2024-09-10T11:08:45.342 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T11:08:45.366 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T11:08:45.458 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899074 branch: wip-rishabh-testing-20240909.165402-squid description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: null first_in_suite: false job_id: '7899074' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.0 ' name: rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rishabh-testing-20240909.165402-squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-testing-20240909.165402-squid sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b owner: scheduled_rishabh@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 9893 sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b sleep_before_teardown: 0 subset: 197/1024 suite: fs suite_branch: wip-rishabh-testing-20240909.165402-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b targets: smithi144.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFgBoeUJxZ9Q0MEvw9oPNqdmN8CwvjoCOk+D3Fs2uGg54aR7vsxi9xALdjvQNLGX40qZJlyEbe/tyRkEsc6dIsQ= smithi178.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFKNuELFom6OiF57hrOGVx0NbOlwjx27HckL5uQHtLqsruacLxpz7SfWXvbCUj0RCWTGKAa6Mba7v8yZrXYnoyA= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.0 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.0 roleless: true - print: '**** done end installing v18.2.0 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay true - cephadm.shell: host.a: - ceph fs set cephfs inline_data true --yes-i-really-really-mean-it - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - ceph-fuse: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_06:58:10 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-09-10T11:08:45.458 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa; will attempt to use it 2024-09-10T11:08:45.459 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks 2024-09-10T11:08:45.459 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T11:08:45.461 INFO:teuthology.task.internal:Checking packages... 2024-09-10T11:08:45.481 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1cf819b7212a6048e92fc78c4eff963344f6d56b' 2024-09-10T11:08:45.481 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T11:08:45.481 INFO:teuthology.packaging:ref: None 2024-09-10T11:08:45.481 INFO:teuthology.packaging:tag: None 2024-09-10T11:08:45.481 INFO:teuthology.packaging:branch: wip-rishabh-testing-20240909.165402-squid 2024-09-10T11:08:45.481 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:08:45.482 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rishabh-testing-20240909.165402-squid 2024-09-10T11:08:45.622 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-305.g1cf819b7 2024-09-10T11:08:45.624 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T11:08:45.634 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T11:08:45.634 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T11:08:45.647 INFO:teuthology.task.internal:Saving configuration 2024-09-10T11:08:45.665 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T11:08:45.680 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T11:08:45.705 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi144.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899074', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 11:03:27.280172', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFgBoeUJxZ9Q0MEvw9oPNqdmN8CwvjoCOk+D3Fs2uGg54aR7vsxi9xALdjvQNLGX40qZJlyEbe/tyRkEsc6dIsQ='} 2024-09-10T11:08:45.726 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi178.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899074', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 11:03:27.278991', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFKNuELFom6OiF57hrOGVx0NbOlwjx27HckL5uQHtLqsruacLxpz7SfWXvbCUj0RCWTGKAa6Mba7v8yZrXYnoyA='} 2024-09-10T11:08:45.726 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T11:08:45.738 INFO:teuthology.task.internal:roles: ubuntu@smithi144.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-09-10T11:08:45.738 INFO:teuthology.task.internal:roles: ubuntu@smithi178.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-09-10T11:08:45.738 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T11:08:45.805 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5586af7be0>, signals=[15]) 2024-09-10T11:08:45.805 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T11:08:45.819 INFO:teuthology.task.internal:Opening connections... 2024-09-10T11:08:45.819 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T11:08:45.820 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:08:45.897 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi178.front.sepia.ceph.com 2024-09-10T11:08:45.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:08:45.975 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T11:08:45.986 DEBUG:teuthology.orchestra.run.smithi144:> uname -m 2024-09-10T11:08:46.005 INFO:teuthology.orchestra.run.smithi144.stdout:x86_64 2024-09-10T11:08:46.005 DEBUG:teuthology.orchestra.run.smithi144:> cat /etc/os-release 2024-09-10T11:08:46.064 INFO:teuthology.orchestra.run.smithi144.stdout:NAME="CentOS Stream" 2024-09-10T11:08:46.064 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION="9" 2024-09-10T11:08:46.064 INFO:teuthology.orchestra.run.smithi144.stdout:ID="centos" 2024-09-10T11:08:46.064 INFO:teuthology.orchestra.run.smithi144.stdout:ID_LIKE="rhel fedora" 2024-09-10T11:08:46.064 INFO:teuthology.orchestra.run.smithi144.stdout:VERSION_ID="9" 2024-09-10T11:08:46.064 INFO:teuthology.orchestra.run.smithi144.stdout:PLATFORM_ID="platform:el9" 2024-09-10T11:08:46.064 INFO:teuthology.orchestra.run.smithi144.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T11:08:46.064 INFO:teuthology.orchestra.run.smithi144.stdout:ANSI_COLOR="0;31" 2024-09-10T11:08:46.064 INFO:teuthology.orchestra.run.smithi144.stdout:LOGO="fedora-logo-icon" 2024-09-10T11:08:46.065 INFO:teuthology.orchestra.run.smithi144.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T11:08:46.065 INFO:teuthology.orchestra.run.smithi144.stdout:HOME_URL="https://centos.org/" 2024-09-10T11:08:46.065 INFO:teuthology.orchestra.run.smithi144.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T11:08:46.065 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T11:08:46.065 INFO:teuthology.orchestra.run.smithi144.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T11:08:46.065 INFO:teuthology.lock.ops:Updating smithi144.front.sepia.ceph.com on lock server 2024-09-10T11:08:46.094 DEBUG:teuthology.orchestra.run.smithi178:> uname -m 2024-09-10T11:08:46.112 INFO:teuthology.orchestra.run.smithi178.stdout:x86_64 2024-09-10T11:08:46.113 DEBUG:teuthology.orchestra.run.smithi178:> cat /etc/os-release 2024-09-10T11:08:46.170 INFO:teuthology.orchestra.run.smithi178.stdout:NAME="CentOS Stream" 2024-09-10T11:08:46.170 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION="9" 2024-09-10T11:08:46.170 INFO:teuthology.orchestra.run.smithi178.stdout:ID="centos" 2024-09-10T11:08:46.170 INFO:teuthology.orchestra.run.smithi178.stdout:ID_LIKE="rhel fedora" 2024-09-10T11:08:46.171 INFO:teuthology.orchestra.run.smithi178.stdout:VERSION_ID="9" 2024-09-10T11:08:46.171 INFO:teuthology.orchestra.run.smithi178.stdout:PLATFORM_ID="platform:el9" 2024-09-10T11:08:46.171 INFO:teuthology.orchestra.run.smithi178.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T11:08:46.171 INFO:teuthology.orchestra.run.smithi178.stdout:ANSI_COLOR="0;31" 2024-09-10T11:08:46.171 INFO:teuthology.orchestra.run.smithi178.stdout:LOGO="fedora-logo-icon" 2024-09-10T11:08:46.171 INFO:teuthology.orchestra.run.smithi178.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T11:08:46.171 INFO:teuthology.orchestra.run.smithi178.stdout:HOME_URL="https://centos.org/" 2024-09-10T11:08:46.171 INFO:teuthology.orchestra.run.smithi178.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T11:08:46.171 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T11:08:46.171 INFO:teuthology.orchestra.run.smithi178.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T11:08:46.172 INFO:teuthology.lock.ops:Updating smithi178.front.sepia.ceph.com on lock server 2024-09-10T11:08:46.194 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T11:08:46.208 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T11:08:46.237 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T11:08:46.237 DEBUG:teuthology.orchestra.run.smithi144:> test '!' -e /home/ubuntu/cephtest 2024-09-10T11:08:46.240 DEBUG:teuthology.orchestra.run.smithi178:> test '!' -e /home/ubuntu/cephtest 2024-09-10T11:08:46.257 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T11:08:46.267 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T11:08:46.268 DEBUG:teuthology.orchestra.run.smithi144:> test -z $(ls -A /var/lib/ceph) 2024-09-10T11:08:46.297 DEBUG:teuthology.orchestra.run.smithi178:> test -z $(ls -A /var/lib/ceph) 2024-09-10T11:08:46.420 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T11:08:46.479 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T11:08:46.498 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T11:08:46.498 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T11:08:46.498 DEBUG:teuthology.orchestra.run.smithi144:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T11:08:46.498 DEBUG:teuthology.orchestra.run.smithi178:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T11:08:46.515 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:08:46.515 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-09-10T11:08:46.516 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:08:46.516 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-09-10T11:08:46.573 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-503.el9.x86_64 2024-09-10T11:08:46.573 INFO:teuthology.task.kernel:Running kernel on smithi144: 5.14.0-503.el9.x86_64 2024-09-10T11:08:46.573 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum install -y kernel 2024-09-10T11:08:46.574 INFO:teuthology.orchestra.run.smithi178.stdout:5.14.0-503.el9.x86_64 2024-09-10T11:08:46.575 INFO:teuthology.task.kernel:Running kernel on smithi178: 5.14.0-503.el9.x86_64 2024-09-10T11:08:46.575 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum install -y kernel 2024-09-10T11:08:50.350 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 4.0 MB/s | 8.2 MB 00:02 2024-09-10T11:08:52.648 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 1.9 MB/s | 8.2 MB 00:04 2024-09-10T11:08:53.409 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 10 MB/s | 20 MB 00:01 2024-09-10T11:08:54.557 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 26 MB/s | 20 MB 00:00 2024-09-10T11:08:57.668 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.5 MB 00:00 2024-09-10T11:08:59.310 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 19 kB 00:00 2024-09-10T11:09:00.066 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-10T11:09:00.198 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-10T11:09:01.724 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 41 kB/s | 19 kB 00:00 2024-09-10T11:09:02.687 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-10T11:09:05.695 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-10T11:09:07.718 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T11:09:07.719 INFO:teuthology.orchestra.run.smithi144.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T11:09:07.806 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:09:07.810 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:09:07.811 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-09-10T11:09:07.811 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:09:07.811 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:09:07.811 INFO:teuthology.orchestra.run.smithi144.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T11:09:07.811 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-10T11:09:07.811 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T11:09:07.811 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T11:09:07.811 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T11:09:07.811 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:09:07.811 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:09:07.811 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:09:07.812 INFO:teuthology.orchestra.run.smithi144.stdout:Install 4 Packages 2024-09-10T11:09:07.812 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:09:07.812 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 79 M 2024-09-10T11:09:07.812 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 126 M 2024-09-10T11:09:07.812 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:09:07.988 INFO:teuthology.orchestra.run.smithi144.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 307 kB/s | 23 kB 00:00 2024-09-10T11:09:08.159 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T11:09:08.347 INFO:teuthology.orchestra.run.smithi144.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-09-10T11:09:08.747 INFO:teuthology.orchestra.run.smithi144.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 41 MB/s | 35 MB 00:00 2024-09-10T11:09:09.148 INFO:teuthology.orchestra.run.smithi144.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 25 MB/s | 29 MB 00:01 2024-09-10T11:09:09.148 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:09:09.148 INFO:teuthology.orchestra.run.smithi144.stdout:Total 59 MB/s | 79 MB 00:01 2024-09-10T11:09:09.832 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:09:10.195 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:09:10.196 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:09:10.198 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T11:09:10.199 INFO:teuthology.orchestra.run.smithi178.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T11:09:10.303 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:09:10.313 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:09:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-09-10T11:09:10.313 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:09:10.313 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:09:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T11:09:10.313 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-10T11:09:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T11:09:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T11:09:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T11:09:10.313 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:09:10.314 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:09:10.314 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:09:10.314 INFO:teuthology.orchestra.run.smithi178.stdout:Install 4 Packages 2024-09-10T11:09:10.314 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:09:10.314 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 79 M 2024-09-10T11:09:10.314 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 126 M 2024-09-10T11:09:10.314 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:09:10.994 INFO:teuthology.orchestra.run.smithi178.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 85 kB/s | 23 kB 00:00 2024-09-10T11:09:11.153 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:09:11.154 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:09:13.589 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:09:14.387 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T11:09:14.466 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T11:09:17.245 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T11:09:17.393 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T11:09:17.984 INFO:teuthology.orchestra.run.smithi178.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 2.2 MB/s | 16 MB 00:07 2024-09-10T11:09:22.205 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T11:09:22.402 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:09:27.257 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:09:43.739 INFO:teuthology.orchestra.run.smithi178.stdout:(3/4): kernel-modules-core-5.14.0-505.el9.x86_6 893 kB/s | 29 MB 00:32 2024-09-10T11:09:49.197 INFO:teuthology.orchestra.run.smithi178.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 919 kB/s | 35 MB 00:38 2024-09-10T11:09:49.197 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:09:49.197 INFO:teuthology.orchestra.run.smithi178.stdout:Total 2.0 MB/s | 79 MB 00:38 2024-09-10T11:09:49.871 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:09:50.244 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:09:50.244 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:09:51.196 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:09:51.196 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:09:53.655 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:09:54.297 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T11:09:54.405 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T11:09:57.193 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T11:09:57.335 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T11:10:02.168 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T11:10:02.236 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:10:05.323 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:10:05.363 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:10:05.655 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:10:05.655 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T11:10:05.655 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T11:10:05.655 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T11:10:06.497 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:10:06.497 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:10:06.498 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:10:06.498 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T11:10:06.498 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T11:10:06.498 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T11:10:06.498 INFO:teuthology.orchestra.run.smithi144.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T11:10:06.498 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:10:06.498 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:10:06.862 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T11:10:06.939 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T11:10:06.941 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T11:10:06.941 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T11:10:06.941 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T11:10:06.941 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T11:10:06.941 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2024-09-10T11:10:06.941 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi144.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T11:10:06.941 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T11:10:07.000 INFO:teuthology.orchestra.run.smithi144.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T11:10:07.000 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T11:10:07.000 DEBUG:teuthology.orchestra.run.smithi144:> sudo rpm -qi grub2-tools 2024-09-10T11:10:07.040 INFO:teuthology.orchestra.run.smithi144.stdout:Name : grub2-tools 2024-09-10T11:10:07.040 INFO:teuthology.orchestra.run.smithi144.stdout:Epoch : 1 2024-09-10T11:10:07.040 INFO:teuthology.orchestra.run.smithi144.stdout:Version : 2.06 2024-09-10T11:10:07.040 INFO:teuthology.orchestra.run.smithi144.stdout:Release : 82.el9 2024-09-10T11:10:07.041 INFO:teuthology.orchestra.run.smithi144.stdout:Architecture: x86_64 2024-09-10T11:10:07.041 INFO:teuthology.orchestra.run.smithi144.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T11:10:07.041 INFO:teuthology.orchestra.run.smithi144.stdout:Group : Unspecified 2024-09-10T11:10:07.041 INFO:teuthology.orchestra.run.smithi144.stdout:Size : 8274670 2024-09-10T11:10:07.041 INFO:teuthology.orchestra.run.smithi144.stdout:License : GPLv3+ 2024-09-10T11:10:07.041 INFO:teuthology.orchestra.run.smithi144.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T11:10:07.041 INFO:teuthology.orchestra.run.smithi144.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T11:10:07.041 INFO:teuthology.orchestra.run.smithi144.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T11:10:07.041 INFO:teuthology.orchestra.run.smithi144.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T11:10:07.041 INFO:teuthology.orchestra.run.smithi144.stdout:Packager : builder@centos.org 2024-09-10T11:10:07.041 INFO:teuthology.orchestra.run.smithi144.stdout:Vendor : CentOS 2024-09-10T11:10:07.042 INFO:teuthology.orchestra.run.smithi144.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T11:10:07.042 INFO:teuthology.orchestra.run.smithi144.stdout:Summary : Support tools for GRUB. 2024-09-10T11:10:07.042 INFO:teuthology.orchestra.run.smithi144.stdout:Description : 2024-09-10T11:10:07.042 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:10:07.042 INFO:teuthology.orchestra.run.smithi144.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T11:10:07.042 INFO:teuthology.orchestra.run.smithi144.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T11:10:07.042 INFO:teuthology.orchestra.run.smithi144.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T11:10:07.042 INFO:teuthology.orchestra.run.smithi144.stdout:hardware devices. 2024-09-10T11:10:07.042 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:10:07.042 INFO:teuthology.orchestra.run.smithi144.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T11:10:07.043 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T11:10:07.043 INFO:teuthology.task.kernel:Updating grub on smithi144 to boot 5.14.0-505.el9.x86_64 2024-09-10T11:10:07.044 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T11:10:07.072 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:10:07.702 INFO:teuthology.orchestra.run.smithi144.stderr:Generating grub configuration file ... 2024-09-10T11:10:09.018 INFO:teuthology.orchestra.run.smithi144.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T11:10:09.047 INFO:teuthology.orchestra.run.smithi144.stderr:done 2024-09-10T11:10:09.049 DEBUG:teuthology.orchestra.run.smithi144:> mktemp 2024-09-10T11:10:09.089 INFO:teuthology.orchestra.run.smithi144.stdout:/tmp/tmp.KTu2tfwnEg 2024-09-10T11:10:09.089 DEBUG:teuthology.orchestra.run.smithi144:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.KTu2tfwnEg 2024-09-10T11:10:09.156 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0666 /tmp/tmp.KTu2tfwnEg 2024-09-10T11:10:09.300 DEBUG:teuthology.orchestra.remote:smithi144:/tmp/tmp.KTu2tfwnEg is 6KB 2024-09-10T11:10:09.348 DEBUG:teuthology.orchestra.run.smithi144:> rm -fr /tmp/tmp.KTu2tfwnEg 2024-09-10T11:10:09.363 DEBUG:teuthology.orchestra.run.smithi144:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T11:10:09.426 INFO:teuthology.orchestra.run.smithi144.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T11:10:09.426 INFO:teuthology.orchestra.run.smithi144.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T11:10:09.426 INFO:teuthology.orchestra.run.smithi144.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T11:10:09.426 INFO:teuthology.orchestra.run.smithi144.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T11:10:09.427 DEBUG:teuthology.orchestra.run.smithi144:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T11:10:09.577 DEBUG:teuthology.orchestra.run.smithi144:> sudo shutdown -r now 2024-09-10T11:10:39.579 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2024-09-10T11:10:39.580 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-10T11:10:39.581 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:10:45.251 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:10:45.290 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:10:45.605 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:10:45.605 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T11:10:45.605 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T11:10:45.605 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T11:10:46.496 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T11:10:46.496 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:10:46.496 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:10:46.496 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T11:10:46.496 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T11:10:46.496 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T11:10:46.496 INFO:teuthology.orchestra.run.smithi178.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T11:10:46.496 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:10:46.497 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:10:46.841 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T11:10:46.933 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T11:10:46.933 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T11:10:46.934 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T11:10:46.934 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T11:10:46.934 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T11:10:46.934 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2024-09-10T11:10:46.934 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi178.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T11:10:46.934 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T11:10:46.993 INFO:teuthology.orchestra.run.smithi178.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T11:10:46.993 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T11:10:46.993 DEBUG:teuthology.orchestra.run.smithi178:> sudo rpm -qi grub2-tools 2024-09-10T11:10:47.027 INFO:teuthology.orchestra.run.smithi178.stdout:Name : grub2-tools 2024-09-10T11:10:47.027 INFO:teuthology.orchestra.run.smithi178.stdout:Epoch : 1 2024-09-10T11:10:47.027 INFO:teuthology.orchestra.run.smithi178.stdout:Version : 2.06 2024-09-10T11:10:47.027 INFO:teuthology.orchestra.run.smithi178.stdout:Release : 82.el9 2024-09-10T11:10:47.027 INFO:teuthology.orchestra.run.smithi178.stdout:Architecture: x86_64 2024-09-10T11:10:47.027 INFO:teuthology.orchestra.run.smithi178.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T11:10:47.027 INFO:teuthology.orchestra.run.smithi178.stdout:Group : Unspecified 2024-09-10T11:10:47.027 INFO:teuthology.orchestra.run.smithi178.stdout:Size : 8274670 2024-09-10T11:10:47.028 INFO:teuthology.orchestra.run.smithi178.stdout:License : GPLv3+ 2024-09-10T11:10:47.028 INFO:teuthology.orchestra.run.smithi178.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T11:10:47.028 INFO:teuthology.orchestra.run.smithi178.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T11:10:47.028 INFO:teuthology.orchestra.run.smithi178.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T11:10:47.028 INFO:teuthology.orchestra.run.smithi178.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T11:10:47.028 INFO:teuthology.orchestra.run.smithi178.stdout:Packager : builder@centos.org 2024-09-10T11:10:47.028 INFO:teuthology.orchestra.run.smithi178.stdout:Vendor : CentOS 2024-09-10T11:10:47.028 INFO:teuthology.orchestra.run.smithi178.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T11:10:47.028 INFO:teuthology.orchestra.run.smithi178.stdout:Summary : Support tools for GRUB. 2024-09-10T11:10:47.028 INFO:teuthology.orchestra.run.smithi178.stdout:Description : 2024-09-10T11:10:47.028 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:10:47.028 INFO:teuthology.orchestra.run.smithi178.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T11:10:47.028 INFO:teuthology.orchestra.run.smithi178.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T11:10:47.028 INFO:teuthology.orchestra.run.smithi178.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T11:10:47.029 INFO:teuthology.orchestra.run.smithi178.stdout:hardware devices. 2024-09-10T11:10:47.029 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:10:47.029 INFO:teuthology.orchestra.run.smithi178.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T11:10:47.030 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T11:10:47.030 INFO:teuthology.task.kernel:Updating grub on smithi178 to boot 5.14.0-505.el9.x86_64 2024-09-10T11:10:47.030 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T11:10:47.695 INFO:teuthology.orchestra.run.smithi178.stderr:Generating grub configuration file ... 2024-09-10T11:10:48.994 INFO:teuthology.orchestra.run.smithi178.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T11:10:49.028 INFO:teuthology.orchestra.run.smithi178.stderr:done 2024-09-10T11:10:49.030 DEBUG:teuthology.orchestra.run.smithi178:> mktemp 2024-09-10T11:10:49.045 INFO:teuthology.orchestra.run.smithi178.stdout:/tmp/tmp.Qh7gUL4e3M 2024-09-10T11:10:49.046 DEBUG:teuthology.orchestra.run.smithi178:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Qh7gUL4e3M 2024-09-10T11:10:49.134 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0666 /tmp/tmp.Qh7gUL4e3M 2024-09-10T11:10:49.231 DEBUG:teuthology.orchestra.remote:smithi178:/tmp/tmp.Qh7gUL4e3M is 6KB 2024-09-10T11:10:49.251 DEBUG:teuthology.orchestra.run.smithi178:> rm -fr /tmp/tmp.Qh7gUL4e3M 2024-09-10T11:10:49.265 DEBUG:teuthology.orchestra.run.smithi178:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T11:10:49.328 INFO:teuthology.orchestra.run.smithi178.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T11:10:49.329 INFO:teuthology.orchestra.run.smithi178.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T11:10:49.329 INFO:teuthology.orchestra.run.smithi178.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T11:10:49.329 INFO:teuthology.orchestra.run.smithi178.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T11:10:49.330 DEBUG:teuthology.orchestra.run.smithi178:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T11:10:49.496 DEBUG:teuthology.orchestra.run.smithi178:> sudo shutdown -r now 2024-09-10T11:10:58.109 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-09-10T11:11:07.117 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-10T11:11:07.118 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:11:10.173 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.144 2024-09-10T11:11:19.508 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2024-09-10T11:11:19.508 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-10T11:11:19.509 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:11:22.175 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-10T11:11:22.175 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:11:38.045 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-09-10T11:11:47.054 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-10T11:11:47.055 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:11:50.109 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.178 2024-09-10T11:11:54.081 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-09-10T11:11:54.466 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-10T11:11:54.466 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T11:11:54.467 DEBUG:teuthology.orchestra.run.smithi144:> uname -r 2024-09-10T11:11:54.481 INFO:teuthology.orchestra.run.smithi144.stdout:5.14.0-505.el9.x86_64 2024-09-10T11:11:54.481 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T11:11:54.481 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T11:11:54.481 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T11:11:55.482 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-09-10T11:11:55.482 DEBUG:teuthology.orchestra.run.smithi144:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T11:11:55.577 INFO:teuthology.orchestra.run.smithi144.stdout:ttyS1 2024-09-10T11:11:55.616 DEBUG:teuthology.parallel:result is None 2024-09-10T11:12:02.110 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-10T11:12:02.111 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:13:02.113 DEBUG:teuthology.orchestra.remote:timed out 2024-09-10T11:13:17.115 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-10T11:13:17.116 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:13:17.539 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-09-10T11:13:17.891 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-10T11:13:17.891 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "5.14.0-505.el9.x86_64"... 2024-09-10T11:13:17.891 DEBUG:teuthology.orchestra.run.smithi178:> uname -r 2024-09-10T11:13:17.906 INFO:teuthology.orchestra.run.smithi178.stdout:5.14.0-505.el9.x86_64 2024-09-10T11:13:17.906 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T11:13:17.907 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T11:13:17.907 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T11:13:18.908 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-09-10T11:13:18.908 DEBUG:teuthology.orchestra.run.smithi178:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T11:13:19.002 INFO:teuthology.orchestra.run.smithi178.stdout:ttyS1 2024-09-10T11:13:19.020 DEBUG:teuthology.parallel:result is None 2024-09-10T11:13:19.020 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T11:13:19.030 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T11:13:19.031 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T11:13:19.034 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T11:13:19.052 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T11:13:19.058 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T11:13:19.065 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T11:13:19.065 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T11:13:19.093 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T11:13:19.149 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T11:13:19.155 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T11:13:19.155 DEBUG:teuthology.orchestra.run.smithi144:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T11:13:19.158 DEBUG:teuthology.orchestra.run.smithi178:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T11:13:19.192 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T11:13:19.193 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T11:13:19.203 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T11:13:19.204 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T11:13:19.206 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T11:13:19.213 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T11:13:19.213 DEBUG:teuthology.orchestra.run.smithi144:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T11:13:19.248 DEBUG:teuthology.orchestra.run.smithi178:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T11:13:19.276 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T11:13:19.285 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T11:13:19.285 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T11:13:19.317 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T11:13:19.333 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T11:13:19.416 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T11:13:19.462 DEBUG:teuthology.orchestra.run.smithi144:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T11:13:19.545 DEBUG:teuthology.orchestra.run.smithi144:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T11:13:19.586 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:13:19.586 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T11:13:19.651 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T11:13:19.697 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T11:13:19.753 DEBUG:teuthology.orchestra.run.smithi178:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T11:13:19.838 DEBUG:teuthology.orchestra.run.smithi178:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T11:13:19.879 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T11:13:19.879 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T11:13:19.946 DEBUG:teuthology.orchestra.run.smithi144:> sudo service rsyslog restart 2024-09-10T11:13:19.948 DEBUG:teuthology.orchestra.run.smithi178:> sudo service rsyslog restart 2024-09-10T11:13:19.998 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T11:13:20.040 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T11:13:20.474 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T11:13:20.481 INFO:teuthology.task.internal:Starting timer... 2024-09-10T11:13:20.482 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T11:13:20.492 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T11:13:20.500 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-10T11:13:20.500 DEBUG:teuthology.orchestra.run.smithi144:> sudo service auditd rotate 2024-09-10T11:13:20.574 INFO:teuthology.orchestra.run.smithi144.stdout:Rotating logs: 2024-09-10T11:13:20.576 DEBUG:teuthology.orchestra.run.smithi178:> sudo service auditd rotate 2024-09-10T11:13:20.620 INFO:teuthology.orchestra.run.smithi178.stdout:Rotating logs: 2024-09-10T11:13:20.622 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T11:13:20.622 DEBUG:teuthology.orchestra.run.smithi144:> /usr/sbin/getenforce 2024-09-10T11:13:20.647 INFO:teuthology.orchestra.run.smithi144.stdout:Permissive 2024-09-10T11:13:20.648 DEBUG:teuthology.orchestra.run.smithi178:> /usr/sbin/getenforce 2024-09-10T11:13:20.683 INFO:teuthology.orchestra.run.smithi178.stdout:Permissive 2024-09-10T11:13:20.684 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi144.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi178.front.sepia.ceph.com': 'permissive'} 2024-09-10T11:13:20.684 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T11:13:20.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:13:20.716 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T11:13:20.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:13:20.753 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T11:13:20.753 DEBUG:teuthology.orchestra.run.smithi144:> sudo /usr/sbin/setenforce permissive 2024-09-10T11:13:20.788 DEBUG:teuthology.orchestra.run.smithi178:> sudo /usr/sbin/setenforce permissive 2024-09-10T11:13:20.822 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T11:13:20.831 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T11:13:20.834 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T11:13:20.938 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T11:13:20.948 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T11:13:20.949 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi144.front.sepia.ceph.com,smithi178.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T11:17:02.920 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi144.front.sepia.ceph.com'), Remote(name='ubuntu@smithi178.front.sepia.ceph.com')] 2024-09-10T11:17:02.921 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-10T11:17:02.923 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi144.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:17:03.002 DEBUG:teuthology.orchestra.run.smithi144:> true 2024-09-10T11:17:03.079 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi144.front.sepia.ceph.com' 2024-09-10T11:17:03.079 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-10T11:17:03.080 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi178.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T11:17:03.157 DEBUG:teuthology.orchestra.run.smithi178:> true 2024-09-10T11:17:03.236 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi178.front.sepia.ceph.com' 2024-09-10T11:17:03.236 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T11:17:03.247 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T11:17:03.248 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T11:17:03.248 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T11:17:03.251 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T11:17:03.251 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T11:17:03.277 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T11:17:03.292 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T11:17:03.305 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T11:17:03.316 INFO:teuthology.orchestra.run.smithi144.stderr:sudo: ntpd: command not found 2024-09-10T11:17:03.319 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T11:17:03.327 INFO:teuthology.orchestra.run.smithi144.stdout:506 Cannot talk to daemon 2024-09-10T11:17:03.341 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T11:17:03.343 INFO:teuthology.orchestra.run.smithi178.stderr:sudo: ntpd: command not found 2024-09-10T11:17:03.354 INFO:teuthology.orchestra.run.smithi178.stdout:506 Cannot talk to daemon 2024-09-10T11:17:03.355 INFO:teuthology.orchestra.run.smithi144.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T11:17:03.368 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T11:17:03.382 INFO:teuthology.orchestra.run.smithi178.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T11:17:03.419 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-09-10T11:17:03.423 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T11:17:03.423 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-09-10T11:17:03.423 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:17:03.424 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:17:03.424 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:17:03.424 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:17:03.443 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-09-10T11:17:03.448 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T11:17:03.448 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-09-10T11:17:03.449 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:17:03.449 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:17:03.449 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:17:03.449 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T11:17:03.449 INFO:teuthology.run_tasks:Running task install... 2024-09-10T11:17:03.457 DEBUG:teuthology.task.install:project ceph 2024-09-10T11:17:03.457 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'}} 2024-09-10T11:17:03.457 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.0', 'flavor': 'default', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T11:17:03.458 INFO:teuthology.task.install:Using flavor: default 2024-09-10T11:17:03.464 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T11:17:03.464 INFO:teuthology.task.install:extra packages: [] 2024-09-10T11:17:03.464 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-09-10T11:17:03.465 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-09-10T11:17:03.465 INFO:teuthology.packaging:ref: None 2024-09-10T11:17:03.465 INFO:teuthology.packaging:tag: v18.2.0 2024-09-10T11:17:03.465 INFO:teuthology.packaging:branch: None 2024-09-10T11:17:03.465 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:17:03.596 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph-ci v18.2.0^{} -> 5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-09-10T11:17:03.596 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-09-10T11:17:03.599 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-09-10T11:17:03.599 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-09-10T11:17:03.599 INFO:teuthology.packaging:ref: None 2024-09-10T11:17:03.599 INFO:teuthology.packaging:tag: v18.2.0 2024-09-10T11:17:03.599 INFO:teuthology.packaging:branch: None 2024-09-10T11:17:03.599 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:17:03.599 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-09-10T11:17:03.747 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-09-10T11:17:03.748 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-09-10T11:17:03.751 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-09-10T11:17:03.751 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-09-10T11:17:03.795 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T11:17:03.795 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:17:03.795 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T11:17:03.803 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T11:17:03.803 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T11:17:03.803 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T11:17:03.821 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T11:17:03.821 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-09-10T11:17:03.822 INFO:teuthology.packaging:ref: None 2024-09-10T11:17:03.822 INFO:teuthology.packaging:tag: v18.2.0 2024-09-10T11:17:03.822 INFO:teuthology.packaging:branch: None 2024-09-10T11:17:03.822 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:17:03.822 DEBUG:teuthology.orchestra.run.smithi144:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T11:17:03.830 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T11:17:03.830 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-09-10T11:17:03.830 INFO:teuthology.packaging:ref: None 2024-09-10T11:17:03.830 INFO:teuthology.packaging:tag: v18.2.0 2024-09-10T11:17:03.830 INFO:teuthology.packaging:branch: None 2024-09-10T11:17:03.830 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:17:03.831 DEBUG:teuthology.orchestra.run.smithi178:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T11:17:03.888 DEBUG:teuthology.orchestra.run.smithi144:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T11:17:03.897 DEBUG:teuthology.orchestra.run.smithi178:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T11:17:03.964 DEBUG:teuthology.orchestra.run.smithi144:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T11:17:03.972 DEBUG:teuthology.orchestra.run.smithi178:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T11:17:04.028 INFO:teuthology.orchestra.run.smithi144.stdout:check_obsoletes = 1 2024-09-10T11:17:04.030 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum clean all 2024-09-10T11:17:04.037 INFO:teuthology.orchestra.run.smithi178.stdout:check_obsoletes = 1 2024-09-10T11:17:04.039 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum clean all 2024-09-10T11:17:04.262 INFO:teuthology.orchestra.run.smithi144.stdout:45 files removed 2024-09-10T11:17:04.269 INFO:teuthology.orchestra.run.smithi178.stdout:45 files removed 2024-09-10T11:17:04.290 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-radosgw 2024-09-10T11:17:04.295 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-radosgw 2024-09-10T11:17:04.695 INFO:teuthology.orchestra.run.smithi144.stdout:ceph packages for x86_64 655 kB/s | 76 kB 00:00 2024-09-10T11:17:04.717 INFO:teuthology.orchestra.run.smithi178.stdout:ceph packages for x86_64 557 kB/s | 76 kB 00:00 2024-09-10T11:17:04.865 INFO:teuthology.orchestra.run.smithi178.stdout:ceph noarch packages 92 kB/s | 9.3 kB 00:00 2024-09-10T11:17:04.887 INFO:teuthology.orchestra.run.smithi144.stdout:ceph noarch packages 64 kB/s | 9.3 kB 00:00 2024-09-10T11:17:05.032 INFO:teuthology.orchestra.run.smithi178.stdout:ceph source packages 18 kB/s | 2.2 kB 00:00 2024-09-10T11:17:05.046 INFO:teuthology.orchestra.run.smithi144.stdout:ceph source packages 18 kB/s | 2.2 kB 00:00 2024-09-10T11:17:06.244 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - BaseOS 7.0 MB/s | 8.2 MB 00:01 2024-09-10T11:17:06.804 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - BaseOS 4.8 MB/s | 8.2 MB 00:01 2024-09-10T11:17:08.357 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:00 2024-09-10T11:17:10.030 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - AppStream 9.6 MB/s | 20 MB 00:02 2024-09-10T11:17:13.192 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - CRB 6.3 MB/s | 6.5 MB 00:01 2024-09-10T11:17:14.717 INFO:teuthology.orchestra.run.smithi178.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 19 kB 00:00 2024-09-10T11:17:15.406 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - CRB 4.3 MB/s | 6.5 MB 00:01 2024-09-10T11:17:15.722 INFO:teuthology.orchestra.run.smithi178.stdout:Extra Packages for Enterprise Linux 26 MB/s | 23 MB 00:00 2024-09-10T11:17:17.215 INFO:teuthology.orchestra.run.smithi144.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-10T11:17:18.311 INFO:teuthology.orchestra.run.smithi144.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-10T11:17:21.086 INFO:teuthology.orchestra.run.smithi178.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-10T11:17:23.495 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:17:23.505 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:17:23.505 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-10T11:17:23.505 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:17:23.505 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:17:23.505 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-09-10T11:17:23.505 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrading: 2024-09-10T11:17:23.505 INFO:teuthology.orchestra.run.smithi178.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-09-10T11:17:23.505 INFO:teuthology.orchestra.run.smithi178.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-09-10T11:17:23.505 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-10T11:17:23.505 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T11:17:23.505 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-09-10T11:17:23.505 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-09-10T11:17:23.506 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-09-10T11:17:23.506 INFO:teuthology.orchestra.run.smithi178.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-10T11:17:23.506 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T11:17:23.506 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T11:17:23.506 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T11:17:23.506 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-09-10T11:17:23.506 INFO:teuthology.orchestra.run.smithi178.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T11:17:23.506 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T11:17:23.506 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T11:17:23.506 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-09-10T11:17:23.506 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T11:17:23.506 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-09-10T11:17:23.507 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T11:17:23.507 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T11:17:23.507 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T11:17:23.507 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-09-10T11:17:23.507 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-09-10T11:17:23.507 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-09-10T11:17:23.507 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T11:17:23.507 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-09-10T11:17:23.507 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-09-10T11:17:23.507 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-09-10T11:17:23.507 INFO:teuthology.orchestra.run.smithi178.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T11:17:23.507 INFO:teuthology.orchestra.run.smithi178.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T11:17:23.507 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:17:23.508 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:17:23.508 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:17:23.508 INFO:teuthology.orchestra.run.smithi178.stdout:Install 28 Packages 2024-09-10T11:17:23.508 INFO:teuthology.orchestra.run.smithi178.stdout:Upgrade 2 Packages 2024-09-10T11:17:23.508 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:17:23.508 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 53 M 2024-09-10T11:17:23.508 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:17:23.767 INFO:teuthology.orchestra.run.smithi144.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-10T11:17:24.037 INFO:teuthology.orchestra.run.smithi178.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 11 MB/s | 5.2 MB 00:00 2024-09-10T11:17:24.146 INFO:teuthology.orchestra.run.smithi178.stdout:(2/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 13 MB/s | 7.6 MB 00:00 2024-09-10T11:17:24.171 INFO:teuthology.orchestra.run.smithi178.stdout:(3/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 180 kB/s | 24 kB 00:00 2024-09-10T11:17:24.212 INFO:teuthology.orchestra.run.smithi178.stdout:(4/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 9.6 MB/s | 653 kB 00:00 2024-09-10T11:17:24.238 INFO:teuthology.orchestra.run.smithi178.stdout:(5/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 6.9 MB/s | 474 kB 00:00 2024-09-10T11:17:24.413 INFO:teuthology.orchestra.run.smithi178.stdout:(6/30): ceph-common-18.2.0-0.el9.x86_64.rpm 21 MB/s | 18 MB 00:00 2024-09-10T11:17:24.438 INFO:teuthology.orchestra.run.smithi178.stdout:(7/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 224 kB/s | 45 kB 00:00 2024-09-10T11:17:24.463 INFO:teuthology.orchestra.run.smithi178.stdout:(8/30): python3-ceph-common-18.2.0-0.el9.x86_64 2.3 MB/s | 119 kB 00:00 2024-09-10T11:17:24.489 INFO:teuthology.orchestra.run.smithi178.stdout:(9/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-09-10T11:17:24.514 INFO:teuthology.orchestra.run.smithi178.stdout:(10/30): python3-rados-18.2.0-0.el9.x86_64.rpm 6.2 MB/s | 321 kB 00:00 2024-09-10T11:17:24.539 INFO:teuthology.orchestra.run.smithi178.stdout:(11/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 5.8 MB/s | 297 kB 00:00 2024-09-10T11:17:24.565 INFO:teuthology.orchestra.run.smithi178.stdout:(12/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 1.9 MB/s | 99 kB 00:00 2024-09-10T11:17:24.632 INFO:teuthology.orchestra.run.smithi178.stdout:(13/30): librgw2-18.2.0-0.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-10T11:17:24.690 INFO:teuthology.orchestra.run.smithi178.stdout:(14/30): boost-program-options-1.75.0-8.el9.x86 710 kB/s | 107 kB 00:00 2024-09-10T11:17:24.707 INFO:teuthology.orchestra.run.smithi178.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 602 kB/s | 45 kB 00:00 2024-09-10T11:17:24.732 INFO:teuthology.orchestra.run.smithi178.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 958 kB/s | 160 kB 00:00 2024-09-10T11:17:24.766 INFO:teuthology.orchestra.run.smithi178.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 1.2 MB/s | 42 kB 00:00 2024-09-10T11:17:24.791 INFO:teuthology.orchestra.run.smithi178.stdout:(18/30): librdkafka-1.6.1-102.el9.x86_64.rpm 6.4 MB/s | 662 kB 00:00 2024-09-10T11:17:24.825 INFO:teuthology.orchestra.run.smithi178.stdout:(19/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-10T11:17:24.867 INFO:teuthology.orchestra.run.smithi178.stdout:(20/30): fmt-8.1.1-5.el9.x86_64.rpm 1.1 MB/s | 111 kB 00:00 2024-09-10T11:17:24.884 INFO:teuthology.orchestra.run.smithi178.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.9 MB/s | 26 kB 00:00 2024-09-10T11:17:24.901 INFO:teuthology.orchestra.run.smithi178.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 2.9 MB/s | 49 kB 00:00 2024-09-10T11:17:24.926 INFO:teuthology.orchestra.run.smithi178.stdout:(23/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 2.4 MB/s | 309 kB 00:00 2024-09-10T11:17:24.951 INFO:teuthology.orchestra.run.smithi178.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-10T11:17:24.985 INFO:teuthology.orchestra.run.smithi178.stdout:(25/30): re2-20211101-20.el9.x86_64.rpm 5.6 MB/s | 191 kB 00:00 2024-09-10T11:17:25.027 INFO:teuthology.orchestra.run.smithi178.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-09-10T11:17:25.111 INFO:teuthology.orchestra.run.smithi178.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-10T11:17:25.169 INFO:teuthology.orchestra.run.smithi178.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-09-10T11:17:25.236 INFO:teuthology.orchestra.run.smithi178.stdout:(29/30): librados2-18.2.0-0.el9.x86_64.rpm 16 MB/s | 3.3 MB 00:00 2024-09-10T11:17:25.286 INFO:teuthology.orchestra.run.smithi178.stdout:(30/30): librbd1-18.2.0-0.el9.x86_64.rpm 17 MB/s | 3.0 MB 00:00 2024-09-10T11:17:25.291 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:17:25.291 INFO:teuthology.orchestra.run.smithi178.stdout:Total 30 MB/s | 53 MB 00:01 2024-09-10T11:17:25.359 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:17:25.393 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:17:25.394 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:17:26.013 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:17:26.013 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:17:26.159 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:17:26.173 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:17:26.173 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-10T11:17:26.173 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:17:26.173 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:17:26.173 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-09-10T11:17:26.173 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrading: 2024-09-10T11:17:26.173 INFO:teuthology.orchestra.run.smithi144.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-09-10T11:17:26.173 INFO:teuthology.orchestra.run.smithi144.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-09-10T11:17:26.173 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-10T11:17:26.174 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-10T11:17:26.174 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-09-10T11:17:26.174 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-09-10T11:17:26.174 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-09-10T11:17:26.174 INFO:teuthology.orchestra.run.smithi144.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-09-10T11:17:26.174 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T11:17:26.174 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-10T11:17:26.174 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-10T11:17:26.174 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-09-10T11:17:26.174 INFO:teuthology.orchestra.run.smithi144.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T11:17:26.174 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T11:17:26.174 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T11:17:26.174 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-09-10T11:17:26.174 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T11:17:26.175 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-09-10T11:17:26.175 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T11:17:26.175 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T11:17:26.175 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-10T11:17:26.175 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-09-10T11:17:26.175 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-09-10T11:17:26.175 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-09-10T11:17:26.175 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T11:17:26.175 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-09-10T11:17:26.175 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-09-10T11:17:26.175 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-09-10T11:17:26.175 INFO:teuthology.orchestra.run.smithi144.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-10T11:17:26.175 INFO:teuthology.orchestra.run.smithi144.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T11:17:26.175 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:17:26.175 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:17:26.176 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:17:26.176 INFO:teuthology.orchestra.run.smithi144.stdout:Install 28 Packages 2024-09-10T11:17:26.176 INFO:teuthology.orchestra.run.smithi144.stdout:Upgrade 2 Packages 2024-09-10T11:17:26.176 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:17:26.176 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 53 M 2024-09-10T11:17:26.176 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:17:26.676 INFO:teuthology.orchestra.run.smithi144.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 43 MB/s | 5.2 MB 00:00 2024-09-10T11:17:26.702 INFO:teuthology.orchestra.run.smithi144.stdout:(2/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 968 kB/s | 24 kB 00:00 2024-09-10T11:17:26.735 INFO:teuthology.orchestra.run.smithi144.stdout:(3/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 19 MB/s | 653 kB 00:00 2024-09-10T11:17:26.777 INFO:teuthology.orchestra.run.smithi144.stdout:(4/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 11 MB/s | 474 kB 00:00 2024-09-10T11:17:26.870 INFO:teuthology.orchestra.run.smithi144.stdout:(5/30): librgw2-18.2.0-0.el9.x86_64.rpm 48 MB/s | 4.4 MB 00:00 2024-09-10T11:17:26.894 INFO:teuthology.orchestra.run.smithi144.stdout:(6/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-09-10T11:17:26.986 INFO:teuthology.orchestra.run.smithi144.stdout:(7/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 18 MB/s | 7.6 MB 00:00 2024-09-10T11:17:27.012 INFO:teuthology.orchestra.run.smithi144.stdout:(8/30): python3-ceph-common-18.2.0-0.el9.x86_64 1.0 MB/s | 119 kB 00:00 2024-09-10T11:17:27.037 INFO:teuthology.orchestra.run.smithi144.stdout:(9/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-09-10T11:17:27.062 INFO:teuthology.orchestra.run.smithi144.stdout:(10/30): python3-rados-18.2.0-0.el9.x86_64.rpm 6.2 MB/s | 321 kB 00:00 2024-09-10T11:17:27.087 INFO:teuthology.orchestra.run.smithi144.stdout:(11/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 5.8 MB/s | 297 kB 00:00 2024-09-10T11:17:27.113 INFO:teuthology.orchestra.run.smithi144.stdout:(12/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 1.9 MB/s | 99 kB 00:00 2024-09-10T11:17:27.338 INFO:teuthology.orchestra.run.smithi144.stdout:(13/30): ceph-common-18.2.0-0.el9.x86_64.rpm 23 MB/s | 18 MB 00:00 2024-09-10T11:17:27.399 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:17:27.495 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-10T11:17:27.525 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-10T11:17:27.574 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-10T11:17:27.680 INFO:teuthology.orchestra.run.smithi144.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 132 kB/s | 45 kB 00:00 2024-09-10T11:17:27.756 INFO:teuthology.orchestra.run.smithi144.stdout:(15/30): boost-program-options-1.75.0-8.el9.x86 160 kB/s | 107 kB 00:00 2024-09-10T11:17:27.780 INFO:teuthology.orchestra.run.smithi144.stdout:(16/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 240 kB/s | 160 kB 00:00 2024-09-10T11:17:27.889 INFO:teuthology.orchestra.run.smithi144.stdout:(17/30): python3-prettytable-0.7.2-27.el9.noarc 390 kB/s | 42 kB 00:00 2024-09-10T11:17:27.931 INFO:teuthology.orchestra.run.smithi144.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 2.6 MB/s | 111 kB 00:00 2024-09-10T11:17:27.943 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-10T11:17:27.965 INFO:teuthology.orchestra.run.smithi144.stdout:(19/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 9.1 MB/s | 309 kB 00:00 2024-09-10T11:17:27.998 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-09-10T11:17:28.015 INFO:teuthology.orchestra.run.smithi144.stdout:(20/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.1 MB/s | 292 kB 00:00 2024-09-10T11:17:28.065 INFO:teuthology.orchestra.run.smithi144.stdout:(21/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 521 kB/s | 26 kB 00:00 2024-09-10T11:17:28.090 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-09-10T11:17:28.091 INFO:teuthology.orchestra.run.smithi144.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T11:17:28.124 INFO:teuthology.orchestra.run.smithi144.stdout:(23/30): librdkafka-1.6.1-102.el9.x86_64.rpm 1.5 MB/s | 662 kB 00:00 2024-09-10T11:17:28.149 INFO:teuthology.orchestra.run.smithi144.stdout:(24/30): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-09-10T11:17:28.178 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-09-10T11:17:28.208 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-09-10T11:17:28.225 INFO:teuthology.orchestra.run.smithi144.stdout:(25/30): libarrow-9.0.0-11.el9.x86_64.rpm 17 MB/s | 4.4 MB 00:00 2024-09-10T11:17:28.250 INFO:teuthology.orchestra.run.smithi144.stdout:(26/30): re2-20211101-20.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-10T11:17:28.280 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-09-10T11:17:28.316 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-10T11:17:28.325 INFO:teuthology.orchestra.run.smithi144.stdout:(27/30): librados2-18.2.0-0.el9.x86_64.rpm 44 MB/s | 3.3 MB 00:00 2024-09-10T11:17:28.359 INFO:teuthology.orchestra.run.smithi144.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-10T11:17:28.362 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-09-10T11:17:28.409 INFO:teuthology.orchestra.run.smithi144.stdout:(29/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.9 MB/s | 839 kB 00:00 2024-09-10T11:17:28.433 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-09-10T11:17:28.460 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-09-10T11:17:28.476 INFO:teuthology.orchestra.run.smithi144.stdout:(30/30): librbd1-18.2.0-0.el9.x86_64.rpm 20 MB/s | 3.0 MB 00:00 2024-09-10T11:17:28.481 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:17:28.481 INFO:teuthology.orchestra.run.smithi144.stdout:Total 23 MB/s | 53 MB 00:02 2024-09-10T11:17:28.526 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-09-10T11:17:28.554 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:17:28.589 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:17:28.590 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:17:28.631 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-10T11:17:28.659 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-10T11:17:28.701 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-10T11:17:28.732 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-09-10T11:17:28.759 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-10T11:17:28.795 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-10T11:17:28.828 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-10T11:17:29.126 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-10T11:17:29.174 INFO:teuthology.orchestra.run.smithi178.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-09-10T11:17:29.265 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-09-10T11:17:29.271 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:17:29.271 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:17:29.304 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-09-10T11:17:29.349 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-10T11:17:29.741 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-10T11:17:29.769 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-09-10T11:17:29.840 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-09-10T11:17:29.914 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-09-10T11:17:29.954 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-09-10T11:17:30.661 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:17:30.745 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-09-10T11:17:30.773 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-09-10T11:17:30.832 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-09-10T11:17:31.194 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-09-10T11:17:31.255 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-09-10T11:17:31.348 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-09-10T11:17:31.435 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-09-10T11:17:31.474 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-09-10T11:17:31.536 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-09-10T11:17:31.565 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-09-10T11:17:31.613 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-09-10T11:17:31.685 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-09-10T11:17:31.710 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-09-10T11:17:31.775 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-09-10T11:17:31.881 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : re2-1:20211101-20.el9.x86_64 12/32 2024-09-10T11:17:31.917 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-09-10T11:17:31.967 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-09-10T11:17:31.998 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-09-10T11:17:32.020 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-09-10T11:17:32.025 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-09-10T11:17:32.059 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-09-10T11:17:32.061 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-09-10T11:17:32.094 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-09-10T11:17:32.403 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-09-10T11:17:32.457 INFO:teuthology.orchestra.run.smithi144.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-09-10T11:17:32.547 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-09-10T11:17:32.587 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-09-10T11:17:32.606 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-09-10T11:17:32.640 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-09-10T11:17:32.645 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-09-10T11:17:32.741 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-09-10T11:17:32.741 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T11:17:32.741 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T11:17:32.741 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:17:32.771 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-09-10T11:17:33.040 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-09-10T11:17:33.085 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-09-10T11:17:33.157 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-09-10T11:17:33.232 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-09-10T11:17:33.262 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-09-10T11:17:35.351 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-09-10T11:17:35.392 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-09-10T11:17:35.979 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-09-10T11:17:36.021 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-09-10T11:17:36.122 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-09-10T11:17:36.123 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T11:17:36.123 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T11:17:36.123 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:17:36.163 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-09-10T11:17:42.463 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-09-10T11:17:42.463 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /sys 2024-09-10T11:17:42.463 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /proc 2024-09-10T11:17:42.463 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /mnt 2024-09-10T11:17:42.463 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /var/tmp 2024-09-10T11:17:42.463 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /home 2024-09-10T11:17:42.463 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /root 2024-09-10T11:17:42.464 INFO:teuthology.orchestra.run.smithi178.stdout:skipping the directory /tmp 2024-09-10T11:17:42.464 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:17:43.212 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-09-10T11:17:43.237 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-09-10T11:17:43.237 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:17:43.238 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T11:17:43.238 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T11:17:43.238 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T11:17:43.238 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:17:43.239 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-10T11:17:43.314 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-10T11:17:43.315 INFO:teuthology.orchestra.run.smithi178.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-10T11:17:44.464 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-10T11:17:44.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-09-10T11:17:44.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-09-10T11:17:44.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-09-10T11:17:44.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-09-10T11:17:44.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-09-10T11:17:44.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-09-10T11:17:44.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-09-10T11:17:44.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-09-10T11:17:44.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-09-10T11:17:44.465 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-09-10T11:17:44.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-09-10T11:17:44.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-09-10T11:17:44.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-09-10T11:17:44.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-10T11:17:44.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-10T11:17:44.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-10T11:17:44.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-10T11:17:44.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-10T11:17:44.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-10T11:17:44.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-10T11:17:44.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-09-10T11:17:44.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-10T11:17:44.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-10T11:17:44.466 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-10T11:17:44.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-10T11:17:44.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-10T11:17:44.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-10T11:17:44.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-10T11:17:44.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-09-10T11:17:44.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-10T11:17:44.467 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-09-10T11:17:45.458 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-10T11:17:45.458 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:17:45.458 INFO:teuthology.orchestra.run.smithi178.stdout:Upgraded: 2024-09-10T11:17:45.458 INFO:teuthology.orchestra.run.smithi178.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:45.458 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:17:45.458 INFO:teuthology.orchestra.run.smithi178.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T11:17:45.458 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:45.458 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:45.459 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:45.459 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:45.459 INFO:teuthology.orchestra.run.smithi178.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-10T11:17:45.459 INFO:teuthology.orchestra.run.smithi178.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T11:17:45.459 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T11:17:45.459 INFO:teuthology.orchestra.run.smithi178.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T11:17:45.459 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:45.459 INFO:teuthology.orchestra.run.smithi178.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T11:17:45.459 INFO:teuthology.orchestra.run.smithi178.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T11:17:45.459 INFO:teuthology.orchestra.run.smithi178.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T11:17:45.459 INFO:teuthology.orchestra.run.smithi178.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:45.459 INFO:teuthology.orchestra.run.smithi178.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T11:17:45.459 INFO:teuthology.orchestra.run.smithi178.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:45.460 INFO:teuthology.orchestra.run.smithi178.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T11:17:45.460 INFO:teuthology.orchestra.run.smithi178.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T11:17:45.460 INFO:teuthology.orchestra.run.smithi178.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T11:17:45.460 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:45.460 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:45.460 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:45.460 INFO:teuthology.orchestra.run.smithi178.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T11:17:45.460 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:45.460 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:45.460 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:45.460 INFO:teuthology.orchestra.run.smithi178.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T11:17:45.460 INFO:teuthology.orchestra.run.smithi178.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T11:17:45.460 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:17:45.461 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:17:45.870 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-test 2024-09-10T11:17:46.093 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-09-10T11:17:46.093 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /sys 2024-09-10T11:17:46.093 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /proc 2024-09-10T11:17:46.094 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /mnt 2024-09-10T11:17:46.094 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /var/tmp 2024-09-10T11:17:46.094 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /home 2024-09-10T11:17:46.094 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /root 2024-09-10T11:17:46.094 INFO:teuthology.orchestra.run.smithi144.stdout:skipping the directory /tmp 2024-09-10T11:17:46.094 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:17:46.440 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:25 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:17:46.577 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:17:46.577 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:17:46.578 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-09-10T11:17:46.578 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:17:46.578 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:17:46.578 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-09-10T11:17:46.578 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-10T11:17:46.578 INFO:teuthology.orchestra.run.smithi178.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-09-10T11:17:46.578 INFO:teuthology.orchestra.run.smithi178.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T11:17:46.578 INFO:teuthology.orchestra.run.smithi178.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T11:17:46.578 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:17:46.578 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:17:46.579 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:17:46.579 INFO:teuthology.orchestra.run.smithi178.stdout:Install 4 Packages 2024-09-10T11:17:46.579 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:17:46.579 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 40 M 2024-09-10T11:17:46.579 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 165 M 2024-09-10T11:17:46.579 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:17:46.720 INFO:teuthology.orchestra.run.smithi178.stdout:(1/4): libcephsqlite-18.2.0-0.el9.x86_64.rpm 1.7 MB/s | 161 kB 00:00 2024-09-10T11:17:46.812 INFO:teuthology.orchestra.run.smithi178.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.6 MB/s | 304 kB 00:00 2024-09-10T11:17:46.829 INFO:teuthology.orchestra.run.smithi178.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 587 kB/s | 64 kB 00:00 2024-09-10T11:17:46.841 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-09-10T11:17:46.866 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-09-10T11:17:46.866 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:17:46.866 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T11:17:46.867 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T11:17:46.867 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T11:17:46.867 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:17:46.868 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-10T11:17:46.951 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-09-10T11:17:46.952 INFO:teuthology.orchestra.run.smithi144.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-10T11:17:47.479 INFO:teuthology.orchestra.run.smithi178.stdout:(4/4): ceph-test-18.2.0-0.el9.x86_64.rpm 47 MB/s | 40 MB 00:00 2024-09-10T11:17:47.481 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:17:47.482 INFO:teuthology.orchestra.run.smithi178.stdout:Total 44 MB/s | 40 MB 00:00 2024-09-10T11:17:47.510 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:17:47.529 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:17:47.529 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:17:47.863 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:17:47.864 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:17:48.057 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-09-10T11:17:48.057 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-09-10T11:17:48.057 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-09-10T11:17:48.057 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-09-10T11:17:48.057 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-09-10T11:17:48.057 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-09-10T11:17:48.058 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-09-10T11:17:48.058 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-09-10T11:17:48.058 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-09-10T11:17:48.058 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-09-10T11:17:48.058 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-09-10T11:17:48.058 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-09-10T11:17:48.058 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-09-10T11:17:48.058 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-09-10T11:17:48.058 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-09-10T11:17:48.058 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-09-10T11:17:48.058 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-09-10T11:17:48.058 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-09-10T11:17:48.059 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-09-10T11:17:48.059 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-09-10T11:17:48.059 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-09-10T11:17:48.059 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-09-10T11:17:48.059 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-09-10T11:17:48.059 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-09-10T11:17:48.059 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-09-10T11:17:48.059 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-09-10T11:17:48.059 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-09-10T11:17:48.059 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : re2-1:20211101-20.el9.x86_64 27/32 2024-09-10T11:17:48.059 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-09-10T11:17:48.059 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-09-10T11:17:48.059 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-09-10T11:17:48.059 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-09-10T11:17:48.337 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:17:48.417 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T11:17:48.462 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T11:17:48.489 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-09-10T11:17:48.975 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-09-10T11:17:48.975 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:17:48.975 INFO:teuthology.orchestra.run.smithi144.stdout:Upgraded: 2024-09-10T11:17:48.976 INFO:teuthology.orchestra.run.smithi144.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:48.976 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:17:48.976 INFO:teuthology.orchestra.run.smithi144.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-10T11:17:48.976 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:48.976 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:48.976 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:48.976 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:48.976 INFO:teuthology.orchestra.run.smithi144.stdout: fmt-8.1.1-5.el9.x86_64 2024-09-10T11:17:48.976 INFO:teuthology.orchestra.run.smithi144.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T11:17:48.976 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-10T11:17:48.976 INFO:teuthology.orchestra.run.smithi144.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-10T11:17:48.976 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:48.977 INFO:teuthology.orchestra.run.smithi144.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T11:17:48.977 INFO:teuthology.orchestra.run.smithi144.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T11:17:48.977 INFO:teuthology.orchestra.run.smithi144.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T11:17:48.977 INFO:teuthology.orchestra.run.smithi144.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:48.977 INFO:teuthology.orchestra.run.smithi144.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T11:17:48.977 INFO:teuthology.orchestra.run.smithi144.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:48.977 INFO:teuthology.orchestra.run.smithi144.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T11:17:48.977 INFO:teuthology.orchestra.run.smithi144.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T11:17:48.977 INFO:teuthology.orchestra.run.smithi144.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-10T11:17:48.977 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:48.977 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:48.977 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:48.978 INFO:teuthology.orchestra.run.smithi144.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T11:17:48.978 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:48.978 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:48.978 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:48.978 INFO:teuthology.orchestra.run.smithi144.stdout: re2-1:20211101-20.el9.x86_64 2024-09-10T11:17:48.978 INFO:teuthology.orchestra.run.smithi144.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T11:17:48.978 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:17:48.978 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:17:49.353 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-test 2024-09-10T11:17:49.928 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:26 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:17:50.061 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-09-10T11:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-09-10T11:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-10T11:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-09-10T11:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T11:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T11:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:17:50.062 INFO:teuthology.orchestra.run.smithi144.stdout:Install 4 Packages 2024-09-10T11:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 40 M 2024-09-10T11:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 165 M 2024-09-10T11:17:50.063 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:17:50.365 INFO:teuthology.orchestra.run.smithi144.stdout:(1/4): libcephsqlite-18.2.0-0.el9.x86_64.rpm 3.4 MB/s | 161 kB 00:00 2024-09-10T11:17:50.449 INFO:teuthology.orchestra.run.smithi144.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 2.3 MB/s | 304 kB 00:00 2024-09-10T11:17:50.474 INFO:teuthology.orchestra.run.smithi144.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 587 kB/s | 64 kB 00:00 2024-09-10T11:17:50.883 INFO:teuthology.orchestra.run.smithi144.stdout:(4/4): ceph-test-18.2.0-0.el9.x86_64.rpm 70 MB/s | 40 MB 00:00 2024-09-10T11:17:50.885 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:17:50.885 INFO:teuthology.orchestra.run.smithi144.stdout:Total 49 MB/s | 40 MB 00:00 2024-09-10T11:17:50.912 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:17:50.932 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:17:50.932 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:17:51.263 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:17:51.264 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:17:51.745 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:17:51.844 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T11:17:51.890 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T11:17:51.918 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-09-10T11:17:52.181 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-09-10T11:17:52.288 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-09-10T11:17:54.345 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-09-10T11:17:54.345 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/4 2024-09-10T11:17:54.346 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/4 2024-09-10T11:17:54.346 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T11:17:54.655 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T11:17:54.655 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:17:54.655 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:17:54.655 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-test-2:18.2.0-0.el9.x86_64 libcephsqlite-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:54.655 INFO:teuthology.orchestra.run.smithi178.stdout: socat-1.7.4.1-6.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T11:17:54.655 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:17:54.656 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:17:54.966 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph 2024-09-10T11:17:55.537 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:34 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:17:55.663 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-09-10T11:17:55.678 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:17:55.681 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:17:55.681 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-10T11:17:55.681 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:17:55.681 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:17:55.681 INFO:teuthology.orchestra.run.smithi178.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-09-10T11:17:55.681 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-10T11:17:55.681 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-09-10T11:17:55.681 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-09-10T11:17:55.681 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-09-10T11:17:55.681 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-09-10T11:17:55.681 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-09-10T11:17:55.681 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T11:17:55.682 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T11:17:55.682 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T11:17:55.682 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T11:17:55.682 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T11:17:55.682 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T11:17:55.682 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T11:17:55.682 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T11:17:55.682 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T11:17:55.682 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T11:17:55.682 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T11:17:55.682 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T11:17:55.682 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T11:17:55.683 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T11:17:55.683 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T11:17:55.683 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T11:17:55.683 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T11:17:55.683 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T11:17:55.683 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T11:17:55.683 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T11:17:55.683 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T11:17:55.683 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T11:17:55.683 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T11:17:55.683 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T11:17:55.683 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T11:17:55.683 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T11:17:55.684 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T11:17:55.684 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:17:55.684 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:17:55.684 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:17:55.684 INFO:teuthology.orchestra.run.smithi178.stdout:Install 33 Packages 2024-09-10T11:17:55.684 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:17:55.684 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 30 M 2024-09-10T11:17:55.685 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 109 M 2024-09-10T11:17:55.685 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:17:55.732 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-09-10T11:17:55.963 INFO:teuthology.orchestra.run.smithi178.stdout:(1/33): ceph-18.2.0-0.el9.x86_64.rpm 98 kB/s | 6.4 kB 00:00 2024-09-10T11:17:56.022 INFO:teuthology.orchestra.run.smithi178.stdout:(2/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 12 MB/s | 1.4 MB 00:00 2024-09-10T11:17:56.080 INFO:teuthology.orchestra.run.smithi178.stdout:(3/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 11 MB/s | 2.1 MB 00:00 2024-09-10T11:17:56.464 INFO:teuthology.orchestra.run.smithi178.stdout:(4/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 40 MB/s | 18 MB 00:00 2024-09-10T11:17:56.489 INFO:teuthology.orchestra.run.smithi178.stdout:(5/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 588 kB/s | 240 kB 00:00 2024-09-10T11:17:56.564 INFO:teuthology.orchestra.run.smithi178.stdout:(6/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 7.3 MB/s | 4.4 MB 00:00 2024-09-10T11:17:56.673 INFO:teuthology.orchestra.run.smithi178.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 983 kB/s | 106 kB 00:00 2024-09-10T11:17:56.698 INFO:teuthology.orchestra.run.smithi178.stdout:(8/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-10T11:17:56.740 INFO:teuthology.orchestra.run.smithi178.stdout:(9/33): python3-pycparser-2.20-6.el9.noarch.rpm 2.0 MB/s | 135 kB 00:00 2024-09-10T11:17:56.774 INFO:teuthology.orchestra.run.smithi178.stdout:(10/33): python3-requests-2.25.1-8.el9.noarch.r 1.6 MB/s | 125 kB 00:00 2024-09-10T11:17:56.799 INFO:teuthology.orchestra.run.smithi178.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-10T11:17:56.849 INFO:teuthology.orchestra.run.smithi178.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 3.4 MB/s | 1.2 MB 00:00 2024-09-10T11:17:56.933 INFO:teuthology.orchestra.run.smithi178.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 260 kB/s | 35 kB 00:00 2024-09-10T11:17:56.967 INFO:teuthology.orchestra.run.smithi178.stdout:(14/33): python3-mako-1.1.4-6.el9.noarch.rpm 894 kB/s | 172 kB 00:00 2024-09-10T11:17:56.992 INFO:teuthology.orchestra.run.smithi178.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 358 kB/s | 51 kB 00:00 2024-09-10T11:17:57.017 INFO:teuthology.orchestra.run.smithi178.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 497 kB/s | 42 kB 00:00 2024-09-10T11:17:57.068 INFO:teuthology.orchestra.run.smithi178.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 433 kB/s | 43 kB 00:00 2024-09-10T11:17:57.101 INFO:teuthology.orchestra.run.smithi178.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 341 kB/s | 11 kB 00:00 2024-09-10T11:17:57.126 INFO:teuthology.orchestra.run.smithi178.stdout:(19/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-10T11:17:57.151 INFO:teuthology.orchestra.run.smithi178.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-10T11:17:57.177 INFO:teuthology.orchestra.run.smithi178.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 936 kB/s | 23 kB 00:00 2024-09-10T11:17:57.202 INFO:teuthology.orchestra.run.smithi178.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 390 kB/s | 19 kB 00:00 2024-09-10T11:17:57.228 INFO:teuthology.orchestra.run.smithi178.stdout:(23/33): python3-jaraco-text-3.2.0-6.el9.noarch 392 kB/s | 20 kB 00:00 2024-09-10T11:17:57.252 INFO:teuthology.orchestra.run.smithi178.stdout:(24/33): python3-logutils-0.3.5-21.el9.noarch.r 922 kB/s | 46 kB 00:00 2024-09-10T11:17:57.278 INFO:teuthology.orchestra.run.smithi178.stdout:(25/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-10T11:17:57.303 INFO:teuthology.orchestra.run.smithi178.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-10T11:17:57.328 INFO:teuthology.orchestra.run.smithi178.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-10T11:17:57.353 INFO:teuthology.orchestra.run.smithi178.stdout:(28/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-10T11:17:57.379 INFO:teuthology.orchestra.run.smithi178.stdout:(29/33): python3-tempora-5.0.0-2.el9.noarch.rpm 712 kB/s | 36 kB 00:00 2024-09-10T11:17:57.404 INFO:teuthology.orchestra.run.smithi178.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-09-10T11:17:57.429 INFO:teuthology.orchestra.run.smithi178.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 798 kB/s | 20 kB 00:00 2024-09-10T11:17:57.454 INFO:teuthology.orchestra.run.smithi178.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T11:17:57.480 INFO:teuthology.orchestra.run.smithi178.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-10T11:17:57.485 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:17:57.485 INFO:teuthology.orchestra.run.smithi178.stdout:Total 17 MB/s | 30 MB 00:01 2024-09-10T11:17:57.609 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:17:57.647 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:17:57.647 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:17:57.692 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-09-10T11:17:57.692 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/4 2024-09-10T11:17:57.693 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/4 2024-09-10T11:17:57.693 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T11:17:57.989 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T11:17:57.990 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:17:57.990 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:17:57.990 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-test-2:18.2.0-0.el9.x86_64 libcephsqlite-2:18.2.0-0.el9.x86_64 2024-09-10T11:17:57.990 INFO:teuthology.orchestra.run.smithi144.stdout: socat-1.7.4.1-6.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T11:17:57.990 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:17:57.990 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:17:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:17:58.064 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:17:58.292 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph 2024-09-10T11:17:58.872 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:35 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:17:59.016 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:17:59.018 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:17:59.018 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-10T11:17:59.018 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:17:59.018 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:17:59.018 INFO:teuthology.orchestra.run.smithi144.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-09-10T11:17:59.019 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-10T11:17:59.019 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-09-10T11:17:59.019 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-09-10T11:17:59.019 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-09-10T11:17:59.019 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-09-10T11:17:59.019 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-09-10T11:17:59.019 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T11:17:59.019 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T11:17:59.019 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T11:17:59.019 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T11:17:59.019 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T11:17:59.019 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T11:17:59.019 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T11:17:59.020 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-10T11:17:59.020 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T11:17:59.020 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T11:17:59.020 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T11:17:59.020 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T11:17:59.020 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T11:17:59.020 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T11:17:59.020 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T11:17:59.020 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T11:17:59.020 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T11:17:59.020 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T11:17:59.020 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T11:17:59.021 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T11:17:59.021 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T11:17:59.021 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T11:17:59.021 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T11:17:59.021 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T11:17:59.021 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T11:17:59.021 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T11:17:59.021 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T11:17:59.021 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:17:59.021 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:17:59.021 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:17:59.021 INFO:teuthology.orchestra.run.smithi144.stdout:Install 33 Packages 2024-09-10T11:17:59.022 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:17:59.022 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 30 M 2024-09-10T11:17:59.022 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 109 M 2024-09-10T11:17:59.022 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:17:59.279 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:17:59.349 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-10T11:17:59.381 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-10T11:17:59.423 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-10T11:17:59.514 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-10T11:17:59.593 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-10T11:17:59.609 INFO:teuthology.orchestra.run.smithi144.stdout:(1/33): ceph-18.2.0-0.el9.x86_64.rpm 96 kB/s | 6.4 kB 00:00 2024-09-10T11:17:59.634 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-10T11:17:59.651 INFO:teuthology.orchestra.run.smithi144.stdout:(2/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 19 MB/s | 2.1 MB 00:00 2024-09-10T11:17:59.668 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-10T11:17:59.693 INFO:teuthology.orchestra.run.smithi144.stdout:(3/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 9.6 MB/s | 1.4 MB 00:00 2024-09-10T11:17:59.699 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-10T11:17:59.718 INFO:teuthology.orchestra.run.smithi144.stdout:(4/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 9.4 MB/s | 240 kB 00:00 2024-09-10T11:17:59.733 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-10T11:17:59.766 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-10T11:17:59.802 INFO:teuthology.orchestra.run.smithi144.stdout:(5/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 23 MB/s | 4.4 MB 00:00 2024-09-10T11:17:59.807 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-10T11:17:59.841 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-10T11:17:59.876 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-10T11:17:59.915 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-10T11:17:59.988 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-10T11:18:00.044 INFO:teuthology.orchestra.run.smithi144.stdout:(6/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 45 MB/s | 18 MB 00:00 2024-09-10T11:18:00.069 INFO:teuthology.orchestra.run.smithi144.stdout:(7/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 723 kB/s | 253 kB 00:00 2024-09-10T11:18:00.111 INFO:teuthology.orchestra.run.smithi144.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-09-10T11:18:00.134 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-10T11:18:00.153 INFO:teuthology.orchestra.run.smithi144.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.9 MB/s | 125 kB 00:00 2024-09-10T11:18:00.178 INFO:teuthology.orchestra.run.smithi144.stdout:(10/33): python3-ply-3.11-14.el9.noarch.rpm 795 kB/s | 106 kB 00:00 2024-09-10T11:18:00.203 INFO:teuthology.orchestra.run.smithi144.stdout:(11/33): python3-urllib3-1.26.5-6.el9.noarch.rp 4.2 MB/s | 215 kB 00:00 2024-09-10T11:18:00.226 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-10T11:18:00.279 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-10T11:18:00.304 INFO:teuthology.orchestra.run.smithi144.stdout:(12/33): python3-cryptography-36.0.1-4.el9.x86_ 2.4 MB/s | 1.2 MB 00:00 2024-09-10T11:18:00.317 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-10T11:18:00.366 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-10T11:18:00.422 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-10T11:18:00.496 INFO:teuthology.orchestra.run.smithi144.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 119 kB/s | 35 kB 00:00 2024-09-10T11:18:00.616 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-10T11:18:00.646 INFO:teuthology.orchestra.run.smithi144.stdout:(14/33): python3-pytz-2021.1-5.el9.noarch.rpm 149 kB/s | 51 kB 00:00 2024-09-10T11:18:00.671 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-10T11:18:00.680 INFO:teuthology.orchestra.run.smithi144.stdout:(15/33): python3-toml-0.10.2-6.el9.noarch.rpm 226 kB/s | 42 kB 00:00 2024-09-10T11:18:00.713 INFO:teuthology.orchestra.run.smithi144.stdout:(16/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 648 kB/s | 43 kB 00:00 2024-09-10T11:18:00.741 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-10T11:18:00.747 INFO:teuthology.orchestra.run.smithi144.stdout:(17/33): python3-mako-1.1.4-6.el9.noarch.rpm 303 kB/s | 172 kB 00:00 2024-09-10T11:18:00.781 INFO:teuthology.orchestra.run.smithi144.stdout:(18/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-10T11:18:00.806 INFO:teuthology.orchestra.run.smithi144.stdout:(19/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 183 kB/s | 11 kB 00:00 2024-09-10T11:18:00.831 INFO:teuthology.orchestra.run.smithi144.stdout:(20/33): python3-jaraco-classes-3.2.1-5.el9.noa 363 kB/s | 18 kB 00:00 2024-09-10T11:18:00.838 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-10T11:18:00.856 INFO:teuthology.orchestra.run.smithi144.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 459 kB/s | 23 kB 00:00 2024-09-10T11:18:00.878 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-10T11:18:00.881 INFO:teuthology.orchestra.run.smithi144.stdout:(22/33): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-10T11:18:00.907 INFO:teuthology.orchestra.run.smithi144.stdout:(23/33): python3-cherrypy-18.6.1-2.el9.noarch.r 1.8 MB/s | 358 kB 00:00 2024-09-10T11:18:00.932 INFO:teuthology.orchestra.run.smithi144.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 261 kB/s | 20 kB 00:00 2024-09-10T11:18:00.958 INFO:teuthology.orchestra.run.smithi144.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 614 kB/s | 46 kB 00:00 2024-09-10T11:18:00.983 INFO:teuthology.orchestra.run.smithi144.stdout:(26/33): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-10T11:18:00.991 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-10T11:18:01.017 INFO:teuthology.orchestra.run.smithi144.stdout:(27/33): python3-portend-3.1.0-2.el9.noarch.rpm 280 kB/s | 16 kB 00:00 2024-09-10T11:18:01.042 INFO:teuthology.orchestra.run.smithi144.stdout:(28/33): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2024-09-10T11:18:01.067 INFO:teuthology.orchestra.run.smithi144.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.0 MB/s | 90 kB 00:00 2024-09-10T11:18:01.092 INFO:teuthology.orchestra.run.smithi144.stdout:(30/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-10T11:18:01.118 INFO:teuthology.orchestra.run.smithi144.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 801 kB/s | 20 kB 00:00 2024-09-10T11:18:01.143 INFO:teuthology.orchestra.run.smithi144.stdout:(32/33): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-10T11:18:01.157 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-09-10T11:18:01.168 INFO:teuthology.orchestra.run.smithi144.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.2 MB/s | 427 kB 00:00 2024-09-10T11:18:01.173 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:01.173 INFO:teuthology.orchestra.run.smithi144.stdout:Total 14 MB/s | 30 MB 00:02 2024-09-10T11:18:01.181 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-09-10T11:18:01.206 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-09-10T11:18:01.207 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:18:01.207 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T11:18:01.207 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T11:18:01.207 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T11:18:01.207 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:01.293 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:18:01.332 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:18:01.333 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:18:01.747 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:18:01.747 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:18:02.757 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-09-10T11:18:02.786 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-09-10T11:18:02.787 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:18:02.787 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T11:18:02.787 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T11:18:02.787 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T11:18:02.787 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:03.035 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:18:03.096 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-09-10T11:18:03.128 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-09-10T11:18:03.170 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-09-10T11:18:03.233 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-09-10T11:18:03.257 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-09-10T11:18:03.257 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:18:03.257 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T11:18:03.258 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T11:18:03.258 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T11:18:03.258 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:03.261 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-09-10T11:18:03.339 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-09-10T11:18:03.384 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/33 2024-09-10T11:18:03.419 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-09-10T11:18:03.457 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-09-10T11:18:03.467 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-09-10T11:18:03.491 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-09-10T11:18:03.491 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:18:03.491 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T11:18:03.491 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T11:18:03.491 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T11:18:03.492 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:03.492 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-09-10T11:18:03.526 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-09-10T11:18:03.556 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-09-10T11:18:03.566 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-09-10T11:18:03.601 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-09-10T11:18:03.636 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-09-10T11:18:03.673 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-09-10T11:18:03.746 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-09-10T11:18:03.886 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-09-10T11:18:03.973 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-09-10T11:18:04.035 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 18/33 2024-09-10T11:18:04.073 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-09-10T11:18:04.122 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-09-10T11:18:04.178 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-09-10T11:18:04.362 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-09-10T11:18:04.411 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-09-10T11:18:04.489 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-09-10T11:18:04.586 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-09-10T11:18:04.627 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-09-10T11:18:04.742 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-09-10T11:18:04.907 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-09-10T11:18:04.931 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-09-10T11:18:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-09-10T11:18:04.957 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:18:04.957 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T11:18:04.957 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T11:18:04.957 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T11:18:04.957 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:05.302 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-09-10T11:18:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-09-10T11:18:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-09-10T11:18:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-09-10T11:18:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-09-10T11:18:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-09-10T11:18:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-09-10T11:18:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-10T11:18:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-10T11:18:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-10T11:18:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-10T11:18:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-10T11:18:05.303 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-10T11:18:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-10T11:18:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-10T11:18:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-10T11:18:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-10T11:18:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-10T11:18:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-10T11:18:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-10T11:18:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-10T11:18:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-10T11:18:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-10T11:18:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-10T11:18:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-10T11:18:05.304 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-10T11:18:05.305 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-10T11:18:05.305 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-10T11:18:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-10T11:18:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-10T11:18:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-10T11:18:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-10T11:18:05.306 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-10T11:18:06.310 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-10T11:18:06.310 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:06.310 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:18:06.310 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-09-10T11:18:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T11:18:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T11:18:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T11:18:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T11:18:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T11:18:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T11:18:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T11:18:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T11:18:06.311 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T11:18:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T11:18:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T11:18:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T11:18:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T11:18:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T11:18:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T11:18:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T11:18:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T11:18:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T11:18:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T11:18:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T11:18:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T11:18:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T11:18:06.312 INFO:teuthology.orchestra.run.smithi178.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T11:18:06.313 INFO:teuthology.orchestra.run.smithi178.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T11:18:06.313 INFO:teuthology.orchestra.run.smithi178.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T11:18:06.313 INFO:teuthology.orchestra.run.smithi178.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T11:18:06.313 INFO:teuthology.orchestra.run.smithi178.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T11:18:06.313 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:06.313 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:06.524 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-09-10T11:18:06.553 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-09-10T11:18:06.553 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:18:06.553 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T11:18:06.553 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T11:18:06.553 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T11:18:06.553 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:06.714 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-base 2024-09-10T11:18:07.000 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-09-10T11:18:07.024 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-09-10T11:18:07.024 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:18:07.024 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T11:18:07.024 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T11:18:07.025 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T11:18:07.025 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:07.234 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-09-10T11:18:07.257 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-09-10T11:18:07.257 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:18:07.258 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T11:18:07.258 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T11:18:07.258 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T11:18:07.258 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:07.279 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:46 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:07.328 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-09-10T11:18:07.369 INFO:teuthology.orchestra.run.smithi178.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:07.410 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:07.411 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-10T11:18:07.411 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:07.494 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install cephadm 2024-09-10T11:18:08.055 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:47 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:08.186 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:08.187 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:08.187 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-09-10T11:18:08.187 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:08.187 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:18:08.187 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-09-10T11:18:08.187 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:08.187 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:18:08.188 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:08.188 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-09-10T11:18:08.188 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:08.188 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 209 k 2024-09-10T11:18:08.188 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 200 k 2024-09-10T11:18:08.188 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:18:08.265 INFO:teuthology.orchestra.run.smithi178.stdout:cephadm-18.2.0-0.el9.noarch.rpm 2.6 MB/s | 209 kB 00:00 2024-09-10T11:18:08.266 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:08.266 INFO:teuthology.orchestra.run.smithi178.stdout:Total 2.6 MB/s | 209 kB 00:00 2024-09-10T11:18:08.266 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:18:08.271 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:18:08.271 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:18:08.274 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:18:08.274 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:18:08.354 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:18:08.819 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-09-10T11:18:08.903 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-09-10T11:18:09.124 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-09-10T11:18:09.124 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-09-10T11:18:09.124 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-09-10T11:18:09.124 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-09-10T11:18:09.124 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-09-10T11:18:09.125 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-09-10T11:18:09.125 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-09-10T11:18:09.125 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-09-10T11:18:09.125 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-09-10T11:18:09.125 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-09-10T11:18:09.125 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-09-10T11:18:09.125 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-09-10T11:18:09.125 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/33 2024-09-10T11:18:09.125 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-09-10T11:18:09.125 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-09-10T11:18:09.125 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-09-10T11:18:09.125 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-09-10T11:18:09.125 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-09-10T11:18:09.126 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-09-10T11:18:09.126 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-09-10T11:18:09.126 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-09-10T11:18:09.126 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-09-10T11:18:09.126 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-09-10T11:18:09.126 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-09-10T11:18:09.126 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-09-10T11:18:09.126 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-09-10T11:18:09.126 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-09-10T11:18:09.126 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-09-10T11:18:09.126 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-09-10T11:18:09.126 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-09-10T11:18:09.126 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-09-10T11:18:09.127 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 31/33 2024-09-10T11:18:09.127 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-09-10T11:18:09.271 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-09-10T11:18:09.535 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-09-10T11:18:09.535 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:09.536 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:18:09.536 INFO:teuthology.orchestra.run.smithi178.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-09-10T11:18:09.536 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:09.536 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:09.778 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T11:18:09.937 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-09-10T11:18:09.937 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:09.937 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:18:09.937 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:09.937 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:09.938 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:09.938 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-09-10T11:18:09.938 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:09.938 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:09.938 INFO:teuthology.orchestra.run.smithi144.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T11:18:09.938 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T11:18:09.938 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T11:18:09.938 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T11:18:09.938 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T11:18:09.938 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T11:18:09.938 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T11:18:09.938 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T11:18:09.938 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T11:18:09.939 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T11:18:09.939 INFO:teuthology.orchestra.run.smithi144.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T11:18:09.939 INFO:teuthology.orchestra.run.smithi144.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T11:18:09.939 INFO:teuthology.orchestra.run.smithi144.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T11:18:09.939 INFO:teuthology.orchestra.run.smithi144.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T11:18:09.939 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T11:18:09.939 INFO:teuthology.orchestra.run.smithi144.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T11:18:09.939 INFO:teuthology.orchestra.run.smithi144.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T11:18:09.939 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T11:18:09.939 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T11:18:09.939 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T11:18:09.939 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T11:18:09.939 INFO:teuthology.orchestra.run.smithi144.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T11:18:09.940 INFO:teuthology.orchestra.run.smithi144.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T11:18:09.940 INFO:teuthology.orchestra.run.smithi144.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T11:18:09.940 INFO:teuthology.orchestra.run.smithi144.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T11:18:09.940 INFO:teuthology.orchestra.run.smithi144.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T11:18:09.940 INFO:teuthology.orchestra.run.smithi144.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T11:18:09.940 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:09.940 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:10.291 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-base 2024-09-10T11:18:10.346 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:49 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:10.481 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:10.482 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repo Size 2024-09-10T11:18:10.482 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:10.482 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:18:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-09-10T11:18:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:10.482 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:18:10.482 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:10.482 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-09-10T11:18:10.482 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:10.482 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 142 k 2024-09-10T11:18:10.482 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 399 k 2024-09-10T11:18:10.482 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:18:10.548 INFO:teuthology.orchestra.run.smithi178.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 2.1 MB/s | 142 kB 00:00 2024-09-10T11:18:10.549 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:10.549 INFO:teuthology.orchestra.run.smithi178.stdout:Total 2.1 MB/s | 142 kB 00:00 2024-09-10T11:18:10.549 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:18:10.556 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:18:10.556 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:18:10.613 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:18:10.614 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:18:10.758 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:18:10.816 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:10.841 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:10.841 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:18:10.841 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T11:18:10.841 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:10.873 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:47 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:10.963 INFO:teuthology.orchestra.run.smithi144.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:11.005 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:11.006 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-10T11:18:11.006 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:11.082 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install cephadm 2024-09-10T11:18:11.644 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:48 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:11.777 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:11.778 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:11.778 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-09-10T11:18:11.778 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:11.778 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:18:11.778 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-09-10T11:18:11.778 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:11.779 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:18:11.779 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:11.779 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-09-10T11:18:11.779 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:11.779 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 209 k 2024-09-10T11:18:11.779 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 200 k 2024-09-10T11:18:11.779 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:18:11.833 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:11.833 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:11.833 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:18:11.833 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:11.834 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:11.834 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:11.843 INFO:teuthology.orchestra.run.smithi144.stdout:cephadm-18.2.0-0.el9.noarch.rpm 3.2 MB/s | 209 kB 00:00 2024-09-10T11:18:11.844 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:11.844 INFO:teuthology.orchestra.run.smithi144.stdout:Total 3.1 MB/s | 209 kB 00:00 2024-09-10T11:18:11.844 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:18:11.849 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:18:11.849 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:18:11.852 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:18:11.853 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:18:11.924 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:18:12.067 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr 2024-09-10T11:18:12.398 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-09-10T11:18:12.489 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-09-10T11:18:12.668 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:51 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:12.762 INFO:teuthology.orchestra.run.smithi178.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:12.803 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:12.804 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-10T11:18:12.805 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:12.862 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-09-10T11:18:12.883 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T11:18:13.120 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-09-10T11:18:13.120 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:13.120 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:18:13.120 INFO:teuthology.orchestra.run.smithi144.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-09-10T11:18:13.120 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:13.120 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:13.400 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T11:18:13.448 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:52 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:13.580 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:13.581 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-10T11:18:13.581 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:13.581 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:18:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-09-10T11:18:13.581 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-10T11:18:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-09-10T11:18:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-09-10T11:18:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T11:18:13.581 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T11:18:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T11:18:13.582 INFO:teuthology.orchestra.run.smithi178.stdout:Installing weak dependencies: 2024-09-10T11:18:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T11:18:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:13.582 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:18:13.582 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:13.582 INFO:teuthology.orchestra.run.smithi178.stdout:Install 7 Packages 2024-09-10T11:18:13.582 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:13.583 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 2.0 M 2024-09-10T11:18:13.583 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 8.1 M 2024-09-10T11:18:13.583 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:18:13.811 INFO:teuthology.orchestra.run.smithi178.stdout:(1/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 400 kB/s | 23 kB 00:00 2024-09-10T11:18:13.827 INFO:teuthology.orchestra.run.smithi178.stdout:(2/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 197 kB/s | 15 kB 00:00 2024-09-10T11:18:13.869 INFO:teuthology.orchestra.run.smithi178.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 158 kB/s | 9.0 kB 00:00 2024-09-10T11:18:13.886 INFO:teuthology.orchestra.run.smithi178.stdout:(4/7): python3-jwt-2.4.0-1.el9.noarch.rpm 698 kB/s | 41 kB 00:00 2024-09-10T11:18:13.903 INFO:teuthology.orchestra.run.smithi178.stdout:(5/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 917 kB/s | 31 kB 00:00 2024-09-10T11:18:13.945 INFO:teuthology.orchestra.run.smithi178.stdout:(6/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 8.8 MB/s | 1.7 MB 00:00 2024-09-10T11:18:13.970 INFO:teuthology.orchestra.run.smithi178.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 2.2 MB/s | 188 kB 00:00 2024-09-10T11:18:13.973 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:13.974 INFO:teuthology.orchestra.run.smithi178.stdout:Total 5.1 MB/s | 2.0 MB 00:00 2024-09-10T11:18:13.974 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:18:13.981 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:50 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:13.982 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:18:13.982 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:18:14.060 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:18:14.060 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:18:14.120 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:14.136 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repo Size 2024-09-10T11:18:14.136 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:14.136 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:18:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-09-10T11:18:14.136 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:14.137 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:18:14.137 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:14.137 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-09-10T11:18:14.137 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:14.137 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 142 k 2024-09-10T11:18:14.137 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 399 k 2024-09-10T11:18:14.137 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:18:14.167 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 3.1 MB/s | 142 kB 00:00 2024-09-10T11:18:14.168 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:14.168 INFO:teuthology.orchestra.run.smithi144.stdout:Total 3.0 MB/s | 142 kB 00:00 2024-09-10T11:18:14.169 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:18:14.175 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:18:14.175 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:18:14.231 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:18:14.232 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:18:14.369 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:18:14.380 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:18:14.446 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-10T11:18:14.451 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:14.475 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:14.476 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:18:14.476 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T11:18:14.476 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:14.497 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-10T11:18:14.552 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-10T11:18:14.584 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-10T11:18:14.613 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-09-10T11:18:14.842 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-09-10T11:18:14.874 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-09-10T11:18:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-09-10T11:18:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-09-10T11:18:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-09-10T11:18:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-09-10T11:18:15.396 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-10T11:18:15.397 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-10T11:18:15.397 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-10T11:18:15.464 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:15.464 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:15.464 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:18:15.464 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:15.464 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:15.464 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:15.665 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-10T11:18:15.666 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:15.666 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:18:15.666 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-09-10T11:18:15.666 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-09-10T11:18:15.666 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-09-10T11:18:15.666 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T11:18:15.666 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T11:18:15.666 INFO:teuthology.orchestra.run.smithi178.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T11:18:15.666 INFO:teuthology.orchestra.run.smithi178.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T11:18:15.666 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:15.666 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:15.712 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr 2024-09-10T11:18:15.888 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T11:18:16.298 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:53 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:16.393 INFO:teuthology.orchestra.run.smithi144.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:16.435 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:16.436 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-10T11:18:16.436 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:16.447 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:00:55 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:16.519 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T11:18:16.581 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:16.582 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:16.582 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-10T11:18:16.582 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:16.582 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:18:16.582 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-09-10T11:18:16.582 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-10T11:18:16.582 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T11:18:16.582 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T11:18:16.582 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T11:18:16.582 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T11:18:16.582 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T11:18:16.583 INFO:teuthology.orchestra.run.smithi178.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T11:18:16.583 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T11:18:16.583 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T11:18:16.583 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T11:18:16.583 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T11:18:16.583 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T11:18:16.583 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:16.583 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:18:16.583 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:16.583 INFO:teuthology.orchestra.run.smithi178.stdout:Install 12 Packages 2024-09-10T11:18:16.583 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:16.584 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 38 M 2024-09-10T11:18:16.584 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 212 M 2024-09-10T11:18:16.584 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:18:17.035 INFO:teuthology.orchestra.run.smithi178.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.0-0. 26 MB/s | 7.4 MB 00:00 2024-09-10T11:18:17.092 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:54 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:17.224 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:17.225 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-10T11:18:17.225 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:17.225 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:18:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-09-10T11:18:17.225 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-10T11:18:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-09-10T11:18:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-09-10T11:18:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T11:18:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T11:18:17.225 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T11:18:17.226 INFO:teuthology.orchestra.run.smithi144.stdout:Installing weak dependencies: 2024-09-10T11:18:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T11:18:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:17.226 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:18:17.226 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:17.226 INFO:teuthology.orchestra.run.smithi144.stdout:Install 7 Packages 2024-09-10T11:18:17.226 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:17.226 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 2.0 M 2024-09-10T11:18:17.227 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 8.1 M 2024-09-10T11:18:17.227 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:18:17.346 INFO:teuthology.orchestra.run.smithi144.stdout:(1/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 397 kB/s | 23 kB 00:00 2024-09-10T11:18:17.371 INFO:teuthology.orchestra.run.smithi144.stdout:(2/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 176 kB/s | 15 kB 00:00 2024-09-10T11:18:17.413 INFO:teuthology.orchestra.run.smithi144.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-10T11:18:17.438 INFO:teuthology.orchestra.run.smithi144.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-10T11:18:17.463 INFO:teuthology.orchestra.run.smithi144.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 443 kB/s | 41 kB 00:00 2024-09-10T11:18:17.488 INFO:teuthology.orchestra.run.smithi144.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T11:18:17.530 INFO:teuthology.orchestra.run.smithi144.stdout:(7/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 7.0 MB/s | 1.7 MB 00:00 2024-09-10T11:18:17.534 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:17.534 INFO:teuthology.orchestra.run.smithi144.stdout:Total 6.5 MB/s | 2.0 MB 00:00 2024-09-10T11:18:17.534 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:18:17.542 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:18:17.542 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:18:17.610 INFO:teuthology.orchestra.run.smithi178.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 58 kB/s | 33 kB 00:00 2024-09-10T11:18:17.621 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:18:17.622 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:18:17.636 INFO:teuthology.orchestra.run.smithi178.stdout:(3/12): libquadmath-11.5.0-2.el9.x86_64.rpm 216 kB/s | 191 kB 00:00 2024-09-10T11:18:17.702 INFO:teuthology.orchestra.run.smithi178.stdout:(4/12): libgfortran-11.5.0-2.el9.x86_64.rpm 841 kB/s | 801 kB 00:00 2024-09-10T11:18:17.919 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:18:17.982 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-10T11:18:18.016 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-10T11:18:18.070 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-10T11:18:18.103 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-10T11:18:18.139 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-09-10T11:18:18.144 INFO:teuthology.orchestra.run.smithi178.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 36 kB/s | 18 kB 00:00 2024-09-10T11:18:18.261 INFO:teuthology.orchestra.run.smithi178.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-10T11:18:18.368 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-09-10T11:18:18.401 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-09-10T11:18:18.678 INFO:teuthology.orchestra.run.smithi178.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.8 MB/s | 3.0 MB 00:01 2024-09-10T11:18:18.909 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-09-10T11:18:18.910 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-09-10T11:18:18.910 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-09-10T11:18:18.910 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-09-10T11:18:18.910 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-10T11:18:18.910 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-10T11:18:18.910 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-10T11:18:19.070 INFO:teuthology.orchestra.run.smithi178.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 303 kB/s | 245 kB 00:00 2024-09-10T11:18:19.185 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-10T11:18:19.186 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:19.186 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:18:19.186 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-09-10T11:18:19.186 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-09-10T11:18:19.186 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-09-10T11:18:19.186 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T11:18:19.186 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T11:18:19.186 INFO:teuthology.orchestra.run.smithi144.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T11:18:19.186 INFO:teuthology.orchestra.run.smithi144.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T11:18:19.186 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:19.186 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:19.303 INFO:teuthology.orchestra.run.smithi178.stdout:(9/12): openblas-openmp-0.3.26-2.el9.x86_64.rpm 4.3 MB/s | 4.9 MB 00:01 2024-09-10T11:18:19.407 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T11:18:19.862 INFO:teuthology.orchestra.run.smithi178.stdout:(10/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 317 kB/s | 250 kB 00:00 2024-09-10T11:18:19.937 INFO:teuthology.orchestra.run.smithi178.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 4.1 MB/s | 5.1 MB 00:01 2024-09-10T11:18:19.966 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:00:56 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:20.098 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:20.099 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:20.099 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-10T11:18:20.099 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:20.099 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:18:20.099 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-09-10T11:18:20.100 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-10T11:18:20.100 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T11:18:20.100 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T11:18:20.100 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T11:18:20.100 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T11:18:20.100 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T11:18:20.100 INFO:teuthology.orchestra.run.smithi144.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T11:18:20.100 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T11:18:20.100 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T11:18:20.100 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T11:18:20.100 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T11:18:20.100 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T11:18:20.100 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:20.101 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:18:20.101 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:20.101 INFO:teuthology.orchestra.run.smithi144.stdout:Install 12 Packages 2024-09-10T11:18:20.101 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:20.101 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 38 M 2024-09-10T11:18:20.101 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 212 M 2024-09-10T11:18:20.101 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:18:20.604 INFO:teuthology.orchestra.run.smithi144.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.0-0. 51 MB/s | 7.4 MB 00:00 2024-09-10T11:18:20.721 INFO:teuthology.orchestra.run.smithi144.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 281 kB/s | 33 kB 00:00 2024-09-10T11:18:20.729 INFO:teuthology.orchestra.run.smithi178.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-10T11:18:20.731 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:20.731 INFO:teuthology.orchestra.run.smithi178.stdout:Total 9.2 MB/s | 38 MB 00:04 2024-09-10T11:18:21.038 INFO:teuthology.orchestra.run.smithi144.stdout:(3/12): libquadmath-11.5.0-2.el9.x86_64.rpm 330 kB/s | 191 kB 00:00 2024-09-10T11:18:21.092 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:18:21.118 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:18:21.118 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:18:21.155 INFO:teuthology.orchestra.run.smithi144.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 156 kB/s | 18 kB 00:00 2024-09-10T11:18:21.239 INFO:teuthology.orchestra.run.smithi144.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 461 kB/s | 38 kB 00:00 2024-09-10T11:18:21.281 INFO:teuthology.orchestra.run.smithi144.stdout:(6/12): libgfortran-11.5.0-2.el9.x86_64.rpm 976 kB/s | 801 kB 00:00 2024-09-10T11:18:21.481 INFO:teuthology.orchestra.run.smithi144.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.2 MB/s | 245 kB 00:00 2024-09-10T11:18:21.573 INFO:teuthology.orchestra.run.smithi144.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 3.5 MB/s | 3.0 MB 00:00 2024-09-10T11:18:21.607 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:18:21.607 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:18:21.723 INFO:teuthology.orchestra.run.smithi144.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 1.6 MB/s | 250 kB 00:00 2024-09-10T11:18:22.361 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:18:22.437 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T11:18:22.477 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T11:18:22.576 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T11:18:22.620 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T11:18:22.803 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T11:18:22.835 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T11:18:22.914 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T11:18:23.240 INFO:teuthology.orchestra.run.smithi144.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 2.5 MB/s | 4.9 MB 00:02 2024-09-10T11:18:23.374 INFO:teuthology.orchestra.run.smithi144.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.7 MB/s | 5.1 MB 00:01 2024-09-10T11:18:23.571 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T11:18:23.690 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T11:18:24.929 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T11:18:25.958 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T11:18:25.984 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-09-10T11:18:28.604 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-09-10T11:18:28.604 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-09-10T11:18:28.605 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T11:18:28.605 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T11:18:28.605 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T11:18:28.605 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T11:18:28.605 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T11:18:28.605 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T11:18:28.605 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T11:18:28.605 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T11:18:28.605 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T11:18:28.605 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T11:18:28.782 INFO:teuthology.orchestra.run.smithi144.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 2.3 MB/s | 16 MB 00:07 2024-09-10T11:18:28.782 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:28.782 INFO:teuthology.orchestra.run.smithi144.stdout:Total 4.4 MB/s | 38 MB 00:08 2024-09-10T11:18:29.051 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T11:18:29.051 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:29.051 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:18:29.051 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-09-10T11:18:29.052 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T11:18:29.052 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T11:18:29.052 INFO:teuthology.orchestra.run.smithi178.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T11:18:29.052 INFO:teuthology.orchestra.run.smithi178.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T11:18:29.052 INFO:teuthology.orchestra.run.smithi178.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T11:18:29.052 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T11:18:29.052 INFO:teuthology.orchestra.run.smithi178.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T11:18:29.052 INFO:teuthology.orchestra.run.smithi178.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T11:18:29.052 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T11:18:29.052 INFO:teuthology.orchestra.run.smithi178.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T11:18:29.052 INFO:teuthology.orchestra.run.smithi178.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T11:18:29.052 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:29.052 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:29.141 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:18:29.168 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:18:29.169 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:18:29.464 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-rook 2024-09-10T11:18:29.690 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:18:29.691 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:18:30.062 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:09 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:30.196 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:30.197 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:30.197 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-10T11:18:30.198 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:30.198 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:18:30.198 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-09-10T11:18:30.198 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-10T11:18:30.198 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T11:18:30.198 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T11:18:30.198 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T11:18:30.198 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T11:18:30.198 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T11:18:30.198 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T11:18:30.198 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T11:18:30.199 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T11:18:30.199 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T11:18:30.199 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T11:18:30.199 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T11:18:30.199 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T11:18:30.199 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:30.199 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:18:30.199 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:30.199 INFO:teuthology.orchestra.run.smithi178.stdout:Install 13 Packages 2024-09-10T11:18:30.199 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:30.200 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 2.2 M 2024-09-10T11:18:30.200 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 26 M 2024-09-10T11:18:30.200 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:18:30.342 INFO:teuthology.orchestra.run.smithi178.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 744 kB/s | 47 kB 00:00 2024-09-10T11:18:30.367 INFO:teuthology.orchestra.run.smithi178.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 288 kB/s | 26 kB 00:00 2024-09-10T11:18:30.393 INFO:teuthology.orchestra.run.smithi178.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 168 kB/s | 19 kB 00:00 2024-09-10T11:18:30.451 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:18:30.485 INFO:teuthology.orchestra.run.smithi178.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-09-10T11:18:30.510 INFO:teuthology.orchestra.run.smithi178.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.3 MB/s | 279 kB 00:00 2024-09-10T11:18:30.535 INFO:teuthology.orchestra.run.smithi178.stdout:(6/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.1 MB/s | 222 kB 00:00 2024-09-10T11:18:30.540 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T11:18:30.560 INFO:teuthology.orchestra.run.smithi178.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 712 kB/s | 54 kB 00:00 2024-09-10T11:18:30.579 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T11:18:30.610 INFO:teuthology.orchestra.run.smithi178.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 320 kB/s | 32 kB 00:00 2024-09-10T11:18:30.636 INFO:teuthology.orchestra.run.smithi178.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 141 kB/s | 14 kB 00:00 2024-09-10T11:18:30.661 INFO:teuthology.orchestra.run.smithi178.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T11:18:30.681 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T11:18:30.686 INFO:teuthology.orchestra.run.smithi178.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.9 MB/s | 220 kB 00:00 2024-09-10T11:18:30.711 INFO:teuthology.orchestra.run.smithi178.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.7 MB/s | 90 kB 00:00 2024-09-10T11:18:30.714 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T11:18:30.745 INFO:teuthology.orchestra.run.smithi178.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.7 MB/s | 1.0 MB 00:00 2024-09-10T11:18:30.748 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:30.748 INFO:teuthology.orchestra.run.smithi178.stdout:Total 4.0 MB/s | 2.2 MB 00:00 2024-09-10T11:18:30.812 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:18:30.827 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:18:30.827 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:18:30.899 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T11:18:30.915 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:18:30.916 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:18:30.929 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T11:18:31.014 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T11:18:31.422 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:18:31.585 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T11:18:31.637 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T11:18:31.693 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T11:18:31.699 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T11:18:31.730 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T11:18:31.772 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T11:18:31.826 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T11:18:31.874 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T11:18:31.985 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T11:18:32.032 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T11:18:32.743 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T11:18:32.795 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T11:18:32.829 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T11:18:32.875 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T11:18:32.899 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-09-10T11:18:33.079 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T11:18:33.677 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-09-10T11:18:33.677 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-09-10T11:18:33.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T11:18:33.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T11:18:33.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T11:18:33.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T11:18:33.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T11:18:33.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T11:18:33.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T11:18:33.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T11:18:33.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T11:18:33.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T11:18:33.678 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T11:18:34.035 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T11:18:34.035 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:34.036 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:18:34.036 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-09-10T11:18:34.036 INFO:teuthology.orchestra.run.smithi178.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T11:18:34.036 INFO:teuthology.orchestra.run.smithi178.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T11:18:34.036 INFO:teuthology.orchestra.run.smithi178.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T11:18:34.036 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T11:18:34.036 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T11:18:34.036 INFO:teuthology.orchestra.run.smithi178.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T11:18:34.036 INFO:teuthology.orchestra.run.smithi178.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T11:18:34.036 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T11:18:34.036 INFO:teuthology.orchestra.run.smithi178.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T11:18:34.036 INFO:teuthology.orchestra.run.smithi178.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T11:18:34.036 INFO:teuthology.orchestra.run.smithi178.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T11:18:34.037 INFO:teuthology.orchestra.run.smithi178.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T11:18:34.037 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:34.037 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:34.120 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T11:18:34.163 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-09-10T11:18:34.283 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T11:18:34.865 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:13 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:34.999 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:35.000 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:35.000 INFO:teuthology.orchestra.run.smithi178.stdout: Package Arch Version Repository Size 2024-09-10T11:18:35.000 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:35.000 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:18:35.000 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-09-10T11:18:35.001 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-10T11:18:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T11:18:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T11:18:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T11:18:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T11:18:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T11:18:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:35.001 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:18:35.001 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:35.001 INFO:teuthology.orchestra.run.smithi178.stdout:Install 6 Packages 2024-09-10T11:18:35.001 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:35.002 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 7.0 M 2024-09-10T11:18:35.002 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 33 M 2024-09-10T11:18:35.002 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:18:35.273 INFO:teuthology.orchestra.run.smithi178.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 1.9 MB/s | 127 kB 00:00 2024-09-10T11:18:35.323 INFO:teuthology.orchestra.run.smithi178.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 11 MB/s | 548 kB 00:00 2024-09-10T11:18:35.348 INFO:teuthology.orchestra.run.smithi178.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.2 MB/s | 58 kB 00:00 2024-09-10T11:18:35.373 INFO:teuthology.orchestra.run.smithi178.stdout:(4/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.5 MB/s | 248 kB 00:00 2024-09-10T11:18:35.399 INFO:teuthology.orchestra.run.smithi178.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.5 MB/s | 74 kB 00:00 2024-09-10T11:18:36.166 INFO:teuthology.orchestra.run.smithi178.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 6.2 MB/s | 6.0 MB 00:00 2024-09-10T11:18:36.168 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:36.168 INFO:teuthology.orchestra.run.smithi178.stdout:Total 6.0 MB/s | 7.0 MB 00:01 2024-09-10T11:18:36.239 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:18:36.250 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:18:36.250 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:18:36.349 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:18:36.349 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:18:36.660 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:18:36.785 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-09-10T11:18:36.786 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-09-10T11:18:36.786 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T11:18:36.786 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T11:18:36.786 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T11:18:36.786 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T11:18:36.786 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T11:18:36.786 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T11:18:36.786 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T11:18:36.786 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T11:18:36.786 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T11:18:36.786 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T11:18:36.830 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T11:18:36.875 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T11:18:37.238 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T11:18:37.239 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:37.239 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:18:37.239 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-09-10T11:18:37.239 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T11:18:37.239 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T11:18:37.239 INFO:teuthology.orchestra.run.smithi144.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T11:18:37.239 INFO:teuthology.orchestra.run.smithi144.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T11:18:37.239 INFO:teuthology.orchestra.run.smithi144.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T11:18:37.239 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T11:18:37.239 INFO:teuthology.orchestra.run.smithi144.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T11:18:37.239 INFO:teuthology.orchestra.run.smithi144.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T11:18:37.239 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T11:18:37.239 INFO:teuthology.orchestra.run.smithi144.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T11:18:37.240 INFO:teuthology.orchestra.run.smithi144.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T11:18:37.240 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:37.240 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:37.383 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T11:18:37.469 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T11:18:37.537 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T11:18:37.563 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-09-10T11:18:37.623 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-rook 2024-09-10T11:18:38.223 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:15 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:38.367 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:38.369 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:38.369 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-10T11:18:38.369 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:38.369 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:18:38.369 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-09-10T11:18:38.369 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-10T11:18:38.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T11:18:38.369 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T11:18:38.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T11:18:38.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T11:18:38.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T11:18:38.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T11:18:38.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T11:18:38.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T11:18:38.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T11:18:38.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T11:18:38.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T11:18:38.370 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T11:18:38.370 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:38.371 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:18:38.371 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:38.371 INFO:teuthology.orchestra.run.smithi144.stdout:Install 13 Packages 2024-09-10T11:18:38.371 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:38.371 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 2.2 M 2024-09-10T11:18:38.372 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 26 M 2024-09-10T11:18:38.372 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:18:38.398 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-09-10T11:18:38.398 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-09-10T11:18:38.398 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T11:18:38.398 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T11:18:38.398 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T11:18:38.398 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T11:18:38.454 INFO:teuthology.orchestra.run.smithi144.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 1.6 MB/s | 47 kB 00:00 2024-09-10T11:18:38.513 INFO:teuthology.orchestra.run.smithi144.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 219 kB/s | 19 kB 00:00 2024-09-10T11:18:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:(3/13): python3-jsonpatch-1.21-16.el9.noarch.rp 245 kB/s | 26 kB 00:00 2024-09-10T11:18:38.614 INFO:teuthology.orchestra.run.smithi144.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.4 MB/s | 222 kB 00:00 2024-09-10T11:18:38.639 INFO:teuthology.orchestra.run.smithi144.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-09-10T11:18:38.664 INFO:teuthology.orchestra.run.smithi144.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.0 MB/s | 54 kB 00:00 2024-09-10T11:18:38.670 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T11:18:38.670 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:38.671 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:18:38.671 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-09-10T11:18:38.671 INFO:teuthology.orchestra.run.smithi178.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T11:18:38.671 INFO:teuthology.orchestra.run.smithi178.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T11:18:38.671 INFO:teuthology.orchestra.run.smithi178.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T11:18:38.671 INFO:teuthology.orchestra.run.smithi178.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T11:18:38.671 INFO:teuthology.orchestra.run.smithi178.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T11:18:38.671 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:38.671 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:38.698 INFO:teuthology.orchestra.run.smithi144.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.6 MB/s | 279 kB 00:00 2024-09-10T11:18:38.748 INFO:teuthology.orchestra.run.smithi144.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 296 kB/s | 32 kB 00:00 2024-09-10T11:18:38.773 INFO:teuthology.orchestra.run.smithi144.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 129 kB/s | 14 kB 00:00 2024-09-10T11:18:38.798 INFO:teuthology.orchestra.run.smithi144.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-10T11:18:38.832 INFO:teuthology.orchestra.run.smithi144.stdout:(11/13): python3-google-auth-2.34.0-1.el9.noarc 1.8 MB/s | 220 kB 00:00 2024-09-10T11:18:38.857 INFO:teuthology.orchestra.run.smithi144.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-09-10T11:18:38.891 INFO:teuthology.orchestra.run.smithi144.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.3 MB/s | 1.0 MB 00:00 2024-09-10T11:18:38.894 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:38.894 INFO:teuthology.orchestra.run.smithi144.stdout:Total 4.2 MB/s | 2.2 MB 00:00 2024-09-10T11:18:38.909 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install ceph-fuse 2024-09-10T11:18:38.961 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:18:38.976 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:18:38.976 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:18:39.066 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:18:39.066 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:18:39.471 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:18 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:39.578 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:18:39.604 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:39.605 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:39.605 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-09-10T11:18:39.605 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:39.605 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:18:39.605 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-09-10T11:18:39.605 INFO:teuthology.orchestra.run.smithi178.stdout:Installing dependencies: 2024-09-10T11:18:39.605 INFO:teuthology.orchestra.run.smithi178.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T11:18:39.605 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:39.606 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:18:39.606 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:39.606 INFO:teuthology.orchestra.run.smithi178.stdout:Install 2 Packages 2024-09-10T11:18:39.606 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:39.606 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 914 k 2024-09-10T11:18:39.606 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 2.7 M 2024-09-10T11:18:39.606 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:18:39.701 INFO:teuthology.orchestra.run.smithi178.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 11 MB/s | 835 kB 00:00 2024-09-10T11:18:39.742 INFO:teuthology.orchestra.run.smithi178.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 691 kB/s | 80 kB 00:00 2024-09-10T11:18:39.743 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:39.743 INFO:teuthology.orchestra.run.smithi178.stdout:Total 6.5 MB/s | 914 kB 00:00 2024-09-10T11:18:39.754 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:18:39.756 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T11:18:39.775 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:18:39.775 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:18:39.817 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T11:18:39.859 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:18:39.859 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:18:39.873 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T11:18:39.910 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T11:18:39.951 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T11:18:40.048 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:18:40.053 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T11:18:40.162 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T11:18:40.200 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T11:18:40.211 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T11:18:40.265 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-09-10T11:18:40.923 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T11:18:40.954 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-09-10T11:18:40.954 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-09-10T11:18:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T11:18:41.018 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T11:18:41.072 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T11:18:41.096 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-09-10T11:18:41.239 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T11:18:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:41.240 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:18:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T11:18:41.240 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:41.240 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:41.530 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install librados-devel 2024-09-10T11:18:41.931 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-09-10T11:18:41.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-09-10T11:18:41.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T11:18:41.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T11:18:41.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T11:18:41.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T11:18:41.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T11:18:41.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T11:18:41.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T11:18:41.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T11:18:41.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T11:18:41.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T11:18:41.932 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T11:18:42.108 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:21 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:42.245 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:42.245 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:42.246 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-09-10T11:18:42.246 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:42.246 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:18:42.246 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-09-10T11:18:42.246 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:42.246 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:18:42.246 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:42.246 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-09-10T11:18:42.246 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:42.246 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 126 k 2024-09-10T11:18:42.246 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 456 k 2024-09-10T11:18:42.246 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:18:42.266 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T11:18:42.266 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:42.266 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:18:42.266 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-09-10T11:18:42.266 INFO:teuthology.orchestra.run.smithi144.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T11:18:42.266 INFO:teuthology.orchestra.run.smithi144.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T11:18:42.266 INFO:teuthology.orchestra.run.smithi144.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T11:18:42.266 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T11:18:42.266 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T11:18:42.266 INFO:teuthology.orchestra.run.smithi144.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T11:18:42.266 INFO:teuthology.orchestra.run.smithi144.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T11:18:42.266 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T11:18:42.266 INFO:teuthology.orchestra.run.smithi144.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T11:18:42.267 INFO:teuthology.orchestra.run.smithi144.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T11:18:42.267 INFO:teuthology.orchestra.run.smithi144.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T11:18:42.267 INFO:teuthology.orchestra.run.smithi144.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T11:18:42.267 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:42.267 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:42.304 INFO:teuthology.orchestra.run.smithi178.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 2.1 MB/s | 126 kB 00:00 2024-09-10T11:18:42.305 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:42.305 INFO:teuthology.orchestra.run.smithi178.stdout:Total 2.1 MB/s | 126 kB 00:00 2024-09-10T11:18:42.305 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:18:42.311 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:18:42.311 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:18:42.373 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:18:42.374 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:18:42.515 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T11:18:42.520 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:18:42.627 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:43.025 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:43.083 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:20 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:43.218 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:43.219 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:43.219 INFO:teuthology.orchestra.run.smithi144.stdout: Package Arch Version Repository Size 2024-09-10T11:18:43.219 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:43.219 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:18:43.219 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-09-10T11:18:43.219 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-10T11:18:43.219 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T11:18:43.219 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T11:18:43.220 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T11:18:43.220 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T11:18:43.220 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T11:18:43.220 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:43.220 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:18:43.220 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:43.220 INFO:teuthology.orchestra.run.smithi144.stdout:Install 6 Packages 2024-09-10T11:18:43.220 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:43.221 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 7.0 M 2024-09-10T11:18:43.221 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 33 M 2024-09-10T11:18:43.221 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:18:43.288 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:43.288 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:43.288 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:18:43.288 INFO:teuthology.orchestra.run.smithi178.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:43.288 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:43.288 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:43.551 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install libcephfs2 2024-09-10T11:18:43.660 INFO:teuthology.orchestra.run.smithi144.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 2.3 MB/s | 127 kB 00:00 2024-09-10T11:18:43.735 INFO:teuthology.orchestra.run.smithi144.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-10T11:18:43.761 INFO:teuthology.orchestra.run.smithi144.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 2.3 MB/s | 58 kB 00:00 2024-09-10T11:18:43.786 INFO:teuthology.orchestra.run.smithi144.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.9 MB/s | 74 kB 00:00 2024-09-10T11:18:43.886 INFO:teuthology.orchestra.run.smithi144.stdout:(5/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 883 kB/s | 248 kB 00:00 2024-09-10T11:18:44.131 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:23 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:44.224 INFO:teuthology.orchestra.run.smithi178.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:44.267 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:44.268 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-10T11:18:44.268 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:44.362 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install libcephfs-devel 2024-09-10T11:18:44.745 INFO:teuthology.orchestra.run.smithi144.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 5.2 MB/s | 6.0 MB 00:01 2024-09-10T11:18:44.747 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:44.747 INFO:teuthology.orchestra.run.smithi144.stdout:Total 4.6 MB/s | 7.0 MB 00:01 2024-09-10T11:18:44.820 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:18:44.832 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:18:44.832 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:18:44.932 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:18:44.933 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:18:44.955 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:23 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:45.091 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:45.091 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:45.091 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-09-10T11:18:45.091 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:45.091 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:18:45.091 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-09-10T11:18:45.091 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:45.091 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:18:45.091 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:45.091 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-09-10T11:18:45.091 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:45.092 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 30 k 2024-09-10T11:18:45.092 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 137 k 2024-09-10T11:18:45.092 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:18:45.166 INFO:teuthology.orchestra.run.smithi178.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 406 kB/s | 30 kB 00:00 2024-09-10T11:18:45.167 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:45.167 INFO:teuthology.orchestra.run.smithi178.stdout:Total 399 kB/s | 30 kB 00:00 2024-09-10T11:18:45.167 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:18:45.170 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:18:45.170 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:18:45.190 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:18:45.190 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:18:45.238 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:18:45.276 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:18:45.380 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:45.412 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T11:18:45.463 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T11:18:45.739 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:45.968 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:45.968 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:45.968 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:18:45.968 INFO:teuthology.orchestra.run.smithi178.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:45.968 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:45.968 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:45.976 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T11:18:46.058 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T11:18:46.117 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T11:18:46.151 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-09-10T11:18:46.155 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install librados2 2024-09-10T11:18:46.722 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:25 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:46.812 INFO:teuthology.orchestra.run.smithi178.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:46.855 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:46.855 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-10T11:18:46.856 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-09-10T11:18:46.924 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-09-10T11:18:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T11:18:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T11:18:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T11:18:46.925 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T11:18:46.933 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install librbd1 2024-09-10T11:18:47.189 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T11:18:47.189 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:47.189 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:18:47.189 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-09-10T11:18:47.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T11:18:47.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T11:18:47.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T11:18:47.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T11:18:47.190 INFO:teuthology.orchestra.run.smithi144.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T11:18:47.190 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:47.190 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:47.432 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install ceph-fuse 2024-09-10T11:18:47.501 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:26 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:47.591 INFO:teuthology.orchestra.run.smithi178.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:47.633 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:47.634 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-10T11:18:47.634 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:47.711 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-rados 2024-09-10T11:18:47.995 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:24 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:48.128 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:48.128 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:48.128 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-09-10T11:18:48.128 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:48.128 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:18:48.128 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-09-10T11:18:48.128 INFO:teuthology.orchestra.run.smithi144.stdout:Installing dependencies: 2024-09-10T11:18:48.129 INFO:teuthology.orchestra.run.smithi144.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T11:18:48.129 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:48.129 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:18:48.129 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:48.129 INFO:teuthology.orchestra.run.smithi144.stdout:Install 2 Packages 2024-09-10T11:18:48.129 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:48.129 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 914 k 2024-09-10T11:18:48.130 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 2.7 M 2024-09-10T11:18:48.130 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:18:48.273 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:27 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:48.288 INFO:teuthology.orchestra.run.smithi144.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 13 MB/s | 835 kB 00:00 2024-09-10T11:18:48.363 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:48.372 INFO:teuthology.orchestra.run.smithi144.stdout:(2/2): fuse-2.9.9-16.el9.x86_64.rpm 552 kB/s | 80 kB 00:00 2024-09-10T11:18:48.372 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:48.373 INFO:teuthology.orchestra.run.smithi144.stdout:Total 3.7 MB/s | 914 kB 00:00 2024-09-10T11:18:48.384 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:18:48.405 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:48.406 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:18:48.406 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:18:48.406 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-10T11:18:48.407 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:48.482 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-rgw 2024-09-10T11:18:48.489 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:18:48.489 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:18:48.691 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:18:48.841 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T11:18:48.905 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-09-10T11:18:49.050 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:28 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:49.140 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:49.182 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:49.183 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-10T11:18:49.183 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:49.262 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-cephfs 2024-09-10T11:18:49.630 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-09-10T11:18:49.630 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-09-10T11:18:49.822 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:28 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:49.912 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:49.923 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T11:18:49.923 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:49.923 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:18:49.923 INFO:teuthology.orchestra.run.smithi144.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T11:18:49.924 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:49.924 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:49.955 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:49.956 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-10T11:18:49.956 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:50.036 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install python3-rbd 2024-09-10T11:18:50.201 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install librados-devel 2024-09-10T11:18:50.601 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:29 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:50.691 INFO:teuthology.orchestra.run.smithi178.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:50.734 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:50.735 INFO:teuthology.orchestra.run.smithi178.stdout:Nothing to do. 2024-09-10T11:18:50.735 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:50.779 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:27 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:50.828 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install rbd-fuse 2024-09-10T11:18:50.917 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:50.918 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-09-10T11:18:50.918 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:50.918 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:18:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-09-10T11:18:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:50.918 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:18:50.918 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:50.918 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-09-10T11:18:50.918 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:50.918 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 126 k 2024-09-10T11:18:50.919 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 456 k 2024-09-10T11:18:50.919 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:18:50.961 INFO:teuthology.orchestra.run.smithi144.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 2.9 MB/s | 126 kB 00:00 2024-09-10T11:18:50.961 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:50.961 INFO:teuthology.orchestra.run.smithi144.stdout:Total 2.8 MB/s | 126 kB 00:00 2024-09-10T11:18:50.962 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:18:50.967 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:18:50.967 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:18:51.030 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:18:51.030 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:18:51.181 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:18:51.282 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:51.390 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:30 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:51.522 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:51.523 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:51.523 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-09-10T11:18:51.523 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:51.523 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:18:51.523 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-09-10T11:18:51.523 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:51.523 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:18:51.523 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:51.523 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-09-10T11:18:51.523 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:51.524 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 86 k 2024-09-10T11:18:51.524 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 230 k 2024-09-10T11:18:51.524 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:18:51.585 INFO:teuthology.orchestra.run.smithi178.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 1.4 MB/s | 86 kB 00:00 2024-09-10T11:18:51.586 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:51.586 INFO:teuthology.orchestra.run.smithi178.stdout:Total 1.3 MB/s | 86 kB 00:00 2024-09-10T11:18:51.586 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:18:51.592 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:18:51.592 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:18:51.655 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:18:51.655 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:18:51.698 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:51.803 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:18:51.907 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:51.968 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:51.968 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:51.968 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:18:51.969 INFO:teuthology.orchestra.run.smithi144.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:51.969 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:51.969 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:52.204 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install libcephfs2 2024-09-10T11:18:52.272 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:52.527 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:52.527 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:52.527 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:18:52.528 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:52.528 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:52.528 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:52.752 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install rbd-mirror 2024-09-10T11:18:52.782 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:29 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:52.875 INFO:teuthology.orchestra.run.smithi144.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:52.917 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:52.918 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-10T11:18:52.919 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:53.021 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install libcephfs-devel 2024-09-10T11:18:53.334 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:32 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:53.471 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:53.471 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:53.472 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-09-10T11:18:53.472 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:53.472 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:18:53.472 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-09-10T11:18:53.472 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:53.472 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:18:53.472 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:53.472 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-09-10T11:18:53.472 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:53.472 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 3.0 M 2024-09-10T11:18:53.472 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 12 M 2024-09-10T11:18:53.473 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:18:53.588 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:30 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:53.621 INFO:teuthology.orchestra.run.smithi178.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 20 MB/s | 3.0 MB 00:00 2024-09-10T11:18:53.622 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:53.622 INFO:teuthology.orchestra.run.smithi178.stdout:Total 20 MB/s | 3.0 MB 00:00 2024-09-10T11:18:53.622 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:18:53.631 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:18:53.631 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:18:53.709 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:18:53.710 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:18:53.724 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:53.724 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:53.724 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-09-10T11:18:53.724 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:53.725 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:18:53.725 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-09-10T11:18:53.725 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:53.725 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:18:53.725 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:18:53.725 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-09-10T11:18:53.725 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:53.726 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 30 k 2024-09-10T11:18:53.726 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 137 k 2024-09-10T11:18:53.726 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:18:53.781 INFO:teuthology.orchestra.run.smithi144.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 549 kB/s | 30 kB 00:00 2024-09-10T11:18:53.782 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:53.782 INFO:teuthology.orchestra.run.smithi144.stdout:Total 535 kB/s | 30 kB 00:00 2024-09-10T11:18:53.782 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:18:53.786 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:18:53.786 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:18:53.805 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:18:53.805 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:18:53.888 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:18:54.003 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:54.123 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:18:54.197 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:54.222 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:54.222 INFO:teuthology.orchestra.run.smithi178.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:18:54.222 INFO:teuthology.orchestra.run.smithi178.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T11:18:54.223 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T11:18:54.223 INFO:teuthology.orchestra.run.smithi178.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T11:18:54.223 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:54.415 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:54.673 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:54.673 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:54.673 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:18:54.673 INFO:teuthology.orchestra.run.smithi144.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:54.673 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:18:54.673 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:54.897 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install librados2 2024-09-10T11:18:55.270 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:55.270 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:55.270 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:18:55.270 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:55.270 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:55.270 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:55.464 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:32 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:55.544 DEBUG:teuthology.orchestra.run.smithi178:> sudo yum -y install rbd-nbd 2024-09-10T11:18:55.554 INFO:teuthology.orchestra.run.smithi144.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:55.596 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:55.598 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-10T11:18:55.598 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:55.686 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install librbd1 2024-09-10T11:18:56.128 INFO:teuthology.orchestra.run.smithi178.stdout:Last metadata expiration check: 0:01:35 ago on Tue 10 Sep 2024 11:17:21 AM UTC. 2024-09-10T11:18:56.247 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:33 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:56.263 INFO:teuthology.orchestra.run.smithi178.stdout:Dependencies resolved. 2024-09-10T11:18:56.263 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:56.263 INFO:teuthology.orchestra.run.smithi178.stdout: Package Architecture Version Repository Size 2024-09-10T11:18:56.263 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:56.264 INFO:teuthology.orchestra.run.smithi178.stdout:Installing: 2024-09-10T11:18:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-09-10T11:18:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:56.264 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction Summary 2024-09-10T11:18:56.264 INFO:teuthology.orchestra.run.smithi178.stdout:================================================================================ 2024-09-10T11:18:56.264 INFO:teuthology.orchestra.run.smithi178.stdout:Install 1 Package 2024-09-10T11:18:56.264 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:56.265 INFO:teuthology.orchestra.run.smithi178.stdout:Total download size: 169 k 2024-09-10T11:18:56.265 INFO:teuthology.orchestra.run.smithi178.stdout:Installed size: 490 k 2024-09-10T11:18:56.265 INFO:teuthology.orchestra.run.smithi178.stdout:Downloading Packages: 2024-09-10T11:18:56.333 INFO:teuthology.orchestra.run.smithi178.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 2.4 MB/s | 169 kB 00:00 2024-09-10T11:18:56.333 INFO:teuthology.orchestra.run.smithi178.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:18:56.334 INFO:teuthology.orchestra.run.smithi178.stdout:Total 2.4 MB/s | 169 kB 00:00 2024-09-10T11:18:56.334 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction check 2024-09-10T11:18:56.337 INFO:teuthology.orchestra.run.smithi144.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:56.341 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction check succeeded. 2024-09-10T11:18:56.341 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction test 2024-09-10T11:18:56.381 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:56.382 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-10T11:18:56.382 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:56.405 INFO:teuthology.orchestra.run.smithi178.stdout:Transaction test succeeded. 2024-09-10T11:18:56.405 INFO:teuthology.orchestra.run.smithi178.stdout:Running transaction 2024-09-10T11:18:56.460 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-rados 2024-09-10T11:18:56.563 INFO:teuthology.orchestra.run.smithi178.stdout: Preparing : 1/1 2024-09-10T11:18:56.671 INFO:teuthology.orchestra.run.smithi178.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:57.029 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:57.072 INFO:teuthology.orchestra.run.smithi178.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:57.119 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:57.162 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:57.163 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-10T11:18:57.163 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:57.250 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-rgw 2024-09-10T11:18:57.358 INFO:teuthology.orchestra.run.smithi178.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:18:57.358 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:57.358 INFO:teuthology.orchestra.run.smithi178.stdout:Installed: 2024-09-10T11:18:57.359 INFO:teuthology.orchestra.run.smithi178.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-09-10T11:18:57.359 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:18:57.359 INFO:teuthology.orchestra.run.smithi178.stdout:Complete! 2024-09-10T11:18:57.637 DEBUG:teuthology.parallel:result is None 2024-09-10T11:18:57.820 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:57.910 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:57.952 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:57.953 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-10T11:18:57.953 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:58.030 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-cephfs 2024-09-10T11:18:58.593 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:35 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:58.684 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:58.727 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:58.728 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-10T11:18:58.728 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:58.806 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install python3-rbd 2024-09-10T11:18:59.368 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:36 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:18:59.457 INFO:teuthology.orchestra.run.smithi144.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-09-10T11:18:59.500 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:18:59.501 INFO:teuthology.orchestra.run.smithi144.stdout:Nothing to do. 2024-09-10T11:18:59.501 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:18:59.580 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install rbd-fuse 2024-09-10T11:19:00.142 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:37 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:19:00.275 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:19:00.276 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:19:00.276 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-09-10T11:19:00.276 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:19:00.276 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:19:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-09-10T11:19:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:19:00.277 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:19:00.277 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:19:00.277 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-09-10T11:19:00.277 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:19:00.277 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 86 k 2024-09-10T11:19:00.277 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 230 k 2024-09-10T11:19:00.277 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:19:00.326 INFO:teuthology.orchestra.run.smithi144.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 1.7 MB/s | 86 kB 00:00 2024-09-10T11:19:00.327 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:19:00.327 INFO:teuthology.orchestra.run.smithi144.stdout:Total 1.7 MB/s | 86 kB 00:00 2024-09-10T11:19:00.327 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:19:00.334 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:19:00.334 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:19:00.396 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:19:00.396 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:19:00.539 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:19:00.648 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:19:01.062 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:19:01.314 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:19:01.314 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:19:01.314 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:19:01.314 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-09-10T11:19:01.314 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:19:01.315 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:19:01.550 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install rbd-mirror 2024-09-10T11:19:02.134 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:39 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:19:02.271 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:19:02.272 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:19:02.272 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-09-10T11:19:02.272 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:19:02.272 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:19:02.272 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-09-10T11:19:02.272 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:19:02.272 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:19:02.272 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:19:02.273 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-09-10T11:19:02.273 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:19:02.273 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 3.0 M 2024-09-10T11:19:02.273 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 12 M 2024-09-10T11:19:02.273 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:19:02.365 INFO:teuthology.orchestra.run.smithi144.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 33 MB/s | 3.0 MB 00:00 2024-09-10T11:19:02.365 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:19:02.365 INFO:teuthology.orchestra.run.smithi144.stdout:Total 33 MB/s | 3.0 MB 00:00 2024-09-10T11:19:02.366 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:19:02.373 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:19:02.374 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:19:02.453 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:19:02.454 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:19:02.879 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:19:03.048 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:19:03.073 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:19:03.073 INFO:teuthology.orchestra.run.smithi144.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T11:19:03.073 INFO:teuthology.orchestra.run.smithi144.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T11:19:03.073 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T11:19:03.073 INFO:teuthology.orchestra.run.smithi144.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T11:19:03.073 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:19:04.067 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:19:04.067 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:19:04.067 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:19:04.067 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-09-10T11:19:04.067 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:19:04.067 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:19:04.316 DEBUG:teuthology.orchestra.run.smithi144:> sudo yum -y install rbd-nbd 2024-09-10T11:19:04.893 INFO:teuthology.orchestra.run.smithi144.stdout:Last metadata expiration check: 0:01:41 ago on Tue 10 Sep 2024 11:17:23 AM UTC. 2024-09-10T11:19:05.032 INFO:teuthology.orchestra.run.smithi144.stdout:Dependencies resolved. 2024-09-10T11:19:05.033 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:19:05.033 INFO:teuthology.orchestra.run.smithi144.stdout: Package Architecture Version Repository Size 2024-09-10T11:19:05.033 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:19:05.033 INFO:teuthology.orchestra.run.smithi144.stdout:Installing: 2024-09-10T11:19:05.033 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-09-10T11:19:05.033 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:19:05.033 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction Summary 2024-09-10T11:19:05.033 INFO:teuthology.orchestra.run.smithi144.stdout:================================================================================ 2024-09-10T11:19:05.033 INFO:teuthology.orchestra.run.smithi144.stdout:Install 1 Package 2024-09-10T11:19:05.033 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:19:05.033 INFO:teuthology.orchestra.run.smithi144.stdout:Total download size: 169 k 2024-09-10T11:19:05.034 INFO:teuthology.orchestra.run.smithi144.stdout:Installed size: 490 k 2024-09-10T11:19:05.034 INFO:teuthology.orchestra.run.smithi144.stdout:Downloading Packages: 2024-09-10T11:19:05.076 INFO:teuthology.orchestra.run.smithi144.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 3.9 MB/s | 169 kB 00:00 2024-09-10T11:19:05.077 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:19:05.077 INFO:teuthology.orchestra.run.smithi144.stdout:Total 3.8 MB/s | 169 kB 00:00 2024-09-10T11:19:05.077 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction check 2024-09-10T11:19:05.085 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction check succeeded. 2024-09-10T11:19:05.085 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction test 2024-09-10T11:19:05.148 INFO:teuthology.orchestra.run.smithi144.stdout:Transaction test succeeded. 2024-09-10T11:19:05.149 INFO:teuthology.orchestra.run.smithi144.stdout:Running transaction 2024-09-10T11:19:05.299 INFO:teuthology.orchestra.run.smithi144.stdout: Preparing : 1/1 2024-09-10T11:19:05.406 INFO:teuthology.orchestra.run.smithi144.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:19:05.849 INFO:teuthology.orchestra.run.smithi144.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:19:06.095 INFO:teuthology.orchestra.run.smithi144.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-09-10T11:19:06.095 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:19:06.096 INFO:teuthology.orchestra.run.smithi144.stdout:Installed: 2024-09-10T11:19:06.096 INFO:teuthology.orchestra.run.smithi144.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-09-10T11:19:06.096 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:19:06.096 INFO:teuthology.orchestra.run.smithi144.stdout:Complete! 2024-09-10T11:19:06.349 DEBUG:teuthology.parallel:result is None 2024-09-10T11:19:06.350 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-09-10T11:19:06.350 INFO:teuthology.packaging:ref: None 2024-09-10T11:19:06.350 INFO:teuthology.packaging:tag: v18.2.0 2024-09-10T11:19:06.350 INFO:teuthology.packaging:branch: None 2024-09-10T11:19:06.350 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:19:06.351 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-09-10T11:19:06.495 DEBUG:teuthology.orchestra.run.smithi144:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T11:19:06.516 INFO:teuthology.orchestra.run.smithi144.stdout:18.2.0-0.el9 2024-09-10T11:19:06.517 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-09-10T11:19:06.517 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-09-10T11:19:06.519 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-09-10T11:19:06.519 INFO:teuthology.packaging:ref: None 2024-09-10T11:19:06.519 INFO:teuthology.packaging:tag: v18.2.0 2024-09-10T11:19:06.519 INFO:teuthology.packaging:branch: None 2024-09-10T11:19:06.519 INFO:teuthology.packaging:sha1: 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:19:06.519 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-09-10T11:19:06.654 DEBUG:teuthology.orchestra.run.smithi178:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T11:19:06.680 INFO:teuthology.orchestra.run.smithi178.stdout:18.2.0-0.el9 2024-09-10T11:19:06.680 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-09-10T11:19:06.680 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-09-10T11:19:06.682 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T11:19:06.682 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:19:06.682 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T11:19:06.708 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T11:19:06.708 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T11:19:06.750 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T11:19:06.750 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:19:06.750 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T11:19:06.775 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T11:19:06.839 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T11:19:06.839 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T11:19:06.868 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T11:19:06.936 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T11:19:06.936 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:19:06.937 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T11:19:06.961 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T11:19:07.025 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T11:19:07.026 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T11:19:07.052 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T11:19:07.117 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T11:19:07.117 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:19:07.117 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T11:19:07.171 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T11:19:07.235 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T11:19:07.236 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T11:19:07.260 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T11:19:07.326 INFO:teuthology.run_tasks:Running task print... 2024-09-10T11:19:07.337 INFO:teuthology.task.print:**** done install task... 2024-09-10T11:19:07.337 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T11:19:07.437 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}}, 'image': 'quay.io/ceph/ceph:v18.2.0', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy'], 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b'} 2024-09-10T11:19:07.437 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.0 2024-09-10T11:19:07.438 INFO:tasks.cephadm:Cluster fsid is 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:19:07.438 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T11:19:07.438 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-09-10T11:19:07.438 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi144': '172.21.15.144', 'mon.smithi178': '172.21.15.178'} 2024-09-10T11:19:07.438 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T11:19:07.438 DEBUG:teuthology.orchestra.run.smithi144:> sudo hostname $(hostname -s) 2024-09-10T11:19:07.464 DEBUG:teuthology.orchestra.run.smithi178:> sudo hostname $(hostname -s) 2024-09-10T11:19:07.489 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-09-10T11:19:07.490 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:19:07.633 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82914/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.5.35+adami05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:14:38.445020', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1cf819b7212a6048e92fc78c4eff963344f6d56b', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/82912/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-305-g1cf819b7', 'node_name': '172.21.2.7+braggi07', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-305.g1cf819b7'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-09 22:05:03.996701', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-testing-20240909.165402-squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-rishabh-testing-20240909.165402-squid/1cf819b7212a6048e92fc78c4eff963344f6d56b/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-10T11:19:07.862 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref reef, sha1 48a7de7e6e1701a88fd571c7e34908ac4b8ed412 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-09-10T11:19:07.864 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/x86_64/flavors/default/cephadm 2024-09-10T11:19:07.864 INFO:tasks.cephadm:Downloading cephadm from url: https://1.chacra.ceph.com/binaries/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/x86_64/flavors/default/cephadm 2024-09-10T11:19:07.864 DEBUG:teuthology.orchestra.run.smithi144:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T11:19:08.205 INFO:teuthology.orchestra.run.smithi144.stdout:-rw-r--r--. 1 ubuntu ubuntu 217036 Sep 10 11:19 /home/ubuntu/cephtest/cephadm 2024-09-10T11:19:08.205 DEBUG:teuthology.orchestra.run.smithi178:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/reef/48a7de7e6e1701a88fd571c7e34908ac4b8ed412/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T11:19:08.551 INFO:teuthology.orchestra.run.smithi178.stdout:-rw-r--r--. 1 ubuntu ubuntu 217036 Sep 10 11:19 /home/ubuntu/cephtest/cephadm 2024-09-10T11:19:08.552 DEBUG:teuthology.orchestra.run.smithi144:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T11:19:08.568 DEBUG:teuthology.orchestra.run.smithi178:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T11:19:08.595 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.0 on all hosts... 2024-09-10T11:19:08.596 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-09-10T11:19:08.611 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-09-10T11:19:10.194 INFO:teuthology.orchestra.run.smithi178.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-09-10T11:19:10.282 INFO:teuthology.orchestra.run.smithi144.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-09-10T11:19:22.108 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:19:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-09-10T11:19:22.108 INFO:teuthology.orchestra.run.smithi144.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-09-10T11:19:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "repo_digests": [ 2024-09-10T11:19:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-09-10T11:19:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-09-10T11:19:22.109 INFO:teuthology.orchestra.run.smithi144.stdout: ] 2024-09-10T11:19:22.109 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:19:22.488 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-10T11:19:22.488 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-09-10T11:19:22.488 INFO:teuthology.orchestra.run.smithi178.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-09-10T11:19:22.488 INFO:teuthology.orchestra.run.smithi178.stdout: "repo_digests": [ 2024-09-10T11:19:22.488 INFO:teuthology.orchestra.run.smithi178.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-09-10T11:19:22.488 INFO:teuthology.orchestra.run.smithi178.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-09-10T11:19:22.488 INFO:teuthology.orchestra.run.smithi178.stdout: ] 2024-09-10T11:19:22.488 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-10T11:19:22.506 DEBUG:teuthology.orchestra.run.smithi144:> sudo mkdir -p /etc/ceph 2024-09-10T11:19:22.534 DEBUG:teuthology.orchestra.run.smithi178:> sudo mkdir -p /etc/ceph 2024-09-10T11:19:22.562 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 777 /etc/ceph 2024-09-10T11:19:22.601 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 777 /etc/ceph 2024-09-10T11:19:22.627 INFO:tasks.cephadm:Writing seed config... 2024-09-10T11:19:22.628 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-09-10T11:19:22.628 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-09-10T11:19:22.628 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-10T11:19:22.628 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-10T11:19:22.628 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-10T11:19:22.628 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-10T11:19:22.629 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-10T11:19:22.629 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-10T11:19:22.629 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-10T11:19:22.629 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T11:19:22.629 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T11:19:22.629 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-10T11:19:22.629 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-10T11:19:22.629 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-10T11:19:22.629 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-10T11:19:22.629 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-10T11:19:22.629 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-10T11:19:22.629 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-10T11:19:22.629 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-10T11:19:22.629 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-10T11:19:22.630 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-10T11:19:22.630 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-10T11:19:22.630 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-10T11:19:22.630 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-09-10T11:19:22.630 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-10T11:19:22.630 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-10T11:19:22.630 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-10T11:19:22.630 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-10T11:19:22.630 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-10T11:19:22.630 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-10T11:19:22.630 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-10T11:19:22.630 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-10T11:19:22.630 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-10T11:19:22.630 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-10T11:19:22.631 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-10T11:19:22.631 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T11:19:22.631 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T11:19:22.631 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T11:19:22.631 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T11:19:22.631 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T11:19:22.631 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-10T11:19:22.631 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-10T11:19:22.631 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-10T11:19:22.632 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:19:22.632 DEBUG:teuthology.orchestra.run.smithi144:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T11:19:22.656 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 7eacd86c-6f66-11ef-bcea-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-10T11:19:22.657 DEBUG:teuthology.orchestra.run.smithi144:mon.smithi144> sudo journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi144.service 2024-09-10T11:19:22.698 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T11:19:22.699 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 -v bootstrap --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.144 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T11:19:22.851 INFO:teuthology.orchestra.run.smithi144.stdout:-------------------------------------------------------------------------------- 2024-09-10T11:19:22.851 INFO:teuthology.orchestra.run.smithi144.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.0', '-v', 'bootstrap', '--fsid', '7eacd86c-6f66-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.144', '--skip-admin-label'] 2024-09-10T11:19:22.875 INFO:teuthology.orchestra.run.smithi144.stdout:/bin/podman: stdout 5.2.2 2024-09-10T11:19:22.876 INFO:teuthology.orchestra.run.smithi144.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T11:19:22.876 INFO:teuthology.orchestra.run.smithi144.stdout:Verifying podman|docker is present... 2024-09-10T11:19:22.898 INFO:teuthology.orchestra.run.smithi144.stdout:/bin/podman: stdout 5.2.2 2024-09-10T11:19:22.898 INFO:teuthology.orchestra.run.smithi144.stdout:Verifying lvm2 is present... 2024-09-10T11:19:22.898 INFO:teuthology.orchestra.run.smithi144.stdout:Verifying time synchronization is in place... 2024-09-10T11:19:22.905 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T11:19:22.905 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T11:19:22.912 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T11:19:22.912 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stdout inactive 2024-09-10T11:19:22.918 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stdout enabled 2024-09-10T11:19:22.925 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stdout active 2024-09-10T11:19:22.925 INFO:teuthology.orchestra.run.smithi144.stdout:Unit chronyd.service is enabled and running 2024-09-10T11:19:22.925 INFO:teuthology.orchestra.run.smithi144.stdout:Repeating the final host check... 2024-09-10T11:19:22.946 INFO:teuthology.orchestra.run.smithi144.stdout:/bin/podman: stdout 5.2.2 2024-09-10T11:19:22.946 INFO:teuthology.orchestra.run.smithi144.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T11:19:22.946 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl is present 2024-09-10T11:19:22.946 INFO:teuthology.orchestra.run.smithi144.stdout:lvcreate is present 2024-09-10T11:19:22.952 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T11:19:22.953 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T11:19:22.959 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T11:19:22.959 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stdout inactive 2024-09-10T11:19:22.965 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stdout enabled 2024-09-10T11:19:22.973 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stdout active 2024-09-10T11:19:22.973 INFO:teuthology.orchestra.run.smithi144.stdout:Unit chronyd.service is enabled and running 2024-09-10T11:19:22.973 INFO:teuthology.orchestra.run.smithi144.stdout:Host looks OK 2024-09-10T11:19:22.973 INFO:teuthology.orchestra.run.smithi144.stdout:Cluster fsid: 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:19:22.973 INFO:teuthology.orchestra.run.smithi144.stdout:Acquiring lock 139622313485648 on /run/cephadm/7eacd86c-6f66-11ef-bcea-c7b262605968.lock 2024-09-10T11:19:22.973 INFO:teuthology.orchestra.run.smithi144.stdout:Lock 139622313485648 acquired on /run/cephadm/7eacd86c-6f66-11ef-bcea-c7b262605968.lock 2024-09-10T11:19:22.973 INFO:teuthology.orchestra.run.smithi144.stdout:Verifying IP 172.21.15.144 port 3300 ... 2024-09-10T11:19:22.973 INFO:teuthology.orchestra.run.smithi144.stdout:Verifying IP 172.21.15.144 port 6789 ... 2024-09-10T11:19:22.974 INFO:teuthology.orchestra.run.smithi144.stdout:Base mon IP(s) is [172.21.15.144:3300, 172.21.15.144:6789], mon addrv is [v2:172.21.15.144:3300,v1:172.21.15.144:6789] 2024-09-10T11:19:22.976 INFO:teuthology.orchestra.run.smithi144.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.144 metric 100 2024-09-10T11:19:22.976 INFO:teuthology.orchestra.run.smithi144.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.144 metric 100 2024-09-10T11:19:22.979 INFO:teuthology.orchestra.run.smithi144.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T11:19:22.979 INFO:teuthology.orchestra.run.smithi144.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-10T11:19:22.979 INFO:teuthology.orchestra.run.smithi144.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-10T11:19:22.981 INFO:teuthology.orchestra.run.smithi144.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T11:19:22.981 INFO:teuthology.orchestra.run.smithi144.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T11:19:22.981 INFO:teuthology.orchestra.run.smithi144.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T11:19:22.981 INFO:teuthology.orchestra.run.smithi144.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T11:19:22.981 INFO:teuthology.orchestra.run.smithi144.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link noprefixroute 2024-09-10T11:19:22.981 INFO:teuthology.orchestra.run.smithi144.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T11:19:22.982 INFO:teuthology.orchestra.run.smithi144.stdout:Mon IP `172.21.15.144` is in CIDR network `172.21.0.0/20` 2024-09-10T11:19:22.982 INFO:teuthology.orchestra.run.smithi144.stdout:Mon IP `172.21.15.144` is in CIDR network `172.21.0.0/20` 2024-09-10T11:19:22.982 INFO:teuthology.orchestra.run.smithi144.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T11:19:22.982 INFO:teuthology.orchestra.run.smithi144.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T11:19:22.983 INFO:teuthology.orchestra.run.smithi144.stdout:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-09-10T11:19:23.820 INFO:teuthology.orchestra.run.smithi144.stdout:/bin/podman: stdout dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-09-10T11:19:23.821 INFO:teuthology.orchestra.run.smithi144.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.0... 2024-09-10T11:19:23.821 INFO:teuthology.orchestra.run.smithi144.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T11:19:23.821 INFO:teuthology.orchestra.run.smithi144.stdout:/bin/podman: stderr Copying blob sha256:3bd20aeff60302f668275dc2005d10679ae56492967a3a5a54fd3dde85333aec 2024-09-10T11:19:23.821 INFO:teuthology.orchestra.run.smithi144.stdout:/bin/podman: stderr Copying blob sha256:46af8f5390d4e94fc57efb422ccb97bb53dfe5b948546bfc191b46557eb2dbd9 2024-09-10T11:19:23.821 INFO:teuthology.orchestra.run.smithi144.stdout:/bin/podman: stderr Copying config sha256:dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-09-10T11:19:23.821 INFO:teuthology.orchestra.run.smithi144.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T11:19:24.724 INFO:teuthology.orchestra.run.smithi144.stdout:ceph: stdout ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-09-10T11:19:24.724 INFO:teuthology.orchestra.run.smithi144.stdout:Ceph version: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-09-10T11:19:24.724 INFO:teuthology.orchestra.run.smithi144.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T11:19:25.754 INFO:teuthology.orchestra.run.smithi144.stdout:stat: stdout 167 167 2024-09-10T11:19:25.755 INFO:teuthology.orchestra.run.smithi144.stdout:Creating initial keys... 2024-09-10T11:19:26.636 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph-authtool: stdout AQA+K+BmI3HPAxAAy1YrunA3NxvMcbepXgAm+w== 2024-09-10T11:19:27.684 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph-authtool: stdout AQA+K+Bmr2S/ORAADMqHrQavhta83xBtWKJlBw== 2024-09-10T11:19:28.598 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph-authtool: stdout AQA/K+BmA4anORAAlkDdZmUxgx+Hr57EbeloWg== 2024-09-10T11:19:28.599 INFO:teuthology.orchestra.run.smithi144.stdout:Creating initial monmap... 2024-09-10T11:19:29.520 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T11:19:29.521 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-09-10T11:19:29.521 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:19:29.521 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T11:19:29.521 INFO:teuthology.orchestra.run.smithi144.stdout:monmaptool for smithi144 [v2:172.21.15.144:3300,v1:172.21.15.144:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T11:19:29.521 INFO:teuthology.orchestra.run.smithi144.stdout:setting min_mon_release = pacific 2024-09-10T11:19:29.521 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/monmaptool: set fsid to 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:19:29.521 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T11:19:29.521 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:19:29.521 INFO:teuthology.orchestra.run.smithi144.stdout:Creating mon... 2024-09-10T11:19:30.409 INFO:teuthology.orchestra.run.smithi144.stdout:create mon.smithi144 on 2024-09-10T11:19:30.612 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T11:19:30.760 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T11:19:30.960 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-7eacd86c-6f66-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-7eacd86c-6f66-11ef-bcea-c7b262605968.target. 2024-09-10T11:19:30.967 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-7eacd86c-6f66-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-7eacd86c-6f66-11ef-bcea-c7b262605968.target. 2024-09-10T11:19:31.357 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi144 2024-09-10T11:19:31.357 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi144.service: Unit ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi144.service not loaded. 2024-09-10T11:19:31.522 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7eacd86c-6f66-11ef-bcea-c7b262605968.target.wants/ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi144.service → /etc/systemd/system/ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@.service. 2024-09-10T11:19:32.076 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T11:19:32.076 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stdout disabled 2024-09-10T11:19:32.082 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T11:19:32.082 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stdout inactive 2024-09-10T11:19:32.083 INFO:teuthology.orchestra.run.smithi144.stdout:firewalld.service is not enabled 2024-09-10T11:19:32.083 INFO:teuthology.orchestra.run.smithi144.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T11:19:32.083 INFO:teuthology.orchestra.run.smithi144.stdout:Waiting for mon to start... 2024-09-10T11:19:32.083 INFO:teuthology.orchestra.run.smithi144.stdout:Waiting for mon... 2024-09-10T11:19:32.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:32 smithi144 systemd[1]: Started Ceph mon.smithi144 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:19:33.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:33 smithi144 ceph-mon[27558]: mon.smithi144 is new leader, mons smithi144 in quorum (ranks 0) 2024-09-10T11:19:33.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:33 smithi144 ceph-mon[27558]: monmap e1: 1 mons at {smithi144=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0]} removed_ranks: {} 2024-09-10T11:19:33.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:33 smithi144 ceph-mon[27558]: fsmap 2024-09-10T11:19:33.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:33 smithi144 ceph-mon[27558]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T11:19:33.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:33 smithi144 ceph-mon[27558]: mgrmap e1: no daemons active 2024-09-10T11:19:33.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:33 smithi144 ceph-mon[27558]: from='client.? 172.21.15.144:0/3205116116' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T11:19:33.220 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T11:19:33.226 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout id: 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:19:33.226 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T11:19:33.226 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 2024-09-10T11:19:33.227 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout services: 2024-09-10T11:19:33.227 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi144 (age 0.564784s) 2024-09-10T11:19:33.227 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T11:19:33.227 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T11:19:33.227 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 2024-09-10T11:19:33.227 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout data: 2024-09-10T11:19:33.227 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T11:19:33.227 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T11:19:33.227 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T11:19:33.227 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T11:19:33.227 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 2024-09-10T11:19:33.227 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.506+0000 7f7b354d6700 1 Processor -- start 2024-09-10T11:19:33.228 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.506+0000 7f7b354d6700 1 -- start start 2024-09-10T11:19:33.228 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.506+0000 7f7b354d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b30105b90 0x7f7b30105fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:19:33.228 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.506+0000 7f7b354d6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b301064e0 con 0x7f7b30105b90 2024-09-10T11:19:33.228 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.506+0000 7f7b2effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b30105b90 0x7f7b30105fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:19:33.228 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.506+0000 7f7b2effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b30105b90 0x7f7b30105fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45690/0 (socket says 172.21.15.144:45690) 2024-09-10T11:19:33.228 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.506+0000 7f7b2effd700 1 -- 172.21.15.144:0/3998680094 learned_addr learned my addr 172.21.15.144:0/3998680094 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:19:33.228 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.507+0000 7f7b2effd700 1 -- 172.21.15.144:0/3998680094 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b30106620 con 0x7f7b30105b90 2024-09-10T11:19:33.228 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.507+0000 7f7b2effd700 1 --2- 172.21.15.144:0/3998680094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b30105b90 0x7f7b30105fa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b200098a0 tx=0x7f7b20009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e538ddf25c06bb36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:19:33.228 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.507+0000 7f7b2dffb700 1 -- 172.21.15.144:0/3998680094 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7b20004180 con 0x7f7b30105b90 2024-09-10T11:19:33.228 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.507+0000 7f7b2dffb700 1 -- 172.21.15.144:0/3998680094 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b200042e0 con 0x7f7b30105b90 2024-09-10T11:19:33.228 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.507+0000 7f7b2dffb700 1 -- 172.21.15.144:0/3998680094 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7b200036a0 con 0x7f7b30105b90 2024-09-10T11:19:33.228 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.507+0000 7f7b354d6700 1 -- 172.21.15.144:0/3998680094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b30105b90 msgr2=0x7f7b30105fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:19:33.228 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.507+0000 7f7b354d6700 1 --2- 172.21.15.144:0/3998680094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b30105b90 0x7f7b30105fa0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f7b200098a0 tx=0x7f7b20009bb0 comp rx=0 tx=0).stop 2024-09-10T11:19:33.229 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.507+0000 7f7b354d6700 1 -- 172.21.15.144:0/3998680094 shutdown_connections 2024-09-10T11:19:33.229 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.507+0000 7f7b354d6700 1 --2- 172.21.15.144:0/3998680094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b30105b90 0x7f7b30105fa0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:19:33.229 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.507+0000 7f7b354d6700 1 -- 172.21.15.144:0/3998680094 >> 172.21.15.144:0/3998680094 conn(0x7f7b30101220 msgr2=0x7f7b30103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:19:33.229 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.508+0000 7f7b354d6700 1 -- 172.21.15.144:0/3998680094 shutdown_connections 2024-09-10T11:19:33.229 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.508+0000 7f7b354d6700 1 -- 172.21.15.144:0/3998680094 wait complete. 2024-09-10T11:19:33.229 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.508+0000 7f7b354d6700 1 Processor -- start 2024-09-10T11:19:33.229 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.508+0000 7f7b354d6700 1 -- start start 2024-09-10T11:19:33.229 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.509+0000 7f7b354d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b30105b90 0x7f7b3007bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:19:33.229 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.509+0000 7f7b354d6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3007c1b0 con 0x7f7b30105b90 2024-09-10T11:19:33.229 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.509+0000 7f7b2effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b30105b90 0x7f7b3007bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:19:33.229 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.509+0000 7f7b2effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b30105b90 0x7f7b3007bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45698/0 (socket says 172.21.15.144:45698) 2024-09-10T11:19:33.229 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.509+0000 7f7b2effd700 1 -- 172.21.15.144:0/3205116116 learned_addr learned my addr 172.21.15.144:0/3205116116 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:19:33.229 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.509+0000 7f7b2effd700 1 -- 172.21.15.144:0/3205116116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b20009550 con 0x7f7b30105b90 2024-09-10T11:19:33.230 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.509+0000 7f7b2effd700 1 --2- 172.21.15.144:0/3205116116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b30105b90 0x7f7b3007bc70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b301068c0 tx=0x7f7b20004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:19:33.230 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.509+0000 7f7b17fff700 1 -- 172.21.15.144:0/3205116116 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7b20003ce0 con 0x7f7b30105b90 2024-09-10T11:19:33.230 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.509+0000 7f7b17fff700 1 -- 172.21.15.144:0/3205116116 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b20020070 con 0x7f7b30105b90 2024-09-10T11:19:33.230 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.509+0000 7f7b354d6700 1 -- 172.21.15.144:0/3205116116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3007a2c0 con 0x7f7b30105b90 2024-09-10T11:19:33.230 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.509+0000 7f7b17fff700 1 -- 172.21.15.144:0/3205116116 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7b20011630 con 0x7f7b30105b90 2024-09-10T11:19:33.230 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.509+0000 7f7b354d6700 1 -- 172.21.15.144:0/3205116116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3007a6e0 con 0x7f7b30105b90 2024-09-10T11:19:33.230 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.510+0000 7f7b17fff700 1 -- 172.21.15.144:0/3205116116 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7b20022030 con 0x7f7b30105b90 2024-09-10T11:19:33.230 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.510+0000 7f7b17fff700 1 -- 172.21.15.144:0/3205116116 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7b20025d00 con 0x7f7b30105b90 2024-09-10T11:19:33.230 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.510+0000 7f7b354d6700 1 -- 172.21.15.144:0/3205116116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b3007b0f0 con 0x7f7b30105b90 2024-09-10T11:19:33.230 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.513+0000 7f7b17fff700 1 -- 172.21.15.144:0/3205116116 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f7b20025480 con 0x7f7b30105b90 2024-09-10T11:19:33.230 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.583+0000 7f7b354d6700 1 -- 172.21.15.144:0/3205116116 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f7b30192410 con 0x7f7b30105b90 2024-09-10T11:19:33.230 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.584+0000 7f7b17fff700 1 -- 172.21.15.144:0/3205116116 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f7b20011c20 con 0x7f7b30105b90 2024-09-10T11:19:33.230 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.584+0000 7f7b354d6700 1 -- 172.21.15.144:0/3205116116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b30105b90 msgr2=0x7f7b3007bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:19:33.230 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.584+0000 7f7b354d6700 1 --2- 172.21.15.144:0/3205116116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b30105b90 0x7f7b3007bc70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f7b301068c0 tx=0x7f7b20004750 comp rx=0 tx=0).stop 2024-09-10T11:19:33.231 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.584+0000 7f7b354d6700 1 -- 172.21.15.144:0/3205116116 shutdown_connections 2024-09-10T11:19:33.231 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.584+0000 7f7b354d6700 1 --2- 172.21.15.144:0/3205116116 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b30105b90 0x7f7b3007bc70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:19:33.231 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.584+0000 7f7b354d6700 1 -- 172.21.15.144:0/3205116116 >> 172.21.15.144:0/3205116116 conn(0x7f7b30101220 msgr2=0x7f7b3018d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:19:33.231 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.585+0000 7f7b354d6700 1 -- 172.21.15.144:0/3205116116 shutdown_connections 2024-09-10T11:19:33.231 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:32.585+0000 7f7b354d6700 1 -- 172.21.15.144:0/3205116116 wait complete. 2024-09-10T11:19:33.231 INFO:teuthology.orchestra.run.smithi144.stdout:mon is available 2024-09-10T11:19:33.231 INFO:teuthology.orchestra.run.smithi144.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T11:19:34.274 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 2024-09-10T11:19:34.274 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T11:19:34.274 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout fsid = 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:19:34.275 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.144:3300,v1:172.21.15.144:6789] 2024-09-10T11:19:34.275 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T11:19:34.275 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T11:19:34.275 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T11:19:34.275 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T11:19:34.275 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 2024-09-10T11:19:34.275 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T11:19:34.275 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T11:19:34.275 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 2024-09-10T11:19:34.275 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T11:19:34.275 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T11:19:34.275 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T11:19:34.275 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T11:19:34.275 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T11:19:34.275 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T11:19:34.276 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.644+0000 7fa162c2a700 1 Processor -- start 2024-09-10T11:19:34.276 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.644+0000 7fa162c2a700 1 -- start start 2024-09-10T11:19:34.276 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.644+0000 7fa162c2a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa15c104fb0 0x7fa15c1053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:19:34.276 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.645+0000 7fa162c2a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c105900 con 0x7fa15c104fb0 2024-09-10T11:19:34.276 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.645+0000 7fa1609c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa15c104fb0 0x7fa15c1053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:19:34.276 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.645+0000 7fa1609c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa15c104fb0 0x7fa15c1053c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45704/0 (socket says 172.21.15.144:45704) 2024-09-10T11:19:34.276 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.645+0000 7fa1609c6700 1 -- 172.21.15.144:0/2936814029 learned_addr learned my addr 172.21.15.144:0/2936814029 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:19:34.276 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.645+0000 7fa1609c6700 1 -- 172.21.15.144:0/2936814029 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c105a40 con 0x7fa15c104fb0 2024-09-10T11:19:34.276 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.645+0000 7fa1609c6700 1 --2- 172.21.15.144:0/2936814029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa15c104fb0 0x7fa15c1053c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa154009e80 tx=0x7fa15400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1876c8355ac99ab6 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:19:34.276 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.645+0000 7fa15b7fe700 1 -- 172.21.15.144:0/2936814029 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa15400e070 con 0x7fa15c104fb0 2024-09-10T11:19:34.276 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.645+0000 7fa15b7fe700 1 -- 172.21.15.144:0/2936814029 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa154003a20 con 0x7fa15c104fb0 2024-09-10T11:19:34.276 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.645+0000 7fa15b7fe700 1 -- 172.21.15.144:0/2936814029 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa154003bd0 con 0x7fa15c104fb0 2024-09-10T11:19:34.276 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.646+0000 7fa162c2a700 1 -- 172.21.15.144:0/2936814029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa15c104fb0 msgr2=0x7fa15c1053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:19:34.276 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.646+0000 7fa162c2a700 1 --2- 172.21.15.144:0/2936814029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa15c104fb0 0x7fa15c1053c0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa154009e80 tx=0x7fa15400b330 comp rx=0 tx=0).stop 2024-09-10T11:19:34.277 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.646+0000 7fa162c2a700 1 -- 172.21.15.144:0/2936814029 shutdown_connections 2024-09-10T11:19:34.277 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.646+0000 7fa162c2a700 1 --2- 172.21.15.144:0/2936814029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa15c104fb0 0x7fa15c1053c0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:19:34.277 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.646+0000 7fa162c2a700 1 -- 172.21.15.144:0/2936814029 >> 172.21.15.144:0/2936814029 conn(0x7fa15c0ff880 msgr2=0x7fa15c101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:19:34.277 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.646+0000 7fa162c2a700 1 -- 172.21.15.144:0/2936814029 shutdown_connections 2024-09-10T11:19:34.277 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.646+0000 7fa162c2a700 1 -- 172.21.15.144:0/2936814029 wait complete. 2024-09-10T11:19:34.277 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.647+0000 7fa162c2a700 1 Processor -- start 2024-09-10T11:19:34.277 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.647+0000 7fa162c2a700 1 -- start start 2024-09-10T11:19:34.277 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.647+0000 7fa162c2a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa15c104fb0 0x7fa15c104730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:19:34.277 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.647+0000 7fa162c2a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c19be70 con 0x7fa15c104fb0 2024-09-10T11:19:34.277 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.647+0000 7fa1609c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa15c104fb0 0x7fa15c104730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:19:34.277 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.647+0000 7fa1609c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa15c104fb0 0x7fa15c104730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45710/0 (socket says 172.21.15.144:45710) 2024-09-10T11:19:34.277 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.647+0000 7fa1609c6700 1 -- 172.21.15.144:0/2883260279 learned_addr learned my addr 172.21.15.144:0/2883260279 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:19:34.277 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.647+0000 7fa1609c6700 1 -- 172.21.15.144:0/2883260279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa154009b30 con 0x7fa15c104fb0 2024-09-10T11:19:34.278 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.647+0000 7fa1609c6700 1 --2- 172.21.15.144:0/2883260279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa15c104fb0 0x7fa15c104730 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa154004000 tx=0x7fa154004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:19:34.278 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.647+0000 7fa159ffb700 1 -- 172.21.15.144:0/2883260279 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa154004240 con 0x7fa15c104fb0 2024-09-10T11:19:34.278 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.647+0000 7fa159ffb700 1 -- 172.21.15.144:0/2883260279 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1540043e0 con 0x7fa15c104fb0 2024-09-10T11:19:34.278 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.647+0000 7fa162c2a700 1 -- 172.21.15.144:0/2883260279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15c19bfb0 con 0x7fa15c104fb0 2024-09-10T11:19:34.278 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.647+0000 7fa159ffb700 1 -- 172.21.15.144:0/2883260279 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1540046e0 con 0x7fa15c104fb0 2024-09-10T11:19:34.278 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.648+0000 7fa162c2a700 1 -- 172.21.15.144:0/2883260279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa15c19c3d0 con 0x7fa15c104fb0 2024-09-10T11:19:34.278 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.648+0000 7fa159ffb700 1 -- 172.21.15.144:0/2883260279 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa154019050 con 0x7fa15c104fb0 2024-09-10T11:19:34.278 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.648+0000 7fa159ffb700 1 -- 172.21.15.144:0/2883260279 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa154025790 con 0x7fa15c104fb0 2024-09-10T11:19:34.278 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.649+0000 7fa162c2a700 1 -- 172.21.15.144:0/2883260279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15c191880 con 0x7fa15c104fb0 2024-09-10T11:19:34.278 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.651+0000 7fa159ffb700 1 -- 172.21.15.144:0/2883260279 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fa15402a060 con 0x7fa15c104fb0 2024-09-10T11:19:34.278 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.719+0000 7fa162c2a700 1 -- 172.21.15.144:0/2883260279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fa15c02ceb0 con 0x7fa15c104fb0 2024-09-10T11:19:34.278 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.722+0000 7fa159ffb700 1 -- 172.21.15.144:0/2883260279 <== mon.0 v2:172.21.15.144:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fa154012d30 con 0x7fa15c104fb0 2024-09-10T11:19:34.278 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.723+0000 7fa159ffb700 1 -- 172.21.15.144:0/2883260279 <== mon.0 v2:172.21.15.144:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+431 (secure 0 0 0) 0x7fa154020030 con 0x7fa15c104fb0 2024-09-10T11:19:34.279 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.724+0000 7fa162c2a700 1 -- 172.21.15.144:0/2883260279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa15c104fb0 msgr2=0x7fa15c104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:19:34.279 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.724+0000 7fa162c2a700 1 --2- 172.21.15.144:0/2883260279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa15c104fb0 0x7fa15c104730 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa154004000 tx=0x7fa154004ad0 comp rx=0 tx=0).stop 2024-09-10T11:19:34.279 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.724+0000 7fa162c2a700 1 -- 172.21.15.144:0/2883260279 shutdown_connections 2024-09-10T11:19:34.279 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.724+0000 7fa162c2a700 1 --2- 172.21.15.144:0/2883260279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa15c104fb0 0x7fa15c104730 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:19:34.279 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.724+0000 7fa162c2a700 1 -- 172.21.15.144:0/2883260279 >> 172.21.15.144:0/2883260279 conn(0x7fa15c0ff880 msgr2=0x7fa15c18c530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:19:34.279 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.724+0000 7fa162c2a700 1 -- 172.21.15.144:0/2883260279 shutdown_connections 2024-09-10T11:19:34.279 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:33.724+0000 7fa162c2a700 1 -- 172.21.15.144:0/2883260279 wait complete. 2024-09-10T11:19:34.279 INFO:teuthology.orchestra.run.smithi144.stdout:Generating new minimal ceph.conf... 2024-09-10T11:19:34.383 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:34 smithi144 ceph-mon[27558]: from='client.? 172.21.15.144:0/2883260279' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T11:19:34.384 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:34 smithi144 ceph-mon[27558]: from='client.? 172.21.15.144:0/2883260279' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T11:19:35.372 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:35 smithi144 ceph-mon[27558]: from='client.? 172.21.15.144:0/3371008581' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:19:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.715+0000 7ff47900b700 1 Processor -- start 2024-09-10T11:19:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.715+0000 7ff47900b700 1 -- start start 2024-09-10T11:19:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.715+0000 7ff47900b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff474105c40 0x7ff474106050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:19:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.715+0000 7ff47900b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff474106590 con 0x7ff474105c40 2024-09-10T11:19:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.715+0000 7ff472d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff474105c40 0x7ff474106050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:19:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.716+0000 7ff472d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff474105c40 0x7ff474106050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45718/0 (socket says 172.21.15.144:45718) 2024-09-10T11:19:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.716+0000 7ff472d9d700 1 -- 172.21.15.144:0/2973759456 learned_addr learned my addr 172.21.15.144:0/2973759456 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:19:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.716+0000 7ff472d9d700 1 -- 172.21.15.144:0/2973759456 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4741066d0 con 0x7ff474105c40 2024-09-10T11:19:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.716+0000 7ff472d9d700 1 --2- 172.21.15.144:0/2973759456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff474105c40 0x7ff474106050 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff4640098a0 tx=0x7ff464009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360a0df4440ce0e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:19:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.716+0000 7ff471d9b700 1 -- 172.21.15.144:0/2973759456 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff464004180 con 0x7ff474105c40 2024-09-10T11:19:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.716+0000 7ff471d9b700 1 -- 172.21.15.144:0/2973759456 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff4640042e0 con 0x7ff474105c40 2024-09-10T11:19:35.379 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.716+0000 7ff471d9b700 1 -- 172.21.15.144:0/2973759456 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff464011a50 con 0x7ff474105c40 2024-09-10T11:19:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.717+0000 7ff47900b700 1 -- 172.21.15.144:0/2973759456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff474105c40 msgr2=0x7ff474106050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:19:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.717+0000 7ff47900b700 1 --2- 172.21.15.144:0/2973759456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff474105c40 0x7ff474106050 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff4640098a0 tx=0x7ff464009bb0 comp rx=0 tx=0).stop 2024-09-10T11:19:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.717+0000 7ff47900b700 1 -- 172.21.15.144:0/2973759456 shutdown_connections 2024-09-10T11:19:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.717+0000 7ff47900b700 1 --2- 172.21.15.144:0/2973759456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff474105c40 0x7ff474106050 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:19:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.717+0000 7ff47900b700 1 -- 172.21.15.144:0/2973759456 >> 172.21.15.144:0/2973759456 conn(0x7ff4741012b0 msgr2=0x7ff4741036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:19:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.717+0000 7ff47900b700 1 -- 172.21.15.144:0/2973759456 shutdown_connections 2024-09-10T11:19:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.717+0000 7ff47900b700 1 -- 172.21.15.144:0/2973759456 wait complete. 2024-09-10T11:19:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.717+0000 7ff47900b700 1 Processor -- start 2024-09-10T11:19:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.718+0000 7ff47900b700 1 -- start start 2024-09-10T11:19:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.718+0000 7ff47900b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff474105c40 0x7ff474199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:19:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.718+0000 7ff47900b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff474199e60 con 0x7ff474105c40 2024-09-10T11:19:35.380 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.718+0000 7ff472d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff474105c40 0x7ff474199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:19:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.718+0000 7ff472d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff474105c40 0x7ff474199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45730/0 (socket says 172.21.15.144:45730) 2024-09-10T11:19:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.718+0000 7ff472d9d700 1 -- 172.21.15.144:0/3371008581 learned_addr learned my addr 172.21.15.144:0/3371008581 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:19:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.718+0000 7ff472d9d700 1 -- 172.21.15.144:0/3371008581 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff464009550 con 0x7ff474105c40 2024-09-10T11:19:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.718+0000 7ff472d9d700 1 --2- 172.21.15.144:0/3371008581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff474105c40 0x7ff474199920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff464003f80 tx=0x7ff464011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:19:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.718+0000 7ff45bfff700 1 -- 172.21.15.144:0/3371008581 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff464003a80 con 0x7ff474105c40 2024-09-10T11:19:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.719+0000 7ff45bfff700 1 -- 172.21.15.144:0/3371008581 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff464003be0 con 0x7ff474105c40 2024-09-10T11:19:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.719+0000 7ff45bfff700 1 -- 172.21.15.144:0/3371008581 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff464019820 con 0x7ff474105c40 2024-09-10T11:19:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.719+0000 7ff47900b700 1 -- 172.21.15.144:0/3371008581 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff47419a060 con 0x7ff474105c40 2024-09-10T11:19:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.719+0000 7ff47900b700 1 -- 172.21.15.144:0/3371008581 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff47419a480 con 0x7ff474105c40 2024-09-10T11:19:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.719+0000 7ff45bfff700 1 -- 172.21.15.144:0/3371008581 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff464019d50 con 0x7ff474105c40 2024-09-10T11:19:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.719+0000 7ff45bfff700 1 -- 172.21.15.144:0/3371008581 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff4640247c0 con 0x7ff474105c40 2024-09-10T11:19:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.720+0000 7ff47900b700 1 -- 172.21.15.144:0/3371008581 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff474062380 con 0x7ff474105c40 2024-09-10T11:19:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.722+0000 7ff45bfff700 1 -- 172.21.15.144:0/3371008581 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7ff46401acc0 con 0x7ff474105c40 2024-09-10T11:19:35.381 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.789+0000 7ff47900b700 1 -- 172.21.15.144:0/3371008581 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7ff474107ab0 con 0x7ff474105c40 2024-09-10T11:19:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.789+0000 7ff45bfff700 1 -- 172.21.15.144:0/3371008581 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7ff464035090 con 0x7ff474105c40 2024-09-10T11:19:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.790+0000 7ff47900b700 1 -- 172.21.15.144:0/3371008581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff474105c40 msgr2=0x7ff474199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:19:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.790+0000 7ff47900b700 1 --2- 172.21.15.144:0/3371008581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff474105c40 0x7ff474199920 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff464003f80 tx=0x7ff464011ab0 comp rx=0 tx=0).stop 2024-09-10T11:19:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.790+0000 7ff47900b700 1 -- 172.21.15.144:0/3371008581 shutdown_connections 2024-09-10T11:19:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.790+0000 7ff47900b700 1 --2- 172.21.15.144:0/3371008581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff474105c40 0x7ff474199920 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:19:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.790+0000 7ff47900b700 1 -- 172.21.15.144:0/3371008581 >> 172.21.15.144:0/3371008581 conn(0x7ff4741012b0 msgr2=0x7ff4741029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:19:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.790+0000 7ff47900b700 1 -- 172.21.15.144:0/3371008581 shutdown_connections 2024-09-10T11:19:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:34.790+0000 7ff47900b700 1 -- 172.21.15.144:0/3371008581 wait complete. 2024-09-10T11:19:35.382 INFO:teuthology.orchestra.run.smithi144.stdout:Restarting the monitor... 2024-09-10T11:19:35.651 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:35 smithi144 systemd[1]: Stopping Ceph mon.smithi144 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:19:35.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:35 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[27554]: 2024-09-10T11:19:35.648+0000 7f8bfb557700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi144 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:19:35.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:35 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[27554]: 2024-09-10T11:19:35.648+0000 7f8bfb557700 -1 mon.smithi144@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T11:19:35.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:35 smithi144 podman[27804]: 2024-09-10 11:19:35.803276863 +0000 UTC m=+0.263674172 container died 096e159fe76084f6796e458469aa09ee160c3894f1cd14dbcc26dba8568df798 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144, ceph=True, RELEASE=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_BRANCH=HEAD) 2024-09-10T11:19:37.335 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:37 smithi144 podman[27804]: 2024-09-10 11:19:37.004025484 +0000 UTC m=+1.464422786 container cleanup 096e159fe76084f6796e458469aa09ee160c3894f1cd14dbcc26dba8568df798 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, org.label-schema.build-date=20231212, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-09-10T11:19:37.335 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:37 smithi144 bash[27804]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144 2024-09-10T11:19:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:38 smithi144 podman[27819]: 2024-09-10 11:19:38.899054635 +0000 UTC m=+3.095310241 container remove 096e159fe76084f6796e458469aa09ee160c3894f1cd14dbcc26dba8568df798 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-09-10T11:19:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:39 smithi144 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi144.service: Deactivated successfully. 2024-09-10T11:19:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:39 smithi144 systemd[1]: Stopped Ceph mon.smithi144 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:19:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:39 smithi144 systemd[1]: Starting Ceph mon.smithi144 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:19:39.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:39 smithi144 podman[27923]: 2024-09-10 11:19:39.16029237 +0000 UTC m=+0.015541981 image pull dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 quay.io/ceph/ceph:v18.2.0 2024-09-10T11:19:40.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:40 smithi144 podman[27923]: 2024-09-10 11:19:40.300998881 +0000 UTC m=+1.156248483 container create 42addb654e4f8b796385356b778525a269a7993950cca0146f524e6c0328c912 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T11:19:42.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 podman[27923]: 2024-09-10 11:19:42.204247557 +0000 UTC m=+3.059497168 container init 42addb654e4f8b796385356b778525a269a7993950cca0146f524e6c0328c912 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20231212, ceph=True, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-09-10T11:19:42.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 podman[27923]: 2024-09-10 11:19:42.208426162 +0000 UTC m=+3.063675765 container start 42addb654e4f8b796385356b778525a269a7993950cca0146f524e6c0328c912 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS) 2024-09-10T11:19:42.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T11:19:42.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable), process ceph-mon, pid 2 2024-09-10T11:19:42.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: pidfile_write: ignore empty --pid-file 2024-09-10T11:19:42.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: load: jerasure load: lrc 2024-09-10T11:19:42.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: RocksDB version: 7.9.2 2024-09-10T11:19:42.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Git sha 0 2024-09-10T11:19:42.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Compile date 2023-08-03 19:21:13 2024-09-10T11:19:42.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: DB SUMMARY 2024-09-10T11:19:42.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: DB Session ID: 604HCIN8S740KHCKAMES 2024-09-10T11:19:42.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: CURRENT file: CURRENT 2024-09-10T11:19:42.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T11:19:42.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-10T11:19:42.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi144/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T11:19:42.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi144/store.db: 000009.log size: 98799 ; 2024-09-10T11:19:42.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.error_if_exists: 0 2024-09-10T11:19:42.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.create_if_missing: 0 2024-09-10T11:19:42.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.paranoid_checks: 1 2024-09-10T11:19:42.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T11:19:42.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T11:19:42.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T11:19:42.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.env: 0x565416990720 2024-09-10T11:19:42.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T11:19:42.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.info_log: 0x5654193132c0 2024-09-10T11:19:42.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T11:19:42.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.statistics: (nil) 2024-09-10T11:19:42.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.use_fsync: 0 2024-09-10T11:19:42.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_log_file_size: 0 2024-09-10T11:19:42.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T11:19:42.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T11:19:42.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T11:19:42.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T11:19:42.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.allow_fallocate: 1 2024-09-10T11:19:42.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T11:19:42.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T11:19:42.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.use_direct_reads: 0 2024-09-10T11:19:42.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T11:19:42.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T11:19:42.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.db_log_dir: 2024-09-10T11:19:42.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.wal_dir: 2024-09-10T11:19:42.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T11:19:42.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T11:19:42.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T11:19:42.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T11:19:42.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T11:19:42.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T11:19:42.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T11:19:42.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T11:19:42.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.write_buffer_manager: 0x5654185a25a0 2024-09-10T11:19:42.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T11:19:42.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T11:19:42.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T11:19:42.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T11:19:42.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T11:19:42.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T11:19:42.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T11:19:42.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T11:19:42.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.unordered_write: 0 2024-09-10T11:19:42.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T11:19:42.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T11:19:42.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T11:19:42.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T11:19:42.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.row_cache: None 2024-09-10T11:19:42.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.wal_filter: None 2024-09-10T11:19:42.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T11:19:42.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T11:19:42.690 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.two_write_queues: 0 2024-09-10T11:19:42.690 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T11:19:42.690 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.wal_compression: 0 2024-09-10T11:19:42.691 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.atomic_flush: 0 2024-09-10T11:19:42.691 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T11:19:42.691 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T11:19:42.691 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T11:19:42.691 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.log_readahead_size: 0 2024-09-10T11:19:42.691 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T11:19:42.691 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T11:19:42.691 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T11:19:42.691 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T11:19:42.691 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T11:19:42.691 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T11:19:42.691 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T11:19:42.691 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_background_jobs: 2 2024-09-10T11:19:42.692 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_background_compactions: -1 2024-09-10T11:19:42.692 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_subcompactions: 1 2024-09-10T11:19:42.692 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T11:19:42.692 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T11:19:42.692 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T11:19:42.692 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T11:19:42.692 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T11:19:42.692 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T11:19:42.692 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T11:19:42.692 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T11:19:42.692 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_open_files: -1 2024-09-10T11:19:42.692 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T11:19:42.692 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T11:19:42.692 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T11:19:42.693 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T11:19:42.693 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_background_flushes: -1 2024-09-10T11:19:42.693 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Compression algorithms supported: 2024-09-10T11:19:42.693 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T11:19:42.693 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: kZSTD supported: 0 2024-09-10T11:19:42.693 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: kXpressCompression supported: 0 2024-09-10T11:19:42.693 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T11:19:42.693 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: kZlibCompression supported: 1 2024-09-10T11:19:42.693 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: kSnappyCompression supported: 1 2024-09-10T11:19:42.693 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: kLZ4Compression supported: 1 2024-09-10T11:19:42.693 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: kBZip2Compression supported: 0 2024-09-10T11:19:42.693 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T11:19:42.693 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T11:19:42.693 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi144/store.db/MANIFEST-000010 2024-09-10T11:19:42.693 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T11:19:42.694 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T11:19:42.694 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.merge_operator: 2024-09-10T11:19:42.694 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compaction_filter: None 2024-09-10T11:19:42.694 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T11:19:42.694 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T11:19:42.694 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T11:19:42.694 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T11:19:42.694 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5654193133c0) 2024-09-10T11:19:42.694 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: cache_index_and_filter_blocks: 1 2024-09-10T11:19:42.694 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T11:19:42.694 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T11:19:42.694 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: pin_top_level_index_and_filter: 1 2024-09-10T11:19:42.694 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: index_type: 0 2024-09-10T11:19:42.694 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: data_block_index_type: 0 2024-09-10T11:19:42.695 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: index_shortening: 1 2024-09-10T11:19:42.695 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T11:19:42.695 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: checksum: 4 2024-09-10T11:19:42.695 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: no_block_cache: 0 2024-09-10T11:19:42.695 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache: 0x565418625350 2024-09-10T11:19:42.695 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache_name: BinnedLRUCache 2024-09-10T11:19:42.695 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache_options: 2024-09-10T11:19:42.695 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: capacity : 536870912 2024-09-10T11:19:42.695 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: num_shard_bits : 4 2024-09-10T11:19:42.695 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: strict_capacity_limit : 0 2024-09-10T11:19:42.695 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: high_pri_pool_ratio: 0.000 2024-09-10T11:19:42.695 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache_compressed: (nil) 2024-09-10T11:19:42.695 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: persistent_cache: (nil) 2024-09-10T11:19:42.695 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_size: 4096 2024-09-10T11:19:42.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_size_deviation: 10 2024-09-10T11:19:42.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_restart_interval: 16 2024-09-10T11:19:42.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: index_block_restart_interval: 1 2024-09-10T11:19:42.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: metadata_block_size: 4096 2024-09-10T11:19:42.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: partition_filters: 0 2024-09-10T11:19:42.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: use_delta_encoding: 1 2024-09-10T11:19:42.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: filter_policy: bloomfilter 2024-09-10T11:19:42.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: whole_key_filtering: 1 2024-09-10T11:19:42.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: verify_compression: 0 2024-09-10T11:19:42.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: read_amp_bytes_per_bit: 0 2024-09-10T11:19:42.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: format_version: 5 2024-09-10T11:19:42.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: enable_index_compression: 1 2024-09-10T11:19:42.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_align: 0 2024-09-10T11:19:42.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: max_auto_readahead_size: 262144 2024-09-10T11:19:42.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: prepopulate_block_cache: 0 2024-09-10T11:19:42.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: initial_auto_readahead_size: 8192 2024-09-10T11:19:42.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T11:19:42.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T11:19:42.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T11:19:42.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compression: NoCompression 2024-09-10T11:19:42.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T11:19:42.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T11:19:42.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T11:19:42.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.num_levels: 7 2024-09-10T11:19:42.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T11:19:42.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T11:19:42.697 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T11:19:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T11:19:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T11:19:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T11:19:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T11:19:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T11:19:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T11:19:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T11:19:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T11:19:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T11:19:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T11:19:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T11:19:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T11:19:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T11:19:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T11:19:42.698 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T11:19:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T11:19:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T11:19:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T11:19:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T11:19:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T11:19:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T11:19:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T11:19:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T11:19:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T11:19:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T11:19:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T11:19:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T11:19:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T11:19:42.699 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T11:19:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T11:19:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T11:19:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T11:19:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T11:19:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T11:19:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T11:19:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T11:19:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T11:19:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T11:19:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T11:19:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T11:19:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T11:19:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T11:19:42.700 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T11:19:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T11:19:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T11:19:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T11:19:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T11:19:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T11:19:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T11:19:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T11:19:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.table_properties_collectors: 2024-09-10T11:19:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.inplace_update_support: 0 2024-09-10T11:19:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T11:19:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T11:19:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T11:19:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T11:19:42.701 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.bloom_locality: 0 2024-09-10T11:19:42.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.max_successive_merges: 0 2024-09-10T11:19:42.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T11:19:42.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T11:19:42.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T11:19:42.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T11:19:42.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.ttl: 2592000 2024-09-10T11:19:42.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T11:19:42.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T11:19:42.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T11:19:42.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.enable_blob_files: false 2024-09-10T11:19:42.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.min_blob_size: 0 2024-09-10T11:19:42.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T11:19:42.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T11:19:42.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T11:19:42.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T11:19:42.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T11:19:42.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T11:19:42.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T11:19:42.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T11:19:42.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi144/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-10T11:19:42.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-10T11:19:42.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e484da36-1ce5-4d20-8c49-e5c565568da8 2024-09-10T11:19:42.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725967182240906, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-10T11:19:42.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-10T11:19:42.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725967182241997, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94045, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 92051, "index_size": 276, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13701, "raw_average_key_size": 49, "raw_value_size": 84559, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725967182, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e484da36-1ce5-4d20-8c49-e5c565568da8", "db_session_id": "604HCIN8S740KHCKAMES", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-10T11:19:42.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725967182242064, "job": 1, "event": "recovery_finished"} 2024-09-10T11:19:42.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-10T11:19:42.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi144/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T11:19:42.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5654186c2000 2024-09-10T11:19:42.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: DB pointer 0x5654186ae000 2024-09-10T11:19:42.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T11:19:42.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T11:19:42.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** DB Stats ** 2024-09-10T11:19:42.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T11:19:42.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T11:19:42.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T11:19:42.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T11:19:42.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T11:19:42.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T11:19:42.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T11:19:42.704 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-09-10T11:19:42.705 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** Compaction Stats [default] ** 2024-09-10T11:19:42.705 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T11:19:42.705 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T11:19:42.705 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: L0 2/0 93.71 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 126.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T11:19:42.705 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Sum 2/0 93.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 126.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T11:19:42.705 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 126.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T11:19:42.705 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-09-10T11:19:42.705 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** Compaction Stats [default] ** 2024-09-10T11:19:42.705 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T11:19:42.705 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T11:19:42.705 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 126.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T11:19:42.705 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-09-10T11:19:42.705 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T11:19:42.705 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-09-10T11:19:42.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T11:19:42.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T11:19:42.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T11:19:42.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T11:19:42.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T11:19:42.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T11:19:42.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative compaction: 0.00 GB write, 26.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T11:19:42.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval compaction: 0.00 GB write, 26.54 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T11:19:42.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T11:19:42.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Block cache BinnedLRUCache@0x565418625350#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-09-10T11:19:42.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-10T11:19:42.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-09-10T11:19:42.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T11:19:42.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: starting mon.smithi144 rank 0 at public addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] at bind addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi144 fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:19:42.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: mon.smithi144@-1(???) e1 preinit fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:19:42.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: mon.smithi144@-1(???).mds e1 new map 2024-09-10T11:19:42.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: mon.smithi144@-1(???).mds e1 print_map 2024-09-10T11:19:42.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: e1 2024-09-10T11:19:42.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:19:42.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:19:42.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: legacy client fscid: -1 2024-09-10T11:19:42.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-09-10T11:19:42.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: No filesystems configured 2024-09-10T11:19:42.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: mon.smithi144@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T11:19:42.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: mon.smithi144@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T11:19:42.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: mon.smithi144@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T11:19:42.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: mon.smithi144@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T11:19:42.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: mon.smithi144@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T11:19:42.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: expand_channel_meta expand map: {default=false} 2024-09-10T11:19:42.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: expand_channel_meta from 'false' to 'false' 2024-09-10T11:19:42.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: expand_channel_meta expanded map: {default=false} 2024-09-10T11:19:42.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: expand_channel_meta expand map: {default=info} 2024-09-10T11:19:42.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: expand_channel_meta from 'info' to 'info' 2024-09-10T11:19:42.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: expand_channel_meta expanded map: {default=info} 2024-09-10T11:19:42.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: expand_channel_meta expand map: {default=daemon} 2024-09-10T11:19:42.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: expand_channel_meta from 'daemon' to 'daemon' 2024-09-10T11:19:42.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: expand_channel_meta expanded map: {default=daemon} 2024-09-10T11:19:42.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: expand_channel_meta expand map: {default=debug} 2024-09-10T11:19:42.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: expand_channel_meta from 'debug' to 'debug' 2024-09-10T11:19:42.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: expand_channel_meta expanded map: {default=debug} 2024-09-10T11:19:42.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: mon.smithi144 is new leader, mons smithi144 in quorum (ranks 0) 2024-09-10T11:19:42.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: monmap e1: 1 mons at {smithi144=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0]} removed_ranks: {} 2024-09-10T11:19:42.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: fsmap 2024-09-10T11:19:42.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T11:19:42.709 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:42 smithi144 ceph-mon[27942]: mgrmap e1: no daemons active 2024-09-10T11:19:43.048 INFO:teuthology.orchestra.run.smithi144.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T11:19:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:43 smithi144 bash[27923]: 42addb654e4f8b796385356b778525a269a7993950cca0146f524e6c0328c912 2024-09-10T11:19:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:43 smithi144 systemd[1]: Started Ceph mon.smithi144 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:19:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:47 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2063482966' entity='client.admin' 2024-09-10T11:19:48.966 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.146+0000 7fbf10a24700 1 Processor -- start 2024-09-10T11:19:48.967 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.146+0000 7fbf10a24700 1 -- start start 2024-09-10T11:19:48.967 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.146+0000 7fbf10a24700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf0c104f80 0x7fbf0c105390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:19:48.967 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.146+0000 7fbf10a24700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c1058d0 con 0x7fbf0c104f80 2024-09-10T11:19:48.967 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.147+0000 7fbf0a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf0c104f80 0x7fbf0c105390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:19:48.967 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.147+0000 7fbf0a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf0c104f80 0x7fbf0c105390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:47060/0 (socket says 172.21.15.144:47060) 2024-09-10T11:19:48.967 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.147+0000 7fbf0a59c700 1 -- 172.21.15.144:0/438608311 learned_addr learned my addr 172.21.15.144:0/438608311 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:19:48.967 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.147+0000 7fbf0a59c700 1 -- 172.21.15.144:0/438608311 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf0c105a10 con 0x7fbf0c104f80 2024-09-10T11:19:48.967 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.147+0000 7fbf0a59c700 1 --2- 172.21.15.144:0/438608311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf0c104f80 0x7fbf0c105390 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0098a0 tx=0x7fbefc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7347648adb40328 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:19:48.967 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.147+0000 7fbf0959a700 1 -- 172.21.15.144:0/438608311 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbefc004180 con 0x7fbf0c104f80 2024-09-10T11:19:48.967 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.147+0000 7fbf0959a700 1 -- 172.21.15.144:0/438608311 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbefc0042e0 con 0x7fbf0c104f80 2024-09-10T11:19:48.967 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.147+0000 7fbf0959a700 1 -- 172.21.15.144:0/438608311 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbefc011a50 con 0x7fbf0c104f80 2024-09-10T11:19:48.968 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.148+0000 7fbf10a24700 1 -- 172.21.15.144:0/438608311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf0c104f80 msgr2=0x7fbf0c105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:19:48.968 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.148+0000 7fbf10a24700 1 --2- 172.21.15.144:0/438608311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf0c104f80 0x7fbf0c105390 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbefc0098a0 tx=0x7fbefc009bb0 comp rx=0 tx=0).stop 2024-09-10T11:19:48.968 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.148+0000 7fbf10a24700 1 -- 172.21.15.144:0/438608311 shutdown_connections 2024-09-10T11:19:48.968 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.148+0000 7fbf10a24700 1 --2- 172.21.15.144:0/438608311 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf0c104f80 0x7fbf0c105390 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:19:48.968 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.148+0000 7fbf10a24700 1 -- 172.21.15.144:0/438608311 >> 172.21.15.144:0/438608311 conn(0x7fbf0c0ff870 msgr2=0x7fbf0c101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:19:48.968 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.148+0000 7fbf10a24700 1 -- 172.21.15.144:0/438608311 shutdown_connections 2024-09-10T11:19:48.968 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.148+0000 7fbf10a24700 1 -- 172.21.15.144:0/438608311 wait complete. 2024-09-10T11:19:48.968 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.149+0000 7fbf10a24700 1 Processor -- start 2024-09-10T11:19:48.968 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.149+0000 7fbf10a24700 1 -- start start 2024-09-10T11:19:48.968 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.149+0000 7fbf10a24700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf0c104f80 0x7fbf0c1987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:19:48.968 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.149+0000 7fbf10a24700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf0c198d30 con 0x7fbf0c104f80 2024-09-10T11:19:48.968 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.149+0000 7fbf0a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf0c104f80 0x7fbf0c1987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:19:48.968 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.149+0000 7fbf0a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf0c104f80 0x7fbf0c1987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:47068/0 (socket says 172.21.15.144:47068) 2024-09-10T11:19:48.969 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.149+0000 7fbf0a59c700 1 -- 172.21.15.144:0/2063482966 learned_addr learned my addr 172.21.15.144:0/2063482966 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:19:48.969 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.149+0000 7fbf0a59c700 1 -- 172.21.15.144:0/2063482966 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc009550 con 0x7fbf0c104f80 2024-09-10T11:19:48.969 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.149+0000 7fbf0a59c700 1 --2- 172.21.15.144:0/2063482966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf0c104f80 0x7fbf0c1987f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbefc006af0 tx=0x7fbefc011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:19:48.969 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.150+0000 7fbefb7fe700 1 -- 172.21.15.144:0/2063482966 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbefc003b90 con 0x7fbf0c104f80 2024-09-10T11:19:48.969 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.150+0000 7fbf10a24700 1 -- 172.21.15.144:0/2063482966 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf0c198f30 con 0x7fbf0c104f80 2024-09-10T11:19:48.969 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.150+0000 7fbefb7fe700 1 -- 172.21.15.144:0/2063482966 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbefc01a8a0 con 0x7fbf0c104f80 2024-09-10T11:19:48.969 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.150+0000 7fbefb7fe700 1 -- 172.21.15.144:0/2063482966 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbefc019880 con 0x7fbf0c104f80 2024-09-10T11:19:48.969 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.150+0000 7fbf10a24700 1 -- 172.21.15.144:0/2063482966 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf0c199350 con 0x7fbf0c104f80 2024-09-10T11:19:48.969 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.150+0000 7fbefb7fe700 1 -- 172.21.15.144:0/2063482966 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbefc02d440 con 0x7fbf0c104f80 2024-09-10T11:19:48.969 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.150+0000 7fbefb7fe700 1 -- 172.21.15.144:0/2063482966 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbefc024860 con 0x7fbf0c104f80 2024-09-10T11:19:48.969 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.151+0000 7fbf10a24700 1 -- 172.21.15.144:0/2063482966 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf0c062380 con 0x7fbf0c104f80 2024-09-10T11:19:48.969 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.153+0000 7fbefb7fe700 1 -- 172.21.15.144:0/2063482966 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fbefc01aa10 con 0x7fbf0c104f80 2024-09-10T11:19:48.970 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.220+0000 7fbf10a24700 1 -- 172.21.15.144:0/2063482966 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fbf0c106960 con 0x7fbf0c104f80 2024-09-10T11:19:48.970 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.223+0000 7fbefb7fe700 1 -- 172.21.15.144:0/2063482966 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fbefc01abb0 con 0x7fbf0c104f80 2024-09-10T11:19:48.970 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.224+0000 7fbf10a24700 1 -- 172.21.15.144:0/2063482966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf0c104f80 msgr2=0x7fbf0c1987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:19:48.970 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.224+0000 7fbf10a24700 1 --2- 172.21.15.144:0/2063482966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf0c104f80 0x7fbf0c1987f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fbefc006af0 tx=0x7fbefc011a90 comp rx=0 tx=0).stop 2024-09-10T11:19:48.970 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.224+0000 7fbf10a24700 1 -- 172.21.15.144:0/2063482966 shutdown_connections 2024-09-10T11:19:48.970 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.224+0000 7fbf10a24700 1 --2- 172.21.15.144:0/2063482966 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf0c104f80 0x7fbf0c1987f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:19:48.970 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.224+0000 7fbf10a24700 1 -- 172.21.15.144:0/2063482966 >> 172.21.15.144:0/2063482966 conn(0x7fbf0c0ff870 msgr2=0x7fbf0c100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:19:48.970 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.224+0000 7fbf10a24700 1 -- 172.21.15.144:0/2063482966 shutdown_connections 2024-09-10T11:19:48.970 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:46.224+0000 7fbf10a24700 1 -- 172.21.15.144:0/2063482966 wait complete. 2024-09-10T11:19:48.970 INFO:teuthology.orchestra.run.smithi144.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T11:19:48.993 INFO:teuthology.orchestra.run.smithi144.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T11:19:48.993 INFO:teuthology.orchestra.run.smithi144.stdout:Creating mgr... 2024-09-10T11:19:48.993 INFO:teuthology.orchestra.run.smithi144.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T11:19:48.993 INFO:teuthology.orchestra.run.smithi144.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T11:19:48.994 INFO:teuthology.orchestra.run.smithi144.stdout:Verifying port 0.0.0.0:8443 ... 2024-09-10T11:19:49.254 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mgr.smithi144.bkogfs 2024-09-10T11:19:49.254 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stderr Failed to reset failed state of unit ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mgr.smithi144.bkogfs.service: Unit ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mgr.smithi144.bkogfs.service not loaded. 2024-09-10T11:19:49.398 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-7eacd86c-6f66-11ef-bcea-c7b262605968.target.wants/ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mgr.smithi144.bkogfs.service → /etc/systemd/system/ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@.service. 2024-09-10T11:19:49.897 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T11:19:49.898 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stdout disabled 2024-09-10T11:19:49.905 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T11:19:49.906 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stdout inactive 2024-09-10T11:19:49.906 INFO:teuthology.orchestra.run.smithi144.stdout:firewalld.service is not enabled 2024-09-10T11:19:49.906 INFO:teuthology.orchestra.run.smithi144.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T11:19:49.916 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T11:19:49.916 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stdout disabled 2024-09-10T11:19:49.924 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T11:19:49.924 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stdout inactive 2024-09-10T11:19:49.924 INFO:teuthology.orchestra.run.smithi144.stdout:firewalld.service is not enabled 2024-09-10T11:19:49.925 INFO:teuthology.orchestra.run.smithi144.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-09-10T11:19:49.925 INFO:teuthology.orchestra.run.smithi144.stdout:Waiting for mgr to start... 2024-09-10T11:19:49.925 INFO:teuthology.orchestra.run.smithi144.stdout:Waiting for mgr... 2024-09-10T11:19:50.784 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:50 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1052057385' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T11:19:51.066 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 2024-09-10T11:19:51.066 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout { 2024-09-10T11:19:51.066 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "fsid": "7eacd86c-6f66-11ef-bcea-c7b262605968", 2024-09-10T11:19:51.066 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T11:19:51.067 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T11:19:51.067 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T11:19:51.067 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T11:19:51.067 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:51.067 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T11:19:51.067 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T11:19:51.067 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 0 2024-09-10T11:19:51.067 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout ], 2024-09-10T11:19:51.067 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T11:19:51.067 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "smithi144" 2024-09-10T11:19:51.067 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout ], 2024-09-10T11:19:51.067 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-09-10T11:19:51.067 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T11:19:51.068 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:19:51.068 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-10T11:19:51.068 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T11:19:51.068 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:51.068 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T11:19:51.068 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:19:51.068 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T11:19:51.068 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T11:19:51.068 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T11:19:51.068 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T11:19:51.068 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T11:19:51.068 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T11:19:51.068 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:51.068 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T11:19:51.069 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T11:19:51.069 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T11:19:51.069 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T11:19:51.069 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T11:19:51.069 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T11:19:51.069 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T11:19:51.069 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T11:19:51.069 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T11:19:51.069 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:51.069 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T11:19:51.070 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:19:51.070 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T11:19:51.070 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T11:19:51.070 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:51.070 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T11:19:51.070 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T11:19:51.070 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T11:19:51.070 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T11:19:51.070 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T11:19:51.070 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T11:19:51.070 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T11:19:51.070 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout ], 2024-09-10T11:19:51.070 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T11:19:51.071 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:51.071 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T11:19:51.071 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:19:51.071 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T11:19:32.021249+0000", 2024-09-10T11:19:51.071 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T11:19:51.071 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:51.071 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T11:19:51.071 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout } 2024-09-10T11:19:51.071 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.386+0000 7f67e66df700 1 Processor -- start 2024-09-10T11:19:51.071 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.386+0000 7f67e66df700 1 -- start start 2024-09-10T11:19:51.071 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.386+0000 7f67e66df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e0106730 0x7f67e0108b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:19:51.071 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.386+0000 7f67e66df700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e0079d60 con 0x7f67e0106730 2024-09-10T11:19:51.071 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.386+0000 7f67dffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e0106730 0x7f67e0108b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:19:51.071 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.386+0000 7f67dffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e0106730 0x7f67e0108b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:47104/0 (socket says 172.21.15.144:47104) 2024-09-10T11:19:51.072 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.386+0000 7f67dffff700 1 -- 172.21.15.144:0/1929526418 learned_addr learned my addr 172.21.15.144:0/1929526418 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:19:51.072 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.387+0000 7f67dffff700 1 -- 172.21.15.144:0/1929526418 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e0079ea0 con 0x7f67e0106730 2024-09-10T11:19:51.072 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.387+0000 7f67dffff700 1 --2- 172.21.15.144:0/1929526418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e0106730 0x7f67e0108b10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67d00098a0 tx=0x7f67d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbff58e9a25291bc server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:19:51.072 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.387+0000 7f67deffd700 1 -- 172.21.15.144:0/1929526418 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f67d0004180 con 0x7f67e0106730 2024-09-10T11:19:51.072 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.387+0000 7f67deffd700 1 -- 172.21.15.144:0/1929526418 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f67d00042e0 con 0x7f67e0106730 2024-09-10T11:19:51.072 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.387+0000 7f67deffd700 1 -- 172.21.15.144:0/1929526418 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f67d0011a50 con 0x7f67e0106730 2024-09-10T11:19:51.072 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.388+0000 7f67e66df700 1 -- 172.21.15.144:0/1929526418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e0106730 msgr2=0x7f67e0108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:19:51.072 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.388+0000 7f67e66df700 1 --2- 172.21.15.144:0/1929526418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e0106730 0x7f67e0108b10 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67d00098a0 tx=0x7f67d0009bb0 comp rx=0 tx=0).stop 2024-09-10T11:19:51.072 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.388+0000 7f67e66df700 1 -- 172.21.15.144:0/1929526418 shutdown_connections 2024-09-10T11:19:51.072 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.388+0000 7f67e66df700 1 --2- 172.21.15.144:0/1929526418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e0106730 0x7f67e0108b10 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:19:51.072 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.388+0000 7f67e66df700 1 -- 172.21.15.144:0/1929526418 >> 172.21.15.144:0/1929526418 conn(0x7f67e0100130 msgr2=0x7f67e0102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:19:51.073 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.388+0000 7f67e66df700 1 -- 172.21.15.144:0/1929526418 shutdown_connections 2024-09-10T11:19:51.073 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.388+0000 7f67e66df700 1 -- 172.21.15.144:0/1929526418 wait complete. 2024-09-10T11:19:51.073 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.388+0000 7f67e66df700 1 Processor -- start 2024-09-10T11:19:51.073 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.388+0000 7f67e66df700 1 -- start start 2024-09-10T11:19:51.073 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.389+0000 7f67e66df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e0106730 0x7f67e019baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:19:51.073 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.389+0000 7f67e66df700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e019bfe0 con 0x7f67e0106730 2024-09-10T11:19:51.073 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.389+0000 7f67dffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e0106730 0x7f67e019baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:19:51.073 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.389+0000 7f67dffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e0106730 0x7f67e019baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:47114/0 (socket says 172.21.15.144:47114) 2024-09-10T11:19:51.073 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.389+0000 7f67dffff700 1 -- 172.21.15.144:0/1052057385 learned_addr learned my addr 172.21.15.144:0/1052057385 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:19:51.073 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.389+0000 7f67dffff700 1 -- 172.21.15.144:0/1052057385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d0009550 con 0x7f67e0106730 2024-09-10T11:19:51.073 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.389+0000 7f67dffff700 1 --2- 172.21.15.144:0/1052057385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e0106730 0x7f67e019baa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f67d0000c00 tx=0x7f67d0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:19:51.074 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.389+0000 7f67dd7fa700 1 -- 172.21.15.144:0/1052057385 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f67d0003b90 con 0x7f67e0106730 2024-09-10T11:19:51.074 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.389+0000 7f67dd7fa700 1 -- 172.21.15.144:0/1052057385 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f67d00248b0 con 0x7f67e0106730 2024-09-10T11:19:51.074 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.389+0000 7f67e66df700 1 -- 172.21.15.144:0/1052057385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e019c240 con 0x7f67e0106730 2024-09-10T11:19:51.074 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.389+0000 7f67dd7fa700 1 -- 172.21.15.144:0/1052057385 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f67d001a8e0 con 0x7f67e0106730 2024-09-10T11:19:51.074 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.389+0000 7f67e66df700 1 -- 172.21.15.144:0/1052057385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e019c6c0 con 0x7f67e0106730 2024-09-10T11:19:51.074 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.390+0000 7f67dd7fa700 1 -- 172.21.15.144:0/1052057385 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f67d00193c0 con 0x7f67e0106730 2024-09-10T11:19:51.074 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.390+0000 7f67dd7fa700 1 -- 172.21.15.144:0/1052057385 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f67d0019670 con 0x7f67e0106730 2024-09-10T11:19:51.074 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.391+0000 7f67e66df700 1 -- 172.21.15.144:0/1052057385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e0109c70 con 0x7f67e0106730 2024-09-10T11:19:51.074 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.393+0000 7f67dd7fa700 1 -- 172.21.15.144:0/1052057385 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f67d0024dd0 con 0x7f67e0106730 2024-09-10T11:19:51.074 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.465+0000 7f67e66df700 1 -- 172.21.15.144:0/1052057385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f67e019f460 con 0x7f67e0106730 2024-09-10T11:19:51.074 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.466+0000 7f67dd7fa700 1 -- 172.21.15.144:0/1052057385 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f67d0024ad0 con 0x7f67e0106730 2024-09-10T11:19:51.074 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.467+0000 7f67e66df700 1 -- 172.21.15.144:0/1052057385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e0106730 msgr2=0x7f67e019baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:19:51.075 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.467+0000 7f67e66df700 1 --2- 172.21.15.144:0/1052057385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e0106730 0x7f67e019baa0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f67d0000c00 tx=0x7f67d0011ab0 comp rx=0 tx=0).stop 2024-09-10T11:19:51.075 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.467+0000 7f67e66df700 1 -- 172.21.15.144:0/1052057385 shutdown_connections 2024-09-10T11:19:51.075 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.467+0000 7f67e66df700 1 --2- 172.21.15.144:0/1052057385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e0106730 0x7f67e019baa0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:19:51.075 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.467+0000 7f67e66df700 1 -- 172.21.15.144:0/1052057385 >> 172.21.15.144:0/1052057385 conn(0x7f67e0100130 msgr2=0x7f67e0100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:19:51.075 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.467+0000 7f67e66df700 1 -- 172.21.15.144:0/1052057385 shutdown_connections 2024-09-10T11:19:51.075 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:50.467+0000 7f67e66df700 1 -- 172.21.15.144:0/1052057385 wait complete. 2024-09-10T11:19:51.075 INFO:teuthology.orchestra.run.smithi144.stdout:mgr not available, waiting (1/15)... 2024-09-10T11:19:53.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:53 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/975586478' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T11:19:54.539 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 2024-09-10T11:19:54.540 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout { 2024-09-10T11:19:54.540 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "fsid": "7eacd86c-6f66-11ef-bcea-c7b262605968", 2024-09-10T11:19:54.540 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T11:19:54.540 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T11:19:54.540 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T11:19:54.540 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T11:19:54.540 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:54.540 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T11:19:54.540 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T11:19:54.540 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 0 2024-09-10T11:19:54.540 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout ], 2024-09-10T11:19:54.540 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T11:19:54.540 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "smithi144" 2024-09-10T11:19:54.541 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout ], 2024-09-10T11:19:54.541 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-09-10T11:19:54.541 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T11:19:54.541 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:19:54.541 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-10T11:19:54.541 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T11:19:54.541 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:54.541 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T11:19:54.541 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:19:54.541 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T11:19:54.541 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T11:19:54.541 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T11:19:54.549 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T11:19:54.549 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T11:19:54.549 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T11:19:54.549 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:54.549 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T11:19:54.549 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T11:19:54.549 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T11:19:54.549 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T11:19:54.549 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T11:19:54.549 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T11:19:54.549 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T11:19:54.549 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T11:19:54.549 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T11:19:54.550 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:54.550 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T11:19:54.550 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:19:54.550 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T11:19:54.550 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T11:19:54.550 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:54.550 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T11:19:54.550 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T11:19:54.550 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T11:19:54.550 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T11:19:54.550 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T11:19:54.550 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T11:19:54.551 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T11:19:54.551 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout ], 2024-09-10T11:19:54.551 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T11:19:54.551 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:54.551 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T11:19:54.551 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:19:54.551 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T11:19:32.021249+0000", 2024-09-10T11:19:54.551 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T11:19:54.551 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:54.551 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T11:19:54.551 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout } 2024-09-10T11:19:54.551 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.432+0000 7f7fc55d9700 1 Processor -- start 2024-09-10T11:19:54.551 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.432+0000 7f7fc55d9700 1 -- start start 2024-09-10T11:19:54.551 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.432+0000 7f7fc55d9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc0104f80 0x7f7fc0105390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:19:54.552 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.432+0000 7f7fc55d9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc01058d0 con 0x7f7fc0104f80 2024-09-10T11:19:54.552 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.432+0000 7f7fbeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc0104f80 0x7f7fc0105390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:19:54.552 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.432+0000 7f7fbeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc0104f80 0x7f7fc0105390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53080/0 (socket says 172.21.15.144:53080) 2024-09-10T11:19:54.552 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.432+0000 7f7fbeffd700 1 -- 172.21.15.144:0/2219901 learned_addr learned my addr 172.21.15.144:0/2219901 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:19:54.552 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.432+0000 7f7fbeffd700 1 -- 172.21.15.144:0/2219901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc0105a10 con 0x7f7fc0104f80 2024-09-10T11:19:54.552 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.432+0000 7f7fbeffd700 1 --2- 172.21.15.144:0/2219901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc0104f80 0x7f7fc0105390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00098a0 tx=0x7f7fb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38139b9cbe4d0f34 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:19:54.552 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.433+0000 7f7fbdffb700 1 -- 172.21.15.144:0/2219901 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fb0004180 con 0x7f7fc0104f80 2024-09-10T11:19:54.552 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.433+0000 7f7fbdffb700 1 -- 172.21.15.144:0/2219901 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fb00042e0 con 0x7f7fc0104f80 2024-09-10T11:19:54.552 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.433+0000 7f7fbdffb700 1 -- 172.21.15.144:0/2219901 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fb0011a50 con 0x7f7fc0104f80 2024-09-10T11:19:54.552 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.433+0000 7f7fc55d9700 1 -- 172.21.15.144:0/2219901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc0104f80 msgr2=0x7f7fc0105390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:19:54.552 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.433+0000 7f7fc55d9700 1 --2- 172.21.15.144:0/2219901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc0104f80 0x7f7fc0105390 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7fb00098a0 tx=0x7f7fb0009bb0 comp rx=0 tx=0).stop 2024-09-10T11:19:54.552 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.433+0000 7f7fc55d9700 1 -- 172.21.15.144:0/2219901 shutdown_connections 2024-09-10T11:19:54.552 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.433+0000 7f7fc55d9700 1 --2- 172.21.15.144:0/2219901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc0104f80 0x7f7fc0105390 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:19:54.553 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.433+0000 7f7fc55d9700 1 -- 172.21.15.144:0/2219901 >> 172.21.15.144:0/2219901 conn(0x7f7fc00ff870 msgr2=0x7f7fc0101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:19:54.553 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.433+0000 7f7fc55d9700 1 -- 172.21.15.144:0/2219901 shutdown_connections 2024-09-10T11:19:54.553 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.433+0000 7f7fc55d9700 1 -- 172.21.15.144:0/2219901 wait complete. 2024-09-10T11:19:54.553 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.434+0000 7f7fc55d9700 1 Processor -- start 2024-09-10T11:19:54.553 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.434+0000 7f7fc55d9700 1 -- start start 2024-09-10T11:19:54.553 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.434+0000 7f7fc55d9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc0104f80 0x7f7fc01998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:19:54.553 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.434+0000 7f7fc55d9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc0199e30 con 0x7f7fc0104f80 2024-09-10T11:19:54.553 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.434+0000 7f7fbeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc0104f80 0x7f7fc01998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:19:54.553 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.434+0000 7f7fbeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc0104f80 0x7f7fc01998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53088/0 (socket says 172.21.15.144:53088) 2024-09-10T11:19:54.553 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.434+0000 7f7fbeffd700 1 -- 172.21.15.144:0/975586478 learned_addr learned my addr 172.21.15.144:0/975586478 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:19:54.553 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.435+0000 7f7fbeffd700 1 -- 172.21.15.144:0/975586478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb0009550 con 0x7f7fc0104f80 2024-09-10T11:19:54.553 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.435+0000 7f7fbeffd700 1 --2- 172.21.15.144:0/975586478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc0104f80 0x7f7fc01998f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0006af0 tx=0x7f7fb0011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:19:54.553 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.435+0000 7f7fa7fff700 1 -- 172.21.15.144:0/975586478 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fb0003b90 con 0x7f7fc0104f80 2024-09-10T11:19:54.553 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.435+0000 7f7fa7fff700 1 -- 172.21.15.144:0/975586478 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f7fb001a8a0 con 0x7f7fc0104f80 2024-09-10T11:19:54.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.435+0000 7f7fa7fff700 1 -- 172.21.15.144:0/975586478 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fb0019880 con 0x7f7fc0104f80 2024-09-10T11:19:54.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.435+0000 7f7fc55d9700 1 -- 172.21.15.144:0/975586478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc019a030 con 0x7f7fc0104f80 2024-09-10T11:19:54.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.435+0000 7f7fc55d9700 1 -- 172.21.15.144:0/975586478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc019a450 con 0x7f7fc0104f80 2024-09-10T11:19:54.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.435+0000 7f7fa7fff700 1 -- 172.21.15.144:0/975586478 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7fb002d440 con 0x7f7fc0104f80 2024-09-10T11:19:54.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.435+0000 7f7fa7fff700 1 -- 172.21.15.144:0/975586478 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7fb0024800 con 0x7f7fc0104f80 2024-09-10T11:19:54.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.436+0000 7f7fc55d9700 1 -- 172.21.15.144:0/975586478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc0062380 con 0x7f7fc0104f80 2024-09-10T11:19:54.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.440+0000 7f7fa7fff700 1 -- 172.21.15.144:0/975586478 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f7fb001aa10 con 0x7f7fc0104f80 2024-09-10T11:19:54.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.513+0000 7f7fc55d9700 1 -- 172.21.15.144:0/975586478 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7fc0106960 con 0x7f7fc0104f80 2024-09-10T11:19:54.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.513+0000 7f7fa7fff700 1 -- 172.21.15.144:0/975586478 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7f7fb001abb0 con 0x7f7fc0104f80 2024-09-10T11:19:54.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.515+0000 7f7fc55d9700 1 -- 172.21.15.144:0/975586478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc0104f80 msgr2=0x7f7fc01998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:19:54.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.515+0000 7f7fc55d9700 1 --2- 172.21.15.144:0/975586478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc0104f80 0x7f7fc01998f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7fb0006af0 tx=0x7f7fb0011a90 comp rx=0 tx=0).stop 2024-09-10T11:19:54.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.515+0000 7f7fc55d9700 1 -- 172.21.15.144:0/975586478 shutdown_connections 2024-09-10T11:19:54.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.515+0000 7f7fc55d9700 1 --2- 172.21.15.144:0/975586478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc0104f80 0x7f7fc01998f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:19:54.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.515+0000 7f7fc55d9700 1 -- 172.21.15.144:0/975586478 >> 172.21.15.144:0/975586478 conn(0x7f7fc00ff870 msgr2=0x7f7fc0104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:19:54.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.515+0000 7f7fc55d9700 1 -- 172.21.15.144:0/975586478 shutdown_connections 2024-09-10T11:19:54.555 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:53.515+0000 7f7fc55d9700 1 -- 172.21.15.144:0/975586478 wait complete. 2024-09-10T11:19:54.555 INFO:teuthology.orchestra.run.smithi144.stdout:mgr not available, waiting (2/15)... 2024-09-10T11:19:56.794 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:56 smithi144 ceph-mon[27942]: Activating manager daemon smithi144.bkogfs 2024-09-10T11:19:56.794 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:56 smithi144 ceph-mon[27942]: mgrmap e2: smithi144.bkogfs(active, starting, since 0.0029428s) 2024-09-10T11:19:56.794 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:56 smithi144 ceph-mon[27942]: from='mgr.14100 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T11:19:56.794 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:56 smithi144 ceph-mon[27942]: from='mgr.14100 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T11:19:56.795 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:56 smithi144 ceph-mon[27942]: from='mgr.14100 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T11:19:56.795 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:56 smithi144 ceph-mon[27942]: from='mgr.14100 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:19:56.795 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:56 smithi144 ceph-mon[27942]: from='mgr.14100 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr metadata", "who": "smithi144.bkogfs", "id": "smithi144.bkogfs"}]: dispatch 2024-09-10T11:19:56.795 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:56 smithi144 ceph-mon[27942]: Manager daemon smithi144.bkogfs is now available 2024-09-10T11:19:56.795 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:56 smithi144 ceph-mon[27942]: from='mgr.14100 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/mirror_snapshot_schedule"}]: dispatch 2024-09-10T11:19:56.795 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:56 smithi144 ceph-mon[27942]: from='mgr.14100 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:19:56.795 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:56 smithi144 ceph-mon[27942]: from='mgr.14100 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/trash_purge_schedule"}]: dispatch 2024-09-10T11:19:56.795 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:56 smithi144 ceph-mon[27942]: from='mgr.14100 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:19:56.795 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:56 smithi144 ceph-mon[27942]: from='mgr.14100 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:19:57.627 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 2024-09-10T11:19:57.628 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout { 2024-09-10T11:19:57.628 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "fsid": "7eacd86c-6f66-11ef-bcea-c7b262605968", 2024-09-10T11:19:57.628 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T11:19:57.628 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T11:19:57.628 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T11:19:57.628 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T11:19:57.628 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:57.628 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T11:19:57.628 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T11:19:57.628 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 0 2024-09-10T11:19:57.628 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout ], 2024-09-10T11:19:57.628 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T11:19:57.629 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "smithi144" 2024-09-10T11:19:57.629 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout ], 2024-09-10T11:19:57.629 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-09-10T11:19:57.629 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T11:19:57.629 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:19:57.629 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-10T11:19:57.629 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T11:19:57.629 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:57.629 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T11:19:57.629 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:19:57.629 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T11:19:57.629 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T11:19:57.629 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T11:19:57.629 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T11:19:57.630 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T11:19:57.630 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T11:19:57.630 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:57.630 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T11:19:57.630 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T11:19:57.630 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T11:19:57.630 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T11:19:57.630 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T11:19:57.630 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T11:19:57.630 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T11:19:57.630 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T11:19:57.630 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T11:19:57.630 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:57.630 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T11:19:57.631 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:19:57.631 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T11:19:57.631 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T11:19:57.631 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:57.631 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T11:19:57.631 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T11:19:57.631 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T11:19:57.631 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T11:19:57.631 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T11:19:57.631 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T11:19:57.631 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T11:19:57.631 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout ], 2024-09-10T11:19:57.631 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T11:19:57.631 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:57.632 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T11:19:57.632 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:19:57.632 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T11:19:32.021249+0000", 2024-09-10T11:19:57.632 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T11:19:57.632 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:19:57.632 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T11:19:57.632 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout } 2024-09-10T11:19:57.632 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.983+0000 7fb06e287700 1 Processor -- start 2024-09-10T11:19:57.632 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.983+0000 7fb06e287700 1 -- start start 2024-09-10T11:19:57.633 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.984+0000 7fb06e287700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06807be20 0x7fb06807a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:19:57.633 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.984+0000 7fb06e287700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06807a860 con 0x7fb06807be20 2024-09-10T11:19:57.633 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.984+0000 7fb067fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06807be20 0x7fb06807a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:19:57.633 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.984+0000 7fb067fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06807be20 0x7fb06807a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53154/0 (socket says 172.21.15.144:53154) 2024-09-10T11:19:57.633 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.984+0000 7fb067fff700 1 -- 172.21.15.144:0/761713020 learned_addr learned my addr 172.21.15.144:0/761713020 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:19:57.633 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.984+0000 7fb067fff700 1 -- 172.21.15.144:0/761713020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb06807a9a0 con 0x7fb06807be20 2024-09-10T11:19:57.633 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.984+0000 7fb067fff700 1 --2- 172.21.15.144:0/761713020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06807be20 0x7fb06807a320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb0580098a0 tx=0x7fb058009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af01e5428ee5f36 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:19:57.633 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.984+0000 7fb066ffd700 1 -- 172.21.15.144:0/761713020 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb058004180 con 0x7fb06807be20 2024-09-10T11:19:57.633 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.984+0000 7fb066ffd700 1 -- 172.21.15.144:0/761713020 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb0580042e0 con 0x7fb06807be20 2024-09-10T11:19:57.633 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.984+0000 7fb066ffd700 1 -- 172.21.15.144:0/761713020 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb058011a50 con 0x7fb06807be20 2024-09-10T11:19:57.633 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.985+0000 7fb06e287700 1 -- 172.21.15.144:0/761713020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06807be20 msgr2=0x7fb06807a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:19:57.633 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.985+0000 7fb06e287700 1 --2- 172.21.15.144:0/761713020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06807be20 0x7fb06807a320 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb0580098a0 tx=0x7fb058009bb0 comp rx=0 tx=0).stop 2024-09-10T11:19:57.633 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.985+0000 7fb06e287700 1 -- 172.21.15.144:0/761713020 shutdown_connections 2024-09-10T11:19:57.634 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.985+0000 7fb06e287700 1 --2- 172.21.15.144:0/761713020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06807be20 0x7fb06807a320 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:19:57.634 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.985+0000 7fb06e287700 1 -- 172.21.15.144:0/761713020 >> 172.21.15.144:0/761713020 conn(0x7fb068101220 msgr2=0x7fb068103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:19:57.634 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.985+0000 7fb06e287700 1 -- 172.21.15.144:0/761713020 shutdown_connections 2024-09-10T11:19:57.634 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.985+0000 7fb06e287700 1 -- 172.21.15.144:0/761713020 wait complete. 2024-09-10T11:19:57.634 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.986+0000 7fb06e287700 1 Processor -- start 2024-09-10T11:19:57.634 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.986+0000 7fb06e287700 1 -- start start 2024-09-10T11:19:57.634 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.986+0000 7fb06e287700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06807be20 0x7fb06819baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:19:57.634 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.986+0000 7fb06e287700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06819c030 con 0x7fb06807be20 2024-09-10T11:19:57.634 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.986+0000 7fb067fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06807be20 0x7fb06819baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:19:57.634 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.986+0000 7fb067fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06807be20 0x7fb06819baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53166/0 (socket says 172.21.15.144:53166) 2024-09-10T11:19:57.634 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.986+0000 7fb067fff700 1 -- 172.21.15.144:0/2976500697 learned_addr learned my addr 172.21.15.144:0/2976500697 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:19:57.634 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.986+0000 7fb067fff700 1 -- 172.21.15.144:0/2976500697 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb058009550 con 0x7fb06807be20 2024-09-10T11:19:57.635 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.986+0000 7fb067fff700 1 --2- 172.21.15.144:0/2976500697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06807be20 0x7fb06819baf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb058000c00 tx=0x7fb058011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:19:57.635 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.987+0000 7fb0657fa700 1 -- 172.21.15.144:0/2976500697 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb058003b90 con 0x7fb06807be20 2024-09-10T11:19:57.635 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.987+0000 7fb0657fa700 1 -- 172.21.15.144:0/2976500697 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb0580248b0 con 0x7fb06807be20 2024-09-10T11:19:57.635 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.987+0000 7fb0657fa700 1 -- 172.21.15.144:0/2976500697 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb05801a8e0 con 0x7fb06807be20 2024-09-10T11:19:57.635 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.987+0000 7fb06e287700 1 -- 172.21.15.144:0/2976500697 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb06819c230 con 0x7fb06807be20 2024-09-10T11:19:57.635 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.987+0000 7fb06e287700 1 -- 172.21.15.144:0/2976500697 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb06819c650 con 0x7fb06807be20 2024-09-10T11:19:57.635 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.987+0000 7fb0657fa700 1 -- 172.21.15.144:0/2976500697 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 2) v1 ==== 44840+0+0 (secure 0 0 0) 0x7fb05801aba0 con 0x7fb06807be20 2024-09-10T11:19:57.635 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.988+0000 7fb0657fa700 1 -- 172.21.15.144:0/2976500697 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb05804be80 con 0x7fb06807be20 2024-09-10T11:19:57.635 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.988+0000 7fb06e287700 1 -- 172.21.15.144:0/2976500697 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb068062380 con 0x7fb06807be20 2024-09-10T11:19:57.636 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:56.990+0000 7fb0657fa700 1 -- 172.21.15.144:0/2976500697 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fb058024440 con 0x7fb06807be20 2024-09-10T11:19:57.636 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:57.059+0000 7fb06e287700 1 -- 172.21.15.144:0/2976500697 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb06819f720 con 0x7fb06807be20 2024-09-10T11:19:57.636 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:57.060+0000 7fb0657fa700 1 -- 172.21.15.144:0/2976500697 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1247 (secure 0 0 0) 0x7fb058046090 con 0x7fb06807be20 2024-09-10T11:19:57.636 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:57.061+0000 7fb06e287700 1 -- 172.21.15.144:0/2976500697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06807be20 msgr2=0x7fb06819baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:19:57.636 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:57.061+0000 7fb06e287700 1 --2- 172.21.15.144:0/2976500697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06807be20 0x7fb06819baf0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb058000c00 tx=0x7fb058011ab0 comp rx=0 tx=0).stop 2024-09-10T11:19:57.636 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:57.061+0000 7fb06e287700 1 -- 172.21.15.144:0/2976500697 shutdown_connections 2024-09-10T11:19:57.636 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:57.061+0000 7fb06e287700 1 --2- 172.21.15.144:0/2976500697 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb06807be20 0x7fb06819baf0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:19:57.636 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:57.061+0000 7fb06e287700 1 -- 172.21.15.144:0/2976500697 >> 172.21.15.144:0/2976500697 conn(0x7fb068101220 msgr2=0x7fb068102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:19:57.636 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:57.061+0000 7fb06e287700 1 -- 172.21.15.144:0/2976500697 shutdown_connections 2024-09-10T11:19:57.636 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:19:57.061+0000 7fb06e287700 1 -- 172.21.15.144:0/2976500697 wait complete. 2024-09-10T11:19:57.636 INFO:teuthology.orchestra.run.smithi144.stdout:mgr not available, waiting (3/15)... 2024-09-10T11:19:57.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:57 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2976500697' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T11:19:57.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:57 smithi144 ceph-mon[27942]: mgrmap e3: smithi144.bkogfs(active, since 1.00655s) 2024-09-10T11:19:59.852 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:19:59 smithi144 ceph-mon[27942]: mgrmap e4: smithi144.bkogfs(active, since 2s) 2024-09-10T11:20:00.863 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:00 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/427292536' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T11:20:00.869 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 2024-09-10T11:20:00.870 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout { 2024-09-10T11:20:00.870 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "fsid": "7eacd86c-6f66-11ef-bcea-c7b262605968", 2024-09-10T11:20:00.870 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T11:20:00.870 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T11:20:00.870 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T11:20:00.870 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T11:20:00.870 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:20:00.870 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T11:20:00.870 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T11:20:00.870 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 0 2024-09-10T11:20:00.871 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout ], 2024-09-10T11:20:00.871 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T11:20:00.871 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "smithi144" 2024-09-10T11:20:00.871 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout ], 2024-09-10T11:20:00.871 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "quorum_age": 18, 2024-09-10T11:20:00.871 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T11:20:00.871 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:20:00.871 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-09-10T11:20:00.871 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T11:20:00.871 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:20:00.871 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T11:20:00.871 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:20:00.871 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T11:20:00.871 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T11:20:00.881 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T11:20:00.881 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T11:20:00.881 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T11:20:00.881 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T11:20:00.881 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:20:00.881 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T11:20:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T11:20:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T11:20:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T11:20:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T11:20:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T11:20:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T11:20:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T11:20:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T11:20:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:20:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T11:20:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:20:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T11:20:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T11:20:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:20:00.882 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T11:20:00.883 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T11:20:00.883 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T11:20:00.883 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T11:20:00.883 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T11:20:00.883 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T11:20:00.883 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T11:20:00.883 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout ], 2024-09-10T11:20:00.883 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T11:20:00.883 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:20:00.883 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T11:20:00.883 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T11:20:00.883 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T11:19:32.021249+0000", 2024-09-10T11:20:00.883 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T11:20:00.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout }, 2024-09-10T11:20:00.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T11:20:00.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout } 2024-09-10T11:20:00.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.046+0000 7fd343e79700 1 Processor -- start 2024-09-10T11:20:00.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.046+0000 7fd343e79700 1 -- start start 2024-09-10T11:20:00.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.046+0000 7fd343e79700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd33c103e70 0x7fd33c104280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:00.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.046+0000 7fd343e79700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c1047c0 con 0x7fd33c103e70 2024-09-10T11:20:00.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.046+0000 7fd341c15700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd33c103e70 0x7fd33c104280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:00.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.046+0000 7fd341c15700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd33c103e70 0x7fd33c104280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53170/0 (socket says 172.21.15.144:53170) 2024-09-10T11:20:00.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.046+0000 7fd341c15700 1 -- 172.21.15.144:0/3821877489 learned_addr learned my addr 172.21.15.144:0/3821877489 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:00.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.047+0000 7fd341c15700 1 -- 172.21.15.144:0/3821877489 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c104900 con 0x7fd33c103e70 2024-09-10T11:20:00.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.047+0000 7fd341c15700 1 --2- 172.21.15.144:0/3821877489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd33c103e70 0x7fd33c104280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0098a0 tx=0x7fd32c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06f43f46ff38af0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:00.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.047+0000 7fd340c13700 1 -- 172.21.15.144:0/3821877489 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd32c004180 con 0x7fd33c103e70 2024-09-10T11:20:00.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.047+0000 7fd340c13700 1 -- 172.21.15.144:0/3821877489 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd32c0042e0 con 0x7fd33c103e70 2024-09-10T11:20:00.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.047+0000 7fd340c13700 1 -- 172.21.15.144:0/3821877489 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd32c011a50 con 0x7fd33c103e70 2024-09-10T11:20:00.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.048+0000 7fd343e79700 1 -- 172.21.15.144:0/3821877489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd33c103e70 msgr2=0x7fd33c104280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:00.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.048+0000 7fd343e79700 1 --2- 172.21.15.144:0/3821877489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd33c103e70 0x7fd33c104280 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd32c0098a0 tx=0x7fd32c009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:00.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.048+0000 7fd343e79700 1 -- 172.21.15.144:0/3821877489 shutdown_connections 2024-09-10T11:20:00.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.048+0000 7fd343e79700 1 --2- 172.21.15.144:0/3821877489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd33c103e70 0x7fd33c104280 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:00.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.048+0000 7fd343e79700 1 -- 172.21.15.144:0/3821877489 >> 172.21.15.144:0/3821877489 conn(0x7fd33c0ff870 msgr2=0x7fd33c101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:00.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.048+0000 7fd343e79700 1 -- 172.21.15.144:0/3821877489 shutdown_connections 2024-09-10T11:20:00.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.048+0000 7fd343e79700 1 -- 172.21.15.144:0/3821877489 wait complete. 2024-09-10T11:20:00.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.048+0000 7fd343e79700 1 Processor -- start 2024-09-10T11:20:00.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.048+0000 7fd343e79700 1 -- start start 2024-09-10T11:20:00.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.049+0000 7fd343e79700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd33c103e70 0x7fd33c1998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:00.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.049+0000 7fd343e79700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33c199e30 con 0x7fd33c103e70 2024-09-10T11:20:00.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.049+0000 7fd341c15700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd33c103e70 0x7fd33c1998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:00.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.049+0000 7fd341c15700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd33c103e70 0x7fd33c1998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53172/0 (socket says 172.21.15.144:53172) 2024-09-10T11:20:00.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.049+0000 7fd341c15700 1 -- 172.21.15.144:0/427292536 learned_addr learned my addr 172.21.15.144:0/427292536 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:00.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.049+0000 7fd341c15700 1 -- 172.21.15.144:0/427292536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32c009550 con 0x7fd33c103e70 2024-09-10T11:20:00.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.049+0000 7fd341c15700 1 --2- 172.21.15.144:0/427292536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd33c103e70 0x7fd33c1998f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd32c000c00 tx=0x7fd32c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:00.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.049+0000 7fd332ffd700 1 -- 172.21.15.144:0/427292536 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd32c003b90 con 0x7fd33c103e70 2024-09-10T11:20:00.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.049+0000 7fd343e79700 1 -- 172.21.15.144:0/427292536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33c19a030 con 0x7fd33c103e70 2024-09-10T11:20:00.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.049+0000 7fd332ffd700 1 -- 172.21.15.144:0/427292536 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fd32c0248b0 con 0x7fd33c103e70 2024-09-10T11:20:00.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.049+0000 7fd332ffd700 1 -- 172.21.15.144:0/427292536 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd32c01a8e0 con 0x7fd33c103e70 2024-09-10T11:20:00.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.049+0000 7fd343e79700 1 -- 172.21.15.144:0/427292536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33c19a450 con 0x7fd33c103e70 2024-09-10T11:20:00.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.050+0000 7fd332ffd700 1 -- 172.21.15.144:0/427292536 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7fd32c01aba0 con 0x7fd33c103e70 2024-09-10T11:20:00.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.050+0000 7fd343e79700 1 -- 172.21.15.144:0/427292536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd33c107a30 con 0x7fd33c103e70 2024-09-10T11:20:00.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.051+0000 7fd332ffd700 1 --2- 172.21.15.144:0/427292536 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd3280383a0 0x7fd32803a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:00.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.051+0000 7fd332ffd700 1 -- 172.21.15.144:0/427292536 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd32c04c260 con 0x7fd33c103e70 2024-09-10T11:20:00.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.051+0000 7fd341414700 1 --2- 172.21.15.144:0/427292536 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd3280383a0 0x7fd32803a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:00.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.051+0000 7fd341414700 1 --2- 172.21.15.144:0/427292536 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd3280383a0 0x7fd32803a850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd338006fd0 tx=0x7fd338006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:00.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.056+0000 7fd332ffd700 1 -- 172.21.15.144:0/427292536 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd32c024440 con 0x7fd33c103e70 2024-09-10T11:20:00.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.321+0000 7fd343e79700 1 -- 172.21.15.144:0/427292536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd33c107c20 con 0x7fd33c103e70 2024-09-10T11:20:00.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.321+0000 7fd332ffd700 1 -- 172.21.15.144:0/427292536 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7fd32c018350 con 0x7fd33c103e70 2024-09-10T11:20:00.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.323+0000 7fd343e79700 1 -- 172.21.15.144:0/427292536 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd3280383a0 msgr2=0x7fd32803a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:00.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.323+0000 7fd343e79700 1 --2- 172.21.15.144:0/427292536 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd3280383a0 0x7fd32803a850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd338006fd0 tx=0x7fd338006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:00.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.323+0000 7fd343e79700 1 -- 172.21.15.144:0/427292536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd33c103e70 msgr2=0x7fd33c1998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:00.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.323+0000 7fd343e79700 1 --2- 172.21.15.144:0/427292536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd33c103e70 0x7fd33c1998f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd32c000c00 tx=0x7fd32c011ab0 comp rx=0 tx=0).stop 2024-09-10T11:20:00.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.323+0000 7fd343e79700 1 -- 172.21.15.144:0/427292536 shutdown_connections 2024-09-10T11:20:00.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.323+0000 7fd343e79700 1 --2- 172.21.15.144:0/427292536 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd3280383a0 0x7fd32803a850 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:00.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.323+0000 7fd343e79700 1 --2- 172.21.15.144:0/427292536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd33c103e70 0x7fd33c1998f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:00.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.323+0000 7fd343e79700 1 -- 172.21.15.144:0/427292536 >> 172.21.15.144:0/427292536 conn(0x7fd33c0ff870 msgr2=0x7fd33c100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:00.888 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.324+0000 7fd343e79700 1 -- 172.21.15.144:0/427292536 shutdown_connections 2024-09-10T11:20:00.888 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:00.324+0000 7fd343e79700 1 -- 172.21.15.144:0/427292536 wait complete. 2024-09-10T11:20:00.888 INFO:teuthology.orchestra.run.smithi144.stdout:mgr is available 2024-09-10T11:20:01.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:01 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/398754492' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T11:20:02.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 2024-09-10T11:20:02.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T11:20:02.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout fsid = 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:20:02.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T11:20:02.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T11:20:02.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T11:20:02.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T11:20:02.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 2024-09-10T11:20:02.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T11:20:02.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T11:20:02.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 2024-09-10T11:20:02.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T11:20:02.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-10T11:20:02.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-10T11:20:02.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-10T11:20:02.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T11:20:02.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T11:20:02.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.282+0000 7f61630ad700 1 Processor -- start 2024-09-10T11:20:02.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.282+0000 7f61630ad700 1 -- start start 2024-09-10T11:20:02.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.282+0000 7f61630ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c104fb0 0x7f615c1053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:02.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.282+0000 7f61630ad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c105900 con 0x7f615c104fb0 2024-09-10T11:20:02.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.282+0000 7f6160e49700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c104fb0 0x7f615c1053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:02.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.282+0000 7f6160e49700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c104fb0 0x7f615c1053c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53186/0 (socket says 172.21.15.144:53186) 2024-09-10T11:20:02.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.282+0000 7f6160e49700 1 -- 172.21.15.144:0/1332313013 learned_addr learned my addr 172.21.15.144:0/1332313013 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:02.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.283+0000 7f6160e49700 1 -- 172.21.15.144:0/1332313013 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c105a40 con 0x7f615c104fb0 2024-09-10T11:20:02.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.283+0000 7f6160e49700 1 --2- 172.21.15.144:0/1332313013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c104fb0 0x7f615c1053c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f615000bb40 tx=0x7f615000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8baa9032a751450 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:02.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.283+0000 7f615b7fe700 1 -- 172.21.15.144:0/1332313013 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f615000dc00 con 0x7f615c104fb0 2024-09-10T11:20:02.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.283+0000 7f615b7fe700 1 -- 172.21.15.144:0/1332313013 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61500044a0 con 0x7f615c104fb0 2024-09-10T11:20:02.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.283+0000 7f615b7fe700 1 -- 172.21.15.144:0/1332313013 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6150010470 con 0x7f615c104fb0 2024-09-10T11:20:02.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.284+0000 7f61630ad700 1 -- 172.21.15.144:0/1332313013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c104fb0 msgr2=0x7f615c1053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:02.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.284+0000 7f61630ad700 1 --2- 172.21.15.144:0/1332313013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c104fb0 0x7f615c1053c0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f615000bb40 tx=0x7f615000be50 comp rx=0 tx=0).stop 2024-09-10T11:20:02.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.284+0000 7f61630ad700 1 -- 172.21.15.144:0/1332313013 shutdown_connections 2024-09-10T11:20:02.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.284+0000 7f61630ad700 1 --2- 172.21.15.144:0/1332313013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c104fb0 0x7f615c1053c0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.284+0000 7f61630ad700 1 -- 172.21.15.144:0/1332313013 >> 172.21.15.144:0/1332313013 conn(0x7f615c0ff880 msgr2=0x7f615c101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.284+0000 7f61630ad700 1 -- 172.21.15.144:0/1332313013 shutdown_connections 2024-09-10T11:20:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.284+0000 7f61630ad700 1 -- 172.21.15.144:0/1332313013 wait complete. 2024-09-10T11:20:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.284+0000 7f61630ad700 1 Processor -- start 2024-09-10T11:20:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.284+0000 7f61630ad700 1 -- start start 2024-09-10T11:20:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.284+0000 7f61630ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c104fb0 0x7f615c197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.284+0000 7f61630ad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c197c50 con 0x7f615c104fb0 2024-09-10T11:20:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.285+0000 7f6160e49700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c104fb0 0x7f615c197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.285+0000 7f6160e49700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c104fb0 0x7f615c197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53188/0 (socket says 172.21.15.144:53188) 2024-09-10T11:20:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.285+0000 7f6160e49700 1 -- 172.21.15.144:0/398754492 learned_addr learned my addr 172.21.15.144:0/398754492 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.285+0000 7f6160e49700 1 -- 172.21.15.144:0/398754492 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615000b7f0 con 0x7f615c104fb0 2024-09-10T11:20:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.285+0000 7f6160e49700 1 --2- 172.21.15.144:0/398754492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c104fb0 0x7f615c197710 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f615000b340 tx=0x7f6150003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.285+0000 7f6159ffb700 1 -- 172.21.15.144:0/398754492 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f615001e070 con 0x7f615c104fb0 2024-09-10T11:20:02.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.285+0000 7f6159ffb700 1 -- 172.21.15.144:0/398754492 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f61500040a0 con 0x7f615c104fb0 2024-09-10T11:20:02.107 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.285+0000 7f6159ffb700 1 -- 172.21.15.144:0/398754492 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f61500228b0 con 0x7f615c104fb0 2024-09-10T11:20:02.107 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.285+0000 7f61630ad700 1 -- 172.21.15.144:0/398754492 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615c197e50 con 0x7f615c104fb0 2024-09-10T11:20:02.107 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.285+0000 7f61630ad700 1 -- 172.21.15.144:0/398754492 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615c198270 con 0x7f615c104fb0 2024-09-10T11:20:02.107 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.286+0000 7f6159ffb700 1 -- 172.21.15.144:0/398754492 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7f615002c440 con 0x7f615c104fb0 2024-09-10T11:20:02.107 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.286+0000 7f6159ffb700 1 --2- 172.21.15.144:0/398754492 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f614c038360 0x7f614c03a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:02.107 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.286+0000 7f6159ffb700 1 -- 172.21.15.144:0/398754492 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f615004e1a0 con 0x7f615c104fb0 2024-09-10T11:20:02.107 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.286+0000 7f615bfff700 1 --2- 172.21.15.144:0/398754492 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f614c038360 0x7f614c03a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:02.107 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.286+0000 7f61630ad700 1 -- 172.21.15.144:0/398754492 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615c191400 con 0x7f615c104fb0 2024-09-10T11:20:02.107 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.287+0000 7f615bfff700 1 --2- 172.21.15.144:0/398754492 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f614c038360 0x7f614c03a810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6148006fd0 tx=0x7f6148006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:02.107 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.291+0000 7f6159ffb700 1 -- 172.21.15.144:0/398754492 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6150027020 con 0x7f615c104fb0 2024-09-10T11:20:02.107 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.479+0000 7f61630ad700 1 -- 172.21.15.144:0/398754492 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f615c106ae0 con 0x7f615c104fb0 2024-09-10T11:20:02.108 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.480+0000 7f6159ffb700 1 -- 172.21.15.144:0/398754492 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f6150021bd0 con 0x7f615c104fb0 2024-09-10T11:20:02.108 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.482+0000 7f61630ad700 1 -- 172.21.15.144:0/398754492 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f614c038360 msgr2=0x7f614c03a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:02.108 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.482+0000 7f61630ad700 1 --2- 172.21.15.144:0/398754492 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f614c038360 0x7f614c03a810 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6148006fd0 tx=0x7f6148006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:02.108 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.482+0000 7f61630ad700 1 -- 172.21.15.144:0/398754492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c104fb0 msgr2=0x7f615c197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:02.108 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.482+0000 7f61630ad700 1 --2- 172.21.15.144:0/398754492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c104fb0 0x7f615c197710 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f615000b340 tx=0x7f6150003cc0 comp rx=0 tx=0).stop 2024-09-10T11:20:02.108 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.482+0000 7f61630ad700 1 -- 172.21.15.144:0/398754492 shutdown_connections 2024-09-10T11:20:02.108 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.482+0000 7f61630ad700 1 --2- 172.21.15.144:0/398754492 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f614c038360 0x7f614c03a810 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:02.108 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.482+0000 7f61630ad700 1 --2- 172.21.15.144:0/398754492 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f615c104fb0 0x7f615c197710 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:02.108 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.482+0000 7f61630ad700 1 -- 172.21.15.144:0/398754492 >> 172.21.15.144:0/398754492 conn(0x7f615c0ff880 msgr2=0x7f615c18e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:02.108 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.483+0000 7f61630ad700 1 -- 172.21.15.144:0/398754492 shutdown_connections 2024-09-10T11:20:02.108 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:01.483+0000 7f61630ad700 1 -- 172.21.15.144:0/398754492 wait complete. 2024-09-10T11:20:02.108 INFO:teuthology.orchestra.run.smithi144.stdout:Enabling cephadm module... 2024-09-10T11:20:03.505 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:03 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/3673418785' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T11:20:03.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.504+0000 7f6ca139d700 1 Processor -- start 2024-09-10T11:20:03.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.504+0000 7f6ca139d700 1 -- start start 2024-09-10T11:20:03.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.505+0000 7f6ca139d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c1064f0 0x7f6c9c1088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:03.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.505+0000 7f6ca139d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c074460 con 0x7f6c9c1064f0 2024-09-10T11:20:03.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.505+0000 7f6c9bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c1064f0 0x7f6c9c1088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:03.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.505+0000 7f6c9bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c1064f0 0x7f6c9c1088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43800/0 (socket says 172.21.15.144:43800) 2024-09-10T11:20:03.884 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.505+0000 7f6c9bfff700 1 -- 172.21.15.144:0/4255770574 learned_addr learned my addr 172.21.15.144:0/4255770574 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:03.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.505+0000 7f6c9bfff700 1 -- 172.21.15.144:0/4255770574 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9c0745a0 con 0x7f6c9c1064f0 2024-09-10T11:20:03.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.505+0000 7f6c9bfff700 1 --2- 172.21.15.144:0/4255770574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c1064f0 0x7f6c9c1088d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0098a0 tx=0x7f6c8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ba48ce671bc19d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:03.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.505+0000 7f6c9affd700 1 -- 172.21.15.144:0/4255770574 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c8c004180 con 0x7f6c9c1064f0 2024-09-10T11:20:03.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.506+0000 7f6c9affd700 1 -- 172.21.15.144:0/4255770574 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c8c0042e0 con 0x7f6c9c1064f0 2024-09-10T11:20:03.885 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.506+0000 7f6c9affd700 1 -- 172.21.15.144:0/4255770574 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c8c011a50 con 0x7f6c9c1064f0 2024-09-10T11:20:03.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.506+0000 7f6ca139d700 1 -- 172.21.15.144:0/4255770574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c1064f0 msgr2=0x7f6c9c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:03.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.506+0000 7f6ca139d700 1 --2- 172.21.15.144:0/4255770574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c1064f0 0x7f6c9c1088d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c0098a0 tx=0x7f6c8c009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:03.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.506+0000 7f6ca139d700 1 -- 172.21.15.144:0/4255770574 shutdown_connections 2024-09-10T11:20:03.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.506+0000 7f6ca139d700 1 --2- 172.21.15.144:0/4255770574 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c1064f0 0x7f6c9c1088d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:03.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.506+0000 7f6ca139d700 1 -- 172.21.15.144:0/4255770574 >> 172.21.15.144:0/4255770574 conn(0x7f6c9c0fff50 msgr2=0x7f6c9c102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:03.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.506+0000 7f6ca139d700 1 -- 172.21.15.144:0/4255770574 shutdown_connections 2024-09-10T11:20:03.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.506+0000 7f6ca139d700 1 -- 172.21.15.144:0/4255770574 wait complete. 2024-09-10T11:20:03.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.507+0000 7f6ca139d700 1 Processor -- start 2024-09-10T11:20:03.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.507+0000 7f6ca139d700 1 -- start start 2024-09-10T11:20:03.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.507+0000 7f6ca139d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c1064f0 0x7f6c9c1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:03.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.507+0000 7f6ca139d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c9c199bf0 con 0x7f6c9c1064f0 2024-09-10T11:20:03.886 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.507+0000 7f6c9bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c1064f0 0x7f6c9c1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:03.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.507+0000 7f6c9bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c1064f0 0x7f6c9c1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43814/0 (socket says 172.21.15.144:43814) 2024-09-10T11:20:03.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.507+0000 7f6c9bfff700 1 -- 172.21.15.144:0/3673418785 learned_addr learned my addr 172.21.15.144:0/3673418785 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:03.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.508+0000 7f6c9bfff700 1 -- 172.21.15.144:0/3673418785 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c009550 con 0x7f6c9c1064f0 2024-09-10T11:20:03.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.508+0000 7f6c9bfff700 1 --2- 172.21.15.144:0/3673418785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c1064f0 0x7f6c9c1996b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c000c00 tx=0x7f6c8c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:03.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.508+0000 7f6c997fa700 1 -- 172.21.15.144:0/3673418785 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c8c003b90 con 0x7f6c9c1064f0 2024-09-10T11:20:03.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.508+0000 7f6c997fa700 1 -- 172.21.15.144:0/3673418785 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f6c8c0248b0 con 0x7f6c9c1064f0 2024-09-10T11:20:03.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.508+0000 7f6ca139d700 1 -- 172.21.15.144:0/3673418785 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c9c199e50 con 0x7f6c9c1064f0 2024-09-10T11:20:03.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.508+0000 7f6c997fa700 1 -- 172.21.15.144:0/3673418785 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c8c01a8e0 con 0x7f6c9c1064f0 2024-09-10T11:20:03.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.508+0000 7f6ca139d700 1 -- 172.21.15.144:0/3673418785 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c9c19a2d0 con 0x7f6c9c1064f0 2024-09-10T11:20:03.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.509+0000 7f6c997fa700 1 -- 172.21.15.144:0/3673418785 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7f6c8c01ab50 con 0x7f6c9c1064f0 2024-09-10T11:20:03.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.509+0000 7f6ca139d700 1 -- 172.21.15.144:0/3673418785 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9c0623f0 con 0x7f6c9c1064f0 2024-09-10T11:20:03.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.509+0000 7f6c997fa700 1 --2- 172.21.15.144:0/3673418785 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6c880383f0 0x7f6c8803a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:03.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.509+0000 7f6c997fa700 1 -- 172.21.15.144:0/3673418785 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6c8c04bff0 con 0x7f6c9c1064f0 2024-09-10T11:20:03.887 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.509+0000 7f6c9b7fe700 1 --2- 172.21.15.144:0/3673418785 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6c880383f0 0x7f6c8803a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:03.888 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.509+0000 7f6c9b7fe700 1 --2- 172.21.15.144:0/3673418785 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6c880383f0 0x7f6c8803a8a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6c84006fd0 tx=0x7f6c84006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:03.888 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.514+0000 7f6c997fa700 1 -- 172.21.15.144:0/3673418785 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6c9c0623f0 con 0x7f6c9c1064f0 2024-09-10T11:20:03.888 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:02.757+0000 7f6ca139d700 1 -- 172.21.15.144:0/3673418785 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f6c9c19cd00 con 0x7f6c9c1064f0 2024-09-10T11:20:03.888 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:03.254+0000 7f6c997fa700 1 -- 172.21.15.144:0/3673418785 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f6c8c024a20 con 0x7f6c9c1064f0 2024-09-10T11:20:03.888 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:03.256+0000 7f6c997fa700 1 -- 172.21.15.144:0/3673418785 <== mon.0 v2:172.21.15.144:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f6c8c050490 con 0x7f6c9c1064f0 2024-09-10T11:20:03.888 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:03.258+0000 7f6ca139d700 1 -- 172.21.15.144:0/3673418785 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6c880383f0 msgr2=0x7f6c8803a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:03.888 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:03.258+0000 7f6ca139d700 1 --2- 172.21.15.144:0/3673418785 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6c880383f0 0x7f6c8803a8a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6c84006fd0 tx=0x7f6c84006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:03.888 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:03.258+0000 7f6ca139d700 1 -- 172.21.15.144:0/3673418785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c1064f0 msgr2=0x7f6c9c1996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:03.888 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:03.258+0000 7f6ca139d700 1 --2- 172.21.15.144:0/3673418785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c1064f0 0x7f6c9c1996b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f6c8c000c00 tx=0x7f6c8c011ab0 comp rx=0 tx=0).stop 2024-09-10T11:20:03.888 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:03.258+0000 7f6ca139d700 1 -- 172.21.15.144:0/3673418785 shutdown_connections 2024-09-10T11:20:03.888 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:03.258+0000 7f6ca139d700 1 --2- 172.21.15.144:0/3673418785 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6c880383f0 0x7f6c8803a8a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:03.888 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:03.258+0000 7f6ca139d700 1 --2- 172.21.15.144:0/3673418785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c9c1064f0 0x7f6c9c1996b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:03.888 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:03.258+0000 7f6ca139d700 1 -- 172.21.15.144:0/3673418785 >> 172.21.15.144:0/3673418785 conn(0x7f6c9c0fff50 msgr2=0x7f6c9c101620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:03.889 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:03.258+0000 7f6ca139d700 1 -- 172.21.15.144:0/3673418785 shutdown_connections 2024-09-10T11:20:03.889 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:03.258+0000 7f6ca139d700 1 -- 172.21.15.144:0/3673418785 wait complete. 2024-09-10T11:20:04.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:04 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/3673418785' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T11:20:04.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:04 smithi144 ceph-mon[27942]: mgrmap e5: smithi144.bkogfs(active, since 6s) 2024-09-10T11:20:05.157 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout { 2024-09-10T11:20:05.157 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-10T11:20:05.158 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T11:20:05.158 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "active_name": "smithi144.bkogfs", 2024-09-10T11:20:05.158 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T11:20:05.158 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout } 2024-09-10T11:20:05.158 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.287+0000 7f98d44bf700 1 Processor -- start 2024-09-10T11:20:05.158 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.287+0000 7f98d44bf700 1 -- start start 2024-09-10T11:20:05.158 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.287+0000 7f98d44bf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc103ea0 0x7f98cc1042b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:05.158 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.287+0000 7f98d44bf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc1047f0 con 0x7f98cc103ea0 2024-09-10T11:20:05.158 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.287+0000 7f98d225b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc103ea0 0x7f98cc1042b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:05.158 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.287+0000 7f98d225b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc103ea0 0x7f98cc1042b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43826/0 (socket says 172.21.15.144:43826) 2024-09-10T11:20:05.158 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.287+0000 7f98d225b700 1 -- 172.21.15.144:0/950507749 learned_addr learned my addr 172.21.15.144:0/950507749 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:05.158 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.287+0000 7f98d225b700 1 -- 172.21.15.144:0/950507749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98cc104930 con 0x7f98cc103ea0 2024-09-10T11:20:05.159 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.288+0000 7f98d225b700 1 --2- 172.21.15.144:0/950507749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc103ea0 0x7f98cc1042b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98c80098a0 tx=0x7f98c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9377c12ee87b4765 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:05.159 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.288+0000 7f98d1259700 1 -- 172.21.15.144:0/950507749 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f98c8004180 con 0x7f98cc103ea0 2024-09-10T11:20:05.159 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.288+0000 7f98d1259700 1 -- 172.21.15.144:0/950507749 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f98c80042e0 con 0x7f98cc103ea0 2024-09-10T11:20:05.159 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.288+0000 7f98d1259700 1 -- 172.21.15.144:0/950507749 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f98c8011a50 con 0x7f98cc103ea0 2024-09-10T11:20:05.159 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.288+0000 7f98d44bf700 1 -- 172.21.15.144:0/950507749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc103ea0 msgr2=0x7f98cc1042b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:05.159 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.288+0000 7f98d44bf700 1 --2- 172.21.15.144:0/950507749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc103ea0 0x7f98cc1042b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f98c80098a0 tx=0x7f98c8009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:05.159 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.288+0000 7f98d44bf700 1 -- 172.21.15.144:0/950507749 shutdown_connections 2024-09-10T11:20:05.159 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.288+0000 7f98d44bf700 1 --2- 172.21.15.144:0/950507749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc103ea0 0x7f98cc1042b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:05.159 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.288+0000 7f98d44bf700 1 -- 172.21.15.144:0/950507749 >> 172.21.15.144:0/950507749 conn(0x7f98cc0ff880 msgr2=0x7f98cc101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:05.159 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.289+0000 7f98d44bf700 1 -- 172.21.15.144:0/950507749 shutdown_connections 2024-09-10T11:20:05.159 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.289+0000 7f98d44bf700 1 -- 172.21.15.144:0/950507749 wait complete. 2024-09-10T11:20:05.159 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.289+0000 7f98d44bf700 1 Processor -- start 2024-09-10T11:20:05.160 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.289+0000 7f98d44bf700 1 -- start start 2024-09-10T11:20:05.160 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.289+0000 7f98d44bf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc103ea0 0x7f98cc199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:05.160 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.289+0000 7f98d44bf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc199e40 con 0x7f98cc103ea0 2024-09-10T11:20:05.160 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.289+0000 7f98d225b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc103ea0 0x7f98cc199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:05.160 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.290+0000 7f98d225b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc103ea0 0x7f98cc199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43830/0 (socket says 172.21.15.144:43830) 2024-09-10T11:20:05.160 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.290+0000 7f98d225b700 1 -- 172.21.15.144:0/903682877 learned_addr learned my addr 172.21.15.144:0/903682877 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:05.160 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.290+0000 7f98d225b700 1 -- 172.21.15.144:0/903682877 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c8009550 con 0x7f98cc103ea0 2024-09-10T11:20:05.160 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.290+0000 7f98d225b700 1 --2- 172.21.15.144:0/903682877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc103ea0 0x7f98cc199900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f98c8006af0 tx=0x7f98c8011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:05.160 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.290+0000 7f98c37fe700 1 -- 172.21.15.144:0/903682877 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f98c8003b90 con 0x7f98cc103ea0 2024-09-10T11:20:05.160 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.290+0000 7f98c37fe700 1 -- 172.21.15.144:0/903682877 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f98c801a8a0 con 0x7f98cc103ea0 2024-09-10T11:20:05.160 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.290+0000 7f98c37fe700 1 -- 172.21.15.144:0/903682877 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f98c8019880 con 0x7f98cc103ea0 2024-09-10T11:20:05.160 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.290+0000 7f98d44bf700 1 -- 172.21.15.144:0/903682877 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc19a040 con 0x7f98cc103ea0 2024-09-10T11:20:05.161 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.290+0000 7f98d44bf700 1 -- 172.21.15.144:0/903682877 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc19a460 con 0x7f98cc103ea0 2024-09-10T11:20:05.161 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.291+0000 7f98c37fe700 1 -- 172.21.15.144:0/903682877 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f98c8019b40 con 0x7f98cc103ea0 2024-09-10T11:20:05.161 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.291+0000 7f98c37fe700 1 --2- 172.21.15.144:0/903682877 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f98b80383b0 0x7f98b803a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:05.161 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.291+0000 7f98c37fe700 1 -- 172.21.15.144:0/903682877 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f98c804c6d0 con 0x7f98cc103ea0 2024-09-10T11:20:05.161 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.291+0000 7f98d1a5a700 1 -- 172.21.15.144:0/903682877 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f98b80383b0 msgr2=0x7f98b803a860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:20:05.161 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.291+0000 7f98d1a5a700 1 --2- 172.21.15.144:0/903682877 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f98b80383b0 0x7f98b803a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T11:20:05.161 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.292+0000 7f98d44bf700 1 -- 172.21.15.144:0/903682877 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc062380 con 0x7f98cc103ea0 2024-09-10T11:20:05.161 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.296+0000 7f98c37fe700 1 -- 172.21.15.144:0/903682877 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f98c802c3a0 con 0x7f98cc103ea0 2024-09-10T11:20:05.161 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.492+0000 7f98d1a5a700 1 -- 172.21.15.144:0/903682877 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f98b80383b0 msgr2=0x7f98b803a860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:20:05.161 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.492+0000 7f98d1a5a700 1 --2- 172.21.15.144:0/903682877 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f98b80383b0 0x7f98b803a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T11:20:05.161 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.564+0000 7f98d44bf700 1 -- 172.21.15.144:0/903682877 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f98cc19cea0 con 0x7f98cc103ea0 2024-09-10T11:20:05.162 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.565+0000 7f98c37fe700 1 -- 172.21.15.144:0/903682877 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f98c8018350 con 0x7f98cc103ea0 2024-09-10T11:20:05.162 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.567+0000 7f98d44bf700 1 -- 172.21.15.144:0/903682877 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f98b80383b0 msgr2=0x7f98b803a860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T11:20:05.162 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.567+0000 7f98d44bf700 1 --2- 172.21.15.144:0/903682877 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f98b80383b0 0x7f98b803a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:05.162 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.567+0000 7f98d44bf700 1 -- 172.21.15.144:0/903682877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc103ea0 msgr2=0x7f98cc199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:05.162 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.567+0000 7f98d44bf700 1 --2- 172.21.15.144:0/903682877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc103ea0 0x7f98cc199900 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f98c8006af0 tx=0x7f98c8011a90 comp rx=0 tx=0).stop 2024-09-10T11:20:05.162 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.567+0000 7f98d44bf700 1 -- 172.21.15.144:0/903682877 shutdown_connections 2024-09-10T11:20:05.162 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.567+0000 7f98d44bf700 1 --2- 172.21.15.144:0/903682877 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f98b80383b0 0x7f98b803a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:05.162 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.567+0000 7f98d44bf700 1 --2- 172.21.15.144:0/903682877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98cc103ea0 0x7f98cc199900 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:05.162 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.567+0000 7f98d44bf700 1 -- 172.21.15.144:0/903682877 >> 172.21.15.144:0/903682877 conn(0x7f98cc0ff880 msgr2=0x7f98cc100f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:05.162 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.567+0000 7f98d44bf700 1 -- 172.21.15.144:0/903682877 shutdown_connections 2024-09-10T11:20:05.162 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:04.567+0000 7f98d44bf700 1 -- 172.21.15.144:0/903682877 wait complete. 2024-09-10T11:20:05.162 INFO:teuthology.orchestra.run.smithi144.stdout:Waiting for the mgr to restart... 2024-09-10T11:20:05.162 INFO:teuthology.orchestra.run.smithi144.stdout:Waiting for mgr epoch 5... 2024-09-10T11:20:05.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:05 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/903682877' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T11:20:10.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: Active manager daemon smithi144.bkogfs restarted 2024-09-10T11:20:10.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: Activating manager daemon smithi144.bkogfs 2024-09-10T11:20:10.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T11:20:10.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: mgrmap e6: smithi144.bkogfs(active, starting, since 0.00337415s) 2024-09-10T11:20:10.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:20:10.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr metadata", "who": "smithi144.bkogfs", "id": "smithi144.bkogfs"}]: dispatch 2024-09-10T11:20:10.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T11:20:10.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T11:20:10.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T11:20:10.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: Manager daemon smithi144.bkogfs is now available 2024-09-10T11:20:10.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:10.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:10.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:20:10.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:20:10.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:20:10.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/mirror_snapshot_schedule"}]: dispatch 2024-09-10T11:20:10.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:10 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/trash_purge_schedule"}]: dispatch 2024-09-10T11:20:11.274 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:11 smithi144 ceph-mon[27942]: Found migration_current of "None". Setting to last migration. 2024-09-10T11:20:11.274 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:11 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:11.274 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:11 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:11.274 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:11 smithi144 ceph-mon[27942]: mgrmap e7: smithi144.bkogfs(active, since 1.00525s) 2024-09-10T11:20:11.575 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout { 2024-09-10T11:20:11.575 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T11:20:11.576 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T11:20:11.576 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout } 2024-09-10T11:20:11.576 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.571+0000 7fbed3d97700 1 Processor -- start 2024-09-10T11:20:11.576 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.571+0000 7fbed3d97700 1 -- start start 2024-09-10T11:20:11.576 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.571+0000 7fbed3d97700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbecc104a80 0x7fbecc104e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:11.576 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.571+0000 7fbed3d97700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc1053d0 con 0x7fbecc104a80 2024-09-10T11:20:11.576 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.571+0000 7fbed1b33700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbecc104a80 0x7fbecc104e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:11.576 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.571+0000 7fbed1b33700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbecc104a80 0x7fbecc104e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43832/0 (socket says 172.21.15.144:43832) 2024-09-10T11:20:11.576 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.571+0000 7fbed1b33700 1 -- 172.21.15.144:0/3484379663 learned_addr learned my addr 172.21.15.144:0/3484379663 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:11.576 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.572+0000 7fbed1b33700 1 -- 172.21.15.144:0/3484379663 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbecc105510 con 0x7fbecc104a80 2024-09-10T11:20:11.577 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.572+0000 7fbed1b33700 1 --2- 172.21.15.144:0/3484379663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbecc104a80 0x7fbecc104e90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbec80098a0 tx=0x7fbec8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d0b6706fb41e5a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:11.577 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.572+0000 7fbed0b31700 1 -- 172.21.15.144:0/3484379663 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbec8004180 con 0x7fbecc104a80 2024-09-10T11:20:11.577 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.572+0000 7fbed0b31700 1 -- 172.21.15.144:0/3484379663 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbec80042e0 con 0x7fbecc104a80 2024-09-10T11:20:11.577 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.572+0000 7fbed0b31700 1 -- 172.21.15.144:0/3484379663 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbec8011a50 con 0x7fbecc104a80 2024-09-10T11:20:11.577 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.573+0000 7fbed3d97700 1 -- 172.21.15.144:0/3484379663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbecc104a80 msgr2=0x7fbecc104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:11.577 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.573+0000 7fbed3d97700 1 --2- 172.21.15.144:0/3484379663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbecc104a80 0x7fbecc104e90 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fbec80098a0 tx=0x7fbec8009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:11.577 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.573+0000 7fbed3d97700 1 -- 172.21.15.144:0/3484379663 shutdown_connections 2024-09-10T11:20:11.577 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.573+0000 7fbed3d97700 1 --2- 172.21.15.144:0/3484379663 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbecc104a80 0x7fbecc104e90 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:11.577 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.573+0000 7fbed3d97700 1 -- 172.21.15.144:0/3484379663 >> 172.21.15.144:0/3484379663 conn(0x7fbecc100110 msgr2=0x7fbecc102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:11.577 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.573+0000 7fbed3d97700 1 -- 172.21.15.144:0/3484379663 shutdown_connections 2024-09-10T11:20:11.577 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.573+0000 7fbed3d97700 1 -- 172.21.15.144:0/3484379663 wait complete. 2024-09-10T11:20:11.577 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.574+0000 7fbed3d97700 1 Processor -- start 2024-09-10T11:20:11.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.574+0000 7fbed3d97700 1 -- start start 2024-09-10T11:20:11.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.574+0000 7fbed3d97700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbecc104a80 0x7fbecc1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:11.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.574+0000 7fbed3d97700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbecc198d00 con 0x7fbecc104a80 2024-09-10T11:20:11.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.574+0000 7fbed1b33700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbecc104a80 0x7fbecc1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:11.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.574+0000 7fbed1b33700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbecc104a80 0x7fbecc1987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43836/0 (socket says 172.21.15.144:43836) 2024-09-10T11:20:11.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.574+0000 7fbed1b33700 1 -- 172.21.15.144:0/1724895466 learned_addr learned my addr 172.21.15.144:0/1724895466 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:11.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.574+0000 7fbed1b33700 1 -- 172.21.15.144:0/1724895466 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec8009550 con 0x7fbecc104a80 2024-09-10T11:20:11.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.575+0000 7fbed1b33700 1 --2- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbecc104a80 0x7fbecc1987c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbec80119e0 tx=0x7fbec8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:11.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.575+0000 7fbec2ffd700 1 -- 172.21.15.144:0/1724895466 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbec8003b90 con 0x7fbecc104a80 2024-09-10T11:20:11.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.575+0000 7fbed3d97700 1 -- 172.21.15.144:0/1724895466 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbecc198f00 con 0x7fbecc104a80 2024-09-10T11:20:11.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.575+0000 7fbec2ffd700 1 -- 172.21.15.144:0/1724895466 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbec801a8a0 con 0x7fbecc104a80 2024-09-10T11:20:11.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.575+0000 7fbec2ffd700 1 -- 172.21.15.144:0/1724895466 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbec8019930 con 0x7fbecc104a80 2024-09-10T11:20:11.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.575+0000 7fbed3d97700 1 -- 172.21.15.144:0/1724895466 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbecc199320 con 0x7fbecc104a80 2024-09-10T11:20:11.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.576+0000 7fbec2ffd700 1 -- 172.21.15.144:0/1724895466 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fbec8019ba0 con 0x7fbecc104a80 2024-09-10T11:20:11.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.576+0000 7fbec2ffd700 1 --2- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 0x7fbeb803a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:11.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.576+0000 7fbec2ffd700 1 -- 172.21.15.144:0/1724895466 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbeb803afc0 con 0x7fbeb8038400 2024-09-10T11:20:11.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.576+0000 7fbec2ffd700 1 -- 172.21.15.144:0/1724895466 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbec804c890 con 0x7fbecc104a80 2024-09-10T11:20:11.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.576+0000 7fbed1332700 1 -- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 msgr2=0x7fbeb803a8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:20:11.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.576+0000 7fbed1332700 1 --2- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 0x7fbeb803a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T11:20:11.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.777+0000 7fbed1332700 1 -- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 msgr2=0x7fbeb803a8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:20:11.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:05.777+0000 7fbed1332700 1 --2- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 0x7fbeb803a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T11:20:11.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:06.178+0000 7fbed1332700 1 -- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 msgr2=0x7fbeb803a8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:20:11.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:06.178+0000 7fbed1332700 1 --2- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 0x7fbeb803a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T11:20:11.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:06.979+0000 7fbed1332700 1 -- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 msgr2=0x7fbeb803a8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:20:11.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:06.979+0000 7fbed1332700 1 --2- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 0x7fbeb803a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T11:20:11.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:08.580+0000 7fbed1332700 1 -- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 msgr2=0x7fbeb803a8b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:20:11.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:08.580+0000 7fbed1332700 1 --2- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 0x7fbeb803a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-10T11:20:11.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:09.968+0000 7fbec2ffd700 1 -- 172.21.15.144:0/1724895466 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mgrmap(e 6) v1 ==== 44851+0+0 (secure 0 0 0) 0x7fbec801aa10 con 0x7fbecc104a80 2024-09-10T11:20:11.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:09.968+0000 7fbec2ffd700 1 -- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 msgr2=0x7fbeb803a8b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T11:20:11.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:09.968+0000 7fbec2ffd700 1 --2- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 0x7fbeb803a8b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:11.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.969+0000 7fbec2ffd700 1 -- 172.21.15.144:0/1724895466 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mgrmap(e 7) v1 ==== 44978+0+0 (secure 0 0 0) 0x7fbec802d620 con 0x7fbecc104a80 2024-09-10T11:20:11.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.969+0000 7fbec2ffd700 1 --2- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 0x7fbeb803a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:11.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.969+0000 7fbec2ffd700 1 -- 172.21.15.144:0/1724895466 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbeb803afc0 con 0x7fbeb8038400 2024-09-10T11:20:11.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.969+0000 7fbed1332700 1 --2- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 0x7fbeb803a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:11.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.970+0000 7fbed1332700 1 --2- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 0x7fbeb803a8b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbebc003a10 tx=0x7fbebc0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:11.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.970+0000 7fbec2ffd700 1 -- 172.21.15.144:0/1724895466 <== mgr.14120 v2:172.21.15.144:6800/2 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fbeb803afc0 con 0x7fbeb8038400 2024-09-10T11:20:11.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.977+0000 7fbed3d97700 1 -- 172.21.15.144:0/1724895466 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fbecc199b00 con 0x7fbeb8038400 2024-09-10T11:20:11.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.977+0000 7fbec2ffd700 1 -- 172.21.15.144:0/1724895466 <== mgr.14120 v2:172.21.15.144:6800/2 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fbecc199b00 con 0x7fbeb8038400 2024-09-10T11:20:11.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.978+0000 7fbed3d97700 1 -- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 msgr2=0x7fbeb803a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:11.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.978+0000 7fbed3d97700 1 --2- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 0x7fbeb803a8b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fbebc003a10 tx=0x7fbebc0092b0 comp rx=0 tx=0).stop 2024-09-10T11:20:11.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.978+0000 7fbed3d97700 1 -- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbecc104a80 msgr2=0x7fbecc1987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:11.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.978+0000 7fbed3d97700 1 --2- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbecc104a80 0x7fbecc1987c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbec80119e0 tx=0x7fbec8011ac0 comp rx=0 tx=0).stop 2024-09-10T11:20:11.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.978+0000 7fbed3d97700 1 -- 172.21.15.144:0/1724895466 shutdown_connections 2024-09-10T11:20:11.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.978+0000 7fbed3d97700 1 --2- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbeb8038400 0x7fbeb803a8b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:11.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.978+0000 7fbed3d97700 1 --2- 172.21.15.144:0/1724895466 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbecc104a80 0x7fbecc1987c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:11.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.978+0000 7fbed3d97700 1 -- 172.21.15.144:0/1724895466 >> 172.21.15.144:0/1724895466 conn(0x7fbecc100110 msgr2=0x7fbecc100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:11.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.978+0000 7fbed3d97700 1 -- 172.21.15.144:0/1724895466 shutdown_connections 2024-09-10T11:20:11.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:10.978+0000 7fbed3d97700 1 -- 172.21.15.144:0/1724895466 wait complete. 2024-09-10T11:20:11.581 INFO:teuthology.orchestra.run.smithi144.stdout:mgr epoch 5 is available 2024-09-10T11:20:11.581 INFO:teuthology.orchestra.run.smithi144.stdout:Setting orchestrator backend to cephadm... 2024-09-10T11:20:12.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:12 smithi144 ceph-mon[27942]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T11:20:12.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:12 smithi144 ceph-mon[27942]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T11:20:12.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:12 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:20:12.913 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.003+0000 7f7175887700 1 Processor -- start 2024-09-10T11:20:12.914 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.003+0000 7f7175887700 1 -- start start 2024-09-10T11:20:12.914 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.003+0000 7f7175887700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7170104a80 0x7f7170104e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:12.914 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.003+0000 7f7175887700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71701053d0 con 0x7f7170104a80 2024-09-10T11:20:12.914 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.003+0000 7f716effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7170104a80 0x7f7170104e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:12.914 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.003+0000 7f716effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7170104a80 0x7f7170104e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43880/0 (socket says 172.21.15.144:43880) 2024-09-10T11:20:12.914 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.003+0000 7f716effd700 1 -- 172.21.15.144:0/1810653875 learned_addr learned my addr 172.21.15.144:0/1810653875 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:12.914 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.003+0000 7f716effd700 1 -- 172.21.15.144:0/1810653875 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7170105510 con 0x7f7170104a80 2024-09-10T11:20:12.914 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.003+0000 7f716effd700 1 --2- 172.21.15.144:0/1810653875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7170104a80 0x7f7170104e90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f71600098a0 tx=0x7f7160009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e48703e52888dd server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:12.914 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.003+0000 7f716dffb700 1 -- 172.21.15.144:0/1810653875 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7160004180 con 0x7f7170104a80 2024-09-10T11:20:12.914 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.004+0000 7f716dffb700 1 -- 172.21.15.144:0/1810653875 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f71600042e0 con 0x7f7170104a80 2024-09-10T11:20:12.914 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.004+0000 7f716dffb700 1 -- 172.21.15.144:0/1810653875 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7160011a50 con 0x7f7170104a80 2024-09-10T11:20:12.914 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.004+0000 7f7175887700 1 -- 172.21.15.144:0/1810653875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7170104a80 msgr2=0x7f7170104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:12.915 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.004+0000 7f7175887700 1 --2- 172.21.15.144:0/1810653875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7170104a80 0x7f7170104e90 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f71600098a0 tx=0x7f7160009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:12.915 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.004+0000 7f7175887700 1 -- 172.21.15.144:0/1810653875 shutdown_connections 2024-09-10T11:20:12.915 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.004+0000 7f7175887700 1 --2- 172.21.15.144:0/1810653875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7170104a80 0x7f7170104e90 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:12.915 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.004+0000 7f7175887700 1 -- 172.21.15.144:0/1810653875 >> 172.21.15.144:0/1810653875 conn(0x7f71701000f0 msgr2=0x7f7170102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:12.915 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.004+0000 7f7175887700 1 -- 172.21.15.144:0/1810653875 shutdown_connections 2024-09-10T11:20:12.915 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.004+0000 7f7175887700 1 -- 172.21.15.144:0/1810653875 wait complete. 2024-09-10T11:20:12.915 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.005+0000 7f7175887700 1 Processor -- start 2024-09-10T11:20:12.915 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.005+0000 7f7175887700 1 -- start start 2024-09-10T11:20:12.915 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.005+0000 7f7175887700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7170104a80 0x7f71701998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:12.915 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.005+0000 7f7175887700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7170199e10 con 0x7f7170104a80 2024-09-10T11:20:12.915 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.005+0000 7f716effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7170104a80 0x7f71701998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:12.915 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.005+0000 7f716effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7170104a80 0x7f71701998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43894/0 (socket says 172.21.15.144:43894) 2024-09-10T11:20:12.915 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.005+0000 7f716effd700 1 -- 172.21.15.144:0/1214867577 learned_addr learned my addr 172.21.15.144:0/1214867577 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:12.915 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.005+0000 7f716effd700 1 -- 172.21.15.144:0/1214867577 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7160009550 con 0x7f7170104a80 2024-09-10T11:20:12.916 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.005+0000 7f716effd700 1 --2- 172.21.15.144:0/1214867577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7170104a80 0x7f71701998d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f71600119e0 tx=0x7f7160011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:12.916 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.006+0000 7f7174885700 1 -- 172.21.15.144:0/1214867577 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7160003b90 con 0x7f7170104a80 2024-09-10T11:20:12.916 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.006+0000 7f7174885700 1 -- 172.21.15.144:0/1214867577 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f716001a8a0 con 0x7f7170104a80 2024-09-10T11:20:12.916 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.006+0000 7f7175887700 1 -- 172.21.15.144:0/1214867577 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f717019a010 con 0x7f7170104a80 2024-09-10T11:20:12.916 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.006+0000 7f7174885700 1 -- 172.21.15.144:0/1214867577 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7160019930 con 0x7f7170104a80 2024-09-10T11:20:12.916 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.006+0000 7f7175887700 1 -- 172.21.15.144:0/1214867577 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717019a430 con 0x7f7170104a80 2024-09-10T11:20:12.916 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.006+0000 7f7174885700 1 -- 172.21.15.144:0/1214867577 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 7) v1 ==== 44978+0+0 (secure 0 0 0) 0x7f7160019bf0 con 0x7f7170104a80 2024-09-10T11:20:12.916 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.006+0000 7f7175887700 1 -- 172.21.15.144:0/1214867577 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7170108f10 con 0x7f7170104a80 2024-09-10T11:20:12.916 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.007+0000 7f7174885700 1 --2- 172.21.15.144:0/1214867577 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7158038290 0x7f715803a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:12.916 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.007+0000 7f7174885700 1 -- 172.21.15.144:0/1214867577 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f716004c750 con 0x7f7170104a80 2024-09-10T11:20:12.916 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.007+0000 7f716e7fc700 1 --2- 172.21.15.144:0/1214867577 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7158038290 0x7f715803a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:12.916 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.007+0000 7f716e7fc700 1 --2- 172.21.15.144:0/1214867577 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7158038290 0x7f715803a740 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7164006fd0 tx=0x7f7164006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:12.917 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.012+0000 7f7174885700 1 -- 172.21.15.144:0/1214867577 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f716002d440 con 0x7f7170104a80 2024-09-10T11:20:12.917 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.022+0000 7f7174885700 1 -- 172.21.15.144:0/1214867577 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f716002c3a0 con 0x7f7170104a80 2024-09-10T11:20:12.917 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.215+0000 7f7175887700 1 -- 172.21.15.144:0/1214867577 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f7170102290 con 0x7f7158038290 2024-09-10T11:20:12.917 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.223+0000 7f7174885700 1 -- 172.21.15.144:0/1214867577 <== mgr.14120 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f7170102290 con 0x7f7158038290 2024-09-10T11:20:12.917 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.225+0000 7f7175887700 1 -- 172.21.15.144:0/1214867577 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7158038290 msgr2=0x7f715803a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:12.917 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.225+0000 7f7175887700 1 --2- 172.21.15.144:0/1214867577 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7158038290 0x7f715803a740 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7164006fd0 tx=0x7f7164006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:12.917 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.225+0000 7f7175887700 1 -- 172.21.15.144:0/1214867577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7170104a80 msgr2=0x7f71701998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:12.917 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.225+0000 7f7175887700 1 --2- 172.21.15.144:0/1214867577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7170104a80 0x7f71701998d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f71600119e0 tx=0x7f7160011ac0 comp rx=0 tx=0).stop 2024-09-10T11:20:12.917 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.225+0000 7f7175887700 1 -- 172.21.15.144:0/1214867577 shutdown_connections 2024-09-10T11:20:12.917 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.225+0000 7f7175887700 1 --2- 172.21.15.144:0/1214867577 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7158038290 0x7f715803a740 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:12.917 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.225+0000 7f7175887700 1 --2- 172.21.15.144:0/1214867577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7170104a80 0x7f71701998d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:12.917 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.225+0000 7f7175887700 1 -- 172.21.15.144:0/1214867577 >> 172.21.15.144:0/1214867577 conn(0x7f71701000f0 msgr2=0x7f7170079940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:12.917 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.225+0000 7f7175887700 1 -- 172.21.15.144:0/1214867577 shutdown_connections 2024-09-10T11:20:12.918 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:12.225+0000 7f7175887700 1 -- 172.21.15.144:0/1214867577 wait complete. 2024-09-10T11:20:13.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:13 smithi144 ceph-mon[27942]: [10/Sep/2024:11:20:11] ENGINE Bus STARTING 2024-09-10T11:20:13.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:13 smithi144 ceph-mon[27942]: [10/Sep/2024:11:20:11] ENGINE Serving on https://172.21.15.144:7150 2024-09-10T11:20:13.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:13 smithi144 ceph-mon[27942]: [10/Sep/2024:11:20:11] ENGINE Serving on http://172.21.15.144:8765 2024-09-10T11:20:13.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:13 smithi144 ceph-mon[27942]: [10/Sep/2024:11:20:11] ENGINE Bus STARTED 2024-09-10T11:20:13.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:13 smithi144 ceph-mon[27942]: mgrmap e8: smithi144.bkogfs(active, since 2s) 2024-09-10T11:20:13.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:13 smithi144 ceph-mon[27942]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:20:13.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:13 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:13.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:13 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:20:14.102 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T11:20:14.102 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.324+0000 7f9a4ca40700 1 Processor -- start 2024-09-10T11:20:14.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.324+0000 7f9a4ca40700 1 -- start start 2024-09-10T11:20:14.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.325+0000 7f9a4ca40700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a48104fb0 0x7f9a481053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:14.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.325+0000 7f9a4ca40700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a48105900 con 0x7f9a48104fb0 2024-09-10T11:20:14.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.325+0000 7f9a4659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a48104fb0 0x7f9a481053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:14.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.325+0000 7f9a4659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a48104fb0 0x7f9a481053c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51248/0 (socket says 172.21.15.144:51248) 2024-09-10T11:20:14.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.325+0000 7f9a4659c700 1 -- 172.21.15.144:0/701517846 learned_addr learned my addr 172.21.15.144:0/701517846 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:14.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.325+0000 7f9a4659c700 1 -- 172.21.15.144:0/701517846 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a48105a40 con 0x7f9a48104fb0 2024-09-10T11:20:14.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.325+0000 7f9a4659c700 1 --2- 172.21.15.144:0/701517846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a48104fb0 0x7f9a481053c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009e80 tx=0x7f9a4000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1894fd83d8cc2f06 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:14.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.325+0000 7f9a4559a700 1 -- 172.21.15.144:0/701517846 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a4000e070 con 0x7f9a48104fb0 2024-09-10T11:20:14.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.325+0000 7f9a4559a700 1 -- 172.21.15.144:0/701517846 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a40003e90 con 0x7f9a48104fb0 2024-09-10T11:20:14.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.325+0000 7f9a4559a700 1 -- 172.21.15.144:0/701517846 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a40004380 con 0x7f9a48104fb0 2024-09-10T11:20:14.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.326+0000 7f9a4ca40700 1 -- 172.21.15.144:0/701517846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a48104fb0 msgr2=0x7f9a481053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:14.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.326+0000 7f9a4ca40700 1 --2- 172.21.15.144:0/701517846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a48104fb0 0x7f9a481053c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009e80 tx=0x7f9a4000b330 comp rx=0 tx=0).stop 2024-09-10T11:20:14.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.326+0000 7f9a4ca40700 1 -- 172.21.15.144:0/701517846 shutdown_connections 2024-09-10T11:20:14.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.326+0000 7f9a4ca40700 1 --2- 172.21.15.144:0/701517846 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a48104fb0 0x7f9a481053c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:14.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.326+0000 7f9a4ca40700 1 -- 172.21.15.144:0/701517846 >> 172.21.15.144:0/701517846 conn(0x7f9a480ff880 msgr2=0x7f9a48101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:14.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.326+0000 7f9a4ca40700 1 -- 172.21.15.144:0/701517846 shutdown_connections 2024-09-10T11:20:14.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.326+0000 7f9a4ca40700 1 -- 172.21.15.144:0/701517846 wait complete. 2024-09-10T11:20:14.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.327+0000 7f9a4ca40700 1 Processor -- start 2024-09-10T11:20:14.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.327+0000 7f9a4ca40700 1 -- start start 2024-09-10T11:20:14.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.327+0000 7f9a4ca40700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a48104fb0 0x7f9a48197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:14.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.327+0000 7f9a4ca40700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a48197c50 con 0x7f9a48104fb0 2024-09-10T11:20:14.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.327+0000 7f9a4659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a48104fb0 0x7f9a48197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:14.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.327+0000 7f9a4659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a48104fb0 0x7f9a48197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51250/0 (socket says 172.21.15.144:51250) 2024-09-10T11:20:14.104 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.327+0000 7f9a4659c700 1 -- 172.21.15.144:0/1114036710 learned_addr learned my addr 172.21.15.144:0/1114036710 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:14.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.327+0000 7f9a4659c700 1 -- 172.21.15.144:0/1114036710 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a40009b30 con 0x7f9a48104fb0 2024-09-10T11:20:14.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.328+0000 7f9a4659c700 1 --2- 172.21.15.144:0/1114036710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a48104fb0 0x7f9a48197710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000b7b0 tx=0x7f9a40003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:14.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.328+0000 7f9a3f7fe700 1 -- 172.21.15.144:0/1114036710 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a4000e040 con 0x7f9a48104fb0 2024-09-10T11:20:14.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.328+0000 7f9a4ca40700 1 -- 172.21.15.144:0/1114036710 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a48197e50 con 0x7f9a48104fb0 2024-09-10T11:20:14.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.328+0000 7f9a3f7fe700 1 -- 172.21.15.144:0/1114036710 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f9a400048a0 con 0x7f9a48104fb0 2024-09-10T11:20:14.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.328+0000 7f9a3f7fe700 1 -- 172.21.15.144:0/1114036710 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9a4001b630 con 0x7f9a48104fb0 2024-09-10T11:20:14.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.328+0000 7f9a4ca40700 1 -- 172.21.15.144:0/1114036710 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a481982d0 con 0x7f9a48104fb0 2024-09-10T11:20:14.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.329+0000 7f9a3f7fe700 1 -- 172.21.15.144:0/1114036710 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f9a40019040 con 0x7f9a48104fb0 2024-09-10T11:20:14.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.329+0000 7f9a4ca40700 1 -- 172.21.15.144:0/1114036710 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a48191400 con 0x7f9a48104fb0 2024-09-10T11:20:14.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.329+0000 7f9a3f7fe700 1 --2- 172.21.15.144:0/1114036710 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9a3403a5b0 0x7f9a3403ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:14.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.329+0000 7f9a3f7fe700 1 -- 172.21.15.144:0/1114036710 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9a4004bde0 con 0x7f9a48104fb0 2024-09-10T11:20:14.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.329+0000 7f9a45d9b700 1 --2- 172.21.15.144:0/1114036710 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9a3403a5b0 0x7f9a3403ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:14.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.329+0000 7f9a45d9b700 1 --2- 172.21.15.144:0/1114036710 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9a3403a5b0 0x7f9a3403ca60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a30006fd0 tx=0x7f9a30006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:14.105 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.334+0000 7f9a3f7fe700 1 -- 172.21.15.144:0/1114036710 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9a400117a0 con 0x7f9a48104fb0 2024-09-10T11:20:14.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.526+0000 7f9a4ca40700 1 -- 172.21.15.144:0/1114036710 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a48061190 con 0x7f9a3403a5b0 2024-09-10T11:20:14.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.527+0000 7f9a3f7fe700 1 -- 172.21.15.144:0/1114036710 <== mgr.14120 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f9a48061190 con 0x7f9a3403a5b0 2024-09-10T11:20:14.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.529+0000 7f9a4ca40700 1 -- 172.21.15.144:0/1114036710 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9a3403a5b0 msgr2=0x7f9a3403ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:14.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.529+0000 7f9a4ca40700 1 --2- 172.21.15.144:0/1114036710 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9a3403a5b0 0x7f9a3403ca60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9a30006fd0 tx=0x7f9a30006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:14.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.529+0000 7f9a4ca40700 1 -- 172.21.15.144:0/1114036710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a48104fb0 msgr2=0x7f9a48197710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:14.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.529+0000 7f9a4ca40700 1 --2- 172.21.15.144:0/1114036710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a48104fb0 0x7f9a48197710 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9a4000b7b0 tx=0x7f9a40003d00 comp rx=0 tx=0).stop 2024-09-10T11:20:14.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.529+0000 7f9a4ca40700 1 -- 172.21.15.144:0/1114036710 shutdown_connections 2024-09-10T11:20:14.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.529+0000 7f9a4ca40700 1 --2- 172.21.15.144:0/1114036710 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9a3403a5b0 0x7f9a3403ca60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:14.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.529+0000 7f9a4ca40700 1 --2- 172.21.15.144:0/1114036710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a48104fb0 0x7f9a48197710 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:14.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.529+0000 7f9a4ca40700 1 -- 172.21.15.144:0/1114036710 >> 172.21.15.144:0/1114036710 conn(0x7f9a480ff880 msgr2=0x7f9a4818e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:14.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.529+0000 7f9a4ca40700 1 -- 172.21.15.144:0/1114036710 shutdown_connections 2024-09-10T11:20:14.106 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:13.529+0000 7f9a4ca40700 1 -- 172.21.15.144:0/1114036710 wait complete. 2024-09-10T11:20:14.106 INFO:teuthology.orchestra.run.smithi144.stdout:Generating ssh key... 2024-09-10T11:20:14.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:14 smithi144 ceph-mon[27942]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:20:15.467 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.487+0000 7f8db4699700 1 Processor -- start 2024-09-10T11:20:15.467 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.487+0000 7f8db4699700 1 -- start start 2024-09-10T11:20:15.467 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.488+0000 7f8db4699700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dac1066f0 0x7f8dac108ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:15.468 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.488+0000 7f8db4699700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac079d40 con 0x7f8dac1066f0 2024-09-10T11:20:15.468 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.488+0000 7f8db2435700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dac1066f0 0x7f8dac108ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:15.468 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.488+0000 7f8db2435700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dac1066f0 0x7f8dac108ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51254/0 (socket says 172.21.15.144:51254) 2024-09-10T11:20:15.468 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.488+0000 7f8db2435700 1 -- 172.21.15.144:0/3003885959 learned_addr learned my addr 172.21.15.144:0/3003885959 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:15.468 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.488+0000 7f8db2435700 1 -- 172.21.15.144:0/3003885959 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac079e80 con 0x7f8dac1066f0 2024-09-10T11:20:15.468 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.488+0000 7f8db2435700 1 --2- 172.21.15.144:0/3003885959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dac1066f0 0x7f8dac108ad0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8da80098a0 tx=0x7f8da8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7e0e2a45afecceb server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:15.468 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.488+0000 7f8db1433700 1 -- 172.21.15.144:0/3003885959 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8da8004180 con 0x7f8dac1066f0 2024-09-10T11:20:15.468 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.488+0000 7f8db1433700 1 -- 172.21.15.144:0/3003885959 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8da80042e0 con 0x7f8dac1066f0 2024-09-10T11:20:15.468 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.488+0000 7f8db1433700 1 -- 172.21.15.144:0/3003885959 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8da8011a50 con 0x7f8dac1066f0 2024-09-10T11:20:15.468 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.489+0000 7f8db4699700 1 -- 172.21.15.144:0/3003885959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dac1066f0 msgr2=0x7f8dac108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:15.468 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.489+0000 7f8db4699700 1 --2- 172.21.15.144:0/3003885959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dac1066f0 0x7f8dac108ad0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8da80098a0 tx=0x7f8da8009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:15.468 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.489+0000 7f8db4699700 1 -- 172.21.15.144:0/3003885959 shutdown_connections 2024-09-10T11:20:15.468 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.489+0000 7f8db4699700 1 --2- 172.21.15.144:0/3003885959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dac1066f0 0x7f8dac108ad0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:15.469 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.489+0000 7f8db4699700 1 -- 172.21.15.144:0/3003885959 >> 172.21.15.144:0/3003885959 conn(0x7f8dac1000f0 msgr2=0x7f8dac102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:15.469 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.489+0000 7f8db4699700 1 -- 172.21.15.144:0/3003885959 shutdown_connections 2024-09-10T11:20:15.469 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.489+0000 7f8db4699700 1 -- 172.21.15.144:0/3003885959 wait complete. 2024-09-10T11:20:15.469 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.490+0000 7f8db4699700 1 Processor -- start 2024-09-10T11:20:15.469 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.490+0000 7f8db4699700 1 -- start start 2024-09-10T11:20:15.469 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.490+0000 7f8db4699700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dac1066f0 0x7f8dac1998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:15.469 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.490+0000 7f8db4699700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dac199e20 con 0x7f8dac1066f0 2024-09-10T11:20:15.469 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.490+0000 7f8db2435700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dac1066f0 0x7f8dac1998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:15.469 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.490+0000 7f8db2435700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dac1066f0 0x7f8dac1998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51266/0 (socket says 172.21.15.144:51266) 2024-09-10T11:20:15.469 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.490+0000 7f8db2435700 1 -- 172.21.15.144:0/3431880994 learned_addr learned my addr 172.21.15.144:0/3431880994 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:15.469 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.490+0000 7f8db2435700 1 -- 172.21.15.144:0/3431880994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8da8009550 con 0x7f8dac1066f0 2024-09-10T11:20:15.469 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.490+0000 7f8db2435700 1 --2- 172.21.15.144:0/3431880994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dac1066f0 0x7f8dac1998e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8da8000c00 tx=0x7f8da8011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:15.469 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.490+0000 7f8d9f7fe700 1 -- 172.21.15.144:0/3431880994 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8da8003b90 con 0x7f8dac1066f0 2024-09-10T11:20:15.469 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.490+0000 7f8d9f7fe700 1 -- 172.21.15.144:0/3431880994 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f8da80248b0 con 0x7f8dac1066f0 2024-09-10T11:20:15.470 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.490+0000 7f8d9f7fe700 1 -- 172.21.15.144:0/3431880994 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8da801a8e0 con 0x7f8dac1066f0 2024-09-10T11:20:15.470 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.490+0000 7f8db4699700 1 -- 172.21.15.144:0/3431880994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dac19a020 con 0x7f8dac1066f0 2024-09-10T11:20:15.470 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.490+0000 7f8db4699700 1 -- 172.21.15.144:0/3431880994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dac19a440 con 0x7f8dac1066f0 2024-09-10T11:20:15.470 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.491+0000 7f8d9f7fe700 1 -- 172.21.15.144:0/3431880994 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f8da801ab50 con 0x7f8dac1066f0 2024-09-10T11:20:15.470 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.491+0000 7f8db4699700 1 -- 172.21.15.144:0/3431880994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dac062380 con 0x7f8dac1066f0 2024-09-10T11:20:15.470 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.491+0000 7f8d9f7fe700 1 --2- 172.21.15.144:0/3431880994 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f8d98038440 0x7f8d9803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:15.470 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.491+0000 7f8d9f7fe700 1 -- 172.21.15.144:0/3431880994 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8da804c5b0 con 0x7f8dac1066f0 2024-09-10T11:20:15.470 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.492+0000 7f8db1c34700 1 --2- 172.21.15.144:0/3431880994 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f8d98038440 0x7f8d9803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:15.470 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.492+0000 7f8db1c34700 1 --2- 172.21.15.144:0/3431880994 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f8d98038440 0x7f8d9803a8f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8da0006fd0 tx=0x7f8da0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:15.470 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.496+0000 7f8d9f7fe700 1 -- 172.21.15.144:0/3431880994 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8da8018ce0 con 0x7f8dac1066f0 2024-09-10T11:20:15.470 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.683+0000 7f8db4699700 1 -- 172.21.15.144:0/3431880994 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dac1023b0 con 0x7f8d98038440 2024-09-10T11:20:15.470 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.861+0000 7f8d9f7fe700 1 -- 172.21.15.144:0/3431880994 <== mgr.14120 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f8dac1023b0 con 0x7f8d98038440 2024-09-10T11:20:15.470 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.863+0000 7f8db4699700 1 -- 172.21.15.144:0/3431880994 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f8d98038440 msgr2=0x7f8d9803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:15.471 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.863+0000 7f8db4699700 1 --2- 172.21.15.144:0/3431880994 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f8d98038440 0x7f8d9803a8f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8da0006fd0 tx=0x7f8da0006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:15.471 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.863+0000 7f8db4699700 1 -- 172.21.15.144:0/3431880994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dac1066f0 msgr2=0x7f8dac1998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:15.471 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.863+0000 7f8db4699700 1 --2- 172.21.15.144:0/3431880994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dac1066f0 0x7f8dac1998e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f8da8000c00 tx=0x7f8da8011ab0 comp rx=0 tx=0).stop 2024-09-10T11:20:15.471 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.863+0000 7f8db4699700 1 -- 172.21.15.144:0/3431880994 shutdown_connections 2024-09-10T11:20:15.471 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.863+0000 7f8db4699700 1 --2- 172.21.15.144:0/3431880994 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f8d98038440 0x7f8d9803a8f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:15.471 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.863+0000 7f8db4699700 1 --2- 172.21.15.144:0/3431880994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8dac1066f0 0x7f8dac1998e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:15.471 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.863+0000 7f8db4699700 1 -- 172.21.15.144:0/3431880994 >> 172.21.15.144:0/3431880994 conn(0x7f8dac1000f0 msgr2=0x7f8dac101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:15.471 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.864+0000 7f8db4699700 1 -- 172.21.15.144:0/3431880994 shutdown_connections 2024-09-10T11:20:15.471 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:14.864+0000 7f8db4699700 1 -- 172.21.15.144:0/3431880994 wait complete. 2024-09-10T11:20:16.065 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:15 smithi144 ceph-mon[27942]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:20:16.065 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:15 smithi144 ceph-mon[27942]: Generating ssh key... 2024-09-10T11:20:16.065 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:15 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:16.065 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:15 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:16.708 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjP9k8BoC0j2Y3iNix001ZW55e0M9N0D5Wy9/1ttxWhPw175KajJlJ217PQXxstMC544bUEXwW8sRnnQShP7VIRP8UsEVwV5FgGfVhNx6tXHDpR85fUDHB0zoo5yzkRJLKhSFBrcvEWWjOYFl+7mPLngZNrPYmc+h8/W+vH5tB1/UmuqnS5cB/nw32QDcHMi9+EN8pnSuU4+umMUkpXTpBhZ8SmU8nirXZHrvnzl9C3Q2IbPx/RrBYHMiF8LPABe5H0uu7qCLNBaZaQQSFO/4A+/diUy3ud+yZx0rtOpleb1ZWN5KcOL29ILfpXo9royhv+sq1QLLB8Vao+V1hoow5jCL3HpWtM/BXKWqixlF6c5Ls/ZY5SayH+mKcSEbWnS32vMnKEYMY3pfVGgWE9FMHIMbSi6ArehHlg0fZyD43YIIDo8Ndpj5aQrrcfia8GwiMSC4xC7phHSUexjDyz7EvYifCdbeWkQaTLTsRj8J0juxK7NVnsFuoQJ25ydRTHx0= ceph-7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:20:16.708 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.862+0000 7f327ef74700 1 Processor -- start 2024-09-10T11:20:16.708 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.862+0000 7f327ef74700 1 -- start start 2024-09-10T11:20:16.708 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.862+0000 7f327ef74700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3278104a80 0x7f3278104e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:16.708 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.862+0000 7f327ef74700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32781053d0 con 0x7f3278104a80 2024-09-10T11:20:16.709 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.862+0000 7f327cd10700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3278104a80 0x7f3278104e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:16.709 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.862+0000 7f327cd10700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3278104a80 0x7f3278104e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51282/0 (socket says 172.21.15.144:51282) 2024-09-10T11:20:16.709 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.862+0000 7f327cd10700 1 -- 172.21.15.144:0/3021318369 learned_addr learned my addr 172.21.15.144:0/3021318369 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:16.709 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.862+0000 7f327cd10700 1 -- 172.21.15.144:0/3021318369 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3278105510 con 0x7f3278104a80 2024-09-10T11:20:16.709 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.862+0000 7f327cd10700 1 --2- 172.21.15.144:0/3021318369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3278104a80 0x7f3278104e90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f326c0098a0 tx=0x7f326c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa33d592d97fe42 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:16.709 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.862+0000 7f32777fe700 1 -- 172.21.15.144:0/3021318369 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f326c004180 con 0x7f3278104a80 2024-09-10T11:20:16.709 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.863+0000 7f32777fe700 1 -- 172.21.15.144:0/3021318369 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f326c0042e0 con 0x7f3278104a80 2024-09-10T11:20:16.709 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.863+0000 7f32777fe700 1 -- 172.21.15.144:0/3021318369 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f326c011a50 con 0x7f3278104a80 2024-09-10T11:20:16.709 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.863+0000 7f327ef74700 1 -- 172.21.15.144:0/3021318369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3278104a80 msgr2=0x7f3278104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:16.709 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.863+0000 7f327ef74700 1 --2- 172.21.15.144:0/3021318369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3278104a80 0x7f3278104e90 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f326c0098a0 tx=0x7f326c009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:16.709 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.863+0000 7f327ef74700 1 -- 172.21.15.144:0/3021318369 shutdown_connections 2024-09-10T11:20:16.709 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.863+0000 7f327ef74700 1 --2- 172.21.15.144:0/3021318369 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3278104a80 0x7f3278104e90 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:16.709 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.863+0000 7f327ef74700 1 -- 172.21.15.144:0/3021318369 >> 172.21.15.144:0/3021318369 conn(0x7f32781000f0 msgr2=0x7f3278102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:16.709 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.863+0000 7f327ef74700 1 -- 172.21.15.144:0/3021318369 shutdown_connections 2024-09-10T11:20:16.710 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.863+0000 7f327ef74700 1 -- 172.21.15.144:0/3021318369 wait complete. 2024-09-10T11:20:16.710 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.864+0000 7f327ef74700 1 Processor -- start 2024-09-10T11:20:16.710 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.864+0000 7f327ef74700 1 -- start start 2024-09-10T11:20:16.710 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.864+0000 7f327ef74700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3278104a80 0x7f3278079b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:16.710 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.864+0000 7f327ef74700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f327819c600 con 0x7f3278104a80 2024-09-10T11:20:16.710 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.864+0000 7f327cd10700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3278104a80 0x7f3278079b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:16.710 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.864+0000 7f327cd10700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3278104a80 0x7f3278079b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51298/0 (socket says 172.21.15.144:51298) 2024-09-10T11:20:16.710 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.864+0000 7f327cd10700 1 -- 172.21.15.144:0/1229073915 learned_addr learned my addr 172.21.15.144:0/1229073915 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:16.710 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.864+0000 7f327cd10700 1 -- 172.21.15.144:0/1229073915 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326c009550 con 0x7f3278104a80 2024-09-10T11:20:16.710 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.864+0000 7f327cd10700 1 --2- 172.21.15.144:0/1229073915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3278104a80 0x7f3278079b70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f326c000c00 tx=0x7f326c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:16.710 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.864+0000 7f3275ffb700 1 -- 172.21.15.144:0/1229073915 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f326c003b90 con 0x7f3278104a80 2024-09-10T11:20:16.710 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.864+0000 7f3275ffb700 1 -- 172.21.15.144:0/1229073915 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f326c0248b0 con 0x7f3278104a80 2024-09-10T11:20:16.710 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.864+0000 7f327ef74700 1 -- 172.21.15.144:0/1229073915 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f327819c740 con 0x7f3278104a80 2024-09-10T11:20:16.711 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.864+0000 7f3275ffb700 1 -- 172.21.15.144:0/1229073915 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f326c01a8e0 con 0x7f3278104a80 2024-09-10T11:20:16.711 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.865+0000 7f327ef74700 1 -- 172.21.15.144:0/1229073915 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f327819cb60 con 0x7f3278104a80 2024-09-10T11:20:16.711 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.865+0000 7f3275ffb700 1 -- 172.21.15.144:0/1229073915 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f326c024a20 con 0x7f3278104a80 2024-09-10T11:20:16.711 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.866+0000 7f3275ffb700 1 --2- 172.21.15.144:0/1229073915 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3268038440 0x7f326803a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:16.711 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.866+0000 7f327ef74700 1 -- 172.21.15.144:0/1229073915 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3278062380 con 0x7f3278104a80 2024-09-10T11:20:16.711 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.866+0000 7f3275ffb700 1 -- 172.21.15.144:0/1229073915 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f326c04c580 con 0x7f3278104a80 2024-09-10T11:20:16.711 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.866+0000 7f3277fff700 1 --2- 172.21.15.144:0/1229073915 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3268038440 0x7f326803a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:16.711 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.866+0000 7f3277fff700 1 --2- 172.21.15.144:0/1229073915 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3268038440 0x7f326803a8f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3264006fd0 tx=0x7f3264006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:16.711 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:15.871+0000 7f3275ffb700 1 -- 172.21.15.144:0/1229073915 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f326c024db0 con 0x7f3278104a80 2024-09-10T11:20:16.711 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:16.064+0000 7f327ef74700 1 -- 172.21.15.144:0/1229073915 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f3278102390 con 0x7f3268038440 2024-09-10T11:20:16.711 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:16.064+0000 7f3275ffb700 1 -- 172.21.15.144:0/1229073915 <== mgr.14120 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f3278102390 con 0x7f3268038440 2024-09-10T11:20:16.712 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:16.066+0000 7f327ef74700 1 -- 172.21.15.144:0/1229073915 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3268038440 msgr2=0x7f326803a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:16.712 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:16.066+0000 7f327ef74700 1 --2- 172.21.15.144:0/1229073915 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3268038440 0x7f326803a8f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3264006fd0 tx=0x7f3264006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:16.712 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:16.067+0000 7f327ef74700 1 -- 172.21.15.144:0/1229073915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3278104a80 msgr2=0x7f3278079b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:16.712 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:16.067+0000 7f327ef74700 1 --2- 172.21.15.144:0/1229073915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3278104a80 0x7f3278079b70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f326c000c00 tx=0x7f326c011ab0 comp rx=0 tx=0).stop 2024-09-10T11:20:16.712 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:16.067+0000 7f327ef74700 1 -- 172.21.15.144:0/1229073915 shutdown_connections 2024-09-10T11:20:16.712 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:16.067+0000 7f327ef74700 1 --2- 172.21.15.144:0/1229073915 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3268038440 0x7f326803a8f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:16.712 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:16.067+0000 7f327ef74700 1 --2- 172.21.15.144:0/1229073915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3278104a80 0x7f3278079b70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:16.712 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:16.067+0000 7f327ef74700 1 -- 172.21.15.144:0/1229073915 >> 172.21.15.144:0/1229073915 conn(0x7f32781000f0 msgr2=0x7f3278101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:16.712 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:16.067+0000 7f327ef74700 1 -- 172.21.15.144:0/1229073915 shutdown_connections 2024-09-10T11:20:16.712 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:16.067+0000 7f327ef74700 1 -- 172.21.15.144:0/1229073915 wait complete. 2024-09-10T11:20:16.712 INFO:teuthology.orchestra.run.smithi144.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T11:20:16.712 INFO:teuthology.orchestra.run.smithi144.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T11:20:16.712 INFO:teuthology.orchestra.run.smithi144.stdout:Adding host smithi144... 2024-09-10T11:20:16.958 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:16 smithi144 ceph-mon[27942]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:20:18.560 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:18 smithi144 ceph-mon[27942]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi144", "addr": "172.21.15.144", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:20:18.560 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:18 smithi144 ceph-mon[27942]: Deploying cephadm binary to smithi144 2024-09-10T11:20:20.233 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout Added host 'smithi144' with addr '172.21.15.144' 2024-09-10T11:20:20.233 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.146+0000 7f0bbcd9a700 1 Processor -- start 2024-09-10T11:20:20.233 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.147+0000 7f0bbcd9a700 1 -- start start 2024-09-10T11:20:20.233 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.147+0000 7f0bbcd9a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bb8104a80 0x7f0bb8104e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.147+0000 7f0bbcd9a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb81053d0 con 0x7f0bb8104a80 2024-09-10T11:20:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.147+0000 7f0bb659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bb8104a80 0x7f0bb8104e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.147+0000 7f0bb659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bb8104a80 0x7f0bb8104e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51300/0 (socket says 172.21.15.144:51300) 2024-09-10T11:20:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.147+0000 7f0bb659c700 1 -- 172.21.15.144:0/804886757 learned_addr learned my addr 172.21.15.144:0/804886757 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.147+0000 7f0bb659c700 1 -- 172.21.15.144:0/804886757 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb8105510 con 0x7f0bb8104a80 2024-09-10T11:20:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.148+0000 7f0bb659c700 1 --2- 172.21.15.144:0/804886757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bb8104a80 0x7f0bb8104e90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80098a0 tx=0x7f0ba8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9e4fa3fc8419050 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.148+0000 7f0bb559a700 1 -- 172.21.15.144:0/804886757 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ba8004180 con 0x7f0bb8104a80 2024-09-10T11:20:20.235 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.148+0000 7f0bb559a700 1 -- 172.21.15.144:0/804886757 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ba80042e0 con 0x7f0bb8104a80 2024-09-10T11:20:20.235 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.148+0000 7f0bb559a700 1 -- 172.21.15.144:0/804886757 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ba8011a50 con 0x7f0bb8104a80 2024-09-10T11:20:20.235 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.148+0000 7f0bbcd9a700 1 -- 172.21.15.144:0/804886757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bb8104a80 msgr2=0x7f0bb8104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:20.235 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.148+0000 7f0bbcd9a700 1 --2- 172.21.15.144:0/804886757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bb8104a80 0x7f0bb8104e90 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80098a0 tx=0x7f0ba8009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:20.235 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.148+0000 7f0bbcd9a700 1 -- 172.21.15.144:0/804886757 shutdown_connections 2024-09-10T11:20:20.235 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.148+0000 7f0bbcd9a700 1 --2- 172.21.15.144:0/804886757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bb8104a80 0x7f0bb8104e90 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:20.236 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.148+0000 7f0bbcd9a700 1 -- 172.21.15.144:0/804886757 >> 172.21.15.144:0/804886757 conn(0x7f0bb81000f0 msgr2=0x7f0bb8102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:20.236 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.149+0000 7f0bbcd9a700 1 -- 172.21.15.144:0/804886757 shutdown_connections 2024-09-10T11:20:20.236 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.149+0000 7f0bbcd9a700 1 -- 172.21.15.144:0/804886757 wait complete. 2024-09-10T11:20:20.236 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.149+0000 7f0bbcd9a700 1 Processor -- start 2024-09-10T11:20:20.236 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.149+0000 7f0bbcd9a700 1 -- start start 2024-09-10T11:20:20.237 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.149+0000 7f0bbcd9a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bb8104a80 0x7f0bb819bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:20.237 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.149+0000 7f0bbcd9a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bb819bff0 con 0x7f0bb8104a80 2024-09-10T11:20:20.237 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.150+0000 7f0bb659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bb8104a80 0x7f0bb819bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:20.237 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.150+0000 7f0bb659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bb8104a80 0x7f0bb819bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51312/0 (socket says 172.21.15.144:51312) 2024-09-10T11:20:20.237 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.150+0000 7f0bb659c700 1 -- 172.21.15.144:0/2285487744 learned_addr learned my addr 172.21.15.144:0/2285487744 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:20.237 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.150+0000 7f0bb659c700 1 -- 172.21.15.144:0/2285487744 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba8009550 con 0x7f0bb8104a80 2024-09-10T11:20:20.237 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.150+0000 7f0bb659c700 1 --2- 172.21.15.144:0/2285487744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bb8104a80 0x7f0bb819bab0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80119e0 tx=0x7f0ba8011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:20.238 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.150+0000 7f0baf7fe700 1 -- 172.21.15.144:0/2285487744 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ba8003b90 con 0x7f0bb8104a80 2024-09-10T11:20:20.238 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.150+0000 7f0bbcd9a700 1 -- 172.21.15.144:0/2285487744 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bb819c1f0 con 0x7f0bb8104a80 2024-09-10T11:20:20.238 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.150+0000 7f0baf7fe700 1 -- 172.21.15.144:0/2285487744 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0ba801a8a0 con 0x7f0bb8104a80 2024-09-10T11:20:20.238 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.150+0000 7f0baf7fe700 1 -- 172.21.15.144:0/2285487744 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ba8019930 con 0x7f0bb8104a80 2024-09-10T11:20:20.238 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.150+0000 7f0bbcd9a700 1 -- 172.21.15.144:0/2285487744 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb819c610 con 0x7f0bb8104a80 2024-09-10T11:20:20.238 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.151+0000 7f0baf7fe700 1 -- 172.21.15.144:0/2285487744 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f0ba8019bf0 con 0x7f0bb8104a80 2024-09-10T11:20:20.238 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.151+0000 7f0baf7fe700 1 --2- 172.21.15.144:0/2285487744 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0ba40383a0 0x7f0ba403a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:20.238 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.151+0000 7f0bbcd9a700 1 -- 172.21.15.144:0/2285487744 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bb81913b0 con 0x7f0bb8104a80 2024-09-10T11:20:20.239 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.151+0000 7f0baf7fe700 1 -- 172.21.15.144:0/2285487744 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0ba804c740 con 0x7f0bb8104a80 2024-09-10T11:20:20.239 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.151+0000 7f0bb5d9b700 1 --2- 172.21.15.144:0/2285487744 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0ba40383a0 0x7f0ba403a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:20.239 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.152+0000 7f0bb5d9b700 1 --2- 172.21.15.144:0/2285487744 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0ba40383a0 0x7f0ba403a850 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0006fd0 tx=0x7f0ba0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:20.239 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.156+0000 7f0baf7fe700 1 -- 172.21.15.144:0/2285487744 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0ba802b930 con 0x7f0bb8104a80 2024-09-10T11:20:20.239 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:17.350+0000 7f0bbcd9a700 1 -- 172.21.15.144:0/2285487744 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi144", "addr": "172.21.15.144", "target": ["mon-mgr", ""]}) v1 -- 0x7f0bb8061190 con 0x7f0ba40383a0 2024-09-10T11:20:20.239 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:19.655+0000 7f0baf7fe700 1 -- 172.21.15.144:0/2285487744 <== mgr.14120 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f0bb8061190 con 0x7f0ba40383a0 2024-09-10T11:20:20.239 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:19.658+0000 7f0bbcd9a700 1 -- 172.21.15.144:0/2285487744 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0ba40383a0 msgr2=0x7f0ba403a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:20.240 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:19.658+0000 7f0bbcd9a700 1 --2- 172.21.15.144:0/2285487744 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0ba40383a0 0x7f0ba403a850 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0006fd0 tx=0x7f0ba0006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:20.240 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:19.658+0000 7f0bbcd9a700 1 -- 172.21.15.144:0/2285487744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bb8104a80 msgr2=0x7f0bb819bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:20.240 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:19.658+0000 7f0bbcd9a700 1 --2- 172.21.15.144:0/2285487744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bb8104a80 0x7f0bb819bab0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80119e0 tx=0x7f0ba8011ac0 comp rx=0 tx=0).stop 2024-09-10T11:20:20.240 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:19.658+0000 7f0bbcd9a700 1 -- 172.21.15.144:0/2285487744 shutdown_connections 2024-09-10T11:20:20.240 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:19.658+0000 7f0bbcd9a700 1 --2- 172.21.15.144:0/2285487744 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0ba40383a0 0x7f0ba403a850 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:20.240 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:19.658+0000 7f0bbcd9a700 1 --2- 172.21.15.144:0/2285487744 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bb8104a80 0x7f0bb819bab0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:20.241 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:19.658+0000 7f0bbcd9a700 1 -- 172.21.15.144:0/2285487744 >> 172.21.15.144:0/2285487744 conn(0x7f0bb81000f0 msgr2=0x7f0bb818e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:20.241 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:19.658+0000 7f0bbcd9a700 1 -- 172.21.15.144:0/2285487744 shutdown_connections 2024-09-10T11:20:20.241 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:19.658+0000 7f0bbcd9a700 1 -- 172.21.15.144:0/2285487744 wait complete. 2024-09-10T11:20:20.241 INFO:teuthology.orchestra.run.smithi144.stdout:Deploying mon service with default placement... 2024-09-10T11:20:20.853 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:20 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:20.853 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:20 smithi144 ceph-mon[27942]: Added host smithi144 2024-09-10T11:20:20.853 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:20 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:20:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:21 smithi144 ceph-mon[27942]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:20:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:21 smithi144 ceph-mon[27942]: Saving service mon spec with placement count:5 2024-09-10T11:20:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:21 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:22.362 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T11:20:22.362 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.636+0000 7f3b1d9e9700 1 Processor -- start 2024-09-10T11:20:22.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.636+0000 7f3b1d9e9700 1 -- start start 2024-09-10T11:20:22.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.637+0000 7f3b1d9e9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b181066d0 0x7f3b18108ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:22.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.637+0000 7f3b1d9e9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b18079d60 con 0x7f3b181066d0 2024-09-10T11:20:22.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.637+0000 7f3b16ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b181066d0 0x7f3b18108ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:22.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.637+0000 7f3b16ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b181066d0 0x7f3b18108ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51322/0 (socket says 172.21.15.144:51322) 2024-09-10T11:20:22.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.637+0000 7f3b16ffd700 1 -- 172.21.15.144:0/2746172416 learned_addr learned my addr 172.21.15.144:0/2746172416 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:22.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.637+0000 7f3b16ffd700 1 -- 172.21.15.144:0/2746172416 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b18079ea0 con 0x7f3b181066d0 2024-09-10T11:20:22.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.637+0000 7f3b16ffd700 1 --2- 172.21.15.144:0/2746172416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b181066d0 0x7f3b18108ab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b080098a0 tx=0x7f3b08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8045fce6e4d2723 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:22.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.637+0000 7f3b15ffb700 1 -- 172.21.15.144:0/2746172416 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b08004180 con 0x7f3b181066d0 2024-09-10T11:20:22.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.638+0000 7f3b15ffb700 1 -- 172.21.15.144:0/2746172416 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b080042e0 con 0x7f3b181066d0 2024-09-10T11:20:22.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.638+0000 7f3b15ffb700 1 -- 172.21.15.144:0/2746172416 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b08011a50 con 0x7f3b181066d0 2024-09-10T11:20:22.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.638+0000 7f3b1d9e9700 1 -- 172.21.15.144:0/2746172416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b181066d0 msgr2=0x7f3b18108ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:22.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.638+0000 7f3b1d9e9700 1 --2- 172.21.15.144:0/2746172416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b181066d0 0x7f3b18108ab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3b080098a0 tx=0x7f3b08009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:22.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.638+0000 7f3b1d9e9700 1 -- 172.21.15.144:0/2746172416 shutdown_connections 2024-09-10T11:20:22.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.638+0000 7f3b1d9e9700 1 --2- 172.21.15.144:0/2746172416 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b181066d0 0x7f3b18108ab0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:22.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.638+0000 7f3b1d9e9700 1 -- 172.21.15.144:0/2746172416 >> 172.21.15.144:0/2746172416 conn(0x7f3b181000f0 msgr2=0x7f3b18102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:22.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.638+0000 7f3b1d9e9700 1 -- 172.21.15.144:0/2746172416 shutdown_connections 2024-09-10T11:20:22.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.638+0000 7f3b1d9e9700 1 -- 172.21.15.144:0/2746172416 wait complete. 2024-09-10T11:20:22.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.639+0000 7f3b1d9e9700 1 Processor -- start 2024-09-10T11:20:22.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.639+0000 7f3b1d9e9700 1 -- start start 2024-09-10T11:20:22.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.639+0000 7f3b1d9e9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b181066d0 0x7f3b181998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:22.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.639+0000 7f3b1d9e9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b18199e10 con 0x7f3b181066d0 2024-09-10T11:20:22.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.639+0000 7f3b16ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b181066d0 0x7f3b181998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:22.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.639+0000 7f3b16ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b181066d0 0x7f3b181998d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51332/0 (socket says 172.21.15.144:51332) 2024-09-10T11:20:22.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.639+0000 7f3b16ffd700 1 -- 172.21.15.144:0/1323533498 learned_addr learned my addr 172.21.15.144:0/1323533498 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:22.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.639+0000 7f3b16ffd700 1 -- 172.21.15.144:0/1323533498 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b08009550 con 0x7f3b181066d0 2024-09-10T11:20:22.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.640+0000 7f3b16ffd700 1 --2- 172.21.15.144:0/1323533498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b181066d0 0x7f3b181998d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3b08000c00 tx=0x7f3b08011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:22.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.640+0000 7f3b1c9e7700 1 -- 172.21.15.144:0/1323533498 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b08003b90 con 0x7f3b181066d0 2024-09-10T11:20:22.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.640+0000 7f3b1d9e9700 1 -- 172.21.15.144:0/1323533498 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1819a070 con 0x7f3b181066d0 2024-09-10T11:20:22.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.640+0000 7f3b1c9e7700 1 -- 172.21.15.144:0/1323533498 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f3b080248b0 con 0x7f3b181066d0 2024-09-10T11:20:22.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.640+0000 7f3b1c9e7700 1 -- 172.21.15.144:0/1323533498 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b0801a8e0 con 0x7f3b181066d0 2024-09-10T11:20:22.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.640+0000 7f3b1d9e9700 1 -- 172.21.15.144:0/1323533498 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1819a4f0 con 0x7f3b181066d0 2024-09-10T11:20:22.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.641+0000 7f3b1c9e7700 1 -- 172.21.15.144:0/1323533498 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f3b0801aba0 con 0x7f3b181066d0 2024-09-10T11:20:22.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.641+0000 7f3b1d9e9700 1 -- 172.21.15.144:0/1323533498 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b180623f0 con 0x7f3b181066d0 2024-09-10T11:20:22.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.641+0000 7f3b1c9e7700 1 --2- 172.21.15.144:0/1323533498 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b000383f0 0x7f3b0003a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:22.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.641+0000 7f3b1c9e7700 1 -- 172.21.15.144:0/1323533498 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3b0804c3a0 con 0x7f3b181066d0 2024-09-10T11:20:22.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.641+0000 7f3b167fc700 1 --2- 172.21.15.144:0/1323533498 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b000383f0 0x7f3b0003a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:22.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.641+0000 7f3b167fc700 1 --2- 172.21.15.144:0/1323533498 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b000383f0 0x7f3b0003a8a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c006fd0 tx=0x7f3b0c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:22.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.645+0000 7f3b1c9e7700 1 -- 172.21.15.144:0/1323533498 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3b180623f0 con 0x7f3b181066d0 2024-09-10T11:20:22.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.851+0000 7f3b1d9e9700 1 -- 172.21.15.144:0/1323533498 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b18062380 con 0x7f3b000383f0 2024-09-10T11:20:22.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.856+0000 7f3b1c9e7700 1 -- 172.21.15.144:0/1323533498 <== mgr.14120 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f3b18062380 con 0x7f3b000383f0 2024-09-10T11:20:22.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.858+0000 7f3b1d9e9700 1 -- 172.21.15.144:0/1323533498 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b000383f0 msgr2=0x7f3b0003a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:22.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.858+0000 7f3b1d9e9700 1 --2- 172.21.15.144:0/1323533498 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b000383f0 0x7f3b0003a8a0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c006fd0 tx=0x7f3b0c006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:22.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.858+0000 7f3b1d9e9700 1 -- 172.21.15.144:0/1323533498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b181066d0 msgr2=0x7f3b181998d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:22.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.858+0000 7f3b1d9e9700 1 --2- 172.21.15.144:0/1323533498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b181066d0 0x7f3b181998d0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f3b08000c00 tx=0x7f3b08011ab0 comp rx=0 tx=0).stop 2024-09-10T11:20:22.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.859+0000 7f3b1d9e9700 1 -- 172.21.15.144:0/1323533498 shutdown_connections 2024-09-10T11:20:22.367 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.859+0000 7f3b1d9e9700 1 --2- 172.21.15.144:0/1323533498 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b000383f0 0x7f3b0003a8a0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:22.367 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.859+0000 7f3b1d9e9700 1 --2- 172.21.15.144:0/1323533498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b181066d0 0x7f3b181998d0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:22.367 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.859+0000 7f3b1d9e9700 1 -- 172.21.15.144:0/1323533498 >> 172.21.15.144:0/1323533498 conn(0x7f3b181000f0 msgr2=0x7f3b18101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:22.367 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.859+0000 7f3b1d9e9700 1 -- 172.21.15.144:0/1323533498 shutdown_connections 2024-09-10T11:20:22.367 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:20.859+0000 7f3b1d9e9700 1 -- 172.21.15.144:0/1323533498 wait complete. 2024-09-10T11:20:22.367 INFO:teuthology.orchestra.run.smithi144.stdout:Deploying mgr service with default placement... 2024-09-10T11:20:23.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:22 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:23.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:22 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:24 smithi144 ceph-mon[27942]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:20:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:24 smithi144 ceph-mon[27942]: Saving service mgr spec with placement count:2 2024-09-10T11:20:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:24 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:24 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:24.206 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T11:20:24.206 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.809+0000 7f33bf3fe700 1 Processor -- start 2024-09-10T11:20:24.206 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.809+0000 7f33bf3fe700 1 -- start start 2024-09-10T11:20:24.207 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.809+0000 7f33bf3fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b8104fb0 0x7f33b81053c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:24.207 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.809+0000 7f33bf3fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b8105900 con 0x7f33b8104fb0 2024-09-10T11:20:24.207 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.809+0000 7f33bd19a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b8104fb0 0x7f33b81053c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:24.207 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.809+0000 7f33bd19a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b8104fb0 0x7f33b81053c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40260/0 (socket says 172.21.15.144:40260) 2024-09-10T11:20:24.207 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.809+0000 7f33bd19a700 1 -- 172.21.15.144:0/67732159 learned_addr learned my addr 172.21.15.144:0/67732159 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:24.207 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.810+0000 7f33bd19a700 1 -- 172.21.15.144:0/67732159 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b8105a40 con 0x7f33b8104fb0 2024-09-10T11:20:24.207 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.810+0000 7f33bd19a700 1 --2- 172.21.15.144:0/67732159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b8104fb0 0x7f33b81053c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f33b400bb40 tx=0x7f33b400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea46f0e705cb9a38 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:24.207 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.810+0000 7f33affff700 1 -- 172.21.15.144:0/67732159 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33b400dc00 con 0x7f33b8104fb0 2024-09-10T11:20:24.207 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.810+0000 7f33affff700 1 -- 172.21.15.144:0/67732159 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f33b40044e0 con 0x7f33b8104fb0 2024-09-10T11:20:24.208 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.810+0000 7f33affff700 1 -- 172.21.15.144:0/67732159 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33b4010470 con 0x7f33b8104fb0 2024-09-10T11:20:24.208 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.811+0000 7f33bf3fe700 1 -- 172.21.15.144:0/67732159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b8104fb0 msgr2=0x7f33b81053c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:24.208 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.811+0000 7f33bf3fe700 1 --2- 172.21.15.144:0/67732159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b8104fb0 0x7f33b81053c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f33b400bb40 tx=0x7f33b400be50 comp rx=0 tx=0).stop 2024-09-10T11:20:24.208 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.811+0000 7f33bf3fe700 1 -- 172.21.15.144:0/67732159 shutdown_connections 2024-09-10T11:20:24.208 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.811+0000 7f33bf3fe700 1 --2- 172.21.15.144:0/67732159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b8104fb0 0x7f33b81053c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:24.208 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.811+0000 7f33bf3fe700 1 -- 172.21.15.144:0/67732159 >> 172.21.15.144:0/67732159 conn(0x7f33b80ff880 msgr2=0x7f33b8101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:24.208 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.811+0000 7f33bf3fe700 1 -- 172.21.15.144:0/67732159 shutdown_connections 2024-09-10T11:20:24.208 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.811+0000 7f33bf3fe700 1 -- 172.21.15.144:0/67732159 wait complete. 2024-09-10T11:20:24.208 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.812+0000 7f33bf3fe700 1 Processor -- start 2024-09-10T11:20:24.208 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.812+0000 7f33bf3fe700 1 -- start start 2024-09-10T11:20:24.208 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.812+0000 7f33bf3fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b8104fb0 0x7f33b8197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:24.208 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.812+0000 7f33bf3fe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b8197c90 con 0x7f33b8104fb0 2024-09-10T11:20:24.208 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.812+0000 7f33bd19a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b8104fb0 0x7f33b8197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:24.209 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.812+0000 7f33bd19a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b8104fb0 0x7f33b8197750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40276/0 (socket says 172.21.15.144:40276) 2024-09-10T11:20:24.209 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.812+0000 7f33bd19a700 1 -- 172.21.15.144:0/2772470 learned_addr learned my addr 172.21.15.144:0/2772470 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:24.209 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.812+0000 7f33bd19a700 1 -- 172.21.15.144:0/2772470 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b400b7f0 con 0x7f33b8104fb0 2024-09-10T11:20:24.209 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.812+0000 7f33bd19a700 1 --2- 172.21.15.144:0/2772470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b8104fb0 0x7f33b8197750 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f33b4003fc0 tx=0x7f33b40040a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:24.209 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.813+0000 7f33ae7fc700 1 -- 172.21.15.144:0/2772470 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33b401e070 con 0x7f33b8104fb0 2024-09-10T11:20:24.209 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.813+0000 7f33ae7fc700 1 -- 172.21.15.144:0/2772470 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f33b400de20 con 0x7f33b8104fb0 2024-09-10T11:20:24.209 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.813+0000 7f33bf3fe700 1 -- 172.21.15.144:0/2772470 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b8197e90 con 0x7f33b8104fb0 2024-09-10T11:20:24.209 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.813+0000 7f33bf3fe700 1 -- 172.21.15.144:0/2772470 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b81982b0 con 0x7f33b8104fb0 2024-09-10T11:20:24.209 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.813+0000 7f33ae7fc700 1 -- 172.21.15.144:0/2772470 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f33b4022a00 con 0x7f33b8104fb0 2024-09-10T11:20:24.209 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.814+0000 7f33ae7fc700 1 -- 172.21.15.144:0/2772470 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f33b402c440 con 0x7f33b8104fb0 2024-09-10T11:20:24.210 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.814+0000 7f33ae7fc700 1 --2- 172.21.15.144:0/2772470 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f33a4038400 0x7f33a403a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:24.210 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.814+0000 7f33ae7fc700 1 -- 172.21.15.144:0/2772470 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f33b404ea60 con 0x7f33b8104fb0 2024-09-10T11:20:24.210 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.814+0000 7f33bf3fe700 1 -- 172.21.15.144:0/2772470 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f339c005320 con 0x7f33b8104fb0 2024-09-10T11:20:24.210 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.814+0000 7f33bc999700 1 --2- 172.21.15.144:0/2772470 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f33a4038400 0x7f33a403a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:24.210 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.814+0000 7f33bc999700 1 --2- 172.21.15.144:0/2772470 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f33a4038400 0x7f33a403a8b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f33a8006fd0 tx=0x7f33a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:24.210 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:22.820+0000 7f33ae7fc700 1 -- 172.21.15.144:0/2772470 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f33b4027070 con 0x7f33b8104fb0 2024-09-10T11:20:24.210 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:23.046+0000 7f33bf3fe700 1 -- 172.21.15.144:0/2772470 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f339c000bf0 con 0x7f33a4038400 2024-09-10T11:20:24.210 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:23.051+0000 7f33ae7fc700 1 -- 172.21.15.144:0/2772470 <== mgr.14120 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f339c000bf0 con 0x7f33a4038400 2024-09-10T11:20:24.210 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:23.053+0000 7f33bf3fe700 1 -- 172.21.15.144:0/2772470 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f33a4038400 msgr2=0x7f33a403a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:24.210 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:23.053+0000 7f33bf3fe700 1 --2- 172.21.15.144:0/2772470 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f33a4038400 0x7f33a403a8b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f33a8006fd0 tx=0x7f33a8006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:24.210 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:23.053+0000 7f33bf3fe700 1 -- 172.21.15.144:0/2772470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b8104fb0 msgr2=0x7f33b8197750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:24.210 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:23.053+0000 7f33bf3fe700 1 --2- 172.21.15.144:0/2772470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b8104fb0 0x7f33b8197750 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f33b4003fc0 tx=0x7f33b40040a0 comp rx=0 tx=0).stop 2024-09-10T11:20:24.210 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:23.053+0000 7f33bf3fe700 1 -- 172.21.15.144:0/2772470 shutdown_connections 2024-09-10T11:20:24.211 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:23.053+0000 7f33bf3fe700 1 --2- 172.21.15.144:0/2772470 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f33a4038400 0x7f33a403a8b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:24.211 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:23.053+0000 7f33bf3fe700 1 --2- 172.21.15.144:0/2772470 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33b8104fb0 0x7f33b8197750 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:24.211 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:23.053+0000 7f33bf3fe700 1 -- 172.21.15.144:0/2772470 >> 172.21.15.144:0/2772470 conn(0x7f33b80ff880 msgr2=0x7f33b818e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:24.211 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:23.053+0000 7f33bf3fe700 1 -- 172.21.15.144:0/2772470 shutdown_connections 2024-09-10T11:20:24.211 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:23.053+0000 7f33bf3fe700 1 -- 172.21.15.144:0/2772470 wait complete. 2024-09-10T11:20:24.211 INFO:teuthology.orchestra.run.smithi144.stdout:Deploying crash service with default placement... 2024-09-10T11:20:25.422 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-09-10T11:20:25.422 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.616+0000 7f0c15cf6700 1 Processor -- start 2024-09-10T11:20:25.422 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.616+0000 7f0c15cf6700 1 -- start start 2024-09-10T11:20:25.422 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.616+0000 7f0c15cf6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c10079d20 0x7f0c10108770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:25.422 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.617+0000 7f0c15cf6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c10108cb0 con 0x7f0c10079d20 2024-09-10T11:20:25.422 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.617+0000 7f0c0f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c10079d20 0x7f0c10108770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:25.422 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.617+0000 7f0c0f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c10079d20 0x7f0c10108770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40288/0 (socket says 172.21.15.144:40288) 2024-09-10T11:20:25.422 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.617+0000 7f0c0f7fe700 1 -- 172.21.15.144:0/730415302 learned_addr learned my addr 172.21.15.144:0/730415302 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:25.422 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.617+0000 7f0c0f7fe700 1 -- 172.21.15.144:0/730415302 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c10108df0 con 0x7f0c10079d20 2024-09-10T11:20:25.423 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.617+0000 7f0c0f7fe700 1 --2- 172.21.15.144:0/730415302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c10079d20 0x7f0c10108770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009e80 tx=0x7f0c0000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0435e2c4cad74e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:25.423 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.617+0000 7f0c0e7fc700 1 -- 172.21.15.144:0/730415302 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c0000e070 con 0x7f0c10079d20 2024-09-10T11:20:25.423 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.617+0000 7f0c0e7fc700 1 -- 172.21.15.144:0/730415302 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0c0000b8c0 con 0x7f0c10079d20 2024-09-10T11:20:25.423 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.617+0000 7f0c0e7fc700 1 -- 172.21.15.144:0/730415302 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c000044d0 con 0x7f0c10079d20 2024-09-10T11:20:25.423 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.618+0000 7f0c15cf6700 1 -- 172.21.15.144:0/730415302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c10079d20 msgr2=0x7f0c10108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:25.423 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.618+0000 7f0c15cf6700 1 --2- 172.21.15.144:0/730415302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c10079d20 0x7f0c10108770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f0c00009e80 tx=0x7f0c0000b330 comp rx=0 tx=0).stop 2024-09-10T11:20:25.423 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.618+0000 7f0c15cf6700 1 -- 172.21.15.144:0/730415302 shutdown_connections 2024-09-10T11:20:25.423 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.618+0000 7f0c15cf6700 1 --2- 172.21.15.144:0/730415302 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c10079d20 0x7f0c10108770 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:25.423 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.618+0000 7f0c15cf6700 1 -- 172.21.15.144:0/730415302 >> 172.21.15.144:0/730415302 conn(0x7f0c10100160 msgr2=0x7f0c10102570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:25.423 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.618+0000 7f0c15cf6700 1 -- 172.21.15.144:0/730415302 shutdown_connections 2024-09-10T11:20:25.423 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.618+0000 7f0c15cf6700 1 -- 172.21.15.144:0/730415302 wait complete. 2024-09-10T11:20:25.423 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.619+0000 7f0c15cf6700 1 Processor -- start 2024-09-10T11:20:25.424 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.619+0000 7f0c15cf6700 1 -- start start 2024-09-10T11:20:25.424 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.619+0000 7f0c15cf6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c10199910 0x7f0c10199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:25.424 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.619+0000 7f0c15cf6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1019a260 con 0x7f0c10199910 2024-09-10T11:20:25.424 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.620+0000 7f0c0f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c10199910 0x7f0c10199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:25.424 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.620+0000 7f0c0f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c10199910 0x7f0c10199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40296/0 (socket says 172.21.15.144:40296) 2024-09-10T11:20:25.424 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.620+0000 7f0c0f7fe700 1 -- 172.21.15.144:0/3824718442 learned_addr learned my addr 172.21.15.144:0/3824718442 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:25.424 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.620+0000 7f0c0f7fe700 1 -- 172.21.15.144:0/3824718442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c00009b30 con 0x7f0c10199910 2024-09-10T11:20:25.424 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.620+0000 7f0c0f7fe700 1 --2- 172.21.15.144:0/3824718442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c10199910 0x7f0c10199d20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0c00004940 tx=0x7f0c00004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:25.424 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.620+0000 7f0c0cff9700 1 -- 172.21.15.144:0/3824718442 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c0000e040 con 0x7f0c10199910 2024-09-10T11:20:25.424 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.620+0000 7f0c15cf6700 1 -- 172.21.15.144:0/3824718442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1019a460 con 0x7f0c10199910 2024-09-10T11:20:25.424 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.620+0000 7f0c0cff9700 1 -- 172.21.15.144:0/3824718442 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0c0000bc40 con 0x7f0c10199910 2024-09-10T11:20:25.424 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.620+0000 7f0c0cff9700 1 -- 172.21.15.144:0/3824718442 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0c00012670 con 0x7f0c10199910 2024-09-10T11:20:25.425 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.620+0000 7f0c15cf6700 1 -- 172.21.15.144:0/3824718442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1019d0c0 con 0x7f0c10199910 2024-09-10T11:20:25.425 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.621+0000 7f0c15cf6700 1 -- 172.21.15.144:0/3824718442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c10062380 con 0x7f0c10199910 2024-09-10T11:20:25.425 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.621+0000 7f0c0cff9700 1 -- 172.21.15.144:0/3824718442 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f0c00019040 con 0x7f0c10199910 2024-09-10T11:20:25.425 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.622+0000 7f0c0cff9700 1 --2- 172.21.15.144:0/3824718442 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0bfc0383f0 0x7f0bfc03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:25.425 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.622+0000 7f0c0cff9700 1 -- 172.21.15.144:0/3824718442 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0c0004bc00 con 0x7f0c10199910 2024-09-10T11:20:25.425 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.622+0000 7f0c0effd700 1 --2- 172.21.15.144:0/3824718442 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0bfc0383f0 0x7f0bfc03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:25.425 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.622+0000 7f0c0effd700 1 --2- 172.21.15.144:0/3824718442 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0bfc0383f0 0x7f0bfc03a8a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8006fd0 tx=0x7f0bf8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:25.425 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.627+0000 7f0c0cff9700 1 -- 172.21.15.144:0/3824718442 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0c00052080 con 0x7f0c10199910 2024-09-10T11:20:25.425 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.839+0000 7f0c15cf6700 1 -- 172.21.15.144:0/3824718442 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c10101610 con 0x7f0bfc0383f0 2024-09-10T11:20:25.425 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.847+0000 7f0c0cff9700 1 -- 172.21.15.144:0/3824718442 <== mgr.14120 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f0c10101610 con 0x7f0bfc0383f0 2024-09-10T11:20:25.425 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.849+0000 7f0c15cf6700 1 -- 172.21.15.144:0/3824718442 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0bfc0383f0 msgr2=0x7f0bfc03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:25.425 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.849+0000 7f0c15cf6700 1 --2- 172.21.15.144:0/3824718442 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0bfc0383f0 0x7f0bfc03a8a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0bf8006fd0 tx=0x7f0bf8006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:25.425 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.849+0000 7f0c15cf6700 1 -- 172.21.15.144:0/3824718442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c10199910 msgr2=0x7f0c10199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:25.426 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.849+0000 7f0c15cf6700 1 --2- 172.21.15.144:0/3824718442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c10199910 0x7f0c10199d20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0c00004940 tx=0x7f0c00004a20 comp rx=0 tx=0).stop 2024-09-10T11:20:25.426 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.849+0000 7f0c15cf6700 1 -- 172.21.15.144:0/3824718442 shutdown_connections 2024-09-10T11:20:25.426 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.849+0000 7f0c15cf6700 1 --2- 172.21.15.144:0/3824718442 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0bfc0383f0 0x7f0bfc03a8a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:25.426 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.849+0000 7f0c15cf6700 1 --2- 172.21.15.144:0/3824718442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0c10199910 0x7f0c10199d20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:25.426 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.849+0000 7f0c15cf6700 1 -- 172.21.15.144:0/3824718442 >> 172.21.15.144:0/3824718442 conn(0x7f0c10100160 msgr2=0x7f0c10100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:25.426 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.849+0000 7f0c15cf6700 1 -- 172.21.15.144:0/3824718442 shutdown_connections 2024-09-10T11:20:25.426 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:24.849+0000 7f0c15cf6700 1 -- 172.21.15.144:0/3824718442 wait complete. 2024-09-10T11:20:25.426 INFO:teuthology.orchestra.run.smithi144.stdout:Deploying ceph-exporter service with default placement... 2024-09-10T11:20:25.792 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:25 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:25.792 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:25 smithi144 ceph-mon[27942]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:20:25.792 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:25 smithi144 ceph-mon[27942]: Saving service crash spec with placement * 2024-09-10T11:20:25.792 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:25 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:27.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:27 smithi144 ceph-mon[27942]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:20:27.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:27 smithi144 ceph-mon[27942]: Saving service ceph-exporter spec with placement * 2024-09-10T11:20:27.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:27 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:27.190 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-09-10T11:20:27.190 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.836+0000 7f8723b35700 1 Processor -- start 2024-09-10T11:20:27.190 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.836+0000 7f8723b35700 1 -- start start 2024-09-10T11:20:27.191 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.836+0000 7f8723b35700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f871c104a80 0x7f871c104e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:27.191 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.836+0000 7f8723b35700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c1053d0 con 0x7f871c104a80 2024-09-10T11:20:27.191 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.836+0000 7f87218d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f871c104a80 0x7f871c104e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:27.191 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.836+0000 7f87218d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f871c104a80 0x7f871c104e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40310/0 (socket says 172.21.15.144:40310) 2024-09-10T11:20:27.191 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.836+0000 7f87218d1700 1 -- 172.21.15.144:0/3933856409 learned_addr learned my addr 172.21.15.144:0/3933856409 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:27.191 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.837+0000 7f87218d1700 1 -- 172.21.15.144:0/3933856409 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c105510 con 0x7f871c104a80 2024-09-10T11:20:27.191 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.837+0000 7f87218d1700 1 --2- 172.21.15.144:0/3933856409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f871c104a80 0x7f871c104e90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f87180098a0 tx=0x7f8718009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=baeec682389cf629 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:27.191 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.837+0000 7f87208cf700 1 -- 172.21.15.144:0/3933856409 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8718004180 con 0x7f871c104a80 2024-09-10T11:20:27.191 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.837+0000 7f87208cf700 1 -- 172.21.15.144:0/3933856409 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f87180042e0 con 0x7f871c104a80 2024-09-10T11:20:27.191 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.837+0000 7f87208cf700 1 -- 172.21.15.144:0/3933856409 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8718011af0 con 0x7f871c104a80 2024-09-10T11:20:27.191 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.837+0000 7f8723b35700 1 -- 172.21.15.144:0/3933856409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f871c104a80 msgr2=0x7f871c104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:27.191 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.837+0000 7f8723b35700 1 --2- 172.21.15.144:0/3933856409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f871c104a80 0x7f871c104e90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f87180098a0 tx=0x7f8718009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:27.191 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.838+0000 7f8723b35700 1 -- 172.21.15.144:0/3933856409 shutdown_connections 2024-09-10T11:20:27.192 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.838+0000 7f8723b35700 1 --2- 172.21.15.144:0/3933856409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f871c104a80 0x7f871c104e90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:27.192 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.838+0000 7f8723b35700 1 -- 172.21.15.144:0/3933856409 >> 172.21.15.144:0/3933856409 conn(0x7f871c1000f0 msgr2=0x7f871c102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:27.192 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.838+0000 7f8723b35700 1 -- 172.21.15.144:0/3933856409 shutdown_connections 2024-09-10T11:20:27.192 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.838+0000 7f8723b35700 1 -- 172.21.15.144:0/3933856409 wait complete. 2024-09-10T11:20:27.192 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.838+0000 7f8723b35700 1 Processor -- start 2024-09-10T11:20:27.192 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.838+0000 7f8723b35700 1 -- start start 2024-09-10T11:20:27.192 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.838+0000 7f8723b35700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f871c104a80 0x7f871c079b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:27.192 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.838+0000 7f8723b35700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c19c640 con 0x7f871c104a80 2024-09-10T11:20:27.192 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.838+0000 7f87218d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f871c104a80 0x7f871c079b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:27.192 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.839+0000 7f87218d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f871c104a80 0x7f871c079b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40320/0 (socket says 172.21.15.144:40320) 2024-09-10T11:20:27.192 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.839+0000 7f87218d1700 1 -- 172.21.15.144:0/2270666703 learned_addr learned my addr 172.21.15.144:0/2270666703 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:27.192 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.839+0000 7f87218d1700 1 -- 172.21.15.144:0/2270666703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8718009550 con 0x7f871c104a80 2024-09-10T11:20:27.192 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.839+0000 7f87218d1700 1 --2- 172.21.15.144:0/2270666703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f871c104a80 0x7f871c079b70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8718006ba0 tx=0x7f8718011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:27.193 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.839+0000 7f8712ffd700 1 -- 172.21.15.144:0/2270666703 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8718003b00 con 0x7f871c104a80 2024-09-10T11:20:27.193 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.839+0000 7f8723b35700 1 -- 172.21.15.144:0/2270666703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871c19c780 con 0x7f871c104a80 2024-09-10T11:20:27.193 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.839+0000 7f8712ffd700 1 -- 172.21.15.144:0/2270666703 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f871801a8e0 con 0x7f871c104a80 2024-09-10T11:20:27.193 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.839+0000 7f8712ffd700 1 -- 172.21.15.144:0/2270666703 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f87180199c0 con 0x7f871c104a80 2024-09-10T11:20:27.193 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.839+0000 7f8723b35700 1 -- 172.21.15.144:0/2270666703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c19cba0 con 0x7f871c104a80 2024-09-10T11:20:27.193 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.840+0000 7f8712ffd700 1 -- 172.21.15.144:0/2270666703 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f8718019c80 con 0x7f871c104a80 2024-09-10T11:20:27.193 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.840+0000 7f8723b35700 1 -- 172.21.15.144:0/2270666703 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871c062380 con 0x7f871c104a80 2024-09-10T11:20:27.193 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.840+0000 7f8712ffd700 1 --2- 172.21.15.144:0/2270666703 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f87080383a0 0x7f870803a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:27.193 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.840+0000 7f8712ffd700 1 -- 172.21.15.144:0/2270666703 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f871804c7d0 con 0x7f871c104a80 2024-09-10T11:20:27.193 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.840+0000 7f87210d0700 1 --2- 172.21.15.144:0/2270666703 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f87080383a0 0x7f870803a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:27.193 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.841+0000 7f87210d0700 1 --2- 172.21.15.144:0/2270666703 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f87080383a0 0x7f870803a850 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f870c006fd0 tx=0x7f870c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:27.193 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:25.845+0000 7f8712ffd700 1 -- 172.21.15.144:0/2270666703 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f871802d440 con 0x7f871c104a80 2024-09-10T11:20:27.194 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:26.059+0000 7f8723b35700 1 -- 172.21.15.144:0/2270666703 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f871c102390 con 0x7f87080383a0 2024-09-10T11:20:27.194 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:26.064+0000 7f8712ffd700 1 -- 172.21.15.144:0/2270666703 <== mgr.14120 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f871c102390 con 0x7f87080383a0 2024-09-10T11:20:27.194 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:26.066+0000 7f8723b35700 1 -- 172.21.15.144:0/2270666703 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f87080383a0 msgr2=0x7f870803a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:27.194 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:26.066+0000 7f8723b35700 1 --2- 172.21.15.144:0/2270666703 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f87080383a0 0x7f870803a850 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f870c006fd0 tx=0x7f870c006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:27.194 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:26.066+0000 7f8723b35700 1 -- 172.21.15.144:0/2270666703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f871c104a80 msgr2=0x7f871c079b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:27.194 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:26.066+0000 7f8723b35700 1 --2- 172.21.15.144:0/2270666703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f871c104a80 0x7f871c079b70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8718006ba0 tx=0x7f8718011b50 comp rx=0 tx=0).stop 2024-09-10T11:20:27.194 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:26.066+0000 7f8723b35700 1 -- 172.21.15.144:0/2270666703 shutdown_connections 2024-09-10T11:20:27.194 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:26.066+0000 7f8723b35700 1 --2- 172.21.15.144:0/2270666703 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f87080383a0 0x7f870803a850 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:27.194 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:26.066+0000 7f8723b35700 1 --2- 172.21.15.144:0/2270666703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f871c104a80 0x7f871c079b70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:27.194 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:26.066+0000 7f8723b35700 1 -- 172.21.15.144:0/2270666703 >> 172.21.15.144:0/2270666703 conn(0x7f871c1000f0 msgr2=0x7f871c101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:27.194 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:26.066+0000 7f8723b35700 1 -- 172.21.15.144:0/2270666703 shutdown_connections 2024-09-10T11:20:27.194 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:26.066+0000 7f8723b35700 1 -- 172.21.15.144:0/2270666703 wait complete. 2024-09-10T11:20:27.194 INFO:teuthology.orchestra.run.smithi144.stdout:Deploying prometheus service with default placement... 2024-09-10T11:20:28.875 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-09-10T11:20:28.875 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.031+0000 7f92881d4700 1 Processor -- start 2024-09-10T11:20:28.875 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.031+0000 7f92881d4700 1 -- start start 2024-09-10T11:20:28.875 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.031+0000 7f92881d4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9280105b70 0x7f9280105f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:28.875 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.031+0000 7f92881d4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92801064c0 con 0x7f9280105b70 2024-09-10T11:20:28.875 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.031+0000 7f9285f70700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9280105b70 0x7f9280105f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:28.875 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.031+0000 7f9285f70700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9280105b70 0x7f9280105f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40324/0 (socket says 172.21.15.144:40324) 2024-09-10T11:20:28.875 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.031+0000 7f9285f70700 1 -- 172.21.15.144:0/2154334069 learned_addr learned my addr 172.21.15.144:0/2154334069 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:28.876 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.032+0000 7f9285f70700 1 -- 172.21.15.144:0/2154334069 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9280106600 con 0x7f9280105b70 2024-09-10T11:20:28.876 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.032+0000 7f9285f70700 1 --2- 172.21.15.144:0/2154334069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9280105b70 0x7f9280105f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f927c0098a0 tx=0x7f927c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d6680cbb3b3276 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:28.876 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.032+0000 7f9284f6e700 1 -- 172.21.15.144:0/2154334069 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f927c004180 con 0x7f9280105b70 2024-09-10T11:20:28.876 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.032+0000 7f9284f6e700 1 -- 172.21.15.144:0/2154334069 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f927c0042e0 con 0x7f9280105b70 2024-09-10T11:20:28.876 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.032+0000 7f9284f6e700 1 -- 172.21.15.144:0/2154334069 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f927c011af0 con 0x7f9280105b70 2024-09-10T11:20:28.876 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.032+0000 7f92881d4700 1 -- 172.21.15.144:0/2154334069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9280105b70 msgr2=0x7f9280105f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:28.876 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.032+0000 7f92881d4700 1 --2- 172.21.15.144:0/2154334069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9280105b70 0x7f9280105f80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f927c0098a0 tx=0x7f927c009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:28.876 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.032+0000 7f92881d4700 1 -- 172.21.15.144:0/2154334069 shutdown_connections 2024-09-10T11:20:28.876 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.032+0000 7f92881d4700 1 --2- 172.21.15.144:0/2154334069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9280105b70 0x7f9280105f80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:28.876 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.032+0000 7f92881d4700 1 -- 172.21.15.144:0/2154334069 >> 172.21.15.144:0/2154334069 conn(0x7f9280101220 msgr2=0x7f9280103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:28.876 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.033+0000 7f92881d4700 1 -- 172.21.15.144:0/2154334069 shutdown_connections 2024-09-10T11:20:28.876 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.033+0000 7f92881d4700 1 -- 172.21.15.144:0/2154334069 wait complete. 2024-09-10T11:20:28.877 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.033+0000 7f92881d4700 1 Processor -- start 2024-09-10T11:20:28.877 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.033+0000 7f92881d4700 1 -- start start 2024-09-10T11:20:28.877 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.033+0000 7f92881d4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9280105b70 0x7f928007bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:28.877 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.033+0000 7f92881d4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f928007c1b0 con 0x7f9280105b70 2024-09-10T11:20:28.877 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.033+0000 7f9285f70700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9280105b70 0x7f928007bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:28.877 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.033+0000 7f9285f70700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9280105b70 0x7f928007bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40330/0 (socket says 172.21.15.144:40330) 2024-09-10T11:20:28.877 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.033+0000 7f9285f70700 1 -- 172.21.15.144:0/2170726732 learned_addr learned my addr 172.21.15.144:0/2170726732 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:28.877 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.034+0000 7f9285f70700 1 -- 172.21.15.144:0/2170726732 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f927c009550 con 0x7f9280105b70 2024-09-10T11:20:28.877 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.034+0000 7f9285f70700 1 --2- 172.21.15.144:0/2170726732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9280105b70 0x7f928007bc70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f927c011ad0 tx=0x7f927c011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:28.877 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.034+0000 7f9272ffd700 1 -- 172.21.15.144:0/2170726732 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f927c003ab0 con 0x7f9280105b70 2024-09-10T11:20:28.877 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.034+0000 7f9272ffd700 1 -- 172.21.15.144:0/2170726732 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f927c003c10 con 0x7f9280105b70 2024-09-10T11:20:28.877 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.034+0000 7f92881d4700 1 -- 172.21.15.144:0/2170726732 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f928007a2c0 con 0x7f9280105b70 2024-09-10T11:20:28.877 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.034+0000 7f9272ffd700 1 -- 172.21.15.144:0/2170726732 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f927c0198e0 con 0x7f9280105b70 2024-09-10T11:20:28.878 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.034+0000 7f92881d4700 1 -- 172.21.15.144:0/2170726732 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f928007a6e0 con 0x7f9280105b70 2024-09-10T11:20:28.878 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.035+0000 7f9272ffd700 1 -- 172.21.15.144:0/2170726732 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f927c019b50 con 0x7f9280105b70 2024-09-10T11:20:28.878 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.035+0000 7f92881d4700 1 -- 172.21.15.144:0/2170726732 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9280062380 con 0x7f9280105b70 2024-09-10T11:20:28.878 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.035+0000 7f9272ffd700 1 --2- 172.21.15.144:0/2170726732 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f926c038440 0x7f926c03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:28.878 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.035+0000 7f9272ffd700 1 -- 172.21.15.144:0/2170726732 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f927c04c210 con 0x7f9280105b70 2024-09-10T11:20:28.878 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.035+0000 7f928576f700 1 --2- 172.21.15.144:0/2170726732 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f926c038440 0x7f926c03a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:28.878 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.036+0000 7f928576f700 1 --2- 172.21.15.144:0/2170726732 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f926c038440 0x7f926c03a8f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9274006fd0 tx=0x7f9274006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:28.878 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.040+0000 7f9272ffd700 1 -- 172.21.15.144:0/2170726732 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f927c01a430 con 0x7f9280105b70 2024-09-10T11:20:28.878 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.240+0000 7f92881d4700 1 -- 172.21.15.144:0/2170726732 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f9280102700 con 0x7f926c038440 2024-09-10T11:20:28.878 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.244+0000 7f9272ffd700 1 -- 172.21.15.144:0/2170726732 <== mgr.14120 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f9280102700 con 0x7f926c038440 2024-09-10T11:20:28.878 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.246+0000 7f92881d4700 1 -- 172.21.15.144:0/2170726732 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f926c038440 msgr2=0x7f926c03a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:28.878 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.246+0000 7f92881d4700 1 --2- 172.21.15.144:0/2170726732 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f926c038440 0x7f926c03a8f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9274006fd0 tx=0x7f9274006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:28.878 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.246+0000 7f92881d4700 1 -- 172.21.15.144:0/2170726732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9280105b70 msgr2=0x7f928007bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:28.879 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.246+0000 7f92881d4700 1 --2- 172.21.15.144:0/2170726732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9280105b70 0x7f928007bc70 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f927c011ad0 tx=0x7f927c011bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:28.879 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.246+0000 7f92881d4700 1 -- 172.21.15.144:0/2170726732 shutdown_connections 2024-09-10T11:20:28.879 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.246+0000 7f92881d4700 1 --2- 172.21.15.144:0/2170726732 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f926c038440 0x7f926c03a8f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:28.879 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.246+0000 7f92881d4700 1 --2- 172.21.15.144:0/2170726732 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9280105b70 0x7f928007bc70 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:28.879 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.246+0000 7f92881d4700 1 -- 172.21.15.144:0/2170726732 >> 172.21.15.144:0/2170726732 conn(0x7f9280101220 msgr2=0x7f9280101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:28.879 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.246+0000 7f92881d4700 1 -- 172.21.15.144:0/2170726732 shutdown_connections 2024-09-10T11:20:28.879 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:28.246+0000 7f92881d4700 1 -- 172.21.15.144:0/2170726732 wait complete. 2024-09-10T11:20:28.879 INFO:teuthology.orchestra.run.smithi144.stdout:Deploying grafana service with default placement... 2024-09-10T11:20:29.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:29 smithi144 ceph-mon[27942]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:20:29.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:29 smithi144 ceph-mon[27942]: Saving service prometheus spec with placement count:1 2024-09-10T11:20:29.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:29 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:30.114 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-09-10T11:20:30.114 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.283+0000 7f218a1cb700 1 Processor -- start 2024-09-10T11:20:30.115 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.283+0000 7f218a1cb700 1 -- start start 2024-09-10T11:20:30.115 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.283+0000 7f218a1cb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2184103e70 0x7f2184104280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:30.115 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.283+0000 7f218a1cb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21841047c0 con 0x7f2184103e70 2024-09-10T11:20:30.115 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.283+0000 7f21837fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2184103e70 0x7f2184104280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:30.115 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.283+0000 7f21837fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2184103e70 0x7f2184104280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40346/0 (socket says 172.21.15.144:40346) 2024-09-10T11:20:30.115 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.283+0000 7f21837fe700 1 -- 172.21.15.144:0/2700840555 learned_addr learned my addr 172.21.15.144:0/2700840555 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:30.115 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.283+0000 7f21837fe700 1 -- 172.21.15.144:0/2700840555 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2184104900 con 0x7f2184103e70 2024-09-10T11:20:30.115 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.284+0000 7f21837fe700 1 --2- 172.21.15.144:0/2700840555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2184103e70 0x7f2184104280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f21740098a0 tx=0x7f2174009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7d7df6f9bbf6e7f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:30.115 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.284+0000 7f21827fc700 1 -- 172.21.15.144:0/2700840555 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2174004180 con 0x7f2184103e70 2024-09-10T11:20:30.115 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.284+0000 7f21827fc700 1 -- 172.21.15.144:0/2700840555 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f21740042e0 con 0x7f2184103e70 2024-09-10T11:20:30.115 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.284+0000 7f21827fc700 1 -- 172.21.15.144:0/2700840555 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2174011af0 con 0x7f2184103e70 2024-09-10T11:20:30.115 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.284+0000 7f218a1cb700 1 -- 172.21.15.144:0/2700840555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2184103e70 msgr2=0x7f2184104280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:30.115 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.284+0000 7f218a1cb700 1 --2- 172.21.15.144:0/2700840555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2184103e70 0x7f2184104280 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f21740098a0 tx=0x7f2174009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:30.116 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.284+0000 7f218a1cb700 1 -- 172.21.15.144:0/2700840555 shutdown_connections 2024-09-10T11:20:30.116 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.284+0000 7f218a1cb700 1 --2- 172.21.15.144:0/2700840555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2184103e70 0x7f2184104280 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:30.116 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.284+0000 7f218a1cb700 1 -- 172.21.15.144:0/2700840555 >> 172.21.15.144:0/2700840555 conn(0x7f21840ff870 msgr2=0x7f2184101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:30.116 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.284+0000 7f218a1cb700 1 -- 172.21.15.144:0/2700840555 shutdown_connections 2024-09-10T11:20:30.116 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.284+0000 7f218a1cb700 1 -- 172.21.15.144:0/2700840555 wait complete. 2024-09-10T11:20:30.116 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.285+0000 7f218a1cb700 1 Processor -- start 2024-09-10T11:20:30.116 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.285+0000 7f218a1cb700 1 -- start start 2024-09-10T11:20:30.116 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.285+0000 7f218a1cb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2184103e70 0x7f2184199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:30.116 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.285+0000 7f218a1cb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2184199e70 con 0x7f2184103e70 2024-09-10T11:20:30.116 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.285+0000 7f21837fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2184103e70 0x7f2184199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:30.116 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.285+0000 7f21837fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2184103e70 0x7f2184199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40360/0 (socket says 172.21.15.144:40360) 2024-09-10T11:20:30.116 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.285+0000 7f21837fe700 1 -- 172.21.15.144:0/910763979 learned_addr learned my addr 172.21.15.144:0/910763979 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:30.116 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.286+0000 7f21837fe700 1 -- 172.21.15.144:0/910763979 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2174009550 con 0x7f2184103e70 2024-09-10T11:20:30.116 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.286+0000 7f21837fe700 1 --2- 172.21.15.144:0/910763979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2184103e70 0x7f2184199930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2174000c00 tx=0x7f2174011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:30.117 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.286+0000 7f2180ff9700 1 -- 172.21.15.144:0/910763979 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2174003a70 con 0x7f2184103e70 2024-09-10T11:20:30.117 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.286+0000 7f2180ff9700 1 -- 172.21.15.144:0/910763979 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2174003bd0 con 0x7f2184103e70 2024-09-10T11:20:30.117 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.286+0000 7f218a1cb700 1 -- 172.21.15.144:0/910763979 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218419a070 con 0x7f2184103e70 2024-09-10T11:20:30.117 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.286+0000 7f2180ff9700 1 -- 172.21.15.144:0/910763979 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21740238c0 con 0x7f2184103e70 2024-09-10T11:20:30.117 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.286+0000 7f218a1cb700 1 -- 172.21.15.144:0/910763979 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218419a490 con 0x7f2184103e70 2024-09-10T11:20:30.117 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.287+0000 7f2180ff9700 1 -- 172.21.15.144:0/910763979 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f2174023b80 con 0x7f2184103e70 2024-09-10T11:20:30.117 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.287+0000 7f2180ff9700 1 --2- 172.21.15.144:0/910763979 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f216c0383f0 0x7f216c03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:30.117 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.287+0000 7f2180ff9700 1 -- 172.21.15.144:0/910763979 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f217404c220 con 0x7f2184103e70 2024-09-10T11:20:30.117 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.287+0000 7f218a1cb700 1 -- 172.21.15.144:0/910763979 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2184062380 con 0x7f2184103e70 2024-09-10T11:20:30.117 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.287+0000 7f2182ffd700 1 --2- 172.21.15.144:0/910763979 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f216c0383f0 0x7f216c03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:30.117 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.287+0000 7f2182ffd700 1 --2- 172.21.15.144:0/910763979 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f216c0383f0 0x7f216c03a8a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2178006fd0 tx=0x7f2178006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:30.117 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.292+0000 7f2180ff9700 1 -- 172.21.15.144:0/910763979 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f217402b300 con 0x7f2184103e70 2024-09-10T11:20:30.117 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.494+0000 7f218a1cb700 1 -- 172.21.15.144:0/910763979 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f2184101b20 con 0x7f216c0383f0 2024-09-10T11:20:30.118 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.499+0000 7f2180ff9700 1 -- 172.21.15.144:0/910763979 <== mgr.14120 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f2184101b20 con 0x7f216c0383f0 2024-09-10T11:20:30.118 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.502+0000 7f218a1cb700 1 -- 172.21.15.144:0/910763979 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f216c0383f0 msgr2=0x7f216c03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:30.118 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.502+0000 7f218a1cb700 1 --2- 172.21.15.144:0/910763979 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f216c0383f0 0x7f216c03a8a0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2178006fd0 tx=0x7f2178006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:30.118 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.502+0000 7f218a1cb700 1 -- 172.21.15.144:0/910763979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2184103e70 msgr2=0x7f2184199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:30.118 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.502+0000 7f218a1cb700 1 --2- 172.21.15.144:0/910763979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2184103e70 0x7f2184199930 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2174000c00 tx=0x7f2174011b50 comp rx=0 tx=0).stop 2024-09-10T11:20:30.118 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.502+0000 7f218a1cb700 1 -- 172.21.15.144:0/910763979 shutdown_connections 2024-09-10T11:20:30.118 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.502+0000 7f218a1cb700 1 --2- 172.21.15.144:0/910763979 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f216c0383f0 0x7f216c03a8a0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:30.118 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.502+0000 7f218a1cb700 1 --2- 172.21.15.144:0/910763979 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2184103e70 0x7f2184199930 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:30.118 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.502+0000 7f218a1cb700 1 -- 172.21.15.144:0/910763979 >> 172.21.15.144:0/910763979 conn(0x7f21840ff870 msgr2=0x7f2184100ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:30.118 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.502+0000 7f218a1cb700 1 -- 172.21.15.144:0/910763979 shutdown_connections 2024-09-10T11:20:30.118 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:29.502+0000 7f218a1cb700 1 -- 172.21.15.144:0/910763979 wait complete. 2024-09-10T11:20:30.118 INFO:teuthology.orchestra.run.smithi144.stdout:Deploying node-exporter service with default placement... 2024-09-10T11:20:30.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:30 smithi144 ceph-mon[27942]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:20:30.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:30 smithi144 ceph-mon[27942]: Saving service grafana spec with placement count:1 2024-09-10T11:20:30.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:30 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:30.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:30 smithi144 ceph-mon[27942]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:20:31.296 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-09-10T11:20:31.296 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.506+0000 7f999b6e7700 1 Processor -- start 2024-09-10T11:20:31.296 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.506+0000 7f999b6e7700 1 -- start start 2024-09-10T11:20:31.296 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.506+0000 7f999b6e7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9994104ae0 0x7f9994104ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:31.296 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.506+0000 7f999b6e7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9994105430 con 0x7f9994104ae0 2024-09-10T11:20:31.296 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.507+0000 7f9999483700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9994104ae0 0x7f9994104ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:31.296 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.507+0000 7f9999483700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9994104ae0 0x7f9994104ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40368/0 (socket says 172.21.15.144:40368) 2024-09-10T11:20:31.296 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.507+0000 7f9999483700 1 -- 172.21.15.144:0/740316097 learned_addr learned my addr 172.21.15.144:0/740316097 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:31.296 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.507+0000 7f9999483700 1 -- 172.21.15.144:0/740316097 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9994105570 con 0x7f9994104ae0 2024-09-10T11:20:31.296 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.507+0000 7f9999483700 1 --2- 172.21.15.144:0/740316097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9994104ae0 0x7f9994104ef0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9990009e80 tx=0x7f999000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fdecd8f39ae4a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:31.297 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.507+0000 7f998bfff700 1 -- 172.21.15.144:0/740316097 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f999000e070 con 0x7f9994104ae0 2024-09-10T11:20:31.297 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.507+0000 7f998bfff700 1 -- 172.21.15.144:0/740316097 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f999000b8c0 con 0x7f9994104ae0 2024-09-10T11:20:31.297 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.507+0000 7f998bfff700 1 -- 172.21.15.144:0/740316097 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99900044d0 con 0x7f9994104ae0 2024-09-10T11:20:31.297 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.508+0000 7f999b6e7700 1 -- 172.21.15.144:0/740316097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9994104ae0 msgr2=0x7f9994104ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:31.297 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.508+0000 7f999b6e7700 1 --2- 172.21.15.144:0/740316097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9994104ae0 0x7f9994104ef0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9990009e80 tx=0x7f999000b330 comp rx=0 tx=0).stop 2024-09-10T11:20:31.297 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.508+0000 7f999b6e7700 1 -- 172.21.15.144:0/740316097 shutdown_connections 2024-09-10T11:20:31.297 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.508+0000 7f999b6e7700 1 --2- 172.21.15.144:0/740316097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9994104ae0 0x7f9994104ef0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:31.297 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.508+0000 7f999b6e7700 1 -- 172.21.15.144:0/740316097 >> 172.21.15.144:0/740316097 conn(0x7f9994100130 msgr2=0x7f9994102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:31.297 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.508+0000 7f999b6e7700 1 -- 172.21.15.144:0/740316097 shutdown_connections 2024-09-10T11:20:31.297 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.508+0000 7f999b6e7700 1 -- 172.21.15.144:0/740316097 wait complete. 2024-09-10T11:20:31.297 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.508+0000 7f999b6e7700 1 Processor -- start 2024-09-10T11:20:31.297 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.509+0000 7f999b6e7700 1 -- start start 2024-09-10T11:20:31.297 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.509+0000 7f999b6e7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9994104ae0 0x7f9994079b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:31.297 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.509+0000 7f999b6e7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f999419c620 con 0x7f9994104ae0 2024-09-10T11:20:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.509+0000 7f9999483700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9994104ae0 0x7f9994079b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.509+0000 7f9999483700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9994104ae0 0x7f9994079b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40378/0 (socket says 172.21.15.144:40378) 2024-09-10T11:20:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.509+0000 7f9999483700 1 -- 172.21.15.144:0/1477889515 learned_addr learned my addr 172.21.15.144:0/1477889515 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.509+0000 7f9999483700 1 -- 172.21.15.144:0/1477889515 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9990009b30 con 0x7f9994104ae0 2024-09-10T11:20:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.509+0000 7f9999483700 1 --2- 172.21.15.144:0/1477889515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9994104ae0 0x7f9994079b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f99900072a0 tx=0x7f99900049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.509+0000 7f998a7fc700 1 -- 172.21.15.144:0/1477889515 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f999000e040 con 0x7f9994104ae0 2024-09-10T11:20:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.509+0000 7f998a7fc700 1 -- 172.21.15.144:0/1477889515 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f999000bd80 con 0x7f9994104ae0 2024-09-10T11:20:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.509+0000 7f999b6e7700 1 -- 172.21.15.144:0/1477889515 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f999419c760 con 0x7f9994104ae0 2024-09-10T11:20:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.509+0000 7f998a7fc700 1 -- 172.21.15.144:0/1477889515 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f99900126c0 con 0x7f9994104ae0 2024-09-10T11:20:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.509+0000 7f999b6e7700 1 -- 172.21.15.144:0/1477889515 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f999419cbe0 con 0x7f9994104ae0 2024-09-10T11:20:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.510+0000 7f998a7fc700 1 -- 172.21.15.144:0/1477889515 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f9990019040 con 0x7f9994104ae0 2024-09-10T11:20:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.510+0000 7f998a7fc700 1 --2- 172.21.15.144:0/1477889515 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f99800383b0 0x7f998003a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:31.298 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.510+0000 7f998a7fc700 1 -- 172.21.15.144:0/1477889515 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f999004cda0 con 0x7f9994104ae0 2024-09-10T11:20:31.299 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.510+0000 7f9998c82700 1 --2- 172.21.15.144:0/1477889515 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f99800383b0 0x7f998003a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:31.299 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.510+0000 7f999b6e7700 1 -- 172.21.15.144:0/1477889515 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9994062380 con 0x7f9994104ae0 2024-09-10T11:20:31.299 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.511+0000 7f9998c82700 1 --2- 172.21.15.144:0/1477889515 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f99800383b0 0x7f998003a860 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9984006fd0 tx=0x7f9984006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:31.299 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.515+0000 7f998a7fc700 1 -- 172.21.15.144:0/1477889515 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f999001f080 con 0x7f9994104ae0 2024-09-10T11:20:31.299 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.717+0000 7f999b6e7700 1 -- 172.21.15.144:0/1477889515 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f999419d190 con 0x7f99800383b0 2024-09-10T11:20:31.299 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.721+0000 7f998a7fc700 1 -- 172.21.15.144:0/1477889515 <== mgr.14120 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f999419d190 con 0x7f99800383b0 2024-09-10T11:20:31.299 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.723+0000 7f999b6e7700 1 -- 172.21.15.144:0/1477889515 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f99800383b0 msgr2=0x7f998003a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:31.299 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.723+0000 7f999b6e7700 1 --2- 172.21.15.144:0/1477889515 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f99800383b0 0x7f998003a860 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9984006fd0 tx=0x7f9984006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:31.299 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.723+0000 7f999b6e7700 1 -- 172.21.15.144:0/1477889515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9994104ae0 msgr2=0x7f9994079b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:31.299 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.723+0000 7f999b6e7700 1 --2- 172.21.15.144:0/1477889515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9994104ae0 0x7f9994079b50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f99900072a0 tx=0x7f99900049f0 comp rx=0 tx=0).stop 2024-09-10T11:20:31.299 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.724+0000 7f999b6e7700 1 -- 172.21.15.144:0/1477889515 shutdown_connections 2024-09-10T11:20:31.299 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.724+0000 7f999b6e7700 1 --2- 172.21.15.144:0/1477889515 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f99800383b0 0x7f998003a860 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:31.299 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.724+0000 7f999b6e7700 1 --2- 172.21.15.144:0/1477889515 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9994104ae0 0x7f9994079b50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:31.300 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.724+0000 7f999b6e7700 1 -- 172.21.15.144:0/1477889515 >> 172.21.15.144:0/1477889515 conn(0x7f9994100130 msgr2=0x7f9994100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:31.300 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.724+0000 7f999b6e7700 1 -- 172.21.15.144:0/1477889515 shutdown_connections 2024-09-10T11:20:31.300 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:30.724+0000 7f999b6e7700 1 -- 172.21.15.144:0/1477889515 wait complete. 2024-09-10T11:20:31.300 INFO:teuthology.orchestra.run.smithi144.stdout:Deploying alertmanager service with default placement... 2024-09-10T11:20:31.919 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:31 smithi144 ceph-mon[27942]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:20:31.919 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:31 smithi144 ceph-mon[27942]: Saving service node-exporter spec with placement * 2024-09-10T11:20:31.919 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:31 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:32.506 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-09-10T11:20:32.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.706+0000 7f201b242700 1 Processor -- start 2024-09-10T11:20:32.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.706+0000 7f201b242700 1 -- start start 2024-09-10T11:20:32.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.706+0000 7f201b242700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2014107240 0x7f2014107650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:32.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.706+0000 7f201b242700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2014107b90 con 0x7f2014107240 2024-09-10T11:20:32.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.706+0000 7f2018fde700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2014107240 0x7f2014107650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:32.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.706+0000 7f2018fde700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2014107240 0x7f2014107650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40380/0 (socket says 172.21.15.144:40380) 2024-09-10T11:20:32.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.706+0000 7f2018fde700 1 -- 172.21.15.144:0/330811532 learned_addr learned my addr 172.21.15.144:0/330811532 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:32.578 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.706+0000 7f2018fde700 1 -- 172.21.15.144:0/330811532 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2014107cd0 con 0x7f2014107240 2024-09-10T11:20:32.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.707+0000 7f2018fde700 1 --2- 172.21.15.144:0/330811532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2014107240 0x7f2014107650 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f20080098a0 tx=0x7f2008009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ca3a845c6a25a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:32.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.707+0000 7f20137fe700 1 -- 172.21.15.144:0/330811532 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2008004180 con 0x7f2014107240 2024-09-10T11:20:32.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.707+0000 7f20137fe700 1 -- 172.21.15.144:0/330811532 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f20080042e0 con 0x7f2014107240 2024-09-10T11:20:32.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.707+0000 7f20137fe700 1 -- 172.21.15.144:0/330811532 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2008011af0 con 0x7f2014107240 2024-09-10T11:20:32.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.707+0000 7f201b242700 1 -- 172.21.15.144:0/330811532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2014107240 msgr2=0x7f2014107650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:32.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.707+0000 7f201b242700 1 --2- 172.21.15.144:0/330811532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2014107240 0x7f2014107650 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f20080098a0 tx=0x7f2008009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:32.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.707+0000 7f201b242700 1 -- 172.21.15.144:0/330811532 shutdown_connections 2024-09-10T11:20:32.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.707+0000 7f201b242700 1 --2- 172.21.15.144:0/330811532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2014107240 0x7f2014107650 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:32.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.707+0000 7f201b242700 1 -- 172.21.15.144:0/330811532 >> 172.21.15.144:0/330811532 conn(0x7f20140ff870 msgr2=0x7f2014101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:32.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.707+0000 7f201b242700 1 -- 172.21.15.144:0/330811532 shutdown_connections 2024-09-10T11:20:32.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.707+0000 7f201b242700 1 -- 172.21.15.144:0/330811532 wait complete. 2024-09-10T11:20:32.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.708+0000 7f201b242700 1 Processor -- start 2024-09-10T11:20:32.579 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.708+0000 7f201b242700 1 -- start start 2024-09-10T11:20:32.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.708+0000 7f201b242700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2014107240 0x7f2014199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:32.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.708+0000 7f201b242700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2014199e50 con 0x7f2014107240 2024-09-10T11:20:32.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.708+0000 7f2018fde700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2014107240 0x7f2014199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:32.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.708+0000 7f2018fde700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2014107240 0x7f2014199910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40382/0 (socket says 172.21.15.144:40382) 2024-09-10T11:20:32.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.708+0000 7f2018fde700 1 -- 172.21.15.144:0/954912831 learned_addr learned my addr 172.21.15.144:0/954912831 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:32.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.709+0000 7f2018fde700 1 -- 172.21.15.144:0/954912831 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2008009550 con 0x7f2014107240 2024-09-10T11:20:32.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.709+0000 7f2018fde700 1 --2- 172.21.15.144:0/954912831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2014107240 0x7f2014199910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2008011ad0 tx=0x7f2008011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:32.580 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.709+0000 7f2011ffb700 1 -- 172.21.15.144:0/954912831 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2008003ab0 con 0x7f2014107240 2024-09-10T11:20:32.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.709+0000 7f2011ffb700 1 -- 172.21.15.144:0/954912831 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2008003c10 con 0x7f2014107240 2024-09-10T11:20:32.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.709+0000 7f201b242700 1 -- 172.21.15.144:0/954912831 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f201419a050 con 0x7f2014107240 2024-09-10T11:20:32.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.709+0000 7f2011ffb700 1 -- 172.21.15.144:0/954912831 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f20080198e0 con 0x7f2014107240 2024-09-10T11:20:32.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.709+0000 7f201b242700 1 -- 172.21.15.144:0/954912831 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f201419a470 con 0x7f2014107240 2024-09-10T11:20:32.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.710+0000 7f2011ffb700 1 -- 172.21.15.144:0/954912831 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f2008019ba0 con 0x7f2014107240 2024-09-10T11:20:32.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.710+0000 7f201b242700 1 -- 172.21.15.144:0/954912831 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2014062380 con 0x7f2014107240 2024-09-10T11:20:32.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.710+0000 7f2011ffb700 1 --2- 172.21.15.144:0/954912831 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f20040383f0 0x7f200403a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:32.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.710+0000 7f2011ffb700 1 -- 172.21.15.144:0/954912831 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f200804c360 con 0x7f2014107240 2024-09-10T11:20:32.581 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.710+0000 7f2013fff700 1 --2- 172.21.15.144:0/954912831 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f20040383f0 0x7f200403a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:32.582 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.711+0000 7f2013fff700 1 --2- 172.21.15.144:0/954912831 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f20040383f0 0x7f200403a8a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2000006fd0 tx=0x7f2000006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:32.582 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.715+0000 7f2011ffb700 1 -- 172.21.15.144:0/954912831 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f200802b300 con 0x7f2014107240 2024-09-10T11:20:32.582 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.917+0000 7f201b242700 1 -- 172.21.15.144:0/954912831 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f2014101b10 con 0x7f20040383f0 2024-09-10T11:20:32.582 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.922+0000 7f2011ffb700 1 -- 172.21.15.144:0/954912831 <== mgr.14120 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f2014101b10 con 0x7f20040383f0 2024-09-10T11:20:32.582 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.924+0000 7f201b242700 1 -- 172.21.15.144:0/954912831 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f20040383f0 msgr2=0x7f200403a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:32.582 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.924+0000 7f201b242700 1 --2- 172.21.15.144:0/954912831 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f20040383f0 0x7f200403a8a0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f2000006fd0 tx=0x7f2000006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:32.582 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.924+0000 7f201b242700 1 -- 172.21.15.144:0/954912831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2014107240 msgr2=0x7f2014199910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:32.582 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.924+0000 7f201b242700 1 --2- 172.21.15.144:0/954912831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2014107240 0x7f2014199910 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2008011ad0 tx=0x7f2008011bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:32.582 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.924+0000 7f201b242700 1 -- 172.21.15.144:0/954912831 shutdown_connections 2024-09-10T11:20:32.582 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.924+0000 7f201b242700 1 --2- 172.21.15.144:0/954912831 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f20040383f0 0x7f200403a8a0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:32.582 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.924+0000 7f201b242700 1 --2- 172.21.15.144:0/954912831 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2014107240 0x7f2014199910 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:32.582 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.924+0000 7f201b242700 1 -- 172.21.15.144:0/954912831 >> 172.21.15.144:0/954912831 conn(0x7f20140ff870 msgr2=0x7f2014100ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:32.583 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.924+0000 7f201b242700 1 -- 172.21.15.144:0/954912831 shutdown_connections 2024-09-10T11:20:32.583 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:31.924+0000 7f201b242700 1 -- 172.21.15.144:0/954912831 wait complete. 2024-09-10T11:20:33.124 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:32 smithi144 ceph-mon[27942]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:20:33.124 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:32 smithi144 ceph-mon[27942]: Saving service alertmanager spec with placement count:1 2024-09-10T11:20:33.125 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:32 smithi144 ceph-mon[27942]: from='mgr.14120 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:33.125 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:32 smithi144 ceph-mon[27942]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:20:33.769 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.902+0000 7ff5ec9a2700 1 Processor -- start 2024-09-10T11:20:33.769 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.902+0000 7ff5ec9a2700 1 -- start start 2024-09-10T11:20:33.769 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.902+0000 7ff5ec9a2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8106510 0x7ff5e81088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:33.769 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.902+0000 7ff5ec9a2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e8079d40 con 0x7ff5e8106510 2024-09-10T11:20:33.769 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.903+0000 7ff5e77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8106510 0x7ff5e81088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:33.769 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.903+0000 7ff5e77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8106510 0x7ff5e81088f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51226/0 (socket says 172.21.15.144:51226) 2024-09-10T11:20:33.770 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.903+0000 7ff5e77fe700 1 -- 172.21.15.144:0/286084348 learned_addr learned my addr 172.21.15.144:0/286084348 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:33.770 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.903+0000 7ff5e77fe700 1 -- 172.21.15.144:0/286084348 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5e8079e80 con 0x7ff5e8106510 2024-09-10T11:20:33.770 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.903+0000 7ff5e77fe700 1 --2- 172.21.15.144:0/286084348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8106510 0x7ff5e81088f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00098a0 tx=0x7ff5d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9e4702781887ef server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:33.770 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.903+0000 7ff5e67fc700 1 -- 172.21.15.144:0/286084348 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5d0004180 con 0x7ff5e8106510 2024-09-10T11:20:33.770 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.903+0000 7ff5e67fc700 1 -- 172.21.15.144:0/286084348 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff5d00042e0 con 0x7ff5e8106510 2024-09-10T11:20:33.770 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.903+0000 7ff5e67fc700 1 -- 172.21.15.144:0/286084348 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5d0011af0 con 0x7ff5e8106510 2024-09-10T11:20:33.770 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.904+0000 7ff5ec9a2700 1 -- 172.21.15.144:0/286084348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8106510 msgr2=0x7ff5e81088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:33.770 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.904+0000 7ff5ec9a2700 1 --2- 172.21.15.144:0/286084348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8106510 0x7ff5e81088f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff5d00098a0 tx=0x7ff5d0009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:33.770 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.904+0000 7ff5ec9a2700 1 -- 172.21.15.144:0/286084348 shutdown_connections 2024-09-10T11:20:33.770 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.904+0000 7ff5ec9a2700 1 --2- 172.21.15.144:0/286084348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8106510 0x7ff5e81088f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:33.770 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.904+0000 7ff5ec9a2700 1 -- 172.21.15.144:0/286084348 >> 172.21.15.144:0/286084348 conn(0x7ff5e80fff10 msgr2=0x7ff5e8102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:33.770 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.904+0000 7ff5ec9a2700 1 -- 172.21.15.144:0/286084348 shutdown_connections 2024-09-10T11:20:33.770 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.904+0000 7ff5ec9a2700 1 -- 172.21.15.144:0/286084348 wait complete. 2024-09-10T11:20:33.771 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.904+0000 7ff5ec9a2700 1 Processor -- start 2024-09-10T11:20:33.771 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.905+0000 7ff5ec9a2700 1 -- start start 2024-09-10T11:20:33.771 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.905+0000 7ff5ec9a2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8106510 0x7ff5e81996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:33.771 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.905+0000 7ff5ec9a2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5e8199c30 con 0x7ff5e8106510 2024-09-10T11:20:33.771 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.905+0000 7ff5e77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8106510 0x7ff5e81996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:33.771 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.905+0000 7ff5e77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8106510 0x7ff5e81996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51238/0 (socket says 172.21.15.144:51238) 2024-09-10T11:20:33.771 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.905+0000 7ff5e77fe700 1 -- 172.21.15.144:0/2555270803 learned_addr learned my addr 172.21.15.144:0/2555270803 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:33.771 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.905+0000 7ff5e77fe700 1 -- 172.21.15.144:0/2555270803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5d0009550 con 0x7ff5e8106510 2024-09-10T11:20:33.771 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.905+0000 7ff5e77fe700 1 --2- 172.21.15.144:0/2555270803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8106510 0x7ff5e81996f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0006ba0 tx=0x7ff5d0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:33.771 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.905+0000 7ff5e4ff9700 1 -- 172.21.15.144:0/2555270803 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5d0003b00 con 0x7ff5e8106510 2024-09-10T11:20:33.772 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.905+0000 7ff5e4ff9700 1 -- 172.21.15.144:0/2555270803 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff5d001a8e0 con 0x7ff5e8106510 2024-09-10T11:20:33.772 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.905+0000 7ff5ec9a2700 1 -- 172.21.15.144:0/2555270803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5e8199e30 con 0x7ff5e8106510 2024-09-10T11:20:33.772 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.905+0000 7ff5e4ff9700 1 -- 172.21.15.144:0/2555270803 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5d0019990 con 0x7ff5e8106510 2024-09-10T11:20:33.772 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.906+0000 7ff5ec9a2700 1 -- 172.21.15.144:0/2555270803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5e819a250 con 0x7ff5e8106510 2024-09-10T11:20:33.772 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.906+0000 7ff5e4ff9700 1 -- 172.21.15.144:0/2555270803 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7ff5d0019c00 con 0x7ff5e8106510 2024-09-10T11:20:33.772 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.907+0000 7ff5ec9a2700 1 -- 172.21.15.144:0/2555270803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5e8062380 con 0x7ff5e8106510 2024-09-10T11:20:33.772 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.907+0000 7ff5e4ff9700 1 --2- 172.21.15.144:0/2555270803 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff5d40383f0 0x7ff5d403a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:33.772 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.907+0000 7ff5e4ff9700 1 -- 172.21.15.144:0/2555270803 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff5d004c750 con 0x7ff5e8106510 2024-09-10T11:20:33.772 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.907+0000 7ff5e6ffd700 1 --2- 172.21.15.144:0/2555270803 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff5d40383f0 0x7ff5d403a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:33.772 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.907+0000 7ff5e6ffd700 1 --2- 172.21.15.144:0/2555270803 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff5d40383f0 0x7ff5d403a8a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8006fd0 tx=0x7ff5d8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:33.772 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:32.912+0000 7ff5e4ff9700 1 -- 172.21.15.144:0/2555270803 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff5d002d440 con 0x7ff5e8106510 2024-09-10T11:20:33.772 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:33.103+0000 7ff5ec9a2700 1 -- 172.21.15.144:0/2555270803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7ff5e819d140 con 0x7ff5e8106510 2024-09-10T11:20:33.772 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:33.107+0000 7ff5e4ff9700 1 -- 172.21.15.144:0/2555270803 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7ff5d002b340 con 0x7ff5e8106510 2024-09-10T11:20:33.773 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:33.110+0000 7ff5ec9a2700 1 -- 172.21.15.144:0/2555270803 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff5d40383f0 msgr2=0x7ff5d403a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:33.773 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:33.110+0000 7ff5ec9a2700 1 --2- 172.21.15.144:0/2555270803 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff5d40383f0 0x7ff5d403a8a0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff5d8006fd0 tx=0x7ff5d8006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:33.773 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:33.110+0000 7ff5ec9a2700 1 -- 172.21.15.144:0/2555270803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8106510 msgr2=0x7ff5e81996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:33.773 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:33.110+0000 7ff5ec9a2700 1 --2- 172.21.15.144:0/2555270803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8106510 0x7ff5e81996f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff5d0006ba0 tx=0x7ff5d0011b50 comp rx=0 tx=0).stop 2024-09-10T11:20:33.773 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:33.110+0000 7ff5ec9a2700 1 -- 172.21.15.144:0/2555270803 shutdown_connections 2024-09-10T11:20:33.773 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:33.110+0000 7ff5ec9a2700 1 --2- 172.21.15.144:0/2555270803 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff5d40383f0 0x7ff5d403a8a0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:33.773 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:33.110+0000 7ff5ec9a2700 1 --2- 172.21.15.144:0/2555270803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5e8106510 0x7ff5e81996f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:33.773 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:33.110+0000 7ff5ec9a2700 1 -- 172.21.15.144:0/2555270803 >> 172.21.15.144:0/2555270803 conn(0x7ff5e80fff10 msgr2=0x7ff5e81015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:33.773 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:33.110+0000 7ff5ec9a2700 1 -- 172.21.15.144:0/2555270803 shutdown_connections 2024-09-10T11:20:33.773 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:33.110+0000 7ff5ec9a2700 1 -- 172.21.15.144:0/2555270803 wait complete. 2024-09-10T11:20:34.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:34 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2555270803' entity='client.admin' 2024-09-10T11:20:34.982 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.201+0000 7fa1b2692700 1 Processor -- start 2024-09-10T11:20:34.983 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.201+0000 7fa1b2692700 1 -- start start 2024-09-10T11:20:34.983 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.201+0000 7fa1b2692700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1ac104a80 0x7fa1ac104e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:34.983 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.201+0000 7fa1b2692700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac1053d0 con 0x7fa1ac104a80 2024-09-10T11:20:34.983 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.201+0000 7fa1abfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1ac104a80 0x7fa1ac104e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:34.983 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.201+0000 7fa1abfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1ac104a80 0x7fa1ac104e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51246/0 (socket says 172.21.15.144:51246) 2024-09-10T11:20:34.983 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.201+0000 7fa1abfff700 1 -- 172.21.15.144:0/4098386382 learned_addr learned my addr 172.21.15.144:0/4098386382 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:34.983 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.202+0000 7fa1abfff700 1 -- 172.21.15.144:0/4098386382 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ac105510 con 0x7fa1ac104a80 2024-09-10T11:20:34.983 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.202+0000 7fa1abfff700 1 --2- 172.21.15.144:0/4098386382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1ac104a80 0x7fa1ac104e90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00098a0 tx=0x7fa1a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4b49086025423d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:34.983 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.202+0000 7fa1aaffd700 1 -- 172.21.15.144:0/4098386382 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1a0004180 con 0x7fa1ac104a80 2024-09-10T11:20:34.983 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.202+0000 7fa1aaffd700 1 -- 172.21.15.144:0/4098386382 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa1a00042e0 con 0x7fa1ac104a80 2024-09-10T11:20:34.983 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.202+0000 7fa1aaffd700 1 -- 172.21.15.144:0/4098386382 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1a0011af0 con 0x7fa1ac104a80 2024-09-10T11:20:34.984 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.202+0000 7fa1b2692700 1 -- 172.21.15.144:0/4098386382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1ac104a80 msgr2=0x7fa1ac104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:34.984 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.202+0000 7fa1b2692700 1 --2- 172.21.15.144:0/4098386382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1ac104a80 0x7fa1ac104e90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00098a0 tx=0x7fa1a0009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:34.984 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.203+0000 7fa1b2692700 1 -- 172.21.15.144:0/4098386382 shutdown_connections 2024-09-10T11:20:34.984 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.203+0000 7fa1b2692700 1 --2- 172.21.15.144:0/4098386382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1ac104a80 0x7fa1ac104e90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:34.984 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.203+0000 7fa1b2692700 1 -- 172.21.15.144:0/4098386382 >> 172.21.15.144:0/4098386382 conn(0x7fa1ac1000f0 msgr2=0x7fa1ac102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:34.984 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.203+0000 7fa1b2692700 1 -- 172.21.15.144:0/4098386382 shutdown_connections 2024-09-10T11:20:34.984 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.203+0000 7fa1b2692700 1 -- 172.21.15.144:0/4098386382 wait complete. 2024-09-10T11:20:34.984 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.203+0000 7fa1b2692700 1 Processor -- start 2024-09-10T11:20:34.984 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.203+0000 7fa1b2692700 1 -- start start 2024-09-10T11:20:34.984 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.204+0000 7fa1b2692700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1ac104a80 0x7fa1ac198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:34.984 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.204+0000 7fa1b2692700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ac198d50 con 0x7fa1ac104a80 2024-09-10T11:20:34.984 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.204+0000 7fa1abfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1ac104a80 0x7fa1ac198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:34.985 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.204+0000 7fa1abfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1ac104a80 0x7fa1ac198810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51252/0 (socket says 172.21.15.144:51252) 2024-09-10T11:20:34.985 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.204+0000 7fa1abfff700 1 -- 172.21.15.144:0/3400717878 learned_addr learned my addr 172.21.15.144:0/3400717878 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:34.985 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.204+0000 7fa1abfff700 1 -- 172.21.15.144:0/3400717878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a0009550 con 0x7fa1ac104a80 2024-09-10T11:20:34.985 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.204+0000 7fa1abfff700 1 --2- 172.21.15.144:0/3400717878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1ac104a80 0x7fa1ac198810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0006ba0 tx=0x7fa1a0011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:34.985 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.204+0000 7fa1a97fa700 1 -- 172.21.15.144:0/3400717878 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1a0003b00 con 0x7fa1ac104a80 2024-09-10T11:20:34.985 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.204+0000 7fa1a97fa700 1 -- 172.21.15.144:0/3400717878 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa1a001a8e0 con 0x7fa1ac104a80 2024-09-10T11:20:34.985 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.204+0000 7fa1a97fa700 1 -- 172.21.15.144:0/3400717878 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1a00199c0 con 0x7fa1ac104a80 2024-09-10T11:20:34.985 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.204+0000 7fa1b2692700 1 -- 172.21.15.144:0/3400717878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ac198f50 con 0x7fa1ac104a80 2024-09-10T11:20:34.985 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.204+0000 7fa1b2692700 1 -- 172.21.15.144:0/3400717878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ac199370 con 0x7fa1ac104a80 2024-09-10T11:20:34.985 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.205+0000 7fa1a97fa700 1 -- 172.21.15.144:0/3400717878 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fa1a0019c80 con 0x7fa1ac104a80 2024-09-10T11:20:34.985 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.205+0000 7fa1b2692700 1 -- 172.21.15.144:0/3400717878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ac108f10 con 0x7fa1ac104a80 2024-09-10T11:20:34.985 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.205+0000 7fa1a97fa700 1 --2- 172.21.15.144:0/3400717878 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa1940383a0 0x7fa19403a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:34.986 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.205+0000 7fa1a97fa700 1 -- 172.21.15.144:0/3400717878 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa1a004c640 con 0x7fa1ac104a80 2024-09-10T11:20:34.986 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.206+0000 7fa1ab7fe700 1 --2- 172.21.15.144:0/3400717878 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa1940383a0 0x7fa19403a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:34.986 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.206+0000 7fa1ab7fe700 1 --2- 172.21.15.144:0/3400717878 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa1940383a0 0x7fa19403a850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa19c006fd0 tx=0x7fa19c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:34.986 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.210+0000 7fa1a97fa700 1 -- 172.21.15.144:0/3400717878 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa1a0018b40 con 0x7fa1ac104a80 2024-09-10T11:20:34.986 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.400+0000 7fa1b2692700 1 -- 172.21.15.144:0/3400717878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fa1ac19c1d0 con 0x7fa1ac104a80 2024-09-10T11:20:34.986 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.404+0000 7fa1a97fa700 1 -- 172.21.15.144:0/3400717878 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7fa1a004c010 con 0x7fa1ac104a80 2024-09-10T11:20:34.986 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.407+0000 7fa1b2692700 1 -- 172.21.15.144:0/3400717878 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa1940383a0 msgr2=0x7fa19403a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:34.986 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.407+0000 7fa1b2692700 1 --2- 172.21.15.144:0/3400717878 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa1940383a0 0x7fa19403a850 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa19c006fd0 tx=0x7fa19c006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:34.986 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.407+0000 7fa1b2692700 1 -- 172.21.15.144:0/3400717878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1ac104a80 msgr2=0x7fa1ac198810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:34.986 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.407+0000 7fa1b2692700 1 --2- 172.21.15.144:0/3400717878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1ac104a80 0x7fa1ac198810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0006ba0 tx=0x7fa1a0011b50 comp rx=0 tx=0).stop 2024-09-10T11:20:34.986 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.407+0000 7fa1b2692700 1 -- 172.21.15.144:0/3400717878 shutdown_connections 2024-09-10T11:20:34.986 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.407+0000 7fa1b2692700 1 --2- 172.21.15.144:0/3400717878 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa1940383a0 0x7fa19403a850 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:34.987 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.407+0000 7fa1b2692700 1 --2- 172.21.15.144:0/3400717878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1ac104a80 0x7fa1ac198810 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:34.987 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.407+0000 7fa1b2692700 1 -- 172.21.15.144:0/3400717878 >> 172.21.15.144:0/3400717878 conn(0x7fa1ac1000f0 msgr2=0x7fa1ac101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:34.987 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.407+0000 7fa1b2692700 1 -- 172.21.15.144:0/3400717878 shutdown_connections 2024-09-10T11:20:34.987 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:34.407+0000 7fa1b2692700 1 -- 172.21.15.144:0/3400717878 wait complete. 2024-09-10T11:20:34.987 INFO:teuthology.orchestra.run.smithi144.stdout:Enabling the dashboard module... 2024-09-10T11:20:35.228 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:35 smithi144 ceph-mon[27942]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:20:35.228 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:35 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/3400717878' entity='client.admin' 2024-09-10T11:20:36.382 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:36 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2397423867' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T11:20:36.788 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.418+0000 7fd5f95ab700 1 Processor -- start 2024-09-10T11:20:36.788 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.418+0000 7fd5f95ab700 1 -- start start 2024-09-10T11:20:36.788 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.418+0000 7fd5f95ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f407be20 0x7fd5f407a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:36.788 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.418+0000 7fd5f95ab700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f407a860 con 0x7fd5f407be20 2024-09-10T11:20:36.789 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.418+0000 7fd5f2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f407be20 0x7fd5f407a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:36.789 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.418+0000 7fd5f2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f407be20 0x7fd5f407a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51264/0 (socket says 172.21.15.144:51264) 2024-09-10T11:20:36.789 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.418+0000 7fd5f2ffd700 1 -- 172.21.15.144:0/2110036091 learned_addr learned my addr 172.21.15.144:0/2110036091 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:36.789 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.418+0000 7fd5f2ffd700 1 -- 172.21.15.144:0/2110036091 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5f407a9a0 con 0x7fd5f407be20 2024-09-10T11:20:36.789 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.419+0000 7fd5f2ffd700 1 --2- 172.21.15.144:0/2110036091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f407be20 0x7fd5f407a320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40098a0 tx=0x7fd5e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe652754cc91675 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:36.789 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.419+0000 7fd5f1ffb700 1 -- 172.21.15.144:0/2110036091 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5e4004180 con 0x7fd5f407be20 2024-09-10T11:20:36.789 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.419+0000 7fd5f1ffb700 1 -- 172.21.15.144:0/2110036091 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd5e40042e0 con 0x7fd5f407be20 2024-09-10T11:20:36.789 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.419+0000 7fd5f1ffb700 1 -- 172.21.15.144:0/2110036091 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5e4011af0 con 0x7fd5f407be20 2024-09-10T11:20:36.789 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.419+0000 7fd5f95ab700 1 -- 172.21.15.144:0/2110036091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f407be20 msgr2=0x7fd5f407a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:36.789 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.419+0000 7fd5f95ab700 1 --2- 172.21.15.144:0/2110036091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f407be20 0x7fd5f407a320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd5e40098a0 tx=0x7fd5e4009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:36.789 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.419+0000 7fd5f95ab700 1 -- 172.21.15.144:0/2110036091 shutdown_connections 2024-09-10T11:20:36.789 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.419+0000 7fd5f95ab700 1 --2- 172.21.15.144:0/2110036091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f407be20 0x7fd5f407a320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:36.789 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.419+0000 7fd5f95ab700 1 -- 172.21.15.144:0/2110036091 >> 172.21.15.144:0/2110036091 conn(0x7fd5f4101220 msgr2=0x7fd5f4103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:36.789 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.419+0000 7fd5f95ab700 1 -- 172.21.15.144:0/2110036091 shutdown_connections 2024-09-10T11:20:36.790 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.420+0000 7fd5f95ab700 1 -- 172.21.15.144:0/2110036091 wait complete. 2024-09-10T11:20:36.790 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.420+0000 7fd5f95ab700 1 Processor -- start 2024-09-10T11:20:36.790 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.420+0000 7fd5f95ab700 1 -- start start 2024-09-10T11:20:36.790 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.420+0000 7fd5f95ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f407be20 0x7fd5f419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:36.790 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.420+0000 7fd5f95ab700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5f419c0b0 con 0x7fd5f407be20 2024-09-10T11:20:36.790 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.421+0000 7fd5f2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f407be20 0x7fd5f419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:36.790 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.421+0000 7fd5f2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f407be20 0x7fd5f419bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51280/0 (socket says 172.21.15.144:51280) 2024-09-10T11:20:36.790 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.421+0000 7fd5f2ffd700 1 -- 172.21.15.144:0/2397423867 learned_addr learned my addr 172.21.15.144:0/2397423867 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:36.790 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.421+0000 7fd5f2ffd700 1 -- 172.21.15.144:0/2397423867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e4009550 con 0x7fd5f407be20 2024-09-10T11:20:36.790 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.421+0000 7fd5f2ffd700 1 --2- 172.21.15.144:0/2397423867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f407be20 0x7fd5f419bb70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4000c00 tx=0x7fd5e4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:36.790 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.421+0000 7fd5dbfff700 1 -- 172.21.15.144:0/2397423867 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5e4003a70 con 0x7fd5f407be20 2024-09-10T11:20:36.790 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.421+0000 7fd5dbfff700 1 -- 172.21.15.144:0/2397423867 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd5e4003bd0 con 0x7fd5f407be20 2024-09-10T11:20:36.791 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.421+0000 7fd5f95ab700 1 -- 172.21.15.144:0/2397423867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5f419c2b0 con 0x7fd5f407be20 2024-09-10T11:20:36.791 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.421+0000 7fd5dbfff700 1 -- 172.21.15.144:0/2397423867 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd5e4023700 con 0x7fd5f407be20 2024-09-10T11:20:36.791 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.421+0000 7fd5f95ab700 1 -- 172.21.15.144:0/2397423867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5f419c6d0 con 0x7fd5f407be20 2024-09-10T11:20:36.791 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.422+0000 7fd5dbfff700 1 -- 172.21.15.144:0/2397423867 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fd5e40239c0 con 0x7fd5f407be20 2024-09-10T11:20:36.791 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.422+0000 7fd5f95ab700 1 -- 172.21.15.144:0/2397423867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5f410bea0 con 0x7fd5f407be20 2024-09-10T11:20:36.791 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.422+0000 7fd5dbfff700 1 --2- 172.21.15.144:0/2397423867 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd5dc0383f0 0x7fd5dc03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:36.791 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.422+0000 7fd5dbfff700 1 -- 172.21.15.144:0/2397423867 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd5e404c290 con 0x7fd5f407be20 2024-09-10T11:20:36.791 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.422+0000 7fd5f27fc700 1 --2- 172.21.15.144:0/2397423867 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd5dc0383f0 0x7fd5dc03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:36.791 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.423+0000 7fd5f27fc700 1 --2- 172.21.15.144:0/2397423867 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd5dc0383f0 0x7fd5dc03a8a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8006fd0 tx=0x7fd5e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:36.791 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.427+0000 7fd5dbfff700 1 -- 172.21.15.144:0/2397423867 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd5e4010c00 con 0x7fd5f407be20 2024-09-10T11:20:36.791 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:35.669+0000 7fd5f95ab700 1 -- 172.21.15.144:0/2397423867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fd5f410c0c0 con 0x7fd5f407be20 2024-09-10T11:20:36.791 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:36.115+0000 7fd5dbfff700 1 -- 172.21.15.144:0/2397423867 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7fd5e401ab70 con 0x7fd5f407be20 2024-09-10T11:20:36.791 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:36.117+0000 7fd5dbfff700 1 -- 172.21.15.144:0/2397423867 <== mon.0 v2:172.21.15.144:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7fd5e4010e30 con 0x7fd5f407be20 2024-09-10T11:20:36.792 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:36.119+0000 7fd5f95ab700 1 -- 172.21.15.144:0/2397423867 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd5dc0383f0 msgr2=0x7fd5dc03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:36.792 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:36.119+0000 7fd5f95ab700 1 --2- 172.21.15.144:0/2397423867 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd5dc0383f0 0x7fd5dc03a8a0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd5e8006fd0 tx=0x7fd5e8006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:36.792 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:36.119+0000 7fd5f95ab700 1 -- 172.21.15.144:0/2397423867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f407be20 msgr2=0x7fd5f419bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:36.792 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:36.119+0000 7fd5f95ab700 1 --2- 172.21.15.144:0/2397423867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f407be20 0x7fd5f419bb70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fd5e4000c00 tx=0x7fd5e4011b50 comp rx=0 tx=0).stop 2024-09-10T11:20:36.792 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:36.119+0000 7fd5f95ab700 1 -- 172.21.15.144:0/2397423867 shutdown_connections 2024-09-10T11:20:36.792 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:36.119+0000 7fd5f95ab700 1 --2- 172.21.15.144:0/2397423867 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd5dc0383f0 0x7fd5dc03a8a0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:36.792 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:36.119+0000 7fd5f95ab700 1 --2- 172.21.15.144:0/2397423867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5f407be20 0x7fd5f419bb70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:36.792 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:36.119+0000 7fd5f95ab700 1 -- 172.21.15.144:0/2397423867 >> 172.21.15.144:0/2397423867 conn(0x7fd5f4101220 msgr2=0x7fd5f4102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:36.792 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:36.119+0000 7fd5f95ab700 1 -- 172.21.15.144:0/2397423867 shutdown_connections 2024-09-10T11:20:36.792 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:36.119+0000 7fd5f95ab700 1 -- 172.21.15.144:0/2397423867 wait complete. 2024-09-10T11:20:37.234 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:37 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2397423867' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T11:20:37.234 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:37 smithi144 ceph-mon[27942]: mgrmap e9: smithi144.bkogfs(active, since 26s) 2024-09-10T11:20:38.180 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout { 2024-09-10T11:20:38.180 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T11:20:38.180 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T11:20:38.180 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "active_name": "smithi144.bkogfs", 2024-09-10T11:20:38.180 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T11:20:38.180 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout } 2024-09-10T11:20:38.180 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.249+0000 7fa9dd80a700 1 Processor -- start 2024-09-10T11:20:38.180 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.249+0000 7fa9dd80a700 1 -- start start 2024-09-10T11:20:38.181 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.249+0000 7fa9dd80a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d8079d20 0x7fa9d810ab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:38.181 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.249+0000 7fa9dd80a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d8106570 con 0x7fa9d8079d20 2024-09-10T11:20:38.181 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.250+0000 7fa9d6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d8079d20 0x7fa9d810ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:38.181 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.250+0000 7fa9d6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d8079d20 0x7fa9d810ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51308/0 (socket says 172.21.15.144:51308) 2024-09-10T11:20:38.181 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.250+0000 7fa9d6ffd700 1 -- 172.21.15.144:0/3290563878 learned_addr learned my addr 172.21.15.144:0/3290563878 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:38.181 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.250+0000 7fa9d6ffd700 1 -- 172.21.15.144:0/3290563878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d810b0c0 con 0x7fa9d8079d20 2024-09-10T11:20:38.181 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.250+0000 7fa9d6ffd700 1 --2- 172.21.15.144:0/3290563878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d8079d20 0x7fa9d810ab80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80098a0 tx=0x7fa9c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7663021b62b47021 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:38.181 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.250+0000 7fa9d5ffb700 1 -- 172.21.15.144:0/3290563878 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9c8004180 con 0x7fa9d8079d20 2024-09-10T11:20:38.181 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.250+0000 7fa9d5ffb700 1 -- 172.21.15.144:0/3290563878 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa9c80042e0 con 0x7fa9d8079d20 2024-09-10T11:20:38.181 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.250+0000 7fa9d5ffb700 1 -- 172.21.15.144:0/3290563878 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9c8011af0 con 0x7fa9d8079d20 2024-09-10T11:20:38.181 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.251+0000 7fa9dd80a700 1 -- 172.21.15.144:0/3290563878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d8079d20 msgr2=0x7fa9d810ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:38.181 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.251+0000 7fa9dd80a700 1 --2- 172.21.15.144:0/3290563878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d8079d20 0x7fa9d810ab80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa9c80098a0 tx=0x7fa9c8009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:38.182 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.251+0000 7fa9dd80a700 1 -- 172.21.15.144:0/3290563878 shutdown_connections 2024-09-10T11:20:38.182 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.251+0000 7fa9dd80a700 1 --2- 172.21.15.144:0/3290563878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d8079d20 0x7fa9d810ab80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:38.182 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.251+0000 7fa9dd80a700 1 -- 172.21.15.144:0/3290563878 >> 172.21.15.144:0/3290563878 conn(0x7fa9d8100130 msgr2=0x7fa9d8102560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:38.182 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.251+0000 7fa9dd80a700 1 -- 172.21.15.144:0/3290563878 shutdown_connections 2024-09-10T11:20:38.182 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.251+0000 7fa9dd80a700 1 -- 172.21.15.144:0/3290563878 wait complete. 2024-09-10T11:20:38.182 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.252+0000 7fa9dd80a700 1 Processor -- start 2024-09-10T11:20:38.182 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.252+0000 7fa9dd80a700 1 -- start start 2024-09-10T11:20:38.182 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.252+0000 7fa9dd80a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d819baf0 0x7fa9d819bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:38.182 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.252+0000 7fa9dd80a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d819c440 con 0x7fa9d819baf0 2024-09-10T11:20:38.182 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.252+0000 7fa9d6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d819baf0 0x7fa9d819bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:38.182 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.252+0000 7fa9d6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d819baf0 0x7fa9d819bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51324/0 (socket says 172.21.15.144:51324) 2024-09-10T11:20:38.182 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.252+0000 7fa9d6ffd700 1 -- 172.21.15.144:0/2602820025 learned_addr learned my addr 172.21.15.144:0/2602820025 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:38.182 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.253+0000 7fa9d6ffd700 1 -- 172.21.15.144:0/2602820025 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c8009550 con 0x7fa9d819baf0 2024-09-10T11:20:38.183 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.253+0000 7fa9d6ffd700 1 --2- 172.21.15.144:0/2602820025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d819baf0 0x7fa9d819bf00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8000c00 tx=0x7fa9c8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:38.183 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.253+0000 7fa9dc808700 1 -- 172.21.15.144:0/2602820025 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9c8003a70 con 0x7fa9d819baf0 2024-09-10T11:20:38.183 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.253+0000 7fa9dc808700 1 -- 172.21.15.144:0/2602820025 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa9c8003bd0 con 0x7fa9d819baf0 2024-09-10T11:20:38.183 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.253+0000 7fa9dc808700 1 -- 172.21.15.144:0/2602820025 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa9c80238c0 con 0x7fa9d819baf0 2024-09-10T11:20:38.183 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.253+0000 7fa9dd80a700 1 -- 172.21.15.144:0/2602820025 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d819c640 con 0x7fa9d819baf0 2024-09-10T11:20:38.183 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.253+0000 7fa9dd80a700 1 -- 172.21.15.144:0/2602820025 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d819f2a0 con 0x7fa9d819baf0 2024-09-10T11:20:38.183 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.254+0000 7fa9dc808700 1 -- 172.21.15.144:0/2602820025 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7fa9c8023b80 con 0x7fa9d819baf0 2024-09-10T11:20:38.183 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.254+0000 7fa9dd80a700 1 -- 172.21.15.144:0/2602820025 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d8079d20 con 0x7fa9d819baf0 2024-09-10T11:20:38.183 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.254+0000 7fa9dc808700 1 --2- 172.21.15.144:0/2602820025 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa9c4038440 0x7fa9c403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:38.183 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.255+0000 7fa9dc808700 1 -- 172.21.15.144:0/2602820025 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa9c804c290 con 0x7fa9d819baf0 2024-09-10T11:20:38.183 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.255+0000 7fa9d67fc700 1 -- 172.21.15.144:0/2602820025 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa9c4038440 msgr2=0x7fa9c403a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:20:38.184 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.255+0000 7fa9d67fc700 1 --2- 172.21.15.144:0/2602820025 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa9c4038440 0x7fa9c403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T11:20:38.184 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.259+0000 7fa9dc808700 1 -- 172.21.15.144:0/2602820025 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa9c802b300 con 0x7fa9d819baf0 2024-09-10T11:20:38.184 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.456+0000 7fa9d67fc700 1 -- 172.21.15.144:0/2602820025 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa9c4038440 msgr2=0x7fa9c403a8f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:20:38.184 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.456+0000 7fa9d67fc700 1 --2- 172.21.15.144:0/2602820025 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa9c4038440 0x7fa9c403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T11:20:38.184 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.606+0000 7fa9dd80a700 1 -- 172.21.15.144:0/2602820025 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fa9d8079f40 con 0x7fa9d819baf0 2024-09-10T11:20:38.184 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.607+0000 7fa9dc808700 1 -- 172.21.15.144:0/2602820025 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7fa9c8010b30 con 0x7fa9d819baf0 2024-09-10T11:20:38.184 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.610+0000 7fa9dd80a700 1 -- 172.21.15.144:0/2602820025 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa9c4038440 msgr2=0x7fa9c403a8f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T11:20:38.184 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.610+0000 7fa9dd80a700 1 --2- 172.21.15.144:0/2602820025 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa9c4038440 0x7fa9c403a8f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:38.184 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.610+0000 7fa9dd80a700 1 -- 172.21.15.144:0/2602820025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d819baf0 msgr2=0x7fa9d819bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:38.184 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.610+0000 7fa9dd80a700 1 --2- 172.21.15.144:0/2602820025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d819baf0 0x7fa9d819bf00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8000c00 tx=0x7fa9c8011b50 comp rx=0 tx=0).stop 2024-09-10T11:20:38.184 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.610+0000 7fa9dd80a700 1 -- 172.21.15.144:0/2602820025 shutdown_connections 2024-09-10T11:20:38.184 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.610+0000 7fa9dd80a700 1 --2- 172.21.15.144:0/2602820025 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa9c4038440 0x7fa9c403a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:38.185 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.610+0000 7fa9dd80a700 1 --2- 172.21.15.144:0/2602820025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9d819baf0 0x7fa9d819bf00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:38.185 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.610+0000 7fa9dd80a700 1 -- 172.21.15.144:0/2602820025 >> 172.21.15.144:0/2602820025 conn(0x7fa9d8100130 msgr2=0x7fa9d8100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:38.185 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.611+0000 7fa9dd80a700 1 -- 172.21.15.144:0/2602820025 shutdown_connections 2024-09-10T11:20:38.185 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:37.611+0000 7fa9dd80a700 1 -- 172.21.15.144:0/2602820025 wait complete. 2024-09-10T11:20:38.185 INFO:teuthology.orchestra.run.smithi144.stdout:Waiting for the mgr to restart... 2024-09-10T11:20:38.185 INFO:teuthology.orchestra.run.smithi144.stdout:Waiting for mgr epoch 9... 2024-09-10T11:20:38.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:38 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2602820025' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T11:20:43.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:43 smithi144 ceph-mon[27942]: Active manager daemon smithi144.bkogfs restarted 2024-09-10T11:20:43.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:43 smithi144 ceph-mon[27942]: Activating manager daemon smithi144.bkogfs 2024-09-10T11:20:43.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:43 smithi144 ceph-mon[27942]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T11:20:43.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:43 smithi144 ceph-mon[27942]: mgrmap e10: smithi144.bkogfs(active, starting, since 0.00376123s) 2024-09-10T11:20:43.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:43 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:20:43.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:43 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr metadata", "who": "smithi144.bkogfs", "id": "smithi144.bkogfs"}]: dispatch 2024-09-10T11:20:43.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:43 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T11:20:43.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:43 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T11:20:43.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:43 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T11:20:43.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:43 smithi144 ceph-mon[27942]: Manager daemon smithi144.bkogfs is now available 2024-09-10T11:20:43.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:43 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:20:43.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:43 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:20:43.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:43 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/mirror_snapshot_schedule"}]: dispatch 2024-09-10T11:20:43.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:43 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/trash_purge_schedule"}]: dispatch 2024-09-10T11:20:44.840 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout { 2024-09-10T11:20:44.840 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T11:20:44.841 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T11:20:44.841 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout } 2024-09-10T11:20:44.841 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.577+0000 7fd85872d700 1 Processor -- start 2024-09-10T11:20:44.841 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.577+0000 7fd85872d700 1 -- start start 2024-09-10T11:20:44.841 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.577+0000 7fd85872d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd850106080 0x7fd850106490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:44.841 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.577+0000 7fd85872d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8501069d0 con 0x7fd850106080 2024-09-10T11:20:44.841 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.577+0000 7fd8564c9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd850106080 0x7fd850106490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:44.841 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.577+0000 7fd8564c9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd850106080 0x7fd850106490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51334/0 (socket says 172.21.15.144:51334) 2024-09-10T11:20:44.841 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.577+0000 7fd8564c9700 1 -- 172.21.15.144:0/3981490385 learned_addr learned my addr 172.21.15.144:0/3981490385 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:44.841 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.577+0000 7fd8564c9700 1 -- 172.21.15.144:0/3981490385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd850106b10 con 0x7fd850106080 2024-09-10T11:20:44.841 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.578+0000 7fd8564c9700 1 --2- 172.21.15.144:0/3981490385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd850106080 0x7fd850106490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0098a0 tx=0x7fd84c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c42116dfa15e0f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:44.841 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.578+0000 7fd8554c7700 1 -- 172.21.15.144:0/3981490385 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd84c004180 con 0x7fd850106080 2024-09-10T11:20:44.841 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.578+0000 7fd8554c7700 1 -- 172.21.15.144:0/3981490385 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd84c0042e0 con 0x7fd850106080 2024-09-10T11:20:44.842 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.578+0000 7fd8554c7700 1 -- 172.21.15.144:0/3981490385 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd84c011af0 con 0x7fd850106080 2024-09-10T11:20:44.842 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.579+0000 7fd85872d700 1 -- 172.21.15.144:0/3981490385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd850106080 msgr2=0x7fd850106490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:44.842 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.579+0000 7fd85872d700 1 --2- 172.21.15.144:0/3981490385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd850106080 0x7fd850106490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd84c0098a0 tx=0x7fd84c009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:44.842 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.579+0000 7fd85872d700 1 -- 172.21.15.144:0/3981490385 shutdown_connections 2024-09-10T11:20:44.842 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.579+0000 7fd85872d700 1 --2- 172.21.15.144:0/3981490385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd850106080 0x7fd850106490 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:44.842 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.579+0000 7fd85872d700 1 -- 172.21.15.144:0/3981490385 >> 172.21.15.144:0/3981490385 conn(0x7fd8500ff880 msgr2=0x7fd850101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:44.842 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.579+0000 7fd85872d700 1 -- 172.21.15.144:0/3981490385 shutdown_connections 2024-09-10T11:20:44.842 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.579+0000 7fd85872d700 1 -- 172.21.15.144:0/3981490385 wait complete. 2024-09-10T11:20:44.842 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.579+0000 7fd85872d700 1 Processor -- start 2024-09-10T11:20:44.842 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.579+0000 7fd85872d700 1 -- start start 2024-09-10T11:20:44.842 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.580+0000 7fd85872d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd850106080 0x7fd850104830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:44.842 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.580+0000 7fd85872d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85019c660 con 0x7fd850106080 2024-09-10T11:20:44.842 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.580+0000 7fd8564c9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd850106080 0x7fd850104830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:44.843 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.580+0000 7fd8564c9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd850106080 0x7fd850104830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51336/0 (socket says 172.21.15.144:51336) 2024-09-10T11:20:44.843 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.580+0000 7fd8564c9700 1 -- 172.21.15.144:0/3951954150 learned_addr learned my addr 172.21.15.144:0/3951954150 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:44.843 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.580+0000 7fd8564c9700 1 -- 172.21.15.144:0/3951954150 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c009550 con 0x7fd850106080 2024-09-10T11:20:44.843 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.580+0000 7fd8564c9700 1 --2- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd850106080 0x7fd850104830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd84c006ba0 tx=0x7fd84c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:44.843 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.580+0000 7fd8477fe700 1 -- 172.21.15.144:0/3951954150 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd84c003a00 con 0x7fd850106080 2024-09-10T11:20:44.843 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.580+0000 7fd8477fe700 1 -- 172.21.15.144:0/3951954150 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd84c003b60 con 0x7fd850106080 2024-09-10T11:20:44.843 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.580+0000 7fd85872d700 1 -- 172.21.15.144:0/3951954150 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85019c7a0 con 0x7fd850106080 2024-09-10T11:20:44.843 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.581+0000 7fd8477fe700 1 -- 172.21.15.144:0/3951954150 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd84c019920 con 0x7fd850106080 2024-09-10T11:20:44.843 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.581+0000 7fd85872d700 1 -- 172.21.15.144:0/3951954150 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85019cbc0 con 0x7fd850106080 2024-09-10T11:20:44.843 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.581+0000 7fd8477fe700 1 -- 172.21.15.144:0/3951954150 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7fd84c019be0 con 0x7fd850106080 2024-09-10T11:20:44.843 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.582+0000 7fd8477fe700 1 --2- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 0x7fd83c03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:44.843 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.582+0000 7fd85872d700 1 -- 172.21.15.144:0/3951954150 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd850062380 con 0x7fd83c0383f0 2024-09-10T11:20:44.843 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.582+0000 7fd8477fe700 1 -- 172.21.15.144:0/3951954150 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fd84c04c540 con 0x7fd850106080 2024-09-10T11:20:44.843 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.582+0000 7fd855cc8700 1 -- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 msgr2=0x7fd83c03a8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:20:44.844 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.582+0000 7fd855cc8700 1 --2- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 0x7fd83c03a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T11:20:44.844 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.783+0000 7fd855cc8700 1 -- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 msgr2=0x7fd83c03a8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:20:44.844 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:38.783+0000 7fd855cc8700 1 --2- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 0x7fd83c03a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T11:20:44.844 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:39.184+0000 7fd855cc8700 1 -- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 msgr2=0x7fd83c03a8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:20:44.844 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:39.184+0000 7fd855cc8700 1 --2- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 0x7fd83c03a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T11:20:44.844 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:39.985+0000 7fd855cc8700 1 -- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 msgr2=0x7fd83c03a8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:20:44.844 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:39.985+0000 7fd855cc8700 1 --2- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 0x7fd83c03a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T11:20:44.844 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:41.586+0000 7fd855cc8700 1 -- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 msgr2=0x7fd83c03a8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:20:44.844 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:41.586+0000 7fd855cc8700 1 --2- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 0x7fd83c03a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-09-10T11:20:44.844 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:42.966+0000 7fd8477fe700 1 -- 172.21.15.144:0/3951954150 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mgrmap(e 10) v1 ==== 44864+0+0 (secure 0 0 0) 0x7fd84c02c3a0 con 0x7fd850106080 2024-09-10T11:20:44.844 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:42.967+0000 7fd8477fe700 1 -- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 msgr2=0x7fd83c03a8a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T11:20:44.844 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:42.967+0000 7fd8477fe700 1 --2- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 0x7fd83c03a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:44.844 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.968+0000 7fd8477fe700 1 -- 172.21.15.144:0/3951954150 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mgrmap(e 11) v1 ==== 44991+0+0 (secure 0 0 0) 0x7fd84c018350 con 0x7fd850106080 2024-09-10T11:20:44.845 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.968+0000 7fd8477fe700 1 --2- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 0x7fd83c03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:44.845 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.968+0000 7fd8477fe700 1 -- 172.21.15.144:0/3951954150 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd850062380 con 0x7fd83c0383f0 2024-09-10T11:20:44.845 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.968+0000 7fd855cc8700 1 --2- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 0x7fd83c03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:44.845 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.969+0000 7fd855cc8700 1 --2- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 0x7fd83c03a8a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd840003a10 tx=0x7fd8400092b0 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:44.845 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.969+0000 7fd8477fe700 1 -- 172.21.15.144:0/3951954150 <== mgr.14164 v2:172.21.15.144:6800/2 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fd850062380 con 0x7fd83c0383f0 2024-09-10T11:20:44.845 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.976+0000 7fd85872d700 1 -- 172.21.15.144:0/3951954150 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fd8501043d0 con 0x7fd83c0383f0 2024-09-10T11:20:44.845 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.976+0000 7fd8477fe700 1 -- 172.21.15.144:0/3951954150 <== mgr.14164 v2:172.21.15.144:6800/2 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fd8501043d0 con 0x7fd83c0383f0 2024-09-10T11:20:44.845 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.976+0000 7fd85872d700 1 -- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 msgr2=0x7fd83c03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:44.845 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.976+0000 7fd85872d700 1 --2- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 0x7fd83c03a8a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fd840003a10 tx=0x7fd8400092b0 comp rx=0 tx=0).stop 2024-09-10T11:20:44.845 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.976+0000 7fd85872d700 1 -- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd850106080 msgr2=0x7fd850104830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:44.845 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.976+0000 7fd85872d700 1 --2- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd850106080 0x7fd850104830 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd84c006ba0 tx=0x7fd84c0047c0 comp rx=0 tx=0).stop 2024-09-10T11:20:44.845 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.976+0000 7fd85872d700 1 -- 172.21.15.144:0/3951954150 shutdown_connections 2024-09-10T11:20:44.845 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.976+0000 7fd85872d700 1 --2- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd83c0383f0 0x7fd83c03a8a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:44.845 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.976+0000 7fd85872d700 1 --2- 172.21.15.144:0/3951954150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd850106080 0x7fd850104830 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:44.846 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.976+0000 7fd85872d700 1 -- 172.21.15.144:0/3951954150 >> 172.21.15.144:0/3951954150 conn(0x7fd8500ff880 msgr2=0x7fd850100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:44.846 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.976+0000 7fd85872d700 1 -- 172.21.15.144:0/3951954150 shutdown_connections 2024-09-10T11:20:44.846 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:43.976+0000 7fd85872d700 1 -- 172.21.15.144:0/3951954150 wait complete. 2024-09-10T11:20:44.846 INFO:teuthology.orchestra.run.smithi144.stdout:mgr epoch 9 is available 2024-09-10T11:20:44.846 INFO:teuthology.orchestra.run.smithi144.stdout:Generating a dashboard self-signed certificate... 2024-09-10T11:20:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:44 smithi144 ceph-mon[27942]: [10/Sep/2024:11:20:43] ENGINE Bus STARTING 2024-09-10T11:20:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:44 smithi144 ceph-mon[27942]: [10/Sep/2024:11:20:43] ENGINE Serving on http://172.21.15.144:8765 2024-09-10T11:20:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:44 smithi144 ceph-mon[27942]: [10/Sep/2024:11:20:43] ENGINE Serving on https://172.21.15.144:7150 2024-09-10T11:20:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:44 smithi144 ceph-mon[27942]: [10/Sep/2024:11:20:43] ENGINE Bus STARTED 2024-09-10T11:20:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:44 smithi144 ceph-mon[27942]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T11:20:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:44 smithi144 ceph-mon[27942]: mgrmap e11: smithi144.bkogfs(active, since 1.00583s) 2024-09-10T11:20:45.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:44 smithi144 ceph-mon[27942]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T11:20:46.246 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:45 smithi144 ceph-mon[27942]: mgrmap e12: smithi144.bkogfs(active, since 2s) 2024-09-10T11:20:46.246 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:45 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:46.246 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:45 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:46.246 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:45 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:46.253 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T11:20:46.253 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.294+0000 7fd9eb3bd700 1 Processor -- start 2024-09-10T11:20:46.253 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.295+0000 7fd9eb3bd700 1 -- start start 2024-09-10T11:20:46.253 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.295+0000 7fd9eb3bd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e4103e70 0x7fd9e4104280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:46.253 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.295+0000 7fd9eb3bd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e41047c0 con 0x7fd9e4103e70 2024-09-10T11:20:46.253 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.295+0000 7fd9e9159700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e4103e70 0x7fd9e4104280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:46.253 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.295+0000 7fd9e9159700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e4103e70 0x7fd9e4104280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35960/0 (socket says 172.21.15.144:35960) 2024-09-10T11:20:46.253 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.295+0000 7fd9e9159700 1 -- 172.21.15.144:0/2894433506 learned_addr learned my addr 172.21.15.144:0/2894433506 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:46.253 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.295+0000 7fd9e9159700 1 -- 172.21.15.144:0/2894433506 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e4104900 con 0x7fd9e4103e70 2024-09-10T11:20:46.253 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.295+0000 7fd9e9159700 1 --2- 172.21.15.144:0/2894433506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e4103e70 0x7fd9e4104280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009e80 tx=0x7fd9d400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b77af79af658bb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:46.254 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.296+0000 7fd9e3fff700 1 -- 172.21.15.144:0/2894433506 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9d400e070 con 0x7fd9e4103e70 2024-09-10T11:20:46.254 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.296+0000 7fd9e3fff700 1 -- 172.21.15.144:0/2894433506 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd9d400b8c0 con 0x7fd9e4103e70 2024-09-10T11:20:46.254 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.296+0000 7fd9e3fff700 1 -- 172.21.15.144:0/2894433506 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9d40044d0 con 0x7fd9e4103e70 2024-09-10T11:20:46.254 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.296+0000 7fd9eb3bd700 1 -- 172.21.15.144:0/2894433506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e4103e70 msgr2=0x7fd9e4104280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:46.254 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.296+0000 7fd9eb3bd700 1 --2- 172.21.15.144:0/2894433506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e4103e70 0x7fd9e4104280 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd9d4009e80 tx=0x7fd9d400b330 comp rx=0 tx=0).stop 2024-09-10T11:20:46.254 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.296+0000 7fd9eb3bd700 1 -- 172.21.15.144:0/2894433506 shutdown_connections 2024-09-10T11:20:46.254 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.296+0000 7fd9eb3bd700 1 --2- 172.21.15.144:0/2894433506 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e4103e70 0x7fd9e4104280 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:46.254 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.296+0000 7fd9eb3bd700 1 -- 172.21.15.144:0/2894433506 >> 172.21.15.144:0/2894433506 conn(0x7fd9e40ff870 msgr2=0x7fd9e4101c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:46.254 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.296+0000 7fd9eb3bd700 1 -- 172.21.15.144:0/2894433506 shutdown_connections 2024-09-10T11:20:46.254 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.296+0000 7fd9eb3bd700 1 -- 172.21.15.144:0/2894433506 wait complete. 2024-09-10T11:20:46.254 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.297+0000 7fd9eb3bd700 1 Processor -- start 2024-09-10T11:20:46.254 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.297+0000 7fd9eb3bd700 1 -- start start 2024-09-10T11:20:46.254 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.297+0000 7fd9eb3bd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e4103e70 0x7fd9e4199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:46.255 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.297+0000 7fd9eb3bd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e4199e40 con 0x7fd9e4103e70 2024-09-10T11:20:46.255 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.297+0000 7fd9e9159700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e4103e70 0x7fd9e4199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:46.255 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.297+0000 7fd9e9159700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e4103e70 0x7fd9e4199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35976/0 (socket says 172.21.15.144:35976) 2024-09-10T11:20:46.255 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.297+0000 7fd9e9159700 1 -- 172.21.15.144:0/681131180 learned_addr learned my addr 172.21.15.144:0/681131180 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:46.255 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.298+0000 7fd9e9159700 1 -- 172.21.15.144:0/681131180 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d4009b30 con 0x7fd9e4103e70 2024-09-10T11:20:46.255 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.298+0000 7fd9e9159700 1 --2- 172.21.15.144:0/681131180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e4103e70 0x7fd9e4199900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40072a0 tx=0x7fd9d40049f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:46.255 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.298+0000 7fd9e27fc700 1 -- 172.21.15.144:0/681131180 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9d400e040 con 0x7fd9e4103e70 2024-09-10T11:20:46.255 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.298+0000 7fd9e27fc700 1 -- 172.21.15.144:0/681131180 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd9d400bd80 con 0x7fd9e4103e70 2024-09-10T11:20:46.255 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.298+0000 7fd9e27fc700 1 -- 172.21.15.144:0/681131180 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9d40126c0 con 0x7fd9e4103e70 2024-09-10T11:20:46.255 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.298+0000 7fd9eb3bd700 1 -- 172.21.15.144:0/681131180 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e419a040 con 0x7fd9e4103e70 2024-09-10T11:20:46.255 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.298+0000 7fd9eb3bd700 1 -- 172.21.15.144:0/681131180 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e419a4c0 con 0x7fd9e4103e70 2024-09-10T11:20:46.255 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.299+0000 7fd9eb3bd700 1 -- 172.21.15.144:0/681131180 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e4062380 con 0x7fd9e4103e70 2024-09-10T11:20:46.255 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.299+0000 7fd9e27fc700 1 -- 172.21.15.144:0/681131180 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7fd9d4019040 con 0x7fd9e4103e70 2024-09-10T11:20:46.255 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.300+0000 7fd9e27fc700 1 --2- 172.21.15.144:0/681131180 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd9cc038400 0x7fd9cc03a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:46.256 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.300+0000 7fd9e27fc700 1 -- 172.21.15.144:0/681131180 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fd9d40547e0 con 0x7fd9e4103e70 2024-09-10T11:20:46.256 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.300+0000 7fd9e8958700 1 --2- 172.21.15.144:0/681131180 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd9cc038400 0x7fd9cc03a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:46.256 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.300+0000 7fd9e8958700 1 --2- 172.21.15.144:0/681131180 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd9cc038400 0x7fd9cc03a8b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8006fd0 tx=0x7fd9d8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:46.256 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.305+0000 7fd9e27fc700 1 -- 172.21.15.144:0/681131180 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd9d4051060 con 0x7fd9e4103e70 2024-09-10T11:20:46.256 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.508+0000 7fd9eb3bd700 1 -- 172.21.15.144:0/681131180 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9e410a090 con 0x7fd9cc038400 2024-09-10T11:20:46.256 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.683+0000 7fd9e27fc700 1 -- 172.21.15.144:0/681131180 <== mgr.14164 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fd9e410a090 con 0x7fd9cc038400 2024-09-10T11:20:46.256 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.685+0000 7fd9eb3bd700 1 -- 172.21.15.144:0/681131180 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd9cc038400 msgr2=0x7fd9cc03a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:46.256 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.685+0000 7fd9eb3bd700 1 --2- 172.21.15.144:0/681131180 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd9cc038400 0x7fd9cc03a8b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd9d8006fd0 tx=0x7fd9d8006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:46.256 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.685+0000 7fd9eb3bd700 1 -- 172.21.15.144:0/681131180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e4103e70 msgr2=0x7fd9e4199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:46.256 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.685+0000 7fd9eb3bd700 1 --2- 172.21.15.144:0/681131180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e4103e70 0x7fd9e4199900 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd9d40072a0 tx=0x7fd9d40049f0 comp rx=0 tx=0).stop 2024-09-10T11:20:46.256 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.685+0000 7fd9eb3bd700 1 -- 172.21.15.144:0/681131180 shutdown_connections 2024-09-10T11:20:46.256 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.685+0000 7fd9eb3bd700 1 --2- 172.21.15.144:0/681131180 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd9cc038400 0x7fd9cc03a8b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:46.256 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.685+0000 7fd9eb3bd700 1 --2- 172.21.15.144:0/681131180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9e4103e70 0x7fd9e4199900 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:46.257 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.685+0000 7fd9eb3bd700 1 -- 172.21.15.144:0/681131180 >> 172.21.15.144:0/681131180 conn(0x7fd9e40ff870 msgr2=0x7fd9e4100640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:46.257 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.686+0000 7fd9eb3bd700 1 -- 172.21.15.144:0/681131180 shutdown_connections 2024-09-10T11:20:46.257 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:45.686+0000 7fd9eb3bd700 1 -- 172.21.15.144:0/681131180 wait complete. 2024-09-10T11:20:46.257 INFO:teuthology.orchestra.run.smithi144.stdout:Creating initial admin user... 2024-09-10T11:20:47.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:46 smithi144 ceph-mon[27942]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:20:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:46 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:48.217 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:48 smithi144 ceph-mon[27942]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:20:48.217 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:48 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:48.609 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$GGQ2pcYiHRCin5/hBS0Vt.Lip2g0HBmbB18YVAxXn58mA.Q4FzP.q", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725967247, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T11:20:48.609 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.676+0000 7f2b88b55700 1 Processor -- start 2024-09-10T11:20:48.609 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.676+0000 7f2b88b55700 1 -- start start 2024-09-10T11:20:48.609 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.676+0000 7f2b88b55700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b841066f0 0x7f2b84108ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:48.609 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.676+0000 7f2b88b55700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b84079d40 con 0x7f2b841066f0 2024-09-10T11:20:48.609 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.676+0000 7f2b8259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b841066f0 0x7f2b84108ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:48.609 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.676+0000 7f2b8259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b841066f0 0x7f2b84108ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35992/0 (socket says 172.21.15.144:35992) 2024-09-10T11:20:48.609 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.676+0000 7f2b8259c700 1 -- 172.21.15.144:0/1993330667 learned_addr learned my addr 172.21.15.144:0/1993330667 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:48.609 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.676+0000 7f2b8259c700 1 -- 172.21.15.144:0/1993330667 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b84079e80 con 0x7f2b841066f0 2024-09-10T11:20:48.610 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.676+0000 7f2b8259c700 1 --2- 172.21.15.144:0/1993330667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b841066f0 0x7f2b84108ad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2b740098a0 tx=0x7f2b74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2df04cf341368a server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:48.610 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.677+0000 7f2b8159a700 1 -- 172.21.15.144:0/1993330667 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b74004180 con 0x7f2b841066f0 2024-09-10T11:20:48.610 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.677+0000 7f2b8159a700 1 -- 172.21.15.144:0/1993330667 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2b740042e0 con 0x7f2b841066f0 2024-09-10T11:20:48.610 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.677+0000 7f2b8159a700 1 -- 172.21.15.144:0/1993330667 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b74011af0 con 0x7f2b841066f0 2024-09-10T11:20:48.610 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.677+0000 7f2b88b55700 1 -- 172.21.15.144:0/1993330667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b841066f0 msgr2=0x7f2b84108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:48.610 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.677+0000 7f2b88b55700 1 --2- 172.21.15.144:0/1993330667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b841066f0 0x7f2b84108ad0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2b740098a0 tx=0x7f2b74009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:48.610 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.677+0000 7f2b88b55700 1 -- 172.21.15.144:0/1993330667 shutdown_connections 2024-09-10T11:20:48.610 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.677+0000 7f2b88b55700 1 --2- 172.21.15.144:0/1993330667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b841066f0 0x7f2b84108ad0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:48.610 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.677+0000 7f2b88b55700 1 -- 172.21.15.144:0/1993330667 >> 172.21.15.144:0/1993330667 conn(0x7f2b841000f0 msgr2=0x7f2b84102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:48.610 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.677+0000 7f2b88b55700 1 -- 172.21.15.144:0/1993330667 shutdown_connections 2024-09-10T11:20:48.610 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.677+0000 7f2b88b55700 1 -- 172.21.15.144:0/1993330667 wait complete. 2024-09-10T11:20:48.610 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.678+0000 7f2b88b55700 1 Processor -- start 2024-09-10T11:20:48.610 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.678+0000 7f2b88b55700 1 -- start start 2024-09-10T11:20:48.610 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.678+0000 7f2b88b55700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b841066f0 0x7f2b84199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:48.611 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.678+0000 7f2b88b55700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b84199ea0 con 0x7f2b841066f0 2024-09-10T11:20:48.611 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.678+0000 7f2b8259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b841066f0 0x7f2b84199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:48.611 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.678+0000 7f2b8259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b841066f0 0x7f2b84199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36008/0 (socket says 172.21.15.144:36008) 2024-09-10T11:20:48.611 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.678+0000 7f2b8259c700 1 -- 172.21.15.144:0/3415654850 learned_addr learned my addr 172.21.15.144:0/3415654850 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:48.611 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.679+0000 7f2b8259c700 1 -- 172.21.15.144:0/3415654850 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b74009550 con 0x7f2b841066f0 2024-09-10T11:20:48.611 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.679+0000 7f2b8259c700 1 --2- 172.21.15.144:0/3415654850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b841066f0 0x7f2b84199960 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2b74006ba0 tx=0x7f2b74011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:48.611 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.679+0000 7f2b7b7fe700 1 -- 172.21.15.144:0/3415654850 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b74003b00 con 0x7f2b841066f0 2024-09-10T11:20:48.611 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.679+0000 7f2b7b7fe700 1 -- 172.21.15.144:0/3415654850 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2b7401a8e0 con 0x7f2b841066f0 2024-09-10T11:20:48.611 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.679+0000 7f2b7b7fe700 1 -- 172.21.15.144:0/3415654850 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b74019990 con 0x7f2b841066f0 2024-09-10T11:20:48.611 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.679+0000 7f2b88b55700 1 -- 172.21.15.144:0/3415654850 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8419a0a0 con 0x7f2b841066f0 2024-09-10T11:20:48.611 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.679+0000 7f2b88b55700 1 -- 172.21.15.144:0/3415654850 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8419a4c0 con 0x7f2b841066f0 2024-09-10T11:20:48.611 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.680+0000 7f2b7b7fe700 1 -- 172.21.15.144:0/3415654850 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f2b74019c50 con 0x7f2b841066f0 2024-09-10T11:20:48.611 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.680+0000 7f2b88b55700 1 -- 172.21.15.144:0/3415654850 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b84062380 con 0x7f2b841066f0 2024-09-10T11:20:48.611 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.680+0000 7f2b7b7fe700 1 --2- 172.21.15.144:0/3415654850 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2b700383f0 0x7f2b7003a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:48.612 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.680+0000 7f2b7b7fe700 1 -- 172.21.15.144:0/3415654850 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f2b7404c8d0 con 0x7f2b841066f0 2024-09-10T11:20:48.612 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.680+0000 7f2b81d9b700 1 --2- 172.21.15.144:0/3415654850 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2b700383f0 0x7f2b7003a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:48.612 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.681+0000 7f2b81d9b700 1 --2- 172.21.15.144:0/3415654850 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2b700383f0 0x7f2b7003a8a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c006fd0 tx=0x7f2b6c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:48.612 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.685+0000 7f2b7b7fe700 1 -- 172.21.15.144:0/3415654850 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2b7402b700 con 0x7f2b841066f0 2024-09-10T11:20:48.612 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:46.899+0000 7f2b88b55700 1 -- 172.21.15.144:0/3415654850 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f2b840621a0 con 0x7f2b700383f0 2024-09-10T11:20:48.612 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:47.130+0000 7f2b7b7fe700 1 -- 172.21.15.144:0/3415654850 <== mgr.14164 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f2b840621a0 con 0x7f2b700383f0 2024-09-10T11:20:48.612 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:47.132+0000 7f2b88b55700 1 -- 172.21.15.144:0/3415654850 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2b700383f0 msgr2=0x7f2b7003a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:48.612 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:47.132+0000 7f2b88b55700 1 --2- 172.21.15.144:0/3415654850 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2b700383f0 0x7f2b7003a8a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2b6c006fd0 tx=0x7f2b6c006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:48.612 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:47.132+0000 7f2b88b55700 1 -- 172.21.15.144:0/3415654850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b841066f0 msgr2=0x7f2b84199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:48.612 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:47.132+0000 7f2b88b55700 1 --2- 172.21.15.144:0/3415654850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b841066f0 0x7f2b84199960 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f2b74006ba0 tx=0x7f2b74011b50 comp rx=0 tx=0).stop 2024-09-10T11:20:48.612 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:47.132+0000 7f2b88b55700 1 -- 172.21.15.144:0/3415654850 shutdown_connections 2024-09-10T11:20:48.612 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:47.132+0000 7f2b88b55700 1 --2- 172.21.15.144:0/3415654850 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2b700383f0 0x7f2b7003a8a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:48.612 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:47.132+0000 7f2b88b55700 1 --2- 172.21.15.144:0/3415654850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b841066f0 0x7f2b84199960 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:48.612 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:47.132+0000 7f2b88b55700 1 -- 172.21.15.144:0/3415654850 >> 172.21.15.144:0/3415654850 conn(0x7f2b841000f0 msgr2=0x7f2b841017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:48.613 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:47.132+0000 7f2b88b55700 1 -- 172.21.15.144:0/3415654850 shutdown_connections 2024-09-10T11:20:48.613 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:47.132+0000 7f2b88b55700 1 -- 172.21.15.144:0/3415654850 wait complete. 2024-09-10T11:20:48.613 INFO:teuthology.orchestra.run.smithi144.stdout:Fetching dashboard port number... 2024-09-10T11:20:49.831 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T11:20:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.037+0000 7f6722b93700 1 Processor -- start 2024-09-10T11:20:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.037+0000 7f6722b93700 1 -- start start 2024-09-10T11:20:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.037+0000 7f6722b93700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c1066f0 0x7f671c108ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.037+0000 7f6722b93700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c079d40 con 0x7f671c1066f0 2024-09-10T11:20:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.037+0000 7f672092f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c1066f0 0x7f671c108ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.037+0000 7f672092f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c1066f0 0x7f671c108ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36018/0 (socket says 172.21.15.144:36018) 2024-09-10T11:20:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.037+0000 7f672092f700 1 -- 172.21.15.144:0/1706066606 learned_addr learned my addr 172.21.15.144:0/1706066606 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.037+0000 7f672092f700 1 -- 172.21.15.144:0/1706066606 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f671c079e80 con 0x7f671c1066f0 2024-09-10T11:20:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.037+0000 7f672092f700 1 --2- 172.21.15.144:0/1706066606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c1066f0 0x7f671c108ad0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f67100098a0 tx=0x7f6710009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=316902da84d158a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.038+0000 7f671b7fe700 1 -- 172.21.15.144:0/1706066606 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6710004180 con 0x7f671c1066f0 2024-09-10T11:20:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.038+0000 7f671b7fe700 1 -- 172.21.15.144:0/1706066606 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f67100042e0 con 0x7f671c1066f0 2024-09-10T11:20:49.832 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.038+0000 7f671b7fe700 1 -- 172.21.15.144:0/1706066606 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6710011af0 con 0x7f671c1066f0 2024-09-10T11:20:49.833 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.038+0000 7f6722b93700 1 -- 172.21.15.144:0/1706066606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c1066f0 msgr2=0x7f671c108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:49.833 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.038+0000 7f6722b93700 1 --2- 172.21.15.144:0/1706066606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c1066f0 0x7f671c108ad0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f67100098a0 tx=0x7f6710009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:49.833 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.038+0000 7f6722b93700 1 -- 172.21.15.144:0/1706066606 shutdown_connections 2024-09-10T11:20:49.833 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.038+0000 7f6722b93700 1 --2- 172.21.15.144:0/1706066606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c1066f0 0x7f671c108ad0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:49.833 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.038+0000 7f6722b93700 1 -- 172.21.15.144:0/1706066606 >> 172.21.15.144:0/1706066606 conn(0x7f671c1000f0 msgr2=0x7f671c102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:49.833 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.038+0000 7f6722b93700 1 -- 172.21.15.144:0/1706066606 shutdown_connections 2024-09-10T11:20:49.833 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.038+0000 7f6722b93700 1 -- 172.21.15.144:0/1706066606 wait complete. 2024-09-10T11:20:49.833 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.039+0000 7f6722b93700 1 Processor -- start 2024-09-10T11:20:49.833 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.039+0000 7f6722b93700 1 -- start start 2024-09-10T11:20:49.833 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.039+0000 7f6722b93700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c1066f0 0x7f671c106050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:49.833 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.039+0000 7f6722b93700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c106590 con 0x7f671c1066f0 2024-09-10T11:20:49.833 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.039+0000 7f672092f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c1066f0 0x7f671c106050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:49.833 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.039+0000 7f672092f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c1066f0 0x7f671c106050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36032/0 (socket says 172.21.15.144:36032) 2024-09-10T11:20:49.833 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.039+0000 7f672092f700 1 -- 172.21.15.144:0/2102783516 learned_addr learned my addr 172.21.15.144:0/2102783516 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:49.834 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.039+0000 7f672092f700 1 -- 172.21.15.144:0/2102783516 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6710009550 con 0x7f671c1066f0 2024-09-10T11:20:49.834 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.040+0000 7f672092f700 1 --2- 172.21.15.144:0/2102783516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c1066f0 0x7f671c106050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6710000c00 tx=0x7f6710011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:49.834 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.040+0000 7f6719ffb700 1 -- 172.21.15.144:0/2102783516 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6710003a70 con 0x7f671c1066f0 2024-09-10T11:20:49.834 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.040+0000 7f6719ffb700 1 -- 172.21.15.144:0/2102783516 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6710003bd0 con 0x7f671c1066f0 2024-09-10T11:20:49.834 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.040+0000 7f6722b93700 1 -- 172.21.15.144:0/2102783516 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671c1046a0 con 0x7f671c1066f0 2024-09-10T11:20:49.834 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.040+0000 7f6719ffb700 1 -- 172.21.15.144:0/2102783516 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f67100238c0 con 0x7f671c1066f0 2024-09-10T11:20:49.834 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.040+0000 7f6722b93700 1 -- 172.21.15.144:0/2102783516 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671c104ac0 con 0x7f671c1066f0 2024-09-10T11:20:49.834 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.041+0000 7f6719ffb700 1 -- 172.21.15.144:0/2102783516 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f6710023b30 con 0x7f671c1066f0 2024-09-10T11:20:49.834 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.041+0000 7f6722b93700 1 -- 172.21.15.144:0/2102783516 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f671c109c70 con 0x7f671c1066f0 2024-09-10T11:20:49.835 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.041+0000 7f6719ffb700 1 --2- 172.21.15.144:0/2102783516 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f670c038490 0x7f670c03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:49.835 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.041+0000 7f6719ffb700 1 -- 172.21.15.144:0/2102783516 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f671004c360 con 0x7f671c1066f0 2024-09-10T11:20:49.835 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.041+0000 7f671bfff700 1 --2- 172.21.15.144:0/2102783516 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f670c038490 0x7f670c03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:49.835 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.041+0000 7f671bfff700 1 --2- 172.21.15.144:0/2102783516 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f670c038490 0x7f670c03a940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6708006fd0 tx=0x7f6708006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:49.835 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.046+0000 7f6719ffb700 1 -- 172.21.15.144:0/2102783516 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6710010560 con 0x7f671c1066f0 2024-09-10T11:20:49.835 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.131+0000 7f6719ffb700 1 -- 172.21.15.144:0/2102783516 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f671001a430 con 0x7f671c1066f0 2024-09-10T11:20:49.835 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.240+0000 7f6722b93700 1 -- 172.21.15.144:0/2102783516 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f671c105580 con 0x7f671c1066f0 2024-09-10T11:20:49.835 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.241+0000 7f6719ffb700 1 -- 172.21.15.144:0/2102783516 <== mon.0 v2:172.21.15.144:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f6710010950 con 0x7f671c1066f0 2024-09-10T11:20:49.835 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.243+0000 7f6722b93700 1 -- 172.21.15.144:0/2102783516 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f670c038490 msgr2=0x7f670c03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:49.835 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.243+0000 7f6722b93700 1 --2- 172.21.15.144:0/2102783516 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f670c038490 0x7f670c03a940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6708006fd0 tx=0x7f6708006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:49.836 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.243+0000 7f6722b93700 1 -- 172.21.15.144:0/2102783516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c1066f0 msgr2=0x7f671c106050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:49.836 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.243+0000 7f6722b93700 1 --2- 172.21.15.144:0/2102783516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c1066f0 0x7f671c106050 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6710000c00 tx=0x7f6710011b50 comp rx=0 tx=0).stop 2024-09-10T11:20:49.836 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.244+0000 7f6722b93700 1 -- 172.21.15.144:0/2102783516 shutdown_connections 2024-09-10T11:20:49.836 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.244+0000 7f6722b93700 1 --2- 172.21.15.144:0/2102783516 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f670c038490 0x7f670c03a940 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:49.836 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.244+0000 7f6722b93700 1 --2- 172.21.15.144:0/2102783516 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c1066f0 0x7f671c106050 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:49.836 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.244+0000 7f6722b93700 1 -- 172.21.15.144:0/2102783516 >> 172.21.15.144:0/2102783516 conn(0x7f671c1000f0 msgr2=0x7f671c101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:49.836 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.244+0000 7f6722b93700 1 -- 172.21.15.144:0/2102783516 shutdown_connections 2024-09-10T11:20:49.836 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:49.244+0000 7f6722b93700 1 -- 172.21.15.144:0/2102783516 wait complete. 2024-09-10T11:20:49.838 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T11:20:49.838 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stdout disabled 2024-09-10T11:20:49.845 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T11:20:49.845 INFO:teuthology.orchestra.run.smithi144.stdout:systemctl: stdout inactive 2024-09-10T11:20:49.845 INFO:teuthology.orchestra.run.smithi144.stdout:firewalld.service is not enabled 2024-09-10T11:20:49.845 INFO:teuthology.orchestra.run.smithi144.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T11:20:49.847 INFO:teuthology.orchestra.run.smithi144.stdout:Ceph Dashboard is now available at: 2024-09-10T11:20:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:20:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: URL: https://smithi144.front.sepia.ceph.com:8443/ 2024-09-10T11:20:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: User: admin 2024-09-10T11:20:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: Password: 84s8aszpy4 2024-09-10T11:20:49.847 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:20:49.848 INFO:teuthology.orchestra.run.smithi144.stdout:Saving cluster configuration to /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config directory 2024-09-10T11:20:49.848 INFO:teuthology.orchestra.run.smithi144.stdout:Enabling autotune for osd_memory_target 2024-09-10T11:20:50.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:50 smithi144 ceph-mon[27942]: mgrmap e13: smithi144.bkogfs(active, since 6s) 2024-09-10T11:20:50.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:50 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2102783516' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T11:20:51.079 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.270+0000 7f0bb3627700 1 Processor -- start 2024-09-10T11:20:51.079 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.270+0000 7f0bb3627700 1 -- start start 2024-09-10T11:20:51.079 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.270+0000 7f0bb3627700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bac079d20 0x7f0bac108720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:51.080 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.270+0000 7f0bb3627700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac108c60 con 0x7f0bac079d20 2024-09-10T11:20:51.080 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.270+0000 7f0bb13c3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bac079d20 0x7f0bac108720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:51.080 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.270+0000 7f0bb13c3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bac079d20 0x7f0bac108720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36038/0 (socket says 172.21.15.144:36038) 2024-09-10T11:20:51.080 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.270+0000 7f0bb13c3700 1 -- 172.21.15.144:0/2191203143 learned_addr learned my addr 172.21.15.144:0/2191203143 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:51.080 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.271+0000 7f0bb13c3700 1 -- 172.21.15.144:0/2191203143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bac108da0 con 0x7f0bac079d20 2024-09-10T11:20:51.080 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.271+0000 7f0bb13c3700 1 --2- 172.21.15.144:0/2191203143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bac079d20 0x7f0bac108720 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80098a0 tx=0x7f0ba8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394d95e844240cc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:51.080 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.271+0000 7f0ba3fff700 1 -- 172.21.15.144:0/2191203143 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ba8004180 con 0x7f0bac079d20 2024-09-10T11:20:51.080 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.271+0000 7f0ba3fff700 1 -- 172.21.15.144:0/2191203143 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0ba80042e0 con 0x7f0bac079d20 2024-09-10T11:20:51.080 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.271+0000 7f0ba3fff700 1 -- 172.21.15.144:0/2191203143 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ba8011af0 con 0x7f0bac079d20 2024-09-10T11:20:51.080 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.271+0000 7f0bb3627700 1 -- 172.21.15.144:0/2191203143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bac079d20 msgr2=0x7f0bac108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:51.080 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.271+0000 7f0bb3627700 1 --2- 172.21.15.144:0/2191203143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bac079d20 0x7f0bac108720 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0ba80098a0 tx=0x7f0ba8009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:51.080 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.272+0000 7f0bb3627700 1 -- 172.21.15.144:0/2191203143 shutdown_connections 2024-09-10T11:20:51.080 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.272+0000 7f0bb3627700 1 --2- 172.21.15.144:0/2191203143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bac079d20 0x7f0bac108720 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:51.081 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.272+0000 7f0bb3627700 1 -- 172.21.15.144:0/2191203143 >> 172.21.15.144:0/2191203143 conn(0x7f0bac1000f0 msgr2=0x7f0bac102500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:51.081 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.272+0000 7f0bb3627700 1 -- 172.21.15.144:0/2191203143 shutdown_connections 2024-09-10T11:20:51.081 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.272+0000 7f0bb3627700 1 -- 172.21.15.144:0/2191203143 wait complete. 2024-09-10T11:20:51.081 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.272+0000 7f0bb3627700 1 Processor -- start 2024-09-10T11:20:51.081 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.272+0000 7f0bb3627700 1 -- start start 2024-09-10T11:20:51.081 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.272+0000 7f0bb3627700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bac199910 0x7f0bac199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:51.081 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.272+0000 7f0bb3627700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bac19a260 con 0x7f0bac199910 2024-09-10T11:20:51.081 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.272+0000 7f0bb13c3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bac199910 0x7f0bac199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:51.081 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.272+0000 7f0bb13c3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bac199910 0x7f0bac199d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36052/0 (socket says 172.21.15.144:36052) 2024-09-10T11:20:51.081 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.272+0000 7f0bb13c3700 1 -- 172.21.15.144:0/1276611397 learned_addr learned my addr 172.21.15.144:0/1276611397 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:51.081 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.273+0000 7f0bb13c3700 1 -- 172.21.15.144:0/1276611397 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba8009550 con 0x7f0bac199910 2024-09-10T11:20:51.081 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.273+0000 7f0bb13c3700 1 --2- 172.21.15.144:0/1276611397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bac199910 0x7f0bac199d20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8006ba0 tx=0x7f0ba8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:51.081 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.273+0000 7f0ba27fc700 1 -- 172.21.15.144:0/1276611397 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ba8003b00 con 0x7f0bac199910 2024-09-10T11:20:51.082 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.273+0000 7f0bb3627700 1 -- 172.21.15.144:0/1276611397 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bac19a460 con 0x7f0bac199910 2024-09-10T11:20:51.082 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.273+0000 7f0ba27fc700 1 -- 172.21.15.144:0/1276611397 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0ba801a8e0 con 0x7f0bac199910 2024-09-10T11:20:51.082 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.273+0000 7f0ba27fc700 1 -- 172.21.15.144:0/1276611397 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0ba8019990 con 0x7f0bac199910 2024-09-10T11:20:51.082 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.273+0000 7f0bb3627700 1 -- 172.21.15.144:0/1276611397 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bac19d0c0 con 0x7f0bac199910 2024-09-10T11:20:51.082 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.274+0000 7f0bb3627700 1 -- 172.21.15.144:0/1276611397 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bac079d20 con 0x7f0bac199910 2024-09-10T11:20:51.082 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.274+0000 7f0ba27fc700 1 -- 172.21.15.144:0/1276611397 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f0ba8019c50 con 0x7f0bac199910 2024-09-10T11:20:51.082 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.274+0000 7f0ba27fc700 1 --2- 172.21.15.144:0/1276611397 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0b98038490 0x7f0b9803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:51.082 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.275+0000 7f0ba27fc700 1 -- 172.21.15.144:0/1276611397 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f0ba804c750 con 0x7f0bac199910 2024-09-10T11:20:51.082 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.275+0000 7f0bb0bc2700 1 --2- 172.21.15.144:0/1276611397 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0b98038490 0x7f0b9803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:51.082 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.275+0000 7f0bb0bc2700 1 --2- 172.21.15.144:0/1276611397 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0b98038490 0x7f0b9803a940 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c006fd0 tx=0x7f0b9c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:51.082 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.279+0000 7f0ba27fc700 1 -- 172.21.15.144:0/1276611397 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0ba80187c0 con 0x7f0bac199910 2024-09-10T11:20:51.083 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.468+0000 7f0bb3627700 1 -- 172.21.15.144:0/1276611397 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f0bac19a620 con 0x7f0bac199910 2024-09-10T11:20:51.083 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.469+0000 7f0ba27fc700 1 -- 172.21.15.144:0/1276611397 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f0ba80187c0 con 0x7f0bac199910 2024-09-10T11:20:51.083 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.471+0000 7f0bb3627700 1 -- 172.21.15.144:0/1276611397 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0b98038490 msgr2=0x7f0b9803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:51.083 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.471+0000 7f0bb3627700 1 --2- 172.21.15.144:0/1276611397 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0b98038490 0x7f0b9803a940 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0b9c006fd0 tx=0x7f0b9c006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:51.083 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.471+0000 7f0bb3627700 1 -- 172.21.15.144:0/1276611397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bac199910 msgr2=0x7f0bac199d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:51.083 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.471+0000 7f0bb3627700 1 --2- 172.21.15.144:0/1276611397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bac199910 0x7f0bac199d20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8006ba0 tx=0x7f0ba8011b50 comp rx=0 tx=0).stop 2024-09-10T11:20:51.083 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.471+0000 7f0bb3627700 1 -- 172.21.15.144:0/1276611397 shutdown_connections 2024-09-10T11:20:51.083 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.471+0000 7f0bb3627700 1 --2- 172.21.15.144:0/1276611397 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0b98038490 0x7f0b9803a940 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:51.083 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.471+0000 7f0bb3627700 1 --2- 172.21.15.144:0/1276611397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0bac199910 0x7f0bac199d20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:51.083 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.471+0000 7f0bb3627700 1 -- 172.21.15.144:0/1276611397 >> 172.21.15.144:0/1276611397 conn(0x7f0bac1000f0 msgr2=0x7f0bac101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:51.083 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.471+0000 7f0bb3627700 1 -- 172.21.15.144:0/1276611397 shutdown_connections 2024-09-10T11:20:51.083 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:50.471+0000 7f0bb3627700 1 -- 172.21.15.144:0/1276611397 wait complete. 2024-09-10T11:20:52.362 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.494+0000 7f8f6c310700 1 Processor -- start 2024-09-10T11:20:52.362 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.494+0000 7f8f6c310700 1 -- start start 2024-09-10T11:20:52.362 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.494+0000 7f8f6c310700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64105b80 0x7f8f64105f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:52.362 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.495+0000 7f8f6c310700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f641064d0 con 0x7f8f64105b80 2024-09-10T11:20:52.362 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.495+0000 7f8f6a0ac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64105b80 0x7f8f64105f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:52.362 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.495+0000 7f8f6a0ac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64105b80 0x7f8f64105f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36068/0 (socket says 172.21.15.144:36068) 2024-09-10T11:20:52.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.495+0000 7f8f6a0ac700 1 -- 172.21.15.144:0/3804508931 learned_addr learned my addr 172.21.15.144:0/3804508931 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:52.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.495+0000 7f8f6a0ac700 1 -- 172.21.15.144:0/3804508931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f64106610 con 0x7f8f64105b80 2024-09-10T11:20:52.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.495+0000 7f8f6a0ac700 1 --2- 172.21.15.144:0/3804508931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64105b80 0x7f8f64105f90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8f600098a0 tx=0x7f8f60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d6f7a37ade959e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:52.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.495+0000 7f8f690aa700 1 -- 172.21.15.144:0/3804508931 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f60004180 con 0x7f8f64105b80 2024-09-10T11:20:52.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.495+0000 7f8f690aa700 1 -- 172.21.15.144:0/3804508931 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8f600042e0 con 0x7f8f64105b80 2024-09-10T11:20:52.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.495+0000 7f8f690aa700 1 -- 172.21.15.144:0/3804508931 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f60011af0 con 0x7f8f64105b80 2024-09-10T11:20:52.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.496+0000 7f8f6c310700 1 -- 172.21.15.144:0/3804508931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64105b80 msgr2=0x7f8f64105f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:52.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.496+0000 7f8f6c310700 1 --2- 172.21.15.144:0/3804508931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64105b80 0x7f8f64105f90 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8f600098a0 tx=0x7f8f60009bb0 comp rx=0 tx=0).stop 2024-09-10T11:20:52.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.496+0000 7f8f6c310700 1 -- 172.21.15.144:0/3804508931 shutdown_connections 2024-09-10T11:20:52.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.496+0000 7f8f6c310700 1 --2- 172.21.15.144:0/3804508931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64105b80 0x7f8f64105f90 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:52.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.496+0000 7f8f6c310700 1 -- 172.21.15.144:0/3804508931 >> 172.21.15.144:0/3804508931 conn(0x7f8f641011f0 msgr2=0x7f8f64103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:52.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.496+0000 7f8f6c310700 1 -- 172.21.15.144:0/3804508931 shutdown_connections 2024-09-10T11:20:52.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.496+0000 7f8f6c310700 1 -- 172.21.15.144:0/3804508931 wait complete. 2024-09-10T11:20:52.363 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.497+0000 7f8f6c310700 1 Processor -- start 2024-09-10T11:20:52.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.497+0000 7f8f6c310700 1 -- start start 2024-09-10T11:20:52.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.497+0000 7f8f6c310700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64105b80 0x7f8f64199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:52.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.497+0000 7f8f6c310700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f64199e40 con 0x7f8f64105b80 2024-09-10T11:20:52.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.497+0000 7f8f6a0ac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64105b80 0x7f8f64199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:52.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.497+0000 7f8f6a0ac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64105b80 0x7f8f64199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36074/0 (socket says 172.21.15.144:36074) 2024-09-10T11:20:52.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.497+0000 7f8f6a0ac700 1 -- 172.21.15.144:0/147112307 learned_addr learned my addr 172.21.15.144:0/147112307 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:52.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.497+0000 7f8f6a0ac700 1 -- 172.21.15.144:0/147112307 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f60009550 con 0x7f8f64105b80 2024-09-10T11:20:52.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.497+0000 7f8f6a0ac700 1 --2- 172.21.15.144:0/147112307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64105b80 0x7f8f64199900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8f60006ba0 tx=0x7f8f60011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:52.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.497+0000 7f8f5b7fe700 1 -- 172.21.15.144:0/147112307 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f60003b00 con 0x7f8f64105b80 2024-09-10T11:20:52.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.497+0000 7f8f5b7fe700 1 -- 172.21.15.144:0/147112307 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8f6001a8e0 con 0x7f8f64105b80 2024-09-10T11:20:52.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.498+0000 7f8f6c310700 1 -- 172.21.15.144:0/147112307 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f6419a040 con 0x7f8f64105b80 2024-09-10T11:20:52.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.498+0000 7f8f5b7fe700 1 -- 172.21.15.144:0/147112307 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8f600199c0 con 0x7f8f64105b80 2024-09-10T11:20:52.364 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.498+0000 7f8f6c310700 1 -- 172.21.15.144:0/147112307 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f6419a460 con 0x7f8f64105b80 2024-09-10T11:20:52.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.498+0000 7f8f5b7fe700 1 -- 172.21.15.144:0/147112307 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f8f60019c30 con 0x7f8f64105b80 2024-09-10T11:20:52.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.498+0000 7f8f6c310700 1 -- 172.21.15.144:0/147112307 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f64062380 con 0x7f8f64105b80 2024-09-10T11:20:52.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.499+0000 7f8f5b7fe700 1 --2- 172.21.15.144:0/147112307 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f8f500384e0 0x7f8f5003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:52.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.499+0000 7f8f5b7fe700 1 -- 172.21.15.144:0/147112307 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f8f6004c740 con 0x7f8f64105b80 2024-09-10T11:20:52.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.499+0000 7f8f698ab700 1 --2- 172.21.15.144:0/147112307 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f8f500384e0 0x7f8f5003a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:52.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.499+0000 7f8f698ab700 1 --2- 172.21.15.144:0/147112307 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f8f500384e0 0x7f8f5003a990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8f54006fd0 tx=0x7f8f54006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:52.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.503+0000 7f8f5b7fe700 1 -- 172.21.15.144:0/147112307 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8f6001a430 con 0x7f8f64105b80 2024-09-10T11:20:52.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.795+0000 7f8f6c310700 1 -- 172.21.15.144:0/147112307 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f8f6419d2c0 con 0x7f8f64105b80 2024-09-10T11:20:52.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.798+0000 7f8f5b7fe700 1 -- 172.21.15.144:0/147112307 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f8f60018b40 con 0x7f8f64105b80 2024-09-10T11:20:52.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.800+0000 7f8f6c310700 1 -- 172.21.15.144:0/147112307 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f8f500384e0 msgr2=0x7f8f5003a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:52.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.800+0000 7f8f6c310700 1 --2- 172.21.15.144:0/147112307 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f8f500384e0 0x7f8f5003a990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8f54006fd0 tx=0x7f8f54006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:52.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.800+0000 7f8f6c310700 1 -- 172.21.15.144:0/147112307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64105b80 msgr2=0x7f8f64199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:52.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.800+0000 7f8f6c310700 1 --2- 172.21.15.144:0/147112307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64105b80 0x7f8f64199900 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8f60006ba0 tx=0x7f8f60011b50 comp rx=0 tx=0).stop 2024-09-10T11:20:52.365 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.800+0000 7f8f6c310700 1 -- 172.21.15.144:0/147112307 shutdown_connections 2024-09-10T11:20:52.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.800+0000 7f8f6c310700 1 --2- 172.21.15.144:0/147112307 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f8f500384e0 0x7f8f5003a990 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:52.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.800+0000 7f8f6c310700 1 --2- 172.21.15.144:0/147112307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f64105b80 0x7f8f64199900 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:52.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.800+0000 7f8f6c310700 1 -- 172.21.15.144:0/147112307 >> 172.21.15.144:0/147112307 conn(0x7f8f641011f0 msgr2=0x7f8f64101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:52.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.800+0000 7f8f6c310700 1 -- 172.21.15.144:0/147112307 shutdown_connections 2024-09-10T11:20:52.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr 2024-09-10T11:20:51.800+0000 7f8f6c310700 1 -- 172.21.15.144:0/147112307 wait complete. 2024-09-10T11:20:52.366 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T11:20:52.366 INFO:teuthology.orchestra.run.smithi144.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T11:20:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:20:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T11:20:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:20:52.366 INFO:teuthology.orchestra.run.smithi144.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T11:20:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:20:52.366 INFO:teuthology.orchestra.run.smithi144.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T11:20:52.367 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:20:52.367 INFO:teuthology.orchestra.run.smithi144.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T11:20:52.367 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:20:52.367 INFO:teuthology.orchestra.run.smithi144.stdout: ceph telemetry on 2024-09-10T11:20:52.367 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:20:52.367 INFO:teuthology.orchestra.run.smithi144.stdout:For more information see: 2024-09-10T11:20:52.367 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:20:52.367 INFO:teuthology.orchestra.run.smithi144.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-10T11:20:52.367 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:20:52.367 INFO:teuthology.orchestra.run.smithi144.stdout:Bootstrap complete. 2024-09-10T11:20:52.393 INFO:tasks.cephadm:Fetching config... 2024-09-10T11:20:52.394 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:20:52.394 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T11:20:52.410 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T11:20:52.411 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:20:52.411 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T11:20:52.467 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T11:20:52.467 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:20:52.467 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/keyring of=/dev/stdout 2024-09-10T11:20:52.531 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T11:20:52.532 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:20:52.532 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T11:20:52.588 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T11:20:52.588 DEBUG:teuthology.orchestra.run.smithi144:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjP9k8BoC0j2Y3iNix001ZW55e0M9N0D5Wy9/1ttxWhPw175KajJlJ217PQXxstMC544bUEXwW8sRnnQShP7VIRP8UsEVwV5FgGfVhNx6tXHDpR85fUDHB0zoo5yzkRJLKhSFBrcvEWWjOYFl+7mPLngZNrPYmc+h8/W+vH5tB1/UmuqnS5cB/nw32QDcHMi9+EN8pnSuU4+umMUkpXTpBhZ8SmU8nirXZHrvnzl9C3Q2IbPx/RrBYHMiF8LPABe5H0uu7qCLNBaZaQQSFO/4A+/diUy3ud+yZx0rtOpleb1ZWN5KcOL29ILfpXo9royhv+sq1QLLB8Vao+V1hoow5jCL3HpWtM/BXKWqixlF6c5Ls/ZY5SayH+mKcSEbWnS32vMnKEYMY3pfVGgWE9FMHIMbSi6ArehHlg0fZyD43YIIDo8Ndpj5aQrrcfia8GwiMSC4xC7phHSUexjDyz7EvYifCdbeWkQaTLTsRj8J0juxK7NVnsFuoQJ25ydRTHx0= ceph-7eacd86c-6f66-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T11:20:52.662 INFO:teuthology.orchestra.run.smithi144.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjP9k8BoC0j2Y3iNix001ZW55e0M9N0D5Wy9/1ttxWhPw175KajJlJ217PQXxstMC544bUEXwW8sRnnQShP7VIRP8UsEVwV5FgGfVhNx6tXHDpR85fUDHB0zoo5yzkRJLKhSFBrcvEWWjOYFl+7mPLngZNrPYmc+h8/W+vH5tB1/UmuqnS5cB/nw32QDcHMi9+EN8pnSuU4+umMUkpXTpBhZ8SmU8nirXZHrvnzl9C3Q2IbPx/RrBYHMiF8LPABe5H0uu7qCLNBaZaQQSFO/4A+/diUy3ud+yZx0rtOpleb1ZWN5KcOL29ILfpXo9royhv+sq1QLLB8Vao+V1hoow5jCL3HpWtM/BXKWqixlF6c5Ls/ZY5SayH+mKcSEbWnS32vMnKEYMY3pfVGgWE9FMHIMbSi6ArehHlg0fZyD43YIIDo8Ndpj5aQrrcfia8GwiMSC4xC7phHSUexjDyz7EvYifCdbeWkQaTLTsRj8J0juxK7NVnsFuoQJ25ydRTHx0= ceph-7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:20:52.673 DEBUG:teuthology.orchestra.run.smithi178:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjP9k8BoC0j2Y3iNix001ZW55e0M9N0D5Wy9/1ttxWhPw175KajJlJ217PQXxstMC544bUEXwW8sRnnQShP7VIRP8UsEVwV5FgGfVhNx6tXHDpR85fUDHB0zoo5yzkRJLKhSFBrcvEWWjOYFl+7mPLngZNrPYmc+h8/W+vH5tB1/UmuqnS5cB/nw32QDcHMi9+EN8pnSuU4+umMUkpXTpBhZ8SmU8nirXZHrvnzl9C3Q2IbPx/RrBYHMiF8LPABe5H0uu7qCLNBaZaQQSFO/4A+/diUy3ud+yZx0rtOpleb1ZWN5KcOL29ILfpXo9royhv+sq1QLLB8Vao+V1hoow5jCL3HpWtM/BXKWqixlF6c5Ls/ZY5SayH+mKcSEbWnS32vMnKEYMY3pfVGgWE9FMHIMbSi6ArehHlg0fZyD43YIIDo8Ndpj5aQrrcfia8GwiMSC4xC7phHSUexjDyz7EvYifCdbeWkQaTLTsRj8J0juxK7NVnsFuoQJ25ydRTHx0= ceph-7eacd86c-6f66-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T11:20:52.735 INFO:teuthology.orchestra.run.smithi178.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDjP9k8BoC0j2Y3iNix001ZW55e0M9N0D5Wy9/1ttxWhPw175KajJlJ217PQXxstMC544bUEXwW8sRnnQShP7VIRP8UsEVwV5FgGfVhNx6tXHDpR85fUDHB0zoo5yzkRJLKhSFBrcvEWWjOYFl+7mPLngZNrPYmc+h8/W+vH5tB1/UmuqnS5cB/nw32QDcHMi9+EN8pnSuU4+umMUkpXTpBhZ8SmU8nirXZHrvnzl9C3Q2IbPx/RrBYHMiF8LPABe5H0uu7qCLNBaZaQQSFO/4A+/diUy3ud+yZx0rtOpleb1ZWN5KcOL29ILfpXo9royhv+sq1QLLB8Vao+V1hoow5jCL3HpWtM/BXKWqixlF6c5Ls/ZY5SayH+mKcSEbWnS32vMnKEYMY3pfVGgWE9FMHIMbSi6ArehHlg0fZyD43YIIDo8Ndpj5aQrrcfia8GwiMSC4xC7phHSUexjDyz7EvYifCdbeWkQaTLTsRj8J0juxK7NVnsFuoQJ25ydRTHx0= ceph-7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:20:52.748 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T11:20:52.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:52 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/147112307' entity='client.admin' 2024-09-10T11:20:52.952 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:20:55.583 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:55 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:55.583 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:55 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:55.583 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:55 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:20:55.583 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:55 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:20:55.583 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:55 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T11:20:55.583 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:55 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-10T11:20:55.583 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:55 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:20:55.583 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:55 smithi144 ceph-mon[27942]: Deploying daemon ceph-exporter.smithi144 on smithi144 2024-09-10T11:20:55.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.595+0000 7ff1a9818700 1 -- 172.21.15.144:0/2643455827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a40fe750 msgr2=0x7ff1a4100b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:55.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.595+0000 7ff1a9818700 1 --2- 172.21.15.144:0/2643455827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a40fe750 0x7ff1a4100b70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff194009910 tx=0x7ff194009c20 comp rx=0 tx=0).stop 2024-09-10T11:20:55.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.595+0000 7ff1a9818700 1 -- 172.21.15.144:0/2643455827 shutdown_connections 2024-09-10T11:20:55.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.595+0000 7ff1a9818700 1 --2- 172.21.15.144:0/2643455827 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a40fe750 0x7ff1a4100b70 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:55.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.595+0000 7ff1a9818700 1 -- 172.21.15.144:0/2643455827 >> 172.21.15.144:0/2643455827 conn(0x7ff1a40fa380 msgr2=0x7ff1a40fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:55.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.595+0000 7ff1a9818700 1 -- 172.21.15.144:0/2643455827 shutdown_connections 2024-09-10T11:20:55.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.595+0000 7ff1a9818700 1 -- 172.21.15.144:0/2643455827 wait complete. 2024-09-10T11:20:55.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.596+0000 7ff1a9818700 1 Processor -- start 2024-09-10T11:20:55.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.596+0000 7ff1a9818700 1 -- start start 2024-09-10T11:20:55.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.596+0000 7ff1a9818700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a40fe750 0x7ff1a419b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:55.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.596+0000 7ff1a9818700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a419bae0 con 0x7ff1a40fe750 2024-09-10T11:20:55.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.596+0000 7ff1a2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a40fe750 0x7ff1a419b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:55.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.597+0000 7ff1a2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a40fe750 0x7ff1a419b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34228/0 (socket says 172.21.15.144:34228) 2024-09-10T11:20:55.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.597+0000 7ff1a2ffd700 1 -- 172.21.15.144:0/719495249 learned_addr learned my addr 172.21.15.144:0/719495249 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:55.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.597+0000 7ff1a2ffd700 1 -- 172.21.15.144:0/719495249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1940095f0 con 0x7ff1a40fe750 2024-09-10T11:20:55.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.597+0000 7ff1a2ffd700 1 --2- 172.21.15.144:0/719495249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a40fe750 0x7ff1a419b5a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff194005b40 tx=0x7ff194004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:55.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.597+0000 7ff1a8816700 1 -- 172.21.15.144:0/719495249 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff19401d070 con 0x7ff1a40fe750 2024-09-10T11:20:55.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.597+0000 7ff1a9818700 1 -- 172.21.15.144:0/719495249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a419bce0 con 0x7ff1a40fe750 2024-09-10T11:20:55.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.597+0000 7ff1a8816700 1 -- 172.21.15.144:0/719495249 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff19400be10 con 0x7ff1a40fe750 2024-09-10T11:20:55.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.597+0000 7ff1a8816700 1 -- 172.21.15.144:0/719495249 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff19400f460 con 0x7ff1a40fe750 2024-09-10T11:20:55.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.597+0000 7ff1a9818700 1 -- 172.21.15.144:0/719495249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a419c180 con 0x7ff1a40fe750 2024-09-10T11:20:55.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.598+0000 7ff1a8816700 1 -- 172.21.15.144:0/719495249 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7ff19400f6d0 con 0x7ff1a40fe750 2024-09-10T11:20:55.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.598+0000 7ff1a9818700 1 -- 172.21.15.144:0/719495249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a4195a40 con 0x7ff1a40fe750 2024-09-10T11:20:55.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.598+0000 7ff1a8816700 1 --2- 172.21.15.144:0/719495249 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff18c038490 0x7ff18c03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:55.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.598+0000 7ff1a8816700 1 -- 172.21.15.144:0/719495249 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7ff19404c560 con 0x7ff1a40fe750 2024-09-10T11:20:55.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.599+0000 7ff1a27fc700 1 --2- 172.21.15.144:0/719495249 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff18c038490 0x7ff18c03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:55.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.599+0000 7ff1a27fc700 1 --2- 172.21.15.144:0/719495249 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff18c038490 0x7ff18c03a940 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff198006fd0 tx=0x7ff198006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:55.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.604+0000 7ff1a8816700 1 -- 172.21.15.144:0/719495249 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff19402a360 con 0x7ff1a40fe750 2024-09-10T11:20:55.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.802+0000 7ff1a9818700 1 -- 172.21.15.144:0/719495249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7ff1a402cc30 con 0x7ff1a40fe750 2024-09-10T11:20:55.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.806+0000 7ff1a8816700 1 -- 172.21.15.144:0/719495249 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7ff194027030 con 0x7ff1a40fe750 2024-09-10T11:20:55.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.808+0000 7ff1a9818700 1 -- 172.21.15.144:0/719495249 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff18c038490 msgr2=0x7ff18c03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:55.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.808+0000 7ff1a9818700 1 --2- 172.21.15.144:0/719495249 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff18c038490 0x7ff18c03a940 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff198006fd0 tx=0x7ff198006e40 comp rx=0 tx=0).stop 2024-09-10T11:20:55.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.808+0000 7ff1a9818700 1 -- 172.21.15.144:0/719495249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a40fe750 msgr2=0x7ff1a419b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:55.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.808+0000 7ff1a9818700 1 --2- 172.21.15.144:0/719495249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a40fe750 0x7ff1a419b5a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff194005b40 tx=0x7ff194004740 comp rx=0 tx=0).stop 2024-09-10T11:20:55.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.808+0000 7ff1a9818700 1 -- 172.21.15.144:0/719495249 shutdown_connections 2024-09-10T11:20:55.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.808+0000 7ff1a9818700 1 --2- 172.21.15.144:0/719495249 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff18c038490 0x7ff18c03a940 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:55.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.808+0000 7ff1a9818700 1 --2- 172.21.15.144:0/719495249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1a40fe750 0x7ff1a419b5a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:55.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.808+0000 7ff1a9818700 1 -- 172.21.15.144:0/719495249 >> 172.21.15.144:0/719495249 conn(0x7ff1a40fa380 msgr2=0x7ff1a40fb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:55.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.808+0000 7ff1a9818700 1 -- 172.21.15.144:0/719495249 shutdown_connections 2024-09-10T11:20:55.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:55.808+0000 7ff1a9818700 1 -- 172.21.15.144:0/719495249 wait complete. 2024-09-10T11:20:56.459 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T11:20:56.459 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T11:20:57.141 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:20:56 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/719495249' entity='client.admin' 2024-09-10T11:20:57.897 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:20:59.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.935+0000 7fe7e0347700 1 -- 172.21.15.144:0/3197515597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7d80fed30 msgr2=0x7fe7d80ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:20:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.935+0000 7fe7e0347700 1 --2- 172.21.15.144:0/3197515597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7d80fed30 0x7fe7d80ff140 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800b0a0 tx=0x7fe7c800b3b0 comp rx=0 tx=0).stop 2024-09-10T11:20:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.935+0000 7fe7e0347700 1 -- 172.21.15.144:0/3197515597 shutdown_connections 2024-09-10T11:20:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.935+0000 7fe7e0347700 1 --2- 172.21.15.144:0/3197515597 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7d80fed30 0x7fe7d80ff140 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:20:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.935+0000 7fe7e0347700 1 -- 172.21.15.144:0/3197515597 >> 172.21.15.144:0/3197515597 conn(0x7fe7d80fa2e0 msgr2=0x7fe7d80fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:20:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.935+0000 7fe7e0347700 1 -- 172.21.15.144:0/3197515597 shutdown_connections 2024-09-10T11:20:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.935+0000 7fe7e0347700 1 -- 172.21.15.144:0/3197515597 wait complete. 2024-09-10T11:20:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.935+0000 7fe7e0347700 1 Processor -- start 2024-09-10T11:20:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.935+0000 7fe7e0347700 1 -- start start 2024-09-10T11:20:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.936+0000 7fe7e0347700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7d80fed30 0x7fe7d8068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.936+0000 7fe7e0347700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7d8197d20 con 0x7fe7d80fed30 2024-09-10T11:20:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.936+0000 7fe7de0e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7d80fed30 0x7fe7d8068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.936+0000 7fe7de0e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7d80fed30 0x7fe7d8068ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34266/0 (socket says 172.21.15.144:34266) 2024-09-10T11:20:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.936+0000 7fe7de0e3700 1 -- 172.21.15.144:0/3924666285 learned_addr learned my addr 172.21.15.144:0/3924666285 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:20:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.936+0000 7fe7de0e3700 1 -- 172.21.15.144:0/3924666285 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c8009d00 con 0x7fe7d80fed30 2024-09-10T11:20:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.936+0000 7fe7de0e3700 1 --2- 172.21.15.144:0/3924666285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7d80fed30 0x7fe7d8068ac0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800b830 tx=0x7fe7c800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.936+0000 7fe7d77fe700 1 -- 172.21.15.144:0/3924666285 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7c800e040 con 0x7fe7d80fed30 2024-09-10T11:20:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.936+0000 7fe7d77fe700 1 -- 172.21.15.144:0/3924666285 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe7c8012af0 con 0x7fe7d80fed30 2024-09-10T11:20:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.936+0000 7fe7e0347700 1 -- 172.21.15.144:0/3924666285 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7d8197e60 con 0x7fe7d80fed30 2024-09-10T11:20:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.936+0000 7fe7d77fe700 1 -- 172.21.15.144:0/3924666285 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe7c801d9f0 con 0x7fe7d80fed30 2024-09-10T11:20:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.936+0000 7fe7e0347700 1 -- 172.21.15.144:0/3924666285 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7d8198220 con 0x7fe7d80fed30 2024-09-10T11:20:59.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.937+0000 7fe7d77fe700 1 -- 172.21.15.144:0/3924666285 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7fe7c801dcb0 con 0x7fe7d80fed30 2024-09-10T11:20:59.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.937+0000 7fe7e0347700 1 -- 172.21.15.144:0/3924666285 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7d818da10 con 0x7fe7d80fed30 2024-09-10T11:20:59.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.938+0000 7fe7d77fe700 1 --2- 172.21.15.144:0/3924666285 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe7c0038490 0x7fe7c003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:20:59.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.938+0000 7fe7d77fe700 1 -- 172.21.15.144:0/3924666285 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fe7c804c980 con 0x7fe7d80fed30 2024-09-10T11:20:59.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.938+0000 7fe7dd8e2700 1 --2- 172.21.15.144:0/3924666285 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe7c0038490 0x7fe7c003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:20:59.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.938+0000 7fe7dd8e2700 1 --2- 172.21.15.144:0/3924666285 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe7c0038490 0x7fe7c003a940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc006fd0 tx=0x7fe7cc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:20:59.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:20:59.943+0000 7fe7d77fe700 1 -- 172.21.15.144:0/3924666285 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe7c801ae60 con 0x7fe7d80fed30 2024-09-10T11:21:00.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:00.144+0000 7fe7e0347700 1 -- 172.21.15.144:0/3924666285 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7d8061220 con 0x7fe7c0038490 2024-09-10T11:21:00.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:00.148+0000 7fe7d77fe700 1 -- 172.21.15.144:0/3924666285 <== mgr.14164 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe7d8061220 con 0x7fe7c0038490 2024-09-10T11:21:00.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:00.150+0000 7fe7e0347700 1 -- 172.21.15.144:0/3924666285 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe7c0038490 msgr2=0x7fe7c003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:00.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:00.150+0000 7fe7e0347700 1 --2- 172.21.15.144:0/3924666285 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe7c0038490 0x7fe7c003a940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc006fd0 tx=0x7fe7cc006e40 comp rx=0 tx=0).stop 2024-09-10T11:21:00.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:00.150+0000 7fe7e0347700 1 -- 172.21.15.144:0/3924666285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7d80fed30 msgr2=0x7fe7d8068ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:00.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:00.150+0000 7fe7e0347700 1 --2- 172.21.15.144:0/3924666285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7d80fed30 0x7fe7d8068ac0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fe7c800b830 tx=0x7fe7c800bd40 comp rx=0 tx=0).stop 2024-09-10T11:21:00.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:00.150+0000 7fe7e0347700 1 -- 172.21.15.144:0/3924666285 shutdown_connections 2024-09-10T11:21:00.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:00.150+0000 7fe7e0347700 1 --2- 172.21.15.144:0/3924666285 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe7c0038490 0x7fe7c003a940 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:00.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:00.150+0000 7fe7e0347700 1 --2- 172.21.15.144:0/3924666285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7d80fed30 0x7fe7d8068ac0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:00.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:00.150+0000 7fe7e0347700 1 -- 172.21.15.144:0/3924666285 >> 172.21.15.144:0/3924666285 conn(0x7fe7d80fa2e0 msgr2=0x7fe7d80faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:00.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:00.150+0000 7fe7e0347700 1 -- 172.21.15.144:0/3924666285 shutdown_connections 2024-09-10T11:21:00.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:00.150+0000 7fe7e0347700 1 -- 172.21.15.144:0/3924666285 wait complete. 2024-09-10T11:21:00.722 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:00 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:00.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:00 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:00.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:00 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:00.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:00 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:00.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:00 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T11:21:00.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:00 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-10T11:21:00.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:00 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:21:00.723 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:00 smithi144 ceph-mon[27942]: Deploying daemon crash.smithi144 on smithi144 2024-09-10T11:21:00.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:00 smithi144 ceph-mon[27942]: from='client.14189 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:21:00.724 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:00 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:00.742 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi178 2024-09-10T11:21:00.743 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T11:21:00.743 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.conf 2024-09-10T11:21:00.761 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T11:21:00.762 DEBUG:teuthology.orchestra.run.smithi178:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:21:00.820 INFO:tasks.cephadm:Adding host smithi178 to orchestrator... 2024-09-10T11:21:00.820 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph orch host add smithi178 2024-09-10T11:21:01.531 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:21:03.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.504+0000 7f002fa20700 1 -- 172.21.15.144:0/3283036002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00280fed20 msgr2=0x7f00280ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:03.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.504+0000 7f002fa20700 1 --2- 172.21.15.144:0/3283036002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00280fed20 0x7f00280ff130 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0024009910 tx=0x7f0024009c20 comp rx=0 tx=0).stop 2024-09-10T11:21:03.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.504+0000 7f002fa20700 1 -- 172.21.15.144:0/3283036002 shutdown_connections 2024-09-10T11:21:03.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.504+0000 7f002fa20700 1 --2- 172.21.15.144:0/3283036002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00280fed20 0x7f00280ff130 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:03.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.504+0000 7f002fa20700 1 -- 172.21.15.144:0/3283036002 >> 172.21.15.144:0/3283036002 conn(0x7f00280fa370 msgr2=0x7f00280fc7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:03.508 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.504+0000 7f002fa20700 1 -- 172.21.15.144:0/3283036002 shutdown_connections 2024-09-10T11:21:03.508 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.504+0000 7f002fa20700 1 -- 172.21.15.144:0/3283036002 wait complete. 2024-09-10T11:21:03.508 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.505+0000 7f002fa20700 1 Processor -- start 2024-09-10T11:21:03.508 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.505+0000 7f002fa20700 1 -- start start 2024-09-10T11:21:03.508 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.505+0000 7f002fa20700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00280fed20 0x7f0028069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:03.508 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.505+0000 7f002fa20700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0028197cb0 con 0x7f00280fed20 2024-09-10T11:21:03.508 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.505+0000 7f002d7bc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00280fed20 0x7f0028069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:03.508 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.505+0000 7f002d7bc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00280fed20 0x7f0028069390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58038/0 (socket says 172.21.15.144:58038) 2024-09-10T11:21:03.508 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.505+0000 7f002d7bc700 1 -- 172.21.15.144:0/3201101927 learned_addr learned my addr 172.21.15.144:0/3201101927 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:21:03.508 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.506+0000 7f002d7bc700 1 -- 172.21.15.144:0/3201101927 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00240095f0 con 0x7f00280fed20 2024-09-10T11:21:03.508 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.506+0000 7f002d7bc700 1 --2- 172.21.15.144:0/3201101927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00280fed20 0x7f0028069390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0024005b40 tx=0x7f0024004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:03.508 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.506+0000 7f001e7fc700 1 -- 172.21.15.144:0/3201101927 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f002401d070 con 0x7f00280fed20 2024-09-10T11:21:03.508 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.506+0000 7f001e7fc700 1 -- 172.21.15.144:0/3201101927 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f002400be10 con 0x7f00280fed20 2024-09-10T11:21:03.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.506+0000 7f002fa20700 1 -- 172.21.15.144:0/3201101927 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0028197df0 con 0x7f00280fed20 2024-09-10T11:21:03.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.506+0000 7f001e7fc700 1 -- 172.21.15.144:0/3201101927 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f002400f460 con 0x7f00280fed20 2024-09-10T11:21:03.510 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.506+0000 7f002fa20700 1 -- 172.21.15.144:0/3201101927 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0028198230 con 0x7f00280fed20 2024-09-10T11:21:03.510 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.507+0000 7f002fa20700 1 -- 172.21.15.144:0/3201101927 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f002818d980 con 0x7f00280fed20 2024-09-10T11:21:03.510 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.507+0000 7f001e7fc700 1 -- 172.21.15.144:0/3201101927 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f002400f6d0 con 0x7f00280fed20 2024-09-10T11:21:03.510 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.507+0000 7f001e7fc700 1 --2- 172.21.15.144:0/3201101927 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0014038490 0x7f001403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:03.510 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.507+0000 7f001e7fc700 1 -- 172.21.15.144:0/3201101927 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f002404c560 con 0x7f00280fed20 2024-09-10T11:21:03.510 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.508+0000 7f002cfbb700 1 --2- 172.21.15.144:0/3201101927 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0014038490 0x7f001403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:03.510 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.508+0000 7f002cfbb700 1 --2- 172.21.15.144:0/3201101927 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0014038490 0x7f001403a940 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0018006fd0 tx=0x7f0018006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:03.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.512+0000 7f001e7fc700 1 -- 172.21.15.144:0/3201101927 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f002402a360 con 0x7f00280fed20 2024-09-10T11:21:03.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:03.727+0000 7f002fa20700 1 -- 172.21.15.144:0/3201101927 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}) v1 -- 0x7f0028061190 con 0x7f0014038490 2024-09-10T11:21:03.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:03 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:03.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:03 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:03.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:03 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:03.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:03 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:03.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:03 smithi144 ceph-mon[27942]: Deploying daemon node-exporter.smithi144 on smithi144 2024-09-10T11:21:03.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:03 smithi144 ceph-mon[27942]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:03.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:03 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:04.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:04 smithi144 ceph-mon[27942]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:21:05.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:05 smithi144 ceph-mon[27942]: Deploying cephadm binary to smithi178 2024-09-10T11:21:05.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:05 smithi144 ceph-mon[27942]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:06.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:06.169+0000 7f001e7fc700 1 -- 172.21.15.144:0/3201101927 <== mgr.14164 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f0028061190 con 0x7f0014038490 2024-09-10T11:21:06.171 INFO:teuthology.orchestra.run.smithi144.stdout:Added host 'smithi178' with addr '172.21.15.178' 2024-09-10T11:21:06.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:06.171+0000 7f002fa20700 1 -- 172.21.15.144:0/3201101927 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0014038490 msgr2=0x7f001403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:06.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:06.172+0000 7f002fa20700 1 --2- 172.21.15.144:0/3201101927 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0014038490 0x7f001403a940 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0018006fd0 tx=0x7f0018006e40 comp rx=0 tx=0).stop 2024-09-10T11:21:06.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:06.172+0000 7f002fa20700 1 -- 172.21.15.144:0/3201101927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00280fed20 msgr2=0x7f0028069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:06.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:06.172+0000 7f002fa20700 1 --2- 172.21.15.144:0/3201101927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00280fed20 0x7f0028069390 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0024005b40 tx=0x7f0024004740 comp rx=0 tx=0).stop 2024-09-10T11:21:06.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:06.172+0000 7f002fa20700 1 -- 172.21.15.144:0/3201101927 shutdown_connections 2024-09-10T11:21:06.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:06.172+0000 7f002fa20700 1 --2- 172.21.15.144:0/3201101927 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0014038490 0x7f001403a940 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:06.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:06.172+0000 7f002fa20700 1 --2- 172.21.15.144:0/3201101927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00280fed20 0x7f0028069390 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:06.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:06.172+0000 7f002fa20700 1 -- 172.21.15.144:0/3201101927 >> 172.21.15.144:0/3201101927 conn(0x7f00280fa370 msgr2=0x7f00280fb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:06.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:06.172+0000 7f002fa20700 1 -- 172.21.15.144:0/3201101927 shutdown_connections 2024-09-10T11:21:06.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:06.172+0000 7f002fa20700 1 -- 172.21.15.144:0/3201101927 wait complete. 2024-09-10T11:21:06.777 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T11:21:07.245 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:21:07.366 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:07 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:07.367 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:07 smithi144 ceph-mon[27942]: Added host smithi178 2024-09-10T11:21:08.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:08 smithi144 ceph-mon[27942]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:08.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:08 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:08.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:08 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:08.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:08 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:08.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:08 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:08.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:08 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:08.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.790+0000 7f2bca546700 1 -- 172.21.15.144:0/1381656373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bc40fe110 msgr2=0x7f2bc40fe520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:08.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.790+0000 7f2bca546700 1 --2- 172.21.15.144:0/1381656373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bc40fe110 0x7f2bc40fe520 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4009910 tx=0x7f2bb4009c20 comp rx=0 tx=0).stop 2024-09-10T11:21:08.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.790+0000 7f2bca546700 1 -- 172.21.15.144:0/1381656373 shutdown_connections 2024-09-10T11:21:08.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.790+0000 7f2bca546700 1 --2- 172.21.15.144:0/1381656373 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bc40fe110 0x7f2bc40fe520 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:08.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.790+0000 7f2bca546700 1 -- 172.21.15.144:0/1381656373 >> 172.21.15.144:0/1381656373 conn(0x7f2bc40f9a60 msgr2=0x7f2bc40fbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:08.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.790+0000 7f2bca546700 1 -- 172.21.15.144:0/1381656373 shutdown_connections 2024-09-10T11:21:08.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.790+0000 7f2bca546700 1 -- 172.21.15.144:0/1381656373 wait complete. 2024-09-10T11:21:08.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.791+0000 7f2bca546700 1 Processor -- start 2024-09-10T11:21:08.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.791+0000 7f2bca546700 1 -- start start 2024-09-10T11:21:08.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.791+0000 7f2bca546700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bc40fe110 0x7f2bc4194f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:08.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.791+0000 7f2bca546700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bc41954d0 con 0x7f2bc40fe110 2024-09-10T11:21:08.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.791+0000 7f2bc3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bc40fe110 0x7f2bc4194f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:08.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.791+0000 7f2bc3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bc40fe110 0x7f2bc4194f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58056/0 (socket says 172.21.15.144:58056) 2024-09-10T11:21:08.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.791+0000 7f2bc3fff700 1 -- 172.21.15.144:0/226472334 learned_addr learned my addr 172.21.15.144:0/226472334 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:21:08.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.792+0000 7f2bc3fff700 1 -- 172.21.15.144:0/226472334 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb40095f0 con 0x7f2bc40fe110 2024-09-10T11:21:08.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.792+0000 7f2bc3fff700 1 --2- 172.21.15.144:0/226472334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bc40fe110 0x7f2bc4194f90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4005b40 tx=0x7f2bb4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:08.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.792+0000 7f2bc17fa700 1 -- 172.21.15.144:0/226472334 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2bb401d070 con 0x7f2bc40fe110 2024-09-10T11:21:08.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.792+0000 7f2bca546700 1 -- 172.21.15.144:0/226472334 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc41956d0 con 0x7f2bc40fe110 2024-09-10T11:21:08.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.792+0000 7f2bc17fa700 1 -- 172.21.15.144:0/226472334 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2bb400be10 con 0x7f2bc40fe110 2024-09-10T11:21:08.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.792+0000 7f2bc17fa700 1 -- 172.21.15.144:0/226472334 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2bb400f460 con 0x7f2bc40fe110 2024-09-10T11:21:08.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.792+0000 7f2bca546700 1 -- 172.21.15.144:0/226472334 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bc4195b70 con 0x7f2bc40fe110 2024-09-10T11:21:08.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.793+0000 7f2bc17fa700 1 -- 172.21.15.144:0/226472334 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f2bb400f5c0 con 0x7f2bc40fe110 2024-09-10T11:21:08.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.793+0000 7f2bca546700 1 -- 172.21.15.144:0/226472334 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bc418eb20 con 0x7f2bc40fe110 2024-09-10T11:21:08.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.793+0000 7f2bc17fa700 1 --2- 172.21.15.144:0/226472334 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2bac0384e0 0x7f2bac03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:08.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.793+0000 7f2bc17fa700 1 -- 172.21.15.144:0/226472334 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f2bb404c440 con 0x7f2bc40fe110 2024-09-10T11:21:08.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.793+0000 7f2bc37fe700 1 --2- 172.21.15.144:0/226472334 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2bac0384e0 0x7f2bac03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:08.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.794+0000 7f2bc37fe700 1 --2- 172.21.15.144:0/226472334 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2bac0384e0 0x7f2bac03a990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8006fd0 tx=0x7f2bb8006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:08.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:08.798+0000 7f2bc17fa700 1 -- 172.21.15.144:0/226472334 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2bb402a360 con 0x7f2bc40fe110 2024-09-10T11:21:09.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:09.029+0000 7f2bca546700 1 -- 172.21.15.144:0/226472334 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2bc4061190 con 0x7f2bac0384e0 2024-09-10T11:21:09.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:09.030+0000 7f2bc17fa700 1 -- 172.21.15.144:0/226472334 <== mgr.14164 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f2bc4061190 con 0x7f2bac0384e0 2024-09-10T11:21:09.032 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:21:09.032 INFO:teuthology.orchestra.run.smithi144.stdout:[{"addr": "172.21.15.144", "hostname": "smithi144", "labels": [], "status": ""}, {"addr": "172.21.15.178", "hostname": "smithi178", "labels": [], "status": ""}] 2024-09-10T11:21:09.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:09.032+0000 7f2bca546700 1 -- 172.21.15.144:0/226472334 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2bac0384e0 msgr2=0x7f2bac03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:09.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:09.032+0000 7f2bca546700 1 --2- 172.21.15.144:0/226472334 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2bac0384e0 0x7f2bac03a990 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2bb8006fd0 tx=0x7f2bb8006e40 comp rx=0 tx=0).stop 2024-09-10T11:21:09.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:09.032+0000 7f2bca546700 1 -- 172.21.15.144:0/226472334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bc40fe110 msgr2=0x7f2bc4194f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:09.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:09.032+0000 7f2bca546700 1 --2- 172.21.15.144:0/226472334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bc40fe110 0x7f2bc4194f90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4005b40 tx=0x7f2bb4004740 comp rx=0 tx=0).stop 2024-09-10T11:21:09.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:09.033+0000 7f2bca546700 1 -- 172.21.15.144:0/226472334 shutdown_connections 2024-09-10T11:21:09.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:09.033+0000 7f2bca546700 1 --2- 172.21.15.144:0/226472334 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2bac0384e0 0x7f2bac03a990 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:09.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:09.033+0000 7f2bca546700 1 --2- 172.21.15.144:0/226472334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2bc40fe110 0x7f2bc4194f90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:09.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:09.033+0000 7f2bca546700 1 -- 172.21.15.144:0/226472334 >> 172.21.15.144:0/226472334 conn(0x7f2bc40f9a60 msgr2=0x7f2bc40fa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:09.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:09.033+0000 7f2bca546700 1 -- 172.21.15.144:0/226472334 shutdown_connections 2024-09-10T11:21:09.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:09.033+0000 7f2bca546700 1 -- 172.21.15.144:0/226472334 wait complete. 2024-09-10T11:21:09.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:09 smithi144 ceph-mon[27942]: Deploying daemon alertmanager.smithi144 on smithi144 2024-09-10T11:21:10.167 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T11:21:10.168 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T11:21:10.382 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:21:10.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:10 smithi144 ceph-mon[27942]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:10.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:10 smithi144 ceph-mon[27942]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T11:21:12.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:12 smithi144 ceph-mon[27942]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:12.927 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.890+0000 7fe723887700 1 -- 172.21.15.144:0/727320251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe71c0ffe40 msgr2=0x7fe71c100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:12.927 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.890+0000 7fe723887700 1 --2- 172.21.15.144:0/727320251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe71c0ffe40 0x7fe71c100250 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe718009910 tx=0x7fe718009c20 comp rx=0 tx=0).stop 2024-09-10T11:21:12.927 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.890+0000 7fe723887700 1 -- 172.21.15.144:0/727320251 shutdown_connections 2024-09-10T11:21:12.927 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.890+0000 7fe723887700 1 --2- 172.21.15.144:0/727320251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe71c0ffe40 0x7fe71c100250 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:12.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.890+0000 7fe723887700 1 -- 172.21.15.144:0/727320251 >> 172.21.15.144:0/727320251 conn(0x7fe71c0fb3f0 msgr2=0x7fe71c0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:12.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.890+0000 7fe723887700 1 -- 172.21.15.144:0/727320251 shutdown_connections 2024-09-10T11:21:12.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.890+0000 7fe723887700 1 -- 172.21.15.144:0/727320251 wait complete. 2024-09-10T11:21:12.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.891+0000 7fe723887700 1 Processor -- start 2024-09-10T11:21:12.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.891+0000 7fe723887700 1 -- start start 2024-09-10T11:21:12.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.891+0000 7fe723887700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe71c0ffe40 0x7fe71c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:12.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.891+0000 7fe723887700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe71c075030 con 0x7fe71c0ffe40 2024-09-10T11:21:12.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.891+0000 7fe721623700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe71c0ffe40 0x7fe71c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:12.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.891+0000 7fe721623700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe71c0ffe40 0x7fe71c074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:47028/0 (socket says 172.21.15.144:47028) 2024-09-10T11:21:12.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.891+0000 7fe721623700 1 -- 172.21.15.144:0/1659915210 learned_addr learned my addr 172.21.15.144:0/1659915210 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:21:12.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.891+0000 7fe721623700 1 -- 172.21.15.144:0/1659915210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7180095f0 con 0x7fe71c0ffe40 2024-09-10T11:21:12.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.891+0000 7fe721623700 1 --2- 172.21.15.144:0/1659915210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe71c0ffe40 0x7fe71c074af0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe718005b40 tx=0x7fe718004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:12.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.892+0000 7fe7127fc700 1 -- 172.21.15.144:0/1659915210 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe71801d070 con 0x7fe71c0ffe40 2024-09-10T11:21:12.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.892+0000 7fe7127fc700 1 -- 172.21.15.144:0/1659915210 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe71800be10 con 0x7fe71c0ffe40 2024-09-10T11:21:12.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.892+0000 7fe723887700 1 -- 172.21.15.144:0/1659915210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe71c073140 con 0x7fe71c0ffe40 2024-09-10T11:21:12.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.892+0000 7fe7127fc700 1 -- 172.21.15.144:0/1659915210 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe71800f460 con 0x7fe71c0ffe40 2024-09-10T11:21:12.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.892+0000 7fe723887700 1 -- 172.21.15.144:0/1659915210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe71c0735e0 con 0x7fe71c0ffe40 2024-09-10T11:21:12.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.893+0000 7fe7127fc700 1 -- 172.21.15.144:0/1659915210 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7fe71800f6d0 con 0x7fe71c0ffe40 2024-09-10T11:21:12.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.893+0000 7fe723887700 1 -- 172.21.15.144:0/1659915210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71c062380 con 0x7fe71c0ffe40 2024-09-10T11:21:12.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.893+0000 7fe7127fc700 1 --2- 172.21.15.144:0/1659915210 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe708038490 0x7fe70803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:12.930 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.893+0000 7fe7127fc700 1 -- 172.21.15.144:0/1659915210 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fe71804c370 con 0x7fe71c0ffe40 2024-09-10T11:21:12.930 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.893+0000 7fe720e22700 1 --2- 172.21.15.144:0/1659915210 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe708038490 0x7fe70803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:12.930 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.893+0000 7fe720e22700 1 --2- 172.21.15.144:0/1659915210 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe708038490 0x7fe70803a940 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe70c006fd0 tx=0x7fe70c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:12.930 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:12.898+0000 7fe7127fc700 1 -- 172.21.15.144:0/1659915210 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe718020370 con 0x7fe71c0ffe40 2024-09-10T11:21:13.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:13.101+0000 7fe723887700 1 -- 172.21.15.144:0/1659915210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7fe71c18e480 con 0x7fe71c0ffe40 2024-09-10T11:21:13.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:13.269+0000 7fe7127fc700 1 -- 172.21.15.144:0/1659915210 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe718027030 con 0x7fe71c0ffe40 2024-09-10T11:21:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:13.271+0000 7fe723887700 1 -- 172.21.15.144:0/1659915210 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe708038490 msgr2=0x7fe70803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:13.271+0000 7fe723887700 1 --2- 172.21.15.144:0/1659915210 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe708038490 0x7fe70803a940 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe70c006fd0 tx=0x7fe70c006e40 comp rx=0 tx=0).stop 2024-09-10T11:21:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:13.271+0000 7fe723887700 1 -- 172.21.15.144:0/1659915210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe71c0ffe40 msgr2=0x7fe71c074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:13.271+0000 7fe723887700 1 --2- 172.21.15.144:0/1659915210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe71c0ffe40 0x7fe71c074af0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe718005b40 tx=0x7fe718004740 comp rx=0 tx=0).stop 2024-09-10T11:21:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:13.271+0000 7fe723887700 1 -- 172.21.15.144:0/1659915210 shutdown_connections 2024-09-10T11:21:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:13.271+0000 7fe723887700 1 --2- 172.21.15.144:0/1659915210 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe708038490 0x7fe70803a940 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:13.271+0000 7fe723887700 1 --2- 172.21.15.144:0/1659915210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe71c0ffe40 0x7fe71c074af0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:13.271+0000 7fe723887700 1 -- 172.21.15.144:0/1659915210 >> 172.21.15.144:0/1659915210 conn(0x7fe71c0fb3f0 msgr2=0x7fe71c0fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:13.271+0000 7fe723887700 1 -- 172.21.15.144:0/1659915210 shutdown_connections 2024-09-10T11:21:13.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:21:13.271+0000 7fe723887700 1 -- 172.21.15.144:0/1659915210 wait complete. 2024-09-10T11:21:13.274 INFO:teuthology.orchestra.run.smithi144.stderr:adjusted tunables profile to default 2024-09-10T11:21:13.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:13 smithi144 ceph-mon[27942]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:13.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:13 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1659915210' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T11:21:14.320 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:14 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1659915210' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T11:21:14.321 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:14 smithi144 ceph-mon[27942]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T11:21:14.347 INFO:tasks.cephadm:Adding mon.smithi144 on smithi144 2024-09-10T11:21:14.347 INFO:tasks.cephadm:Adding mon.smithi178 on smithi178 2024-09-10T11:21:14.347 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph orch apply mon '2;smithi144:172.21.15.144=smithi144;smithi178:172.21.15.178=smithi178' 2024-09-10T11:21:14.570 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:14.614 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:15.578 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:15 smithi144 ceph-mon[27942]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:16.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.072+0000 7fd3a1742700 1 -- 172.21.15.178:0/1851145542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c100340 msgr2=0x7fd39c100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:16.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.072+0000 7fd3a1742700 1 --2- 172.21.15.178:0/1851145542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c100340 0x7fd39c100750 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd38c009910 tx=0x7fd38c009c20 comp rx=0 tx=0).stop 2024-09-10T11:21:16.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.072+0000 7fd3a1742700 1 -- 172.21.15.178:0/1851145542 shutdown_connections 2024-09-10T11:21:16.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.072+0000 7fd3a1742700 1 --2- 172.21.15.178:0/1851145542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c100340 0x7fd39c100750 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:16.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.072+0000 7fd3a1742700 1 -- 172.21.15.178:0/1851145542 >> 172.21.15.178:0/1851145542 conn(0x7fd39c0f9a60 msgr2=0x7fd39c0fbe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:16.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.072+0000 7fd3a1742700 1 -- 172.21.15.178:0/1851145542 shutdown_connections 2024-09-10T11:21:16.074 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.072+0000 7fd3a1742700 1 -- 172.21.15.178:0/1851145542 wait complete. 2024-09-10T11:21:16.075 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.073+0000 7fd3a1742700 1 Processor -- start 2024-09-10T11:21:16.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.073+0000 7fd3a1742700 1 -- start start 2024-09-10T11:21:16.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.073+0000 7fd3a1742700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c100340 0x7fd39c194fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:16.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.073+0000 7fd3a1742700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd39c195510 con 0x7fd39c100340 2024-09-10T11:21:16.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.074+0000 7fd39affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c100340 0x7fd39c194fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:16.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.074+0000 7fd39affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c100340 0x7fd39c194fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:52006/0 (socket says 172.21.15.178:52006) 2024-09-10T11:21:16.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.074+0000 7fd39affd700 1 -- 172.21.15.178:0/2489314692 learned_addr learned my addr 172.21.15.178:0/2489314692 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:21:16.076 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.074+0000 7fd39affd700 1 -- 172.21.15.178:0/2489314692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c0095f0 con 0x7fd39c100340 2024-09-10T11:21:16.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.074+0000 7fd39affd700 1 --2- 172.21.15.178:0/2489314692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c100340 0x7fd39c194fd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd38c005b40 tx=0x7fd38c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:16.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.074+0000 7fd398ff9700 1 -- 172.21.15.178:0/2489314692 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd38c01d070 con 0x7fd39c100340 2024-09-10T11:21:16.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.074+0000 7fd398ff9700 1 -- 172.21.15.178:0/2489314692 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd38c00be10 con 0x7fd39c100340 2024-09-10T11:21:16.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.074+0000 7fd3a1742700 1 -- 172.21.15.178:0/2489314692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd39c195710 con 0x7fd39c100340 2024-09-10T11:21:16.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.074+0000 7fd398ff9700 1 -- 172.21.15.178:0/2489314692 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd38c00f460 con 0x7fd39c100340 2024-09-10T11:21:16.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.074+0000 7fd3a1742700 1 -- 172.21.15.178:0/2489314692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd39c195bb0 con 0x7fd39c100340 2024-09-10T11:21:16.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.075+0000 7fd398ff9700 1 -- 172.21.15.178:0/2489314692 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7fd38c0227e0 con 0x7fd39c100340 2024-09-10T11:21:16.077 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.075+0000 7fd3a1742700 1 -- 172.21.15.178:0/2489314692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd39c104b70 con 0x7fd39c100340 2024-09-10T11:21:16.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.075+0000 7fd398ff9700 1 --2- 172.21.15.178:0/2489314692 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd3880384e0 0x7fd38803a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:16.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.075+0000 7fd398ff9700 1 -- 172.21.15.178:0/2489314692 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fd38c04c410 con 0x7fd39c100340 2024-09-10T11:21:16.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.076+0000 7fd3927fc700 1 --2- 172.21.15.178:0/2489314692 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd3880384e0 0x7fd38803a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:16.078 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.076+0000 7fd3927fc700 1 --2- 172.21.15.178:0/2489314692 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd3880384e0 0x7fd38803a990 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd384006fd0 tx=0x7fd384006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:16.081 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.080+0000 7fd398ff9700 1 -- 172.21.15.178:0/2489314692 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd38c02ba00 con 0x7fd39c100340 2024-09-10T11:21:16.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.280+0000 7fd3a1742700 1 -- 172.21.15.178:0/2489314692 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi144:172.21.15.144=smithi144;smithi178:172.21.15.178=smithi178", "target": ["mon-mgr", ""]}) v1 -- 0x7fd39c0fb390 con 0x7fd3880384e0 2024-09-10T11:21:16.287 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.286+0000 7fd398ff9700 1 -- 172.21.15.178:0/2489314692 <== mgr.14164 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd39c0fb390 con 0x7fd3880384e0 2024-09-10T11:21:16.287 INFO:teuthology.orchestra.run.smithi178.stdout:Scheduled mon update... 2024-09-10T11:21:16.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.288+0000 7fd3a1742700 1 -- 172.21.15.178:0/2489314692 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd3880384e0 msgr2=0x7fd38803a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:16.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.288+0000 7fd3a1742700 1 --2- 172.21.15.178:0/2489314692 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd3880384e0 0x7fd38803a990 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd384006fd0 tx=0x7fd384006e40 comp rx=0 tx=0).stop 2024-09-10T11:21:16.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.288+0000 7fd3a1742700 1 -- 172.21.15.178:0/2489314692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c100340 msgr2=0x7fd39c194fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:16.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.288+0000 7fd3a1742700 1 --2- 172.21.15.178:0/2489314692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c100340 0x7fd39c194fd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd38c005b40 tx=0x7fd38c004740 comp rx=0 tx=0).stop 2024-09-10T11:21:16.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.289+0000 7fd3a1742700 1 -- 172.21.15.178:0/2489314692 shutdown_connections 2024-09-10T11:21:16.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.289+0000 7fd3a1742700 1 --2- 172.21.15.178:0/2489314692 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd3880384e0 0x7fd38803a990 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:16.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.289+0000 7fd3a1742700 1 --2- 172.21.15.178:0/2489314692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd39c100340 0x7fd39c194fd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:16.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.289+0000 7fd3a1742700 1 -- 172.21.15.178:0/2489314692 >> 172.21.15.178:0/2489314692 conn(0x7fd39c0f9a60 msgr2=0x7fd39c0fa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:16.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.289+0000 7fd3a1742700 1 -- 172.21.15.178:0/2489314692 shutdown_connections 2024-09-10T11:21:16.290 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:16.289+0000 7fd3a1742700 1 -- 172.21.15.178:0/2489314692 wait complete. 2024-09-10T11:21:16.895 DEBUG:teuthology.orchestra.run.smithi178:mon.smithi178> sudo journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi178.service 2024-09-10T11:21:16.897 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:21:16.898 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:21:17.117 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:17.160 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:17.663 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:17 smithi144 ceph-mon[27942]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi144:172.21.15.144=smithi144;smithi178:172.21.15.178=smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:21:17.663 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:17 smithi144 ceph-mon[27942]: Saving service mon spec with placement smithi144:172.21.15.144=smithi144;smithi178:172.21.15.178=smithi178;count:2 2024-09-10T11:21:17.664 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:17 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:17.664 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:17 smithi144 ceph-mon[27942]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:18.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.557+0000 7f4ca5b9f700 1 -- 172.21.15.178:0/1301538522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca00ff2f0 msgr2=0x7f4ca00ff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:18.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.557+0000 7f4ca5b9f700 1 --2- 172.21.15.178:0/1301538522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca00ff2f0 0x7f4ca00ff700 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4c90009910 tx=0x7f4c90009c20 comp rx=0 tx=0).stop 2024-09-10T11:21:18.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.557+0000 7f4ca5b9f700 1 -- 172.21.15.178:0/1301538522 shutdown_connections 2024-09-10T11:21:18.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.557+0000 7f4ca5b9f700 1 --2- 172.21.15.178:0/1301538522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca00ff2f0 0x7f4ca00ff700 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:18.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.557+0000 7f4ca5b9f700 1 -- 172.21.15.178:0/1301538522 >> 172.21.15.178:0/1301538522 conn(0x7f4ca00f9af0 msgr2=0x7f4ca00fbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:18.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.557+0000 7f4ca5b9f700 1 -- 172.21.15.178:0/1301538522 shutdown_connections 2024-09-10T11:21:18.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.557+0000 7f4ca5b9f700 1 -- 172.21.15.178:0/1301538522 wait complete. 2024-09-10T11:21:18.559 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.557+0000 7f4ca5b9f700 1 Processor -- start 2024-09-10T11:21:18.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.558+0000 7f4ca5b9f700 1 -- start start 2024-09-10T11:21:18.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.558+0000 7f4ca5b9f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca00ff2f0 0x7f4ca0194f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:18.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.558+0000 7f4ca5b9f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ca01954a0 con 0x7f4ca00ff2f0 2024-09-10T11:21:18.560 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.558+0000 7f4c9f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca00ff2f0 0x7f4ca0194f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:18.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.558+0000 7f4c9f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca00ff2f0 0x7f4ca0194f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:52014/0 (socket says 172.21.15.178:52014) 2024-09-10T11:21:18.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.558+0000 7f4c9f7fe700 1 -- 172.21.15.178:0/3718207768 learned_addr learned my addr 172.21.15.178:0/3718207768 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:21:18.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.558+0000 7f4c9f7fe700 1 -- 172.21.15.178:0/3718207768 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900095f0 con 0x7f4ca00ff2f0 2024-09-10T11:21:18.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.558+0000 7f4c9f7fe700 1 --2- 172.21.15.178:0/3718207768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca00ff2f0 0x7f4ca0194f60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4c90005b40 tx=0x7f4c90004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:18.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.559+0000 7f4c9cff9700 1 -- 172.21.15.178:0/3718207768 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c9001d070 con 0x7f4ca00ff2f0 2024-09-10T11:21:18.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.559+0000 7f4c9cff9700 1 -- 172.21.15.178:0/3718207768 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4c9000be10 con 0x7f4ca00ff2f0 2024-09-10T11:21:18.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.559+0000 7f4ca5b9f700 1 -- 172.21.15.178:0/3718207768 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ca01956a0 con 0x7f4ca00ff2f0 2024-09-10T11:21:18.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.559+0000 7f4c9cff9700 1 -- 172.21.15.178:0/3718207768 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c9000f460 con 0x7f4ca00ff2f0 2024-09-10T11:21:18.561 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.559+0000 7f4ca5b9f700 1 -- 172.21.15.178:0/3718207768 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ca0195b40 con 0x7f4ca00ff2f0 2024-09-10T11:21:18.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.559+0000 7f4c9cff9700 1 -- 172.21.15.178:0/3718207768 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f4c9000b960 con 0x7f4ca00ff2f0 2024-09-10T11:21:18.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.560+0000 7f4ca5b9f700 1 -- 172.21.15.178:0/3718207768 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ca018e990 con 0x7f4ca00ff2f0 2024-09-10T11:21:18.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.560+0000 7f4c9cff9700 1 --2- 172.21.15.178:0/3718207768 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4c88038490 0x7f4c8803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:18.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.560+0000 7f4c9cff9700 1 -- 172.21.15.178:0/3718207768 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f4c9004c4b0 con 0x7f4ca00ff2f0 2024-09-10T11:21:18.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.560+0000 7f4c9effd700 1 --2- 172.21.15.178:0/3718207768 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4c88038490 0x7f4c8803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:18.562 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.560+0000 7f4c9effd700 1 --2- 172.21.15.178:0/3718207768 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4c88038490 0x7f4c8803a940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4c94006fd0 tx=0x7f4c94006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:18.566 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.565+0000 7f4c9cff9700 1 -- 172.21.15.178:0/3718207768 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4c9002a360 con 0x7f4ca00ff2f0 2024-09-10T11:21:18.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.832+0000 7f4ca5b9f700 1 -- 172.21.15.178:0/3718207768 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4ca002cc30 con 0x7f4ca00ff2f0 2024-09-10T11:21:18.833 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.832+0000 7f4c9cff9700 1 -- 172.21.15.178:0/3718207768 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f4c90027070 con 0x7f4ca00ff2f0 2024-09-10T11:21:18.834 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:21:18.834 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:21:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.834+0000 7f4ca5b9f700 1 -- 172.21.15.178:0/3718207768 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4c88038490 msgr2=0x7f4c8803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.834+0000 7f4ca5b9f700 1 --2- 172.21.15.178:0/3718207768 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4c88038490 0x7f4c8803a940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f4c94006fd0 tx=0x7f4c94006e40 comp rx=0 tx=0).stop 2024-09-10T11:21:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.834+0000 7f4ca5b9f700 1 -- 172.21.15.178:0/3718207768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca00ff2f0 msgr2=0x7f4ca0194f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.834+0000 7f4ca5b9f700 1 --2- 172.21.15.178:0/3718207768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca00ff2f0 0x7f4ca0194f60 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4c90005b40 tx=0x7f4c90004740 comp rx=0 tx=0).stop 2024-09-10T11:21:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.834+0000 7f4ca5b9f700 1 -- 172.21.15.178:0/3718207768 shutdown_connections 2024-09-10T11:21:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.834+0000 7f4ca5b9f700 1 --2- 172.21.15.178:0/3718207768 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4c88038490 0x7f4c8803a940 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.834+0000 7f4ca5b9f700 1 --2- 172.21.15.178:0/3718207768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ca00ff2f0 0x7f4ca0194f60 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.834+0000 7f4ca5b9f700 1 -- 172.21.15.178:0/3718207768 >> 172.21.15.178:0/3718207768 conn(0x7f4ca00f9af0 msgr2=0x7f4ca00fe810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.835+0000 7f4ca5b9f700 1 -- 172.21.15.178:0/3718207768 shutdown_connections 2024-09-10T11:21:18.836 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:18.835+0000 7f4ca5b9f700 1 -- 172.21.15.178:0/3718207768 wait complete. 2024-09-10T11:21:18.837 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:21:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:19 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:19 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:19 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:19 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:19 smithi144 ceph-mon[27942]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-10T11:21:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:19 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:19 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:19 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T11:21:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:19 smithi144 ceph-mon[27942]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T11:21:19.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:19 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:19.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:19 smithi144 ceph-mon[27942]: Deploying daemon grafana.smithi144 on smithi144 2024-09-10T11:21:19.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:19 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/3718207768' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:21:19.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:19 smithi144 ceph-mon[27942]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:20.392 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:21:20.392 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:21:20.589 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:20.639 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:22.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.034+0000 7f70e9841700 1 -- 172.21.15.178:0/3759779291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e40fed30 msgr2=0x7f70e40ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:22.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.034+0000 7f70e9841700 1 --2- 172.21.15.178:0/3759779291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e40fed30 0x7f70e40ff140 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f70d8009910 tx=0x7f70d8009c20 comp rx=0 tx=0).stop 2024-09-10T11:21:22.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.034+0000 7f70e9841700 1 -- 172.21.15.178:0/3759779291 shutdown_connections 2024-09-10T11:21:22.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.034+0000 7f70e9841700 1 --2- 172.21.15.178:0/3759779291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e40fed30 0x7f70e40ff140 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:22.036 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.034+0000 7f70e9841700 1 -- 172.21.15.178:0/3759779291 >> 172.21.15.178:0/3759779291 conn(0x7f70e40fa2e0 msgr2=0x7f70e40fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:22.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.034+0000 7f70e9841700 1 -- 172.21.15.178:0/3759779291 shutdown_connections 2024-09-10T11:21:22.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.034+0000 7f70e9841700 1 -- 172.21.15.178:0/3759779291 wait complete. 2024-09-10T11:21:22.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.034+0000 7f70e9841700 1 Processor -- start 2024-09-10T11:21:22.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.035+0000 7f70e9841700 1 -- start start 2024-09-10T11:21:22.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.035+0000 7f70e9841700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e40fed30 0x7f70e4068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:22.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.035+0000 7f70e9841700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e4197d00 con 0x7f70e40fed30 2024-09-10T11:21:22.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.035+0000 7f70e2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e40fed30 0x7f70e4068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:22.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.035+0000 7f70e2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e40fed30 0x7f70e4068ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:52042/0 (socket says 172.21.15.178:52042) 2024-09-10T11:21:22.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.035+0000 7f70e2ffd700 1 -- 172.21.15.178:0/529591044 learned_addr learned my addr 172.21.15.178:0/529591044 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:21:22.037 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.035+0000 7f70e2ffd700 1 -- 172.21.15.178:0/529591044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d80095f0 con 0x7f70e40fed30 2024-09-10T11:21:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.035+0000 7f70e2ffd700 1 --2- 172.21.15.178:0/529591044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e40fed30 0x7f70e4068ae0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f70d8005b40 tx=0x7f70d8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.035+0000 7f70e883f700 1 -- 172.21.15.178:0/529591044 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70d801d070 con 0x7f70e40fed30 2024-09-10T11:21:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.035+0000 7f70e883f700 1 -- 172.21.15.178:0/529591044 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f70d800be10 con 0x7f70e40fed30 2024-09-10T11:21:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.035+0000 7f70e9841700 1 -- 172.21.15.178:0/529591044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e4197e40 con 0x7f70e40fed30 2024-09-10T11:21:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.036+0000 7f70e883f700 1 -- 172.21.15.178:0/529591044 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f70d800f460 con 0x7f70e40fed30 2024-09-10T11:21:22.039 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.036+0000 7f70e9841700 1 -- 172.21.15.178:0/529591044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e41982e0 con 0x7f70e40fed30 2024-09-10T11:21:22.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.036+0000 7f70e883f700 1 -- 172.21.15.178:0/529591044 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f70d80227e0 con 0x7f70e40fed30 2024-09-10T11:21:22.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.036+0000 7f70e9841700 1 -- 172.21.15.178:0/529591044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e418da10 con 0x7f70e40fed30 2024-09-10T11:21:22.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.037+0000 7f70e883f700 1 --2- 172.21.15.178:0/529591044 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f70cc038490 0x7f70cc03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:22.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.037+0000 7f70e883f700 1 -- 172.21.15.178:0/529591044 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f70d804c440 con 0x7f70e40fed30 2024-09-10T11:21:22.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.037+0000 7f70e27fc700 1 --2- 172.21.15.178:0/529591044 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f70cc038490 0x7f70cc03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:22.040 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.037+0000 7f70e27fc700 1 --2- 172.21.15.178:0/529591044 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f70cc038490 0x7f70cc03a940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f70d4006fd0 tx=0x7f70d4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:22.047 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.046+0000 7f70e883f700 1 -- 172.21.15.178:0/529591044 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f70d802bba0 con 0x7f70e40fed30 2024-09-10T11:21:22.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.320+0000 7f70e9841700 1 -- 172.21.15.178:0/529591044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f70e402cc30 con 0x7f70e40fed30 2024-09-10T11:21:22.322 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.320+0000 7f70e883f700 1 -- 172.21.15.178:0/529591044 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f70d8027030 con 0x7f70e40fed30 2024-09-10T11:21:22.322 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:21:22.322 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:21:22.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.322+0000 7f70e9841700 1 -- 172.21.15.178:0/529591044 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f70cc038490 msgr2=0x7f70cc03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:22.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.322+0000 7f70e9841700 1 --2- 172.21.15.178:0/529591044 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f70cc038490 0x7f70cc03a940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f70d4006fd0 tx=0x7f70d4006e40 comp rx=0 tx=0).stop 2024-09-10T11:21:22.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.322+0000 7f70e9841700 1 -- 172.21.15.178:0/529591044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e40fed30 msgr2=0x7f70e4068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:22.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.322+0000 7f70e9841700 1 --2- 172.21.15.178:0/529591044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e40fed30 0x7f70e4068ae0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f70d8005b40 tx=0x7f70d8004740 comp rx=0 tx=0).stop 2024-09-10T11:21:22.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.322+0000 7f70e9841700 1 -- 172.21.15.178:0/529591044 shutdown_connections 2024-09-10T11:21:22.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.322+0000 7f70e9841700 1 --2- 172.21.15.178:0/529591044 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f70cc038490 0x7f70cc03a940 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:22.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.322+0000 7f70e9841700 1 --2- 172.21.15.178:0/529591044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e40fed30 0x7f70e4068ae0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:22.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.322+0000 7f70e9841700 1 -- 172.21.15.178:0/529591044 >> 172.21.15.178:0/529591044 conn(0x7f70e40fa2e0 msgr2=0x7f70e40faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:22.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.323+0000 7f70e9841700 1 -- 172.21.15.178:0/529591044 shutdown_connections 2024-09-10T11:21:22.324 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:22.323+0000 7f70e9841700 1 -- 172.21.15.178:0/529591044 wait complete. 2024-09-10T11:21:22.325 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:21:22.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:22 smithi144 ceph-mon[27942]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:23.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:23 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/529591044' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:21:23.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:23 smithi144 ceph-mon[27942]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:23.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:23 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:23.929 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:21:23.930 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:21:24.114 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:24.158 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.617+0000 7f9ae04ad700 1 -- 172.21.15.178:0/1939970365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ad80ffe40 msgr2=0x7f9ad8100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.617+0000 7f9ae04ad700 1 --2- 172.21.15.178:0/1939970365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ad80ffe40 0x7f9ad8100250 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8009910 tx=0x7f9ac8009c20 comp rx=0 tx=0).stop 2024-09-10T11:21:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.617+0000 7f9ae04ad700 1 -- 172.21.15.178:0/1939970365 shutdown_connections 2024-09-10T11:21:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.617+0000 7f9ae04ad700 1 --2- 172.21.15.178:0/1939970365 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ad80ffe40 0x7f9ad8100250 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.617+0000 7f9ae04ad700 1 -- 172.21.15.178:0/1939970365 >> 172.21.15.178:0/1939970365 conn(0x7f9ad80fb3f0 msgr2=0x7f9ad80fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.617+0000 7f9ae04ad700 1 -- 172.21.15.178:0/1939970365 shutdown_connections 2024-09-10T11:21:25.619 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.618+0000 7f9ae04ad700 1 -- 172.21.15.178:0/1939970365 wait complete. 2024-09-10T11:21:25.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.618+0000 7f9ae04ad700 1 Processor -- start 2024-09-10T11:21:25.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.618+0000 7f9ae04ad700 1 -- start start 2024-09-10T11:21:25.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.618+0000 7f9ae04ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ad80ffe40 0x7f9ad8194fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:25.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.618+0000 7f9ae04ad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ad81954e0 con 0x7f9ad80ffe40 2024-09-10T11:21:25.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.619+0000 7f9ade249700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ad80ffe40 0x7f9ad8194fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:25.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.619+0000 7f9ade249700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ad80ffe40 0x7f9ad8194fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:53260/0 (socket says 172.21.15.178:53260) 2024-09-10T11:21:25.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.619+0000 7f9ade249700 1 -- 172.21.15.178:0/2493248445 learned_addr learned my addr 172.21.15.178:0/2493248445 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:21:25.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.619+0000 7f9ade249700 1 -- 172.21.15.178:0/2493248445 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac80095f0 con 0x7f9ad80ffe40 2024-09-10T11:21:25.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.619+0000 7f9ade249700 1 --2- 172.21.15.178:0/2493248445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ad80ffe40 0x7f9ad8194fa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005b40 tx=0x7f9ac8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:25.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.619+0000 7f9acf7fe700 1 -- 172.21.15.178:0/2493248445 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ac801d070 con 0x7f9ad80ffe40 2024-09-10T11:21:25.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.619+0000 7f9acf7fe700 1 -- 172.21.15.178:0/2493248445 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9ac800be10 con 0x7f9ad80ffe40 2024-09-10T11:21:25.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.619+0000 7f9acf7fe700 1 -- 172.21.15.178:0/2493248445 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9ac800f460 con 0x7f9ad80ffe40 2024-09-10T11:21:25.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.619+0000 7f9ae04ad700 1 -- 172.21.15.178:0/2493248445 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ad81956e0 con 0x7f9ad80ffe40 2024-09-10T11:21:25.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.619+0000 7f9ae04ad700 1 -- 172.21.15.178:0/2493248445 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ad8195b80 con 0x7f9ad80ffe40 2024-09-10T11:21:25.622 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.620+0000 7f9acf7fe700 1 -- 172.21.15.178:0/2493248445 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f9ac800b960 con 0x7f9ad80ffe40 2024-09-10T11:21:25.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.620+0000 7f9ae04ad700 1 -- 172.21.15.178:0/2493248445 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ad818eaa0 con 0x7f9ad80ffe40 2024-09-10T11:21:25.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.620+0000 7f9acf7fe700 1 --2- 172.21.15.178:0/2493248445 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9ac40384e0 0x7f9ac403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:25.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.620+0000 7f9acf7fe700 1 -- 172.21.15.178:0/2493248445 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f9ac804c280 con 0x7f9ad80ffe40 2024-09-10T11:21:25.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.621+0000 7f9adda48700 1 --2- 172.21.15.178:0/2493248445 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9ac40384e0 0x7f9ac403a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:25.623 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.621+0000 7f9adda48700 1 --2- 172.21.15.178:0/2493248445 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9ac40384e0 0x7f9ac403a990 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4006fd0 tx=0x7f9ad4006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:25.626 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.625+0000 7f9acf7fe700 1 -- 172.21.15.178:0/2493248445 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9ac802ba00 con 0x7f9ad80ffe40 2024-09-10T11:21:25.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.890+0000 7f9ae04ad700 1 -- 172.21.15.178:0/2493248445 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f9ad802cc30 con 0x7f9ad80ffe40 2024-09-10T11:21:25.893 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.891+0000 7f9acf7fe700 1 -- 172.21.15.178:0/2493248445 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f9ac8027070 con 0x7f9ad80ffe40 2024-09-10T11:21:25.893 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:21:25.893 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:21:25.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.893+0000 7f9ae04ad700 1 -- 172.21.15.178:0/2493248445 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9ac40384e0 msgr2=0x7f9ac403a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:25.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.893+0000 7f9ae04ad700 1 --2- 172.21.15.178:0/2493248445 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9ac40384e0 0x7f9ac403a990 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f9ad4006fd0 tx=0x7f9ad4006e40 comp rx=0 tx=0).stop 2024-09-10T11:21:25.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.893+0000 7f9ae04ad700 1 -- 172.21.15.178:0/2493248445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ad80ffe40 msgr2=0x7f9ad8194fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:25.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.893+0000 7f9ae04ad700 1 --2- 172.21.15.178:0/2493248445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ad80ffe40 0x7f9ad8194fa0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f9ac8005b40 tx=0x7f9ac8004740 comp rx=0 tx=0).stop 2024-09-10T11:21:25.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.893+0000 7f9ae04ad700 1 -- 172.21.15.178:0/2493248445 shutdown_connections 2024-09-10T11:21:25.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.893+0000 7f9ae04ad700 1 --2- 172.21.15.178:0/2493248445 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9ac40384e0 0x7f9ac403a990 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:25.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.893+0000 7f9ae04ad700 1 --2- 172.21.15.178:0/2493248445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ad80ffe40 0x7f9ad8194fa0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:25.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.893+0000 7f9ae04ad700 1 -- 172.21.15.178:0/2493248445 >> 172.21.15.178:0/2493248445 conn(0x7f9ad80fb3f0 msgr2=0x7f9ad80fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:25.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.894+0000 7f9ae04ad700 1 -- 172.21.15.178:0/2493248445 shutdown_connections 2024-09-10T11:21:25.895 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:25.894+0000 7f9ae04ad700 1 -- 172.21.15.178:0/2493248445 wait complete. 2024-09-10T11:21:25.896 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:21:26.129 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:26 smithi144 ceph-mon[27942]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:26.129 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:26 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/2493248445' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:21:27.426 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:21:27.427 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:21:27.625 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:27.668 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:28.128 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:28 smithi144 ceph-mon[27942]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:29.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.063+0000 7fa978506700 1 -- 172.21.15.178:0/1392374201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9700fed30 msgr2=0x7fa9700ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:29.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.063+0000 7fa978506700 1 --2- 172.21.15.178:0/1392374201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9700fed30 0x7fa9700ff140 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa964009910 tx=0x7fa964009c20 comp rx=0 tx=0).stop 2024-09-10T11:21:29.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.063+0000 7fa978506700 1 -- 172.21.15.178:0/1392374201 shutdown_connections 2024-09-10T11:21:29.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.063+0000 7fa978506700 1 --2- 172.21.15.178:0/1392374201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9700fed30 0x7fa9700ff140 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:29.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.063+0000 7fa978506700 1 -- 172.21.15.178:0/1392374201 >> 172.21.15.178:0/1392374201 conn(0x7fa9700fa2e0 msgr2=0x7fa9700fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:29.066 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.063+0000 7fa978506700 1 -- 172.21.15.178:0/1392374201 shutdown_connections 2024-09-10T11:21:29.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.063+0000 7fa978506700 1 -- 172.21.15.178:0/1392374201 wait complete. 2024-09-10T11:21:29.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.064+0000 7fa978506700 1 Processor -- start 2024-09-10T11:21:29.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.064+0000 7fa978506700 1 -- start start 2024-09-10T11:21:29.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.064+0000 7fa978506700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9700fed30 0x7fa970194fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:29.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.064+0000 7fa978506700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa970195500 con 0x7fa9700fed30 2024-09-10T11:21:29.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.064+0000 7fa9762a2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9700fed30 0x7fa970194fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:29.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.064+0000 7fa9762a2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9700fed30 0x7fa970194fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:53286/0 (socket says 172.21.15.178:53286) 2024-09-10T11:21:29.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.064+0000 7fa9762a2700 1 -- 172.21.15.178:0/2639311148 learned_addr learned my addr 172.21.15.178:0/2639311148 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:21:29.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.064+0000 7fa9762a2700 1 -- 172.21.15.178:0/2639311148 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9640095f0 con 0x7fa9700fed30 2024-09-10T11:21:29.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.064+0000 7fa9762a2700 1 --2- 172.21.15.178:0/2639311148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9700fed30 0x7fa970194fc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa964005950 tx=0x7fa964009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:29.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.065+0000 7fa9637fe700 1 -- 172.21.15.178:0/2639311148 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa96401d070 con 0x7fa9700fed30 2024-09-10T11:21:29.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.065+0000 7fa9637fe700 1 -- 172.21.15.178:0/2639311148 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa96400be10 con 0x7fa9700fed30 2024-09-10T11:21:29.067 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.065+0000 7fa978506700 1 -- 172.21.15.178:0/2639311148 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa970195700 con 0x7fa9700fed30 2024-09-10T11:21:29.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.065+0000 7fa9637fe700 1 -- 172.21.15.178:0/2639311148 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa96400f460 con 0x7fa9700fed30 2024-09-10T11:21:29.068 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.065+0000 7fa978506700 1 -- 172.21.15.178:0/2639311148 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa970195ba0 con 0x7fa9700fed30 2024-09-10T11:21:29.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.066+0000 7fa9637fe700 1 -- 172.21.15.178:0/2639311148 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7fa96400b960 con 0x7fa9700fed30 2024-09-10T11:21:29.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.066+0000 7fa978506700 1 -- 172.21.15.178:0/2639311148 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa97018ea20 con 0x7fa9700fed30 2024-09-10T11:21:29.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.066+0000 7fa9637fe700 1 --2- 172.21.15.178:0/2639311148 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa95c038490 0x7fa95c03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:29.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.066+0000 7fa9637fe700 1 -- 172.21.15.178:0/2639311148 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fa96404c490 con 0x7fa9700fed30 2024-09-10T11:21:29.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.066+0000 7fa975aa1700 1 --2- 172.21.15.178:0/2639311148 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa95c038490 0x7fa95c03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:29.069 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.066+0000 7fa975aa1700 1 --2- 172.21.15.178:0/2639311148 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa95c038490 0x7fa95c03a940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa96c006fd0 tx=0x7fa96c006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:29.073 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.072+0000 7fa9637fe700 1 -- 172.21.15.178:0/2639311148 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa96402bba0 con 0x7fa9700fed30 2024-09-10T11:21:29.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.334+0000 7fa978506700 1 -- 172.21.15.178:0/2639311148 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa97002cc30 con 0x7fa9700fed30 2024-09-10T11:21:29.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.334+0000 7fa9637fe700 1 -- 172.21.15.178:0/2639311148 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fa964027030 con 0x7fa9700fed30 2024-09-10T11:21:29.336 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:21:29.336 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:21:29.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.336+0000 7fa978506700 1 -- 172.21.15.178:0/2639311148 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa95c038490 msgr2=0x7fa95c03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:29.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.336+0000 7fa978506700 1 --2- 172.21.15.178:0/2639311148 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa95c038490 0x7fa95c03a940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa96c006fd0 tx=0x7fa96c006e40 comp rx=0 tx=0).stop 2024-09-10T11:21:29.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.336+0000 7fa978506700 1 -- 172.21.15.178:0/2639311148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9700fed30 msgr2=0x7fa970194fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:29.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.336+0000 7fa978506700 1 --2- 172.21.15.178:0/2639311148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9700fed30 0x7fa970194fc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa964005950 tx=0x7fa964009e60 comp rx=0 tx=0).stop 2024-09-10T11:21:29.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.336+0000 7fa978506700 1 -- 172.21.15.178:0/2639311148 shutdown_connections 2024-09-10T11:21:29.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.336+0000 7fa978506700 1 --2- 172.21.15.178:0/2639311148 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa95c038490 0x7fa95c03a940 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:29.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.336+0000 7fa978506700 1 --2- 172.21.15.178:0/2639311148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9700fed30 0x7fa970194fc0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:29.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.336+0000 7fa978506700 1 -- 172.21.15.178:0/2639311148 >> 172.21.15.178:0/2639311148 conn(0x7fa9700fa2e0 msgr2=0x7fa9700690c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:29.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.336+0000 7fa978506700 1 -- 172.21.15.178:0/2639311148 shutdown_connections 2024-09-10T11:21:29.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:29.336+0000 7fa978506700 1 -- 172.21.15.178:0/2639311148 wait complete. 2024-09-10T11:21:29.339 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:21:30.161 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:29 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:30.161 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:29 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:30.161 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:29 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:30.161 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:29 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:30.161 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:29 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:30.162 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:29 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:30.162 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:29 smithi144 ceph-mon[27942]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:30.162 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:29 smithi144 ceph-mon[27942]: Deploying daemon prometheus.smithi144 on smithi144 2024-09-10T11:21:30.162 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:29 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/2639311148' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:21:30.905 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:21:30.906 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:21:31.102 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:31.144 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:32.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:32 smithi144 ceph-mon[27942]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:32.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.608+0000 7fa15ba25700 1 -- 172.21.15.178:0/293511581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa154101480 msgr2=0x7fa154101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:32.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.608+0000 7fa15ba25700 1 --2- 172.21.15.178:0/293511581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa154101480 0x7fa154101890 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa150009910 tx=0x7fa150009c20 comp rx=0 tx=0).stop 2024-09-10T11:21:32.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.609+0000 7fa15ba25700 1 -- 172.21.15.178:0/293511581 shutdown_connections 2024-09-10T11:21:32.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.609+0000 7fa15ba25700 1 --2- 172.21.15.178:0/293511581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa154101480 0x7fa154101890 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:32.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.609+0000 7fa15ba25700 1 -- 172.21.15.178:0/293511581 >> 172.21.15.178:0/293511581 conn(0x7fa1540f9af0 msgr2=0x7fa1540fbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:32.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.609+0000 7fa15ba25700 1 -- 172.21.15.178:0/293511581 shutdown_connections 2024-09-10T11:21:32.610 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.609+0000 7fa15ba25700 1 -- 172.21.15.178:0/293511581 wait complete. 2024-09-10T11:21:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.609+0000 7fa15ba25700 1 Processor -- start 2024-09-10T11:21:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.609+0000 7fa15ba25700 1 -- start start 2024-09-10T11:21:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.609+0000 7fa15ba25700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa154101480 0x7fa154194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.609+0000 7fa15ba25700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1541954f0 con 0x7fa154101480 2024-09-10T11:21:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.610+0000 7fa1597c1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa154101480 0x7fa154194fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.610+0000 7fa1597c1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa154101480 0x7fa154194fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:53298/0 (socket says 172.21.15.178:53298) 2024-09-10T11:21:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.610+0000 7fa1597c1700 1 -- 172.21.15.178:0/372428161 learned_addr learned my addr 172.21.15.178:0/372428161 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:21:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.610+0000 7fa1597c1700 1 -- 172.21.15.178:0/372428161 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1500095f0 con 0x7fa154101480 2024-09-10T11:21:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.610+0000 7fa1597c1700 1 --2- 172.21.15.178:0/372428161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa154101480 0x7fa154194fb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa150005b40 tx=0x7fa150004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.610+0000 7fa14a7fc700 1 -- 172.21.15.178:0/372428161 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa15001d070 con 0x7fa154101480 2024-09-10T11:21:32.612 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.610+0000 7fa15ba25700 1 -- 172.21.15.178:0/372428161 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1541956f0 con 0x7fa154101480 2024-09-10T11:21:32.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.610+0000 7fa14a7fc700 1 -- 172.21.15.178:0/372428161 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa15000be10 con 0x7fa154101480 2024-09-10T11:21:32.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.610+0000 7fa14a7fc700 1 -- 172.21.15.178:0/372428161 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa15000f460 con 0x7fa154101480 2024-09-10T11:21:32.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.610+0000 7fa15ba25700 1 -- 172.21.15.178:0/372428161 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa154195b90 con 0x7fa154101480 2024-09-10T11:21:32.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.611+0000 7fa14a7fc700 1 -- 172.21.15.178:0/372428161 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7fa1500227e0 con 0x7fa154101480 2024-09-10T11:21:32.613 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.611+0000 7fa15ba25700 1 -- 172.21.15.178:0/372428161 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15418ea80 con 0x7fa154101480 2024-09-10T11:21:32.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.611+0000 7fa14a7fc700 1 --2- 172.21.15.178:0/372428161 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa140038490 0x7fa14003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:32.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.611+0000 7fa14a7fc700 1 -- 172.21.15.178:0/372428161 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7fa15004c450 con 0x7fa154101480 2024-09-10T11:21:32.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.612+0000 7fa158fc0700 1 --2- 172.21.15.178:0/372428161 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa140038490 0x7fa14003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:32.614 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.612+0000 7fa158fc0700 1 --2- 172.21.15.178:0/372428161 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa140038490 0x7fa14003a940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa144006fd0 tx=0x7fa144006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:32.621 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.620+0000 7fa14a7fc700 1 -- 172.21.15.178:0/372428161 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa15002bba0 con 0x7fa154101480 2024-09-10T11:21:32.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.887+0000 7fa15ba25700 1 -- 172.21.15.178:0/372428161 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fa15402cc30 con 0x7fa154101480 2024-09-10T11:21:32.888 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.887+0000 7fa14a7fc700 1 -- 172.21.15.178:0/372428161 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fa150027070 con 0x7fa154101480 2024-09-10T11:21:32.889 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:21:32.889 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:21:32.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.889+0000 7fa15ba25700 1 -- 172.21.15.178:0/372428161 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa140038490 msgr2=0x7fa14003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:32.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.889+0000 7fa15ba25700 1 --2- 172.21.15.178:0/372428161 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa140038490 0x7fa14003a940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa144006fd0 tx=0x7fa144006e40 comp rx=0 tx=0).stop 2024-09-10T11:21:32.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.889+0000 7fa15ba25700 1 -- 172.21.15.178:0/372428161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa154101480 msgr2=0x7fa154194fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:32.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.889+0000 7fa15ba25700 1 --2- 172.21.15.178:0/372428161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa154101480 0x7fa154194fb0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa150005b40 tx=0x7fa150004740 comp rx=0 tx=0).stop 2024-09-10T11:21:32.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.889+0000 7fa15ba25700 1 -- 172.21.15.178:0/372428161 shutdown_connections 2024-09-10T11:21:32.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.889+0000 7fa15ba25700 1 --2- 172.21.15.178:0/372428161 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa140038490 0x7fa14003a940 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:32.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.889+0000 7fa15ba25700 1 --2- 172.21.15.178:0/372428161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa154101480 0x7fa154194fb0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:32.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.889+0000 7fa15ba25700 1 -- 172.21.15.178:0/372428161 >> 172.21.15.178:0/372428161 conn(0x7fa1540f9af0 msgr2=0x7fa1540fa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:32.891 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.890+0000 7fa15ba25700 1 -- 172.21.15.178:0/372428161 shutdown_connections 2024-09-10T11:21:32.892 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:32.890+0000 7fa15ba25700 1 -- 172.21.15.178:0/372428161 wait complete. 2024-09-10T11:21:32.892 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:21:33.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:33 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/372428161' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:21:33.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:33 smithi144 ceph-mon[27942]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:33.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:33 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:34.503 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:21:34.504 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:21:34.698 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:34.741 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:36.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.148+0000 7f0191e7b700 1 -- 172.21.15.178:0/1492168077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c100920 msgr2=0x7f018c102d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:36.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.148+0000 7f0191e7b700 1 --2- 172.21.15.178:0/1492168077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c100920 0x7f018c102d00 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f017c009910 tx=0x7f017c009c20 comp rx=0 tx=0).stop 2024-09-10T11:21:36.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.148+0000 7f0191e7b700 1 -- 172.21.15.178:0/1492168077 shutdown_connections 2024-09-10T11:21:36.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.148+0000 7f0191e7b700 1 --2- 172.21.15.178:0/1492168077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c100920 0x7f018c102d00 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:36.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.148+0000 7f0191e7b700 1 -- 172.21.15.178:0/1492168077 >> 172.21.15.178:0/1492168077 conn(0x7f018c0fa2e0 msgr2=0x7f018c0fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:36.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.148+0000 7f0191e7b700 1 -- 172.21.15.178:0/1492168077 shutdown_connections 2024-09-10T11:21:36.150 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.148+0000 7f0191e7b700 1 -- 172.21.15.178:0/1492168077 wait complete. 2024-09-10T11:21:36.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.149+0000 7f0191e7b700 1 Processor -- start 2024-09-10T11:21:36.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.149+0000 7f0191e7b700 1 -- start start 2024-09-10T11:21:36.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.149+0000 7f0191e7b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c100920 0x7f018c100280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:36.151 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.149+0000 7f0191e7b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c1007c0 con 0x7f018c100920 2024-09-10T11:21:36.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.149+0000 7f018b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c100920 0x7f018c100280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:36.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.149+0000 7f018b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c100920 0x7f018c100280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:35306/0 (socket says 172.21.15.178:35306) 2024-09-10T11:21:36.152 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.149+0000 7f018b7fe700 1 -- 172.21.15.178:0/2318246203 learned_addr learned my addr 172.21.15.178:0/2318246203 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:21:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.150+0000 7f018b7fe700 1 -- 172.21.15.178:0/2318246203 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f017c0095f0 con 0x7f018c100920 2024-09-10T11:21:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.150+0000 7f018b7fe700 1 --2- 172.21.15.178:0/2318246203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c100920 0x7f018c100280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f017c005b40 tx=0x7f017c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.150+0000 7f0188ff9700 1 -- 172.21.15.178:0/2318246203 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f017c01d070 con 0x7f018c100920 2024-09-10T11:21:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.150+0000 7f0191e7b700 1 -- 172.21.15.178:0/2318246203 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f018c0fe8d0 con 0x7f018c100920 2024-09-10T11:21:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.150+0000 7f0188ff9700 1 -- 172.21.15.178:0/2318246203 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f017c00be10 con 0x7f018c100920 2024-09-10T11:21:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.150+0000 7f0188ff9700 1 -- 172.21.15.178:0/2318246203 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f017c00f460 con 0x7f018c100920 2024-09-10T11:21:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.150+0000 7f0191e7b700 1 -- 172.21.15.178:0/2318246203 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f018c0fed70 con 0x7f018c100920 2024-09-10T11:21:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.151+0000 7f0191e7b700 1 -- 172.21.15.178:0/2318246203 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f018c062380 con 0x7f018c100920 2024-09-10T11:21:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.151+0000 7f0188ff9700 1 -- 172.21.15.178:0/2318246203 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f017c0227e0 con 0x7f018c100920 2024-09-10T11:21:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.151+0000 7f0188ff9700 1 --2- 172.21.15.178:0/2318246203 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0178038490 0x7f017803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.151+0000 7f0188ff9700 1 -- 172.21.15.178:0/2318246203 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f017c04c490 con 0x7f018c100920 2024-09-10T11:21:36.153 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.151+0000 7f018affd700 1 --2- 172.21.15.178:0/2318246203 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0178038490 0x7f017803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:36.154 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.152+0000 7f018affd700 1 --2- 172.21.15.178:0/2318246203 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0178038490 0x7f017803a940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0174006fd0 tx=0x7f0174006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:36.157 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.156+0000 7f0188ff9700 1 -- 172.21.15.178:0/2318246203 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f017c02bba0 con 0x7f018c100920 2024-09-10T11:21:36.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.421+0000 7f0191e7b700 1 -- 172.21.15.178:0/2318246203 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f018c18e4f0 con 0x7f018c100920 2024-09-10T11:21:36.423 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.421+0000 7f0188ff9700 1 -- 172.21.15.178:0/2318246203 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f017c027070 con 0x7f018c100920 2024-09-10T11:21:36.423 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:21:36.423 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:21:36.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.424+0000 7f0191e7b700 1 -- 172.21.15.178:0/2318246203 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0178038490 msgr2=0x7f017803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:36.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.424+0000 7f0191e7b700 1 --2- 172.21.15.178:0/2318246203 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0178038490 0x7f017803a940 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0174006fd0 tx=0x7f0174006e40 comp rx=0 tx=0).stop 2024-09-10T11:21:36.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.424+0000 7f0191e7b700 1 -- 172.21.15.178:0/2318246203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c100920 msgr2=0x7f018c100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:36.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.424+0000 7f0191e7b700 1 --2- 172.21.15.178:0/2318246203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c100920 0x7f018c100280 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f017c005b40 tx=0x7f017c004740 comp rx=0 tx=0).stop 2024-09-10T11:21:36.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.424+0000 7f0191e7b700 1 -- 172.21.15.178:0/2318246203 shutdown_connections 2024-09-10T11:21:36.425 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.424+0000 7f0191e7b700 1 --2- 172.21.15.178:0/2318246203 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0178038490 0x7f017803a940 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:36.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.424+0000 7f0191e7b700 1 --2- 172.21.15.178:0/2318246203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f018c100920 0x7f018c100280 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:36.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.424+0000 7f0191e7b700 1 -- 172.21.15.178:0/2318246203 >> 172.21.15.178:0/2318246203 conn(0x7f018c0fa2e0 msgr2=0x7f018c0faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:36.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.424+0000 7f0191e7b700 1 -- 172.21.15.178:0/2318246203 shutdown_connections 2024-09-10T11:21:36.426 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:36.424+0000 7f0191e7b700 1 -- 172.21.15.178:0/2318246203 wait complete. 2024-09-10T11:21:36.426 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:21:36.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:36 smithi144 ceph-mon[27942]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:37.141 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:37 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/2318246203' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:21:37.890 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:21:37.890 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:21:38.085 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:38.128 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:38.230 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:38 smithi144 ceph-mon[27942]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:39.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.504+0000 7f6855cf8700 1 -- 172.21.15.178:0/1405887960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68500fed30 msgr2=0x7f68500ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:39.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.504+0000 7f6855cf8700 1 --2- 172.21.15.178:0/1405887960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68500fed30 0x7f68500ff140 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f684800b0a0 tx=0x7f684800b3b0 comp rx=0 tx=0).stop 2024-09-10T11:21:39.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.504+0000 7f6855cf8700 1 -- 172.21.15.178:0/1405887960 shutdown_connections 2024-09-10T11:21:39.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.504+0000 7f6855cf8700 1 --2- 172.21.15.178:0/1405887960 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68500fed30 0x7f68500ff140 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:39.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.504+0000 7f6855cf8700 1 -- 172.21.15.178:0/1405887960 >> 172.21.15.178:0/1405887960 conn(0x7f68500fa2e0 msgr2=0x7f68500fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:39.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.504+0000 7f6855cf8700 1 -- 172.21.15.178:0/1405887960 shutdown_connections 2024-09-10T11:21:39.506 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.504+0000 7f6855cf8700 1 -- 172.21.15.178:0/1405887960 wait complete. 2024-09-10T11:21:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.505+0000 7f6855cf8700 1 Processor -- start 2024-09-10T11:21:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.505+0000 7f6855cf8700 1 -- start start 2024-09-10T11:21:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.505+0000 7f6855cf8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68500fed30 0x7f6850068ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:39.507 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.505+0000 7f6855cf8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6850197d00 con 0x7f68500fed30 2024-09-10T11:21:39.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.505+0000 7f684f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68500fed30 0x7f6850068ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:39.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.505+0000 7f684f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68500fed30 0x7f6850068ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:35322/0 (socket says 172.21.15.178:35322) 2024-09-10T11:21:39.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.506+0000 7f684f7fe700 1 -- 172.21.15.178:0/923127561 learned_addr learned my addr 172.21.15.178:0/923127561 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:21:39.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.506+0000 7f684f7fe700 1 -- 172.21.15.178:0/923127561 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6848009d00 con 0x7f68500fed30 2024-09-10T11:21:39.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.506+0000 7f684f7fe700 1 --2- 172.21.15.178:0/923127561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68500fed30 0x7f6850068ae0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f684800b830 tx=0x7f684800bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:39.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.506+0000 7f684cff9700 1 -- 172.21.15.178:0/923127561 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f684800e040 con 0x7f68500fed30 2024-09-10T11:21:39.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.506+0000 7f6855cf8700 1 -- 172.21.15.178:0/923127561 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6850197e40 con 0x7f68500fed30 2024-09-10T11:21:39.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.506+0000 7f684cff9700 1 -- 172.21.15.178:0/923127561 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6848012af0 con 0x7f68500fed30 2024-09-10T11:21:39.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.506+0000 7f684cff9700 1 -- 172.21.15.178:0/923127561 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f684801d9f0 con 0x7f68500fed30 2024-09-10T11:21:39.508 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.506+0000 7f6855cf8700 1 -- 172.21.15.178:0/923127561 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6850198260 con 0x7f68500fed30 2024-09-10T11:21:39.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.507+0000 7f684cff9700 1 -- 172.21.15.178:0/923127561 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 13) v1 ==== 45141+0+0 (secure 0 0 0) 0x7f684801dcb0 con 0x7f68500fed30 2024-09-10T11:21:39.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.507+0000 7f6855cf8700 1 -- 172.21.15.178:0/923127561 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685018da10 con 0x7f68500fed30 2024-09-10T11:21:39.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.507+0000 7f684cff9700 1 --2- 172.21.15.178:0/923127561 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f683c038490 0x7f683c03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:39.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.507+0000 7f684cff9700 1 -- 172.21.15.178:0/923127561 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f684804c960 con 0x7f68500fed30 2024-09-10T11:21:39.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.507+0000 7f684effd700 1 --2- 172.21.15.178:0/923127561 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f683c038490 0x7f683c03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:39.509 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.508+0000 7f684effd700 1 --2- 172.21.15.178:0/923127561 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f683c038490 0x7f683c03a940 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6838006fd0 tx=0x7f6838006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:39.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.512+0000 7f684cff9700 1 -- 172.21.15.178:0/923127561 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f684801abf0 con 0x7f68500fed30 2024-09-10T11:21:39.776 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.775+0000 7f6855cf8700 1 -- 172.21.15.178:0/923127561 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f685002cc30 con 0x7f68500fed30 2024-09-10T11:21:39.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.776+0000 7f684cff9700 1 -- 172.21.15.178:0/923127561 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f684801abf0 con 0x7f68500fed30 2024-09-10T11:21:39.777 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:21:39.777 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:21:39.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.777+0000 7f6855cf8700 1 -- 172.21.15.178:0/923127561 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f683c038490 msgr2=0x7f683c03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:39.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.777+0000 7f6855cf8700 1 --2- 172.21.15.178:0/923127561 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f683c038490 0x7f683c03a940 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f6838006fd0 tx=0x7f6838006e40 comp rx=0 tx=0).stop 2024-09-10T11:21:39.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.777+0000 7f6855cf8700 1 -- 172.21.15.178:0/923127561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68500fed30 msgr2=0x7f6850068ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:39.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.777+0000 7f6855cf8700 1 --2- 172.21.15.178:0/923127561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68500fed30 0x7f6850068ae0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f684800b830 tx=0x7f684800bd40 comp rx=0 tx=0).stop 2024-09-10T11:21:39.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.777+0000 7f6855cf8700 1 -- 172.21.15.178:0/923127561 shutdown_connections 2024-09-10T11:21:39.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.777+0000 7f6855cf8700 1 --2- 172.21.15.178:0/923127561 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f683c038490 0x7f683c03a940 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:39.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.778+0000 7f6855cf8700 1 --2- 172.21.15.178:0/923127561 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68500fed30 0x7f6850068ae0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:39.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.778+0000 7f6855cf8700 1 -- 172.21.15.178:0/923127561 >> 172.21.15.178:0/923127561 conn(0x7f68500fa2e0 msgr2=0x7f68500faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:39.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.778+0000 7f6855cf8700 1 -- 172.21.15.178:0/923127561 shutdown_connections 2024-09-10T11:21:39.779 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:39.778+0000 7f6855cf8700 1 -- 172.21.15.178:0/923127561 wait complete. 2024-09-10T11:21:39.780 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:21:40.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:40 smithi144 ceph-mon[27942]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:40.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:40 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/923127561' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:21:41.361 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:21:41.362 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:21:41.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:41 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:41.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:41 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:41.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:41 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:41.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:41 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T11:21:41.551 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:41.594 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:42.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:42 smithi144 ceph-mon[27942]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:21:42.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:42 smithi144 ceph-mon[27942]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T11:21:42.423 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:42 smithi144 ceph-mon[27942]: mgrmap e14: smithi144.bkogfs(active, since 58s) 2024-09-10T11:21:42.943 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.939+0000 7f5b1ce85700 1 -- 172.21.15.178:0/2760160975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b180fed30 msgr2=0x7f5b180ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.939+0000 7f5b1ce85700 1 --2- 172.21.15.178:0/2760160975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b180fed30 0x7f5b180ff140 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5b08009910 tx=0x7f5b08009c20 comp rx=0 tx=0).stop 2024-09-10T11:21:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.939+0000 7f5b1ce85700 1 -- 172.21.15.178:0/2760160975 shutdown_connections 2024-09-10T11:21:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.939+0000 7f5b1ce85700 1 --2- 172.21.15.178:0/2760160975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b180fed30 0x7f5b180ff140 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.939+0000 7f5b1ce85700 1 -- 172.21.15.178:0/2760160975 >> 172.21.15.178:0/2760160975 conn(0x7f5b180fa380 msgr2=0x7f5b180fc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.939+0000 7f5b1ce85700 1 -- 172.21.15.178:0/2760160975 shutdown_connections 2024-09-10T11:21:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.939+0000 7f5b1ce85700 1 -- 172.21.15.178:0/2760160975 wait complete. 2024-09-10T11:21:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.940+0000 7f5b1ce85700 1 Processor -- start 2024-09-10T11:21:42.944 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.940+0000 7f5b1ce85700 1 -- start start 2024-09-10T11:21:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.940+0000 7f5b1ce85700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b180fed30 0x7f5b180fc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.940+0000 7f5b1ce85700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b180fc850 con 0x7f5b180fed30 2024-09-10T11:21:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.940+0000 7f5b1659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b180fed30 0x7f5b180fc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.940+0000 7f5b1659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b180fed30 0x7f5b180fc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:35340/0 (socket says 172.21.15.178:35340) 2024-09-10T11:21:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.940+0000 7f5b1659c700 1 -- 172.21.15.178:0/577627654 learned_addr learned my addr 172.21.15.178:0/577627654 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:21:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.940+0000 7f5b1659c700 1 -- 172.21.15.178:0/577627654 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b080095f0 con 0x7f5b180fed30 2024-09-10T11:21:42.945 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.940+0000 7f5b1659c700 1 --2- 172.21.15.178:0/577627654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b180fed30 0x7f5b180fc310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5b08005b40 tx=0x7f5b08004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.941+0000 7f5b077fe700 1 -- 172.21.15.178:0/577627654 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5b0801d070 con 0x7f5b180fed30 2024-09-10T11:21:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.941+0000 7f5b1ce85700 1 -- 172.21.15.178:0/577627654 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b180fa960 con 0x7f5b180fed30 2024-09-10T11:21:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.941+0000 7f5b077fe700 1 -- 172.21.15.178:0/577627654 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5b0800be10 con 0x7f5b180fed30 2024-09-10T11:21:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.941+0000 7f5b077fe700 1 -- 172.21.15.178:0/577627654 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5b0800f460 con 0x7f5b180fed30 2024-09-10T11:21:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.941+0000 7f5b1ce85700 1 -- 172.21.15.178:0/577627654 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b180fae00 con 0x7f5b180fed30 2024-09-10T11:21:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.941+0000 7f5b077fe700 1 -- 172.21.15.178:0/577627654 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 14) v1 ==== 45155+0+0 (secure 0 0 0) 0x7f5b0800f6d0 con 0x7f5b180fed30 2024-09-10T11:21:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.942+0000 7f5b1ce85700 1 -- 172.21.15.178:0/577627654 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b18062380 con 0x7f5b180fed30 2024-09-10T11:21:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.942+0000 7f5b077fe700 1 --2- 172.21.15.178:0/577627654 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f5b000384e0 0x7f5b0003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:42.946 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.942+0000 7f5b077fe700 1 -- 172.21.15.178:0/577627654 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f5b0804c560 con 0x7f5b180fed30 2024-09-10T11:21:42.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.942+0000 7f5b15d9b700 1 -- 172.21.15.178:0/577627654 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f5b000384e0 msgr2=0x7f5b0003a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:21:42.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.942+0000 7f5b15d9b700 1 --2- 172.21.15.178:0/577627654 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f5b000384e0 0x7f5b0003a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T11:21:42.947 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:42.947+0000 7f5b077fe700 1 -- 172.21.15.178:0/577627654 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5b08020550 con 0x7f5b180fed30 2024-09-10T11:21:43.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:43.143+0000 7f5b15d9b700 1 -- 172.21.15.178:0/577627654 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f5b000384e0 msgr2=0x7f5b0003a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:21:43.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:43.143+0000 7f5b15d9b700 1 --2- 172.21.15.178:0/577627654 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f5b000384e0 0x7f5b0003a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T11:21:43.211 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:43.210+0000 7f5b1ce85700 1 -- 172.21.15.178:0/577627654 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f5b1818e380 con 0x7f5b180fed30 2024-09-10T11:21:43.212 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:43.211+0000 7f5b077fe700 1 -- 172.21.15.178:0/577627654 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f5b08027030 con 0x7f5b180fed30 2024-09-10T11:21:43.212 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:21:43.212 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:21:43.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:43.212+0000 7f5b1ce85700 1 -- 172.21.15.178:0/577627654 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f5b000384e0 msgr2=0x7f5b0003a990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T11:21:43.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:43.213+0000 7f5b1ce85700 1 --2- 172.21.15.178:0/577627654 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f5b000384e0 0x7f5b0003a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:43.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:43.213+0000 7f5b1ce85700 1 -- 172.21.15.178:0/577627654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b180fed30 msgr2=0x7f5b180fc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:43.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:43.213+0000 7f5b1ce85700 1 --2- 172.21.15.178:0/577627654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b180fed30 0x7f5b180fc310 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5b08005b40 tx=0x7f5b08004740 comp rx=0 tx=0).stop 2024-09-10T11:21:43.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:43.213+0000 7f5b1ce85700 1 -- 172.21.15.178:0/577627654 shutdown_connections 2024-09-10T11:21:43.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:43.213+0000 7f5b1ce85700 1 --2- 172.21.15.178:0/577627654 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f5b000384e0 0x7f5b0003a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:43.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:43.213+0000 7f5b1ce85700 1 --2- 172.21.15.178:0/577627654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b180fed30 0x7f5b180fc310 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:43.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:43.213+0000 7f5b1ce85700 1 -- 172.21.15.178:0/577627654 >> 172.21.15.178:0/577627654 conn(0x7f5b180fa380 msgr2=0x7f5b18069100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:43.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:43.213+0000 7f5b1ce85700 1 -- 172.21.15.178:0/577627654 shutdown_connections 2024-09-10T11:21:43.214 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:43.213+0000 7f5b1ce85700 1 -- 172.21.15.178:0/577627654 wait complete. 2024-09-10T11:21:43.215 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:21:43.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:43 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/577627654' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:21:44.766 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:21:44.766 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:21:44.963 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:45.007 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:46.454 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.453+0000 7f6644439700 1 -- 172.21.15.178:0/3799668378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c0ffe70 msgr2=0x7f663c100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:46.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.453+0000 7f6644439700 1 --2- 172.21.15.178:0/3799668378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c0ffe70 0x7f663c100280 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6638009910 tx=0x7f6638009c20 comp rx=0 tx=0).stop 2024-09-10T11:21:46.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.453+0000 7f6644439700 1 -- 172.21.15.178:0/3799668378 shutdown_connections 2024-09-10T11:21:46.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.453+0000 7f6644439700 1 --2- 172.21.15.178:0/3799668378 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c0ffe70 0x7f663c100280 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:46.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.453+0000 7f6644439700 1 -- 172.21.15.178:0/3799668378 >> 172.21.15.178:0/3799668378 conn(0x7f663c0fb470 msgr2=0x7f663c0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:46.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.453+0000 7f6644439700 1 -- 172.21.15.178:0/3799668378 shutdown_connections 2024-09-10T11:21:46.455 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.453+0000 7f6644439700 1 -- 172.21.15.178:0/3799668378 wait complete. 2024-09-10T11:21:46.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.454+0000 7f6644439700 1 Processor -- start 2024-09-10T11:21:46.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.454+0000 7f6644439700 1 -- start start 2024-09-10T11:21:46.456 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.454+0000 7f6644439700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c0ffe70 0x7f663c197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:46.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.454+0000 7f6644439700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f663c1976d0 con 0x7f663c0ffe70 2024-09-10T11:21:46.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.454+0000 7f66421d5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c0ffe70 0x7f663c197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:46.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.454+0000 7f66421d5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c0ffe70 0x7f663c197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:57440/0 (socket says 172.21.15.178:57440) 2024-09-10T11:21:46.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.454+0000 7f66421d5700 1 -- 172.21.15.178:0/1463103717 learned_addr learned my addr 172.21.15.178:0/1463103717 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:21:46.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.454+0000 7f66421d5700 1 -- 172.21.15.178:0/1463103717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66380095f0 con 0x7f663c0ffe70 2024-09-10T11:21:46.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.454+0000 7f66421d5700 1 --2- 172.21.15.178:0/1463103717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c0ffe70 0x7f663c197190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6638005b40 tx=0x7f6638004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:46.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.455+0000 7f66337fe700 1 -- 172.21.15.178:0/1463103717 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f663801d070 con 0x7f663c0ffe70 2024-09-10T11:21:46.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.455+0000 7f66337fe700 1 -- 172.21.15.178:0/1463103717 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f663800be10 con 0x7f663c0ffe70 2024-09-10T11:21:46.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.455+0000 7f6644439700 1 -- 172.21.15.178:0/1463103717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f663c1978d0 con 0x7f663c0ffe70 2024-09-10T11:21:46.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.455+0000 7f66337fe700 1 -- 172.21.15.178:0/1463103717 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f663800f460 con 0x7f663c0ffe70 2024-09-10T11:21:46.457 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.455+0000 7f6644439700 1 -- 172.21.15.178:0/1463103717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f663c197d70 con 0x7f663c0ffe70 2024-09-10T11:21:46.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.456+0000 7f66337fe700 1 -- 172.21.15.178:0/1463103717 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 14) v1 ==== 45155+0+0 (secure 0 0 0) 0x7f663800b960 con 0x7f663c0ffe70 2024-09-10T11:21:46.458 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.456+0000 7f6644439700 1 -- 172.21.15.178:0/1463103717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663c190e70 con 0x7f663c0ffe70 2024-09-10T11:21:46.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.456+0000 7f66337fe700 1 --2- 172.21.15.178:0/1463103717 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f66280384e0 0x7f662803a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:46.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.456+0000 7f66337fe700 1 -- 172.21.15.178:0/1463103717 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1069+0+0 (secure 0 0 0) 0x7f663804c400 con 0x7f663c0ffe70 2024-09-10T11:21:46.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.456+0000 7f66419d4700 1 -- 172.21.15.178:0/1463103717 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f66280384e0 msgr2=0x7f662803a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:21:46.459 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.456+0000 7f66419d4700 1 --2- 172.21.15.178:0/1463103717 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f66280384e0 0x7f662803a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T11:21:46.461 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.461+0000 7f66337fe700 1 -- 172.21.15.178:0/1463103717 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f663802a360 con 0x7f663c0ffe70 2024-09-10T11:21:46.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.657+0000 7f66419d4700 1 -- 172.21.15.178:0/1463103717 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f66280384e0 msgr2=0x7f662803a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.144:6800/2 2024-09-10T11:21:46.658 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.657+0000 7f66419d4700 1 --2- 172.21.15.178:0/1463103717 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f66280384e0 0x7f662803a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T11:21:46.725 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.724+0000 7f6644439700 1 -- 172.21.15.178:0/1463103717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f663c02cc30 con 0x7f663c0ffe70 2024-09-10T11:21:46.726 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.725+0000 7f66337fe700 1 -- 172.21.15.178:0/1463103717 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f6638027030 con 0x7f663c0ffe70 2024-09-10T11:21:46.726 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:21:46.726 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:21:46.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.727+0000 7f6644439700 1 -- 172.21.15.178:0/1463103717 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f66280384e0 msgr2=0x7f662803a990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T11:21:46.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.727+0000 7f6644439700 1 --2- 172.21.15.178:0/1463103717 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f66280384e0 0x7f662803a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:46.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.727+0000 7f6644439700 1 -- 172.21.15.178:0/1463103717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c0ffe70 msgr2=0x7f663c197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:46.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.727+0000 7f6644439700 1 --2- 172.21.15.178:0/1463103717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c0ffe70 0x7f663c197190 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6638005b40 tx=0x7f6638004740 comp rx=0 tx=0).stop 2024-09-10T11:21:46.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.727+0000 7f6644439700 1 -- 172.21.15.178:0/1463103717 shutdown_connections 2024-09-10T11:21:46.728 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.727+0000 7f6644439700 1 --2- 172.21.15.178:0/1463103717 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f66280384e0 0x7f662803a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:46.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.727+0000 7f6644439700 1 --2- 172.21.15.178:0/1463103717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f663c0ffe70 0x7f663c197190 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:46.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.727+0000 7f6644439700 1 -- 172.21.15.178:0/1463103717 >> 172.21.15.178:0/1463103717 conn(0x7f663c0fb470 msgr2=0x7f663c0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:46.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.727+0000 7f6644439700 1 -- 172.21.15.178:0/1463103717 shutdown_connections 2024-09-10T11:21:46.729 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:46.727+0000 7f6644439700 1 -- 172.21.15.178:0/1463103717 wait complete. 2024-09-10T11:21:46.729 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:21:46.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:46 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/1463103717' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:21:48.287 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:21:48.287 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:21:48.477 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:48.521 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: Active manager daemon smithi144.bkogfs restarted 2024-09-10T11:21:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: Activating manager daemon smithi144.bkogfs 2024-09-10T11:21:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: osdmap e5: 0 total, 0 up, 0 in 2024-09-10T11:21:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: mgrmap e15: smithi144.bkogfs(active, starting, since 0.00545597s) 2024-09-10T11:21:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:21:48.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr metadata", "who": "smithi144.bkogfs", "id": "smithi144.bkogfs"}]: dispatch 2024-09-10T11:21:48.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T11:21:48.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T11:21:48.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T11:21:48.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: Manager daemon smithi144.bkogfs is now available 2024-09-10T11:21:48.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:21:48.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:21:48.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:48.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:21:48.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/mirror_snapshot_schedule"}]: dispatch 2024-09-10T11:21:48.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/trash_purge_schedule"}]: dispatch 2024-09-10T11:21:48.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:48 smithi144 ceph-mon[27942]: [10/Sep/2024:11:21:48] ENGINE Bus STARTING 2024-09-10T11:21:49.661 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:49 smithi144 ceph-mon[27942]: [10/Sep/2024:11:21:48] ENGINE Serving on https://172.21.15.144:7150 2024-09-10T11:21:49.661 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:49 smithi144 ceph-mon[27942]: [10/Sep/2024:11:21:48] ENGINE Serving on http://172.21.15.144:8765 2024-09-10T11:21:49.661 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:49 smithi144 ceph-mon[27942]: [10/Sep/2024:11:21:48] ENGINE Bus STARTED 2024-09-10T11:21:49.661 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:49 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:49.661 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:49 smithi144 ceph-mon[27942]: mgrmap e16: smithi144.bkogfs(active, since 1.0054s) 2024-09-10T11:21:49.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.991+0000 7f34f13a8700 1 -- 172.21.15.178:0/718301975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34ec0fed30 msgr2=0x7f34ec0ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:49.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.991+0000 7f34f13a8700 1 --2- 172.21.15.178:0/718301975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34ec0fed30 0x7f34ec0ff140 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f34dc009910 tx=0x7f34dc009c20 comp rx=0 tx=0).stop 2024-09-10T11:21:49.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.991+0000 7f34f13a8700 1 -- 172.21.15.178:0/718301975 shutdown_connections 2024-09-10T11:21:49.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.991+0000 7f34f13a8700 1 --2- 172.21.15.178:0/718301975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34ec0fed30 0x7f34ec0ff140 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:49.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.991+0000 7f34f13a8700 1 -- 172.21.15.178:0/718301975 >> 172.21.15.178:0/718301975 conn(0x7f34ec0fa2e0 msgr2=0x7f34ec0fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:49.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.991+0000 7f34f13a8700 1 -- 172.21.15.178:0/718301975 shutdown_connections 2024-09-10T11:21:49.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.991+0000 7f34f13a8700 1 -- 172.21.15.178:0/718301975 wait complete. 2024-09-10T11:21:49.993 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.992+0000 7f34f13a8700 1 Processor -- start 2024-09-10T11:21:49.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.992+0000 7f34f13a8700 1 -- start start 2024-09-10T11:21:49.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.992+0000 7f34f13a8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34ec0fed30 0x7f34ec068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:49.994 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.992+0000 7f34f13a8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec195b10 con 0x7f34ec0fed30 2024-09-10T11:21:49.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.992+0000 7f34eaffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34ec0fed30 0x7f34ec068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:49.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.992+0000 7f34eaffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34ec0fed30 0x7f34ec068ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:57468/0 (socket says 172.21.15.178:57468) 2024-09-10T11:21:49.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.992+0000 7f34eaffd700 1 -- 172.21.15.178:0/1973803493 learned_addr learned my addr 172.21.15.178:0/1973803493 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:21:49.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.992+0000 7f34eaffd700 1 -- 172.21.15.178:0/1973803493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34dc0095f0 con 0x7f34ec0fed30 2024-09-10T11:21:49.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.993+0000 7f34eaffd700 1 --2- 172.21.15.178:0/1973803493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34ec0fed30 0x7f34ec068ac0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f34dc005b40 tx=0x7f34dc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:49.995 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.993+0000 7f34d3fff700 1 -- 172.21.15.178:0/1973803493 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34dc01d070 con 0x7f34ec0fed30 2024-09-10T11:21:49.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.993+0000 7f34d3fff700 1 -- 172.21.15.178:0/1973803493 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f34dc00be10 con 0x7f34ec0fed30 2024-09-10T11:21:49.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.993+0000 7f34f13a8700 1 -- 172.21.15.178:0/1973803493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ec19a120 con 0x7f34ec0fed30 2024-09-10T11:21:49.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.993+0000 7f34d3fff700 1 -- 172.21.15.178:0/1973803493 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f34dc00f460 con 0x7f34ec0fed30 2024-09-10T11:21:49.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.993+0000 7f34f13a8700 1 -- 172.21.15.178:0/1973803493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ec19a480 con 0x7f34ec0fed30 2024-09-10T11:21:49.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.994+0000 7f34d3fff700 1 -- 172.21.15.178:0/1973803493 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 16) v1 ==== 45005+0+0 (secure 0 0 0) 0x7f34dc00b960 con 0x7f34ec0fed30 2024-09-10T11:21:49.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.994+0000 7f34f13a8700 1 -- 172.21.15.178:0/1973803493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ec18da10 con 0x7f34ec0fed30 2024-09-10T11:21:49.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.994+0000 7f34d3fff700 1 --2- 172.21.15.178:0/1973803493 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f34d4038340 0x7f34d403a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:49.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.994+0000 7f34d3fff700 1 -- 172.21.15.178:0/1973803493 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7f34dc04c270 con 0x7f34ec0fed30 2024-09-10T11:21:49.996 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.994+0000 7f34ea7fc700 1 --2- 172.21.15.178:0/1973803493 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f34d4038340 0x7f34d403a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:49.997 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.995+0000 7f34ea7fc700 1 --2- 172.21.15.178:0/1973803493 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f34d4038340 0x7f34d403a7f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f34e0006fd0 tx=0x7f34e0006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:50.000 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:49.999+0000 7f34d3fff700 1 -- 172.21.15.178:0/1973803493 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f34dc02bba0 con 0x7f34ec0fed30 2024-09-10T11:21:50.278 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:50.276+0000 7f34f13a8700 1 -- 172.21.15.178:0/1973803493 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f34ec02cc30 con 0x7f34ec0fed30 2024-09-10T11:21:50.279 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:50.276+0000 7f34d3fff700 1 -- 172.21.15.178:0/1973803493 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f34dc027030 con 0x7f34ec0fed30 2024-09-10T11:21:50.279 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:21:50.279 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:21:50.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:50.279+0000 7f34f13a8700 1 -- 172.21.15.178:0/1973803493 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f34d4038340 msgr2=0x7f34d403a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:50.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:50.279+0000 7f34f13a8700 1 --2- 172.21.15.178:0/1973803493 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f34d4038340 0x7f34d403a7f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f34e0006fd0 tx=0x7f34e0006e40 comp rx=0 tx=0).stop 2024-09-10T11:21:50.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:50.279+0000 7f34f13a8700 1 -- 172.21.15.178:0/1973803493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34ec0fed30 msgr2=0x7f34ec068ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:50.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:50.279+0000 7f34f13a8700 1 --2- 172.21.15.178:0/1973803493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34ec0fed30 0x7f34ec068ac0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f34dc005b40 tx=0x7f34dc004740 comp rx=0 tx=0).stop 2024-09-10T11:21:50.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:50.279+0000 7f34f13a8700 1 -- 172.21.15.178:0/1973803493 shutdown_connections 2024-09-10T11:21:50.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:50.279+0000 7f34f13a8700 1 --2- 172.21.15.178:0/1973803493 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f34d4038340 0x7f34d403a7f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:50.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:50.279+0000 7f34f13a8700 1 --2- 172.21.15.178:0/1973803493 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f34ec0fed30 0x7f34ec068ac0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:50.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:50.279+0000 7f34f13a8700 1 -- 172.21.15.178:0/1973803493 >> 172.21.15.178:0/1973803493 conn(0x7f34ec0fa2e0 msgr2=0x7f34ec0faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:50.281 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:50.279+0000 7f34f13a8700 1 -- 172.21.15.178:0/1973803493 shutdown_connections 2024-09-10T11:21:50.282 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:50.279+0000 7f34f13a8700 1 -- 172.21.15.178:0/1973803493 wait complete. 2024-09-10T11:21:50.282 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:21:50.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:50 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:50.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:50 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/1973803493' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:21:50.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:50 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:51.887 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:21:51.888 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:21:51.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:51 smithi144 ceph-mon[27942]: mgrmap e17: smithi144.bkogfs(active, since 2s) 2024-09-10T11:21:52.346 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:52.399 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-10T11:21:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:53 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:53 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:53 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:53 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:53 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:21:54.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.250+0000 7f428122e700 1 -- 172.21.15.178:0/4179317418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f427c0fed30 msgr2=0x7f427c0ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:54.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.250+0000 7f428122e700 1 --2- 172.21.15.178:0/4179317418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f427c0fed30 0x7f427c0ff140 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f426c009910 tx=0x7f426c009c20 comp rx=0 tx=0).stop 2024-09-10T11:21:54.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.250+0000 7f428122e700 1 -- 172.21.15.178:0/4179317418 shutdown_connections 2024-09-10T11:21:54.252 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.250+0000 7f428122e700 1 --2- 172.21.15.178:0/4179317418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f427c0fed30 0x7f427c0ff140 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:54.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.250+0000 7f428122e700 1 -- 172.21.15.178:0/4179317418 >> 172.21.15.178:0/4179317418 conn(0x7f427c0fa2c0 msgr2=0x7f427c0fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:54.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.250+0000 7f428122e700 1 -- 172.21.15.178:0/4179317418 shutdown_connections 2024-09-10T11:21:54.253 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.250+0000 7f428122e700 1 -- 172.21.15.178:0/4179317418 wait complete. 2024-09-10T11:21:54.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.250+0000 7f428122e700 1 Processor -- start 2024-09-10T11:21:54.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.251+0000 7f428122e700 1 -- start start 2024-09-10T11:21:54.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.251+0000 7f428122e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f427c0fed30 0x7f427c0fc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:54.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.251+0000 7f428122e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427c0fc7b0 con 0x7f427c0fed30 2024-09-10T11:21:54.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.251+0000 7f427ad9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f427c0fed30 0x7f427c0fc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:54.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.251+0000 7f427ad9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f427c0fed30 0x7f427c0fc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:57482/0 (socket says 172.21.15.178:57482) 2024-09-10T11:21:54.254 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.251+0000 7f427ad9d700 1 -- 172.21.15.178:0/1919779747 learned_addr learned my addr 172.21.15.178:0/1919779747 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:21:54.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.251+0000 7f427ad9d700 1 -- 172.21.15.178:0/1919779747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c0095f0 con 0x7f427c0fed30 2024-09-10T11:21:54.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.252+0000 7f427ad9d700 1 --2- 172.21.15.178:0/1919779747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f427c0fed30 0x7f427c0fc270 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f426c005b40 tx=0x7f426c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:54.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.252+0000 7f4263fff700 1 -- 172.21.15.178:0/1919779747 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f426c01d070 con 0x7f427c0fed30 2024-09-10T11:21:54.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.252+0000 7f4263fff700 1 -- 172.21.15.178:0/1919779747 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f426c00be10 con 0x7f427c0fed30 2024-09-10T11:21:54.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.252+0000 7f428122e700 1 -- 172.21.15.178:0/1919779747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f427c0fa8c0 con 0x7f427c0fed30 2024-09-10T11:21:54.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.252+0000 7f4263fff700 1 -- 172.21.15.178:0/1919779747 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f426c00f460 con 0x7f427c0fed30 2024-09-10T11:21:54.255 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.252+0000 7f428122e700 1 -- 172.21.15.178:0/1919779747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427c0fad60 con 0x7f427c0fed30 2024-09-10T11:21:54.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.253+0000 7f4263fff700 1 -- 172.21.15.178:0/1919779747 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7f426c00f6d0 con 0x7f427c0fed30 2024-09-10T11:21:54.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.253+0000 7f4263fff700 1 --2- 172.21.15.178:0/1919779747 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4264038580 0x7f426403aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:54.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.253+0000 7f4263fff700 1 -- 172.21.15.178:0/1919779747 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7f426c04c600 con 0x7f427c0fed30 2024-09-10T11:21:54.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.253+0000 7f427a59c700 1 --2- 172.21.15.178:0/1919779747 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4264038580 0x7f426403aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:54.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.253+0000 7f428122e700 1 -- 172.21.15.178:0/1919779747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f427c062380 con 0x7f427c0fed30 2024-09-10T11:21:54.256 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.253+0000 7f427a59c700 1 --2- 172.21.15.178:0/1919779747 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4264038580 0x7f426403aa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4270006fd0 tx=0x7f4270006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:54.260 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.258+0000 7f4263fff700 1 -- 172.21.15.178:0/1919779747 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f426c027070 con 0x7f427c0fed30 2024-09-10T11:21:54.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.543+0000 7f428122e700 1 -- 172.21.15.178:0/1919779747 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f427c18e400 con 0x7f427c0fed30 2024-09-10T11:21:54.545 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.543+0000 7f4263fff700 1 -- 172.21.15.178:0/1919779747 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f427c18e400 con 0x7f427c0fed30 2024-09-10T11:21:54.545 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:21:54.546 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:21:54.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.546+0000 7f428122e700 1 -- 172.21.15.178:0/1919779747 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4264038580 msgr2=0x7f426403aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:54.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.546+0000 7f428122e700 1 --2- 172.21.15.178:0/1919779747 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4264038580 0x7f426403aa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4270006fd0 tx=0x7f4270006e40 comp rx=0 tx=0).stop 2024-09-10T11:21:54.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.546+0000 7f428122e700 1 -- 172.21.15.178:0/1919779747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f427c0fed30 msgr2=0x7f427c0fc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:54.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.546+0000 7f428122e700 1 --2- 172.21.15.178:0/1919779747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f427c0fed30 0x7f427c0fc270 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f426c005b40 tx=0x7f426c004740 comp rx=0 tx=0).stop 2024-09-10T11:21:54.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.546+0000 7f428122e700 1 -- 172.21.15.178:0/1919779747 shutdown_connections 2024-09-10T11:21:54.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.546+0000 7f428122e700 1 --2- 172.21.15.178:0/1919779747 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4264038580 0x7f426403aa30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:54.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.546+0000 7f428122e700 1 --2- 172.21.15.178:0/1919779747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f427c0fed30 0x7f427c0fc270 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:54.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.546+0000 7f428122e700 1 -- 172.21.15.178:0/1919779747 >> 172.21.15.178:0/1919779747 conn(0x7f427c0fa2c0 msgr2=0x7f427c069060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:54.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.546+0000 7f428122e700 1 -- 172.21.15.178:0/1919779747 shutdown_connections 2024-09-10T11:21:54.548 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:54.546+0000 7f428122e700 1 -- 172.21.15.178:0/1919779747 wait complete. 2024-09-10T11:21:54.549 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:21:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:55 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:55 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:55 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:55 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:55 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:21:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:55 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:21:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:55 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:21:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:55 smithi144 ceph-mon[27942]: Updating smithi144:/etc/ceph/ceph.conf 2024-09-10T11:21:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:55 smithi144 ceph-mon[27942]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T11:21:55.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:55 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/1919779747' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:21:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:56 smithi144 ceph-mon[27942]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:21:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:56 smithi144 ceph-mon[27942]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:21:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:56 smithi144 ceph-mon[27942]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:21:56.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:56 smithi144 ceph-mon[27942]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:21:56.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:56 smithi144 ceph-mon[27942]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:21:56.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:56 smithi144 ceph-mon[27942]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:21:56.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:56.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:56.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:56.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:56.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:56.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi178", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T11:21:56.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi178", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-10T11:21:56.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:21:56.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:56 smithi144 ceph-mon[27942]: Deploying daemon ceph-exporter.smithi178 on smithi178 2024-09-10T11:21:56.225 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:21:56.225 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:21:56.412 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.960+0000 7f078cadb700 1 -- 172.21.15.178:0/1362636625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07880fefa0 msgr2=0x7f07880ff370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.960+0000 7f078cadb700 1 --2- 172.21.15.178:0/1362636625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07880fefa0 0x7f07880ff370 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f078000b0a0 tx=0x7f078000b3b0 comp rx=0 tx=0).stop 2024-09-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.960+0000 7f078cadb700 1 -- 172.21.15.178:0/1362636625 shutdown_connections 2024-09-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.960+0000 7f078cadb700 1 --2- 172.21.15.178:0/1362636625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07880fefa0 0x7f07880ff370 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.960+0000 7f078cadb700 1 -- 172.21.15.178:0/1362636625 >> 172.21.15.178:0/1362636625 conn(0x7f07880fa860 msgr2=0x7f07880fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.960+0000 7f078cadb700 1 -- 172.21.15.178:0/1362636625 shutdown_connections 2024-09-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.960+0000 7f078cadb700 1 -- 172.21.15.178:0/1362636625 wait complete. 2024-09-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.961+0000 7f078cadb700 1 Processor -- start 2024-09-10T11:21:57.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.961+0000 7f078cadb700 1 -- start start 2024-09-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.961+0000 7f078cadb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07880fefa0 0x7f0788195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.961+0000 7f078cadb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0788195880 con 0x7f07880fefa0 2024-09-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.962+0000 7f078659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07880fefa0 0x7f0788195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.962+0000 7f078659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07880fefa0 0x7f0788195340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:57020/0 (socket says 172.21.15.178:57020) 2024-09-10T11:21:57.965 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.962+0000 7f078659c700 1 -- 172.21.15.178:0/1623968212 learned_addr learned my addr 172.21.15.178:0/1623968212 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:21:57.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.962+0000 7f078659c700 1 -- 172.21.15.178:0/1623968212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0780009d00 con 0x7f07880fefa0 2024-09-10T11:21:57.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.962+0000 7f078659c700 1 --2- 172.21.15.178:0/1623968212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07880fefa0 0x7f0788195340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f07800062a0 tx=0x7f0780003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:57.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.962+0000 7f077f7fe700 1 -- 172.21.15.178:0/1623968212 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f078000e040 con 0x7f07880fefa0 2024-09-10T11:21:57.966 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.962+0000 7f077f7fe700 1 -- 172.21.15.178:0/1623968212 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f078000bdd0 con 0x7f07880fefa0 2024-09-10T11:21:57.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.962+0000 7f078cadb700 1 -- 172.21.15.178:0/1623968212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0788195a80 con 0x7f07880fefa0 2024-09-10T11:21:57.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.962+0000 7f077f7fe700 1 -- 172.21.15.178:0/1623968212 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f078001d940 con 0x7f07880fefa0 2024-09-10T11:21:57.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.963+0000 7f078cadb700 1 -- 172.21.15.178:0/1623968212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0788195f00 con 0x7f07880fefa0 2024-09-10T11:21:57.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.963+0000 7f077f7fe700 1 -- 172.21.15.178:0/1623968212 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7f0780004250 con 0x7f07880fefa0 2024-09-10T11:21:57.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.963+0000 7f078cadb700 1 -- 172.21.15.178:0/1623968212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0788062380 con 0x7f07880fefa0 2024-09-10T11:21:57.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.964+0000 7f077f7fe700 1 --2- 172.21.15.178:0/1623968212 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0774038580 0x7f077403aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:21:57.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.964+0000 7f077f7fe700 1 -- 172.21.15.178:0/1623968212 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7f078004c3c0 con 0x7f07880fefa0 2024-09-10T11:21:57.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.964+0000 7f0785d9b700 1 --2- 172.21.15.178:0/1623968212 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0774038580 0x7f077403aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:21:57.967 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.964+0000 7f0785d9b700 1 --2- 172.21.15.178:0/1623968212 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0774038580 0x7f077403aa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0770006fd0 tx=0x7f0770006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:21:57.970 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:57.969+0000 7f077f7fe700 1 -- 172.21.15.178:0/1623968212 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0780011570 con 0x7f07880fefa0 2024-09-10T11:21:58.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:58.241+0000 7f078cadb700 1 -- 172.21.15.178:0/1623968212 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f07881966a0 con 0x7f07880fefa0 2024-09-10T11:21:58.244 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:58.242+0000 7f077f7fe700 1 -- 172.21.15.178:0/1623968212 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f0780011390 con 0x7f07880fefa0 2024-09-10T11:21:58.244 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:21:58.245 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:21:58.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:58.244+0000 7f078cadb700 1 -- 172.21.15.178:0/1623968212 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0774038580 msgr2=0x7f077403aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:58.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:58.244+0000 7f078cadb700 1 --2- 172.21.15.178:0/1623968212 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0774038580 0x7f077403aa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0770006fd0 tx=0x7f0770006e40 comp rx=0 tx=0).stop 2024-09-10T11:21:58.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:58.244+0000 7f078cadb700 1 -- 172.21.15.178:0/1623968212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07880fefa0 msgr2=0x7f0788195340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:21:58.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:58.244+0000 7f078cadb700 1 --2- 172.21.15.178:0/1623968212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07880fefa0 0x7f0788195340 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f07800062a0 tx=0x7f0780003ce0 comp rx=0 tx=0).stop 2024-09-10T11:21:58.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:58.244+0000 7f078cadb700 1 -- 172.21.15.178:0/1623968212 shutdown_connections 2024-09-10T11:21:58.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:58.244+0000 7f078cadb700 1 --2- 172.21.15.178:0/1623968212 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0774038580 0x7f077403aa30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:58.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:58.244+0000 7f078cadb700 1 --2- 172.21.15.178:0/1623968212 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f07880fefa0 0x7f0788195340 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:21:58.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:58.244+0000 7f078cadb700 1 -- 172.21.15.178:0/1623968212 >> 172.21.15.178:0/1623968212 conn(0x7f07880fa860 msgr2=0x7f07880687f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:21:58.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:58.244+0000 7f078cadb700 1 -- 172.21.15.178:0/1623968212 shutdown_connections 2024-09-10T11:21:58.247 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:21:58.244+0000 7f078cadb700 1 -- 172.21.15.178:0/1623968212 wait complete. 2024-09-10T11:21:58.248 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:21:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:58 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/1623968212' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:21:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:59 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:59 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:59 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:59 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:21:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:59 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi178", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T11:21:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:59 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi178", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-10T11:21:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:59 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:21:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:21:59 smithi144 ceph-mon[27942]: Deploying daemon crash.smithi178 on smithi178 2024-09-10T11:21:59.986 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:21:59.987 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:22:00.197 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:22:01.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.690+0000 7fb24ecfe700 1 -- 172.21.15.178:0/3748049621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2480fefa0 msgr2=0x7fb2480ff370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:01.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.690+0000 7fb24ecfe700 1 --2- 172.21.15.178:0/3748049621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2480fefa0 0x7fb2480ff370 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fb23800b0a0 tx=0x7fb23800b3b0 comp rx=0 tx=0).stop 2024-09-10T11:22:01.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.691+0000 7fb24ecfe700 1 -- 172.21.15.178:0/3748049621 shutdown_connections 2024-09-10T11:22:01.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.691+0000 7fb24ecfe700 1 --2- 172.21.15.178:0/3748049621 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2480fefa0 0x7fb2480ff370 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:01.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.691+0000 7fb24ecfe700 1 -- 172.21.15.178:0/3748049621 >> 172.21.15.178:0/3748049621 conn(0x7fb2480fa860 msgr2=0x7fb2480fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:22:01.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.691+0000 7fb24ecfe700 1 -- 172.21.15.178:0/3748049621 shutdown_connections 2024-09-10T11:22:01.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.691+0000 7fb24ecfe700 1 -- 172.21.15.178:0/3748049621 wait complete. 2024-09-10T11:22:01.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.691+0000 7fb24ecfe700 1 Processor -- start 2024-09-10T11:22:01.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.691+0000 7fb24ecfe700 1 -- start start 2024-09-10T11:22:01.693 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.691+0000 7fb24ecfe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2480fefa0 0x7fb2481975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:01.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.691+0000 7fb24ecfe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb248197b10 con 0x7fb2480fefa0 2024-09-10T11:22:01.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.692+0000 7fb24ca9a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2480fefa0 0x7fb2481975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:01.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.692+0000 7fb24ca9a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2480fefa0 0x7fb2481975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:57056/0 (socket says 172.21.15.178:57056) 2024-09-10T11:22:01.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.692+0000 7fb24ca9a700 1 -- 172.21.15.178:0/3295372083 learned_addr learned my addr 172.21.15.178:0/3295372083 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:22:01.694 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.692+0000 7fb24ca9a700 1 -- 172.21.15.178:0/3295372083 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb238009d00 con 0x7fb2480fefa0 2024-09-10T11:22:01.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.692+0000 7fb24ca9a700 1 --2- 172.21.15.178:0/3295372083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2480fefa0 0x7fb2481975d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb2380062a0 tx=0x7fb238003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:22:01.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.692+0000 7fb245ffb700 1 -- 172.21.15.178:0/3295372083 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb23800e040 con 0x7fb2480fefa0 2024-09-10T11:22:01.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.692+0000 7fb245ffb700 1 -- 172.21.15.178:0/3295372083 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb23800bdd0 con 0x7fb2480fefa0 2024-09-10T11:22:01.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.692+0000 7fb24ecfe700 1 -- 172.21.15.178:0/3295372083 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb248197d10 con 0x7fb2480fefa0 2024-09-10T11:22:01.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.692+0000 7fb245ffb700 1 -- 172.21.15.178:0/3295372083 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb23801d940 con 0x7fb2480fefa0 2024-09-10T11:22:01.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.692+0000 7fb24ecfe700 1 -- 172.21.15.178:0/3295372083 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb248198190 con 0x7fb2480fefa0 2024-09-10T11:22:01.695 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.693+0000 7fb245ffb700 1 -- 172.21.15.178:0/3295372083 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7fb238004250 con 0x7fb2480fefa0 2024-09-10T11:22:01.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.693+0000 7fb24ecfe700 1 -- 172.21.15.178:0/3295372083 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb248062380 con 0x7fb2480fefa0 2024-09-10T11:22:01.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.694+0000 7fb245ffb700 1 --2- 172.21.15.178:0/3295372083 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb234038580 0x7fb23403aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:01.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.694+0000 7fb245ffb700 1 -- 172.21.15.178:0/3295372083 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7fb23804c340 con 0x7fb2480fefa0 2024-09-10T11:22:01.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.694+0000 7fb247fff700 1 --2- 172.21.15.178:0/3295372083 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb234038580 0x7fb23403aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:01.696 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.694+0000 7fb247fff700 1 --2- 172.21.15.178:0/3295372083 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb234038580 0x7fb23403aa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb230006fd0 tx=0x7fb230006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:22:01.700 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.699+0000 7fb245ffb700 1 -- 172.21.15.178:0/3295372083 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb238011570 con 0x7fb2480fefa0 2024-09-10T11:22:01.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.982+0000 7fb24ecfe700 1 -- 172.21.15.178:0/3295372083 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fb248105cf0 con 0x7fb2480fefa0 2024-09-10T11:22:01.984 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.983+0000 7fb245ffb700 1 -- 172.21.15.178:0/3295372083 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fb238011390 con 0x7fb2480fefa0 2024-09-10T11:22:01.984 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:22:01.984 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:22:01.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.985+0000 7fb24ecfe700 1 -- 172.21.15.178:0/3295372083 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb234038580 msgr2=0x7fb23403aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:01.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.985+0000 7fb24ecfe700 1 --2- 172.21.15.178:0/3295372083 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb234038580 0x7fb23403aa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb230006fd0 tx=0x7fb230006e40 comp rx=0 tx=0).stop 2024-09-10T11:22:01.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.985+0000 7fb24ecfe700 1 -- 172.21.15.178:0/3295372083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2480fefa0 msgr2=0x7fb2481975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:01.986 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.985+0000 7fb24ecfe700 1 --2- 172.21.15.178:0/3295372083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2480fefa0 0x7fb2481975d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb2380062a0 tx=0x7fb238003ce0 comp rx=0 tx=0).stop 2024-09-10T11:22:01.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.985+0000 7fb24ecfe700 1 -- 172.21.15.178:0/3295372083 shutdown_connections 2024-09-10T11:22:01.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.985+0000 7fb24ecfe700 1 --2- 172.21.15.178:0/3295372083 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb234038580 0x7fb23403aa30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:01.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.985+0000 7fb24ecfe700 1 --2- 172.21.15.178:0/3295372083 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2480fefa0 0x7fb2481975d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:01.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.985+0000 7fb24ecfe700 1 -- 172.21.15.178:0/3295372083 >> 172.21.15.178:0/3295372083 conn(0x7fb2480fa860 msgr2=0x7fb248069080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:22:01.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.985+0000 7fb24ecfe700 1 -- 172.21.15.178:0/3295372083 shutdown_connections 2024-09-10T11:22:01.987 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:01.985+0000 7fb24ecfe700 1 -- 172.21.15.178:0/3295372083 wait complete. 2024-09-10T11:22:01.987 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:22:02.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:02 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/3295372083' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:03 smithi144 ceph-mon[27942]: Deploying daemon node-exporter.smithi178 on smithi178 2024-09-10T11:22:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:22:03.695 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:22:03.695 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:22:03.894 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:22:06.512 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.509+0000 7ff36029f700 1 -- 172.21.15.178:0/3027237804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3580ff020 msgr2=0x7ff3580ff3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.509+0000 7ff36029f700 1 --2- 172.21.15.178:0/3027237804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3580ff020 0x7ff3580ff3f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff354009910 tx=0x7ff354009c20 comp rx=0 tx=0).stop 2024-09-10T11:22:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.509+0000 7ff36029f700 1 -- 172.21.15.178:0/3027237804 shutdown_connections 2024-09-10T11:22:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.509+0000 7ff36029f700 1 --2- 172.21.15.178:0/3027237804 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3580ff020 0x7ff3580ff3f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.509+0000 7ff36029f700 1 -- 172.21.15.178:0/3027237804 >> 172.21.15.178:0/3027237804 conn(0x7ff358074af0 msgr2=0x7ff358074ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:22:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.510+0000 7ff36029f700 1 -- 172.21.15.178:0/3027237804 shutdown_connections 2024-09-10T11:22:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.510+0000 7ff36029f700 1 -- 172.21.15.178:0/3027237804 wait complete. 2024-09-10T11:22:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.510+0000 7ff36029f700 1 Processor -- start 2024-09-10T11:22:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.510+0000 7ff36029f700 1 -- start start 2024-09-10T11:22:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.510+0000 7ff36029f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3580ff020 0x7ff358196170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.510+0000 7ff36029f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3581966b0 con 0x7ff3580ff020 2024-09-10T11:22:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.511+0000 7ff35e03b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3580ff020 0x7ff358196170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:06.513 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.511+0000 7ff35e03b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3580ff020 0x7ff358196170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:50432/0 (socket says 172.21.15.178:50432) 2024-09-10T11:22:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.511+0000 7ff35e03b700 1 -- 172.21.15.178:0/2075161099 learned_addr learned my addr 172.21.15.178:0/2075161099 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:22:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.511+0000 7ff35e03b700 1 -- 172.21.15.178:0/2075161099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3540095f0 con 0x7ff3580ff020 2024-09-10T11:22:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.511+0000 7ff35e03b700 1 --2- 172.21.15.178:0/2075161099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3580ff020 0x7ff358196170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff354005b40 tx=0x7ff354004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:22:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.511+0000 7ff34f7fe700 1 -- 172.21.15.178:0/2075161099 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff35401d070 con 0x7ff3580ff020 2024-09-10T11:22:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.511+0000 7ff36029f700 1 -- 172.21.15.178:0/2075161099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3581968b0 con 0x7ff3580ff020 2024-09-10T11:22:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.511+0000 7ff34f7fe700 1 -- 172.21.15.178:0/2075161099 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff3540044d0 con 0x7ff3580ff020 2024-09-10T11:22:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.511+0000 7ff34f7fe700 1 -- 172.21.15.178:0/2075161099 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff354003930 con 0x7ff3580ff020 2024-09-10T11:22:06.514 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.511+0000 7ff36029f700 1 -- 172.21.15.178:0/2075161099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff358196c70 con 0x7ff3580ff020 2024-09-10T11:22:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.512+0000 7ff34f7fe700 1 -- 172.21.15.178:0/2075161099 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7ff354003ba0 con 0x7ff3580ff020 2024-09-10T11:22:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.512+0000 7ff36029f700 1 -- 172.21.15.178:0/2075161099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff358107250 con 0x7ff3580ff020 2024-09-10T11:22:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.513+0000 7ff34f7fe700 1 --2- 172.21.15.178:0/2075161099 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff3440385d0 0x7ff34403aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.513+0000 7ff34f7fe700 1 -- 172.21.15.178:0/2075161099 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7ff35404c2e0 con 0x7ff3580ff020 2024-09-10T11:22:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.513+0000 7ff35d83a700 1 --2- 172.21.15.178:0/2075161099 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff3440385d0 0x7ff34403aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:06.515 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.513+0000 7ff35d83a700 1 --2- 172.21.15.178:0/2075161099 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff3440385d0 0x7ff34403aa80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff348006fd0 tx=0x7ff348006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:22:06.519 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.518+0000 7ff34f7fe700 1 -- 172.21.15.178:0/2075161099 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff35402a540 con 0x7ff3580ff020 2024-09-10T11:22:06.793 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.792+0000 7ff36029f700 1 -- 172.21.15.178:0/2075161099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff358068610 con 0x7ff3580ff020 2024-09-10T11:22:06.794 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.792+0000 7ff34f7fe700 1 -- 172.21.15.178:0/2075161099 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7ff35402a360 con 0x7ff3580ff020 2024-09-10T11:22:06.794 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:22:06.794 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:22:06.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.795+0000 7ff36029f700 1 -- 172.21.15.178:0/2075161099 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff3440385d0 msgr2=0x7ff34403aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:06.796 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.795+0000 7ff36029f700 1 --2- 172.21.15.178:0/2075161099 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff3440385d0 0x7ff34403aa80 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff348006fd0 tx=0x7ff348006e40 comp rx=0 tx=0).stop 2024-09-10T11:22:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.795+0000 7ff36029f700 1 -- 172.21.15.178:0/2075161099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3580ff020 msgr2=0x7ff358196170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.795+0000 7ff36029f700 1 --2- 172.21.15.178:0/2075161099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3580ff020 0x7ff358196170 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ff354005b40 tx=0x7ff354004c80 comp rx=0 tx=0).stop 2024-09-10T11:22:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.795+0000 7ff36029f700 1 -- 172.21.15.178:0/2075161099 shutdown_connections 2024-09-10T11:22:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.795+0000 7ff36029f700 1 --2- 172.21.15.178:0/2075161099 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff3440385d0 0x7ff34403aa80 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.795+0000 7ff36029f700 1 --2- 172.21.15.178:0/2075161099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff3580ff020 0x7ff358196170 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.795+0000 7ff36029f700 1 -- 172.21.15.178:0/2075161099 >> 172.21.15.178:0/2075161099 conn(0x7ff358074af0 msgr2=0x7ff3581031b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:22:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.795+0000 7ff36029f700 1 -- 172.21.15.178:0/2075161099 shutdown_connections 2024-09-10T11:22:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:06.795+0000 7ff36029f700 1 -- 172.21.15.178:0/2075161099 wait complete. 2024-09-10T11:22:06.797 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:22:07.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:07.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:07.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:07.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:07.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi178.vmvxuf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:22:07.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi178.vmvxuf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T11:22:07.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:22:07.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:07.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:07 smithi144 ceph-mon[27942]: Deploying daemon mgr.smithi178.vmvxuf on smithi178 2024-09-10T11:22:07.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:07 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/2075161099' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:08.400 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:22:08.400 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:22:08.684 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:22:09.180 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:08 smithi144 ceph-mon[27942]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:09.180 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:08 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:10.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.954+0000 7f3db89f9700 1 -- 172.21.15.178:0/1414956210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3db40fefa0 msgr2=0x7f3db40ff370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:10.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.954+0000 7f3db89f9700 1 --2- 172.21.15.178:0/1414956210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3db40fefa0 0x7f3db40ff370 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3da4009910 tx=0x7f3da4009c20 comp rx=0 tx=0).stop 2024-09-10T11:22:10.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.954+0000 7f3db89f9700 1 -- 172.21.15.178:0/1414956210 shutdown_connections 2024-09-10T11:22:10.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.954+0000 7f3db89f9700 1 --2- 172.21.15.178:0/1414956210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3db40fefa0 0x7f3db40ff370 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:10.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.954+0000 7f3db89f9700 1 -- 172.21.15.178:0/1414956210 >> 172.21.15.178:0/1414956210 conn(0x7f3db40fa860 msgr2=0x7f3db40fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:22:10.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.954+0000 7f3db89f9700 1 -- 172.21.15.178:0/1414956210 shutdown_connections 2024-09-10T11:22:10.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.954+0000 7f3db89f9700 1 -- 172.21.15.178:0/1414956210 wait complete. 2024-09-10T11:22:10.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.955+0000 7f3db89f9700 1 Processor -- start 2024-09-10T11:22:10.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.955+0000 7f3db89f9700 1 -- start start 2024-09-10T11:22:10.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.955+0000 7f3db89f9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3db40fefa0 0x7f3db4195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:10.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.955+0000 7f3db89f9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db4195880 con 0x7f3db40fefa0 2024-09-10T11:22:10.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.955+0000 7f3db259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3db40fefa0 0x7f3db4195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:10.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.955+0000 7f3db259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3db40fefa0 0x7f3db4195340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:50470/0 (socket says 172.21.15.178:50470) 2024-09-10T11:22:10.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.955+0000 7f3db259c700 1 -- 172.21.15.178:0/1416643085 learned_addr learned my addr 172.21.15.178:0/1416643085 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:22:10.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.956+0000 7f3db259c700 1 -- 172.21.15.178:0/1416643085 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da40095f0 con 0x7f3db40fefa0 2024-09-10T11:22:10.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.956+0000 7f3db259c700 1 --2- 172.21.15.178:0/1416643085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3db40fefa0 0x7f3db4195340 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3da4005b40 tx=0x7f3da4004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:22:10.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.956+0000 7f3da37fe700 1 -- 172.21.15.178:0/1416643085 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3da401d070 con 0x7f3db40fefa0 2024-09-10T11:22:10.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.956+0000 7f3db89f9700 1 -- 172.21.15.178:0/1416643085 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db4195a80 con 0x7f3db40fefa0 2024-09-10T11:22:10.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.956+0000 7f3da37fe700 1 -- 172.21.15.178:0/1416643085 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3da40044d0 con 0x7f3db40fefa0 2024-09-10T11:22:10.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.956+0000 7f3da37fe700 1 -- 172.21.15.178:0/1416643085 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3da4003930 con 0x7f3db40fefa0 2024-09-10T11:22:10.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.956+0000 7f3db89f9700 1 -- 172.21.15.178:0/1416643085 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db4195e40 con 0x7f3db40fefa0 2024-09-10T11:22:10.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.957+0000 7f3da37fe700 1 -- 172.21.15.178:0/1416643085 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7f3da4003bf0 con 0x7f3db40fefa0 2024-09-10T11:22:10.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.957+0000 7f3db89f9700 1 -- 172.21.15.178:0/1416643085 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db4062380 con 0x7f3db40fefa0 2024-09-10T11:22:10.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.957+0000 7f3da37fe700 1 --2- 172.21.15.178:0/1416643085 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3d9c038580 0x7f3d9c03aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:10.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.957+0000 7f3da37fe700 1 -- 172.21.15.178:0/1416643085 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7f3da404c440 con 0x7f3db40fefa0 2024-09-10T11:22:10.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.958+0000 7f3db1d9b700 1 --2- 172.21.15.178:0/1416643085 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3d9c038580 0x7f3d9c03aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:10.960 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.958+0000 7f3db1d9b700 1 --2- 172.21.15.178:0/1416643085 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3d9c038580 0x7f3d9c03aa30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3da8006fd0 tx=0x7f3da8006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:22:10.964 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:10.963+0000 7f3da37fe700 1 -- 172.21.15.178:0/1416643085 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3da402a950 con 0x7f3db40fefa0 2024-09-10T11:22:11.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:11.242+0000 7f3db89f9700 1 -- 172.21.15.178:0/1416643085 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3db41966a0 con 0x7f3db40fefa0 2024-09-10T11:22:11.243 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:11.242+0000 7f3da37fe700 1 -- 172.21.15.178:0/1416643085 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7f3da402a360 con 0x7f3db40fefa0 2024-09-10T11:22:11.244 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:22:11.244 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:22:11.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:11.244+0000 7f3db89f9700 1 -- 172.21.15.178:0/1416643085 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3d9c038580 msgr2=0x7f3d9c03aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:11.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:11.245+0000 7f3db89f9700 1 --2- 172.21.15.178:0/1416643085 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3d9c038580 0x7f3d9c03aa30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3da8006fd0 tx=0x7f3da8006e40 comp rx=0 tx=0).stop 2024-09-10T11:22:11.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:11.245+0000 7f3db89f9700 1 -- 172.21.15.178:0/1416643085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3db40fefa0 msgr2=0x7f3db4195340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:11.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:11.245+0000 7f3db89f9700 1 --2- 172.21.15.178:0/1416643085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3db40fefa0 0x7f3db4195340 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3da4005b40 tx=0x7f3da4004c80 comp rx=0 tx=0).stop 2024-09-10T11:22:11.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:11.245+0000 7f3db89f9700 1 -- 172.21.15.178:0/1416643085 shutdown_connections 2024-09-10T11:22:11.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:11.245+0000 7f3db89f9700 1 --2- 172.21.15.178:0/1416643085 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3d9c038580 0x7f3d9c03aa30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:11.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:11.245+0000 7f3db89f9700 1 --2- 172.21.15.178:0/1416643085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3db40fefa0 0x7f3db4195340 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:11.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:11.245+0000 7f3db89f9700 1 -- 172.21.15.178:0/1416643085 >> 172.21.15.178:0/1416643085 conn(0x7f3db40fa860 msgr2=0x7f3db40687f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:22:11.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:11.245+0000 7f3db89f9700 1 -- 172.21.15.178:0/1416643085 shutdown_connections 2024-09-10T11:22:11.246 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:11.245+0000 7f3db89f9700 1 -- 172.21.15.178:0/1416643085 wait complete. 2024-09-10T11:22:11.247 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:22:11.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:10 smithi144 ceph-mon[27942]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:11.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:10 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:11.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:10 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:11.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:10 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:11.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:10 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:11.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:10 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T11:22:11.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:10 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:11.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:10 smithi144 ceph-mon[27942]: Deploying daemon mon.smithi178 on smithi178 2024-09-10T11:22:12.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:12 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/1416643085' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:13.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:13 smithi144 ceph-mon[27942]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:13.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:13 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:13.483 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:22:13.483 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:22:13.761 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:22:15.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.432+0000 7fe7a6d26700 1 -- 172.21.15.178:0/3899881707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7a01010f0 msgr2=0x7fe7a01014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:15.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.432+0000 7fe7a6d26700 1 --2- 172.21.15.178:0/3899881707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7a01010f0 0x7fe7a01014c0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe794009910 tx=0x7fe794009c20 comp rx=0 tx=0).stop 2024-09-10T11:22:15.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.432+0000 7fe7a6d26700 1 -- 172.21.15.178:0/3899881707 shutdown_connections 2024-09-10T11:22:15.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.432+0000 7fe7a6d26700 1 --2- 172.21.15.178:0/3899881707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7a01010f0 0x7fe7a01014c0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:15.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.432+0000 7fe7a6d26700 1 -- 172.21.15.178:0/3899881707 >> 172.21.15.178:0/3899881707 conn(0x7fe7a0074c10 msgr2=0x7fe7a0075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:22:15.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.432+0000 7fe7a6d26700 1 -- 172.21.15.178:0/3899881707 shutdown_connections 2024-09-10T11:22:15.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.432+0000 7fe7a6d26700 1 -- 172.21.15.178:0/3899881707 wait complete. 2024-09-10T11:22:15.434 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.433+0000 7fe7a6d26700 1 Processor -- start 2024-09-10T11:22:15.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.433+0000 7fe7a6d26700 1 -- start start 2024-09-10T11:22:15.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.433+0000 7fe7a6d26700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7a01010f0 0x7fe7a00fef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:15.435 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.433+0000 7fe7a6d26700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a0100ea0 con 0x7fe7a01010f0 2024-09-10T11:22:15.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:15 smithi144 ceph-mon[27942]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:15.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.433+0000 7fe7a4ac2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7a01010f0 0x7fe7a00fef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:15.436 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.433+0000 7fe7a4ac2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7a01010f0 0x7fe7a00fef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.178:59148/0 (socket says 172.21.15.178:59148) 2024-09-10T11:22:15.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.433+0000 7fe7a4ac2700 1 -- 172.21.15.178:0/3640862237 learned_addr learned my addr 172.21.15.178:0/3640862237 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:22:15.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.434+0000 7fe7a4ac2700 1 -- 172.21.15.178:0/3640862237 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7940095f0 con 0x7fe7a01010f0 2024-09-10T11:22:15.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.434+0000 7fe7a4ac2700 1 --2- 172.21.15.178:0/3640862237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7a01010f0 0x7fe7a00fef20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe794005b40 tx=0x7fe794004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:22:15.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.434+0000 7fe79dffb700 1 -- 172.21.15.178:0/3640862237 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe79401d070 con 0x7fe7a01010f0 2024-09-10T11:22:15.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.434+0000 7fe79dffb700 1 -- 172.21.15.178:0/3640862237 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe7940044d0 con 0x7fe7a01010f0 2024-09-10T11:22:15.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.434+0000 7fe7a6d26700 1 -- 172.21.15.178:0/3640862237 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a00ff460 con 0x7fe7a01010f0 2024-09-10T11:22:15.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.434+0000 7fe79dffb700 1 -- 172.21.15.178:0/3640862237 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe794003930 con 0x7fe7a01010f0 2024-09-10T11:22:15.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.434+0000 7fe7a6d26700 1 -- 172.21.15.178:0/3640862237 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a00ff8e0 con 0x7fe7a01010f0 2024-09-10T11:22:15.437 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.435+0000 7fe7a6d26700 1 -- 172.21.15.178:0/3640862237 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a01077d0 con 0x7fe7a01010f0 2024-09-10T11:22:15.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.436+0000 7fe79dffb700 1 -- 172.21.15.178:0/3640862237 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 17) v1 ==== 45199+0+0 (secure 0 0 0) 0x7fe79400b960 con 0x7fe7a01010f0 2024-09-10T11:22:15.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.436+0000 7fe79dffb700 1 --2- 172.21.15.178:0/3640862237 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe7900385d0 0x7fe79003aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:15.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.436+0000 7fe79dffb700 1 -- 172.21.15.178:0/3640862237 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7fe79404d0b0 con 0x7fe7a01010f0 2024-09-10T11:22:15.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.437+0000 7fe79ffff700 1 --2- 172.21.15.178:0/3640862237 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe7900385d0 0x7fe79003aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:15.438 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.437+0000 7fe79ffff700 1 --2- 172.21.15.178:0/3640862237 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe7900385d0 0x7fe79003aa80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe78c006fd0 tx=0x7fe78c006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:22:15.442 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.441+0000 7fe79dffb700 1 -- 172.21.15.178:0/3640862237 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe79402b7a0 con 0x7fe7a01010f0 2024-09-10T11:22:15.732 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.730+0000 7fe7a6d26700 1 -- 172.21.15.178:0/3640862237 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fe7a00ffbe0 con 0x7fe7a01010f0 2024-09-10T11:22:15.747 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.746+0000 7fe79dffb700 1 -- 172.21.15.178:0/3640862237 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+748 (secure 0 0 0) 0x7fe79402b3e0 con 0x7fe7a01010f0 2024-09-10T11:22:15.748 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:22:15.748 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":1,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:19:28.938528Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T11:22:15.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.748+0000 7fe7a6d26700 1 -- 172.21.15.178:0/3640862237 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe7900385d0 msgr2=0x7fe79003aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:15.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.748+0000 7fe7a6d26700 1 --2- 172.21.15.178:0/3640862237 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe7900385d0 0x7fe79003aa80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe78c006fd0 tx=0x7fe78c006e40 comp rx=0 tx=0).stop 2024-09-10T11:22:15.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.748+0000 7fe7a6d26700 1 -- 172.21.15.178:0/3640862237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7a01010f0 msgr2=0x7fe7a00fef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:15.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.748+0000 7fe7a6d26700 1 --2- 172.21.15.178:0/3640862237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7a01010f0 0x7fe7a00fef20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe794005b40 tx=0x7fe794004c80 comp rx=0 tx=0).stop 2024-09-10T11:22:15.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.748+0000 7fe7a6d26700 1 -- 172.21.15.178:0/3640862237 shutdown_connections 2024-09-10T11:22:15.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.748+0000 7fe7a6d26700 1 --2- 172.21.15.178:0/3640862237 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe7900385d0 0x7fe79003aa80 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:15.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.748+0000 7fe7a6d26700 1 --2- 172.21.15.178:0/3640862237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7a01010f0 0x7fe7a00fef20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:15.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.748+0000 7fe7a6d26700 1 -- 172.21.15.178:0/3640862237 >> 172.21.15.178:0/3640862237 conn(0x7fe7a0074c10 msgr2=0x7fe7a0104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:22:15.750 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.749+0000 7fe7a6d26700 1 -- 172.21.15.178:0/3640862237 shutdown_connections 2024-09-10T11:22:15.751 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:15.749+0000 7fe7a6d26700 1 -- 172.21.15.178:0/3640862237 wait complete. 2024-09-10T11:22:15.751 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 1 2024-09-10T11:22:15.865 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 podman[33480]: 2024-09-10 11:22:15.84157611 +0000 UTC m=+0.358230677 container start 3c1ba210ed3fb6c058a87286198a1cc8c1ce886d2ef5bce99e2c94cce123e6a2 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20231212, RELEASE=HEAD, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True) 2024-09-10T11:22:15.865 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T11:22:16.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable), process ceph-mon, pid 2 2024-09-10T11:22:16.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pidfile_write: ignore empty --pid-file 2024-09-10T11:22:16.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: load: jerasure load: lrc 2024-09-10T11:22:16.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: RocksDB version: 7.9.2 2024-09-10T11:22:16.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Git sha 0 2024-09-10T11:22:16.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Compile date 2023-08-03 19:21:13 2024-09-10T11:22:16.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: DB SUMMARY 2024-09-10T11:22:16.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: DB Session ID: QUKH0J33QW4YH8VHN8AF 2024-09-10T11:22:16.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: CURRENT file: CURRENT 2024-09-10T11:22:16.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T11:22:16.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-10T11:22:16.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi178/store.db dir, Total Num: 0, files: 2024-09-10T11:22:16.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi178/store.db: 000004.log size: 511 ; 2024-09-10T11:22:16.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.error_if_exists: 0 2024-09-10T11:22:16.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.create_if_missing: 0 2024-09-10T11:22:16.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.paranoid_checks: 1 2024-09-10T11:22:16.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T11:22:16.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T11:22:16.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T11:22:16.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.env: 0x557b778b6720 2024-09-10T11:22:16.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T11:22:16.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.info_log: 0x557b7a72f2c0 2024-09-10T11:22:16.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T11:22:16.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.statistics: (nil) 2024-09-10T11:22:16.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.use_fsync: 0 2024-09-10T11:22:16.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_log_file_size: 0 2024-09-10T11:22:16.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T11:22:16.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T11:22:16.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T11:22:16.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T11:22:16.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.allow_fallocate: 1 2024-09-10T11:22:16.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T11:22:16.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T11:22:16.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.use_direct_reads: 0 2024-09-10T11:22:16.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T11:22:16.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T11:22:16.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.db_log_dir: 2024-09-10T11:22:16.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.wal_dir: 2024-09-10T11:22:16.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T11:22:16.121 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T11:22:16.121 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T11:22:16.121 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T11:22:16.121 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T11:22:16.121 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T11:22:16.121 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T11:22:16.121 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T11:22:16.121 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.write_buffer_manager: 0x557b799a03c0 2024-09-10T11:22:16.121 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T11:22:16.121 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T11:22:16.121 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T11:22:16.121 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T11:22:16.121 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T11:22:16.121 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T11:22:16.122 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T11:22:16.122 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T11:22:16.122 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.unordered_write: 0 2024-09-10T11:22:16.122 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T11:22:16.122 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T11:22:16.122 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T11:22:16.122 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T11:22:16.122 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.row_cache: None 2024-09-10T11:22:16.122 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.wal_filter: None 2024-09-10T11:22:16.122 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T11:22:16.122 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T11:22:16.122 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.two_write_queues: 0 2024-09-10T11:22:16.122 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T11:22:16.123 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.wal_compression: 0 2024-09-10T11:22:16.123 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.atomic_flush: 0 2024-09-10T11:22:16.123 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T11:22:16.123 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T11:22:16.123 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T11:22:16.123 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.log_readahead_size: 0 2024-09-10T11:22:16.123 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T11:22:16.123 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T11:22:16.123 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T11:22:16.123 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T11:22:16.123 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T11:22:16.123 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T11:22:16.123 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T11:22:16.125 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_background_jobs: 2 2024-09-10T11:22:16.126 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_background_compactions: -1 2024-09-10T11:22:16.126 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_subcompactions: 1 2024-09-10T11:22:16.126 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T11:22:16.126 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T11:22:16.126 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T11:22:16.126 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T11:22:16.126 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T11:22:16.126 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T11:22:16.126 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T11:22:16.126 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T11:22:16.126 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_open_files: -1 2024-09-10T11:22:16.126 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T11:22:16.126 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T11:22:16.126 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T11:22:16.127 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T11:22:16.127 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_background_flushes: -1 2024-09-10T11:22:16.127 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Compression algorithms supported: 2024-09-10T11:22:16.127 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T11:22:16.127 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: kZSTD supported: 0 2024-09-10T11:22:16.127 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: kXpressCompression supported: 0 2024-09-10T11:22:16.127 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T11:22:16.127 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: kZlibCompression supported: 1 2024-09-10T11:22:16.127 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: kSnappyCompression supported: 1 2024-09-10T11:22:16.127 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: kLZ4Compression supported: 1 2024-09-10T11:22:16.127 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: kBZip2Compression supported: 0 2024-09-10T11:22:16.127 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T11:22:16.127 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T11:22:16.128 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi178/store.db/MANIFEST-000005 2024-09-10T11:22:16.128 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T11:22:16.128 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T11:22:16.128 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.merge_operator: 2024-09-10T11:22:16.128 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compaction_filter: None 2024-09-10T11:22:16.128 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T11:22:16.128 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T11:22:16.128 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T11:22:16.128 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T11:22:16.128 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557b7a72f3c0) 2024-09-10T11:22:16.128 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: cache_index_and_filter_blocks: 1 2024-09-10T11:22:16.128 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T11:22:16.128 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T11:22:16.128 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: pin_top_level_index_and_filter: 1 2024-09-10T11:22:16.129 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: index_type: 0 2024-09-10T11:22:16.129 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: data_block_index_type: 0 2024-09-10T11:22:16.129 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: index_shortening: 1 2024-09-10T11:22:16.129 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T11:22:16.129 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: checksum: 4 2024-09-10T11:22:16.129 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: no_block_cache: 0 2024-09-10T11:22:16.129 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_cache: 0x557b79a411f0 2024-09-10T11:22:16.129 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_cache_name: BinnedLRUCache 2024-09-10T11:22:16.129 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_cache_options: 2024-09-10T11:22:16.129 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: capacity : 536870912 2024-09-10T11:22:16.129 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: num_shard_bits : 4 2024-09-10T11:22:16.129 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: strict_capacity_limit : 0 2024-09-10T11:22:16.129 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: high_pri_pool_ratio: 0.000 2024-09-10T11:22:16.129 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_cache_compressed: (nil) 2024-09-10T11:22:16.130 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: persistent_cache: (nil) 2024-09-10T11:22:16.130 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_size: 4096 2024-09-10T11:22:16.130 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_size_deviation: 10 2024-09-10T11:22:16.130 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_restart_interval: 16 2024-09-10T11:22:16.130 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: index_block_restart_interval: 1 2024-09-10T11:22:16.130 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: metadata_block_size: 4096 2024-09-10T11:22:16.130 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: partition_filters: 0 2024-09-10T11:22:16.130 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: use_delta_encoding: 1 2024-09-10T11:22:16.130 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: filter_policy: bloomfilter 2024-09-10T11:22:16.130 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: whole_key_filtering: 1 2024-09-10T11:22:16.130 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: verify_compression: 0 2024-09-10T11:22:16.130 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: read_amp_bytes_per_bit: 0 2024-09-10T11:22:16.130 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: format_version: 5 2024-09-10T11:22:16.130 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: enable_index_compression: 1 2024-09-10T11:22:16.131 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_align: 0 2024-09-10T11:22:16.131 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: max_auto_readahead_size: 262144 2024-09-10T11:22:16.131 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: prepopulate_block_cache: 0 2024-09-10T11:22:16.131 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: initial_auto_readahead_size: 8192 2024-09-10T11:22:16.131 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T11:22:16.131 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T11:22:16.131 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T11:22:16.131 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compression: NoCompression 2024-09-10T11:22:16.131 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T11:22:16.131 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T11:22:16.131 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T11:22:16.131 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.num_levels: 7 2024-09-10T11:22:16.131 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T11:22:16.131 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T11:22:16.131 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T11:22:16.132 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T11:22:16.132 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T11:22:16.132 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T11:22:16.132 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T11:22:16.132 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T11:22:16.132 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T11:22:16.132 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T11:22:16.132 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T11:22:16.132 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T11:22:16.132 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T11:22:16.132 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T11:22:16.132 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T11:22:16.132 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T11:22:16.133 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T11:22:16.133 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T11:22:16.133 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T11:22:16.133 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T11:22:16.133 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T11:22:16.133 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T11:22:16.133 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T11:22:16.133 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T11:22:16.133 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T11:22:16.133 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T11:22:16.133 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T11:22:16.133 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T11:22:16.133 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T11:22:16.133 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T11:22:16.133 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T11:22:16.134 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T11:22:16.134 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T11:22:16.134 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T11:22:16.134 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T11:22:16.134 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T11:22:16.134 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T11:22:16.134 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T11:22:16.134 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T11:22:16.134 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T11:22:16.134 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T11:22:16.134 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T11:22:16.134 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T11:22:16.134 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T11:22:16.134 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T11:22:16.135 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T11:22:16.135 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T11:22:16.135 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T11:22:16.135 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T11:22:16.135 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T11:22:16.135 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T11:22:16.135 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T11:22:16.135 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T11:22:16.135 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.table_properties_collectors: 2024-09-10T11:22:16.135 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.inplace_update_support: 0 2024-09-10T11:22:16.135 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T11:22:16.135 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T11:22:16.135 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T11:22:16.135 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T11:22:16.136 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.bloom_locality: 0 2024-09-10T11:22:16.136 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.max_successive_merges: 0 2024-09-10T11:22:16.136 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T11:22:16.136 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T11:22:16.136 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T11:22:16.136 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T11:22:16.136 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.ttl: 2592000 2024-09-10T11:22:16.136 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T11:22:16.136 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T11:22:16.136 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T11:22:16.136 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.enable_blob_files: false 2024-09-10T11:22:16.136 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.min_blob_size: 0 2024-09-10T11:22:16.136 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T11:22:16.137 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T11:22:16.137 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T11:22:16.137 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T11:22:16.137 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T11:22:16.137 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T11:22:16.137 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T11:22:16.137 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T11:22:16.137 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi178/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T11:22:16.137 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-10T11:22:16.137 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 97046055-eaba-4a4e-bd53-c95f3747ca37 2024-09-10T11:22:16.137 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725967335870956, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T11:22:16.137 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-10T11:22:16.137 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725967335871563, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725967335, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "97046055-eaba-4a4e-bd53-c95f3747ca37", "db_session_id": "QUKH0J33QW4YH8VHN8AF", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-10T11:22:16.137 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725967335871669, "job": 1, "event": "recovery_finished"} 2024-09-10T11:22:16.138 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-10T11:22:16.138 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi178/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T11:22:16.138 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x557b79ade000 2024-09-10T11:22:16.138 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: DB pointer 0x557b79aca000 2024-09-10T11:22:16.138 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T11:22:16.138 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T11:22:16.138 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: ** DB Stats ** 2024-09-10T11:22:16.138 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T11:22:16.138 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T11:22:16.138 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T11:22:16.138 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T11:22:16.138 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T11:22:16.138 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T11:22:16.138 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T11:22:16.139 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: 2024-09-10T11:22:16.139 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: ** Compaction Stats [default] ** 2024-09-10T11:22:16.139 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T11:22:16.139 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T11:22:16.139 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T11:22:16.139 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T11:22:16.139 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T11:22:16.139 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: 2024-09-10T11:22:16.139 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: ** Compaction Stats [default] ** 2024-09-10T11:22:16.139 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T11:22:16.139 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T11:22:16.139 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-10T11:22:16.139 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: 2024-09-10T11:22:16.140 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T11:22:16.140 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: 2024-09-10T11:22:16.140 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T11:22:16.140 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T11:22:16.140 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T11:22:16.140 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T11:22:16.140 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T11:22:16.140 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T11:22:16.140 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Cumulative compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T11:22:16.140 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Interval compaction: 0.00 GB write, 0.43 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T11:22:16.140 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T11:22:16.140 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Block cache BinnedLRUCache@0x557b79a411f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-10T11:22:16.140 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-10T11:22:16.140 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: 2024-09-10T11:22:16.141 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T11:22:16.141 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178 does not exist in monmap, will attempt to join an existing cluster 2024-09-10T11:22:16.141 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: using public_addr v2:172.21.15.178:0/0 -> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-10T11:22:16.141 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: starting mon.smithi178 rank -1 at public addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] at bind addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi178 fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:22:16.141 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(???) e0 preinit fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:22:16.141 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(synchronizing).mds e1 new map 2024-09-10T11:22:16.141 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(synchronizing).mds e1 print_map 2024-09-10T11:22:16.141 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: e1 2024-09-10T11:22:16.141 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:22:16.141 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:22:16.141 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: legacy client fscid: -1 2024-09-10T11:22:16.141 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: 2024-09-10T11:22:16.141 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: No filesystems configured 2024-09-10T11:22:16.142 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T11:22:16.142 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T11:22:16.142 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T11:22:16.142 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T11:22:16.142 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T11:22:16.142 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T11:22:16.142 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2024-09-10T11:22:16.142 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T11:22:16.142 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T11:22:16.142 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T11:22:16.142 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T11:22:16.142 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:22:16.142 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Deploying cephadm binary to smithi178 2024-09-10T11:22:16.142 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.143 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.143 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Added host smithi178 2024-09-10T11:22:16.143 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.143 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.143 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.143 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.143 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.143 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.143 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Deploying daemon alertmanager.smithi144 on smithi144 2024-09-10T11:22:16.143 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.143 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T11:22:16.143 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.143 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.143 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/1659915210' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T11:22:16.144 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/1659915210' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T11:22:16.144 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T11:22:16.144 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.144 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi144:172.21.15.144=smithi144;smithi178:172.21.15.178=smithi178", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:22:16.144 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Saving service mon spec with placement smithi144:172.21.15.144=smithi144;smithi178:172.21.15.178=smithi178;count:2 2024-09-10T11:22:16.144 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.144 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.144 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.144 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.144 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.144 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.144 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Regenerating cephadm self-signed grafana TLS certificates 2024-09-10T11:22:16.144 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.145 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.145 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T11:22:16.145 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T11:22:16.145 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.145 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Deploying daemon grafana.smithi144 on smithi144 2024-09-10T11:22:16.145 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/3718207768' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.145 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.145 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.145 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/529591044' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.145 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.145 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.145 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.145 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/2493248445' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.145 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Deploying daemon prometheus.smithi144 on smithi144 2024-09-10T11:22:16.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/2639311148' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/372428161' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/2318246203' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.147 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.147 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.147 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/923127561' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.147 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.147 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.147 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.147 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T11:22:16.147 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.147 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14164 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T11:22:16.147 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mgrmap e14: smithi144.bkogfs(active, since 58s) 2024-09-10T11:22:16.147 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/577627654' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.147 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/1463103717' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.147 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Active manager daemon smithi144.bkogfs restarted 2024-09-10T11:22:16.148 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Activating manager daemon smithi144.bkogfs 2024-09-10T11:22:16.148 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: osdmap e5: 0 total, 0 up, 0 in 2024-09-10T11:22:16.148 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mgrmap e15: smithi144.bkogfs(active, starting, since 0.00545597s) 2024-09-10T11:22:16.148 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:22:16.148 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr metadata", "who": "smithi144.bkogfs", "id": "smithi144.bkogfs"}]: dispatch 2024-09-10T11:22:16.148 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T11:22:16.148 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T11:22:16.148 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T11:22:16.148 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Manager daemon smithi144.bkogfs is now available 2024-09-10T11:22:16.148 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.148 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.148 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.148 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:22:16.149 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/mirror_snapshot_schedule"}]: dispatch 2024-09-10T11:22:16.149 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/trash_purge_schedule"}]: dispatch 2024-09-10T11:22:16.149 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: [10/Sep/2024:11:21:48] ENGINE Bus STARTING 2024-09-10T11:22:16.149 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: [10/Sep/2024:11:21:48] ENGINE Serving on https://172.21.15.144:7150 2024-09-10T11:22:16.149 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: [10/Sep/2024:11:21:48] ENGINE Serving on http://172.21.15.144:8765 2024-09-10T11:22:16.149 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: [10/Sep/2024:11:21:48] ENGINE Bus STARTED 2024-09-10T11:22:16.149 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.149 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mgrmap e16: smithi144.bkogfs(active, since 1.0054s) 2024-09-10T11:22:16.149 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.149 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/1973803493' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.149 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.149 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mgrmap e17: smithi144.bkogfs(active, since 2s) 2024-09-10T11:22:16.149 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.149 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.150 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.150 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.150 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd/host:smithi144", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:22:16.150 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.150 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.150 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.150 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.150 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd/host:smithi178", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:22:16.150 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:16.150 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:22:16.150 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Updating smithi144:/etc/ceph/ceph.conf 2024-09-10T11:22:16.150 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T11:22:16.150 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/1919779747' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.150 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:22:16.150 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:22:16.151 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:22:16.151 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:22:16.151 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:22:16.151 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:22:16.151 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.151 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.151 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.151 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.151 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.151 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi178", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T11:22:16.151 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi178", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-10T11:22:16.151 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:16.151 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Deploying daemon ceph-exporter.smithi178 on smithi178 2024-09-10T11:22:16.151 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/1623968212' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.152 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.152 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.152 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.152 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.152 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi178", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T11:22:16.152 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi178", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-09-10T11:22:16.152 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:16.152 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Deploying daemon crash.smithi178 on smithi178 2024-09-10T11:22:16.152 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/3295372083' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.152 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.152 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.152 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.152 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.152 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Deploying daemon node-exporter.smithi178 on smithi178 2024-09-10T11:22:16.153 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.153 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:22:16.153 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.153 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.153 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.153 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.153 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi178.vmvxuf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:22:16.153 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi178.vmvxuf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T11:22:16.153 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:22:16.153 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:16.153 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Deploying daemon mgr.smithi178.vmvxuf on smithi178 2024-09-10T11:22:16.153 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/2075161099' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.153 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.153 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.154 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.154 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.154 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.154 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.154 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.154 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T11:22:16.154 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:16.154 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: Deploying daemon mon.smithi178 on smithi178 2024-09-10T11:22:16.154 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/1416643085' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:16.154 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.154 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:16.155 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:16.155 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(synchronizing).paxosservice(auth 1..8) refresh upgraded, format 0 -> 3 2024-09-10T11:22:16.155 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: expand_channel_meta expand map: {default=false} 2024-09-10T11:22:16.155 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: expand_channel_meta from 'false' to 'false' 2024-09-10T11:22:16.155 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: expand_channel_meta expanded map: {default=false} 2024-09-10T11:22:16.155 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: expand_channel_meta expand map: {default=info} 2024-09-10T11:22:16.155 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: expand_channel_meta from 'info' to 'info' 2024-09-10T11:22:16.155 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: expand_channel_meta expanded map: {default=info} 2024-09-10T11:22:16.155 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: expand_channel_meta expand map: {default=daemon} 2024-09-10T11:22:16.155 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: expand_channel_meta from 'daemon' to 'daemon' 2024-09-10T11:22:16.155 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: expand_channel_meta expanded map: {default=daemon} 2024-09-10T11:22:16.155 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: expand_channel_meta expand map: {default=debug} 2024-09-10T11:22:16.155 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: expand_channel_meta from 'debug' to 'debug' 2024-09-10T11:22:16.156 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: expand_channel_meta expanded map: {default=debug} 2024-09-10T11:22:16.156 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 ceph-mon[33514]: mon.smithi178@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete,mon_cluster_log_to_file 2024-09-10T11:22:16.156 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 bash[33480]: 3c1ba210ed3fb6c058a87286198a1cc8c1ce886d2ef5bce99e2c94cce123e6a2 2024-09-10T11:22:16.156 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:15 smithi178 systemd[1]: Started Ceph mon.smithi178 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:22:17.555 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-09-10T11:22:17.555 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T11:22:17.751 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:22:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:22:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: mon.smithi144 calling monitor election 2024-09-10T11:22:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:22:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/2240634365' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/crt"}]: dispatch 2024-09-10T11:22:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:22:21.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:21.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:22:21.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: mon.smithi178 calling monitor election 2024-09-10T11:22:21.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:22:21.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:22:21.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:21.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:22:21.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:22:21.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: mon.smithi144 is new leader, mons smithi144,smithi178 in quorum (ranks 0,1) 2024-09-10T11:22:21.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: monmap e2: 2 mons at {smithi144=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0],smithi178=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0]} removed_ranks: {} 2024-09-10T11:22:21.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: fsmap 2024-09-10T11:22:21.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: osdmap e5: 0 total, 0 up, 0 in 2024-09-10T11:22:21.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: mgrmap e17: smithi144.bkogfs(active, since 33s) 2024-09-10T11:22:21.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: Standby manager daemon smithi178.vmvxuf started 2024-09-10T11:22:21.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/2240634365' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T11:22:21.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: overall HEALTH_OK 2024-09-10T11:22:21.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/2240634365' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/key"}]: dispatch 2024-09-10T11:22:21.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/2240634365' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T11:22:21.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:21.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:21.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:21.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:21.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:21.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:21.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:22:21.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: mon.smithi144 calling monitor election 2024-09-10T11:22:21.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:22:21.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/2240634365' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/crt"}]: dispatch 2024-09-10T11:22:21.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:22:21.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:21.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:22:21.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: mon.smithi178 calling monitor election 2024-09-10T11:22:21.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:22:21.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:22:21.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:21.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:22:21.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:22:21.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: mon.smithi144 is new leader, mons smithi144,smithi178 in quorum (ranks 0,1) 2024-09-10T11:22:21.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: monmap e2: 2 mons at {smithi144=[v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0],smithi178=[v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0]} removed_ranks: {} 2024-09-10T11:22:21.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: fsmap 2024-09-10T11:22:21.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: osdmap e5: 0 total, 0 up, 0 in 2024-09-10T11:22:21.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: mgrmap e17: smithi144.bkogfs(active, since 33s) 2024-09-10T11:22:21.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: Standby manager daemon smithi178.vmvxuf started 2024-09-10T11:22:21.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/2240634365' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T11:22:21.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: overall HEALTH_OK 2024-09-10T11:22:21.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/2240634365' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/key"}]: dispatch 2024-09-10T11:22:21.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/2240634365' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T11:22:21.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:21.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:21.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:21.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:21.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:22.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.135+0000 7f22fc8f6700 1 -- 172.21.15.178:0/2031713055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e00035c0 msgr2=0x7f22e0005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:22.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.135+0000 7f22fc8f6700 1 --2- 172.21.15.178:0/2031713055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e00035c0 0x7f22e0005a40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f22f80ffea0 tx=0x7f22ec00feb0 comp rx=0 tx=0).stop 2024-09-10T11:22:22.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.135+0000 7f22fc8f6700 1 -- 172.21.15.178:0/2031713055 shutdown_connections 2024-09-10T11:22:22.136 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.135+0000 7f22fc8f6700 1 --2- 172.21.15.178:0/2031713055 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e00035c0 0x7f22e0005a40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:22.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.135+0000 7f22fc8f6700 1 -- 172.21.15.178:0/2031713055 >> 172.21.15.178:0/2031713055 conn(0x7f22f80fba50 msgr2=0x7f22f80fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:22:22.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.135+0000 7f22fc8f6700 1 -- 172.21.15.178:0/2031713055 shutdown_connections 2024-09-10T11:22:22.137 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.135+0000 7f22fc8f6700 1 -- 172.21.15.178:0/2031713055 wait complete. 2024-09-10T11:22:22.138 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22fc8f6700 1 Processor -- start 2024-09-10T11:22:22.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22fc8f6700 1 -- start start 2024-09-10T11:22:22.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22fc8f6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e00035c0 0x7f22f8195530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:22.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22fc8f6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22f8195a70 0x7f22f8199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:22.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22fc8f6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f8196080 con 0x7f22e00035c0 2024-09-10T11:22:22.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22fc8f6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f81961f0 con 0x7f22f8195a70 2024-09-10T11:22:22.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22f5d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22f8195a70 0x7f22f8199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:22.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22f5d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22f8195a70 0x7f22f8199ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:38800/0 (socket says 172.21.15.178:38800) 2024-09-10T11:22:22.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22f5d9b700 1 -- 172.21.15.178:0/2142111363 learned_addr learned my addr 172.21.15.178:0/2142111363 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:22:22.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22f659c700 1 --2- 172.21.15.178:0/2142111363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e00035c0 0x7f22f8195530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:22.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22f5d9b700 1 -- 172.21.15.178:0/2142111363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22f8195a70 msgr2=0x7f22f8199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-10T11:22:22.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22f5d9b700 1 -- 172.21.15.178:0/2142111363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22f8195a70 msgr2=0x7f22f8199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T11:22:22.139 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22f5d9b700 1 --2- 172.21.15.178:0/2142111363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22f8195a70 0x7f22f8199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-10T11:22:22.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22f5d9b700 1 --2- 172.21.15.178:0/2142111363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22f8195a70 0x7f22f8199ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T11:22:22.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22f659c700 1 -- 172.21.15.178:0/2142111363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22f8195a70 msgr2=0x7f22f8199ee0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T11:22:22.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22f659c700 1 --2- 172.21.15.178:0/2142111363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22f8195a70 0x7f22f8199ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:22.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.136+0000 7f22f659c700 1 -- 172.21.15.178:0/2142111363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec00f900 con 0x7f22e00035c0 2024-09-10T11:22:22.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.137+0000 7f22f659c700 1 --2- 172.21.15.178:0/2142111363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e00035c0 0x7f22f8195530 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f22f80ffea0 tx=0x7f22ec004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:22:22.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.137+0000 7f22e77fe700 1 -- 172.21.15.178:0/2142111363 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22ec004b50 con 0x7f22e00035c0 2024-09-10T11:22:22.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.137+0000 7f22e77fe700 1 -- 172.21.15.178:0/2142111363 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f22ec013e10 con 0x7f22e00035c0 2024-09-10T11:22:22.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.137+0000 7f22fc8f6700 1 -- 172.21.15.178:0/2142111363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f819a4e0 con 0x7f22e00035c0 2024-09-10T11:22:22.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.137+0000 7f22e77fe700 1 -- 172.21.15.178:0/2142111363 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22ec011cd0 con 0x7f22e00035c0 2024-09-10T11:22:22.140 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.137+0000 7f22fc8f6700 1 -- 172.21.15.178:0/2142111363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f819aa30 con 0x7f22e00035c0 2024-09-10T11:22:22.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.138+0000 7f22fc8f6700 1 -- 172.21.15.178:0/2142111363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f8062380 con 0x7f22e00035c0 2024-09-10T11:22:22.142 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.138+0000 7f22e77fe700 1 -- 172.21.15.178:0/2142111363 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7f22ec013960 con 0x7f22e00035c0 2024-09-10T11:22:22.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.139+0000 7f22e77fe700 1 --2- 172.21.15.178:0/2142111363 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f22d806c700 0x7f22d806ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:22.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.139+0000 7f22e77fe700 1 -- 172.21.15.178:0/2142111363 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7f22ec08d080 con 0x7f22e00035c0 2024-09-10T11:22:22.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.139+0000 7f22f5d9b700 1 --2- 172.21.15.178:0/2142111363 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f22d806c700 0x7f22d806ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:22.144 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.140+0000 7f22f5d9b700 1 --2- 172.21.15.178:0/2142111363 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f22d806c700 0x7f22d806ebb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009180 tx=0x7f22e8008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:22:22.145 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.144+0000 7f22e77fe700 1 -- 172.21.15.178:0/2142111363 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f22ec0588f0 con 0x7f22e00035c0 2024-09-10T11:22:22.364 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:21 smithi178 ceph-mon[33514]: mgrmap e18: smithi144.bkogfs(active, since 33s), standbys: smithi178.vmvxuf 2024-09-10T11:22:22.364 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:21 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr metadata", "who": "smithi178.vmvxuf", "id": "smithi178.vmvxuf"}]: dispatch 2024-09-10T11:22:22.364 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:21 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:22:22.364 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:21 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:22:22.418 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.417+0000 7f22fc8f6700 1 -- 172.21.15.178:0/2142111363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f22f81a0180 con 0x7f22e00035c0 2024-09-10T11:22:22.419 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.417+0000 7f22e77fe700 1 -- 172.21.15.178:0/2142111363 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1026 (secure 0 0 0) 0x7f22ec031090 con 0x7f22e00035c0 2024-09-10T11:22:22.419 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:22:22.419 INFO:teuthology.orchestra.run.smithi178.stdout:{"epoch":2,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","modified":"2024-09-10T11:22:15.900195Z","created":"2024-09-10T11:19:28.938528Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi144","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:3300","nonce":0},{"type":"v1","addr":"172.21.15.144:6789","nonce":0}]},"addr":"172.21.15.144:6789/0","public_addr":"172.21.15.144:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi178","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:3300","nonce":0},{"type":"v1","addr":"172.21.15.178:6789","nonce":0}]},"addr":"172.21.15.178:6789/0","public_addr":"172.21.15.178:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T11:22:22.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.420+0000 7f22fc8f6700 1 -- 172.21.15.178:0/2142111363 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f22d806c700 msgr2=0x7f22d806ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:22.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.420+0000 7f22fc8f6700 1 --2- 172.21.15.178:0/2142111363 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f22d806c700 0x7f22d806ebb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f22e8009180 tx=0x7f22e8008040 comp rx=0 tx=0).stop 2024-09-10T11:22:22.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.420+0000 7f22fc8f6700 1 -- 172.21.15.178:0/2142111363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e00035c0 msgr2=0x7f22f8195530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:22.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.420+0000 7f22fc8f6700 1 --2- 172.21.15.178:0/2142111363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e00035c0 0x7f22f8195530 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f22f80ffea0 tx=0x7f22ec004f90 comp rx=0 tx=0).stop 2024-09-10T11:22:22.421 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.420+0000 7f22fc8f6700 1 -- 172.21.15.178:0/2142111363 shutdown_connections 2024-09-10T11:22:22.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.420+0000 7f22fc8f6700 1 --2- 172.21.15.178:0/2142111363 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f22d806c700 0x7f22d806ebb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:22.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.420+0000 7f22fc8f6700 1 --2- 172.21.15.178:0/2142111363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f22e00035c0 0x7f22f8195530 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:22.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.420+0000 7f22fc8f6700 1 --2- 172.21.15.178:0/2142111363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f22f8195a70 0x7f22f8199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:22.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.420+0000 7f22fc8f6700 1 -- 172.21.15.178:0/2142111363 >> 172.21.15.178:0/2142111363 conn(0x7f22f80fba50 msgr2=0x7f22f818d480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:22:22.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.420+0000 7f22fc8f6700 1 -- 172.21.15.178:0/2142111363 shutdown_connections 2024-09-10T11:22:22.422 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:22:22.420+0000 7f22fc8f6700 1 -- 172.21.15.178:0/2142111363 wait complete. 2024-09-10T11:22:22.422 INFO:teuthology.orchestra.run.smithi178.stderr:dumped monmap epoch 2 2024-09-10T11:22:22.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:21 smithi144 ceph-mon[27942]: mgrmap e18: smithi144.bkogfs(active, since 33s), standbys: smithi178.vmvxuf 2024-09-10T11:22:22.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:21 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr metadata", "who": "smithi178.vmvxuf", "id": "smithi178.vmvxuf"}]: dispatch 2024-09-10T11:22:22.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:21 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:22:22.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:21 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:22:23.072 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T11:22:23.072 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T11:22:23.282 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:22:23.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:22 smithi178 ceph-mon[33514]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:23.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:22 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/2142111363' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:23.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:22 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:23.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:22 smithi144 ceph-mon[27942]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:23.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:22 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/2142111363' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T11:22:23.402 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:22 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:24.669 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:24.669 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:24.669 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:24.669 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:22:24.669 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: Updating smithi144:/etc/ceph/ceph.conf 2024-09-10T11:22:24.669 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T11:22:24.669 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:22:24.669 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:22:24.669 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:24.670 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:24.670 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:24.670 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:24.670 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:24.670 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:24.670 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: Reconfiguring mon.smithi144 (unknown last config time)... 2024-09-10T11:22:24.670 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T11:22:24.670 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T11:22:24.670 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:24.670 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:24 smithi144 ceph-mon[27942]: Reconfiguring daemon mon.smithi144 on smithi144 2024-09-10T11:22:24.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:24.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:24.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:24.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:22:24.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: Updating smithi144:/etc/ceph/ceph.conf 2024-09-10T11:22:24.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T11:22:24.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:22:24.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:22:24.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:24.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:24.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:24.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:24.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:24.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:24.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: Reconfiguring mon.smithi144 (unknown last config time)... 2024-09-10T11:22:24.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T11:22:24.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T11:22:24.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:24.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:24 smithi178 ceph-mon[33514]: Reconfiguring daemon mon.smithi144 on smithi144 2024-09-10T11:22:25.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.485+0000 7fcb70b68700 1 -- 172.21.15.144:0/3925239926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c0ff040 msgr2=0x7fcb6c0ff410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:25.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.485+0000 7fcb70b68700 1 --2- 172.21.15.144:0/3925239926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c0ff040 0x7fcb6c0ff410 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcb54009910 tx=0x7fcb54009c20 comp rx=0 tx=0).stop 2024-09-10T11:22:25.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.486+0000 7fcb70b68700 1 -- 172.21.15.144:0/3925239926 shutdown_connections 2024-09-10T11:22:25.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.486+0000 7fcb70b68700 1 --2- 172.21.15.144:0/3925239926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c0ff040 0x7fcb6c0ff410 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:25.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.486+0000 7fcb70b68700 1 -- 172.21.15.144:0/3925239926 >> 172.21.15.144:0/3925239926 conn(0x7fcb6c0fa900 msgr2=0x7fcb6c0fcd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:22:25.489 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.486+0000 7fcb70b68700 1 -- 172.21.15.144:0/3925239926 shutdown_connections 2024-09-10T11:22:25.489 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.486+0000 7fcb70b68700 1 -- 172.21.15.144:0/3925239926 wait complete. 2024-09-10T11:22:25.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.486+0000 7fcb70b68700 1 Processor -- start 2024-09-10T11:22:25.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.487+0000 7fcb70b68700 1 -- start start 2024-09-10T11:22:25.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.487+0000 7fcb70b68700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c0ff040 0x7fcb6c196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:25.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.487+0000 7fcb70b68700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb6c196870 0x7fcb6c196ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:25.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.487+0000 7fcb70b68700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c19f430 con 0x7fcb6c0ff040 2024-09-10T11:22:25.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.487+0000 7fcb70b68700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6c19f570 con 0x7fcb6c196870 2024-09-10T11:22:25.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.487+0000 7fcb6a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c0ff040 0x7fcb6c196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:25.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.487+0000 7fcb6a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c0ff040 0x7fcb6c196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40540/0 (socket says 172.21.15.144:40540) 2024-09-10T11:22:25.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.487+0000 7fcb6a59c700 1 -- 172.21.15.144:0/1527338509 learned_addr learned my addr 172.21.15.144:0/1527338509 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:22:25.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.487+0000 7fcb69d9b700 1 --2- 172.21.15.144:0/1527338509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb6c196870 0x7fcb6c196ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:25.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.487+0000 7fcb6a59c700 1 -- 172.21.15.144:0/1527338509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb6c196870 msgr2=0x7fcb6c196ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:25.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.487+0000 7fcb6a59c700 1 --2- 172.21.15.144:0/1527338509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb6c196870 0x7fcb6c196ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:25.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.487+0000 7fcb6a59c700 1 -- 172.21.15.144:0/1527338509 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb540095f0 con 0x7fcb6c0ff040 2024-09-10T11:22:25.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.487+0000 7fcb6a59c700 1 --2- 172.21.15.144:0/1527338509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c0ff040 0x7fcb6c196330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcb540092e0 tx=0x7fcb5400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:22:25.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.488+0000 7fcb637fe700 1 -- 172.21.15.144:0/1527338509 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb5401d070 con 0x7fcb6c0ff040 2024-09-10T11:22:25.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.488+0000 7fcb637fe700 1 -- 172.21.15.144:0/1527338509 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcb54005140 con 0x7fcb6c0ff040 2024-09-10T11:22:25.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.488+0000 7fcb637fe700 1 -- 172.21.15.144:0/1527338509 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb54017440 con 0x7fcb6c0ff040 2024-09-10T11:22:25.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.488+0000 7fcb70b68700 1 -- 172.21.15.144:0/1527338509 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb6c19f710 con 0x7fcb6c0ff040 2024-09-10T11:22:25.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.488+0000 7fcb70b68700 1 -- 172.21.15.144:0/1527338509 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb6c19fc60 con 0x7fcb6c0ff040 2024-09-10T11:22:25.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.489+0000 7fcb70b68700 1 -- 172.21.15.144:0/1527338509 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb6c062380 con 0x7fcb6c0ff040 2024-09-10T11:22:25.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.489+0000 7fcb637fe700 1 -- 172.21.15.144:0/1527338509 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7fcb5400f460 con 0x7fcb6c0ff040 2024-09-10T11:22:25.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.489+0000 7fcb637fe700 1 --2- 172.21.15.144:0/1527338509 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcb5806c760 0x7fcb5806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:25.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.489+0000 7fcb637fe700 1 -- 172.21.15.144:0/1527338509 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7fcb5408ba10 con 0x7fcb6c0ff040 2024-09-10T11:22:25.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.490+0000 7fcb69d9b700 1 --2- 172.21.15.144:0/1527338509 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcb5806c760 0x7fcb5806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:25.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.490+0000 7fcb69d9b700 1 --2- 172.21.15.144:0/1527338509 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcb5806c760 0x7fcb5806ec10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0078e0 tx=0x7fcb5c008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:22:25.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.494+0000 7fcb637fe700 1 -- 172.21.15.144:0/1527338509 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcb54057300 con 0x7fcb6c0ff040 2024-09-10T11:22:25.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.689+0000 7fcb70b68700 1 -- 172.21.15.144:0/1527338509 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fcb6c1a0010 con 0x7fcb6c0ff040 2024-09-10T11:22:25.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.690+0000 7fcb637fe700 1 -- 172.21.15.144:0/1527338509 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+227 (secure 0 0 0) 0x7fcb54026090 con 0x7fcb6c0ff040 2024-09-10T11:22:25.693 INFO:teuthology.orchestra.run.smithi144.stdout:# minimal ceph.conf for 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:22:25.693 INFO:teuthology.orchestra.run.smithi144.stdout:[global] 2024-09-10T11:22:25.693 INFO:teuthology.orchestra.run.smithi144.stdout: fsid = 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:22:25.693 INFO:teuthology.orchestra.run.smithi144.stdout: mon_host = [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] 2024-09-10T11:22:25.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.692+0000 7fcb70b68700 1 -- 172.21.15.144:0/1527338509 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcb5806c760 msgr2=0x7fcb5806ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:25.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.692+0000 7fcb70b68700 1 --2- 172.21.15.144:0/1527338509 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcb5806c760 0x7fcb5806ec10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c0078e0 tx=0x7fcb5c008040 comp rx=0 tx=0).stop 2024-09-10T11:22:25.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.692+0000 7fcb70b68700 1 -- 172.21.15.144:0/1527338509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c0ff040 msgr2=0x7fcb6c196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:25.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.692+0000 7fcb70b68700 1 --2- 172.21.15.144:0/1527338509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c0ff040 0x7fcb6c196330 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcb540092e0 tx=0x7fcb5400bda0 comp rx=0 tx=0).stop 2024-09-10T11:22:25.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.692+0000 7fcb70b68700 1 -- 172.21.15.144:0/1527338509 shutdown_connections 2024-09-10T11:22:25.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.692+0000 7fcb70b68700 1 --2- 172.21.15.144:0/1527338509 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcb5806c760 0x7fcb5806ec10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:25.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.692+0000 7fcb70b68700 1 --2- 172.21.15.144:0/1527338509 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb6c0ff040 0x7fcb6c196330 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:25.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.692+0000 7fcb70b68700 1 --2- 172.21.15.144:0/1527338509 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb6c196870 0x7fcb6c196ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:25.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.692+0000 7fcb70b68700 1 -- 172.21.15.144:0/1527338509 >> 172.21.15.144:0/1527338509 conn(0x7fcb6c0fa900 msgr2=0x7fcb6c104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:22:25.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.692+0000 7fcb70b68700 1 -- 172.21.15.144:0/1527338509 shutdown_connections 2024-09-10T11:22:25.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:25.692+0000 7fcb70b68700 1 -- 172.21.15.144:0/1527338509 wait complete. 2024-09-10T11:22:26.034 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:25 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1527338509' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:26.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:25 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/1527338509' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:26.943 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T11:22:26.943 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:22:26.943 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T11:22:26.970 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:22:26.970 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:22:27.035 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T11:22:27.035 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T11:22:27.068 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T11:22:27.068 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:22:27.138 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T11:22:27.138 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:22:27.138 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T11:22:27.154 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T11:22:27.154 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_1 2024-09-10T11:22:27.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:27 smithi144 ceph-mon[27942]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:27 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:27 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:27 smithi144 ceph-mon[27942]: Reconfiguring mgr.smithi144.bkogfs (unknown last config time)... 2024-09-10T11:22:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:27 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.bkogfs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:22:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:27 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:22:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:27 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:27 smithi144 ceph-mon[27942]: Reconfiguring daemon mgr.smithi144.bkogfs on smithi144 2024-09-10T11:22:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T11:22:27.199 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:22:27.199 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-10T11:22:27.200 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:22:27.200 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:22:27.200 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-10 11:21:39.498759555 +0000 2024-09-10T11:22:27.200 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-10 11:16:41.731356708 +0000 2024-09-10T11:22:27.200 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-10 11:16:41.731356708 +0000 2024-09-10T11:22:27.200 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-09-10 11:16:41.731356708 +0000 2024-09-10T11:22:27.200 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T11:22:27.265 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-10T11:22:27.265 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-10T11:22:27.265 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000225555 s, 2.3 MB/s 2024-09-10T11:22:27.266 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T11:22:27.322 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_2 2024-09-10T11:22:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:27 smithi178 ceph-mon[33514]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:27 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:27 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:27 smithi178 ceph-mon[33514]: Reconfiguring mgr.smithi144.bkogfs (unknown last config time)... 2024-09-10T11:22:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:27 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.bkogfs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:22:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:27 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:22:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:27 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:27 smithi178 ceph-mon[33514]: Reconfiguring daemon mgr.smithi144.bkogfs on smithi144 2024-09-10T11:22:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T11:22:27.379 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:22:27.379 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-10T11:22:27.380 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:22:27.380 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:22:27.380 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-10 11:21:39.498759555 +0000 2024-09-10T11:22:27.380 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-10 11:16:42.071350539 +0000 2024-09-10T11:22:27.380 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-10 11:16:42.071350539 +0000 2024-09-10T11:22:27.380 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-09-10 11:16:42.071350539 +0000 2024-09-10T11:22:27.380 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T11:22:27.444 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-10T11:22:27.444 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-10T11:22:27.444 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000237825 s, 2.2 MB/s 2024-09-10T11:22:27.446 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T11:22:27.502 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_3 2024-09-10T11:22:27.557 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T11:22:27.557 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:22:27.557 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-10T11:22:27.557 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:22:27.557 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:22:27.557 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-10 11:21:39.499759537 +0000 2024-09-10T11:22:27.558 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-10 11:16:42.402344532 +0000 2024-09-10T11:22:27.558 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-10 11:16:42.402344532 +0000 2024-09-10T11:22:27.558 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-09-10 11:16:42.402344532 +0000 2024-09-10T11:22:27.558 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T11:22:27.620 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-10T11:22:27.621 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-10T11:22:27.621 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.00019029 s, 2.7 MB/s 2024-09-10T11:22:27.622 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T11:22:27.678 DEBUG:teuthology.orchestra.run.smithi144:> stat /dev/vg_nvme/lv_4 2024-09-10T11:22:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T11:22:27.733 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:22:27.733 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-10T11:22:27.733 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:22:27.733 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:22:27.734 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 2024-09-10 11:21:39.499759537 +0000 2024-09-10T11:22:27.734 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-10 11:16:42.729338598 +0000 2024-09-10T11:22:27.734 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-10 11:16:42.729338598 +0000 2024-09-10T11:22:27.734 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: 2024-09-10 11:16:42.729338598 +0000 2024-09-10T11:22:27.734 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T11:22:27.797 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records in 2024-09-10T11:22:27.797 INFO:teuthology.orchestra.run.smithi144.stderr:1+0 records out 2024-09-10T11:22:27.797 INFO:teuthology.orchestra.run.smithi144.stderr:512 bytes copied, 0.000197881 s, 2.6 MB/s 2024-09-10T11:22:27.798 DEBUG:teuthology.orchestra.run.smithi144:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T11:22:27.855 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T11:22:27.855 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T11:22:27.875 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T11:22:27.875 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_1 2024-09-10T11:22:27.933 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T11:22:27.934 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:22:27.934 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-10T11:22:27.934 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:22:27.934 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:22:27.934 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-10 11:22:13.723552778 +0000 2024-09-10T11:22:27.934 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-10 11:16:41.749306058 +0000 2024-09-10T11:22:27.934 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-10 11:16:41.749306058 +0000 2024-09-10T11:22:27.934 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-10 11:16:41.749306058 +0000 2024-09-10T11:22:27.934 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T11:22:28.001 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-10T11:22:28.002 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-10T11:22:28.002 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000132619 s, 3.9 MB/s 2024-09-10T11:22:28.003 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T11:22:28.062 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_2 2024-09-10T11:22:28.119 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T11:22:28.119 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:22:28.120 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-10T11:22:28.120 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:22:28.120 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:22:28.120 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-10 11:22:13.724552759 +0000 2024-09-10T11:22:28.120 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-10 11:16:42.076299803 +0000 2024-09-10T11:22:28.120 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-10 11:16:42.076299803 +0000 2024-09-10T11:22:28.120 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-10 11:16:42.076299803 +0000 2024-09-10T11:22:28.120 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T11:22:28.186 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-10T11:22:28.186 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-10T11:22:28.186 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.000144096 s, 3.6 MB/s 2024-09-10T11:22:28.187 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T11:22:28.245 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_3 2024-09-10T11:22:28.301 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T11:22:28.301 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:22:28.301 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-10T11:22:28.302 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:22:28.302 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:22:28.302 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-10 11:22:13.724552759 +0000 2024-09-10T11:22:28.302 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-10 11:16:42.381293970 +0000 2024-09-10T11:22:28.302 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-10 11:16:42.381293970 +0000 2024-09-10T11:22:28.302 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-10 11:16:42.381293970 +0000 2024-09-10T11:22:28.302 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T11:22:28.366 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-10T11:22:28.366 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-10T11:22:28.366 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 0.00012003 s, 4.3 MB/s 2024-09-10T11:22:28.367 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T11:22:28.425 DEBUG:teuthology.orchestra.run.smithi178:> stat /dev/vg_nvme/lv_4 2024-09-10T11:22:28.481 INFO:teuthology.orchestra.run.smithi178.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T11:22:28.481 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T11:22:28.481 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-09-10T11:22:28.481 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:22:28.481 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T11:22:28.481 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 2024-09-10 11:22:13.725552740 +0000 2024-09-10T11:22:28.481 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-10 11:16:42.683288193 +0000 2024-09-10T11:22:28.481 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-10 11:16:42.683288193 +0000 2024-09-10T11:22:28.481 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: 2024-09-10 11:16:42.683288193 +0000 2024-09-10T11:22:28.482 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T11:22:28.545 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records in 2024-09-10T11:22:28.545 INFO:teuthology.orchestra.run.smithi178.stderr:1+0 records out 2024-09-10T11:22:28.545 INFO:teuthology.orchestra.run.smithi178.stderr:512 bytes copied, 9.4688e-05 s, 5.4 MB/s 2024-09-10T11:22:28.546 DEBUG:teuthology.orchestra.run.smithi178:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T11:22:28.603 INFO:tasks.cephadm:Deploying osd.0 on smithi144 with /dev/vg_nvme/lv_4... 2024-09-10T11:22:28.603 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T11:22:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:29 smithi144 ceph-mon[27942]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:29 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:29 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:29 smithi144 ceph-mon[27942]: Reconfiguring ceph-exporter.smithi144 (monmap changed)... 2024-09-10T11:22:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:29 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T11:22:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:29 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:29 smithi144 ceph-mon[27942]: Reconfiguring daemon ceph-exporter.smithi144 on smithi144 2024-09-10T11:22:29.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:29 smithi178 ceph-mon[33514]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:29.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:29 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:29.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:29 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:29.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:29 smithi178 ceph-mon[33514]: Reconfiguring ceph-exporter.smithi144 (monmap changed)... 2024-09-10T11:22:29.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:29 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T11:22:29.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:29 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:29.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:29 smithi178 ceph-mon[33514]: Reconfiguring daemon ceph-exporter.smithi144 on smithi144 2024-09-10T11:22:29.663 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:22:30.826 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:30 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:30.827 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:30 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:30.827 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:30 smithi144 ceph-mon[27942]: Reconfiguring crash.smithi144 (monmap changed)... 2024-09-10T11:22:30.827 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:30 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T11:22:30.827 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:30 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:30.827 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:30 smithi144 ceph-mon[27942]: Reconfiguring daemon crash.smithi144 on smithi144 2024-09-10T11:22:30.827 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:30 smithi144 ceph-mon[27942]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:30.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:30 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:30.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:30 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:30.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:30 smithi178 ceph-mon[33514]: Reconfiguring crash.smithi144 (monmap changed)... 2024-09-10T11:22:30.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:30 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T11:22:30.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:30 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:30.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:30 smithi178 ceph-mon[33514]: Reconfiguring daemon crash.smithi144 on smithi144 2024-09-10T11:22:30.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:30 smithi178 ceph-mon[33514]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:32.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:32 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:32.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:32 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:32.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:32 smithi144 ceph-mon[27942]: Reconfiguring alertmanager.smithi144 (dependencies changed)... 2024-09-10T11:22:32.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:32 smithi144 ceph-mon[27942]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:32.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:32 smithi144 ceph-mon[27942]: Reconfiguring daemon alertmanager.smithi144 on smithi144 2024-09-10T11:22:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:32 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:32 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:32 smithi178 ceph-mon[33514]: Reconfiguring alertmanager.smithi144 (dependencies changed)... 2024-09-10T11:22:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:32 smithi178 ceph-mon[33514]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:32 smithi178 ceph-mon[33514]: Reconfiguring daemon alertmanager.smithi144 on smithi144 2024-09-10T11:22:33.293 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:22:33.314 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_4 2024-09-10T11:22:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:33 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:22:34.119 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:22:34.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:33 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:22:35.134 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:34 smithi144 ceph-mon[27942]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:34 smithi178 ceph-mon[33514]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:37.225 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:36 smithi144 ceph-mon[27942]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:36 smithi178 ceph-mon[33514]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:37.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.482+0000 7fcc16e17700 1 -- 172.21.15.144:0/1085740246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc100ff050 msgr2=0x7fcc100ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:37.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.482+0000 7fcc16e17700 1 --2- 172.21.15.144:0/1085740246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc100ff050 0x7fcc100ff460 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009910 tx=0x7fcc04009c20 comp rx=0 tx=0).stop 2024-09-10T11:22:37.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.482+0000 7fcc16e17700 1 -- 172.21.15.144:0/1085740246 shutdown_connections 2024-09-10T11:22:37.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.482+0000 7fcc16e17700 1 --2- 172.21.15.144:0/1085740246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc10100250 0x7fcc101006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:37.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.482+0000 7fcc16e17700 1 --2- 172.21.15.144:0/1085740246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc100ff050 0x7fcc100ff460 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:37.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.482+0000 7fcc16e17700 1 -- 172.21.15.144:0/1085740246 >> 172.21.15.144:0/1085740246 conn(0x7fcc100fa5e0 msgr2=0x7fcc100fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:22:37.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.482+0000 7fcc16e17700 1 -- 172.21.15.144:0/1085740246 shutdown_connections 2024-09-10T11:22:37.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.482+0000 7fcc16e17700 1 -- 172.21.15.144:0/1085740246 wait complete. 2024-09-10T11:22:37.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.483+0000 7fcc16e17700 1 Processor -- start 2024-09-10T11:22:37.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.483+0000 7fcc16e17700 1 -- start start 2024-09-10T11:22:37.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.483+0000 7fcc16e17700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc100ff050 0x7fcc10195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:37.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.483+0000 7fcc16e17700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc10100250 0x7fcc10195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:37.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.483+0000 7fcc16e17700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc101965b0 con 0x7fcc10100250 2024-09-10T11:22:37.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.483+0000 7fcc16e17700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc101966f0 con 0x7fcc100ff050 2024-09-10T11:22:37.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.483+0000 7fcc0ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc10100250 0x7fcc10195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:37.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.483+0000 7fcc0ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc10100250 0x7fcc10195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45098/0 (socket says 172.21.15.144:45098) 2024-09-10T11:22:37.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.483+0000 7fcc0ffff700 1 -- 172.21.15.144:0/1858244885 learned_addr learned my addr 172.21.15.144:0/1858244885 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:22:37.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.483+0000 7fcc14bb3700 1 --2- 172.21.15.144:0/1858244885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc100ff050 0x7fcc10195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:37.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.484+0000 7fcc0ffff700 1 -- 172.21.15.144:0/1858244885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc100ff050 msgr2=0x7fcc10195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:22:37.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.484+0000 7fcc0ffff700 1 --2- 172.21.15.144:0/1858244885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc100ff050 0x7fcc10195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:22:37.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.484+0000 7fcc0ffff700 1 -- 172.21.15.144:0/1858244885 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc040095f0 con 0x7fcc10100250 2024-09-10T11:22:37.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.484+0000 7fcc0ffff700 1 --2- 172.21.15.144:0/1858244885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc10100250 0x7fcc10195f90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00b4b0 tx=0x7fcbfc00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:22:37.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.484+0000 7fcc0dffb700 1 -- 172.21.15.144:0/1858244885 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbfc011800 con 0x7fcc10100250 2024-09-10T11:22:37.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.484+0000 7fcc0dffb700 1 -- 172.21.15.144:0/1858244885 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcbfc011e10 con 0x7fcc10100250 2024-09-10T11:22:37.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.484+0000 7fcc0dffb700 1 -- 172.21.15.144:0/1858244885 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbfc00f2e0 con 0x7fcc10100250 2024-09-10T11:22:37.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.484+0000 7fcc16e17700 1 -- 172.21.15.144:0/1858244885 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1019b150 con 0x7fcc10100250 2024-09-10T11:22:37.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.485+0000 7fcc16e17700 1 -- 172.21.15.144:0/1858244885 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc101017f0 con 0x7fcc10100250 2024-09-10T11:22:37.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.486+0000 7fcc16e17700 1 -- 172.21.15.144:0/1858244885 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc10107c90 con 0x7fcc10100250 2024-09-10T11:22:37.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.486+0000 7fcc0dffb700 1 -- 172.21.15.144:0/1858244885 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7fcbfc010380 con 0x7fcc10100250 2024-09-10T11:22:37.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.487+0000 7fcc0dffb700 1 --2- 172.21.15.144:0/1858244885 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcc0006c600 0x7fcc0006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:22:37.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.487+0000 7fcc0dffb700 1 -- 172.21.15.144:0/1858244885 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1198+0+0 (secure 0 0 0) 0x7fcbfc08a170 con 0x7fcc10100250 2024-09-10T11:22:37.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.487+0000 7fcc14bb3700 1 --2- 172.21.15.144:0/1858244885 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcc0006c600 0x7fcc0006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:22:37.489 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.488+0000 7fcc14bb3700 1 --2- 172.21.15.144:0/1858244885 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcc0006c600 0x7fcc0006eab0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcc04006010 tx=0x7fcc04005280 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:22:37.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.492+0000 7fcc0dffb700 1 -- 172.21.15.144:0/1858244885 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcbfc0594f0 con 0x7fcc10100250 2024-09-10T11:22:37.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:22:37.700+0000 7fcc16e17700 1 -- 172.21.15.144:0/1858244885 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc10068ae0 con 0x7fcc0006c600 2024-09-10T11:22:38.540 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:38 smithi144 ceph-mon[27942]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:22:38.540 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:38 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T11:22:38.540 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:38 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T11:22:38.540 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:38 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:38.540 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:38 smithi144 ceph-mon[27942]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:38 smithi178 ceph-mon[33514]: from='client.14252 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:22:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:38 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T11:22:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:38 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T11:22:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:38 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:38 smithi178 ceph-mon[33514]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:40 smithi144 ceph-mon[27942]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:40 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:40 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:40 smithi178 ceph-mon[33514]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:40 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:40 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:41 smithi178 ceph-mon[33514]: Reconfiguring grafana.smithi144 (dependencies changed)... 2024-09-10T11:22:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:41 smithi178 ceph-mon[33514]: Reconfiguring daemon grafana.smithi144 on smithi144 2024-09-10T11:22:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:41 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/2139644110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5612a382-bb30-4314-8f65-0925157cf09d"}]: dispatch 2024-09-10T11:22:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:41 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/2139644110' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5612a382-bb30-4314-8f65-0925157cf09d"}]': finished 2024-09-10T11:22:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:41 smithi178 ceph-mon[33514]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T11:22:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:41 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:22:42.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:41 smithi144 ceph-mon[27942]: Reconfiguring grafana.smithi144 (dependencies changed)... 2024-09-10T11:22:42.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:41 smithi144 ceph-mon[27942]: Reconfiguring daemon grafana.smithi144 on smithi144 2024-09-10T11:22:42.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:41 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2139644110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5612a382-bb30-4314-8f65-0925157cf09d"}]: dispatch 2024-09-10T11:22:42.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:41 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2139644110' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5612a382-bb30-4314-8f65-0925157cf09d"}]': finished 2024-09-10T11:22:42.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:41 smithi144 ceph-mon[27942]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T11:22:42.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:41 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:22:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:42 smithi178 ceph-mon[33514]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:42 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/3885179623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T11:22:43.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:42 smithi144 ceph-mon[27942]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:43.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:42 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/3885179623' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T11:22:45.254 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:44 smithi144 ceph-mon[27942]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:44 smithi178 ceph-mon[33514]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:47.036 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:46 smithi144 ceph-mon[27942]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:46 smithi178 ceph-mon[33514]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:48 smithi178 ceph-mon[33514]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:22:48.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:48 smithi144 ceph-mon[27942]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:22:51.042 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:50 smithi144 ceph-mon[27942]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:51.043 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:50 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:51.043 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:50 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:50 smithi178 ceph-mon[33514]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:50 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:50 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:52.268 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:51 smithi178 ceph-mon[33514]: Reconfiguring prometheus.smithi144 (dependencies changed)... 2024-09-10T11:22:52.268 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:51 smithi178 ceph-mon[33514]: Reconfiguring daemon prometheus.smithi144 on smithi144 2024-09-10T11:22:52.305 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:51 smithi144 ceph-mon[27942]: Reconfiguring prometheus.smithi144 (dependencies changed)... 2024-09-10T11:22:52.306 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:51 smithi144 ceph-mon[27942]: Reconfiguring daemon prometheus.smithi144 on smithi144 2024-09-10T11:22:53.362 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:52 smithi144 ceph-mon[27942]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:53.362 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:52 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T11:22:53.362 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:52 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:53.362 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:52 smithi144 ceph-mon[27942]: Deploying daemon osd.0 on smithi144 2024-09-10T11:22:53.366 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:52 smithi178 ceph-mon[33514]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:52 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T11:22:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:52 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:52 smithi178 ceph-mon[33514]: Deploying daemon osd.0 on smithi144 2024-09-10T11:22:55.075 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:54 smithi144 ceph-mon[27942]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:54 smithi178 ceph-mon[33514]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:56 smithi178 ceph-mon[33514]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:56 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:56 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:56 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi178", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T11:22:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:56 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:57.250 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:56 smithi144 ceph-mon[27942]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:57.250 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:57.250 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:57.250 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi178", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T11:22:57.250 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:58.094 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:57 smithi144 ceph-mon[27942]: Reconfiguring ceph-exporter.smithi178 (monmap changed)... 2024-09-10T11:22:58.094 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:57 smithi144 ceph-mon[27942]: Reconfiguring daemon ceph-exporter.smithi178 on smithi178 2024-09-10T11:22:58.094 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:57 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:58.094 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:57 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:58.094 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:57 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi178", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T11:22:58.094 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:57 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:58.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:57 smithi178 ceph-mon[33514]: Reconfiguring ceph-exporter.smithi178 (monmap changed)... 2024-09-10T11:22:58.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:57 smithi178 ceph-mon[33514]: Reconfiguring daemon ceph-exporter.smithi178 on smithi178 2024-09-10T11:22:58.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:57 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:58.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:57 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:22:58.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:57 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi178", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T11:22:58.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:57 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:22:59.081 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:58 smithi144 ceph-mon[27942]: Reconfiguring crash.smithi178 (monmap changed)... 2024-09-10T11:22:59.081 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:58 smithi144 ceph-mon[27942]: Reconfiguring daemon crash.smithi178 on smithi178 2024-09-10T11:22:59.081 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:22:58 smithi144 ceph-mon[27942]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:22:59.089 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:58 smithi178 ceph-mon[33514]: Reconfiguring crash.smithi178 (monmap changed)... 2024-09-10T11:22:59.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:58 smithi178 ceph-mon[33514]: Reconfiguring daemon crash.smithi178 on smithi178 2024-09-10T11:22:59.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:22:58 smithi178 ceph-mon[33514]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:23:00.335 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:00 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:00.335 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:00 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:00.335 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:00 smithi178 ceph-mon[33514]: Reconfiguring mgr.smithi178.vmvxuf (monmap changed)... 2024-09-10T11:23:00.335 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:00 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi178.vmvxuf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:23:00.335 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:00 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:23:00.335 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:00 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:00.335 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:00 smithi178 ceph-mon[33514]: Reconfiguring daemon mgr.smithi178.vmvxuf on smithi178 2024-09-10T11:23:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:00 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:00 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:00 smithi144 ceph-mon[27942]: Reconfiguring mgr.smithi178.vmvxuf (monmap changed)... 2024-09-10T11:23:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:00 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi178.vmvxuf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:23:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:00 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:23:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:00 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:00 smithi144 ceph-mon[27942]: Reconfiguring daemon mgr.smithi178.vmvxuf on smithi178 2024-09-10T11:23:01.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:01 smithi144 ceph-mon[27942]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:23:01.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:01 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:01.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:01 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:01.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:01 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T11:23:01.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:01 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T11:23:01.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:01 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:01.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:01 smithi178 ceph-mon[33514]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:23:01.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:01 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:01.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:01 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:01.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:01 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T11:23:01.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:01 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T11:23:01.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:01 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:02.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: Reconfiguring mon.smithi178 (monmap changed)... 2024-09-10T11:23:02.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: Reconfiguring daemon mon.smithi178 on smithi178 2024-09-10T11:23:02.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:02.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:02.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:02.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:02.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T11:23:02.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi144.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T11:23:02.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:02.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T11:23:02.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T11:23:02.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:02.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:23:02.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T11:23:02.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:02.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:23:02.501 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: Reconfiguring mon.smithi178 (monmap changed)... 2024-09-10T11:23:02.501 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: Reconfiguring daemon mon.smithi178 on smithi178 2024-09-10T11:23:02.501 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:02.501 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:02.501 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:02.501 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:02.501 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T11:23:02.502 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi144.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T11:23:02.502 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:02.502 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T11:23:02.502 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T11:23:02.502 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:02.502 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:23:02.502 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T11:23:02.502 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:02.502 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:23:03.278 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:03 smithi144 ceph-mon[27942]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T11:23:03.278 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:03 smithi144 ceph-mon[27942]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi144.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T11:23:03.278 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:03 smithi144 ceph-mon[27942]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T11:23:03.278 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:03 smithi144 ceph-mon[27942]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T11:23:03.278 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:03 smithi144 ceph-mon[27942]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:23:03.278 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:03 smithi144 ceph-mon[27942]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:23:03.278 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:03 smithi144 ceph-mon[27942]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T11:23:03.278 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:23:03.326 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:03 smithi178 ceph-mon[33514]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T11:23:03.326 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:03 smithi178 ceph-mon[33514]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi144.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T11:23:03.326 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:03 smithi178 ceph-mon[33514]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T11:23:03.326 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:03 smithi178 ceph-mon[33514]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi144.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T11:23:03.327 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:03 smithi178 ceph-mon[33514]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:23:03.327 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:03 smithi178 ceph-mon[33514]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:23:03.327 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:03 smithi178 ceph-mon[33514]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi144.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T11:23:03.327 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:03 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:23:04.843 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:04 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:04.843 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:04 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:04.844 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:04 smithi144 ceph-mon[27942]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:23:04.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:04.919+0000 7fcc0dffb700 1 -- 172.21.15.144:0/1858244885 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcc10068ae0 con 0x7fcc0006c600 2024-09-10T11:23:04.922 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 0 on host 'smithi144' 2024-09-10T11:23:04.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:04.922+0000 7fcc16e17700 1 -- 172.21.15.144:0/1858244885 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcc0006c600 msgr2=0x7fcc0006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:23:04.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:04.922+0000 7fcc16e17700 1 --2- 172.21.15.144:0/1858244885 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcc0006c600 0x7fcc0006eab0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fcc04006010 tx=0x7fcc04005280 comp rx=0 tx=0).stop 2024-09-10T11:23:04.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:04.922+0000 7fcc16e17700 1 -- 172.21.15.144:0/1858244885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc10100250 msgr2=0x7fcc10195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:23:04.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:04.922+0000 7fcc16e17700 1 --2- 172.21.15.144:0/1858244885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc10100250 0x7fcc10195f90 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc00b4b0 tx=0x7fcbfc00b7c0 comp rx=0 tx=0).stop 2024-09-10T11:23:04.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:04.922+0000 7fcc16e17700 1 -- 172.21.15.144:0/1858244885 shutdown_connections 2024-09-10T11:23:04.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:04.922+0000 7fcc16e17700 1 --2- 172.21.15.144:0/1858244885 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcc0006c600 0x7fcc0006eab0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:23:04.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:04.922+0000 7fcc16e17700 1 --2- 172.21.15.144:0/1858244885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc100ff050 0x7fcc10195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:23:04.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:04.922+0000 7fcc16e17700 1 --2- 172.21.15.144:0/1858244885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc10100250 0x7fcc10195f90 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:23:04.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:04.922+0000 7fcc16e17700 1 -- 172.21.15.144:0/1858244885 >> 172.21.15.144:0/1858244885 conn(0x7fcc100fa5e0 msgr2=0x7fcc100683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:23:04.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:04.922+0000 7fcc16e17700 1 -- 172.21.15.144:0/1858244885 shutdown_connections 2024-09-10T11:23:04.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:04.922+0000 7fcc16e17700 1 -- 172.21.15.144:0/1858244885 wait complete. 2024-09-10T11:23:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:04 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:04 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:04 smithi178 ceph-mon[33514]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:23:05.529 DEBUG:teuthology.orchestra.run.smithi144:osd.0> sudo journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T11:23:05.531 INFO:tasks.cephadm:Deploying osd.1 on smithi144 with /dev/vg_nvme/lv_3... 2024-09-10T11:23:05.531 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T11:23:06.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:05 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:06.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:05 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:06.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:05 smithi144 ceph-mon[27942]: from='osd.0 [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T11:23:06.703 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:05 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:06.704 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:05 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:06.704 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:05 smithi178 ceph-mon[33514]: from='osd.0 [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T11:23:06.704 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:23:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:06 smithi144 ceph-mon[27942]: from='osd.0 [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T11:23:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:06 smithi144 ceph-mon[27942]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T11:23:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:06 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:23:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:06 smithi144 ceph-mon[27942]: from='osd.0 [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T11:23:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:06 smithi144 ceph-mon[27942]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:23:06.934 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:23:06 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0[42921]: 2024-09-10T11:23:06.880+0000 7f4da636f700 -1 osd.0 0 waiting for initial osdmap 2024-09-10T11:23:06.934 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:23:06 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0[42921]: 2024-09-10T11:23:06.884+0000 7f4d9d15a700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:23:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:06 smithi178 ceph-mon[33514]: from='osd.0 [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T11:23:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:06 smithi178 ceph-mon[33514]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T11:23:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:06 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:23:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:06 smithi178 ceph-mon[33514]: from='osd.0 [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T11:23:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:06 smithi178 ceph-mon[33514]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:23:07.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:07 smithi144 ceph-mon[27942]: from='osd.0 [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T11:23:07.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:07 smithi144 ceph-mon[27942]: osdmap e8: 1 total, 0 up, 1 in 2024-09-10T11:23:07.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:23:07.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:23:08.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:07 smithi178 ceph-mon[33514]: from='osd.0 [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T11:23:08.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:07 smithi178 ceph-mon[33514]: osdmap e8: 1 total, 0 up, 1 in 2024-09-10T11:23:08.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:07 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:23:08.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:07 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:23:09.086 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:08 smithi144 ceph-mon[27942]: purged_snaps scrub starts 2024-09-10T11:23:09.086 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:08 smithi144 ceph-mon[27942]: purged_snaps scrub ok 2024-09-10T11:23:09.086 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:08 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:23:09.086 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:08 smithi144 ceph-mon[27942]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:23:09.086 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:08 smithi144 ceph-mon[27942]: osd.0 [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408] boot 2024-09-10T11:23:09.086 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:08 smithi144 ceph-mon[27942]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T11:23:09.086 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:08 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:23:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:08 smithi178 ceph-mon[33514]: purged_snaps scrub starts 2024-09-10T11:23:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:08 smithi178 ceph-mon[33514]: purged_snaps scrub ok 2024-09-10T11:23:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:08 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:23:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:08 smithi178 ceph-mon[33514]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T11:23:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:08 smithi178 ceph-mon[33514]: osd.0 [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408] boot 2024-09-10T11:23:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:08 smithi178 ceph-mon[33514]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T11:23:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:08 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:23:09.643 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:23:09.667 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_3 2024-09-10T11:23:09.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:09 smithi144 ceph-mon[27942]: osdmap e10: 1 total, 1 up, 1 in 2024-09-10T11:23:10.332 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:23:10.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:09 smithi178 ceph-mon[33514]: osdmap e10: 1 total, 1 up, 1 in 2024-09-10T11:23:11.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:10 smithi144 ceph-mon[27942]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:11.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:10 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:11.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:10 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:10 smithi178 ceph-mon[33514]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:10 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:10 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:12.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.025+0000 7f9bd400f700 1 -- 172.21.15.144:0/387356166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bcc100250 msgr2=0x7f9bcc1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:23:12.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.025+0000 7f9bd400f700 1 --2- 172.21.15.144:0/387356166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bcc100250 0x7f9bcc1006a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc009910 tx=0x7f9bbc009c20 comp rx=0 tx=0).stop 2024-09-10T11:23:12.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.025+0000 7f9bd400f700 1 -- 172.21.15.144:0/387356166 shutdown_connections 2024-09-10T11:23:12.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.025+0000 7f9bd400f700 1 --2- 172.21.15.144:0/387356166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bcc100250 0x7f9bcc1006a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:23:12.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.025+0000 7f9bd400f700 1 --2- 172.21.15.144:0/387356166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bcc0ff050 0x7f9bcc0ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:23:12.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.025+0000 7f9bd400f700 1 -- 172.21.15.144:0/387356166 >> 172.21.15.144:0/387356166 conn(0x7f9bcc0fa5e0 msgr2=0x7f9bcc0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:23:12.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.025+0000 7f9bd400f700 1 -- 172.21.15.144:0/387356166 shutdown_connections 2024-09-10T11:23:12.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.025+0000 7f9bd400f700 1 -- 172.21.15.144:0/387356166 wait complete. 2024-09-10T11:23:12.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.025+0000 7f9bd400f700 1 Processor -- start 2024-09-10T11:23:12.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.025+0000 7f9bd400f700 1 -- start start 2024-09-10T11:23:12.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bd400f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bcc0ff050 0x7f9bcc197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:23:12.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bd400f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bcc100250 0x7f9bcc1980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:23:12.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bd400f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc1986d0 con 0x7f9bcc0ff050 2024-09-10T11:23:12.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bd400f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bcc198810 con 0x7f9bcc100250 2024-09-10T11:23:12.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bd1dab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bcc0ff050 0x7f9bcc197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:23:12.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bd1dab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bcc0ff050 0x7f9bcc197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33098/0 (socket says 172.21.15.144:33098) 2024-09-10T11:23:12.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bd1dab700 1 -- 172.21.15.144:0/1796662484 learned_addr learned my addr 172.21.15.144:0/1796662484 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:23:12.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bd15aa700 1 --2- 172.21.15.144:0/1796662484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bcc100250 0x7f9bcc1980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:23:12.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bd1dab700 1 -- 172.21.15.144:0/1796662484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bcc100250 msgr2=0x7f9bcc1980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:23:12.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bd1dab700 1 --2- 172.21.15.144:0/1796662484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bcc100250 0x7f9bcc1980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:23:12.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bd1dab700 1 -- 172.21.15.144:0/1796662484 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bbc0095f0 con 0x7f9bcc0ff050 2024-09-10T11:23:12.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bd1dab700 1 --2- 172.21.15.144:0/1796662484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bcc0ff050 0x7f9bcc197b70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800d900 tx=0x7f9bc800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:23:12.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bc2ffd700 1 -- 172.21.15.144:0/1796662484 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc80041f0 con 0x7f9bcc0ff050 2024-09-10T11:23:12.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bc2ffd700 1 -- 172.21.15.144:0/1796662484 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9bc8004350 con 0x7f9bcc0ff050 2024-09-10T11:23:12.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bc2ffd700 1 -- 172.21.15.144:0/1796662484 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bc8003d20 con 0x7f9bcc0ff050 2024-09-10T11:23:12.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.026+0000 7f9bd400f700 1 -- 172.21.15.144:0/1796662484 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bcc19d2c0 con 0x7f9bcc0ff050 2024-09-10T11:23:12.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.027+0000 7f9bd400f700 1 -- 172.21.15.144:0/1796662484 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bcc19d810 con 0x7f9bcc0ff050 2024-09-10T11:23:12.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.028+0000 7f9bd400f700 1 -- 172.21.15.144:0/1796662484 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bcc106340 con 0x7f9bcc0ff050 2024-09-10T11:23:12.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.028+0000 7f9bc2ffd700 1 -- 172.21.15.144:0/1796662484 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7f9bc8009730 con 0x7f9bcc0ff050 2024-09-10T11:23:12.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.028+0000 7f9bc2ffd700 1 --2- 172.21.15.144:0/1796662484 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9bb806c600 0x7f9bb806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:23:12.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.029+0000 7f9bd15aa700 1 --2- 172.21.15.144:0/1796662484 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9bb806c600 0x7f9bb806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:23:12.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.029+0000 7f9bc2ffd700 1 -- 172.21.15.144:0/1796662484 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1810+0+0 (secure 0 0 0) 0x7f9bc808a740 con 0x7f9bcc0ff050 2024-09-10T11:23:12.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.029+0000 7f9bd15aa700 1 --2- 172.21.15.144:0/1796662484 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9bb806c600 0x7f9bb806eab0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc006010 tx=0x7f9bbc00b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:23:12.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.033+0000 7f9bc2ffd700 1 -- 172.21.15.144:0/1796662484 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9bc8059850 con 0x7f9bcc0ff050 2024-09-10T11:23:12.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:12.234+0000 7f9bd400f700 1 -- 172.21.15.144:0/1796662484 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f9bcc061190 con 0x7f9bb806c600 2024-09-10T11:23:13.077 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:12 smithi144 ceph-mon[27942]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:13.077 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:12 smithi144 ceph-mon[27942]: from='client.14270 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:23:13.077 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:12 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T11:23:13.078 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:12 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T11:23:13.078 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:12 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:12 smithi178 ceph-mon[33514]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:12 smithi178 ceph-mon[33514]: from='client.14270 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:23:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:12 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T11:23:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:12 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T11:23:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:12 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:14 smithi144 ceph-mon[27942]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:14 smithi178 ceph-mon[33514]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:16.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:16 smithi178 ceph-mon[33514]: Detected new or changed devices on smithi144 2024-09-10T11:23:16.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:16.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:16.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:23:16.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:16 smithi178 ceph-mon[33514]: Adjusting osd_memory_target on smithi144 to 11840M 2024-09-10T11:23:16.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:16.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:16.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:23:16.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:16.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:23:16.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:16 smithi178 ceph-mon[33514]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:16.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:16 smithi144 ceph-mon[27942]: Detected new or changed devices on smithi144 2024-09-10T11:23:16.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:16.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:16.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:23:16.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:16 smithi144 ceph-mon[27942]: Adjusting osd_memory_target on smithi144 to 11840M 2024-09-10T11:23:16.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:16.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:16.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:23:16.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:16.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:23:16.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:16 smithi144 ceph-mon[27942]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:17.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:17 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1671784254' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68977841-c0e7-44ce-8e18-96af001f56f7"}]: dispatch 2024-09-10T11:23:17.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:17 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1671784254' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68977841-c0e7-44ce-8e18-96af001f56f7"}]': finished 2024-09-10T11:23:17.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:17 smithi144 ceph-mon[27942]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T11:23:17.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:17 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:23:17.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:17 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/66677373' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T11:23:17.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:17 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/1671784254' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "68977841-c0e7-44ce-8e18-96af001f56f7"}]: dispatch 2024-09-10T11:23:17.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:17 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/1671784254' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "68977841-c0e7-44ce-8e18-96af001f56f7"}]': finished 2024-09-10T11:23:17.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:17 smithi178 ceph-mon[33514]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T11:23:17.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:17 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:23:17.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:17 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/66677373' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T11:23:18.575 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:18 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:18.576 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:18 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:18.576 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:18 smithi144 ceph-mon[27942]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:18.576 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:18 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:23:18.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:18 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:18.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:18 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:18.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:18 smithi178 ceph-mon[33514]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:18.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:18 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:23:21.063 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:21.064 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:21.064 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:21.064 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:23:21.064 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:21.064 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:20 smithi144 ceph-mon[27942]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:23:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:20 smithi178 ceph-mon[33514]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:23.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:22 smithi144 ceph-mon[27942]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:23.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:22 smithi178 ceph-mon[33514]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:24 smithi144 ceph-mon[27942]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:24 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T11:23:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:24 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:24 smithi144 ceph-mon[27942]: Deploying daemon osd.1 on smithi144 2024-09-10T11:23:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:24 smithi178 ceph-mon[33514]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:24 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T11:23:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:24 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:24 smithi178 ceph-mon[33514]: Deploying daemon osd.1 on smithi144 2024-09-10T11:23:27.079 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:26 smithi144 ceph-mon[27942]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:26 smithi178 ceph-mon[33514]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:28.566 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:28 smithi144 ceph-mon[27942]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:28.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:28 smithi178 ceph-mon[33514]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:30.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:30 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:30.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:30 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:30.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:30 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:23:30.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:30 smithi178 ceph-mon[33514]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:30.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:30 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:30.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:30 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:30.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:30 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:23:30.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:30 smithi144 ceph-mon[27942]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:32.359 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:32 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:32.359 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:32 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:32.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:32.390+0000 7f9bc2ffd700 1 -- 172.21.15.144:0/1796662484 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9bcc061190 con 0x7f9bb806c600 2024-09-10T11:23:32.392 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 1 on host 'smithi144' 2024-09-10T11:23:32.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:32.392+0000 7f9bd400f700 1 -- 172.21.15.144:0/1796662484 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9bb806c600 msgr2=0x7f9bb806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:23:32.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:32.392+0000 7f9bd400f700 1 --2- 172.21.15.144:0/1796662484 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9bb806c600 0x7f9bb806eab0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9bbc006010 tx=0x7f9bbc00b540 comp rx=0 tx=0).stop 2024-09-10T11:23:32.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:32.392+0000 7f9bd400f700 1 -- 172.21.15.144:0/1796662484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bcc0ff050 msgr2=0x7f9bcc197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:23:32.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:32.392+0000 7f9bd400f700 1 --2- 172.21.15.144:0/1796662484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bcc0ff050 0x7f9bcc197b70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9bc800d900 tx=0x7f9bc800dc10 comp rx=0 tx=0).stop 2024-09-10T11:23:32.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:32.392+0000 7f9bd400f700 1 -- 172.21.15.144:0/1796662484 shutdown_connections 2024-09-10T11:23:32.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:32.392+0000 7f9bd400f700 1 --2- 172.21.15.144:0/1796662484 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9bb806c600 0x7f9bb806eab0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:23:32.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:32.392+0000 7f9bd400f700 1 --2- 172.21.15.144:0/1796662484 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9bcc0ff050 0x7f9bcc197b70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:23:32.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:32.392+0000 7f9bd400f700 1 --2- 172.21.15.144:0/1796662484 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9bcc100250 0x7f9bcc1980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:23:32.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:32.392+0000 7f9bd400f700 1 -- 172.21.15.144:0/1796662484 >> 172.21.15.144:0/1796662484 conn(0x7f9bcc0fa5e0 msgr2=0x7f9bcc103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:23:32.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:32.393+0000 7f9bd400f700 1 -- 172.21.15.144:0/1796662484 shutdown_connections 2024-09-10T11:23:32.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:32.393+0000 7f9bd400f700 1 -- 172.21.15.144:0/1796662484 wait complete. 2024-09-10T11:23:32.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:32 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:32.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:32 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:33.352 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:33 smithi144 ceph-mon[27942]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:33.353 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:33 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:33.353 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:33 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:33.353 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:33 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:23:33.374 DEBUG:teuthology.orchestra.run.smithi144:osd.1> sudo journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T11:23:33.376 INFO:tasks.cephadm:Deploying osd.2 on smithi144 with /dev/vg_nvme/lv_2... 2024-09-10T11:23:33.376 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T11:23:33.481 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:33 smithi178 ceph-mon[33514]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:33.481 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:33 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:33.481 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:33 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:33.481 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:33 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:23:34.257 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:23:34.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:34 smithi178 ceph-mon[33514]: from='osd.1 [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T11:23:34.379 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:34 smithi144 ceph-mon[27942]: from='osd.1 [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T11:23:35.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:35 smithi144 ceph-mon[27942]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:35.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:35 smithi144 ceph-mon[27942]: from='osd.1 [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T11:23:35.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:35 smithi144 ceph-mon[27942]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T11:23:35.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:35 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:23:35.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:35 smithi144 ceph-mon[27942]: from='osd.1 [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T11:23:35.434 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:23:35 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1[48489]: 2024-09-10T11:23:35.133+0000 7fa59ace7700 -1 osd.1 0 waiting for initial osdmap 2024-09-10T11:23:35.434 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:23:35 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1[48489]: 2024-09-10T11:23:35.139+0000 7fa5932d5700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:23:35.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:35 smithi178 ceph-mon[33514]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:35.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:35 smithi178 ceph-mon[33514]: from='osd.1 [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T11:23:35.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:35 smithi178 ceph-mon[33514]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T11:23:35.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:35 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:23:35.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:35 smithi178 ceph-mon[33514]: from='osd.1 [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T11:23:36.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:36 smithi144 ceph-mon[27942]: from='osd.1 [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T11:23:36.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:36 smithi144 ceph-mon[27942]: osdmap e13: 2 total, 1 up, 2 in 2024-09-10T11:23:36.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:36 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:23:36.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:36 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:23:36.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:36 smithi178 ceph-mon[33514]: from='osd.1 [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T11:23:36.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:36 smithi178 ceph-mon[33514]: osdmap e13: 2 total, 1 up, 2 in 2024-09-10T11:23:36.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:36 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:23:36.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:36 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:23:37.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:37 smithi144 ceph-mon[27942]: purged_snaps scrub starts 2024-09-10T11:23:37.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:37 smithi144 ceph-mon[27942]: purged_snaps scrub ok 2024-09-10T11:23:37.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:37 smithi144 ceph-mon[27942]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:37.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:37 smithi144 ceph-mon[27942]: osd.1 [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549] boot 2024-09-10T11:23:37.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:37 smithi144 ceph-mon[27942]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T11:23:37.422 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:37 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:23:37.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:37 smithi178 ceph-mon[33514]: purged_snaps scrub starts 2024-09-10T11:23:37.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:37 smithi178 ceph-mon[33514]: purged_snaps scrub ok 2024-09-10T11:23:37.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:37 smithi178 ceph-mon[33514]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-10T11:23:37.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:37 smithi178 ceph-mon[33514]: osd.1 [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549] boot 2024-09-10T11:23:37.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:37 smithi178 ceph-mon[33514]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T11:23:37.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:37 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:23:38.415 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:38 smithi144 ceph-mon[27942]: osdmap e15: 2 total, 2 up, 2 in 2024-09-10T11:23:38.418 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:23:38.439 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi144:vg_nvme/lv_2 2024-09-10T11:23:38.488 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:38 smithi178 ceph-mon[33514]: osdmap e15: 2 total, 2 up, 2 in 2024-09-10T11:23:38.656 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:23:39.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:39 smithi144 ceph-mon[27942]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:39.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:39 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:39.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:39 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:39.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:39 smithi178 ceph-mon[33514]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:39.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:39 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:39.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:39 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:41.417 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:41 smithi144 ceph-mon[27942]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:41.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:41 smithi178 ceph-mon[33514]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:41.802 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.781+0000 7f6b6cc3b700 1 -- 172.21.15.144:0/2781158433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b68100190 msgr2=0x7f6b681005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:23:41.802 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.781+0000 7f6b6cc3b700 1 --2- 172.21.15.144:0/2781158433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b68100190 0x7f6b681005a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f6b58009910 tx=0x7f6b58009c20 comp rx=0 tx=0).stop 2024-09-10T11:23:41.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.782+0000 7f6b6cc3b700 1 -- 172.21.15.144:0/2781158433 shutdown_connections 2024-09-10T11:23:41.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.782+0000 7f6b6cc3b700 1 --2- 172.21.15.144:0/2781158433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b68101390 0x7f6b681017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:23:41.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.782+0000 7f6b6cc3b700 1 --2- 172.21.15.144:0/2781158433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b68100190 0x7f6b681005a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:23:41.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.782+0000 7f6b6cc3b700 1 -- 172.21.15.144:0/2781158433 >> 172.21.15.144:0/2781158433 conn(0x7f6b680fb720 msgr2=0x7f6b680fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:23:41.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.782+0000 7f6b6cc3b700 1 -- 172.21.15.144:0/2781158433 shutdown_connections 2024-09-10T11:23:41.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.782+0000 7f6b6cc3b700 1 -- 172.21.15.144:0/2781158433 wait complete. 2024-09-10T11:23:41.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.782+0000 7f6b6cc3b700 1 Processor -- start 2024-09-10T11:23:41.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.782+0000 7f6b6cc3b700 1 -- start start 2024-09-10T11:23:41.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.783+0000 7f6b6cc3b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b68100190 0x7f6b68195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:23:41.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.783+0000 7f6b6cc3b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b68101390 0x7f6b68195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:23:41.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.783+0000 7f6b6cc3b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b68196570 con 0x7f6b68101390 2024-09-10T11:23:41.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.783+0000 7f6b6cc3b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b681966b0 con 0x7f6b68100190 2024-09-10T11:23:41.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.783+0000 7f6b65d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b68101390 0x7f6b68195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:23:41.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.783+0000 7f6b65d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b68101390 0x7f6b68195f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59470/0 (socket says 172.21.15.144:59470) 2024-09-10T11:23:41.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.783+0000 7f6b65d9b700 1 -- 172.21.15.144:0/2714256595 learned_addr learned my addr 172.21.15.144:0/2714256595 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:23:41.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.783+0000 7f6b6659c700 1 --2- 172.21.15.144:0/2714256595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b68100190 0x7f6b68195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:23:41.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.783+0000 7f6b65d9b700 1 -- 172.21.15.144:0/2714256595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b68100190 msgr2=0x7f6b68195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:23:41.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.783+0000 7f6b65d9b700 1 --2- 172.21.15.144:0/2714256595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b68100190 0x7f6b68195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:23:41.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.783+0000 7f6b65d9b700 1 -- 172.21.15.144:0/2714256595 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b580095f0 con 0x7f6b68101390 2024-09-10T11:23:41.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.783+0000 7f6b65d9b700 1 --2- 172.21.15.144:0/2714256595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b68101390 0x7f6b68195f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00b4b0 tx=0x7f6b5c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:23:41.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.783+0000 7f6b577fe700 1 -- 172.21.15.144:0/2714256595 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b5c011800 con 0x7f6b68101390 2024-09-10T11:23:41.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.784+0000 7f6b6cc3b700 1 -- 172.21.15.144:0/2714256595 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b6819b160 con 0x7f6b68101390 2024-09-10T11:23:41.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.784+0000 7f6b577fe700 1 -- 172.21.15.144:0/2714256595 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6b5c011e10 con 0x7f6b68101390 2024-09-10T11:23:41.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.784+0000 7f6b577fe700 1 -- 172.21.15.144:0/2714256595 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b5c00f2e0 con 0x7f6b68101390 2024-09-10T11:23:41.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.784+0000 7f6b6cc3b700 1 -- 172.21.15.144:0/2714256595 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b6819b730 con 0x7f6b68101390 2024-09-10T11:23:41.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.785+0000 7f6b6cc3b700 1 -- 172.21.15.144:0/2714256595 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b68108d60 con 0x7f6b68101390 2024-09-10T11:23:41.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.785+0000 7f6b577fe700 1 -- 172.21.15.144:0/2714256595 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7f6b5c00f510 con 0x7f6b68101390 2024-09-10T11:23:41.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.785+0000 7f6b577fe700 1 --2- 172.21.15.144:0/2714256595 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6b5006c600 0x7f6b5006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:23:41.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.786+0000 7f6b577fe700 1 -- 172.21.15.144:0/2714256595 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2242+0+0 (secure 0 0 0) 0x7f6b5c08a6c0 con 0x7f6b68101390 2024-09-10T11:23:41.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.786+0000 7f6b6659c700 1 --2- 172.21.15.144:0/2714256595 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6b5006c600 0x7f6b5006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:23:41.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.786+0000 7f6b6659c700 1 --2- 172.21.15.144:0/2714256595 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6b5006c600 0x7f6b5006eab0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b58006010 tx=0x7f6b5800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:23:41.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:41.793+0000 7f6b577fe700 1 -- 172.21.15.144:0/2714256595 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6b5c0595a0 con 0x7f6b68101390 2024-09-10T11:23:42.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:23:42.010+0000 7f6b6cc3b700 1 -- 172.21.15.144:0/2714256595 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6b68105ce0 con 0x7f6b5006c600 2024-09-10T11:23:42.290 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:42 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T11:23:42.290 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:42 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T11:23:42.290 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:42 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:42.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:42 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T11:23:42.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:42 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T11:23:42.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:42 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:43.379 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:43 smithi144 ceph-mon[27942]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:43.380 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:43 smithi144 ceph-mon[27942]: from='client.14288 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:23:43.380 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:43 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:43.380 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:43 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:43.380 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:43 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:23:43.380 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:43 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:23:43.380 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:43 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:43.380 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:43 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:43.380 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:43 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:23:43.380 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:43 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:43.380 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:43 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:23:43.445 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:43 smithi178 ceph-mon[33514]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:43.445 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:43 smithi178 ceph-mon[33514]: from='client.14288 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi144:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:23:43.445 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:43 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:43.445 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:43 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:43.445 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:43 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:23:43.445 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:43 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:23:43.445 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:43 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:43.445 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:43 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:43.446 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:43 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:23:43.446 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:43 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:43.446 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:43 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:23:44.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:44 smithi144 ceph-mon[27942]: Detected new or changed devices on smithi144 2024-09-10T11:23:44.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:44 smithi144 ceph-mon[27942]: Adjusting osd_memory_target on smithi144 to 5920M 2024-09-10T11:23:44.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:44 smithi178 ceph-mon[33514]: Detected new or changed devices on smithi144 2024-09-10T11:23:44.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:44 smithi178 ceph-mon[33514]: Adjusting osd_memory_target on smithi144 to 5920M 2024-09-10T11:23:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:45 smithi144 ceph-mon[27942]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:45 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:45 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:45 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/600443263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f07849c-6967-4e40-a8e8-c5f928879f8b"}]: dispatch 2024-09-10T11:23:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:45 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/600443263' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f07849c-6967-4e40-a8e8-c5f928879f8b"}]': finished 2024-09-10T11:23:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:45 smithi144 ceph-mon[27942]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T11:23:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:45 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:23:45.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:45 smithi178 ceph-mon[33514]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:45.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:45 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:45.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:45 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:45.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:45 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/600443263' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8f07849c-6967-4e40-a8e8-c5f928879f8b"}]: dispatch 2024-09-10T11:23:45.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:45 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/600443263' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8f07849c-6967-4e40-a8e8-c5f928879f8b"}]': finished 2024-09-10T11:23:45.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:45 smithi178 ceph-mon[33514]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T11:23:45.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:45 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:23:46.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:46 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/4191126992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T11:23:46.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:46 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/4191126992' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T11:23:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:47 smithi144 ceph-mon[27942]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:47.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:47 smithi178 ceph-mon[33514]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:48 smithi178 ceph-mon[33514]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:23:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:23:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:48 smithi144 ceph-mon[27942]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:23:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:23:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:51.144 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:50 smithi144 ceph-mon[27942]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:50 smithi178 ceph-mon[33514]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:53.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:52 smithi144 ceph-mon[27942]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:53.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:52 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T11:23:53.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:52 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:53.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:52 smithi144 ceph-mon[27942]: Deploying daemon osd.2 on smithi144 2024-09-10T11:23:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:52 smithi178 ceph-mon[33514]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:52 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T11:23:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:52 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:23:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:52 smithi178 ceph-mon[33514]: Deploying daemon osd.2 on smithi144 2024-09-10T11:23:55.262 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:54 smithi144 ceph-mon[27942]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:54 smithi178 ceph-mon[33514]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:57.238 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:56 smithi144 ceph-mon[27942]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:56 smithi178 ceph-mon[33514]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:23:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:23:58 smithi178 ceph-mon[33514]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:23:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:23:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:23:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:23:58 smithi144 ceph-mon[27942]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:24:01.144 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:00 smithi144 ceph-mon[27942]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:24:01.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:24:01.178+0000 7f6b577fe700 1 -- 172.21.15.144:0/2714256595 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6b68105ce0 con 0x7f6b5006c600 2024-09-10T11:24:01.180 INFO:teuthology.orchestra.run.smithi144.stdout:Created osd(s) 2 on host 'smithi144' 2024-09-10T11:24:01.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:24:01.181+0000 7f6b6cc3b700 1 -- 172.21.15.144:0/2714256595 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6b5006c600 msgr2=0x7f6b5006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:24:01.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:24:01.181+0000 7f6b6cc3b700 1 --2- 172.21.15.144:0/2714256595 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6b5006c600 0x7f6b5006eab0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f6b58006010 tx=0x7f6b5800b540 comp rx=0 tx=0).stop 2024-09-10T11:24:01.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:24:01.181+0000 7f6b6cc3b700 1 -- 172.21.15.144:0/2714256595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b68101390 msgr2=0x7f6b68195f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:24:01.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:24:01.181+0000 7f6b6cc3b700 1 --2- 172.21.15.144:0/2714256595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b68101390 0x7f6b68195f50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6b5c00b4b0 tx=0x7f6b5c00b7c0 comp rx=0 tx=0).stop 2024-09-10T11:24:01.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:24:01.181+0000 7f6b6cc3b700 1 -- 172.21.15.144:0/2714256595 shutdown_connections 2024-09-10T11:24:01.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:24:01.181+0000 7f6b6cc3b700 1 --2- 172.21.15.144:0/2714256595 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6b5006c600 0x7f6b5006eab0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:01.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:24:01.181+0000 7f6b6cc3b700 1 --2- 172.21.15.144:0/2714256595 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b68100190 0x7f6b68195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:01.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:24:01.181+0000 7f6b6cc3b700 1 --2- 172.21.15.144:0/2714256595 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b68101390 0x7f6b68195f50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:01.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:24:01.181+0000 7f6b6cc3b700 1 -- 172.21.15.144:0/2714256595 >> 172.21.15.144:0/2714256595 conn(0x7f6b680fb720 msgr2=0x7f6b681045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:24:01.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:24:01.181+0000 7f6b6cc3b700 1 -- 172.21.15.144:0/2714256595 shutdown_connections 2024-09-10T11:24:01.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:24:01.181+0000 7f6b6cc3b700 1 -- 172.21.15.144:0/2714256595 wait complete. 2024-09-10T11:24:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:00 smithi178 ceph-mon[33514]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:24:02.072 DEBUG:teuthology.orchestra.run.smithi144:osd.2> sudo journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T11:24:02.074 INFO:tasks.cephadm:Deploying osd.3 on smithi178 with /dev/vg_nvme/lv_4... 2024-09-10T11:24:02.074 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-10T11:24:02.283 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:24:02.329 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:02.329 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:02.329 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:02 smithi144 ceph-mon[27942]: from='osd.2 [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T11:24:02.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:02.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:02.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:02 smithi178 ceph-mon[33514]: from='osd.2 [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T11:24:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:03 smithi178 ceph-mon[33514]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:24:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:03 smithi178 ceph-mon[33514]: from='osd.2 [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T11:24:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:03 smithi178 ceph-mon[33514]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T11:24:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:03 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:24:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:03 smithi178 ceph-mon[33514]: from='osd.2 [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T11:24:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:03 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:24:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:03 smithi144 ceph-mon[27942]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:24:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:03 smithi144 ceph-mon[27942]: from='osd.2 [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T11:24:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:03 smithi144 ceph-mon[27942]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T11:24:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:24:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:03 smithi144 ceph-mon[27942]: from='osd.2 [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T11:24:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:24:03.434 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:24:03 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2[53971]: 2024-09-10T11:24:03.187+0000 7f081f8f1700 -1 osd.2 0 waiting for initial osdmap 2024-09-10T11:24:03.434 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:24:03 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2[53971]: 2024-09-10T11:24:03.193+0000 7f081a6e4700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:24:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:04 smithi178 ceph-mon[33514]: from='osd.2 [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T11:24:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:04 smithi178 ceph-mon[33514]: osdmap e18: 3 total, 2 up, 3 in 2024-09-10T11:24:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:04 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:24:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:04 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:24:04.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:04 smithi144 ceph-mon[27942]: from='osd.2 [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]': finished 2024-09-10T11:24:04.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:04 smithi144 ceph-mon[27942]: osdmap e18: 3 total, 2 up, 3 in 2024-09-10T11:24:04.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:04 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:24:04.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:04 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:24:04.676 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:24:04.694 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_4 2024-09-10T11:24:04.891 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:24:05.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:05 smithi144 ceph-mon[27942]: purged_snaps scrub starts 2024-09-10T11:24:05.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:05 smithi144 ceph-mon[27942]: purged_snaps scrub ok 2024-09-10T11:24:05.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:05 smithi144 ceph-mon[27942]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:24:05.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:05 smithi144 ceph-mon[27942]: osd.2 [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368] boot 2024-09-10T11:24:05.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:05 smithi144 ceph-mon[27942]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T11:24:05.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:05 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:24:05.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:05 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:05.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:05 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:05.562 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:05 smithi178 ceph-mon[33514]: purged_snaps scrub starts 2024-09-10T11:24:05.562 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:05 smithi178 ceph-mon[33514]: purged_snaps scrub ok 2024-09-10T11:24:05.562 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:05 smithi178 ceph-mon[33514]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-10T11:24:05.562 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:05 smithi178 ceph-mon[33514]: osd.2 [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368] boot 2024-09-10T11:24:05.562 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:05 smithi178 ceph-mon[33514]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T11:24:05.562 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:05 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:24:05.562 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:05 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:05.562 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:05 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.334+0000 7f97e08de700 1 -- 172.21.15.178:0/1346993004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97dc103500 msgr2=0x7f97dc1058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:24:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.334+0000 7f97e08de700 1 --2- 172.21.15.178:0/1346993004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97dc103500 0x7f97dc1058e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009910 tx=0x7f97c4009c20 comp rx=0 tx=0).stop 2024-09-10T11:24:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.334+0000 7f97e08de700 1 -- 172.21.15.178:0/1346993004 shutdown_connections 2024-09-10T11:24:06.335 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.334+0000 7f97e08de700 1 --2- 172.21.15.178:0/1346993004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97dc103500 0x7f97dc1058e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:06.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.334+0000 7f97e08de700 1 --2- 172.21.15.178:0/1346993004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97dc100be0 0x7f97dc102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:06.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.334+0000 7f97e08de700 1 -- 172.21.15.178:0/1346993004 >> 172.21.15.178:0/1346993004 conn(0x7f97dc0fa5e0 msgr2=0x7f97dc0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:24:06.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.334+0000 7f97e08de700 1 -- 172.21.15.178:0/1346993004 shutdown_connections 2024-09-10T11:24:06.336 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.334+0000 7f97e08de700 1 -- 172.21.15.178:0/1346993004 wait complete. 2024-09-10T11:24:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.335+0000 7f97e08de700 1 Processor -- start 2024-09-10T11:24:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.335+0000 7f97e08de700 1 -- start start 2024-09-10T11:24:06.337 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.335+0000 7f97e08de700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97dc100be0 0x7f97dc195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:24:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.335+0000 7f97e08de700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97dc103500 0x7f97dc195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:24:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.335+0000 7f97e08de700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc196480 con 0x7f97dc103500 2024-09-10T11:24:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.335+0000 7f97e08de700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc1965c0 con 0x7f97dc100be0 2024-09-10T11:24:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.335+0000 7f97da59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97dc100be0 0x7f97dc195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:24:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.335+0000 7f97da59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97dc100be0 0x7f97dc195920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:55090/0 (socket says 172.21.15.178:55090) 2024-09-10T11:24:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.335+0000 7f97da59c700 1 -- 172.21.15.178:0/484950048 learned_addr learned my addr 172.21.15.178:0/484950048 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:24:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.335+0000 7f97d1d9b700 1 --2- 172.21.15.178:0/484950048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97dc103500 0x7f97dc195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:24:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.335+0000 7f97da59c700 1 -- 172.21.15.178:0/484950048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97dc103500 msgr2=0x7f97dc195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:24:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.335+0000 7f97da59c700 1 --2- 172.21.15.178:0/484950048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97dc103500 0x7f97dc195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.335+0000 7f97da59c700 1 -- 172.21.15.178:0/484950048 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c40095f0 con 0x7f97dc100be0 2024-09-10T11:24:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.335+0000 7f97da59c700 1 --2- 172.21.15.178:0/484950048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97dc100be0 0x7f97dc195920 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00e920 tx=0x7f97cc00ec30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:24:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.336+0000 7f97d3fff700 1 -- 172.21.15.178:0/484950048 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97cc00c9d0 con 0x7f97dc100be0 2024-09-10T11:24:06.338 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.336+0000 7f97e08de700 1 -- 172.21.15.178:0/484950048 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97dc19b070 con 0x7f97dc100be0 2024-09-10T11:24:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.336+0000 7f97d3fff700 1 -- 172.21.15.178:0/484950048 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f97cc010430 con 0x7f97dc100be0 2024-09-10T11:24:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.336+0000 7f97d3fff700 1 -- 172.21.15.178:0/484950048 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97cc018650 con 0x7f97dc100be0 2024-09-10T11:24:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.336+0000 7f97e08de700 1 -- 172.21.15.178:0/484950048 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97dc19b5c0 con 0x7f97dc100be0 2024-09-10T11:24:06.339 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.337+0000 7f97e08de700 1 -- 172.21.15.178:0/484950048 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97dc18fb70 con 0x7f97dc100be0 2024-09-10T11:24:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.337+0000 7f97d3fff700 1 -- 172.21.15.178:0/484950048 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 18) v1 ==== 89860+0+0 (secure 0 0 0) 0x7f97cc0105a0 con 0x7f97dc100be0 2024-09-10T11:24:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.338+0000 7f97d3fff700 1 --2- 172.21.15.178:0/484950048 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f97c806c600 0x7f97c806eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:24:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.338+0000 7f97d3fff700 1 -- 172.21.15.178:0/484950048 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3049+0+0 (secure 0 0 0) 0x7f97cc014070 con 0x7f97dc100be0 2024-09-10T11:24:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.338+0000 7f97d1d9b700 1 --2- 172.21.15.178:0/484950048 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f97c806c600 0x7f97c806eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:24:06.340 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.338+0000 7f97d1d9b700 1 --2- 172.21.15.178:0/484950048 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f97c806c600 0x7f97c806eab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009fd0 tx=0x7f97c4005940 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:24:06.343 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.342+0000 7f97d3fff700 1 -- 172.21.15.178:0/484950048 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f97cc051720 con 0x7f97dc100be0 2024-09-10T11:24:06.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:06 smithi144 ceph-mon[27942]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T11:24:06.538 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:06.537+0000 7f97e08de700 1 -- 172.21.15.178:0/484950048 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f97dc061190 con 0x7f97c806c600 2024-09-10T11:24:06.538 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:06 smithi178 ceph-mon[33514]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T11:24:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:07 smithi178 ceph-mon[33514]: pgmap v77: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:07 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T11:24:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:07 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T11:24:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:07 smithi178 ceph-mon[33514]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T11:24:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:07 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T11:24:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:07 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T11:24:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:07 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T11:24:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:07 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:07.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:07 smithi144 ceph-mon[27942]: pgmap v77: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:07.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-10T11:24:07.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-10T11:24:07.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:07 smithi144 ceph-mon[27942]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T11:24:07.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T11:24:07.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T11:24:07.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T11:24:07.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:07.977 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:24:07 smithi144 sudo[56377]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T11:24:08.494 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:24:07 smithi144 sudo[56377]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T11:24:08.494 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:24:07 smithi144 sudo[56377]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-09-10T11:24:08.494 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:24:07 smithi144 sudo[56377]: pam_unix(sudo:session): session closed for user root 2024-09-10T11:24:08.495 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:08.318+0000 7f97d3fff700 1 -- 172.21.15.178:0/484950048 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f97cc056710 con 0x7f97dc100be0 2024-09-10T11:24:08.495 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:24:07 smithi144 sudo[56435]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T11:24:08.495 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:24:07 smithi144 sudo[56435]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T11:24:08.495 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:24:07 smithi144 sudo[56435]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-09-10T11:24:08.495 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:24:07 smithi144 sudo[56435]: pam_unix(sudo:session): session closed for user root 2024-09-10T11:24:08.525 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:24:08 smithi144 sudo[56455]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T11:24:08.525 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:24:08 smithi144 sudo[56455]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T11:24:08.525 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:24:08 smithi144 sudo[56455]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-09-10T11:24:08.525 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:24:08 smithi144 sudo[56455]: pam_unix(sudo:session): session closed for user root 2024-09-10T11:24:08.525 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:24:08.525 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: Detected new or changed devices on smithi144 2024-09-10T11:24:08.525 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:08.525 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:08.525 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:24:08.526 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:24:08.526 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:24:08.526 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: Adjusting osd_memory_target on smithi144 to 3946M 2024-09-10T11:24:08.526 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:08.526 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:08.526 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:24:08.526 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:08.526 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T11:24:08.526 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T11:24:08.526 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:24:08.526 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:08 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:08.527 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: from='client.24135 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:24:08.527 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: Detected new or changed devices on smithi144 2024-09-10T11:24:08.527 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:08.527 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:08.527 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:24:08.527 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:24:08.527 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:24:08.527 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: Adjusting osd_memory_target on smithi144 to 3946M 2024-09-10T11:24:08.527 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:08.527 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:08.527 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:24:08.527 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:08.527 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T11:24:08.528 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: osdmap e22: 3 total, 3 up, 3 in 2024-09-10T11:24:08.528 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:24:08.528 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:08 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:08.780 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:24:08 smithi144 sudo[56505]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T11:24:08.780 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:24:08 smithi144 sudo[56505]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T11:24:08.780 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:24:08 smithi144 sudo[56505]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-09-10T11:24:08.780 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:24:08 smithi144 sudo[56505]: pam_unix(sudo:session): session closed for user root 2024-09-10T11:24:09.042 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:24:08 smithi144 sudo[56559]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T11:24:09.042 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:24:08 smithi144 sudo[56559]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T11:24:09.042 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:24:08 smithi144 sudo[56559]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-09-10T11:24:09.042 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:24:08 smithi144 sudo[56559]: pam_unix(sudo:session): session closed for user root 2024-09-10T11:24:09.305 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:24:09 smithi144 sudo[56581]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T11:24:09.305 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:24:09 smithi144 sudo[56581]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T11:24:09.305 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:24:09 smithi144 sudo[56581]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-09-10T11:24:09.305 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:24:09 smithi144 sudo[56581]: pam_unix(sudo:session): session closed for user root 2024-09-10T11:24:09.305 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 sudo[56591]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T11:24:09.305 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 sudo[56591]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T11:24:09.305 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 sudo[56591]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-09-10T11:24:09.305 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 sudo[56591]: pam_unix(sudo:session): session closed for user root 2024-09-10T11:24:09.555 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 sudo[56636]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T11:24:09.555 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 sudo[56636]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T11:24:09.555 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 sudo[56636]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-09-10T11:24:09.555 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 sudo[56636]: pam_unix(sudo:session): session closed for user root 2024-09-10T11:24:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:09 smithi178 ceph-mon[33514]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:09 smithi178 ceph-mon[33514]: mgrmap e19: smithi144.bkogfs(active, since 2m), standbys: smithi178.vmvxuf 2024-09-10T11:24:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:09 smithi178 ceph-mon[33514]: osdmap e23: 3 total, 3 up, 3 in 2024-09-10T11:24:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:09 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/3935093560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd8b7396-3962-4c06-ab46-d746cead4b1d"}]: dispatch 2024-09-10T11:24:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:09 smithi178 ceph-mon[33514]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd8b7396-3962-4c06-ab46-d746cead4b1d"}]: dispatch 2024-09-10T11:24:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:09 smithi178 ceph-mon[33514]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd8b7396-3962-4c06-ab46-d746cead4b1d"}]': finished 2024-09-10T11:24:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:09 smithi178 ceph-mon[33514]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T11:24:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:09 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:24:09.886 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 ceph-mon[27942]: pgmap v80: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:09.887 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 ceph-mon[27942]: mgrmap e19: smithi144.bkogfs(active, since 2m), standbys: smithi178.vmvxuf 2024-09-10T11:24:09.887 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 ceph-mon[27942]: osdmap e23: 3 total, 3 up, 3 in 2024-09-10T11:24:09.887 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/3935093560' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd8b7396-3962-4c06-ab46-d746cead4b1d"}]: dispatch 2024-09-10T11:24:09.887 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 ceph-mon[27942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fd8b7396-3962-4c06-ab46-d746cead4b1d"}]: dispatch 2024-09-10T11:24:09.887 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 ceph-mon[27942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fd8b7396-3962-4c06-ab46-d746cead4b1d"}]': finished 2024-09-10T11:24:09.887 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 ceph-mon[27942]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T11:24:09.887 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:09 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:24:10.051 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:09 smithi178 sudo[37475]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T11:24:10.051 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:09 smithi178 sudo[37475]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T11:24:10.051 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:09 smithi178 sudo[37475]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-09-10T11:24:10.051 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:09 smithi178 sudo[37475]: pam_unix(sudo:session): session closed for user root 2024-09-10T11:24:10.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:10 smithi178 sudo[37478]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T11:24:10.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:10 smithi178 sudo[37478]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T11:24:10.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:10 smithi178 sudo[37478]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-09-10T11:24:10.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:10 smithi178 sudo[37478]: pam_unix(sudo:session): session closed for user root 2024-09-10T11:24:10.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:10 smithi178 ceph-mon[33514]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T11:24:10.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:10 smithi178 ceph-mon[33514]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T11:24:10.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:10 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:24:10.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:10 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:24:10.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:10 smithi178 ceph-mon[33514]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T11:24:10.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:10 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:24:10.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:10 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:24:10.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:10 smithi178 ceph-mon[33514]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:10.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:10 smithi178 ceph-mon[33514]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T11:24:10.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:10 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/4151294753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T11:24:10.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:10 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:10.923 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:10 smithi144 ceph-mon[27942]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T11:24:10.924 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:10 smithi144 ceph-mon[27942]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T11:24:10.924 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:10 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:24:10.924 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:10 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:24:10.924 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:10 smithi144 ceph-mon[27942]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T11:24:10.924 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:10 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:24:10.924 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:10 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:24:10.924 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:10 smithi144 ceph-mon[27942]: pgmap v83: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:10.924 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:10 smithi144 ceph-mon[27942]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T11:24:10.924 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:10 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/4151294753' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T11:24:10.924 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:10 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:12 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:12 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:12 smithi178 ceph-mon[33514]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:12 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:12 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:24:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:12 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:12 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:12 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:12 smithi144 ceph-mon[27942]: pgmap v84: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:12 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:12 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:24:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:12 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:14 smithi178 ceph-mon[33514]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:15.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:14 smithi144 ceph-mon[27942]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:16 smithi178 ceph-mon[33514]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T11:24:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:16 smithi178 ceph-mon[33514]: Deploying daemon osd.3 on smithi178 2024-09-10T11:24:17.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:16 smithi144 ceph-mon[27942]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:17.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T11:24:17.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:17.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:16 smithi144 ceph-mon[27942]: Deploying daemon osd.3 on smithi178 2024-09-10T11:24:18.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:18 smithi178 ceph-mon[33514]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:18.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:18 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:24:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:18 smithi144 ceph-mon[27942]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:18 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:24:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:20 smithi178 ceph-mon[33514]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:21.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:20 smithi144 ceph-mon[27942]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:22.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:22 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:22.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:22 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:22.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:22 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:24:22.556 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:22 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:22.556 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:22 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:22.556 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:22 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:24:23.386 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:23 smithi178 ceph-mon[33514]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:23.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:23 smithi144 ceph-mon[27942]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:24.355 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:24.353+0000 7f97d3fff700 1 -- 172.21.15.178:0/484950048 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f97dc061190 con 0x7f97c806c600 2024-09-10T11:24:24.355 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 3 on host 'smithi178' 2024-09-10T11:24:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:24.356+0000 7f97e08de700 1 -- 172.21.15.178:0/484950048 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f97c806c600 msgr2=0x7f97c806eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:24:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:24.356+0000 7f97e08de700 1 --2- 172.21.15.178:0/484950048 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f97c806c600 0x7f97c806eab0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009fd0 tx=0x7f97c4005940 comp rx=0 tx=0).stop 2024-09-10T11:24:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:24.356+0000 7f97e08de700 1 -- 172.21.15.178:0/484950048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97dc100be0 msgr2=0x7f97dc195920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:24:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:24.356+0000 7f97e08de700 1 --2- 172.21.15.178:0/484950048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97dc100be0 0x7f97dc195920 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f97cc00e920 tx=0x7f97cc00ec30 comp rx=0 tx=0).stop 2024-09-10T11:24:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:24.356+0000 7f97e08de700 1 -- 172.21.15.178:0/484950048 shutdown_connections 2024-09-10T11:24:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:24.356+0000 7f97e08de700 1 --2- 172.21.15.178:0/484950048 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f97c806c600 0x7f97c806eab0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:24.358 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:24.356+0000 7f97e08de700 1 --2- 172.21.15.178:0/484950048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f97dc100be0 0x7f97dc195920 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:24.356+0000 7f97e08de700 1 --2- 172.21.15.178:0/484950048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f97dc103500 0x7f97dc195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:24.356+0000 7f97e08de700 1 -- 172.21.15.178:0/484950048 >> 172.21.15.178:0/484950048 conn(0x7f97dc0fa5e0 msgr2=0x7f97dc0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:24:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:24.357+0000 7f97e08de700 1 -- 172.21.15.178:0/484950048 shutdown_connections 2024-09-10T11:24:24.359 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:24.357+0000 7f97e08de700 1 -- 172.21.15.178:0/484950048 wait complete. 2024-09-10T11:24:25.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:25 smithi144 ceph-mon[27942]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:25.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:25 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:25.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:25 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:25.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:25 smithi144 ceph-mon[27942]: from='osd.3 [v2:172.21.15.178:6800/1366035688,v1:172.21.15.178:6801/1366035688]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T11:24:25.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:25 smithi144 ceph-mon[27942]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T11:24:25.564 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:25 smithi178 ceph-mon[33514]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:25.564 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:25 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:25.564 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:25 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:25.565 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:25 smithi178 ceph-mon[33514]: from='osd.3 [v2:172.21.15.178:6800/1366035688,v1:172.21.15.178:6801/1366035688]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T11:24:25.565 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:25 smithi178 ceph-mon[33514]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T11:24:25.586 DEBUG:teuthology.orchestra.run.smithi178:osd.3> sudo journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T11:24:25.588 INFO:tasks.cephadm:Deploying osd.4 on smithi178 with /dev/vg_nvme/lv_3... 2024-09-10T11:24:25.588 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-10T11:24:25.832 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:24:26.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:26 smithi178 ceph-mon[33514]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T11:24:26.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:26 smithi178 ceph-mon[33514]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T11:24:26.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:26 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:24:26.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:26 smithi178 ceph-mon[33514]: from='osd.3 [v2:172.21.15.178:6800/1366035688,v1:172.21.15.178:6801/1366035688]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:24:26.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:26 smithi178 ceph-mon[33514]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:24:26.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:26 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:26.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:26 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:26.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:26 smithi178 ceph-mon[33514]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:26.618 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:24:26 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3[39525]: 2024-09-10T11:24:26.326+0000 7f22e6dad700 -1 osd.3 0 waiting for initial osdmap 2024-09-10T11:24:26.618 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:24:26 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3[39525]: 2024-09-10T11:24:26.331+0000 7f22e33a3700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:24:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:26 smithi144 ceph-mon[27942]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T11:24:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:26 smithi144 ceph-mon[27942]: osdmap e25: 4 total, 3 up, 4 in 2024-09-10T11:24:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:26 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:24:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:26 smithi144 ceph-mon[27942]: from='osd.3 [v2:172.21.15.178:6800/1366035688,v1:172.21.15.178:6801/1366035688]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:24:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:26 smithi144 ceph-mon[27942]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:24:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:26 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:26 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:26 smithi144 ceph-mon[27942]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-09-10T11:24:27.450 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:27 smithi178 ceph-mon[33514]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T11:24:27.450 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:27 smithi178 ceph-mon[33514]: osdmap e26: 4 total, 3 up, 4 in 2024-09-10T11:24:27.451 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:27 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:24:27.451 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:27 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:24:27.451 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:27 smithi178 ceph-mon[33514]: osd.3 [v2:172.21.15.178:6800/1366035688,v1:172.21.15.178:6801/1366035688] boot 2024-09-10T11:24:27.451 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:27 smithi178 ceph-mon[33514]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T11:24:27.451 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:27 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:24:27.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:27 smithi144 ceph-mon[27942]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T11:24:27.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:27 smithi144 ceph-mon[27942]: osdmap e26: 4 total, 3 up, 4 in 2024-09-10T11:24:27.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:27 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:24:27.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:27 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:24:27.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:27 smithi144 ceph-mon[27942]: osd.3 [v2:172.21.15.178:6800/1366035688,v1:172.21.15.178:6801/1366035688] boot 2024-09-10T11:24:27.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:27 smithi144 ceph-mon[27942]: osdmap e27: 4 total, 4 up, 4 in 2024-09-10T11:24:27.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:27 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:24:28.559 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:28 smithi178 ceph-mon[33514]: purged_snaps scrub starts 2024-09-10T11:24:28.559 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:28 smithi178 ceph-mon[33514]: purged_snaps scrub ok 2024-09-10T11:24:28.559 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:28 smithi178 ceph-mon[33514]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:28.559 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:28 smithi178 ceph-mon[33514]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T11:24:28.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:28 smithi144 ceph-mon[27942]: purged_snaps scrub starts 2024-09-10T11:24:28.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:28 smithi144 ceph-mon[27942]: purged_snaps scrub ok 2024-09-10T11:24:28.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:28 smithi144 ceph-mon[27942]: pgmap v95: 1 pgs: 1 peering; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:28.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:28 smithi144 ceph-mon[27942]: osdmap e28: 4 total, 4 up, 4 in 2024-09-10T11:24:29.219 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:24:29.239 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_3 2024-09-10T11:24:29.800 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:24:30.508 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:30 smithi178 ceph-mon[33514]: osdmap e29: 4 total, 4 up, 4 in 2024-09-10T11:24:30.509 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:30 smithi178 ceph-mon[33514]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:30.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:30 smithi144 ceph-mon[27942]: osdmap e29: 4 total, 4 up, 4 in 2024-09-10T11:24:30.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:30 smithi144 ceph-mon[27942]: pgmap v98: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:31.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:31 smithi178 ceph-mon[33514]: osdmap e30: 4 total, 4 up, 4 in 2024-09-10T11:24:31.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:31 smithi178 ceph-mon[33514]: Detected new or changed devices on smithi178 2024-09-10T11:24:31.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:31 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:31.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:31 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:31.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:31 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:24:31.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:31 smithi178 ceph-mon[33514]: Adjusting osd_memory_target on smithi178 to 14911M 2024-09-10T11:24:31.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:31 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:31.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:31 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:31.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:31 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:24:31.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:31 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:31.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:31 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:24:31.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:31 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:31.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:31 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:24:31.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:31 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:31.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:31 smithi144 ceph-mon[27942]: osdmap e30: 4 total, 4 up, 4 in 2024-09-10T11:24:31.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:31 smithi144 ceph-mon[27942]: Detected new or changed devices on smithi178 2024-09-10T11:24:31.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:31 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:31.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:31 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:31.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:31 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:24:31.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:31 smithi144 ceph-mon[27942]: Adjusting osd_memory_target on smithi178 to 14911M 2024-09-10T11:24:31.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:31 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:31.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:31 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:31.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:31 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:24:31.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:31 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:31.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:31 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:24:31.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:31 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:31.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:31 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:24:31.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:31 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:31.956 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.952+0000 7fdac5452700 1 -- 172.21.15.178:0/1941389263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac0100150 msgr2=0x7fdac0100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:24:31.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.952+0000 7fdac5452700 1 --2- 172.21.15.178:0/1941389263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac0100150 0x7fdac0100560 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8009910 tx=0x7fdaa8009c20 comp rx=0 tx=0).stop 2024-09-10T11:24:31.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.952+0000 7fdac5452700 1 -- 172.21.15.178:0/1941389263 shutdown_connections 2024-09-10T11:24:31.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.952+0000 7fdac5452700 1 --2- 172.21.15.178:0/1941389263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac0101350 0x7fdac01017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:31.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.952+0000 7fdac5452700 1 --2- 172.21.15.178:0/1941389263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac0100150 0x7fdac0100560 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:31.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.952+0000 7fdac5452700 1 -- 172.21.15.178:0/1941389263 >> 172.21.15.178:0/1941389263 conn(0x7fdac00fb6e0 msgr2=0x7fdac00fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:24:31.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.953+0000 7fdac5452700 1 -- 172.21.15.178:0/1941389263 shutdown_connections 2024-09-10T11:24:31.957 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.953+0000 7fdac5452700 1 -- 172.21.15.178:0/1941389263 wait complete. 2024-09-10T11:24:31.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.953+0000 7fdac5452700 1 Processor -- start 2024-09-10T11:24:31.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.953+0000 7fdac5452700 1 -- start start 2024-09-10T11:24:31.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.953+0000 7fdac5452700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac0100150 0x7fdac01959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:24:31.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.953+0000 7fdac5452700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac0101350 0x7fdac0195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:24:31.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.953+0000 7fdac5452700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac0196500 con 0x7fdac0101350 2024-09-10T11:24:31.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.953+0000 7fdac5452700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac0196640 con 0x7fdac0100150 2024-09-10T11:24:31.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.954+0000 7fdabeffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac0100150 0x7fdac01959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:24:31.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.954+0000 7fdabeffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac0100150 0x7fdac01959a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:42846/0 (socket says 172.21.15.178:42846) 2024-09-10T11:24:31.958 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.954+0000 7fdabeffd700 1 -- 172.21.15.178:0/4287705775 learned_addr learned my addr 172.21.15.178:0/4287705775 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:24:31.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.954+0000 7fdabe7fc700 1 --2- 172.21.15.178:0/4287705775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac0101350 0x7fdac0195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:24:31.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.954+0000 7fdabeffd700 1 -- 172.21.15.178:0/4287705775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac0101350 msgr2=0x7fdac0195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:24:31.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.954+0000 7fdabeffd700 1 --2- 172.21.15.178:0/4287705775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac0101350 0x7fdac0195ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:31.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.954+0000 7fdabeffd700 1 -- 172.21.15.178:0/4287705775 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa80095f0 con 0x7fdac0100150 2024-09-10T11:24:31.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.954+0000 7fdabe7fc700 1 --2- 172.21.15.178:0/4287705775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac0101350 0x7fdac0195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:24:31.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.954+0000 7fdabeffd700 1 --2- 172.21.15.178:0/4287705775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac0100150 0x7fdac01959a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800bda0 tx=0x7fdaa800be80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:24:31.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.954+0000 7fdab7fff700 1 -- 172.21.15.178:0/4287705775 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaa801d070 con 0x7fdac0100150 2024-09-10T11:24:31.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.954+0000 7fdab7fff700 1 -- 172.21.15.178:0/4287705775 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdaa8004c50 con 0x7fdac0100150 2024-09-10T11:24:31.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.954+0000 7fdac5452700 1 -- 172.21.15.178:0/4287705775 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac019b090 con 0x7fdac0100150 2024-09-10T11:24:31.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.954+0000 7fdab7fff700 1 -- 172.21.15.178:0/4287705775 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaa800f690 con 0x7fdac0100150 2024-09-10T11:24:31.959 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.954+0000 7fdac5452700 1 -- 172.21.15.178:0/4287705775 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac019b580 con 0x7fdac0100150 2024-09-10T11:24:31.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.955+0000 7fdac5452700 1 -- 172.21.15.178:0/4287705775 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac0108ca0 con 0x7fdac0100150 2024-09-10T11:24:31.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.956+0000 7fdab7fff700 1 -- 172.21.15.178:0/4287705775 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fdaa80229b0 con 0x7fdac0100150 2024-09-10T11:24:31.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.956+0000 7fdab7fff700 1 --2- 172.21.15.178:0/4287705775 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdaac06c680 0x7fdaac06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:24:31.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.956+0000 7fdab7fff700 1 -- 172.21.15.178:0/4287705775 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 3602+0+0 (secure 0 0 0) 0x7fdaa808c890 con 0x7fdac0100150 2024-09-10T11:24:31.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.956+0000 7fdabe7fc700 1 --2- 172.21.15.178:0/4287705775 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdaac06c680 0x7fdaac06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:24:31.961 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.957+0000 7fdabe7fc700 1 --2- 172.21.15.178:0/4287705775 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdaac06c680 0x7fdaac06eb30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdab0005950 tx=0x7fdab0005f50 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:24:31.963 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:31.961+0000 7fdab7fff700 1 -- 172.21.15.178:0/4287705775 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdaa805b1c0 con 0x7fdac0100150 2024-09-10T11:24:32.158 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:32.156+0000 7fdac5452700 1 -- 172.21.15.178:0/4287705775 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fdac0105ca0 con 0x7fdaac06c680 2024-09-10T11:24:32.614 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:32 smithi178 ceph-mon[33514]: osdmap e31: 4 total, 4 up, 4 in 2024-09-10T11:24:32.614 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:32 smithi178 ceph-mon[33514]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:32.615 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:32 smithi178 ceph-mon[33514]: from='client.24159 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:24:32.615 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:32 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T11:24:32.615 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:32 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T11:24:32.615 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:32 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:32.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:32 smithi144 ceph-mon[27942]: osdmap e31: 4 total, 4 up, 4 in 2024-09-10T11:24:32.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:32 smithi144 ceph-mon[27942]: pgmap v101: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:32.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:32 smithi144 ceph-mon[27942]: from='client.24159 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:24:32.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:32 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T11:24:32.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:32 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T11:24:32.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:32 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:33.576 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:33 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:24:33.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:33 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:24:34.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:34 smithi178 ceph-mon[33514]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-10T11:24:34.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:34 smithi144 ceph-mon[27942]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 109 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-10T11:24:35.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:35 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/2507169751' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc5df32a-01cd-45af-8a8e-9759fe330737"}]: dispatch 2024-09-10T11:24:35.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:35 smithi178 ceph-mon[33514]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc5df32a-01cd-45af-8a8e-9759fe330737"}]: dispatch 2024-09-10T11:24:35.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:35 smithi178 ceph-mon[33514]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc5df32a-01cd-45af-8a8e-9759fe330737"}]': finished 2024-09-10T11:24:35.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:35 smithi178 ceph-mon[33514]: osdmap e32: 5 total, 4 up, 5 in 2024-09-10T11:24:35.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:35 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:24:35.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:35 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/2507169751' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc5df32a-01cd-45af-8a8e-9759fe330737"}]: dispatch 2024-09-10T11:24:35.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:35 smithi144 ceph-mon[27942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bc5df32a-01cd-45af-8a8e-9759fe330737"}]: dispatch 2024-09-10T11:24:35.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:35 smithi144 ceph-mon[27942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bc5df32a-01cd-45af-8a8e-9759fe330737"}]': finished 2024-09-10T11:24:35.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:35 smithi144 ceph-mon[27942]: osdmap e32: 5 total, 4 up, 5 in 2024-09-10T11:24:35.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:35 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:24:36.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:36 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/741708699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T11:24:36.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:36 smithi178 ceph-mon[33514]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T11:24:36.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:36 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/741708699' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T11:24:36.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:36 smithi144 ceph-mon[27942]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T11:24:38.594 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:38 smithi178 ceph-mon[33514]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-10T11:24:38.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:38 smithi144 ceph-mon[27942]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 76 KiB/s, 0 objects/s recovering 2024-09-10T11:24:41.210 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:40 smithi178 ceph-mon[33514]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-10T11:24:41.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:40 smithi144 ceph-mon[27942]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 67 KiB/s, 0 objects/s recovering 2024-09-10T11:24:42.318 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:41 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T11:24:42.319 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:41 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:42.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:41 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T11:24:42.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:41 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:43.240 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:42 smithi178 ceph-mon[33514]: Deploying daemon osd.4 on smithi178 2024-09-10T11:24:43.240 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:42 smithi178 ceph-mon[33514]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T11:24:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:42 smithi144 ceph-mon[27942]: Deploying daemon osd.4 on smithi178 2024-09-10T11:24:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:42 smithi144 ceph-mon[27942]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T11:24:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:44 smithi178 ceph-mon[33514]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:44 smithi144 ceph-mon[27942]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:46 smithi178 ceph-mon[33514]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:46 smithi144 ceph-mon[27942]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:48.321 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:47 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:24:48.321 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:47 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:48.322 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:47 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:48.322 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:47 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:24:48.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:47 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:24:48.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:47 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:48.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:47 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:48.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:47 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:24:49.226 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:48 smithi178 ceph-mon[33514]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:49.235 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:48 smithi144 ceph-mon[27942]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:50.168 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:50.166+0000 7fdab7fff700 1 -- 172.21.15.178:0/4287705775 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdac0105ca0 con 0x7fdaac06c680 2024-09-10T11:24:50.168 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 4 on host 'smithi178' 2024-09-10T11:24:50.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:50.169+0000 7fdac5452700 1 -- 172.21.15.178:0/4287705775 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdaac06c680 msgr2=0x7fdaac06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:24:50.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:50.169+0000 7fdac5452700 1 --2- 172.21.15.178:0/4287705775 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdaac06c680 0x7fdaac06eb30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fdab0005950 tx=0x7fdab0005f50 comp rx=0 tx=0).stop 2024-09-10T11:24:50.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:50.169+0000 7fdac5452700 1 -- 172.21.15.178:0/4287705775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac0100150 msgr2=0x7fdac01959a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:24:50.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:50.169+0000 7fdac5452700 1 --2- 172.21.15.178:0/4287705775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac0100150 0x7fdac01959a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fdaa800bda0 tx=0x7fdaa800be80 comp rx=0 tx=0).stop 2024-09-10T11:24:50.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:50.169+0000 7fdac5452700 1 -- 172.21.15.178:0/4287705775 shutdown_connections 2024-09-10T11:24:50.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:50.169+0000 7fdac5452700 1 --2- 172.21.15.178:0/4287705775 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdaac06c680 0x7fdaac06eb30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:50.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:50.169+0000 7fdac5452700 1 --2- 172.21.15.178:0/4287705775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdac0100150 0x7fdac01959a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:50.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:50.169+0000 7fdac5452700 1 --2- 172.21.15.178:0/4287705775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdac0101350 0x7fdac0195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:50.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:50.169+0000 7fdac5452700 1 -- 172.21.15.178:0/4287705775 >> 172.21.15.178:0/4287705775 conn(0x7fdac00fb6e0 msgr2=0x7fdac0104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:24:50.170 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:50.169+0000 7fdac5452700 1 -- 172.21.15.178:0/4287705775 shutdown_connections 2024-09-10T11:24:50.171 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:50.169+0000 7fdac5452700 1 -- 172.21.15.178:0/4287705775 wait complete. 2024-09-10T11:24:50.746 DEBUG:teuthology.orchestra.run.smithi178:osd.4> sudo journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T11:24:50.748 INFO:tasks.cephadm:Deploying osd.5 on smithi178 with /dev/vg_nvme/lv_2... 2024-09-10T11:24:50.748 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-10T11:24:51.013 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:24:50 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[44175]: 2024-09-10T11:24:50.868+0000 7f887870d640 -1 osd.4 0 log_to_monitors true 2024-09-10T11:24:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:51 smithi178 ceph-mon[33514]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:51 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:51 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:51 smithi178 ceph-mon[33514]: from='osd.4 [v2:172.21.15.178:6808/280235677,v1:172.21.15.178:6809/280235677]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T11:24:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:51 smithi178 ceph-mon[33514]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T11:24:51.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:51 smithi144 ceph-mon[27942]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:51.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:51 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:51.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:51 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:51.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:51 smithi144 ceph-mon[27942]: from='osd.4 [v2:172.21.15.178:6808/280235677,v1:172.21.15.178:6809/280235677]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T11:24:51.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:51 smithi144 ceph-mon[27942]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T11:24:51.502 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:24:52.277 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:24:52 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[44175]: 2024-09-10T11:24:52.134+0000 7f886d583700 -1 osd.4 0 waiting for initial osdmap 2024-09-10T11:24:52.277 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:24:52 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[44175]: 2024-09-10T11:24:52.140+0000 7f8866372700 -1 osd.4 34 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:24:52.277 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:52 smithi178 ceph-mon[33514]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T11:24:52.277 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:52 smithi178 ceph-mon[33514]: osdmap e33: 5 total, 4 up, 5 in 2024-09-10T11:24:52.277 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:52 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:24:52.277 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:52 smithi178 ceph-mon[33514]: from='osd.4 [v2:172.21.15.178:6808/280235677,v1:172.21.15.178:6809/280235677]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:24:52.278 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:52 smithi178 ceph-mon[33514]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:24:52.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:52 smithi144 ceph-mon[27942]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T11:24:52.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:52 smithi144 ceph-mon[27942]: osdmap e33: 5 total, 4 up, 5 in 2024-09-10T11:24:52.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:52 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:24:52.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:52 smithi144 ceph-mon[27942]: from='osd.4 [v2:172.21.15.178:6808/280235677,v1:172.21.15.178:6809/280235677]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:24:52.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:52 smithi144 ceph-mon[27942]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:24:53.404 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:53 smithi178 ceph-mon[33514]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:53.404 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:53 smithi178 ceph-mon[33514]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T11:24:53.404 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:53 smithi178 ceph-mon[33514]: osdmap e34: 5 total, 4 up, 5 in 2024-09-10T11:24:53.404 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:53 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:24:53.404 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:53 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:24:53.405 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:53 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:53.405 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:53 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:53.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:53 smithi144 ceph-mon[27942]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-09-10T11:24:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:53 smithi144 ceph-mon[27942]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T11:24:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:53 smithi144 ceph-mon[27942]: osdmap e34: 5 total, 4 up, 5 in 2024-09-10T11:24:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:53 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:24:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:53 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:24:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:53 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:53 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:54 smithi178 ceph-mon[33514]: purged_snaps scrub starts 2024-09-10T11:24:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:54 smithi178 ceph-mon[33514]: purged_snaps scrub ok 2024-09-10T11:24:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:54 smithi178 ceph-mon[33514]: osd.4 [v2:172.21.15.178:6808/280235677,v1:172.21.15.178:6809/280235677] boot 2024-09-10T11:24:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:54 smithi178 ceph-mon[33514]: osdmap e35: 5 total, 5 up, 5 in 2024-09-10T11:24:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:54 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:24:54.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:54 smithi144 ceph-mon[27942]: purged_snaps scrub starts 2024-09-10T11:24:54.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:54 smithi144 ceph-mon[27942]: purged_snaps scrub ok 2024-09-10T11:24:54.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:54 smithi144 ceph-mon[27942]: osd.4 [v2:172.21.15.178:6808/280235677,v1:172.21.15.178:6809/280235677] boot 2024-09-10T11:24:54.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:54 smithi144 ceph-mon[27942]: osdmap e35: 5 total, 5 up, 5 in 2024-09-10T11:24:54.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:54 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:24:54.728 INFO:teuthology.orchestra.run.smithi178.stdout: 2024-09-10T11:24:54.747 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi178:vg_nvme/lv_2 2024-09-10T11:24:55.272 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:24:55.379 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:55 smithi178 ceph-mon[33514]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:24:55.379 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:55 smithi178 ceph-mon[33514]: osdmap e36: 5 total, 5 up, 5 in 2024-09-10T11:24:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:55 smithi144 ceph-mon[27942]: pgmap v116: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:24:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:55 smithi144 ceph-mon[27942]: osdmap e36: 5 total, 5 up, 5 in 2024-09-10T11:24:57.028 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.025+0000 7f775323e700 1 -- 172.21.15.178:0/2232768995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f774c0ffa40 msgr2=0x7f774c0ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:24:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.025+0000 7f775323e700 1 --2- 172.21.15.178:0/2232768995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f774c0ffa40 0x7f774c0ffe90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7738009910 tx=0x7f7738009c20 comp rx=0 tx=0).stop 2024-09-10T11:24:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.025+0000 7f775323e700 1 -- 172.21.15.178:0/2232768995 shutdown_connections 2024-09-10T11:24:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.025+0000 7f775323e700 1 --2- 172.21.15.178:0/2232768995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f774c0ffa40 0x7f774c0ffe90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.025+0000 7f775323e700 1 --2- 172.21.15.178:0/2232768995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f774c0fe840 0x7f774c0fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.025+0000 7f775323e700 1 -- 172.21.15.178:0/2232768995 >> 172.21.15.178:0/2232768995 conn(0x7f774c0f9dd0 msgr2=0x7f774c0fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:24:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.025+0000 7f775323e700 1 -- 172.21.15.178:0/2232768995 shutdown_connections 2024-09-10T11:24:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.025+0000 7f775323e700 1 -- 172.21.15.178:0/2232768995 wait complete. 2024-09-10T11:24:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.026+0000 7f775323e700 1 Processor -- start 2024-09-10T11:24:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.026+0000 7f775323e700 1 -- start start 2024-09-10T11:24:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.026+0000 7f775323e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f774c0fe840 0x7f774c1017e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:24:57.029 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.026+0000 7f775323e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f774c0ffa40 0x7f774c19a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:24:57.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.026+0000 7f775323e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c19ad00 con 0x7f774c0ffa40 2024-09-10T11:24:57.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.026+0000 7f775323e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774c19ae70 con 0x7f774c0fe840 2024-09-10T11:24:57.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.026+0000 7f7750fda700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f774c0fe840 0x7f774c1017e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:24:57.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.026+0000 7f7750fda700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f774c0fe840 0x7f774c1017e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:56490/0 (socket says 172.21.15.178:56490) 2024-09-10T11:24:57.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.026+0000 7f7750fda700 1 -- 172.21.15.178:0/591805747 learned_addr learned my addr 172.21.15.178:0/591805747 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:24:57.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.027+0000 7f774bfff700 1 --2- 172.21.15.178:0/591805747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f774c0ffa40 0x7f774c19a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:24:57.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.027+0000 7f7750fda700 1 -- 172.21.15.178:0/591805747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f774c0ffa40 msgr2=0x7f774c19a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:24:57.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.027+0000 7f7750fda700 1 --2- 172.21.15.178:0/591805747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f774c0ffa40 0x7f774c19a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:24:57.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.027+0000 7f7750fda700 1 -- 172.21.15.178:0/591805747 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77380095f0 con 0x7f774c0fe840 2024-09-10T11:24:57.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.027+0000 7f7750fda700 1 --2- 172.21.15.178:0/591805747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f774c0fe840 0x7f774c1017e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f774000d900 tx=0x7f774000dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:24:57.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.027+0000 7f7749ffb700 1 -- 172.21.15.178:0/591805747 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f774000b920 con 0x7f774c0fe840 2024-09-10T11:24:57.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.027+0000 7f775323e700 1 -- 172.21.15.178:0/591805747 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f774c19b150 con 0x7f774c0fe840 2024-09-10T11:24:57.030 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.027+0000 7f7749ffb700 1 -- 172.21.15.178:0/591805747 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7740010460 con 0x7f774c0fe840 2024-09-10T11:24:57.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.027+0000 7f7749ffb700 1 -- 172.21.15.178:0/591805747 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f774000f590 con 0x7f774c0fe840 2024-09-10T11:24:57.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.027+0000 7f775323e700 1 -- 172.21.15.178:0/591805747 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f774c19b6a0 con 0x7f774c0fe840 2024-09-10T11:24:57.031 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.028+0000 7f775323e700 1 -- 172.21.15.178:0/591805747 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f774c107390 con 0x7f774c0fe840 2024-09-10T11:24:57.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.029+0000 7f7749ffb700 1 -- 172.21.15.178:0/591805747 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f77400105d0 con 0x7f774c0fe840 2024-09-10T11:24:57.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.029+0000 7f7749ffb700 1 --2- 172.21.15.178:0/591805747 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f773c06c680 0x7f773c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:24:57.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.029+0000 7f7749ffb700 1 -- 172.21.15.178:0/591805747 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 4034+0+0 (secure 0 0 0) 0x7f774008af80 con 0x7f774c0fe840 2024-09-10T11:24:57.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.029+0000 7f774bfff700 1 --2- 172.21.15.178:0/591805747 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f773c06c680 0x7f773c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:24:57.032 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.030+0000 7f774bfff700 1 --2- 172.21.15.178:0/591805747 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f773c06c680 0x7f773c06eb30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7738005090 tx=0x7f773801a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:24:57.034 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.033+0000 7f7749ffb700 1 -- 172.21.15.178:0/591805747 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7740059680 con 0x7f774c0fe840 2024-09-10T11:24:57.238 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:24:57.237+0000 7f775323e700 1 -- 172.21.15.178:0/591805747 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f774c103d80 con 0x7f773c06c680 2024-09-10T11:24:57.239 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:57 smithi178 ceph-mon[33514]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:24:57.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:57 smithi144 ceph-mon[27942]: pgmap v118: 1 pgs: 1 peering; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:24:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='client.24177 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:24:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T11:24:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T11:24:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:24:58.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:24:58.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:58.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:58.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:24:58.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:58.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:24:58.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:58.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:24:58.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:58 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:58.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='client.24177 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi178:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:24:58.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T11:24:58.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T11:24:58.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:58.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:58.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:58.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:24:58.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:24:58.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:58.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:58.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:24:58.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:58.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:24:58.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:24:58.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:24:58.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:58 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:24:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:59 smithi178 ceph-mon[33514]: Detected new or changed devices on smithi178 2024-09-10T11:24:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:59 smithi178 ceph-mon[33514]: Adjusting osd_memory_target on smithi178 to 7455M 2024-09-10T11:24:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:24:59 smithi178 ceph-mon[33514]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:24:59.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:59 smithi144 ceph-mon[27942]: Detected new or changed devices on smithi178 2024-09-10T11:24:59.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:59 smithi144 ceph-mon[27942]: Adjusting osd_memory_target on smithi178 to 7455M 2024-09-10T11:24:59.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:24:59 smithi144 ceph-mon[27942]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:00 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/3226318744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52ee296f-c365-4e5b-b18a-a281a0f58e04"}]: dispatch 2024-09-10T11:25:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:00 smithi144 ceph-mon[27942]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52ee296f-c365-4e5b-b18a-a281a0f58e04"}]: dispatch 2024-09-10T11:25:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:00 smithi144 ceph-mon[27942]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52ee296f-c365-4e5b-b18a-a281a0f58e04"}]': finished 2024-09-10T11:25:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:00 smithi144 ceph-mon[27942]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T11:25:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:00 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:25:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:00 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/3226318744' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52ee296f-c365-4e5b-b18a-a281a0f58e04"}]: dispatch 2024-09-10T11:25:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:00 smithi178 ceph-mon[33514]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "52ee296f-c365-4e5b-b18a-a281a0f58e04"}]: dispatch 2024-09-10T11:25:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:00 smithi178 ceph-mon[33514]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "52ee296f-c365-4e5b-b18a-a281a0f58e04"}]': finished 2024-09-10T11:25:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:00 smithi178 ceph-mon[33514]: osdmap e37: 6 total, 5 up, 6 in 2024-09-10T11:25:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:00 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:25:01.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:01 smithi144 ceph-mon[27942]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:01.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:01 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/43079150' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T11:25:01.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:01 smithi178 ceph-mon[33514]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:01.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:01 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/43079150' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T11:25:03.418 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:03 smithi178 ceph-mon[33514]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:03.418 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:03 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:25:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:03 smithi144 ceph-mon[27942]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:25:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:05 smithi178 ceph-mon[33514]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:05.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:05 smithi144 ceph-mon[27942]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:07.408 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:07 smithi178 ceph-mon[33514]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:07.408 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:07 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T11:25:07.408 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:07 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:25:07.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:07 smithi144 ceph-mon[27942]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:07.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T11:25:07.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:25:08.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:08 smithi144 ceph-mon[27942]: Deploying daemon osd.5 on smithi178 2024-09-10T11:25:08.466 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:08 smithi178 ceph-mon[33514]: Deploying daemon osd.5 on smithi178 2024-09-10T11:25:09.430 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:09 smithi178 ceph-mon[33514]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:09.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:09 smithi144 ceph-mon[27942]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:11.407 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:11 smithi178 ceph-mon[33514]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:11.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:11 smithi144 ceph-mon[27942]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:12 smithi178 ceph-mon[33514]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:12 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:12 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:12 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:25:13.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:12 smithi144 ceph-mon[27942]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:13.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:12 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:13.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:12 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:13.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:12 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:25:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:14 smithi178 ceph-mon[33514]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:15.377 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:14 smithi144 ceph-mon[27942]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:15.555 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:15.553+0000 7f7749ffb700 1 -- 172.21.15.178:0/591805747 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f774c103d80 con 0x7f773c06c680 2024-09-10T11:25:15.555 INFO:teuthology.orchestra.run.smithi178.stdout:Created osd(s) 5 on host 'smithi178' 2024-09-10T11:25:15.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:15.555+0000 7f775323e700 1 -- 172.21.15.178:0/591805747 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f773c06c680 msgr2=0x7f773c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:15.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:15.555+0000 7f775323e700 1 --2- 172.21.15.178:0/591805747 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f773c06c680 0x7f773c06eb30 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7738005090 tx=0x7f773801a040 comp rx=0 tx=0).stop 2024-09-10T11:25:15.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:15.556+0000 7f775323e700 1 -- 172.21.15.178:0/591805747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f774c0fe840 msgr2=0x7f774c1017e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:15.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:15.556+0000 7f775323e700 1 --2- 172.21.15.178:0/591805747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f774c0fe840 0x7f774c1017e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f774000d900 tx=0x7f774000dc10 comp rx=0 tx=0).stop 2024-09-10T11:25:15.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:15.556+0000 7f775323e700 1 -- 172.21.15.178:0/591805747 shutdown_connections 2024-09-10T11:25:15.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:15.556+0000 7f775323e700 1 --2- 172.21.15.178:0/591805747 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f773c06c680 0x7f773c06eb30 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:15.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:15.556+0000 7f775323e700 1 --2- 172.21.15.178:0/591805747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f774c0fe840 0x7f774c1017e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:15.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:15.556+0000 7f775323e700 1 --2- 172.21.15.178:0/591805747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f774c0ffa40 0x7f774c19a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:15.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:15.556+0000 7f775323e700 1 -- 172.21.15.178:0/591805747 >> 172.21.15.178:0/591805747 conn(0x7f774c0f9dd0 msgr2=0x7f774c0fc140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:15.557 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:15.556+0000 7f775323e700 1 -- 172.21.15.178:0/591805747 shutdown_connections 2024-09-10T11:25:15.558 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:15.556+0000 7f775323e700 1 -- 172.21.15.178:0/591805747 wait complete. 2024-09-10T11:25:16.594 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:16.594 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:16.594 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:16 smithi178 ceph-mon[33514]: from='osd.5 [v2:172.21.15.178:6816/3024778612,v1:172.21.15.178:6817/3024778612]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T11:25:16.594 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:16 smithi178 ceph-mon[33514]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T11:25:16.594 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:16 smithi178 ceph-mon[33514]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:16.614 DEBUG:teuthology.orchestra.run.smithi178:osd.5> sudo journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T11:25:16.616 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-09-10T11:25:16.616 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T11:25:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:16.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:16.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:16.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:16 smithi144 ceph-mon[27942]: from='osd.5 [v2:172.21.15.178:6816/3024778612,v1:172.21.15.178:6817/3024778612]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T11:25:16.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:16 smithi144 ceph-mon[27942]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T11:25:16.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:16 smithi144 ceph-mon[27942]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:17.753 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:17 smithi178 ceph-mon[33514]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T11:25:17.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:17 smithi178 ceph-mon[33514]: osdmap e38: 6 total, 5 up, 6 in 2024-09-10T11:25:17.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:17 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:25:17.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:17 smithi178 ceph-mon[33514]: from='osd.5 [v2:172.21.15.178:6816/3024778612,v1:172.21.15.178:6817/3024778612]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:25:17.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:17 smithi178 ceph-mon[33514]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:25:17.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:17 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:17.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:17 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:17.754 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:25:17 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[48821]: 2024-09-10T11:25:17.554+0000 7f1d310d6700 -1 osd.5 0 waiting for initial osdmap 2024-09-10T11:25:17.754 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:25:17 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[48821]: 2024-09-10T11:25:17.562+0000 7f1d2bec9700 -1 osd.5 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:25:17.855 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:17 smithi144 ceph-mon[27942]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T11:25:17.856 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:17 smithi144 ceph-mon[27942]: osdmap e38: 6 total, 5 up, 6 in 2024-09-10T11:25:17.856 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:17 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:25:17.856 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:17 smithi144 ceph-mon[27942]: from='osd.5 [v2:172.21.15.178:6816/3024778612,v1:172.21.15.178:6817/3024778612]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:25:17.856 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:17 smithi144 ceph-mon[27942]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:25:17.856 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:17 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:17.856 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:17 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:18.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.410+0000 7f68778dd700 1 -- 172.21.15.144:0/2716686805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68700ff060 msgr2=0x7f68700ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:18.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.410+0000 7f68778dd700 1 --2- 172.21.15.144:0/2716686805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68700ff060 0x7f68700ff470 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6860009910 tx=0x7f6860009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:18.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.411+0000 7f68778dd700 1 -- 172.21.15.144:0/2716686805 shutdown_connections 2024-09-10T11:25:18.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.411+0000 7f68778dd700 1 --2- 172.21.15.144:0/2716686805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6870100260 0x7f68701006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:18.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.411+0000 7f68778dd700 1 --2- 172.21.15.144:0/2716686805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68700ff060 0x7f68700ff470 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:18.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.411+0000 7f68778dd700 1 -- 172.21.15.144:0/2716686805 >> 172.21.15.144:0/2716686805 conn(0x7f68700fa610 msgr2=0x7f68700fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:18.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.411+0000 7f68778dd700 1 -- 172.21.15.144:0/2716686805 shutdown_connections 2024-09-10T11:25:18.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.411+0000 7f68778dd700 1 -- 172.21.15.144:0/2716686805 wait complete. 2024-09-10T11:25:18.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.411+0000 7f68778dd700 1 Processor -- start 2024-09-10T11:25:18.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.411+0000 7f68778dd700 1 -- start start 2024-09-10T11:25:18.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.411+0000 7f68778dd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68700ff060 0x7f6870069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:18.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.411+0000 7f68778dd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6870100260 0x7f687019a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:18.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.411+0000 7f68778dd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687019aca0 con 0x7f68700ff060 2024-09-10T11:25:18.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.411+0000 7f68778dd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687019ade0 con 0x7f6870100260 2024-09-10T11:25:18.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.411+0000 7f6875679700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68700ff060 0x7f6870069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:18.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.412+0000 7f6874e78700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6870100260 0x7f687019a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:18.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.412+0000 7f6875679700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68700ff060 0x7f6870069930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55590/0 (socket says 172.21.15.144:55590) 2024-09-10T11:25:18.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.412+0000 7f6875679700 1 -- 172.21.15.144:0/131830749 learned_addr learned my addr 172.21.15.144:0/131830749 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:18.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.412+0000 7f6875679700 1 -- 172.21.15.144:0/131830749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6870100260 msgr2=0x7f687019a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:18.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.412+0000 7f6875679700 1 --2- 172.21.15.144:0/131830749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6870100260 0x7f687019a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:18.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.412+0000 7f6875679700 1 -- 172.21.15.144:0/131830749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68600095f0 con 0x7f68700ff060 2024-09-10T11:25:18.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.412+0000 7f6874e78700 1 --2- 172.21.15.144:0/131830749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6870100260 0x7f687019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:25:18.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.412+0000 7f6875679700 1 --2- 172.21.15.144:0/131830749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68700ff060 0x7f6870069930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f686000bda0 tx=0x7f686000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:18.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.412+0000 7f68667fc700 1 -- 172.21.15.144:0/131830749 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f686001d070 con 0x7f68700ff060 2024-09-10T11:25:18.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.412+0000 7f68667fc700 1 -- 172.21.15.144:0/131830749 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f686000f460 con 0x7f68700ff060 2024-09-10T11:25:18.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.412+0000 7f68778dd700 1 -- 172.21.15.144:0/131830749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687019b060 con 0x7f68700ff060 2024-09-10T11:25:18.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.412+0000 7f68667fc700 1 -- 172.21.15.144:0/131830749 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6860021620 con 0x7f68700ff060 2024-09-10T11:25:18.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.412+0000 7f68778dd700 1 -- 172.21.15.144:0/131830749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687019b550 con 0x7f68700ff060 2024-09-10T11:25:18.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.413+0000 7f68778dd700 1 -- 172.21.15.144:0/131830749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6870107bb0 con 0x7f68700ff060 2024-09-10T11:25:18.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.414+0000 7f68667fc700 1 -- 172.21.15.144:0/131830749 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f686000f5d0 con 0x7f68700ff060 2024-09-10T11:25:18.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.414+0000 7f68667fc700 1 --2- 172.21.15.144:0/131830749 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f685c06c680 0x7f685c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:18.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.414+0000 7f68667fc700 1 -- 172.21.15.144:0/131830749 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4186+0+0 (secure 0 0 0) 0x7f686008c6a0 con 0x7f68700ff060 2024-09-10T11:25:18.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.414+0000 7f6874e78700 1 --2- 172.21.15.144:0/131830749 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f685c06c680 0x7f685c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:18.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.414+0000 7f6874e78700 1 --2- 172.21.15.144:0/131830749 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f685c06c680 0x7f685c06eb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f686c005950 tx=0x7f686c00b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:18.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.419+0000 7f68667fc700 1 -- 172.21.15.144:0/131830749 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f686005ad10 con 0x7f68700ff060 2024-09-10T11:25:18.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.611+0000 7f68778dd700 1 -- 172.21.15.144:0/131830749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f687019bc50 con 0x7f68700ff060 2024-09-10T11:25:18.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.612+0000 7f68667fc700 1 -- 172.21.15.144:0/131830749 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v40) v1 ==== 74+0+130 (secure 0 0 0) 0x7f686005a8a0 con 0x7f68700ff060 2024-09-10T11:25:18.614 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:25:18.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.614+0000 7f68778dd700 1 -- 172.21.15.144:0/131830749 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f685c06c680 msgr2=0x7f685c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:18.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.614+0000 7f68778dd700 1 --2- 172.21.15.144:0/131830749 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f685c06c680 0x7f685c06eb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f686c005950 tx=0x7f686c00b410 comp rx=0 tx=0).stop 2024-09-10T11:25:18.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.614+0000 7f68778dd700 1 -- 172.21.15.144:0/131830749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68700ff060 msgr2=0x7f6870069930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:18.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.614+0000 7f68778dd700 1 --2- 172.21.15.144:0/131830749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68700ff060 0x7f6870069930 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f686000bda0 tx=0x7f686000bdd0 comp rx=0 tx=0).stop 2024-09-10T11:25:18.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.614+0000 7f68778dd700 1 -- 172.21.15.144:0/131830749 shutdown_connections 2024-09-10T11:25:18.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.614+0000 7f68778dd700 1 --2- 172.21.15.144:0/131830749 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f685c06c680 0x7f685c06eb30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:18.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.614+0000 7f68778dd700 1 --2- 172.21.15.144:0/131830749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68700ff060 0x7f6870069930 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:18.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.614+0000 7f68778dd700 1 --2- 172.21.15.144:0/131830749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6870100260 0x7f687019a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:18.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.614+0000 7f68778dd700 1 -- 172.21.15.144:0/131830749 >> 172.21.15.144:0/131830749 conn(0x7f68700fa610 msgr2=0x7f6870103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:18.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.614+0000 7f68778dd700 1 -- 172.21.15.144:0/131830749 shutdown_connections 2024-09-10T11:25:18.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:18.614+0000 7f68778dd700 1 -- 172.21.15.144:0/131830749 wait complete. 2024-09-10T11:25:18.807 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:18 smithi178 ceph-mon[33514]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T11:25:18.807 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:18 smithi178 ceph-mon[33514]: osdmap e39: 6 total, 5 up, 6 in 2024-09-10T11:25:18.807 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:18 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:25:18.808 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:18 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:25:18.808 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:18 smithi178 ceph-mon[33514]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:18.808 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:18 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:25:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:18 smithi144 ceph-mon[27942]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]': finished 2024-09-10T11:25:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:18 smithi144 ceph-mon[27942]: osdmap e39: 6 total, 5 up, 6 in 2024-09-10T11:25:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:18 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:25:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:18 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:25:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:18 smithi144 ceph-mon[27942]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-09-10T11:25:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:18 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:25:19.260 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":40,"num_osds":6,"num_up_osds":6,"osd_up_since":1725967518,"num_in_osds":6,"osd_in_since":1725967499,"num_remapped_pgs":0} 2024-09-10T11:25:19.260 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T11:25:19.472 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:19.809 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:19 smithi144 ceph-mon[27942]: purged_snaps scrub starts 2024-09-10T11:25:19.810 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:19 smithi144 ceph-mon[27942]: purged_snaps scrub ok 2024-09-10T11:25:19.810 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:19 smithi144 ceph-mon[27942]: osd.5 [v2:172.21.15.178:6816/3024778612,v1:172.21.15.178:6817/3024778612] boot 2024-09-10T11:25:19.810 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:19 smithi144 ceph-mon[27942]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T11:25:19.810 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:19 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:25:19.810 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:19 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/131830749' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T11:25:19.845 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:19 smithi178 ceph-mon[33514]: purged_snaps scrub starts 2024-09-10T11:25:19.846 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:19 smithi178 ceph-mon[33514]: purged_snaps scrub ok 2024-09-10T11:25:19.846 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:19 smithi178 ceph-mon[33514]: osd.5 [v2:172.21.15.178:6816/3024778612,v1:172.21.15.178:6817/3024778612] boot 2024-09-10T11:25:19.846 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:19 smithi178 ceph-mon[33514]: osdmap e40: 6 total, 6 up, 6 in 2024-09-10T11:25:19.846 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:19 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:25:19.846 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:19 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/131830749' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T11:25:20.834 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T11:25:20.834 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:20.834 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: Detected new or changed devices on smithi178 2024-09-10T11:25:20.835 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:20.835 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:20.835 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:25:20.835 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:25:20.835 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:25:20.835 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: Adjusting osd_memory_target on smithi178 to 4970M 2024-09-10T11:25:20.835 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:20.835 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:25:20.835 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:25:20.835 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:20.835 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:25:20.835 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:25:20.835 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:25:20.836 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:20 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: osdmap e41: 6 total, 6 up, 6 in 2024-09-10T11:25:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 161 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: Detected new or changed devices on smithi178 2024-09-10T11:25:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:20.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:25:20.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:25:20.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:25:20.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: Adjusting osd_memory_target on smithi178 to 4970M 2024-09-10T11:25:20.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:20.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:25:20.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:25:20.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:20.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:25:20.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:25:20.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:25:20.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:20 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:25:21.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.179+0000 7fa4152fa700 1 -- 172.21.15.144:0/3399261488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4100ff2c0 msgr2=0x7fa4100ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:21.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.179+0000 7fa4152fa700 1 --2- 172.21.15.144:0/3399261488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4100ff2c0 0x7fa4100ff690 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa404009910 tx=0x7fa404009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:21.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.179+0000 7fa4152fa700 1 -- 172.21.15.144:0/3399261488 shutdown_connections 2024-09-10T11:25:21.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.179+0000 7fa4152fa700 1 --2- 172.21.15.144:0/3399261488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4100ffc60 0x7fa410103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:21.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.179+0000 7fa4152fa700 1 --2- 172.21.15.144:0/3399261488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4100ff2c0 0x7fa4100ff690 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:21.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.179+0000 7fa4152fa700 1 -- 172.21.15.144:0/3399261488 >> 172.21.15.144:0/3399261488 conn(0x7fa4100fab80 msgr2=0x7fa4100fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:21.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.179+0000 7fa4152fa700 1 -- 172.21.15.144:0/3399261488 shutdown_connections 2024-09-10T11:25:21.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.179+0000 7fa4152fa700 1 -- 172.21.15.144:0/3399261488 wait complete. 2024-09-10T11:25:21.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.180+0000 7fa4152fa700 1 Processor -- start 2024-09-10T11:25:21.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.180+0000 7fa4152fa700 1 -- start start 2024-09-10T11:25:21.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.180+0000 7fa4152fa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4100ff2c0 0x7fa410100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:21.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.180+0000 7fa4152fa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4100ffc60 0x7fa410101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:21.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.180+0000 7fa4152fa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa410101750 con 0x7fa4100ff2c0 2024-09-10T11:25:21.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.180+0000 7fa4152fa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa410101890 con 0x7fa4100ffc60 2024-09-10T11:25:21.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.180+0000 7fa40effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4100ff2c0 0x7fa410100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:21.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.180+0000 7fa40effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4100ff2c0 0x7fa410100cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55610/0 (socket says 172.21.15.144:55610) 2024-09-10T11:25:21.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.180+0000 7fa40effd700 1 -- 172.21.15.144:0/1285466771 learned_addr learned my addr 172.21.15.144:0/1285466771 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:21.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.180+0000 7fa40e7fc700 1 --2- 172.21.15.144:0/1285466771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4100ffc60 0x7fa410101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:21.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.180+0000 7fa40effd700 1 -- 172.21.15.144:0/1285466771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4100ffc60 msgr2=0x7fa410101210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:21.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.180+0000 7fa40effd700 1 --2- 172.21.15.144:0/1285466771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4100ffc60 0x7fa410101210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:21.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.180+0000 7fa40effd700 1 -- 172.21.15.144:0/1285466771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4040095f0 con 0x7fa4100ff2c0 2024-09-10T11:25:21.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.181+0000 7fa40effd700 1 --2- 172.21.15.144:0/1285466771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4100ff2c0 0x7fa410100cd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa40400fa00 tx=0x7fa40400fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:21.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.181+0000 7fa3f7fff700 1 -- 172.21.15.144:0/1285466771 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa40401d070 con 0x7fa4100ff2c0 2024-09-10T11:25:21.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.181+0000 7fa4152fa700 1 -- 172.21.15.144:0/1285466771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa410101b10 con 0x7fa4100ff2c0 2024-09-10T11:25:21.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.181+0000 7fa3f7fff700 1 -- 172.21.15.144:0/1285466771 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa40400fe90 con 0x7fa4100ff2c0 2024-09-10T11:25:21.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.181+0000 7fa3f7fff700 1 -- 172.21.15.144:0/1285466771 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa404022470 con 0x7fa4100ff2c0 2024-09-10T11:25:21.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.181+0000 7fa4152fa700 1 -- 172.21.15.144:0/1285466771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4101a2240 con 0x7fa4100ff2c0 2024-09-10T11:25:21.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.182+0000 7fa4152fa700 1 -- 172.21.15.144:0/1285466771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4101085e0 con 0x7fa4100ff2c0 2024-09-10T11:25:21.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.182+0000 7fa3f7fff700 1 -- 172.21.15.144:0/1285466771 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fa404003680 con 0x7fa4100ff2c0 2024-09-10T11:25:21.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.183+0000 7fa3f7fff700 1 --2- 172.21.15.144:0/1285466771 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa3f806c680 0x7fa3f806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:21.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.183+0000 7fa3f7fff700 1 -- 172.21.15.144:0/1285466771 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fa40408c840 con 0x7fa4100ff2c0 2024-09-10T11:25:21.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.183+0000 7fa40e7fc700 1 --2- 172.21.15.144:0/1285466771 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa3f806c680 0x7fa3f806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:21.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.183+0000 7fa40e7fc700 1 --2- 172.21.15.144:0/1285466771 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa3f806c680 0x7fa3f806eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa410102470 tx=0x7fa400009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:21.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.187+0000 7fa3f7fff700 1 -- 172.21.15.144:0/1285466771 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa404057300 con 0x7fa4100ff2c0 2024-09-10T11:25:21.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.375+0000 7fa4152fa700 1 -- 172.21.15.144:0/1285466771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa41004ea50 con 0x7fa4100ff2c0 2024-09-10T11:25:21.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.376+0000 7fa3f7fff700 1 -- 172.21.15.144:0/1285466771 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11107 (secure 0 0 0) 0x7fa40405a920 con 0x7fa4100ff2c0 2024-09-10T11:25:21.378 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:25:21.378 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":41,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","created":"2024-09-10T11:19:32.020786+0000","modified":"2024-09-10T11:25:19.556870+0000","last_up_change":"2024-09-10T11:25:18.551589+0000","last_in_change":"2024-09-10T11:24:59.765382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:24:06.211746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5612a382-bb30-4314-8f65-0925157cf09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2127975408},{"type":"v1","addr":"172.21.15.144:6803","nonce":2127975408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2127975408},{"type":"v1","addr":"172.21.15.144:6805","nonce":2127975408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2127975408},{"type":"v1","addr":"172.21.15.144:6809","nonce":2127975408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2127975408},{"type":"v1","addr":"172.21.15.144:6807","nonce":2127975408}]},"public_addr":"172.21.15.144:6803/2127975408","cluster_addr":"172.21.15.144:6805/2127975408","heartbeat_back_addr":"172.21.15.144:6809/2127975408","heartbeat_front_addr":"172.21.15.144:6807/2127975408","state":["exists","up"]},{"osd":1,"uuid":"68977841-c0e7-44ce-8e18-96af001f56f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1548583549},{"type":"v1","addr":"172.21.15.144:6811","nonce":1548583549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1548583549},{"type":"v1","addr":"172.21.15.144:6813","nonce":1548583549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1548583549},{"type":"v1","addr":"172.21.15.144:6817","nonce":1548583549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1548583549},{"type":"v1","addr":"172.21.15.144:6815","nonce":1548583549}]},"public_addr":"172.21.15.144:6811/1548583549","cluster_addr":"172.21.15.144:6813/1548583549","heartbeat_back_addr":"172.21.15.144:6817/1548583549","heartbeat_front_addr":"172.21.15.144:6815/1548583549","state":["exists","up"]},{"osd":2,"uuid":"8f07849c-6967-4e40-a8e8-c5f928879f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3584068368},{"type":"v1","addr":"172.21.15.144:6819","nonce":3584068368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3584068368},{"type":"v1","addr":"172.21.15.144:6821","nonce":3584068368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3584068368},{"type":"v1","addr":"172.21.15.144:6825","nonce":3584068368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3584068368},{"type":"v1","addr":"172.21.15.144:6823","nonce":3584068368}]},"public_addr":"172.21.15.144:6819/3584068368","cluster_addr":"172.21.15.144:6821/3584068368","heartbeat_back_addr":"172.21.15.144:6825/3584068368","heartbeat_front_addr":"172.21.15.144:6823/3584068368","state":["exists","up"]},{"osd":3,"uuid":"fd8b7396-3962-4c06-ab46-d746cead4b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1366035688},{"type":"v1","addr":"172.21.15.178:6801","nonce":1366035688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1366035688},{"type":"v1","addr":"172.21.15.178:6803","nonce":1366035688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1366035688},{"type":"v1","addr":"172.21.15.178:6807","nonce":1366035688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1366035688},{"type":"v1","addr":"172.21.15.178:6805","nonce":1366035688}]},"public_addr":"172.21.15.178:6801/1366035688","cluster_addr":"172.21.15.178:6803/1366035688","heartbeat_back_addr":"172.21.15.178:6807/1366035688","heartbeat_front_addr":"172.21.15.178:6805/1366035688","state":["exists","up"]},{"osd":4,"uuid":"bc5df32a-01cd-45af-8a8e-9759fe330737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":280235677},{"type":"v1","addr":"172.21.15.178:6809","nonce":280235677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":280235677},{"type":"v1","addr":"172.21.15.178:6811","nonce":280235677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":280235677},{"type":"v1","addr":"172.21.15.178:6815","nonce":280235677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":280235677},{"type":"v1","addr":"172.21.15.178:6813","nonce":280235677}]},"public_addr":"172.21.15.178:6809/280235677","cluster_addr":"172.21.15.178:6811/280235677","heartbeat_back_addr":"172.21.15.178:6815/280235677","heartbeat_front_addr":"172.21.15.178:6813/280235677","state":["exists","up"]},{"osd":5,"uuid":"52ee296f-c365-4e5b-b18a-a281a0f58e04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3024778612},{"type":"v1","addr":"172.21.15.178:6817","nonce":3024778612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3024778612},{"type":"v1","addr":"172.21.15.178:6819","nonce":3024778612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3024778612},{"type":"v1","addr":"172.21.15.178:6823","nonce":3024778612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3024778612},{"type":"v1","addr":"172.21.15.178:6821","nonce":3024778612}]},"public_addr":"172.21.15.178:6817/3024778612","cluster_addr":"172.21.15.178:6819/3024778612","heartbeat_back_addr":"172.21.15.178:6823/3024778612","heartbeat_front_addr":"172.21.15.178:6821/3024778612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:23:06.331946+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:23:34.303341+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:24:02.370765+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:24:26.055226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:24:51.902849+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:25:16.897994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:0/3499523310":"2024-09-11T11:21:47.868365+0000","172.21.15.144:0/2387813590":"2024-09-11T11:20:42.965172+0000","172.21.15.144:0/3102311729":"2024-09-11T11:20:42.965172+0000","172.21.15.144:6801/2":"2024-09-11T11:20:09.967048+0000","172.21.15.144:0/898696771":"2024-09-11T11:21:47.868365+0000","172.21.15.144:6800/2":"2024-09-11T11:20:09.967048+0000","172.21.15.144:0/4053102056":"2024-09-11T11:21:47.868365+0000","172.21.15.144:0/2667523333":"2024-09-11T11:20:42.965172+0000","172.21.15.144:0/2543679831":"2024-09-11T11:20:09.967048+0000","172.21.15.144:0/3946165961":"2024-09-11T11:20:09.967048+0000","172.21.15.144:0/1513284721":"2024-09-11T11:20:09.967048+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:25:21.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.378+0000 7fa4152fa700 1 -- 172.21.15.144:0/1285466771 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa3f806c680 msgr2=0x7fa3f806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:21.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.378+0000 7fa4152fa700 1 --2- 172.21.15.144:0/1285466771 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa3f806c680 0x7fa3f806eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa410102470 tx=0x7fa400009260 comp rx=0 tx=0).stop 2024-09-10T11:25:21.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.378+0000 7fa4152fa700 1 -- 172.21.15.144:0/1285466771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4100ff2c0 msgr2=0x7fa410100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:21.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.378+0000 7fa4152fa700 1 --2- 172.21.15.144:0/1285466771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4100ff2c0 0x7fa410100cd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa40400fa00 tx=0x7fa40400fa30 comp rx=0 tx=0).stop 2024-09-10T11:25:21.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.378+0000 7fa4152fa700 1 -- 172.21.15.144:0/1285466771 shutdown_connections 2024-09-10T11:25:21.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.378+0000 7fa4152fa700 1 --2- 172.21.15.144:0/1285466771 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa3f806c680 0x7fa3f806eb30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:21.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.378+0000 7fa4152fa700 1 --2- 172.21.15.144:0/1285466771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4100ff2c0 0x7fa410100cd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:21.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.378+0000 7fa4152fa700 1 --2- 172.21.15.144:0/1285466771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4100ffc60 0x7fa410101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:21.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.378+0000 7fa4152fa700 1 -- 172.21.15.144:0/1285466771 >> 172.21.15.144:0/1285466771 conn(0x7fa4100fab80 msgr2=0x7fa410068ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:21.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.378+0000 7fa4152fa700 1 -- 172.21.15.144:0/1285466771 shutdown_connections 2024-09-10T11:25:21.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:21.378+0000 7fa4152fa700 1 -- 172.21.15.144:0/1285466771 wait complete. 2024-09-10T11:25:21.747 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:21 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1285466771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T11:25:21.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:21 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/1285466771' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T11:25:22.119 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T11:24:06.211746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-10T11:25:22.119 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T11:25:22.325 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:22.758 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:22 smithi144 ceph-mon[27942]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:22 smithi178 ceph-mon[33514]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:24.040 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.038+0000 7f7fc4e2b700 1 -- 172.21.15.144:0/1141908724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc01003e0 msgr2=0x7f7fc01007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:24.040 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.038+0000 7f7fc4e2b700 1 --2- 172.21.15.144:0/1141908724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc01003e0 0x7f7fc01007b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4009910 tx=0x7f7fb4009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:24.040 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.039+0000 7f7fc4e2b700 1 -- 172.21.15.144:0/1141908724 shutdown_connections 2024-09-10T11:25:24.040 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.039+0000 7f7fc4e2b700 1 --2- 172.21.15.144:0/1141908724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fc0100d80 0x7f7fc0104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:24.041 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.039+0000 7f7fc4e2b700 1 --2- 172.21.15.144:0/1141908724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc01003e0 0x7f7fc01007b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:24.041 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.039+0000 7f7fc4e2b700 1 -- 172.21.15.144:0/1141908724 >> 172.21.15.144:0/1141908724 conn(0x7f7fc00fbca0 msgr2=0x7f7fc00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:24.041 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.039+0000 7f7fc4e2b700 1 -- 172.21.15.144:0/1141908724 shutdown_connections 2024-09-10T11:25:24.041 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.039+0000 7f7fc4e2b700 1 -- 172.21.15.144:0/1141908724 wait complete. 2024-09-10T11:25:24.042 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.039+0000 7f7fc4e2b700 1 Processor -- start 2024-09-10T11:25:24.042 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.039+0000 7f7fc4e2b700 1 -- start start 2024-09-10T11:25:24.042 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7fc4e2b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc01003e0 0x7f7fc0195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:24.042 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7fc4e2b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fc0100d80 0x7f7fc01962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:24.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7fc4e2b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc01969b0 con 0x7f7fc01003e0 2024-09-10T11:25:24.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7fc4e2b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc019a740 con 0x7f7fc0100d80 2024-09-10T11:25:24.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7fbe59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc01003e0 0x7f7fc0195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:24.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7fbe59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc01003e0 0x7f7fc0195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37454/0 (socket says 172.21.15.144:37454) 2024-09-10T11:25:24.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7fbe59c700 1 -- 172.21.15.144:0/2007173185 learned_addr learned my addr 172.21.15.144:0/2007173185 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:24.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7fbdd9b700 1 --2- 172.21.15.144:0/2007173185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fc0100d80 0x7f7fc01962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:24.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7fbe59c700 1 -- 172.21.15.144:0/2007173185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fc0100d80 msgr2=0x7f7fc01962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:24.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7fbe59c700 1 --2- 172.21.15.144:0/2007173185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fc0100d80 0x7f7fc01962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:24.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7fbe59c700 1 -- 172.21.15.144:0/2007173185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb40095f0 con 0x7f7fc01003e0 2024-09-10T11:25:24.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7fbe59c700 1 --2- 172.21.15.144:0/2007173185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc01003e0 0x7f7fc0195d90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4006010 tx=0x7f7fb400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:24.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7faf7fe700 1 -- 172.21.15.144:0/2007173185 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fb401d070 con 0x7f7fc01003e0 2024-09-10T11:25:24.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7faf7fe700 1 -- 172.21.15.144:0/2007173185 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7fb4004fa0 con 0x7f7fc01003e0 2024-09-10T11:25:24.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7fc4e2b700 1 -- 172.21.15.144:0/2007173185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc019a9c0 con 0x7f7fc01003e0 2024-09-10T11:25:24.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.040+0000 7f7faf7fe700 1 -- 172.21.15.144:0/2007173185 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fb400f690 con 0x7f7fc01003e0 2024-09-10T11:25:24.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.041+0000 7f7fc4e2b700 1 -- 172.21.15.144:0/2007173185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc019aeb0 con 0x7f7fc01003e0 2024-09-10T11:25:24.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.042+0000 7f7fc4e2b700 1 -- 172.21.15.144:0/2007173185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc01086f0 con 0x7f7fc01003e0 2024-09-10T11:25:24.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.042+0000 7f7faf7fe700 1 -- 172.21.15.144:0/2007173185 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f7fb4022470 con 0x7f7fc01003e0 2024-09-10T11:25:24.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.042+0000 7f7faf7fe700 1 --2- 172.21.15.144:0/2007173185 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7fa806c680 0x7f7fa806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:24.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.043+0000 7f7faf7fe700 1 -- 172.21.15.144:0/2007173185 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f7fb408cc70 con 0x7f7fc01003e0 2024-09-10T11:25:24.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.043+0000 7f7fbdd9b700 1 --2- 172.21.15.144:0/2007173185 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7fa806c680 0x7f7fa806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:24.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.043+0000 7f7fbdd9b700 1 --2- 172.21.15.144:0/2007173185 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7fa806c680 0x7f7fa806eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7fc01973b0 tx=0x7f7fb0009190 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:24.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.047+0000 7f7faf7fe700 1 -- 172.21.15.144:0/2007173185 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7fb4057730 con 0x7f7fc01003e0 2024-09-10T11:25:24.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.233+0000 7f7fc4e2b700 1 -- 172.21.15.144:0/2007173185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7fc019b190 con 0x7f7fc01003e0 2024-09-10T11:25:24.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.234+0000 7f7faf7fe700 1 -- 172.21.15.144:0/2007173185 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v41) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7fb405ad50 con 0x7f7fc01003e0 2024-09-10T11:25:24.235 INFO:teuthology.orchestra.run.smithi144.stdout:pg_num: 1 2024-09-10T11:25:24.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.236+0000 7f7fc4e2b700 1 -- 172.21.15.144:0/2007173185 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7fa806c680 msgr2=0x7f7fa806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:24.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.236+0000 7f7fc4e2b700 1 --2- 172.21.15.144:0/2007173185 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7fa806c680 0x7f7fa806eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7fc01973b0 tx=0x7f7fb0009190 comp rx=0 tx=0).stop 2024-09-10T11:25:24.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.236+0000 7f7fc4e2b700 1 -- 172.21.15.144:0/2007173185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc01003e0 msgr2=0x7f7fc0195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:24.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.236+0000 7f7fc4e2b700 1 --2- 172.21.15.144:0/2007173185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc01003e0 0x7f7fc0195d90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f7fb4006010 tx=0x7f7fb400bd70 comp rx=0 tx=0).stop 2024-09-10T11:25:24.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.236+0000 7f7fc4e2b700 1 -- 172.21.15.144:0/2007173185 shutdown_connections 2024-09-10T11:25:24.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.236+0000 7f7fc4e2b700 1 --2- 172.21.15.144:0/2007173185 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7fa806c680 0x7f7fa806eb30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:24.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.236+0000 7f7fc4e2b700 1 --2- 172.21.15.144:0/2007173185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fc01003e0 0x7f7fc0195d90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:24.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.236+0000 7f7fc4e2b700 1 --2- 172.21.15.144:0/2007173185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fc0100d80 0x7f7fc01962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:24.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.236+0000 7f7fc4e2b700 1 -- 172.21.15.144:0/2007173185 >> 172.21.15.144:0/2007173185 conn(0x7f7fc00fbca0 msgr2=0x7f7fc00fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:24.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.236+0000 7f7fc4e2b700 1 -- 172.21.15.144:0/2007173185 shutdown_connections 2024-09-10T11:25:24.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:24.236+0000 7f7fc4e2b700 1 -- 172.21.15.144:0/2007173185 wait complete. 2024-09-10T11:25:24.959 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T11:25:24.960 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T11:25:25.157 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:25.277 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:24 smithi144 ceph-mon[27942]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:25.277 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:24 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2007173185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T11:25:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:24 smithi178 ceph-mon[33514]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:24 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/2007173185' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T11:25:26.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.842+0000 7fe8661e3700 1 -- 172.21.15.144:0/3817798775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe860100150 msgr2=0x7fe860100560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:26.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.842+0000 7fe8661e3700 1 --2- 172.21.15.144:0/3817798775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe860100150 0x7fe860100560 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe850009910 tx=0x7fe850009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:26.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.843+0000 7fe8661e3700 1 -- 172.21.15.144:0/3817798775 shutdown_connections 2024-09-10T11:25:26.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.843+0000 7fe8661e3700 1 --2- 172.21.15.144:0/3817798775 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe860101350 0x7fe8601017a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:26.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.843+0000 7fe8661e3700 1 --2- 172.21.15.144:0/3817798775 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe860100150 0x7fe860100560 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:26.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.843+0000 7fe8661e3700 1 -- 172.21.15.144:0/3817798775 >> 172.21.15.144:0/3817798775 conn(0x7fe8600fb6e0 msgr2=0x7fe8600fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:26.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.843+0000 7fe8661e3700 1 -- 172.21.15.144:0/3817798775 shutdown_connections 2024-09-10T11:25:26.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.843+0000 7fe8661e3700 1 -- 172.21.15.144:0/3817798775 wait complete. 2024-09-10T11:25:26.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.843+0000 7fe8661e3700 1 Processor -- start 2024-09-10T11:25:26.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.843+0000 7fe8661e3700 1 -- start start 2024-09-10T11:25:26.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.844+0000 7fe8661e3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe860100150 0x7fe860195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:26.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.844+0000 7fe8661e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe860101350 0x7fe860195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:26.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.844+0000 7fe8661e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe860196570 con 0x7fe860101350 2024-09-10T11:25:26.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.844+0000 7fe8661e3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8601966b0 con 0x7fe860100150 2024-09-10T11:25:26.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.844+0000 7fe85effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe860101350 0x7fe860195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:26.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.844+0000 7fe85effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe860101350 0x7fe860195f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37472/0 (socket says 172.21.15.144:37472) 2024-09-10T11:25:26.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.844+0000 7fe85effd700 1 -- 172.21.15.144:0/1988669152 learned_addr learned my addr 172.21.15.144:0/1988669152 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:26.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.844+0000 7fe85f7fe700 1 --2- 172.21.15.144:0/1988669152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe860100150 0x7fe860195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:26.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.844+0000 7fe85effd700 1 -- 172.21.15.144:0/1988669152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe860100150 msgr2=0x7fe860195a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:26.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.844+0000 7fe85effd700 1 --2- 172.21.15.144:0/1988669152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe860100150 0x7fe860195a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:26.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.844+0000 7fe85effd700 1 -- 172.21.15.144:0/1988669152 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8500095f0 con 0x7fe860101350 2024-09-10T11:25:26.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.844+0000 7fe85effd700 1 --2- 172.21.15.144:0/1988669152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe860101350 0x7fe860195f50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe85400b4b0 tx=0x7fe85400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:26.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.844+0000 7fe85cff9700 1 -- 172.21.15.144:0/1988669152 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe854011800 con 0x7fe860101350 2024-09-10T11:25:26.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.844+0000 7fe85cff9700 1 -- 172.21.15.144:0/1988669152 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe854011e10 con 0x7fe860101350 2024-09-10T11:25:26.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.845+0000 7fe8661e3700 1 -- 172.21.15.144:0/1988669152 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe86019b160 con 0x7fe860101350 2024-09-10T11:25:26.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.845+0000 7fe85cff9700 1 -- 172.21.15.144:0/1988669152 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe85400f2e0 con 0x7fe860101350 2024-09-10T11:25:26.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.845+0000 7fe8661e3700 1 -- 172.21.15.144:0/1988669152 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe86019b730 con 0x7fe860101350 2024-09-10T11:25:26.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.845+0000 7fe8661e3700 1 -- 172.21.15.144:0/1988669152 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe860108d20 con 0x7fe860101350 2024-09-10T11:25:26.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.846+0000 7fe85cff9700 1 -- 172.21.15.144:0/1988669152 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fe854010380 con 0x7fe860101350 2024-09-10T11:25:26.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.846+0000 7fe85cff9700 1 --2- 172.21.15.144:0/1988669152 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe84806c680 0x7fe84806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:26.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.846+0000 7fe85cff9700 1 -- 172.21.15.144:0/1988669152 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fe85408af50 con 0x7fe860101350 2024-09-10T11:25:26.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.846+0000 7fe85f7fe700 1 --2- 172.21.15.144:0/1988669152 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe84806c680 0x7fe84806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:26.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.847+0000 7fe85f7fe700 1 --2- 172.21.15.144:0/1988669152 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe84806c680 0x7fe84806eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe850006010 tx=0x7fe85000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:26.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:26.851+0000 7fe85cff9700 1 -- 172.21.15.144:0/1988669152 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe854059520 con 0x7fe860101350 2024-09-10T11:25:27.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:27.116+0000 7fe8661e3700 1 -- 172.21.15.144:0/1988669152 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fe86019be80 con 0x7fe860101350 2024-09-10T11:25:27.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:27.122+0000 7fe85cff9700 1 -- 172.21.15.144:0/1988669152 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7fe854015050 con 0x7fe860101350 2024-09-10T11:25:27.124 INFO:teuthology.orchestra.run.smithi144.stdout:[client.0] 2024-09-10T11:25:27.124 INFO:teuthology.orchestra.run.smithi144.stdout: key = AQCnLOBm+kAIBxAAZ7WHmvGOkbUndo/rxzCepw== 2024-09-10T11:25:27.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:27.124+0000 7fe8661e3700 1 -- 172.21.15.144:0/1988669152 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe84806c680 msgr2=0x7fe84806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:27.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:27.124+0000 7fe8661e3700 1 --2- 172.21.15.144:0/1988669152 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe84806c680 0x7fe84806eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe850006010 tx=0x7fe85000b540 comp rx=0 tx=0).stop 2024-09-10T11:25:27.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:27.124+0000 7fe8661e3700 1 -- 172.21.15.144:0/1988669152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe860101350 msgr2=0x7fe860195f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:27.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:27.124+0000 7fe8661e3700 1 --2- 172.21.15.144:0/1988669152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe860101350 0x7fe860195f50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe85400b4b0 tx=0x7fe85400b7c0 comp rx=0 tx=0).stop 2024-09-10T11:25:27.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:27.124+0000 7fe8661e3700 1 -- 172.21.15.144:0/1988669152 shutdown_connections 2024-09-10T11:25:27.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:27.124+0000 7fe8661e3700 1 --2- 172.21.15.144:0/1988669152 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fe84806c680 0x7fe84806eb30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:27.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:27.124+0000 7fe8661e3700 1 --2- 172.21.15.144:0/1988669152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe860100150 0x7fe860195a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:27.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:27.124+0000 7fe8661e3700 1 --2- 172.21.15.144:0/1988669152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe860101350 0x7fe860195f50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:27.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:27.124+0000 7fe8661e3700 1 -- 172.21.15.144:0/1988669152 >> 172.21.15.144:0/1988669152 conn(0x7fe8600fb6e0 msgr2=0x7fe860104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:27.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:27.125+0000 7fe8661e3700 1 -- 172.21.15.144:0/1988669152 shutdown_connections 2024-09-10T11:25:27.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:27.125+0000 7fe8661e3700 1 -- 172.21.15.144:0/1988669152 wait complete. 2024-09-10T11:25:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:26 smithi178 ceph-mon[33514]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:27.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:26 smithi144 ceph-mon[27942]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:27.649 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:25:27.649 DEBUG:teuthology.orchestra.run.smithi144:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T11:25:27.649 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T11:25:27.687 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T11:25:27.898 INFO:teuthology.orchestra.run.smithi178.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:25:28.019 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:27 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/1988669152' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T11:25:28.019 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:27 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/1988669152' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T11:25:28.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:27 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1988669152' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T11:25:28.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:27 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1988669152' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T11:25:29.146 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:28 smithi178 ceph-mon[33514]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:29.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:28 smithi144 ceph-mon[27942]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:29.499 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.495+0000 7f35b78d5700 1 -- 172.21.15.178:0/393280386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b0101370 msgr2=0x7f35b01017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:29.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.495+0000 7f35b78d5700 1 --2- 172.21.15.178:0/393280386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b0101370 0x7f35b01017c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f35ac009910 tx=0x7f35ac009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:29.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.495+0000 7f35b78d5700 1 -- 172.21.15.178:0/393280386 shutdown_connections 2024-09-10T11:25:29.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.495+0000 7f35b78d5700 1 --2- 172.21.15.178:0/393280386 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b0101370 0x7f35b01017c0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:29.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.495+0000 7f35b78d5700 1 --2- 172.21.15.178:0/393280386 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b0100170 0x7f35b0100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:29.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.495+0000 7f35b78d5700 1 -- 172.21.15.178:0/393280386 >> 172.21.15.178:0/393280386 conn(0x7f35b00fb6e0 msgr2=0x7f35b00fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:29.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.496+0000 7f35b78d5700 1 -- 172.21.15.178:0/393280386 shutdown_connections 2024-09-10T11:25:29.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.496+0000 7f35b78d5700 1 -- 172.21.15.178:0/393280386 wait complete. 2024-09-10T11:25:29.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.496+0000 7f35b78d5700 1 Processor -- start 2024-09-10T11:25:29.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.496+0000 7f35b78d5700 1 -- start start 2024-09-10T11:25:29.500 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.496+0000 7f35b78d5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b0100170 0x7f35b0195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:29.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35b78d5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b0101370 0x7f35b0195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:29.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35b78d5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b0196450 con 0x7f35b0101370 2024-09-10T11:25:29.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35b78d5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b0196590 con 0x7f35b0100170 2024-09-10T11:25:29.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35b5671700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b0100170 0x7f35b0195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:29.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35b5671700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b0100170 0x7f35b0195980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.178:36124/0 (socket says 172.21.15.178:36124) 2024-09-10T11:25:29.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35b5671700 1 -- 172.21.15.178:0/2669075447 learned_addr learned my addr 172.21.15.178:0/2669075447 (peer_addr_for_me v2:172.21.15.178:0/0) 2024-09-10T11:25:29.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35b5671700 1 -- 172.21.15.178:0/2669075447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b0101370 msgr2=0x7f35b0195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:29.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35b4e70700 1 --2- 172.21.15.178:0/2669075447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b0101370 0x7f35b0195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:29.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35b5671700 1 --2- 172.21.15.178:0/2669075447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b0101370 0x7f35b0195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:29.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35b5671700 1 -- 172.21.15.178:0/2669075447 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac0095f0 con 0x7f35b0100170 2024-09-10T11:25:29.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35b5671700 1 --2- 172.21.15.178:0/2669075447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b0100170 0x7f35b0195980 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f35a400c910 tx=0x7f35a400ccd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:29.501 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35a27fc700 1 -- 172.21.15.178:0/2669075447 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35a40041f0 con 0x7f35b0100170 2024-09-10T11:25:29.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35a27fc700 1 -- 172.21.15.178:0/2669075447 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f35a4007db0 con 0x7f35b0100170 2024-09-10T11:25:29.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35b78d5700 1 -- 172.21.15.178:0/2669075447 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b019aff0 con 0x7f35b0100170 2024-09-10T11:25:29.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35a27fc700 1 -- 172.21.15.178:0/2669075447 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35a4003d60 con 0x7f35b0100170 2024-09-10T11:25:29.502 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.497+0000 7f35b78d5700 1 -- 172.21.15.178:0/2669075447 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b019b510 con 0x7f35b0100170 2024-09-10T11:25:29.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.498+0000 7f35b78d5700 1 -- 172.21.15.178:0/2669075447 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b0108cc0 con 0x7f35b0100170 2024-09-10T11:25:29.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.499+0000 7f35a27fc700 1 -- 172.21.15.178:0/2669075447 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f35a4007900 con 0x7f35b0100170 2024-09-10T11:25:29.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.499+0000 7f35a27fc700 1 --2- 172.21.15.178:0/2669075447 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f359c06c680 0x7f359c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:29.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.499+0000 7f35a27fc700 1 -- 172.21.15.178:0/2669075447 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f35a408b1c0 con 0x7f35b0100170 2024-09-10T11:25:29.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.500+0000 7f35b4e70700 1 --2- 172.21.15.178:0/2669075447 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f359c06c680 0x7f359c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:29.503 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.500+0000 7f35b4e70700 1 --2- 172.21.15.178:0/2669075447 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f359c06c680 0x7f359c06eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f35ac006010 tx=0x7f35ac00b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:29.505 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.504+0000 7f35a27fc700 1 -- 172.21.15.178:0/2669075447 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f35a4059710 con 0x7f35b0100170 2024-09-10T11:25:29.770 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.768+0000 7f35b78d5700 1 -- 172.21.15.178:0/2669075447 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f35b019bbb0 con 0x7f35b0100170 2024-09-10T11:25:29.775 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.773+0000 7f35a27fc700 1 -- 172.21.15.178:0/2669075447 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f35a40592a0 con 0x7f35b0100170 2024-09-10T11:25:29.775 INFO:teuthology.orchestra.run.smithi178.stdout:[client.1] 2024-09-10T11:25:29.775 INFO:teuthology.orchestra.run.smithi178.stdout: key = AQCpLOBmmub0LRAAjWLayeCJyspMSzevEPT57A== 2024-09-10T11:25:29.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.775+0000 7f35b78d5700 1 -- 172.21.15.178:0/2669075447 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f359c06c680 msgr2=0x7f359c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:29.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.775+0000 7f35b78d5700 1 --2- 172.21.15.178:0/2669075447 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f359c06c680 0x7f359c06eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f35ac006010 tx=0x7f35ac00b540 comp rx=0 tx=0).stop 2024-09-10T11:25:29.777 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.775+0000 7f35b78d5700 1 -- 172.21.15.178:0/2669075447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b0100170 msgr2=0x7f35b0195980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:29.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.775+0000 7f35b78d5700 1 --2- 172.21.15.178:0/2669075447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b0100170 0x7f35b0195980 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f35a400c910 tx=0x7f35a400ccd0 comp rx=0 tx=0).stop 2024-09-10T11:25:29.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.775+0000 7f35b78d5700 1 -- 172.21.15.178:0/2669075447 shutdown_connections 2024-09-10T11:25:29.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.775+0000 7f35b78d5700 1 --2- 172.21.15.178:0/2669075447 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f359c06c680 0x7f359c06eb30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:29.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.775+0000 7f35b78d5700 1 --2- 172.21.15.178:0/2669075447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35b0100170 0x7f35b0195980 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:29.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.775+0000 7f35b78d5700 1 --2- 172.21.15.178:0/2669075447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35b0101370 0x7f35b0195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:29.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.775+0000 7f35b78d5700 1 -- 172.21.15.178:0/2669075447 >> 172.21.15.178:0/2669075447 conn(0x7f35b00fb6e0 msgr2=0x7f35b01045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:29.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.775+0000 7f35b78d5700 1 -- 172.21.15.178:0/2669075447 shutdown_connections 2024-09-10T11:25:29.778 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:25:29.775+0000 7f35b78d5700 1 -- 172.21.15.178:0/2669075447 wait complete. 2024-09-10T11:25:30.097 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:29 smithi178 ceph-mon[33514]: from='client.? 172.21.15.178:0/2669075447' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T11:25:30.098 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:29 smithi178 ceph-mon[33514]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T11:25:30.098 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:29 smithi178 ceph-mon[33514]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T11:25:30.394 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T11:25:30.394 DEBUG:teuthology.orchestra.run.smithi178:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-10T11:25:30.395 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T11:25:30.432 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T11:25:30.432 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T11:25:30.432 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T11:25:30.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:29 smithi144 ceph-mon[27942]: from='client.? 172.21.15.178:0/2669075447' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T11:25:30.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:29 smithi144 ceph-mon[27942]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T11:25:30.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:29 smithi144 ceph-mon[27942]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T11:25:30.655 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:31.072 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:30 smithi144 ceph-mon[27942]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:30 smithi178 ceph-mon[33514]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:32.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.259+0000 7fcc63a3d700 1 -- 172.21.15.144:0/234988627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c0ff2d0 msgr2=0x7fcc5c0ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:32.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.259+0000 7fcc63a3d700 1 --2- 172.21.15.144:0/234988627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c0ff2d0 0x7fcc5c0ff6a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c009910 tx=0x7fcc4c009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:32.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.259+0000 7fcc63a3d700 1 -- 172.21.15.144:0/234988627 shutdown_connections 2024-09-10T11:25:32.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.259+0000 7fcc63a3d700 1 --2- 172.21.15.144:0/234988627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc5c0ffc70 0x7fcc5c103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:32.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.259+0000 7fcc63a3d700 1 --2- 172.21.15.144:0/234988627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c0ff2d0 0x7fcc5c0ff6a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:32.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.259+0000 7fcc63a3d700 1 -- 172.21.15.144:0/234988627 >> 172.21.15.144:0/234988627 conn(0x7fcc5c0fabb0 msgr2=0x7fcc5c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:32.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.259+0000 7fcc63a3d700 1 -- 172.21.15.144:0/234988627 shutdown_connections 2024-09-10T11:25:32.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.259+0000 7fcc63a3d700 1 -- 172.21.15.144:0/234988627 wait complete. 2024-09-10T11:25:32.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.260+0000 7fcc63a3d700 1 Processor -- start 2024-09-10T11:25:32.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.260+0000 7fcc63a3d700 1 -- start start 2024-09-10T11:25:32.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.260+0000 7fcc63a3d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c0ff2d0 0x7fcc5c196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:32.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.260+0000 7fcc63a3d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc5c0ffc70 0x7fcc5c1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:32.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.260+0000 7fcc63a3d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c1977f0 con 0x7fcc5c0ff2d0 2024-09-10T11:25:32.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.260+0000 7fcc63a3d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5c197930 con 0x7fcc5c0ffc70 2024-09-10T11:25:32.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.260+0000 7fcc617d9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c0ff2d0 0x7fcc5c196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:32.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.260+0000 7fcc617d9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c0ff2d0 0x7fcc5c196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37504/0 (socket says 172.21.15.144:37504) 2024-09-10T11:25:32.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.260+0000 7fcc617d9700 1 -- 172.21.15.144:0/1991289066 learned_addr learned my addr 172.21.15.144:0/1991289066 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:32.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.260+0000 7fcc617d9700 1 -- 172.21.15.144:0/1991289066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc5c0ffc70 msgr2=0x7fcc5c1971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:32.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.260+0000 7fcc60fd8700 1 --2- 172.21.15.144:0/1991289066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc5c0ffc70 0x7fcc5c1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:32.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.260+0000 7fcc617d9700 1 --2- 172.21.15.144:0/1991289066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc5c0ffc70 0x7fcc5c1971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:32.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.260+0000 7fcc617d9700 1 -- 172.21.15.144:0/1991289066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc4c0095f0 con 0x7fcc5c0ff2d0 2024-09-10T11:25:32.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.260+0000 7fcc60fd8700 1 --2- 172.21.15.144:0/1991289066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc5c0ffc70 0x7fcc5c1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:25:32.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.261+0000 7fcc617d9700 1 --2- 172.21.15.144:0/1991289066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c0ff2d0 0x7fcc5c196c90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c006010 tx=0x7fcc4c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:32.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.261+0000 7fcc5a7fc700 1 -- 172.21.15.144:0/1991289066 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc4c01d070 con 0x7fcc5c0ff2d0 2024-09-10T11:25:32.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.261+0000 7fcc63a3d700 1 -- 172.21.15.144:0/1991289066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc5c19fd00 con 0x7fcc5c0ff2d0 2024-09-10T11:25:32.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.261+0000 7fcc5a7fc700 1 -- 172.21.15.144:0/1991289066 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcc4c00f460 con 0x7fcc5c0ff2d0 2024-09-10T11:25:32.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.261+0000 7fcc5a7fc700 1 -- 172.21.15.144:0/1991289066 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc4c021640 con 0x7fcc5c0ff2d0 2024-09-10T11:25:32.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.261+0000 7fcc63a3d700 1 -- 172.21.15.144:0/1991289066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc5c1a0110 con 0x7fcc5c0ff2d0 2024-09-10T11:25:32.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.262+0000 7fcc63a3d700 1 -- 172.21.15.144:0/1991289066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc5c1075e0 con 0x7fcc5c0ff2d0 2024-09-10T11:25:32.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.262+0000 7fcc5a7fc700 1 -- 172.21.15.144:0/1991289066 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fcc4c00faf0 con 0x7fcc5c0ff2d0 2024-09-10T11:25:32.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.263+0000 7fcc5a7fc700 1 --2- 172.21.15.144:0/1991289066 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcc4406c680 0x7fcc4406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:32.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.263+0000 7fcc60fd8700 1 --2- 172.21.15.144:0/1991289066 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcc4406c680 0x7fcc4406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:32.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.263+0000 7fcc5a7fc700 1 -- 172.21.15.144:0/1991289066 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fcc4c08c720 con 0x7fcc5c0ff2d0 2024-09-10T11:25:32.266 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.263+0000 7fcc60fd8700 1 --2- 172.21.15.144:0/1991289066 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcc4406c680 0x7fcc4406eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c069600 tx=0x7fcc500073e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:32.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.268+0000 7fcc5a7fc700 1 -- 172.21.15.144:0/1991289066 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcc4c057260 con 0x7fcc5c0ff2d0 2024-09-10T11:25:32.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.519+0000 7fcc63a3d700 1 -- 172.21.15.144:0/1991289066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fcc5c02cc30 con 0x7fcc5c0ff2d0 2024-09-10T11:25:32.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.522+0000 7fcc5a7fc700 1 -- 172.21.15.144:0/1991289066 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+172836 (secure 0 0 0) 0x7fcc4c05a880 con 0x7fcc5c0ff2d0 2024-09-10T11:25:32.528 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:25:32.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.524+0000 7fcc63a3d700 1 -- 172.21.15.144:0/1991289066 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcc4406c680 msgr2=0x7fcc4406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:32.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.524+0000 7fcc63a3d700 1 --2- 172.21.15.144:0/1991289066 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcc4406c680 0x7fcc4406eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c069600 tx=0x7fcc500073e0 comp rx=0 tx=0).stop 2024-09-10T11:25:32.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.524+0000 7fcc63a3d700 1 -- 172.21.15.144:0/1991289066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c0ff2d0 msgr2=0x7fcc5c196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:32.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.524+0000 7fcc63a3d700 1 --2- 172.21.15.144:0/1991289066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c0ff2d0 0x7fcc5c196c90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fcc4c006010 tx=0x7fcc4c00be20 comp rx=0 tx=0).stop 2024-09-10T11:25:32.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.524+0000 7fcc63a3d700 1 -- 172.21.15.144:0/1991289066 shutdown_connections 2024-09-10T11:25:32.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.524+0000 7fcc63a3d700 1 --2- 172.21.15.144:0/1991289066 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fcc4406c680 0x7fcc4406eb30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:32.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.524+0000 7fcc63a3d700 1 --2- 172.21.15.144:0/1991289066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcc5c0ff2d0 0x7fcc5c196c90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:32.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.524+0000 7fcc63a3d700 1 --2- 172.21.15.144:0/1991289066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcc5c0ffc70 0x7fcc5c1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:32.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.524+0000 7fcc63a3d700 1 -- 172.21.15.144:0/1991289066 >> 172.21.15.144:0/1991289066 conn(0x7fcc5c0fabb0 msgr2=0x7fcc5c0fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:32.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.524+0000 7fcc63a3d700 1 -- 172.21.15.144:0/1991289066 shutdown_connections 2024-09-10T11:25:32.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:32.524+0000 7fcc63a3d700 1 -- 172.21.15.144:0/1991289066 wait complete. 2024-09-10T11:25:33.104 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:32 smithi144 ceph-mon[27942]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:33.104 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:32 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1991289066' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T11:25:33.104 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:32 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:25:33.124 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":19,"active_gid":14213,"active_name":"smithi144.bkogfs","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6800","nonce":2},{"type":"v1","addr":"172.21.15.144:6801","nonce":2}]},"active_addr":"172.21.15.144:6801/2","active_change":"2024-09-10T11:21:47.868521+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14236,"name":"smithi178.vmvxuf","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.144:8443/","prometheus":"http://172.21.15.144:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":1470901008}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":2988003891}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":337525076}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.144:0","nonce":2490688975}]}]} 2024-09-10T11:25:33.130 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T11:25:33.130 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T11:25:33.131 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T11:25:33.326 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:32 smithi178 ceph-mon[33514]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:32 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/1991289066' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T11:25:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:32 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:25:35.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.100+0000 7f2eb624a700 1 -- 172.21.15.144:0/2856616062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb00ff290 msgr2=0x7f2eb00ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:35.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.100+0000 7f2eb624a700 1 --2- 172.21.15.144:0/2856616062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb00ff290 0x7f2eb00ff660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0009910 tx=0x7f2ea0009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:35.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.100+0000 7f2eb624a700 1 -- 172.21.15.144:0/2856616062 shutdown_connections 2024-09-10T11:25:35.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.100+0000 7f2eb624a700 1 --2- 172.21.15.144:0/2856616062 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eb00ffba0 0x7f2eb010a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:35.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.100+0000 7f2eb624a700 1 --2- 172.21.15.144:0/2856616062 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb00ff290 0x7f2eb00ff660 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:35.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.100+0000 7f2eb624a700 1 -- 172.21.15.144:0/2856616062 >> 172.21.15.144:0/2856616062 conn(0x7f2eb0074850 msgr2=0x7f2eb0074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:35.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.100+0000 7f2eb624a700 1 -- 172.21.15.144:0/2856616062 shutdown_connections 2024-09-10T11:25:35.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.100+0000 7f2eb624a700 1 -- 172.21.15.144:0/2856616062 wait complete. 2024-09-10T11:25:35.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.100+0000 7f2eb624a700 1 Processor -- start 2024-09-10T11:25:35.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.100+0000 7f2eb624a700 1 -- start start 2024-09-10T11:25:35.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eb624a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eb00ff290 0x7f2eb0195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:35.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eb624a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb00ffba0 0x7f2eb0196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:35.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eb624a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb0196a30 con 0x7f2eb00ffba0 2024-09-10T11:25:35.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eb624a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2eb019a7c0 con 0x7f2eb00ff290 2024-09-10T11:25:35.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eaeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb00ffba0 0x7f2eb0196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:35.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eaeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb00ffba0 0x7f2eb0196350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39998/0 (socket says 172.21.15.144:39998) 2024-09-10T11:25:35.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eaeffd700 1 -- 172.21.15.144:0/4064318381 learned_addr learned my addr 172.21.15.144:0/4064318381 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:35.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eaf7fe700 1 --2- 172.21.15.144:0/4064318381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eb00ff290 0x7f2eb0195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:35.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eaeffd700 1 -- 172.21.15.144:0/4064318381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eb00ff290 msgr2=0x7f2eb0195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:35.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eaeffd700 1 --2- 172.21.15.144:0/4064318381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eb00ff290 0x7f2eb0195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:35.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eaeffd700 1 -- 172.21.15.144:0/4064318381 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ea00095f0 con 0x7f2eb00ffba0 2024-09-10T11:25:35.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eaeffd700 1 --2- 172.21.15.144:0/4064318381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb00ffba0 0x7f2eb0196350 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400b4b0 tx=0x7f2ea400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:35.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eacff9700 1 -- 172.21.15.144:0/4064318381 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ea4011800 con 0x7f2eb00ffba0 2024-09-10T11:25:35.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eacff9700 1 -- 172.21.15.144:0/4064318381 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2ea4011e10 con 0x7f2eb00ffba0 2024-09-10T11:25:35.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eb624a700 1 -- 172.21.15.144:0/4064318381 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2eb019aaa0 con 0x7f2eb00ffba0 2024-09-10T11:25:35.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eacff9700 1 -- 172.21.15.144:0/4064318381 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ea400f2e0 con 0x7f2eb00ffba0 2024-09-10T11:25:35.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.101+0000 7f2eb624a700 1 -- 172.21.15.144:0/4064318381 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2eb019aff0 con 0x7f2eb00ffba0 2024-09-10T11:25:35.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.102+0000 7f2eb624a700 1 -- 172.21.15.144:0/4064318381 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2eb0107d10 con 0x7f2eb00ffba0 2024-09-10T11:25:35.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.103+0000 7f2eacff9700 1 -- 172.21.15.144:0/4064318381 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f2ea4010380 con 0x7f2eb00ffba0 2024-09-10T11:25:35.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.103+0000 7f2eacff9700 1 --2- 172.21.15.144:0/4064318381 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2e9806c680 0x7f2e9806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:35.106 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:34 smithi144 ceph-mon[27942]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:35.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.103+0000 7f2eacff9700 1 -- 172.21.15.144:0/4064318381 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f2ea408b070 con 0x7f2eb00ffba0 2024-09-10T11:25:35.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.103+0000 7f2eaf7fe700 1 --2- 172.21.15.144:0/4064318381 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2e9806c680 0x7f2e9806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:35.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.104+0000 7f2eaf7fe700 1 --2- 172.21.15.144:0/4064318381 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2e9806c680 0x7f2e9806eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0006010 tx=0x7f2ea001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:35.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.108+0000 7f2eacff9700 1 -- 172.21.15.144:0/4064318381 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2ea4055b30 con 0x7f2eb00ffba0 2024-09-10T11:25:35.298 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.297+0000 7f2eb624a700 1 -- 172.21.15.144:0/4064318381 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2eb004ea50 con 0x7f2eb00ffba0 2024-09-10T11:25:35.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.298+0000 7f2eacff9700 1 -- 172.21.15.144:0/4064318381 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11107 (secure 0 0 0) 0x7f2ea4059150 con 0x7f2eb00ffba0 2024-09-10T11:25:35.299 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:25:35.300 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":41,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","created":"2024-09-10T11:19:32.020786+0000","modified":"2024-09-10T11:25:19.556870+0000","last_up_change":"2024-09-10T11:25:18.551589+0000","last_in_change":"2024-09-10T11:24:59.765382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:24:06.211746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5612a382-bb30-4314-8f65-0925157cf09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2127975408},{"type":"v1","addr":"172.21.15.144:6803","nonce":2127975408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2127975408},{"type":"v1","addr":"172.21.15.144:6805","nonce":2127975408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2127975408},{"type":"v1","addr":"172.21.15.144:6809","nonce":2127975408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2127975408},{"type":"v1","addr":"172.21.15.144:6807","nonce":2127975408}]},"public_addr":"172.21.15.144:6803/2127975408","cluster_addr":"172.21.15.144:6805/2127975408","heartbeat_back_addr":"172.21.15.144:6809/2127975408","heartbeat_front_addr":"172.21.15.144:6807/2127975408","state":["exists","up"]},{"osd":1,"uuid":"68977841-c0e7-44ce-8e18-96af001f56f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1548583549},{"type":"v1","addr":"172.21.15.144:6811","nonce":1548583549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1548583549},{"type":"v1","addr":"172.21.15.144:6813","nonce":1548583549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1548583549},{"type":"v1","addr":"172.21.15.144:6817","nonce":1548583549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1548583549},{"type":"v1","addr":"172.21.15.144:6815","nonce":1548583549}]},"public_addr":"172.21.15.144:6811/1548583549","cluster_addr":"172.21.15.144:6813/1548583549","heartbeat_back_addr":"172.21.15.144:6817/1548583549","heartbeat_front_addr":"172.21.15.144:6815/1548583549","state":["exists","up"]},{"osd":2,"uuid":"8f07849c-6967-4e40-a8e8-c5f928879f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3584068368},{"type":"v1","addr":"172.21.15.144:6819","nonce":3584068368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3584068368},{"type":"v1","addr":"172.21.15.144:6821","nonce":3584068368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3584068368},{"type":"v1","addr":"172.21.15.144:6825","nonce":3584068368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3584068368},{"type":"v1","addr":"172.21.15.144:6823","nonce":3584068368}]},"public_addr":"172.21.15.144:6819/3584068368","cluster_addr":"172.21.15.144:6821/3584068368","heartbeat_back_addr":"172.21.15.144:6825/3584068368","heartbeat_front_addr":"172.21.15.144:6823/3584068368","state":["exists","up"]},{"osd":3,"uuid":"fd8b7396-3962-4c06-ab46-d746cead4b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1366035688},{"type":"v1","addr":"172.21.15.178:6801","nonce":1366035688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1366035688},{"type":"v1","addr":"172.21.15.178:6803","nonce":1366035688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1366035688},{"type":"v1","addr":"172.21.15.178:6807","nonce":1366035688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1366035688},{"type":"v1","addr":"172.21.15.178:6805","nonce":1366035688}]},"public_addr":"172.21.15.178:6801/1366035688","cluster_addr":"172.21.15.178:6803/1366035688","heartbeat_back_addr":"172.21.15.178:6807/1366035688","heartbeat_front_addr":"172.21.15.178:6805/1366035688","state":["exists","up"]},{"osd":4,"uuid":"bc5df32a-01cd-45af-8a8e-9759fe330737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":280235677},{"type":"v1","addr":"172.21.15.178:6809","nonce":280235677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":280235677},{"type":"v1","addr":"172.21.15.178:6811","nonce":280235677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":280235677},{"type":"v1","addr":"172.21.15.178:6815","nonce":280235677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":280235677},{"type":"v1","addr":"172.21.15.178:6813","nonce":280235677}]},"public_addr":"172.21.15.178:6809/280235677","cluster_addr":"172.21.15.178:6811/280235677","heartbeat_back_addr":"172.21.15.178:6815/280235677","heartbeat_front_addr":"172.21.15.178:6813/280235677","state":["exists","up"]},{"osd":5,"uuid":"52ee296f-c365-4e5b-b18a-a281a0f58e04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3024778612},{"type":"v1","addr":"172.21.15.178:6817","nonce":3024778612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3024778612},{"type":"v1","addr":"172.21.15.178:6819","nonce":3024778612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3024778612},{"type":"v1","addr":"172.21.15.178:6823","nonce":3024778612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3024778612},{"type":"v1","addr":"172.21.15.178:6821","nonce":3024778612}]},"public_addr":"172.21.15.178:6817/3024778612","cluster_addr":"172.21.15.178:6819/3024778612","heartbeat_back_addr":"172.21.15.178:6823/3024778612","heartbeat_front_addr":"172.21.15.178:6821/3024778612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:23:06.331946+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:23:34.303341+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:24:02.370765+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:24:26.055226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:24:51.902849+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:25:16.897994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:0/3499523310":"2024-09-11T11:21:47.868365+0000","172.21.15.144:0/2387813590":"2024-09-11T11:20:42.965172+0000","172.21.15.144:0/3102311729":"2024-09-11T11:20:42.965172+0000","172.21.15.144:6801/2":"2024-09-11T11:20:09.967048+0000","172.21.15.144:0/898696771":"2024-09-11T11:21:47.868365+0000","172.21.15.144:6800/2":"2024-09-11T11:20:09.967048+0000","172.21.15.144:0/4053102056":"2024-09-11T11:21:47.868365+0000","172.21.15.144:0/2667523333":"2024-09-11T11:20:42.965172+0000","172.21.15.144:0/2543679831":"2024-09-11T11:20:09.967048+0000","172.21.15.144:0/3946165961":"2024-09-11T11:20:09.967048+0000","172.21.15.144:0/1513284721":"2024-09-11T11:20:09.967048+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:25:35.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.300+0000 7f2eb624a700 1 -- 172.21.15.144:0/4064318381 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2e9806c680 msgr2=0x7f2e9806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:35.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.300+0000 7f2eb624a700 1 --2- 172.21.15.144:0/4064318381 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2e9806c680 0x7f2e9806eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f2ea0006010 tx=0x7f2ea001a040 comp rx=0 tx=0).stop 2024-09-10T11:25:35.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.300+0000 7f2eb624a700 1 -- 172.21.15.144:0/4064318381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb00ffba0 msgr2=0x7f2eb0196350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:35.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.300+0000 7f2eb624a700 1 --2- 172.21.15.144:0/4064318381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb00ffba0 0x7f2eb0196350 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2ea400b4b0 tx=0x7f2ea400b7c0 comp rx=0 tx=0).stop 2024-09-10T11:25:35.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.300+0000 7f2eb624a700 1 -- 172.21.15.144:0/4064318381 shutdown_connections 2024-09-10T11:25:35.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.300+0000 7f2eb624a700 1 --2- 172.21.15.144:0/4064318381 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f2e9806c680 0x7f2e9806eb30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:35.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.300+0000 7f2eb624a700 1 --2- 172.21.15.144:0/4064318381 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2eb00ff290 0x7f2eb0195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:35.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.300+0000 7f2eb624a700 1 --2- 172.21.15.144:0/4064318381 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2eb00ffba0 0x7f2eb0196350 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:35.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.300+0000 7f2eb624a700 1 -- 172.21.15.144:0/4064318381 >> 172.21.15.144:0/4064318381 conn(0x7f2eb0074850 msgr2=0x7f2eb0104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:35.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.300+0000 7f2eb624a700 1 -- 172.21.15.144:0/4064318381 shutdown_connections 2024-09-10T11:25:35.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:35.300+0000 7f2eb624a700 1 -- 172.21.15.144:0/4064318381 wait complete. 2024-09-10T11:25:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:34 smithi178 ceph-mon[33514]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:35.898 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T11:25:35.899 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T11:25:36.099 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:36.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:35 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/4064318381' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T11:25:36.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:35 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/4064318381' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T11:25:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:36 smithi144 ceph-mon[27942]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:36 smithi178 ceph-mon[33514]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:37.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.787+0000 7fd4f8379700 1 -- 172.21.15.144:0/1887923520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4f0068630 msgr2=0x7fd4f0068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:37.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.787+0000 7fd4f8379700 1 --2- 172.21.15.144:0/1887923520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4f0068630 0x7fd4f0068a00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4009910 tx=0x7fd4e4009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:37.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.787+0000 7fd4f8379700 1 -- 172.21.15.144:0/1887923520 shutdown_connections 2024-09-10T11:25:37.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.787+0000 7fd4f8379700 1 --2- 172.21.15.144:0/1887923520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f0068fd0 0x7fd4f0104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:37.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.787+0000 7fd4f8379700 1 --2- 172.21.15.144:0/1887923520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4f0068630 0x7fd4f0068a00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:37.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.787+0000 7fd4f8379700 1 -- 172.21.15.144:0/1887923520 >> 172.21.15.144:0/1887923520 conn(0x7fd4f0074f50 msgr2=0x7fd4f0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:37.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.787+0000 7fd4f8379700 1 -- 172.21.15.144:0/1887923520 shutdown_connections 2024-09-10T11:25:37.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.787+0000 7fd4f8379700 1 -- 172.21.15.144:0/1887923520 wait complete. 2024-09-10T11:25:37.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.788+0000 7fd4f8379700 1 Processor -- start 2024-09-10T11:25:37.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.788+0000 7fd4f8379700 1 -- start start 2024-09-10T11:25:37.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.788+0000 7fd4f8379700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f0068630 0x7fd4f0195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:37.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.788+0000 7fd4f8379700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4f0068fd0 0x7fd4f0196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:37.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.788+0000 7fd4f8379700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f01968c0 con 0x7fd4f0068fd0 2024-09-10T11:25:37.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.788+0000 7fd4f8379700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f0196a00 con 0x7fd4f0068630 2024-09-10T11:25:37.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.788+0000 7fd4f5914700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4f0068fd0 0x7fd4f0196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:37.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.788+0000 7fd4f5914700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4f0068fd0 0x7fd4f0196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40014/0 (socket says 172.21.15.144:40014) 2024-09-10T11:25:37.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.788+0000 7fd4f5914700 1 -- 172.21.15.144:0/210390301 learned_addr learned my addr 172.21.15.144:0/210390301 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:37.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.788+0000 7fd4f6115700 1 --2- 172.21.15.144:0/210390301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f0068630 0x7fd4f0195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:37.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.788+0000 7fd4f5914700 1 -- 172.21.15.144:0/210390301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f0068630 msgr2=0x7fd4f0195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:37.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.788+0000 7fd4f5914700 1 --2- 172.21.15.144:0/210390301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f0068630 0x7fd4f0195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:37.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.788+0000 7fd4f5914700 1 -- 172.21.15.144:0/210390301 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e40095f0 con 0x7fd4f0068fd0 2024-09-10T11:25:37.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.789+0000 7fd4f5914700 1 --2- 172.21.15.144:0/210390301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4f0068fd0 0x7fd4f0196270 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00c960 tx=0x7fd4ec00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:37.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.789+0000 7fd4e37fe700 1 -- 172.21.15.144:0/210390301 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4ec007a70 con 0x7fd4f0068fd0 2024-09-10T11:25:37.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.789+0000 7fd4e37fe700 1 -- 172.21.15.144:0/210390301 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd4ec007bd0 con 0x7fd4f0068fd0 2024-09-10T11:25:37.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.789+0000 7fd4f8379700 1 -- 172.21.15.144:0/210390301 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f019a850 con 0x7fd4f0068fd0 2024-09-10T11:25:37.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.789+0000 7fd4e37fe700 1 -- 172.21.15.144:0/210390301 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4ec018630 con 0x7fd4f0068fd0 2024-09-10T11:25:37.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.789+0000 7fd4f8379700 1 -- 172.21.15.144:0/210390301 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f019ada0 con 0x7fd4f0068fd0 2024-09-10T11:25:37.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.790+0000 7fd4f8379700 1 -- 172.21.15.144:0/210390301 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f01086f0 con 0x7fd4f0068fd0 2024-09-10T11:25:37.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.790+0000 7fd4e37fe700 1 -- 172.21.15.144:0/210390301 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fd4ec00fa90 con 0x7fd4f0068fd0 2024-09-10T11:25:37.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.791+0000 7fd4e37fe700 1 --2- 172.21.15.144:0/210390301 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd4dc06c680 0x7fd4dc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:37.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.791+0000 7fd4f6115700 1 --2- 172.21.15.144:0/210390301 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd4dc06c680 0x7fd4dc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:37.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.791+0000 7fd4e37fe700 1 -- 172.21.15.144:0/210390301 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fd4ec08b530 con 0x7fd4f0068fd0 2024-09-10T11:25:37.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.791+0000 7fd4f6115700 1 --2- 172.21.15.144:0/210390301 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd4dc06c680 0x7fd4dc06eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400b5c0 tx=0x7fd4e401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:37.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.795+0000 7fd4e37fe700 1 -- 172.21.15.144:0/210390301 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd4ec059a80 con 0x7fd4f0068fd0 2024-09-10T11:25:37.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.985+0000 7fd4f8379700 1 -- 172.21.15.144:0/210390301 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4f002cc30 con 0x7fd4f0068fd0 2024-09-10T11:25:37.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.986+0000 7fd4e37fe700 1 -- 172.21.15.144:0/210390301 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+11107 (secure 0 0 0) 0x7fd4ec059610 con 0x7fd4f0068fd0 2024-09-10T11:25:37.987 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:25:37.987 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":41,"fsid":"7eacd86c-6f66-11ef-bcea-c7b262605968","created":"2024-09-10T11:19:32.020786+0000","modified":"2024-09-10T11:25:19.556870+0000","last_up_change":"2024-09-10T11:25:18.551589+0000","last_in_change":"2024-09-10T11:24:59.765382+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T11:24:06.211746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5612a382-bb30-4314-8f65-0925157cf09d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6802","nonce":2127975408},{"type":"v1","addr":"172.21.15.144:6803","nonce":2127975408}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6804","nonce":2127975408},{"type":"v1","addr":"172.21.15.144:6805","nonce":2127975408}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6808","nonce":2127975408},{"type":"v1","addr":"172.21.15.144:6809","nonce":2127975408}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6806","nonce":2127975408},{"type":"v1","addr":"172.21.15.144:6807","nonce":2127975408}]},"public_addr":"172.21.15.144:6803/2127975408","cluster_addr":"172.21.15.144:6805/2127975408","heartbeat_back_addr":"172.21.15.144:6809/2127975408","heartbeat_front_addr":"172.21.15.144:6807/2127975408","state":["exists","up"]},{"osd":1,"uuid":"68977841-c0e7-44ce-8e18-96af001f56f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":30,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6810","nonce":1548583549},{"type":"v1","addr":"172.21.15.144:6811","nonce":1548583549}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6812","nonce":1548583549},{"type":"v1","addr":"172.21.15.144:6813","nonce":1548583549}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6816","nonce":1548583549},{"type":"v1","addr":"172.21.15.144:6817","nonce":1548583549}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6814","nonce":1548583549},{"type":"v1","addr":"172.21.15.144:6815","nonce":1548583549}]},"public_addr":"172.21.15.144:6811/1548583549","cluster_addr":"172.21.15.144:6813/1548583549","heartbeat_back_addr":"172.21.15.144:6817/1548583549","heartbeat_front_addr":"172.21.15.144:6815/1548583549","state":["exists","up"]},{"osd":2,"uuid":"8f07849c-6967-4e40-a8e8-c5f928879f8b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6818","nonce":3584068368},{"type":"v1","addr":"172.21.15.144:6819","nonce":3584068368}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6820","nonce":3584068368},{"type":"v1","addr":"172.21.15.144:6821","nonce":3584068368}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6824","nonce":3584068368},{"type":"v1","addr":"172.21.15.144:6825","nonce":3584068368}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6822","nonce":3584068368},{"type":"v1","addr":"172.21.15.144:6823","nonce":3584068368}]},"public_addr":"172.21.15.144:6819/3584068368","cluster_addr":"172.21.15.144:6821/3584068368","heartbeat_back_addr":"172.21.15.144:6825/3584068368","heartbeat_front_addr":"172.21.15.144:6823/3584068368","state":["exists","up"]},{"osd":3,"uuid":"fd8b7396-3962-4c06-ab46-d746cead4b1d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6800","nonce":1366035688},{"type":"v1","addr":"172.21.15.178:6801","nonce":1366035688}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6802","nonce":1366035688},{"type":"v1","addr":"172.21.15.178:6803","nonce":1366035688}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6806","nonce":1366035688},{"type":"v1","addr":"172.21.15.178:6807","nonce":1366035688}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6804","nonce":1366035688},{"type":"v1","addr":"172.21.15.178:6805","nonce":1366035688}]},"public_addr":"172.21.15.178:6801/1366035688","cluster_addr":"172.21.15.178:6803/1366035688","heartbeat_back_addr":"172.21.15.178:6807/1366035688","heartbeat_front_addr":"172.21.15.178:6805/1366035688","state":["exists","up"]},{"osd":4,"uuid":"bc5df32a-01cd-45af-8a8e-9759fe330737","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":35,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6808","nonce":280235677},{"type":"v1","addr":"172.21.15.178:6809","nonce":280235677}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6810","nonce":280235677},{"type":"v1","addr":"172.21.15.178:6811","nonce":280235677}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6814","nonce":280235677},{"type":"v1","addr":"172.21.15.178:6815","nonce":280235677}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6812","nonce":280235677},{"type":"v1","addr":"172.21.15.178:6813","nonce":280235677}]},"public_addr":"172.21.15.178:6809/280235677","cluster_addr":"172.21.15.178:6811/280235677","heartbeat_back_addr":"172.21.15.178:6815/280235677","heartbeat_front_addr":"172.21.15.178:6813/280235677","state":["exists","up"]},{"osd":5,"uuid":"52ee296f-c365-4e5b-b18a-a281a0f58e04","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":40,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6816","nonce":3024778612},{"type":"v1","addr":"172.21.15.178:6817","nonce":3024778612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6818","nonce":3024778612},{"type":"v1","addr":"172.21.15.178:6819","nonce":3024778612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6822","nonce":3024778612},{"type":"v1","addr":"172.21.15.178:6823","nonce":3024778612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6820","nonce":3024778612},{"type":"v1","addr":"172.21.15.178:6821","nonce":3024778612}]},"public_addr":"172.21.15.178:6817/3024778612","cluster_addr":"172.21.15.178:6819/3024778612","heartbeat_back_addr":"172.21.15.178:6823/3024778612","heartbeat_front_addr":"172.21.15.178:6821/3024778612","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:23:06.331946+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:23:34.303341+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:24:02.370765+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:24:26.055226+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:24:51.902849+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T11:25:16.897994+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.144:0/3499523310":"2024-09-11T11:21:47.868365+0000","172.21.15.144:0/2387813590":"2024-09-11T11:20:42.965172+0000","172.21.15.144:0/3102311729":"2024-09-11T11:20:42.965172+0000","172.21.15.144:6801/2":"2024-09-11T11:20:09.967048+0000","172.21.15.144:0/898696771":"2024-09-11T11:21:47.868365+0000","172.21.15.144:6800/2":"2024-09-11T11:20:09.967048+0000","172.21.15.144:0/4053102056":"2024-09-11T11:21:47.868365+0000","172.21.15.144:0/2667523333":"2024-09-11T11:20:42.965172+0000","172.21.15.144:0/2543679831":"2024-09-11T11:20:09.967048+0000","172.21.15.144:0/3946165961":"2024-09-11T11:20:09.967048+0000","172.21.15.144:0/1513284721":"2024-09-11T11:20:09.967048+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T11:25:37.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.988+0000 7fd4f8379700 1 -- 172.21.15.144:0/210390301 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd4dc06c680 msgr2=0x7fd4dc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:37.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.988+0000 7fd4f8379700 1 --2- 172.21.15.144:0/210390301 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd4dc06c680 0x7fd4dc06eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400b5c0 tx=0x7fd4e401a040 comp rx=0 tx=0).stop 2024-09-10T11:25:37.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.988+0000 7fd4f8379700 1 -- 172.21.15.144:0/210390301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4f0068fd0 msgr2=0x7fd4f0196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:37.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.988+0000 7fd4f8379700 1 --2- 172.21.15.144:0/210390301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4f0068fd0 0x7fd4f0196270 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec00c960 tx=0x7fd4ec00cd20 comp rx=0 tx=0).stop 2024-09-10T11:25:37.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.988+0000 7fd4f8379700 1 -- 172.21.15.144:0/210390301 shutdown_connections 2024-09-10T11:25:37.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.988+0000 7fd4f8379700 1 --2- 172.21.15.144:0/210390301 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd4dc06c680 0x7fd4dc06eb30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:37.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.988+0000 7fd4f8379700 1 --2- 172.21.15.144:0/210390301 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4f0068630 0x7fd4f0195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:37.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.988+0000 7fd4f8379700 1 --2- 172.21.15.144:0/210390301 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4f0068fd0 0x7fd4f0196270 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:37.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.988+0000 7fd4f8379700 1 -- 172.21.15.144:0/210390301 >> 172.21.15.144:0/210390301 conn(0x7fd4f0074f50 msgr2=0x7fd4f00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:37.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.988+0000 7fd4f8379700 1 -- 172.21.15.144:0/210390301 shutdown_connections 2024-09-10T11:25:37.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:37.988+0000 7fd4f8379700 1 -- 172.21.15.144:0/210390301 wait complete. 2024-09-10T11:25:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:38 smithi178 ceph-mon[33514]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:38 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/210390301' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T11:25:38.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:38 smithi144 ceph-mon[27942]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:38.657 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:38 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/210390301' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T11:25:38.682 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T11:25:38.683 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T11:25:38.683 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T11:25:38.684 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T11:25:38.684 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T11:25:38.684 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T11:25:38.946 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:38.998 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:40.387 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:40.387 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:40.388 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:40.407 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:41.033 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:40 smithi144 ceph-mon[27942]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:40 smithi178 ceph-mon[33514]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:43.111 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:42 smithi144 ceph-mon[27942]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:42 smithi178 ceph-mon[33514]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:43.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.606+0000 7f488a3f0700 1 -- 172.21.15.144:0/145464395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4884068630 msgr2=0x7f4884068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:43.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.606+0000 7f488a3f0700 1 --2- 172.21.15.144:0/145464395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4884068630 0x7f4884068a00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f4874009910 tx=0x7f4874009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:43.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.606+0000 7f488a3f0700 1 -- 172.21.15.144:0/145464395 shutdown_connections 2024-09-10T11:25:43.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.606+0000 7f488a3f0700 1 --2- 172.21.15.144:0/145464395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4884068fd0 0x7f4884104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:43.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.606+0000 7f488a3f0700 1 --2- 172.21.15.144:0/145464395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4884068630 0x7f4884068a00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:43.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.607+0000 7f488a3f0700 1 -- 172.21.15.144:0/145464395 >> 172.21.15.144:0/145464395 conn(0x7f4884074f50 msgr2=0x7f4884075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:43.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.607+0000 7f488a3f0700 1 -- 172.21.15.144:0/145464395 shutdown_connections 2024-09-10T11:25:43.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.607+0000 7f488a3f0700 1 -- 172.21.15.144:0/145464395 wait complete. 2024-09-10T11:25:43.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.607+0000 7f488a3f0700 1 Processor -- start 2024-09-10T11:25:43.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.607+0000 7f488a3f0700 1 -- start start 2024-09-10T11:25:43.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f488a3f0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4884068630 0x7f4884195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:43.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f488a3f0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4884068fd0 0x7f4884196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:43.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f488a3f0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48841968c0 con 0x7f4884068fd0 2024-09-10T11:25:43.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f488a3f0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4884196a00 con 0x7f4884068630 2024-09-10T11:25:43.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f48837fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4884068fd0 0x7f4884196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:43.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f48837fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4884068fd0 0x7f4884196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39390/0 (socket says 172.21.15.144:39390) 2024-09-10T11:25:43.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f48837fe700 1 -- 172.21.15.144:0/2639281873 learned_addr learned my addr 172.21.15.144:0/2639281873 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:43.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f4883fff700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4884068630 0x7f4884195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:43.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f48837fe700 1 -- 172.21.15.144:0/2639281873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4884068630 msgr2=0x7f4884195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:43.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f48837fe700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4884068630 0x7f4884195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:43.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f48837fe700 1 -- 172.21.15.144:0/2639281873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48740095f0 con 0x7f4884068fd0 2024-09-10T11:25:43.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f4883fff700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4884068630 0x7f4884195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:25:43.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f48837fe700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4884068fd0 0x7f4884196270 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f487800c960 tx=0x7f487800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:43.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f48817fa700 1 -- 172.21.15.144:0/2639281873 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4878007a70 con 0x7f4884068fd0 2024-09-10T11:25:43.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f488a3f0700 1 -- 172.21.15.144:0/2639281873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f488419a850 con 0x7f4884068fd0 2024-09-10T11:25:43.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f48817fa700 1 -- 172.21.15.144:0/2639281873 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4878007bd0 con 0x7f4884068fd0 2024-09-10T11:25:43.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f48817fa700 1 -- 172.21.15.144:0/2639281873 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4878018630 con 0x7f4884068fd0 2024-09-10T11:25:43.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.608+0000 7f488a3f0700 1 -- 172.21.15.144:0/2639281873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488419ada0 con 0x7f4884068fd0 2024-09-10T11:25:43.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.609+0000 7f488a3f0700 1 -- 172.21.15.144:0/2639281873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f488410d670 con 0x7f4884068fd0 2024-09-10T11:25:43.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.610+0000 7f48817fa700 1 -- 172.21.15.144:0/2639281873 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f487800f450 con 0x7f4884068fd0 2024-09-10T11:25:43.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.610+0000 7f48817fa700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f486c06c680 0x7f486c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:43.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.610+0000 7f48817fa700 1 -- 172.21.15.144:0/2639281873 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f487808b510 con 0x7f4884068fd0 2024-09-10T11:25:43.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.610+0000 7f4883fff700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f486c06c680 0x7f486c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:43.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.610+0000 7f48817fa700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549] conn(0x7f486c072210 0x7f486c074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:43.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.610+0000 7f48817fa700 1 -- 172.21.15.144:0/2639281873 --> [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f486c074cd0 con 0x7f486c072210 2024-09-10T11:25:43.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.610+0000 7f48817fa700 1 -- 172.21.15.144:0/2639281873 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f487808dbe0 con 0x7f4884068fd0 2024-09-10T11:25:43.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.610+0000 7f488898d700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549] conn(0x7f486c072210 0x7f486c074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:43.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.611+0000 7f4883fff700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f486c06c680 0x7f486c06eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4874005b40 tx=0x7f487401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:43.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.611+0000 7f488898d700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549] conn(0x7f486c072210 0x7f486c074620 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:43.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.611+0000 7f48817fa700 1 -- 172.21.15.144:0/2639281873 <== osd.1 v2:172.21.15.144:6810/1548583549 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f486c074cd0 con 0x7f486c072210 2024-09-10T11:25:43.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.627+0000 7f488a3f0700 1 -- 172.21.15.144:0/2639281873 --> [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f488404ea50 con 0x7f486c072210 2024-09-10T11:25:43.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.627+0000 7f48817fa700 1 -- 172.21.15.144:0/2639281873 <== osd.1 v2:172.21.15.144:6810/1548583549 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f488404ea50 con 0x7f486c072210 2024-09-10T11:25:43.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.627+0000 7f488a3f0700 1 -- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549] conn(0x7f486c072210 msgr2=0x7f486c074620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:43.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.627+0000 7f488a3f0700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549] conn(0x7f486c072210 0x7f486c074620 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:43.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.627+0000 7f488a3f0700 1 -- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f486c06c680 msgr2=0x7f486c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:43.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.627+0000 7f488a3f0700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f486c06c680 0x7f486c06eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f4874005b40 tx=0x7f487401a040 comp rx=0 tx=0).stop 2024-09-10T11:25:43.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.627+0000 7f488a3f0700 1 -- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4884068fd0 msgr2=0x7f4884196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:43.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.627+0000 7f488a3f0700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4884068fd0 0x7f4884196270 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f487800c960 tx=0x7f487800cd20 comp rx=0 tx=0).stop 2024-09-10T11:25:43.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.627+0000 7f488a3f0700 1 -- 172.21.15.144:0/2639281873 shutdown_connections 2024-09-10T11:25:43.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.627+0000 7f488a3f0700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:6810/1548583549,v1:172.21.15.144:6811/1548583549] conn(0x7f486c072210 0x7f486c074620 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:43.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.627+0000 7f488a3f0700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f486c06c680 0x7f486c06eb30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:43.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.627+0000 7f488a3f0700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4884068630 0x7f4884195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:43.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.627+0000 7f488a3f0700 1 --2- 172.21.15.144:0/2639281873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4884068fd0 0x7f4884196270 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:43.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.627+0000 7f488a3f0700 1 -- 172.21.15.144:0/2639281873 >> 172.21.15.144:0/2639281873 conn(0x7f4884074f50 msgr2=0x7f48840fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:43.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.628+0000 7f488a3f0700 1 -- 172.21.15.144:0/2639281873 shutdown_connections 2024-09-10T11:25:43.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:43.628+0000 7f488a3f0700 1 -- 172.21.15.144:0/2639281873 wait complete. 2024-09-10T11:25:44.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.804+0000 7f61c249b700 1 -- 172.21.15.144:0/320444679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61bc100400 msgr2=0x7f61bc1007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:44.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.804+0000 7f61c249b700 1 --2- 172.21.15.144:0/320444679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61bc100400 0x7f61bc1007d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f61ac009910 tx=0x7f61ac009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:44.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.804+0000 7f61c249b700 1 -- 172.21.15.144:0/320444679 shutdown_connections 2024-09-10T11:25:44.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.804+0000 7f61c249b700 1 --2- 172.21.15.144:0/320444679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61bc100da0 0x7f61bc104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:44.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.804+0000 7f61c249b700 1 --2- 172.21.15.144:0/320444679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61bc100400 0x7f61bc1007d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:44.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.804+0000 7f61c249b700 1 -- 172.21.15.144:0/320444679 >> 172.21.15.144:0/320444679 conn(0x7f61bc0fbca0 msgr2=0x7f61bc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:44.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.804+0000 7f61c249b700 1 -- 172.21.15.144:0/320444679 shutdown_connections 2024-09-10T11:25:44.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.804+0000 7f61c249b700 1 -- 172.21.15.144:0/320444679 wait complete. 2024-09-10T11:25:44.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.805+0000 7f61c249b700 1 Processor -- start 2024-09-10T11:25:44.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.805+0000 7f61c249b700 1 -- start start 2024-09-10T11:25:44.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.805+0000 7f61c249b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61bc100400 0x7f61bc195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:44.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.805+0000 7f61c249b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61bc100da0 0x7f61bc1962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:44.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.805+0000 7f61c249b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc1969d0 con 0x7f61bc100400 2024-09-10T11:25:44.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.805+0000 7f61c249b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc19a760 con 0x7f61bc100da0 2024-09-10T11:25:44.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.805+0000 7f61bbfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61bc100400 0x7f61bc195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:44.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.805+0000 7f61bbfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61bc100400 0x7f61bc195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39404/0 (socket says 172.21.15.144:39404) 2024-09-10T11:25:44.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.805+0000 7f61bbfff700 1 -- 172.21.15.144:0/607532959 learned_addr learned my addr 172.21.15.144:0/607532959 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:44.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.805+0000 7f61bb7fe700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61bc100da0 0x7f61bc1962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:44.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.806+0000 7f61bbfff700 1 -- 172.21.15.144:0/607532959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61bc100da0 msgr2=0x7f61bc1962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:44.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.806+0000 7f61bbfff700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61bc100da0 0x7f61bc1962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:44.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.806+0000 7f61bbfff700 1 -- 172.21.15.144:0/607532959 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac0095f0 con 0x7f61bc100400 2024-09-10T11:25:44.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.806+0000 7f61bbfff700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61bc100400 0x7f61bc195db0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f61ac006010 tx=0x7f61ac00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:44.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.806+0000 7f61b97fa700 1 -- 172.21.15.144:0/607532959 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61ac01d070 con 0x7f61bc100400 2024-09-10T11:25:44.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.806+0000 7f61b97fa700 1 -- 172.21.15.144:0/607532959 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f61ac004fa0 con 0x7f61bc100400 2024-09-10T11:25:44.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.806+0000 7f61c249b700 1 -- 172.21.15.144:0/607532959 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61bc19a9e0 con 0x7f61bc100400 2024-09-10T11:25:44.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.806+0000 7f61b97fa700 1 -- 172.21.15.144:0/607532959 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61ac00f690 con 0x7f61bc100400 2024-09-10T11:25:44.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.806+0000 7f61c249b700 1 -- 172.21.15.144:0/607532959 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61bc19aed0 con 0x7f61bc100400 2024-09-10T11:25:44.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.807+0000 7f61c249b700 1 -- 172.21.15.144:0/607532959 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f61bc10d690 con 0x7f61bc100400 2024-09-10T11:25:44.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.807+0000 7f61b97fa700 1 -- 172.21.15.144:0/607532959 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f61ac022470 con 0x7f61bc100400 2024-09-10T11:25:44.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.808+0000 7f61b97fa700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f61a806c680 0x7f61a806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:44.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.808+0000 7f61bb7fe700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f61a806c680 0x7f61a806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:44.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.808+0000 7f61b97fa700 1 -- 172.21.15.144:0/607532959 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f61ac08d5c0 con 0x7f61bc100400 2024-09-10T11:25:44.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.808+0000 7f61b97fa700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.178:6800/1366035688,v1:172.21.15.178:6801/1366035688] conn(0x7f61a8072210 0x7f61a8074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:44.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.808+0000 7f61b97fa700 1 -- 172.21.15.144:0/607532959 --> [v2:172.21.15.178:6800/1366035688,v1:172.21.15.178:6801/1366035688] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f61a8074cd0 con 0x7f61a8072210 2024-09-10T11:25:44.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.808+0000 7f61b97fa700 1 -- 172.21.15.144:0/607532959 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61ac08da20 con 0x7f61bc100400 2024-09-10T11:25:44.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.808+0000 7f61bb7fe700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f61a806c680 0x7f61a806eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f61bc1973d0 tx=0x7f61a4009190 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:44.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.808+0000 7f61c0a38700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.178:6800/1366035688,v1:172.21.15.178:6801/1366035688] conn(0x7f61a8072210 0x7f61a8074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:44.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.809+0000 7f61c0a38700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.178:6800/1366035688,v1:172.21.15.178:6801/1366035688] conn(0x7f61a8072210 0x7f61a8074620 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:44.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.810+0000 7f61b97fa700 1 -- 172.21.15.144:0/607532959 <== osd.3 v2:172.21.15.178:6800/1366035688 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f61a8074cd0 con 0x7f61a8072210 2024-09-10T11:25:44.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.825+0000 7f61c249b700 1 -- 172.21.15.144:0/607532959 --> [v2:172.21.15.178:6800/1366035688,v1:172.21.15.178:6801/1366035688] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f61bc04ea50 con 0x7f61a8072210 2024-09-10T11:25:44.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.826+0000 7f61b97fa700 1 -- 172.21.15.144:0/607532959 <== osd.3 v2:172.21.15.178:6800/1366035688 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f61bc04ea50 con 0x7f61a8072210 2024-09-10T11:25:44.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.826+0000 7f61c249b700 1 -- 172.21.15.144:0/607532959 >> [v2:172.21.15.178:6800/1366035688,v1:172.21.15.178:6801/1366035688] conn(0x7f61a8072210 msgr2=0x7f61a8074620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:44.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.826+0000 7f61c249b700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.178:6800/1366035688,v1:172.21.15.178:6801/1366035688] conn(0x7f61a8072210 0x7f61a8074620 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:44.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.826+0000 7f61c249b700 1 -- 172.21.15.144:0/607532959 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f61a806c680 msgr2=0x7f61a806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:44.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.826+0000 7f61c249b700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f61a806c680 0x7f61a806eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f61bc1973d0 tx=0x7f61a4009190 comp rx=0 tx=0).stop 2024-09-10T11:25:44.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.826+0000 7f61c249b700 1 -- 172.21.15.144:0/607532959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61bc100400 msgr2=0x7f61bc195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:44.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.826+0000 7f61c249b700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61bc100400 0x7f61bc195db0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f61ac006010 tx=0x7f61ac00bd70 comp rx=0 tx=0).stop 2024-09-10T11:25:44.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.826+0000 7f61c249b700 1 -- 172.21.15.144:0/607532959 shutdown_connections 2024-09-10T11:25:44.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.826+0000 7f61c249b700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.178:6800/1366035688,v1:172.21.15.178:6801/1366035688] conn(0x7f61a8072210 0x7f61a8074620 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:44.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.826+0000 7f61c249b700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f61a806c680 0x7f61a806eb30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:44.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.826+0000 7f61c249b700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61bc100400 0x7f61bc195db0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:44.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.826+0000 7f61c249b700 1 --2- 172.21.15.144:0/607532959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61bc100da0 0x7f61bc1962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:44.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.826+0000 7f61c249b700 1 -- 172.21.15.144:0/607532959 >> 172.21.15.144:0/607532959 conn(0x7f61bc0fbca0 msgr2=0x7f61bc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:44.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.826+0000 7f61c249b700 1 -- 172.21.15.144:0/607532959 shutdown_connections 2024-09-10T11:25:44.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:44.826+0000 7f61c249b700 1 -- 172.21.15.144:0/607532959 wait complete. 2024-09-10T11:25:45.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.008+0000 7fad588df700 1 -- 172.21.15.144:0/3099733408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad540ff310 msgr2=0x7fad540ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:45.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.008+0000 7fad588df700 1 --2- 172.21.15.144:0/3099733408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad540ff310 0x7fad540ff6e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fad44009910 tx=0x7fad44009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:45.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.008+0000 7fad588df700 1 -- 172.21.15.144:0/3099733408 shutdown_connections 2024-09-10T11:25:45.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.008+0000 7fad588df700 1 --2- 172.21.15.144:0/3099733408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad540ffcb0 0x7fad54103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:45.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.008+0000 7fad588df700 1 --2- 172.21.15.144:0/3099733408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad540ff310 0x7fad540ff6e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:45.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.008+0000 7fad588df700 1 -- 172.21.15.144:0/3099733408 >> 172.21.15.144:0/3099733408 conn(0x7fad540fabb0 msgr2=0x7fad540fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:45.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.008+0000 7fad588df700 1 -- 172.21.15.144:0/3099733408 shutdown_connections 2024-09-10T11:25:45.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.008+0000 7fad588df700 1 -- 172.21.15.144:0/3099733408 wait complete. 2024-09-10T11:25:45.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.008+0000 7fad588df700 1 Processor -- start 2024-09-10T11:25:45.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.009+0000 7fad588df700 1 -- start start 2024-09-10T11:25:45.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.009+0000 7fad588df700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad540ff310 0x7fad5419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:45.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.009+0000 7fad588df700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad540ffcb0 0x7fad5419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:45.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.009+0000 7fad588df700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5419b5f0 con 0x7fad540ff310 2024-09-10T11:25:45.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.009+0000 7fad588df700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad54194aa0 con 0x7fad540ffcb0 2024-09-10T11:25:45.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.009+0000 7fad5259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad540ff310 0x7fad5419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:45.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.009+0000 7fad5259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad540ff310 0x7fad5419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39436/0 (socket says 172.21.15.144:39436) 2024-09-10T11:25:45.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.009+0000 7fad5259c700 1 -- 172.21.15.144:0/4230741645 learned_addr learned my addr 172.21.15.144:0/4230741645 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:45.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.009+0000 7fad4bfff700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad540ffcb0 0x7fad5419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:45.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.009+0000 7fad5259c700 1 -- 172.21.15.144:0/4230741645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad540ffcb0 msgr2=0x7fad5419af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:45.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.009+0000 7fad5259c700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad540ffcb0 0x7fad5419af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:45.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.009+0000 7fad5259c700 1 -- 172.21.15.144:0/4230741645 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad440095f0 con 0x7fad540ff310 2024-09-10T11:25:45.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.010+0000 7fad5259c700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad540ff310 0x7fad5419aa20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fad44006010 tx=0x7fad4400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:45.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.010+0000 7fad4b7fe700 1 -- 172.21.15.144:0/4230741645 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad4401d070 con 0x7fad540ff310 2024-09-10T11:25:45.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.010+0000 7fad4b7fe700 1 -- 172.21.15.144:0/4230741645 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fad4400f460 con 0x7fad540ff310 2024-09-10T11:25:45.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.010+0000 7fad4b7fe700 1 -- 172.21.15.144:0/4230741645 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad44021640 con 0x7fad540ff310 2024-09-10T11:25:45.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.010+0000 7fad588df700 1 -- 172.21.15.144:0/4230741645 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad54194d20 con 0x7fad540ff310 2024-09-10T11:25:45.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.010+0000 7fad588df700 1 -- 172.21.15.144:0/4230741645 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad541951c0 con 0x7fad540ff310 2024-09-10T11:25:45.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.011+0000 7fad588df700 1 -- 172.21.15.144:0/4230741645 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fad5410c4f0 con 0x7fad540ff310 2024-09-10T11:25:45.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.012+0000 7fad4b7fe700 1 -- 172.21.15.144:0/4230741645 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fad4400faf0 con 0x7fad540ff310 2024-09-10T11:25:45.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.012+0000 7fad4b7fe700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fad40070a90 0x7fad40072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:45.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.012+0000 7fad4b7fe700 1 -- 172.21.15.144:0/4230741645 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fad4408c810 con 0x7fad540ff310 2024-09-10T11:25:45.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.012+0000 7fad4bfff700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fad40070a90 0x7fad40072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:45.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.012+0000 7fad4b7fe700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368] conn(0x7fad40076620 0x7fad40078a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:45.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.012+0000 7fad4b7fe700 1 -- 172.21.15.144:0/4230741645 --> [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fad400790e0 con 0x7fad40076620 2024-09-10T11:25:45.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.012+0000 7fad4b7fe700 1 -- 172.21.15.144:0/4230741645 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad440912a0 con 0x7fad540ff310 2024-09-10T11:25:45.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.012+0000 7fad52d9d700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368] conn(0x7fad40076620 0x7fad40078a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:45.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.013+0000 7fad4bfff700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fad40070a90 0x7fad40072f40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fad3c005fd0 tx=0x7fad3c00c500 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:45.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.013+0000 7fad52d9d700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368] conn(0x7fad40076620 0x7fad40078a30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:45.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.014+0000 7fad4b7fe700 1 -- 172.21.15.144:0/4230741645 <== osd.2 v2:172.21.15.144:6818/3584068368 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fad400790e0 con 0x7fad40076620 2024-09-10T11:25:45.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.031+0000 7fad588df700 1 -- 172.21.15.144:0/4230741645 --> [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fad54195730 con 0x7fad40076620 2024-09-10T11:25:45.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.031+0000 7fad4b7fe700 1 -- 172.21.15.144:0/4230741645 <== osd.2 v2:172.21.15.144:6818/3584068368 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fad54195730 con 0x7fad40076620 2024-09-10T11:25:45.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.031+0000 7fad588df700 1 -- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368] conn(0x7fad40076620 msgr2=0x7fad40078a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:45.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.031+0000 7fad588df700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368] conn(0x7fad40076620 0x7fad40078a30 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:45.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.032+0000 7fad588df700 1 -- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fad40070a90 msgr2=0x7fad40072f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:45.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.032+0000 7fad588df700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fad40070a90 0x7fad40072f40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fad3c005fd0 tx=0x7fad3c00c500 comp rx=0 tx=0).stop 2024-09-10T11:25:45.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.032+0000 7fad588df700 1 -- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad540ff310 msgr2=0x7fad5419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:45.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.032+0000 7fad588df700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad540ff310 0x7fad5419aa20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fad44006010 tx=0x7fad4400be20 comp rx=0 tx=0).stop 2024-09-10T11:25:45.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.032+0000 7fad588df700 1 -- 172.21.15.144:0/4230741645 shutdown_connections 2024-09-10T11:25:45.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.032+0000 7fad588df700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fad40070a90 0x7fad40072f40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:45.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.032+0000 7fad588df700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:6818/3584068368,v1:172.21.15.144:6819/3584068368] conn(0x7fad40076620 0x7fad40078a30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:45.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.032+0000 7fad588df700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad540ff310 0x7fad5419aa20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:45.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.032+0000 7fad588df700 1 --2- 172.21.15.144:0/4230741645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad540ffcb0 0x7fad5419af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:45.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.032+0000 7fad588df700 1 -- 172.21.15.144:0/4230741645 >> 172.21.15.144:0/4230741645 conn(0x7fad540fabb0 msgr2=0x7fad540fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:45.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.032+0000 7fad588df700 1 -- 172.21.15.144:0/4230741645 shutdown_connections 2024-09-10T11:25:45.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:45.032+0000 7fad588df700 1 -- 172.21.15.144:0/4230741645 wait complete. 2024-09-10T11:25:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:44 smithi178 ceph-mon[33514]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:44 smithi144 ceph-mon[27942]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:46.242 INFO:teuthology.orchestra.run.smithi144.stdout:60129542172 2024-09-10T11:25:46.243 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T11:25:46.814 INFO:teuthology.orchestra.run.smithi144.stdout:115964117010 2024-09-10T11:25:46.914 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T11:25:47.174 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:46 smithi144 ceph-mon[27942]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:46 smithi178 ceph-mon[33514]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:47.846 INFO:teuthology.orchestra.run.smithi144.stdout:81604378646 2024-09-10T11:25:47.847 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T11:25:48.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.531+0000 7f7b32f7f700 1 -- 172.21.15.144:0/1319316148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c0ffca0 msgr2=0x7f7b2c103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:48.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.531+0000 7f7b32f7f700 1 --2- 172.21.15.144:0/1319316148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c0ffca0 0x7f7b2c103c70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c00b0a0 tx=0x7f7b1c00b3b0 comp rx=0 tx=0).stop 2024-09-10T11:25:48.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.531+0000 7f7b32f7f700 1 -- 172.21.15.144:0/1319316148 shutdown_connections 2024-09-10T11:25:48.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.531+0000 7f7b32f7f700 1 --2- 172.21.15.144:0/1319316148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c0ffca0 0x7f7b2c103c70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:48.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.531+0000 7f7b32f7f700 1 --2- 172.21.15.144:0/1319316148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b2c0ff300 0x7f7b2c0ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:48.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.531+0000 7f7b32f7f700 1 -- 172.21.15.144:0/1319316148 >> 172.21.15.144:0/1319316148 conn(0x7f7b2c0faba0 msgr2=0x7f7b2c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:48.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.531+0000 7f7b32f7f700 1 -- 172.21.15.144:0/1319316148 shutdown_connections 2024-09-10T11:25:48.533 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:48 smithi144 ceph-mon[27942]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:48.533 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:25:48.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.531+0000 7f7b32f7f700 1 -- 172.21.15.144:0/1319316148 wait complete. 2024-09-10T11:25:48.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.532+0000 7f7b32f7f700 1 Processor -- start 2024-09-10T11:25:48.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.532+0000 7f7b32f7f700 1 -- start start 2024-09-10T11:25:48.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.532+0000 7f7b32f7f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b2c0ff300 0x7f7b2c196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:48.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.532+0000 7f7b32f7f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c0ffca0 0x7f7b2c197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:48.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.532+0000 7f7b32f7f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c1977a0 con 0x7f7b2c0ffca0 2024-09-10T11:25:48.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.532+0000 7f7b32f7f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2c197910 con 0x7f7b2c0ff300 2024-09-10T11:25:48.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.532+0000 7f7b2bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c0ffca0 0x7f7b2c197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:48.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.532+0000 7f7b2bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c0ffca0 0x7f7b2c197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39468/0 (socket says 172.21.15.144:39468) 2024-09-10T11:25:48.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.532+0000 7f7b2bfff700 1 -- 172.21.15.144:0/2762098297 learned_addr learned my addr 172.21.15.144:0/2762098297 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:48.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.532+0000 7f7b30d1b700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b2c0ff300 0x7f7b2c196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:48.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.532+0000 7f7b2bfff700 1 -- 172.21.15.144:0/2762098297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b2c0ff300 msgr2=0x7f7b2c196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:48.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.532+0000 7f7b2bfff700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b2c0ff300 0x7f7b2c196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:48.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.532+0000 7f7b2bfff700 1 -- 172.21.15.144:0/2762098297 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b1c009d00 con 0x7f7b2c0ffca0 2024-09-10T11:25:48.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.533+0000 7f7b2bfff700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c0ffca0 0x7f7b2c197180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c006130 tx=0x7f7b1c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:48.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.533+0000 7f7b29ffb700 1 -- 172.21.15.144:0/2762098297 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b1c00e040 con 0x7f7b2c0ffca0 2024-09-10T11:25:48.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.533+0000 7f7b32f7f700 1 -- 172.21.15.144:0/2762098297 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b2c19fcb0 con 0x7f7b2c0ffca0 2024-09-10T11:25:48.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.533+0000 7f7b29ffb700 1 -- 172.21.15.144:0/2762098297 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7b1c0044f0 con 0x7f7b2c0ffca0 2024-09-10T11:25:48.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.533+0000 7f7b29ffb700 1 -- 172.21.15.144:0/2762098297 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b1c01d7d0 con 0x7f7b2c0ffca0 2024-09-10T11:25:48.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.533+0000 7f7b32f7f700 1 -- 172.21.15.144:0/2762098297 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b2c1a0120 con 0x7f7b2c0ffca0 2024-09-10T11:25:48.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.534+0000 7f7b32f7f700 1 -- 172.21.15.144:0/2762098297 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7b2c10c430 con 0x7f7b2c0ffca0 2024-09-10T11:25:48.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.535+0000 7f7b29ffb700 1 -- 172.21.15.144:0/2762098297 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f7b1c004040 con 0x7f7b2c0ffca0 2024-09-10T11:25:48.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.535+0000 7f7b29ffb700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7b1806c680 0x7f7b1806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:48.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.535+0000 7f7b29ffb700 1 -- 172.21.15.144:0/2762098297 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f7b1c08d8a0 con 0x7f7b2c0ffca0 2024-09-10T11:25:48.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.535+0000 7f7b30d1b700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7b1806c680 0x7f7b1806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:48.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.535+0000 7f7b29ffb700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.178:6816/3024778612,v1:172.21.15.178:6817/3024778612] conn(0x7f7b18072210 0x7f7b18074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:48.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.535+0000 7f7b29ffb700 1 -- 172.21.15.144:0/2762098297 --> [v2:172.21.15.178:6816/3024778612,v1:172.21.15.178:6817/3024778612] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b18074cd0 con 0x7f7b18072210 2024-09-10T11:25:48.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.535+0000 7f7b29ffb700 1 -- 172.21.15.144:0/2762098297 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b1c08dd00 con 0x7f7b2c0ffca0 2024-09-10T11:25:48.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.536+0000 7f7b3151c700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.178:6816/3024778612,v1:172.21.15.178:6817/3024778612] conn(0x7f7b18072210 0x7f7b18074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:48.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.536+0000 7f7b30d1b700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7b1806c680 0x7f7b1806eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b24005950 tx=0x7f7b2400b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:48.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.536+0000 7f7b3151c700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.178:6816/3024778612,v1:172.21.15.178:6817/3024778612] conn(0x7f7b18072210 0x7f7b18074620 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:48.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.537+0000 7f7b29ffb700 1 -- 172.21.15.144:0/2762098297 <== osd.5 v2:172.21.15.178:6816/3024778612 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7b18074cd0 con 0x7f7b18072210 2024-09-10T11:25:48.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.552+0000 7f7b32f7f700 1 -- 172.21.15.144:0/2762098297 --> [v2:172.21.15.178:6816/3024778612,v1:172.21.15.178:6817/3024778612] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7b2c04ea50 con 0x7f7b18072210 2024-09-10T11:25:48.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.553+0000 7f7b29ffb700 1 -- 172.21.15.144:0/2762098297 <== osd.5 v2:172.21.15.178:6816/3024778612 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7b2c04ea50 con 0x7f7b18072210 2024-09-10T11:25:48.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.553+0000 7f7b32f7f700 1 -- 172.21.15.144:0/2762098297 >> [v2:172.21.15.178:6816/3024778612,v1:172.21.15.178:6817/3024778612] conn(0x7f7b18072210 msgr2=0x7f7b18074620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:48.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.553+0000 7f7b32f7f700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.178:6816/3024778612,v1:172.21.15.178:6817/3024778612] conn(0x7f7b18072210 0x7f7b18074620 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:48.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.553+0000 7f7b32f7f700 1 -- 172.21.15.144:0/2762098297 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7b1806c680 msgr2=0x7f7b1806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:48.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.553+0000 7f7b32f7f700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7b1806c680 0x7f7b1806eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b24005950 tx=0x7f7b2400b410 comp rx=0 tx=0).stop 2024-09-10T11:25:48.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.553+0000 7f7b32f7f700 1 -- 172.21.15.144:0/2762098297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c0ffca0 msgr2=0x7f7b2c197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:48.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.553+0000 7f7b32f7f700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c0ffca0 0x7f7b2c197180 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b1c006130 tx=0x7f7b1c00cd20 comp rx=0 tx=0).stop 2024-09-10T11:25:48.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.553+0000 7f7b32f7f700 1 -- 172.21.15.144:0/2762098297 shutdown_connections 2024-09-10T11:25:48.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.553+0000 7f7b32f7f700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.178:6816/3024778612,v1:172.21.15.178:6817/3024778612] conn(0x7f7b18072210 0x7f7b18074620 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:48.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.553+0000 7f7b32f7f700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7b1806c680 0x7f7b1806eb30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:48.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.553+0000 7f7b32f7f700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b2c0ff300 0x7f7b2c196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:48.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.553+0000 7f7b32f7f700 1 --2- 172.21.15.144:0/2762098297 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b2c0ffca0 0x7f7b2c197180 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:48.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.553+0000 7f7b32f7f700 1 -- 172.21.15.144:0/2762098297 >> 172.21.15.144:0/2762098297 conn(0x7f7b2c0faba0 msgr2=0x7f7b2c0fc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:48.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.553+0000 7f7b32f7f700 1 -- 172.21.15.144:0/2762098297 shutdown_connections 2024-09-10T11:25:48.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:48.554+0000 7f7b32f7f700 1 -- 172.21.15.144:0/2762098297 wait complete. 2024-09-10T11:25:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:48 smithi178 ceph-mon[33514]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:25:49.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.350+0000 7f3b4e979700 1 -- 172.21.15.144:0/1559927339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b48100d70 msgr2=0x7f3b48104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:49.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.350+0000 7f3b4e979700 1 --2- 172.21.15.144:0/1559927339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b48100d70 0x7f3b48104d40 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009910 tx=0x7f3b3c009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:49.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.350+0000 7f3b4e979700 1 -- 172.21.15.144:0/1559927339 shutdown_connections 2024-09-10T11:25:49.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.350+0000 7f3b4e979700 1 --2- 172.21.15.144:0/1559927339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b48100d70 0x7f3b48104d40 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.350+0000 7f3b4e979700 1 --2- 172.21.15.144:0/1559927339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b481003d0 0x7f3b481007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.350+0000 7f3b4e979700 1 -- 172.21.15.144:0/1559927339 >> 172.21.15.144:0/1559927339 conn(0x7f3b480fbc90 msgr2=0x7f3b480fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:49.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.350+0000 7f3b4e979700 1 -- 172.21.15.144:0/1559927339 shutdown_connections 2024-09-10T11:25:49.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.350+0000 7f3b4e979700 1 -- 172.21.15.144:0/1559927339 wait complete. 2024-09-10T11:25:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.351+0000 7f3b4e979700 1 Processor -- start 2024-09-10T11:25:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.351+0000 7f3b4e979700 1 -- start start 2024-09-10T11:25:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.351+0000 7f3b4e979700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b481003d0 0x7f3b48195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.351+0000 7f3b4e979700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b48100d70 0x7f3b48196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.351+0000 7f3b4e979700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b48196960 con 0x7f3b481003d0 2024-09-10T11:25:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.351+0000 7f3b4e979700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b4819a6f0 con 0x7f3b48100d70 2024-09-10T11:25:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.352+0000 7f3b47fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b481003d0 0x7f3b48195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.352+0000 7f3b47fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b481003d0 0x7f3b48195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39478/0 (socket says 172.21.15.144:39478) 2024-09-10T11:25:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.352+0000 7f3b47fff700 1 -- 172.21.15.144:0/2211656046 learned_addr learned my addr 172.21.15.144:0/2211656046 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.352+0000 7f3b477fe700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b48100d70 0x7f3b48196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.352+0000 7f3b47fff700 1 -- 172.21.15.144:0/2211656046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b48100d70 msgr2=0x7f3b48196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.352+0000 7f3b47fff700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b48100d70 0x7f3b48196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.352+0000 7f3b47fff700 1 -- 172.21.15.144:0/2211656046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c0095f0 con 0x7f3b481003d0 2024-09-10T11:25:49.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.352+0000 7f3b47fff700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b481003d0 0x7f3b48195d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800d8d0 tx=0x7f3b3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:49.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.352+0000 7f3b457fa700 1 -- 172.21.15.144:0/2211656046 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b38009940 con 0x7f3b481003d0 2024-09-10T11:25:49.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.352+0000 7f3b457fa700 1 -- 172.21.15.144:0/2211656046 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3b38010460 con 0x7f3b481003d0 2024-09-10T11:25:49.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.352+0000 7f3b4e979700 1 -- 172.21.15.144:0/2211656046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b4819a9d0 con 0x7f3b481003d0 2024-09-10T11:25:49.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.352+0000 7f3b457fa700 1 -- 172.21.15.144:0/2211656046 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b3800f5f0 con 0x7f3b481003d0 2024-09-10T11:25:49.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.352+0000 7f3b4e979700 1 -- 172.21.15.144:0/2211656046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b4819af20 con 0x7f3b481003d0 2024-09-10T11:25:49.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.353+0000 7f3b4e979700 1 -- 172.21.15.144:0/2211656046 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3b4810d5c0 con 0x7f3b481003d0 2024-09-10T11:25:49.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.354+0000 7f3b457fa700 1 -- 172.21.15.144:0/2211656046 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f3b380105d0 con 0x7f3b481003d0 2024-09-10T11:25:49.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.354+0000 7f3b457fa700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b3006c680 0x7f3b3006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:49.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.354+0000 7f3b457fa700 1 -- 172.21.15.144:0/2211656046 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f3b3808b980 con 0x7f3b481003d0 2024-09-10T11:25:49.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.354+0000 7f3b477fe700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b3006c680 0x7f3b3006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:49.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.354+0000 7f3b457fa700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408] conn(0x7f3b30072210 0x7f3b30074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:49.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.354+0000 7f3b457fa700 1 -- 172.21.15.144:0/2211656046 --> [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3b30074cd0 con 0x7f3b30072210 2024-09-10T11:25:49.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.354+0000 7f3b457fa700 1 -- 172.21.15.144:0/2211656046 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b3808bde0 con 0x7f3b481003d0 2024-09-10T11:25:49.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.354+0000 7f3b4cf16700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408] conn(0x7f3b30072210 0x7f3b30074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:49.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.355+0000 7f3b477fe700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b3006c680 0x7f3b3006eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3b48197360 tx=0x7f3b3c01a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:49.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.355+0000 7f3b4cf16700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408] conn(0x7f3b30072210 0x7f3b30074620 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:49.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.355+0000 7f3b457fa700 1 -- 172.21.15.144:0/2211656046 <== osd.0 v2:172.21.15.144:6802/2127975408 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3b30074cd0 con 0x7f3b30072210 2024-09-10T11:25:49.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.373+0000 7f3b4e979700 1 -- 172.21.15.144:0/2211656046 --> [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3b4804ea50 con 0x7f3b30072210 2024-09-10T11:25:49.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.374+0000 7f3b457fa700 1 -- 172.21.15.144:0/2211656046 <== osd.0 v2:172.21.15.144:6802/2127975408 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f3b4804ea50 con 0x7f3b30072210 2024-09-10T11:25:49.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.374+0000 7f3b4e979700 1 -- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408] conn(0x7f3b30072210 msgr2=0x7f3b30074620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:49.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.374+0000 7f3b4e979700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408] conn(0x7f3b30072210 0x7f3b30074620 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.374+0000 7f3b4e979700 1 -- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b3006c680 msgr2=0x7f3b3006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:49.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.374+0000 7f3b4e979700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b3006c680 0x7f3b3006eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3b48197360 tx=0x7f3b3c01a040 comp rx=0 tx=0).stop 2024-09-10T11:25:49.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.374+0000 7f3b4e979700 1 -- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b481003d0 msgr2=0x7f3b48195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:49.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.374+0000 7f3b4e979700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b481003d0 0x7f3b48195d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800d8d0 tx=0x7f3b3800dc90 comp rx=0 tx=0).stop 2024-09-10T11:25:49.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.374+0000 7f3b4e979700 1 -- 172.21.15.144:0/2211656046 shutdown_connections 2024-09-10T11:25:49.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.374+0000 7f3b4e979700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:6802/2127975408,v1:172.21.15.144:6803/2127975408] conn(0x7f3b30072210 0x7f3b30074620 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.374+0000 7f3b4e979700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b3006c680 0x7f3b3006eb30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.374+0000 7f3b4e979700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b481003d0 0x7f3b48195d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.374+0000 7f3b4e979700 1 --2- 172.21.15.144:0/2211656046 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b48100d70 0x7f3b48196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.374+0000 7f3b4e979700 1 -- 172.21.15.144:0/2211656046 >> 172.21.15.144:0/2211656046 conn(0x7f3b480fbc90 msgr2=0x7f3b480fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:49.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.374+0000 7f3b4e979700 1 -- 172.21.15.144:0/2211656046 shutdown_connections 2024-09-10T11:25:49.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.374+0000 7f3b4e979700 1 -- 172.21.15.144:0/2211656046 wait complete. 2024-09-10T11:25:49.965 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.962+0000 7f3a2a974700 1 -- 172.21.15.144:0/2326751877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a240ffc70 msgr2=0x7f3a24103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:49.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.962+0000 7f3a2a974700 1 --2- 172.21.15.144:0/2326751877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a240ffc70 0x7f3a24103c40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3a14009910 tx=0x7f3a14009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:49.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.962+0000 7f3a2a974700 1 -- 172.21.15.144:0/2326751877 shutdown_connections 2024-09-10T11:25:49.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.962+0000 7f3a2a974700 1 --2- 172.21.15.144:0/2326751877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a240ffc70 0x7f3a24103c40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.962+0000 7f3a2a974700 1 --2- 172.21.15.144:0/2326751877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a240ff2d0 0x7f3a240ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.962+0000 7f3a2a974700 1 -- 172.21.15.144:0/2326751877 >> 172.21.15.144:0/2326751877 conn(0x7f3a240fabb0 msgr2=0x7f3a240fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:49.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.962+0000 7f3a2a974700 1 -- 172.21.15.144:0/2326751877 shutdown_connections 2024-09-10T11:25:49.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.962+0000 7f3a2a974700 1 -- 172.21.15.144:0/2326751877 wait complete. 2024-09-10T11:25:49.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.963+0000 7f3a2a974700 1 Processor -- start 2024-09-10T11:25:49.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.963+0000 7f3a2a974700 1 -- start start 2024-09-10T11:25:49.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.963+0000 7f3a2a974700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a240ff2d0 0x7f3a24196bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:49.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.963+0000 7f3a2a974700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a240ffc70 0x7f3a24197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:49.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.963+0000 7f3a2a974700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a24197730 con 0x7f3a240ffc70 2024-09-10T11:25:49.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.963+0000 7f3a2a974700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a24197870 con 0x7f3a240ff2d0 2024-09-10T11:25:49.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.963+0000 7f3a237fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a240ffc70 0x7f3a24197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:49.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.963+0000 7f3a237fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a240ffc70 0x7f3a24197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39494/0 (socket says 172.21.15.144:39494) 2024-09-10T11:25:49.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.963+0000 7f3a237fe700 1 -- 172.21.15.144:0/905523750 learned_addr learned my addr 172.21.15.144:0/905523750 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:49.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.963+0000 7f3a23fff700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a240ff2d0 0x7f3a24196bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:49.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.963+0000 7f3a237fe700 1 -- 172.21.15.144:0/905523750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a240ff2d0 msgr2=0x7f3a24196bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:49.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.963+0000 7f3a237fe700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a240ff2d0 0x7f3a24196bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.963+0000 7f3a237fe700 1 -- 172.21.15.144:0/905523750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a140095f0 con 0x7f3a240ffc70 2024-09-10T11:25:49.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.964+0000 7f3a237fe700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a240ffc70 0x7f3a24197110 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3a140092e0 tx=0x7f3a1400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:49.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.964+0000 7f3a217fa700 1 -- 172.21.15.144:0/905523750 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a1401d070 con 0x7f3a240ffc70 2024-09-10T11:25:49.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.964+0000 7f3a2a974700 1 -- 172.21.15.144:0/905523750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2419fc40 con 0x7f3a240ffc70 2024-09-10T11:25:49.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.964+0000 7f3a217fa700 1 -- 172.21.15.144:0/905523750 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3a1400f460 con 0x7f3a240ffc70 2024-09-10T11:25:49.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.964+0000 7f3a217fa700 1 -- 172.21.15.144:0/905523750 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a14017630 con 0x7f3a240ffc70 2024-09-10T11:25:49.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.964+0000 7f3a2a974700 1 -- 172.21.15.144:0/905523750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a241a0050 con 0x7f3a240ffc70 2024-09-10T11:25:49.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.965+0000 7f3a2a974700 1 -- 172.21.15.144:0/905523750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3a2410c480 con 0x7f3a240ffc70 2024-09-10T11:25:49.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.965+0000 7f3a217fa700 1 -- 172.21.15.144:0/905523750 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f3a1400f5d0 con 0x7f3a240ffc70 2024-09-10T11:25:49.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.966+0000 7f3a217fa700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3a0c06c680 0x7f3a0c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:49.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.966+0000 7f3a23fff700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3a0c06c680 0x7f3a0c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:49.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.966+0000 7f3a217fa700 1 -- 172.21.15.144:0/905523750 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f3a1408cd30 con 0x7f3a240ffc70 2024-09-10T11:25:49.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.966+0000 7f3a217fa700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.178:6808/280235677,v1:172.21.15.178:6809/280235677] conn(0x7f3a0c072210 0x7f3a0c074620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:49.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.966+0000 7f3a217fa700 1 -- 172.21.15.144:0/905523750 --> [v2:172.21.15.178:6808/280235677,v1:172.21.15.178:6809/280235677] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3a0c074cd0 con 0x7f3a0c072210 2024-09-10T11:25:49.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.966+0000 7f3a217fa700 1 -- 172.21.15.144:0/905523750 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_get_version_reply(handle=1 version=41) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a1408d190 con 0x7f3a240ffc70 2024-09-10T11:25:49.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.966+0000 7f3a28f11700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.178:6808/280235677,v1:172.21.15.178:6809/280235677] conn(0x7f3a0c072210 0x7f3a0c074620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:49.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.966+0000 7f3a23fff700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3a0c06c680 0x7f3a0c06eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a18005950 tx=0x7f3a180058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:49.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.967+0000 7f3a28f11700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.178:6808/280235677,v1:172.21.15.178:6809/280235677] conn(0x7f3a0c072210 0x7f3a0c074620 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:49.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.968+0000 7f3a217fa700 1 -- 172.21.15.144:0/905523750 <== osd.4 v2:172.21.15.178:6808/280235677 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3a0c074cd0 con 0x7f3a0c072210 2024-09-10T11:25:49.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.983+0000 7f3a2a974700 1 -- 172.21.15.144:0/905523750 --> [v2:172.21.15.178:6808/280235677,v1:172.21.15.178:6809/280235677] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3a2404ea50 con 0x7f3a0c072210 2024-09-10T11:25:49.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.984+0000 7f3a217fa700 1 -- 172.21.15.144:0/905523750 <== osd.4 v2:172.21.15.178:6808/280235677 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3a2404ea50 con 0x7f3a0c072210 2024-09-10T11:25:49.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.984+0000 7f3a2a974700 1 -- 172.21.15.144:0/905523750 >> [v2:172.21.15.178:6808/280235677,v1:172.21.15.178:6809/280235677] conn(0x7f3a0c072210 msgr2=0x7f3a0c074620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:49.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.984+0000 7f3a2a974700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.178:6808/280235677,v1:172.21.15.178:6809/280235677] conn(0x7f3a0c072210 0x7f3a0c074620 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.984+0000 7f3a2a974700 1 -- 172.21.15.144:0/905523750 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3a0c06c680 msgr2=0x7f3a0c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:49.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.984+0000 7f3a2a974700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3a0c06c680 0x7f3a0c06eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f3a18005950 tx=0x7f3a180058e0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.984+0000 7f3a2a974700 1 -- 172.21.15.144:0/905523750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a240ffc70 msgr2=0x7f3a24197110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:49.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.984+0000 7f3a2a974700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a240ffc70 0x7f3a24197110 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3a140092e0 tx=0x7f3a1400be20 comp rx=0 tx=0).stop 2024-09-10T11:25:49.986 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.984+0000 7f3a2a974700 1 -- 172.21.15.144:0/905523750 shutdown_connections 2024-09-10T11:25:49.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.984+0000 7f3a2a974700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.178:6808/280235677,v1:172.21.15.178:6809/280235677] conn(0x7f3a0c072210 0x7f3a0c074620 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.984+0000 7f3a2a974700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3a0c06c680 0x7f3a0c06eb30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.984+0000 7f3a2a974700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3a240ff2d0 0x7f3a24196bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.984+0000 7f3a2a974700 1 --2- 172.21.15.144:0/905523750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3a240ffc70 0x7f3a24197110 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:49.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.984+0000 7f3a2a974700 1 -- 172.21.15.144:0/905523750 >> 172.21.15.144:0/905523750 conn(0x7f3a240fabb0 msgr2=0x7f3a240fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:49.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.984+0000 7f3a2a974700 1 -- 172.21.15.144:0/905523750 shutdown_connections 2024-09-10T11:25:49.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:49.984+0000 7f3a2a974700 1 -- 172.21.15.144:0/905523750 wait complete. 2024-09-10T11:25:50.655 INFO:teuthology.orchestra.run.smithi144.stdout:171798691849 2024-09-10T11:25:50.655 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T11:25:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:50 smithi178 ceph-mon[33514]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:51.369 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:50 smithi144 ceph-mon[27942]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:51.390 INFO:teuthology.orchestra.run.smithi144.stdout:150323855373 2024-09-10T11:25:51.391 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T11:25:51.443 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:51.859 INFO:teuthology.orchestra.run.smithi144.stdout:38654705699 2024-09-10T11:25:51.859 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T11:25:51.896 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:51.897 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:53.021 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:52 smithi144 ceph-mon[27942]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:52 smithi178 ceph-mon[33514]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:54 smithi144 ceph-mon[27942]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:55.323 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:55.323 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:55.366 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:25:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:54 smithi178 ceph-mon[33514]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:56.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.024+0000 7f34737de700 1 -- 172.21.15.144:0/3074701919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f346c0ff290 msgr2=0x7f346c0ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:56.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.024+0000 7f34737de700 1 --2- 172.21.15.144:0/3074701919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f346c0ff290 0x7f346c0ff660 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f345c009910 tx=0x7f345c009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:56.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.024+0000 7f34737de700 1 -- 172.21.15.144:0/3074701919 shutdown_connections 2024-09-10T11:25:56.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.024+0000 7f34737de700 1 --2- 172.21.15.144:0/3074701919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f346c0ffba0 0x7f346c10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:56.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.024+0000 7f34737de700 1 --2- 172.21.15.144:0/3074701919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f346c0ff290 0x7f346c0ff660 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:56.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.024+0000 7f34737de700 1 -- 172.21.15.144:0/3074701919 >> 172.21.15.144:0/3074701919 conn(0x7f346c074850 msgr2=0x7f346c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:56.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.025+0000 7f34737de700 1 -- 172.21.15.144:0/3074701919 shutdown_connections 2024-09-10T11:25:56.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.025+0000 7f34737de700 1 -- 172.21.15.144:0/3074701919 wait complete. 2024-09-10T11:25:56.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.025+0000 7f34737de700 1 Processor -- start 2024-09-10T11:25:56.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.025+0000 7f34737de700 1 -- start start 2024-09-10T11:25:56.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.025+0000 7f34737de700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f346c0ff290 0x7f346c195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:56.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.025+0000 7f34737de700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f346c0ffba0 0x7f346c1962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:56.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.025+0000 7f34737de700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c196990 con 0x7f346c0ffba0 2024-09-10T11:25:56.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.025+0000 7f34737de700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f346c19a720 con 0x7f346c0ff290 2024-09-10T11:25:56.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.025+0000 7f3470d79700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f346c0ffba0 0x7f346c1962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:56.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.026+0000 7f3470d79700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f346c0ffba0 0x7f346c1962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54054/0 (socket says 172.21.15.144:54054) 2024-09-10T11:25:56.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.026+0000 7f3470d79700 1 -- 172.21.15.144:0/1869265771 learned_addr learned my addr 172.21.15.144:0/1869265771 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:56.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.026+0000 7f347157a700 1 --2- 172.21.15.144:0/1869265771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f346c0ff290 0x7f346c195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:56.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.026+0000 7f3470d79700 1 -- 172.21.15.144:0/1869265771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f346c0ff290 msgr2=0x7f346c195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:56.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.026+0000 7f3470d79700 1 --2- 172.21.15.144:0/1869265771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f346c0ff290 0x7f346c195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:56.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.026+0000 7f3470d79700 1 -- 172.21.15.144:0/1869265771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c0095f0 con 0x7f346c0ffba0 2024-09-10T11:25:56.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.026+0000 7f347157a700 1 --2- 172.21.15.144:0/1869265771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f346c0ff290 0x7f346c195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:25:56.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.026+0000 7f3470d79700 1 --2- 172.21.15.144:0/1869265771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f346c0ffba0 0x7f346c1962b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f346800d8d0 tx=0x7f346800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:56.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.026+0000 7f34627fc700 1 -- 172.21.15.144:0/1869265771 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3468009940 con 0x7f346c0ffba0 2024-09-10T11:25:56.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.026+0000 7f34737de700 1 -- 172.21.15.144:0/1869265771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f346c19aa00 con 0x7f346c0ffba0 2024-09-10T11:25:56.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.026+0000 7f34627fc700 1 -- 172.21.15.144:0/1869265771 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3468010460 con 0x7f346c0ffba0 2024-09-10T11:25:56.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.026+0000 7f34627fc700 1 -- 172.21.15.144:0/1869265771 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f346800f5f0 con 0x7f346c0ffba0 2024-09-10T11:25:56.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.026+0000 7f34737de700 1 -- 172.21.15.144:0/1869265771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f346c19af50 con 0x7f346c0ffba0 2024-09-10T11:25:56.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.027+0000 7f34737de700 1 -- 172.21.15.144:0/1869265771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f346c107d10 con 0x7f346c0ffba0 2024-09-10T11:25:56.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.028+0000 7f34627fc700 1 -- 172.21.15.144:0/1869265771 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f34680105d0 con 0x7f346c0ffba0 2024-09-10T11:25:56.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.028+0000 7f34627fc700 1 --2- 172.21.15.144:0/1869265771 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f345806c680 0x7f345806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:56.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.028+0000 7f34627fc700 1 -- 172.21.15.144:0/1869265771 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f346808b4a0 con 0x7f346c0ffba0 2024-09-10T11:25:56.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.028+0000 7f347157a700 1 --2- 172.21.15.144:0/1869265771 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f345806c680 0x7f345806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:56.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.028+0000 7f347157a700 1 --2- 172.21.15.144:0/1869265771 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f345806c680 0x7f345806eb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f345c005b40 tx=0x7f345c01a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:56.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.033+0000 7f34627fc700 1 -- 172.21.15.144:0/1869265771 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3468055f60 con 0x7f346c0ffba0 2024-09-10T11:25:56.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.228+0000 7f34737de700 1 -- 172.21.15.144:0/1869265771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f346c19b170 con 0x7f346c0ffba0 2024-09-10T11:25:56.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.228+0000 7f34627fc700 1 -- 172.21.15.144:0/1869265771 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3468059580 con 0x7f346c0ffba0 2024-09-10T11:25:56.230 INFO:teuthology.orchestra.run.smithi144.stdout:60129542174 2024-09-10T11:25:56.232 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.230+0000 7f34737de700 1 -- 172.21.15.144:0/1869265771 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f345806c680 msgr2=0x7f345806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:56.232 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.230+0000 7f34737de700 1 --2- 172.21.15.144:0/1869265771 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f345806c680 0x7f345806eb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f345c005b40 tx=0x7f345c01a040 comp rx=0 tx=0).stop 2024-09-10T11:25:56.232 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.230+0000 7f34737de700 1 -- 172.21.15.144:0/1869265771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f346c0ffba0 msgr2=0x7f346c1962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:56.232 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.230+0000 7f34737de700 1 --2- 172.21.15.144:0/1869265771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f346c0ffba0 0x7f346c1962b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f346800d8d0 tx=0x7f346800dc90 comp rx=0 tx=0).stop 2024-09-10T11:25:56.232 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.230+0000 7f34737de700 1 -- 172.21.15.144:0/1869265771 shutdown_connections 2024-09-10T11:25:56.232 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.230+0000 7f34737de700 1 --2- 172.21.15.144:0/1869265771 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f345806c680 0x7f345806eb30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:56.232 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.230+0000 7f34737de700 1 --2- 172.21.15.144:0/1869265771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f346c0ff290 0x7f346c195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:56.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.230+0000 7f34737de700 1 --2- 172.21.15.144:0/1869265771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f346c0ffba0 0x7f346c1962b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:56.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.230+0000 7f34737de700 1 -- 172.21.15.144:0/1869265771 >> 172.21.15.144:0/1869265771 conn(0x7f346c074850 msgr2=0x7f346c104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:56.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.231+0000 7f34737de700 1 -- 172.21.15.144:0/1869265771 shutdown_connections 2024-09-10T11:25:56.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:56.231+0000 7f34737de700 1 -- 172.21.15.144:0/1869265771 wait complete. 2024-09-10T11:25:56.849 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542172 got 60129542174 for osd.1 2024-09-10T11:25:56.849 DEBUG:teuthology.parallel:result is None 2024-09-10T11:25:57.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.056+0000 7fb344a7d700 1 -- 172.21.15.144:0/1172221126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb340068630 msgr2=0x7fb340068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:57.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.056+0000 7fb344a7d700 1 --2- 172.21.15.144:0/1172221126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb340068630 0x7fb340068a00 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb334009910 tx=0x7fb334009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:57.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.057+0000 7fb344a7d700 1 -- 172.21.15.144:0/1172221126 shutdown_connections 2024-09-10T11:25:57.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.057+0000 7fb344a7d700 1 --2- 172.21.15.144:0/1172221126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb340068fd0 0x7fb340104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:57.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.057+0000 7fb344a7d700 1 --2- 172.21.15.144:0/1172221126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb340068630 0x7fb340068a00 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:57.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.057+0000 7fb344a7d700 1 -- 172.21.15.144:0/1172221126 >> 172.21.15.144:0/1172221126 conn(0x7fb340074f50 msgr2=0x7fb340075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:57.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.057+0000 7fb344a7d700 1 -- 172.21.15.144:0/1172221126 shutdown_connections 2024-09-10T11:25:57.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.057+0000 7fb344a7d700 1 -- 172.21.15.144:0/1172221126 wait complete. 2024-09-10T11:25:57.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.057+0000 7fb344a7d700 1 Processor -- start 2024-09-10T11:25:57.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.057+0000 7fb344a7d700 1 -- start start 2024-09-10T11:25:57.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb344a7d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb340068630 0x7fb340197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:57.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb344a7d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb340068fd0 0x7fb340198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:57.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb344a7d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb340198c00 con 0x7fb340068fd0 2024-09-10T11:25:57.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb344a7d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb34019c990 con 0x7fb340068630 2024-09-10T11:25:57.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb33dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb340068fd0 0x7fb340198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:57.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb33dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb340068fd0 0x7fb340198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54062/0 (socket says 172.21.15.144:54062) 2024-09-10T11:25:57.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb33dd9b700 1 -- 172.21.15.144:0/35605565 learned_addr learned my addr 172.21.15.144:0/35605565 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:57.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb33e59c700 1 --2- 172.21.15.144:0/35605565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb340068630 0x7fb340197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:57.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb33dd9b700 1 -- 172.21.15.144:0/35605565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb340068630 msgr2=0x7fb340197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:57.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb33dd9b700 1 --2- 172.21.15.144:0/35605565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb340068630 0x7fb340197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:57.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb33dd9b700 1 -- 172.21.15.144:0/35605565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3340095f0 con 0x7fb340068fd0 2024-09-10T11:25:57.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb33dd9b700 1 --2- 172.21.15.144:0/35605565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb340068fd0 0x7fb340198520 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb33000b480 tx=0x7fb33000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:57.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb32f7fe700 1 -- 172.21.15.144:0/35605565 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb3300118c0 con 0x7fb340068fd0 2024-09-10T11:25:57.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb32f7fe700 1 -- 172.21.15.144:0/35605565 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb330010380 con 0x7fb340068fd0 2024-09-10T11:25:57.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb32f7fe700 1 -- 172.21.15.144:0/35605565 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb33000f4f0 con 0x7fb340068fd0 2024-09-10T11:25:57.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb344a7d700 1 -- 172.21.15.144:0/35605565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34019cc70 con 0x7fb340068fd0 2024-09-10T11:25:57.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.058+0000 7fb344a7d700 1 -- 172.21.15.144:0/35605565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb34019d1c0 con 0x7fb340068fd0 2024-09-10T11:25:57.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.059+0000 7fb344a7d700 1 -- 172.21.15.144:0/35605565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb340104570 con 0x7fb340068fd0 2024-09-10T11:25:57.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.060+0000 7fb32f7fe700 1 -- 172.21.15.144:0/35605565 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fb3300104f0 con 0x7fb340068fd0 2024-09-10T11:25:57.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.060+0000 7fb32f7fe700 1 --2- 172.21.15.144:0/35605565 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb32806c680 0x7fb32806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:57.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.060+0000 7fb32f7fe700 1 -- 172.21.15.144:0/35605565 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fb33008b0e0 con 0x7fb340068fd0 2024-09-10T11:25:57.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.060+0000 7fb33e59c700 1 --2- 172.21.15.144:0/35605565 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb32806c680 0x7fb32806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:57.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.061+0000 7fb33e59c700 1 --2- 172.21.15.144:0/35605565 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb32806c680 0x7fb32806eb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb33400b5c0 tx=0x7fb33401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:57.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.065+0000 7fb32f7fe700 1 -- 172.21.15.144:0/35605565 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb330055ba0 con 0x7fb340068fd0 2024-09-10T11:25:57.171 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:56 smithi144 ceph-mon[27942]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:57.172 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:56 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1869265771' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T11:25:57.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.261+0000 7fb344a7d700 1 -- 172.21.15.144:0/35605565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb34004ea50 con 0x7fb340068fd0 2024-09-10T11:25:57.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.262+0000 7fb32f7fe700 1 -- 172.21.15.144:0/35605565 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb3300591c0 con 0x7fb340068fd0 2024-09-10T11:25:57.263 INFO:teuthology.orchestra.run.smithi144.stdout:81604378649 2024-09-10T11:25:57.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.264+0000 7fb344a7d700 1 -- 172.21.15.144:0/35605565 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb32806c680 msgr2=0x7fb32806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:57.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.264+0000 7fb344a7d700 1 --2- 172.21.15.144:0/35605565 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb32806c680 0x7fb32806eb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb33400b5c0 tx=0x7fb33401a040 comp rx=0 tx=0).stop 2024-09-10T11:25:57.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.264+0000 7fb344a7d700 1 -- 172.21.15.144:0/35605565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb340068fd0 msgr2=0x7fb340198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:57.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.264+0000 7fb344a7d700 1 --2- 172.21.15.144:0/35605565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb340068fd0 0x7fb340198520 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb33000b480 tx=0x7fb33000b840 comp rx=0 tx=0).stop 2024-09-10T11:25:57.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.264+0000 7fb344a7d700 1 -- 172.21.15.144:0/35605565 shutdown_connections 2024-09-10T11:25:57.266 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.264+0000 7fb344a7d700 1 --2- 172.21.15.144:0/35605565 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb32806c680 0x7fb32806eb30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:57.266 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.264+0000 7fb344a7d700 1 --2- 172.21.15.144:0/35605565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb340068630 0x7fb340197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:57.266 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.264+0000 7fb344a7d700 1 --2- 172.21.15.144:0/35605565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb340068fd0 0x7fb340198520 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:57.266 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.264+0000 7fb344a7d700 1 -- 172.21.15.144:0/35605565 >> 172.21.15.144:0/35605565 conn(0x7fb340074f50 msgr2=0x7fb3400fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:57.266 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.264+0000 7fb344a7d700 1 -- 172.21.15.144:0/35605565 shutdown_connections 2024-09-10T11:25:57.266 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:57.264+0000 7fb344a7d700 1 -- 172.21.15.144:0/35605565 wait complete. 2024-09-10T11:25:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:56 smithi178 ceph-mon[33514]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:56 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/1869265771' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T11:25:58.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:57 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/35605565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T11:25:58.279 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378646 got 81604378649 for osd.2 2024-09-10T11:25:58.279 DEBUG:teuthology.parallel:result is None 2024-09-10T11:25:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:57 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/35605565' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T11:25:59.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.238+0000 7f6b569ce700 1 -- 172.21.15.144:0/647695696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b500ff2b0 msgr2=0x7f6b500ff680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:59.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.238+0000 7f6b569ce700 1 --2- 172.21.15.144:0/647695696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b500ff2b0 0x7f6b500ff680 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6b40009910 tx=0x7f6b40009c20 comp rx=0 tx=0).stop 2024-09-10T11:25:59.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.238+0000 7f6b569ce700 1 -- 172.21.15.144:0/647695696 shutdown_connections 2024-09-10T11:25:59.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.238+0000 7f6b569ce700 1 --2- 172.21.15.144:0/647695696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b500ffc50 0x7f6b50103c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:59.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.238+0000 7f6b569ce700 1 --2- 172.21.15.144:0/647695696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b500ff2b0 0x7f6b500ff680 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:59.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.238+0000 7f6b569ce700 1 -- 172.21.15.144:0/647695696 >> 172.21.15.144:0/647695696 conn(0x7f6b500fab50 msgr2=0x7f6b500fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:59.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.238+0000 7f6b569ce700 1 -- 172.21.15.144:0/647695696 shutdown_connections 2024-09-10T11:25:59.240 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:58 smithi144 ceph-mon[27942]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:59.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.238+0000 7f6b569ce700 1 -- 172.21.15.144:0/647695696 wait complete. 2024-09-10T11:25:59.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.239+0000 7f6b569ce700 1 Processor -- start 2024-09-10T11:25:59.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.239+0000 7f6b569ce700 1 -- start start 2024-09-10T11:25:59.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.239+0000 7f6b569ce700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b500ff2b0 0x7f6b50195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:59.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.239+0000 7f6b569ce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b500ffc50 0x7f6b501962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.239+0000 7f6b569ce700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b501969d0 con 0x7f6b500ffc50 2024-09-10T11:25:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.239+0000 7f6b569ce700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b5019a760 con 0x7f6b500ff2b0 2024-09-10T11:25:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.239+0000 7f6b4f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b500ffc50 0x7f6b501962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.239+0000 7f6b4f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b500ffc50 0x7f6b501962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54074/0 (socket says 172.21.15.144:54074) 2024-09-10T11:25:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.239+0000 7f6b4f7fe700 1 -- 172.21.15.144:0/3851727384 learned_addr learned my addr 172.21.15.144:0/3851727384 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.239+0000 7f6b4f7fe700 1 -- 172.21.15.144:0/3851727384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b500ff2b0 msgr2=0x7f6b50195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.239+0000 7f6b4f7fe700 1 --2- 172.21.15.144:0/3851727384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b500ff2b0 0x7f6b50195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.239+0000 7f6b4f7fe700 1 -- 172.21.15.144:0/3851727384 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b400095f0 con 0x7f6b500ffc50 2024-09-10T11:25:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.240+0000 7f6b4f7fe700 1 --2- 172.21.15.144:0/3851727384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b500ffc50 0x7f6b501962f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400b480 tx=0x7f6b4400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.240+0000 7f6b4d7fa700 1 -- 172.21.15.144:0/3851727384 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b440118c0 con 0x7f6b500ffc50 2024-09-10T11:25:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.240+0000 7f6b4d7fa700 1 -- 172.21.15.144:0/3851727384 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6b44010380 con 0x7f6b500ffc50 2024-09-10T11:25:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.240+0000 7f6b569ce700 1 -- 172.21.15.144:0/3851727384 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b5019aa40 con 0x7f6b500ffc50 2024-09-10T11:25:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.240+0000 7f6b4d7fa700 1 -- 172.21.15.144:0/3851727384 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b4400f4f0 con 0x7f6b500ffc50 2024-09-10T11:25:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.240+0000 7f6b569ce700 1 -- 172.21.15.144:0/3851727384 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b5019af90 con 0x7f6b500ffc50 2024-09-10T11:25:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.241+0000 7f6b569ce700 1 -- 172.21.15.144:0/3851727384 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b5004ea50 con 0x7f6b500ffc50 2024-09-10T11:25:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.241+0000 7f6b4d7fa700 1 -- 172.21.15.144:0/3851727384 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f6b440104f0 con 0x7f6b500ffc50 2024-09-10T11:25:59.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.242+0000 7f6b4d7fa700 1 --2- 172.21.15.144:0/3851727384 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6b3806c680 0x7f6b3806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:59.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.242+0000 7f6b4d7fa700 1 -- 172.21.15.144:0/3851727384 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f6b4408b020 con 0x7f6b500ffc50 2024-09-10T11:25:59.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.242+0000 7f6b4ffff700 1 --2- 172.21.15.144:0/3851727384 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6b3806c680 0x7f6b3806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:59.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.242+0000 7f6b4ffff700 1 --2- 172.21.15.144:0/3851727384 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6b3806c680 0x7f6b3806eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b40006010 tx=0x7f6b4000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:59.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.246+0000 7f6b4d7fa700 1 -- 172.21.15.144:0/3851727384 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6b44055b60 con 0x7f6b500ffc50 2024-09-10T11:25:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:58 smithi178 ceph-mon[33514]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:25:59.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.430+0000 7fbc6c6c9700 1 -- 172.21.15.144:0/488990909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc640ff320 msgr2=0x7fbc640ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:59.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.430+0000 7fbc6c6c9700 1 --2- 172.21.15.144:0/488990909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc640ff320 0x7fbc640ff6f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009940 tx=0x7fbc54009c50 comp rx=0 tx=0).stop 2024-09-10T11:25:59.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.430+0000 7fbc6c6c9700 1 -- 172.21.15.144:0/488990909 shutdown_connections 2024-09-10T11:25:59.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.430+0000 7fbc6c6c9700 1 --2- 172.21.15.144:0/488990909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc640ffc30 0x7fbc6410a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:59.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.430+0000 7fbc6c6c9700 1 --2- 172.21.15.144:0/488990909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc640ff320 0x7fbc640ff6f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:59.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.430+0000 7fbc6c6c9700 1 -- 172.21.15.144:0/488990909 >> 172.21.15.144:0/488990909 conn(0x7fbc640748b0 msgr2=0x7fbc64074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:59.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.430+0000 7fbc6c6c9700 1 -- 172.21.15.144:0/488990909 shutdown_connections 2024-09-10T11:25:59.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.430+0000 7fbc6c6c9700 1 -- 172.21.15.144:0/488990909 wait complete. 2024-09-10T11:25:59.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.431+0000 7fbc6c6c9700 1 Processor -- start 2024-09-10T11:25:59.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.431+0000 7fbc6c6c9700 1 -- start start 2024-09-10T11:25:59.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.431+0000 7fbc6c6c9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc640ff320 0x7fbc6419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:59.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.431+0000 7fbc6c6c9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc640ffc30 0x7fbc6419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:59.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.431+0000 7fbc6c6c9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6419c560 con 0x7fbc640ff320 2024-09-10T11:25:59.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.431+0000 7fbc6c6c9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc64195aa0 con 0x7fbc640ffc30 2024-09-10T11:25:59.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.431+0000 7fbc6a465700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc640ff320 0x7fbc6419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:59.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.431+0000 7fbc6a465700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc640ff320 0x7fbc6419ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54096/0 (socket says 172.21.15.144:54096) 2024-09-10T11:25:59.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.431+0000 7fbc6a465700 1 -- 172.21.15.144:0/1266546882 learned_addr learned my addr 172.21.15.144:0/1266546882 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:25:59.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.431+0000 7fbc69c64700 1 --2- 172.21.15.144:0/1266546882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc640ffc30 0x7fbc6419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:59.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.431+0000 7fbc6a465700 1 -- 172.21.15.144:0/1266546882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc640ffc30 msgr2=0x7fbc6419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:59.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.431+0000 7fbc6a465700 1 --2- 172.21.15.144:0/1266546882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc640ffc30 0x7fbc6419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:59.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.431+0000 7fbc6a465700 1 -- 172.21.15.144:0/1266546882 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc540095f0 con 0x7fbc640ff320 2024-09-10T11:25:59.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.432+0000 7fbc6a465700 1 --2- 172.21.15.144:0/1266546882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc640ff320 0x7fbc6419ba20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbc540055f0 tx=0x7fbc54005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:59.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.432+0000 7fbc5b7fe700 1 -- 172.21.15.144:0/1266546882 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc5401d070 con 0x7fbc640ff320 2024-09-10T11:25:59.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.432+0000 7fbc5b7fe700 1 -- 172.21.15.144:0/1266546882 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbc5400f460 con 0x7fbc640ff320 2024-09-10T11:25:59.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.432+0000 7fbc6c6c9700 1 -- 172.21.15.144:0/1266546882 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc64195d80 con 0x7fbc640ff320 2024-09-10T11:25:59.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.432+0000 7fbc5b7fe700 1 -- 172.21.15.144:0/1266546882 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc54021640 con 0x7fbc640ff320 2024-09-10T11:25:59.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.432+0000 7fbc6c6c9700 1 -- 172.21.15.144:0/1266546882 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc641962d0 con 0x7fbc640ff320 2024-09-10T11:25:59.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.433+0000 7fbc6c6c9700 1 -- 172.21.15.144:0/1266546882 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc64107da0 con 0x7fbc640ff320 2024-09-10T11:25:59.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.433+0000 7fbc5b7fe700 1 -- 172.21.15.144:0/1266546882 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fbc5400fab0 con 0x7fbc640ff320 2024-09-10T11:25:59.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.434+0000 7fbc5b7fe700 1 --2- 172.21.15.144:0/1266546882 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbc5006c680 0x7fbc5006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:25:59.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.434+0000 7fbc69c64700 1 --2- 172.21.15.144:0/1266546882 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbc5006c680 0x7fbc5006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:25:59.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.434+0000 7fbc5b7fe700 1 -- 172.21.15.144:0/1266546882 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fbc5408c770 con 0x7fbc640ff320 2024-09-10T11:25:59.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.434+0000 7fbc69c64700 1 --2- 172.21.15.144:0/1266546882 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbc5006c680 0x7fbc5006eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc640ff0e0 tx=0x7fbc6000a380 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:25:59.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.438+0000 7fbc5b7fe700 1 -- 172.21.15.144:0/1266546882 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbc5405acc0 con 0x7fbc640ff320 2024-09-10T11:25:59.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.442+0000 7f6b569ce700 1 -- 172.21.15.144:0/3851727384 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6b50197110 con 0x7f6b500ffc50 2024-09-10T11:25:59.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.442+0000 7f6b4d7fa700 1 -- 172.21.15.144:0/3851727384 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6b44059180 con 0x7f6b500ffc50 2024-09-10T11:25:59.444 INFO:teuthology.orchestra.run.smithi144.stdout:115964117013 2024-09-10T11:25:59.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.444+0000 7f6b569ce700 1 -- 172.21.15.144:0/3851727384 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6b3806c680 msgr2=0x7f6b3806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:59.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.444+0000 7f6b569ce700 1 --2- 172.21.15.144:0/3851727384 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6b3806c680 0x7f6b3806eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6b40006010 tx=0x7f6b4000b540 comp rx=0 tx=0).stop 2024-09-10T11:25:59.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.444+0000 7f6b569ce700 1 -- 172.21.15.144:0/3851727384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b500ffc50 msgr2=0x7f6b501962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:59.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.444+0000 7f6b569ce700 1 --2- 172.21.15.144:0/3851727384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b500ffc50 0x7f6b501962f0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6b4400b480 tx=0x7f6b4400b840 comp rx=0 tx=0).stop 2024-09-10T11:25:59.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.445+0000 7f6b569ce700 1 -- 172.21.15.144:0/3851727384 shutdown_connections 2024-09-10T11:25:59.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.445+0000 7f6b569ce700 1 --2- 172.21.15.144:0/3851727384 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6b3806c680 0x7f6b3806eb30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:59.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.445+0000 7f6b569ce700 1 --2- 172.21.15.144:0/3851727384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b500ff2b0 0x7f6b50195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:59.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.445+0000 7f6b569ce700 1 --2- 172.21.15.144:0/3851727384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b500ffc50 0x7f6b501962f0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:59.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.445+0000 7f6b569ce700 1 -- 172.21.15.144:0/3851727384 >> 172.21.15.144:0/3851727384 conn(0x7f6b500fab50 msgr2=0x7f6b50068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:59.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.445+0000 7f6b569ce700 1 -- 172.21.15.144:0/3851727384 shutdown_connections 2024-09-10T11:25:59.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.445+0000 7f6b569ce700 1 -- 172.21.15.144:0/3851727384 wait complete. 2024-09-10T11:25:59.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.634+0000 7fbc6c6c9700 1 -- 172.21.15.144:0/1266546882 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbc6404ea50 con 0x7fbc640ff320 2024-09-10T11:25:59.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.634+0000 7fbc5b7fe700 1 -- 172.21.15.144:0/1266546882 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fbc5405a850 con 0x7fbc640ff320 2024-09-10T11:25:59.636 INFO:teuthology.orchestra.run.smithi144.stdout:150323855374 2024-09-10T11:25:59.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.636+0000 7fbc6c6c9700 1 -- 172.21.15.144:0/1266546882 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbc5006c680 msgr2=0x7fbc5006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:59.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.636+0000 7fbc6c6c9700 1 --2- 172.21.15.144:0/1266546882 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbc5006c680 0x7fbc5006eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbc640ff0e0 tx=0x7fbc6000a380 comp rx=0 tx=0).stop 2024-09-10T11:25:59.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.636+0000 7fbc6c6c9700 1 -- 172.21.15.144:0/1266546882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc640ff320 msgr2=0x7fbc6419ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:25:59.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.636+0000 7fbc6c6c9700 1 --2- 172.21.15.144:0/1266546882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc640ff320 0x7fbc6419ba20 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fbc540055f0 tx=0x7fbc54005620 comp rx=0 tx=0).stop 2024-09-10T11:25:59.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.636+0000 7fbc6c6c9700 1 -- 172.21.15.144:0/1266546882 shutdown_connections 2024-09-10T11:25:59.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.636+0000 7fbc6c6c9700 1 --2- 172.21.15.144:0/1266546882 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fbc5006c680 0x7fbc5006eb30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:59.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.636+0000 7fbc6c6c9700 1 --2- 172.21.15.144:0/1266546882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc640ff320 0x7fbc6419ba20 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:59.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.636+0000 7fbc6c6c9700 1 --2- 172.21.15.144:0/1266546882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc640ffc30 0x7fbc6419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:25:59.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.636+0000 7fbc6c6c9700 1 -- 172.21.15.144:0/1266546882 >> 172.21.15.144:0/1266546882 conn(0x7fbc640748b0 msgr2=0x7fbc64104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:25:59.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.637+0000 7fbc6c6c9700 1 -- 172.21.15.144:0/1266546882 shutdown_connections 2024-09-10T11:25:59.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:25:59.637+0000 7fbc6c6c9700 1 -- 172.21.15.144:0/1266546882 wait complete. 2024-09-10T11:26:00.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:59 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/3851727384' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T11:26:00.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:25:59 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1266546882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T11:26:00.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:59 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/3851727384' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T11:26:00.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:25:59 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/1266546882' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T11:26:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:00 smithi144 ceph-mon[27942]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:00 smithi178 ceph-mon[33514]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:01.604 INFO:tasks.cephadm.ceph_manager.ceph:need seq 150323855373 got 150323855374 for osd.4 2024-09-10T11:26:01.604 DEBUG:teuthology.parallel:result is None 2024-09-10T11:26:02.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.442+0000 7f8172c5d700 1 -- 172.21.15.144:0/1174090161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816c100d70 msgr2=0x7f816c104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:02.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.442+0000 7f8172c5d700 1 --2- 172.21.15.144:0/1174090161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816c100d70 0x7f816c104d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f815c009910 tx=0x7f815c009c20 comp rx=0 tx=0).stop 2024-09-10T11:26:02.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.442+0000 7f8172c5d700 1 -- 172.21.15.144:0/1174090161 shutdown_connections 2024-09-10T11:26:02.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.442+0000 7f8172c5d700 1 --2- 172.21.15.144:0/1174090161 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816c100d70 0x7f816c104d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:02.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.442+0000 7f8172c5d700 1 --2- 172.21.15.144:0/1174090161 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f816c1003d0 0x7f816c1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:02.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.442+0000 7f8172c5d700 1 -- 172.21.15.144:0/1174090161 >> 172.21.15.144:0/1174090161 conn(0x7f816c0fbc90 msgr2=0x7f816c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:02.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.443+0000 7f8172c5d700 1 -- 172.21.15.144:0/1174090161 shutdown_connections 2024-09-10T11:26:02.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.443+0000 7f8172c5d700 1 -- 172.21.15.144:0/1174090161 wait complete. 2024-09-10T11:26:02.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.443+0000 7f8172c5d700 1 Processor -- start 2024-09-10T11:26:02.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.443+0000 7f8172c5d700 1 -- start start 2024-09-10T11:26:02.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.443+0000 7f8172c5d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f816c1003d0 0x7f816c195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:02.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.443+0000 7f8172c5d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816c100d70 0x7f816c196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:02.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.443+0000 7f8172c5d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c196a00 con 0x7f816c100d70 2024-09-10T11:26:02.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.443+0000 7f8172c5d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816c19a790 con 0x7f816c1003d0 2024-09-10T11:26:02.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.443+0000 7f816bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816c100d70 0x7f816c196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:02.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.443+0000 7f816bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816c100d70 0x7f816c196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51828/0 (socket says 172.21.15.144:51828) 2024-09-10T11:26:02.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.443+0000 7f816bfff700 1 -- 172.21.15.144:0/4188618059 learned_addr learned my addr 172.21.15.144:0/4188618059 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:02.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.444+0000 7f81709f9700 1 --2- 172.21.15.144:0/4188618059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f816c1003d0 0x7f816c195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:02.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.444+0000 7f816bfff700 1 -- 172.21.15.144:0/4188618059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f816c1003d0 msgr2=0x7f816c195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:02.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.444+0000 7f816bfff700 1 --2- 172.21.15.144:0/4188618059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f816c1003d0 0x7f816c195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:02.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.444+0000 7f816bfff700 1 -- 172.21.15.144:0/4188618059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c0095f0 con 0x7f816c100d70 2024-09-10T11:26:02.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.444+0000 7f816bfff700 1 --2- 172.21.15.144:0/4188618059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816c100d70 0x7f816c196320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f815c0092e0 tx=0x7f815c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:02.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.444+0000 7f8169ffb700 1 -- 172.21.15.144:0/4188618059 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f815c01d070 con 0x7f816c100d70 2024-09-10T11:26:02.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.444+0000 7f8169ffb700 1 -- 172.21.15.144:0/4188618059 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f815c004fa0 con 0x7f816c100d70 2024-09-10T11:26:02.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.444+0000 7f8169ffb700 1 -- 172.21.15.144:0/4188618059 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f815c00f690 con 0x7f816c100d70 2024-09-10T11:26:02.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.444+0000 7f8172c5d700 1 -- 172.21.15.144:0/4188618059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816c19aa10 con 0x7f816c100d70 2024-09-10T11:26:02.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.444+0000 7f8172c5d700 1 -- 172.21.15.144:0/4188618059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816c19af00 con 0x7f816c100d70 2024-09-10T11:26:02.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.445+0000 7f8172c5d700 1 -- 172.21.15.144:0/4188618059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816c108690 con 0x7f816c100d70 2024-09-10T11:26:02.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.446+0000 7f8169ffb700 1 -- 172.21.15.144:0/4188618059 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f815c022470 con 0x7f816c100d70 2024-09-10T11:26:02.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.446+0000 7f8169ffb700 1 --2- 172.21.15.144:0/4188618059 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f815806c680 0x7f815806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:02.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.446+0000 7f8169ffb700 1 -- 172.21.15.144:0/4188618059 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f815c08cc30 con 0x7f816c100d70 2024-09-10T11:26:02.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.446+0000 7f81709f9700 1 --2- 172.21.15.144:0/4188618059 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f815806c680 0x7f815806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:02.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.447+0000 7f81709f9700 1 --2- 172.21.15.144:0/4188618059 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f815806c680 0x7f815806eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f816400a400 tx=0x7f816400d040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:02.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.451+0000 7f8169ffb700 1 -- 172.21.15.144:0/4188618059 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f815c0576f0 con 0x7f816c100d70 2024-09-10T11:26:02.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.646+0000 7f8172c5d700 1 -- 172.21.15.144:0/4188618059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f816c19b2a0 con 0x7f816c100d70 2024-09-10T11:26:02.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.646+0000 7f8169ffb700 1 -- 172.21.15.144:0/4188618059 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f815c05ad10 con 0x7f816c100d70 2024-09-10T11:26:02.648 INFO:teuthology.orchestra.run.smithi144.stdout:38654705701 2024-09-10T11:26:02.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.648+0000 7f8172c5d700 1 -- 172.21.15.144:0/4188618059 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f815806c680 msgr2=0x7f815806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:02.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.648+0000 7f8172c5d700 1 --2- 172.21.15.144:0/4188618059 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f815806c680 0x7f815806eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f816400a400 tx=0x7f816400d040 comp rx=0 tx=0).stop 2024-09-10T11:26:02.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.648+0000 7f8172c5d700 1 -- 172.21.15.144:0/4188618059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816c100d70 msgr2=0x7f816c196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:02.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.648+0000 7f8172c5d700 1 --2- 172.21.15.144:0/4188618059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816c100d70 0x7f816c196320 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f815c0092e0 tx=0x7f815c00bd70 comp rx=0 tx=0).stop 2024-09-10T11:26:02.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.648+0000 7f8172c5d700 1 -- 172.21.15.144:0/4188618059 shutdown_connections 2024-09-10T11:26:02.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.648+0000 7f8172c5d700 1 --2- 172.21.15.144:0/4188618059 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f815806c680 0x7f815806eb30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:02.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.648+0000 7f8172c5d700 1 --2- 172.21.15.144:0/4188618059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f816c1003d0 0x7f816c195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:02.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.648+0000 7f8172c5d700 1 --2- 172.21.15.144:0/4188618059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f816c100d70 0x7f816c196320 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:02.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.648+0000 7f8172c5d700 1 -- 172.21.15.144:0/4188618059 >> 172.21.15.144:0/4188618059 conn(0x7f816c0fbc90 msgr2=0x7f816c0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:02.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.649+0000 7f8172c5d700 1 -- 172.21.15.144:0/4188618059 shutdown_connections 2024-09-10T11:26:02.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:02.649+0000 7f8172c5d700 1 -- 172.21.15.144:0/4188618059 wait complete. 2024-09-10T11:26:02.846 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117010 got 115964117013 for osd.3 2024-09-10T11:26:02.847 DEBUG:teuthology.parallel:result is None 2024-09-10T11:26:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:02 smithi178 ceph-mon[33514]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:02 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/4188618059' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T11:26:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:26:03.429 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:02 smithi144 ceph-mon[27942]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:03.430 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:02 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/4188618059' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T11:26:03.430 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:26:03.456 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705701 for osd.0 2024-09-10T11:26:03.456 DEBUG:teuthology.parallel:result is None 2024-09-10T11:26:04.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.561+0000 7fc70a95e700 1 -- 172.21.15.144:0/224200645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7040ff290 msgr2=0x7fc7040ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:04.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.561+0000 7fc70a95e700 1 --2- 172.21.15.144:0/224200645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7040ff290 0x7fc7040ff660 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc6f4009940 tx=0x7fc6f4009c50 comp rx=0 tx=0).stop 2024-09-10T11:26:04.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.561+0000 7fc70a95e700 1 -- 172.21.15.144:0/224200645 shutdown_connections 2024-09-10T11:26:04.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.561+0000 7fc70a95e700 1 --2- 172.21.15.144:0/224200645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7040ffba0 0x7fc70410a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:04.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.561+0000 7fc70a95e700 1 --2- 172.21.15.144:0/224200645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7040ff290 0x7fc7040ff660 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:04.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.561+0000 7fc70a95e700 1 -- 172.21.15.144:0/224200645 >> 172.21.15.144:0/224200645 conn(0x7fc704074850 msgr2=0x7fc704074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:04.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.561+0000 7fc70a95e700 1 -- 172.21.15.144:0/224200645 shutdown_connections 2024-09-10T11:26:04.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.561+0000 7fc70a95e700 1 -- 172.21.15.144:0/224200645 wait complete. 2024-09-10T11:26:04.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.562+0000 7fc70a95e700 1 Processor -- start 2024-09-10T11:26:04.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.562+0000 7fc70a95e700 1 -- start start 2024-09-10T11:26:04.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.562+0000 7fc70a95e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7040ff290 0x7fc704195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:04.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.562+0000 7fc70a95e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7040ffba0 0x7fc7041963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:04.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.562+0000 7fc70a95e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc704196a80 con 0x7fc7040ff290 2024-09-10T11:26:04.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.562+0000 7fc70a95e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc70419a810 con 0x7fc7040ffba0 2024-09-10T11:26:04.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.562+0000 7fc703fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7040ff290 0x7fc704195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:04.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.562+0000 7fc703fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7040ff290 0x7fc704195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51846/0 (socket says 172.21.15.144:51846) 2024-09-10T11:26:04.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.562+0000 7fc703fff700 1 -- 172.21.15.144:0/3819707856 learned_addr learned my addr 172.21.15.144:0/3819707856 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:04.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.562+0000 7fc6fbfff700 1 --2- 172.21.15.144:0/3819707856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7040ffba0 0x7fc7041963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:04.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.562+0000 7fc703fff700 1 -- 172.21.15.144:0/3819707856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7040ffba0 msgr2=0x7fc7041963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:04.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.562+0000 7fc703fff700 1 --2- 172.21.15.144:0/3819707856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7040ffba0 0x7fc7041963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:04.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.562+0000 7fc703fff700 1 -- 172.21.15.144:0/3819707856 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f40095f0 con 0x7fc7040ff290 2024-09-10T11:26:04.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.563+0000 7fc703fff700 1 --2- 172.21.15.144:0/3819707856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7040ff290 0x7fc704195e60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40055f0 tx=0x7fc6f40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:04.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.563+0000 7fc701ffb700 1 -- 172.21.15.144:0/3819707856 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6f401d070 con 0x7fc7040ff290 2024-09-10T11:26:04.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.563+0000 7fc70a95e700 1 -- 172.21.15.144:0/3819707856 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc70419aa90 con 0x7fc7040ff290 2024-09-10T11:26:04.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.563+0000 7fc701ffb700 1 -- 172.21.15.144:0/3819707856 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc6f4003ba0 con 0x7fc7040ff290 2024-09-10T11:26:04.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.563+0000 7fc701ffb700 1 -- 172.21.15.144:0/3819707856 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6f4021640 con 0x7fc7040ff290 2024-09-10T11:26:04.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.563+0000 7fc70a95e700 1 -- 172.21.15.144:0/3819707856 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc70419af80 con 0x7fc7040ff290 2024-09-10T11:26:04.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.564+0000 7fc70a95e700 1 -- 172.21.15.144:0/3819707856 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc704107d10 con 0x7fc7040ff290 2024-09-10T11:26:04.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.564+0000 7fc701ffb700 1 -- 172.21.15.144:0/3819707856 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fc6f4003d10 con 0x7fc7040ff290 2024-09-10T11:26:04.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.565+0000 7fc701ffb700 1 --2- 172.21.15.144:0/3819707856 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc6f006c680 0x7fc6f006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:04.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.565+0000 7fc701ffb700 1 -- 172.21.15.144:0/3819707856 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fc6f408c8f0 con 0x7fc7040ff290 2024-09-10T11:26:04.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.565+0000 7fc6fbfff700 1 --2- 172.21.15.144:0/3819707856 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc6f006c680 0x7fc6f006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:04.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.565+0000 7fc6fbfff700 1 --2- 172.21.15.144:0/3819707856 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc6f006c680 0x7fc6f006eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec006fd0 tx=0x7fc6ec008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:04.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.569+0000 7fc701ffb700 1 -- 172.21.15.144:0/3819707856 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc6f40573b0 con 0x7fc7040ff290 2024-09-10T11:26:04.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.762+0000 7fc70a95e700 1 -- 172.21.15.144:0/3819707856 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fc70404ea50 con 0x7fc7040ff290 2024-09-10T11:26:04.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.763+0000 7fc701ffb700 1 -- 172.21.15.144:0/3819707856 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fc6f405a9d0 con 0x7fc7040ff290 2024-09-10T11:26:04.764 INFO:teuthology.orchestra.run.smithi144.stdout:171798691852 2024-09-10T11:26:04.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.765+0000 7fc70a95e700 1 -- 172.21.15.144:0/3819707856 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc6f006c680 msgr2=0x7fc6f006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:04.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.765+0000 7fc70a95e700 1 --2- 172.21.15.144:0/3819707856 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc6f006c680 0x7fc6f006eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc6ec006fd0 tx=0x7fc6ec008040 comp rx=0 tx=0).stop 2024-09-10T11:26:04.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.765+0000 7fc70a95e700 1 -- 172.21.15.144:0/3819707856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7040ff290 msgr2=0x7fc704195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:04.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.765+0000 7fc70a95e700 1 --2- 172.21.15.144:0/3819707856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7040ff290 0x7fc704195e60 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc6f40055f0 tx=0x7fc6f40056d0 comp rx=0 tx=0).stop 2024-09-10T11:26:04.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.765+0000 7fc70a95e700 1 -- 172.21.15.144:0/3819707856 shutdown_connections 2024-09-10T11:26:04.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.765+0000 7fc70a95e700 1 --2- 172.21.15.144:0/3819707856 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc6f006c680 0x7fc6f006eb30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:04.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.765+0000 7fc70a95e700 1 --2- 172.21.15.144:0/3819707856 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7040ff290 0x7fc704195e60 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:04.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.765+0000 7fc70a95e700 1 --2- 172.21.15.144:0/3819707856 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7040ffba0 0x7fc7041963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:04.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.765+0000 7fc70a95e700 1 -- 172.21.15.144:0/3819707856 >> 172.21.15.144:0/3819707856 conn(0x7fc704074850 msgr2=0x7fc704104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:04.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.765+0000 7fc70a95e700 1 -- 172.21.15.144:0/3819707856 shutdown_connections 2024-09-10T11:26:04.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:04.765+0000 7fc70a95e700 1 -- 172.21.15.144:0/3819707856 wait complete. 2024-09-10T11:26:05.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:04 smithi144 ceph-mon[27942]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:05.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:04 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/3819707856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T11:26:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:04 smithi178 ceph-mon[33514]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:04 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/3819707856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T11:26:05.382 INFO:tasks.cephadm.ceph_manager.ceph:need seq 171798691849 got 171798691852 for osd.5 2024-09-10T11:26:05.382 DEBUG:teuthology.parallel:result is None 2024-09-10T11:26:05.382 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T11:26:05.383 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T11:26:05.585 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:07.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.133+0000 7fb834c76700 1 -- 172.21.15.144:0/2619554079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8300ff2c0 msgr2=0x7fb8300ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:07.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.133+0000 7fb834c76700 1 --2- 172.21.15.144:0/2619554079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8300ff2c0 0x7fb8300ff690 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb82800b0a0 tx=0x7fb82800b3b0 comp rx=0 tx=0).stop 2024-09-10T11:26:07.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.133+0000 7fb834c76700 1 -- 172.21.15.144:0/2619554079 shutdown_connections 2024-09-10T11:26:07.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.133+0000 7fb834c76700 1 --2- 172.21.15.144:0/2619554079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8300ffc60 0x7fb830103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:07.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.133+0000 7fb834c76700 1 --2- 172.21.15.144:0/2619554079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8300ff2c0 0x7fb8300ff690 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:07.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.133+0000 7fb834c76700 1 -- 172.21.15.144:0/2619554079 >> 172.21.15.144:0/2619554079 conn(0x7fb8300fab80 msgr2=0x7fb8300fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:07.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.133+0000 7fb834c76700 1 -- 172.21.15.144:0/2619554079 shutdown_connections 2024-09-10T11:26:07.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.133+0000 7fb834c76700 1 -- 172.21.15.144:0/2619554079 wait complete. 2024-09-10T11:26:07.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.134+0000 7fb834c76700 1 Processor -- start 2024-09-10T11:26:07.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.134+0000 7fb834c76700 1 -- start start 2024-09-10T11:26:07.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.134+0000 7fb834c76700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8300ff2c0 0x7fb830197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:07.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.134+0000 7fb834c76700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8300ffc60 0x7fb8301984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:07.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.134+0000 7fb834c76700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb830198b80 con 0x7fb8300ffc60 2024-09-10T11:26:07.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.134+0000 7fb834c76700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb83019c910 con 0x7fb8300ff2c0 2024-09-10T11:26:07.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.134+0000 7fb82dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8300ffc60 0x7fb8301984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:07.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.134+0000 7fb82dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8300ffc60 0x7fb8301984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51870/0 (socket says 172.21.15.144:51870) 2024-09-10T11:26:07.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.134+0000 7fb82dd9b700 1 -- 172.21.15.144:0/2918210985 learned_addr learned my addr 172.21.15.144:0/2918210985 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:07.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.134+0000 7fb82e59c700 1 --2- 172.21.15.144:0/2918210985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8300ff2c0 0x7fb830197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:07.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.134+0000 7fb82dd9b700 1 -- 172.21.15.144:0/2918210985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8300ff2c0 msgr2=0x7fb830197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:07.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.134+0000 7fb82dd9b700 1 --2- 172.21.15.144:0/2918210985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8300ff2c0 0x7fb830197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:07.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.134+0000 7fb82dd9b700 1 -- 172.21.15.144:0/2918210985 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb828009d00 con 0x7fb8300ffc60 2024-09-10T11:26:07.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.134+0000 7fb82e59c700 1 --2- 172.21.15.144:0/2918210985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8300ff2c0 0x7fb830197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:26:07.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.135+0000 7fb82dd9b700 1 --2- 172.21.15.144:0/2918210985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8300ffc60 0x7fb8301984a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb81800c8f0 tx=0x7fb81800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:07.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.135+0000 7fb8277fe700 1 -- 172.21.15.144:0/2918210985 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb818007a50 con 0x7fb8300ffc60 2024-09-10T11:26:07.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.135+0000 7fb834c76700 1 -- 172.21.15.144:0/2918210985 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83019cbf0 con 0x7fb8300ffc60 2024-09-10T11:26:07.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.135+0000 7fb8277fe700 1 -- 172.21.15.144:0/2918210985 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb818007bb0 con 0x7fb8300ffc60 2024-09-10T11:26:07.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.135+0000 7fb8277fe700 1 -- 172.21.15.144:0/2918210985 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb818018630 con 0x7fb8300ffc60 2024-09-10T11:26:07.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.135+0000 7fb834c76700 1 -- 172.21.15.144:0/2918210985 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83019d140 con 0x7fb8300ffc60 2024-09-10T11:26:07.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.136+0000 7fb834c76700 1 -- 172.21.15.144:0/2918210985 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83004ea50 con 0x7fb8300ffc60 2024-09-10T11:26:07.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.136+0000 7fb8277fe700 1 -- 172.21.15.144:0/2918210985 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fb818007d20 con 0x7fb8300ffc60 2024-09-10T11:26:07.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.137+0000 7fb8277fe700 1 --2- 172.21.15.144:0/2918210985 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb81c06c680 0x7fb81c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:07.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.137+0000 7fb82e59c700 1 --2- 172.21.15.144:0/2918210985 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb81c06c680 0x7fb81c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:07.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.137+0000 7fb8277fe700 1 -- 172.21.15.144:0/2918210985 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fb81808b460 con 0x7fb8300ffc60 2024-09-10T11:26:07.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.137+0000 7fb82e59c700 1 --2- 172.21.15.144:0/2918210985 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb81c06c680 0x7fb81c06eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb828009480 tx=0x7fb828006000 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:07.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.141+0000 7fb8277fe700 1 -- 172.21.15.144:0/2918210985 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb818055f20 con 0x7fb8300ffc60 2024-09-10T11:26:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:06 smithi144 ceph-mon[27942]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:07.333 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.330+0000 7fb834c76700 1 -- 172.21.15.144:0/2918210985 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb83019d680 con 0x7fb81c06c680 2024-09-10T11:26:07.334 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.332+0000 7fb8277fe700 1 -- 172.21.15.144:0/2918210985 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27980 (secure 0 0 0) 0x7fb83019d680 con 0x7fb81c06c680 2024-09-10T11:26:07.335 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:26:07.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.334+0000 7fb834c76700 1 -- 172.21.15.144:0/2918210985 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb81c06c680 msgr2=0x7fb81c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:07.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.334+0000 7fb834c76700 1 --2- 172.21.15.144:0/2918210985 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb81c06c680 0x7fb81c06eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb828009480 tx=0x7fb828006000 comp rx=0 tx=0).stop 2024-09-10T11:26:07.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.334+0000 7fb834c76700 1 -- 172.21.15.144:0/2918210985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8300ffc60 msgr2=0x7fb8301984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:07.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.334+0000 7fb834c76700 1 --2- 172.21.15.144:0/2918210985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8300ffc60 0x7fb8301984a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb81800c8f0 tx=0x7fb81800ccb0 comp rx=0 tx=0).stop 2024-09-10T11:26:07.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.334+0000 7fb834c76700 1 -- 172.21.15.144:0/2918210985 shutdown_connections 2024-09-10T11:26:07.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.334+0000 7fb834c76700 1 --2- 172.21.15.144:0/2918210985 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fb81c06c680 0x7fb81c06eb30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:07.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.334+0000 7fb834c76700 1 --2- 172.21.15.144:0/2918210985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8300ff2c0 0x7fb830197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:07.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.334+0000 7fb834c76700 1 --2- 172.21.15.144:0/2918210985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8300ffc60 0x7fb8301984a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:07.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.334+0000 7fb834c76700 1 -- 172.21.15.144:0/2918210985 >> 172.21.15.144:0/2918210985 conn(0x7fb8300fab80 msgr2=0x7fb830102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:07.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.334+0000 7fb834c76700 1 -- 172.21.15.144:0/2918210985 shutdown_connections 2024-09-10T11:26:07.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:07.334+0000 7fb834c76700 1 -- 172.21.15.144:0/2918210985 wait complete. 2024-09-10T11:26:07.338 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2024-09-10T11:26:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:06 smithi178 ceph-mon[33514]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:08.060 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-09-10T11:26:05.918429+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165588,"kb_used_data":3876,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305324,"statfs":{"total":575970213888,"available":575800651776,"internally_reserved":0,"allocated":3969024,"data_stored":2588226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002586"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":184,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T11:25:18.554641+0000","last_change":"2024-09-10T11:24:53.133067+0000","last_active":"2024-09-10T11:25:18.554641+0000","last_peered":"2024-09-10T11:25:18.554641+0000","last_clean":"2024-09-10T11:25:18.554641+0000","last_became_active":"2024-09-10T11:24:53.132549+0000","last_became_peered":"2024-09-10T11:24:53.132549+0000","last_unstale":"2024-09-10T11:25:18.554641+0000","last_undegraded":"2024-09-10T11:25:18.554641+0000","last_fullsized":"2024-09-10T11:25:18.554641+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:24:06.261913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:24:06.261913+0000","last_clean_scrub_stamp":"2024-09-10T11:24:06.261913+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:48.099586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":136187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]}]},{"osd":4,"up_from":35,"seq":150323855376,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":136187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 11:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.46400000000000002}]},{"osd":1,"last update":"Tue Sep 10 11:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.58999999999999997}]},{"osd":2,"last update":"Tue Sep 10 11:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.57299999999999995}]},{"osd":3,"last update":"Tue Sep 10 11:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.31900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]}]},{"osd":3,"up_from":27,"seq":115964117014,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":726555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 11:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.61899999999999999}]},{"osd":1,"last update":"Tue Sep 10 11:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.63500000000000001}]},{"osd":2,"last update":"Tue Sep 10 11:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.66400000000000003}]},{"osd":4,"last update":"Tue Sep 10 11:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.48999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]}]},{"osd":2,"up_from":19,"seq":81604378651,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":136187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 11:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.49299999999999999}]},{"osd":1,"last update":"Tue Sep 10 11:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.59699999999999998}]},{"osd":3,"last update":"Tue Sep 10 11:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.749}]},{"osd":4,"last update":"Tue Sep 10 11:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":0,"up_from":9,"seq":38654705702,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":726555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 11:25:39 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33600000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.25600000000000001},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.307}]},{"osd":2,"last update":"Tue Sep 10 11:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.33600000000000002}]},{"osd":3,"last update":"Tue Sep 10 11:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.40999999999999998}]},{"osd":4,"last update":"Tue Sep 10 11:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55100000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":726555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 11:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.311,"15min":0.29999999999999999},"min":{"1min":0.25900000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.309},{"interface":"front","average":{"1min":0.378,"5min":0.32800000000000001,"15min":0.32000000000000001},"min":{"1min":0.252,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.35699999999999998}]},{"osd":2,"last update":"Tue Sep 10 11:25:05 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.40100000000000002}]},{"osd":3,"last update":"Tue Sep 10 11:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.59199999999999997}]},{"osd":4,"last update":"Tue Sep 10 11:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":6.851,"5min":6.851,"15min":6.851},"last":0.78200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60399999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T11:26:08.062 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T11:26:08.261 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:08.608 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:08 smithi144 ceph-mon[27942]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T11:26:08.608 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:08 smithi144 ceph-mon[27942]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:08.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:08 smithi178 ceph-mon[33514]: from='client.14428 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T11:26:08.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:08 smithi178 ceph-mon[33514]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:09.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.952+0000 7fdf1d059700 1 -- 172.21.15.144:0/245465197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf180ff2c0 msgr2=0x7fdf180ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:09.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.952+0000 7fdf1d059700 1 --2- 172.21.15.144:0/245465197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf180ff2c0 0x7fdf180ff690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fdf08009910 tx=0x7fdf08009c20 comp rx=0 tx=0).stop 2024-09-10T11:26:09.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.952+0000 7fdf1d059700 1 -- 172.21.15.144:0/245465197 shutdown_connections 2024-09-10T11:26:09.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.952+0000 7fdf1d059700 1 --2- 172.21.15.144:0/245465197 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf180ffc60 0x7fdf18103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:09.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.952+0000 7fdf1d059700 1 --2- 172.21.15.144:0/245465197 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf180ff2c0 0x7fdf180ff690 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:09.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.952+0000 7fdf1d059700 1 -- 172.21.15.144:0/245465197 >> 172.21.15.144:0/245465197 conn(0x7fdf180fab80 msgr2=0x7fdf180fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:09.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.952+0000 7fdf1d059700 1 -- 172.21.15.144:0/245465197 shutdown_connections 2024-09-10T11:26:09.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.952+0000 7fdf1d059700 1 -- 172.21.15.144:0/245465197 wait complete. 2024-09-10T11:26:09.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.953+0000 7fdf1d059700 1 Processor -- start 2024-09-10T11:26:09.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.953+0000 7fdf1d059700 1 -- start start 2024-09-10T11:26:09.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.953+0000 7fdf1d059700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf180ff2c0 0x7fdf18197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:09.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.953+0000 7fdf1d059700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf180ffc60 0x7fdf18198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:09.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.953+0000 7fdf1d059700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf18198b70 con 0x7fdf180ffc60 2024-09-10T11:26:09.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.953+0000 7fdf1d059700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1819c900 con 0x7fdf180ff2c0 2024-09-10T11:26:09.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.953+0000 7fdf1659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf180ffc60 0x7fdf18198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:09.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.953+0000 7fdf1659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf180ffc60 0x7fdf18198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51884/0 (socket says 172.21.15.144:51884) 2024-09-10T11:26:09.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.953+0000 7fdf1659c700 1 -- 172.21.15.144:0/3347608145 learned_addr learned my addr 172.21.15.144:0/3347608145 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:09.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.953+0000 7fdf16d9d700 1 --2- 172.21.15.144:0/3347608145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf180ff2c0 0x7fdf18197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:09.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.954+0000 7fdf1659c700 1 -- 172.21.15.144:0/3347608145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf180ff2c0 msgr2=0x7fdf18197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:09.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.954+0000 7fdf1659c700 1 --2- 172.21.15.144:0/3347608145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf180ff2c0 0x7fdf18197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:09.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.954+0000 7fdf1659c700 1 -- 172.21.15.144:0/3347608145 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf080095f0 con 0x7fdf180ffc60 2024-09-10T11:26:09.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.954+0000 7fdf1659c700 1 --2- 172.21.15.144:0/3347608145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf180ffc60 0x7fdf18198490 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00d900 tx=0x7fdf0c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:09.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.954+0000 7fdefffff700 1 -- 172.21.15.144:0/3347608145 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf0c009900 con 0x7fdf180ffc60 2024-09-10T11:26:09.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.954+0000 7fdf1d059700 1 -- 172.21.15.144:0/3347608145 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1819cbe0 con 0x7fdf180ffc60 2024-09-10T11:26:09.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.954+0000 7fdf1d059700 1 -- 172.21.15.144:0/3347608145 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1819d130 con 0x7fdf180ffc60 2024-09-10T11:26:09.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.954+0000 7fdefffff700 1 -- 172.21.15.144:0/3347608145 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdf0c010460 con 0x7fdf180ffc60 2024-09-10T11:26:09.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.954+0000 7fdefffff700 1 -- 172.21.15.144:0/3347608145 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf0c00f5f0 con 0x7fdf180ffc60 2024-09-10T11:26:09.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.955+0000 7fdf1d059700 1 -- 172.21.15.144:0/3347608145 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf04005320 con 0x7fdf180ffc60 2024-09-10T11:26:09.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.955+0000 7fdefffff700 1 -- 172.21.15.144:0/3347608145 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fdf0c009a60 con 0x7fdf180ffc60 2024-09-10T11:26:09.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.956+0000 7fdefffff700 1 --2- 172.21.15.144:0/3347608145 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdf0006c680 0x7fdf0006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:09.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.956+0000 7fdefffff700 1 -- 172.21.15.144:0/3347608145 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fdf0c08af70 con 0x7fdf180ffc60 2024-09-10T11:26:09.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.956+0000 7fdf16d9d700 1 --2- 172.21.15.144:0/3347608145 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdf0006c680 0x7fdf0006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:09.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.956+0000 7fdf16d9d700 1 --2- 172.21.15.144:0/3347608145 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdf0006c680 0x7fdf0006eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdf08006010 tx=0x7fdf0801a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:09.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:09.964+0000 7fdefffff700 1 -- 172.21.15.144:0/3347608145 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdf0c055a30 con 0x7fdf180ffc60 2024-09-10T11:26:10.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:10.157+0000 7fdf1d059700 1 -- 172.21.15.144:0/3347608145 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdf04000bf0 con 0x7fdf0006c680 2024-09-10T11:26:10.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:10.158+0000 7fdefffff700 1 -- 172.21.15.144:0/3347608145 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27893 (secure 0 0 0) 0x7fdf04000bf0 con 0x7fdf0006c680 2024-09-10T11:26:10.161 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:26:10.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:10.160+0000 7fdf1d059700 1 -- 172.21.15.144:0/3347608145 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdf0006c680 msgr2=0x7fdf0006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:10.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:10.160+0000 7fdf1d059700 1 --2- 172.21.15.144:0/3347608145 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdf0006c680 0x7fdf0006eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdf08006010 tx=0x7fdf0801a040 comp rx=0 tx=0).stop 2024-09-10T11:26:10.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:10.161+0000 7fdf1d059700 1 -- 172.21.15.144:0/3347608145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf180ffc60 msgr2=0x7fdf18198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:10.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:10.161+0000 7fdf1d059700 1 --2- 172.21.15.144:0/3347608145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf180ffc60 0x7fdf18198490 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00d900 tx=0x7fdf0c00dc10 comp rx=0 tx=0).stop 2024-09-10T11:26:10.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:10.161+0000 7fdf1d059700 1 -- 172.21.15.144:0/3347608145 shutdown_connections 2024-09-10T11:26:10.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:10.161+0000 7fdf1d059700 1 --2- 172.21.15.144:0/3347608145 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdf0006c680 0x7fdf0006eb30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:10.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:10.161+0000 7fdf1d059700 1 --2- 172.21.15.144:0/3347608145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf180ff2c0 0x7fdf18197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:10.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:10.161+0000 7fdf1d059700 1 --2- 172.21.15.144:0/3347608145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf180ffc60 0x7fdf18198490 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:10.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:10.161+0000 7fdf1d059700 1 -- 172.21.15.144:0/3347608145 >> 172.21.15.144:0/3347608145 conn(0x7fdf180fab80 msgr2=0x7fdf18069570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:10.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:10.161+0000 7fdf1d059700 1 -- 172.21.15.144:0/3347608145 shutdown_connections 2024-09-10T11:26:10.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:10.161+0000 7fdf1d059700 1 -- 172.21.15.144:0/3347608145 wait complete. 2024-09-10T11:26:10.164 INFO:teuthology.orchestra.run.smithi144.stderr:dumped all 2024-09-10T11:26:10.740 INFO:teuthology.orchestra.run.smithi144.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-09-10T11:26:09.919023+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165588,"kb_used_data":3876,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305324,"statfs":{"total":575970213888,"available":575800651776,"internally_reserved":0,"allocated":3969024,"data_stored":2588226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002180"},"pg_stats":[{"pgid":"1.0","version":"24'117","reported_seq":184,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-09-10T11:25:18.554641+0000","last_change":"2024-09-10T11:24:53.133067+0000","last_active":"2024-09-10T11:25:18.554641+0000","last_peered":"2024-09-10T11:25:18.554641+0000","last_clean":"2024-09-10T11:25:18.554641+0000","last_became_active":"2024-09-10T11:24:53.132549+0000","last_became_peered":"2024-09-10T11:24:53.132549+0000","last_unstale":"2024-09-10T11:25:18.554641+0000","last_undegraded":"2024-09-10T11:25:18.554641+0000","last_fullsized":"2024-09-10T11:25:18.554641+0000","mapping_epoch":34,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":35,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T11:24:06.261913+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T11:24:06.261913+0000","last_clean_scrub_stamp":"2024-09-10T11:24:06.261913+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T17:38:48.099586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":40,"seq":171798691854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":136187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]}]},{"osd":4,"up_from":35,"seq":150323855377,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":136187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 11:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.52900000000000003}]},{"osd":1,"last update":"Tue Sep 10 11:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.51700000000000002}]},{"osd":2,"last update":"Tue Sep 10 11:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.57499999999999996}]},{"osd":3,"last update":"Tue Sep 10 11:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"last":0.34300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40300000000000002}]}]},{"osd":3,"up_from":27,"seq":115964117015,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":726555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 11:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.53700000000000003}]},{"osd":1,"last update":"Tue Sep 10 11:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.501}]},{"osd":2,"last update":"Tue Sep 10 11:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.55000000000000004}]},{"osd":4,"last update":"Tue Sep 10 11:25:57 2024","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"last":0.37},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.442}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]}]},{"osd":2,"up_from":19,"seq":81604378651,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":136187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 11:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.49299999999999999}]},{"osd":1,"last update":"Tue Sep 10 11:25:07 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.377,"15min":0.377},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"last":0.59699999999999998}]},{"osd":3,"last update":"Tue Sep 10 11:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.749}]},{"osd":4,"last update":"Tue Sep 10 11:25:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"last":0.63500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":726555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 11:25:39 2024","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.33600000000000002,"15min":0.33400000000000002},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"last":0.313},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.33500000000000002,"15min":0.33100000000000002},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"last":0.34499999999999997}]},{"osd":2,"last update":"Tue Sep 10 11:26:08 2024","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.35799999999999998,"15min":0.36199999999999999},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.46700000000000003,"5min":0.55200000000000005,"15min":0.55200000000000005},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.373,"5min":0.38500000000000001,"15min":0.38700000000000001},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.51300000000000001,"5min":0.627,"15min":0.627},"last":0.36399999999999999}]},{"osd":3,"last update":"Tue Sep 10 11:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.65300000000000002}]},{"osd":4,"last update":"Tue Sep 10 11:25:55 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.755}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":726555,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 11:25:40 2024","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.311,"15min":0.29999999999999999},"min":{"1min":0.25900000000000001,"5min":0.221,"15min":0.221},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.32800000000000001,"15min":0.32000000000000001},"min":{"1min":0.252,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.315}]},{"osd":2,"last update":"Tue Sep 10 11:26:05 2024","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.372,"15min":0.374},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38900000000000001,"15min":0.39200000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.55100000000000005,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.28000000000000003}]},{"osd":3,"last update":"Tue Sep 10 11:25:29 2024","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.5}]},{"osd":4,"last update":"Tue Sep 10 11:25:54 2024","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":6.851,"5min":6.851,"15min":6.851},"last":0.52800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T11:26:10.741 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T11:26:10.741 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T11:26:10.741 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T11:26:10.741 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T11:26:10.950 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:11.305 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:10 smithi144 ceph-mon[27942]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:11.305 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:10 smithi144 ceph-mon[27942]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T11:26:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:10 smithi178 ceph-mon[33514]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:10 smithi178 ceph-mon[33514]: from='client.14432 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T11:26:12.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.540+0000 7fac32bc6700 1 -- 172.21.15.144:0/4009686761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac2c0ffd30 msgr2=0x7fac2c103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:12.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.540+0000 7fac32bc6700 1 --2- 172.21.15.144:0/4009686761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac2c0ffd30 0x7fac2c103ce0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fac1c00b0a0 tx=0x7fac1c00b3b0 comp rx=0 tx=0).stop 2024-09-10T11:26:12.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.540+0000 7fac32bc6700 1 -- 172.21.15.144:0/4009686761 shutdown_connections 2024-09-10T11:26:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.540+0000 7fac32bc6700 1 --2- 172.21.15.144:0/4009686761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac2c0ffd30 0x7fac2c103ce0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.540+0000 7fac32bc6700 1 --2- 172.21.15.144:0/4009686761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac2c0ff390 0x7fac2c0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.540+0000 7fac32bc6700 1 -- 172.21.15.144:0/4009686761 >> 172.21.15.144:0/4009686761 conn(0x7fac2c074f50 msgr2=0x7fac2c075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.540+0000 7fac32bc6700 1 -- 172.21.15.144:0/4009686761 shutdown_connections 2024-09-10T11:26:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.540+0000 7fac32bc6700 1 -- 172.21.15.144:0/4009686761 wait complete. 2024-09-10T11:26:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.541+0000 7fac32bc6700 1 Processor -- start 2024-09-10T11:26:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.541+0000 7fac32bc6700 1 -- start start 2024-09-10T11:26:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.541+0000 7fac32bc6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac2c0ff390 0x7fac2c100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.541+0000 7fac32bc6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac2c0ffd30 0x7fac2c1012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.541+0000 7fac32bc6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c101800 con 0x7fac2c0ffd30 2024-09-10T11:26:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.541+0000 7fac32bc6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac2c101940 con 0x7fac2c0ff390 2024-09-10T11:26:12.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.541+0000 7fac2bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac2c0ffd30 0x7fac2c1012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:12.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.541+0000 7fac2bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac2c0ffd30 0x7fac2c1012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:48436/0 (socket says 172.21.15.144:48436) 2024-09-10T11:26:12.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.541+0000 7fac2bfff700 1 -- 172.21.15.144:0/3559082496 learned_addr learned my addr 172.21.15.144:0/3559082496 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:12.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.542+0000 7fac2bfff700 1 -- 172.21.15.144:0/3559082496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac2c0ff390 msgr2=0x7fac2c100d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:12.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.542+0000 7fac30962700 1 --2- 172.21.15.144:0/3559082496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac2c0ff390 0x7fac2c100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:12.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.542+0000 7fac2bfff700 1 --2- 172.21.15.144:0/3559082496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac2c0ff390 0x7fac2c100d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:12.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.542+0000 7fac2bfff700 1 -- 172.21.15.144:0/3559082496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c009d00 con 0x7fac2c0ffd30 2024-09-10T11:26:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.542+0000 7fac30962700 1 --2- 172.21.15.144:0/3559082496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac2c0ff390 0x7fac2c100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:26:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.542+0000 7fac2bfff700 1 --2- 172.21.15.144:0/3559082496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac2c0ffd30 0x7fac2c1012c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fac1c006130 tx=0x7fac1c00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.542+0000 7fac29ffb700 1 -- 172.21.15.144:0/3559082496 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac1c00e040 con 0x7fac2c0ffd30 2024-09-10T11:26:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.542+0000 7fac29ffb700 1 -- 172.21.15.144:0/3559082496 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fac1c003e60 con 0x7fac2c0ffd30 2024-09-10T11:26:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.542+0000 7fac32bc6700 1 -- 172.21.15.144:0/3559082496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac2c101bc0 con 0x7fac2c0ffd30 2024-09-10T11:26:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.542+0000 7fac29ffb700 1 -- 172.21.15.144:0/3559082496 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac1c01d930 con 0x7fac2c0ffd30 2024-09-10T11:26:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.542+0000 7fac32bc6700 1 -- 172.21.15.144:0/3559082496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac2c1a2340 con 0x7fac2c0ffd30 2024-09-10T11:26:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.543+0000 7fac32bc6700 1 -- 172.21.15.144:0/3559082496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac2c04ea50 con 0x7fac2c0ffd30 2024-09-10T11:26:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.544+0000 7fac29ffb700 1 -- 172.21.15.144:0/3559082496 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fac1c004880 con 0x7fac2c0ffd30 2024-09-10T11:26:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.544+0000 7fac29ffb700 1 --2- 172.21.15.144:0/3559082496 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fac1806c680 0x7fac1806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.544+0000 7fac29ffb700 1 -- 172.21.15.144:0/3559082496 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fac1c08cf10 con 0x7fac2c0ffd30 2024-09-10T11:26:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.544+0000 7fac30962700 1 --2- 172.21.15.144:0/3559082496 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fac1806c680 0x7fac1806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.545+0000 7fac30962700 1 --2- 172.21.15.144:0/3559082496 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fac1806c680 0x7fac1806eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fac14007900 tx=0x7fac14008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:12.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.549+0000 7fac29ffb700 1 -- 172.21.15.144:0/3559082496 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fac1c0579d0 con 0x7fac2c0ffd30 2024-09-10T11:26:12.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.787+0000 7fac32bc6700 1 -- 172.21.15.144:0/3559082496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fac2c1022b0 con 0x7fac2c0ffd30 2024-09-10T11:26:12.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.788+0000 7fac29ffb700 1 -- 172.21.15.144:0/3559082496 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fac1c05aff0 con 0x7fac2c0ffd30 2024-09-10T11:26:12.790 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:26:12.790 INFO:teuthology.orchestra.run.smithi144.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T11:26:12.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.790+0000 7fac32bc6700 1 -- 172.21.15.144:0/3559082496 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fac1806c680 msgr2=0x7fac1806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:12.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.790+0000 7fac32bc6700 1 --2- 172.21.15.144:0/3559082496 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fac1806c680 0x7fac1806eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fac14007900 tx=0x7fac14008040 comp rx=0 tx=0).stop 2024-09-10T11:26:12.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.790+0000 7fac32bc6700 1 -- 172.21.15.144:0/3559082496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac2c0ffd30 msgr2=0x7fac2c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:12.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.790+0000 7fac32bc6700 1 --2- 172.21.15.144:0/3559082496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac2c0ffd30 0x7fac2c1012c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fac1c006130 tx=0x7fac1c00cdd0 comp rx=0 tx=0).stop 2024-09-10T11:26:12.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.790+0000 7fac32bc6700 1 -- 172.21.15.144:0/3559082496 shutdown_connections 2024-09-10T11:26:12.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.790+0000 7fac32bc6700 1 --2- 172.21.15.144:0/3559082496 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fac1806c680 0x7fac1806eb30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:12.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.790+0000 7fac32bc6700 1 --2- 172.21.15.144:0/3559082496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac2c0ff390 0x7fac2c100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:12.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.790+0000 7fac32bc6700 1 --2- 172.21.15.144:0/3559082496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac2c0ffd30 0x7fac2c1012c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:12.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.790+0000 7fac32bc6700 1 -- 172.21.15.144:0/3559082496 >> 172.21.15.144:0/3559082496 conn(0x7fac2c074f50 msgr2=0x7fac2c103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:12.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.790+0000 7fac32bc6700 1 -- 172.21.15.144:0/3559082496 shutdown_connections 2024-09-10T11:26:12.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:12.790+0000 7fac32bc6700 1 -- 172.21.15.144:0/3559082496 wait complete. 2024-09-10T11:26:13.161 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:12 smithi144 ceph-mon[27942]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:13.161 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:12 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/3559082496' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T11:26:13.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:12 smithi178 ceph-mon[33514]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:13.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:12 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/3559082496' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T11:26:13.560 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T11:26:13.560 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T11:26:13.560 INFO:teuthology.run_tasks:Running task print... 2024-09-10T11:26:13.577 INFO:teuthology.task.print:**** done end installing v18.2.0 cephadm ... 2024-09-10T11:26:13.577 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T11:26:13.586 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T11:26:13.586 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-09-10T11:26:13.785 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:14 smithi144 ceph-mon[27942]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:14 smithi178 ceph-mon[33514]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:15.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.548+0000 7f96a7fea700 1 -- 172.21.15.144:0/4177886863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a01013f0 msgr2=0x7f96a0101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:15.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.548+0000 7f96a7fea700 1 --2- 172.21.15.144:0/4177886863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a01013f0 0x7f96a0101840 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f9690009910 tx=0x7f9690009c20 comp rx=0 tx=0).stop 2024-09-10T11:26:15.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.548+0000 7f96a7fea700 1 -- 172.21.15.144:0/4177886863 shutdown_connections 2024-09-10T11:26:15.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.548+0000 7f96a7fea700 1 --2- 172.21.15.144:0/4177886863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a01013f0 0x7f96a0101840 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:15.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.548+0000 7f96a7fea700 1 --2- 172.21.15.144:0/4177886863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a00fe410 0x7f96a00fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:15.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.548+0000 7f96a7fea700 1 -- 172.21.15.144:0/4177886863 >> 172.21.15.144:0/4177886863 conn(0x7f96a00f9d80 msgr2=0x7f96a00fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:15.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.548+0000 7f96a7fea700 1 -- 172.21.15.144:0/4177886863 shutdown_connections 2024-09-10T11:26:15.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.548+0000 7f96a7fea700 1 -- 172.21.15.144:0/4177886863 wait complete. 2024-09-10T11:26:15.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.548+0000 7f96a7fea700 1 Processor -- start 2024-09-10T11:26:15.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.549+0000 7f96a7fea700 1 -- start start 2024-09-10T11:26:15.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.549+0000 7f96a7fea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a00fe410 0x7f96a0100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:15.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.549+0000 7f96a7fea700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a01013f0 0x7f96a00ff3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:15.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.549+0000 7f96a7fea700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00ff8e0 con 0x7f96a00fe410 2024-09-10T11:26:15.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.549+0000 7f96a7fea700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96a00ffa20 con 0x7f96a01013f0 2024-09-10T11:26:15.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.549+0000 7f96a5d86700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a00fe410 0x7f96a0100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:15.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.549+0000 7f96a5d86700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a00fe410 0x7f96a0100d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:48466/0 (socket says 172.21.15.144:48466) 2024-09-10T11:26:15.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.549+0000 7f96a5d86700 1 -- 172.21.15.144:0/774520681 learned_addr learned my addr 172.21.15.144:0/774520681 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:15.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.549+0000 7f96a5585700 1 --2- 172.21.15.144:0/774520681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a01013f0 0x7f96a00ff3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:15.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.549+0000 7f96a5d86700 1 -- 172.21.15.144:0/774520681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a01013f0 msgr2=0x7f96a00ff3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:15.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.549+0000 7f96a5d86700 1 --2- 172.21.15.144:0/774520681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a01013f0 0x7f96a00ff3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:15.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.549+0000 7f96a5d86700 1 -- 172.21.15.144:0/774520681 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96900095f0 con 0x7f96a00fe410 2024-09-10T11:26:15.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.549+0000 7f96a5585700 1 --2- 172.21.15.144:0/774520681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a01013f0 0x7f96a00ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:26:15.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.549+0000 7f96a5d86700 1 --2- 172.21.15.144:0/774520681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a00fe410 0x7f96a0100d50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f969c007e90 tx=0x7f969c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:15.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.549+0000 7f9696ffd700 1 -- 172.21.15.144:0/774520681 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f969c00dda0 con 0x7f96a00fe410 2024-09-10T11:26:15.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.550+0000 7f96a7fea700 1 -- 172.21.15.144:0/774520681 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96a00ffd00 con 0x7f96a00fe410 2024-09-10T11:26:15.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.550+0000 7f9696ffd700 1 -- 172.21.15.144:0/774520681 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f969c00f040 con 0x7f96a00fe410 2024-09-10T11:26:15.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.550+0000 7f9696ffd700 1 -- 172.21.15.144:0/774520681 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f969c012610 con 0x7f96a00fe410 2024-09-10T11:26:15.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.550+0000 7f96a7fea700 1 -- 172.21.15.144:0/774520681 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96a019d9d0 con 0x7f96a00fe410 2024-09-10T11:26:15.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.551+0000 7f96a7fea700 1 -- 172.21.15.144:0/774520681 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96a0108cb0 con 0x7f96a00fe410 2024-09-10T11:26:15.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.551+0000 7f9696ffd700 1 -- 172.21.15.144:0/774520681 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f969c013aa0 con 0x7f96a00fe410 2024-09-10T11:26:15.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.552+0000 7f9696ffd700 1 --2- 172.21.15.144:0/774520681 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f968c06c680 0x7f968c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:15.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.552+0000 7f9696ffd700 1 -- 172.21.15.144:0/774520681 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f969c08b470 con 0x7f96a00fe410 2024-09-10T11:26:15.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.552+0000 7f96a5585700 1 --2- 172.21.15.144:0/774520681 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f968c06c680 0x7f968c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:15.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.552+0000 7f96a5585700 1 --2- 172.21.15.144:0/774520681 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f968c06c680 0x7f968c06eb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f969000b5c0 tx=0x7f9690005e20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:15.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.556+0000 7f9696ffd700 1 -- 172.21.15.144:0/774520681 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f969c0599c0 con 0x7f96a00fe410 2024-09-10T11:26:15.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.746+0000 7f96a7fea700 1 -- 172.21.15.144:0/774520681 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7f96a0108e70 con 0x7f96a00fe410 2024-09-10T11:26:15.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.750+0000 7f9696ffd700 1 -- 172.21.15.144:0/774520681 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7f969c059550 con 0x7f96a00fe410 2024-09-10T11:26:15.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.753+0000 7f96a7fea700 1 -- 172.21.15.144:0/774520681 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f968c06c680 msgr2=0x7f968c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:15.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.753+0000 7f96a7fea700 1 --2- 172.21.15.144:0/774520681 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f968c06c680 0x7f968c06eb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f969000b5c0 tx=0x7f9690005e20 comp rx=0 tx=0).stop 2024-09-10T11:26:15.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.753+0000 7f96a7fea700 1 -- 172.21.15.144:0/774520681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a00fe410 msgr2=0x7f96a0100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:15.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.753+0000 7f96a7fea700 1 --2- 172.21.15.144:0/774520681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a00fe410 0x7f96a0100d50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f969c007e90 tx=0x7f969c00d3b0 comp rx=0 tx=0).stop 2024-09-10T11:26:15.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.753+0000 7f96a7fea700 1 -- 172.21.15.144:0/774520681 shutdown_connections 2024-09-10T11:26:15.755 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.753+0000 7f96a7fea700 1 --2- 172.21.15.144:0/774520681 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f968c06c680 0x7f968c06eb30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:15.755 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.753+0000 7f96a7fea700 1 --2- 172.21.15.144:0/774520681 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96a00fe410 0x7f96a0100d50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:15.755 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.753+0000 7f96a7fea700 1 --2- 172.21.15.144:0/774520681 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96a01013f0 0x7f96a00ff3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:15.755 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.753+0000 7f96a7fea700 1 -- 172.21.15.144:0/774520681 >> 172.21.15.144:0/774520681 conn(0x7f96a00f9d80 msgr2=0x7f96a0104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:15.755 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.753+0000 7f96a7fea700 1 -- 172.21.15.144:0/774520681 shutdown_connections 2024-09-10T11:26:15.755 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:15.753+0000 7f96a7fea700 1 -- 172.21.15.144:0/774520681 wait complete. 2024-09-10T11:26:16.513 INFO:teuthology.run_tasks:Running task print... 2024-09-10T11:26:16.522 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-09-10T11:26:16.522 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T11:26:16.529 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T11:26:16.529 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- bash -c 'ceph orch status' 2024-09-10T11:26:16.733 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:17.066 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:16 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/774520681' entity='client.admin' 2024-09-10T11:26:17.067 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:26:17.067 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:16 smithi144 ceph-mon[27942]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:17.067 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:26:17.067 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:26:17.067 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:16 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:16 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/774520681' entity='client.admin' 2024-09-10T11:26:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:26:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:16 smithi178 ceph-mon[33514]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:26:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:26:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:16 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:18.310 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.309+0000 7f6fe16bf700 1 -- 172.21.15.144:0/250223348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fdc1002b0 msgr2=0x7f6fdc100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:18.310 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.309+0000 7f6fe16bf700 1 --2- 172.21.15.144:0/250223348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fdc1002b0 0x7f6fdc100700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009910 tx=0x7f6fcc009c20 comp rx=0 tx=0).stop 2024-09-10T11:26:18.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.309+0000 7f6fe16bf700 1 -- 172.21.15.144:0/250223348 shutdown_connections 2024-09-10T11:26:18.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.309+0000 7f6fe16bf700 1 --2- 172.21.15.144:0/250223348 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fdc1002b0 0x7f6fdc100700 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:18.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.309+0000 7f6fe16bf700 1 --2- 172.21.15.144:0/250223348 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fdc0ff0b0 0x7f6fdc0ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:18.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.309+0000 7f6fe16bf700 1 -- 172.21.15.144:0/250223348 >> 172.21.15.144:0/250223348 conn(0x7f6fdc0fa660 msgr2=0x7f6fdc0fca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:18.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.309+0000 7f6fe16bf700 1 -- 172.21.15.144:0/250223348 shutdown_connections 2024-09-10T11:26:18.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.309+0000 7f6fe16bf700 1 -- 172.21.15.144:0/250223348 wait complete. 2024-09-10T11:26:18.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.310+0000 7f6fe16bf700 1 Processor -- start 2024-09-10T11:26:18.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.310+0000 7f6fe16bf700 1 -- start start 2024-09-10T11:26:18.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.310+0000 7f6fe16bf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fdc0ff0b0 0x7f6fdc1949a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:18.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.310+0000 7f6fe16bf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fdc1002b0 0x7f6fdc194ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:18.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.310+0000 7f6fe16bf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc195500 con 0x7f6fdc1002b0 2024-09-10T11:26:18.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.310+0000 7f6fe16bf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fdc195640 con 0x7f6fdc0ff0b0 2024-09-10T11:26:18.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.310+0000 7f6fda7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fdc1002b0 0x7f6fdc194ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:18.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.310+0000 7f6fda7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fdc1002b0 0x7f6fdc194ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:48472/0 (socket says 172.21.15.144:48472) 2024-09-10T11:26:18.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.310+0000 7f6fda7fc700 1 -- 172.21.15.144:0/2045753044 learned_addr learned my addr 172.21.15.144:0/2045753044 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:18.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.310+0000 7f6fda7fc700 1 -- 172.21.15.144:0/2045753044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fdc0ff0b0 msgr2=0x7f6fdc1949a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:18.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.310+0000 7f6fda7fc700 1 --2- 172.21.15.144:0/2045753044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fdc0ff0b0 0x7f6fdc1949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:18.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.310+0000 7f6fda7fc700 1 -- 172.21.15.144:0/2045753044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fcc0095f0 con 0x7f6fdc1002b0 2024-09-10T11:26:18.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.311+0000 7f6fda7fc700 1 --2- 172.21.15.144:0/2045753044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fdc1002b0 0x7f6fdc194ee0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc006010 tx=0x7f6fcc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:18.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.311+0000 7f6fd3fff700 1 -- 172.21.15.144:0/2045753044 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fcc01d070 con 0x7f6fdc1002b0 2024-09-10T11:26:18.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.311+0000 7f6fe16bf700 1 -- 172.21.15.144:0/2045753044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fdc19a090 con 0x7f6fdc1002b0 2024-09-10T11:26:18.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.311+0000 7f6fd3fff700 1 -- 172.21.15.144:0/2045753044 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6fcc00f460 con 0x7f6fdc1002b0 2024-09-10T11:26:18.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.311+0000 7f6fd3fff700 1 -- 172.21.15.144:0/2045753044 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fcc021640 con 0x7f6fdc1002b0 2024-09-10T11:26:18.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.311+0000 7f6fe16bf700 1 -- 172.21.15.144:0/2045753044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fdc19a600 con 0x7f6fdc1002b0 2024-09-10T11:26:18.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.312+0000 7f6fe16bf700 1 -- 172.21.15.144:0/2045753044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fdc107ca0 con 0x7f6fdc1002b0 2024-09-10T11:26:18.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.312+0000 7f6fd3fff700 1 -- 172.21.15.144:0/2045753044 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f6fcc00fab0 con 0x7f6fdc1002b0 2024-09-10T11:26:18.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.313+0000 7f6fd3fff700 1 --2- 172.21.15.144:0/2045753044 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6fc806c680 0x7f6fc806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:18.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.313+0000 7f6fd3fff700 1 -- 172.21.15.144:0/2045753044 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f6fcc08c8b0 con 0x7f6fdc1002b0 2024-09-10T11:26:18.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.313+0000 7f6fdaffd700 1 --2- 172.21.15.144:0/2045753044 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6fc806c680 0x7f6fc806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:18.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.313+0000 7f6fdaffd700 1 --2- 172.21.15.144:0/2045753044 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6fc806c680 0x7f6fc806eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc100110 tx=0x7f6fc4006d20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:18.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.317+0000 7f6fd3fff700 1 -- 172.21.15.144:0/2045753044 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6fcc05ae00 con 0x7f6fdc1002b0 2024-09-10T11:26:18.527 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.525+0000 7f6fe16bf700 1 -- 172.21.15.144:0/2045753044 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fdc104c00 con 0x7f6fc806c680 2024-09-10T11:26:18.528 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.527+0000 7f6fd3fff700 1 -- 172.21.15.144:0/2045753044 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6fdc104c00 con 0x7f6fc806c680 2024-09-10T11:26:18.528 INFO:teuthology.orchestra.run.smithi144.stdout:Backend: cephadm 2024-09-10T11:26:18.528 INFO:teuthology.orchestra.run.smithi144.stdout:Available: Yes 2024-09-10T11:26:18.528 INFO:teuthology.orchestra.run.smithi144.stdout:Paused: No 2024-09-10T11:26:18.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.529+0000 7f6fe16bf700 1 -- 172.21.15.144:0/2045753044 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6fc806c680 msgr2=0x7f6fc806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:18.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.529+0000 7f6fe16bf700 1 --2- 172.21.15.144:0/2045753044 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6fc806c680 0x7f6fc806eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6fdc100110 tx=0x7f6fc4006d20 comp rx=0 tx=0).stop 2024-09-10T11:26:18.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.529+0000 7f6fe16bf700 1 -- 172.21.15.144:0/2045753044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fdc1002b0 msgr2=0x7f6fdc194ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:18.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.529+0000 7f6fe16bf700 1 --2- 172.21.15.144:0/2045753044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fdc1002b0 0x7f6fdc194ee0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc006010 tx=0x7f6fcc00be20 comp rx=0 tx=0).stop 2024-09-10T11:26:18.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.529+0000 7f6fe16bf700 1 -- 172.21.15.144:0/2045753044 shutdown_connections 2024-09-10T11:26:18.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.529+0000 7f6fe16bf700 1 --2- 172.21.15.144:0/2045753044 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f6fc806c680 0x7f6fc806eb30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:18.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.529+0000 7f6fe16bf700 1 --2- 172.21.15.144:0/2045753044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fdc0ff0b0 0x7f6fdc1949a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:18.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.529+0000 7f6fe16bf700 1 --2- 172.21.15.144:0/2045753044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fdc1002b0 0x7f6fdc194ee0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:18.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.529+0000 7f6fe16bf700 1 -- 172.21.15.144:0/2045753044 >> 172.21.15.144:0/2045753044 conn(0x7f6fdc0fa660 msgr2=0x7f6fdc1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:18.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.529+0000 7f6fe16bf700 1 -- 172.21.15.144:0/2045753044 shutdown_connections 2024-09-10T11:26:18.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:18.529+0000 7f6fe16bf700 1 -- 172.21.15.144:0/2045753044 wait complete. 2024-09-10T11:26:18.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:18 smithi178 ceph-mon[33514]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:18.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:18 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:26:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:18 smithi144 ceph-mon[27942]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:18 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:26:19.240 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-10T11:26:19.439 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:19.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:19 smithi178 ceph-mon[33514]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:26:19.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:19 smithi144 ceph-mon[27942]: from='client.14444 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:26:20.602 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:20 smithi144 ceph-mon[27942]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:20.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:20 smithi178 ceph-mon[33514]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:21.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.107+0000 7f8458117700 1 -- 172.21.15.144:0/2915087016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8450100c60 msgr2=0x7f8450103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:21.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.107+0000 7f8458117700 1 --2- 172.21.15.144:0/2915087016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8450100c60 0x7f8450103040 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8440009910 tx=0x7f8440009c20 comp rx=0 tx=0).stop 2024-09-10T11:26:21.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.107+0000 7f8458117700 1 -- 172.21.15.144:0/2915087016 shutdown_connections 2024-09-10T11:26:21.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.107+0000 7f8458117700 1 --2- 172.21.15.144:0/2915087016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8450103580 0x7f8450105960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:21.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.107+0000 7f8458117700 1 --2- 172.21.15.144:0/2915087016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8450100c60 0x7f8450103040 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:21.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.107+0000 7f8458117700 1 -- 172.21.15.144:0/2915087016 >> 172.21.15.144:0/2915087016 conn(0x7f84500fa5e0 msgr2=0x7f84500fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:21.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.107+0000 7f8458117700 1 -- 172.21.15.144:0/2915087016 shutdown_connections 2024-09-10T11:26:21.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.107+0000 7f8458117700 1 -- 172.21.15.144:0/2915087016 wait complete. 2024-09-10T11:26:21.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f8458117700 1 Processor -- start 2024-09-10T11:26:21.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f8458117700 1 -- start start 2024-09-10T11:26:21.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f8458117700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8450100c60 0x7f8450197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:21.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f8458117700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8450103580 0x7f84501980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:21.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f8458117700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84501986c0 con 0x7f8450100c60 2024-09-10T11:26:21.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f8458117700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8450198800 con 0x7f8450103580 2024-09-10T11:26:21.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f8455eb3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8450100c60 0x7f8450197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:21.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f8455eb3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8450100c60 0x7f8450197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:48484/0 (socket says 172.21.15.144:48484) 2024-09-10T11:26:21.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f8455eb3700 1 -- 172.21.15.144:0/1421635733 learned_addr learned my addr 172.21.15.144:0/1421635733 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:21.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f84556b2700 1 --2- 172.21.15.144:0/1421635733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8450103580 0x7f84501980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:21.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f8455eb3700 1 -- 172.21.15.144:0/1421635733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8450103580 msgr2=0x7f84501980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:21.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f8455eb3700 1 --2- 172.21.15.144:0/1421635733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8450103580 0x7f84501980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:21.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f8455eb3700 1 -- 172.21.15.144:0/1421635733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84400095f0 con 0x7f8450100c60 2024-09-10T11:26:21.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f84556b2700 1 --2- 172.21.15.144:0/1421635733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8450103580 0x7f84501980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:26:21.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f8455eb3700 1 --2- 172.21.15.144:0/1421635733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8450100c60 0x7f8450197b60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8440006010 tx=0x7f844000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:21.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.108+0000 7f844effd700 1 -- 172.21.15.144:0/1421635733 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f844001d070 con 0x7f8450100c60 2024-09-10T11:26:21.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.109+0000 7f844effd700 1 -- 172.21.15.144:0/1421635733 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f844000f460 con 0x7f8450100c60 2024-09-10T11:26:21.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.109+0000 7f8458117700 1 -- 172.21.15.144:0/1421635733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f845019d250 con 0x7f8450100c60 2024-09-10T11:26:21.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.109+0000 7f844effd700 1 -- 172.21.15.144:0/1421635733 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8440017630 con 0x7f8450100c60 2024-09-10T11:26:21.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.109+0000 7f8458117700 1 -- 172.21.15.144:0/1421635733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845019d740 con 0x7f8450100c60 2024-09-10T11:26:21.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.109+0000 7f8458117700 1 -- 172.21.15.144:0/1421635733 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84500fc1c0 con 0x7f8450100c60 2024-09-10T11:26:21.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.110+0000 7f844effd700 1 -- 172.21.15.144:0/1421635733 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f844000f5d0 con 0x7f8450100c60 2024-09-10T11:26:21.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.110+0000 7f844effd700 1 --2- 172.21.15.144:0/1421635733 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f843806c680 0x7f843806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:21.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.111+0000 7f844effd700 1 -- 172.21.15.144:0/1421635733 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f844008c9d0 con 0x7f8450100c60 2024-09-10T11:26:21.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.111+0000 7f84556b2700 1 --2- 172.21.15.144:0/1421635733 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f843806c680 0x7f843806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:21.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.111+0000 7f84556b2700 1 --2- 172.21.15.144:0/1421635733 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f843806c680 0x7f843806eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8444009dd0 tx=0x7f8444009040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:21.119 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.115+0000 7f844effd700 1 -- 172.21.15.144:0/1421635733 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f844005af20 con 0x7f8450100c60 2024-09-10T11:26:21.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.312+0000 7f8458117700 1 -- 172.21.15.144:0/1421635733 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8450061190 con 0x7f843806c680 2024-09-10T11:26:21.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.321+0000 7f844effd700 1 -- 172.21.15.144:0/1421635733 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2820 (secure 0 0 0) 0x7f8450061190 con 0x7f843806c680 2024-09-10T11:26:21.323 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:26:21.323 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (3m) 2m ago 5m 25.8M - 0.25.0 c8568f914cd2 b5c0bfae59de 2024-09-10T11:26:21.323 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (5m) 2m ago 5m 7025k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:26:21.323 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (4m) 64s ago 4m 6933k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:26:21.323 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (5m) 2m ago 5m 6665k - 18.2.0 dc2bc1663786 3311fb04aa0c 2024-09-10T11:26:21.323 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (4m) 64s ago 4m 6651k - 18.2.0 dc2bc1663786 6e408e8476d4 2024-09-10T11:26:21.323 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (3m) 2m ago 4m 79.6M - 9.4.7 954c08fa6188 e9a1aec955f6 2024-09-10T11:26:21.323 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:9283,8765,8443 running (6m) 2m ago 6m 478M - 18.2.0 dc2bc1663786 71a7e1c8a428 2024-09-10T11:26:21.323 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (4m) 64s ago 4m 437M - 18.2.0 dc2bc1663786 86f93471b98b 2024-09-10T11:26:21.323 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (6m) 2m ago 6m 43.4M 2048M 18.2.0 dc2bc1663786 42addb654e4f 2024-09-10T11:26:21.323 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (4m) 64s ago 4m 41.0M 2048M 18.2.0 dc2bc1663786 3c1ba210ed3f 2024-09-10T11:26:21.324 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (5m) 2m ago 5m 16.3M - 1.5.0 0da6a335fe13 dfbf0c488d17 2024-09-10T11:26:21.324 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (4m) 64s ago 4m 15.1M - 1.5.0 0da6a335fe13 ff2cac118c8d 2024-09-10T11:26:21.324 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (3m) 2m ago 3m 40.7M 3946M 18.2.0 dc2bc1663786 2435febebdb5 2024-09-10T11:26:21.324 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (2m) 2m ago 2m 40.3M 3946M 18.2.0 dc2bc1663786 b55fe920ff9d 2024-09-10T11:26:21.324 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (2m) 2m ago 2m 39.6M 3946M 18.2.0 dc2bc1663786 3ed5618eac33 2024-09-10T11:26:21.324 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (2m) 64s ago 2m 42.0M 4970M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:26:21.324 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (94s) 64s ago 94s 39.3M 4970M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:26:21.324 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (69s) 64s ago 69s 11.5M 4970M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:26:21.324 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (3m) 2m ago 4m 39.9M - 2.43.0 a07b618ecd1d a463a804d232 2024-09-10T11:26:21.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.324+0000 7f8458117700 1 -- 172.21.15.144:0/1421635733 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f843806c680 msgr2=0x7f843806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:21.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.324+0000 7f8458117700 1 --2- 172.21.15.144:0/1421635733 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f843806c680 0x7f843806eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8444009dd0 tx=0x7f8444009040 comp rx=0 tx=0).stop 2024-09-10T11:26:21.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.324+0000 7f8458117700 1 -- 172.21.15.144:0/1421635733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8450100c60 msgr2=0x7f8450197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:21.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.324+0000 7f8458117700 1 --2- 172.21.15.144:0/1421635733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8450100c60 0x7f8450197b60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f8440006010 tx=0x7f844000be20 comp rx=0 tx=0).stop 2024-09-10T11:26:21.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.324+0000 7f8458117700 1 -- 172.21.15.144:0/1421635733 shutdown_connections 2024-09-10T11:26:21.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.324+0000 7f8458117700 1 --2- 172.21.15.144:0/1421635733 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f843806c680 0x7f843806eb30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:21.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.324+0000 7f8458117700 1 --2- 172.21.15.144:0/1421635733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8450100c60 0x7f8450197b60 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:21.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.324+0000 7f8458117700 1 --2- 172.21.15.144:0/1421635733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8450103580 0x7f84501980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:21.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.324+0000 7f8458117700 1 -- 172.21.15.144:0/1421635733 >> 172.21.15.144:0/1421635733 conn(0x7f84500fa5e0 msgr2=0x7f84500ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:21.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.324+0000 7f8458117700 1 -- 172.21.15.144:0/1421635733 shutdown_connections 2024-09-10T11:26:21.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:21.324+0000 7f8458117700 1 -- 172.21.15.144:0/1421635733 wait complete. 2024-09-10T11:26:21.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:21 smithi144 ceph-mon[27942]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:26:21.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:21 smithi178 ceph-mon[33514]: from='client.14448 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:26:21.899 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-10T11:26:22.098 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:22.538 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:22 smithi144 ceph-mon[27942]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:22.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:22 smithi178 ceph-mon[33514]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.672+0000 7f54630dc700 1 -- 172.21.15.144:0/4188772859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c1035a0 msgr2=0x7f545c105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.672+0000 7f54630dc700 1 --2- 172.21.15.144:0/4188772859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c1035a0 0x7f545c105980 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5448009910 tx=0x7f5448009c20 comp rx=0 tx=0).stop 2024-09-10T11:26:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.672+0000 7f54630dc700 1 -- 172.21.15.144:0/4188772859 shutdown_connections 2024-09-10T11:26:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.672+0000 7f54630dc700 1 --2- 172.21.15.144:0/4188772859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c1035a0 0x7f545c105980 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:23.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.672+0000 7f54630dc700 1 --2- 172.21.15.144:0/4188772859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f545c100c80 0x7f545c103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:23.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.672+0000 7f54630dc700 1 -- 172.21.15.144:0/4188772859 >> 172.21.15.144:0/4188772859 conn(0x7f545c0fa600 msgr2=0x7f545c0fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:23.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.672+0000 7f54630dc700 1 -- 172.21.15.144:0/4188772859 shutdown_connections 2024-09-10T11:26:23.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.672+0000 7f54630dc700 1 -- 172.21.15.144:0/4188772859 wait complete. 2024-09-10T11:26:23.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.672+0000 7f54630dc700 1 Processor -- start 2024-09-10T11:26:23.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.672+0000 7f54630dc700 1 -- start start 2024-09-10T11:26:23.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.672+0000 7f54630dc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f545c100c80 0x7f545c195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:23.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.672+0000 7f54630dc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c1035a0 0x7f545c195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:23.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.673+0000 7f54630dc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c1964e0 con 0x7f545c1035a0 2024-09-10T11:26:23.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.673+0000 7f54630dc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c196620 con 0x7f545c100c80 2024-09-10T11:26:23.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.673+0000 7f545bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c1035a0 0x7f545c195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:23.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.673+0000 7f545bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c1035a0 0x7f545c195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57656/0 (socket says 172.21.15.144:57656) 2024-09-10T11:26:23.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.673+0000 7f545bfff700 1 -- 172.21.15.144:0/3134659385 learned_addr learned my addr 172.21.15.144:0/3134659385 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:23.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.673+0000 7f5460e78700 1 --2- 172.21.15.144:0/3134659385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f545c100c80 0x7f545c195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:23.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.673+0000 7f545bfff700 1 -- 172.21.15.144:0/3134659385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f545c100c80 msgr2=0x7f545c195980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:23.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.673+0000 7f545bfff700 1 --2- 172.21.15.144:0/3134659385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f545c100c80 0x7f545c195980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:23.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.673+0000 7f545bfff700 1 -- 172.21.15.144:0/3134659385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54480095f0 con 0x7f545c1035a0 2024-09-10T11:26:23.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.673+0000 7f545bfff700 1 --2- 172.21.15.144:0/3134659385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c1035a0 0x7f545c195ec0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5448006010 tx=0x7f544800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:23.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.673+0000 7f5459ffb700 1 -- 172.21.15.144:0/3134659385 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f544801d070 con 0x7f545c1035a0 2024-09-10T11:26:23.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.673+0000 7f5459ffb700 1 -- 172.21.15.144:0/3134659385 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f544800f460 con 0x7f545c1035a0 2024-09-10T11:26:23.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.673+0000 7f54630dc700 1 -- 172.21.15.144:0/3134659385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545c19b070 con 0x7f545c1035a0 2024-09-10T11:26:23.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.673+0000 7f5459ffb700 1 -- 172.21.15.144:0/3134659385 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5448021640 con 0x7f545c1035a0 2024-09-10T11:26:23.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.673+0000 7f54630dc700 1 -- 172.21.15.144:0/3134659385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545c19b560 con 0x7f545c1035a0 2024-09-10T11:26:23.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.674+0000 7f54630dc700 1 -- 172.21.15.144:0/3134659385 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545c18fb90 con 0x7f545c1035a0 2024-09-10T11:26:23.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.675+0000 7f5459ffb700 1 -- 172.21.15.144:0/3134659385 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f544800fab0 con 0x7f545c1035a0 2024-09-10T11:26:23.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.675+0000 7f5459ffb700 1 --2- 172.21.15.144:0/3134659385 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f544c06c680 0x7f544c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:23.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.676+0000 7f5460e78700 1 --2- 172.21.15.144:0/3134659385 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f544c06c680 0x7f544c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:23.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.676+0000 7f5459ffb700 1 -- 172.21.15.144:0/3134659385 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f544808c770 con 0x7f545c1035a0 2024-09-10T11:26:23.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.676+0000 7f5460e78700 1 --2- 172.21.15.144:0/3134659385 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f544c06c680 0x7f544c06eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5450005950 tx=0x7f545000b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:23.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.680+0000 7f5459ffb700 1 -- 172.21.15.144:0/3134659385 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f544805acc0 con 0x7f545c1035a0 2024-09-10T11:26:23.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.879+0000 7f54630dc700 1 -- 172.21.15.144:0/3134659385 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f545c061190 con 0x7f544c06c680 2024-09-10T11:26:23.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.884+0000 7f5459ffb700 1 -- 172.21.15.144:0/3134659385 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1310 (secure 0 0 0) 0x7f545c061190 con 0x7f544c06c680 2024-09-10T11:26:23.885 INFO:teuthology.orchestra.run.smithi144.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-10T11:26:23.885 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-09-10T11:26:23.886 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter 2/2 2m ago 5m * 2024-09-10T11:26:23.886 INFO:teuthology.orchestra.run.smithi144.stdout:crash 2/2 2m ago 5m * 2024-09-10T11:26:23.886 INFO:teuthology.orchestra.run.smithi144.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-09-10T11:26:23.886 INFO:teuthology.orchestra.run.smithi144.stdout:mgr 2/2 2m ago 6m count:2 2024-09-10T11:26:23.886 INFO:teuthology.orchestra.run.smithi144.stdout:mon 2/2 2m ago 5m smithi144:172.21.15.144=smithi144;smithi178:172.21.15.178=smithi178;count:2 2024-09-10T11:26:23.886 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-09-10T11:26:23.886 INFO:teuthology.orchestra.run.smithi144.stdout:osd 6 2m ago - 2024-09-10T11:26:23.886 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-09-10T11:26:23.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.886+0000 7f54630dc700 1 -- 172.21.15.144:0/3134659385 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f544c06c680 msgr2=0x7f544c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:23.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.886+0000 7f54630dc700 1 --2- 172.21.15.144:0/3134659385 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f544c06c680 0x7f544c06eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5450005950 tx=0x7f545000b410 comp rx=0 tx=0).stop 2024-09-10T11:26:23.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.886+0000 7f54630dc700 1 -- 172.21.15.144:0/3134659385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c1035a0 msgr2=0x7f545c195ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:23.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.886+0000 7f54630dc700 1 --2- 172.21.15.144:0/3134659385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c1035a0 0x7f545c195ec0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5448006010 tx=0x7f544800bd70 comp rx=0 tx=0).stop 2024-09-10T11:26:23.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.886+0000 7f54630dc700 1 -- 172.21.15.144:0/3134659385 shutdown_connections 2024-09-10T11:26:23.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.886+0000 7f54630dc700 1 --2- 172.21.15.144:0/3134659385 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f544c06c680 0x7f544c06eb30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:23.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.886+0000 7f54630dc700 1 --2- 172.21.15.144:0/3134659385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f545c100c80 0x7f545c195980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:23.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.886+0000 7f54630dc700 1 --2- 172.21.15.144:0/3134659385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f545c1035a0 0x7f545c195ec0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:23.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.886+0000 7f54630dc700 1 -- 172.21.15.144:0/3134659385 >> 172.21.15.144:0/3134659385 conn(0x7f545c0fa600 msgr2=0x7f545c0fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:23.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.887+0000 7f54630dc700 1 -- 172.21.15.144:0/3134659385 shutdown_connections 2024-09-10T11:26:23.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:23.887+0000 7f54630dc700 1 -- 172.21.15.144:0/3134659385 wait complete. 2024-09-10T11:26:24.520 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-10T11:26:24.721 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:25.152 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:24 smithi144 ceph-mon[27942]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:26:25.152 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:24 smithi144 ceph-mon[27942]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:24 smithi178 ceph-mon[33514]: from='client.14452 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:26:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:24 smithi178 ceph-mon[33514]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:26.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.378+0000 7febe317d700 1 -- 172.21.15.144:0/881763239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc1035a0 msgr2=0x7febdc105980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:26.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.378+0000 7febe317d700 1 --2- 172.21.15.144:0/881763239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc1035a0 0x7febdc105980 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7febc8009910 tx=0x7febc8009c20 comp rx=0 tx=0).stop 2024-09-10T11:26:26.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.378+0000 7febe317d700 1 -- 172.21.15.144:0/881763239 shutdown_connections 2024-09-10T11:26:26.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.378+0000 7febe317d700 1 --2- 172.21.15.144:0/881763239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc1035a0 0x7febdc105980 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:26.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.378+0000 7febe317d700 1 --2- 172.21.15.144:0/881763239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febdc100c80 0x7febdc103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:26.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.378+0000 7febe317d700 1 -- 172.21.15.144:0/881763239 >> 172.21.15.144:0/881763239 conn(0x7febdc0fa5c0 msgr2=0x7febdc0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:26.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.378+0000 7febe317d700 1 -- 172.21.15.144:0/881763239 shutdown_connections 2024-09-10T11:26:26.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.378+0000 7febe317d700 1 -- 172.21.15.144:0/881763239 wait complete. 2024-09-10T11:26:26.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.379+0000 7febe317d700 1 Processor -- start 2024-09-10T11:26:26.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.379+0000 7febe317d700 1 -- start start 2024-09-10T11:26:26.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.379+0000 7febe317d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc100c80 0x7febdc195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:26.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.379+0000 7febe317d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febdc1035a0 0x7febdc195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:26.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.379+0000 7febe317d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc196420 con 0x7febdc100c80 2024-09-10T11:26:26.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.379+0000 7febe317d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febdc196560 con 0x7febdc1035a0 2024-09-10T11:26:26.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.379+0000 7febe0f19700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc100c80 0x7febdc195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:26.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.379+0000 7febe0f19700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc100c80 0x7febdc195950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57676/0 (socket says 172.21.15.144:57676) 2024-09-10T11:26:26.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.379+0000 7febe0f19700 1 -- 172.21.15.144:0/4225522463 learned_addr learned my addr 172.21.15.144:0/4225522463 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:26.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.379+0000 7febdbfff700 1 --2- 172.21.15.144:0/4225522463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febdc1035a0 0x7febdc195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:26.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.380+0000 7febe0f19700 1 -- 172.21.15.144:0/4225522463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febdc1035a0 msgr2=0x7febdc195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:26.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.380+0000 7febe0f19700 1 --2- 172.21.15.144:0/4225522463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febdc1035a0 0x7febdc195e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:26.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.380+0000 7febe0f19700 1 -- 172.21.15.144:0/4225522463 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc80095f0 con 0x7febdc100c80 2024-09-10T11:26:26.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.380+0000 7febe0f19700 1 --2- 172.21.15.144:0/4225522463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc100c80 0x7febdc195950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7febd000c930 tx=0x7febd000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:26.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.380+0000 7febd9ffb700 1 -- 172.21.15.144:0/4225522463 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febd0007ad0 con 0x7febdc100c80 2024-09-10T11:26:26.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.380+0000 7febe317d700 1 -- 172.21.15.144:0/4225522463 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febdc19afc0 con 0x7febdc100c80 2024-09-10T11:26:26.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.380+0000 7febd9ffb700 1 -- 172.21.15.144:0/4225522463 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7febd000ce80 con 0x7febdc100c80 2024-09-10T11:26:26.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.380+0000 7febd9ffb700 1 -- 172.21.15.144:0/4225522463 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febd0018630 con 0x7febdc100c80 2024-09-10T11:26:26.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.380+0000 7febe317d700 1 -- 172.21.15.144:0/4225522463 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febdc19b4e0 con 0x7febdc100c80 2024-09-10T11:26:26.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.381+0000 7febe317d700 1 -- 172.21.15.144:0/4225522463 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febdc18fb80 con 0x7febdc100c80 2024-09-10T11:26:26.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.381+0000 7febd9ffb700 1 -- 172.21.15.144:0/4225522463 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7febd0007c30 con 0x7febdc100c80 2024-09-10T11:26:26.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.382+0000 7febd9ffb700 1 --2- 172.21.15.144:0/4225522463 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7febcc06c680 0x7febcc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:26.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.382+0000 7febdbfff700 1 --2- 172.21.15.144:0/4225522463 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7febcc06c680 0x7febcc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:26.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.382+0000 7febd9ffb700 1 -- 172.21.15.144:0/4225522463 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7febd008b4a0 con 0x7febdc100c80 2024-09-10T11:26:26.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.382+0000 7febdbfff700 1 --2- 172.21.15.144:0/4225522463 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7febcc06c680 0x7febcc06eb30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7febc8006010 tx=0x7febc800b560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:26.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.386+0000 7febd9ffb700 1 -- 172.21.15.144:0/4225522463 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7febd00599f0 con 0x7febdc100c80 2024-09-10T11:26:26.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.582+0000 7febe317d700 1 -- 172.21.15.144:0/4225522463 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7febdc061190 con 0x7febcc06c680 2024-09-10T11:26:26.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.583+0000 7febd9ffb700 1 -- 172.21.15.144:0/4225522463 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7febdc061190 con 0x7febcc06c680 2024-09-10T11:26:26.584 INFO:teuthology.orchestra.run.smithi144.stdout:HOST ADDR LABELS STATUS 2024-09-10T11:26:26.584 INFO:teuthology.orchestra.run.smithi144.stdout:smithi144 172.21.15.144 2024-09-10T11:26:26.584 INFO:teuthology.orchestra.run.smithi144.stdout:smithi178 172.21.15.178 2024-09-10T11:26:26.585 INFO:teuthology.orchestra.run.smithi144.stdout:2 hosts in cluster 2024-09-10T11:26:26.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.585+0000 7febe317d700 1 -- 172.21.15.144:0/4225522463 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7febcc06c680 msgr2=0x7febcc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:26.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.585+0000 7febe317d700 1 --2- 172.21.15.144:0/4225522463 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7febcc06c680 0x7febcc06eb30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7febc8006010 tx=0x7febc800b560 comp rx=0 tx=0).stop 2024-09-10T11:26:26.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.585+0000 7febe317d700 1 -- 172.21.15.144:0/4225522463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc100c80 msgr2=0x7febdc195950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:26.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.585+0000 7febe317d700 1 --2- 172.21.15.144:0/4225522463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc100c80 0x7febdc195950 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7febd000c930 tx=0x7febd000cc40 comp rx=0 tx=0).stop 2024-09-10T11:26:26.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.585+0000 7febe317d700 1 -- 172.21.15.144:0/4225522463 shutdown_connections 2024-09-10T11:26:26.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.585+0000 7febe317d700 1 --2- 172.21.15.144:0/4225522463 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7febcc06c680 0x7febcc06eb30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:26.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.585+0000 7febe317d700 1 --2- 172.21.15.144:0/4225522463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febdc100c80 0x7febdc195950 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:26.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.585+0000 7febe317d700 1 --2- 172.21.15.144:0/4225522463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febdc1035a0 0x7febdc195e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:26.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.585+0000 7febe317d700 1 -- 172.21.15.144:0/4225522463 >> 172.21.15.144:0/4225522463 conn(0x7febdc0fa5c0 msgr2=0x7febdc0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:26.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.585+0000 7febe317d700 1 -- 172.21.15.144:0/4225522463 shutdown_connections 2024-09-10T11:26:26.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:26.585+0000 7febe317d700 1 -- 172.21.15.144:0/4225522463 wait complete. 2024-09-10T11:26:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:26 smithi144 ceph-mon[27942]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:27.267 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-10T11:26:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:26 smithi178 ceph-mon[33514]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:27.466 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:28.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:27 smithi144 ceph-mon[27942]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:26:28.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:27 smithi178 ceph-mon[33514]: from='client.14456 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:26:29.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.211+0000 7f4779e53700 1 -- 172.21.15.144:0/788735903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47740ff080 msgr2=0x7f47740ff490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:29.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.211+0000 7f4779e53700 1 --2- 172.21.15.144:0/788735903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47740ff080 0x7f47740ff490 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4764009910 tx=0x7f4764009c20 comp rx=0 tx=0).stop 2024-09-10T11:26:29.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.211+0000 7f4779e53700 1 -- 172.21.15.144:0/788735903 shutdown_connections 2024-09-10T11:26:29.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.211+0000 7f4779e53700 1 --2- 172.21.15.144:0/788735903 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4774100280 0x7f47741006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:29.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.211+0000 7f4779e53700 1 --2- 172.21.15.144:0/788735903 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47740ff080 0x7f47740ff490 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:29.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.211+0000 7f4779e53700 1 -- 172.21.15.144:0/788735903 >> 172.21.15.144:0/788735903 conn(0x7f47740fa630 msgr2=0x7f47740fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:29.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.211+0000 7f4779e53700 1 -- 172.21.15.144:0/788735903 shutdown_connections 2024-09-10T11:26:29.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.211+0000 7f4779e53700 1 -- 172.21.15.144:0/788735903 wait complete. 2024-09-10T11:26:29.214 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:28 smithi144 ceph-mon[27942]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:29.216 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.212+0000 7f4779e53700 1 Processor -- start 2024-09-10T11:26:29.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.212+0000 7f4779e53700 1 -- start start 2024-09-10T11:26:29.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.212+0000 7f4779e53700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47740ff080 0x7f4774195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:29.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.212+0000 7f4779e53700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4774100280 0x7f4774195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:29.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.212+0000 7f4779e53700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47741965a0 con 0x7f4774100280 2024-09-10T11:26:29.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.212+0000 7f4779e53700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47741966e0 con 0x7f47740ff080 2024-09-10T11:26:29.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.212+0000 7f476bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4774100280 0x7f4774195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:29.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.212+0000 7f476bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4774100280 0x7f4774195f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57690/0 (socket says 172.21.15.144:57690) 2024-09-10T11:26:29.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.212+0000 7f476bfff700 1 -- 172.21.15.144:0/1721094188 learned_addr learned my addr 172.21.15.144:0/1721094188 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:29.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.212+0000 7f47737fe700 1 --2- 172.21.15.144:0/1721094188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47740ff080 0x7f4774195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:29.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.213+0000 7f476bfff700 1 -- 172.21.15.144:0/1721094188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47740ff080 msgr2=0x7f4774195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:29.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.213+0000 7f476bfff700 1 --2- 172.21.15.144:0/1721094188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47740ff080 0x7f4774195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:29.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.213+0000 7f476bfff700 1 -- 172.21.15.144:0/1721094188 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47640095f0 con 0x7f4774100280 2024-09-10T11:26:29.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.213+0000 7f476bfff700 1 --2- 172.21.15.144:0/1721094188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4774100280 0x7f4774195f80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f475c00b4b0 tx=0x7f475c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:29.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.213+0000 7f47717fa700 1 -- 172.21.15.144:0/1721094188 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f475c011800 con 0x7f4774100280 2024-09-10T11:26:29.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.213+0000 7f4779e53700 1 -- 172.21.15.144:0/1721094188 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477419b190 con 0x7f4774100280 2024-09-10T11:26:29.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.213+0000 7f47717fa700 1 -- 172.21.15.144:0/1721094188 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f475c011e10 con 0x7f4774100280 2024-09-10T11:26:29.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.213+0000 7f47717fa700 1 -- 172.21.15.144:0/1721094188 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f475c00f2e0 con 0x7f4774100280 2024-09-10T11:26:29.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.213+0000 7f4779e53700 1 -- 172.21.15.144:0/1721094188 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f477419b760 con 0x7f4774100280 2024-09-10T11:26:29.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.214+0000 7f4779e53700 1 -- 172.21.15.144:0/1721094188 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4774107c50 con 0x7f4774100280 2024-09-10T11:26:29.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.215+0000 7f47717fa700 1 -- 172.21.15.144:0/1721094188 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f475c010380 con 0x7f4774100280 2024-09-10T11:26:29.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.215+0000 7f47717fa700 1 --2- 172.21.15.144:0/1721094188 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f476006c680 0x7f476006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:29.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.215+0000 7f47717fa700 1 -- 172.21.15.144:0/1721094188 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7f475c08b5a0 con 0x7f4774100280 2024-09-10T11:26:29.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.215+0000 7f47737fe700 1 --2- 172.21.15.144:0/1721094188 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f476006c680 0x7f476006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:29.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.216+0000 7f47737fe700 1 --2- 172.21.15.144:0/1721094188 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f476006c680 0x7f476006eb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4764006010 tx=0x7f476400b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:29.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.219+0000 7f47717fa700 1 -- 172.21.15.144:0/1721094188 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f475c053720 con 0x7f4774100280 2024-09-10T11:26:29.338 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:28 smithi178 ceph-mon[33514]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:29.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.425+0000 7f4779e53700 1 -- 172.21.15.144:0/1721094188 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f47740694a0 con 0x7f476006c680 2024-09-10T11:26:29.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.426+0000 7f47717fa700 1 -- 172.21.15.144:0/1721094188 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f47740694a0 con 0x7f476006c680 2024-09-10T11:26:29.428 INFO:teuthology.orchestra.run.smithi144.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-10T11:26:29.428 INFO:teuthology.orchestra.run.smithi144.stdout:smithi144 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62050085400BGN 372G No 2m ago LVM detected, locked 2024-09-10T11:26:29.428 INFO:teuthology.orchestra.run.smithi144.stdout:smithi178 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75130029480DGN 447G No 69s ago LVM detected, locked 2024-09-10T11:26:29.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.428+0000 7f4779e53700 1 -- 172.21.15.144:0/1721094188 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f476006c680 msgr2=0x7f476006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:29.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.428+0000 7f4779e53700 1 --2- 172.21.15.144:0/1721094188 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f476006c680 0x7f476006eb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4764006010 tx=0x7f476400b540 comp rx=0 tx=0).stop 2024-09-10T11:26:29.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.428+0000 7f4779e53700 1 -- 172.21.15.144:0/1721094188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4774100280 msgr2=0x7f4774195f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:29.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.428+0000 7f4779e53700 1 --2- 172.21.15.144:0/1721094188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4774100280 0x7f4774195f80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f475c00b4b0 tx=0x7f475c00b7c0 comp rx=0 tx=0).stop 2024-09-10T11:26:29.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.429+0000 7f4779e53700 1 -- 172.21.15.144:0/1721094188 shutdown_connections 2024-09-10T11:26:29.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.429+0000 7f4779e53700 1 --2- 172.21.15.144:0/1721094188 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f476006c680 0x7f476006eb30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:29.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.429+0000 7f4779e53700 1 --2- 172.21.15.144:0/1721094188 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47740ff080 0x7f4774195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:29.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.429+0000 7f4779e53700 1 --2- 172.21.15.144:0/1721094188 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4774100280 0x7f4774195f80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:29.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.429+0000 7f4779e53700 1 -- 172.21.15.144:0/1721094188 >> 172.21.15.144:0/1721094188 conn(0x7f47740fa630 msgr2=0x7f4774068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:29.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.429+0000 7f4779e53700 1 -- 172.21.15.144:0/1721094188 shutdown_connections 2024-09-10T11:26:29.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:29.429+0000 7f4779e53700 1 -- 172.21.15.144:0/1721094188 wait complete. 2024-09-10T11:26:30.093 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T11:26:30.103 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T11:26:30.104 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-09-10T11:26:30.307 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:31.085 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:30 smithi144 ceph-mon[27942]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:26:31.085 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:30 smithi144 ceph-mon[27942]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:30 smithi178 ceph-mon[33514]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:26:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:30 smithi178 ceph-mon[33514]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:31.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.783+0000 7fdc9cc29700 1 -- 172.21.15.144:0/2304682789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc98100260 msgr2=0x7fdc981006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:31.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.783+0000 7fdc9cc29700 1 --2- 172.21.15.144:0/2304682789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc98100260 0x7fdc981006b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c009910 tx=0x7fdc8c009c20 comp rx=0 tx=0).stop 2024-09-10T11:26:31.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.783+0000 7fdc9cc29700 1 -- 172.21.15.144:0/2304682789 shutdown_connections 2024-09-10T11:26:31.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.783+0000 7fdc9cc29700 1 --2- 172.21.15.144:0/2304682789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc98100260 0x7fdc981006b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:31.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.783+0000 7fdc9cc29700 1 --2- 172.21.15.144:0/2304682789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc980ff060 0x7fdc980ff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:31.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.783+0000 7fdc9cc29700 1 -- 172.21.15.144:0/2304682789 >> 172.21.15.144:0/2304682789 conn(0x7fdc980fa610 msgr2=0x7fdc980fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:31.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.783+0000 7fdc9cc29700 1 -- 172.21.15.144:0/2304682789 shutdown_connections 2024-09-10T11:26:31.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.783+0000 7fdc9cc29700 1 -- 172.21.15.144:0/2304682789 wait complete. 2024-09-10T11:26:31.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.784+0000 7fdc9cc29700 1 Processor -- start 2024-09-10T11:26:31.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.784+0000 7fdc9cc29700 1 -- start start 2024-09-10T11:26:31.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.784+0000 7fdc9cc29700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc980ff060 0x7fdc98194860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:31.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.784+0000 7fdc9cc29700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc98100260 0x7fdc98194da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:31.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.784+0000 7fdc9cc29700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc981953c0 con 0x7fdc98100260 2024-09-10T11:26:31.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.784+0000 7fdc9cc29700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc98195500 con 0x7fdc980ff060 2024-09-10T11:26:31.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.785+0000 7fdc95d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc98100260 0x7fdc98194da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:31.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.785+0000 7fdc95d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc98100260 0x7fdc98194da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57706/0 (socket says 172.21.15.144:57706) 2024-09-10T11:26:31.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.785+0000 7fdc95d9b700 1 -- 172.21.15.144:0/3926635488 learned_addr learned my addr 172.21.15.144:0/3926635488 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:31.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.785+0000 7fdc9659c700 1 --2- 172.21.15.144:0/3926635488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc980ff060 0x7fdc98194860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:31.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.785+0000 7fdc95d9b700 1 -- 172.21.15.144:0/3926635488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc980ff060 msgr2=0x7fdc98194860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:31.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.785+0000 7fdc95d9b700 1 --2- 172.21.15.144:0/3926635488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc980ff060 0x7fdc98194860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:31.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.785+0000 7fdc95d9b700 1 -- 172.21.15.144:0/3926635488 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc8c0095f0 con 0x7fdc98100260 2024-09-10T11:26:31.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.785+0000 7fdc95d9b700 1 --2- 172.21.15.144:0/3926635488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc98100260 0x7fdc98194da0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c006010 tx=0x7fdc8c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:31.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.785+0000 7fdc877fe700 1 -- 172.21.15.144:0/3926635488 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc8c01d070 con 0x7fdc98100260 2024-09-10T11:26:31.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.785+0000 7fdc877fe700 1 -- 172.21.15.144:0/3926635488 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdc8c00f460 con 0x7fdc98100260 2024-09-10T11:26:31.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.785+0000 7fdc9cc29700 1 -- 172.21.15.144:0/3926635488 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc98199f50 con 0x7fdc98100260 2024-09-10T11:26:31.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.785+0000 7fdc877fe700 1 -- 172.21.15.144:0/3926635488 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc8c021640 con 0x7fdc98100260 2024-09-10T11:26:31.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.785+0000 7fdc9cc29700 1 -- 172.21.15.144:0/3926635488 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc9819a410 con 0x7fdc98100260 2024-09-10T11:26:31.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.786+0000 7fdc9cc29700 1 -- 172.21.15.144:0/3926635488 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc98107bb0 con 0x7fdc98100260 2024-09-10T11:26:31.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.787+0000 7fdc877fe700 1 -- 172.21.15.144:0/3926635488 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fdc8c00fab0 con 0x7fdc98100260 2024-09-10T11:26:31.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.787+0000 7fdc877fe700 1 --2- 172.21.15.144:0/3926635488 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdc8006c680 0x7fdc8006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:31.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.787+0000 7fdc877fe700 1 -- 172.21.15.144:0/3926635488 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4466+0+0 (secure 0 0 0) 0x7fdc8c08c8d0 con 0x7fdc98100260 2024-09-10T11:26:31.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.787+0000 7fdc9659c700 1 --2- 172.21.15.144:0/3926635488 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdc8006c680 0x7fdc8006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:31.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.788+0000 7fdc9659c700 1 --2- 172.21.15.144:0/3926635488 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdc8006c680 0x7fdc8006eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdc981000c0 tx=0x7fdc8800a380 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:31.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:31.792+0000 7fdc877fe700 1 -- 172.21.15.144:0/3926635488 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdc8c05ae20 con 0x7fdc98100260 2024-09-10T11:26:32.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:32.011+0000 7fdc9cc29700 1 -- 172.21.15.144:0/3926635488 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc98104bb0 con 0x7fdc8006c680 2024-09-10T11:26:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:32 smithi178 ceph-mon[33514]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:32 smithi178 ceph-mon[33514]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:26:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:32 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-09-10T11:26:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:32 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:26:33.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:32 smithi144 ceph-mon[27942]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:33.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:32 smithi144 ceph-mon[27942]: from='client.14464 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:26:33.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:32 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-09-10T11:26:33.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:32 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:26:34.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:34.022+0000 7fdc877fe700 1 -- 172.21.15.144:0/3926635488 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdc98104bb0 con 0x7fdc8006c680 2024-09-10T11:26:34.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:34.025+0000 7fdc9cc29700 1 -- 172.21.15.144:0/3926635488 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdc8006c680 msgr2=0x7fdc8006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:34.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:34.025+0000 7fdc9cc29700 1 --2- 172.21.15.144:0/3926635488 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdc8006c680 0x7fdc8006eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdc981000c0 tx=0x7fdc8800a380 comp rx=0 tx=0).stop 2024-09-10T11:26:34.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:34.025+0000 7fdc9cc29700 1 -- 172.21.15.144:0/3926635488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc98100260 msgr2=0x7fdc98194da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:34.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:34.025+0000 7fdc9cc29700 1 --2- 172.21.15.144:0/3926635488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc98100260 0x7fdc98194da0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c006010 tx=0x7fdc8c00bd70 comp rx=0 tx=0).stop 2024-09-10T11:26:34.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:34.025+0000 7fdc9cc29700 1 -- 172.21.15.144:0/3926635488 shutdown_connections 2024-09-10T11:26:34.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:34.025+0000 7fdc9cc29700 1 --2- 172.21.15.144:0/3926635488 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdc8006c680 0x7fdc8006eb30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:34.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:34.025+0000 7fdc9cc29700 1 --2- 172.21.15.144:0/3926635488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc980ff060 0x7fdc98194860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:34.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:34.025+0000 7fdc9cc29700 1 --2- 172.21.15.144:0/3926635488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc98100260 0x7fdc98194da0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:34.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:34.025+0000 7fdc9cc29700 1 -- 172.21.15.144:0/3926635488 >> 172.21.15.144:0/3926635488 conn(0x7fdc980fa610 msgr2=0x7fdc98103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:34.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:34.026+0000 7fdc9cc29700 1 -- 172.21.15.144:0/3926635488 shutdown_connections 2024-09-10T11:26:34.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:34.026+0000 7fdc9cc29700 1 -- 172.21.15.144:0/3926635488 wait complete. 2024-09-10T11:26:34.334 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:33 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-09-10T11:26:34.334 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:33 smithi144 ceph-mon[27942]: osdmap e42: 6 total, 6 up, 6 in 2024-09-10T11:26:34.334 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:33 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-09-10T11:26:34.334 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[27938]: 2024-09-10T11:26:34.003+0000 7f4ebd379700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:26:34.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:33 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-09-10T11:26:34.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:33 smithi178 ceph-mon[33514]: osdmap e42: 6 total, 6 up, 6 in 2024-09-10T11:26:34.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:33 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-09-10T11:26:34.691 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T11:26:35.229 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: pgmap v173: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:35.229 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-09-10T11:26:35.229 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: osdmap e43: 6 total, 6 up, 6 in 2024-09-10T11:26:35.229 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-09-10T11:26:35.229 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:26:35.229 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T11:26:35.229 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: osdmap e44: 6 total, 6 up, 6 in 2024-09-10T11:26:35.230 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-09-10T11:26:35.230 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: fsmap cephfs:0 2024-09-10T11:26:35.230 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: Saving service mds.cephfs spec with placement count:4 2024-09-10T11:26:35.230 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:35.230 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:26:35.230 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:26:35.230 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:26:35.230 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:35.230 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.bfyvwv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:26:35.230 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.bfyvwv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T11:26:35.230 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:34 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:26:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: pgmap v173: 33 pgs: 6 creating+peering, 1 active+clean, 26 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-09-10T11:26:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: osdmap e43: 6 total, 6 up, 6 in 2024-09-10T11:26:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-09-10T11:26:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:26:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T11:26:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: osdmap e44: 6 total, 6 up, 6 in 2024-09-10T11:26:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-09-10T11:26:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: fsmap cephfs:0 2024-09-10T11:26:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: Saving service mds.cephfs spec with placement count:4 2024-09-10T11:26:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:26:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:26:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:26:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:35.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.bfyvwv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:26:35.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.bfyvwv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T11:26:35.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:34 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:26:35.501 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:36.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:36 smithi144 ceph-mon[27942]: Deploying daemon mds.cephfs.smithi144.bfyvwv on smithi144 2024-09-10T11:26:36.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:36 smithi144 ceph-mon[27942]: osdmap e45: 6 total, 6 up, 6 in 2024-09-10T11:26:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:36 smithi178 ceph-mon[33514]: Deploying daemon mds.cephfs.smithi144.bfyvwv on smithi144 2024-09-10T11:26:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:36 smithi178 ceph-mon[33514]: osdmap e45: 6 total, 6 up, 6 in 2024-09-10T11:26:37.349 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:37 smithi144 ceph-mon[27942]: pgmap v177: 65 pgs: 2 creating+activating, 6 creating+peering, 21 active+clean, 36 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:37.349 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:37 smithi144 ceph-mon[27942]: osdmap e46: 6 total, 6 up, 6 in 2024-09-10T11:26:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:37 smithi178 ceph-mon[33514]: pgmap v177: 65 pgs: 2 creating+activating, 6 creating+peering, 21 active+clean, 36 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:37 smithi178 ceph-mon[33514]: osdmap e46: 6 total, 6 up, 6 in 2024-09-10T11:26:37.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.849+0000 7f643b270700 1 -- 172.21.15.144:0/440321633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6434100190 msgr2=0x7f64341005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:37.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.849+0000 7f643b270700 1 --2- 172.21.15.144:0/440321633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6434100190 0x7f64341005a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6428009910 tx=0x7f6428009c20 comp rx=0 tx=0).stop 2024-09-10T11:26:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.849+0000 7f643b270700 1 -- 172.21.15.144:0/440321633 shutdown_connections 2024-09-10T11:26:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.849+0000 7f643b270700 1 --2- 172.21.15.144:0/440321633 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6434101390 0x7f64341017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.849+0000 7f643b270700 1 --2- 172.21.15.144:0/440321633 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6434100190 0x7f64341005a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.849+0000 7f643b270700 1 -- 172.21.15.144:0/440321633 >> 172.21.15.144:0/440321633 conn(0x7f64340fb720 msgr2=0x7f64340fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.849+0000 7f643b270700 1 -- 172.21.15.144:0/440321633 shutdown_connections 2024-09-10T11:26:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.849+0000 7f643b270700 1 -- 172.21.15.144:0/440321633 wait complete. 2024-09-10T11:26:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.850+0000 7f643b270700 1 Processor -- start 2024-09-10T11:26:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.850+0000 7f643b270700 1 -- start start 2024-09-10T11:26:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.850+0000 7f643b270700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6434100190 0x7f6434074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.850+0000 7f643b270700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6434101390 0x7f6434073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.850+0000 7f643b270700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6434073680 con 0x7f6434100190 2024-09-10T11:26:37.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.850+0000 7f643b270700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64340737c0 con 0x7f6434101390 2024-09-10T11:26:37.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.850+0000 7f643900c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6434100190 0x7f6434074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:37.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.850+0000 7f643900c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6434100190 0x7f6434074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38836/0 (socket says 172.21.15.144:38836) 2024-09-10T11:26:37.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.850+0000 7f643900c700 1 -- 172.21.15.144:0/1276424696 learned_addr learned my addr 172.21.15.144:0/1276424696 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:37.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.850+0000 7f643900c700 1 -- 172.21.15.144:0/1276424696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6434101390 msgr2=0x7f6434073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:37.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.850+0000 7f643900c700 1 --2- 172.21.15.144:0/1276424696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6434101390 0x7f6434073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:37.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.850+0000 7f643900c700 1 -- 172.21.15.144:0/1276424696 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64280095f0 con 0x7f6434100190 2024-09-10T11:26:37.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.850+0000 7f643900c700 1 --2- 172.21.15.144:0/1276424696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6434100190 0x7f6434074af0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f642800bda0 tx=0x7f642800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:37.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.851+0000 7f64267fc700 1 -- 172.21.15.144:0/1276424696 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f642801d070 con 0x7f6434100190 2024-09-10T11:26:37.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.851+0000 7f64267fc700 1 -- 172.21.15.144:0/1276424696 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6428004c50 con 0x7f6434100190 2024-09-10T11:26:37.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.851+0000 7f643b270700 1 -- 172.21.15.144:0/1276424696 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6434073a40 con 0x7f6434100190 2024-09-10T11:26:37.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.851+0000 7f64267fc700 1 -- 172.21.15.144:0/1276424696 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f642800f690 con 0x7f6434100190 2024-09-10T11:26:37.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.851+0000 7f643b270700 1 -- 172.21.15.144:0/1276424696 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6434073f30 con 0x7f6434100190 2024-09-10T11:26:37.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.852+0000 7f643b270700 1 -- 172.21.15.144:0/1276424696 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6434108ce0 con 0x7f6434100190 2024-09-10T11:26:37.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.852+0000 7f64267fc700 1 -- 172.21.15.144:0/1276424696 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f64280229b0 con 0x7f6434100190 2024-09-10T11:26:37.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.853+0000 7f64267fc700 1 --2- 172.21.15.144:0/1276424696 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f642006c680 0x7f642006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:37.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.853+0000 7f64267fc700 1 -- 172.21.15.144:0/1276424696 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f642808cf40 con 0x7f6434100190 2024-09-10T11:26:37.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.853+0000 7f643880b700 1 --2- 172.21.15.144:0/1276424696 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f642006c680 0x7f642006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:37.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.853+0000 7f643880b700 1 --2- 172.21.15.144:0/1276424696 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f642006c680 0x7f642006eb30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f643000a990 tx=0x7f6430005c80 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:37.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:37.857+0000 7f64267fc700 1 -- 172.21.15.144:0/1276424696 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f642805b210 con 0x7f6434100190 2024-09-10T11:26:38.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:38.095+0000 7f643b270700 1 -- 172.21.15.144:0/1276424696 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f643419de70 con 0x7f6434100190 2024-09-10T11:26:38.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:38.096+0000 7f64267fc700 1 -- 172.21.15.144:0/1276424696 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1093 (secure 0 0 0) 0x7f642805ada0 con 0x7f6434100190 2024-09-10T11:26:38.098 INFO:teuthology.orchestra.run.smithi144.stdout:e2 2024-09-10T11:26:38.098 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:26:38.098 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:26:38.098 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:26:38.098 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:26:38.098 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:26:38.099 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:26:38.099 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 2 2024-09-10T11:26:38.099 INFO:teuthology.orchestra.run.smithi144.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-09-10T11:26:38.099 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:26:38.099 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:26:34.004556+0000 2024-09-10T11:26:38.099 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:26:38.099 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:26:38.099 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:26:38.099 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:26:38.099 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:26:38.099 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:26:38.099 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:26:38.100 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 0 2024-09-10T11:26:38.100 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:26:38.100 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 1 2024-09-10T11:26:38.100 INFO:teuthology.orchestra.run.smithi144.stdout:in 2024-09-10T11:26:38.100 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:26:38.100 INFO:teuthology.orchestra.run.smithi144.stdout:failed 2024-09-10T11:26:38.100 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:26:38.100 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:26:38.100 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:26:38.100 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:26:38.100 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data disabled 2024-09-10T11:26:38.100 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:26:38.100 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:26:38.100 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 0 2024-09-10T11:26:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:26:38.101 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:26:38.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:38.098+0000 7f643b270700 1 -- 172.21.15.144:0/1276424696 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f642006c680 msgr2=0x7f642006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:38.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:38.098+0000 7f643b270700 1 --2- 172.21.15.144:0/1276424696 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f642006c680 0x7f642006eb30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f643000a990 tx=0x7f6430005c80 comp rx=0 tx=0).stop 2024-09-10T11:26:38.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:38.098+0000 7f643b270700 1 -- 172.21.15.144:0/1276424696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6434100190 msgr2=0x7f6434074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:38.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:38.098+0000 7f643b270700 1 --2- 172.21.15.144:0/1276424696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6434100190 0x7f6434074af0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f642800bda0 tx=0x7f642800be80 comp rx=0 tx=0).stop 2024-09-10T11:26:38.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:38.098+0000 7f643b270700 1 -- 172.21.15.144:0/1276424696 shutdown_connections 2024-09-10T11:26:38.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:38.098+0000 7f643b270700 1 --2- 172.21.15.144:0/1276424696 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f642006c680 0x7f642006eb30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:38.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:38.098+0000 7f643b270700 1 --2- 172.21.15.144:0/1276424696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6434100190 0x7f6434074af0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:38.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:38.098+0000 7f643b270700 1 --2- 172.21.15.144:0/1276424696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6434101390 0x7f6434073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:38.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:38.098+0000 7f643b270700 1 -- 172.21.15.144:0/1276424696 >> 172.21.15.144:0/1276424696 conn(0x7f64340fb720 msgr2=0x7f64341045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:38.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:38.098+0000 7f643b270700 1 -- 172.21.15.144:0/1276424696 shutdown_connections 2024-09-10T11:26:38.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:38.098+0000 7f643b270700 1 -- 172.21.15.144:0/1276424696 wait complete. 2024-09-10T11:26:38.103 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 2 2024-09-10T11:26:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:38 smithi178 ceph-mon[33514]: pgmap v179: 65 pgs: 2 creating+activating, 6 creating+peering, 48 active+clean, 9 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:38 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/1276424696' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:26:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:38 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:38 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:38.659 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:38 smithi144 ceph-mon[27942]: pgmap v179: 65 pgs: 2 creating+activating, 6 creating+peering, 48 active+clean, 9 unknown; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:26:38.659 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:38 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1276424696' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:26:38.660 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:38 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:38.660 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:38 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:39.066 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T11:26:39.078 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T11:26:39.078 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-09-10T11:26:39.279 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:39.605 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:39 smithi144 ceph-mon[27942]: mds.? [v2:172.21.15.144:6826/3978020411,v1:172.21.15.144:6827/3978020411] up:boot 2024-09-10T11:26:39.606 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:39 smithi144 ceph-mon[27942]: daemon mds.cephfs.smithi144.bfyvwv assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-09-10T11:26:39.606 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:39 smithi144 ceph-mon[27942]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T11:26:39.606 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:39 smithi144 ceph-mon[27942]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T11:26:39.606 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:39 smithi144 ceph-mon[27942]: Cluster is now healthy 2024-09-10T11:26:39.606 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:39 smithi144 ceph-mon[27942]: fsmap cephfs:0 1 up:standby 2024-09-10T11:26:39.606 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:39 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:26:39.606 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:39 smithi144 ceph-mon[27942]: fsmap cephfs:1 {0=cephfs.smithi144.bfyvwv=up:creating} 2024-09-10T11:26:39.606 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:39 smithi144 ceph-mon[27942]: daemon mds.cephfs.smithi144.bfyvwv is now active in filesystem cephfs as rank 0 2024-09-10T11:26:39.606 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:39 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:39.606 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:39 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.nmhrmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:26:39.606 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:39 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.nmhrmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T11:26:39.606 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:39 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:26:39.607 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:39 smithi144 ceph-mon[27942]: Deploying daemon mds.cephfs.smithi178.nmhrmh on smithi178 2024-09-10T11:26:39.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:39 smithi178 ceph-mon[33514]: mds.? [v2:172.21.15.144:6826/3978020411,v1:172.21.15.144:6827/3978020411] up:boot 2024-09-10T11:26:39.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:39 smithi178 ceph-mon[33514]: daemon mds.cephfs.smithi144.bfyvwv assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-09-10T11:26:39.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:39 smithi178 ceph-mon[33514]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T11:26:39.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:39 smithi178 ceph-mon[33514]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T11:26:39.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:39 smithi178 ceph-mon[33514]: Cluster is now healthy 2024-09-10T11:26:39.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:39 smithi178 ceph-mon[33514]: fsmap cephfs:0 1 up:standby 2024-09-10T11:26:39.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:39 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:26:39.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:39 smithi178 ceph-mon[33514]: fsmap cephfs:1 {0=cephfs.smithi144.bfyvwv=up:creating} 2024-09-10T11:26:39.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:39 smithi178 ceph-mon[33514]: daemon mds.cephfs.smithi144.bfyvwv is now active in filesystem cephfs as rank 0 2024-09-10T11:26:39.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:39 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:39.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:39 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.nmhrmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:26:39.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:39 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.nmhrmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T11:26:39.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:39 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:26:39.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:39 smithi178 ceph-mon[33514]: Deploying daemon mds.cephfs.smithi178.nmhrmh on smithi178 2024-09-10T11:26:40.553 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:40 smithi178 ceph-mon[33514]: mds.? [v2:172.21.15.144:6826/3978020411,v1:172.21.15.144:6827/3978020411] up:active 2024-09-10T11:26:40.553 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:40 smithi178 ceph-mon[33514]: fsmap cephfs:1 {0=cephfs.smithi144.bfyvwv=up:active} 2024-09-10T11:26:40.553 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:40 smithi178 ceph-mon[33514]: pgmap v180: 65 pgs: 2 creating+activating, 63 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-09-10T11:26:40.637 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:40 smithi144 ceph-mon[27942]: mds.? [v2:172.21.15.144:6826/3978020411,v1:172.21.15.144:6827/3978020411] up:active 2024-09-10T11:26:40.637 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:40 smithi144 ceph-mon[27942]: fsmap cephfs:1 {0=cephfs.smithi144.bfyvwv=up:active} 2024-09-10T11:26:40.637 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:40 smithi144 ceph-mon[27942]: pgmap v180: 65 pgs: 2 creating+activating, 63 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.3 KiB/s wr, 1 op/s 2024-09-10T11:26:40.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.744+0000 7f5bde908700 1 -- 172.21.15.144:0/3448590740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd8103500 msgr2=0x7f5bd81058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:40.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.744+0000 7f5bde908700 1 --2- 172.21.15.144:0/3448590740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd8103500 0x7f5bd81058e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009910 tx=0x7f5bc0009c20 comp rx=0 tx=0).stop 2024-09-10T11:26:40.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.744+0000 7f5bde908700 1 -- 172.21.15.144:0/3448590740 shutdown_connections 2024-09-10T11:26:40.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.744+0000 7f5bde908700 1 --2- 172.21.15.144:0/3448590740 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd8103500 0x7f5bd81058e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:40.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.744+0000 7f5bde908700 1 --2- 172.21.15.144:0/3448590740 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd8100be0 0x7f5bd8102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:40.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.744+0000 7f5bde908700 1 -- 172.21.15.144:0/3448590740 >> 172.21.15.144:0/3448590740 conn(0x7f5bd80fa600 msgr2=0x7f5bd80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:40.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.744+0000 7f5bde908700 1 -- 172.21.15.144:0/3448590740 shutdown_connections 2024-09-10T11:26:40.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.744+0000 7f5bde908700 1 -- 172.21.15.144:0/3448590740 wait complete. 2024-09-10T11:26:40.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.745+0000 7f5bde908700 1 Processor -- start 2024-09-10T11:26:40.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.745+0000 7f5bde908700 1 -- start start 2024-09-10T11:26:40.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.745+0000 7f5bde908700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd8100be0 0x7f5bd81959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:40.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.745+0000 7f5bde908700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd8103500 0x7f5bd8195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:40.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.745+0000 7f5bde908700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd8196500 con 0x7f5bd8103500 2024-09-10T11:26:40.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.745+0000 7f5bde908700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd8196640 con 0x7f5bd8100be0 2024-09-10T11:26:40.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.745+0000 7f5bcffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd8103500 0x7f5bd8195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:40.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.745+0000 7f5bcffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd8103500 0x7f5bd8195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38870/0 (socket says 172.21.15.144:38870) 2024-09-10T11:26:40.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.745+0000 7f5bcffff700 1 -- 172.21.15.144:0/4090633200 learned_addr learned my addr 172.21.15.144:0/4090633200 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:40.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.745+0000 7f5bd7fff700 1 --2- 172.21.15.144:0/4090633200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd8100be0 0x7f5bd81959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:40.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.745+0000 7f5bcffff700 1 -- 172.21.15.144:0/4090633200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd8100be0 msgr2=0x7f5bd81959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:40.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.745+0000 7f5bcffff700 1 --2- 172.21.15.144:0/4090633200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd8100be0 0x7f5bd81959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:40.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.745+0000 7f5bcffff700 1 -- 172.21.15.144:0/4090633200 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc00095f0 con 0x7f5bd8103500 2024-09-10T11:26:40.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.745+0000 7f5bcffff700 1 --2- 172.21.15.144:0/4090633200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd8103500 0x7f5bd8195ee0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0006010 tx=0x7f5bc000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:40.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.746+0000 7f5bd5ffb700 1 -- 172.21.15.144:0/4090633200 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bc001d070 con 0x7f5bd8103500 2024-09-10T11:26:40.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.746+0000 7f5bd5ffb700 1 -- 172.21.15.144:0/4090633200 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5bc000f460 con 0x7f5bd8103500 2024-09-10T11:26:40.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.746+0000 7f5bde908700 1 -- 172.21.15.144:0/4090633200 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd819b090 con 0x7f5bd8103500 2024-09-10T11:26:40.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.746+0000 7f5bd5ffb700 1 -- 172.21.15.144:0/4090633200 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bc0021640 con 0x7f5bd8103500 2024-09-10T11:26:40.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.746+0000 7f5bde908700 1 -- 172.21.15.144:0/4090633200 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd819b600 con 0x7f5bd8103500 2024-09-10T11:26:40.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.747+0000 7f5bde908700 1 -- 172.21.15.144:0/4090633200 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd818fbf0 con 0x7f5bd8103500 2024-09-10T11:26:40.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.747+0000 7f5bd5ffb700 1 -- 172.21.15.144:0/4090633200 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f5bc000fab0 con 0x7f5bd8103500 2024-09-10T11:26:40.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.748+0000 7f5bd5ffb700 1 --2- 172.21.15.144:0/4090633200 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f5bc406c680 0x7f5bc406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:40.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.748+0000 7f5bd5ffb700 1 -- 172.21.15.144:0/4090633200 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5296+0+0 (secure 0 0 0) 0x7f5bc008cc30 con 0x7f5bd8103500 2024-09-10T11:26:40.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.748+0000 7f5bd7fff700 1 --2- 172.21.15.144:0/4090633200 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f5bc406c680 0x7f5bc406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:40.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.748+0000 7f5bd7fff700 1 --2- 172.21.15.144:0/4090633200 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f5bc406c680 0x7f5bc406eb30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8006fd0 tx=0x7f5bc8008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:40.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.752+0000 7f5bd5ffb700 1 -- 172.21.15.144:0/4090633200 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5bc005aed0 con 0x7f5bd8103500 2024-09-10T11:26:40.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:40.997+0000 7f5bde908700 1 -- 172.21.15.144:0/4090633200 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f5bd8061960 con 0x7f5bd8103500 2024-09-10T11:26:41.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:41.372+0000 7f5bd5ffb700 1 -- 172.21.15.144:0/4090633200 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v6) v1 ==== 105+0+0 (secure 0 0 0) 0x7f5bc00260b0 con 0x7f5bd8103500 2024-09-10T11:26:41.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:41.376+0000 7f5bde908700 1 -- 172.21.15.144:0/4090633200 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f5bc406c680 msgr2=0x7f5bc406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:41.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:41.376+0000 7f5bde908700 1 --2- 172.21.15.144:0/4090633200 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f5bc406c680 0x7f5bc406eb30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5bc8006fd0 tx=0x7f5bc8008040 comp rx=0 tx=0).stop 2024-09-10T11:26:41.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:41.376+0000 7f5bde908700 1 -- 172.21.15.144:0/4090633200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd8103500 msgr2=0x7f5bd8195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:41.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:41.376+0000 7f5bde908700 1 --2- 172.21.15.144:0/4090633200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd8103500 0x7f5bd8195ee0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0006010 tx=0x7f5bc000be20 comp rx=0 tx=0).stop 2024-09-10T11:26:41.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:41.376+0000 7f5bde908700 1 -- 172.21.15.144:0/4090633200 shutdown_connections 2024-09-10T11:26:41.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:41.376+0000 7f5bde908700 1 --2- 172.21.15.144:0/4090633200 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f5bc406c680 0x7f5bc406eb30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:41.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:41.376+0000 7f5bde908700 1 --2- 172.21.15.144:0/4090633200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd8100be0 0x7f5bd81959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:41.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:41.376+0000 7f5bde908700 1 --2- 172.21.15.144:0/4090633200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd8103500 0x7f5bd8195ee0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:41.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:41.376+0000 7f5bde908700 1 -- 172.21.15.144:0/4090633200 >> 172.21.15.144:0/4090633200 conn(0x7f5bd80fa600 msgr2=0x7f5bd80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:41.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:41.376+0000 7f5bde908700 1 -- 172.21.15.144:0/4090633200 shutdown_connections 2024-09-10T11:26:41.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:41.376+0000 7f5bde908700 1 -- 172.21.15.144:0/4090633200 wait complete. 2024-09-10T11:26:41.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:41 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/4090633200' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-09-10T11:26:41.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:41 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:41.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:41 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:41.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:41 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:41.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:41 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jaoyvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:26:41.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:41 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jaoyvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T11:26:41.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:41 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:26:41.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:41 smithi178 ceph-mon[33514]: Deploying daemon mds.cephfs.smithi144.jaoyvs on smithi144 2024-09-10T11:26:41.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:41 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/4090633200' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-09-10T11:26:41.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:41 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:41.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:41 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:41.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:41 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:41.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:41 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jaoyvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:26:41.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:41 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jaoyvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T11:26:41.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:41 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:26:41.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:41 smithi144 ceph-mon[27942]: Deploying daemon mds.cephfs.smithi144.jaoyvs on smithi144 2024-09-10T11:26:41.990 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T11:26:42.001 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T11:26:42.001 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay true' 2024-09-10T11:26:42.629 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:42 smithi178 ceph-mon[33514]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T11:26:42.629 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:42 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/4090633200' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-09-10T11:26:42.629 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:42 smithi178 ceph-mon[33514]: mds.? [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] up:boot 2024-09-10T11:26:42.629 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:42 smithi178 ceph-mon[33514]: daemon mds.cephfs.smithi178.nmhrmh assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T11:26:42.629 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:42 smithi178 ceph-mon[33514]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-10T11:26:42.629 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:42 smithi178 ceph-mon[33514]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T11:26:42.629 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:42 smithi178 ceph-mon[33514]: fsmap cephfs:1 {0=cephfs.smithi144.bfyvwv=up:active} 1 up:standby 2024-09-10T11:26:42.630 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:42 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.nmhrmh"}]: dispatch 2024-09-10T11:26:42.630 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:42 smithi178 ceph-mon[33514]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi178.nmhrmh=up:creating} 2024-09-10T11:26:42.630 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:42 smithi178 ceph-mon[33514]: daemon mds.cephfs.smithi178.nmhrmh is now active in filesystem cephfs as rank 1 2024-09-10T11:26:42.630 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:42 smithi178 ceph-mon[33514]: pgmap v181: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.0 KiB/s wr, 2 op/s 2024-09-10T11:26:42.648 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:42.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:42 smithi144 ceph-mon[27942]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-10T11:26:42.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:42 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/4090633200' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-09-10T11:26:42.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:42 smithi144 ceph-mon[27942]: mds.? [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] up:boot 2024-09-10T11:26:42.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:42 smithi144 ceph-mon[27942]: daemon mds.cephfs.smithi178.nmhrmh assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-10T11:26:42.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:42 smithi144 ceph-mon[27942]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-10T11:26:42.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:42 smithi144 ceph-mon[27942]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-10T11:26:42.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:42 smithi144 ceph-mon[27942]: fsmap cephfs:1 {0=cephfs.smithi144.bfyvwv=up:active} 1 up:standby 2024-09-10T11:26:42.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:42 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.nmhrmh"}]: dispatch 2024-09-10T11:26:42.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:42 smithi144 ceph-mon[27942]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi178.nmhrmh=up:creating} 2024-09-10T11:26:42.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:42 smithi144 ceph-mon[27942]: daemon mds.cephfs.smithi178.nmhrmh is now active in filesystem cephfs as rank 1 2024-09-10T11:26:42.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:42 smithi144 ceph-mon[27942]: pgmap v181: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.0 KiB/s wr, 2 op/s 2024-09-10T11:26:43.569 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:43 smithi144 ceph-mon[27942]: mds.? [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] up:active 2024-09-10T11:26:43.569 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:43 smithi144 ceph-mon[27942]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2024-09-10T11:26:43.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:43 smithi178 ceph-mon[33514]: mds.? [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] up:active 2024-09-10T11:26:43.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:43 smithi178 ceph-mon[33514]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2024-09-10T11:26:44.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:44 smithi144 ceph-mon[27942]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 7 op/s 2024-09-10T11:26:44.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.699+0000 7fa38dc1c700 1 -- 172.21.15.144:0/2938673881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa388103520 msgr2=0x7fa388105900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:44.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.699+0000 7fa38dc1c700 1 --2- 172.21.15.144:0/2938673881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa388103520 0x7fa388105900 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa37c009910 tx=0x7fa37c009c20 comp rx=0 tx=0).stop 2024-09-10T11:26:44.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.699+0000 7fa38dc1c700 1 -- 172.21.15.144:0/2938673881 shutdown_connections 2024-09-10T11:26:44.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.699+0000 7fa38dc1c700 1 --2- 172.21.15.144:0/2938673881 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa388103520 0x7fa388105900 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:44.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.699+0000 7fa38dc1c700 1 --2- 172.21.15.144:0/2938673881 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa388100c00 0x7fa388102fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:44.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.699+0000 7fa38dc1c700 1 -- 172.21.15.144:0/2938673881 >> 172.21.15.144:0/2938673881 conn(0x7fa3880fa600 msgr2=0x7fa3880fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:44.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.699+0000 7fa38dc1c700 1 -- 172.21.15.144:0/2938673881 shutdown_connections 2024-09-10T11:26:44.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.699+0000 7fa38dc1c700 1 -- 172.21.15.144:0/2938673881 wait complete. 2024-09-10T11:26:44.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.699+0000 7fa38dc1c700 1 Processor -- start 2024-09-10T11:26:44.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.700+0000 7fa38dc1c700 1 -- start start 2024-09-10T11:26:44.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.700+0000 7fa38dc1c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa388100c00 0x7fa388197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:44.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.700+0000 7fa38dc1c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa388103520 0x7fa3881980c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:44.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.700+0000 7fa38dc1c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3881986e0 con 0x7fa388103520 2024-09-10T11:26:44.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.700+0000 7fa38dc1c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa388198820 con 0x7fa388100c00 2024-09-10T11:26:44.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.700+0000 7fa386ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa388103520 0x7fa3881980c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:44.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.700+0000 7fa386ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa388103520 0x7fa3881980c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42392/0 (socket says 172.21.15.144:42392) 2024-09-10T11:26:44.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.700+0000 7fa386ffd700 1 -- 172.21.15.144:0/4238123032 learned_addr learned my addr 172.21.15.144:0/4238123032 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:44.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.700+0000 7fa3877fe700 1 --2- 172.21.15.144:0/4238123032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa388100c00 0x7fa388197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:44.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.700+0000 7fa386ffd700 1 -- 172.21.15.144:0/4238123032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa388100c00 msgr2=0x7fa388197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:44.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.700+0000 7fa386ffd700 1 --2- 172.21.15.144:0/4238123032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa388100c00 0x7fa388197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:44.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.701+0000 7fa386ffd700 1 -- 172.21.15.144:0/4238123032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa37c0095f0 con 0x7fa388103520 2024-09-10T11:26:44.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.701+0000 7fa386ffd700 1 --2- 172.21.15.144:0/4238123032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa388103520 0x7fa3881980c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa37c006010 tx=0x7fa37c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:44.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.701+0000 7fa384ff9700 1 -- 172.21.15.144:0/4238123032 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa37c01d070 con 0x7fa388103520 2024-09-10T11:26:44.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.701+0000 7fa384ff9700 1 -- 172.21.15.144:0/4238123032 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa37c00f460 con 0x7fa388103520 2024-09-10T11:26:44.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.701+0000 7fa384ff9700 1 -- 172.21.15.144:0/4238123032 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa37c021640 con 0x7fa388103520 2024-09-10T11:26:44.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.701+0000 7fa38dc1c700 1 -- 172.21.15.144:0/4238123032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38819d270 con 0x7fa388103520 2024-09-10T11:26:44.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.702+0000 7fa38dc1c700 1 -- 172.21.15.144:0/4238123032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38819d790 con 0x7fa388103520 2024-09-10T11:26:44.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.702+0000 7fa38dc1c700 1 -- 172.21.15.144:0/4238123032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3880fc1e0 con 0x7fa388103520 2024-09-10T11:26:44.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.703+0000 7fa384ff9700 1 -- 172.21.15.144:0/4238123032 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fa37c00fab0 con 0x7fa388103520 2024-09-10T11:26:44.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.703+0000 7fa384ff9700 1 --2- 172.21.15.144:0/4238123032 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa37006c6d0 0x7fa37006eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:44.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.703+0000 7fa384ff9700 1 -- 172.21.15.144:0/4238123032 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fa37c08cb00 con 0x7fa388103520 2024-09-10T11:26:44.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.703+0000 7fa3877fe700 1 --2- 172.21.15.144:0/4238123032 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa37006c6d0 0x7fa37006eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:44.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.704+0000 7fa3877fe700 1 --2- 172.21.15.144:0/4238123032 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa37006c6d0 0x7fa37006eb80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa378006fd0 tx=0x7fa378008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:44.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.708+0000 7fa384ff9700 1 -- 172.21.15.144:0/4238123032 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa37c05ae60 con 0x7fa388103520 2024-09-10T11:26:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:44 smithi178 ceph-mon[33514]: pgmap v182: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 7 op/s 2024-09-10T11:26:44.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:44.948+0000 7fa38dc1c700 1 -- 172.21.15.144:0/4238123032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fa38802cc30 con 0x7fa388103520 2024-09-10T11:26:45.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:45.392+0000 7fa384ff9700 1 -- 172.21.15.144:0/4238123032 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v9) v1 ==== 121+0+0 (secure 0 0 0) 0x7fa37c0260c0 con 0x7fa388103520 2024-09-10T11:26:45.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:45.395+0000 7fa38dc1c700 1 -- 172.21.15.144:0/4238123032 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa37006c6d0 msgr2=0x7fa37006eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:45.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:45.395+0000 7fa38dc1c700 1 --2- 172.21.15.144:0/4238123032 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa37006c6d0 0x7fa37006eb80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa378006fd0 tx=0x7fa378008040 comp rx=0 tx=0).stop 2024-09-10T11:26:45.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:45.395+0000 7fa38dc1c700 1 -- 172.21.15.144:0/4238123032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa388103520 msgr2=0x7fa3881980c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:45.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:45.395+0000 7fa38dc1c700 1 --2- 172.21.15.144:0/4238123032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa388103520 0x7fa3881980c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa37c006010 tx=0x7fa37c00be20 comp rx=0 tx=0).stop 2024-09-10T11:26:45.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:45.395+0000 7fa38dc1c700 1 -- 172.21.15.144:0/4238123032 shutdown_connections 2024-09-10T11:26:45.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:45.395+0000 7fa38dc1c700 1 --2- 172.21.15.144:0/4238123032 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fa37006c6d0 0x7fa37006eb80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:45.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:45.395+0000 7fa38dc1c700 1 --2- 172.21.15.144:0/4238123032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa388100c00 0x7fa388197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:45.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:45.395+0000 7fa38dc1c700 1 --2- 172.21.15.144:0/4238123032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa388103520 0x7fa3881980c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:45.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:45.395+0000 7fa38dc1c700 1 -- 172.21.15.144:0/4238123032 >> 172.21.15.144:0/4238123032 conn(0x7fa3880fa600 msgr2=0x7fa3880ff280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:45.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:45.395+0000 7fa38dc1c700 1 -- 172.21.15.144:0/4238123032 shutdown_connections 2024-09-10T11:26:45.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:45.395+0000 7fa38dc1c700 1 -- 172.21.15.144:0/4238123032 wait complete. 2024-09-10T11:26:45.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:45 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/4238123032' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T11:26:45.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:45 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:45.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:45 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:45.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:45 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:45.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:45 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.czjrsa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:26:45.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:45 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.czjrsa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T11:26:45.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:45 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:26:45.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:45 smithi144 ceph-mon[27942]: Deploying daemon mds.cephfs.smithi178.czjrsa on smithi178 2024-09-10T11:26:45.706 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:45 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/4238123032' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-10T11:26:45.706 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:45 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:45.706 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:45 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:45.706 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:45 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:45.707 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:45 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.czjrsa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:26:45.707 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:45 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.czjrsa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-10T11:26:45.707 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:45 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:26:45.707 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:45 smithi178 ceph-mon[33514]: Deploying daemon mds.cephfs.smithi178.czjrsa on smithi178 2024-09-10T11:26:46.005 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T11:26:46.015 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T11:26:46.015 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data true --yes-i-really-really-mean-it' 2024-09-10T11:26:46.239 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:46.678 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:46 smithi144 ceph-mon[27942]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-10T11:26:46.678 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:46 smithi144 ceph-mon[27942]: Cluster is now healthy 2024-09-10T11:26:46.678 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:46 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/4238123032' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T11:26:46.678 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:46 smithi144 ceph-mon[27942]: mds.? [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] up:boot 2024-09-10T11:26:46.678 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:46 smithi144 ceph-mon[27942]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby 2024-09-10T11:26:46.679 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:46 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jaoyvs"}]: dispatch 2024-09-10T11:26:46.679 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:46 smithi144 ceph-mon[27942]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby-replay 2024-09-10T11:26:46.679 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:46 smithi144 ceph-mon[27942]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-10T11:26:46.744 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:46 smithi178 ceph-mon[33514]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-10T11:26:46.744 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:46 smithi178 ceph-mon[33514]: Cluster is now healthy 2024-09-10T11:26:46.744 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:46 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/4238123032' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-10T11:26:46.744 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:46 smithi178 ceph-mon[33514]: mds.? [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] up:boot 2024-09-10T11:26:46.744 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:46 smithi178 ceph-mon[33514]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby 2024-09-10T11:26:46.744 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:46 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jaoyvs"}]: dispatch 2024-09-10T11:26:46.744 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:46 smithi178 ceph-mon[33514]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby-replay 2024-09-10T11:26:46.744 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:46 smithi178 ceph-mon[33514]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-10T11:26:47.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.834+0000 7fdf4dbbb700 1 -- 172.21.15.144:0/3557884420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf480ff050 msgr2=0x7fdf480ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:47.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.834+0000 7fdf4dbbb700 1 --2- 172.21.15.144:0/3557884420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf480ff050 0x7fdf480ff460 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fdf38009910 tx=0x7fdf38009c20 comp rx=0 tx=0).stop 2024-09-10T11:26:47.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.834+0000 7fdf4dbbb700 1 -- 172.21.15.144:0/3557884420 shutdown_connections 2024-09-10T11:26:47.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.834+0000 7fdf4dbbb700 1 --2- 172.21.15.144:0/3557884420 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf48100250 0x7fdf481006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:47.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.834+0000 7fdf4dbbb700 1 --2- 172.21.15.144:0/3557884420 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf480ff050 0x7fdf480ff460 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:47.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.834+0000 7fdf4dbbb700 1 -- 172.21.15.144:0/3557884420 >> 172.21.15.144:0/3557884420 conn(0x7fdf480fa5c0 msgr2=0x7fdf480fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:47.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.834+0000 7fdf4dbbb700 1 -- 172.21.15.144:0/3557884420 shutdown_connections 2024-09-10T11:26:47.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.834+0000 7fdf4dbbb700 1 -- 172.21.15.144:0/3557884420 wait complete. 2024-09-10T11:26:47.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.834+0000 7fdf4dbbb700 1 Processor -- start 2024-09-10T11:26:47.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.835+0000 7fdf4dbbb700 1 -- start start 2024-09-10T11:26:47.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.835+0000 7fdf4dbbb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf480ff050 0x7fdf481948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:47.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.835+0000 7fdf4dbbb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf48100250 0x7fdf48194de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:47.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.835+0000 7fdf4dbbb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf48195400 con 0x7fdf480ff050 2024-09-10T11:26:47.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.835+0000 7fdf4dbbb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf48195540 con 0x7fdf48100250 2024-09-10T11:26:47.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.835+0000 7fdf477fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf480ff050 0x7fdf481948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:47.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.835+0000 7fdf477fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf480ff050 0x7fdf481948a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42422/0 (socket says 172.21.15.144:42422) 2024-09-10T11:26:47.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.835+0000 7fdf477fe700 1 -- 172.21.15.144:0/467188644 learned_addr learned my addr 172.21.15.144:0/467188644 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:47.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.835+0000 7fdf46ffd700 1 --2- 172.21.15.144:0/467188644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf48100250 0x7fdf48194de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:47.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.835+0000 7fdf477fe700 1 -- 172.21.15.144:0/467188644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf48100250 msgr2=0x7fdf48194de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:47.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.835+0000 7fdf477fe700 1 --2- 172.21.15.144:0/467188644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf48100250 0x7fdf48194de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:47.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.835+0000 7fdf477fe700 1 -- 172.21.15.144:0/467188644 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf380095f0 con 0x7fdf480ff050 2024-09-10T11:26:47.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.835+0000 7fdf477fe700 1 --2- 172.21.15.144:0/467188644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf480ff050 0x7fdf481948a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800bda0 tx=0x7fdf3800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:47.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.835+0000 7fdf44ff9700 1 -- 172.21.15.144:0/467188644 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf3801d070 con 0x7fdf480ff050 2024-09-10T11:26:47.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.836+0000 7fdf44ff9700 1 -- 172.21.15.144:0/467188644 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdf3800f460 con 0x7fdf480ff050 2024-09-10T11:26:47.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.836+0000 7fdf4dbbb700 1 -- 172.21.15.144:0/467188644 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf48199f90 con 0x7fdf480ff050 2024-09-10T11:26:47.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.836+0000 7fdf44ff9700 1 -- 172.21.15.144:0/467188644 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf38021620 con 0x7fdf480ff050 2024-09-10T11:26:47.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.836+0000 7fdf4dbbb700 1 -- 172.21.15.144:0/467188644 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf4819a400 con 0x7fdf480ff050 2024-09-10T11:26:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.837+0000 7fdf4dbbb700 1 -- 172.21.15.144:0/467188644 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf48107ba0 con 0x7fdf480ff050 2024-09-10T11:26:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.837+0000 7fdf44ff9700 1 -- 172.21.15.144:0/467188644 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fdf3800f5d0 con 0x7fdf480ff050 2024-09-10T11:26:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.837+0000 7fdf44ff9700 1 --2- 172.21.15.144:0/467188644 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdf3006c680 0x7fdf3006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.838+0000 7fdf46ffd700 1 --2- 172.21.15.144:0/467188644 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdf3006c680 0x7fdf3006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.838+0000 7fdf44ff9700 1 -- 172.21.15.144:0/467188644 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5296+0+0 (secure 0 0 0) 0x7fdf3808ca60 con 0x7fdf480ff050 2024-09-10T11:26:47.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.838+0000 7fdf46ffd700 1 --2- 172.21.15.144:0/467188644 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdf3006c680 0x7fdf3006eb30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c007c60 tx=0x7fdf3c0073d0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:47.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:47.842+0000 7fdf44ff9700 1 -- 172.21.15.144:0/467188644 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdf3805ac80 con 0x7fdf480ff050 2024-09-10T11:26:48.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:48.110+0000 7fdf4dbbb700 1 -- 172.21.15.144:0/467188644 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdf48107d90 con 0x7fdf480ff050 2024-09-10T11:26:48.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:48.351+0000 7fdf44ff9700 1 -- 172.21.15.144:0/467188644 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]=0 inline data enabled v11) v1 ==== 168+0+0 (secure 0 0 0) 0x7fdf3805a810 con 0x7fdf480ff050 2024-09-10T11:26:48.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:48.353+0000 7fdf4dbbb700 1 -- 172.21.15.144:0/467188644 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdf3006c680 msgr2=0x7fdf3006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:48.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:48.353+0000 7fdf4dbbb700 1 --2- 172.21.15.144:0/467188644 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdf3006c680 0x7fdf3006eb30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c007c60 tx=0x7fdf3c0073d0 comp rx=0 tx=0).stop 2024-09-10T11:26:48.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:48.353+0000 7fdf4dbbb700 1 -- 172.21.15.144:0/467188644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf480ff050 msgr2=0x7fdf481948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:48.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:48.353+0000 7fdf4dbbb700 1 --2- 172.21.15.144:0/467188644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf480ff050 0x7fdf481948a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fdf3800bda0 tx=0x7fdf3800bdd0 comp rx=0 tx=0).stop 2024-09-10T11:26:48.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:48.353+0000 7fdf4dbbb700 1 -- 172.21.15.144:0/467188644 shutdown_connections 2024-09-10T11:26:48.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:48.353+0000 7fdf4dbbb700 1 --2- 172.21.15.144:0/467188644 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdf3006c680 0x7fdf3006eb30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:48.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:48.353+0000 7fdf4dbbb700 1 --2- 172.21.15.144:0/467188644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf480ff050 0x7fdf481948a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:48.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:48.353+0000 7fdf4dbbb700 1 --2- 172.21.15.144:0/467188644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf48100250 0x7fdf48194de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:48.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:48.353+0000 7fdf4dbbb700 1 -- 172.21.15.144:0/467188644 >> 172.21.15.144:0/467188644 conn(0x7fdf480fa5c0 msgr2=0x7fdf48103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:48.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:48.354+0000 7fdf4dbbb700 1 -- 172.21.15.144:0/467188644 shutdown_connections 2024-09-10T11:26:48.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:48.354+0000 7fdf4dbbb700 1 -- 172.21.15.144:0/467188644 wait complete. 2024-09-10T11:26:48.356 INFO:teuthology.orchestra.run.smithi144.stderr:inline data enabled 2024-09-10T11:26:48.909 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:48.909 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:48.909 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:48.909 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:48.909 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:48.909 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:26:48.909 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 601 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-10T11:26:48.910 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:48.910 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:26:48.910 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/467188644' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T11:26:48.910 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-09-10T11:26:48.910 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: mds.? [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] up:boot 2024-09-10T11:26:48.910 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/467188644' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T11:26:48.910 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby-replay 1 up:standby 2024-09-10T11:26:48.910 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.czjrsa"}]: dispatch 2024-09-10T11:26:48.910 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:48 smithi144 ceph-mon[27942]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:26:48.911 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T11:26:48.921 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T11:26:48.921 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T11:26:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:26:49.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 601 B/s rd, 2.2 KiB/s wr, 7 op/s 2024-09-10T11:26:49.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:49.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:26:49.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/467188644' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T11:26:49.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: Health check failed: 1 filesystem with deprecated feature inline_data (FS_INLINE_DATA_DEPRECATED) 2024-09-10T11:26:49.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: mds.? [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] up:boot 2024-09-10T11:26:49.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/467188644' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "true", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T11:26:49.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby-replay 1 up:standby 2024-09-10T11:26:49.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.czjrsa"}]: dispatch 2024-09-10T11:26:49.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:48 smithi178 ceph-mon[33514]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:26:49.688 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:51.144 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:50 smithi144 ceph-mon[27942]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-10T11:26:51.144 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:50 smithi144 ceph-mon[27942]: mds.? [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] up:standby-replay 2024-09-10T11:26:51.144 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:50 smithi144 ceph-mon[27942]: mds.? [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] up:active 2024-09-10T11:26:51.144 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:50 smithi144 ceph-mon[27942]: Dropping low affinity active daemon mds.cephfs.smithi144.bfyvwv in favor of higher affinity standby. 2024-09-10T11:26:51.144 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:50 smithi144 ceph-mon[27942]: Replacing daemon mds.cephfs.smithi144.bfyvwv as rank 0 with standby daemon mds.cephfs.smithi144.jaoyvs 2024-09-10T11:26:51.145 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:50 smithi144 ceph-mon[27942]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T11:26:51.145 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:50 smithi144 ceph-mon[27942]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:26:51.145 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:50 smithi144 ceph-mon[27942]: osdmap e47: 6 total, 6 up, 6 in 2024-09-10T11:26:51.145 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:50 smithi144 ceph-mon[27942]: fsmap cephfs:2/2 {0=cephfs.smithi144.jaoyvs=up:replay,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby-replay 2024-09-10T11:26:51.145 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:50 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:51.145 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:50 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:51.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.234+0000 7fdb3feb7700 1 -- 172.21.15.144:0/1314340705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb38100190 msgr2=0x7fdb381005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:51.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.234+0000 7fdb3feb7700 1 --2- 172.21.15.144:0/1314340705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb38100190 0x7fdb381005a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdb3000b0a0 tx=0x7fdb3000b3b0 comp rx=0 tx=0).stop 2024-09-10T11:26:51.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.234+0000 7fdb3feb7700 1 -- 172.21.15.144:0/1314340705 shutdown_connections 2024-09-10T11:26:51.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.234+0000 7fdb3feb7700 1 --2- 172.21.15.144:0/1314340705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb38101390 0x7fdb381017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:51.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.234+0000 7fdb3feb7700 1 --2- 172.21.15.144:0/1314340705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb38100190 0x7fdb381005a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:51.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.234+0000 7fdb3feb7700 1 -- 172.21.15.144:0/1314340705 >> 172.21.15.144:0/1314340705 conn(0x7fdb380fb720 msgr2=0x7fdb380fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:51.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.234+0000 7fdb3feb7700 1 -- 172.21.15.144:0/1314340705 shutdown_connections 2024-09-10T11:26:51.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.234+0000 7fdb3feb7700 1 -- 172.21.15.144:0/1314340705 wait complete. 2024-09-10T11:26:51.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.235+0000 7fdb3feb7700 1 Processor -- start 2024-09-10T11:26:51.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.235+0000 7fdb3feb7700 1 -- start start 2024-09-10T11:26:51.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.235+0000 7fdb3feb7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb38100190 0x7fdb38195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:51.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.235+0000 7fdb3feb7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb38101390 0x7fdb38195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:51.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.235+0000 7fdb3feb7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb381964f0 con 0x7fdb38100190 2024-09-10T11:26:51.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.235+0000 7fdb3feb7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb38196630 con 0x7fdb38101390 2024-09-10T11:26:51.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.235+0000 7fdb3dc53700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb38100190 0x7fdb38195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:51.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.235+0000 7fdb3dc53700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb38100190 0x7fdb38195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42462/0 (socket says 172.21.15.144:42462) 2024-09-10T11:26:51.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.235+0000 7fdb3dc53700 1 -- 172.21.15.144:0/482345110 learned_addr learned my addr 172.21.15.144:0/482345110 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:51.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.235+0000 7fdb3d452700 1 --2- 172.21.15.144:0/482345110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb38101390 0x7fdb38195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:51.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.235+0000 7fdb3dc53700 1 -- 172.21.15.144:0/482345110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb38101390 msgr2=0x7fdb38195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:51.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.235+0000 7fdb3dc53700 1 --2- 172.21.15.144:0/482345110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb38101390 0x7fdb38195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:51.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.235+0000 7fdb3dc53700 1 -- 172.21.15.144:0/482345110 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb30009d00 con 0x7fdb38100190 2024-09-10T11:26:51.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.235+0000 7fdb3dc53700 1 --2- 172.21.15.144:0/482345110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb38100190 0x7fdb38195990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdb300077b0 tx=0x7fdb30009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:51.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.236+0000 7fdb36ffd700 1 -- 172.21.15.144:0/482345110 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb3000e040 con 0x7fdb38100190 2024-09-10T11:26:51.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.236+0000 7fdb36ffd700 1 -- 172.21.15.144:0/482345110 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdb300044f0 con 0x7fdb38100190 2024-09-10T11:26:51.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.236+0000 7fdb3feb7700 1 -- 172.21.15.144:0/482345110 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3819b080 con 0x7fdb38100190 2024-09-10T11:26:51.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.236+0000 7fdb36ffd700 1 -- 172.21.15.144:0/482345110 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb3001d9e0 con 0x7fdb38100190 2024-09-10T11:26:51.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.236+0000 7fdb3feb7700 1 -- 172.21.15.144:0/482345110 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3819b5f0 con 0x7fdb38100190 2024-09-10T11:26:51.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.236+0000 7fdb3feb7700 1 -- 172.21.15.144:0/482345110 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb38108ce0 con 0x7fdb38100190 2024-09-10T11:26:51.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.237+0000 7fdb36ffd700 1 -- 172.21.15.144:0/482345110 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fdb30004040 con 0x7fdb38100190 2024-09-10T11:26:51.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.238+0000 7fdb36ffd700 1 --2- 172.21.15.144:0/482345110 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdb2006c680 0x7fdb2006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:51.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.238+0000 7fdb3d452700 1 --2- 172.21.15.144:0/482345110 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdb2006c680 0x7fdb2006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:51.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.238+0000 7fdb36ffd700 1 -- 172.21.15.144:0/482345110 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fdb3008d260 con 0x7fdb38100190 2024-09-10T11:26:51.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.238+0000 7fdb3d452700 1 --2- 172.21.15.144:0/482345110 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdb2006c680 0x7fdb2006eb30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c005950 tx=0x7fdb2c0058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:51.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.242+0000 7fdb36ffd700 1 -- 172.21.15.144:0/482345110 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdb3005b420 con 0x7fdb38100190 2024-09-10T11:26:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:50 smithi178 ceph-mon[33514]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-09-10T11:26:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:50 smithi178 ceph-mon[33514]: mds.? [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] up:standby-replay 2024-09-10T11:26:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:50 smithi178 ceph-mon[33514]: mds.? [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] up:active 2024-09-10T11:26:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:50 smithi178 ceph-mon[33514]: Dropping low affinity active daemon mds.cephfs.smithi144.bfyvwv in favor of higher affinity standby. 2024-09-10T11:26:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:50 smithi178 ceph-mon[33514]: Replacing daemon mds.cephfs.smithi144.bfyvwv as rank 0 with standby daemon mds.cephfs.smithi144.jaoyvs 2024-09-10T11:26:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:50 smithi178 ceph-mon[33514]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T11:26:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:50 smithi178 ceph-mon[33514]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:26:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:50 smithi178 ceph-mon[33514]: osdmap e47: 6 total, 6 up, 6 in 2024-09-10T11:26:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:50 smithi178 ceph-mon[33514]: fsmap cephfs:2/2 {0=cephfs.smithi144.jaoyvs=up:replay,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby-replay 2024-09-10T11:26:51.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:50 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:51.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:50 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:51.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.488+0000 7fdb3feb7700 1 -- 172.21.15.144:0/482345110 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdb3819c030 con 0x7fdb38100190 2024-09-10T11:26:51.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.489+0000 7fdb36ffd700 1 -- 172.21.15.144:0/482345110 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1855 (secure 0 0 0) 0x7fdb3005afb0 con 0x7fdb38100190 2024-09-10T11:26:51.491 INFO:teuthology.orchestra.run.smithi144.stdout:e15 2024-09-10T11:26:51.491 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:26:51.491 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:26:51.491 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:26:51.491 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:26:51.491 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:26:51.491 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:26:51.492 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 15 2024-09-10T11:26:51.492 INFO:teuthology.orchestra.run.smithi144.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T11:26:51.492 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:26:51.492 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:26:50.935716+0000 2024-09-10T11:26:51.492 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:26:51.492 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:26:51.492 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:26:51.492 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:26:51.492 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:26:51.492 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:26:51.492 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:26:51.492 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 47 2024-09-10T11:26:51.493 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:26:51.493 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:26:51.493 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:26:51.493 INFO:teuthology.orchestra.run.smithi144.stdout:up {0=14488,1=24267} 2024-09-10T11:26:51.493 INFO:teuthology.orchestra.run.smithi144.stdout:failed 2024-09-10T11:26:51.493 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:26:51.493 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:26:51.493 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:26:51.493 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:26:51.493 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:26:51.493 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:26:51.494 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:26:51.494 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:26:51.494 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{0:14488} state up:resolve seq 3 join_fscid=1 addr [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:26:51.494 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:26:51.494 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{1:24273} state up:standby-replay seq 1 addr [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:26:51.494 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:26:51.494 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:26:51.494 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:26:51.494 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:26:51.494 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:14498} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/1752150537,v1:172.21.15.144:6827/1752150537] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:26:51.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.491+0000 7fdb3feb7700 1 -- 172.21.15.144:0/482345110 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdb2006c680 msgr2=0x7fdb2006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:51.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.491+0000 7fdb3feb7700 1 --2- 172.21.15.144:0/482345110 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdb2006c680 0x7fdb2006eb30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c005950 tx=0x7fdb2c0058e0 comp rx=0 tx=0).stop 2024-09-10T11:26:51.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.491+0000 7fdb3feb7700 1 -- 172.21.15.144:0/482345110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb38100190 msgr2=0x7fdb38195990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:51.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.491+0000 7fdb3feb7700 1 --2- 172.21.15.144:0/482345110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb38100190 0x7fdb38195990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdb300077b0 tx=0x7fdb30009810 comp rx=0 tx=0).stop 2024-09-10T11:26:51.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.491+0000 7fdb3feb7700 1 -- 172.21.15.144:0/482345110 shutdown_connections 2024-09-10T11:26:51.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.491+0000 7fdb3feb7700 1 --2- 172.21.15.144:0/482345110 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fdb2006c680 0x7fdb2006eb30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:51.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.491+0000 7fdb3feb7700 1 --2- 172.21.15.144:0/482345110 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb38100190 0x7fdb38195990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:51.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.491+0000 7fdb3feb7700 1 --2- 172.21.15.144:0/482345110 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb38101390 0x7fdb38195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:51.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.491+0000 7fdb3feb7700 1 -- 172.21.15.144:0/482345110 >> 172.21.15.144:0/482345110 conn(0x7fdb380fb720 msgr2=0x7fdb381045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:51.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.491+0000 7fdb3feb7700 1 -- 172.21.15.144:0/482345110 shutdown_connections 2024-09-10T11:26:51.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:51.491+0000 7fdb3feb7700 1 -- 172.21.15.144:0/482345110 wait complete. 2024-09-10T11:26:51.496 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 15 2024-09-10T11:26:52.168 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:51 smithi144 ceph-mon[27942]: mds.? [v2:172.21.15.144:6826/1752150537,v1:172.21.15.144:6827/1752150537] up:boot 2024-09-10T11:26:52.168 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:51 smithi144 ceph-mon[27942]: mds.? [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] up:resolve 2024-09-10T11:26:52.168 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:51 smithi144 ceph-mon[27942]: fsmap cephfs:2/2 {0=cephfs.smithi144.jaoyvs=up:resolve,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby-replay 1 up:standby 2024-09-10T11:26:52.168 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:51 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:26:52.168 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:51 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/482345110' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:26:52.189 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-09-10T11:26:52.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:51 smithi178 ceph-mon[33514]: mds.? [v2:172.21.15.144:6826/1752150537,v1:172.21.15.144:6827/1752150537] up:boot 2024-09-10T11:26:52.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:51 smithi178 ceph-mon[33514]: mds.? [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] up:resolve 2024-09-10T11:26:52.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:51 smithi178 ceph-mon[33514]: fsmap cephfs:2/2 {0=cephfs.smithi144.jaoyvs=up:resolve,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby-replay 1 up:standby 2024-09-10T11:26:52.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:51 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:26:52.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:51 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/482345110' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:26:52.734 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:53.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:52 smithi144 ceph-mon[27942]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-10T11:26:53.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:52 smithi144 ceph-mon[27942]: mds.? [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] up:reconnect 2024-09-10T11:26:53.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:52 smithi144 ceph-mon[27942]: mds.? [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] up:standby-replay 2024-09-10T11:26:53.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:52 smithi144 ceph-mon[27942]: fsmap cephfs:2/2 {0=cephfs.smithi144.jaoyvs=up:reconnect,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby-replay 1 up:standby 2024-09-10T11:26:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:52 smithi178 ceph-mon[33514]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.4 KiB/s rd, 1.8 KiB/s wr, 7 op/s 2024-09-10T11:26:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:52 smithi178 ceph-mon[33514]: mds.? [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] up:reconnect 2024-09-10T11:26:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:52 smithi178 ceph-mon[33514]: mds.? [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] up:standby-replay 2024-09-10T11:26:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:52 smithi178 ceph-mon[33514]: fsmap cephfs:2/2 {0=cephfs.smithi144.jaoyvs=up:reconnect,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby-replay 1 up:standby 2024-09-10T11:26:54.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:53 smithi144 ceph-mon[27942]: mds.? [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] up:rejoin 2024-09-10T11:26:54.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:53 smithi144 ceph-mon[27942]: fsmap cephfs:2/2 {0=cephfs.smithi144.jaoyvs=up:rejoin,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby-replay 1 up:standby 2024-09-10T11:26:54.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:53 smithi144 ceph-mon[27942]: daemon mds.cephfs.smithi144.jaoyvs is now active in filesystem cephfs as rank 0 2024-09-10T11:26:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:53 smithi178 ceph-mon[33514]: mds.? [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] up:rejoin 2024-09-10T11:26:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:53 smithi178 ceph-mon[33514]: fsmap cephfs:2/2 {0=cephfs.smithi144.jaoyvs=up:rejoin,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby-replay 1 up:standby 2024-09-10T11:26:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:53 smithi178 ceph-mon[33514]: daemon mds.cephfs.smithi144.jaoyvs is now active in filesystem cephfs as rank 0 2024-09-10T11:26:55.026 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:54 smithi144 ceph-mon[27942]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 716 B/s wr, 5 op/s 2024-09-10T11:26:55.026 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:54 smithi144 ceph-mon[27942]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T11:26:55.027 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:54 smithi144 ceph-mon[27942]: mds.? [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] up:active 2024-09-10T11:26:55.027 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:54 smithi144 ceph-mon[27942]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby-replay 1 up:standby 2024-09-10T11:26:55.027 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:54 smithi144 ceph-mon[27942]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:26:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:54 smithi178 ceph-mon[33514]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.6 KiB/s rd, 716 B/s wr, 5 op/s 2024-09-10T11:26:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:54 smithi178 ceph-mon[33514]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T11:26:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:54 smithi178 ceph-mon[33514]: mds.? [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] up:active 2024-09-10T11:26:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:54 smithi178 ceph-mon[33514]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 1 up:standby-replay 1 up:standby 2024-09-10T11:26:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:54 smithi178 ceph-mon[33514]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:26:56.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.168+0000 7f1f5dd80700 1 -- 172.21.15.144:0/1378622524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f580ffa20 msgr2=0x7f1f580ffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:56.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.168+0000 7f1f5dd80700 1 --2- 172.21.15.144:0/1378622524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f580ffa20 0x7f1f580ffe70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00aa20 tx=0x7f1f4c00ad30 comp rx=0 tx=0).stop 2024-09-10T11:26:56.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.168+0000 7f1f5dd80700 1 -- 172.21.15.144:0/1378622524 shutdown_connections 2024-09-10T11:26:56.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.168+0000 7f1f5dd80700 1 --2- 172.21.15.144:0/1378622524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f580ffa20 0x7f1f580ffe70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:56.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.168+0000 7f1f5dd80700 1 --2- 172.21.15.144:0/1378622524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f580fe820 0x7f1f580fec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:56.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.168+0000 7f1f5dd80700 1 -- 172.21.15.144:0/1378622524 >> 172.21.15.144:0/1378622524 conn(0x7f1f580f9db0 msgr2=0x7f1f580fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:56.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.169+0000 7f1f5dd80700 1 -- 172.21.15.144:0/1378622524 shutdown_connections 2024-09-10T11:26:56.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.169+0000 7f1f5dd80700 1 -- 172.21.15.144:0/1378622524 wait complete. 2024-09-10T11:26:56.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.169+0000 7f1f5dd80700 1 Processor -- start 2024-09-10T11:26:56.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.169+0000 7f1f5dd80700 1 -- start start 2024-09-10T11:26:56.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.169+0000 7f1f5dd80700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f580fe820 0x7f1f581959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:56.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.169+0000 7f1f5dd80700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f580ffa20 0x7f1f58195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:56.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.169+0000 7f1f5dd80700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f58196510 con 0x7f1f580fe820 2024-09-10T11:26:56.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.169+0000 7f1f5dd80700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f58196650 con 0x7f1f580ffa20 2024-09-10T11:26:56.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.170+0000 7f1f577fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f580fe820 0x7f1f581959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:56.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.170+0000 7f1f577fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f580fe820 0x7f1f581959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:46158/0 (socket says 172.21.15.144:46158) 2024-09-10T11:26:56.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.170+0000 7f1f577fe700 1 -- 172.21.15.144:0/3619008924 learned_addr learned my addr 172.21.15.144:0/3619008924 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:56.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.170+0000 7f1f56ffd700 1 --2- 172.21.15.144:0/3619008924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f580ffa20 0x7f1f58195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:56.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.170+0000 7f1f577fe700 1 -- 172.21.15.144:0/3619008924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f580ffa20 msgr2=0x7f1f58195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:56.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.170+0000 7f1f577fe700 1 --2- 172.21.15.144:0/3619008924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f580ffa20 0x7f1f58195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:56.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.170+0000 7f1f577fe700 1 -- 172.21.15.144:0/3619008924 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f4c00a700 con 0x7f1f580fe820 2024-09-10T11:26:56.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.170+0000 7f1f577fe700 1 --2- 172.21.15.144:0/3619008924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f580fe820 0x7f1f581959b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800d900 tx=0x7f1f4800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:56.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.170+0000 7f1f54ff9700 1 -- 172.21.15.144:0/3619008924 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f480041f0 con 0x7f1f580fe820 2024-09-10T11:26:56.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.170+0000 7f1f54ff9700 1 -- 172.21.15.144:0/3619008924 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1f48004350 con 0x7f1f580fe820 2024-09-10T11:26:56.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.170+0000 7f1f5dd80700 1 -- 172.21.15.144:0/3619008924 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5819b100 con 0x7f1f580fe820 2024-09-10T11:26:56.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.170+0000 7f1f54ff9700 1 -- 172.21.15.144:0/3619008924 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f48003d20 con 0x7f1f580fe820 2024-09-10T11:26:56.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.170+0000 7f1f5dd80700 1 -- 172.21.15.144:0/3619008924 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5819b6d0 con 0x7f1f580fe820 2024-09-10T11:26:56.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.171+0000 7f1f5dd80700 1 -- 172.21.15.144:0/3619008924 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f5818f9d0 con 0x7f1f580fe820 2024-09-10T11:26:56.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.172+0000 7f1f54ff9700 1 -- 172.21.15.144:0/3619008924 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f1f48009730 con 0x7f1f580fe820 2024-09-10T11:26:56.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.172+0000 7f1f54ff9700 1 --2- 172.21.15.144:0/3619008924 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f1f4006c680 0x7f1f4006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:56.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.172+0000 7f1f54ff9700 1 -- 172.21.15.144:0/3619008924 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f1f4808b730 con 0x7f1f580fe820 2024-09-10T11:26:56.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.172+0000 7f1f56ffd700 1 --2- 172.21.15.144:0/3619008924 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f1f4006c680 0x7f1f4006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:56.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.172+0000 7f1f56ffd700 1 --2- 172.21.15.144:0/3619008924 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f1f4006c680 0x7f1f4006eb30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c6d0 tx=0x7f1f4c005de0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:56.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.176+0000 7f1f54ff9700 1 -- 172.21.15.144:0/3619008924 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1f480598f0 con 0x7f1f580fe820 2024-09-10T11:26:56.467 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.465+0000 7f1f5dd80700 1 -- 172.21.15.144:0/3619008924 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f5802d0c0 con 0x7f1f580fe820 2024-09-10T11:26:56.467 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.466+0000 7f1f54ff9700 1 -- 172.21.15.144:0/3619008924 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4782 (secure 0 0 0) 0x7f1f48003e80 con 0x7f1f580fe820 2024-09-10T11:26:56.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.468+0000 7f1f5dd80700 1 -- 172.21.15.144:0/3619008924 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f1f4006c680 msgr2=0x7f1f4006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:56.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.468+0000 7f1f5dd80700 1 --2- 172.21.15.144:0/3619008924 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f1f4006c680 0x7f1f4006eb30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00c6d0 tx=0x7f1f4c005de0 comp rx=0 tx=0).stop 2024-09-10T11:26:56.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.468+0000 7f1f5dd80700 1 -- 172.21.15.144:0/3619008924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f580fe820 msgr2=0x7f1f581959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:56.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.468+0000 7f1f5dd80700 1 --2- 172.21.15.144:0/3619008924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f580fe820 0x7f1f581959b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1f4800d900 tx=0x7f1f4800dc10 comp rx=0 tx=0).stop 2024-09-10T11:26:56.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.468+0000 7f1f5dd80700 1 -- 172.21.15.144:0/3619008924 shutdown_connections 2024-09-10T11:26:56.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.468+0000 7f1f5dd80700 1 --2- 172.21.15.144:0/3619008924 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f1f4006c680 0x7f1f4006eb30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:56.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.468+0000 7f1f5dd80700 1 --2- 172.21.15.144:0/3619008924 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1f580fe820 0x7f1f581959b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:56.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.468+0000 7f1f5dd80700 1 --2- 172.21.15.144:0/3619008924 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1f580ffa20 0x7f1f58195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:56.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.468+0000 7f1f5dd80700 1 -- 172.21.15.144:0/3619008924 >> 172.21.15.144:0/3619008924 conn(0x7f1f580f9db0 msgr2=0x7f1f58102e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:56.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.468+0000 7f1f5dd80700 1 -- 172.21.15.144:0/3619008924 shutdown_connections 2024-09-10T11:26:56.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:56.468+0000 7f1f5dd80700 1 -- 172.21.15.144:0/3619008924 wait complete. 2024-09-10T11:26:56.470 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 19 2024-09-10T11:26:56.480 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:26:57.029 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:56 smithi144 ceph-mon[27942]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.1 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T11:26:57.029 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:57.030 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:57.030 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:26:57.030 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:26:57.030 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:57.030 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:56 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/3619008924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T11:26:57.030 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:56 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:26:57.050 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-09-10T11:26:57.264 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:26:57.270 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:56 smithi178 ceph-mon[33514]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.1 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T11:26:57.270 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:56 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:57.270 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:56 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:57.270 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:56 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:26:57.270 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:56 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:26:57.270 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:56 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:57.271 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:56 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/3619008924' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T11:26:57.271 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:56 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:26:58.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:58 smithi178 ceph-mon[33514]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T11:26:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:58 smithi144 ceph-mon[27942]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.7 KiB/s rd, 0 B/s wr, 10 op/s 2024-09-10T11:26:59.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.336+0000 7ff515bce700 1 -- 172.21.15.144:0/1328176099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff510100c30 msgr2=0x7ff510103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:59.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.336+0000 7ff515bce700 1 --2- 172.21.15.144:0/1328176099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff510100c30 0x7ff510103010 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff500009910 tx=0x7ff500009c20 comp rx=0 tx=0).stop 2024-09-10T11:26:59.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.336+0000 7ff515bce700 1 -- 172.21.15.144:0/1328176099 shutdown_connections 2024-09-10T11:26:59.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.336+0000 7ff515bce700 1 --2- 172.21.15.144:0/1328176099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff510103550 0x7ff510105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:59.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.336+0000 7ff515bce700 1 --2- 172.21.15.144:0/1328176099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff510100c30 0x7ff510103010 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:59.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.336+0000 7ff515bce700 1 -- 172.21.15.144:0/1328176099 >> 172.21.15.144:0/1328176099 conn(0x7ff5100fa630 msgr2=0x7ff5100fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:59.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.336+0000 7ff515bce700 1 -- 172.21.15.144:0/1328176099 shutdown_connections 2024-09-10T11:26:59.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.336+0000 7ff515bce700 1 -- 172.21.15.144:0/1328176099 wait complete. 2024-09-10T11:26:59.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.337+0000 7ff515bce700 1 Processor -- start 2024-09-10T11:26:59.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.337+0000 7ff515bce700 1 -- start start 2024-09-10T11:26:59.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.337+0000 7ff515bce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff510100c30 0x7ff510100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:59.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.337+0000 7ff515bce700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff510103550 0x7ff5100febe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:59.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.337+0000 7ff515bce700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100ff120 con 0x7ff510100c30 2024-09-10T11:26:59.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.337+0000 7ff515bce700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5100ff260 con 0x7ff510103550 2024-09-10T11:26:59.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.338+0000 7ff50f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff510100c30 0x7ff510100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:59.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.338+0000 7ff50f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff510100c30 0x7ff510100590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:46168/0 (socket says 172.21.15.144:46168) 2024-09-10T11:26:59.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.338+0000 7ff50f7fe700 1 -- 172.21.15.144:0/4187209548 learned_addr learned my addr 172.21.15.144:0/4187209548 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:26:59.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.338+0000 7ff50effd700 1 --2- 172.21.15.144:0/4187209548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff510103550 0x7ff5100febe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:59.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.338+0000 7ff50f7fe700 1 -- 172.21.15.144:0/4187209548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff510103550 msgr2=0x7ff5100febe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:59.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.338+0000 7ff50f7fe700 1 --2- 172.21.15.144:0/4187209548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff510103550 0x7ff5100febe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:59.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.338+0000 7ff50f7fe700 1 -- 172.21.15.144:0/4187209548 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5000095f0 con 0x7ff510100c30 2024-09-10T11:26:59.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.338+0000 7ff50f7fe700 1 --2- 172.21.15.144:0/4187209548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff510100c30 0x7ff510100590 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff500005b40 tx=0x7ff50000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:59.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.338+0000 7ff50cff9700 1 -- 172.21.15.144:0/4187209548 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff50001d070 con 0x7ff510100c30 2024-09-10T11:26:59.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.338+0000 7ff50cff9700 1 -- 172.21.15.144:0/4187209548 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff50000f460 con 0x7ff510100c30 2024-09-10T11:26:59.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.338+0000 7ff515bce700 1 -- 172.21.15.144:0/4187209548 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5100ff4e0 con 0x7ff510100c30 2024-09-10T11:26:59.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.338+0000 7ff515bce700 1 -- 172.21.15.144:0/4187209548 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5100ff9d0 con 0x7ff510100c30 2024-09-10T11:26:59.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.338+0000 7ff50cff9700 1 -- 172.21.15.144:0/4187209548 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff500021620 con 0x7ff510100c30 2024-09-10T11:26:59.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.339+0000 7ff515bce700 1 -- 172.21.15.144:0/4187209548 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4fc005320 con 0x7ff510100c30 2024-09-10T11:26:59.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.340+0000 7ff50cff9700 1 -- 172.21.15.144:0/4187209548 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7ff500003a60 con 0x7ff510100c30 2024-09-10T11:26:59.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.340+0000 7ff50cff9700 1 --2- 172.21.15.144:0/4187209548 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff4f806c680 0x7ff4f806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:26:59.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.340+0000 7ff50cff9700 1 -- 172.21.15.144:0/4187209548 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7ff50008ca30 con 0x7ff510100c30 2024-09-10T11:26:59.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.340+0000 7ff50effd700 1 --2- 172.21.15.144:0/4187209548 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff4f806c680 0x7ff4f806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:26:59.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.340+0000 7ff50effd700 1 --2- 172.21.15.144:0/4187209548 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff4f806c680 0x7ff4f806eb30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff504007c60 tx=0x7ff5040073d0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:26:59.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.344+0000 7ff50cff9700 1 -- 172.21.15.144:0/4187209548 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff50005abf0 con 0x7ff510100c30 2024-09-10T11:26:59.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.633+0000 7ff515bce700 1 -- 172.21.15.144:0/4187209548 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7ff4fc005190 con 0x7ff510100c30 2024-09-10T11:26:59.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.634+0000 7ff50cff9700 1 -- 172.21.15.144:0/4187209548 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v19) v1 ==== 78+0+83 (secure 0 0 0) 0x7ff50005a780 con 0x7ff510100c30 2024-09-10T11:26:59.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.636+0000 7ff515bce700 1 -- 172.21.15.144:0/4187209548 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff4f806c680 msgr2=0x7ff4f806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:59.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.636+0000 7ff515bce700 1 --2- 172.21.15.144:0/4187209548 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff4f806c680 0x7ff4f806eb30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ff504007c60 tx=0x7ff5040073d0 comp rx=0 tx=0).stop 2024-09-10T11:26:59.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.636+0000 7ff515bce700 1 -- 172.21.15.144:0/4187209548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff510100c30 msgr2=0x7ff510100590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:26:59.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.636+0000 7ff515bce700 1 --2- 172.21.15.144:0/4187209548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff510100c30 0x7ff510100590 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff500005b40 tx=0x7ff50000bd70 comp rx=0 tx=0).stop 2024-09-10T11:26:59.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.636+0000 7ff515bce700 1 -- 172.21.15.144:0/4187209548 shutdown_connections 2024-09-10T11:26:59.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.636+0000 7ff515bce700 1 --2- 172.21.15.144:0/4187209548 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff4f806c680 0x7ff4f806eb30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:59.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.636+0000 7ff515bce700 1 --2- 172.21.15.144:0/4187209548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff510100c30 0x7ff510100590 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:59.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.636+0000 7ff515bce700 1 --2- 172.21.15.144:0/4187209548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff510103550 0x7ff5100febe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:26:59.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.636+0000 7ff515bce700 1 -- 172.21.15.144:0/4187209548 >> 172.21.15.144:0/4187209548 conn(0x7ff5100fa630 msgr2=0x7ff5100fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:26:59.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.636+0000 7ff515bce700 1 -- 172.21.15.144:0/4187209548 shutdown_connections 2024-09-10T11:26:59.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:26:59.636+0000 7ff515bce700 1 -- 172.21.15.144:0/4187209548 wait complete. 2024-09-10T11:26:59.647 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:26:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:59 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:26:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:26:59 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:27:00.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:59 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:27:00.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:26:59 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:27:00.312 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-09-10T11:27:00.329 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T11:27:00.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:00 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/4187209548' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-09-10T11:27:00.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:00 smithi144 ceph-mon[27942]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T11:27:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:00 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/4187209548' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-09-10T11:27:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:00 smithi178 ceph-mon[33514]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 10 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T11:27:01.556 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:27:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:02 smithi178 ceph-mon[33514]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T11:27:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:27:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:02 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:27:03.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:02 smithi144 ceph-mon[27942]: pgmap v192: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.7 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T11:27:03.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:27:03.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:02 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:27:03.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.769+0000 7f500f53a700 1 -- 172.21.15.144:0/3573445910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50081003b0 msgr2=0x7f5008100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:03.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.769+0000 7f500f53a700 1 --2- 172.21.15.144:0/3573445910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50081003b0 0x7f5008100780 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f500000b0a0 tx=0x7f500000b3b0 comp rx=0 tx=0).stop 2024-09-10T11:27:03.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.769+0000 7f500f53a700 1 -- 172.21.15.144:0/3573445910 shutdown_connections 2024-09-10T11:27:03.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.769+0000 7f500f53a700 1 --2- 172.21.15.144:0/3573445910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5008100d50 0x7f5008104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:03.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.769+0000 7f500f53a700 1 --2- 172.21.15.144:0/3573445910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50081003b0 0x7f5008100780 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:03.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.769+0000 7f500f53a700 1 -- 172.21.15.144:0/3573445910 >> 172.21.15.144:0/3573445910 conn(0x7f50080fbc30 msgr2=0x7f50080fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:27:03.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.769+0000 7f500f53a700 1 -- 172.21.15.144:0/3573445910 shutdown_connections 2024-09-10T11:27:03.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.769+0000 7f500f53a700 1 -- 172.21.15.144:0/3573445910 wait complete. 2024-09-10T11:27:03.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.770+0000 7f500f53a700 1 Processor -- start 2024-09-10T11:27:03.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.770+0000 7f500f53a700 1 -- start start 2024-09-10T11:27:03.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.770+0000 7f500f53a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50081003b0 0x7f5008195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:03.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.770+0000 7f500f53a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5008100d50 0x7f5008196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:03.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.770+0000 7f500f53a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5008196a40 con 0x7f50081003b0 2024-09-10T11:27:03.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.770+0000 7f500f53a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500819a7d0 con 0x7f5008100d50 2024-09-10T11:27:03.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.770+0000 7f500d2d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50081003b0 0x7f5008195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:03.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.770+0000 7f500d2d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50081003b0 0x7f5008195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35552/0 (socket says 172.21.15.144:35552) 2024-09-10T11:27:03.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.770+0000 7f500d2d6700 1 -- 172.21.15.144:0/1419192636 learned_addr learned my addr 172.21.15.144:0/1419192636 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:27:03.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.770+0000 7f500cad5700 1 --2- 172.21.15.144:0/1419192636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5008100d50 0x7f5008196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:03.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.770+0000 7f500d2d6700 1 -- 172.21.15.144:0/1419192636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5008100d50 msgr2=0x7f5008196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:03.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.771+0000 7f500d2d6700 1 --2- 172.21.15.144:0/1419192636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5008100d50 0x7f5008196360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:03.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.771+0000 7f500d2d6700 1 -- 172.21.15.144:0/1419192636 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5000009d00 con 0x7f50081003b0 2024-09-10T11:27:03.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.771+0000 7f500d2d6700 1 --2- 172.21.15.144:0/1419192636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50081003b0 0x7f5008195e20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5000000f80 tx=0x7f500000cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:27:03.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.771+0000 7f50067fc700 1 -- 172.21.15.144:0/1419192636 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f500000e040 con 0x7f50081003b0 2024-09-10T11:27:03.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.771+0000 7f50067fc700 1 -- 172.21.15.144:0/1419192636 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5000003de0 con 0x7f50081003b0 2024-09-10T11:27:03.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.771+0000 7f500f53a700 1 -- 172.21.15.144:0/1419192636 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500819aa50 con 0x7f50081003b0 2024-09-10T11:27:03.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.771+0000 7f50067fc700 1 -- 172.21.15.144:0/1419192636 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f500001d9a0 con 0x7f50081003b0 2024-09-10T11:27:03.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.771+0000 7f500f53a700 1 -- 172.21.15.144:0/1419192636 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500819af40 con 0x7f50081003b0 2024-09-10T11:27:03.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.772+0000 7f500f53a700 1 -- 172.21.15.144:0/1419192636 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50081086c0 con 0x7f50081003b0 2024-09-10T11:27:03.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.772+0000 7f50067fc700 1 -- 172.21.15.144:0/1419192636 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f5000004040 con 0x7f50081003b0 2024-09-10T11:27:03.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.773+0000 7f50067fc700 1 --2- 172.21.15.144:0/1419192636 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4ff006c680 0x7f4ff006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:03.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.773+0000 7f50067fc700 1 -- 172.21.15.144:0/1419192636 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f500008d1c0 con 0x7f50081003b0 2024-09-10T11:27:03.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.773+0000 7f500cad5700 1 --2- 172.21.15.144:0/1419192636 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4ff006c680 0x7f4ff006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:03.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.773+0000 7f500cad5700 1 --2- 172.21.15.144:0/1419192636 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4ff006c680 0x7f4ff006eb30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5008197440 tx=0x7f4ffc009190 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:27:03.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:03.777+0000 7f50067fc700 1 -- 172.21.15.144:0/1419192636 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f50000578f0 con 0x7f50081003b0 2024-09-10T11:27:04.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:04.021+0000 7f500f53a700 1 -- 172.21.15.144:0/1419192636 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f500804ea50 con 0x7f50081003b0 2024-09-10T11:27:04.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:04.021+0000 7f50067fc700 1 -- 172.21.15.144:0/1419192636 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4782 (secure 0 0 0) 0x7f500005af10 con 0x7f50081003b0 2024-09-10T11:27:04.024 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:27:04.024 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":19,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:26:53.953224+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14488,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14488":{"gid":14488,"name":"cephfs.smithi144.jaoyvs","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6829/2138997251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2138997251},{"type":"v1","addr":"172.21.15.144:6829","nonce":2138997251}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14498":{"gid":14498,"name":"cephfs.smithi144.bfyvwv","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6827/1752150537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1752150537},{"type":"v1","addr":"172.21.15.144:6827","nonce":1752150537}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi178.nmhrmh","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6825/2986758450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2986758450},{"type":"v1","addr":"172.21.15.178:6825","nonce":2986758450}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi178.czjrsa","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.178:6827/3262518638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3262518638},{"type":"v1","addr":"172.21.15.178:6827","nonce":3262518638}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-09-10T11:27:04.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:04.023+0000 7f500f53a700 1 -- 172.21.15.144:0/1419192636 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4ff006c680 msgr2=0x7f4ff006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:04.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:04.023+0000 7f500f53a700 1 --2- 172.21.15.144:0/1419192636 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4ff006c680 0x7f4ff006eb30 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5008197440 tx=0x7f4ffc009190 comp rx=0 tx=0).stop 2024-09-10T11:27:04.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:04.023+0000 7f500f53a700 1 -- 172.21.15.144:0/1419192636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50081003b0 msgr2=0x7f5008195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:04.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:04.023+0000 7f500f53a700 1 --2- 172.21.15.144:0/1419192636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50081003b0 0x7f5008195e20 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5000000f80 tx=0x7f500000cd00 comp rx=0 tx=0).stop 2024-09-10T11:27:04.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:04.023+0000 7f500f53a700 1 -- 172.21.15.144:0/1419192636 shutdown_connections 2024-09-10T11:27:04.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:04.023+0000 7f500f53a700 1 --2- 172.21.15.144:0/1419192636 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4ff006c680 0x7f4ff006eb30 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:04.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:04.023+0000 7f500f53a700 1 --2- 172.21.15.144:0/1419192636 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50081003b0 0x7f5008195e20 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:04.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:04.023+0000 7f500f53a700 1 --2- 172.21.15.144:0/1419192636 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5008100d50 0x7f5008196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:04.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:04.023+0000 7f500f53a700 1 -- 172.21.15.144:0/1419192636 >> 172.21.15.144:0/1419192636 conn(0x7f50080fbc30 msgr2=0x7f50080fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:27:04.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:04.024+0000 7f500f53a700 1 -- 172.21.15.144:0/1419192636 shutdown_connections 2024-09-10T11:27:04.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:04.024+0000 7f500f53a700 1 -- 172.21.15.144:0/1419192636 wait complete. 2024-09-10T11:27:04.027 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 19 2024-09-10T11:27:04.807 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 19, 'max_mds': 2, 'flags': 50} 2024-09-10T11:27:04.807 INFO:teuthology.run_tasks:Running task ceph-fuse... 2024-09-10T11:27:04.839 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2024-09-10T11:27:04.839 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.1': {}} 2024-09-10T11:27:04.839 INFO:tasks.ceph_fuse:client.0 config is {} 2024-09-10T11:27:04.839 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T11:27:04.839 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T11:27:04.840 INFO:tasks.ceph_fuse:client.1 config is {} 2024-09-10T11:27:04.840 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-10T11:27:04.840 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T11:27:04.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:04.840 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2024-09-10T11:27:04.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:04.859 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link delete ceph-brx 2024-09-10T11:27:04.931 INFO:teuthology.orchestra.run.smithi178.stderr:Cannot find device "ceph-brx" 2024-09-10T11:27:04.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:27:04.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:04.933 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-09-10T11:27:04.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:04.950 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link delete ceph-brx 2024-09-10T11:27:05.019 INFO:teuthology.orchestra.run.smithi144.stderr:Cannot find device "ceph-brx" 2024-09-10T11:27:05.021 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T11:27:05.021 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2024-09-10T11:27:05.021 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T11:27:05.022 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T11:27:05.084 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:04 smithi144 ceph-mon[27942]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.1 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T11:27:05.084 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:04 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1419192636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T11:27:05.276 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:27:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:04 smithi178 ceph-mon[33514]: pgmap v193: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 9.1 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T11:27:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:04 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/1419192636' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T11:27:07.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.997+0000 7fd94f5b0700 1 -- 172.21.15.144:0/3115005554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9480ffd20 msgr2=0x7fd948103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:07.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.997+0000 7fd94f5b0700 1 --2- 172.21.15.144:0/3115005554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9480ffd20 0x7fd948103cf0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd938009910 tx=0x7fd938009c20 comp rx=0 tx=0).stop 2024-09-10T11:27:07.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.997+0000 7fd94f5b0700 1 -- 172.21.15.144:0/3115005554 shutdown_connections 2024-09-10T11:27:07.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.997+0000 7fd94f5b0700 1 --2- 172.21.15.144:0/3115005554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9480ffd20 0x7fd948103cf0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:07.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.997+0000 7fd94f5b0700 1 --2- 172.21.15.144:0/3115005554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9480ff380 0x7fd9480ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:07.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.997+0000 7fd94f5b0700 1 -- 172.21.15.144:0/3115005554 >> 172.21.15.144:0/3115005554 conn(0x7fd9480fac20 msgr2=0x7fd9480fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:27:07.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.997+0000 7fd94f5b0700 1 -- 172.21.15.144:0/3115005554 shutdown_connections 2024-09-10T11:27:07.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.997+0000 7fd94f5b0700 1 -- 172.21.15.144:0/3115005554 wait complete. 2024-09-10T11:27:07.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.998+0000 7fd94f5b0700 1 Processor -- start 2024-09-10T11:27:07.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.998+0000 7fd94f5b0700 1 -- start start 2024-09-10T11:27:07.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.998+0000 7fd94f5b0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9480ff380 0x7fd948196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:07.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.998+0000 7fd94f5b0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9480ffd20 0x7fd948197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:07.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.998+0000 7fd94f5b0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9481977b0 con 0x7fd9480ffd20 2024-09-10T11:27:07.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.998+0000 7fd94f5b0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9481978f0 con 0x7fd9480ff380 2024-09-10T11:27:07.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.998+0000 7fd94cb4b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9480ffd20 0x7fd948197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:07.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.998+0000 7fd94cb4b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9480ffd20 0x7fd948197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35564/0 (socket says 172.21.15.144:35564) 2024-09-10T11:27:07.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.998+0000 7fd94cb4b700 1 -- 172.21.15.144:0/3299898855 learned_addr learned my addr 172.21.15.144:0/3299898855 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:27:07.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.998+0000 7fd94d34c700 1 --2- 172.21.15.144:0/3299898855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9480ff380 0x7fd948196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:07.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.998+0000 7fd94cb4b700 1 -- 172.21.15.144:0/3299898855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9480ff380 msgr2=0x7fd948196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:07.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.998+0000 7fd94cb4b700 1 --2- 172.21.15.144:0/3299898855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9480ff380 0x7fd948196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:07.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.998+0000 7fd94cb4b700 1 -- 172.21.15.144:0/3299898855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9380095f0 con 0x7fd9480ffd20 2024-09-10T11:27:07.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.999+0000 7fd94cb4b700 1 --2- 172.21.15.144:0/3299898855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9480ffd20 0x7fd948197190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd93800bda0 tx=0x7fd93800bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:27:07.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.999+0000 7fd93e7fc700 1 -- 172.21.15.144:0/3299898855 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd93801d070 con 0x7fd9480ffd20 2024-09-10T11:27:07.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.999+0000 7fd93e7fc700 1 -- 172.21.15.144:0/3299898855 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd93800f460 con 0x7fd9480ffd20 2024-09-10T11:27:07.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.999+0000 7fd94f5b0700 1 -- 172.21.15.144:0/3299898855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd94819fcc0 con 0x7fd9480ffd20 2024-09-10T11:27:07.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.999+0000 7fd93e7fc700 1 -- 172.21.15.144:0/3299898855 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd938017630 con 0x7fd9480ffd20 2024-09-10T11:27:07.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:06.999+0000 7fd94f5b0700 1 -- 172.21.15.144:0/3299898855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9481a00d0 con 0x7fd9480ffd20 2024-09-10T11:27:07.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.000+0000 7fd94f5b0700 1 -- 172.21.15.144:0/3299898855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9481076f0 con 0x7fd9480ffd20 2024-09-10T11:27:07.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.000+0000 7fd93e7fc700 1 -- 172.21.15.144:0/3299898855 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fd93800fab0 con 0x7fd9480ffd20 2024-09-10T11:27:07.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.001+0000 7fd93e7fc700 1 --2- 172.21.15.144:0/3299898855 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd93406c680 0x7fd93406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:07.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.001+0000 7fd93e7fc700 1 -- 172.21.15.144:0/3299898855 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fd93808cc40 con 0x7fd9480ffd20 2024-09-10T11:27:07.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.001+0000 7fd94d34c700 1 --2- 172.21.15.144:0/3299898855 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd93406c680 0x7fd93406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:07.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.001+0000 7fd94d34c700 1 --2- 172.21.15.144:0/3299898855 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd93406c680 0x7fd93406eb30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd944007be0 tx=0x7fd9440073d0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:27:07.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.005+0000 7fd93e7fc700 1 -- 172.21.15.144:0/3299898855 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd938057370 con 0x7fd9480ffd20 2024-09-10T11:27:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:06 smithi144 ceph-mon[27942]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.5 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T11:27:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:06 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:27:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:06 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:27:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:06 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:27:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:06 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:27:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:06 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:27:07.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.241+0000 7fd94f5b0700 1 -- 172.21.15.144:0/3299898855 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd94804ea50 con 0x7fd9480ffd20 2024-09-10T11:27:07.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.241+0000 7fd93e7fc700 1 -- 172.21.15.144:0/3299898855 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+83 (secure 0 0 0) 0x7fd93805a990 con 0x7fd9480ffd20 2024-09-10T11:27:07.244 INFO:teuthology.orchestra.run.smithi144.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-09-10T11:27:07.246 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.244+0000 7fd94f5b0700 1 -- 172.21.15.144:0/3299898855 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd93406c680 msgr2=0x7fd93406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:07.246 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.244+0000 7fd94f5b0700 1 --2- 172.21.15.144:0/3299898855 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd93406c680 0x7fd93406eb30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd944007be0 tx=0x7fd9440073d0 comp rx=0 tx=0).stop 2024-09-10T11:27:07.246 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.244+0000 7fd94f5b0700 1 -- 172.21.15.144:0/3299898855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9480ffd20 msgr2=0x7fd948197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:07.246 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.244+0000 7fd94f5b0700 1 --2- 172.21.15.144:0/3299898855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9480ffd20 0x7fd948197190 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fd93800bda0 tx=0x7fd93800bdd0 comp rx=0 tx=0).stop 2024-09-10T11:27:07.246 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.244+0000 7fd94f5b0700 1 -- 172.21.15.144:0/3299898855 shutdown_connections 2024-09-10T11:27:07.246 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.244+0000 7fd94f5b0700 1 --2- 172.21.15.144:0/3299898855 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd93406c680 0x7fd93406eb30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:07.246 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.244+0000 7fd94f5b0700 1 --2- 172.21.15.144:0/3299898855 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9480ff380 0x7fd948196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:07.246 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.244+0000 7fd94f5b0700 1 --2- 172.21.15.144:0/3299898855 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9480ffd20 0x7fd948197190 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:07.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.244+0000 7fd94f5b0700 1 -- 172.21.15.144:0/3299898855 >> 172.21.15.144:0/3299898855 conn(0x7fd9480fac20 msgr2=0x7fd9480fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:27:07.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.244+0000 7fd94f5b0700 1 -- 172.21.15.144:0/3299898855 shutdown_connections 2024-09-10T11:27:07.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07.244+0000 7fd94f5b0700 1 -- 172.21.15.144:0/3299898855 wait complete. 2024-09-10T11:27:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:06 smithi178 ceph-mon[33514]: pgmap v194: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.5 KiB/s rd, 0 B/s wr, 9 op/s 2024-09-10T11:27:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:06 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:27:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:06 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:27:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:06 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:27:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:06 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:27:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:06 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:27:07.826 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:27:07.826 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:27:07.826 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi144.front.sepia.ceph.com 2024-09-10T11:27:07.826 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-10T11:27:07.826 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-10T11:27:07.826 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:27:07.827 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:27:07.827 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:27:07.827 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:27:07.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:07.827 DEBUG:teuthology.orchestra.run.smithi144:> ip addr 2024-09-10T11:27:07.882 INFO:teuthology.orchestra.run.smithi144.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:27:07.882 INFO:teuthology.orchestra.run.smithi144.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:27:07.882 INFO:teuthology.orchestra.run.smithi144.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:27:07.882 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:27:07.882 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 ::1/128 scope host 2024-09-10T11:27:07.882 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:27:07.882 INFO:teuthology.orchestra.run.smithi144.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:27:07.882 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:22 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:27:07.882 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f0 2024-09-10T11:27:07.883 INFO:teuthology.orchestra.run.smithi144.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:27:07.883 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:d9:38:23 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:27:07.883 INFO:teuthology.orchestra.run.smithi144.stdout: altname enp7s0f1 2024-09-10T11:27:07.883 INFO:teuthology.orchestra.run.smithi144.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:27:07.883 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5e brd ff:ff:ff:ff:ff:ff 2024-09-10T11:27:07.883 INFO:teuthology.orchestra.run.smithi144.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:27:07.883 INFO:teuthology.orchestra.run.smithi144.stdout: link/ether 0c:c4:7a:88:7f:5f brd ff:ff:ff:ff:ff:ff 2024-09-10T11:27:07.883 INFO:teuthology.orchestra.run.smithi144.stdout: inet 172.21.15.144/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:27:07.883 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft 42281sec preferred_lft 42281sec 2024-09-10T11:27:07.883 INFO:teuthology.orchestra.run.smithi144.stdout: inet6 fe80::ec4:7aff:fe88:7f5f/64 scope link noprefixroute 2024-09-10T11:27:07.883 INFO:teuthology.orchestra.run.smithi144.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:27:07.884 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T11:27:07.884 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:27:07.884 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-09-10T11:27:07.884 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add name ceph-brx type bridge 2024-09-10T11:27:07.884 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr flush dev ceph-brx 2024-09-10T11:27:07.884 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set ceph-brx up 2024-09-10T11:27:07.884 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T11:27:07.884 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-09-10T11:27:07.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:07 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:27:08.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:27:08.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:08.086 DEBUG:teuthology.orchestra.run.smithi144:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:27:08.110 INFO:teuthology.orchestra.run.smithi144.stdout:1 2024-09-10T11:27:08.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:08.112 DEBUG:teuthology.orchestra.run.smithi144:> route 2024-09-10T11:27:08.189 INFO:teuthology.orchestra.run.smithi144.stdout:Kernel IP routing table 2024-09-10T11:27:08.189 INFO:teuthology.orchestra.run.smithi144.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:27:08.189 INFO:teuthology.orchestra.run.smithi144.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:27:08.189 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:27:08.189 INFO:teuthology.orchestra.run.smithi144.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:27:08.190 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:27:08.190 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-09-10T11:27:08.190 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:27:08.190 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:27:08.190 DEBUG:teuthology.orchestra.run.smithi144:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:27:08.190 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-09-10T11:27:08.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:27:08.285 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:07 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/3299898855' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T11:27:08.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:07 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/3299898855' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T11:27:08.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:27:08.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:08.521 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list 2024-09-10T11:27:08.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:08.537 DEBUG:teuthology.orchestra.run.smithi144:> ip netns list-id 2024-09-10T11:27:08.592 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:27:08.592 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-09-10T11:27:08.592 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:27:08.593 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-10T11:27:08.593 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-09-10T11:27:08.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:27:08.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:27:08.710 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-10T11:27:08.710 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:27:08.710 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-09-10T11:27:08.710 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-10T11:27:08.710 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T11:27:08.710 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-10T11:27:08.710 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-10T11:27:08.711 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-10T11:27:08.711 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-09-10T11:27:08.758 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:27:08.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:27:08.834 DEBUG:teuthology.orchestra.run.smithi144:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:27:08.835 DEBUG:teuthology.orchestra.run.smithi144:> set -e 2024-09-10T11:27:08.835 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set brx.0 up 2024-09-10T11:27:08.835 DEBUG:teuthology.orchestra.run.smithi144:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T11:27:08.835 DEBUG:teuthology.orchestra.run.smithi144:> ') 2024-09-10T11:27:08.883 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:27:08.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:27:08.911 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2024-09-10T11:27:08.911 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:27:08.911 DEBUG:teuthology.orchestra.run.smithi144:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-10T11:27:08.927 INFO:teuthology.orchestra.run.smithi144.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-10T11:27:08.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:27:08.927 DEBUG:teuthology.orchestra.run.smithi144:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:27:08.982 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2024-09-10T11:27:09.050 DEBUG:teuthology.orchestra.run.smithi144:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:27:09.106 INFO:teuthology.orchestra.run.smithi144.stdout:/proc 2024-09-10T11:27:09.106 INFO:teuthology.orchestra.run.smithi144.stdout:/sys 2024-09-10T11:27:09.106 INFO:teuthology.orchestra.run.smithi144.stdout:/dev 2024-09-10T11:27:09.107 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/security 2024-09-10T11:27:09.107 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/shm 2024-09-10T11:27:09.107 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/pts 2024-09-10T11:27:09.107 INFO:teuthology.orchestra.run.smithi144.stdout:/run 2024-09-10T11:27:09.107 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/cgroup 2024-09-10T11:27:09.107 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/pstore 2024-09-10T11:27:09.107 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/bpf 2024-09-10T11:27:09.107 INFO:teuthology.orchestra.run.smithi144.stdout:/ 2024-09-10T11:27:09.107 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/selinux 2024-09-10T11:27:09.107 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:27:09.107 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/debug 2024-09-10T11:27:09.107 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/mqueue 2024-09-10T11:27:09.108 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/hugepages 2024-09-10T11:27:09.108 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/tracing 2024-09-10T11:27:09.108 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/fuse/connections 2024-09-10T11:27:09.108 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/config 2024-09-10T11:27:09.108 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:27:09.108 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:27:09.108 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:27:09.108 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:27:09.108 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/1000 2024-09-10T11:27:09.108 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph 2024-09-10T11:27:09.108 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay 2024-09-10T11:27:09.109 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/e82cc84cd3cfab6e4ea3cc8540d65e99bd8f850fc035221c252202830e1393ff/merged 2024-09-10T11:27:09.109 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/5f2fd51fecae4ab418d65a5a7fe8012ab2dfc0f9c6fc15835dbce4bad3102a50/merged 2024-09-10T11:27:09.109 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/0 2024-09-10T11:27:09.109 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:27:09.109 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/c912de9a0d77b4327c2afd3fd0c81b9612045a40fc567e2fac5983b2bbaa4520/merged 2024-09-10T11:27:09.109 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/44b5b176daac6ffdae08927966a14276311c5420390e4f3931a45c72b56bb3a4/merged 2024-09-10T11:27:09.109 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/326154c7d77a22fefb3e0e5652255bb455c684bb1198d231a914181796b162f0/merged 2024-09-10T11:27:09.109 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/3cc12d46a3106ed249ead4bd472a39e75ff506bf9af697e5437c9862c5f67d5b/merged 2024-09-10T11:27:09.109 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/a2c445e344fec462fffb53b98a98877780b007070d1c4b7bac17dee2006ba431/merged 2024-09-10T11:27:09.109 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/96d4e04afd35b4c0d1f1628abe603766cdd5effd297caf579a67ef32ed948185/merged 2024-09-10T11:27:09.109 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/1455014bffed2674968bb5502c51ca605cb860bcddee9871f3c2c815b9d34e25/merged 2024-09-10T11:27:09.110 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/1efd8298fdaa223ff6c45309dd4e99b2d670aacf06c85eb2f0427fd19cdb3edb/merged 2024-09-10T11:27:09.110 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/dfffa705e36ff83303f8e1a5e81b5b0a9866f47d1b0a431d3c50790ad2739ad0/merged 2024-09-10T11:27:09.110 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/c37c9f7c54608acf6b8fe1f7fe26499bc954b68c5b72229a4d5b14105755e699/merged 2024-09-10T11:27:09.110 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/a3cb62015f7bcaacc1f48e5c564cc534b0f35a06c08042ff0424826e9de50f7c/merged 2024-09-10T11:27:09.110 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns 2024-09-10T11:27:09.110 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:27:09.110 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:27:09.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:09.110 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2024-09-10T11:27:09.162 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:27:09.163 DEBUG:teuthology.orchestra.run.smithi144:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2024-09-10T11:27:09.205 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2024-09-10T11:27:09.231 DEBUG:teuthology.orchestra.run.smithi144:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:27:09.245 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:08 smithi144 ceph-mon[27942]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T11:27:09.260 INFO:teuthology.orchestra.run.smithi144.stdout:/proc 2024-09-10T11:27:09.260 INFO:teuthology.orchestra.run.smithi144.stdout:/sys 2024-09-10T11:27:09.260 INFO:teuthology.orchestra.run.smithi144.stdout:/dev 2024-09-10T11:27:09.260 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/security 2024-09-10T11:27:09.260 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/shm 2024-09-10T11:27:09.260 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/pts 2024-09-10T11:27:09.260 INFO:teuthology.orchestra.run.smithi144.stdout:/run 2024-09-10T11:27:09.260 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/cgroup 2024-09-10T11:27:09.260 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/pstore 2024-09-10T11:27:09.261 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/bpf 2024-09-10T11:27:09.261 INFO:teuthology.orchestra.run.smithi144.stdout:/ 2024-09-10T11:27:09.261 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/selinux 2024-09-10T11:27:09.261 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:27:09.261 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/debug 2024-09-10T11:27:09.261 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/mqueue 2024-09-10T11:27:09.261 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/hugepages 2024-09-10T11:27:09.261 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/tracing 2024-09-10T11:27:09.261 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/fuse/connections 2024-09-10T11:27:09.261 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/config 2024-09-10T11:27:09.261 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:27:09.261 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:27:09.261 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:27:09.261 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:27:09.262 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/1000 2024-09-10T11:27:09.262 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph 2024-09-10T11:27:09.262 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay 2024-09-10T11:27:09.262 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/e82cc84cd3cfab6e4ea3cc8540d65e99bd8f850fc035221c252202830e1393ff/merged 2024-09-10T11:27:09.262 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/5f2fd51fecae4ab418d65a5a7fe8012ab2dfc0f9c6fc15835dbce4bad3102a50/merged 2024-09-10T11:27:09.262 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/0 2024-09-10T11:27:09.262 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:27:09.262 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/c912de9a0d77b4327c2afd3fd0c81b9612045a40fc567e2fac5983b2bbaa4520/merged 2024-09-10T11:27:09.262 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/44b5b176daac6ffdae08927966a14276311c5420390e4f3931a45c72b56bb3a4/merged 2024-09-10T11:27:09.262 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/326154c7d77a22fefb3e0e5652255bb455c684bb1198d231a914181796b162f0/merged 2024-09-10T11:27:09.262 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/3cc12d46a3106ed249ead4bd472a39e75ff506bf9af697e5437c9862c5f67d5b/merged 2024-09-10T11:27:09.262 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/a2c445e344fec462fffb53b98a98877780b007070d1c4b7bac17dee2006ba431/merged 2024-09-10T11:27:09.262 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/96d4e04afd35b4c0d1f1628abe603766cdd5effd297caf579a67ef32ed948185/merged 2024-09-10T11:27:09.262 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/1455014bffed2674968bb5502c51ca605cb860bcddee9871f3c2c815b9d34e25/merged 2024-09-10T11:27:09.263 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/1efd8298fdaa223ff6c45309dd4e99b2d670aacf06c85eb2f0427fd19cdb3edb/merged 2024-09-10T11:27:09.263 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/dfffa705e36ff83303f8e1a5e81b5b0a9866f47d1b0a431d3c50790ad2739ad0/merged 2024-09-10T11:27:09.263 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/c37c9f7c54608acf6b8fe1f7fe26499bc954b68c5b72229a4d5b14105755e699/merged 2024-09-10T11:27:09.263 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/a3cb62015f7bcaacc1f48e5c564cc534b0f35a06c08042ff0424826e9de50f7c/merged 2024-09-10T11:27:09.263 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns 2024-09-10T11:27:09.263 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:27:09.263 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:27:09.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:09.263 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2024-09-10T11:27:09.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi144.stderr:2024-09-10T11:27:09.321+0000 7f5069282480 -1 init, newargv = 0x55c2f1c083d0 newargc=15 2024-09-10T11:27:09.324 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi144.stderr:ceph-fuse[65863]: starting ceph client 2024-09-10T11:27:09.331 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi144.stderr:ceph-fuse[65863]: starting fuse 2024-09-10T11:27:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:08 smithi178 ceph-mon[33514]: pgmap v195: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T11:27:10.416 DEBUG:teuthology.orchestra.run.smithi144:> sudo modprobe fuse 2024-09-10T11:27:10.445 DEBUG:teuthology.orchestra.run.smithi144:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:27:10.502 INFO:teuthology.orchestra.run.smithi144.stdout:/proc 2024-09-10T11:27:10.502 INFO:teuthology.orchestra.run.smithi144.stdout:/sys 2024-09-10T11:27:10.502 INFO:teuthology.orchestra.run.smithi144.stdout:/dev 2024-09-10T11:27:10.502 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/security 2024-09-10T11:27:10.503 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/shm 2024-09-10T11:27:10.503 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/pts 2024-09-10T11:27:10.503 INFO:teuthology.orchestra.run.smithi144.stdout:/run 2024-09-10T11:27:10.503 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/cgroup 2024-09-10T11:27:10.503 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/pstore 2024-09-10T11:27:10.503 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/bpf 2024-09-10T11:27:10.503 INFO:teuthology.orchestra.run.smithi144.stdout:/ 2024-09-10T11:27:10.503 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/selinux 2024-09-10T11:27:10.503 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:27:10.503 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/debug 2024-09-10T11:27:10.503 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/mqueue 2024-09-10T11:27:10.503 INFO:teuthology.orchestra.run.smithi144.stdout:/dev/hugepages 2024-09-10T11:27:10.503 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/tracing 2024-09-10T11:27:10.504 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/fs/fuse/connections 2024-09-10T11:27:10.504 INFO:teuthology.orchestra.run.smithi144.stdout:/sys/kernel/config 2024-09-10T11:27:10.504 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:27:10.504 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:27:10.504 INFO:teuthology.orchestra.run.smithi144.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:27:10.504 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:27:10.504 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/1000 2024-09-10T11:27:10.504 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/ceph 2024-09-10T11:27:10.504 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay 2024-09-10T11:27:10.504 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/e82cc84cd3cfab6e4ea3cc8540d65e99bd8f850fc035221c252202830e1393ff/merged 2024-09-10T11:27:10.504 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/5f2fd51fecae4ab418d65a5a7fe8012ab2dfc0f9c6fc15835dbce4bad3102a50/merged 2024-09-10T11:27:10.504 INFO:teuthology.orchestra.run.smithi144.stdout:/run/user/0 2024-09-10T11:27:10.504 INFO:teuthology.orchestra.run.smithi144.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:27:10.504 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/c912de9a0d77b4327c2afd3fd0c81b9612045a40fc567e2fac5983b2bbaa4520/merged 2024-09-10T11:27:10.505 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/44b5b176daac6ffdae08927966a14276311c5420390e4f3931a45c72b56bb3a4/merged 2024-09-10T11:27:10.505 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/326154c7d77a22fefb3e0e5652255bb455c684bb1198d231a914181796b162f0/merged 2024-09-10T11:27:10.505 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/3cc12d46a3106ed249ead4bd472a39e75ff506bf9af697e5437c9862c5f67d5b/merged 2024-09-10T11:27:10.505 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/a2c445e344fec462fffb53b98a98877780b007070d1c4b7bac17dee2006ba431/merged 2024-09-10T11:27:10.505 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/96d4e04afd35b4c0d1f1628abe603766cdd5effd297caf579a67ef32ed948185/merged 2024-09-10T11:27:10.505 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/1455014bffed2674968bb5502c51ca605cb860bcddee9871f3c2c815b9d34e25/merged 2024-09-10T11:27:10.505 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/1efd8298fdaa223ff6c45309dd4e99b2d670aacf06c85eb2f0427fd19cdb3edb/merged 2024-09-10T11:27:10.505 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/dfffa705e36ff83303f8e1a5e81b5b0a9866f47d1b0a431d3c50790ad2739ad0/merged 2024-09-10T11:27:10.505 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/c37c9f7c54608acf6b8fe1f7fe26499bc954b68c5b72229a4d5b14105755e699/merged 2024-09-10T11:27:10.505 INFO:teuthology.orchestra.run.smithi144.stdout:/var/lib/containers/storage/overlay/a3cb62015f7bcaacc1f48e5c564cc534b0f35a06c08042ff0424826e9de50f7c/merged 2024-09-10T11:27:10.505 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns 2024-09-10T11:27:10.505 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:27:10.505 INFO:teuthology.orchestra.run.smithi144.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-10T11:27:10.506 INFO:teuthology.orchestra.run.smithi144.stdout:/home/ubuntu/cephtest/mnt.0 2024-09-10T11:27:10.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:10.506 DEBUG:teuthology.orchestra.run.smithi144:> ls /sys/fs/fuse/connections 2024-09-10T11:27:10.557 INFO:teuthology.orchestra.run.smithi144.stdout:141 2024-09-10T11:27:10.558 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [141] 2024-09-10T11:27:10.558 DEBUG:teuthology.orchestra.run.smithi144:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:27:10.558 DEBUG:teuthology.orchestra.run.smithi144:> import glob 2024-09-10T11:27:10.558 DEBUG:teuthology.orchestra.run.smithi144:> import re 2024-09-10T11:27:10.558 DEBUG:teuthology.orchestra.run.smithi144:> import os 2024-09-10T11:27:10.558 DEBUG:teuthology.orchestra.run.smithi144:> import subprocess 2024-09-10T11:27:10.558 DEBUG:teuthology.orchestra.run.smithi144:> 2024-09-10T11:27:10.558 DEBUG:teuthology.orchestra.run.smithi144:> def _find_admin_socket(client_name): 2024-09-10T11:27:10.558 DEBUG:teuthology.orchestra.run.smithi144:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2024-09-10T11:27:10.558 DEBUG:teuthology.orchestra.run.smithi144:> files = glob.glob(asok_path) 2024-09-10T11:27:10.558 DEBUG:teuthology.orchestra.run.smithi144:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> # Given a non-glob path, it better be there 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> if "*" not in asok_path: 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> assert(len(files) == 1) 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> return files[0] 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> for f in files: 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> contents = proc_f.read() 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> if mountpoint in contents: 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> return f 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> 2024-09-10T11:27:10.559 DEBUG:teuthology.orchestra.run.smithi144:> print(_find_admin_socket("client.0")) 2024-09-10T11:27:10.560 DEBUG:teuthology.orchestra.run.smithi144:> ' 2024-09-10T11:27:10.674 INFO:teuthology.orchestra.run.smithi144.stdout:/var/run/ceph/ceph-client.0.65863.asok 2024-09-10T11:27:10.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:27:10.685 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.65863.asok 2024-09-10T11:27:10.685 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:10.686 DEBUG:teuthology.orchestra.run.smithi144:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.65863.asok status 2024-09-10T11:27:10.817 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:27:10.817 INFO:teuthology.orchestra.run.smithi144.stdout: "metadata": { 2024-09-10T11:27:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_sha1": "5dd24139a1eada541a3bc16b6941c5dde975e26d", 2024-09-10T11:27:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-09-10T11:27:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: "entity_id": "0", 2024-09-10T11:27:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: "hostname": "smithi144", 2024-09-10T11:27:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2024-09-10T11:27:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: "pid": "65863", 2024-09-10T11:27:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: "root": "/" 2024-09-10T11:27:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:27:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_count": 0, 2024-09-10T11:27:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: "dentry_pinned_count": 0, 2024-09-10T11:27:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: "id": 14522, 2024-09-10T11:27:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: "inst": { 2024-09-10T11:27:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: "name": { 2024-09-10T11:27:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "client", 2024-09-10T11:27:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: "num": 14522 2024-09-10T11:27:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:27:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2024-09-10T11:27:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2024-09-10T11:27:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:27:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 3895676988 2024-09-10T11:27:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:27:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:27:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": { 2024-09-10T11:27:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: "type": "v1", 2024-09-10T11:27:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:27:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: "nonce": 3895676988 2024-09-10T11:27:10.820 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:27:10.820 INFO:teuthology.orchestra.run.smithi144.stdout: "inst_str": "client.14522 192.168.0.1:0/3895676988", 2024-09-10T11:27:10.820 INFO:teuthology.orchestra.run.smithi144.stdout: "addr_str": "192.168.0.1:0/3895676988", 2024-09-10T11:27:10.820 INFO:teuthology.orchestra.run.smithi144.stdout: "inode_count": 1, 2024-09-10T11:27:10.820 INFO:teuthology.orchestra.run.smithi144.stdout: "mds_epoch": 19, 2024-09-10T11:27:10.820 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch": 47, 2024-09-10T11:27:10.820 INFO:teuthology.orchestra.run.smithi144.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:27:10.820 INFO:teuthology.orchestra.run.smithi144.stdout: "blocklisted": false, 2024-09-10T11:27:10.820 INFO:teuthology.orchestra.run.smithi144.stdout: "fs_name": "cephfs" 2024-09-10T11:27:10.820 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:27:10.826 DEBUG:tasks.ceph_fuse:passing mntargs=[] 2024-09-10T11:27:10.826 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs ls 2024-09-10T11:27:11.024 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:27:11.273 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:10 smithi144 ceph-mon[27942]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T11:27:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:10 smithi178 ceph-mon[33514]: pgmap v196: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 3.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-10T11:27:12.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.537+0000 7feb5f70c700 1 -- 172.21.15.144:0/4236966130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb581003b0 msgr2=0x7feb58100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:12.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.537+0000 7feb5f70c700 1 --2- 172.21.15.144:0/4236966130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb581003b0 0x7feb58100780 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7feb48009910 tx=0x7feb48009c20 comp rx=0 tx=0).stop 2024-09-10T11:27:12.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.537+0000 7feb5f70c700 1 -- 172.21.15.144:0/4236966130 shutdown_connections 2024-09-10T11:27:12.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.537+0000 7feb5f70c700 1 --2- 172.21.15.144:0/4236966130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb58100d50 0x7feb58104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:12.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.537+0000 7feb5f70c700 1 --2- 172.21.15.144:0/4236966130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb581003b0 0x7feb58100780 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:12.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.537+0000 7feb5f70c700 1 -- 172.21.15.144:0/4236966130 >> 172.21.15.144:0/4236966130 conn(0x7feb580fbc30 msgr2=0x7feb580fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:27:12.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.537+0000 7feb5f70c700 1 -- 172.21.15.144:0/4236966130 shutdown_connections 2024-09-10T11:27:12.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.537+0000 7feb5f70c700 1 -- 172.21.15.144:0/4236966130 wait complete. 2024-09-10T11:27:12.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.538+0000 7feb5f70c700 1 Processor -- start 2024-09-10T11:27:12.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.538+0000 7feb5f70c700 1 -- start start 2024-09-10T11:27:12.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.538+0000 7feb5f70c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb581003b0 0x7feb58073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:12.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.538+0000 7feb5f70c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb58100d50 0x7feb58073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:12.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.538+0000 7feb5f70c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb58073ad0 con 0x7feb58100d50 2024-09-10T11:27:12.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.538+0000 7feb5f70c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb58073c10 con 0x7feb581003b0 2024-09-10T11:27:12.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.538+0000 7feb5cca7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb58100d50 0x7feb58073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:12.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.538+0000 7feb5cca7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb58100d50 0x7feb58073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59970/0 (socket says 172.21.15.144:59970) 2024-09-10T11:27:12.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.538+0000 7feb5cca7700 1 -- 172.21.15.144:0/3791751218 learned_addr learned my addr 172.21.15.144:0/3791751218 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:27:12.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.538+0000 7feb5d4a8700 1 --2- 172.21.15.144:0/3791751218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb581003b0 0x7feb58073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:12.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.538+0000 7feb5cca7700 1 -- 172.21.15.144:0/3791751218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb581003b0 msgr2=0x7feb58073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:12.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.538+0000 7feb5cca7700 1 --2- 172.21.15.144:0/3791751218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb581003b0 0x7feb58073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:12.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.539+0000 7feb5cca7700 1 -- 172.21.15.144:0/3791751218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb480095f0 con 0x7feb58100d50 2024-09-10T11:27:12.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.539+0000 7feb5cca7700 1 --2- 172.21.15.144:0/3791751218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb58100d50 0x7feb58073590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feb5400d900 tx=0x7feb5400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:27:12.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.539+0000 7feb4e7fc700 1 -- 172.21.15.144:0/3791751218 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb54009900 con 0x7feb58100d50 2024-09-10T11:27:12.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.539+0000 7feb4e7fc700 1 -- 172.21.15.144:0/3791751218 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7feb54010460 con 0x7feb58100d50 2024-09-10T11:27:12.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.539+0000 7feb5f70c700 1 -- 172.21.15.144:0/3791751218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb581a1ee0 con 0x7feb58100d50 2024-09-10T11:27:12.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.539+0000 7feb4e7fc700 1 -- 172.21.15.144:0/3791751218 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb5400f5f0 con 0x7feb58100d50 2024-09-10T11:27:12.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.539+0000 7feb5f70c700 1 -- 172.21.15.144:0/3791751218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb581a22f0 con 0x7feb58100d50 2024-09-10T11:27:12.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.540+0000 7feb5f70c700 1 -- 172.21.15.144:0/3791751218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb581086c0 con 0x7feb58100d50 2024-09-10T11:27:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.540+0000 7feb4e7fc700 1 -- 172.21.15.144:0/3791751218 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7feb540105d0 con 0x7feb58100d50 2024-09-10T11:27:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.541+0000 7feb4e7fc700 1 --2- 172.21.15.144:0/3791751218 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7feb4406c680 0x7feb4406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.541+0000 7feb4e7fc700 1 -- 172.21.15.144:0/3791751218 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7feb5408b3d0 con 0x7feb58100d50 2024-09-10T11:27:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.541+0000 7feb5d4a8700 1 --2- 172.21.15.144:0/3791751218 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7feb4406c680 0x7feb4406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:12.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.541+0000 7feb5d4a8700 1 --2- 172.21.15.144:0/3791751218 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7feb4406c680 0x7feb4406eb30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feb48006010 tx=0x7feb4801a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:27:12.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.545+0000 7feb4e7fc700 1 -- 172.21.15.144:0/3791751218 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7feb54055b00 con 0x7feb58100d50 2024-09-10T11:27:12.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.777+0000 7feb5f70c700 1 -- 172.21.15.144:0/3791751218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feb5804ea50 con 0x7feb58100d50 2024-09-10T11:27:12.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.778+0000 7feb4e7fc700 1 -- 172.21.15.144:0/3791751218 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+83 (secure 0 0 0) 0x7feb54089e60 con 0x7feb58100d50 2024-09-10T11:27:12.779 INFO:teuthology.orchestra.run.smithi144.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-09-10T11:27:12.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.780+0000 7feb5f70c700 1 -- 172.21.15.144:0/3791751218 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7feb4406c680 msgr2=0x7feb4406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:12.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.780+0000 7feb5f70c700 1 --2- 172.21.15.144:0/3791751218 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7feb4406c680 0x7feb4406eb30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7feb48006010 tx=0x7feb4801a040 comp rx=0 tx=0).stop 2024-09-10T11:27:12.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.780+0000 7feb5f70c700 1 -- 172.21.15.144:0/3791751218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb58100d50 msgr2=0x7feb58073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:12.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.780+0000 7feb5f70c700 1 --2- 172.21.15.144:0/3791751218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb58100d50 0x7feb58073590 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7feb5400d900 tx=0x7feb5400dc10 comp rx=0 tx=0).stop 2024-09-10T11:27:12.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.780+0000 7feb5f70c700 1 -- 172.21.15.144:0/3791751218 shutdown_connections 2024-09-10T11:27:12.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.780+0000 7feb5f70c700 1 --2- 172.21.15.144:0/3791751218 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7feb4406c680 0x7feb4406eb30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:12.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.780+0000 7feb5f70c700 1 --2- 172.21.15.144:0/3791751218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb581003b0 0x7feb58073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:12.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.780+0000 7feb5f70c700 1 --2- 172.21.15.144:0/3791751218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb58100d50 0x7feb58073590 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:12.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.780+0000 7feb5f70c700 1 -- 172.21.15.144:0/3791751218 >> 172.21.15.144:0/3791751218 conn(0x7feb580fbc30 msgr2=0x7feb580fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:27:12.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.780+0000 7feb5f70c700 1 -- 172.21.15.144:0/3791751218 shutdown_connections 2024-09-10T11:27:12.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:12.780+0000 7feb5f70c700 1 -- 172.21.15.144:0/3791751218 wait complete. 2024-09-10T11:27:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:12 smithi178 ceph-mon[33514]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-09-10T11:27:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:12 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/3791751218' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T11:27:13.375 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:12 smithi144 ceph-mon[27942]: pgmap v197: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 3 op/s 2024-09-10T11:27:13.375 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:12 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/3791751218' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-10T11:27:13.397 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-10T11:27:13.397 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-10T11:27:13.397 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi178.front.sepia.ceph.com 2024-09-10T11:27:13.397 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-10T11:27:13.397 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-10T11:27:13.397 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-10T11:27:13.397 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-10T11:27:13.397 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-10T11:27:13.397 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:27:13.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:13.398 DEBUG:teuthology.orchestra.run.smithi178:> ip addr 2024-09-10T11:27:13.417 INFO:teuthology.orchestra.run.smithi178.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-10T11:27:13.417 INFO:teuthology.orchestra.run.smithi178.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-10T11:27:13.417 INFO:teuthology.orchestra.run.smithi178.stdout: inet 127.0.0.1/8 scope host lo 2024-09-10T11:27:13.417 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:27:13.417 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 ::1/128 scope host 2024-09-10T11:27:13.417 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:27:13.417 INFO:teuthology.orchestra.run.smithi178.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:27:13.417 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:20 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:27:13.417 INFO:teuthology.orchestra.run.smithi178.stdout: altname enp7s0f0 2024-09-10T11:27:13.417 INFO:teuthology.orchestra.run.smithi178.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:27:13.418 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:d9:3b:21 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:27:13.418 INFO:teuthology.orchestra.run.smithi178.stdout: altname enp7s0f1 2024-09-10T11:27:13.418 INFO:teuthology.orchestra.run.smithi178.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-10T11:27:13.418 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f8 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:27:13.418 INFO:teuthology.orchestra.run.smithi178.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-10T11:27:13.418 INFO:teuthology.orchestra.run.smithi178.stdout: link/ether 0c:c4:7a:88:7e:f9 brd ff:ff:ff:ff:ff:ff 2024-09-10T11:27:13.418 INFO:teuthology.orchestra.run.smithi178.stdout: inet 172.21.15.178/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-10T11:27:13.418 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft 42328sec preferred_lft 42328sec 2024-09-10T11:27:13.418 INFO:teuthology.orchestra.run.smithi178.stdout: inet6 fe80::ec4:7aff:fe88:7ef9/64 scope link noprefixroute 2024-09-10T11:27:13.418 INFO:teuthology.orchestra.run.smithi178.stdout: valid_lft forever preferred_lft forever 2024-09-10T11:27:13.418 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-10T11:27:13.419 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:27:13.419 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-09-10T11:27:13.419 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link add name ceph-brx type bridge 2024-09-10T11:27:13.419 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip addr flush dev ceph-brx 2024-09-10T11:27:13.419 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set ceph-brx up 2024-09-10T11:27:13.419 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-10T11:27:13.419 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-09-10T11:27:13.511 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:27:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:27:13.634 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:27:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:27:13.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:13.641 DEBUG:teuthology.orchestra.run.smithi178:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-10T11:27:13.666 INFO:teuthology.orchestra.run.smithi178.stdout:1 2024-09-10T11:27:13.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:13.667 DEBUG:teuthology.orchestra.run.smithi178:> route 2024-09-10T11:27:13.740 INFO:teuthology.orchestra.run.smithi178.stdout:Kernel IP routing table 2024-09-10T11:27:13.740 INFO:teuthology.orchestra.run.smithi178.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-10T11:27:13.740 INFO:teuthology.orchestra.run.smithi178.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-10T11:27:13.740 INFO:teuthology.orchestra.run.smithi178.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-10T11:27:13.741 INFO:teuthology.orchestra.run.smithi178.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-10T11:27:13.741 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:27:13.741 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-09-10T11:27:13.741 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-10T11:27:13.741 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-10T11:27:13.741 DEBUG:teuthology.orchestra.run.smithi178:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-10T11:27:13.741 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-09-10T11:27:13.828 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:27:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:27:14.020 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:27:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:27:14.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:14.026 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list 2024-09-10T11:27:14.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:14.042 DEBUG:teuthology.orchestra.run.smithi178:> ip netns list-id 2024-09-10T11:27:14.098 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:27:14.098 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-09-10T11:27:14.098 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:27:14.099 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-10T11:27:14.099 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-09-10T11:27:14.186 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:27:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:27:14.209 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:27:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:27:14.215 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-10T11:27:14.216 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:27:14.216 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-09-10T11:27:14.216 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-10T11:27:14.216 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-10T11:27:14.216 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-10T11:27:14.216 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-10T11:27:14.216 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-10T11:27:14.216 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-09-10T11:27:14.266 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:27:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:27:14.347 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:27:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:27:14.353 DEBUG:teuthology.orchestra.run.smithi178:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-10T11:27:14.353 DEBUG:teuthology.orchestra.run.smithi178:> set -e 2024-09-10T11:27:14.353 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set brx.0 up 2024-09-10T11:27:14.353 DEBUG:teuthology.orchestra.run.smithi178:> sudo ip link set dev brx.0 master ceph-brx 2024-09-10T11:27:14.353 DEBUG:teuthology.orchestra.run.smithi178:> ') 2024-09-10T11:27:14.401 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:27:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-10T11:27:14.424 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:27:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:27:14.432 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2024-09-10T11:27:14.432 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:27:14.432 DEBUG:teuthology.orchestra.run.smithi178:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-10T11:27:14.447 INFO:teuthology.orchestra.run.smithi178.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-10T11:27:14.447 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-10T11:27:14.447 DEBUG:teuthology.orchestra.run.smithi178:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:27:14.502 DEBUG:teuthology.orchestra.run.smithi178:> sudo modprobe fuse 2024-09-10T11:27:14.570 DEBUG:teuthology.orchestra.run.smithi178:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:27:14.625 INFO:teuthology.orchestra.run.smithi178.stdout:/proc 2024-09-10T11:27:14.626 INFO:teuthology.orchestra.run.smithi178.stdout:/sys 2024-09-10T11:27:14.626 INFO:teuthology.orchestra.run.smithi178.stdout:/dev 2024-09-10T11:27:14.626 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/security 2024-09-10T11:27:14.626 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/shm 2024-09-10T11:27:14.626 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/pts 2024-09-10T11:27:14.626 INFO:teuthology.orchestra.run.smithi178.stdout:/run 2024-09-10T11:27:14.626 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup 2024-09-10T11:27:14.626 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/pstore 2024-09-10T11:27:14.626 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/bpf 2024-09-10T11:27:14.626 INFO:teuthology.orchestra.run.smithi178.stdout:/ 2024-09-10T11:27:14.626 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/selinux 2024-09-10T11:27:14.627 INFO:teuthology.orchestra.run.smithi178.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:27:14.627 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/hugepages 2024-09-10T11:27:14.627 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/mqueue 2024-09-10T11:27:14.627 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/debug 2024-09-10T11:27:14.627 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/tracing 2024-09-10T11:27:14.627 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/fuse/connections 2024-09-10T11:27:14.627 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/config 2024-09-10T11:27:14.627 INFO:teuthology.orchestra.run.smithi178.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:27:14.627 INFO:teuthology.orchestra.run.smithi178.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:27:14.627 INFO:teuthology.orchestra.run.smithi178.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:27:14.627 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:27:14.627 INFO:teuthology.orchestra.run.smithi178.stdout:/run/user/1000 2024-09-10T11:27:14.627 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/ceph 2024-09-10T11:27:14.627 INFO:teuthology.orchestra.run.smithi178.stdout:/run/user/0 2024-09-10T11:27:14.628 INFO:teuthology.orchestra.run.smithi178.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:27:14.628 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay 2024-09-10T11:27:14.628 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/44fcf51dea9f97bcb7097162993f00940e5ea6a807faa47aecd926070fc1254b/merged 2024-09-10T11:27:14.628 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/05d7aa54672307c0824246d2b71dbba100a85a75fcef6b99d3be834aca8ae6ee/merged 2024-09-10T11:27:14.628 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/c1d29e916b59df0ed123fa4ff29238a8ade786f90b6421638d3262dc01fe3f4a/merged 2024-09-10T11:27:14.628 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/fe8f53fc806e4f113dc2ebca411c68700637989ff5be64d628ab007d5fb450b8/merged 2024-09-10T11:27:14.628 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/7dd7337b31395d97142f764605735b017bdc94ea7485d78b8281ba1d652b53e8/merged 2024-09-10T11:27:14.628 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/282695fb5d7a04275ad200208bdf16c6a8581e85e608da48a3bf28e0a9023a68/merged 2024-09-10T11:27:14.628 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/96ca227c2fee88fe2fa5a1941156232a83802d7b9eb67a2cdf9771238bb8d90a/merged 2024-09-10T11:27:14.628 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/aa06580b9d53c141b894807e35f6a69905aaefbfee2f358845e215e627da7464/merged 2024-09-10T11:27:14.628 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/1286582ec95ccfc24649dc84acbecaa3b8abb363781d6d7caae77cafcb64b36f/merged 2024-09-10T11:27:14.628 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/f23b33b31e90227a2c57de568c7b7c8dc8d6ab63b35327095b2b990c9a7b7dc8/merged 2024-09-10T11:27:14.629 INFO:teuthology.orchestra.run.smithi178.stdout:/run/netns 2024-09-10T11:27:14.629 INFO:teuthology.orchestra.run.smithi178.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:27:14.629 INFO:teuthology.orchestra.run.smithi178.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:27:14.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:14.629 DEBUG:teuthology.orchestra.run.smithi178:> ls /sys/fs/fuse/connections 2024-09-10T11:27:14.680 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-10T11:27:14.681 DEBUG:teuthology.orchestra.run.smithi178:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2024-09-10T11:27:14.723 DEBUG:teuthology.orchestra.run.smithi178:> sudo modprobe fuse 2024-09-10T11:27:14.747 DEBUG:teuthology.orchestra.run.smithi178:> cat /proc/self/mounts | awk '{print $2}' 2024-09-10T11:27:14.803 INFO:teuthology.orchestra.run.smithi178.stdout:/proc 2024-09-10T11:27:14.803 INFO:teuthology.orchestra.run.smithi178.stdout:/sys 2024-09-10T11:27:14.803 INFO:teuthology.orchestra.run.smithi178.stdout:/dev 2024-09-10T11:27:14.803 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/security 2024-09-10T11:27:14.804 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/shm 2024-09-10T11:27:14.804 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/pts 2024-09-10T11:27:14.804 INFO:teuthology.orchestra.run.smithi178.stdout:/run 2024-09-10T11:27:14.804 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/cgroup 2024-09-10T11:27:14.804 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/pstore 2024-09-10T11:27:14.804 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/bpf 2024-09-10T11:27:14.804 INFO:teuthology.orchestra.run.smithi178.stdout:/ 2024-09-10T11:27:14.804 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/selinux 2024-09-10T11:27:14.804 INFO:teuthology.orchestra.run.smithi178.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:27:14.804 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/hugepages 2024-09-10T11:27:14.804 INFO:teuthology.orchestra.run.smithi178.stdout:/dev/mqueue 2024-09-10T11:27:14.804 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/debug 2024-09-10T11:27:14.805 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/tracing 2024-09-10T11:27:14.805 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/fs/fuse/connections 2024-09-10T11:27:14.805 INFO:teuthology.orchestra.run.smithi178.stdout:/sys/kernel/config 2024-09-10T11:27:14.805 INFO:teuthology.orchestra.run.smithi178.stdout:/run/credentials/systemd-sysctl.service 2024-09-10T11:27:14.805 INFO:teuthology.orchestra.run.smithi178.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-10T11:27:14.805 INFO:teuthology.orchestra.run.smithi178.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-10T11:27:14.805 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/nfs/rpc_pipefs 2024-09-10T11:27:14.805 INFO:teuthology.orchestra.run.smithi178.stdout:/run/user/1000 2024-09-10T11:27:14.805 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/ceph 2024-09-10T11:27:14.805 INFO:teuthology.orchestra.run.smithi178.stdout:/run/user/0 2024-09-10T11:27:14.805 INFO:teuthology.orchestra.run.smithi178.stdout:/proc/sys/fs/binfmt_misc 2024-09-10T11:27:14.805 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay 2024-09-10T11:27:14.805 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/44fcf51dea9f97bcb7097162993f00940e5ea6a807faa47aecd926070fc1254b/merged 2024-09-10T11:27:14.806 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/05d7aa54672307c0824246d2b71dbba100a85a75fcef6b99d3be834aca8ae6ee/merged 2024-09-10T11:27:14.806 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/c1d29e916b59df0ed123fa4ff29238a8ade786f90b6421638d3262dc01fe3f4a/merged 2024-09-10T11:27:14.806 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/fe8f53fc806e4f113dc2ebca411c68700637989ff5be64d628ab007d5fb450b8/merged 2024-09-10T11:27:14.806 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/7dd7337b31395d97142f764605735b017bdc94ea7485d78b8281ba1d652b53e8/merged 2024-09-10T11:27:14.806 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/282695fb5d7a04275ad200208bdf16c6a8581e85e608da48a3bf28e0a9023a68/merged 2024-09-10T11:27:14.806 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/96ca227c2fee88fe2fa5a1941156232a83802d7b9eb67a2cdf9771238bb8d90a/merged 2024-09-10T11:27:14.806 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/aa06580b9d53c141b894807e35f6a69905aaefbfee2f358845e215e627da7464/merged 2024-09-10T11:27:14.806 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/1286582ec95ccfc24649dc84acbecaa3b8abb363781d6d7caae77cafcb64b36f/merged 2024-09-10T11:27:14.806 INFO:teuthology.orchestra.run.smithi178.stdout:/var/lib/containers/storage/overlay/f23b33b31e90227a2c57de568c7b7c8dc8d6ab63b35327095b2b990c9a7b7dc8/merged 2024-09-10T11:27:14.806 INFO:teuthology.orchestra.run.smithi178.stdout:/run/netns 2024-09-10T11:27:14.806 INFO:teuthology.orchestra.run.smithi178.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:27:14.806 INFO:teuthology.orchestra.run.smithi178.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-10T11:27:14.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:14.808 DEBUG:teuthology.orchestra.run.smithi178:> ls /sys/fs/fuse/connections 2024-09-10T11:27:14.840 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi178.stderr:2024-09-10T11:27:14.837+0000 7f826100c480 -1 init, newargv = 0x559b8867b530 newargc=15 2024-09-10T11:27:14.840 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi178.stderr:ceph-fuse[53275]: starting ceph client 2024-09-10T11:27:14.848 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi178.stderr:ceph-fuse[53275]: starting fuse 2024-09-10T11:27:14.856 INFO:teuthology.orchestra.run.smithi178.stdout:129 2024-09-10T11:27:14.856 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [129] 2024-09-10T11:27:14.857 DEBUG:teuthology.orchestra.run.smithi178:> sudo stdin-killer -- python3 -c ' 2024-09-10T11:27:14.857 DEBUG:teuthology.orchestra.run.smithi178:> import glob 2024-09-10T11:27:14.857 DEBUG:teuthology.orchestra.run.smithi178:> import re 2024-09-10T11:27:14.857 DEBUG:teuthology.orchestra.run.smithi178:> import os 2024-09-10T11:27:14.857 DEBUG:teuthology.orchestra.run.smithi178:> import subprocess 2024-09-10T11:27:14.857 DEBUG:teuthology.orchestra.run.smithi178:> 2024-09-10T11:27:14.857 DEBUG:teuthology.orchestra.run.smithi178:> def _find_admin_socket(client_name): 2024-09-10T11:27:14.857 DEBUG:teuthology.orchestra.run.smithi178:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2024-09-10T11:27:14.857 DEBUG:teuthology.orchestra.run.smithi178:> files = glob.glob(asok_path) 2024-09-10T11:27:14.857 DEBUG:teuthology.orchestra.run.smithi178:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2024-09-10T11:27:14.857 DEBUG:teuthology.orchestra.run.smithi178:> 2024-09-10T11:27:14.857 DEBUG:teuthology.orchestra.run.smithi178:> # Given a non-glob path, it better be there 2024-09-10T11:27:14.858 DEBUG:teuthology.orchestra.run.smithi178:> if "*" not in asok_path: 2024-09-10T11:27:14.858 DEBUG:teuthology.orchestra.run.smithi178:> assert(len(files) == 1) 2024-09-10T11:27:14.858 DEBUG:teuthology.orchestra.run.smithi178:> return files[0] 2024-09-10T11:27:14.858 DEBUG:teuthology.orchestra.run.smithi178:> 2024-09-10T11:27:14.858 DEBUG:teuthology.orchestra.run.smithi178:> for f in files: 2024-09-10T11:27:14.858 DEBUG:teuthology.orchestra.run.smithi178:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-10T11:27:14.858 DEBUG:teuthology.orchestra.run.smithi178:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-10T11:27:14.858 DEBUG:teuthology.orchestra.run.smithi178:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-10T11:27:14.858 DEBUG:teuthology.orchestra.run.smithi178:> contents = proc_f.read() 2024-09-10T11:27:14.858 DEBUG:teuthology.orchestra.run.smithi178:> if mountpoint in contents: 2024-09-10T11:27:14.858 DEBUG:teuthology.orchestra.run.smithi178:> return f 2024-09-10T11:27:14.858 DEBUG:teuthology.orchestra.run.smithi178:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-10T11:27:14.858 DEBUG:teuthology.orchestra.run.smithi178:> 2024-09-10T11:27:14.858 DEBUG:teuthology.orchestra.run.smithi178:> print(_find_admin_socket("client.1")) 2024-09-10T11:27:14.858 DEBUG:teuthology.orchestra.run.smithi178:> ' 2024-09-10T11:27:14.974 INFO:teuthology.orchestra.run.smithi178.stdout:/var/run/ceph/ceph-client.1.53275.asok 2024-09-10T11:27:14.977 INFO:teuthology.orchestra.run.smithi178.stderr:2024-09-10T11:27:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-10T11:27:14.985 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.53275.asok 2024-09-10T11:27:14.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:14.985 DEBUG:teuthology.orchestra.run.smithi178:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.53275.asok status 2024-09-10T11:27:15.118 INFO:teuthology.orchestra.run.smithi178.stdout:{ 2024-09-10T11:27:15.118 INFO:teuthology.orchestra.run.smithi178.stdout: "metadata": { 2024-09-10T11:27:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_sha1": "5dd24139a1eada541a3bc16b6941c5dde975e26d", 2024-09-10T11:27:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-09-10T11:27:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "entity_id": "1", 2024-09-10T11:27:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "hostname": "smithi178", 2024-09-10T11:27:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2024-09-10T11:27:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "pid": "53275", 2024-09-10T11:27:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "root": "/" 2024-09-10T11:27:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T11:27:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "dentry_count": 0, 2024-09-10T11:27:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "dentry_pinned_count": 0, 2024-09-10T11:27:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "id": 24305, 2024-09-10T11:27:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "inst": { 2024-09-10T11:27:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "name": { 2024-09-10T11:27:15.119 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "client", 2024-09-10T11:27:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "num": 24305 2024-09-10T11:27:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T11:27:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": { 2024-09-10T11:27:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-10T11:27:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:27:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 2242343795 2024-09-10T11:27:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: } 2024-09-10T11:27:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T11:27:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": { 2024-09-10T11:27:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "type": "v1", 2024-09-10T11:27:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "addr": "192.168.0.1:0", 2024-09-10T11:27:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "nonce": 2242343795 2024-09-10T11:27:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: }, 2024-09-10T11:27:15.120 INFO:teuthology.orchestra.run.smithi178.stdout: "inst_str": "client.24305 192.168.0.1:0/2242343795", 2024-09-10T11:27:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: "addr_str": "192.168.0.1:0/2242343795", 2024-09-10T11:27:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: "inode_count": 1, 2024-09-10T11:27:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: "mds_epoch": 19, 2024-09-10T11:27:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_epoch": 47, 2024-09-10T11:27:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: "osd_epoch_barrier": 0, 2024-09-10T11:27:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: "blocklisted": false, 2024-09-10T11:27:15.121 INFO:teuthology.orchestra.run.smithi178.stdout: "fs_name": "cephfs" 2024-09-10T11:27:15.121 INFO:teuthology.orchestra.run.smithi178.stdout:} 2024-09-10T11:27:15.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:15.128 DEBUG:teuthology.orchestra.run.smithi144:> stat --file-system '--printf=%T 2024-09-10T11:27:15.128 DEBUG:teuthology.orchestra.run.smithi144:> ' -- /home/ubuntu/cephtest/mnt.0 2024-09-10T11:27:15.149 INFO:teuthology.orchestra.run.smithi144.stdout:fuseblk 2024-09-10T11:27:15.149 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2024-09-10T11:27:15.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:15.149 DEBUG:teuthology.orchestra.run.smithi144:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-10T11:27:15.225 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:15.225 DEBUG:teuthology.orchestra.run.smithi178:> stat --file-system '--printf=%T 2024-09-10T11:27:15.225 DEBUG:teuthology.orchestra.run.smithi178:> ' -- /home/ubuntu/cephtest/mnt.1 2024-09-10T11:27:15.242 INFO:teuthology.orchestra.run.smithi178.stdout:fuseblk 2024-09-10T11:27:15.242 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2024-09-10T11:27:15.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T11:27:15.243 DEBUG:teuthology.orchestra.run.smithi178:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-10T11:27:15.302 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:15 smithi178 ceph-mon[33514]: pgmap v198: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-10T11:27:15.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:15 smithi144 ceph-mon[27942]: pgmap v198: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 5 op/s 2024-09-10T11:27:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:17 smithi178 ceph-mon[33514]: pgmap v199: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-09-10T11:27:17.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:17 smithi144 ceph-mon[27942]: pgmap v199: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s rd, 341 B/s wr, 4 op/s 2024-09-10T11:27:18.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:18 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:27:18.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:18 smithi144 ceph-mon[27942]: pgmap v200: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-10T11:27:18.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:18 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:27:18.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:18 smithi178 ceph-mon[33514]: pgmap v200: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-10T11:27:20.081 INFO:teuthology.run_tasks:Running task print... 2024-09-10T11:27:20.092 INFO:teuthology.task.print:**** done client 2024-09-10T11:27:20.092 INFO:teuthology.run_tasks:Running task parallel... 2024-09-10T11:27:20.104 INFO:teuthology.task.parallel:starting parallel... 2024-09-10T11:27:20.105 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-09-10T11:27:20.105 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-10T11:27:20.106 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T11:27:20.106 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-09-10T11:27:20.106 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-09-10T11:27:20.106 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-09-10T11:27:20.111 INFO:tasks.workunit:Pulling workunits from ref 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:27:20.111 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T11:27:20.112 INFO:tasks.workunit:timeout=3h 2024-09-10T11:27:20.112 INFO:tasks.workunit:cleanup=True 2024-09-10T11:27:20.112 DEBUG:teuthology.orchestra.run.smithi144:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T11:27:20.132 INFO:teuthology.orchestra.run.smithi144.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-10T11:27:20.132 INFO:teuthology.orchestra.run.smithi144.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-10T11:27:20.132 INFO:teuthology.orchestra.run.smithi144.stdout:Device: 8dh/141d Inode: 1 Links: 2 2024-09-10T11:27:20.132 INFO:teuthology.orchestra.run.smithi144.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:27:20.132 INFO:teuthology.orchestra.run.smithi144.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-10T11:27:20.132 INFO:teuthology.orchestra.run.smithi144.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-10T11:27:20.133 INFO:teuthology.orchestra.run.smithi144.stdout:Modify: 2024-09-10 11:26:38.360934681 +0000 2024-09-10T11:27:20.133 INFO:teuthology.orchestra.run.smithi144.stdout:Change: 2024-09-10 11:27:15.221305098 +0000 2024-09-10T11:27:20.133 INFO:teuthology.orchestra.run.smithi144.stdout: Birth: - 2024-09-10T11:27:20.133 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-10T11:27:20.133 DEBUG:teuthology.orchestra.run.smithi144:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-10T11:27:20.206 DEBUG:teuthology.orchestra.run.smithi178:> stat -- /home/ubuntu/cephtest/mnt.1 2024-09-10T11:27:20.228 INFO:teuthology.orchestra.run.smithi178.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-09-10T11:27:20.228 INFO:teuthology.orchestra.run.smithi178.stdout: Size: 0 Blocks: 1 IO Block: 4194304 directory 2024-09-10T11:27:20.228 INFO:teuthology.orchestra.run.smithi178.stdout:Device: 81h/129d Inode: 1 Links: 3 2024-09-10T11:27:20.228 INFO:teuthology.orchestra.run.smithi178.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T11:27:20.228 INFO:teuthology.orchestra.run.smithi178.stdout:Context: system_u:object_r:fusefs_t:s0 2024-09-10T11:27:20.228 INFO:teuthology.orchestra.run.smithi178.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-09-10T11:27:20.228 INFO:teuthology.orchestra.run.smithi178.stdout:Modify: 2024-09-10 11:27:20.198835040 +0000 2024-09-10T11:27:20.228 INFO:teuthology.orchestra.run.smithi178.stdout:Change: 2024-09-10 11:27:20.198835040 +0000 2024-09-10T11:27:20.228 INFO:teuthology.orchestra.run.smithi178.stdout: Birth: - 2024-09-10T11:27:20.229 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-09-10T11:27:20.229 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-09-10T11:27:20.305 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:27:20.305 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:27:20.323 INFO:tasks.workunit.client.0.smithi144.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T11:27:20.334 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:27:20.363 INFO:tasks.workunit.client.1.smithi178.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-09-10T11:27:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:20 smithi178 ceph-mon[33514]: pgmap v201: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-10T11:27:21.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:20 smithi144 ceph-mon[27942]: pgmap v201: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-10T11:27:21.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.917+0000 7f3b393ec700 1 -- 172.21.15.144:0/2372218667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b340ff080 msgr2=0x7f3b340ff490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:21.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.917+0000 7f3b393ec700 1 --2- 172.21.15.144:0/2372218667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b340ff080 0x7f3b340ff490 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3b24009910 tx=0x7f3b24009c20 comp rx=0 tx=0).stop 2024-09-10T11:27:21.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.917+0000 7f3b393ec700 1 -- 172.21.15.144:0/2372218667 shutdown_connections 2024-09-10T11:27:21.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.917+0000 7f3b393ec700 1 --2- 172.21.15.144:0/2372218667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b34100280 0x7f3b341006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:21.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.917+0000 7f3b393ec700 1 --2- 172.21.15.144:0/2372218667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b340ff080 0x7f3b340ff490 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:21.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.917+0000 7f3b393ec700 1 -- 172.21.15.144:0/2372218667 >> 172.21.15.144:0/2372218667 conn(0x7f3b340fa630 msgr2=0x7f3b340fca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:27:21.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.917+0000 7f3b393ec700 1 -- 172.21.15.144:0/2372218667 shutdown_connections 2024-09-10T11:27:21.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.917+0000 7f3b393ec700 1 -- 172.21.15.144:0/2372218667 wait complete. 2024-09-10T11:27:21.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.918+0000 7f3b393ec700 1 Processor -- start 2024-09-10T11:27:21.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.918+0000 7f3b393ec700 1 -- start start 2024-09-10T11:27:21.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.918+0000 7f3b393ec700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b340ff080 0x7f3b34195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:21.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.918+0000 7f3b393ec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b34100280 0x7f3b34195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:21.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.918+0000 7f3b393ec700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b341965a0 con 0x7f3b34100280 2024-09-10T11:27:21.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.918+0000 7f3b393ec700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b341966e0 con 0x7f3b340ff080 2024-09-10T11:27:21.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.919+0000 7f3b327fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b34100280 0x7f3b34195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:21.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.919+0000 7f3b327fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b34100280 0x7f3b34195f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59990/0 (socket says 172.21.15.144:59990) 2024-09-10T11:27:21.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.919+0000 7f3b327fc700 1 -- 172.21.15.144:0/4142047718 learned_addr learned my addr 172.21.15.144:0/4142047718 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:27:21.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.919+0000 7f3b32ffd700 1 --2- 172.21.15.144:0/4142047718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b340ff080 0x7f3b34195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:21.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.919+0000 7f3b327fc700 1 -- 172.21.15.144:0/4142047718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b340ff080 msgr2=0x7f3b34195a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:21.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.919+0000 7f3b327fc700 1 --2- 172.21.15.144:0/4142047718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b340ff080 0x7f3b34195a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:21.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.919+0000 7f3b327fc700 1 -- 172.21.15.144:0/4142047718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b240095f0 con 0x7f3b34100280 2024-09-10T11:27:21.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.919+0000 7f3b32ffd700 1 --2- 172.21.15.144:0/4142047718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b340ff080 0x7f3b34195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:27:21.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.919+0000 7f3b327fc700 1 --2- 172.21.15.144:0/4142047718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b34100280 0x7f3b34195f80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800b4b0 tx=0x7f3b2800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:27:21.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.919+0000 7f3b1bfff700 1 -- 172.21.15.144:0/4142047718 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b28011800 con 0x7f3b34100280 2024-09-10T11:27:21.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.919+0000 7f3b1bfff700 1 -- 172.21.15.144:0/4142047718 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3b28011e10 con 0x7f3b34100280 2024-09-10T11:27:21.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.919+0000 7f3b393ec700 1 -- 172.21.15.144:0/4142047718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3419b190 con 0x7f3b34100280 2024-09-10T11:27:21.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.919+0000 7f3b1bfff700 1 -- 172.21.15.144:0/4142047718 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b2800f2e0 con 0x7f3b34100280 2024-09-10T11:27:21.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.919+0000 7f3b393ec700 1 -- 172.21.15.144:0/4142047718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3419b760 con 0x7f3b34100280 2024-09-10T11:27:21.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.920+0000 7f3b393ec700 1 -- 172.21.15.144:0/4142047718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b34107c50 con 0x7f3b34100280 2024-09-10T11:27:21.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.921+0000 7f3b1bfff700 1 -- 172.21.15.144:0/4142047718 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f3b28010380 con 0x7f3b34100280 2024-09-10T11:27:21.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.921+0000 7f3b1bfff700 1 --2- 172.21.15.144:0/4142047718 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b1c06c680 0x7f3b1c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:21.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.921+0000 7f3b32ffd700 1 --2- 172.21.15.144:0/4142047718 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b1c06c680 0x7f3b1c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:21.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.921+0000 7f3b1bfff700 1 -- 172.21.15.144:0/4142047718 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f3b2808b490 con 0x7f3b34100280 2024-09-10T11:27:21.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.922+0000 7f3b32ffd700 1 --2- 172.21.15.144:0/4142047718 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b1c06c680 0x7f3b1c06eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3b24006010 tx=0x7f3b24005920 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:27:21.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:21.925+0000 7f3b1bfff700 1 -- 172.21.15.144:0/4142047718 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3b28059650 con 0x7f3b34100280 2024-09-10T11:27:22.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:22.124+0000 7f3b393ec700 1 -- 172.21.15.144:0/4142047718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f3b34107e10 con 0x7f3b34100280 2024-09-10T11:27:22.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:22.125+0000 7f3b1bfff700 1 -- 172.21.15.144:0/4142047718 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7f3b280591e0 con 0x7f3b34100280 2024-09-10T11:27:22.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:22.127+0000 7f3b393ec700 1 -- 172.21.15.144:0/4142047718 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b1c06c680 msgr2=0x7f3b1c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:22.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:22.127+0000 7f3b393ec700 1 --2- 172.21.15.144:0/4142047718 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b1c06c680 0x7f3b1c06eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3b24006010 tx=0x7f3b24005920 comp rx=0 tx=0).stop 2024-09-10T11:27:22.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:22.127+0000 7f3b393ec700 1 -- 172.21.15.144:0/4142047718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b34100280 msgr2=0x7f3b34195f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:22.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:22.127+0000 7f3b393ec700 1 --2- 172.21.15.144:0/4142047718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b34100280 0x7f3b34195f80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3b2800b4b0 tx=0x7f3b2800b870 comp rx=0 tx=0).stop 2024-09-10T11:27:22.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:22.127+0000 7f3b393ec700 1 -- 172.21.15.144:0/4142047718 shutdown_connections 2024-09-10T11:27:22.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:22.127+0000 7f3b393ec700 1 --2- 172.21.15.144:0/4142047718 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f3b1c06c680 0x7f3b1c06eb30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:22.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:22.127+0000 7f3b393ec700 1 --2- 172.21.15.144:0/4142047718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b340ff080 0x7f3b34195a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:22.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:22.127+0000 7f3b393ec700 1 --2- 172.21.15.144:0/4142047718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b34100280 0x7f3b34195f80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:22.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:22.127+0000 7f3b393ec700 1 -- 172.21.15.144:0/4142047718 >> 172.21.15.144:0/4142047718 conn(0x7f3b340fa630 msgr2=0x7f3b34068cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:27:22.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:22.127+0000 7f3b393ec700 1 -- 172.21.15.144:0/4142047718 shutdown_connections 2024-09-10T11:27:22.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:22.127+0000 7f3b393ec700 1 -- 172.21.15.144:0/4142047718 wait complete. 2024-09-10T11:27:22.679 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-09-10T11:27:22.929 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:27:23.284 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:22 smithi144 ceph-mon[27942]: pgmap v202: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T11:27:23.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:22 smithi178 ceph-mon[33514]: pgmap v202: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T11:27:24.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.624+0000 7f5701e1b700 1 -- 172.21.15.144:0/1308352111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56fc101760 msgr2=0x7f56fc101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:24.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.624+0000 7f5701e1b700 1 --2- 172.21.15.144:0/1308352111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56fc101760 0x7f56fc101b70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009910 tx=0x7f56ec009c20 comp rx=0 tx=0).stop 2024-09-10T11:27:24.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.625+0000 7f5701e1b700 1 -- 172.21.15.144:0/1308352111 shutdown_connections 2024-09-10T11:27:24.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.625+0000 7f5701e1b700 1 --2- 172.21.15.144:0/1308352111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56fc102140 0x7f56fc0fe220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:24.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.625+0000 7f5701e1b700 1 --2- 172.21.15.144:0/1308352111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56fc101760 0x7f56fc101b70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:24.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.625+0000 7f5701e1b700 1 -- 172.21.15.144:0/1308352111 >> 172.21.15.144:0/1308352111 conn(0x7f56fc0f9db0 msgr2=0x7f56fc0fc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:27:24.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.625+0000 7f5701e1b700 1 -- 172.21.15.144:0/1308352111 shutdown_connections 2024-09-10T11:27:24.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.625+0000 7f5701e1b700 1 -- 172.21.15.144:0/1308352111 wait complete. 2024-09-10T11:27:24.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.625+0000 7f5701e1b700 1 Processor -- start 2024-09-10T11:27:24.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.625+0000 7f5701e1b700 1 -- start start 2024-09-10T11:27:24.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f5701e1b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56fc101760 0x7f56fc195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:24.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f5701e1b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56fc102140 0x7f56fc195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:24.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f5701e1b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc196560 con 0x7f56fc101760 2024-09-10T11:27:24.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f5701e1b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56fc1966a0 con 0x7f56fc102140 2024-09-10T11:27:24.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f56fb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56fc101760 0x7f56fc195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:24.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f56fb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56fc101760 0x7f56fc195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50686/0 (socket says 172.21.15.144:50686) 2024-09-10T11:27:24.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f56fb7fe700 1 -- 172.21.15.144:0/1762340314 learned_addr learned my addr 172.21.15.144:0/1762340314 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:27:24.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f56faffd700 1 --2- 172.21.15.144:0/1762340314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56fc102140 0x7f56fc195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:24.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f56fb7fe700 1 -- 172.21.15.144:0/1762340314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56fc102140 msgr2=0x7f56fc195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:24.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f56fb7fe700 1 --2- 172.21.15.144:0/1762340314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56fc102140 0x7f56fc195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:24.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f56fb7fe700 1 -- 172.21.15.144:0/1762340314 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec0095f0 con 0x7f56fc101760 2024-09-10T11:27:24.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f56faffd700 1 --2- 172.21.15.144:0/1762340314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56fc102140 0x7f56fc195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:27:24.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f56fb7fe700 1 --2- 172.21.15.144:0/1762340314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56fc101760 0x7f56fc195a00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00bda0 tx=0x7f56ec00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:27:24.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f56f8ff9700 1 -- 172.21.15.144:0/1762340314 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56ec01d070 con 0x7f56fc101760 2024-09-10T11:27:24.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f5701e1b700 1 -- 172.21.15.144:0/1762340314 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56fc19b0f0 con 0x7f56fc101760 2024-09-10T11:27:24.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f56f8ff9700 1 -- 172.21.15.144:0/1762340314 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f56ec004c50 con 0x7f56fc101760 2024-09-10T11:27:24.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f56f8ff9700 1 -- 172.21.15.144:0/1762340314 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56ec00f690 con 0x7f56fc101760 2024-09-10T11:27:24.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.626+0000 7f5701e1b700 1 -- 172.21.15.144:0/1762340314 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56fc19b660 con 0x7f56fc101760 2024-09-10T11:27:24.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.628+0000 7f5701e1b700 1 -- 172.21.15.144:0/1762340314 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56fc108d70 con 0x7f56fc101760 2024-09-10T11:27:24.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.628+0000 7f56f8ff9700 1 -- 172.21.15.144:0/1762340314 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f56ec0229b0 con 0x7f56fc101760 2024-09-10T11:27:24.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.628+0000 7f56f8ff9700 1 --2- 172.21.15.144:0/1762340314 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f56e406c680 0x7f56e406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:24.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.629+0000 7f56f8ff9700 1 -- 172.21.15.144:0/1762340314 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f56ec08cf60 con 0x7f56fc101760 2024-09-10T11:27:24.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.629+0000 7f56faffd700 1 --2- 172.21.15.144:0/1762340314 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f56e406c680 0x7f56e406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:24.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.629+0000 7f56faffd700 1 --2- 172.21.15.144:0/1762340314 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f56e406c680 0x7f56e406eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f56fc068d60 tx=0x7f56f0008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:27:24.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.633+0000 7f56f8ff9700 1 -- 172.21.15.144:0/1762340314 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f56ec0bf720 con 0x7f56fc101760 2024-09-10T11:27:24.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.827+0000 7f5701e1b700 1 -- 172.21.15.144:0/1762340314 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f56fc108f30 con 0x7f56fc101760 2024-09-10T11:27:24.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.828+0000 7f56f8ff9700 1 -- 172.21.15.144:0/1762340314 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f56ec05b1d0 con 0x7f56fc101760 2024-09-10T11:27:24.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.830+0000 7f5701e1b700 1 -- 172.21.15.144:0/1762340314 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f56e406c680 msgr2=0x7f56e406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:24.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.830+0000 7f5701e1b700 1 --2- 172.21.15.144:0/1762340314 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f56e406c680 0x7f56e406eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f56fc068d60 tx=0x7f56f0008040 comp rx=0 tx=0).stop 2024-09-10T11:27:24.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.830+0000 7f5701e1b700 1 -- 172.21.15.144:0/1762340314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56fc101760 msgr2=0x7f56fc195a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:24.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.830+0000 7f5701e1b700 1 --2- 172.21.15.144:0/1762340314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56fc101760 0x7f56fc195a00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f56ec00bda0 tx=0x7f56ec00be80 comp rx=0 tx=0).stop 2024-09-10T11:27:24.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.830+0000 7f5701e1b700 1 -- 172.21.15.144:0/1762340314 shutdown_connections 2024-09-10T11:27:24.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.830+0000 7f5701e1b700 1 --2- 172.21.15.144:0/1762340314 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f56e406c680 0x7f56e406eb30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:24.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.830+0000 7f5701e1b700 1 --2- 172.21.15.144:0/1762340314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56fc101760 0x7f56fc195a00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:24.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.830+0000 7f5701e1b700 1 --2- 172.21.15.144:0/1762340314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56fc102140 0x7f56fc195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:24.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.830+0000 7f5701e1b700 1 -- 172.21.15.144:0/1762340314 >> 172.21.15.144:0/1762340314 conn(0x7f56fc0f9db0 msgr2=0x7f56fc104730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:27:24.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.830+0000 7f5701e1b700 1 -- 172.21.15.144:0/1762340314 shutdown_connections 2024-09-10T11:27:24.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:24.830+0000 7f5701e1b700 1 -- 172.21.15.144:0/1762340314 wait complete. 2024-09-10T11:27:25.147 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:24 smithi144 ceph-mon[27942]: pgmap v203: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-10T11:27:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:24 smithi178 ceph-mon[33514]: pgmap v203: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1023 B/s wr, 6 op/s 2024-09-10T11:27:25.529 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph config set global log_to_journald false --force' 2024-09-10T11:27:25.730 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:27:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:26 smithi144 ceph-mon[27942]: pgmap v204: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-10T11:27:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:26 smithi178 ceph-mon[33514]: pgmap v204: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 767 B/s wr, 4 op/s 2024-09-10T11:27:27.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.532+0000 7fac73c67700 1 -- 172.21.15.144:0/3975860780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6c0ff1f0 msgr2=0x7fac6c0ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:27.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.532+0000 7fac73c67700 1 --2- 172.21.15.144:0/3975860780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6c0ff1f0 0x7fac6c0ff660 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fac5c009910 tx=0x7fac5c009c20 comp rx=0 tx=0).stop 2024-09-10T11:27:27.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.532+0000 7fac73c67700 1 -- 172.21.15.144:0/3975860780 shutdown_connections 2024-09-10T11:27:27.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.532+0000 7fac73c67700 1 --2- 172.21.15.144:0/3975860780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6c0ff1f0 0x7fac6c0ff660 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:27.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.532+0000 7fac73c67700 1 --2- 172.21.15.144:0/3975860780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac6c0fe480 0x7fac6c0fe890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:27.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.532+0000 7fac73c67700 1 -- 172.21.15.144:0/3975860780 >> 172.21.15.144:0/3975860780 conn(0x7fac6c0f9dd0 msgr2=0x7fac6c0fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:27:27.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.532+0000 7fac73c67700 1 -- 172.21.15.144:0/3975860780 shutdown_connections 2024-09-10T11:27:27.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.532+0000 7fac73c67700 1 -- 172.21.15.144:0/3975860780 wait complete. 2024-09-10T11:27:27.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.533+0000 7fac73c67700 1 Processor -- start 2024-09-10T11:27:27.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.533+0000 7fac73c67700 1 -- start start 2024-09-10T11:27:27.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.533+0000 7fac73c67700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac6c0fe480 0x7fac6c19bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:27.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.533+0000 7fac73c67700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6c0ff1f0 0x7fac6c19c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:27.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.533+0000 7fac73c67700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c19cb20 con 0x7fac6c0ff1f0 2024-09-10T11:27:27.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.533+0000 7fac73c67700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac6c19cc60 con 0x7fac6c0fe480 2024-09-10T11:27:27.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.533+0000 7fac71202700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6c0ff1f0 0x7fac6c19c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:27.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.533+0000 7fac71202700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6c0ff1f0 0x7fac6c19c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50706/0 (socket says 172.21.15.144:50706) 2024-09-10T11:27:27.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.533+0000 7fac71202700 1 -- 172.21.15.144:0/2878522056 learned_addr learned my addr 172.21.15.144:0/2878522056 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:27:27.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.533+0000 7fac71a03700 1 --2- 172.21.15.144:0/2878522056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac6c0fe480 0x7fac6c19bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:27.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.533+0000 7fac71202700 1 -- 172.21.15.144:0/2878522056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac6c0fe480 msgr2=0x7fac6c19bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:27.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.533+0000 7fac71202700 1 --2- 172.21.15.144:0/2878522056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac6c0fe480 0x7fac6c19bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:27.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.533+0000 7fac71202700 1 -- 172.21.15.144:0/2878522056 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac5c0095f0 con 0x7fac6c0ff1f0 2024-09-10T11:27:27.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.534+0000 7fac71202700 1 --2- 172.21.15.144:0/2878522056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6c0ff1f0 0x7fac6c19c500 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0092e0 tx=0x7fac5c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:27:27.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.534+0000 7fac62ffd700 1 -- 172.21.15.144:0/2878522056 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac5c01d070 con 0x7fac6c0ff1f0 2024-09-10T11:27:27.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.534+0000 7fac62ffd700 1 -- 172.21.15.144:0/2878522056 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fac5c00f460 con 0x7fac6c0ff1f0 2024-09-10T11:27:27.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.534+0000 7fac62ffd700 1 -- 172.21.15.144:0/2878522056 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac5c017630 con 0x7fac6c0ff1f0 2024-09-10T11:27:27.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.534+0000 7fac73c67700 1 -- 172.21.15.144:0/2878522056 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6c1a16b0 con 0x7fac6c0ff1f0 2024-09-10T11:27:27.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.534+0000 7fac73c67700 1 -- 172.21.15.144:0/2878522056 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6c1a1ba0 con 0x7fac6c0ff1f0 2024-09-10T11:27:27.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.535+0000 7fac73c67700 1 -- 172.21.15.144:0/2878522056 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac6c10f310 con 0x7fac6c0ff1f0 2024-09-10T11:27:27.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.535+0000 7fac62ffd700 1 -- 172.21.15.144:0/2878522056 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fac5c00fa00 con 0x7fac6c0ff1f0 2024-09-10T11:27:27.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.536+0000 7fac62ffd700 1 --2- 172.21.15.144:0/2878522056 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fac5806c680 0x7fac5806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:27.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.536+0000 7fac62ffd700 1 -- 172.21.15.144:0/2878522056 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fac5c08cd00 con 0x7fac6c0ff1f0 2024-09-10T11:27:27.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.536+0000 7fac71a03700 1 --2- 172.21.15.144:0/2878522056 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fac5806c680 0x7fac5806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:27.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.536+0000 7fac71a03700 1 --2- 172.21.15.144:0/2878522056 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fac5806c680 0x7fac5806eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fac68005950 tx=0x7fac6800b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:27:27.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.540+0000 7fac62ffd700 1 -- 172.21.15.144:0/2878522056 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fac5c090030 con 0x7fac6c0ff1f0 2024-09-10T11:27:27.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.740+0000 7fac73c67700 1 -- 172.21.15.144:0/2878522056 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fac6c1a22a0 con 0x7fac6c0ff1f0 2024-09-10T11:27:27.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.740+0000 7fac62ffd700 1 -- 172.21.15.144:0/2878522056 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7fac5c05aec0 con 0x7fac6c0ff1f0 2024-09-10T11:27:27.743 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.742+0000 7fac73c67700 1 -- 172.21.15.144:0/2878522056 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fac5806c680 msgr2=0x7fac5806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:27.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.742+0000 7fac73c67700 1 --2- 172.21.15.144:0/2878522056 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fac5806c680 0x7fac5806eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fac68005950 tx=0x7fac6800b410 comp rx=0 tx=0).stop 2024-09-10T11:27:27.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.742+0000 7fac73c67700 1 -- 172.21.15.144:0/2878522056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6c0ff1f0 msgr2=0x7fac6c19c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:27.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.742+0000 7fac73c67700 1 --2- 172.21.15.144:0/2878522056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6c0ff1f0 0x7fac6c19c500 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fac5c0092e0 tx=0x7fac5c00bd70 comp rx=0 tx=0).stop 2024-09-10T11:27:27.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.742+0000 7fac73c67700 1 -- 172.21.15.144:0/2878522056 shutdown_connections 2024-09-10T11:27:27.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.742+0000 7fac73c67700 1 --2- 172.21.15.144:0/2878522056 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fac5806c680 0x7fac5806eb30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:27.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.742+0000 7fac73c67700 1 --2- 172.21.15.144:0/2878522056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac6c0fe480 0x7fac6c19bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:27.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.742+0000 7fac73c67700 1 --2- 172.21.15.144:0/2878522056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac6c0ff1f0 0x7fac6c19c500 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:27.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.742+0000 7fac73c67700 1 -- 172.21.15.144:0/2878522056 >> 172.21.15.144:0/2878522056 conn(0x7fac6c0f9dd0 msgr2=0x7fac6c102420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:27:27.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.742+0000 7fac73c67700 1 -- 172.21.15.144:0/2878522056 shutdown_connections 2024-09-10T11:27:27.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:27.742+0000 7fac73c67700 1 -- 172.21.15.144:0/2878522056 wait complete. 2024-09-10T11:27:28.428 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-09-10T11:27:28.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:28 smithi178 ceph-mon[33514]: pgmap v205: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 852 B/s wr, 5 op/s 2024-09-10T11:27:28.628 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:27:28.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:28 smithi144 ceph-mon[27942]: pgmap v205: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.7 KiB/s rd, 852 B/s wr, 5 op/s 2024-09-10T11:27:30.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.182+0000 7f4e3ff12700 1 -- 172.21.15.144:0/1733049757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e38100170 msgr2=0x7f4e38100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:30.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.182+0000 7f4e3ff12700 1 --2- 172.21.15.144:0/1733049757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e38100170 0x7f4e38100580 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4e34009910 tx=0x7f4e34009c20 comp rx=0 tx=0).stop 2024-09-10T11:27:30.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.182+0000 7f4e3ff12700 1 -- 172.21.15.144:0/1733049757 shutdown_connections 2024-09-10T11:27:30.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.182+0000 7f4e3ff12700 1 --2- 172.21.15.144:0/1733049757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e38101370 0x7f4e381017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:30.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.182+0000 7f4e3ff12700 1 --2- 172.21.15.144:0/1733049757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e38100170 0x7f4e38100580 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:30.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.182+0000 7f4e3ff12700 1 -- 172.21.15.144:0/1733049757 >> 172.21.15.144:0/1733049757 conn(0x7f4e380fb6e0 msgr2=0x7f4e380fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:27:30.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.183+0000 7f4e3ff12700 1 -- 172.21.15.144:0/1733049757 shutdown_connections 2024-09-10T11:27:30.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.183+0000 7f4e3ff12700 1 -- 172.21.15.144:0/1733049757 wait complete. 2024-09-10T11:27:30.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.183+0000 7f4e3ff12700 1 Processor -- start 2024-09-10T11:27:30.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.183+0000 7f4e3ff12700 1 -- start start 2024-09-10T11:27:30.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.183+0000 7f4e3ff12700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e38100170 0x7f4e38197c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:30.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.183+0000 7f4e3ff12700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e38101370 0x7f4e38198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:30.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.183+0000 7f4e3ff12700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e381987a0 con 0x7f4e38100170 2024-09-10T11:27:30.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.183+0000 7f4e3ff12700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3819d1b0 con 0x7f4e38101370 2024-09-10T11:27:30.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.183+0000 7f4e3dcae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e38100170 0x7f4e38197c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:30.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.183+0000 7f4e3dcae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e38100170 0x7f4e38197c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50726/0 (socket says 172.21.15.144:50726) 2024-09-10T11:27:30.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.183+0000 7f4e3dcae700 1 -- 172.21.15.144:0/2662841994 learned_addr learned my addr 172.21.15.144:0/2662841994 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:27:30.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.184+0000 7f4e3d4ad700 1 --2- 172.21.15.144:0/2662841994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e38101370 0x7f4e38198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:30.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.184+0000 7f4e3dcae700 1 -- 172.21.15.144:0/2662841994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e38101370 msgr2=0x7f4e38198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:27:30.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.184+0000 7f4e3dcae700 1 --2- 172.21.15.144:0/2662841994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e38101370 0x7f4e38198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:27:30.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.184+0000 7f4e3dcae700 1 -- 172.21.15.144:0/2662841994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e340095f0 con 0x7f4e38100170 2024-09-10T11:27:30.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.184+0000 7f4e3dcae700 1 --2- 172.21.15.144:0/2662841994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e38100170 0x7f4e38197c40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400bda0 tx=0x7f4e3400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:27:30.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.184+0000 7f4e2effd700 1 -- 172.21.15.144:0/2662841994 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e3401d070 con 0x7f4e38100170 2024-09-10T11:27:30.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.184+0000 7f4e2effd700 1 -- 172.21.15.144:0/2662841994 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4e34022470 con 0x7f4e38100170 2024-09-10T11:27:30.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.184+0000 7f4e3ff12700 1 -- 172.21.15.144:0/2662841994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3819d350 con 0x7f4e38100170 2024-09-10T11:27:30.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.184+0000 7f4e2effd700 1 -- 172.21.15.144:0/2662841994 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e3400f650 con 0x7f4e38100170 2024-09-10T11:27:30.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.184+0000 7f4e3ff12700 1 -- 172.21.15.144:0/2662841994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3819d890 con 0x7f4e38100170 2024-09-10T11:27:30.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.185+0000 7f4e3ff12700 1 -- 172.21.15.144:0/2662841994 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e38105450 con 0x7f4e38100170 2024-09-10T11:27:30.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.186+0000 7f4e2effd700 1 -- 172.21.15.144:0/2662841994 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f4e340225e0 con 0x7f4e38100170 2024-09-10T11:27:30.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.186+0000 7f4e2effd700 1 --2- 172.21.15.144:0/2662841994 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4e2406c680 0x7f4e2406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:27:30.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.186+0000 7f4e2effd700 1 -- 172.21.15.144:0/2662841994 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f4e3408ce60 con 0x7f4e38100170 2024-09-10T11:27:30.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.186+0000 7f4e3d4ad700 1 --2- 172.21.15.144:0/2662841994 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4e2406c680 0x7f4e2406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:27:30.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.186+0000 7f4e3d4ad700 1 --2- 172.21.15.144:0/2662841994 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4e2406c680 0x7f4e2406eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009a10 tx=0x7f4e28009190 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:27:30.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.190+0000 7f4e2effd700 1 -- 172.21.15.144:0/2662841994 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4e34057640 con 0x7f4e38100170 2024-09-10T11:27:30.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:27:30.397+0000 7f4e3ff12700 1 -- 172.21.15.144:0/2662841994 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f4e38061240 con 0x7f4e2406c680 2024-09-10T11:27:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:30 smithi178 ceph-mon[33514]: pgmap v206: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-10T11:27:31.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:30 smithi144 ceph-mon[27942]: pgmap v206: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.5 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-10T11:27:32.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:31 smithi178 ceph-mon[33514]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:27:32.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:31 smithi144 ceph-mon[27942]: from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:27:33.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:32 smithi144 ceph-mon[27942]: pgmap v207: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-10T11:27:33.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:32 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:27:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:32 smithi178 ceph-mon[33514]: pgmap v207: 65 pgs: 65 active+clean; 587 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-10T11:27:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:32 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:27:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:34 smithi178 ceph-mon[33514]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-10T11:27:35.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:34 smithi144 ceph-mon[27942]: pgmap v208: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-10T11:27:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:37 smithi178 ceph-mon[33514]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-10T11:27:37.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:37 smithi144 ceph-mon[27942]: pgmap v209: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 4 op/s 2024-09-10T11:27:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:38 smithi178 ceph-mon[33514]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-10T11:27:38.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:38 smithi144 ceph-mon[27942]: pgmap v210: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-10T11:27:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:40 smithi178 ceph-mon[33514]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T11:27:41.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:40 smithi144 ceph-mon[27942]: pgmap v211: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T11:27:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:42 smithi178 ceph-mon[33514]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-10T11:27:43.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:42 smithi144 ceph-mon[27942]: pgmap v212: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-10T11:27:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:44 smithi178 ceph-mon[33514]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T11:27:45.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:44 smithi144 ceph-mon[27942]: pgmap v213: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T11:27:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:46 smithi178 ceph-mon[33514]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:27:47.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:46 smithi144 ceph-mon[27942]: pgmap v214: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:27:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:27:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:48 smithi144 ceph-mon[27942]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:27:48.866 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:27:48.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:48 smithi178 ceph-mon[33514]: pgmap v215: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:27:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:50 smithi178 ceph-mon[33514]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:27:51.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:50 smithi144 ceph-mon[27942]: pgmap v216: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:27:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:52 smithi178 ceph-mon[33514]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:27:53.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:52 smithi144 ceph-mon[27942]: pgmap v217: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:27:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:54 smithi178 ceph-mon[33514]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:27:55.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:54 smithi144 ceph-mon[27942]: pgmap v218: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:27:57.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:57 smithi144 ceph-mon[27942]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:27:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:57 smithi178 ceph-mon[33514]: pgmap v219: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:27:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:27:58 smithi144 ceph-mon[27942]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:27:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:27:58 smithi178 ceph-mon[33514]: pgmap v220: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:28:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:00 smithi144 ceph-mon[27942]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:28:01.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:00 smithi178 ceph-mon[33514]: pgmap v221: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:28:02.266 INFO:teuthology.orchestra.run.smithi144.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:28:02.266 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:02.263+0000 7f4e2effd700 1 -- 172.21.15.144:0/2662841994 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f4e38061240 con 0x7f4e2406c680 2024-09-10T11:28:02.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:02.266+0000 7f4e3ff12700 1 -- 172.21.15.144:0/2662841994 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4e2406c680 msgr2=0x7f4e2406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:02.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:02.266+0000 7f4e3ff12700 1 --2- 172.21.15.144:0/2662841994 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4e2406c680 0x7f4e2406eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009a10 tx=0x7f4e28009190 comp rx=0 tx=0).stop 2024-09-10T11:28:02.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:02.266+0000 7f4e3ff12700 1 -- 172.21.15.144:0/2662841994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e38100170 msgr2=0x7f4e38197c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:02.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:02.266+0000 7f4e3ff12700 1 --2- 172.21.15.144:0/2662841994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e38100170 0x7f4e38197c40 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4e3400bda0 tx=0x7f4e3400bdd0 comp rx=0 tx=0).stop 2024-09-10T11:28:02.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:02.266+0000 7f4e3ff12700 1 -- 172.21.15.144:0/2662841994 shutdown_connections 2024-09-10T11:28:02.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:02.266+0000 7f4e3ff12700 1 --2- 172.21.15.144:0/2662841994 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f4e2406c680 0x7f4e2406eb30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:02.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:02.266+0000 7f4e3ff12700 1 --2- 172.21.15.144:0/2662841994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4e38100170 0x7f4e38197c40 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:02.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:02.266+0000 7f4e3ff12700 1 --2- 172.21.15.144:0/2662841994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4e38101370 0x7f4e38198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:02.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:02.266+0000 7f4e3ff12700 1 -- 172.21.15.144:0/2662841994 >> 172.21.15.144:0/2662841994 conn(0x7f4e380fb6e0 msgr2=0x7f4e38102590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:02.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:02.266+0000 7f4e3ff12700 1 -- 172.21.15.144:0/2662841994 shutdown_connections 2024-09-10T11:28:02.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:02.266+0000 7f4e3ff12700 1 -- 172.21.15.144:0/2662841994 wait complete. 2024-09-10T11:28:03.085 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-09-10T11:28:03.356 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:28:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:03 smithi144 ceph-mon[27942]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:28:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:03 smithi144 ceph-mon[27942]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:28:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:28:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:28:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:28:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:28:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:03 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:03.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:03 smithi178 ceph-mon[33514]: pgmap v222: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:28:03.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:03 smithi178 ceph-mon[33514]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:28:03.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:03 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:03.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:03 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:28:03.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:03 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:28:03.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:03 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:28:03.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:03 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:28:03.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:03 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:04.511 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:04 smithi144 ceph-mon[27942]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:28:04.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:04 smithi178 ceph-mon[33514]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:28:05.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:05 smithi144 ceph-mon[27942]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:28:05.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:05 smithi178 ceph-mon[33514]: pgmap v223: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:28:06.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.343+0000 7f9eabf04700 1 -- 172.21.15.144:0/3299647250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea41021a0 msgr2=0x7f9ea40fe260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:06.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.343+0000 7f9eabf04700 1 --2- 172.21.15.144:0/3299647250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea41021a0 0x7f9ea40fe260 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009910 tx=0x7f9e94009c20 comp rx=0 tx=0).stop 2024-09-10T11:28:06.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.343+0000 7f9eabf04700 1 -- 172.21.15.144:0/3299647250 shutdown_connections 2024-09-10T11:28:06.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.343+0000 7f9eabf04700 1 --2- 172.21.15.144:0/3299647250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea41021a0 0x7f9ea40fe260 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:06.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.343+0000 7f9eabf04700 1 --2- 172.21.15.144:0/3299647250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea41017c0 0x7f9ea4101bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:06.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.343+0000 7f9eabf04700 1 -- 172.21.15.144:0/3299647250 >> 172.21.15.144:0/3299647250 conn(0x7f9ea40f9dd0 msgr2=0x7f9ea40fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:06.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.343+0000 7f9eabf04700 1 -- 172.21.15.144:0/3299647250 shutdown_connections 2024-09-10T11:28:06.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.343+0000 7f9eabf04700 1 -- 172.21.15.144:0/3299647250 wait complete. 2024-09-10T11:28:06.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.345+0000 7f9eabf04700 1 Processor -- start 2024-09-10T11:28:06.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.345+0000 7f9eabf04700 1 -- start start 2024-09-10T11:28:06.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.345+0000 7f9eabf04700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea41017c0 0x7f9ea4195a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:06.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.345+0000 7f9eabf04700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea41021a0 0x7f9ea4195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:06.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.345+0000 7f9ea9ca0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea41017c0 0x7f9ea4195a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:06.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.345+0000 7f9ea9ca0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea41017c0 0x7f9ea4195a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33078/0 (socket says 172.21.15.144:33078) 2024-09-10T11:28:06.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.345+0000 7f9ea9ca0700 1 -- 172.21.15.144:0/2431221053 learned_addr learned my addr 172.21.15.144:0/2431221053 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:28:06.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.345+0000 7f9eabf04700 1 -- 172.21.15.144:0/2431221053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea41965a0 con 0x7f9ea41017c0 2024-09-10T11:28:06.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.345+0000 7f9eabf04700 1 -- 172.21.15.144:0/2431221053 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea41966e0 con 0x7f9ea41021a0 2024-09-10T11:28:06.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.346+0000 7f9ea9ca0700 1 -- 172.21.15.144:0/2431221053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea41021a0 msgr2=0x7f9ea4195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:06.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.346+0000 7f9ea9ca0700 1 --2- 172.21.15.144:0/2431221053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea41021a0 0x7f9ea4195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:06.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.346+0000 7f9ea9ca0700 1 -- 172.21.15.144:0/2431221053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e940095f0 con 0x7f9ea41017c0 2024-09-10T11:28:06.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.346+0000 7f9ea9ca0700 1 --2- 172.21.15.144:0/2431221053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea41017c0 0x7f9ea4195a40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000e8f0 tx=0x7f9ea000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:06.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.346+0000 7f9e9affd700 1 -- 172.21.15.144:0/2431221053 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ea0009790 con 0x7f9ea41017c0 2024-09-10T11:28:06.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.346+0000 7f9eabf04700 1 -- 172.21.15.144:0/2431221053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea419b190 con 0x7f9ea41017c0 2024-09-10T11:28:06.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.346+0000 7f9e9affd700 1 -- 172.21.15.144:0/2431221053 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9ea00044d0 con 0x7f9ea41017c0 2024-09-10T11:28:06.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.346+0000 7f9eabf04700 1 -- 172.21.15.144:0/2431221053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea419b680 con 0x7f9ea41017c0 2024-09-10T11:28:06.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.347+0000 7f9e9affd700 1 -- 172.21.15.144:0/2431221053 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ea0003d70 con 0x7f9ea41017c0 2024-09-10T11:28:06.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.348+0000 7f9eabf04700 1 -- 172.21.15.144:0/2431221053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea4108d90 con 0x7f9ea41017c0 2024-09-10T11:28:06.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.348+0000 7f9e9affd700 1 -- 172.21.15.144:0/2431221053 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f9ea0004020 con 0x7f9ea41017c0 2024-09-10T11:28:06.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.349+0000 7f9e9affd700 1 --2- 172.21.15.144:0/2431221053 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9e9006c680 0x7f9e9006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:06.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.349+0000 7f9e9affd700 1 -- 172.21.15.144:0/2431221053 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f9ea0014070 con 0x7f9ea41017c0 2024-09-10T11:28:06.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.349+0000 7f9ea949f700 1 --2- 172.21.15.144:0/2431221053 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9e9006c680 0x7f9e9006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:06.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.350+0000 7f9ea949f700 1 --2- 172.21.15.144:0/2431221053 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9e9006c680 0x7f9e9006eb30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009fd0 tx=0x7f9e94000bc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:06.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.353+0000 7f9e9affd700 1 -- 172.21.15.144:0/2431221053 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9ea0059ef0 con 0x7f9ea41017c0 2024-09-10T11:28:06.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.774+0000 7f9eabf04700 1 -- 172.21.15.144:0/2431221053 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ea4105eb0 con 0x7f9e9006c680 2024-09-10T11:28:06.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.776+0000 7f9e9affd700 1 -- 172.21.15.144:0/2431221053 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f9ea4105eb0 con 0x7f9e9006c680 2024-09-10T11:28:06.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.779+0000 7f9eabf04700 1 -- 172.21.15.144:0/2431221053 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9e9006c680 msgr2=0x7f9e9006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:06.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.779+0000 7f9eabf04700 1 --2- 172.21.15.144:0/2431221053 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9e9006c680 0x7f9e9006eb30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f9e94009fd0 tx=0x7f9e94000bc0 comp rx=0 tx=0).stop 2024-09-10T11:28:06.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.779+0000 7f9eabf04700 1 -- 172.21.15.144:0/2431221053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea41017c0 msgr2=0x7f9ea4195a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:06.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.779+0000 7f9eabf04700 1 --2- 172.21.15.144:0/2431221053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea41017c0 0x7f9ea4195a40 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000e8f0 tx=0x7f9ea000ecb0 comp rx=0 tx=0).stop 2024-09-10T11:28:06.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.779+0000 7f9eabf04700 1 -- 172.21.15.144:0/2431221053 shutdown_connections 2024-09-10T11:28:06.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.779+0000 7f9eabf04700 1 --2- 172.21.15.144:0/2431221053 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9e9006c680 0x7f9e9006eb30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:06.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.779+0000 7f9eabf04700 1 --2- 172.21.15.144:0/2431221053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ea41017c0 0x7f9ea4195a40 secure :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9ea000e8f0 tx=0x7f9ea000ecb0 comp rx=0 tx=0).stop 2024-09-10T11:28:06.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.779+0000 7f9eabf04700 1 --2- 172.21.15.144:0/2431221053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ea41021a0 0x7f9ea4195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:06.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.779+0000 7f9eabf04700 1 -- 172.21.15.144:0/2431221053 >> 172.21.15.144:0/2431221053 conn(0x7f9ea40f9dd0 msgr2=0x7f9ea4104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:06.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.779+0000 7f9eabf04700 1 -- 172.21.15.144:0/2431221053 shutdown_connections 2024-09-10T11:28:06.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.779+0000 7f9eabf04700 1 -- 172.21.15.144:0/2431221053 wait complete. 2024-09-10T11:28:06.796 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:28:06.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.948+0000 7f9d4e7b0700 1 -- 172.21.15.144:0/2996104261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d48100250 msgr2=0x7f9d481006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:06.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.948+0000 7f9d4e7b0700 1 --2- 172.21.15.144:0/2996104261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d48100250 0x7f9d481006a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c009910 tx=0x7f9d3c009c20 comp rx=0 tx=0).stop 2024-09-10T11:28:06.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.948+0000 7f9d4e7b0700 1 -- 172.21.15.144:0/2996104261 shutdown_connections 2024-09-10T11:28:06.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.949+0000 7f9d4e7b0700 1 --2- 172.21.15.144:0/2996104261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d48100250 0x7f9d481006a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:06.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.949+0000 7f9d4e7b0700 1 --2- 172.21.15.144:0/2996104261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d480ff050 0x7f9d480ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:06.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.949+0000 7f9d4e7b0700 1 -- 172.21.15.144:0/2996104261 >> 172.21.15.144:0/2996104261 conn(0x7f9d480fa5e0 msgr2=0x7f9d480fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:06.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.949+0000 7f9d4e7b0700 1 -- 172.21.15.144:0/2996104261 shutdown_connections 2024-09-10T11:28:06.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.949+0000 7f9d4e7b0700 1 -- 172.21.15.144:0/2996104261 wait complete. 2024-09-10T11:28:06.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.949+0000 7f9d4e7b0700 1 Processor -- start 2024-09-10T11:28:06.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.950+0000 7f9d4e7b0700 1 -- start start 2024-09-10T11:28:06.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.950+0000 7f9d4e7b0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d480ff050 0x7f9d48068f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:06.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.950+0000 7f9d4e7b0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d481988c0 0x7f9d4819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:06.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.950+0000 7f9d4e7b0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4819b2c0 con 0x7f9d480ff050 2024-09-10T11:28:06.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.950+0000 7f9d4e7b0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d4819b400 con 0x7f9d481988c0 2024-09-10T11:28:06.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.950+0000 7f9d47fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d480ff050 0x7f9d48068f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:06.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.950+0000 7f9d477fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d481988c0 0x7f9d4819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:06.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.950+0000 7f9d477fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d481988c0 0x7f9d4819aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.144:47018/0 (socket says 172.21.15.144:47018) 2024-09-10T11:28:06.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.950+0000 7f9d477fe700 1 -- 172.21.15.144:0/1952391445 learned_addr learned my addr 172.21.15.144:0/1952391445 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:28:06.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.951+0000 7f9d477fe700 1 -- 172.21.15.144:0/1952391445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d480ff050 msgr2=0x7f9d48068f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:06.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.951+0000 7f9d477fe700 1 --2- 172.21.15.144:0/1952391445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d480ff050 0x7f9d48068f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:06.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.951+0000 7f9d477fe700 1 -- 172.21.15.144:0/1952391445 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3c0095f0 con 0x7f9d481988c0 2024-09-10T11:28:06.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.951+0000 7f9d477fe700 1 --2- 172.21.15.144:0/1952391445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d481988c0 0x7f9d4819aca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c005b40 tx=0x7f9d3c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:06.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.951+0000 7f9d457fa700 1 -- 172.21.15.144:0/1952391445 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d3c01d070 con 0x7f9d481988c0 2024-09-10T11:28:06.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.951+0000 7f9d4e7b0700 1 -- 172.21.15.144:0/1952391445 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d481946a0 con 0x7f9d481988c0 2024-09-10T11:28:06.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.951+0000 7f9d457fa700 1 -- 172.21.15.144:0/1952391445 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9d3c005dc0 con 0x7f9d481988c0 2024-09-10T11:28:06.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.951+0000 7f9d457fa700 1 -- 172.21.15.144:0/1952391445 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d3c022470 con 0x7f9d481988c0 2024-09-10T11:28:06.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.951+0000 7f9d4e7b0700 1 -- 172.21.15.144:0/1952391445 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d48194b90 con 0x7f9d481988c0 2024-09-10T11:28:06.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.953+0000 7f9d4e7b0700 1 -- 172.21.15.144:0/1952391445 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d48107ba0 con 0x7f9d481988c0 2024-09-10T11:28:06.955 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.954+0000 7f9d457fa700 1 -- 172.21.15.144:0/1952391445 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f9d3c003680 con 0x7f9d481988c0 2024-09-10T11:28:06.955 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.954+0000 7f9d457fa700 1 --2- 172.21.15.144:0/1952391445 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9d3006c680 0x7f9d3006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:06.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.954+0000 7f9d457fa700 1 -- 172.21.15.144:0/1952391445 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f9d3c08ccc0 con 0x7f9d481988c0 2024-09-10T11:28:06.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.955+0000 7f9d47fff700 1 --2- 172.21.15.144:0/1952391445 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9d3006c680 0x7f9d3006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:06.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.955+0000 7f9d47fff700 1 --2- 172.21.15.144:0/1952391445 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9d3006c680 0x7f9d3006eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9d481000b0 tx=0x7f9d38012040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:06.964 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:06.963+0000 7f9d457fa700 1 -- 172.21.15.144:0/1952391445 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9d3c05ae80 con 0x7f9d481988c0 2024-09-10T11:28:07.334 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:07 smithi144 ceph-mon[27942]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:28:07.334 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:07.334 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:07 smithi144 ceph-mon[27942]: Upgrade: Target is version 19.1.1-305-g1cf819b7 (unknown) 2024-09-10T11:28:07.334 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:07 smithi144 ceph-mon[27942]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, digests ['quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559'] 2024-09-10T11:28:07.334 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:28:07.335 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:07 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:07.335 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:07 smithi144 ceph-mon[27942]: Upgrade: Need to upgrade myself (mgr.smithi144.bkogfs) 2024-09-10T11:28:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:07 smithi178 ceph-mon[33514]: pgmap v224: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:28:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:07 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:07 smithi178 ceph-mon[33514]: Upgrade: Target is version 19.1.1-305-g1cf819b7 (unknown) 2024-09-10T11:28:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:07 smithi178 ceph-mon[33514]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, digests ['quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559'] 2024-09-10T11:28:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:07 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:28:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:07 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:07 smithi178 ceph-mon[33514]: Upgrade: Need to upgrade myself (mgr.smithi144.bkogfs) 2024-09-10T11:28:07.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.387+0000 7f9d4e7b0700 1 -- 172.21.15.144:0/1952391445 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d48104ba0 con 0x7f9d3006c680 2024-09-10T11:28:07.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.388+0000 7f9d457fa700 1 -- 172.21.15.144:0/1952391445 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f9d48104ba0 con 0x7f9d3006c680 2024-09-10T11:28:07.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.391+0000 7f9d4e7b0700 1 -- 172.21.15.144:0/1952391445 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9d3006c680 msgr2=0x7f9d3006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:07.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.391+0000 7f9d4e7b0700 1 --2- 172.21.15.144:0/1952391445 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9d3006c680 0x7f9d3006eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9d481000b0 tx=0x7f9d38012040 comp rx=0 tx=0).stop 2024-09-10T11:28:07.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.391+0000 7f9d4e7b0700 1 -- 172.21.15.144:0/1952391445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d481988c0 msgr2=0x7f9d4819aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:07.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.391+0000 7f9d4e7b0700 1 --2- 172.21.15.144:0/1952391445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d481988c0 0x7f9d4819aca0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f9d3c005b40 tx=0x7f9d3c00fa00 comp rx=0 tx=0).stop 2024-09-10T11:28:07.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.392+0000 7f9d4e7b0700 1 -- 172.21.15.144:0/1952391445 shutdown_connections 2024-09-10T11:28:07.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.392+0000 7f9d4e7b0700 1 --2- 172.21.15.144:0/1952391445 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f9d3006c680 0x7f9d3006eb30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:07.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.392+0000 7f9d4e7b0700 1 --2- 172.21.15.144:0/1952391445 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d480ff050 0x7f9d48068f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:07.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.392+0000 7f9d4e7b0700 1 --2- 172.21.15.144:0/1952391445 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d481988c0 0x7f9d4819aca0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:07.395 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.392+0000 7f9d4e7b0700 1 -- 172.21.15.144:0/1952391445 >> 172.21.15.144:0/1952391445 conn(0x7f9d480fa5e0 msgr2=0x7f9d48103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:07.395 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.392+0000 7f9d4e7b0700 1 -- 172.21.15.144:0/1952391445 shutdown_connections 2024-09-10T11:28:07.395 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.392+0000 7f9d4e7b0700 1 -- 172.21.15.144:0/1952391445 wait complete. 2024-09-10T11:28:07.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.536+0000 7f1794e7e700 1 -- 172.21.15.144:0/3258653325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17900fe410 msgr2=0x7f17900fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:07.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.536+0000 7f1794e7e700 1 --2- 172.21.15.144:0/3258653325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17900fe410 0x7f17900fe820 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f1780009910 tx=0x7f1780009c20 comp rx=0 tx=0).stop 2024-09-10T11:28:07.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.536+0000 7f1794e7e700 1 -- 172.21.15.144:0/3258653325 shutdown_connections 2024-09-10T11:28:07.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.536+0000 7f1794e7e700 1 --2- 172.21.15.144:0/3258653325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17900ff180 0x7f17900ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:07.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.536+0000 7f1794e7e700 1 --2- 172.21.15.144:0/3258653325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17900fe410 0x7f17900fe820 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:07.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.536+0000 7f1794e7e700 1 -- 172.21.15.144:0/3258653325 >> 172.21.15.144:0/3258653325 conn(0x7f17900f9d80 msgr2=0x7f17900fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:07.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.536+0000 7f1794e7e700 1 -- 172.21.15.144:0/3258653325 shutdown_connections 2024-09-10T11:28:07.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.536+0000 7f1794e7e700 1 -- 172.21.15.144:0/3258653325 wait complete. 2024-09-10T11:28:07.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.537+0000 7f1794e7e700 1 Processor -- start 2024-09-10T11:28:07.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.537+0000 7f1794e7e700 1 -- start start 2024-09-10T11:28:07.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.537+0000 7f1794e7e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17900fe410 0x7f179019bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:07.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.538+0000 7f1794e7e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17900ff180 0x7f179019c500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:07.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.538+0000 7f1794e7e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179019cb20 con 0x7f17900ff180 2024-09-10T11:28:07.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.538+0000 7f1794e7e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179019cc60 con 0x7f17900fe410 2024-09-10T11:28:07.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.538+0000 7f178e59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17900fe410 0x7f179019bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:07.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.538+0000 7f178dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17900ff180 0x7f179019c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:07.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.538+0000 7f178dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17900ff180 0x7f179019c500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33122/0 (socket says 172.21.15.144:33122) 2024-09-10T11:28:07.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.538+0000 7f178dd9b700 1 -- 172.21.15.144:0/3154344609 learned_addr learned my addr 172.21.15.144:0/3154344609 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:28:07.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.539+0000 7f178dd9b700 1 -- 172.21.15.144:0/3154344609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17900fe410 msgr2=0x7f179019bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:07.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.539+0000 7f178dd9b700 1 --2- 172.21.15.144:0/3154344609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17900fe410 0x7f179019bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:07.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.539+0000 7f178dd9b700 1 -- 172.21.15.144:0/3154344609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17800095f0 con 0x7f17900ff180 2024-09-10T11:28:07.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.539+0000 7f178dd9b700 1 --2- 172.21.15.144:0/3154344609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17900ff180 0x7f179019c500 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f178400b4b0 tx=0x7f178400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:07.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.540+0000 7f177f7fe700 1 -- 172.21.15.144:0/3154344609 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1784011800 con 0x7f17900ff180 2024-09-10T11:28:07.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.540+0000 7f1794e7e700 1 -- 172.21.15.144:0/3154344609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17901a1710 con 0x7f17900ff180 2024-09-10T11:28:07.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.540+0000 7f1794e7e700 1 -- 172.21.15.144:0/3154344609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17901a1ce0 con 0x7f17900ff180 2024-09-10T11:28:07.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.540+0000 7f177f7fe700 1 -- 172.21.15.144:0/3154344609 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1784011e10 con 0x7f17900ff180 2024-09-10T11:28:07.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.540+0000 7f177f7fe700 1 -- 172.21.15.144:0/3154344609 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f178400f2e0 con 0x7f17900ff180 2024-09-10T11:28:07.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.542+0000 7f1794e7e700 1 -- 172.21.15.144:0/3154344609 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1770005320 con 0x7f17900ff180 2024-09-10T11:28:07.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.544+0000 7f177f7fe700 1 -- 172.21.15.144:0/3154344609 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f1784011960 con 0x7f17900ff180 2024-09-10T11:28:07.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.545+0000 7f177f7fe700 1 --2- 172.21.15.144:0/3154344609 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f177806c6d0 0x7f177806eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:07.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.545+0000 7f177f7fe700 1 -- 172.21.15.144:0/3154344609 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f178408b8c0 con 0x7f17900ff180 2024-09-10T11:28:07.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.545+0000 7f178e59c700 1 --2- 172.21.15.144:0/3154344609 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f177806c6d0 0x7f177806eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:07.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.546+0000 7f178e59c700 1 --2- 172.21.15.144:0/3154344609 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f177806c6d0 0x7f177806eb80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f178000b5c0 tx=0x7f1780005fb0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:07.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.551+0000 7f177f7fe700 1 -- 172.21.15.144:0/3154344609 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1784053720 con 0x7f17900ff180 2024-09-10T11:28:07.927 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.925+0000 7f1794e7e700 1 -- 172.21.15.144:0/3154344609 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1770000bf0 con 0x7f177806c6d0 2024-09-10T11:28:07.947 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.946+0000 7f177f7fe700 1 -- 172.21.15.144:0/3154344609 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7f1770000bf0 con 0x7f177806c6d0 2024-09-10T11:28:07.948 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:28:07.948 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (5m) 61s ago 6m 25.0M - 0.25.0 c8568f914cd2 b5c0bfae59de 2024-09-10T11:28:07.948 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (7m) 61s ago 7m 7465k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:28:07.948 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (6m) 69s ago 6m 7411k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:28:07.948 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (7m) 61s ago 7m 6665k - 18.2.0 dc2bc1663786 3311fb04aa0c 2024-09-10T11:28:07.948 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (6m) 69s ago 6m 6651k - 18.2.0 dc2bc1663786 6e408e8476d4 2024-09-10T11:28:07.948 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (5m) 61s ago 6m 84.7M - 9.4.7 954c08fa6188 e9a1aec955f6 2024-09-10T11:28:07.948 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (90s) 61s ago 89s 12.6M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:28:07.948 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (83s) 61s ago 82s 16.2M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:28:07.949 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (80s) 69s ago 80s 16.1M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:28:07.949 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (87s) 69s ago 86s 18.4M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:28:07.949 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:9283,8765,8443 running (8m) 61s ago 8m 487M - 18.2.0 dc2bc1663786 71a7e1c8a428 2024-09-10T11:28:07.949 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (5m) 69s ago 5m 437M - 18.2.0 dc2bc1663786 86f93471b98b 2024-09-10T11:28:07.949 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (8m) 61s ago 8m 44.4M 2048M 18.2.0 dc2bc1663786 42addb654e4f 2024-09-10T11:28:07.949 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (5m) 69s ago 5m 35.3M 2048M 18.2.0 dc2bc1663786 3c1ba210ed3f 2024-09-10T11:28:07.949 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (7m) 61s ago 7m 16.0M - 1.5.0 0da6a335fe13 dfbf0c488d17 2024-09-10T11:28:07.949 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (6m) 69s ago 6m 15.2M - 1.5.0 0da6a335fe13 ff2cac118c8d 2024-09-10T11:28:07.949 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (5m) 61s ago 5m 47.9M 3946M 18.2.0 dc2bc1663786 2435febebdb5 2024-09-10T11:28:07.949 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (4m) 61s ago 4m 48.0M 3946M 18.2.0 dc2bc1663786 b55fe920ff9d 2024-09-10T11:28:07.949 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (4m) 61s ago 4m 46.3M 3946M 18.2.0 dc2bc1663786 3ed5618eac33 2024-09-10T11:28:07.949 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (3m) 69s ago 3m 46.6M 4970M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:28:07.949 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (3m) 69s ago 3m 44.0M 4970M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:28:07.950 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (2m) 69s ago 2m 44.5M 4970M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:28:07.950 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (5m) 61s ago 6m 60.8M - 2.43.0 a07b618ecd1d a463a804d232 2024-09-10T11:28:07.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.949+0000 7f1794e7e700 1 -- 172.21.15.144:0/3154344609 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f177806c6d0 msgr2=0x7f177806eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:07.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.949+0000 7f1794e7e700 1 --2- 172.21.15.144:0/3154344609 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f177806c6d0 0x7f177806eb80 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f178000b5c0 tx=0x7f1780005fb0 comp rx=0 tx=0).stop 2024-09-10T11:28:07.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.949+0000 7f1794e7e700 1 -- 172.21.15.144:0/3154344609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17900ff180 msgr2=0x7f179019c500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:07.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.949+0000 7f1794e7e700 1 --2- 172.21.15.144:0/3154344609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17900ff180 0x7f179019c500 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f178400b4b0 tx=0x7f178400b870 comp rx=0 tx=0).stop 2024-09-10T11:28:07.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.949+0000 7f1794e7e700 1 -- 172.21.15.144:0/3154344609 shutdown_connections 2024-09-10T11:28:07.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.949+0000 7f1794e7e700 1 --2- 172.21.15.144:0/3154344609 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f177806c6d0 0x7f177806eb80 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:07.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.949+0000 7f1794e7e700 1 --2- 172.21.15.144:0/3154344609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17900fe410 0x7f179019bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:07.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.949+0000 7f1794e7e700 1 --2- 172.21.15.144:0/3154344609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17900ff180 0x7f179019c500 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:07.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.949+0000 7f1794e7e700 1 -- 172.21.15.144:0/3154344609 >> 172.21.15.144:0/3154344609 conn(0x7f17900f9d80 msgr2=0x7f17901023b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:07.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.949+0000 7f1794e7e700 1 -- 172.21.15.144:0/3154344609 shutdown_connections 2024-09-10T11:28:07.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:07.949+0000 7f1794e7e700 1 -- 172.21.15.144:0/3154344609 wait complete. 2024-09-10T11:28:08.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.118+0000 7f0a8bcd9700 1 -- 172.21.15.144:0/2253001370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a840ff050 msgr2=0x7f0a840ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:08.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.118+0000 7f0a8bcd9700 1 --2- 172.21.15.144:0/2253001370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a840ff050 0x7f0a840ff460 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009910 tx=0x7f0a80009c20 comp rx=0 tx=0).stop 2024-09-10T11:28:08.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.119+0000 7f0a8bcd9700 1 -- 172.21.15.144:0/2253001370 shutdown_connections 2024-09-10T11:28:08.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.119+0000 7f0a8bcd9700 1 --2- 172.21.15.144:0/2253001370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a84100250 0x7f0a841006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:08.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.119+0000 7f0a8bcd9700 1 --2- 172.21.15.144:0/2253001370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a840ff050 0x7f0a840ff460 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:08.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.119+0000 7f0a8bcd9700 1 -- 172.21.15.144:0/2253001370 >> 172.21.15.144:0/2253001370 conn(0x7f0a840fa5e0 msgr2=0x7f0a840fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:08.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.119+0000 7f0a8bcd9700 1 -- 172.21.15.144:0/2253001370 shutdown_connections 2024-09-10T11:28:08.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.119+0000 7f0a8bcd9700 1 -- 172.21.15.144:0/2253001370 wait complete. 2024-09-10T11:28:08.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.120+0000 7f0a8bcd9700 1 Processor -- start 2024-09-10T11:28:08.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.120+0000 7f0a8bcd9700 1 -- start start 2024-09-10T11:28:08.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.120+0000 7f0a8bcd9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a840ff050 0x7f0a84068f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:08.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.120+0000 7f0a8bcd9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a84100250 0x7f0a8419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:08.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.121+0000 7f0a89a75700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a840ff050 0x7f0a84068f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:08.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.121+0000 7f0a89274700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a84100250 0x7f0a8419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:08.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.121+0000 7f0a89274700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a84100250 0x7f0a8419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33136/0 (socket says 172.21.15.144:33136) 2024-09-10T11:28:08.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.121+0000 7f0a89a75700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a840ff050 0x7f0a84068f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.144:47054/0 (socket says 172.21.15.144:47054) 2024-09-10T11:28:08.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.121+0000 7f0a89a75700 1 -- 172.21.15.144:0/3401891307 learned_addr learned my addr 172.21.15.144:0/3401891307 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:28:08.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.121+0000 7f0a8bcd9700 1 -- 172.21.15.144:0/3401891307 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8419aa80 con 0x7f0a84100250 2024-09-10T11:28:08.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.121+0000 7f0a8bcd9700 1 -- 172.21.15.144:0/3401891307 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8419abc0 con 0x7f0a840ff050 2024-09-10T11:28:08.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.121+0000 7f0a89a75700 1 -- 172.21.15.144:0/3401891307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a84100250 msgr2=0x7f0a8419a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:08.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.121+0000 7f0a89a75700 1 --2- 172.21.15.144:0/3401891307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a84100250 0x7f0a8419a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:08.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.121+0000 7f0a89a75700 1 -- 172.21.15.144:0/3401891307 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a800095f0 con 0x7f0a840ff050 2024-09-10T11:28:08.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.121+0000 7f0a89274700 1 --2- 172.21.15.144:0/3401891307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a84100250 0x7f0a8419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:28:08.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.122+0000 7f0a89a75700 1 --2- 172.21.15.144:0/3401891307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a840ff050 0x7f0a84068f90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000fa10 tx=0x7f0a8000fa40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:08.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.122+0000 7f0a7affd700 1 -- 172.21.15.144:0/3401891307 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a8001d070 con 0x7f0a840ff050 2024-09-10T11:28:08.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.122+0000 7f0a7affd700 1 -- 172.21.15.144:0/3401891307 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0a80005640 con 0x7f0a840ff050 2024-09-10T11:28:08.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.122+0000 7f0a7affd700 1 -- 172.21.15.144:0/3401891307 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a80017700 con 0x7f0a840ff050 2024-09-10T11:28:08.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.122+0000 7f0a8bcd9700 1 -- 172.21.15.144:0/3401891307 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8419ae40 con 0x7f0a840ff050 2024-09-10T11:28:08.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.122+0000 7f0a8bcd9700 1 -- 172.21.15.144:0/3401891307 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a8419b250 con 0x7f0a840ff050 2024-09-10T11:28:08.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.124+0000 7f0a78ff9700 1 -- 172.21.15.144:0/3401891307 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a6c0052f0 con 0x7f0a840ff050 2024-09-10T11:28:08.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.124+0000 7f0a7affd700 1 -- 172.21.15.144:0/3401891307 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f0a80003680 con 0x7f0a840ff050 2024-09-10T11:28:08.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.125+0000 7f0a7affd700 1 --2- 172.21.15.144:0/3401891307 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0a7006c680 0x7f0a7006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:08.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.125+0000 7f0a7affd700 1 -- 172.21.15.144:0/3401891307 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f0a8008d090 con 0x7f0a840ff050 2024-09-10T11:28:08.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.126+0000 7f0a89274700 1 --2- 172.21.15.144:0/3401891307 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0a7006c680 0x7f0a7006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:08.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.126+0000 7f0a89274700 1 --2- 172.21.15.144:0/3401891307 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0a7006c680 0x7f0a7006eb30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0a740114e0 tx=0x7f0a74009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:08.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.133+0000 7f0a7affd700 1 -- 172.21.15.144:0/3401891307 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0a8005b230 con 0x7f0a840ff050 2024-09-10T11:28:08.188 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:08 smithi144 ceph-mon[27942]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 on smithi178 2024-09-10T11:28:08.188 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:08 smithi144 ceph-mon[27942]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:08.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:08 smithi178 ceph-mon[33514]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 on smithi178 2024-09-10T11:28:08.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:08 smithi178 ceph-mon[33514]: from='client.14548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:08.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.677+0000 7f0a78ff9700 1 -- 172.21.15.144:0/3401891307 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0a6c0061d0 con 0x7f0a840ff050 2024-09-10T11:28:08.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.678+0000 7f0a7affd700 1 -- 172.21.15.144:0/3401891307 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f0a8005adc0 con 0x7f0a840ff050 2024-09-10T11:28:08.679 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:28:08.679 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:28:08.680 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-09-10T11:28:08.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:28:08.680 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:28:08.680 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-09-10T11:28:08.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:28:08.680 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:28:08.680 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-09-10T11:28:08.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:28:08.680 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:28:08.680 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:28:08.680 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:28:08.680 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:28:08.680 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-09-10T11:28:08.681 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:28:08.681 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:28:08.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.681+0000 7f0a78ff9700 1 -- 172.21.15.144:0/3401891307 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0a7006c680 msgr2=0x7f0a7006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:08.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.681+0000 7f0a78ff9700 1 --2- 172.21.15.144:0/3401891307 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0a7006c680 0x7f0a7006eb30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f0a740114e0 tx=0x7f0a74009260 comp rx=0 tx=0).stop 2024-09-10T11:28:08.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.681+0000 7f0a78ff9700 1 -- 172.21.15.144:0/3401891307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a840ff050 msgr2=0x7f0a84068f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:08.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.681+0000 7f0a78ff9700 1 --2- 172.21.15.144:0/3401891307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a840ff050 0x7f0a84068f90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0a8000fa10 tx=0x7f0a8000fa40 comp rx=0 tx=0).stop 2024-09-10T11:28:08.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.682+0000 7f0a78ff9700 1 -- 172.21.15.144:0/3401891307 shutdown_connections 2024-09-10T11:28:08.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.682+0000 7f0a78ff9700 1 --2- 172.21.15.144:0/3401891307 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f0a7006c680 0x7f0a7006eb30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:08.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.682+0000 7f0a78ff9700 1 --2- 172.21.15.144:0/3401891307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a840ff050 0x7f0a84068f90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:08.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.682+0000 7f0a78ff9700 1 --2- 172.21.15.144:0/3401891307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a84100250 0x7f0a8419a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:08.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.682+0000 7f0a78ff9700 1 -- 172.21.15.144:0/3401891307 >> 172.21.15.144:0/3401891307 conn(0x7f0a840fa5e0 msgr2=0x7f0a840fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:08.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.682+0000 7f0a78ff9700 1 -- 172.21.15.144:0/3401891307 shutdown_connections 2024-09-10T11:28:08.684 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.682+0000 7f0a78ff9700 1 -- 172.21.15.144:0/3401891307 wait complete. 2024-09-10T11:28:08.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.829+0000 7f7f728bb700 1 -- 172.21.15.144:0/3046520074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c1006d0 msgr2=0x7f7f6c100ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:08.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.829+0000 7f7f728bb700 1 --2- 172.21.15.144:0/3046520074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c1006d0 0x7f7f6c100ae0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c009910 tx=0x7f7f5c009c20 comp rx=0 tx=0).stop 2024-09-10T11:28:08.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.829+0000 7f7f728bb700 1 -- 172.21.15.144:0/3046520074 shutdown_connections 2024-09-10T11:28:08.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.829+0000 7f7f728bb700 1 --2- 172.21.15.144:0/3046520074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f6c0ff2a0 0x7f7f6c0ff710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:08.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.829+0000 7f7f728bb700 1 --2- 172.21.15.144:0/3046520074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c1006d0 0x7f7f6c100ae0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:08.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.829+0000 7f7f728bb700 1 -- 172.21.15.144:0/3046520074 >> 172.21.15.144:0/3046520074 conn(0x7f7f6c0f9d80 msgr2=0x7f7f6c0fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:08.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.829+0000 7f7f728bb700 1 -- 172.21.15.144:0/3046520074 shutdown_connections 2024-09-10T11:28:08.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.829+0000 7f7f728bb700 1 -- 172.21.15.144:0/3046520074 wait complete. 2024-09-10T11:28:08.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.830+0000 7f7f728bb700 1 Processor -- start 2024-09-10T11:28:08.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.830+0000 7f7f728bb700 1 -- start start 2024-09-10T11:28:08.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.831+0000 7f7f728bb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c0ff2a0 0x7f7f6c194830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:08.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.831+0000 7f7f728bb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f6c1006d0 0x7f7f6c194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:08.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.831+0000 7f7f6bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c0ff2a0 0x7f7f6c194830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:08.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.831+0000 7f7f6bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c0ff2a0 0x7f7f6c194830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33140/0 (socket says 172.21.15.144:33140) 2024-09-10T11:28:08.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.831+0000 7f7f6bfff700 1 -- 172.21.15.144:0/3387002249 learned_addr learned my addr 172.21.15.144:0/3387002249 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:28:08.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.831+0000 7f7f728bb700 1 -- 172.21.15.144:0/3387002249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c195390 con 0x7f7f6c0ff2a0 2024-09-10T11:28:08.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.831+0000 7f7f728bb700 1 -- 172.21.15.144:0/3387002249 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6c1954d0 con 0x7f7f6c1006d0 2024-09-10T11:28:08.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.831+0000 7f7f6bfff700 1 -- 172.21.15.144:0/3387002249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f6c1006d0 msgr2=0x7f7f6c194d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:08.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.831+0000 7f7f6bfff700 1 --2- 172.21.15.144:0/3387002249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f6c1006d0 0x7f7f6c194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:08.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.831+0000 7f7f6bfff700 1 -- 172.21.15.144:0/3387002249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f5c0095f0 con 0x7f7f6c0ff2a0 2024-09-10T11:28:08.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.832+0000 7f7f6bfff700 1 --2- 172.21.15.144:0/3387002249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c0ff2a0 0x7f7f6c194830 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c000c00 tx=0x7f7f5c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:08.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.832+0000 7f7f69ffb700 1 -- 172.21.15.144:0/3387002249 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f5c01d070 con 0x7f7f6c0ff2a0 2024-09-10T11:28:08.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.832+0000 7f7f69ffb700 1 -- 172.21.15.144:0/3387002249 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7f5c005dc0 con 0x7f7f6c0ff2a0 2024-09-10T11:28:08.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.832+0000 7f7f69ffb700 1 -- 172.21.15.144:0/3387002249 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f5c0226a0 con 0x7f7f6c0ff2a0 2024-09-10T11:28:08.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.832+0000 7f7f728bb700 1 -- 172.21.15.144:0/3387002249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f6c199f20 con 0x7f7f6c0ff2a0 2024-09-10T11:28:08.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.832+0000 7f7f728bb700 1 -- 172.21.15.144:0/3387002249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f6c19a300 con 0x7f7f6c0ff2a0 2024-09-10T11:28:08.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.833+0000 7f7f728bb700 1 -- 172.21.15.144:0/3387002249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f6c107c40 con 0x7f7f6c0ff2a0 2024-09-10T11:28:08.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.834+0000 7f7f69ffb700 1 -- 172.21.15.144:0/3387002249 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f7f5c003680 con 0x7f7f6c0ff2a0 2024-09-10T11:28:08.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.835+0000 7f7f69ffb700 1 --2- 172.21.15.144:0/3387002249 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7f5806c680 0x7f7f5806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:08.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.835+0000 7f7f69ffb700 1 -- 172.21.15.144:0/3387002249 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f7f5c08cd80 con 0x7f7f6c0ff2a0 2024-09-10T11:28:08.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.836+0000 7f7f63fff700 1 --2- 172.21.15.144:0/3387002249 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7f5806c680 0x7f7f5806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:08.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.836+0000 7f7f63fff700 1 --2- 172.21.15.144:0/3387002249 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7f5806c680 0x7f7f5806eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7f54005950 tx=0x7f7f540058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:08.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:08.840+0000 7f7f69ffb700 1 -- 172.21.15.144:0/3387002249 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7f5c090030 con 0x7f7f6c0ff2a0 2024-09-10T11:28:09.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:09.079+0000 7f7f728bb700 1 -- 172.21.15.144:0/3387002249 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f6c104c00 con 0x7f7f5806c680 2024-09-10T11:28:09.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:09.080+0000 7f7f69ffb700 1 -- 172.21.15.144:0/3387002249 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f7f6c104c00 con 0x7f7f5806c680 2024-09-10T11:28:09.081 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:28:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:28:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:28:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T11:28:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [], 2024-09-10T11:28:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "0/2 daemons upgraded", 2024-09-10T11:28:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 image on host smithi178", 2024-09-10T11:28:09.082 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:28:09.082 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:28:09.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:09.083+0000 7f7f728bb700 1 -- 172.21.15.144:0/3387002249 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7f5806c680 msgr2=0x7f7f5806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:09.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:09.083+0000 7f7f728bb700 1 --2- 172.21.15.144:0/3387002249 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7f5806c680 0x7f7f5806eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f7f54005950 tx=0x7f7f540058e0 comp rx=0 tx=0).stop 2024-09-10T11:28:09.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:09.083+0000 7f7f728bb700 1 -- 172.21.15.144:0/3387002249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c0ff2a0 msgr2=0x7f7f6c194830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:09.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:09.083+0000 7f7f728bb700 1 --2- 172.21.15.144:0/3387002249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c0ff2a0 0x7f7f6c194830 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f7f5c000c00 tx=0x7f7f5c00f740 comp rx=0 tx=0).stop 2024-09-10T11:28:09.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:09.083+0000 7f7f728bb700 1 -- 172.21.15.144:0/3387002249 shutdown_connections 2024-09-10T11:28:09.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:09.083+0000 7f7f728bb700 1 --2- 172.21.15.144:0/3387002249 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7f5806c680 0x7f7f5806eb30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:09.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:09.083+0000 7f7f728bb700 1 --2- 172.21.15.144:0/3387002249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f6c0ff2a0 0x7f7f6c194830 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:09.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:09.083+0000 7f7f728bb700 1 --2- 172.21.15.144:0/3387002249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f6c1006d0 0x7f7f6c194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:09.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:09.083+0000 7f7f728bb700 1 -- 172.21.15.144:0/3387002249 >> 172.21.15.144:0/3387002249 conn(0x7f7f6c0f9d80 msgr2=0x7f7f6c1034e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:09.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:09.083+0000 7f7f728bb700 1 -- 172.21.15.144:0/3387002249 shutdown_connections 2024-09-10T11:28:09.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:09.083+0000 7f7f728bb700 1 -- 172.21.15.144:0/3387002249 wait complete. 2024-09-10T11:28:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:09 smithi178 ceph-mon[33514]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:09 smithi178 ceph-mon[33514]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:09 smithi178 ceph-mon[33514]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:28:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:09 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/3401891307' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:28:09.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:09 smithi144 ceph-mon[27942]: from='client.24319 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:09.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:09 smithi144 ceph-mon[27942]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:09.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:09 smithi144 ceph-mon[27942]: pgmap v225: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:28:09.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:09 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/3401891307' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:28:10.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:10 smithi178 ceph-mon[33514]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:10.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:10 smithi144 ceph-mon[27942]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:11 smithi178 ceph-mon[33514]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:28:11.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:11 smithi144 ceph-mon[27942]: pgmap v226: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:28:13.366 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:13 smithi178 ceph-mon[33514]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:28:13.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:13 smithi144 ceph-mon[27942]: pgmap v227: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:28:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:15 smithi178 ceph-mon[33514]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:28:15.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:15 smithi144 ceph-mon[27942]: pgmap v228: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:28:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:17 smithi178 ceph-mon[33514]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:28:17.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:17 smithi144 ceph-mon[27942]: pgmap v229: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:28:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:18 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:28:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:18 smithi144 ceph-mon[27942]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:28:18.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:18 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:28:18.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:18 smithi178 ceph-mon[33514]: pgmap v230: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:28:21.366 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:21 smithi178 ceph-mon[33514]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:28:21.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:21 smithi144 ceph-mon[27942]: pgmap v231: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:28:23.366 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:23 smithi178 ceph-mon[33514]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:28:23.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:23 smithi144 ceph-mon[27942]: pgmap v232: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:28:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:25 smithi178 ceph-mon[33514]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:28:25.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:25 smithi144 ceph-mon[27942]: pgmap v233: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:28:25.658 INFO:tasks.workunit.client.0.smithi144.stderr:Note: switching to '1cf819b7212a6048e92fc78c4eff963344f6d56b'. 2024-09-10T11:28:25.658 INFO:tasks.workunit.client.0.smithi144.stderr: 2024-09-10T11:28:25.659 INFO:tasks.workunit.client.0.smithi144.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T11:28:25.659 INFO:tasks.workunit.client.0.smithi144.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T11:28:25.659 INFO:tasks.workunit.client.0.smithi144.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T11:28:25.659 INFO:tasks.workunit.client.0.smithi144.stderr: 2024-09-10T11:28:25.659 INFO:tasks.workunit.client.0.smithi144.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T11:28:25.659 INFO:tasks.workunit.client.0.smithi144.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T11:28:25.659 INFO:tasks.workunit.client.0.smithi144.stderr: 2024-09-10T11:28:25.659 INFO:tasks.workunit.client.0.smithi144.stderr: git switch -c 2024-09-10T11:28:25.659 INFO:tasks.workunit.client.0.smithi144.stderr: 2024-09-10T11:28:25.659 INFO:tasks.workunit.client.0.smithi144.stderr:Or undo this operation with: 2024-09-10T11:28:25.660 INFO:tasks.workunit.client.0.smithi144.stderr: 2024-09-10T11:28:25.660 INFO:tasks.workunit.client.0.smithi144.stderr: git switch - 2024-09-10T11:28:25.660 INFO:tasks.workunit.client.0.smithi144.stderr: 2024-09-10T11:28:25.660 INFO:tasks.workunit.client.0.smithi144.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T11:28:25.660 INFO:tasks.workunit.client.0.smithi144.stderr: 2024-09-10T11:28:25.660 INFO:tasks.workunit.client.0.smithi144.stderr:HEAD is now at 1cf819b7212 Merge PR #59672 into wip-rishabh-testing-20240909.165402-squid 2024-09-10T11:28:25.665 DEBUG:teuthology.orchestra.run.smithi144:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T11:28:25.850 INFO:tasks.workunit.client.0.smithi144.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T11:28:25.853 INFO:tasks.workunit.client.0.smithi144.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T11:28:25.853 INFO:tasks.workunit.client.0.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T11:28:26.001 INFO:tasks.workunit.client.1.smithi178.stderr:Updating files: 85% (10808/12697) Updating files: 86% (10920/12697) Updating files: 87% (11047/12697) Updating files: 88% (11174/12697) Updating files: 89% (11301/12697) Updating files: 90% (11428/12697) Updating files: 91% (11555/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T11:28:26.479 INFO:tasks.workunit.client.1.smithi178.stderr:Note: switching to '1cf819b7212a6048e92fc78c4eff963344f6d56b'. 2024-09-10T11:28:26.479 INFO:tasks.workunit.client.1.smithi178.stderr: 2024-09-10T11:28:26.479 INFO:tasks.workunit.client.1.smithi178.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T11:28:26.479 INFO:tasks.workunit.client.1.smithi178.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T11:28:26.480 INFO:tasks.workunit.client.1.smithi178.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T11:28:26.480 INFO:tasks.workunit.client.1.smithi178.stderr: 2024-09-10T11:28:26.480 INFO:tasks.workunit.client.1.smithi178.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T11:28:26.480 INFO:tasks.workunit.client.1.smithi178.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T11:28:26.480 INFO:tasks.workunit.client.1.smithi178.stderr: 2024-09-10T11:28:26.480 INFO:tasks.workunit.client.1.smithi178.stderr: git switch -c 2024-09-10T11:28:26.480 INFO:tasks.workunit.client.1.smithi178.stderr: 2024-09-10T11:28:26.480 INFO:tasks.workunit.client.1.smithi178.stderr:Or undo this operation with: 2024-09-10T11:28:26.480 INFO:tasks.workunit.client.1.smithi178.stderr: 2024-09-10T11:28:26.480 INFO:tasks.workunit.client.1.smithi178.stderr: git switch - 2024-09-10T11:28:26.480 INFO:tasks.workunit.client.1.smithi178.stderr: 2024-09-10T11:28:26.480 INFO:tasks.workunit.client.1.smithi178.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T11:28:26.480 INFO:tasks.workunit.client.1.smithi178.stderr: 2024-09-10T11:28:26.481 INFO:tasks.workunit.client.1.smithi178.stderr:HEAD is now at 1cf819b7212 Merge PR #59672 into wip-rishabh-testing-20240909.165402-squid 2024-09-10T11:28:26.485 DEBUG:teuthology.orchestra.run.smithi178:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-09-10T11:28:26.690 INFO:tasks.workunit.client.1.smithi178.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T11:28:26.692 INFO:tasks.workunit.client.1.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-10T11:28:26.692 INFO:tasks.workunit.client.1.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T11:28:27.332 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:27 smithi144 ceph-mon[27942]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:28:27.366 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:27 smithi178 ceph-mon[33514]: pgmap v234: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:28:28.497 INFO:tasks.workunit.client.1.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T11:28:28.598 INFO:tasks.workunit.client.1.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T11:28:28.647 INFO:tasks.workunit.client.1.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-09-10T11:28:28.649 INFO:tasks.workunit.client.1.smithi178.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-10T11:28:28.649 INFO:tasks.workunit.client.1.smithi178.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T11:28:28.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:28 smithi144 ceph-mon[27942]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:28:28.694 INFO:tasks.workunit.client.1.smithi178.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-09-10T11:28:28.698 DEBUG:teuthology.orchestra.run.smithi178:> set -ex 2024-09-10T11:28:28.698 DEBUG:teuthology.orchestra.run.smithi178:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-09-10T11:28:28.716 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-09-10T11:28:28.719 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-09-10T11:28:28.720 DEBUG:teuthology.orchestra.run.smithi178:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1cf819b7212a6048e92fc78c4eff963344f6d56b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-09-10T11:28:28.789 INFO:tasks.workunit.client.1.smithi178.stderr:+ mkdir -p fsstress 2024-09-10T11:28:28.792 INFO:tasks.workunit.client.1.smithi178.stderr:+ pushd fsstress 2024-09-10T11:28:28.793 INFO:tasks.workunit.client.1.smithi178.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-10T11:28:28.793 INFO:tasks.workunit.client.1.smithi178.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-09-10T11:28:28.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:28 smithi178 ceph-mon[33514]: pgmap v235: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:28:29.654 INFO:tasks.workunit.client.1.smithi178.stderr:+ tar xzf ltp-full.tgz 2024-09-10T11:28:31.366 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:31 smithi178 ceph-mon[33514]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:28:31.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:31 smithi144 ceph-mon[27942]: pgmap v236: 65 pgs: 65 active+clean; 590 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:28:31.960 INFO:tasks.workunit.client.0.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T11:28:32.310 INFO:tasks.workunit.client.0.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T11:28:32.360 INFO:tasks.workunit.client.0.smithi144.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T11:28:32.361 INFO:tasks.workunit.client.0.smithi144.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T11:28:32.362 INFO:tasks.workunit.client.0.smithi144.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T11:28:32.442 INFO:tasks.workunit.client.0.smithi144.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T11:28:32.445 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T11:28:32.446 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T11:28:32.470 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-09-10T11:28:32.471 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-09-10T11:28:32.471 DEBUG:teuthology.orchestra.run.smithi144:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1cf819b7212a6048e92fc78c4eff963344f6d56b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-09-10T11:28:32.536 INFO:tasks.workunit.client.0.smithi144.stderr:+ mkdir -p fsstress 2024-09-10T11:28:32.538 INFO:tasks.workunit.client.0.smithi144.stderr:+ pushd fsstress 2024-09-10T11:28:32.539 INFO:tasks.workunit.client.0.smithi144.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-10T11:28:32.540 INFO:tasks.workunit.client.0.smithi144.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-09-10T11:28:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:33 smithi178 ceph-mon[33514]: pgmap v237: 65 pgs: 65 active+clean; 656 KiB data, 167 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 5.6 KiB/s wr, 3 op/s 2024-09-10T11:28:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:33 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:28:33.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:33 smithi144 ceph-mon[27942]: pgmap v237: 65 pgs: 65 active+clean; 656 KiB data, 167 MiB used, 536 GiB / 536 GiB avail; 1.7 KiB/s rd, 5.6 KiB/s wr, 3 op/s 2024-09-10T11:28:33.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:33 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:28:33.754 INFO:tasks.workunit.client.0.smithi144.stderr:+ tar xzf ltp-full.tgz 2024-09-10T11:28:35.366 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:35 smithi178 ceph-mon[33514]: pgmap v238: 65 pgs: 65 active+clean; 14 MiB data, 195 MiB used, 536 GiB / 536 GiB avail; 31 KiB/s rd, 1.2 MiB/s wr, 27 op/s 2024-09-10T11:28:35.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:35 smithi144 ceph-mon[27942]: pgmap v238: 65 pgs: 65 active+clean; 14 MiB data, 195 MiB used, 536 GiB / 536 GiB avail; 31 KiB/s rd, 1.2 MiB/s wr, 27 op/s 2024-09-10T11:28:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:36 smithi178 ceph-mon[33514]: Upgrade: Updating mgr.smithi178.vmvxuf 2024-09-10T11:28:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:36 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:36 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi178.vmvxuf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:28:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:36 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:28:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:36 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:28:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:36 smithi178 ceph-mon[33514]: Deploying daemon mgr.smithi178.vmvxuf on smithi178 2024-09-10T11:28:36.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:36 smithi144 ceph-mon[27942]: Upgrade: Updating mgr.smithi178.vmvxuf 2024-09-10T11:28:36.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:36 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:36.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:36 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi178.vmvxuf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:28:36.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:36 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:28:36.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:36 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:28:36.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:36 smithi144 ceph-mon[27942]: Deploying daemon mgr.smithi178.vmvxuf on smithi178 2024-09-10T11:28:37.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:37 smithi144 ceph-mon[27942]: pgmap v239: 65 pgs: 65 active+clean; 24 MiB data, 216 MiB used, 536 GiB / 536 GiB avail; 31 KiB/s rd, 1.9 MiB/s wr, 48 op/s 2024-09-10T11:28:37.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:37 smithi178 ceph-mon[33514]: pgmap v239: 65 pgs: 65 active+clean; 24 MiB data, 216 MiB used, 536 GiB / 536 GiB avail; 31 KiB/s rd, 1.9 MiB/s wr, 48 op/s 2024-09-10T11:28:38.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:38 smithi144 ceph-mon[27942]: pgmap v240: 65 pgs: 65 active+clean; 34 MiB data, 249 MiB used, 536 GiB / 536 GiB avail; 468 KiB/s rd, 2.8 MiB/s wr, 80 op/s 2024-09-10T11:28:38.711 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:38 smithi178 ceph-mon[33514]: pgmap v240: 65 pgs: 65 active+clean; 34 MiB data, 249 MiB used, 536 GiB / 536 GiB avail; 468 KiB/s rd, 2.8 MiB/s wr, 80 op/s 2024-09-10T11:28:39.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.216+0000 7fc2767fb700 1 -- 172.21.15.144:0/1815337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2700ff050 msgr2=0x7fc2700ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:39.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.216+0000 7fc2767fb700 1 --2- 172.21.15.144:0/1815337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2700ff050 0x7fc2700ff460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fc260009910 tx=0x7fc260009c20 comp rx=0 tx=0).stop 2024-09-10T11:28:39.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.216+0000 7fc2767fb700 1 -- 172.21.15.144:0/1815337 shutdown_connections 2024-09-10T11:28:39.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.216+0000 7fc2767fb700 1 --2- 172.21.15.144:0/1815337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc270100250 0x7fc2701006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:39.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.216+0000 7fc2767fb700 1 --2- 172.21.15.144:0/1815337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2700ff050 0x7fc2700ff460 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:39.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.216+0000 7fc2767fb700 1 -- 172.21.15.144:0/1815337 >> 172.21.15.144:0/1815337 conn(0x7fc2700fa600 msgr2=0x7fc2700fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:39.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.216+0000 7fc2767fb700 1 -- 172.21.15.144:0/1815337 shutdown_connections 2024-09-10T11:28:39.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.216+0000 7fc2767fb700 1 -- 172.21.15.144:0/1815337 wait complete. 2024-09-10T11:28:39.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.217+0000 7fc2767fb700 1 Processor -- start 2024-09-10T11:28:39.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.217+0000 7fc2767fb700 1 -- start start 2024-09-10T11:28:39.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.217+0000 7fc2767fb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2700ff050 0x7fc270194870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:39.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.217+0000 7fc2767fb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc270100250 0x7fc270194db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:39.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.217+0000 7fc2767fb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2701953d0 con 0x7fc270100250 2024-09-10T11:28:39.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.217+0000 7fc2767fb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc270195510 con 0x7fc2700ff050 2024-09-10T11:28:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.217+0000 7fc26f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc270100250 0x7fc270194db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.218+0000 7fc26f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc270100250 0x7fc270194db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38210/0 (socket says 172.21.15.144:38210) 2024-09-10T11:28:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.218+0000 7fc26ffff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2700ff050 0x7fc270194870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.218+0000 7fc26f7fe700 1 -- 172.21.15.144:0/1546839045 learned_addr learned my addr 172.21.15.144:0/1546839045 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:28:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.218+0000 7fc26ffff700 1 -- 172.21.15.144:0/1546839045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc270100250 msgr2=0x7fc270194db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.218+0000 7fc26ffff700 1 --2- 172.21.15.144:0/1546839045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc270100250 0x7fc270194db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.218+0000 7fc26ffff700 1 -- 172.21.15.144:0/1546839045 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2600095f0 con 0x7fc2700ff050 2024-09-10T11:28:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.218+0000 7fc26ffff700 1 --2- 172.21.15.144:0/1546839045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2700ff050 0x7fc270194870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc260009910 tx=0x7fc26000bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.218+0000 7fc26d7fa700 1 -- 172.21.15.144:0/1546839045 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc26001d070 con 0x7fc2700ff050 2024-09-10T11:28:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.218+0000 7fc26d7fa700 1 -- 172.21.15.144:0/1546839045 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc26000f460 con 0x7fc2700ff050 2024-09-10T11:28:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.218+0000 7fc2767fb700 1 -- 172.21.15.144:0/1546839045 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc270068da0 con 0x7fc2700ff050 2024-09-10T11:28:39.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.219+0000 7fc2767fb700 1 -- 172.21.15.144:0/1546839045 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc270069290 con 0x7fc2700ff050 2024-09-10T11:28:39.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.219+0000 7fc26d7fa700 1 -- 172.21.15.144:0/1546839045 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc260021640 con 0x7fc2700ff050 2024-09-10T11:28:39.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.220+0000 7fc2767fb700 1 -- 172.21.15.144:0/1546839045 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc270107ba0 con 0x7fc2700ff050 2024-09-10T11:28:39.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.221+0000 7fc26d7fa700 1 -- 172.21.15.144:0/1546839045 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fc260003a60 con 0x7fc2700ff050 2024-09-10T11:28:39.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.221+0000 7fc26d7fa700 1 --2- 172.21.15.144:0/1546839045 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc25806c680 0x7fc25806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:39.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.221+0000 7fc26d7fa700 1 -- 172.21.15.144:0/1546839045 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fc26008c8c0 con 0x7fc2700ff050 2024-09-10T11:28:39.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.221+0000 7fc26f7fe700 1 --2- 172.21.15.144:0/1546839045 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc25806c680 0x7fc25806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:39.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.222+0000 7fc26f7fe700 1 --2- 172.21.15.144:0/1546839045 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc25806c680 0x7fc25806eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc264007d10 tx=0x7fc264007480 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:39.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.226+0000 7fc26d7fa700 1 -- 172.21.15.144:0/1546839045 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc26005ab00 con 0x7fc2700ff050 2024-09-10T11:28:39.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.510+0000 7fc2767fb700 1 -- 172.21.15.144:0/1546839045 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc270104ba0 con 0x7fc25806c680 2024-09-10T11:28:39.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.512+0000 7fc26d7fa700 1 -- 172.21.15.144:0/1546839045 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fc270104ba0 con 0x7fc25806c680 2024-09-10T11:28:39.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.514+0000 7fc2767fb700 1 -- 172.21.15.144:0/1546839045 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc25806c680 msgr2=0x7fc25806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:39.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.514+0000 7fc2767fb700 1 --2- 172.21.15.144:0/1546839045 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc25806c680 0x7fc25806eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc264007d10 tx=0x7fc264007480 comp rx=0 tx=0).stop 2024-09-10T11:28:39.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.514+0000 7fc2767fb700 1 -- 172.21.15.144:0/1546839045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2700ff050 msgr2=0x7fc270194870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:39.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.514+0000 7fc2767fb700 1 --2- 172.21.15.144:0/1546839045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2700ff050 0x7fc270194870 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc260009910 tx=0x7fc26000bd70 comp rx=0 tx=0).stop 2024-09-10T11:28:39.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.514+0000 7fc2767fb700 1 -- 172.21.15.144:0/1546839045 shutdown_connections 2024-09-10T11:28:39.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.514+0000 7fc2767fb700 1 --2- 172.21.15.144:0/1546839045 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc25806c680 0x7fc25806eb30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:39.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.514+0000 7fc2767fb700 1 --2- 172.21.15.144:0/1546839045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2700ff050 0x7fc270194870 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:39.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.514+0000 7fc2767fb700 1 --2- 172.21.15.144:0/1546839045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc270100250 0x7fc270194db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:39.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.514+0000 7fc2767fb700 1 -- 172.21.15.144:0/1546839045 >> 172.21.15.144:0/1546839045 conn(0x7fc2700fa600 msgr2=0x7fc270103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:39.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.514+0000 7fc2767fb700 1 -- 172.21.15.144:0/1546839045 shutdown_connections 2024-09-10T11:28:39.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.515+0000 7fc2767fb700 1 -- 172.21.15.144:0/1546839045 wait complete. 2024-09-10T11:28:39.527 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:28:39.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.648+0000 7fd552cc6700 1 -- 172.21.15.144:0/2967558498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c0fe410 msgr2=0x7fd54c0fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:39.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.648+0000 7fd552cc6700 1 --2- 172.21.15.144:0/2967558498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c0fe410 0x7fd54c0fe820 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd540009910 tx=0x7fd540009c20 comp rx=0 tx=0).stop 2024-09-10T11:28:39.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.648+0000 7fd552cc6700 1 -- 172.21.15.144:0/2967558498 shutdown_connections 2024-09-10T11:28:39.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.648+0000 7fd552cc6700 1 --2- 172.21.15.144:0/2967558498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd54c1013f0 0x7fd54c101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:39.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.648+0000 7fd552cc6700 1 --2- 172.21.15.144:0/2967558498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c0fe410 0x7fd54c0fe820 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:39.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.648+0000 7fd552cc6700 1 -- 172.21.15.144:0/2967558498 >> 172.21.15.144:0/2967558498 conn(0x7fd54c0f9d80 msgr2=0x7fd54c0fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:39.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.649+0000 7fd552cc6700 1 -- 172.21.15.144:0/2967558498 shutdown_connections 2024-09-10T11:28:39.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.649+0000 7fd552cc6700 1 -- 172.21.15.144:0/2967558498 wait complete. 2024-09-10T11:28:39.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.649+0000 7fd552cc6700 1 Processor -- start 2024-09-10T11:28:39.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.649+0000 7fd552cc6700 1 -- start start 2024-09-10T11:28:39.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.650+0000 7fd552cc6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c0fe410 0x7fd54c195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:39.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.650+0000 7fd552cc6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd54c1013f0 0x7fd54c195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:39.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.650+0000 7fd552cc6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c1964e0 con 0x7fd54c0fe410 2024-09-10T11:28:39.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.650+0000 7fd552cc6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd54c196620 con 0x7fd54c1013f0 2024-09-10T11:28:39.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.650+0000 7fd550a62700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c0fe410 0x7fd54c195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:39.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.650+0000 7fd550a62700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c0fe410 0x7fd54c195980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38240/0 (socket says 172.21.15.144:38240) 2024-09-10T11:28:39.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.650+0000 7fd550a62700 1 -- 172.21.15.144:0/1508049614 learned_addr learned my addr 172.21.15.144:0/1508049614 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:28:39.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.650+0000 7fd54bfff700 1 --2- 172.21.15.144:0/1508049614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd54c1013f0 0x7fd54c195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:39.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.650+0000 7fd550a62700 1 -- 172.21.15.144:0/1508049614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd54c1013f0 msgr2=0x7fd54c195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:39.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.650+0000 7fd550a62700 1 --2- 172.21.15.144:0/1508049614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd54c1013f0 0x7fd54c195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:39.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.650+0000 7fd550a62700 1 -- 172.21.15.144:0/1508049614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5400095f0 con 0x7fd54c0fe410 2024-09-10T11:28:39.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.650+0000 7fd550a62700 1 --2- 172.21.15.144:0/1508049614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c0fe410 0x7fd54c195980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd540000c00 tx=0x7fd54001cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:39.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.651+0000 7fd549ffb700 1 -- 172.21.15.144:0/1508049614 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5400052a0 con 0x7fd54c0fe410 2024-09-10T11:28:39.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.651+0000 7fd549ffb700 1 -- 172.21.15.144:0/1508049614 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd540020070 con 0x7fd54c0fe410 2024-09-10T11:28:39.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.651+0000 7fd552cc6700 1 -- 172.21.15.144:0/1508049614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd54c0ff1e0 con 0x7fd54c0fe410 2024-09-10T11:28:39.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.651+0000 7fd549ffb700 1 -- 172.21.15.144:0/1508049614 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd54000f460 con 0x7fd54c0fe410 2024-09-10T11:28:39.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.651+0000 7fd552cc6700 1 -- 172.21.15.144:0/1508049614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd54c0ff6d0 con 0x7fd54c0fe410 2024-09-10T11:28:39.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.652+0000 7fd552cc6700 1 -- 172.21.15.144:0/1508049614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd54c108e20 con 0x7fd54c0fe410 2024-09-10T11:28:39.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.653+0000 7fd549ffb700 1 -- 172.21.15.144:0/1508049614 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fd540003710 con 0x7fd54c0fe410 2024-09-10T11:28:39.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.654+0000 7fd549ffb700 1 --2- 172.21.15.144:0/1508049614 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd53c06c680 0x7fd53c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:39.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.654+0000 7fd549ffb700 1 -- 172.21.15.144:0/1508049614 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fd54008ce10 con 0x7fd54c0fe410 2024-09-10T11:28:39.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.654+0000 7fd54bfff700 1 --2- 172.21.15.144:0/1508049614 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd53c06c680 0x7fd53c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:39.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.654+0000 7fd54bfff700 1 --2- 172.21.15.144:0/1508049614 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd53c06c680 0x7fd53c06eb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5380096d0 tx=0x7fd538009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:39.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.660+0000 7fd549ffb700 1 -- 172.21.15.144:0/1508049614 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd54005af70 con 0x7fd54c0fe410 2024-09-10T11:28:39.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.948+0000 7fd552cc6700 1 -- 172.21.15.144:0/1508049614 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd54c105d40 con 0x7fd53c06c680 2024-09-10T11:28:39.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.949+0000 7fd549ffb700 1 -- 172.21.15.144:0/1508049614 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fd54c105d40 con 0x7fd53c06c680 2024-09-10T11:28:39.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.951+0000 7fd552cc6700 1 -- 172.21.15.144:0/1508049614 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd53c06c680 msgr2=0x7fd53c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:39.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.951+0000 7fd552cc6700 1 --2- 172.21.15.144:0/1508049614 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd53c06c680 0x7fd53c06eb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd5380096d0 tx=0x7fd538009260 comp rx=0 tx=0).stop 2024-09-10T11:28:39.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.951+0000 7fd552cc6700 1 -- 172.21.15.144:0/1508049614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c0fe410 msgr2=0x7fd54c195980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:39.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.951+0000 7fd552cc6700 1 --2- 172.21.15.144:0/1508049614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c0fe410 0x7fd54c195980 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fd540000c00 tx=0x7fd54001cb00 comp rx=0 tx=0).stop 2024-09-10T11:28:39.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.952+0000 7fd552cc6700 1 -- 172.21.15.144:0/1508049614 shutdown_connections 2024-09-10T11:28:39.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.952+0000 7fd552cc6700 1 --2- 172.21.15.144:0/1508049614 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fd53c06c680 0x7fd53c06eb30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:39.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.952+0000 7fd552cc6700 1 --2- 172.21.15.144:0/1508049614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd54c0fe410 0x7fd54c195980 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:39.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.952+0000 7fd552cc6700 1 --2- 172.21.15.144:0/1508049614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd54c1013f0 0x7fd54c195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:39.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.952+0000 7fd552cc6700 1 -- 172.21.15.144:0/1508049614 >> 172.21.15.144:0/1508049614 conn(0x7fd54c0f9d80 msgr2=0x7fd54c104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:39.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.952+0000 7fd552cc6700 1 -- 172.21.15.144:0/1508049614 shutdown_connections 2024-09-10T11:28:39.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:39.952+0000 7fd552cc6700 1 -- 172.21.15.144:0/1508049614 wait complete. 2024-09-10T11:28:40.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.088+0000 7ff39715b700 1 -- 172.21.15.144:0/185842836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff390100190 msgr2=0x7ff3901005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:40.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.088+0000 7ff39715b700 1 --2- 172.21.15.144:0/185842836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff390100190 0x7ff3901005a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ff384009910 tx=0x7ff384009c20 comp rx=0 tx=0).stop 2024-09-10T11:28:40.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.088+0000 7ff39715b700 1 -- 172.21.15.144:0/185842836 shutdown_connections 2024-09-10T11:28:40.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.088+0000 7ff39715b700 1 --2- 172.21.15.144:0/185842836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390101390 0x7ff3901017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:40.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.088+0000 7ff39715b700 1 --2- 172.21.15.144:0/185842836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff390100190 0x7ff3901005a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:40.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.088+0000 7ff39715b700 1 -- 172.21.15.144:0/185842836 >> 172.21.15.144:0/185842836 conn(0x7ff3900fb740 msgr2=0x7ff3900fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:40.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.088+0000 7ff39715b700 1 -- 172.21.15.144:0/185842836 shutdown_connections 2024-09-10T11:28:40.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.088+0000 7ff39715b700 1 -- 172.21.15.144:0/185842836 wait complete. 2024-09-10T11:28:40.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.089+0000 7ff39715b700 1 Processor -- start 2024-09-10T11:28:40.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.089+0000 7ff39715b700 1 -- start start 2024-09-10T11:28:40.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.089+0000 7ff39715b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390100190 0x7ff390074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:40.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.089+0000 7ff39715b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff390101390 0x7ff390073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:40.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.089+0000 7ff39715b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff390073680 con 0x7ff390101390 2024-09-10T11:28:40.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.089+0000 7ff38ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff390101390 0x7ff390073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:40.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.089+0000 7ff38ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff390101390 0x7ff390073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38258/0 (socket says 172.21.15.144:38258) 2024-09-10T11:28:40.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.089+0000 7ff38ffff700 1 -- 172.21.15.144:0/2518935658 learned_addr learned my addr 172.21.15.144:0/2518935658 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:28:40.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.089+0000 7ff394ef7700 1 --2- 172.21.15.144:0/2518935658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390100190 0x7ff390074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:40.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.089+0000 7ff39715b700 1 -- 172.21.15.144:0/2518935658 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3900737c0 con 0x7ff390100190 2024-09-10T11:28:40.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.090+0000 7ff38ffff700 1 -- 172.21.15.144:0/2518935658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390100190 msgr2=0x7ff390074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:40.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.090+0000 7ff38ffff700 1 --2- 172.21.15.144:0/2518935658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390100190 0x7ff390074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:40.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.090+0000 7ff38ffff700 1 -- 172.21.15.144:0/2518935658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3840095f0 con 0x7ff390101390 2024-09-10T11:28:40.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.090+0000 7ff38ffff700 1 --2- 172.21.15.144:0/2518935658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff390101390 0x7ff390073140 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00e920 tx=0x7ff37c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:40.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.090+0000 7ff38dffb700 1 -- 172.21.15.144:0/2518935658 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff37c00c9d0 con 0x7ff390101390 2024-09-10T11:28:40.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.090+0000 7ff38dffb700 1 -- 172.21.15.144:0/2518935658 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff37c010430 con 0x7ff390101390 2024-09-10T11:28:40.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.090+0000 7ff39715b700 1 -- 172.21.15.144:0/2518935658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff390073aa0 con 0x7ff390101390 2024-09-10T11:28:40.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.090+0000 7ff38dffb700 1 -- 172.21.15.144:0/2518935658 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff37c018650 con 0x7ff390101390 2024-09-10T11:28:40.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.091+0000 7ff39715b700 1 -- 172.21.15.144:0/2518935658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39019d9f0 con 0x7ff390101390 2024-09-10T11:28:40.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.092+0000 7ff39715b700 1 -- 172.21.15.144:0/2518935658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff390108ce0 con 0x7ff390101390 2024-09-10T11:28:40.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.093+0000 7ff38dffb700 1 -- 172.21.15.144:0/2518935658 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7ff37c0105a0 con 0x7ff390101390 2024-09-10T11:28:40.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.094+0000 7ff38dffb700 1 --2- 172.21.15.144:0/2518935658 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff38006c680 0x7ff38006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:40.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.094+0000 7ff38dffb700 1 -- 172.21.15.144:0/2518935658 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7ff37c014070 con 0x7ff390101390 2024-09-10T11:28:40.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.094+0000 7ff394ef7700 1 --2- 172.21.15.144:0/2518935658 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff38006c680 0x7ff38006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:40.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.094+0000 7ff394ef7700 1 --2- 172.21.15.144:0/2518935658 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff38006c680 0x7ff38006eb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff384009fd0 tx=0x7ff384005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:40.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.099+0000 7ff38dffb700 1 -- 172.21.15.144:0/2518935658 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff37c059d80 con 0x7ff390101390 2024-09-10T11:28:40.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:39 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:40.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:39 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:40.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:39 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:28:40.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.405+0000 7ff39715b700 1 -- 172.21.15.144:0/2518935658 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff390105ce0 con 0x7ff38006c680 2024-09-10T11:28:40.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:39 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:40.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:39 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:40.407 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:39 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:28:40.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.418+0000 7ff38dffb700 1 -- 172.21.15.144:0/2518935658 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3576 (secure 0 0 0) 0x7ff390105ce0 con 0x7ff38006c680 2024-09-10T11:28:40.420 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:28:40.420 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (6m) 93s ago 7m 25.0M - 0.25.0 c8568f914cd2 b5c0bfae59de 2024-09-10T11:28:40.420 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (7m) 93s ago 7m 7465k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:28:40.420 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (6m) 101s ago 6m 7411k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:28:40.420 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (7m) 93s ago 7m 6665k - 18.2.0 dc2bc1663786 3311fb04aa0c 2024-09-10T11:28:40.420 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (6m) 101s ago 6m 6651k - 18.2.0 dc2bc1663786 6e408e8476d4 2024-09-10T11:28:40.420 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (5m) 93s ago 7m 84.7M - 9.4.7 954c08fa6188 e9a1aec955f6 2024-09-10T11:28:40.420 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (2m) 93s ago 2m 12.6M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:28:40.420 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (115s) 93s ago 115s 16.2M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:28:40.420 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (113s) 101s ago 112s 16.1M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:28:40.421 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (119s) 101s ago 119s 18.4M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:28:40.421 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:9283,8765,8443 running (8m) 93s ago 8m 487M - 18.2.0 dc2bc1663786 71a7e1c8a428 2024-09-10T11:28:40.421 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 starting - - - - 2024-09-10T11:28:40.421 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (9m) 93s ago 9m 44.4M 2048M 18.2.0 dc2bc1663786 42addb654e4f 2024-09-10T11:28:40.421 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (6m) 101s ago 6m 35.3M 2048M 18.2.0 dc2bc1663786 3c1ba210ed3f 2024-09-10T11:28:40.421 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (7m) 93s ago 7m 16.0M - 1.5.0 0da6a335fe13 dfbf0c488d17 2024-09-10T11:28:40.421 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (6m) 101s ago 6m 15.2M - 1.5.0 0da6a335fe13 ff2cac118c8d 2024-09-10T11:28:40.421 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (5m) 93s ago 5m 47.9M 3946M 18.2.0 dc2bc1663786 2435febebdb5 2024-09-10T11:28:40.421 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (5m) 93s ago 5m 48.0M 3946M 18.2.0 dc2bc1663786 b55fe920ff9d 2024-09-10T11:28:40.421 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (4m) 93s ago 4m 46.3M 3946M 18.2.0 dc2bc1663786 3ed5618eac33 2024-09-10T11:28:40.421 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (4m) 101s ago 4m 46.6M 4970M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:28:40.421 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (3m) 101s ago 3m 44.0M 4970M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:28:40.421 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (3m) 101s ago 3m 44.5M 4970M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:28:40.422 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (5m) 93s ago 7m 60.8M - 2.43.0 a07b618ecd1d a463a804d232 2024-09-10T11:28:40.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.421+0000 7ff39715b700 1 -- 172.21.15.144:0/2518935658 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff38006c680 msgr2=0x7ff38006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:40.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.421+0000 7ff39715b700 1 --2- 172.21.15.144:0/2518935658 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff38006c680 0x7ff38006eb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff384009fd0 tx=0x7ff384005a90 comp rx=0 tx=0).stop 2024-09-10T11:28:40.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.421+0000 7ff39715b700 1 -- 172.21.15.144:0/2518935658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff390101390 msgr2=0x7ff390073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:40.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.421+0000 7ff39715b700 1 --2- 172.21.15.144:0/2518935658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff390101390 0x7ff390073140 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff37c00e920 tx=0x7ff37c00ec30 comp rx=0 tx=0).stop 2024-09-10T11:28:40.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.421+0000 7ff39715b700 1 -- 172.21.15.144:0/2518935658 shutdown_connections 2024-09-10T11:28:40.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.421+0000 7ff39715b700 1 --2- 172.21.15.144:0/2518935658 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7ff38006c680 0x7ff38006eb30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:40.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.421+0000 7ff39715b700 1 --2- 172.21.15.144:0/2518935658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff390100190 0x7ff390074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:40.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.421+0000 7ff39715b700 1 --2- 172.21.15.144:0/2518935658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff390101390 0x7ff390073140 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:40.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.421+0000 7ff39715b700 1 -- 172.21.15.144:0/2518935658 >> 172.21.15.144:0/2518935658 conn(0x7ff3900fb740 msgr2=0x7ff3901045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:40.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.421+0000 7ff39715b700 1 -- 172.21.15.144:0/2518935658 shutdown_connections 2024-09-10T11:28:40.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.422+0000 7ff39715b700 1 -- 172.21.15.144:0/2518935658 wait complete. 2024-09-10T11:28:40.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.533+0000 7f7634a7a700 1 -- 172.21.15.144:0/4206154937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7630103500 msgr2=0x7f76301058e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:40.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.533+0000 7f7634a7a700 1 --2- 172.21.15.144:0/4206154937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7630103500 0x7f76301058e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7624009910 tx=0x7f7624009c20 comp rx=0 tx=0).stop 2024-09-10T11:28:40.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.534+0000 7f7634a7a700 1 -- 172.21.15.144:0/4206154937 shutdown_connections 2024-09-10T11:28:40.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.534+0000 7f7634a7a700 1 --2- 172.21.15.144:0/4206154937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7630103500 0x7f76301058e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:40.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.534+0000 7f7634a7a700 1 --2- 172.21.15.144:0/4206154937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7630100be0 0x7f7630102fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:40.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.534+0000 7f7634a7a700 1 -- 172.21.15.144:0/4206154937 >> 172.21.15.144:0/4206154937 conn(0x7f76300fa5e0 msgr2=0x7f76300fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:40.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.534+0000 7f7634a7a700 1 -- 172.21.15.144:0/4206154937 shutdown_connections 2024-09-10T11:28:40.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.534+0000 7f7634a7a700 1 -- 172.21.15.144:0/4206154937 wait complete. 2024-09-10T11:28:40.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.535+0000 7f7634a7a700 1 Processor -- start 2024-09-10T11:28:40.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.535+0000 7f7634a7a700 1 -- start start 2024-09-10T11:28:40.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.535+0000 7f7634a7a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7630100be0 0x7f7630195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:40.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.535+0000 7f7634a7a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7630103500 0x7f7630195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:40.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.535+0000 7f7634a7a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76301964b0 con 0x7f7630103500 2024-09-10T11:28:40.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.535+0000 7f7634a7a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76301965f0 con 0x7f7630100be0 2024-09-10T11:28:40.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.535+0000 7f762dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7630103500 0x7f7630195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:40.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.535+0000 7f762dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7630103500 0x7f7630195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38268/0 (socket says 172.21.15.144:38268) 2024-09-10T11:28:40.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.535+0000 7f762dd9b700 1 -- 172.21.15.144:0/2902739468 learned_addr learned my addr 172.21.15.144:0/2902739468 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:28:40.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.535+0000 7f762e59c700 1 --2- 172.21.15.144:0/2902739468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7630100be0 0x7f7630195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:40.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.536+0000 7f762dd9b700 1 -- 172.21.15.144:0/2902739468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7630100be0 msgr2=0x7f7630195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:40.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.536+0000 7f762dd9b700 1 --2- 172.21.15.144:0/2902739468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7630100be0 0x7f7630195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:40.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.536+0000 7f762dd9b700 1 -- 172.21.15.144:0/2902739468 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76240095f0 con 0x7f7630103500 2024-09-10T11:28:40.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.536+0000 7f762dd9b700 1 --2- 172.21.15.144:0/2902739468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7630103500 0x7f7630195e90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7624000c00 tx=0x7f762400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:40.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.536+0000 7f761f7fe700 1 -- 172.21.15.144:0/2902739468 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f762401d070 con 0x7f7630103500 2024-09-10T11:28:40.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.536+0000 7f7634a7a700 1 -- 172.21.15.144:0/2902739468 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763019b040 con 0x7f7630103500 2024-09-10T11:28:40.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.536+0000 7f7634a7a700 1 -- 172.21.15.144:0/2902739468 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763019b530 con 0x7f7630103500 2024-09-10T11:28:40.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.536+0000 7f761f7fe700 1 -- 172.21.15.144:0/2902739468 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7624005dc0 con 0x7f7630103500 2024-09-10T11:28:40.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.536+0000 7f761f7fe700 1 -- 172.21.15.144:0/2902739468 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76240226a0 con 0x7f7630103500 2024-09-10T11:28:40.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.537+0000 7f7634a7a700 1 -- 172.21.15.144:0/2902739468 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7610005320 con 0x7f7630103500 2024-09-10T11:28:40.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.538+0000 7f761f7fe700 1 -- 172.21.15.144:0/2902739468 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7f762400faf0 con 0x7f7630103500 2024-09-10T11:28:40.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.538+0000 7f761f7fe700 1 --2- 172.21.15.144:0/2902739468 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7618070a90 0x7f7618072f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:40.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.538+0000 7f761f7fe700 1 -- 172.21.15.144:0/2902739468 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7f762408cbe0 con 0x7f7630103500 2024-09-10T11:28:40.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.539+0000 7f762e59c700 1 --2- 172.21.15.144:0/2902739468 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7618070a90 0x7f7618072f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:40.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.539+0000 7f762e59c700 1 --2- 172.21.15.144:0/2902739468 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7618070a90 0x7f7618072f40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7620009c60 tx=0x7f7620009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:40.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.543+0000 7f761f7fe700 1 -- 172.21.15.144:0/2902739468 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f762405ae20 con 0x7f7630103500 2024-09-10T11:28:40.932 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.931+0000 7f7634a7a700 1 -- 172.21.15.144:0/2902739468 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7610006200 con 0x7f7630103500 2024-09-10T11:28:40.934 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:28:40.934 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:28:40.934 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-09-10T11:28:40.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:28:40.934 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:28:40.934 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-09-10T11:28:40.934 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:28:40.934 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:28:40.935 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-09-10T11:28:40.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:28:40.935 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:28:40.935 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:28:40.935 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:28:40.935 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:28:40.935 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-09-10T11:28:40.935 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:28:40.935 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:28:40.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.932+0000 7f761f7fe700 1 -- 172.21.15.144:0/2902739468 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f76240213d0 con 0x7f7630103500 2024-09-10T11:28:40.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.935+0000 7f7634a7a700 1 -- 172.21.15.144:0/2902739468 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7618070a90 msgr2=0x7f7618072f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:40.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.935+0000 7f7634a7a700 1 --2- 172.21.15.144:0/2902739468 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7618070a90 0x7f7618072f40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7620009c60 tx=0x7f7620009260 comp rx=0 tx=0).stop 2024-09-10T11:28:40.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.935+0000 7f7634a7a700 1 -- 172.21.15.144:0/2902739468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7630103500 msgr2=0x7f7630195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:40.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.935+0000 7f7634a7a700 1 --2- 172.21.15.144:0/2902739468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7630103500 0x7f7630195e90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f7624000c00 tx=0x7f762400f690 comp rx=0 tx=0).stop 2024-09-10T11:28:40.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.935+0000 7f7634a7a700 1 -- 172.21.15.144:0/2902739468 shutdown_connections 2024-09-10T11:28:40.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.935+0000 7f7634a7a700 1 --2- 172.21.15.144:0/2902739468 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7f7618070a90 0x7f7618072f40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:40.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.935+0000 7f7634a7a700 1 --2- 172.21.15.144:0/2902739468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7630100be0 0x7f7630195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:40.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.935+0000 7f7634a7a700 1 --2- 172.21.15.144:0/2902739468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7630103500 0x7f7630195e90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:40.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.935+0000 7f7634a7a700 1 -- 172.21.15.144:0/2902739468 >> 172.21.15.144:0/2902739468 conn(0x7f76300fa5e0 msgr2=0x7f76300fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:40.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.936+0000 7f7634a7a700 1 -- 172.21.15.144:0/2902739468 shutdown_connections 2024-09-10T11:28:40.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:40.936+0000 7f7634a7a700 1 -- 172.21.15.144:0/2902739468 wait complete. 2024-09-10T11:28:41.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.078+0000 7fc56e514700 1 -- 172.21.15.144:0/1748122112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc568100c30 msgr2=0x7fc568103010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:41.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.078+0000 7fc56e514700 1 --2- 172.21.15.144:0/1748122112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc568100c30 0x7fc568103010 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc558009910 tx=0x7fc558009c20 comp rx=0 tx=0).stop 2024-09-10T11:28:41.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.078+0000 7fc56e514700 1 -- 172.21.15.144:0/1748122112 shutdown_connections 2024-09-10T11:28:41.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.078+0000 7fc56e514700 1 --2- 172.21.15.144:0/1748122112 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc568103550 0x7fc568105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:41.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.078+0000 7fc56e514700 1 --2- 172.21.15.144:0/1748122112 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc568100c30 0x7fc568103010 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:41.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.078+0000 7fc56e514700 1 -- 172.21.15.144:0/1748122112 >> 172.21.15.144:0/1748122112 conn(0x7fc5680fa630 msgr2=0x7fc5680fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:41.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.078+0000 7fc56e514700 1 -- 172.21.15.144:0/1748122112 shutdown_connections 2024-09-10T11:28:41.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.078+0000 7fc56e514700 1 -- 172.21.15.144:0/1748122112 wait complete. 2024-09-10T11:28:41.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.079+0000 7fc56e514700 1 Processor -- start 2024-09-10T11:28:41.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.079+0000 7fc56e514700 1 -- start start 2024-09-10T11:28:41.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.079+0000 7fc56e514700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc568100c30 0x7fc568195900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:41.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.079+0000 7fc56e514700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc568103550 0x7fc568195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:41.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.079+0000 7fc56e514700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc568196460 con 0x7fc568100c30 2024-09-10T11:28:41.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.079+0000 7fc56e514700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5681965a0 con 0x7fc568103550 2024-09-10T11:28:41.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.079+0000 7fc567fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc568100c30 0x7fc568195900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.079+0000 7fc567fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc568100c30 0x7fc568195900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38294/0 (socket says 172.21.15.144:38294) 2024-09-10T11:28:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.079+0000 7fc567fff700 1 -- 172.21.15.144:0/596695765 learned_addr learned my addr 172.21.15.144:0/596695765 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:28:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.080+0000 7fc5677fe700 1 --2- 172.21.15.144:0/596695765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc568103550 0x7fc568195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.080+0000 7fc5677fe700 1 -- 172.21.15.144:0/596695765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc568100c30 msgr2=0x7fc568195900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.080+0000 7fc5677fe700 1 --2- 172.21.15.144:0/596695765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc568100c30 0x7fc568195900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.080+0000 7fc5677fe700 1 -- 172.21.15.144:0/596695765 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5580095f0 con 0x7fc568103550 2024-09-10T11:28:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.080+0000 7fc567fff700 1 --2- 172.21.15.144:0/596695765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc568100c30 0x7fc568195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:28:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.080+0000 7fc5677fe700 1 --2- 172.21.15.144:0/596695765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc568103550 0x7fc568195e40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00e970 tx=0x7fc55c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.080+0000 7fc5657fa700 1 -- 172.21.15.144:0/596695765 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc55c00c9d0 con 0x7fc568103550 2024-09-10T11:28:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.080+0000 7fc56e514700 1 -- 172.21.15.144:0/596695765 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc56819b050 con 0x7fc568103550 2024-09-10T11:28:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.080+0000 7fc5657fa700 1 -- 172.21.15.144:0/596695765 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc55c010430 con 0x7fc568103550 2024-09-10T11:28:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.080+0000 7fc5657fa700 1 -- 172.21.15.144:0/596695765 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc55c018650 con 0x7fc568103550 2024-09-10T11:28:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.080+0000 7fc56e514700 1 -- 172.21.15.144:0/596695765 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc56819b570 con 0x7fc568103550 2024-09-10T11:28:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.082+0000 7fc56e514700 1 -- 172.21.15.144:0/596695765 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc568108ce0 con 0x7fc568103550 2024-09-10T11:28:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.082+0000 7fc5657fa700 1 -- 172.21.15.144:0/596695765 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 19) v1 ==== 89916+0+0 (secure 0 0 0) 0x7fc55c00cb30 con 0x7fc568103550 2024-09-10T11:28:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.082+0000 7fc5657fa700 1 --2- 172.21.15.144:0/596695765 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc55006c680 0x7fc55006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:28:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.082+0000 7fc5657fa700 1 -- 172.21.15.144:0/596695765 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5382+0+0 (secure 0 0 0) 0x7fc55c014070 con 0x7fc568103550 2024-09-10T11:28:41.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.083+0000 7fc567fff700 1 --2- 172.21.15.144:0/596695765 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc55006c680 0x7fc55006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:28:41.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.083+0000 7fc567fff700 1 --2- 172.21.15.144:0/596695765 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc55006c680 0x7fc55006eb30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc558009fd0 tx=0x7fc558005940 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:28:41.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.091+0000 7fc5657fa700 1 -- 172.21.15.144:0/596695765 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc55c051720 con 0x7fc568103550 2024-09-10T11:28:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:41 smithi178 ceph-mon[33514]: from='client.24333 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:41 smithi178 ceph-mon[33514]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:41 smithi178 ceph-mon[33514]: pgmap v241: 65 pgs: 65 active+clean; 37 MiB data, 291 MiB used, 536 GiB / 536 GiB avail; 468 KiB/s rd, 3.1 MiB/s wr, 122 op/s 2024-09-10T11:28:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:41 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/2902739468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:28:41.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:41 smithi144 ceph-mon[27942]: from='client.24333 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:41 smithi144 ceph-mon[27942]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:41 smithi144 ceph-mon[27942]: pgmap v241: 65 pgs: 65 active+clean; 37 MiB data, 291 MiB used, 536 GiB / 536 GiB avail; 468 KiB/s rd, 3.1 MiB/s wr, 122 op/s 2024-09-10T11:28:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:41 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2902739468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:28:41.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.340+0000 7fc56e514700 1 -- 172.21.15.144:0/596695765 --> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5681041b0 con 0x7fc55006c680 2024-09-10T11:28:41.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.342+0000 7fc5657fa700 1 -- 172.21.15.144:0/596695765 <== mgr.14213 v2:172.21.15.144:6800/2 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fc5681041b0 con 0x7fc55006c680 2024-09-10T11:28:41.343 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:28:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:28:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:28:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T11:28:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [], 2024-09-10T11:28:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "", 2024-09-10T11:28:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mgr daemons", 2024-09-10T11:28:41.343 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:28:41.344 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:28:41.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.344+0000 7fc56e514700 1 -- 172.21.15.144:0/596695765 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc55006c680 msgr2=0x7fc55006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:41.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.344+0000 7fc56e514700 1 --2- 172.21.15.144:0/596695765 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc55006c680 0x7fc55006eb30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc558009fd0 tx=0x7fc558005940 comp rx=0 tx=0).stop 2024-09-10T11:28:41.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.344+0000 7fc56e514700 1 -- 172.21.15.144:0/596695765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc568103550 msgr2=0x7fc568195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:28:41.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.344+0000 7fc56e514700 1 --2- 172.21.15.144:0/596695765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc568103550 0x7fc568195e40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc55c00e970 tx=0x7fc55c00ec80 comp rx=0 tx=0).stop 2024-09-10T11:28:41.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.344+0000 7fc56e514700 1 -- 172.21.15.144:0/596695765 shutdown_connections 2024-09-10T11:28:41.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.344+0000 7fc56e514700 1 --2- 172.21.15.144:0/596695765 >> [v2:172.21.15.144:6800/2,v1:172.21.15.144:6801/2] conn(0x7fc55006c680 0x7fc55006eb30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:41.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.344+0000 7fc56e514700 1 --2- 172.21.15.144:0/596695765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc568100c30 0x7fc568195900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:41.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.344+0000 7fc56e514700 1 --2- 172.21.15.144:0/596695765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc568103550 0x7fc568195e40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:28:41.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.344+0000 7fc56e514700 1 -- 172.21.15.144:0/596695765 >> 172.21.15.144:0/596695765 conn(0x7fc5680fa630 msgr2=0x7fc5680fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:28:41.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.345+0000 7fc56e514700 1 -- 172.21.15.144:0/596695765 shutdown_connections 2024-09-10T11:28:41.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:28:41.345+0000 7fc56e514700 1 -- 172.21.15.144:0/596695765 wait complete. 2024-09-10T11:28:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:42 smithi178 ceph-mon[33514]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:42 smithi178 ceph-mon[33514]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:28:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:42 smithi178 ceph-mon[33514]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:42.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:42 smithi144 ceph-mon[27942]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:42.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:42 smithi144 ceph-mon[27942]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:28:42.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:42 smithi144 ceph-mon[27942]: from='client.24349 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:28:43.155 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:43 smithi178 ceph-mon[33514]: pgmap v242: 65 pgs: 65 active+clean; 38 MiB data, 297 MiB used, 536 GiB / 536 GiB avail; 467 KiB/s rd, 3.2 MiB/s wr, 132 op/s 2024-09-10T11:28:43.156 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:43 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:43.156 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:43 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:43.156 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:43 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:43.156 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:43 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:43 smithi144 ceph-mon[27942]: pgmap v242: 65 pgs: 65 active+clean; 38 MiB data, 297 MiB used, 536 GiB / 536 GiB avail; 467 KiB/s rd, 3.2 MiB/s wr, 132 op/s 2024-09-10T11:28:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:43 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:43 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:43 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:43 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:45 smithi178 ceph-mon[33514]: pgmap v243: 65 pgs: 65 active+clean; 54 MiB data, 343 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 171 op/s 2024-09-10T11:28:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:45 smithi178 ceph-mon[33514]: Standby manager daemon smithi178.vmvxuf restarted 2024-09-10T11:28:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:45 smithi178 ceph-mon[33514]: Standby manager daemon smithi178.vmvxuf started 2024-09-10T11:28:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:45 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/crt"}]: dispatch 2024-09-10T11:28:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:45 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T11:28:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:45 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/key"}]: dispatch 2024-09-10T11:28:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:45 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T11:28:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:45 smithi144 ceph-mon[27942]: pgmap v243: 65 pgs: 65 active+clean; 54 MiB data, 343 MiB used, 536 GiB / 536 GiB avail; 1.7 MiB/s rd, 4.6 MiB/s wr, 171 op/s 2024-09-10T11:28:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:45 smithi144 ceph-mon[27942]: Standby manager daemon smithi178.vmvxuf restarted 2024-09-10T11:28:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:45 smithi144 ceph-mon[27942]: Standby manager daemon smithi178.vmvxuf started 2024-09-10T11:28:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:45 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/crt"}]: dispatch 2024-09-10T11:28:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:45 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T11:28:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:45 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/key"}]: dispatch 2024-09-10T11:28:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:45 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T11:28:46.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:46 smithi178 ceph-mon[33514]: mgrmap e20: smithi144.bkogfs(active, since 6m), standbys: smithi178.vmvxuf 2024-09-10T11:28:46.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:46 smithi144 ceph-mon[27942]: mgrmap e20: smithi144.bkogfs(active, since 6m), standbys: smithi178.vmvxuf 2024-09-10T11:28:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:47 smithi178 ceph-mon[33514]: pgmap v244: 65 pgs: 65 active+clean; 63 MiB data, 366 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 175 op/s 2024-09-10T11:28:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:47 smithi144 ceph-mon[27942]: pgmap v244: 65 pgs: 65 active+clean; 63 MiB data, 366 MiB used, 536 GiB / 536 GiB avail; 2.0 MiB/s rd, 4.1 MiB/s wr, 175 op/s 2024-09-10T11:28:48.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:48 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:28:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:48 smithi144 ceph-mon[27942]: pgmap v245: 65 pgs: 65 active+clean; 77 MiB data, 387 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 182 op/s 2024-09-10T11:28:48.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:48 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:28:48.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:48 smithi178 ceph-mon[33514]: pgmap v245: 65 pgs: 65 active+clean; 77 MiB data, 387 MiB used, 536 GiB / 536 GiB avail; 2.8 MiB/s rd, 4.5 MiB/s wr, 182 op/s 2024-09-10T11:28:51.361 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:51 smithi178 ceph-mon[33514]: pgmap v246: 65 pgs: 65 active+clean; 79 MiB data, 426 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 171 op/s 2024-09-10T11:28:51.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:51 smithi144 ceph-mon[27942]: pgmap v246: 65 pgs: 65 active+clean; 79 MiB data, 426 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 3.9 MiB/s wr, 171 op/s 2024-09-10T11:28:52.478 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:52.479 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:52.479 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:28:52.479 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:28:52.479 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:52.479 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:28:52.479 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr fail", "who": "smithi144.bkogfs"}]: dispatch 2024-09-10T11:28:52.479 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: osdmap e48: 6 total, 6 up, 6 in 2024-09-10T11:28:52.479 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "mgr fail", "who": "smithi144.bkogfs"}]': finished 2024-09-10T11:28:52.479 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: mgrmap e21: smithi178.vmvxuf(active, starting, since 0.00750642s) 2024-09-10T11:28:52.479 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:28:52.480 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:28:52.480 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jaoyvs"}]: dispatch 2024-09-10T11:28:52.480 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:28:52.480 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.nmhrmh"}]: dispatch 2024-09-10T11:28:52.480 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.czjrsa"}]: dispatch 2024-09-10T11:28:52.480 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mgr metadata", "who": "smithi178.vmvxuf", "id": "smithi178.vmvxuf"}]: dispatch 2024-09-10T11:28:52.480 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:28:52.480 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:28:52.480 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:28:52.480 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:28:52.480 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:28:52.480 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:28:52.481 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T11:28:52.481 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T11:28:52.481 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T11:28:52.481 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: Manager daemon smithi178.vmvxuf is now available 2024-09-10T11:28:52.481 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: Migrating agent root cert to cert store 2024-09-10T11:28:52.481 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:28:52.481 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: Migrating agent root key to cert store 2024-09-10T11:28:52.481 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:28:52.481 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: Checking for cert/key for grafana.smithi144 2024-09-10T11:28:52.481 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: Migrating grafana.smithi144 cert to cert store 2024-09-10T11:28:52.481 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:28:52.481 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: Migrating grafana.smithi144 key to cert store 2024-09-10T11:28:52.481 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:28:52.481 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:28:52.482 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:28:52.482 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:28:52.482 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi178.vmvxuf/mirror_snapshot_schedule"}]: dispatch 2024-09-10T11:28:52.482 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi178.vmvxuf/mirror_snapshot_schedule"}]: dispatch 2024-09-10T11:28:52.482 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi178.vmvxuf/trash_purge_schedule"}]: dispatch 2024-09-10T11:28:52.482 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:52 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi178.vmvxuf/trash_purge_schedule"}]: dispatch 2024-09-10T11:28:52.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:52.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:52.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:28:52.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:28:52.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' 2024-09-10T11:28:52.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:28:52.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr fail", "who": "smithi144.bkogfs"}]: dispatch 2024-09-10T11:28:52.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: osdmap e48: 6 total, 6 up, 6 in 2024-09-10T11:28:52.619 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.14213 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "mgr fail", "who": "smithi144.bkogfs"}]': finished 2024-09-10T11:28:52.619 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: mgrmap e21: smithi178.vmvxuf(active, starting, since 0.00750642s) 2024-09-10T11:28:52.619 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:28:52.619 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:28:52.619 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jaoyvs"}]: dispatch 2024-09-10T11:28:52.619 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:28:52.619 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.nmhrmh"}]: dispatch 2024-09-10T11:28:52.619 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.czjrsa"}]: dispatch 2024-09-10T11:28:52.619 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mgr metadata", "who": "smithi178.vmvxuf", "id": "smithi178.vmvxuf"}]: dispatch 2024-09-10T11:28:52.619 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:28:52.619 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:28:52.620 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:28:52.620 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:28:52.620 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:28:52.620 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:28:52.620 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T11:28:52.620 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T11:28:52.620 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T11:28:52.620 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: Manager daemon smithi178.vmvxuf is now available 2024-09-10T11:28:52.620 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: Migrating agent root cert to cert store 2024-09-10T11:28:52.620 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:28:52.620 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: Migrating agent root key to cert store 2024-09-10T11:28:52.621 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:28:52.621 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: Checking for cert/key for grafana.smithi144 2024-09-10T11:28:52.621 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: Migrating grafana.smithi144 cert to cert store 2024-09-10T11:28:52.621 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:28:52.621 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: Migrating grafana.smithi144 key to cert store 2024-09-10T11:28:52.621 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:28:52.621 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:28:52.621 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:28:52.621 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:28:52.621 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi178.vmvxuf/mirror_snapshot_schedule"}]: dispatch 2024-09-10T11:28:52.621 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi178.vmvxuf/mirror_snapshot_schedule"}]: dispatch 2024-09-10T11:28:52.622 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi178.vmvxuf/trash_purge_schedule"}]: dispatch 2024-09-10T11:28:52.622 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:52 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi178.vmvxuf/trash_purge_schedule"}]: dispatch 2024-09-10T11:28:53.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:53 smithi178 ceph-mon[33514]: Deploying cephadm binary to smithi144 2024-09-10T11:28:53.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:53 smithi178 ceph-mon[33514]: Deploying cephadm binary to smithi178 2024-09-10T11:28:53.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:53 smithi178 ceph-mon[33514]: mgrmap e22: smithi178.vmvxuf(active, since 1.01007s) 2024-09-10T11:28:53.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:53 smithi178 ceph-mon[33514]: pgmap v3: 65 pgs: 65 active+clean; 102 MiB data, 508 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:28:53.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:53 smithi144 ceph-mon[27942]: Deploying cephadm binary to smithi144 2024-09-10T11:28:53.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:53 smithi144 ceph-mon[27942]: Deploying cephadm binary to smithi178 2024-09-10T11:28:53.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:53 smithi144 ceph-mon[27942]: mgrmap e22: smithi178.vmvxuf(active, since 1.01007s) 2024-09-10T11:28:53.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:53 smithi144 ceph-mon[27942]: pgmap v3: 65 pgs: 65 active+clean; 102 MiB data, 508 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:28:54.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:54 smithi178 ceph-mon[33514]: [10/Sep/2024:11:28:53] ENGINE Bus STARTING 2024-09-10T11:28:54.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:54 smithi178 ceph-mon[33514]: [10/Sep/2024:11:28:53] ENGINE Serving on http://172.21.15.178:8765 2024-09-10T11:28:54.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:54 smithi178 ceph-mon[33514]: pgmap v4: 65 pgs: 65 active+clean; 102 MiB data, 508 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:28:54.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:54 smithi178 ceph-mon[33514]: mgrmap e23: smithi178.vmvxuf(active, since 2s) 2024-09-10T11:28:54.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:54 smithi178 ceph-mon[33514]: [10/Sep/2024:11:28:53] ENGINE Serving on https://172.21.15.178:7150 2024-09-10T11:28:54.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:54 smithi178 ceph-mon[33514]: [10/Sep/2024:11:28:53] ENGINE Bus STARTED 2024-09-10T11:28:54.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:54 smithi178 ceph-mon[33514]: [10/Sep/2024:11:28:53] ENGINE Error in HTTPServer.serve 2024-09-10T11:28:54.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Traceback (most recent call last): 2024-09-10T11:28:54.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T11:28:54.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: self._connections.run(self.expiration_interval) 2024-09-10T11:28:54.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T11:28:54.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: self._run(expiration_interval) 2024-09-10T11:28:54.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T11:28:54.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T11:28:54.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T11:28:54.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T11:28:54.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T11:28:54.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: s = self.context.wrap_socket( 2024-09-10T11:28:54.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T11:28:54.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: return self.sslsocket_class._create( 2024-09-10T11:28:54.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T11:28:54.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: self.do_handshake() 2024-09-10T11:28:54.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T11:28:54.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: self._sslobj.do_handshake() 2024-09-10T11:28:54.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T11:28:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:54 smithi144 ceph-mon[27942]: [10/Sep/2024:11:28:53] ENGINE Bus STARTING 2024-09-10T11:28:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:54 smithi144 ceph-mon[27942]: [10/Sep/2024:11:28:53] ENGINE Serving on http://172.21.15.178:8765 2024-09-10T11:28:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:54 smithi144 ceph-mon[27942]: pgmap v4: 65 pgs: 65 active+clean; 102 MiB data, 508 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:28:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:54 smithi144 ceph-mon[27942]: mgrmap e23: smithi178.vmvxuf(active, since 2s) 2024-09-10T11:28:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:54 smithi144 ceph-mon[27942]: [10/Sep/2024:11:28:53] ENGINE Serving on https://172.21.15.178:7150 2024-09-10T11:28:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:54 smithi144 ceph-mon[27942]: [10/Sep/2024:11:28:53] ENGINE Bus STARTED 2024-09-10T11:28:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:54 smithi144 ceph-mon[27942]: [10/Sep/2024:11:28:53] ENGINE Error in HTTPServer.serve 2024-09-10T11:28:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Traceback (most recent call last): 2024-09-10T11:28:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T11:28:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: self._connections.run(self.expiration_interval) 2024-09-10T11:28:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T11:28:54.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: self._run(expiration_interval) 2024-09-10T11:28:54.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T11:28:54.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T11:28:54.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T11:28:54.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T11:28:54.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T11:28:54.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: s = self.context.wrap_socket( 2024-09-10T11:28:54.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T11:28:54.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: return self.sslsocket_class._create( 2024-09-10T11:28:54.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T11:28:54.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: self.do_handshake() 2024-09-10T11:28:54.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T11:28:54.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: self._sslobj.do_handshake() 2024-09-10T11:28:54.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T11:28:56.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:56 smithi144 ceph-mon[27942]: pgmap v5: 65 pgs: 65 active+clean; 102 MiB data, 508 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:28:56.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:56 smithi144 ceph-mon[27942]: mgrmap e24: smithi178.vmvxuf(active, since 4s) 2024-09-10T11:28:56.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:56 smithi178 ceph-mon[33514]: pgmap v5: 65 pgs: 65 active+clean; 102 MiB data, 508 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:28:56.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:56 smithi178 ceph-mon[33514]: mgrmap e24: smithi178.vmvxuf(active, since 4s) 2024-09-10T11:28:58.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:28:58 smithi144 ceph-mon[27942]: pgmap v6: 65 pgs: 65 active+clean; 102 MiB data, 508 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:28:58.696 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:28:58 smithi178 ceph-mon[33514]: pgmap v6: 65 pgs: 65 active+clean; 102 MiB data, 508 MiB used, 536 GiB / 536 GiB avail 2024-09-10T11:29:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:00 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:00 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:00 smithi178 ceph-mon[33514]: pgmap v7: 65 pgs: 65 active+clean; 122 MiB data, 584 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 166 op/s 2024-09-10T11:29:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:00 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:00 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:00 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:00 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:00 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:00.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:00 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:00.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:00 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:00.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:00 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:00.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:00 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:00.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:00 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:00.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:00 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:00.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:00 smithi144 ceph-mon[27942]: pgmap v7: 65 pgs: 65 active+clean; 122 MiB data, 584 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.9 MiB/s wr, 166 op/s 2024-09-10T11:29:00.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:00 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:00.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:00 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:00.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:00 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:00.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:00 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:00.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:00 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:00.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:00 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:00.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:00 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:00.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:00 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:00.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:00 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:01.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:01 smithi178 ceph-mon[33514]: Adjusting osd_memory_target on smithi144 to 1216M 2024-09-10T11:29:01.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:01 smithi178 ceph-mon[33514]: Standby manager daemon smithi144.bkogfs started 2024-09-10T11:29:01.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:01 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.bkogfs/crt"}]: dispatch 2024-09-10T11:29:01.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:01 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T11:29:01.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:01 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.bkogfs/key"}]: dispatch 2024-09-10T11:29:01.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:01 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T11:29:01.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:01 smithi144 ceph-mon[27942]: Adjusting osd_memory_target on smithi144 to 1216M 2024-09-10T11:29:01.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:01 smithi144 ceph-mon[27942]: Standby manager daemon smithi144.bkogfs started 2024-09-10T11:29:01.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:01 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.bkogfs/crt"}]: dispatch 2024-09-10T11:29:01.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:01 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T11:29:01.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:01 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.bkogfs/key"}]: dispatch 2024-09-10T11:29:01.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:01 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.144:0/2' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T11:29:02.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:02 smithi178 ceph-mon[33514]: mgrmap e25: smithi178.vmvxuf(active, since 9s), standbys: smithi144.bkogfs 2024-09-10T11:29:02.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:02 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mgr metadata", "who": "smithi144.bkogfs", "id": "smithi144.bkogfs"}]: dispatch 2024-09-10T11:29:02.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:02 smithi178 ceph-mon[33514]: pgmap v8: 65 pgs: 65 active+clean; 122 MiB data, 584 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 129 op/s 2024-09-10T11:29:02.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:02 smithi144 ceph-mon[27942]: mgrmap e25: smithi178.vmvxuf(active, since 9s), standbys: smithi144.bkogfs 2024-09-10T11:29:02.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:02 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mgr metadata", "who": "smithi144.bkogfs", "id": "smithi144.bkogfs"}]: dispatch 2024-09-10T11:29:02.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:02 smithi144 ceph-mon[27942]: pgmap v8: 65 pgs: 65 active+clean; 122 MiB data, 584 MiB used, 536 GiB / 536 GiB avail; 1.9 MiB/s rd, 2.2 MiB/s wr, 129 op/s 2024-09-10T11:29:04.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:04 smithi178 ceph-mon[33514]: pgmap v9: 65 pgs: 65 active+clean; 141 MiB data, 660 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 234 op/s 2024-09-10T11:29:04.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:04 smithi144 ceph-mon[27942]: pgmap v9: 65 pgs: 65 active+clean; 141 MiB data, 660 MiB used, 536 GiB / 536 GiB avail; 2.7 MiB/s rd, 3.6 MiB/s wr, 234 op/s 2024-09-10T11:29:06.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:06 smithi178 ceph-mon[33514]: pgmap v10: 65 pgs: 65 active+clean; 141 MiB data, 660 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-09-10T11:29:06.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:06 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:06.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:06 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:06 smithi144 ceph-mon[27942]: pgmap v10: 65 pgs: 65 active+clean; 141 MiB data, 660 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-09-10T11:29:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:06 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:06 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:07.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:07 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:29:07.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:07 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:07.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:07 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:07.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:07 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:29:07.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:07 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:07.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:07 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:08.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:08 smithi178 ceph-mon[33514]: pgmap v11: 65 pgs: 65 active+clean; 141 MiB data, 660 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-09-10T11:29:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:08 smithi144 ceph-mon[27942]: pgmap v11: 65 pgs: 65 active+clean; 141 MiB data, 660 MiB used, 536 GiB / 536 GiB avail; 2.4 MiB/s rd, 3.3 MiB/s wr, 214 op/s 2024-09-10T11:29:10.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:10 smithi144 ceph-mon[27942]: pgmap v12: 65 pgs: 65 active+clean; 169 MiB data, 717 MiB used, 536 GiB / 536 GiB avail; 3.5 MiB/s rd, 5.7 MiB/s wr, 423 op/s 2024-09-10T11:29:10.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:10 smithi178 ceph-mon[33514]: pgmap v12: 65 pgs: 65 active+clean; 169 MiB data, 717 MiB used, 536 GiB / 536 GiB avail; 3.5 MiB/s rd, 5.7 MiB/s wr, 423 op/s 2024-09-10T11:29:11.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.458+0000 7f5afc1e4700 1 -- 172.21.15.144:0/2921598326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5af40ff090 msgr2=0x7f5af40ff4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:11.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.458+0000 7f5afc1e4700 1 --2- 172.21.15.144:0/2921598326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5af40ff090 0x7f5af40ff4a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009910 tx=0x7f5ae8009c20 comp rx=0 tx=0).stop 2024-09-10T11:29:11.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.458+0000 7f5afc1e4700 1 -- 172.21.15.144:0/2921598326 shutdown_connections 2024-09-10T11:29:11.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.458+0000 7f5afc1e4700 1 --2- 172.21.15.144:0/2921598326 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5af4100290 0x7f5af41006e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:11.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.458+0000 7f5afc1e4700 1 --2- 172.21.15.144:0/2921598326 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5af40ff090 0x7f5af40ff4a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:11.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.458+0000 7f5afc1e4700 1 -- 172.21.15.144:0/2921598326 >> 172.21.15.144:0/2921598326 conn(0x7f5af40fa660 msgr2=0x7f5af40fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:11.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.458+0000 7f5afc1e4700 1 -- 172.21.15.144:0/2921598326 shutdown_connections 2024-09-10T11:29:11.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.458+0000 7f5afc1e4700 1 -- 172.21.15.144:0/2921598326 wait complete. 2024-09-10T11:29:11.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.459+0000 7f5afc1e4700 1 Processor -- start 2024-09-10T11:29:11.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.459+0000 7f5afc1e4700 1 -- start start 2024-09-10T11:29:11.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.459+0000 7f5afc1e4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5af40ff090 0x7f5af4194890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:11.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.459+0000 7f5afc1e4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5af4100290 0x7f5af4194dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:11.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.459+0000 7f5afc1e4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af41953f0 con 0x7f5af40ff090 2024-09-10T11:29:11.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.459+0000 7f5afc1e4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af4195530 con 0x7f5af4100290 2024-09-10T11:29:11.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.459+0000 7f5af9f80700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5af40ff090 0x7f5af4194890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:11.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.459+0000 7f5af9f80700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5af40ff090 0x7f5af4194890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56584/0 (socket says 172.21.15.144:56584) 2024-09-10T11:29:11.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.459+0000 7f5af9f80700 1 -- 172.21.15.144:0/4049395031 learned_addr learned my addr 172.21.15.144:0/4049395031 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:29:11.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.459+0000 7f5af977f700 1 --2- 172.21.15.144:0/4049395031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5af4100290 0x7f5af4194dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:11.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.460+0000 7f5af9f80700 1 -- 172.21.15.144:0/4049395031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5af4100290 msgr2=0x7f5af4194dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:11.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.460+0000 7f5af9f80700 1 --2- 172.21.15.144:0/4049395031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5af4100290 0x7f5af4194dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:11.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.460+0000 7f5af9f80700 1 -- 172.21.15.144:0/4049395031 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae80095f0 con 0x7f5af40ff090 2024-09-10T11:29:11.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.460+0000 7f5af9f80700 1 --2- 172.21.15.144:0/4049395031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5af40ff090 0x7f5af4194890 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8000c00 tx=0x7f5ae801a850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:11.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.460+0000 7f5ae6ffd700 1 -- 172.21.15.144:0/4049395031 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ae801f070 con 0x7f5af40ff090 2024-09-10T11:29:11.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.460+0000 7f5ae6ffd700 1 -- 172.21.15.144:0/4049395031 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5ae8005dc0 con 0x7f5af40ff090 2024-09-10T11:29:11.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.460+0000 7f5afc1e4700 1 -- 172.21.15.144:0/4049395031 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af4068d70 con 0x7f5af40ff090 2024-09-10T11:29:11.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.460+0000 7f5ae6ffd700 1 -- 172.21.15.144:0/4049395031 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ae800f460 con 0x7f5af40ff090 2024-09-10T11:29:11.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.460+0000 7f5afc1e4700 1 -- 172.21.15.144:0/4049395031 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af4069260 con 0x7f5af40ff090 2024-09-10T11:29:11.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.461+0000 7f5afc1e4700 1 -- 172.21.15.144:0/4049395031 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af4107be0 con 0x7f5af40ff090 2024-09-10T11:29:11.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.462+0000 7f5ae6ffd700 1 -- 172.21.15.144:0/4049395031 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) v1 ==== 91089+0+0 (secure 0 0 0) 0x7f5ae8023070 con 0x7f5af40ff090 2024-09-10T11:29:11.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.462+0000 7f5ae6ffd700 1 --2- 172.21.15.144:0/4049395031 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f5ae006d3c0 0x7f5ae006f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:11.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.462+0000 7f5ae6ffd700 1 -- 172.21.15.144:0/4049395031 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5ae808ff80 con 0x7f5af40ff090 2024-09-10T11:29:11.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.462+0000 7f5af977f700 1 --2- 172.21.15.144:0/4049395031 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f5ae006d3c0 0x7f5ae006f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:11.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.463+0000 7f5af977f700 1 --2- 172.21.15.144:0/4049395031 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f5ae006d3c0 0x7f5ae006f870 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009c60 tx=0x7f5af0009260 comp rx=0 tx=0).ready entity=mgr.24329 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:11.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.469+0000 7f5ae6ffd700 1 -- 172.21.15.144:0/4049395031 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f5ae805d2f0 con 0x7f5af40ff090 2024-09-10T11:29:11.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.739+0000 7f5afc1e4700 1 -- 172.21.15.144:0/4049395031 --> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5af4104be0 con 0x7f5ae006d3c0 2024-09-10T11:29:11.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.741+0000 7f5ae6ffd700 1 -- 172.21.15.144:0/4049395031 <== mgr.24329 v2:172.21.15.178:6828/2974090415 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f5af4104be0 con 0x7f5ae006d3c0 2024-09-10T11:29:11.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.743+0000 7f5afc1e4700 1 -- 172.21.15.144:0/4049395031 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f5ae006d3c0 msgr2=0x7f5ae006f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:11.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.743+0000 7f5afc1e4700 1 --2- 172.21.15.144:0/4049395031 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f5ae006d3c0 0x7f5ae006f870 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f5af0009c60 tx=0x7f5af0009260 comp rx=0 tx=0).stop 2024-09-10T11:29:11.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.743+0000 7f5afc1e4700 1 -- 172.21.15.144:0/4049395031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5af40ff090 msgr2=0x7f5af4194890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:11.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.743+0000 7f5afc1e4700 1 --2- 172.21.15.144:0/4049395031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5af40ff090 0x7f5af4194890 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8000c00 tx=0x7f5ae801a850 comp rx=0 tx=0).stop 2024-09-10T11:29:11.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.743+0000 7f5afc1e4700 1 -- 172.21.15.144:0/4049395031 shutdown_connections 2024-09-10T11:29:11.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.743+0000 7f5afc1e4700 1 --2- 172.21.15.144:0/4049395031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5af40ff090 0x7f5af4194890 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:11.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.743+0000 7f5afc1e4700 1 --2- 172.21.15.144:0/4049395031 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f5ae006d3c0 0x7f5ae006f870 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:11.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.743+0000 7f5afc1e4700 1 --2- 172.21.15.144:0/4049395031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5af4100290 0x7f5af4194dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:11.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.743+0000 7f5afc1e4700 1 -- 172.21.15.144:0/4049395031 >> 172.21.15.144:0/4049395031 conn(0x7f5af40fa660 msgr2=0x7f5af41034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:11.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.743+0000 7f5afc1e4700 1 -- 172.21.15.144:0/4049395031 shutdown_connections 2024-09-10T11:29:11.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.743+0000 7f5afc1e4700 1 -- 172.21.15.144:0/4049395031 wait complete. 2024-09-10T11:29:11.755 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:29:11.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.871+0000 7fdb22066700 1 -- 172.21.15.144:0/453839945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c0ff050 msgr2=0x7fdb1c0ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:11.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.871+0000 7fdb22066700 1 --2- 172.21.15.144:0/453839945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c0ff050 0x7fdb1c0ff460 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c009910 tx=0x7fdb0c009c20 comp rx=0 tx=0).stop 2024-09-10T11:29:11.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.871+0000 7fdb22066700 1 -- 172.21.15.144:0/453839945 shutdown_connections 2024-09-10T11:29:11.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.871+0000 7fdb22066700 1 --2- 172.21.15.144:0/453839945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb1c100250 0x7fdb1c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:11.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.871+0000 7fdb22066700 1 --2- 172.21.15.144:0/453839945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c0ff050 0x7fdb1c0ff460 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:11.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.871+0000 7fdb22066700 1 -- 172.21.15.144:0/453839945 >> 172.21.15.144:0/453839945 conn(0x7fdb1c0fa600 msgr2=0x7fdb1c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:11.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.871+0000 7fdb22066700 1 -- 172.21.15.144:0/453839945 shutdown_connections 2024-09-10T11:29:11.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.871+0000 7fdb22066700 1 -- 172.21.15.144:0/453839945 wait complete. 2024-09-10T11:29:11.874 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.871+0000 7fdb22066700 1 Processor -- start 2024-09-10T11:29:11.874 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.872+0000 7fdb22066700 1 -- start start 2024-09-10T11:29:11.874 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.872+0000 7fdb22066700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c0ff050 0x7fdb1c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:11.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.872+0000 7fdb22066700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb1c100250 0x7fdb1c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:11.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.872+0000 7fdb22066700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c1986c0 con 0x7fdb1c0ff050 2024-09-10T11:29:11.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.872+0000 7fdb22066700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1c198800 con 0x7fdb1c100250 2024-09-10T11:29:11.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.872+0000 7fdb1b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c0ff050 0x7fdb1c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:11.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.872+0000 7fdb1b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c0ff050 0x7fdb1c197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56604/0 (socket says 172.21.15.144:56604) 2024-09-10T11:29:11.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.872+0000 7fdb1b7fe700 1 -- 172.21.15.144:0/2814595020 learned_addr learned my addr 172.21.15.144:0/2814595020 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:29:11.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.872+0000 7fdb1affd700 1 --2- 172.21.15.144:0/2814595020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb1c100250 0x7fdb1c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:11.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.872+0000 7fdb1b7fe700 1 -- 172.21.15.144:0/2814595020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb1c100250 msgr2=0x7fdb1c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:11.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.872+0000 7fdb1b7fe700 1 --2- 172.21.15.144:0/2814595020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb1c100250 0x7fdb1c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:11.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.872+0000 7fdb1b7fe700 1 -- 172.21.15.144:0/2814595020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb0c0095f0 con 0x7fdb1c0ff050 2024-09-10T11:29:11.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.873+0000 7fdb1b7fe700 1 --2- 172.21.15.144:0/2814595020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c0ff050 0x7fdb1c197b60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c000c00 tx=0x7fdb0c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:11.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.873+0000 7fdb18ff9700 1 -- 172.21.15.144:0/2814595020 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb0c01d070 con 0x7fdb1c0ff050 2024-09-10T11:29:11.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.873+0000 7fdb18ff9700 1 -- 172.21.15.144:0/2814595020 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdb0c005dc0 con 0x7fdb1c0ff050 2024-09-10T11:29:11.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.873+0000 7fdb22066700 1 -- 172.21.15.144:0/2814595020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1c19d250 con 0x7fdb1c0ff050 2024-09-10T11:29:11.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.873+0000 7fdb18ff9700 1 -- 172.21.15.144:0/2814595020 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb0c022470 con 0x7fdb1c0ff050 2024-09-10T11:29:11.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.873+0000 7fdb22066700 1 -- 172.21.15.144:0/2814595020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1c19d740 con 0x7fdb1c0ff050 2024-09-10T11:29:11.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.874+0000 7fdb22066700 1 -- 172.21.15.144:0/2814595020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb1c107ba0 con 0x7fdb1c0ff050 2024-09-10T11:29:11.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.875+0000 7fdb18ff9700 1 -- 172.21.15.144:0/2814595020 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) v1 ==== 91089+0+0 (secure 0 0 0) 0x7fdb0c003810 con 0x7fdb1c0ff050 2024-09-10T11:29:11.877 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.875+0000 7fdb18ff9700 1 --2- 172.21.15.144:0/2814595020 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7fdb0406d3c0 0x7fdb0406f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:11.877 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.875+0000 7fdb18ff9700 1 -- 172.21.15.144:0/2814595020 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdb0c08e110 con 0x7fdb1c0ff050 2024-09-10T11:29:11.877 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.875+0000 7fdb1affd700 1 --2- 172.21.15.144:0/2814595020 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7fdb0406d3c0 0x7fdb0406f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:11.877 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.876+0000 7fdb1affd700 1 --2- 172.21.15.144:0/2814595020 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7fdb0406d3c0 0x7fdb0406f870 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009c60 tx=0x7fdb10009260 comp rx=0 tx=0).ready entity=mgr.24329 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:11.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:11.880+0000 7fdb18ff9700 1 -- 172.21.15.144:0/2814595020 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7fdb0c05b480 con 0x7fdb1c0ff050 2024-09-10T11:29:12.122 INFO:tasks.workunit.client.1.smithi178.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-09-10T11:29:12.128 INFO:tasks.workunit.client.1.smithi178.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-10T11:29:12.129 INFO:tasks.workunit.client.1.smithi178.stderr:+ make 2024-09-10T11:29:12.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.182+0000 7fdb22066700 1 -- 172.21.15.144:0/2814595020 --> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb1c069350 con 0x7fdb0406d3c0 2024-09-10T11:29:12.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.184+0000 7fdb18ff9700 1 -- 172.21.15.144:0/2814595020 <== mgr.24329 v2:172.21.15.178:6828/2974090415 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fdb1c069350 con 0x7fdb0406d3c0 2024-09-10T11:29:12.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.187+0000 7fdb22066700 1 -- 172.21.15.144:0/2814595020 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7fdb0406d3c0 msgr2=0x7fdb0406f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:12.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.187+0000 7fdb22066700 1 --2- 172.21.15.144:0/2814595020 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7fdb0406d3c0 0x7fdb0406f870 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fdb10009c60 tx=0x7fdb10009260 comp rx=0 tx=0).stop 2024-09-10T11:29:12.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.187+0000 7fdb22066700 1 -- 172.21.15.144:0/2814595020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c0ff050 msgr2=0x7fdb1c197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:12.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.187+0000 7fdb22066700 1 --2- 172.21.15.144:0/2814595020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c0ff050 0x7fdb1c197b60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fdb0c000c00 tx=0x7fdb0c00f690 comp rx=0 tx=0).stop 2024-09-10T11:29:12.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.188+0000 7fdb22066700 1 -- 172.21.15.144:0/2814595020 shutdown_connections 2024-09-10T11:29:12.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.188+0000 7fdb22066700 1 --2- 172.21.15.144:0/2814595020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb1c0ff050 0x7fdb1c197b60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:12.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.188+0000 7fdb22066700 1 --2- 172.21.15.144:0/2814595020 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7fdb0406d3c0 0x7fdb0406f870 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:12.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.188+0000 7fdb22066700 1 --2- 172.21.15.144:0/2814595020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb1c100250 0x7fdb1c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:12.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.188+0000 7fdb22066700 1 -- 172.21.15.144:0/2814595020 >> 172.21.15.144:0/2814595020 conn(0x7fdb1c0fa600 msgr2=0x7fdb1c068c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:12.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.188+0000 7fdb22066700 1 -- 172.21.15.144:0/2814595020 shutdown_connections 2024-09-10T11:29:12.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.188+0000 7fdb22066700 1 -- 172.21.15.144:0/2814595020 wait complete. 2024-09-10T11:29:12.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.337+0000 7f0985a16700 1 -- 172.21.15.144:0/2826093754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0980100ca0 msgr2=0x7f0980103080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:12.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.337+0000 7f0985a16700 1 --2- 172.21.15.144:0/2826093754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0980100ca0 0x7f0980103080 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f0970009910 tx=0x7f0970009c20 comp rx=0 tx=0).stop 2024-09-10T11:29:12.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.338+0000 7f0985a16700 1 -- 172.21.15.144:0/2826093754 shutdown_connections 2024-09-10T11:29:12.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.338+0000 7f0985a16700 1 --2- 172.21.15.144:0/2826093754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09801035c0 0x7f09801059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:12.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.338+0000 7f0985a16700 1 --2- 172.21.15.144:0/2826093754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0980100ca0 0x7f0980103080 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:12.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.338+0000 7f0985a16700 1 -- 172.21.15.144:0/2826093754 >> 172.21.15.144:0/2826093754 conn(0x7f09800fa5e0 msgr2=0x7f09800fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:12.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.338+0000 7f0985a16700 1 -- 172.21.15.144:0/2826093754 shutdown_connections 2024-09-10T11:29:12.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.338+0000 7f0985a16700 1 -- 172.21.15.144:0/2826093754 wait complete. 2024-09-10T11:29:12.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.339+0000 7f0985a16700 1 Processor -- start 2024-09-10T11:29:12.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.339+0000 7f0985a16700 1 -- start start 2024-09-10T11:29:12.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.339+0000 7f0985a16700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0980100ca0 0x7f0980195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:12.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.339+0000 7f0985a16700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09801035c0 0x7f0980195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:12.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.339+0000 7f0985a16700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09801964b0 con 0x7f09801035c0 2024-09-10T11:29:12.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.339+0000 7f0985a16700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09801965f0 con 0x7f0980100ca0 2024-09-10T11:29:12.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.339+0000 7f097e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09801035c0 0x7f0980195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:12.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.339+0000 7f097e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09801035c0 0x7f0980195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57104/0 (socket says 172.21.15.144:57104) 2024-09-10T11:29:12.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.339+0000 7f097e7fc700 1 -- 172.21.15.144:0/833362916 learned_addr learned my addr 172.21.15.144:0/833362916 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:29:12.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.339+0000 7f097effd700 1 --2- 172.21.15.144:0/833362916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0980100ca0 0x7f0980195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:12.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.340+0000 7f097e7fc700 1 -- 172.21.15.144:0/833362916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0980100ca0 msgr2=0x7f0980195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:12.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.340+0000 7f097e7fc700 1 --2- 172.21.15.144:0/833362916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0980100ca0 0x7f0980195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:12.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.340+0000 7f097e7fc700 1 -- 172.21.15.144:0/833362916 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09700095f0 con 0x7f09801035c0 2024-09-10T11:29:12.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.340+0000 7f097effd700 1 --2- 172.21.15.144:0/833362916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0980100ca0 0x7f0980195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:29:12.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.340+0000 7f097e7fc700 1 --2- 172.21.15.144:0/833362916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09801035c0 0x7f0980195e90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f097400e920 tx=0x7f097400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:12.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.340+0000 7f0984a14700 1 -- 172.21.15.144:0/833362916 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f097400c9d0 con 0x7f09801035c0 2024-09-10T11:29:12.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.340+0000 7f0985a16700 1 -- 172.21.15.144:0/833362916 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098019b0a0 con 0x7f09801035c0 2024-09-10T11:29:12.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.340+0000 7f0985a16700 1 -- 172.21.15.144:0/833362916 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098019b5f0 con 0x7f09801035c0 2024-09-10T11:29:12.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.340+0000 7f0984a14700 1 -- 172.21.15.144:0/833362916 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0974010430 con 0x7f09801035c0 2024-09-10T11:29:12.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.340+0000 7f0984a14700 1 -- 172.21.15.144:0/833362916 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0974018650 con 0x7f09801035c0 2024-09-10T11:29:12.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.341+0000 7f0985a16700 1 -- 172.21.15.144:0/833362916 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096c005320 con 0x7f09801035c0 2024-09-10T11:29:12.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.342+0000 7f0984a14700 1 -- 172.21.15.144:0/833362916 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) v1 ==== 91089+0+0 (secure 0 0 0) 0x7f097400cb30 con 0x7f09801035c0 2024-09-10T11:29:12.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.343+0000 7f0984a14700 1 --2- 172.21.15.144:0/833362916 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f096806d3c0 0x7f096806f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:12.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.343+0000 7f0984a14700 1 -- 172.21.15.144:0/833362916 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0974014070 con 0x7f09801035c0 2024-09-10T11:29:12.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.343+0000 7f097effd700 1 --2- 172.21.15.144:0/833362916 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f096806d3c0 0x7f096806f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:12.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.344+0000 7f097effd700 1 --2- 172.21.15.144:0/833362916 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f096806d3c0 0x7f096806f870 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0970009fd0 tx=0x7f0970005940 comp rx=0 tx=0).ready entity=mgr.24329 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:12.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.350+0000 7f0984a14700 1 -- 172.21.15.144:0/833362916 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f09740539d0 con 0x7f09801035c0 2024-09-10T11:29:12.382 INFO:tasks.workunit.client.1.smithi178.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-09-10T11:29:12.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.629+0000 7f0985a16700 1 -- 172.21.15.144:0/833362916 --> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f096c000bf0 con 0x7f096806d3c0 2024-09-10T11:29:12.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.640+0000 7f0984a14700 1 -- 172.21.15.144:0/833362916 <== mgr.24329 v2:172.21.15.178:6828/2974090415 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f096c000bf0 con 0x7f096806d3c0 2024-09-10T11:29:12.641 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:29:12.641 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (6m) 13s ago 7m 24.5M - 0.25.0 c8568f914cd2 b5c0bfae59de 2024-09-10T11:29:12.641 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (8m) 13s ago 8m 7680k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:29:12.641 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (7m) 6s ago 7m 7629k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:29:12.641 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (8m) 13s ago 8m 6665k - 18.2.0 dc2bc1663786 3311fb04aa0c 2024-09-10T11:29:12.642 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (7m) 6s ago 7m 6651k - 18.2.0 dc2bc1663786 6e408e8476d4 2024-09-10T11:29:12.642 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (6m) 13s ago 7m 88.6M - 9.4.7 954c08fa6188 e9a1aec955f6 2024-09-10T11:29:12.642 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (2m) 13s ago 2m 76.5M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:29:12.642 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (2m) 13s ago 2m 103M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:29:12.642 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (2m) 6s ago 2m 60.0M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:29:12.642 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (2m) 6s ago 2m 90.1M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:29:12.642 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:9283,8765,8443 running (9m) 13s ago 9m 171M - 18.2.0 dc2bc1663786 71a7e1c8a428 2024-09-10T11:29:12.642 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (34s) 6s ago 7m 480M - 19.1.1-305-g1cf819b7 41d671128357 859a740b57c8 2024-09-10T11:29:12.642 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (9m) 13s ago 9m 48.8M 2048M 18.2.0 dc2bc1663786 42addb654e4f 2024-09-10T11:29:12.642 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (6m) 6s ago 6m 40.4M 2048M 18.2.0 dc2bc1663786 3c1ba210ed3f 2024-09-10T11:29:12.642 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (8m) 13s ago 8m 16.3M - 1.5.0 0da6a335fe13 dfbf0c488d17 2024-09-10T11:29:12.642 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (7m) 6s ago 7m 15.2M - 1.5.0 0da6a335fe13 ff2cac118c8d 2024-09-10T11:29:12.642 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (6m) 13s ago 6m 78.9M 1216M 18.2.0 dc2bc1663786 2435febebdb5 2024-09-10T11:29:12.642 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (5m) 13s ago 5m 83.0M 1216M 18.2.0 dc2bc1663786 b55fe920ff9d 2024-09-10T11:29:12.643 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (5m) 13s ago 5m 70.1M 1216M 18.2.0 dc2bc1663786 3ed5618eac33 2024-09-10T11:29:12.643 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (4m) 6s ago 4m 76.3M 4970M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:29:12.643 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (4m) 6s ago 4m 70.5M 4970M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:29:12.643 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (4m) 6s ago 4m 68.7M 4970M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:29:12.643 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (6m) 13s ago 7m 65.9M - 2.43.0 a07b618ecd1d a463a804d232 2024-09-10T11:29:12.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.643+0000 7f0985a16700 1 -- 172.21.15.144:0/833362916 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f096806d3c0 msgr2=0x7f096806f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:12.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.643+0000 7f0985a16700 1 --2- 172.21.15.144:0/833362916 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f096806d3c0 0x7f096806f870 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f0970009fd0 tx=0x7f0970005940 comp rx=0 tx=0).stop 2024-09-10T11:29:12.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.643+0000 7f0985a16700 1 -- 172.21.15.144:0/833362916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09801035c0 msgr2=0x7f0980195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:12.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.643+0000 7f0985a16700 1 --2- 172.21.15.144:0/833362916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09801035c0 0x7f0980195e90 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f097400e920 tx=0x7f097400ec30 comp rx=0 tx=0).stop 2024-09-10T11:29:12.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.643+0000 7f0985a16700 1 -- 172.21.15.144:0/833362916 shutdown_connections 2024-09-10T11:29:12.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.643+0000 7f0985a16700 1 --2- 172.21.15.144:0/833362916 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f096806d3c0 0x7f096806f870 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:12.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.643+0000 7f0985a16700 1 --2- 172.21.15.144:0/833362916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0980100ca0 0x7f0980195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:12.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.643+0000 7f0985a16700 1 --2- 172.21.15.144:0/833362916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09801035c0 0x7f0980195e90 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:12.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.643+0000 7f0985a16700 1 -- 172.21.15.144:0/833362916 >> 172.21.15.144:0/833362916 conn(0x7f09800fa5e0 msgr2=0x7f09800fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:12.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.643+0000 7f0985a16700 1 -- 172.21.15.144:0/833362916 shutdown_connections 2024-09-10T11:29:12.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.643+0000 7f0985a16700 1 -- 172.21.15.144:0/833362916 wait complete. 2024-09-10T11:29:12.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.764+0000 7ff2bb6b3700 1 -- 172.21.15.144:0/2381570024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b40fe7d0 msgr2=0x7ff2b40febe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:12.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.764+0000 7ff2bb6b3700 1 --2- 172.21.15.144:0/2381570024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b40fe7d0 0x7ff2b40febe0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0009910 tx=0x7ff2b0009c20 comp rx=0 tx=0).stop 2024-09-10T11:29:12.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.764+0000 7ff2bb6b3700 1 -- 172.21.15.144:0/2381570024 shutdown_connections 2024-09-10T11:29:12.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.764+0000 7ff2bb6b3700 1 --2- 172.21.15.144:0/2381570024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b40ff9d0 0x7ff2b40ffe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:12.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.764+0000 7ff2bb6b3700 1 --2- 172.21.15.144:0/2381570024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b40fe7d0 0x7ff2b40febe0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:12.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.764+0000 7ff2bb6b3700 1 -- 172.21.15.144:0/2381570024 >> 172.21.15.144:0/2381570024 conn(0x7ff2b40f9d80 msgr2=0x7ff2b40fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:12.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.764+0000 7ff2bb6b3700 1 -- 172.21.15.144:0/2381570024 shutdown_connections 2024-09-10T11:29:12.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.764+0000 7ff2bb6b3700 1 -- 172.21.15.144:0/2381570024 wait complete. 2024-09-10T11:29:12.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.765+0000 7ff2bb6b3700 1 Processor -- start 2024-09-10T11:29:12.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.765+0000 7ff2bb6b3700 1 -- start start 2024-09-10T11:29:12.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.765+0000 7ff2bb6b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b40fe7d0 0x7ff2b4195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:12.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.765+0000 7ff2bb6b3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b40ff9d0 0x7ff2b4195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:12.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.765+0000 7ff2bb6b3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b41964d0 con 0x7ff2b40fe7d0 2024-09-10T11:29:12.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.765+0000 7ff2bb6b3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b4196610 con 0x7ff2b40ff9d0 2024-09-10T11:29:12.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.765+0000 7ff2b944f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b40fe7d0 0x7ff2b4195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:12.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.765+0000 7ff2b944f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b40fe7d0 0x7ff2b4195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57122/0 (socket says 172.21.15.144:57122) 2024-09-10T11:29:12.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.765+0000 7ff2b944f700 1 -- 172.21.15.144:0/2105550128 learned_addr learned my addr 172.21.15.144:0/2105550128 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:29:12.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.765+0000 7ff2b8c4e700 1 --2- 172.21.15.144:0/2105550128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b40ff9d0 0x7ff2b4195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:12.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.765+0000 7ff2b944f700 1 -- 172.21.15.144:0/2105550128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b40ff9d0 msgr2=0x7ff2b4195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:12.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.765+0000 7ff2b944f700 1 --2- 172.21.15.144:0/2105550128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b40ff9d0 0x7ff2b4195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:12.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.765+0000 7ff2b944f700 1 -- 172.21.15.144:0/2105550128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b00095f0 con 0x7ff2b40fe7d0 2024-09-10T11:29:12.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.766+0000 7ff2b944f700 1 --2- 172.21.15.144:0/2105550128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b40fe7d0 0x7ff2b4195970 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000fa00 tx=0x7ff2b000fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:12.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.766+0000 7ff2aa7fc700 1 -- 172.21.15.144:0/2105550128 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2b001d070 con 0x7ff2b40fe7d0 2024-09-10T11:29:12.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.766+0000 7ff2bb6b3700 1 -- 172.21.15.144:0/2105550128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b419b060 con 0x7ff2b40fe7d0 2024-09-10T11:29:12.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.766+0000 7ff2bb6b3700 1 -- 172.21.15.144:0/2105550128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b419b550 con 0x7ff2b40fe7d0 2024-09-10T11:29:12.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.766+0000 7ff2aa7fc700 1 -- 172.21.15.144:0/2105550128 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff2b000fe90 con 0x7ff2b40fe7d0 2024-09-10T11:29:12.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.766+0000 7ff2aa7fc700 1 -- 172.21.15.144:0/2105550128 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2b0017630 con 0x7ff2b40fe7d0 2024-09-10T11:29:12.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.767+0000 7ff2bb6b3700 1 -- 172.21.15.144:0/2105550128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff298005320 con 0x7ff2b40fe7d0 2024-09-10T11:29:12.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.768+0000 7ff2aa7fc700 1 -- 172.21.15.144:0/2105550128 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) v1 ==== 91089+0+0 (secure 0 0 0) 0x7ff2b0003680 con 0x7ff2b40fe7d0 2024-09-10T11:29:12.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.768+0000 7ff2aa7fc700 1 --2- 172.21.15.144:0/2105550128 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7ff2a006d3c0 0x7ff2a006f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:12.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.768+0000 7ff2aa7fc700 1 -- 172.21.15.144:0/2105550128 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff2b008df90 con 0x7ff2b40fe7d0 2024-09-10T11:29:12.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.768+0000 7ff2b8c4e700 1 --2- 172.21.15.144:0/2105550128 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7ff2a006d3c0 0x7ff2a006f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:12.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.769+0000 7ff2b8c4e700 1 --2- 172.21.15.144:0/2105550128 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7ff2a006d3c0 0x7ff2a006f870 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009c60 tx=0x7ff2a4009260 comp rx=0 tx=0).ready entity=mgr.24329 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:12.774 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:12.773+0000 7ff2aa7fc700 1 -- 172.21.15.144:0/2105550128 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7ff2b005b300 con 0x7ff2b40fe7d0 2024-09-10T11:29:12.891 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:12 smithi144 ceph-mon[27942]: pgmap v13: 65 pgs: 65 active+clean; 169 MiB data, 717 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 326 op/s 2024-09-10T11:29:12.891 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:12 smithi144 ceph-mon[27942]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:13.088 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:12 smithi178 ceph-mon[33514]: pgmap v13: 65 pgs: 65 active+clean; 169 MiB data, 717 MiB used, 536 GiB / 536 GiB avail; 2.1 MiB/s rd, 4.0 MiB/s wr, 326 op/s 2024-09-10T11:29:13.089 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:12 smithi178 ceph-mon[33514]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:13.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.130+0000 7ff2bb6b3700 1 -- 172.21.15.144:0/2105550128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff298005cc0 con 0x7ff2b40fe7d0 2024-09-10T11:29:13.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.131+0000 7ff2aa7fc700 1 -- 172.21.15.144:0/2105550128 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+770 (secure 0 0 0) 0x7ff2b0003e60 con 0x7ff2b40fe7d0 2024-09-10T11:29:13.133 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:29:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:29:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-09-10T11:29:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:29:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:29:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:29:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 1 2024-09-10T11:29:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:29:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:29:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-09-10T11:29:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:29:13.133 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:29:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:29:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:29:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:29:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 13, 2024-09-10T11:29:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 1 2024-09-10T11:29:13.134 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:29:13.134 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:29:13.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.134+0000 7ff2bb6b3700 1 -- 172.21.15.144:0/2105550128 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7ff2a006d3c0 msgr2=0x7ff2a006f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:13.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.134+0000 7ff2bb6b3700 1 --2- 172.21.15.144:0/2105550128 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7ff2a006d3c0 0x7ff2a006f870 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff2a4009c60 tx=0x7ff2a4009260 comp rx=0 tx=0).stop 2024-09-10T11:29:13.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.134+0000 7ff2bb6b3700 1 -- 172.21.15.144:0/2105550128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b40fe7d0 msgr2=0x7ff2b4195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:13.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.134+0000 7ff2bb6b3700 1 --2- 172.21.15.144:0/2105550128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b40fe7d0 0x7ff2b4195970 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000fa00 tx=0x7ff2b000fa30 comp rx=0 tx=0).stop 2024-09-10T11:29:13.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.134+0000 7ff2bb6b3700 1 -- 172.21.15.144:0/2105550128 shutdown_connections 2024-09-10T11:29:13.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.134+0000 7ff2bb6b3700 1 --2- 172.21.15.144:0/2105550128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff2b40fe7d0 0x7ff2b4195970 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:13.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.134+0000 7ff2bb6b3700 1 --2- 172.21.15.144:0/2105550128 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7ff2a006d3c0 0x7ff2a006f870 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:13.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.134+0000 7ff2bb6b3700 1 --2- 172.21.15.144:0/2105550128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff2b40ff9d0 0x7ff2b4195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:13.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.134+0000 7ff2bb6b3700 1 -- 172.21.15.144:0/2105550128 >> 172.21.15.144:0/2105550128 conn(0x7ff2b40f9d80 msgr2=0x7ff2b4104e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:13.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.134+0000 7ff2bb6b3700 1 -- 172.21.15.144:0/2105550128 shutdown_connections 2024-09-10T11:29:13.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.134+0000 7ff2bb6b3700 1 -- 172.21.15.144:0/2105550128 wait complete. 2024-09-10T11:29:13.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.259+0000 7f4a869a5700 1 -- 172.21.15.144:0/2385769513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a80100170 msgr2=0x7f4a80100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:13.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.259+0000 7f4a869a5700 1 --2- 172.21.15.144:0/2385769513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a80100170 0x7f4a80100580 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009910 tx=0x7f4a70009c20 comp rx=0 tx=0).stop 2024-09-10T11:29:13.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.259+0000 7f4a869a5700 1 -- 172.21.15.144:0/2385769513 shutdown_connections 2024-09-10T11:29:13.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.259+0000 7f4a869a5700 1 --2- 172.21.15.144:0/2385769513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a80101370 0x7f4a801017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:13.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.259+0000 7f4a869a5700 1 --2- 172.21.15.144:0/2385769513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a80100170 0x7f4a80100580 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:13.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.259+0000 7f4a869a5700 1 -- 172.21.15.144:0/2385769513 >> 172.21.15.144:0/2385769513 conn(0x7f4a800fb720 msgr2=0x7f4a800fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:13.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.259+0000 7f4a869a5700 1 -- 172.21.15.144:0/2385769513 shutdown_connections 2024-09-10T11:29:13.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.259+0000 7f4a869a5700 1 -- 172.21.15.144:0/2385769513 wait complete. 2024-09-10T11:29:13.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.260+0000 7f4a869a5700 1 Processor -- start 2024-09-10T11:29:13.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.260+0000 7f4a869a5700 1 -- start start 2024-09-10T11:29:13.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.260+0000 7f4a869a5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a80100170 0x7f4a80074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:13.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.260+0000 7f4a869a5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a80101370 0x7f4a80073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:13.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.260+0000 7f4a869a5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a80073680 con 0x7f4a80100170 2024-09-10T11:29:13.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.260+0000 7f4a869a5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a800737c0 con 0x7f4a80101370 2024-09-10T11:29:13.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.260+0000 7f4a7ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a80100170 0x7f4a80074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:13.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.260+0000 7f4a7f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a80101370 0x7f4a80073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:13.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.260+0000 7f4a7ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a80100170 0x7f4a80074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57134/0 (socket says 172.21.15.144:57134) 2024-09-10T11:29:13.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.261+0000 7f4a7ffff700 1 -- 172.21.15.144:0/2618582902 learned_addr learned my addr 172.21.15.144:0/2618582902 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:29:13.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.261+0000 7f4a7f7fe700 1 -- 172.21.15.144:0/2618582902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a80100170 msgr2=0x7f4a80074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:13.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.261+0000 7f4a7f7fe700 1 --2- 172.21.15.144:0/2618582902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a80100170 0x7f4a80074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:13.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.261+0000 7f4a7f7fe700 1 -- 172.21.15.144:0/2618582902 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a700095f0 con 0x7f4a80101370 2024-09-10T11:29:13.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.261+0000 7f4a7f7fe700 1 --2- 172.21.15.144:0/2618582902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a80101370 0x7f4a80073140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400e970 tx=0x7f4a7400ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:13.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.261+0000 7f4a7d7fa700 1 -- 172.21.15.144:0/2618582902 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a7400c9d0 con 0x7f4a80101370 2024-09-10T11:29:13.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.261+0000 7f4a7d7fa700 1 -- 172.21.15.144:0/2618582902 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4a74010430 con 0x7f4a80101370 2024-09-10T11:29:13.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.261+0000 7f4a869a5700 1 -- 172.21.15.144:0/2618582902 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a80073aa0 con 0x7f4a80101370 2024-09-10T11:29:13.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.261+0000 7f4a7d7fa700 1 -- 172.21.15.144:0/2618582902 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a74018650 con 0x7f4a80101370 2024-09-10T11:29:13.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.261+0000 7f4a869a5700 1 -- 172.21.15.144:0/2618582902 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8019d9e0 con 0x7f4a80101370 2024-09-10T11:29:13.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.262+0000 7f4a869a5700 1 -- 172.21.15.144:0/2618582902 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a80108cc0 con 0x7f4a80101370 2024-09-10T11:29:13.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.263+0000 7f4a7d7fa700 1 -- 172.21.15.144:0/2618582902 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) v1 ==== 91089+0+0 (secure 0 0 0) 0x7f4a7400cb30 con 0x7f4a80101370 2024-09-10T11:29:13.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.263+0000 7f4a7d7fa700 1 --2- 172.21.15.144:0/2618582902 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f4a6806d3c0 0x7f4a6806f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:13.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.264+0000 7f4a7d7fa700 1 -- 172.21.15.144:0/2618582902 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4a74014070 con 0x7f4a80101370 2024-09-10T11:29:13.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.264+0000 7f4a7ffff700 1 --2- 172.21.15.144:0/2618582902 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f4a6806d3c0 0x7f4a6806f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:13.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.264+0000 7f4a7ffff700 1 --2- 172.21.15.144:0/2618582902 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f4a6806d3c0 0x7f4a6806f870 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009fd0 tx=0x7f4a70005940 comp rx=0 tx=0).ready entity=mgr.24329 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:13.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.269+0000 7f4a7d7fa700 1 -- 172.21.15.144:0/2618582902 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f4a740539d0 con 0x7f4a80101370 2024-09-10T11:29:13.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.555+0000 7f4a869a5700 1 -- 172.21.15.144:0/2618582902 --> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a80105cc0 con 0x7f4a6806d3c0 2024-09-10T11:29:13.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.557+0000 7f4a7d7fa700 1 -- 172.21.15.144:0/2618582902 <== mgr.24329 v2:172.21.15.178:6828/2974090415 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f4a80105cc0 con 0x7f4a6806d3c0 2024-09-10T11:29:13.558 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:29:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:29:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:29:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T11:29:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [], 2024-09-10T11:29:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "1/2 daemons upgraded", 2024-09-10T11:29:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "", 2024-09-10T11:29:13.558 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:29:13.558 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:29:13.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.560+0000 7f4a869a5700 1 -- 172.21.15.144:0/2618582902 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f4a6806d3c0 msgr2=0x7f4a6806f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:13.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.560+0000 7f4a869a5700 1 --2- 172.21.15.144:0/2618582902 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f4a6806d3c0 0x7f4a6806f870 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f4a70009fd0 tx=0x7f4a70005940 comp rx=0 tx=0).stop 2024-09-10T11:29:13.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.560+0000 7f4a869a5700 1 -- 172.21.15.144:0/2618582902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a80101370 msgr2=0x7f4a80073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:13.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.560+0000 7f4a869a5700 1 --2- 172.21.15.144:0/2618582902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a80101370 0x7f4a80073140 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f4a7400e970 tx=0x7f4a7400ec80 comp rx=0 tx=0).stop 2024-09-10T11:29:13.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.560+0000 7f4a869a5700 1 -- 172.21.15.144:0/2618582902 shutdown_connections 2024-09-10T11:29:13.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.560+0000 7f4a869a5700 1 --2- 172.21.15.144:0/2618582902 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a80100170 0x7f4a80074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:13.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.560+0000 7f4a869a5700 1 --2- 172.21.15.144:0/2618582902 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f4a6806d3c0 0x7f4a6806f870 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:13.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.560+0000 7f4a869a5700 1 --2- 172.21.15.144:0/2618582902 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a80101370 0x7f4a80073140 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:13.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.560+0000 7f4a869a5700 1 -- 172.21.15.144:0/2618582902 >> 172.21.15.144:0/2618582902 conn(0x7f4a800fb720 msgr2=0x7f4a801045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:13.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.560+0000 7f4a869a5700 1 -- 172.21.15.144:0/2618582902 shutdown_connections 2024-09-10T11:29:13.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:13.560+0000 7f4a869a5700 1 -- 172.21.15.144:0/2618582902 wait complete. 2024-09-10T11:29:13.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:13 smithi144 ceph-mon[27942]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:13.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:13 smithi144 ceph-mon[27942]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:13.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:13 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2105550128' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:29:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:13 smithi178 ceph-mon[33514]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:13 smithi178 ceph-mon[33514]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:13 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/2105550128' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:29:14.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:14 smithi144 ceph-mon[27942]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:14.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:14 smithi144 ceph-mon[27942]: pgmap v14: 65 pgs: 65 active+clean; 195 MiB data, 831 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s rd, 6.3 MiB/s wr, 472 op/s 2024-09-10T11:29:14.970 INFO:tasks.workunit.client.1.smithi178.stderr:++ readlink -f fsstress 2024-09-10T11:29:14.971 INFO:tasks.workunit.client.1.smithi178.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-09-10T11:29:14.971 INFO:tasks.workunit.client.1.smithi178.stderr:+ popd 2024-09-10T11:29:14.972 INFO:tasks.workunit.client.1.smithi178.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-09-10T11:29:14.973 INFO:tasks.workunit.client.1.smithi178.stderr:+ popd 2024-09-10T11:29:14.973 INFO:tasks.workunit.client.1.smithi178.stdout:~/cephtest/mnt.1/client.1/tmp 2024-09-10T11:29:14.973 INFO:tasks.workunit.client.1.smithi178.stderr:++ mktemp -d -p . 2024-09-10T11:29:14.976 INFO:tasks.workunit.client.1.smithi178.stderr:+ T=./tmp.KdfX4V8Omh 2024-09-10T11:29:14.976 INFO:tasks.workunit.client.1.smithi178.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.KdfX4V8Omh -l 1 -n 1000 -p 10 -v 2024-09-10T11:29:14.979 INFO:tasks.workunit.client.1.smithi178.stdout:seed = 1726930052 2024-09-10T11:29:14.984 INFO:tasks.workunit.client.1.smithi178.stdout:0/0: chown . 5 1 2024-09-10T11:29:14.984 INFO:tasks.workunit.client.1.smithi178.stdout:0/1: fsync - no filename 2024-09-10T11:29:14.984 INFO:tasks.workunit.client.1.smithi178.stdout:0/2: rename - no filename 2024-09-10T11:29:14.984 INFO:tasks.workunit.client.1.smithi178.stdout:0/3: chown . 1 1 2024-09-10T11:29:14.984 INFO:tasks.workunit.client.1.smithi178.stdout:0/4: chown . 144149 1 2024-09-10T11:29:14.984 INFO:tasks.workunit.client.1.smithi178.stdout:0/5: truncate - no filename 2024-09-10T11:29:14.984 INFO:tasks.workunit.client.1.smithi178.stdout:0/6: chown . 373 1 2024-09-10T11:29:14.984 INFO:tasks.workunit.client.1.smithi178.stdout:0/7: truncate - no filename 2024-09-10T11:29:14.985 INFO:tasks.workunit.client.1.smithi178.stdout:0/8: dwrite - no filename 2024-09-10T11:29:14.985 INFO:tasks.workunit.client.1.smithi178.stdout:0/9: fdatasync - no filename 2024-09-10T11:29:14.986 INFO:tasks.workunit.client.1.smithi178.stdout:0/10: mkdir d0 0 2024-09-10T11:29:14.986 INFO:tasks.workunit.client.1.smithi178.stdout:0/11: write - no filename 2024-09-10T11:29:14.986 INFO:tasks.workunit.client.1.smithi178.stdout:0/12: readlink - no filename 2024-09-10T11:29:14.987 INFO:tasks.workunit.client.1.smithi178.stdout:0/13: truncate - no filename 2024-09-10T11:29:14.987 INFO:tasks.workunit.client.1.smithi178.stdout:0/14: truncate - no filename 2024-09-10T11:29:14.987 INFO:tasks.workunit.client.1.smithi178.stdout:2/0: read - no filename 2024-09-10T11:29:14.988 INFO:tasks.workunit.client.1.smithi178.stdout:3/0: write - no filename 2024-09-10T11:29:14.989 INFO:tasks.workunit.client.1.smithi178.stdout:0/15: creat d0/f1 x:0 0 0 2024-09-10T11:29:14.990 INFO:tasks.workunit.client.1.smithi178.stdout:2/1: symlink l0 0 2024-09-10T11:29:14.992 INFO:tasks.workunit.client.1.smithi178.stdout:5/0: write - no filename 2024-09-10T11:29:14.993 INFO:tasks.workunit.client.1.smithi178.stdout:5/1: dread - no filename 2024-09-10T11:29:14.993 INFO:tasks.workunit.client.1.smithi178.stdout:5/2: rmdir - no directory 2024-09-10T11:29:14.993 INFO:tasks.workunit.client.1.smithi178.stdout:3/1: symlink l0 0 2024-09-10T11:29:14.993 INFO:tasks.workunit.client.1.smithi178.stdout:2/2: mkdir d1 0 2024-09-10T11:29:14.993 INFO:tasks.workunit.client.1.smithi178.stdout:2/3: dread - no filename 2024-09-10T11:29:14.994 INFO:tasks.workunit.client.1.smithi178.stdout:6/0: dread - no filename 2024-09-10T11:29:14.995 INFO:tasks.workunit.client.1.smithi178.stdout:3/2: creat f1 x:0 0 0 2024-09-10T11:29:14.996 INFO:tasks.workunit.client.1.smithi178.stdout:3/3: write f1 [302901,23862] 0 2024-09-10T11:29:14.997 INFO:tasks.workunit.client.1.smithi178.stdout:5/3: creat f0 x:0 0 0 2024-09-10T11:29:14.998 INFO:tasks.workunit.client.1.smithi178.stdout:3/4: write f1 [1339575,75000] 0 2024-09-10T11:29:14.998 INFO:tasks.workunit.client.1.smithi178.stdout:2/4: creat d1/f2 x:0 0 0 2024-09-10T11:29:14.998 INFO:tasks.workunit.client.1.smithi178.stdout:7/0: dwrite - no filename 2024-09-10T11:29:14.999 INFO:tasks.workunit.client.1.smithi178.stdout:2/5: readlink l0 0 2024-09-10T11:29:14.999 INFO:tasks.workunit.client.1.smithi178.stdout:2/6: read - d1/f2 zero size 2024-09-10T11:29:15.000 INFO:tasks.workunit.client.1.smithi178.stdout:5/4: rename f0 to f1 0 2024-09-10T11:29:15.001 INFO:tasks.workunit.client.1.smithi178.stdout:6/1: creat f0 x:0 0 0 2024-09-10T11:29:15.001 INFO:tasks.workunit.client.1.smithi178.stdout:2/7: dread - d1/f2 zero size 2024-09-10T11:29:15.001 INFO:tasks.workunit.client.1.smithi178.stdout:3/5: symlink l2 0 2024-09-10T11:29:15.002 INFO:tasks.workunit.client.1.smithi178.stdout:6/2: dread - f0 zero size 2024-09-10T11:29:15.003 INFO:tasks.workunit.client.1.smithi178.stdout:8/0: dwrite - no filename 2024-09-10T11:29:15.003 INFO:tasks.workunit.client.1.smithi178.stdout:8/1: dwrite - no filename 2024-09-10T11:29:15.003 INFO:tasks.workunit.client.1.smithi178.stdout:2/8: write d1/f2 [1021535,540] 0 2024-09-10T11:29:15.003 INFO:tasks.workunit.client.1.smithi178.stdout:8/2: chown . 4914 1 2024-09-10T11:29:15.003 INFO:tasks.workunit.client.1.smithi178.stdout:8/3: rmdir - no directory 2024-09-10T11:29:15.003 INFO:tasks.workunit.client.1.smithi178.stdout:8/4: write - no filename 2024-09-10T11:29:15.003 INFO:tasks.workunit.client.1.smithi178.stdout:8/5: dread - no filename 2024-09-10T11:29:15.003 INFO:tasks.workunit.client.1.smithi178.stdout:8/6: truncate - no filename 2024-09-10T11:29:15.003 INFO:tasks.workunit.client.1.smithi178.stdout:8/7: dread - no filename 2024-09-10T11:29:15.004 INFO:tasks.workunit.client.1.smithi178.stdout:8/8: chown . 0 1 2024-09-10T11:29:15.004 INFO:tasks.workunit.client.1.smithi178.stdout:8/9: rename - no filename 2024-09-10T11:29:15.004 INFO:tasks.workunit.client.1.smithi178.stdout:6/3: chown f0 17 1 2024-09-10T11:29:15.004 INFO:tasks.workunit.client.1.smithi178.stdout:3/6: chown l2 3525524 1 2024-09-10T11:29:15.005 INFO:tasks.workunit.client.1.smithi178.stdout:7/1: creat f0 x:0 0 0 2024-09-10T11:29:15.006 INFO:tasks.workunit.client.1.smithi178.stdout:9/0: rename - no filename 2024-09-10T11:29:15.006 INFO:tasks.workunit.client.1.smithi178.stdout:6/4: creat f1 x:0 0 0 2024-09-10T11:29:15.006 INFO:tasks.workunit.client.1.smithi178.stdout:9/1: fsync - no filename 2024-09-10T11:29:15.007 INFO:tasks.workunit.client.1.smithi178.stdout:8/10: creat f0 x:0 0 0 2024-09-10T11:29:15.007 INFO:tasks.workunit.client.1.smithi178.stdout:7/2: creat f1 x:0 0 0 2024-09-10T11:29:15.008 INFO:tasks.workunit.client.1.smithi178.stdout:7/3: dread - f0 zero size 2024-09-10T11:29:15.010 INFO:tasks.workunit.client.1.smithi178.stdout:8/11: symlink l1 0 2024-09-10T11:29:15.011 INFO:tasks.workunit.client.1.smithi178.stdout:9/2: symlink l0 0 2024-09-10T11:29:15.011 INFO:tasks.workunit.client.1.smithi178.stdout:9/3: truncate - no filename 2024-09-10T11:29:15.012 INFO:tasks.workunit.client.1.smithi178.stdout:8/12: dread - f0 zero size 2024-09-10T11:29:15.014 INFO:tasks.workunit.client.1.smithi178.stdout:9/4: chown l0 236 1 2024-09-10T11:29:15.014 INFO:tasks.workunit.client.1.smithi178.stdout:9/5: write - no filename 2024-09-10T11:29:15.017 INFO:tasks.workunit.client.1.smithi178.stdout:9/6: chown l0 22 1 2024-09-10T11:29:15.019 INFO:tasks.workunit.client.1.smithi178.stdout:8/13: write f0 [667921,125960] 0 2024-09-10T11:29:15.024 INFO:tasks.workunit.client.1.smithi178.stdout:3/7: dwrite f1 [0,4194304] 0 2024-09-10T11:29:15.025 INFO:tasks.workunit.client.1.smithi178.stdout:8/14: write f0 [1283877,73654] 0 2024-09-10T11:29:15.028 INFO:tasks.workunit.client.1.smithi178.stdout:9/7: unlink l0 0 2024-09-10T11:29:15.029 INFO:tasks.workunit.client.1.smithi178.stdout:9/8: dread - no filename 2024-09-10T11:29:15.029 INFO:tasks.workunit.client.1.smithi178.stdout:9/9: write - no filename 2024-09-10T11:29:15.029 INFO:tasks.workunit.client.1.smithi178.stdout:9/10: rename - no filename 2024-09-10T11:29:15.029 INFO:tasks.workunit.client.1.smithi178.stdout:9/11: dwrite - no filename 2024-09-10T11:29:15.029 INFO:tasks.workunit.client.1.smithi178.stdout:9/12: rename - no filename 2024-09-10T11:29:15.029 INFO:tasks.workunit.client.1.smithi178.stdout:9/13: dwrite - no filename 2024-09-10T11:29:15.029 INFO:tasks.workunit.client.1.smithi178.stdout:9/14: write - no filename 2024-09-10T11:29:15.030 INFO:tasks.workunit.client.1.smithi178.stdout:6/5: dwrite f0 [0,4194304] 0 2024-09-10T11:29:15.030 INFO:tasks.workunit.client.1.smithi178.stdout:9/15: chown . 15927218 1 2024-09-10T11:29:15.031 INFO:tasks.workunit.client.1.smithi178.stdout:3/8: chown l2 940457 1 2024-09-10T11:29:15.031 INFO:tasks.workunit.client.1.smithi178.stdout:6/6: rmdir - no directory 2024-09-10T11:29:15.031 INFO:tasks.workunit.client.1.smithi178.stdout:9/16: chown . 92451 1 2024-09-10T11:29:15.031 INFO:tasks.workunit.client.1.smithi178.stdout:9/17: rename - no filename 2024-09-10T11:29:15.032 INFO:tasks.workunit.client.1.smithi178.stdout:9/18: chown . 128334514 1 2024-09-10T11:29:15.032 INFO:tasks.workunit.client.1.smithi178.stdout:7/4: dwrite f0 [0,4194304] 0 2024-09-10T11:29:15.033 INFO:tasks.workunit.client.1.smithi178.stdout:8/15: rename f0 to f2 0 2024-09-10T11:29:15.033 INFO:tasks.workunit.client.1.smithi178.stdout:3/9: chown f1 290656 1 2024-09-10T11:29:15.033 INFO:tasks.workunit.client.1.smithi178.stdout:6/7: truncate f0 5234382 0 2024-09-10T11:29:15.034 INFO:tasks.workunit.client.1.smithi178.stdout:9/19: creat f1 x:0 0 0 2024-09-10T11:29:15.034 INFO:tasks.workunit.client.1.smithi178.stdout:8/16: readlink l1 0 2024-09-10T11:29:15.036 INFO:tasks.workunit.client.1.smithi178.stdout:3/10: symlink l3 0 2024-09-10T11:29:15.036 INFO:tasks.workunit.client.1.smithi178.stdout:3/11: chown f1 720 1 2024-09-10T11:29:15.037 INFO:tasks.workunit.client.1.smithi178.stdout:3/12: rmdir - no directory 2024-09-10T11:29:15.038 INFO:tasks.workunit.client.1.smithi178.stdout:9/20: unlink f1 0 2024-09-10T11:29:15.038 INFO:tasks.workunit.client.1.smithi178.stdout:9/21: link - no file 2024-09-10T11:29:15.038 INFO:tasks.workunit.client.1.smithi178.stdout:9/22: rename - no filename 2024-09-10T11:29:15.038 INFO:tasks.workunit.client.1.smithi178.stdout:9/23: unlink - no file 2024-09-10T11:29:15.038 INFO:tasks.workunit.client.1.smithi178.stdout:9/24: link - no file 2024-09-10T11:29:15.038 INFO:tasks.workunit.client.1.smithi178.stdout:9/25: dread - no filename 2024-09-10T11:29:15.043 INFO:tasks.workunit.client.1.smithi178.stdout:8/17: dread f2 [0,4194304] 0 2024-09-10T11:29:15.044 INFO:tasks.workunit.client.1.smithi178.stdout:3/13: creat f4 x:0 0 0 2024-09-10T11:29:15.045 INFO:tasks.workunit.client.1.smithi178.stdout:9/26: mkdir d2 0 2024-09-10T11:29:15.048 INFO:tasks.workunit.client.1.smithi178.stdout:3/14: dread f1 [0,4194304] 0 2024-09-10T11:29:15.049 INFO:tasks.workunit.client.1.smithi178.stdout:3/15: readlink l0 0 2024-09-10T11:29:15.051 INFO:tasks.workunit.client.1.smithi178.stdout:9/27: creat d2/f3 x:0 0 0 2024-09-10T11:29:15.052 INFO:tasks.workunit.client.1.smithi178.stdout:3/16: rename f4 to f5 0 2024-09-10T11:29:15.055 INFO:tasks.workunit.client.1.smithi178.stdout:9/28: link d2/f3 d2/f4 0 2024-09-10T11:29:15.057 INFO:tasks.workunit.client.1.smithi178.stdout:9/29: mkdir d2/d5 0 2024-09-10T11:29:15.059 INFO:tasks.workunit.client.1.smithi178.stdout:9/30: mknod d2/c6 0 2024-09-10T11:29:15.060 INFO:tasks.workunit.client.1.smithi178.stdout:3/17: dwrite f1 [0,4194304] 0 2024-09-10T11:29:15.065 INFO:tasks.workunit.client.1.smithi178.stdout:9/31: rmdir d2/d5 0 2024-09-10T11:29:15.066 INFO:tasks.workunit.client.1.smithi178.stdout:3/18: dread f1 [0,4194304] 0 2024-09-10T11:29:15.067 INFO:tasks.workunit.client.1.smithi178.stdout:9/32: truncate d2/f3 122310 0 2024-09-10T11:29:15.067 INFO:tasks.workunit.client.1.smithi178.stdout:3/19: chown l2 3 1 2024-09-10T11:29:15.068 INFO:tasks.workunit.client.1.smithi178.stdout:3/20: fdatasync f5 0 2024-09-10T11:29:15.069 INFO:tasks.workunit.client.1.smithi178.stdout:9/33: write d2/f4 [332744,12156] 0 2024-09-10T11:29:15.071 INFO:tasks.workunit.client.1.smithi178.stdout:9/34: creat d2/f7 x:0 0 0 2024-09-10T11:29:15.073 INFO:tasks.workunit.client.1.smithi178.stdout:9/35: mkdir d2/d8 0 2024-09-10T11:29:15.080 INFO:tasks.workunit.client.1.smithi178.stdout:9/36: dwrite d2/f3 [0,4194304] 0 2024-09-10T11:29:15.086 INFO:tasks.workunit.client.1.smithi178.stdout:9/37: dread d2/f4 [0,4194304] 0 2024-09-10T11:29:15.088 INFO:tasks.workunit.client.1.smithi178.stdout:9/38: creat d2/d8/f9 x:0 0 0 2024-09-10T11:29:15.111 INFO:tasks.workunit.client.1.smithi178.stdout:9/39: creat d2/d8/fa x:0 0 0 2024-09-10T11:29:15.111 INFO:tasks.workunit.client.1.smithi178.stdout:9/40: readlink - no filename 2024-09-10T11:29:15.112 INFO:tasks.workunit.client.1.smithi178.stdout:9/41: readlink - no filename 2024-09-10T11:29:15.114 INFO:tasks.workunit.client.1.smithi178.stdout:9/42: link d2/d8/f9 d2/d8/fb 0 2024-09-10T11:29:15.115 INFO:tasks.workunit.client.1.smithi178.stdout:9/43: mkdir d2/dc 0 2024-09-10T11:29:15.115 INFO:tasks.workunit.client.1.smithi178.stdout:9/44: chown d2/c6 110841588 1 2024-09-10T11:29:15.116 INFO:tasks.workunit.client.1.smithi178.stdout:9/45: dread - d2/d8/fa zero size 2024-09-10T11:29:15.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:14 smithi178 ceph-mon[33514]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:15.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:14 smithi178 ceph-mon[33514]: pgmap v14: 65 pgs: 65 active+clean; 195 MiB data, 831 MiB used, 536 GiB / 536 GiB avail; 3.2 MiB/s rd, 6.3 MiB/s wr, 472 op/s 2024-09-10T11:29:15.231 INFO:tasks.workunit.client.1.smithi178.stdout:9/46: write d2/d8/f9 [542102,65640] 0 2024-09-10T11:29:15.233 INFO:tasks.workunit.client.1.smithi178.stdout:9/47: truncate d2/d8/fa 802132 0 2024-09-10T11:29:15.249 INFO:tasks.workunit.client.1.smithi178.stdout:9/48: link d2/d8/f9 d2/fd 0 2024-09-10T11:29:15.257 INFO:tasks.workunit.client.1.smithi178.stdout:9/49: dwrite d2/d8/fa [0,4194304] 0 2024-09-10T11:29:15.266 INFO:tasks.workunit.client.1.smithi178.stdout:9/50: dwrite d2/f3 [0,4194304] 0 2024-09-10T11:29:15.267 INFO:tasks.workunit.client.1.smithi178.stdout:9/51: write d2/f3 [1570771,20081] 0 2024-09-10T11:29:15.270 INFO:tasks.workunit.client.1.smithi178.stdout:9/52: symlink d2/le 0 2024-09-10T11:29:15.273 INFO:tasks.workunit.client.1.smithi178.stdout:9/53: write d2/d8/fb [37201,27443] 0 2024-09-10T11:29:15.275 INFO:tasks.workunit.client.1.smithi178.stdout:9/54: mkdir d2/df 0 2024-09-10T11:29:15.314 INFO:tasks.workunit.client.1.smithi178.stdout:0/16: fsync d0/f1 0 2024-09-10T11:29:15.315 INFO:tasks.workunit.client.1.smithi178.stdout:0/17: write d0/f1 [1037294,127788] 0 2024-09-10T11:29:15.319 INFO:tasks.workunit.client.1.smithi178.stdout:5/5: rename f1 to f2 0 2024-09-10T11:29:15.322 INFO:tasks.workunit.client.1.smithi178.stdout:5/6: symlink l3 0 2024-09-10T11:29:15.323 INFO:tasks.workunit.client.1.smithi178.stdout:0/18: dwrite d0/f1 [0,4194304] 0 2024-09-10T11:29:15.324 INFO:tasks.workunit.client.1.smithi178.stdout:7/5: getdents . 0 2024-09-10T11:29:15.324 INFO:tasks.workunit.client.1.smithi178.stdout:2/9: truncate d1/f2 443055 0 2024-09-10T11:29:15.325 INFO:tasks.workunit.client.1.smithi178.stdout:0/19: chown d0/f1 7893802 1 2024-09-10T11:29:15.325 INFO:tasks.workunit.client.1.smithi178.stdout:7/6: chown f0 44 1 2024-09-10T11:29:15.327 INFO:tasks.workunit.client.1.smithi178.stdout:6/8: write f0 [6231294,42456] 0 2024-09-10T11:29:15.327 INFO:tasks.workunit.client.1.smithi178.stdout:7/7: fsync f1 0 2024-09-10T11:29:15.327 INFO:tasks.workunit.client.1.smithi178.stdout:0/20: read d0/f1 [1144752,71580] 0 2024-09-10T11:29:15.335 INFO:tasks.workunit.client.1.smithi178.stdout:7/8: rename f1 to f2 0 2024-09-10T11:29:15.341 INFO:tasks.workunit.client.1.smithi178.stdout:5/7: dwrite f2 [0,4194304] 0 2024-09-10T11:29:15.341 INFO:tasks.workunit.client.1.smithi178.stdout:5/8: rmdir - no directory 2024-09-10T11:29:15.343 INFO:tasks.workunit.client.1.smithi178.stdout:5/9: chown f2 286870 1 2024-09-10T11:29:15.344 INFO:tasks.workunit.client.1.smithi178.stdout:7/9: creat f3 x:0 0 0 2024-09-10T11:29:15.347 INFO:tasks.workunit.client.1.smithi178.stdout:8/18: truncate f2 380877 0 2024-09-10T11:29:15.348 INFO:tasks.workunit.client.1.smithi178.stdout:3/21: fsync f5 0 2024-09-10T11:29:15.348 INFO:tasks.workunit.client.1.smithi178.stdout:7/10: truncate f0 4408529 0 2024-09-10T11:29:15.349 INFO:tasks.workunit.client.1.smithi178.stdout:6/9: dwrite f1 [0,4194304] 0 2024-09-10T11:29:15.352 INFO:tasks.workunit.client.1.smithi178.stdout:7/11: read f0 [4066208,10737] 0 2024-09-10T11:29:15.352 INFO:tasks.workunit.client.1.smithi178.stdout:3/22: dread - f5 zero size 2024-09-10T11:29:15.353 INFO:tasks.workunit.client.1.smithi178.stdout:7/12: chown f0 28153 1 2024-09-10T11:29:15.353 INFO:tasks.workunit.client.1.smithi178.stdout:7/13: readlink - no filename 2024-09-10T11:29:15.354 INFO:tasks.workunit.client.1.smithi178.stdout:7/14: readlink - no filename 2024-09-10T11:29:15.355 INFO:tasks.workunit.client.1.smithi178.stdout:0/21: dwrite d0/f1 [0,4194304] 0 2024-09-10T11:29:15.359 INFO:tasks.workunit.client.1.smithi178.stdout:5/10: link f2 f4 0 2024-09-10T11:29:15.363 INFO:tasks.workunit.client.1.smithi178.stdout:6/10: dread f0 [0,4194304] 0 2024-09-10T11:29:15.363 INFO:tasks.workunit.client.1.smithi178.stdout:3/23: dread f1 [0,4194304] 0 2024-09-10T11:29:15.364 INFO:tasks.workunit.client.1.smithi178.stdout:3/24: readlink l0 0 2024-09-10T11:29:15.368 INFO:tasks.workunit.client.1.smithi178.stdout:5/11: creat f5 x:0 0 0 2024-09-10T11:29:15.374 INFO:tasks.workunit.client.1.smithi178.stdout:6/11: mknod c2 0 2024-09-10T11:29:15.375 INFO:tasks.workunit.client.1.smithi178.stdout:3/25: symlink l6 0 2024-09-10T11:29:15.375 INFO:tasks.workunit.client.1.smithi178.stdout:3/26: chown l6 86199 1 2024-09-10T11:29:15.376 INFO:tasks.workunit.client.1.smithi178.stdout:5/12: mknod c6 0 2024-09-10T11:29:15.378 INFO:tasks.workunit.client.1.smithi178.stdout:6/12: mkdir d3 0 2024-09-10T11:29:15.379 INFO:tasks.workunit.client.1.smithi178.stdout:3/27: creat f7 x:0 0 0 2024-09-10T11:29:15.380 INFO:tasks.workunit.client.1.smithi178.stdout:5/13: symlink l7 0 2024-09-10T11:29:15.382 INFO:tasks.workunit.client.1.smithi178.stdout:6/13: mkdir d3/d4 0 2024-09-10T11:29:15.384 INFO:tasks.workunit.client.1.smithi178.stdout:5/14: unlink l7 0 2024-09-10T11:29:15.385 INFO:tasks.workunit.client.1.smithi178.stdout:6/14: creat d3/f5 x:0 0 0 2024-09-10T11:29:15.388 INFO:tasks.workunit.client.1.smithi178.stdout:3/28: dwrite f7 [0,4194304] 0 2024-09-10T11:29:15.389 INFO:tasks.workunit.client.1.smithi178.stdout:3/29: dread - f5 zero size 2024-09-10T11:29:15.389 INFO:tasks.workunit.client.1.smithi178.stdout:3/30: chown f1 5122 1 2024-09-10T11:29:15.392 INFO:tasks.workunit.client.1.smithi178.stdout:3/31: creat f8 x:0 0 0 2024-09-10T11:29:15.395 INFO:tasks.workunit.client.1.smithi178.stdout:3/32: symlink l9 0 2024-09-10T11:29:15.398 INFO:tasks.workunit.client.1.smithi178.stdout:3/33: write f1 [2601068,12772] 0 2024-09-10T11:29:15.398 INFO:tasks.workunit.client.1.smithi178.stdout:9/55: truncate d2/d8/fa 1930260 0 2024-09-10T11:29:15.399 INFO:tasks.workunit.client.1.smithi178.stdout:6/15: dwrite f0 [4194304,4194304] 0 2024-09-10T11:29:15.400 INFO:tasks.workunit.client.1.smithi178.stdout:3/34: read f1 [2787433,52583] 0 2024-09-10T11:29:15.400 INFO:tasks.workunit.client.1.smithi178.stdout:9/56: readlink d2/le 0 2024-09-10T11:29:15.403 INFO:tasks.workunit.client.1.smithi178.stdout:6/16: unlink c2 0 2024-09-10T11:29:15.405 INFO:tasks.workunit.client.1.smithi178.stdout:3/35: symlink la 0 2024-09-10T11:29:15.411 INFO:tasks.workunit.client.1.smithi178.stdout:8/19: dread f2 [0,4194304] 0 2024-09-10T11:29:15.413 INFO:tasks.workunit.client.1.smithi178.stdout:3/36: truncate f5 932252 0 2024-09-10T11:29:15.420 INFO:tasks.workunit.client.1.smithi178.stdout:9/57: dwrite d2/f7 [0,4194304] 0 2024-09-10T11:29:15.420 INFO:tasks.workunit.client.1.smithi178.stdout:6/17: dwrite f0 [4194304,4194304] 0 2024-09-10T11:29:15.420 INFO:tasks.workunit.client.1.smithi178.stdout:7/15: truncate f0 2271653 0 2024-09-10T11:29:15.420 INFO:tasks.workunit.client.1.smithi178.stdout:0/22: truncate d0/f1 3983735 0 2024-09-10T11:29:15.421 INFO:tasks.workunit.client.1.smithi178.stdout:0/23: readlink - no filename 2024-09-10T11:29:15.421 INFO:tasks.workunit.client.1.smithi178.stdout:9/58: readlink d2/le 0 2024-09-10T11:29:15.422 INFO:tasks.workunit.client.1.smithi178.stdout:9/59: chown d2/d8/f9 502 1 2024-09-10T11:29:15.434 INFO:tasks.workunit.client.1.smithi178.stdout:8/20: write f2 [140191,81484] 0 2024-09-10T11:29:15.437 INFO:tasks.workunit.client.1.smithi178.stdout:8/21: mknod c3 0 2024-09-10T11:29:15.439 INFO:tasks.workunit.client.1.smithi178.stdout:3/37: truncate f1 4006900 0 2024-09-10T11:29:15.439 INFO:tasks.workunit.client.1.smithi178.stdout:8/22: dread f2 [0,4194304] 0 2024-09-10T11:29:15.441 INFO:tasks.workunit.client.1.smithi178.stdout:6/18: fsync f0 0 2024-09-10T11:29:15.441 INFO:tasks.workunit.client.1.smithi178.stdout:3/38: mkdir db 0 2024-09-10T11:29:15.442 INFO:tasks.workunit.client.1.smithi178.stdout:2/10: truncate d1/f2 159881 0 2024-09-10T11:29:15.443 INFO:tasks.workunit.client.1.smithi178.stdout:6/19: truncate d3/f5 873551 0 2024-09-10T11:29:15.444 INFO:tasks.workunit.client.1.smithi178.stdout:8/23: rename c3 to c4 0 2024-09-10T11:29:15.447 INFO:tasks.workunit.client.1.smithi178.stdout:9/60: truncate d2/f7 2854587 0 2024-09-10T11:29:15.447 INFO:tasks.workunit.client.1.smithi178.stdout:9/61: fsync d2/fd 0 2024-09-10T11:29:15.448 INFO:tasks.workunit.client.1.smithi178.stdout:7/16: write f0 [2045381,3707] 0 2024-09-10T11:29:15.449 INFO:tasks.workunit.client.1.smithi178.stdout:3/39: symlink db/lc 0 2024-09-10T11:29:15.449 INFO:tasks.workunit.client.1.smithi178.stdout:8/24: write f2 [310596,130608] 0 2024-09-10T11:29:15.453 INFO:tasks.workunit.client.1.smithi178.stdout:6/20: link d3/f5 d3/d4/f6 0 2024-09-10T11:29:15.454 INFO:tasks.workunit.client.1.smithi178.stdout:7/17: fsync f2 0 2024-09-10T11:29:15.456 INFO:tasks.workunit.client.1.smithi178.stdout:2/11: rename d1/f2 to d1/f3 0 2024-09-10T11:29:15.459 INFO:tasks.workunit.client.1.smithi178.stdout:3/40: rename f5 to db/fd 0 2024-09-10T11:29:15.459 INFO:tasks.workunit.client.1.smithi178.stdout:3/41: chown l9 453 1 2024-09-10T11:29:15.460 INFO:tasks.workunit.client.1.smithi178.stdout:6/21: mkdir d3/d7 0 2024-09-10T11:29:15.461 INFO:tasks.workunit.client.1.smithi178.stdout:6/22: chown d3/d7 60661 1 2024-09-10T11:29:15.461 INFO:tasks.workunit.client.1.smithi178.stdout:7/18: creat f4 x:0 0 0 2024-09-10T11:29:15.461 INFO:tasks.workunit.client.1.smithi178.stdout:0/24: truncate d0/f1 3091720 0 2024-09-10T11:29:15.462 INFO:tasks.workunit.client.1.smithi178.stdout:9/62: getdents d2/df 0 2024-09-10T11:29:15.466 INFO:tasks.workunit.client.1.smithi178.stdout:8/25: unlink c4 0 2024-09-10T11:29:15.466 INFO:tasks.workunit.client.1.smithi178.stdout:8/26: readlink l1 0 2024-09-10T11:29:15.467 INFO:tasks.workunit.client.1.smithi178.stdout:3/42: creat db/fe x:0 0 0 2024-09-10T11:29:15.468 INFO:tasks.workunit.client.1.smithi178.stdout:7/19: symlink l5 0 2024-09-10T11:29:15.471 INFO:tasks.workunit.client.1.smithi178.stdout:3/43: dread db/fd [0,4194304] 0 2024-09-10T11:29:15.472 INFO:tasks.workunit.client.1.smithi178.stdout:6/23: rename d3/d4 to d3/d7/d8 0 2024-09-10T11:29:15.474 INFO:tasks.workunit.client.1.smithi178.stdout:8/27: dwrite f2 [0,4194304] 0 2024-09-10T11:29:15.476 INFO:tasks.workunit.client.1.smithi178.stdout:0/25: chown d0/f1 7539875 1 2024-09-10T11:29:15.480 INFO:tasks.workunit.client.1.smithi178.stdout:3/44: rename db/fd to db/ff 0 2024-09-10T11:29:15.482 INFO:tasks.workunit.client.1.smithi178.stdout:9/63: creat d2/dc/f10 x:0 0 0 2024-09-10T11:29:15.483 INFO:tasks.workunit.client.1.smithi178.stdout:9/64: readlink d2/le 0 2024-09-10T11:29:15.484 INFO:tasks.workunit.client.1.smithi178.stdout:2/12: fsync d1/f3 0 2024-09-10T11:29:15.486 INFO:tasks.workunit.client.1.smithi178.stdout:6/24: symlink d3/d7/l9 0 2024-09-10T11:29:15.490 INFO:tasks.workunit.client.1.smithi178.stdout:3/45: creat db/f10 x:0 0 0 2024-09-10T11:29:15.492 INFO:tasks.workunit.client.1.smithi178.stdout:2/13: symlink d1/l4 0 2024-09-10T11:29:15.493 INFO:tasks.workunit.client.1.smithi178.stdout:9/65: dread d2/d8/fa [0,4194304] 0 2024-09-10T11:29:15.497 INFO:tasks.workunit.client.1.smithi178.stdout:8/28: write f2 [4771433,110050] 0 2024-09-10T11:29:15.498 INFO:tasks.workunit.client.1.smithi178.stdout:8/29: read f2 [187726,62192] 0 2024-09-10T11:29:15.499 INFO:tasks.workunit.client.1.smithi178.stdout:9/66: dread d2/d8/f9 [0,4194304] 0 2024-09-10T11:29:15.500 INFO:tasks.workunit.client.1.smithi178.stdout:9/67: stat d2/f4 0 2024-09-10T11:29:15.500 INFO:tasks.workunit.client.1.smithi178.stdout:8/30: write f2 [643823,73819] 0 2024-09-10T11:29:15.500 INFO:tasks.workunit.client.1.smithi178.stdout:3/46: truncate f1 1670109 0 2024-09-10T11:29:15.505 INFO:tasks.workunit.client.1.smithi178.stdout:3/47: symlink db/l11 0 2024-09-10T11:29:15.505 INFO:tasks.workunit.client.1.smithi178.stdout:2/14: write d1/f3 [898692,17719] 0 2024-09-10T11:29:15.507 INFO:tasks.workunit.client.1.smithi178.stdout:9/68: dread d2/d8/fa [0,4194304] 0 2024-09-10T11:29:15.509 INFO:tasks.workunit.client.1.smithi178.stdout:0/26: truncate d0/f1 722550 0 2024-09-10T11:29:15.510 INFO:tasks.workunit.client.1.smithi178.stdout:2/15: rmdir d1 39 2024-09-10T11:29:15.512 INFO:tasks.workunit.client.1.smithi178.stdout:6/25: dwrite d3/d7/d8/f6 [0,4194304] 0 2024-09-10T11:29:15.517 INFO:tasks.workunit.client.1.smithi178.stdout:8/31: dread f2 [0,4194304] 0 2024-09-10T11:29:15.518 INFO:tasks.workunit.client.1.smithi178.stdout:0/27: creat d0/f2 x:0 0 0 2024-09-10T11:29:15.518 INFO:tasks.workunit.client.1.smithi178.stdout:0/28: chown d0 118 1 2024-09-10T11:29:15.518 INFO:tasks.workunit.client.1.smithi178.stdout:3/48: dread f7 [0,4194304] 0 2024-09-10T11:29:15.521 INFO:tasks.workunit.client.1.smithi178.stdout:8/32: fdatasync f2 0 2024-09-10T11:29:15.521 INFO:tasks.workunit.client.1.smithi178.stdout:2/16: stat d1/f3 0 2024-09-10T11:29:15.546 INFO:tasks.workunit.client.1.smithi178.stdout:0/29: mknod d0/c3 0 2024-09-10T11:29:15.546 INFO:tasks.workunit.client.1.smithi178.stdout:2/17: stat d1/l4 0 2024-09-10T11:29:15.548 INFO:tasks.workunit.client.1.smithi178.stdout:2/18: readlink l0 0 2024-09-10T11:29:15.549 INFO:tasks.workunit.client.1.smithi178.stdout:9/69: dread d2/f4 [0,4194304] 0 2024-09-10T11:29:15.549 INFO:tasks.workunit.client.1.smithi178.stdout:6/26: dread f1 [0,4194304] 0 2024-09-10T11:29:15.550 INFO:tasks.workunit.client.1.smithi178.stdout:2/19: truncate d1/f3 1458764 0 2024-09-10T11:29:15.556 INFO:tasks.workunit.client.1.smithi178.stdout:3/49: dwrite db/f10 [0,4194304] 0 2024-09-10T11:29:15.566 INFO:tasks.workunit.client.1.smithi178.stdout:2/20: dread d1/f3 [0,4194304] 0 2024-09-10T11:29:15.566 INFO:tasks.workunit.client.1.smithi178.stdout:3/50: dwrite f8 [0,4194304] 0 2024-09-10T11:29:15.568 INFO:tasks.workunit.client.1.smithi178.stdout:3/51: truncate f7 4223545 0 2024-09-10T11:29:15.571 INFO:tasks.workunit.client.1.smithi178.stdout:2/21: dread d1/f3 [0,4194304] 0 2024-09-10T11:29:15.575 INFO:tasks.workunit.client.1.smithi178.stdout:3/52: dread f8 [0,4194304] 0 2024-09-10T11:29:15.672 INFO:tasks.workunit.client.1.smithi178.stdout:8/33: unlink l1 0 2024-09-10T11:29:15.672 INFO:tasks.workunit.client.1.smithi178.stdout:8/34: write f2 [251579,53529] 0 2024-09-10T11:29:15.674 INFO:tasks.workunit.client.1.smithi178.stdout:6/27: creat d3/fa x:0 0 0 2024-09-10T11:29:15.675 INFO:tasks.workunit.client.1.smithi178.stdout:6/28: stat d3/d7/d8/f6 0 2024-09-10T11:29:15.679 INFO:tasks.workunit.client.1.smithi178.stdout:2/22: creat d1/f5 x:0 0 0 2024-09-10T11:29:15.680 INFO:tasks.workunit.client.1.smithi178.stdout:8/35: mkdir d5 0 2024-09-10T11:29:15.682 INFO:tasks.workunit.client.1.smithi178.stdout:8/36: write f2 [3962429,68564] 0 2024-09-10T11:29:15.683 INFO:tasks.workunit.client.1.smithi178.stdout:8/37: chown d5 209259 1 2024-09-10T11:29:15.685 INFO:tasks.workunit.client.1.smithi178.stdout:6/29: dwrite d3/d7/d8/f6 [4194304,4194304] 0 2024-09-10T11:29:15.688 INFO:tasks.workunit.client.1.smithi178.stdout:3/53: link l9 db/l12 0 2024-09-10T11:29:15.689 INFO:tasks.workunit.client.1.smithi178.stdout:6/30: creat d3/fb x:0 0 0 2024-09-10T11:29:15.691 INFO:tasks.workunit.client.1.smithi178.stdout:8/38: getdents d5 0 2024-09-10T11:29:15.693 INFO:tasks.workunit.client.1.smithi178.stdout:6/31: chown d3/f5 2 1 2024-09-10T11:29:15.695 INFO:tasks.workunit.client.1.smithi178.stdout:6/32: read f1 [88424,78239] 0 2024-09-10T11:29:15.696 INFO:tasks.workunit.client.1.smithi178.stdout:3/54: truncate f1 239916 0 2024-09-10T11:29:15.697 INFO:tasks.workunit.client.1.smithi178.stdout:2/23: dwrite d1/f3 [0,4194304] 0 2024-09-10T11:29:15.699 INFO:tasks.workunit.client.1.smithi178.stdout:2/24: chown l0 73696 1 2024-09-10T11:29:15.704 INFO:tasks.workunit.client.1.smithi178.stdout:8/39: dwrite f2 [0,4194304] 0 2024-09-10T11:29:15.705 INFO:tasks.workunit.client.1.smithi178.stdout:2/25: symlink d1/l6 0 2024-09-10T11:29:15.711 INFO:tasks.workunit.client.1.smithi178.stdout:3/55: dwrite f7 [0,4194304] 0 2024-09-10T11:29:15.713 INFO:tasks.workunit.client.1.smithi178.stdout:8/40: link f2 d5/f6 0 2024-09-10T11:29:15.716 INFO:tasks.workunit.client.1.smithi178.stdout:3/56: symlink db/l13 0 2024-09-10T11:29:15.719 INFO:tasks.workunit.client.1.smithi178.stdout:3/57: mknod db/c14 0 2024-09-10T11:29:15.719 INFO:tasks.workunit.client.1.smithi178.stdout:3/58: dread - db/fe zero size 2024-09-10T11:29:15.720 INFO:tasks.workunit.client.1.smithi178.stdout:8/41: link f2 d5/f7 0 2024-09-10T11:29:15.721 INFO:tasks.workunit.client.1.smithi178.stdout:3/59: fsync db/ff 0 2024-09-10T11:29:15.721 INFO:tasks.workunit.client.1.smithi178.stdout:3/60: chown db/lc 103831446 1 2024-09-10T11:29:15.726 INFO:tasks.workunit.client.1.smithi178.stdout:3/61: link db/fe db/f15 0 2024-09-10T11:29:15.729 INFO:tasks.workunit.client.1.smithi178.stdout:8/42: dwrite d5/f6 [0,4194304] 0 2024-09-10T11:29:15.739 INFO:tasks.workunit.client.1.smithi178.stdout:8/43: dwrite d5/f6 [4194304,4194304] 0 2024-09-10T11:29:15.742 INFO:tasks.workunit.client.1.smithi178.stdout:8/44: creat d5/f8 x:0 0 0 2024-09-10T11:29:15.793 INFO:tasks.workunit.client.1.smithi178.stdout:9/70: truncate d2/f7 59951 0 2024-09-10T11:29:15.793 INFO:tasks.workunit.client.1.smithi178.stdout:9/71: stat d2/d8 0 2024-09-10T11:29:15.793 INFO:tasks.workunit.client.1.smithi178.stdout:9/72: write d2/dc/f10 [361745,25484] 0 2024-09-10T11:29:15.797 INFO:tasks.workunit.client.1.smithi178.stdout:9/73: unlink d2/d8/fb 0 2024-09-10T11:29:15.797 INFO:tasks.workunit.client.1.smithi178.stdout:9/74: chown d2/f3 1331 1 2024-09-10T11:29:15.797 INFO:tasks.workunit.client.1.smithi178.stdout:2/26: getdents d1 0 2024-09-10T11:29:15.798 INFO:tasks.workunit.client.1.smithi178.stdout:2/27: rmdir d1 39 2024-09-10T11:29:15.935 INFO:tasks.workunit.client.1.smithi178.stdout:2/28: truncate d1/f5 83179 0 2024-09-10T11:29:15.936 INFO:tasks.workunit.client.1.smithi178.stdout:8/45: truncate f2 2195997 0 2024-09-10T11:29:15.936 INFO:tasks.workunit.client.1.smithi178.stdout:2/29: dread d1/f5 [0,4194304] 0 2024-09-10T11:29:15.937 INFO:tasks.workunit.client.1.smithi178.stdout:2/30: fsync d1/f5 0 2024-09-10T11:29:15.939 INFO:tasks.workunit.client.1.smithi178.stdout:9/75: getdents d2/d8 0 2024-09-10T11:29:15.940 INFO:tasks.workunit.client.1.smithi178.stdout:8/46: symlink d5/l9 0 2024-09-10T11:29:15.941 INFO:tasks.workunit.client.1.smithi178.stdout:2/31: mkdir d1/d7 0 2024-09-10T11:29:15.942 INFO:tasks.workunit.client.1.smithi178.stdout:2/32: chown d1/f3 40158 1 2024-09-10T11:29:15.942 INFO:tasks.workunit.client.1.smithi178.stdout:2/33: write d1/f5 [874326,30091] 0 2024-09-10T11:29:15.944 INFO:tasks.workunit.client.1.smithi178.stdout:9/76: mknod d2/df/c11 0 2024-09-10T11:29:15.944 INFO:tasks.workunit.client.1.smithi178.stdout:8/47: creat d5/fa x:0 0 0 2024-09-10T11:29:15.945 INFO:tasks.workunit.client.1.smithi178.stdout:2/34: mkdir d1/d7/d8 0 2024-09-10T11:29:15.946 INFO:tasks.workunit.client.1.smithi178.stdout:9/77: rmdir d2/d8 39 2024-09-10T11:29:15.950 INFO:tasks.workunit.client.1.smithi178.stdout:2/35: dread d1/f3 [0,4194304] 0 2024-09-10T11:29:15.950 INFO:tasks.workunit.client.1.smithi178.stdout:2/36: stat d1/l6 0 2024-09-10T11:29:15.952 INFO:tasks.workunit.client.1.smithi178.stdout:8/48: rmdir d5 39 2024-09-10T11:29:15.954 INFO:tasks.workunit.client.1.smithi178.stdout:2/37: dread d1/f5 [0,4194304] 0 2024-09-10T11:29:15.954 INFO:tasks.workunit.client.1.smithi178.stdout:8/49: mkdir d5/db 0 2024-09-10T11:29:15.956 INFO:tasks.workunit.client.1.smithi178.stdout:8/50: write d5/fa [333483,104293] 0 2024-09-10T11:29:15.957 INFO:tasks.workunit.client.1.smithi178.stdout:2/38: creat d1/d7/d8/f9 x:0 0 0 2024-09-10T11:29:15.960 INFO:tasks.workunit.client.1.smithi178.stdout:9/78: dwrite d2/fd [0,4194304] 0 2024-09-10T11:29:15.965 INFO:tasks.workunit.client.1.smithi178.stdout:8/51: truncate d5/f6 1100660 0 2024-09-10T11:29:15.965 INFO:tasks.workunit.client.1.smithi178.stdout:8/52: rename d5 to d5/dc 22 2024-09-10T11:29:15.966 INFO:tasks.workunit.client.1.smithi178.stdout:2/39: truncate d1/f3 1114762 0 2024-09-10T11:29:15.967 INFO:tasks.workunit.client.1.smithi178.stdout:9/79: getdents d2 0 2024-09-10T11:29:15.968 INFO:tasks.workunit.client.1.smithi178.stdout:2/40: dread d1/f5 [0,4194304] 0 2024-09-10T11:29:15.969 INFO:tasks.workunit.client.1.smithi178.stdout:2/41: dread - d1/d7/d8/f9 zero size 2024-09-10T11:29:15.970 INFO:tasks.workunit.client.1.smithi178.stdout:8/53: mknod d5/db/cd 0 2024-09-10T11:29:15.970 INFO:tasks.workunit.client.1.smithi178.stdout:8/54: readlink d5/l9 0 2024-09-10T11:29:15.973 INFO:tasks.workunit.client.1.smithi178.stdout:8/55: creat d5/db/fe x:0 0 0 2024-09-10T11:29:15.973 INFO:tasks.workunit.client.1.smithi178.stdout:9/80: link d2/d8/fa d2/dc/f12 0 2024-09-10T11:29:15.974 INFO:tasks.workunit.client.1.smithi178.stdout:9/81: readlink d2/le 0 2024-09-10T11:29:15.975 INFO:tasks.workunit.client.1.smithi178.stdout:9/82: chown d2/df 439299 1 2024-09-10T11:29:15.978 INFO:tasks.workunit.client.1.smithi178.stdout:9/83: mkdir d2/d13 0 2024-09-10T11:29:15.979 INFO:tasks.workunit.client.1.smithi178.stdout:8/56: dread d5/fa [0,4194304] 0 2024-09-10T11:29:15.979 INFO:tasks.workunit.client.1.smithi178.stdout:2/42: dwrite d1/d7/d8/f9 [0,4194304] 0 2024-09-10T11:29:15.980 INFO:tasks.workunit.client.1.smithi178.stdout:8/57: write d5/f8 [743104,9300] 0 2024-09-10T11:29:15.981 INFO:tasks.workunit.client.1.smithi178.stdout:2/43: write d1/d7/d8/f9 [1893898,16676] 0 2024-09-10T11:29:15.984 INFO:tasks.workunit.client.1.smithi178.stdout:8/58: creat d5/ff x:0 0 0 2024-09-10T11:29:15.984 INFO:tasks.workunit.client.1.smithi178.stdout:8/59: read d5/fa [152788,30860] 0 2024-09-10T11:29:15.985 INFO:tasks.workunit.client.1.smithi178.stdout:2/44: write d1/f5 [580110,103874] 0 2024-09-10T11:29:15.985 INFO:tasks.workunit.client.1.smithi178.stdout:8/60: readlink d5/l9 0 2024-09-10T11:29:15.990 INFO:tasks.workunit.client.1.smithi178.stdout:2/45: symlink d1/d7/d8/la 0 2024-09-10T11:29:15.994 INFO:tasks.workunit.client.1.smithi178.stdout:8/61: dwrite d5/ff [0,4194304] 0 2024-09-10T11:29:15.997 INFO:tasks.workunit.client.1.smithi178.stdout:8/62: chown d5/db/fe 26 1 2024-09-10T11:29:15.998 INFO:tasks.workunit.client.1.smithi178.stdout:8/63: readlink d5/l9 0 2024-09-10T11:29:16.000 INFO:tasks.workunit.client.1.smithi178.stdout:8/64: write d5/ff [2678471,42829] 0 2024-09-10T11:29:16.000 INFO:tasks.workunit.client.1.smithi178.stdout:2/46: write d1/f3 [1465908,80286] 0 2024-09-10T11:29:16.002 INFO:tasks.workunit.client.1.smithi178.stdout:9/84: dwrite d2/d8/f9 [0,4194304] 0 2024-09-10T11:29:16.005 INFO:tasks.workunit.client.1.smithi178.stdout:8/65: write d5/fa [427414,86290] 0 2024-09-10T11:29:16.005 INFO:tasks.workunit.client.1.smithi178.stdout:2/47: symlink d1/d7/d8/lb 0 2024-09-10T11:29:16.006 INFO:tasks.workunit.client.1.smithi178.stdout:8/66: mkdir d5/db/d10 0 2024-09-10T11:29:16.008 INFO:tasks.workunit.client.1.smithi178.stdout:2/48: rename d1/d7/d8/la to d1/lc 0 2024-09-10T11:29:16.010 INFO:tasks.workunit.client.1.smithi178.stdout:8/67: mknod d5/c11 0 2024-09-10T11:29:16.011 INFO:tasks.workunit.client.1.smithi178.stdout:8/68: chown d5/db/cd 3603 1 2024-09-10T11:29:16.012 INFO:tasks.workunit.client.1.smithi178.stdout:2/49: unlink l0 0 2024-09-10T11:29:16.013 INFO:tasks.workunit.client.1.smithi178.stdout:9/85: getdents d2/df 0 2024-09-10T11:29:16.014 INFO:tasks.workunit.client.1.smithi178.stdout:8/69: mknod d5/c12 0 2024-09-10T11:29:16.014 INFO:tasks.workunit.client.1.smithi178.stdout:9/86: write d2/d8/f9 [9067,8452] 0 2024-09-10T11:29:16.017 INFO:tasks.workunit.client.1.smithi178.stdout:2/50: link d1/d7/d8/f9 d1/d7/d8/fd 0 2024-09-10T11:29:16.018 INFO:tasks.workunit.client.1.smithi178.stdout:9/87: chown d2/dc 12071 1 2024-09-10T11:29:16.018 INFO:tasks.workunit.client.1.smithi178.stdout:8/70: dread f2 [0,4194304] 0 2024-09-10T11:29:16.019 INFO:tasks.workunit.client.1.smithi178.stdout:2/51: unlink d1/l6 0 2024-09-10T11:29:16.021 INFO:tasks.workunit.client.1.smithi178.stdout:8/71: rename d5/f6 to d5/f13 0 2024-09-10T11:29:16.022 INFO:tasks.workunit.client.1.smithi178.stdout:2/52: mkdir d1/d7/de 0 2024-09-10T11:29:16.023 INFO:tasks.workunit.client.1.smithi178.stdout:8/72: creat d5/db/f14 x:0 0 0 2024-09-10T11:29:16.025 INFO:tasks.workunit.client.1.smithi178.stdout:9/88: link d2/le d2/df/l14 0 2024-09-10T11:29:16.025 INFO:tasks.workunit.client.1.smithi178.stdout:8/73: mknod d5/db/d10/c15 0 2024-09-10T11:29:16.026 INFO:tasks.workunit.client.1.smithi178.stdout:9/89: rmdir d2/d8 39 2024-09-10T11:29:16.026 INFO:tasks.workunit.client.1.smithi178.stdout:8/74: stat d5/db/d10/c15 0 2024-09-10T11:29:16.028 INFO:tasks.workunit.client.1.smithi178.stdout:2/53: dread d1/d7/d8/f9 [0,4194304] 0 2024-09-10T11:29:16.029 INFO:tasks.workunit.client.1.smithi178.stdout:8/75: symlink d5/l16 0 2024-09-10T11:29:16.031 INFO:tasks.workunit.client.1.smithi178.stdout:2/54: mkdir d1/d7/df 0 2024-09-10T11:29:16.031 INFO:tasks.workunit.client.1.smithi178.stdout:9/90: creat d2/d13/f15 x:0 0 0 2024-09-10T11:29:16.032 INFO:tasks.workunit.client.1.smithi178.stdout:8/76: creat d5/db/f17 x:0 0 0 2024-09-10T11:29:16.032 INFO:tasks.workunit.client.1.smithi178.stdout:9/91: rmdir d2 39 2024-09-10T11:29:16.038 INFO:tasks.workunit.client.1.smithi178.stdout:2/55: dwrite d1/f5 [0,4194304] 0 2024-09-10T11:29:16.040 INFO:tasks.workunit.client.1.smithi178.stdout:9/92: dread d2/d8/f9 [0,4194304] 0 2024-09-10T11:29:16.040 INFO:tasks.workunit.client.1.smithi178.stdout:2/56: rmdir d1/d7 39 2024-09-10T11:29:16.043 INFO:tasks.workunit.client.1.smithi178.stdout:2/57: write d1/d7/d8/f9 [2170524,9891] 0 2024-09-10T11:29:16.044 INFO:tasks.workunit.client.1.smithi178.stdout:9/93: dread d2/f4 [0,4194304] 0 2024-09-10T11:29:16.046 INFO:tasks.workunit.client.1.smithi178.stdout:9/94: readlink d2/le 0 2024-09-10T11:29:16.048 INFO:tasks.workunit.client.1.smithi178.stdout:9/95: dread d2/f7 [0,4194304] 0 2024-09-10T11:29:16.056 INFO:tasks.workunit.client.1.smithi178.stdout:9/96: dwrite d2/d8/f9 [0,4194304] 0 2024-09-10T11:29:16.059 INFO:tasks.workunit.client.1.smithi178.stdout:9/97: rmdir d2/dc 39 2024-09-10T11:29:16.063 INFO:tasks.workunit.client.1.smithi178.stdout:9/98: mkdir d2/dc/d16 0 2024-09-10T11:29:16.066 INFO:tasks.workunit.client.1.smithi178.stdout:9/99: dread d2/d8/f9 [0,4194304] 0 2024-09-10T11:29:16.069 INFO:tasks.workunit.client.1.smithi178.stdout:9/100: fdatasync d2/f3 0 2024-09-10T11:29:16.070 INFO:tasks.workunit.client.1.smithi178.stdout:9/101: write d2/f3 [3226168,47743] 0 2024-09-10T11:29:16.077 INFO:tasks.workunit.client.1.smithi178.stdout:9/102: dwrite d2/dc/f10 [0,4194304] 0 2024-09-10T11:29:16.078 INFO:tasks.workunit.client.1.smithi178.stdout:9/103: stat d2/dc/d16 0 2024-09-10T11:29:16.079 INFO:tasks.workunit.client.1.smithi178.stdout:9/104: stat d2/df/c11 0 2024-09-10T11:29:16.080 INFO:tasks.workunit.client.1.smithi178.stdout:9/105: chown d2/d13 310 1 2024-09-10T11:29:16.086 INFO:tasks.workunit.client.1.smithi178.stdout:9/106: dwrite d2/f3 [0,4194304] 0 2024-09-10T11:29:16.089 INFO:tasks.workunit.client.1.smithi178.stdout:9/107: creat d2/d8/f17 x:0 0 0 2024-09-10T11:29:16.095 INFO:tasks.workunit.client.1.smithi178.stdout:9/108: rename d2/dc/f10 to d2/f18 0 2024-09-10T11:29:16.096 INFO:tasks.workunit.client.1.smithi178.stdout:9/109: readlink d2/df/l14 0 2024-09-10T11:29:16.241 INFO:tasks.workunit.client.1.smithi178.stdout:8/77: write f2 [1498923,115972] 0 2024-09-10T11:29:16.242 INFO:tasks.workunit.client.1.smithi178.stdout:2/58: truncate d1/f5 403217 0 2024-09-10T11:29:16.243 INFO:tasks.workunit.client.1.smithi178.stdout:9/110: fdatasync d2/f3 0 2024-09-10T11:29:16.243 INFO:tasks.workunit.client.1.smithi178.stdout:8/78: chown d5/f8 2174 1 2024-09-10T11:29:16.243 INFO:tasks.workunit.client.1.smithi178.stdout:2/59: fdatasync d1/d7/d8/f9 0 2024-09-10T11:29:16.247 INFO:tasks.workunit.client.1.smithi178.stdout:9/111: mkdir d2/d19 0 2024-09-10T11:29:16.247 INFO:tasks.workunit.client.1.smithi178.stdout:2/60: dread d1/f3 [0,4194304] 0 2024-09-10T11:29:16.250 INFO:tasks.workunit.client.1.smithi178.stdout:2/61: dread d1/f3 [0,4194304] 0 2024-09-10T11:29:16.251 INFO:tasks.workunit.client.1.smithi178.stdout:9/112: getdents d2/df 0 2024-09-10T11:29:16.252 INFO:tasks.workunit.client.1.smithi178.stdout:2/62: mkdir d1/d7/d10 0 2024-09-10T11:29:16.254 INFO:tasks.workunit.client.1.smithi178.stdout:2/63: write d1/f5 [654605,29806] 0 2024-09-10T11:29:16.254 INFO:tasks.workunit.client.1.smithi178.stdout:8/79: truncate d5/ff 3151656 0 2024-09-10T11:29:16.256 INFO:tasks.workunit.client.1.smithi178.stdout:9/113: rename d2/dc/f12 to d2/d8/f1a 0 2024-09-10T11:29:16.258 INFO:tasks.workunit.client.1.smithi178.stdout:8/80: dread f2 [0,4194304] 0 2024-09-10T11:29:16.260 INFO:tasks.workunit.client.1.smithi178.stdout:2/64: rename d1/d7/d8 to d1/d7/de/d11 0 2024-09-10T11:29:16.264 INFO:tasks.workunit.client.1.smithi178.stdout:8/81: creat d5/db/d10/f18 x:0 0 0 2024-09-10T11:29:16.267 INFO:tasks.workunit.client.1.smithi178.stdout:8/82: creat d5/db/f19 x:0 0 0 2024-09-10T11:29:16.268 INFO:tasks.workunit.client.1.smithi178.stdout:2/65: dwrite d1/d7/de/d11/fd [0,4194304] 0 2024-09-10T11:29:16.269 INFO:tasks.workunit.client.1.smithi178.stdout:8/83: write d5/f8 [897116,81432] 0 2024-09-10T11:29:16.270 INFO:tasks.workunit.client.1.smithi178.stdout:9/114: dwrite d2/fd [0,4194304] 0 2024-09-10T11:29:16.270 INFO:tasks.workunit.client.1.smithi178.stdout:2/66: fsync d1/f5 0 2024-09-10T11:29:16.274 INFO:tasks.workunit.client.1.smithi178.stdout:2/67: mkdir d1/d12 0 2024-09-10T11:29:16.276 INFO:tasks.workunit.client.1.smithi178.stdout:9/115: rmdir d2/dc/d16 0 2024-09-10T11:29:16.279 INFO:tasks.workunit.client.1.smithi178.stdout:8/84: dwrite d5/db/fe [0,4194304] 0 2024-09-10T11:29:16.284 INFO:tasks.workunit.client.1.smithi178.stdout:8/85: symlink d5/db/d10/l1a 0 2024-09-10T11:29:16.286 INFO:tasks.workunit.client.1.smithi178.stdout:2/68: dwrite d1/f5 [0,4194304] 0 2024-09-10T11:29:16.286 INFO:tasks.workunit.client.1.smithi178.stdout:8/86: write d5/f8 [223151,6071] 0 2024-09-10T11:29:16.289 INFO:tasks.workunit.client.1.smithi178.stdout:2/69: write d1/d7/de/d11/f9 [1023622,13142] 0 2024-09-10T11:29:16.290 INFO:tasks.workunit.client.1.smithi178.stdout:2/70: chown d1/f3 40088649 1 2024-09-10T11:29:16.290 INFO:tasks.workunit.client.1.smithi178.stdout:9/116: getdents d2/d13 0 2024-09-10T11:29:16.295 INFO:tasks.workunit.client.1.smithi178.stdout:9/117: rename d2/f7 to d2/df/f1b 0 2024-09-10T11:29:16.298 INFO:tasks.workunit.client.1.smithi178.stdout:2/71: creat d1/f13 x:0 0 0 2024-09-10T11:29:16.299 INFO:tasks.workunit.client.1.smithi178.stdout:8/87: rmdir d5/db/d10 39 2024-09-10T11:29:16.309 INFO:tasks.workunit.client.1.smithi178.stdout:9/118: dwrite d2/f4 [0,4194304] 0 2024-09-10T11:29:16.315 INFO:tasks.workunit.client.1.smithi178.stdout:8/88: dwrite d5/db/f17 [0,4194304] 0 2024-09-10T11:29:16.315 INFO:tasks.workunit.client.1.smithi178.stdout:2/72: dwrite d1/f3 [0,4194304] 0 2024-09-10T11:29:16.317 INFO:tasks.workunit.client.1.smithi178.stdout:2/73: write d1/f3 [1970991,39712] 0 2024-09-10T11:29:16.322 INFO:tasks.workunit.client.1.smithi178.stdout:8/89: symlink d5/l1b 0 2024-09-10T11:29:16.328 INFO:tasks.workunit.client.1.smithi178.stdout:8/90: unlink d5/db/d10/f18 0 2024-09-10T11:29:16.329 INFO:tasks.workunit.client.1.smithi178.stdout:8/91: write d5/db/f14 [495544,9230] 0 2024-09-10T11:29:16.330 INFO:tasks.workunit.client.1.smithi178.stdout:9/119: write d2/d8/fa [335389,94498] 0 2024-09-10T11:29:16.331 INFO:tasks.workunit.client.1.smithi178.stdout:9/120: dread - d2/d8/f17 zero size 2024-09-10T11:29:16.331 INFO:tasks.workunit.client.1.smithi178.stdout:8/92: mknod d5/c1c 0 2024-09-10T11:29:16.334 INFO:tasks.workunit.client.1.smithi178.stdout:8/93: write f2 [2627736,15815] 0 2024-09-10T11:29:16.336 INFO:tasks.workunit.client.1.smithi178.stdout:8/94: mkdir d5/d1d 0 2024-09-10T11:29:16.337 INFO:tasks.workunit.client.1.smithi178.stdout:9/121: dread d2/f4 [0,4194304] 0 2024-09-10T11:29:16.340 INFO:tasks.workunit.client.1.smithi178.stdout:9/122: mkdir d2/d19/d1c 0 2024-09-10T11:29:16.340 INFO:tasks.workunit.client.1.smithi178.stdout:9/123: chown d2/dc 7138327 1 2024-09-10T11:29:16.341 INFO:tasks.workunit.client.1.smithi178.stdout:8/95: dread d5/db/fe [0,4194304] 0 2024-09-10T11:29:16.342 INFO:tasks.workunit.client.1.smithi178.stdout:9/124: truncate d2/f4 4704943 0 2024-09-10T11:29:16.344 INFO:tasks.workunit.client.1.smithi178.stdout:9/125: symlink d2/d8/l1d 0 2024-09-10T11:29:16.346 INFO:tasks.workunit.client.1.smithi178.stdout:8/96: dread d5/f13 [0,4194304] 0 2024-09-10T11:29:16.346 INFO:tasks.workunit.client.1.smithi178.stdout:9/126: truncate d2/d13/f15 128491 0 2024-09-10T11:29:16.349 INFO:tasks.workunit.client.1.smithi178.stdout:9/127: fdatasync d2/df/f1b 0 2024-09-10T11:29:16.350 INFO:tasks.workunit.client.1.smithi178.stdout:8/97: mkdir d5/db/d1e 0 2024-09-10T11:29:16.350 INFO:tasks.workunit.client.1.smithi178.stdout:8/98: chown d5/c11 0 1 2024-09-10T11:29:16.351 INFO:tasks.workunit.client.1.smithi178.stdout:8/99: dread - d5/db/f19 zero size 2024-09-10T11:29:16.351 INFO:tasks.workunit.client.1.smithi178.stdout:9/128: creat d2/df/f1e x:0 0 0 2024-09-10T11:29:16.353 INFO:tasks.workunit.client.1.smithi178.stdout:8/100: creat d5/f1f x:0 0 0 2024-09-10T11:29:16.357 INFO:tasks.workunit.client.1.smithi178.stdout:9/129: truncate d2/fd 2236556 0 2024-09-10T11:29:16.369 INFO:tasks.workunit.client.1.smithi178.stdout:9/130: truncate d2/d8/f9 950517 0 2024-09-10T11:29:16.370 INFO:tasks.workunit.client.1.smithi178.stdout:9/131: write d2/d8/f17 [393524,51270] 0 2024-09-10T11:29:16.371 INFO:tasks.workunit.client.1.smithi178.stdout:9/132: write d2/df/f1e [844650,106671] 0 2024-09-10T11:29:16.374 INFO:tasks.workunit.client.1.smithi178.stdout:9/133: creat d2/d13/f1f x:0 0 0 2024-09-10T11:29:16.376 INFO:tasks.workunit.client.1.smithi178.stdout:9/134: chown d2/d8/f9 0 1 2024-09-10T11:29:16.382 INFO:tasks.workunit.client.1.smithi178.stdout:9/135: unlink d2/fd 0 2024-09-10T11:29:16.385 INFO:tasks.workunit.client.1.smithi178.stdout:9/136: symlink d2/df/l20 0 2024-09-10T11:29:16.393 INFO:tasks.workunit.client.1.smithi178.stdout:9/137: write d2/d8/f9 [205942,75467] 0 2024-09-10T11:29:16.400 INFO:tasks.workunit.client.1.smithi178.stdout:9/138: dwrite d2/df/f1e [0,4194304] 0 2024-09-10T11:29:16.401 INFO:tasks.workunit.client.1.smithi178.stdout:9/139: truncate d2/f3 4990832 0 2024-09-10T11:29:16.402 INFO:tasks.workunit.client.1.smithi178.stdout:9/140: chown d2/d13/f1f 272 1 2024-09-10T11:29:16.403 INFO:tasks.workunit.client.1.smithi178.stdout:9/141: chown d2/f3 0 1 2024-09-10T11:29:16.411 INFO:tasks.workunit.client.1.smithi178.stdout:9/142: dwrite d2/d13/f1f [0,4194304] 0 2024-09-10T11:29:16.415 INFO:tasks.workunit.client.1.smithi178.stdout:9/143: unlink d2/f3 0 2024-09-10T11:29:16.417 INFO:tasks.workunit.client.1.smithi178.stdout:9/144: creat d2/d13/f21 x:0 0 0 2024-09-10T11:29:16.418 INFO:tasks.workunit.client.1.smithi178.stdout:9/145: dread - d2/d13/f21 zero size 2024-09-10T11:29:16.421 INFO:tasks.workunit.client.1.smithi178.stdout:9/146: creat d2/d19/d1c/f22 x:0 0 0 2024-09-10T11:29:16.422 INFO:tasks.workunit.client.1.smithi178.stdout:9/147: mknod d2/d19/d1c/c23 0 2024-09-10T11:29:16.425 INFO:tasks.workunit.client.1.smithi178.stdout:9/148: rename d2/d13/f21 to d2/d19/d1c/f24 0 2024-09-10T11:29:16.432 INFO:tasks.workunit.client.1.smithi178.stdout:9/149: dwrite d2/f18 [0,4194304] 0 2024-09-10T11:29:16.435 INFO:tasks.workunit.client.1.smithi178.stdout:9/150: fsync d2/df/f1b 0 2024-09-10T11:29:16.441 INFO:tasks.workunit.client.1.smithi178.stdout:9/151: rename d2/df/l20 to d2/d8/l25 0 2024-09-10T11:29:16.454 INFO:tasks.workunit.client.1.smithi178.stdout:9/152: rename d2/df/f1e to d2/f26 0 2024-09-10T11:29:16.459 INFO:tasks.workunit.client.1.smithi178.stdout:9/153: write d2/d8/f9 [1275716,67091] 0 2024-09-10T11:29:16.464 INFO:tasks.workunit.client.1.smithi178.stdout:9/154: dread d2/d13/f1f [0,4194304] 0 2024-09-10T11:29:16.474 INFO:tasks.workunit.client.1.smithi178.stdout:9/155: dwrite d2/f26 [4194304,4194304] 0 2024-09-10T11:29:16.477 INFO:tasks.workunit.client.1.smithi178.stdout:9/156: chown d2/f18 106578894 1 2024-09-10T11:29:16.478 INFO:tasks.workunit.client.1.smithi178.stdout:9/157: chown d2/df 13916506 1 2024-09-10T11:29:16.483 INFO:tasks.workunit.client.1.smithi178.stdout:9/158: truncate d2/f26 4390640 0 2024-09-10T11:29:16.485 INFO:tasks.workunit.client.1.smithi178.stdout:9/159: chown d2/d19/d1c/f22 1 1 2024-09-10T11:29:16.494 INFO:tasks.workunit.client.1.smithi178.stdout:9/160: dwrite d2/df/f1b [0,4194304] 0 2024-09-10T11:29:16.498 INFO:tasks.workunit.client.1.smithi178.stdout:9/161: chown d2/c6 1009151 1 2024-09-10T11:29:16.500 INFO:tasks.workunit.client.1.smithi178.stdout:9/162: mknod d2/d13/c27 0 2024-09-10T11:29:16.501 INFO:tasks.workunit.client.1.smithi178.stdout:9/163: write d2/d8/f1a [950130,76921] 0 2024-09-10T11:29:16.502 INFO:tasks.workunit.client.1.smithi178.stdout:9/164: write d2/d13/f15 [679055,30732] 0 2024-09-10T11:29:16.504 INFO:tasks.workunit.client.1.smithi178.stdout:9/165: creat d2/df/f28 x:0 0 0 2024-09-10T11:29:16.505 INFO:tasks.workunit.client.1.smithi178.stdout:9/166: dread - d2/d19/d1c/f24 zero size 2024-09-10T11:29:16.507 INFO:tasks.workunit.client.1.smithi178.stdout:9/167: write d2/f26 [1407524,60888] 0 2024-09-10T11:29:16.509 INFO:tasks.workunit.client.1.smithi178.stdout:9/168: write d2/d13/f1f [3309424,104313] 0 2024-09-10T11:29:16.513 INFO:tasks.workunit.client.1.smithi178.stdout:9/169: dread d2/d8/f17 [0,4194304] 0 2024-09-10T11:29:16.514 INFO:tasks.workunit.client.1.smithi178.stdout:9/170: write d2/df/f1b [2463640,100151] 0 2024-09-10T11:29:16.523 INFO:tasks.workunit.client.1.smithi178.stdout:9/171: dwrite d2/d13/f15 [0,4194304] 0 2024-09-10T11:29:16.524 INFO:tasks.workunit.client.1.smithi178.stdout:9/172: write d2/f26 [605074,36605] 0 2024-09-10T11:29:16.527 INFO:tasks.workunit.client.1.smithi178.stdout:9/173: mkdir d2/d19/d29 0 2024-09-10T11:29:16.529 INFO:tasks.workunit.client.1.smithi178.stdout:9/174: mkdir d2/d19/d1c/d2a 0 2024-09-10T11:29:16.530 INFO:tasks.workunit.client.1.smithi178.stdout:9/175: truncate d2/d8/f17 456592 0 2024-09-10T11:29:16.539 INFO:tasks.workunit.client.1.smithi178.stdout:9/176: dwrite d2/d8/f1a [0,4194304] 0 2024-09-10T11:29:16.559 INFO:tasks.workunit.client.1.smithi178.stdout:9/177: getdents d2/dc 0 2024-09-10T11:29:16.567 INFO:tasks.workunit.client.1.smithi178.stdout:9/178: dwrite d2/d8/f9 [0,4194304] 0 2024-09-10T11:29:16.678 INFO:tasks.workunit.client.1.smithi178.stdout:9/179: mknod d2/d13/c2b 0 2024-09-10T11:29:16.681 INFO:tasks.workunit.client.1.smithi178.stdout:9/180: link d2/df/f1b d2/d19/d1c/f2c 0 2024-09-10T11:29:16.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:16 smithi178 ceph-mon[33514]: pgmap v15: 65 pgs: 65 active+clean; 195 MiB data, 831 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 355 op/s 2024-09-10T11:29:16.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:16 smithi144 ceph-mon[27942]: pgmap v15: 65 pgs: 65 active+clean; 195 MiB data, 831 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 355 op/s 2024-09-10T11:29:18.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:18 smithi178 ceph-mon[33514]: pgmap v16: 65 pgs: 65 active+clean; 195 MiB data, 831 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 355 op/s 2024-09-10T11:29:18.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:18 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:18.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:18 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:18.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:18 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:18.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:18 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:18.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:18 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:18.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:18 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:18.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:18 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:18.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:18 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:18.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:18 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:18.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:18 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:29:18.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:18 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:29:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:18 smithi144 ceph-mon[27942]: pgmap v16: 65 pgs: 65 active+clean; 195 MiB data, 831 MiB used, 536 GiB / 536 GiB avail; 2.2 MiB/s rd, 4.6 MiB/s wr, 355 op/s 2024-09-10T11:29:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:18 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:18 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:18 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:18 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:18 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:18 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:18 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:18.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:18 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T11:29:18.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:18 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:18.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:18 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:29:18.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:18 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:29:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:19 smithi144 ceph-mon[27942]: Adjusting osd_memory_target on smithi178 to 2239M 2024-09-10T11:29:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:19 smithi144 ceph-mon[27942]: Updating smithi144:/etc/ceph/ceph.conf 2024-09-10T11:29:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:19 smithi144 ceph-mon[27942]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T11:29:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:19 smithi144 ceph-mon[27942]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:29:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:19 smithi144 ceph-mon[27942]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:29:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:19 smithi144 ceph-mon[27942]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:29:19.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:19 smithi178 ceph-mon[33514]: Adjusting osd_memory_target on smithi178 to 2239M 2024-09-10T11:29:19.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:19 smithi178 ceph-mon[33514]: Updating smithi144:/etc/ceph/ceph.conf 2024-09-10T11:29:19.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:19 smithi178 ceph-mon[33514]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T11:29:19.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:19 smithi178 ceph-mon[33514]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:29:19.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:19 smithi178 ceph-mon[33514]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:29:19.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:19 smithi178 ceph-mon[33514]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:29:20.530 INFO:tasks.workunit.client.1.smithi178.stdout:0/30: sync 2024-09-10T11:29:20.530 INFO:tasks.workunit.client.1.smithi178.stdout:5/15: sync 2024-09-10T11:29:20.530 INFO:tasks.workunit.client.1.smithi178.stdout:1/0: sync 2024-09-10T11:29:20.530 INFO:tasks.workunit.client.1.smithi178.stdout:4/0: sync 2024-09-10T11:29:20.530 INFO:tasks.workunit.client.1.smithi178.stdout:1/1: dwrite - no filename 2024-09-10T11:29:20.530 INFO:tasks.workunit.client.1.smithi178.stdout:1/2: write - no filename 2024-09-10T11:29:20.530 INFO:tasks.workunit.client.1.smithi178.stdout:4/1: write - no filename 2024-09-10T11:29:20.530 INFO:tasks.workunit.client.1.smithi178.stdout:1/3: rename - no filename 2024-09-10T11:29:20.530 INFO:tasks.workunit.client.1.smithi178.stdout:1/4: rename - no filename 2024-09-10T11:29:20.530 INFO:tasks.workunit.client.1.smithi178.stdout:1/5: dwrite - no filename 2024-09-10T11:29:20.531 INFO:tasks.workunit.client.1.smithi178.stdout:1/6: fsync - no filename 2024-09-10T11:29:20.531 INFO:tasks.workunit.client.1.smithi178.stdout:3/62: sync 2024-09-10T11:29:20.531 INFO:tasks.workunit.client.1.smithi178.stdout:8/101: sync 2024-09-10T11:29:20.531 INFO:tasks.workunit.client.1.smithi178.stdout:9/181: sync 2024-09-10T11:29:20.531 INFO:tasks.workunit.client.1.smithi178.stdout:2/74: sync 2024-09-10T11:29:20.531 INFO:tasks.workunit.client.1.smithi178.stdout:7/20: sync 2024-09-10T11:29:20.531 INFO:tasks.workunit.client.1.smithi178.stdout:6/33: sync 2024-09-10T11:29:20.532 INFO:tasks.workunit.client.1.smithi178.stdout:6/34: write d3/fa [922095,56049] 0 2024-09-10T11:29:20.534 INFO:tasks.workunit.client.1.smithi178.stdout:7/21: mkdir d6 0 2024-09-10T11:29:20.534 INFO:tasks.workunit.client.1.smithi178.stdout:1/7: mknod c0 0 2024-09-10T11:29:20.534 INFO:tasks.workunit.client.1.smithi178.stdout:1/8: write - no filename 2024-09-10T11:29:20.535 INFO:tasks.workunit.client.1.smithi178.stdout:1/9: dwrite - no filename 2024-09-10T11:29:20.535 INFO:tasks.workunit.client.1.smithi178.stdout:7/22: dread - f2 zero size 2024-09-10T11:29:20.535 INFO:tasks.workunit.client.1.smithi178.stdout:4/2: mkdir d0 0 2024-09-10T11:29:20.536 INFO:tasks.workunit.client.1.smithi178.stdout:3/63: creat db/f16 x:0 0 0 2024-09-10T11:29:20.536 INFO:tasks.workunit.client.1.smithi178.stdout:0/31: dread d0/f1 [0,4194304] 0 2024-09-10T11:29:20.538 INFO:tasks.workunit.client.1.smithi178.stdout:9/182: mkdir d2/d19/d1c/d2d 0 2024-09-10T11:29:20.539 INFO:tasks.workunit.client.1.smithi178.stdout:6/35: mknod d3/cc 0 2024-09-10T11:29:20.540 INFO:tasks.workunit.client.1.smithi178.stdout:1/10: creat f1 x:0 0 0 2024-09-10T11:29:20.542 INFO:tasks.workunit.client.1.smithi178.stdout:3/64: symlink db/l17 0 2024-09-10T11:29:20.543 INFO:tasks.workunit.client.1.smithi178.stdout:6/36: write d3/fa [1241686,7726] 0 2024-09-10T11:29:20.543 INFO:tasks.workunit.client.1.smithi178.stdout:5/16: dread f2 [0,4194304] 0 2024-09-10T11:29:20.544 INFO:tasks.workunit.client.1.smithi178.stdout:4/3: symlink d0/l1 0 2024-09-10T11:29:20.544 INFO:tasks.workunit.client.1.smithi178.stdout:4/4: dread - no filename 2024-09-10T11:29:20.545 INFO:tasks.workunit.client.1.smithi178.stdout:7/23: mknod d6/c7 0 2024-09-10T11:29:20.547 INFO:tasks.workunit.client.1.smithi178.stdout:8/102: dwrite d5/f8 [0,4194304] 0 2024-09-10T11:29:20.547 INFO:tasks.workunit.client.1.smithi178.stdout:4/5: rename d0 to d0/d2 22 2024-09-10T11:29:20.548 INFO:tasks.workunit.client.1.smithi178.stdout:9/183: creat d2/d13/f2e x:0 0 0 2024-09-10T11:29:20.548 INFO:tasks.workunit.client.1.smithi178.stdout:2/75: dwrite d1/f5 [0,4194304] 0 2024-09-10T11:29:20.549 INFO:tasks.workunit.client.1.smithi178.stdout:1/11: symlink l2 0 2024-09-10T11:29:20.550 INFO:tasks.workunit.client.1.smithi178.stdout:3/65: creat db/f18 x:0 0 0 2024-09-10T11:29:20.551 INFO:tasks.workunit.client.1.smithi178.stdout:5/17: mknod c8 0 2024-09-10T11:29:20.553 INFO:tasks.workunit.client.1.smithi178.stdout:5/18: chown f2 5569 1 2024-09-10T11:29:20.553 INFO:tasks.workunit.client.1.smithi178.stdout:9/184: dread d2/d8/f17 [0,4194304] 0 2024-09-10T11:29:20.554 INFO:tasks.workunit.client.1.smithi178.stdout:1/12: mkdir d3 0 2024-09-10T11:29:20.558 INFO:tasks.workunit.client.1.smithi178.stdout:3/66: read f1 [38980,48325] 0 2024-09-10T11:29:20.559 INFO:tasks.workunit.client.1.smithi178.stdout:8/103: mkdir d5/db/d10/d20 0 2024-09-10T11:29:20.561 INFO:tasks.workunit.client.1.smithi178.stdout:2/76: rename d1/d7/de to d1/d7/d10/d14 0 2024-09-10T11:29:20.561 INFO:tasks.workunit.client.1.smithi178.stdout:7/24: dwrite f2 [0,4194304] 0 2024-09-10T11:29:20.562 INFO:tasks.workunit.client.1.smithi178.stdout:3/67: dread - db/f15 zero size 2024-09-10T11:29:20.563 INFO:tasks.workunit.client.1.smithi178.stdout:5/19: mkdir d9 0 2024-09-10T11:29:20.572 INFO:tasks.workunit.client.1.smithi178.stdout:1/13: symlink d3/l4 0 2024-09-10T11:29:20.572 INFO:tasks.workunit.client.1.smithi178.stdout:5/20: truncate f2 4654285 0 2024-09-10T11:29:20.573 INFO:tasks.workunit.client.1.smithi178.stdout:6/37: getdents d3 0 2024-09-10T11:29:20.574 INFO:tasks.workunit.client.1.smithi178.stdout:3/68: fsync f1 0 2024-09-10T11:29:20.575 INFO:tasks.workunit.client.1.smithi178.stdout:0/32: dwrite d0/f1 [0,4194304] 0 2024-09-10T11:29:20.580 INFO:tasks.workunit.client.1.smithi178.stdout:9/185: rename d2/f4 to d2/dc/f2f 0 2024-09-10T11:29:20.581 INFO:tasks.workunit.client.1.smithi178.stdout:8/104: rename d5/db/d1e to d5/db/d1e/d21 22 2024-09-10T11:29:20.582 INFO:tasks.workunit.client.1.smithi178.stdout:2/77: dwrite d1/f3 [0,4194304] 0 2024-09-10T11:29:20.584 INFO:tasks.workunit.client.1.smithi178.stdout:8/105: read d5/f7 [1596490,92561] 0 2024-09-10T11:29:20.584 INFO:tasks.workunit.client.1.smithi178.stdout:2/78: write d1/f3 [5149175,46639] 0 2024-09-10T11:29:20.585 INFO:tasks.workunit.client.1.smithi178.stdout:1/14: mknod d3/c5 0 2024-09-10T11:29:20.589 INFO:tasks.workunit.client.1.smithi178.stdout:8/106: readlink d5/l1b 0 2024-09-10T11:29:20.589 INFO:tasks.workunit.client.1.smithi178.stdout:6/38: dwrite d3/fa [0,4194304] 0 2024-09-10T11:29:20.589 INFO:tasks.workunit.client.1.smithi178.stdout:0/33: mknod d0/c4 0 2024-09-10T11:29:20.591 INFO:tasks.workunit.client.1.smithi178.stdout:0/34: chown d0 461588 1 2024-09-10T11:29:20.592 INFO:tasks.workunit.client.1.smithi178.stdout:1/15: write f1 [14197,129431] 0 2024-09-10T11:29:20.595 INFO:tasks.workunit.client.1.smithi178.stdout:0/35: read d0/f1 [847495,117691] 0 2024-09-10T11:29:20.596 INFO:tasks.workunit.client.1.smithi178.stdout:1/16: mknod d3/c6 0 2024-09-10T11:29:20.597 INFO:tasks.workunit.client.1.smithi178.stdout:7/25: getdents d6 0 2024-09-10T11:29:20.598 INFO:tasks.workunit.client.1.smithi178.stdout:1/17: fdatasync f1 0 2024-09-10T11:29:20.598 INFO:tasks.workunit.client.1.smithi178.stdout:8/107: mkdir d5/d1d/d22 0 2024-09-10T11:29:20.601 INFO:tasks.workunit.client.1.smithi178.stdout:2/79: rename d1/d7/d10/d14/d11/fd to d1/d12/f15 0 2024-09-10T11:29:20.602 INFO:tasks.workunit.client.1.smithi178.stdout:5/21: dwrite f4 [0,4194304] 0 2024-09-10T11:29:20.602 INFO:tasks.workunit.client.1.smithi178.stdout:2/80: write d1/f13 [505805,117190] 0 2024-09-10T11:29:20.604 INFO:tasks.workunit.client.1.smithi178.stdout:0/36: creat d0/f5 x:0 0 0 2024-09-10T11:29:20.607 INFO:tasks.workunit.client.1.smithi178.stdout:1/18: creat d3/f7 x:0 0 0 2024-09-10T11:29:20.609 INFO:tasks.workunit.client.1.smithi178.stdout:6/39: rename d3/d7/l9 to d3/d7/d8/ld 0 2024-09-10T11:29:20.613 INFO:tasks.workunit.client.1.smithi178.stdout:2/81: creat d1/d7/f16 x:0 0 0 2024-09-10T11:29:20.613 INFO:tasks.workunit.client.1.smithi178.stdout:6/40: chown d3/d7/d8/f6 119544 1 2024-09-10T11:29:20.614 INFO:tasks.workunit.client.1.smithi178.stdout:0/37: rename d0/f5 to d0/f6 0 2024-09-10T11:29:20.616 INFO:tasks.workunit.client.1.smithi178.stdout:8/108: rename d5/db/f19 to d5/d1d/d22/f23 0 2024-09-10T11:29:20.617 INFO:tasks.workunit.client.1.smithi178.stdout:1/19: dwrite d3/f7 [0,4194304] 0 2024-09-10T11:29:20.619 INFO:tasks.workunit.client.1.smithi178.stdout:2/82: dread d1/f5 [0,4194304] 0 2024-09-10T11:29:20.628 INFO:tasks.workunit.client.1.smithi178.stdout:9/186: symlink d2/d19/d1c/l30 0 2024-09-10T11:29:20.635 INFO:tasks.workunit.client.1.smithi178.stdout:5/22: dwrite f5 [0,4194304] 0 2024-09-10T11:29:20.636 INFO:tasks.workunit.client.1.smithi178.stdout:1/20: rename d3/f7 to d3/f8 0 2024-09-10T11:29:20.636 INFO:tasks.workunit.client.1.smithi178.stdout:9/187: chown d2/f18 0 1 2024-09-10T11:29:20.637 INFO:tasks.workunit.client.1.smithi178.stdout:8/109: truncate d5/ff 3392310 0 2024-09-10T11:29:20.639 INFO:tasks.workunit.client.1.smithi178.stdout:6/41: truncate d3/fa 206010 0 2024-09-10T11:29:20.643 INFO:tasks.workunit.client.1.smithi178.stdout:1/21: mkdir d3/d9 0 2024-09-10T11:29:20.644 INFO:tasks.workunit.client.1.smithi178.stdout:6/42: write d3/fb [649798,119449] 0 2024-09-10T11:29:20.644 INFO:tasks.workunit.client.1.smithi178.stdout:1/22: readlink l2 0 2024-09-10T11:29:20.644 INFO:tasks.workunit.client.1.smithi178.stdout:0/38: dwrite d0/f2 [0,4194304] 0 2024-09-10T11:29:20.645 INFO:tasks.workunit.client.1.smithi178.stdout:9/188: mkdir d2/df/d31 0 2024-09-10T11:29:20.647 INFO:tasks.workunit.client.1.smithi178.stdout:2/83: getdents d1/d7/d10/d14 0 2024-09-10T11:29:20.651 INFO:tasks.workunit.client.1.smithi178.stdout:6/43: symlink d3/le 0 2024-09-10T11:29:20.651 INFO:tasks.workunit.client.1.smithi178.stdout:8/110: dread d5/db/fe [0,4194304] 0 2024-09-10T11:29:20.651 INFO:tasks.workunit.client.1.smithi178.stdout:5/23: dwrite f2 [0,4194304] 0 2024-09-10T11:29:20.652 INFO:tasks.workunit.client.1.smithi178.stdout:5/24: rename d9 to d9/da 22 2024-09-10T11:29:20.653 INFO:tasks.workunit.client.1.smithi178.stdout:9/189: readlink d2/d8/l1d 0 2024-09-10T11:29:20.656 INFO:tasks.workunit.client.1.smithi178.stdout:2/84: creat d1/d12/f17 x:0 0 0 2024-09-10T11:29:20.657 INFO:tasks.workunit.client.1.smithi178.stdout:1/23: creat d3/d9/fa x:0 0 0 2024-09-10T11:29:20.658 INFO:tasks.workunit.client.1.smithi178.stdout:8/111: creat d5/f24 x:0 0 0 2024-09-10T11:29:20.661 INFO:tasks.workunit.client.1.smithi178.stdout:2/85: symlink d1/d7/d10/l18 0 2024-09-10T11:29:20.661 INFO:tasks.workunit.client.1.smithi178.stdout:1/24: creat d3/d9/fb x:0 0 0 2024-09-10T11:29:20.662 INFO:tasks.workunit.client.1.smithi178.stdout:1/25: write f1 [199945,17207] 0 2024-09-10T11:29:20.663 INFO:tasks.workunit.client.1.smithi178.stdout:6/44: link f1 d3/d7/ff 0 2024-09-10T11:29:20.664 INFO:tasks.workunit.client.1.smithi178.stdout:5/25: symlink d9/lb 0 2024-09-10T11:29:20.666 INFO:tasks.workunit.client.1.smithi178.stdout:1/26: creat d3/d9/fc x:0 0 0 2024-09-10T11:29:20.668 INFO:tasks.workunit.client.1.smithi178.stdout:1/27: readlink d3/l4 0 2024-09-10T11:29:20.669 INFO:tasks.workunit.client.1.smithi178.stdout:1/28: truncate f1 641327 0 2024-09-10T11:29:20.669 INFO:tasks.workunit.client.1.smithi178.stdout:1/29: dread - d3/d9/fc zero size 2024-09-10T11:29:20.669 INFO:tasks.workunit.client.1.smithi178.stdout:1/30: read d3/f8 [2713625,40768] 0 2024-09-10T11:29:20.669 INFO:tasks.workunit.client.1.smithi178.stdout:5/26: mkdir d9/dc 0 2024-09-10T11:29:20.670 INFO:tasks.workunit.client.1.smithi178.stdout:1/31: creat d3/d9/fd x:0 0 0 2024-09-10T11:29:20.670 INFO:tasks.workunit.client.1.smithi178.stdout:1/32: chown d3 3 1 2024-09-10T11:29:20.671 INFO:tasks.workunit.client.1.smithi178.stdout:1/33: fdatasync d3/d9/fa 0 2024-09-10T11:29:20.671 INFO:tasks.workunit.client.1.smithi178.stdout:9/190: link d2/c6 d2/d19/d1c/d2a/c32 0 2024-09-10T11:29:20.672 INFO:tasks.workunit.client.1.smithi178.stdout:5/27: mkdir d9/dc/dd 0 2024-09-10T11:29:20.673 INFO:tasks.workunit.client.1.smithi178.stdout:5/28: write f2 [4707503,126190] 0 2024-09-10T11:29:20.674 INFO:tasks.workunit.client.1.smithi178.stdout:1/34: mkdir d3/de 0 2024-09-10T11:29:20.675 INFO:tasks.workunit.client.1.smithi178.stdout:1/35: write d3/d9/fb [674841,51175] 0 2024-09-10T11:29:20.676 INFO:tasks.workunit.client.1.smithi178.stdout:1/36: dread - d3/d9/fa zero size 2024-09-10T11:29:20.678 INFO:tasks.workunit.client.1.smithi178.stdout:6/45: rename d3/fa to d3/d7/f10 0 2024-09-10T11:29:20.682 INFO:tasks.workunit.client.1.smithi178.stdout:6/46: creat d3/f11 x:0 0 0 2024-09-10T11:29:20.682 INFO:tasks.workunit.client.1.smithi178.stdout:6/47: write d3/f11 [966949,73099] 0 2024-09-10T11:29:20.685 INFO:tasks.workunit.client.1.smithi178.stdout:9/191: rename d2/df/l14 to d2/d8/l33 0 2024-09-10T11:29:20.688 INFO:tasks.workunit.client.1.smithi178.stdout:9/192: creat d2/d8/f34 x:0 0 0 2024-09-10T11:29:20.689 INFO:tasks.workunit.client.1.smithi178.stdout:1/37: fsync d3/d9/fc 0 2024-09-10T11:29:20.690 INFO:tasks.workunit.client.1.smithi178.stdout:8/112: truncate d5/f8 3747874 0 2024-09-10T11:29:20.691 INFO:tasks.workunit.client.1.smithi178.stdout:1/38: mknod d3/d9/cf 0 2024-09-10T11:29:20.692 INFO:tasks.workunit.client.1.smithi178.stdout:1/39: write d3/d9/fa [820687,106484] 0 2024-09-10T11:29:20.699 INFO:tasks.workunit.client.1.smithi178.stdout:1/40: dwrite d3/d9/fc [0,4194304] 0 2024-09-10T11:29:20.701 INFO:tasks.workunit.client.1.smithi178.stdout:6/48: dread d3/d7/f10 [0,4194304] 0 2024-09-10T11:29:20.703 INFO:tasks.workunit.client.1.smithi178.stdout:8/113: truncate d5/db/f14 166978 0 2024-09-10T11:29:20.706 INFO:tasks.workunit.client.1.smithi178.stdout:8/114: creat d5/d1d/f25 x:0 0 0 2024-09-10T11:29:20.709 INFO:tasks.workunit.client.1.smithi178.stdout:6/49: dwrite d3/d7/d8/f6 [4194304,4194304] 0 2024-09-10T11:29:20.713 INFO:tasks.workunit.client.1.smithi178.stdout:8/115: getdents d5/db/d10 0 2024-09-10T11:29:20.715 INFO:tasks.workunit.client.1.smithi178.stdout:8/116: write d5/f7 [1757455,2248] 0 2024-09-10T11:29:20.716 INFO:tasks.workunit.client.1.smithi178.stdout:8/117: truncate d5/f1f 398371 0 2024-09-10T11:29:20.726 INFO:tasks.workunit.client.1.smithi178.stdout:6/50: dwrite f1 [0,4194304] 0 2024-09-10T11:29:20.727 INFO:tasks.workunit.client.1.smithi178.stdout:6/51: rename d3/d7 to d3/d7/d12 22 2024-09-10T11:29:20.728 INFO:tasks.workunit.client.1.smithi178.stdout:8/118: dwrite d5/ff [0,4194304] 0 2024-09-10T11:29:20.730 INFO:tasks.workunit.client.1.smithi178.stdout:6/52: truncate d3/d7/f10 74404 0 2024-09-10T11:29:20.732 INFO:tasks.workunit.client.1.smithi178.stdout:8/119: creat d5/db/d10/d20/f26 x:0 0 0 2024-09-10T11:29:20.738 INFO:tasks.workunit.client.1.smithi178.stdout:6/53: truncate f1 4151125 0 2024-09-10T11:29:20.741 INFO:tasks.workunit.client.1.smithi178.stdout:6/54: mkdir d3/d7/d8/d13 0 2024-09-10T11:29:20.741 INFO:tasks.workunit.client.1.smithi178.stdout:6/55: readlink d3/le 0 2024-09-10T11:29:20.742 INFO:tasks.workunit.client.1.smithi178.stdout:6/56: creat d3/f14 x:0 0 0 2024-09-10T11:29:20.744 INFO:tasks.workunit.client.1.smithi178.stdout:6/57: mkdir d3/d7/d8/d15 0 2024-09-10T11:29:20.745 INFO:tasks.workunit.client.1.smithi178.stdout:6/58: chown d3/d7/d8/d15 30 1 2024-09-10T11:29:20.745 INFO:tasks.workunit.client.1.smithi178.stdout:6/59: truncate d3/f14 467714 0 2024-09-10T11:29:20.747 INFO:tasks.workunit.client.1.smithi178.stdout:6/60: creat d3/d7/d8/f16 x:0 0 0 2024-09-10T11:29:20.755 INFO:tasks.workunit.client.1.smithi178.stdout:6/61: write d3/d7/ff [383006,96355] 0 2024-09-10T11:29:20.763 INFO:tasks.workunit.client.1.smithi178.stdout:6/62: dwrite d3/d7/d8/f16 [0,4194304] 0 2024-09-10T11:29:20.811 INFO:tasks.workunit.client.1.smithi178.stdout:6/63: write d3/d7/f10 [177111,48774] 0 2024-09-10T11:29:20.813 INFO:tasks.workunit.client.1.smithi178.stdout:6/64: symlink d3/d7/l17 0 2024-09-10T11:29:20.819 INFO:tasks.workunit.client.1.smithi178.stdout:6/65: dwrite d3/d7/d8/f6 [0,4194304] 0 2024-09-10T11:29:20.821 INFO:tasks.workunit.client.1.smithi178.stdout:6/66: mkdir d3/d7/d8/d13/d18 0 2024-09-10T11:29:20.822 INFO:tasks.workunit.client.1.smithi178.stdout:6/67: write d3/f5 [6092270,57578] 0 2024-09-10T11:29:20.824 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:20 smithi144 ceph-mon[27942]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:29:20.824 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:20 smithi144 ceph-mon[27942]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:29:20.824 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:20 smithi144 ceph-mon[27942]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:29:20.824 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:20 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.824 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:20 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.824 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:20 smithi144 ceph-mon[27942]: pgmap v17: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.8 MiB/s rd, 22 MiB/s wr, 501 op/s 2024-09-10T11:29:20.824 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:20 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.824 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:20 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.824 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:20 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.824 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:20 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.824 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:20 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.825 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:20 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.825 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:20 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.825 INFO:tasks.workunit.client.1.smithi178.stdout:6/68: unlink d3/f5 0 2024-09-10T11:29:20.826 INFO:tasks.workunit.client.1.smithi178.stdout:6/69: chown d3/cc 201029867 1 2024-09-10T11:29:20.827 INFO:tasks.workunit.client.1.smithi178.stdout:4/6: sync 2024-09-10T11:29:20.827 INFO:tasks.workunit.client.1.smithi178.stdout:4/7: dwrite - no filename 2024-09-10T11:29:20.827 INFO:tasks.workunit.client.1.smithi178.stdout:4/8: dread - no filename 2024-09-10T11:29:20.828 INFO:tasks.workunit.client.1.smithi178.stdout:6/70: readlink d3/d7/d8/ld 0 2024-09-10T11:29:20.828 INFO:tasks.workunit.client.1.smithi178.stdout:6/71: stat f1 0 2024-09-10T11:29:20.830 INFO:tasks.workunit.client.1.smithi178.stdout:6/72: mkdir d3/d7/d8/d19 0 2024-09-10T11:29:20.831 INFO:tasks.workunit.client.1.smithi178.stdout:4/9: link d0/l1 d0/l3 0 2024-09-10T11:29:20.836 INFO:tasks.workunit.client.1.smithi178.stdout:4/10: link d0/l3 d0/l4 0 2024-09-10T11:29:20.836 INFO:tasks.workunit.client.1.smithi178.stdout:4/11: write - no filename 2024-09-10T11:29:20.838 INFO:tasks.workunit.client.1.smithi178.stdout:4/12: creat d0/f5 x:0 0 0 2024-09-10T11:29:20.839 INFO:tasks.workunit.client.1.smithi178.stdout:4/13: truncate d0/f5 712389 0 2024-09-10T11:29:20.840 INFO:tasks.workunit.client.1.smithi178.stdout:4/14: dread d0/f5 [0,4194304] 0 2024-09-10T11:29:20.841 INFO:tasks.workunit.client.1.smithi178.stdout:4/15: dread d0/f5 [0,4194304] 0 2024-09-10T11:29:20.843 INFO:tasks.workunit.client.1.smithi178.stdout:4/16: write d0/f5 [359485,98567] 0 2024-09-10T11:29:20.843 INFO:tasks.workunit.client.1.smithi178.stdout:6/73: truncate d3/d7/ff 929913 0 2024-09-10T11:29:20.845 INFO:tasks.workunit.client.1.smithi178.stdout:4/17: mknod d0/c6 0 2024-09-10T11:29:20.852 INFO:tasks.workunit.client.1.smithi178.stdout:4/18: dwrite d0/f5 [0,4194304] 0 2024-09-10T11:29:20.853 INFO:tasks.workunit.client.1.smithi178.stdout:4/19: chown d0/c6 0 1 2024-09-10T11:29:20.853 INFO:tasks.workunit.client.1.smithi178.stdout:6/74: link d3/d7/d8/ld d3/d7/d8/d19/l1a 0 2024-09-10T11:29:20.857 INFO:tasks.workunit.client.1.smithi178.stdout:6/75: symlink d3/d7/d8/d13/d18/l1b 0 2024-09-10T11:29:20.860 INFO:tasks.workunit.client.1.smithi178.stdout:4/20: dwrite d0/f5 [0,4194304] 0 2024-09-10T11:29:20.862 INFO:tasks.workunit.client.1.smithi178.stdout:6/76: truncate d3/d7/ff 1924647 0 2024-09-10T11:29:20.864 INFO:tasks.workunit.client.1.smithi178.stdout:4/21: symlink d0/l7 0 2024-09-10T11:29:20.866 INFO:tasks.workunit.client.1.smithi178.stdout:4/22: mkdir d0/d8 0 2024-09-10T11:29:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:20 smithi178 ceph-mon[33514]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:29:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:20 smithi178 ceph-mon[33514]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:29:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:20 smithi178 ceph-mon[33514]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:29:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:20 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:20 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:20 smithi178 ceph-mon[33514]: pgmap v17: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.8 MiB/s rd, 22 MiB/s wr, 501 op/s 2024-09-10T11:29:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:20 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:20 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:20 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:20 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:20 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:20 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:20 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:20.868 INFO:tasks.workunit.client.1.smithi178.stdout:6/77: mkdir d3/d7/d8/d15/d1c 0 2024-09-10T11:29:20.869 INFO:tasks.workunit.client.1.smithi178.stdout:6/78: write d3/f11 [1058105,60213] 0 2024-09-10T11:29:20.875 INFO:tasks.workunit.client.1.smithi178.stdout:4/23: dwrite d0/f5 [0,4194304] 0 2024-09-10T11:29:20.878 INFO:tasks.workunit.client.1.smithi178.stdout:4/24: write d0/f5 [1927172,42945] 0 2024-09-10T11:29:20.881 INFO:tasks.workunit.client.1.smithi178.stdout:4/25: mkdir d0/d9 0 2024-09-10T11:29:20.881 INFO:tasks.workunit.client.1.smithi178.stdout:6/79: dwrite d3/d7/d8/f16 [0,4194304] 0 2024-09-10T11:29:20.882 INFO:tasks.workunit.client.1.smithi178.stdout:4/26: write d0/f5 [158449,25007] 0 2024-09-10T11:29:20.890 INFO:tasks.workunit.client.1.smithi178.stdout:4/27: dread d0/f5 [0,4194304] 0 2024-09-10T11:29:20.891 INFO:tasks.workunit.client.1.smithi178.stdout:6/80: dwrite d3/f14 [0,4194304] 0 2024-09-10T11:29:20.893 INFO:tasks.workunit.client.1.smithi178.stdout:4/28: mkdir d0/d8/da 0 2024-09-10T11:29:20.894 INFO:tasks.workunit.client.1.smithi178.stdout:6/81: creat d3/d7/d8/d15/d1c/f1d x:0 0 0 2024-09-10T11:29:20.897 INFO:tasks.workunit.client.1.smithi178.stdout:6/82: symlink d3/d7/d8/d13/d18/l1e 0 2024-09-10T11:29:20.899 INFO:tasks.workunit.client.1.smithi178.stdout:6/83: rmdir d3/d7/d8/d19 39 2024-09-10T11:29:20.905 INFO:tasks.workunit.client.1.smithi178.stdout:6/84: link d3/d7/ff d3/d7/d8/f1f 0 2024-09-10T11:29:20.906 INFO:tasks.workunit.client.1.smithi178.stdout:6/85: chown d3/d7/d8/d15 149271 1 2024-09-10T11:29:20.907 INFO:tasks.workunit.client.1.smithi178.stdout:4/29: truncate d0/f5 760585 0 2024-09-10T11:29:20.910 INFO:tasks.workunit.client.1.smithi178.stdout:4/30: creat d0/d9/fb x:0 0 0 2024-09-10T11:29:20.911 INFO:tasks.workunit.client.1.smithi178.stdout:4/31: fdatasync d0/f5 0 2024-09-10T11:29:20.912 INFO:tasks.workunit.client.1.smithi178.stdout:4/32: chown d0/d9 13524 1 2024-09-10T11:29:20.912 INFO:tasks.workunit.client.1.smithi178.stdout:6/86: dwrite d3/f14 [0,4194304] 0 2024-09-10T11:29:20.914 INFO:tasks.workunit.client.1.smithi178.stdout:4/33: readlink d0/l4 0 2024-09-10T11:29:20.914 INFO:tasks.workunit.client.1.smithi178.stdout:4/34: chown d0/d8/da 846 1 2024-09-10T11:29:20.915 INFO:tasks.workunit.client.1.smithi178.stdout:4/35: write d0/f5 [926541,54406] 0 2024-09-10T11:29:20.916 INFO:tasks.workunit.client.1.smithi178.stdout:4/36: read - d0/d9/fb zero size 2024-09-10T11:29:20.917 INFO:tasks.workunit.client.1.smithi178.stdout:6/87: unlink d3/d7/d8/f16 0 2024-09-10T11:29:20.917 INFO:tasks.workunit.client.1.smithi178.stdout:4/37: write d0/d9/fb [57622,112717] 0 2024-09-10T11:29:20.932 INFO:tasks.workunit.client.1.smithi178.stdout:6/88: dwrite d3/d7/d8/f1f [0,4194304] 0 2024-09-10T11:29:20.936 INFO:tasks.workunit.client.1.smithi178.stdout:6/89: rmdir d3 39 2024-09-10T11:29:20.941 INFO:tasks.workunit.client.1.smithi178.stdout:6/90: mkdir d3/d7/d8/d15/d1c/d20 0 2024-09-10T11:29:20.945 INFO:tasks.workunit.client.1.smithi178.stdout:6/91: rename d3/d7/d8/f6 to d3/d7/d8/d19/f21 0 2024-09-10T11:29:20.947 INFO:tasks.workunit.client.1.smithi178.stdout:6/92: creat d3/d7/d8/d15/f22 x:0 0 0 2024-09-10T11:29:20.951 INFO:tasks.workunit.client.1.smithi178.stdout:6/93: link d3/d7/d8/ld d3/d7/d8/l23 0 2024-09-10T11:29:20.956 INFO:tasks.workunit.client.1.smithi178.stdout:6/94: write f0 [1588305,75135] 0 2024-09-10T11:29:20.957 INFO:tasks.workunit.client.1.smithi178.stdout:6/95: rmdir d3/d7/d8/d13 39 2024-09-10T11:29:20.959 INFO:tasks.workunit.client.1.smithi178.stdout:6/96: mkdir d3/d24 0 2024-09-10T11:29:20.961 INFO:tasks.workunit.client.1.smithi178.stdout:6/97: rename d3/d7/f10 to d3/d7/d8/d15/f25 0 2024-09-10T11:29:20.966 INFO:tasks.workunit.client.1.smithi178.stdout:6/98: dread d3/d7/d8/f1f [0,4194304] 0 2024-09-10T11:29:20.973 INFO:tasks.workunit.client.1.smithi178.stdout:6/99: dwrite d3/fb [0,4194304] 0 2024-09-10T11:29:20.979 INFO:tasks.workunit.client.1.smithi178.stdout:6/100: dread d3/d7/d8/d19/f21 [0,4194304] 0 2024-09-10T11:29:20.987 INFO:tasks.workunit.client.1.smithi178.stdout:6/101: dwrite d3/fb [0,4194304] 0 2024-09-10T11:29:20.990 INFO:tasks.workunit.client.1.smithi178.stdout:6/102: creat d3/d7/d8/d19/f26 x:0 0 0 2024-09-10T11:29:20.997 INFO:tasks.workunit.client.1.smithi178.stdout:6/103: link d3/d7/d8/d13/d18/l1b d3/d7/d8/d15/d1c/d20/l27 0 2024-09-10T11:29:21.002 INFO:tasks.workunit.client.1.smithi178.stdout:6/104: dwrite f0 [4194304,4194304] 0 2024-09-10T11:29:21.006 INFO:tasks.workunit.client.1.smithi178.stdout:6/105: creat d3/d7/d8/d15/d1c/d20/f28 x:0 0 0 2024-09-10T11:29:21.008 INFO:tasks.workunit.client.1.smithi178.stdout:6/106: chown d3/d7/d8/d13/d18/l1e 1174 1 2024-09-10T11:29:21.010 INFO:tasks.workunit.client.1.smithi178.stdout:6/107: creat d3/f29 x:0 0 0 2024-09-10T11:29:21.011 INFO:tasks.workunit.client.1.smithi178.stdout:6/108: write d3/f29 [20123,19409] 0 2024-09-10T11:29:21.019 INFO:tasks.workunit.client.1.smithi178.stdout:6/109: rename d3/d7/d8/d13 to d3/d7/d8/d15/d1c/d2a 0 2024-09-10T11:29:21.022 INFO:tasks.workunit.client.1.smithi178.stdout:6/110: symlink d3/d7/d8/d19/l2b 0 2024-09-10T11:29:21.022 INFO:tasks.workunit.client.1.smithi178.stdout:6/111: dread - d3/d7/d8/d15/f22 zero size 2024-09-10T11:29:21.024 INFO:tasks.workunit.client.1.smithi178.stdout:6/112: write d3/f29 [183710,19314] 0 2024-09-10T11:29:21.025 INFO:tasks.workunit.client.1.smithi178.stdout:6/113: chown d3 38 1 2024-09-10T11:29:21.026 INFO:tasks.workunit.client.1.smithi178.stdout:3/69: sync 2024-09-10T11:29:21.028 INFO:tasks.workunit.client.1.smithi178.stdout:6/114: creat d3/d7/d8/d15/d1c/d2a/f2c x:0 0 0 2024-09-10T11:29:21.029 INFO:tasks.workunit.client.1.smithi178.stdout:3/70: fdatasync db/f10 0 2024-09-10T11:29:21.030 INFO:tasks.workunit.client.1.smithi178.stdout:6/115: read d3/d7/d8/d19/f21 [3226191,50376] 0 2024-09-10T11:29:21.034 INFO:tasks.workunit.client.1.smithi178.stdout:3/71: dread db/f10 [0,4194304] 0 2024-09-10T11:29:21.035 INFO:tasks.workunit.client.1.smithi178.stdout:6/116: chown d3/d7/d8/d15/d1c 200646960 1 2024-09-10T11:29:21.035 INFO:tasks.workunit.client.1.smithi178.stdout:3/72: rename db to db/d19 22 2024-09-10T11:29:21.036 INFO:tasks.workunit.client.1.smithi178.stdout:3/73: fdatasync f7 0 2024-09-10T11:29:21.036 INFO:tasks.workunit.client.1.smithi178.stdout:3/74: chown la 12 1 2024-09-10T11:29:21.037 INFO:tasks.workunit.client.1.smithi178.stdout:3/75: dread - db/f15 zero size 2024-09-10T11:29:21.038 INFO:tasks.workunit.client.1.smithi178.stdout:6/117: truncate d3/d7/d8/d15/d1c/d2a/f2c 777462 0 2024-09-10T11:29:21.040 INFO:tasks.workunit.client.1.smithi178.stdout:6/118: readlink d3/d7/d8/d19/l1a 0 2024-09-10T11:29:21.118 INFO:tasks.workunit.client.1.smithi178.stdout:3/76: dwrite db/f18 [0,4194304] 0 2024-09-10T11:29:21.120 INFO:tasks.workunit.client.1.smithi178.stdout:6/119: dread f1 [0,4194304] 0 2024-09-10T11:29:21.121 INFO:tasks.workunit.client.1.smithi178.stdout:3/77: creat db/f1a x:0 0 0 2024-09-10T11:29:21.121 INFO:tasks.workunit.client.1.smithi178.stdout:6/120: chown d3/d7/d8 193431973 1 2024-09-10T11:29:21.123 INFO:tasks.workunit.client.1.smithi178.stdout:3/78: mkdir db/d1b 0 2024-09-10T11:29:21.125 INFO:tasks.workunit.client.1.smithi178.stdout:3/79: chown l9 11 1 2024-09-10T11:29:21.125 INFO:tasks.workunit.client.1.smithi178.stdout:6/121: rename d3/f29 to d3/d7/d8/d15/d1c/d20/f2d 0 2024-09-10T11:29:21.127 INFO:tasks.workunit.client.1.smithi178.stdout:3/80: symlink db/l1c 0 2024-09-10T11:29:21.130 INFO:tasks.workunit.client.1.smithi178.stdout:6/122: dread d3/d7/ff [0,4194304] 0 2024-09-10T11:29:21.141 INFO:tasks.workunit.client.1.smithi178.stdout:6/123: dwrite f1 [0,4194304] 0 2024-09-10T11:29:21.142 INFO:tasks.workunit.client.1.smithi178.stdout:3/81: dwrite db/f15 [0,4194304] 0 2024-09-10T11:29:21.144 INFO:tasks.workunit.client.1.smithi178.stdout:6/124: rmdir d3/d7 39 2024-09-10T11:29:21.146 INFO:tasks.workunit.client.1.smithi178.stdout:3/82: creat db/f1d x:0 0 0 2024-09-10T11:29:21.152 INFO:tasks.workunit.client.1.smithi178.stdout:3/83: truncate db/ff 801706 0 2024-09-10T11:29:21.157 INFO:tasks.workunit.client.1.smithi178.stdout:3/84: mknod db/c1e 0 2024-09-10T11:29:21.158 INFO:tasks.workunit.client.1.smithi178.stdout:3/85: chown db/f18 8142917 1 2024-09-10T11:29:21.159 INFO:tasks.workunit.client.1.smithi178.stdout:6/125: dwrite d3/d7/d8/d15/d1c/d20/f28 [0,4194304] 0 2024-09-10T11:29:21.159 INFO:tasks.workunit.client.1.smithi178.stdout:3/86: chown db/l1c 3197 1 2024-09-10T11:29:21.165 INFO:tasks.workunit.client.1.smithi178.stdout:3/87: dread db/f18 [0,4194304] 0 2024-09-10T11:29:21.167 INFO:tasks.workunit.client.1.smithi178.stdout:6/126: link d3/f14 d3/d24/f2e 0 2024-09-10T11:29:21.168 INFO:tasks.workunit.client.1.smithi178.stdout:3/88: rename db/l11 to db/l1f 0 2024-09-10T11:29:21.169 INFO:tasks.workunit.client.1.smithi178.stdout:3/89: write db/f15 [4844278,95011] 0 2024-09-10T11:29:21.170 INFO:tasks.workunit.client.1.smithi178.stdout:6/127: mkdir d3/d7/d8/d15/d2f 0 2024-09-10T11:29:21.171 INFO:tasks.workunit.client.1.smithi178.stdout:6/128: write d3/d7/ff [1171641,116346] 0 2024-09-10T11:29:21.176 INFO:tasks.workunit.client.1.smithi178.stdout:6/129: unlink d3/d7/d8/d15/d1c/d2a/d18/l1e 0 2024-09-10T11:29:21.179 INFO:tasks.workunit.client.1.smithi178.stdout:6/130: rename d3/d7/d8/d19/f21 to d3/d24/f30 0 2024-09-10T11:29:21.189 INFO:tasks.workunit.client.1.smithi178.stdout:3/90: getdents db 0 2024-09-10T11:29:21.190 INFO:tasks.workunit.client.1.smithi178.stdout:3/91: fdatasync db/f16 0 2024-09-10T11:29:21.192 INFO:tasks.workunit.client.1.smithi178.stdout:6/131: dwrite d3/d7/d8/d19/f26 [0,4194304] 0 2024-09-10T11:29:21.194 INFO:tasks.workunit.client.1.smithi178.stdout:7/26: sync 2024-09-10T11:29:21.194 INFO:tasks.workunit.client.1.smithi178.stdout:3/92: write f8 [3119484,96290] 0 2024-09-10T11:29:21.195 INFO:tasks.workunit.client.1.smithi178.stdout:0/39: sync 2024-09-10T11:29:21.195 INFO:tasks.workunit.client.1.smithi178.stdout:3/93: readlink l3 0 2024-09-10T11:29:21.204 INFO:tasks.workunit.client.1.smithi178.stdout:6/132: rename d3/le to d3/d7/d8/d15/l31 0 2024-09-10T11:29:21.206 INFO:tasks.workunit.client.1.smithi178.stdout:6/133: rmdir d3/d24 39 2024-09-10T11:29:21.207 INFO:tasks.workunit.client.1.smithi178.stdout:0/40: rename d0/f2 to d0/f7 0 2024-09-10T11:29:21.208 INFO:tasks.workunit.client.1.smithi178.stdout:3/94: rmdir db/d1b 0 2024-09-10T11:29:21.209 INFO:tasks.workunit.client.1.smithi178.stdout:3/95: truncate db/ff 1576864 0 2024-09-10T11:29:21.210 INFO:tasks.workunit.client.1.smithi178.stdout:3/96: fdatasync db/fe 0 2024-09-10T11:29:21.211 INFO:tasks.workunit.client.1.smithi178.stdout:6/134: rmdir d3/d7/d8/d15/d2f 0 2024-09-10T11:29:21.212 INFO:tasks.workunit.client.1.smithi178.stdout:6/135: chown d3/f11 10 1 2024-09-10T11:29:21.213 INFO:tasks.workunit.client.1.smithi178.stdout:0/41: rmdir d0 39 2024-09-10T11:29:21.215 INFO:tasks.workunit.client.1.smithi178.stdout:3/97: mkdir db/d20 0 2024-09-10T11:29:21.219 INFO:tasks.workunit.client.1.smithi178.stdout:3/98: creat db/d20/f21 x:0 0 0 2024-09-10T11:29:21.222 INFO:tasks.workunit.client.1.smithi178.stdout:3/99: symlink db/d20/l22 0 2024-09-10T11:29:21.224 INFO:tasks.workunit.client.1.smithi178.stdout:3/100: stat db/f16 0 2024-09-10T11:29:21.225 INFO:tasks.workunit.client.1.smithi178.stdout:6/136: dwrite d3/d7/d8/d15/f22 [0,4194304] 0 2024-09-10T11:29:21.233 INFO:tasks.workunit.client.1.smithi178.stdout:0/42: dwrite d0/f7 [0,4194304] 0 2024-09-10T11:29:21.233 INFO:tasks.workunit.client.1.smithi178.stdout:3/101: truncate db/f15 1986202 0 2024-09-10T11:29:21.235 INFO:tasks.workunit.client.1.smithi178.stdout:3/102: truncate db/f1d 108460 0 2024-09-10T11:29:21.236 INFO:tasks.workunit.client.1.smithi178.stdout:6/137: dread f0 [4194304,4194304] 0 2024-09-10T11:29:21.238 INFO:tasks.workunit.client.1.smithi178.stdout:3/103: write db/d20/f21 [604536,129540] 0 2024-09-10T11:29:21.238 INFO:tasks.workunit.client.1.smithi178.stdout:0/43: creat d0/f8 x:0 0 0 2024-09-10T11:29:21.241 INFO:tasks.workunit.client.1.smithi178.stdout:0/44: creat d0/f9 x:0 0 0 2024-09-10T11:29:21.244 INFO:tasks.workunit.client.1.smithi178.stdout:0/45: mknod d0/ca 0 2024-09-10T11:29:21.245 INFO:tasks.workunit.client.1.smithi178.stdout:6/138: dread d3/d24/f30 [4194304,4194304] 0 2024-09-10T11:29:21.245 INFO:tasks.workunit.client.1.smithi178.stdout:0/46: mkdir d0/db 0 2024-09-10T11:29:21.247 INFO:tasks.workunit.client.1.smithi178.stdout:6/139: mkdir d3/d7/d32 0 2024-09-10T11:29:21.248 INFO:tasks.workunit.client.1.smithi178.stdout:0/47: mkdir d0/dc 0 2024-09-10T11:29:21.248 INFO:tasks.workunit.client.1.smithi178.stdout:0/48: chown d0/f9 14532 1 2024-09-10T11:29:21.249 INFO:tasks.workunit.client.1.smithi178.stdout:6/140: write d3/d7/d8/d15/d1c/d20/f28 [4647396,53221] 0 2024-09-10T11:29:21.249 INFO:tasks.workunit.client.1.smithi178.stdout:0/49: dread - d0/f6 zero size 2024-09-10T11:29:21.252 INFO:tasks.workunit.client.1.smithi178.stdout:0/50: write d0/f6 [819983,40558] 0 2024-09-10T11:29:21.252 INFO:tasks.workunit.client.1.smithi178.stdout:6/141: mknod d3/d7/d8/d15/c33 0 2024-09-10T11:29:21.255 INFO:tasks.workunit.client.1.smithi178.stdout:0/51: read d0/f6 [7991,15891] 0 2024-09-10T11:29:21.256 INFO:tasks.workunit.client.1.smithi178.stdout:6/142: mknod d3/d24/c34 0 2024-09-10T11:29:21.258 INFO:tasks.workunit.client.1.smithi178.stdout:0/52: write d0/f1 [21632,100155] 0 2024-09-10T11:29:21.258 INFO:tasks.workunit.client.1.smithi178.stdout:6/143: mkdir d3/d7/d35 0 2024-09-10T11:29:21.260 INFO:tasks.workunit.client.1.smithi178.stdout:6/144: mknod d3/d7/c36 0 2024-09-10T11:29:21.264 INFO:tasks.workunit.client.1.smithi178.stdout:6/145: creat d3/f37 x:0 0 0 2024-09-10T11:29:21.266 INFO:tasks.workunit.client.1.smithi178.stdout:6/146: write d3/d7/d8/d15/d1c/d20/f2d [433665,128409] 0 2024-09-10T11:29:21.266 INFO:tasks.workunit.client.1.smithi178.stdout:0/53: truncate d0/f1 696224 0 2024-09-10T11:29:21.267 INFO:tasks.workunit.client.1.smithi178.stdout:0/54: rename d0/db to d0/db/dd 22 2024-09-10T11:29:21.269 INFO:tasks.workunit.client.1.smithi178.stdout:0/55: truncate d0/f7 4332470 0 2024-09-10T11:29:21.269 INFO:tasks.workunit.client.1.smithi178.stdout:0/56: stat d0/c4 0 2024-09-10T11:29:21.269 INFO:tasks.workunit.client.1.smithi178.stdout:6/147: creat d3/d7/d8/d15/d1c/d2a/d18/f38 x:0 0 0 2024-09-10T11:29:21.270 INFO:tasks.workunit.client.1.smithi178.stdout:6/148: dread - d3/d7/d8/d15/d1c/f1d zero size 2024-09-10T11:29:21.271 INFO:tasks.workunit.client.1.smithi178.stdout:0/57: write d0/f7 [1917639,32331] 0 2024-09-10T11:29:21.274 INFO:tasks.workunit.client.1.smithi178.stdout:6/149: rename d3/f11 to d3/d7/d8/d19/f39 0 2024-09-10T11:29:21.274 INFO:tasks.workunit.client.1.smithi178.stdout:0/58: creat d0/dc/fe x:0 0 0 2024-09-10T11:29:21.276 INFO:tasks.workunit.client.1.smithi178.stdout:0/59: creat d0/dc/ff x:0 0 0 2024-09-10T11:29:21.277 INFO:tasks.workunit.client.1.smithi178.stdout:0/60: read - d0/f8 zero size 2024-09-10T11:29:21.277 INFO:tasks.workunit.client.1.smithi178.stdout:6/150: creat d3/f3a x:0 0 0 2024-09-10T11:29:21.281 INFO:tasks.workunit.client.1.smithi178.stdout:0/61: symlink d0/dc/l10 0 2024-09-10T11:29:21.281 INFO:tasks.workunit.client.1.smithi178.stdout:0/62: readlink d0/dc/l10 0 2024-09-10T11:29:21.281 INFO:tasks.workunit.client.1.smithi178.stdout:6/151: mknod d3/d7/d35/c3b 0 2024-09-10T11:29:21.281 INFO:tasks.workunit.client.1.smithi178.stdout:6/152: write d3/f37 [913808,46497] 0 2024-09-10T11:29:21.281 INFO:tasks.workunit.client.1.smithi178.stdout:0/63: mknod d0/dc/c11 0 2024-09-10T11:29:21.283 INFO:tasks.workunit.client.1.smithi178.stdout:0/64: mkdir d0/d12 0 2024-09-10T11:29:21.285 INFO:tasks.workunit.client.1.smithi178.stdout:6/153: link d3/fb d3/d7/d8/d15/d1c/d2a/f3c 0 2024-09-10T11:29:21.285 INFO:tasks.workunit.client.1.smithi178.stdout:0/65: read d0/f6 [466999,111911] 0 2024-09-10T11:29:21.289 INFO:tasks.workunit.client.1.smithi178.stdout:0/66: rename d0/dc/fe to d0/dc/f13 0 2024-09-10T11:29:21.291 INFO:tasks.workunit.client.1.smithi178.stdout:0/67: mkdir d0/db/d14 0 2024-09-10T11:29:21.292 INFO:tasks.workunit.client.1.smithi178.stdout:0/68: chown d0/c4 28688228 1 2024-09-10T11:29:21.293 INFO:tasks.workunit.client.1.smithi178.stdout:6/154: getdents d3 0 2024-09-10T11:29:21.295 INFO:tasks.workunit.client.1.smithi178.stdout:6/155: symlink d3/d7/d35/l3d 0 2024-09-10T11:29:21.296 INFO:tasks.workunit.client.1.smithi178.stdout:6/156: truncate d3/f37 1908180 0 2024-09-10T11:29:21.298 INFO:tasks.workunit.client.1.smithi178.stdout:6/157: fsync d3/fb 0 2024-09-10T11:29:21.303 INFO:tasks.workunit.client.1.smithi178.stdout:6/158: rmdir d3/d7/d32 0 2024-09-10T11:29:21.306 INFO:tasks.workunit.client.1.smithi178.stdout:6/159: rename d3/cc to d3/d7/c3e 0 2024-09-10T11:29:21.317 INFO:tasks.workunit.client.1.smithi178.stdout:6/160: dwrite d3/d7/ff [0,4194304] 0 2024-09-10T11:29:21.324 INFO:tasks.workunit.client.1.smithi178.stdout:6/161: truncate d3/d7/d8/d15/f25 151148 0 2024-09-10T11:29:21.331 INFO:tasks.workunit.client.1.smithi178.stdout:6/162: dwrite d3/d7/d8/d15/d1c/d2a/f2c [0,4194304] 0 2024-09-10T11:29:21.334 INFO:tasks.workunit.client.1.smithi178.stdout:6/163: read d3/d7/d8/d15/d1c/d20/f2d [327395,29642] 0 2024-09-10T11:29:21.336 INFO:tasks.workunit.client.1.smithi178.stdout:6/164: truncate d3/f3a 507269 0 2024-09-10T11:29:21.338 INFO:tasks.workunit.client.1.smithi178.stdout:6/165: mknod d3/d7/d8/d19/c3f 0 2024-09-10T11:29:21.342 INFO:tasks.workunit.client.1.smithi178.stdout:6/166: truncate d3/f14 164549 0 2024-09-10T11:29:21.346 INFO:tasks.workunit.client.1.smithi178.stdout:6/167: dread d3/d7/d8/d15/f22 [0,4194304] 0 2024-09-10T11:29:21.350 INFO:tasks.workunit.client.1.smithi178.stdout:6/168: rename d3/d7/ff to d3/d7/d8/d15/d1c/f40 0 2024-09-10T11:29:21.353 INFO:tasks.workunit.client.1.smithi178.stdout:6/169: creat d3/d7/d8/d15/d1c/d2a/f41 x:0 0 0 2024-09-10T11:29:21.353 INFO:tasks.workunit.client.1.smithi178.stdout:6/170: stat d3/d7/l17 0 2024-09-10T11:29:21.355 INFO:tasks.workunit.client.1.smithi178.stdout:6/171: write f0 [3239034,39063] 0 2024-09-10T11:29:21.359 INFO:tasks.workunit.client.1.smithi178.stdout:6/172: getdents d3 0 2024-09-10T11:29:21.361 INFO:tasks.workunit.client.1.smithi178.stdout:6/173: mkdir d3/d42 0 2024-09-10T11:29:21.365 INFO:tasks.workunit.client.1.smithi178.stdout:6/174: chown d3/d7/d8/d15/d1c/d20/f2d 67863 1 2024-09-10T11:29:21.367 INFO:tasks.workunit.client.1.smithi178.stdout:6/175: rmdir d3/d7/d8/d19 39 2024-09-10T11:29:21.371 INFO:tasks.workunit.client.1.smithi178.stdout:6/176: dread d3/f37 [0,4194304] 0 2024-09-10T11:29:21.372 INFO:tasks.workunit.client.1.smithi178.stdout:6/177: chown d3/d7/d8/d15/d1c/d20/f28 21 1 2024-09-10T11:29:21.378 INFO:tasks.workunit.client.1.smithi178.stdout:6/178: dwrite d3/d7/d8/d15/d1c/d20/f28 [0,4194304] 0 2024-09-10T11:29:21.381 INFO:tasks.workunit.client.1.smithi178.stdout:6/179: creat d3/d7/d8/d15/d1c/d2a/d18/f43 x:0 0 0 2024-09-10T11:29:21.383 INFO:tasks.workunit.client.1.smithi178.stdout:6/180: write d3/d7/d8/d15/d1c/f1d [68070,118062] 0 2024-09-10T11:29:21.388 INFO:tasks.workunit.client.1.smithi178.stdout:6/181: dread d3/d7/d8/d15/f22 [0,4194304] 0 2024-09-10T11:29:21.390 INFO:tasks.workunit.client.1.smithi178.stdout:6/182: mknod d3/d7/c44 0 2024-09-10T11:29:21.392 INFO:tasks.workunit.client.1.smithi178.stdout:6/183: rename d3/f37 to d3/d7/d8/d15/d1c/f45 0 2024-09-10T11:29:21.399 INFO:tasks.workunit.client.1.smithi178.stdout:9/193: sync 2024-09-10T11:29:21.399 INFO:tasks.workunit.client.1.smithi178.stdout:5/29: sync 2024-09-10T11:29:21.399 INFO:tasks.workunit.client.1.smithi178.stdout:2/86: sync 2024-09-10T11:29:21.400 INFO:tasks.workunit.client.1.smithi178.stdout:2/87: chown d1/d7/d10/d14/d11 1 1 2024-09-10T11:29:21.401 INFO:tasks.workunit.client.1.smithi178.stdout:6/184: mknod d3/d7/d35/c46 0 2024-09-10T11:29:21.401 INFO:tasks.workunit.client.1.smithi178.stdout:9/194: write d2/f26 [1350425,98998] 0 2024-09-10T11:29:21.403 INFO:tasks.workunit.client.1.smithi178.stdout:5/30: mknod d9/dc/dd/ce 0 2024-09-10T11:29:21.403 INFO:tasks.workunit.client.1.smithi178.stdout:2/88: rmdir d1/d7/d10/d14/d11 39 2024-09-10T11:29:21.405 INFO:tasks.workunit.client.1.smithi178.stdout:5/31: write f4 [5072025,37768] 0 2024-09-10T11:29:21.405 INFO:tasks.workunit.client.1.smithi178.stdout:6/185: creat d3/d7/d8/d15/d1c/d20/f47 x:0 0 0 2024-09-10T11:29:21.408 INFO:tasks.workunit.client.1.smithi178.stdout:5/32: creat d9/ff x:0 0 0 2024-09-10T11:29:21.409 INFO:tasks.workunit.client.1.smithi178.stdout:6/186: mkdir d3/d7/d8/d15/d48 0 2024-09-10T11:29:21.411 INFO:tasks.workunit.client.1.smithi178.stdout:9/195: dwrite d2/d13/f2e [0,4194304] 0 2024-09-10T11:29:21.411 INFO:tasks.workunit.client.1.smithi178.stdout:2/89: rename d1/d7/df to d1/d7/d10/d14/d11/d19 0 2024-09-10T11:29:21.416 INFO:tasks.workunit.client.1.smithi178.stdout:2/90: rmdir d1/d12 39 2024-09-10T11:29:21.420 INFO:tasks.workunit.client.1.smithi178.stdout:6/187: dwrite d3/d7/d8/d15/f22 [0,4194304] 0 2024-09-10T11:29:21.421 INFO:tasks.workunit.client.1.smithi178.stdout:2/91: write d1/f5 [3758911,9155] 0 2024-09-10T11:29:21.422 INFO:tasks.workunit.client.1.smithi178.stdout:5/33: link l3 d9/l10 0 2024-09-10T11:29:21.423 INFO:tasks.workunit.client.1.smithi178.stdout:9/196: write d2/df/f1b [1431729,18887] 0 2024-09-10T11:29:21.424 INFO:tasks.workunit.client.1.smithi178.stdout:5/34: mkdir d9/d11 0 2024-09-10T11:29:21.426 INFO:tasks.workunit.client.1.smithi178.stdout:9/197: rmdir d2/d19 39 2024-09-10T11:29:21.426 INFO:tasks.workunit.client.1.smithi178.stdout:5/35: chown d9/dc/dd/ce 14504 1 2024-09-10T11:29:21.427 INFO:tasks.workunit.client.1.smithi178.stdout:2/92: creat d1/d7/d10/d14/f1a x:0 0 0 2024-09-10T11:29:21.430 INFO:tasks.workunit.client.1.smithi178.stdout:2/93: truncate d1/d12/f17 52731 0 2024-09-10T11:29:21.431 INFO:tasks.workunit.client.1.smithi178.stdout:5/36: creat d9/dc/f12 x:0 0 0 2024-09-10T11:29:21.435 INFO:tasks.workunit.client.1.smithi178.stdout:6/188: write d3/fb [679590,66810] 0 2024-09-10T11:29:21.437 INFO:tasks.workunit.client.1.smithi178.stdout:5/37: rename d9/l10 to d9/d11/l13 0 2024-09-10T11:29:21.437 INFO:tasks.workunit.client.1.smithi178.stdout:9/198: dread d2/dc/f2f [0,4194304] 0 2024-09-10T11:29:21.438 INFO:tasks.workunit.client.1.smithi178.stdout:2/94: creat d1/d7/d10/d14/f1b x:0 0 0 2024-09-10T11:29:21.439 INFO:tasks.workunit.client.1.smithi178.stdout:2/95: chown d1 500 1 2024-09-10T11:29:21.440 INFO:tasks.workunit.client.1.smithi178.stdout:2/96: write d1/d7/f16 [926089,105199] 0 2024-09-10T11:29:21.441 INFO:tasks.workunit.client.1.smithi178.stdout:5/38: rename d9/dc/f12 to d9/d11/f14 0 2024-09-10T11:29:21.443 INFO:tasks.workunit.client.1.smithi178.stdout:2/97: symlink d1/d7/l1c 0 2024-09-10T11:29:21.444 INFO:tasks.workunit.client.1.smithi178.stdout:2/98: write d1/f5 [1808855,14986] 0 2024-09-10T11:29:21.446 INFO:tasks.workunit.client.1.smithi178.stdout:9/199: getdents d2/d8 0 2024-09-10T11:29:21.447 INFO:tasks.workunit.client.1.smithi178.stdout:5/39: dread f4 [0,4194304] 0 2024-09-10T11:29:21.449 INFO:tasks.workunit.client.1.smithi178.stdout:6/189: write d3/d24/f2e [399237,90220] 0 2024-09-10T11:29:21.450 INFO:tasks.workunit.client.1.smithi178.stdout:5/40: creat d9/dc/f15 x:0 0 0 2024-09-10T11:29:21.452 INFO:tasks.workunit.client.1.smithi178.stdout:5/41: chown d9 207046467 1 2024-09-10T11:29:21.453 INFO:tasks.workunit.client.1.smithi178.stdout:6/190: unlink d3/d7/d35/c46 0 2024-09-10T11:29:21.456 INFO:tasks.workunit.client.1.smithi178.stdout:5/42: mkdir d9/dc/dd/d16 0 2024-09-10T11:29:21.457 INFO:tasks.workunit.client.1.smithi178.stdout:2/99: dwrite d1/f3 [4194304,4194304] 0 2024-09-10T11:29:21.460 INFO:tasks.workunit.client.1.smithi178.stdout:6/191: mknod d3/d7/d8/d15/d48/c49 0 2024-09-10T11:29:21.462 INFO:tasks.workunit.client.1.smithi178.stdout:9/200: dwrite d2/d19/d1c/f22 [0,4194304] 0 2024-09-10T11:29:21.468 INFO:tasks.workunit.client.1.smithi178.stdout:5/43: getdents d9 0 2024-09-10T11:29:21.469 INFO:tasks.workunit.client.1.smithi178.stdout:6/192: mknod d3/d7/d8/d15/d1c/c4a 0 2024-09-10T11:29:21.470 INFO:tasks.workunit.client.1.smithi178.stdout:5/44: mkdir d9/dc/d17 0 2024-09-10T11:29:21.471 INFO:tasks.workunit.client.1.smithi178.stdout:6/193: write d3/d7/d8/d15/d1c/d2a/f3c [1513932,54755] 0 2024-09-10T11:29:21.471 INFO:tasks.workunit.client.1.smithi178.stdout:9/201: symlink d2/d19/d1c/d2a/l35 0 2024-09-10T11:29:21.472 INFO:tasks.workunit.client.1.smithi178.stdout:2/100: truncate d1/f3 7874910 0 2024-09-10T11:29:21.474 INFO:tasks.workunit.client.1.smithi178.stdout:2/101: write d1/f13 [649964,31356] 0 2024-09-10T11:29:21.474 INFO:tasks.workunit.client.1.smithi178.stdout:5/45: creat d9/d11/f18 x:0 0 0 2024-09-10T11:29:21.475 INFO:tasks.workunit.client.1.smithi178.stdout:6/194: mkdir d3/d7/d8/d15/d1c/d20/d4b 0 2024-09-10T11:29:21.478 INFO:tasks.workunit.client.1.smithi178.stdout:9/202: rename d2/c6 to d2/d8/c36 0 2024-09-10T11:29:21.479 INFO:tasks.workunit.client.1.smithi178.stdout:5/46: mkdir d9/d19 0 2024-09-10T11:29:21.482 INFO:tasks.workunit.client.1.smithi178.stdout:6/195: write d3/d7/d8/d15/d1c/f40 [1267067,48480] 0 2024-09-10T11:29:21.484 INFO:tasks.workunit.client.1.smithi178.stdout:6/196: chown d3/d7/d8/d15/d1c/d20/f28 3906707 1 2024-09-10T11:29:21.484 INFO:tasks.workunit.client.1.smithi178.stdout:5/47: creat d9/d19/f1a x:0 0 0 2024-09-10T11:29:21.487 INFO:tasks.workunit.client.1.smithi178.stdout:9/203: getdents d2/d19 0 2024-09-10T11:29:21.487 INFO:tasks.workunit.client.1.smithi178.stdout:5/48: truncate d9/d11/f18 382756 0 2024-09-10T11:29:21.491 INFO:tasks.workunit.client.1.smithi178.stdout:9/204: chown d2/df/f28 44 1 2024-09-10T11:29:21.493 INFO:tasks.workunit.client.1.smithi178.stdout:6/197: dread d3/d7/d8/d15/d1c/d20/f2d [0,4194304] 0 2024-09-10T11:29:21.494 INFO:tasks.workunit.client.1.smithi178.stdout:2/102: dwrite d1/f3 [4194304,4194304] 0 2024-09-10T11:29:21.496 INFO:tasks.workunit.client.1.smithi178.stdout:9/205: dread d2/d8/f17 [0,4194304] 0 2024-09-10T11:29:21.496 INFO:tasks.workunit.client.1.smithi178.stdout:9/206: stat d2/df 0 2024-09-10T11:29:21.498 INFO:tasks.workunit.client.1.smithi178.stdout:9/207: dread - d2/d8/f34 zero size 2024-09-10T11:29:21.498 INFO:tasks.workunit.client.1.smithi178.stdout:6/198: readlink d3/d7/d8/d15/d1c/d2a/d18/l1b 0 2024-09-10T11:29:21.500 INFO:tasks.workunit.client.1.smithi178.stdout:2/103: symlink d1/d12/l1d 0 2024-09-10T11:29:21.500 INFO:tasks.workunit.client.1.smithi178.stdout:9/208: read d2/df/f1b [1499198,130958] 0 2024-09-10T11:29:21.501 INFO:tasks.workunit.client.1.smithi178.stdout:2/104: chown d1/d7/d10 32 1 2024-09-10T11:29:21.501 INFO:tasks.workunit.client.1.smithi178.stdout:5/49: dwrite f4 [0,4194304] 0 2024-09-10T11:29:21.506 INFO:tasks.workunit.client.1.smithi178.stdout:2/105: creat d1/d7/d10/f1e x:0 0 0 2024-09-10T11:29:21.507 INFO:tasks.workunit.client.1.smithi178.stdout:9/209: mkdir d2/df/d31/d37 0 2024-09-10T11:29:21.511 INFO:tasks.workunit.client.1.smithi178.stdout:2/106: dread d1/d12/f15 [0,4194304] 0 2024-09-10T11:29:21.514 INFO:tasks.workunit.client.1.smithi178.stdout:2/107: creat d1/d7/d10/f1f x:0 0 0 2024-09-10T11:29:21.515 INFO:tasks.workunit.client.1.smithi178.stdout:6/199: write d3/d7/d8/d15/d1c/f45 [1897295,43838] 0 2024-09-10T11:29:21.518 INFO:tasks.workunit.client.1.smithi178.stdout:5/50: truncate d9/d11/f18 49110 0 2024-09-10T11:29:21.519 INFO:tasks.workunit.client.1.smithi178.stdout:9/210: truncate d2/d13/f2e 1009558 0 2024-09-10T11:29:21.523 INFO:tasks.workunit.client.1.smithi178.stdout:2/108: dwrite d1/d7/d10/d14/d11/f9 [4194304,4194304] 0 2024-09-10T11:29:21.529 INFO:tasks.workunit.client.1.smithi178.stdout:9/211: dread d2/d19/d1c/f22 [0,4194304] 0 2024-09-10T11:29:21.529 INFO:tasks.workunit.client.1.smithi178.stdout:2/109: mknod d1/c20 0 2024-09-10T11:29:21.529 INFO:tasks.workunit.client.1.smithi178.stdout:5/51: dwrite d9/ff [0,4194304] 0 2024-09-10T11:29:21.536 INFO:tasks.workunit.client.1.smithi178.stdout:5/52: creat d9/dc/dd/d16/f1b x:0 0 0 2024-09-10T11:29:21.537 INFO:tasks.workunit.client.1.smithi178.stdout:5/53: chown d9/d19 2 1 2024-09-10T11:29:21.540 INFO:tasks.workunit.client.1.smithi178.stdout:5/54: chown d9/d19/f1a 322 1 2024-09-10T11:29:21.540 INFO:tasks.workunit.client.1.smithi178.stdout:2/110: dwrite d1/d7/d10/f1e [0,4194304] 0 2024-09-10T11:29:21.540 INFO:tasks.workunit.client.1.smithi178.stdout:9/212: truncate d2/d8/f9 591925 0 2024-09-10T11:29:21.542 INFO:tasks.workunit.client.1.smithi178.stdout:5/55: creat d9/dc/dd/d16/f1c x:0 0 0 2024-09-10T11:29:21.544 INFO:tasks.workunit.client.1.smithi178.stdout:5/56: dread d9/d11/f18 [0,4194304] 0 2024-09-10T11:29:21.545 INFO:tasks.workunit.client.1.smithi178.stdout:2/111: symlink d1/d7/d10/d14/l21 0 2024-09-10T11:29:21.546 INFO:tasks.workunit.client.1.smithi178.stdout:5/57: write d9/dc/f15 [1040116,126936] 0 2024-09-10T11:29:21.550 INFO:tasks.workunit.client.1.smithi178.stdout:2/112: mkdir d1/d22 0 2024-09-10T11:29:21.550 INFO:tasks.workunit.client.1.smithi178.stdout:5/58: mkdir d9/dc/d17/d1d 0 2024-09-10T11:29:21.552 INFO:tasks.workunit.client.1.smithi178.stdout:5/59: readlink l3 0 2024-09-10T11:29:21.552 INFO:tasks.workunit.client.1.smithi178.stdout:2/113: write d1/d12/f17 [142393,61963] 0 2024-09-10T11:29:21.555 INFO:tasks.workunit.client.1.smithi178.stdout:9/213: dwrite d2/d13/f2e [0,4194304] 0 2024-09-10T11:29:21.555 INFO:tasks.workunit.client.1.smithi178.stdout:2/114: write d1/f13 [1490946,32694] 0 2024-09-10T11:29:21.556 INFO:tasks.workunit.client.1.smithi178.stdout:5/60: creat d9/dc/d17/f1e x:0 0 0 2024-09-10T11:29:21.556 INFO:tasks.workunit.client.1.smithi178.stdout:2/115: fdatasync d1/d7/d10/d14/f1a 0 2024-09-10T11:29:21.558 INFO:tasks.workunit.client.1.smithi178.stdout:5/61: truncate d9/dc/f15 2169660 0 2024-09-10T11:29:21.558 INFO:tasks.workunit.client.1.smithi178.stdout:9/214: mknod d2/d13/c38 0 2024-09-10T11:29:21.558 INFO:tasks.workunit.client.1.smithi178.stdout:5/62: fsync d9/ff 0 2024-09-10T11:29:21.560 INFO:tasks.workunit.client.1.smithi178.stdout:5/63: write d9/d19/f1a [970408,5803] 0 2024-09-10T11:29:21.562 INFO:tasks.workunit.client.1.smithi178.stdout:9/215: creat d2/d19/f39 x:0 0 0 2024-09-10T11:29:21.563 INFO:tasks.workunit.client.1.smithi178.stdout:5/64: write d9/d11/f14 [705046,42686] 0 2024-09-10T11:29:21.564 INFO:tasks.workunit.client.1.smithi178.stdout:2/116: creat d1/d7/d10/d14/f23 x:0 0 0 2024-09-10T11:29:21.565 INFO:tasks.workunit.client.1.smithi178.stdout:5/65: write f2 [1225299,37908] 0 2024-09-10T11:29:21.567 INFO:tasks.workunit.client.1.smithi178.stdout:9/216: symlink d2/d19/d1c/d2d/l3a 0 2024-09-10T11:29:21.568 INFO:tasks.workunit.client.1.smithi178.stdout:5/66: mknod d9/d11/c1f 0 2024-09-10T11:29:21.568 INFO:tasks.workunit.client.1.smithi178.stdout:9/217: dread d2/d8/f17 [0,4194304] 0 2024-09-10T11:29:21.569 INFO:tasks.workunit.client.1.smithi178.stdout:9/218: chown d2/d13/f2e 995684 1 2024-09-10T11:29:21.569 INFO:tasks.workunit.client.1.smithi178.stdout:9/219: chown d2/d13/f2e 189073 1 2024-09-10T11:29:21.569 INFO:tasks.workunit.client.1.smithi178.stdout:5/67: mknod d9/dc/c20 0 2024-09-10T11:29:21.570 INFO:tasks.workunit.client.1.smithi178.stdout:2/117: rmdir d1/d7/d10/d14 39 2024-09-10T11:29:21.572 INFO:tasks.workunit.client.1.smithi178.stdout:5/68: read d9/ff [2716289,120225] 0 2024-09-10T11:29:21.572 INFO:tasks.workunit.client.1.smithi178.stdout:9/220: creat d2/df/f3b x:0 0 0 2024-09-10T11:29:21.573 INFO:tasks.workunit.client.1.smithi178.stdout:2/118: fsync d1/d12/f15 0 2024-09-10T11:29:21.573 INFO:tasks.workunit.client.1.smithi178.stdout:5/69: write d9/dc/d17/f1e [335443,86141] 0 2024-09-10T11:29:21.574 INFO:tasks.workunit.client.1.smithi178.stdout:5/70: dread - d9/dc/dd/d16/f1c zero size 2024-09-10T11:29:21.577 INFO:tasks.workunit.client.1.smithi178.stdout:5/71: unlink c6 0 2024-09-10T11:29:21.580 INFO:tasks.workunit.client.1.smithi178.stdout:5/72: getdents d9/dc/d17/d1d 0 2024-09-10T11:29:21.582 INFO:tasks.workunit.client.1.smithi178.stdout:5/73: symlink d9/d19/l21 0 2024-09-10T11:29:21.584 INFO:tasks.workunit.client.1.smithi178.stdout:5/74: symlink d9/dc/dd/d16/l22 0 2024-09-10T11:29:21.586 INFO:tasks.workunit.client.1.smithi178.stdout:2/119: truncate d1/f5 3635974 0 2024-09-10T11:29:21.588 INFO:tasks.workunit.client.1.smithi178.stdout:5/75: link d9/d11/f14 d9/dc/dd/f23 0 2024-09-10T11:29:21.593 INFO:tasks.workunit.client.1.smithi178.stdout:9/221: dwrite d2/d8/f9 [0,4194304] 0 2024-09-10T11:29:21.593 INFO:tasks.workunit.client.1.smithi178.stdout:5/76: write d9/dc/dd/f23 [1708584,109644] 0 2024-09-10T11:29:21.595 INFO:tasks.workunit.client.1.smithi178.stdout:2/120: dwrite d1/f13 [0,4194304] 0 2024-09-10T11:29:21.599 INFO:tasks.workunit.client.1.smithi178.stdout:5/77: symlink d9/dc/d17/l24 0 2024-09-10T11:29:21.601 INFO:tasks.workunit.client.1.smithi178.stdout:5/78: symlink d9/d19/l25 0 2024-09-10T11:29:21.605 INFO:tasks.workunit.client.1.smithi178.stdout:9/222: dread d2/f18 [0,4194304] 0 2024-09-10T11:29:21.606 INFO:tasks.workunit.client.1.smithi178.stdout:5/79: symlink d9/dc/l26 0 2024-09-10T11:29:21.609 INFO:tasks.workunit.client.1.smithi178.stdout:9/223: write d2/dc/f2f [2840387,13599] 0 2024-09-10T11:29:21.612 INFO:tasks.workunit.client.1.smithi178.stdout:2/121: dwrite d1/d7/d10/d14/f1a [0,4194304] 0 2024-09-10T11:29:21.617 INFO:tasks.workunit.client.1.smithi178.stdout:2/122: read d1/d7/f16 [674400,110348] 0 2024-09-10T11:29:21.618 INFO:tasks.workunit.client.1.smithi178.stdout:2/123: chown d1/d7/d10/d14/l21 15357 1 2024-09-10T11:29:21.619 INFO:tasks.workunit.client.1.smithi178.stdout:9/224: dread d2/df/f1b [0,4194304] 0 2024-09-10T11:29:21.620 INFO:tasks.workunit.client.1.smithi178.stdout:5/80: truncate d9/dc/f15 488325 0 2024-09-10T11:29:21.621 INFO:tasks.workunit.client.1.smithi178.stdout:2/124: mkdir d1/d7/d24 0 2024-09-10T11:29:21.622 INFO:tasks.workunit.client.1.smithi178.stdout:5/81: write d9/dc/dd/d16/f1b [449844,118251] 0 2024-09-10T11:29:21.624 INFO:tasks.workunit.client.1.smithi178.stdout:9/225: creat d2/d13/f3c x:0 0 0 2024-09-10T11:29:21.629 INFO:tasks.workunit.client.1.smithi178.stdout:5/82: dread - d9/dc/dd/d16/f1c zero size 2024-09-10T11:29:21.630 INFO:tasks.workunit.client.1.smithi178.stdout:9/226: mkdir d2/df/d31/d37/d3d 0 2024-09-10T11:29:21.636 INFO:tasks.workunit.client.1.smithi178.stdout:9/227: creat d2/d19/d1c/d2d/f3e x:0 0 0 2024-09-10T11:29:21.637 INFO:tasks.workunit.client.1.smithi178.stdout:2/125: dwrite d1/f3 [8388608,4194304] 0 2024-09-10T11:29:21.641 INFO:tasks.workunit.client.1.smithi178.stdout:2/126: mknod d1/d22/c25 0 2024-09-10T11:29:21.642 INFO:tasks.workunit.client.1.smithi178.stdout:9/228: rename d2/d19/d1c/d2d/f3e to d2/f3f 0 2024-09-10T11:29:21.643 INFO:tasks.workunit.client.1.smithi178.stdout:9/229: stat d2/df/d31/d37/d3d 0 2024-09-10T11:29:21.644 INFO:tasks.workunit.client.1.smithi178.stdout:2/127: creat d1/d22/f26 x:0 0 0 2024-09-10T11:29:21.645 INFO:tasks.workunit.client.1.smithi178.stdout:2/128: write d1/d7/d10/f1e [5151947,68770] 0 2024-09-10T11:29:21.648 INFO:tasks.workunit.client.1.smithi178.stdout:9/230: rename d2/d19/d29 to d2/d13/d40 0 2024-09-10T11:29:21.653 INFO:tasks.workunit.client.1.smithi178.stdout:2/129: unlink d1/f5 0 2024-09-10T11:29:21.661 INFO:tasks.workunit.client.1.smithi178.stdout:9/231: truncate d2/dc/f2f 4412809 0 2024-09-10T11:29:21.661 INFO:tasks.workunit.client.1.smithi178.stdout:2/130: dwrite d1/d7/d10/d14/f1a [0,4194304] 0 2024-09-10T11:29:21.663 INFO:tasks.workunit.client.1.smithi178.stdout:1/41: sync 2024-09-10T11:29:21.667 INFO:tasks.workunit.client.1.smithi178.stdout:1/42: creat d3/d9/f10 x:0 0 0 2024-09-10T11:29:21.668 INFO:tasks.workunit.client.1.smithi178.stdout:1/43: fsync d3/d9/fb 0 2024-09-10T11:29:21.670 INFO:tasks.workunit.client.1.smithi178.stdout:2/131: dread d1/d7/f16 [0,4194304] 0 2024-09-10T11:29:21.670 INFO:tasks.workunit.client.1.smithi178.stdout:2/132: readlink d1/d7/d10/l18 0 2024-09-10T11:29:21.674 INFO:tasks.workunit.client.1.smithi178.stdout:2/133: creat d1/d12/f27 x:0 0 0 2024-09-10T11:29:21.680 INFO:tasks.workunit.client.1.smithi178.stdout:2/134: dread d1/d7/f16 [0,4194304] 0 2024-09-10T11:29:21.681 INFO:tasks.workunit.client.1.smithi178.stdout:1/44: dwrite d3/d9/fc [0,4194304] 0 2024-09-10T11:29:21.682 INFO:tasks.workunit.client.1.smithi178.stdout:9/232: dwrite d2/df/f28 [0,4194304] 0 2024-09-10T11:29:21.683 INFO:tasks.workunit.client.1.smithi178.stdout:1/45: write f1 [702788,53330] 0 2024-09-10T11:29:21.683 INFO:tasks.workunit.client.1.smithi178.stdout:2/135: write d1/f3 [9566938,106401] 0 2024-09-10T11:29:21.684 INFO:tasks.workunit.client.1.smithi178.stdout:1/46: stat d3/d9 0 2024-09-10T11:29:21.684 INFO:tasks.workunit.client.1.smithi178.stdout:1/47: rename d3/d9 to d3/d9/d11 22 2024-09-10T11:29:21.689 INFO:tasks.workunit.client.1.smithi178.stdout:2/136: mknod d1/d12/c28 0 2024-09-10T11:29:21.691 INFO:tasks.workunit.client.1.smithi178.stdout:2/137: fdatasync d1/d12/f17 0 2024-09-10T11:29:21.692 INFO:tasks.workunit.client.1.smithi178.stdout:9/233: truncate d2/dc/f2f 3476105 0 2024-09-10T11:29:21.697 INFO:tasks.workunit.client.1.smithi178.stdout:9/234: mkdir d2/d19/d1c/d2a/d41 0 2024-09-10T11:29:21.699 INFO:tasks.workunit.client.1.smithi178.stdout:1/48: dwrite d3/f8 [0,4194304] 0 2024-09-10T11:29:21.700 INFO:tasks.workunit.client.1.smithi178.stdout:9/235: symlink d2/d19/l42 0 2024-09-10T11:29:21.701 INFO:tasks.workunit.client.1.smithi178.stdout:2/138: dread d1/f3 [4194304,4194304] 0 2024-09-10T11:29:21.702 INFO:tasks.workunit.client.1.smithi178.stdout:1/49: mkdir d3/d9/d12 0 2024-09-10T11:29:21.702 INFO:tasks.workunit.client.1.smithi178.stdout:1/50: stat d3/de 0 2024-09-10T11:29:21.704 INFO:tasks.workunit.client.1.smithi178.stdout:2/139: creat d1/d12/f29 x:0 0 0 2024-09-10T11:29:21.705 INFO:tasks.workunit.client.1.smithi178.stdout:2/140: write d1/d7/d10/d14/f1a [2093043,110298] 0 2024-09-10T11:29:21.706 INFO:tasks.workunit.client.1.smithi178.stdout:1/51: unlink c0 0 2024-09-10T11:29:21.707 INFO:tasks.workunit.client.1.smithi178.stdout:9/236: dread d2/d13/f15 [0,4194304] 0 2024-09-10T11:29:21.707 INFO:tasks.workunit.client.1.smithi178.stdout:2/141: truncate d1/d12/f29 130044 0 2024-09-10T11:29:21.708 INFO:tasks.workunit.client.1.smithi178.stdout:1/52: truncate d3/d9/fa 978488 0 2024-09-10T11:29:21.710 INFO:tasks.workunit.client.1.smithi178.stdout:1/53: write d3/d9/fd [909690,21976] 0 2024-09-10T11:29:21.711 INFO:tasks.workunit.client.1.smithi178.stdout:1/54: chown d3/d9/fc 847 1 2024-09-10T11:29:21.712 INFO:tasks.workunit.client.1.smithi178.stdout:2/142: rename d1/d7/d10/d14/f1a to d1/d7/d10/d14/d11/f2a 0 2024-09-10T11:29:21.713 INFO:tasks.workunit.client.1.smithi178.stdout:2/143: dread - d1/d7/d10/d14/f1b zero size 2024-09-10T11:29:21.715 INFO:tasks.workunit.client.1.smithi178.stdout:2/144: symlink d1/d7/d24/l2b 0 2024-09-10T11:29:21.716 INFO:tasks.workunit.client.1.smithi178.stdout:2/145: fsync d1/d12/f27 0 2024-09-10T11:29:21.719 INFO:tasks.workunit.client.1.smithi178.stdout:2/146: creat d1/d7/d24/f2c x:0 0 0 2024-09-10T11:29:21.720 INFO:tasks.workunit.client.1.smithi178.stdout:1/55: dread d3/d9/fd [0,4194304] 0 2024-09-10T11:29:21.722 INFO:tasks.workunit.client.1.smithi178.stdout:2/147: creat d1/d7/d10/d14/d11/d19/f2d x:0 0 0 2024-09-10T11:29:21.724 INFO:tasks.workunit.client.1.smithi178.stdout:1/56: dread d3/d9/fc [0,4194304] 0 2024-09-10T11:29:21.726 INFO:tasks.workunit.client.1.smithi178.stdout:1/57: write d3/d9/fb [586302,16791] 0 2024-09-10T11:29:21.727 INFO:tasks.workunit.client.1.smithi178.stdout:2/148: creat d1/f2e x:0 0 0 2024-09-10T11:29:21.728 INFO:tasks.workunit.client.1.smithi178.stdout:2/149: dread - d1/d22/f26 zero size 2024-09-10T11:29:21.729 INFO:tasks.workunit.client.1.smithi178.stdout:1/58: write d3/d9/fc [3926744,4137] 0 2024-09-10T11:29:21.732 INFO:tasks.workunit.client.1.smithi178.stdout:9/237: dwrite d2/d13/f1f [0,4194304] 0 2024-09-10T11:29:21.732 INFO:tasks.workunit.client.1.smithi178.stdout:2/150: write d1/d7/d10/f1e [5380417,23632] 0 2024-09-10T11:29:21.732 INFO:tasks.workunit.client.1.smithi178.stdout:1/59: write d3/d9/fa [1207100,74714] 0 2024-09-10T11:29:21.733 INFO:tasks.workunit.client.1.smithi178.stdout:2/151: chown d1/d7/d10/l18 119688258 1 2024-09-10T11:29:21.737 INFO:tasks.workunit.client.1.smithi178.stdout:1/60: rename d3/d9/d12 to d3/d13 0 2024-09-10T11:29:21.737 INFO:tasks.workunit.client.1.smithi178.stdout:2/152: rename d1/d22 to d1/d22/d2f 22 2024-09-10T11:29:21.737 INFO:tasks.workunit.client.1.smithi178.stdout:1/61: chown d3/de 211934 1 2024-09-10T11:29:21.741 INFO:tasks.workunit.client.1.smithi178.stdout:1/62: creat d3/d9/f14 x:0 0 0 2024-09-10T11:29:21.742 INFO:tasks.workunit.client.1.smithi178.stdout:9/238: dwrite d2/d19/f39 [0,4194304] 0 2024-09-10T11:29:21.744 INFO:tasks.workunit.client.1.smithi178.stdout:1/63: truncate d3/d9/f14 521194 0 2024-09-10T11:29:21.747 INFO:tasks.workunit.client.1.smithi178.stdout:8/120: sync 2024-09-10T11:29:21.747 INFO:tasks.workunit.client.1.smithi178.stdout:9/239: creat d2/d13/f43 x:0 0 0 2024-09-10T11:29:21.749 INFO:tasks.workunit.client.1.smithi178.stdout:9/240: readlink d2/d19/l42 0 2024-09-10T11:29:21.750 INFO:tasks.workunit.client.1.smithi178.stdout:1/64: dread d3/d9/fd [0,4194304] 0 2024-09-10T11:29:21.751 INFO:tasks.workunit.client.1.smithi178.stdout:2/153: dwrite d1/f3 [12582912,4194304] 0 2024-09-10T11:29:21.756 INFO:tasks.workunit.client.1.smithi178.stdout:2/154: creat d1/d22/f30 x:0 0 0 2024-09-10T11:29:21.758 INFO:tasks.workunit.client.1.smithi178.stdout:1/65: rmdir d3 39 2024-09-10T11:29:21.761 INFO:tasks.workunit.client.1.smithi178.stdout:9/241: dread d2/d13/f15 [0,4194304] 0 2024-09-10T11:29:21.761 INFO:tasks.workunit.client.1.smithi178.stdout:8/121: dwrite d5/db/d10/d20/f26 [0,4194304] 0 2024-09-10T11:29:21.762 INFO:tasks.workunit.client.1.smithi178.stdout:2/155: write d1/d7/d10/d14/d11/f2a [4941879,13698] 0 2024-09-10T11:29:21.764 INFO:tasks.workunit.client.1.smithi178.stdout:9/242: fsync d2/d19/d1c/f22 0 2024-09-10T11:29:21.766 INFO:tasks.workunit.client.1.smithi178.stdout:2/156: creat d1/d7/d10/f31 x:0 0 0 2024-09-10T11:29:21.767 INFO:tasks.workunit.client.1.smithi178.stdout:9/243: fsync d2/d19/d1c/f2c 0 2024-09-10T11:29:21.770 INFO:tasks.workunit.client.1.smithi178.stdout:1/66: dwrite d3/d9/f10 [0,4194304] 0 2024-09-10T11:29:21.771 INFO:tasks.workunit.client.1.smithi178.stdout:1/67: stat d3/d9/fd 0 2024-09-10T11:29:21.775 INFO:tasks.workunit.client.1.smithi178.stdout:1/68: write d3/d9/f14 [458412,113576] 0 2024-09-10T11:29:21.777 INFO:tasks.workunit.client.1.smithi178.stdout:1/69: stat d3/d9/f14 0 2024-09-10T11:29:21.778 INFO:tasks.workunit.client.1.smithi178.stdout:2/157: dwrite d1/d12/f27 [0,4194304] 0 2024-09-10T11:29:21.781 INFO:tasks.workunit.client.1.smithi178.stdout:9/244: dwrite d2/d19/d1c/f22 [0,4194304] 0 2024-09-10T11:29:21.786 INFO:tasks.workunit.client.1.smithi178.stdout:2/158: dread d1/d12/f27 [0,4194304] 0 2024-09-10T11:29:21.789 INFO:tasks.workunit.client.1.smithi178.stdout:9/245: rmdir d2/d19/d1c/d2a/d41 0 2024-09-10T11:29:21.792 INFO:tasks.workunit.client.1.smithi178.stdout:9/246: creat d2/dc/f44 x:0 0 0 2024-09-10T11:29:21.792 INFO:tasks.workunit.client.1.smithi178.stdout:2/159: dread d1/f3 [12582912,4194304] 0 2024-09-10T11:29:21.793 INFO:tasks.workunit.client.1.smithi178.stdout:9/247: write d2/d8/f9 [3383698,101299] 0 2024-09-10T11:29:21.796 INFO:tasks.workunit.client.1.smithi178.stdout:2/160: mknod d1/d12/c32 0 2024-09-10T11:29:21.797 INFO:tasks.workunit.client.1.smithi178.stdout:9/248: creat d2/d19/d1c/d2d/f45 x:0 0 0 2024-09-10T11:29:21.797 INFO:tasks.workunit.client.1.smithi178.stdout:2/161: read - d1/d7/d10/f1f zero size 2024-09-10T11:29:21.798 INFO:tasks.workunit.client.1.smithi178.stdout:2/162: write d1/f13 [2078980,2643] 0 2024-09-10T11:29:21.800 INFO:tasks.workunit.client.1.smithi178.stdout:2/163: dread - d1/d7/d10/d14/f1b zero size 2024-09-10T11:29:21.802 INFO:tasks.workunit.client.1.smithi178.stdout:9/249: dread d2/d13/f1f [0,4194304] 0 2024-09-10T11:29:21.802 INFO:tasks.workunit.client.1.smithi178.stdout:2/164: mknod d1/d7/d10/d14/d11/d19/c33 0 2024-09-10T11:29:21.804 INFO:tasks.workunit.client.1.smithi178.stdout:9/250: mknod d2/d19/d1c/c46 0 2024-09-10T11:29:21.808 INFO:tasks.workunit.client.1.smithi178.stdout:2/165: rename d1/d12 to d1/d7/d10/d14/d11/d19/d34 0 2024-09-10T11:29:21.810 INFO:tasks.workunit.client.1.smithi178.stdout:9/251: dwrite d2/d8/f9 [0,4194304] 0 2024-09-10T11:29:21.811 INFO:tasks.workunit.client.1.smithi178.stdout:9/252: dread - d2/d8/f34 zero size 2024-09-10T11:29:21.811 INFO:tasks.workunit.client.1.smithi178.stdout:9/253: chown d2/df/d31 243016 1 2024-09-10T11:29:21.812 INFO:tasks.workunit.client.1.smithi178.stdout:2/166: rename d1/d7/d10/d14/d11/d19/d34/f27 to d1/d7/d10/d14/d11/d19/f35 0 2024-09-10T11:29:21.814 INFO:tasks.workunit.client.1.smithi178.stdout:9/254: unlink d2/d19/d1c/c46 0 2024-09-10T11:29:21.815 INFO:tasks.workunit.client.1.smithi178.stdout:2/167: dread d1/d7/d10/d14/d11/d19/d34/f17 [0,4194304] 0 2024-09-10T11:29:21.818 INFO:tasks.workunit.client.1.smithi178.stdout:9/255: getdents d2/df/d31/d37/d3d 0 2024-09-10T11:29:21.818 INFO:tasks.workunit.client.1.smithi178.stdout:9/256: chown d2/df 6223 1 2024-09-10T11:29:21.820 INFO:tasks.workunit.client.1.smithi178.stdout:2/168: dread d1/f13 [0,4194304] 0 2024-09-10T11:29:21.823 INFO:tasks.workunit.client.1.smithi178.stdout:2/169: creat d1/d7/d10/d14/d11/d19/f36 x:0 0 0 2024-09-10T11:29:21.827 INFO:tasks.workunit.client.1.smithi178.stdout:2/170: link d1/d7/d10/d14/d11/d19/f35 d1/d7/d10/d14/d11/d19/f37 0 2024-09-10T11:29:21.831 INFO:tasks.workunit.client.1.smithi178.stdout:9/257: dwrite d2/d19/d1c/f24 [0,4194304] 0 2024-09-10T11:29:21.833 INFO:tasks.workunit.client.1.smithi178.stdout:9/258: write d2/df/f28 [1214688,42878] 0 2024-09-10T11:29:21.837 INFO:tasks.workunit.client.1.smithi178.stdout:2/171: dwrite d1/d7/d10/d14/d11/d19/d34/f15 [0,4194304] 0 2024-09-10T11:29:21.837 INFO:tasks.workunit.client.1.smithi178.stdout:2/172: read d1/d7/f16 [828133,35882] 0 2024-09-10T11:29:21.842 INFO:tasks.workunit.client.1.smithi178.stdout:9/259: truncate d2/d8/f9 684526 0 2024-09-10T11:29:21.843 INFO:tasks.workunit.client.1.smithi178.stdout:2/173: creat d1/d7/d10/f38 x:0 0 0 2024-09-10T11:29:21.845 INFO:tasks.workunit.client.1.smithi178.stdout:2/174: mknod d1/d7/d24/c39 0 2024-09-10T11:29:21.847 INFO:tasks.workunit.client.1.smithi178.stdout:2/175: creat d1/d22/f3a x:0 0 0 2024-09-10T11:29:21.856 INFO:tasks.workunit.client.1.smithi178.stdout:9/260: dwrite d2/f18 [0,4194304] 0 2024-09-10T11:29:21.856 INFO:tasks.workunit.client.1.smithi178.stdout:2/176: rmdir d1/d7 39 2024-09-10T11:29:21.860 INFO:tasks.workunit.client.1.smithi178.stdout:9/261: getdents d2/df/d31/d37 0 2024-09-10T11:29:21.861 INFO:tasks.workunit.client.1.smithi178.stdout:2/177: creat d1/d7/d10/d14/f3b x:0 0 0 2024-09-10T11:29:21.862 INFO:tasks.workunit.client.1.smithi178.stdout:2/178: fdatasync d1/d7/d10/d14/d11/d19/d34/f15 0 2024-09-10T11:29:21.864 INFO:tasks.workunit.client.1.smithi178.stdout:2/179: mknod d1/d7/c3c 0 2024-09-10T11:29:21.866 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:21 smithi178 ceph-mon[33514]: Reconfiguring prometheus.smithi144 (dependencies changed)... 2024-09-10T11:29:21.866 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:21 smithi178 ceph-mon[33514]: Reconfiguring daemon prometheus.smithi144 on smithi144 2024-09-10T11:29:21.867 INFO:tasks.workunit.client.1.smithi178.stdout:2/180: creat d1/f3d x:0 0 0 2024-09-10T11:29:21.869 INFO:tasks.workunit.client.1.smithi178.stdout:2/181: chown d1/d7/d10/d14/d11/d19/d34/c32 676254 1 2024-09-10T11:29:21.871 INFO:tasks.workunit.client.1.smithi178.stdout:9/262: dwrite d2/d19/d1c/f2c [4194304,4194304] 0 2024-09-10T11:29:21.872 INFO:tasks.workunit.client.1.smithi178.stdout:9/263: chown d2/d13/c2b 4614611 1 2024-09-10T11:29:21.874 INFO:tasks.workunit.client.1.smithi178.stdout:9/264: readlink d2/d19/d1c/d2d/l3a 0 2024-09-10T11:29:21.874 INFO:tasks.workunit.client.1.smithi178.stdout:9/265: stat d2/d8/f17 0 2024-09-10T11:29:21.875 INFO:tasks.workunit.client.1.smithi178.stdout:2/182: creat d1/d7/d10/d14/f3e x:0 0 0 2024-09-10T11:29:21.879 INFO:tasks.workunit.client.1.smithi178.stdout:9/266: creat d2/df/d31/f47 x:0 0 0 2024-09-10T11:29:21.879 INFO:tasks.workunit.client.1.smithi178.stdout:2/183: symlink d1/l3f 0 2024-09-10T11:29:21.885 INFO:tasks.workunit.client.1.smithi178.stdout:2/184: truncate d1/d7/d10/d14/d11/d19/f37 1911673 0 2024-09-10T11:29:21.885 INFO:tasks.workunit.client.1.smithi178.stdout:2/185: stat d1/d7/f16 0 2024-09-10T11:29:21.887 INFO:tasks.workunit.client.1.smithi178.stdout:2/186: rmdir d1/d22 39 2024-09-10T11:29:21.888 INFO:tasks.workunit.client.1.smithi178.stdout:2/187: dread - d1/d7/d10/f38 zero size 2024-09-10T11:29:21.889 INFO:tasks.workunit.client.1.smithi178.stdout:2/188: chown d1/d7/d10/d14/d11/d19/d34/c32 13152431 1 2024-09-10T11:29:21.890 INFO:tasks.workunit.client.1.smithi178.stdout:2/189: write d1/d7/d10/d14/d11/f2a [2476278,108033] 0 2024-09-10T11:29:21.895 INFO:tasks.workunit.client.1.smithi178.stdout:9/267: truncate d2/df/f28 850100 0 2024-09-10T11:29:21.899 INFO:tasks.workunit.client.1.smithi178.stdout:2/190: truncate d1/d7/d10/d14/d11/d19/d34/f17 145487 0 2024-09-10T11:29:21.901 INFO:tasks.workunit.client.1.smithi178.stdout:9/268: link d2/d19/f39 d2/d13/f48 0 2024-09-10T11:29:21.905 INFO:tasks.workunit.client.1.smithi178.stdout:2/191: dwrite d1/d7/d10/d14/d11/f9 [0,4194304] 0 2024-09-10T11:29:21.906 INFO:tasks.workunit.client.1.smithi178.stdout:9/269: creat d2/d19/d1c/d2a/f49 x:0 0 0 2024-09-10T11:29:21.908 INFO:tasks.workunit.client.1.smithi178.stdout:9/270: mkdir d2/df/d31/d4a 0 2024-09-10T11:29:21.909 INFO:tasks.workunit.client.1.smithi178.stdout:2/192: mkdir d1/d7/d10/d14/d11/d19/d34/d40 0 2024-09-10T11:29:21.916 INFO:tasks.workunit.client.1.smithi178.stdout:9/271: dwrite d2/d8/f17 [0,4194304] 0 2024-09-10T11:29:21.920 INFO:tasks.workunit.client.1.smithi178.stdout:9/272: mkdir d2/d13/d40/d4b 0 2024-09-10T11:29:21.922 INFO:tasks.workunit.client.1.smithi178.stdout:9/273: readlink d2/d8/l33 0 2024-09-10T11:29:21.923 INFO:tasks.workunit.client.1.smithi178.stdout:2/193: rename d1/d7/d10 to d1/d41 0 2024-09-10T11:29:21.925 INFO:tasks.workunit.client.1.smithi178.stdout:9/274: symlink d2/df/d31/l4c 0 2024-09-10T11:29:21.926 INFO:tasks.workunit.client.1.smithi178.stdout:2/194: creat d1/d7/d24/f42 x:0 0 0 2024-09-10T11:29:21.927 INFO:tasks.workunit.client.1.smithi178.stdout:2/195: write d1/d41/f38 [396095,14841] 0 2024-09-10T11:29:21.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:21 smithi144 ceph-mon[27942]: Reconfiguring prometheus.smithi144 (dependencies changed)... 2024-09-10T11:29:21.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:21 smithi144 ceph-mon[27942]: Reconfiguring daemon prometheus.smithi144 on smithi144 2024-09-10T11:29:21.936 INFO:tasks.workunit.client.1.smithi178.stdout:2/196: dwrite d1/d41/f38 [0,4194304] 0 2024-09-10T11:29:21.938 INFO:tasks.workunit.client.1.smithi178.stdout:9/275: dwrite d2/d8/fa [0,4194304] 0 2024-09-10T11:29:21.940 INFO:tasks.workunit.client.1.smithi178.stdout:2/197: read - d1/d22/f26 zero size 2024-09-10T11:29:21.941 INFO:tasks.workunit.client.1.smithi178.stdout:9/276: unlink d2/d8/l25 0 2024-09-10T11:29:21.943 INFO:tasks.workunit.client.1.smithi178.stdout:9/277: mknod d2/d19/d1c/c4d 0 2024-09-10T11:29:21.945 INFO:tasks.workunit.client.1.smithi178.stdout:9/278: readlink d2/le 0 2024-09-10T11:29:21.947 INFO:tasks.workunit.client.1.smithi178.stdout:2/198: dread d1/d41/f38 [0,4194304] 0 2024-09-10T11:29:21.947 INFO:tasks.workunit.client.1.smithi178.stdout:9/279: mknod d2/d8/c4e 0 2024-09-10T11:29:21.948 INFO:tasks.workunit.client.1.smithi178.stdout:2/199: chown d1/d41/d14/d11/lb 58291 1 2024-09-10T11:29:21.949 INFO:tasks.workunit.client.1.smithi178.stdout:2/200: dread - d1/d41/d14/f1b zero size 2024-09-10T11:29:21.954 INFO:tasks.workunit.client.1.smithi178.stdout:2/201: write d1/d41/d14/d11/d19/d34/f15 [5292121,85925] 0 2024-09-10T11:29:21.956 INFO:tasks.workunit.client.1.smithi178.stdout:2/202: readlink d1/d7/l1c 0 2024-09-10T11:29:21.958 INFO:tasks.workunit.client.1.smithi178.stdout:9/280: dwrite d2/d19/d1c/d2a/f49 [0,4194304] 0 2024-09-10T11:29:21.959 INFO:tasks.workunit.client.1.smithi178.stdout:9/281: dread - d2/d19/d1c/d2d/f45 zero size 2024-09-10T11:29:21.960 INFO:tasks.workunit.client.1.smithi178.stdout:2/203: mkdir d1/d7/d24/d43 0 2024-09-10T11:29:21.961 INFO:tasks.workunit.client.1.smithi178.stdout:2/204: chown d1/d22 141422 1 2024-09-10T11:29:21.963 INFO:tasks.workunit.client.1.smithi178.stdout:2/205: write d1/d22/f3a [240251,92903] 0 2024-09-10T11:29:21.969 INFO:tasks.workunit.client.1.smithi178.stdout:2/206: creat d1/d41/f44 x:0 0 0 2024-09-10T11:29:21.971 INFO:tasks.workunit.client.1.smithi178.stdout:2/207: mknod d1/d7/c45 0 2024-09-10T11:29:21.974 INFO:tasks.workunit.client.1.smithi178.stdout:9/282: dwrite d2/d19/f39 [4194304,4194304] 0 2024-09-10T11:29:21.976 INFO:tasks.workunit.client.1.smithi178.stdout:2/208: truncate d1/d41/d14/d11/d19/f2d 87432 0 2024-09-10T11:29:21.980 INFO:tasks.workunit.client.1.smithi178.stdout:2/209: truncate d1/d22/f30 226485 0 2024-09-10T11:29:21.980 INFO:tasks.workunit.client.1.smithi178.stdout:9/283: dread d2/d8/f17 [0,4194304] 0 2024-09-10T11:29:21.981 INFO:tasks.workunit.client.1.smithi178.stdout:9/284: stat d2/df/f3b 0 2024-09-10T11:29:21.981 INFO:tasks.workunit.client.1.smithi178.stdout:2/210: stat d1/d7/d24/d43 0 2024-09-10T11:29:21.985 INFO:tasks.workunit.client.1.smithi178.stdout:2/211: mkdir d1/d7/d24/d46 0 2024-09-10T11:29:21.985 INFO:tasks.workunit.client.1.smithi178.stdout:9/285: write d2/dc/f2f [636956,66601] 0 2024-09-10T11:29:21.987 INFO:tasks.workunit.client.1.smithi178.stdout:2/212: chown d1/d41/d14/d11/d19/d34/c28 1872715 1 2024-09-10T11:29:21.987 INFO:tasks.workunit.client.1.smithi178.stdout:9/286: readlink d2/d19/d1c/d2d/l3a 0 2024-09-10T11:29:21.989 INFO:tasks.workunit.client.1.smithi178.stdout:2/213: unlink d1/l3f 0 2024-09-10T11:29:21.989 INFO:tasks.workunit.client.1.smithi178.stdout:9/287: getdents d2/df/d31/d4a 0 2024-09-10T11:29:21.991 INFO:tasks.workunit.client.1.smithi178.stdout:2/214: write d1/d41/d14/d11/d19/f2d [344965,112329] 0 2024-09-10T11:29:21.995 INFO:tasks.workunit.client.1.smithi178.stdout:2/215: rmdir d1 39 2024-09-10T11:29:21.999 INFO:tasks.workunit.client.1.smithi178.stdout:2/216: creat d1/d41/d14/d11/d19/d34/f47 x:0 0 0 2024-09-10T11:29:22.001 INFO:tasks.workunit.client.1.smithi178.stdout:9/288: dwrite d2/df/f28 [0,4194304] 0 2024-09-10T11:29:22.005 INFO:tasks.workunit.client.1.smithi178.stdout:9/289: creat d2/df/d31/d37/f4f x:0 0 0 2024-09-10T11:29:22.006 INFO:tasks.workunit.client.1.smithi178.stdout:9/290: dread - d2/d13/f3c zero size 2024-09-10T11:29:22.006 INFO:tasks.workunit.client.1.smithi178.stdout:2/217: dread d1/d41/d14/d11/d19/d34/f15 [0,4194304] 0 2024-09-10T11:29:22.009 INFO:tasks.workunit.client.1.smithi178.stdout:9/291: creat d2/d8/f50 x:0 0 0 2024-09-10T11:29:22.012 INFO:tasks.workunit.client.1.smithi178.stdout:9/292: mkdir d2/d51 0 2024-09-10T11:29:22.017 INFO:tasks.workunit.client.1.smithi178.stdout:2/218: write d1/d41/d14/d11/d19/f37 [1186184,121977] 0 2024-09-10T11:29:22.019 INFO:tasks.workunit.client.1.smithi178.stdout:2/219: read d1/d41/d14/d11/d19/f37 [1234680,103299] 0 2024-09-10T11:29:22.021 INFO:tasks.workunit.client.1.smithi178.stdout:2/220: read - d1/d41/d14/d11/d19/d34/f47 zero size 2024-09-10T11:29:22.021 INFO:tasks.workunit.client.1.smithi178.stdout:4/38: sync 2024-09-10T11:29:22.025 INFO:tasks.workunit.client.1.smithi178.stdout:2/221: link d1/d41/d14/d11/f9 d1/d7/d24/d46/f48 0 2024-09-10T11:29:22.029 INFO:tasks.workunit.client.1.smithi178.stdout:4/39: dwrite d0/d9/fb [0,4194304] 0 2024-09-10T11:29:22.033 INFO:tasks.workunit.client.1.smithi178.stdout:4/40: mkdir d0/d9/dc 0 2024-09-10T11:29:22.035 INFO:tasks.workunit.client.1.smithi178.stdout:2/222: dwrite d1/d41/d14/f1b [0,4194304] 0 2024-09-10T11:29:22.037 INFO:tasks.workunit.client.1.smithi178.stdout:4/41: mknod d0/cd 0 2024-09-10T11:29:22.042 INFO:tasks.workunit.client.1.smithi178.stdout:4/42: getdents d0/d8 0 2024-09-10T11:29:22.045 INFO:tasks.workunit.client.1.smithi178.stdout:2/223: dwrite d1/f3d [0,4194304] 0 2024-09-10T11:29:22.047 INFO:tasks.workunit.client.1.smithi178.stdout:4/43: mknod d0/d9/ce 0 2024-09-10T11:29:22.048 INFO:tasks.workunit.client.1.smithi178.stdout:4/44: readlink d0/l1 0 2024-09-10T11:29:22.057 INFO:tasks.workunit.client.1.smithi178.stdout:2/224: dwrite d1/d22/f30 [0,4194304] 0 2024-09-10T11:29:22.059 INFO:tasks.workunit.client.1.smithi178.stdout:2/225: fdatasync d1/d7/d24/d46/f48 0 2024-09-10T11:29:22.062 INFO:tasks.workunit.client.1.smithi178.stdout:2/226: mknod d1/d7/d24/d43/c49 0 2024-09-10T11:29:22.065 INFO:tasks.workunit.client.1.smithi178.stdout:2/227: getdents d1/d41/d14 0 2024-09-10T11:29:22.068 INFO:tasks.workunit.client.1.smithi178.stdout:2/228: creat d1/d41/d14/d11/f4a x:0 0 0 2024-09-10T11:29:22.069 INFO:tasks.workunit.client.1.smithi178.stdout:4/45: truncate d0/f5 93040 0 2024-09-10T11:29:22.071 INFO:tasks.workunit.client.1.smithi178.stdout:4/46: getdents d0/d9/dc 0 2024-09-10T11:29:22.073 INFO:tasks.workunit.client.1.smithi178.stdout:4/47: mkdir d0/d8/df 0 2024-09-10T11:29:22.077 INFO:tasks.workunit.client.1.smithi178.stdout:2/229: dwrite d1/d7/d24/f2c [0,4194304] 0 2024-09-10T11:29:22.078 INFO:tasks.workunit.client.1.smithi178.stdout:4/48: dread d0/f5 [0,4194304] 0 2024-09-10T11:29:22.080 INFO:tasks.workunit.client.1.smithi178.stdout:2/230: fdatasync d1/d41/d14/d11/d19/d34/f15 0 2024-09-10T11:29:22.081 INFO:tasks.workunit.client.1.smithi178.stdout:2/231: fsync d1/d41/f1f 0 2024-09-10T11:29:22.082 INFO:tasks.workunit.client.1.smithi178.stdout:2/232: fdatasync d1/d41/f1f 0 2024-09-10T11:29:22.082 INFO:tasks.workunit.client.1.smithi178.stdout:4/49: getdents d0/d8/da 0 2024-09-10T11:29:22.085 INFO:tasks.workunit.client.1.smithi178.stdout:4/50: mkdir d0/d10 0 2024-09-10T11:29:22.088 INFO:tasks.workunit.client.1.smithi178.stdout:4/51: mknod d0/d10/c11 0 2024-09-10T11:29:22.090 INFO:tasks.workunit.client.1.smithi178.stdout:4/52: dread d0/f5 [0,4194304] 0 2024-09-10T11:29:22.090 INFO:tasks.workunit.client.1.smithi178.stdout:2/233: dwrite d1/d7/f16 [0,4194304] 0 2024-09-10T11:29:22.093 INFO:tasks.workunit.client.1.smithi178.stdout:4/53: creat d0/d9/f12 x:0 0 0 2024-09-10T11:29:22.093 INFO:tasks.workunit.client.1.smithi178.stdout:4/54: chown d0/l4 16 1 2024-09-10T11:29:22.096 INFO:tasks.workunit.client.1.smithi178.stdout:2/234: rename d1/d41/d14/d11/d19/d34/d40 to d1/d41/d14/d11/d19/d4b 0 2024-09-10T11:29:22.098 INFO:tasks.workunit.client.1.smithi178.stdout:2/235: write d1/d41/d14/d11/f2a [699168,48908] 0 2024-09-10T11:29:22.098 INFO:tasks.workunit.client.1.smithi178.stdout:4/55: creat d0/d9/f13 x:0 0 0 2024-09-10T11:29:22.099 INFO:tasks.workunit.client.1.smithi178.stdout:2/236: chown d1/d41/d14/d11/lb 0 1 2024-09-10T11:29:22.100 INFO:tasks.workunit.client.1.smithi178.stdout:4/56: dread d0/f5 [0,4194304] 0 2024-09-10T11:29:22.101 INFO:tasks.workunit.client.1.smithi178.stdout:2/237: chown d1/d41/d14/d11/f2a 3080737 1 2024-09-10T11:29:22.101 INFO:tasks.workunit.client.1.smithi178.stdout:2/238: chown d1/d7/f16 30095 1 2024-09-10T11:29:22.104 INFO:tasks.workunit.client.1.smithi178.stdout:2/239: mknod d1/d41/d14/d11/c4c 0 2024-09-10T11:29:22.104 INFO:tasks.workunit.client.1.smithi178.stdout:4/57: link d0/c6 d0/d9/dc/c14 0 2024-09-10T11:29:22.106 INFO:tasks.workunit.client.1.smithi178.stdout:2/240: mkdir d1/d22/d4d 0 2024-09-10T11:29:22.109 INFO:tasks.workunit.client.1.smithi178.stdout:2/241: fdatasync d1/d7/d24/d46/f48 0 2024-09-10T11:29:22.112 INFO:tasks.workunit.client.1.smithi178.stdout:4/58: dwrite d0/d9/f13 [0,4194304] 0 2024-09-10T11:29:22.113 INFO:tasks.workunit.client.1.smithi178.stdout:4/59: readlink d0/l7 0 2024-09-10T11:29:22.113 INFO:tasks.workunit.client.1.smithi178.stdout:4/60: readlink d0/l7 0 2024-09-10T11:29:22.116 INFO:tasks.workunit.client.1.smithi178.stdout:4/61: mknod d0/d8/df/c15 0 2024-09-10T11:29:22.118 INFO:tasks.workunit.client.1.smithi178.stdout:2/242: dwrite d1/f3 [12582912,4194304] 0 2024-09-10T11:29:22.122 INFO:tasks.workunit.client.1.smithi178.stdout:2/243: mknod d1/d7/d24/c4e 0 2024-09-10T11:29:22.125 INFO:tasks.workunit.client.1.smithi178.stdout:4/62: dwrite d0/d9/f13 [4194304,4194304] 0 2024-09-10T11:29:22.127 INFO:tasks.workunit.client.1.smithi178.stdout:2/244: getdents d1/d22 0 2024-09-10T11:29:22.143 INFO:tasks.workunit.client.0.smithi144.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-09-10T11:29:22.150 INFO:tasks.workunit.client.0.smithi144.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-10T11:29:22.151 INFO:tasks.workunit.client.0.smithi144.stderr:+ make 2024-09-10T11:29:22.181 INFO:tasks.workunit.client.1.smithi178.stdout:7/27: sync 2024-09-10T11:29:22.192 INFO:tasks.workunit.client.1.smithi178.stdout:2/245: write d1/d41/d14/d11/f4a [69188,75011] 0 2024-09-10T11:29:22.193 INFO:tasks.workunit.client.1.smithi178.stdout:2/246: readlink d1/d41/d14/l21 0 2024-09-10T11:29:22.194 INFO:tasks.workunit.client.1.smithi178.stdout:2/247: chown d1/d41/d14/l21 15 1 2024-09-10T11:29:22.195 INFO:tasks.workunit.client.1.smithi178.stdout:2/248: truncate d1/d41/d14/d11/f4a 1018895 0 2024-09-10T11:29:22.196 INFO:tasks.workunit.client.1.smithi178.stdout:2/249: write d1/d41/d14/d11/d19/f36 [503940,69252] 0 2024-09-10T11:29:22.197 INFO:tasks.workunit.client.1.smithi178.stdout:2/250: chown d1/f3 497 1 2024-09-10T11:29:22.197 INFO:tasks.workunit.client.1.smithi178.stdout:2/251: dread - d1/d7/d24/f42 zero size 2024-09-10T11:29:22.199 INFO:tasks.workunit.client.1.smithi178.stdout:2/252: creat d1/d41/d14/d11/d19/f4f x:0 0 0 2024-09-10T11:29:22.303 INFO:tasks.workunit.client.1.smithi178.stdout:7/28: dread f0 [0,4194304] 0 2024-09-10T11:29:22.303 INFO:tasks.workunit.client.1.smithi178.stdout:2/253: mknod d1/d7/d24/d43/c50 0 2024-09-10T11:29:22.303 INFO:tasks.workunit.client.1.smithi178.stdout:2/254: stat d1/d7/d24/d43/c49 0 2024-09-10T11:29:22.306 INFO:tasks.workunit.client.1.smithi178.stdout:2/255: creat d1/d41/f51 x:0 0 0 2024-09-10T11:29:22.307 INFO:tasks.workunit.client.0.smithi144.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-09-10T11:29:22.307 INFO:tasks.workunit.client.1.smithi178.stdout:2/256: mknod d1/d41/d14/d11/c52 0 2024-09-10T11:29:22.311 INFO:tasks.workunit.client.1.smithi178.stdout:7/29: dwrite f4 [0,4194304] 0 2024-09-10T11:29:22.311 INFO:tasks.workunit.client.1.smithi178.stdout:2/257: dread d1/d41/d14/d11/d19/f35 [0,4194304] 0 2024-09-10T11:29:22.312 INFO:tasks.workunit.client.1.smithi178.stdout:7/30: read f2 [610154,30299] 0 2024-09-10T11:29:22.313 INFO:tasks.workunit.client.1.smithi178.stdout:7/31: truncate f4 5241820 0 2024-09-10T11:29:22.315 INFO:tasks.workunit.client.1.smithi178.stdout:2/258: mkdir d1/d41/d14/d11/d53 0 2024-09-10T11:29:22.319 INFO:tasks.workunit.client.1.smithi178.stdout:2/259: unlink d1/d41/d14/d11/d19/d34/f47 0 2024-09-10T11:29:22.327 INFO:tasks.workunit.client.1.smithi178.stdout:7/32: dwrite f3 [0,4194304] 0 2024-09-10T11:29:22.328 INFO:tasks.workunit.client.1.smithi178.stdout:7/33: mknod d6/c8 0 2024-09-10T11:29:22.331 INFO:tasks.workunit.client.1.smithi178.stdout:2/260: truncate d1/d41/d14/d11/d19/f36 347742 0 2024-09-10T11:29:22.331 INFO:tasks.workunit.client.1.smithi178.stdout:2/261: fdatasync d1/d41/f51 0 2024-09-10T11:29:22.335 INFO:tasks.workunit.client.1.smithi178.stdout:2/262: dread d1/f3d [0,4194304] 0 2024-09-10T11:29:22.337 INFO:tasks.workunit.client.1.smithi178.stdout:2/263: mkdir d1/d41/d14/d11/d19/d54 0 2024-09-10T11:29:22.338 INFO:tasks.workunit.client.1.smithi178.stdout:3/104: sync 2024-09-10T11:29:22.339 INFO:tasks.workunit.client.1.smithi178.stdout:0/69: sync 2024-09-10T11:29:22.341 INFO:tasks.workunit.client.1.smithi178.stdout:0/70: creat d0/db/f15 x:0 0 0 2024-09-10T11:29:22.342 INFO:tasks.workunit.client.1.smithi178.stdout:3/105: symlink db/d20/l23 0 2024-09-10T11:29:22.343 INFO:tasks.workunit.client.1.smithi178.stdout:2/264: dwrite d1/d7/f16 [0,4194304] 0 2024-09-10T11:29:22.344 INFO:tasks.workunit.client.1.smithi178.stdout:3/106: symlink db/l24 0 2024-09-10T11:29:22.347 INFO:tasks.workunit.client.1.smithi178.stdout:0/71: rename d0/f6 to d0/dc/f16 0 2024-09-10T11:29:22.349 INFO:tasks.workunit.client.1.smithi178.stdout:2/265: mkdir d1/d41/d14/d11/d19/d55 0 2024-09-10T11:29:22.349 INFO:tasks.workunit.client.1.smithi178.stdout:7/34: dwrite f2 [0,4194304] 0 2024-09-10T11:29:22.355 INFO:tasks.workunit.client.1.smithi178.stdout:3/107: rename db/lc to db/d20/l25 0 2024-09-10T11:29:22.356 INFO:tasks.workunit.client.1.smithi178.stdout:7/35: symlink d6/l9 0 2024-09-10T11:29:22.359 INFO:tasks.workunit.client.1.smithi178.stdout:3/108: mkdir db/d26 0 2024-09-10T11:29:22.360 INFO:tasks.workunit.client.1.smithi178.stdout:7/36: symlink d6/la 0 2024-09-10T11:29:22.363 INFO:tasks.workunit.client.1.smithi178.stdout:2/266: dwrite d1/d41/d14/d11/d19/f37 [0,4194304] 0 2024-09-10T11:29:22.365 INFO:tasks.workunit.client.1.smithi178.stdout:7/37: creat d6/fb x:0 0 0 2024-09-10T11:29:22.365 INFO:tasks.workunit.client.1.smithi178.stdout:2/267: read d1/d41/d14/d11/d19/d34/f29 [118221,125848] 0 2024-09-10T11:29:22.367 INFO:tasks.workunit.client.1.smithi178.stdout:2/268: symlink d1/d41/d14/d11/l56 0 2024-09-10T11:29:22.368 INFO:tasks.workunit.client.1.smithi178.stdout:7/38: creat d6/fc x:0 0 0 2024-09-10T11:29:22.371 INFO:tasks.workunit.client.1.smithi178.stdout:0/72: truncate d0/dc/f16 213383 0 2024-09-10T11:29:22.371 INFO:tasks.workunit.client.1.smithi178.stdout:0/73: stat d0/ca 0 2024-09-10T11:29:22.374 INFO:tasks.workunit.client.1.smithi178.stdout:7/39: rename f2 to d6/fd 0 2024-09-10T11:29:22.375 INFO:tasks.workunit.client.1.smithi178.stdout:2/269: creat d1/d41/d14/d11/d19/d4b/f57 x:0 0 0 2024-09-10T11:29:22.376 INFO:tasks.workunit.client.1.smithi178.stdout:3/109: getdents db/d20 0 2024-09-10T11:29:22.377 INFO:tasks.workunit.client.1.smithi178.stdout:7/40: mknod d6/ce 0 2024-09-10T11:29:22.379 INFO:tasks.workunit.client.1.smithi178.stdout:2/270: creat d1/d41/d14/d11/d19/d34/f58 x:0 0 0 2024-09-10T11:29:22.379 INFO:tasks.workunit.client.1.smithi178.stdout:2/271: chown d1/d41/f1e 1 1 2024-09-10T11:29:22.381 INFO:tasks.workunit.client.1.smithi178.stdout:7/41: unlink f0 0 2024-09-10T11:29:22.382 INFO:tasks.workunit.client.1.smithi178.stdout:3/110: fsync db/ff 0 2024-09-10T11:29:22.385 INFO:tasks.workunit.client.1.smithi178.stdout:0/74: dread d0/f1 [0,4194304] 0 2024-09-10T11:29:22.385 INFO:tasks.workunit.client.1.smithi178.stdout:3/111: write db/d20/f21 [1127559,21102] 0 2024-09-10T11:29:22.386 INFO:tasks.workunit.client.1.smithi178.stdout:3/112: fsync db/f1a 0 2024-09-10T11:29:22.388 INFO:tasks.workunit.client.1.smithi178.stdout:2/272: dwrite d1/d41/d14/f3e [0,4194304] 0 2024-09-10T11:29:22.388 INFO:tasks.workunit.client.1.smithi178.stdout:7/42: link d6/fb d6/ff 0 2024-09-10T11:29:22.390 INFO:tasks.workunit.client.1.smithi178.stdout:2/273: truncate d1/d22/f3a 681509 0 2024-09-10T11:29:22.390 INFO:tasks.workunit.client.1.smithi178.stdout:0/75: mknod d0/db/d14/c17 0 2024-09-10T11:29:22.390 INFO:tasks.workunit.client.1.smithi178.stdout:7/43: mkdir d6/d10 0 2024-09-10T11:29:22.391 INFO:tasks.workunit.client.1.smithi178.stdout:7/44: write f3 [2088281,126929] 0 2024-09-10T11:29:22.395 INFO:tasks.workunit.client.1.smithi178.stdout:2/274: rename d1/d7/d24/d46/f48 to d1/d41/d14/f59 0 2024-09-10T11:29:22.397 INFO:tasks.workunit.client.1.smithi178.stdout:7/45: write d6/ff [798651,27885] 0 2024-09-10T11:29:22.400 INFO:tasks.workunit.client.1.smithi178.stdout:0/76: dwrite d0/dc/f13 [0,4194304] 0 2024-09-10T11:29:22.401 INFO:tasks.workunit.client.1.smithi178.stdout:7/46: creat d6/f11 x:0 0 0 2024-09-10T11:29:22.402 INFO:tasks.workunit.client.1.smithi178.stdout:7/47: write d6/f11 [33895,82346] 0 2024-09-10T11:29:22.405 INFO:tasks.workunit.client.1.smithi178.stdout:0/77: mknod d0/d12/c18 0 2024-09-10T11:29:22.406 INFO:tasks.workunit.client.1.smithi178.stdout:7/48: write f4 [136361,7865] 0 2024-09-10T11:29:22.408 INFO:tasks.workunit.client.1.smithi178.stdout:7/49: stat d6/c7 0 2024-09-10T11:29:22.408 INFO:tasks.workunit.client.1.smithi178.stdout:2/275: dwrite d1/d41/f51 [0,4194304] 0 2024-09-10T11:29:22.409 INFO:tasks.workunit.client.1.smithi178.stdout:0/78: rename d0/db/d14 to d0/d19 0 2024-09-10T11:29:22.409 INFO:tasks.workunit.client.1.smithi178.stdout:0/79: fdatasync d0/f8 0 2024-09-10T11:29:22.410 INFO:tasks.workunit.client.1.smithi178.stdout:7/50: mknod d6/c12 0 2024-09-10T11:29:22.411 INFO:tasks.workunit.client.1.smithi178.stdout:7/51: dread - d6/fc zero size 2024-09-10T11:29:22.414 INFO:tasks.workunit.client.1.smithi178.stdout:0/80: rename d0/c4 to d0/db/c1a 0 2024-09-10T11:29:22.416 INFO:tasks.workunit.client.1.smithi178.stdout:2/276: dwrite d1/d22/f3a [0,4194304] 0 2024-09-10T11:29:22.421 INFO:tasks.workunit.client.1.smithi178.stdout:2/277: rename d1/f3 to d1/d7/d24/f5a 0 2024-09-10T11:29:22.422 INFO:tasks.workunit.client.1.smithi178.stdout:0/81: chown d0/db/c1a 352555 1 2024-09-10T11:29:22.424 INFO:tasks.workunit.client.1.smithi178.stdout:0/82: chown d0/dc/f16 72001 1 2024-09-10T11:29:22.425 INFO:tasks.workunit.client.1.smithi178.stdout:0/83: write d0/dc/f13 [1474262,55063] 0 2024-09-10T11:29:22.428 INFO:tasks.workunit.client.1.smithi178.stdout:2/278: mkdir d1/d41/d14/d11/d19/d34/d5b 0 2024-09-10T11:29:22.430 INFO:tasks.workunit.client.1.smithi178.stdout:2/279: read d1/d22/f3a [1790255,32133] 0 2024-09-10T11:29:22.431 INFO:tasks.workunit.client.1.smithi178.stdout:3/113: dread f7 [0,4194304] 0 2024-09-10T11:29:22.433 INFO:tasks.workunit.client.1.smithi178.stdout:0/84: dwrite d0/dc/f13 [0,4194304] 0 2024-09-10T11:29:22.433 INFO:tasks.workunit.client.1.smithi178.stdout:3/114: write db/f18 [2155964,14509] 0 2024-09-10T11:29:22.434 INFO:tasks.workunit.client.1.smithi178.stdout:3/115: dread - db/f1a zero size 2024-09-10T11:29:22.435 INFO:tasks.workunit.client.1.smithi178.stdout:2/280: creat d1/d41/d14/d11/d19/d55/f5c x:0 0 0 2024-09-10T11:29:22.435 INFO:tasks.workunit.client.1.smithi178.stdout:0/85: mknod d0/d19/c1b 0 2024-09-10T11:29:22.437 INFO:tasks.workunit.client.1.smithi178.stdout:0/86: write d0/f7 [4757262,118707] 0 2024-09-10T11:29:22.437 INFO:tasks.workunit.client.1.smithi178.stdout:3/116: creat db/f27 x:0 0 0 2024-09-10T11:29:22.438 INFO:tasks.workunit.client.1.smithi178.stdout:2/281: mkdir d1/d41/d5d 0 2024-09-10T11:29:22.439 INFO:tasks.workunit.client.1.smithi178.stdout:3/117: chown db/f10 10691502 1 2024-09-10T11:29:22.439 INFO:tasks.workunit.client.1.smithi178.stdout:0/87: write d0/dc/ff [254759,93165] 0 2024-09-10T11:29:22.442 INFO:tasks.workunit.client.1.smithi178.stdout:2/282: mkdir d1/d41/d14/d11/d19/d34/d5e 0 2024-09-10T11:29:22.442 INFO:tasks.workunit.client.1.smithi178.stdout:0/88: fsync d0/dc/ff 0 2024-09-10T11:29:22.443 INFO:tasks.workunit.client.1.smithi178.stdout:0/89: write d0/dc/f13 [3597387,121931] 0 2024-09-10T11:29:22.444 INFO:tasks.workunit.client.1.smithi178.stdout:0/90: dread - d0/f9 zero size 2024-09-10T11:29:22.446 INFO:tasks.workunit.client.1.smithi178.stdout:2/283: dread d1/d41/f51 [0,4194304] 0 2024-09-10T11:29:22.447 INFO:tasks.workunit.client.1.smithi178.stdout:0/91: creat d0/dc/f1c x:0 0 0 2024-09-10T11:29:22.448 INFO:tasks.workunit.client.1.smithi178.stdout:0/92: write d0/dc/f13 [13998,99481] 0 2024-09-10T11:29:22.449 INFO:tasks.workunit.client.1.smithi178.stdout:2/284: unlink d1/d41/d14/d11/c52 0 2024-09-10T11:29:22.449 INFO:tasks.workunit.client.1.smithi178.stdout:0/93: write d0/dc/ff [522570,47137] 0 2024-09-10T11:29:22.450 INFO:tasks.workunit.client.1.smithi178.stdout:0/94: write d0/f9 [1032194,26192] 0 2024-09-10T11:29:22.451 INFO:tasks.workunit.client.1.smithi178.stdout:0/95: dread - d0/f8 zero size 2024-09-10T11:29:22.452 INFO:tasks.workunit.client.1.smithi178.stdout:2/285: symlink d1/d41/d14/d11/d19/d34/d5e/l5f 0 2024-09-10T11:29:22.453 INFO:tasks.workunit.client.1.smithi178.stdout:2/286: chown d1/d41/f1f 92158 1 2024-09-10T11:29:22.453 INFO:tasks.workunit.client.1.smithi178.stdout:0/96: creat d0/d19/f1d x:0 0 0 2024-09-10T11:29:22.455 INFO:tasks.workunit.client.1.smithi178.stdout:2/287: truncate d1/d41/f31 177952 0 2024-09-10T11:29:22.456 INFO:tasks.workunit.client.1.smithi178.stdout:0/97: creat d0/d19/f1e x:0 0 0 2024-09-10T11:29:22.456 INFO:tasks.workunit.client.1.smithi178.stdout:0/98: stat d0/d19 0 2024-09-10T11:29:22.457 INFO:tasks.workunit.client.1.smithi178.stdout:3/118: dwrite db/fe [0,4194304] 0 2024-09-10T11:29:22.458 INFO:tasks.workunit.client.1.smithi178.stdout:0/99: dread - d0/d19/f1d zero size 2024-09-10T11:29:22.459 INFO:tasks.workunit.client.1.smithi178.stdout:2/288: mkdir d1/d41/d14/d11/d19/d34/d5b/d60 0 2024-09-10T11:29:22.464 INFO:tasks.workunit.client.1.smithi178.stdout:0/100: dwrite d0/f7 [0,4194304] 0 2024-09-10T11:29:22.464 INFO:tasks.workunit.client.1.smithi178.stdout:2/289: rename d1/d7/d24/d43/c49 to d1/d22/c61 0 2024-09-10T11:29:22.465 INFO:tasks.workunit.client.1.smithi178.stdout:0/101: chown d0/f9 261 1 2024-09-10T11:29:22.468 INFO:tasks.workunit.client.1.smithi178.stdout:3/119: link f1 db/d20/f28 0 2024-09-10T11:29:22.468 INFO:tasks.workunit.client.1.smithi178.stdout:0/102: dread d0/f9 [0,4194304] 0 2024-09-10T11:29:22.469 INFO:tasks.workunit.client.1.smithi178.stdout:0/103: stat d0/d12/c18 0 2024-09-10T11:29:22.469 INFO:tasks.workunit.client.1.smithi178.stdout:0/104: read d0/f7 [39724,85977] 0 2024-09-10T11:29:22.470 INFO:tasks.workunit.client.1.smithi178.stdout:2/290: creat d1/d41/d14/d11/d53/f62 x:0 0 0 2024-09-10T11:29:22.471 INFO:tasks.workunit.client.1.smithi178.stdout:2/291: write d1/d41/d14/d11/d19/f4f [355948,91946] 0 2024-09-10T11:29:22.472 INFO:tasks.workunit.client.1.smithi178.stdout:0/105: mkdir d0/dc/d1f 0 2024-09-10T11:29:22.473 INFO:tasks.workunit.client.1.smithi178.stdout:6/200: sync 2024-09-10T11:29:22.473 INFO:tasks.workunit.client.1.smithi178.stdout:0/106: write d0/f8 [544269,61182] 0 2024-09-10T11:29:22.474 INFO:tasks.workunit.client.1.smithi178.stdout:2/292: unlink d1/c20 0 2024-09-10T11:29:22.475 INFO:tasks.workunit.client.1.smithi178.stdout:6/201: write d3/d7/d8/d15/d1c/d2a/f41 [794144,121974] 0 2024-09-10T11:29:22.479 INFO:tasks.workunit.client.1.smithi178.stdout:6/202: rename d3/d7/d8/f1f to d3/d7/d35/f4c 0 2024-09-10T11:29:22.482 INFO:tasks.workunit.client.1.smithi178.stdout:6/203: read d3/f14 [184788,71028] 0 2024-09-10T11:29:22.483 INFO:tasks.workunit.client.1.smithi178.stdout:0/107: dwrite d0/f7 [0,4194304] 0 2024-09-10T11:29:22.483 INFO:tasks.workunit.client.1.smithi178.stdout:2/293: dread d1/d7/d24/f2c [0,4194304] 0 2024-09-10T11:29:22.484 INFO:tasks.workunit.client.1.smithi178.stdout:6/204: read - d3/d7/d8/d15/d1c/d2a/d18/f43 zero size 2024-09-10T11:29:22.485 INFO:tasks.workunit.client.1.smithi178.stdout:0/108: mknod d0/d12/c20 0 2024-09-10T11:29:22.486 INFO:tasks.workunit.client.1.smithi178.stdout:0/109: fsync d0/dc/ff 0 2024-09-10T11:29:22.487 INFO:tasks.workunit.client.1.smithi178.stdout:0/110: chown d0/d19 120874 1 2024-09-10T11:29:22.571 INFO:tasks.workunit.client.1.smithi178.stdout:6/205: creat d3/d7/d8/d15/d1c/f4d x:0 0 0 2024-09-10T11:29:22.572 INFO:tasks.workunit.client.1.smithi178.stdout:0/111: creat d0/dc/f21 x:0 0 0 2024-09-10T11:29:22.572 INFO:tasks.workunit.client.1.smithi178.stdout:2/294: dwrite d1/d41/d14/d11/d19/f4f [0,4194304] 0 2024-09-10T11:29:22.575 INFO:tasks.workunit.client.1.smithi178.stdout:0/112: read - d0/db/f15 zero size 2024-09-10T11:29:22.575 INFO:tasks.workunit.client.1.smithi178.stdout:6/206: dread - d3/d7/d8/d15/d1c/f4d zero size 2024-09-10T11:29:22.579 INFO:tasks.workunit.client.1.smithi178.stdout:2/295: rename d1/d41/d14/d11/d19/d34/f58 to d1/d7/d24/d46/f63 0 2024-09-10T11:29:22.585 INFO:tasks.workunit.client.1.smithi178.stdout:6/207: dwrite d3/d7/d8/d15/d1c/d20/f47 [0,4194304] 0 2024-09-10T11:29:22.589 INFO:tasks.workunit.client.1.smithi178.stdout:2/296: link d1/d41/d14/d11/d19/f36 d1/d41/d14/d11/d19/d34/d5b/d60/f64 0 2024-09-10T11:29:22.591 INFO:tasks.workunit.client.1.smithi178.stdout:6/208: mkdir d3/d4e 0 2024-09-10T11:29:22.593 INFO:tasks.workunit.client.1.smithi178.stdout:2/297: mknod d1/d41/d14/d11/c65 0 2024-09-10T11:29:22.594 INFO:tasks.workunit.client.1.smithi178.stdout:2/298: read d1/d41/d14/d11/f4a [605684,40385] 0 2024-09-10T11:29:22.601 INFO:tasks.workunit.client.1.smithi178.stdout:2/299: link d1/d22/f30 d1/d41/d14/d11/d19/d4b/f66 0 2024-09-10T11:29:22.601 INFO:tasks.workunit.client.1.smithi178.stdout:2/300: readlink d1/d41/d14/d11/lb 0 2024-09-10T11:29:22.604 INFO:tasks.workunit.client.1.smithi178.stdout:6/209: dread d3/d7/d8/d15/f22 [0,4194304] 0 2024-09-10T11:29:22.608 INFO:tasks.workunit.client.1.smithi178.stdout:2/301: dwrite d1/d7/d24/f5a [4194304,4194304] 0 2024-09-10T11:29:22.617 INFO:tasks.workunit.client.1.smithi178.stdout:6/210: dread d3/d7/d8/d15/d1c/d2a/f3c [0,4194304] 0 2024-09-10T11:29:22.618 INFO:tasks.workunit.client.1.smithi178.stdout:6/211: chown d3/d24 4407717 1 2024-09-10T11:29:22.619 INFO:tasks.workunit.client.1.smithi178.stdout:6/212: fsync f0 0 2024-09-10T11:29:22.621 INFO:tasks.workunit.client.1.smithi178.stdout:2/302: dwrite d1/d41/d14/d11/d19/d34/f29 [0,4194304] 0 2024-09-10T11:29:22.622 INFO:tasks.workunit.client.1.smithi178.stdout:6/213: unlink d3/d24/c34 0 2024-09-10T11:29:22.623 INFO:tasks.workunit.client.1.smithi178.stdout:5/83: sync 2024-09-10T11:29:22.624 INFO:tasks.workunit.client.1.smithi178.stdout:5/84: write d9/d19/f1a [96082,49397] 0 2024-09-10T11:29:22.626 INFO:tasks.workunit.client.1.smithi178.stdout:2/303: write d1/d41/d14/f3b [704205,24048] 0 2024-09-10T11:29:22.630 INFO:tasks.workunit.client.1.smithi178.stdout:5/85: rename f5 to d9/dc/dd/d16/f27 0 2024-09-10T11:29:22.633 INFO:tasks.workunit.client.1.smithi178.stdout:2/304: dwrite d1/f3d [0,4194304] 0 2024-09-10T11:29:22.636 INFO:tasks.workunit.client.1.smithi178.stdout:2/305: creat d1/d41/d14/d11/d19/d4b/f67 x:0 0 0 2024-09-10T11:29:22.637 INFO:tasks.workunit.client.1.smithi178.stdout:5/86: truncate d9/ff 2365739 0 2024-09-10T11:29:22.638 INFO:tasks.workunit.client.1.smithi178.stdout:5/87: stat d9/dc/d17/d1d 0 2024-09-10T11:29:22.638 INFO:tasks.workunit.client.1.smithi178.stdout:6/214: truncate d3/d7/d8/d15/d1c/f45 1383854 0 2024-09-10T11:29:22.642 INFO:tasks.workunit.client.1.smithi178.stdout:5/88: rename d9/d11/f18 to d9/f28 0 2024-09-10T11:29:22.643 INFO:tasks.workunit.client.1.smithi178.stdout:6/215: creat d3/d7/d8/d15/d1c/d2a/d18/f4f x:0 0 0 2024-09-10T11:29:22.643 INFO:tasks.workunit.client.1.smithi178.stdout:2/306: dwrite d1/d41/d14/d11/d19/f2d [0,4194304] 0 2024-09-10T11:29:22.644 INFO:tasks.workunit.client.1.smithi178.stdout:5/89: rmdir d9 39 2024-09-10T11:29:22.645 INFO:tasks.workunit.client.1.smithi178.stdout:2/307: write d1/d41/d14/d11/d19/f2d [2427160,99286] 0 2024-09-10T11:29:22.648 INFO:tasks.workunit.client.1.smithi178.stdout:2/308: fsync d1/d41/d14/f23 0 2024-09-10T11:29:22.649 INFO:tasks.workunit.client.1.smithi178.stdout:5/90: rename d9/dc/c20 to d9/dc/d17/c29 0 2024-09-10T11:29:22.657 INFO:tasks.workunit.client.1.smithi178.stdout:5/91: dwrite f2 [0,4194304] 0 2024-09-10T11:29:22.659 INFO:tasks.workunit.client.1.smithi178.stdout:6/216: dread d3/d7/d8/d15/d1c/d2a/f2c [0,4194304] 0 2024-09-10T11:29:22.666 INFO:tasks.workunit.client.1.smithi178.stdout:5/92: dwrite f2 [0,4194304] 0 2024-09-10T11:29:22.669 INFO:tasks.workunit.client.1.smithi178.stdout:6/217: dwrite d3/d7/d8/d15/d1c/d20/f47 [4194304,4194304] 0 2024-09-10T11:29:22.669 INFO:tasks.workunit.client.1.smithi178.stdout:5/93: mknod d9/dc/c2a 0 2024-09-10T11:29:22.672 INFO:tasks.workunit.client.1.smithi178.stdout:5/94: read d9/d19/f1a [277440,107732] 0 2024-09-10T11:29:22.674 INFO:tasks.workunit.client.1.smithi178.stdout:6/218: unlink d3/d7/d8/d15/d1c/c4a 0 2024-09-10T11:29:22.674 INFO:tasks.workunit.client.1.smithi178.stdout:5/95: creat d9/dc/f2b x:0 0 0 2024-09-10T11:29:22.675 INFO:tasks.workunit.client.1.smithi178.stdout:6/219: chown d3/f3a 563 1 2024-09-10T11:29:22.676 INFO:tasks.workunit.client.1.smithi178.stdout:5/96: chown d9/d19/l21 31915576 1 2024-09-10T11:29:22.677 INFO:tasks.workunit.client.1.smithi178.stdout:6/220: dread - d3/d7/d8/d15/d1c/f4d zero size 2024-09-10T11:29:22.677 INFO:tasks.workunit.client.1.smithi178.stdout:5/97: creat d9/d19/f2c x:0 0 0 2024-09-10T11:29:22.678 INFO:tasks.workunit.client.1.smithi178.stdout:6/221: rename d3 to d3/d7/d35/d50 22 2024-09-10T11:29:22.680 INFO:tasks.workunit.client.1.smithi178.stdout:5/98: unlink c8 0 2024-09-10T11:29:22.681 INFO:tasks.workunit.client.1.smithi178.stdout:6/222: creat d3/d7/d8/d15/d1c/d2a/f51 x:0 0 0 2024-09-10T11:29:22.682 INFO:tasks.workunit.client.1.smithi178.stdout:5/99: creat d9/dc/dd/f2d x:0 0 0 2024-09-10T11:29:22.682 INFO:tasks.workunit.client.1.smithi178.stdout:6/223: write d3/d7/d8/d15/d1c/d2a/f2c [456688,86166] 0 2024-09-10T11:29:22.686 INFO:tasks.workunit.client.1.smithi178.stdout:6/224: unlink d3/d7/d8/d15/d1c/d20/f28 0 2024-09-10T11:29:22.688 INFO:tasks.workunit.client.1.smithi178.stdout:5/100: link d9/d11/l13 d9/dc/dd/d16/l2e 0 2024-09-10T11:29:22.690 INFO:tasks.workunit.client.1.smithi178.stdout:5/101: rename d9/dc/dd/d16/f1b to d9/d11/f2f 0 2024-09-10T11:29:22.692 INFO:tasks.workunit.client.1.smithi178.stdout:5/102: creat d9/f30 x:0 0 0 2024-09-10T11:29:22.693 INFO:tasks.workunit.client.1.smithi178.stdout:6/225: link d3/d7/d8/d15/f25 d3/d7/d8/d15/d1c/d20/d4b/f52 0 2024-09-10T11:29:22.693 INFO:tasks.workunit.client.1.smithi178.stdout:5/103: rename d9/d11 to d9/d11/d31 22 2024-09-10T11:29:22.694 INFO:tasks.workunit.client.1.smithi178.stdout:5/104: creat d9/f32 x:0 0 0 2024-09-10T11:29:22.696 INFO:tasks.workunit.client.1.smithi178.stdout:5/105: mknod d9/d19/c33 0 2024-09-10T11:29:22.697 INFO:tasks.workunit.client.1.smithi178.stdout:5/106: truncate d9/d11/f2f 1203799 0 2024-09-10T11:29:22.699 INFO:tasks.workunit.client.1.smithi178.stdout:6/226: dwrite d3/f3a [0,4194304] 0 2024-09-10T11:29:22.709 INFO:tasks.workunit.client.1.smithi178.stdout:6/227: dwrite d3/d7/d8/d15/d1c/d2a/f3c [0,4194304] 0 2024-09-10T11:29:22.709 INFO:tasks.workunit.client.1.smithi178.stdout:5/107: rmdir d9/dc 39 2024-09-10T11:29:22.709 INFO:tasks.workunit.client.1.smithi178.stdout:5/108: dread - d9/d19/f2c zero size 2024-09-10T11:29:22.709 INFO:tasks.workunit.client.1.smithi178.stdout:6/228: creat d3/d7/d8/d15/d1c/f53 x:0 0 0 2024-09-10T11:29:22.709 INFO:tasks.workunit.client.1.smithi178.stdout:6/229: chown d3/d7/d35 52861028 1 2024-09-10T11:29:22.710 INFO:tasks.workunit.client.1.smithi178.stdout:6/230: write f0 [5144679,23334] 0 2024-09-10T11:29:22.711 INFO:tasks.workunit.client.1.smithi178.stdout:5/109: chown d9/dc/d17/f1e 2921273 1 2024-09-10T11:29:22.720 INFO:tasks.workunit.client.1.smithi178.stdout:6/231: dwrite f0 [8388608,4194304] 0 2024-09-10T11:29:22.720 INFO:tasks.workunit.client.1.smithi178.stdout:5/110: dwrite d9/d19/f1a [0,4194304] 0 2024-09-10T11:29:22.723 INFO:tasks.workunit.client.1.smithi178.stdout:5/111: creat d9/dc/d17/f34 x:0 0 0 2024-09-10T11:29:22.725 INFO:tasks.workunit.client.1.smithi178.stdout:5/112: fsync d9/dc/d17/f1e 0 2024-09-10T11:29:22.726 INFO:tasks.workunit.client.1.smithi178.stdout:5/113: rename d9/dc to d9/dc/d17/d35 22 2024-09-10T11:29:22.727 INFO:tasks.workunit.client.1.smithi178.stdout:5/114: chown d9/dc/dd/ce 48 1 2024-09-10T11:29:22.730 INFO:tasks.workunit.client.1.smithi178.stdout:5/115: unlink d9/d11/l13 0 2024-09-10T11:29:22.731 INFO:tasks.workunit.client.1.smithi178.stdout:6/232: truncate d3/f3a 1124523 0 2024-09-10T11:29:22.732 INFO:tasks.workunit.client.1.smithi178.stdout:5/116: rename d9/d19/l21 to d9/d11/l36 0 2024-09-10T11:29:22.734 INFO:tasks.workunit.client.1.smithi178.stdout:5/117: write d9/d11/f2f [1007899,36328] 0 2024-09-10T11:29:22.735 INFO:tasks.workunit.client.1.smithi178.stdout:5/118: chown d9/f30 7434 1 2024-09-10T11:29:22.737 INFO:tasks.workunit.client.1.smithi178.stdout:5/119: fsync d9/ff 0 2024-09-10T11:29:22.739 INFO:tasks.workunit.client.1.smithi178.stdout:6/233: dwrite d3/d7/d8/d15/d1c/f40 [0,4194304] 0 2024-09-10T11:29:22.746 INFO:tasks.workunit.client.1.smithi178.stdout:5/120: dwrite d9/d19/f1a [0,4194304] 0 2024-09-10T11:29:22.752 INFO:tasks.workunit.client.1.smithi178.stdout:8/122: sync 2024-09-10T11:29:22.776 INFO:tasks.workunit.client.1.smithi178.stdout:5/121: mkdir d9/dc/dd/d37 0 2024-09-10T11:29:22.776 INFO:tasks.workunit.client.1.smithi178.stdout:5/122: fsync d9/f32 0 2024-09-10T11:29:22.778 INFO:tasks.workunit.client.1.smithi178.stdout:8/123: dread d5/db/f14 [0,4194304] 0 2024-09-10T11:29:22.779 INFO:tasks.workunit.client.1.smithi178.stdout:6/234: creat d3/d7/d8/d19/f54 x:0 0 0 2024-09-10T11:29:22.779 INFO:tasks.workunit.client.1.smithi178.stdout:5/123: symlink d9/dc/dd/l38 0 2024-09-10T11:29:22.780 INFO:tasks.workunit.client.1.smithi178.stdout:8/124: symlink d5/l27 0 2024-09-10T11:29:22.780 INFO:tasks.workunit.client.1.smithi178.stdout:5/124: write d9/dc/dd/d16/f1c [979110,88106] 0 2024-09-10T11:29:22.781 INFO:tasks.workunit.client.1.smithi178.stdout:5/125: truncate d9/d19/f2c 537378 0 2024-09-10T11:29:22.784 INFO:tasks.workunit.client.1.smithi178.stdout:6/235: dread d3/d7/d8/d15/d1c/d2a/f3c [0,4194304] 0 2024-09-10T11:29:22.784 INFO:tasks.workunit.client.1.smithi178.stdout:6/236: fdatasync f0 0 2024-09-10T11:29:22.786 INFO:tasks.workunit.client.1.smithi178.stdout:5/126: link d9/dc/dd/d16/l22 d9/dc/d17/l39 0 2024-09-10T11:29:22.788 INFO:tasks.workunit.client.1.smithi178.stdout:6/237: creat d3/d7/d8/d15/d1c/d20/d4b/f55 x:0 0 0 2024-09-10T11:29:22.788 INFO:tasks.workunit.client.1.smithi178.stdout:8/125: write d5/fa [440202,9958] 0 2024-09-10T11:29:22.790 INFO:tasks.workunit.client.1.smithi178.stdout:8/126: fdatasync d5/f1f 0 2024-09-10T11:29:22.793 INFO:tasks.workunit.client.1.smithi178.stdout:8/127: creat d5/db/d10/d20/f28 x:0 0 0 2024-09-10T11:29:22.794 INFO:tasks.workunit.client.1.smithi178.stdout:5/127: dwrite d9/dc/f2b [0,4194304] 0 2024-09-10T11:29:22.797 INFO:tasks.workunit.client.1.smithi178.stdout:8/128: symlink d5/db/d10/d20/l29 0 2024-09-10T11:29:22.798 INFO:tasks.workunit.client.1.smithi178.stdout:6/238: dwrite d3/d7/d8/d19/f54 [0,4194304] 0 2024-09-10T11:29:22.798 INFO:tasks.workunit.client.1.smithi178.stdout:8/129: readlink d5/db/d10/d20/l29 0 2024-09-10T11:29:22.802 INFO:tasks.workunit.client.1.smithi178.stdout:8/130: symlink d5/db/d10/l2a 0 2024-09-10T11:29:22.803 INFO:tasks.workunit.client.1.smithi178.stdout:5/128: dwrite f4 [4194304,4194304] 0 2024-09-10T11:29:22.806 INFO:tasks.workunit.client.1.smithi178.stdout:5/129: mknod d9/dc/d17/c3a 0 2024-09-10T11:29:22.809 INFO:tasks.workunit.client.1.smithi178.stdout:5/130: stat l3 0 2024-09-10T11:29:22.809 INFO:tasks.workunit.client.1.smithi178.stdout:8/131: getdents d5/d1d 0 2024-09-10T11:29:22.810 INFO:tasks.workunit.client.1.smithi178.stdout:6/239: dwrite d3/d7/d8/d15/d1c/d2a/d18/f38 [0,4194304] 0 2024-09-10T11:29:22.817 INFO:tasks.workunit.client.1.smithi178.stdout:8/132: write d5/db/d10/d20/f26 [4174740,76683] 0 2024-09-10T11:29:22.817 INFO:tasks.workunit.client.1.smithi178.stdout:6/240: mkdir d3/d7/d8/d56 0 2024-09-10T11:29:22.826 INFO:tasks.workunit.client.1.smithi178.stdout:6/241: dwrite d3/d7/d8/d15/d1c/d2a/d18/f4f [0,4194304] 0 2024-09-10T11:29:22.828 INFO:tasks.workunit.client.1.smithi178.stdout:3/120: sync 2024-09-10T11:29:22.828 INFO:tasks.workunit.client.1.smithi178.stdout:0/113: sync 2024-09-10T11:29:22.828 INFO:tasks.workunit.client.1.smithi178.stdout:7/52: sync 2024-09-10T11:29:22.828 INFO:tasks.workunit.client.1.smithi178.stdout:2/309: sync 2024-09-10T11:29:22.828 INFO:tasks.workunit.client.1.smithi178.stdout:4/63: sync 2024-09-10T11:29:22.828 INFO:tasks.workunit.client.1.smithi178.stdout:1/70: sync 2024-09-10T11:29:22.828 INFO:tasks.workunit.client.1.smithi178.stdout:9/293: sync 2024-09-10T11:29:22.829 INFO:tasks.workunit.client.1.smithi178.stdout:5/131: fsync f2 0 2024-09-10T11:29:22.829 INFO:tasks.workunit.client.1.smithi178.stdout:6/242: mkdir d3/d42/d57 0 2024-09-10T11:29:22.831 INFO:tasks.workunit.client.1.smithi178.stdout:3/121: write f8 [2473309,23996] 0 2024-09-10T11:29:22.831 INFO:tasks.workunit.client.1.smithi178.stdout:7/53: mknod d6/d10/c13 0 2024-09-10T11:29:22.833 INFO:tasks.workunit.client.1.smithi178.stdout:1/71: write d3/d9/fa [902479,17540] 0 2024-09-10T11:29:22.834 INFO:tasks.workunit.client.1.smithi178.stdout:9/294: unlink d2/d8/f34 0 2024-09-10T11:29:22.835 INFO:tasks.workunit.client.1.smithi178.stdout:4/64: truncate d0/d9/f12 80422 0 2024-09-10T11:29:22.836 INFO:tasks.workunit.client.1.smithi178.stdout:6/243: rmdir d3/d7/d8/d15/d1c/d20 39 2024-09-10T11:29:22.837 INFO:tasks.workunit.client.1.smithi178.stdout:3/122: rmdir db/d20 39 2024-09-10T11:29:22.839 INFO:tasks.workunit.client.1.smithi178.stdout:7/54: write d6/ff [1563969,27244] 0 2024-09-10T11:29:22.840 INFO:tasks.workunit.client.1.smithi178.stdout:6/244: chown f1 2132 1 2024-09-10T11:29:22.841 INFO:tasks.workunit.client.1.smithi178.stdout:3/123: read db/f1d [87395,61099] 0 2024-09-10T11:29:22.843 INFO:tasks.workunit.client.1.smithi178.stdout:9/295: mknod d2/df/d31/d37/c52 0 2024-09-10T11:29:22.845 INFO:tasks.workunit.client.1.smithi178.stdout:7/55: creat d6/f14 x:0 0 0 2024-09-10T11:29:22.848 INFO:tasks.workunit.client.1.smithi178.stdout:9/296: fsync d2/d8/fa 0 2024-09-10T11:29:22.851 INFO:tasks.workunit.client.1.smithi178.stdout:0/114: dwrite d0/dc/f21 [0,4194304] 0 2024-09-10T11:29:22.851 INFO:tasks.workunit.client.1.smithi178.stdout:4/65: link d0/l1 d0/d9/dc/l16 0 2024-09-10T11:29:22.852 INFO:tasks.workunit.client.1.smithi178.stdout:2/310: dwrite d1/d7/d24/f42 [0,4194304] 0 2024-09-10T11:29:22.853 INFO:tasks.workunit.client.1.smithi178.stdout:7/56: dread d6/f11 [0,4194304] 0 2024-09-10T11:29:22.855 INFO:tasks.workunit.client.1.smithi178.stdout:5/132: dread d9/dc/dd/d16/f27 [0,4194304] 0 2024-09-10T11:29:22.858 INFO:tasks.workunit.client.1.smithi178.stdout:1/72: dwrite d3/d9/fa [0,4194304] 0 2024-09-10T11:29:22.860 INFO:tasks.workunit.client.1.smithi178.stdout:7/57: unlink d6/f11 0 2024-09-10T11:29:22.861 INFO:tasks.workunit.client.1.smithi178.stdout:4/66: creat d0/d8/da/f17 x:0 0 0 2024-09-10T11:29:22.863 INFO:tasks.workunit.client.1.smithi178.stdout:5/133: symlink d9/dc/dd/d37/l3b 0 2024-09-10T11:29:22.863 INFO:tasks.workunit.client.1.smithi178.stdout:4/67: stat d0/l7 0 2024-09-10T11:29:22.864 INFO:tasks.workunit.client.1.smithi178.stdout:7/58: read d6/ff [1076696,115957] 0 2024-09-10T11:29:22.865 INFO:tasks.workunit.client.1.smithi178.stdout:0/115: fsync d0/dc/f21 0 2024-09-10T11:29:22.866 INFO:tasks.workunit.client.1.smithi178.stdout:4/68: write d0/d9/f13 [4795368,867] 0 2024-09-10T11:29:22.866 INFO:tasks.workunit.client.1.smithi178.stdout:9/297: rename d2/d19/l42 to d2/l53 0 2024-09-10T11:29:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:22 smithi178 ceph-mon[33514]: pgmap v18: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 20 MiB/s wr, 292 op/s 2024-09-10T11:29:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:22 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:29:22.869 INFO:tasks.workunit.client.1.smithi178.stdout:2/311: creat d1/d41/d14/d11/d19/d34/d5b/d60/f68 x:0 0 0 2024-09-10T11:29:22.870 INFO:tasks.workunit.client.1.smithi178.stdout:9/298: read d2/d8/f17 [3077872,81372] 0 2024-09-10T11:29:22.871 INFO:tasks.workunit.client.1.smithi178.stdout:3/124: dwrite db/d20/f28 [0,4194304] 0 2024-09-10T11:29:22.871 INFO:tasks.workunit.client.1.smithi178.stdout:7/59: mkdir d6/d10/d15 0 2024-09-10T11:29:22.873 INFO:tasks.workunit.client.1.smithi178.stdout:7/60: readlink d6/la 0 2024-09-10T11:29:22.873 INFO:tasks.workunit.client.1.smithi178.stdout:2/312: write d1/d41/f31 [1212286,81537] 0 2024-09-10T11:29:22.874 INFO:tasks.workunit.client.1.smithi178.stdout:0/116: rename d0/f7 to d0/f22 0 2024-09-10T11:29:22.876 INFO:tasks.workunit.client.1.smithi178.stdout:1/73: rename f1 to d3/d13/f15 0 2024-09-10T11:29:22.876 INFO:tasks.workunit.client.1.smithi178.stdout:4/69: creat d0/f18 x:0 0 0 2024-09-10T11:29:22.877 INFO:tasks.workunit.client.1.smithi178.stdout:9/299: mkdir d2/d19/d1c/d2d/d54 0 2024-09-10T11:29:22.878 INFO:tasks.workunit.client.1.smithi178.stdout:7/61: fsync f3 0 2024-09-10T11:29:22.879 INFO:tasks.workunit.client.1.smithi178.stdout:4/70: chown d0/d8/df 42 1 2024-09-10T11:29:22.879 INFO:tasks.workunit.client.1.smithi178.stdout:5/134: dwrite d9/dc/f2b [0,4194304] 0 2024-09-10T11:29:22.880 INFO:tasks.workunit.client.1.smithi178.stdout:2/313: truncate d1/d7/f16 5141795 0 2024-09-10T11:29:22.883 INFO:tasks.workunit.client.1.smithi178.stdout:9/300: unlink d2/d19/d1c/c23 0 2024-09-10T11:29:22.884 INFO:tasks.workunit.client.1.smithi178.stdout:7/62: fsync d6/ff 0 2024-09-10T11:29:22.888 INFO:tasks.workunit.client.1.smithi178.stdout:3/125: link db/c14 db/c29 0 2024-09-10T11:29:22.890 INFO:tasks.workunit.client.1.smithi178.stdout:9/301: creat d2/d19/d1c/d2d/f55 x:0 0 0 2024-09-10T11:29:22.890 INFO:tasks.workunit.client.1.smithi178.stdout:5/135: dread d9/dc/d17/f1e [0,4194304] 0 2024-09-10T11:29:22.892 INFO:tasks.workunit.client.1.smithi178.stdout:3/126: read f7 [2520240,102992] 0 2024-09-10T11:29:22.894 INFO:tasks.workunit.client.1.smithi178.stdout:1/74: dwrite d3/d13/f15 [0,4194304] 0 2024-09-10T11:29:22.895 INFO:tasks.workunit.client.1.smithi178.stdout:0/117: truncate d0/f9 367337 0 2024-09-10T11:29:22.896 INFO:tasks.workunit.client.1.smithi178.stdout:9/302: mkdir d2/d13/d40/d56 0 2024-09-10T11:29:22.897 INFO:tasks.workunit.client.1.smithi178.stdout:1/75: stat d3/c6 0 2024-09-10T11:29:22.898 INFO:tasks.workunit.client.1.smithi178.stdout:3/127: symlink db/d26/l2a 0 2024-09-10T11:29:22.900 INFO:tasks.workunit.client.1.smithi178.stdout:7/63: dread d6/fd [0,4194304] 0 2024-09-10T11:29:22.901 INFO:tasks.workunit.client.1.smithi178.stdout:9/303: symlink d2/d19/d1c/d2a/l57 0 2024-09-10T11:29:22.901 INFO:tasks.workunit.client.1.smithi178.stdout:4/71: dwrite d0/f5 [0,4194304] 0 2024-09-10T11:29:22.903 INFO:tasks.workunit.client.1.smithi178.stdout:7/64: fsync d6/ff 0 2024-09-10T11:29:22.904 INFO:tasks.workunit.client.1.smithi178.stdout:1/76: write d3/d9/f14 [280002,40780] 0 2024-09-10T11:29:22.905 INFO:tasks.workunit.client.1.smithi178.stdout:3/128: symlink db/l2b 0 2024-09-10T11:29:22.906 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:22 smithi144 ceph-mon[27942]: pgmap v18: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 20 MiB/s wr, 292 op/s 2024-09-10T11:29:22.906 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:22 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:29:22.906 INFO:tasks.workunit.client.1.smithi178.stdout:9/304: write d2/d13/f43 [701202,25137] 0 2024-09-10T11:29:22.907 INFO:tasks.workunit.client.1.smithi178.stdout:3/129: stat la 0 2024-09-10T11:29:22.907 INFO:tasks.workunit.client.1.smithi178.stdout:4/72: write d0/d9/fb [2757142,8052] 0 2024-09-10T11:29:22.907 INFO:tasks.workunit.client.1.smithi178.stdout:7/65: dread - d6/f14 zero size 2024-09-10T11:29:22.913 INFO:tasks.workunit.client.1.smithi178.stdout:5/136: dwrite f4 [0,4194304] 0 2024-09-10T11:29:22.913 INFO:tasks.workunit.client.1.smithi178.stdout:4/73: mknod d0/d10/c19 0 2024-09-10T11:29:22.914 INFO:tasks.workunit.client.1.smithi178.stdout:3/130: write db/f16 [662019,50247] 0 2024-09-10T11:29:22.916 INFO:tasks.workunit.client.1.smithi178.stdout:9/305: write d2/df/f3b [709793,107670] 0 2024-09-10T11:29:22.917 INFO:tasks.workunit.client.1.smithi178.stdout:7/66: creat d6/d10/d15/f16 x:0 0 0 2024-09-10T11:29:22.917 INFO:tasks.workunit.client.1.smithi178.stdout:0/118: dwrite d0/db/f15 [0,4194304] 0 2024-09-10T11:29:22.920 INFO:tasks.workunit.client.1.smithi178.stdout:0/119: read d0/f1 [120044,20987] 0 2024-09-10T11:29:22.923 INFO:tasks.workunit.client.1.smithi178.stdout:7/67: symlink d6/d10/l17 0 2024-09-10T11:29:22.923 INFO:tasks.workunit.client.1.smithi178.stdout:4/74: mkdir d0/d8/da/d1a 0 2024-09-10T11:29:22.927 INFO:tasks.workunit.client.1.smithi178.stdout:3/131: dread db/ff [0,4194304] 0 2024-09-10T11:29:22.927 INFO:tasks.workunit.client.1.smithi178.stdout:7/68: dread - d6/f14 zero size 2024-09-10T11:29:22.928 INFO:tasks.workunit.client.1.smithi178.stdout:1/77: dwrite d3/f8 [0,4194304] 0 2024-09-10T11:29:22.931 INFO:tasks.workunit.client.1.smithi178.stdout:0/120: getdents d0/dc/d1f 0 2024-09-10T11:29:22.932 INFO:tasks.workunit.client.1.smithi178.stdout:3/132: creat db/d26/f2c x:0 0 0 2024-09-10T11:29:22.934 INFO:tasks.workunit.client.1.smithi178.stdout:4/75: rename d0/d8/df/c15 to d0/d10/c1b 0 2024-09-10T11:29:22.934 INFO:tasks.workunit.client.1.smithi178.stdout:5/137: dwrite d9/f30 [0,4194304] 0 2024-09-10T11:29:22.937 INFO:tasks.workunit.client.1.smithi178.stdout:9/306: dwrite d2/dc/f2f [0,4194304] 0 2024-09-10T11:29:22.937 INFO:tasks.workunit.client.1.smithi178.stdout:0/121: unlink d0/dc/f1c 0 2024-09-10T11:29:22.942 INFO:tasks.workunit.client.1.smithi178.stdout:1/78: creat d3/de/f16 x:0 0 0 2024-09-10T11:29:22.943 INFO:tasks.workunit.client.1.smithi178.stdout:9/307: chown d2/d13/f15 226 1 2024-09-10T11:29:22.946 INFO:tasks.workunit.client.1.smithi178.stdout:1/79: stat d3/d9/fc 0 2024-09-10T11:29:22.947 INFO:tasks.workunit.client.1.smithi178.stdout:5/138: rename d9/dc/dd/f23 to d9/dc/d17/f3c 0 2024-09-10T11:29:22.948 INFO:tasks.workunit.client.1.smithi178.stdout:0/122: mknod d0/d19/c23 0 2024-09-10T11:29:22.950 INFO:tasks.workunit.client.1.smithi178.stdout:7/69: dwrite d6/fb [0,4194304] 0 2024-09-10T11:29:22.950 INFO:tasks.workunit.client.1.smithi178.stdout:9/308: chown d2/df/d31/d37/d3d 17410 1 2024-09-10T11:29:22.952 INFO:tasks.workunit.client.1.smithi178.stdout:7/70: dread - d6/fc zero size 2024-09-10T11:29:22.952 INFO:tasks.workunit.client.1.smithi178.stdout:4/76: dread d0/d9/fb [0,4194304] 0 2024-09-10T11:29:22.953 INFO:tasks.workunit.client.1.smithi178.stdout:3/133: dwrite db/f16 [0,4194304] 0 2024-09-10T11:29:22.956 INFO:tasks.workunit.client.1.smithi178.stdout:0/123: fdatasync d0/dc/f16 0 2024-09-10T11:29:22.956 INFO:tasks.workunit.client.1.smithi178.stdout:5/139: write d9/dc/dd/d16/f27 [2654287,29246] 0 2024-09-10T11:29:22.960 INFO:tasks.workunit.client.1.smithi178.stdout:3/134: truncate f7 4369724 0 2024-09-10T11:29:22.961 INFO:tasks.workunit.client.1.smithi178.stdout:4/77: write d0/d9/f12 [205379,121231] 0 2024-09-10T11:29:22.962 INFO:tasks.workunit.client.1.smithi178.stdout:9/309: write d2/f26 [2720549,97647] 0 2024-09-10T11:29:22.962 INFO:tasks.workunit.client.1.smithi178.stdout:1/80: link d3/d9/fa d3/d13/f17 0 2024-09-10T11:29:22.964 INFO:tasks.workunit.client.1.smithi178.stdout:1/81: chown d3/l4 4920 1 2024-09-10T11:29:22.967 INFO:tasks.workunit.client.1.smithi178.stdout:0/124: fsync d0/dc/f16 0 2024-09-10T11:29:22.969 INFO:tasks.workunit.client.1.smithi178.stdout:5/140: creat d9/dc/d17/f3d x:0 0 0 2024-09-10T11:29:22.969 INFO:tasks.workunit.client.1.smithi178.stdout:4/78: creat d0/f1c x:0 0 0 2024-09-10T11:29:22.972 INFO:tasks.workunit.client.1.smithi178.stdout:5/141: readlink d9/dc/dd/d16/l2e 0 2024-09-10T11:29:22.973 INFO:tasks.workunit.client.1.smithi178.stdout:1/82: mknod d3/de/c18 0 2024-09-10T11:29:22.975 INFO:tasks.workunit.client.1.smithi178.stdout:3/135: dwrite db/d26/f2c [0,4194304] 0 2024-09-10T11:29:22.977 INFO:tasks.workunit.client.1.smithi178.stdout:5/142: mkdir d9/dc/dd/d37/d3e 0 2024-09-10T11:29:22.980 INFO:tasks.workunit.client.1.smithi178.stdout:3/136: chown db/l24 1 1 2024-09-10T11:29:22.981 INFO:tasks.workunit.client.1.smithi178.stdout:7/71: getdents d6/d10 0 2024-09-10T11:29:22.981 INFO:tasks.workunit.client.1.smithi178.stdout:9/310: dwrite d2/d13/f2e [0,4194304] 0 2024-09-10T11:29:22.985 INFO:tasks.workunit.client.1.smithi178.stdout:7/72: write f3 [3048423,11660] 0 2024-09-10T11:29:22.985 INFO:tasks.workunit.client.1.smithi178.stdout:4/79: mknod d0/d9/dc/c1d 0 2024-09-10T11:29:22.986 INFO:tasks.workunit.client.1.smithi178.stdout:3/137: write f7 [1465989,32958] 0 2024-09-10T11:29:22.987 INFO:tasks.workunit.client.1.smithi178.stdout:7/73: dread - d6/f14 zero size 2024-09-10T11:29:22.990 INFO:tasks.workunit.client.1.smithi178.stdout:1/83: dread d3/d13/f17 [0,4194304] 0 2024-09-10T11:29:22.993 INFO:tasks.workunit.client.1.smithi178.stdout:5/143: dread f4 [4194304,4194304] 0 2024-09-10T11:29:22.995 INFO:tasks.workunit.client.1.smithi178.stdout:5/144: chown d9 212 1 2024-09-10T11:29:22.996 INFO:tasks.workunit.client.1.smithi178.stdout:0/125: dwrite d0/f22 [4194304,4194304] 0 2024-09-10T11:29:22.998 INFO:tasks.workunit.client.1.smithi178.stdout:8/133: sync 2024-09-10T11:29:23.002 INFO:tasks.workunit.client.1.smithi178.stdout:0/126: write d0/f22 [5779074,122335] 0 2024-09-10T11:29:23.003 INFO:tasks.workunit.client.1.smithi178.stdout:1/84: write d3/d9/f14 [1192880,90127] 0 2024-09-10T11:29:23.007 INFO:tasks.workunit.client.1.smithi178.stdout:0/127: rename d0/d19/f1e to d0/db/f24 0 2024-09-10T11:29:23.008 INFO:tasks.workunit.client.1.smithi178.stdout:8/134: stat d5/db/fe 0 2024-09-10T11:29:23.009 INFO:tasks.workunit.client.1.smithi178.stdout:4/80: truncate d0/d9/f13 7780928 0 2024-09-10T11:29:23.009 INFO:tasks.workunit.client.1.smithi178.stdout:7/74: dwrite d6/f14 [0,4194304] 0 2024-09-10T11:29:23.010 INFO:tasks.workunit.client.1.smithi178.stdout:1/85: mknod d3/de/c19 0 2024-09-10T11:29:23.012 INFO:tasks.workunit.client.1.smithi178.stdout:9/311: dread d2/d19/f39 [0,4194304] 0 2024-09-10T11:29:23.013 INFO:tasks.workunit.client.1.smithi178.stdout:5/145: write d9/f28 [999805,51301] 0 2024-09-10T11:29:23.013 INFO:tasks.workunit.client.1.smithi178.stdout:0/128: creat d0/d19/f25 x:0 0 0 2024-09-10T11:29:23.014 INFO:tasks.workunit.client.1.smithi178.stdout:1/86: mkdir d3/d9/d1a 0 2024-09-10T11:29:23.014 INFO:tasks.workunit.client.1.smithi178.stdout:3/138: dread db/fe [0,4194304] 0 2024-09-10T11:29:23.016 INFO:tasks.workunit.client.1.smithi178.stdout:3/139: stat db/d20/l23 0 2024-09-10T11:29:23.017 INFO:tasks.workunit.client.1.smithi178.stdout:0/129: write d0/d19/f1d [341870,42294] 0 2024-09-10T11:29:23.017 INFO:tasks.workunit.client.1.smithi178.stdout:3/140: chown db/l24 946815 1 2024-09-10T11:29:23.017 INFO:tasks.workunit.client.1.smithi178.stdout:3/141: readlink l9 0 2024-09-10T11:29:23.018 INFO:tasks.workunit.client.1.smithi178.stdout:7/75: rename d6/l9 to d6/l18 0 2024-09-10T11:29:23.019 INFO:tasks.workunit.client.1.smithi178.stdout:4/81: symlink d0/d8/da/d1a/l1e 0 2024-09-10T11:29:23.019 INFO:tasks.workunit.client.1.smithi178.stdout:9/312: truncate d2/d19/f39 8871726 0 2024-09-10T11:29:23.020 INFO:tasks.workunit.client.1.smithi178.stdout:1/87: symlink d3/l1b 0 2024-09-10T11:29:23.023 INFO:tasks.workunit.client.1.smithi178.stdout:7/76: mknod d6/d10/d15/c19 0 2024-09-10T11:29:23.025 INFO:tasks.workunit.client.1.smithi178.stdout:8/135: link d5/db/d10/d20/f26 d5/db/d10/d20/f2b 0 2024-09-10T11:29:23.026 INFO:tasks.workunit.client.1.smithi178.stdout:0/130: creat d0/dc/d1f/f26 x:0 0 0 2024-09-10T11:29:23.028 INFO:tasks.workunit.client.1.smithi178.stdout:5/146: dread d9/d11/f2f [0,4194304] 0 2024-09-10T11:29:23.028 INFO:tasks.workunit.client.1.smithi178.stdout:7/77: read - d6/d10/d15/f16 zero size 2024-09-10T11:29:23.028 INFO:tasks.workunit.client.1.smithi178.stdout:1/88: link d3/d9/f10 d3/f1c 0 2024-09-10T11:29:23.029 INFO:tasks.workunit.client.1.smithi178.stdout:3/142: dwrite f7 [0,4194304] 0 2024-09-10T11:29:23.030 INFO:tasks.workunit.client.1.smithi178.stdout:0/131: dread - d0/dc/d1f/f26 zero size 2024-09-10T11:29:23.030 INFO:tasks.workunit.client.1.smithi178.stdout:8/136: fdatasync d5/db/fe 0 2024-09-10T11:29:23.030 INFO:tasks.workunit.client.1.smithi178.stdout:7/78: stat d6/c12 0 2024-09-10T11:29:23.031 INFO:tasks.workunit.client.1.smithi178.stdout:3/143: dread - db/f1a zero size 2024-09-10T11:29:23.032 INFO:tasks.workunit.client.1.smithi178.stdout:5/147: rmdir d9 39 2024-09-10T11:29:23.033 INFO:tasks.workunit.client.1.smithi178.stdout:7/79: rename d6 to d6/d10/d15/d1a 22 2024-09-10T11:29:23.036 INFO:tasks.workunit.client.1.smithi178.stdout:3/144: truncate db/f27 647907 0 2024-09-10T11:29:23.038 INFO:tasks.workunit.client.1.smithi178.stdout:7/80: fdatasync d6/fd 0 2024-09-10T11:29:23.041 INFO:tasks.workunit.client.1.smithi178.stdout:9/313: dread d2/d13/f15 [0,4194304] 0 2024-09-10T11:29:23.042 INFO:tasks.workunit.client.1.smithi178.stdout:5/148: mkdir d9/dc/dd/d3f 0 2024-09-10T11:29:23.043 INFO:tasks.workunit.client.1.smithi178.stdout:7/81: symlink d6/d10/l1b 0 2024-09-10T11:29:23.046 INFO:tasks.workunit.client.1.smithi178.stdout:7/82: write f3 [3663682,40228] 0 2024-09-10T11:29:23.047 INFO:tasks.workunit.client.1.smithi178.stdout:7/83: fsync d6/ff 0 2024-09-10T11:29:23.049 INFO:tasks.workunit.client.1.smithi178.stdout:0/132: rmdir d0/dc/d1f 39 2024-09-10T11:29:23.049 INFO:tasks.workunit.client.1.smithi178.stdout:7/84: write d6/ff [4100501,89305] 0 2024-09-10T11:29:23.050 INFO:tasks.workunit.client.1.smithi178.stdout:9/314: symlink d2/d19/l58 0 2024-09-10T11:29:23.052 INFO:tasks.workunit.client.1.smithi178.stdout:7/85: dread - d6/fc zero size 2024-09-10T11:29:23.054 INFO:tasks.workunit.client.1.smithi178.stdout:0/133: creat d0/d12/f27 x:0 0 0 2024-09-10T11:29:23.054 INFO:tasks.workunit.client.1.smithi178.stdout:6/245: sync 2024-09-10T11:29:23.054 INFO:tasks.workunit.client.1.smithi178.stdout:2/314: sync 2024-09-10T11:29:23.055 INFO:tasks.workunit.client.1.smithi178.stdout:9/315: mkdir d2/d51/d59 0 2024-09-10T11:29:23.055 INFO:tasks.workunit.client.1.smithi178.stdout:1/89: dread d3/d9/fc [0,4194304] 0 2024-09-10T11:29:23.056 INFO:tasks.workunit.client.1.smithi178.stdout:3/145: truncate f1 1435872 0 2024-09-10T11:29:23.058 INFO:tasks.workunit.client.1.smithi178.stdout:5/149: dwrite d9/d19/f1a [0,4194304] 0 2024-09-10T11:29:23.060 INFO:tasks.workunit.client.1.smithi178.stdout:3/146: chown db/l13 3828126 1 2024-09-10T11:29:23.060 INFO:tasks.workunit.client.1.smithi178.stdout:1/90: read d3/d13/f17 [3760869,10902] 0 2024-09-10T11:29:23.061 INFO:tasks.workunit.client.1.smithi178.stdout:4/82: truncate d0/d9/f13 2189156 0 2024-09-10T11:29:23.063 INFO:tasks.workunit.client.1.smithi178.stdout:9/316: write d2/f3f [696474,117101] 0 2024-09-10T11:29:23.065 INFO:tasks.workunit.client.1.smithi178.stdout:2/315: creat d1/f69 x:0 0 0 2024-09-10T11:29:23.065 INFO:tasks.workunit.client.1.smithi178.stdout:1/91: mknod d3/de/c1d 0 2024-09-10T11:29:23.066 INFO:tasks.workunit.client.1.smithi178.stdout:5/150: dread d9/dc/d17/f1e [0,4194304] 0 2024-09-10T11:29:23.066 INFO:tasks.workunit.client.1.smithi178.stdout:8/137: dwrite d5/db/f17 [0,4194304] 0 2024-09-10T11:29:23.067 INFO:tasks.workunit.client.1.smithi178.stdout:5/151: chown d9/dc/dd/d3f 94 1 2024-09-10T11:29:23.067 INFO:tasks.workunit.client.1.smithi178.stdout:6/246: creat d3/d7/d8/f58 x:0 0 0 2024-09-10T11:29:23.069 INFO:tasks.workunit.client.1.smithi178.stdout:4/83: symlink d0/d8/df/l1f 0 2024-09-10T11:29:23.070 INFO:tasks.workunit.client.1.smithi178.stdout:7/86: getdents d6/d10 0 2024-09-10T11:29:23.073 INFO:tasks.workunit.client.1.smithi178.stdout:3/147: symlink db/l2d 0 2024-09-10T11:29:23.074 INFO:tasks.workunit.client.1.smithi178.stdout:9/317: stat d2/d8/f9 0 2024-09-10T11:29:23.075 INFO:tasks.workunit.client.1.smithi178.stdout:0/134: getdents d0/dc/d1f 0 2024-09-10T11:29:23.076 INFO:tasks.workunit.client.1.smithi178.stdout:8/138: creat d5/db/d10/d20/f2c x:0 0 0 2024-09-10T11:29:23.076 INFO:tasks.workunit.client.1.smithi178.stdout:4/84: creat d0/d8/df/f20 x:0 0 0 2024-09-10T11:29:23.078 INFO:tasks.workunit.client.1.smithi178.stdout:9/318: chown d2/d19/d1c 96 1 2024-09-10T11:29:23.079 INFO:tasks.workunit.client.1.smithi178.stdout:7/87: creat d6/d10/d15/f1c x:0 0 0 2024-09-10T11:29:23.079 INFO:tasks.workunit.client.1.smithi178.stdout:6/247: chown d3/d7/d8/d15/d1c/d20/l27 149577 1 2024-09-10T11:29:23.079 INFO:tasks.workunit.client.1.smithi178.stdout:4/85: mkdir d0/d21 0 2024-09-10T11:29:23.081 INFO:tasks.workunit.client.1.smithi178.stdout:8/139: symlink d5/db/d10/l2d 0 2024-09-10T11:29:23.087 INFO:tasks.workunit.client.1.smithi178.stdout:5/152: dwrite d9/d11/f2f [0,4194304] 0 2024-09-10T11:29:23.089 INFO:tasks.workunit.client.1.smithi178.stdout:2/316: truncate d1/d41/d14/d11/d19/d34/d5b/d60/f64 585725 0 2024-09-10T11:29:23.089 INFO:tasks.workunit.client.1.smithi178.stdout:3/148: write db/f10 [1348173,35422] 0 2024-09-10T11:29:23.091 INFO:tasks.workunit.client.1.smithi178.stdout:1/92: truncate d3/d9/fd 543088 0 2024-09-10T11:29:23.094 INFO:tasks.workunit.client.1.smithi178.stdout:0/135: creat d0/d19/f28 x:0 0 0 2024-09-10T11:29:23.094 INFO:tasks.workunit.client.1.smithi178.stdout:1/93: write d3/f8 [1702278,36752] 0 2024-09-10T11:29:23.095 INFO:tasks.workunit.client.1.smithi178.stdout:9/319: mkdir d2/df/d31/d5a 0 2024-09-10T11:29:23.096 INFO:tasks.workunit.client.1.smithi178.stdout:7/88: mkdir d6/d10/d1d 0 2024-09-10T11:29:23.096 INFO:tasks.workunit.client.1.smithi178.stdout:0/136: stat d0/f22 0 2024-09-10T11:29:23.097 INFO:tasks.workunit.client.1.smithi178.stdout:5/153: symlink d9/dc/l40 0 2024-09-10T11:29:23.106 INFO:tasks.workunit.client.1.smithi178.stdout:5/154: stat d9/dc/dd/d37 0 2024-09-10T11:29:23.106 INFO:tasks.workunit.client.1.smithi178.stdout:8/140: dwrite d5/db/d10/d20/f28 [0,4194304] 0 2024-09-10T11:29:23.110 INFO:tasks.workunit.client.1.smithi178.stdout:4/86: dwrite d0/d9/fb [0,4194304] 0 2024-09-10T11:29:23.110 INFO:tasks.workunit.client.1.smithi178.stdout:6/248: getdents d3/d7/d8/d19 0 2024-09-10T11:29:23.113 INFO:tasks.workunit.client.1.smithi178.stdout:2/317: write d1/d41/d14/f3e [3971679,25246] 0 2024-09-10T11:29:23.114 INFO:tasks.workunit.client.1.smithi178.stdout:7/89: dread d6/fd [0,4194304] 0 2024-09-10T11:29:23.115 INFO:tasks.workunit.client.1.smithi178.stdout:3/149: symlink db/l2e 0 2024-09-10T11:29:23.118 INFO:tasks.workunit.client.1.smithi178.stdout:1/94: write d3/d9/f10 [4425588,12741] 0 2024-09-10T11:29:23.123 INFO:tasks.workunit.client.1.smithi178.stdout:0/137: dread d0/f22 [0,4194304] 0 2024-09-10T11:29:23.127 INFO:tasks.workunit.client.1.smithi178.stdout:7/90: mkdir d6/d10/d1e 0 2024-09-10T11:29:23.128 INFO:tasks.workunit.client.1.smithi178.stdout:3/150: creat db/d26/f2f x:0 0 0 2024-09-10T11:29:23.130 INFO:tasks.workunit.client.1.smithi178.stdout:0/138: symlink d0/l29 0 2024-09-10T11:29:23.131 INFO:tasks.workunit.client.1.smithi178.stdout:1/95: rename d3/d9/cf to d3/d13/c1e 0 2024-09-10T11:29:23.132 INFO:tasks.workunit.client.1.smithi178.stdout:8/141: dwrite f2 [0,4194304] 0 2024-09-10T11:29:23.133 INFO:tasks.workunit.client.1.smithi178.stdout:9/320: link d2/d19/d1c/l30 d2/d19/d1c/d2d/l5b 0 2024-09-10T11:29:23.133 INFO:tasks.workunit.client.1.smithi178.stdout:7/91: write d6/fd [164399,112520] 0 2024-09-10T11:29:23.136 INFO:tasks.workunit.client.1.smithi178.stdout:6/249: dread d3/f14 [0,4194304] 0 2024-09-10T11:29:23.137 INFO:tasks.workunit.client.1.smithi178.stdout:4/87: getdents d0/d10 0 2024-09-10T11:29:23.137 INFO:tasks.workunit.client.1.smithi178.stdout:0/139: fsync d0/dc/f21 0 2024-09-10T11:29:23.138 INFO:tasks.workunit.client.1.smithi178.stdout:7/92: symlink d6/l1f 0 2024-09-10T11:29:23.139 INFO:tasks.workunit.client.1.smithi178.stdout:8/142: rename d5/db to d5/db/d10/d2e 22 2024-09-10T11:29:23.142 INFO:tasks.workunit.client.1.smithi178.stdout:5/155: dwrite d9/dc/dd/d16/f27 [0,4194304] 0 2024-09-10T11:29:23.142 INFO:tasks.workunit.client.1.smithi178.stdout:4/88: dread - d0/f1c zero size 2024-09-10T11:29:23.142 INFO:tasks.workunit.client.1.smithi178.stdout:7/93: dread - d6/fc zero size 2024-09-10T11:29:23.144 INFO:tasks.workunit.client.1.smithi178.stdout:0/140: symlink d0/l2a 0 2024-09-10T11:29:23.149 INFO:tasks.workunit.client.1.smithi178.stdout:6/250: unlink d3/d24/f2e 0 2024-09-10T11:29:23.149 INFO:tasks.workunit.client.1.smithi178.stdout:0/141: dread - d0/d19/f28 zero size 2024-09-10T11:29:23.149 INFO:tasks.workunit.client.1.smithi178.stdout:7/94: write f4 [3691721,29540] 0 2024-09-10T11:29:23.150 INFO:tasks.workunit.client.1.smithi178.stdout:4/89: write d0/f18 [383831,95455] 0 2024-09-10T11:29:23.151 INFO:tasks.workunit.client.1.smithi178.stdout:0/142: chown d0/f8 6620 1 2024-09-10T11:29:23.152 INFO:tasks.workunit.client.1.smithi178.stdout:8/143: creat d5/db/d10/f2f x:0 0 0 2024-09-10T11:29:23.156 INFO:tasks.workunit.client.1.smithi178.stdout:5/156: creat d9/dc/dd/d16/f41 x:0 0 0 2024-09-10T11:29:23.156 INFO:tasks.workunit.client.1.smithi178.stdout:5/157: read f2 [8350880,29656] 0 2024-09-10T11:29:23.157 INFO:tasks.workunit.client.1.smithi178.stdout:0/143: unlink d0/c3 0 2024-09-10T11:29:23.158 INFO:tasks.workunit.client.1.smithi178.stdout:3/151: dread db/f10 [0,4194304] 0 2024-09-10T11:29:23.158 INFO:tasks.workunit.client.1.smithi178.stdout:0/144: readlink d0/l29 0 2024-09-10T11:29:23.160 INFO:tasks.workunit.client.1.smithi178.stdout:7/95: creat d6/d10/d1d/f20 x:0 0 0 2024-09-10T11:29:23.161 INFO:tasks.workunit.client.1.smithi178.stdout:3/152: fsync db/f16 0 2024-09-10T11:29:23.166 INFO:tasks.workunit.client.1.smithi178.stdout:7/96: write d6/d10/d15/f16 [852002,18269] 0 2024-09-10T11:29:23.166 INFO:tasks.workunit.client.1.smithi178.stdout:5/158: mkdir d9/dc/dd/d37/d42 0 2024-09-10T11:29:23.166 INFO:tasks.workunit.client.1.smithi178.stdout:0/145: read d0/dc/f16 [170629,36170] 0 2024-09-10T11:29:23.166 INFO:tasks.workunit.client.1.smithi178.stdout:6/251: getdents d3/d24 0 2024-09-10T11:29:23.167 INFO:tasks.workunit.client.1.smithi178.stdout:3/153: unlink f7 0 2024-09-10T11:29:23.170 INFO:tasks.workunit.client.1.smithi178.stdout:9/321: truncate d2/d8/f17 3268145 0 2024-09-10T11:29:23.170 INFO:tasks.workunit.client.1.smithi178.stdout:4/90: dwrite d0/d9/f12 [0,4194304] 0 2024-09-10T11:29:23.171 INFO:tasks.workunit.client.1.smithi178.stdout:2/318: dread d1/d41/d14/f1b [0,4194304] 0 2024-09-10T11:29:23.171 INFO:tasks.workunit.client.1.smithi178.stdout:6/252: unlink d3/d7/d8/d15/c33 0 2024-09-10T11:29:23.173 INFO:tasks.workunit.client.1.smithi178.stdout:4/91: write d0/d9/f12 [1570291,34335] 0 2024-09-10T11:29:23.179 INFO:tasks.workunit.client.1.smithi178.stdout:7/97: creat d6/d10/d15/f21 x:0 0 0 2024-09-10T11:29:23.181 INFO:tasks.workunit.client.1.smithi178.stdout:9/322: mknod d2/df/c5c 0 2024-09-10T11:29:23.182 INFO:tasks.workunit.client.1.smithi178.stdout:6/253: mknod d3/d7/d8/d15/d1c/d2a/c59 0 2024-09-10T11:29:23.183 INFO:tasks.workunit.client.1.smithi178.stdout:7/98: dread - d6/d10/d15/f1c zero size 2024-09-10T11:29:23.186 INFO:tasks.workunit.client.1.smithi178.stdout:5/159: truncate d9/dc/f15 1320901 0 2024-09-10T11:29:23.189 INFO:tasks.workunit.client.1.smithi178.stdout:1/96: dread d3/f1c [0,4194304] 0 2024-09-10T11:29:23.191 INFO:tasks.workunit.client.1.smithi178.stdout:0/146: dwrite d0/f22 [0,4194304] 0 2024-09-10T11:29:23.191 INFO:tasks.workunit.client.1.smithi178.stdout:2/319: read d1/d41/d14/d11/d19/d34/f17 [73643,77284] 0 2024-09-10T11:29:23.193 INFO:tasks.workunit.client.1.smithi178.stdout:7/99: mkdir d6/d10/d1d/d22 0 2024-09-10T11:29:23.194 INFO:tasks.workunit.client.1.smithi178.stdout:8/144: dwrite d5/db/d10/d20/f2b [0,4194304] 0 2024-09-10T11:29:23.195 INFO:tasks.workunit.client.1.smithi178.stdout:4/92: truncate d0/f5 4113166 0 2024-09-10T11:29:23.197 INFO:tasks.workunit.client.1.smithi178.stdout:3/154: dwrite db/f10 [4194304,4194304] 0 2024-09-10T11:29:23.197 INFO:tasks.workunit.client.1.smithi178.stdout:6/254: creat d3/d7/d35/f5a x:0 0 0 2024-09-10T11:29:23.198 INFO:tasks.workunit.client.1.smithi178.stdout:4/93: fsync d0/f18 0 2024-09-10T11:29:23.199 INFO:tasks.workunit.client.1.smithi178.stdout:9/323: creat d2/d19/d1c/d2d/d54/f5d x:0 0 0 2024-09-10T11:29:23.201 INFO:tasks.workunit.client.1.smithi178.stdout:2/320: rmdir d1/d7/d24/d43 39 2024-09-10T11:29:23.202 INFO:tasks.workunit.client.1.smithi178.stdout:5/160: symlink d9/dc/d17/d1d/l43 0 2024-09-10T11:29:23.203 INFO:tasks.workunit.client.1.smithi178.stdout:9/324: readlink d2/d8/l33 0 2024-09-10T11:29:23.205 INFO:tasks.workunit.client.1.smithi178.stdout:7/100: creat d6/f23 x:0 0 0 2024-09-10T11:29:23.205 INFO:tasks.workunit.client.1.smithi178.stdout:8/145: write d5/db/d10/d20/f2c [350246,95497] 0 2024-09-10T11:29:23.207 INFO:tasks.workunit.client.1.smithi178.stdout:7/101: stat d6 0 2024-09-10T11:29:23.207 INFO:tasks.workunit.client.1.smithi178.stdout:3/155: symlink db/d20/l30 0 2024-09-10T11:29:23.209 INFO:tasks.workunit.client.1.smithi178.stdout:7/102: chown d6/d10 60 1 2024-09-10T11:29:23.209 INFO:tasks.workunit.client.1.smithi178.stdout:6/255: dread - d3/d7/d8/d15/d1c/d2a/f51 zero size 2024-09-10T11:29:23.209 INFO:tasks.workunit.client.1.smithi178.stdout:2/321: chown d1/f69 4111815 1 2024-09-10T11:29:23.212 INFO:tasks.workunit.client.1.smithi178.stdout:1/97: dwrite d3/f8 [4194304,4194304] 0 2024-09-10T11:29:23.214 INFO:tasks.workunit.client.1.smithi178.stdout:4/94: chown d0/d9/f13 575 1 2024-09-10T11:29:23.217 INFO:tasks.workunit.client.1.smithi178.stdout:8/146: mkdir d5/d30 0 2024-09-10T11:29:23.218 INFO:tasks.workunit.client.1.smithi178.stdout:1/98: truncate d3/d13/f17 4557271 0 2024-09-10T11:29:23.218 INFO:tasks.workunit.client.1.smithi178.stdout:8/147: readlink d5/l9 0 2024-09-10T11:29:23.220 INFO:tasks.workunit.client.1.smithi178.stdout:5/161: creat d9/dc/d17/f44 x:0 0 0 2024-09-10T11:29:23.221 INFO:tasks.workunit.client.1.smithi178.stdout:9/325: dwrite d2/d13/f3c [0,4194304] 0 2024-09-10T11:29:23.221 INFO:tasks.workunit.client.1.smithi178.stdout:1/99: fsync d3/de/f16 0 2024-09-10T11:29:23.223 INFO:tasks.workunit.client.1.smithi178.stdout:3/156: symlink db/d26/l31 0 2024-09-10T11:29:23.224 INFO:tasks.workunit.client.1.smithi178.stdout:8/148: write d5/ff [3981787,32489] 0 2024-09-10T11:29:23.225 INFO:tasks.workunit.client.1.smithi178.stdout:0/147: getdents d0/dc 0 2024-09-10T11:29:23.225 INFO:tasks.workunit.client.1.smithi178.stdout:2/322: unlink d1/d22/c25 0 2024-09-10T11:29:23.227 INFO:tasks.workunit.client.1.smithi178.stdout:7/103: mkdir d6/d10/d1e/d24 0 2024-09-10T11:29:23.229 INFO:tasks.workunit.client.1.smithi178.stdout:2/323: write d1/d41/f1f [855519,25625] 0 2024-09-10T11:29:23.233 INFO:tasks.workunit.client.1.smithi178.stdout:6/256: dwrite d3/d24/f30 [8388608,4194304] 0 2024-09-10T11:29:23.256 INFO:tasks.workunit.client.1.smithi178.stdout:5/162: mknod d9/dc/dd/d37/c45 0 2024-09-10T11:29:23.256 INFO:tasks.workunit.client.1.smithi178.stdout:5/163: chown d9/dc/dd/f2d 6 1 2024-09-10T11:29:23.257 INFO:tasks.workunit.client.1.smithi178.stdout:3/157: creat db/d26/f32 x:0 0 0 2024-09-10T11:29:23.259 INFO:tasks.workunit.client.1.smithi178.stdout:0/148: creat d0/dc/f2b x:0 0 0 2024-09-10T11:29:23.261 INFO:tasks.workunit.client.1.smithi178.stdout:2/324: creat d1/d7/f6a x:0 0 0 2024-09-10T11:29:23.262 INFO:tasks.workunit.client.1.smithi178.stdout:6/257: creat d3/d42/f5b x:0 0 0 2024-09-10T11:29:23.263 INFO:tasks.workunit.client.1.smithi178.stdout:1/100: fdatasync d3/d9/fd 0 2024-09-10T11:29:23.264 INFO:tasks.workunit.client.1.smithi178.stdout:5/164: symlink d9/dc/dd/l46 0 2024-09-10T11:29:23.264 INFO:tasks.workunit.client.1.smithi178.stdout:2/325: read d1/d7/d24/f5a [12305061,108329] 0 2024-09-10T11:29:23.265 INFO:tasks.workunit.client.1.smithi178.stdout:3/158: creat db/d20/f33 x:0 0 0 2024-09-10T11:29:23.266 INFO:tasks.workunit.client.1.smithi178.stdout:9/326: write d2/d13/f15 [200952,9498] 0 2024-09-10T11:29:23.267 INFO:tasks.workunit.client.1.smithi178.stdout:8/149: symlink d5/db/d1e/l31 0 2024-09-10T11:29:23.267 INFO:tasks.workunit.client.1.smithi178.stdout:2/326: stat d1/d41/d14/d11/l56 0 2024-09-10T11:29:23.268 INFO:tasks.workunit.client.1.smithi178.stdout:0/149: mkdir d0/d2c 0 2024-09-10T11:29:23.269 INFO:tasks.workunit.client.1.smithi178.stdout:1/101: rename d3/de/f16 to d3/d9/d1a/f1f 0 2024-09-10T11:29:23.273 INFO:tasks.workunit.client.1.smithi178.stdout:6/258: dwrite d3/d7/d8/d15/d1c/f1d [0,4194304] 0 2024-09-10T11:29:23.273 INFO:tasks.workunit.client.1.smithi178.stdout:7/104: truncate d6/ff 3118118 0 2024-09-10T11:29:23.274 INFO:tasks.workunit.client.1.smithi178.stdout:9/327: mkdir d2/d19/d5e 0 2024-09-10T11:29:23.280 INFO:tasks.workunit.client.1.smithi178.stdout:9/328: chown d2/d19/d1c 101745249 1 2024-09-10T11:29:23.283 INFO:tasks.workunit.client.1.smithi178.stdout:3/159: dwrite db/f1a [0,4194304] 0 2024-09-10T11:29:23.285 INFO:tasks.workunit.client.1.smithi178.stdout:7/105: link d6/fd d6/d10/d1d/f25 0 2024-09-10T11:29:23.286 INFO:tasks.workunit.client.1.smithi178.stdout:6/259: unlink d3/d7/d8/d19/f54 0 2024-09-10T11:29:23.286 INFO:tasks.workunit.client.1.smithi178.stdout:2/327: dwrite d1/d41/d14/f23 [0,4194304] 0 2024-09-10T11:29:23.290 INFO:tasks.workunit.client.1.smithi178.stdout:1/102: dread d3/d13/f15 [0,4194304] 0 2024-09-10T11:29:23.290 INFO:tasks.workunit.client.1.smithi178.stdout:7/106: chown d6/d10/l17 40322144 1 2024-09-10T11:29:23.295 INFO:tasks.workunit.client.1.smithi178.stdout:3/160: dread f1 [0,4194304] 0 2024-09-10T11:29:23.299 INFO:tasks.workunit.client.1.smithi178.stdout:0/150: dwrite d0/dc/d1f/f26 [0,4194304] 0 2024-09-10T11:29:23.301 INFO:tasks.workunit.client.1.smithi178.stdout:3/161: creat db/d20/f34 x:0 0 0 2024-09-10T11:29:23.304 INFO:tasks.workunit.client.1.smithi178.stdout:0/151: mkdir d0/dc/d1f/d2d 0 2024-09-10T11:29:23.305 INFO:tasks.workunit.client.1.smithi178.stdout:9/329: truncate d2/d13/f2e 3246519 0 2024-09-10T11:29:23.305 INFO:tasks.workunit.client.1.smithi178.stdout:6/260: dread d3/d7/d8/d15/d1c/f40 [0,4194304] 0 2024-09-10T11:29:23.305 INFO:tasks.workunit.client.1.smithi178.stdout:2/328: link d1/d41/d14/d11/f4a d1/d41/d14/d11/d19/d34/d5b/f6b 0 2024-09-10T11:29:23.307 INFO:tasks.workunit.client.1.smithi178.stdout:1/103: dwrite d3/f8 [4194304,4194304] 0 2024-09-10T11:29:23.309 INFO:tasks.workunit.client.1.smithi178.stdout:9/330: write d2/d13/f43 [455214,68216] 0 2024-09-10T11:29:23.314 INFO:tasks.workunit.client.1.smithi178.stdout:6/261: symlink d3/d7/d8/d15/d1c/d2a/l5c 0 2024-09-10T11:29:23.314 INFO:tasks.workunit.client.1.smithi178.stdout:7/107: dwrite d6/d10/d1d/f20 [0,4194304] 0 2024-09-10T11:29:23.315 INFO:tasks.workunit.client.1.smithi178.stdout:3/162: getdents db 0 2024-09-10T11:29:23.320 INFO:tasks.workunit.client.1.smithi178.stdout:1/104: rename d3/f8 to d3/d9/f20 0 2024-09-10T11:29:23.320 INFO:tasks.workunit.client.1.smithi178.stdout:0/152: link d0/f9 d0/f2e 0 2024-09-10T11:29:23.322 INFO:tasks.workunit.client.1.smithi178.stdout:1/105: stat d3/de/c18 0 2024-09-10T11:29:23.323 INFO:tasks.workunit.client.1.smithi178.stdout:2/329: dwrite d1/f2e [0,4194304] 0 2024-09-10T11:29:23.329 INFO:tasks.workunit.client.1.smithi178.stdout:3/163: dwrite db/f10 [4194304,4194304] 0 2024-09-10T11:29:23.329 INFO:tasks.workunit.client.1.smithi178.stdout:7/108: dread d6/d10/d1d/f20 [0,4194304] 0 2024-09-10T11:29:23.331 INFO:tasks.workunit.client.1.smithi178.stdout:3/164: write db/f1a [2499683,103868] 0 2024-09-10T11:29:23.331 INFO:tasks.workunit.client.1.smithi178.stdout:7/109: write f3 [804781,70001] 0 2024-09-10T11:29:23.332 INFO:tasks.workunit.client.1.smithi178.stdout:3/165: dread - db/d26/f2f zero size 2024-09-10T11:29:23.333 INFO:tasks.workunit.client.1.smithi178.stdout:3/166: stat db/f15 0 2024-09-10T11:29:23.335 INFO:tasks.workunit.client.1.smithi178.stdout:9/331: fsync d2/d19/f39 0 2024-09-10T11:29:23.336 INFO:tasks.workunit.client.1.smithi178.stdout:6/262: rename d3/fb to d3/d7/d8/d15/d1c/d2a/f5d 0 2024-09-10T11:29:23.337 INFO:tasks.workunit.client.1.smithi178.stdout:3/167: write db/f16 [1585681,81217] 0 2024-09-10T11:29:23.340 INFO:tasks.workunit.client.1.smithi178.stdout:0/153: fdatasync d0/f9 0 2024-09-10T11:29:23.341 INFO:tasks.workunit.client.1.smithi178.stdout:6/263: dread - d3/d7/d8/d15/d1c/d2a/d18/f43 zero size 2024-09-10T11:29:23.342 INFO:tasks.workunit.client.1.smithi178.stdout:9/332: creat d2/df/d31/d37/f5f x:0 0 0 2024-09-10T11:29:23.342 INFO:tasks.workunit.client.1.smithi178.stdout:3/168: mkdir db/d35 0 2024-09-10T11:29:23.343 INFO:tasks.workunit.client.1.smithi178.stdout:7/110: truncate d6/d10/d1d/f25 3835418 0 2024-09-10T11:29:23.343 INFO:tasks.workunit.client.1.smithi178.stdout:0/154: chown d0/f1 79 1 2024-09-10T11:29:23.344 INFO:tasks.workunit.client.1.smithi178.stdout:9/333: chown d2/d13/f3c 6895 1 2024-09-10T11:29:23.344 INFO:tasks.workunit.client.1.smithi178.stdout:1/106: dwrite d3/d9/d1a/f1f [0,4194304] 0 2024-09-10T11:29:23.345 INFO:tasks.workunit.client.1.smithi178.stdout:2/330: rmdir d1/d41/d14/d11/d19/d54 0 2024-09-10T11:29:23.346 INFO:tasks.workunit.client.1.smithi178.stdout:2/331: chown d1/d41/d14/d11/d19/d34/d5b 369325 1 2024-09-10T11:29:23.348 INFO:tasks.workunit.client.1.smithi178.stdout:0/155: creat d0/d12/f2f x:0 0 0 2024-09-10T11:29:23.349 INFO:tasks.workunit.client.1.smithi178.stdout:7/111: creat d6/d10/d1e/f26 x:0 0 0 2024-09-10T11:29:23.349 INFO:tasks.workunit.client.1.smithi178.stdout:9/334: chown d2/l53 528864 1 2024-09-10T11:29:23.351 INFO:tasks.workunit.client.1.smithi178.stdout:1/107: write d3/d9/fc [2221622,93696] 0 2024-09-10T11:29:23.351 INFO:tasks.workunit.client.1.smithi178.stdout:6/264: mknod d3/c5e 0 2024-09-10T11:29:23.352 INFO:tasks.workunit.client.1.smithi178.stdout:9/335: stat d2/d19/d1c/d2d/d54 0 2024-09-10T11:29:23.354 INFO:tasks.workunit.client.1.smithi178.stdout:2/332: rename d1/d41/d14/f3b to d1/d7/f6c 0 2024-09-10T11:29:23.355 INFO:tasks.workunit.client.1.smithi178.stdout:9/336: fsync d2/df/f3b 0 2024-09-10T11:29:23.356 INFO:tasks.workunit.client.1.smithi178.stdout:9/337: fsync d2/df/d31/f47 0 2024-09-10T11:29:23.357 INFO:tasks.workunit.client.1.smithi178.stdout:6/265: fsync d3/d7/d8/d15/d1c/f40 0 2024-09-10T11:29:23.358 INFO:tasks.workunit.client.1.smithi178.stdout:1/108: dread d3/d13/f15 [0,4194304] 0 2024-09-10T11:29:23.359 INFO:tasks.workunit.client.1.smithi178.stdout:2/333: mknod d1/d41/d14/d11/d19/d34/d5e/c6d 0 2024-09-10T11:29:23.361 INFO:tasks.workunit.client.1.smithi178.stdout:2/334: chown d1/d41/d14/d11/d19/f37 0 1 2024-09-10T11:29:23.361 INFO:tasks.workunit.client.1.smithi178.stdout:9/338: mkdir d2/d19/d1c/d2d/d54/d60 0 2024-09-10T11:29:23.363 INFO:tasks.workunit.client.1.smithi178.stdout:6/266: mkdir d3/d7/d8/d15/d1c/d20/d4b/d5f 0 2024-09-10T11:29:23.364 INFO:tasks.workunit.client.1.smithi178.stdout:7/112: getdents d6 0 2024-09-10T11:29:23.365 INFO:tasks.workunit.client.1.smithi178.stdout:1/109: mkdir d3/d9/d21 0 2024-09-10T11:29:23.365 INFO:tasks.workunit.client.1.smithi178.stdout:0/156: link d0/l29 d0/dc/l30 0 2024-09-10T11:29:23.365 INFO:tasks.workunit.client.1.smithi178.stdout:0/157: chown d0/f22 156080918 1 2024-09-10T11:29:23.366 INFO:tasks.workunit.client.1.smithi178.stdout:9/339: mknod d2/df/c61 0 2024-09-10T11:29:23.368 INFO:tasks.workunit.client.1.smithi178.stdout:6/267: rename d3/d7/d8/ld to d3/d7/d35/l60 0 2024-09-10T11:29:23.368 INFO:tasks.workunit.client.1.smithi178.stdout:7/113: creat d6/d10/d1d/f27 x:0 0 0 2024-09-10T11:29:23.370 INFO:tasks.workunit.client.1.smithi178.stdout:6/268: truncate d3/d7/d8/d15/d1c/d20/f2d 1127514 0 2024-09-10T11:29:23.370 INFO:tasks.workunit.client.1.smithi178.stdout:2/335: truncate d1/d41/d14/d11/f9 7927318 0 2024-09-10T11:29:23.371 INFO:tasks.workunit.client.1.smithi178.stdout:9/340: unlink d2/d19/d1c/d2a/f49 0 2024-09-10T11:29:23.372 INFO:tasks.workunit.client.1.smithi178.stdout:7/114: symlink d6/l28 0 2024-09-10T11:29:23.374 INFO:tasks.workunit.client.1.smithi178.stdout:9/341: fdatasync d2/df/d31/d37/f5f 0 2024-09-10T11:29:23.374 INFO:tasks.workunit.client.1.smithi178.stdout:2/336: write d1/f13 [5138715,28413] 0 2024-09-10T11:29:23.376 INFO:tasks.workunit.client.1.smithi178.stdout:6/269: truncate d3/d7/d8/d15/d1c/d20/f2d 1565694 0 2024-09-10T11:29:23.378 INFO:tasks.workunit.client.1.smithi178.stdout:1/110: dwrite d3/d9/f20 [0,4194304] 0 2024-09-10T11:29:23.378 INFO:tasks.workunit.client.1.smithi178.stdout:0/158: dwrite d0/f1 [0,4194304] 0 2024-09-10T11:29:23.379 INFO:tasks.workunit.client.1.smithi178.stdout:2/337: mkdir d1/d41/d14/d11/d19/d55/d6e 0 2024-09-10T11:29:23.380 INFO:tasks.workunit.client.1.smithi178.stdout:1/111: chown d3/c5 5 1 2024-09-10T11:29:23.380 INFO:tasks.workunit.client.1.smithi178.stdout:2/338: fsync d1/f3d 0 2024-09-10T11:29:23.383 INFO:tasks.workunit.client.1.smithi178.stdout:1/112: write d3/d9/fc [3830253,21108] 0 2024-09-10T11:29:23.385 INFO:tasks.workunit.client.1.smithi178.stdout:6/270: rename d3/d7/d8/d15/d1c/d2a/d18/f4f to d3/d42/f61 0 2024-09-10T11:29:23.387 INFO:tasks.workunit.client.1.smithi178.stdout:9/342: dwrite d2/df/d31/d37/f5f [0,4194304] 0 2024-09-10T11:29:23.388 INFO:tasks.workunit.client.1.smithi178.stdout:2/339: dread d1/d41/d14/d11/d19/d34/f17 [0,4194304] 0 2024-09-10T11:29:23.389 INFO:tasks.workunit.client.1.smithi178.stdout:0/159: mknod d0/dc/d1f/d2d/c31 0 2024-09-10T11:29:23.390 INFO:tasks.workunit.client.1.smithi178.stdout:0/160: chown d0/dc/f2b 9582019 1 2024-09-10T11:29:23.396 INFO:tasks.workunit.client.1.smithi178.stdout:6/271: dread d3/d7/d35/f4c [0,4194304] 0 2024-09-10T11:29:23.397 INFO:tasks.workunit.client.1.smithi178.stdout:9/343: truncate d2/d19/f39 3754979 0 2024-09-10T11:29:23.402 INFO:tasks.workunit.client.1.smithi178.stdout:9/344: mknod d2/d19/d1c/d2a/c62 0 2024-09-10T11:29:23.403 INFO:tasks.workunit.client.1.smithi178.stdout:2/340: dwrite d1/d7/f6c [0,4194304] 0 2024-09-10T11:29:23.404 INFO:tasks.workunit.client.1.smithi178.stdout:1/113: truncate d3/d9/f14 18690 0 2024-09-10T11:29:23.407 INFO:tasks.workunit.client.1.smithi178.stdout:6/272: creat d3/d7/d8/d15/d1c/d20/d4b/f62 x:0 0 0 2024-09-10T11:29:23.408 INFO:tasks.workunit.client.1.smithi178.stdout:6/273: truncate d3/d42/f5b 650548 0 2024-09-10T11:29:23.408 INFO:tasks.workunit.client.1.smithi178.stdout:9/345: creat d2/df/d31/d37/f63 x:0 0 0 2024-09-10T11:29:23.409 INFO:tasks.workunit.client.1.smithi178.stdout:1/114: creat d3/f22 x:0 0 0 2024-09-10T11:29:23.411 INFO:tasks.workunit.client.1.smithi178.stdout:1/115: truncate d3/d9/fb 943524 0 2024-09-10T11:29:23.412 INFO:tasks.workunit.client.1.smithi178.stdout:2/341: rename d1/d41/d14 to d1/d41/d14/d6f 22 2024-09-10T11:29:23.413 INFO:tasks.workunit.client.1.smithi178.stdout:6/274: creat d3/d7/d35/f63 x:0 0 0 2024-09-10T11:29:23.418 INFO:tasks.workunit.client.1.smithi178.stdout:1/116: mknod d3/d9/d21/c23 0 2024-09-10T11:29:23.418 INFO:tasks.workunit.client.1.smithi178.stdout:9/346: link d2/df/d31/l4c d2/df/l64 0 2024-09-10T11:29:23.418 INFO:tasks.workunit.client.1.smithi178.stdout:2/342: dwrite d1/d7/f6c [0,4194304] 0 2024-09-10T11:29:23.420 INFO:tasks.workunit.client.1.smithi178.stdout:6/275: link d3/d7/d8/d19/l2b d3/d24/l64 0 2024-09-10T11:29:23.430 INFO:tasks.workunit.client.1.smithi178.stdout:2/343: fsync d1/d7/d24/f2c 0 2024-09-10T11:29:23.430 INFO:tasks.workunit.client.1.smithi178.stdout:9/347: unlink d2/df/d31/d37/f4f 0 2024-09-10T11:29:23.432 INFO:tasks.workunit.client.1.smithi178.stdout:1/117: dread d3/d13/f15 [0,4194304] 0 2024-09-10T11:29:23.433 INFO:tasks.workunit.client.1.smithi178.stdout:2/344: mknod d1/d41/d5d/c70 0 2024-09-10T11:29:23.434 INFO:tasks.workunit.client.1.smithi178.stdout:2/345: chown d1/d7/d24/f42 653874 1 2024-09-10T11:29:23.435 INFO:tasks.workunit.client.1.smithi178.stdout:1/118: rmdir d3/d9/d1a 39 2024-09-10T11:29:23.436 INFO:tasks.workunit.client.1.smithi178.stdout:1/119: chown d3/de/c19 143426 1 2024-09-10T11:29:23.437 INFO:tasks.workunit.client.1.smithi178.stdout:2/346: write d1/d41/f31 [179038,97626] 0 2024-09-10T11:29:23.439 INFO:tasks.workunit.client.1.smithi178.stdout:6/276: dwrite d3/d7/d8/d19/f39 [0,4194304] 0 2024-09-10T11:29:23.440 INFO:tasks.workunit.client.1.smithi178.stdout:1/120: creat d3/d13/f24 x:0 0 0 2024-09-10T11:29:23.443 INFO:tasks.workunit.client.1.smithi178.stdout:2/347: rename d1/d41/d14/d11/d19/f4f to d1/d41/d5d/f71 0 2024-09-10T11:29:23.445 INFO:tasks.workunit.client.1.smithi178.stdout:9/348: dwrite d2/d13/f2e [0,4194304] 0 2024-09-10T11:29:23.449 INFO:tasks.workunit.client.1.smithi178.stdout:1/121: rename d3/d9/f20 to d3/d9/f25 0 2024-09-10T11:29:23.453 INFO:tasks.workunit.client.1.smithi178.stdout:2/348: dwrite d1/d41/d14/d11/d19/d55/f5c [0,4194304] 0 2024-09-10T11:29:23.454 INFO:tasks.workunit.client.1.smithi178.stdout:1/122: mkdir d3/d26 0 2024-09-10T11:29:23.454 INFO:tasks.workunit.client.1.smithi178.stdout:1/123: chown d3/f22 1210 1 2024-09-10T11:29:23.455 INFO:tasks.workunit.client.1.smithi178.stdout:2/349: write d1/d41/d14/d11/d19/d4b/f57 [396288,117358] 0 2024-09-10T11:29:23.455 INFO:tasks.workunit.client.1.smithi178.stdout:9/349: read d2/d8/f9 [168318,112491] 0 2024-09-10T11:29:23.458 INFO:tasks.workunit.client.1.smithi178.stdout:6/277: write d3/d7/d8/d15/d1c/d20/d4b/f52 [150714,120084] 0 2024-09-10T11:29:23.459 INFO:tasks.workunit.client.1.smithi178.stdout:2/350: mknod d1/d7/c72 0 2024-09-10T11:29:23.466 INFO:tasks.workunit.client.1.smithi178.stdout:2/351: rename d1/d41/f38 to d1/d41/d14/d11/d19/d55/f73 0 2024-09-10T11:29:23.467 INFO:tasks.workunit.client.1.smithi178.stdout:9/350: dwrite d2/dc/f2f [0,4194304] 0 2024-09-10T11:29:23.467 INFO:tasks.workunit.client.1.smithi178.stdout:1/124: dwrite d3/d13/f17 [4194304,4194304] 0 2024-09-10T11:29:23.471 INFO:tasks.workunit.client.1.smithi178.stdout:9/351: symlink d2/d19/d1c/l65 0 2024-09-10T11:29:23.471 INFO:tasks.workunit.client.1.smithi178.stdout:2/352: chown d1/d41/d14/d11/f4a 6338 1 2024-09-10T11:29:23.473 INFO:tasks.workunit.client.1.smithi178.stdout:6/278: rename d3/d7/d8/d19/l2b to d3/d7/d8/d15/d48/l65 0 2024-09-10T11:29:23.475 INFO:tasks.workunit.client.1.smithi178.stdout:1/125: dread d3/d9/fc [0,4194304] 0 2024-09-10T11:29:23.478 INFO:tasks.workunit.client.1.smithi178.stdout:6/279: write d3/d7/d8/d15/d1c/d2a/f41 [72733,78720] 0 2024-09-10T11:29:23.479 INFO:tasks.workunit.client.1.smithi178.stdout:1/126: symlink d3/de/l27 0 2024-09-10T11:29:23.482 INFO:tasks.workunit.client.1.smithi178.stdout:9/352: dwrite d2/d8/fa [0,4194304] 0 2024-09-10T11:29:23.483 INFO:tasks.workunit.client.1.smithi178.stdout:2/353: dwrite d1/d22/f26 [0,4194304] 0 2024-09-10T11:29:23.483 INFO:tasks.workunit.client.1.smithi178.stdout:6/280: chown d3/d7/d8/d15/d1c/d2a/l5c 13611 1 2024-09-10T11:29:23.484 INFO:tasks.workunit.client.1.smithi178.stdout:1/127: creat d3/d26/f28 x:0 0 0 2024-09-10T11:29:23.485 INFO:tasks.workunit.client.1.smithi178.stdout:9/353: write d2/d8/f50 [309820,109555] 0 2024-09-10T11:29:23.487 INFO:tasks.workunit.client.1.smithi178.stdout:6/281: creat d3/d42/f66 x:0 0 0 2024-09-10T11:29:23.487 INFO:tasks.workunit.client.1.smithi178.stdout:9/354: stat d2/l53 0 2024-09-10T11:29:23.487 INFO:tasks.workunit.client.1.smithi178.stdout:1/128: creat d3/d13/f29 x:0 0 0 2024-09-10T11:29:23.488 INFO:tasks.workunit.client.1.smithi178.stdout:2/354: chown d1/d7/d24/d46/f63 0 1 2024-09-10T11:29:23.488 INFO:tasks.workunit.client.1.smithi178.stdout:1/129: chown d3/d13/f24 191 1 2024-09-10T11:29:23.490 INFO:tasks.workunit.client.1.smithi178.stdout:6/282: stat d3/d7/d8/d15/d1c/d2a/c59 0 2024-09-10T11:29:23.491 INFO:tasks.workunit.client.1.smithi178.stdout:1/130: rmdir d3/de 39 2024-09-10T11:29:23.493 INFO:tasks.workunit.client.1.smithi178.stdout:6/283: write d3/d7/d8/d19/f26 [2079180,1229] 0 2024-09-10T11:29:23.494 INFO:tasks.workunit.client.1.smithi178.stdout:1/131: mkdir d3/d26/d2a 0 2024-09-10T11:29:23.495 INFO:tasks.workunit.client.1.smithi178.stdout:2/355: mknod d1/d41/d14/d11/d19/d55/d6e/c74 0 2024-09-10T11:29:23.499 INFO:tasks.workunit.client.1.smithi178.stdout:6/284: rename d3/d7/d8/d15/d1c/d20/d4b/d5f to d3/d4e/d67 0 2024-09-10T11:29:23.499 INFO:tasks.workunit.client.1.smithi178.stdout:6/285: dread - d3/d7/d35/f5a zero size 2024-09-10T11:29:23.500 INFO:tasks.workunit.client.1.smithi178.stdout:1/132: mkdir d3/d13/d2b 0 2024-09-10T11:29:23.500 INFO:tasks.workunit.client.1.smithi178.stdout:9/355: write d2/d8/f17 [347047,124400] 0 2024-09-10T11:29:23.501 INFO:tasks.workunit.client.1.smithi178.stdout:1/133: chown d3/d13/f24 32272686 1 2024-09-10T11:29:23.505 INFO:tasks.workunit.client.1.smithi178.stdout:9/356: fsync d2/d13/f1f 0 2024-09-10T11:29:23.507 INFO:tasks.workunit.client.1.smithi178.stdout:1/134: write d3/d9/d1a/f1f [4849263,39823] 0 2024-09-10T11:29:23.507 INFO:tasks.workunit.client.1.smithi178.stdout:9/357: write d2/dc/f44 [941041,88392] 0 2024-09-10T11:29:23.511 INFO:tasks.workunit.client.1.smithi178.stdout:6/286: creat d3/d42/d57/f68 x:0 0 0 2024-09-10T11:29:23.512 INFO:tasks.workunit.client.1.smithi178.stdout:2/356: truncate d1/d22/f26 1819116 0 2024-09-10T11:29:23.513 INFO:tasks.workunit.client.1.smithi178.stdout:1/135: write d3/f1c [3731393,87563] 0 2024-09-10T11:29:23.514 INFO:tasks.workunit.client.1.smithi178.stdout:2/357: truncate d1/d41/d14/f1b 4964697 0 2024-09-10T11:29:23.515 INFO:tasks.workunit.client.1.smithi178.stdout:2/358: write d1/d7/f6c [1553387,124015] 0 2024-09-10T11:29:23.516 INFO:tasks.workunit.client.1.smithi178.stdout:6/287: rename d3/d4e/d67 to d3/d24/d69 0 2024-09-10T11:29:23.517 INFO:tasks.workunit.client.1.smithi178.stdout:2/359: dread d1/d41/d14/d11/d19/d34/f17 [0,4194304] 0 2024-09-10T11:29:23.518 INFO:tasks.workunit.client.1.smithi178.stdout:9/358: truncate d2/dc/f2f 3559185 0 2024-09-10T11:29:23.519 INFO:tasks.workunit.client.1.smithi178.stdout:2/360: write d1/d7/f6c [3115321,86357] 0 2024-09-10T11:29:23.519 INFO:tasks.workunit.client.1.smithi178.stdout:6/288: chown d3/d7/d8/d15/l31 91 1 2024-09-10T11:29:23.522 INFO:tasks.workunit.client.1.smithi178.stdout:2/361: creat d1/d41/d14/d11/d19/d55/d6e/f75 x:0 0 0 2024-09-10T11:29:23.524 INFO:tasks.workunit.client.1.smithi178.stdout:9/359: truncate d2/d19/f39 2905035 0 2024-09-10T11:29:23.528 INFO:tasks.workunit.client.1.smithi178.stdout:9/360: mkdir d2/d19/d1c/d2d/d54/d60/d66 0 2024-09-10T11:29:23.529 INFO:tasks.workunit.client.1.smithi178.stdout:6/289: rmdir d3/d24/d69 0 2024-09-10T11:29:23.530 INFO:tasks.workunit.client.1.smithi178.stdout:2/362: rmdir d1/d41 39 2024-09-10T11:29:23.531 INFO:tasks.workunit.client.1.smithi178.stdout:6/290: fsync d3/d7/d8/d15/d1c/d2a/d18/f38 0 2024-09-10T11:29:23.536 INFO:tasks.workunit.client.1.smithi178.stdout:2/363: creat d1/d41/d5d/f76 x:0 0 0 2024-09-10T11:29:23.538 INFO:tasks.workunit.client.1.smithi178.stdout:9/361: dwrite d2/df/d31/d37/f63 [0,4194304] 0 2024-09-10T11:29:23.538 INFO:tasks.workunit.client.1.smithi178.stdout:2/364: mknod d1/d22/c77 0 2024-09-10T11:29:23.541 INFO:tasks.workunit.client.1.smithi178.stdout:9/362: creat d2/d19/d1c/d2a/f67 x:0 0 0 2024-09-10T11:29:23.542 INFO:tasks.workunit.client.1.smithi178.stdout:6/291: dwrite d3/d7/d35/f5a [0,4194304] 0 2024-09-10T11:29:23.543 INFO:tasks.workunit.client.1.smithi178.stdout:2/365: mkdir d1/d22/d4d/d78 0 2024-09-10T11:29:23.545 INFO:tasks.workunit.client.1.smithi178.stdout:9/363: mkdir d2/df/d31/d4a/d68 0 2024-09-10T11:29:23.547 INFO:tasks.workunit.client.1.smithi178.stdout:6/292: rename d3/f3a to d3/d42/d57/f6a 0 2024-09-10T11:29:23.547 INFO:tasks.workunit.client.1.smithi178.stdout:2/366: getdents d1/d22/d4d/d78 0 2024-09-10T11:29:23.548 INFO:tasks.workunit.client.1.smithi178.stdout:9/364: mknod d2/d13/d40/d4b/c69 0 2024-09-10T11:29:23.554 INFO:tasks.workunit.client.1.smithi178.stdout:6/293: dwrite d3/d42/d57/f68 [0,4194304] 0 2024-09-10T11:29:23.556 INFO:tasks.workunit.client.1.smithi178.stdout:6/294: mknod d3/d4e/c6b 0 2024-09-10T11:29:23.559 INFO:tasks.workunit.client.1.smithi178.stdout:9/365: truncate d2/d19/d1c/f24 4183300 0 2024-09-10T11:29:23.560 INFO:tasks.workunit.client.1.smithi178.stdout:6/295: symlink d3/d7/d8/d15/d48/l6c 0 2024-09-10T11:29:23.563 INFO:tasks.workunit.client.1.smithi178.stdout:6/296: mknod d3/d7/d8/d15/d1c/d20/c6d 0 2024-09-10T11:29:23.565 INFO:tasks.workunit.client.1.smithi178.stdout:2/367: dwrite d1/d41/d14/d11/d19/d34/d5b/d60/f64 [0,4194304] 0 2024-09-10T11:29:23.567 INFO:tasks.workunit.client.1.smithi178.stdout:2/368: fdatasync d1/d41/d14/d11/f2a 0 2024-09-10T11:29:23.571 INFO:tasks.workunit.client.1.smithi178.stdout:2/369: dread d1/d41/d14/d11/d19/d34/d5b/f6b [0,4194304] 0 2024-09-10T11:29:23.577 INFO:tasks.workunit.client.1.smithi178.stdout:2/370: mknod d1/d41/d14/d11/d19/d4b/c79 0 2024-09-10T11:29:23.579 INFO:tasks.workunit.client.1.smithi178.stdout:6/297: dwrite d3/d7/d8/d15/d1c/f1d [4194304,4194304] 0 2024-09-10T11:29:23.583 INFO:tasks.workunit.client.1.smithi178.stdout:6/298: mkdir d3/d6e 0 2024-09-10T11:29:23.584 INFO:tasks.workunit.client.1.smithi178.stdout:2/371: write d1/d22/f30 [3182488,18231] 0 2024-09-10T11:29:23.587 INFO:tasks.workunit.client.1.smithi178.stdout:2/372: symlink d1/d41/d14/d11/d53/l7a 0 2024-09-10T11:29:23.592 INFO:tasks.workunit.client.1.smithi178.stdout:6/299: dwrite d3/d7/d8/d15/d1c/f53 [0,4194304] 0 2024-09-10T11:29:23.592 INFO:tasks.workunit.client.1.smithi178.stdout:2/373: link d1/d41/d14/d11/lb d1/d41/d14/l7b 0 2024-09-10T11:29:23.592 INFO:tasks.workunit.client.1.smithi178.stdout:2/374: chown d1/d22/f3a 62 1 2024-09-10T11:29:23.593 INFO:tasks.workunit.client.1.smithi178.stdout:2/375: readlink d1/d41/d14/d11/d19/d34/l1d 0 2024-09-10T11:29:23.595 INFO:tasks.workunit.client.1.smithi178.stdout:4/95: sync 2024-09-10T11:29:23.595 INFO:tasks.workunit.client.1.smithi178.stdout:6/300: unlink d3/d7/d8/d15/d1c/d2a/f51 0 2024-09-10T11:29:23.597 INFO:tasks.workunit.client.1.smithi178.stdout:6/301: creat d3/d7/d35/f6f x:0 0 0 2024-09-10T11:29:23.598 INFO:tasks.workunit.client.1.smithi178.stdout:2/376: mkdir d1/d41/d14/d11/d19/d34/d5b/d7c 0 2024-09-10T11:29:23.601 INFO:tasks.workunit.client.1.smithi178.stdout:6/302: creat d3/d7/d8/d15/d1c/d2a/d18/f70 x:0 0 0 2024-09-10T11:29:23.602 INFO:tasks.workunit.client.1.smithi178.stdout:4/96: dwrite d0/f18 [0,4194304] 0 2024-09-10T11:29:23.603 INFO:tasks.workunit.client.1.smithi178.stdout:4/97: truncate d0/f1c 1008121 0 2024-09-10T11:29:23.606 INFO:tasks.workunit.client.1.smithi178.stdout:4/98: write d0/d9/f12 [2838864,45116] 0 2024-09-10T11:29:23.608 INFO:tasks.workunit.client.1.smithi178.stdout:6/303: dread d3/d7/d8/d15/d1c/f40 [0,4194304] 0 2024-09-10T11:29:23.608 INFO:tasks.workunit.client.1.smithi178.stdout:4/99: rmdir d0/d9/dc 39 2024-09-10T11:29:23.609 INFO:tasks.workunit.client.1.smithi178.stdout:6/304: chown d3/d7/d8/d15/d1c/d20/d4b 9946 1 2024-09-10T11:29:23.611 INFO:tasks.workunit.client.1.smithi178.stdout:4/100: getdents d0 0 2024-09-10T11:29:23.613 INFO:tasks.workunit.client.1.smithi178.stdout:4/101: rmdir d0/d9 39 2024-09-10T11:29:23.621 INFO:tasks.workunit.client.1.smithi178.stdout:4/102: dread d0/d9/f13 [0,4194304] 0 2024-09-10T11:29:23.621 INFO:tasks.workunit.client.1.smithi178.stdout:4/103: write d0/d8/da/f17 [459946,112086] 0 2024-09-10T11:29:23.624 INFO:tasks.workunit.client.1.smithi178.stdout:4/104: symlink d0/d8/df/l22 0 2024-09-10T11:29:23.627 INFO:tasks.workunit.client.1.smithi178.stdout:4/105: dread d0/f18 [0,4194304] 0 2024-09-10T11:29:23.629 INFO:tasks.workunit.client.1.smithi178.stdout:6/305: dread d3/d7/d8/d15/d1c/d20/f47 [4194304,4194304] 0 2024-09-10T11:29:23.630 INFO:tasks.workunit.client.1.smithi178.stdout:6/306: write d3/d7/d8/d15/d1c/f4d [656325,46999] 0 2024-09-10T11:29:23.632 INFO:tasks.workunit.client.1.smithi178.stdout:6/307: getdents d3/d6e 0 2024-09-10T11:29:23.635 INFO:tasks.workunit.client.1.smithi178.stdout:6/308: dread d3/d7/d8/d15/d1c/f1d [4194304,4194304] 0 2024-09-10T11:29:23.637 INFO:tasks.workunit.client.1.smithi178.stdout:6/309: creat d3/d6e/f71 x:0 0 0 2024-09-10T11:29:23.638 INFO:tasks.workunit.client.1.smithi178.stdout:6/310: write d3/d7/d8/d15/d1c/d2a/d18/f43 [821363,93260] 0 2024-09-10T11:29:23.642 INFO:tasks.workunit.client.1.smithi178.stdout:6/311: dread d3/d7/d8/d15/d1c/d2a/d18/f43 [0,4194304] 0 2024-09-10T11:29:23.647 INFO:tasks.workunit.client.1.smithi178.stdout:6/312: truncate d3/d42/f61 2187009 0 2024-09-10T11:29:23.648 INFO:tasks.workunit.client.1.smithi178.stdout:6/313: mkdir d3/d7/d8/d72 0 2024-09-10T11:29:23.649 INFO:tasks.workunit.client.1.smithi178.stdout:6/314: write d3/d7/d8/d15/d1c/f40 [2702854,17449] 0 2024-09-10T11:29:23.654 INFO:tasks.workunit.client.1.smithi178.stdout:6/315: dread d3/d7/d8/d15/d1c/f45 [0,4194304] 0 2024-09-10T11:29:23.658 INFO:tasks.workunit.client.1.smithi178.stdout:6/316: dread d3/d7/d8/d15/d1c/f53 [0,4194304] 0 2024-09-10T11:29:23.665 INFO:tasks.workunit.client.1.smithi178.stdout:6/317: dwrite d3/d7/d8/d15/d1c/f45 [0,4194304] 0 2024-09-10T11:29:23.669 INFO:tasks.workunit.client.1.smithi178.stdout:6/318: fsync d3/f14 0 2024-09-10T11:29:23.671 INFO:tasks.workunit.client.1.smithi178.stdout:6/319: mknod d3/d7/d35/c73 0 2024-09-10T11:29:23.671 INFO:tasks.workunit.client.1.smithi178.stdout:6/320: chown d3/d7/d8/d15 864760 1 2024-09-10T11:29:23.672 INFO:tasks.workunit.client.1.smithi178.stdout:6/321: truncate d3/d7/d8/d15/d1c/d2a/d18/f70 9251 0 2024-09-10T11:29:23.673 INFO:tasks.workunit.client.1.smithi178.stdout:6/322: dread - d3/d42/f66 zero size 2024-09-10T11:29:23.678 INFO:tasks.workunit.client.1.smithi178.stdout:6/323: truncate d3/d7/d8/d15/f25 209371 0 2024-09-10T11:29:23.678 INFO:tasks.workunit.client.1.smithi178.stdout:6/324: read d3/d7/d8/d19/f39 [1442616,11957] 0 2024-09-10T11:29:23.680 INFO:tasks.workunit.client.1.smithi178.stdout:6/325: mkdir d3/d74 0 2024-09-10T11:29:23.683 INFO:tasks.workunit.client.1.smithi178.stdout:6/326: mkdir d3/d7/d8/d72/d75 0 2024-09-10T11:29:23.735 INFO:tasks.workunit.client.1.smithi178.stdout:8/150: sync 2024-09-10T11:29:23.736 INFO:tasks.workunit.client.1.smithi178.stdout:5/165: sync 2024-09-10T11:29:23.736 INFO:tasks.workunit.client.1.smithi178.stdout:5/166: dread - d9/dc/d17/f3d zero size 2024-09-10T11:29:23.737 INFO:tasks.workunit.client.1.smithi178.stdout:5/167: fdatasync d9/dc/dd/d16/f41 0 2024-09-10T11:29:23.737 INFO:tasks.workunit.client.1.smithi178.stdout:5/168: readlink d9/dc/dd/d37/l3b 0 2024-09-10T11:29:23.739 INFO:tasks.workunit.client.1.smithi178.stdout:8/151: dread d5/db/d10/d20/f26 [0,4194304] 0 2024-09-10T11:29:23.739 INFO:tasks.workunit.client.1.smithi178.stdout:5/169: creat d9/dc/f47 x:0 0 0 2024-09-10T11:29:23.741 INFO:tasks.workunit.client.1.smithi178.stdout:5/170: mknod d9/dc/d17/d1d/c48 0 2024-09-10T11:29:23.742 INFO:tasks.workunit.client.1.smithi178.stdout:8/152: mknod d5/db/d10/d20/c32 0 2024-09-10T11:29:23.742 INFO:tasks.workunit.client.1.smithi178.stdout:5/171: write d9/f30 [33588,36422] 0 2024-09-10T11:29:23.742 INFO:tasks.workunit.client.1.smithi178.stdout:8/153: fdatasync d5/db/fe 0 2024-09-10T11:29:23.743 INFO:tasks.workunit.client.1.smithi178.stdout:5/172: rmdir d9/dc/d17/d1d 39 2024-09-10T11:29:23.746 INFO:tasks.workunit.client.1.smithi178.stdout:5/173: creat d9/dc/dd/d3f/f49 x:0 0 0 2024-09-10T11:29:23.747 INFO:tasks.workunit.client.1.smithi178.stdout:5/174: dread - d9/dc/f47 zero size 2024-09-10T11:29:23.749 INFO:tasks.workunit.client.1.smithi178.stdout:8/154: write d5/db/f14 [734910,24742] 0 2024-09-10T11:29:23.749 INFO:tasks.workunit.client.1.smithi178.stdout:8/155: stat d5/f13 0 2024-09-10T11:29:23.749 INFO:tasks.workunit.client.1.smithi178.stdout:5/175: mknod d9/dc/d17/d1d/c4a 0 2024-09-10T11:29:23.750 INFO:tasks.workunit.client.1.smithi178.stdout:8/156: write f2 [5172794,94213] 0 2024-09-10T11:29:23.752 INFO:tasks.workunit.client.1.smithi178.stdout:5/176: mkdir d9/dc/d17/d4b 0 2024-09-10T11:29:23.753 INFO:tasks.workunit.client.1.smithi178.stdout:8/157: mkdir d5/d30/d33 0 2024-09-10T11:29:23.753 INFO:tasks.workunit.client.1.smithi178.stdout:5/177: dread - d9/dc/dd/d16/f41 zero size 2024-09-10T11:29:23.755 INFO:tasks.workunit.client.1.smithi178.stdout:8/158: mknod d5/d1d/d22/c34 0 2024-09-10T11:29:23.760 INFO:tasks.workunit.client.1.smithi178.stdout:5/178: dwrite d9/d11/f2f [0,4194304] 0 2024-09-10T11:29:23.763 INFO:tasks.workunit.client.1.smithi178.stdout:5/179: truncate d9/dc/d17/f3d 355320 0 2024-09-10T11:29:23.764 INFO:tasks.workunit.client.1.smithi178.stdout:8/159: dwrite d5/d1d/d22/f23 [0,4194304] 0 2024-09-10T11:29:23.764 INFO:tasks.workunit.client.1.smithi178.stdout:8/160: readlink d5/l16 0 2024-09-10T11:29:23.765 INFO:tasks.workunit.client.1.smithi178.stdout:5/180: read d9/ff [1546507,7604] 0 2024-09-10T11:29:23.765 INFO:tasks.workunit.client.1.smithi178.stdout:8/161: chown d5/f1f 0 1 2024-09-10T11:29:23.773 INFO:tasks.workunit.client.1.smithi178.stdout:5/181: dwrite d9/d19/f1a [0,4194304] 0 2024-09-10T11:29:23.775 INFO:tasks.workunit.client.1.smithi178.stdout:5/182: symlink d9/dc/dd/d37/d3e/l4c 0 2024-09-10T11:29:23.776 INFO:tasks.workunit.client.1.smithi178.stdout:5/183: dread - d9/dc/dd/d16/f41 zero size 2024-09-10T11:29:23.778 INFO:tasks.workunit.client.1.smithi178.stdout:5/184: symlink d9/dc/d17/d1d/l4d 0 2024-09-10T11:29:23.780 INFO:tasks.workunit.client.1.smithi178.stdout:5/185: write d9/dc/d17/f44 [8479,28531] 0 2024-09-10T11:29:23.781 INFO:tasks.workunit.client.1.smithi178.stdout:8/162: fdatasync d5/d1d/d22/f23 0 2024-09-10T11:29:23.783 INFO:tasks.workunit.client.1.smithi178.stdout:8/163: mkdir d5/db/d10/d35 0 2024-09-10T11:29:23.784 INFO:tasks.workunit.client.1.smithi178.stdout:5/186: dread d9/dc/dd/d16/f27 [0,4194304] 0 2024-09-10T11:29:23.784 INFO:tasks.workunit.client.1.smithi178.stdout:5/187: fsync d9/dc/d17/f34 0 2024-09-10T11:29:23.785 INFO:tasks.workunit.client.1.smithi178.stdout:5/188: stat d9/lb 0 2024-09-10T11:29:23.786 INFO:tasks.workunit.client.1.smithi178.stdout:5/189: chown d9/dc/dd 0 1 2024-09-10T11:29:23.786 INFO:tasks.workunit.client.1.smithi178.stdout:8/164: dread d5/db/d10/d20/f2c [0,4194304] 0 2024-09-10T11:29:23.786 INFO:tasks.workunit.client.1.smithi178.stdout:5/190: chown d9/dc/d17/f1e 46 1 2024-09-10T11:29:23.788 INFO:tasks.workunit.client.1.smithi178.stdout:8/165: creat d5/d1d/d22/f36 x:0 0 0 2024-09-10T11:29:23.789 INFO:tasks.workunit.client.1.smithi178.stdout:5/191: write d9/dc/dd/d16/f27 [4602156,125719] 0 2024-09-10T11:29:23.789 INFO:tasks.workunit.client.1.smithi178.stdout:8/166: creat d5/f37 x:0 0 0 2024-09-10T11:29:23.790 INFO:tasks.workunit.client.1.smithi178.stdout:8/167: write d5/db/d10/d20/f28 [1473884,82566] 0 2024-09-10T11:29:23.792 INFO:tasks.workunit.client.1.smithi178.stdout:5/192: link d9/f28 d9/dc/dd/d37/d42/f4e 0 2024-09-10T11:29:23.793 INFO:tasks.workunit.client.1.smithi178.stdout:5/193: stat d9/dc/dd/ce 0 2024-09-10T11:29:23.795 INFO:tasks.workunit.client.1.smithi178.stdout:5/194: symlink d9/dc/d17/d1d/l4f 0 2024-09-10T11:29:23.795 INFO:tasks.workunit.client.1.smithi178.stdout:5/195: readlink d9/dc/d17/d1d/l4f 0 2024-09-10T11:29:23.796 INFO:tasks.workunit.client.1.smithi178.stdout:8/168: truncate d5/db/fe 1981453 0 2024-09-10T11:29:23.796 INFO:tasks.workunit.client.1.smithi178.stdout:8/169: chown d5/db/d10/d20/f28 326926271 1 2024-09-10T11:29:23.796 INFO:tasks.workunit.client.1.smithi178.stdout:8/170: stat d5/db/d10/c15 0 2024-09-10T11:29:23.799 INFO:tasks.workunit.client.1.smithi178.stdout:8/171: getdents d5/d1d/d22 0 2024-09-10T11:29:23.800 INFO:tasks.workunit.client.1.smithi178.stdout:8/172: truncate d5/f24 356389 0 2024-09-10T11:29:23.801 INFO:tasks.workunit.client.1.smithi178.stdout:8/173: symlink d5/d30/l38 0 2024-09-10T11:29:23.803 INFO:tasks.workunit.client.1.smithi178.stdout:8/174: creat d5/db/d1e/f39 x:0 0 0 2024-09-10T11:29:23.803 INFO:tasks.workunit.client.1.smithi178.stdout:8/175: readlink d5/db/d10/l2d 0 2024-09-10T11:29:23.808 INFO:tasks.workunit.client.1.smithi178.stdout:0/161: sync 2024-09-10T11:29:23.808 INFO:tasks.workunit.client.1.smithi178.stdout:3/169: sync 2024-09-10T11:29:23.819 INFO:tasks.workunit.client.1.smithi178.stdout:8/176: mkdir d5/d30/d33/d3a 0 2024-09-10T11:29:23.823 INFO:tasks.workunit.client.1.smithi178.stdout:0/162: dread d0/f8 [0,4194304] 0 2024-09-10T11:29:23.825 INFO:tasks.workunit.client.1.smithi178.stdout:0/163: read - d0/db/f24 zero size 2024-09-10T11:29:23.826 INFO:tasks.workunit.client.1.smithi178.stdout:8/177: dwrite d5/db/d10/f2f [0,4194304] 0 2024-09-10T11:29:23.828 INFO:tasks.workunit.client.1.smithi178.stdout:3/170: truncate db/f10 1629315 0 2024-09-10T11:29:23.829 INFO:tasks.workunit.client.1.smithi178.stdout:8/178: creat d5/db/d10/d20/f3b x:0 0 0 2024-09-10T11:29:23.831 INFO:tasks.workunit.client.1.smithi178.stdout:0/164: link d0/f8 d0/db/f32 0 2024-09-10T11:29:23.832 INFO:tasks.workunit.client.1.smithi178.stdout:3/171: mkdir db/d36 0 2024-09-10T11:29:23.834 INFO:tasks.workunit.client.1.smithi178.stdout:8/179: link d5/db/d10/d20/f28 d5/db/d10/d35/f3c 0 2024-09-10T11:29:23.835 INFO:tasks.workunit.client.1.smithi178.stdout:0/165: mkdir d0/d2c/d33 0 2024-09-10T11:29:23.836 INFO:tasks.workunit.client.1.smithi178.stdout:0/166: creat d0/d12/f34 x:0 0 0 2024-09-10T11:29:23.841 INFO:tasks.workunit.client.1.smithi178.stdout:8/180: dread d5/f7 [0,4194304] 0 2024-09-10T11:29:23.843 INFO:tasks.workunit.client.1.smithi178.stdout:0/167: link d0/dc/l30 d0/d19/l35 0 2024-09-10T11:29:23.843 INFO:tasks.workunit.client.1.smithi178.stdout:3/172: dwrite db/f10 [0,4194304] 0 2024-09-10T11:29:23.849 INFO:tasks.workunit.client.1.smithi178.stdout:3/173: creat db/d35/f37 x:0 0 0 2024-09-10T11:29:23.850 INFO:tasks.workunit.client.1.smithi178.stdout:8/181: dread d5/f13 [0,4194304] 0 2024-09-10T11:29:23.852 INFO:tasks.workunit.client.1.smithi178.stdout:3/174: stat db/l17 0 2024-09-10T11:29:23.853 INFO:tasks.workunit.client.1.smithi178.stdout:8/182: write d5/d1d/d22/f36 [572606,129024] 0 2024-09-10T11:29:23.854 INFO:tasks.workunit.client.1.smithi178.stdout:0/168: dwrite d0/d12/f27 [0,4194304] 0 2024-09-10T11:29:23.854 INFO:tasks.workunit.client.1.smithi178.stdout:3/175: truncate db/f27 1219761 0 2024-09-10T11:29:23.855 INFO:tasks.workunit.client.1.smithi178.stdout:0/169: chown d0/d19/c17 332 1 2024-09-10T11:29:23.857 INFO:tasks.workunit.client.1.smithi178.stdout:8/183: unlink d5/l27 0 2024-09-10T11:29:23.858 INFO:tasks.workunit.client.1.smithi178.stdout:0/170: chown d0/dc/ff 0 1 2024-09-10T11:29:23.860 INFO:tasks.workunit.client.1.smithi178.stdout:8/184: rmdir d5/db/d10/d35 39 2024-09-10T11:29:23.862 INFO:tasks.workunit.client.1.smithi178.stdout:0/171: mknod d0/dc/d1f/d2d/c36 0 2024-09-10T11:29:23.865 INFO:tasks.workunit.client.1.smithi178.stdout:0/172: creat d0/d2c/f37 x:0 0 0 2024-09-10T11:29:23.868 INFO:tasks.workunit.client.1.smithi178.stdout:3/176: dread db/f16 [0,4194304] 0 2024-09-10T11:29:23.869 INFO:tasks.workunit.client.1.smithi178.stdout:0/173: write d0/db/f24 [65978,5186] 0 2024-09-10T11:29:23.871 INFO:tasks.workunit.client.1.smithi178.stdout:8/185: dwrite d5/f7 [4194304,4194304] 0 2024-09-10T11:29:23.872 INFO:tasks.workunit.client.1.smithi178.stdout:3/177: symlink db/d36/l38 0 2024-09-10T11:29:23.874 INFO:tasks.workunit.client.1.smithi178.stdout:0/174: rename d0/dc/f16 to d0/db/f38 0 2024-09-10T11:29:23.876 INFO:tasks.workunit.client.1.smithi178.stdout:8/186: mkdir d5/db/d3d 0 2024-09-10T11:29:23.881 INFO:tasks.workunit.client.1.smithi178.stdout:3/178: dwrite db/d20/f21 [0,4194304] 0 2024-09-10T11:29:23.888 INFO:tasks.workunit.client.1.smithi178.stdout:8/187: fdatasync f2 0 2024-09-10T11:29:23.890 INFO:tasks.workunit.client.1.smithi178.stdout:3/179: dwrite db/fe [4194304,4194304] 0 2024-09-10T11:29:23.892 INFO:tasks.workunit.client.1.smithi178.stdout:8/188: mknod d5/d30/d33/c3e 0 2024-09-10T11:29:23.895 INFO:tasks.workunit.client.1.smithi178.stdout:8/189: fdatasync d5/db/d10/d20/f28 0 2024-09-10T11:29:23.899 INFO:tasks.workunit.client.1.smithi178.stdout:8/190: link d5/db/d10/d20/l29 d5/db/d1e/l3f 0 2024-09-10T11:29:23.900 INFO:tasks.workunit.client.1.smithi178.stdout:8/191: write d5/db/d10/f2f [4382629,59122] 0 2024-09-10T11:29:23.902 INFO:tasks.workunit.client.1.smithi178.stdout:8/192: rmdir d5/db/d10/d35 39 2024-09-10T11:29:23.903 INFO:tasks.workunit.client.1.smithi178.stdout:8/193: write d5/fa [193544,3805] 0 2024-09-10T11:29:23.904 INFO:tasks.workunit.client.1.smithi178.stdout:8/194: fsync d5/db/d10/d20/f28 0 2024-09-10T11:29:23.908 INFO:tasks.workunit.client.1.smithi178.stdout:8/195: write d5/db/fe [64174,128771] 0 2024-09-10T11:29:23.912 INFO:tasks.workunit.client.1.smithi178.stdout:8/196: dread f2 [4194304,4194304] 0 2024-09-10T11:29:23.912 INFO:tasks.workunit.client.1.smithi178.stdout:8/197: write d5/d1d/d22/f23 [4118637,61867] 0 2024-09-10T11:29:23.915 INFO:tasks.workunit.client.1.smithi178.stdout:8/198: rmdir d5/db/d10/d20 39 2024-09-10T11:29:23.917 INFO:tasks.workunit.client.1.smithi178.stdout:9/366: sync 2024-09-10T11:29:23.917 INFO:tasks.workunit.client.1.smithi178.stdout:1/136: sync 2024-09-10T11:29:23.917 INFO:tasks.workunit.client.1.smithi178.stdout:7/115: sync 2024-09-10T11:29:23.917 INFO:tasks.workunit.client.1.smithi178.stdout:2/377: sync 2024-09-10T11:29:23.917 INFO:tasks.workunit.client.1.smithi178.stdout:6/327: sync 2024-09-10T11:29:23.917 INFO:tasks.workunit.client.1.smithi178.stdout:4/106: sync 2024-09-10T11:29:23.918 INFO:tasks.workunit.client.1.smithi178.stdout:7/116: readlink l5 0 2024-09-10T11:29:23.918 INFO:tasks.workunit.client.1.smithi178.stdout:8/199: creat d5/db/d10/d20/f40 x:0 0 0 2024-09-10T11:29:23.918 INFO:tasks.workunit.client.1.smithi178.stdout:6/328: stat d3/d7/d35 0 2024-09-10T11:29:23.918 INFO:tasks.workunit.client.1.smithi178.stdout:9/367: chown d2/d13/d40 20014 1 2024-09-10T11:29:23.918 INFO:tasks.workunit.client.1.smithi178.stdout:1/137: chown d3/l4 729228 1 2024-09-10T11:29:23.919 INFO:tasks.workunit.client.1.smithi178.stdout:4/107: unlink d0/d10/c19 0 2024-09-10T11:29:23.921 INFO:tasks.workunit.client.1.smithi178.stdout:6/329: mknod d3/d7/d35/c76 0 2024-09-10T11:29:23.922 INFO:tasks.workunit.client.1.smithi178.stdout:2/378: mkdir d1/d7/d24/d7d 0 2024-09-10T11:29:23.967 INFO:tasks.workunit.client.1.smithi178.stdout:8/200: rename d5/c1c to d5/db/d10/c41 0 2024-09-10T11:29:23.968 INFO:tasks.workunit.client.1.smithi178.stdout:4/108: chown d0/l4 0 1 2024-09-10T11:29:23.971 INFO:tasks.workunit.client.1.smithi178.stdout:2/379: write d1/d41/f51 [2598290,86195] 0 2024-09-10T11:29:23.973 INFO:tasks.workunit.client.1.smithi178.stdout:6/330: write d3/d7/d8/d15/d1c/f1d [8302784,106104] 0 2024-09-10T11:29:23.973 INFO:tasks.workunit.client.1.smithi178.stdout:7/117: dwrite d6/d10/d1e/f26 [0,4194304] 0 2024-09-10T11:29:23.975 INFO:tasks.workunit.client.1.smithi178.stdout:9/368: dwrite d2/d8/f50 [0,4194304] 0 2024-09-10T11:29:23.979 INFO:tasks.workunit.client.1.smithi178.stdout:2/380: symlink d1/d41/d5d/l7e 0 2024-09-10T11:29:23.981 INFO:tasks.workunit.client.1.smithi178.stdout:7/118: mkdir d6/d29 0 2024-09-10T11:29:23.990 INFO:tasks.workunit.client.1.smithi178.stdout:7/119: unlink d6/d10/c13 0 2024-09-10T11:29:23.990 INFO:tasks.workunit.client.1.smithi178.stdout:8/201: dwrite d5/f24 [0,4194304] 0 2024-09-10T11:29:23.994 INFO:tasks.workunit.client.1.smithi178.stdout:7/120: fsync d6/d10/d15/f1c 0 2024-09-10T11:29:23.996 INFO:tasks.workunit.client.1.smithi178.stdout:4/109: dwrite d0/d9/f12 [0,4194304] 0 2024-09-10T11:29:23.997 INFO:tasks.workunit.client.1.smithi178.stdout:2/381: getdents d1/d7/d24 0 2024-09-10T11:29:23.998 INFO:tasks.workunit.client.1.smithi178.stdout:7/121: rename d6/d10 to d6/d10/d2a 22 2024-09-10T11:29:24.002 INFO:tasks.workunit.client.1.smithi178.stdout:9/369: dwrite d2/df/d31/d37/f5f [0,4194304] 0 2024-09-10T11:29:24.003 INFO:tasks.workunit.client.1.smithi178.stdout:5/196: sync 2024-09-10T11:29:24.003 INFO:tasks.workunit.client.1.smithi178.stdout:0/175: sync 2024-09-10T11:29:24.003 INFO:tasks.workunit.client.1.smithi178.stdout:6/331: dwrite d3/d7/d8/d15/d1c/f1d [0,4194304] 0 2024-09-10T11:29:24.007 INFO:tasks.workunit.client.1.smithi178.stdout:7/122: mkdir d6/d10/d15/d2b 0 2024-09-10T11:29:24.013 INFO:tasks.workunit.client.1.smithi178.stdout:9/370: creat d2/d19/d5e/f6a x:0 0 0 2024-09-10T11:29:24.016 INFO:tasks.workunit.client.1.smithi178.stdout:5/197: rename d9/dc/dd/d16/l22 to d9/dc/dd/d16/l50 0 2024-09-10T11:29:24.017 INFO:tasks.workunit.client.1.smithi178.stdout:6/332: dwrite d3/d6e/f71 [0,4194304] 0 2024-09-10T11:29:24.019 INFO:tasks.workunit.client.1.smithi178.stdout:4/110: link d0/d9/ce d0/c23 0 2024-09-10T11:29:24.019 INFO:tasks.workunit.client.1.smithi178.stdout:0/176: mkdir d0/dc/d39 0 2024-09-10T11:29:24.019 INFO:tasks.workunit.client.1.smithi178.stdout:7/123: creat d6/d10/d15/f2c x:0 0 0 2024-09-10T11:29:24.020 INFO:tasks.workunit.client.1.smithi178.stdout:5/198: fdatasync d9/dc/dd/d16/f41 0 2024-09-10T11:29:24.025 INFO:tasks.workunit.client.1.smithi178.stdout:9/371: truncate d2/d13/f48 78381 0 2024-09-10T11:29:24.027 INFO:tasks.workunit.client.1.smithi178.stdout:4/111: symlink d0/d21/l24 0 2024-09-10T11:29:24.029 INFO:tasks.workunit.client.1.smithi178.stdout:9/372: symlink d2/df/d31/d37/l6b 0 2024-09-10T11:29:24.030 INFO:tasks.workunit.client.1.smithi178.stdout:6/333: truncate d3/d42/d57/f6a 249841 0 2024-09-10T11:29:24.032 INFO:tasks.workunit.client.1.smithi178.stdout:4/112: mknod d0/d10/c25 0 2024-09-10T11:29:24.033 INFO:tasks.workunit.client.1.smithi178.stdout:5/199: dread d9/dc/dd/d16/f1c [0,4194304] 0 2024-09-10T11:29:24.035 INFO:tasks.workunit.client.1.smithi178.stdout:6/334: dread d3/d7/d8/d15/d1c/f4d [0,4194304] 0 2024-09-10T11:29:24.035 INFO:tasks.workunit.client.1.smithi178.stdout:9/373: chown d2/df/l64 830 1 2024-09-10T11:29:24.037 INFO:tasks.workunit.client.1.smithi178.stdout:7/124: write d6/ff [3183754,33186] 0 2024-09-10T11:29:24.040 INFO:tasks.workunit.client.1.smithi178.stdout:9/374: symlink d2/dc/l6c 0 2024-09-10T11:29:24.041 INFO:tasks.workunit.client.1.smithi178.stdout:4/113: creat d0/d9/dc/f26 x:0 0 0 2024-09-10T11:29:24.043 INFO:tasks.workunit.client.1.smithi178.stdout:4/114: readlink d0/l4 0 2024-09-10T11:29:24.044 INFO:tasks.workunit.client.1.smithi178.stdout:6/335: getdents d3/d7/d8/d56 0 2024-09-10T11:29:24.046 INFO:tasks.workunit.client.1.smithi178.stdout:2/382: dread d1/d41/d14/f59 [0,4194304] 0 2024-09-10T11:29:24.048 INFO:tasks.workunit.client.1.smithi178.stdout:7/125: rename d6/d10/d15 to d6/d10/d1e/d2d 0 2024-09-10T11:29:24.049 INFO:tasks.workunit.client.1.smithi178.stdout:0/177: dwrite d0/f9 [0,4194304] 0 2024-09-10T11:29:24.050 INFO:tasks.workunit.client.1.smithi178.stdout:4/115: creat d0/d8/da/f27 x:0 0 0 2024-09-10T11:29:24.052 INFO:tasks.workunit.client.1.smithi178.stdout:7/126: chown d6/ff 818488 1 2024-09-10T11:29:24.053 INFO:tasks.workunit.client.1.smithi178.stdout:2/383: unlink d1/d41/d14/f3e 0 2024-09-10T11:29:24.053 INFO:tasks.workunit.client.1.smithi178.stdout:5/200: dwrite d9/dc/dd/d16/f27 [4194304,4194304] 0 2024-09-10T11:29:24.054 INFO:tasks.workunit.client.1.smithi178.stdout:0/178: creat d0/db/f3a x:0 0 0 2024-09-10T11:29:24.055 INFO:tasks.workunit.client.1.smithi178.stdout:7/127: write d6/d10/d1d/f27 [483717,27351] 0 2024-09-10T11:29:24.058 INFO:tasks.workunit.client.1.smithi178.stdout:7/128: readlink d6/d10/l1b 0 2024-09-10T11:29:24.059 INFO:tasks.workunit.client.1.smithi178.stdout:6/336: truncate f0 4697194 0 2024-09-10T11:29:24.061 INFO:tasks.workunit.client.1.smithi178.stdout:7/129: readlink d6/d10/l17 0 2024-09-10T11:29:24.061 INFO:tasks.workunit.client.1.smithi178.stdout:5/201: creat d9/f51 x:0 0 0 2024-09-10T11:29:24.061 INFO:tasks.workunit.client.1.smithi178.stdout:6/337: rename d3/d42 to d3/d42/d57/d77 22 2024-09-10T11:29:24.064 INFO:tasks.workunit.client.1.smithi178.stdout:2/384: mknod d1/d41/d14/d11/d19/d34/d5e/c7f 0 2024-09-10T11:29:24.064 INFO:tasks.workunit.client.1.smithi178.stdout:7/130: write d6/f23 [429248,99500] 0 2024-09-10T11:29:24.067 INFO:tasks.workunit.client.1.smithi178.stdout:4/116: dwrite d0/d9/fb [0,4194304] 0 2024-09-10T11:29:24.072 INFO:tasks.workunit.client.1.smithi178.stdout:7/131: dwrite f3 [0,4194304] 0 2024-09-10T11:29:24.087 INFO:tasks.workunit.client.1.smithi178.stdout:4/117: mknod d0/d9/dc/c28 0 2024-09-10T11:29:24.091 INFO:tasks.workunit.client.1.smithi178.stdout:7/132: symlink d6/d10/d1e/d2d/d2b/l2e 0 2024-09-10T11:29:24.091 INFO:tasks.workunit.client.1.smithi178.stdout:0/179: truncate d0/db/f15 2396527 0 2024-09-10T11:29:24.092 INFO:tasks.workunit.client.1.smithi178.stdout:4/118: creat d0/d8/df/f29 x:0 0 0 2024-09-10T11:29:24.092 INFO:tasks.workunit.client.1.smithi178.stdout:7/133: dread - d6/d10/d1e/d2d/f21 zero size 2024-09-10T11:29:24.093 INFO:tasks.workunit.client.1.smithi178.stdout:5/202: write d9/dc/dd/d37/d42/f4e [2070918,123384] 0 2024-09-10T11:29:24.093 INFO:tasks.workunit.client.1.smithi178.stdout:4/119: chown d0/d9/fb 106 1 2024-09-10T11:29:24.095 INFO:tasks.workunit.client.1.smithi178.stdout:7/134: write d6/d10/d1e/d2d/f16 [824951,56809] 0 2024-09-10T11:29:24.096 INFO:tasks.workunit.client.1.smithi178.stdout:2/385: write d1/d41/d14/f59 [767515,23761] 0 2024-09-10T11:29:24.097 INFO:tasks.workunit.client.1.smithi178.stdout:6/338: truncate d3/d42/d57/f68 2363945 0 2024-09-10T11:29:24.099 INFO:tasks.workunit.client.1.smithi178.stdout:5/203: mknod d9/dc/d17/d1d/c52 0 2024-09-10T11:29:24.100 INFO:tasks.workunit.client.1.smithi178.stdout:0/180: dread d0/f2e [0,4194304] 0 2024-09-10T11:29:24.102 INFO:tasks.workunit.client.1.smithi178.stdout:4/120: mknod d0/d9/c2a 0 2024-09-10T11:29:24.103 INFO:tasks.workunit.client.1.smithi178.stdout:6/339: mkdir d3/d42/d57/d78 0 2024-09-10T11:29:24.103 INFO:tasks.workunit.client.1.smithi178.stdout:4/121: chown d0/d10 39850 1 2024-09-10T11:29:24.104 INFO:tasks.workunit.client.1.smithi178.stdout:4/122: chown d0/f1c 387107 1 2024-09-10T11:29:24.106 INFO:tasks.workunit.client.1.smithi178.stdout:5/204: symlink d9/dc/dd/d16/l53 0 2024-09-10T11:29:24.107 INFO:tasks.workunit.client.1.smithi178.stdout:0/181: truncate d0/dc/f21 4735050 0 2024-09-10T11:29:24.107 INFO:tasks.workunit.client.1.smithi178.stdout:4/123: creat d0/d8/da/f2b x:0 0 0 2024-09-10T11:29:24.110 INFO:tasks.workunit.client.1.smithi178.stdout:7/135: dread d6/fb [0,4194304] 0 2024-09-10T11:29:24.113 INFO:tasks.workunit.client.1.smithi178.stdout:0/182: creat d0/d2c/d33/f3b x:0 0 0 2024-09-10T11:29:24.113 INFO:tasks.workunit.client.1.smithi178.stdout:4/124: truncate d0/d8/da/f2b 942482 0 2024-09-10T11:29:24.114 INFO:tasks.workunit.client.1.smithi178.stdout:2/386: truncate d1/d7/d24/f42 3071721 0 2024-09-10T11:29:24.115 INFO:tasks.workunit.client.1.smithi178.stdout:7/136: symlink d6/d10/d1d/l2f 0 2024-09-10T11:29:24.115 INFO:tasks.workunit.client.1.smithi178.stdout:6/340: dwrite d3/d7/d35/f6f [0,4194304] 0 2024-09-10T11:29:24.118 INFO:tasks.workunit.client.1.smithi178.stdout:6/341: read d3/d7/d35/f4c [3007153,58142] 0 2024-09-10T11:29:24.121 INFO:tasks.workunit.client.1.smithi178.stdout:4/125: creat d0/d10/f2c x:0 0 0 2024-09-10T11:29:24.122 INFO:tasks.workunit.client.1.smithi178.stdout:5/205: dwrite d9/f30 [0,4194304] 0 2024-09-10T11:29:24.125 INFO:tasks.workunit.client.1.smithi178.stdout:7/137: creat d6/f30 x:0 0 0 2024-09-10T11:29:24.126 INFO:tasks.workunit.client.1.smithi178.stdout:4/126: chown d0/f5 31 1 2024-09-10T11:29:24.127 INFO:tasks.workunit.client.1.smithi178.stdout:2/387: write d1/d41/d14/f1b [5625629,5987] 0 2024-09-10T11:29:24.128 INFO:tasks.workunit.client.1.smithi178.stdout:6/342: symlink d3/d7/d8/d15/d1c/d20/l79 0 2024-09-10T11:29:24.130 INFO:tasks.workunit.client.1.smithi178.stdout:0/183: dwrite d0/dc/f21 [4194304,4194304] 0 2024-09-10T11:29:24.133 INFO:tasks.workunit.client.1.smithi178.stdout:5/206: creat d9/dc/d17/d1d/f54 x:0 0 0 2024-09-10T11:29:24.138 INFO:tasks.workunit.client.1.smithi178.stdout:3/180: sync 2024-09-10T11:29:24.139 INFO:tasks.workunit.client.1.smithi178.stdout:2/388: chown d1/d41/d14/d11/d19/d34/d5b/d60 53700628 1 2024-09-10T11:29:24.144 INFO:tasks.workunit.client.1.smithi178.stdout:0/184: unlink d0/d19/f28 0 2024-09-10T11:29:24.145 INFO:tasks.workunit.client.1.smithi178.stdout:7/138: dwrite d6/f14 [4194304,4194304] 0 2024-09-10T11:29:24.149 INFO:tasks.workunit.client.1.smithi178.stdout:2/389: truncate d1/f69 405951 0 2024-09-10T11:29:24.152 INFO:tasks.workunit.client.1.smithi178.stdout:4/127: dwrite d0/f1c [0,4194304] 0 2024-09-10T11:29:24.156 INFO:tasks.workunit.client.1.smithi178.stdout:5/207: dread d9/f30 [0,4194304] 0 2024-09-10T11:29:24.159 INFO:tasks.workunit.client.1.smithi178.stdout:7/139: fsync d6/ff 0 2024-09-10T11:29:24.159 INFO:tasks.workunit.client.1.smithi178.stdout:2/390: mknod d1/d41/d14/d11/d19/d55/c80 0 2024-09-10T11:29:24.160 INFO:tasks.workunit.client.1.smithi178.stdout:6/343: dwrite d3/d24/f30 [8388608,4194304] 0 2024-09-10T11:29:24.162 INFO:tasks.workunit.client.1.smithi178.stdout:0/185: write d0/db/f32 [1590920,117039] 0 2024-09-10T11:29:24.165 INFO:tasks.workunit.client.1.smithi178.stdout:2/391: truncate d1/d7/f6a 1047616 0 2024-09-10T11:29:24.166 INFO:tasks.workunit.client.1.smithi178.stdout:7/140: rmdir d6/d10 39 2024-09-10T11:29:24.167 INFO:tasks.workunit.client.1.smithi178.stdout:5/208: mkdir d9/dc/dd/d37/d42/d55 0 2024-09-10T11:29:24.169 INFO:tasks.workunit.client.1.smithi178.stdout:6/344: symlink d3/d24/l7a 0 2024-09-10T11:29:24.169 INFO:tasks.workunit.client.1.smithi178.stdout:5/209: stat d9/d19/c33 0 2024-09-10T11:29:24.170 INFO:tasks.workunit.client.1.smithi178.stdout:4/128: rename d0/d8/da/f17 to d0/d9/f2d 0 2024-09-10T11:29:24.171 INFO:tasks.workunit.client.1.smithi178.stdout:0/186: symlink d0/d19/l3c 0 2024-09-10T11:29:24.171 INFO:tasks.workunit.client.1.smithi178.stdout:3/181: dwrite db/d20/f28 [0,4194304] 0 2024-09-10T11:29:24.172 INFO:tasks.workunit.client.1.smithi178.stdout:2/392: dread d1/d41/d14/d11/d19/d34/f17 [0,4194304] 0 2024-09-10T11:29:24.173 INFO:tasks.workunit.client.1.smithi178.stdout:5/210: dread - d9/dc/dd/d16/f41 zero size 2024-09-10T11:29:24.174 INFO:tasks.workunit.client.1.smithi178.stdout:3/182: stat db/d36 0 2024-09-10T11:29:24.174 INFO:tasks.workunit.client.1.smithi178.stdout:7/141: chown d6/d10/d1d/d22 46337341 1 2024-09-10T11:29:24.177 INFO:tasks.workunit.client.1.smithi178.stdout:0/187: mknod d0/d2c/c3d 0 2024-09-10T11:29:24.178 INFO:tasks.workunit.client.1.smithi178.stdout:4/129: rename d0/l1 to d0/d9/dc/l2e 0 2024-09-10T11:29:24.178 INFO:tasks.workunit.client.1.smithi178.stdout:2/393: chown d1/d41/d14/d11/d53/f62 485015162 1 2024-09-10T11:29:24.180 INFO:tasks.workunit.client.1.smithi178.stdout:7/142: creat d6/d10/d1d/f31 x:0 0 0 2024-09-10T11:29:24.181 INFO:tasks.workunit.client.1.smithi178.stdout:5/211: mknod d9/dc/d17/d1d/c56 0 2024-09-10T11:29:24.183 INFO:tasks.workunit.client.1.smithi178.stdout:2/394: readlink d1/d41/d14/d11/d19/d34/d5e/l5f 0 2024-09-10T11:29:24.184 INFO:tasks.workunit.client.1.smithi178.stdout:6/345: truncate d3/d7/d8/d19/f39 1602713 0 2024-09-10T11:29:24.186 INFO:tasks.workunit.client.1.smithi178.stdout:5/212: creat d9/d19/f57 x:0 0 0 2024-09-10T11:29:24.187 INFO:tasks.workunit.client.1.smithi178.stdout:4/130: link d0/d8/df/l1f d0/d8/da/l2f 0 2024-09-10T11:29:24.189 INFO:tasks.workunit.client.1.smithi178.stdout:2/395: write d1/d41/d14/d11/d19/d4b/f67 [706254,80335] 0 2024-09-10T11:29:24.190 INFO:tasks.workunit.client.1.smithi178.stdout:7/143: getdents d6/d10 0 2024-09-10T11:29:24.190 INFO:tasks.workunit.client.1.smithi178.stdout:6/346: creat d3/d7/d8/d15/d1c/d20/d4b/f7b x:0 0 0 2024-09-10T11:29:24.192 INFO:tasks.workunit.client.1.smithi178.stdout:4/131: creat d0/d8/da/d1a/f30 x:0 0 0 2024-09-10T11:29:24.192 INFO:tasks.workunit.client.1.smithi178.stdout:1/138: sync 2024-09-10T11:29:24.192 INFO:tasks.workunit.client.1.smithi178.stdout:8/202: sync 2024-09-10T11:29:24.193 INFO:tasks.workunit.client.1.smithi178.stdout:2/396: fsync d1/d41/f44 0 2024-09-10T11:29:24.193 INFO:tasks.workunit.client.1.smithi178.stdout:3/183: dwrite db/d26/f2f [0,4194304] 0 2024-09-10T11:29:24.194 INFO:tasks.workunit.client.1.smithi178.stdout:0/188: dwrite d0/f22 [4194304,4194304] 0 2024-09-10T11:29:24.197 INFO:tasks.workunit.client.1.smithi178.stdout:0/189: chown d0/d19 16338 1 2024-09-10T11:29:24.198 INFO:tasks.workunit.client.1.smithi178.stdout:2/397: fdatasync d1/d22/f30 0 2024-09-10T11:29:24.201 INFO:tasks.workunit.client.1.smithi178.stdout:3/184: dread f1 [0,4194304] 0 2024-09-10T11:29:24.201 INFO:tasks.workunit.client.1.smithi178.stdout:5/213: rename d9/dc/d17/d1d/c52 to d9/dc/dd/d3f/c58 0 2024-09-10T11:29:24.203 INFO:tasks.workunit.client.1.smithi178.stdout:8/203: unlink d5/db/d10/f2f 0 2024-09-10T11:29:24.204 INFO:tasks.workunit.client.1.smithi178.stdout:4/132: creat d0/d8/da/f31 x:0 0 0 2024-09-10T11:29:24.205 INFO:tasks.workunit.client.1.smithi178.stdout:5/214: unlink d9/dc/d17/d1d/l4d 0 2024-09-10T11:29:24.206 INFO:tasks.workunit.client.1.smithi178.stdout:4/133: write d0/d8/da/f2b [973574,94502] 0 2024-09-10T11:29:24.208 INFO:tasks.workunit.client.1.smithi178.stdout:7/144: symlink d6/d10/d1d/d22/l32 0 2024-09-10T11:29:24.209 INFO:tasks.workunit.client.1.smithi178.stdout:6/347: fsync d3/d7/d8/d15/f25 0 2024-09-10T11:29:24.212 INFO:tasks.workunit.client.1.smithi178.stdout:2/398: mknod d1/d7/d24/d7d/c81 0 2024-09-10T11:29:24.214 INFO:tasks.workunit.client.1.smithi178.stdout:1/139: creat d3/f2c x:0 0 0 2024-09-10T11:29:24.214 INFO:tasks.workunit.client.1.smithi178.stdout:6/348: write d3/d7/d8/d19/f26 [3069366,63695] 0 2024-09-10T11:29:24.217 INFO:tasks.workunit.client.1.smithi178.stdout:4/134: dread d0/d8/da/f2b [0,4194304] 0 2024-09-10T11:29:24.218 INFO:tasks.workunit.client.1.smithi178.stdout:8/204: symlink d5/db/d10/d35/l42 0 2024-09-10T11:29:24.220 INFO:tasks.workunit.client.1.smithi178.stdout:2/399: rmdir d1/d41/d14/d11 39 2024-09-10T11:29:24.220 INFO:tasks.workunit.client.1.smithi178.stdout:1/140: creat d3/d13/f2d x:0 0 0 2024-09-10T11:29:24.221 INFO:tasks.workunit.client.1.smithi178.stdout:3/185: dread db/d20/f28 [0,4194304] 0 2024-09-10T11:29:24.223 INFO:tasks.workunit.client.1.smithi178.stdout:4/135: creat d0/d10/f32 x:0 0 0 2024-09-10T11:29:24.223 INFO:tasks.workunit.client.1.smithi178.stdout:3/186: fsync db/f10 0 2024-09-10T11:29:24.227 INFO:tasks.workunit.client.1.smithi178.stdout:7/145: dwrite d6/ff [0,4194304] 0 2024-09-10T11:29:24.232 INFO:tasks.workunit.client.1.smithi178.stdout:3/187: write db/d20/f34 [394630,6323] 0 2024-09-10T11:29:24.237 INFO:tasks.workunit.client.1.smithi178.stdout:7/146: chown d6/f14 3 1 2024-09-10T11:29:24.237 INFO:tasks.workunit.client.1.smithi178.stdout:1/141: rename d3/d9/fc to d3/de/f2e 0 2024-09-10T11:29:24.238 INFO:tasks.workunit.client.1.smithi178.stdout:6/349: dwrite d3/d7/d8/d15/d1c/d20/f47 [4194304,4194304] 0 2024-09-10T11:29:24.238 INFO:tasks.workunit.client.1.smithi178.stdout:2/400: creat d1/d41/d14/f82 x:0 0 0 2024-09-10T11:29:24.239 INFO:tasks.workunit.client.1.smithi178.stdout:5/215: dread d9/dc/f2b [0,4194304] 0 2024-09-10T11:29:24.241 INFO:tasks.workunit.client.1.smithi178.stdout:7/147: symlink d6/d10/l33 0 2024-09-10T11:29:24.245 INFO:tasks.workunit.client.1.smithi178.stdout:3/188: creat db/f39 x:0 0 0 2024-09-10T11:29:24.245 INFO:tasks.workunit.client.1.smithi178.stdout:8/205: dwrite d5/db/d10/d20/f28 [0,4194304] 0 2024-09-10T11:29:24.245 INFO:tasks.workunit.client.1.smithi178.stdout:4/136: dwrite d0/f1c [0,4194304] 0 2024-09-10T11:29:24.246 INFO:tasks.workunit.client.1.smithi178.stdout:2/401: stat d1/d41/d14/d11/d19/d34/d5e/c7f 0 2024-09-10T11:29:24.247 INFO:tasks.workunit.client.1.smithi178.stdout:3/189: write db/f39 [271532,65537] 0 2024-09-10T11:29:24.248 INFO:tasks.workunit.client.1.smithi178.stdout:7/148: mkdir d6/d10/d34 0 2024-09-10T11:29:24.254 INFO:tasks.workunit.client.1.smithi178.stdout:8/206: unlink d5/l9 0 2024-09-10T11:29:24.254 INFO:tasks.workunit.client.1.smithi178.stdout:7/149: chown d6/d10/d1e/d2d/f21 4 1 2024-09-10T11:29:24.257 INFO:tasks.workunit.client.1.smithi178.stdout:4/137: chown d0/l4 2634 1 2024-09-10T11:29:24.258 INFO:tasks.workunit.client.1.smithi178.stdout:6/350: write d3/d7/d8/d15/d1c/f53 [2270235,90361] 0 2024-09-10T11:29:24.262 INFO:tasks.workunit.client.1.smithi178.stdout:2/402: dread d1/d41/d14/d11/d19/f36 [0,4194304] 0 2024-09-10T11:29:24.262 INFO:tasks.workunit.client.1.smithi178.stdout:4/138: symlink d0/d9/dc/l33 0 2024-09-10T11:29:24.265 INFO:tasks.workunit.client.1.smithi178.stdout:5/216: write d9/dc/f15 [1378821,85400] 0 2024-09-10T11:29:24.269 INFO:tasks.workunit.client.1.smithi178.stdout:4/139: dread - d0/d8/da/d1a/f30 zero size 2024-09-10T11:29:24.269 INFO:tasks.workunit.client.1.smithi178.stdout:1/142: truncate d3/d9/f25 3706583 0 2024-09-10T11:29:24.271 INFO:tasks.workunit.client.1.smithi178.stdout:3/190: dwrite db/f15 [8388608,4194304] 0 2024-09-10T11:29:24.277 INFO:tasks.workunit.client.1.smithi178.stdout:8/207: dwrite d5/db/f17 [0,4194304] 0 2024-09-10T11:29:24.278 INFO:tasks.workunit.client.1.smithi178.stdout:1/143: symlink d3/d26/l2f 0 2024-09-10T11:29:24.285 INFO:tasks.workunit.client.1.smithi178.stdout:4/140: truncate d0/d8/da/d1a/f30 143164 0 2024-09-10T11:29:24.285 INFO:tasks.workunit.client.1.smithi178.stdout:7/150: dwrite d6/d10/d1e/d2d/f1c [0,4194304] 0 2024-09-10T11:29:24.286 INFO:tasks.workunit.client.1.smithi178.stdout:5/217: rename d9/dc/d17/d1d/c48 to d9/dc/dd/d16/c59 0 2024-09-10T11:29:24.288 INFO:tasks.workunit.client.1.smithi178.stdout:4/141: chown d0/d10/f32 15515898 1 2024-09-10T11:29:24.289 INFO:tasks.workunit.client.1.smithi178.stdout:6/351: dwrite d3/d42/f66 [0,4194304] 0 2024-09-10T11:29:24.289 INFO:tasks.workunit.client.1.smithi178.stdout:5/218: fsync d9/dc/d17/f1e 0 2024-09-10T11:29:24.289 INFO:tasks.workunit.client.1.smithi178.stdout:8/208: mknod d5/db/d1e/c43 0 2024-09-10T11:29:24.290 INFO:tasks.workunit.client.1.smithi178.stdout:2/403: dwrite d1/d41/d14/d11/d19/f2d [0,4194304] 0 2024-09-10T11:29:24.290 INFO:tasks.workunit.client.1.smithi178.stdout:7/151: creat d6/d10/d1e/f35 x:0 0 0 2024-09-10T11:29:24.291 INFO:tasks.workunit.client.1.smithi178.stdout:1/144: rename d3/de/c19 to d3/d13/d2b/c30 0 2024-09-10T11:29:24.295 INFO:tasks.workunit.client.1.smithi178.stdout:4/142: creat d0/d9/dc/f34 x:0 0 0 2024-09-10T11:29:24.296 INFO:tasks.workunit.client.1.smithi178.stdout:8/209: rmdir d5/db/d10/d35 39 2024-09-10T11:29:24.299 INFO:tasks.workunit.client.1.smithi178.stdout:5/219: mkdir d9/dc/dd/d3f/d5a 0 2024-09-10T11:29:24.301 INFO:tasks.workunit.client.1.smithi178.stdout:1/145: mknod d3/d13/c31 0 2024-09-10T11:29:24.303 INFO:tasks.workunit.client.1.smithi178.stdout:7/152: dwrite d6/f14 [4194304,4194304] 0 2024-09-10T11:29:24.303 INFO:tasks.workunit.client.1.smithi178.stdout:2/404: mknod d1/c83 0 2024-09-10T11:29:24.304 INFO:tasks.workunit.client.1.smithi178.stdout:7/153: write d6/f14 [1953997,52229] 0 2024-09-10T11:29:24.304 INFO:tasks.workunit.client.1.smithi178.stdout:2/405: readlink d1/d41/d14/d11/d19/d34/l1d 0 2024-09-10T11:29:24.306 INFO:tasks.workunit.client.1.smithi178.stdout:6/352: mkdir d3/d42/d57/d78/d7c 0 2024-09-10T11:29:24.307 INFO:tasks.workunit.client.1.smithi178.stdout:1/146: creat d3/d13/f32 x:0 0 0 2024-09-10T11:29:24.308 INFO:tasks.workunit.client.1.smithi178.stdout:6/353: truncate d3/d7/d8/f58 105002 0 2024-09-10T11:29:24.309 INFO:tasks.workunit.client.1.smithi178.stdout:8/210: creat d5/db/d10/d35/f44 x:0 0 0 2024-09-10T11:29:24.312 INFO:tasks.workunit.client.1.smithi178.stdout:4/143: rmdir d0/d9/dc 39 2024-09-10T11:29:24.316 INFO:tasks.workunit.client.1.smithi178.stdout:2/406: rename d1/d41/d14/d11/d19 to d1/d7/d24/d7d/d84 0 2024-09-10T11:29:24.316 INFO:tasks.workunit.client.1.smithi178.stdout:6/354: rename d3 to d3/d42/d7d 22 2024-09-10T11:29:24.318 INFO:tasks.workunit.client.1.smithi178.stdout:7/154: creat d6/d10/d1e/d24/f36 x:0 0 0 2024-09-10T11:29:24.319 INFO:tasks.workunit.client.1.smithi178.stdout:7/155: write d6/d10/d1e/d2d/f1c [2899425,110732] 0 2024-09-10T11:29:24.319 INFO:tasks.workunit.client.1.smithi178.stdout:4/144: symlink d0/d10/l35 0 2024-09-10T11:29:24.319 INFO:tasks.workunit.client.1.smithi178.stdout:1/147: mknod d3/c33 0 2024-09-10T11:29:24.321 INFO:tasks.workunit.client.1.smithi178.stdout:2/407: read d1/d7/d24/f42 [2911384,26649] 0 2024-09-10T11:29:24.324 INFO:tasks.workunit.client.1.smithi178.stdout:5/220: truncate d9/dc/d17/f3c 377291 0 2024-09-10T11:29:24.325 INFO:tasks.workunit.client.1.smithi178.stdout:4/145: mknod d0/d8/c36 0 2024-09-10T11:29:24.327 INFO:tasks.workunit.client.1.smithi178.stdout:8/211: truncate f2 886191 0 2024-09-10T11:29:24.328 INFO:tasks.workunit.client.1.smithi178.stdout:8/212: write d5/f24 [4376188,38449] 0 2024-09-10T11:29:24.330 INFO:tasks.workunit.client.1.smithi178.stdout:4/146: chown d0/d8/da/l2f 163033 1 2024-09-10T11:29:24.330 INFO:tasks.workunit.client.1.smithi178.stdout:8/213: write d5/db/d10/d20/f28 [3646578,31442] 0 2024-09-10T11:29:24.330 INFO:tasks.workunit.client.1.smithi178.stdout:1/148: creat d3/d26/d2a/f34 x:0 0 0 2024-09-10T11:29:24.331 INFO:tasks.workunit.client.1.smithi178.stdout:1/149: stat d3/c5 0 2024-09-10T11:29:24.332 INFO:tasks.workunit.client.1.smithi178.stdout:4/147: readlink d0/d8/da/d1a/l1e 0 2024-09-10T11:29:24.332 INFO:tasks.workunit.client.1.smithi178.stdout:1/150: dread - d3/d13/f29 zero size 2024-09-10T11:29:24.333 INFO:tasks.workunit.client.1.smithi178.stdout:8/214: write d5/db/d10/d35/f3c [3939730,124968] 0 2024-09-10T11:29:24.337 INFO:tasks.workunit.client.1.smithi178.stdout:4/148: chown d0/d9/dc/f26 9700081 1 2024-09-10T11:29:24.337 INFO:tasks.workunit.client.1.smithi178.stdout:8/215: rmdir d5/d1d 39 2024-09-10T11:29:24.342 INFO:tasks.workunit.client.1.smithi178.stdout:4/149: creat d0/d21/f37 x:0 0 0 2024-09-10T11:29:24.342 INFO:tasks.workunit.client.1.smithi178.stdout:7/156: dwrite d6/fd [0,4194304] 0 2024-09-10T11:29:24.342 INFO:tasks.workunit.client.1.smithi178.stdout:4/150: stat d0/d8/c36 0 2024-09-10T11:29:24.342 INFO:tasks.workunit.client.1.smithi178.stdout:1/151: dwrite d3/d13/f24 [0,4194304] 0 2024-09-10T11:29:24.350 INFO:tasks.workunit.client.1.smithi178.stdout:8/216: write d5/f8 [4020204,100516] 0 2024-09-10T11:29:24.352 INFO:tasks.workunit.client.1.smithi178.stdout:8/217: read d5/db/d10/d35/f3c [2444788,64748] 0 2024-09-10T11:29:24.357 INFO:tasks.workunit.client.1.smithi178.stdout:7/157: dwrite d6/f30 [0,4194304] 0 2024-09-10T11:29:24.359 INFO:tasks.workunit.client.1.smithi178.stdout:1/152: dwrite d3/d13/f24 [0,4194304] 0 2024-09-10T11:29:24.360 INFO:tasks.workunit.client.1.smithi178.stdout:9/375: sync 2024-09-10T11:29:24.361 INFO:tasks.workunit.client.1.smithi178.stdout:4/151: dwrite d0/f18 [0,4194304] 0 2024-09-10T11:29:24.362 INFO:tasks.workunit.client.1.smithi178.stdout:7/158: dread - d6/d10/d1e/d2d/f2c zero size 2024-09-10T11:29:24.363 INFO:tasks.workunit.client.1.smithi178.stdout:9/376: fdatasync d2/dc/f44 0 2024-09-10T11:29:24.365 INFO:tasks.workunit.client.1.smithi178.stdout:1/153: mkdir d3/d35 0 2024-09-10T11:29:24.368 INFO:tasks.workunit.client.1.smithi178.stdout:4/152: stat d0/d8/da/l2f 0 2024-09-10T11:29:24.368 INFO:tasks.workunit.client.1.smithi178.stdout:9/377: fdatasync d2/d19/d1c/d2d/f45 0 2024-09-10T11:29:24.369 INFO:tasks.workunit.client.1.smithi178.stdout:4/153: stat d0/d10/c1b 0 2024-09-10T11:29:24.369 INFO:tasks.workunit.client.1.smithi178.stdout:8/218: dwrite d5/db/d10/d20/f26 [0,4194304] 0 2024-09-10T11:29:24.371 INFO:tasks.workunit.client.1.smithi178.stdout:1/154: truncate d3/d26/f28 826658 0 2024-09-10T11:29:24.371 INFO:tasks.workunit.client.1.smithi178.stdout:4/154: dread - d0/d9/dc/f34 zero size 2024-09-10T11:29:24.372 INFO:tasks.workunit.client.1.smithi178.stdout:7/159: mknod d6/c37 0 2024-09-10T11:29:24.372 INFO:tasks.workunit.client.1.smithi178.stdout:9/378: read - d2/d19/d1c/d2d/f55 zero size 2024-09-10T11:29:24.374 INFO:tasks.workunit.client.1.smithi178.stdout:7/160: truncate d6/d10/d1e/f35 353713 0 2024-09-10T11:29:24.376 INFO:tasks.workunit.client.1.smithi178.stdout:9/379: creat d2/d19/d1c/d2a/f6d x:0 0 0 2024-09-10T11:29:24.379 INFO:tasks.workunit.client.1.smithi178.stdout:1/155: rename d3/d26/l2f to d3/d26/d2a/l36 0 2024-09-10T11:29:24.379 INFO:tasks.workunit.client.1.smithi178.stdout:9/380: chown d2/df/d31/l4c 464 1 2024-09-10T11:29:24.380 INFO:tasks.workunit.client.1.smithi178.stdout:7/161: symlink d6/d10/d1e/d2d/l38 0 2024-09-10T11:29:24.381 INFO:tasks.workunit.client.1.smithi178.stdout:4/155: dread d0/d9/f13 [0,4194304] 0 2024-09-10T11:29:24.381 INFO:tasks.workunit.client.1.smithi178.stdout:8/219: dwrite d5/db/d10/d20/f40 [0,4194304] 0 2024-09-10T11:29:24.381 INFO:tasks.workunit.client.1.smithi178.stdout:1/156: dread - d3/d13/f2d zero size 2024-09-10T11:29:24.383 INFO:tasks.workunit.client.1.smithi178.stdout:4/156: rmdir d0/d8 39 2024-09-10T11:29:24.383 INFO:tasks.workunit.client.1.smithi178.stdout:1/157: write d3/f1c [2508720,74646] 0 2024-09-10T11:29:24.387 INFO:tasks.workunit.client.1.smithi178.stdout:8/220: creat d5/db/d1e/f45 x:0 0 0 2024-09-10T11:29:24.388 INFO:tasks.workunit.client.1.smithi178.stdout:7/162: dwrite d6/d10/d1e/d2d/f21 [0,4194304] 0 2024-09-10T11:29:24.388 INFO:tasks.workunit.client.1.smithi178.stdout:8/221: write d5/db/f17 [1117877,1149] 0 2024-09-10T11:29:24.390 INFO:tasks.workunit.client.1.smithi178.stdout:9/381: creat d2/df/d31/d37/d3d/f6e x:0 0 0 2024-09-10T11:29:24.391 INFO:tasks.workunit.client.1.smithi178.stdout:4/157: unlink d0/d10/c11 0 2024-09-10T11:29:24.397 INFO:tasks.workunit.client.1.smithi178.stdout:7/163: creat d6/d10/d1e/d2d/d2b/f39 x:0 0 0 2024-09-10T11:29:24.398 INFO:tasks.workunit.client.1.smithi178.stdout:8/222: unlink f2 0 2024-09-10T11:29:24.402 INFO:tasks.workunit.client.1.smithi178.stdout:7/164: write d6/d10/d1e/d2d/f21 [16165,2958] 0 2024-09-10T11:29:24.402 INFO:tasks.workunit.client.1.smithi178.stdout:8/223: mknod d5/db/c46 0 2024-09-10T11:29:24.404 INFO:tasks.workunit.client.1.smithi178.stdout:4/158: dwrite d0/d9/dc/f26 [0,4194304] 0 2024-09-10T11:29:24.405 INFO:tasks.workunit.client.1.smithi178.stdout:1/158: dwrite d3/de/f2e [0,4194304] 0 2024-09-10T11:29:24.407 INFO:tasks.workunit.client.1.smithi178.stdout:1/159: write d3/d13/f2d [541176,26651] 0 2024-09-10T11:29:24.411 INFO:tasks.workunit.client.1.smithi178.stdout:7/165: creat d6/d10/d1d/d22/f3a x:0 0 0 2024-09-10T11:29:24.411 INFO:tasks.workunit.client.1.smithi178.stdout:7/166: stat d6/d10/d1e/d2d/f2c 0 2024-09-10T11:29:24.414 INFO:tasks.workunit.client.1.smithi178.stdout:7/167: mkdir d6/d10/d1d/d22/d3b 0 2024-09-10T11:29:24.414 INFO:tasks.workunit.client.1.smithi178.stdout:1/160: creat d3/d9/d1a/f37 x:0 0 0 2024-09-10T11:29:24.415 INFO:tasks.workunit.client.1.smithi178.stdout:8/224: rename d5/d1d/d22/c34 to d5/d1d/c47 0 2024-09-10T11:29:24.417 INFO:tasks.workunit.client.1.smithi178.stdout:1/161: rename d3/c6 to d3/d35/c38 0 2024-09-10T11:29:24.418 INFO:tasks.workunit.client.1.smithi178.stdout:1/162: write d3/d13/f24 [2691170,63211] 0 2024-09-10T11:29:24.418 INFO:tasks.workunit.client.1.smithi178.stdout:4/159: write d0/f5 [4198406,46503] 0 2024-09-10T11:29:24.419 INFO:tasks.workunit.client.1.smithi178.stdout:4/160: chown d0/d9/dc/l16 2703425 1 2024-09-10T11:29:24.422 INFO:tasks.workunit.client.1.smithi178.stdout:4/161: write d0/f5 [3551467,56388] 0 2024-09-10T11:29:24.425 INFO:tasks.workunit.client.1.smithi178.stdout:1/163: dwrite d3/d13/f24 [0,4194304] 0 2024-09-10T11:29:24.426 INFO:tasks.workunit.client.1.smithi178.stdout:8/225: write d5/db/d10/d20/f2b [2535513,90268] 0 2024-09-10T11:29:24.426 INFO:tasks.workunit.client.1.smithi178.stdout:1/164: readlink d3/l4 0 2024-09-10T11:29:24.427 INFO:tasks.workunit.client.1.smithi178.stdout:1/165: stat d3/d35 0 2024-09-10T11:29:24.430 INFO:tasks.workunit.client.1.smithi178.stdout:1/166: rmdir d3/d13/d2b 39 2024-09-10T11:29:24.430 INFO:tasks.workunit.client.1.smithi178.stdout:4/162: creat d0/f38 x:0 0 0 2024-09-10T11:29:24.434 INFO:tasks.workunit.client.1.smithi178.stdout:8/226: mkdir d5/d30/d48 0 2024-09-10T11:29:24.435 INFO:tasks.workunit.client.1.smithi178.stdout:4/163: symlink d0/d8/df/l39 0 2024-09-10T11:29:24.436 INFO:tasks.workunit.client.1.smithi178.stdout:1/167: dwrite d3/d9/fb [0,4194304] 0 2024-09-10T11:29:24.437 INFO:tasks.workunit.client.1.smithi178.stdout:4/164: dread d0/d8/da/d1a/f30 [0,4194304] 0 2024-09-10T11:29:24.439 INFO:tasks.workunit.client.1.smithi178.stdout:8/227: rename d5/db/f17 to d5/d30/d33/f49 0 2024-09-10T11:29:24.440 INFO:tasks.workunit.client.1.smithi178.stdout:1/168: rmdir d3/d13 39 2024-09-10T11:29:24.441 INFO:tasks.workunit.client.1.smithi178.stdout:4/165: dread d0/d9/f13 [0,4194304] 0 2024-09-10T11:29:24.441 INFO:tasks.workunit.client.1.smithi178.stdout:8/228: chown d5/l16 175470 1 2024-09-10T11:29:24.442 INFO:tasks.workunit.client.1.smithi178.stdout:1/169: fsync d3/d13/f17 0 2024-09-10T11:29:24.448 INFO:tasks.workunit.client.1.smithi178.stdout:1/170: dread d3/d13/f24 [0,4194304] 0 2024-09-10T11:29:24.449 INFO:tasks.workunit.client.1.smithi178.stdout:1/171: write d3/d9/d1a/f37 [918473,46318] 0 2024-09-10T11:29:24.451 INFO:tasks.workunit.client.1.smithi178.stdout:4/166: truncate d0/f18 3821999 0 2024-09-10T11:29:24.452 INFO:tasks.workunit.client.1.smithi178.stdout:4/167: chown d0/d10/f32 0 1 2024-09-10T11:29:24.452 INFO:tasks.workunit.client.1.smithi178.stdout:1/172: mknod d3/d13/d2b/c39 0 2024-09-10T11:29:24.452 INFO:tasks.workunit.client.1.smithi178.stdout:8/229: truncate d5/db/fe 1641733 0 2024-09-10T11:29:24.453 INFO:tasks.workunit.client.1.smithi178.stdout:4/168: truncate d0/d8/df/f20 17724 0 2024-09-10T11:29:24.454 INFO:tasks.workunit.client.1.smithi178.stdout:4/169: symlink d0/d8/df/l3a 0 2024-09-10T11:29:24.455 INFO:tasks.workunit.client.1.smithi178.stdout:4/170: creat d0/d8/df/f3b x:0 0 0 2024-09-10T11:29:24.456 INFO:tasks.workunit.client.1.smithi178.stdout:4/171: truncate d0/d8/da/f27 459218 0 2024-09-10T11:29:24.463 INFO:tasks.workunit.client.1.smithi178.stdout:4/172: truncate d0/f5 629745 0 2024-09-10T11:29:24.464 INFO:tasks.workunit.client.1.smithi178.stdout:4/173: write d0/d9/fb [1740996,35728] 0 2024-09-10T11:29:24.469 INFO:tasks.workunit.client.1.smithi178.stdout:4/174: write d0/d9/f13 [489322,63035] 0 2024-09-10T11:29:24.471 INFO:tasks.workunit.client.1.smithi178.stdout:4/175: symlink d0/d21/l3c 0 2024-09-10T11:29:24.472 INFO:tasks.workunit.client.1.smithi178.stdout:4/176: mkdir d0/d9/d3d 0 2024-09-10T11:29:24.474 INFO:tasks.workunit.client.1.smithi178.stdout:4/177: mkdir d0/d8/d3e 0 2024-09-10T11:29:24.475 INFO:tasks.workunit.client.1.smithi178.stdout:4/178: creat d0/d8/da/f3f x:0 0 0 2024-09-10T11:29:24.476 INFO:tasks.workunit.client.1.smithi178.stdout:4/179: rmdir d0/d8 39 2024-09-10T11:29:24.478 INFO:tasks.workunit.client.1.smithi178.stdout:4/180: getdents d0/d9/dc 0 2024-09-10T11:29:24.478 INFO:tasks.workunit.client.1.smithi178.stdout:4/181: read d0/f1c [2590816,80876] 0 2024-09-10T11:29:24.481 INFO:tasks.workunit.client.1.smithi178.stdout:4/182: dread d0/f1c [0,4194304] 0 2024-09-10T11:29:24.483 INFO:tasks.workunit.client.1.smithi178.stdout:4/183: dread d0/d9/f13 [0,4194304] 0 2024-09-10T11:29:24.484 INFO:tasks.workunit.client.1.smithi178.stdout:4/184: fsync d0/d10/f32 0 2024-09-10T11:29:24.487 INFO:tasks.workunit.client.1.smithi178.stdout:4/185: unlink d0/c6 0 2024-09-10T11:29:24.487 INFO:tasks.workunit.client.1.smithi178.stdout:4/186: chown d0/d21/l3c 1111192 1 2024-09-10T11:29:24.562 INFO:tasks.workunit.client.1.smithi178.stdout:3/191: sync 2024-09-10T11:29:24.562 INFO:tasks.workunit.client.1.smithi178.stdout:0/190: sync 2024-09-10T11:29:24.564 INFO:tasks.workunit.client.1.smithi178.stdout:0/191: creat d0/dc/d1f/d2d/f3e x:0 0 0 2024-09-10T11:29:24.567 INFO:tasks.workunit.client.1.smithi178.stdout:0/192: fdatasync d0/db/f38 0 2024-09-10T11:29:24.568 INFO:tasks.workunit.client.1.smithi178.stdout:3/192: dwrite db/d20/f34 [0,4194304] 0 2024-09-10T11:29:24.569 INFO:tasks.workunit.client.1.smithi178.stdout:3/193: readlink la 0 2024-09-10T11:29:24.569 INFO:tasks.workunit.client.1.smithi178.stdout:3/194: readlink db/d36/l38 0 2024-09-10T11:29:24.570 INFO:tasks.workunit.client.1.smithi178.stdout:0/193: mkdir d0/d12/d3f 0 2024-09-10T11:29:24.573 INFO:tasks.workunit.client.1.smithi178.stdout:0/194: rename d0/dc/d1f/d2d/c36 to d0/d19/c40 0 2024-09-10T11:29:24.574 INFO:tasks.workunit.client.1.smithi178.stdout:3/195: truncate db/ff 2596729 0 2024-09-10T11:29:24.575 INFO:tasks.workunit.client.1.smithi178.stdout:0/195: mkdir d0/d2c/d33/d41 0 2024-09-10T11:29:24.576 INFO:tasks.workunit.client.1.smithi178.stdout:0/196: write d0/d19/f25 [697907,127001] 0 2024-09-10T11:29:24.581 INFO:tasks.workunit.client.1.smithi178.stdout:0/197: fsync d0/d12/f2f 0 2024-09-10T11:29:24.582 INFO:tasks.workunit.client.1.smithi178.stdout:0/198: fdatasync d0/d12/f34 0 2024-09-10T11:29:24.583 INFO:tasks.workunit.client.1.smithi178.stdout:3/196: dwrite db/f10 [4194304,4194304] 0 2024-09-10T11:29:24.585 INFO:tasks.workunit.client.1.smithi178.stdout:3/197: write db/d20/f33 [761220,62953] 0 2024-09-10T11:29:24.586 INFO:tasks.workunit.client.1.smithi178.stdout:0/199: getdents d0/dc/d1f/d2d 0 2024-09-10T11:29:24.594 INFO:tasks.workunit.client.1.smithi178.stdout:0/200: dwrite d0/db/f32 [0,4194304] 0 2024-09-10T11:29:24.594 INFO:tasks.workunit.client.1.smithi178.stdout:3/198: dwrite db/f1a [0,4194304] 0 2024-09-10T11:29:24.595 INFO:tasks.workunit.client.1.smithi178.stdout:0/201: fdatasync d0/d12/f2f 0 2024-09-10T11:29:24.634 INFO:tasks.workunit.client.1.smithi178.stdout:2/408: sync 2024-09-10T11:29:24.634 INFO:tasks.workunit.client.1.smithi178.stdout:6/355: sync 2024-09-10T11:29:24.669 INFO:tasks.workunit.client.1.smithi178.stdout:3/199: mkdir db/d35/d3a 0 2024-09-10T11:29:24.671 INFO:tasks.workunit.client.1.smithi178.stdout:0/202: creat d0/dc/d1f/d2d/f42 x:0 0 0 2024-09-10T11:29:24.673 INFO:tasks.workunit.client.1.smithi178.stdout:6/356: creat d3/d7/d8/d15/d1c/d2a/d18/f7e x:0 0 0 2024-09-10T11:29:24.674 INFO:tasks.workunit.client.1.smithi178.stdout:3/200: rename db/d26/f2f to db/d26/f3b 0 2024-09-10T11:29:24.674 INFO:tasks.workunit.client.1.smithi178.stdout:2/409: mknod d1/d41/c85 0 2024-09-10T11:29:24.676 INFO:tasks.workunit.client.1.smithi178.stdout:0/203: mkdir d0/db/d43 0 2024-09-10T11:29:24.676 INFO:tasks.workunit.client.1.smithi178.stdout:7/168: sync 2024-09-10T11:29:24.676 INFO:tasks.workunit.client.1.smithi178.stdout:5/221: sync 2024-09-10T11:29:24.676 INFO:tasks.workunit.client.1.smithi178.stdout:9/382: sync 2024-09-10T11:29:24.676 INFO:tasks.workunit.client.1.smithi178.stdout:1/173: sync 2024-09-10T11:29:24.676 INFO:tasks.workunit.client.1.smithi178.stdout:4/187: sync 2024-09-10T11:29:24.676 INFO:tasks.workunit.client.1.smithi178.stdout:8/230: sync 2024-09-10T11:29:24.676 INFO:tasks.workunit.client.1.smithi178.stdout:0/204: dread - d0/d2c/f37 zero size 2024-09-10T11:29:24.677 INFO:tasks.workunit.client.1.smithi178.stdout:3/201: creat db/d20/f3c x:0 0 0 2024-09-10T11:29:24.678 INFO:tasks.workunit.client.1.smithi178.stdout:9/383: readlink d2/df/l64 0 2024-09-10T11:29:24.678 INFO:tasks.workunit.client.1.smithi178.stdout:4/188: dread - d0/d21/f37 zero size 2024-09-10T11:29:24.678 INFO:tasks.workunit.client.1.smithi178.stdout:2/410: creat d1/d7/d24/d43/f86 x:0 0 0 2024-09-10T11:29:24.678 INFO:tasks.workunit.client.1.smithi178.stdout:0/205: chown d0/d2c 2373 1 2024-09-10T11:29:24.678 INFO:tasks.workunit.client.1.smithi178.stdout:7/169: chown d6/fc 120 1 2024-09-10T11:29:24.679 INFO:tasks.workunit.client.1.smithi178.stdout:6/357: symlink d3/d7/d8/d15/d1c/l7f 0 2024-09-10T11:29:24.680 INFO:tasks.workunit.client.1.smithi178.stdout:8/231: chown d5/f7 2 1 2024-09-10T11:29:24.680 INFO:tasks.workunit.client.1.smithi178.stdout:1/174: truncate d3/de/f2e 4314829 0 2024-09-10T11:29:24.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:24 smithi144 ceph-mon[27942]: pgmap v19: 65 pgs: 65 active+clean; 541 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 4.4 MiB/s rd, 54 MiB/s wr, 402 op/s 2024-09-10T11:29:24.686 INFO:tasks.workunit.client.1.smithi178.stdout:9/384: mknod d2/c6f 0 2024-09-10T11:29:24.687 INFO:tasks.workunit.client.1.smithi178.stdout:3/202: dwrite db/f16 [0,4194304] 0 2024-09-10T11:29:24.687 INFO:tasks.workunit.client.1.smithi178.stdout:1/175: rmdir d3/d13 39 2024-09-10T11:29:24.688 INFO:tasks.workunit.client.1.smithi178.stdout:3/203: chown l9 561864 1 2024-09-10T11:29:24.688 INFO:tasks.workunit.client.1.smithi178.stdout:1/176: readlink d3/l4 0 2024-09-10T11:29:24.689 INFO:tasks.workunit.client.1.smithi178.stdout:4/189: dwrite d0/d8/da/f27 [0,4194304] 0 2024-09-10T11:29:24.689 INFO:tasks.workunit.client.1.smithi178.stdout:9/385: dread - d2/df/d31/d37/d3d/f6e zero size 2024-09-10T11:29:24.689 INFO:tasks.workunit.client.1.smithi178.stdout:8/232: mknod d5/d1d/d22/c4a 0 2024-09-10T11:29:24.691 INFO:tasks.workunit.client.1.smithi178.stdout:6/358: creat d3/d7/d8/d15/d1c/d20/f80 x:0 0 0 2024-09-10T11:29:24.695 INFO:tasks.workunit.client.1.smithi178.stdout:9/386: truncate d2/df/d31/f47 284619 0 2024-09-10T11:29:24.695 INFO:tasks.workunit.client.1.smithi178.stdout:5/222: creat d9/dc/dd/d37/d42/d55/f5b x:0 0 0 2024-09-10T11:29:24.695 INFO:tasks.workunit.client.1.smithi178.stdout:6/359: read d3/d7/d35/f5a [3295860,48112] 0 2024-09-10T11:29:24.697 INFO:tasks.workunit.client.1.smithi178.stdout:7/170: creat d6/d29/f3c x:0 0 0 2024-09-10T11:29:24.697 INFO:tasks.workunit.client.1.smithi178.stdout:7/171: dread - d6/d10/d1e/d2d/d2b/f39 zero size 2024-09-10T11:29:24.699 INFO:tasks.workunit.client.1.smithi178.stdout:5/223: dread d9/dc/d17/f44 [0,4194304] 0 2024-09-10T11:29:24.699 INFO:tasks.workunit.client.1.smithi178.stdout:0/206: link d0/dc/d1f/f26 d0/d12/d3f/f44 0 2024-09-10T11:29:24.700 INFO:tasks.workunit.client.1.smithi178.stdout:5/224: readlink d9/dc/l26 0 2024-09-10T11:29:24.700 INFO:tasks.workunit.client.1.smithi178.stdout:1/177: chown d3/d9/f25 305939 1 2024-09-10T11:29:24.701 INFO:tasks.workunit.client.1.smithi178.stdout:8/233: creat d5/d30/f4b x:0 0 0 2024-09-10T11:29:24.703 INFO:tasks.workunit.client.1.smithi178.stdout:2/411: link d1/d7/d24/d7d/d84/f2d d1/d7/d24/d43/f87 0 2024-09-10T11:29:24.704 INFO:tasks.workunit.client.1.smithi178.stdout:2/412: fsync d1/d41/f1f 0 2024-09-10T11:29:24.705 INFO:tasks.workunit.client.1.smithi178.stdout:6/360: creat d3/d24/f81 x:0 0 0 2024-09-10T11:29:24.706 INFO:tasks.workunit.client.1.smithi178.stdout:9/387: creat d2/df/d31/d37/f70 x:0 0 0 2024-09-10T11:29:24.707 INFO:tasks.workunit.client.1.smithi178.stdout:7/172: mknod d6/d10/c3d 0 2024-09-10T11:29:24.707 INFO:tasks.workunit.client.1.smithi178.stdout:9/388: write d2/d8/f1a [3956837,120168] 0 2024-09-10T11:29:24.708 INFO:tasks.workunit.client.1.smithi178.stdout:0/207: creat d0/d2c/d33/f45 x:0 0 0 2024-09-10T11:29:24.710 INFO:tasks.workunit.client.1.smithi178.stdout:7/173: write d6/d10/d1e/d2d/d2b/f39 [865170,41049] 0 2024-09-10T11:29:24.711 INFO:tasks.workunit.client.1.smithi178.stdout:9/389: dread - d2/d19/d1c/d2d/d54/f5d zero size 2024-09-10T11:29:24.711 INFO:tasks.workunit.client.1.smithi178.stdout:0/208: dread - d0/dc/f2b zero size 2024-09-10T11:29:24.711 INFO:tasks.workunit.client.1.smithi178.stdout:1/178: rmdir d3 39 2024-09-10T11:29:24.713 INFO:tasks.workunit.client.1.smithi178.stdout:5/225: symlink d9/dc/dd/d37/d42/d55/l5c 0 2024-09-10T11:29:24.714 INFO:tasks.workunit.client.1.smithi178.stdout:6/361: dwrite d3/d7/d8/d15/d1c/d2a/f41 [0,4194304] 0 2024-09-10T11:29:24.715 INFO:tasks.workunit.client.1.smithi178.stdout:6/362: write d3/d24/f81 [31438,45102] 0 2024-09-10T11:29:24.719 INFO:tasks.workunit.client.1.smithi178.stdout:2/413: mkdir d1/d7/d24/d46/d88 0 2024-09-10T11:29:24.720 INFO:tasks.workunit.client.1.smithi178.stdout:2/414: write d1/d41/f51 [1804757,19669] 0 2024-09-10T11:29:24.721 INFO:tasks.workunit.client.1.smithi178.stdout:2/415: chown d1/d7/d24/f5a 330031261 1 2024-09-10T11:29:24.723 INFO:tasks.workunit.client.1.smithi178.stdout:4/190: creat d0/f40 x:0 0 0 2024-09-10T11:29:24.724 INFO:tasks.workunit.client.1.smithi178.stdout:4/191: read d0/d9/fb [2478085,76089] 0 2024-09-10T11:29:24.726 INFO:tasks.workunit.client.1.smithi178.stdout:9/390: stat d2/d19/d1c/l30 0 2024-09-10T11:29:24.727 INFO:tasks.workunit.client.1.smithi178.stdout:9/391: dread - d2/d19/d1c/d2d/d54/f5d zero size 2024-09-10T11:29:24.730 INFO:tasks.workunit.client.1.smithi178.stdout:8/234: mknod d5/d30/d33/d3a/c4c 0 2024-09-10T11:29:24.734 INFO:tasks.workunit.client.1.smithi178.stdout:3/204: truncate db/f15 839519 0 2024-09-10T11:29:24.736 INFO:tasks.workunit.client.1.smithi178.stdout:1/179: stat d3/d13/f15 0 2024-09-10T11:29:24.737 INFO:tasks.workunit.client.1.smithi178.stdout:9/392: mknod d2/df/d31/d37/c71 0 2024-09-10T11:29:24.738 INFO:tasks.workunit.client.1.smithi178.stdout:9/393: read d2/df/d31/f47 [190341,50218] 0 2024-09-10T11:29:24.738 INFO:tasks.workunit.client.1.smithi178.stdout:9/394: fdatasync d2/f3f 0 2024-09-10T11:29:24.740 INFO:tasks.workunit.client.1.smithi178.stdout:8/235: write d5/f24 [4159030,24248] 0 2024-09-10T11:29:24.741 INFO:tasks.workunit.client.1.smithi178.stdout:8/236: read d5/db/d10/d20/f26 [378505,115537] 0 2024-09-10T11:29:24.742 INFO:tasks.workunit.client.1.smithi178.stdout:3/205: mkdir db/d20/d3d 0 2024-09-10T11:29:24.743 INFO:tasks.workunit.client.1.smithi178.stdout:2/416: write d1/d41/d14/d11/f4a [1454037,18705] 0 2024-09-10T11:29:24.748 INFO:tasks.workunit.client.1.smithi178.stdout:1/180: mkdir d3/d26/d3a 0 2024-09-10T11:29:24.749 INFO:tasks.workunit.client.1.smithi178.stdout:0/209: dwrite d0/dc/d1f/f26 [0,4194304] 0 2024-09-10T11:29:24.749 INFO:tasks.workunit.client.1.smithi178.stdout:0/210: dread - d0/d12/f2f zero size 2024-09-10T11:29:24.749 INFO:tasks.workunit.client.1.smithi178.stdout:7/174: rename d6/d10/d1d/d22/l32 to d6/l3e 0 2024-09-10T11:29:24.749 INFO:tasks.workunit.client.1.smithi178.stdout:6/363: truncate d3/d7/d8/d15/d1c/d2a/d18/f43 428829 0 2024-09-10T11:29:24.750 INFO:tasks.workunit.client.1.smithi178.stdout:7/175: read d6/d10/d1e/f35 [64810,129816] 0 2024-09-10T11:29:24.750 INFO:tasks.workunit.client.1.smithi178.stdout:6/364: chown d3/d7/d35/f63 101 1 2024-09-10T11:29:24.753 INFO:tasks.workunit.client.1.smithi178.stdout:4/192: truncate d0/d9/f2d 208264 0 2024-09-10T11:29:24.758 INFO:tasks.workunit.client.1.smithi178.stdout:2/417: unlink d1/d7/d24/d7d/d84/d34/d5b/d60/f68 0 2024-09-10T11:29:24.760 INFO:tasks.workunit.client.1.smithi178.stdout:0/211: write d0/f9 [1093222,101291] 0 2024-09-10T11:29:24.762 INFO:tasks.workunit.client.1.smithi178.stdout:5/226: getdents d9/d11 0 2024-09-10T11:29:24.763 INFO:tasks.workunit.client.1.smithi178.stdout:5/227: readlink d9/d11/l36 0 2024-09-10T11:29:24.763 INFO:tasks.workunit.client.1.smithi178.stdout:6/365: chown d3/d7/d8/l23 3335 1 2024-09-10T11:29:24.764 INFO:tasks.workunit.client.1.smithi178.stdout:6/366: readlink d3/d7/d8/d15/l31 0 2024-09-10T11:29:24.766 INFO:tasks.workunit.client.1.smithi178.stdout:7/176: rmdir d6/d10/d1e/d2d 39 2024-09-10T11:29:24.770 INFO:tasks.workunit.client.1.smithi178.stdout:3/206: unlink db/l13 0 2024-09-10T11:29:24.771 INFO:tasks.workunit.client.1.smithi178.stdout:6/367: dwrite d3/d7/d8/d15/d1c/d2a/d18/f7e [0,4194304] 0 2024-09-10T11:29:24.771 INFO:tasks.workunit.client.1.smithi178.stdout:3/207: dread - db/d20/f3c zero size 2024-09-10T11:29:24.773 INFO:tasks.workunit.client.1.smithi178.stdout:2/418: write d1/d41/d5d/f71 [3724919,97571] 0 2024-09-10T11:29:24.773 INFO:tasks.workunit.client.1.smithi178.stdout:9/395: truncate d2/f3f 138032 0 2024-09-10T11:29:24.777 INFO:tasks.workunit.client.1.smithi178.stdout:2/419: fsync d1/d41/f1f 0 2024-09-10T11:29:24.778 INFO:tasks.workunit.client.1.smithi178.stdout:0/212: symlink d0/db/d43/l46 0 2024-09-10T11:29:24.778 INFO:tasks.workunit.client.1.smithi178.stdout:7/177: truncate d6/d10/d1e/d2d/f2c 72841 0 2024-09-10T11:29:24.779 INFO:tasks.workunit.client.1.smithi178.stdout:2/420: write d1/f2e [4490845,58672] 0 2024-09-10T11:29:24.779 INFO:tasks.workunit.client.1.smithi178.stdout:3/208: mkdir db/d20/d3e 0 2024-09-10T11:29:24.781 INFO:tasks.workunit.client.1.smithi178.stdout:5/228: getdents d9/d11 0 2024-09-10T11:29:24.781 INFO:tasks.workunit.client.1.smithi178.stdout:6/368: fdatasync d3/d7/d8/d15/d1c/d2a/f3c 0 2024-09-10T11:29:24.781 INFO:tasks.workunit.client.1.smithi178.stdout:9/396: rmdir d2/d51 39 2024-09-10T11:29:24.781 INFO:tasks.workunit.client.1.smithi178.stdout:4/193: link d0/d8/df/l3a d0/d8/da/d1a/l41 0 2024-09-10T11:29:24.782 INFO:tasks.workunit.client.1.smithi178.stdout:8/237: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:24.786 INFO:tasks.workunit.client.1.smithi178.stdout:0/213: mknod d0/dc/d1f/d2d/c47 0 2024-09-10T11:29:24.786 INFO:tasks.workunit.client.1.smithi178.stdout:5/229: write d9/dc/dd/d37/d42/f4e [1633989,81699] 0 2024-09-10T11:29:24.788 INFO:tasks.workunit.client.1.smithi178.stdout:2/421: readlink d1/d41/d14/l7b 0 2024-09-10T11:29:24.788 INFO:tasks.workunit.client.1.smithi178.stdout:4/194: unlink d0/d8/da/d1a/f30 0 2024-09-10T11:29:24.789 INFO:tasks.workunit.client.1.smithi178.stdout:0/214: chown d0/dc/d1f/d2d/c31 67962789 1 2024-09-10T11:29:24.790 INFO:tasks.workunit.client.1.smithi178.stdout:2/422: chown d1/d41/f44 648887 1 2024-09-10T11:29:24.791 INFO:tasks.workunit.client.1.smithi178.stdout:7/178: dwrite d6/fb [0,4194304] 0 2024-09-10T11:29:24.796 INFO:tasks.workunit.client.1.smithi178.stdout:9/397: dwrite d2/d19/d1c/f22 [0,4194304] 0 2024-09-10T11:29:24.796 INFO:tasks.workunit.client.1.smithi178.stdout:6/369: symlink d3/d7/d8/d72/l82 0 2024-09-10T11:29:24.796 INFO:tasks.workunit.client.1.smithi178.stdout:8/238: chown d5/f1f 496816 1 2024-09-10T11:29:24.799 INFO:tasks.workunit.client.1.smithi178.stdout:0/215: mknod d0/d2c/d33/c48 0 2024-09-10T11:29:24.802 INFO:tasks.workunit.client.1.smithi178.stdout:2/423: mkdir d1/d7/d24/d7d/d84/d4b/d89 0 2024-09-10T11:29:24.805 INFO:tasks.workunit.client.1.smithi178.stdout:9/398: rmdir d2/df/d31 39 2024-09-10T11:29:24.805 INFO:tasks.workunit.client.1.smithi178.stdout:7/179: dwrite d6/fb [4194304,4194304] 0 2024-09-10T11:29:24.806 INFO:tasks.workunit.client.1.smithi178.stdout:0/216: symlink d0/db/l49 0 2024-09-10T11:29:24.807 INFO:tasks.workunit.client.1.smithi178.stdout:4/195: chown d0/d9/dc/f34 13285242 1 2024-09-10T11:29:24.807 INFO:tasks.workunit.client.1.smithi178.stdout:9/399: readlink d2/l53 0 2024-09-10T11:29:24.807 INFO:tasks.workunit.client.1.smithi178.stdout:3/209: link db/l12 db/d35/d3a/l3f 0 2024-09-10T11:29:24.808 INFO:tasks.workunit.client.1.smithi178.stdout:0/217: fdatasync d0/d12/f34 0 2024-09-10T11:29:24.809 INFO:tasks.workunit.client.1.smithi178.stdout:2/424: symlink d1/d7/l8a 0 2024-09-10T11:29:24.811 INFO:tasks.workunit.client.1.smithi178.stdout:3/210: write db/f39 [1029206,14770] 0 2024-09-10T11:29:24.812 INFO:tasks.workunit.client.1.smithi178.stdout:5/230: dread d9/ff [0,4194304] 0 2024-09-10T11:29:24.812 INFO:tasks.workunit.client.1.smithi178.stdout:0/218: dread - d0/d2c/d33/f45 zero size 2024-09-10T11:29:24.812 INFO:tasks.workunit.client.1.smithi178.stdout:6/370: mknod d3/d42/d57/d78/d7c/c83 0 2024-09-10T11:29:24.814 INFO:tasks.workunit.client.1.smithi178.stdout:9/400: creat d2/d13/d40/d4b/f72 x:0 0 0 2024-09-10T11:29:24.814 INFO:tasks.workunit.client.1.smithi178.stdout:7/180: write d6/d10/d1e/f35 [679086,124614] 0 2024-09-10T11:29:24.815 INFO:tasks.workunit.client.1.smithi178.stdout:3/211: read db/f10 [362485,125497] 0 2024-09-10T11:29:24.816 INFO:tasks.workunit.client.1.smithi178.stdout:7/181: dread - d6/d29/f3c zero size 2024-09-10T11:29:24.817 INFO:tasks.workunit.client.1.smithi178.stdout:0/219: read d0/d19/f25 [596155,95623] 0 2024-09-10T11:29:24.817 INFO:tasks.workunit.client.1.smithi178.stdout:8/239: dread d5/db/d10/d20/f2b [0,4194304] 0 2024-09-10T11:29:24.817 INFO:tasks.workunit.client.1.smithi178.stdout:6/371: creat d3/d42/f84 x:0 0 0 2024-09-10T11:29:24.818 INFO:tasks.workunit.client.1.smithi178.stdout:2/425: creat d1/d41/d14/d11/d53/f8b x:0 0 0 2024-09-10T11:29:24.819 INFO:tasks.workunit.client.1.smithi178.stdout:4/196: creat d0/f42 x:0 0 0 2024-09-10T11:29:24.820 INFO:tasks.workunit.client.1.smithi178.stdout:3/212: stat db/ff 0 2024-09-10T11:29:24.820 INFO:tasks.workunit.client.1.smithi178.stdout:9/401: creat d2/d8/f73 x:0 0 0 2024-09-10T11:29:24.821 INFO:tasks.workunit.client.1.smithi178.stdout:0/220: mknod d0/db/c4a 0 2024-09-10T11:29:24.821 INFO:tasks.workunit.client.1.smithi178.stdout:8/240: symlink d5/d30/d33/l4d 0 2024-09-10T11:29:24.823 INFO:tasks.workunit.client.1.smithi178.stdout:2/426: chown d1/d41/d14/l21 102 1 2024-09-10T11:29:24.824 INFO:tasks.workunit.client.1.smithi178.stdout:4/197: symlink d0/d8/df/l43 0 2024-09-10T11:29:24.824 INFO:tasks.workunit.client.1.smithi178.stdout:5/231: dwrite d9/dc/d17/f3d [0,4194304] 0 2024-09-10T11:29:24.824 INFO:tasks.workunit.client.1.smithi178.stdout:3/213: mkdir db/d26/d40 0 2024-09-10T11:29:24.826 INFO:tasks.workunit.client.1.smithi178.stdout:4/198: read - d0/d10/f32 zero size 2024-09-10T11:29:24.827 INFO:tasks.workunit.client.1.smithi178.stdout:3/214: write db/d20/f34 [3039408,53243] 0 2024-09-10T11:29:24.827 INFO:tasks.workunit.client.1.smithi178.stdout:9/402: chown d2/df/d31/d37/f70 56702234 1 2024-09-10T11:29:24.830 INFO:tasks.workunit.client.1.smithi178.stdout:8/241: write d5/fa [1047428,63378] 0 2024-09-10T11:29:24.831 INFO:tasks.workunit.client.1.smithi178.stdout:2/427: rename d1/d7/c45 to d1/d7/d24/d7d/d84/c8c 0 2024-09-10T11:29:24.834 INFO:tasks.workunit.client.1.smithi178.stdout:8/242: creat d5/d1d/d22/f4e x:0 0 0 2024-09-10T11:29:24.835 INFO:tasks.workunit.client.1.smithi178.stdout:0/221: dread d0/dc/ff [0,4194304] 0 2024-09-10T11:29:24.836 INFO:tasks.workunit.client.1.smithi178.stdout:8/243: dread d5/f1f [0,4194304] 0 2024-09-10T11:29:24.836 INFO:tasks.workunit.client.1.smithi178.stdout:3/215: dread f1 [0,4194304] 0 2024-09-10T11:29:24.838 INFO:tasks.workunit.client.1.smithi178.stdout:7/182: truncate d6/d10/d1e/d2d/d2b/f39 207020 0 2024-09-10T11:29:24.839 INFO:tasks.workunit.client.1.smithi178.stdout:0/222: mknod d0/d12/c4b 0 2024-09-10T11:29:24.840 INFO:tasks.workunit.client.1.smithi178.stdout:8/244: creat d5/d30/f4f x:0 0 0 2024-09-10T11:29:24.841 INFO:tasks.workunit.client.1.smithi178.stdout:5/232: link d9/dc/dd/d16/l2e d9/dc/d17/d1d/l5d 0 2024-09-10T11:29:24.842 INFO:tasks.workunit.client.1.smithi178.stdout:8/245: truncate d5/d30/f4b 204440 0 2024-09-10T11:29:24.842 INFO:tasks.workunit.client.1.smithi178.stdout:2/428: truncate d1/f13 1587959 0 2024-09-10T11:29:24.843 INFO:tasks.workunit.client.1.smithi178.stdout:3/216: symlink db/d20/d3d/l41 0 2024-09-10T11:29:24.844 INFO:tasks.workunit.client.1.smithi178.stdout:7/183: creat d6/d10/d34/f3f x:0 0 0 2024-09-10T11:29:24.845 INFO:tasks.workunit.client.1.smithi178.stdout:8/246: creat d5/d1d/f50 x:0 0 0 2024-09-10T11:29:24.845 INFO:tasks.workunit.client.1.smithi178.stdout:4/199: dwrite d0/d9/fb [4194304,4194304] 0 2024-09-10T11:29:24.848 INFO:tasks.workunit.client.1.smithi178.stdout:5/233: creat d9/f5e x:0 0 0 2024-09-10T11:29:24.849 INFO:tasks.workunit.client.1.smithi178.stdout:2/429: mkdir d1/d41/d8d 0 2024-09-10T11:29:24.850 INFO:tasks.workunit.client.1.smithi178.stdout:7/184: symlink d6/d10/d34/l40 0 2024-09-10T11:29:24.851 INFO:tasks.workunit.client.1.smithi178.stdout:3/217: fsync db/fe 0 2024-09-10T11:29:24.851 INFO:tasks.workunit.client.1.smithi178.stdout:2/430: write d1/d41/d14/f82 [868612,130316] 0 2024-09-10T11:29:24.852 INFO:tasks.workunit.client.1.smithi178.stdout:0/223: getdents d0/d19 0 2024-09-10T11:29:24.853 INFO:tasks.workunit.client.1.smithi178.stdout:7/185: truncate d6/d10/d1d/f27 1056172 0 2024-09-10T11:29:24.854 INFO:tasks.workunit.client.1.smithi178.stdout:7/186: fdatasync d6/ff 0 2024-09-10T11:29:24.854 INFO:tasks.workunit.client.1.smithi178.stdout:8/247: symlink d5/db/d1e/l51 0 2024-09-10T11:29:24.856 INFO:tasks.workunit.client.1.smithi178.stdout:8/248: chown d5/db/d1e/f39 15487 1 2024-09-10T11:29:24.857 INFO:tasks.workunit.client.1.smithi178.stdout:5/234: rename d9/f30 to d9/dc/dd/d37/d42/f5f 0 2024-09-10T11:29:24.859 INFO:tasks.workunit.client.1.smithi178.stdout:8/249: fsync d5/db/d1e/f45 0 2024-09-10T11:29:24.860 INFO:tasks.workunit.client.1.smithi178.stdout:8/250: read - d5/d30/f4f zero size 2024-09-10T11:29:24.860 INFO:tasks.workunit.client.1.smithi178.stdout:3/218: dwrite db/f1a [0,4194304] 0 2024-09-10T11:29:24.861 INFO:tasks.workunit.client.1.smithi178.stdout:2/431: mkdir d1/d7/d8e 0 2024-09-10T11:29:24.864 INFO:tasks.workunit.client.1.smithi178.stdout:7/187: creat d6/d10/d1e/d2d/d2b/f41 x:0 0 0 2024-09-10T11:29:24.866 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:24 smithi178 ceph-mon[33514]: pgmap v19: 65 pgs: 65 active+clean; 541 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 4.4 MiB/s rd, 54 MiB/s wr, 402 op/s 2024-09-10T11:29:24.867 INFO:tasks.workunit.client.1.smithi178.stdout:0/224: rename d0/db to d0/dc/d39/d4c 0 2024-09-10T11:29:24.868 INFO:tasks.workunit.client.1.smithi178.stdout:5/235: fdatasync d9/dc/dd/d16/f1c 0 2024-09-10T11:29:24.870 INFO:tasks.workunit.client.1.smithi178.stdout:2/432: dwrite d1/d7/d24/d7d/d84/d4b/f57 [0,4194304] 0 2024-09-10T11:29:24.870 INFO:tasks.workunit.client.1.smithi178.stdout:8/251: chown d5/ff 19193660 1 2024-09-10T11:29:24.872 INFO:tasks.workunit.client.1.smithi178.stdout:7/188: creat d6/d29/f42 x:0 0 0 2024-09-10T11:29:24.873 INFO:tasks.workunit.client.1.smithi178.stdout:3/219: mkdir db/d35/d3a/d42 0 2024-09-10T11:29:24.876 INFO:tasks.workunit.client.1.smithi178.stdout:8/252: mkdir d5/db/d10/d20/d52 0 2024-09-10T11:29:24.877 INFO:tasks.workunit.client.1.smithi178.stdout:2/433: mknod d1/d7/d24/d7d/d84/d34/d5b/d60/c8f 0 2024-09-10T11:29:24.879 INFO:tasks.workunit.client.1.smithi178.stdout:5/236: dwrite d9/f51 [0,4194304] 0 2024-09-10T11:29:24.884 INFO:tasks.workunit.client.1.smithi178.stdout:5/237: fdatasync d9/dc/f2b 0 2024-09-10T11:29:24.887 INFO:tasks.workunit.client.1.smithi178.stdout:8/253: mkdir d5/db/d10/d53 0 2024-09-10T11:29:24.888 INFO:tasks.workunit.client.1.smithi178.stdout:0/225: truncate d0/f22 942982 0 2024-09-10T11:29:24.888 INFO:tasks.workunit.client.1.smithi178.stdout:5/238: write d9/dc/dd/f2d [917906,101306] 0 2024-09-10T11:29:24.890 INFO:tasks.workunit.client.1.smithi178.stdout:8/254: symlink d5/d30/d33/l54 0 2024-09-10T11:29:24.892 INFO:tasks.workunit.client.1.smithi178.stdout:7/189: truncate d6/fd 3154138 0 2024-09-10T11:29:24.893 INFO:tasks.workunit.client.1.smithi178.stdout:5/239: chown d9/dc/dd/ce 3259 1 2024-09-10T11:29:24.894 INFO:tasks.workunit.client.1.smithi178.stdout:0/226: creat d0/d2c/f4d x:0 0 0 2024-09-10T11:29:24.895 INFO:tasks.workunit.client.1.smithi178.stdout:2/434: dwrite d1/d7/d24/d7d/d84/d55/f73 [4194304,4194304] 0 2024-09-10T11:29:24.895 INFO:tasks.workunit.client.1.smithi178.stdout:1/181: sync 2024-09-10T11:29:24.902 INFO:tasks.workunit.client.1.smithi178.stdout:3/220: dwrite db/d20/f28 [0,4194304] 0 2024-09-10T11:29:24.908 INFO:tasks.workunit.client.1.smithi178.stdout:7/190: rename d6/d10/d1d/f25 to d6/d10/d1d/d22/d3b/f43 0 2024-09-10T11:29:24.912 INFO:tasks.workunit.client.1.smithi178.stdout:5/240: dread d9/dc/dd/d16/f1c [0,4194304] 0 2024-09-10T11:29:24.916 INFO:tasks.workunit.client.1.smithi178.stdout:5/241: mknod d9/d11/c60 0 2024-09-10T11:29:24.916 INFO:tasks.workunit.client.1.smithi178.stdout:3/221: link db/l24 db/d35/d3a/l43 0 2024-09-10T11:29:24.918 INFO:tasks.workunit.client.1.smithi178.stdout:1/182: dwrite d3/d9/f10 [0,4194304] 0 2024-09-10T11:29:24.919 INFO:tasks.workunit.client.1.smithi178.stdout:7/191: truncate d6/d10/d1e/d2d/f21 4536274 0 2024-09-10T11:29:24.920 INFO:tasks.workunit.client.1.smithi178.stdout:1/183: readlink d3/l4 0 2024-09-10T11:29:24.922 INFO:tasks.workunit.client.1.smithi178.stdout:8/255: dwrite d5/db/d10/d20/f28 [0,4194304] 0 2024-09-10T11:29:24.924 INFO:tasks.workunit.client.1.smithi178.stdout:1/184: chown d3/d26/d2a/f34 2060007 1 2024-09-10T11:29:24.925 INFO:tasks.workunit.client.1.smithi178.stdout:0/227: dread d0/dc/f13 [0,4194304] 0 2024-09-10T11:29:24.925 INFO:tasks.workunit.client.1.smithi178.stdout:5/242: dread d9/dc/f15 [0,4194304] 0 2024-09-10T11:29:24.926 INFO:tasks.workunit.client.1.smithi178.stdout:5/243: stat d9/dc/d17/d1d 0 2024-09-10T11:29:24.926 INFO:tasks.workunit.client.1.smithi178.stdout:0/228: write d0/dc/f21 [3316640,34270] 0 2024-09-10T11:29:24.927 INFO:tasks.workunit.client.1.smithi178.stdout:2/435: dread d1/f2e [0,4194304] 0 2024-09-10T11:29:24.927 INFO:tasks.workunit.client.1.smithi178.stdout:3/222: rename db/l2b to db/d35/d3a/l44 0 2024-09-10T11:29:24.928 INFO:tasks.workunit.client.1.smithi178.stdout:3/223: fsync db/d26/f2c 0 2024-09-10T11:29:24.930 INFO:tasks.workunit.client.1.smithi178.stdout:0/229: dread d0/d19/f1d [0,4194304] 0 2024-09-10T11:29:24.933 INFO:tasks.workunit.client.1.smithi178.stdout:1/185: dread d3/d9/f14 [0,4194304] 0 2024-09-10T11:29:24.934 INFO:tasks.workunit.client.1.smithi178.stdout:5/244: chown l3 1403084 1 2024-09-10T11:29:24.934 INFO:tasks.workunit.client.1.smithi178.stdout:7/192: dread d6/d10/d1d/d22/d3b/f43 [0,4194304] 0 2024-09-10T11:29:24.935 INFO:tasks.workunit.client.1.smithi178.stdout:5/245: chown d9/dc/l40 1620328 1 2024-09-10T11:29:24.936 INFO:tasks.workunit.client.1.smithi178.stdout:5/246: dread - d9/dc/d17/d1d/f54 zero size 2024-09-10T11:29:24.937 INFO:tasks.workunit.client.1.smithi178.stdout:2/436: symlink d1/d7/d24/d7d/d84/d55/l90 0 2024-09-10T11:29:24.938 INFO:tasks.workunit.client.1.smithi178.stdout:3/224: unlink db/d26/l2a 0 2024-09-10T11:29:24.941 INFO:tasks.workunit.client.1.smithi178.stdout:3/225: truncate db/d26/f32 182820 0 2024-09-10T11:29:24.942 INFO:tasks.workunit.client.1.smithi178.stdout:8/256: creat d5/db/d10/f55 x:0 0 0 2024-09-10T11:29:24.942 INFO:tasks.workunit.client.1.smithi178.stdout:2/437: rmdir d1/d22 39 2024-09-10T11:29:24.943 INFO:tasks.workunit.client.1.smithi178.stdout:7/193: dwrite d6/d10/d1e/d2d/f21 [4194304,4194304] 0 2024-09-10T11:29:24.944 INFO:tasks.workunit.client.1.smithi178.stdout:2/438: fdatasync d1/d7/f16 0 2024-09-10T11:29:24.945 INFO:tasks.workunit.client.1.smithi178.stdout:0/230: write d0/d19/f25 [1200041,42722] 0 2024-09-10T11:29:24.946 INFO:tasks.workunit.client.1.smithi178.stdout:1/186: link d3/f22 d3/d26/f3b 0 2024-09-10T11:29:24.947 INFO:tasks.workunit.client.1.smithi178.stdout:8/257: symlink d5/d30/l56 0 2024-09-10T11:29:24.949 INFO:tasks.workunit.client.1.smithi178.stdout:2/439: symlink d1/d7/d24/l91 0 2024-09-10T11:29:24.953 INFO:tasks.workunit.client.1.smithi178.stdout:7/194: creat d6/d10/d34/f44 x:0 0 0 2024-09-10T11:29:24.953 INFO:tasks.workunit.client.1.smithi178.stdout:0/231: readlink d0/l29 0 2024-09-10T11:29:24.953 INFO:tasks.workunit.client.1.smithi178.stdout:8/258: mknod d5/d30/c57 0 2024-09-10T11:29:24.953 INFO:tasks.workunit.client.1.smithi178.stdout:3/226: write db/ff [1554760,86362] 0 2024-09-10T11:29:24.954 INFO:tasks.workunit.client.1.smithi178.stdout:0/232: write d0/f8 [1494842,27524] 0 2024-09-10T11:29:24.958 INFO:tasks.workunit.client.1.smithi178.stdout:5/247: dwrite d9/dc/f15 [0,4194304] 0 2024-09-10T11:29:24.958 INFO:tasks.workunit.client.1.smithi178.stdout:0/233: write d0/d2c/f4d [471727,56768] 0 2024-09-10T11:29:24.961 INFO:tasks.workunit.client.1.smithi178.stdout:7/195: creat d6/d10/d1e/f45 x:0 0 0 2024-09-10T11:29:24.964 INFO:tasks.workunit.client.1.smithi178.stdout:5/248: mknod d9/d11/c61 0 2024-09-10T11:29:24.965 INFO:tasks.workunit.client.1.smithi178.stdout:2/440: link d1/d7/d24/d43/f87 d1/d7/d24/d7d/d84/d55/d6e/f92 0 2024-09-10T11:29:24.967 INFO:tasks.workunit.client.1.smithi178.stdout:3/227: rename db/f10 to db/f45 0 2024-09-10T11:29:24.968 INFO:tasks.workunit.client.1.smithi178.stdout:5/249: mkdir d9/dc/dd/d37/d42/d62 0 2024-09-10T11:29:24.968 INFO:tasks.workunit.client.1.smithi178.stdout:0/234: dread d0/f8 [0,4194304] 0 2024-09-10T11:29:24.969 INFO:tasks.workunit.client.1.smithi178.stdout:1/187: dread d3/d9/f25 [0,4194304] 0 2024-09-10T11:29:24.970 INFO:tasks.workunit.client.1.smithi178.stdout:2/441: unlink d1/d41/d14/d11/d53/f62 0 2024-09-10T11:29:24.970 INFO:tasks.workunit.client.1.smithi178.stdout:7/196: getdents d6/d29 0 2024-09-10T11:29:24.970 INFO:tasks.workunit.client.1.smithi178.stdout:0/235: read d0/dc/f21 [1068698,15938] 0 2024-09-10T11:29:24.971 INFO:tasks.workunit.client.1.smithi178.stdout:1/188: chown d3/d26/d2a 1732807 1 2024-09-10T11:29:24.972 INFO:tasks.workunit.client.1.smithi178.stdout:3/228: chown db/d35/d3a/l43 1 1 2024-09-10T11:29:24.972 INFO:tasks.workunit.client.1.smithi178.stdout:7/197: chown d6/d10/d1e/d2d/f16 1929 1 2024-09-10T11:29:24.972 INFO:tasks.workunit.client.1.smithi178.stdout:3/229: chown db/d20/l23 211 1 2024-09-10T11:29:24.973 INFO:tasks.workunit.client.1.smithi178.stdout:0/236: truncate d0/dc/d1f/d2d/f3e 746845 0 2024-09-10T11:29:24.985 INFO:tasks.workunit.client.1.smithi178.stdout:3/230: dread db/d20/f21 [0,4194304] 0 2024-09-10T11:29:24.989 INFO:tasks.workunit.client.1.smithi178.stdout:3/231: dwrite db/d26/f2c [0,4194304] 0 2024-09-10T11:29:25.008 INFO:tasks.workunit.client.1.smithi178.stdout:5/250: mknod d9/dc/dd/d37/c63 0 2024-09-10T11:29:25.010 INFO:tasks.workunit.client.1.smithi178.stdout:2/442: unlink d1/d41/f31 0 2024-09-10T11:29:25.011 INFO:tasks.workunit.client.1.smithi178.stdout:7/198: symlink d6/d10/d1e/d24/l46 0 2024-09-10T11:29:25.012 INFO:tasks.workunit.client.1.smithi178.stdout:7/199: write d6/d10/d1d/f27 [773806,108854] 0 2024-09-10T11:29:25.014 INFO:tasks.workunit.client.1.smithi178.stdout:0/237: unlink d0/d2c/f4d 0 2024-09-10T11:29:25.015 INFO:tasks.workunit.client.1.smithi178.stdout:5/251: mknod d9/dc/dd/d37/c64 0 2024-09-10T11:29:25.017 INFO:tasks.workunit.client.1.smithi178.stdout:7/200: dread d6/f30 [0,4194304] 0 2024-09-10T11:29:25.021 INFO:tasks.workunit.client.1.smithi178.stdout:3/232: mkdir db/d35/d3a/d42/d46 0 2024-09-10T11:29:25.021 INFO:tasks.workunit.client.1.smithi178.stdout:3/233: read db/f1a [756993,76090] 0 2024-09-10T11:29:25.022 INFO:tasks.workunit.client.1.smithi178.stdout:3/234: chown db/f15 658 1 2024-09-10T11:29:25.022 INFO:tasks.workunit.client.1.smithi178.stdout:0/238: rename d0/d2c/d33/f3b to d0/f4e 0 2024-09-10T11:29:25.023 INFO:tasks.workunit.client.1.smithi178.stdout:1/189: creat d3/d9/f3c x:0 0 0 2024-09-10T11:29:25.024 INFO:tasks.workunit.client.1.smithi178.stdout:2/443: creat d1/d7/d24/d7d/d84/d4b/d89/f93 x:0 0 0 2024-09-10T11:29:25.030 INFO:tasks.workunit.client.1.smithi178.stdout:3/235: dwrite db/d20/f33 [0,4194304] 0 2024-09-10T11:29:25.031 INFO:tasks.workunit.client.1.smithi178.stdout:7/201: write d6/d10/d1d/d22/d3b/f43 [381370,23918] 0 2024-09-10T11:29:25.032 INFO:tasks.workunit.client.1.smithi178.stdout:5/252: link d9/dc/d17/c3a d9/dc/dd/d37/d3e/c65 0 2024-09-10T11:29:25.034 INFO:tasks.workunit.client.1.smithi178.stdout:1/190: link d3/d9/f3c d3/d13/d2b/f3d 0 2024-09-10T11:29:25.035 INFO:tasks.workunit.client.1.smithi178.stdout:5/253: chown d9/dc/f47 238 1 2024-09-10T11:29:25.037 INFO:tasks.workunit.client.1.smithi178.stdout:0/239: dwrite d0/dc/f13 [0,4194304] 0 2024-09-10T11:29:25.037 INFO:tasks.workunit.client.1.smithi178.stdout:3/236: unlink db/l2e 0 2024-09-10T11:29:25.039 INFO:tasks.workunit.client.1.smithi178.stdout:0/240: dread - d0/d2c/f37 zero size 2024-09-10T11:29:25.039 INFO:tasks.workunit.client.1.smithi178.stdout:7/202: rename d6/l18 to d6/d10/d1e/d24/l47 0 2024-09-10T11:29:25.040 INFO:tasks.workunit.client.1.smithi178.stdout:2/444: getdents d1/d7/d24/d7d/d84/d34/d5e 0 2024-09-10T11:29:25.043 INFO:tasks.workunit.client.1.smithi178.stdout:2/445: write d1/d7/d24/d7d/d84/d55/f73 [1648243,39403] 0 2024-09-10T11:29:25.046 INFO:tasks.workunit.client.1.smithi178.stdout:2/446: chown d1/d7/d24/d7d/d84/d34/d5b/d60/f64 609332517 1 2024-09-10T11:29:25.047 INFO:tasks.workunit.client.1.smithi178.stdout:1/191: rename d3/d13/d2b/c30 to d3/d26/d3a/c3e 0 2024-09-10T11:29:25.048 INFO:tasks.workunit.client.1.smithi178.stdout:0/241: link d0/l29 d0/d12/d3f/l4f 0 2024-09-10T11:29:25.049 INFO:tasks.workunit.client.1.smithi178.stdout:5/254: rmdir d9/dc/d17 39 2024-09-10T11:29:25.050 INFO:tasks.workunit.client.1.smithi178.stdout:3/237: dwrite db/d26/f3b [0,4194304] 0 2024-09-10T11:29:25.054 INFO:tasks.workunit.client.1.smithi178.stdout:0/242: rename d0/d19 to d0/dc/d39/d50 0 2024-09-10T11:29:25.057 INFO:tasks.workunit.client.1.smithi178.stdout:1/192: dwrite d3/d9/fb [0,4194304] 0 2024-09-10T11:29:25.059 INFO:tasks.workunit.client.1.smithi178.stdout:3/238: fdatasync db/f1a 0 2024-09-10T11:29:25.061 INFO:tasks.workunit.client.1.smithi178.stdout:0/243: creat d0/d12/f51 x:0 0 0 2024-09-10T11:29:25.062 INFO:tasks.workunit.client.1.smithi178.stdout:2/447: getdents d1/d41/d5d 0 2024-09-10T11:29:25.074 INFO:tasks.workunit.client.1.smithi178.stdout:5/255: dread d9/f28 [0,4194304] 0 2024-09-10T11:29:25.075 INFO:tasks.workunit.client.1.smithi178.stdout:1/193: truncate d3/d26/f28 105607 0 2024-09-10T11:29:25.075 INFO:tasks.workunit.client.1.smithi178.stdout:3/239: write db/fe [166114,131025] 0 2024-09-10T11:29:25.077 INFO:tasks.workunit.client.1.smithi178.stdout:0/244: dwrite d0/d12/f2f [0,4194304] 0 2024-09-10T11:29:25.077 INFO:tasks.workunit.client.1.smithi178.stdout:1/194: fdatasync d3/d13/f17 0 2024-09-10T11:29:25.080 INFO:tasks.workunit.client.1.smithi178.stdout:2/448: dwrite d1/d41/d5d/f71 [4194304,4194304] 0 2024-09-10T11:29:25.085 INFO:tasks.workunit.client.1.smithi178.stdout:0/245: symlink d0/dc/l52 0 2024-09-10T11:29:25.087 INFO:tasks.workunit.client.1.smithi178.stdout:5/256: truncate d9/dc/d17/f34 544910 0 2024-09-10T11:29:25.088 INFO:tasks.workunit.client.1.smithi178.stdout:3/240: dwrite db/d35/f37 [0,4194304] 0 2024-09-10T11:29:25.090 INFO:tasks.workunit.client.1.smithi178.stdout:3/241: write db/d26/f3b [319837,82919] 0 2024-09-10T11:29:25.098 INFO:tasks.workunit.client.1.smithi178.stdout:5/257: read d9/dc/dd/f2d [773631,70428] 0 2024-09-10T11:29:25.103 INFO:tasks.workunit.client.1.smithi178.stdout:2/449: dread d1/d41/d14/f1b [4194304,4194304] 0 2024-09-10T11:29:25.105 INFO:tasks.workunit.client.1.smithi178.stdout:0/246: rename d0/dc/d1f/d2d/f3e to d0/f53 0 2024-09-10T11:29:25.108 INFO:tasks.workunit.client.1.smithi178.stdout:0/247: dread d0/dc/d39/d4c/f24 [0,4194304] 0 2024-09-10T11:29:25.112 INFO:tasks.workunit.client.1.smithi178.stdout:0/248: write d0/dc/d39/d50/f25 [1324704,9367] 0 2024-09-10T11:29:25.115 INFO:tasks.workunit.client.1.smithi178.stdout:0/249: chown d0/dc/d1f/d2d/c31 3619212 1 2024-09-10T11:29:25.115 INFO:tasks.workunit.client.1.smithi178.stdout:3/242: truncate db/f1d 355367 0 2024-09-10T11:29:25.116 INFO:tasks.workunit.client.1.smithi178.stdout:2/450: dwrite d1/d7/d24/d7d/d84/d4b/d89/f93 [0,4194304] 0 2024-09-10T11:29:25.120 INFO:tasks.workunit.client.1.smithi178.stdout:0/250: symlink d0/d2c/l54 0 2024-09-10T11:29:25.121 INFO:tasks.workunit.client.1.smithi178.stdout:0/251: chown d0/f8 184 1 2024-09-10T11:29:25.124 INFO:tasks.workunit.client.1.smithi178.stdout:2/451: creat d1/d7/d24/d7d/d84/d55/f94 x:0 0 0 2024-09-10T11:29:25.126 INFO:tasks.workunit.client.1.smithi178.stdout:3/243: dwrite db/f18 [0,4194304] 0 2024-09-10T11:29:25.129 INFO:tasks.workunit.client.1.smithi178.stdout:3/244: write db/d20/f33 [640150,22681] 0 2024-09-10T11:29:25.133 INFO:tasks.workunit.client.1.smithi178.stdout:2/452: creat d1/d7/d24/d46/d88/f95 x:0 0 0 2024-09-10T11:29:25.134 INFO:tasks.workunit.client.1.smithi178.stdout:3/245: creat db/d36/f47 x:0 0 0 2024-09-10T11:29:25.140 INFO:tasks.workunit.client.1.smithi178.stdout:5/258: dwrite d9/ff [0,4194304] 0 2024-09-10T11:29:25.141 INFO:tasks.workunit.client.1.smithi178.stdout:2/453: creat d1/d7/d24/d7d/d84/d34/d5e/f96 x:0 0 0 2024-09-10T11:29:25.145 INFO:tasks.workunit.client.1.smithi178.stdout:5/259: creat d9/d19/f66 x:0 0 0 2024-09-10T11:29:25.146 INFO:tasks.workunit.client.1.smithi178.stdout:6/372: sync 2024-09-10T11:29:25.146 INFO:tasks.workunit.client.1.smithi178.stdout:9/403: sync 2024-09-10T11:29:25.146 INFO:tasks.workunit.client.1.smithi178.stdout:3/246: dwrite f8 [0,4194304] 0 2024-09-10T11:29:25.146 INFO:tasks.workunit.client.1.smithi178.stdout:0/252: truncate d0/dc/f13 1186753 0 2024-09-10T11:29:25.150 INFO:tasks.workunit.client.1.smithi178.stdout:2/454: dread d1/d7/d24/d7d/d84/d4b/f67 [0,4194304] 0 2024-09-10T11:29:25.156 INFO:tasks.workunit.client.1.smithi178.stdout:6/373: symlink d3/d4e/l85 0 2024-09-10T11:29:25.156 INFO:tasks.workunit.client.1.smithi178.stdout:3/247: mknod db/d20/c48 0 2024-09-10T11:29:25.157 INFO:tasks.workunit.client.1.smithi178.stdout:0/253: creat d0/dc/d39/f55 x:0 0 0 2024-09-10T11:29:25.158 INFO:tasks.workunit.client.1.smithi178.stdout:5/260: dread d9/f51 [0,4194304] 0 2024-09-10T11:29:25.160 INFO:tasks.workunit.client.1.smithi178.stdout:0/254: readlink d0/dc/l30 0 2024-09-10T11:29:25.161 INFO:tasks.workunit.client.1.smithi178.stdout:2/455: write d1/d7/d24/d7d/d84/f36 [4381242,6961] 0 2024-09-10T11:29:25.162 INFO:tasks.workunit.client.1.smithi178.stdout:0/255: dread - d0/d2c/d33/f45 zero size 2024-09-10T11:29:25.163 INFO:tasks.workunit.client.1.smithi178.stdout:5/261: creat d9/d19/f67 x:0 0 0 2024-09-10T11:29:25.163 INFO:tasks.workunit.client.1.smithi178.stdout:9/404: write d2/d19/f39 [986066,114505] 0 2024-09-10T11:29:25.164 INFO:tasks.workunit.client.1.smithi178.stdout:2/456: mkdir d1/d41/d14/d11/d53/d97 0 2024-09-10T11:29:25.165 INFO:tasks.workunit.client.1.smithi178.stdout:5/262: readlink d9/dc/l26 0 2024-09-10T11:29:25.166 INFO:tasks.workunit.client.1.smithi178.stdout:0/256: symlink d0/d2c/d33/l56 0 2024-09-10T11:29:25.167 INFO:tasks.workunit.client.1.smithi178.stdout:9/405: symlink d2/dc/l74 0 2024-09-10T11:29:25.167 INFO:tasks.workunit.client.1.smithi178.stdout:3/248: link db/d20/l22 db/d26/l49 0 2024-09-10T11:29:25.168 INFO:tasks.workunit.client.1.smithi178.stdout:9/406: chown d2/df/d31 125545 1 2024-09-10T11:29:25.170 INFO:tasks.workunit.client.1.smithi178.stdout:3/249: fsync db/f1a 0 2024-09-10T11:29:25.171 INFO:tasks.workunit.client.1.smithi178.stdout:2/457: fdatasync d1/d22/f3a 0 2024-09-10T11:29:25.173 INFO:tasks.workunit.client.1.smithi178.stdout:9/407: rename d2/d19/d1c/d2d/d54 to d2/d13/d75 0 2024-09-10T11:29:25.175 INFO:tasks.workunit.client.1.smithi178.stdout:3/250: chown db/f1d 12625897 1 2024-09-10T11:29:25.175 INFO:tasks.workunit.client.1.smithi178.stdout:6/374: dread d3/d7/d35/f6f [0,4194304] 0 2024-09-10T11:29:25.177 INFO:tasks.workunit.client.1.smithi178.stdout:0/257: dread d0/f22 [0,4194304] 0 2024-09-10T11:29:25.177 INFO:tasks.workunit.client.1.smithi178.stdout:9/408: truncate d2/df/d31/d37/f70 900025 0 2024-09-10T11:29:25.178 INFO:tasks.workunit.client.1.smithi178.stdout:3/251: chown db/d35/d3a/l44 1839 1 2024-09-10T11:29:25.178 INFO:tasks.workunit.client.1.smithi178.stdout:0/258: readlink d0/dc/l52 0 2024-09-10T11:29:25.179 INFO:tasks.workunit.client.1.smithi178.stdout:3/252: readlink l9 0 2024-09-10T11:29:25.180 INFO:tasks.workunit.client.1.smithi178.stdout:9/409: creat d2/df/d31/f76 x:0 0 0 2024-09-10T11:29:25.181 INFO:tasks.workunit.client.1.smithi178.stdout:0/259: creat d0/dc/d39/d4c/f57 x:0 0 0 2024-09-10T11:29:25.181 INFO:tasks.workunit.client.1.smithi178.stdout:5/263: dread d9/d19/f1a [0,4194304] 0 2024-09-10T11:29:25.181 INFO:tasks.workunit.client.1.smithi178.stdout:9/410: stat d2/d8/l1d 0 2024-09-10T11:29:25.183 INFO:tasks.workunit.client.1.smithi178.stdout:6/375: rename d3/d7/d8/d15/d1c/d20/d4b/f62 to d3/d42/d57/d78/f86 0 2024-09-10T11:29:25.184 INFO:tasks.workunit.client.1.smithi178.stdout:3/253: creat db/d35/d3a/f4a x:0 0 0 2024-09-10T11:29:25.185 INFO:tasks.workunit.client.1.smithi178.stdout:6/376: write d3/f14 [195730,81904] 0 2024-09-10T11:29:25.187 INFO:tasks.workunit.client.1.smithi178.stdout:2/458: truncate d1/d7/d24/d7d/d84/d34/f15 5210220 0 2024-09-10T11:29:25.189 INFO:tasks.workunit.client.1.smithi178.stdout:5/264: unlink d9/dc/d17/f1e 0 2024-09-10T11:29:25.189 INFO:tasks.workunit.client.1.smithi178.stdout:3/254: creat db/d35/d3a/f4b x:0 0 0 2024-09-10T11:29:25.191 INFO:tasks.workunit.client.1.smithi178.stdout:5/265: write d9/dc/f15 [1567818,112254] 0 2024-09-10T11:29:25.191 INFO:tasks.workunit.client.1.smithi178.stdout:2/459: write d1/d7/f16 [6047650,74468] 0 2024-09-10T11:29:25.196 INFO:tasks.workunit.client.1.smithi178.stdout:9/411: dread d2/d13/f15 [0,4194304] 0 2024-09-10T11:29:25.196 INFO:tasks.workunit.client.1.smithi178.stdout:6/377: dread d3/d7/d8/d15/d1c/d20/f2d [0,4194304] 0 2024-09-10T11:29:25.196 INFO:tasks.workunit.client.1.smithi178.stdout:5/266: dread - d9/dc/d17/d1d/f54 zero size 2024-09-10T11:29:25.199 INFO:tasks.workunit.client.1.smithi178.stdout:6/378: mkdir d3/d6e/d87 0 2024-09-10T11:29:25.200 INFO:tasks.workunit.client.1.smithi178.stdout:0/260: dwrite d0/f22 [0,4194304] 0 2024-09-10T11:29:25.200 INFO:tasks.workunit.client.1.smithi178.stdout:6/379: chown d3/d7/d8/d19 883724546 1 2024-09-10T11:29:25.204 INFO:tasks.workunit.client.1.smithi178.stdout:6/380: symlink d3/d42/d57/d78/d7c/l88 0 2024-09-10T11:29:25.207 INFO:tasks.workunit.client.1.smithi178.stdout:5/267: dwrite d9/d19/f1a [0,4194304] 0 2024-09-10T11:29:25.209 INFO:tasks.workunit.client.1.smithi178.stdout:2/460: truncate d1/d7/f16 4383919 0 2024-09-10T11:29:25.211 INFO:tasks.workunit.client.1.smithi178.stdout:0/261: dwrite d0/d12/f2f [0,4194304] 0 2024-09-10T11:29:25.212 INFO:tasks.workunit.client.1.smithi178.stdout:2/461: chown d1/f13 772 1 2024-09-10T11:29:25.212 INFO:tasks.workunit.client.1.smithi178.stdout:0/262: dread - d0/dc/d1f/d2d/f42 zero size 2024-09-10T11:29:25.215 INFO:tasks.workunit.client.1.smithi178.stdout:5/268: mkdir d9/dc/dd/d37/d68 0 2024-09-10T11:29:25.217 INFO:tasks.workunit.client.1.smithi178.stdout:5/269: readlink d9/dc/dd/l38 0 2024-09-10T11:29:25.217 INFO:tasks.workunit.client.1.smithi178.stdout:6/381: creat d3/d7/d8/d15/d1c/f89 x:0 0 0 2024-09-10T11:29:25.218 INFO:tasks.workunit.client.1.smithi178.stdout:2/462: creat d1/d7/d24/d7d/d84/f98 x:0 0 0 2024-09-10T11:29:25.221 INFO:tasks.workunit.client.1.smithi178.stdout:6/382: mknod d3/d42/d57/d78/c8a 0 2024-09-10T11:29:25.223 INFO:tasks.workunit.client.1.smithi178.stdout:0/263: dread d0/d12/f2f [0,4194304] 0 2024-09-10T11:29:25.225 INFO:tasks.workunit.client.1.smithi178.stdout:2/463: creat d1/d41/d14/d11/d53/d97/f99 x:0 0 0 2024-09-10T11:29:25.227 INFO:tasks.workunit.client.1.smithi178.stdout:5/270: dwrite d9/ff [4194304,4194304] 0 2024-09-10T11:29:25.229 INFO:tasks.workunit.client.1.smithi178.stdout:0/264: fsync d0/d12/f2f 0 2024-09-10T11:29:25.231 INFO:tasks.workunit.client.1.smithi178.stdout:6/383: link d3/d7/d8/f58 d3/d7/d8/d19/f8b 0 2024-09-10T11:29:25.235 INFO:tasks.workunit.client.1.smithi178.stdout:0/265: dwrite d0/f9 [0,4194304] 0 2024-09-10T11:29:25.238 INFO:tasks.workunit.client.1.smithi178.stdout:5/271: unlink d9/dc/d17/c3a 0 2024-09-10T11:29:25.241 INFO:tasks.workunit.client.1.smithi178.stdout:6/384: symlink d3/d7/d8/d72/d75/l8c 0 2024-09-10T11:29:25.242 INFO:tasks.workunit.client.1.smithi178.stdout:2/464: write d1/d7/d24/d7d/d84/f2d [1624270,121242] 0 2024-09-10T11:29:25.242 INFO:tasks.workunit.client.1.smithi178.stdout:5/272: rename d9/dc/f47 to d9/dc/dd/f69 0 2024-09-10T11:29:25.244 INFO:tasks.workunit.client.1.smithi178.stdout:0/266: dread d0/dc/d1f/f26 [0,4194304] 0 2024-09-10T11:29:25.245 INFO:tasks.workunit.client.1.smithi178.stdout:6/385: symlink d3/d7/d8/d19/l8d 0 2024-09-10T11:29:25.245 INFO:tasks.workunit.client.1.smithi178.stdout:0/267: dread - d0/dc/f2b zero size 2024-09-10T11:29:25.245 INFO:tasks.workunit.client.1.smithi178.stdout:5/273: read - d9/dc/d17/d1d/f54 zero size 2024-09-10T11:29:25.246 INFO:tasks.workunit.client.1.smithi178.stdout:2/465: mknod d1/d7/d24/d7d/d84/c9a 0 2024-09-10T11:29:25.246 INFO:tasks.workunit.client.0.smithi144.stderr:++ readlink -f fsstress 2024-09-10T11:29:25.246 INFO:tasks.workunit.client.1.smithi178.stdout:0/268: dread - d0/dc/f2b zero size 2024-09-10T11:29:25.247 INFO:tasks.workunit.client.1.smithi178.stdout:5/274: dread - d9/dc/dd/d3f/f49 zero size 2024-09-10T11:29:25.247 INFO:tasks.workunit.client.1.smithi178.stdout:0/269: readlink d0/d2c/d33/l56 0 2024-09-10T11:29:25.248 INFO:tasks.workunit.client.0.smithi144.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-09-10T11:29:25.248 INFO:tasks.workunit.client.0.smithi144.stderr:+ popd 2024-09-10T11:29:25.249 INFO:tasks.workunit.client.1.smithi178.stdout:6/386: mkdir d3/d7/d8/d15/d1c/d2a/d18/d8e 0 2024-09-10T11:29:25.249 INFO:tasks.workunit.client.0.smithi144.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-09-10T11:29:25.249 INFO:tasks.workunit.client.0.smithi144.stderr:+ popd 2024-09-10T11:29:25.250 INFO:tasks.workunit.client.0.smithi144.stdout:~/cephtest/mnt.0/client.0/tmp 2024-09-10T11:29:25.250 INFO:tasks.workunit.client.0.smithi144.stderr:++ mktemp -d -p . 2024-09-10T11:29:25.251 INFO:tasks.workunit.client.1.smithi178.stdout:5/275: chown d9/dc/d17/l39 11620 1 2024-09-10T11:29:25.251 INFO:tasks.workunit.client.1.smithi178.stdout:6/387: dread - d3/d42/d57/d78/f86 zero size 2024-09-10T11:29:25.252 INFO:tasks.workunit.client.1.smithi178.stdout:6/388: readlink d3/d7/l17 0 2024-09-10T11:29:25.253 INFO:tasks.workunit.client.0.smithi144.stderr:+ T=./tmp.hgnv0hcqfM 2024-09-10T11:29:25.253 INFO:tasks.workunit.client.0.smithi144.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.hgnv0hcqfM -l 1 -n 1000 -p 10 -v 2024-09-10T11:29:25.254 INFO:tasks.workunit.client.1.smithi178.stdout:5/276: unlink d9/dc/l26 0 2024-09-10T11:29:25.255 INFO:tasks.workunit.client.1.smithi178.stdout:0/270: dwrite d0/dc/d39/d4c/f3a [0,4194304] 0 2024-09-10T11:29:25.257 INFO:tasks.workunit.client.0.smithi144.stdout:seed = 1726201955 2024-09-10T11:29:25.259 INFO:tasks.workunit.client.1.smithi178.stdout:2/466: dwrite d1/d7/d24/d7d/d84/d4b/f67 [0,4194304] 0 2024-09-10T11:29:25.261 INFO:tasks.workunit.client.1.smithi178.stdout:0/271: symlink d0/dc/d1f/l58 0 2024-09-10T11:29:25.262 INFO:tasks.workunit.client.1.smithi178.stdout:5/277: mknod d9/dc/d17/d1d/c6a 0 2024-09-10T11:29:25.262 INFO:tasks.workunit.client.1.smithi178.stdout:2/467: write d1/d41/f51 [2338416,119100] 0 2024-09-10T11:29:25.263 INFO:tasks.workunit.client.1.smithi178.stdout:5/278: rename d9/dc/dd/d37/d42 to d9/dc/dd/d37/d42/d6b 22 2024-09-10T11:29:25.266 INFO:tasks.workunit.client.0.smithi144.stdout:0/0: creat f0 x:0 0 0 2024-09-10T11:29:25.267 INFO:tasks.workunit.client.1.smithi178.stdout:0/272: rmdir d0/dc/d1f/d2d 39 2024-09-10T11:29:25.269 INFO:tasks.workunit.client.0.smithi144.stdout:1/0: symlink l0 0 2024-09-10T11:29:25.269 INFO:tasks.workunit.client.0.smithi144.stdout:2/0: unlink - no file 2024-09-10T11:29:25.269 INFO:tasks.workunit.client.0.smithi144.stdout:2/1: write - no filename 2024-09-10T11:29:25.269 INFO:tasks.workunit.client.0.smithi144.stdout:1/1: dwrite - no filename 2024-09-10T11:29:25.269 INFO:tasks.workunit.client.0.smithi144.stdout:2/2: rename - no filename 2024-09-10T11:29:25.269 INFO:tasks.workunit.client.0.smithi144.stdout:1/2: dwrite - no filename 2024-09-10T11:29:25.272 INFO:tasks.workunit.client.0.smithi144.stdout:1/3: unlink l0 0 2024-09-10T11:29:25.272 INFO:tasks.workunit.client.0.smithi144.stdout:1/4: dread - no filename 2024-09-10T11:29:25.272 INFO:tasks.workunit.client.0.smithi144.stdout:2/3: symlink l0 0 2024-09-10T11:29:25.273 INFO:tasks.workunit.client.0.smithi144.stdout:2/4: dwrite - no filename 2024-09-10T11:29:25.273 INFO:tasks.workunit.client.0.smithi144.stdout:3/0: write - no filename 2024-09-10T11:29:25.273 INFO:tasks.workunit.client.1.smithi178.stdout:2/468: rename d1/d7/d24/d7d/d84/d55/d6e to d1/d22/d4d/d9b 0 2024-09-10T11:29:25.275 INFO:tasks.workunit.client.1.smithi178.stdout:5/279: dread f4 [0,4194304] 0 2024-09-10T11:29:25.276 INFO:tasks.workunit.client.0.smithi144.stdout:2/5: mknod c1 0 2024-09-10T11:29:25.276 INFO:tasks.workunit.client.0.smithi144.stdout:2/6: rmdir - no directory 2024-09-10T11:29:25.276 INFO:tasks.workunit.client.0.smithi144.stdout:4/0: write - no filename 2024-09-10T11:29:25.276 INFO:tasks.workunit.client.0.smithi144.stdout:4/1: write - no filename 2024-09-10T11:29:25.276 INFO:tasks.workunit.client.0.smithi144.stdout:4/2: write - no filename 2024-09-10T11:29:25.276 INFO:tasks.workunit.client.0.smithi144.stdout:4/3: stat - no entries 2024-09-10T11:29:25.277 INFO:tasks.workunit.client.0.smithi144.stdout:4/4: dwrite - no filename 2024-09-10T11:29:25.277 INFO:tasks.workunit.client.1.smithi178.stdout:5/280: symlink d9/dc/dd/l6c 0 2024-09-10T11:29:25.278 INFO:tasks.workunit.client.0.smithi144.stdout:5/0: dwrite - no filename 2024-09-10T11:29:25.279 INFO:tasks.workunit.client.0.smithi144.stdout:3/1: creat f0 x:0 0 0 2024-09-10T11:29:25.279 INFO:tasks.workunit.client.0.smithi144.stdout:0/1: dwrite f0 [0,4194304] 0 2024-09-10T11:29:25.280 INFO:tasks.workunit.client.0.smithi144.stdout:2/7: creat f2 x:0 0 0 2024-09-10T11:29:25.282 INFO:tasks.workunit.client.1.smithi178.stdout:2/469: dwrite d1/d22/f26 [0,4194304] 0 2024-09-10T11:29:25.283 INFO:tasks.workunit.client.0.smithi144.stdout:4/5: creat f0 x:0 0 0 2024-09-10T11:29:25.283 INFO:tasks.workunit.client.1.smithi178.stdout:5/281: mkdir d9/dc/dd/d37/d42/d55/d6d 0 2024-09-10T11:29:25.284 INFO:tasks.workunit.client.0.smithi144.stdout:2/8: symlink l3 0 2024-09-10T11:29:25.286 INFO:tasks.workunit.client.0.smithi144.stdout:5/1: getdents . 0 2024-09-10T11:29:25.286 INFO:tasks.workunit.client.0.smithi144.stdout:5/2: rmdir - no directory 2024-09-10T11:29:25.286 INFO:tasks.workunit.client.0.smithi144.stdout:5/3: write - no filename 2024-09-10T11:29:25.288 INFO:tasks.workunit.client.1.smithi178.stdout:5/282: chown d9/dc/d17/f3c 5301158 1 2024-09-10T11:29:25.288 INFO:tasks.workunit.client.0.smithi144.stdout:4/6: chown f0 2 1 2024-09-10T11:29:25.289 INFO:tasks.workunit.client.1.smithi178.stdout:2/470: truncate d1/d41/d14/d11/d53/d97/f99 171085 0 2024-09-10T11:29:25.289 INFO:tasks.workunit.client.0.smithi144.stdout:2/9: rename l3 to l4 0 2024-09-10T11:29:25.289 INFO:tasks.workunit.client.0.smithi144.stdout:7/0: write - no filename 2024-09-10T11:29:25.289 INFO:tasks.workunit.client.0.smithi144.stdout:7/1: fdatasync - no filename 2024-09-10T11:29:25.289 INFO:tasks.workunit.client.0.smithi144.stdout:7/2: dwrite - no filename 2024-09-10T11:29:25.290 INFO:tasks.workunit.client.0.smithi144.stdout:4/7: stat f0 0 2024-09-10T11:29:25.290 INFO:tasks.workunit.client.0.smithi144.stdout:4/8: readlink - no filename 2024-09-10T11:29:25.292 INFO:tasks.workunit.client.0.smithi144.stdout:6/0: creat f0 x:0 0 0 2024-09-10T11:29:25.293 INFO:tasks.workunit.client.0.smithi144.stdout:2/10: truncate f2 816182 0 2024-09-10T11:29:25.293 INFO:tasks.workunit.client.1.smithi178.stdout:0/273: dwrite d0/dc/d39/d50/f1d [0,4194304] 0 2024-09-10T11:29:25.293 INFO:tasks.workunit.client.1.smithi178.stdout:2/471: symlink d1/d7/l9c 0 2024-09-10T11:29:25.295 INFO:tasks.workunit.client.0.smithi144.stdout:8/0: rename - no filename 2024-09-10T11:29:25.295 INFO:tasks.workunit.client.0.smithi144.stdout:6/1: write f0 [335562,59417] 0 2024-09-10T11:29:25.295 INFO:tasks.workunit.client.0.smithi144.stdout:3/2: dwrite f0 [0,4194304] 0 2024-09-10T11:29:25.298 INFO:tasks.workunit.client.0.smithi144.stdout:4/9: creat f1 x:0 0 0 2024-09-10T11:29:25.299 INFO:tasks.workunit.client.0.smithi144.stdout:5/4: getdents . 0 2024-09-10T11:29:25.299 INFO:tasks.workunit.client.0.smithi144.stdout:2/11: symlink l5 0 2024-09-10T11:29:25.299 INFO:tasks.workunit.client.0.smithi144.stdout:7/3: getdents . 0 2024-09-10T11:29:25.301 INFO:tasks.workunit.client.1.smithi178.stdout:2/472: mkdir d1/d7/d24/d46/d88/d9d 0 2024-09-10T11:29:25.301 INFO:tasks.workunit.client.0.smithi144.stdout:3/3: creat f1 x:0 0 0 2024-09-10T11:29:25.301 INFO:tasks.workunit.client.0.smithi144.stdout:9/0: rename - no filename 2024-09-10T11:29:25.301 INFO:tasks.workunit.client.0.smithi144.stdout:9/1: rename - no filename 2024-09-10T11:29:25.302 INFO:tasks.workunit.client.0.smithi144.stdout:9/2: dwrite - no filename 2024-09-10T11:29:25.302 INFO:tasks.workunit.client.0.smithi144.stdout:4/10: creat f2 x:0 0 0 2024-09-10T11:29:25.302 INFO:tasks.workunit.client.0.smithi144.stdout:2/12: write f2 [912734,57467] 0 2024-09-10T11:29:25.302 INFO:tasks.workunit.client.0.smithi144.stdout:6/2: mknod c1 0 2024-09-10T11:29:25.302 INFO:tasks.workunit.client.1.smithi178.stdout:5/283: dread d9/d11/f14 [0,4194304] 0 2024-09-10T11:29:25.303 INFO:tasks.workunit.client.0.smithi144.stdout:6/3: truncate f0 832763 0 2024-09-10T11:29:25.303 INFO:tasks.workunit.client.0.smithi144.stdout:8/1: mkdir d0 0 2024-09-10T11:29:25.303 INFO:tasks.workunit.client.0.smithi144.stdout:8/2: write - no filename 2024-09-10T11:29:25.305 INFO:tasks.workunit.client.1.smithi178.stdout:5/284: write d9/dc/dd/d37/d42/f5f [3925221,57973] 0 2024-09-10T11:29:25.306 INFO:tasks.workunit.client.1.smithi178.stdout:2/473: symlink d1/d22/d4d/d9b/l9e 0 2024-09-10T11:29:25.310 INFO:tasks.workunit.client.0.smithi144.stdout:4/11: mknod c3 0 2024-09-10T11:29:25.310 INFO:tasks.workunit.client.0.smithi144.stdout:4/12: dread - f1 zero size 2024-09-10T11:29:25.310 INFO:tasks.workunit.client.0.smithi144.stdout:4/13: chown f2 76055 1 2024-09-10T11:29:25.311 INFO:tasks.workunit.client.0.smithi144.stdout:2/13: unlink l5 0 2024-09-10T11:29:25.312 INFO:tasks.workunit.client.1.smithi178.stdout:0/274: truncate d0/f22 2096987 0 2024-09-10T11:29:25.313 INFO:tasks.workunit.client.1.smithi178.stdout:0/275: stat d0/dc/d1f/l58 0 2024-09-10T11:29:25.314 INFO:tasks.workunit.client.0.smithi144.stdout:6/4: symlink l2 0 2024-09-10T11:29:25.315 INFO:tasks.workunit.client.1.smithi178.stdout:5/285: dwrite d9/f32 [0,4194304] 0 2024-09-10T11:29:25.316 INFO:tasks.workunit.client.0.smithi144.stdout:5/5: mkdir d0 0 2024-09-10T11:29:25.317 INFO:tasks.workunit.client.0.smithi144.stdout:7/4: symlink l0 0 2024-09-10T11:29:25.317 INFO:tasks.workunit.client.0.smithi144.stdout:7/5: dwrite - no filename 2024-09-10T11:29:25.317 INFO:tasks.workunit.client.0.smithi144.stdout:7/6: truncate - no filename 2024-09-10T11:29:25.318 INFO:tasks.workunit.client.1.smithi178.stdout:2/474: fdatasync d1/d22/f26 0 2024-09-10T11:29:25.318 INFO:tasks.workunit.client.1.smithi178.stdout:0/276: truncate d0/dc/d39/d4c/f38 1210693 0 2024-09-10T11:29:25.319 INFO:tasks.workunit.client.1.smithi178.stdout:5/286: creat d9/dc/dd/d37/d42/d55/f6e x:0 0 0 2024-09-10T11:29:25.319 INFO:tasks.workunit.client.0.smithi144.stdout:9/3: creat f0 x:0 0 0 2024-09-10T11:29:25.320 INFO:tasks.workunit.client.0.smithi144.stdout:2/14: dwrite f2 [0,4194304] 0 2024-09-10T11:29:25.321 INFO:tasks.workunit.client.1.smithi178.stdout:5/287: write d9/ff [1662529,41721] 0 2024-09-10T11:29:25.322 INFO:tasks.workunit.client.0.smithi144.stdout:4/14: rename f0 to f4 0 2024-09-10T11:29:25.322 INFO:tasks.workunit.client.0.smithi144.stdout:4/15: read - f1 zero size 2024-09-10T11:29:25.323 INFO:tasks.workunit.client.0.smithi144.stdout:2/15: truncate f2 4874655 0 2024-09-10T11:29:25.325 INFO:tasks.workunit.client.0.smithi144.stdout:3/4: link f0 f2 0 2024-09-10T11:29:25.326 INFO:tasks.workunit.client.1.smithi178.stdout:5/288: unlink d9/dc/d17/d1d/l5d 0 2024-09-10T11:29:25.326 INFO:tasks.workunit.client.0.smithi144.stdout:8/3: mknod d0/c1 0 2024-09-10T11:29:25.327 INFO:tasks.workunit.client.0.smithi144.stdout:6/5: mknod c3 0 2024-09-10T11:29:25.329 INFO:tasks.workunit.client.1.smithi178.stdout:5/289: chown d9/dc/f15 1 1 2024-09-10T11:29:25.332 INFO:tasks.workunit.client.0.smithi144.stdout:9/4: symlink l1 0 2024-09-10T11:29:25.332 INFO:tasks.workunit.client.1.smithi178.stdout:0/277: dwrite d0/f8 [0,4194304] 0 2024-09-10T11:29:25.332 INFO:tasks.workunit.client.1.smithi178.stdout:2/475: dwrite d1/d7/d24/d7d/d84/d55/f73 [4194304,4194304] 0 2024-09-10T11:29:25.332 INFO:tasks.workunit.client.1.smithi178.stdout:5/290: dread - d9/dc/d17/d1d/f54 zero size 2024-09-10T11:29:25.334 INFO:tasks.workunit.client.1.smithi178.stdout:2/476: readlink d1/d41/l18 0 2024-09-10T11:29:25.336 INFO:tasks.workunit.client.1.smithi178.stdout:5/291: truncate d9/dc/dd/f2d 188173 0 2024-09-10T11:29:25.336 INFO:tasks.workunit.client.0.smithi144.stdout:2/16: rename f2 to f6 0 2024-09-10T11:29:25.337 INFO:tasks.workunit.client.1.smithi178.stdout:0/278: symlink d0/d12/l59 0 2024-09-10T11:29:25.340 INFO:tasks.workunit.client.1.smithi178.stdout:0/279: creat d0/f5a x:0 0 0 2024-09-10T11:29:25.340 INFO:tasks.workunit.client.0.smithi144.stdout:3/5: rename f1 to f3 0 2024-09-10T11:29:25.340 INFO:tasks.workunit.client.0.smithi144.stdout:3/6: dread - f3 zero size 2024-09-10T11:29:25.344 INFO:tasks.workunit.client.1.smithi178.stdout:2/477: truncate d1/d7/d24/d7d/d84/d4b/f66 2615908 0 2024-09-10T11:29:25.344 INFO:tasks.workunit.client.1.smithi178.stdout:0/280: dread d0/dc/ff [0,4194304] 0 2024-09-10T11:29:25.344 INFO:tasks.workunit.client.0.smithi144.stdout:6/6: mknod c4 0 2024-09-10T11:29:25.345 INFO:tasks.workunit.client.1.smithi178.stdout:5/292: dread d9/dc/dd/d37/d42/f4e [0,4194304] 0 2024-09-10T11:29:25.346 INFO:tasks.workunit.client.0.smithi144.stdout:9/5: creat f2 x:0 0 0 2024-09-10T11:29:25.346 INFO:tasks.workunit.client.1.smithi178.stdout:5/293: write d9/dc/f2b [1523809,49841] 0 2024-09-10T11:29:25.347 INFO:tasks.workunit.client.0.smithi144.stdout:2/17: mkdir d7 0 2024-09-10T11:29:25.349 INFO:tasks.workunit.client.0.smithi144.stdout:6/7: mknod c5 0 2024-09-10T11:29:25.350 INFO:tasks.workunit.client.0.smithi144.stdout:5/6: rmdir d0 0 2024-09-10T11:29:25.350 INFO:tasks.workunit.client.0.smithi144.stdout:5/7: chown . 80618642 1 2024-09-10T11:29:25.352 INFO:tasks.workunit.client.1.smithi178.stdout:8/259: sync 2024-09-10T11:29:25.352 INFO:tasks.workunit.client.1.smithi178.stdout:4/200: sync 2024-09-10T11:29:25.353 INFO:tasks.workunit.client.1.smithi178.stdout:8/260: chown d5/d30/c57 739518 1 2024-09-10T11:29:25.354 INFO:tasks.workunit.client.1.smithi178.stdout:8/261: chown d5/d30/d33/c3e 1832286972 1 2024-09-10T11:29:25.354 INFO:tasks.workunit.client.1.smithi178.stdout:8/262: fsync d5/fa 0 2024-09-10T11:29:25.358 INFO:tasks.workunit.client.1.smithi178.stdout:4/201: dwrite d0/d8/df/f3b [0,4194304] 0 2024-09-10T11:29:25.389 INFO:tasks.workunit.client.0.smithi144.stdout:9/6: mknod c3 0 2024-09-10T11:29:25.390 INFO:tasks.workunit.client.0.smithi144.stdout:6/8: rename f0 to f6 0 2024-09-10T11:29:25.393 INFO:tasks.workunit.client.1.smithi178.stdout:8/263: rmdir d5/d30 39 2024-09-10T11:29:25.394 INFO:tasks.workunit.client.0.smithi144.stdout:9/7: mknod c4 0 2024-09-10T11:29:25.395 INFO:tasks.workunit.client.0.smithi144.stdout:9/8: dread - f0 zero size 2024-09-10T11:29:25.396 INFO:tasks.workunit.client.0.smithi144.stdout:6/9: rename c3 to c7 0 2024-09-10T11:29:25.397 INFO:tasks.workunit.client.1.smithi178.stdout:5/294: getdents d9/dc/d17/d4b 0 2024-09-10T11:29:25.397 INFO:tasks.workunit.client.0.smithi144.stdout:6/10: truncate f6 1117425 0 2024-09-10T11:29:25.398 INFO:tasks.workunit.client.0.smithi144.stdout:9/9: unlink f2 0 2024-09-10T11:29:25.398 INFO:tasks.workunit.client.0.smithi144.stdout:9/10: dread - f0 zero size 2024-09-10T11:29:25.399 INFO:tasks.workunit.client.1.smithi178.stdout:0/281: dwrite d0/f4e [0,4194304] 0 2024-09-10T11:29:25.402 INFO:tasks.workunit.client.0.smithi144.stdout:6/11: creat f8 x:0 0 0 2024-09-10T11:29:25.404 INFO:tasks.workunit.client.0.smithi144.stdout:6/12: mkdir d9 0 2024-09-10T11:29:25.409 INFO:tasks.workunit.client.0.smithi144.stdout:9/11: dwrite f0 [0,4194304] 0 2024-09-10T11:29:25.412 INFO:tasks.workunit.client.1.smithi178.stdout:8/264: unlink d5/db/d10/f55 0 2024-09-10T11:29:25.412 INFO:tasks.workunit.client.1.smithi178.stdout:8/265: chown d5/db/d10/d35 0 1 2024-09-10T11:29:25.413 INFO:tasks.workunit.client.1.smithi178.stdout:8/266: chown d5/db/d10/l1a 11 1 2024-09-10T11:29:25.415 INFO:tasks.workunit.client.0.smithi144.stdout:6/13: dread f6 [0,4194304] 0 2024-09-10T11:29:25.416 INFO:tasks.workunit.client.1.smithi178.stdout:0/282: creat d0/dc/d39/f5b x:0 0 0 2024-09-10T11:29:25.417 INFO:tasks.workunit.client.0.smithi144.stdout:6/14: dread - f8 zero size 2024-09-10T11:29:25.418 INFO:tasks.workunit.client.0.smithi144.stdout:6/15: readlink l2 0 2024-09-10T11:29:25.419 INFO:tasks.workunit.client.1.smithi178.stdout:8/267: fsync d5/f37 0 2024-09-10T11:29:25.420 INFO:tasks.workunit.client.1.smithi178.stdout:8/268: dread - d5/db/d10/d35/f44 zero size 2024-09-10T11:29:25.420 INFO:tasks.workunit.client.0.smithi144.stdout:9/12: dwrite f0 [0,4194304] 0 2024-09-10T11:29:25.421 INFO:tasks.workunit.client.1.smithi178.stdout:5/295: rmdir d9/dc/dd/d37/d68 0 2024-09-10T11:29:25.421 INFO:tasks.workunit.client.0.smithi144.stdout:6/16: mknod d9/ca 0 2024-09-10T11:29:25.422 INFO:tasks.workunit.client.1.smithi178.stdout:5/296: dread - d9/dc/dd/d37/d42/d55/f6e zero size 2024-09-10T11:29:25.422 INFO:tasks.workunit.client.1.smithi178.stdout:5/297: read f2 [1632263,85518] 0 2024-09-10T11:29:25.422 INFO:tasks.workunit.client.1.smithi178.stdout:0/283: mknod d0/d12/d3f/c5c 0 2024-09-10T11:29:25.424 INFO:tasks.workunit.client.1.smithi178.stdout:8/269: unlink d5/db/d10/l2d 0 2024-09-10T11:29:25.424 INFO:tasks.workunit.client.0.smithi144.stdout:6/17: dread f6 [0,4194304] 0 2024-09-10T11:29:25.425 INFO:tasks.workunit.client.1.smithi178.stdout:0/284: rmdir d0/d12/d3f 39 2024-09-10T11:29:25.426 INFO:tasks.workunit.client.0.smithi144.stdout:6/18: symlink d9/lb 0 2024-09-10T11:29:25.426 INFO:tasks.workunit.client.0.smithi144.stdout:6/19: chown d9 259443443 1 2024-09-10T11:29:25.427 INFO:tasks.workunit.client.1.smithi178.stdout:5/298: rename d9/dc/dd/d37/d42/d55/f5b to d9/dc/dd/d37/d42/d62/f6f 0 2024-09-10T11:29:25.429 INFO:tasks.workunit.client.1.smithi178.stdout:8/270: symlink d5/db/d10/l58 0 2024-09-10T11:29:25.431 INFO:tasks.workunit.client.1.smithi178.stdout:0/285: rmdir d0/dc/d1f/d2d 39 2024-09-10T11:29:25.432 INFO:tasks.workunit.client.1.smithi178.stdout:8/271: mknod d5/db/d10/d20/c59 0 2024-09-10T11:29:25.434 INFO:tasks.workunit.client.1.smithi178.stdout:5/299: dwrite d9/f32 [4194304,4194304] 0 2024-09-10T11:29:25.435 INFO:tasks.workunit.client.0.smithi144.stdout:6/20: dwrite f6 [0,4194304] 0 2024-09-10T11:29:25.436 INFO:tasks.workunit.client.1.smithi178.stdout:8/272: read d5/db/fe [682070,71922] 0 2024-09-10T11:29:25.437 INFO:tasks.workunit.client.1.smithi178.stdout:5/300: chown d9/dc/dd/d37/d42/f5f 1555171598 1 2024-09-10T11:29:25.439 INFO:tasks.workunit.client.0.smithi144.stdout:6/21: mkdir d9/dc 0 2024-09-10T11:29:25.439 INFO:tasks.workunit.client.1.smithi178.stdout:5/301: readlink d9/dc/dd/d37/l3b 0 2024-09-10T11:29:25.440 INFO:tasks.workunit.client.0.smithi144.stdout:6/22: write f8 [136512,88574] 0 2024-09-10T11:29:25.442 INFO:tasks.workunit.client.1.smithi178.stdout:8/273: creat d5/db/d10/d35/f5a x:0 0 0 2024-09-10T11:29:25.442 INFO:tasks.workunit.client.1.smithi178.stdout:0/286: dwrite d0/dc/d1f/d2d/f42 [0,4194304] 0 2024-09-10T11:29:25.444 INFO:tasks.workunit.client.0.smithi144.stdout:6/23: mkdir d9/dd 0 2024-09-10T11:29:25.446 INFO:tasks.workunit.client.0.smithi144.stdout:6/24: rmdir d9 39 2024-09-10T11:29:25.450 INFO:tasks.workunit.client.1.smithi178.stdout:0/287: mknod d0/dc/d39/d4c/c5d 0 2024-09-10T11:29:25.455 INFO:tasks.workunit.client.1.smithi178.stdout:0/288: mkdir d0/dc/d39/d50/d5e 0 2024-09-10T11:29:25.455 INFO:tasks.workunit.client.1.smithi178.stdout:8/274: dwrite d5/f8 [0,4194304] 0 2024-09-10T11:29:25.457 INFO:tasks.workunit.client.1.smithi178.stdout:5/302: dwrite d9/f28 [0,4194304] 0 2024-09-10T11:29:25.459 INFO:tasks.workunit.client.1.smithi178.stdout:8/275: fsync d5/db/d10/d35/f5a 0 2024-09-10T11:29:25.464 INFO:tasks.workunit.client.1.smithi178.stdout:8/276: rename d5/db/d1e/l51 to d5/d1d/l5b 0 2024-09-10T11:29:25.469 INFO:tasks.workunit.client.1.smithi178.stdout:8/277: write d5/db/d10/d20/f2c [778677,85075] 0 2024-09-10T11:29:25.470 INFO:tasks.workunit.client.1.smithi178.stdout:5/303: dwrite d9/dc/dd/f69 [0,4194304] 0 2024-09-10T11:29:25.471 INFO:tasks.workunit.client.1.smithi178.stdout:0/289: dread d0/dc/d39/d4c/f15 [0,4194304] 0 2024-09-10T11:29:25.472 INFO:tasks.workunit.client.1.smithi178.stdout:0/290: write d0/d12/f2f [1991428,119646] 0 2024-09-10T11:29:25.473 INFO:tasks.workunit.client.1.smithi178.stdout:5/304: truncate f2 8872990 0 2024-09-10T11:29:25.473 INFO:tasks.workunit.client.1.smithi178.stdout:0/291: dread - d0/d12/f51 zero size 2024-09-10T11:29:25.474 INFO:tasks.workunit.client.1.smithi178.stdout:5/305: chown d9/d11 4 1 2024-09-10T11:29:25.475 INFO:tasks.workunit.client.1.smithi178.stdout:8/278: write d5/d30/f4b [1140668,30873] 0 2024-09-10T11:29:25.477 INFO:tasks.workunit.client.1.smithi178.stdout:0/292: write d0/dc/d39/d50/f1d [2649191,124592] 0 2024-09-10T11:29:25.478 INFO:tasks.workunit.client.1.smithi178.stdout:8/279: fdatasync d5/d30/d33/f49 0 2024-09-10T11:29:25.480 INFO:tasks.workunit.client.1.smithi178.stdout:5/306: getdents d9/dc 0 2024-09-10T11:29:25.481 INFO:tasks.workunit.client.1.smithi178.stdout:5/307: stat d9/dc/dd/d37 0 2024-09-10T11:29:25.482 INFO:tasks.workunit.client.1.smithi178.stdout:8/280: mkdir d5/db/d10/d20/d52/d5c 0 2024-09-10T11:29:25.482 INFO:tasks.workunit.client.1.smithi178.stdout:0/293: fsync d0/f22 0 2024-09-10T11:29:25.483 INFO:tasks.workunit.client.1.smithi178.stdout:8/281: chown d5/d30/d33/l4d 57 1 2024-09-10T11:29:25.484 INFO:tasks.workunit.client.1.smithi178.stdout:5/308: creat d9/dc/dd/d16/f70 x:0 0 0 2024-09-10T11:29:25.486 INFO:tasks.workunit.client.1.smithi178.stdout:0/294: creat d0/dc/d39/d4c/d43/f5f x:0 0 0 2024-09-10T11:29:25.486 INFO:tasks.workunit.client.1.smithi178.stdout:5/309: dread d9/dc/d17/f34 [0,4194304] 0 2024-09-10T11:29:25.487 INFO:tasks.workunit.client.1.smithi178.stdout:5/310: stat d9/dc/dd/d37/d42/f4e 0 2024-09-10T11:29:25.488 INFO:tasks.workunit.client.1.smithi178.stdout:8/282: mknod d5/db/d3d/c5d 0 2024-09-10T11:29:25.489 INFO:tasks.workunit.client.1.smithi178.stdout:8/283: fdatasync d5/d1d/f50 0 2024-09-10T11:29:25.491 INFO:tasks.workunit.client.1.smithi178.stdout:5/311: write d9/dc/d17/f34 [1554823,15368] 0 2024-09-10T11:29:25.492 INFO:tasks.workunit.client.1.smithi178.stdout:8/284: creat d5/db/d1e/f5e x:0 0 0 2024-09-10T11:29:25.492 INFO:tasks.workunit.client.1.smithi178.stdout:0/295: link d0/ca d0/dc/d39/c60 0 2024-09-10T11:29:25.493 INFO:tasks.workunit.client.1.smithi178.stdout:0/296: stat d0/dc/d39/d50 0 2024-09-10T11:29:25.494 INFO:tasks.workunit.client.1.smithi178.stdout:5/312: creat d9/f71 x:0 0 0 2024-09-10T11:29:25.496 INFO:tasks.workunit.client.1.smithi178.stdout:8/285: link d5/db/d10/d35/f3c d5/db/d3d/f5f 0 2024-09-10T11:29:25.498 INFO:tasks.workunit.client.1.smithi178.stdout:5/313: rename d9/d11/l36 to d9/dc/l72 0 2024-09-10T11:29:25.500 INFO:tasks.workunit.client.1.smithi178.stdout:0/297: creat d0/dc/f61 x:0 0 0 2024-09-10T11:29:25.501 INFO:tasks.workunit.client.1.smithi178.stdout:8/286: rename d5/d1d/d22/f23 to d5/d1d/f60 0 2024-09-10T11:29:25.505 INFO:tasks.workunit.client.1.smithi178.stdout:8/287: unlink d5/d1d/l5b 0 2024-09-10T11:29:25.507 INFO:tasks.workunit.client.1.smithi178.stdout:0/298: getdents d0/dc/d1f 0 2024-09-10T11:29:25.508 INFO:tasks.workunit.client.1.smithi178.stdout:0/299: chown d0/d2c/d33/c48 0 1 2024-09-10T11:29:25.509 INFO:tasks.workunit.client.1.smithi178.stdout:8/288: mkdir d5/d61 0 2024-09-10T11:29:25.510 INFO:tasks.workunit.client.1.smithi178.stdout:0/300: mknod d0/dc/d1f/d2d/c62 0 2024-09-10T11:29:25.512 INFO:tasks.workunit.client.1.smithi178.stdout:8/289: truncate d5/db/fe 1084727 0 2024-09-10T11:29:25.513 INFO:tasks.workunit.client.1.smithi178.stdout:0/301: symlink d0/dc/d39/l63 0 2024-09-10T11:29:25.514 INFO:tasks.workunit.client.1.smithi178.stdout:0/302: write d0/d12/f34 [229288,46381] 0 2024-09-10T11:29:25.514 INFO:tasks.workunit.client.1.smithi178.stdout:8/290: creat d5/db/d1e/f62 x:0 0 0 2024-09-10T11:29:25.515 INFO:tasks.workunit.client.1.smithi178.stdout:5/314: link d9/d19/f67 d9/dc/d17/d1d/f73 0 2024-09-10T11:29:25.515 INFO:tasks.workunit.client.1.smithi178.stdout:8/291: write d5/db/d1e/f62 [369727,88215] 0 2024-09-10T11:29:25.516 INFO:tasks.workunit.client.1.smithi178.stdout:0/303: write d0/dc/d1f/f26 [2143884,27215] 0 2024-09-10T11:29:25.520 INFO:tasks.workunit.client.1.smithi178.stdout:8/292: symlink d5/d61/l63 0 2024-09-10T11:29:25.520 INFO:tasks.workunit.client.1.smithi178.stdout:8/293: stat d5/db/cd 0 2024-09-10T11:29:25.521 INFO:tasks.workunit.client.1.smithi178.stdout:0/304: creat d0/d12/f64 x:0 0 0 2024-09-10T11:29:25.522 INFO:tasks.workunit.client.1.smithi178.stdout:8/294: read d5/f7 [683195,122775] 0 2024-09-10T11:29:25.523 INFO:tasks.workunit.client.1.smithi178.stdout:0/305: creat d0/d2c/f65 x:0 0 0 2024-09-10T11:29:25.523 INFO:tasks.workunit.client.1.smithi178.stdout:0/306: readlink d0/dc/d39/d50/l35 0 2024-09-10T11:29:25.525 INFO:tasks.workunit.client.1.smithi178.stdout:0/307: chown d0/d2c/c3d 1 1 2024-09-10T11:29:25.526 INFO:tasks.workunit.client.1.smithi178.stdout:8/295: fdatasync d5/f13 0 2024-09-10T11:29:25.526 INFO:tasks.workunit.client.1.smithi178.stdout:0/308: truncate d0/dc/d39/f5b 749925 0 2024-09-10T11:29:25.529 INFO:tasks.workunit.client.1.smithi178.stdout:5/315: truncate f4 4971626 0 2024-09-10T11:29:25.530 INFO:tasks.workunit.client.1.smithi178.stdout:8/296: creat d5/db/d10/d53/f64 x:0 0 0 2024-09-10T11:29:25.531 INFO:tasks.workunit.client.1.smithi178.stdout:8/297: write d5/d1d/f25 [83061,126764] 0 2024-09-10T11:29:25.535 INFO:tasks.workunit.client.1.smithi178.stdout:5/316: dread d9/f32 [4194304,4194304] 0 2024-09-10T11:29:25.542 INFO:tasks.workunit.client.1.smithi178.stdout:5/317: dread d9/dc/dd/d37/d42/f4e [0,4194304] 0 2024-09-10T11:29:25.545 INFO:tasks.workunit.client.1.smithi178.stdout:5/318: creat d9/dc/dd/d37/d42/d55/f74 x:0 0 0 2024-09-10T11:29:25.545 INFO:tasks.workunit.client.1.smithi178.stdout:8/298: dread d5/d30/d33/f49 [0,4194304] 0 2024-09-10T11:29:25.546 INFO:tasks.workunit.client.1.smithi178.stdout:8/299: dread - d5/d1d/f50 zero size 2024-09-10T11:29:25.547 INFO:tasks.workunit.client.1.smithi178.stdout:5/319: creat d9/dc/dd/d37/f75 x:0 0 0 2024-09-10T11:29:25.548 INFO:tasks.workunit.client.1.smithi178.stdout:5/320: readlink d9/dc/d17/d1d/l4f 0 2024-09-10T11:29:25.549 INFO:tasks.workunit.client.1.smithi178.stdout:8/300: link d5/d30/d33/c3e d5/db/d3d/c65 0 2024-09-10T11:29:25.549 INFO:tasks.workunit.client.1.smithi178.stdout:8/301: readlink d5/db/d10/l2a 0 2024-09-10T11:29:25.550 INFO:tasks.workunit.client.1.smithi178.stdout:5/321: rename d9/lb to d9/dc/d17/d1d/l76 0 2024-09-10T11:29:25.551 INFO:tasks.workunit.client.1.smithi178.stdout:8/302: symlink d5/db/d3d/l66 0 2024-09-10T11:29:25.552 INFO:tasks.workunit.client.1.smithi178.stdout:8/303: dread - d5/db/d10/d35/f5a zero size 2024-09-10T11:29:25.554 INFO:tasks.workunit.client.1.smithi178.stdout:8/304: mkdir d5/db/d10/d20/d52/d67 0 2024-09-10T11:29:25.555 INFO:tasks.workunit.client.1.smithi178.stdout:5/322: link d9/d19/c33 d9/c77 0 2024-09-10T11:29:25.555 INFO:tasks.workunit.client.0.smithi144.stdout:1/5: getdents . 0 2024-09-10T11:29:25.555 INFO:tasks.workunit.client.0.smithi144.stdout:1/6: chown . 4 1 2024-09-10T11:29:25.556 INFO:tasks.workunit.client.0.smithi144.stdout:1/7: creat f1 x:0 0 0 2024-09-10T11:29:25.557 INFO:tasks.workunit.client.0.smithi144.stdout:1/8: truncate f1 702612 0 2024-09-10T11:29:25.557 INFO:tasks.workunit.client.1.smithi178.stdout:5/323: dread d9/dc/d17/f44 [0,4194304] 0 2024-09-10T11:29:25.559 INFO:tasks.workunit.client.0.smithi144.stdout:1/9: rename f1 to f2 0 2024-09-10T11:29:25.562 INFO:tasks.workunit.client.0.smithi144.stdout:1/10: write f2 [817558,112722] 0 2024-09-10T11:29:25.563 INFO:tasks.workunit.client.1.smithi178.stdout:8/305: write d5/f7 [1215383,74576] 0 2024-09-10T11:29:25.563 INFO:tasks.workunit.client.1.smithi178.stdout:5/324: write d9/dc/dd/f2d [114002,94611] 0 2024-09-10T11:29:25.564 INFO:tasks.workunit.client.0.smithi144.stdout:0/2: truncate f0 653825 0 2024-09-10T11:29:25.564 INFO:tasks.workunit.client.1.smithi178.stdout:8/306: chown d5/db/d10/d53 39420460 1 2024-09-10T11:29:25.565 INFO:tasks.workunit.client.0.smithi144.stdout:1/11: creat f3 x:0 0 0 2024-09-10T11:29:25.565 INFO:tasks.workunit.client.1.smithi178.stdout:8/307: chown d5/db/d10/d20/f3b 0 1 2024-09-10T11:29:25.567 INFO:tasks.workunit.client.0.smithi144.stdout:0/3: creat f1 x:0 0 0 2024-09-10T11:29:25.569 INFO:tasks.workunit.client.0.smithi144.stdout:1/12: rename f3 to f4 0 2024-09-10T11:29:25.569 INFO:tasks.workunit.client.0.smithi144.stdout:1/13: stat f4 0 2024-09-10T11:29:25.570 INFO:tasks.workunit.client.0.smithi144.stdout:0/4: creat f2 x:0 0 0 2024-09-10T11:29:25.572 INFO:tasks.workunit.client.0.smithi144.stdout:1/14: mkdir d5 0 2024-09-10T11:29:25.573 INFO:tasks.workunit.client.0.smithi144.stdout:0/5: rename f1 to f3 0 2024-09-10T11:29:25.575 INFO:tasks.workunit.client.1.smithi178.stdout:5/325: dwrite d9/d11/f14 [0,4194304] 0 2024-09-10T11:29:25.576 INFO:tasks.workunit.client.0.smithi144.stdout:0/6: fdatasync f0 0 2024-09-10T11:29:25.577 INFO:tasks.workunit.client.0.smithi144.stdout:1/15: creat d5/f6 x:0 0 0 2024-09-10T11:29:25.579 INFO:tasks.workunit.client.0.smithi144.stdout:0/7: write f3 [767079,126238] 0 2024-09-10T11:29:25.581 INFO:tasks.workunit.client.0.smithi144.stdout:1/16: creat d5/f7 x:0 0 0 2024-09-10T11:29:25.586 INFO:tasks.workunit.client.0.smithi144.stdout:0/8: dwrite f2 [0,4194304] 0 2024-09-10T11:29:25.587 INFO:tasks.workunit.client.0.smithi144.stdout:4/16: unlink f4 0 2024-09-10T11:29:25.588 INFO:tasks.workunit.client.0.smithi144.stdout:0/9: fdatasync f0 0 2024-09-10T11:29:25.589 INFO:tasks.workunit.client.1.smithi178.stdout:5/326: dwrite f4 [0,4194304] 0 2024-09-10T11:29:25.590 INFO:tasks.workunit.client.0.smithi144.stdout:1/17: dwrite d5/f6 [0,4194304] 0 2024-09-10T11:29:25.591 INFO:tasks.workunit.client.0.smithi144.stdout:8/4: getdents d0 0 2024-09-10T11:29:25.592 INFO:tasks.workunit.client.0.smithi144.stdout:1/18: truncate d5/f7 127084 0 2024-09-10T11:29:25.594 INFO:tasks.workunit.client.0.smithi144.stdout:8/5: mknod d0/c2 0 2024-09-10T11:29:25.598 INFO:tasks.workunit.client.1.smithi178.stdout:5/327: dwrite d9/dc/dd/d37/f75 [0,4194304] 0 2024-09-10T11:29:25.601 INFO:tasks.workunit.client.1.smithi178.stdout:5/328: rmdir d9/dc/dd/d37/d42 39 2024-09-10T11:29:25.606 INFO:tasks.workunit.client.1.smithi178.stdout:5/329: creat d9/dc/dd/d37/d3e/f78 x:0 0 0 2024-09-10T11:29:25.607 INFO:tasks.workunit.client.1.smithi178.stdout:5/330: fdatasync d9/dc/dd/f69 0 2024-09-10T11:29:25.609 INFO:tasks.workunit.client.0.smithi144.stdout:6/25: symlink d9/dc/le 0 2024-09-10T11:29:25.609 INFO:tasks.workunit.client.0.smithi144.stdout:6/26: chown d9/dd 319761001 1 2024-09-10T11:29:25.610 INFO:tasks.workunit.client.1.smithi178.stdout:5/331: unlink d9/d11/c60 0 2024-09-10T11:29:25.614 INFO:tasks.workunit.client.0.smithi144.stdout:6/27: link d9/lb d9/dc/lf 0 2024-09-10T11:29:25.614 INFO:tasks.workunit.client.0.smithi144.stdout:6/28: stat c4 0 2024-09-10T11:29:25.615 INFO:tasks.workunit.client.0.smithi144.stdout:1/19: fsync f4 0 2024-09-10T11:29:25.616 INFO:tasks.workunit.client.1.smithi178.stdout:5/332: symlink d9/dc/dd/d37/d3e/l79 0 2024-09-10T11:29:25.618 INFO:tasks.workunit.client.0.smithi144.stdout:1/20: truncate d5/f7 694258 0 2024-09-10T11:29:25.620 INFO:tasks.workunit.client.1.smithi178.stdout:5/333: getdents d9/dc/dd/d37/d42/d62 0 2024-09-10T11:29:25.622 INFO:tasks.workunit.client.0.smithi144.stdout:0/10: dwrite f0 [0,4194304] 0 2024-09-10T11:29:25.623 INFO:tasks.workunit.client.0.smithi144.stdout:1/21: link d5/f6 d5/f8 0 2024-09-10T11:29:25.624 INFO:tasks.workunit.client.1.smithi178.stdout:5/334: dread d9/dc/d17/f3c [0,4194304] 0 2024-09-10T11:29:25.625 INFO:tasks.workunit.client.0.smithi144.stdout:6/29: dwrite f6 [0,4194304] 0 2024-09-10T11:29:25.625 INFO:tasks.workunit.client.0.smithi144.stdout:0/11: creat f4 x:0 0 0 2024-09-10T11:29:25.627 INFO:tasks.workunit.client.0.smithi144.stdout:0/12: write f2 [3592600,19847] 0 2024-09-10T11:29:25.627 INFO:tasks.workunit.client.1.smithi178.stdout:5/335: readlink d9/dc/d17/d1d/l76 0 2024-09-10T11:29:25.628 INFO:tasks.workunit.client.1.smithi178.stdout:5/336: chown d9/dc/dd/d37/d42/f5f 77501 1 2024-09-10T11:29:25.630 INFO:tasks.workunit.client.1.smithi178.stdout:7/203: sync 2024-09-10T11:29:25.632 INFO:tasks.workunit.client.0.smithi144.stdout:6/30: rename c1 to d9/dd/c10 0 2024-09-10T11:29:25.633 INFO:tasks.workunit.client.0.smithi144.stdout:6/31: write f6 [3699045,41982] 0 2024-09-10T11:29:25.634 INFO:tasks.workunit.client.0.smithi144.stdout:6/32: write f6 [589737,60900] 0 2024-09-10T11:29:25.635 INFO:tasks.workunit.client.0.smithi144.stdout:0/13: dread f2 [0,4194304] 0 2024-09-10T11:29:25.636 INFO:tasks.workunit.client.0.smithi144.stdout:6/33: write f6 [1006625,16057] 0 2024-09-10T11:29:25.638 INFO:tasks.workunit.client.0.smithi144.stdout:0/14: creat f5 x:0 0 0 2024-09-10T11:29:25.639 INFO:tasks.workunit.client.0.smithi144.stdout:0/15: read f2 [30789,89606] 0 2024-09-10T11:29:25.639 INFO:tasks.workunit.client.0.smithi144.stdout:0/16: rmdir - no directory 2024-09-10T11:29:25.641 INFO:tasks.workunit.client.0.smithi144.stdout:0/17: dread - f4 zero size 2024-09-10T11:29:25.643 INFO:tasks.workunit.client.1.smithi178.stdout:7/204: dread d6/d10/d1e/f35 [0,4194304] 0 2024-09-10T11:29:25.644 INFO:tasks.workunit.client.0.smithi144.stdout:6/34: dread f8 [0,4194304] 0 2024-09-10T11:29:25.645 INFO:tasks.workunit.client.0.smithi144.stdout:1/22: read f2 [465143,78262] 0 2024-09-10T11:29:25.647 INFO:tasks.workunit.client.0.smithi144.stdout:6/35: mkdir d9/dd/d11 0 2024-09-10T11:29:25.648 INFO:tasks.workunit.client.0.smithi144.stdout:1/23: symlink d5/l9 0 2024-09-10T11:29:25.648 INFO:tasks.workunit.client.1.smithi178.stdout:7/205: symlink d6/d10/d1e/d2d/d2b/l48 0 2024-09-10T11:29:25.649 INFO:tasks.workunit.client.0.smithi144.stdout:1/24: stat f2 0 2024-09-10T11:29:25.649 INFO:tasks.workunit.client.0.smithi144.stdout:6/36: dread f8 [0,4194304] 0 2024-09-10T11:29:25.650 INFO:tasks.workunit.client.0.smithi144.stdout:1/25: chown f4 91 1 2024-09-10T11:29:25.650 INFO:tasks.workunit.client.0.smithi144.stdout:1/26: chown d5/f6 3 1 2024-09-10T11:29:25.652 INFO:tasks.workunit.client.0.smithi144.stdout:1/27: symlink d5/la 0 2024-09-10T11:29:25.653 INFO:tasks.workunit.client.1.smithi178.stdout:5/337: getdents d9/dc/dd 0 2024-09-10T11:29:25.653 INFO:tasks.workunit.client.0.smithi144.stdout:1/28: readlink d5/la 0 2024-09-10T11:29:25.654 INFO:tasks.workunit.client.1.smithi178.stdout:7/206: mkdir d6/d10/d1e/d24/d49 0 2024-09-10T11:29:25.656 INFO:tasks.workunit.client.0.smithi144.stdout:6/37: link c7 d9/dd/d11/c12 0 2024-09-10T11:29:25.659 INFO:tasks.workunit.client.0.smithi144.stdout:6/38: mkdir d9/dc/d13 0 2024-09-10T11:29:25.660 INFO:tasks.workunit.client.1.smithi178.stdout:8/308: read d5/db/d10/d20/f40 [2270757,124658] 0 2024-09-10T11:29:25.661 INFO:tasks.workunit.client.1.smithi178.stdout:7/207: mkdir d6/d10/d1d/d4a 0 2024-09-10T11:29:25.662 INFO:tasks.workunit.client.0.smithi144.stdout:6/39: rename d9/dd/d11 to d9/dd/d11/d14 22 2024-09-10T11:29:25.664 INFO:tasks.workunit.client.0.smithi144.stdout:1/29: dwrite d5/f6 [0,4194304] 0 2024-09-10T11:29:25.664 INFO:tasks.workunit.client.1.smithi178.stdout:7/208: symlink d6/d10/d1d/d22/l4b 0 2024-09-10T11:29:25.665 INFO:tasks.workunit.client.1.smithi178.stdout:7/209: read d6/ff [5921362,1485] 0 2024-09-10T11:29:25.666 INFO:tasks.workunit.client.0.smithi144.stdout:6/40: mknod d9/dc/d13/c15 0 2024-09-10T11:29:25.669 INFO:tasks.workunit.client.1.smithi178.stdout:7/210: write d6/d29/f3c [632251,14297] 0 2024-09-10T11:29:25.671 INFO:tasks.workunit.client.1.smithi178.stdout:5/338: dwrite d9/dc/d17/d1d/f73 [0,4194304] 0 2024-09-10T11:29:25.672 INFO:tasks.workunit.client.1.smithi178.stdout:7/211: write d6/d10/d1e/d24/f36 [433443,8379] 0 2024-09-10T11:29:25.674 INFO:tasks.workunit.client.1.smithi178.stdout:8/309: dwrite d5/db/d3d/f5f [0,4194304] 0 2024-09-10T11:29:25.674 INFO:tasks.workunit.client.1.smithi178.stdout:7/212: chown d6/d10/l33 0 1 2024-09-10T11:29:25.675 INFO:tasks.workunit.client.0.smithi144.stdout:1/30: dwrite d5/f6 [4194304,4194304] 0 2024-09-10T11:29:25.676 INFO:tasks.workunit.client.1.smithi178.stdout:5/339: creat d9/dc/dd/d37/d42/d55/f7a x:0 0 0 2024-09-10T11:29:25.678 INFO:tasks.workunit.client.0.smithi144.stdout:6/41: dwrite f6 [0,4194304] 0 2024-09-10T11:29:25.678 INFO:tasks.workunit.client.1.smithi178.stdout:7/213: rmdir d6/d10/d1d/d22 39 2024-09-10T11:29:25.680 INFO:tasks.workunit.client.1.smithi178.stdout:5/340: mknod d9/d11/c7b 0 2024-09-10T11:29:25.680 INFO:tasks.workunit.client.1.smithi178.stdout:7/214: readlink d6/l1f 0 2024-09-10T11:29:25.683 INFO:tasks.workunit.client.0.smithi144.stdout:6/42: creat d9/dc/f16 x:0 0 0 2024-09-10T11:29:25.684 INFO:tasks.workunit.client.1.smithi178.stdout:2/478: dread d1/d7/d24/d7d/d84/f36 [0,4194304] 0 2024-09-10T11:29:25.686 INFO:tasks.workunit.client.1.smithi178.stdout:7/215: mkdir d6/d29/d4c 0 2024-09-10T11:29:25.686 INFO:tasks.workunit.client.1.smithi178.stdout:8/310: dwrite d5/d30/d33/f49 [0,4194304] 0 2024-09-10T11:29:25.688 INFO:tasks.workunit.client.1.smithi178.stdout:8/311: truncate d5/d1d/f25 799929 0 2024-09-10T11:29:25.689 INFO:tasks.workunit.client.1.smithi178.stdout:8/312: write d5/f37 [839791,47232] 0 2024-09-10T11:29:25.689 INFO:tasks.workunit.client.1.smithi178.stdout:5/341: symlink d9/dc/dd/l7c 0 2024-09-10T11:29:25.690 INFO:tasks.workunit.client.0.smithi144.stdout:6/43: dread f6 [0,4194304] 0 2024-09-10T11:29:25.692 INFO:tasks.workunit.client.1.smithi178.stdout:7/216: rename d6/fc to d6/d10/d34/f4d 0 2024-09-10T11:29:25.694 INFO:tasks.workunit.client.1.smithi178.stdout:8/313: write d5/db/d3d/f5f [4050906,71997] 0 2024-09-10T11:29:25.694 INFO:tasks.workunit.client.1.smithi178.stdout:5/342: chown d9/d11/c7b 1266971954 1 2024-09-10T11:29:25.694 INFO:tasks.workunit.client.0.smithi144.stdout:6/44: rmdir d9/dc/d13 39 2024-09-10T11:29:25.696 INFO:tasks.workunit.client.0.smithi144.stdout:6/45: write f8 [670436,44570] 0 2024-09-10T11:29:25.697 INFO:tasks.workunit.client.1.smithi178.stdout:7/217: fsync d6/d10/d1e/f35 0 2024-09-10T11:29:25.700 INFO:tasks.workunit.client.0.smithi144.stdout:6/46: unlink d9/ca 0 2024-09-10T11:29:25.701 INFO:tasks.workunit.client.1.smithi178.stdout:8/314: rename d5/fa to d5/db/d10/d20/d52/f68 0 2024-09-10T11:29:25.702 INFO:tasks.workunit.client.1.smithi178.stdout:5/343: dread d9/d11/f14 [0,4194304] 0 2024-09-10T11:29:25.703 INFO:tasks.workunit.client.0.smithi144.stdout:6/47: creat d9/dd/d11/f17 x:0 0 0 2024-09-10T11:29:25.703 INFO:tasks.workunit.client.0.smithi144.stdout:6/48: chown d9/dd 538 1 2024-09-10T11:29:25.705 INFO:tasks.workunit.client.0.smithi144.stdout:6/49: readlink d9/lb 0 2024-09-10T11:29:25.705 INFO:tasks.workunit.client.1.smithi178.stdout:8/315: creat d5/f69 x:0 0 0 2024-09-10T11:29:25.706 INFO:tasks.workunit.client.1.smithi178.stdout:5/344: mkdir d9/dc/dd/d3f/d5a/d7d 0 2024-09-10T11:29:25.706 INFO:tasks.workunit.client.0.smithi144.stdout:6/50: write f6 [764035,94220] 0 2024-09-10T11:29:25.708 INFO:tasks.workunit.client.1.smithi178.stdout:5/345: unlink d9/dc/dd/d37/d3e/l79 0 2024-09-10T11:29:25.710 INFO:tasks.workunit.client.1.smithi178.stdout:8/316: dread d5/db/d3d/f5f [0,4194304] 0 2024-09-10T11:29:25.712 INFO:tasks.workunit.client.1.smithi178.stdout:8/317: creat d5/d30/f6a x:0 0 0 2024-09-10T11:29:25.713 INFO:tasks.workunit.client.1.smithi178.stdout:5/346: mknod d9/dc/dd/d37/d3e/c7e 0 2024-09-10T11:29:25.714 INFO:tasks.workunit.client.1.smithi178.stdout:7/218: truncate d6/f23 447612 0 2024-09-10T11:29:25.715 INFO:tasks.workunit.client.1.smithi178.stdout:8/318: mknod d5/d30/d33/d3a/c6b 0 2024-09-10T11:29:25.716 INFO:tasks.workunit.client.1.smithi178.stdout:5/347: getdents d9/dc/dd/d3f/d5a/d7d 0 2024-09-10T11:29:25.717 INFO:tasks.workunit.client.1.smithi178.stdout:5/348: chown d9/dc/d17/d1d 459494 1 2024-09-10T11:29:25.721 INFO:tasks.workunit.client.1.smithi178.stdout:7/219: dwrite d6/fd [0,4194304] 0 2024-09-10T11:29:25.723 INFO:tasks.workunit.client.1.smithi178.stdout:7/220: readlink d6/d10/d34/l40 0 2024-09-10T11:29:25.728 INFO:tasks.workunit.client.0.smithi144.stdout:0/18: getdents . 0 2024-09-10T11:29:25.729 INFO:tasks.workunit.client.1.smithi178.stdout:5/349: dwrite d9/dc/d17/f3d [0,4194304] 0 2024-09-10T11:29:25.729 INFO:tasks.workunit.client.1.smithi178.stdout:7/221: write d6/d10/d1e/f35 [1123343,38441] 0 2024-09-10T11:29:25.735 INFO:tasks.workunit.client.0.smithi144.stdout:0/19: dwrite f3 [0,4194304] 0 2024-09-10T11:29:25.736 INFO:tasks.workunit.client.1.smithi178.stdout:3/255: sync 2024-09-10T11:29:25.736 INFO:tasks.workunit.client.1.smithi178.stdout:1/195: sync 2024-09-10T11:29:25.739 INFO:tasks.workunit.client.1.smithi178.stdout:8/319: dread d5/ff [0,4194304] 0 2024-09-10T11:29:25.741 INFO:tasks.workunit.client.0.smithi144.stdout:0/20: mknod c6 0 2024-09-10T11:29:25.741 INFO:tasks.workunit.client.0.smithi144.stdout:0/21: readlink - no filename 2024-09-10T11:29:25.741 INFO:tasks.workunit.client.0.smithi144.stdout:0/22: stat c6 0 2024-09-10T11:29:25.741 INFO:tasks.workunit.client.0.smithi144.stdout:0/23: rmdir - no directory 2024-09-10T11:29:25.742 INFO:tasks.workunit.client.1.smithi178.stdout:8/320: dread - d5/db/d1e/f39 zero size 2024-09-10T11:29:25.744 INFO:tasks.workunit.client.1.smithi178.stdout:2/479: dread d1/d7/f16 [0,4194304] 0 2024-09-10T11:29:25.744 INFO:tasks.workunit.client.1.smithi178.stdout:7/222: getdents d6/d10/d1d/d4a 0 2024-09-10T11:29:25.744 INFO:tasks.workunit.client.0.smithi144.stdout:1/31: truncate d5/f6 1393376 0 2024-09-10T11:29:25.747 INFO:tasks.workunit.client.0.smithi144.stdout:1/32: dread d5/f7 [0,4194304] 0 2024-09-10T11:29:25.748 INFO:tasks.workunit.client.0.smithi144.stdout:0/24: dwrite f0 [0,4194304] 0 2024-09-10T11:29:25.749 INFO:tasks.workunit.client.0.smithi144.stdout:1/33: truncate f4 542010 0 2024-09-10T11:29:25.750 INFO:tasks.workunit.client.1.smithi178.stdout:7/223: creat d6/d10/d34/f4e x:0 0 0 2024-09-10T11:29:25.750 INFO:tasks.workunit.client.1.smithi178.stdout:5/350: dread d9/dc/d17/f3d [0,4194304] 0 2024-09-10T11:29:25.750 INFO:tasks.workunit.client.1.smithi178.stdout:7/224: fsync d6/d10/d1d/f31 0 2024-09-10T11:29:25.752 INFO:tasks.workunit.client.1.smithi178.stdout:1/196: link d3/d9/fd d3/d13/f3f 0 2024-09-10T11:29:25.755 INFO:tasks.workunit.client.0.smithi144.stdout:0/25: dread f3 [0,4194304] 0 2024-09-10T11:29:25.757 INFO:tasks.workunit.client.1.smithi178.stdout:3/256: dread db/fe [0,4194304] 0 2024-09-10T11:29:25.757 INFO:tasks.workunit.client.1.smithi178.stdout:5/351: unlink d9/ff 0 2024-09-10T11:29:25.759 INFO:tasks.workunit.client.0.smithi144.stdout:0/26: write f2 [3514820,90179] 0 2024-09-10T11:29:25.759 INFO:tasks.workunit.client.0.smithi144.stdout:0/27: stat f3 0 2024-09-10T11:29:25.760 INFO:tasks.workunit.client.1.smithi178.stdout:5/352: chown d9/d19/f1a 2 1 2024-09-10T11:29:25.761 INFO:tasks.workunit.client.1.smithi178.stdout:5/353: truncate d9/f28 5104551 0 2024-09-10T11:29:25.761 INFO:tasks.workunit.client.1.smithi178.stdout:7/225: fdatasync d6/d10/d1d/d22/f3a 0 2024-09-10T11:29:25.762 INFO:tasks.workunit.client.1.smithi178.stdout:8/321: getdents d5/db/d10/d53 0 2024-09-10T11:29:25.762 INFO:tasks.workunit.client.0.smithi144.stdout:1/34: dwrite d5/f7 [0,4194304] 0 2024-09-10T11:29:25.763 INFO:tasks.workunit.client.1.smithi178.stdout:2/480: dwrite d1/d7/d24/d7d/d84/d34/d5b/f6b [0,4194304] 0 2024-09-10T11:29:25.769 INFO:tasks.workunit.client.0.smithi144.stdout:1/35: rename f2 to d5/fb 0 2024-09-10T11:29:25.769 INFO:tasks.workunit.client.1.smithi178.stdout:5/354: chown d9/dc/dd/d37/d42/f5f 0 1 2024-09-10T11:29:25.770 INFO:tasks.workunit.client.1.smithi178.stdout:3/257: dwrite db/f16 [4194304,4194304] 0 2024-09-10T11:29:25.770 INFO:tasks.workunit.client.0.smithi144.stdout:0/28: dwrite f2 [0,4194304] 0 2024-09-10T11:29:25.770 INFO:tasks.workunit.client.0.smithi144.stdout:1/36: chown d5/f8 22681754 1 2024-09-10T11:29:25.770 INFO:tasks.workunit.client.1.smithi178.stdout:5/355: read d9/dc/dd/d37/d42/f4e [4009518,59324] 0 2024-09-10T11:29:25.771 INFO:tasks.workunit.client.1.smithi178.stdout:3/258: chown db/ff 115058 1 2024-09-10T11:29:25.772 INFO:tasks.workunit.client.1.smithi178.stdout:8/322: creat d5/d30/d33/d3a/f6c x:0 0 0 2024-09-10T11:29:25.773 INFO:tasks.workunit.client.1.smithi178.stdout:3/259: truncate db/f15 881787 0 2024-09-10T11:29:25.774 INFO:tasks.workunit.client.0.smithi144.stdout:1/37: creat d5/fc x:0 0 0 2024-09-10T11:29:25.775 INFO:tasks.workunit.client.1.smithi178.stdout:8/323: chown d5/db/d10/d20/d52/f68 48523 1 2024-09-10T11:29:25.776 INFO:tasks.workunit.client.0.smithi144.stdout:1/38: write d5/f7 [5189615,26477] 0 2024-09-10T11:29:25.779 INFO:tasks.workunit.client.0.smithi144.stdout:1/39: mkdir d5/dd 0 2024-09-10T11:29:25.783 INFO:tasks.workunit.client.1.smithi178.stdout:2/481: mkdir d1/d41/d8d/d9f 0 2024-09-10T11:29:25.784 INFO:tasks.workunit.client.0.smithi144.stdout:0/29: dwrite f3 [0,4194304] 0 2024-09-10T11:29:25.785 INFO:tasks.workunit.client.0.smithi144.stdout:1/40: link d5/f7 d5/fe 0 2024-09-10T11:29:25.786 INFO:tasks.workunit.client.1.smithi178.stdout:5/356: creat d9/dc/dd/f7f x:0 0 0 2024-09-10T11:29:25.787 INFO:tasks.workunit.client.0.smithi144.stdout:1/41: mkdir d5/df 0 2024-09-10T11:29:25.788 INFO:tasks.workunit.client.1.smithi178.stdout:7/226: symlink d6/d10/d1e/d24/d49/l4f 0 2024-09-10T11:29:25.790 INFO:tasks.workunit.client.1.smithi178.stdout:8/324: mknod d5/db/d3d/c6d 0 2024-09-10T11:29:25.791 INFO:tasks.workunit.client.1.smithi178.stdout:8/325: stat d5/db/d10/d20/f3b 0 2024-09-10T11:29:25.792 INFO:tasks.workunit.client.0.smithi144.stdout:1/42: creat d5/df/f10 x:0 0 0 2024-09-10T11:29:25.793 INFO:tasks.workunit.client.1.smithi178.stdout:2/482: creat d1/d7/d24/d7d/fa0 x:0 0 0 2024-09-10T11:29:25.794 INFO:tasks.workunit.client.1.smithi178.stdout:2/483: readlink d1/d41/d5d/l7e 0 2024-09-10T11:29:25.794 INFO:tasks.workunit.client.0.smithi144.stdout:0/30: dwrite f2 [0,4194304] 0 2024-09-10T11:29:25.795 INFO:tasks.workunit.client.0.smithi144.stdout:1/43: mknod d5/c11 0 2024-09-10T11:29:25.797 INFO:tasks.workunit.client.1.smithi178.stdout:5/357: unlink d9/dc/dd/d37/d42/d55/l5c 0 2024-09-10T11:29:25.797 INFO:tasks.workunit.client.1.smithi178.stdout:7/227: mkdir d6/d10/d1d/d22/d50 0 2024-09-10T11:29:25.797 INFO:tasks.workunit.client.0.smithi144.stdout:0/31: write f4 [420497,92372] 0 2024-09-10T11:29:25.799 INFO:tasks.workunit.client.1.smithi178.stdout:7/228: stat d6/d10/d1e/f26 0 2024-09-10T11:29:25.799 INFO:tasks.workunit.client.1.smithi178.stdout:5/358: read d9/dc/d17/f3d [1992395,62800] 0 2024-09-10T11:29:25.799 INFO:tasks.workunit.client.1.smithi178.stdout:2/484: chown d1/d41/d14/d11/f9 2138789313 1 2024-09-10T11:29:25.800 INFO:tasks.workunit.client.1.smithi178.stdout:8/326: symlink d5/db/l6e 0 2024-09-10T11:29:25.805 INFO:tasks.workunit.client.1.smithi178.stdout:8/327: symlink d5/db/d3d/l6f 0 2024-09-10T11:29:25.805 INFO:tasks.workunit.client.1.smithi178.stdout:7/229: mknod d6/d29/d4c/c51 0 2024-09-10T11:29:25.806 INFO:tasks.workunit.client.0.smithi144.stdout:1/44: rmdir d5 39 2024-09-10T11:29:25.806 INFO:tasks.workunit.client.0.smithi144.stdout:0/32: dwrite f4 [0,4194304] 0 2024-09-10T11:29:25.807 INFO:tasks.workunit.client.1.smithi178.stdout:7/230: dread - d6/d10/d1d/d22/f3a zero size 2024-09-10T11:29:25.809 INFO:tasks.workunit.client.0.smithi144.stdout:1/45: creat d5/f12 x:0 0 0 2024-09-10T11:29:25.809 INFO:tasks.workunit.client.0.smithi144.stdout:1/46: fsync f4 0 2024-09-10T11:29:25.845 INFO:tasks.workunit.client.0.smithi144.stdout:1/47: creat d5/dd/f13 x:0 0 0 2024-09-10T11:29:25.846 INFO:tasks.workunit.client.1.smithi178.stdout:3/260: truncate db/ff 1040575 0 2024-09-10T11:29:25.847 INFO:tasks.workunit.client.1.smithi178.stdout:7/231: symlink d6/l52 0 2024-09-10T11:29:25.848 INFO:tasks.workunit.client.0.smithi144.stdout:0/33: dwrite f0 [0,4194304] 0 2024-09-10T11:29:25.848 INFO:tasks.workunit.client.0.smithi144.stdout:1/48: write d5/f7 [3195227,2235] 0 2024-09-10T11:29:25.848 INFO:tasks.workunit.client.0.smithi144.stdout:1/49: chown f4 1331 1 2024-09-10T11:29:25.849 INFO:tasks.workunit.client.1.smithi178.stdout:3/261: mknod db/c4c 0 2024-09-10T11:29:25.850 INFO:tasks.workunit.client.1.smithi178.stdout:7/232: symlink d6/d10/d1d/d4a/l53 0 2024-09-10T11:29:25.851 INFO:tasks.workunit.client.0.smithi144.stdout:0/34: getdents . 0 2024-09-10T11:29:25.852 INFO:tasks.workunit.client.1.smithi178.stdout:3/262: dread db/fe [0,4194304] 0 2024-09-10T11:29:25.852 INFO:tasks.workunit.client.1.smithi178.stdout:3/263: chown db/d20/f28 2 1 2024-09-10T11:29:25.853 INFO:tasks.workunit.client.1.smithi178.stdout:7/233: symlink d6/d10/d1e/d2d/d2b/l54 0 2024-09-10T11:29:25.854 INFO:tasks.workunit.client.1.smithi178.stdout:7/234: write d6/d10/d1e/d24/f36 [656691,110987] 0 2024-09-10T11:29:25.857 INFO:tasks.workunit.client.0.smithi144.stdout:1/50: dwrite d5/fe [0,4194304] 0 2024-09-10T11:29:25.858 INFO:tasks.workunit.client.0.smithi144.stdout:1/51: chown f4 3 1 2024-09-10T11:29:25.858 INFO:tasks.workunit.client.1.smithi178.stdout:7/235: mkdir d6/d29/d4c/d55 0 2024-09-10T11:29:25.858 INFO:tasks.workunit.client.0.smithi144.stdout:0/35: dread f3 [0,4194304] 0 2024-09-10T11:29:25.859 INFO:tasks.workunit.client.1.smithi178.stdout:7/236: dread - d6/d10/d1d/d22/f3a zero size 2024-09-10T11:29:25.859 INFO:tasks.workunit.client.1.smithi178.stdout:7/237: dread - d6/d10/d34/f4e zero size 2024-09-10T11:29:25.861 INFO:tasks.workunit.client.0.smithi144.stdout:0/36: mknod c7 0 2024-09-10T11:29:25.862 INFO:tasks.workunit.client.0.smithi144.stdout:1/52: link d5/fb d5/f14 0 2024-09-10T11:29:25.863 INFO:tasks.workunit.client.0.smithi144.stdout:1/53: chown d5/fc 14328 1 2024-09-10T11:29:25.863 INFO:tasks.workunit.client.0.smithi144.stdout:0/37: symlink l8 0 2024-09-10T11:29:25.864 INFO:tasks.workunit.client.1.smithi178.stdout:7/238: dwrite d6/ff [4194304,4194304] 0 2024-09-10T11:29:25.865 INFO:tasks.workunit.client.0.smithi144.stdout:0/38: mknod c9 0 2024-09-10T11:29:25.865 INFO:tasks.workunit.client.1.smithi178.stdout:7/239: write d6/d10/d1e/d2d/f16 [308614,17027] 0 2024-09-10T11:29:25.869 INFO:tasks.workunit.client.0.smithi144.stdout:0/39: dread f3 [0,4194304] 0 2024-09-10T11:29:25.871 INFO:tasks.workunit.client.1.smithi178.stdout:7/240: dread d6/f30 [0,4194304] 0 2024-09-10T11:29:25.871 INFO:tasks.workunit.client.0.smithi144.stdout:0/40: mkdir da 0 2024-09-10T11:29:25.873 INFO:tasks.workunit.client.0.smithi144.stdout:0/41: symlink da/lb 0 2024-09-10T11:29:25.873 INFO:tasks.workunit.client.1.smithi178.stdout:7/241: mknod d6/d10/d1e/d24/d49/c56 0 2024-09-10T11:29:25.874 INFO:tasks.workunit.client.0.smithi144.stdout:0/42: truncate f5 187231 0 2024-09-10T11:29:25.874 INFO:tasks.workunit.client.0.smithi144.stdout:0/43: readlink l8 0 2024-09-10T11:29:25.874 INFO:tasks.workunit.client.0.smithi144.stdout:1/54: fsync d5/dd/f13 0 2024-09-10T11:29:25.875 INFO:tasks.workunit.client.0.smithi144.stdout:0/44: rmdir da 39 2024-09-10T11:29:25.876 INFO:tasks.workunit.client.1.smithi178.stdout:7/242: creat d6/d10/d1d/d4a/f57 x:0 0 0 2024-09-10T11:29:25.878 INFO:tasks.workunit.client.1.smithi178.stdout:7/243: symlink d6/d10/d1d/d22/d3b/l58 0 2024-09-10T11:29:25.880 INFO:tasks.workunit.client.0.smithi144.stdout:0/45: rename f3 to da/fc 0 2024-09-10T11:29:25.880 INFO:tasks.workunit.client.1.smithi178.stdout:7/244: creat d6/d10/d1d/d22/d50/f59 x:0 0 0 2024-09-10T11:29:25.881 INFO:tasks.workunit.client.1.smithi178.stdout:7/245: fdatasync d6/d10/d1e/f26 0 2024-09-10T11:29:25.883 INFO:tasks.workunit.client.1.smithi178.stdout:7/246: creat d6/d10/d1d/d22/d3b/f5a x:0 0 0 2024-09-10T11:29:25.884 INFO:tasks.workunit.client.0.smithi144.stdout:0/46: truncate da/fc 2642312 0 2024-09-10T11:29:25.890 INFO:tasks.workunit.client.0.smithi144.stdout:0/47: dwrite f5 [0,4194304] 0 2024-09-10T11:29:25.890 INFO:tasks.workunit.client.0.smithi144.stdout:0/48: write f4 [233989,22095] 0 2024-09-10T11:29:25.893 INFO:tasks.workunit.client.0.smithi144.stdout:0/49: mknod da/cd 0 2024-09-10T11:29:25.893 INFO:tasks.workunit.client.1.smithi178.stdout:6/389: sync 2024-09-10T11:29:25.893 INFO:tasks.workunit.client.1.smithi178.stdout:9/412: sync 2024-09-10T11:29:25.895 INFO:tasks.workunit.client.1.smithi178.stdout:6/390: mknod d3/d42/c8f 0 2024-09-10T11:29:25.897 INFO:tasks.workunit.client.1.smithi178.stdout:9/413: rename d2/df/d31/d4a/d68 to d2/d19/d1c/d77 0 2024-09-10T11:29:25.898 INFO:tasks.workunit.client.1.smithi178.stdout:9/414: chown d2/df/d31/d37/c52 98650486 1 2024-09-10T11:29:25.899 INFO:tasks.workunit.client.1.smithi178.stdout:9/415: write d2/d19/d1c/f22 [2023588,105204] 0 2024-09-10T11:29:25.916 INFO:tasks.workunit.client.1.smithi178.stdout:3/264: dwrite db/f1d [0,4194304] 0 2024-09-10T11:29:25.917 INFO:tasks.workunit.client.1.smithi178.stdout:7/247: truncate d6/fd 850223 0 2024-09-10T11:29:25.960 INFO:tasks.workunit.client.1.smithi178.stdout:7/248: read f4 [4056032,91909] 0 2024-09-10T11:29:25.961 INFO:tasks.workunit.client.1.smithi178.stdout:7/249: read d6/ff [7798110,107702] 0 2024-09-10T11:29:25.963 INFO:tasks.workunit.client.1.smithi178.stdout:7/250: mknod d6/d10/d1d/d22/c5b 0 2024-09-10T11:29:25.964 INFO:tasks.workunit.client.1.smithi178.stdout:9/416: dwrite d2/d19/d1c/f24 [4194304,4194304] 0 2024-09-10T11:29:25.970 INFO:tasks.workunit.client.1.smithi178.stdout:7/251: dread d6/d29/f3c [0,4194304] 0 2024-09-10T11:29:25.974 INFO:tasks.workunit.client.1.smithi178.stdout:9/417: dwrite d2/d8/f73 [0,4194304] 0 2024-09-10T11:29:25.976 INFO:tasks.workunit.client.1.smithi178.stdout:3/265: dread db/d20/f34 [0,4194304] 0 2024-09-10T11:29:25.977 INFO:tasks.workunit.client.1.smithi178.stdout:3/266: write db/d26/f3b [1114874,20471] 0 2024-09-10T11:29:25.978 INFO:tasks.workunit.client.1.smithi178.stdout:9/418: creat d2/d19/d5e/f78 x:0 0 0 2024-09-10T11:29:25.980 INFO:tasks.workunit.client.1.smithi178.stdout:7/252: dwrite d6/d10/d1d/d22/f3a [0,4194304] 0 2024-09-10T11:29:25.982 INFO:tasks.workunit.client.1.smithi178.stdout:7/253: rmdir d6/d10/d1d/d22 39 2024-09-10T11:29:26.005 INFO:tasks.workunit.client.1.smithi178.stdout:9/419: dread d2/d13/f3c [0,4194304] 0 2024-09-10T11:29:26.007 INFO:tasks.workunit.client.1.smithi178.stdout:9/420: unlink d2/d13/f2e 0 2024-09-10T11:29:26.008 INFO:tasks.workunit.client.1.smithi178.stdout:9/421: chown d2/dc/f2f 350598 1 2024-09-10T11:29:26.012 INFO:tasks.workunit.client.1.smithi178.stdout:9/422: rename d2/df/d31/d37/f70 to d2/d19/d1c/f79 0 2024-09-10T11:29:26.012 INFO:tasks.workunit.client.1.smithi178.stdout:9/423: readlink d2/l53 0 2024-09-10T11:29:26.014 INFO:tasks.workunit.client.1.smithi178.stdout:9/424: creat d2/d19/d5e/f7a x:0 0 0 2024-09-10T11:29:26.016 INFO:tasks.workunit.client.1.smithi178.stdout:9/425: truncate d2/df/d31/d37/d3d/f6e 54774 0 2024-09-10T11:29:26.020 INFO:tasks.workunit.client.1.smithi178.stdout:4/202: sync 2024-09-10T11:29:26.020 INFO:tasks.workunit.client.1.smithi178.stdout:0/309: sync 2024-09-10T11:29:26.021 INFO:tasks.workunit.client.1.smithi178.stdout:9/426: link d2/d13/f1f d2/df/d31/d5a/f7b 0 2024-09-10T11:29:26.021 INFO:tasks.workunit.client.1.smithi178.stdout:0/310: fsync d0/dc/d1f/d2d/f42 0 2024-09-10T11:29:26.021 INFO:tasks.workunit.client.1.smithi178.stdout:9/427: stat d2/d13 0 2024-09-10T11:29:26.024 INFO:tasks.workunit.client.1.smithi178.stdout:9/428: mknod d2/d19/d1c/c7c 0 2024-09-10T11:29:26.027 INFO:tasks.workunit.client.1.smithi178.stdout:9/429: symlink d2/df/d31/d4a/l7d 0 2024-09-10T11:29:26.030 INFO:tasks.workunit.client.1.smithi178.stdout:9/430: unlink d2/d19/d1c/d2d/l5b 0 2024-09-10T11:29:26.038 INFO:tasks.workunit.client.1.smithi178.stdout:0/311: dwrite d0/dc/f13 [0,4194304] 0 2024-09-10T11:29:26.038 INFO:tasks.workunit.client.1.smithi178.stdout:0/312: chown d0/f9 8 1 2024-09-10T11:29:26.043 INFO:tasks.workunit.client.1.smithi178.stdout:0/313: unlink d0/dc/d39/d4c/f38 0 2024-09-10T11:29:26.046 INFO:tasks.workunit.client.1.smithi178.stdout:0/314: getdents d0/dc/d39/d50/d5e 0 2024-09-10T11:29:26.048 INFO:tasks.workunit.client.1.smithi178.stdout:0/315: mknod d0/d2c/d33/c66 0 2024-09-10T11:29:26.055 INFO:tasks.workunit.client.1.smithi178.stdout:0/316: dwrite d0/d12/d3f/f44 [0,4194304] 0 2024-09-10T11:29:26.057 INFO:tasks.workunit.client.1.smithi178.stdout:0/317: mknod d0/d12/d3f/c67 0 2024-09-10T11:29:26.058 INFO:tasks.workunit.client.1.smithi178.stdout:0/318: dread - d0/d2c/d33/f45 zero size 2024-09-10T11:29:26.061 INFO:tasks.workunit.client.1.smithi178.stdout:0/319: creat d0/dc/d39/f68 x:0 0 0 2024-09-10T11:29:26.064 INFO:tasks.workunit.client.1.smithi178.stdout:0/320: mknod d0/dc/d39/d50/d5e/c69 0 2024-09-10T11:29:26.065 INFO:tasks.workunit.client.1.smithi178.stdout:0/321: read - d0/dc/d39/d4c/d43/f5f zero size 2024-09-10T11:29:26.066 INFO:tasks.workunit.client.1.smithi178.stdout:0/322: write d0/dc/d39/d4c/f57 [707674,85586] 0 2024-09-10T11:29:26.067 INFO:tasks.workunit.client.1.smithi178.stdout:0/323: chown d0/dc/d39/d4c/d43/f5f 1993796 1 2024-09-10T11:29:26.068 INFO:tasks.workunit.client.1.smithi178.stdout:0/324: dread - d0/dc/d39/f55 zero size 2024-09-10T11:29:26.071 INFO:tasks.workunit.client.1.smithi178.stdout:0/325: rename d0/d12/l59 to d0/dc/d39/d50/d5e/l6a 0 2024-09-10T11:29:26.076 INFO:tasks.workunit.client.1.smithi178.stdout:0/326: dread d0/dc/d39/d50/f25 [0,4194304] 0 2024-09-10T11:29:26.079 INFO:tasks.workunit.client.1.smithi178.stdout:0/327: dread d0/f9 [0,4194304] 0 2024-09-10T11:29:26.082 INFO:tasks.workunit.client.1.smithi178.stdout:0/328: unlink d0/dc/d39/d4c/c5d 0 2024-09-10T11:29:26.082 INFO:tasks.workunit.client.1.smithi178.stdout:0/329: readlink d0/dc/l52 0 2024-09-10T11:29:26.084 INFO:tasks.workunit.client.1.smithi178.stdout:0/330: mknod d0/dc/d39/d50/d5e/c6b 0 2024-09-10T11:29:26.086 INFO:tasks.workunit.client.1.smithi178.stdout:0/331: read d0/dc/d39/d4c/f57 [418832,21003] 0 2024-09-10T11:29:26.088 INFO:tasks.workunit.client.1.smithi178.stdout:0/332: mknod d0/dc/d1f/d2d/c6c 0 2024-09-10T11:29:26.089 INFO:tasks.workunit.client.1.smithi178.stdout:0/333: dread d0/f53 [0,4194304] 0 2024-09-10T11:29:26.101 INFO:tasks.workunit.client.1.smithi178.stdout:0/334: dwrite d0/dc/d39/d4c/f15 [0,4194304] 0 2024-09-10T11:29:26.104 INFO:tasks.workunit.client.1.smithi178.stdout:0/335: fsync d0/f22 0 2024-09-10T11:29:26.107 INFO:tasks.workunit.client.1.smithi178.stdout:0/336: write d0/dc/d39/d4c/f3a [3805338,17445] 0 2024-09-10T11:29:26.117 INFO:tasks.workunit.client.1.smithi178.stdout:1/197: sync 2024-09-10T11:29:26.134 INFO:tasks.workunit.client.1.smithi178.stdout:0/337: truncate d0/dc/d39/d4c/d43/f5f 165246 0 2024-09-10T11:29:26.135 INFO:tasks.workunit.client.1.smithi178.stdout:1/198: chown d3/d13/d2b/f3d 5 1 2024-09-10T11:29:26.135 INFO:tasks.workunit.client.1.smithi178.stdout:1/199: stat d3/c33 0 2024-09-10T11:29:26.135 INFO:tasks.workunit.client.1.smithi178.stdout:0/338: write d0/dc/d39/d4c/f15 [579905,88629] 0 2024-09-10T11:29:26.136 INFO:tasks.workunit.client.1.smithi178.stdout:1/200: write d3/d9/d1a/f37 [297586,99370] 0 2024-09-10T11:29:26.136 INFO:tasks.workunit.client.1.smithi178.stdout:0/339: stat d0/dc/d1f/d2d/c31 0 2024-09-10T11:29:26.137 INFO:tasks.workunit.client.1.smithi178.stdout:0/340: fdatasync d0/dc/d39/d50/f1d 0 2024-09-10T11:29:26.140 INFO:tasks.workunit.client.1.smithi178.stdout:1/201: creat d3/f40 x:0 0 0 2024-09-10T11:29:26.141 INFO:tasks.workunit.client.1.smithi178.stdout:1/202: write d3/d9/fb [4282972,129558] 0 2024-09-10T11:29:26.189 INFO:tasks.workunit.client.1.smithi178.stdout:0/341: symlink d0/d2c/d33/l6d 0 2024-09-10T11:29:26.191 INFO:tasks.workunit.client.1.smithi178.stdout:0/342: fsync d0/f53 0 2024-09-10T11:29:26.191 INFO:tasks.workunit.client.1.smithi178.stdout:0/343: dread - d0/d12/f51 zero size 2024-09-10T11:29:26.192 INFO:tasks.workunit.client.1.smithi178.stdout:1/203: fsync d3/d26/f28 0 2024-09-10T11:29:26.192 INFO:tasks.workunit.client.1.smithi178.stdout:1/204: readlink l2 0 2024-09-10T11:29:26.196 INFO:tasks.workunit.client.1.smithi178.stdout:1/205: rename d3/d13/f32 to d3/d26/d2a/f41 0 2024-09-10T11:29:26.203 INFO:tasks.workunit.client.1.smithi178.stdout:0/344: dwrite d0/dc/f21 [4194304,4194304] 0 2024-09-10T11:29:26.203 INFO:tasks.workunit.client.1.smithi178.stdout:0/345: fdatasync d0/f5a 0 2024-09-10T11:29:26.205 INFO:tasks.workunit.client.1.smithi178.stdout:0/346: rename d0 to d0/dc/d6e 22 2024-09-10T11:29:26.209 INFO:tasks.workunit.client.1.smithi178.stdout:0/347: creat d0/d12/f6f x:0 0 0 2024-09-10T11:29:26.209 INFO:tasks.workunit.client.1.smithi178.stdout:1/206: dread d3/de/f2e [0,4194304] 0 2024-09-10T11:29:26.217 INFO:tasks.workunit.client.1.smithi178.stdout:1/207: dwrite d3/f40 [0,4194304] 0 2024-09-10T11:29:26.218 INFO:tasks.workunit.client.1.smithi178.stdout:0/348: dwrite d0/d12/d3f/f44 [0,4194304] 0 2024-09-10T11:29:26.220 INFO:tasks.workunit.client.1.smithi178.stdout:0/349: write d0/d12/f34 [217406,54085] 0 2024-09-10T11:29:26.223 INFO:tasks.workunit.client.1.smithi178.stdout:1/208: creat d3/d9/f42 x:0 0 0 2024-09-10T11:29:26.224 INFO:tasks.workunit.client.1.smithi178.stdout:0/350: mknod d0/dc/d39/d4c/d43/c70 0 2024-09-10T11:29:26.226 INFO:tasks.workunit.client.1.smithi178.stdout:1/209: truncate d3/d26/f28 597502 0 2024-09-10T11:29:26.226 INFO:tasks.workunit.client.1.smithi178.stdout:0/351: write d0/dc/d39/d4c/f3a [3415660,14410] 0 2024-09-10T11:29:26.227 INFO:tasks.workunit.client.1.smithi178.stdout:1/210: write d3/d13/f17 [1413380,106305] 0 2024-09-10T11:29:26.230 INFO:tasks.workunit.client.1.smithi178.stdout:0/352: creat d0/dc/d39/d50/f71 x:0 0 0 2024-09-10T11:29:26.232 INFO:tasks.workunit.client.1.smithi178.stdout:0/353: truncate d0/f5a 1019813 0 2024-09-10T11:29:26.232 INFO:tasks.workunit.client.1.smithi178.stdout:1/211: rename d3/d13/f24 to d3/de/f43 0 2024-09-10T11:29:26.233 INFO:tasks.workunit.client.1.smithi178.stdout:1/212: chown d3/d26/f3b 1923689506 1 2024-09-10T11:29:26.233 INFO:tasks.workunit.client.1.smithi178.stdout:0/354: chown d0/dc/d39/d4c/f15 503803 1 2024-09-10T11:29:26.254 INFO:tasks.workunit.client.1.smithi178.stdout:1/213: dread d3/d13/f15 [0,4194304] 0 2024-09-10T11:29:26.256 INFO:tasks.workunit.client.1.smithi178.stdout:1/214: symlink d3/d35/l44 0 2024-09-10T11:29:26.263 INFO:tasks.workunit.client.1.smithi178.stdout:1/215: truncate d3/d26/f28 896681 0 2024-09-10T11:29:26.264 INFO:tasks.workunit.client.1.smithi178.stdout:1/216: write d3/d9/d1a/f1f [493458,7303] 0 2024-09-10T11:29:26.266 INFO:tasks.workunit.client.1.smithi178.stdout:1/217: write d3/de/f43 [3101383,115389] 0 2024-09-10T11:29:26.270 INFO:tasks.workunit.client.1.smithi178.stdout:1/218: write d3/f22 [948544,35678] 0 2024-09-10T11:29:26.282 INFO:tasks.workunit.client.1.smithi178.stdout:1/219: dread d3/d9/f25 [0,4194304] 0 2024-09-10T11:29:26.284 INFO:tasks.workunit.client.1.smithi178.stdout:1/220: creat d3/de/f45 x:0 0 0 2024-09-10T11:29:26.285 INFO:tasks.workunit.client.1.smithi178.stdout:1/221: mknod d3/d26/d3a/c46 0 2024-09-10T11:29:26.298 INFO:tasks.workunit.client.1.smithi178.stdout:1/222: dread d3/d13/f17 [0,4194304] 0 2024-09-10T11:29:26.301 INFO:tasks.workunit.client.1.smithi178.stdout:1/223: getdents d3/d9 0 2024-09-10T11:29:26.303 INFO:tasks.workunit.client.1.smithi178.stdout:1/224: unlink d3/f2c 0 2024-09-10T11:29:26.304 INFO:tasks.workunit.client.1.smithi178.stdout:1/225: chown d3/d13/d2b/f3d 1 1 2024-09-10T11:29:26.306 INFO:tasks.workunit.client.1.smithi178.stdout:1/226: creat d3/f47 x:0 0 0 2024-09-10T11:29:26.308 INFO:tasks.workunit.client.1.smithi178.stdout:1/227: mkdir d3/d9/d1a/d48 0 2024-09-10T11:29:26.309 INFO:tasks.workunit.client.1.smithi178.stdout:1/228: write d3/f22 [1915738,39310] 0 2024-09-10T11:29:26.310 INFO:tasks.workunit.client.1.smithi178.stdout:1/229: read d3/d13/f2d [247983,67328] 0 2024-09-10T11:29:26.311 INFO:tasks.workunit.client.1.smithi178.stdout:1/230: mknod d3/d26/c49 0 2024-09-10T11:29:26.316 INFO:tasks.workunit.client.1.smithi178.stdout:1/231: dwrite d3/d13/f15 [0,4194304] 0 2024-09-10T11:29:26.320 INFO:tasks.workunit.client.1.smithi178.stdout:1/232: link d3/d9/f3c d3/de/f4a 0 2024-09-10T11:29:26.323 INFO:tasks.workunit.client.1.smithi178.stdout:1/233: rename d3/d13 to d3/d9/d21/d4b 0 2024-09-10T11:29:26.327 INFO:tasks.workunit.client.1.smithi178.stdout:1/234: rename l2 to d3/d9/d21/d4b/l4c 0 2024-09-10T11:29:26.328 INFO:tasks.workunit.client.1.smithi178.stdout:1/235: chown d3/de/l27 3759 1 2024-09-10T11:29:26.329 INFO:tasks.workunit.client.1.smithi178.stdout:1/236: mknod d3/d26/d2a/c4d 0 2024-09-10T11:29:26.331 INFO:tasks.workunit.client.1.smithi178.stdout:5/359: sync 2024-09-10T11:29:26.331 INFO:tasks.workunit.client.1.smithi178.stdout:1/237: creat d3/d26/d3a/f4e x:0 0 0 2024-09-10T11:29:26.333 INFO:tasks.workunit.client.1.smithi178.stdout:5/360: creat d9/dc/dd/d37/d42/d62/f80 x:0 0 0 2024-09-10T11:29:26.335 INFO:tasks.workunit.client.1.smithi178.stdout:5/361: unlink d9/dc/dd/l46 0 2024-09-10T11:29:26.336 INFO:tasks.workunit.client.1.smithi178.stdout:1/238: dwrite d3/de/f45 [0,4194304] 0 2024-09-10T11:29:26.340 INFO:tasks.workunit.client.1.smithi178.stdout:1/239: unlink d3/c5 0 2024-09-10T11:29:26.341 INFO:tasks.workunit.client.1.smithi178.stdout:1/240: write d3/d9/d1a/f37 [116311,62026] 0 2024-09-10T11:29:26.345 INFO:tasks.workunit.client.1.smithi178.stdout:1/241: symlink d3/d9/d1a/d48/l4f 0 2024-09-10T11:29:26.348 INFO:tasks.workunit.client.1.smithi178.stdout:1/242: dread d3/d9/d1a/f37 [0,4194304] 0 2024-09-10T11:29:26.348 INFO:tasks.workunit.client.1.smithi178.stdout:1/243: chown d3/d9/d21 502286 1 2024-09-10T11:29:26.350 INFO:tasks.workunit.client.1.smithi178.stdout:1/244: fsync d3/d26/d2a/f41 0 2024-09-10T11:29:26.350 INFO:tasks.workunit.client.1.smithi178.stdout:5/362: truncate d9/dc/dd/d37/d42/f5f 478147 0 2024-09-10T11:29:26.351 INFO:tasks.workunit.client.1.smithi178.stdout:5/363: write d9/dc/dd/d37/d3e/f78 [421107,51146] 0 2024-09-10T11:29:26.353 INFO:tasks.workunit.client.1.smithi178.stdout:5/364: dread - d9/dc/dd/d37/d42/d62/f80 zero size 2024-09-10T11:29:26.356 INFO:tasks.workunit.client.1.smithi178.stdout:1/245: dwrite d3/de/f45 [0,4194304] 0 2024-09-10T11:29:26.360 INFO:tasks.workunit.client.1.smithi178.stdout:1/246: unlink d3/de/c18 0 2024-09-10T11:29:26.363 INFO:tasks.workunit.client.1.smithi178.stdout:1/247: mkdir d3/d26/d3a/d50 0 2024-09-10T11:29:26.363 INFO:tasks.workunit.client.1.smithi178.stdout:5/365: dwrite d9/dc/dd/d37/d42/d55/f74 [0,4194304] 0 2024-09-10T11:29:26.364 INFO:tasks.workunit.client.1.smithi178.stdout:5/366: chown d9/d19 177 1 2024-09-10T11:29:26.365 INFO:tasks.workunit.client.1.smithi178.stdout:5/367: write d9/dc/d17/d1d/f73 [2302280,53779] 0 2024-09-10T11:29:26.366 INFO:tasks.workunit.client.1.smithi178.stdout:1/248: chown d3/d26/d2a/l36 26 1 2024-09-10T11:29:26.368 INFO:tasks.workunit.client.1.smithi178.stdout:1/249: creat d3/d26/f51 x:0 0 0 2024-09-10T11:29:26.369 INFO:tasks.workunit.client.1.smithi178.stdout:1/250: chown d3/d9/fa 16254378 1 2024-09-10T11:29:26.443 INFO:tasks.workunit.client.1.smithi178.stdout:1/251: dwrite d3/d9/fd [0,4194304] 0 2024-09-10T11:29:26.444 INFO:tasks.workunit.client.1.smithi178.stdout:1/252: write d3/d26/d3a/f4e [718413,22966] 0 2024-09-10T11:29:26.451 INFO:tasks.workunit.client.1.smithi178.stdout:1/253: dwrite d3/d9/d21/d4b/f3f [0,4194304] 0 2024-09-10T11:29:26.453 INFO:tasks.workunit.client.1.smithi178.stdout:1/254: symlink d3/d9/l52 0 2024-09-10T11:29:26.455 INFO:tasks.workunit.client.1.smithi178.stdout:1/255: creat d3/d26/d3a/f53 x:0 0 0 2024-09-10T11:29:26.461 INFO:tasks.workunit.client.1.smithi178.stdout:1/256: rename d3/d26 to d3/d9/d54 0 2024-09-10T11:29:26.462 INFO:tasks.workunit.client.1.smithi178.stdout:1/257: chown d3/d9/d54/d3a/d50 644 1 2024-09-10T11:29:26.463 INFO:tasks.workunit.client.1.smithi178.stdout:1/258: read - d3/d9/d54/d2a/f34 zero size 2024-09-10T11:29:26.465 INFO:tasks.workunit.client.1.smithi178.stdout:1/259: symlink d3/d9/d21/d4b/d2b/l55 0 2024-09-10T11:29:26.471 INFO:tasks.workunit.client.1.smithi178.stdout:1/260: dwrite d3/d9/d54/f51 [0,4194304] 0 2024-09-10T11:29:26.475 INFO:tasks.workunit.client.1.smithi178.stdout:1/261: unlink d3/d9/f14 0 2024-09-10T11:29:26.477 INFO:tasks.workunit.client.1.smithi178.stdout:1/262: creat d3/d9/d54/f56 x:0 0 0 2024-09-10T11:29:26.479 INFO:tasks.workunit.client.1.smithi178.stdout:1/263: creat d3/d9/d21/d4b/d2b/f57 x:0 0 0 2024-09-10T11:29:26.482 INFO:tasks.workunit.client.1.smithi178.stdout:1/264: rename d3/d35/c38 to d3/d35/c58 0 2024-09-10T11:29:26.483 INFO:tasks.workunit.client.1.smithi178.stdout:1/265: creat d3/d9/f59 x:0 0 0 2024-09-10T11:29:26.486 INFO:tasks.workunit.client.1.smithi178.stdout:1/266: link d3/d9/d54/d2a/f34 d3/d9/d21/d4b/f5a 0 2024-09-10T11:29:26.487 INFO:tasks.workunit.client.1.smithi178.stdout:1/267: truncate d3/f47 539654 0 2024-09-10T11:29:26.487 INFO:tasks.workunit.client.1.smithi178.stdout:1/268: stat d3/d9/d54 0 2024-09-10T11:29:26.494 INFO:tasks.workunit.client.1.smithi178.stdout:1/269: rmdir d3/d9/d54 39 2024-09-10T11:29:26.513 INFO:tasks.workunit.client.1.smithi178.stdout:8/328: sync 2024-09-10T11:29:26.513 INFO:tasks.workunit.client.1.smithi178.stdout:3/267: sync 2024-09-10T11:29:26.513 INFO:tasks.workunit.client.1.smithi178.stdout:6/391: sync 2024-09-10T11:29:26.513 INFO:tasks.workunit.client.1.smithi178.stdout:2/485: sync 2024-09-10T11:29:26.514 INFO:tasks.workunit.client.1.smithi178.stdout:2/486: write d1/d7/f6a [573395,3063] 0 2024-09-10T11:29:26.516 INFO:tasks.workunit.client.1.smithi178.stdout:6/392: fsync d3/d7/d8/d19/f8b 0 2024-09-10T11:29:26.517 INFO:tasks.workunit.client.1.smithi178.stdout:1/270: dread d3/d9/f10 [0,4194304] 0 2024-09-10T11:29:26.517 INFO:tasks.workunit.client.1.smithi178.stdout:8/329: mkdir d5/db/d10/d70 0 2024-09-10T11:29:26.519 INFO:tasks.workunit.client.1.smithi178.stdout:1/271: chown d3/de/f4a 1877917 1 2024-09-10T11:29:26.520 INFO:tasks.workunit.client.1.smithi178.stdout:8/330: symlink d5/db/d10/d20/l71 0 2024-09-10T11:29:26.525 INFO:tasks.workunit.client.1.smithi178.stdout:2/487: truncate d1/d41/f1e 3940392 0 2024-09-10T11:29:26.527 INFO:tasks.workunit.client.1.smithi178.stdout:1/272: write d3/d9/d21/d4b/f2d [839665,42631] 0 2024-09-10T11:29:26.529 INFO:tasks.workunit.client.1.smithi178.stdout:6/393: dwrite d3/d42/d57/d78/f86 [0,4194304] 0 2024-09-10T11:29:26.533 INFO:tasks.workunit.client.1.smithi178.stdout:2/488: write d1/d7/d24/d43/f87 [4693603,101956] 0 2024-09-10T11:29:26.534 INFO:tasks.workunit.client.1.smithi178.stdout:1/273: mknod d3/d35/c5b 0 2024-09-10T11:29:26.535 INFO:tasks.workunit.client.1.smithi178.stdout:8/331: dwrite d5/db/d10/d53/f64 [0,4194304] 0 2024-09-10T11:29:26.535 INFO:tasks.workunit.client.1.smithi178.stdout:3/268: dread db/d35/f37 [0,4194304] 0 2024-09-10T11:29:26.539 INFO:tasks.workunit.client.1.smithi178.stdout:3/269: write db/d26/f32 [943596,98288] 0 2024-09-10T11:29:26.540 INFO:tasks.workunit.client.1.smithi178.stdout:6/394: mkdir d3/d7/d8/d15/d1c/d2a/d18/d8e/d90 0 2024-09-10T11:29:26.544 INFO:tasks.workunit.client.1.smithi178.stdout:6/395: readlink d3/d7/d8/d15/d1c/d20/l79 0 2024-09-10T11:29:26.544 INFO:tasks.workunit.client.1.smithi178.stdout:8/332: getdents d5/db/d10/d20 0 2024-09-10T11:29:26.546 INFO:tasks.workunit.client.1.smithi178.stdout:8/333: dread - d5/d30/d33/d3a/f6c zero size 2024-09-10T11:29:26.547 INFO:tasks.workunit.client.1.smithi178.stdout:2/489: dwrite d1/d41/d14/f23 [4194304,4194304] 0 2024-09-10T11:29:26.547 INFO:tasks.workunit.client.1.smithi178.stdout:1/274: rename d3/d9/d21/d4b/f29 to d3/d9/d1a/f5c 0 2024-09-10T11:29:26.548 INFO:tasks.workunit.client.1.smithi178.stdout:3/270: chown db/d26/l49 2429 1 2024-09-10T11:29:26.548 INFO:tasks.workunit.client.1.smithi178.stdout:1/275: chown d3/d35/c5b 1172469 1 2024-09-10T11:29:26.549 INFO:tasks.workunit.client.1.smithi178.stdout:2/490: fdatasync d1/d41/d14/d11/d53/d97/f99 0 2024-09-10T11:29:26.549 INFO:tasks.workunit.client.1.smithi178.stdout:6/396: unlink d3/d42/f84 0 2024-09-10T11:29:26.550 INFO:tasks.workunit.client.1.smithi178.stdout:8/334: symlink d5/db/d3d/l72 0 2024-09-10T11:29:26.550 INFO:tasks.workunit.client.1.smithi178.stdout:2/491: write d1/f69 [168142,38282] 0 2024-09-10T11:29:26.553 INFO:tasks.workunit.client.1.smithi178.stdout:8/335: chown d5/f37 143042902 1 2024-09-10T11:29:26.555 INFO:tasks.workunit.client.1.smithi178.stdout:1/276: mkdir d3/d9/d54/d3a/d5d 0 2024-09-10T11:29:26.555 INFO:tasks.workunit.client.1.smithi178.stdout:8/336: symlink d5/d1d/l73 0 2024-09-10T11:29:26.559 INFO:tasks.workunit.client.1.smithi178.stdout:3/271: dwrite db/d35/d3a/f4b [0,4194304] 0 2024-09-10T11:29:26.562 INFO:tasks.workunit.client.1.smithi178.stdout:3/272: write db/d26/f2c [432625,29999] 0 2024-09-10T11:29:26.566 INFO:tasks.workunit.client.1.smithi178.stdout:2/492: dwrite d1/d7/d24/d7d/d84/d34/f29 [0,4194304] 0 2024-09-10T11:29:26.572 INFO:tasks.workunit.client.1.smithi178.stdout:2/493: truncate d1/d7/d24/d7d/fa0 144747 0 2024-09-10T11:29:26.575 INFO:tasks.workunit.client.1.smithi178.stdout:8/337: dwrite d5/db/d10/d20/f2b [4194304,4194304] 0 2024-09-10T11:29:26.575 INFO:tasks.workunit.client.1.smithi178.stdout:3/273: dread db/ff [0,4194304] 0 2024-09-10T11:29:26.576 INFO:tasks.workunit.client.1.smithi178.stdout:6/397: dread f1 [0,4194304] 0 2024-09-10T11:29:26.578 INFO:tasks.workunit.client.1.smithi178.stdout:2/494: write d1/d7/d24/d7d/d84/d34/f29 [4247392,79332] 0 2024-09-10T11:29:26.582 INFO:tasks.workunit.client.1.smithi178.stdout:8/338: getdents d5/db/d10/d20 0 2024-09-10T11:29:26.582 INFO:tasks.workunit.client.1.smithi178.stdout:6/398: mkdir d3/d74/d91 0 2024-09-10T11:29:26.582 INFO:tasks.workunit.client.1.smithi178.stdout:3/274: link db/l17 db/d36/l4d 0 2024-09-10T11:29:26.582 INFO:tasks.workunit.client.1.smithi178.stdout:2/495: mkdir d1/d7/d24/d7d/d84/d55/da1 0 2024-09-10T11:29:26.583 INFO:tasks.workunit.client.1.smithi178.stdout:8/339: mknod d5/d1d/c74 0 2024-09-10T11:29:26.588 INFO:tasks.workunit.client.1.smithi178.stdout:3/275: creat db/d35/d3a/d42/d46/f4e x:0 0 0 2024-09-10T11:29:26.588 INFO:tasks.workunit.client.1.smithi178.stdout:8/340: rmdir d5/db/d10 39 2024-09-10T11:29:26.588 INFO:tasks.workunit.client.1.smithi178.stdout:2/496: dread d1/d7/d24/d7d/d84/d34/f17 [0,4194304] 0 2024-09-10T11:29:26.592 INFO:tasks.workunit.client.1.smithi178.stdout:8/341: mkdir d5/d30/d33/d75 0 2024-09-10T11:29:26.593 INFO:tasks.workunit.client.1.smithi178.stdout:8/342: fdatasync d5/f24 0 2024-09-10T11:29:26.593 INFO:tasks.workunit.client.1.smithi178.stdout:3/276: creat db/f4f x:0 0 0 2024-09-10T11:29:26.593 INFO:tasks.workunit.client.1.smithi178.stdout:8/343: truncate d5/d30/f4f 494068 0 2024-09-10T11:29:26.593 INFO:tasks.workunit.client.1.smithi178.stdout:2/497: symlink d1/d41/d8d/d9f/la2 0 2024-09-10T11:29:26.594 INFO:tasks.workunit.client.1.smithi178.stdout:8/344: mkdir d5/d1d/d22/d76 0 2024-09-10T11:29:26.596 INFO:tasks.workunit.client.1.smithi178.stdout:2/498: creat d1/d7/d24/d7d/d84/d4b/d89/fa3 x:0 0 0 2024-09-10T11:29:26.597 INFO:tasks.workunit.client.1.smithi178.stdout:6/399: write f0 [2943871,47235] 0 2024-09-10T11:29:26.600 INFO:tasks.workunit.client.1.smithi178.stdout:6/400: readlink d3/d24/l64 0 2024-09-10T11:29:26.602 INFO:tasks.workunit.client.1.smithi178.stdout:8/345: dwrite d5/db/d10/d35/f5a [0,4194304] 0 2024-09-10T11:29:26.603 INFO:tasks.workunit.client.1.smithi178.stdout:6/401: creat d3/d42/d57/d78/d7c/f92 x:0 0 0 2024-09-10T11:29:26.603 INFO:tasks.workunit.client.1.smithi178.stdout:3/277: truncate f8 2276699 0 2024-09-10T11:29:26.604 INFO:tasks.workunit.client.1.smithi178.stdout:2/499: write d1/d22/f30 [461136,105175] 0 2024-09-10T11:29:26.604 INFO:tasks.workunit.client.1.smithi178.stdout:8/346: write d5/db/d10/d20/f2b [4817266,49002] 0 2024-09-10T11:29:26.606 INFO:tasks.workunit.client.1.smithi178.stdout:8/347: chown d5/db/d10/d53 19200447 1 2024-09-10T11:29:26.607 INFO:tasks.workunit.client.1.smithi178.stdout:6/402: creat d3/d7/d8/d15/d1c/d20/f93 x:0 0 0 2024-09-10T11:29:26.609 INFO:tasks.workunit.client.1.smithi178.stdout:2/500: write d1/d7/d24/d7d/d84/d4b/f66 [2635995,80610] 0 2024-09-10T11:29:26.612 INFO:tasks.workunit.client.1.smithi178.stdout:3/278: dwrite db/f1d [0,4194304] 0 2024-09-10T11:29:26.612 INFO:tasks.workunit.client.1.smithi178.stdout:6/403: symlink d3/d7/d8/d15/d48/l94 0 2024-09-10T11:29:26.616 INFO:tasks.workunit.client.1.smithi178.stdout:8/348: dwrite d5/db/d10/d20/f3b [0,4194304] 0 2024-09-10T11:29:26.616 INFO:tasks.workunit.client.1.smithi178.stdout:2/501: rename d1/d7/d24/f5a to d1/d7/d24/d46/fa4 0 2024-09-10T11:29:26.617 INFO:tasks.workunit.client.1.smithi178.stdout:3/279: creat db/d36/f50 x:0 0 0 2024-09-10T11:29:26.617 INFO:tasks.workunit.client.1.smithi178.stdout:8/349: readlink d5/d1d/l73 0 2024-09-10T11:29:26.618 INFO:tasks.workunit.client.1.smithi178.stdout:2/502: mknod d1/d22/ca5 0 2024-09-10T11:29:26.619 INFO:tasks.workunit.client.1.smithi178.stdout:2/503: readlink d1/d41/d14/d11/lb 0 2024-09-10T11:29:26.620 INFO:tasks.workunit.client.1.smithi178.stdout:8/350: dread d5/db/d1e/f62 [0,4194304] 0 2024-09-10T11:29:26.620 INFO:tasks.workunit.client.1.smithi178.stdout:2/504: chown d1/d7/d24/c39 103761 1 2024-09-10T11:29:26.621 INFO:tasks.workunit.client.1.smithi178.stdout:2/505: read d1/d7/d24/d7d/fa0 [59124,43475] 0 2024-09-10T11:29:26.624 INFO:tasks.workunit.client.1.smithi178.stdout:6/404: rename d3/d7/c44 to d3/d7/d8/c95 0 2024-09-10T11:29:26.626 INFO:tasks.workunit.client.1.smithi178.stdout:2/506: dread d1/d7/f16 [0,4194304] 0 2024-09-10T11:29:26.626 INFO:tasks.workunit.client.1.smithi178.stdout:6/405: fdatasync d3/d7/d8/d15/d1c/f53 0 2024-09-10T11:29:26.629 INFO:tasks.workunit.client.1.smithi178.stdout:8/351: rename d5/db/l6e to d5/d30/d33/l77 0 2024-09-10T11:29:26.629 INFO:tasks.workunit.client.1.smithi178.stdout:6/406: read d3/d7/d8/d15/d1c/d2a/f2c [1257501,228] 0 2024-09-10T11:29:26.631 INFO:tasks.workunit.client.1.smithi178.stdout:8/352: write d5/f8 [3944735,123494] 0 2024-09-10T11:29:26.632 INFO:tasks.workunit.client.1.smithi178.stdout:3/280: getdents db 0 2024-09-10T11:29:26.632 INFO:tasks.workunit.client.1.smithi178.stdout:6/407: creat d3/d42/f96 x:0 0 0 2024-09-10T11:29:26.633 INFO:tasks.workunit.client.1.smithi178.stdout:8/353: write d5/db/d10/d20/f26 [6979761,40708] 0 2024-09-10T11:29:26.634 INFO:tasks.workunit.client.1.smithi178.stdout:3/281: write db/f16 [6982462,102803] 0 2024-09-10T11:29:26.635 INFO:tasks.workunit.client.1.smithi178.stdout:6/408: mkdir d3/d42/d97 0 2024-09-10T11:29:26.636 INFO:tasks.workunit.client.1.smithi178.stdout:3/282: unlink l2 0 2024-09-10T11:29:26.637 INFO:tasks.workunit.client.1.smithi178.stdout:8/354: write d5/db/d10/d35/f3c [435781,14810] 0 2024-09-10T11:29:26.639 INFO:tasks.workunit.client.1.smithi178.stdout:3/283: creat db/d20/d3d/f51 x:0 0 0 2024-09-10T11:29:26.640 INFO:tasks.workunit.client.1.smithi178.stdout:2/507: dread d1/d41/d14/d11/f2a [0,4194304] 0 2024-09-10T11:29:26.641 INFO:tasks.workunit.client.1.smithi178.stdout:3/284: creat db/d20/f52 x:0 0 0 2024-09-10T11:29:26.643 INFO:tasks.workunit.client.1.smithi178.stdout:3/285: write db/f39 [228160,81538] 0 2024-09-10T11:29:26.643 INFO:tasks.workunit.client.1.smithi178.stdout:8/355: creat d5/db/d10/d70/f78 x:0 0 0 2024-09-10T11:29:26.645 INFO:tasks.workunit.client.1.smithi178.stdout:8/356: chown d5/db/fe 3 1 2024-09-10T11:29:26.646 INFO:tasks.workunit.client.1.smithi178.stdout:3/286: creat db/d20/d3d/f53 x:0 0 0 2024-09-10T11:29:26.650 INFO:tasks.workunit.client.1.smithi178.stdout:8/357: dwrite d5/f7 [0,4194304] 0 2024-09-10T11:29:26.652 INFO:tasks.workunit.client.1.smithi178.stdout:3/287: getdents db/d35/d3a/d42/d46 0 2024-09-10T11:29:26.652 INFO:tasks.workunit.client.1.smithi178.stdout:3/288: dread - db/d20/f52 zero size 2024-09-10T11:29:26.653 INFO:tasks.workunit.client.1.smithi178.stdout:8/358: creat d5/db/d10/d20/f79 x:0 0 0 2024-09-10T11:29:26.654 INFO:tasks.workunit.client.1.smithi178.stdout:8/359: dread - d5/d1d/d22/f4e zero size 2024-09-10T11:29:26.655 INFO:tasks.workunit.client.1.smithi178.stdout:7/254: sync 2024-09-10T11:29:26.655 INFO:tasks.workunit.client.1.smithi178.stdout:4/203: sync 2024-09-10T11:29:26.655 INFO:tasks.workunit.client.1.smithi178.stdout:3/289: mknod db/d26/d40/c54 0 2024-09-10T11:29:26.657 INFO:tasks.workunit.client.1.smithi178.stdout:7/255: write d6/d10/d1e/d2d/f1c [585470,25240] 0 2024-09-10T11:29:26.659 INFO:tasks.workunit.client.1.smithi178.stdout:8/360: dread d5/db/d10/d20/d52/f68 [0,4194304] 0 2024-09-10T11:29:26.660 INFO:tasks.workunit.client.1.smithi178.stdout:8/361: write d5/db/d1e/f39 [268964,26028] 0 2024-09-10T11:29:26.660 INFO:tasks.workunit.client.1.smithi178.stdout:3/290: unlink db/d35/d3a/l3f 0 2024-09-10T11:29:26.662 INFO:tasks.workunit.client.1.smithi178.stdout:6/409: write d3/d42/d57/f68 [1823839,114117] 0 2024-09-10T11:29:26.662 INFO:tasks.workunit.client.1.smithi178.stdout:8/362: chown d5/d61/l63 6 1 2024-09-10T11:29:26.665 INFO:tasks.workunit.client.1.smithi178.stdout:4/204: creat d0/d9/d3d/f44 x:0 0 0 2024-09-10T11:29:26.665 INFO:tasks.workunit.client.1.smithi178.stdout:3/291: mknod db/d20/c55 0 2024-09-10T11:29:26.665 INFO:tasks.workunit.client.1.smithi178.stdout:2/508: truncate d1/d41/d14/d11/f4a 3511697 0 2024-09-10T11:29:26.666 INFO:tasks.workunit.client.1.smithi178.stdout:8/363: chown d5/db/d1e/l31 293299426 1 2024-09-10T11:29:26.667 INFO:tasks.workunit.client.1.smithi178.stdout:6/410: symlink d3/d42/l98 0 2024-09-10T11:29:26.669 INFO:tasks.workunit.client.1.smithi178.stdout:7/256: dwrite d6/d10/d1e/f45 [0,4194304] 0 2024-09-10T11:29:26.669 INFO:tasks.workunit.client.1.smithi178.stdout:3/292: truncate db/d20/d3d/f51 240601 0 2024-09-10T11:29:26.670 INFO:tasks.workunit.client.1.smithi178.stdout:2/509: truncate d1/d41/d14/f1b 5916402 0 2024-09-10T11:29:26.670 INFO:tasks.workunit.client.1.smithi178.stdout:6/411: fdatasync d3/d7/d35/f63 0 2024-09-10T11:29:26.671 INFO:tasks.workunit.client.1.smithi178.stdout:4/205: read d0/f18 [3111086,127856] 0 2024-09-10T11:29:26.672 INFO:tasks.workunit.client.1.smithi178.stdout:4/206: dread - d0/f42 zero size 2024-09-10T11:29:26.672 INFO:tasks.workunit.client.1.smithi178.stdout:6/412: fsync d3/d42/f96 0 2024-09-10T11:29:26.673 INFO:tasks.workunit.client.1.smithi178.stdout:7/257: creat d6/d10/f5c x:0 0 0 2024-09-10T11:29:26.673 INFO:tasks.workunit.client.1.smithi178.stdout:8/364: truncate d5/f1f 306392 0 2024-09-10T11:29:26.673 INFO:tasks.workunit.client.1.smithi178.stdout:4/207: chown d0/d9/d3d 2329 1 2024-09-10T11:29:26.673 INFO:tasks.workunit.client.1.smithi178.stdout:3/293: mkdir db/d20/d3d/d56 0 2024-09-10T11:29:26.674 INFO:tasks.workunit.client.1.smithi178.stdout:7/258: chown d6/d10/d34/f4d 1901441366 1 2024-09-10T11:29:26.675 INFO:tasks.workunit.client.1.smithi178.stdout:3/294: read db/ff [466156,36618] 0 2024-09-10T11:29:26.675 INFO:tasks.workunit.client.1.smithi178.stdout:7/259: stat d6/d10/d1d/f31 0 2024-09-10T11:29:26.675 INFO:tasks.workunit.client.1.smithi178.stdout:6/413: write d3/d7/d35/f6f [1854297,5204] 0 2024-09-10T11:29:26.678 INFO:tasks.workunit.client.1.smithi178.stdout:4/208: mknod d0/d9/c45 0 2024-09-10T11:29:26.678 INFO:tasks.workunit.client.1.smithi178.stdout:3/295: truncate db/f4f 778174 0 2024-09-10T11:29:26.678 INFO:tasks.workunit.client.1.smithi178.stdout:8/365: write d5/db/d10/d20/f40 [5006349,14450] 0 2024-09-10T11:29:26.681 INFO:tasks.workunit.client.1.smithi178.stdout:6/414: write d3/d7/d8/d15/d1c/f45 [2064287,57080] 0 2024-09-10T11:29:26.683 INFO:tasks.workunit.client.1.smithi178.stdout:8/366: chown d5/d30/d33/c3e 0 1 2024-09-10T11:29:26.683 INFO:tasks.workunit.client.1.smithi178.stdout:2/510: dread d1/d7/d24/f42 [0,4194304] 0 2024-09-10T11:29:26.684 INFO:tasks.workunit.client.1.smithi178.stdout:4/209: mknod d0/d9/dc/c46 0 2024-09-10T11:29:26.684 INFO:tasks.workunit.client.1.smithi178.stdout:8/367: chown d5/f24 124 1 2024-09-10T11:29:26.684 INFO:tasks.workunit.client.1.smithi178.stdout:6/415: symlink d3/d7/d8/d15/l99 0 2024-09-10T11:29:26.685 INFO:tasks.workunit.client.1.smithi178.stdout:7/260: fdatasync d6/d10/d1e/f45 0 2024-09-10T11:29:26.685 INFO:tasks.workunit.client.1.smithi178.stdout:3/296: symlink db/d20/d3e/l57 0 2024-09-10T11:29:26.686 INFO:tasks.workunit.client.1.smithi178.stdout:4/210: write d0/d21/f37 [988322,26375] 0 2024-09-10T11:29:26.687 INFO:tasks.workunit.client.1.smithi178.stdout:2/511: mknod d1/d41/d8d/d9f/ca6 0 2024-09-10T11:29:26.689 INFO:tasks.workunit.client.1.smithi178.stdout:4/211: write d0/d9/f12 [375691,83804] 0 2024-09-10T11:29:26.690 INFO:tasks.workunit.client.1.smithi178.stdout:4/212: readlink d0/d21/l24 0 2024-09-10T11:29:26.692 INFO:tasks.workunit.client.1.smithi178.stdout:8/368: symlink d5/d1d/d22/d76/l7a 0 2024-09-10T11:29:26.693 INFO:tasks.workunit.client.1.smithi178.stdout:6/416: rename d3/d7/d8/d15/d1c/d20/l27 to d3/d7/d8/d15/d1c/d20/l9a 0 2024-09-10T11:29:26.695 INFO:tasks.workunit.client.1.smithi178.stdout:8/369: fsync d5/db/d10/d35/f5a 0 2024-09-10T11:29:26.696 INFO:tasks.workunit.client.1.smithi178.stdout:2/512: creat d1/d7/d24/d7d/d84/d34/d5b/d7c/fa7 x:0 0 0 2024-09-10T11:29:26.698 INFO:tasks.workunit.client.1.smithi178.stdout:3/297: dwrite db/d35/d3a/f4b [0,4194304] 0 2024-09-10T11:29:26.701 INFO:tasks.workunit.client.1.smithi178.stdout:7/261: dwrite d6/d29/f3c [0,4194304] 0 2024-09-10T11:29:26.702 INFO:tasks.workunit.client.1.smithi178.stdout:8/370: write d5/d30/d33/f49 [1501651,19369] 0 2024-09-10T11:29:26.705 INFO:tasks.workunit.client.1.smithi178.stdout:2/513: dread - d1/d7/d24/d7d/d84/d4b/d89/fa3 zero size 2024-09-10T11:29:26.709 INFO:tasks.workunit.client.1.smithi178.stdout:3/298: mkdir db/d35/d58 0 2024-09-10T11:29:26.709 INFO:tasks.workunit.client.1.smithi178.stdout:4/213: dwrite d0/d8/da/f27 [0,4194304] 0 2024-09-10T11:29:26.713 INFO:tasks.workunit.client.1.smithi178.stdout:8/371: creat d5/db/d10/d20/f7b x:0 0 0 2024-09-10T11:29:26.717 INFO:tasks.workunit.client.1.smithi178.stdout:4/214: readlink d0/d8/df/l22 0 2024-09-10T11:29:26.719 INFO:tasks.workunit.client.1.smithi178.stdout:2/514: link d1/d7/d24/d43/f87 d1/d7/d8e/fa8 0 2024-09-10T11:29:26.719 INFO:tasks.workunit.client.1.smithi178.stdout:3/299: truncate f8 682541 0 2024-09-10T11:29:26.720 INFO:tasks.workunit.client.1.smithi178.stdout:6/417: dwrite d3/d7/d8/d15/d1c/d2a/d18/f70 [0,4194304] 0 2024-09-10T11:29:26.721 INFO:tasks.workunit.client.1.smithi178.stdout:7/262: dwrite d6/f14 [0,4194304] 0 2024-09-10T11:29:26.726 INFO:tasks.workunit.client.1.smithi178.stdout:2/515: creat d1/d41/d5d/fa9 x:0 0 0 2024-09-10T11:29:26.731 INFO:tasks.workunit.client.1.smithi178.stdout:4/215: write d0/d8/da/f2b [1345141,44013] 0 2024-09-10T11:29:26.735 INFO:tasks.workunit.client.1.smithi178.stdout:3/300: dwrite db/d20/f33 [0,4194304] 0 2024-09-10T11:29:26.738 INFO:tasks.workunit.client.1.smithi178.stdout:6/418: dwrite d3/d7/d8/d15/d1c/d20/f47 [4194304,4194304] 0 2024-09-10T11:29:26.740 INFO:tasks.workunit.client.1.smithi178.stdout:4/216: symlink d0/d8/da/l47 0 2024-09-10T11:29:26.745 INFO:tasks.workunit.client.1.smithi178.stdout:7/263: dread d6/d10/d1e/f26 [0,4194304] 0 2024-09-10T11:29:26.745 INFO:tasks.workunit.client.1.smithi178.stdout:2/516: dread d1/d7/d24/d7d/d84/d34/f29 [0,4194304] 0 2024-09-10T11:29:26.746 INFO:tasks.workunit.client.1.smithi178.stdout:6/419: truncate d3/d7/d8/d15/d1c/f89 1042470 0 2024-09-10T11:29:26.749 INFO:tasks.workunit.client.1.smithi178.stdout:4/217: creat d0/d10/f48 x:0 0 0 2024-09-10T11:29:26.754 INFO:tasks.workunit.client.1.smithi178.stdout:2/517: write d1/d7/d24/d7d/d84/d4b/d89/fa3 [589308,69082] 0 2024-09-10T11:29:26.754 INFO:tasks.workunit.client.1.smithi178.stdout:2/518: readlink d1/d41/d5d/l7e 0 2024-09-10T11:29:26.755 INFO:tasks.workunit.client.1.smithi178.stdout:4/218: link d0/d9/fb d0/d8/df/f49 0 2024-09-10T11:29:26.755 INFO:tasks.workunit.client.1.smithi178.stdout:2/519: write d1/d7/d24/d7d/d84/d55/f94 [483103,125669] 0 2024-09-10T11:29:26.758 INFO:tasks.workunit.client.1.smithi178.stdout:9/431: sync 2024-09-10T11:29:26.758 INFO:tasks.workunit.client.1.smithi178.stdout:1/277: sync 2024-09-10T11:29:26.758 INFO:tasks.workunit.client.1.smithi178.stdout:0/355: sync 2024-09-10T11:29:26.758 INFO:tasks.workunit.client.1.smithi178.stdout:5/368: sync 2024-09-10T11:29:26.761 INFO:tasks.workunit.client.1.smithi178.stdout:0/356: readlink d0/d2c/d33/l56 0 2024-09-10T11:29:26.762 INFO:tasks.workunit.client.1.smithi178.stdout:5/369: symlink d9/dc/dd/d3f/d5a/l81 0 2024-09-10T11:29:26.763 INFO:tasks.workunit.client.1.smithi178.stdout:2/520: symlink d1/d7/d24/d7d/d84/d34/d5b/laa 0 2024-09-10T11:29:26.764 INFO:tasks.workunit.client.1.smithi178.stdout:3/301: dread f1 [0,4194304] 0 2024-09-10T11:29:26.765 INFO:tasks.workunit.client.1.smithi178.stdout:9/432: mknod d2/d19/c7e 0 2024-09-10T11:29:26.766 INFO:tasks.workunit.client.1.smithi178.stdout:0/357: mkdir d0/d12/d3f/d72 0 2024-09-10T11:29:26.767 INFO:tasks.workunit.client.1.smithi178.stdout:5/370: unlink d9/d19/f57 0 2024-09-10T11:29:26.772 INFO:tasks.workunit.client.1.smithi178.stdout:6/420: write d3/d7/d8/d15/d1c/f40 [5196266,3690] 0 2024-09-10T11:29:26.773 INFO:tasks.workunit.client.1.smithi178.stdout:9/433: dread d2/df/d31/d37/d3d/f6e [0,4194304] 0 2024-09-10T11:29:26.774 INFO:tasks.workunit.client.1.smithi178.stdout:2/521: chown d1/d7/d24/d7d/d84/d34/d5b/d60/c8f 587 1 2024-09-10T11:29:26.776 INFO:tasks.workunit.client.1.smithi178.stdout:8/372: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:26.778 INFO:tasks.workunit.client.1.smithi178.stdout:4/219: dwrite d0/d9/dc/f26 [4194304,4194304] 0 2024-09-10T11:29:26.779 INFO:tasks.workunit.client.1.smithi178.stdout:1/278: dwrite d3/d9/f42 [0,4194304] 0 2024-09-10T11:29:26.780 INFO:tasks.workunit.client.1.smithi178.stdout:2/522: write d1/d41/d5d/f76 [637589,87808] 0 2024-09-10T11:29:26.781 INFO:tasks.workunit.client.1.smithi178.stdout:5/371: rename d9/dc/dd/d37/c64 to d9/dc/dd/d3f/d5a/c82 0 2024-09-10T11:29:26.782 INFO:tasks.workunit.client.1.smithi178.stdout:3/302: link db/d26/l49 db/d35/d3a/d42/l59 0 2024-09-10T11:29:26.782 INFO:tasks.workunit.client.1.smithi178.stdout:1/279: readlink d3/d9/d54/d2a/l36 0 2024-09-10T11:29:26.783 INFO:tasks.workunit.client.1.smithi178.stdout:9/434: mknod d2/df/d31/d37/c7f 0 2024-09-10T11:29:26.785 INFO:tasks.workunit.client.1.smithi178.stdout:4/220: symlink d0/d10/l4a 0 2024-09-10T11:29:26.788 INFO:tasks.workunit.client.1.smithi178.stdout:8/373: mknod d5/db/d10/d20/d52/d67/c7c 0 2024-09-10T11:29:26.789 INFO:tasks.workunit.client.1.smithi178.stdout:7/264: dread d6/f14 [4194304,4194304] 0 2024-09-10T11:29:26.790 INFO:tasks.workunit.client.1.smithi178.stdout:7/265: readlink d6/l1f 0 2024-09-10T11:29:26.792 INFO:tasks.workunit.client.1.smithi178.stdout:1/280: unlink d3/d9/d1a/d48/l4f 0 2024-09-10T11:29:26.793 INFO:tasks.workunit.client.1.smithi178.stdout:9/435: mknod d2/d13/d75/c80 0 2024-09-10T11:29:26.794 INFO:tasks.workunit.client.1.smithi178.stdout:7/266: chown d6/d29/d4c/d55 77 1 2024-09-10T11:29:26.796 INFO:tasks.workunit.client.1.smithi178.stdout:6/421: write d3/d42/d57/f6a [1269887,25405] 0 2024-09-10T11:29:26.797 INFO:tasks.workunit.client.1.smithi178.stdout:2/523: write d1/d7/f16 [1816843,128068] 0 2024-09-10T11:29:26.799 INFO:tasks.workunit.client.1.smithi178.stdout:5/372: dwrite d9/dc/dd/d16/f1c [0,4194304] 0 2024-09-10T11:29:26.800 INFO:tasks.workunit.client.1.smithi178.stdout:7/267: read d6/f14 [3452787,130983] 0 2024-09-10T11:29:26.806 INFO:tasks.workunit.client.1.smithi178.stdout:5/373: rmdir d9/dc/dd/d37/d42 39 2024-09-10T11:29:26.807 INFO:tasks.workunit.client.1.smithi178.stdout:0/358: dread d0/d12/f27 [0,4194304] 0 2024-09-10T11:29:26.807 INFO:tasks.workunit.client.1.smithi178.stdout:3/303: dwrite db/d20/f34 [0,4194304] 0 2024-09-10T11:29:26.808 INFO:tasks.workunit.client.1.smithi178.stdout:4/221: dread d0/f18 [0,4194304] 0 2024-09-10T11:29:26.810 INFO:tasks.workunit.client.1.smithi178.stdout:0/359: chown d0/dc/d39/d4c/f3a 21891 1 2024-09-10T11:29:26.814 INFO:tasks.workunit.client.1.smithi178.stdout:6/422: rename d3/d42/d57/d78/f86 to d3/d7/d8/d15/d1c/d20/d4b/f9b 0 2024-09-10T11:29:26.815 INFO:tasks.workunit.client.1.smithi178.stdout:8/374: truncate d5/db/d10/d20/f2c 427856 0 2024-09-10T11:29:26.816 INFO:tasks.workunit.client.1.smithi178.stdout:2/524: truncate d1/d41/d14/d11/f9 3000219 0 2024-09-10T11:29:26.816 INFO:tasks.workunit.client.1.smithi178.stdout:5/374: dwrite d9/dc/dd/d3f/f49 [0,4194304] 0 2024-09-10T11:29:26.818 INFO:tasks.workunit.client.1.smithi178.stdout:4/222: rmdir d0/d9 39 2024-09-10T11:29:26.819 INFO:tasks.workunit.client.1.smithi178.stdout:5/375: write d9/f5e [1001105,5657] 0 2024-09-10T11:29:26.821 INFO:tasks.workunit.client.1.smithi178.stdout:9/436: truncate d2/d13/f3c 2495929 0 2024-09-10T11:29:26.825 INFO:tasks.workunit.client.1.smithi178.stdout:0/360: dwrite d0/f8 [4194304,4194304] 0 2024-09-10T11:29:26.834 INFO:tasks.workunit.client.1.smithi178.stdout:1/281: dwrite d3/d9/d54/d2a/f34 [0,4194304] 0 2024-09-10T11:29:26.836 INFO:tasks.workunit.client.1.smithi178.stdout:6/423: dwrite d3/d42/f96 [0,4194304] 0 2024-09-10T11:29:26.843 INFO:tasks.workunit.client.1.smithi178.stdout:9/437: dwrite d2/f26 [0,4194304] 0 2024-09-10T11:29:26.844 INFO:tasks.workunit.client.1.smithi178.stdout:0/361: dread d0/dc/d39/d4c/f15 [0,4194304] 0 2024-09-10T11:29:26.844 INFO:tasks.workunit.client.1.smithi178.stdout:5/376: dwrite d9/dc/d17/d1d/f54 [0,4194304] 0 2024-09-10T11:29:26.844 INFO:tasks.workunit.client.1.smithi178.stdout:1/282: write d3/d9/d54/d2a/f41 [109222,123090] 0 2024-09-10T11:29:26.848 INFO:tasks.workunit.client.1.smithi178.stdout:4/223: creat d0/d8/da/d1a/f4b x:0 0 0 2024-09-10T11:29:26.851 INFO:tasks.workunit.client.1.smithi178.stdout:5/377: dread d9/dc/dd/d3f/f49 [0,4194304] 0 2024-09-10T11:29:26.852 INFO:tasks.workunit.client.1.smithi178.stdout:2/525: rename d1/d7/d24/d7d/d84/d34 to d1/d7/d24/d46/d88/d9d/dab 0 2024-09-10T11:29:26.857 INFO:tasks.workunit.client.1.smithi178.stdout:6/424: mknod d3/d7/d8/d15/d1c/d2a/d18/d8e/c9c 0 2024-09-10T11:29:26.858 INFO:tasks.workunit.client.1.smithi178.stdout:9/438: chown d2/d8/c36 12102268 1 2024-09-10T11:29:26.858 INFO:tasks.workunit.client.1.smithi178.stdout:7/268: link d6/la d6/d10/d1e/l5d 0 2024-09-10T11:29:26.860 INFO:tasks.workunit.client.1.smithi178.stdout:7/269: truncate d6/d10/d1e/d2d/f16 1045932 0 2024-09-10T11:29:26.866 INFO:tasks.workunit.client.1.smithi178.stdout:5/378: rmdir d9 39 2024-09-10T11:29:26.866 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:26 smithi178 ceph-mon[33514]: pgmap v20: 65 pgs: 65 active+clean; 541 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 52 MiB/s wr, 255 op/s 2024-09-10T11:29:26.868 INFO:tasks.workunit.client.1.smithi178.stdout:4/224: dread d0/d9/f13 [0,4194304] 0 2024-09-10T11:29:26.868 INFO:tasks.workunit.client.1.smithi178.stdout:6/425: dread d3/d7/d35/f6f [0,4194304] 0 2024-09-10T11:29:26.869 INFO:tasks.workunit.client.1.smithi178.stdout:4/225: write d0/d8/df/f3b [2942572,6243] 0 2024-09-10T11:29:26.869 INFO:tasks.workunit.client.1.smithi178.stdout:0/362: rename d0/f1 to d0/dc/d39/d4c/f73 0 2024-09-10T11:29:26.870 INFO:tasks.workunit.client.1.smithi178.stdout:6/426: read - d3/d7/d8/d15/d1c/d20/d4b/f7b zero size 2024-09-10T11:29:26.870 INFO:tasks.workunit.client.1.smithi178.stdout:8/375: creat d5/db/f7d x:0 0 0 2024-09-10T11:29:26.871 INFO:tasks.workunit.client.1.smithi178.stdout:8/376: dread - d5/f69 zero size 2024-09-10T11:29:26.872 INFO:tasks.workunit.client.1.smithi178.stdout:2/526: dread d1/f3d [0,4194304] 0 2024-09-10T11:29:26.872 INFO:tasks.workunit.client.1.smithi178.stdout:3/304: getdents db/d35 0 2024-09-10T11:29:26.879 INFO:tasks.workunit.client.1.smithi178.stdout:9/439: fsync d2/d8/f9 0 2024-09-10T11:29:26.884 INFO:tasks.workunit.client.1.smithi178.stdout:9/440: dwrite d2/d19/d1c/f24 [8388608,4194304] 0 2024-09-10T11:29:26.886 INFO:tasks.workunit.client.1.smithi178.stdout:0/363: creat d0/dc/d1f/f74 x:0 0 0 2024-09-10T11:29:26.887 INFO:tasks.workunit.client.1.smithi178.stdout:6/427: creat d3/d7/f9d x:0 0 0 2024-09-10T11:29:26.890 INFO:tasks.workunit.client.1.smithi178.stdout:1/283: creat d3/d9/f5e x:0 0 0 2024-09-10T11:29:26.892 INFO:tasks.workunit.client.1.smithi178.stdout:9/441: dwrite d2/df/d31/d37/f63 [0,4194304] 0 2024-09-10T11:29:26.893 INFO:tasks.workunit.client.1.smithi178.stdout:9/442: chown d2/df/d31/l4c 1231962 1 2024-09-10T11:29:26.895 INFO:tasks.workunit.client.1.smithi178.stdout:7/270: symlink d6/d10/d1e/l5e 0 2024-09-10T11:29:26.896 INFO:tasks.workunit.client.1.smithi178.stdout:9/443: read d2/df/d31/d5a/f7b [1896632,78117] 0 2024-09-10T11:29:26.896 INFO:tasks.workunit.client.1.smithi178.stdout:7/271: stat d6/ff 0 2024-09-10T11:29:26.897 INFO:tasks.workunit.client.1.smithi178.stdout:5/379: symlink d9/dc/dd/l83 0 2024-09-10T11:29:26.897 INFO:tasks.workunit.client.1.smithi178.stdout:5/380: chown d9/dc/d17/f3d 1393 1 2024-09-10T11:29:26.898 INFO:tasks.workunit.client.1.smithi178.stdout:4/226: creat d0/d8/d3e/f4c x:0 0 0 2024-09-10T11:29:26.901 INFO:tasks.workunit.client.1.smithi178.stdout:8/377: mknod d5/c7e 0 2024-09-10T11:29:26.901 INFO:tasks.workunit.client.1.smithi178.stdout:8/378: readlink d5/d61/l63 0 2024-09-10T11:29:26.902 INFO:tasks.workunit.client.1.smithi178.stdout:0/364: mkdir d0/dc/d39/d75 0 2024-09-10T11:29:26.905 INFO:tasks.workunit.client.1.smithi178.stdout:2/527: rename d1/lc to d1/d7/d24/lac 0 2024-09-10T11:29:26.905 INFO:tasks.workunit.client.1.smithi178.stdout:2/528: stat d1/d22 0 2024-09-10T11:29:26.907 INFO:tasks.workunit.client.1.smithi178.stdout:2/529: truncate d1/d7/d24/d7d/d84/d55/f94 850693 0 2024-09-10T11:29:26.907 INFO:tasks.workunit.client.1.smithi178.stdout:1/284: mkdir d3/d9/d21/d4b/d2b/d5f 0 2024-09-10T11:29:26.913 INFO:tasks.workunit.client.1.smithi178.stdout:5/381: chown l3 15 1 2024-09-10T11:29:26.914 INFO:tasks.workunit.client.1.smithi178.stdout:4/227: mknod d0/d8/d3e/c4d 0 2024-09-10T11:29:26.915 INFO:tasks.workunit.client.1.smithi178.stdout:1/285: dwrite d3/f22 [0,4194304] 0 2024-09-10T11:29:26.916 INFO:tasks.workunit.client.1.smithi178.stdout:0/365: write d0/f9 [699311,14146] 0 2024-09-10T11:29:26.916 INFO:tasks.workunit.client.1.smithi178.stdout:4/228: fdatasync d0/d8/da/f27 0 2024-09-10T11:29:26.917 INFO:tasks.workunit.client.1.smithi178.stdout:0/366: write d0/f9 [5035119,96010] 0 2024-09-10T11:29:26.917 INFO:tasks.workunit.client.1.smithi178.stdout:4/229: chown d0/d10/l35 7602 1 2024-09-10T11:29:26.919 INFO:tasks.workunit.client.1.smithi178.stdout:8/379: rename d5/ff to d5/db/d10/d20/d52/d5c/f7f 0 2024-09-10T11:29:26.919 INFO:tasks.workunit.client.1.smithi178.stdout:4/230: dread - d0/f42 zero size 2024-09-10T11:29:26.921 INFO:tasks.workunit.client.1.smithi178.stdout:3/305: link db/d36/l38 db/l5a 0 2024-09-10T11:29:26.922 INFO:tasks.workunit.client.1.smithi178.stdout:2/530: dread d1/d7/d24/d7d/d84/f36 [0,4194304] 0 2024-09-10T11:29:26.922 INFO:tasks.workunit.client.1.smithi178.stdout:8/380: truncate d5/d1d/d22/f4e 1012446 0 2024-09-10T11:29:26.923 INFO:tasks.workunit.client.1.smithi178.stdout:8/381: fdatasync d5/db/d10/d20/f40 0 2024-09-10T11:29:26.924 INFO:tasks.workunit.client.1.smithi178.stdout:8/382: readlink d5/db/d3d/l6f 0 2024-09-10T11:29:26.927 INFO:tasks.workunit.client.1.smithi178.stdout:6/428: link d3/d7/l17 d3/d42/d97/l9e 0 2024-09-10T11:29:26.928 INFO:tasks.workunit.client.1.smithi178.stdout:6/429: chown d3/d42 0 1 2024-09-10T11:29:26.929 INFO:tasks.workunit.client.1.smithi178.stdout:0/367: symlink d0/dc/d39/d50/l76 0 2024-09-10T11:29:26.930 INFO:tasks.workunit.client.1.smithi178.stdout:1/286: dwrite d3/d9/fa [8388608,4194304] 0 2024-09-10T11:29:26.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:26 smithi144 ceph-mon[27942]: pgmap v20: 65 pgs: 65 active+clean; 541 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 52 MiB/s wr, 255 op/s 2024-09-10T11:29:26.934 INFO:tasks.workunit.client.1.smithi178.stdout:8/383: creat d5/db/d10/d70/f80 x:0 0 0 2024-09-10T11:29:26.934 INFO:tasks.workunit.client.1.smithi178.stdout:9/444: dwrite d2/d8/f9 [0,4194304] 0 2024-09-10T11:29:26.934 INFO:tasks.workunit.client.1.smithi178.stdout:6/430: fdatasync d3/d7/d8/d15/d1c/d2a/f2c 0 2024-09-10T11:29:26.935 INFO:tasks.workunit.client.1.smithi178.stdout:0/368: creat d0/dc/d39/d4c/f77 x:0 0 0 2024-09-10T11:29:26.936 INFO:tasks.workunit.client.1.smithi178.stdout:9/445: read d2/d13/f48 [665718,17460] 0 2024-09-10T11:29:26.937 INFO:tasks.workunit.client.1.smithi178.stdout:6/431: chown d3/d7/d8/d15/d1c/d2a/d18/d8e/c9c 3 1 2024-09-10T11:29:26.937 INFO:tasks.workunit.client.1.smithi178.stdout:5/382: creat d9/dc/f84 x:0 0 0 2024-09-10T11:29:26.939 INFO:tasks.workunit.client.1.smithi178.stdout:4/231: link d0/d8/d3e/f4c d0/d8/d3e/f4e 0 2024-09-10T11:29:26.940 INFO:tasks.workunit.client.1.smithi178.stdout:8/384: mknod d5/d61/c81 0 2024-09-10T11:29:26.945 INFO:tasks.workunit.client.1.smithi178.stdout:0/369: creat d0/dc/d39/d50/d5e/f78 x:0 0 0 2024-09-10T11:29:26.946 INFO:tasks.workunit.client.1.smithi178.stdout:9/446: mkdir d2/d19/d81 0 2024-09-10T11:29:26.948 INFO:tasks.workunit.client.1.smithi178.stdout:1/287: mknod d3/d9/d21/d4b/c60 0 2024-09-10T11:29:26.949 INFO:tasks.workunit.client.1.smithi178.stdout:6/432: truncate d3/d7/d8/d15/f25 593013 0 2024-09-10T11:29:26.950 INFO:tasks.workunit.client.1.smithi178.stdout:1/288: fsync d3/d9/d54/f3b 0 2024-09-10T11:29:26.951 INFO:tasks.workunit.client.1.smithi178.stdout:4/232: mkdir d0/d9/d3d/d4f 0 2024-09-10T11:29:26.952 INFO:tasks.workunit.client.1.smithi178.stdout:8/385: mkdir d5/db/d10/d53/d82 0 2024-09-10T11:29:26.954 INFO:tasks.workunit.client.1.smithi178.stdout:1/289: chown d3/de/f2e 77 1 2024-09-10T11:29:26.956 INFO:tasks.workunit.client.1.smithi178.stdout:2/531: getdents d1/d7/d24/d43 0 2024-09-10T11:29:26.956 INFO:tasks.workunit.client.1.smithi178.stdout:6/433: mknod d3/d7/d8/d19/c9f 0 2024-09-10T11:29:26.956 INFO:tasks.workunit.client.1.smithi178.stdout:8/386: write d5/db/d10/d20/f2b [1151970,61836] 0 2024-09-10T11:29:26.957 INFO:tasks.workunit.client.1.smithi178.stdout:5/383: symlink d9/dc/dd/d37/d3e/l85 0 2024-09-10T11:29:26.958 INFO:tasks.workunit.client.1.smithi178.stdout:9/447: dwrite d2/dc/f44 [0,4194304] 0 2024-09-10T11:29:26.958 INFO:tasks.workunit.client.1.smithi178.stdout:4/233: mkdir d0/d9/dc/d50 0 2024-09-10T11:29:26.964 INFO:tasks.workunit.client.1.smithi178.stdout:3/306: dwrite db/f45 [0,4194304] 0 2024-09-10T11:29:26.967 INFO:tasks.workunit.client.1.smithi178.stdout:3/307: chown db/f15 118504 1 2024-09-10T11:29:26.967 INFO:tasks.workunit.client.1.smithi178.stdout:1/290: rename d3/de to d3/d9/d21/d4b/d2b/d5f/d61 0 2024-09-10T11:29:26.973 INFO:tasks.workunit.client.1.smithi178.stdout:2/532: symlink d1/d7/d24/d46/d88/d9d/dab/d5e/lad 0 2024-09-10T11:29:26.973 INFO:tasks.workunit.client.1.smithi178.stdout:6/434: creat d3/fa0 x:0 0 0 2024-09-10T11:29:26.974 INFO:tasks.workunit.client.1.smithi178.stdout:9/448: dwrite d2/df/d31/f76 [0,4194304] 0 2024-09-10T11:29:26.974 INFO:tasks.workunit.client.1.smithi178.stdout:8/387: rename d5/d61/l63 to d5/d1d/l83 0 2024-09-10T11:29:26.975 INFO:tasks.workunit.client.1.smithi178.stdout:2/533: readlink d1/d7/l9c 0 2024-09-10T11:29:26.976 INFO:tasks.workunit.client.1.smithi178.stdout:1/291: mknod d3/d9/d54/d2a/c62 0 2024-09-10T11:29:26.977 INFO:tasks.workunit.client.1.smithi178.stdout:4/234: truncate d0/d9/fb 5789185 0 2024-09-10T11:29:26.979 INFO:tasks.workunit.client.1.smithi178.stdout:3/308: write db/d20/f28 [3795737,119661] 0 2024-09-10T11:29:26.979 INFO:tasks.workunit.client.1.smithi178.stdout:9/449: symlink d2/d13/l82 0 2024-09-10T11:29:26.980 INFO:tasks.workunit.client.1.smithi178.stdout:8/388: read d5/f37 [651426,6794] 0 2024-09-10T11:29:26.983 INFO:tasks.workunit.client.1.smithi178.stdout:4/235: unlink d0/l3 0 2024-09-10T11:29:26.984 INFO:tasks.workunit.client.1.smithi178.stdout:8/389: fsync d5/db/d1e/f5e 0 2024-09-10T11:29:26.984 INFO:tasks.workunit.client.1.smithi178.stdout:0/370: dwrite d0/dc/d39/d4c/f24 [0,4194304] 0 2024-09-10T11:29:26.985 INFO:tasks.workunit.client.1.smithi178.stdout:1/292: write d3/d9/d21/d4b/f5a [4054236,112222] 0 2024-09-10T11:29:26.988 INFO:tasks.workunit.client.1.smithi178.stdout:8/390: fsync d5/db/d10/d53/f64 0 2024-09-10T11:29:26.990 INFO:tasks.workunit.client.1.smithi178.stdout:0/371: write d0/d12/f64 [316730,75959] 0 2024-09-10T11:29:26.992 INFO:tasks.workunit.client.1.smithi178.stdout:3/309: dread db/d20/f33 [0,4194304] 0 2024-09-10T11:29:26.993 INFO:tasks.workunit.client.1.smithi178.stdout:9/450: creat d2/df/d31/d37/d3d/f83 x:0 0 0 2024-09-10T11:29:26.993 INFO:tasks.workunit.client.1.smithi178.stdout:6/435: write d3/d7/d8/d15/d1c/d2a/f2c [4382630,72238] 0 2024-09-10T11:29:26.996 INFO:tasks.workunit.client.1.smithi178.stdout:8/391: symlink d5/db/d10/d20/d52/l84 0 2024-09-10T11:29:27.001 INFO:tasks.workunit.client.1.smithi178.stdout:0/372: dread d0/f5a [0,4194304] 0 2024-09-10T11:29:27.001 INFO:tasks.workunit.client.1.smithi178.stdout:5/384: dread d9/dc/dd/f69 [0,4194304] 0 2024-09-10T11:29:27.001 INFO:tasks.workunit.client.1.smithi178.stdout:6/436: symlink d3/d4e/la1 0 2024-09-10T11:29:27.002 INFO:tasks.workunit.client.1.smithi178.stdout:9/451: symlink d2/d19/d1c/d77/l84 0 2024-09-10T11:29:27.003 INFO:tasks.workunit.client.1.smithi178.stdout:4/236: dwrite d0/d10/f48 [0,4194304] 0 2024-09-10T11:29:27.003 INFO:tasks.workunit.client.1.smithi178.stdout:8/392: symlink d5/db/d10/d53/l85 0 2024-09-10T11:29:27.008 INFO:tasks.workunit.client.1.smithi178.stdout:5/385: creat d9/f86 x:0 0 0 2024-09-10T11:29:27.009 INFO:tasks.workunit.client.1.smithi178.stdout:3/310: getdents db/d35/d3a/d42 0 2024-09-10T11:29:27.010 INFO:tasks.workunit.client.1.smithi178.stdout:9/452: fsync d2/df/d31/f47 0 2024-09-10T11:29:27.012 INFO:tasks.workunit.client.1.smithi178.stdout:4/237: mkdir d0/d8/da/d1a/d51 0 2024-09-10T11:29:27.014 INFO:tasks.workunit.client.1.smithi178.stdout:6/437: rename d3/d7/d8/d15/d1c/f45 to d3/d7/d8/d15/d1c/d2a/fa2 0 2024-09-10T11:29:27.014 INFO:tasks.workunit.client.1.smithi178.stdout:8/393: creat d5/d30/d33/d75/f86 x:0 0 0 2024-09-10T11:29:27.016 INFO:tasks.workunit.client.1.smithi178.stdout:1/293: dwrite d3/d9/d21/d4b/d2b/f3d [0,4194304] 0 2024-09-10T11:29:27.017 INFO:tasks.workunit.client.1.smithi178.stdout:3/311: dread db/ff [0,4194304] 0 2024-09-10T11:29:27.017 INFO:tasks.workunit.client.1.smithi178.stdout:2/534: dread d1/d41/f51 [0,4194304] 0 2024-09-10T11:29:27.017 INFO:tasks.workunit.client.1.smithi178.stdout:0/373: dwrite d0/dc/d39/f55 [0,4194304] 0 2024-09-10T11:29:27.018 INFO:tasks.workunit.client.1.smithi178.stdout:9/453: mknod d2/d13/d40/d56/c85 0 2024-09-10T11:29:27.019 INFO:tasks.workunit.client.1.smithi178.stdout:0/374: chown d0/dc/d39/d75 39104880 1 2024-09-10T11:29:27.019 INFO:tasks.workunit.client.1.smithi178.stdout:3/312: chown db/d35/d3a/l44 431109312 1 2024-09-10T11:29:27.022 INFO:tasks.workunit.client.1.smithi178.stdout:4/238: creat d0/d9/d3d/d4f/f52 x:0 0 0 2024-09-10T11:29:27.022 INFO:tasks.workunit.client.1.smithi178.stdout:9/454: fdatasync d2/df/d31/d37/f63 0 2024-09-10T11:29:27.023 INFO:tasks.workunit.client.1.smithi178.stdout:8/394: rmdir d5/db/d10/d20 39 2024-09-10T11:29:27.026 INFO:tasks.workunit.client.1.smithi178.stdout:2/535: unlink d1/d7/d24/d46/d88/d9d/dab/l1d 0 2024-09-10T11:29:27.027 INFO:tasks.workunit.client.1.smithi178.stdout:1/294: mkdir d3/d9/d54/d3a/d63 0 2024-09-10T11:29:27.031 INFO:tasks.workunit.client.1.smithi178.stdout:6/438: mkdir d3/d7/d8/d56/da3 0 2024-09-10T11:29:27.032 INFO:tasks.workunit.client.1.smithi178.stdout:8/395: read d5/db/f14 [164278,72857] 0 2024-09-10T11:29:27.033 INFO:tasks.workunit.client.1.smithi178.stdout:9/455: rename d2/df/d31/d37/f63 to d2/d13/f86 0 2024-09-10T11:29:27.034 INFO:tasks.workunit.client.1.smithi178.stdout:0/375: write d0/dc/d39/d4c/f73 [1156344,36881] 0 2024-09-10T11:29:27.035 INFO:tasks.workunit.client.1.smithi178.stdout:5/386: dwrite d9/dc/dd/d37/d42/d55/f6e [0,4194304] 0 2024-09-10T11:29:27.035 INFO:tasks.workunit.client.1.smithi178.stdout:6/439: chown d3/d4e/c6b 1 1 2024-09-10T11:29:27.035 INFO:tasks.workunit.client.1.smithi178.stdout:4/239: fdatasync d0/d9/f2d 0 2024-09-10T11:29:27.038 INFO:tasks.workunit.client.1.smithi178.stdout:3/313: unlink db/l5a 0 2024-09-10T11:29:27.041 INFO:tasks.workunit.client.1.smithi178.stdout:6/440: chown d3/d7/d8/d72/d75/l8c 122 1 2024-09-10T11:29:27.044 INFO:tasks.workunit.client.1.smithi178.stdout:6/441: readlink d3/d4e/l85 0 2024-09-10T11:29:27.045 INFO:tasks.workunit.client.1.smithi178.stdout:9/456: dread d2/f3f [0,4194304] 0 2024-09-10T11:29:27.046 INFO:tasks.workunit.client.1.smithi178.stdout:5/387: symlink d9/dc/dd/d3f/d5a/d7d/l87 0 2024-09-10T11:29:27.047 INFO:tasks.workunit.client.1.smithi178.stdout:4/240: mknod d0/d9/dc/d50/c53 0 2024-09-10T11:29:27.048 INFO:tasks.workunit.client.1.smithi178.stdout:1/295: link d3/d9/d21/d4b/d2b/d5f/d61/c1d d3/d9/d54/d3a/d50/c64 0 2024-09-10T11:29:27.050 INFO:tasks.workunit.client.1.smithi178.stdout:3/314: dwrite db/f1d [0,4194304] 0 2024-09-10T11:29:27.050 INFO:tasks.workunit.client.1.smithi178.stdout:2/536: write d1/d7/d24/d46/d88/d9d/dab/f15 [2606475,92964] 0 2024-09-10T11:29:27.050 INFO:tasks.workunit.client.1.smithi178.stdout:5/388: mknod d9/c88 0 2024-09-10T11:29:27.053 INFO:tasks.workunit.client.1.smithi178.stdout:2/537: fsync d1/d7/f16 0 2024-09-10T11:29:27.053 INFO:tasks.workunit.client.1.smithi178.stdout:1/296: read d3/d9/d54/d3a/f4e [223325,98156] 0 2024-09-10T11:29:27.054 INFO:tasks.workunit.client.1.smithi178.stdout:1/297: dread - d3/d9/f5e zero size 2024-09-10T11:29:27.054 INFO:tasks.workunit.client.1.smithi178.stdout:3/315: write db/d35/d3a/f4b [3180664,112635] 0 2024-09-10T11:29:27.055 INFO:tasks.workunit.client.1.smithi178.stdout:5/389: chown d9/dc/dd/l6c 3 1 2024-09-10T11:29:27.056 INFO:tasks.workunit.client.1.smithi178.stdout:2/538: fsync d1/d7/d24/d7d/d84/d55/f73 0 2024-09-10T11:29:27.058 INFO:tasks.workunit.client.1.smithi178.stdout:0/376: dread d0/dc/d39/d4c/f3a [0,4194304] 0 2024-09-10T11:29:27.066 INFO:tasks.workunit.client.1.smithi178.stdout:0/377: dwrite d0/dc/d39/d50/f71 [0,4194304] 0 2024-09-10T11:29:27.067 INFO:tasks.workunit.client.1.smithi178.stdout:7/272: sync 2024-09-10T11:29:27.080 INFO:tasks.workunit.client.1.smithi178.stdout:0/378: dread d0/d12/f2f [0,4194304] 0 2024-09-10T11:29:27.166 INFO:tasks.workunit.client.1.smithi178.stdout:8/396: sync 2024-09-10T11:29:27.190 INFO:tasks.workunit.client.1.smithi178.stdout:6/442: creat d3/d7/d8/d15/d1c/d2a/d18/d8e/fa4 x:0 0 0 2024-09-10T11:29:27.194 INFO:tasks.workunit.client.1.smithi178.stdout:9/457: write d2/d13/f3c [3500821,100019] 0 2024-09-10T11:29:27.195 INFO:tasks.workunit.client.1.smithi178.stdout:1/298: symlink d3/d9/d54/l65 0 2024-09-10T11:29:27.195 INFO:tasks.workunit.client.1.smithi178.stdout:9/458: readlink d2/d19/d1c/l65 0 2024-09-10T11:29:27.196 INFO:tasks.workunit.client.1.smithi178.stdout:5/390: creat d9/dc/dd/d37/d42/f89 x:0 0 0 2024-09-10T11:29:27.197 INFO:tasks.workunit.client.1.smithi178.stdout:9/459: truncate d2/df/d31/f76 4607984 0 2024-09-10T11:29:27.197 INFO:tasks.workunit.client.1.smithi178.stdout:3/316: truncate db/f1a 4672522 0 2024-09-10T11:29:27.198 INFO:tasks.workunit.client.1.smithi178.stdout:9/460: read d2/df/d31/f76 [3615436,13500] 0 2024-09-10T11:29:27.198 INFO:tasks.workunit.client.1.smithi178.stdout:2/539: creat d1/d22/fae x:0 0 0 2024-09-10T11:29:27.199 INFO:tasks.workunit.client.1.smithi178.stdout:7/273: unlink d6/d10/d34/f4d 0 2024-09-10T11:29:27.204 INFO:tasks.workunit.client.1.smithi178.stdout:0/379: rmdir d0/dc/d39/d50/d5e 39 2024-09-10T11:29:27.205 INFO:tasks.workunit.client.1.smithi178.stdout:0/380: stat d0/dc/d39 0 2024-09-10T11:29:27.205 INFO:tasks.workunit.client.1.smithi178.stdout:8/397: rmdir d5/d1d/d22/d76 39 2024-09-10T11:29:27.205 INFO:tasks.workunit.client.1.smithi178.stdout:6/443: creat d3/d7/d8/d15/fa5 x:0 0 0 2024-09-10T11:29:27.205 INFO:tasks.workunit.client.1.smithi178.stdout:7/274: dwrite d6/d10/d1e/f45 [0,4194304] 0 2024-09-10T11:29:27.207 INFO:tasks.workunit.client.1.smithi178.stdout:6/444: read d3/d42/d57/f68 [1316352,126409] 0 2024-09-10T11:29:27.209 INFO:tasks.workunit.client.1.smithi178.stdout:1/299: creat d3/d9/d1a/d48/f66 x:0 0 0 2024-09-10T11:29:27.212 INFO:tasks.workunit.client.1.smithi178.stdout:3/317: rename db/d36/f47 to db/d35/f5b 0 2024-09-10T11:29:27.216 INFO:tasks.workunit.client.1.smithi178.stdout:2/540: fdatasync d1/d7/d24/d7d/fa0 0 2024-09-10T11:29:27.217 INFO:tasks.workunit.client.1.smithi178.stdout:8/398: creat d5/d30/f87 x:0 0 0 2024-09-10T11:29:27.219 INFO:tasks.workunit.client.1.smithi178.stdout:4/241: link d0/d10/l35 d0/d9/dc/d50/l54 0 2024-09-10T11:29:27.222 INFO:tasks.workunit.client.1.smithi178.stdout:6/445: mkdir d3/d7/d8/d15/d1c/d20/da6 0 2024-09-10T11:29:27.223 INFO:tasks.workunit.client.1.smithi178.stdout:1/300: creat d3/d9/d21/f67 x:0 0 0 2024-09-10T11:29:27.226 INFO:tasks.workunit.client.1.smithi178.stdout:5/391: rename d9/dc/dd/d37/d42/d55/f74 to d9/dc/dd/d37/d42/f8a 0 2024-09-10T11:29:27.227 INFO:tasks.workunit.client.1.smithi178.stdout:8/399: dread d5/d30/d33/f49 [0,4194304] 0 2024-09-10T11:29:27.230 INFO:tasks.workunit.client.1.smithi178.stdout:0/381: mknod d0/d12/d3f/d72/c79 0 2024-09-10T11:29:27.235 INFO:tasks.workunit.client.1.smithi178.stdout:1/301: creat d3/d35/f68 x:0 0 0 2024-09-10T11:29:27.236 INFO:tasks.workunit.client.1.smithi178.stdout:0/382: dwrite d0/dc/d39/d4c/f73 [4194304,4194304] 0 2024-09-10T11:29:27.237 INFO:tasks.workunit.client.1.smithi178.stdout:9/461: creat d2/d51/d59/f87 x:0 0 0 2024-09-10T11:29:27.237 INFO:tasks.workunit.client.1.smithi178.stdout:1/302: write d3/d9/d21/d4b/d2b/d5f/d61/f4a [4881181,46477] 0 2024-09-10T11:29:27.238 INFO:tasks.workunit.client.1.smithi178.stdout:9/462: chown d2/d19/d1c/d2a/f67 101391 1 2024-09-10T11:29:27.240 INFO:tasks.workunit.client.1.smithi178.stdout:3/318: rename db/d26/l31 to db/d35/d58/l5c 0 2024-09-10T11:29:27.240 INFO:tasks.workunit.client.1.smithi178.stdout:3/319: write db/d20/f28 [3954846,108024] 0 2024-09-10T11:29:27.246 INFO:tasks.workunit.client.1.smithi178.stdout:3/320: dwrite db/d26/f32 [0,4194304] 0 2024-09-10T11:29:27.248 INFO:tasks.workunit.client.1.smithi178.stdout:3/321: read db/d20/f34 [1922000,115769] 0 2024-09-10T11:29:27.248 INFO:tasks.workunit.client.1.smithi178.stdout:6/446: rename d3/d7/d8/l23 to d3/d7/d35/la7 0 2024-09-10T11:29:27.253 INFO:tasks.workunit.client.1.smithi178.stdout:1/303: dwrite d3/d9/f10 [0,4194304] 0 2024-09-10T11:29:27.253 INFO:tasks.workunit.client.1.smithi178.stdout:6/447: fsync d3/d7/d35/f5a 0 2024-09-10T11:29:27.256 INFO:tasks.workunit.client.1.smithi178.stdout:9/463: getdents d2/d13/d40 0 2024-09-10T11:29:27.257 INFO:tasks.workunit.client.1.smithi178.stdout:9/464: dread d2/f3f [0,4194304] 0 2024-09-10T11:29:27.257 INFO:tasks.workunit.client.1.smithi178.stdout:6/448: symlink d3/d6e/la8 0 2024-09-10T11:29:27.258 INFO:tasks.workunit.client.1.smithi178.stdout:1/304: write d3/d9/d54/d3a/f4e [1752484,60195] 0 2024-09-10T11:29:27.261 INFO:tasks.workunit.client.1.smithi178.stdout:0/383: rename d0/f53 to d0/dc/d1f/f7a 0 2024-09-10T11:29:27.263 INFO:tasks.workunit.client.1.smithi178.stdout:1/305: dwrite d3/d9/fd [0,4194304] 0 2024-09-10T11:29:27.265 INFO:tasks.workunit.client.1.smithi178.stdout:0/384: creat d0/d2c/f7b x:0 0 0 2024-09-10T11:29:27.268 INFO:tasks.workunit.client.1.smithi178.stdout:0/385: mkdir d0/dc/d39/d7c 0 2024-09-10T11:29:27.276 INFO:tasks.workunit.client.1.smithi178.stdout:0/386: dwrite d0/d2c/f7b [0,4194304] 0 2024-09-10T11:29:27.277 INFO:tasks.workunit.client.1.smithi178.stdout:1/306: dread d3/d9/d54/f51 [0,4194304] 0 2024-09-10T11:29:27.277 INFO:tasks.workunit.client.1.smithi178.stdout:0/387: write d0/dc/d39/d4c/f24 [3784737,112107] 0 2024-09-10T11:29:27.280 INFO:tasks.workunit.client.1.smithi178.stdout:1/307: rename d3/l1b to d3/d35/l69 0 2024-09-10T11:29:27.281 INFO:tasks.workunit.client.1.smithi178.stdout:1/308: readlink d3/d9/d54/l65 0 2024-09-10T11:29:27.282 INFO:tasks.workunit.client.1.smithi178.stdout:1/309: fdatasync d3/d9/d54/d2a/f34 0 2024-09-10T11:29:27.284 INFO:tasks.workunit.client.1.smithi178.stdout:0/388: dwrite d0/dc/d39/d50/f71 [0,4194304] 0 2024-09-10T11:29:27.286 INFO:tasks.workunit.client.1.smithi178.stdout:0/389: stat d0/dc/d1f/l58 0 2024-09-10T11:29:27.291 INFO:tasks.workunit.client.1.smithi178.stdout:0/390: creat d0/dc/d39/d50/d5e/f7d x:0 0 0 2024-09-10T11:29:27.291 INFO:tasks.workunit.client.1.smithi178.stdout:0/391: fdatasync d0/d2c/f37 0 2024-09-10T11:29:27.293 INFO:tasks.workunit.client.1.smithi178.stdout:0/392: chown d0/dc/d1f/d2d/c62 303 1 2024-09-10T11:29:27.294 INFO:tasks.workunit.client.1.smithi178.stdout:1/310: dread d3/d9/f25 [0,4194304] 0 2024-09-10T11:29:27.294 INFO:tasks.workunit.client.1.smithi178.stdout:0/393: rmdir d0/dc/d1f/d2d 39 2024-09-10T11:29:27.297 INFO:tasks.workunit.client.1.smithi178.stdout:0/394: mknod d0/dc/d39/d50/c7e 0 2024-09-10T11:29:27.300 INFO:tasks.workunit.client.1.smithi178.stdout:1/311: dwrite d3/f47 [0,4194304] 0 2024-09-10T11:29:27.300 INFO:tasks.workunit.client.1.smithi178.stdout:0/395: mknod d0/dc/d39/d50/c7f 0 2024-09-10T11:29:27.304 INFO:tasks.workunit.client.1.smithi178.stdout:0/396: symlink d0/dc/d39/d75/l80 0 2024-09-10T11:29:27.306 INFO:tasks.workunit.client.1.smithi178.stdout:0/397: creat d0/d2c/d33/f81 x:0 0 0 2024-09-10T11:29:27.306 INFO:tasks.workunit.client.1.smithi178.stdout:0/398: write d0/dc/d39/d4c/d43/f5f [375086,63649] 0 2024-09-10T11:29:27.307 INFO:tasks.workunit.client.1.smithi178.stdout:0/399: read - d0/dc/d39/f68 zero size 2024-09-10T11:29:27.321 INFO:tasks.workunit.client.1.smithi178.stdout:4/242: rmdir d0/d10 39 2024-09-10T11:29:27.334 INFO:tasks.workunit.client.1.smithi178.stdout:2/541: dwrite d1/d22/d4d/d9b/f92 [4194304,4194304] 0 2024-09-10T11:29:27.343 INFO:tasks.workunit.client.1.smithi178.stdout:7/275: write d6/d10/d1e/d2d/d2b/f39 [858254,66911] 0 2024-09-10T11:29:27.345 INFO:tasks.workunit.client.1.smithi178.stdout:3/322: readlink db/d35/d58/l5c 0 2024-09-10T11:29:27.348 INFO:tasks.workunit.client.1.smithi178.stdout:7/276: link d6/ff d6/d10/d1e/d2d/f5f 0 2024-09-10T11:29:27.349 INFO:tasks.workunit.client.1.smithi178.stdout:3/323: mknod db/d20/d3d/d56/c5d 0 2024-09-10T11:29:27.351 INFO:tasks.workunit.client.1.smithi178.stdout:8/400: dwrite d5/db/d1e/f62 [0,4194304] 0 2024-09-10T11:29:27.352 INFO:tasks.workunit.client.1.smithi178.stdout:8/401: chown d5/d30/f4f 7967381 1 2024-09-10T11:29:27.353 INFO:tasks.workunit.client.1.smithi178.stdout:8/402: chown d5/db/d10/d53/d82 184499 1 2024-09-10T11:29:27.353 INFO:tasks.workunit.client.1.smithi178.stdout:7/277: unlink d6/d10/d1e/d2d/f1c 0 2024-09-10T11:29:27.356 INFO:tasks.workunit.client.1.smithi178.stdout:3/324: unlink db/d35/d58/l5c 0 2024-09-10T11:29:27.358 INFO:tasks.workunit.client.1.smithi178.stdout:7/278: write f4 [1804713,108494] 0 2024-09-10T11:29:27.359 INFO:tasks.workunit.client.1.smithi178.stdout:3/325: creat db/d35/f5e x:0 0 0 2024-09-10T11:29:27.360 INFO:tasks.workunit.client.1.smithi178.stdout:8/403: dwrite d5/d30/f87 [0,4194304] 0 2024-09-10T11:29:27.364 INFO:tasks.workunit.client.1.smithi178.stdout:9/465: write d2/df/d31/d5a/f7b [4126769,111545] 0 2024-09-10T11:29:27.365 INFO:tasks.workunit.client.1.smithi178.stdout:3/326: rmdir db 39 2024-09-10T11:29:27.365 INFO:tasks.workunit.client.1.smithi178.stdout:6/449: write d3/d7/d8/d19/f8b [349500,5867] 0 2024-09-10T11:29:27.366 INFO:tasks.workunit.client.1.smithi178.stdout:2/542: dread d1/d41/d14/f59 [0,4194304] 0 2024-09-10T11:29:27.368 INFO:tasks.workunit.client.1.smithi178.stdout:7/279: mkdir d6/d10/d1d/d22/d60 0 2024-09-10T11:29:27.368 INFO:tasks.workunit.client.1.smithi178.stdout:8/404: fdatasync d5/db/d10/d20/f2b 0 2024-09-10T11:29:27.368 INFO:tasks.workunit.client.1.smithi178.stdout:0/400: fsync d0/d2c/f7b 0 2024-09-10T11:29:27.371 INFO:tasks.workunit.client.1.smithi178.stdout:6/450: write d3/d7/d8/d15/d1c/d2a/d18/f70 [2356862,38297] 0 2024-09-10T11:29:27.371 INFO:tasks.workunit.client.1.smithi178.stdout:9/466: rename d2/df/d31/d5a to d2/d13/d40/d4b/d88 0 2024-09-10T11:29:27.373 INFO:tasks.workunit.client.1.smithi178.stdout:9/467: dread - d2/d13/d40/d4b/f72 zero size 2024-09-10T11:29:27.373 INFO:tasks.workunit.client.1.smithi178.stdout:2/543: dread d1/d41/f51 [0,4194304] 0 2024-09-10T11:29:27.373 INFO:tasks.workunit.client.1.smithi178.stdout:7/280: mkdir d6/d10/d61 0 2024-09-10T11:29:27.374 INFO:tasks.workunit.client.1.smithi178.stdout:3/327: creat db/d35/f5f x:0 0 0 2024-09-10T11:29:27.376 INFO:tasks.workunit.client.1.smithi178.stdout:1/312: truncate d3/d9/d21/d4b/f5a 839214 0 2024-09-10T11:29:27.378 INFO:tasks.workunit.client.1.smithi178.stdout:0/401: creat d0/d12/d3f/d72/f82 x:0 0 0 2024-09-10T11:29:27.380 INFO:tasks.workunit.client.1.smithi178.stdout:0/402: fsync d0/d12/d3f/f44 0 2024-09-10T11:29:27.380 INFO:tasks.workunit.client.1.smithi178.stdout:6/451: truncate d3/d42/d57/f68 507334 0 2024-09-10T11:29:27.381 INFO:tasks.workunit.client.1.smithi178.stdout:9/468: dwrite d2/d13/f1f [0,4194304] 0 2024-09-10T11:29:27.382 INFO:tasks.workunit.client.1.smithi178.stdout:3/328: fsync db/f15 0 2024-09-10T11:29:27.384 INFO:tasks.workunit.client.1.smithi178.stdout:8/405: creat d5/db/f88 x:0 0 0 2024-09-10T11:29:27.386 INFO:tasks.workunit.client.1.smithi178.stdout:2/544: dread d1/d7/d24/f42 [0,4194304] 0 2024-09-10T11:29:27.387 INFO:tasks.workunit.client.1.smithi178.stdout:9/469: write d2/df/d31/f47 [1008802,57225] 0 2024-09-10T11:29:27.388 INFO:tasks.workunit.client.1.smithi178.stdout:0/403: creat d0/d12/d3f/f83 x:0 0 0 2024-09-10T11:29:27.389 INFO:tasks.workunit.client.1.smithi178.stdout:7/281: fsync d6/ff 0 2024-09-10T11:29:27.390 INFO:tasks.workunit.client.1.smithi178.stdout:3/329: dread db/f27 [0,4194304] 0 2024-09-10T11:29:27.391 INFO:tasks.workunit.client.1.smithi178.stdout:1/313: creat d3/d9/d54/d3a/d63/f6a x:0 0 0 2024-09-10T11:29:27.392 INFO:tasks.workunit.client.1.smithi178.stdout:3/330: dread - db/d35/f5f zero size 2024-09-10T11:29:27.394 INFO:tasks.workunit.client.1.smithi178.stdout:6/452: dwrite d3/d7/d8/d15/d1c/d20/f93 [0,4194304] 0 2024-09-10T11:29:27.394 INFO:tasks.workunit.client.1.smithi178.stdout:3/331: rename db/d35 to db/d35/d3a/d42/d46/d60 22 2024-09-10T11:29:27.397 INFO:tasks.workunit.client.1.smithi178.stdout:6/453: rename d3 to d3/d7/d8/d15/d1c/d20/d4b/da9 22 2024-09-10T11:29:27.399 INFO:tasks.workunit.client.1.smithi178.stdout:2/545: symlink d1/d22/d4d/d78/laf 0 2024-09-10T11:29:27.401 INFO:tasks.workunit.client.1.smithi178.stdout:3/332: mknod db/c61 0 2024-09-10T11:29:27.401 INFO:tasks.workunit.client.1.smithi178.stdout:5/392: sync 2024-09-10T11:29:27.401 INFO:tasks.workunit.client.1.smithi178.stdout:1/314: dread d3/d9/d21/d4b/d2b/d5f/d61/f2e [4194304,4194304] 0 2024-09-10T11:29:27.403 INFO:tasks.workunit.client.1.smithi178.stdout:7/282: truncate d6/fd 1631398 0 2024-09-10T11:29:27.405 INFO:tasks.workunit.client.1.smithi178.stdout:3/333: dread db/f45 [0,4194304] 0 2024-09-10T11:29:27.407 INFO:tasks.workunit.client.1.smithi178.stdout:8/406: link d5/db/d3d/c5d d5/d30/d33/c89 0 2024-09-10T11:29:27.408 INFO:tasks.workunit.client.1.smithi178.stdout:8/407: chown d5/db/d1e/f5e 3460 1 2024-09-10T11:29:27.414 INFO:tasks.workunit.client.1.smithi178.stdout:0/404: dwrite d0/dc/d39/d4c/f15 [0,4194304] 0 2024-09-10T11:29:27.419 INFO:tasks.workunit.client.1.smithi178.stdout:7/283: fsync d6/f23 0 2024-09-10T11:29:27.421 INFO:tasks.workunit.client.1.smithi178.stdout:6/454: write d3/d42/f61 [196299,85286] 0 2024-09-10T11:29:27.421 INFO:tasks.workunit.client.1.smithi178.stdout:2/546: creat d1/d41/d14/fb0 x:0 0 0 2024-09-10T11:29:27.422 INFO:tasks.workunit.client.1.smithi178.stdout:1/315: rename d3/d35/c5b to d3/d9/d21/d4b/c6b 0 2024-09-10T11:29:27.423 INFO:tasks.workunit.client.1.smithi178.stdout:8/408: dread d5/db/d10/d20/d52/d5c/f7f [0,4194304] 0 2024-09-10T11:29:27.426 INFO:tasks.workunit.client.1.smithi178.stdout:2/547: chown d1/d7/d24/d43/f87 1 1 2024-09-10T11:29:27.426 INFO:tasks.workunit.client.1.smithi178.stdout:7/284: write d6/d10/d1e/d24/f36 [58913,75401] 0 2024-09-10T11:29:27.427 INFO:tasks.workunit.client.1.smithi178.stdout:1/316: chown d3/d9/d21/d4b/f2d 900 1 2024-09-10T11:29:27.428 INFO:tasks.workunit.client.1.smithi178.stdout:5/393: rename d9/dc/f15 to d9/dc/dd/d3f/f8b 0 2024-09-10T11:29:27.428 INFO:tasks.workunit.client.1.smithi178.stdout:0/405: dwrite d0/dc/f21 [0,4194304] 0 2024-09-10T11:29:27.429 INFO:tasks.workunit.client.1.smithi178.stdout:8/409: read - d5/db/d10/d35/f44 zero size 2024-09-10T11:29:27.431 INFO:tasks.workunit.client.1.smithi178.stdout:2/548: mkdir d1/d41/d8d/db1 0 2024-09-10T11:29:27.433 INFO:tasks.workunit.client.1.smithi178.stdout:6/455: write d3/d7/d8/d15/d1c/d20/d4b/f9b [3297313,67684] 0 2024-09-10T11:29:27.433 INFO:tasks.workunit.client.1.smithi178.stdout:7/285: creat d6/d10/d1d/d4a/f62 x:0 0 0 2024-09-10T11:29:27.435 INFO:tasks.workunit.client.1.smithi178.stdout:0/406: write d0/dc/d39/d4c/d43/f5f [478205,77455] 0 2024-09-10T11:29:27.435 INFO:tasks.workunit.client.1.smithi178.stdout:2/549: write d1/f69 [703743,109588] 0 2024-09-10T11:29:27.435 INFO:tasks.workunit.client.1.smithi178.stdout:5/394: unlink d9/dc/dd/l38 0 2024-09-10T11:29:27.435 INFO:tasks.workunit.client.1.smithi178.stdout:3/334: dwrite f8 [0,4194304] 0 2024-09-10T11:29:27.441 INFO:tasks.workunit.client.1.smithi178.stdout:8/410: truncate d5/f1f 128108 0 2024-09-10T11:29:27.441 INFO:tasks.workunit.client.1.smithi178.stdout:3/335: truncate db/d36/f50 144007 0 2024-09-10T11:29:27.441 INFO:tasks.workunit.client.1.smithi178.stdout:0/407: write d0/dc/d39/d50/d5e/f7d [123124,25562] 0 2024-09-10T11:29:27.442 INFO:tasks.workunit.client.1.smithi178.stdout:1/317: mkdir d3/d9/d54/d3a/d50/d6c 0 2024-09-10T11:29:27.444 INFO:tasks.workunit.client.1.smithi178.stdout:7/286: symlink d6/d10/d1e/d24/d49/l63 0 2024-09-10T11:29:27.445 INFO:tasks.workunit.client.1.smithi178.stdout:1/318: write d3/d9/d54/d3a/f4e [2115187,114227] 0 2024-09-10T11:29:27.445 INFO:tasks.workunit.client.1.smithi178.stdout:6/456: mkdir d3/d74/d91/daa 0 2024-09-10T11:29:27.445 INFO:tasks.workunit.client.1.smithi178.stdout:7/287: read - d6/d10/d1e/d2d/d2b/f41 zero size 2024-09-10T11:29:27.446 INFO:tasks.workunit.client.1.smithi178.stdout:3/336: creat db/d26/d40/f62 x:0 0 0 2024-09-10T11:29:27.447 INFO:tasks.workunit.client.1.smithi178.stdout:3/337: fdatasync db/f18 0 2024-09-10T11:29:27.448 INFO:tasks.workunit.client.1.smithi178.stdout:3/338: stat db/d20/d3d/l41 0 2024-09-10T11:29:27.448 INFO:tasks.workunit.client.1.smithi178.stdout:5/395: creat d9/dc/d17/d1d/f8c x:0 0 0 2024-09-10T11:29:27.448 INFO:tasks.workunit.client.1.smithi178.stdout:2/550: symlink d1/d7/d24/d7d/d84/d55/da1/lb2 0 2024-09-10T11:29:27.449 INFO:tasks.workunit.client.1.smithi178.stdout:3/339: fsync db/f1d 0 2024-09-10T11:29:27.449 INFO:tasks.workunit.client.1.smithi178.stdout:8/411: mknod d5/c8a 0 2024-09-10T11:29:27.452 INFO:tasks.workunit.client.1.smithi178.stdout:5/396: chown d9/dc/dd/d16/f27 43830 1 2024-09-10T11:29:27.453 INFO:tasks.workunit.client.1.smithi178.stdout:3/340: mkdir db/d20/d3d/d63 0 2024-09-10T11:29:27.454 INFO:tasks.workunit.client.1.smithi178.stdout:8/412: write d5/d1d/d22/f36 [527156,122891] 0 2024-09-10T11:29:27.454 INFO:tasks.workunit.client.1.smithi178.stdout:5/397: chown d9/d19 657 1 2024-09-10T11:29:27.454 INFO:tasks.workunit.client.1.smithi178.stdout:6/457: creat d3/d7/d8/fab x:0 0 0 2024-09-10T11:29:27.457 INFO:tasks.workunit.client.1.smithi178.stdout:1/319: dwrite d3/d9/d54/d3a/f4e [0,4194304] 0 2024-09-10T11:29:27.459 INFO:tasks.workunit.client.1.smithi178.stdout:8/413: fsync d5/db/d10/d70/f78 0 2024-09-10T11:29:27.462 INFO:tasks.workunit.client.1.smithi178.stdout:3/341: fdatasync db/d20/f33 0 2024-09-10T11:29:27.462 INFO:tasks.workunit.client.1.smithi178.stdout:8/414: fdatasync d5/d30/f6a 0 2024-09-10T11:29:27.468 INFO:tasks.workunit.client.1.smithi178.stdout:0/408: dread d0/dc/d39/d50/f1d [0,4194304] 0 2024-09-10T11:29:27.469 INFO:tasks.workunit.client.1.smithi178.stdout:1/320: unlink d3/d9/d21/d4b/f3f 0 2024-09-10T11:29:27.470 INFO:tasks.workunit.client.1.smithi178.stdout:6/458: dwrite f0 [0,4194304] 0 2024-09-10T11:29:27.475 INFO:tasks.workunit.client.1.smithi178.stdout:2/551: dread d1/d41/d14/d11/f4a [0,4194304] 0 2024-09-10T11:29:27.475 INFO:tasks.workunit.client.1.smithi178.stdout:4/243: sync 2024-09-10T11:29:27.477 INFO:tasks.workunit.client.1.smithi178.stdout:1/321: creat d3/d9/d1a/f6d x:0 0 0 2024-09-10T11:29:27.480 INFO:tasks.workunit.client.1.smithi178.stdout:8/415: dwrite d5/db/d10/d35/f3c [0,4194304] 0 2024-09-10T11:29:27.480 INFO:tasks.workunit.client.1.smithi178.stdout:7/288: dwrite d6/d10/d1e/d2d/f5f [4194304,4194304] 0 2024-09-10T11:29:27.488 INFO:tasks.workunit.client.1.smithi178.stdout:5/398: dread d9/dc/d17/f3d [0,4194304] 0 2024-09-10T11:29:27.490 INFO:tasks.workunit.client.1.smithi178.stdout:8/416: fdatasync d5/db/d10/d20/f79 0 2024-09-10T11:29:27.490 INFO:tasks.workunit.client.1.smithi178.stdout:4/244: chown d0/d9/dc/c14 0 1 2024-09-10T11:29:27.490 INFO:tasks.workunit.client.1.smithi178.stdout:0/409: dwrite d0/dc/d1f/f74 [0,4194304] 0 2024-09-10T11:29:27.493 INFO:tasks.workunit.client.1.smithi178.stdout:1/322: rename d3/d9/d54/d3a/d5d to d3/d9/d54/d3a/d6e 0 2024-09-10T11:29:27.493 INFO:tasks.workunit.client.1.smithi178.stdout:3/342: dread f1 [0,4194304] 0 2024-09-10T11:29:27.495 INFO:tasks.workunit.client.1.smithi178.stdout:3/343: readlink db/d20/l30 0 2024-09-10T11:29:27.498 INFO:tasks.workunit.client.1.smithi178.stdout:1/323: read d3/d9/d21/d4b/d2b/d5f/d61/f2e [533535,130483] 0 2024-09-10T11:29:27.501 INFO:tasks.workunit.client.1.smithi178.stdout:1/324: chown d3/d9/d21/d4b/d2b/d5f/d61/l27 51076429 1 2024-09-10T11:29:27.502 INFO:tasks.workunit.client.1.smithi178.stdout:3/344: dwrite db/d20/d3d/f53 [0,4194304] 0 2024-09-10T11:29:27.503 INFO:tasks.workunit.client.1.smithi178.stdout:3/345: truncate db/f15 910623 0 2024-09-10T11:29:27.520 INFO:tasks.workunit.client.1.smithi178.stdout:7/289: symlink d6/d10/d1e/l64 0 2024-09-10T11:29:27.523 INFO:tasks.workunit.client.1.smithi178.stdout:0/410: truncate d0/dc/d1f/d2d/f42 5227047 0 2024-09-10T11:29:27.528 INFO:tasks.workunit.client.1.smithi178.stdout:3/346: mkdir db/d20/d3d/d56/d64 0 2024-09-10T11:29:27.528 INFO:tasks.workunit.client.1.smithi178.stdout:7/290: creat d6/d10/d1d/d22/d3b/f65 x:0 0 0 2024-09-10T11:29:27.529 INFO:tasks.workunit.client.1.smithi178.stdout:8/417: link d5/db/f88 d5/d30/d33/d75/f8b 0 2024-09-10T11:29:27.531 INFO:tasks.workunit.client.1.smithi178.stdout:0/411: unlink d0/dc/d39/d50/c1b 0 2024-09-10T11:29:27.532 INFO:tasks.workunit.client.1.smithi178.stdout:6/459: dwrite d3/d42/d57/f68 [0,4194304] 0 2024-09-10T11:29:27.533 INFO:tasks.workunit.client.1.smithi178.stdout:6/460: stat d3/d7/d8/d15/d1c/d2a/d18/d8e/d90 0 2024-09-10T11:29:27.534 INFO:tasks.workunit.client.1.smithi178.stdout:3/347: rename db/d20/f21 to db/d36/f65 0 2024-09-10T11:29:27.535 INFO:tasks.workunit.client.1.smithi178.stdout:3/348: chown db/d26/f3b 50 1 2024-09-10T11:29:27.537 INFO:tasks.workunit.client.1.smithi178.stdout:5/399: link d9/dc/dd/d16/f1c d9/dc/dd/d37/f8d 0 2024-09-10T11:29:27.537 INFO:tasks.workunit.client.1.smithi178.stdout:8/418: rmdir d5/db/d10/d70 39 2024-09-10T11:29:27.538 INFO:tasks.workunit.client.1.smithi178.stdout:0/412: rmdir d0/d12/d3f 39 2024-09-10T11:29:27.538 INFO:tasks.workunit.client.1.smithi178.stdout:8/419: dread - d5/db/d10/d20/f7b zero size 2024-09-10T11:29:27.540 INFO:tasks.workunit.client.1.smithi178.stdout:3/349: creat db/d20/d3d/d56/f66 x:0 0 0 2024-09-10T11:29:27.541 INFO:tasks.workunit.client.1.smithi178.stdout:8/420: mkdir d5/d30/d33/d8c 0 2024-09-10T11:29:27.542 INFO:tasks.workunit.client.1.smithi178.stdout:8/421: chown d5/d30/f87 1456 1 2024-09-10T11:29:27.543 INFO:tasks.workunit.client.1.smithi178.stdout:8/422: readlink d5/db/d10/d53/l85 0 2024-09-10T11:29:27.544 INFO:tasks.workunit.client.1.smithi178.stdout:3/350: rename db/c4c to db/d35/d58/c67 0 2024-09-10T11:29:27.547 INFO:tasks.workunit.client.1.smithi178.stdout:3/351: mkdir db/d26/d40/d68 0 2024-09-10T11:29:27.549 INFO:tasks.workunit.client.1.smithi178.stdout:3/352: stat db/d20/d3d 0 2024-09-10T11:29:27.552 INFO:tasks.workunit.client.1.smithi178.stdout:7/291: rmdir d6/d10/d1d/d22/d3b 39 2024-09-10T11:29:27.553 INFO:tasks.workunit.client.1.smithi178.stdout:6/461: dread d3/d7/d8/d15/d1c/f40 [0,4194304] 0 2024-09-10T11:29:27.555 INFO:tasks.workunit.client.1.smithi178.stdout:7/292: readlink d6/d10/l33 0 2024-09-10T11:29:27.555 INFO:tasks.workunit.client.1.smithi178.stdout:8/423: dwrite d5/f37 [0,4194304] 0 2024-09-10T11:29:27.555 INFO:tasks.workunit.client.1.smithi178.stdout:4/245: truncate d0/d8/df/f3b 67632 0 2024-09-10T11:29:27.560 INFO:tasks.workunit.client.1.smithi178.stdout:7/293: dread - d6/d10/d34/f3f zero size 2024-09-10T11:29:27.562 INFO:tasks.workunit.client.1.smithi178.stdout:5/400: truncate d9/dc/dd/d16/f27 3628499 0 2024-09-10T11:29:27.563 INFO:tasks.workunit.client.1.smithi178.stdout:4/246: fsync d0/d10/f48 0 2024-09-10T11:29:27.564 INFO:tasks.workunit.client.1.smithi178.stdout:3/353: dwrite db/f4f [0,4194304] 0 2024-09-10T11:29:27.565 INFO:tasks.workunit.client.1.smithi178.stdout:0/413: truncate d0/dc/d39/f55 1795899 0 2024-09-10T11:29:27.568 INFO:tasks.workunit.client.1.smithi178.stdout:5/401: truncate d9/dc/d17/d1d/f73 4446910 0 2024-09-10T11:29:27.570 INFO:tasks.workunit.client.1.smithi178.stdout:0/414: write d0/dc/d39/f68 [856359,28420] 0 2024-09-10T11:29:27.571 INFO:tasks.workunit.client.1.smithi178.stdout:7/294: dread d6/d10/d1e/d2d/d2b/f39 [0,4194304] 0 2024-09-10T11:29:27.572 INFO:tasks.workunit.client.1.smithi178.stdout:4/247: stat d0/d8/df/f3b 0 2024-09-10T11:29:27.573 INFO:tasks.workunit.client.1.smithi178.stdout:7/295: write d6/d10/d1e/d24/f36 [1040786,64937] 0 2024-09-10T11:29:27.574 INFO:tasks.workunit.client.1.smithi178.stdout:4/248: read d0/d21/f37 [99401,86033] 0 2024-09-10T11:29:27.574 INFO:tasks.workunit.client.1.smithi178.stdout:7/296: dread - d6/d10/f5c zero size 2024-09-10T11:29:27.574 INFO:tasks.workunit.client.1.smithi178.stdout:0/415: symlink d0/dc/d39/l84 0 2024-09-10T11:29:27.575 INFO:tasks.workunit.client.1.smithi178.stdout:6/462: truncate d3/d7/d8/d15/d1c/d20/d4b/f52 293050 0 2024-09-10T11:29:27.576 INFO:tasks.workunit.client.1.smithi178.stdout:4/249: fsync d0/d8/df/f29 0 2024-09-10T11:29:27.580 INFO:tasks.workunit.client.1.smithi178.stdout:7/297: truncate d6/d10/d1d/f27 1604787 0 2024-09-10T11:29:27.581 INFO:tasks.workunit.client.1.smithi178.stdout:8/424: getdents d5/d1d/d22/d76 0 2024-09-10T11:29:27.584 INFO:tasks.workunit.client.1.smithi178.stdout:8/425: chown d5/db/f7d 18314568 1 2024-09-10T11:29:27.584 INFO:tasks.workunit.client.1.smithi178.stdout:9/470: sync 2024-09-10T11:29:27.584 INFO:tasks.workunit.client.1.smithi178.stdout:5/402: dwrite d9/f32 [0,4194304] 0 2024-09-10T11:29:27.585 INFO:tasks.workunit.client.1.smithi178.stdout:6/463: mknod d3/d7/d8/d15/d1c/d20/d4b/cac 0 2024-09-10T11:29:27.586 INFO:tasks.workunit.client.1.smithi178.stdout:0/416: dwrite d0/dc/f2b [0,4194304] 0 2024-09-10T11:29:27.592 INFO:tasks.workunit.client.1.smithi178.stdout:7/298: creat d6/d10/d1e/d2d/f66 x:0 0 0 2024-09-10T11:29:27.594 INFO:tasks.workunit.client.1.smithi178.stdout:8/426: write d5/db/d1e/f45 [77021,15561] 0 2024-09-10T11:29:27.594 INFO:tasks.workunit.client.1.smithi178.stdout:7/299: readlink l5 0 2024-09-10T11:29:27.603 INFO:tasks.workunit.client.1.smithi178.stdout:3/354: dwrite db/f1a [0,4194304] 0 2024-09-10T11:29:27.605 INFO:tasks.workunit.client.1.smithi178.stdout:4/250: rename d0/d9/dc/c1d to d0/c55 0 2024-09-10T11:29:27.607 INFO:tasks.workunit.client.1.smithi178.stdout:0/417: mkdir d0/dc/d39/d7c/d85 0 2024-09-10T11:29:27.609 INFO:tasks.workunit.client.1.smithi178.stdout:3/355: fsync db/d20/f28 0 2024-09-10T11:29:27.609 INFO:tasks.workunit.client.1.smithi178.stdout:9/471: dwrite d2/d13/f3c [0,4194304] 0 2024-09-10T11:29:27.610 INFO:tasks.workunit.client.1.smithi178.stdout:4/251: dread - d0/d8/da/f31 zero size 2024-09-10T11:29:27.612 INFO:tasks.workunit.client.1.smithi178.stdout:7/300: mknod d6/d10/d1d/d22/d60/c67 0 2024-09-10T11:29:27.614 INFO:tasks.workunit.client.1.smithi178.stdout:5/403: dwrite d9/dc/d17/d1d/f54 [4194304,4194304] 0 2024-09-10T11:29:27.614 INFO:tasks.workunit.client.1.smithi178.stdout:0/418: chown d0/dc/d39/d4c/f24 16980949 1 2024-09-10T11:29:27.615 INFO:tasks.workunit.client.1.smithi178.stdout:8/427: dwrite d5/f37 [0,4194304] 0 2024-09-10T11:29:27.616 INFO:tasks.workunit.client.1.smithi178.stdout:3/356: write db/d20/d3d/f53 [2756547,51488] 0 2024-09-10T11:29:27.617 INFO:tasks.workunit.client.1.smithi178.stdout:7/301: fdatasync d6/d10/d1e/f26 0 2024-09-10T11:29:27.625 INFO:tasks.workunit.client.1.smithi178.stdout:7/302: fsync d6/d10/d34/f44 0 2024-09-10T11:29:27.625 INFO:tasks.workunit.client.1.smithi178.stdout:5/404: truncate d9/d19/f1a 4380000 0 2024-09-10T11:29:27.626 INFO:tasks.workunit.client.1.smithi178.stdout:0/419: write d0/d12/f51 [564590,124736] 0 2024-09-10T11:29:27.630 INFO:tasks.workunit.client.1.smithi178.stdout:6/464: dread d3/d7/d8/d19/f26 [0,4194304] 0 2024-09-10T11:29:27.631 INFO:tasks.workunit.client.1.smithi178.stdout:4/252: getdents d0/d8/da/d1a/d51 0 2024-09-10T11:29:27.634 INFO:tasks.workunit.client.1.smithi178.stdout:8/428: symlink d5/d1d/d22/d76/l8d 0 2024-09-10T11:29:27.635 INFO:tasks.workunit.client.1.smithi178.stdout:0/420: mknod d0/dc/d39/d50/d5e/c86 0 2024-09-10T11:29:27.638 INFO:tasks.workunit.client.1.smithi178.stdout:7/303: rename d6/d10/d1d/d22/d60/c67 to d6/d10/d61/c68 0 2024-09-10T11:29:27.638 INFO:tasks.workunit.client.1.smithi178.stdout:8/429: write d5/f69 [108670,130808] 0 2024-09-10T11:29:27.638 INFO:tasks.workunit.client.1.smithi178.stdout:3/357: dwrite db/d35/f5f [0,4194304] 0 2024-09-10T11:29:27.640 INFO:tasks.workunit.client.1.smithi178.stdout:9/472: dread d2/d13/d40/d4b/d88/f7b [0,4194304] 0 2024-09-10T11:29:27.641 INFO:tasks.workunit.client.1.smithi178.stdout:5/405: link d9/dc/f84 d9/dc/dd/d16/f8e 0 2024-09-10T11:29:27.645 INFO:tasks.workunit.client.1.smithi178.stdout:4/253: write d0/d9/f2d [232121,59408] 0 2024-09-10T11:29:27.645 INFO:tasks.workunit.client.1.smithi178.stdout:3/358: unlink db/f15 0 2024-09-10T11:29:27.648 INFO:tasks.workunit.client.1.smithi178.stdout:5/406: mknod d9/dc/dd/c8f 0 2024-09-10T11:29:27.649 INFO:tasks.workunit.client.1.smithi178.stdout:7/304: mknod d6/c69 0 2024-09-10T11:29:27.649 INFO:tasks.workunit.client.1.smithi178.stdout:6/465: dwrite d3/d7/d8/d15/d1c/f53 [0,4194304] 0 2024-09-10T11:29:27.649 INFO:tasks.workunit.client.1.smithi178.stdout:0/421: dwrite d0/d12/f51 [0,4194304] 0 2024-09-10T11:29:27.652 INFO:tasks.workunit.client.1.smithi178.stdout:5/407: chown d9/dc/dd/f2d 2339568 1 2024-09-10T11:29:27.652 INFO:tasks.workunit.client.1.smithi178.stdout:7/305: truncate d6/d10/d34/f4e 656176 0 2024-09-10T11:29:27.655 INFO:tasks.workunit.client.1.smithi178.stdout:0/422: dread d0/d12/f64 [0,4194304] 0 2024-09-10T11:29:27.655 INFO:tasks.workunit.client.1.smithi178.stdout:6/466: unlink d3/d7/d8/d15/d1c/f89 0 2024-09-10T11:29:27.655 INFO:tasks.workunit.client.1.smithi178.stdout:8/430: link d5/db/d10/d20/l71 d5/l8e 0 2024-09-10T11:29:27.658 INFO:tasks.workunit.client.1.smithi178.stdout:6/467: chown d3/d7/d8/d72/d75 23666757 1 2024-09-10T11:29:27.660 INFO:tasks.workunit.client.1.smithi178.stdout:6/468: write d3/d7/d8/d15/d1c/d2a/f2c [1882191,52961] 0 2024-09-10T11:29:27.663 INFO:tasks.workunit.client.1.smithi178.stdout:3/359: getdents db/d35 0 2024-09-10T11:29:27.665 INFO:tasks.workunit.client.1.smithi178.stdout:8/431: creat d5/d61/f8f x:0 0 0 2024-09-10T11:29:27.666 INFO:tasks.workunit.client.1.smithi178.stdout:2/552: sync 2024-09-10T11:29:27.666 INFO:tasks.workunit.client.1.smithi178.stdout:4/254: write d0/d9/fb [5521446,60339] 0 2024-09-10T11:29:27.668 INFO:tasks.workunit.client.1.smithi178.stdout:6/469: symlink d3/d7/d8/d15/d1c/d2a/d18/lad 0 2024-09-10T11:29:27.670 INFO:tasks.workunit.client.1.smithi178.stdout:3/360: chown db/d36/l38 316467170 1 2024-09-10T11:29:27.670 INFO:tasks.workunit.client.1.smithi178.stdout:5/408: dwrite d9/d11/f14 [0,4194304] 0 2024-09-10T11:29:27.671 INFO:tasks.workunit.client.1.smithi178.stdout:8/432: symlink d5/d30/d33/d75/l90 0 2024-09-10T11:29:27.672 INFO:tasks.workunit.client.1.smithi178.stdout:6/470: chown d3/d7/d8/d15/d48/c49 250 1 2024-09-10T11:29:27.673 INFO:tasks.workunit.client.1.smithi178.stdout:5/409: chown d9/dc/dd 17000 1 2024-09-10T11:29:27.675 INFO:tasks.workunit.client.1.smithi178.stdout:0/423: creat d0/d12/f87 x:0 0 0 2024-09-10T11:29:27.675 INFO:tasks.workunit.client.1.smithi178.stdout:9/473: dread d2/f26 [0,4194304] 0 2024-09-10T11:29:27.675 INFO:tasks.workunit.client.1.smithi178.stdout:3/361: read - db/d20/d3d/d56/f66 zero size 2024-09-10T11:29:27.675 INFO:tasks.workunit.client.1.smithi178.stdout:4/255: fsync d0/d8/df/f49 0 2024-09-10T11:29:27.677 INFO:tasks.workunit.client.1.smithi178.stdout:2/553: fdatasync d1/f2e 0 2024-09-10T11:29:27.678 INFO:tasks.workunit.client.1.smithi178.stdout:6/471: chown d3/d7/d8/d15/d1c/d20/f47 22397 1 2024-09-10T11:29:27.678 INFO:tasks.workunit.client.1.smithi178.stdout:9/474: chown d2/d19/d5e 1093 1 2024-09-10T11:29:27.679 INFO:tasks.workunit.client.1.smithi178.stdout:3/362: creat db/d26/d40/f69 x:0 0 0 2024-09-10T11:29:27.680 INFO:tasks.workunit.client.1.smithi178.stdout:4/256: write d0/d9/fb [3868834,10607] 0 2024-09-10T11:29:27.682 INFO:tasks.workunit.client.1.smithi178.stdout:0/424: dread d0/dc/d39/f5b [0,4194304] 0 2024-09-10T11:29:27.683 INFO:tasks.workunit.client.1.smithi178.stdout:2/554: chown d1/d7/d24/d7d/d84/f37 0 1 2024-09-10T11:29:27.688 INFO:tasks.workunit.client.1.smithi178.stdout:7/306: dread d6/d10/d1e/d2d/f21 [0,4194304] 0 2024-09-10T11:29:27.690 INFO:tasks.workunit.client.1.smithi178.stdout:4/257: creat d0/d8/f56 x:0 0 0 2024-09-10T11:29:27.691 INFO:tasks.workunit.client.1.smithi178.stdout:1/325: sync 2024-09-10T11:29:27.696 INFO:tasks.workunit.client.1.smithi178.stdout:0/425: write d0/d12/f27 [4179889,67230] 0 2024-09-10T11:29:27.698 INFO:tasks.workunit.client.1.smithi178.stdout:4/258: dread - d0/d10/f32 zero size 2024-09-10T11:29:27.700 INFO:tasks.workunit.client.1.smithi178.stdout:2/555: write d1/d7/d24/d46/d88/d9d/dab/f29 [4418405,54699] 0 2024-09-10T11:29:27.702 INFO:tasks.workunit.client.1.smithi178.stdout:3/363: dwrite db/d20/d3d/d56/f66 [0,4194304] 0 2024-09-10T11:29:27.703 INFO:tasks.workunit.client.1.smithi178.stdout:1/326: mkdir d3/d9/d21/d6f 0 2024-09-10T11:29:27.706 INFO:tasks.workunit.client.1.smithi178.stdout:8/433: dwrite d5/db/d10/d20/f2c [0,4194304] 0 2024-09-10T11:29:27.713 INFO:tasks.workunit.client.1.smithi178.stdout:3/364: dread db/d36/f50 [0,4194304] 0 2024-09-10T11:29:27.713 INFO:tasks.workunit.client.1.smithi178.stdout:9/475: dread d2/d13/f86 [0,4194304] 0 2024-09-10T11:29:27.714 INFO:tasks.workunit.client.1.smithi178.stdout:5/410: dread d9/f51 [0,4194304] 0 2024-09-10T11:29:27.715 INFO:tasks.workunit.client.1.smithi178.stdout:3/365: stat l6 0 2024-09-10T11:29:27.716 INFO:tasks.workunit.client.1.smithi178.stdout:4/259: dread d0/d9/f2d [0,4194304] 0 2024-09-10T11:29:27.716 INFO:tasks.workunit.client.1.smithi178.stdout:2/556: mkdir d1/d7/d24/d46/d88/d9d/dab/d5e/db3 0 2024-09-10T11:29:27.716 INFO:tasks.workunit.client.1.smithi178.stdout:8/434: dread - d5/db/d1e/f5e zero size 2024-09-10T11:29:27.719 INFO:tasks.workunit.client.1.smithi178.stdout:6/472: dread d3/d42/f66 [0,4194304] 0 2024-09-10T11:29:27.721 INFO:tasks.workunit.client.1.smithi178.stdout:7/307: dwrite d6/d10/d1d/d22/d50/f59 [0,4194304] 0 2024-09-10T11:29:27.721 INFO:tasks.workunit.client.1.smithi178.stdout:9/476: mkdir d2/d13/d40/d89 0 2024-09-10T11:29:27.721 INFO:tasks.workunit.client.1.smithi178.stdout:3/366: write db/f4f [1723967,57768] 0 2024-09-10T11:29:27.722 INFO:tasks.workunit.client.1.smithi178.stdout:8/435: fsync d5/db/d1e/f39 0 2024-09-10T11:29:27.727 INFO:tasks.workunit.client.1.smithi178.stdout:6/473: fsync d3/d7/d8/d15/d1c/f1d 0 2024-09-10T11:29:27.727 INFO:tasks.workunit.client.1.smithi178.stdout:7/308: readlink d6/d10/d1e/d24/d49/l63 0 2024-09-10T11:29:27.728 INFO:tasks.workunit.client.1.smithi178.stdout:9/477: fsync d2/df/d31/f76 0 2024-09-10T11:29:27.733 INFO:tasks.workunit.client.1.smithi178.stdout:0/426: dwrite d0/dc/d39/f5b [0,4194304] 0 2024-09-10T11:29:27.733 INFO:tasks.workunit.client.1.smithi178.stdout:1/327: dwrite d3/d9/d21/d4b/d2b/d5f/d61/f45 [0,4194304] 0 2024-09-10T11:29:27.733 INFO:tasks.workunit.client.1.smithi178.stdout:7/309: dread - d6/d10/d1d/d4a/f62 zero size 2024-09-10T11:29:27.738 INFO:tasks.workunit.client.1.smithi178.stdout:4/260: truncate d0/f5 81706 0 2024-09-10T11:29:27.739 INFO:tasks.workunit.client.1.smithi178.stdout:9/478: read d2/df/d31/d37/f5f [3663217,39109] 0 2024-09-10T11:29:27.742 INFO:tasks.workunit.client.1.smithi178.stdout:8/436: creat d5/d30/d33/d75/f91 x:0 0 0 2024-09-10T11:29:27.742 INFO:tasks.workunit.client.1.smithi178.stdout:3/367: mkdir db/d20/d3d/d63/d6a 0 2024-09-10T11:29:27.744 INFO:tasks.workunit.client.1.smithi178.stdout:1/328: unlink d3/d9/d54/f56 0 2024-09-10T11:29:27.744 INFO:tasks.workunit.client.1.smithi178.stdout:2/557: dwrite d1/d7/d24/d7d/d84/d4b/f66 [0,4194304] 0 2024-09-10T11:29:27.745 INFO:tasks.workunit.client.1.smithi178.stdout:9/479: dread d2/f3f [0,4194304] 0 2024-09-10T11:29:27.745 INFO:tasks.workunit.client.1.smithi178.stdout:4/261: read - d0/d10/f2c zero size 2024-09-10T11:29:27.747 INFO:tasks.workunit.client.1.smithi178.stdout:7/310: rename d6/d10/d1d/d22/d60 to d6/d10/d34/d6a 0 2024-09-10T11:29:27.750 INFO:tasks.workunit.client.1.smithi178.stdout:3/368: rmdir db/d26/d40 39 2024-09-10T11:29:27.752 INFO:tasks.workunit.client.1.smithi178.stdout:8/437: dread d5/db/d1e/f45 [0,4194304] 0 2024-09-10T11:29:27.752 INFO:tasks.workunit.client.1.smithi178.stdout:7/311: fsync d6/f30 0 2024-09-10T11:29:27.755 INFO:tasks.workunit.client.1.smithi178.stdout:1/329: mknod d3/d9/d1a/d48/c70 0 2024-09-10T11:29:27.756 INFO:tasks.workunit.client.1.smithi178.stdout:4/262: creat d0/d8/da/d1a/f57 x:0 0 0 2024-09-10T11:29:27.756 INFO:tasks.workunit.client.1.smithi178.stdout:9/480: truncate d2/d19/d1c/f24 13356485 0 2024-09-10T11:29:27.757 INFO:tasks.workunit.client.1.smithi178.stdout:3/369: write db/d20/d3d/f51 [1119744,47173] 0 2024-09-10T11:29:27.758 INFO:tasks.workunit.client.1.smithi178.stdout:8/438: rmdir d5/db/d10/d20/d52 39 2024-09-10T11:29:27.759 INFO:tasks.workunit.client.1.smithi178.stdout:5/411: dwrite d9/dc/dd/d16/f1c [0,4194304] 0 2024-09-10T11:29:27.761 INFO:tasks.workunit.client.1.smithi178.stdout:0/427: dwrite d0/dc/f2b [0,4194304] 0 2024-09-10T11:29:27.761 INFO:tasks.workunit.client.1.smithi178.stdout:7/312: chown d6/d10/d1e/d24/d49/l4f 53608054 1 2024-09-10T11:29:27.764 INFO:tasks.workunit.client.1.smithi178.stdout:1/330: chown d3/d9/d21/d4b/d2b/d5f/d61/f43 3 1 2024-09-10T11:29:27.764 INFO:tasks.workunit.client.1.smithi178.stdout:3/370: read db/d26/f2c [3599406,4481] 0 2024-09-10T11:29:27.765 INFO:tasks.workunit.client.1.smithi178.stdout:2/558: dread d1/d7/d24/d46/d88/d9d/dab/f15 [0,4194304] 0 2024-09-10T11:29:27.765 INFO:tasks.workunit.client.1.smithi178.stdout:5/412: write d9/d19/f67 [1853102,21080] 0 2024-09-10T11:29:27.765 INFO:tasks.workunit.client.1.smithi178.stdout:9/481: mknod d2/d19/d1c/d2d/c8a 0 2024-09-10T11:29:27.771 INFO:tasks.workunit.client.1.smithi178.stdout:7/313: mknod d6/c6b 0 2024-09-10T11:29:27.772 INFO:tasks.workunit.client.1.smithi178.stdout:7/314: write d6/d10/d1e/d2d/d2b/f41 [318808,34108] 0 2024-09-10T11:29:27.772 INFO:tasks.workunit.client.1.smithi178.stdout:2/559: dread d1/f69 [0,4194304] 0 2024-09-10T11:29:27.774 INFO:tasks.workunit.client.1.smithi178.stdout:8/439: creat d5/db/d10/d53/d82/f92 x:0 0 0 2024-09-10T11:29:27.777 INFO:tasks.workunit.client.1.smithi178.stdout:9/482: mknod d2/c8b 0 2024-09-10T11:29:27.778 INFO:tasks.workunit.client.1.smithi178.stdout:9/483: chown d2/df/d31/d37/d3d/f6e 267 1 2024-09-10T11:29:27.781 INFO:tasks.workunit.client.1.smithi178.stdout:1/331: fdatasync d3/d9/d54/d2a/f34 0 2024-09-10T11:29:27.782 INFO:tasks.workunit.client.1.smithi178.stdout:2/560: creat d1/d7/d24/d7d/d84/d55/fb4 x:0 0 0 2024-09-10T11:29:27.783 INFO:tasks.workunit.client.1.smithi178.stdout:2/561: stat d1/d41/d5d 0 2024-09-10T11:29:27.784 INFO:tasks.workunit.client.1.smithi178.stdout:5/413: creat d9/f90 x:0 0 0 2024-09-10T11:29:27.786 INFO:tasks.workunit.client.1.smithi178.stdout:0/428: dwrite d0/d12/f2f [0,4194304] 0 2024-09-10T11:29:27.787 INFO:tasks.workunit.client.1.smithi178.stdout:8/440: write d5/db/d10/d20/d52/f68 [1033052,119627] 0 2024-09-10T11:29:27.791 INFO:tasks.workunit.client.1.smithi178.stdout:8/441: readlink d5/db/d10/l58 0 2024-09-10T11:29:27.792 INFO:tasks.workunit.client.1.smithi178.stdout:3/371: write db/fe [748153,99802] 0 2024-09-10T11:29:27.792 INFO:tasks.workunit.client.1.smithi178.stdout:5/414: unlink d9/dc/d17/d1d/l76 0 2024-09-10T11:29:27.793 INFO:tasks.workunit.client.1.smithi178.stdout:2/562: write d1/d7/d24/d46/d88/d9d/dab/d5e/f96 [792589,60611] 0 2024-09-10T11:29:27.793 INFO:tasks.workunit.client.1.smithi178.stdout:9/484: dwrite d2/d13/f43 [0,4194304] 0 2024-09-10T11:29:27.798 INFO:tasks.workunit.client.1.smithi178.stdout:9/485: write d2/d13/f3c [2904429,54698] 0 2024-09-10T11:29:27.803 INFO:tasks.workunit.client.1.smithi178.stdout:1/332: dread d3/d9/d54/d3a/f4e [0,4194304] 0 2024-09-10T11:29:27.803 INFO:tasks.workunit.client.1.smithi178.stdout:9/486: chown d2/dc/l6c 47898 1 2024-09-10T11:29:27.804 INFO:tasks.workunit.client.1.smithi178.stdout:7/315: dwrite d6/f23 [0,4194304] 0 2024-09-10T11:29:27.804 INFO:tasks.workunit.client.1.smithi178.stdout:3/372: mkdir db/d20/d3e/d6b 0 2024-09-10T11:29:27.805 INFO:tasks.workunit.client.1.smithi178.stdout:7/316: read - d6/d10/f5c zero size 2024-09-10T11:29:27.809 INFO:tasks.workunit.client.1.smithi178.stdout:9/487: unlink d2/d19/d1c/d2a/c62 0 2024-09-10T11:29:27.810 INFO:tasks.workunit.client.1.smithi178.stdout:1/333: symlink d3/d9/d54/d3a/d50/l71 0 2024-09-10T11:29:27.812 INFO:tasks.workunit.client.1.smithi178.stdout:0/429: creat d0/dc/d39/d50/f88 x:0 0 0 2024-09-10T11:29:27.812 INFO:tasks.workunit.client.1.smithi178.stdout:3/373: dwrite db/d20/f52 [0,4194304] 0 2024-09-10T11:29:27.816 INFO:tasks.workunit.client.1.smithi178.stdout:7/317: symlink d6/d10/d1d/d22/d50/l6c 0 2024-09-10T11:29:27.817 INFO:tasks.workunit.client.1.smithi178.stdout:9/488: fdatasync d2/d13/f43 0 2024-09-10T11:29:27.817 INFO:tasks.workunit.client.1.smithi178.stdout:2/563: truncate d1/d7/f6c 2007503 0 2024-09-10T11:29:27.819 INFO:tasks.workunit.client.1.smithi178.stdout:0/430: mkdir d0/d2c/d89 0 2024-09-10T11:29:27.819 INFO:tasks.workunit.client.1.smithi178.stdout:2/564: fsync d1/d7/d24/f2c 0 2024-09-10T11:29:27.819 INFO:tasks.workunit.client.1.smithi178.stdout:3/374: fsync db/ff 0 2024-09-10T11:29:27.821 INFO:tasks.workunit.client.1.smithi178.stdout:7/318: unlink d6/d10/d1e/l5e 0 2024-09-10T11:29:27.822 INFO:tasks.workunit.client.1.smithi178.stdout:1/334: mknod d3/d9/c72 0 2024-09-10T11:29:27.822 INFO:tasks.workunit.client.1.smithi178.stdout:8/442: getdents d5/db/d10/d70 0 2024-09-10T11:29:27.823 INFO:tasks.workunit.client.1.smithi178.stdout:8/443: fdatasync d5/d1d/f50 0 2024-09-10T11:29:27.824 INFO:tasks.workunit.client.1.smithi178.stdout:9/489: mknod d2/df/d31/d37/c8c 0 2024-09-10T11:29:27.826 INFO:tasks.workunit.client.1.smithi178.stdout:0/431: creat d0/dc/d39/d4c/d43/f8a x:0 0 0 2024-09-10T11:29:27.829 INFO:tasks.workunit.client.1.smithi178.stdout:2/565: write d1/d7/d24/d46/d88/d9d/dab/d5b/d60/f64 [3960593,67435] 0 2024-09-10T11:29:27.830 INFO:tasks.workunit.client.1.smithi178.stdout:7/319: unlink d6/f23 0 2024-09-10T11:29:27.830 INFO:tasks.workunit.client.1.smithi178.stdout:2/566: fdatasync d1/d41/d5d/f71 0 2024-09-10T11:29:27.831 INFO:tasks.workunit.client.1.smithi178.stdout:1/335: creat d3/d9/d21/d4b/d2b/d5f/f73 x:0 0 0 2024-09-10T11:29:27.832 INFO:tasks.workunit.client.1.smithi178.stdout:8/444: symlink d5/db/d10/d53/d82/l93 0 2024-09-10T11:29:27.833 INFO:tasks.workunit.client.1.smithi178.stdout:3/375: creat db/d20/d3d/d63/d6a/f6c x:0 0 0 2024-09-10T11:29:27.834 INFO:tasks.workunit.client.1.smithi178.stdout:8/445: stat d5/db/f88 0 2024-09-10T11:29:27.837 INFO:tasks.workunit.client.1.smithi178.stdout:9/490: dread d2/d19/d1c/f22 [0,4194304] 0 2024-09-10T11:29:27.837 INFO:tasks.workunit.client.1.smithi178.stdout:1/336: fdatasync d3/d9/d54/f51 0 2024-09-10T11:29:27.838 INFO:tasks.workunit.client.1.smithi178.stdout:3/376: rename db/d20/d3d/l41 to db/d20/d3e/l6d 0 2024-09-10T11:29:27.838 INFO:tasks.workunit.client.1.smithi178.stdout:9/491: chown d2/f26 3491966 1 2024-09-10T11:29:27.840 INFO:tasks.workunit.client.1.smithi178.stdout:5/415: dread d9/dc/dd/d37/d42/f4e [0,4194304] 0 2024-09-10T11:29:27.841 INFO:tasks.workunit.client.1.smithi178.stdout:7/320: creat d6/d10/d1d/d22/d3b/f6d x:0 0 0 2024-09-10T11:29:27.842 INFO:tasks.workunit.client.1.smithi178.stdout:1/337: creat d3/d35/f74 x:0 0 0 2024-09-10T11:29:27.842 INFO:tasks.workunit.client.1.smithi178.stdout:6/474: sync 2024-09-10T11:29:27.843 INFO:tasks.workunit.client.1.smithi178.stdout:7/321: fdatasync d6/d10/d1d/d22/d50/f59 0 2024-09-10T11:29:27.846 INFO:tasks.workunit.client.1.smithi178.stdout:0/432: dread d0/dc/d39/d50/f71 [0,4194304] 0 2024-09-10T11:29:27.848 INFO:tasks.workunit.client.1.smithi178.stdout:0/433: read d0/dc/d39/d4c/f24 [746875,24632] 0 2024-09-10T11:29:27.851 INFO:tasks.workunit.client.1.smithi178.stdout:7/322: rename d6/d29/f42 to d6/d10/d34/f6e 0 2024-09-10T11:29:27.853 INFO:tasks.workunit.client.1.smithi178.stdout:6/475: symlink d3/d6e/d87/lae 0 2024-09-10T11:29:27.854 INFO:tasks.workunit.client.1.smithi178.stdout:8/446: dread d5/d30/f87 [0,4194304] 0 2024-09-10T11:29:27.854 INFO:tasks.workunit.client.1.smithi178.stdout:1/338: creat d3/d9/d21/d6f/f75 x:0 0 0 2024-09-10T11:29:27.854 INFO:tasks.workunit.client.1.smithi178.stdout:7/323: write d6/d10/d1e/d2d/d2b/f41 [348566,94348] 0 2024-09-10T11:29:27.855 INFO:tasks.workunit.client.1.smithi178.stdout:2/567: fsync d1/d7/f6c 0 2024-09-10T11:29:27.857 INFO:tasks.workunit.client.1.smithi178.stdout:9/492: write d2/df/d31/d37/d3d/f6e [430778,84848] 0 2024-09-10T11:29:27.859 INFO:tasks.workunit.client.1.smithi178.stdout:0/434: dwrite d0/dc/d1f/d2d/f42 [0,4194304] 0 2024-09-10T11:29:27.859 INFO:tasks.workunit.client.1.smithi178.stdout:5/416: truncate d9/f5e 810346 0 2024-09-10T11:29:27.861 INFO:tasks.workunit.client.1.smithi178.stdout:6/476: rmdir d3/d7/d8/d15/d1c/d2a 39 2024-09-10T11:29:27.864 INFO:tasks.workunit.client.1.smithi178.stdout:7/324: dread d6/d10/d1e/d2d/d2b/f41 [0,4194304] 0 2024-09-10T11:29:27.866 INFO:tasks.workunit.client.1.smithi178.stdout:3/377: dwrite db/d36/f50 [0,4194304] 0 2024-09-10T11:29:27.867 INFO:tasks.workunit.client.1.smithi178.stdout:9/493: truncate d2/f18 3335630 0 2024-09-10T11:29:27.867 INFO:tasks.workunit.client.1.smithi178.stdout:7/325: write d6/d10/d1d/d22/d3b/f65 [905788,57726] 0 2024-09-10T11:29:27.869 INFO:tasks.workunit.client.1.smithi178.stdout:9/494: chown d2/d19/d1c/c7c 768887 1 2024-09-10T11:29:27.871 INFO:tasks.workunit.client.1.smithi178.stdout:5/417: dread d9/d11/f14 [0,4194304] 0 2024-09-10T11:29:27.873 INFO:tasks.workunit.client.1.smithi178.stdout:6/477: creat d3/d7/d8/d72/faf x:0 0 0 2024-09-10T11:29:27.875 INFO:tasks.workunit.client.1.smithi178.stdout:7/326: mkdir d6/d10/d34/d6a/d6f 0 2024-09-10T11:29:27.876 INFO:tasks.workunit.client.1.smithi178.stdout:9/495: creat d2/d13/d75/d60/f8d x:0 0 0 2024-09-10T11:29:27.877 INFO:tasks.workunit.client.1.smithi178.stdout:4/263: sync 2024-09-10T11:29:27.880 INFO:tasks.workunit.client.1.smithi178.stdout:5/418: mkdir d9/d19/d91 0 2024-09-10T11:29:27.884 INFO:tasks.workunit.client.1.smithi178.stdout:0/435: dwrite d0/dc/d39/d4c/f24 [0,4194304] 0 2024-09-10T11:29:27.884 INFO:tasks.workunit.client.1.smithi178.stdout:9/496: fdatasync d2/dc/f2f 0 2024-09-10T11:29:27.885 INFO:tasks.workunit.client.1.smithi178.stdout:3/378: write db/f27 [1280570,17626] 0 2024-09-10T11:29:27.885 INFO:tasks.workunit.client.1.smithi178.stdout:4/264: mknod d0/d8/da/c58 0 2024-09-10T11:29:27.886 INFO:tasks.workunit.client.1.smithi178.stdout:0/436: chown d0/d2c/l54 56 1 2024-09-10T11:29:27.887 INFO:tasks.workunit.client.1.smithi178.stdout:8/447: getdents d5/db/d3d 0 2024-09-10T11:29:27.889 INFO:tasks.workunit.client.1.smithi178.stdout:1/339: getdents d3/d9/d54/d3a/d63 0 2024-09-10T11:29:27.890 INFO:tasks.workunit.client.1.smithi178.stdout:5/419: rename d9/d19/f2c to d9/d19/f92 0 2024-09-10T11:29:27.890 INFO:tasks.workunit.client.1.smithi178.stdout:8/448: dread - d5/db/d10/d70/f78 zero size 2024-09-10T11:29:27.891 INFO:tasks.workunit.client.1.smithi178.stdout:6/478: rename d3/d7/d8/d15/f25 to d3/d7/fb0 0 2024-09-10T11:29:27.892 INFO:tasks.workunit.client.1.smithi178.stdout:5/420: chown d9/dc/dd/l83 2253873 1 2024-09-10T11:29:27.893 INFO:tasks.workunit.client.1.smithi178.stdout:3/379: creat db/d35/d58/f6e x:0 0 0 2024-09-10T11:29:27.894 INFO:tasks.workunit.client.1.smithi178.stdout:3/380: stat la 0 2024-09-10T11:29:27.894 INFO:tasks.workunit.client.1.smithi178.stdout:0/437: dread d0/dc/d39/f68 [0,4194304] 0 2024-09-10T11:29:27.895 INFO:tasks.workunit.client.1.smithi178.stdout:4/265: truncate d0/f18 839244 0 2024-09-10T11:29:27.895 INFO:tasks.workunit.client.1.smithi178.stdout:3/381: readlink db/d20/l23 0 2024-09-10T11:29:27.896 INFO:tasks.workunit.client.1.smithi178.stdout:2/568: dread d1/d7/f16 [0,4194304] 0 2024-09-10T11:29:27.897 INFO:tasks.workunit.client.1.smithi178.stdout:1/340: creat d3/d9/d54/d2a/f76 x:0 0 0 2024-09-10T11:29:27.897 INFO:tasks.workunit.client.1.smithi178.stdout:2/569: stat d1/d7/d24/d46/d88/d9d/dab/d5e 0 2024-09-10T11:29:27.898 INFO:tasks.workunit.client.1.smithi178.stdout:7/327: rename d6/d29 to d6/d10/d1e/d24/d49/d70 0 2024-09-10T11:29:27.900 INFO:tasks.workunit.client.1.smithi178.stdout:6/479: write d3/d7/d8/d15/d1c/d2a/d18/d8e/fa4 [900874,25317] 0 2024-09-10T11:29:27.905 INFO:tasks.workunit.client.1.smithi178.stdout:1/341: creat d3/d9/d54/d3a/d6e/f77 x:0 0 0 2024-09-10T11:29:27.906 INFO:tasks.workunit.client.1.smithi178.stdout:2/570: creat d1/d7/d24/d7d/fb5 x:0 0 0 2024-09-10T11:29:27.907 INFO:tasks.workunit.client.1.smithi178.stdout:2/571: chown d1/d22/d4d/d9b/c74 3 1 2024-09-10T11:29:27.908 INFO:tasks.workunit.client.1.smithi178.stdout:3/382: rename db/fe to db/d20/d3d/d63/f6f 0 2024-09-10T11:29:27.909 INFO:tasks.workunit.client.1.smithi178.stdout:7/328: mknod d6/d10/d1e/d24/c71 0 2024-09-10T11:29:27.909 INFO:tasks.workunit.client.1.smithi178.stdout:3/383: stat db/d20/d3d/f51 0 2024-09-10T11:29:27.911 INFO:tasks.workunit.client.1.smithi178.stdout:6/480: symlink d3/d7/d8/d72/lb1 0 2024-09-10T11:29:27.912 INFO:tasks.workunit.client.1.smithi178.stdout:9/497: link d2/d19/d1c/f2c d2/d19/f8e 0 2024-09-10T11:29:27.913 INFO:tasks.workunit.client.1.smithi178.stdout:9/498: chown d2/d51 200597505 1 2024-09-10T11:29:27.916 INFO:tasks.workunit.client.1.smithi178.stdout:4/266: rename d0/d8/da/f31 to d0/d8/df/f59 0 2024-09-10T11:29:27.919 INFO:tasks.workunit.client.1.smithi178.stdout:0/438: link d0/d12/c18 d0/dc/d39/d7c/d85/c8b 0 2024-09-10T11:29:27.919 INFO:tasks.workunit.client.1.smithi178.stdout:1/342: mkdir d3/d9/d54/d3a/d50/d6c/d78 0 2024-09-10T11:29:27.921 INFO:tasks.workunit.client.1.smithi178.stdout:7/329: dwrite d6/d10/d1d/d22/f3a [0,4194304] 0 2024-09-10T11:29:27.923 INFO:tasks.workunit.client.1.smithi178.stdout:4/267: truncate d0/d10/f32 883699 0 2024-09-10T11:29:27.925 INFO:tasks.workunit.client.1.smithi178.stdout:8/449: dwrite d5/f1f [0,4194304] 0 2024-09-10T11:29:27.928 INFO:tasks.workunit.client.1.smithi178.stdout:2/572: dread d1/d7/d24/d7d/d84/d55/f73 [0,4194304] 0 2024-09-10T11:29:27.929 INFO:tasks.workunit.client.1.smithi178.stdout:9/499: rename d2/d13/f48 to d2/d51/f8f 0 2024-09-10T11:29:27.930 INFO:tasks.workunit.client.1.smithi178.stdout:2/573: write d1/f2e [1681290,99966] 0 2024-09-10T11:29:27.931 INFO:tasks.workunit.client.1.smithi178.stdout:0/439: mkdir d0/dc/d39/d4c/d43/d8c 0 2024-09-10T11:29:27.931 INFO:tasks.workunit.client.1.smithi178.stdout:7/330: mknod d6/d10/d1e/d24/d49/c72 0 2024-09-10T11:29:27.932 INFO:tasks.workunit.client.1.smithi178.stdout:5/421: dwrite d9/f5e [0,4194304] 0 2024-09-10T11:29:27.933 INFO:tasks.workunit.client.1.smithi178.stdout:9/500: fdatasync d2/d19/d1c/d2a/f67 0 2024-09-10T11:29:27.933 INFO:tasks.workunit.client.1.smithi178.stdout:4/268: creat d0/d10/f5a x:0 0 0 2024-09-10T11:29:27.936 INFO:tasks.workunit.client.1.smithi178.stdout:6/481: truncate d3/d24/f30 10652167 0 2024-09-10T11:29:27.942 INFO:tasks.workunit.client.1.smithi178.stdout:3/384: getdents db/d36 0 2024-09-10T11:29:27.943 INFO:tasks.workunit.client.1.smithi178.stdout:5/422: mknod d9/d11/c93 0 2024-09-10T11:29:27.944 INFO:tasks.workunit.client.1.smithi178.stdout:9/501: unlink d2/df/c61 0 2024-09-10T11:29:27.944 INFO:tasks.workunit.client.1.smithi178.stdout:2/574: rename d1/f13 to d1/d7/d24/d7d/d84/fb6 0 2024-09-10T11:29:27.946 INFO:tasks.workunit.client.1.smithi178.stdout:8/450: symlink d5/d1d/d22/l94 0 2024-09-10T11:29:27.948 INFO:tasks.workunit.client.1.smithi178.stdout:3/385: stat db/d35/d3a/d42/l59 0 2024-09-10T11:29:27.949 INFO:tasks.workunit.client.1.smithi178.stdout:0/440: symlink d0/dc/d39/d7c/d85/l8d 0 2024-09-10T11:29:27.949 INFO:tasks.workunit.client.1.smithi178.stdout:7/331: dwrite d6/d10/d1d/d4a/f62 [0,4194304] 0 2024-09-10T11:29:27.949 INFO:tasks.workunit.client.1.smithi178.stdout:5/423: dread d9/dc/d17/f44 [0,4194304] 0 2024-09-10T11:29:27.950 INFO:tasks.workunit.client.1.smithi178.stdout:1/343: write d3/d9/fd [4309686,107435] 0 2024-09-10T11:29:27.956 INFO:tasks.workunit.client.1.smithi178.stdout:9/502: link d2/d13/f1f d2/d19/d1c/d77/f90 0 2024-09-10T11:29:27.958 INFO:tasks.workunit.client.1.smithi178.stdout:4/269: link d0/cd d0/d9/dc/c5b 0 2024-09-10T11:29:27.961 INFO:tasks.workunit.client.1.smithi178.stdout:9/503: chown d2/df/d31 675 1 2024-09-10T11:29:27.964 INFO:tasks.workunit.client.1.smithi178.stdout:1/344: mknod d3/d9/d54/c79 0 2024-09-10T11:29:27.964 INFO:tasks.workunit.client.1.smithi178.stdout:8/451: rename d5/c11 to d5/db/d1e/c95 0 2024-09-10T11:29:27.968 INFO:tasks.workunit.client.1.smithi178.stdout:4/270: chown d0/d9/d3d/d4f/f52 1147600757 1 2024-09-10T11:29:27.970 INFO:tasks.workunit.client.1.smithi178.stdout:7/332: creat d6/d10/d1d/f73 x:0 0 0 2024-09-10T11:29:27.971 INFO:tasks.workunit.client.1.smithi178.stdout:2/575: dwrite d1/d41/d5d/fa9 [0,4194304] 0 2024-09-10T11:29:27.971 INFO:tasks.workunit.client.1.smithi178.stdout:3/386: dwrite db/d20/d3d/f53 [0,4194304] 0 2024-09-10T11:29:27.972 INFO:tasks.workunit.client.1.smithi178.stdout:6/482: dread d3/d7/d8/d15/d1c/d20/d4b/f9b [0,4194304] 0 2024-09-10T11:29:27.974 INFO:tasks.workunit.client.1.smithi178.stdout:4/271: dread - d0/d9/d3d/f44 zero size 2024-09-10T11:29:27.975 INFO:tasks.workunit.client.1.smithi178.stdout:2/576: chown d1/d41/d8d/d9f 409447 1 2024-09-10T11:29:27.975 INFO:tasks.workunit.client.1.smithi178.stdout:0/441: truncate d0/dc/d39/d4c/f15 3208981 0 2024-09-10T11:29:27.977 INFO:tasks.workunit.client.1.smithi178.stdout:7/333: creat d6/d10/d34/f74 x:0 0 0 2024-09-10T11:29:27.978 INFO:tasks.workunit.client.1.smithi178.stdout:1/345: truncate d3/d9/d21/d4b/d2b/f57 710118 0 2024-09-10T11:29:27.978 INFO:tasks.workunit.client.1.smithi178.stdout:5/424: dwrite d9/dc/dd/d37/d42/f89 [0,4194304] 0 2024-09-10T11:29:27.985 INFO:tasks.workunit.client.1.smithi178.stdout:5/425: read d9/dc/dd/f2d [198615,31685] 0 2024-09-10T11:29:27.989 INFO:tasks.workunit.client.1.smithi178.stdout:9/504: dread d2/df/d31/f76 [0,4194304] 0 2024-09-10T11:29:27.993 INFO:tasks.workunit.client.1.smithi178.stdout:6/483: link d3/d42/d57/f68 d3/d7/d8/d15/d1c/d20/fb2 0 2024-09-10T11:29:27.994 INFO:tasks.workunit.client.1.smithi178.stdout:7/334: truncate d6/d10/d1e/d2d/f2c 878667 0 2024-09-10T11:29:27.995 INFO:tasks.workunit.client.1.smithi178.stdout:4/272: mknod d0/d8/da/d1a/d51/c5c 0 2024-09-10T11:29:27.995 INFO:tasks.workunit.client.1.smithi178.stdout:5/426: write d9/f32 [1983769,125737] 0 2024-09-10T11:29:27.996 INFO:tasks.workunit.client.1.smithi178.stdout:9/505: mknod d2/d19/d5e/c91 0 2024-09-10T11:29:27.997 INFO:tasks.workunit.client.1.smithi178.stdout:1/346: creat d3/d9/d21/d4b/d2b/d5f/d61/f7a x:0 0 0 2024-09-10T11:29:27.997 INFO:tasks.workunit.client.1.smithi178.stdout:2/577: dwrite d1/d22/f26 [0,4194304] 0 2024-09-10T11:29:28.001 INFO:tasks.workunit.client.1.smithi178.stdout:6/484: creat d3/d7/d8/d72/fb3 x:0 0 0 2024-09-10T11:29:28.001 INFO:tasks.workunit.client.1.smithi178.stdout:3/387: dwrite db/d20/d3d/d56/f66 [0,4194304] 0 2024-09-10T11:29:28.007 INFO:tasks.workunit.client.1.smithi178.stdout:0/442: dwrite d0/dc/d39/d4c/f57 [0,4194304] 0 2024-09-10T11:29:28.011 INFO:tasks.workunit.client.1.smithi178.stdout:3/388: mkdir db/d20/d70 0 2024-09-10T11:29:28.014 INFO:tasks.workunit.client.1.smithi178.stdout:6/485: dwrite d3/d7/d8/d15/d1c/d2a/d18/f70 [4194304,4194304] 0 2024-09-10T11:29:28.016 INFO:tasks.workunit.client.1.smithi178.stdout:6/486: write d3/d7/d8/d72/fb3 [547782,60755] 0 2024-09-10T11:29:28.021 INFO:tasks.workunit.client.1.smithi178.stdout:8/452: dread d5/db/d10/d20/f40 [0,4194304] 0 2024-09-10T11:29:28.023 INFO:tasks.workunit.client.1.smithi178.stdout:8/453: dread d5/db/d1e/f45 [0,4194304] 0 2024-09-10T11:29:28.025 INFO:tasks.workunit.client.1.smithi178.stdout:8/454: read d5/d1d/f25 [485522,123423] 0 2024-09-10T11:29:28.050 INFO:tasks.workunit.client.1.smithi178.stdout:2/578: dread d1/d22/d4d/d9b/f92 [4194304,4194304] 0 2024-09-10T11:29:28.055 INFO:tasks.workunit.client.1.smithi178.stdout:2/579: dwrite d1/d7/d24/d7d/d84/d55/fb4 [0,4194304] 0 2024-09-10T11:29:28.083 INFO:tasks.workunit.client.1.smithi178.stdout:2/580: dread d1/d22/d4d/d9b/f92 [0,4194304] 0 2024-09-10T11:29:28.150 INFO:tasks.workunit.client.1.smithi178.stdout:7/335: rename d6/d10/d1d/d22/d3b/l58 to d6/l75 0 2024-09-10T11:29:28.151 INFO:tasks.workunit.client.1.smithi178.stdout:1/347: truncate d3/d9/d54/d2a/f34 1334945 0 2024-09-10T11:29:28.152 INFO:tasks.workunit.client.1.smithi178.stdout:5/427: creat d9/dc/dd/d37/d3e/f94 x:0 0 0 2024-09-10T11:29:28.153 INFO:tasks.workunit.client.1.smithi178.stdout:5/428: write f2 [5895512,129743] 0 2024-09-10T11:29:28.155 INFO:tasks.workunit.client.1.smithi178.stdout:4/273: dwrite d0/f18 [0,4194304] 0 2024-09-10T11:29:28.156 INFO:tasks.workunit.client.1.smithi178.stdout:4/274: write d0/d10/f5a [140317,8830] 0 2024-09-10T11:29:28.211 INFO:tasks.workunit.client.1.smithi178.stdout:6/487: fdatasync d3/d7/d8/d15/d1c/d20/fb2 0 2024-09-10T11:29:28.212 INFO:tasks.workunit.client.1.smithi178.stdout:6/488: chown d3/d7/d8/d15/d1c 0 1 2024-09-10T11:29:28.214 INFO:tasks.workunit.client.1.smithi178.stdout:2/581: creat d1/d41/d8d/fb7 x:0 0 0 2024-09-10T11:29:28.214 INFO:tasks.workunit.client.1.smithi178.stdout:0/443: mknod d0/dc/d39/d4c/d43/d8c/c8e 0 2024-09-10T11:29:28.215 INFO:tasks.workunit.client.1.smithi178.stdout:7/336: mknod d6/d10/d1e/d24/d49/d70/c76 0 2024-09-10T11:29:28.216 INFO:tasks.workunit.client.1.smithi178.stdout:7/337: chown d6/d10/d1e/d2d/f21 31 1 2024-09-10T11:29:28.218 INFO:tasks.workunit.client.1.smithi178.stdout:1/348: creat d3/d9/d21/d4b/d2b/d5f/d61/f7b x:0 0 0 2024-09-10T11:29:28.220 INFO:tasks.workunit.client.1.smithi178.stdout:7/338: dread d6/d10/d1d/f27 [0,4194304] 0 2024-09-10T11:29:28.221 INFO:tasks.workunit.client.1.smithi178.stdout:4/275: mkdir d0/d8/da/d1a/d51/d5d 0 2024-09-10T11:29:28.222 INFO:tasks.workunit.client.1.smithi178.stdout:7/339: chown d6/d10/d1e/d24/d49/l63 45 1 2024-09-10T11:29:28.222 INFO:tasks.workunit.client.1.smithi178.stdout:9/506: link d2/f18 d2/d19/d1c/d2a/f92 0 2024-09-10T11:29:28.222 INFO:tasks.workunit.client.1.smithi178.stdout:9/507: write d2/d19/d5e/f6a [287124,19713] 0 2024-09-10T11:29:28.229 INFO:tasks.workunit.client.1.smithi178.stdout:8/455: mknod d5/db/d10/c96 0 2024-09-10T11:29:28.229 INFO:tasks.workunit.client.1.smithi178.stdout:2/582: symlink d1/d7/d24/d46/lb8 0 2024-09-10T11:29:28.229 INFO:tasks.workunit.client.1.smithi178.stdout:0/444: symlink d0/d2c/d33/l8f 0 2024-09-10T11:29:28.229 INFO:tasks.workunit.client.1.smithi178.stdout:2/583: dread d1/d7/f16 [4194304,4194304] 0 2024-09-10T11:29:28.229 INFO:tasks.workunit.client.1.smithi178.stdout:2/584: chown d1/d22 0 1 2024-09-10T11:29:28.230 INFO:tasks.workunit.client.1.smithi178.stdout:1/349: creat d3/d9/d54/d3a/d6e/f7c x:0 0 0 2024-09-10T11:29:28.231 INFO:tasks.workunit.client.1.smithi178.stdout:2/585: chown d1/d7/d24/d7d/d84/c9a 2508 1 2024-09-10T11:29:28.231 INFO:tasks.workunit.client.1.smithi178.stdout:4/276: write d0/d9/f2d [601589,23867] 0 2024-09-10T11:29:28.232 INFO:tasks.workunit.client.1.smithi178.stdout:2/586: chown d1/d22/d4d/d9b 7489 1 2024-09-10T11:29:28.232 INFO:tasks.workunit.client.1.smithi178.stdout:7/340: creat d6/d10/d61/f77 x:0 0 0 2024-09-10T11:29:28.232 INFO:tasks.workunit.client.1.smithi178.stdout:3/389: link db/l1c db/d35/d3a/d42/d46/l71 0 2024-09-10T11:29:28.233 INFO:tasks.workunit.client.1.smithi178.stdout:4/277: chown d0/d8/da/d1a/d51/c5c 9780 1 2024-09-10T11:29:28.233 INFO:tasks.workunit.client.1.smithi178.stdout:3/390: dread - db/d35/f5b zero size 2024-09-10T11:29:28.234 INFO:tasks.workunit.client.1.smithi178.stdout:6/489: creat d3/d42/d97/fb4 x:0 0 0 2024-09-10T11:29:28.236 INFO:tasks.workunit.client.1.smithi178.stdout:0/445: mkdir d0/dc/d39/d4c/d43/d90 0 2024-09-10T11:29:28.237 INFO:tasks.workunit.client.1.smithi178.stdout:5/429: rmdir d9/dc/d17/d4b 0 2024-09-10T11:29:28.240 INFO:tasks.workunit.client.1.smithi178.stdout:7/341: mknod d6/d10/d1d/d4a/c78 0 2024-09-10T11:29:28.241 INFO:tasks.workunit.client.1.smithi178.stdout:3/391: dwrite db/f39 [0,4194304] 0 2024-09-10T11:29:28.243 INFO:tasks.workunit.client.1.smithi178.stdout:6/490: dwrite d3/d7/d8/d15/d1c/d2a/f41 [0,4194304] 0 2024-09-10T11:29:28.245 INFO:tasks.workunit.client.1.smithi178.stdout:5/430: dread d9/dc/dd/d37/d3e/f78 [0,4194304] 0 2024-09-10T11:29:28.245 INFO:tasks.workunit.client.1.smithi178.stdout:0/446: rename d0/dc/d39/l63 to d0/d2c/d33/d41/l91 0 2024-09-10T11:29:28.253 INFO:tasks.workunit.client.1.smithi178.stdout:5/431: fsync d9/f51 0 2024-09-10T11:29:28.256 INFO:tasks.workunit.client.1.smithi178.stdout:7/342: dwrite d6/f14 [0,4194304] 0 2024-09-10T11:29:28.257 INFO:tasks.workunit.client.1.smithi178.stdout:6/491: mknod d3/cb5 0 2024-09-10T11:29:28.259 INFO:tasks.workunit.client.1.smithi178.stdout:3/392: truncate db/f1d 2352693 0 2024-09-10T11:29:28.261 INFO:tasks.workunit.client.1.smithi178.stdout:0/447: dwrite d0/d12/f64 [0,4194304] 0 2024-09-10T11:29:28.263 INFO:tasks.workunit.client.1.smithi178.stdout:5/432: mknod d9/c95 0 2024-09-10T11:29:28.267 INFO:tasks.workunit.client.1.smithi178.stdout:3/393: creat db/f72 x:0 0 0 2024-09-10T11:29:28.268 INFO:tasks.workunit.client.1.smithi178.stdout:1/350: truncate d3/d9/fd 1088320 0 2024-09-10T11:29:28.269 INFO:tasks.workunit.client.1.smithi178.stdout:2/587: write d1/d41/f1e [134256,75715] 0 2024-09-10T11:29:28.271 INFO:tasks.workunit.client.1.smithi178.stdout:5/433: rename d9/dc/dd/d37/l3b to d9/dc/dd/d37/d42/d62/l96 0 2024-09-10T11:29:28.275 INFO:tasks.workunit.client.1.smithi178.stdout:6/492: link d3/d7/d8/d15/d1c/f1d d3/d42/fb6 0 2024-09-10T11:29:28.275 INFO:tasks.workunit.client.1.smithi178.stdout:0/448: symlink d0/d2c/d89/l92 0 2024-09-10T11:29:28.278 INFO:tasks.workunit.client.1.smithi178.stdout:3/394: creat db/d20/d70/f73 x:0 0 0 2024-09-10T11:29:28.279 INFO:tasks.workunit.client.1.smithi178.stdout:5/434: dwrite d9/f90 [0,4194304] 0 2024-09-10T11:29:28.279 INFO:tasks.workunit.client.1.smithi178.stdout:1/351: mkdir d3/d7d 0 2024-09-10T11:29:28.280 INFO:tasks.workunit.client.1.smithi178.stdout:1/352: chown d3/d9/d54 30 1 2024-09-10T11:29:28.281 INFO:tasks.workunit.client.1.smithi178.stdout:7/343: getdents d6/d10 0 2024-09-10T11:29:28.284 INFO:tasks.workunit.client.1.smithi178.stdout:0/449: unlink d0/f9 0 2024-09-10T11:29:28.285 INFO:tasks.workunit.client.1.smithi178.stdout:6/493: symlink d3/d7/d35/lb7 0 2024-09-10T11:29:28.288 INFO:tasks.workunit.client.1.smithi178.stdout:6/494: truncate d3/d7/d8/d15/d1c/d2a/d18/d8e/fa4 1816976 0 2024-09-10T11:29:28.291 INFO:tasks.workunit.client.1.smithi178.stdout:5/435: rename d9/dc/f84 to d9/d19/f97 0 2024-09-10T11:29:28.292 INFO:tasks.workunit.client.1.smithi178.stdout:7/344: mkdir d6/d10/d1d/d4a/d79 0 2024-09-10T11:29:28.293 INFO:tasks.workunit.client.1.smithi178.stdout:7/345: write d6/d10/d1d/f31 [994763,60595] 0 2024-09-10T11:29:28.297 INFO:tasks.workunit.client.1.smithi178.stdout:3/395: mkdir db/d26/d74 0 2024-09-10T11:29:28.297 INFO:tasks.workunit.client.1.smithi178.stdout:6/495: creat d3/d7/d8/d15/d1c/d20/fb8 x:0 0 0 2024-09-10T11:29:28.298 INFO:tasks.workunit.client.1.smithi178.stdout:5/436: creat d9/dc/dd/d3f/f98 x:0 0 0 2024-09-10T11:29:28.298 INFO:tasks.workunit.client.1.smithi178.stdout:3/396: fdatasync db/d20/d3d/d63/d6a/f6c 0 2024-09-10T11:29:28.299 INFO:tasks.workunit.client.1.smithi178.stdout:2/588: dwrite d1/d7/d24/d7d/d84/f37 [0,4194304] 0 2024-09-10T11:29:28.301 INFO:tasks.workunit.client.1.smithi178.stdout:5/437: mknod d9/dc/dd/d16/c99 0 2024-09-10T11:29:28.304 INFO:tasks.workunit.client.1.smithi178.stdout:7/346: rename d6/d10/d1d/d22/d3b/f43 to d6/d10/d1e/d24/d49/d70/d4c/f7a 0 2024-09-10T11:29:28.307 INFO:tasks.workunit.client.1.smithi178.stdout:2/589: mkdir d1/d41/d14/d11/db9 0 2024-09-10T11:29:28.308 INFO:tasks.workunit.client.1.smithi178.stdout:5/438: dwrite d9/dc/dd/d37/d42/f89 [0,4194304] 0 2024-09-10T11:29:28.309 INFO:tasks.workunit.client.1.smithi178.stdout:5/439: fdatasync d9/dc/d17/f34 0 2024-09-10T11:29:28.310 INFO:tasks.workunit.client.1.smithi178.stdout:5/440: dread - d9/dc/dd/d37/d42/d62/f80 zero size 2024-09-10T11:29:28.313 INFO:tasks.workunit.client.0.smithi144.stdout:7/7: sync 2024-09-10T11:29:28.314 INFO:tasks.workunit.client.1.smithi178.stdout:7/347: creat d6/d10/d1e/d24/f7b x:0 0 0 2024-09-10T11:29:28.315 INFO:tasks.workunit.client.1.smithi178.stdout:7/348: chown d6/d10/d1e/d2d/l38 0 1 2024-09-10T11:29:28.315 INFO:tasks.workunit.client.1.smithi178.stdout:2/590: mknod d1/d7/d24/d46/d88/cba 0 2024-09-10T11:29:28.316 INFO:tasks.workunit.client.0.smithi144.stdout:7/8: creat f1 x:0 0 0 2024-09-10T11:29:28.318 INFO:tasks.workunit.client.1.smithi178.stdout:0/450: truncate d0/dc/d1f/f74 1595502 0 2024-09-10T11:29:28.318 INFO:tasks.workunit.client.0.smithi144.stdout:7/9: mknod c2 0 2024-09-10T11:29:28.319 INFO:tasks.workunit.client.0.smithi144.stdout:7/10: write f1 [423860,53936] 0 2024-09-10T11:29:28.324 INFO:tasks.workunit.client.0.smithi144.stdout:7/11: dwrite f1 [0,4194304] 0 2024-09-10T11:29:28.400 INFO:tasks.workunit.client.0.smithi144.stdout:5/8: sync 2024-09-10T11:29:28.400 INFO:tasks.workunit.client.0.smithi144.stdout:3/7: sync 2024-09-10T11:29:28.400 INFO:tasks.workunit.client.0.smithi144.stdout:2/18: sync 2024-09-10T11:29:28.401 INFO:tasks.workunit.client.1.smithi178.stdout:9/508: sync 2024-09-10T11:29:28.402 INFO:tasks.workunit.client.1.smithi178.stdout:7/349: mkdir d6/d10/d1e/d24/d49/d7c 0 2024-09-10T11:29:28.402 INFO:tasks.workunit.client.0.smithi144.stdout:5/9: mknod c1 0 2024-09-10T11:29:28.403 INFO:tasks.workunit.client.1.smithi178.stdout:3/397: link db/d20/l23 db/d20/d3d/d63/d6a/l75 0 2024-09-10T11:29:28.404 INFO:tasks.workunit.client.0.smithi144.stdout:2/19: creat d7/f8 x:0 0 0 2024-09-10T11:29:28.405 INFO:tasks.workunit.client.1.smithi178.stdout:2/591: mkdir d1/d7/d24/d46/d88/d9d/dab/dbb 0 2024-09-10T11:29:28.405 INFO:tasks.workunit.client.0.smithi144.stdout:5/10: symlink l2 0 2024-09-10T11:29:28.405 INFO:tasks.workunit.client.0.smithi144.stdout:5/11: dwrite - no filename 2024-09-10T11:29:28.406 INFO:tasks.workunit.client.0.smithi144.stdout:5/12: readlink l2 0 2024-09-10T11:29:28.407 INFO:tasks.workunit.client.0.smithi144.stdout:2/20: mknod d7/c9 0 2024-09-10T11:29:28.409 INFO:tasks.workunit.client.1.smithi178.stdout:7/350: dread d6/d10/d1e/f35 [0,4194304] 0 2024-09-10T11:29:28.409 INFO:tasks.workunit.client.0.smithi144.stdout:3/8: dwrite f3 [0,4194304] 0 2024-09-10T11:29:28.409 INFO:tasks.workunit.client.0.smithi144.stdout:5/13: symlink l3 0 2024-09-10T11:29:28.409 INFO:tasks.workunit.client.1.smithi178.stdout:7/351: dread - d6/d10/d1d/d4a/f57 zero size 2024-09-10T11:29:28.410 INFO:tasks.workunit.client.0.smithi144.stdout:5/14: chown c1 3 1 2024-09-10T11:29:28.410 INFO:tasks.workunit.client.0.smithi144.stdout:5/15: readlink l3 0 2024-09-10T11:29:28.410 INFO:tasks.workunit.client.0.smithi144.stdout:5/16: fdatasync - no filename 2024-09-10T11:29:28.411 INFO:tasks.workunit.client.0.smithi144.stdout:3/9: read f0 [1616024,117280] 0 2024-09-10T11:29:28.412 INFO:tasks.workunit.client.0.smithi144.stdout:5/17: mkdir d4 0 2024-09-10T11:29:28.412 INFO:tasks.workunit.client.1.smithi178.stdout:9/509: fsync d2/d19/d1c/d2a/f92 0 2024-09-10T11:29:28.414 INFO:tasks.workunit.client.0.smithi144.stdout:2/21: dread f6 [0,4194304] 0 2024-09-10T11:29:28.414 INFO:tasks.workunit.client.0.smithi144.stdout:3/10: mkdir d4 0 2024-09-10T11:29:28.414 INFO:tasks.workunit.client.1.smithi178.stdout:3/398: mknod db/d20/d70/c76 0 2024-09-10T11:29:28.415 INFO:tasks.workunit.client.1.smithi178.stdout:2/592: write d1/d41/f51 [1710165,31772] 0 2024-09-10T11:29:28.417 INFO:tasks.workunit.client.0.smithi144.stdout:5/18: unlink l3 0 2024-09-10T11:29:28.419 INFO:tasks.workunit.client.0.smithi144.stdout:2/22: mkdir d7/da 0 2024-09-10T11:29:28.420 INFO:tasks.workunit.client.0.smithi144.stdout:2/23: write d7/f8 [73599,129169] 0 2024-09-10T11:29:28.421 INFO:tasks.workunit.client.0.smithi144.stdout:3/11: mkdir d4/d5 0 2024-09-10T11:29:28.423 INFO:tasks.workunit.client.1.smithi178.stdout:9/510: write d2/d19/d1c/f22 [183153,5504] 0 2024-09-10T11:29:28.424 INFO:tasks.workunit.client.0.smithi144.stdout:3/12: unlink f0 0 2024-09-10T11:29:28.425 INFO:tasks.workunit.client.0.smithi144.stdout:5/19: mkdir d4/d5 0 2024-09-10T11:29:28.426 INFO:tasks.workunit.client.0.smithi144.stdout:5/20: chown d4 0 1 2024-09-10T11:29:28.426 INFO:tasks.workunit.client.0.smithi144.stdout:5/21: truncate - no filename 2024-09-10T11:29:28.426 INFO:tasks.workunit.client.0.smithi144.stdout:5/22: dwrite - no filename 2024-09-10T11:29:28.426 INFO:tasks.workunit.client.0.smithi144.stdout:5/23: fdatasync - no filename 2024-09-10T11:29:28.428 INFO:tasks.workunit.client.0.smithi144.stdout:3/13: symlink d4/l6 0 2024-09-10T11:29:28.429 INFO:tasks.workunit.client.0.smithi144.stdout:2/24: creat d7/da/fb x:0 0 0 2024-09-10T11:29:28.429 INFO:tasks.workunit.client.0.smithi144.stdout:2/25: fdatasync d7/da/fb 0 2024-09-10T11:29:28.433 INFO:tasks.workunit.client.0.smithi144.stdout:5/24: rename c1 to d4/d5/c6 0 2024-09-10T11:29:28.433 INFO:tasks.workunit.client.0.smithi144.stdout:5/25: read - no filename 2024-09-10T11:29:28.433 INFO:tasks.workunit.client.0.smithi144.stdout:5/26: rename d4 to d4/d5/d7 22 2024-09-10T11:29:28.433 INFO:tasks.workunit.client.0.smithi144.stdout:2/26: dread d7/f8 [0,4194304] 0 2024-09-10T11:29:28.433 INFO:tasks.workunit.client.0.smithi144.stdout:3/14: read f2 [842982,128644] 0 2024-09-10T11:29:28.438 INFO:tasks.workunit.client.0.smithi144.stdout:2/27: rename l4 to d7/da/lc 0 2024-09-10T11:29:28.442 INFO:tasks.workunit.client.0.smithi144.stdout:3/15: link d4/l6 d4/l7 0 2024-09-10T11:29:28.446 INFO:tasks.workunit.client.0.smithi144.stdout:3/16: rmdir d4/d5 0 2024-09-10T11:29:28.446 INFO:tasks.workunit.client.0.smithi144.stdout:3/17: read f2 [172230,77825] 0 2024-09-10T11:29:28.448 INFO:tasks.workunit.client.0.smithi144.stdout:3/18: symlink d4/l8 0 2024-09-10T11:29:28.451 INFO:tasks.workunit.client.0.smithi144.stdout:3/19: rename d4/l8 to d4/l9 0 2024-09-10T11:29:28.467 INFO:tasks.workunit.client.0.smithi144.stdout:7/12: dwrite f1 [4194304,4194304] 0 2024-09-10T11:29:28.474 INFO:tasks.workunit.client.1.smithi178.stdout:3/399: creat db/d35/f77 x:0 0 0 2024-09-10T11:29:28.476 INFO:tasks.workunit.client.1.smithi178.stdout:5/441: dwrite d9/d19/f97 [0,4194304] 0 2024-09-10T11:29:28.479 INFO:tasks.workunit.client.0.smithi144.stdout:5/27: rename d4/d5/c6 to d4/d5/c8 0 2024-09-10T11:29:28.480 INFO:tasks.workunit.client.0.smithi144.stdout:5/28: dwrite - no filename 2024-09-10T11:29:28.480 INFO:tasks.workunit.client.0.smithi144.stdout:5/29: chown d4/d5 1078889 1 2024-09-10T11:29:28.480 INFO:tasks.workunit.client.0.smithi144.stdout:5/30: write - no filename 2024-09-10T11:29:28.480 INFO:tasks.workunit.client.0.smithi144.stdout:5/31: dread - no filename 2024-09-10T11:29:28.481 INFO:tasks.workunit.client.1.smithi178.stdout:0/451: dwrite d0/dc/f13 [4194304,4194304] 0 2024-09-10T11:29:28.486 INFO:tasks.workunit.client.1.smithi178.stdout:7/352: dread d6/ff [4194304,4194304] 0 2024-09-10T11:29:28.487 INFO:tasks.workunit.client.0.smithi144.stdout:2/28: dwrite f6 [4194304,4194304] 0 2024-09-10T11:29:28.488 INFO:tasks.workunit.client.0.smithi144.stdout:2/29: truncate d7/da/fb 700523 0 2024-09-10T11:29:28.489 INFO:tasks.workunit.client.0.smithi144.stdout:2/30: fsync d7/da/fb 0 2024-09-10T11:29:28.490 INFO:tasks.workunit.client.1.smithi178.stdout:7/353: creat d6/d10/d1d/d4a/f7d x:0 0 0 2024-09-10T11:29:28.492 INFO:tasks.workunit.client.0.smithi144.stdout:5/32: mknod d4/d5/c9 0 2024-09-10T11:29:28.492 INFO:tasks.workunit.client.0.smithi144.stdout:5/33: dwrite - no filename 2024-09-10T11:29:28.493 INFO:tasks.workunit.client.0.smithi144.stdout:5/34: fdatasync - no filename 2024-09-10T11:29:28.493 INFO:tasks.workunit.client.0.smithi144.stdout:5/35: fdatasync - no filename 2024-09-10T11:29:28.493 INFO:tasks.workunit.client.0.smithi144.stdout:3/20: write f2 [815756,38228] 0 2024-09-10T11:29:28.496 INFO:tasks.workunit.client.0.smithi144.stdout:2/31: creat d7/da/fd x:0 0 0 2024-09-10T11:29:28.497 INFO:tasks.workunit.client.0.smithi144.stdout:2/32: write f6 [9326801,65872] 0 2024-09-10T11:29:28.497 INFO:tasks.workunit.client.1.smithi178.stdout:7/354: mkdir d6/d10/d1d/d22/d50/d7e 0 2024-09-10T11:29:28.499 INFO:tasks.workunit.client.1.smithi178.stdout:3/400: write db/f1d [878020,113146] 0 2024-09-10T11:29:28.500 INFO:tasks.workunit.client.1.smithi178.stdout:5/442: dread d9/f51 [0,4194304] 0 2024-09-10T11:29:28.501 INFO:tasks.workunit.client.1.smithi178.stdout:7/355: creat d6/d10/d34/d6a/f7f x:0 0 0 2024-09-10T11:29:28.501 INFO:tasks.workunit.client.1.smithi178.stdout:9/511: truncate d2/dc/f44 2461421 0 2024-09-10T11:29:28.506 INFO:tasks.workunit.client.0.smithi144.stdout:2/33: mkdir d7/de 0 2024-09-10T11:29:28.507 INFO:tasks.workunit.client.1.smithi178.stdout:7/356: rename d6/d10/d1e/d24/l46 to d6/d10/d1d/d4a/d79/l80 0 2024-09-10T11:29:28.507 INFO:tasks.workunit.client.0.smithi144.stdout:5/36: unlink d4/d5/c8 0 2024-09-10T11:29:28.508 INFO:tasks.workunit.client.0.smithi144.stdout:3/21: mkdir d4/da 0 2024-09-10T11:29:28.509 INFO:tasks.workunit.client.0.smithi144.stdout:2/34: creat d7/da/ff x:0 0 0 2024-09-10T11:29:28.511 INFO:tasks.workunit.client.1.smithi178.stdout:3/401: dwrite db/d20/d3d/f53 [4194304,4194304] 0 2024-09-10T11:29:28.511 INFO:tasks.workunit.client.1.smithi178.stdout:9/512: rename d2/d13/d40/d4b to d2/d13/d40/d89/d93 0 2024-09-10T11:29:28.512 INFO:tasks.workunit.client.1.smithi178.stdout:5/443: dwrite d9/dc/dd/d16/f8e [0,4194304] 0 2024-09-10T11:29:28.512 INFO:tasks.workunit.client.0.smithi144.stdout:5/37: creat d4/fa x:0 0 0 2024-09-10T11:29:28.512 INFO:tasks.workunit.client.0.smithi144.stdout:5/38: stat d4/d5 0 2024-09-10T11:29:28.513 INFO:tasks.workunit.client.1.smithi178.stdout:3/402: fdatasync db/d35/f5e 0 2024-09-10T11:29:28.514 INFO:tasks.workunit.client.0.smithi144.stdout:2/35: creat d7/f10 x:0 0 0 2024-09-10T11:29:28.514 INFO:tasks.workunit.client.0.smithi144.stdout:7/13: truncate f1 2277375 0 2024-09-10T11:29:28.516 INFO:tasks.workunit.client.0.smithi144.stdout:2/36: truncate d7/da/fd 181195 0 2024-09-10T11:29:28.517 INFO:tasks.workunit.client.1.smithi178.stdout:9/513: unlink d2/d19/d1c/f22 0 2024-09-10T11:29:28.520 INFO:tasks.workunit.client.1.smithi178.stdout:7/357: mknod d6/d10/d1e/c81 0 2024-09-10T11:29:28.521 INFO:tasks.workunit.client.0.smithi144.stdout:5/39: symlink d4/lb 0 2024-09-10T11:29:28.521 INFO:tasks.workunit.client.0.smithi144.stdout:7/14: symlink l3 0 2024-09-10T11:29:28.522 INFO:tasks.workunit.client.0.smithi144.stdout:7/15: readlink l3 0 2024-09-10T11:29:28.522 INFO:tasks.workunit.client.0.smithi144.stdout:5/40: chown l2 0 1 2024-09-10T11:29:28.524 INFO:tasks.workunit.client.1.smithi178.stdout:0/452: dread d0/d2c/f7b [0,4194304] 0 2024-09-10T11:29:28.524 INFO:tasks.workunit.client.1.smithi178.stdout:9/514: fsync d2/f18 0 2024-09-10T11:29:28.525 INFO:tasks.workunit.client.0.smithi144.stdout:7/16: creat f4 x:0 0 0 2024-09-10T11:29:28.525 INFO:tasks.workunit.client.1.smithi178.stdout:5/444: dwrite d9/f71 [0,4194304] 0 2024-09-10T11:29:28.527 INFO:tasks.workunit.client.0.smithi144.stdout:2/37: dwrite d7/da/fb [0,4194304] 0 2024-09-10T11:29:28.527 INFO:tasks.workunit.client.0.smithi144.stdout:5/41: symlink d4/d5/lc 0 2024-09-10T11:29:28.529 INFO:tasks.workunit.client.0.smithi144.stdout:5/42: truncate d4/fa 1015572 0 2024-09-10T11:29:28.529 INFO:tasks.workunit.client.0.smithi144.stdout:7/17: mkdir d5 0 2024-09-10T11:29:28.530 INFO:tasks.workunit.client.1.smithi178.stdout:3/403: mknod db/d20/d3d/d56/d64/c78 0 2024-09-10T11:29:28.530 INFO:tasks.workunit.client.0.smithi144.stdout:5/43: chown d4/lb 3199908 1 2024-09-10T11:29:28.531 INFO:tasks.workunit.client.0.smithi144.stdout:7/18: write f4 [965736,14798] 0 2024-09-10T11:29:28.532 INFO:tasks.workunit.client.0.smithi144.stdout:2/38: readlink d7/da/lc 0 2024-09-10T11:29:28.532 INFO:tasks.workunit.client.0.smithi144.stdout:5/44: dread d4/fa [0,4194304] 0 2024-09-10T11:29:28.533 INFO:tasks.workunit.client.1.smithi178.stdout:7/358: dwrite f3 [0,4194304] 0 2024-09-10T11:29:28.534 INFO:tasks.workunit.client.1.smithi178.stdout:0/453: rmdir d0/dc/d1f 39 2024-09-10T11:29:28.534 INFO:tasks.workunit.client.1.smithi178.stdout:3/404: dread - db/d20/d3d/d63/d6a/f6c zero size 2024-09-10T11:29:28.535 INFO:tasks.workunit.client.0.smithi144.stdout:2/39: creat d7/f11 x:0 0 0 2024-09-10T11:29:28.536 INFO:tasks.workunit.client.0.smithi144.stdout:5/45: symlink d4/ld 0 2024-09-10T11:29:28.537 INFO:tasks.workunit.client.1.smithi178.stdout:5/445: dread d9/f71 [0,4194304] 0 2024-09-10T11:29:28.538 INFO:tasks.workunit.client.1.smithi178.stdout:9/515: creat d2/df/d31/d37/f94 x:0 0 0 2024-09-10T11:29:28.540 INFO:tasks.workunit.client.1.smithi178.stdout:0/454: mkdir d0/d2c/d93 0 2024-09-10T11:29:28.541 INFO:tasks.workunit.client.1.smithi178.stdout:3/405: chown db/d35/d3a 26543 1 2024-09-10T11:29:28.541 INFO:tasks.workunit.client.1.smithi178.stdout:5/446: rmdir d9/dc/dd/d37/d42/d62 39 2024-09-10T11:29:28.541 INFO:tasks.workunit.client.1.smithi178.stdout:5/447: dread - d9/f86 zero size 2024-09-10T11:29:28.542 INFO:tasks.workunit.client.0.smithi144.stdout:7/19: symlink d5/l6 0 2024-09-10T11:29:28.543 INFO:tasks.workunit.client.1.smithi178.stdout:7/359: mkdir d6/d10/d1d/d22/d50/d7e/d82 0 2024-09-10T11:29:28.545 INFO:tasks.workunit.client.0.smithi144.stdout:2/40: creat d7/da/f12 x:0 0 0 2024-09-10T11:29:28.545 INFO:tasks.workunit.client.0.smithi144.stdout:2/41: chown d7/da/fb 6993705 1 2024-09-10T11:29:28.546 INFO:tasks.workunit.client.0.smithi144.stdout:2/42: write d7/da/f12 [855781,4273] 0 2024-09-10T11:29:28.548 INFO:tasks.workunit.client.1.smithi178.stdout:5/448: mkdir d9/d19/d91/d9a 0 2024-09-10T11:29:28.548 INFO:tasks.workunit.client.1.smithi178.stdout:7/360: dread - d6/d10/d1e/d2d/f66 zero size 2024-09-10T11:29:28.551 INFO:tasks.workunit.client.0.smithi144.stdout:7/20: creat d5/f7 x:0 0 0 2024-09-10T11:29:28.553 INFO:tasks.workunit.client.0.smithi144.stdout:2/43: creat d7/da/f13 x:0 0 0 2024-09-10T11:29:28.554 INFO:tasks.workunit.client.0.smithi144.stdout:2/44: read d7/da/fd [160623,65356] 0 2024-09-10T11:29:28.555 INFO:tasks.workunit.client.1.smithi178.stdout:3/406: dwrite db/d20/f3c [0,4194304] 0 2024-09-10T11:29:28.555 INFO:tasks.workunit.client.0.smithi144.stdout:5/46: dwrite d4/fa [0,4194304] 0 2024-09-10T11:29:28.556 INFO:tasks.workunit.client.1.smithi178.stdout:7/361: mkdir d6/d10/d1d/d4a/d79/d83 0 2024-09-10T11:29:28.557 INFO:tasks.workunit.client.0.smithi144.stdout:4/17: sync 2024-09-10T11:29:28.557 INFO:tasks.workunit.client.0.smithi144.stdout:9/13: sync 2024-09-10T11:29:28.558 INFO:tasks.workunit.client.1.smithi178.stdout:3/407: mkdir db/d20/d3e/d6b/d79 0 2024-09-10T11:29:28.559 INFO:tasks.workunit.client.0.smithi144.stdout:3/22: truncate f3 608669 0 2024-09-10T11:29:28.560 INFO:tasks.workunit.client.1.smithi178.stdout:0/455: dwrite d0/d2c/f65 [0,4194304] 0 2024-09-10T11:29:28.565 INFO:tasks.workunit.client.0.smithi144.stdout:5/47: dread d4/fa [0,4194304] 0 2024-09-10T11:29:28.565 INFO:tasks.workunit.client.1.smithi178.stdout:3/408: write db/d20/d70/f73 [186461,51995] 0 2024-09-10T11:29:28.566 INFO:tasks.workunit.client.0.smithi144.stdout:4/18: unlink c3 0 2024-09-10T11:29:28.566 INFO:tasks.workunit.client.0.smithi144.stdout:5/48: readlink l2 0 2024-09-10T11:29:28.567 INFO:tasks.workunit.client.0.smithi144.stdout:4/19: stat f2 0 2024-09-10T11:29:28.567 INFO:tasks.workunit.client.0.smithi144.stdout:2/45: dwrite d7/da/f13 [0,4194304] 0 2024-09-10T11:29:28.568 INFO:tasks.workunit.client.0.smithi144.stdout:9/14: dread f0 [0,4194304] 0 2024-09-10T11:29:28.571 INFO:tasks.workunit.client.1.smithi178.stdout:0/456: mkdir d0/dc/d39/d75/d94 0 2024-09-10T11:29:28.572 INFO:tasks.workunit.client.1.smithi178.stdout:0/457: dread - d0/d2c/f37 zero size 2024-09-10T11:29:28.572 INFO:tasks.workunit.client.0.smithi144.stdout:7/21: rename f4 to d5/f8 0 2024-09-10T11:29:28.573 INFO:tasks.workunit.client.1.smithi178.stdout:3/409: creat db/d26/d40/f7a x:0 0 0 2024-09-10T11:29:28.573 INFO:tasks.workunit.client.0.smithi144.stdout:7/22: chown l0 0 1 2024-09-10T11:29:28.576 INFO:tasks.workunit.client.0.smithi144.stdout:5/49: unlink d4/d5/lc 0 2024-09-10T11:29:28.577 INFO:tasks.workunit.client.1.smithi178.stdout:0/458: write d0/dc/d1f/f26 [3754565,39814] 0 2024-09-10T11:29:28.578 INFO:tasks.workunit.client.0.smithi144.stdout:9/15: rename c3 to c5 0 2024-09-10T11:29:28.579 INFO:tasks.workunit.client.0.smithi144.stdout:9/16: rmdir - no directory 2024-09-10T11:29:28.581 INFO:tasks.workunit.client.1.smithi178.stdout:0/459: mkdir d0/d2c/d95 0 2024-09-10T11:29:28.583 INFO:tasks.workunit.client.0.smithi144.stdout:5/50: creat d4/fe x:0 0 0 2024-09-10T11:29:28.584 INFO:tasks.workunit.client.0.smithi144.stdout:5/51: read - d4/fe zero size 2024-09-10T11:29:28.585 INFO:tasks.workunit.client.0.smithi144.stdout:5/52: write d4/fe [439807,10082] 0 2024-09-10T11:29:28.586 INFO:tasks.workunit.client.1.smithi178.stdout:0/460: rename d0/dc/d39/c60 to d0/d2c/d95/c96 0 2024-09-10T11:29:28.586 INFO:tasks.workunit.client.0.smithi144.stdout:9/17: rename c4 to c6 0 2024-09-10T11:29:28.590 INFO:tasks.workunit.client.1.smithi178.stdout:0/461: getdents d0/dc/d1f/d2d 0 2024-09-10T11:29:28.592 INFO:tasks.workunit.client.1.smithi178.stdout:1/353: sync 2024-09-10T11:29:28.592 INFO:tasks.workunit.client.1.smithi178.stdout:4/278: sync 2024-09-10T11:29:28.592 INFO:tasks.workunit.client.1.smithi178.stdout:8/456: sync 2024-09-10T11:29:28.592 INFO:tasks.workunit.client.1.smithi178.stdout:6/496: sync 2024-09-10T11:29:28.593 INFO:tasks.workunit.client.0.smithi144.stdout:4/20: getdents . 0 2024-09-10T11:29:28.593 INFO:tasks.workunit.client.0.smithi144.stdout:4/21: rmdir - no directory 2024-09-10T11:29:28.593 INFO:tasks.workunit.client.0.smithi144.stdout:4/22: dread - f2 zero size 2024-09-10T11:29:28.593 INFO:tasks.workunit.client.0.smithi144.stdout:4/23: fdatasync f1 0 2024-09-10T11:29:28.596 INFO:tasks.workunit.client.1.smithi178.stdout:0/462: chown d0/dc/d39/f55 180547514 1 2024-09-10T11:29:28.596 INFO:tasks.workunit.client.0.smithi144.stdout:4/24: creat f5 x:0 0 0 2024-09-10T11:29:28.598 INFO:tasks.workunit.client.1.smithi178.stdout:8/457: chown d5/d30/d33/l77 25255221 1 2024-09-10T11:29:28.599 INFO:tasks.workunit.client.0.smithi144.stdout:4/25: mknod c6 0 2024-09-10T11:29:28.600 INFO:tasks.workunit.client.0.smithi144.stdout:4/26: dread - f1 zero size 2024-09-10T11:29:28.600 INFO:tasks.workunit.client.0.smithi144.stdout:4/27: fdatasync f1 0 2024-09-10T11:29:28.601 INFO:tasks.workunit.client.0.smithi144.stdout:9/18: unlink c5 0 2024-09-10T11:29:28.602 INFO:tasks.workunit.client.1.smithi178.stdout:8/458: write d5/db/d10/d20/f2c [2495485,56829] 0 2024-09-10T11:29:28.604 INFO:tasks.workunit.client.1.smithi178.stdout:0/463: creat d0/d12/d3f/d72/f97 x:0 0 0 2024-09-10T11:29:28.605 INFO:tasks.workunit.client.0.smithi144.stdout:9/19: dread f0 [0,4194304] 0 2024-09-10T11:29:28.605 INFO:tasks.workunit.client.0.smithi144.stdout:4/28: unlink f2 0 2024-09-10T11:29:28.605 INFO:tasks.workunit.client.0.smithi144.stdout:9/20: read f0 [2602678,68784] 0 2024-09-10T11:29:28.606 INFO:tasks.workunit.client.0.smithi144.stdout:3/23: truncate f2 1184030 0 2024-09-10T11:29:28.606 INFO:tasks.workunit.client.1.smithi178.stdout:8/459: write d5/db/d1e/f45 [31121,27330] 0 2024-09-10T11:29:28.607 INFO:tasks.workunit.client.1.smithi178.stdout:0/464: write d0/dc/d39/d4c/f73 [9031032,18993] 0 2024-09-10T11:29:28.607 INFO:tasks.workunit.client.0.smithi144.stdout:7/23: truncate f1 13174 0 2024-09-10T11:29:28.608 INFO:tasks.workunit.client.1.smithi178.stdout:6/497: dread f1 [0,4194304] 0 2024-09-10T11:29:28.609 INFO:tasks.workunit.client.1.smithi178.stdout:8/460: write d5/d61/f8f [51279,99214] 0 2024-09-10T11:29:28.610 INFO:tasks.workunit.client.0.smithi144.stdout:9/21: symlink l7 0 2024-09-10T11:29:28.610 INFO:tasks.workunit.client.0.smithi144.stdout:7/24: write d5/f7 [862769,114032] 0 2024-09-10T11:29:28.611 INFO:tasks.workunit.client.1.smithi178.stdout:6/498: chown d3/d24/l7a 394 1 2024-09-10T11:29:28.611 INFO:tasks.workunit.client.1.smithi178.stdout:8/461: fsync d5/f7 0 2024-09-10T11:29:28.612 INFO:tasks.workunit.client.0.smithi144.stdout:5/53: truncate d4/fa 1306962 0 2024-09-10T11:29:28.613 INFO:tasks.workunit.client.0.smithi144.stdout:4/29: dwrite f1 [0,4194304] 0 2024-09-10T11:29:28.615 INFO:tasks.workunit.client.1.smithi178.stdout:4/279: write d0/f5 [342480,123078] 0 2024-09-10T11:29:28.615 INFO:tasks.workunit.client.0.smithi144.stdout:9/22: mknod c8 0 2024-09-10T11:29:28.616 INFO:tasks.workunit.client.1.smithi178.stdout:1/354: dread d3/d9/d21/d4b/d2b/d5f/d61/f4a [0,4194304] 0 2024-09-10T11:29:28.616 INFO:tasks.workunit.client.1.smithi178.stdout:6/499: fdatasync d3/d7/d8/d15/f22 0 2024-09-10T11:29:28.617 INFO:tasks.workunit.client.1.smithi178.stdout:8/462: creat d5/db/d3d/f97 x:0 0 0 2024-09-10T11:29:28.618 INFO:tasks.workunit.client.1.smithi178.stdout:4/280: write d0/d8/da/f27 [2506957,108975] 0 2024-09-10T11:29:28.619 INFO:tasks.workunit.client.1.smithi178.stdout:6/500: readlink d3/d42/l98 0 2024-09-10T11:29:28.619 INFO:tasks.workunit.client.0.smithi144.stdout:4/30: creat f7 x:0 0 0 2024-09-10T11:29:28.621 INFO:tasks.workunit.client.0.smithi144.stdout:7/25: dwrite d5/f7 [0,4194304] 0 2024-09-10T11:29:28.621 INFO:tasks.workunit.client.0.smithi144.stdout:4/31: chown f5 179305 1 2024-09-10T11:29:28.622 INFO:tasks.workunit.client.1.smithi178.stdout:8/463: creat d5/db/d10/d53/f98 x:0 0 0 2024-09-10T11:29:28.622 INFO:tasks.workunit.client.0.smithi144.stdout:9/23: dread f0 [0,4194304] 0 2024-09-10T11:29:28.623 INFO:tasks.workunit.client.1.smithi178.stdout:0/465: dread d0/dc/d39/d50/f25 [0,4194304] 0 2024-09-10T11:29:28.623 INFO:tasks.workunit.client.0.smithi144.stdout:3/24: getdents d4 0 2024-09-10T11:29:28.624 INFO:tasks.workunit.client.1.smithi178.stdout:1/355: mkdir d3/d9/d54/d3a/d63/d7e 0 2024-09-10T11:29:28.626 INFO:tasks.workunit.client.1.smithi178.stdout:6/501: truncate d3/d42/d57/f68 1027353 0 2024-09-10T11:29:28.627 INFO:tasks.workunit.client.0.smithi144.stdout:4/32: mknod c8 0 2024-09-10T11:29:28.627 INFO:tasks.workunit.client.0.smithi144.stdout:4/33: rmdir - no directory 2024-09-10T11:29:28.628 INFO:tasks.workunit.client.1.smithi178.stdout:6/502: chown d3/d7/d8/d72 8 1 2024-09-10T11:29:28.631 INFO:tasks.workunit.client.1.smithi178.stdout:4/281: mknod d0/c5e 0 2024-09-10T11:29:28.632 INFO:tasks.workunit.client.1.smithi178.stdout:0/466: chown d0/f5a 3192 1 2024-09-10T11:29:28.635 INFO:tasks.workunit.client.0.smithi144.stdout:3/25: creat d4/fb x:0 0 0 2024-09-10T11:29:28.635 INFO:tasks.workunit.client.0.smithi144.stdout:7/26: fdatasync f1 0 2024-09-10T11:29:28.636 INFO:tasks.workunit.client.0.smithi144.stdout:4/34: creat f9 x:0 0 0 2024-09-10T11:29:28.636 INFO:tasks.workunit.client.1.smithi178.stdout:8/464: dwrite d5/db/d10/d20/f3b [0,4194304] 0 2024-09-10T11:29:28.636 INFO:tasks.workunit.client.0.smithi144.stdout:9/24: dwrite f0 [0,4194304] 0 2024-09-10T11:29:28.637 INFO:tasks.workunit.client.0.smithi144.stdout:5/54: getdents d4 0 2024-09-10T11:29:28.638 INFO:tasks.workunit.client.1.smithi178.stdout:8/465: chown d5/db/d3d/f97 0 1 2024-09-10T11:29:28.639 INFO:tasks.workunit.client.0.smithi144.stdout:4/35: chown c8 180307 1 2024-09-10T11:29:28.639 INFO:tasks.workunit.client.0.smithi144.stdout:4/36: readlink - no filename 2024-09-10T11:29:28.640 INFO:tasks.workunit.client.0.smithi144.stdout:9/25: write f0 [4028332,81017] 0 2024-09-10T11:29:28.643 INFO:tasks.workunit.client.1.smithi178.stdout:4/282: rename d0/d9/dc/c5b to d0/d8/da/d1a/d51/d5d/c5f 0 2024-09-10T11:29:28.644 INFO:tasks.workunit.client.0.smithi144.stdout:4/37: unlink c8 0 2024-09-10T11:29:28.647 INFO:tasks.workunit.client.1.smithi178.stdout:8/466: dwrite d5/d1d/d22/f36 [0,4194304] 0 2024-09-10T11:29:28.647 INFO:tasks.workunit.client.1.smithi178.stdout:6/503: getdents d3/d7/d8/d56 0 2024-09-10T11:29:28.649 INFO:tasks.workunit.client.0.smithi144.stdout:3/26: creat d4/da/fc x:0 0 0 2024-09-10T11:29:28.652 INFO:tasks.workunit.client.0.smithi144.stdout:5/55: dwrite d4/fe [0,4194304] 0 2024-09-10T11:29:28.652 INFO:tasks.workunit.client.0.smithi144.stdout:3/27: truncate d4/fb 47693 0 2024-09-10T11:29:28.655 INFO:tasks.workunit.client.1.smithi178.stdout:0/467: rename d0/dc/d39/d4c/f24 to d0/dc/d39/d4c/d43/d8c/f98 0 2024-09-10T11:29:28.655 INFO:tasks.workunit.client.0.smithi144.stdout:7/27: dread d5/f7 [0,4194304] 0 2024-09-10T11:29:28.656 INFO:tasks.workunit.client.0.smithi144.stdout:3/28: dread - d4/da/fc zero size 2024-09-10T11:29:28.657 INFO:tasks.workunit.client.1.smithi178.stdout:4/283: creat d0/d8/da/f60 x:0 0 0 2024-09-10T11:29:28.658 INFO:tasks.workunit.client.1.smithi178.stdout:1/356: dread d3/d9/f25 [0,4194304] 0 2024-09-10T11:29:28.660 INFO:tasks.workunit.client.1.smithi178.stdout:4/284: write d0/d8/da/f60 [596699,39719] 0 2024-09-10T11:29:28.660 INFO:tasks.workunit.client.0.smithi144.stdout:9/26: dread f0 [0,4194304] 0 2024-09-10T11:29:28.663 INFO:tasks.workunit.client.0.smithi144.stdout:5/56: mknod d4/d5/cf 0 2024-09-10T11:29:28.664 INFO:tasks.workunit.client.1.smithi178.stdout:0/468: mknod d0/c99 0 2024-09-10T11:29:28.667 INFO:tasks.workunit.client.1.smithi178.stdout:1/357: creat d3/d9/d54/f7f x:0 0 0 2024-09-10T11:29:28.667 INFO:tasks.workunit.client.0.smithi144.stdout:4/38: dwrite f1 [4194304,4194304] 0 2024-09-10T11:29:28.668 INFO:tasks.workunit.client.1.smithi178.stdout:8/467: getdents d5/db/d3d 0 2024-09-10T11:29:28.669 INFO:tasks.workunit.client.0.smithi144.stdout:7/28: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:28.669 INFO:tasks.workunit.client.1.smithi178.stdout:0/469: rename d0/dc/d1f/d2d/c47 to d0/d12/d3f/c9a 0 2024-09-10T11:29:28.670 INFO:tasks.workunit.client.0.smithi144.stdout:5/57: symlink d4/l10 0 2024-09-10T11:29:28.671 INFO:tasks.workunit.client.1.smithi178.stdout:4/285: dread d0/d8/df/f3b [0,4194304] 0 2024-09-10T11:29:28.672 INFO:tasks.workunit.client.1.smithi178.stdout:8/468: creat d5/d30/d33/f99 x:0 0 0 2024-09-10T11:29:28.675 INFO:tasks.workunit.client.0.smithi144.stdout:5/58: stat d4/d5 0 2024-09-10T11:29:28.676 INFO:tasks.workunit.client.1.smithi178.stdout:1/358: rename d3/d7d to d3/d9/d54/d3a/d50/d80 0 2024-09-10T11:29:28.678 INFO:tasks.workunit.client.0.smithi144.stdout:4/39: link f7 fa 0 2024-09-10T11:29:28.678 INFO:tasks.workunit.client.0.smithi144.stdout:4/40: rmdir - no directory 2024-09-10T11:29:28.678 INFO:tasks.workunit.client.0.smithi144.stdout:3/29: dwrite d4/da/fc [0,4194304] 0 2024-09-10T11:29:28.679 INFO:tasks.workunit.client.1.smithi178.stdout:4/286: creat d0/d9/d3d/f61 x:0 0 0 2024-09-10T11:29:28.680 INFO:tasks.workunit.client.1.smithi178.stdout:8/469: mknod d5/d1d/c9a 0 2024-09-10T11:29:28.680 INFO:tasks.workunit.client.0.smithi144.stdout:5/59: unlink d4/fe 0 2024-09-10T11:29:28.681 INFO:tasks.workunit.client.1.smithi178.stdout:4/287: write d0/d9/d3d/d4f/f52 [924915,20466] 0 2024-09-10T11:29:28.681 INFO:tasks.workunit.client.0.smithi144.stdout:9/27: dwrite f0 [0,4194304] 0 2024-09-10T11:29:28.681 INFO:tasks.workunit.client.1.smithi178.stdout:8/470: truncate d5/f69 453268 0 2024-09-10T11:29:28.683 INFO:tasks.workunit.client.0.smithi144.stdout:4/41: symlink lb 0 2024-09-10T11:29:28.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:28 smithi144 ceph-mon[27942]: pgmap v21: 65 pgs: 65 active+clean; 541 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 52 MiB/s wr, 255 op/s 2024-09-10T11:29:28.685 INFO:tasks.workunit.client.1.smithi178.stdout:1/359: symlink d3/d9/d21/d4b/d2b/d5f/l81 0 2024-09-10T11:29:28.687 INFO:tasks.workunit.client.0.smithi144.stdout:3/30: dread f2 [0,4194304] 0 2024-09-10T11:29:28.687 INFO:tasks.workunit.client.0.smithi144.stdout:4/42: symlink lc 0 2024-09-10T11:29:28.688 INFO:tasks.workunit.client.1.smithi178.stdout:6/504: dwrite d3/d7/d8/d15/f22 [0,4194304] 0 2024-09-10T11:29:28.689 INFO:tasks.workunit.client.0.smithi144.stdout:3/31: write d4/da/fc [827965,107705] 0 2024-09-10T11:29:28.692 INFO:tasks.workunit.client.1.smithi178.stdout:8/471: rename d5/d61/c81 to d5/d61/c9b 0 2024-09-10T11:29:28.694 INFO:tasks.workunit.client.1.smithi178.stdout:0/470: getdents d0/dc 0 2024-09-10T11:29:28.696 INFO:tasks.workunit.client.0.smithi144.stdout:4/43: dwrite f5 [0,4194304] 0 2024-09-10T11:29:28.697 INFO:tasks.workunit.client.0.smithi144.stdout:5/60: getdents d4/d5 0 2024-09-10T11:29:28.697 INFO:tasks.workunit.client.0.smithi144.stdout:4/44: read f1 [7578619,59837] 0 2024-09-10T11:29:28.698 INFO:tasks.workunit.client.0.smithi144.stdout:3/32: mkdir d4/dd 0 2024-09-10T11:29:28.699 INFO:tasks.workunit.client.0.smithi144.stdout:4/45: truncate f9 180859 0 2024-09-10T11:29:28.699 INFO:tasks.workunit.client.1.smithi178.stdout:6/505: link d3/d7/d8/d19/f8b d3/d7/d8/d15/d1c/d2a/fb9 0 2024-09-10T11:29:28.701 INFO:tasks.workunit.client.1.smithi178.stdout:6/506: fsync d3/d42/d57/d78/d7c/f92 0 2024-09-10T11:29:28.704 INFO:tasks.workunit.client.0.smithi144.stdout:4/46: creat fd x:0 0 0 2024-09-10T11:29:28.704 INFO:tasks.workunit.client.1.smithi178.stdout:0/471: link d0/d12/d3f/f83 d0/dc/d39/d7c/f9b 0 2024-09-10T11:29:28.705 INFO:tasks.workunit.client.1.smithi178.stdout:8/472: getdents d5/d1d/d22/d76 0 2024-09-10T11:29:28.707 INFO:tasks.workunit.client.0.smithi144.stdout:4/47: rename lb to le 0 2024-09-10T11:29:28.708 INFO:tasks.workunit.client.1.smithi178.stdout:6/507: rename d3/d7/d8/d15/d1c/d2a/d18/f38 to d3/d6e/fba 0 2024-09-10T11:29:28.708 INFO:tasks.workunit.client.0.smithi144.stdout:9/28: fsync f0 0 2024-09-10T11:29:28.712 INFO:tasks.workunit.client.1.smithi178.stdout:6/508: fdatasync d3/d7/d8/d15/d1c/d2a/f41 0 2024-09-10T11:29:28.714 INFO:tasks.workunit.client.0.smithi144.stdout:4/48: symlink lf 0 2024-09-10T11:29:28.714 INFO:tasks.workunit.client.0.smithi144.stdout:4/49: chown fd 5616375 1 2024-09-10T11:29:28.715 INFO:tasks.workunit.client.0.smithi144.stdout:4/50: read - fa zero size 2024-09-10T11:29:28.717 INFO:tasks.workunit.client.1.smithi178.stdout:0/472: dwrite d0/d2c/f37 [0,4194304] 0 2024-09-10T11:29:28.722 INFO:tasks.workunit.client.1.smithi178.stdout:4/288: dwrite d0/d21/f37 [0,4194304] 0 2024-09-10T11:29:28.723 INFO:tasks.workunit.client.0.smithi144.stdout:9/29: truncate f0 1447527 0 2024-09-10T11:29:28.725 INFO:tasks.workunit.client.0.smithi144.stdout:3/33: write f2 [1993609,95853] 0 2024-09-10T11:29:28.725 INFO:tasks.workunit.client.1.smithi178.stdout:1/360: dread d3/d9/d21/d4b/f15 [0,4194304] 0 2024-09-10T11:29:28.725 INFO:tasks.workunit.client.0.smithi144.stdout:3/34: fdatasync d4/fb 0 2024-09-10T11:29:28.727 INFO:tasks.workunit.client.0.smithi144.stdout:3/35: dread d4/fb [0,4194304] 0 2024-09-10T11:29:28.727 INFO:tasks.workunit.client.0.smithi144.stdout:5/61: write d4/fa [1967655,59089] 0 2024-09-10T11:29:28.730 INFO:tasks.workunit.client.0.smithi144.stdout:8/6: sync 2024-09-10T11:29:28.730 INFO:tasks.workunit.client.0.smithi144.stdout:8/7: dread - no filename 2024-09-10T11:29:28.730 INFO:tasks.workunit.client.0.smithi144.stdout:8/8: write - no filename 2024-09-10T11:29:28.730 INFO:tasks.workunit.client.1.smithi178.stdout:8/473: dwrite d5/d30/d33/f49 [0,4194304] 0 2024-09-10T11:29:28.733 INFO:tasks.workunit.client.1.smithi178.stdout:4/289: mkdir d0/d8/da/d1a/d51/d5d/d62 0 2024-09-10T11:29:28.734 INFO:tasks.workunit.client.1.smithi178.stdout:1/361: rmdir d3/d9/d54/d3a/d50/d6c 39 2024-09-10T11:29:28.734 INFO:tasks.workunit.client.1.smithi178.stdout:0/473: truncate d0/dc/d39/d4c/f3a 1013270 0 2024-09-10T11:29:28.736 INFO:tasks.workunit.client.0.smithi144.stdout:5/62: rename l2 to d4/d5/l11 0 2024-09-10T11:29:28.736 INFO:tasks.workunit.client.1.smithi178.stdout:8/474: mknod d5/db/d3d/c9c 0 2024-09-10T11:29:28.737 INFO:tasks.workunit.client.1.smithi178.stdout:0/474: truncate d0/d2c/f65 4905863 0 2024-09-10T11:29:28.737 INFO:tasks.workunit.client.0.smithi144.stdout:5/63: write d4/fa [1763001,113914] 0 2024-09-10T11:29:28.738 INFO:tasks.workunit.client.1.smithi178.stdout:8/475: write d5/f1f [2900322,6961] 0 2024-09-10T11:29:28.740 INFO:tasks.workunit.client.1.smithi178.stdout:4/290: mkdir d0/d63 0 2024-09-10T11:29:28.740 INFO:tasks.workunit.client.0.smithi144.stdout:8/9: unlink d0/c2 0 2024-09-10T11:29:28.740 INFO:tasks.workunit.client.0.smithi144.stdout:8/10: dread - no filename 2024-09-10T11:29:28.741 INFO:tasks.workunit.client.0.smithi144.stdout:8/11: chown d0/c1 13327162 1 2024-09-10T11:29:28.741 INFO:tasks.workunit.client.1.smithi178.stdout:6/509: link d3/d7/d8/d15/d1c/d20/l9a d3/d74/d91/daa/lbb 0 2024-09-10T11:29:28.743 INFO:tasks.workunit.client.1.smithi178.stdout:0/475: dread d0/dc/d39/d50/d5e/f7d [0,4194304] 0 2024-09-10T11:29:28.743 INFO:tasks.workunit.client.0.smithi144.stdout:3/36: creat d4/dd/fe x:0 0 0 2024-09-10T11:29:28.743 INFO:tasks.workunit.client.1.smithi178.stdout:4/291: mknod d0/d8/d3e/c64 0 2024-09-10T11:29:28.744 INFO:tasks.workunit.client.1.smithi178.stdout:8/476: symlink d5/l9d 0 2024-09-10T11:29:28.745 INFO:tasks.workunit.client.1.smithi178.stdout:0/476: write d0/d2c/f65 [4069833,112647] 0 2024-09-10T11:29:28.745 INFO:tasks.workunit.client.1.smithi178.stdout:4/292: chown d0/d8/df/f29 1536343185 1 2024-09-10T11:29:28.748 INFO:tasks.workunit.client.1.smithi178.stdout:0/477: truncate d0/d2c/d33/f81 509569 0 2024-09-10T11:29:28.748 INFO:tasks.workunit.client.1.smithi178.stdout:1/362: rename d3/d9/d54/d3a/d50/l71 to d3/d9/d54/d3a/d50/l82 0 2024-09-10T11:29:28.749 INFO:tasks.workunit.client.1.smithi178.stdout:4/293: mknod d0/d8/da/c65 0 2024-09-10T11:29:28.749 INFO:tasks.workunit.client.0.smithi144.stdout:8/12: symlink d0/l3 0 2024-09-10T11:29:28.749 INFO:tasks.workunit.client.0.smithi144.stdout:8/13: dwrite - no filename 2024-09-10T11:29:28.750 INFO:tasks.workunit.client.0.smithi144.stdout:8/14: fdatasync - no filename 2024-09-10T11:29:28.750 INFO:tasks.workunit.client.0.smithi144.stdout:9/30: dread f0 [0,4194304] 0 2024-09-10T11:29:28.750 INFO:tasks.workunit.client.1.smithi178.stdout:0/478: creat d0/dc/d39/f9c x:0 0 0 2024-09-10T11:29:28.750 INFO:tasks.workunit.client.1.smithi178.stdout:1/363: write d3/d9/d21/d4b/d2b/d5f/d61/f43 [1765715,31698] 0 2024-09-10T11:29:28.752 INFO:tasks.workunit.client.0.smithi144.stdout:4/51: dwrite fa [0,4194304] 0 2024-09-10T11:29:28.754 INFO:tasks.workunit.client.0.smithi144.stdout:3/37: fsync f3 0 2024-09-10T11:29:28.757 INFO:tasks.workunit.client.1.smithi178.stdout:1/364: mknod d3/d9/d1a/c83 0 2024-09-10T11:29:28.757 INFO:tasks.workunit.client.0.smithi144.stdout:9/31: creat f9 x:0 0 0 2024-09-10T11:29:28.759 INFO:tasks.workunit.client.0.smithi144.stdout:8/15: symlink d0/l4 0 2024-09-10T11:29:28.760 INFO:tasks.workunit.client.1.smithi178.stdout:1/365: rmdir d3/d9/d21/d6f 39 2024-09-10T11:29:28.760 INFO:tasks.workunit.client.0.smithi144.stdout:5/64: link d4/lb d4/l12 0 2024-09-10T11:29:28.763 INFO:tasks.workunit.client.1.smithi178.stdout:0/479: dread d0/dc/d1f/f74 [0,4194304] 0 2024-09-10T11:29:28.763 INFO:tasks.workunit.client.1.smithi178.stdout:2/593: sync 2024-09-10T11:29:28.764 INFO:tasks.workunit.client.0.smithi144.stdout:9/32: mknod ca 0 2024-09-10T11:29:28.764 INFO:tasks.workunit.client.1.smithi178.stdout:1/366: rename d3/d9/d21/d4b/c60 to d3/d9/d54/d2a/c84 0 2024-09-10T11:29:28.765 INFO:tasks.workunit.client.1.smithi178.stdout:1/367: chown d3/d9/d54/d3a/f53 6364779 1 2024-09-10T11:29:28.766 INFO:tasks.workunit.client.1.smithi178.stdout:2/594: write d1/d41/d14/d11/f9 [3317949,106986] 0 2024-09-10T11:29:28.767 INFO:tasks.workunit.client.0.smithi144.stdout:8/16: creat d0/f5 x:0 0 0 2024-09-10T11:29:28.768 INFO:tasks.workunit.client.0.smithi144.stdout:8/17: write d0/f5 [280242,23135] 0 2024-09-10T11:29:28.770 INFO:tasks.workunit.client.1.smithi178.stdout:0/480: link d0/dc/f13 d0/dc/d39/f9d 0 2024-09-10T11:29:28.771 INFO:tasks.workunit.client.0.smithi144.stdout:4/52: truncate fa 2862752 0 2024-09-10T11:29:28.771 INFO:tasks.workunit.client.0.smithi144.stdout:4/53: rmdir - no directory 2024-09-10T11:29:28.772 INFO:tasks.workunit.client.1.smithi178.stdout:1/368: creat d3/d9/d21/d4b/d2b/d5f/f85 x:0 0 0 2024-09-10T11:29:28.773 INFO:tasks.workunit.client.0.smithi144.stdout:3/38: truncate f2 107658 0 2024-09-10T11:29:28.773 INFO:tasks.workunit.client.0.smithi144.stdout:4/54: dread f9 [0,4194304] 0 2024-09-10T11:29:28.774 INFO:tasks.workunit.client.1.smithi178.stdout:0/481: fsync d0/d12/d3f/f44 0 2024-09-10T11:29:28.774 INFO:tasks.workunit.client.0.smithi144.stdout:4/55: chown lc 534672 1 2024-09-10T11:29:28.775 INFO:tasks.workunit.client.0.smithi144.stdout:3/39: dread d4/fb [0,4194304] 0 2024-09-10T11:29:28.777 INFO:tasks.workunit.client.0.smithi144.stdout:3/40: chown d4/dd/fe 2 1 2024-09-10T11:29:28.777 INFO:tasks.workunit.client.0.smithi144.stdout:8/18: dwrite d0/f5 [0,4194304] 0 2024-09-10T11:29:28.777 INFO:tasks.workunit.client.1.smithi178.stdout:4/294: dwrite d0/d8/d3e/f4e [0,4194304] 0 2024-09-10T11:29:28.778 INFO:tasks.workunit.client.0.smithi144.stdout:3/41: chown d4 13521630 1 2024-09-10T11:29:28.778 INFO:tasks.workunit.client.0.smithi144.stdout:3/42: chown d4/da/fc 16974788 1 2024-09-10T11:29:28.779 INFO:tasks.workunit.client.0.smithi144.stdout:8/19: creat d0/f6 x:0 0 0 2024-09-10T11:29:28.780 INFO:tasks.workunit.client.0.smithi144.stdout:8/20: chown d0/c1 122390 1 2024-09-10T11:29:28.780 INFO:tasks.workunit.client.0.smithi144.stdout:8/21: stat d0/f5 0 2024-09-10T11:29:28.781 INFO:tasks.workunit.client.1.smithi178.stdout:8/477: dread d5/db/d10/d20/f26 [0,4194304] 0 2024-09-10T11:29:28.781 INFO:tasks.workunit.client.1.smithi178.stdout:1/369: dread - d3/d9/d21/d4b/d2b/d5f/f85 zero size 2024-09-10T11:29:28.781 INFO:tasks.workunit.client.0.smithi144.stdout:3/43: symlink d4/da/lf 0 2024-09-10T11:29:28.781 INFO:tasks.workunit.client.0.smithi144.stdout:8/22: write d0/f5 [3529709,129173] 0 2024-09-10T11:29:28.786 INFO:tasks.workunit.client.1.smithi178.stdout:0/482: creat d0/dc/f9e x:0 0 0 2024-09-10T11:29:28.787 INFO:tasks.workunit.client.1.smithi178.stdout:4/295: rename d0 to d0/d9/dc/d50/d66 22 2024-09-10T11:29:28.787 INFO:tasks.workunit.client.1.smithi178.stdout:2/595: dwrite d1/d41/d14/fb0 [0,4194304] 0 2024-09-10T11:29:28.787 INFO:tasks.workunit.client.0.smithi144.stdout:3/44: rename d4/da/lf to d4/l10 0 2024-09-10T11:29:28.788 INFO:tasks.workunit.client.1.smithi178.stdout:1/370: fsync d3/d9/d1a/d48/f66 0 2024-09-10T11:29:28.790 INFO:tasks.workunit.client.1.smithi178.stdout:8/478: write d5/db/d10/d20/f28 [4549186,75527] 0 2024-09-10T11:29:28.790 INFO:tasks.workunit.client.1.smithi178.stdout:4/296: mknod d0/d21/c67 0 2024-09-10T11:29:28.792 INFO:tasks.workunit.client.0.smithi144.stdout:8/23: rename d0/c1 to d0/c7 0 2024-09-10T11:29:28.792 INFO:tasks.workunit.client.0.smithi144.stdout:8/24: fdatasync d0/f6 0 2024-09-10T11:29:28.794 INFO:tasks.workunit.client.1.smithi178.stdout:4/297: mknod d0/d21/c68 0 2024-09-10T11:29:28.795 INFO:tasks.workunit.client.0.smithi144.stdout:3/45: symlink d4/dd/l11 0 2024-09-10T11:29:28.797 INFO:tasks.workunit.client.0.smithi144.stdout:8/25: rename d0/f6 to d0/f8 0 2024-09-10T11:29:28.797 INFO:tasks.workunit.client.1.smithi178.stdout:8/479: creat d5/d1d/d22/d76/f9e x:0 0 0 2024-09-10T11:29:28.800 INFO:tasks.workunit.client.1.smithi178.stdout:4/298: read d0/f5 [44054,89503] 0 2024-09-10T11:29:28.800 INFO:tasks.workunit.client.0.smithi144.stdout:3/46: creat d4/dd/f12 x:0 0 0 2024-09-10T11:29:28.800 INFO:tasks.workunit.client.1.smithi178.stdout:1/371: creat d3/d9/d21/d4b/d2b/d5f/f86 x:0 0 0 2024-09-10T11:29:28.802 INFO:tasks.workunit.client.0.smithi144.stdout:3/47: mknod d4/da/c13 0 2024-09-10T11:29:28.803 INFO:tasks.workunit.client.0.smithi144.stdout:3/48: chown d4/da 972 1 2024-09-10T11:29:28.805 INFO:tasks.workunit.client.0.smithi144.stdout:4/56: truncate f7 1982846 0 2024-09-10T11:29:28.805 INFO:tasks.workunit.client.1.smithi178.stdout:8/480: rename d5/db/d10/d20/f2b to d5/db/d10/d53/f9f 0 2024-09-10T11:29:28.805 INFO:tasks.workunit.client.0.smithi144.stdout:3/49: write d4/dd/f12 [648087,110324] 0 2024-09-10T11:29:28.806 INFO:tasks.workunit.client.1.smithi178.stdout:1/372: symlink d3/d9/d21/d4b/d2b/d5f/l87 0 2024-09-10T11:29:28.806 INFO:tasks.workunit.client.0.smithi144.stdout:4/57: dread - fd zero size 2024-09-10T11:29:28.807 INFO:tasks.workunit.client.1.smithi178.stdout:1/373: chown d3/c33 5334667 1 2024-09-10T11:29:28.808 INFO:tasks.workunit.client.0.smithi144.stdout:8/26: dwrite d0/f8 [0,4194304] 0 2024-09-10T11:29:28.809 INFO:tasks.workunit.client.0.smithi144.stdout:3/50: symlink d4/dd/l14 0 2024-09-10T11:29:28.810 INFO:tasks.workunit.client.1.smithi178.stdout:4/299: rename d0/f18 to d0/d8/da/d1a/f69 0 2024-09-10T11:29:28.812 INFO:tasks.workunit.client.1.smithi178.stdout:1/374: creat d3/d9/d21/d4b/d2b/f88 x:0 0 0 2024-09-10T11:29:28.814 INFO:tasks.workunit.client.1.smithi178.stdout:0/483: creat d0/dc/d39/d50/f9f x:0 0 0 2024-09-10T11:29:28.815 INFO:tasks.workunit.client.1.smithi178.stdout:4/300: mknod d0/d8/da/d1a/d51/d5d/c6a 0 2024-09-10T11:29:28.816 INFO:tasks.workunit.client.0.smithi144.stdout:3/51: dread d4/dd/f12 [0,4194304] 0 2024-09-10T11:29:28.816 INFO:tasks.workunit.client.1.smithi178.stdout:8/481: link d5/c8a d5/d30/d33/d8c/ca0 0 2024-09-10T11:29:28.817 INFO:tasks.workunit.client.1.smithi178.stdout:4/301: truncate d0/f40 872317 0 2024-09-10T11:29:28.818 INFO:tasks.workunit.client.1.smithi178.stdout:4/302: chown d0/d21/c68 10311900 1 2024-09-10T11:29:28.819 INFO:tasks.workunit.client.1.smithi178.stdout:0/484: dread d0/dc/d39/d4c/d43/f5f [0,4194304] 0 2024-09-10T11:29:28.821 INFO:tasks.workunit.client.1.smithi178.stdout:1/375: symlink d3/d9/d54/d3a/d50/l89 0 2024-09-10T11:29:28.821 INFO:tasks.workunit.client.0.smithi144.stdout:8/27: dwrite d0/f8 [0,4194304] 0 2024-09-10T11:29:28.822 INFO:tasks.workunit.client.1.smithi178.stdout:1/376: chown d3/d9/d21/d4b/d2b/f3d 3417113 1 2024-09-10T11:29:28.824 INFO:tasks.workunit.client.0.smithi144.stdout:8/28: write d0/f8 [3381386,116291] 0 2024-09-10T11:29:28.826 INFO:tasks.workunit.client.0.smithi144.stdout:3/52: link d4/dd/l11 d4/dd/l15 0 2024-09-10T11:29:28.828 INFO:tasks.workunit.client.1.smithi178.stdout:8/482: dwrite d5/d1d/d22/f36 [0,4194304] 0 2024-09-10T11:29:28.828 INFO:tasks.workunit.client.1.smithi178.stdout:1/377: write d3/d9/d21/d4b/d2b/d5f/d61/f7b [280579,99666] 0 2024-09-10T11:29:28.829 INFO:tasks.workunit.client.0.smithi144.stdout:8/29: mkdir d0/d9 0 2024-09-10T11:29:28.829 INFO:tasks.workunit.client.0.smithi144.stdout:4/58: dwrite f1 [0,4194304] 0 2024-09-10T11:29:28.831 INFO:tasks.workunit.client.1.smithi178.stdout:0/485: getdents d0/d2c/d33/d41 0 2024-09-10T11:29:28.832 INFO:tasks.workunit.client.0.smithi144.stdout:8/30: mkdir d0/da 0 2024-09-10T11:29:28.833 INFO:tasks.workunit.client.0.smithi144.stdout:4/59: creat f10 x:0 0 0 2024-09-10T11:29:28.834 INFO:tasks.workunit.client.1.smithi178.stdout:0/486: fsync d0/f5a 0 2024-09-10T11:29:28.837 INFO:tasks.workunit.client.1.smithi178.stdout:2/596: dwrite d1/d22/d4d/d9b/f75 [0,4194304] 0 2024-09-10T11:29:28.837 INFO:tasks.workunit.client.1.smithi178.stdout:0/487: chown d0/dc/d39/d50/c7e 317901 1 2024-09-10T11:29:28.837 INFO:tasks.workunit.client.0.smithi144.stdout:3/53: dwrite d4/da/fc [4194304,4194304] 0 2024-09-10T11:29:28.841 INFO:tasks.workunit.client.1.smithi178.stdout:2/597: write d1/d7/d24/d7d/d84/d4b/f66 [1600567,45638] 0 2024-09-10T11:29:28.843 INFO:tasks.workunit.client.0.smithi144.stdout:8/31: dread d0/f8 [0,4194304] 0 2024-09-10T11:29:28.846 INFO:tasks.workunit.client.1.smithi178.stdout:2/598: truncate d1/d41/f51 5236953 0 2024-09-10T11:29:28.847 INFO:tasks.workunit.client.0.smithi144.stdout:4/60: dwrite fd [0,4194304] 0 2024-09-10T11:29:28.851 INFO:tasks.workunit.client.1.smithi178.stdout:2/599: chown d1/d7/d24/f2c 1128 1 2024-09-10T11:29:28.855 INFO:tasks.workunit.client.0.smithi144.stdout:8/32: dread d0/f5 [0,4194304] 0 2024-09-10T11:29:28.857 INFO:tasks.workunit.client.1.smithi178.stdout:8/483: dwrite d5/db/d10/d20/f26 [0,4194304] 0 2024-09-10T11:29:28.858 INFO:tasks.workunit.client.0.smithi144.stdout:4/61: rename f10 to f11 0 2024-09-10T11:29:28.859 INFO:tasks.workunit.client.1.smithi178.stdout:8/484: chown d5/d1d/c9a 159 1 2024-09-10T11:29:28.859 INFO:tasks.workunit.client.1.smithi178.stdout:4/303: dwrite d0/f5 [0,4194304] 0 2024-09-10T11:29:28.862 INFO:tasks.workunit.client.0.smithi144.stdout:8/33: unlink d0/f5 0 2024-09-10T11:29:28.864 INFO:tasks.workunit.client.0.smithi144.stdout:8/34: fsync d0/f8 0 2024-09-10T11:29:28.864 INFO:tasks.workunit.client.1.smithi178.stdout:2/600: write d1/d7/d24/d7d/d84/d4b/f67 [835889,113061] 0 2024-09-10T11:29:28.866 INFO:tasks.workunit.client.1.smithi178.stdout:0/488: dwrite d0/dc/ff [0,4194304] 0 2024-09-10T11:29:28.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:28 smithi178 ceph-mon[33514]: pgmap v21: 65 pgs: 65 active+clean; 541 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 52 MiB/s wr, 255 op/s 2024-09-10T11:29:28.867 INFO:tasks.workunit.client.1.smithi178.stdout:1/378: dread d3/d9/d21/d4b/d2b/d5f/d61/f45 [0,4194304] 0 2024-09-10T11:29:28.869 INFO:tasks.workunit.client.0.smithi144.stdout:8/35: symlink d0/lb 0 2024-09-10T11:29:28.870 INFO:tasks.workunit.client.1.smithi178.stdout:2/601: chown d1/d41/f51 14 1 2024-09-10T11:29:28.873 INFO:tasks.workunit.client.0.smithi144.stdout:8/36: creat d0/fc x:0 0 0 2024-09-10T11:29:28.873 INFO:tasks.workunit.client.1.smithi178.stdout:0/489: mknod d0/dc/d39/d50/d5e/ca0 0 2024-09-10T11:29:28.873 INFO:tasks.workunit.client.1.smithi178.stdout:1/379: stat d3/d9/d21/d4b/d2b/d5f/d61/f2e 0 2024-09-10T11:29:28.877 INFO:tasks.workunit.client.0.smithi144.stdout:8/37: unlink d0/l4 0 2024-09-10T11:29:28.877 INFO:tasks.workunit.client.0.smithi144.stdout:8/38: rename d0 to d0/dd 22 2024-09-10T11:29:28.878 INFO:tasks.workunit.client.1.smithi178.stdout:1/380: readlink d3/d9/l52 0 2024-09-10T11:29:28.880 INFO:tasks.workunit.client.0.smithi144.stdout:8/39: creat d0/d9/fe x:0 0 0 2024-09-10T11:29:28.881 INFO:tasks.workunit.client.1.smithi178.stdout:2/602: dread d1/d7/d24/d7d/d84/f2d [0,4194304] 0 2024-09-10T11:29:28.882 INFO:tasks.workunit.client.0.smithi144.stdout:8/40: creat d0/d9/ff x:0 0 0 2024-09-10T11:29:28.882 INFO:tasks.workunit.client.1.smithi178.stdout:8/485: dwrite d5/db/d10/d20/f2c [4194304,4194304] 0 2024-09-10T11:29:28.884 INFO:tasks.workunit.client.1.smithi178.stdout:8/486: rename d5 to d5/d1d/da1 22 2024-09-10T11:29:28.886 INFO:tasks.workunit.client.1.smithi178.stdout:8/487: dread - d5/d30/d33/f99 zero size 2024-09-10T11:29:28.890 INFO:tasks.workunit.client.0.smithi144.stdout:8/41: dwrite d0/d9/ff [0,4194304] 0 2024-09-10T11:29:28.891 INFO:tasks.workunit.client.0.smithi144.stdout:3/54: write f2 [644243,58812] 0 2024-09-10T11:29:28.892 INFO:tasks.workunit.client.1.smithi178.stdout:2/603: symlink d1/d7/d24/d46/d88/d9d/dab/d5b/d7c/lbc 0 2024-09-10T11:29:28.892 INFO:tasks.workunit.client.1.smithi178.stdout:0/490: dwrite d0/dc/d39/d50/d5e/f7d [0,4194304] 0 2024-09-10T11:29:28.892 INFO:tasks.workunit.client.0.smithi144.stdout:8/42: write d0/f8 [1302488,109567] 0 2024-09-10T11:29:28.894 INFO:tasks.workunit.client.0.smithi144.stdout:4/62: getdents . 0 2024-09-10T11:29:28.895 INFO:tasks.workunit.client.0.smithi144.stdout:4/63: chown f1 1647 1 2024-09-10T11:29:28.895 INFO:tasks.workunit.client.1.smithi178.stdout:1/381: rename d3/d9/d21/d4b/d2b/d5f/d61 to d3/d9/d54/d3a/d6e/d8a 0 2024-09-10T11:29:28.895 INFO:tasks.workunit.client.0.smithi144.stdout:4/64: chown fd 3338171 1 2024-09-10T11:29:28.897 INFO:tasks.workunit.client.1.smithi178.stdout:1/382: fsync d3/f40 0 2024-09-10T11:29:28.897 INFO:tasks.workunit.client.1.smithi178.stdout:8/488: rmdir d5/db/d10/d20/d52 39 2024-09-10T11:29:28.897 INFO:tasks.workunit.client.0.smithi144.stdout:3/55: symlink d4/dd/l16 0 2024-09-10T11:29:28.902 INFO:tasks.workunit.client.0.smithi144.stdout:8/43: creat d0/f10 x:0 0 0 2024-09-10T11:29:28.902 INFO:tasks.workunit.client.1.smithi178.stdout:2/604: truncate d1/d7/f6c 1003182 0 2024-09-10T11:29:28.904 INFO:tasks.workunit.client.0.smithi144.stdout:4/65: creat f12 x:0 0 0 2024-09-10T11:29:28.904 INFO:tasks.workunit.client.1.smithi178.stdout:4/304: truncate d0/f5 3743593 0 2024-09-10T11:29:28.906 INFO:tasks.workunit.client.0.smithi144.stdout:4/66: read f9 [135527,124306] 0 2024-09-10T11:29:28.906 INFO:tasks.workunit.client.0.smithi144.stdout:3/56: mkdir d4/da/d17 0 2024-09-10T11:29:28.906 INFO:tasks.workunit.client.1.smithi178.stdout:0/491: dwrite d0/d12/f64 [0,4194304] 0 2024-09-10T11:29:28.908 INFO:tasks.workunit.client.1.smithi178.stdout:1/383: mknod d3/d9/d54/d3a/d63/d7e/c8b 0 2024-09-10T11:29:28.910 INFO:tasks.workunit.client.1.smithi178.stdout:4/305: write d0/d8/f56 [455831,71385] 0 2024-09-10T11:29:28.911 INFO:tasks.workunit.client.0.smithi144.stdout:8/44: dwrite d0/fc [0,4194304] 0 2024-09-10T11:29:28.912 INFO:tasks.workunit.client.0.smithi144.stdout:4/67: rename lc to l13 0 2024-09-10T11:29:28.913 INFO:tasks.workunit.client.0.smithi144.stdout:8/45: truncate d0/f10 910042 0 2024-09-10T11:29:28.913 INFO:tasks.workunit.client.1.smithi178.stdout:1/384: stat d3/d9/d54 0 2024-09-10T11:29:28.913 INFO:tasks.workunit.client.1.smithi178.stdout:2/605: symlink d1/d41/d14/d11/lbd 0 2024-09-10T11:29:28.913 INFO:tasks.workunit.client.0.smithi144.stdout:8/46: fsync d0/f8 0 2024-09-10T11:29:28.914 INFO:tasks.workunit.client.1.smithi178.stdout:0/492: rmdir d0/d12/d3f 39 2024-09-10T11:29:28.914 INFO:tasks.workunit.client.1.smithi178.stdout:8/489: dwrite d5/db/f7d [0,4194304] 0 2024-09-10T11:29:28.915 INFO:tasks.workunit.client.0.smithi144.stdout:3/57: mkdir d4/da/d17/d18 0 2024-09-10T11:29:28.916 INFO:tasks.workunit.client.1.smithi178.stdout:4/306: creat d0/d8/da/f6b x:0 0 0 2024-09-10T11:29:28.916 INFO:tasks.workunit.client.0.smithi144.stdout:8/47: dread d0/f10 [0,4194304] 0 2024-09-10T11:29:28.919 INFO:tasks.workunit.client.1.smithi178.stdout:8/490: creat d5/d61/fa2 x:0 0 0 2024-09-10T11:29:28.919 INFO:tasks.workunit.client.0.smithi144.stdout:3/58: symlink d4/dd/l19 0 2024-09-10T11:29:28.920 INFO:tasks.workunit.client.1.smithi178.stdout:2/606: write d1/d7/d24/d7d/d84/d4b/d89/fa3 [524808,79163] 0 2024-09-10T11:29:28.920 INFO:tasks.workunit.client.1.smithi178.stdout:1/385: creat d3/d9/d54/d3a/d63/d7e/f8c x:0 0 0 2024-09-10T11:29:28.922 INFO:tasks.workunit.client.1.smithi178.stdout:4/307: symlink d0/d8/da/d1a/d51/d5d/l6c 0 2024-09-10T11:29:28.922 INFO:tasks.workunit.client.0.smithi144.stdout:3/59: mkdir d4/d1a 0 2024-09-10T11:29:28.923 INFO:tasks.workunit.client.0.smithi144.stdout:8/48: fdatasync d0/d9/ff 0 2024-09-10T11:29:28.925 INFO:tasks.workunit.client.1.smithi178.stdout:4/308: truncate d0/d10/f5a 274280 0 2024-09-10T11:29:28.926 INFO:tasks.workunit.client.0.smithi144.stdout:8/49: chown d0/c7 2000896 1 2024-09-10T11:29:28.927 INFO:tasks.workunit.client.0.smithi144.stdout:8/50: stat d0 0 2024-09-10T11:29:28.928 INFO:tasks.workunit.client.1.smithi178.stdout:1/386: chown d3/d9/d21/d6f/f75 37405 1 2024-09-10T11:29:28.928 INFO:tasks.workunit.client.0.smithi144.stdout:3/60: dread d4/da/fc [0,4194304] 0 2024-09-10T11:29:28.929 INFO:tasks.workunit.client.1.smithi178.stdout:8/491: dwrite d5/d30/f4b [0,4194304] 0 2024-09-10T11:29:28.930 INFO:tasks.workunit.client.0.smithi144.stdout:4/68: readlink l13 0 2024-09-10T11:29:28.930 INFO:tasks.workunit.client.0.smithi144.stdout:0/50: sync 2024-09-10T11:29:28.930 INFO:tasks.workunit.client.0.smithi144.stdout:1/55: sync 2024-09-10T11:29:28.930 INFO:tasks.workunit.client.0.smithi144.stdout:2/46: sync 2024-09-10T11:29:28.930 INFO:tasks.workunit.client.0.smithi144.stdout:6/51: sync 2024-09-10T11:29:28.931 INFO:tasks.workunit.client.0.smithi144.stdout:8/51: creat d0/d9/f11 x:0 0 0 2024-09-10T11:29:28.931 INFO:tasks.workunit.client.0.smithi144.stdout:0/51: chown f0 55956 1 2024-09-10T11:29:28.932 INFO:tasks.workunit.client.1.smithi178.stdout:4/309: unlink d0/d10/f48 0 2024-09-10T11:29:28.932 INFO:tasks.workunit.client.0.smithi144.stdout:2/47: chown d7/f11 2 1 2024-09-10T11:29:28.932 INFO:tasks.workunit.client.0.smithi144.stdout:1/56: chown d5/df/f10 117950259 1 2024-09-10T11:29:28.932 INFO:tasks.workunit.client.0.smithi144.stdout:6/52: dread - d9/dd/d11/f17 zero size 2024-09-10T11:29:28.937 INFO:tasks.workunit.client.0.smithi144.stdout:3/61: dread f2 [0,4194304] 0 2024-09-10T11:29:28.937 INFO:tasks.workunit.client.0.smithi144.stdout:8/52: write d0/d9/f11 [967472,116362] 0 2024-09-10T11:29:28.937 INFO:tasks.workunit.client.1.smithi178.stdout:4/310: unlink d0/d9/f2d 0 2024-09-10T11:29:28.941 INFO:tasks.workunit.client.1.smithi178.stdout:8/492: dwrite d5/db/d3d/f97 [0,4194304] 0 2024-09-10T11:29:28.941 INFO:tasks.workunit.client.0.smithi144.stdout:0/52: mknod da/ce 0 2024-09-10T11:29:28.941 INFO:tasks.workunit.client.0.smithi144.stdout:1/57: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:28.942 INFO:tasks.workunit.client.1.smithi178.stdout:9/516: sync 2024-09-10T11:29:28.942 INFO:tasks.workunit.client.0.smithi144.stdout:6/53: write f8 [474657,19770] 0 2024-09-10T11:29:28.943 INFO:tasks.workunit.client.1.smithi178.stdout:4/311: readlink d0/d9/dc/l2e 0 2024-09-10T11:29:28.943 INFO:tasks.workunit.client.0.smithi144.stdout:2/48: creat d7/da/f14 x:0 0 0 2024-09-10T11:29:28.945 INFO:tasks.workunit.client.1.smithi178.stdout:4/312: symlink d0/d9/d3d/d4f/l6d 0 2024-09-10T11:29:28.946 INFO:tasks.workunit.client.0.smithi144.stdout:1/58: creat d5/f15 x:0 0 0 2024-09-10T11:29:28.950 INFO:tasks.workunit.client.0.smithi144.stdout:4/69: dwrite fd [0,4194304] 0 2024-09-10T11:29:28.950 INFO:tasks.workunit.client.1.smithi178.stdout:9/517: read d2/df/d31/d37/f5f [2238747,104911] 0 2024-09-10T11:29:28.950 INFO:tasks.workunit.client.1.smithi178.stdout:4/313: creat d0/f6e x:0 0 0 2024-09-10T11:29:28.953 INFO:tasks.workunit.client.0.smithi144.stdout:1/59: creat d5/df/f16 x:0 0 0 2024-09-10T11:29:28.955 INFO:tasks.workunit.client.1.smithi178.stdout:9/518: rename d2/d13/f1f to d2/d19/d1c/d2d/f95 0 2024-09-10T11:29:28.957 INFO:tasks.workunit.client.0.smithi144.stdout:1/60: dread - d5/df/f16 zero size 2024-09-10T11:29:28.958 INFO:tasks.workunit.client.1.smithi178.stdout:4/314: dwrite d0/d9/dc/f34 [0,4194304] 0 2024-09-10T11:29:28.960 INFO:tasks.workunit.client.1.smithi178.stdout:9/519: write d2/d13/f86 [672276,74159] 0 2024-09-10T11:29:28.960 INFO:tasks.workunit.client.0.smithi144.stdout:2/49: dwrite d7/da/fb [4194304,4194304] 0 2024-09-10T11:29:28.962 INFO:tasks.workunit.client.0.smithi144.stdout:6/54: dread f6 [0,4194304] 0 2024-09-10T11:29:28.963 INFO:tasks.workunit.client.1.smithi178.stdout:0/493: dread d0/f4e [0,4194304] 0 2024-09-10T11:29:28.963 INFO:tasks.workunit.client.0.smithi144.stdout:2/50: stat d7 0 2024-09-10T11:29:28.964 INFO:tasks.workunit.client.1.smithi178.stdout:4/315: unlink d0/d8/da/l2f 0 2024-09-10T11:29:28.965 INFO:tasks.workunit.client.1.smithi178.stdout:9/520: symlink d2/d19/d1c/d2d/l96 0 2024-09-10T11:29:28.965 INFO:tasks.workunit.client.1.smithi178.stdout:2/607: dread d1/d41/d14/d11/f9 [0,4194304] 0 2024-09-10T11:29:28.967 INFO:tasks.workunit.client.0.smithi144.stdout:6/55: write f8 [575288,42349] 0 2024-09-10T11:29:28.968 INFO:tasks.workunit.client.1.smithi178.stdout:0/494: rename d0/d2c/d33/d41 to d0/d2c/d89/da1 0 2024-09-10T11:29:28.969 INFO:tasks.workunit.client.1.smithi178.stdout:4/316: mknod d0/d8/d3e/c6f 0 2024-09-10T11:29:28.972 INFO:tasks.workunit.client.0.smithi144.stdout:6/56: creat d9/dd/d11/f18 x:0 0 0 2024-09-10T11:29:28.972 INFO:tasks.workunit.client.1.smithi178.stdout:4/317: fsync d0/d9/d3d/d4f/f52 0 2024-09-10T11:29:28.974 INFO:tasks.workunit.client.0.smithi144.stdout:2/51: link d7/da/fb d7/f15 0 2024-09-10T11:29:28.975 INFO:tasks.workunit.client.0.smithi144.stdout:6/57: write f8 [1493105,10700] 0 2024-09-10T11:29:28.977 INFO:tasks.workunit.client.0.smithi144.stdout:6/58: dread - d9/dd/d11/f18 zero size 2024-09-10T11:29:28.977 INFO:tasks.workunit.client.0.smithi144.stdout:2/52: chown d7/c9 2 1 2024-09-10T11:29:28.978 INFO:tasks.workunit.client.0.smithi144.stdout:1/61: dwrite d5/df/f16 [0,4194304] 0 2024-09-10T11:29:28.978 INFO:tasks.workunit.client.1.smithi178.stdout:2/608: dwrite d1/d41/f1f [0,4194304] 0 2024-09-10T11:29:28.980 INFO:tasks.workunit.client.1.smithi178.stdout:4/318: chown d0/d8/df/l3a 1649516 1 2024-09-10T11:29:28.981 INFO:tasks.workunit.client.1.smithi178.stdout:4/319: readlink d0/d9/d3d/d4f/l6d 0 2024-09-10T11:29:28.983 INFO:tasks.workunit.client.1.smithi178.stdout:1/387: truncate d3/d9/fa 1935543 0 2024-09-10T11:29:28.983 INFO:tasks.workunit.client.0.smithi144.stdout:3/62: write f3 [465004,20041] 0 2024-09-10T11:29:28.983 INFO:tasks.workunit.client.0.smithi144.stdout:2/53: write d7/da/fb [5373582,1711] 0 2024-09-10T11:29:28.985 INFO:tasks.workunit.client.1.smithi178.stdout:0/495: getdents d0/d12 0 2024-09-10T11:29:28.986 INFO:tasks.workunit.client.0.smithi144.stdout:1/62: fsync d5/f15 0 2024-09-10T11:29:28.986 INFO:tasks.workunit.client.0.smithi144.stdout:6/59: creat d9/dc/d13/f19 x:0 0 0 2024-09-10T11:29:28.987 INFO:tasks.workunit.client.0.smithi144.stdout:0/53: truncate f4 3957614 0 2024-09-10T11:29:28.987 INFO:tasks.workunit.client.0.smithi144.stdout:6/60: readlink d9/dc/lf 0 2024-09-10T11:29:28.988 INFO:tasks.workunit.client.1.smithi178.stdout:1/388: write d3/d9/d54/d3a/d63/d7e/f8c [156561,47457] 0 2024-09-10T11:29:28.988 INFO:tasks.workunit.client.0.smithi144.stdout:3/63: fdatasync d4/dd/f12 0 2024-09-10T11:29:28.988 INFO:tasks.workunit.client.1.smithi178.stdout:0/496: write d0/dc/d39/f5b [1747118,60638] 0 2024-09-10T11:29:28.989 INFO:tasks.workunit.client.1.smithi178.stdout:4/320: fdatasync d0/d8/df/f59 0 2024-09-10T11:29:28.991 INFO:tasks.workunit.client.1.smithi178.stdout:1/389: write d3/d9/d54/d3a/d6e/d8a/f7b [636217,121243] 0 2024-09-10T11:29:28.991 INFO:tasks.workunit.client.0.smithi144.stdout:1/63: mknod d5/dd/c17 0 2024-09-10T11:29:28.992 INFO:tasks.workunit.client.1.smithi178.stdout:4/321: dread d0/d8/df/f3b [0,4194304] 0 2024-09-10T11:29:28.993 INFO:tasks.workunit.client.0.smithi144.stdout:6/61: creat d9/dd/f1a x:0 0 0 2024-09-10T11:29:28.993 INFO:tasks.workunit.client.0.smithi144.stdout:2/54: dread f6 [4194304,4194304] 0 2024-09-10T11:29:28.993 INFO:tasks.workunit.client.1.smithi178.stdout:1/390: readlink d3/d9/d21/d4b/d2b/d5f/l81 0 2024-09-10T11:29:28.993 INFO:tasks.workunit.client.0.smithi144.stdout:1/64: truncate d5/f12 223349 0 2024-09-10T11:29:28.994 INFO:tasks.workunit.client.0.smithi144.stdout:3/64: creat d4/dd/f1b x:0 0 0 2024-09-10T11:29:28.995 INFO:tasks.workunit.client.1.smithi178.stdout:1/391: write d3/d9/d1a/d48/f66 [858096,122865] 0 2024-09-10T11:29:28.995 INFO:tasks.workunit.client.0.smithi144.stdout:3/65: truncate d4/fb 473312 0 2024-09-10T11:29:28.996 INFO:tasks.workunit.client.1.smithi178.stdout:4/322: chown d0/c55 1646606 1 2024-09-10T11:29:28.996 INFO:tasks.workunit.client.0.smithi144.stdout:6/62: mkdir d9/dd/d11/d1b 0 2024-09-10T11:29:28.998 INFO:tasks.workunit.client.1.smithi178.stdout:1/392: mkdir d3/d9/d54/d8d 0 2024-09-10T11:29:28.998 INFO:tasks.workunit.client.0.smithi144.stdout:3/66: mknod d4/c1c 0 2024-09-10T11:29:28.998 INFO:tasks.workunit.client.1.smithi178.stdout:4/323: symlink d0/d9/d3d/l70 0 2024-09-10T11:29:28.999 INFO:tasks.workunit.client.0.smithi144.stdout:6/63: creat d9/dd/d11/f1c x:0 0 0 2024-09-10T11:29:29.000 INFO:tasks.workunit.client.0.smithi144.stdout:6/64: readlink d9/dc/lf 0 2024-09-10T11:29:29.000 INFO:tasks.workunit.client.1.smithi178.stdout:1/393: symlink d3/d9/l8e 0 2024-09-10T11:29:29.000 INFO:tasks.workunit.client.0.smithi144.stdout:2/55: creat d7/de/f16 x:0 0 0 2024-09-10T11:29:29.002 INFO:tasks.workunit.client.0.smithi144.stdout:3/67: mkdir d4/da/d1d 0 2024-09-10T11:29:29.004 INFO:tasks.workunit.client.1.smithi178.stdout:1/394: rmdir d3/d9/d54/d3a/d50 39 2024-09-10T11:29:29.005 INFO:tasks.workunit.client.1.smithi178.stdout:0/497: dread d0/dc/d39/d4c/f15 [0,4194304] 0 2024-09-10T11:29:29.007 INFO:tasks.workunit.client.0.smithi144.stdout:3/68: rmdir d4/da/d17 39 2024-09-10T11:29:29.009 INFO:tasks.workunit.client.0.smithi144.stdout:1/65: getdents d5/df 0 2024-09-10T11:29:29.009 INFO:tasks.workunit.client.1.smithi178.stdout:1/395: symlink d3/d9/d54/d3a/l8f 0 2024-09-10T11:29:29.010 INFO:tasks.workunit.client.0.smithi144.stdout:2/56: link d7/f11 d7/da/f17 0 2024-09-10T11:29:29.011 INFO:tasks.workunit.client.1.smithi178.stdout:1/396: readlink d3/d9/d21/d4b/d2b/d5f/l87 0 2024-09-10T11:29:29.014 INFO:tasks.workunit.client.0.smithi144.stdout:3/69: mknod d4/d1a/c1e 0 2024-09-10T11:29:29.015 INFO:tasks.workunit.client.0.smithi144.stdout:6/65: dwrite d9/dc/f16 [0,4194304] 0 2024-09-10T11:29:29.016 INFO:tasks.workunit.client.1.smithi178.stdout:1/397: rename d3/d9/f5e to d3/d9/d1a/d48/f90 0 2024-09-10T11:29:29.016 INFO:tasks.workunit.client.0.smithi144.stdout:0/54: unlink f4 0 2024-09-10T11:29:29.019 INFO:tasks.workunit.client.0.smithi144.stdout:4/70: dwrite f9 [0,4194304] 0 2024-09-10T11:29:29.021 INFO:tasks.workunit.client.1.smithi178.stdout:0/498: dread d0/d12/f27 [0,4194304] 0 2024-09-10T11:29:29.021 INFO:tasks.workunit.client.0.smithi144.stdout:3/70: chown d4/dd/f12 21027 1 2024-09-10T11:29:29.024 INFO:tasks.workunit.client.0.smithi144.stdout:0/55: mkdir da/df 0 2024-09-10T11:29:29.026 INFO:tasks.workunit.client.0.smithi144.stdout:4/71: write f9 [438083,52503] 0 2024-09-10T11:29:29.028 INFO:tasks.workunit.client.1.smithi178.stdout:9/521: dwrite d2/df/d31/d37/f5f [4194304,4194304] 0 2024-09-10T11:29:29.028 INFO:tasks.workunit.client.1.smithi178.stdout:4/324: truncate d0/d9/dc/f34 3697132 0 2024-09-10T11:29:29.035 INFO:tasks.workunit.client.1.smithi178.stdout:2/609: dwrite d1/d7/d24/d46/d88/d9d/dab/f17 [0,4194304] 0 2024-09-10T11:29:29.035 INFO:tasks.workunit.client.1.smithi178.stdout:9/522: chown d2/d19/d1c/d2d/c8a 10151 1 2024-09-10T11:29:29.041 INFO:tasks.workunit.client.1.smithi178.stdout:4/325: getdents d0/d8/d3e 0 2024-09-10T11:29:29.044 INFO:tasks.workunit.client.0.smithi144.stdout:2/57: dwrite d7/da/fb [4194304,4194304] 0 2024-09-10T11:29:29.048 INFO:tasks.workunit.client.1.smithi178.stdout:1/398: dread d3/d9/d21/d4b/f5a [0,4194304] 0 2024-09-10T11:29:29.049 INFO:tasks.workunit.client.0.smithi144.stdout:1/66: dwrite d5/fc [0,4194304] 0 2024-09-10T11:29:29.049 INFO:tasks.workunit.client.0.smithi144.stdout:2/58: readlink l0 0 2024-09-10T11:29:29.049 INFO:tasks.workunit.client.1.smithi178.stdout:4/326: creat d0/d63/f71 x:0 0 0 2024-09-10T11:29:29.049 INFO:tasks.workunit.client.1.smithi178.stdout:9/523: symlink d2/d13/d75/l97 0 2024-09-10T11:29:29.051 INFO:tasks.workunit.client.1.smithi178.stdout:4/327: read - d0/d8/da/d1a/f4b zero size 2024-09-10T11:29:29.053 INFO:tasks.workunit.client.1.smithi178.stdout:9/524: fsync d2/d51/d59/f87 0 2024-09-10T11:29:29.054 INFO:tasks.workunit.client.1.smithi178.stdout:1/399: rmdir d3/d35 39 2024-09-10T11:29:29.054 INFO:tasks.workunit.client.0.smithi144.stdout:6/66: dwrite d9/dd/d11/f17 [0,4194304] 0 2024-09-10T11:29:29.056 INFO:tasks.workunit.client.1.smithi178.stdout:0/499: dwrite d0/dc/d39/d4c/d43/f8a [0,4194304] 0 2024-09-10T11:29:29.057 INFO:tasks.workunit.client.0.smithi144.stdout:2/59: dread - d7/da/f14 zero size 2024-09-10T11:29:29.058 INFO:tasks.workunit.client.1.smithi178.stdout:2/610: rename d1/d7/d24/c39 to d1/d7/d24/d46/d88/d9d/dab/cbe 0 2024-09-10T11:29:29.058 INFO:tasks.workunit.client.0.smithi144.stdout:0/56: dread da/fc [0,4194304] 0 2024-09-10T11:29:29.060 INFO:tasks.workunit.client.1.smithi178.stdout:5/449: sync 2024-09-10T11:29:29.060 INFO:tasks.workunit.client.1.smithi178.stdout:7/362: sync 2024-09-10T11:29:29.060 INFO:tasks.workunit.client.1.smithi178.stdout:3/410: sync 2024-09-10T11:29:29.060 INFO:tasks.workunit.client.1.smithi178.stdout:6/510: sync 2024-09-10T11:29:29.061 INFO:tasks.workunit.client.1.smithi178.stdout:9/525: symlink d2/dc/l98 0 2024-09-10T11:29:29.062 INFO:tasks.workunit.client.0.smithi144.stdout:3/71: dwrite d4/dd/fe [0,4194304] 0 2024-09-10T11:29:29.062 INFO:tasks.workunit.client.0.smithi144.stdout:0/57: write f5 [665521,43909] 0 2024-09-10T11:29:29.063 INFO:tasks.workunit.client.1.smithi178.stdout:7/363: chown d6/d10/d1d/d22/d50/d7e 85063960 1 2024-09-10T11:29:29.063 INFO:tasks.workunit.client.1.smithi178.stdout:0/500: unlink d0/dc/d39/l84 0 2024-09-10T11:29:29.063 INFO:tasks.workunit.client.0.smithi144.stdout:2/60: creat d7/da/f18 x:0 0 0 2024-09-10T11:29:29.067 INFO:tasks.workunit.client.0.smithi144.stdout:6/67: fsync d9/dd/d11/f1c 0 2024-09-10T11:29:29.067 INFO:tasks.workunit.client.0.smithi144.stdout:1/67: getdents d5/dd 0 2024-09-10T11:29:29.068 INFO:tasks.workunit.client.0.smithi144.stdout:2/61: read d7/da/f13 [3527960,1052] 0 2024-09-10T11:29:29.068 INFO:tasks.workunit.client.1.smithi178.stdout:0/501: dread d0/f5a [0,4194304] 0 2024-09-10T11:29:29.069 INFO:tasks.workunit.client.1.smithi178.stdout:6/511: creat d3/d7/d8/d15/fbc x:0 0 0 2024-09-10T11:29:29.069 INFO:tasks.workunit.client.1.smithi178.stdout:5/450: dread - d9/dc/dd/d37/d42/d62/f80 zero size 2024-09-10T11:29:29.070 INFO:tasks.workunit.client.1.smithi178.stdout:3/411: creat db/d35/d58/f7b x:0 0 0 2024-09-10T11:29:29.071 INFO:tasks.workunit.client.1.smithi178.stdout:7/364: creat d6/d10/d61/f84 x:0 0 0 2024-09-10T11:29:29.072 INFO:tasks.workunit.client.1.smithi178.stdout:9/526: symlink d2/df/d31/l99 0 2024-09-10T11:29:29.072 INFO:tasks.workunit.client.0.smithi144.stdout:6/68: write d9/dd/d11/f17 [3427936,21239] 0 2024-09-10T11:29:29.072 INFO:tasks.workunit.client.0.smithi144.stdout:2/62: stat d7/f8 0 2024-09-10T11:29:29.072 INFO:tasks.workunit.client.0.smithi144.stdout:0/58: unlink da/lb 0 2024-09-10T11:29:29.072 INFO:tasks.workunit.client.0.smithi144.stdout:2/63: fdatasync d7/da/f18 0 2024-09-10T11:29:29.073 INFO:tasks.workunit.client.1.smithi178.stdout:3/412: write db/f72 [251397,49675] 0 2024-09-10T11:29:29.075 INFO:tasks.workunit.client.1.smithi178.stdout:7/365: creat d6/d10/d1d/d4a/d79/f85 x:0 0 0 2024-09-10T11:29:29.075 INFO:tasks.workunit.client.0.smithi144.stdout:6/69: creat d9/dd/f1d x:0 0 0 2024-09-10T11:29:29.076 INFO:tasks.workunit.client.0.smithi144.stdout:3/72: creat d4/da/d17/f1f x:0 0 0 2024-09-10T11:29:29.076 INFO:tasks.workunit.client.1.smithi178.stdout:6/512: creat d3/d7/fbd x:0 0 0 2024-09-10T11:29:29.077 INFO:tasks.workunit.client.0.smithi144.stdout:2/64: creat d7/da/f19 x:0 0 0 2024-09-10T11:29:29.078 INFO:tasks.workunit.client.1.smithi178.stdout:9/527: fsync d2/d19/d1c/d2d/f55 0 2024-09-10T11:29:29.078 INFO:tasks.workunit.client.1.smithi178.stdout:4/328: dread d0/d8/d3e/f4c [0,4194304] 0 2024-09-10T11:29:29.078 INFO:tasks.workunit.client.0.smithi144.stdout:0/59: dwrite f5 [0,4194304] 0 2024-09-10T11:29:29.082 INFO:tasks.workunit.client.1.smithi178.stdout:6/513: fdatasync d3/d7/d35/f5a 0 2024-09-10T11:29:29.084 INFO:tasks.workunit.client.1.smithi178.stdout:9/528: rename d2/d19/d81 to d2/d13/d40/d56/d9a 0 2024-09-10T11:29:29.084 INFO:tasks.workunit.client.0.smithi144.stdout:6/70: creat d9/dc/d13/f1e x:0 0 0 2024-09-10T11:29:29.085 INFO:tasks.workunit.client.0.smithi144.stdout:6/71: fdatasync d9/dc/d13/f19 0 2024-09-10T11:29:29.085 INFO:tasks.workunit.client.0.smithi144.stdout:2/65: creat d7/de/f1a x:0 0 0 2024-09-10T11:29:29.086 INFO:tasks.workunit.client.1.smithi178.stdout:6/514: rename d3 to d3/d7/d8/d15/d1c/d20/d4b/dbe 22 2024-09-10T11:29:29.088 INFO:tasks.workunit.client.1.smithi178.stdout:1/400: dread d3/d9/d54/f51 [0,4194304] 0 2024-09-10T11:29:29.088 INFO:tasks.workunit.client.0.smithi144.stdout:6/72: mkdir d9/d1f 0 2024-09-10T11:29:29.089 INFO:tasks.workunit.client.1.smithi178.stdout:5/451: getdents d9/dc/dd/d37/d42/d55 0 2024-09-10T11:29:29.091 INFO:tasks.workunit.client.0.smithi144.stdout:3/73: fdatasync d4/dd/fe 0 2024-09-10T11:29:29.091 INFO:tasks.workunit.client.0.smithi144.stdout:2/66: creat d7/de/f1b x:0 0 0 2024-09-10T11:29:29.093 INFO:tasks.workunit.client.1.smithi178.stdout:9/529: mkdir d2/d9b 0 2024-09-10T11:29:29.093 INFO:tasks.workunit.client.0.smithi144.stdout:1/68: truncate d5/fc 2614187 0 2024-09-10T11:29:29.095 INFO:tasks.workunit.client.0.smithi144.stdout:3/74: dread f3 [0,4194304] 0 2024-09-10T11:29:29.095 INFO:tasks.workunit.client.0.smithi144.stdout:2/67: truncate d7/da/f18 762701 0 2024-09-10T11:29:29.098 INFO:tasks.workunit.client.0.smithi144.stdout:6/73: dwrite d9/dd/f1a [0,4194304] 0 2024-09-10T11:29:29.098 INFO:tasks.workunit.client.0.smithi144.stdout:1/69: creat d5/f18 x:0 0 0 2024-09-10T11:29:29.099 INFO:tasks.workunit.client.1.smithi178.stdout:5/452: dwrite d9/dc/d17/d1d/f8c [0,4194304] 0 2024-09-10T11:29:29.099 INFO:tasks.workunit.client.0.smithi144.stdout:6/74: stat d9/dd/d11/d1b 0 2024-09-10T11:29:29.099 INFO:tasks.workunit.client.0.smithi144.stdout:3/75: dread d4/dd/f12 [0,4194304] 0 2024-09-10T11:29:29.099 INFO:tasks.workunit.client.0.smithi144.stdout:7/29: sync 2024-09-10T11:29:29.100 INFO:tasks.workunit.client.0.smithi144.stdout:2/68: mkdir d7/d1c 0 2024-09-10T11:29:29.100 INFO:tasks.workunit.client.0.smithi144.stdout:3/76: read f2 [290275,126139] 0 2024-09-10T11:29:29.102 INFO:tasks.workunit.client.0.smithi144.stdout:6/75: write d9/dc/f16 [1890686,28625] 0 2024-09-10T11:29:29.103 INFO:tasks.workunit.client.0.smithi144.stdout:2/69: dread - d7/da/ff zero size 2024-09-10T11:29:29.104 INFO:tasks.workunit.client.0.smithi144.stdout:1/70: rmdir d5/df 39 2024-09-10T11:29:29.108 INFO:tasks.workunit.client.1.smithi178.stdout:5/453: creat d9/dc/dd/d3f/d5a/d7d/f9b x:0 0 0 2024-09-10T11:29:29.109 INFO:tasks.workunit.client.0.smithi144.stdout:0/60: fdatasync f5 0 2024-09-10T11:29:29.110 INFO:tasks.workunit.client.0.smithi144.stdout:2/70: creat d7/f1d x:0 0 0 2024-09-10T11:29:29.110 INFO:tasks.workunit.client.1.smithi178.stdout:0/502: truncate d0/d2c/f37 2003490 0 2024-09-10T11:29:29.110 INFO:tasks.workunit.client.1.smithi178.stdout:3/413: truncate db/d20/d3d/d56/f66 3866237 0 2024-09-10T11:29:29.110 INFO:tasks.workunit.client.1.smithi178.stdout:2/611: dread d1/d7/d24/d7d/d84/d55/f5c [0,4194304] 0 2024-09-10T11:29:29.112 INFO:tasks.workunit.client.1.smithi178.stdout:5/454: dread - d9/dc/dd/f7f zero size 2024-09-10T11:29:29.113 INFO:tasks.workunit.client.0.smithi144.stdout:6/76: creat d9/dc/d13/f20 x:0 0 0 2024-09-10T11:29:29.115 INFO:tasks.workunit.client.1.smithi178.stdout:2/612: symlink d1/d7/d24/lbf 0 2024-09-10T11:29:29.115 INFO:tasks.workunit.client.0.smithi144.stdout:6/77: read - d9/dd/d11/f18 zero size 2024-09-10T11:29:29.115 INFO:tasks.workunit.client.1.smithi178.stdout:0/503: symlink d0/d2c/d95/la2 0 2024-09-10T11:29:29.115 INFO:tasks.workunit.client.0.smithi144.stdout:7/30: dwrite d5/f7 [0,4194304] 0 2024-09-10T11:29:29.117 INFO:tasks.workunit.client.0.smithi144.stdout:2/71: rename d7/d1c to d7/da/d1e 0 2024-09-10T11:29:29.118 INFO:tasks.workunit.client.0.smithi144.stdout:7/31: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:29.119 INFO:tasks.workunit.client.0.smithi144.stdout:2/72: dread - d7/da/f17 zero size 2024-09-10T11:29:29.120 INFO:tasks.workunit.client.1.smithi178.stdout:5/455: unlink d9/dc/l40 0 2024-09-10T11:29:29.120 INFO:tasks.workunit.client.0.smithi144.stdout:6/78: creat d9/dd/d11/f21 x:0 0 0 2024-09-10T11:29:29.123 INFO:tasks.workunit.client.1.smithi178.stdout:0/504: mknod d0/dc/d39/d7c/ca3 0 2024-09-10T11:29:29.124 INFO:tasks.workunit.client.1.smithi178.stdout:1/401: dwrite d3/d9/d54/f28 [0,4194304] 0 2024-09-10T11:29:29.125 INFO:tasks.workunit.client.1.smithi178.stdout:2/613: rename d1/d7/d24/d7d/fb5 to d1/d7/d8e/fc0 0 2024-09-10T11:29:29.125 INFO:tasks.workunit.client.1.smithi178.stdout:9/530: dwrite d2/dc/f2f [0,4194304] 0 2024-09-10T11:29:29.126 INFO:tasks.workunit.client.0.smithi144.stdout:1/71: dwrite d5/f14 [0,4194304] 0 2024-09-10T11:29:29.133 INFO:tasks.workunit.client.1.smithi178.stdout:0/505: mkdir d0/dc/d39/da4 0 2024-09-10T11:29:29.137 INFO:tasks.workunit.client.1.smithi178.stdout:1/402: mknod d3/d9/d54/d2a/c91 0 2024-09-10T11:29:29.141 INFO:tasks.workunit.client.1.smithi178.stdout:2/614: dwrite d1/d41/d5d/fa9 [0,4194304] 0 2024-09-10T11:29:29.141 INFO:tasks.workunit.client.1.smithi178.stdout:3/414: dread db/f4f [0,4194304] 0 2024-09-10T11:29:29.142 INFO:tasks.workunit.client.1.smithi178.stdout:9/531: link d2/d19/d1c/d2a/f92 d2/df/f9c 0 2024-09-10T11:29:29.147 INFO:tasks.workunit.client.1.smithi178.stdout:2/615: readlink d1/d7/d24/d7d/d84/d55/da1/lb2 0 2024-09-10T11:29:29.154 INFO:tasks.workunit.client.1.smithi178.stdout:1/403: write d3/d9/f25 [4114600,24913] 0 2024-09-10T11:29:29.154 INFO:tasks.workunit.client.1.smithi178.stdout:2/616: dread d1/d41/d14/f59 [0,4194304] 0 2024-09-10T11:29:29.155 INFO:tasks.workunit.client.1.smithi178.stdout:9/532: dwrite d2/d13/d75/d60/f8d [0,4194304] 0 2024-09-10T11:29:29.156 INFO:tasks.workunit.client.1.smithi178.stdout:5/456: dread d9/dc/dd/f69 [0,4194304] 0 2024-09-10T11:29:29.157 INFO:tasks.workunit.client.1.smithi178.stdout:5/457: chown d9/f32 1889897 1 2024-09-10T11:29:29.158 INFO:tasks.workunit.client.1.smithi178.stdout:5/458: chown d9/dc/dd/l6c 1 1 2024-09-10T11:29:29.159 INFO:tasks.workunit.client.1.smithi178.stdout:2/617: mkdir d1/d41/d14/d11/db9/dc1 0 2024-09-10T11:29:29.161 INFO:tasks.workunit.client.1.smithi178.stdout:5/459: creat d9/dc/dd/d3f/d5a/f9c x:0 0 0 2024-09-10T11:29:29.164 INFO:tasks.workunit.client.1.smithi178.stdout:1/404: link d3/d9/d54/d3a/d6e/f7c d3/d9/d54/d2a/f92 0 2024-09-10T11:29:29.165 INFO:tasks.workunit.client.1.smithi178.stdout:2/618: symlink d1/d7/d24/d46/d88/d9d/dab/lc2 0 2024-09-10T11:29:29.167 INFO:tasks.workunit.client.1.smithi178.stdout:5/460: creat d9/dc/dd/d37/d42/d55/f9d x:0 0 0 2024-09-10T11:29:29.167 INFO:tasks.workunit.client.1.smithi178.stdout:1/405: mknod d3/d9/d54/d3a/d63/c93 0 2024-09-10T11:29:29.168 INFO:tasks.workunit.client.1.smithi178.stdout:2/619: creat d1/d22/d4d/d78/fc3 x:0 0 0 2024-09-10T11:29:29.170 INFO:tasks.workunit.client.1.smithi178.stdout:2/620: write d1/d41/d14/d11/d53/f8b [903586,115359] 0 2024-09-10T11:29:29.170 INFO:tasks.workunit.client.1.smithi178.stdout:5/461: rename d9/d19/c33 to d9/dc/d17/d1d/c9e 0 2024-09-10T11:29:29.172 INFO:tasks.workunit.client.1.smithi178.stdout:5/462: write d9/d19/f97 [3321107,105980] 0 2024-09-10T11:29:29.174 INFO:tasks.workunit.client.1.smithi178.stdout:2/621: rename d1/d7/c72 to d1/d41/d8d/cc4 0 2024-09-10T11:29:29.174 INFO:tasks.workunit.client.1.smithi178.stdout:5/463: chown d9/dc/d17/d1d/c4a 0 1 2024-09-10T11:29:29.174 INFO:tasks.workunit.client.1.smithi178.stdout:5/464: readlink l3 0 2024-09-10T11:29:29.176 INFO:tasks.workunit.client.1.smithi178.stdout:3/415: dread db/f1a [0,4194304] 0 2024-09-10T11:29:29.176 INFO:tasks.workunit.client.1.smithi178.stdout:8/493: sync 2024-09-10T11:29:29.176 INFO:tasks.workunit.client.1.smithi178.stdout:2/622: read d1/d7/d24/d46/d88/d9d/dab/f15 [239638,11917] 0 2024-09-10T11:29:29.179 INFO:tasks.workunit.client.1.smithi178.stdout:3/416: mknod db/d35/d58/c7c 0 2024-09-10T11:29:29.180 INFO:tasks.workunit.client.1.smithi178.stdout:8/494: unlink d5/d1d/l83 0 2024-09-10T11:29:29.181 INFO:tasks.workunit.client.1.smithi178.stdout:5/465: mkdir d9/dc/dd/d37/d42/d9f 0 2024-09-10T11:29:29.182 INFO:tasks.workunit.client.1.smithi178.stdout:1/406: dread d3/f22 [0,4194304] 0 2024-09-10T11:29:29.183 INFO:tasks.workunit.client.1.smithi178.stdout:8/495: creat d5/db/d1e/fa3 x:0 0 0 2024-09-10T11:29:29.184 INFO:tasks.workunit.client.1.smithi178.stdout:5/466: symlink d9/la0 0 2024-09-10T11:29:29.185 INFO:tasks.workunit.client.1.smithi178.stdout:2/623: dwrite d1/d41/f1e [0,4194304] 0 2024-09-10T11:29:29.186 INFO:tasks.workunit.client.1.smithi178.stdout:1/407: creat d3/d9/d21/d4b/d2b/f94 x:0 0 0 2024-09-10T11:29:29.186 INFO:tasks.workunit.client.1.smithi178.stdout:8/496: write d5/d1d/d22/f36 [4067350,24518] 0 2024-09-10T11:29:29.190 INFO:tasks.workunit.client.1.smithi178.stdout:2/624: creat d1/d7/d24/d46/d88/d9d/dab/fc5 x:0 0 0 2024-09-10T11:29:29.191 INFO:tasks.workunit.client.1.smithi178.stdout:2/625: chown d1/d41/d5d 46 1 2024-09-10T11:29:29.192 INFO:tasks.workunit.client.1.smithi178.stdout:5/467: link d9/f86 d9/dc/dd/d3f/d5a/d7d/fa1 0 2024-09-10T11:29:29.193 INFO:tasks.workunit.client.0.smithi144.stdout:7/32: creat d5/f9 x:0 0 0 2024-09-10T11:29:29.193 INFO:tasks.workunit.client.0.smithi144.stdout:7/33: fsync f1 0 2024-09-10T11:29:29.194 INFO:tasks.workunit.client.0.smithi144.stdout:7/34: truncate f1 475907 0 2024-09-10T11:29:29.195 INFO:tasks.workunit.client.0.smithi144.stdout:0/61: truncate f0 710177 0 2024-09-10T11:29:29.196 INFO:tasks.workunit.client.1.smithi178.stdout:8/497: mknod d5/db/d10/d70/ca4 0 2024-09-10T11:29:29.197 INFO:tasks.workunit.client.1.smithi178.stdout:8/498: chown d5/db/d10/l58 1 1 2024-09-10T11:29:29.198 INFO:tasks.workunit.client.1.smithi178.stdout:2/626: mkdir d1/d7/d24/d46/d88/dc6 0 2024-09-10T11:29:29.200 INFO:tasks.workunit.client.1.smithi178.stdout:5/468: write d9/dc/f2b [5002557,26280] 0 2024-09-10T11:29:29.200 INFO:tasks.workunit.client.0.smithi144.stdout:7/35: rename d5/f9 to d5/fa 0 2024-09-10T11:29:29.201 INFO:tasks.workunit.client.1.smithi178.stdout:3/417: truncate db/f4f 3791119 0 2024-09-10T11:29:29.201 INFO:tasks.workunit.client.0.smithi144.stdout:2/73: link l0 d7/da/d1e/l1f 0 2024-09-10T11:29:29.204 INFO:tasks.workunit.client.0.smithi144.stdout:1/72: dwrite d5/f6 [0,4194304] 0 2024-09-10T11:29:29.204 INFO:tasks.workunit.client.1.smithi178.stdout:2/627: unlink d1/d7/d24/d7d/d84/d4b/c79 0 2024-09-10T11:29:29.205 INFO:tasks.workunit.client.0.smithi144.stdout:1/73: fdatasync d5/f12 0 2024-09-10T11:29:29.208 INFO:tasks.workunit.client.1.smithi178.stdout:5/469: creat d9/d11/fa2 x:0 0 0 2024-09-10T11:29:29.209 INFO:tasks.workunit.client.0.smithi144.stdout:0/62: fsync f0 0 2024-09-10T11:29:29.210 INFO:tasks.workunit.client.1.smithi178.stdout:5/470: readlink d9/dc/dd/d3f/d5a/d7d/l87 0 2024-09-10T11:29:29.210 INFO:tasks.workunit.client.1.smithi178.stdout:3/418: readlink db/d36/l4d 0 2024-09-10T11:29:29.210 INFO:tasks.workunit.client.1.smithi178.stdout:3/419: chown db/l24 72360 1 2024-09-10T11:29:29.212 INFO:tasks.workunit.client.0.smithi144.stdout:7/36: dwrite f1 [0,4194304] 0 2024-09-10T11:29:29.214 INFO:tasks.workunit.client.0.smithi144.stdout:2/74: dread d7/da/fb [0,4194304] 0 2024-09-10T11:29:29.215 INFO:tasks.workunit.client.0.smithi144.stdout:1/74: getdents d5/df 0 2024-09-10T11:29:29.216 INFO:tasks.workunit.client.0.smithi144.stdout:7/37: creat d5/fb x:0 0 0 2024-09-10T11:29:29.217 INFO:tasks.workunit.client.0.smithi144.stdout:7/38: chown f1 23 1 2024-09-10T11:29:29.217 INFO:tasks.workunit.client.1.smithi178.stdout:5/471: mknod d9/ca3 0 2024-09-10T11:29:29.218 INFO:tasks.workunit.client.1.smithi178.stdout:8/499: getdents d5/d30/d33/d8c 0 2024-09-10T11:29:29.219 INFO:tasks.workunit.client.0.smithi144.stdout:2/75: unlink d7/de/f1a 0 2024-09-10T11:29:29.219 INFO:tasks.workunit.client.0.smithi144.stdout:7/39: write d5/fb [730805,20171] 0 2024-09-10T11:29:29.221 INFO:tasks.workunit.client.0.smithi144.stdout:0/63: write da/fc [393246,107225] 0 2024-09-10T11:29:29.221 INFO:tasks.workunit.client.1.smithi178.stdout:3/420: getdents db/d20/d3e/d6b 0 2024-09-10T11:29:29.222 INFO:tasks.workunit.client.1.smithi178.stdout:8/500: write d5/db/d1e/f45 [134158,82186] 0 2024-09-10T11:29:29.222 INFO:tasks.workunit.client.1.smithi178.stdout:1/408: dwrite d3/d9/d54/d3a/d6e/d8a/f2e [0,4194304] 0 2024-09-10T11:29:29.223 INFO:tasks.workunit.client.0.smithi144.stdout:0/64: chown da/cd 56403 1 2024-09-10T11:29:29.223 INFO:tasks.workunit.client.1.smithi178.stdout:5/472: creat d9/d19/d91/fa4 x:0 0 0 2024-09-10T11:29:29.224 INFO:tasks.workunit.client.0.smithi144.stdout:1/75: dwrite d5/df/f10 [0,4194304] 0 2024-09-10T11:29:29.224 INFO:tasks.workunit.client.1.smithi178.stdout:5/473: fdatasync d9/d19/f1a 0 2024-09-10T11:29:29.227 INFO:tasks.workunit.client.1.smithi178.stdout:1/409: write d3/d9/d54/d2a/f76 [484685,113069] 0 2024-09-10T11:29:29.228 INFO:tasks.workunit.client.0.smithi144.stdout:1/76: write d5/f7 [2578714,100681] 0 2024-09-10T11:29:29.234 INFO:tasks.workunit.client.1.smithi178.stdout:2/628: truncate d1/d7/d8e/fa8 3800281 0 2024-09-10T11:29:29.236 INFO:tasks.workunit.client.1.smithi178.stdout:5/474: creat d9/dc/dd/d37/d42/d9f/fa5 x:0 0 0 2024-09-10T11:29:29.237 INFO:tasks.workunit.client.0.smithi144.stdout:0/65: dwrite f0 [0,4194304] 0 2024-09-10T11:29:29.237 INFO:tasks.workunit.client.1.smithi178.stdout:1/410: creat d3/d9/d21/d4b/d2b/d5f/f95 x:0 0 0 2024-09-10T11:29:29.239 INFO:tasks.workunit.client.0.smithi144.stdout:1/77: rename d5/df/f10 to d5/df/f19 0 2024-09-10T11:29:29.239 INFO:tasks.workunit.client.1.smithi178.stdout:5/475: stat d9/d19/d91/fa4 0 2024-09-10T11:29:29.240 INFO:tasks.workunit.client.1.smithi178.stdout:2/629: symlink d1/d41/d14/d11/d53/lc7 0 2024-09-10T11:29:29.241 INFO:tasks.workunit.client.0.smithi144.stdout:7/40: dwrite d5/fa [0,4194304] 0 2024-09-10T11:29:29.242 INFO:tasks.workunit.client.1.smithi178.stdout:3/421: dwrite db/f1a [0,4194304] 0 2024-09-10T11:29:29.243 INFO:tasks.workunit.client.0.smithi144.stdout:1/78: mkdir d5/df/d1a 0 2024-09-10T11:29:29.243 INFO:tasks.workunit.client.0.smithi144.stdout:0/66: dread f2 [0,4194304] 0 2024-09-10T11:29:29.244 INFO:tasks.workunit.client.1.smithi178.stdout:5/476: dread - d9/dc/dd/d3f/d5a/f9c zero size 2024-09-10T11:29:29.245 INFO:tasks.workunit.client.0.smithi144.stdout:1/79: dread d5/f12 [0,4194304] 0 2024-09-10T11:29:29.246 INFO:tasks.workunit.client.0.smithi144.stdout:7/41: rmdir d5 39 2024-09-10T11:29:29.247 INFO:tasks.workunit.client.1.smithi178.stdout:3/422: readlink db/l12 0 2024-09-10T11:29:29.248 INFO:tasks.workunit.client.1.smithi178.stdout:1/411: rename d3/d9/d54/d2a/c62 to d3/d9/d21/d4b/d2b/c96 0 2024-09-10T11:29:29.252 INFO:tasks.workunit.client.1.smithi178.stdout:3/423: readlink db/d20/d3d/d63/d6a/l75 0 2024-09-10T11:29:29.253 INFO:tasks.workunit.client.0.smithi144.stdout:0/67: rename c7 to da/c10 0 2024-09-10T11:29:29.253 INFO:tasks.workunit.client.1.smithi178.stdout:2/630: dwrite d1/d22/f3a [0,4194304] 0 2024-09-10T11:29:29.255 INFO:tasks.workunit.client.0.smithi144.stdout:1/80: dwrite d5/fb [0,4194304] 0 2024-09-10T11:29:29.257 INFO:tasks.workunit.client.0.smithi144.stdout:0/68: unlink c9 0 2024-09-10T11:29:29.258 INFO:tasks.workunit.client.0.smithi144.stdout:7/42: mkdir d5/dc 0 2024-09-10T11:29:29.259 INFO:tasks.workunit.client.0.smithi144.stdout:0/69: write f5 [4626386,26068] 0 2024-09-10T11:29:29.260 INFO:tasks.workunit.client.0.smithi144.stdout:0/70: write f5 [2947428,85672] 0 2024-09-10T11:29:29.264 INFO:tasks.workunit.client.1.smithi178.stdout:3/424: write db/d20/d3d/d63/f6f [257179,128555] 0 2024-09-10T11:29:29.265 INFO:tasks.workunit.client.1.smithi178.stdout:1/412: dwrite d3/d9/d1a/f6d [0,4194304] 0 2024-09-10T11:29:29.268 INFO:tasks.workunit.client.1.smithi178.stdout:2/631: link d1/c83 d1/d7/d24/d46/d88/d9d/dab/cc8 0 2024-09-10T11:29:29.270 INFO:tasks.workunit.client.0.smithi144.stdout:0/71: dwrite f0 [0,4194304] 0 2024-09-10T11:29:29.270 INFO:tasks.workunit.client.1.smithi178.stdout:3/425: truncate db/d35/f77 402715 0 2024-09-10T11:29:29.270 INFO:tasks.workunit.client.0.smithi144.stdout:7/43: link l0 d5/dc/ld 0 2024-09-10T11:29:29.275 INFO:tasks.workunit.client.0.smithi144.stdout:1/81: getdents d5 0 2024-09-10T11:29:29.276 INFO:tasks.workunit.client.0.smithi144.stdout:7/44: unlink c2 0 2024-09-10T11:29:29.276 INFO:tasks.workunit.client.0.smithi144.stdout:1/82: fdatasync d5/dd/f13 0 2024-09-10T11:29:29.276 INFO:tasks.workunit.client.1.smithi178.stdout:1/413: unlink d3/d9/d54/d2a/f76 0 2024-09-10T11:29:29.277 INFO:tasks.workunit.client.0.smithi144.stdout:7/45: chown d5/f7 12011 1 2024-09-10T11:29:29.279 INFO:tasks.workunit.client.1.smithi178.stdout:3/426: mknod db/d35/c7d 0 2024-09-10T11:29:29.281 INFO:tasks.workunit.client.0.smithi144.stdout:1/83: creat d5/f1b x:0 0 0 2024-09-10T11:29:29.287 INFO:tasks.workunit.client.0.smithi144.stdout:0/72: rmdir da/df 0 2024-09-10T11:29:29.288 INFO:tasks.workunit.client.1.smithi178.stdout:2/632: write d1/d7/d24/d7d/d84/fb6 [288483,421] 0 2024-09-10T11:29:29.288 INFO:tasks.workunit.client.0.smithi144.stdout:7/46: creat d5/dc/fe x:0 0 0 2024-09-10T11:29:29.289 INFO:tasks.workunit.client.1.smithi178.stdout:8/501: dread d5/f7 [0,4194304] 0 2024-09-10T11:29:29.291 INFO:tasks.workunit.client.0.smithi144.stdout:7/47: symlink d5/lf 0 2024-09-10T11:29:29.293 INFO:tasks.workunit.client.0.smithi144.stdout:7/48: creat d5/dc/f10 x:0 0 0 2024-09-10T11:29:29.296 INFO:tasks.workunit.client.0.smithi144.stdout:7/49: mkdir d5/dc/d11 0 2024-09-10T11:29:29.296 INFO:tasks.workunit.client.0.smithi144.stdout:7/50: chown d5/dc/fe 9244 1 2024-09-10T11:29:29.299 INFO:tasks.workunit.client.1.smithi178.stdout:8/502: dread - d5/db/f88 zero size 2024-09-10T11:29:29.299 INFO:tasks.workunit.client.1.smithi178.stdout:5/477: dread d9/f90 [0,4194304] 0 2024-09-10T11:29:29.299 INFO:tasks.workunit.client.1.smithi178.stdout:3/427: truncate db/f16 2090714 0 2024-09-10T11:29:29.299 INFO:tasks.workunit.client.0.smithi144.stdout:7/51: mkdir d5/dc/d12 0 2024-09-10T11:29:29.301 INFO:tasks.workunit.client.1.smithi178.stdout:2/633: dwrite d1/d22/fae [0,4194304] 0 2024-09-10T11:29:29.301 INFO:tasks.workunit.client.1.smithi178.stdout:1/414: dwrite d3/d9/d54/f51 [0,4194304] 0 2024-09-10T11:29:29.302 INFO:tasks.workunit.client.0.smithi144.stdout:7/52: link d5/fa d5/dc/d12/f13 0 2024-09-10T11:29:29.303 INFO:tasks.workunit.client.1.smithi178.stdout:7/366: sync 2024-09-10T11:29:29.306 INFO:tasks.workunit.client.1.smithi178.stdout:3/428: mkdir db/d20/d3e/d6b/d7e 0 2024-09-10T11:29:29.307 INFO:tasks.workunit.client.0.smithi144.stdout:7/53: dread d5/fa [0,4194304] 0 2024-09-10T11:29:29.308 INFO:tasks.workunit.client.1.smithi178.stdout:7/367: stat d6/fd 0 2024-09-10T11:29:29.310 INFO:tasks.workunit.client.0.smithi144.stdout:7/54: mkdir d5/dc/d14 0 2024-09-10T11:29:29.311 INFO:tasks.workunit.client.1.smithi178.stdout:1/415: creat d3/d9/d54/d3a/d63/f97 x:0 0 0 2024-09-10T11:29:29.314 INFO:tasks.workunit.client.1.smithi178.stdout:8/503: rename d5/db/d10/d20/d52/l84 to d5/db/d10/d20/d52/la5 0 2024-09-10T11:29:29.315 INFO:tasks.workunit.client.0.smithi144.stdout:7/55: dread d5/dc/d12/f13 [0,4194304] 0 2024-09-10T11:29:29.315 INFO:tasks.workunit.client.1.smithi178.stdout:1/416: dread d3/d9/d21/d4b/d2b/f57 [0,4194304] 0 2024-09-10T11:29:29.319 INFO:tasks.workunit.client.0.smithi144.stdout:7/56: mknod d5/dc/d11/c15 0 2024-09-10T11:29:29.320 INFO:tasks.workunit.client.0.smithi144.stdout:7/57: rename d5/dc/d12 to d5/dc/d12/d16 22 2024-09-10T11:29:29.321 INFO:tasks.workunit.client.1.smithi178.stdout:8/504: dread d5/f13 [0,4194304] 0 2024-09-10T11:29:29.322 INFO:tasks.workunit.client.1.smithi178.stdout:1/417: readlink d3/d9/l8e 0 2024-09-10T11:29:29.323 INFO:tasks.workunit.client.1.smithi178.stdout:2/634: getdents d1/d7 0 2024-09-10T11:29:29.325 INFO:tasks.workunit.client.1.smithi178.stdout:8/505: chown d5/d1d/l73 247 1 2024-09-10T11:29:29.325 INFO:tasks.workunit.client.1.smithi178.stdout:7/368: link d6/c12 d6/d10/c86 0 2024-09-10T11:29:29.326 INFO:tasks.workunit.client.0.smithi144.stdout:7/58: creat d5/dc/d11/f17 x:0 0 0 2024-09-10T11:29:29.326 INFO:tasks.workunit.client.1.smithi178.stdout:2/635: stat d1/d22/c77 0 2024-09-10T11:29:29.326 INFO:tasks.workunit.client.0.smithi144.stdout:1/84: read d5/df/f19 [2091917,95840] 0 2024-09-10T11:29:29.327 INFO:tasks.workunit.client.0.smithi144.stdout:7/59: readlink l0 0 2024-09-10T11:29:29.328 INFO:tasks.workunit.client.0.smithi144.stdout:1/85: chown d5/f7 1791278 1 2024-09-10T11:29:29.329 INFO:tasks.workunit.client.0.smithi144.stdout:1/86: dread - d5/f1b zero size 2024-09-10T11:29:29.330 INFO:tasks.workunit.client.1.smithi178.stdout:1/418: mkdir d3/d9/d98 0 2024-09-10T11:29:29.330 INFO:tasks.workunit.client.0.smithi144.stdout:7/60: write d5/fa [2144166,7715] 0 2024-09-10T11:29:29.330 INFO:tasks.workunit.client.0.smithi144.stdout:7/61: chown d5/dc/d14 1062306469 1 2024-09-10T11:29:29.331 INFO:tasks.workunit.client.1.smithi178.stdout:5/478: dread d9/dc/d17/d1d/f54 [0,4194304] 0 2024-09-10T11:29:29.332 INFO:tasks.workunit.client.1.smithi178.stdout:3/429: dwrite db/d35/f5b [0,4194304] 0 2024-09-10T11:29:29.332 INFO:tasks.workunit.client.0.smithi144.stdout:1/87: chown d5/f18 36461597 1 2024-09-10T11:29:29.333 INFO:tasks.workunit.client.1.smithi178.stdout:2/636: unlink d1/d7/d24/d7d/c81 0 2024-09-10T11:29:29.334 INFO:tasks.workunit.client.0.smithi144.stdout:7/62: mkdir d5/dc/d11/d18 0 2024-09-10T11:29:29.336 INFO:tasks.workunit.client.1.smithi178.stdout:8/506: symlink d5/db/d10/d35/la6 0 2024-09-10T11:29:29.337 INFO:tasks.workunit.client.1.smithi178.stdout:7/369: rename d6/l75 to d6/d10/d1e/d24/d49/d7c/l87 0 2024-09-10T11:29:29.337 INFO:tasks.workunit.client.0.smithi144.stdout:1/88: mknod d5/dd/c1c 0 2024-09-10T11:29:29.338 INFO:tasks.workunit.client.1.smithi178.stdout:5/479: write d9/dc/dd/d16/f8e [1172389,114143] 0 2024-09-10T11:29:29.339 INFO:tasks.workunit.client.0.smithi144.stdout:7/63: truncate d5/f8 1337951 0 2024-09-10T11:29:29.340 INFO:tasks.workunit.client.1.smithi178.stdout:7/370: write d6/d10/d1d/d22/d3b/f5a [162977,115051] 0 2024-09-10T11:29:29.341 INFO:tasks.workunit.client.1.smithi178.stdout:1/419: chown d3/d9/d54/d3a/d50/l89 7 1 2024-09-10T11:29:29.341 INFO:tasks.workunit.client.1.smithi178.stdout:7/371: chown d6/d10/d34/f74 4 1 2024-09-10T11:29:29.342 INFO:tasks.workunit.client.1.smithi178.stdout:2/637: mknod d1/d7/d24/d46/d88/d9d/dab/cc9 0 2024-09-10T11:29:29.347 INFO:tasks.workunit.client.0.smithi144.stdout:7/64: symlink d5/dc/d11/d18/l19 0 2024-09-10T11:29:29.349 INFO:tasks.workunit.client.1.smithi178.stdout:7/372: symlink d6/d10/d34/d6a/d6f/l88 0 2024-09-10T11:29:29.351 INFO:tasks.workunit.client.1.smithi178.stdout:8/507: getdents d5/db/d10/d20/d52 0 2024-09-10T11:29:29.352 INFO:tasks.workunit.client.0.smithi144.stdout:1/89: getdents d5/dd 0 2024-09-10T11:29:29.353 INFO:tasks.workunit.client.0.smithi144.stdout:1/90: fdatasync d5/df/f16 0 2024-09-10T11:29:29.354 INFO:tasks.workunit.client.1.smithi178.stdout:1/420: rename d3/d35/f68 to d3/d35/f99 0 2024-09-10T11:29:29.355 INFO:tasks.workunit.client.1.smithi178.stdout:5/480: dwrite d9/d19/f67 [0,4194304] 0 2024-09-10T11:29:29.355 INFO:tasks.workunit.client.0.smithi144.stdout:1/91: readlink d5/l9 0 2024-09-10T11:29:29.356 INFO:tasks.workunit.client.1.smithi178.stdout:3/430: truncate db/f27 1056571 0 2024-09-10T11:29:29.357 INFO:tasks.workunit.client.1.smithi178.stdout:7/373: symlink d6/d10/d1d/d4a/d79/l89 0 2024-09-10T11:29:29.358 INFO:tasks.workunit.client.1.smithi178.stdout:1/421: chown d3/d9/d54/d2a/f41 4049822 1 2024-09-10T11:29:29.360 INFO:tasks.workunit.client.1.smithi178.stdout:2/638: dwrite d1/d41/d8d/fb7 [0,4194304] 0 2024-09-10T11:29:29.361 INFO:tasks.workunit.client.1.smithi178.stdout:8/508: fsync d5/db/f14 0 2024-09-10T11:29:29.361 INFO:tasks.workunit.client.1.smithi178.stdout:5/481: truncate d9/dc/dd/f2d 1128575 0 2024-09-10T11:29:29.362 INFO:tasks.workunit.client.1.smithi178.stdout:5/482: chown d9/dc/d17/d1d 2 1 2024-09-10T11:29:29.365 INFO:tasks.workunit.client.1.smithi178.stdout:5/483: chown d9/dc/dd/d3f/f8b 207 1 2024-09-10T11:29:29.368 INFO:tasks.workunit.client.1.smithi178.stdout:2/639: creat d1/d22/fca x:0 0 0 2024-09-10T11:29:29.370 INFO:tasks.workunit.client.1.smithi178.stdout:2/640: fdatasync d1/d41/f51 0 2024-09-10T11:29:29.370 INFO:tasks.workunit.client.1.smithi178.stdout:8/509: mkdir d5/d30/d33/d8c/da7 0 2024-09-10T11:29:29.371 INFO:tasks.workunit.client.1.smithi178.stdout:7/374: write d6/d10/d1e/d2d/f16 [1303728,80486] 0 2024-09-10T11:29:29.373 INFO:tasks.workunit.client.1.smithi178.stdout:5/484: rename d9/dc/dd/f2d to d9/dc/dd/d37/d3e/fa6 0 2024-09-10T11:29:29.375 INFO:tasks.workunit.client.1.smithi178.stdout:1/422: rename d3/d9/d54/d3a/d63 to d3/d9/d54/d3a/d63/d9a 22 2024-09-10T11:29:29.375 INFO:tasks.workunit.client.1.smithi178.stdout:7/375: write d6/f14 [1664899,101995] 0 2024-09-10T11:29:29.377 INFO:tasks.workunit.client.1.smithi178.stdout:7/376: fdatasync f3 0 2024-09-10T11:29:29.378 INFO:tasks.workunit.client.1.smithi178.stdout:4/329: sync 2024-09-10T11:29:29.381 INFO:tasks.workunit.client.1.smithi178.stdout:3/431: dwrite db/d35/d58/f7b [0,4194304] 0 2024-09-10T11:29:29.381 INFO:tasks.workunit.client.1.smithi178.stdout:1/423: stat d3/d9/d21/d4b/d2b/l55 0 2024-09-10T11:29:29.381 INFO:tasks.workunit.client.1.smithi178.stdout:7/377: readlink d6/d10/d1d/d4a/l53 0 2024-09-10T11:29:29.381 INFO:tasks.workunit.client.1.smithi178.stdout:5/485: mknod d9/dc/dd/d3f/ca7 0 2024-09-10T11:29:29.383 INFO:tasks.workunit.client.1.smithi178.stdout:2/641: getdents d1/d41/d14/d11/db9 0 2024-09-10T11:29:29.383 INFO:tasks.workunit.client.1.smithi178.stdout:5/486: readlink d9/dc/d17/l24 0 2024-09-10T11:29:29.384 INFO:tasks.workunit.client.1.smithi178.stdout:3/432: dread - db/d20/d3d/d63/d6a/f6c zero size 2024-09-10T11:29:29.385 INFO:tasks.workunit.client.1.smithi178.stdout:4/330: symlink d0/d8/l72 0 2024-09-10T11:29:29.387 INFO:tasks.workunit.client.1.smithi178.stdout:7/378: creat d6/d10/d1e/d2d/f8a x:0 0 0 2024-09-10T11:29:29.391 INFO:tasks.workunit.client.1.smithi178.stdout:8/510: dwrite d5/d1d/d22/d76/f9e [0,4194304] 0 2024-09-10T11:29:29.391 INFO:tasks.workunit.client.1.smithi178.stdout:4/331: symlink d0/d8/da/l73 0 2024-09-10T11:29:29.392 INFO:tasks.workunit.client.1.smithi178.stdout:8/511: write d5/db/d10/d20/f2c [9053772,60898] 0 2024-09-10T11:29:29.393 INFO:tasks.workunit.client.1.smithi178.stdout:5/487: mknod d9/dc/dd/d37/d3e/ca8 0 2024-09-10T11:29:29.394 INFO:tasks.workunit.client.1.smithi178.stdout:5/488: readlink d9/dc/dd/l83 0 2024-09-10T11:29:29.394 INFO:tasks.workunit.client.1.smithi178.stdout:5/489: stat d9/d11/c93 0 2024-09-10T11:29:29.399 INFO:tasks.workunit.client.1.smithi178.stdout:7/379: symlink d6/d10/d1d/d22/l8b 0 2024-09-10T11:29:29.400 INFO:tasks.workunit.client.1.smithi178.stdout:8/512: rename d5/db/c46 to d5/d30/d33/d3a/ca8 0 2024-09-10T11:29:29.401 INFO:tasks.workunit.client.1.smithi178.stdout:5/490: rmdir d9/dc/dd/d16 39 2024-09-10T11:29:29.403 INFO:tasks.workunit.client.0.smithi144.stdout:5/65: sync 2024-09-10T11:29:29.404 INFO:tasks.workunit.client.1.smithi178.stdout:4/332: write d0/d8/df/f3b [845446,69302] 0 2024-09-10T11:29:29.404 INFO:tasks.workunit.client.1.smithi178.stdout:8/513: read - d5/db/d10/d53/d82/f92 zero size 2024-09-10T11:29:29.404 INFO:tasks.workunit.client.1.smithi178.stdout:2/642: getdents d1/d7/d24/d7d/d84/d55/da1 0 2024-09-10T11:29:29.406 INFO:tasks.workunit.client.0.smithi144.stdout:5/66: dread d4/fa [0,4194304] 0 2024-09-10T11:29:29.486 INFO:tasks.workunit.client.0.smithi144.stdout:5/67: unlink d4/lb 0 2024-09-10T11:29:29.486 INFO:tasks.workunit.client.0.smithi144.stdout:5/68: symlink d4/d5/l13 0 2024-09-10T11:29:29.486 INFO:tasks.workunit.client.0.smithi144.stdout:5/69: mknod d4/c14 0 2024-09-10T11:29:29.486 INFO:tasks.workunit.client.0.smithi144.stdout:5/70: dread d4/fa [0,4194304] 0 2024-09-10T11:29:29.487 INFO:tasks.workunit.client.0.smithi144.stdout:5/71: creat d4/f15 x:0 0 0 2024-09-10T11:29:29.487 INFO:tasks.workunit.client.0.smithi144.stdout:5/72: symlink d4/l16 0 2024-09-10T11:29:29.487 INFO:tasks.workunit.client.0.smithi144.stdout:5/73: write d4/fa [2611828,3618] 0 2024-09-10T11:29:29.487 INFO:tasks.workunit.client.0.smithi144.stdout:5/74: write d4/fa [442508,99382] 0 2024-09-10T11:29:29.487 INFO:tasks.workunit.client.0.smithi144.stdout:5/75: symlink d4/d5/l17 0 2024-09-10T11:29:29.492 INFO:tasks.workunit.client.1.smithi178.stdout:5/491: creat d9/d19/d91/fa9 x:0 0 0 2024-09-10T11:29:29.492 INFO:tasks.workunit.client.1.smithi178.stdout:4/333: chown d0/d21/l3c 198941 1 2024-09-10T11:29:29.492 INFO:tasks.workunit.client.1.smithi178.stdout:1/424: dwrite d3/d9/d21/d4b/f5a [0,4194304] 0 2024-09-10T11:29:29.492 INFO:tasks.workunit.client.1.smithi178.stdout:8/514: creat d5/db/d1e/fa9 x:0 0 0 2024-09-10T11:29:29.492 INFO:tasks.workunit.client.1.smithi178.stdout:2/643: mkdir d1/d7/dcb 0 2024-09-10T11:29:29.492 INFO:tasks.workunit.client.1.smithi178.stdout:8/515: stat d5/d30/d33/d75/f91 0 2024-09-10T11:29:29.492 INFO:tasks.workunit.client.1.smithi178.stdout:3/433: dwrite db/f4f [0,4194304] 0 2024-09-10T11:29:29.492 INFO:tasks.workunit.client.1.smithi178.stdout:1/425: unlink d3/d9/f3c 0 2024-09-10T11:29:29.492 INFO:tasks.workunit.client.1.smithi178.stdout:8/516: readlink d5/db/d10/l1a 0 2024-09-10T11:29:29.493 INFO:tasks.workunit.client.1.smithi178.stdout:1/426: read - d3/d9/d1a/d48/f90 zero size 2024-09-10T11:29:29.493 INFO:tasks.workunit.client.1.smithi178.stdout:6/515: sync 2024-09-10T11:29:29.493 INFO:tasks.workunit.client.1.smithi178.stdout:0/506: sync 2024-09-10T11:29:29.493 INFO:tasks.workunit.client.1.smithi178.stdout:9/533: sync 2024-09-10T11:29:29.493 INFO:tasks.workunit.client.1.smithi178.stdout:5/492: rename d9/dc/dd/d37/d42/f89 to d9/d11/faa 0 2024-09-10T11:29:29.493 INFO:tasks.workunit.client.1.smithi178.stdout:2/644: truncate d1/d22/fca 526000 0 2024-09-10T11:29:29.493 INFO:tasks.workunit.client.1.smithi178.stdout:8/517: mkdir d5/d30/d33/d8c/daa 0 2024-09-10T11:29:29.493 INFO:tasks.workunit.client.1.smithi178.stdout:5/493: write d9/dc/dd/d37/d42/f8a [301027,114308] 0 2024-09-10T11:29:29.493 INFO:tasks.workunit.client.1.smithi178.stdout:1/427: unlink d3/d9/d54/d3a/d6e/d8a/c1d 0 2024-09-10T11:29:29.493 INFO:tasks.workunit.client.1.smithi178.stdout:6/516: creat d3/d7/d8/d15/fbf x:0 0 0 2024-09-10T11:29:29.493 INFO:tasks.workunit.client.1.smithi178.stdout:8/518: chown d5/db/d1e/fa9 44 1 2024-09-10T11:29:29.493 INFO:tasks.workunit.client.1.smithi178.stdout:2/645: write d1/d41/d14/d11/d53/d97/f99 [231892,91861] 0 2024-09-10T11:29:29.493 INFO:tasks.workunit.client.1.smithi178.stdout:0/507: readlink d0/dc/d39/d50/d5e/l6a 0 2024-09-10T11:29:29.494 INFO:tasks.workunit.client.1.smithi178.stdout:4/334: dwrite d0/d9/f13 [0,4194304] 0 2024-09-10T11:29:29.494 INFO:tasks.workunit.client.1.smithi178.stdout:7/380: dread f4 [0,4194304] 0 2024-09-10T11:29:29.494 INFO:tasks.workunit.client.1.smithi178.stdout:9/534: rename d2/d8/l1d to d2/l9d 0 2024-09-10T11:29:29.494 INFO:tasks.workunit.client.1.smithi178.stdout:4/335: stat d0/d10/l35 0 2024-09-10T11:29:29.494 INFO:tasks.workunit.client.1.smithi178.stdout:8/519: write d5/d1d/f50 [797182,56967] 0 2024-09-10T11:29:29.494 INFO:tasks.workunit.client.1.smithi178.stdout:2/646: chown d1/d7/d24/d43/f87 4026552 1 2024-09-10T11:29:29.494 INFO:tasks.workunit.client.1.smithi178.stdout:7/381: fsync d6/f30 0 2024-09-10T11:29:29.494 INFO:tasks.workunit.client.1.smithi178.stdout:9/535: creat d2/d19/d5e/f9e x:0 0 0 2024-09-10T11:29:29.494 INFO:tasks.workunit.client.1.smithi178.stdout:0/508: creat d0/d12/d3f/fa5 x:0 0 0 2024-09-10T11:29:29.494 INFO:tasks.workunit.client.1.smithi178.stdout:5/494: rename d9/c88 to d9/dc/dd/d3f/d5a/cab 0 2024-09-10T11:29:29.494 INFO:tasks.workunit.client.1.smithi178.stdout:1/428: creat d3/d9/d54/d3a/d50/f9b x:0 0 0 2024-09-10T11:29:29.494 INFO:tasks.workunit.client.1.smithi178.stdout:6/517: dread d3/d7/d8/d15/d1c/f4d [0,4194304] 0 2024-09-10T11:29:29.494 INFO:tasks.workunit.client.1.smithi178.stdout:9/536: mknod d2/d9b/c9f 0 2024-09-10T11:29:29.495 INFO:tasks.workunit.client.1.smithi178.stdout:8/520: mknod d5/d30/d33/d8c/cab 0 2024-09-10T11:29:29.495 INFO:tasks.workunit.client.1.smithi178.stdout:4/336: dwrite d0/d8/df/f29 [0,4194304] 0 2024-09-10T11:29:29.495 INFO:tasks.workunit.client.1.smithi178.stdout:5/495: write d9/dc/dd/d37/d42/d62/f6f [779029,44680] 0 2024-09-10T11:29:29.495 INFO:tasks.workunit.client.1.smithi178.stdout:5/496: dread - d9/dc/dd/d37/d42/d62/f80 zero size 2024-09-10T11:29:29.495 INFO:tasks.workunit.client.1.smithi178.stdout:9/537: dwrite d2/d8/fa [0,4194304] 0 2024-09-10T11:29:29.495 INFO:tasks.workunit.client.1.smithi178.stdout:2/647: creat d1/d7/d24/d46/d88/d9d/dab/d5b/d60/fcc x:0 0 0 2024-09-10T11:29:29.495 INFO:tasks.workunit.client.1.smithi178.stdout:0/509: dread d0/dc/d39/d4c/f3a [0,4194304] 0 2024-09-10T11:29:29.495 INFO:tasks.workunit.client.1.smithi178.stdout:9/538: fdatasync d2/dc/f2f 0 2024-09-10T11:29:29.495 INFO:tasks.workunit.client.1.smithi178.stdout:0/510: dread - d0/d12/d3f/d72/f82 zero size 2024-09-10T11:29:29.495 INFO:tasks.workunit.client.1.smithi178.stdout:1/429: write d3/d9/d54/d3a/f4e [1077879,91920] 0 2024-09-10T11:29:29.495 INFO:tasks.workunit.client.1.smithi178.stdout:2/648: dread - d1/d7/d24/d46/d88/d9d/dab/d5b/d60/fcc zero size 2024-09-10T11:29:29.495 INFO:tasks.workunit.client.1.smithi178.stdout:5/497: chown d9/dc/dd/d37/d42/d55/f7a 178998933 1 2024-09-10T11:29:29.495 INFO:tasks.workunit.client.1.smithi178.stdout:6/518: write d3/d7/d8/d15/d1c/f40 [675903,124553] 0 2024-09-10T11:29:29.495 INFO:tasks.workunit.client.1.smithi178.stdout:7/382: link d6/d10/d61/f77 d6/d10/d1e/d24/d49/d7c/f8c 0 2024-09-10T11:29:29.496 INFO:tasks.workunit.client.1.smithi178.stdout:8/521: mknod d5/d30/d33/d8c/cac 0 2024-09-10T11:29:29.496 INFO:tasks.workunit.client.1.smithi178.stdout:5/498: chown d9/dc/dd/d3f/f8b 1337 1 2024-09-10T11:29:29.496 INFO:tasks.workunit.client.1.smithi178.stdout:4/337: mknod d0/d21/c74 0 2024-09-10T11:29:29.496 INFO:tasks.workunit.client.1.smithi178.stdout:4/338: dread - d0/d8/da/d1a/f4b zero size 2024-09-10T11:29:29.496 INFO:tasks.workunit.client.1.smithi178.stdout:9/539: unlink d2/d13/f86 0 2024-09-10T11:29:29.496 INFO:tasks.workunit.client.1.smithi178.stdout:5/499: dwrite d9/d19/d91/fa9 [0,4194304] 0 2024-09-10T11:29:29.496 INFO:tasks.workunit.client.1.smithi178.stdout:7/383: creat d6/d10/d34/f8d x:0 0 0 2024-09-10T11:29:29.496 INFO:tasks.workunit.client.1.smithi178.stdout:7/384: chown d6/d10/d1e/d24/d49/d7c 9 1 2024-09-10T11:29:29.496 INFO:tasks.workunit.client.1.smithi178.stdout:6/519: creat d3/d7/d8/d15/d1c/d2a/fc0 x:0 0 0 2024-09-10T11:29:29.496 INFO:tasks.workunit.client.1.smithi178.stdout:2/649: unlink d1/d7/d24/d46/d88/d9d/dab/d5e/c7f 0 2024-09-10T11:29:29.496 INFO:tasks.workunit.client.1.smithi178.stdout:2/650: stat d1/d41/d5d/l7e 0 2024-09-10T11:29:29.496 INFO:tasks.workunit.client.1.smithi178.stdout:8/522: mkdir d5/db/d10/d20/d52/d5c/dad 0 2024-09-10T11:29:29.496 INFO:tasks.workunit.client.1.smithi178.stdout:2/651: dread d1/d41/f1e [0,4194304] 0 2024-09-10T11:29:29.496 INFO:tasks.workunit.client.1.smithi178.stdout:9/540: mknod d2/d13/d40/d89/d93/d88/ca0 0 2024-09-10T11:29:29.497 INFO:tasks.workunit.client.1.smithi178.stdout:9/541: fdatasync d2/d19/d5e/f78 0 2024-09-10T11:29:29.497 INFO:tasks.workunit.client.1.smithi178.stdout:2/652: dwrite d1/d7/d24/d7d/d84/d4b/d89/f93 [0,4194304] 0 2024-09-10T11:29:29.497 INFO:tasks.workunit.client.1.smithi178.stdout:0/511: mknod d0/dc/d1f/ca6 0 2024-09-10T11:29:29.503 INFO:tasks.workunit.client.1.smithi178.stdout:9/542: unlink d2/d19/d5e/f6a 0 2024-09-10T11:29:29.503 INFO:tasks.workunit.client.1.smithi178.stdout:9/543: stat d2/df/d31/d4a 0 2024-09-10T11:29:29.504 INFO:tasks.workunit.client.1.smithi178.stdout:9/544: write d2/d8/f1a [2501460,101518] 0 2024-09-10T11:29:29.507 INFO:tasks.workunit.client.1.smithi178.stdout:5/500: write d9/dc/dd/d37/d42/f5f [348635,117975] 0 2024-09-10T11:29:29.508 INFO:tasks.workunit.client.1.smithi178.stdout:4/339: creat d0/f75 x:0 0 0 2024-09-10T11:29:29.509 INFO:tasks.workunit.client.1.smithi178.stdout:4/340: write d0/d8/df/f20 [254674,37208] 0 2024-09-10T11:29:29.510 INFO:tasks.workunit.client.1.smithi178.stdout:2/653: rename d1/d7/d24/d46/d88/d9d/dab/d5b/f6b to d1/d7/d24/d46/d88/fcd 0 2024-09-10T11:29:29.512 INFO:tasks.workunit.client.1.smithi178.stdout:7/385: creat d6/d10/d1e/f8e x:0 0 0 2024-09-10T11:29:29.514 INFO:tasks.workunit.client.1.smithi178.stdout:7/386: write d6/d10/d1e/f26 [1120722,108221] 0 2024-09-10T11:29:29.514 INFO:tasks.workunit.client.1.smithi178.stdout:1/430: mknod d3/d9/d54/d3a/d50/d6c/d78/c9c 0 2024-09-10T11:29:29.518 INFO:tasks.workunit.client.1.smithi178.stdout:9/545: symlink d2/df/d31/d37/d3d/la1 0 2024-09-10T11:29:29.519 INFO:tasks.workunit.client.1.smithi178.stdout:8/523: creat d5/fae x:0 0 0 2024-09-10T11:29:29.519 INFO:tasks.workunit.client.1.smithi178.stdout:5/501: mknod d9/dc/d17/d1d/cac 0 2024-09-10T11:29:29.520 INFO:tasks.workunit.client.1.smithi178.stdout:0/512: symlink d0/dc/d39/d50/la7 0 2024-09-10T11:29:29.521 INFO:tasks.workunit.client.1.smithi178.stdout:5/502: chown f2 143 1 2024-09-10T11:29:29.521 INFO:tasks.workunit.client.1.smithi178.stdout:4/341: rmdir d0/d9/dc 39 2024-09-10T11:29:29.523 INFO:tasks.workunit.client.1.smithi178.stdout:0/513: chown d0/dc/d39/d50/d5e/f78 177 1 2024-09-10T11:29:29.524 INFO:tasks.workunit.client.1.smithi178.stdout:7/387: write d6/d10/d1e/d2d/f21 [4261003,3607] 0 2024-09-10T11:29:29.525 INFO:tasks.workunit.client.1.smithi178.stdout:8/524: symlink d5/d30/d33/d75/laf 0 2024-09-10T11:29:29.525 INFO:tasks.workunit.client.1.smithi178.stdout:6/520: dwrite d3/d7/fb0 [0,4194304] 0 2024-09-10T11:29:29.527 INFO:tasks.workunit.client.1.smithi178.stdout:4/342: creat d0/d63/f76 x:0 0 0 2024-09-10T11:29:29.529 INFO:tasks.workunit.client.1.smithi178.stdout:4/343: chown d0/d21/l24 1087 1 2024-09-10T11:29:29.533 INFO:tasks.workunit.client.1.smithi178.stdout:7/388: rmdir d6/d10/d1e/d2d 39 2024-09-10T11:29:29.535 INFO:tasks.workunit.client.1.smithi178.stdout:6/521: mknod d3/d6e/d87/cc1 0 2024-09-10T11:29:29.535 INFO:tasks.workunit.client.1.smithi178.stdout:8/525: unlink d5/d30/d33/d75/l90 0 2024-09-10T11:29:29.537 INFO:tasks.workunit.client.1.smithi178.stdout:2/654: dwrite d1/d7/d24/d46/f63 [0,4194304] 0 2024-09-10T11:29:29.537 INFO:tasks.workunit.client.1.smithi178.stdout:4/344: rename d0/d63/f71 to d0/d21/f77 0 2024-09-10T11:29:29.538 INFO:tasks.workunit.client.1.smithi178.stdout:7/389: rmdir d6/d10/d34/d6a/d6f 39 2024-09-10T11:29:29.540 INFO:tasks.workunit.client.1.smithi178.stdout:9/546: rmdir d2/df/d31/d37/d3d 39 2024-09-10T11:29:29.540 INFO:tasks.workunit.client.1.smithi178.stdout:2/655: chown d1/d7/d24/d46/d88/d9d/dab/c28 47994 1 2024-09-10T11:29:29.541 INFO:tasks.workunit.client.1.smithi178.stdout:4/345: creat d0/d63/f78 x:0 0 0 2024-09-10T11:29:29.541 INFO:tasks.workunit.client.1.smithi178.stdout:8/526: chown d5/db/d3d/c5d 430300 1 2024-09-10T11:29:29.543 INFO:tasks.workunit.client.1.smithi178.stdout:0/514: dread d0/dc/ff [0,4194304] 0 2024-09-10T11:29:29.544 INFO:tasks.workunit.client.1.smithi178.stdout:4/346: write d0/f75 [529272,19156] 0 2024-09-10T11:29:29.545 INFO:tasks.workunit.client.1.smithi178.stdout:4/347: fsync d0/d8/df/f20 0 2024-09-10T11:29:29.546 INFO:tasks.workunit.client.1.smithi178.stdout:2/656: symlink d1/d41/d14/d11/lce 0 2024-09-10T11:29:29.546 INFO:tasks.workunit.client.1.smithi178.stdout:2/657: stat d1/d7/d24/f2c 0 2024-09-10T11:29:29.548 INFO:tasks.workunit.client.1.smithi178.stdout:7/390: rename d6/d10/d1d/f31 to d6/d10/d1d/d22/d50/d7e/f8f 0 2024-09-10T11:29:29.549 INFO:tasks.workunit.client.1.smithi178.stdout:8/527: unlink d5/db/d10/d20/f40 0 2024-09-10T11:29:29.550 INFO:tasks.workunit.client.1.smithi178.stdout:6/522: getdents d3/d74/d91 0 2024-09-10T11:29:29.551 INFO:tasks.workunit.client.1.smithi178.stdout:5/503: truncate f4 253841 0 2024-09-10T11:29:29.552 INFO:tasks.workunit.client.1.smithi178.stdout:8/528: chown d5/db/d10/d20/d52 3 1 2024-09-10T11:29:29.552 INFO:tasks.workunit.client.1.smithi178.stdout:0/515: creat d0/dc/d1f/d2d/fa8 x:0 0 0 2024-09-10T11:29:29.554 INFO:tasks.workunit.client.1.smithi178.stdout:6/523: stat d3/d7/d8/d15/d1c/d2a/f41 0 2024-09-10T11:29:29.554 INFO:tasks.workunit.client.1.smithi178.stdout:8/529: read - d5/fae zero size 2024-09-10T11:29:29.554 INFO:tasks.workunit.client.1.smithi178.stdout:4/348: symlink d0/d21/l79 0 2024-09-10T11:29:29.555 INFO:tasks.workunit.client.1.smithi178.stdout:1/431: dwrite d3/d9/fd [0,4194304] 0 2024-09-10T11:29:29.555 INFO:tasks.workunit.client.1.smithi178.stdout:0/516: dread - d0/d12/f6f zero size 2024-09-10T11:29:29.559 INFO:tasks.workunit.client.1.smithi178.stdout:6/524: write d3/d7/fbd [788716,7291] 0 2024-09-10T11:29:29.560 INFO:tasks.workunit.client.1.smithi178.stdout:2/658: dread d1/d7/f6c [0,4194304] 0 2024-09-10T11:29:29.561 INFO:tasks.workunit.client.1.smithi178.stdout:8/530: chown d5/l8e 13 1 2024-09-10T11:29:29.561 INFO:tasks.workunit.client.1.smithi178.stdout:4/349: creat d0/d8/da/d1a/d51/f7a x:0 0 0 2024-09-10T11:29:29.562 INFO:tasks.workunit.client.1.smithi178.stdout:6/525: chown d3/d7/c36 7 1 2024-09-10T11:29:29.563 INFO:tasks.workunit.client.1.smithi178.stdout:4/350: chown d0/f38 52 1 2024-09-10T11:29:29.564 INFO:tasks.workunit.client.1.smithi178.stdout:0/517: creat d0/dc/d39/d7c/fa9 x:0 0 0 2024-09-10T11:29:29.567 INFO:tasks.workunit.client.1.smithi178.stdout:8/531: creat d5/d1d/fb0 x:0 0 0 2024-09-10T11:29:29.568 INFO:tasks.workunit.client.1.smithi178.stdout:2/659: mkdir d1/d7/d24/d43/dcf 0 2024-09-10T11:29:29.569 INFO:tasks.workunit.client.1.smithi178.stdout:9/547: dwrite d2/df/f28 [0,4194304] 0 2024-09-10T11:29:29.570 INFO:tasks.workunit.client.1.smithi178.stdout:7/391: dread d6/fb [4194304,4194304] 0 2024-09-10T11:29:29.570 INFO:tasks.workunit.client.1.smithi178.stdout:5/504: creat d9/dc/dd/fad x:0 0 0 2024-09-10T11:29:29.572 INFO:tasks.workunit.client.1.smithi178.stdout:6/526: rename d3/d42 to d3/d74/d91/daa/dc2 0 2024-09-10T11:29:29.574 INFO:tasks.workunit.client.1.smithi178.stdout:2/660: symlink d1/d22/d4d/ld0 0 2024-09-10T11:29:29.576 INFO:tasks.workunit.client.1.smithi178.stdout:9/548: creat d2/d13/fa2 x:0 0 0 2024-09-10T11:29:29.577 INFO:tasks.workunit.client.1.smithi178.stdout:8/532: symlink d5/db/d10/d20/d52/d67/lb1 0 2024-09-10T11:29:29.578 INFO:tasks.workunit.client.1.smithi178.stdout:5/505: symlink d9/lae 0 2024-09-10T11:29:29.581 INFO:tasks.workunit.client.1.smithi178.stdout:8/533: write d5/fae [514153,95788] 0 2024-09-10T11:29:29.584 INFO:tasks.workunit.client.1.smithi178.stdout:7/392: dread d6/d10/d1d/f27 [0,4194304] 0 2024-09-10T11:29:29.584 INFO:tasks.workunit.client.1.smithi178.stdout:6/527: dread d3/d7/d8/d15/d1c/d20/d4b/f52 [0,4194304] 0 2024-09-10T11:29:29.585 INFO:tasks.workunit.client.1.smithi178.stdout:9/549: unlink d2/d19/d1c/l30 0 2024-09-10T11:29:29.590 INFO:tasks.workunit.client.1.smithi178.stdout:3/434: sync 2024-09-10T11:29:29.591 INFO:tasks.workunit.client.1.smithi178.stdout:6/528: mkdir d3/d74/d91/daa/dc2/d57/d78/dc3 0 2024-09-10T11:29:29.594 INFO:tasks.workunit.client.1.smithi178.stdout:1/432: dread d3/f1c [0,4194304] 0 2024-09-10T11:29:29.594 INFO:tasks.workunit.client.1.smithi178.stdout:8/534: dwrite d5/d30/f6a [0,4194304] 0 2024-09-10T11:29:29.601 INFO:tasks.workunit.client.1.smithi178.stdout:3/435: symlink db/d20/d3d/d63/l7f 0 2024-09-10T11:29:29.603 INFO:tasks.workunit.client.1.smithi178.stdout:7/393: getdents d6 0 2024-09-10T11:29:29.603 INFO:tasks.workunit.client.1.smithi178.stdout:5/506: write d9/dc/dd/d16/f27 [3146502,129376] 0 2024-09-10T11:29:29.605 INFO:tasks.workunit.client.1.smithi178.stdout:1/433: read - d3/d9/d1a/f5c zero size 2024-09-10T11:29:29.605 INFO:tasks.workunit.client.1.smithi178.stdout:6/529: write d3/d7/d8/d15/d1c/d20/d4b/f7b [594123,129791] 0 2024-09-10T11:29:29.606 INFO:tasks.workunit.client.1.smithi178.stdout:9/550: dwrite d2/df/d31/f47 [0,4194304] 0 2024-09-10T11:29:29.607 INFO:tasks.workunit.client.1.smithi178.stdout:8/535: mkdir d5/db/d10/d53/d82/db2 0 2024-09-10T11:29:29.608 INFO:tasks.workunit.client.1.smithi178.stdout:0/518: dread d0/dc/d39/d4c/f57 [0,4194304] 0 2024-09-10T11:29:29.609 INFO:tasks.workunit.client.1.smithi178.stdout:3/436: creat db/d26/f80 x:0 0 0 2024-09-10T11:29:29.614 INFO:tasks.workunit.client.1.smithi178.stdout:3/437: dread - db/d26/f80 zero size 2024-09-10T11:29:29.614 INFO:tasks.workunit.client.1.smithi178.stdout:7/394: creat d6/d10/d1e/d24/d49/d70/d4c/d55/f90 x:0 0 0 2024-09-10T11:29:29.615 INFO:tasks.workunit.client.1.smithi178.stdout:1/434: rename d3/d9/d1a/f37 to d3/d9/d1a/f9d 0 2024-09-10T11:29:29.617 INFO:tasks.workunit.client.1.smithi178.stdout:5/507: mkdir d9/dc/dd/d37/d42/d62/daf 0 2024-09-10T11:29:29.617 INFO:tasks.workunit.client.1.smithi178.stdout:2/661: dread d1/d7/d24/d7d/d84/f2d [0,4194304] 0 2024-09-10T11:29:29.618 INFO:tasks.workunit.client.1.smithi178.stdout:6/530: rename d3/d7/d8/d72/d75/l8c to d3/d6e/lc4 0 2024-09-10T11:29:29.622 INFO:tasks.workunit.client.1.smithi178.stdout:1/435: mkdir d3/d9/d54/d3a/d50/d9e 0 2024-09-10T11:29:29.623 INFO:tasks.workunit.client.1.smithi178.stdout:8/536: dwrite d5/db/d1e/fa3 [0,4194304] 0 2024-09-10T11:29:29.691 INFO:tasks.workunit.client.1.smithi178.stdout:5/508: mkdir d9/dc/d17/db0 0 2024-09-10T11:29:29.692 INFO:tasks.workunit.client.1.smithi178.stdout:9/551: rename d2/d19/d1c/d2d/f45 to d2/df/d31/d37/fa3 0 2024-09-10T11:29:29.692 INFO:tasks.workunit.client.1.smithi178.stdout:7/395: dread d6/d10/d1d/d22/d50/d7e/f8f [0,4194304] 0 2024-09-10T11:29:29.696 INFO:tasks.workunit.client.1.smithi178.stdout:9/552: mknod d2/d51/ca4 0 2024-09-10T11:29:29.696 INFO:tasks.workunit.client.1.smithi178.stdout:2/662: truncate d1/d22/d4d/d9b/f92 1414701 0 2024-09-10T11:29:29.697 INFO:tasks.workunit.client.1.smithi178.stdout:0/519: dwrite d0/dc/d1f/f7a [0,4194304] 0 2024-09-10T11:29:29.697 INFO:tasks.workunit.client.0.smithi144.stdout:9/33: sync 2024-09-10T11:29:29.698 INFO:tasks.workunit.client.0.smithi144.stdout:4/72: sync 2024-09-10T11:29:29.698 INFO:tasks.workunit.client.0.smithi144.stdout:8/53: sync 2024-09-10T11:29:29.699 INFO:tasks.workunit.client.0.smithi144.stdout:8/54: write d0/d9/ff [2807449,79780] 0 2024-09-10T11:29:29.701 INFO:tasks.workunit.client.1.smithi178.stdout:1/436: write d3/d9/d21/d4b/d2b/f94 [865392,78979] 0 2024-09-10T11:29:29.702 INFO:tasks.workunit.client.0.smithi144.stdout:8/55: mknod d0/da/c12 0 2024-09-10T11:29:29.703 INFO:tasks.workunit.client.1.smithi178.stdout:5/509: creat d9/dc/d17/d1d/fb1 x:0 0 0 2024-09-10T11:29:29.704 INFO:tasks.workunit.client.1.smithi178.stdout:9/553: mkdir d2/d19/d1c/da5 0 2024-09-10T11:29:29.705 INFO:tasks.workunit.client.0.smithi144.stdout:4/73: dread f1 [0,4194304] 0 2024-09-10T11:29:29.706 INFO:tasks.workunit.client.1.smithi178.stdout:3/438: dread db/d26/f2c [0,4194304] 0 2024-09-10T11:29:29.706 INFO:tasks.workunit.client.1.smithi178.stdout:2/663: symlink d1/d41/d14/d11/db9/dc1/ld1 0 2024-09-10T11:29:29.707 INFO:tasks.workunit.client.0.smithi144.stdout:9/34: dwrite f9 [0,4194304] 0 2024-09-10T11:29:29.707 INFO:tasks.workunit.client.0.smithi144.stdout:4/74: fsync fd 0 2024-09-10T11:29:29.707 INFO:tasks.workunit.client.1.smithi178.stdout:0/520: chown d0/dc/d39/d50/d5e/l6a 144 1 2024-09-10T11:29:29.707 INFO:tasks.workunit.client.0.smithi144.stdout:4/75: fdatasync f9 0 2024-09-10T11:29:29.708 INFO:tasks.workunit.client.1.smithi178.stdout:3/439: readlink db/l1f 0 2024-09-10T11:29:29.710 INFO:tasks.workunit.client.1.smithi178.stdout:8/537: dwrite d5/d30/d33/f49 [0,4194304] 0 2024-09-10T11:29:29.710 INFO:tasks.workunit.client.0.smithi144.stdout:8/56: dread d0/d9/ff [0,4194304] 0 2024-09-10T11:29:29.711 INFO:tasks.workunit.client.1.smithi178.stdout:1/437: mkdir d3/d9/d54/d3a/d9f 0 2024-09-10T11:29:29.716 INFO:tasks.workunit.client.1.smithi178.stdout:1/438: dread - d3/d9/d54/d3a/d63/f97 zero size 2024-09-10T11:29:29.716 INFO:tasks.workunit.client.1.smithi178.stdout:3/440: read f1 [192838,22745] 0 2024-09-10T11:29:29.718 INFO:tasks.workunit.client.1.smithi178.stdout:0/521: unlink d0/dc/f9e 0 2024-09-10T11:29:29.719 INFO:tasks.workunit.client.1.smithi178.stdout:7/396: dwrite d6/d10/d1e/d2d/f16 [0,4194304] 0 2024-09-10T11:29:29.719 INFO:tasks.workunit.client.1.smithi178.stdout:9/554: symlink d2/d13/d40/d89/d93/d88/la6 0 2024-09-10T11:29:29.721 INFO:tasks.workunit.client.1.smithi178.stdout:2/664: mkdir d1/d7/d24/d46/d88/d9d/dd2 0 2024-09-10T11:29:29.722 INFO:tasks.workunit.client.0.smithi144.stdout:4/76: getdents . 0 2024-09-10T11:29:29.724 INFO:tasks.workunit.client.0.smithi144.stdout:8/57: link d0/f10 d0/f13 0 2024-09-10T11:29:29.724 INFO:tasks.workunit.client.1.smithi178.stdout:8/538: mknod d5/cb3 0 2024-09-10T11:29:29.725 INFO:tasks.workunit.client.1.smithi178.stdout:3/441: write db/d26/f2c [2973878,98919] 0 2024-09-10T11:29:29.725 INFO:tasks.workunit.client.1.smithi178.stdout:2/665: write d1/d7/d24/d7d/d84/f35 [3981519,4705] 0 2024-09-10T11:29:29.725 INFO:tasks.workunit.client.0.smithi144.stdout:4/77: mkdir d14 0 2024-09-10T11:29:29.726 INFO:tasks.workunit.client.1.smithi178.stdout:0/522: mknod d0/d2c/d33/caa 0 2024-09-10T11:29:29.728 INFO:tasks.workunit.client.1.smithi178.stdout:5/510: dwrite d9/dc/dd/d3f/d5a/f9c [0,4194304] 0 2024-09-10T11:29:29.728 INFO:tasks.workunit.client.1.smithi178.stdout:1/439: mkdir d3/da0 0 2024-09-10T11:29:29.730 INFO:tasks.workunit.client.0.smithi144.stdout:4/78: rename f11 to d14/f15 0 2024-09-10T11:29:29.731 INFO:tasks.workunit.client.1.smithi178.stdout:8/539: mkdir d5/d30/db4 0 2024-09-10T11:29:29.731 INFO:tasks.workunit.client.0.smithi144.stdout:8/58: fdatasync d0/f10 0 2024-09-10T11:29:29.732 INFO:tasks.workunit.client.1.smithi178.stdout:1/440: write d3/d9/d54/d2a/f34 [1917851,28958] 0 2024-09-10T11:29:29.732 INFO:tasks.workunit.client.0.smithi144.stdout:9/35: dwrite f0 [0,4194304] 0 2024-09-10T11:29:29.733 INFO:tasks.workunit.client.0.smithi144.stdout:8/59: readlink d0/lb 0 2024-09-10T11:29:29.735 INFO:tasks.workunit.client.1.smithi178.stdout:4/351: sync 2024-09-10T11:29:29.737 INFO:tasks.workunit.client.1.smithi178.stdout:4/352: truncate d0/d10/f2c 942070 0 2024-09-10T11:29:29.737 INFO:tasks.workunit.client.0.smithi144.stdout:9/36: mknod cb 0 2024-09-10T11:29:29.738 INFO:tasks.workunit.client.0.smithi144.stdout:8/60: mknod d0/da/c14 0 2024-09-10T11:29:29.739 INFO:tasks.workunit.client.1.smithi178.stdout:4/353: write d0/d8/df/f3b [711158,103714] 0 2024-09-10T11:29:29.740 INFO:tasks.workunit.client.0.smithi144.stdout:4/79: link f1 d14/f16 0 2024-09-10T11:29:29.741 INFO:tasks.workunit.client.1.smithi178.stdout:7/397: rename d6/d10/d1e/f26 to d6/d10/d1e/d24/d49/d70/d4c/d55/f91 0 2024-09-10T11:29:29.742 INFO:tasks.workunit.client.0.smithi144.stdout:9/37: creat fc x:0 0 0 2024-09-10T11:29:29.743 INFO:tasks.workunit.client.0.smithi144.stdout:9/38: fsync f9 0 2024-09-10T11:29:29.744 INFO:tasks.workunit.client.0.smithi144.stdout:8/61: dread d0/d9/f11 [0,4194304] 0 2024-09-10T11:29:29.745 INFO:tasks.workunit.client.0.smithi144.stdout:4/80: symlink d14/l17 0 2024-09-10T11:29:29.745 INFO:tasks.workunit.client.1.smithi178.stdout:0/523: dwrite d0/dc/d39/d4c/f77 [0,4194304] 0 2024-09-10T11:29:29.746 INFO:tasks.workunit.client.1.smithi178.stdout:5/511: symlink d9/d11/lb2 0 2024-09-10T11:29:29.749 INFO:tasks.workunit.client.1.smithi178.stdout:9/555: dread d2/d8/f17 [0,4194304] 0 2024-09-10T11:29:29.749 INFO:tasks.workunit.client.1.smithi178.stdout:1/441: mknod d3/d9/d21/d6f/ca1 0 2024-09-10T11:29:29.749 INFO:tasks.workunit.client.0.smithi144.stdout:9/39: symlink ld 0 2024-09-10T11:29:29.750 INFO:tasks.workunit.client.1.smithi178.stdout:3/442: mkdir db/d20/d3e/d6b/d79/d81 0 2024-09-10T11:29:29.751 INFO:tasks.workunit.client.0.smithi144.stdout:8/62: rename d0/da/c12 to d0/d9/c15 0 2024-09-10T11:29:29.753 INFO:tasks.workunit.client.1.smithi178.stdout:4/354: symlink d0/d10/l7b 0 2024-09-10T11:29:29.753 INFO:tasks.workunit.client.0.smithi144.stdout:4/81: rename d14/f16 to d14/f18 0 2024-09-10T11:29:29.755 INFO:tasks.workunit.client.1.smithi178.stdout:8/540: rename d5/db/d10/d20/d52/d67/lb1 to d5/d30/d33/d8c/da7/lb5 0 2024-09-10T11:29:29.756 INFO:tasks.workunit.client.1.smithi178.stdout:7/398: dwrite d6/d10/d34/f74 [0,4194304] 0 2024-09-10T11:29:29.757 INFO:tasks.workunit.client.0.smithi144.stdout:9/40: dwrite f9 [0,4194304] 0 2024-09-10T11:29:29.760 INFO:tasks.workunit.client.0.smithi144.stdout:8/63: dread d0/f8 [0,4194304] 0 2024-09-10T11:29:29.760 INFO:tasks.workunit.client.1.smithi178.stdout:4/355: creat d0/d8/df/f7c x:0 0 0 2024-09-10T11:29:29.761 INFO:tasks.workunit.client.0.smithi144.stdout:8/64: fdatasync d0/d9/fe 0 2024-09-10T11:29:29.762 INFO:tasks.workunit.client.1.smithi178.stdout:4/356: stat d0/d10/l35 0 2024-09-10T11:29:29.763 INFO:tasks.workunit.client.1.smithi178.stdout:0/524: rename d0/d2c/d33/c66 to d0/dc/d39/d4c/cab 0 2024-09-10T11:29:29.764 INFO:tasks.workunit.client.1.smithi178.stdout:8/541: creat d5/d30/fb6 x:0 0 0 2024-09-10T11:29:29.765 INFO:tasks.workunit.client.0.smithi144.stdout:8/65: symlink d0/l16 0 2024-09-10T11:29:29.766 INFO:tasks.workunit.client.0.smithi144.stdout:8/66: chown d0/d9/fe 1057595 1 2024-09-10T11:29:29.767 INFO:tasks.workunit.client.1.smithi178.stdout:0/525: chown d0/d12/d3f/d72 538601 1 2024-09-10T11:29:29.769 INFO:tasks.workunit.client.1.smithi178.stdout:1/442: link d3/d9/fb d3/d9/d54/d3a/d9f/fa2 0 2024-09-10T11:29:29.770 INFO:tasks.workunit.client.0.smithi144.stdout:8/67: creat d0/da/f17 x:0 0 0 2024-09-10T11:29:29.771 INFO:tasks.workunit.client.1.smithi178.stdout:5/512: rmdir d9/d19/d91/d9a 0 2024-09-10T11:29:29.771 INFO:tasks.workunit.client.1.smithi178.stdout:1/443: write d3/d9/f59 [34316,38766] 0 2024-09-10T11:29:29.771 INFO:tasks.workunit.client.1.smithi178.stdout:4/357: dread d0/d10/f5a [0,4194304] 0 2024-09-10T11:29:29.771 INFO:tasks.workunit.client.1.smithi178.stdout:6/531: sync 2024-09-10T11:29:29.772 INFO:tasks.workunit.client.1.smithi178.stdout:7/399: creat d6/d10/d1d/d22/d50/d7e/d82/f92 x:0 0 0 2024-09-10T11:29:29.774 INFO:tasks.workunit.client.1.smithi178.stdout:8/542: fdatasync d5/d1d/f25 0 2024-09-10T11:29:29.774 INFO:tasks.workunit.client.0.smithi144.stdout:8/68: rename d0/l3 to d0/da/l18 0 2024-09-10T11:29:29.775 INFO:tasks.workunit.client.1.smithi178.stdout:5/513: truncate d9/dc/dd/d37/d42/f5f 535590 0 2024-09-10T11:29:29.775 INFO:tasks.workunit.client.0.smithi144.stdout:9/41: truncate f0 2644652 0 2024-09-10T11:29:29.776 INFO:tasks.workunit.client.1.smithi178.stdout:2/666: write d1/d7/d24/d7d/d84/d55/f5c [3599438,128401] 0 2024-09-10T11:29:29.777 INFO:tasks.workunit.client.1.smithi178.stdout:2/667: stat d1/d7/d24/d46/d88/d9d/dab/f17 0 2024-09-10T11:29:29.778 INFO:tasks.workunit.client.0.smithi144.stdout:9/42: symlink le 0 2024-09-10T11:29:29.778 INFO:tasks.workunit.client.0.smithi144.stdout:4/82: dwrite fa [0,4194304] 0 2024-09-10T11:29:29.780 INFO:tasks.workunit.client.1.smithi178.stdout:4/358: symlink d0/d8/d3e/l7d 0 2024-09-10T11:29:29.782 INFO:tasks.workunit.client.1.smithi178.stdout:4/359: chown d0/d21/c74 270 1 2024-09-10T11:29:29.783 INFO:tasks.workunit.client.0.smithi144.stdout:9/43: symlink lf 0 2024-09-10T11:29:29.784 INFO:tasks.workunit.client.0.smithi144.stdout:4/83: symlink d14/l19 0 2024-09-10T11:29:29.785 INFO:tasks.workunit.client.1.smithi178.stdout:3/443: dread db/d20/d3d/f53 [0,4194304] 0 2024-09-10T11:29:29.785 INFO:tasks.workunit.client.1.smithi178.stdout:6/532: mkdir d3/d74/d91/daa/dc2/d57/d78/d7c/dc5 0 2024-09-10T11:29:29.786 INFO:tasks.workunit.client.1.smithi178.stdout:2/668: dwrite d1/d41/d14/f82 [0,4194304] 0 2024-09-10T11:29:29.786 INFO:tasks.workunit.client.0.smithi144.stdout:9/44: write f9 [1011773,122745] 0 2024-09-10T11:29:29.788 INFO:tasks.workunit.client.0.smithi144.stdout:8/69: dwrite d0/d9/f11 [0,4194304] 0 2024-09-10T11:29:29.789 INFO:tasks.workunit.client.1.smithi178.stdout:3/444: rename db/d20 to db/d20/d3d/d63/d82 22 2024-09-10T11:29:29.789 INFO:tasks.workunit.client.1.smithi178.stdout:3/445: stat db/d20/d3d/d56 0 2024-09-10T11:29:29.790 INFO:tasks.workunit.client.0.smithi144.stdout:9/45: creat f10 x:0 0 0 2024-09-10T11:29:29.791 INFO:tasks.workunit.client.0.smithi144.stdout:9/46: readlink le 0 2024-09-10T11:29:29.793 INFO:tasks.workunit.client.0.smithi144.stdout:8/70: rename d0/f8 to d0/da/f19 0 2024-09-10T11:29:29.794 INFO:tasks.workunit.client.1.smithi178.stdout:1/444: mkdir d3/d9/d54/d3a/d50/d6c/d78/da3 0 2024-09-10T11:29:29.794 INFO:tasks.workunit.client.0.smithi144.stdout:8/71: fsync d0/d9/fe 0 2024-09-10T11:29:29.795 INFO:tasks.workunit.client.0.smithi144.stdout:9/47: mkdir d11 0 2024-09-10T11:29:29.796 INFO:tasks.workunit.client.1.smithi178.stdout:7/400: dwrite d6/d10/d1e/d2d/d2b/f39 [0,4194304] 0 2024-09-10T11:29:29.796 INFO:tasks.workunit.client.1.smithi178.stdout:8/543: mkdir d5/db7 0 2024-09-10T11:29:29.796 INFO:tasks.workunit.client.0.smithi144.stdout:4/84: write d14/f18 [8254793,53362] 0 2024-09-10T11:29:29.800 INFO:tasks.workunit.client.1.smithi178.stdout:7/401: dread d6/d10/d1e/d2d/f2c [0,4194304] 0 2024-09-10T11:29:29.801 INFO:tasks.workunit.client.1.smithi178.stdout:2/669: mknod d1/d22/d4d/d78/cd3 0 2024-09-10T11:29:29.806 INFO:tasks.workunit.client.0.smithi144.stdout:4/85: truncate fd 1044437 0 2024-09-10T11:29:29.809 INFO:tasks.workunit.client.1.smithi178.stdout:2/670: symlink d1/d41/d5d/ld4 0 2024-09-10T11:29:29.811 INFO:tasks.workunit.client.0.smithi144.stdout:4/86: dread f1 [0,4194304] 0 2024-09-10T11:29:29.811 INFO:tasks.workunit.client.1.smithi178.stdout:6/533: dread d3/d7/d8/d15/d1c/d20/fb2 [0,4194304] 0 2024-09-10T11:29:29.812 INFO:tasks.workunit.client.0.smithi144.stdout:9/48: truncate f0 1585886 0 2024-09-10T11:29:29.814 INFO:tasks.workunit.client.1.smithi178.stdout:9/556: dwrite d2/d19/f39 [0,4194304] 0 2024-09-10T11:29:29.814 INFO:tasks.workunit.client.1.smithi178.stdout:7/402: mkdir d6/d10/d1d/d22/d93 0 2024-09-10T11:29:29.814 INFO:tasks.workunit.client.0.smithi144.stdout:9/49: mkdir d11/d12 0 2024-09-10T11:29:29.815 INFO:tasks.workunit.client.1.smithi178.stdout:2/671: read d1/d7/d24/d46/d88/d9d/dab/f29 [3356638,9108] 0 2024-09-10T11:29:29.817 INFO:tasks.workunit.client.1.smithi178.stdout:5/514: getdents d9/dc/dd/d37/d42 0 2024-09-10T11:29:29.818 INFO:tasks.workunit.client.0.smithi144.stdout:4/87: dwrite f7 [0,4194304] 0 2024-09-10T11:29:29.818 INFO:tasks.workunit.client.0.smithi144.stdout:4/88: chown f5 95645 1 2024-09-10T11:29:29.819 INFO:tasks.workunit.client.1.smithi178.stdout:1/445: dwrite d3/d9/d1a/d48/f90 [0,4194304] 0 2024-09-10T11:29:29.822 INFO:tasks.workunit.client.1.smithi178.stdout:4/360: getdents d0/d8/df 0 2024-09-10T11:29:29.822 INFO:tasks.workunit.client.0.smithi144.stdout:4/89: mkdir d14/d1a 0 2024-09-10T11:29:29.823 INFO:tasks.workunit.client.0.smithi144.stdout:4/90: stat d14 0 2024-09-10T11:29:29.823 INFO:tasks.workunit.client.1.smithi178.stdout:3/446: getdents db/d20/d3e 0 2024-09-10T11:29:29.823 INFO:tasks.workunit.client.0.smithi144.stdout:4/91: stat c6 0 2024-09-10T11:29:29.824 INFO:tasks.workunit.client.0.smithi144.stdout:4/92: write f9 [299266,45049] 0 2024-09-10T11:29:29.825 INFO:tasks.workunit.client.1.smithi178.stdout:5/515: creat d9/dc/dd/d16/fb3 x:0 0 0 2024-09-10T11:29:29.827 INFO:tasks.workunit.client.1.smithi178.stdout:3/447: dread db/d35/f77 [0,4194304] 0 2024-09-10T11:29:29.827 INFO:tasks.workunit.client.0.smithi144.stdout:4/93: creat d14/f1b x:0 0 0 2024-09-10T11:29:29.827 INFO:tasks.workunit.client.0.smithi144.stdout:4/94: readlink d14/l17 0 2024-09-10T11:29:29.828 INFO:tasks.workunit.client.1.smithi178.stdout:7/403: write d6/d10/d1e/d2d/f5f [9223853,92489] 0 2024-09-10T11:29:29.831 INFO:tasks.workunit.client.0.smithi144.stdout:4/95: creat d14/d1a/f1c x:0 0 0 2024-09-10T11:29:29.831 INFO:tasks.workunit.client.1.smithi178.stdout:9/557: creat d2/d13/d75/d60/d66/fa7 x:0 0 0 2024-09-10T11:29:29.833 INFO:tasks.workunit.client.1.smithi178.stdout:3/448: mknod db/d35/c83 0 2024-09-10T11:29:29.835 INFO:tasks.workunit.client.1.smithi178.stdout:4/361: symlink d0/d9/dc/d50/l7e 0 2024-09-10T11:29:29.835 INFO:tasks.workunit.client.0.smithi144.stdout:4/96: link f7 d14/f1d 0 2024-09-10T11:29:29.841 INFO:tasks.workunit.client.0.smithi144.stdout:4/97: rename d14/d1a to d14/d1e 0 2024-09-10T11:29:29.842 INFO:tasks.workunit.client.1.smithi178.stdout:0/526: write d0/dc/d1f/f74 [378911,115599] 0 2024-09-10T11:29:29.843 INFO:tasks.workunit.client.1.smithi178.stdout:7/404: rename d6/d10/d1d/d22/c5b to d6/d10/d34/d6a/c94 0 2024-09-10T11:29:29.847 INFO:tasks.workunit.client.1.smithi178.stdout:3/449: rename db/f18 to db/d20/d3d/d63/d6a/f84 0 2024-09-10T11:29:29.848 INFO:tasks.workunit.client.1.smithi178.stdout:0/527: dwrite d0/d12/d3f/f44 [4194304,4194304] 0 2024-09-10T11:29:29.848 INFO:tasks.workunit.client.0.smithi144.stdout:6/79: sync 2024-09-10T11:29:29.849 INFO:tasks.workunit.client.0.smithi144.stdout:3/77: sync 2024-09-10T11:29:29.849 INFO:tasks.workunit.client.1.smithi178.stdout:7/405: mknod d6/d10/d1d/d22/d50/d7e/d82/c95 0 2024-09-10T11:29:29.851 INFO:tasks.workunit.client.1.smithi178.stdout:4/362: rename d0/d8/d3e/c64 to d0/d9/d3d/c7f 0 2024-09-10T11:29:29.851 INFO:tasks.workunit.client.0.smithi144.stdout:3/78: rmdir d4/d1a 39 2024-09-10T11:29:29.853 INFO:tasks.workunit.client.0.smithi144.stdout:3/79: dread f2 [0,4194304] 0 2024-09-10T11:29:29.853 INFO:tasks.workunit.client.0.smithi144.stdout:6/80: mknod d9/dc/d13/c22 0 2024-09-10T11:29:29.853 INFO:tasks.workunit.client.1.smithi178.stdout:0/528: fsync d0/dc/f61 0 2024-09-10T11:29:29.854 INFO:tasks.workunit.client.1.smithi178.stdout:3/450: dwrite db/d26/d40/f62 [0,4194304] 0 2024-09-10T11:29:29.854 INFO:tasks.workunit.client.1.smithi178.stdout:0/529: readlink d0/dc/d39/d50/la7 0 2024-09-10T11:29:29.856 INFO:tasks.workunit.client.1.smithi178.stdout:7/406: stat d6/d10/d34/d6a/d6f/l88 0 2024-09-10T11:29:29.858 INFO:tasks.workunit.client.0.smithi144.stdout:6/81: creat d9/dc/f23 x:0 0 0 2024-09-10T11:29:29.859 INFO:tasks.workunit.client.1.smithi178.stdout:7/407: read - d6/d10/d1e/d2d/f66 zero size 2024-09-10T11:29:29.859 INFO:tasks.workunit.client.1.smithi178.stdout:4/363: unlink d0/l7 0 2024-09-10T11:29:29.860 INFO:tasks.workunit.client.0.smithi144.stdout:6/82: mknod d9/c24 0 2024-09-10T11:29:29.861 INFO:tasks.workunit.client.1.smithi178.stdout:0/530: mkdir d0/dc/d1f/d2d/dac 0 2024-09-10T11:29:29.862 INFO:tasks.workunit.client.1.smithi178.stdout:7/408: chown d6/d10/d1d/d22/d50/d7e/d82/c95 113019 1 2024-09-10T11:29:29.862 INFO:tasks.workunit.client.0.smithi144.stdout:3/80: truncate d4/da/fc 5136889 0 2024-09-10T11:29:29.863 INFO:tasks.workunit.client.0.smithi144.stdout:6/83: truncate d9/dc/d13/f20 900238 0 2024-09-10T11:29:29.863 INFO:tasks.workunit.client.1.smithi178.stdout:3/451: creat db/d20/d3d/f85 x:0 0 0 2024-09-10T11:29:29.863 INFO:tasks.workunit.client.1.smithi178.stdout:6/534: dwrite d3/d7/d8/f58 [0,4194304] 0 2024-09-10T11:29:29.864 INFO:tasks.workunit.client.0.smithi144.stdout:6/84: truncate d9/dd/d11/f21 589124 0 2024-09-10T11:29:29.865 INFO:tasks.workunit.client.1.smithi178.stdout:3/452: readlink db/d20/d3e/l57 0 2024-09-10T11:29:29.865 INFO:tasks.workunit.client.0.smithi144.stdout:6/85: fdatasync d9/dc/d13/f20 0 2024-09-10T11:29:29.866 INFO:tasks.workunit.client.1.smithi178.stdout:4/364: unlink d0/d10/f32 0 2024-09-10T11:29:29.866 INFO:tasks.workunit.client.1.smithi178.stdout:5/516: write d9/dc/dd/d37/d3e/f78 [1480413,46691] 0 2024-09-10T11:29:29.873 INFO:tasks.workunit.client.0.smithi144.stdout:3/81: dwrite d4/da/d17/f1f [0,4194304] 0 2024-09-10T11:29:29.874 INFO:tasks.workunit.client.1.smithi178.stdout:6/535: symlink d3/d7/d8/d15/d1c/d20/d4b/lc6 0 2024-09-10T11:29:29.875 INFO:tasks.workunit.client.1.smithi178.stdout:2/672: truncate d1/d41/d5d/fa9 810366 0 2024-09-10T11:29:29.877 INFO:tasks.workunit.client.1.smithi178.stdout:7/409: truncate d6/d10/d1e/d2d/f2c 1434020 0 2024-09-10T11:29:29.878 INFO:tasks.workunit.client.0.smithi144.stdout:6/86: dwrite d9/dc/f23 [0,4194304] 0 2024-09-10T11:29:29.878 INFO:tasks.workunit.client.1.smithi178.stdout:9/558: truncate d2/f26 777237 0 2024-09-10T11:29:29.881 INFO:tasks.workunit.client.1.smithi178.stdout:3/453: mknod db/d36/c86 0 2024-09-10T11:29:29.882 INFO:tasks.workunit.client.1.smithi178.stdout:6/536: dwrite d3/d74/d91/daa/dc2/f61 [0,4194304] 0 2024-09-10T11:29:29.884 INFO:tasks.workunit.client.1.smithi178.stdout:6/537: chown d3/d7/d8/d15/d1c/d20/fb2 240 1 2024-09-10T11:29:29.884 INFO:tasks.workunit.client.0.smithi144.stdout:3/82: chown d4/da/fc 3 1 2024-09-10T11:29:29.886 INFO:tasks.workunit.client.0.smithi144.stdout:6/87: rename d9/dc/f23 to d9/dc/f25 0 2024-09-10T11:29:29.886 INFO:tasks.workunit.client.0.smithi144.stdout:6/88: chown d9/dc/d13/c15 3817400 1 2024-09-10T11:29:29.890 INFO:tasks.workunit.client.0.smithi144.stdout:6/89: rename f6 to d9/d1f/f26 0 2024-09-10T11:29:29.891 INFO:tasks.workunit.client.0.smithi144.stdout:6/90: write f8 [648387,99462] 0 2024-09-10T11:29:29.892 INFO:tasks.workunit.client.0.smithi144.stdout:6/91: write d9/dd/f1d [969463,52171] 0 2024-09-10T11:29:29.893 INFO:tasks.workunit.client.0.smithi144.stdout:6/92: chown d9/dc/f16 1 1 2024-09-10T11:29:29.894 INFO:tasks.workunit.client.0.smithi144.stdout:6/93: readlink d9/lb 0 2024-09-10T11:29:29.894 INFO:tasks.workunit.client.0.smithi144.stdout:6/94: readlink l2 0 2024-09-10T11:29:29.896 INFO:tasks.workunit.client.0.smithi144.stdout:6/95: creat d9/dd/d11/f27 x:0 0 0 2024-09-10T11:29:29.902 INFO:tasks.workunit.client.0.smithi144.stdout:6/96: dwrite d9/dd/d11/f18 [0,4194304] 0 2024-09-10T11:29:29.905 INFO:tasks.workunit.client.0.smithi144.stdout:6/97: creat d9/dc/f28 x:0 0 0 2024-09-10T11:29:29.909 INFO:tasks.workunit.client.0.smithi144.stdout:6/98: getdents d9/dc/d13 0 2024-09-10T11:29:29.910 INFO:tasks.workunit.client.0.smithi144.stdout:6/99: mknod d9/d1f/c29 0 2024-09-10T11:29:29.911 INFO:tasks.workunit.client.0.smithi144.stdout:6/100: fsync d9/dd/d11/f1c 0 2024-09-10T11:29:29.912 INFO:tasks.workunit.client.0.smithi144.stdout:6/101: creat d9/f2a x:0 0 0 2024-09-10T11:29:29.913 INFO:tasks.workunit.client.0.smithi144.stdout:6/102: truncate d9/dc/d13/f1e 77938 0 2024-09-10T11:29:29.914 INFO:tasks.workunit.client.0.smithi144.stdout:6/103: fsync d9/dd/d11/f21 0 2024-09-10T11:29:29.919 INFO:tasks.workunit.client.0.smithi144.stdout:6/104: truncate d9/dc/d13/f1e 54636 0 2024-09-10T11:29:29.920 INFO:tasks.workunit.client.0.smithi144.stdout:6/105: mknod d9/dd/c2b 0 2024-09-10T11:29:29.921 INFO:tasks.workunit.client.0.smithi144.stdout:6/106: mkdir d9/d1f/d2c 0 2024-09-10T11:29:29.924 INFO:tasks.workunit.client.0.smithi144.stdout:6/107: write d9/dc/f25 [3607452,98079] 0 2024-09-10T11:29:29.924 INFO:tasks.workunit.client.0.smithi144.stdout:6/108: fsync d9/dc/d13/f20 0 2024-09-10T11:29:29.926 INFO:tasks.workunit.client.0.smithi144.stdout:6/109: creat d9/dd/d11/f2d x:0 0 0 2024-09-10T11:29:29.927 INFO:tasks.workunit.client.0.smithi144.stdout:6/110: write f8 [1916836,125580] 0 2024-09-10T11:29:29.928 INFO:tasks.workunit.client.0.smithi144.stdout:6/111: dread - d9/dc/f28 zero size 2024-09-10T11:29:29.941 INFO:tasks.workunit.client.1.smithi178.stdout:1/446: sync 2024-09-10T11:29:29.941 INFO:tasks.workunit.client.1.smithi178.stdout:8/544: sync 2024-09-10T11:29:29.941 INFO:tasks.workunit.client.1.smithi178.stdout:8/545: chown d5/db/d10/d20/d52/d67 171 1 2024-09-10T11:29:29.944 INFO:tasks.workunit.client.1.smithi178.stdout:4/365: creat d0/d8/da/d1a/d51/d5d/f80 x:0 0 0 2024-09-10T11:29:29.949 INFO:tasks.workunit.client.1.smithi178.stdout:2/673: rename d1/d7/l1c to d1/d7/d24/d46/d88/d9d/dab/d5b/ld5 0 2024-09-10T11:29:29.949 INFO:tasks.workunit.client.1.smithi178.stdout:7/410: rename d6/d10/d1d/d22 to d6/d10/d1d/d22/d50/d7e/d82/d96 22 2024-09-10T11:29:29.949 INFO:tasks.workunit.client.1.smithi178.stdout:6/538: fsync d3/d7/d8/d15/d1c/d2a/fa2 0 2024-09-10T11:29:29.950 INFO:tasks.workunit.client.1.smithi178.stdout:1/447: creat d3/d35/fa4 x:0 0 0 2024-09-10T11:29:29.951 INFO:tasks.workunit.client.1.smithi178.stdout:8/546: mkdir d5/d30/db8 0 2024-09-10T11:29:29.952 INFO:tasks.workunit.client.1.smithi178.stdout:0/531: creat d0/dc/d1f/fad x:0 0 0 2024-09-10T11:29:29.953 INFO:tasks.workunit.client.1.smithi178.stdout:5/517: getdents d9/dc/dd/d37/d42/d55/d6d 0 2024-09-10T11:29:29.955 INFO:tasks.workunit.client.1.smithi178.stdout:3/454: rename db/d35/c83 to db/d20/d3d/c87 0 2024-09-10T11:29:29.956 INFO:tasks.workunit.client.1.smithi178.stdout:2/674: mkdir d1/d7/d24/d7d/d84/d55/da1/dd6 0 2024-09-10T11:29:29.956 INFO:tasks.workunit.client.1.smithi178.stdout:3/455: chown db/d35/d58/c7c 7 1 2024-09-10T11:29:29.957 INFO:tasks.workunit.client.1.smithi178.stdout:7/411: mknod d6/d10/d1d/d22/d50/c97 0 2024-09-10T11:29:29.957 INFO:tasks.workunit.client.1.smithi178.stdout:0/532: dread d0/d12/f34 [0,4194304] 0 2024-09-10T11:29:29.957 INFO:tasks.workunit.client.1.smithi178.stdout:1/448: mknod d3/d9/d1a/d48/ca5 0 2024-09-10T11:29:29.959 INFO:tasks.workunit.client.1.smithi178.stdout:1/449: chown d3/d9/d54/d2a/f34 91740 1 2024-09-10T11:29:29.960 INFO:tasks.workunit.client.1.smithi178.stdout:5/518: mknod d9/dc/dd/d3f/d5a/d7d/cb4 0 2024-09-10T11:29:29.962 INFO:tasks.workunit.client.1.smithi178.stdout:6/539: dread d3/d7/d8/d15/d1c/d2a/f2c [0,4194304] 0 2024-09-10T11:29:29.963 INFO:tasks.workunit.client.1.smithi178.stdout:3/456: dwrite db/d26/f3b [0,4194304] 0 2024-09-10T11:29:29.963 INFO:tasks.workunit.client.1.smithi178.stdout:4/366: link d0/d8/da/f27 d0/d9/f81 0 2024-09-10T11:29:29.964 INFO:tasks.workunit.client.1.smithi178.stdout:4/367: dread - d0/d63/f76 zero size 2024-09-10T11:29:29.966 INFO:tasks.workunit.client.1.smithi178.stdout:7/412: mkdir d6/d10/d1d/d22/d50/d98 0 2024-09-10T11:29:29.967 INFO:tasks.workunit.client.1.smithi178.stdout:7/413: chown d6/c69 216404 1 2024-09-10T11:29:29.969 INFO:tasks.workunit.client.1.smithi178.stdout:9/559: getdents d2/d19/d1c 0 2024-09-10T11:29:29.974 INFO:tasks.workunit.client.1.smithi178.stdout:0/533: mkdir d0/dc/d1f/dae 0 2024-09-10T11:29:29.975 INFO:tasks.workunit.client.1.smithi178.stdout:0/534: fdatasync d0/dc/d39/d50/f9f 0 2024-09-10T11:29:29.976 INFO:tasks.workunit.client.1.smithi178.stdout:8/547: creat d5/db/d10/d20/fb9 x:0 0 0 2024-09-10T11:29:29.980 INFO:tasks.workunit.client.1.smithi178.stdout:9/560: creat d2/d13/d40/d89/d93/d88/fa8 x:0 0 0 2024-09-10T11:29:29.980 INFO:tasks.workunit.client.1.smithi178.stdout:9/561: chown d2/df/f28 1 1 2024-09-10T11:29:29.981 INFO:tasks.workunit.client.1.smithi178.stdout:1/450: truncate d3/d9/d21/d4b/f17 1939507 0 2024-09-10T11:29:29.982 INFO:tasks.workunit.client.1.smithi178.stdout:5/519: symlink d9/lb5 0 2024-09-10T11:29:29.982 INFO:tasks.workunit.client.1.smithi178.stdout:5/520: chown d9/f32 3101 1 2024-09-10T11:29:29.985 INFO:tasks.workunit.client.1.smithi178.stdout:4/368: mknod d0/d8/da/d1a/d51/d5d/d62/c82 0 2024-09-10T11:29:29.987 INFO:tasks.workunit.client.1.smithi178.stdout:9/562: creat d2/d19/d1c/d2a/fa9 x:0 0 0 2024-09-10T11:29:29.988 INFO:tasks.workunit.client.1.smithi178.stdout:2/675: getdents d1/d41/d5d 0 2024-09-10T11:29:29.989 INFO:tasks.workunit.client.1.smithi178.stdout:5/521: creat d9/d19/fb6 x:0 0 0 2024-09-10T11:29:29.990 INFO:tasks.workunit.client.1.smithi178.stdout:0/535: mknod d0/dc/d39/da4/caf 0 2024-09-10T11:29:29.992 INFO:tasks.workunit.client.1.smithi178.stdout:4/369: chown d0/d9/ce 6160 1 2024-09-10T11:29:29.992 INFO:tasks.workunit.client.1.smithi178.stdout:7/414: link d6/d10/d61/f77 d6/d10/d1d/d22/d93/f99 0 2024-09-10T11:29:29.993 INFO:tasks.workunit.client.1.smithi178.stdout:9/563: dwrite d2/d19/f39 [4194304,4194304] 0 2024-09-10T11:29:29.994 INFO:tasks.workunit.client.1.smithi178.stdout:7/415: truncate d6/d10/d34/f3f 695491 0 2024-09-10T11:29:29.995 INFO:tasks.workunit.client.1.smithi178.stdout:9/564: fsync d2/d13/d40/d89/d93/d88/fa8 0 2024-09-10T11:29:29.995 INFO:tasks.workunit.client.1.smithi178.stdout:9/565: chown d2/d19/d1c/f2c 6 1 2024-09-10T11:29:29.997 INFO:tasks.workunit.client.1.smithi178.stdout:1/451: rename d3/d9/d21/d4b/d2b/f57 to d3/d9/d1a/d48/fa6 0 2024-09-10T11:29:29.998 INFO:tasks.workunit.client.1.smithi178.stdout:9/566: dread d2/f3f [0,4194304] 0 2024-09-10T11:29:29.999 INFO:tasks.workunit.client.1.smithi178.stdout:2/676: unlink d1/d7/d24/d46/d88/d9d/dab/d5e/lad 0 2024-09-10T11:29:30.000 INFO:tasks.workunit.client.1.smithi178.stdout:3/457: getdents db/d20/d3d/d56 0 2024-09-10T11:29:30.002 INFO:tasks.workunit.client.1.smithi178.stdout:3/458: stat db/d20/d3d/f85 0 2024-09-10T11:29:30.003 INFO:tasks.workunit.client.1.smithi178.stdout:3/459: stat db/d20/d3e/d6b/d7e 0 2024-09-10T11:29:30.003 INFO:tasks.workunit.client.1.smithi178.stdout:1/452: creat d3/d9/d54/d3a/d63/d7e/fa7 x:0 0 0 2024-09-10T11:29:30.004 INFO:tasks.workunit.client.1.smithi178.stdout:7/416: mknod d6/d10/d1e/c9a 0 2024-09-10T11:29:30.007 INFO:tasks.workunit.client.1.smithi178.stdout:7/417: dread - d6/d10/d61/f84 zero size 2024-09-10T11:29:30.008 INFO:tasks.workunit.client.0.smithi144.stdout:2/76: sync 2024-09-10T11:29:30.008 INFO:tasks.workunit.client.0.smithi144.stdout:1/92: sync 2024-09-10T11:29:30.008 INFO:tasks.workunit.client.0.smithi144.stdout:5/76: sync 2024-09-10T11:29:30.008 INFO:tasks.workunit.client.0.smithi144.stdout:7/65: sync 2024-09-10T11:29:30.008 INFO:tasks.workunit.client.0.smithi144.stdout:0/73: sync 2024-09-10T11:29:30.010 INFO:tasks.workunit.client.1.smithi178.stdout:5/522: rename d9/dc/dd/d16/f1c to d9/dc/dd/fb7 0 2024-09-10T11:29:30.011 INFO:tasks.workunit.client.1.smithi178.stdout:2/677: dwrite d1/d7/d24/d43/f86 [0,4194304] 0 2024-09-10T11:29:30.012 INFO:tasks.workunit.client.1.smithi178.stdout:1/453: mknod d3/d9/d54/d3a/d50/d9e/ca8 0 2024-09-10T11:29:30.013 INFO:tasks.workunit.client.0.smithi144.stdout:2/77: write d7/f8 [255641,86818] 0 2024-09-10T11:29:30.013 INFO:tasks.workunit.client.1.smithi178.stdout:7/418: dread - d6/d10/d1d/d4a/f57 zero size 2024-09-10T11:29:30.013 INFO:tasks.workunit.client.0.smithi144.stdout:1/93: creat d5/dd/f1d x:0 0 0 2024-09-10T11:29:30.014 INFO:tasks.workunit.client.0.smithi144.stdout:7/66: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:30.015 INFO:tasks.workunit.client.0.smithi144.stdout:5/77: readlink d4/l12 0 2024-09-10T11:29:30.015 INFO:tasks.workunit.client.0.smithi144.stdout:1/94: chown f4 1315507 1 2024-09-10T11:29:30.016 INFO:tasks.workunit.client.0.smithi144.stdout:1/95: fsync d5/df/f16 0 2024-09-10T11:29:30.016 INFO:tasks.workunit.client.0.smithi144.stdout:7/67: write d5/f7 [1044617,31080] 0 2024-09-10T11:29:30.019 INFO:tasks.workunit.client.0.smithi144.stdout:2/78: creat d7/de/f20 x:0 0 0 2024-09-10T11:29:30.020 INFO:tasks.workunit.client.0.smithi144.stdout:5/78: dread - d4/f15 zero size 2024-09-10T11:29:30.022 INFO:tasks.workunit.client.1.smithi178.stdout:3/460: dwrite db/d26/f2c [0,4194304] 0 2024-09-10T11:29:30.023 INFO:tasks.workunit.client.0.smithi144.stdout:1/96: rename f4 to d5/df/d1a/f1e 0 2024-09-10T11:29:30.023 INFO:tasks.workunit.client.1.smithi178.stdout:9/567: dread d2/d13/f43 [0,4194304] 0 2024-09-10T11:29:30.023 INFO:tasks.workunit.client.1.smithi178.stdout:1/454: chown d3/d9/fa 23407941 1 2024-09-10T11:29:30.025 INFO:tasks.workunit.client.0.smithi144.stdout:7/68: dwrite f1 [0,4194304] 0 2024-09-10T11:29:30.026 INFO:tasks.workunit.client.1.smithi178.stdout:4/370: rename d0/d8/d3e/l7d to d0/d9/dc/l83 0 2024-09-10T11:29:30.026 INFO:tasks.workunit.client.0.smithi144.stdout:2/79: unlink d7/da/lc 0 2024-09-10T11:29:30.027 INFO:tasks.workunit.client.0.smithi144.stdout:1/97: rmdir d5/df 39 2024-09-10T11:29:30.029 INFO:tasks.workunit.client.0.smithi144.stdout:7/69: mknod d5/dc/d11/d18/c1a 0 2024-09-10T11:29:30.030 INFO:tasks.workunit.client.0.smithi144.stdout:7/70: fdatasync d5/dc/fe 0 2024-09-10T11:29:30.032 INFO:tasks.workunit.client.1.smithi178.stdout:5/523: dwrite d9/dc/f2b [0,4194304] 0 2024-09-10T11:29:30.034 INFO:tasks.workunit.client.1.smithi178.stdout:2/678: mkdir d1/d7/d24/d46/d88/d9d/dab/d5b/d60/dd7 0 2024-09-10T11:29:30.035 INFO:tasks.workunit.client.1.smithi178.stdout:4/371: write d0/d8/df/f3b [4980,27063] 0 2024-09-10T11:29:30.042 INFO:tasks.workunit.client.1.smithi178.stdout:3/461: dwrite db/f4f [0,4194304] 0 2024-09-10T11:29:30.044 INFO:tasks.workunit.client.1.smithi178.stdout:4/372: stat d0/d8/da/f27 0 2024-09-10T11:29:30.044 INFO:tasks.workunit.client.1.smithi178.stdout:1/455: rename d3/d9/d1a/f1f to d3/d35/fa9 0 2024-09-10T11:29:30.044 INFO:tasks.workunit.client.1.smithi178.stdout:3/462: fsync db/d35/d3a/f4b 0 2024-09-10T11:29:30.045 INFO:tasks.workunit.client.0.smithi144.stdout:7/71: mknod d5/c1b 0 2024-09-10T11:29:30.046 INFO:tasks.workunit.client.1.smithi178.stdout:1/456: stat d3/d9/d54/d3a/d6e/d8a 0 2024-09-10T11:29:30.046 INFO:tasks.workunit.client.1.smithi178.stdout:3/463: readlink l9 0 2024-09-10T11:29:30.048 INFO:tasks.workunit.client.1.smithi178.stdout:2/679: read d1/d41/d14/d11/d53/d97/f99 [157461,1398] 0 2024-09-10T11:29:30.048 INFO:tasks.workunit.client.1.smithi178.stdout:0/536: dread d0/dc/f21 [0,4194304] 0 2024-09-10T11:29:30.048 INFO:tasks.workunit.client.1.smithi178.stdout:5/524: mkdir d9/d19/d91/db8 0 2024-09-10T11:29:30.048 INFO:tasks.workunit.client.0.smithi144.stdout:7/72: chown d5/dc/d11/c15 26851800 1 2024-09-10T11:29:30.049 INFO:tasks.workunit.client.0.smithi144.stdout:1/98: dread d5/fe [0,4194304] 0 2024-09-10T11:29:30.051 INFO:tasks.workunit.client.0.smithi144.stdout:1/99: write d5/f12 [784236,45559] 0 2024-09-10T11:29:30.052 INFO:tasks.workunit.client.1.smithi178.stdout:4/373: fdatasync d0/d9/f81 0 2024-09-10T11:29:30.054 INFO:tasks.workunit.client.1.smithi178.stdout:5/525: symlink d9/d19/lb9 0 2024-09-10T11:29:30.054 INFO:tasks.workunit.client.1.smithi178.stdout:2/680: chown d1/d22/d4d/d78/laf 34 1 2024-09-10T11:29:30.055 INFO:tasks.workunit.client.0.smithi144.stdout:7/73: mknod d5/dc/d12/c1c 0 2024-09-10T11:29:30.056 INFO:tasks.workunit.client.0.smithi144.stdout:1/100: fsync d5/fb 0 2024-09-10T11:29:30.057 INFO:tasks.workunit.client.1.smithi178.stdout:1/457: symlink d3/d9/d54/d3a/d50/d80/laa 0 2024-09-10T11:29:30.059 INFO:tasks.workunit.client.1.smithi178.stdout:7/419: dwrite d6/d10/d1e/f35 [0,4194304] 0 2024-09-10T11:29:30.059 INFO:tasks.workunit.client.1.smithi178.stdout:5/526: fdatasync d9/dc/dd/f69 0 2024-09-10T11:29:30.059 INFO:tasks.workunit.client.0.smithi144.stdout:7/74: mknod d5/dc/d12/c1d 0 2024-09-10T11:29:30.063 INFO:tasks.workunit.client.0.smithi144.stdout:5/79: truncate d4/fa 594521 0 2024-09-10T11:29:30.065 INFO:tasks.workunit.client.0.smithi144.stdout:1/101: creat d5/df/f1f x:0 0 0 2024-09-10T11:29:30.065 INFO:tasks.workunit.client.1.smithi178.stdout:4/374: truncate d0/d8/da/f3f 763229 0 2024-09-10T11:29:30.065 INFO:tasks.workunit.client.0.smithi144.stdout:2/80: truncate d7/da/f18 97376 0 2024-09-10T11:29:30.069 INFO:tasks.workunit.client.1.smithi178.stdout:7/420: creat d6/d10/d34/f9b x:0 0 0 2024-09-10T11:29:30.069 INFO:tasks.workunit.client.0.smithi144.stdout:1/102: chown d5/df/f19 95 1 2024-09-10T11:29:30.070 INFO:tasks.workunit.client.1.smithi178.stdout:1/458: mknod d3/d9/d54/d3a/d50/d6c/cab 0 2024-09-10T11:29:30.071 INFO:tasks.workunit.client.1.smithi178.stdout:0/537: dwrite d0/dc/d1f/f26 [4194304,4194304] 0 2024-09-10T11:29:30.071 INFO:tasks.workunit.client.0.smithi144.stdout:1/103: unlink d5/df/f1f 0 2024-09-10T11:29:30.072 INFO:tasks.workunit.client.1.smithi178.stdout:9/568: dread d2/d13/f15 [0,4194304] 0 2024-09-10T11:29:30.073 INFO:tasks.workunit.client.0.smithi144.stdout:5/80: mkdir d4/d5/d18 0 2024-09-10T11:29:30.073 INFO:tasks.workunit.client.1.smithi178.stdout:7/421: unlink d6/d10/l1b 0 2024-09-10T11:29:30.074 INFO:tasks.workunit.client.0.smithi144.stdout:7/75: getdents d5/dc/d11 0 2024-09-10T11:29:30.075 INFO:tasks.workunit.client.0.smithi144.stdout:2/81: dwrite f6 [0,4194304] 0 2024-09-10T11:29:30.075 INFO:tasks.workunit.client.1.smithi178.stdout:4/375: mknod d0/d9/d3d/d4f/c84 0 2024-09-10T11:29:30.076 INFO:tasks.workunit.client.0.smithi144.stdout:7/76: truncate d5/fb 1744362 0 2024-09-10T11:29:30.077 INFO:tasks.workunit.client.0.smithi144.stdout:2/82: write d7/f1d [438229,101910] 0 2024-09-10T11:29:30.077 INFO:tasks.workunit.client.1.smithi178.stdout:3/464: dread f8 [0,4194304] 0 2024-09-10T11:29:30.078 INFO:tasks.workunit.client.1.smithi178.stdout:0/538: readlink d0/dc/d39/d50/l3c 0 2024-09-10T11:29:30.080 INFO:tasks.workunit.client.0.smithi144.stdout:7/77: truncate d5/dc/f10 276901 0 2024-09-10T11:29:30.081 INFO:tasks.workunit.client.0.smithi144.stdout:2/83: mkdir d7/da/d1e/d21 0 2024-09-10T11:29:30.081 INFO:tasks.workunit.client.1.smithi178.stdout:4/376: chown d0/d9/c2a 1 1 2024-09-10T11:29:30.081 INFO:tasks.workunit.client.1.smithi178.stdout:2/681: dwrite d1/d22/f26 [4194304,4194304] 0 2024-09-10T11:29:30.081 INFO:tasks.workunit.client.1.smithi178.stdout:9/569: mknod d2/d13/d75/caa 0 2024-09-10T11:29:30.085 INFO:tasks.workunit.client.1.smithi178.stdout:5/527: dwrite d9/dc/dd/f69 [0,4194304] 0 2024-09-10T11:29:30.085 INFO:tasks.workunit.client.1.smithi178.stdout:9/570: chown d2/d19/d1c/c4d 149 1 2024-09-10T11:29:30.085 INFO:tasks.workunit.client.1.smithi178.stdout:0/539: dread - d0/d12/d3f/d72/f82 zero size 2024-09-10T11:29:30.090 INFO:tasks.workunit.client.0.smithi144.stdout:5/81: write d4/fa [427641,60377] 0 2024-09-10T11:29:30.092 INFO:tasks.workunit.client.0.smithi144.stdout:1/104: dwrite d5/f18 [0,4194304] 0 2024-09-10T11:29:30.092 INFO:tasks.workunit.client.0.smithi144.stdout:2/84: creat d7/de/f22 x:0 0 0 2024-09-10T11:29:30.093 INFO:tasks.workunit.client.1.smithi178.stdout:2/682: symlink d1/d7/d24/d43/ld8 0 2024-09-10T11:29:30.095 INFO:tasks.workunit.client.0.smithi144.stdout:2/85: dread - d7/da/f17 zero size 2024-09-10T11:29:30.095 INFO:tasks.workunit.client.0.smithi144.stdout:7/78: dwrite d5/fb [0,4194304] 0 2024-09-10T11:29:30.097 INFO:tasks.workunit.client.0.smithi144.stdout:7/79: readlink d5/dc/ld 0 2024-09-10T11:29:30.097 INFO:tasks.workunit.client.0.smithi144.stdout:2/86: chown d7/da/d1e/d21 7112902 1 2024-09-10T11:29:30.098 INFO:tasks.workunit.client.0.smithi144.stdout:5/82: symlink d4/d5/l19 0 2024-09-10T11:29:30.098 INFO:tasks.workunit.client.0.smithi144.stdout:2/87: chown d7/f1d 0 1 2024-09-10T11:29:30.100 INFO:tasks.workunit.client.0.smithi144.stdout:7/80: chown d5/dc/d11/d18/l19 9 1 2024-09-10T11:29:30.101 INFO:tasks.workunit.client.1.smithi178.stdout:7/422: dwrite d6/d10/d1e/d2d/d2b/f41 [0,4194304] 0 2024-09-10T11:29:30.101 INFO:tasks.workunit.client.0.smithi144.stdout:2/88: dread - d7/de/f1b zero size 2024-09-10T11:29:30.102 INFO:tasks.workunit.client.0.smithi144.stdout:1/105: read - d5/dd/f13 zero size 2024-09-10T11:29:30.108 INFO:tasks.workunit.client.0.smithi144.stdout:7/81: creat d5/dc/d11/d18/f1e x:0 0 0 2024-09-10T11:29:30.108 INFO:tasks.workunit.client.0.smithi144.stdout:2/89: write d7/da/fb [2488290,3526] 0 2024-09-10T11:29:30.109 INFO:tasks.workunit.client.1.smithi178.stdout:4/377: dwrite d0/d8/d3e/f4e [4194304,4194304] 0 2024-09-10T11:29:30.110 INFO:tasks.workunit.client.0.smithi144.stdout:7/82: stat d5/dc/d11/f17 0 2024-09-10T11:29:30.114 INFO:tasks.workunit.client.1.smithi178.stdout:7/423: dread d6/d10/d1e/d2d/f16 [0,4194304] 0 2024-09-10T11:29:30.115 INFO:tasks.workunit.client.0.smithi144.stdout:5/83: dwrite d4/fa [0,4194304] 0 2024-09-10T11:29:30.119 INFO:tasks.workunit.client.0.smithi144.stdout:5/84: write d4/fa [1328418,105543] 0 2024-09-10T11:29:30.124 INFO:tasks.workunit.client.0.smithi144.stdout:1/106: dwrite d5/f12 [0,4194304] 0 2024-09-10T11:29:30.127 INFO:tasks.workunit.client.1.smithi178.stdout:0/540: dread d0/dc/d39/d4c/f15 [0,4194304] 0 2024-09-10T11:29:30.130 INFO:tasks.workunit.client.0.smithi144.stdout:5/85: symlink d4/l1a 0 2024-09-10T11:29:30.132 INFO:tasks.workunit.client.0.smithi144.stdout:1/107: rename d5/df/d1a to d5/dd/d20 0 2024-09-10T11:29:30.135 INFO:tasks.workunit.client.0.smithi144.stdout:1/108: write d5/f18 [3050334,12571] 0 2024-09-10T11:29:30.136 INFO:tasks.workunit.client.1.smithi178.stdout:0/541: dwrite d0/dc/d39/d4c/f73 [8388608,4194304] 0 2024-09-10T11:29:30.136 INFO:tasks.workunit.client.0.smithi144.stdout:1/109: dread - d5/f1b zero size 2024-09-10T11:29:30.137 INFO:tasks.workunit.client.0.smithi144.stdout:5/86: creat d4/d5/d18/f1b x:0 0 0 2024-09-10T11:29:30.139 INFO:tasks.workunit.client.0.smithi144.stdout:7/83: dwrite d5/f8 [0,4194304] 0 2024-09-10T11:29:30.142 INFO:tasks.workunit.client.1.smithi178.stdout:0/542: chown d0/dc/d39/f68 996 1 2024-09-10T11:29:30.142 INFO:tasks.workunit.client.0.smithi144.stdout:5/87: symlink d4/d5/l1c 0 2024-09-10T11:29:30.143 INFO:tasks.workunit.client.1.smithi178.stdout:2/683: dread d1/d7/d24/d7d/d84/d55/f5c [0,4194304] 0 2024-09-10T11:29:30.145 INFO:tasks.workunit.client.0.smithi144.stdout:7/84: creat d5/dc/d12/f1f x:0 0 0 2024-09-10T11:29:30.146 INFO:tasks.workunit.client.1.smithi178.stdout:3/465: dread db/d35/f5b [0,4194304] 0 2024-09-10T11:29:30.147 INFO:tasks.workunit.client.0.smithi144.stdout:1/110: truncate d5/fc 3464383 0 2024-09-10T11:29:30.153 INFO:tasks.workunit.client.0.smithi144.stdout:5/88: dwrite d4/fa [0,4194304] 0 2024-09-10T11:29:30.156 INFO:tasks.workunit.client.0.smithi144.stdout:1/111: unlink d5/la 0 2024-09-10T11:29:30.158 INFO:tasks.workunit.client.0.smithi144.stdout:5/89: rename d4/l12 to d4/d5/l1d 0 2024-09-10T11:29:30.159 INFO:tasks.workunit.client.0.smithi144.stdout:1/112: creat d5/dd/f21 x:0 0 0 2024-09-10T11:29:30.160 INFO:tasks.workunit.client.0.smithi144.stdout:7/85: rmdir d5/dc 39 2024-09-10T11:29:30.162 INFO:tasks.workunit.client.0.smithi144.stdout:5/90: symlink d4/d5/l1e 0 2024-09-10T11:29:30.163 INFO:tasks.workunit.client.0.smithi144.stdout:1/113: stat d5/f8 0 2024-09-10T11:29:30.164 INFO:tasks.workunit.client.0.smithi144.stdout:1/114: dread - d5/f1b zero size 2024-09-10T11:29:30.165 INFO:tasks.workunit.client.0.smithi144.stdout:7/86: chown d5/dc/d11/d18/f1e 2273 1 2024-09-10T11:29:30.166 INFO:tasks.workunit.client.0.smithi144.stdout:5/91: creat d4/d5/d18/f1f x:0 0 0 2024-09-10T11:29:30.166 INFO:tasks.workunit.client.0.smithi144.stdout:5/92: truncate d4/f15 155906 0 2024-09-10T11:29:30.167 INFO:tasks.workunit.client.1.smithi178.stdout:3/466: dread db/d20/f28 [0,4194304] 0 2024-09-10T11:29:30.169 INFO:tasks.workunit.client.1.smithi178.stdout:3/467: write db/d26/f32 [3222853,130430] 0 2024-09-10T11:29:30.171 INFO:tasks.workunit.client.0.smithi144.stdout:5/93: mkdir d4/d20 0 2024-09-10T11:29:30.173 INFO:tasks.workunit.client.0.smithi144.stdout:5/94: write d4/fa [1620868,126854] 0 2024-09-10T11:29:30.174 INFO:tasks.workunit.client.0.smithi144.stdout:5/95: chown d4/d20 466028 1 2024-09-10T11:29:30.176 INFO:tasks.workunit.client.0.smithi144.stdout:5/96: symlink d4/d5/d18/l21 0 2024-09-10T11:29:30.176 INFO:tasks.workunit.client.0.smithi144.stdout:8/72: sync 2024-09-10T11:29:30.177 INFO:tasks.workunit.client.0.smithi144.stdout:5/97: truncate d4/d5/d18/f1f 137144 0 2024-09-10T11:29:30.178 INFO:tasks.workunit.client.0.smithi144.stdout:8/73: write d0/d9/fe [263023,52332] 0 2024-09-10T11:29:30.178 INFO:tasks.workunit.client.0.smithi144.stdout:5/98: dread d4/f15 [0,4194304] 0 2024-09-10T11:29:30.180 INFO:tasks.workunit.client.0.smithi144.stdout:5/99: dread d4/d5/d18/f1f [0,4194304] 0 2024-09-10T11:29:30.181 INFO:tasks.workunit.client.0.smithi144.stdout:7/87: truncate f1 744748 0 2024-09-10T11:29:30.183 INFO:tasks.workunit.client.0.smithi144.stdout:7/88: creat d5/dc/f20 x:0 0 0 2024-09-10T11:29:30.184 INFO:tasks.workunit.client.0.smithi144.stdout:1/115: dwrite d5/fc [0,4194304] 0 2024-09-10T11:29:30.184 INFO:tasks.workunit.client.0.smithi144.stdout:5/100: rmdir d4/d5/d18 39 2024-09-10T11:29:30.185 INFO:tasks.workunit.client.0.smithi144.stdout:1/116: chown d5/fc 240134 1 2024-09-10T11:29:30.190 INFO:tasks.workunit.client.0.smithi144.stdout:5/101: mkdir d4/d20/d22 0 2024-09-10T11:29:30.191 INFO:tasks.workunit.client.0.smithi144.stdout:7/89: dwrite d5/fa [0,4194304] 0 2024-09-10T11:29:30.192 INFO:tasks.workunit.client.0.smithi144.stdout:7/90: chown d5/dc/f10 2937 1 2024-09-10T11:29:30.193 INFO:tasks.workunit.client.0.smithi144.stdout:5/102: fdatasync d4/d5/d18/f1b 0 2024-09-10T11:29:30.193 INFO:tasks.workunit.client.0.smithi144.stdout:5/103: dread - d4/d5/d18/f1b zero size 2024-09-10T11:29:30.196 INFO:tasks.workunit.client.0.smithi144.stdout:7/91: creat d5/dc/d12/f21 x:0 0 0 2024-09-10T11:29:30.196 INFO:tasks.workunit.client.0.smithi144.stdout:1/117: truncate d5/fc 908765 0 2024-09-10T11:29:30.197 INFO:tasks.workunit.client.0.smithi144.stdout:5/104: read d4/f15 [109172,35870] 0 2024-09-10T11:29:30.198 INFO:tasks.workunit.client.0.smithi144.stdout:7/92: creat d5/dc/d12/f22 x:0 0 0 2024-09-10T11:29:30.200 INFO:tasks.workunit.client.0.smithi144.stdout:7/93: read d5/fb [2460254,43631] 0 2024-09-10T11:29:30.201 INFO:tasks.workunit.client.1.smithi178.stdout:5/528: creat d9/dc/d17/d1d/fba x:0 0 0 2024-09-10T11:29:30.201 INFO:tasks.workunit.client.1.smithi178.stdout:5/529: stat d9/dc/dd/ce 0 2024-09-10T11:29:30.203 INFO:tasks.workunit.client.0.smithi144.stdout:5/105: creat d4/d5/f23 x:0 0 0 2024-09-10T11:29:30.205 INFO:tasks.workunit.client.0.smithi144.stdout:5/106: chown d4/d5/l13 47470 1 2024-09-10T11:29:30.206 INFO:tasks.workunit.client.0.smithi144.stdout:1/118: dwrite d5/f1b [0,4194304] 0 2024-09-10T11:29:30.206 INFO:tasks.workunit.client.0.smithi144.stdout:1/119: stat d5/dd/d20 0 2024-09-10T11:29:30.207 INFO:tasks.workunit.client.0.smithi144.stdout:1/120: fsync d5/dd/f21 0 2024-09-10T11:29:30.212 INFO:tasks.workunit.client.0.smithi144.stdout:1/121: dread d5/fe [0,4194304] 0 2024-09-10T11:29:30.213 INFO:tasks.workunit.client.0.smithi144.stdout:5/107: getdents d4/d5 0 2024-09-10T11:29:30.214 INFO:tasks.workunit.client.0.smithi144.stdout:7/94: truncate d5/f7 902903 0 2024-09-10T11:29:30.216 INFO:tasks.workunit.client.0.smithi144.stdout:5/108: dread d4/f15 [0,4194304] 0 2024-09-10T11:29:30.220 INFO:tasks.workunit.client.0.smithi144.stdout:7/95: symlink d5/dc/d11/d18/l23 0 2024-09-10T11:29:30.221 INFO:tasks.workunit.client.0.smithi144.stdout:5/109: unlink d4/d5/l1d 0 2024-09-10T11:29:30.221 INFO:tasks.workunit.client.0.smithi144.stdout:1/122: dread d5/df/f19 [0,4194304] 0 2024-09-10T11:29:30.227 INFO:tasks.workunit.client.0.smithi144.stdout:1/123: getdents d5/dd/d20 0 2024-09-10T11:29:30.230 INFO:tasks.workunit.client.0.smithi144.stdout:7/96: dwrite d5/fa [0,4194304] 0 2024-09-10T11:29:30.230 INFO:tasks.workunit.client.0.smithi144.stdout:1/124: mkdir d5/dd/d22 0 2024-09-10T11:29:30.232 INFO:tasks.workunit.client.0.smithi144.stdout:5/110: dwrite d4/fa [0,4194304] 0 2024-09-10T11:29:30.232 INFO:tasks.workunit.client.0.smithi144.stdout:1/125: dread - d5/dd/f13 zero size 2024-09-10T11:29:30.233 INFO:tasks.workunit.client.0.smithi144.stdout:7/97: truncate d5/dc/f20 494904 0 2024-09-10T11:29:30.238 INFO:tasks.workunit.client.1.smithi178.stdout:6/540: sync 2024-09-10T11:29:30.239 INFO:tasks.workunit.client.1.smithi178.stdout:6/541: chown d3/d7/d8/d15/d1c/d20/fb8 1815 1 2024-09-10T11:29:30.240 INFO:tasks.workunit.client.0.smithi144.stdout:1/126: creat d5/f23 x:0 0 0 2024-09-10T11:29:30.240 INFO:tasks.workunit.client.1.smithi178.stdout:6/542: chown d3/d74/d91/daa/dc2/d97 96 1 2024-09-10T11:29:30.243 INFO:tasks.workunit.client.1.smithi178.stdout:1/459: rename d3/d9/d54 to d3/d9/d98/dac 0 2024-09-10T11:29:30.244 INFO:tasks.workunit.client.0.smithi144.stdout:1/127: truncate d5/f7 5339771 0 2024-09-10T11:29:30.248 INFO:tasks.workunit.client.1.smithi178.stdout:4/378: creat d0/d8/d3e/f85 x:0 0 0 2024-09-10T11:29:30.248 INFO:tasks.workunit.client.1.smithi178.stdout:4/379: fdatasync d0/d8/da/f6b 0 2024-09-10T11:29:30.249 INFO:tasks.workunit.client.1.smithi178.stdout:4/380: readlink d0/d8/df/l43 0 2024-09-10T11:29:30.252 INFO:tasks.workunit.client.0.smithi144.stdout:7/98: fdatasync d5/fa 0 2024-09-10T11:29:30.252 INFO:tasks.workunit.client.1.smithi178.stdout:2/684: creat d1/d22/d4d/d78/fd9 x:0 0 0 2024-09-10T11:29:30.253 INFO:tasks.workunit.client.0.smithi144.stdout:1/128: dwrite d5/fb [0,4194304] 0 2024-09-10T11:29:30.254 INFO:tasks.workunit.client.1.smithi178.stdout:0/543: creat d0/d2c/d89/da1/fb0 x:0 0 0 2024-09-10T11:29:30.255 INFO:tasks.workunit.client.0.smithi144.stdout:5/111: dwrite d4/f15 [0,4194304] 0 2024-09-10T11:29:30.256 INFO:tasks.workunit.client.1.smithi178.stdout:4/381: dwrite d0/d9/d3d/f44 [0,4194304] 0 2024-09-10T11:29:30.257 INFO:tasks.workunit.client.1.smithi178.stdout:0/544: read d0/dc/d39/d50/f1d [3137190,104430] 0 2024-09-10T11:29:30.258 INFO:tasks.workunit.client.1.smithi178.stdout:4/382: dread - d0/d9/d3d/f61 zero size 2024-09-10T11:29:30.259 INFO:tasks.workunit.client.1.smithi178.stdout:4/383: dread - d0/d8/da/d1a/f4b zero size 2024-09-10T11:29:30.260 INFO:tasks.workunit.client.0.smithi144.stdout:5/112: dread - d4/d5/f23 zero size 2024-09-10T11:29:30.260 INFO:tasks.workunit.client.0.smithi144.stdout:7/99: creat d5/dc/d11/d18/f24 x:0 0 0 2024-09-10T11:29:30.262 INFO:tasks.workunit.client.0.smithi144.stdout:1/129: dread d5/df/f19 [0,4194304] 0 2024-09-10T11:29:30.263 INFO:tasks.workunit.client.0.smithi144.stdout:5/113: mkdir d4/d20/d24 0 2024-09-10T11:29:30.263 INFO:tasks.workunit.client.1.smithi178.stdout:5/530: chown d9/dc/dd/d37/d42/d62/l96 57504 1 2024-09-10T11:29:30.264 INFO:tasks.workunit.client.0.smithi144.stdout:1/130: write d5/f8 [3388544,130476] 0 2024-09-10T11:29:30.264 INFO:tasks.workunit.client.1.smithi178.stdout:0/545: read d0/dc/d1f/d2d/f42 [4570995,106093] 0 2024-09-10T11:29:30.265 INFO:tasks.workunit.client.0.smithi144.stdout:7/100: symlink d5/dc/d11/l25 0 2024-09-10T11:29:30.266 INFO:tasks.workunit.client.1.smithi178.stdout:0/546: chown d0/dc/d1f/d2d/fa8 0 1 2024-09-10T11:29:30.268 INFO:tasks.workunit.client.1.smithi178.stdout:0/547: write d0/dc/d39/d4c/f73 [4721980,37468] 0 2024-09-10T11:29:30.273 INFO:tasks.workunit.client.0.smithi144.stdout:1/131: creat d5/dd/d22/f24 x:0 0 0 2024-09-10T11:29:30.273 INFO:tasks.workunit.client.1.smithi178.stdout:0/548: truncate d0/dc/d39/d50/f88 75060 0 2024-09-10T11:29:30.273 INFO:tasks.workunit.client.1.smithi178.stdout:0/549: chown d0/d12/f2f 4769 1 2024-09-10T11:29:30.273 INFO:tasks.workunit.client.1.smithi178.stdout:0/550: write d0/dc/d39/d7c/fa9 [789590,103474] 0 2024-09-10T11:29:30.274 INFO:tasks.workunit.client.1.smithi178.stdout:1/460: rmdir d3/d9/d98/dac/d3a/d63/d7e 39 2024-09-10T11:29:30.274 INFO:tasks.workunit.client.0.smithi144.stdout:5/114: creat d4/d5/f25 x:0 0 0 2024-09-10T11:29:30.274 INFO:tasks.workunit.client.0.smithi144.stdout:5/115: read d4/fa [1598989,94874] 0 2024-09-10T11:29:30.275 INFO:tasks.workunit.client.1.smithi178.stdout:0/551: read d0/d2c/d33/f81 [72660,127257] 0 2024-09-10T11:29:30.275 INFO:tasks.workunit.client.0.smithi144.stdout:1/132: fsync d5/f15 0 2024-09-10T11:29:30.275 INFO:tasks.workunit.client.0.smithi144.stdout:7/101: truncate d5/fa 2152327 0 2024-09-10T11:29:30.276 INFO:tasks.workunit.client.0.smithi144.stdout:1/133: chown d5/fb 52037881 1 2024-09-10T11:29:30.277 INFO:tasks.workunit.client.0.smithi144.stdout:7/102: chown d5/dc/d12/f22 418277 1 2024-09-10T11:29:30.278 INFO:tasks.workunit.client.0.smithi144.stdout:5/116: symlink d4/d5/d18/l26 0 2024-09-10T11:29:30.280 INFO:tasks.workunit.client.0.smithi144.stdout:5/117: stat d4/d5/d18/f1f 0 2024-09-10T11:29:30.280 INFO:tasks.workunit.client.0.smithi144.stdout:7/103: creat d5/dc/d11/f26 x:0 0 0 2024-09-10T11:29:30.282 INFO:tasks.workunit.client.0.smithi144.stdout:5/118: write d4/d5/f25 [538282,112194] 0 2024-09-10T11:29:30.283 INFO:tasks.workunit.client.0.smithi144.stdout:1/134: dread d5/fb [0,4194304] 0 2024-09-10T11:29:30.284 INFO:tasks.workunit.client.0.smithi144.stdout:1/135: chown d5/df 20 1 2024-09-10T11:29:30.287 INFO:tasks.workunit.client.0.smithi144.stdout:7/104: dread f1 [0,4194304] 0 2024-09-10T11:29:30.288 INFO:tasks.workunit.client.0.smithi144.stdout:1/136: write d5/fe [2190741,118778] 0 2024-09-10T11:29:30.290 INFO:tasks.workunit.client.1.smithi178.stdout:7/424: sync 2024-09-10T11:29:30.290 INFO:tasks.workunit.client.1.smithi178.stdout:8/548: sync 2024-09-10T11:29:30.291 INFO:tasks.workunit.client.0.smithi144.stdout:7/105: dread f1 [0,4194304] 0 2024-09-10T11:29:30.291 INFO:tasks.workunit.client.1.smithi178.stdout:8/549: dread - d5/db/d10/d35/f44 zero size 2024-09-10T11:29:30.293 INFO:tasks.workunit.client.1.smithi178.stdout:6/543: rmdir d3/d7/d8/d72/d75 0 2024-09-10T11:29:30.294 INFO:tasks.workunit.client.0.smithi144.stdout:5/119: dwrite d4/fa [0,4194304] 0 2024-09-10T11:29:30.295 INFO:tasks.workunit.client.0.smithi144.stdout:5/120: write d4/d5/f25 [161902,46704] 0 2024-09-10T11:29:30.297 INFO:tasks.workunit.client.0.smithi144.stdout:7/106: fsync d5/dc/d12/f13 0 2024-09-10T11:29:30.300 INFO:tasks.workunit.client.1.smithi178.stdout:7/425: symlink d6/d10/d1e/d2d/d2b/l9c 0 2024-09-10T11:29:30.301 INFO:tasks.workunit.client.0.smithi144.stdout:1/137: dwrite d5/f14 [0,4194304] 0 2024-09-10T11:29:30.303 INFO:tasks.workunit.client.0.smithi144.stdout:7/107: fdatasync d5/f7 0 2024-09-10T11:29:30.305 INFO:tasks.workunit.client.0.smithi144.stdout:1/138: mknod d5/dd/d22/c25 0 2024-09-10T11:29:30.305 INFO:tasks.workunit.client.0.smithi144.stdout:7/108: dread - d5/dc/d11/f17 zero size 2024-09-10T11:29:30.306 INFO:tasks.workunit.client.0.smithi144.stdout:5/121: getdents d4 0 2024-09-10T11:29:30.307 INFO:tasks.workunit.client.1.smithi178.stdout:7/426: unlink d6/d10/d34/d6a/d6f/l88 0 2024-09-10T11:29:30.307 INFO:tasks.workunit.client.0.smithi144.stdout:7/109: readlink d5/dc/ld 0 2024-09-10T11:29:30.308 INFO:tasks.workunit.client.0.smithi144.stdout:1/139: symlink d5/dd/d20/l26 0 2024-09-10T11:29:30.309 INFO:tasks.workunit.client.0.smithi144.stdout:5/122: stat d4/d5/cf 0 2024-09-10T11:29:30.309 INFO:tasks.workunit.client.0.smithi144.stdout:5/123: stat d4/d5/d18/l21 0 2024-09-10T11:29:30.310 INFO:tasks.workunit.client.0.smithi144.stdout:7/110: write d5/f7 [710777,129853] 0 2024-09-10T11:29:30.311 INFO:tasks.workunit.client.1.smithi178.stdout:8/550: link d5/d1d/d22/d76/f9e d5/db/d10/d53/d82/fba 0 2024-09-10T11:29:30.311 INFO:tasks.workunit.client.0.smithi144.stdout:7/111: read - d5/dc/d11/d18/f24 zero size 2024-09-10T11:29:30.312 INFO:tasks.workunit.client.1.smithi178.stdout:8/551: write d5/db/d10/d53/f64 [648550,125440] 0 2024-09-10T11:29:30.314 INFO:tasks.workunit.client.1.smithi178.stdout:8/552: write d5/db/d10/d53/f64 [4111426,57586] 0 2024-09-10T11:29:30.315 INFO:tasks.workunit.client.1.smithi178.stdout:8/553: chown d5/d30/d33/c3e 0 1 2024-09-10T11:29:30.315 INFO:tasks.workunit.client.0.smithi144.stdout:7/112: truncate d5/dc/d11/d18/f1e 691964 0 2024-09-10T11:29:30.316 INFO:tasks.workunit.client.1.smithi178.stdout:6/544: link d3/d6e/f71 d3/d24/fc7 0 2024-09-10T11:29:30.320 INFO:tasks.workunit.client.0.smithi144.stdout:7/113: truncate d5/dc/d11/d18/f24 454594 0 2024-09-10T11:29:30.321 INFO:tasks.workunit.client.0.smithi144.stdout:1/140: dwrite d5/dd/d20/f1e [0,4194304] 0 2024-09-10T11:29:30.321 INFO:tasks.workunit.client.0.smithi144.stdout:7/114: chown d5/dc/d12 842425 1 2024-09-10T11:29:30.323 INFO:tasks.workunit.client.1.smithi178.stdout:8/554: dwrite d5/d30/d33/f99 [0,4194304] 0 2024-09-10T11:29:30.324 INFO:tasks.workunit.client.1.smithi178.stdout:8/555: stat d5/d30/f4b 0 2024-09-10T11:29:30.325 INFO:tasks.workunit.client.0.smithi144.stdout:1/141: creat d5/dd/d22/f27 x:0 0 0 2024-09-10T11:29:30.326 INFO:tasks.workunit.client.0.smithi144.stdout:7/115: write d5/dc/f20 [761169,67446] 0 2024-09-10T11:29:30.328 INFO:tasks.workunit.client.1.smithi178.stdout:8/556: mkdir d5/db/d10/d20/d52/d67/dbb 0 2024-09-10T11:29:30.329 INFO:tasks.workunit.client.1.smithi178.stdout:6/545: link d3/d7/d35/c73 d3/d7/d8/d15/d1c/d20/da6/cc8 0 2024-09-10T11:29:30.329 INFO:tasks.workunit.client.1.smithi178.stdout:8/557: chown d5/db/d10/d70/ca4 1535765 1 2024-09-10T11:29:30.329 INFO:tasks.workunit.client.0.smithi144.stdout:5/124: dwrite d4/d5/d18/f1f [0,4194304] 0 2024-09-10T11:29:30.330 INFO:tasks.workunit.client.0.smithi144.stdout:5/125: stat d4/l16 0 2024-09-10T11:29:30.331 INFO:tasks.workunit.client.1.smithi178.stdout:8/558: truncate d5/d30/fb6 190632 0 2024-09-10T11:29:30.333 INFO:tasks.workunit.client.0.smithi144.stdout:1/142: mknod d5/c28 0 2024-09-10T11:29:30.333 INFO:tasks.workunit.client.0.smithi144.stdout:1/143: fsync d5/f15 0 2024-09-10T11:29:30.337 INFO:tasks.workunit.client.1.smithi178.stdout:9/571: rename d2/d51/d59 to d2/d19/d1c/d2d/dab 0 2024-09-10T11:29:30.340 INFO:tasks.workunit.client.0.smithi144.stdout:7/116: rename d5/dc/d11/d18/c1a to d5/dc/d14/c27 0 2024-09-10T11:29:30.342 INFO:tasks.workunit.client.0.smithi144.stdout:5/126: chown d4/d5/l17 694 1 2024-09-10T11:29:30.344 INFO:tasks.workunit.client.0.smithi144.stdout:1/144: dread d5/f12 [0,4194304] 0 2024-09-10T11:29:30.345 INFO:tasks.workunit.client.1.smithi178.stdout:8/559: getdents d5/d1d/d22 0 2024-09-10T11:29:30.345 INFO:tasks.workunit.client.0.smithi144.stdout:7/117: mknod d5/c28 0 2024-09-10T11:29:30.345 INFO:tasks.workunit.client.0.smithi144.stdout:5/127: symlink d4/d5/l27 0 2024-09-10T11:29:30.347 INFO:tasks.workunit.client.0.smithi144.stdout:7/118: read d5/dc/d12/f13 [1592039,130706] 0 2024-09-10T11:29:30.347 INFO:tasks.workunit.client.1.smithi178.stdout:8/560: mknod d5/d30/d33/d75/cbc 0 2024-09-10T11:29:30.349 INFO:tasks.workunit.client.0.smithi144.stdout:7/119: dread d5/dc/d11/d18/f24 [0,4194304] 0 2024-09-10T11:29:30.350 INFO:tasks.workunit.client.1.smithi178.stdout:3/468: unlink db/d20/l22 0 2024-09-10T11:29:30.350 INFO:tasks.workunit.client.1.smithi178.stdout:8/561: rmdir d5/d1d 39 2024-09-10T11:29:30.350 INFO:tasks.workunit.client.1.smithi178.stdout:3/469: chown db/d36/c86 9637877 1 2024-09-10T11:29:30.351 INFO:tasks.workunit.client.0.smithi144.stdout:5/128: creat d4/d5/f28 x:0 0 0 2024-09-10T11:29:30.352 INFO:tasks.workunit.client.0.smithi144.stdout:7/120: truncate d5/fb 3729187 0 2024-09-10T11:29:30.352 INFO:tasks.workunit.client.0.smithi144.stdout:5/129: chown d4/d5/cf 0 1 2024-09-10T11:29:30.353 INFO:tasks.workunit.client.0.smithi144.stdout:1/145: truncate d5/fe 1691327 0 2024-09-10T11:29:30.353 INFO:tasks.workunit.client.0.smithi144.stdout:7/121: fdatasync d5/dc/d12/f1f 0 2024-09-10T11:29:30.354 INFO:tasks.workunit.client.0.smithi144.stdout:1/146: read - d5/dd/d22/f27 zero size 2024-09-10T11:29:30.354 INFO:tasks.workunit.client.1.smithi178.stdout:8/562: creat d5/d30/db8/fbd x:0 0 0 2024-09-10T11:29:30.355 INFO:tasks.workunit.client.0.smithi144.stdout:7/122: fdatasync d5/dc/d11/f17 0 2024-09-10T11:29:30.356 INFO:tasks.workunit.client.0.smithi144.stdout:1/147: rename d5/dd to d5/dd/d29 22 2024-09-10T11:29:30.357 INFO:tasks.workunit.client.0.smithi144.stdout:7/123: dread d5/dc/d11/d18/f24 [0,4194304] 0 2024-09-10T11:29:30.358 INFO:tasks.workunit.client.0.smithi144.stdout:5/130: link d4/d5/l17 d4/d20/l29 0 2024-09-10T11:29:30.358 INFO:tasks.workunit.client.0.smithi144.stdout:7/124: write d5/f8 [240550,3155] 0 2024-09-10T11:29:30.362 INFO:tasks.workunit.client.1.smithi178.stdout:8/563: dwrite d5/db/d10/d70/f78 [0,4194304] 0 2024-09-10T11:29:30.365 INFO:tasks.workunit.client.0.smithi144.stdout:7/125: creat d5/f29 x:0 0 0 2024-09-10T11:29:30.366 INFO:tasks.workunit.client.0.smithi144.stdout:5/131: dwrite d4/f15 [0,4194304] 0 2024-09-10T11:29:30.367 INFO:tasks.workunit.client.0.smithi144.stdout:5/132: dread - d4/d5/f28 zero size 2024-09-10T11:29:30.368 INFO:tasks.workunit.client.0.smithi144.stdout:5/133: write d4/d5/d18/f1f [2289317,10104] 0 2024-09-10T11:29:30.370 INFO:tasks.workunit.client.0.smithi144.stdout:7/126: mkdir d5/dc/d2a 0 2024-09-10T11:29:30.370 INFO:tasks.workunit.client.1.smithi178.stdout:8/564: dread d5/db/d1e/f39 [0,4194304] 0 2024-09-10T11:29:30.373 INFO:tasks.workunit.client.0.smithi144.stdout:1/148: write d5/fe [1856659,95856] 0 2024-09-10T11:29:30.373 INFO:tasks.workunit.client.0.smithi144.stdout:7/127: mkdir d5/dc/d12/d2b 0 2024-09-10T11:29:30.374 INFO:tasks.workunit.client.1.smithi178.stdout:0/552: mkdir d0/db1 0 2024-09-10T11:29:30.376 INFO:tasks.workunit.client.0.smithi144.stdout:1/149: truncate d5/f14 5048592 0 2024-09-10T11:29:30.376 INFO:tasks.workunit.client.0.smithi144.stdout:4/98: sync 2024-09-10T11:29:30.376 INFO:tasks.workunit.client.0.smithi144.stdout:9/50: sync 2024-09-10T11:29:30.379 INFO:tasks.workunit.client.1.smithi178.stdout:0/553: chown d0/dc/ff 9798183 1 2024-09-10T11:29:30.380 INFO:tasks.workunit.client.1.smithi178.stdout:2/685: write d1/d7/d24/d46/d88/fcd [3264486,100960] 0 2024-09-10T11:29:30.380 INFO:tasks.workunit.client.0.smithi144.stdout:1/150: unlink d5/f14 0 2024-09-10T11:29:30.382 INFO:tasks.workunit.client.0.smithi144.stdout:7/128: dwrite d5/f8 [0,4194304] 0 2024-09-10T11:29:30.384 INFO:tasks.workunit.client.1.smithi178.stdout:8/565: dwrite d5/db/d1e/f62 [0,4194304] 0 2024-09-10T11:29:30.385 INFO:tasks.workunit.client.0.smithi144.stdout:4/99: dread f7 [0,4194304] 0 2024-09-10T11:29:30.385 INFO:tasks.workunit.client.0.smithi144.stdout:5/134: link d4/d5/c9 d4/d20/d24/c2a 0 2024-09-10T11:29:30.386 INFO:tasks.workunit.client.1.smithi178.stdout:4/384: dwrite d0/f38 [0,4194304] 0 2024-09-10T11:29:30.387 INFO:tasks.workunit.client.0.smithi144.stdout:1/151: truncate d5/dd/f13 789071 0 2024-09-10T11:29:30.387 INFO:tasks.workunit.client.1.smithi178.stdout:0/554: mknod d0/dc/d39/d50/cb2 0 2024-09-10T11:29:30.388 INFO:tasks.workunit.client.0.smithi144.stdout:7/129: creat d5/dc/d12/f2c x:0 0 0 2024-09-10T11:29:30.388 INFO:tasks.workunit.client.1.smithi178.stdout:8/566: write d5/db/d10/d20/f7b [555536,52542] 0 2024-09-10T11:29:30.389 INFO:tasks.workunit.client.1.smithi178.stdout:0/555: write d0/dc/d1f/fad [76911,45928] 0 2024-09-10T11:29:30.390 INFO:tasks.workunit.client.0.smithi144.stdout:4/100: creat d14/f1f x:0 0 0 2024-09-10T11:29:30.391 INFO:tasks.workunit.client.1.smithi178.stdout:5/531: rename d9/f90 to d9/d19/d91/fbb 0 2024-09-10T11:29:30.391 INFO:tasks.workunit.client.0.smithi144.stdout:7/130: read d5/dc/d11/d18/f24 [377060,13424] 0 2024-09-10T11:29:30.394 INFO:tasks.workunit.client.0.smithi144.stdout:1/152: dwrite d5/dd/f21 [0,4194304] 0 2024-09-10T11:29:30.398 INFO:tasks.workunit.client.1.smithi178.stdout:8/567: unlink d5/db/d10/d20/f3b 0 2024-09-10T11:29:30.399 INFO:tasks.workunit.client.0.smithi144.stdout:4/101: creat d14/d1e/f20 x:0 0 0 2024-09-10T11:29:30.401 INFO:tasks.workunit.client.0.smithi144.stdout:4/102: write f9 [3591291,120835] 0 2024-09-10T11:29:30.402 INFO:tasks.workunit.client.0.smithi144.stdout:5/135: link d4/d5/l13 d4/d20/l2b 0 2024-09-10T11:29:30.405 INFO:tasks.workunit.client.1.smithi178.stdout:2/686: rmdir d1/d7/d24/d46/d88/d9d/dd2 0 2024-09-10T11:29:30.406 INFO:tasks.workunit.client.0.smithi144.stdout:4/103: unlink f1 0 2024-09-10T11:29:30.409 INFO:tasks.workunit.client.1.smithi178.stdout:5/532: dwrite d9/d11/faa [0,4194304] 0 2024-09-10T11:29:30.410 INFO:tasks.workunit.client.1.smithi178.stdout:5/533: dread - d9/d11/fa2 zero size 2024-09-10T11:29:30.411 INFO:tasks.workunit.client.1.smithi178.stdout:6/546: rename d3/d7/d8/d15/d1c/d20 to d3/d74/dc9 0 2024-09-10T11:29:30.411 INFO:tasks.workunit.client.0.smithi144.stdout:5/136: creat d4/f2c x:0 0 0 2024-09-10T11:29:30.415 INFO:tasks.workunit.client.0.smithi144.stdout:4/104: dread f9 [0,4194304] 0 2024-09-10T11:29:30.415 INFO:tasks.workunit.client.0.smithi144.stdout:9/51: write f0 [1619289,109036] 0 2024-09-10T11:29:30.420 INFO:tasks.workunit.client.0.smithi144.stdout:1/153: getdents d5 0 2024-09-10T11:29:30.422 INFO:tasks.workunit.client.0.smithi144.stdout:4/105: stat d14/f18 0 2024-09-10T11:29:30.423 INFO:tasks.workunit.client.0.smithi144.stdout:1/154: write d5/dd/f13 [283568,76360] 0 2024-09-10T11:29:30.425 INFO:tasks.workunit.client.0.smithi144.stdout:7/131: write d5/fb [592253,24661] 0 2024-09-10T11:29:30.425 INFO:tasks.workunit.client.1.smithi178.stdout:6/547: getdents d3/d74/d91/daa/dc2/d57/d78/dc3 0 2024-09-10T11:29:30.427 INFO:tasks.workunit.client.1.smithi178.stdout:5/534: dread d9/dc/d17/d1d/f73 [0,4194304] 0 2024-09-10T11:29:30.427 INFO:tasks.workunit.client.0.smithi144.stdout:9/52: dwrite f9 [4194304,4194304] 0 2024-09-10T11:29:30.428 INFO:tasks.workunit.client.0.smithi144.stdout:4/106: creat d14/d1e/f21 x:0 0 0 2024-09-10T11:29:30.428 INFO:tasks.workunit.client.1.smithi178.stdout:5/535: read d9/d11/faa [1894733,85564] 0 2024-09-10T11:29:30.429 INFO:tasks.workunit.client.0.smithi144.stdout:9/53: chown cb 51143654 1 2024-09-10T11:29:30.430 INFO:tasks.workunit.client.1.smithi178.stdout:8/568: getdents d5/d1d/d22/d76 0 2024-09-10T11:29:30.430 INFO:tasks.workunit.client.1.smithi178.stdout:5/536: write d9/dc/dd/d16/f27 [2734990,37292] 0 2024-09-10T11:29:30.434 INFO:tasks.workunit.client.1.smithi178.stdout:7/427: symlink d6/d10/d1d/l9d 0 2024-09-10T11:29:30.434 INFO:tasks.workunit.client.0.smithi144.stdout:7/132: symlink d5/dc/d12/d2b/l2d 0 2024-09-10T11:29:30.436 INFO:tasks.workunit.client.1.smithi178.stdout:7/428: stat d6/d10/d1d/d22/d50/c97 0 2024-09-10T11:29:30.436 INFO:tasks.workunit.client.1.smithi178.stdout:5/537: fsync d9/f86 0 2024-09-10T11:29:30.438 INFO:tasks.workunit.client.0.smithi144.stdout:1/155: getdents d5/dd 0 2024-09-10T11:29:30.440 INFO:tasks.workunit.client.1.smithi178.stdout:4/385: mknod d0/d8/c86 0 2024-09-10T11:29:30.442 INFO:tasks.workunit.client.1.smithi178.stdout:1/461: symlink d3/d9/lad 0 2024-09-10T11:29:30.442 INFO:tasks.workunit.client.1.smithi178.stdout:7/429: dread d6/d10/d1d/d22/d3b/f5a [0,4194304] 0 2024-09-10T11:29:30.443 INFO:tasks.workunit.client.0.smithi144.stdout:9/54: fdatasync f0 0 2024-09-10T11:29:30.445 INFO:tasks.workunit.client.0.smithi144.stdout:7/133: link d5/dc/ld d5/dc/d11/d18/l2e 0 2024-09-10T11:29:30.445 INFO:tasks.workunit.client.0.smithi144.stdout:5/137: getdents d4/d20 0 2024-09-10T11:29:30.446 INFO:tasks.workunit.client.1.smithi178.stdout:4/386: symlink d0/d10/l87 0 2024-09-10T11:29:30.447 INFO:tasks.workunit.client.1.smithi178.stdout:2/687: dread d1/d41/d14/f1b [0,4194304] 0 2024-09-10T11:29:30.447 INFO:tasks.workunit.client.0.smithi144.stdout:9/55: dread f0 [0,4194304] 0 2024-09-10T11:29:30.448 INFO:tasks.workunit.client.0.smithi144.stdout:5/138: chown d4/d5/cf 25 1 2024-09-10T11:29:30.449 INFO:tasks.workunit.client.1.smithi178.stdout:9/572: symlink d2/d13/lac 0 2024-09-10T11:29:30.449 INFO:tasks.workunit.client.1.smithi178.stdout:5/538: dread d9/dc/dd/d37/d42/f5f [0,4194304] 0 2024-09-10T11:29:30.449 INFO:tasks.workunit.client.1.smithi178.stdout:7/430: dread d6/d10/d34/f74 [0,4194304] 0 2024-09-10T11:29:30.451 INFO:tasks.workunit.client.1.smithi178.stdout:7/431: fdatasync d6/d10/d1e/f35 0 2024-09-10T11:29:30.451 INFO:tasks.workunit.client.1.smithi178.stdout:5/539: write d9/dc/d17/d1d/fb1 [994162,105894] 0 2024-09-10T11:29:30.452 INFO:tasks.workunit.client.1.smithi178.stdout:3/470: rename db/d35/c7d to db/d20/d3d/d63/c88 0 2024-09-10T11:29:30.452 INFO:tasks.workunit.client.0.smithi144.stdout:7/134: mkdir d5/dc/d12/d2f 0 2024-09-10T11:29:30.453 INFO:tasks.workunit.client.0.smithi144.stdout:7/135: read - d5/dc/d12/f2c zero size 2024-09-10T11:29:30.453 INFO:tasks.workunit.client.0.smithi144.stdout:9/56: dread f9 [0,4194304] 0 2024-09-10T11:29:30.454 INFO:tasks.workunit.client.1.smithi178.stdout:7/432: truncate d6/d10/d34/f4e 1004943 0 2024-09-10T11:29:30.454 INFO:tasks.workunit.client.0.smithi144.stdout:7/136: read d5/fb [2521313,86340] 0 2024-09-10T11:29:30.454 INFO:tasks.workunit.client.1.smithi178.stdout:3/471: write db/d35/f5f [3317717,56502] 0 2024-09-10T11:29:30.455 INFO:tasks.workunit.client.1.smithi178.stdout:4/387: symlink d0/d8/da/l88 0 2024-09-10T11:29:30.455 INFO:tasks.workunit.client.0.smithi144.stdout:4/107: mkdir d14/d1e/d22 0 2024-09-10T11:29:30.455 INFO:tasks.workunit.client.0.smithi144.stdout:5/139: mkdir d4/d20/d24/d2d 0 2024-09-10T11:29:30.456 INFO:tasks.workunit.client.1.smithi178.stdout:3/472: fsync db/d20/f3c 0 2024-09-10T11:29:30.456 INFO:tasks.workunit.client.1.smithi178.stdout:7/433: chown d6/d10/d1e/d2d/f8a 93193 1 2024-09-10T11:29:30.456 INFO:tasks.workunit.client.1.smithi178.stdout:2/688: mknod d1/d22/d4d/d78/cda 0 2024-09-10T11:29:30.457 INFO:tasks.workunit.client.1.smithi178.stdout:9/573: mkdir d2/d51/dad 0 2024-09-10T11:29:30.458 INFO:tasks.workunit.client.1.smithi178.stdout:1/462: mkdir d3/d9/d98/dac/d3a/dae 0 2024-09-10T11:29:30.459 INFO:tasks.workunit.client.0.smithi144.stdout:5/140: rmdir d4/d20 39 2024-09-10T11:29:30.460 INFO:tasks.workunit.client.0.smithi144.stdout:9/57: symlink d11/d12/l13 0 2024-09-10T11:29:30.461 INFO:tasks.workunit.client.0.smithi144.stdout:4/108: creat d14/d1e/d22/f23 x:0 0 0 2024-09-10T11:29:30.464 INFO:tasks.workunit.client.0.smithi144.stdout:5/141: fdatasync d4/d5/d18/f1f 0 2024-09-10T11:29:30.466 INFO:tasks.workunit.client.0.smithi144.stdout:7/137: getdents d5/dc 0 2024-09-10T11:29:30.466 INFO:tasks.workunit.client.0.smithi144.stdout:4/109: read d14/f18 [1546667,37932] 0 2024-09-10T11:29:30.467 INFO:tasks.workunit.client.1.smithi178.stdout:4/388: mknod d0/d8/da/d1a/d51/c89 0 2024-09-10T11:29:30.468 INFO:tasks.workunit.client.0.smithi144.stdout:4/110: chown f9 42455952 1 2024-09-10T11:29:30.468 INFO:tasks.workunit.client.1.smithi178.stdout:5/540: dwrite d9/dc/dd/d37/d3e/fa6 [0,4194304] 0 2024-09-10T11:29:30.469 INFO:tasks.workunit.client.0.smithi144.stdout:7/138: mknod d5/dc/d11/c30 0 2024-09-10T11:29:30.470 INFO:tasks.workunit.client.0.smithi144.stdout:5/142: write d4/fa [496333,12913] 0 2024-09-10T11:29:30.470 INFO:tasks.workunit.client.0.smithi144.stdout:9/58: dwrite f10 [0,4194304] 0 2024-09-10T11:29:30.470 INFO:tasks.workunit.client.1.smithi178.stdout:3/473: rename db/d20/l25 to db/d26/d40/l89 0 2024-09-10T11:29:30.471 INFO:tasks.workunit.client.0.smithi144.stdout:4/111: mknod d14/d1e/d22/c24 0 2024-09-10T11:29:30.472 INFO:tasks.workunit.client.1.smithi178.stdout:0/556: truncate d0/f8 7688736 0 2024-09-10T11:29:30.474 INFO:tasks.workunit.client.1.smithi178.stdout:6/548: write d3/d7/d35/f6f [4906427,93835] 0 2024-09-10T11:29:30.475 INFO:tasks.workunit.client.0.smithi144.stdout:9/59: rename f9 to d11/d12/f14 0 2024-09-10T11:29:30.476 INFO:tasks.workunit.client.1.smithi178.stdout:9/574: read d2/f26 [119336,2876] 0 2024-09-10T11:29:30.478 INFO:tasks.workunit.client.0.smithi144.stdout:7/139: unlink d5/dc/d11/d18/l2e 0 2024-09-10T11:29:30.478 INFO:tasks.workunit.client.0.smithi144.stdout:7/140: chown d5/dc/d14 210 1 2024-09-10T11:29:30.479 INFO:tasks.workunit.client.1.smithi178.stdout:2/689: symlink d1/ldb 0 2024-09-10T11:29:30.479 INFO:tasks.workunit.client.1.smithi178.stdout:4/389: dwrite d0/f40 [0,4194304] 0 2024-09-10T11:29:30.480 INFO:tasks.workunit.client.1.smithi178.stdout:2/690: chown d1 262 1 2024-09-10T11:29:30.480 INFO:tasks.workunit.client.0.smithi144.stdout:9/60: creat d11/f15 x:0 0 0 2024-09-10T11:29:30.482 INFO:tasks.workunit.client.1.smithi178.stdout:5/541: mkdir d9/dc/dd/d3f/d5a/dbc 0 2024-09-10T11:29:30.483 INFO:tasks.workunit.client.0.smithi144.stdout:4/112: rename c6 to d14/c25 0 2024-09-10T11:29:30.484 INFO:tasks.workunit.client.1.smithi178.stdout:2/691: fsync d1/d7/f6a 0 2024-09-10T11:29:30.484 INFO:tasks.workunit.client.0.smithi144.stdout:9/61: symlink d11/l16 0 2024-09-10T11:29:30.485 INFO:tasks.workunit.client.1.smithi178.stdout:3/474: creat db/d20/f8a x:0 0 0 2024-09-10T11:29:30.485 INFO:tasks.workunit.client.0.smithi144.stdout:9/62: write fc [60430,16311] 0 2024-09-10T11:29:30.487 INFO:tasks.workunit.client.1.smithi178.stdout:8/569: truncate d5/db/d10/d20/f28 2432716 0 2024-09-10T11:29:30.487 INFO:tasks.workunit.client.0.smithi144.stdout:7/141: mknod d5/dc/d12/d2f/c31 0 2024-09-10T11:29:30.488 INFO:tasks.workunit.client.1.smithi178.stdout:7/434: link d6/d10/d1d/d22/d50/d7e/d82/f92 d6/d10/d1d/d22/d93/f9e 0 2024-09-10T11:29:30.488 INFO:tasks.workunit.client.1.smithi178.stdout:6/549: mknod d3/d74/dc9/d4b/cca 0 2024-09-10T11:29:30.488 INFO:tasks.workunit.client.0.smithi144.stdout:7/142: truncate d5/dc/f10 1167334 0 2024-09-10T11:29:30.491 INFO:tasks.workunit.client.1.smithi178.stdout:3/475: write db/d36/f50 [4558604,76955] 0 2024-09-10T11:29:30.492 INFO:tasks.workunit.client.0.smithi144.stdout:9/63: link fc d11/f17 0 2024-09-10T11:29:30.492 INFO:tasks.workunit.client.0.smithi144.stdout:4/113: link lf d14/d1e/l26 0 2024-09-10T11:29:30.493 INFO:tasks.workunit.client.1.smithi178.stdout:9/575: mkdir d2/d13/d40/d89/d93/d88/dae 0 2024-09-10T11:29:30.493 INFO:tasks.workunit.client.0.smithi144.stdout:9/64: chown d11/l16 1 1 2024-09-10T11:29:30.493 INFO:tasks.workunit.client.0.smithi144.stdout:7/143: symlink d5/dc/d12/l32 0 2024-09-10T11:29:30.494 INFO:tasks.workunit.client.0.smithi144.stdout:4/114: read - d14/d1e/f20 zero size 2024-09-10T11:29:30.494 INFO:tasks.workunit.client.1.smithi178.stdout:0/557: dwrite d0/d12/d3f/d72/f82 [0,4194304] 0 2024-09-10T11:29:30.494 INFO:tasks.workunit.client.0.smithi144.stdout:7/144: dread - d5/dc/d12/f1f zero size 2024-09-10T11:29:30.495 INFO:tasks.workunit.client.0.smithi144.stdout:7/145: dread - d5/f29 zero size 2024-09-10T11:29:30.497 INFO:tasks.workunit.client.0.smithi144.stdout:4/115: creat d14/d1e/f27 x:0 0 0 2024-09-10T11:29:30.502 INFO:tasks.workunit.client.1.smithi178.stdout:5/542: dread d9/dc/dd/d16/f27 [0,4194304] 0 2024-09-10T11:29:30.502 INFO:tasks.workunit.client.0.smithi144.stdout:4/116: unlink f7 0 2024-09-10T11:29:30.507 INFO:tasks.workunit.client.1.smithi178.stdout:4/390: mkdir d0/d9/dc/d8a 0 2024-09-10T11:29:30.511 INFO:tasks.workunit.client.0.smithi144.stdout:9/65: dwrite d11/f17 [0,4194304] 0 2024-09-10T11:29:30.513 INFO:tasks.workunit.client.1.smithi178.stdout:3/476: symlink db/d20/d3d/d56/l8b 0 2024-09-10T11:29:30.513 INFO:tasks.workunit.client.0.smithi144.stdout:7/146: dwrite d5/dc/f20 [0,4194304] 0 2024-09-10T11:29:30.514 INFO:tasks.workunit.client.1.smithi178.stdout:9/576: rmdir d2/d13/d75 39 2024-09-10T11:29:30.514 INFO:tasks.workunit.client.0.smithi144.stdout:7/147: write d5/f29 [215018,45954] 0 2024-09-10T11:29:30.515 INFO:tasks.workunit.client.1.smithi178.stdout:5/543: dread d9/dc/dd/d37/d42/d55/f6e [0,4194304] 0 2024-09-10T11:29:30.515 INFO:tasks.workunit.client.1.smithi178.stdout:6/550: rmdir d3/d7/d8/d15/d1c/d2a/d18/d8e 39 2024-09-10T11:29:30.516 INFO:tasks.workunit.client.0.smithi144.stdout:7/148: dread - d5/dc/d12/f1f zero size 2024-09-10T11:29:30.516 INFO:tasks.workunit.client.0.smithi144.stdout:4/117: rename d14/d1e/d22/c24 to d14/d1e/d22/c28 0 2024-09-10T11:29:30.516 INFO:tasks.workunit.client.1.smithi178.stdout:5/544: chown d9/dc/dd/d37/d42/d55 47322 1 2024-09-10T11:29:30.518 INFO:tasks.workunit.client.1.smithi178.stdout:7/435: creat d6/d10/d1d/d22/d50/d7e/d82/f9f x:0 0 0 2024-09-10T11:29:30.519 INFO:tasks.workunit.client.1.smithi178.stdout:9/577: dread d2/d8/f17 [0,4194304] 0 2024-09-10T11:29:30.521 INFO:tasks.workunit.client.1.smithi178.stdout:2/692: rename d1/d22/d4d/d78/fc3 to d1/d7/d24/d46/d88/d9d/dab/d5e/fdc 0 2024-09-10T11:29:30.522 INFO:tasks.workunit.client.0.smithi144.stdout:9/66: rename c6 to d11/d12/c18 0 2024-09-10T11:29:30.523 INFO:tasks.workunit.client.0.smithi144.stdout:7/149: symlink d5/dc/d14/l33 0 2024-09-10T11:29:30.523 INFO:tasks.workunit.client.1.smithi178.stdout:2/693: write d1/d41/d14/d11/d53/f8b [739569,102907] 0 2024-09-10T11:29:30.526 INFO:tasks.workunit.client.0.smithi144.stdout:4/118: rename le to d14/d1e/l29 0 2024-09-10T11:29:30.528 INFO:tasks.workunit.client.1.smithi178.stdout:4/391: creat d0/d8/da/d1a/f8b x:0 0 0 2024-09-10T11:29:30.530 INFO:tasks.workunit.client.1.smithi178.stdout:8/570: dread d5/db/f7d [0,4194304] 0 2024-09-10T11:29:30.531 INFO:tasks.workunit.client.0.smithi144.stdout:4/119: symlink d14/d1e/l2a 0 2024-09-10T11:29:30.533 INFO:tasks.workunit.client.0.smithi144.stdout:4/120: chown d14/d1e/d22/f23 14 1 2024-09-10T11:29:30.533 INFO:tasks.workunit.client.1.smithi178.stdout:9/578: truncate d2/df/f3b 551683 0 2024-09-10T11:29:30.535 INFO:tasks.workunit.client.1.smithi178.stdout:0/558: rename d0/dc/d1f/d2d/c62 to d0/dc/d39/d4c/d43/cb3 0 2024-09-10T11:29:30.541 INFO:tasks.workunit.client.0.smithi144.stdout:4/121: dwrite f12 [0,4194304] 0 2024-09-10T11:29:30.543 INFO:tasks.workunit.client.1.smithi178.stdout:6/551: link d3/d74/dc9/fb8 d3/d74/d91/daa/dc2/d57/fcb 0 2024-09-10T11:29:30.545 INFO:tasks.workunit.client.1.smithi178.stdout:2/694: dread d1/d7/d24/d46/d88/d9d/dab/f17 [0,4194304] 0 2024-09-10T11:29:30.546 INFO:tasks.workunit.client.1.smithi178.stdout:8/571: creat d5/db/d3d/fbe x:0 0 0 2024-09-10T11:29:30.546 INFO:tasks.workunit.client.1.smithi178.stdout:2/695: readlink d1/d7/d24/l91 0 2024-09-10T11:29:30.548 INFO:tasks.workunit.client.1.smithi178.stdout:8/572: read d5/db/f14 [168916,67782] 0 2024-09-10T11:29:30.548 INFO:tasks.workunit.client.1.smithi178.stdout:5/545: unlink d9/dc/dd/d3f/c58 0 2024-09-10T11:29:30.550 INFO:tasks.workunit.client.1.smithi178.stdout:5/546: chown d9/dc/d17/l24 282 1 2024-09-10T11:29:30.554 INFO:tasks.workunit.client.0.smithi144.stdout:9/67: dwrite d11/f17 [4194304,4194304] 0 2024-09-10T11:29:30.554 INFO:tasks.workunit.client.1.smithi178.stdout:7/436: rename d6/d10/d1e/d24/d49/d70/d4c to d6/d10/d1e/d2d/d2b/da0 0 2024-09-10T11:29:30.555 INFO:tasks.workunit.client.0.smithi144.stdout:4/122: mkdir d14/d1e/d22/d2b 0 2024-09-10T11:29:30.557 INFO:tasks.workunit.client.1.smithi178.stdout:6/552: mknod d3/d7/ccc 0 2024-09-10T11:29:30.557 INFO:tasks.workunit.client.1.smithi178.stdout:0/559: dwrite d0/dc/d39/d50/f25 [0,4194304] 0 2024-09-10T11:29:30.558 INFO:tasks.workunit.client.0.smithi144.stdout:9/68: rmdir d11 39 2024-09-10T11:29:30.560 INFO:tasks.workunit.client.1.smithi178.stdout:2/696: symlink d1/d7/d24/d46/ldd 0 2024-09-10T11:29:30.560 INFO:tasks.workunit.client.0.smithi144.stdout:4/123: mknod d14/c2c 0 2024-09-10T11:29:30.561 INFO:tasks.workunit.client.0.smithi144.stdout:4/124: write f5 [3592770,15750] 0 2024-09-10T11:29:30.561 INFO:tasks.workunit.client.0.smithi144.stdout:9/69: mknod d11/d12/c19 0 2024-09-10T11:29:30.562 INFO:tasks.workunit.client.0.smithi144.stdout:4/125: chown f9 251846 1 2024-09-10T11:29:30.563 INFO:tasks.workunit.client.1.smithi178.stdout:8/573: unlink d5/d30/d33/d75/laf 0 2024-09-10T11:29:30.565 INFO:tasks.workunit.client.0.smithi144.stdout:9/70: write d11/d12/f14 [6816122,36772] 0 2024-09-10T11:29:30.566 INFO:tasks.workunit.client.1.smithi178.stdout:9/579: creat d2/df/d31/d37/d3d/faf x:0 0 0 2024-09-10T11:29:30.568 INFO:tasks.workunit.client.1.smithi178.stdout:7/437: mkdir d6/d10/d1d/d22/d3b/da1 0 2024-09-10T11:29:30.568 INFO:tasks.workunit.client.1.smithi178.stdout:4/392: creat d0/d9/dc/f8c x:0 0 0 2024-09-10T11:29:30.570 INFO:tasks.workunit.client.1.smithi178.stdout:2/697: symlink d1/d7/lde 0 2024-09-10T11:29:30.570 INFO:tasks.workunit.client.0.smithi144.stdout:9/71: rename d11/d12/l13 to d11/l1a 0 2024-09-10T11:29:30.571 INFO:tasks.workunit.client.0.smithi144.stdout:9/72: write d11/f17 [5470685,55483] 0 2024-09-10T11:29:30.573 INFO:tasks.workunit.client.0.smithi144.stdout:9/73: write d11/f17 [6305428,18064] 0 2024-09-10T11:29:30.573 INFO:tasks.workunit.client.1.smithi178.stdout:5/547: mkdir d9/dc/dd/d37/dbd 0 2024-09-10T11:29:30.576 INFO:tasks.workunit.client.1.smithi178.stdout:4/393: mknod d0/d8/d3e/c8d 0 2024-09-10T11:29:30.576 INFO:tasks.workunit.client.0.smithi144.stdout:9/74: mkdir d11/d12/d1b 0 2024-09-10T11:29:30.576 INFO:tasks.workunit.client.1.smithi178.stdout:8/574: creat d5/db7/fbf x:0 0 0 2024-09-10T11:29:30.578 INFO:tasks.workunit.client.1.smithi178.stdout:3/477: truncate db/d35/f5b 1596506 0 2024-09-10T11:29:30.578 INFO:tasks.workunit.client.1.smithi178.stdout:3/478: fsync db/d35/d58/f6e 0 2024-09-10T11:29:30.581 INFO:tasks.workunit.client.1.smithi178.stdout:3/479: chown db/d26/d40/c54 450884 1 2024-09-10T11:29:30.585 INFO:tasks.workunit.client.0.smithi144.stdout:9/75: truncate fc 5292322 0 2024-09-10T11:29:30.586 INFO:tasks.workunit.client.1.smithi178.stdout:8/575: read d5/db/d3d/f5f [1046254,28286] 0 2024-09-10T11:29:30.587 INFO:tasks.workunit.client.1.smithi178.stdout:0/560: getdents d0/d2c/d89 0 2024-09-10T11:29:30.588 INFO:tasks.workunit.client.0.smithi144.stdout:9/76: creat d11/f1c x:0 0 0 2024-09-10T11:29:30.588 INFO:tasks.workunit.client.1.smithi178.stdout:6/553: dread d3/d7/d8/d15/d1c/f53 [0,4194304] 0 2024-09-10T11:29:30.588 INFO:tasks.workunit.client.1.smithi178.stdout:4/394: dwrite d0/f38 [4194304,4194304] 0 2024-09-10T11:29:30.588 INFO:tasks.workunit.client.1.smithi178.stdout:9/580: link d2/d19/d1c/d2a/f6d d2/d13/d40/d89/d93/d88/fb0 0 2024-09-10T11:29:30.591 INFO:tasks.workunit.client.1.smithi178.stdout:7/438: getdents d6/d10/d1e/d24/d49/d7c 0 2024-09-10T11:29:30.594 INFO:tasks.workunit.client.1.smithi178.stdout:2/698: dread d1/d7/d24/d7d/d84/f35 [0,4194304] 0 2024-09-10T11:29:30.594 INFO:tasks.workunit.client.1.smithi178.stdout:0/561: creat d0/dc/d39/d4c/fb4 x:0 0 0 2024-09-10T11:29:30.595 INFO:tasks.workunit.client.1.smithi178.stdout:7/439: stat d6/d10/d1e/d24/d49 0 2024-09-10T11:29:30.597 INFO:tasks.workunit.client.0.smithi144.stdout:9/77: dread fc [0,4194304] 0 2024-09-10T11:29:30.598 INFO:tasks.workunit.client.1.smithi178.stdout:2/699: fdatasync d1/d7/d24/d7d/d84/fb6 0 2024-09-10T11:29:30.598 INFO:tasks.workunit.client.1.smithi178.stdout:9/581: creat d2/d19/d1c/d2d/dab/fb1 x:0 0 0 2024-09-10T11:29:30.600 INFO:tasks.workunit.client.1.smithi178.stdout:6/554: dwrite d3/d7/d8/d19/f8b [0,4194304] 0 2024-09-10T11:29:30.601 INFO:tasks.workunit.client.1.smithi178.stdout:4/395: mkdir d0/d9/dc/d8e 0 2024-09-10T11:29:30.602 INFO:tasks.workunit.client.0.smithi144.stdout:9/78: unlink f0 0 2024-09-10T11:29:30.602 INFO:tasks.workunit.client.0.smithi144.stdout:9/79: chown ca 1 1 2024-09-10T11:29:30.602 INFO:tasks.workunit.client.0.smithi144.stdout:9/80: read - d11/f15 zero size 2024-09-10T11:29:30.603 INFO:tasks.workunit.client.0.smithi144.stdout:9/81: readlink ld 0 2024-09-10T11:29:30.603 INFO:tasks.workunit.client.0.smithi144.stdout:9/82: chown cb 39909 1 2024-09-10T11:29:30.606 INFO:tasks.workunit.client.1.smithi178.stdout:8/576: rmdir d5/d30/d48 0 2024-09-10T11:29:30.607 INFO:tasks.workunit.client.1.smithi178.stdout:6/555: write d3/d7/d8/d15/d1c/d2a/f41 [4149249,62679] 0 2024-09-10T11:29:30.609 INFO:tasks.workunit.client.1.smithi178.stdout:0/562: unlink d0/d2c/d33/caa 0 2024-09-10T11:29:30.609 INFO:tasks.workunit.client.1.smithi178.stdout:8/577: readlink d5/db/d1e/l31 0 2024-09-10T11:29:30.609 INFO:tasks.workunit.client.1.smithi178.stdout:7/440: mkdir d6/d10/d1e/d24/d49/d70/da2 0 2024-09-10T11:29:30.612 INFO:tasks.workunit.client.0.smithi144.stdout:9/83: dwrite d11/f17 [0,4194304] 0 2024-09-10T11:29:30.614 INFO:tasks.workunit.client.1.smithi178.stdout:4/396: dread d0/d9/d3d/f44 [0,4194304] 0 2024-09-10T11:29:30.616 INFO:tasks.workunit.client.1.smithi178.stdout:4/397: write d0/f38 [8075425,8234] 0 2024-09-10T11:29:30.616 INFO:tasks.workunit.client.0.smithi144.stdout:9/84: dread fc [0,4194304] 0 2024-09-10T11:29:30.617 INFO:tasks.workunit.client.1.smithi178.stdout:6/556: dwrite d3/d74/dc9/f47 [8388608,4194304] 0 2024-09-10T11:29:30.619 INFO:tasks.workunit.client.1.smithi178.stdout:6/557: write d3/d7/d8/d15/d1c/d2a/fc0 [551900,116162] 0 2024-09-10T11:29:30.620 INFO:tasks.workunit.client.0.smithi144.stdout:9/85: creat d11/d12/f1d x:0 0 0 2024-09-10T11:29:30.621 INFO:tasks.workunit.client.0.smithi144.stdout:9/86: dread - d11/d12/f1d zero size 2024-09-10T11:29:30.626 INFO:tasks.workunit.client.0.smithi144.stdout:3/83: sync 2024-09-10T11:29:30.627 INFO:tasks.workunit.client.1.smithi178.stdout:9/582: mknod d2/df/d31/d4a/cb2 0 2024-09-10T11:29:30.629 INFO:tasks.workunit.client.1.smithi178.stdout:2/700: getdents d1/d41/d5d 0 2024-09-10T11:29:30.630 INFO:tasks.workunit.client.0.smithi144.stdout:9/87: dwrite d11/f17 [0,4194304] 0 2024-09-10T11:29:30.631 INFO:tasks.workunit.client.1.smithi178.stdout:0/563: creat d0/dc/d1f/d2d/fb5 x:0 0 0 2024-09-10T11:29:30.632 INFO:tasks.workunit.client.1.smithi178.stdout:7/441: mknod d6/d10/d1e/d24/ca3 0 2024-09-10T11:29:30.635 INFO:tasks.workunit.client.1.smithi178.stdout:4/398: mknod d0/d8/da/d1a/d51/d5d/c8f 0 2024-09-10T11:29:30.637 INFO:tasks.workunit.client.1.smithi178.stdout:6/558: symlink d3/d74/lcd 0 2024-09-10T11:29:30.637 INFO:tasks.workunit.client.0.smithi144.stdout:3/84: dwrite d4/fb [0,4194304] 0 2024-09-10T11:29:30.637 INFO:tasks.workunit.client.0.smithi144.stdout:9/88: rename d11/d12/c18 to d11/d12/c1e 0 2024-09-10T11:29:30.638 INFO:tasks.workunit.client.0.smithi144.stdout:9/89: chown d11/f17 129755 1 2024-09-10T11:29:30.639 INFO:tasks.workunit.client.1.smithi178.stdout:6/559: write d3/d7/d8/d15/d1c/d2a/f41 [3606570,26989] 0 2024-09-10T11:29:30.640 INFO:tasks.workunit.client.1.smithi178.stdout:6/560: chown d3/d74/dc9/d4b 29726945 1 2024-09-10T11:29:30.640 INFO:tasks.workunit.client.0.smithi144.stdout:3/85: chown d4/l6 6181 1 2024-09-10T11:29:30.641 INFO:tasks.workunit.client.0.smithi144.stdout:3/86: stat d4/da 0 2024-09-10T11:29:30.641 INFO:tasks.workunit.client.1.smithi178.stdout:2/701: unlink d1/d41/d5d/ld4 0 2024-09-10T11:29:30.643 INFO:tasks.workunit.client.1.smithi178.stdout:8/578: mkdir d5/db/d10/d20/dc0 0 2024-09-10T11:29:30.644 INFO:tasks.workunit.client.0.smithi144.stdout:3/87: creat d4/dd/f20 x:0 0 0 2024-09-10T11:29:30.647 INFO:tasks.workunit.client.0.smithi144.stdout:9/90: truncate fc 3001260 0 2024-09-10T11:29:30.648 INFO:tasks.workunit.client.0.smithi144.stdout:3/88: dread d4/fb [0,4194304] 0 2024-09-10T11:29:30.649 INFO:tasks.workunit.client.0.smithi144.stdout:9/91: creat d11/f1f x:0 0 0 2024-09-10T11:29:30.651 INFO:tasks.workunit.client.0.smithi144.stdout:3/89: getdents d4/da/d17/d18 0 2024-09-10T11:29:30.651 INFO:tasks.workunit.client.1.smithi178.stdout:6/561: symlink d3/d7/d35/lce 0 2024-09-10T11:29:30.652 INFO:tasks.workunit.client.1.smithi178.stdout:6/562: chown d3/d7/d8/f58 35 1 2024-09-10T11:29:30.652 INFO:tasks.workunit.client.0.smithi144.stdout:9/92: rename d11/d12/f1d to d11/d12/d1b/f20 0 2024-09-10T11:29:30.653 INFO:tasks.workunit.client.1.smithi178.stdout:8/579: creat d5/db/d10/d53/d82/fc1 x:0 0 0 2024-09-10T11:29:30.653 INFO:tasks.workunit.client.0.smithi144.stdout:3/90: mkdir d4/dd/d21 0 2024-09-10T11:29:30.654 INFO:tasks.workunit.client.1.smithi178.stdout:4/399: getdents d0/d9/dc/d8a 0 2024-09-10T11:29:30.655 INFO:tasks.workunit.client.0.smithi144.stdout:9/93: creat d11/f21 x:0 0 0 2024-09-10T11:29:30.655 INFO:tasks.workunit.client.0.smithi144.stdout:9/94: dread - d11/f1c zero size 2024-09-10T11:29:30.658 INFO:tasks.workunit.client.0.smithi144.stdout:3/91: dread d4/da/d17/f1f [0,4194304] 0 2024-09-10T11:29:30.659 INFO:tasks.workunit.client.0.smithi144.stdout:3/92: stat d4/c1c 0 2024-09-10T11:29:30.660 INFO:tasks.workunit.client.1.smithi178.stdout:8/580: dwrite d5/d1d/d22/f36 [0,4194304] 0 2024-09-10T11:29:30.660 INFO:tasks.workunit.client.1.smithi178.stdout:6/563: rename d3/d74/d91/daa/dc2/d57/d78/dc3 to d3/d74/d91/daa/dc2/dcf 0 2024-09-10T11:29:30.661 INFO:tasks.workunit.client.0.smithi144.stdout:3/93: write f3 [619018,84058] 0 2024-09-10T11:29:30.662 INFO:tasks.workunit.client.1.smithi178.stdout:4/400: creat d0/d8/da/f90 x:0 0 0 2024-09-10T11:29:30.662 INFO:tasks.workunit.client.1.smithi178.stdout:8/581: fsync d5/db/d1e/fa9 0 2024-09-10T11:29:30.663 INFO:tasks.workunit.client.0.smithi144.stdout:9/95: dwrite d11/f1f [0,4194304] 0 2024-09-10T11:29:30.663 INFO:tasks.workunit.client.1.smithi178.stdout:2/702: creat d1/d41/fdf x:0 0 0 2024-09-10T11:29:30.665 INFO:tasks.workunit.client.1.smithi178.stdout:8/582: read d5/d1d/f25 [467312,71223] 0 2024-09-10T11:29:30.666 INFO:tasks.workunit.client.1.smithi178.stdout:6/564: symlink d3/d74/dc9/d4b/ld0 0 2024-09-10T11:29:30.668 INFO:tasks.workunit.client.0.smithi144.stdout:9/96: mkdir d11/d12/d1b/d22 0 2024-09-10T11:29:30.672 INFO:tasks.workunit.client.1.smithi178.stdout:6/565: unlink d3/d7/d35/c76 0 2024-09-10T11:29:30.673 INFO:tasks.workunit.client.0.smithi144.stdout:3/94: dwrite f2 [0,4194304] 0 2024-09-10T11:29:30.674 INFO:tasks.workunit.client.1.smithi178.stdout:4/401: dwrite d0/d8/f56 [0,4194304] 0 2024-09-10T11:29:30.675 INFO:tasks.workunit.client.1.smithi178.stdout:2/703: link d1/d22/ca5 d1/d7/d24/d46/ce0 0 2024-09-10T11:29:30.678 INFO:tasks.workunit.client.1.smithi178.stdout:6/566: mknod d3/d6e/d87/cd1 0 2024-09-10T11:29:30.678 INFO:tasks.workunit.client.0.smithi144.stdout:3/95: rename d4/da to d4/dd/d22 0 2024-09-10T11:29:30.679 INFO:tasks.workunit.client.1.smithi178.stdout:9/583: write d2/f18 [2214362,39608] 0 2024-09-10T11:29:30.681 INFO:tasks.workunit.client.1.smithi178.stdout:2/704: chown d1/d41/d8d/cc4 1768 1 2024-09-10T11:29:30.682 INFO:tasks.workunit.client.1.smithi178.stdout:4/402: rename d0/d8/c86 to d0/d8/da/c91 0 2024-09-10T11:29:30.687 INFO:tasks.workunit.client.1.smithi178.stdout:4/403: chown d0/f40 7508 1 2024-09-10T11:29:30.687 INFO:tasks.workunit.client.1.smithi178.stdout:9/584: chown d2/df/d31/d37/f5f 5838151 1 2024-09-10T11:29:30.688 INFO:tasks.workunit.client.0.smithi144.stdout:9/97: truncate d11/d12/f14 7176150 0 2024-09-10T11:29:30.690 INFO:tasks.workunit.client.1.smithi178.stdout:8/583: dread d5/f13 [0,4194304] 0 2024-09-10T11:29:30.691 INFO:tasks.workunit.client.0.smithi144.stdout:9/98: dread d11/f17 [0,4194304] 0 2024-09-10T11:29:30.691 INFO:tasks.workunit.client.1.smithi178.stdout:0/564: dwrite d0/f5a [0,4194304] 0 2024-09-10T11:29:30.692 INFO:tasks.workunit.client.1.smithi178.stdout:7/442: dwrite d6/d10/d1d/d22/d50/d7e/d82/f92 [0,4194304] 0 2024-09-10T11:29:30.692 INFO:tasks.workunit.client.0.smithi144.stdout:9/99: write d11/f1f [3370988,100966] 0 2024-09-10T11:29:30.693 INFO:tasks.workunit.client.0.smithi144.stdout:9/100: stat d11/f1c 0 2024-09-10T11:29:30.693 INFO:tasks.workunit.client.1.smithi178.stdout:8/584: dread - d5/d61/fa2 zero size 2024-09-10T11:29:30.695 INFO:tasks.workunit.client.1.smithi178.stdout:8/585: stat d5/db/d10/d20/d52/d5c/f7f 0 2024-09-10T11:29:30.695 INFO:tasks.workunit.client.1.smithi178.stdout:0/565: chown d0/dc/d39/d7c/d85 35942578 1 2024-09-10T11:29:30.695 INFO:tasks.workunit.client.1.smithi178.stdout:7/443: stat d6/d10/d1e/f8e 0 2024-09-10T11:29:30.696 INFO:tasks.workunit.client.0.smithi144.stdout:9/101: symlink d11/d12/d1b/l23 0 2024-09-10T11:29:30.700 INFO:tasks.workunit.client.1.smithi178.stdout:7/444: dread - d6/d10/d1d/d22/d50/d7e/d82/f9f zero size 2024-09-10T11:29:30.700 INFO:tasks.workunit.client.1.smithi178.stdout:8/586: symlink d5/d30/d33/lc2 0 2024-09-10T11:29:30.702 INFO:tasks.workunit.client.1.smithi178.stdout:8/587: truncate d5/d30/db8/fbd 625391 0 2024-09-10T11:29:30.702 INFO:tasks.workunit.client.0.smithi144.stdout:9/102: dwrite f10 [0,4194304] 0 2024-09-10T11:29:30.703 INFO:tasks.workunit.client.0.smithi144.stdout:9/103: readlink d11/l16 0 2024-09-10T11:29:30.706 INFO:tasks.workunit.client.0.smithi144.stdout:9/104: creat d11/d12/d1b/f24 x:0 0 0 2024-09-10T11:29:30.706 INFO:tasks.workunit.client.1.smithi178.stdout:9/585: mknod d2/df/cb3 0 2024-09-10T11:29:30.708 INFO:tasks.workunit.client.1.smithi178.stdout:4/404: link d0/d8/df/l3a d0/d8/da/d1a/l92 0 2024-09-10T11:29:30.709 INFO:tasks.workunit.client.1.smithi178.stdout:4/405: stat d0/d8/df/l39 0 2024-09-10T11:29:30.709 INFO:tasks.workunit.client.0.smithi144.stdout:9/105: symlink d11/l25 0 2024-09-10T11:29:30.714 INFO:tasks.workunit.client.1.smithi178.stdout:4/406: creat d0/d21/f93 x:0 0 0 2024-09-10T11:29:30.715 INFO:tasks.workunit.client.1.smithi178.stdout:0/566: link d0/dc/d39/d50/cb2 d0/dc/d1f/cb6 0 2024-09-10T11:29:30.716 INFO:tasks.workunit.client.1.smithi178.stdout:0/567: chown d0/dc/d39/d50/f9f 16 1 2024-09-10T11:29:30.718 INFO:tasks.workunit.client.0.smithi144.stdout:9/106: dwrite fc [0,4194304] 0 2024-09-10T11:29:30.721 INFO:tasks.workunit.client.1.smithi178.stdout:8/588: dread d5/db/d1e/fa3 [0,4194304] 0 2024-09-10T11:29:30.721 INFO:tasks.workunit.client.1.smithi178.stdout:9/586: link d2/dc/l98 d2/d13/d40/d56/d9a/lb4 0 2024-09-10T11:29:30.722 INFO:tasks.workunit.client.1.smithi178.stdout:2/705: truncate d1/d41/d14/f82 15185 0 2024-09-10T11:29:30.727 INFO:tasks.workunit.client.0.smithi144.stdout:9/107: dwrite d11/f1c [0,4194304] 0 2024-09-10T11:29:30.728 INFO:tasks.workunit.client.0.smithi144.stdout:9/108: dread - d11/d12/d1b/f20 zero size 2024-09-10T11:29:30.731 INFO:tasks.workunit.client.1.smithi178.stdout:2/706: creat d1/d7/d24/d43/fe1 x:0 0 0 2024-09-10T11:29:30.731 INFO:tasks.workunit.client.1.smithi178.stdout:0/568: link d0/dc/d39/d4c/cab d0/dc/d1f/d2d/dac/cb7 0 2024-09-10T11:29:30.732 INFO:tasks.workunit.client.0.smithi144.stdout:9/109: unlink d11/f1f 0 2024-09-10T11:29:30.733 INFO:tasks.workunit.client.1.smithi178.stdout:8/589: symlink d5/db/d10/d20/d52/d5c/dad/lc3 0 2024-09-10T11:29:30.733 INFO:tasks.workunit.client.1.smithi178.stdout:7/445: dwrite d6/fd [0,4194304] 0 2024-09-10T11:29:30.735 INFO:tasks.workunit.client.0.smithi144.stdout:9/110: creat d11/d12/f26 x:0 0 0 2024-09-10T11:29:30.736 INFO:tasks.workunit.client.0.smithi144.stdout:9/111: chown d11/d12/d1b/l23 18468 1 2024-09-10T11:29:30.736 INFO:tasks.workunit.client.1.smithi178.stdout:9/587: mknod d2/d13/d40/d89/d93/d88/dae/cb5 0 2024-09-10T11:29:30.737 INFO:tasks.workunit.client.1.smithi178.stdout:4/407: dread d0/f1c [0,4194304] 0 2024-09-10T11:29:30.738 INFO:tasks.workunit.client.0.smithi144.stdout:9/112: creat d11/d12/d1b/f27 x:0 0 0 2024-09-10T11:29:30.738 INFO:tasks.workunit.client.0.smithi144.stdout:9/113: stat d11/f21 0 2024-09-10T11:29:30.739 INFO:tasks.workunit.client.1.smithi178.stdout:4/408: write d0/d21/f93 [942076,10980] 0 2024-09-10T11:29:30.740 INFO:tasks.workunit.client.1.smithi178.stdout:2/707: mknod d1/d7/d24/d46/d88/d9d/ce2 0 2024-09-10T11:29:30.741 INFO:tasks.workunit.client.1.smithi178.stdout:7/446: readlink d6/d10/d1d/d22/d50/l6c 0 2024-09-10T11:29:30.743 INFO:tasks.workunit.client.0.smithi144.stdout:9/114: rename le to d11/d12/d1b/d22/l28 0 2024-09-10T11:29:30.743 INFO:tasks.workunit.client.1.smithi178.stdout:2/708: fsync d1/d7/d24/f2c 0 2024-09-10T11:29:30.745 INFO:tasks.workunit.client.1.smithi178.stdout:9/588: dread d2/d13/f15 [0,4194304] 0 2024-09-10T11:29:30.746 INFO:tasks.workunit.client.1.smithi178.stdout:4/409: creat d0/f94 x:0 0 0 2024-09-10T11:29:30.746 INFO:tasks.workunit.client.1.smithi178.stdout:0/569: link d0/dc/d39/d50/c7f d0/dc/d39/d50/cb8 0 2024-09-10T11:29:30.747 INFO:tasks.workunit.client.1.smithi178.stdout:7/447: symlink d6/d10/d1e/d24/d49/d70/da2/la4 0 2024-09-10T11:29:30.747 INFO:tasks.workunit.client.0.smithi144.stdout:9/115: link lf d11/d12/d1b/d22/l29 0 2024-09-10T11:29:30.749 INFO:tasks.workunit.client.1.smithi178.stdout:4/410: fdatasync d0/d8/da/f2b 0 2024-09-10T11:29:30.751 INFO:tasks.workunit.client.0.smithi144.stdout:9/116: truncate d11/d12/f14 6020508 0 2024-09-10T11:29:30.752 INFO:tasks.workunit.client.1.smithi178.stdout:4/411: write d0/d21/f37 [2742319,29488] 0 2024-09-10T11:29:30.753 INFO:tasks.workunit.client.0.smithi144.stdout:9/117: mkdir d11/d2a 0 2024-09-10T11:29:30.753 INFO:tasks.workunit.client.0.smithi144.stdout:9/118: chown d11/d12/d1b/f24 35971717 1 2024-09-10T11:29:30.755 INFO:tasks.workunit.client.1.smithi178.stdout:4/412: fsync d0/d9/d3d/f61 0 2024-09-10T11:29:30.755 INFO:tasks.workunit.client.0.smithi144.stdout:9/119: creat d11/f2b x:0 0 0 2024-09-10T11:29:30.756 INFO:tasks.workunit.client.1.smithi178.stdout:9/589: dwrite d2/d19/d5e/f78 [0,4194304] 0 2024-09-10T11:29:30.757 INFO:tasks.workunit.client.0.smithi144.stdout:9/120: getdents d11/d2a 0 2024-09-10T11:29:30.759 INFO:tasks.workunit.client.1.smithi178.stdout:7/448: dwrite d6/d10/d1e/d2d/f66 [0,4194304] 0 2024-09-10T11:29:30.759 INFO:tasks.workunit.client.1.smithi178.stdout:9/590: readlink d2/d13/d40/d89/d93/d88/la6 0 2024-09-10T11:29:30.760 INFO:tasks.workunit.client.1.smithi178.stdout:9/591: fsync d2/df/d31/d37/d3d/f6e 0 2024-09-10T11:29:30.760 INFO:tasks.workunit.client.0.smithi144.stdout:9/121: mkdir d11/d12/d1b/d2c 0 2024-09-10T11:29:30.762 INFO:tasks.workunit.client.1.smithi178.stdout:2/709: rename d1/d7/d24/d7d/d84/d55/f94 to d1/d41/d14/d11/db9/fe3 0 2024-09-10T11:29:30.763 INFO:tasks.workunit.client.0.smithi144.stdout:9/122: mkdir d11/d2a/d2d 0 2024-09-10T11:29:30.763 INFO:tasks.workunit.client.0.smithi144.stdout:6/112: sync 2024-09-10T11:29:30.763 INFO:tasks.workunit.client.0.smithi144.stdout:0/74: sync 2024-09-10T11:29:30.763 INFO:tasks.workunit.client.0.smithi144.stdout:8/74: sync 2024-09-10T11:29:30.764 INFO:tasks.workunit.client.0.smithi144.stdout:2/90: sync 2024-09-10T11:29:30.764 INFO:tasks.workunit.client.1.smithi178.stdout:8/590: dread d5/db/d3d/f97 [0,4194304] 0 2024-09-10T11:29:30.764 INFO:tasks.workunit.client.1.smithi178.stdout:0/570: symlink d0/dc/d39/d4c/d43/d8c/lb9 0 2024-09-10T11:29:30.765 INFO:tasks.workunit.client.0.smithi144.stdout:0/75: write da/fc [3245637,78007] 0 2024-09-10T11:29:30.768 INFO:tasks.workunit.client.1.smithi178.stdout:7/449: creat d6/d10/d1d/d22/d3b/fa5 x:0 0 0 2024-09-10T11:29:30.768 INFO:tasks.workunit.client.0.smithi144.stdout:8/75: symlink d0/da/l1a 0 2024-09-10T11:29:30.773 INFO:tasks.workunit.client.0.smithi144.stdout:8/76: mkdir d0/d1b 0 2024-09-10T11:29:30.773 INFO:tasks.workunit.client.1.smithi178.stdout:0/571: mknod d0/d2c/d89/da1/cba 0 2024-09-10T11:29:30.774 INFO:tasks.workunit.client.0.smithi144.stdout:6/113: creat d9/dc/f2e x:0 0 0 2024-09-10T11:29:30.775 INFO:tasks.workunit.client.0.smithi144.stdout:6/114: truncate d9/dc/d13/f20 1828955 0 2024-09-10T11:29:30.776 INFO:tasks.workunit.client.0.smithi144.stdout:8/77: creat d0/da/f1c x:0 0 0 2024-09-10T11:29:30.777 INFO:tasks.workunit.client.0.smithi144.stdout:2/91: getdents d7/da/d1e 0 2024-09-10T11:29:30.777 INFO:tasks.workunit.client.1.smithi178.stdout:9/592: creat d2/d13/fb6 x:0 0 0 2024-09-10T11:29:30.779 INFO:tasks.workunit.client.1.smithi178.stdout:7/450: dwrite d6/d10/d1d/f73 [0,4194304] 0 2024-09-10T11:29:30.780 INFO:tasks.workunit.client.0.smithi144.stdout:8/78: creat d0/d9/f1d x:0 0 0 2024-09-10T11:29:30.780 INFO:tasks.workunit.client.1.smithi178.stdout:9/593: fsync d2/d19/d1c/d2d/f55 0 2024-09-10T11:29:30.781 INFO:tasks.workunit.client.0.smithi144.stdout:6/115: mkdir d9/dd/d11/d1b/d2f 0 2024-09-10T11:29:30.782 INFO:tasks.workunit.client.1.smithi178.stdout:4/413: rename d0/d8/da/l88 to d0/l95 0 2024-09-10T11:29:30.784 INFO:tasks.workunit.client.0.smithi144.stdout:2/92: dwrite d7/f10 [0,4194304] 0 2024-09-10T11:29:30.784 INFO:tasks.workunit.client.1.smithi178.stdout:4/414: truncate d0/f94 631833 0 2024-09-10T11:29:30.785 INFO:tasks.workunit.client.0.smithi144.stdout:2/93: chown d7/de/f16 4551073 1 2024-09-10T11:29:30.787 INFO:tasks.workunit.client.0.smithi144.stdout:8/79: rename d0/l16 to d0/l1e 0 2024-09-10T11:29:30.788 INFO:tasks.workunit.client.1.smithi178.stdout:8/591: dread d5/d30/f4b [0,4194304] 0 2024-09-10T11:29:30.788 INFO:tasks.workunit.client.1.smithi178.stdout:9/594: mknod d2/df/d31/d4a/cb7 0 2024-09-10T11:29:30.789 INFO:tasks.workunit.client.0.smithi144.stdout:9/123: fsync d11/d12/f14 0 2024-09-10T11:29:30.790 INFO:tasks.workunit.client.1.smithi178.stdout:2/710: rename d1/d7/d24/d46/d88/d9d/dab/cbe to d1/d41/d14/d11/db9/dc1/ce4 0 2024-09-10T11:29:30.791 INFO:tasks.workunit.client.0.smithi144.stdout:2/94: mknod d7/c23 0 2024-09-10T11:29:30.791 INFO:tasks.workunit.client.1.smithi178.stdout:4/415: creat d0/d8/da/f96 x:0 0 0 2024-09-10T11:29:30.792 INFO:tasks.workunit.client.1.smithi178.stdout:9/595: rmdir d2/d19/d1c 39 2024-09-10T11:29:30.793 INFO:tasks.workunit.client.1.smithi178.stdout:0/572: getdents d0/d2c/d89/da1 0 2024-09-10T11:29:30.795 INFO:tasks.workunit.client.1.smithi178.stdout:7/451: mkdir d6/d10/d1d/d22/d50/d98/da6 0 2024-09-10T11:29:30.795 INFO:tasks.workunit.client.0.smithi144.stdout:6/116: mknod d9/dc/c30 0 2024-09-10T11:29:30.795 INFO:tasks.workunit.client.1.smithi178.stdout:8/592: mkdir d5/db/d10/d70/dc4 0 2024-09-10T11:29:30.796 INFO:tasks.workunit.client.1.smithi178.stdout:4/416: rmdir d0/d9/dc 39 2024-09-10T11:29:30.797 INFO:tasks.workunit.client.1.smithi178.stdout:0/573: truncate d0/dc/f61 736552 0 2024-09-10T11:29:30.798 INFO:tasks.workunit.client.0.smithi144.stdout:9/124: mknod d11/d2a/c2e 0 2024-09-10T11:29:30.799 INFO:tasks.workunit.client.1.smithi178.stdout:2/711: symlink d1/d7/d24/d46/d88/d9d/dab/d5b/d60/le5 0 2024-09-10T11:29:30.799 INFO:tasks.workunit.client.0.smithi144.stdout:9/125: dread - d11/f15 zero size 2024-09-10T11:29:30.800 INFO:tasks.workunit.client.1.smithi178.stdout:7/452: mkdir d6/d10/d1d/d22/d50/d7e/da7 0 2024-09-10T11:29:30.800 INFO:tasks.workunit.client.0.smithi144.stdout:2/95: symlink d7/da/l24 0 2024-09-10T11:29:30.803 INFO:tasks.workunit.client.1.smithi178.stdout:4/417: creat d0/d9/d3d/d4f/f97 x:0 0 0 2024-09-10T11:29:30.803 INFO:tasks.workunit.client.0.smithi144.stdout:9/126: truncate d11/f21 747347 0 2024-09-10T11:29:30.803 INFO:tasks.workunit.client.0.smithi144.stdout:0/76: truncate f2 817204 0 2024-09-10T11:29:30.804 INFO:tasks.workunit.client.0.smithi144.stdout:6/117: dwrite d9/dd/f1d [0,4194304] 0 2024-09-10T11:29:30.805 INFO:tasks.workunit.client.0.smithi144.stdout:6/118: chown d9/dc/f28 42 1 2024-09-10T11:29:30.806 INFO:tasks.workunit.client.0.smithi144.stdout:8/80: rename d0/da to d0/d9/d1f 0 2024-09-10T11:29:30.806 INFO:tasks.workunit.client.1.smithi178.stdout:2/712: read d1/d41/d14/fb0 [2489301,63581] 0 2024-09-10T11:29:30.806 INFO:tasks.workunit.client.1.smithi178.stdout:8/593: dwrite d5/db/d3d/fbe [0,4194304] 0 2024-09-10T11:29:30.807 INFO:tasks.workunit.client.0.smithi144.stdout:8/81: stat d0/d9/d1f/f1c 0 2024-09-10T11:29:30.807 INFO:tasks.workunit.client.0.smithi144.stdout:8/82: chown d0/d9 3048 1 2024-09-10T11:29:30.810 INFO:tasks.workunit.client.1.smithi178.stdout:8/594: fdatasync d5/db/d10/d70/f80 0 2024-09-10T11:29:30.812 INFO:tasks.workunit.client.1.smithi178.stdout:7/453: dwrite d6/f14 [8388608,4194304] 0 2024-09-10T11:29:30.814 INFO:tasks.workunit.client.1.smithi178.stdout:8/595: fsync d5/db/d10/d20/f2c 0 2024-09-10T11:29:30.815 INFO:tasks.workunit.client.0.smithi144.stdout:2/96: rename d7/da/fb to d7/da/d1e/f25 0 2024-09-10T11:29:30.815 INFO:tasks.workunit.client.1.smithi178.stdout:7/454: read f3 [1620612,114446] 0 2024-09-10T11:29:30.817 INFO:tasks.workunit.client.1.smithi178.stdout:7/455: dread - d6/d10/d34/f44 zero size 2024-09-10T11:29:30.818 INFO:tasks.workunit.client.0.smithi144.stdout:6/119: rmdir d9 39 2024-09-10T11:29:30.822 INFO:tasks.workunit.client.1.smithi178.stdout:8/596: dwrite d5/db/d10/d53/f98 [0,4194304] 0 2024-09-10T11:29:30.825 INFO:tasks.workunit.client.0.smithi144.stdout:6/120: unlink f8 0 2024-09-10T11:29:30.825 INFO:tasks.workunit.client.0.smithi144.stdout:2/97: dwrite d7/da/f19 [0,4194304] 0 2024-09-10T11:29:30.828 INFO:tasks.workunit.client.0.smithi144.stdout:0/77: dread f2 [0,4194304] 0 2024-09-10T11:29:30.829 INFO:tasks.workunit.client.0.smithi144.stdout:0/78: fsync f0 0 2024-09-10T11:29:30.829 INFO:tasks.workunit.client.0.smithi144.stdout:0/79: fsync da/fc 0 2024-09-10T11:29:30.830 INFO:tasks.workunit.client.0.smithi144.stdout:0/80: chown da/ce 2 1 2024-09-10T11:29:30.831 INFO:tasks.workunit.client.1.smithi178.stdout:8/597: dwrite d5/db/d10/d35/f44 [0,4194304] 0 2024-09-10T11:29:30.831 INFO:tasks.workunit.client.0.smithi144.stdout:8/83: getdents d0 0 2024-09-10T11:29:30.840 INFO:tasks.workunit.client.0.smithi144.stdout:2/98: link d7/da/f12 d7/f26 0 2024-09-10T11:29:30.842 INFO:tasks.workunit.client.0.smithi144.stdout:8/84: mkdir d0/d1b/d20 0 2024-09-10T11:29:30.843 INFO:tasks.workunit.client.0.smithi144.stdout:2/99: mknod d7/c27 0 2024-09-10T11:29:30.846 INFO:tasks.workunit.client.0.smithi144.stdout:2/100: creat d7/de/f28 x:0 0 0 2024-09-10T11:29:30.846 INFO:tasks.workunit.client.0.smithi144.stdout:0/81: link da/c10 da/c11 0 2024-09-10T11:29:30.848 INFO:tasks.workunit.client.0.smithi144.stdout:2/101: mkdir d7/d29 0 2024-09-10T11:29:30.849 INFO:tasks.workunit.client.0.smithi144.stdout:0/82: unlink da/fc 0 2024-09-10T11:29:30.851 INFO:tasks.workunit.client.0.smithi144.stdout:8/85: dwrite d0/d9/d1f/f19 [0,4194304] 0 2024-09-10T11:29:30.851 INFO:tasks.workunit.client.0.smithi144.stdout:8/86: readlink d0/lb 0 2024-09-10T11:29:30.851 INFO:tasks.workunit.client.0.smithi144.stdout:2/102: dread d7/da/f18 [0,4194304] 0 2024-09-10T11:29:30.854 INFO:tasks.workunit.client.0.smithi144.stdout:8/87: truncate d0/f10 972339 0 2024-09-10T11:29:30.858 INFO:tasks.workunit.client.0.smithi144.stdout:8/88: rmdir d0/d1b/d20 0 2024-09-10T11:29:30.859 INFO:tasks.workunit.client.0.smithi144.stdout:8/89: rmdir d0/d9/d1f 39 2024-09-10T11:29:30.861 INFO:tasks.workunit.client.1.smithi178.stdout:1/463: sync 2024-09-10T11:29:30.862 INFO:tasks.workunit.client.0.smithi144.stdout:8/90: mkdir d0/d9/d21 0 2024-09-10T11:29:30.866 INFO:tasks.workunit.client.0.smithi144.stdout:2/103: write d7/f11 [566193,97583] 0 2024-09-10T11:29:30.867 INFO:tasks.workunit.client.0.smithi144.stdout:8/91: dread d0/f13 [0,4194304] 0 2024-09-10T11:29:30.867 INFO:tasks.workunit.client.0.smithi144.stdout:0/83: truncate f5 2368874 0 2024-09-10T11:29:30.869 INFO:tasks.workunit.client.0.smithi144.stdout:0/84: mknod da/c12 0 2024-09-10T11:29:30.869 INFO:tasks.workunit.client.0.smithi144.stdout:2/104: dread d7/da/fd [0,4194304] 0 2024-09-10T11:29:30.869 INFO:tasks.workunit.client.0.smithi144.stdout:8/92: mknod d0/d9/d21/c22 0 2024-09-10T11:29:30.870 INFO:tasks.workunit.client.0.smithi144.stdout:0/85: creat da/f13 x:0 0 0 2024-09-10T11:29:30.872 INFO:tasks.workunit.client.0.smithi144.stdout:2/105: link d7/da/fd d7/da/d1e/d21/f2a 0 2024-09-10T11:29:30.873 INFO:tasks.workunit.client.0.smithi144.stdout:2/106: stat d7/da/d1e 0 2024-09-10T11:29:30.876 INFO:tasks.workunit.client.0.smithi144.stdout:8/93: dwrite d0/d9/fe [0,4194304] 0 2024-09-10T11:29:30.877 INFO:tasks.workunit.client.0.smithi144.stdout:2/107: dread d7/da/f17 [0,4194304] 0 2024-09-10T11:29:30.878 INFO:tasks.workunit.client.0.smithi144.stdout:0/86: getdents da 0 2024-09-10T11:29:30.878 INFO:tasks.workunit.client.0.smithi144.stdout:2/108: fsync f6 0 2024-09-10T11:29:30.880 INFO:tasks.workunit.client.0.smithi144.stdout:8/94: write d0/d9/d1f/f1c [347894,111749] 0 2024-09-10T11:29:30.887 INFO:tasks.workunit.client.0.smithi144.stdout:0/87: symlink da/l14 0 2024-09-10T11:29:30.887 INFO:tasks.workunit.client.0.smithi144.stdout:2/109: rename d7/de/f20 to d7/de/f2b 0 2024-09-10T11:29:30.887 INFO:tasks.workunit.client.0.smithi144.stdout:2/110: chown d7/da/f12 721688 1 2024-09-10T11:29:30.887 INFO:tasks.workunit.client.0.smithi144.stdout:0/88: rmdir da 39 2024-09-10T11:29:30.887 INFO:tasks.workunit.client.0.smithi144.stdout:8/95: mkdir d0/d9/d23 0 2024-09-10T11:29:30.889 INFO:tasks.workunit.client.0.smithi144.stdout:8/96: rename d0/d9/d1f/f17 to d0/f24 0 2024-09-10T11:29:30.891 INFO:tasks.workunit.client.0.smithi144.stdout:0/89: rename da/f13 to da/f15 0 2024-09-10T11:29:30.891 INFO:tasks.workunit.client.0.smithi144.stdout:8/97: dread d0/f10 [0,4194304] 0 2024-09-10T11:29:30.892 INFO:tasks.workunit.client.1.smithi178.stdout:0/574: rename d0/d2c/d89/da1/fb0 to d0/dc/d39/d7c/fbb 0 2024-09-10T11:29:30.894 INFO:tasks.workunit.client.0.smithi144.stdout:0/90: creat da/f16 x:0 0 0 2024-09-10T11:29:30.895 INFO:tasks.workunit.client.0.smithi144.stdout:8/98: mkdir d0/d9/d21/d25 0 2024-09-10T11:29:30.897 INFO:tasks.workunit.client.0.smithi144.stdout:0/91: write da/f15 [172081,94817] 0 2024-09-10T11:29:30.898 INFO:tasks.workunit.client.0.smithi144.stdout:8/99: mkdir d0/d9/d23/d26 0 2024-09-10T11:29:30.898 INFO:tasks.workunit.client.1.smithi178.stdout:4/418: symlink d0/d21/l98 0 2024-09-10T11:29:30.898 INFO:tasks.workunit.client.0.smithi144.stdout:8/100: chown d0/d9/d1f 29877 1 2024-09-10T11:29:30.899 INFO:tasks.workunit.client.1.smithi178.stdout:2/713: mknod d1/d7/d24/d46/d88/d9d/ce6 0 2024-09-10T11:29:30.900 INFO:tasks.workunit.client.0.smithi144.stdout:8/101: chown d0/d9/f11 18 1 2024-09-10T11:29:30.901 INFO:tasks.workunit.client.0.smithi144.stdout:2/111: write d7/da/fd [581016,92741] 0 2024-09-10T11:29:30.902 INFO:tasks.workunit.client.0.smithi144.stdout:2/112: fsync d7/da/ff 0 2024-09-10T11:29:30.904 INFO:tasks.workunit.client.1.smithi178.stdout:0/575: rmdir d0/d2c/d89 39 2024-09-10T11:29:30.904 INFO:tasks.workunit.client.1.smithi178.stdout:9/596: dwrite d2/d13/d75/f5d [0,4194304] 0 2024-09-10T11:29:30.905 INFO:tasks.workunit.client.0.smithi144.stdout:0/92: dwrite f0 [0,4194304] 0 2024-09-10T11:29:30.905 INFO:tasks.workunit.client.0.smithi144.stdout:2/113: mknod d7/da/d1e/c2c 0 2024-09-10T11:29:30.906 INFO:tasks.workunit.client.0.smithi144.stdout:2/114: stat d7/da/d1e/c2c 0 2024-09-10T11:29:30.906 INFO:tasks.workunit.client.0.smithi144.stdout:2/115: chown d7/de/f16 26 1 2024-09-10T11:29:30.907 INFO:tasks.workunit.client.0.smithi144.stdout:0/93: rmdir da 39 2024-09-10T11:29:30.908 INFO:tasks.workunit.client.1.smithi178.stdout:7/456: creat d6/d10/d1d/d22/d93/fa8 x:0 0 0 2024-09-10T11:29:30.908 INFO:tasks.workunit.client.1.smithi178.stdout:4/419: readlink d0/d8/df/l3a 0 2024-09-10T11:29:30.908 INFO:tasks.workunit.client.1.smithi178.stdout:2/714: unlink d1/d41/d5d/f76 0 2024-09-10T11:29:30.910 INFO:tasks.workunit.client.1.smithi178.stdout:8/598: mknod d5/db/d10/d70/dc4/cc5 0 2024-09-10T11:29:30.913 INFO:tasks.workunit.client.0.smithi144.stdout:2/116: readlink d7/da/d1e/l1f 0 2024-09-10T11:29:30.914 INFO:tasks.workunit.client.0.smithi144.stdout:2/117: write d7/de/f16 [692080,75667] 0 2024-09-10T11:29:30.916 INFO:tasks.workunit.client.1.smithi178.stdout:1/464: dwrite d3/d9/d21/d4b/d2b/d5f/f85 [0,4194304] 0 2024-09-10T11:29:30.917 INFO:tasks.workunit.client.1.smithi178.stdout:7/457: dread - d6/d10/d1d/d22/d50/d7e/d82/f9f zero size 2024-09-10T11:29:30.919 INFO:tasks.workunit.client.0.smithi144.stdout:0/94: symlink da/l17 0 2024-09-10T11:29:30.920 INFO:tasks.workunit.client.0.smithi144.stdout:8/102: dwrite d0/f13 [0,4194304] 0 2024-09-10T11:29:30.920 INFO:tasks.workunit.client.0.smithi144.stdout:0/95: chown da/cd 39 1 2024-09-10T11:29:30.921 INFO:tasks.workunit.client.1.smithi178.stdout:0/576: creat d0/dc/d1f/dae/fbc x:0 0 0 2024-09-10T11:29:30.921 INFO:tasks.workunit.client.0.smithi144.stdout:2/118: unlink d7/da/ff 0 2024-09-10T11:29:30.921 INFO:tasks.workunit.client.0.smithi144.stdout:0/96: readlink da/l14 0 2024-09-10T11:29:30.923 INFO:tasks.workunit.client.1.smithi178.stdout:8/599: symlink d5/d61/lc6 0 2024-09-10T11:29:30.924 INFO:tasks.workunit.client.1.smithi178.stdout:2/715: chown d1/d7/d24/d46/d88/d9d/dab/d5b/d7c/fa7 21193 1 2024-09-10T11:29:30.925 INFO:tasks.workunit.client.1.smithi178.stdout:9/597: symlink d2/d19/d1c/da5/lb8 0 2024-09-10T11:29:30.926 INFO:tasks.workunit.client.1.smithi178.stdout:4/420: dwrite d0/d21/f93 [0,4194304] 0 2024-09-10T11:29:30.930 INFO:tasks.workunit.client.0.smithi144.stdout:2/119: dwrite d7/f10 [0,4194304] 0 2024-09-10T11:29:30.930 INFO:tasks.workunit.client.0.smithi144.stdout:0/97: dwrite f0 [0,4194304] 0 2024-09-10T11:29:30.932 INFO:tasks.workunit.client.1.smithi178.stdout:5/548: sync 2024-09-10T11:29:30.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:30 smithi144 ceph-mon[27942]: pgmap v22: 65 pgs: 65 active+clean; 1.1 GiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 29 MiB/s rd, 155 MiB/s wr, 426 op/s 2024-09-10T11:29:30.934 INFO:tasks.workunit.client.1.smithi178.stdout:1/465: getdents d3/d35 0 2024-09-10T11:29:30.934 INFO:tasks.workunit.client.0.smithi144.stdout:2/120: mknod d7/da/d1e/c2d 0 2024-09-10T11:29:30.934 INFO:tasks.workunit.client.0.smithi144.stdout:0/98: dread da/f15 [0,4194304] 0 2024-09-10T11:29:30.935 INFO:tasks.workunit.client.1.smithi178.stdout:2/716: truncate d1/d22/d4d/d78/fd9 687153 0 2024-09-10T11:29:30.935 INFO:tasks.workunit.client.1.smithi178.stdout:8/600: rename d5/d30/d33/c3e to d5/db/d3d/cc7 0 2024-09-10T11:29:30.936 INFO:tasks.workunit.client.1.smithi178.stdout:1/466: chown d3/d9/d98/dac/d2a/c91 5 1 2024-09-10T11:29:30.937 INFO:tasks.workunit.client.1.smithi178.stdout:9/598: creat d2/d13/d75/d60/d66/fb9 x:0 0 0 2024-09-10T11:29:30.938 INFO:tasks.workunit.client.0.smithi144.stdout:0/99: fsync f2 0 2024-09-10T11:29:30.939 INFO:tasks.workunit.client.0.smithi144.stdout:0/100: read f2 [764802,13415] 0 2024-09-10T11:29:30.940 INFO:tasks.workunit.client.1.smithi178.stdout:4/421: mknod d0/d9/d3d/d4f/c99 0 2024-09-10T11:29:30.940 INFO:tasks.workunit.client.0.smithi144.stdout:0/101: stat da/c12 0 2024-09-10T11:29:30.940 INFO:tasks.workunit.client.0.smithi144.stdout:0/102: stat c6 0 2024-09-10T11:29:30.942 INFO:tasks.workunit.client.1.smithi178.stdout:1/467: readlink d3/d9/lad 0 2024-09-10T11:29:30.944 INFO:tasks.workunit.client.1.smithi178.stdout:7/458: link d6/d10/d1d/d22/d50/d7e/f8f d6/d10/d1e/d24/d49/d7c/fa9 0 2024-09-10T11:29:30.946 INFO:tasks.workunit.client.1.smithi178.stdout:8/601: creat d5/db7/fc8 x:0 0 0 2024-09-10T11:29:30.947 INFO:tasks.workunit.client.1.smithi178.stdout:7/459: write d6/d10/d1d/d22/d3b/f6d [369531,35096] 0 2024-09-10T11:29:30.948 INFO:tasks.workunit.client.1.smithi178.stdout:4/422: rmdir d0/d8/da/d1a/d51/d5d/d62 39 2024-09-10T11:29:30.948 INFO:tasks.workunit.client.1.smithi178.stdout:7/460: dread - d6/d10/d34/f6e zero size 2024-09-10T11:29:30.950 INFO:tasks.workunit.client.0.smithi144.stdout:0/103: dwrite da/f16 [0,4194304] 0 2024-09-10T11:29:30.950 INFO:tasks.workunit.client.1.smithi178.stdout:2/717: creat d1/d7/d24/d46/d88/d9d/dab/dbb/fe7 x:0 0 0 2024-09-10T11:29:30.951 INFO:tasks.workunit.client.1.smithi178.stdout:9/599: symlink d2/d13/lba 0 2024-09-10T11:29:30.951 INFO:tasks.workunit.client.1.smithi178.stdout:1/468: read d3/d9/d98/dac/d3a/d63/d7e/f8c [54664,42056] 0 2024-09-10T11:29:30.952 INFO:tasks.workunit.client.1.smithi178.stdout:0/577: getdents d0/dc/d39/d4c/d43/d8c 0 2024-09-10T11:29:30.954 INFO:tasks.workunit.client.1.smithi178.stdout:4/423: rmdir d0/d8/df 39 2024-09-10T11:29:30.954 INFO:tasks.workunit.client.1.smithi178.stdout:1/469: chown d3/d9/fa 185211798 1 2024-09-10T11:29:30.954 INFO:tasks.workunit.client.1.smithi178.stdout:9/600: write d2/d13/d75/f5d [2256515,109667] 0 2024-09-10T11:29:30.957 INFO:tasks.workunit.client.0.smithi144.stdout:0/104: getdents da 0 2024-09-10T11:29:30.957 INFO:tasks.workunit.client.1.smithi178.stdout:4/424: read - d0/d8/da/f96 zero size 2024-09-10T11:29:30.957 INFO:tasks.workunit.client.1.smithi178.stdout:2/718: creat d1/d7/d24/fe8 x:0 0 0 2024-09-10T11:29:30.958 INFO:tasks.workunit.client.1.smithi178.stdout:1/470: chown d3/d9/d98/dac/d3a/d63/f97 46 1 2024-09-10T11:29:30.960 INFO:tasks.workunit.client.1.smithi178.stdout:5/549: dread d9/dc/dd/d37/f75 [0,4194304] 0 2024-09-10T11:29:30.961 INFO:tasks.workunit.client.1.smithi178.stdout:7/461: mkdir d6/d10/d1d/daa 0 2024-09-10T11:29:30.963 INFO:tasks.workunit.client.1.smithi178.stdout:8/602: getdents d5/db/d10/d70 0 2024-09-10T11:29:30.963 INFO:tasks.workunit.client.1.smithi178.stdout:1/471: creat d3/d9/d21/d6f/faf x:0 0 0 2024-09-10T11:29:30.965 INFO:tasks.workunit.client.1.smithi178.stdout:2/719: rename d1/d7/d24/d43/dcf to d1/d7/d8e/de9 0 2024-09-10T11:29:30.965 INFO:tasks.workunit.client.1.smithi178.stdout:1/472: stat d3/d9/d98/dac/d3a/c46 0 2024-09-10T11:29:30.965 INFO:tasks.workunit.client.1.smithi178.stdout:4/425: mkdir d0/d9/dc/d9a 0 2024-09-10T11:29:30.967 INFO:tasks.workunit.client.1.smithi178.stdout:7/462: unlink d6/d10/d1e/d2d/d2b/l9c 0 2024-09-10T11:29:30.967 INFO:tasks.workunit.client.1.smithi178.stdout:9/601: dwrite d2/d13/d75/d60/d66/fa7 [0,4194304] 0 2024-09-10T11:29:30.969 INFO:tasks.workunit.client.1.smithi178.stdout:5/550: creat d9/fbe x:0 0 0 2024-09-10T11:29:30.970 INFO:tasks.workunit.client.1.smithi178.stdout:1/473: write d3/d35/f99 [708527,80756] 0 2024-09-10T11:29:30.972 INFO:tasks.workunit.client.1.smithi178.stdout:7/463: fdatasync d6/d10/d34/f3f 0 2024-09-10T11:29:30.973 INFO:tasks.workunit.client.1.smithi178.stdout:8/603: dread d5/db/d1e/fa3 [0,4194304] 0 2024-09-10T11:29:30.973 INFO:tasks.workunit.client.1.smithi178.stdout:4/426: rename d0/d8/da/d1a/l92 to d0/d9/d3d/l9b 0 2024-09-10T11:29:30.974 INFO:tasks.workunit.client.1.smithi178.stdout:2/720: symlink d1/d7/d24/d46/d88/d9d/dab/d5b/d60/dd7/lea 0 2024-09-10T11:29:30.974 INFO:tasks.workunit.client.1.smithi178.stdout:9/602: mknod d2/d19/d1c/d77/cbb 0 2024-09-10T11:29:30.976 INFO:tasks.workunit.client.1.smithi178.stdout:4/427: write d0/d8/d3e/f85 [971215,72050] 0 2024-09-10T11:29:30.978 INFO:tasks.workunit.client.1.smithi178.stdout:5/551: unlink d9/dc/dd/d3f/d5a/d7d/fa1 0 2024-09-10T11:29:30.980 INFO:tasks.workunit.client.1.smithi178.stdout:8/604: unlink d5/db/d3d/l72 0 2024-09-10T11:29:30.981 INFO:tasks.workunit.client.1.smithi178.stdout:4/428: dread d0/d21/f93 [0,4194304] 0 2024-09-10T11:29:30.981 INFO:tasks.workunit.client.1.smithi178.stdout:8/605: dread - d5/db7/fbf zero size 2024-09-10T11:29:30.982 INFO:tasks.workunit.client.1.smithi178.stdout:7/464: rename d6/d10/c3d to d6/d10/d1e/d2d/d2b/da0/cab 0 2024-09-10T11:29:30.982 INFO:tasks.workunit.client.1.smithi178.stdout:4/429: readlink d0/d21/l79 0 2024-09-10T11:29:30.983 INFO:tasks.workunit.client.1.smithi178.stdout:9/603: creat d2/d13/d40/d89/d93/d88/dae/fbc x:0 0 0 2024-09-10T11:29:30.987 INFO:tasks.workunit.client.1.smithi178.stdout:2/721: creat d1/feb x:0 0 0 2024-09-10T11:29:30.988 INFO:tasks.workunit.client.1.smithi178.stdout:9/604: rmdir d2/d19/d1c 39 2024-09-10T11:29:30.988 INFO:tasks.workunit.client.1.smithi178.stdout:2/722: stat d1/feb 0 2024-09-10T11:29:30.988 INFO:tasks.workunit.client.1.smithi178.stdout:5/552: dwrite d9/dc/dd/fad [0,4194304] 0 2024-09-10T11:29:30.989 INFO:tasks.workunit.client.1.smithi178.stdout:4/430: creat d0/d8/da/d1a/d51/f9c x:0 0 0 2024-09-10T11:29:30.991 INFO:tasks.workunit.client.1.smithi178.stdout:8/606: creat d5/db/d10/d20/d52/d67/dbb/fc9 x:0 0 0 2024-09-10T11:29:30.993 INFO:tasks.workunit.client.1.smithi178.stdout:4/431: fsync d0/d9/d3d/f44 0 2024-09-10T11:29:30.994 INFO:tasks.workunit.client.1.smithi178.stdout:7/465: dwrite d6/d10/d1e/f8e [0,4194304] 0 2024-09-10T11:29:30.995 INFO:tasks.workunit.client.1.smithi178.stdout:9/605: readlink d2/d19/d1c/d2d/l3a 0 2024-09-10T11:29:30.997 INFO:tasks.workunit.client.1.smithi178.stdout:1/474: write d3/d9/fb [3473426,130251] 0 2024-09-10T11:29:30.999 INFO:tasks.workunit.client.0.smithi144.stdout:1/156: sync 2024-09-10T11:29:31.000 INFO:tasks.workunit.client.0.smithi144.stdout:1/157: chown d5/f23 14793 1 2024-09-10T11:29:31.001 INFO:tasks.workunit.client.1.smithi178.stdout:4/432: fdatasync d0/d8/da/f27 0 2024-09-10T11:29:31.001 INFO:tasks.workunit.client.1.smithi178.stdout:8/607: unlink d5/db/d10/d20/d52/f68 0 2024-09-10T11:29:31.002 INFO:tasks.workunit.client.0.smithi144.stdout:1/158: mkdir d5/d2a 0 2024-09-10T11:29:31.002 INFO:tasks.workunit.client.1.smithi178.stdout:9/606: creat d2/d13/d40/d89/fbd x:0 0 0 2024-09-10T11:29:31.003 INFO:tasks.workunit.client.0.smithi144.stdout:1/159: chown d5/f15 3184248 1 2024-09-10T11:29:31.004 INFO:tasks.workunit.client.1.smithi178.stdout:1/475: rmdir d3/d9/d21/d6f 39 2024-09-10T11:29:31.004 INFO:tasks.workunit.client.1.smithi178.stdout:4/433: chown d0/d9/dc/c46 171884080 1 2024-09-10T11:29:31.004 INFO:tasks.workunit.client.1.smithi178.stdout:7/466: mknod d6/d10/d1e/cac 0 2024-09-10T11:29:31.005 INFO:tasks.workunit.client.0.smithi144.stdout:1/160: rmdir d5 39 2024-09-10T11:29:31.008 INFO:tasks.workunit.client.1.smithi178.stdout:7/467: write d6/d10/d1d/d22/d50/f59 [2581172,76535] 0 2024-09-10T11:29:31.008 INFO:tasks.workunit.client.1.smithi178.stdout:9/607: dread d2/d19/d1c/f24 [12582912,4194304] 0 2024-09-10T11:29:31.009 INFO:tasks.workunit.client.0.smithi144.stdout:1/161: mknod d5/dd/c2b 0 2024-09-10T11:29:31.009 INFO:tasks.workunit.client.1.smithi178.stdout:9/608: chown d2/le 307166 1 2024-09-10T11:29:31.012 INFO:tasks.workunit.client.1.smithi178.stdout:9/609: write d2/d19/d1c/d2d/dab/fb1 [552691,35326] 0 2024-09-10T11:29:31.012 INFO:tasks.workunit.client.1.smithi178.stdout:4/434: unlink d0/d8/df/l43 0 2024-09-10T11:29:31.013 INFO:tasks.workunit.client.0.smithi144.stdout:1/162: getdents d5/df 0 2024-09-10T11:29:31.014 INFO:tasks.workunit.client.0.smithi144.stdout:1/163: dread - d5/dd/f1d zero size 2024-09-10T11:29:31.014 INFO:tasks.workunit.client.1.smithi178.stdout:9/610: readlink d2/d19/d1c/d2a/l57 0 2024-09-10T11:29:31.014 INFO:tasks.workunit.client.1.smithi178.stdout:2/723: dread d1/d7/d24/f2c [0,4194304] 0 2024-09-10T11:29:31.015 INFO:tasks.workunit.client.0.smithi144.stdout:1/164: write d5/dd/f13 [1588704,41296] 0 2024-09-10T11:29:31.017 INFO:tasks.workunit.client.1.smithi178.stdout:1/476: link d3/d35/l69 d3/d9/d21/d4b/d2b/d5f/lb0 0 2024-09-10T11:29:31.018 INFO:tasks.workunit.client.1.smithi178.stdout:7/468: dread d6/d10/d1e/d2d/d2b/da0/f7a [0,4194304] 0 2024-09-10T11:29:31.020 INFO:tasks.workunit.client.1.smithi178.stdout:2/724: rmdir d1/d7/d24/d43 39 2024-09-10T11:29:31.021 INFO:tasks.workunit.client.1.smithi178.stdout:4/435: mkdir d0/d9d 0 2024-09-10T11:29:31.022 INFO:tasks.workunit.client.1.smithi178.stdout:9/611: fsync d2/d13/d40/d89/d93/d88/f7b 0 2024-09-10T11:29:31.022 INFO:tasks.workunit.client.1.smithi178.stdout:1/477: mknod d3/d35/cb1 0 2024-09-10T11:29:31.023 INFO:tasks.workunit.client.1.smithi178.stdout:4/436: write d0/f94 [799378,59026] 0 2024-09-10T11:29:31.024 INFO:tasks.workunit.client.1.smithi178.stdout:7/469: mknod d6/d10/d1d/d22/d50/d98/cad 0 2024-09-10T11:29:31.024 INFO:tasks.workunit.client.1.smithi178.stdout:9/612: chown d2/df/f28 1 1 2024-09-10T11:29:31.025 INFO:tasks.workunit.client.1.smithi178.stdout:1/478: rmdir d3/d9/d21/d4b/d2b 39 2024-09-10T11:29:31.030 INFO:tasks.workunit.client.1.smithi178.stdout:9/613: write d2/d8/f9 [3721962,90802] 0 2024-09-10T11:29:31.030 INFO:tasks.workunit.client.0.smithi144.stdout:1/165: unlink d5/dd/d20/f1e 0 2024-09-10T11:29:31.032 INFO:tasks.workunit.client.1.smithi178.stdout:1/479: symlink d3/d9/d21/d4b/lb2 0 2024-09-10T11:29:31.033 INFO:tasks.workunit.client.1.smithi178.stdout:8/608: dwrite d5/db/fe [0,4194304] 0 2024-09-10T11:29:31.035 INFO:tasks.workunit.client.1.smithi178.stdout:8/609: fsync d5/db/d3d/fbe 0 2024-09-10T11:29:31.035 INFO:tasks.workunit.client.1.smithi178.stdout:9/614: readlink d2/d13/d40/d56/d9a/lb4 0 2024-09-10T11:29:31.036 INFO:tasks.workunit.client.1.smithi178.stdout:7/470: rename d6/d10/l33 to d6/d10/d1e/d2d/d2b/da0/d55/lae 0 2024-09-10T11:29:31.037 INFO:tasks.workunit.client.1.smithi178.stdout:1/480: creat d3/d9/d98/dac/d3a/dae/fb3 x:0 0 0 2024-09-10T11:29:31.040 INFO:tasks.workunit.client.1.smithi178.stdout:9/615: mkdir d2/d13/d40/d56/dbe 0 2024-09-10T11:29:31.041 INFO:tasks.workunit.client.1.smithi178.stdout:7/471: unlink d6/d10/d1e/d24/d49/d70/c76 0 2024-09-10T11:29:31.042 INFO:tasks.workunit.client.1.smithi178.stdout:4/437: write d0/d21/f93 [3523588,59546] 0 2024-09-10T11:29:31.043 INFO:tasks.workunit.client.1.smithi178.stdout:9/616: write d2/d13/d75/d60/d66/fa7 [4719826,20491] 0 2024-09-10T11:29:31.046 INFO:tasks.workunit.client.1.smithi178.stdout:9/617: chown d2/d51/dad 15138793 1 2024-09-10T11:29:31.048 INFO:tasks.workunit.client.1.smithi178.stdout:7/472: stat d6/d10/d1e/d2d/d2b/da0/cab 0 2024-09-10T11:29:31.048 INFO:tasks.workunit.client.1.smithi178.stdout:4/438: rename d0/f40 to d0/d8/df/f9e 0 2024-09-10T11:29:31.048 INFO:tasks.workunit.client.1.smithi178.stdout:9/618: symlink d2/d9b/lbf 0 2024-09-10T11:29:31.049 INFO:tasks.workunit.client.1.smithi178.stdout:4/439: fdatasync d0/d9/fb 0 2024-09-10T11:29:31.050 INFO:tasks.workunit.client.1.smithi178.stdout:9/619: chown d2/d13/d40/d89/fbd 809526992 1 2024-09-10T11:29:31.052 INFO:tasks.workunit.client.1.smithi178.stdout:4/440: write d0/d8/da/d1a/f8b [338519,60530] 0 2024-09-10T11:29:31.052 INFO:tasks.workunit.client.1.smithi178.stdout:7/473: creat d6/d10/d34/d6a/d6f/faf x:0 0 0 2024-09-10T11:29:31.054 INFO:tasks.workunit.client.1.smithi178.stdout:9/620: chown d2/d19/d1c/d2a/fa9 0 1 2024-09-10T11:29:31.054 INFO:tasks.workunit.client.1.smithi178.stdout:4/441: write d0/d8/da/d1a/f57 [761123,58000] 0 2024-09-10T11:29:31.055 INFO:tasks.workunit.client.1.smithi178.stdout:7/474: write d6/d10/d1d/d22/d93/fa8 [878392,114522] 0 2024-09-10T11:29:31.056 INFO:tasks.workunit.client.0.smithi144.stdout:5/143: sync 2024-09-10T11:29:31.056 INFO:tasks.workunit.client.0.smithi144.stdout:5/144: fsync d4/d5/f25 0 2024-09-10T11:29:31.057 INFO:tasks.workunit.client.0.smithi144.stdout:1/166: mknod d5/c2c 0 2024-09-10T11:29:31.059 INFO:tasks.workunit.client.1.smithi178.stdout:7/475: write d6/d10/d1e/f8e [370196,62665] 0 2024-09-10T11:29:31.060 INFO:tasks.workunit.client.0.smithi144.stdout:5/145: dread d4/fa [0,4194304] 0 2024-09-10T11:29:31.064 INFO:tasks.workunit.client.1.smithi178.stdout:4/442: getdents d0/d8/da/d1a 0 2024-09-10T11:29:31.064 INFO:tasks.workunit.client.0.smithi144.stdout:5/146: dread d4/fa [0,4194304] 0 2024-09-10T11:29:31.064 INFO:tasks.workunit.client.1.smithi178.stdout:1/481: dwrite d3/d9/d98/dac/d3a/d63/d7e/f8c [0,4194304] 0 2024-09-10T11:29:31.068 INFO:tasks.workunit.client.0.smithi144.stdout:1/167: truncate d5/f1b 2990171 0 2024-09-10T11:29:31.068 INFO:tasks.workunit.client.1.smithi178.stdout:9/621: dwrite d2/d8/fa [0,4194304] 0 2024-09-10T11:29:31.071 INFO:tasks.workunit.client.0.smithi144.stdout:5/147: rename d4/d5/d18/f1b to d4/d20/d24/d2d/f2e 0 2024-09-10T11:29:31.073 INFO:tasks.workunit.client.1.smithi178.stdout:7/476: dwrite d6/d10/f5c [0,4194304] 0 2024-09-10T11:29:31.073 INFO:tasks.workunit.client.1.smithi178.stdout:1/482: link d3/d35/fa4 d3/d35/fb4 0 2024-09-10T11:29:31.075 INFO:tasks.workunit.client.0.smithi144.stdout:5/148: symlink d4/d20/l2f 0 2024-09-10T11:29:31.077 INFO:tasks.workunit.client.0.smithi144.stdout:5/149: dread - d4/d5/f28 zero size 2024-09-10T11:29:31.079 INFO:tasks.workunit.client.0.smithi144.stdout:1/168: dwrite d5/fc [0,4194304] 0 2024-09-10T11:29:31.079 INFO:tasks.workunit.client.0.smithi144.stdout:5/150: chown d4/l1a 777397 1 2024-09-10T11:29:31.080 INFO:tasks.workunit.client.1.smithi178.stdout:7/477: fdatasync d6/d10/d1d/d22/d50/d7e/f8f 0 2024-09-10T11:29:31.080 INFO:tasks.workunit.client.0.smithi144.stdout:1/169: chown d5/dd/c1c 1 1 2024-09-10T11:29:31.081 INFO:tasks.workunit.client.1.smithi178.stdout:9/622: write d2/df/f1b [5740823,19864] 0 2024-09-10T11:29:31.082 INFO:tasks.workunit.client.1.smithi178.stdout:9/623: chown d2/d13/d40/d56 246013 1 2024-09-10T11:29:31.082 INFO:tasks.workunit.client.0.smithi144.stdout:1/170: write d5/f23 [532311,57799] 0 2024-09-10T11:29:31.084 INFO:tasks.workunit.client.1.smithi178.stdout:1/483: dwrite d3/d9/d98/dac/d3a/f4e [0,4194304] 0 2024-09-10T11:29:31.088 INFO:tasks.workunit.client.0.smithi144.stdout:1/171: rename d5/fe to d5/d2a/f2d 0 2024-09-10T11:29:31.088 INFO:tasks.workunit.client.0.smithi144.stdout:1/172: rename d5 to d5/d2e 22 2024-09-10T11:29:31.092 INFO:tasks.workunit.client.1.smithi178.stdout:1/484: mkdir d3/d9/d98/dac/d3a/d50/db5 0 2024-09-10T11:29:31.093 INFO:tasks.workunit.client.1.smithi178.stdout:9/624: dwrite d2/f18 [0,4194304] 0 2024-09-10T11:29:31.095 INFO:tasks.workunit.client.0.smithi144.stdout:1/173: creat d5/dd/d20/f2f x:0 0 0 2024-09-10T11:29:31.095 INFO:tasks.workunit.client.0.smithi144.stdout:5/151: link d4/d5/l1c d4/d5/l30 0 2024-09-10T11:29:31.097 INFO:tasks.workunit.client.0.smithi144.stdout:1/174: mknod d5/d2a/c30 0 2024-09-10T11:29:31.098 INFO:tasks.workunit.client.1.smithi178.stdout:1/485: creat d3/d9/d21/fb6 x:0 0 0 2024-09-10T11:29:31.101 INFO:tasks.workunit.client.1.smithi178.stdout:9/625: dwrite d2/d19/d1c/d2d/dab/fb1 [0,4194304] 0 2024-09-10T11:29:31.101 INFO:tasks.workunit.client.0.smithi144.stdout:1/175: mknod d5/df/c31 0 2024-09-10T11:29:31.102 INFO:tasks.workunit.client.0.smithi144.stdout:5/152: truncate d4/d5/f25 401652 0 2024-09-10T11:29:31.102 INFO:tasks.workunit.client.1.smithi178.stdout:9/626: chown d2/d19/d1c/d2a/f92 1178 1 2024-09-10T11:29:31.105 INFO:tasks.workunit.client.1.smithi178.stdout:9/627: write d2/d19/d1c/d2d/dab/fb1 [1505364,100832] 0 2024-09-10T11:29:31.106 INFO:tasks.workunit.client.1.smithi178.stdout:6/567: sync 2024-09-10T11:29:31.106 INFO:tasks.workunit.client.1.smithi178.stdout:3/480: sync 2024-09-10T11:29:31.108 INFO:tasks.workunit.client.0.smithi144.stdout:1/176: dwrite d5/f18 [0,4194304] 0 2024-09-10T11:29:31.111 INFO:tasks.workunit.client.1.smithi178.stdout:1/486: dwrite d3/d9/d98/dac/d3a/d63/f6a [0,4194304] 0 2024-09-10T11:29:31.112 INFO:tasks.workunit.client.0.smithi144.stdout:5/153: rename d4/d5/l1e to d4/d5/l31 0 2024-09-10T11:29:31.113 INFO:tasks.workunit.client.1.smithi178.stdout:7/478: write d6/d10/d34/f74 [1048531,32720] 0 2024-09-10T11:29:31.115 INFO:tasks.workunit.client.1.smithi178.stdout:3/481: symlink db/d20/d3d/d56/d64/l8c 0 2024-09-10T11:29:31.116 INFO:tasks.workunit.client.0.smithi144.stdout:1/177: link d5/f15 d5/f32 0 2024-09-10T11:29:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:30 smithi178 ceph-mon[33514]: pgmap v22: 65 pgs: 65 active+clean; 1.1 GiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 29 MiB/s rd, 155 MiB/s wr, 426 op/s 2024-09-10T11:29:31.119 INFO:tasks.workunit.client.1.smithi178.stdout:7/479: creat d6/d10/d1e/d2d/d2b/da0/d55/fb0 x:0 0 0 2024-09-10T11:29:31.119 INFO:tasks.workunit.client.1.smithi178.stdout:1/487: creat d3/d9/d21/fb7 x:0 0 0 2024-09-10T11:29:31.120 INFO:tasks.workunit.client.1.smithi178.stdout:3/482: truncate db/f27 811128 0 2024-09-10T11:29:31.121 INFO:tasks.workunit.client.0.smithi144.stdout:5/154: dwrite d4/fa [0,4194304] 0 2024-09-10T11:29:31.124 INFO:tasks.workunit.client.1.smithi178.stdout:7/480: dread d6/d10/d1d/d22/d3b/f5a [0,4194304] 0 2024-09-10T11:29:31.124 INFO:tasks.workunit.client.1.smithi178.stdout:1/488: write d3/d9/d21/d4b/d2b/d5f/f73 [396936,81833] 0 2024-09-10T11:29:31.126 INFO:tasks.workunit.client.1.smithi178.stdout:3/483: mknod db/d26/d40/c8d 0 2024-09-10T11:29:31.128 INFO:tasks.workunit.client.1.smithi178.stdout:7/481: mkdir d6/d10/d34/d6a/d6f/db1 0 2024-09-10T11:29:31.129 INFO:tasks.workunit.client.1.smithi178.stdout:1/489: read d3/d9/d98/dac/d3a/d6e/d8a/f4a [2967434,83335] 0 2024-09-10T11:29:31.130 INFO:tasks.workunit.client.1.smithi178.stdout:1/490: chown d3/l4 753 1 2024-09-10T11:29:31.131 INFO:tasks.workunit.client.1.smithi178.stdout:1/491: truncate d3/d9/d21/fb6 70028 0 2024-09-10T11:29:31.131 INFO:tasks.workunit.client.1.smithi178.stdout:3/484: rename db/d26/f3b to db/d20/d70/f8e 0 2024-09-10T11:29:31.132 INFO:tasks.workunit.client.1.smithi178.stdout:3/485: stat l6 0 2024-09-10T11:29:31.132 INFO:tasks.workunit.client.1.smithi178.stdout:3/486: fsync db/f39 0 2024-09-10T11:29:31.168 INFO:tasks.workunit.client.1.smithi178.stdout:7/482: rename d6/d10/d1e/d24 to d6/d10/d1d/d22/d50/d7e/d82/db2 0 2024-09-10T11:29:31.168 INFO:tasks.workunit.client.1.smithi178.stdout:1/492: mkdir d3/d9/d98/dac/d3a/d50/d6c/d78/db8 0 2024-09-10T11:29:31.169 INFO:tasks.workunit.client.0.smithi144.stdout:1/178: rmdir d5/dd/d20 39 2024-09-10T11:29:31.170 INFO:tasks.workunit.client.1.smithi178.stdout:9/628: write d2/d19/d1c/d77/f90 [3196930,54837] 0 2024-09-10T11:29:31.170 INFO:tasks.workunit.client.1.smithi178.stdout:1/493: fdatasync d3/f1c 0 2024-09-10T11:29:31.172 INFO:tasks.workunit.client.1.smithi178.stdout:3/487: creat db/d20/d3d/d56/d64/f8f x:0 0 0 2024-09-10T11:29:31.172 INFO:tasks.workunit.client.0.smithi144.stdout:5/155: unlink d4/f2c 0 2024-09-10T11:29:31.174 INFO:tasks.workunit.client.0.smithi144.stdout:1/179: creat d5/df/f33 x:0 0 0 2024-09-10T11:29:31.174 INFO:tasks.workunit.client.1.smithi178.stdout:3/488: stat db/d20/l23 0 2024-09-10T11:29:31.175 INFO:tasks.workunit.client.0.smithi144.stdout:5/156: rmdir d4/d20/d24/d2d 39 2024-09-10T11:29:31.176 INFO:tasks.workunit.client.0.smithi144.stdout:5/157: dread - d4/d5/f23 zero size 2024-09-10T11:29:31.177 INFO:tasks.workunit.client.0.smithi144.stdout:5/158: fsync d4/d5/d18/f1f 0 2024-09-10T11:29:31.177 INFO:tasks.workunit.client.0.smithi144.stdout:1/180: rename d5/dd/d22/f27 to d5/d2a/f34 0 2024-09-10T11:29:31.179 INFO:tasks.workunit.client.1.smithi178.stdout:1/494: write d3/d9/d98/dac/d3a/d6e/d8a/f7b [1141337,110673] 0 2024-09-10T11:29:31.182 INFO:tasks.workunit.client.0.smithi144.stdout:1/181: creat d5/f35 x:0 0 0 2024-09-10T11:29:31.182 INFO:tasks.workunit.client.1.smithi178.stdout:7/483: dwrite d6/fb [0,4194304] 0 2024-09-10T11:29:31.184 INFO:tasks.workunit.client.1.smithi178.stdout:9/629: dread d2/d8/f17 [0,4194304] 0 2024-09-10T11:29:31.184 INFO:tasks.workunit.client.0.smithi144.stdout:5/159: mkdir d4/d5/d32 0 2024-09-10T11:29:31.185 INFO:tasks.workunit.client.1.smithi178.stdout:6/568: dwrite d3/d74/d91/daa/dc2/f5b [0,4194304] 0 2024-09-10T11:29:31.185 INFO:tasks.workunit.client.1.smithi178.stdout:3/489: link db/f1d db/d26/d74/f90 0 2024-09-10T11:29:31.188 INFO:tasks.workunit.client.1.smithi178.stdout:3/490: read - db/d20/d3d/f85 zero size 2024-09-10T11:29:31.188 INFO:tasks.workunit.client.1.smithi178.stdout:7/484: write d6/d10/d1d/d22/d50/d7e/d82/f9f [746502,24098] 0 2024-09-10T11:29:31.193 INFO:tasks.workunit.client.1.smithi178.stdout:3/491: fdatasync db/d26/d40/f69 0 2024-09-10T11:29:31.196 INFO:tasks.workunit.client.1.smithi178.stdout:1/495: link d3/d9/d21/d4b/d2b/f88 d3/d9/d21/d4b/d2b/fb9 0 2024-09-10T11:29:31.197 INFO:tasks.workunit.client.1.smithi178.stdout:9/630: dwrite d2/dc/f2f [0,4194304] 0 2024-09-10T11:29:31.198 INFO:tasks.workunit.client.1.smithi178.stdout:9/631: stat d2/d8/f73 0 2024-09-10T11:29:31.199 INFO:tasks.workunit.client.1.smithi178.stdout:3/492: mknod db/d20/d3e/d6b/d79/c91 0 2024-09-10T11:29:31.201 INFO:tasks.workunit.client.1.smithi178.stdout:9/632: rmdir d2/d19/d1c 39 2024-09-10T11:29:31.203 INFO:tasks.workunit.client.1.smithi178.stdout:3/493: write db/d20/d3d/d63/d6a/f6c [688598,76936] 0 2024-09-10T11:29:31.203 INFO:tasks.workunit.client.1.smithi178.stdout:1/496: getdents d3/d9/d98/dac/d3a/d50/d6c 0 2024-09-10T11:29:31.204 INFO:tasks.workunit.client.1.smithi178.stdout:9/633: chown d2/d13/d75/l97 817596 1 2024-09-10T11:29:31.205 INFO:tasks.workunit.client.1.smithi178.stdout:1/497: dread - d3/d9/d21/fb7 zero size 2024-09-10T11:29:31.206 INFO:tasks.workunit.client.1.smithi178.stdout:3/494: mknod db/c92 0 2024-09-10T11:29:31.207 INFO:tasks.workunit.client.1.smithi178.stdout:6/569: dwrite d3/d7/d35/f5a [0,4194304] 0 2024-09-10T11:29:31.207 INFO:tasks.workunit.client.1.smithi178.stdout:9/634: write d2/df/d31/d37/f94 [919277,115778] 0 2024-09-10T11:29:31.208 INFO:tasks.workunit.client.1.smithi178.stdout:3/495: dread - db/d20/d3d/d56/d64/f8f zero size 2024-09-10T11:29:31.211 INFO:tasks.workunit.client.1.smithi178.stdout:1/498: unlink d3/d9/d98/dac/d3a/d6e/d8a/f4a 0 2024-09-10T11:29:31.216 INFO:tasks.workunit.client.1.smithi178.stdout:3/496: dwrite f1 [0,4194304] 0 2024-09-10T11:29:31.216 INFO:tasks.workunit.client.1.smithi178.stdout:6/570: link d3/d7/d8/d15/fbc d3/d74/d91/daa/dc2/d57/d78/d7c/fd2 0 2024-09-10T11:29:31.222 INFO:tasks.workunit.client.1.smithi178.stdout:3/497: unlink db/d20/l23 0 2024-09-10T11:29:31.223 INFO:tasks.workunit.client.1.smithi178.stdout:3/498: chown db/d20/d3d/d56/l8b 261845 1 2024-09-10T11:29:31.223 INFO:tasks.workunit.client.1.smithi178.stdout:3/499: stat db/d35/d3a/l43 0 2024-09-10T11:29:31.225 INFO:tasks.workunit.client.1.smithi178.stdout:1/499: link d3/d9/l52 d3/d9/d98/dac/d3a/d63/lba 0 2024-09-10T11:29:31.228 INFO:tasks.workunit.client.1.smithi178.stdout:1/500: creat d3/d9/d98/dac/d3a/d50/d6c/fbb x:0 0 0 2024-09-10T11:29:31.229 INFO:tasks.workunit.client.1.smithi178.stdout:3/500: mknod db/d35/d3a/d42/c93 0 2024-09-10T11:29:31.230 INFO:tasks.workunit.client.1.smithi178.stdout:1/501: symlink d3/d35/lbc 0 2024-09-10T11:29:31.231 INFO:tasks.workunit.client.1.smithi178.stdout:3/501: symlink db/d20/d3d/d56/d64/l94 0 2024-09-10T11:29:31.232 INFO:tasks.workunit.client.1.smithi178.stdout:1/502: read d3/d9/d21/d4b/f5a [2463079,55339] 0 2024-09-10T11:29:31.233 INFO:tasks.workunit.client.1.smithi178.stdout:3/502: creat db/d20/f95 x:0 0 0 2024-09-10T11:29:31.234 INFO:tasks.workunit.client.1.smithi178.stdout:6/571: dread d3/d24/fc7 [0,4194304] 0 2024-09-10T11:29:31.234 INFO:tasks.workunit.client.1.smithi178.stdout:3/503: fdatasync db/d20/d3d/f85 0 2024-09-10T11:29:31.235 INFO:tasks.workunit.client.1.smithi178.stdout:6/572: stat d3/d74/dc9/d4b/cca 0 2024-09-10T11:29:31.236 INFO:tasks.workunit.client.1.smithi178.stdout:9/635: dread d2/d13/d75/d60/f8d [0,4194304] 0 2024-09-10T11:29:31.237 INFO:tasks.workunit.client.1.smithi178.stdout:3/504: creat db/d20/d3d/d63/f96 x:0 0 0 2024-09-10T11:29:31.240 INFO:tasks.workunit.client.1.smithi178.stdout:6/573: dread d3/d74/dc9/f47 [8388608,4194304] 0 2024-09-10T11:29:31.242 INFO:tasks.workunit.client.1.smithi178.stdout:1/503: getdents d3/d9/d98/dac/d3a/d63 0 2024-09-10T11:29:31.243 INFO:tasks.workunit.client.1.smithi178.stdout:3/505: creat db/f97 x:0 0 0 2024-09-10T11:29:31.246 INFO:tasks.workunit.client.1.smithi178.stdout:9/636: link d2/df/d31/d37/f94 d2/df/d31/fc0 0 2024-09-10T11:29:31.247 INFO:tasks.workunit.client.1.smithi178.stdout:6/574: write d3/d74/dc9/d4b/f9b [4804769,52247] 0 2024-09-10T11:29:31.249 INFO:tasks.workunit.client.0.smithi144.stdout:7/150: sync 2024-09-10T11:29:31.250 INFO:tasks.workunit.client.0.smithi144.stdout:1/182: symlink d5/df/l36 0 2024-09-10T11:29:31.251 INFO:tasks.workunit.client.0.smithi144.stdout:5/160: read - d4/d5/f28 zero size 2024-09-10T11:29:31.251 INFO:tasks.workunit.client.0.smithi144.stdout:7/151: read - d5/dc/d12/f21 zero size 2024-09-10T11:29:31.253 INFO:tasks.workunit.client.1.smithi178.stdout:1/504: rename d3/d9/d98/dac/d2a/c84 to d3/d9/d98/dac/d3a/d50/db5/cbd 0 2024-09-10T11:29:31.254 INFO:tasks.workunit.client.0.smithi144.stdout:5/161: creat d4/d5/d32/f33 x:0 0 0 2024-09-10T11:29:31.255 INFO:tasks.workunit.client.1.smithi178.stdout:6/575: mknod d3/d7/d8/d15/d1c/d2a/cd3 0 2024-09-10T11:29:31.256 INFO:tasks.workunit.client.1.smithi178.stdout:3/506: mkdir db/d20/d3e/d6b/d7e/d98 0 2024-09-10T11:29:31.257 INFO:tasks.workunit.client.1.smithi178.stdout:9/637: rmdir d2/d19/d1c/d77 39 2024-09-10T11:29:31.262 INFO:tasks.workunit.client.1.smithi178.stdout:6/576: symlink d3/d74/d91/daa/dc2/d97/ld4 0 2024-09-10T11:29:31.262 INFO:tasks.workunit.client.1.smithi178.stdout:6/577: write f0 [904280,49756] 0 2024-09-10T11:29:31.263 INFO:tasks.workunit.client.1.smithi178.stdout:3/507: mknod db/d26/d40/c99 0 2024-09-10T11:29:31.263 INFO:tasks.workunit.client.1.smithi178.stdout:6/578: read d3/d74/dc9/f93 [1363533,19621] 0 2024-09-10T11:29:31.266 INFO:tasks.workunit.client.1.smithi178.stdout:6/579: creat d3/d7/d8/d72/fd5 x:0 0 0 2024-09-10T11:29:31.268 INFO:tasks.workunit.client.1.smithi178.stdout:1/505: getdents d3/d9/d98/dac/d3a/d50/d6c 0 2024-09-10T11:29:31.270 INFO:tasks.workunit.client.1.smithi178.stdout:6/580: dread d3/d24/f81 [0,4194304] 0 2024-09-10T11:29:31.270 INFO:tasks.workunit.client.1.smithi178.stdout:3/508: getdents db/d20/d3d 0 2024-09-10T11:29:31.271 INFO:tasks.workunit.client.1.smithi178.stdout:1/506: creat d3/d9/d98/dac/d8d/fbe x:0 0 0 2024-09-10T11:29:31.272 INFO:tasks.workunit.client.1.smithi178.stdout:3/509: mkdir db/d26/d9a 0 2024-09-10T11:29:31.274 INFO:tasks.workunit.client.1.smithi178.stdout:3/510: mkdir db/d9b 0 2024-09-10T11:29:31.276 INFO:tasks.workunit.client.1.smithi178.stdout:1/507: rename d3/d9/f10 to d3/d9/d98/dac/d3a/d50/d6c/d78/db8/fbf 0 2024-09-10T11:29:31.277 INFO:tasks.workunit.client.1.smithi178.stdout:6/581: dwrite d3/d7/d8/d15/fa5 [0,4194304] 0 2024-09-10T11:29:31.278 INFO:tasks.workunit.client.1.smithi178.stdout:3/511: mkdir db/d20/d9c 0 2024-09-10T11:29:31.279 INFO:tasks.workunit.client.1.smithi178.stdout:3/512: chown db/f4f 46199630 1 2024-09-10T11:29:31.281 INFO:tasks.workunit.client.1.smithi178.stdout:3/513: write f8 [4779372,74907] 0 2024-09-10T11:29:31.281 INFO:tasks.workunit.client.1.smithi178.stdout:6/582: dread d3/d7/d8/d72/fb3 [0,4194304] 0 2024-09-10T11:29:31.285 INFO:tasks.workunit.client.1.smithi178.stdout:0/578: sync 2024-09-10T11:29:31.287 INFO:tasks.workunit.client.1.smithi178.stdout:1/508: truncate d3/d9/d21/d4b/f2d 129996 0 2024-09-10T11:29:31.287 INFO:tasks.workunit.client.1.smithi178.stdout:0/579: mkdir d0/d2c/d95/dbd 0 2024-09-10T11:29:31.287 INFO:tasks.workunit.client.1.smithi178.stdout:6/583: write d3/d74/dc9/fb2 [1675027,51200] 0 2024-09-10T11:29:31.289 INFO:tasks.workunit.client.1.smithi178.stdout:1/509: write d3/d9/d21/d4b/d2b/d5f/f85 [4519914,99467] 0 2024-09-10T11:29:31.290 INFO:tasks.workunit.client.1.smithi178.stdout:6/584: creat d3/d74/d91/daa/fd6 x:0 0 0 2024-09-10T11:29:31.291 INFO:tasks.workunit.client.1.smithi178.stdout:1/510: chown d3/d9/d98/dac/d3a/d50/d80/laa 443403528 1 2024-09-10T11:29:31.294 INFO:tasks.workunit.client.1.smithi178.stdout:6/585: creat d3/d7/d8/d15/d1c/d2a/d18/d8e/d90/fd7 x:0 0 0 2024-09-10T11:29:31.298 INFO:tasks.workunit.client.1.smithi178.stdout:6/586: symlink d3/d7/d8/d15/d1c/d2a/d18/ld8 0 2024-09-10T11:29:31.298 INFO:tasks.workunit.client.1.smithi178.stdout:1/511: dwrite d3/d9/d98/dac/d3a/d63/f6a [4194304,4194304] 0 2024-09-10T11:29:31.301 INFO:tasks.workunit.client.1.smithi178.stdout:6/587: unlink d3/d74/dc9/l9a 0 2024-09-10T11:29:31.302 INFO:tasks.workunit.client.1.smithi178.stdout:1/512: mknod d3/d9/d98/dac/d3a/d50/cc0 0 2024-09-10T11:29:31.302 INFO:tasks.workunit.client.1.smithi178.stdout:6/588: stat d3/d24/f30 0 2024-09-10T11:29:31.346 INFO:tasks.workunit.client.0.smithi144.stdout:1/183: dwrite d5/dd/d22/f24 [0,4194304] 0 2024-09-10T11:29:31.347 INFO:tasks.workunit.client.0.smithi144.stdout:7/152: dwrite d5/dc/f10 [0,4194304] 0 2024-09-10T11:29:31.347 INFO:tasks.workunit.client.0.smithi144.stdout:5/162: getdents d4/d20/d24/d2d 0 2024-09-10T11:29:31.351 INFO:tasks.workunit.client.0.smithi144.stdout:1/184: rename d5/f35 to d5/dd/d22/f37 0 2024-09-10T11:29:31.353 INFO:tasks.workunit.client.0.smithi144.stdout:7/153: creat d5/dc/d11/d18/f34 x:0 0 0 2024-09-10T11:29:31.354 INFO:tasks.workunit.client.0.smithi144.stdout:7/154: chown d5/dc/d12/f2c 6 1 2024-09-10T11:29:31.355 INFO:tasks.workunit.client.0.smithi144.stdout:1/185: creat d5/f38 x:0 0 0 2024-09-10T11:29:31.355 INFO:tasks.workunit.client.0.smithi144.stdout:7/155: stat d5/dc/d11/d18/f1e 0 2024-09-10T11:29:31.356 INFO:tasks.workunit.client.1.smithi178.stdout:1/513: dread d3/d9/d21/fb6 [0,4194304] 0 2024-09-10T11:29:31.358 INFO:tasks.workunit.client.1.smithi178.stdout:6/589: symlink d3/d74/d91/daa/dc2/ld9 0 2024-09-10T11:29:31.358 INFO:tasks.workunit.client.0.smithi144.stdout:1/186: rmdir d5/dd/d20 39 2024-09-10T11:29:31.361 INFO:tasks.workunit.client.0.smithi144.stdout:7/156: link d5/dc/d11/d18/f1e d5/dc/d2a/f35 0 2024-09-10T11:29:31.361 INFO:tasks.workunit.client.0.smithi144.stdout:5/163: dwrite d4/d5/f25 [0,4194304] 0 2024-09-10T11:29:31.363 INFO:tasks.workunit.client.0.smithi144.stdout:5/164: truncate d4/d20/d24/d2d/f2e 280120 0 2024-09-10T11:29:31.363 INFO:tasks.workunit.client.0.smithi144.stdout:5/165: dread - d4/d5/f28 zero size 2024-09-10T11:29:31.364 INFO:tasks.workunit.client.1.smithi178.stdout:1/514: dread d3/d9/d1a/d48/f66 [0,4194304] 0 2024-09-10T11:29:31.365 INFO:tasks.workunit.client.0.smithi144.stdout:7/157: symlink d5/dc/d12/l36 0 2024-09-10T11:29:31.366 INFO:tasks.workunit.client.1.smithi178.stdout:6/590: dwrite d3/d74/dc9/f47 [8388608,4194304] 0 2024-09-10T11:29:31.367 INFO:tasks.workunit.client.0.smithi144.stdout:1/187: creat d5/dd/d20/f39 x:0 0 0 2024-09-10T11:29:31.368 INFO:tasks.workunit.client.0.smithi144.stdout:5/166: creat d4/d20/f34 x:0 0 0 2024-09-10T11:29:31.370 INFO:tasks.workunit.client.0.smithi144.stdout:1/188: symlink d5/df/l3a 0 2024-09-10T11:29:31.371 INFO:tasks.workunit.client.0.smithi144.stdout:5/167: write d4/d20/d24/d2d/f2e [1179526,176] 0 2024-09-10T11:29:31.373 INFO:tasks.workunit.client.0.smithi144.stdout:7/158: creat d5/f37 x:0 0 0 2024-09-10T11:29:31.374 INFO:tasks.workunit.client.0.smithi144.stdout:7/159: fdatasync d5/f29 0 2024-09-10T11:29:31.375 INFO:tasks.workunit.client.0.smithi144.stdout:5/168: mknod d4/d20/d22/c35 0 2024-09-10T11:29:31.380 INFO:tasks.workunit.client.0.smithi144.stdout:7/160: creat d5/f38 x:0 0 0 2024-09-10T11:29:31.381 INFO:tasks.workunit.client.0.smithi144.stdout:5/169: rmdir d4 39 2024-09-10T11:29:31.381 INFO:tasks.workunit.client.0.smithi144.stdout:7/161: chown d5/dc/d12/c1d 0 1 2024-09-10T11:29:31.383 INFO:tasks.workunit.client.0.smithi144.stdout:7/162: dread - d5/dc/d12/f2c zero size 2024-09-10T11:29:31.384 INFO:tasks.workunit.client.0.smithi144.stdout:5/170: symlink d4/d20/d22/l36 0 2024-09-10T11:29:31.385 INFO:tasks.workunit.client.0.smithi144.stdout:1/189: dwrite d5/f1b [0,4194304] 0 2024-09-10T11:29:31.386 INFO:tasks.workunit.client.0.smithi144.stdout:7/163: truncate d5/dc/d12/f21 726190 0 2024-09-10T11:29:31.386 INFO:tasks.workunit.client.0.smithi144.stdout:7/164: fsync d5/f37 0 2024-09-10T11:29:31.387 INFO:tasks.workunit.client.0.smithi144.stdout:1/190: fsync d5/f7 0 2024-09-10T11:29:31.387 INFO:tasks.workunit.client.0.smithi144.stdout:1/191: readlink d5/l9 0 2024-09-10T11:29:31.393 INFO:tasks.workunit.client.0.smithi144.stdout:1/192: getdents d5 0 2024-09-10T11:29:31.393 INFO:tasks.workunit.client.0.smithi144.stdout:5/171: dwrite d4/f15 [0,4194304] 0 2024-09-10T11:29:31.394 INFO:tasks.workunit.client.1.smithi178.stdout:5/553: sync 2024-09-10T11:29:31.396 INFO:tasks.workunit.client.0.smithi144.stdout:7/165: dwrite d5/dc/d11/d18/f34 [0,4194304] 0 2024-09-10T11:29:31.397 INFO:tasks.workunit.client.0.smithi144.stdout:7/166: write d5/dc/d12/f2c [755234,53692] 0 2024-09-10T11:29:31.398 INFO:tasks.workunit.client.1.smithi178.stdout:5/554: dread - d9/dc/dd/d37/d42/d9f/fa5 zero size 2024-09-10T11:29:31.400 INFO:tasks.workunit.client.1.smithi178.stdout:6/591: rename d3/d7/d8/d15/d1c/d2a/f3c to d3/d6e/fda 0 2024-09-10T11:29:31.400 INFO:tasks.workunit.client.0.smithi144.stdout:1/193: mkdir d5/dd/d3b 0 2024-09-10T11:29:31.401 INFO:tasks.workunit.client.0.smithi144.stdout:1/194: write d5/fc [153984,43871] 0 2024-09-10T11:29:31.402 INFO:tasks.workunit.client.1.smithi178.stdout:1/515: getdents d3/d9/d21/d6f 0 2024-09-10T11:29:31.403 INFO:tasks.workunit.client.0.smithi144.stdout:1/195: write d5/dd/f21 [3133508,122952] 0 2024-09-10T11:29:31.404 INFO:tasks.workunit.client.1.smithi178.stdout:5/555: link d9/f5e d9/dc/dd/d37/d42/d62/daf/fbf 0 2024-09-10T11:29:31.405 INFO:tasks.workunit.client.0.smithi144.stdout:1/196: dread - d5/f32 zero size 2024-09-10T11:29:31.406 INFO:tasks.workunit.client.0.smithi144.stdout:7/167: symlink d5/dc/l39 0 2024-09-10T11:29:31.406 INFO:tasks.workunit.client.0.smithi144.stdout:5/172: dwrite d4/d5/d32/f33 [0,4194304] 0 2024-09-10T11:29:31.409 INFO:tasks.workunit.client.1.smithi178.stdout:6/592: rename d3/d74/d91/daa/dc2/d57/d78/d7c/l88 to d3/d7/d8/ldb 0 2024-09-10T11:29:31.411 INFO:tasks.workunit.client.1.smithi178.stdout:5/556: chown d9/dc/dd/d16/c59 185936 1 2024-09-10T11:29:31.412 INFO:tasks.workunit.client.0.smithi144.stdout:7/168: mknod d5/dc/c3a 0 2024-09-10T11:29:31.412 INFO:tasks.workunit.client.0.smithi144.stdout:5/173: dread d4/f15 [0,4194304] 0 2024-09-10T11:29:31.412 INFO:tasks.workunit.client.1.smithi178.stdout:5/557: fdatasync d9/dc/dd/f69 0 2024-09-10T11:29:31.413 INFO:tasks.workunit.client.0.smithi144.stdout:5/174: truncate d4/d5/f28 1035020 0 2024-09-10T11:29:31.414 INFO:tasks.workunit.client.1.smithi178.stdout:1/516: dwrite d3/d35/fa9 [0,4194304] 0 2024-09-10T11:29:31.415 INFO:tasks.workunit.client.1.smithi178.stdout:6/593: write d3/d7/d8/d15/d1c/d2a/f41 [5246468,84363] 0 2024-09-10T11:29:31.416 INFO:tasks.workunit.client.0.smithi144.stdout:1/197: dread d5/f7 [0,4194304] 0 2024-09-10T11:29:31.418 INFO:tasks.workunit.client.0.smithi144.stdout:5/175: dread d4/fa [0,4194304] 0 2024-09-10T11:29:31.419 INFO:tasks.workunit.client.0.smithi144.stdout:7/169: link d5/c1b d5/dc/d11/c3b 0 2024-09-10T11:29:31.419 INFO:tasks.workunit.client.1.smithi178.stdout:6/594: write d3/d7/d8/d15/d1c/f53 [976377,127532] 0 2024-09-10T11:29:31.425 INFO:tasks.workunit.client.0.smithi144.stdout:1/198: getdents d5/d2a 0 2024-09-10T11:29:31.426 INFO:tasks.workunit.client.0.smithi144.stdout:1/199: fdatasync d5/dd/f13 0 2024-09-10T11:29:31.426 INFO:tasks.workunit.client.0.smithi144.stdout:3/96: sync 2024-09-10T11:29:31.427 INFO:tasks.workunit.client.0.smithi144.stdout:4/126: sync 2024-09-10T11:29:31.429 INFO:tasks.workunit.client.0.smithi144.stdout:7/170: dwrite d5/dc/fe [0,4194304] 0 2024-09-10T11:29:31.429 INFO:tasks.workunit.client.0.smithi144.stdout:1/200: write d5/f7 [668385,57298] 0 2024-09-10T11:29:31.430 INFO:tasks.workunit.client.0.smithi144.stdout:7/171: chown d5/dc/d11/d18 61593 1 2024-09-10T11:29:31.430 INFO:tasks.workunit.client.0.smithi144.stdout:1/201: chown d5/df/f19 2471518 1 2024-09-10T11:29:31.431 INFO:tasks.workunit.client.0.smithi144.stdout:7/172: write d5/f8 [2830787,81911] 0 2024-09-10T11:29:31.432 INFO:tasks.workunit.client.1.smithi178.stdout:5/558: dwrite d9/dc/dd/d37/f8d [4194304,4194304] 0 2024-09-10T11:29:31.433 INFO:tasks.workunit.client.1.smithi178.stdout:1/517: dwrite d3/d9/d1a/f5c [0,4194304] 0 2024-09-10T11:29:31.433 INFO:tasks.workunit.client.0.smithi144.stdout:3/97: dread f2 [0,4194304] 0 2024-09-10T11:29:31.437 INFO:tasks.workunit.client.0.smithi144.stdout:5/176: truncate d4/d5/f28 18465 0 2024-09-10T11:29:31.437 INFO:tasks.workunit.client.0.smithi144.stdout:1/202: unlink d5/f1b 0 2024-09-10T11:29:31.438 INFO:tasks.workunit.client.1.smithi178.stdout:6/595: dwrite d3/d7/d8/d19/f8b [0,4194304] 0 2024-09-10T11:29:31.440 INFO:tasks.workunit.client.0.smithi144.stdout:7/173: creat d5/dc/f3c x:0 0 0 2024-09-10T11:29:31.442 INFO:tasks.workunit.client.1.smithi178.stdout:5/559: mknod d9/dc/dd/d3f/d5a/dbc/cc0 0 2024-09-10T11:29:31.442 INFO:tasks.workunit.client.0.smithi144.stdout:7/174: dread - d5/dc/d11/f17 zero size 2024-09-10T11:29:31.443 INFO:tasks.workunit.client.1.smithi178.stdout:6/596: mkdir d3/d74/dc9/ddc 0 2024-09-10T11:29:31.444 INFO:tasks.workunit.client.0.smithi144.stdout:5/177: readlink d4/d5/l17 0 2024-09-10T11:29:31.445 INFO:tasks.workunit.client.1.smithi178.stdout:1/518: link d3/d9/d98/dac/d2a/f41 d3/da0/fc1 0 2024-09-10T11:29:31.446 INFO:tasks.workunit.client.1.smithi178.stdout:6/597: chown d3/d74/d91/daa/dc2/d57/f6a 6755528 1 2024-09-10T11:29:31.446 INFO:tasks.workunit.client.0.smithi144.stdout:1/203: creat d5/f3c x:0 0 0 2024-09-10T11:29:31.449 INFO:tasks.workunit.client.0.smithi144.stdout:7/175: rmdir d5/dc/d11/d18 39 2024-09-10T11:29:31.450 INFO:tasks.workunit.client.0.smithi144.stdout:1/204: mkdir d5/dd/d20/d3d 0 2024-09-10T11:29:31.451 INFO:tasks.workunit.client.1.smithi178.stdout:6/598: read d3/d7/d8/d19/f26 [1127646,81711] 0 2024-09-10T11:29:31.451 INFO:tasks.workunit.client.1.smithi178.stdout:5/560: dwrite d9/dc/dd/f7f [0,4194304] 0 2024-09-10T11:29:31.453 INFO:tasks.workunit.client.0.smithi144.stdout:1/205: dread - d5/f32 zero size 2024-09-10T11:29:31.454 INFO:tasks.workunit.client.0.smithi144.stdout:7/176: rmdir d5/dc/d12 39 2024-09-10T11:29:31.458 INFO:tasks.workunit.client.0.smithi144.stdout:7/177: truncate d5/dc/f10 4631758 0 2024-09-10T11:29:31.461 INFO:tasks.workunit.client.0.smithi144.stdout:7/178: stat d5/f29 0 2024-09-10T11:29:31.462 INFO:tasks.workunit.client.1.smithi178.stdout:1/519: dwrite d3/d9/d21/fb6 [0,4194304] 0 2024-09-10T11:29:31.463 INFO:tasks.workunit.client.0.smithi144.stdout:5/178: dwrite d4/fa [4194304,4194304] 0 2024-09-10T11:29:31.463 INFO:tasks.workunit.client.0.smithi144.stdout:1/206: unlink d5/f18 0 2024-09-10T11:29:31.466 INFO:tasks.workunit.client.0.smithi144.stdout:9/127: sync 2024-09-10T11:29:31.467 INFO:tasks.workunit.client.0.smithi144.stdout:4/127: dread f5 [0,4194304] 0 2024-09-10T11:29:31.467 INFO:tasks.workunit.client.0.smithi144.stdout:2/121: sync 2024-09-10T11:29:31.468 INFO:tasks.workunit.client.0.smithi144.stdout:1/207: chown d5/c28 1 1 2024-09-10T11:29:31.469 INFO:tasks.workunit.client.1.smithi178.stdout:1/520: rename d3/d9/d98/dac/d3a/d50/d6c/d78/da3 to d3/d9/d98/dac/d3a/d50/d9e/dc2 0 2024-09-10T11:29:31.470 INFO:tasks.workunit.client.0.smithi144.stdout:5/179: mknod d4/d5/d32/c37 0 2024-09-10T11:29:31.471 INFO:tasks.workunit.client.0.smithi144.stdout:4/128: readlink d14/l17 0 2024-09-10T11:29:31.474 INFO:tasks.workunit.client.1.smithi178.stdout:1/521: fdatasync d3/d9/d21/d4b/d2b/f3d 0 2024-09-10T11:29:31.475 INFO:tasks.workunit.client.0.smithi144.stdout:4/129: stat d14/f1f 0 2024-09-10T11:29:31.478 INFO:tasks.workunit.client.1.smithi178.stdout:1/522: symlink d3/d35/lc3 0 2024-09-10T11:29:31.478 INFO:tasks.workunit.client.0.smithi144.stdout:1/208: unlink d5/f3c 0 2024-09-10T11:29:31.478 INFO:tasks.workunit.client.0.smithi144.stdout:2/122: chown d7/da/f14 2944601 1 2024-09-10T11:29:31.478 INFO:tasks.workunit.client.0.smithi144.stdout:3/98: dread d4/dd/fe [0,4194304] 0 2024-09-10T11:29:31.479 INFO:tasks.workunit.client.0.smithi144.stdout:5/180: mkdir d4/d5/d18/d38 0 2024-09-10T11:29:31.481 INFO:tasks.workunit.client.0.smithi144.stdout:9/128: truncate d11/d12/d1b/f27 993982 0 2024-09-10T11:29:31.481 INFO:tasks.workunit.client.0.smithi144.stdout:3/99: stat d4/fb 0 2024-09-10T11:29:31.482 INFO:tasks.workunit.client.0.smithi144.stdout:4/130: creat d14/d1e/d22/f2d x:0 0 0 2024-09-10T11:29:31.483 INFO:tasks.workunit.client.0.smithi144.stdout:7/179: dwrite d5/f8 [0,4194304] 0 2024-09-10T11:29:31.483 INFO:tasks.workunit.client.0.smithi144.stdout:1/209: mknod d5/dd/d22/c3e 0 2024-09-10T11:29:31.484 INFO:tasks.workunit.client.0.smithi144.stdout:3/100: chown d4/dd/l14 2486 1 2024-09-10T11:29:31.489 INFO:tasks.workunit.client.0.smithi144.stdout:7/180: chown d5/dc/d11/c15 0 1 2024-09-10T11:29:31.489 INFO:tasks.workunit.client.1.smithi178.stdout:5/561: dwrite d9/dc/dd/d37/d42/d62/daf/fbf [0,4194304] 0 2024-09-10T11:29:31.491 INFO:tasks.workunit.client.0.smithi144.stdout:5/181: rename d4/d5/l1c to d4/d5/d32/l39 0 2024-09-10T11:29:31.491 INFO:tasks.workunit.client.0.smithi144.stdout:7/181: chown d5/f7 242 1 2024-09-10T11:29:31.495 INFO:tasks.workunit.client.0.smithi144.stdout:3/101: creat d4/dd/d21/f23 x:0 0 0 2024-09-10T11:29:31.497 INFO:tasks.workunit.client.0.smithi144.stdout:9/129: dwrite d11/f1c [0,4194304] 0 2024-09-10T11:29:31.500 INFO:tasks.workunit.client.0.smithi144.stdout:5/182: chown d4/d20/d24/c2a 484 1 2024-09-10T11:29:31.500 INFO:tasks.workunit.client.0.smithi144.stdout:4/131: dwrite d14/d1e/f21 [0,4194304] 0 2024-09-10T11:29:31.502 INFO:tasks.workunit.client.0.smithi144.stdout:9/130: read d11/f17 [2632498,64742] 0 2024-09-10T11:29:31.502 INFO:tasks.workunit.client.0.smithi144.stdout:2/123: dwrite d7/da/d1e/f25 [4194304,4194304] 0 2024-09-10T11:29:31.502 INFO:tasks.workunit.client.0.smithi144.stdout:4/132: read f5 [492358,88635] 0 2024-09-10T11:29:31.504 INFO:tasks.workunit.client.1.smithi178.stdout:1/523: dwrite d3/d35/fb4 [0,4194304] 0 2024-09-10T11:29:31.506 INFO:tasks.workunit.client.1.smithi178.stdout:1/524: stat d3/da0/fc1 0 2024-09-10T11:29:31.512 INFO:tasks.workunit.client.1.smithi178.stdout:1/525: creat d3/d9/fc4 x:0 0 0 2024-09-10T11:29:31.513 INFO:tasks.workunit.client.0.smithi144.stdout:5/183: fsync d4/d5/d32/f33 0 2024-09-10T11:29:31.515 INFO:tasks.workunit.client.0.smithi144.stdout:7/182: dread d5/dc/fe [0,4194304] 0 2024-09-10T11:29:31.516 INFO:tasks.workunit.client.0.smithi144.stdout:9/131: creat d11/d12/d1b/d22/f2f x:0 0 0 2024-09-10T11:29:31.516 INFO:tasks.workunit.client.0.smithi144.stdout:3/102: dwrite d4/fb [0,4194304] 0 2024-09-10T11:29:31.518 INFO:tasks.workunit.client.0.smithi144.stdout:7/183: write d5/f29 [751075,9233] 0 2024-09-10T11:29:31.521 INFO:tasks.workunit.client.0.smithi144.stdout:3/103: truncate d4/dd/f1b 145048 0 2024-09-10T11:29:31.522 INFO:tasks.workunit.client.0.smithi144.stdout:4/133: write f9 [1192430,128949] 0 2024-09-10T11:29:31.523 INFO:tasks.workunit.client.0.smithi144.stdout:5/184: symlink d4/d20/d24/d2d/l3a 0 2024-09-10T11:29:31.527 INFO:tasks.workunit.client.0.smithi144.stdout:9/132: mkdir d11/d12/d1b/d22/d30 0 2024-09-10T11:29:31.527 INFO:tasks.workunit.client.0.smithi144.stdout:5/185: readlink d4/d5/d18/l21 0 2024-09-10T11:29:31.528 INFO:tasks.workunit.client.1.smithi178.stdout:1/526: truncate d3/d9/d1a/f6d 830218 0 2024-09-10T11:29:31.529 INFO:tasks.workunit.client.1.smithi178.stdout:1/527: write d3/d35/f99 [103929,10053] 0 2024-09-10T11:29:31.529 INFO:tasks.workunit.client.0.smithi144.stdout:7/184: stat d5/dc/d12/d2f/c31 0 2024-09-10T11:29:31.530 INFO:tasks.workunit.client.0.smithi144.stdout:4/134: write fa [1389830,40686] 0 2024-09-10T11:29:31.531 INFO:tasks.workunit.client.0.smithi144.stdout:6/121: sync 2024-09-10T11:29:31.531 INFO:tasks.workunit.client.0.smithi144.stdout:8/103: sync 2024-09-10T11:29:31.531 INFO:tasks.workunit.client.0.smithi144.stdout:0/105: sync 2024-09-10T11:29:31.533 INFO:tasks.workunit.client.0.smithi144.stdout:7/185: fdatasync d5/f38 0 2024-09-10T11:29:31.536 INFO:tasks.workunit.client.0.smithi144.stdout:2/124: dwrite d7/da/f17 [0,4194304] 0 2024-09-10T11:29:31.538 INFO:tasks.workunit.client.0.smithi144.stdout:3/104: dread d4/dd/d22/d17/f1f [0,4194304] 0 2024-09-10T11:29:31.539 INFO:tasks.workunit.client.0.smithi144.stdout:1/210: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:31.544 INFO:tasks.workunit.client.0.smithi144.stdout:5/186: dwrite d4/d20/d24/d2d/f2e [0,4194304] 0 2024-09-10T11:29:31.546 INFO:tasks.workunit.client.0.smithi144.stdout:9/133: truncate d11/d12/d1b/d22/f2f 10136 0 2024-09-10T11:29:31.547 INFO:tasks.workunit.client.1.smithi178.stdout:2/725: sync 2024-09-10T11:29:31.549 INFO:tasks.workunit.client.0.smithi144.stdout:6/122: creat d9/dc/d13/f31 x:0 0 0 2024-09-10T11:29:31.550 INFO:tasks.workunit.client.0.smithi144.stdout:0/106: dread f2 [0,4194304] 0 2024-09-10T11:29:31.550 INFO:tasks.workunit.client.0.smithi144.stdout:5/187: mknod d4/d20/d22/c3b 0 2024-09-10T11:29:31.552 INFO:tasks.workunit.client.1.smithi178.stdout:2/726: mknod d1/d7/d24/d46/d88/d9d/dab/d5b/d7c/cec 0 2024-09-10T11:29:31.553 INFO:tasks.workunit.client.0.smithi144.stdout:9/134: write fc [4158731,128040] 0 2024-09-10T11:29:31.554 INFO:tasks.workunit.client.0.smithi144.stdout:4/135: creat d14/d1e/d22/d2b/f2e x:0 0 0 2024-09-10T11:29:31.556 INFO:tasks.workunit.client.0.smithi144.stdout:9/135: dread - d11/f15 zero size 2024-09-10T11:29:31.556 INFO:tasks.workunit.client.0.smithi144.stdout:1/211: truncate d5/fb 2888637 0 2024-09-10T11:29:31.558 INFO:tasks.workunit.client.0.smithi144.stdout:0/107: chown da/c11 13698017 1 2024-09-10T11:29:31.558 INFO:tasks.workunit.client.0.smithi144.stdout:2/125: write d7/da/f18 [56696,123029] 0 2024-09-10T11:29:31.559 INFO:tasks.workunit.client.0.smithi144.stdout:0/108: chown da/f15 10964 1 2024-09-10T11:29:31.559 INFO:tasks.workunit.client.0.smithi144.stdout:8/104: rename d0/d9/d1f/l18 to d0/d9/d21/l27 0 2024-09-10T11:29:31.560 INFO:tasks.workunit.client.0.smithi144.stdout:3/105: dread d4/dd/d22/d17/f1f [0,4194304] 0 2024-09-10T11:29:31.560 INFO:tasks.workunit.client.0.smithi144.stdout:4/136: creat d14/d1e/f2f x:0 0 0 2024-09-10T11:29:31.561 INFO:tasks.workunit.client.0.smithi144.stdout:7/186: creat d5/dc/d11/f3d x:0 0 0 2024-09-10T11:29:31.561 INFO:tasks.workunit.client.0.smithi144.stdout:2/126: write d7/de/f22 [290608,87409] 0 2024-09-10T11:29:31.562 INFO:tasks.workunit.client.0.smithi144.stdout:4/137: dread - d14/d1e/f27 zero size 2024-09-10T11:29:31.562 INFO:tasks.workunit.client.0.smithi144.stdout:0/109: write f0 [109435,83754] 0 2024-09-10T11:29:31.564 INFO:tasks.workunit.client.0.smithi144.stdout:8/105: symlink d0/d1b/l28 0 2024-09-10T11:29:31.565 INFO:tasks.workunit.client.0.smithi144.stdout:8/106: stat d0/d9 0 2024-09-10T11:29:31.567 INFO:tasks.workunit.client.0.smithi144.stdout:9/136: link d11/f2b d11/d12/d1b/d2c/f31 0 2024-09-10T11:29:31.568 INFO:tasks.workunit.client.0.smithi144.stdout:1/212: dread d5/df/f19 [0,4194304] 0 2024-09-10T11:29:31.569 INFO:tasks.workunit.client.0.smithi144.stdout:5/188: truncate d4/f15 477375 0 2024-09-10T11:29:31.571 INFO:tasks.workunit.client.0.smithi144.stdout:5/189: fdatasync d4/d5/f23 0 2024-09-10T11:29:31.571 INFO:tasks.workunit.client.1.smithi178.stdout:2/727: truncate d1/d41/d14/f1b 2656671 0 2024-09-10T11:29:31.573 INFO:tasks.workunit.client.1.smithi178.stdout:2/728: truncate d1/d41/d14/d11/d53/f8b 1705680 0 2024-09-10T11:29:31.575 INFO:tasks.workunit.client.0.smithi144.stdout:3/106: rename d4/dd/d22/c13 to d4/d1a/c24 0 2024-09-10T11:29:31.576 INFO:tasks.workunit.client.0.smithi144.stdout:9/137: truncate d11/d12/d1b/f20 470043 0 2024-09-10T11:29:31.579 INFO:tasks.workunit.client.0.smithi144.stdout:2/127: mknod d7/d29/c2e 0 2024-09-10T11:29:31.579 INFO:tasks.workunit.client.0.smithi144.stdout:4/138: dread f12 [0,4194304] 0 2024-09-10T11:29:31.581 INFO:tasks.workunit.client.0.smithi144.stdout:2/128: chown d7/da 3880369 1 2024-09-10T11:29:31.582 INFO:tasks.workunit.client.0.smithi144.stdout:5/190: mknod d4/d20/d24/d2d/c3c 0 2024-09-10T11:29:31.584 INFO:tasks.workunit.client.0.smithi144.stdout:2/129: write d7/da/f17 [4895679,101570] 0 2024-09-10T11:29:31.584 INFO:tasks.workunit.client.0.smithi144.stdout:0/110: link da/l14 da/l18 0 2024-09-10T11:29:31.586 INFO:tasks.workunit.client.0.smithi144.stdout:1/213: creat d5/dd/f3f x:0 0 0 2024-09-10T11:29:31.588 INFO:tasks.workunit.client.0.smithi144.stdout:7/187: dwrite d5/dc/fe [0,4194304] 0 2024-09-10T11:29:31.589 INFO:tasks.workunit.client.0.smithi144.stdout:8/107: dwrite d0/d9/d1f/f19 [0,4194304] 0 2024-09-10T11:29:31.589 INFO:tasks.workunit.client.0.smithi144.stdout:2/130: fsync d7/de/f1b 0 2024-09-10T11:29:31.590 INFO:tasks.workunit.client.0.smithi144.stdout:9/138: dread f10 [0,4194304] 0 2024-09-10T11:29:31.590 INFO:tasks.workunit.client.1.smithi178.stdout:2/729: dwrite d1/d7/d24/d43/f87 [0,4194304] 0 2024-09-10T11:29:31.591 INFO:tasks.workunit.client.0.smithi144.stdout:1/214: write d5/dd/f3f [499666,5981] 0 2024-09-10T11:29:31.594 INFO:tasks.workunit.client.0.smithi144.stdout:1/215: dread - d5/f15 zero size 2024-09-10T11:29:31.594 INFO:tasks.workunit.client.0.smithi144.stdout:9/139: chown d11/d12/d1b/l23 44976833 1 2024-09-10T11:29:31.595 INFO:tasks.workunit.client.0.smithi144.stdout:3/107: rename d4/dd/l16 to d4/l25 0 2024-09-10T11:29:31.598 INFO:tasks.workunit.client.1.smithi178.stdout:2/730: dwrite d1/d41/d14/d11/f4a [0,4194304] 0 2024-09-10T11:29:31.599 INFO:tasks.workunit.client.0.smithi144.stdout:2/131: creat d7/de/f2f x:0 0 0 2024-09-10T11:29:31.600 INFO:tasks.workunit.client.1.smithi178.stdout:2/731: chown d1/d7/d24/d46/d88/d9d/dab/d5b/d60/fcc 0 1 2024-09-10T11:29:31.601 INFO:tasks.workunit.client.0.smithi144.stdout:5/191: creat d4/d5/f3d x:0 0 0 2024-09-10T11:29:31.605 INFO:tasks.workunit.client.0.smithi144.stdout:7/188: symlink d5/dc/l3e 0 2024-09-10T11:29:31.606 INFO:tasks.workunit.client.0.smithi144.stdout:1/216: fsync d5/dd/f1d 0 2024-09-10T11:29:31.607 INFO:tasks.workunit.client.0.smithi144.stdout:2/132: chown d7/da/l24 1448945 1 2024-09-10T11:29:31.608 INFO:tasks.workunit.client.0.smithi144.stdout:4/139: rename d14/d1e/l29 to d14/d1e/d22/d2b/l30 0 2024-09-10T11:29:31.610 INFO:tasks.workunit.client.0.smithi144.stdout:0/111: dwrite da/f16 [0,4194304] 0 2024-09-10T11:29:31.610 INFO:tasks.workunit.client.0.smithi144.stdout:1/217: fdatasync d5/f6 0 2024-09-10T11:29:31.611 INFO:tasks.workunit.client.0.smithi144.stdout:4/140: chown d14/d1e/d22 1 1 2024-09-10T11:29:31.612 INFO:tasks.workunit.client.0.smithi144.stdout:7/189: write d5/f7 [404960,90216] 0 2024-09-10T11:29:31.613 INFO:tasks.workunit.client.0.smithi144.stdout:8/108: dwrite d0/fc [0,4194304] 0 2024-09-10T11:29:31.613 INFO:tasks.workunit.client.0.smithi144.stdout:0/112: readlink da/l17 0 2024-09-10T11:29:31.615 INFO:tasks.workunit.client.1.smithi178.stdout:2/732: dwrite d1/d7/d24/f2c [0,4194304] 0 2024-09-10T11:29:31.615 INFO:tasks.workunit.client.0.smithi144.stdout:9/140: rmdir d11/d12/d1b/d22/d30 0 2024-09-10T11:29:31.618 INFO:tasks.workunit.client.1.smithi178.stdout:2/733: fsync d1/d7/d24/d46/d88/d9d/dab/d5e/f96 0 2024-09-10T11:29:31.618 INFO:tasks.workunit.client.0.smithi144.stdout:4/141: dread - d14/f1f zero size 2024-09-10T11:29:31.618 INFO:tasks.workunit.client.0.smithi144.stdout:1/218: write d5/f23 [45817,22874] 0 2024-09-10T11:29:31.621 INFO:tasks.workunit.client.1.smithi178.stdout:2/734: fsync d1/d7/d24/d46/d88/d9d/dab/f15 0 2024-09-10T11:29:31.621 INFO:tasks.workunit.client.0.smithi144.stdout:0/113: rmdir da 39 2024-09-10T11:29:31.623 INFO:tasks.workunit.client.0.smithi144.stdout:2/133: creat d7/da/d1e/d21/f30 x:0 0 0 2024-09-10T11:29:31.624 INFO:tasks.workunit.client.0.smithi144.stdout:3/108: dwrite d4/dd/d21/f23 [0,4194304] 0 2024-09-10T11:29:31.627 INFO:tasks.workunit.client.0.smithi144.stdout:1/219: creat d5/d2a/f40 x:0 0 0 2024-09-10T11:29:31.628 INFO:tasks.workunit.client.0.smithi144.stdout:4/142: creat d14/d1e/d22/f31 x:0 0 0 2024-09-10T11:29:31.628 INFO:tasks.workunit.client.1.smithi178.stdout:2/735: mknod d1/d7/d24/d46/d88/d9d/dab/d5e/ced 0 2024-09-10T11:29:31.629 INFO:tasks.workunit.client.1.smithi178.stdout:2/736: stat d1/d7/d24/lbf 0 2024-09-10T11:29:31.629 INFO:tasks.workunit.client.0.smithi144.stdout:1/220: stat d5/dd/d22/f24 0 2024-09-10T11:29:31.630 INFO:tasks.workunit.client.0.smithi144.stdout:9/141: mknod d11/d12/d1b/c32 0 2024-09-10T11:29:31.632 INFO:tasks.workunit.client.0.smithi144.stdout:7/190: symlink d5/dc/d2a/l3f 0 2024-09-10T11:29:31.634 INFO:tasks.workunit.client.0.smithi144.stdout:3/109: unlink d4/l9 0 2024-09-10T11:29:31.636 INFO:tasks.workunit.client.0.smithi144.stdout:8/109: creat d0/d9/d21/d25/f29 x:0 0 0 2024-09-10T11:29:31.636 INFO:tasks.workunit.client.1.smithi178.stdout:2/737: creat d1/d41/fee x:0 0 0 2024-09-10T11:29:31.637 INFO:tasks.workunit.client.0.smithi144.stdout:8/110: chown d0/d9 164704 1 2024-09-10T11:29:31.637 INFO:tasks.workunit.client.0.smithi144.stdout:2/134: dwrite d7/de/f16 [0,4194304] 0 2024-09-10T11:29:31.639 INFO:tasks.workunit.client.1.smithi178.stdout:2/738: mkdir d1/d41/d14/d11/db9/def 0 2024-09-10T11:29:31.641 INFO:tasks.workunit.client.0.smithi144.stdout:7/191: mkdir d5/dc/d14/d40 0 2024-09-10T11:29:31.642 INFO:tasks.workunit.client.0.smithi144.stdout:8/111: creat d0/d1b/f2a x:0 0 0 2024-09-10T11:29:31.644 INFO:tasks.workunit.client.0.smithi144.stdout:2/135: mknod d7/da/d1e/c31 0 2024-09-10T11:29:31.644 INFO:tasks.workunit.client.1.smithi178.stdout:2/739: creat d1/d7/d24/d7d/d84/d4b/d89/ff0 x:0 0 0 2024-09-10T11:29:31.646 INFO:tasks.workunit.client.0.smithi144.stdout:7/192: mkdir d5/dc/d2a/d41 0 2024-09-10T11:29:31.647 INFO:tasks.workunit.client.0.smithi144.stdout:7/193: fsync d5/f7 0 2024-09-10T11:29:31.647 INFO:tasks.workunit.client.0.smithi144.stdout:7/194: fsync d5/dc/d11/f26 0 2024-09-10T11:29:31.649 INFO:tasks.workunit.client.0.smithi144.stdout:0/114: truncate f0 3325525 0 2024-09-10T11:29:31.650 INFO:tasks.workunit.client.0.smithi144.stdout:2/136: mknod d7/c32 0 2024-09-10T11:29:31.651 INFO:tasks.workunit.client.0.smithi144.stdout:2/137: write d7/da/f17 [1775674,23729] 0 2024-09-10T11:29:31.651 INFO:tasks.workunit.client.0.smithi144.stdout:1/221: rmdir d5/dd/d3b 0 2024-09-10T11:29:31.654 INFO:tasks.workunit.client.0.smithi144.stdout:7/195: rmdir d5/dc/d12/d2b 39 2024-09-10T11:29:31.656 INFO:tasks.workunit.client.0.smithi144.stdout:3/110: truncate d4/dd/d21/f23 3336715 0 2024-09-10T11:29:31.657 INFO:tasks.workunit.client.1.smithi178.stdout:4/443: sync 2024-09-10T11:29:31.657 INFO:tasks.workunit.client.1.smithi178.stdout:9/638: sync 2024-09-10T11:29:31.657 INFO:tasks.workunit.client.1.smithi178.stdout:3/514: sync 2024-09-10T11:29:31.657 INFO:tasks.workunit.client.1.smithi178.stdout:8/610: sync 2024-09-10T11:29:31.657 INFO:tasks.workunit.client.1.smithi178.stdout:7/485: sync 2024-09-10T11:29:31.658 INFO:tasks.workunit.client.1.smithi178.stdout:6/599: sync 2024-09-10T11:29:31.658 INFO:tasks.workunit.client.1.smithi178.stdout:0/580: sync 2024-09-10T11:29:31.659 INFO:tasks.workunit.client.0.smithi144.stdout:1/222: rename d5/dd/f21 to d5/f41 0 2024-09-10T11:29:31.659 INFO:tasks.workunit.client.1.smithi178.stdout:7/486: stat d6/d10/d34/f8d 0 2024-09-10T11:29:31.659 INFO:tasks.workunit.client.0.smithi144.stdout:9/142: dwrite d11/d12/d1b/d2c/f31 [0,4194304] 0 2024-09-10T11:29:31.660 INFO:tasks.workunit.client.0.smithi144.stdout:3/111: dread d4/dd/f1b [0,4194304] 0 2024-09-10T11:29:31.661 INFO:tasks.workunit.client.1.smithi178.stdout:6/600: stat d3/d7/d8/d15/fbc 0 2024-09-10T11:29:31.661 INFO:tasks.workunit.client.0.smithi144.stdout:2/138: unlink d7/da/d1e/c2c 0 2024-09-10T11:29:31.662 INFO:tasks.workunit.client.1.smithi178.stdout:8/611: write d5/d30/d33/f49 [2403345,86459] 0 2024-09-10T11:29:31.663 INFO:tasks.workunit.client.1.smithi178.stdout:0/581: mknod d0/d2c/d33/cbe 0 2024-09-10T11:29:31.663 INFO:tasks.workunit.client.1.smithi178.stdout:7/487: readlink d6/d10/d1d/l9d 0 2024-09-10T11:29:31.665 INFO:tasks.workunit.client.0.smithi144.stdout:4/143: dwrite fd [0,4194304] 0 2024-09-10T11:29:31.667 INFO:tasks.workunit.client.1.smithi178.stdout:4/444: creat d0/d9/dc/d8e/f9f x:0 0 0 2024-09-10T11:29:31.667 INFO:tasks.workunit.client.0.smithi144.stdout:4/144: fsync d14/d1e/f27 0 2024-09-10T11:29:31.669 INFO:tasks.workunit.client.1.smithi178.stdout:7/488: creat d6/d10/fb3 x:0 0 0 2024-09-10T11:29:31.669 INFO:tasks.workunit.client.0.smithi144.stdout:7/196: dread d5/dc/f10 [0,4194304] 0 2024-09-10T11:29:31.670 INFO:tasks.workunit.client.1.smithi178.stdout:7/489: chown d6/d10/d34/d6a/d6f/faf 176702 1 2024-09-10T11:29:31.674 INFO:tasks.workunit.client.0.smithi144.stdout:8/112: truncate d0/d9/d1f/f19 2223124 0 2024-09-10T11:29:31.676 INFO:tasks.workunit.client.1.smithi178.stdout:5/562: sync 2024-09-10T11:29:31.676 INFO:tasks.workunit.client.1.smithi178.stdout:1/528: sync 2024-09-10T11:29:31.676 INFO:tasks.workunit.client.0.smithi144.stdout:2/139: dwrite d7/da/f18 [0,4194304] 0 2024-09-10T11:29:31.678 INFO:tasks.workunit.client.1.smithi178.stdout:6/601: creat d3/d74/dc9/da6/fdd x:0 0 0 2024-09-10T11:29:31.680 INFO:tasks.workunit.client.0.smithi144.stdout:3/112: mknod d4/dd/d21/c26 0 2024-09-10T11:29:31.681 INFO:tasks.workunit.client.1.smithi178.stdout:5/563: write d9/d11/fa2 [379370,50942] 0 2024-09-10T11:29:31.682 INFO:tasks.workunit.client.0.smithi144.stdout:4/145: creat d14/d1e/f32 x:0 0 0 2024-09-10T11:29:31.683 INFO:tasks.workunit.client.1.smithi178.stdout:0/582: creat d0/fbf x:0 0 0 2024-09-10T11:29:31.684 INFO:tasks.workunit.client.0.smithi144.stdout:4/146: dread - d14/d1e/d22/f23 zero size 2024-09-10T11:29:31.685 INFO:tasks.workunit.client.0.smithi144.stdout:9/143: unlink d11/l1a 0 2024-09-10T11:29:31.685 INFO:tasks.workunit.client.0.smithi144.stdout:0/115: dwrite da/f15 [0,4194304] 0 2024-09-10T11:29:31.686 INFO:tasks.workunit.client.1.smithi178.stdout:7/490: dwrite d6/d10/d61/f84 [0,4194304] 0 2024-09-10T11:29:31.686 INFO:tasks.workunit.client.1.smithi178.stdout:6/602: write d3/d7/d8/d15/d1c/d2a/d18/f70 [262264,24570] 0 2024-09-10T11:29:31.687 INFO:tasks.workunit.client.0.smithi144.stdout:9/144: read - d11/d12/d1b/f24 zero size 2024-09-10T11:29:31.688 INFO:tasks.workunit.client.1.smithi178.stdout:0/583: chown d0/dc/d39/da4 3 1 2024-09-10T11:29:31.689 INFO:tasks.workunit.client.0.smithi144.stdout:4/147: truncate d14/d1e/d22/f23 182182 0 2024-09-10T11:29:31.689 INFO:tasks.workunit.client.1.smithi178.stdout:3/515: link db/d20/d3d/d63/f6f db/d20/d3d/d63/d6a/f9d 0 2024-09-10T11:29:31.690 INFO:tasks.workunit.client.0.smithi144.stdout:7/197: symlink d5/dc/d14/d40/l42 0 2024-09-10T11:29:31.692 INFO:tasks.workunit.client.1.smithi178.stdout:8/612: rename d5/d1d/l73 to d5/db/d10/lca 0 2024-09-10T11:29:31.693 INFO:tasks.workunit.client.1.smithi178.stdout:9/639: dread d2/d19/f8e [4194304,4194304] 0 2024-09-10T11:29:31.693 INFO:tasks.workunit.client.1.smithi178.stdout:2/740: truncate d1/d7/d24/d46/d88/fcd 1480853 0 2024-09-10T11:29:31.695 INFO:tasks.workunit.client.0.smithi144.stdout:3/113: creat d4/dd/d22/d17/d18/f27 x:0 0 0 2024-09-10T11:29:31.695 INFO:tasks.workunit.client.0.smithi144.stdout:0/116: dread da/f16 [0,4194304] 0 2024-09-10T11:29:31.696 INFO:tasks.workunit.client.0.smithi144.stdout:1/223: getdents d5/d2a 0 2024-09-10T11:29:31.702 INFO:tasks.workunit.client.0.smithi144.stdout:0/117: dread da/f16 [0,4194304] 0 2024-09-10T11:29:31.702 INFO:tasks.workunit.client.0.smithi144.stdout:9/145: symlink d11/d12/d1b/d22/l33 0 2024-09-10T11:29:31.702 INFO:tasks.workunit.client.1.smithi178.stdout:6/603: readlink d3/d7/d35/l60 0 2024-09-10T11:29:31.704 INFO:tasks.workunit.client.1.smithi178.stdout:2/741: dwrite d1/d7/d24/d7d/d84/d4b/f67 [0,4194304] 0 2024-09-10T11:29:31.705 INFO:tasks.workunit.client.0.smithi144.stdout:3/114: mkdir d4/d28 0 2024-09-10T11:29:31.705 INFO:tasks.workunit.client.1.smithi178.stdout:3/516: mknod db/c9e 0 2024-09-10T11:29:31.705 INFO:tasks.workunit.client.1.smithi178.stdout:3/517: readlink l9 0 2024-09-10T11:29:31.706 INFO:tasks.workunit.client.0.smithi144.stdout:1/224: dread d5/f6 [0,4194304] 0 2024-09-10T11:29:31.706 INFO:tasks.workunit.client.1.smithi178.stdout:3/518: fdatasync db/f72 0 2024-09-10T11:29:31.707 INFO:tasks.workunit.client.1.smithi178.stdout:8/613: write d5/d30/f87 [949773,35886] 0 2024-09-10T11:29:31.707 INFO:tasks.workunit.client.0.smithi144.stdout:9/146: mknod d11/d2a/c34 0 2024-09-10T11:29:31.708 INFO:tasks.workunit.client.0.smithi144.stdout:2/140: truncate d7/da/d1e/d21/f2a 305638 0 2024-09-10T11:29:31.708 INFO:tasks.workunit.client.1.smithi178.stdout:0/584: dread d0/dc/d39/d50/f1d [0,4194304] 0 2024-09-10T11:29:31.709 INFO:tasks.workunit.client.1.smithi178.stdout:7/491: rename d6/d10/d1e/d2d/f66 to d6/d10/d1d/d22/d50/d98/da6/fb4 0 2024-09-10T11:29:31.709 INFO:tasks.workunit.client.0.smithi144.stdout:2/141: dread - d7/de/f2f zero size 2024-09-10T11:29:31.710 INFO:tasks.workunit.client.1.smithi178.stdout:1/529: creat d3/d9/d98/dac/d3a/d9f/fc5 x:0 0 0 2024-09-10T11:29:31.712 INFO:tasks.workunit.client.1.smithi178.stdout:4/445: link d0/d8/da/d1a/l41 d0/d9/dc/d8a/la0 0 2024-09-10T11:29:31.713 INFO:tasks.workunit.client.0.smithi144.stdout:4/148: link d14/f1d d14/d1e/d22/d2b/f33 0 2024-09-10T11:29:31.714 INFO:tasks.workunit.client.1.smithi178.stdout:6/604: symlink d3/d74/d91/daa/dc2/d57/d78/lde 0 2024-09-10T11:29:31.716 INFO:tasks.workunit.client.1.smithi178.stdout:2/742: dread d1/d7/d24/d7d/d84/d4b/d89/fa3 [0,4194304] 0 2024-09-10T11:29:31.717 INFO:tasks.workunit.client.1.smithi178.stdout:8/614: fsync d5/db/d3d/f97 0 2024-09-10T11:29:31.717 INFO:tasks.workunit.client.0.smithi144.stdout:4/149: mkdir d14/d1e/d22/d34 0 2024-09-10T11:29:31.718 INFO:tasks.workunit.client.1.smithi178.stdout:3/519: chown db/l1c 63140 1 2024-09-10T11:29:31.718 INFO:tasks.workunit.client.0.smithi144.stdout:3/115: unlink d4/l25 0 2024-09-10T11:29:31.719 INFO:tasks.workunit.client.1.smithi178.stdout:8/615: dread - d5/db7/fbf zero size 2024-09-10T11:29:31.719 INFO:tasks.workunit.client.0.smithi144.stdout:1/225: mknod d5/dd/d20/d3d/c42 0 2024-09-10T11:29:31.720 INFO:tasks.workunit.client.1.smithi178.stdout:0/585: dwrite d0/dc/f61 [0,4194304] 0 2024-09-10T11:29:31.720 INFO:tasks.workunit.client.1.smithi178.stdout:1/530: mkdir d3/d35/dc6 0 2024-09-10T11:29:31.722 INFO:tasks.workunit.client.1.smithi178.stdout:7/492: fsync d6/d10/d34/f6e 0 2024-09-10T11:29:31.723 INFO:tasks.workunit.client.0.smithi144.stdout:9/147: dread fc [0,4194304] 0 2024-09-10T11:29:31.724 INFO:tasks.workunit.client.0.smithi144.stdout:0/118: dwrite da/f16 [0,4194304] 0 2024-09-10T11:29:31.725 INFO:tasks.workunit.client.1.smithi178.stdout:5/564: creat d9/dc/dd/fc1 x:0 0 0 2024-09-10T11:29:31.726 INFO:tasks.workunit.client.0.smithi144.stdout:3/116: creat d4/dd/d22/d1d/f29 x:0 0 0 2024-09-10T11:29:31.729 INFO:tasks.workunit.client.0.smithi144.stdout:2/142: dwrite f6 [0,4194304] 0 2024-09-10T11:29:31.729 INFO:tasks.workunit.client.0.smithi144.stdout:1/226: truncate d5/f15 388741 0 2024-09-10T11:29:31.733 INFO:tasks.workunit.client.0.smithi144.stdout:2/143: read d7/f10 [2787412,89790] 0 2024-09-10T11:29:31.736 INFO:tasks.workunit.client.0.smithi144.stdout:9/148: mknod d11/c35 0 2024-09-10T11:29:31.737 INFO:tasks.workunit.client.0.smithi144.stdout:1/227: creat d5/dd/d22/f43 x:0 0 0 2024-09-10T11:29:31.738 INFO:tasks.workunit.client.0.smithi144.stdout:4/150: dwrite d14/d1e/f1c [0,4194304] 0 2024-09-10T11:29:31.740 INFO:tasks.workunit.client.0.smithi144.stdout:1/228: write d5/d2a/f2d [419786,115494] 0 2024-09-10T11:29:31.742 INFO:tasks.workunit.client.0.smithi144.stdout:4/151: write f9 [3800110,124503] 0 2024-09-10T11:29:31.744 INFO:tasks.workunit.client.0.smithi144.stdout:3/117: dwrite d4/fb [0,4194304] 0 2024-09-10T11:29:31.744 INFO:tasks.workunit.client.0.smithi144.stdout:9/149: write d11/f17 [1313119,10265] 0 2024-09-10T11:29:31.749 INFO:tasks.workunit.client.1.smithi178.stdout:7/493: symlink d6/d10/lb5 0 2024-09-10T11:29:31.749 INFO:tasks.workunit.client.0.smithi144.stdout:9/150: fdatasync d11/f15 0 2024-09-10T11:29:31.749 INFO:tasks.workunit.client.0.smithi144.stdout:4/152: dread - d14/d1e/d22/f31 zero size 2024-09-10T11:29:31.750 INFO:tasks.workunit.client.0.smithi144.stdout:9/151: fsync d11/f2b 0 2024-09-10T11:29:31.750 INFO:tasks.workunit.client.1.smithi178.stdout:7/494: write d6/d10/d1d/d4a/f62 [1280659,14820] 0 2024-09-10T11:29:31.752 INFO:tasks.workunit.client.0.smithi144.stdout:3/118: write f2 [3238832,45554] 0 2024-09-10T11:29:31.754 INFO:tasks.workunit.client.0.smithi144.stdout:1/229: mkdir d5/dd/d20/d44 0 2024-09-10T11:29:31.755 INFO:tasks.workunit.client.0.smithi144.stdout:3/119: fdatasync d4/dd/d22/d17/d18/f27 0 2024-09-10T11:29:31.755 INFO:tasks.workunit.client.0.smithi144.stdout:1/230: chown d5/d2a 109913987 1 2024-09-10T11:29:31.756 INFO:tasks.workunit.client.0.smithi144.stdout:3/120: chown d4/d1a 23 1 2024-09-10T11:29:31.757 INFO:tasks.workunit.client.0.smithi144.stdout:1/231: chown d5/dd/f1d 343265 1 2024-09-10T11:29:31.759 INFO:tasks.workunit.client.1.smithi178.stdout:4/446: creat d0/d8/da/d1a/d51/d5d/fa1 x:0 0 0 2024-09-10T11:29:31.759 INFO:tasks.workunit.client.1.smithi178.stdout:6/605: link d3/d4e/c6b d3/d6e/d87/cdf 0 2024-09-10T11:29:31.759 INFO:tasks.workunit.client.0.smithi144.stdout:3/121: creat d4/dd/d22/d1d/f2a x:0 0 0 2024-09-10T11:29:31.760 INFO:tasks.workunit.client.1.smithi178.stdout:6/606: chown d3/d74 21 1 2024-09-10T11:29:31.760 INFO:tasks.workunit.client.0.smithi144.stdout:9/152: dread d11/f1c [0,4194304] 0 2024-09-10T11:29:31.761 INFO:tasks.workunit.client.1.smithi178.stdout:1/531: creat d3/d9/d98/dac/d3a/fc7 x:0 0 0 2024-09-10T11:29:31.761 INFO:tasks.workunit.client.0.smithi144.stdout:3/122: read - d4/dd/d22/d1d/f2a zero size 2024-09-10T11:29:31.761 INFO:tasks.workunit.client.0.smithi144.stdout:1/232: creat d5/dd/d20/d44/f45 x:0 0 0 2024-09-10T11:29:31.762 INFO:tasks.workunit.client.1.smithi178.stdout:1/532: write d3/d35/f74 [411167,10137] 0 2024-09-10T11:29:31.762 INFO:tasks.workunit.client.0.smithi144.stdout:9/153: fsync f10 0 2024-09-10T11:29:31.763 INFO:tasks.workunit.client.0.smithi144.stdout:1/233: chown d5/dd/f1d 58098 1 2024-09-10T11:29:31.763 INFO:tasks.workunit.client.1.smithi178.stdout:1/533: stat d3/d9/d98/dac/f7f 0 2024-09-10T11:29:31.764 INFO:tasks.workunit.client.0.smithi144.stdout:1/234: read - d5/dd/d20/f39 zero size 2024-09-10T11:29:31.766 INFO:tasks.workunit.client.0.smithi144.stdout:3/123: rename d4/l6 to d4/dd/d22/d1d/l2b 0 2024-09-10T11:29:31.766 INFO:tasks.workunit.client.1.smithi178.stdout:1/534: fdatasync d3/d9/d98/dac/d3a/d63/d7e/f8c 0 2024-09-10T11:29:31.766 INFO:tasks.workunit.client.1.smithi178.stdout:6/607: getdents d3/d74/dc9/ddc 0 2024-09-10T11:29:31.767 INFO:tasks.workunit.client.0.smithi144.stdout:9/154: rename d11/d2a to d11/d36 0 2024-09-10T11:29:31.767 INFO:tasks.workunit.client.0.smithi144.stdout:9/155: fdatasync d11/d12/d1b/f27 0 2024-09-10T11:29:31.769 INFO:tasks.workunit.client.0.smithi144.stdout:3/124: mknod d4/d28/c2c 0 2024-09-10T11:29:31.770 INFO:tasks.workunit.client.1.smithi178.stdout:1/535: symlink d3/d9/d98/dac/lc8 0 2024-09-10T11:29:31.770 INFO:tasks.workunit.client.0.smithi144.stdout:3/125: chown d4/fb 2 1 2024-09-10T11:29:31.771 INFO:tasks.workunit.client.1.smithi178.stdout:9/640: dwrite d2/df/d31/d37/fa3 [0,4194304] 0 2024-09-10T11:29:31.772 INFO:tasks.workunit.client.1.smithi178.stdout:1/536: write d3/d9/d98/dac/f28 [848237,76359] 0 2024-09-10T11:29:31.773 INFO:tasks.workunit.client.1.smithi178.stdout:6/608: unlink d3/d7/d8/d15/l99 0 2024-09-10T11:29:31.775 INFO:tasks.workunit.client.0.smithi144.stdout:1/235: truncate d5/f12 1208989 0 2024-09-10T11:29:31.777 INFO:tasks.workunit.client.1.smithi178.stdout:1/537: creat d3/d9/d98/dac/d3a/d50/db5/fc9 x:0 0 0 2024-09-10T11:29:31.778 INFO:tasks.workunit.client.0.smithi144.stdout:1/236: stat d5/dd/d20/d44/f45 0 2024-09-10T11:29:31.779 INFO:tasks.workunit.client.0.smithi144.stdout:0/119: fsync da/f16 0 2024-09-10T11:29:31.779 INFO:tasks.workunit.client.0.smithi144.stdout:4/153: fdatasync d14/d1e/f1c 0 2024-09-10T11:29:31.779 INFO:tasks.workunit.client.1.smithi178.stdout:1/538: chown d3/d9/d98/dac/l65 43 1 2024-09-10T11:29:31.780 INFO:tasks.workunit.client.1.smithi178.stdout:2/743: write d1/d7/d24/d7d/fa0 [887651,119049] 0 2024-09-10T11:29:31.782 INFO:tasks.workunit.client.0.smithi144.stdout:4/154: dread - d14/d1e/f20 zero size 2024-09-10T11:29:31.782 INFO:tasks.workunit.client.0.smithi144.stdout:3/126: dwrite d4/dd/d22/d1d/f2a [0,4194304] 0 2024-09-10T11:29:31.782 INFO:tasks.workunit.client.0.smithi144.stdout:9/156: dwrite d11/d12/d1b/f20 [0,4194304] 0 2024-09-10T11:29:31.784 INFO:tasks.workunit.client.0.smithi144.stdout:3/127: read d4/fb [2942787,124993] 0 2024-09-10T11:29:31.785 INFO:tasks.workunit.client.1.smithi178.stdout:5/565: truncate d9/d19/f67 3814678 0 2024-09-10T11:29:31.785 INFO:tasks.workunit.client.0.smithi144.stdout:4/155: write d14/f1f [405859,118787] 0 2024-09-10T11:29:31.786 INFO:tasks.workunit.client.0.smithi144.stdout:0/120: creat da/f19 x:0 0 0 2024-09-10T11:29:31.786 INFO:tasks.workunit.client.1.smithi178.stdout:2/744: chown d1/d22/d4d/d9b/f75 99476 1 2024-09-10T11:29:31.787 INFO:tasks.workunit.client.0.smithi144.stdout:1/237: rename d5/dd/d20/f39 to d5/d2a/f46 0 2024-09-10T11:29:31.791 INFO:tasks.workunit.client.1.smithi178.stdout:4/447: write d0/d9/f81 [3837221,16354] 0 2024-09-10T11:29:31.791 INFO:tasks.workunit.client.0.smithi144.stdout:9/157: mknod d11/d12/d1b/d22/c37 0 2024-09-10T11:29:31.795 INFO:tasks.workunit.client.1.smithi178.stdout:1/539: getdents d3/d9/d98/dac/d3a/d50/d9e/dc2 0 2024-09-10T11:29:31.796 INFO:tasks.workunit.client.0.smithi144.stdout:3/128: chown d4/dd/l11 10923539 1 2024-09-10T11:29:31.796 INFO:tasks.workunit.client.0.smithi144.stdout:3/129: chown d4/dd/l15 47505 1 2024-09-10T11:29:31.797 INFO:tasks.workunit.client.1.smithi178.stdout:9/641: dread d2/d13/f43 [0,4194304] 0 2024-09-10T11:29:31.798 INFO:tasks.workunit.client.1.smithi178.stdout:5/566: fdatasync d9/dc/dd/d37/d3e/f78 0 2024-09-10T11:29:31.800 INFO:tasks.workunit.client.1.smithi178.stdout:3/520: dwrite db/d35/f77 [0,4194304] 0 2024-09-10T11:29:31.800 INFO:tasks.workunit.client.0.smithi144.stdout:0/121: chown da/l18 76 1 2024-09-10T11:29:31.800 INFO:tasks.workunit.client.1.smithi178.stdout:0/586: dwrite d0/dc/d39/f9d [4194304,4194304] 0 2024-09-10T11:29:31.801 INFO:tasks.workunit.client.0.smithi144.stdout:4/156: mkdir d14/d1e/d22/d2b/d35 0 2024-09-10T11:29:31.805 INFO:tasks.workunit.client.0.smithi144.stdout:1/238: mknod d5/dd/d22/c47 0 2024-09-10T11:29:31.806 INFO:tasks.workunit.client.0.smithi144.stdout:9/158: rmdir d11/d36 39 2024-09-10T11:29:31.808 INFO:tasks.workunit.client.0.smithi144.stdout:0/122: creat da/f1a x:0 0 0 2024-09-10T11:29:31.809 INFO:tasks.workunit.client.1.smithi178.stdout:9/642: rmdir d2/d13/d75 39 2024-09-10T11:29:31.810 INFO:tasks.workunit.client.0.smithi144.stdout:3/130: dwrite d4/dd/d22/d1d/f29 [0,4194304] 0 2024-09-10T11:29:31.811 INFO:tasks.workunit.client.1.smithi178.stdout:0/587: chown d0/dc/l10 451791 1 2024-09-10T11:29:31.813 INFO:tasks.workunit.client.1.smithi178.stdout:4/448: readlink d0/d9/dc/d8a/la0 0 2024-09-10T11:29:31.814 INFO:tasks.workunit.client.0.smithi144.stdout:4/157: dwrite f9 [0,4194304] 0 2024-09-10T11:29:31.814 INFO:tasks.workunit.client.1.smithi178.stdout:3/521: read db/d26/d40/f62 [144706,30386] 0 2024-09-10T11:29:31.816 INFO:tasks.workunit.client.0.smithi144.stdout:1/239: unlink d5/d2a/f40 0 2024-09-10T11:29:31.816 INFO:tasks.workunit.client.0.smithi144.stdout:1/240: readlink d5/df/l3a 0 2024-09-10T11:29:31.817 INFO:tasks.workunit.client.1.smithi178.stdout:7/495: dwrite d6/d10/d61/f77 [0,4194304] 0 2024-09-10T11:29:31.818 INFO:tasks.workunit.client.1.smithi178.stdout:1/540: getdents d3/d9/d21 0 2024-09-10T11:29:31.821 INFO:tasks.workunit.client.0.smithi144.stdout:9/159: mkdir d11/d36/d2d/d38 0 2024-09-10T11:29:31.821 INFO:tasks.workunit.client.0.smithi144.stdout:9/160: fdatasync d11/d12/d1b/f20 0 2024-09-10T11:29:31.821 INFO:tasks.workunit.client.1.smithi178.stdout:6/609: dwrite d3/d74/dc9/fb8 [0,4194304] 0 2024-09-10T11:29:31.822 INFO:tasks.workunit.client.0.smithi144.stdout:9/161: read d11/d12/d1b/d22/f2f [9560,50440] 0 2024-09-10T11:29:31.824 INFO:tasks.workunit.client.0.smithi144.stdout:4/158: symlink d14/l36 0 2024-09-10T11:29:31.824 INFO:tasks.workunit.client.1.smithi178.stdout:2/745: write d1/d7/d24/d7d/d84/d4b/f57 [4407823,99727] 0 2024-09-10T11:29:31.829 INFO:tasks.workunit.client.0.smithi144.stdout:9/162: dwrite d11/f15 [0,4194304] 0 2024-09-10T11:29:31.833 INFO:tasks.workunit.client.1.smithi178.stdout:2/746: fdatasync d1/d7/d24/d7d/d84/f36 0 2024-09-10T11:29:31.834 INFO:tasks.workunit.client.1.smithi178.stdout:3/522: mkdir db/d35/d3a/d9f 0 2024-09-10T11:29:31.837 INFO:tasks.workunit.client.1.smithi178.stdout:4/449: rename d0/d8/da/d1a/d51/f9c to d0/d10/fa2 0 2024-09-10T11:29:31.839 INFO:tasks.workunit.client.1.smithi178.stdout:7/496: mknod d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d70/da2/cb6 0 2024-09-10T11:29:31.840 INFO:tasks.workunit.client.1.smithi178.stdout:4/450: read - d0/d9/d3d/d4f/f97 zero size 2024-09-10T11:29:31.843 INFO:tasks.workunit.client.0.smithi144.stdout:1/241: creat d5/dd/f48 x:0 0 0 2024-09-10T11:29:31.844 INFO:tasks.workunit.client.1.smithi178.stdout:3/523: mknod db/d20/ca0 0 2024-09-10T11:29:31.844 INFO:tasks.workunit.client.1.smithi178.stdout:7/497: mkdir d6/d10/d1d/d22/d93/db7 0 2024-09-10T11:29:31.847 INFO:tasks.workunit.client.1.smithi178.stdout:4/451: rename d0/d8/d3e/c8d to d0/d8/df/ca3 0 2024-09-10T11:29:31.847 INFO:tasks.workunit.client.1.smithi178.stdout:1/541: dread d3/d9/d21/d4b/d2b/d5f/f85 [0,4194304] 0 2024-09-10T11:29:31.849 INFO:tasks.workunit.client.1.smithi178.stdout:2/747: link d1/ldb d1/d7/d8e/de9/lf1 0 2024-09-10T11:29:31.849 INFO:tasks.workunit.client.0.smithi144.stdout:9/163: symlink d11/d36/d2d/d38/l39 0 2024-09-10T11:29:31.850 INFO:tasks.workunit.client.1.smithi178.stdout:9/643: truncate d2/df/d31/d37/f5f 2076928 0 2024-09-10T11:29:31.851 INFO:tasks.workunit.client.0.smithi144.stdout:9/164: creat d11/d12/d1b/d22/f3a x:0 0 0 2024-09-10T11:29:31.852 INFO:tasks.workunit.client.1.smithi178.stdout:3/524: symlink db/d20/d3e/d6b/d7e/d98/la1 0 2024-09-10T11:29:31.852 INFO:tasks.workunit.client.0.smithi144.stdout:0/123: write f2 [878434,74696] 0 2024-09-10T11:29:31.854 INFO:tasks.workunit.client.1.smithi178.stdout:7/498: creat d6/d10/d1d/fb8 x:0 0 0 2024-09-10T11:29:31.857 INFO:tasks.workunit.client.1.smithi178.stdout:2/748: creat d1/d7/d24/d46/d88/dc6/ff2 x:0 0 0 2024-09-10T11:29:31.858 INFO:tasks.workunit.client.0.smithi144.stdout:9/165: symlink d11/d12/d1b/d22/l3b 0 2024-09-10T11:29:31.859 INFO:tasks.workunit.client.1.smithi178.stdout:5/567: dread d9/dc/dd/d3f/f49 [0,4194304] 0 2024-09-10T11:29:31.860 INFO:tasks.workunit.client.0.smithi144.stdout:1/242: getdents d5/dd/d20/d44 0 2024-09-10T11:29:31.861 INFO:tasks.workunit.client.0.smithi144.stdout:3/131: dwrite d4/dd/f1b [0,4194304] 0 2024-09-10T11:29:31.861 INFO:tasks.workunit.client.0.smithi144.stdout:1/243: chown d5/dd/d22 640 1 2024-09-10T11:29:31.861 INFO:tasks.workunit.client.1.smithi178.stdout:4/452: dwrite d0/f5 [0,4194304] 0 2024-09-10T11:29:31.862 INFO:tasks.workunit.client.0.smithi144.stdout:0/124: dread f0 [0,4194304] 0 2024-09-10T11:29:31.863 INFO:tasks.workunit.client.1.smithi178.stdout:9/644: truncate d2/dc/f44 1500269 0 2024-09-10T11:29:31.864 INFO:tasks.workunit.client.0.smithi144.stdout:0/125: dread - da/f1a zero size 2024-09-10T11:29:31.864 INFO:tasks.workunit.client.0.smithi144.stdout:0/126: write da/f15 [1965165,97760] 0 2024-09-10T11:29:31.865 INFO:tasks.workunit.client.0.smithi144.stdout:3/132: symlink d4/d1a/l2d 0 2024-09-10T11:29:31.865 INFO:tasks.workunit.client.1.smithi178.stdout:2/749: symlink d1/d41/d14/d11/db9/lf3 0 2024-09-10T11:29:31.866 INFO:tasks.workunit.client.0.smithi144.stdout:0/127: chown f5 3725107 1 2024-09-10T11:29:31.866 INFO:tasks.workunit.client.0.smithi144.stdout:0/128: dread - da/f19 zero size 2024-09-10T11:29:31.867 INFO:tasks.workunit.client.0.smithi144.stdout:9/166: creat d11/f3c x:0 0 0 2024-09-10T11:29:31.867 INFO:tasks.workunit.client.1.smithi178.stdout:5/568: mknod d9/dc/dd/d37/d42/d62/cc2 0 2024-09-10T11:29:31.869 INFO:tasks.workunit.client.0.smithi144.stdout:3/133: creat d4/dd/d22/d17/d18/f2e x:0 0 0 2024-09-10T11:29:31.869 INFO:tasks.workunit.client.0.smithi144.stdout:0/129: dread f2 [0,4194304] 0 2024-09-10T11:29:31.871 INFO:tasks.workunit.client.1.smithi178.stdout:7/499: creat d6/d10/d1e/d2d/d2b/da0/fb9 x:0 0 0 2024-09-10T11:29:31.871 INFO:tasks.workunit.client.0.smithi144.stdout:0/130: chown c6 451 1 2024-09-10T11:29:31.872 INFO:tasks.workunit.client.1.smithi178.stdout:7/500: truncate d6/d10/d1e/d2d/f8a 108343 0 2024-09-10T11:29:31.873 INFO:tasks.workunit.client.0.smithi144.stdout:9/167: creat d11/d12/d1b/d22/f3d x:0 0 0 2024-09-10T11:29:31.873 INFO:tasks.workunit.client.1.smithi178.stdout:7/501: fsync d6/d10/f5c 0 2024-09-10T11:29:31.874 INFO:tasks.workunit.client.1.smithi178.stdout:9/645: unlink d2/d13/d40/d89/d93/d88/f7b 0 2024-09-10T11:29:31.875 INFO:tasks.workunit.client.0.smithi144.stdout:3/134: mkdir d4/dd/d21/d2f 0 2024-09-10T11:29:31.875 INFO:tasks.workunit.client.1.smithi178.stdout:9/646: chown d2/d9b 3350715 1 2024-09-10T11:29:31.875 INFO:tasks.workunit.client.1.smithi178.stdout:7/502: fdatasync d6/d10/d1d/d22/d50/d7e/d82/f9f 0 2024-09-10T11:29:31.877 INFO:tasks.workunit.client.1.smithi178.stdout:7/503: read f4 [4239365,72749] 0 2024-09-10T11:29:31.877 INFO:tasks.workunit.client.1.smithi178.stdout:5/569: truncate d9/dc/dd/d16/f70 951139 0 2024-09-10T11:29:31.878 INFO:tasks.workunit.client.0.smithi144.stdout:0/131: rename da/l17 to da/l1b 0 2024-09-10T11:29:31.881 INFO:tasks.workunit.client.1.smithi178.stdout:2/750: creat d1/d41/d14/d11/db9/def/ff4 x:0 0 0 2024-09-10T11:29:31.881 INFO:tasks.workunit.client.1.smithi178.stdout:3/525: truncate db/d26/f32 305770 0 2024-09-10T11:29:31.882 INFO:tasks.workunit.client.0.smithi144.stdout:9/168: unlink d11/d12/d1b/d22/f3a 0 2024-09-10T11:29:31.884 INFO:tasks.workunit.client.1.smithi178.stdout:5/570: mkdir d9/d11/dc3 0 2024-09-10T11:29:31.884 INFO:tasks.workunit.client.0.smithi144.stdout:9/169: fdatasync d11/d12/d1b/d2c/f31 0 2024-09-10T11:29:31.884 INFO:tasks.workunit.client.0.smithi144.stdout:3/135: creat d4/dd/d22/d17/f30 x:0 0 0 2024-09-10T11:29:31.884 INFO:tasks.workunit.client.0.smithi144.stdout:3/136: chown d4/fb 920282088 1 2024-09-10T11:29:31.885 INFO:tasks.workunit.client.1.smithi178.stdout:5/571: fdatasync d9/dc/dd/f7f 0 2024-09-10T11:29:31.885 INFO:tasks.workunit.client.1.smithi178.stdout:9/647: write d2/d13/d75/d60/f8d [4260595,15991] 0 2024-09-10T11:29:31.886 INFO:tasks.workunit.client.0.smithi144.stdout:3/137: chown d4/dd/d22/d17 818 1 2024-09-10T11:29:31.886 INFO:tasks.workunit.client.0.smithi144.stdout:1/244: dread d5/f12 [0,4194304] 0 2024-09-10T11:29:31.887 INFO:tasks.workunit.client.1.smithi178.stdout:5/572: chown d9/la0 779277 1 2024-09-10T11:29:31.887 INFO:tasks.workunit.client.1.smithi178.stdout:3/526: rename db/d20/d3e/d6b/d7e to db/d20/d3d/d63/da2 0 2024-09-10T11:29:31.888 INFO:tasks.workunit.client.0.smithi144.stdout:9/170: dread f10 [0,4194304] 0 2024-09-10T11:29:31.889 INFO:tasks.workunit.client.1.smithi178.stdout:7/504: symlink d6/d10/d34/d6a/lba 0 2024-09-10T11:29:31.889 INFO:tasks.workunit.client.1.smithi178.stdout:4/453: dread d0/d9/d3d/f44 [0,4194304] 0 2024-09-10T11:29:31.889 INFO:tasks.workunit.client.1.smithi178.stdout:5/573: write d9/f32 [1142853,16401] 0 2024-09-10T11:29:31.890 INFO:tasks.workunit.client.1.smithi178.stdout:3/527: chown db/d36/f50 782006 1 2024-09-10T11:29:31.891 INFO:tasks.workunit.client.0.smithi144.stdout:1/245: getdents d5/d2a 0 2024-09-10T11:29:31.892 INFO:tasks.workunit.client.1.smithi178.stdout:7/505: write d6/d10/d1e/d2d/f5f [4158041,86089] 0 2024-09-10T11:29:31.893 INFO:tasks.workunit.client.0.smithi144.stdout:9/171: dread d11/d12/d1b/d2c/f31 [0,4194304] 0 2024-09-10T11:29:31.893 INFO:tasks.workunit.client.1.smithi178.stdout:3/528: read db/d20/f33 [2457644,91212] 0 2024-09-10T11:29:31.893 INFO:tasks.workunit.client.0.smithi144.stdout:3/138: rmdir d4/dd/d22/d17/d18 39 2024-09-10T11:29:31.894 INFO:tasks.workunit.client.0.smithi144.stdout:9/172: chown d11/d36/d2d/d38/l39 35972 1 2024-09-10T11:29:31.895 INFO:tasks.workunit.client.1.smithi178.stdout:9/648: rmdir d2/d51 39 2024-09-10T11:29:31.895 INFO:tasks.workunit.client.1.smithi178.stdout:3/529: chown db/d20/d3e/d6b/d79/d81 0 1 2024-09-10T11:29:31.895 INFO:tasks.workunit.client.1.smithi178.stdout:3/530: chown db/d26/d9a 1539 1 2024-09-10T11:29:31.897 INFO:tasks.workunit.client.1.smithi178.stdout:3/531: dread - db/d20/d3d/f85 zero size 2024-09-10T11:29:31.899 INFO:tasks.workunit.client.0.smithi144.stdout:9/173: mkdir d11/d36/d3e 0 2024-09-10T11:29:31.899 INFO:tasks.workunit.client.1.smithi178.stdout:2/751: symlink d1/d7/d24/d46/d88/d9d/dab/d5e/db3/lf5 0 2024-09-10T11:29:31.900 INFO:tasks.workunit.client.1.smithi178.stdout:2/752: write d1/d7/d24/d46/d88/dc6/ff2 [299496,85685] 0 2024-09-10T11:29:31.901 INFO:tasks.workunit.client.0.smithi144.stdout:3/139: link d4/dd/d22/d17/f30 d4/d1a/f31 0 2024-09-10T11:29:31.901 INFO:tasks.workunit.client.0.smithi144.stdout:1/246: write d5/f15 [467794,96159] 0 2024-09-10T11:29:31.902 INFO:tasks.workunit.client.0.smithi144.stdout:1/247: dread - d5/dd/f1d zero size 2024-09-10T11:29:31.904 INFO:tasks.workunit.client.0.smithi144.stdout:1/248: dread d5/f12 [0,4194304] 0 2024-09-10T11:29:31.905 INFO:tasks.workunit.client.0.smithi144.stdout:9/174: link l7 d11/d36/d3e/l3f 0 2024-09-10T11:29:31.906 INFO:tasks.workunit.client.0.smithi144.stdout:9/175: chown d11/l16 8024348 1 2024-09-10T11:29:31.908 INFO:tasks.workunit.client.0.smithi144.stdout:3/140: creat d4/f32 x:0 0 0 2024-09-10T11:29:31.908 INFO:tasks.workunit.client.0.smithi144.stdout:9/176: chown d11/f2b 471623109 1 2024-09-10T11:29:31.910 INFO:tasks.workunit.client.1.smithi178.stdout:9/649: truncate d2/d19/d1c/d2d/f95 3844390 0 2024-09-10T11:29:31.913 INFO:tasks.workunit.client.1.smithi178.stdout:4/454: dread d0/d8/df/f49 [0,4194304] 0 2024-09-10T11:29:31.913 INFO:tasks.workunit.client.1.smithi178.stdout:4/455: chown d0/d8/da/d1a/d51/d5d/fa1 31623529 1 2024-09-10T11:29:31.913 INFO:tasks.workunit.client.0.smithi144.stdout:3/141: write d4/dd/f1b [3392250,89432] 0 2024-09-10T11:29:31.913 INFO:tasks.workunit.client.0.smithi144.stdout:9/177: mknod d11/d36/d2d/c40 0 2024-09-10T11:29:31.913 INFO:tasks.workunit.client.0.smithi144.stdout:1/249: dwrite d5/f38 [0,4194304] 0 2024-09-10T11:29:31.914 INFO:tasks.workunit.client.1.smithi178.stdout:2/753: link d1/d22/d4d/d9b/f92 d1/d41/d14/d11/db9/ff6 0 2024-09-10T11:29:31.915 INFO:tasks.workunit.client.0.smithi144.stdout:3/142: mknod d4/d1a/c33 0 2024-09-10T11:29:31.917 INFO:tasks.workunit.client.0.smithi144.stdout:1/250: mknod d5/d2a/c49 0 2024-09-10T11:29:31.918 INFO:tasks.workunit.client.1.smithi178.stdout:9/650: creat d2/d13/d40/d89/fc1 x:0 0 0 2024-09-10T11:29:31.922 INFO:tasks.workunit.client.1.smithi178.stdout:3/532: rename db/d20/d3d/d56/d64/l8c to db/d35/d3a/d42/la3 0 2024-09-10T11:29:31.922 INFO:tasks.workunit.client.1.smithi178.stdout:7/506: write d6/d10/d1d/d22/d3b/f5a [696456,97205] 0 2024-09-10T11:29:31.923 INFO:tasks.workunit.client.1.smithi178.stdout:9/651: mkdir d2/d13/d40/d89/d93/dc2 0 2024-09-10T11:29:31.923 INFO:tasks.workunit.client.0.smithi144.stdout:9/178: dwrite d11/d12/d1b/d22/f3d [0,4194304] 0 2024-09-10T11:29:31.924 INFO:tasks.workunit.client.1.smithi178.stdout:4/456: symlink d0/d9/dc/d9a/la4 0 2024-09-10T11:29:31.925 INFO:tasks.workunit.client.1.smithi178.stdout:5/574: write d9/f86 [978662,4424] 0 2024-09-10T11:29:31.925 INFO:tasks.workunit.client.0.smithi144.stdout:1/251: dread d5/df/f19 [0,4194304] 0 2024-09-10T11:29:31.926 INFO:tasks.workunit.client.0.smithi144.stdout:9/179: mkdir d11/d12/d1b/d41 0 2024-09-10T11:29:31.927 INFO:tasks.workunit.client.1.smithi178.stdout:5/575: fsync d9/dc/f2b 0 2024-09-10T11:29:31.927 INFO:tasks.workunit.client.0.smithi144.stdout:3/143: truncate d4/dd/d22/d1d/f29 206173 0 2024-09-10T11:29:31.928 INFO:tasks.workunit.client.1.smithi178.stdout:4/457: mkdir d0/d9d/da5 0 2024-09-10T11:29:31.931 INFO:tasks.workunit.client.0.smithi144.stdout:9/180: symlink d11/d12/d1b/l42 0 2024-09-10T11:29:31.932 INFO:tasks.workunit.client.1.smithi178.stdout:2/754: rename d1/d7/d24/d46/d88/d9d/dab/cc9 to d1/d41/d14/cf7 0 2024-09-10T11:29:31.932 INFO:tasks.workunit.client.0.smithi144.stdout:1/252: dread d5/fc [0,4194304] 0 2024-09-10T11:29:31.933 INFO:tasks.workunit.client.1.smithi178.stdout:7/507: creat d6/d10/d1d/d22/fbb x:0 0 0 2024-09-10T11:29:31.934 INFO:tasks.workunit.client.0.smithi144.stdout:9/181: mknod d11/d36/d2d/c43 0 2024-09-10T11:29:31.936 INFO:tasks.workunit.client.1.smithi178.stdout:5/576: getdents d9/dc 0 2024-09-10T11:29:31.937 INFO:tasks.workunit.client.0.smithi144.stdout:3/144: dwrite d4/dd/f20 [0,4194304] 0 2024-09-10T11:29:31.938 INFO:tasks.workunit.client.0.smithi144.stdout:3/145: chown d4 1413923 1 2024-09-10T11:29:31.940 INFO:tasks.workunit.client.0.smithi144.stdout:9/182: dread d11/f2b [0,4194304] 0 2024-09-10T11:29:31.940 INFO:tasks.workunit.client.1.smithi178.stdout:4/458: dwrite d0/d8/d3e/f4c [4194304,4194304] 0 2024-09-10T11:29:31.940 INFO:tasks.workunit.client.0.smithi144.stdout:1/253: truncate d5/dd/f13 1609791 0 2024-09-10T11:29:31.941 INFO:tasks.workunit.client.1.smithi178.stdout:5/577: write d9/dc/dd/d37/d42/d9f/fa5 [436523,4426] 0 2024-09-10T11:29:31.943 INFO:tasks.workunit.client.0.smithi144.stdout:9/183: dread d11/d12/d1b/f27 [0,4194304] 0 2024-09-10T11:29:31.945 INFO:tasks.workunit.client.0.smithi144.stdout:3/146: mkdir d4/dd/d22/d17/d18/d34 0 2024-09-10T11:29:31.946 INFO:tasks.workunit.client.1.smithi178.stdout:3/533: dread db/d26/f2c [0,4194304] 0 2024-09-10T11:29:31.946 INFO:tasks.workunit.client.1.smithi178.stdout:5/578: fdatasync d9/d11/f14 0 2024-09-10T11:29:31.946 INFO:tasks.workunit.client.1.smithi178.stdout:4/459: fdatasync d0/d8/da/f96 0 2024-09-10T11:29:31.951 INFO:tasks.workunit.client.1.smithi178.stdout:2/755: dwrite d1/d7/d24/d46/d88/d9d/dab/d5b/d60/fcc [0,4194304] 0 2024-09-10T11:29:31.951 INFO:tasks.workunit.client.0.smithi144.stdout:3/147: mkdir d4/dd/d22/d1d/d35 0 2024-09-10T11:29:31.952 INFO:tasks.workunit.client.1.smithi178.stdout:5/579: chown d9/dc/d17/d1d/c4a 1419 1 2024-09-10T11:29:31.953 INFO:tasks.workunit.client.0.smithi144.stdout:9/184: dwrite d11/d12/d1b/f24 [0,4194304] 0 2024-09-10T11:29:31.953 INFO:tasks.workunit.client.0.smithi144.stdout:1/254: getdents d5/dd/d20 0 2024-09-10T11:29:31.954 INFO:tasks.workunit.client.0.smithi144.stdout:9/185: write d11/d12/d1b/f20 [2930236,33425] 0 2024-09-10T11:29:31.955 INFO:tasks.workunit.client.1.smithi178.stdout:5/580: creat d9/d11/fc4 x:0 0 0 2024-09-10T11:29:31.955 INFO:tasks.workunit.client.1.smithi178.stdout:7/508: dwrite d6/d10/d1e/d2d/d2b/da0/d55/f91 [0,4194304] 0 2024-09-10T11:29:31.956 INFO:tasks.workunit.client.1.smithi178.stdout:3/534: mknod db/d20/d3d/d63/ca4 0 2024-09-10T11:29:31.958 INFO:tasks.workunit.client.1.smithi178.stdout:2/756: creat d1/d7/d24/d7d/d84/d4b/ff8 x:0 0 0 2024-09-10T11:29:31.958 INFO:tasks.workunit.client.1.smithi178.stdout:7/509: chown d6/d10/d1e 1569 1 2024-09-10T11:29:31.960 INFO:tasks.workunit.client.1.smithi178.stdout:3/535: creat db/d20/fa5 x:0 0 0 2024-09-10T11:29:31.961 INFO:tasks.workunit.client.0.smithi144.stdout:3/148: rename d4/fb to d4/d28/f36 0 2024-09-10T11:29:31.962 INFO:tasks.workunit.client.0.smithi144.stdout:9/186: fdatasync d11/d12/d1b/d22/f2f 0 2024-09-10T11:29:31.963 INFO:tasks.workunit.client.1.smithi178.stdout:7/510: readlink d6/d10/d1e/d2d/d2b/l2e 0 2024-09-10T11:29:31.963 INFO:tasks.workunit.client.1.smithi178.stdout:2/757: dread d1/d22/fca [0,4194304] 0 2024-09-10T11:29:31.964 INFO:tasks.workunit.client.1.smithi178.stdout:3/536: rmdir db/d26/d40 39 2024-09-10T11:29:31.967 INFO:tasks.workunit.client.0.smithi144.stdout:1/255: rename d5/dd/d20 to d5/d4a 0 2024-09-10T11:29:31.967 INFO:tasks.workunit.client.0.smithi144.stdout:9/187: dread d11/d12/d1b/d2c/f31 [0,4194304] 0 2024-09-10T11:29:31.969 INFO:tasks.workunit.client.0.smithi144.stdout:9/188: write f10 [2539297,58469] 0 2024-09-10T11:29:31.971 INFO:tasks.workunit.client.1.smithi178.stdout:5/581: link d9/dc/d17/f3d d9/d19/d91/db8/fc5 0 2024-09-10T11:29:31.972 INFO:tasks.workunit.client.1.smithi178.stdout:4/460: dwrite d0/d9/dc/f8c [0,4194304] 0 2024-09-10T11:29:31.973 INFO:tasks.workunit.client.0.smithi144.stdout:1/256: unlink d5/dd/d22/f43 0 2024-09-10T11:29:31.973 INFO:tasks.workunit.client.1.smithi178.stdout:7/511: mknod d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d7c/cbc 0 2024-09-10T11:29:31.975 INFO:tasks.workunit.client.1.smithi178.stdout:3/537: mknod db/d20/d3d/d63/ca6 0 2024-09-10T11:29:31.977 INFO:tasks.workunit.client.0.smithi144.stdout:9/189: unlink d11/d12/d1b/f24 0 2024-09-10T11:29:31.978 INFO:tasks.workunit.client.0.smithi144.stdout:9/190: chown d11/f3c 55630 1 2024-09-10T11:29:31.979 INFO:tasks.workunit.client.0.smithi144.stdout:1/257: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:31.980 INFO:tasks.workunit.client.1.smithi178.stdout:2/758: rename d1/d41/d14/d11/d53/l7a to d1/d41/d14/d11/db9/lf9 0 2024-09-10T11:29:31.981 INFO:tasks.workunit.client.0.smithi144.stdout:9/191: symlink d11/d36/d2d/l44 0 2024-09-10T11:29:31.983 INFO:tasks.workunit.client.0.smithi144.stdout:1/258: mkdir d5/df/d4b 0 2024-09-10T11:29:31.983 INFO:tasks.workunit.client.1.smithi178.stdout:4/461: unlink d0/d9/dc/f26 0 2024-09-10T11:29:31.984 INFO:tasks.workunit.client.0.smithi144.stdout:1/259: write d5/f6 [1445383,88041] 0 2024-09-10T11:29:31.985 INFO:tasks.workunit.client.1.smithi178.stdout:5/582: creat d9/dc/dd/d37/dbd/fc6 x:0 0 0 2024-09-10T11:29:31.985 INFO:tasks.workunit.client.0.smithi144.stdout:9/192: symlink d11/d36/d3e/l45 0 2024-09-10T11:29:31.986 INFO:tasks.workunit.client.0.smithi144.stdout:3/149: write d4/dd/d22/d1d/f29 [952269,49838] 0 2024-09-10T11:29:31.986 INFO:tasks.workunit.client.0.smithi144.stdout:6/123: sync 2024-09-10T11:29:31.986 INFO:tasks.workunit.client.0.smithi144.stdout:5/192: sync 2024-09-10T11:29:31.986 INFO:tasks.workunit.client.1.smithi178.stdout:3/538: creat db/d20/d3e/d6b/fa7 x:0 0 0 2024-09-10T11:29:31.987 INFO:tasks.workunit.client.0.smithi144.stdout:1/260: mkdir d5/dd/d22/d4c 0 2024-09-10T11:29:31.987 INFO:tasks.workunit.client.1.smithi178.stdout:3/539: fsync db/f39 0 2024-09-10T11:29:31.989 INFO:tasks.workunit.client.1.smithi178.stdout:7/512: rename d6/d10/d61 to d6/d10/d1e/d2d/d2b/da0/d55/dbd 0 2024-09-10T11:29:31.989 INFO:tasks.workunit.client.0.smithi144.stdout:9/193: symlink d11/d36/l46 0 2024-09-10T11:29:31.991 INFO:tasks.workunit.client.1.smithi178.stdout:3/540: dread db/ff [0,4194304] 0 2024-09-10T11:29:31.992 INFO:tasks.workunit.client.0.smithi144.stdout:6/124: creat d9/dd/d11/d1b/f32 x:0 0 0 2024-09-10T11:29:31.993 INFO:tasks.workunit.client.1.smithi178.stdout:2/759: symlink d1/d7/d24/d7d/d84/d55/da1/lfa 0 2024-09-10T11:29:31.996 INFO:tasks.workunit.client.0.smithi144.stdout:3/150: rename d4/dd/f20 to d4/f37 0 2024-09-10T11:29:31.997 INFO:tasks.workunit.client.1.smithi178.stdout:7/513: creat d6/d10/d1d/d22/d3b/fbe x:0 0 0 2024-09-10T11:29:31.997 INFO:tasks.workunit.client.0.smithi144.stdout:5/193: dwrite d4/d20/d24/d2d/f2e [0,4194304] 0 2024-09-10T11:29:31.999 INFO:tasks.workunit.client.0.smithi144.stdout:1/261: dwrite d5/dd/d22/f24 [0,4194304] 0 2024-09-10T11:29:31.999 INFO:tasks.workunit.client.0.smithi144.stdout:5/194: stat d4/d20/d22/c35 0 2024-09-10T11:29:31.999 INFO:tasks.workunit.client.0.smithi144.stdout:6/125: dread d9/dc/d13/f1e [0,4194304] 0 2024-09-10T11:29:31.999 INFO:tasks.workunit.client.1.smithi178.stdout:7/514: write d6/d10/d1d/d22/f3a [1857876,117376] 0 2024-09-10T11:29:32.001 INFO:tasks.workunit.client.0.smithi144.stdout:1/262: write d5/df/f16 [3653870,119327] 0 2024-09-10T11:29:32.005 INFO:tasks.workunit.client.1.smithi178.stdout:8/616: sync 2024-09-10T11:29:32.005 INFO:tasks.workunit.client.0.smithi144.stdout:3/151: write d4/dd/d21/f23 [2082188,90356] 0 2024-09-10T11:29:32.005 INFO:tasks.workunit.client.0.smithi144.stdout:9/194: getdents d11/d36/d2d/d38 0 2024-09-10T11:29:32.007 INFO:tasks.workunit.client.1.smithi178.stdout:7/515: mkdir d6/dbf 0 2024-09-10T11:29:32.009 INFO:tasks.workunit.client.0.smithi144.stdout:1/263: creat d5/d2a/f4d x:0 0 0 2024-09-10T11:29:32.014 INFO:tasks.workunit.client.0.smithi144.stdout:1/264: dread d5/f32 [0,4194304] 0 2024-09-10T11:29:32.015 INFO:tasks.workunit.client.0.smithi144.stdout:3/152: dread d4/dd/d22/d1d/f29 [0,4194304] 0 2024-09-10T11:29:32.016 INFO:tasks.workunit.client.1.smithi178.stdout:8/617: creat d5/db/d10/d70/fcb x:0 0 0 2024-09-10T11:29:32.017 INFO:tasks.workunit.client.1.smithi178.stdout:8/618: chown d5/d30/d33/d8c/cab 63 1 2024-09-10T11:29:32.017 INFO:tasks.workunit.client.0.smithi144.stdout:5/195: rename d4/d5/f3d to d4/d20/d24/f3e 0 2024-09-10T11:29:32.019 INFO:tasks.workunit.client.0.smithi144.stdout:6/126: truncate d9/dd/d11/f21 12885 0 2024-09-10T11:29:32.019 INFO:tasks.workunit.client.1.smithi178.stdout:4/462: getdents d0/d8/da/d1a/d51/d5d 0 2024-09-10T11:29:32.020 INFO:tasks.workunit.client.1.smithi178.stdout:4/463: stat d0/d10/l4a 0 2024-09-10T11:29:32.021 INFO:tasks.workunit.client.0.smithi144.stdout:9/195: link d11/d12/d1b/l23 d11/d36/d3e/l47 0 2024-09-10T11:29:32.022 INFO:tasks.workunit.client.0.smithi144.stdout:3/153: truncate d4/dd/d22/fc 1316155 0 2024-09-10T11:29:32.023 INFO:tasks.workunit.client.1.smithi178.stdout:4/464: readlink d0/d9/dc/l33 0 2024-09-10T11:29:32.024 INFO:tasks.workunit.client.0.smithi144.stdout:1/265: truncate d5/fb 700779 0 2024-09-10T11:29:32.024 INFO:tasks.workunit.client.1.smithi178.stdout:8/619: creat d5/db/d10/d20/fcc x:0 0 0 2024-09-10T11:29:32.025 INFO:tasks.workunit.client.0.smithi144.stdout:5/196: dread d4/d5/f28 [0,4194304] 0 2024-09-10T11:29:32.025 INFO:tasks.workunit.client.0.smithi144.stdout:1/266: chown d5/d4a/d44 18893 1 2024-09-10T11:29:32.026 INFO:tasks.workunit.client.0.smithi144.stdout:3/154: write d4/dd/d22/d1d/f29 [577147,56827] 0 2024-09-10T11:29:32.027 INFO:tasks.workunit.client.1.smithi178.stdout:2/760: dread d1/d7/d24/d46/f63 [0,4194304] 0 2024-09-10T11:29:32.027 INFO:tasks.workunit.client.1.smithi178.stdout:4/465: unlink d0/d9/fb 0 2024-09-10T11:29:32.028 INFO:tasks.workunit.client.1.smithi178.stdout:5/583: write d9/dc/dd/d37/f75 [208886,56386] 0 2024-09-10T11:29:32.029 INFO:tasks.workunit.client.1.smithi178.stdout:4/466: dread - d0/d8/da/f90 zero size 2024-09-10T11:29:32.029 INFO:tasks.workunit.client.1.smithi178.stdout:8/620: creat d5/d30/d33/d8c/fcd x:0 0 0 2024-09-10T11:29:32.030 INFO:tasks.workunit.client.0.smithi144.stdout:5/197: fdatasync d4/d5/f25 0 2024-09-10T11:29:32.031 INFO:tasks.workunit.client.1.smithi178.stdout:5/584: write d9/dc/dd/d37/d3e/f78 [2120115,84786] 0 2024-09-10T11:29:32.032 INFO:tasks.workunit.client.1.smithi178.stdout:2/761: mkdir d1/d41/d8d/dfb 0 2024-09-10T11:29:32.033 INFO:tasks.workunit.client.1.smithi178.stdout:4/467: mknod d0/d10/ca6 0 2024-09-10T11:29:32.033 INFO:tasks.workunit.client.0.smithi144.stdout:3/155: creat d4/d28/f38 x:0 0 0 2024-09-10T11:29:32.035 INFO:tasks.workunit.client.1.smithi178.stdout:8/621: readlink d5/db/d10/d20/l29 0 2024-09-10T11:29:32.038 INFO:tasks.workunit.client.1.smithi178.stdout:3/541: dread db/d26/d40/f62 [0,4194304] 0 2024-09-10T11:29:32.038 INFO:tasks.workunit.client.0.smithi144.stdout:9/196: dwrite d11/f3c [0,4194304] 0 2024-09-10T11:29:32.039 INFO:tasks.workunit.client.0.smithi144.stdout:6/127: dwrite d9/dc/d13/f31 [0,4194304] 0 2024-09-10T11:29:32.039 INFO:tasks.workunit.client.1.smithi178.stdout:8/622: dread d5/d30/fb6 [0,4194304] 0 2024-09-10T11:29:32.040 INFO:tasks.workunit.client.0.smithi144.stdout:5/198: mkdir d4/d5/d3f 0 2024-09-10T11:29:32.045 INFO:tasks.workunit.client.1.smithi178.stdout:3/542: mknod db/d20/d3d/ca8 0 2024-09-10T11:29:32.045 INFO:tasks.workunit.client.1.smithi178.stdout:5/585: dwrite d9/dc/dd/d37/d42/d9f/fa5 [0,4194304] 0 2024-09-10T11:29:32.049 INFO:tasks.workunit.client.0.smithi144.stdout:5/199: write d4/d5/d18/f1f [3637553,31893] 0 2024-09-10T11:29:32.050 INFO:tasks.workunit.client.1.smithi178.stdout:8/623: rename d5/d30/db8/fbd to d5/d1d/d22/d76/fce 0 2024-09-10T11:29:32.050 INFO:tasks.workunit.client.0.smithi144.stdout:6/128: creat d9/dd/f33 x:0 0 0 2024-09-10T11:29:32.051 INFO:tasks.workunit.client.0.smithi144.stdout:1/267: dwrite d5/df/f19 [0,4194304] 0 2024-09-10T11:29:32.053 INFO:tasks.workunit.client.1.smithi178.stdout:2/762: link d1/d7/d24/d46/d88/d9d/dab/d5e/l5f d1/d7/d24/d7d/d84/d55/lfc 0 2024-09-10T11:29:32.053 INFO:tasks.workunit.client.1.smithi178.stdout:4/468: link d0/d9/dc/l33 d0/d8/da/d1a/d51/d5d/d62/la7 0 2024-09-10T11:29:32.054 INFO:tasks.workunit.client.1.smithi178.stdout:8/624: read - d5/db7/fc8 zero size 2024-09-10T11:29:32.054 INFO:tasks.workunit.client.0.smithi144.stdout:3/156: fsync d4/dd/d22/fc 0 2024-09-10T11:29:32.058 INFO:tasks.workunit.client.0.smithi144.stdout:9/197: dwrite d11/d12/d1b/f20 [0,4194304] 0 2024-09-10T11:29:32.061 INFO:tasks.workunit.client.1.smithi178.stdout:7/516: dwrite d6/d10/d34/d6a/d6f/faf [0,4194304] 0 2024-09-10T11:29:32.061 INFO:tasks.workunit.client.1.smithi178.stdout:4/469: write d0/d9/f81 [4743195,36472] 0 2024-09-10T11:29:32.064 INFO:tasks.workunit.client.0.smithi144.stdout:6/129: symlink d9/dd/l34 0 2024-09-10T11:29:32.064 INFO:tasks.workunit.client.0.smithi144.stdout:3/157: mknod d4/dd/d21/c39 0 2024-09-10T11:29:32.067 INFO:tasks.workunit.client.1.smithi178.stdout:4/470: read - d0/d8/da/d1a/d51/d5d/fa1 zero size 2024-09-10T11:29:32.067 INFO:tasks.workunit.client.1.smithi178.stdout:2/763: creat d1/d7/d24/d46/d88/d9d/dab/d5e/db3/ffd x:0 0 0 2024-09-10T11:29:32.069 INFO:tasks.workunit.client.0.smithi144.stdout:9/198: dwrite f10 [4194304,4194304] 0 2024-09-10T11:29:32.070 INFO:tasks.workunit.client.1.smithi178.stdout:7/517: mkdir d6/d10/d1e/d2d/d2b/da0/dc0 0 2024-09-10T11:29:32.076 INFO:tasks.workunit.client.0.smithi144.stdout:9/199: mknod d11/d12/d1b/c48 0 2024-09-10T11:29:32.078 INFO:tasks.workunit.client.0.smithi144.stdout:1/268: write d5/f12 [2119513,35590] 0 2024-09-10T11:29:32.079 INFO:tasks.workunit.client.0.smithi144.stdout:6/130: getdents d9 0 2024-09-10T11:29:32.080 INFO:tasks.workunit.client.0.smithi144.stdout:6/131: fsync d9/dd/d11/f18 0 2024-09-10T11:29:32.080 INFO:tasks.workunit.client.0.smithi144.stdout:9/200: rmdir d11/d12/d1b/d22 39 2024-09-10T11:29:32.081 INFO:tasks.workunit.client.1.smithi178.stdout:2/764: fsync d1/d7/d24/d46/d88/d9d/dab/f17 0 2024-09-10T11:29:32.081 INFO:tasks.workunit.client.0.smithi144.stdout:6/132: dread - d9/dc/f2e zero size 2024-09-10T11:29:32.081 INFO:tasks.workunit.client.0.smithi144.stdout:9/201: write f10 [7917961,19458] 0 2024-09-10T11:29:32.082 INFO:tasks.workunit.client.0.smithi144.stdout:3/158: rmdir d4/dd/d22/d1d/d35 0 2024-09-10T11:29:32.083 INFO:tasks.workunit.client.0.smithi144.stdout:5/200: write d4/d5/f28 [246862,1771] 0 2024-09-10T11:29:32.083 INFO:tasks.workunit.client.1.smithi178.stdout:2/765: write d1/d22/fae [3337090,124357] 0 2024-09-10T11:29:32.083 INFO:tasks.workunit.client.0.smithi144.stdout:9/202: read d11/f15 [1697903,45072] 0 2024-09-10T11:29:32.084 INFO:tasks.workunit.client.1.smithi178.stdout:2/766: readlink d1/d41/d14/l21 0 2024-09-10T11:29:32.084 INFO:tasks.workunit.client.0.smithi144.stdout:1/269: mkdir d5/d2a/d4e 0 2024-09-10T11:29:32.084 INFO:tasks.workunit.client.1.smithi178.stdout:2/767: stat d1/d7/d8e/fa8 0 2024-09-10T11:29:32.085 INFO:tasks.workunit.client.0.smithi144.stdout:1/270: fsync d5/f38 0 2024-09-10T11:29:32.088 INFO:tasks.workunit.client.1.smithi178.stdout:2/768: chown d1/d7/d8e/de9/lf1 12211 1 2024-09-10T11:29:32.088 INFO:tasks.workunit.client.1.smithi178.stdout:3/543: write db/d20/d3d/d63/d6a/f9d [1353746,75111] 0 2024-09-10T11:29:32.090 INFO:tasks.workunit.client.0.smithi144.stdout:6/133: unlink d9/dd/d11/f17 0 2024-09-10T11:29:32.090 INFO:tasks.workunit.client.1.smithi178.stdout:8/625: dread d5/f37 [0,4194304] 0 2024-09-10T11:29:32.093 INFO:tasks.workunit.client.1.smithi178.stdout:8/626: stat d5/db/d10/d20/d52/d67 0 2024-09-10T11:29:32.093 INFO:tasks.workunit.client.0.smithi144.stdout:1/271: dwrite d5/dd/f3f [0,4194304] 0 2024-09-10T11:29:32.094 INFO:tasks.workunit.client.1.smithi178.stdout:2/769: creat d1/d41/d14/d11/db9/dc1/ffe x:0 0 0 2024-09-10T11:29:32.094 INFO:tasks.workunit.client.0.smithi144.stdout:1/272: truncate d5/fc 4985200 0 2024-09-10T11:29:32.096 INFO:tasks.workunit.client.1.smithi178.stdout:3/544: rmdir db/d20/d9c 0 2024-09-10T11:29:32.097 INFO:tasks.workunit.client.1.smithi178.stdout:5/586: dwrite d9/dc/d17/d1d/f73 [0,4194304] 0 2024-09-10T11:29:32.097 INFO:tasks.workunit.client.1.smithi178.stdout:3/545: stat db/f1a 0 2024-09-10T11:29:32.100 INFO:tasks.workunit.client.1.smithi178.stdout:8/627: mknod d5/db/d10/d20/d52/d5c/ccf 0 2024-09-10T11:29:32.102 INFO:tasks.workunit.client.1.smithi178.stdout:2/770: creat d1/d7/d24/d7d/d84/d55/da1/fff x:0 0 0 2024-09-10T11:29:32.106 INFO:tasks.workunit.client.0.smithi144.stdout:6/134: symlink d9/dd/l35 0 2024-09-10T11:29:32.108 INFO:tasks.workunit.client.1.smithi178.stdout:2/771: read d1/d41/d14/d11/f9 [2576036,99070] 0 2024-09-10T11:29:32.109 INFO:tasks.workunit.client.0.smithi144.stdout:1/273: rename d5/df/c31 to d5/c4f 0 2024-09-10T11:29:32.110 INFO:tasks.workunit.client.1.smithi178.stdout:5/587: dwrite d9/dc/dd/fb7 [4194304,4194304] 0 2024-09-10T11:29:32.110 INFO:tasks.workunit.client.0.smithi144.stdout:9/203: truncate d11/d12/d1b/d22/f2f 407263 0 2024-09-10T11:29:32.111 INFO:tasks.workunit.client.1.smithi178.stdout:5/588: stat d9/dc 0 2024-09-10T11:29:32.112 INFO:tasks.workunit.client.1.smithi178.stdout:8/628: symlink d5/db/d10/ld0 0 2024-09-10T11:29:32.112 INFO:tasks.workunit.client.1.smithi178.stdout:4/471: dread d0/d9/dc/f34 [0,4194304] 0 2024-09-10T11:29:32.116 INFO:tasks.workunit.client.0.smithi144.stdout:6/135: creat d9/d1f/f36 x:0 0 0 2024-09-10T11:29:32.116 INFO:tasks.workunit.client.0.smithi144.stdout:6/136: chown d9/dd/d11 3650373 1 2024-09-10T11:29:32.116 INFO:tasks.workunit.client.0.smithi144.stdout:5/201: link d4/d20/d22/l36 d4/d5/d32/l40 0 2024-09-10T11:29:32.117 INFO:tasks.workunit.client.1.smithi178.stdout:8/629: unlink d5/c7e 0 2024-09-10T11:29:32.117 INFO:tasks.workunit.client.0.smithi144.stdout:3/159: dwrite d4/d1a/f31 [0,4194304] 0 2024-09-10T11:29:32.118 INFO:tasks.workunit.client.0.smithi144.stdout:5/202: chown d4/d20/d24/c2a 403958752 1 2024-09-10T11:29:32.119 INFO:tasks.workunit.client.0.smithi144.stdout:9/204: dread d11/f3c [0,4194304] 0 2024-09-10T11:29:32.119 INFO:tasks.workunit.client.1.smithi178.stdout:7/518: dwrite d6/d10/d1e/d2d/f2c [0,4194304] 0 2024-09-10T11:29:32.120 INFO:tasks.workunit.client.0.smithi144.stdout:1/274: link d5/df/f16 d5/dd/d22/d4c/f50 0 2024-09-10T11:29:32.120 INFO:tasks.workunit.client.1.smithi178.stdout:2/772: creat d1/d41/d14/d11/db9/f100 x:0 0 0 2024-09-10T11:29:32.121 INFO:tasks.workunit.client.0.smithi144.stdout:6/137: creat d9/d1f/f37 x:0 0 0 2024-09-10T11:29:32.121 INFO:tasks.workunit.client.1.smithi178.stdout:4/472: unlink d0/d21/c67 0 2024-09-10T11:29:32.123 INFO:tasks.workunit.client.1.smithi178.stdout:3/546: getdents db 0 2024-09-10T11:29:32.123 INFO:tasks.workunit.client.0.smithi144.stdout:5/203: mknod d4/d20/c41 0 2024-09-10T11:29:32.124 INFO:tasks.workunit.client.1.smithi178.stdout:2/773: write d1/d7/f16 [3172992,55735] 0 2024-09-10T11:29:32.124 INFO:tasks.workunit.client.1.smithi178.stdout:5/589: symlink d9/d11/dc3/lc7 0 2024-09-10T11:29:32.125 INFO:tasks.workunit.client.0.smithi144.stdout:1/275: write d5/f23 [200940,40366] 0 2024-09-10T11:29:32.127 INFO:tasks.workunit.client.0.smithi144.stdout:6/138: mkdir d9/d38 0 2024-09-10T11:29:32.128 INFO:tasks.workunit.client.1.smithi178.stdout:8/630: dread d5/fae [0,4194304] 0 2024-09-10T11:29:32.128 INFO:tasks.workunit.client.1.smithi178.stdout:2/774: mknod d1/d41/d14/d11/db9/dc1/c101 0 2024-09-10T11:29:32.131 INFO:tasks.workunit.client.1.smithi178.stdout:4/473: symlink d0/d8/la8 0 2024-09-10T11:29:32.131 INFO:tasks.workunit.client.0.smithi144.stdout:3/160: rename d4/d1a/f31 to d4/dd/d21/d2f/f3a 0 2024-09-10T11:29:32.131 INFO:tasks.workunit.client.0.smithi144.stdout:8/113: sync 2024-09-10T11:29:32.132 INFO:tasks.workunit.client.1.smithi178.stdout:5/590: truncate d9/dc/dd/d3f/f98 1028323 0 2024-09-10T11:29:32.132 INFO:tasks.workunit.client.0.smithi144.stdout:9/205: symlink d11/d12/d1b/l49 0 2024-09-10T11:29:32.134 INFO:tasks.workunit.client.1.smithi178.stdout:7/519: dwrite d6/d10/d1d/d22/d3b/f65 [0,4194304] 0 2024-09-10T11:29:32.134 INFO:tasks.workunit.client.1.smithi178.stdout:4/474: creat d0/d63/fa9 x:0 0 0 2024-09-10T11:29:32.136 INFO:tasks.workunit.client.1.smithi178.stdout:5/591: symlink d9/d19/lc8 0 2024-09-10T11:29:32.136 INFO:tasks.workunit.client.1.smithi178.stdout:2/775: creat d1/d7/d24/d46/f102 x:0 0 0 2024-09-10T11:29:32.139 INFO:tasks.workunit.client.1.smithi178.stdout:4/475: read d0/d8/da/f27 [1557060,13263] 0 2024-09-10T11:29:32.140 INFO:tasks.workunit.client.1.smithi178.stdout:5/592: rmdir d9/dc/d17 39 2024-09-10T11:29:32.140 INFO:tasks.workunit.client.1.smithi178.stdout:4/476: chown d0/d8/da/d1a/d51/c5c 12 1 2024-09-10T11:29:32.141 INFO:tasks.workunit.client.0.smithi144.stdout:3/161: creat d4/d1a/f3b x:0 0 0 2024-09-10T11:29:32.141 INFO:tasks.workunit.client.0.smithi144.stdout:3/162: chown f3 5928616 1 2024-09-10T11:29:32.142 INFO:tasks.workunit.client.0.smithi144.stdout:9/206: truncate d11/d12/f14 779477 0 2024-09-10T11:29:32.145 INFO:tasks.workunit.client.1.smithi178.stdout:4/477: fdatasync d0/d21/f77 0 2024-09-10T11:29:32.145 INFO:tasks.workunit.client.1.smithi178.stdout:7/520: dwrite d6/d10/d1e/d2d/d2b/da0/f7a [0,4194304] 0 2024-09-10T11:29:32.146 INFO:tasks.workunit.client.0.smithi144.stdout:5/204: mknod d4/d5/d18/d38/c42 0 2024-09-10T11:29:32.147 INFO:tasks.workunit.client.0.smithi144.stdout:5/205: dread - d4/d5/f23 zero size 2024-09-10T11:29:32.148 INFO:tasks.workunit.client.0.smithi144.stdout:6/139: creat d9/d1f/d2c/f39 x:0 0 0 2024-09-10T11:29:32.149 INFO:tasks.workunit.client.0.smithi144.stdout:6/140: stat d9/dd/l34 0 2024-09-10T11:29:32.151 INFO:tasks.workunit.client.0.smithi144.stdout:8/114: dread d0/f13 [0,4194304] 0 2024-09-10T11:29:32.151 INFO:tasks.workunit.client.1.smithi178.stdout:4/478: symlink d0/d9/d3d/laa 0 2024-09-10T11:29:32.152 INFO:tasks.workunit.client.1.smithi178.stdout:7/521: chown d6/d10/d1d/d22/d50/d7e/d82/db2/l47 44789856 1 2024-09-10T11:29:32.152 INFO:tasks.workunit.client.0.smithi144.stdout:3/163: symlink d4/dd/d21/l3c 0 2024-09-10T11:29:32.154 INFO:tasks.workunit.client.1.smithi178.stdout:5/593: dread d9/dc/d17/d1d/fb1 [0,4194304] 0 2024-09-10T11:29:32.155 INFO:tasks.workunit.client.0.smithi144.stdout:9/207: creat d11/d12/d1b/d22/f4a x:0 0 0 2024-09-10T11:29:32.156 INFO:tasks.workunit.client.1.smithi178.stdout:7/522: creat d6/d10/d1e/d2d/d2b/fc1 x:0 0 0 2024-09-10T11:29:32.158 INFO:tasks.workunit.client.1.smithi178.stdout:4/479: fdatasync d0/d8/df/f49 0 2024-09-10T11:29:32.158 INFO:tasks.workunit.client.0.smithi144.stdout:5/206: creat d4/d20/f43 x:0 0 0 2024-09-10T11:29:32.160 INFO:tasks.workunit.client.0.smithi144.stdout:6/141: mkdir d9/dd/d3a 0 2024-09-10T11:29:32.160 INFO:tasks.workunit.client.1.smithi178.stdout:5/594: creat d9/dc/dd/d3f/d5a/dbc/fc9 x:0 0 0 2024-09-10T11:29:32.160 INFO:tasks.workunit.client.1.smithi178.stdout:4/480: truncate d0/f6e 972955 0 2024-09-10T11:29:32.161 INFO:tasks.workunit.client.1.smithi178.stdout:7/523: mknod d6/cc2 0 2024-09-10T11:29:32.163 INFO:tasks.workunit.client.0.smithi144.stdout:3/164: symlink d4/dd/d22/d1d/l3d 0 2024-09-10T11:29:32.164 INFO:tasks.workunit.client.1.smithi178.stdout:8/631: dread d5/f1f [0,4194304] 0 2024-09-10T11:29:32.165 INFO:tasks.workunit.client.1.smithi178.stdout:5/595: write d9/dc/dd/d37/d42/f4e [4252770,17139] 0 2024-09-10T11:29:32.165 INFO:tasks.workunit.client.1.smithi178.stdout:4/481: write d0/d10/f2c [902796,130463] 0 2024-09-10T11:29:32.166 INFO:tasks.workunit.client.0.smithi144.stdout:9/208: rename d11/d36/d2d/l44 to d11/d36/l4b 0 2024-09-10T11:29:32.169 INFO:tasks.workunit.client.1.smithi178.stdout:7/524: write d6/d10/d1d/f20 [4293369,62733] 0 2024-09-10T11:29:32.169 INFO:tasks.workunit.client.1.smithi178.stdout:8/632: readlink d5/db/d10/d20/d52/la5 0 2024-09-10T11:29:32.170 INFO:tasks.workunit.client.0.smithi144.stdout:3/165: write d4/dd/d21/d2f/f3a [2588449,92759] 0 2024-09-10T11:29:32.171 INFO:tasks.workunit.client.1.smithi178.stdout:5/596: creat d9/dc/dd/d3f/d5a/dbc/fca x:0 0 0 2024-09-10T11:29:32.172 INFO:tasks.workunit.client.0.smithi144.stdout:5/207: mkdir d4/d5/d3f/d44 0 2024-09-10T11:29:32.173 INFO:tasks.workunit.client.1.smithi178.stdout:7/525: write d6/d10/d1e/d2d/d2b/da0/d55/dbd/f84 [1754576,129157] 0 2024-09-10T11:29:32.173 INFO:tasks.workunit.client.1.smithi178.stdout:5/597: creat d9/d11/fcb x:0 0 0 2024-09-10T11:29:32.174 INFO:tasks.workunit.client.0.smithi144.stdout:6/142: symlink d9/dd/d3a/l3b 0 2024-09-10T11:29:32.175 INFO:tasks.workunit.client.1.smithi178.stdout:8/633: rename d5/db/d10/d20/d52/la5 to d5/d61/ld1 0 2024-09-10T11:29:32.176 INFO:tasks.workunit.client.0.smithi144.stdout:2/144: sync 2024-09-10T11:29:32.176 INFO:tasks.workunit.client.0.smithi144.stdout:7/198: sync 2024-09-10T11:29:32.176 INFO:tasks.workunit.client.0.smithi144.stdout:1/276: dwrite d5/fb [0,4194304] 0 2024-09-10T11:29:32.179 INFO:tasks.workunit.client.0.smithi144.stdout:5/208: symlink d4/d20/d22/l45 0 2024-09-10T11:29:32.180 INFO:tasks.workunit.client.0.smithi144.stdout:2/145: stat d7/da/f19 0 2024-09-10T11:29:32.180 INFO:tasks.workunit.client.0.smithi144.stdout:1/277: readlink d5/d4a/l26 0 2024-09-10T11:29:32.181 INFO:tasks.workunit.client.1.smithi178.stdout:4/482: link d0/d8/df/l22 d0/d8/d3e/lab 0 2024-09-10T11:29:32.181 INFO:tasks.workunit.client.1.smithi178.stdout:7/526: dwrite d6/d10/d1d/d22/d93/fa8 [0,4194304] 0 2024-09-10T11:29:32.181 INFO:tasks.workunit.client.0.smithi144.stdout:6/143: mknod d9/d1f/d2c/c3c 0 2024-09-10T11:29:32.181 INFO:tasks.workunit.client.0.smithi144.stdout:3/166: dread f3 [0,4194304] 0 2024-09-10T11:29:32.182 INFO:tasks.workunit.client.0.smithi144.stdout:2/146: chown d7 2535 1 2024-09-10T11:29:32.182 INFO:tasks.workunit.client.0.smithi144.stdout:8/115: dread d0/d9/fe [0,4194304] 0 2024-09-10T11:29:32.182 INFO:tasks.workunit.client.1.smithi178.stdout:5/598: mkdir d9/dc/dd/d37/d42/d55/d6d/dcc 0 2024-09-10T11:29:32.183 INFO:tasks.workunit.client.0.smithi144.stdout:7/199: write d5/dc/d12/f21 [738530,101926] 0 2024-09-10T11:29:32.183 INFO:tasks.workunit.client.1.smithi178.stdout:0/588: sync 2024-09-10T11:29:32.184 INFO:tasks.workunit.client.0.smithi144.stdout:6/144: chown d9/dc/f16 6532 1 2024-09-10T11:29:32.185 INFO:tasks.workunit.client.0.smithi144.stdout:9/209: dwrite d11/f3c [0,4194304] 0 2024-09-10T11:29:32.187 INFO:tasks.workunit.client.1.smithi178.stdout:0/589: write d0/dc/d1f/f7a [638033,1002] 0 2024-09-10T11:29:32.187 INFO:tasks.workunit.client.1.smithi178.stdout:5/599: write d9/dc/dd/d3f/d5a/dbc/fca [729261,38729] 0 2024-09-10T11:29:32.188 INFO:tasks.workunit.client.1.smithi178.stdout:7/527: stat d6/c12 0 2024-09-10T11:29:32.188 INFO:tasks.workunit.client.0.smithi144.stdout:3/167: write d4/dd/f12 [43697,13231] 0 2024-09-10T11:29:32.188 INFO:tasks.workunit.client.1.smithi178.stdout:4/483: rename d0/d9/dc to d0/d9/dc/dac 22 2024-09-10T11:29:32.189 INFO:tasks.workunit.client.0.smithi144.stdout:8/116: write d0/fc [3174878,48345] 0 2024-09-10T11:29:32.193 INFO:tasks.workunit.client.1.smithi178.stdout:4/484: write d0/f38 [6270737,220] 0 2024-09-10T11:29:32.193 INFO:tasks.workunit.client.1.smithi178.stdout:7/528: truncate d6/d10/d1d/d22/d3b/f5a 1182436 0 2024-09-10T11:29:32.193 INFO:tasks.workunit.client.1.smithi178.stdout:0/590: chown d0/d2c/d89/l92 41 1 2024-09-10T11:29:32.194 INFO:tasks.workunit.client.1.smithi178.stdout:8/634: write d5/db/f14 [426396,26795] 0 2024-09-10T11:29:32.197 INFO:tasks.workunit.client.0.smithi144.stdout:6/145: rmdir d9/dd/d11/d1b 39 2024-09-10T11:29:32.200 INFO:tasks.workunit.client.1.smithi178.stdout:8/635: write d5/db/d3d/f97 [2910128,59077] 0 2024-09-10T11:29:32.200 INFO:tasks.workunit.client.1.smithi178.stdout:5/600: dwrite d9/dc/dd/d16/fb3 [0,4194304] 0 2024-09-10T11:29:32.201 INFO:tasks.workunit.client.0.smithi144.stdout:2/147: dwrite d7/de/f2f [0,4194304] 0 2024-09-10T11:29:32.202 INFO:tasks.workunit.client.1.smithi178.stdout:0/591: mkdir d0/dc/d1f/dc0 0 2024-09-10T11:29:32.202 INFO:tasks.workunit.client.0.smithi144.stdout:1/278: dwrite d5/f23 [0,4194304] 0 2024-09-10T11:29:32.202 INFO:tasks.workunit.client.0.smithi144.stdout:3/168: unlink d4/d1a/l2d 0 2024-09-10T11:29:32.203 INFO:tasks.workunit.client.1.smithi178.stdout:7/529: truncate d6/d10/d1d/f27 981753 0 2024-09-10T11:29:32.203 INFO:tasks.workunit.client.1.smithi178.stdout:0/592: chown d0/d12/f2f 469410742 1 2024-09-10T11:29:32.203 INFO:tasks.workunit.client.1.smithi178.stdout:7/530: fsync d6/d10/d34/f8d 0 2024-09-10T11:29:32.207 INFO:tasks.workunit.client.1.smithi178.stdout:4/485: unlink d0/d9/dc/l33 0 2024-09-10T11:29:32.207 INFO:tasks.workunit.client.1.smithi178.stdout:8/636: rmdir d5/db/d10/d35 39 2024-09-10T11:29:32.209 INFO:tasks.workunit.client.0.smithi144.stdout:3/169: write d4/dd/d22/d17/f30 [163257,22458] 0 2024-09-10T11:29:32.210 INFO:tasks.workunit.client.1.smithi178.stdout:0/593: read d0/dc/f13 [489735,9524] 0 2024-09-10T11:29:32.210 INFO:tasks.workunit.client.0.smithi144.stdout:7/200: creat d5/dc/d12/d2b/f43 x:0 0 0 2024-09-10T11:29:32.211 INFO:tasks.workunit.client.0.smithi144.stdout:8/117: rename d0/d9/f11 to d0/d9/d21/f2b 0 2024-09-10T11:29:32.212 INFO:tasks.workunit.client.0.smithi144.stdout:3/170: readlink d4/dd/l15 0 2024-09-10T11:29:32.212 INFO:tasks.workunit.client.0.smithi144.stdout:6/146: symlink d9/d1f/d2c/l3d 0 2024-09-10T11:29:32.213 INFO:tasks.workunit.client.0.smithi144.stdout:5/209: creat d4/d20/d24/f46 x:0 0 0 2024-09-10T11:29:32.214 INFO:tasks.workunit.client.1.smithi178.stdout:5/601: mkdir d9/dc/d17/dcd 0 2024-09-10T11:29:32.214 INFO:tasks.workunit.client.0.smithi144.stdout:9/210: dwrite d11/f21 [0,4194304] 0 2024-09-10T11:29:32.215 INFO:tasks.workunit.client.0.smithi144.stdout:2/148: symlink d7/da/d1e/d21/l33 0 2024-09-10T11:29:32.218 INFO:tasks.workunit.client.1.smithi178.stdout:5/602: write d9/d19/f97 [2316341,43855] 0 2024-09-10T11:29:32.219 INFO:tasks.workunit.client.0.smithi144.stdout:6/147: read - d9/dd/d11/f27 zero size 2024-09-10T11:29:32.219 INFO:tasks.workunit.client.0.smithi144.stdout:5/210: write d4/d5/d32/f33 [1252536,131014] 0 2024-09-10T11:29:32.220 INFO:tasks.workunit.client.0.smithi144.stdout:5/211: fdatasync d4/d20/f34 0 2024-09-10T11:29:32.221 INFO:tasks.workunit.client.0.smithi144.stdout:1/279: rename d5/dd/d22/d4c to d5/df/d4b/d51 0 2024-09-10T11:29:32.221 INFO:tasks.workunit.client.1.smithi178.stdout:0/594: write d0/dc/d39/d7c/fa9 [27744,25236] 0 2024-09-10T11:29:32.222 INFO:tasks.workunit.client.0.smithi144.stdout:1/280: dread - d5/dd/f1d zero size 2024-09-10T11:29:32.223 INFO:tasks.workunit.client.0.smithi144.stdout:7/201: dread d5/f29 [0,4194304] 0 2024-09-10T11:29:32.223 INFO:tasks.workunit.client.1.smithi178.stdout:5/603: truncate d9/d11/f14 5086698 0 2024-09-10T11:29:32.224 INFO:tasks.workunit.client.1.smithi178.stdout:4/486: dwrite d0/d9/d3d/f61 [0,4194304] 0 2024-09-10T11:29:32.226 INFO:tasks.workunit.client.1.smithi178.stdout:4/487: rmdir d0/d8/da/d1a/d51/d5d/d62 39 2024-09-10T11:29:32.227 INFO:tasks.workunit.client.0.smithi144.stdout:8/118: dwrite d0/d9/d21/f2b [0,4194304] 0 2024-09-10T11:29:32.231 INFO:tasks.workunit.client.0.smithi144.stdout:9/211: mknod d11/d12/c4c 0 2024-09-10T11:29:32.231 INFO:tasks.workunit.client.0.smithi144.stdout:7/202: dwrite d5/dc/d11/f26 [0,4194304] 0 2024-09-10T11:29:32.232 INFO:tasks.workunit.client.0.smithi144.stdout:6/148: mkdir d9/dd/d3e 0 2024-09-10T11:29:32.233 INFO:tasks.workunit.client.1.smithi178.stdout:7/531: dread d6/d10/d34/f74 [0,4194304] 0 2024-09-10T11:29:32.233 INFO:tasks.workunit.client.1.smithi178.stdout:4/488: symlink d0/d63/lad 0 2024-09-10T11:29:32.235 INFO:tasks.workunit.client.1.smithi178.stdout:8/637: dwrite d5/db/d10/d53/f9f [0,4194304] 0 2024-09-10T11:29:32.239 INFO:tasks.workunit.client.0.smithi144.stdout:7/203: rename d5/f29 to d5/dc/d14/f44 0 2024-09-10T11:29:32.242 INFO:tasks.workunit.client.0.smithi144.stdout:3/171: creat d4/dd/d22/f3e x:0 0 0 2024-09-10T11:29:32.245 INFO:tasks.workunit.client.0.smithi144.stdout:3/172: fsync d4/dd/d22/d1d/f29 0 2024-09-10T11:29:32.245 INFO:tasks.workunit.client.0.smithi144.stdout:5/212: unlink d4/d5/l31 0 2024-09-10T11:29:32.246 INFO:tasks.workunit.client.1.smithi178.stdout:7/532: mknod d6/d10/d1e/d2d/d2b/cc3 0 2024-09-10T11:29:32.248 INFO:tasks.workunit.client.0.smithi144.stdout:5/213: chown d4/d5/d18/d38 244 1 2024-09-10T11:29:32.249 INFO:tasks.workunit.client.0.smithi144.stdout:8/119: mknod d0/d9/c2c 0 2024-09-10T11:29:32.250 INFO:tasks.workunit.client.1.smithi178.stdout:8/638: dwrite d5/db/d10/d20/d52/d67/dbb/fc9 [0,4194304] 0 2024-09-10T11:29:32.250 INFO:tasks.workunit.client.0.smithi144.stdout:4/159: sync 2024-09-10T11:29:32.250 INFO:tasks.workunit.client.0.smithi144.stdout:0/132: sync 2024-09-10T11:29:32.251 INFO:tasks.workunit.client.0.smithi144.stdout:3/173: truncate d4/dd/d21/f23 3704511 0 2024-09-10T11:29:32.252 INFO:tasks.workunit.client.0.smithi144.stdout:9/212: dwrite d11/d12/d1b/d2c/f31 [0,4194304] 0 2024-09-10T11:29:32.252 INFO:tasks.workunit.client.1.smithi178.stdout:7/533: write d6/d10/d1e/d2d/d2b/da0/fb9 [651173,106522] 0 2024-09-10T11:29:32.253 INFO:tasks.workunit.client.0.smithi144.stdout:7/204: symlink d5/dc/d12/d2b/l45 0 2024-09-10T11:29:32.255 INFO:tasks.workunit.client.0.smithi144.stdout:1/281: link d5/c11 d5/df/d4b/c52 0 2024-09-10T11:29:32.256 INFO:tasks.workunit.client.0.smithi144.stdout:6/149: mknod d9/dd/d11/d1b/d2f/c3f 0 2024-09-10T11:29:32.257 INFO:tasks.workunit.client.1.smithi178.stdout:5/604: truncate d9/dc/dd/f69 2639872 0 2024-09-10T11:29:32.257 INFO:tasks.workunit.client.0.smithi144.stdout:7/205: dread - d5/dc/d12/d2b/f43 zero size 2024-09-10T11:29:32.257 INFO:tasks.workunit.client.0.smithi144.stdout:6/150: stat d9 0 2024-09-10T11:29:32.260 INFO:tasks.workunit.client.0.smithi144.stdout:1/282: truncate d5/d4a/f2f 766229 0 2024-09-10T11:29:32.261 INFO:tasks.workunit.client.1.smithi178.stdout:4/489: getdents d0/d21 0 2024-09-10T11:29:32.261 INFO:tasks.workunit.client.0.smithi144.stdout:5/214: mkdir d4/d5/d32/d47 0 2024-09-10T11:29:32.262 INFO:tasks.workunit.client.0.smithi144.stdout:2/149: write d7/da/d1e/d21/f2a [106566,80094] 0 2024-09-10T11:29:32.263 INFO:tasks.workunit.client.0.smithi144.stdout:4/160: creat d14/d1e/d22/f37 x:0 0 0 2024-09-10T11:29:32.264 INFO:tasks.workunit.client.1.smithi178.stdout:0/595: dread d0/dc/d39/d4c/d43/f8a [0,4194304] 0 2024-09-10T11:29:32.264 INFO:tasks.workunit.client.0.smithi144.stdout:5/215: write d4/d20/f34 [795214,77559] 0 2024-09-10T11:29:32.265 INFO:tasks.workunit.client.0.smithi144.stdout:0/133: creat da/f1c x:0 0 0 2024-09-10T11:29:32.265 INFO:tasks.workunit.client.1.smithi178.stdout:7/534: rename d6/d10/d1d/d22/l8b to d6/d10/d1d/d22/d50/d7e/d82/lc4 0 2024-09-10T11:29:32.266 INFO:tasks.workunit.client.1.smithi178.stdout:0/596: chown d0/dc/d39/d50/d5e/f7d 4940332 1 2024-09-10T11:29:32.267 INFO:tasks.workunit.client.1.smithi178.stdout:7/535: fdatasync d6/d10/d1d/d22/d3b/fa5 0 2024-09-10T11:29:32.267 INFO:tasks.workunit.client.1.smithi178.stdout:4/490: unlink d0/d8/da/l73 0 2024-09-10T11:29:32.267 INFO:tasks.workunit.client.0.smithi144.stdout:2/150: write d7/f11 [689543,67265] 0 2024-09-10T11:29:32.269 INFO:tasks.workunit.client.1.smithi178.stdout:0/597: dread - d0/d12/f87 zero size 2024-09-10T11:29:32.270 INFO:tasks.workunit.client.0.smithi144.stdout:3/174: creat d4/d28/f3f x:0 0 0 2024-09-10T11:29:32.270 INFO:tasks.workunit.client.0.smithi144.stdout:4/161: readlink lf 0 2024-09-10T11:29:32.271 INFO:tasks.workunit.client.1.smithi178.stdout:4/491: mkdir d0/d8/da/dae 0 2024-09-10T11:29:32.273 INFO:tasks.workunit.client.0.smithi144.stdout:6/151: creat d9/dd/d3e/f40 x:0 0 0 2024-09-10T11:29:32.275 INFO:tasks.workunit.client.0.smithi144.stdout:2/151: symlink d7/l34 0 2024-09-10T11:29:32.275 INFO:tasks.workunit.client.1.smithi178.stdout:5/605: dread d9/f51 [0,4194304] 0 2024-09-10T11:29:32.276 INFO:tasks.workunit.client.1.smithi178.stdout:7/536: dwrite d6/d10/fb3 [0,4194304] 0 2024-09-10T11:29:32.276 INFO:tasks.workunit.client.0.smithi144.stdout:1/283: dwrite d5/d4a/d44/f45 [0,4194304] 0 2024-09-10T11:29:32.278 INFO:tasks.workunit.client.0.smithi144.stdout:6/152: dread - d9/dc/f28 zero size 2024-09-10T11:29:32.279 INFO:tasks.workunit.client.0.smithi144.stdout:1/284: rename d5/d4a to d5/d4a/d44/d53 22 2024-09-10T11:29:32.279 INFO:tasks.workunit.client.1.smithi178.stdout:5/606: write d9/dc/dd/d37/f75 [2663455,59374] 0 2024-09-10T11:29:32.281 INFO:tasks.workunit.client.0.smithi144.stdout:9/213: creat d11/d12/d1b/f4d x:0 0 0 2024-09-10T11:29:32.285 INFO:tasks.workunit.client.0.smithi144.stdout:3/175: dread d4/dd/d22/d1d/f2a [0,4194304] 0 2024-09-10T11:29:32.286 INFO:tasks.workunit.client.0.smithi144.stdout:6/153: dread - d9/f2a zero size 2024-09-10T11:29:32.287 INFO:tasks.workunit.client.1.smithi178.stdout:7/537: dwrite d6/d10/d1e/d2d/f8a [0,4194304] 0 2024-09-10T11:29:32.289 INFO:tasks.workunit.client.0.smithi144.stdout:3/176: read d4/dd/f1b [2572880,28000] 0 2024-09-10T11:29:32.289 INFO:tasks.workunit.client.0.smithi144.stdout:5/216: dread d4/d20/d24/d2d/f2e [0,4194304] 0 2024-09-10T11:29:32.289 INFO:tasks.workunit.client.0.smithi144.stdout:8/120: getdents d0/d1b 0 2024-09-10T11:29:32.289 INFO:tasks.workunit.client.1.smithi178.stdout:7/538: read - d6/d10/d1d/d4a/d79/f85 zero size 2024-09-10T11:29:32.290 INFO:tasks.workunit.client.0.smithi144.stdout:4/162: dwrite fd [0,4194304] 0 2024-09-10T11:29:32.290 INFO:tasks.workunit.client.1.smithi178.stdout:5/607: rmdir d9/dc/dd/d37 39 2024-09-10T11:29:32.291 INFO:tasks.workunit.client.0.smithi144.stdout:6/154: chown d9/d1f/f26 11338529 1 2024-09-10T11:29:32.293 INFO:tasks.workunit.client.0.smithi144.stdout:9/214: creat d11/d36/f4e x:0 0 0 2024-09-10T11:29:32.293 INFO:tasks.workunit.client.0.smithi144.stdout:4/163: fsync d14/d1e/f2f 0 2024-09-10T11:29:32.294 INFO:tasks.workunit.client.0.smithi144.stdout:6/155: fdatasync d9/dc/d13/f31 0 2024-09-10T11:29:32.298 INFO:tasks.workunit.client.1.smithi178.stdout:7/539: mknod d6/d10/d1d/d22/d50/d98/cc5 0 2024-09-10T11:29:32.300 INFO:tasks.workunit.client.1.smithi178.stdout:5/608: fdatasync d9/dc/dd/d37/d42/d55/f6e 0 2024-09-10T11:29:32.301 INFO:tasks.workunit.client.1.smithi178.stdout:7/540: write d6/d10/d1e/d2d/d2b/da0/d55/f90 [305200,29969] 0 2024-09-10T11:29:32.305 INFO:tasks.workunit.client.0.smithi144.stdout:5/217: chown d4/d5/l30 261 1 2024-09-10T11:29:32.308 INFO:tasks.workunit.client.1.smithi178.stdout:5/609: unlink d9/d11/dc3/lc7 0 2024-09-10T11:29:32.309 INFO:tasks.workunit.client.0.smithi144.stdout:6/156: creat d9/f41 x:0 0 0 2024-09-10T11:29:32.309 INFO:tasks.workunit.client.0.smithi144.stdout:5/218: write d4/d5/f28 [1035958,59289] 0 2024-09-10T11:29:32.309 INFO:tasks.workunit.client.0.smithi144.stdout:0/134: write f0 [1310701,103645] 0 2024-09-10T11:29:32.310 INFO:tasks.workunit.client.1.smithi178.stdout:7/541: mknod d6/d10/d1d/d22/d50/d98/da6/cc6 0 2024-09-10T11:29:32.311 INFO:tasks.workunit.client.0.smithi144.stdout:0/135: chown da/l18 4426 1 2024-09-10T11:29:32.313 INFO:tasks.workunit.client.0.smithi144.stdout:9/215: creat d11/d12/d1b/d22/f4f x:0 0 0 2024-09-10T11:29:32.313 INFO:tasks.workunit.client.0.smithi144.stdout:8/121: dwrite d0/d9/d21/d25/f29 [0,4194304] 0 2024-09-10T11:29:32.314 INFO:tasks.workunit.client.0.smithi144.stdout:1/285: dwrite d5/d2a/f46 [0,4194304] 0 2024-09-10T11:29:32.317 INFO:tasks.workunit.client.1.smithi178.stdout:5/610: dwrite d9/dc/dd/d37/d42/d55/f9d [0,4194304] 0 2024-09-10T11:29:32.321 INFO:tasks.workunit.client.1.smithi178.stdout:7/542: dwrite d6/d10/d1d/d4a/d79/f85 [0,4194304] 0 2024-09-10T11:29:32.323 INFO:tasks.workunit.client.0.smithi144.stdout:9/216: stat d11/d12/d1b/d22/f4f 0 2024-09-10T11:29:32.327 INFO:tasks.workunit.client.0.smithi144.stdout:2/152: getdents d7/da/d1e 0 2024-09-10T11:29:32.328 INFO:tasks.workunit.client.0.smithi144.stdout:1/286: creat d5/df/d4b/f54 x:0 0 0 2024-09-10T11:29:32.332 INFO:tasks.workunit.client.0.smithi144.stdout:8/122: rename d0/l1e to d0/l2d 0 2024-09-10T11:29:32.332 INFO:tasks.workunit.client.1.smithi178.stdout:4/492: truncate d0/d8/d3e/f85 203612 0 2024-09-10T11:29:32.333 INFO:tasks.workunit.client.0.smithi144.stdout:3/177: rename d4/d1a/c33 to d4/c40 0 2024-09-10T11:29:32.334 INFO:tasks.workunit.client.0.smithi144.stdout:9/217: read d11/d12/d1b/f20 [279368,81772] 0 2024-09-10T11:29:32.335 INFO:tasks.workunit.client.1.smithi178.stdout:8/639: dread d5/db/d10/d35/f5a [0,4194304] 0 2024-09-10T11:29:32.337 INFO:tasks.workunit.client.1.smithi178.stdout:7/543: dread d6/d10/d34/f4e [0,4194304] 0 2024-09-10T11:29:32.337 INFO:tasks.workunit.client.1.smithi178.stdout:5/611: dwrite d9/dc/dd/d16/fb3 [0,4194304] 0 2024-09-10T11:29:32.338 INFO:tasks.workunit.client.1.smithi178.stdout:5/612: chown d9/dc/dd/d3f/f49 2531554 1 2024-09-10T11:29:32.340 INFO:tasks.workunit.client.0.smithi144.stdout:0/136: dwrite da/f19 [0,4194304] 0 2024-09-10T11:29:32.341 INFO:tasks.workunit.client.0.smithi144.stdout:6/157: dwrite d9/dd/d11/f1c [0,4194304] 0 2024-09-10T11:29:32.342 INFO:tasks.workunit.client.1.smithi178.stdout:4/493: creat d0/d9/d3d/d4f/faf x:0 0 0 2024-09-10T11:29:32.343 INFO:tasks.workunit.client.1.smithi178.stdout:4/494: fdatasync d0/d9/f13 0 2024-09-10T11:29:32.343 INFO:tasks.workunit.client.0.smithi144.stdout:5/219: dwrite d4/d20/d24/f46 [0,4194304] 0 2024-09-10T11:29:32.344 INFO:tasks.workunit.client.0.smithi144.stdout:0/137: write da/f1c [31663,74293] 0 2024-09-10T11:29:32.344 INFO:tasks.workunit.client.0.smithi144.stdout:1/287: write d5/d2a/f34 [760396,115668] 0 2024-09-10T11:29:32.350 INFO:tasks.workunit.client.0.smithi144.stdout:8/123: unlink d0/d9/d1f/c14 0 2024-09-10T11:29:32.351 INFO:tasks.workunit.client.1.smithi178.stdout:5/613: creat d9/dc/d17/d1d/fce x:0 0 0 2024-09-10T11:29:32.354 INFO:tasks.workunit.client.1.smithi178.stdout:5/614: write d9/dc/dd/d37/f8d [2575671,80483] 0 2024-09-10T11:29:32.354 INFO:tasks.workunit.client.1.smithi178.stdout:4/495: truncate d0/d8/df/f29 4272640 0 2024-09-10T11:29:32.354 INFO:tasks.workunit.client.0.smithi144.stdout:4/164: rename d14/f18 to d14/d1e/d22/d2b/d35/f38 0 2024-09-10T11:29:32.354 INFO:tasks.workunit.client.0.smithi144.stdout:1/288: write d5/f8 [3016749,40447] 0 2024-09-10T11:29:32.356 INFO:tasks.workunit.client.1.smithi178.stdout:5/615: write d9/dc/dd/d16/fb3 [1670269,3232] 0 2024-09-10T11:29:32.356 INFO:tasks.workunit.client.1.smithi178.stdout:6/610: sync 2024-09-10T11:29:32.356 INFO:tasks.workunit.client.1.smithi178.stdout:1/542: sync 2024-09-10T11:29:32.356 INFO:tasks.workunit.client.1.smithi178.stdout:9/652: sync 2024-09-10T11:29:32.359 INFO:tasks.workunit.client.0.smithi144.stdout:1/289: chown d5/d2a/f4d 16223706 1 2024-09-10T11:29:32.360 INFO:tasks.workunit.client.0.smithi144.stdout:3/178: dread d4/dd/d22/d1d/f2a [0,4194304] 0 2024-09-10T11:29:32.361 INFO:tasks.workunit.client.1.smithi178.stdout:4/496: creat d0/d8/da/d1a/d51/fb0 x:0 0 0 2024-09-10T11:29:32.361 INFO:tasks.workunit.client.0.smithi144.stdout:1/290: fdatasync d5/f12 0 2024-09-10T11:29:32.362 INFO:tasks.workunit.client.1.smithi178.stdout:5/616: mknod d9/dc/dd/d3f/d5a/d7d/ccf 0 2024-09-10T11:29:32.367 INFO:tasks.workunit.client.0.smithi144.stdout:6/158: dwrite d9/dd/f1a [0,4194304] 0 2024-09-10T11:29:32.368 INFO:tasks.workunit.client.0.smithi144.stdout:2/153: getdents d7 0 2024-09-10T11:29:32.369 INFO:tasks.workunit.client.1.smithi178.stdout:7/544: getdents d6/d10/d1e/d2d/d2b 0 2024-09-10T11:29:32.370 INFO:tasks.workunit.client.1.smithi178.stdout:6/611: creat d3/d74/d91/daa/dc2/dcf/fe0 x:0 0 0 2024-09-10T11:29:32.370 INFO:tasks.workunit.client.1.smithi178.stdout:1/543: creat d3/d9/d98/fca x:0 0 0 2024-09-10T11:29:32.371 INFO:tasks.workunit.client.0.smithi144.stdout:3/179: creat d4/dd/d21/f41 x:0 0 0 2024-09-10T11:29:32.372 INFO:tasks.workunit.client.0.smithi144.stdout:4/165: mknod d14/c39 0 2024-09-10T11:29:32.375 INFO:tasks.workunit.client.0.smithi144.stdout:8/124: dread d0/f13 [0,4194304] 0 2024-09-10T11:29:32.376 INFO:tasks.workunit.client.0.smithi144.stdout:9/218: truncate d11/f3c 2555803 0 2024-09-10T11:29:32.377 INFO:tasks.workunit.client.0.smithi144.stdout:0/138: truncate f0 325737 0 2024-09-10T11:29:32.378 INFO:tasks.workunit.client.0.smithi144.stdout:1/291: write d5/fc [841031,128257] 0 2024-09-10T11:29:32.379 INFO:tasks.workunit.client.0.smithi144.stdout:4/166: unlink d14/f1d 0 2024-09-10T11:29:32.379 INFO:tasks.workunit.client.1.smithi178.stdout:6/612: mkdir d3/d7/d8/d56/de1 0 2024-09-10T11:29:32.380 INFO:tasks.workunit.client.0.smithi144.stdout:5/220: write d4/f15 [829709,68802] 0 2024-09-10T11:29:32.382 INFO:tasks.workunit.client.0.smithi144.stdout:6/159: rename d9/dd/d3a/l3b to d9/dd/d11/d1b/l42 0 2024-09-10T11:29:32.383 INFO:tasks.workunit.client.1.smithi178.stdout:1/544: dwrite d3/d9/d98/dac/d3a/d6e/f77 [0,4194304] 0 2024-09-10T11:29:32.383 INFO:tasks.workunit.client.0.smithi144.stdout:8/125: truncate d0/d9/d1f/f1c 1290858 0 2024-09-10T11:29:32.384 INFO:tasks.workunit.client.1.smithi178.stdout:8/640: dwrite d5/d30/fb6 [0,4194304] 0 2024-09-10T11:29:32.384 INFO:tasks.workunit.client.0.smithi144.stdout:2/154: dwrite d7/f11 [0,4194304] 0 2024-09-10T11:29:32.385 INFO:tasks.workunit.client.0.smithi144.stdout:3/180: creat d4/dd/d21/f42 x:0 0 0 2024-09-10T11:29:32.385 INFO:tasks.workunit.client.0.smithi144.stdout:0/139: symlink da/l1d 0 2024-09-10T11:29:32.385 INFO:tasks.workunit.client.1.smithi178.stdout:6/613: unlink d3/d7/d35/f63 0 2024-09-10T11:29:32.386 INFO:tasks.workunit.client.0.smithi144.stdout:1/292: dread d5/f32 [0,4194304] 0 2024-09-10T11:29:32.386 INFO:tasks.workunit.client.1.smithi178.stdout:4/497: dread d0/d8/df/f29 [0,4194304] 0 2024-09-10T11:29:32.387 INFO:tasks.workunit.client.0.smithi144.stdout:9/219: mknod d11/d12/d1b/d22/c50 0 2024-09-10T11:29:32.387 INFO:tasks.workunit.client.0.smithi144.stdout:2/155: readlink d7/l34 0 2024-09-10T11:29:32.391 INFO:tasks.workunit.client.0.smithi144.stdout:9/220: write fc [4953356,13843] 0 2024-09-10T11:29:32.392 INFO:tasks.workunit.client.1.smithi178.stdout:4/498: chown d0/d8/da/f6b 51698126 1 2024-09-10T11:29:32.392 INFO:tasks.workunit.client.0.smithi144.stdout:0/140: symlink da/l1e 0 2024-09-10T11:29:32.393 INFO:tasks.workunit.client.1.smithi178.stdout:6/614: creat d3/d7/d8/d56/de1/fe2 x:0 0 0 2024-09-10T11:29:32.394 INFO:tasks.workunit.client.0.smithi144.stdout:5/221: symlink d4/d5/l48 0 2024-09-10T11:29:32.396 INFO:tasks.workunit.client.0.smithi144.stdout:2/156: dread d7/de/f2f [0,4194304] 0 2024-09-10T11:29:32.397 INFO:tasks.workunit.client.0.smithi144.stdout:3/181: creat d4/dd/d22/d17/d18/f43 x:0 0 0 2024-09-10T11:29:32.398 INFO:tasks.workunit.client.0.smithi144.stdout:5/222: mknod d4/d5/d32/c49 0 2024-09-10T11:29:32.399 INFO:tasks.workunit.client.0.smithi144.stdout:2/157: write d7/de/f28 [760966,71542] 0 2024-09-10T11:29:32.400 INFO:tasks.workunit.client.0.smithi144.stdout:0/141: rename da/l1e to da/l1f 0 2024-09-10T11:29:32.400 INFO:tasks.workunit.client.0.smithi144.stdout:5/223: read - d4/d20/d24/f3e zero size 2024-09-10T11:29:32.400 INFO:tasks.workunit.client.1.smithi178.stdout:8/641: getdents d5/db/d1e 0 2024-09-10T11:29:32.401 INFO:tasks.workunit.client.0.smithi144.stdout:0/142: chown da/ce 1 1 2024-09-10T11:29:32.401 INFO:tasks.workunit.client.0.smithi144.stdout:5/224: chown d4/d20/l29 1614776 1 2024-09-10T11:29:32.402 INFO:tasks.workunit.client.0.smithi144.stdout:4/167: link d14/f1b d14/f3a 0 2024-09-10T11:29:32.404 INFO:tasks.workunit.client.0.smithi144.stdout:1/293: link d5/f15 d5/df/f55 0 2024-09-10T11:29:32.405 INFO:tasks.workunit.client.0.smithi144.stdout:4/168: truncate d14/d1e/d22/f37 703371 0 2024-09-10T11:29:32.405 INFO:tasks.workunit.client.0.smithi144.stdout:1/294: readlink d5/l9 0 2024-09-10T11:29:32.406 INFO:tasks.workunit.client.0.smithi144.stdout:2/158: creat d7/da/f35 x:0 0 0 2024-09-10T11:29:32.406 INFO:tasks.workunit.client.0.smithi144.stdout:4/169: fsync d14/d1e/f1c 0 2024-09-10T11:29:32.406 INFO:tasks.workunit.client.1.smithi178.stdout:4/499: rename d0/d9/ce to d0/d9/dc/cb1 0 2024-09-10T11:29:32.407 INFO:tasks.workunit.client.0.smithi144.stdout:0/143: dread da/f15 [0,4194304] 0 2024-09-10T11:29:32.407 INFO:tasks.workunit.client.0.smithi144.stdout:1/295: write d5/df/f33 [864306,103407] 0 2024-09-10T11:29:32.408 INFO:tasks.workunit.client.1.smithi178.stdout:5/617: truncate d9/d19/d91/fbb 3466783 0 2024-09-10T11:29:32.409 INFO:tasks.workunit.client.0.smithi144.stdout:5/225: mkdir d4/d5/d18/d38/d4a 0 2024-09-10T11:29:32.410 INFO:tasks.workunit.client.1.smithi178.stdout:1/545: getdents d3/d9/d98/dac/d3a/d50/d80 0 2024-09-10T11:29:32.410 INFO:tasks.workunit.client.1.smithi178.stdout:5/618: truncate d9/d11/fa2 902783 0 2024-09-10T11:29:32.411 INFO:tasks.workunit.client.1.smithi178.stdout:8/642: symlink d5/ld2 0 2024-09-10T11:29:32.412 INFO:tasks.workunit.client.0.smithi144.stdout:9/221: link d11/d12/d1b/d22/c37 d11/d12/d1b/d22/c51 0 2024-09-10T11:29:32.412 INFO:tasks.workunit.client.1.smithi178.stdout:4/500: symlink d0/d9/dc/d50/lb2 0 2024-09-10T11:29:32.413 INFO:tasks.workunit.client.1.smithi178.stdout:4/501: dread - d0/f42 zero size 2024-09-10T11:29:32.413 INFO:tasks.workunit.client.1.smithi178.stdout:6/615: getdents d3/d74/d91/daa/dc2/d57 0 2024-09-10T11:29:32.413 INFO:tasks.workunit.client.0.smithi144.stdout:4/170: rmdir d14/d1e/d22/d2b 39 2024-09-10T11:29:32.415 INFO:tasks.workunit.client.1.smithi178.stdout:5/619: creat d9/dc/dd/d37/d42/d55/fd0 x:0 0 0 2024-09-10T11:29:32.417 INFO:tasks.workunit.client.1.smithi178.stdout:4/502: mkdir d0/d9d/db3 0 2024-09-10T11:29:32.417 INFO:tasks.workunit.client.0.smithi144.stdout:2/159: mkdir d7/de/d36 0 2024-09-10T11:29:32.418 INFO:tasks.workunit.client.0.smithi144.stdout:4/171: dread d14/f1f [0,4194304] 0 2024-09-10T11:29:32.419 INFO:tasks.workunit.client.1.smithi178.stdout:5/620: creat d9/dc/d17/db0/fd1 x:0 0 0 2024-09-10T11:29:32.419 INFO:tasks.workunit.client.0.smithi144.stdout:0/144: rename da/l14 to da/l20 0 2024-09-10T11:29:32.419 INFO:tasks.workunit.client.0.smithi144.stdout:4/172: truncate d14/d1e/d22/f37 1686536 0 2024-09-10T11:29:32.421 INFO:tasks.workunit.client.1.smithi178.stdout:4/503: write d0/d8/da/f27 [733371,68911] 0 2024-09-10T11:29:32.421 INFO:tasks.workunit.client.0.smithi144.stdout:6/160: write d9/dc/d13/f1e [346115,42375] 0 2024-09-10T11:29:32.423 INFO:tasks.workunit.client.1.smithi178.stdout:6/616: dwrite d3/fa0 [0,4194304] 0 2024-09-10T11:29:32.425 INFO:tasks.workunit.client.1.smithi178.stdout:8/643: write d5/db/d10/d20/f28 [3423356,117445] 0 2024-09-10T11:29:32.426 INFO:tasks.workunit.client.1.smithi178.stdout:6/617: read - d3/d7/d8/d15/fbf zero size 2024-09-10T11:29:32.427 INFO:tasks.workunit.client.0.smithi144.stdout:5/226: mkdir d4/d20/d24/d2d/d4b 0 2024-09-10T11:29:32.431 INFO:tasks.workunit.client.1.smithi178.stdout:8/644: unlink d5/db/d10/d53/d82/f92 0 2024-09-10T11:29:32.432 INFO:tasks.workunit.client.0.smithi144.stdout:2/160: dwrite d7/f8 [0,4194304] 0 2024-09-10T11:29:32.433 INFO:tasks.workunit.client.0.smithi144.stdout:4/173: dwrite f9 [4194304,4194304] 0 2024-09-10T11:29:32.436 INFO:tasks.workunit.client.1.smithi178.stdout:5/621: rename d9/dc/d17/d1d/f73 to d9/dc/dd/d37/d42/d55/d6d/dcc/fd2 0 2024-09-10T11:29:32.436 INFO:tasks.workunit.client.0.smithi144.stdout:0/145: dread da/f19 [0,4194304] 0 2024-09-10T11:29:32.436 INFO:tasks.workunit.client.1.smithi178.stdout:4/504: link d0/d9/c2a d0/d63/cb4 0 2024-09-10T11:29:32.437 INFO:tasks.workunit.client.1.smithi178.stdout:5/622: fdatasync d9/dc/dd/d37/f8d 0 2024-09-10T11:29:32.438 INFO:tasks.workunit.client.0.smithi144.stdout:6/161: rmdir d9/dc 39 2024-09-10T11:29:32.442 INFO:tasks.workunit.client.1.smithi178.stdout:6/618: symlink d3/d7/d8/d15/d1c/le3 0 2024-09-10T11:29:32.443 INFO:tasks.workunit.client.1.smithi178.stdout:4/505: creat d0/d21/fb5 x:0 0 0 2024-09-10T11:29:32.443 INFO:tasks.workunit.client.0.smithi144.stdout:3/182: getdents d4/dd/d21/d2f 0 2024-09-10T11:29:32.444 INFO:tasks.workunit.client.1.smithi178.stdout:5/623: creat d9/dc/dd/d3f/d5a/fd3 x:0 0 0 2024-09-10T11:29:32.445 INFO:tasks.workunit.client.0.smithi144.stdout:3/183: read d4/dd/d22/d1d/f2a [274130,69291] 0 2024-09-10T11:29:32.446 INFO:tasks.workunit.client.0.smithi144.stdout:0/146: dwrite da/f16 [0,4194304] 0 2024-09-10T11:29:32.446 INFO:tasks.workunit.client.1.smithi178.stdout:6/619: write d3/d7/d8/d15/d1c/d2a/d18/d8e/fa4 [2129088,52283] 0 2024-09-10T11:29:32.449 INFO:tasks.workunit.client.0.smithi144.stdout:2/161: creat d7/de/f37 x:0 0 0 2024-09-10T11:29:32.449 INFO:tasks.workunit.client.1.smithi178.stdout:8/645: link d5/db/d10/d20/c32 d5/d30/d33/cd3 0 2024-09-10T11:29:32.452 INFO:tasks.workunit.client.0.smithi144.stdout:4/174: rename d14/d1e/f1c to d14/d1e/d22/d34/f3b 0 2024-09-10T11:29:32.453 INFO:tasks.workunit.client.1.smithi178.stdout:6/620: unlink d3/cb5 0 2024-09-10T11:29:32.453 INFO:tasks.workunit.client.0.smithi144.stdout:4/175: write d14/d1e/d22/f2d [1042258,76997] 0 2024-09-10T11:29:32.454 INFO:tasks.workunit.client.0.smithi144.stdout:6/162: fsync d9/dc/f28 0 2024-09-10T11:29:32.454 INFO:tasks.workunit.client.1.smithi178.stdout:8/646: rmdir d5/db/d10/d20/d52/d5c 39 2024-09-10T11:29:32.455 INFO:tasks.workunit.client.1.smithi178.stdout:4/506: dwrite d0/d8/d3e/f4e [4194304,4194304] 0 2024-09-10T11:29:32.457 INFO:tasks.workunit.client.0.smithi144.stdout:5/227: symlink d4/d20/d24/d2d/d4b/l4c 0 2024-09-10T11:29:32.458 INFO:tasks.workunit.client.1.smithi178.stdout:5/624: dwrite d9/dc/dd/d37/d42/d62/daf/fbf [0,4194304] 0 2024-09-10T11:29:32.459 INFO:tasks.workunit.client.0.smithi144.stdout:3/184: creat d4/d1a/f44 x:0 0 0 2024-09-10T11:29:32.459 INFO:tasks.workunit.client.1.smithi178.stdout:4/507: stat d0/d8/df/f3b 0 2024-09-10T11:29:32.460 INFO:tasks.workunit.client.0.smithi144.stdout:0/147: creat da/f21 x:0 0 0 2024-09-10T11:29:32.460 INFO:tasks.workunit.client.1.smithi178.stdout:4/508: chown d0/d8/d3e 1 1 2024-09-10T11:29:32.461 INFO:tasks.workunit.client.1.smithi178.stdout:5/625: write d9/dc/dd/fc1 [206459,38981] 0 2024-09-10T11:29:32.461 INFO:tasks.workunit.client.0.smithi144.stdout:5/228: write d4/d5/d32/f33 [4384727,18172] 0 2024-09-10T11:29:32.462 INFO:tasks.workunit.client.1.smithi178.stdout:5/626: stat d9/dc/dd/d37/d42/d55/f6e 0 2024-09-10T11:29:32.463 INFO:tasks.workunit.client.0.smithi144.stdout:3/185: write d4/dd/f12 [858927,81505] 0 2024-09-10T11:29:32.464 INFO:tasks.workunit.client.0.smithi144.stdout:0/148: read da/f1c [33808,98551] 0 2024-09-10T11:29:32.464 INFO:tasks.workunit.client.1.smithi178.stdout:4/509: write d0/d9/dc/f34 [4719521,122268] 0 2024-09-10T11:29:32.466 INFO:tasks.workunit.client.1.smithi178.stdout:5/627: truncate d9/dc/dd/d3f/d5a/dbc/fca 1396749 0 2024-09-10T11:29:32.467 INFO:tasks.workunit.client.0.smithi144.stdout:4/176: dwrite d14/d1e/d22/f23 [0,4194304] 0 2024-09-10T11:29:32.467 INFO:tasks.workunit.client.1.smithi178.stdout:5/628: chown d9/dc/dd/d16/c99 161083936 1 2024-09-10T11:29:32.467 INFO:tasks.workunit.client.1.smithi178.stdout:8/647: creat d5/db/d10/d20/d52/d5c/fd4 x:0 0 0 2024-09-10T11:29:32.468 INFO:tasks.workunit.client.0.smithi144.stdout:3/186: write d4/dd/d21/f42 [149067,1376] 0 2024-09-10T11:29:32.468 INFO:tasks.workunit.client.1.smithi178.stdout:5/629: dread - d9/dc/d17/d1d/fce zero size 2024-09-10T11:29:32.468 INFO:tasks.workunit.client.1.smithi178.stdout:8/648: write d5/d30/d33/f49 [4167463,90979] 0 2024-09-10T11:29:32.473 INFO:tasks.workunit.client.0.smithi144.stdout:2/162: symlink d7/l38 0 2024-09-10T11:29:32.473 INFO:tasks.workunit.client.0.smithi144.stdout:2/163: chown f6 16243054 1 2024-09-10T11:29:32.473 INFO:tasks.workunit.client.0.smithi144.stdout:2/164: dread - d7/de/f37 zero size 2024-09-10T11:29:32.473 INFO:tasks.workunit.client.0.smithi144.stdout:1/296: truncate d5/fb 3692744 0 2024-09-10T11:29:32.475 INFO:tasks.workunit.client.0.smithi144.stdout:6/163: creat d9/d1f/f43 x:0 0 0 2024-09-10T11:29:32.476 INFO:tasks.workunit.client.1.smithi178.stdout:4/510: mkdir d0/d8/da/d1a/d51/d5d/db6 0 2024-09-10T11:29:32.478 INFO:tasks.workunit.client.1.smithi178.stdout:8/649: write d5/fae [1516366,83182] 0 2024-09-10T11:29:32.480 INFO:tasks.workunit.client.0.smithi144.stdout:2/165: dread d7/de/f22 [0,4194304] 0 2024-09-10T11:29:32.481 INFO:tasks.workunit.client.1.smithi178.stdout:4/511: write d0/d8/df/f3b [1275271,104282] 0 2024-09-10T11:29:32.481 INFO:tasks.workunit.client.1.smithi178.stdout:5/630: rename d9/f28 to d9/d19/d91/db8/fd4 0 2024-09-10T11:29:32.481 INFO:tasks.workunit.client.1.smithi178.stdout:8/650: fdatasync d5/d30/d33/d75/f86 0 2024-09-10T11:29:32.481 INFO:tasks.workunit.client.0.smithi144.stdout:2/166: readlink l0 0 2024-09-10T11:29:32.482 INFO:tasks.workunit.client.1.smithi178.stdout:6/621: dread d3/d7/d8/d15/d1c/d2a/fc0 [0,4194304] 0 2024-09-10T11:29:32.483 INFO:tasks.workunit.client.0.smithi144.stdout:6/164: dwrite d9/dc/f16 [0,4194304] 0 2024-09-10T11:29:32.484 INFO:tasks.workunit.client.0.smithi144.stdout:0/149: rmdir da 39 2024-09-10T11:29:32.485 INFO:tasks.workunit.client.1.smithi178.stdout:4/512: mknod d0/d9/dc/d9a/cb7 0 2024-09-10T11:29:32.486 INFO:tasks.workunit.client.1.smithi178.stdout:5/631: mkdir d9/dd5 0 2024-09-10T11:29:32.488 INFO:tasks.workunit.client.1.smithi178.stdout:8/651: mknod d5/db/cd5 0 2024-09-10T11:29:32.488 INFO:tasks.workunit.client.1.smithi178.stdout:4/513: write d0/d9/d3d/d4f/f52 [937430,55420] 0 2024-09-10T11:29:32.490 INFO:tasks.workunit.client.0.smithi144.stdout:1/297: mknod d5/c56 0 2024-09-10T11:29:32.490 INFO:tasks.workunit.client.1.smithi178.stdout:5/632: dread - d9/dc/dd/d37/d42/d55/f7a zero size 2024-09-10T11:29:32.492 INFO:tasks.workunit.client.1.smithi178.stdout:5/633: chown d9/dc/dd/d37/d3e/f78 1664234 1 2024-09-10T11:29:32.492 INFO:tasks.workunit.client.1.smithi178.stdout:6/622: dread d3/d74/dc9/d4b/f7b [0,4194304] 0 2024-09-10T11:29:32.494 INFO:tasks.workunit.client.1.smithi178.stdout:6/623: fsync d3/d74/dc9/fb8 0 2024-09-10T11:29:32.494 INFO:tasks.workunit.client.1.smithi178.stdout:4/514: creat d0/d9d/da5/fb8 x:0 0 0 2024-09-10T11:29:32.495 INFO:tasks.workunit.client.0.smithi144.stdout:9/222: dwrite d11/f3c [0,4194304] 0 2024-09-10T11:29:32.496 INFO:tasks.workunit.client.0.smithi144.stdout:2/167: symlink d7/da/l39 0 2024-09-10T11:29:32.497 INFO:tasks.workunit.client.1.smithi178.stdout:8/652: link d5/db/d1e/c95 d5/d30/db4/cd6 0 2024-09-10T11:29:32.498 INFO:tasks.workunit.client.1.smithi178.stdout:6/624: dread - d3/d74/dc9/f80 zero size 2024-09-10T11:29:32.499 INFO:tasks.workunit.client.0.smithi144.stdout:6/165: symlink d9/d1f/d2c/l44 0 2024-09-10T11:29:32.500 INFO:tasks.workunit.client.0.smithi144.stdout:9/223: write d11/d12/d1b/f27 [1160027,108713] 0 2024-09-10T11:29:32.500 INFO:tasks.workunit.client.0.smithi144.stdout:1/298: dwrite d5/fc [4194304,4194304] 0 2024-09-10T11:29:32.501 INFO:tasks.workunit.client.1.smithi178.stdout:4/515: symlink d0/d8/da/dae/lb9 0 2024-09-10T11:29:32.502 INFO:tasks.workunit.client.0.smithi144.stdout:3/187: mknod d4/c45 0 2024-09-10T11:29:32.504 INFO:tasks.workunit.client.1.smithi178.stdout:8/653: creat d5/db/d10/d20/dc0/fd7 x:0 0 0 2024-09-10T11:29:32.505 INFO:tasks.workunit.client.0.smithi144.stdout:9/224: fsync d11/d12/d1b/d22/f3d 0 2024-09-10T11:29:32.507 INFO:tasks.workunit.client.1.smithi178.stdout:8/654: chown d5/d30/d33/d75/f86 56779650 1 2024-09-10T11:29:32.507 INFO:tasks.workunit.client.1.smithi178.stdout:5/634: dwrite d9/dc/dd/d3f/f8b [0,4194304] 0 2024-09-10T11:29:32.509 INFO:tasks.workunit.client.1.smithi178.stdout:8/655: fdatasync d5/db/d1e/f5e 0 2024-09-10T11:29:32.512 INFO:tasks.workunit.client.0.smithi144.stdout:9/225: creat d11/d36/f52 x:0 0 0 2024-09-10T11:29:32.513 INFO:tasks.workunit.client.0.smithi144.stdout:2/168: dwrite d7/f8 [0,4194304] 0 2024-09-10T11:29:32.514 INFO:tasks.workunit.client.1.smithi178.stdout:5/635: unlink d9/d19/f97 0 2024-09-10T11:29:32.515 INFO:tasks.workunit.client.0.smithi144.stdout:6/166: dread d9/dc/f16 [0,4194304] 0 2024-09-10T11:29:32.516 INFO:tasks.workunit.client.1.smithi178.stdout:6/625: rename d3/d7/d8/d15/d48/c49 to d3/d74/d91/daa/dc2/ce4 0 2024-09-10T11:29:32.518 INFO:tasks.workunit.client.1.smithi178.stdout:8/656: creat d5/db/d10/d53/d82/fd8 x:0 0 0 2024-09-10T11:29:32.518 INFO:tasks.workunit.client.0.smithi144.stdout:1/299: dwrite d5/d2a/f46 [0,4194304] 0 2024-09-10T11:29:32.519 INFO:tasks.workunit.client.1.smithi178.stdout:8/657: dread - d5/db7/fbf zero size 2024-09-10T11:29:32.519 INFO:tasks.workunit.client.1.smithi178.stdout:4/516: symlink d0/d9/dc/d8a/lba 0 2024-09-10T11:29:32.520 INFO:tasks.workunit.client.0.smithi144.stdout:9/226: mkdir d11/d12/d1b/d22/d53 0 2024-09-10T11:29:32.520 INFO:tasks.workunit.client.0.smithi144.stdout:3/188: creat d4/dd/d22/d17/f46 x:0 0 0 2024-09-10T11:29:32.521 INFO:tasks.workunit.client.1.smithi178.stdout:8/658: readlink d5/db/d1e/l31 0 2024-09-10T11:29:32.521 INFO:tasks.workunit.client.0.smithi144.stdout:9/227: write d11/d36/f52 [380495,79187] 0 2024-09-10T11:29:32.524 INFO:tasks.workunit.client.0.smithi144.stdout:3/189: write d4/dd/d22/d17/f30 [1867945,49724] 0 2024-09-10T11:29:32.524 INFO:tasks.workunit.client.0.smithi144.stdout:1/300: mknod d5/c57 0 2024-09-10T11:29:32.524 INFO:tasks.workunit.client.0.smithi144.stdout:1/301: chown d5/df/f19 1843289 1 2024-09-10T11:29:32.525 INFO:tasks.workunit.client.0.smithi144.stdout:3/190: fsync d4/dd/f12 0 2024-09-10T11:29:32.525 INFO:tasks.workunit.client.0.smithi144.stdout:1/302: truncate d5/dd/f1d 410420 0 2024-09-10T11:29:32.526 INFO:tasks.workunit.client.0.smithi144.stdout:2/169: link d7/c32 d7/de/d36/c3a 0 2024-09-10T11:29:32.526 INFO:tasks.workunit.client.1.smithi178.stdout:6/626: creat d3/d7/d8/d15/d1c/d2a/d18/d8e/d90/fe5 x:0 0 0 2024-09-10T11:29:32.527 INFO:tasks.workunit.client.0.smithi144.stdout:3/191: dread - d4/dd/d22/d17/d18/f27 zero size 2024-09-10T11:29:32.528 INFO:tasks.workunit.client.0.smithi144.stdout:1/303: write d5/dd/f1d [363596,60044] 0 2024-09-10T11:29:32.528 INFO:tasks.workunit.client.0.smithi144.stdout:2/170: chown d7/de/f2f 4273456 1 2024-09-10T11:29:32.529 INFO:tasks.workunit.client.1.smithi178.stdout:6/627: mknod d3/d7/d8/d56/da3/ce6 0 2024-09-10T11:29:32.530 INFO:tasks.workunit.client.0.smithi144.stdout:6/167: creat d9/f45 x:0 0 0 2024-09-10T11:29:32.530 INFO:tasks.workunit.client.1.smithi178.stdout:6/628: write d3/fa0 [2981344,77349] 0 2024-09-10T11:29:32.531 INFO:tasks.workunit.client.0.smithi144.stdout:2/171: stat d7/de/f37 0 2024-09-10T11:29:32.532 INFO:tasks.workunit.client.1.smithi178.stdout:6/629: fdatasync d3/d7/d8/d15/d1c/d2a/d18/d8e/fa4 0 2024-09-10T11:29:32.533 INFO:tasks.workunit.client.1.smithi178.stdout:8/659: getdents d5/d30 0 2024-09-10T11:29:32.534 INFO:tasks.workunit.client.1.smithi178.stdout:6/630: read - d3/d7/d8/fab zero size 2024-09-10T11:29:32.535 INFO:tasks.workunit.client.0.smithi144.stdout:9/228: dwrite d11/f15 [0,4194304] 0 2024-09-10T11:29:32.536 INFO:tasks.workunit.client.0.smithi144.stdout:2/172: creat d7/de/f3b x:0 0 0 2024-09-10T11:29:32.538 INFO:tasks.workunit.client.0.smithi144.stdout:3/192: link d4/c45 d4/dd/d21/c47 0 2024-09-10T11:29:32.538 INFO:tasks.workunit.client.1.smithi178.stdout:8/660: mknod d5/db/cd9 0 2024-09-10T11:29:32.540 INFO:tasks.workunit.client.0.smithi144.stdout:2/173: creat d7/da/f3c x:0 0 0 2024-09-10T11:29:32.541 INFO:tasks.workunit.client.1.smithi178.stdout:8/661: getdents d5/db/d10/d53/d82/db2 0 2024-09-10T11:29:32.542 INFO:tasks.workunit.client.0.smithi144.stdout:9/229: truncate d11/d12/d1b/d22/f4f 515243 0 2024-09-10T11:29:32.542 INFO:tasks.workunit.client.0.smithi144.stdout:6/168: dwrite d9/dc/d13/f19 [0,4194304] 0 2024-09-10T11:29:32.543 INFO:tasks.workunit.client.1.smithi178.stdout:8/662: stat d5/db/d10/d20/d52/d5c/dad/lc3 0 2024-09-10T11:29:32.543 INFO:tasks.workunit.client.0.smithi144.stdout:6/169: fsync d9/dd/d3e/f40 0 2024-09-10T11:29:32.547 INFO:tasks.workunit.client.0.smithi144.stdout:9/230: creat d11/d12/d1b/d22/d53/f54 x:0 0 0 2024-09-10T11:29:32.550 INFO:tasks.workunit.client.0.smithi144.stdout:9/231: stat d11/d12/d1b/d2c 0 2024-09-10T11:29:32.552 INFO:tasks.workunit.client.0.smithi144.stdout:6/170: creat d9/dd/d11/f46 x:0 0 0 2024-09-10T11:29:32.553 INFO:tasks.workunit.client.0.smithi144.stdout:6/171: fdatasync d9/f2a 0 2024-09-10T11:29:32.554 INFO:tasks.workunit.client.0.smithi144.stdout:2/174: dwrite d7/de/f2b [0,4194304] 0 2024-09-10T11:29:32.555 INFO:tasks.workunit.client.0.smithi144.stdout:9/232: truncate d11/d12/f26 579701 0 2024-09-10T11:29:32.556 INFO:tasks.workunit.client.1.smithi178.stdout:6/631: dread d3/d74/d91/daa/dc2/fb6 [4194304,4194304] 0 2024-09-10T11:29:32.556 INFO:tasks.workunit.client.0.smithi144.stdout:3/193: dwrite d4/dd/d22/d1d/f2a [0,4194304] 0 2024-09-10T11:29:32.557 INFO:tasks.workunit.client.0.smithi144.stdout:3/194: dread - d4/d1a/f44 zero size 2024-09-10T11:29:32.559 INFO:tasks.workunit.client.1.smithi178.stdout:6/632: mknod d3/d74/d91/daa/dc2/ce7 0 2024-09-10T11:29:32.561 INFO:tasks.workunit.client.0.smithi144.stdout:9/233: readlink l7 0 2024-09-10T11:29:32.562 INFO:tasks.workunit.client.1.smithi178.stdout:6/633: mknod d3/d4e/ce8 0 2024-09-10T11:29:32.563 INFO:tasks.workunit.client.0.smithi144.stdout:3/195: truncate d4/dd/d22/f3e 634528 0 2024-09-10T11:29:32.564 INFO:tasks.workunit.client.0.smithi144.stdout:9/234: creat d11/d36/d3e/f55 x:0 0 0 2024-09-10T11:29:32.564 INFO:tasks.workunit.client.0.smithi144.stdout:2/175: stat d7/c32 0 2024-09-10T11:29:32.565 INFO:tasks.workunit.client.1.smithi178.stdout:6/634: mkdir d3/d6e/de9 0 2024-09-10T11:29:32.565 INFO:tasks.workunit.client.1.smithi178.stdout:6/635: chown d3/d4e/la1 933501333 1 2024-09-10T11:29:32.567 INFO:tasks.workunit.client.1.smithi178.stdout:3/547: sync 2024-09-10T11:29:32.568 INFO:tasks.workunit.client.0.smithi144.stdout:3/196: creat d4/dd/d22/d17/d18/f48 x:0 0 0 2024-09-10T11:29:32.569 INFO:tasks.workunit.client.1.smithi178.stdout:3/548: mknod db/d26/d40/ca9 0 2024-09-10T11:29:32.570 INFO:tasks.workunit.client.0.smithi144.stdout:9/235: dread d11/d12/d1b/f20 [0,4194304] 0 2024-09-10T11:29:32.570 INFO:tasks.workunit.client.0.smithi144.stdout:3/197: mknod d4/dd/d22/d17/d18/c49 0 2024-09-10T11:29:32.571 INFO:tasks.workunit.client.0.smithi144.stdout:9/236: write d11/f3c [1528823,54295] 0 2024-09-10T11:29:32.573 INFO:tasks.workunit.client.0.smithi144.stdout:9/237: dread - d11/d36/d3e/f55 zero size 2024-09-10T11:29:32.575 INFO:tasks.workunit.client.0.smithi144.stdout:3/198: link d4/c45 d4/dd/d22/d17/d18/d34/c4a 0 2024-09-10T11:29:32.576 INFO:tasks.workunit.client.1.smithi178.stdout:3/549: dwrite db/d35/d58/f6e [0,4194304] 0 2024-09-10T11:29:32.576 INFO:tasks.workunit.client.1.smithi178.stdout:8/663: dread d5/db/d10/d53/d82/fba [0,4194304] 0 2024-09-10T11:29:32.577 INFO:tasks.workunit.client.1.smithi178.stdout:8/664: write d5/db/d10/d53/d82/fd8 [87050,82429] 0 2024-09-10T11:29:32.578 INFO:tasks.workunit.client.0.smithi144.stdout:3/199: write d4/dd/d22/d17/d18/f48 [797663,84718] 0 2024-09-10T11:29:32.580 INFO:tasks.workunit.client.0.smithi144.stdout:3/200: write d4/d1a/f44 [793483,9949] 0 2024-09-10T11:29:32.581 INFO:tasks.workunit.client.0.smithi144.stdout:9/238: dwrite d11/d12/d1b/d22/f4a [0,4194304] 0 2024-09-10T11:29:32.581 INFO:tasks.workunit.client.0.smithi144.stdout:3/201: dread - d4/d1a/f3b zero size 2024-09-10T11:29:32.582 INFO:tasks.workunit.client.1.smithi178.stdout:8/665: stat d5/db/d10/d53/d82/fc1 0 2024-09-10T11:29:32.582 INFO:tasks.workunit.client.0.smithi144.stdout:9/239: readlink d11/l25 0 2024-09-10T11:29:32.583 INFO:tasks.workunit.client.1.smithi178.stdout:3/550: mknod db/d26/caa 0 2024-09-10T11:29:32.587 INFO:tasks.workunit.client.0.smithi144.stdout:3/202: write d4/dd/d22/d17/f1f [2747758,28542] 0 2024-09-10T11:29:32.590 INFO:tasks.workunit.client.1.smithi178.stdout:3/551: rename db/d20/d3d/f51 to db/d20/d3e/d6b/d79/d81/fab 0 2024-09-10T11:29:32.593 INFO:tasks.workunit.client.0.smithi144.stdout:9/240: dwrite d11/d36/f52 [0,4194304] 0 2024-09-10T11:29:32.595 INFO:tasks.workunit.client.0.smithi144.stdout:6/172: dread d9/dd/f1d [0,4194304] 0 2024-09-10T11:29:32.597 INFO:tasks.workunit.client.0.smithi144.stdout:6/173: stat d9/dc 0 2024-09-10T11:29:32.601 INFO:tasks.workunit.client.1.smithi178.stdout:3/552: rename db/d35/d3a/d42/d46/l71 to db/d26/d40/lac 0 2024-09-10T11:29:32.603 INFO:tasks.workunit.client.1.smithi178.stdout:8/666: dread d5/d30/f87 [0,4194304] 0 2024-09-10T11:29:32.604 INFO:tasks.workunit.client.0.smithi144.stdout:3/203: dwrite d4/dd/d22/d17/f30 [4194304,4194304] 0 2024-09-10T11:29:32.604 INFO:tasks.workunit.client.1.smithi178.stdout:8/667: read - d5/d30/d33/d75/f8b zero size 2024-09-10T11:29:32.607 INFO:tasks.workunit.client.0.smithi144.stdout:5/229: truncate d4/fa 5218007 0 2024-09-10T11:29:32.608 INFO:tasks.workunit.client.1.smithi178.stdout:6/636: dread d3/d74/d91/daa/dc2/f61 [0,4194304] 0 2024-09-10T11:29:32.611 INFO:tasks.workunit.client.0.smithi144.stdout:5/230: stat d4/d5/l48 0 2024-09-10T11:29:32.611 INFO:tasks.workunit.client.0.smithi144.stdout:3/204: mkdir d4/dd/d22/d17/d18/d34/d4b 0 2024-09-10T11:29:32.611 INFO:tasks.workunit.client.0.smithi144.stdout:9/241: dwrite d11/f1c [0,4194304] 0 2024-09-10T11:29:32.616 INFO:tasks.workunit.client.0.smithi144.stdout:6/174: dwrite d9/dd/f1a [4194304,4194304] 0 2024-09-10T11:29:32.617 INFO:tasks.workunit.client.0.smithi144.stdout:3/205: dread - d4/dd/d22/d17/f46 zero size 2024-09-10T11:29:32.618 INFO:tasks.workunit.client.1.smithi178.stdout:8/668: unlink d5/db/d3d/c65 0 2024-09-10T11:29:32.619 INFO:tasks.workunit.client.0.smithi144.stdout:3/206: chown d4/dd/d21/d2f 599 1 2024-09-10T11:29:32.619 INFO:tasks.workunit.client.1.smithi178.stdout:8/669: readlink d5/l9d 0 2024-09-10T11:29:32.619 INFO:tasks.workunit.client.0.smithi144.stdout:4/177: write d14/d1e/d22/d2b/f33 [1172672,107768] 0 2024-09-10T11:29:32.621 INFO:tasks.workunit.client.0.smithi144.stdout:4/178: write fa [3419275,45382] 0 2024-09-10T11:29:32.622 INFO:tasks.workunit.client.0.smithi144.stdout:5/231: mknod d4/d5/d18/d38/c4d 0 2024-09-10T11:29:32.624 INFO:tasks.workunit.client.0.smithi144.stdout:6/175: rmdir d9/dd/d3e 39 2024-09-10T11:29:32.627 INFO:tasks.workunit.client.0.smithi144.stdout:4/179: truncate d14/f3a 939614 0 2024-09-10T11:29:32.628 INFO:tasks.workunit.client.0.smithi144.stdout:9/242: mknod d11/d12/d1b/d41/c56 0 2024-09-10T11:29:32.628 INFO:tasks.workunit.client.0.smithi144.stdout:0/150: dwrite f0 [0,4194304] 0 2024-09-10T11:29:32.631 INFO:tasks.workunit.client.0.smithi144.stdout:6/176: unlink d9/f41 0 2024-09-10T11:29:32.634 INFO:tasks.workunit.client.0.smithi144.stdout:3/207: rename d4/dd/d22/d17/f46 to d4/dd/d22/d17/d18/f4c 0 2024-09-10T11:29:32.635 INFO:tasks.workunit.client.0.smithi144.stdout:6/177: truncate d9/f45 531740 0 2024-09-10T11:29:32.635 INFO:tasks.workunit.client.0.smithi144.stdout:0/151: write da/f21 [368008,4082] 0 2024-09-10T11:29:32.639 INFO:tasks.workunit.client.0.smithi144.stdout:2/176: truncate d7/de/f2f 1714405 0 2024-09-10T11:29:32.640 INFO:tasks.workunit.client.0.smithi144.stdout:4/180: mkdir d14/d1e/d22/d2b/d35/d3c 0 2024-09-10T11:29:32.641 INFO:tasks.workunit.client.0.smithi144.stdout:0/152: symlink da/l22 0 2024-09-10T11:29:32.642 INFO:tasks.workunit.client.0.smithi144.stdout:5/232: dwrite d4/d5/f25 [0,4194304] 0 2024-09-10T11:29:32.642 INFO:tasks.workunit.client.0.smithi144.stdout:6/178: chown d9/dd/d11/d1b/f32 60 1 2024-09-10T11:29:32.654 INFO:tasks.workunit.client.1.smithi178.stdout:8/670: mkdir d5/db/dda 0 2024-09-10T11:29:32.659 INFO:tasks.workunit.client.1.smithi178.stdout:4/517: link d0/f5 d0/d8/da/d1a/d51/d5d/fbb 0 2024-09-10T11:29:32.661 INFO:tasks.workunit.client.1.smithi178.stdout:8/671: creat d5/d30/d33/d8c/daa/fdb x:0 0 0 2024-09-10T11:29:32.662 INFO:tasks.workunit.client.1.smithi178.stdout:8/672: creat d5/d1d/fdc x:0 0 0 2024-09-10T11:29:32.663 INFO:tasks.workunit.client.1.smithi178.stdout:8/673: chown d5/db/d10/c15 580527451 1 2024-09-10T11:29:32.664 INFO:tasks.workunit.client.1.smithi178.stdout:5/636: truncate d9/d11/fa2 67646 0 2024-09-10T11:29:32.666 INFO:tasks.workunit.client.1.smithi178.stdout:4/518: rename d0/d9/c2a to d0/d9/dc/cbc 0 2024-09-10T11:29:32.666 INFO:tasks.workunit.client.1.smithi178.stdout:5/637: mkdir d9/dc/dd/d3f/dd6 0 2024-09-10T11:29:32.667 INFO:tasks.workunit.client.1.smithi178.stdout:5/638: chown d9/dc/dd/d3f/ca7 124941727 1 2024-09-10T11:29:32.668 INFO:tasks.workunit.client.1.smithi178.stdout:8/674: rename d5/db/d10/d70/f80 to d5/db/d10/d20/dc0/fdd 0 2024-09-10T11:29:32.669 INFO:tasks.workunit.client.1.smithi178.stdout:4/519: dread - d0/d8/df/f7c zero size 2024-09-10T11:29:32.669 INFO:tasks.workunit.client.1.smithi178.stdout:4/520: dread - d0/d21/f77 zero size 2024-09-10T11:29:32.672 INFO:tasks.workunit.client.1.smithi178.stdout:5/639: mkdir d9/dc/dd7 0 2024-09-10T11:29:32.673 INFO:tasks.workunit.client.1.smithi178.stdout:5/640: read d9/dc/dd/d3f/f98 [872432,104551] 0 2024-09-10T11:29:32.674 INFO:tasks.workunit.client.1.smithi178.stdout:4/521: link d0/d63/f76 d0/d9/d3d/fbd 0 2024-09-10T11:29:32.675 INFO:tasks.workunit.client.1.smithi178.stdout:5/641: fdatasync d9/d19/d91/fa4 0 2024-09-10T11:29:32.676 INFO:tasks.workunit.client.1.smithi178.stdout:5/642: dread - d9/d11/fc4 zero size 2024-09-10T11:29:32.677 INFO:tasks.workunit.client.1.smithi178.stdout:8/675: dwrite d5/d1d/d22/f4e [0,4194304] 0 2024-09-10T11:29:32.679 INFO:tasks.workunit.client.1.smithi178.stdout:3/553: truncate db/d35/f77 2905281 0 2024-09-10T11:29:32.681 INFO:tasks.workunit.client.1.smithi178.stdout:5/643: mkdir d9/dc/dd/d37/d42/d55/d6d/dcc/dd8 0 2024-09-10T11:29:32.686 INFO:tasks.workunit.client.1.smithi178.stdout:4/522: dwrite d0/d8/da/d1a/f4b [0,4194304] 0 2024-09-10T11:29:32.686 INFO:tasks.workunit.client.1.smithi178.stdout:5/644: fsync d9/dc/dd/d37/d42/d9f/fa5 0 2024-09-10T11:29:32.689 INFO:tasks.workunit.client.1.smithi178.stdout:4/523: chown d0/d8/da/d1a/f69 3262 1 2024-09-10T11:29:32.690 INFO:tasks.workunit.client.1.smithi178.stdout:6/637: dwrite d3/d24/f30 [0,4194304] 0 2024-09-10T11:29:32.697 INFO:tasks.workunit.client.1.smithi178.stdout:2/776: sync 2024-09-10T11:29:32.699 INFO:tasks.workunit.client.1.smithi178.stdout:8/676: dread d5/db/d10/d53/f64 [0,4194304] 0 2024-09-10T11:29:32.699 INFO:tasks.workunit.client.1.smithi178.stdout:5/645: dwrite d9/d19/f67 [4194304,4194304] 0 2024-09-10T11:29:32.704 INFO:tasks.workunit.client.1.smithi178.stdout:2/777: fdatasync d1/d7/d24/d46/d88/d9d/dab/d5e/fdc 0 2024-09-10T11:29:32.705 INFO:tasks.workunit.client.1.smithi178.stdout:3/554: write db/f27 [1022445,6424] 0 2024-09-10T11:29:32.705 INFO:tasks.workunit.client.1.smithi178.stdout:8/677: symlink d5/db/d10/d20/d52/lde 0 2024-09-10T11:29:32.706 INFO:tasks.workunit.client.1.smithi178.stdout:2/778: dread - d1/d7/d8e/fc0 zero size 2024-09-10T11:29:32.707 INFO:tasks.workunit.client.1.smithi178.stdout:2/779: fdatasync d1/d22/d4d/d9b/f75 0 2024-09-10T11:29:32.709 INFO:tasks.workunit.client.1.smithi178.stdout:3/555: read db/d35/f5f [433333,78423] 0 2024-09-10T11:29:32.711 INFO:tasks.workunit.client.1.smithi178.stdout:2/780: truncate d1/d22/fca 305246 0 2024-09-10T11:29:32.713 INFO:tasks.workunit.client.1.smithi178.stdout:3/556: dread db/f1a [4194304,4194304] 0 2024-09-10T11:29:32.714 INFO:tasks.workunit.client.1.smithi178.stdout:8/678: link d5/db/d10/d20/f79 d5/db/d10/d20/d52/d67/fdf 0 2024-09-10T11:29:32.714 INFO:tasks.workunit.client.1.smithi178.stdout:3/557: write db/d20/f28 [1658648,12943] 0 2024-09-10T11:29:32.718 INFO:tasks.workunit.client.1.smithi178.stdout:2/781: rename d1/d7/d24/d46/d88/d9d/dab/f29 to d1/d7/f103 0 2024-09-10T11:29:32.719 INFO:tasks.workunit.client.1.smithi178.stdout:3/558: mknod db/d35/d3a/d42/d46/cad 0 2024-09-10T11:29:32.719 INFO:tasks.workunit.client.1.smithi178.stdout:8/679: creat d5/db/d10/d20/d52/d67/dbb/fe0 x:0 0 0 2024-09-10T11:29:32.720 INFO:tasks.workunit.client.1.smithi178.stdout:3/559: fsync db/d35/f5e 0 2024-09-10T11:29:32.720 INFO:tasks.workunit.client.1.smithi178.stdout:5/646: dwrite d9/d11/f2f [0,4194304] 0 2024-09-10T11:29:32.721 INFO:tasks.workunit.client.1.smithi178.stdout:8/680: fsync d5/d30/d33/f49 0 2024-09-10T11:29:32.724 INFO:tasks.workunit.client.1.smithi178.stdout:5/647: mknod d9/dc/dd/d37/d42/cd9 0 2024-09-10T11:29:32.725 INFO:tasks.workunit.client.1.smithi178.stdout:8/681: creat d5/db/dda/fe1 x:0 0 0 2024-09-10T11:29:32.729 INFO:tasks.workunit.client.1.smithi178.stdout:8/682: mknod d5/d30/d33/d3a/ce2 0 2024-09-10T11:29:32.730 INFO:tasks.workunit.client.1.smithi178.stdout:8/683: dread - d5/db7/fc8 zero size 2024-09-10T11:29:32.730 INFO:tasks.workunit.client.1.smithi178.stdout:8/684: fdatasync d5/db/d3d/fbe 0 2024-09-10T11:29:32.733 INFO:tasks.workunit.client.1.smithi178.stdout:8/685: write d5/d30/d33/f99 [1853464,9297] 0 2024-09-10T11:29:32.736 INFO:tasks.workunit.client.1.smithi178.stdout:3/560: dwrite db/d20/d3d/d56/f66 [0,4194304] 0 2024-09-10T11:29:32.769 INFO:tasks.workunit.client.0.smithi144.stdout:0/153: write da/f21 [1193302,98357] 0 2024-09-10T11:29:32.769 INFO:tasks.workunit.client.0.smithi144.stdout:3/208: rename d4/d1a/c1e to d4/dd/d21/d2f/c4d 0 2024-09-10T11:29:32.771 INFO:tasks.workunit.client.1.smithi178.stdout:8/686: creat d5/db/d10/d20/dc0/fe3 x:0 0 0 2024-09-10T11:29:32.771 INFO:tasks.workunit.client.0.smithi144.stdout:3/209: read - d4/dd/d22/d17/d18/f2e zero size 2024-09-10T11:29:32.772 INFO:tasks.workunit.client.0.smithi144.stdout:9/243: truncate d11/d12/d1b/f27 106475 0 2024-09-10T11:29:32.773 INFO:tasks.workunit.client.1.smithi178.stdout:3/561: chown db/d26/l49 603 1 2024-09-10T11:29:32.776 INFO:tasks.workunit.client.1.smithi178.stdout:5/648: unlink d9/dc/dd/d37/d42/f5f 0 2024-09-10T11:29:32.778 INFO:tasks.workunit.client.1.smithi178.stdout:8/687: dwrite d5/db/d10/d35/f3c [0,4194304] 0 2024-09-10T11:29:32.778 INFO:tasks.workunit.client.1.smithi178.stdout:5/649: write d9/dc/dd/d3f/d5a/dbc/fc9 [786284,3883] 0 2024-09-10T11:29:32.780 INFO:tasks.workunit.client.1.smithi178.stdout:5/650: dread - d9/dc/d17/db0/fd1 zero size 2024-09-10T11:29:32.781 INFO:tasks.workunit.client.1.smithi178.stdout:3/562: link f8 db/d20/d3d/d56/d64/fae 0 2024-09-10T11:29:32.781 INFO:tasks.workunit.client.1.smithi178.stdout:3/563: chown db/d20/f28 267965564 1 2024-09-10T11:29:32.782 INFO:tasks.workunit.client.1.smithi178.stdout:8/688: mkdir d5/db/d10/de4 0 2024-09-10T11:29:32.782 INFO:tasks.workunit.client.1.smithi178.stdout:5/651: chown d9/dc/d17/f44 4488 1 2024-09-10T11:29:32.783 INFO:tasks.workunit.client.1.smithi178.stdout:3/564: chown db/d20/f34 69 1 2024-09-10T11:29:32.783 INFO:tasks.workunit.client.1.smithi178.stdout:2/782: dread d1/d7/d24/d7d/d84/d4b/f57 [0,4194304] 0 2024-09-10T11:29:32.783 INFO:tasks.workunit.client.0.smithi144.stdout:9/244: chown d11 3 1 2024-09-10T11:29:32.783 INFO:tasks.workunit.client.1.smithi178.stdout:8/689: dread - d5/d30/d33/d8c/fcd zero size 2024-09-10T11:29:32.784 INFO:tasks.workunit.client.1.smithi178.stdout:5/652: read - d9/dc/dd/d3f/d5a/fd3 zero size 2024-09-10T11:29:32.786 INFO:tasks.workunit.client.0.smithi144.stdout:0/154: mknod da/c23 0 2024-09-10T11:29:32.786 INFO:tasks.workunit.client.0.smithi144.stdout:2/177: dwrite d7/da/d1e/d21/f2a [0,4194304] 0 2024-09-10T11:29:32.787 INFO:tasks.workunit.client.1.smithi178.stdout:3/565: creat db/d20/d3d/d56/faf x:0 0 0 2024-09-10T11:29:32.787 INFO:tasks.workunit.client.0.smithi144.stdout:6/179: write d9/dc/f16 [2256964,55013] 0 2024-09-10T11:29:32.788 INFO:tasks.workunit.client.0.smithi144.stdout:4/181: mknod d14/d1e/d22/d2b/c3d 0 2024-09-10T11:29:32.788 INFO:tasks.workunit.client.0.smithi144.stdout:9/245: fsync d11/d12/d1b/d22/f4f 0 2024-09-10T11:29:32.790 INFO:tasks.workunit.client.0.smithi144.stdout:3/210: fsync d4/d28/f36 0 2024-09-10T11:29:32.792 INFO:tasks.workunit.client.0.smithi144.stdout:3/211: write d4/dd/fe [2787612,64376] 0 2024-09-10T11:29:32.792 INFO:tasks.workunit.client.0.smithi144.stdout:5/233: rmdir d4/d5/d18/d38/d4a 0 2024-09-10T11:29:32.794 INFO:tasks.workunit.client.0.smithi144.stdout:2/178: dread d7/de/f2b [0,4194304] 0 2024-09-10T11:29:32.797 INFO:tasks.workunit.client.0.smithi144.stdout:0/155: rename da/cd to da/c24 0 2024-09-10T11:29:32.799 INFO:tasks.workunit.client.1.smithi178.stdout:2/783: dwrite d1/d7/d24/d46/d88/d9d/dab/fc5 [0,4194304] 0 2024-09-10T11:29:32.799 INFO:tasks.workunit.client.1.smithi178.stdout:5/653: dwrite d9/f71 [0,4194304] 0 2024-09-10T11:29:32.799 INFO:tasks.workunit.client.0.smithi144.stdout:9/246: symlink d11/d12/d1b/d41/l57 0 2024-09-10T11:29:32.800 INFO:tasks.workunit.client.0.smithi144.stdout:5/234: dwrite d4/d20/d24/f46 [0,4194304] 0 2024-09-10T11:29:32.800 INFO:tasks.workunit.client.0.smithi144.stdout:4/182: mkdir d14/d1e/d22/d34/d3e 0 2024-09-10T11:29:32.801 INFO:tasks.workunit.client.0.smithi144.stdout:9/247: chown d11/f3c 1648 1 2024-09-10T11:29:32.802 INFO:tasks.workunit.client.1.smithi178.stdout:8/690: link d5/db/d10/d20/f28 d5/db/fe5 0 2024-09-10T11:29:32.802 INFO:tasks.workunit.client.1.smithi178.stdout:3/566: link db/l12 db/d20/d3d/d63/d6a/lb0 0 2024-09-10T11:29:32.804 INFO:tasks.workunit.client.1.smithi178.stdout:2/784: creat d1/d7/d24/d46/d88/d9d/dab/d5e/f104 x:0 0 0 2024-09-10T11:29:32.806 INFO:tasks.workunit.client.1.smithi178.stdout:2/785: chown d1/d7/d24/d43 9227 1 2024-09-10T11:29:32.807 INFO:tasks.workunit.client.1.smithi178.stdout:3/567: symlink db/d20/d3d/d56/d64/lb1 0 2024-09-10T11:29:32.807 INFO:tasks.workunit.client.0.smithi144.stdout:3/212: unlink d4/dd/d22/d17/d18/f27 0 2024-09-10T11:29:32.807 INFO:tasks.workunit.client.1.smithi178.stdout:2/786: readlink d1/d7/d24/l2b 0 2024-09-10T11:29:32.808 INFO:tasks.workunit.client.0.smithi144.stdout:2/179: mkdir d7/de/d3d 0 2024-09-10T11:29:32.809 INFO:tasks.workunit.client.0.smithi144.stdout:0/156: read da/f15 [1173258,63277] 0 2024-09-10T11:29:32.810 INFO:tasks.workunit.client.0.smithi144.stdout:5/235: creat d4/d20/d24/d2d/d4b/f4e x:0 0 0 2024-09-10T11:29:32.813 INFO:tasks.workunit.client.1.smithi178.stdout:2/787: creat d1/d41/d8d/f105 x:0 0 0 2024-09-10T11:29:32.813 INFO:tasks.workunit.client.0.smithi144.stdout:9/248: rename d11/d12/d1b/d22/f4f to d11/d36/f58 0 2024-09-10T11:29:32.815 INFO:tasks.workunit.client.1.smithi178.stdout:5/654: rename d9/d19/d91 to d9/dc/dd/d37/dbd/dda 0 2024-09-10T11:29:32.817 INFO:tasks.workunit.client.1.smithi178.stdout:2/788: fdatasync d1/d7/d24/d7d/d84/d4b/f57 0 2024-09-10T11:29:32.818 INFO:tasks.workunit.client.1.smithi178.stdout:3/568: link db/d20/d3d/f53 db/d20/d3d/d63/da2/fb2 0 2024-09-10T11:29:32.820 INFO:tasks.workunit.client.0.smithi144.stdout:4/183: link fd d14/d1e/d22/d34/d3e/f3f 0 2024-09-10T11:29:32.820 INFO:tasks.workunit.client.0.smithi144.stdout:4/184: stat d14/f3a 0 2024-09-10T11:29:32.821 INFO:tasks.workunit.client.0.smithi144.stdout:4/185: chown d14/d1e/l2a 23824884 1 2024-09-10T11:29:32.822 INFO:tasks.workunit.client.1.smithi178.stdout:5/655: creat d9/dc/dd/d37/d3e/fdb x:0 0 0 2024-09-10T11:29:32.822 INFO:tasks.workunit.client.0.smithi144.stdout:5/236: mkdir d4/d5/d18/d38/d4f 0 2024-09-10T11:29:32.826 INFO:tasks.workunit.client.0.smithi144.stdout:9/249: creat d11/d36/f59 x:0 0 0 2024-09-10T11:29:32.826 INFO:tasks.workunit.client.0.smithi144.stdout:9/250: truncate d11/d12/d1b/f4d 214786 0 2024-09-10T11:29:32.827 INFO:tasks.workunit.client.1.smithi178.stdout:2/789: unlink d1/d22/fca 0 2024-09-10T11:29:32.832 INFO:tasks.workunit.client.0.smithi144.stdout:5/237: link d4/d20/f43 d4/d5/d32/d47/f50 0 2024-09-10T11:29:32.833 INFO:tasks.workunit.client.0.smithi144.stdout:3/213: getdents d4/dd/d21/d2f 0 2024-09-10T11:29:32.834 INFO:tasks.workunit.client.0.smithi144.stdout:3/214: dread - d4/dd/d21/f41 zero size 2024-09-10T11:29:32.837 INFO:tasks.workunit.client.1.smithi178.stdout:2/790: truncate d1/f69 792302 0 2024-09-10T11:29:32.838 INFO:tasks.workunit.client.0.smithi144.stdout:3/215: dread d4/dd/d22/d1d/f29 [0,4194304] 0 2024-09-10T11:29:32.838 INFO:tasks.workunit.client.1.smithi178.stdout:8/691: dwrite d5/d30/d33/d3a/f6c [0,4194304] 0 2024-09-10T11:29:32.839 INFO:tasks.workunit.client.0.smithi144.stdout:3/216: dread - d4/d28/f38 zero size 2024-09-10T11:29:32.839 INFO:tasks.workunit.client.0.smithi144.stdout:0/157: dwrite da/f1c [0,4194304] 0 2024-09-10T11:29:32.840 INFO:tasks.workunit.client.0.smithi144.stdout:9/251: rename d11/d12/d1b/d22/c51 to d11/c5a 0 2024-09-10T11:29:32.841 INFO:tasks.workunit.client.0.smithi144.stdout:3/217: rename d4 to d4/d28/d4e 22 2024-09-10T11:29:32.841 INFO:tasks.workunit.client.0.smithi144.stdout:3/218: chown d4/d1a 1800 1 2024-09-10T11:29:32.842 INFO:tasks.workunit.client.0.smithi144.stdout:2/180: truncate d7/da/f19 2368949 0 2024-09-10T11:29:32.842 INFO:tasks.workunit.client.0.smithi144.stdout:9/252: write d11/d12/d1b/d2c/f31 [4639833,105864] 0 2024-09-10T11:29:32.843 INFO:tasks.workunit.client.0.smithi144.stdout:0/158: symlink da/l25 0 2024-09-10T11:29:32.844 INFO:tasks.workunit.client.1.smithi178.stdout:8/692: creat d5/d30/d33/d8c/da7/fe6 x:0 0 0 2024-09-10T11:29:32.844 INFO:tasks.workunit.client.0.smithi144.stdout:0/159: stat l8 0 2024-09-10T11:29:32.845 INFO:tasks.workunit.client.0.smithi144.stdout:3/219: write d4/dd/d21/f42 [238740,121763] 0 2024-09-10T11:29:32.845 INFO:tasks.workunit.client.1.smithi178.stdout:8/693: write d5/db/d10/d53/f98 [1715954,105051] 0 2024-09-10T11:29:32.851 INFO:tasks.workunit.client.0.smithi144.stdout:5/238: truncate d4/d5/d18/f1f 129618 0 2024-09-10T11:29:32.856 INFO:tasks.workunit.client.0.smithi144.stdout:4/186: dwrite d14/f1b [0,4194304] 0 2024-09-10T11:29:32.856 INFO:tasks.workunit.client.0.smithi144.stdout:9/253: dread d11/f1c [0,4194304] 0 2024-09-10T11:29:32.856 INFO:tasks.workunit.client.0.smithi144.stdout:5/239: mkdir d4/d5/d18/d51 0 2024-09-10T11:29:32.857 INFO:tasks.workunit.client.0.smithi144.stdout:3/220: getdents d4/dd/d22/d1d 0 2024-09-10T11:29:32.858 INFO:tasks.workunit.client.0.smithi144.stdout:4/187: chown d14/f1b 8791228 1 2024-09-10T11:29:32.859 INFO:tasks.workunit.client.0.smithi144.stdout:2/181: dwrite d7/de/f28 [0,4194304] 0 2024-09-10T11:29:32.861 INFO:tasks.workunit.client.0.smithi144.stdout:9/254: write d11/d12/d1b/d22/f3d [3175216,97065] 0 2024-09-10T11:29:32.861 INFO:tasks.workunit.client.0.smithi144.stdout:5/240: mkdir d4/d52 0 2024-09-10T11:29:32.867 INFO:tasks.workunit.client.1.smithi178.stdout:2/791: truncate d1/d22/f26 1340546 0 2024-09-10T11:29:32.869 INFO:tasks.workunit.client.0.smithi144.stdout:5/241: symlink d4/d5/d18/l53 0 2024-09-10T11:29:32.871 INFO:tasks.workunit.client.0.smithi144.stdout:3/221: dwrite d4/dd/d22/d17/d18/f4c [0,4194304] 0 2024-09-10T11:29:32.872 INFO:tasks.workunit.client.1.smithi178.stdout:2/792: creat d1/d7/d24/d46/d88/d9d/dab/d5b/f106 x:0 0 0 2024-09-10T11:29:32.874 INFO:tasks.workunit.client.0.smithi144.stdout:3/222: rename d4/dd/d22/d17/d18 to d4/dd/d22/d17/d18/d34/d4b/d4f 22 2024-09-10T11:29:32.877 INFO:tasks.workunit.client.0.smithi144.stdout:9/255: creat d11/f5b x:0 0 0 2024-09-10T11:29:32.877 INFO:tasks.workunit.client.0.smithi144.stdout:2/182: link d7/f26 d7/de/d3d/f3e 0 2024-09-10T11:29:32.877 INFO:tasks.workunit.client.1.smithi178.stdout:2/793: link d1/d22/d4d/d9b/f75 d1/d7/d24/d7d/d84/d55/f107 0 2024-09-10T11:29:32.878 INFO:tasks.workunit.client.0.smithi144.stdout:2/183: dread - d7/de/f1b zero size 2024-09-10T11:29:32.880 INFO:tasks.workunit.client.0.smithi144.stdout:3/223: mknod d4/c50 0 2024-09-10T11:29:32.880 INFO:tasks.workunit.client.0.smithi144.stdout:9/256: read d11/d12/f14 [78404,25390] 0 2024-09-10T11:29:32.881 INFO:tasks.workunit.client.1.smithi178.stdout:2/794: mkdir d1/d7/d24/d46/d88/d9d/dab/d5b/d60/dd7/d108 0 2024-09-10T11:29:32.882 INFO:tasks.workunit.client.1.smithi178.stdout:8/694: dread d5/db/d10/d53/f98 [0,4194304] 0 2024-09-10T11:29:32.883 INFO:tasks.workunit.client.0.smithi144.stdout:2/184: rename d7/da/f14 to d7/d29/f3f 0 2024-09-10T11:29:32.883 INFO:tasks.workunit.client.1.smithi178.stdout:8/695: fdatasync d5/db/d10/d20/fb9 0 2024-09-10T11:29:32.884 INFO:tasks.workunit.client.0.smithi144.stdout:3/224: chown d4/d1a/c24 1144 1 2024-09-10T11:29:32.887 INFO:tasks.workunit.client.0.smithi144.stdout:5/242: truncate d4/d5/d18/f1f 267205 0 2024-09-10T11:29:32.888 INFO:tasks.workunit.client.1.smithi178.stdout:8/696: mkdir d5/db/d3d/de7 0 2024-09-10T11:29:32.888 INFO:tasks.workunit.client.1.smithi178.stdout:2/795: fdatasync d1/d7/d24/d46/fa4 0 2024-09-10T11:29:32.888 INFO:tasks.workunit.client.0.smithi144.stdout:4/188: dread d14/d1e/d22/d2b/f33 [0,4194304] 0 2024-09-10T11:29:32.889 INFO:tasks.workunit.client.1.smithi178.stdout:8/697: dread - d5/d30/d33/d8c/da7/fe6 zero size 2024-09-10T11:29:32.889 INFO:tasks.workunit.client.0.smithi144.stdout:2/185: mknod d7/de/d36/c40 0 2024-09-10T11:29:32.890 INFO:tasks.workunit.client.1.smithi178.stdout:2/796: fsync d1/d7/d24/d7d/d84/f98 0 2024-09-10T11:29:32.895 INFO:tasks.workunit.client.1.smithi178.stdout:2/797: rename d1/d7/d24/d46/d88/d9d/dab/d5e/f104 to d1/d22/d4d/d9b/f109 0 2024-09-10T11:29:32.896 INFO:tasks.workunit.client.0.smithi144.stdout:5/243: write d4/d5/d32/d47/f50 [889845,113119] 0 2024-09-10T11:29:32.897 INFO:tasks.workunit.client.0.smithi144.stdout:3/225: truncate d4/dd/d22/d17/d18/f48 138709 0 2024-09-10T11:29:32.899 INFO:tasks.workunit.client.0.smithi144.stdout:4/189: dwrite d14/d1e/f2f [0,4194304] 0 2024-09-10T11:29:32.901 INFO:tasks.workunit.client.0.smithi144.stdout:3/226: creat d4/dd/f51 x:0 0 0 2024-09-10T11:29:32.903 INFO:tasks.workunit.client.1.smithi178.stdout:8/698: link d5/db/d10/d20/d52/lde d5/db/d10/d20/d52/d67/dbb/le8 0 2024-09-10T11:29:32.903 INFO:tasks.workunit.client.0.smithi144.stdout:9/257: dwrite d11/d12/f14 [0,4194304] 0 2024-09-10T11:29:32.905 INFO:tasks.workunit.client.0.smithi144.stdout:3/227: dread d4/dd/d22/f3e [0,4194304] 0 2024-09-10T11:29:32.905 INFO:tasks.workunit.client.0.smithi144.stdout:4/190: read - d14/f15 zero size 2024-09-10T11:29:32.906 INFO:tasks.workunit.client.0.smithi144.stdout:4/191: chown d14/d1e/f2f 2973 1 2024-09-10T11:29:32.906 INFO:tasks.workunit.client.0.smithi144.stdout:3/228: write d4/dd/d22/fc [726614,41398] 0 2024-09-10T11:29:32.907 INFO:tasks.workunit.client.1.smithi178.stdout:8/699: rmdir d5/d30/d33/d3a 39 2024-09-10T11:29:32.910 INFO:tasks.workunit.client.0.smithi144.stdout:3/229: read d4/dd/d22/d17/f1f [138158,92786] 0 2024-09-10T11:29:32.912 INFO:tasks.workunit.client.0.smithi144.stdout:3/230: truncate d4/dd/d21/f41 439766 0 2024-09-10T11:29:32.912 INFO:tasks.workunit.client.0.smithi144.stdout:5/244: dread d4/fa [4194304,4194304] 0 2024-09-10T11:29:32.913 INFO:tasks.workunit.client.1.smithi178.stdout:2/798: getdents d1/d7/d24/d46/d88/d9d/dab/d5b 0 2024-09-10T11:29:32.913 INFO:tasks.workunit.client.0.smithi144.stdout:4/192: mkdir d14/d1e/d22/d34/d3e/d40 0 2024-09-10T11:29:32.914 INFO:tasks.workunit.client.1.smithi178.stdout:8/700: dread d5/d61/f8f [0,4194304] 0 2024-09-10T11:29:32.914 INFO:tasks.workunit.client.1.smithi178.stdout:8/701: chown d5/db/d3d 28784382 1 2024-09-10T11:29:32.918 INFO:tasks.workunit.client.0.smithi144.stdout:2/186: dread f6 [4194304,4194304] 0 2024-09-10T11:29:32.919 INFO:tasks.workunit.client.0.smithi144.stdout:4/193: dread d14/d1e/d22/f37 [0,4194304] 0 2024-09-10T11:29:32.922 INFO:tasks.workunit.client.0.smithi144.stdout:3/231: dwrite d4/dd/d21/f42 [0,4194304] 0 2024-09-10T11:29:32.922 INFO:tasks.workunit.client.1.smithi178.stdout:8/702: dwrite d5/d1d/f25 [0,4194304] 0 2024-09-10T11:29:32.922 INFO:tasks.workunit.client.0.smithi144.stdout:4/194: mknod d14/d1e/d22/d34/c41 0 2024-09-10T11:29:32.923 INFO:tasks.workunit.client.1.smithi178.stdout:8/703: chown d5/d1d 312 1 2024-09-10T11:29:32.924 INFO:tasks.workunit.client.1.smithi178.stdout:8/704: dread - d5/db/d1e/fa9 zero size 2024-09-10T11:29:32.924 INFO:tasks.workunit.client.0.smithi144.stdout:4/195: chown d14/c25 38 1 2024-09-10T11:29:32.925 INFO:tasks.workunit.client.0.smithi144.stdout:5/245: dread d4/d5/d18/f1f [0,4194304] 0 2024-09-10T11:29:32.927 INFO:tasks.workunit.client.0.smithi144.stdout:3/232: symlink d4/dd/d22/d17/l52 0 2024-09-10T11:29:32.927 INFO:tasks.workunit.client.0.smithi144.stdout:4/196: truncate d14/d1e/d22/d2b/f33 4763056 0 2024-09-10T11:29:32.927 INFO:tasks.workunit.client.0.smithi144.stdout:3/233: write f2 [23640,16003] 0 2024-09-10T11:29:32.928 INFO:tasks.workunit.client.1.smithi178.stdout:8/705: rename d5/db/d10/d20/d52/d5c/fd4 to d5/d30/db8/fe9 0 2024-09-10T11:29:32.932 INFO:tasks.workunit.client.1.smithi178.stdout:2/799: dwrite d1/d7/d24/f42 [0,4194304] 0 2024-09-10T11:29:32.933 INFO:tasks.workunit.client.0.smithi144.stdout:5/246: read d4/d5/d32/f33 [1738879,118818] 0 2024-09-10T11:29:32.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:32 smithi144 ceph-mon[27942]: pgmap v23: 65 pgs: 65 active+clean; 1.1 GiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 28 MiB/s rd, 138 MiB/s wr, 280 op/s 2024-09-10T11:29:32.934 INFO:tasks.workunit.client.0.smithi144.stdout:4/197: symlink d14/d1e/d22/l42 0 2024-09-10T11:29:32.935 INFO:tasks.workunit.client.0.smithi144.stdout:4/198: chown d14/d1e/f2f 728289 1 2024-09-10T11:29:32.935 INFO:tasks.workunit.client.0.smithi144.stdout:3/234: creat d4/dd/d22/d17/d18/d34/d4b/f53 x:0 0 0 2024-09-10T11:29:32.936 INFO:tasks.workunit.client.0.smithi144.stdout:3/235: chown d4/dd/d21/f23 158932 1 2024-09-10T11:29:32.938 INFO:tasks.workunit.client.0.smithi144.stdout:3/236: chown d4/dd/d22/d17/f1f 1479975 1 2024-09-10T11:29:32.942 INFO:tasks.workunit.client.0.smithi144.stdout:7/206: sync 2024-09-10T11:29:32.943 INFO:tasks.workunit.client.0.smithi144.stdout:8/126: sync 2024-09-10T11:29:32.943 INFO:tasks.workunit.client.1.smithi178.stdout:8/706: truncate d5/db/fe 853425 0 2024-09-10T11:29:32.945 INFO:tasks.workunit.client.0.smithi144.stdout:5/247: getdents d4/d20/d22 0 2024-09-10T11:29:32.945 INFO:tasks.workunit.client.0.smithi144.stdout:3/237: chown d4/dd/d22/d17/d18/f2e 105 1 2024-09-10T11:29:32.947 INFO:tasks.workunit.client.1.smithi178.stdout:8/707: chown d5/d30/d33/d3a/f6c 32 1 2024-09-10T11:29:32.948 INFO:tasks.workunit.client.0.smithi144.stdout:8/127: symlink d0/d9/d1f/l2e 0 2024-09-10T11:29:32.949 INFO:tasks.workunit.client.1.smithi178.stdout:2/800: dwrite d1/d7/d24/d7d/d84/d55/f73 [0,4194304] 0 2024-09-10T11:29:32.949 INFO:tasks.workunit.client.0.smithi144.stdout:9/258: dread d11/d12/d1b/d22/f3d [0,4194304] 0 2024-09-10T11:29:32.950 INFO:tasks.workunit.client.1.smithi178.stdout:8/708: unlink d5/f69 0 2024-09-10T11:29:32.950 INFO:tasks.workunit.client.0.smithi144.stdout:3/238: dread d4/d1a/f44 [0,4194304] 0 2024-09-10T11:29:32.951 INFO:tasks.workunit.client.0.smithi144.stdout:4/199: write d14/f1f [802453,119669] 0 2024-09-10T11:29:32.952 INFO:tasks.workunit.client.0.smithi144.stdout:8/128: creat d0/d1b/f2f x:0 0 0 2024-09-10T11:29:32.953 INFO:tasks.workunit.client.1.smithi178.stdout:8/709: creat d5/d30/d33/d8c/fea x:0 0 0 2024-09-10T11:29:32.953 INFO:tasks.workunit.client.0.smithi144.stdout:5/248: rmdir d4/d5/d18/d38/d4f 0 2024-09-10T11:29:32.954 INFO:tasks.workunit.client.1.smithi178.stdout:8/710: fdatasync d5/db/d10/d70/fcb 0 2024-09-10T11:29:32.954 INFO:tasks.workunit.client.1.smithi178.stdout:1/546: sync 2024-09-10T11:29:32.954 INFO:tasks.workunit.client.1.smithi178.stdout:9/653: sync 2024-09-10T11:29:32.954 INFO:tasks.workunit.client.1.smithi178.stdout:0/598: sync 2024-09-10T11:29:32.954 INFO:tasks.workunit.client.1.smithi178.stdout:7/545: sync 2024-09-10T11:29:32.955 INFO:tasks.workunit.client.1.smithi178.stdout:2/801: rename d1/d7/d24/d7d/d84/f37 to d1/d41/d14/d11/db9/dc1/f10a 0 2024-09-10T11:29:32.957 INFO:tasks.workunit.client.1.smithi178.stdout:8/711: creat d5/db/d3d/feb x:0 0 0 2024-09-10T11:29:32.958 INFO:tasks.workunit.client.0.smithi144.stdout:3/239: dread d4/dd/f1b [0,4194304] 0 2024-09-10T11:29:32.958 INFO:tasks.workunit.client.1.smithi178.stdout:1/547: write d3/d9/d98/dac/d3a/d9f/fa2 [1667059,112245] 0 2024-09-10T11:29:32.959 INFO:tasks.workunit.client.0.smithi144.stdout:9/259: rename d11/f5b to d11/d12/d1b/d41/f5c 0 2024-09-10T11:29:32.959 INFO:tasks.workunit.client.0.smithi144.stdout:8/129: write d0/f10 [1454025,39845] 0 2024-09-10T11:29:32.960 INFO:tasks.workunit.client.1.smithi178.stdout:0/599: fdatasync d0/dc/d39/d50/f71 0 2024-09-10T11:29:32.961 INFO:tasks.workunit.client.0.smithi144.stdout:3/240: fsync d4/dd/d22/f3e 0 2024-09-10T11:29:32.961 INFO:tasks.workunit.client.0.smithi144.stdout:3/241: stat d4/dd 0 2024-09-10T11:29:32.962 INFO:tasks.workunit.client.1.smithi178.stdout:7/546: creat d6/d10/d1d/d22/d50/fc7 x:0 0 0 2024-09-10T11:29:32.963 INFO:tasks.workunit.client.0.smithi144.stdout:4/200: rename d14/c2c to d14/d1e/d22/d34/c43 0 2024-09-10T11:29:32.964 INFO:tasks.workunit.client.1.smithi178.stdout:8/712: symlink d5/d61/lec 0 2024-09-10T11:29:32.965 INFO:tasks.workunit.client.1.smithi178.stdout:1/548: symlink d3/d9/d1a/d48/lcb 0 2024-09-10T11:29:32.967 INFO:tasks.workunit.client.1.smithi178.stdout:0/600: fsync d0/dc/d39/d7c/f9b 0 2024-09-10T11:29:32.967 INFO:tasks.workunit.client.0.smithi144.stdout:9/260: rename ld to d11/d12/l5d 0 2024-09-10T11:29:32.968 INFO:tasks.workunit.client.1.smithi178.stdout:1/549: chown d3/d9/d98/dac/d2a/c91 13 1 2024-09-10T11:29:32.969 INFO:tasks.workunit.client.1.smithi178.stdout:2/802: unlink d1/d41/d14/cf7 0 2024-09-10T11:29:32.969 INFO:tasks.workunit.client.0.smithi144.stdout:3/242: dwrite d4/d28/f38 [0,4194304] 0 2024-09-10T11:29:32.970 INFO:tasks.workunit.client.0.smithi144.stdout:3/243: write d4/f32 [388945,51866] 0 2024-09-10T11:29:32.971 INFO:tasks.workunit.client.1.smithi178.stdout:8/713: stat d5/d30/db4/cd6 0 2024-09-10T11:29:32.972 INFO:tasks.workunit.client.0.smithi144.stdout:4/201: creat d14/d1e/d22/d34/d3e/d40/f44 x:0 0 0 2024-09-10T11:29:32.974 INFO:tasks.workunit.client.0.smithi144.stdout:5/249: write d4/d5/d18/f1f [272131,44756] 0 2024-09-10T11:29:32.974 INFO:tasks.workunit.client.0.smithi144.stdout:9/261: write d11/f1c [2199715,129577] 0 2024-09-10T11:29:32.977 INFO:tasks.workunit.client.0.smithi144.stdout:9/262: chown d11/d12/d1b/d22/l33 0 1 2024-09-10T11:29:32.978 INFO:tasks.workunit.client.1.smithi178.stdout:9/654: rename d2/d19/d1c/d2a/f6d to d2/df/d31/d37/d3d/fc3 0 2024-09-10T11:29:32.979 INFO:tasks.workunit.client.0.smithi144.stdout:8/130: truncate d0/d9/d1f/f1c 1276948 0 2024-09-10T11:29:32.979 INFO:tasks.workunit.client.0.smithi144.stdout:3/244: dwrite d4/dd/d21/d2f/f3a [4194304,4194304] 0 2024-09-10T11:29:32.982 INFO:tasks.workunit.client.1.smithi178.stdout:1/550: mkdir d3/d9/d98/dac/d3a/d50/db5/dcc 0 2024-09-10T11:29:32.983 INFO:tasks.workunit.client.0.smithi144.stdout:3/245: readlink d4/dd/l11 0 2024-09-10T11:29:32.983 INFO:tasks.workunit.client.0.smithi144.stdout:8/131: read - d0/d9/f1d zero size 2024-09-10T11:29:32.983 INFO:tasks.workunit.client.0.smithi144.stdout:9/263: dread d11/d12/f26 [0,4194304] 0 2024-09-10T11:29:32.983 INFO:tasks.workunit.client.0.smithi144.stdout:9/264: stat l1 0 2024-09-10T11:29:32.983 INFO:tasks.workunit.client.0.smithi144.stdout:5/250: mknod d4/d20/d24/d2d/d4b/c54 0 2024-09-10T11:29:32.984 INFO:tasks.workunit.client.0.smithi144.stdout:9/265: write d11/f21 [3661126,32571] 0 2024-09-10T11:29:32.984 INFO:tasks.workunit.client.0.smithi144.stdout:3/246: mknod d4/dd/d21/c54 0 2024-09-10T11:29:32.986 INFO:tasks.workunit.client.1.smithi178.stdout:8/714: creat d5/d61/fed x:0 0 0 2024-09-10T11:29:32.987 INFO:tasks.workunit.client.1.smithi178.stdout:0/601: dwrite d0/dc/d39/d4c/f57 [4194304,4194304] 0 2024-09-10T11:29:32.988 INFO:tasks.workunit.client.1.smithi178.stdout:2/803: getdents d1/d7/d24/d7d/d84/d55/da1/dd6 0 2024-09-10T11:29:32.993 INFO:tasks.workunit.client.1.smithi178.stdout:0/602: symlink d0/dc/d39/d4c/d43/lc1 0 2024-09-10T11:29:32.993 INFO:tasks.workunit.client.0.smithi144.stdout:3/247: mkdir d4/dd/d22/d17/d18/d34/d4b/d55 0 2024-09-10T11:29:32.994 INFO:tasks.workunit.client.0.smithi144.stdout:8/132: dwrite d0/d1b/f2a [0,4194304] 0 2024-09-10T11:29:32.995 INFO:tasks.workunit.client.1.smithi178.stdout:2/804: unlink d1/d41/d14/d11/db9/dc1/f10a 0 2024-09-10T11:29:32.996 INFO:tasks.workunit.client.0.smithi144.stdout:4/202: dwrite d14/d1e/d22/d2b/d35/f38 [4194304,4194304] 0 2024-09-10T11:29:32.997 INFO:tasks.workunit.client.1.smithi178.stdout:0/603: fsync d0/dc/d39/d4c/d43/f8a 0 2024-09-10T11:29:32.997 INFO:tasks.workunit.client.0.smithi144.stdout:5/251: getdents d4/d20/d24/d2d 0 2024-09-10T11:29:32.999 INFO:tasks.workunit.client.1.smithi178.stdout:2/805: mkdir d1/d7/d8e/d10b 0 2024-09-10T11:29:33.000 INFO:tasks.workunit.client.0.smithi144.stdout:3/248: mknod d4/dd/d22/d17/c56 0 2024-09-10T11:29:33.002 INFO:tasks.workunit.client.1.smithi178.stdout:8/715: dwrite d5/db/d10/d20/dc0/fe3 [0,4194304] 0 2024-09-10T11:29:33.006 INFO:tasks.workunit.client.1.smithi178.stdout:2/806: mknod d1/d41/d14/d11/db9/dc1/c10c 0 2024-09-10T11:29:33.011 INFO:tasks.workunit.client.0.smithi144.stdout:3/249: dread d4/dd/d21/f42 [0,4194304] 0 2024-09-10T11:29:33.011 INFO:tasks.workunit.client.0.smithi144.stdout:8/133: dwrite d0/f24 [0,4194304] 0 2024-09-10T11:29:33.012 INFO:tasks.workunit.client.1.smithi178.stdout:1/551: dread d3/d9/d98/dac/d3a/d50/d6c/d78/db8/fbf [0,4194304] 0 2024-09-10T11:29:33.013 INFO:tasks.workunit.client.1.smithi178.stdout:9/655: write d2/d19/d1c/d77/f90 [1650945,110500] 0 2024-09-10T11:29:33.014 INFO:tasks.workunit.client.0.smithi144.stdout:8/134: stat d0/c7 0 2024-09-10T11:29:33.016 INFO:tasks.workunit.client.1.smithi178.stdout:0/604: dwrite d0/dc/f61 [0,4194304] 0 2024-09-10T11:29:33.018 INFO:tasks.workunit.client.1.smithi178.stdout:8/716: dread d5/db/d10/d35/f5a [0,4194304] 0 2024-09-10T11:29:33.021 INFO:tasks.workunit.client.1.smithi178.stdout:2/807: symlink d1/l10d 0 2024-09-10T11:29:33.022 INFO:tasks.workunit.client.0.smithi144.stdout:5/252: truncate d4/d5/d32/d47/f50 585006 0 2024-09-10T11:29:33.026 INFO:tasks.workunit.client.0.smithi144.stdout:9/266: dread d11/f17 [0,4194304] 0 2024-09-10T11:29:33.026 INFO:tasks.workunit.client.1.smithi178.stdout:1/552: symlink d3/d9/d98/dac/d3a/d9f/lcd 0 2024-09-10T11:29:33.031 INFO:tasks.workunit.client.0.smithi144.stdout:3/250: getdents d4/dd/d22/d17/d18 0 2024-09-10T11:29:33.032 INFO:tasks.workunit.client.1.smithi178.stdout:2/808: creat d1/d7/d24/d46/d88/d9d/dab/d5b/f10e x:0 0 0 2024-09-10T11:29:33.032 INFO:tasks.workunit.client.1.smithi178.stdout:7/547: dread d6/d10/d1d/f20 [0,4194304] 0 2024-09-10T11:29:33.033 INFO:tasks.workunit.client.0.smithi144.stdout:5/253: creat d4/f55 x:0 0 0 2024-09-10T11:29:33.034 INFO:tasks.workunit.client.1.smithi178.stdout:7/548: fsync d6/d10/d1e/d2d/d2b/da0/f7a 0 2024-09-10T11:29:33.035 INFO:tasks.workunit.client.1.smithi178.stdout:8/717: creat d5/d30/db4/fee x:0 0 0 2024-09-10T11:29:33.036 INFO:tasks.workunit.client.1.smithi178.stdout:8/718: fsync d5/d30/fb6 0 2024-09-10T11:29:33.037 INFO:tasks.workunit.client.1.smithi178.stdout:2/809: unlink d1/ldb 0 2024-09-10T11:29:33.037 INFO:tasks.workunit.client.0.smithi144.stdout:8/135: dwrite d0/f24 [4194304,4194304] 0 2024-09-10T11:29:33.038 INFO:tasks.workunit.client.1.smithi178.stdout:9/656: link d2/df/d31/d37/c8c d2/d13/d40/d56/dbe/cc4 0 2024-09-10T11:29:33.038 INFO:tasks.workunit.client.0.smithi144.stdout:5/254: mkdir d4/d5/d56 0 2024-09-10T11:29:33.039 INFO:tasks.workunit.client.0.smithi144.stdout:4/203: dread f9 [0,4194304] 0 2024-09-10T11:29:33.039 INFO:tasks.workunit.client.0.smithi144.stdout:9/267: link d11/d36/c34 d11/d12/d1b/d41/c5e 0 2024-09-10T11:29:33.039 INFO:tasks.workunit.client.1.smithi178.stdout:9/657: write d2/d19/d1c/d2a/f92 [1452242,40298] 0 2024-09-10T11:29:33.039 INFO:tasks.workunit.client.0.smithi144.stdout:8/136: dread - d0/d1b/f2f zero size 2024-09-10T11:29:33.042 INFO:tasks.workunit.client.0.smithi144.stdout:5/255: truncate d4/d20/d24/f3e 870628 0 2024-09-10T11:29:33.043 INFO:tasks.workunit.client.0.smithi144.stdout:4/204: write d14/d1e/d22/d34/d3e/d40/f44 [506925,42427] 0 2024-09-10T11:29:33.043 INFO:tasks.workunit.client.0.smithi144.stdout:3/251: link d4/d28/c2c d4/c57 0 2024-09-10T11:29:33.044 INFO:tasks.workunit.client.1.smithi178.stdout:9/658: write d2/d19/f8e [7705218,1645] 0 2024-09-10T11:29:33.047 INFO:tasks.workunit.client.1.smithi178.stdout:8/719: mkdir d5/def 0 2024-09-10T11:29:33.048 INFO:tasks.workunit.client.1.smithi178.stdout:9/659: read d2/df/d31/d37/d3d/f6e [492450,14827] 0 2024-09-10T11:29:33.051 INFO:tasks.workunit.client.1.smithi178.stdout:8/720: creat d5/db/d10/de4/ff0 x:0 0 0 2024-09-10T11:29:33.052 INFO:tasks.workunit.client.0.smithi144.stdout:4/205: creat d14/d1e/f45 x:0 0 0 2024-09-10T11:29:33.054 INFO:tasks.workunit.client.1.smithi178.stdout:8/721: creat d5/db/d10/d35/ff1 x:0 0 0 2024-09-10T11:29:33.054 INFO:tasks.workunit.client.0.smithi144.stdout:9/268: dwrite d11/d36/f52 [0,4194304] 0 2024-09-10T11:29:33.060 INFO:tasks.workunit.client.0.smithi144.stdout:5/256: dwrite d4/d5/f28 [0,4194304] 0 2024-09-10T11:29:33.062 INFO:tasks.workunit.client.0.smithi144.stdout:9/269: write d11/d36/f59 [612213,95935] 0 2024-09-10T11:29:33.062 INFO:tasks.workunit.client.0.smithi144.stdout:4/206: mknod d14/d1e/d22/d2b/c46 0 2024-09-10T11:29:33.062 INFO:tasks.workunit.client.1.smithi178.stdout:9/660: dread d2/d8/fa [0,4194304] 0 2024-09-10T11:29:33.064 INFO:tasks.workunit.client.0.smithi144.stdout:3/252: dwrite d4/dd/d22/fc [0,4194304] 0 2024-09-10T11:29:33.067 INFO:tasks.workunit.client.1.smithi178.stdout:9/661: dread d2/d8/f1a [0,4194304] 0 2024-09-10T11:29:33.067 INFO:tasks.workunit.client.0.smithi144.stdout:9/270: symlink d11/d12/l5f 0 2024-09-10T11:29:33.068 INFO:tasks.workunit.client.0.smithi144.stdout:4/207: write d14/f1f [187827,860] 0 2024-09-10T11:29:33.069 INFO:tasks.workunit.client.1.smithi178.stdout:7/549: write d6/d10/d1e/f45 [962943,59159] 0 2024-09-10T11:29:33.069 INFO:tasks.workunit.client.1.smithi178.stdout:9/662: dread - d2/d13/fb6 zero size 2024-09-10T11:29:33.070 INFO:tasks.workunit.client.1.smithi178.stdout:7/550: stat d6/d10/d1e/d2d/f21 0 2024-09-10T11:29:33.071 INFO:tasks.workunit.client.1.smithi178.stdout:9/663: chown d2/d19/d1c/d2a/f92 221061 1 2024-09-10T11:29:33.071 INFO:tasks.workunit.client.1.smithi178.stdout:7/551: chown d6/d10/d1d/d4a/f57 204631 1 2024-09-10T11:29:33.072 INFO:tasks.workunit.client.0.smithi144.stdout:3/253: write d4/dd/d21/f42 [1378038,53664] 0 2024-09-10T11:29:33.072 INFO:tasks.workunit.client.0.smithi144.stdout:8/137: dwrite d0/d9/d1f/f19 [0,4194304] 0 2024-09-10T11:29:33.073 INFO:tasks.workunit.client.1.smithi178.stdout:7/552: fdatasync d6/d10/d1e/f8e 0 2024-09-10T11:29:33.073 INFO:tasks.workunit.client.1.smithi178.stdout:2/810: write d1/d41/d14/f82 [933552,97050] 0 2024-09-10T11:29:33.074 INFO:tasks.workunit.client.0.smithi144.stdout:9/271: write d11/d36/f52 [1044068,111031] 0 2024-09-10T11:29:33.078 INFO:tasks.workunit.client.0.smithi144.stdout:8/138: fsync d0/fc 0 2024-09-10T11:29:33.079 INFO:tasks.workunit.client.1.smithi178.stdout:9/664: rename d2/df/d31/d37/f5f to d2/d8/fc5 0 2024-09-10T11:29:33.079 INFO:tasks.workunit.client.1.smithi178.stdout:0/605: dread d0/f8 [0,4194304] 0 2024-09-10T11:29:33.081 INFO:tasks.workunit.client.0.smithi144.stdout:5/257: dwrite d4/d20/d24/d2d/d4b/f4e [0,4194304] 0 2024-09-10T11:29:33.081 INFO:tasks.workunit.client.1.smithi178.stdout:7/553: mknod d6/d10/d1d/d22/d50/d98/da6/cc8 0 2024-09-10T11:29:33.082 INFO:tasks.workunit.client.0.smithi144.stdout:6/180: sync 2024-09-10T11:29:33.082 INFO:tasks.workunit.client.0.smithi144.stdout:1/304: sync 2024-09-10T11:29:33.083 INFO:tasks.workunit.client.1.smithi178.stdout:2/811: dread - d1/d7/d24/d46/d88/d9d/dab/d5b/d7c/fa7 zero size 2024-09-10T11:29:33.084 INFO:tasks.workunit.client.1.smithi178.stdout:7/554: write d6/d10/d34/f6e [28734,116907] 0 2024-09-10T11:29:33.085 INFO:tasks.workunit.client.1.smithi178.stdout:2/812: stat d1/d22/d4d/d9b/f109 0 2024-09-10T11:29:33.089 INFO:tasks.workunit.client.1.smithi178.stdout:2/813: chown d1/d7/d24/d46/f102 6 1 2024-09-10T11:29:33.090 INFO:tasks.workunit.client.0.smithi144.stdout:8/139: creat d0/d9/d23/d26/f30 x:0 0 0 2024-09-10T11:29:33.091 INFO:tasks.workunit.client.0.smithi144.stdout:1/305: symlink d5/df/d4b/l58 0 2024-09-10T11:29:33.093 INFO:tasks.workunit.client.0.smithi144.stdout:3/254: dwrite d4/dd/d22/d17/d18/f4c [0,4194304] 0 2024-09-10T11:29:33.093 INFO:tasks.workunit.client.0.smithi144.stdout:4/208: link fd d14/d1e/d22/f47 0 2024-09-10T11:29:33.094 INFO:tasks.workunit.client.1.smithi178.stdout:0/606: dwrite d0/dc/d1f/f26 [0,4194304] 0 2024-09-10T11:29:33.095 INFO:tasks.workunit.client.0.smithi144.stdout:5/258: dread d4/d20/d24/d2d/d4b/f4e [0,4194304] 0 2024-09-10T11:29:33.095 INFO:tasks.workunit.client.0.smithi144.stdout:8/140: chown d0/d9/d23/d26/f30 219 1 2024-09-10T11:29:33.095 INFO:tasks.workunit.client.1.smithi178.stdout:7/555: rename d6/d10/d1e/f8e to d6/fc9 0 2024-09-10T11:29:33.096 INFO:tasks.workunit.client.0.smithi144.stdout:1/306: write d5/d2a/f4d [209289,33421] 0 2024-09-10T11:29:33.096 INFO:tasks.workunit.client.0.smithi144.stdout:5/259: readlink d4/d5/d18/l53 0 2024-09-10T11:29:33.097 INFO:tasks.workunit.client.0.smithi144.stdout:3/255: write d4/dd/d22/d1d/f2a [2757464,71678] 0 2024-09-10T11:29:33.097 INFO:tasks.workunit.client.0.smithi144.stdout:6/181: rename d9/dc to d9/dd/d3e/d47 0 2024-09-10T11:29:33.098 INFO:tasks.workunit.client.0.smithi144.stdout:5/260: write d4/d5/f23 [193370,22483] 0 2024-09-10T11:29:33.099 INFO:tasks.workunit.client.1.smithi178.stdout:0/607: dread d0/f8 [4194304,4194304] 0 2024-09-10T11:29:33.102 INFO:tasks.workunit.client.1.smithi178.stdout:7/556: mkdir d6/d10/d1d/dca 0 2024-09-10T11:29:33.105 INFO:tasks.workunit.client.1.smithi178.stdout:0/608: read d0/dc/d39/d50/d5e/f7d [1044280,25109] 0 2024-09-10T11:29:33.105 INFO:tasks.workunit.client.0.smithi144.stdout:1/307: creat d5/df/d4b/d51/f59 x:0 0 0 2024-09-10T11:29:33.106 INFO:tasks.workunit.client.1.smithi178.stdout:7/557: symlink d6/d10/d1d/d22/d50/d7e/lcb 0 2024-09-10T11:29:33.107 INFO:tasks.workunit.client.1.smithi178.stdout:7/558: chown d6/d10/d1d/d4a/d79/d83 0 1 2024-09-10T11:29:33.107 INFO:tasks.workunit.client.0.smithi144.stdout:1/308: chown d5/dd/f1d 4077 1 2024-09-10T11:29:33.108 INFO:tasks.workunit.client.0.smithi144.stdout:6/182: rmdir d9/dd/d3e/d47/d13 39 2024-09-10T11:29:33.110 INFO:tasks.workunit.client.0.smithi144.stdout:3/256: dwrite d4/dd/d22/d17/d18/f2e [0,4194304] 0 2024-09-10T11:29:33.111 INFO:tasks.workunit.client.0.smithi144.stdout:5/261: symlink d4/d20/d24/d2d/l57 0 2024-09-10T11:29:33.115 INFO:tasks.workunit.client.1.smithi178.stdout:9/665: write d2/df/d31/d37/d3d/f83 [197318,122086] 0 2024-09-10T11:29:33.115 INFO:tasks.workunit.client.1.smithi178.stdout:0/609: dread d0/dc/d39/d4c/f3a [0,4194304] 0 2024-09-10T11:29:33.116 INFO:tasks.workunit.client.0.smithi144.stdout:1/309: symlink d5/l5a 0 2024-09-10T11:29:33.116 INFO:tasks.workunit.client.1.smithi178.stdout:7/559: dwrite d6/d10/d34/d6a/d6f/faf [0,4194304] 0 2024-09-10T11:29:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:32 smithi178 ceph-mon[33514]: pgmap v23: 65 pgs: 65 active+clean; 1.1 GiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 28 MiB/s rd, 138 MiB/s wr, 280 op/s 2024-09-10T11:29:33.118 INFO:tasks.workunit.client.0.smithi144.stdout:1/310: write d5/f23 [2430200,63341] 0 2024-09-10T11:29:33.120 INFO:tasks.workunit.client.0.smithi144.stdout:8/141: dread d0/d9/d1f/f1c [0,4194304] 0 2024-09-10T11:29:33.120 INFO:tasks.workunit.client.0.smithi144.stdout:5/262: chown d4/d20/d22/l36 28665 1 2024-09-10T11:29:33.123 INFO:tasks.workunit.client.0.smithi144.stdout:6/183: dwrite d9/dd/f1a [4194304,4194304] 0 2024-09-10T11:29:33.124 INFO:tasks.workunit.client.1.smithi178.stdout:9/666: creat d2/d19/d1c/da5/fc6 x:0 0 0 2024-09-10T11:29:33.124 INFO:tasks.workunit.client.0.smithi144.stdout:4/209: rename d14/d1e/f27 to d14/d1e/d22/f48 0 2024-09-10T11:29:33.126 INFO:tasks.workunit.client.0.smithi144.stdout:6/184: write d9/dd/d11/f46 [274008,60742] 0 2024-09-10T11:29:33.126 INFO:tasks.workunit.client.1.smithi178.stdout:7/560: creat d6/dbf/fcc x:0 0 0 2024-09-10T11:29:33.128 INFO:tasks.workunit.client.1.smithi178.stdout:7/561: chown d6/d10/d1d/d22/d50/d98/da6/cc8 1375 1 2024-09-10T11:29:33.131 INFO:tasks.workunit.client.0.smithi144.stdout:8/142: creat d0/d1b/f31 x:0 0 0 2024-09-10T11:29:33.131 INFO:tasks.workunit.client.1.smithi178.stdout:9/667: rmdir d2/d51/dad 0 2024-09-10T11:29:33.132 INFO:tasks.workunit.client.0.smithi144.stdout:3/257: creat d4/f58 x:0 0 0 2024-09-10T11:29:33.133 INFO:tasks.workunit.client.0.smithi144.stdout:1/311: rename d5/dd/d22/f37 to d5/d2a/f5b 0 2024-09-10T11:29:33.133 INFO:tasks.workunit.client.0.smithi144.stdout:4/210: symlink d14/d1e/d22/d2b/d35/l49 0 2024-09-10T11:29:33.135 INFO:tasks.workunit.client.1.smithi178.stdout:7/562: creat d6/d10/d1d/d4a/d79/d83/fcd x:0 0 0 2024-09-10T11:29:33.135 INFO:tasks.workunit.client.1.smithi178.stdout:9/668: creat d2/df/d31/d37/d3d/fc7 x:0 0 0 2024-09-10T11:29:33.138 INFO:tasks.workunit.client.0.smithi144.stdout:6/185: rename d9/dd/c2b to d9/dd/d3a/c48 0 2024-09-10T11:29:33.138 INFO:tasks.workunit.client.1.smithi178.stdout:0/610: truncate d0/dc/d39/d50/d5e/f7d 2219666 0 2024-09-10T11:29:33.139 INFO:tasks.workunit.client.0.smithi144.stdout:1/312: symlink d5/df/d4b/l5c 0 2024-09-10T11:29:33.140 INFO:tasks.workunit.client.1.smithi178.stdout:7/563: write d6/d10/d1e/d2d/d2b/da0/d55/fb0 [395513,42458] 0 2024-09-10T11:29:33.140 INFO:tasks.workunit.client.0.smithi144.stdout:3/258: dwrite d4/dd/d21/f42 [0,4194304] 0 2024-09-10T11:29:33.141 INFO:tasks.workunit.client.0.smithi144.stdout:1/313: fsync d5/d4a/d44/f45 0 2024-09-10T11:29:33.141 INFO:tasks.workunit.client.1.smithi178.stdout:2/814: dread d1/d7/d24/d7d/d84/d55/f107 [0,4194304] 0 2024-09-10T11:29:33.142 INFO:tasks.workunit.client.1.smithi178.stdout:0/611: readlink d0/dc/d39/d50/l76 0 2024-09-10T11:29:33.143 INFO:tasks.workunit.client.0.smithi144.stdout:1/314: chown d5/df/d4b/f54 0 1 2024-09-10T11:29:33.145 INFO:tasks.workunit.client.1.smithi178.stdout:9/669: dwrite d2/d13/d75/f5d [0,4194304] 0 2024-09-10T11:29:33.146 INFO:tasks.workunit.client.0.smithi144.stdout:1/315: truncate d5/df/d4b/d51/f59 228303 0 2024-09-10T11:29:33.150 INFO:tasks.workunit.client.1.smithi178.stdout:2/815: creat d1/d7/d24/d7d/d84/d55/f10f x:0 0 0 2024-09-10T11:29:33.150 INFO:tasks.workunit.client.1.smithi178.stdout:9/670: chown d2/df/cb3 315763 1 2024-09-10T11:29:33.151 INFO:tasks.workunit.client.0.smithi144.stdout:3/259: rename d4/dd/l19 to d4/dd/d22/d17/d18/d34/d4b/d55/l59 0 2024-09-10T11:29:33.152 INFO:tasks.workunit.client.0.smithi144.stdout:5/263: write d4/d20/f43 [697629,22419] 0 2024-09-10T11:29:33.153 INFO:tasks.workunit.client.0.smithi144.stdout:4/211: creat d14/d1e/d22/f4a x:0 0 0 2024-09-10T11:29:33.153 INFO:tasks.workunit.client.1.smithi178.stdout:0/612: dwrite d0/dc/d1f/f7a [4194304,4194304] 0 2024-09-10T11:29:33.155 INFO:tasks.workunit.client.0.smithi144.stdout:6/186: link d9/d1f/f26 d9/dd/f49 0 2024-09-10T11:29:33.156 INFO:tasks.workunit.client.0.smithi144.stdout:8/143: dread d0/fc [0,4194304] 0 2024-09-10T11:29:33.157 INFO:tasks.workunit.client.1.smithi178.stdout:2/816: creat d1/d41/d8d/d9f/f110 x:0 0 0 2024-09-10T11:29:33.159 INFO:tasks.workunit.client.0.smithi144.stdout:1/316: rename d5/d2a/d4e to d5/df/d4b/d5d 0 2024-09-10T11:29:33.159 INFO:tasks.workunit.client.1.smithi178.stdout:9/671: symlink d2/d19/d1c/da5/lc8 0 2024-09-10T11:29:33.160 INFO:tasks.workunit.client.0.smithi144.stdout:5/264: mknod d4/d5/d56/c58 0 2024-09-10T11:29:33.160 INFO:tasks.workunit.client.0.smithi144.stdout:1/317: fsync d5/f23 0 2024-09-10T11:29:33.163 INFO:tasks.workunit.client.1.smithi178.stdout:0/613: symlink d0/d2c/d93/lc2 0 2024-09-10T11:29:33.164 INFO:tasks.workunit.client.1.smithi178.stdout:9/672: creat d2/d19/d1c/d77/fc9 x:0 0 0 2024-09-10T11:29:33.164 INFO:tasks.workunit.client.1.smithi178.stdout:0/614: chown d0/dc/d1f/dae/fbc 809 1 2024-09-10T11:29:33.165 INFO:tasks.workunit.client.1.smithi178.stdout:4/524: sync 2024-09-10T11:29:33.166 INFO:tasks.workunit.client.1.smithi178.stdout:9/673: unlink d2/d13/fa2 0 2024-09-10T11:29:33.173 INFO:tasks.workunit.client.1.smithi178.stdout:4/525: dwrite d0/d8/da/f2b [0,4194304] 0 2024-09-10T11:29:33.175 INFO:tasks.workunit.client.1.smithi178.stdout:7/564: dread d6/fd [0,4194304] 0 2024-09-10T11:29:33.175 INFO:tasks.workunit.client.1.smithi178.stdout:4/526: dread - d0/d10/fa2 zero size 2024-09-10T11:29:33.179 INFO:tasks.workunit.client.1.smithi178.stdout:7/565: symlink d6/d10/d1e/d2d/d2b/da0/lce 0 2024-09-10T11:29:33.182 INFO:tasks.workunit.client.1.smithi178.stdout:7/566: rename d6/d10 to d6/d10/d1e/d2d/d2b/dcf 22 2024-09-10T11:29:33.184 INFO:tasks.workunit.client.1.smithi178.stdout:0/615: truncate d0/dc/d1f/f74 228961 0 2024-09-10T11:29:33.186 INFO:tasks.workunit.client.1.smithi178.stdout:0/616: stat d0/dc/d1f/d2d/dac 0 2024-09-10T11:29:33.188 INFO:tasks.workunit.client.1.smithi178.stdout:7/567: creat d6/d10/d1d/daa/fd0 x:0 0 0 2024-09-10T11:29:33.188 INFO:tasks.workunit.client.1.smithi178.stdout:2/817: dwrite d1/d7/d24/d43/f87 [0,4194304] 0 2024-09-10T11:29:33.188 INFO:tasks.workunit.client.1.smithi178.stdout:9/674: dread d2/df/d31/d37/fa3 [0,4194304] 0 2024-09-10T11:29:33.192 INFO:tasks.workunit.client.1.smithi178.stdout:2/818: truncate d1/d41/d8d/d9f/f110 576975 0 2024-09-10T11:29:33.192 INFO:tasks.workunit.client.1.smithi178.stdout:7/568: symlink d6/d10/d1d/d22/d93/db7/ld1 0 2024-09-10T11:29:33.192 INFO:tasks.workunit.client.1.smithi178.stdout:0/617: truncate d0/dc/d39/d4c/f32 5088838 0 2024-09-10T11:29:33.194 INFO:tasks.workunit.client.1.smithi178.stdout:4/527: dread d0/d8/f56 [0,4194304] 0 2024-09-10T11:29:33.194 INFO:tasks.workunit.client.1.smithi178.stdout:7/569: symlink d6/d10/d1d/d4a/ld2 0 2024-09-10T11:29:33.196 INFO:tasks.workunit.client.1.smithi178.stdout:4/528: read - d0/d9/d3d/d4f/faf zero size 2024-09-10T11:29:33.198 INFO:tasks.workunit.client.1.smithi178.stdout:7/570: symlink d6/d10/d1e/ld3 0 2024-09-10T11:29:33.198 INFO:tasks.workunit.client.1.smithi178.stdout:0/618: dread d0/dc/d39/d4c/f57 [4194304,4194304] 0 2024-09-10T11:29:33.203 INFO:tasks.workunit.client.1.smithi178.stdout:7/571: creat d6/d10/d1d/dca/fd4 x:0 0 0 2024-09-10T11:29:33.205 INFO:tasks.workunit.client.1.smithi178.stdout:2/819: write d1/d7/d24/d7d/d84/d4b/d89/fa3 [735521,89649] 0 2024-09-10T11:29:33.206 INFO:tasks.workunit.client.1.smithi178.stdout:2/820: chown d1/d41/fdf 520290135 1 2024-09-10T11:29:33.209 INFO:tasks.workunit.client.1.smithi178.stdout:7/572: truncate d6/d10/d1e/d2d/d2b/da0/f7a 3798000 0 2024-09-10T11:29:33.209 INFO:tasks.workunit.client.1.smithi178.stdout:4/529: truncate d0/f38 4206320 0 2024-09-10T11:29:33.210 INFO:tasks.workunit.client.1.smithi178.stdout:0/619: dread d0/dc/d39/d50/f1d [0,4194304] 0 2024-09-10T11:29:33.211 INFO:tasks.workunit.client.1.smithi178.stdout:4/530: symlink d0/d9/dc/d8e/lbe 0 2024-09-10T11:29:33.213 INFO:tasks.workunit.client.1.smithi178.stdout:0/620: chown d0/dc/d39/d50/cb8 18 1 2024-09-10T11:29:33.216 INFO:tasks.workunit.client.1.smithi178.stdout:2/821: truncate d1/d7/d24/d7d/d84/d55/f73 503977 0 2024-09-10T11:29:33.221 INFO:tasks.workunit.client.1.smithi178.stdout:4/531: dwrite d0/d8/da/d1a/f8b [0,4194304] 0 2024-09-10T11:29:33.222 INFO:tasks.workunit.client.1.smithi178.stdout:2/822: truncate d1/d41/d14/d11/db9/def/ff4 333785 0 2024-09-10T11:29:33.224 INFO:tasks.workunit.client.1.smithi178.stdout:2/823: dread - d1/d7/d8e/fc0 zero size 2024-09-10T11:29:33.224 INFO:tasks.workunit.client.1.smithi178.stdout:7/573: dread d6/d10/d1e/d2d/d2b/da0/d55/f91 [0,4194304] 0 2024-09-10T11:29:33.225 INFO:tasks.workunit.client.1.smithi178.stdout:4/532: chown d0/d8/da/f3f 12887 1 2024-09-10T11:29:33.226 INFO:tasks.workunit.client.1.smithi178.stdout:4/533: dread - d0/d8/da/f90 zero size 2024-09-10T11:29:33.227 INFO:tasks.workunit.client.1.smithi178.stdout:7/574: mkdir d6/d10/d1d/d22/d50/d98/da6/dd5 0 2024-09-10T11:29:33.229 INFO:tasks.workunit.client.1.smithi178.stdout:4/534: truncate d0/f5 3448281 0 2024-09-10T11:29:33.230 INFO:tasks.workunit.client.1.smithi178.stdout:0/621: dread d0/dc/d39/d50/f25 [0,4194304] 0 2024-09-10T11:29:33.232 INFO:tasks.workunit.client.1.smithi178.stdout:7/575: creat d6/d10/d1d/fd6 x:0 0 0 2024-09-10T11:29:33.234 INFO:tasks.workunit.client.0.smithi144.stdout:1/318: creat d5/dd/f5e x:0 0 0 2024-09-10T11:29:33.235 INFO:tasks.workunit.client.0.smithi144.stdout:8/144: dwrite d0/d9/d21/d25/f29 [0,4194304] 0 2024-09-10T11:29:33.237 INFO:tasks.workunit.client.1.smithi178.stdout:4/535: dwrite d0/d63/f78 [0,4194304] 0 2024-09-10T11:29:33.238 INFO:tasks.workunit.client.0.smithi144.stdout:8/145: chown d0/d9/d21/c22 289 1 2024-09-10T11:29:33.238 INFO:tasks.workunit.client.1.smithi178.stdout:4/536: fsync d0/d9/d3d/d4f/f52 0 2024-09-10T11:29:33.239 INFO:tasks.workunit.client.0.smithi144.stdout:1/319: symlink d5/df/d4b/d51/l5f 0 2024-09-10T11:29:33.241 INFO:tasks.workunit.client.0.smithi144.stdout:1/320: stat d5/d2a/c49 0 2024-09-10T11:29:33.244 INFO:tasks.workunit.client.0.smithi144.stdout:5/265: dwrite d4/d20/d24/d2d/d4b/f4e [4194304,4194304] 0 2024-09-10T11:29:33.244 INFO:tasks.workunit.client.0.smithi144.stdout:8/146: symlink d0/d9/d23/d26/l32 0 2024-09-10T11:29:33.245 INFO:tasks.workunit.client.1.smithi178.stdout:0/622: write d0/dc/d1f/d2d/f42 [5050687,123979] 0 2024-09-10T11:29:33.245 INFO:tasks.workunit.client.0.smithi144.stdout:3/260: dwrite d4/dd/f1b [0,4194304] 0 2024-09-10T11:29:33.246 INFO:tasks.workunit.client.0.smithi144.stdout:1/321: chown d5/dd/f5e 0 1 2024-09-10T11:29:33.246 INFO:tasks.workunit.client.1.smithi178.stdout:7/576: dread d6/d10/d1d/d22/d50/d7e/d82/db2/f36 [0,4194304] 0 2024-09-10T11:29:33.247 INFO:tasks.workunit.client.0.smithi144.stdout:5/266: stat d4/d5/c9 0 2024-09-10T11:29:33.251 INFO:tasks.workunit.client.0.smithi144.stdout:8/147: truncate d0/d1b/f2f 539346 0 2024-09-10T11:29:33.251 INFO:tasks.workunit.client.1.smithi178.stdout:4/537: dwrite d0/d63/f78 [0,4194304] 0 2024-09-10T11:29:33.252 INFO:tasks.workunit.client.0.smithi144.stdout:3/261: rename d4/dd/d21 to d4/dd/d21/d2f/d5a 22 2024-09-10T11:29:33.252 INFO:tasks.workunit.client.0.smithi144.stdout:1/322: symlink d5/df/d4b/d51/l60 0 2024-09-10T11:29:33.252 INFO:tasks.workunit.client.1.smithi178.stdout:4/538: fsync d0/d8/da/f6b 0 2024-09-10T11:29:33.253 INFO:tasks.workunit.client.1.smithi178.stdout:4/539: fdatasync d0/d9/d3d/f61 0 2024-09-10T11:29:33.255 INFO:tasks.workunit.client.0.smithi144.stdout:3/262: creat d4/dd/d21/f5b x:0 0 0 2024-09-10T11:29:33.255 INFO:tasks.workunit.client.1.smithi178.stdout:7/577: symlink d6/d10/d1d/d22/d3b/ld7 0 2024-09-10T11:29:33.256 INFO:tasks.workunit.client.0.smithi144.stdout:3/263: chown d4/dd/d22/d17/d18/d34 9 1 2024-09-10T11:29:33.256 INFO:tasks.workunit.client.0.smithi144.stdout:1/323: mknod d5/df/d4b/c61 0 2024-09-10T11:29:33.257 INFO:tasks.workunit.client.0.smithi144.stdout:3/264: write d4/d1a/f3b [914099,106119] 0 2024-09-10T11:29:33.258 INFO:tasks.workunit.client.0.smithi144.stdout:8/148: symlink d0/d9/l33 0 2024-09-10T11:29:33.259 INFO:tasks.workunit.client.0.smithi144.stdout:3/265: dread - d4/d28/f3f zero size 2024-09-10T11:29:33.260 INFO:tasks.workunit.client.1.smithi178.stdout:2/824: dread d1/d22/f30 [0,4194304] 0 2024-09-10T11:29:33.261 INFO:tasks.workunit.client.1.smithi178.stdout:4/540: rename d0/d21 to d0/d9/dc/d8e/dbf 0 2024-09-10T11:29:33.262 INFO:tasks.workunit.client.0.smithi144.stdout:8/149: rmdir d0/d9/d21 39 2024-09-10T11:29:33.263 INFO:tasks.workunit.client.1.smithi178.stdout:7/578: creat d6/dbf/fd8 x:0 0 0 2024-09-10T11:29:33.265 INFO:tasks.workunit.client.0.smithi144.stdout:3/266: creat d4/dd/d22/d17/d18/d34/f5c x:0 0 0 2024-09-10T11:29:33.266 INFO:tasks.workunit.client.0.smithi144.stdout:8/150: dread - d0/d9/d23/d26/f30 zero size 2024-09-10T11:29:33.268 INFO:tasks.workunit.client.0.smithi144.stdout:1/324: dwrite d5/df/d4b/d51/f59 [0,4194304] 0 2024-09-10T11:29:33.269 INFO:tasks.workunit.client.1.smithi178.stdout:4/541: mkdir d0/d8/dc0 0 2024-09-10T11:29:33.271 INFO:tasks.workunit.client.1.smithi178.stdout:7/579: creat d6/d10/d1d/d22/d3b/fd9 x:0 0 0 2024-09-10T11:29:33.273 INFO:tasks.workunit.client.0.smithi144.stdout:1/325: mkdir d5/df/d4b/d62 0 2024-09-10T11:29:33.274 INFO:tasks.workunit.client.1.smithi178.stdout:2/825: dwrite d1/d7/d24/d46/f63 [0,4194304] 0 2024-09-10T11:29:33.274 INFO:tasks.workunit.client.0.smithi144.stdout:1/326: dread - d5/dd/f5e zero size 2024-09-10T11:29:33.274 INFO:tasks.workunit.client.0.smithi144.stdout:1/327: readlink d5/d4a/l26 0 2024-09-10T11:29:33.277 INFO:tasks.workunit.client.0.smithi144.stdout:1/328: mkdir d5/d4a/d63 0 2024-09-10T11:29:33.279 INFO:tasks.workunit.client.0.smithi144.stdout:1/329: write d5/dd/f5e [97323,63104] 0 2024-09-10T11:29:33.280 INFO:tasks.workunit.client.1.smithi178.stdout:2/826: write d1/d7/d24/d46/d88/d9d/dab/d5e/db3/ffd [917830,127637] 0 2024-09-10T11:29:33.284 INFO:tasks.workunit.client.0.smithi144.stdout:8/151: truncate d0/d9/fe 2407196 0 2024-09-10T11:29:33.286 INFO:tasks.workunit.client.1.smithi178.stdout:4/542: dwrite d0/d8/da/f6b [0,4194304] 0 2024-09-10T11:29:33.289 INFO:tasks.workunit.client.0.smithi144.stdout:5/267: dread d4/d5/d32/f33 [0,4194304] 0 2024-09-10T11:29:33.290 INFO:tasks.workunit.client.0.smithi144.stdout:8/152: dread d0/f24 [4194304,4194304] 0 2024-09-10T11:29:33.291 INFO:tasks.workunit.client.0.smithi144.stdout:8/153: chown d0/fc 30122 1 2024-09-10T11:29:33.291 INFO:tasks.workunit.client.1.smithi178.stdout:2/827: creat d1/d7/d24/d46/d88/d9d/dab/d5b/d60/dd7/d108/f111 x:0 0 0 2024-09-10T11:29:33.292 INFO:tasks.workunit.client.0.smithi144.stdout:5/268: rmdir d4/d20 39 2024-09-10T11:29:33.294 INFO:tasks.workunit.client.0.smithi144.stdout:8/154: mkdir d0/d9/d1f/d34 0 2024-09-10T11:29:33.295 INFO:tasks.workunit.client.0.smithi144.stdout:5/269: write d4/d20/d24/f46 [3054202,83851] 0 2024-09-10T11:29:33.296 INFO:tasks.workunit.client.1.smithi178.stdout:4/543: creat d0/d9/fc1 x:0 0 0 2024-09-10T11:29:33.297 INFO:tasks.workunit.client.1.smithi178.stdout:4/544: dread - d0/d9/dc/d8e/dbf/fb5 zero size 2024-09-10T11:29:33.299 INFO:tasks.workunit.client.1.smithi178.stdout:7/580: dread d6/d10/d1e/d2d/d2b/f39 [0,4194304] 0 2024-09-10T11:29:33.299 INFO:tasks.workunit.client.1.smithi178.stdout:2/828: link d1/d41/d5d/c70 d1/d7/d8e/d10b/c112 0 2024-09-10T11:29:33.300 INFO:tasks.workunit.client.1.smithi178.stdout:4/545: write d0/d10/fa2 [458065,119872] 0 2024-09-10T11:29:33.300 INFO:tasks.workunit.client.0.smithi144.stdout:8/155: rename d0/d9/c15 to d0/d9/d1f/c35 0 2024-09-10T11:29:33.301 INFO:tasks.workunit.client.1.smithi178.stdout:7/581: write d6/d10/d1d/d22/d93/f99 [439725,118851] 0 2024-09-10T11:29:33.301 INFO:tasks.workunit.client.1.smithi178.stdout:4/546: readlink d0/d8/da/dae/lb9 0 2024-09-10T11:29:33.303 INFO:tasks.workunit.client.1.smithi178.stdout:4/547: fdatasync d0/d9/dc/d8e/dbf/fb5 0 2024-09-10T11:29:33.303 INFO:tasks.workunit.client.0.smithi144.stdout:5/270: dwrite d4/f55 [0,4194304] 0 2024-09-10T11:29:33.304 INFO:tasks.workunit.client.1.smithi178.stdout:7/582: chown d6/d10/d1e/d2d/d2b/l2e 30304779 1 2024-09-10T11:29:33.306 INFO:tasks.workunit.client.0.smithi144.stdout:5/271: rmdir d4/d5/d32/d47 39 2024-09-10T11:29:33.306 INFO:tasks.workunit.client.1.smithi178.stdout:2/829: write d1/d7/d24/d7d/d84/f35 [2608605,27772] 0 2024-09-10T11:29:33.310 INFO:tasks.workunit.client.0.smithi144.stdout:3/267: dread d4/dd/d22/d17/f1f [0,4194304] 0 2024-09-10T11:29:33.311 INFO:tasks.workunit.client.0.smithi144.stdout:8/156: truncate d0/f24 2184228 0 2024-09-10T11:29:33.312 INFO:tasks.workunit.client.1.smithi178.stdout:2/830: creat d1/d7/d24/d46/d88/d9d/dab/d5e/f113 x:0 0 0 2024-09-10T11:29:33.313 INFO:tasks.workunit.client.0.smithi144.stdout:5/272: getdents d4/d5/d3f/d44 0 2024-09-10T11:29:33.313 INFO:tasks.workunit.client.1.smithi178.stdout:7/583: dwrite d6/d10/d1d/d22/d3b/fa5 [0,4194304] 0 2024-09-10T11:29:33.316 INFO:tasks.workunit.client.0.smithi144.stdout:5/273: mkdir d4/d5/d3f/d59 0 2024-09-10T11:29:33.316 INFO:tasks.workunit.client.1.smithi178.stdout:7/584: creat d6/d10/d1e/d2d/d2b/fda x:0 0 0 2024-09-10T11:29:33.317 INFO:tasks.workunit.client.1.smithi178.stdout:2/831: creat d1/d7/d24/d7d/d84/d55/da1/f114 x:0 0 0 2024-09-10T11:29:33.319 INFO:tasks.workunit.client.0.smithi144.stdout:5/274: unlink d4/d5/d32/d47/f50 0 2024-09-10T11:29:33.320 INFO:tasks.workunit.client.1.smithi178.stdout:7/585: rmdir d6/d10/d1e/d2d/d2b/da0 39 2024-09-10T11:29:33.320 INFO:tasks.workunit.client.0.smithi144.stdout:3/268: link d4/dd/d22/d17/d18/f48 d4/dd/d21/f5d 0 2024-09-10T11:29:33.321 INFO:tasks.workunit.client.1.smithi178.stdout:2/832: symlink d1/d41/d14/d11/d53/l115 0 2024-09-10T11:29:33.321 INFO:tasks.workunit.client.0.smithi144.stdout:8/157: link d0/d9/d1f/f1c d0/d9/d23/d26/f36 0 2024-09-10T11:29:33.321 INFO:tasks.workunit.client.0.smithi144.stdout:8/158: dread - d0/d9/f1d zero size 2024-09-10T11:29:33.322 INFO:tasks.workunit.client.0.smithi144.stdout:8/159: chown d0/d9/f1d 156 1 2024-09-10T11:29:33.323 INFO:tasks.workunit.client.1.smithi178.stdout:7/586: write d6/d10/d1e/d2d/d2b/da0/d55/dbd/f77 [5220773,20356] 0 2024-09-10T11:29:33.323 INFO:tasks.workunit.client.0.smithi144.stdout:3/269: symlink d4/dd/d22/d17/d18/d34/l5e 0 2024-09-10T11:29:33.324 INFO:tasks.workunit.client.1.smithi178.stdout:2/833: unlink d1/d7/l8a 0 2024-09-10T11:29:33.325 INFO:tasks.workunit.client.0.smithi144.stdout:3/270: dread d4/d1a/f44 [0,4194304] 0 2024-09-10T11:29:33.325 INFO:tasks.workunit.client.1.smithi178.stdout:7/587: write d6/d10/d1d/daa/fd0 [85574,58971] 0 2024-09-10T11:29:33.328 INFO:tasks.workunit.client.0.smithi144.stdout:3/271: truncate d4/dd/d22/d17/d18/d34/f5c 74483 0 2024-09-10T11:29:33.328 INFO:tasks.workunit.client.1.smithi178.stdout:7/588: write d6/d10/d1d/daa/fd0 [375922,84116] 0 2024-09-10T11:29:33.329 INFO:tasks.workunit.client.1.smithi178.stdout:2/834: unlink d1/d41/d14/d11/c4c 0 2024-09-10T11:29:33.331 INFO:tasks.workunit.client.0.smithi144.stdout:3/272: creat d4/d1a/f5f x:0 0 0 2024-09-10T11:29:33.332 INFO:tasks.workunit.client.0.smithi144.stdout:5/275: write d4/fa [1499970,79674] 0 2024-09-10T11:29:33.333 INFO:tasks.workunit.client.1.smithi178.stdout:7/589: mknod d6/d10/d1d/cdb 0 2024-09-10T11:29:33.333 INFO:tasks.workunit.client.0.smithi144.stdout:8/160: write d0/f24 [2212438,113080] 0 2024-09-10T11:29:33.340 INFO:tasks.workunit.client.0.smithi144.stdout:8/161: mkdir d0/d9/d23/d37 0 2024-09-10T11:29:33.341 INFO:tasks.workunit.client.1.smithi178.stdout:2/835: getdents d1/d7/d24/d46/d88/d9d/dab/d5b/d7c 0 2024-09-10T11:29:33.341 INFO:tasks.workunit.client.1.smithi178.stdout:7/590: dwrite d6/d10/d1e/d2d/d2b/f41 [4194304,4194304] 0 2024-09-10T11:29:33.341 INFO:tasks.workunit.client.0.smithi144.stdout:5/276: rename d4/d5/d32/l40 to d4/d5/l5a 0 2024-09-10T11:29:33.344 INFO:tasks.workunit.client.1.smithi178.stdout:2/836: rename d1/d7/d24/d46/d88/d9d/dab/d5e/f96 to d1/d41/d5d/f116 0 2024-09-10T11:29:33.344 INFO:tasks.workunit.client.0.smithi144.stdout:8/162: dwrite d0/d9/f1d [0,4194304] 0 2024-09-10T11:29:33.344 INFO:tasks.workunit.client.1.smithi178.stdout:2/837: readlink d1/d7/d24/d7d/d84/d55/l90 0 2024-09-10T11:29:33.345 INFO:tasks.workunit.client.1.smithi178.stdout:7/591: mkdir d6/d10/d1d/d22/d50/d98/ddc 0 2024-09-10T11:29:33.346 INFO:tasks.workunit.client.1.smithi178.stdout:7/592: dread - d6/d10/d1e/d2d/d2b/fc1 zero size 2024-09-10T11:29:33.347 INFO:tasks.workunit.client.0.smithi144.stdout:5/277: link d4/d5/l48 d4/d5/d18/d38/l5b 0 2024-09-10T11:29:33.347 INFO:tasks.workunit.client.0.smithi144.stdout:8/163: symlink d0/d9/d1f/d34/l38 0 2024-09-10T11:29:33.348 INFO:tasks.workunit.client.1.smithi178.stdout:7/593: write d6/dbf/fcc [385873,48390] 0 2024-09-10T11:29:33.348 INFO:tasks.workunit.client.1.smithi178.stdout:7/594: readlink d6/d10/lb5 0 2024-09-10T11:29:33.350 INFO:tasks.workunit.client.1.smithi178.stdout:7/595: fdatasync d6/dbf/fcc 0 2024-09-10T11:29:33.351 INFO:tasks.workunit.client.1.smithi178.stdout:2/838: rename d1/d7/d24/d46/d88/d9d/dab/d5b/d7c/cec to d1/d7/d24/c117 0 2024-09-10T11:29:33.351 INFO:tasks.workunit.client.0.smithi144.stdout:5/278: unlink d4/l10 0 2024-09-10T11:29:33.352 INFO:tasks.workunit.client.0.smithi144.stdout:8/164: fsync d0/d9/fe 0 2024-09-10T11:29:33.352 INFO:tasks.workunit.client.1.smithi178.stdout:7/596: mkdir d6/dbf/ddd 0 2024-09-10T11:29:33.353 INFO:tasks.workunit.client.1.smithi178.stdout:2/839: mkdir d1/d7/d24/d7d/d84/d4b/d118 0 2024-09-10T11:29:33.355 INFO:tasks.workunit.client.1.smithi178.stdout:7/597: symlink d6/d10/d34/d6a/lde 0 2024-09-10T11:29:33.356 INFO:tasks.workunit.client.0.smithi144.stdout:0/160: sync 2024-09-10T11:29:33.356 INFO:tasks.workunit.client.0.smithi144.stdout:2/187: sync 2024-09-10T11:29:33.358 INFO:tasks.workunit.client.1.smithi178.stdout:7/598: symlink d6/d10/d1e/ldf 0 2024-09-10T11:29:33.358 INFO:tasks.workunit.client.0.smithi144.stdout:5/279: dwrite d4/d5/f28 [0,4194304] 0 2024-09-10T11:29:33.358 INFO:tasks.workunit.client.0.smithi144.stdout:0/161: read da/f16 [3554828,111517] 0 2024-09-10T11:29:33.359 INFO:tasks.workunit.client.1.smithi178.stdout:2/840: dread d1/d41/d14/d11/d53/f8b [0,4194304] 0 2024-09-10T11:29:33.361 INFO:tasks.workunit.client.1.smithi178.stdout:7/599: mknod d6/ce0 0 2024-09-10T11:29:33.361 INFO:tasks.workunit.client.0.smithi144.stdout:0/162: creat da/f26 x:0 0 0 2024-09-10T11:29:33.363 INFO:tasks.workunit.client.1.smithi178.stdout:2/841: write d1/d22/d4d/d9b/f75 [1680166,49321] 0 2024-09-10T11:29:33.364 INFO:tasks.workunit.client.0.smithi144.stdout:0/163: fdatasync da/f16 0 2024-09-10T11:29:33.365 INFO:tasks.workunit.client.1.smithi178.stdout:7/600: read d6/d10/d1d/d22/d93/fa8 [2387044,86987] 0 2024-09-10T11:29:33.366 INFO:tasks.workunit.client.1.smithi178.stdout:7/601: creat d6/d10/d1d/d22/d93/fe1 x:0 0 0 2024-09-10T11:29:33.366 INFO:tasks.workunit.client.0.smithi144.stdout:0/164: truncate da/f15 3518393 0 2024-09-10T11:29:33.367 INFO:tasks.workunit.client.0.smithi144.stdout:0/165: rename da to da/d27 22 2024-09-10T11:29:33.367 INFO:tasks.workunit.client.1.smithi178.stdout:7/602: fsync d6/d10/d1d/f73 0 2024-09-10T11:29:33.367 INFO:tasks.workunit.client.0.smithi144.stdout:0/166: chown da/l25 1133 1 2024-09-10T11:29:33.373 INFO:tasks.workunit.client.0.smithi144.stdout:0/167: dwrite da/f16 [4194304,4194304] 0 2024-09-10T11:29:33.374 INFO:tasks.workunit.client.0.smithi144.stdout:0/168: creat da/f28 x:0 0 0 2024-09-10T11:29:33.379 INFO:tasks.workunit.client.1.smithi178.stdout:2/842: dread d1/d22/fae [0,4194304] 0 2024-09-10T11:29:33.381 INFO:tasks.workunit.client.1.smithi178.stdout:2/843: mknod d1/d7/d24/d7d/d84/d55/da1/c119 0 2024-09-10T11:29:33.383 INFO:tasks.workunit.client.1.smithi178.stdout:2/844: rename d1/d7/d24/d46/d88/d9d/dab/c28 to d1/d41/d8d/c11a 0 2024-09-10T11:29:33.386 INFO:tasks.workunit.client.1.smithi178.stdout:2/845: dread d1/d7/d24/f42 [0,4194304] 0 2024-09-10T11:29:33.391 INFO:tasks.workunit.client.1.smithi178.stdout:6/638: sync 2024-09-10T11:29:33.391 INFO:tasks.workunit.client.1.smithi178.stdout:5/656: sync 2024-09-10T11:29:33.391 INFO:tasks.workunit.client.1.smithi178.stdout:3/569: sync 2024-09-10T11:29:33.393 INFO:tasks.workunit.client.1.smithi178.stdout:3/570: symlink db/d35/lb3 0 2024-09-10T11:29:33.396 INFO:tasks.workunit.client.1.smithi178.stdout:6/639: mkdir d3/d74/d91/daa/dc2/d57/dea 0 2024-09-10T11:29:33.401 INFO:tasks.workunit.client.1.smithi178.stdout:6/640: fdatasync d3/d7/d8/d15/d1c/d2a/f2c 0 2024-09-10T11:29:33.403 INFO:tasks.workunit.client.1.smithi178.stdout:6/641: write d3/d7/d8/d15/f22 [730520,24217] 0 2024-09-10T11:29:33.405 INFO:tasks.workunit.client.0.smithi144.stdout:3/273: truncate d4/dd/d22/d17/d18/f2e 1146048 0 2024-09-10T11:29:33.410 INFO:tasks.workunit.client.0.smithi144.stdout:3/274: read d4/dd/d21/f5d [100171,122105] 0 2024-09-10T11:29:33.411 INFO:tasks.workunit.client.0.smithi144.stdout:3/275: truncate d4/d28/f3f 964447 0 2024-09-10T11:29:33.412 INFO:tasks.workunit.client.1.smithi178.stdout:7/603: getdents d6/d10/d1d/d22/d93 0 2024-09-10T11:29:33.412 INFO:tasks.workunit.client.0.smithi144.stdout:7/207: sync 2024-09-10T11:29:33.412 INFO:tasks.workunit.client.0.smithi144.stdout:9/272: sync 2024-09-10T11:29:33.413 INFO:tasks.workunit.client.0.smithi144.stdout:3/276: creat d4/dd/d21/d2f/f60 x:0 0 0 2024-09-10T11:29:33.413 INFO:tasks.workunit.client.0.smithi144.stdout:8/165: truncate d0/d9/d21/f2b 172884 0 2024-09-10T11:29:33.414 INFO:tasks.workunit.client.1.smithi178.stdout:7/604: fdatasync d6/d10/d1e/d2d/d2b/da0/d55/f91 0 2024-09-10T11:29:33.415 INFO:tasks.workunit.client.0.smithi144.stdout:2/188: write d7/f10 [1530909,128267] 0 2024-09-10T11:29:33.416 INFO:tasks.workunit.client.1.smithi178.stdout:6/642: fdatasync d3/d7/d8/d15/f22 0 2024-09-10T11:29:33.417 INFO:tasks.workunit.client.0.smithi144.stdout:9/273: mknod d11/d12/d1b/d22/d53/c60 0 2024-09-10T11:29:33.417 INFO:tasks.workunit.client.0.smithi144.stdout:9/274: fsync f10 0 2024-09-10T11:29:33.417 INFO:tasks.workunit.client.1.smithi178.stdout:2/846: truncate d1/d22/d4d/d9b/f75 711625 0 2024-09-10T11:29:33.418 INFO:tasks.workunit.client.0.smithi144.stdout:3/277: chown d4/dd/d22/d17/d18/d34/c4a 2332015 1 2024-09-10T11:29:33.418 INFO:tasks.workunit.client.0.smithi144.stdout:9/275: chown d11/d12/d1b/d22 143264 1 2024-09-10T11:29:33.419 INFO:tasks.workunit.client.0.smithi144.stdout:8/166: mknod d0/d9/c39 0 2024-09-10T11:29:33.419 INFO:tasks.workunit.client.1.smithi178.stdout:7/605: rename d6/d10/d1e/d2d/d2b/l54 to d6/d10/d1e/d2d/d2b/da0/d55/le2 0 2024-09-10T11:29:33.420 INFO:tasks.workunit.client.0.smithi144.stdout:2/189: dread d7/f1d [0,4194304] 0 2024-09-10T11:29:33.420 INFO:tasks.workunit.client.0.smithi144.stdout:9/276: fdatasync d11/d12/f14 0 2024-09-10T11:29:33.420 INFO:tasks.workunit.client.0.smithi144.stdout:5/280: truncate d4/d5/f28 1731896 0 2024-09-10T11:29:33.421 INFO:tasks.workunit.client.1.smithi178.stdout:5/657: dread d9/dc/dd/d37/d42/d55/f9d [0,4194304] 0 2024-09-10T11:29:33.421 INFO:tasks.workunit.client.1.smithi178.stdout:7/606: stat d6/d10/d1e/d2d/d2b/da0/d55/dbd 0 2024-09-10T11:29:33.422 INFO:tasks.workunit.client.0.smithi144.stdout:8/167: chown d0/d9/c2c 7631893 1 2024-09-10T11:29:33.423 INFO:tasks.workunit.client.1.smithi178.stdout:2/847: rmdir d1/d41/d14/d11 39 2024-09-10T11:29:33.424 INFO:tasks.workunit.client.0.smithi144.stdout:8/168: write d0/d9/d21/d25/f29 [2452209,17913] 0 2024-09-10T11:29:33.426 INFO:tasks.workunit.client.0.smithi144.stdout:0/169: write da/f19 [585879,107390] 0 2024-09-10T11:29:33.427 INFO:tasks.workunit.client.1.smithi178.stdout:7/607: unlink d6/d10/d1d/d22/d50/d7e/d82/db2/ca3 0 2024-09-10T11:29:33.430 INFO:tasks.workunit.client.0.smithi144.stdout:0/170: write da/f26 [738348,123321] 0 2024-09-10T11:29:33.433 INFO:tasks.workunit.client.1.smithi178.stdout:2/848: symlink d1/d7/d24/d46/d88/d9d/dab/d5b/l11b 0 2024-09-10T11:29:33.433 INFO:tasks.workunit.client.0.smithi144.stdout:5/281: dwrite d4/f55 [0,4194304] 0 2024-09-10T11:29:33.434 INFO:tasks.workunit.client.1.smithi178.stdout:7/608: creat d6/d10/d1d/d22/d50/fe3 x:0 0 0 2024-09-10T11:29:33.435 INFO:tasks.workunit.client.1.smithi178.stdout:2/849: write d1/d7/d24/d7d/d84/d4b/d89/ff0 [890824,13360] 0 2024-09-10T11:29:33.437 INFO:tasks.workunit.client.1.smithi178.stdout:7/609: write d6/d10/d1d/d22/d50/d7e/d82/f92 [1276007,39828] 0 2024-09-10T11:29:33.438 INFO:tasks.workunit.client.1.smithi178.stdout:2/850: dread - d1/d7/d24/d46/d88/d9d/dab/d5e/fdc zero size 2024-09-10T11:29:33.440 INFO:tasks.workunit.client.1.smithi178.stdout:2/851: write d1/d7/d24/d46/d88/d9d/dab/f15 [2405081,99760] 0 2024-09-10T11:29:33.442 INFO:tasks.workunit.client.0.smithi144.stdout:8/169: dread d0/f13 [0,4194304] 0 2024-09-10T11:29:33.444 INFO:tasks.workunit.client.0.smithi144.stdout:7/208: link d5/dc/ld d5/dc/d12/d2f/l46 0 2024-09-10T11:29:33.444 INFO:tasks.workunit.client.1.smithi178.stdout:7/610: rename d6/d10/d1d/d4a/ld2 to d6/d10/d1d/dca/le4 0 2024-09-10T11:29:33.445 INFO:tasks.workunit.client.0.smithi144.stdout:3/278: dread d4/dd/d22/d17/f30 [0,4194304] 0 2024-09-10T11:29:33.445 INFO:tasks.workunit.client.0.smithi144.stdout:7/209: stat d5/dc/d12/d2f/c31 0 2024-09-10T11:29:33.447 INFO:tasks.workunit.client.0.smithi144.stdout:5/282: mkdir d4/d20/d22/d5c 0 2024-09-10T11:29:33.450 INFO:tasks.workunit.client.1.smithi178.stdout:2/852: mkdir d1/d7/d24/d11c 0 2024-09-10T11:29:33.451 INFO:tasks.workunit.client.1.smithi178.stdout:7/611: dwrite d6/d10/d1d/d22/d93/f9e [0,4194304] 0 2024-09-10T11:29:33.452 INFO:tasks.workunit.client.0.smithi144.stdout:3/279: mknod d4/dd/d21/d2f/c61 0 2024-09-10T11:29:33.456 INFO:tasks.workunit.client.0.smithi144.stdout:7/210: rmdir d5/dc/d11 39 2024-09-10T11:29:33.456 INFO:tasks.workunit.client.1.smithi178.stdout:5/658: dread d9/dc/dd/d37/dbd/dda/db8/fd4 [0,4194304] 0 2024-09-10T11:29:33.457 INFO:tasks.workunit.client.1.smithi178.stdout:7/612: unlink d6/d10/d1d/d22/d50/fc7 0 2024-09-10T11:29:33.458 INFO:tasks.workunit.client.0.smithi144.stdout:2/190: dwrite d7/f26 [0,4194304] 0 2024-09-10T11:29:33.459 INFO:tasks.workunit.client.0.smithi144.stdout:8/170: creat d0/d9/d23/d37/f3a x:0 0 0 2024-09-10T11:29:33.460 INFO:tasks.workunit.client.0.smithi144.stdout:3/280: creat d4/dd/d21/f62 x:0 0 0 2024-09-10T11:29:33.462 INFO:tasks.workunit.client.1.smithi178.stdout:5/659: symlink d9/dc/dd/d37/d42/d9f/ldc 0 2024-09-10T11:29:33.465 INFO:tasks.workunit.client.0.smithi144.stdout:7/211: creat d5/dc/d12/d2f/f47 x:0 0 0 2024-09-10T11:29:33.466 INFO:tasks.workunit.client.0.smithi144.stdout:2/191: creat d7/f41 x:0 0 0 2024-09-10T11:29:33.466 INFO:tasks.workunit.client.1.smithi178.stdout:7/613: dwrite d6/d10/fb3 [0,4194304] 0 2024-09-10T11:29:33.467 INFO:tasks.workunit.client.0.smithi144.stdout:3/281: mkdir d4/dd/d22/d17/d18/d34/d4b/d63 0 2024-09-10T11:29:33.467 INFO:tasks.workunit.client.0.smithi144.stdout:0/171: dread f2 [0,4194304] 0 2024-09-10T11:29:33.470 INFO:tasks.workunit.client.1.smithi178.stdout:5/660: symlink d9/dc/dd/d37/d42/d55/d6d/dcc/ldd 0 2024-09-10T11:29:33.474 INFO:tasks.workunit.client.1.smithi178.stdout:5/661: mknod d9/d19/cde 0 2024-09-10T11:29:33.475 INFO:tasks.workunit.client.0.smithi144.stdout:8/171: dwrite d0/d1b/f31 [0,4194304] 0 2024-09-10T11:29:33.475 INFO:tasks.workunit.client.0.smithi144.stdout:9/277: dwrite d11/d12/d1b/f27 [0,4194304] 0 2024-09-10T11:29:33.477 INFO:tasks.workunit.client.0.smithi144.stdout:9/278: chown d11/d12/d1b/d41/l57 2 1 2024-09-10T11:29:33.478 INFO:tasks.workunit.client.1.smithi178.stdout:7/614: dwrite d6/d10/d1d/d22/d50/f59 [0,4194304] 0 2024-09-10T11:29:33.478 INFO:tasks.workunit.client.0.smithi144.stdout:4/212: sync 2024-09-10T11:29:33.478 INFO:tasks.workunit.client.0.smithi144.stdout:4/213: chown d14/l36 454766667 1 2024-09-10T11:29:33.480 INFO:tasks.workunit.client.0.smithi144.stdout:4/214: chown d14/d1e/d22/d2b/d35/l49 18993 1 2024-09-10T11:29:33.480 INFO:tasks.workunit.client.1.smithi178.stdout:7/615: chown d6/d10/d1d/f73 10491 1 2024-09-10T11:29:33.480 INFO:tasks.workunit.client.1.smithi178.stdout:2/853: dread d1/d41/d8d/fb7 [0,4194304] 0 2024-09-10T11:29:33.481 INFO:tasks.workunit.client.1.smithi178.stdout:5/662: symlink d9/dc/ldf 0 2024-09-10T11:29:33.481 INFO:tasks.workunit.client.0.smithi144.stdout:4/215: chown d14/d1e/d22/d2b/c3d 16018987 1 2024-09-10T11:29:33.482 INFO:tasks.workunit.client.0.smithi144.stdout:7/212: symlink d5/dc/d12/l48 0 2024-09-10T11:29:33.484 INFO:tasks.workunit.client.0.smithi144.stdout:7/213: readlink d5/dc/d12/l32 0 2024-09-10T11:29:33.484 INFO:tasks.workunit.client.1.smithi178.stdout:7/616: rmdir d6/d10/d1d/d22/d3b 39 2024-09-10T11:29:33.487 INFO:tasks.workunit.client.1.smithi178.stdout:2/854: symlink d1/d7/d24/d7d/d84/l11d 0 2024-09-10T11:29:33.488 INFO:tasks.workunit.client.1.smithi178.stdout:5/663: read - d9/dc/dd/d16/f41 zero size 2024-09-10T11:29:33.491 INFO:tasks.workunit.client.0.smithi144.stdout:4/216: mkdir d14/d1e/d22/d34/d3e/d40/d4b 0 2024-09-10T11:29:33.493 INFO:tasks.workunit.client.0.smithi144.stdout:5/283: dread d4/fa [0,4194304] 0 2024-09-10T11:29:33.495 INFO:tasks.workunit.client.0.smithi144.stdout:2/192: getdents d7/da/d1e 0 2024-09-10T11:29:33.495 INFO:tasks.workunit.client.1.smithi178.stdout:5/664: fdatasync d9/dc/dd/d37/d42/d62/f80 0 2024-09-10T11:29:33.496 INFO:tasks.workunit.client.0.smithi144.stdout:0/172: dwrite f5 [0,4194304] 0 2024-09-10T11:29:33.497 INFO:tasks.workunit.client.1.smithi178.stdout:7/617: link d6/d10/d1d/d22/l4b d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d7c/le5 0 2024-09-10T11:29:33.499 INFO:tasks.workunit.client.0.smithi144.stdout:8/172: creat d0/d9/f3b x:0 0 0 2024-09-10T11:29:33.499 INFO:tasks.workunit.client.1.smithi178.stdout:5/665: creat d9/dc/dd/d37/dbd/dda/db8/fe0 x:0 0 0 2024-09-10T11:29:33.499 INFO:tasks.workunit.client.0.smithi144.stdout:0/173: write da/f19 [4172142,60049] 0 2024-09-10T11:29:33.500 INFO:tasks.workunit.client.0.smithi144.stdout:0/174: rename da to da/d29 22 2024-09-10T11:29:33.501 INFO:tasks.workunit.client.0.smithi144.stdout:3/282: truncate d4/dd/d22/fc 4047163 0 2024-09-10T11:29:33.502 INFO:tasks.workunit.client.0.smithi144.stdout:4/217: truncate d14/d1e/d22/d2b/f2e 198078 0 2024-09-10T11:29:33.503 INFO:tasks.workunit.client.0.smithi144.stdout:2/193: mkdir d7/d42 0 2024-09-10T11:29:33.504 INFO:tasks.workunit.client.1.smithi178.stdout:5/666: creat d9/dc/dd/d37/d42/d55/d6d/fe1 x:0 0 0 2024-09-10T11:29:33.505 INFO:tasks.workunit.client.0.smithi144.stdout:9/279: truncate d11/f15 525297 0 2024-09-10T11:29:33.505 INFO:tasks.workunit.client.0.smithi144.stdout:5/284: dwrite d4/d5/d18/f1f [0,4194304] 0 2024-09-10T11:29:33.508 INFO:tasks.workunit.client.0.smithi144.stdout:0/175: creat da/f2a x:0 0 0 2024-09-10T11:29:33.509 INFO:tasks.workunit.client.1.smithi178.stdout:5/667: write d9/dc/dd/d37/d42/f4e [2927625,116163] 0 2024-09-10T11:29:33.509 INFO:tasks.workunit.client.0.smithi144.stdout:3/283: symlink d4/dd/d21/d2f/l64 0 2024-09-10T11:29:33.514 INFO:tasks.workunit.client.1.smithi178.stdout:5/668: mkdir d9/dc/dd/d37/d42/d55/d6d/dcc/de2 0 2024-09-10T11:29:33.514 INFO:tasks.workunit.client.0.smithi144.stdout:7/214: getdents d5/dc/d2a 0 2024-09-10T11:29:33.515 INFO:tasks.workunit.client.0.smithi144.stdout:0/176: creat da/f2b x:0 0 0 2024-09-10T11:29:33.515 INFO:tasks.workunit.client.0.smithi144.stdout:4/218: mkdir d14/d1e/d22/d34/d3e/d40/d4b/d4c 0 2024-09-10T11:29:33.516 INFO:tasks.workunit.client.1.smithi178.stdout:5/669: chown d9/dc/dd/d37/d42/d62 923 1 2024-09-10T11:29:33.516 INFO:tasks.workunit.client.0.smithi144.stdout:5/285: mkdir d4/d20/d22/d5c/d5d 0 2024-09-10T11:29:33.518 INFO:tasks.workunit.client.0.smithi144.stdout:7/215: fdatasync d5/dc/f3c 0 2024-09-10T11:29:33.519 INFO:tasks.workunit.client.1.smithi178.stdout:5/670: unlink d9/dc/d17/d1d/cac 0 2024-09-10T11:29:33.520 INFO:tasks.workunit.client.1.smithi178.stdout:2/855: dwrite d1/d41/f44 [0,4194304] 0 2024-09-10T11:29:33.525 INFO:tasks.workunit.client.0.smithi144.stdout:3/284: dwrite d4/dd/d22/d17/d18/d34/d4b/f53 [0,4194304] 0 2024-09-10T11:29:33.526 INFO:tasks.workunit.client.0.smithi144.stdout:5/286: chown d4/d5/l5a 7 1 2024-09-10T11:29:33.527 INFO:tasks.workunit.client.1.smithi178.stdout:2/856: fsync d1/d41/d14/d11/d53/f8b 0 2024-09-10T11:29:33.527 INFO:tasks.workunit.client.1.smithi178.stdout:5/671: dread d9/dc/dd/d37/d42/d62/f6f [0,4194304] 0 2024-09-10T11:29:33.529 INFO:tasks.workunit.client.0.smithi144.stdout:6/187: sync 2024-09-10T11:29:33.529 INFO:tasks.workunit.client.0.smithi144.stdout:1/330: sync 2024-09-10T11:29:33.531 INFO:tasks.workunit.client.0.smithi144.stdout:9/280: rename d11/f3c to d11/d12/d1b/f61 0 2024-09-10T11:29:33.534 INFO:tasks.workunit.client.1.smithi178.stdout:5/672: symlink d9/dc/dd/d37/dbd/dda/db8/le3 0 2024-09-10T11:29:33.535 INFO:tasks.workunit.client.1.smithi178.stdout:5/673: chown d9/d19/cde 252 1 2024-09-10T11:29:33.536 INFO:tasks.workunit.client.0.smithi144.stdout:2/194: write d7/da/f19 [136273,111898] 0 2024-09-10T11:29:33.538 INFO:tasks.workunit.client.0.smithi144.stdout:3/285: rmdir d4/dd/d22/d17/d18/d34/d4b/d55 39 2024-09-10T11:29:33.539 INFO:tasks.workunit.client.1.smithi178.stdout:5/674: dread d9/dc/dd/fc1 [0,4194304] 0 2024-09-10T11:29:33.542 INFO:tasks.workunit.client.0.smithi144.stdout:6/188: dwrite d9/dd/d11/f27 [0,4194304] 0 2024-09-10T11:29:33.544 INFO:tasks.workunit.client.0.smithi144.stdout:8/173: dwrite d0/d9/d23/d26/f36 [0,4194304] 0 2024-09-10T11:29:33.544 INFO:tasks.workunit.client.1.smithi178.stdout:5/675: mkdir d9/de4 0 2024-09-10T11:29:33.544 INFO:tasks.workunit.client.0.smithi144.stdout:6/189: fdatasync d9/dd/d11/f46 0 2024-09-10T11:29:33.545 INFO:tasks.workunit.client.1.smithi178.stdout:5/676: fsync d9/f51 0 2024-09-10T11:29:33.545 INFO:tasks.workunit.client.0.smithi144.stdout:7/216: rename d5/dc/fe to d5/dc/d2a/d41/f49 0 2024-09-10T11:29:33.546 INFO:tasks.workunit.client.1.smithi178.stdout:7/618: dread d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d70/f3c [0,4194304] 0 2024-09-10T11:29:33.548 INFO:tasks.workunit.client.0.smithi144.stdout:7/217: dread - d5/f37 zero size 2024-09-10T11:29:33.548 INFO:tasks.workunit.client.0.smithi144.stdout:0/177: dread da/f21 [0,4194304] 0 2024-09-10T11:29:33.548 INFO:tasks.workunit.client.0.smithi144.stdout:5/287: creat d4/d5/d3f/d59/f5e x:0 0 0 2024-09-10T11:29:33.551 INFO:tasks.workunit.client.0.smithi144.stdout:6/190: chown d9/d1f/f37 56473 1 2024-09-10T11:29:33.553 INFO:tasks.workunit.client.1.smithi178.stdout:5/677: unlink d9/d11/f14 0 2024-09-10T11:29:33.554 INFO:tasks.workunit.client.1.smithi178.stdout:5/678: stat d9/dc/dd/d37/d42/d55/d6d/fe1 0 2024-09-10T11:29:33.555 INFO:tasks.workunit.client.0.smithi144.stdout:6/191: dread - d9/dd/f33 zero size 2024-09-10T11:29:33.555 INFO:tasks.workunit.client.1.smithi178.stdout:1/553: sync 2024-09-10T11:29:33.556 INFO:tasks.workunit.client.0.smithi144.stdout:8/174: dread d0/d9/d23/d26/f36 [0,4194304] 0 2024-09-10T11:29:33.557 INFO:tasks.workunit.client.1.smithi178.stdout:7/619: mkdir d6/d10/d1d/de6 0 2024-09-10T11:29:33.557 INFO:tasks.workunit.client.0.smithi144.stdout:2/195: dwrite d7/de/f28 [0,4194304] 0 2024-09-10T11:29:33.558 INFO:tasks.workunit.client.0.smithi144.stdout:2/196: truncate d7/f41 22460 0 2024-09-10T11:29:33.559 INFO:tasks.workunit.client.1.smithi178.stdout:2/857: truncate d1/d7/d24/d7d/d84/d4b/d89/f93 1509849 0 2024-09-10T11:29:33.560 INFO:tasks.workunit.client.1.smithi178.stdout:5/679: read d9/dc/dd/d16/f70 [522533,67117] 0 2024-09-10T11:29:33.561 INFO:tasks.workunit.client.0.smithi144.stdout:0/178: mknod da/c2c 0 2024-09-10T11:29:33.561 INFO:tasks.workunit.client.0.smithi144.stdout:0/179: chown da/f26 548394 1 2024-09-10T11:29:33.564 INFO:tasks.workunit.client.1.smithi178.stdout:5/680: symlink d9/dc/dd/d3f/d5a/dbc/le5 0 2024-09-10T11:29:33.566 INFO:tasks.workunit.client.0.smithi144.stdout:0/180: dread da/f26 [0,4194304] 0 2024-09-10T11:29:33.567 INFO:tasks.workunit.client.0.smithi144.stdout:7/218: creat d5/dc/d12/d2f/f4a x:0 0 0 2024-09-10T11:29:33.567 INFO:tasks.workunit.client.0.smithi144.stdout:4/219: truncate d14/d1e/d22/d34/f3b 1097428 0 2024-09-10T11:29:33.570 INFO:tasks.workunit.client.0.smithi144.stdout:1/331: dwrite d5/df/f16 [0,4194304] 0 2024-09-10T11:29:33.572 INFO:tasks.workunit.client.1.smithi178.stdout:5/681: write d9/dc/dd/fad [3156533,19845] 0 2024-09-10T11:29:33.572 INFO:tasks.workunit.client.0.smithi144.stdout:6/192: readlink d9/dd/d11/d1b/l42 0 2024-09-10T11:29:33.577 INFO:tasks.workunit.client.1.smithi178.stdout:7/620: dwrite d6/d10/d1e/d2d/d2b/f39 [0,4194304] 0 2024-09-10T11:29:33.579 INFO:tasks.workunit.client.1.smithi178.stdout:2/858: dwrite d1/d7/d24/d46/d88/d9d/dab/dbb/fe7 [0,4194304] 0 2024-09-10T11:29:33.579 INFO:tasks.workunit.client.0.smithi144.stdout:0/181: symlink da/l2d 0 2024-09-10T11:29:33.579 INFO:tasks.workunit.client.0.smithi144.stdout:0/182: stat da/ce 0 2024-09-10T11:29:33.579 INFO:tasks.workunit.client.0.smithi144.stdout:9/281: link d11/d12/d1b/d22/l33 d11/l62 0 2024-09-10T11:29:33.580 INFO:tasks.workunit.client.1.smithi178.stdout:5/682: symlink d9/dc/dd/d37/dbd/dda/db8/le6 0 2024-09-10T11:29:33.580 INFO:tasks.workunit.client.0.smithi144.stdout:2/197: mkdir d7/de/d3d/d43 0 2024-09-10T11:29:33.581 INFO:tasks.workunit.client.1.smithi178.stdout:7/621: write d6/d10/d1d/d4a/f57 [428162,19490] 0 2024-09-10T11:29:33.583 INFO:tasks.workunit.client.0.smithi144.stdout:2/198: write d7/de/f28 [354765,27195] 0 2024-09-10T11:29:33.583 INFO:tasks.workunit.client.0.smithi144.stdout:1/332: mkdir d5/dd/d64 0 2024-09-10T11:29:33.585 INFO:tasks.workunit.client.0.smithi144.stdout:3/286: dread d4/dd/d21/f23 [0,4194304] 0 2024-09-10T11:29:33.586 INFO:tasks.workunit.client.0.smithi144.stdout:9/282: write d11/d12/d1b/d22/f3d [2175104,125607] 0 2024-09-10T11:29:33.587 INFO:tasks.workunit.client.0.smithi144.stdout:0/183: dwrite da/f19 [0,4194304] 0 2024-09-10T11:29:33.587 INFO:tasks.workunit.client.1.smithi178.stdout:2/859: creat d1/d7/d24/d7d/f11e x:0 0 0 2024-09-10T11:29:33.587 INFO:tasks.workunit.client.0.smithi144.stdout:9/283: fdatasync d11/d12/d1b/d22/d53/f54 0 2024-09-10T11:29:33.589 INFO:tasks.workunit.client.0.smithi144.stdout:6/193: mkdir d9/dd/d3e/d47/d13/d4a 0 2024-09-10T11:29:33.595 INFO:tasks.workunit.client.0.smithi144.stdout:4/220: rename d14/f1f to d14/d1e/d22/f4d 0 2024-09-10T11:29:33.595 INFO:tasks.workunit.client.0.smithi144.stdout:2/199: creat d7/da/f44 x:0 0 0 2024-09-10T11:29:33.595 INFO:tasks.workunit.client.0.smithi144.stdout:0/184: creat da/f2e x:0 0 0 2024-09-10T11:29:33.598 INFO:tasks.workunit.client.0.smithi144.stdout:3/287: write d4/f37 [332559,34769] 0 2024-09-10T11:29:33.599 INFO:tasks.workunit.client.0.smithi144.stdout:7/219: getdents d5/dc/d12/d2b 0 2024-09-10T11:29:33.600 INFO:tasks.workunit.client.0.smithi144.stdout:0/185: unlink da/c2c 0 2024-09-10T11:29:33.603 INFO:tasks.workunit.client.0.smithi144.stdout:1/333: rename d5/d4a/d3d to d5/d2a/d65 0 2024-09-10T11:29:33.603 INFO:tasks.workunit.client.0.smithi144.stdout:6/194: dread d9/dd/d11/f46 [0,4194304] 0 2024-09-10T11:29:33.605 INFO:tasks.workunit.client.0.smithi144.stdout:3/288: unlink d4/dd/d22/d17/f1f 0 2024-09-10T11:29:33.606 INFO:tasks.workunit.client.1.smithi178.stdout:1/554: dread d3/d9/d98/dac/f3b [0,4194304] 0 2024-09-10T11:29:33.607 INFO:tasks.workunit.client.0.smithi144.stdout:0/186: mknod da/c2f 0 2024-09-10T11:29:33.607 INFO:tasks.workunit.client.0.smithi144.stdout:9/284: dwrite fc [4194304,4194304] 0 2024-09-10T11:29:33.613 INFO:tasks.workunit.client.0.smithi144.stdout:3/289: dwrite d4/dd/f51 [0,4194304] 0 2024-09-10T11:29:33.614 INFO:tasks.workunit.client.1.smithi178.stdout:1/555: dwrite d3/d9/fc4 [0,4194304] 0 2024-09-10T11:29:33.615 INFO:tasks.workunit.client.0.smithi144.stdout:5/288: truncate d4/fa 3457591 0 2024-09-10T11:29:33.615 INFO:tasks.workunit.client.0.smithi144.stdout:3/290: write d4/dd/f51 [3538975,83879] 0 2024-09-10T11:29:33.616 INFO:tasks.workunit.client.0.smithi144.stdout:4/221: symlink d14/d1e/d22/d2b/d35/d3c/l4e 0 2024-09-10T11:29:33.617 INFO:tasks.workunit.client.0.smithi144.stdout:4/222: truncate d14/f1b 5138893 0 2024-09-10T11:29:33.619 INFO:tasks.workunit.client.0.smithi144.stdout:1/334: creat d5/df/d4b/d5d/f66 x:0 0 0 2024-09-10T11:29:33.629 INFO:tasks.workunit.client.0.smithi144.stdout:0/187: creat da/f30 x:0 0 0 2024-09-10T11:29:33.631 INFO:tasks.workunit.client.0.smithi144.stdout:9/285: creat d11/d12/d1b/d22/f63 x:0 0 0 2024-09-10T11:29:33.632 INFO:tasks.workunit.client.0.smithi144.stdout:2/200: truncate d7/f10 3549435 0 2024-09-10T11:29:33.634 INFO:tasks.workunit.client.0.smithi144.stdout:7/220: link d5/dc/d12/f1f d5/dc/d2a/f4b 0 2024-09-10T11:29:33.634 INFO:tasks.workunit.client.1.smithi178.stdout:1/556: dwrite d3/d9/d98/dac/d3a/d63/d7e/fa7 [0,4194304] 0 2024-09-10T11:29:33.636 INFO:tasks.workunit.client.1.smithi178.stdout:5/683: dwrite f4 [0,4194304] 0 2024-09-10T11:29:33.637 INFO:tasks.workunit.client.0.smithi144.stdout:6/195: mknod d9/dd/d3e/d47/c4b 0 2024-09-10T11:29:33.637 INFO:tasks.workunit.client.0.smithi144.stdout:4/223: mkdir d14/d1e/d22/d2b/d35/d4f 0 2024-09-10T11:29:33.639 INFO:tasks.workunit.client.0.smithi144.stdout:1/335: creat d5/df/d4b/d5d/f67 x:0 0 0 2024-09-10T11:29:33.646 INFO:tasks.workunit.client.0.smithi144.stdout:2/201: mkdir d7/de/d36/d45 0 2024-09-10T11:29:33.646 INFO:tasks.workunit.client.0.smithi144.stdout:9/286: write d11/d12/d1b/f61 [887790,22972] 0 2024-09-10T11:29:33.649 INFO:tasks.workunit.client.0.smithi144.stdout:7/221: rmdir d5/dc/d12/d2b 39 2024-09-10T11:29:33.652 INFO:tasks.workunit.client.0.smithi144.stdout:1/336: dwrite d5/d2a/f4d [0,4194304] 0 2024-09-10T11:29:33.653 INFO:tasks.workunit.client.0.smithi144.stdout:1/337: readlink d5/l5a 0 2024-09-10T11:29:33.653 INFO:tasks.workunit.client.0.smithi144.stdout:0/188: dread da/f16 [0,4194304] 0 2024-09-10T11:29:33.654 INFO:tasks.workunit.client.0.smithi144.stdout:3/291: rename d4/dd/d22/d17/d18/d34/d4b/d55 to d4/dd/d22/d17/d18/d34/d65 0 2024-09-10T11:29:33.654 INFO:tasks.workunit.client.0.smithi144.stdout:1/338: write d5/f8 [2894697,82925] 0 2024-09-10T11:29:33.657 INFO:tasks.workunit.client.0.smithi144.stdout:4/224: creat d14/d1e/f50 x:0 0 0 2024-09-10T11:29:33.659 INFO:tasks.workunit.client.0.smithi144.stdout:9/287: unlink c8 0 2024-09-10T11:29:33.660 INFO:tasks.workunit.client.1.smithi178.stdout:1/557: write d3/d9/d21/d4b/f17 [474442,27470] 0 2024-09-10T11:29:33.660 INFO:tasks.workunit.client.1.smithi178.stdout:5/684: dread d9/f32 [0,4194304] 0 2024-09-10T11:29:33.662 INFO:tasks.workunit.client.1.smithi178.stdout:1/558: truncate d3/da0/fc1 1091461 0 2024-09-10T11:29:33.662 INFO:tasks.workunit.client.0.smithi144.stdout:7/222: chown d5/c1b 4983561 1 2024-09-10T11:29:33.664 INFO:tasks.workunit.client.0.smithi144.stdout:0/189: symlink da/l31 0 2024-09-10T11:29:33.664 INFO:tasks.workunit.client.0.smithi144.stdout:3/292: rmdir d4/dd/d22/d17/d18 39 2024-09-10T11:29:33.666 INFO:tasks.workunit.client.1.smithi178.stdout:1/559: truncate d3/d9/d21/d4b/d2b/d5f/f86 516340 0 2024-09-10T11:29:33.667 INFO:tasks.workunit.client.0.smithi144.stdout:0/190: write f5 [221749,97784] 0 2024-09-10T11:29:33.667 INFO:tasks.workunit.client.0.smithi144.stdout:9/288: dwrite d11/d36/d3e/f55 [0,4194304] 0 2024-09-10T11:29:33.667 INFO:tasks.workunit.client.0.smithi144.stdout:4/225: readlink d14/l19 0 2024-09-10T11:29:33.668 INFO:tasks.workunit.client.1.smithi178.stdout:5/685: dwrite d9/fbe [0,4194304] 0 2024-09-10T11:29:33.670 INFO:tasks.workunit.client.0.smithi144.stdout:5/289: getdents d4/d20/d24/d2d 0 2024-09-10T11:29:33.671 INFO:tasks.workunit.client.0.smithi144.stdout:9/289: write d11/d12/d1b/f27 [1764368,66148] 0 2024-09-10T11:29:33.672 INFO:tasks.workunit.client.0.smithi144.stdout:6/196: creat d9/dd/d11/f4c x:0 0 0 2024-09-10T11:29:33.676 INFO:tasks.workunit.client.0.smithi144.stdout:3/293: readlink d4/dd/d22/d1d/l2b 0 2024-09-10T11:29:33.677 INFO:tasks.workunit.client.0.smithi144.stdout:0/191: creat da/f32 x:0 0 0 2024-09-10T11:29:33.677 INFO:tasks.workunit.client.0.smithi144.stdout:3/294: stat d4/dd/l15 0 2024-09-10T11:29:33.678 INFO:tasks.workunit.client.0.smithi144.stdout:0/192: chown da/f2b 23384205 1 2024-09-10T11:29:33.680 INFO:tasks.workunit.client.0.smithi144.stdout:4/226: symlink d14/d1e/d22/d34/l51 0 2024-09-10T11:29:33.680 INFO:tasks.workunit.client.1.smithi178.stdout:5/686: getdents d9/dc/d17 0 2024-09-10T11:29:33.681 INFO:tasks.workunit.client.0.smithi144.stdout:4/227: chown d14/d1e 3545145 1 2024-09-10T11:29:33.681 INFO:tasks.workunit.client.1.smithi178.stdout:1/560: dwrite d3/d9/d98/dac/d3a/d50/d6c/d78/db8/fbf [0,4194304] 0 2024-09-10T11:29:33.682 INFO:tasks.workunit.client.1.smithi178.stdout:1/561: fdatasync d3/d35/f74 0 2024-09-10T11:29:33.685 INFO:tasks.workunit.client.0.smithi144.stdout:3/295: dwrite d4/dd/d21/f5b [0,4194304] 0 2024-09-10T11:29:33.686 INFO:tasks.workunit.client.1.smithi178.stdout:5/687: unlink d9/dc/d17/f3c 0 2024-09-10T11:29:33.687 INFO:tasks.workunit.client.1.smithi178.stdout:1/562: readlink d3/d9/d21/d4b/d2b/l55 0 2024-09-10T11:29:33.691 INFO:tasks.workunit.client.0.smithi144.stdout:6/197: creat d9/dd/d3e/f4d x:0 0 0 2024-09-10T11:29:33.691 INFO:tasks.workunit.client.0.smithi144.stdout:7/223: dread d5/fa [0,4194304] 0 2024-09-10T11:29:33.691 INFO:tasks.workunit.client.1.smithi178.stdout:5/688: rmdir d9/dc/dd/d3f 39 2024-09-10T11:29:33.692 INFO:tasks.workunit.client.0.smithi144.stdout:4/228: dread d14/d1e/d22/d2b/d35/f38 [4194304,4194304] 0 2024-09-10T11:29:33.692 INFO:tasks.workunit.client.0.smithi144.stdout:0/193: symlink da/l33 0 2024-09-10T11:29:33.695 INFO:tasks.workunit.client.0.smithi144.stdout:6/198: dread d9/dd/d3e/d47/d13/f20 [0,4194304] 0 2024-09-10T11:29:33.696 INFO:tasks.workunit.client.0.smithi144.stdout:2/202: write d7/de/f2f [2169641,106459] 0 2024-09-10T11:29:33.696 INFO:tasks.workunit.client.1.smithi178.stdout:5/689: mknod d9/dc/dd/d16/ce7 0 2024-09-10T11:29:33.697 INFO:tasks.workunit.client.0.smithi144.stdout:6/199: readlink d9/dd/d11/d1b/l42 0 2024-09-10T11:29:33.697 INFO:tasks.workunit.client.0.smithi144.stdout:1/339: getdents d5/df/d4b 0 2024-09-10T11:29:33.704 INFO:tasks.workunit.client.0.smithi144.stdout:5/290: creat d4/d5/d32/f5f x:0 0 0 2024-09-10T11:29:33.704 INFO:tasks.workunit.client.0.smithi144.stdout:1/340: dwrite d5/df/d4b/d51/f50 [0,4194304] 0 2024-09-10T11:29:33.705 INFO:tasks.workunit.client.1.smithi178.stdout:5/690: dwrite d9/dc/dd/d37/dbd/dda/db8/fd4 [0,4194304] 0 2024-09-10T11:29:33.705 INFO:tasks.workunit.client.0.smithi144.stdout:1/341: dread - d5/df/d4b/d5d/f67 zero size 2024-09-10T11:29:33.706 INFO:tasks.workunit.client.0.smithi144.stdout:0/194: mkdir da/d34 0 2024-09-10T11:29:33.710 INFO:tasks.workunit.client.1.smithi178.stdout:1/563: dwrite d3/d9/d98/dac/d3a/d6e/f7c [0,4194304] 0 2024-09-10T11:29:33.710 INFO:tasks.workunit.client.0.smithi144.stdout:2/203: creat d7/d29/f46 x:0 0 0 2024-09-10T11:29:33.715 INFO:tasks.workunit.client.1.smithi178.stdout:1/564: unlink d3/d9/d98/dac/d2a/c4d 0 2024-09-10T11:29:33.720 INFO:tasks.workunit.client.1.smithi178.stdout:1/565: symlink d3/d9/d98/dac/d3a/d6e/lce 0 2024-09-10T11:29:33.721 INFO:tasks.workunit.client.0.smithi144.stdout:5/291: write d4/d5/d32/f33 [2672226,38780] 0 2024-09-10T11:29:33.723 INFO:tasks.workunit.client.1.smithi178.stdout:1/566: creat d3/d9/d98/dac/d3a/d63/fcf x:0 0 0 2024-09-10T11:29:33.724 INFO:tasks.workunit.client.0.smithi144.stdout:9/290: getdents d11/d12/d1b/d22 0 2024-09-10T11:29:33.724 INFO:tasks.workunit.client.0.smithi144.stdout:1/342: dwrite d5/df/f33 [0,4194304] 0 2024-09-10T11:29:33.724 INFO:tasks.workunit.client.0.smithi144.stdout:6/200: mknod d9/c4e 0 2024-09-10T11:29:33.727 INFO:tasks.workunit.client.0.smithi144.stdout:3/296: dread d4/dd/d22/d1d/f2a [0,4194304] 0 2024-09-10T11:29:33.727 INFO:tasks.workunit.client.1.smithi178.stdout:1/567: creat d3/d9/d98/dac/d3a/d50/db5/dcc/fd0 x:0 0 0 2024-09-10T11:29:33.727 INFO:tasks.workunit.client.0.smithi144.stdout:7/224: creat d5/dc/d11/f4c x:0 0 0 2024-09-10T11:29:33.730 INFO:tasks.workunit.client.0.smithi144.stdout:4/229: getdents d14 0 2024-09-10T11:29:33.731 INFO:tasks.workunit.client.1.smithi178.stdout:1/568: write d3/d9/d98/dac/d3a/d63/d7e/fa7 [3674335,127118] 0 2024-09-10T11:29:33.731 INFO:tasks.workunit.client.0.smithi144.stdout:0/195: dwrite da/f2a [0,4194304] 0 2024-09-10T11:29:33.731 INFO:tasks.workunit.client.0.smithi144.stdout:2/204: mkdir d7/de/d36/d45/d47 0 2024-09-10T11:29:33.732 INFO:tasks.workunit.client.0.smithi144.stdout:1/343: chown d5/df/d4b/d5d/f67 446 1 2024-09-10T11:29:33.734 INFO:tasks.workunit.client.0.smithi144.stdout:5/292: write d4/d20/d24/f3e [900650,83187] 0 2024-09-10T11:29:33.735 INFO:tasks.workunit.client.0.smithi144.stdout:6/201: unlink d9/f45 0 2024-09-10T11:29:33.737 INFO:tasks.workunit.client.0.smithi144.stdout:9/291: mkdir d11/d12/d1b/d22/d53/d64 0 2024-09-10T11:29:33.737 INFO:tasks.workunit.client.0.smithi144.stdout:7/225: fsync d5/f38 0 2024-09-10T11:29:33.738 INFO:tasks.workunit.client.0.smithi144.stdout:0/196: fdatasync da/f32 0 2024-09-10T11:29:33.739 INFO:tasks.workunit.client.0.smithi144.stdout:7/226: stat d5 0 2024-09-10T11:29:33.741 INFO:tasks.workunit.client.0.smithi144.stdout:6/202: mkdir d9/d1f/d2c/d4f 0 2024-09-10T11:29:33.742 INFO:tasks.workunit.client.0.smithi144.stdout:5/293: symlink d4/d5/d3f/l60 0 2024-09-10T11:29:33.743 INFO:tasks.workunit.client.0.smithi144.stdout:3/297: dwrite d4/dd/f51 [0,4194304] 0 2024-09-10T11:29:33.743 INFO:tasks.workunit.client.1.smithi178.stdout:5/691: dread d9/dc/dd/d16/fb3 [0,4194304] 0 2024-09-10T11:29:33.746 INFO:tasks.workunit.client.1.smithi178.stdout:5/692: creat d9/d11/fe8 x:0 0 0 2024-09-10T11:29:33.747 INFO:tasks.workunit.client.1.smithi178.stdout:8/722: sync 2024-09-10T11:29:33.747 INFO:tasks.workunit.client.0.smithi144.stdout:5/294: truncate d4/d5/d32/f33 4567304 0 2024-09-10T11:29:33.748 INFO:tasks.workunit.client.0.smithi144.stdout:4/230: rename d14/f1b to d14/d1e/d22/d34/d3e/d40/d4b/f52 0 2024-09-10T11:29:33.748 INFO:tasks.workunit.client.1.smithi178.stdout:8/723: stat d5/c8a 0 2024-09-10T11:29:33.750 INFO:tasks.workunit.client.0.smithi144.stdout:6/203: unlink d9/dd/d11/f27 0 2024-09-10T11:29:33.750 INFO:tasks.workunit.client.0.smithi144.stdout:5/295: fdatasync d4/d5/d18/f1f 0 2024-09-10T11:29:33.750 INFO:tasks.workunit.client.0.smithi144.stdout:0/197: dread da/f2a [0,4194304] 0 2024-09-10T11:29:33.751 INFO:tasks.workunit.client.0.smithi144.stdout:1/344: mknod d5/d4a/d63/c68 0 2024-09-10T11:29:33.752 INFO:tasks.workunit.client.0.smithi144.stdout:6/204: dread - d9/d1f/f43 zero size 2024-09-10T11:29:33.752 INFO:tasks.workunit.client.0.smithi144.stdout:7/227: dwrite d5/dc/d12/d2f/f4a [0,4194304] 0 2024-09-10T11:29:33.754 INFO:tasks.workunit.client.1.smithi178.stdout:8/724: getdents d5/d30/d33/d8c/da7 0 2024-09-10T11:29:33.755 INFO:tasks.workunit.client.0.smithi144.stdout:6/205: fdatasync d9/dd/d3e/d47/f16 0 2024-09-10T11:29:33.755 INFO:tasks.workunit.client.0.smithi144.stdout:0/198: write da/f2b [281400,98225] 0 2024-09-10T11:29:33.760 INFO:tasks.workunit.client.1.smithi178.stdout:1/569: write d3/d9/d21/f67 [673377,55710] 0 2024-09-10T11:29:33.760 INFO:tasks.workunit.client.0.smithi144.stdout:7/228: dread d5/fa [0,4194304] 0 2024-09-10T11:29:33.763 INFO:tasks.workunit.client.0.smithi144.stdout:1/345: dwrite d5/f8 [0,4194304] 0 2024-09-10T11:29:33.763 INFO:tasks.workunit.client.0.smithi144.stdout:2/205: link d7/f10 d7/d42/f48 0 2024-09-10T11:29:33.763 INFO:tasks.workunit.client.0.smithi144.stdout:7/229: write d5/dc/d12/d2f/f47 [186757,130612] 0 2024-09-10T11:29:33.763 INFO:tasks.workunit.client.0.smithi144.stdout:3/298: rename d4/dd/d22 to d4/dd/d22/d17/d18/d34/d4b/d66 22 2024-09-10T11:29:33.764 INFO:tasks.workunit.client.0.smithi144.stdout:1/346: chown d5/d2a/d65 23033052 1 2024-09-10T11:29:33.764 INFO:tasks.workunit.client.0.smithi144.stdout:2/206: chown d7/f15 9141 1 2024-09-10T11:29:33.765 INFO:tasks.workunit.client.1.smithi178.stdout:8/725: dwrite d5/db/d1e/f62 [0,4194304] 0 2024-09-10T11:29:33.766 INFO:tasks.workunit.client.0.smithi144.stdout:1/347: write d5/f6 [2026758,62261] 0 2024-09-10T11:29:33.767 INFO:tasks.workunit.client.1.smithi178.stdout:8/726: readlink d5/db/d10/d53/d82/l93 0 2024-09-10T11:29:33.768 INFO:tasks.workunit.client.0.smithi144.stdout:6/206: symlink d9/dd/d3e/d47/d13/l50 0 2024-09-10T11:29:33.770 INFO:tasks.workunit.client.0.smithi144.stdout:9/292: link d11/d36/d3e/l3f d11/d12/d1b/d2c/l65 0 2024-09-10T11:29:33.771 INFO:tasks.workunit.client.1.smithi178.stdout:1/570: dwrite d3/f22 [4194304,4194304] 0 2024-09-10T11:29:33.774 INFO:tasks.workunit.client.1.smithi178.stdout:8/727: rename d5/db/d10/d20/l71 to d5/d1d/d22/lf2 0 2024-09-10T11:29:33.775 INFO:tasks.workunit.client.0.smithi144.stdout:3/299: dwrite d4/dd/d21/f62 [0,4194304] 0 2024-09-10T11:29:33.778 INFO:tasks.workunit.client.0.smithi144.stdout:7/230: creat d5/dc/d12/d2f/f4d x:0 0 0 2024-09-10T11:29:33.778 INFO:tasks.workunit.client.1.smithi178.stdout:1/571: rename d3/d9/d98/dac/d3a/d63 to d3/d9/d98/dd1 0 2024-09-10T11:29:33.779 INFO:tasks.workunit.client.0.smithi144.stdout:2/207: stat d7/de/d36/c3a 0 2024-09-10T11:29:33.779 INFO:tasks.workunit.client.0.smithi144.stdout:7/231: readlink d5/dc/d14/d40/l42 0 2024-09-10T11:29:33.780 INFO:tasks.workunit.client.0.smithi144.stdout:6/207: unlink c5 0 2024-09-10T11:29:33.781 INFO:tasks.workunit.client.0.smithi144.stdout:5/296: rename d4/d5/l17 to d4/d5/l61 0 2024-09-10T11:29:33.782 INFO:tasks.workunit.client.0.smithi144.stdout:4/231: readlink d14/l36 0 2024-09-10T11:29:33.782 INFO:tasks.workunit.client.1.smithi178.stdout:8/728: mknod d5/d30/db4/cf3 0 2024-09-10T11:29:33.783 INFO:tasks.workunit.client.0.smithi144.stdout:3/300: mkdir d4/d28/d67 0 2024-09-10T11:29:33.783 INFO:tasks.workunit.client.1.smithi178.stdout:1/572: mknod d3/d9/d1a/d48/cd2 0 2024-09-10T11:29:33.784 INFO:tasks.workunit.client.0.smithi144.stdout:0/199: rmdir da/d34 0 2024-09-10T11:29:33.785 INFO:tasks.workunit.client.1.smithi178.stdout:8/729: symlink d5/d30/lf4 0 2024-09-10T11:29:33.785 INFO:tasks.workunit.client.1.smithi178.stdout:8/730: read - d5/d61/fed zero size 2024-09-10T11:29:33.787 INFO:tasks.workunit.client.1.smithi178.stdout:1/573: mkdir d3/d9/d21/d4b/dd3 0 2024-09-10T11:29:33.788 INFO:tasks.workunit.client.0.smithi144.stdout:6/208: creat d9/dd/d3a/f51 x:0 0 0 2024-09-10T11:29:33.791 INFO:tasks.workunit.client.1.smithi178.stdout:8/731: creat d5/db/ff5 x:0 0 0 2024-09-10T11:29:33.791 INFO:tasks.workunit.client.1.smithi178.stdout:1/574: symlink d3/d9/d98/dac/d3a/d50/d80/ld4 0 2024-09-10T11:29:33.792 INFO:tasks.workunit.client.0.smithi144.stdout:4/232: chown d14/d1e/d22/d34 1 1 2024-09-10T11:29:33.792 INFO:tasks.workunit.client.0.smithi144.stdout:9/293: symlink d11/d12/d1b/l66 0 2024-09-10T11:29:33.792 INFO:tasks.workunit.client.0.smithi144.stdout:6/209: write d9/d1f/f43 [253458,43904] 0 2024-09-10T11:29:33.792 INFO:tasks.workunit.client.0.smithi144.stdout:6/210: truncate d9/d1f/d2c/f39 847838 0 2024-09-10T11:29:33.793 INFO:tasks.workunit.client.0.smithi144.stdout:5/297: creat d4/d5/d3f/f62 x:0 0 0 2024-09-10T11:29:33.793 INFO:tasks.workunit.client.1.smithi178.stdout:1/575: chown d3/d9/d21/d4b/f17 0 1 2024-09-10T11:29:33.793 INFO:tasks.workunit.client.0.smithi144.stdout:5/298: stat d4/d5/l30 0 2024-09-10T11:29:33.798 INFO:tasks.workunit.client.1.smithi178.stdout:1/576: write d3/d9/d98/dac/d3a/d50/d6c/d78/db8/fbf [4886478,102839] 0 2024-09-10T11:29:33.798 INFO:tasks.workunit.client.1.smithi178.stdout:8/732: dwrite d5/d30/d33/d75/f86 [0,4194304] 0 2024-09-10T11:29:33.799 INFO:tasks.workunit.client.0.smithi144.stdout:7/232: mknod d5/dc/d11/c4e 0 2024-09-10T11:29:33.800 INFO:tasks.workunit.client.1.smithi178.stdout:8/733: truncate d5/db/d1e/fa3 4639331 0 2024-09-10T11:29:33.800 INFO:tasks.workunit.client.0.smithi144.stdout:9/294: creat d11/d12/d1b/d22/f67 x:0 0 0 2024-09-10T11:29:33.801 INFO:tasks.workunit.client.0.smithi144.stdout:4/233: dread d14/d1e/d22/f2d [0,4194304] 0 2024-09-10T11:29:33.807 INFO:tasks.workunit.client.0.smithi144.stdout:9/295: unlink d11/d12/d1b/d22/l28 0 2024-09-10T11:29:33.809 INFO:tasks.workunit.client.0.smithi144.stdout:8/175: sync 2024-09-10T11:29:33.810 INFO:tasks.workunit.client.1.smithi178.stdout:8/734: mknod d5/db/d10/d70/cf6 0 2024-09-10T11:29:33.811 INFO:tasks.workunit.client.0.smithi144.stdout:5/299: dwrite d4/d5/f23 [0,4194304] 0 2024-09-10T11:29:33.812 INFO:tasks.workunit.client.0.smithi144.stdout:2/208: write d7/d42/f48 [3294014,121114] 0 2024-09-10T11:29:33.814 INFO:tasks.workunit.client.1.smithi178.stdout:1/577: symlink d3/d9/d98/dac/d3a/d50/ld5 0 2024-09-10T11:29:33.815 INFO:tasks.workunit.client.0.smithi144.stdout:4/234: link l13 d14/l53 0 2024-09-10T11:29:33.816 INFO:tasks.workunit.client.1.smithi178.stdout:8/735: fdatasync d5/db/d1e/fa3 0 2024-09-10T11:29:33.818 INFO:tasks.workunit.client.0.smithi144.stdout:1/348: dread d5/f41 [0,4194304] 0 2024-09-10T11:29:33.818 INFO:tasks.workunit.client.0.smithi144.stdout:7/233: link d5/dc/d12/d2b/l45 d5/dc/d12/d2f/l4f 0 2024-09-10T11:29:33.818 INFO:tasks.workunit.client.0.smithi144.stdout:5/300: chown d4/d20/d24/d2d/c3c 370156271 1 2024-09-10T11:29:33.818 INFO:tasks.workunit.client.1.smithi178.stdout:8/736: truncate d5/db/d10/d70/fcb 812642 0 2024-09-10T11:29:33.821 INFO:tasks.workunit.client.1.smithi178.stdout:8/737: write d5/db/d10/d20/f26 [863233,116574] 0 2024-09-10T11:29:33.821 INFO:tasks.workunit.client.0.smithi144.stdout:4/235: write fa [3694555,8365] 0 2024-09-10T11:29:33.821 INFO:tasks.workunit.client.0.smithi144.stdout:3/301: dwrite d4/dd/d21/f5d [0,4194304] 0 2024-09-10T11:29:33.825 INFO:tasks.workunit.client.1.smithi178.stdout:8/738: read d5/d1d/f60 [3865918,123422] 0 2024-09-10T11:29:33.830 INFO:tasks.workunit.client.0.smithi144.stdout:8/176: getdents d0/d9 0 2024-09-10T11:29:33.833 INFO:tasks.workunit.client.1.smithi178.stdout:8/739: link d5/cb3 d5/d1d/d22/cf7 0 2024-09-10T11:29:33.833 INFO:tasks.workunit.client.0.smithi144.stdout:8/177: stat d0/d9/f3b 0 2024-09-10T11:29:33.833 INFO:tasks.workunit.client.0.smithi144.stdout:3/302: symlink d4/dd/d22/d17/d18/d34/l68 0 2024-09-10T11:29:33.833 INFO:tasks.workunit.client.0.smithi144.stdout:2/209: dread d7/de/d3d/f3e [0,4194304] 0 2024-09-10T11:29:33.834 INFO:tasks.workunit.client.0.smithi144.stdout:9/296: dwrite d11/d12/d1b/d22/f4a [0,4194304] 0 2024-09-10T11:29:33.835 INFO:tasks.workunit.client.1.smithi178.stdout:1/578: truncate d3/d9/d98/dac/d3a/d50/d6c/d78/db8/fbf 3291918 0 2024-09-10T11:29:33.837 INFO:tasks.workunit.client.0.smithi144.stdout:5/301: dwrite d4/d5/f25 [0,4194304] 0 2024-09-10T11:29:33.837 INFO:tasks.workunit.client.0.smithi144.stdout:8/178: chown d0/d9/d23/d26/l32 7626459 1 2024-09-10T11:29:33.838 INFO:tasks.workunit.client.0.smithi144.stdout:2/210: mknod d7/c49 0 2024-09-10T11:29:33.838 INFO:tasks.workunit.client.0.smithi144.stdout:3/303: fsync d4/dd/d22/d17/d18/d34/f5c 0 2024-09-10T11:29:33.839 INFO:tasks.workunit.client.1.smithi178.stdout:8/740: dwrite d5/d1d/fb0 [0,4194304] 0 2024-09-10T11:29:33.841 INFO:tasks.workunit.client.0.smithi144.stdout:2/211: fsync d7/da/f3c 0 2024-09-10T11:29:33.842 INFO:tasks.workunit.client.0.smithi144.stdout:9/297: write d11/d12/d1b/d22/f3d [3291754,50065] 0 2024-09-10T11:29:33.844 INFO:tasks.workunit.client.1.smithi178.stdout:8/741: mknod d5/db/d10/d53/d82/db2/cf8 0 2024-09-10T11:29:33.849 INFO:tasks.workunit.client.0.smithi144.stdout:8/179: symlink d0/d9/d23/l3c 0 2024-09-10T11:29:33.853 INFO:tasks.workunit.client.0.smithi144.stdout:5/302: creat d4/d20/d24/f63 x:0 0 0 2024-09-10T11:29:33.854 INFO:tasks.workunit.client.0.smithi144.stdout:3/304: getdents d4/dd/d22/d17/d18/d34/d4b/d63 0 2024-09-10T11:29:33.859 INFO:tasks.workunit.client.0.smithi144.stdout:1/349: dwrite d5/fb [0,4194304] 0 2024-09-10T11:29:33.859 INFO:tasks.workunit.client.0.smithi144.stdout:8/180: write d0/d9/d1f/f1c [3972895,44311] 0 2024-09-10T11:29:33.860 INFO:tasks.workunit.client.0.smithi144.stdout:4/236: dwrite d14/d1e/d22/f48 [0,4194304] 0 2024-09-10T11:29:33.864 INFO:tasks.workunit.client.0.smithi144.stdout:2/212: dwrite d7/da/f3c [0,4194304] 0 2024-09-10T11:29:33.866 INFO:tasks.workunit.client.0.smithi144.stdout:2/213: chown c1 62 1 2024-09-10T11:29:33.867 INFO:tasks.workunit.client.0.smithi144.stdout:5/303: mkdir d4/d20/d22/d5c/d5d/d64 0 2024-09-10T11:29:33.868 INFO:tasks.workunit.client.0.smithi144.stdout:8/181: write d0/f10 [1860834,21576] 0 2024-09-10T11:29:33.871 INFO:tasks.workunit.client.0.smithi144.stdout:4/237: rmdir d14/d1e/d22/d34/d3e 39 2024-09-10T11:29:33.871 INFO:tasks.workunit.client.0.smithi144.stdout:1/350: creat d5/d4a/d44/f69 x:0 0 0 2024-09-10T11:29:33.872 INFO:tasks.workunit.client.0.smithi144.stdout:3/305: symlink d4/dd/d22/d17/d18/d34/d4b/l69 0 2024-09-10T11:29:33.872 INFO:tasks.workunit.client.0.smithi144.stdout:7/234: dread d5/dc/d2a/d41/f49 [0,4194304] 0 2024-09-10T11:29:33.875 INFO:tasks.workunit.client.0.smithi144.stdout:8/182: mkdir d0/d9/d23/d3d 0 2024-09-10T11:29:33.875 INFO:tasks.workunit.client.0.smithi144.stdout:1/351: chown d5/f38 52827 1 2024-09-10T11:29:33.876 INFO:tasks.workunit.client.0.smithi144.stdout:5/304: write d4/d5/d18/f1f [4233946,129594] 0 2024-09-10T11:29:33.877 INFO:tasks.workunit.client.0.smithi144.stdout:3/306: read d4/dd/d22/f3e [88374,99128] 0 2024-09-10T11:29:33.879 INFO:tasks.workunit.client.0.smithi144.stdout:3/307: chown d4/l7 26 1 2024-09-10T11:29:33.879 INFO:tasks.workunit.client.0.smithi144.stdout:7/235: creat d5/dc/d14/f50 x:0 0 0 2024-09-10T11:29:33.880 INFO:tasks.workunit.client.0.smithi144.stdout:1/352: rmdir d5/d2a/d65 39 2024-09-10T11:29:33.881 INFO:tasks.workunit.client.0.smithi144.stdout:9/298: dread d11/f2b [0,4194304] 0 2024-09-10T11:29:33.881 INFO:tasks.workunit.client.0.smithi144.stdout:7/236: stat f1 0 2024-09-10T11:29:33.882 INFO:tasks.workunit.client.0.smithi144.stdout:8/183: mknod d0/d9/d21/c3e 0 2024-09-10T11:29:33.883 INFO:tasks.workunit.client.0.smithi144.stdout:2/214: dwrite d7/da/f19 [0,4194304] 0 2024-09-10T11:29:33.883 INFO:tasks.workunit.client.0.smithi144.stdout:5/305: creat d4/d5/d3f/f65 x:0 0 0 2024-09-10T11:29:33.884 INFO:tasks.workunit.client.0.smithi144.stdout:7/237: dread - d5/dc/f3c zero size 2024-09-10T11:29:33.884 INFO:tasks.workunit.client.0.smithi144.stdout:3/308: fdatasync d4/dd/d21/d2f/f3a 0 2024-09-10T11:29:33.884 INFO:tasks.workunit.client.0.smithi144.stdout:2/215: readlink l0 0 2024-09-10T11:29:33.885 INFO:tasks.workunit.client.1.smithi178.stdout:1/579: dread d3/d9/d21/d4b/f15 [0,4194304] 0 2024-09-10T11:29:33.886 INFO:tasks.workunit.client.0.smithi144.stdout:1/353: creat d5/d4a/f6a x:0 0 0 2024-09-10T11:29:33.886 INFO:tasks.workunit.client.0.smithi144.stdout:5/306: chown d4/d5/d3f/f62 8110990 1 2024-09-10T11:29:33.887 INFO:tasks.workunit.client.1.smithi178.stdout:8/742: dread d5/d1d/d22/d76/f9e [0,4194304] 0 2024-09-10T11:29:33.888 INFO:tasks.workunit.client.1.smithi178.stdout:8/743: stat d5/db/d10/d53/d82/fd8 0 2024-09-10T11:29:33.890 INFO:tasks.workunit.client.0.smithi144.stdout:7/238: truncate d5/dc/d12/f1f 255180 0 2024-09-10T11:29:33.891 INFO:tasks.workunit.client.1.smithi178.stdout:8/744: creat d5/d30/ff9 x:0 0 0 2024-09-10T11:29:33.892 INFO:tasks.workunit.client.0.smithi144.stdout:8/184: getdents d0/d1b 0 2024-09-10T11:29:33.893 INFO:tasks.workunit.client.0.smithi144.stdout:5/307: fdatasync d4/d5/f28 0 2024-09-10T11:29:33.895 INFO:tasks.workunit.client.0.smithi144.stdout:3/309: symlink d4/dd/d22/l6a 0 2024-09-10T11:29:33.899 INFO:tasks.workunit.client.0.smithi144.stdout:9/299: dread d11/d12/d1b/d2c/f31 [0,4194304] 0 2024-09-10T11:29:33.902 INFO:tasks.workunit.client.1.smithi178.stdout:1/580: dwrite d3/f1c [0,4194304] 0 2024-09-10T11:29:33.902 INFO:tasks.workunit.client.0.smithi144.stdout:3/310: write d4/dd/d22/d17/f30 [2760308,35801] 0 2024-09-10T11:29:33.903 INFO:tasks.workunit.client.0.smithi144.stdout:8/185: rename d0/d1b/f31 to d0/f3f 0 2024-09-10T11:29:33.903 INFO:tasks.workunit.client.1.smithi178.stdout:1/581: chown d3/c33 566 1 2024-09-10T11:29:33.906 INFO:tasks.workunit.client.1.smithi178.stdout:8/745: truncate d5/d30/f4b 4153230 0 2024-09-10T11:29:33.907 INFO:tasks.workunit.client.0.smithi144.stdout:2/216: dwrite d7/de/f1b [0,4194304] 0 2024-09-10T11:29:33.909 INFO:tasks.workunit.client.1.smithi178.stdout:1/582: getdents d3/d9/d98/dac/d2a 0 2024-09-10T11:29:33.909 INFO:tasks.workunit.client.0.smithi144.stdout:2/217: chown d7/da/d1e/d21 230204689 1 2024-09-10T11:29:33.913 INFO:tasks.workunit.client.0.smithi144.stdout:4/238: dread d14/d1e/f21 [0,4194304] 0 2024-09-10T11:29:33.913 INFO:tasks.workunit.client.0.smithi144.stdout:4/239: fsync d14/d1e/f45 0 2024-09-10T11:29:33.915 INFO:tasks.workunit.client.1.smithi178.stdout:1/583: rename d3/d35/cb1 to d3/d9/d98/dac/d3a/d50/cd6 0 2024-09-10T11:29:33.915 INFO:tasks.workunit.client.0.smithi144.stdout:7/239: dread d5/dc/d12/d2f/f4a [0,4194304] 0 2024-09-10T11:29:33.917 INFO:tasks.workunit.client.1.smithi178.stdout:8/746: truncate d5/db/d3d/f97 604303 0 2024-09-10T11:29:33.918 INFO:tasks.workunit.client.1.smithi178.stdout:1/584: read - d3/d9/d98/dac/d3a/d6e/d8a/f7a zero size 2024-09-10T11:29:33.919 INFO:tasks.workunit.client.1.smithi178.stdout:8/747: write d5/d30/d33/d8c/daa/fdb [769304,1477] 0 2024-09-10T11:29:33.920 INFO:tasks.workunit.client.0.smithi144.stdout:3/311: creat d4/dd/d22/f6b x:0 0 0 2024-09-10T11:29:33.920 INFO:tasks.workunit.client.0.smithi144.stdout:2/218: symlink d7/de/d36/d45/l4a 0 2024-09-10T11:29:33.920 INFO:tasks.workunit.client.0.smithi144.stdout:5/308: dwrite d4/d5/f28 [0,4194304] 0 2024-09-10T11:29:33.920 INFO:tasks.workunit.client.1.smithi178.stdout:1/585: chown d3/d9/d98/dac/d3a/d50/d80 384321 1 2024-09-10T11:29:33.921 INFO:tasks.workunit.client.0.smithi144.stdout:8/186: unlink d0/d9/d21/c22 0 2024-09-10T11:29:33.921 INFO:tasks.workunit.client.0.smithi144.stdout:1/354: dread d5/fc [0,4194304] 0 2024-09-10T11:29:33.921 INFO:tasks.workunit.client.0.smithi144.stdout:7/240: mknod d5/dc/d2a/c51 0 2024-09-10T11:29:33.922 INFO:tasks.workunit.client.1.smithi178.stdout:8/748: write d5/db/d1e/f39 [793860,35110] 0 2024-09-10T11:29:33.922 INFO:tasks.workunit.client.1.smithi178.stdout:1/586: chown d3/d9/d21/d6f/f75 0 1 2024-09-10T11:29:33.923 INFO:tasks.workunit.client.0.smithi144.stdout:9/300: rename d11/d12/d1b/d22/f67 to d11/d12/d1b/d22/d53/d64/f68 0 2024-09-10T11:29:33.924 INFO:tasks.workunit.client.1.smithi178.stdout:8/749: write d5/db/d10/d20/dc0/fe3 [4305552,48759] 0 2024-09-10T11:29:33.924 INFO:tasks.workunit.client.1.smithi178.stdout:1/587: dread - d3/d9/d98/dac/d3a/dae/fb3 zero size 2024-09-10T11:29:33.925 INFO:tasks.workunit.client.1.smithi178.stdout:8/750: chown d5/db 13 1 2024-09-10T11:29:33.926 INFO:tasks.workunit.client.0.smithi144.stdout:8/187: dread - d0/d9/f3b zero size 2024-09-10T11:29:33.929 INFO:tasks.workunit.client.1.smithi178.stdout:8/751: mkdir d5/db/dda/dfa 0 2024-09-10T11:29:33.930 INFO:tasks.workunit.client.0.smithi144.stdout:5/309: chown d4/d5/d32/f5f 976 1 2024-09-10T11:29:33.937 INFO:tasks.workunit.client.0.smithi144.stdout:7/241: dread d5/dc/d14/f44 [0,4194304] 0 2024-09-10T11:29:33.939 INFO:tasks.workunit.client.0.smithi144.stdout:5/310: write d4/d5/d3f/d59/f5e [498877,100792] 0 2024-09-10T11:29:33.939 INFO:tasks.workunit.client.1.smithi178.stdout:8/752: link d5/db/d10/d70/ca4 d5/d30/d33/d75/cfb 0 2024-09-10T11:29:33.941 INFO:tasks.workunit.client.0.smithi144.stdout:1/355: dread d5/df/d4b/d51/f50 [0,4194304] 0 2024-09-10T11:29:33.941 INFO:tasks.workunit.client.0.smithi144.stdout:3/312: dread d4/dd/d22/d17/f30 [0,4194304] 0 2024-09-10T11:29:33.943 INFO:tasks.workunit.client.0.smithi144.stdout:2/219: dwrite f6 [8388608,4194304] 0 2024-09-10T11:29:33.946 INFO:tasks.workunit.client.0.smithi144.stdout:9/301: dwrite d11/d36/f52 [0,4194304] 0 2024-09-10T11:29:33.946 INFO:tasks.workunit.client.0.smithi144.stdout:5/311: symlink d4/d5/d3f/d44/l66 0 2024-09-10T11:29:33.947 INFO:tasks.workunit.client.0.smithi144.stdout:3/313: symlink d4/d1a/l6c 0 2024-09-10T11:29:33.947 INFO:tasks.workunit.client.0.smithi144.stdout:8/188: dwrite d0/d9/d21/d25/f29 [0,4194304] 0 2024-09-10T11:29:33.952 INFO:tasks.workunit.client.0.smithi144.stdout:9/302: chown d11/d12/l5f 15862502 1 2024-09-10T11:29:33.953 INFO:tasks.workunit.client.0.smithi144.stdout:8/189: stat d0/f3f 0 2024-09-10T11:29:33.953 INFO:tasks.workunit.client.0.smithi144.stdout:2/220: rmdir d7/de/d36/d45 39 2024-09-10T11:29:33.959 INFO:tasks.workunit.client.0.smithi144.stdout:7/242: dwrite d5/dc/d11/f3d [0,4194304] 0 2024-09-10T11:29:33.959 INFO:tasks.workunit.client.0.smithi144.stdout:8/190: chown d0/d9/d1f/l1a 920059 1 2024-09-10T11:29:33.960 INFO:tasks.workunit.client.0.smithi144.stdout:1/356: link d5/dd/f1d d5/d2a/d65/f6b 0 2024-09-10T11:29:33.961 INFO:tasks.workunit.client.0.smithi144.stdout:2/221: creat d7/d29/f4b x:0 0 0 2024-09-10T11:29:33.962 INFO:tasks.workunit.client.0.smithi144.stdout:9/303: mknod d11/d12/d1b/d22/d53/d64/c69 0 2024-09-10T11:29:33.963 INFO:tasks.workunit.client.0.smithi144.stdout:7/243: write d5/dc/d11/f4c [750477,35512] 0 2024-09-10T11:29:33.963 INFO:tasks.workunit.client.0.smithi144.stdout:4/240: dread f9 [0,4194304] 0 2024-09-10T11:29:33.966 INFO:tasks.workunit.client.0.smithi144.stdout:3/314: dwrite d4/dd/d21/f41 [0,4194304] 0 2024-09-10T11:29:33.966 INFO:tasks.workunit.client.1.smithi178.stdout:8/753: dread d5/db/d10/d35/f5a [0,4194304] 0 2024-09-10T11:29:33.967 INFO:tasks.workunit.client.0.smithi144.stdout:7/244: write d5/dc/d12/d2b/f43 [986443,25] 0 2024-09-10T11:29:33.968 INFO:tasks.workunit.client.1.smithi178.stdout:8/754: write d5/db/d10/d20/fb9 [405528,77850] 0 2024-09-10T11:29:33.971 INFO:tasks.workunit.client.0.smithi144.stdout:8/191: creat d0/d9/d23/f40 x:0 0 0 2024-09-10T11:29:33.973 INFO:tasks.workunit.client.0.smithi144.stdout:1/357: unlink d5/dd/f3f 0 2024-09-10T11:29:33.976 INFO:tasks.workunit.client.1.smithi178.stdout:8/755: link d5/d30/d33/d75/f86 d5/db/d10/d20/dc0/ffc 0 2024-09-10T11:29:33.979 INFO:tasks.workunit.client.1.smithi178.stdout:8/756: creat d5/db/d3d/de7/ffd x:0 0 0 2024-09-10T11:29:33.980 INFO:tasks.workunit.client.0.smithi144.stdout:3/315: creat d4/dd/d22/d17/d18/f6d x:0 0 0 2024-09-10T11:29:33.981 INFO:tasks.workunit.client.0.smithi144.stdout:5/312: rename d4/d5/d3f to d4/d5/d18/d67 0 2024-09-10T11:29:33.982 INFO:tasks.workunit.client.1.smithi178.stdout:8/757: mknod d5/db/d10/d70/cfe 0 2024-09-10T11:29:33.984 INFO:tasks.workunit.client.0.smithi144.stdout:2/222: dwrite d7/d29/f3f [0,4194304] 0 2024-09-10T11:29:33.984 INFO:tasks.workunit.client.0.smithi144.stdout:1/358: mknod d5/d4a/c6c 0 2024-09-10T11:29:33.985 INFO:tasks.workunit.client.0.smithi144.stdout:2/223: chown d7/l34 765 1 2024-09-10T11:29:33.986 INFO:tasks.workunit.client.0.smithi144.stdout:9/304: symlink d11/d12/d1b/l6a 0 2024-09-10T11:29:33.986 INFO:tasks.workunit.client.0.smithi144.stdout:2/224: truncate d7/f41 471452 0 2024-09-10T11:29:33.990 INFO:tasks.workunit.client.1.smithi178.stdout:8/758: dwrite d5/d1d/fb0 [4194304,4194304] 0 2024-09-10T11:29:33.992 INFO:tasks.workunit.client.0.smithi144.stdout:8/192: rename d0/d9/c2c to d0/d9/d21/d25/c41 0 2024-09-10T11:29:33.993 INFO:tasks.workunit.client.1.smithi178.stdout:8/759: read d5/db/d10/d20/d52/d67/dbb/fc9 [392183,45299] 0 2024-09-10T11:29:33.998 INFO:tasks.workunit.client.0.smithi144.stdout:2/225: symlink d7/de/l4c 0 2024-09-10T11:29:33.999 INFO:tasks.workunit.client.1.smithi178.stdout:8/760: rename d5/d1d/d22/d76/fce to d5/db/d10/d53/fff 0 2024-09-10T11:29:34.000 INFO:tasks.workunit.client.1.smithi178.stdout:8/761: write d5/fae [782743,61351] 0 2024-09-10T11:29:34.000 INFO:tasks.workunit.client.0.smithi144.stdout:9/305: creat d11/d12/d1b/d22/d53/d64/f6b x:0 0 0 2024-09-10T11:29:34.001 INFO:tasks.workunit.client.1.smithi178.stdout:8/762: read d5/d1d/d22/f4e [4123532,60656] 0 2024-09-10T11:29:34.003 INFO:tasks.workunit.client.1.smithi178.stdout:8/763: read d5/f13 [831887,40521] 0 2024-09-10T11:29:34.004 INFO:tasks.workunit.client.0.smithi144.stdout:7/245: creat d5/dc/d12/f52 x:0 0 0 2024-09-10T11:29:34.009 INFO:tasks.workunit.client.1.smithi178.stdout:8/764: dwrite d5/db/f14 [0,4194304] 0 2024-09-10T11:29:34.009 INFO:tasks.workunit.client.0.smithi144.stdout:1/359: symlink d5/dd/d64/l6d 0 2024-09-10T11:29:34.010 INFO:tasks.workunit.client.0.smithi144.stdout:1/360: rename d5/df to d5/df/d6e 22 2024-09-10T11:29:34.011 INFO:tasks.workunit.client.1.smithi178.stdout:8/765: write d5/db/d1e/fa9 [354176,33637] 0 2024-09-10T11:29:34.012 INFO:tasks.workunit.client.1.smithi178.stdout:8/766: dread - d5/d61/fa2 zero size 2024-09-10T11:29:34.014 INFO:tasks.workunit.client.0.smithi144.stdout:0/200: sync 2024-09-10T11:29:34.016 INFO:tasks.workunit.client.0.smithi144.stdout:5/313: creat d4/d5/d32/f68 x:0 0 0 2024-09-10T11:29:34.017 INFO:tasks.workunit.client.0.smithi144.stdout:3/316: link d4/dd/d22/d1d/l3d d4/dd/d22/d17/d18/d34/d4b/d63/l6e 0 2024-09-10T11:29:34.017 INFO:tasks.workunit.client.1.smithi178.stdout:8/767: link d5/db/d1e/fa9 d5/db/d10/d35/f100 0 2024-09-10T11:29:34.019 INFO:tasks.workunit.client.0.smithi144.stdout:4/241: getdents d14/d1e/d22/d34/d3e/d40 0 2024-09-10T11:29:34.020 INFO:tasks.workunit.client.1.smithi178.stdout:8/768: symlink d5/d30/d33/l101 0 2024-09-10T11:29:34.020 INFO:tasks.workunit.client.0.smithi144.stdout:1/361: rename d5/f32 to d5/df/d4b/d5d/f6f 0 2024-09-10T11:29:34.020 INFO:tasks.workunit.client.0.smithi144.stdout:1/362: chown d5/l9 219405459 1 2024-09-10T11:29:34.021 INFO:tasks.workunit.client.1.smithi178.stdout:8/769: truncate d5/db/d10/d53/fff 1061287 0 2024-09-10T11:29:34.022 INFO:tasks.workunit.client.0.smithi144.stdout:5/314: read d4/d20/d24/d2d/d4b/f4e [815832,102400] 0 2024-09-10T11:29:34.022 INFO:tasks.workunit.client.0.smithi144.stdout:0/201: mknod da/c35 0 2024-09-10T11:29:34.023 INFO:tasks.workunit.client.0.smithi144.stdout:3/317: dread d4/dd/d21/f41 [0,4194304] 0 2024-09-10T11:29:34.024 INFO:tasks.workunit.client.0.smithi144.stdout:3/318: chown d4/dd/d21/d2f 28522767 1 2024-09-10T11:29:34.024 INFO:tasks.workunit.client.0.smithi144.stdout:3/319: write d4/dd/fe [830198,103345] 0 2024-09-10T11:29:34.025 INFO:tasks.workunit.client.0.smithi144.stdout:8/193: mknod d0/d9/d1f/d34/c42 0 2024-09-10T11:29:34.025 INFO:tasks.workunit.client.0.smithi144.stdout:4/242: fsync f9 0 2024-09-10T11:29:34.027 INFO:tasks.workunit.client.0.smithi144.stdout:1/363: rename d5/d4a/l26 to d5/d2a/d65/l70 0 2024-09-10T11:29:34.027 INFO:tasks.workunit.client.0.smithi144.stdout:2/226: link d7/c49 d7/de/d3d/c4d 0 2024-09-10T11:29:34.028 INFO:tasks.workunit.client.0.smithi144.stdout:9/306: link ca d11/d36/d3e/c6c 0 2024-09-10T11:29:34.030 INFO:tasks.workunit.client.0.smithi144.stdout:7/246: creat d5/dc/d12/f53 x:0 0 0 2024-09-10T11:29:34.031 INFO:tasks.workunit.client.0.smithi144.stdout:0/202: creat da/f36 x:0 0 0 2024-09-10T11:29:34.031 INFO:tasks.workunit.client.0.smithi144.stdout:0/203: fsync f5 0 2024-09-10T11:29:34.032 INFO:tasks.workunit.client.0.smithi144.stdout:5/315: fdatasync d4/d20/d24/d2d/f2e 0 2024-09-10T11:29:34.041 INFO:tasks.workunit.client.1.smithi178.stdout:9/675: sync 2024-09-10T11:29:34.043 INFO:tasks.workunit.client.1.smithi178.stdout:9/676: write d2/df/d31/d37/d3d/faf [176312,37610] 0 2024-09-10T11:29:34.046 INFO:tasks.workunit.client.1.smithi178.stdout:9/677: link d2/df/d31/fc0 d2/d8/fca 0 2024-09-10T11:29:34.048 INFO:tasks.workunit.client.1.smithi178.stdout:9/678: mknod d2/d13/d75/d60/d66/ccb 0 2024-09-10T11:29:34.048 INFO:tasks.workunit.client.1.smithi178.stdout:9/679: chown d2/d13/d40/d89/d93/d88/fa8 73 1 2024-09-10T11:29:34.054 INFO:tasks.workunit.client.1.smithi178.stdout:9/680: dwrite d2/d19/f39 [8388608,4194304] 0 2024-09-10T11:29:34.055 INFO:tasks.workunit.client.1.smithi178.stdout:9/681: chown d2/df/d31/l99 440 1 2024-09-10T11:29:34.060 INFO:tasks.workunit.client.1.smithi178.stdout:9/682: rename d2/d13/d40/d56/c85 to d2/d19/ccc 0 2024-09-10T11:29:34.062 INFO:tasks.workunit.client.1.smithi178.stdout:9/683: symlink d2/d13/d40/d89/d93/lcd 0 2024-09-10T11:29:34.067 INFO:tasks.workunit.client.1.smithi178.stdout:9/684: read d2/d8/f9 [3183713,71228] 0 2024-09-10T11:29:34.073 INFO:tasks.workunit.client.1.smithi178.stdout:9/685: dread d2/d8/fca [0,4194304] 0 2024-09-10T11:29:34.074 INFO:tasks.workunit.client.1.smithi178.stdout:9/686: fsync d2/d13/f3c 0 2024-09-10T11:29:34.075 INFO:tasks.workunit.client.1.smithi178.stdout:9/687: write d2/df/d31/f47 [2496531,68816] 0 2024-09-10T11:29:34.079 INFO:tasks.workunit.client.1.smithi178.stdout:9/688: truncate d2/df/d31/d37/f94 1625112 0 2024-09-10T11:29:34.081 INFO:tasks.workunit.client.1.smithi178.stdout:9/689: getdents d2/d8 0 2024-09-10T11:29:34.083 INFO:tasks.workunit.client.1.smithi178.stdout:9/690: symlink d2/d13/lce 0 2024-09-10T11:29:34.110 INFO:tasks.workunit.client.0.smithi144.stdout:7/247: dread f1 [0,4194304] 0 2024-09-10T11:29:34.110 INFO:tasks.workunit.client.0.smithi144.stdout:1/364: symlink d5/df/d4b/d5d/l71 0 2024-09-10T11:29:34.111 INFO:tasks.workunit.client.0.smithi144.stdout:0/204: rename da/c35 to da/c37 0 2024-09-10T11:29:34.112 INFO:tasks.workunit.client.0.smithi144.stdout:1/365: chown d5/df/d4b/d51/f59 20410 1 2024-09-10T11:29:34.115 INFO:tasks.workunit.client.0.smithi144.stdout:0/205: unlink da/ce 0 2024-09-10T11:29:34.118 INFO:tasks.workunit.client.0.smithi144.stdout:9/307: creat d11/d12/d1b/f6d x:0 0 0 2024-09-10T11:29:34.121 INFO:tasks.workunit.client.0.smithi144.stdout:7/248: dwrite d5/dc/d12/f53 [0,4194304] 0 2024-09-10T11:29:34.121 INFO:tasks.workunit.client.0.smithi144.stdout:2/227: dread d7/f11 [0,4194304] 0 2024-09-10T11:29:34.122 INFO:tasks.workunit.client.0.smithi144.stdout:8/194: link d0/d9/d1f/l1a d0/d9/d21/l43 0 2024-09-10T11:29:34.122 INFO:tasks.workunit.client.0.smithi144.stdout:9/308: write d11/f17 [3197344,130113] 0 2024-09-10T11:29:34.122 INFO:tasks.workunit.client.0.smithi144.stdout:5/316: dwrite d4/d5/d32/f5f [0,4194304] 0 2024-09-10T11:29:34.123 INFO:tasks.workunit.client.0.smithi144.stdout:1/366: rmdir d5/df 39 2024-09-10T11:29:34.126 INFO:tasks.workunit.client.0.smithi144.stdout:0/206: dread da/f2a [0,4194304] 0 2024-09-10T11:29:34.127 INFO:tasks.workunit.client.0.smithi144.stdout:8/195: unlink d0/c7 0 2024-09-10T11:29:34.132 INFO:tasks.workunit.client.0.smithi144.stdout:5/317: creat d4/d52/f69 x:0 0 0 2024-09-10T11:29:34.134 INFO:tasks.workunit.client.0.smithi144.stdout:0/207: mknod da/c38 0 2024-09-10T11:29:34.137 INFO:tasks.workunit.client.0.smithi144.stdout:1/367: write d5/df/f16 [4663424,80940] 0 2024-09-10T11:29:34.139 INFO:tasks.workunit.client.0.smithi144.stdout:5/318: creat d4/d5/d32/d47/f6a x:0 0 0 2024-09-10T11:29:34.142 INFO:tasks.workunit.client.0.smithi144.stdout:9/309: link d11/d12/d1b/d2c/l65 d11/d12/d1b/d41/l6e 0 2024-09-10T11:29:34.147 INFO:tasks.workunit.client.0.smithi144.stdout:7/249: dwrite d5/dc/f3c [0,4194304] 0 2024-09-10T11:29:34.147 INFO:tasks.workunit.client.0.smithi144.stdout:1/368: creat d5/d2a/d65/f72 x:0 0 0 2024-09-10T11:29:34.148 INFO:tasks.workunit.client.0.smithi144.stdout:4/243: dwrite d14/d1e/d22/d34/f3b [0,4194304] 0 2024-09-10T11:29:34.149 INFO:tasks.workunit.client.0.smithi144.stdout:2/228: dwrite d7/d29/f4b [0,4194304] 0 2024-09-10T11:29:34.150 INFO:tasks.workunit.client.0.smithi144.stdout:9/310: mkdir d11/d12/d6f 0 2024-09-10T11:29:34.153 INFO:tasks.workunit.client.0.smithi144.stdout:8/196: dread d0/d9/d21/d25/f29 [0,4194304] 0 2024-09-10T11:29:34.155 INFO:tasks.workunit.client.0.smithi144.stdout:1/369: rmdir d5/d4a/d63 39 2024-09-10T11:29:34.157 INFO:tasks.workunit.client.0.smithi144.stdout:0/208: dwrite da/f2e [0,4194304] 0 2024-09-10T11:29:34.158 INFO:tasks.workunit.client.0.smithi144.stdout:2/229: mkdir d7/de/d3d/d4e 0 2024-09-10T11:29:34.163 INFO:tasks.workunit.client.0.smithi144.stdout:9/311: dwrite d11/d12/d1b/f61 [0,4194304] 0 2024-09-10T11:29:34.164 INFO:tasks.workunit.client.0.smithi144.stdout:1/370: rename d5/dd/d22 to d5/d2a/d65/d73 0 2024-09-10T11:29:34.165 INFO:tasks.workunit.client.0.smithi144.stdout:0/209: creat da/f39 x:0 0 0 2024-09-10T11:29:34.165 INFO:tasks.workunit.client.0.smithi144.stdout:9/312: write d11/d36/f59 [164760,1315] 0 2024-09-10T11:29:34.167 INFO:tasks.workunit.client.0.smithi144.stdout:4/244: creat d14/f54 x:0 0 0 2024-09-10T11:29:34.168 INFO:tasks.workunit.client.0.smithi144.stdout:0/210: chown da/c38 36263253 1 2024-09-10T11:29:34.170 INFO:tasks.workunit.client.0.smithi144.stdout:9/313: write d11/d36/f4e [431156,42738] 0 2024-09-10T11:29:34.170 INFO:tasks.workunit.client.0.smithi144.stdout:7/250: link d5/dc/d11/c30 d5/dc/d2a/d41/c54 0 2024-09-10T11:29:34.171 INFO:tasks.workunit.client.0.smithi144.stdout:8/197: dwrite d0/d9/f1d [0,4194304] 0 2024-09-10T11:29:34.172 INFO:tasks.workunit.client.0.smithi144.stdout:1/371: chown d5/c11 4485145 1 2024-09-10T11:29:34.188 INFO:tasks.workunit.client.0.smithi144.stdout:0/211: mkdir da/d3a 0 2024-09-10T11:29:34.190 INFO:tasks.workunit.client.0.smithi144.stdout:6/211: sync 2024-09-10T11:29:34.190 INFO:tasks.workunit.client.0.smithi144.stdout:0/212: read - da/f30 zero size 2024-09-10T11:29:34.190 INFO:tasks.workunit.client.0.smithi144.stdout:5/319: dwrite d4/d20/d24/d2d/d4b/f4e [4194304,4194304] 0 2024-09-10T11:29:34.191 INFO:tasks.workunit.client.0.smithi144.stdout:2/230: creat d7/de/d36/f4f x:0 0 0 2024-09-10T11:29:34.192 INFO:tasks.workunit.client.0.smithi144.stdout:9/314: rename d11/d12/d1b/d22/d53/d64/f6b to d11/d36/d2d/f70 0 2024-09-10T11:29:34.192 INFO:tasks.workunit.client.0.smithi144.stdout:8/198: stat d0/d9/d23/d26/f36 0 2024-09-10T11:29:34.194 INFO:tasks.workunit.client.0.smithi144.stdout:7/251: fdatasync d5/dc/d11/d18/f24 0 2024-09-10T11:29:34.195 INFO:tasks.workunit.client.0.smithi144.stdout:2/231: write d7/da/d1e/f25 [1646130,98002] 0 2024-09-10T11:29:34.211 INFO:tasks.workunit.client.1.smithi178.stdout:4/548: sync 2024-09-10T11:29:34.211 INFO:tasks.workunit.client.1.smithi178.stdout:0/623: sync 2024-09-10T11:29:34.213 INFO:tasks.workunit.client.1.smithi178.stdout:4/549: mkdir d0/d9/d3d/dc2 0 2024-09-10T11:29:34.213 INFO:tasks.workunit.client.1.smithi178.stdout:4/550: dread - d0/d8/da/f90 zero size 2024-09-10T11:29:34.214 INFO:tasks.workunit.client.1.smithi178.stdout:0/624: creat d0/dc/d39/d50/fc3 x:0 0 0 2024-09-10T11:29:34.218 INFO:tasks.workunit.client.1.smithi178.stdout:0/625: rename d0/dc/d39/d4c/d43 to d0/dc/dc4 0 2024-09-10T11:29:34.219 INFO:tasks.workunit.client.1.smithi178.stdout:4/551: dwrite d0/d8/da/f6b [0,4194304] 0 2024-09-10T11:29:34.221 INFO:tasks.workunit.client.1.smithi178.stdout:0/626: mknod d0/dc/d39/cc5 0 2024-09-10T11:29:34.222 INFO:tasks.workunit.client.1.smithi178.stdout:0/627: fdatasync d0/dc/d1f/fad 0 2024-09-10T11:29:34.225 INFO:tasks.workunit.client.1.smithi178.stdout:0/628: symlink d0/d2c/d89/lc6 0 2024-09-10T11:29:34.225 INFO:tasks.workunit.client.1.smithi178.stdout:0/629: readlink d0/d2c/d33/l8f 0 2024-09-10T11:29:34.227 INFO:tasks.workunit.client.1.smithi178.stdout:0/630: creat d0/dc/dc4/d90/fc7 x:0 0 0 2024-09-10T11:29:34.229 INFO:tasks.workunit.client.1.smithi178.stdout:0/631: write d0/dc/dc4/d8c/f98 [1681288,86640] 0 2024-09-10T11:29:34.230 INFO:tasks.workunit.client.1.smithi178.stdout:0/632: symlink d0/dc/d39/d7c/lc8 0 2024-09-10T11:29:34.232 INFO:tasks.workunit.client.1.smithi178.stdout:0/633: getdents d0/d12 0 2024-09-10T11:29:34.233 INFO:tasks.workunit.client.1.smithi178.stdout:0/634: chown d0/dc/d39/d4c/f15 2670436 1 2024-09-10T11:29:34.234 INFO:tasks.workunit.client.1.smithi178.stdout:0/635: creat d0/d12/d3f/fc9 x:0 0 0 2024-09-10T11:29:34.242 INFO:tasks.workunit.client.1.smithi178.stdout:0/636: write d0/d2c/f7b [998569,27513] 0 2024-09-10T11:29:34.243 INFO:tasks.workunit.client.1.smithi178.stdout:0/637: creat d0/dc/d39/d50/d5e/fca x:0 0 0 2024-09-10T11:29:34.245 INFO:tasks.workunit.client.1.smithi178.stdout:0/638: creat d0/d2c/d95/dbd/fcb x:0 0 0 2024-09-10T11:29:34.246 INFO:tasks.workunit.client.1.smithi178.stdout:0/639: mknod d0/db1/ccc 0 2024-09-10T11:29:34.260 INFO:tasks.workunit.client.1.smithi178.stdout:0/640: dread d0/dc/f21 [0,4194304] 0 2024-09-10T11:29:34.260 INFO:tasks.workunit.client.1.smithi178.stdout:0/641: stat d0/d2c/d93 0 2024-09-10T11:29:34.270 INFO:tasks.workunit.client.1.smithi178.stdout:0/642: dread d0/dc/ff [0,4194304] 0 2024-09-10T11:29:34.279 INFO:tasks.workunit.client.1.smithi178.stdout:0/643: dread d0/d2c/f37 [0,4194304] 0 2024-09-10T11:29:34.282 INFO:tasks.workunit.client.1.smithi178.stdout:0/644: getdents d0/d2c/d33 0 2024-09-10T11:29:34.285 INFO:tasks.workunit.client.1.smithi178.stdout:0/645: write d0/dc/f2b [3762645,33854] 0 2024-09-10T11:29:34.287 INFO:tasks.workunit.client.1.smithi178.stdout:0/646: creat d0/dc/d39/d50/d5e/fcd x:0 0 0 2024-09-10T11:29:34.288 INFO:tasks.workunit.client.1.smithi178.stdout:0/647: stat d0/dc/d39/d50/l3c 0 2024-09-10T11:29:34.288 INFO:tasks.workunit.client.1.smithi178.stdout:0/648: read - d0/d12/d3f/fa5 zero size 2024-09-10T11:29:34.317 INFO:tasks.workunit.client.0.smithi144.stdout:5/320: write d4/d5/d18/d67/d59/f5e [938720,9170] 0 2024-09-10T11:29:34.318 INFO:tasks.workunit.client.0.smithi144.stdout:6/212: symlink d9/d1f/d2c/d4f/l52 0 2024-09-10T11:29:34.319 INFO:tasks.workunit.client.0.smithi144.stdout:1/372: dwrite d5/f12 [0,4194304] 0 2024-09-10T11:29:34.320 INFO:tasks.workunit.client.0.smithi144.stdout:8/199: symlink d0/d9/d23/l44 0 2024-09-10T11:29:34.320 INFO:tasks.workunit.client.0.smithi144.stdout:4/245: write d14/d1e/d22/d34/d3e/f3f [181699,62773] 0 2024-09-10T11:29:34.323 INFO:tasks.workunit.client.0.smithi144.stdout:9/315: creat d11/d12/d1b/f71 x:0 0 0 2024-09-10T11:29:34.326 INFO:tasks.workunit.client.0.smithi144.stdout:2/232: dread d7/f11 [0,4194304] 0 2024-09-10T11:29:34.326 INFO:tasks.workunit.client.0.smithi144.stdout:2/233: stat d7/de/d3d/d4e 0 2024-09-10T11:29:34.327 INFO:tasks.workunit.client.0.smithi144.stdout:6/213: rename d9/dd/d11/f2d to d9/d1f/d2c/f53 0 2024-09-10T11:29:34.327 INFO:tasks.workunit.client.0.smithi144.stdout:7/252: getdents d5/dc/d11/d18 0 2024-09-10T11:29:34.328 INFO:tasks.workunit.client.0.smithi144.stdout:4/246: fsync d14/f3a 0 2024-09-10T11:29:34.329 INFO:tasks.workunit.client.0.smithi144.stdout:4/247: chown d14/d1e/d22/d2b 388365977 1 2024-09-10T11:29:34.329 INFO:tasks.workunit.client.0.smithi144.stdout:9/316: symlink d11/d36/d2d/l72 0 2024-09-10T11:29:34.330 INFO:tasks.workunit.client.0.smithi144.stdout:1/373: creat d5/d2a/f74 x:0 0 0 2024-09-10T11:29:34.332 INFO:tasks.workunit.client.0.smithi144.stdout:1/374: stat d5/df/d4b/d51/f50 0 2024-09-10T11:29:34.333 INFO:tasks.workunit.client.0.smithi144.stdout:7/253: creat d5/dc/d2a/f55 x:0 0 0 2024-09-10T11:29:34.334 INFO:tasks.workunit.client.0.smithi144.stdout:4/248: mknod d14/d1e/d22/d34/c55 0 2024-09-10T11:29:34.335 INFO:tasks.workunit.client.0.smithi144.stdout:7/254: stat d5/dc/d2a/c51 0 2024-09-10T11:29:34.335 INFO:tasks.workunit.client.0.smithi144.stdout:1/375: creat d5/d4a/d44/f75 x:0 0 0 2024-09-10T11:29:34.335 INFO:tasks.workunit.client.0.smithi144.stdout:2/234: dwrite d7/da/d1e/f25 [4194304,4194304] 0 2024-09-10T11:29:34.336 INFO:tasks.workunit.client.0.smithi144.stdout:1/376: fdatasync d5/d2a/f46 0 2024-09-10T11:29:34.338 INFO:tasks.workunit.client.0.smithi144.stdout:9/317: rename d11/d36/l4b to d11/d12/d1b/d22/d53/d64/l73 0 2024-09-10T11:29:34.341 INFO:tasks.workunit.client.0.smithi144.stdout:5/321: truncate d4/d5/f28 2057977 0 2024-09-10T11:29:34.342 INFO:tasks.workunit.client.0.smithi144.stdout:7/255: unlink d5/dc/d12/f2c 0 2024-09-10T11:29:34.343 INFO:tasks.workunit.client.0.smithi144.stdout:4/249: rmdir d14/d1e/d22/d2b/d35 39 2024-09-10T11:29:34.345 INFO:tasks.workunit.client.0.smithi144.stdout:9/318: creat d11/d12/d1b/d22/f74 x:0 0 0 2024-09-10T11:29:34.346 INFO:tasks.workunit.client.0.smithi144.stdout:7/256: write d5/f7 [327702,42421] 0 2024-09-10T11:29:34.349 INFO:tasks.workunit.client.0.smithi144.stdout:9/319: write d11/f2b [5450663,1022] 0 2024-09-10T11:29:34.349 INFO:tasks.workunit.client.0.smithi144.stdout:5/322: truncate d4/d5/d18/d67/f62 294398 0 2024-09-10T11:29:34.349 INFO:tasks.workunit.client.0.smithi144.stdout:2/235: dwrite d7/da/d1e/d21/f30 [0,4194304] 0 2024-09-10T11:29:34.352 INFO:tasks.workunit.client.0.smithi144.stdout:2/236: fdatasync d7/d29/f46 0 2024-09-10T11:29:34.352 INFO:tasks.workunit.client.0.smithi144.stdout:1/377: dwrite d5/df/d4b/f54 [0,4194304] 0 2024-09-10T11:29:34.360 INFO:tasks.workunit.client.1.smithi178.stdout:3/571: sync 2024-09-10T11:29:34.361 INFO:tasks.workunit.client.1.smithi178.stdout:3/572: unlink db/f97 0 2024-09-10T11:29:34.362 INFO:tasks.workunit.client.0.smithi144.stdout:6/214: dread d9/dd/f49 [0,4194304] 0 2024-09-10T11:29:34.362 INFO:tasks.workunit.client.0.smithi144.stdout:6/215: readlink d9/d1f/d2c/l44 0 2024-09-10T11:29:34.362 INFO:tasks.workunit.client.0.smithi144.stdout:6/216: write d9/dd/d3e/d47/f2e [334678,5100] 0 2024-09-10T11:29:34.362 INFO:tasks.workunit.client.0.smithi144.stdout:6/217: stat d9/dd/d3e/d47/f16 0 2024-09-10T11:29:34.362 INFO:tasks.workunit.client.0.smithi144.stdout:6/218: write d9/dd/d3e/d47/d13/f1e [362640,7817] 0 2024-09-10T11:29:34.363 INFO:tasks.workunit.client.0.smithi144.stdout:2/237: rename d7/f26 to d7/de/d3d/f50 0 2024-09-10T11:29:34.363 INFO:tasks.workunit.client.1.smithi178.stdout:3/573: write db/d20/d3d/d56/faf [610069,82666] 0 2024-09-10T11:29:34.363 INFO:tasks.workunit.client.0.smithi144.stdout:2/238: write d7/f8 [4212572,129459] 0 2024-09-10T11:29:34.364 INFO:tasks.workunit.client.0.smithi144.stdout:7/257: creat d5/dc/f56 x:0 0 0 2024-09-10T11:29:34.367 INFO:tasks.workunit.client.1.smithi178.stdout:3/574: readlink db/d20/d3d/d63/d6a/lb0 0 2024-09-10T11:29:34.369 INFO:tasks.workunit.client.1.smithi178.stdout:3/575: chown db/d20/d3d/f53 17495 1 2024-09-10T11:29:34.371 INFO:tasks.workunit.client.0.smithi144.stdout:7/258: creat d5/dc/d14/d40/f57 x:0 0 0 2024-09-10T11:29:34.371 INFO:tasks.workunit.client.0.smithi144.stdout:4/250: link d14/l17 d14/d1e/d22/d2b/d35/d4f/l56 0 2024-09-10T11:29:34.372 INFO:tasks.workunit.client.0.smithi144.stdout:1/378: dwrite d5/df/d4b/d5d/f6f [0,4194304] 0 2024-09-10T11:29:34.375 INFO:tasks.workunit.client.0.smithi144.stdout:6/219: mknod d9/dd/d11/c54 0 2024-09-10T11:29:34.375 INFO:tasks.workunit.client.0.smithi144.stdout:6/220: readlink d9/d1f/d2c/l3d 0 2024-09-10T11:29:34.376 INFO:tasks.workunit.client.0.smithi144.stdout:5/323: truncate d4/f55 2711946 0 2024-09-10T11:29:34.377 INFO:tasks.workunit.client.1.smithi178.stdout:3/576: dwrite db/d20/d3d/d63/d6a/f6c [0,4194304] 0 2024-09-10T11:29:34.378 INFO:tasks.workunit.client.0.smithi144.stdout:5/324: fdatasync d4/d5/f25 0 2024-09-10T11:29:34.378 INFO:tasks.workunit.client.0.smithi144.stdout:9/320: link d11/l62 d11/d12/d1b/d41/l75 0 2024-09-10T11:29:34.379 INFO:tasks.workunit.client.0.smithi144.stdout:7/259: mkdir d5/dc/d14/d40/d58 0 2024-09-10T11:29:34.382 INFO:tasks.workunit.client.0.smithi144.stdout:1/379: creat d5/df/d4b/d5d/f76 x:0 0 0 2024-09-10T11:29:34.384 INFO:tasks.workunit.client.0.smithi144.stdout:6/221: mknod d9/dd/d3a/c55 0 2024-09-10T11:29:34.388 INFO:tasks.workunit.client.0.smithi144.stdout:2/239: dread d7/de/f28 [0,4194304] 0 2024-09-10T11:29:34.389 INFO:tasks.workunit.client.0.smithi144.stdout:7/260: dwrite d5/f38 [0,4194304] 0 2024-09-10T11:29:34.390 INFO:tasks.workunit.client.0.smithi144.stdout:9/321: creat d11/d36/d2d/d38/f76 x:0 0 0 2024-09-10T11:29:34.390 INFO:tasks.workunit.client.0.smithi144.stdout:2/240: stat d7/de/d36/f4f 0 2024-09-10T11:29:34.391 INFO:tasks.workunit.client.0.smithi144.stdout:9/322: stat d11/d12/d1b/l6a 0 2024-09-10T11:29:34.392 INFO:tasks.workunit.client.1.smithi178.stdout:3/577: dread db/d35/f37 [0,4194304] 0 2024-09-10T11:29:34.393 INFO:tasks.workunit.client.0.smithi144.stdout:1/380: mknod d5/dd/c77 0 2024-09-10T11:29:34.393 INFO:tasks.workunit.client.0.smithi144.stdout:7/261: symlink d5/dc/d14/l59 0 2024-09-10T11:29:34.395 INFO:tasks.workunit.client.0.smithi144.stdout:2/241: chown d7/c49 4464267 1 2024-09-10T11:29:34.396 INFO:tasks.workunit.client.0.smithi144.stdout:7/262: write d5/dc/d12/f22 [276617,92876] 0 2024-09-10T11:29:34.397 INFO:tasks.workunit.client.0.smithi144.stdout:5/325: creat d4/f6b x:0 0 0 2024-09-10T11:29:34.399 INFO:tasks.workunit.client.0.smithi144.stdout:4/251: dread d14/f3a [0,4194304] 0 2024-09-10T11:29:34.400 INFO:tasks.workunit.client.0.smithi144.stdout:5/326: write d4/d5/f25 [4887135,128363] 0 2024-09-10T11:29:34.400 INFO:tasks.workunit.client.0.smithi144.stdout:7/263: write d5/dc/d12/d2f/f4d [216651,43504] 0 2024-09-10T11:29:34.403 INFO:tasks.workunit.client.0.smithi144.stdout:6/222: creat d9/dd/d11/f56 x:0 0 0 2024-09-10T11:29:34.404 INFO:tasks.workunit.client.0.smithi144.stdout:9/323: creat d11/d12/d1b/f77 x:0 0 0 2024-09-10T11:29:34.405 INFO:tasks.workunit.client.1.smithi178.stdout:3/578: write db/d20/f34 [3107093,118232] 0 2024-09-10T11:29:34.405 INFO:tasks.workunit.client.1.smithi178.stdout:3/579: readlink la 0 2024-09-10T11:29:34.405 INFO:tasks.workunit.client.1.smithi178.stdout:3/580: chown db/d20/d3d 990 1 2024-09-10T11:29:34.407 INFO:tasks.workunit.client.0.smithi144.stdout:1/381: link d5/c28 d5/dd/d64/c78 0 2024-09-10T11:29:34.407 INFO:tasks.workunit.client.1.smithi178.stdout:3/581: symlink db/d9b/lb4 0 2024-09-10T11:29:34.408 INFO:tasks.workunit.client.0.smithi144.stdout:4/252: creat d14/d1e/d22/d34/d3e/d40/d4b/f57 x:0 0 0 2024-09-10T11:29:34.408 INFO:tasks.workunit.client.0.smithi144.stdout:1/382: write d5/df/d4b/d5d/f6f [3766047,128862] 0 2024-09-10T11:29:34.410 INFO:tasks.workunit.client.1.smithi178.stdout:3/582: symlink db/d20/d3d/d56/d64/lb5 0 2024-09-10T11:29:34.410 INFO:tasks.workunit.client.0.smithi144.stdout:5/327: rename d4/f15 to d4/d5/d18/d67/d59/f6c 0 2024-09-10T11:29:34.412 INFO:tasks.workunit.client.0.smithi144.stdout:5/328: write d4/d20/d24/f3e [1479065,64236] 0 2024-09-10T11:29:34.412 INFO:tasks.workunit.client.1.smithi178.stdout:3/583: mkdir db/d26/d40/d68/db6 0 2024-09-10T11:29:34.415 INFO:tasks.workunit.client.1.smithi178.stdout:3/584: symlink db/d35/d3a/d9f/lb7 0 2024-09-10T11:29:34.415 INFO:tasks.workunit.client.0.smithi144.stdout:5/329: write d4/d5/f23 [3189682,96118] 0 2024-09-10T11:29:34.415 INFO:tasks.workunit.client.1.smithi178.stdout:6/643: sync 2024-09-10T11:29:34.416 INFO:tasks.workunit.client.0.smithi144.stdout:3/320: sync 2024-09-10T11:29:34.416 INFO:tasks.workunit.client.0.smithi144.stdout:0/213: sync 2024-09-10T11:29:34.417 INFO:tasks.workunit.client.0.smithi144.stdout:1/383: mkdir d5/dd/d64/d79 0 2024-09-10T11:29:34.417 INFO:tasks.workunit.client.0.smithi144.stdout:4/253: dwrite d14/d1e/d22/d2b/f33 [0,4194304] 0 2024-09-10T11:29:34.418 INFO:tasks.workunit.client.1.smithi178.stdout:3/585: getdents db/d35/d3a 0 2024-09-10T11:29:34.419 INFO:tasks.workunit.client.0.smithi144.stdout:2/242: truncate d7/f11 2576614 0 2024-09-10T11:29:34.421 INFO:tasks.workunit.client.0.smithi144.stdout:9/324: mkdir d11/d12/d1b/d2c/d78 0 2024-09-10T11:29:34.422 INFO:tasks.workunit.client.0.smithi144.stdout:2/243: chown d7/de/f2f 223726 1 2024-09-10T11:29:34.423 INFO:tasks.workunit.client.1.smithi178.stdout:6/644: dwrite d3/d7/d8/d15/f22 [0,4194304] 0 2024-09-10T11:29:34.425 INFO:tasks.workunit.client.0.smithi144.stdout:3/321: write d4/dd/d22/d17/d18/d34/f5c [1060666,87739] 0 2024-09-10T11:29:34.426 INFO:tasks.workunit.client.0.smithi144.stdout:5/330: mkdir d4/d5/d18/d38/d6d 0 2024-09-10T11:29:34.427 INFO:tasks.workunit.client.0.smithi144.stdout:0/214: dread da/f2b [0,4194304] 0 2024-09-10T11:29:34.427 INFO:tasks.workunit.client.1.smithi178.stdout:6/645: symlink d3/d74/d91/daa/dc2/leb 0 2024-09-10T11:29:34.428 INFO:tasks.workunit.client.0.smithi144.stdout:6/223: write d9/dd/d11/f21 [576229,66484] 0 2024-09-10T11:29:34.433 INFO:tasks.workunit.client.1.smithi178.stdout:6/646: mknod d3/d7/d8/d15/d1c/d2a/d18/d8e/d90/cec 0 2024-09-10T11:29:34.433 INFO:tasks.workunit.client.0.smithi144.stdout:4/254: write f5 [1241901,17878] 0 2024-09-10T11:29:34.433 INFO:tasks.workunit.client.0.smithi144.stdout:7/264: dwrite d5/dc/d11/d18/f24 [0,4194304] 0 2024-09-10T11:29:34.435 INFO:tasks.workunit.client.1.smithi178.stdout:6/647: rmdir d3/d7/d8/d56 39 2024-09-10T11:29:34.435 INFO:tasks.workunit.client.0.smithi144.stdout:9/325: creat d11/d36/f79 x:0 0 0 2024-09-10T11:29:34.436 INFO:tasks.workunit.client.0.smithi144.stdout:4/255: fsync d14/d1e/d22/d2b/f2e 0 2024-09-10T11:29:34.438 INFO:tasks.workunit.client.0.smithi144.stdout:6/224: write d9/d1f/d2c/f53 [859675,41917] 0 2024-09-10T11:29:34.439 INFO:tasks.workunit.client.1.smithi178.stdout:6/648: creat d3/d7/d8/d56/fed x:0 0 0 2024-09-10T11:29:34.439 INFO:tasks.workunit.client.0.smithi144.stdout:5/331: symlink d4/d52/l6e 0 2024-09-10T11:29:34.442 INFO:tasks.workunit.client.0.smithi144.stdout:3/322: dread d4/dd/d22/d1d/f29 [0,4194304] 0 2024-09-10T11:29:34.442 INFO:tasks.workunit.client.0.smithi144.stdout:0/215: creat da/d3a/f3b x:0 0 0 2024-09-10T11:29:34.443 INFO:tasks.workunit.client.1.smithi178.stdout:6/649: unlink d3/d7/d8/d15/d1c/d2a/f2c 0 2024-09-10T11:29:34.444 INFO:tasks.workunit.client.0.smithi144.stdout:6/225: dread - d9/dd/d3a/f51 zero size 2024-09-10T11:29:34.446 INFO:tasks.workunit.client.0.smithi144.stdout:9/326: creat d11/d12/d1b/d22/f7a x:0 0 0 2024-09-10T11:29:34.447 INFO:tasks.workunit.client.1.smithi178.stdout:6/650: mkdir d3/d74/d91/dee 0 2024-09-10T11:29:34.447 INFO:tasks.workunit.client.0.smithi144.stdout:4/256: dread d14/d1e/d22/f48 [0,4194304] 0 2024-09-10T11:29:34.448 INFO:tasks.workunit.client.0.smithi144.stdout:0/216: symlink da/l3c 0 2024-09-10T11:29:34.449 INFO:tasks.workunit.client.1.smithi178.stdout:6/651: symlink d3/d7/d8/d56/de1/lef 0 2024-09-10T11:29:34.449 INFO:tasks.workunit.client.0.smithi144.stdout:4/257: fsync d14/d1e/f2f 0 2024-09-10T11:29:34.451 INFO:tasks.workunit.client.0.smithi144.stdout:6/226: write d9/dd/f1d [1167877,130460] 0 2024-09-10T11:29:34.452 INFO:tasks.workunit.client.1.smithi178.stdout:3/586: dread db/d26/d74/f90 [0,4194304] 0 2024-09-10T11:29:34.452 INFO:tasks.workunit.client.0.smithi144.stdout:7/265: creat d5/f5a x:0 0 0 2024-09-10T11:29:34.453 INFO:tasks.workunit.client.0.smithi144.stdout:9/327: mkdir d11/d12/d1b/d7b 0 2024-09-10T11:29:34.455 INFO:tasks.workunit.client.1.smithi178.stdout:3/587: creat db/d35/d3a/d9f/fb8 x:0 0 0 2024-09-10T11:29:34.456 INFO:tasks.workunit.client.0.smithi144.stdout:6/227: fsync d9/f2a 0 2024-09-10T11:29:34.456 INFO:tasks.workunit.client.0.smithi144.stdout:3/323: dwrite d4/d1a/f3b [0,4194304] 0 2024-09-10T11:29:34.456 INFO:tasks.workunit.client.0.smithi144.stdout:0/217: rename da/l3c to da/d3a/l3d 0 2024-09-10T11:29:34.456 INFO:tasks.workunit.client.1.smithi178.stdout:3/588: write db/d20/d3d/d63/d6a/f9d [2418268,100868] 0 2024-09-10T11:29:34.457 INFO:tasks.workunit.client.0.smithi144.stdout:7/266: fsync d5/dc/d11/f17 0 2024-09-10T11:29:34.458 INFO:tasks.workunit.client.0.smithi144.stdout:1/384: write d5/dd/f13 [1779510,11772] 0 2024-09-10T11:29:34.459 INFO:tasks.workunit.client.1.smithi178.stdout:3/589: rename db/d36/f65 to db/d20/fb9 0 2024-09-10T11:29:34.460 INFO:tasks.workunit.client.0.smithi144.stdout:9/328: write d11/d12/d1b/f61 [1060026,5675] 0 2024-09-10T11:29:34.461 INFO:tasks.workunit.client.0.smithi144.stdout:2/244: truncate d7/da/f12 743557 0 2024-09-10T11:29:34.461 INFO:tasks.workunit.client.0.smithi144.stdout:1/385: chown d5/d2a/c30 48321 1 2024-09-10T11:29:34.463 INFO:tasks.workunit.client.1.smithi178.stdout:2/860: sync 2024-09-10T11:29:34.463 INFO:tasks.workunit.client.1.smithi178.stdout:5/693: sync 2024-09-10T11:29:34.463 INFO:tasks.workunit.client.1.smithi178.stdout:0/649: sync 2024-09-10T11:29:34.463 INFO:tasks.workunit.client.1.smithi178.stdout:1/588: sync 2024-09-10T11:29:34.464 INFO:tasks.workunit.client.1.smithi178.stdout:7/622: sync 2024-09-10T11:29:34.464 INFO:tasks.workunit.client.1.smithi178.stdout:4/552: sync 2024-09-10T11:29:34.464 INFO:tasks.workunit.client.1.smithi178.stdout:8/770: sync 2024-09-10T11:29:34.464 INFO:tasks.workunit.client.1.smithi178.stdout:9/691: sync 2024-09-10T11:29:34.464 INFO:tasks.workunit.client.0.smithi144.stdout:6/228: read d9/dd/d3e/d47/f25 [495404,107710] 0 2024-09-10T11:29:34.464 INFO:tasks.workunit.client.0.smithi144.stdout:5/332: dwrite d4/d20/d24/f63 [0,4194304] 0 2024-09-10T11:29:34.465 INFO:tasks.workunit.client.0.smithi144.stdout:7/267: read d5/dc/d11/f4c [51025,79533] 0 2024-09-10T11:29:34.465 INFO:tasks.workunit.client.1.smithi178.stdout:9/692: readlink d2/df/l64 0 2024-09-10T11:29:34.465 INFO:tasks.workunit.client.1.smithi178.stdout:3/590: unlink db/d35/d3a/d42/l59 0 2024-09-10T11:29:34.466 INFO:tasks.workunit.client.0.smithi144.stdout:3/324: creat d4/dd/f6f x:0 0 0 2024-09-10T11:29:34.467 INFO:tasks.workunit.client.0.smithi144.stdout:0/218: rmdir da 39 2024-09-10T11:29:34.467 INFO:tasks.workunit.client.1.smithi178.stdout:4/553: chown d0/d8/da/f90 219021713 1 2024-09-10T11:29:34.468 INFO:tasks.workunit.client.1.smithi178.stdout:8/771: rmdir d5/d1d 39 2024-09-10T11:29:34.468 INFO:tasks.workunit.client.1.smithi178.stdout:7/623: chown d6/d10/d1d/d4a/f62 4 1 2024-09-10T11:29:34.470 INFO:tasks.workunit.client.1.smithi178.stdout:9/693: chown d2/d19/d1c/d2d/c8a 414645 1 2024-09-10T11:29:34.471 INFO:tasks.workunit.client.1.smithi178.stdout:0/650: write d0/d2c/d95/dbd/fcb [768475,3272] 0 2024-09-10T11:29:34.471 INFO:tasks.workunit.client.1.smithi178.stdout:3/591: write db/d20/d70/f8e [735472,95025] 0 2024-09-10T11:29:34.474 INFO:tasks.workunit.client.0.smithi144.stdout:9/329: dread d11/d12/f26 [0,4194304] 0 2024-09-10T11:29:34.474 INFO:tasks.workunit.client.0.smithi144.stdout:4/258: creat d14/d1e/d22/d34/d3e/d40/d4b/d4c/f58 x:0 0 0 2024-09-10T11:29:34.475 INFO:tasks.workunit.client.1.smithi178.stdout:9/694: creat d2/dc/fcf x:0 0 0 2024-09-10T11:29:34.477 INFO:tasks.workunit.client.0.smithi144.stdout:7/268: symlink d5/dc/d2a/d41/l5b 0 2024-09-10T11:29:34.478 INFO:tasks.workunit.client.1.smithi178.stdout:1/589: truncate d3/d9/d98/dac/d3a/d50/f9b 1046901 0 2024-09-10T11:29:34.478 INFO:tasks.workunit.client.0.smithi144.stdout:5/333: dwrite d4/d5/f23 [0,4194304] 0 2024-09-10T11:29:34.479 INFO:tasks.workunit.client.1.smithi178.stdout:3/592: stat db/l2d 0 2024-09-10T11:29:34.479 INFO:tasks.workunit.client.1.smithi178.stdout:5/694: truncate d9/dc/dd/d37/d42/d55/d6d/fe1 981750 0 2024-09-10T11:29:34.479 INFO:tasks.workunit.client.1.smithi178.stdout:4/554: fdatasync d0/d8/df/f49 0 2024-09-10T11:29:34.479 INFO:tasks.workunit.client.0.smithi144.stdout:3/325: symlink d4/dd/l70 0 2024-09-10T11:29:34.479 INFO:tasks.workunit.client.0.smithi144.stdout:4/259: write d14/d1e/f50 [994146,119105] 0 2024-09-10T11:29:34.480 INFO:tasks.workunit.client.0.smithi144.stdout:9/330: write d11/d12/d1b/f61 [2792086,82113] 0 2024-09-10T11:29:34.481 INFO:tasks.workunit.client.1.smithi178.stdout:8/772: symlink d5/d1d/d22/d76/l102 0 2024-09-10T11:29:34.482 INFO:tasks.workunit.client.0.smithi144.stdout:8/200: sync 2024-09-10T11:29:34.483 INFO:tasks.workunit.client.0.smithi144.stdout:1/386: dwrite d5/fc [0,4194304] 0 2024-09-10T11:29:34.484 INFO:tasks.workunit.client.0.smithi144.stdout:1/387: fdatasync d5/dd/f48 0 2024-09-10T11:29:34.484 INFO:tasks.workunit.client.0.smithi144.stdout:5/334: chown d4/d5/d32/l39 228 1 2024-09-10T11:29:34.485 INFO:tasks.workunit.client.1.smithi178.stdout:6/652: dread d3/d74/d91/daa/dc2/f66 [0,4194304] 0 2024-09-10T11:29:34.485 INFO:tasks.workunit.client.1.smithi178.stdout:7/624: mknod d6/d10/d34/d6a/d6f/ce7 0 2024-09-10T11:29:34.485 INFO:tasks.workunit.client.0.smithi144.stdout:0/219: unlink da/c23 0 2024-09-10T11:29:34.485 INFO:tasks.workunit.client.0.smithi144.stdout:1/388: chown d5/d2a 0 1 2024-09-10T11:29:34.486 INFO:tasks.workunit.client.1.smithi178.stdout:2/861: dwrite d1/d41/d5d/f116 [0,4194304] 0 2024-09-10T11:29:34.487 INFO:tasks.workunit.client.0.smithi144.stdout:6/229: creat d9/dd/d3e/d47/f57 x:0 0 0 2024-09-10T11:29:34.487 INFO:tasks.workunit.client.1.smithi178.stdout:0/651: mkdir d0/dc/d39/da4/dce 0 2024-09-10T11:29:34.488 INFO:tasks.workunit.client.0.smithi144.stdout:7/269: fdatasync d5/dc/d2a/f35 0 2024-09-10T11:29:34.489 INFO:tasks.workunit.client.1.smithi178.stdout:9/695: mkdir d2/d13/d75/d60/dd0 0 2024-09-10T11:29:34.491 INFO:tasks.workunit.client.0.smithi144.stdout:3/326: symlink d4/dd/l71 0 2024-09-10T11:29:34.495 INFO:tasks.workunit.client.0.smithi144.stdout:8/201: mknod d0/d9/d23/d26/c45 0 2024-09-10T11:29:34.498 INFO:tasks.workunit.client.0.smithi144.stdout:4/260: rename d14/d1e/d22/d2b/d35/l49 to d14/d1e/d22/d2b/d35/d3c/l59 0 2024-09-10T11:29:34.498 INFO:tasks.workunit.client.0.smithi144.stdout:9/331: dwrite d11/d12/d1b/f27 [0,4194304] 0 2024-09-10T11:29:34.500 INFO:tasks.workunit.client.1.smithi178.stdout:3/593: dwrite db/f27 [0,4194304] 0 2024-09-10T11:29:34.502 INFO:tasks.workunit.client.1.smithi178.stdout:2/862: chown d1/d7/d24/d46/d88/d9d/dab/d5e/fdc 432 1 2024-09-10T11:29:34.503 INFO:tasks.workunit.client.0.smithi144.stdout:0/220: creat da/d3a/f3e x:0 0 0 2024-09-10T11:29:34.504 INFO:tasks.workunit.client.0.smithi144.stdout:9/332: chown d11/d12/d1b/l66 36 1 2024-09-10T11:29:34.506 INFO:tasks.workunit.client.1.smithi178.stdout:6/653: dread - d3/d74/d91/daa/dc2/d97/fb4 zero size 2024-09-10T11:29:34.507 INFO:tasks.workunit.client.0.smithi144.stdout:5/335: dwrite d4/d5/d18/d67/f65 [0,4194304] 0 2024-09-10T11:29:34.509 INFO:tasks.workunit.client.0.smithi144.stdout:7/270: dwrite d5/f38 [0,4194304] 0 2024-09-10T11:29:34.510 INFO:tasks.workunit.client.0.smithi144.stdout:6/230: chown d9/dd/d3a/c48 5888367 1 2024-09-10T11:29:34.512 INFO:tasks.workunit.client.1.smithi178.stdout:9/696: creat d2/d13/d75/d60/d66/fd1 x:0 0 0 2024-09-10T11:29:34.512 INFO:tasks.workunit.client.1.smithi178.stdout:2/863: write d1/d22/d4d/d9b/f92 [57135,63276] 0 2024-09-10T11:29:34.515 INFO:tasks.workunit.client.0.smithi144.stdout:1/389: rename d5/d4a/f6a to d5/dd/d64/f7a 0 2024-09-10T11:29:34.516 INFO:tasks.workunit.client.0.smithi144.stdout:5/336: truncate d4/d20/f34 1163103 0 2024-09-10T11:29:34.516 INFO:tasks.workunit.client.0.smithi144.stdout:2/245: truncate d7/da/d1e/f25 4803045 0 2024-09-10T11:29:34.518 INFO:tasks.workunit.client.1.smithi178.stdout:8/773: link d5/d30/d33/l101 d5/db/d3d/de7/l103 0 2024-09-10T11:29:34.518 INFO:tasks.workunit.client.0.smithi144.stdout:5/337: write d4/d5/f23 [4679173,6116] 0 2024-09-10T11:29:34.518 INFO:tasks.workunit.client.1.smithi178.stdout:6/654: write d3/d74/dc9/fb8 [2093211,118005] 0 2024-09-10T11:29:34.519 INFO:tasks.workunit.client.0.smithi144.stdout:1/390: dread - d5/d2a/d65/f72 zero size 2024-09-10T11:29:34.519 INFO:tasks.workunit.client.0.smithi144.stdout:4/261: mknod d14/d1e/d22/d2b/d35/d3c/c5a 0 2024-09-10T11:29:34.520 INFO:tasks.workunit.client.0.smithi144.stdout:8/202: dwrite d0/d9/d23/f40 [0,4194304] 0 2024-09-10T11:29:34.522 INFO:tasks.workunit.client.0.smithi144.stdout:2/246: read d7/d42/f48 [2325978,89652] 0 2024-09-10T11:29:34.522 INFO:tasks.workunit.client.0.smithi144.stdout:0/221: symlink da/l3f 0 2024-09-10T11:29:34.525 INFO:tasks.workunit.client.1.smithi178.stdout:8/774: fsync d5/db/d1e/f62 0 2024-09-10T11:29:34.527 INFO:tasks.workunit.client.1.smithi178.stdout:7/625: dwrite d6/d10/d1e/d2d/d2b/da0/d55/dbd/f77 [4194304,4194304] 0 2024-09-10T11:29:34.528 INFO:tasks.workunit.client.0.smithi144.stdout:8/203: truncate d0/d9/d23/d37/f3a 387237 0 2024-09-10T11:29:34.528 INFO:tasks.workunit.client.0.smithi144.stdout:7/271: rmdir d5/dc/d2a 39 2024-09-10T11:29:34.529 INFO:tasks.workunit.client.0.smithi144.stdout:5/338: truncate d4/d5/d18/d67/d59/f5e 1551457 0 2024-09-10T11:29:34.529 INFO:tasks.workunit.client.1.smithi178.stdout:9/697: write d2/dc/fcf [521185,16130] 0 2024-09-10T11:29:34.531 INFO:tasks.workunit.client.1.smithi178.stdout:3/594: link db/d20/d3d/d56/d64/fae db/d9b/fba 0 2024-09-10T11:29:34.531 INFO:tasks.workunit.client.1.smithi178.stdout:5/695: dwrite d9/dc/dd/d37/d42/f8a [4194304,4194304] 0 2024-09-10T11:29:34.533 INFO:tasks.workunit.client.0.smithi144.stdout:6/231: creat d9/d1f/d2c/d4f/f58 x:0 0 0 2024-09-10T11:29:34.538 INFO:tasks.workunit.client.1.smithi178.stdout:4/555: write d0/f5 [3474692,102259] 0 2024-09-10T11:29:34.538 INFO:tasks.workunit.client.1.smithi178.stdout:6/655: mknod d3/d7/d8/d15/cf0 0 2024-09-10T11:29:34.539 INFO:tasks.workunit.client.1.smithi178.stdout:1/590: dwrite d3/d9/d98/dac/d3a/f53 [0,4194304] 0 2024-09-10T11:29:34.539 INFO:tasks.workunit.client.0.smithi144.stdout:7/272: dwrite d5/dc/d12/f21 [0,4194304] 0 2024-09-10T11:29:34.540 INFO:tasks.workunit.client.1.smithi178.stdout:6/656: chown d3/c5e 1826350 1 2024-09-10T11:29:34.544 INFO:tasks.workunit.client.1.smithi178.stdout:4/556: write d0/d9/dc/d8e/dbf/f37 [1377232,57601] 0 2024-09-10T11:29:34.545 INFO:tasks.workunit.client.1.smithi178.stdout:2/864: symlink d1/d7/d24/d46/d88/d9d/dab/d5b/l11f 0 2024-09-10T11:29:34.545 INFO:tasks.workunit.client.0.smithi144.stdout:9/333: mknod d11/d12/d6f/c7c 0 2024-09-10T11:29:34.548 INFO:tasks.workunit.client.0.smithi144.stdout:9/334: chown d11/d36/d2d/c43 268236583 1 2024-09-10T11:29:34.548 INFO:tasks.workunit.client.0.smithi144.stdout:9/335: fdatasync d11/d12/d1b/d22/f3d 0 2024-09-10T11:29:34.549 INFO:tasks.workunit.client.0.smithi144.stdout:9/336: dread d11/d12/f26 [0,4194304] 0 2024-09-10T11:29:34.549 INFO:tasks.workunit.client.0.smithi144.stdout:0/222: symlink da/d3a/l40 0 2024-09-10T11:29:34.550 INFO:tasks.workunit.client.0.smithi144.stdout:2/247: write d7/de/f2b [2953608,38865] 0 2024-09-10T11:29:34.550 INFO:tasks.workunit.client.0.smithi144.stdout:9/337: read d11/d12/d1b/d2c/f31 [1214540,115167] 0 2024-09-10T11:29:34.551 INFO:tasks.workunit.client.1.smithi178.stdout:6/657: read f1 [1290309,119350] 0 2024-09-10T11:29:34.555 INFO:tasks.workunit.client.0.smithi144.stdout:9/338: truncate d11/d12/d1b/d22/f7a 974195 0 2024-09-10T11:29:34.555 INFO:tasks.workunit.client.0.smithi144.stdout:9/339: fdatasync f10 0 2024-09-10T11:29:34.560 INFO:tasks.workunit.client.1.smithi178.stdout:1/591: dwrite d3/d9/d21/fb7 [0,4194304] 0 2024-09-10T11:29:34.564 INFO:tasks.workunit.client.1.smithi178.stdout:6/658: creat d3/d6e/ff1 x:0 0 0 2024-09-10T11:29:34.565 INFO:tasks.workunit.client.1.smithi178.stdout:7/626: creat d6/d10/d1e/d2d/d2b/da0/dc0/fe8 x:0 0 0 2024-09-10T11:29:34.566 INFO:tasks.workunit.client.1.smithi178.stdout:9/698: dread d2/df/d31/d37/d3d/f6e [0,4194304] 0 2024-09-10T11:29:34.566 INFO:tasks.workunit.client.0.smithi144.stdout:2/248: unlink d7/da/d1e/d21/f30 0 2024-09-10T11:29:34.568 INFO:tasks.workunit.client.0.smithi144.stdout:0/223: dread f2 [0,4194304] 0 2024-09-10T11:29:34.568 INFO:tasks.workunit.client.0.smithi144.stdout:4/262: link d14/d1e/d22/f31 d14/d1e/d22/d2b/d35/f5b 0 2024-09-10T11:29:34.569 INFO:tasks.workunit.client.1.smithi178.stdout:6/659: fdatasync d3/d74/dc9/da6/fdd 0 2024-09-10T11:29:34.570 INFO:tasks.workunit.client.1.smithi178.stdout:5/696: creat d9/dc/dd/d3f/fe9 x:0 0 0 2024-09-10T11:29:34.573 INFO:tasks.workunit.client.0.smithi144.stdout:6/232: rmdir d9 39 2024-09-10T11:29:34.574 INFO:tasks.workunit.client.1.smithi178.stdout:6/660: fsync d3/d74/d91/daa/dc2/d57/d78/d7c/f92 0 2024-09-10T11:29:34.575 INFO:tasks.workunit.client.1.smithi178.stdout:0/652: dread d0/dc/f13 [0,4194304] 0 2024-09-10T11:29:34.575 INFO:tasks.workunit.client.0.smithi144.stdout:3/327: link d4/dd/d21/d2f/c61 d4/dd/d22/d17/c72 0 2024-09-10T11:29:34.576 INFO:tasks.workunit.client.0.smithi144.stdout:3/328: dread - d4/f58 zero size 2024-09-10T11:29:34.576 INFO:tasks.workunit.client.1.smithi178.stdout:6/661: readlink d3/d74/dc9/d4b/lc6 0 2024-09-10T11:29:34.576 INFO:tasks.workunit.client.1.smithi178.stdout:9/699: dwrite d2/f18 [4194304,4194304] 0 2024-09-10T11:29:34.577 INFO:tasks.workunit.client.0.smithi144.stdout:5/339: mkdir d4/d20/d24/d6f 0 2024-09-10T11:29:34.577 INFO:tasks.workunit.client.1.smithi178.stdout:1/592: rename d3/d9/d98/dac/d2a/f41 to d3/d9/d21/d4b/d2b/d5f/fd7 0 2024-09-10T11:29:34.579 INFO:tasks.workunit.client.0.smithi144.stdout:5/340: read d4/d5/f23 [2289536,69962] 0 2024-09-10T11:29:34.580 INFO:tasks.workunit.client.1.smithi178.stdout:1/593: write d3/d9/fa [1224909,128644] 0 2024-09-10T11:29:34.581 INFO:tasks.workunit.client.1.smithi178.stdout:0/653: fdatasync d0/dc/f21 0 2024-09-10T11:29:34.582 INFO:tasks.workunit.client.1.smithi178.stdout:5/697: write d9/dc/d17/d1d/fb1 [1233166,99307] 0 2024-09-10T11:29:34.584 INFO:tasks.workunit.client.1.smithi178.stdout:5/698: chown d9/dc/dd/d3f/d5a/l81 1026 1 2024-09-10T11:29:34.585 INFO:tasks.workunit.client.1.smithi178.stdout:7/627: unlink d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d7c/le5 0 2024-09-10T11:29:34.585 INFO:tasks.workunit.client.0.smithi144.stdout:9/340: dwrite d11/d12/d1b/d22/d53/d64/f68 [0,4194304] 0 2024-09-10T11:29:34.586 INFO:tasks.workunit.client.1.smithi178.stdout:1/594: chown d3/d9/d98/dac/d3a/d50/d6c/cab 32288 1 2024-09-10T11:29:34.588 INFO:tasks.workunit.client.1.smithi178.stdout:6/662: creat d3/d7/d8/d56/de1/ff2 x:0 0 0 2024-09-10T11:29:34.589 INFO:tasks.workunit.client.0.smithi144.stdout:7/273: creat d5/dc/d11/f5c x:0 0 0 2024-09-10T11:29:34.589 INFO:tasks.workunit.client.0.smithi144.stdout:3/329: mknod d4/dd/c73 0 2024-09-10T11:29:34.590 INFO:tasks.workunit.client.0.smithi144.stdout:1/391: getdents d5/df/d4b/d5d 0 2024-09-10T11:29:34.591 INFO:tasks.workunit.client.1.smithi178.stdout:9/700: link d2/d13/d75/f5d d2/df/d31/d37/d3d/fd2 0 2024-09-10T11:29:34.592 INFO:tasks.workunit.client.0.smithi144.stdout:6/233: write d9/dd/d3e/d47/f57 [392655,74937] 0 2024-09-10T11:29:34.592 INFO:tasks.workunit.client.1.smithi178.stdout:9/701: rename d2/d13/d40/d56 to d2/d13/d40/d56/dbe/dd3 22 2024-09-10T11:29:34.594 INFO:tasks.workunit.client.1.smithi178.stdout:7/628: symlink d6/d10/d1d/d4a/d79/d83/le9 0 2024-09-10T11:29:34.594 INFO:tasks.workunit.client.0.smithi144.stdout:6/234: truncate d9/f2a 895959 0 2024-09-10T11:29:34.595 INFO:tasks.workunit.client.1.smithi178.stdout:5/699: creat d9/dc/dd/d37/d42/d55/d6d/fea x:0 0 0 2024-09-10T11:29:34.596 INFO:tasks.workunit.client.0.smithi144.stdout:5/341: creat d4/d5/d18/d38/d6d/f70 x:0 0 0 2024-09-10T11:29:34.596 INFO:tasks.workunit.client.0.smithi144.stdout:6/235: fsync d9/d1f/f36 0 2024-09-10T11:29:34.597 INFO:tasks.workunit.client.0.smithi144.stdout:5/342: chown d4/d5 1093826 1 2024-09-10T11:29:34.597 INFO:tasks.workunit.client.0.smithi144.stdout:7/274: mknod d5/dc/d14/d40/c5d 0 2024-09-10T11:29:34.598 INFO:tasks.workunit.client.0.smithi144.stdout:4/263: link d14/d1e/d22/d34/l51 d14/d1e/d22/d34/d3e/d40/d4b/l5c 0 2024-09-10T11:29:34.599 INFO:tasks.workunit.client.1.smithi178.stdout:9/702: fsync d2/d19/d1c/d2a/f67 0 2024-09-10T11:29:34.602 INFO:tasks.workunit.client.0.smithi144.stdout:3/330: creat d4/dd/d22/d17/d18/d34/d4b/d63/f74 x:0 0 0 2024-09-10T11:29:34.603 INFO:tasks.workunit.client.1.smithi178.stdout:6/663: mkdir d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3 0 2024-09-10T11:29:34.603 INFO:tasks.workunit.client.1.smithi178.stdout:3/595: write db/d20/d3d/d56/d64/fae [948808,66138] 0 2024-09-10T11:29:34.604 INFO:tasks.workunit.client.1.smithi178.stdout:1/595: rename d3/f47 to d3/fd8 0 2024-09-10T11:29:34.605 INFO:tasks.workunit.client.0.smithi144.stdout:7/275: mkdir d5/dc/d14/d40/d5e 0 2024-09-10T11:29:34.606 INFO:tasks.workunit.client.1.smithi178.stdout:4/557: dread d0/d9/dc/f8c [0,4194304] 0 2024-09-10T11:29:34.607 INFO:tasks.workunit.client.1.smithi178.stdout:6/664: readlink d3/d74/d91/daa/dc2/l98 0 2024-09-10T11:29:34.607 INFO:tasks.workunit.client.1.smithi178.stdout:2/865: dwrite d1/f2e [0,4194304] 0 2024-09-10T11:29:34.609 INFO:tasks.workunit.client.0.smithi144.stdout:6/236: dwrite d9/dd/d3e/d47/f57 [0,4194304] 0 2024-09-10T11:29:34.609 INFO:tasks.workunit.client.1.smithi178.stdout:9/703: mknod d2/d19/d1c/d77/cd4 0 2024-09-10T11:29:34.609 INFO:tasks.workunit.client.1.smithi178.stdout:7/629: read d6/d10/d1d/d22/d3b/fa5 [2570592,98896] 0 2024-09-10T11:29:34.610 INFO:tasks.workunit.client.1.smithi178.stdout:8/775: truncate d5/d30/f87 3153304 0 2024-09-10T11:29:34.610 INFO:tasks.workunit.client.1.smithi178.stdout:3/596: creat db/d20/d3d/d63/fbb x:0 0 0 2024-09-10T11:29:34.611 INFO:tasks.workunit.client.0.smithi144.stdout:2/249: rmdir d7/da 39 2024-09-10T11:29:34.613 INFO:tasks.workunit.client.1.smithi178.stdout:9/704: chown d2/d13/c38 85 1 2024-09-10T11:29:34.616 INFO:tasks.workunit.client.0.smithi144.stdout:9/341: dread d11/d12/d1b/d22/f4a [0,4194304] 0 2024-09-10T11:29:34.617 INFO:tasks.workunit.client.0.smithi144.stdout:4/264: getdents d14/d1e/d22/d34/d3e/d40 0 2024-09-10T11:29:34.618 INFO:tasks.workunit.client.1.smithi178.stdout:4/558: symlink d0/d8/dc0/lc3 0 2024-09-10T11:29:34.619 INFO:tasks.workunit.client.0.smithi144.stdout:3/331: truncate d4/dd/d22/d17/d18/f2e 1423906 0 2024-09-10T11:29:34.624 INFO:tasks.workunit.client.0.smithi144.stdout:6/237: symlink d9/d38/l59 0 2024-09-10T11:29:34.624 INFO:tasks.workunit.client.1.smithi178.stdout:2/866: rmdir d1/d41/d8d/d9f 39 2024-09-10T11:29:34.628 INFO:tasks.workunit.client.0.smithi144.stdout:9/342: creat d11/d12/d1b/d2c/d78/f7d x:0 0 0 2024-09-10T11:29:34.628 INFO:tasks.workunit.client.0.smithi144.stdout:5/343: dread d4/d5/d18/f1f [0,4194304] 0 2024-09-10T11:29:34.630 INFO:tasks.workunit.client.1.smithi178.stdout:6/665: truncate d3/d7/f9d 666580 0 2024-09-10T11:29:34.630 INFO:tasks.workunit.client.1.smithi178.stdout:0/654: dread d0/f4e [0,4194304] 0 2024-09-10T11:29:34.631 INFO:tasks.workunit.client.1.smithi178.stdout:7/630: creat d6/d10/d1d/fea x:0 0 0 2024-09-10T11:29:34.632 INFO:tasks.workunit.client.0.smithi144.stdout:7/276: dwrite d5/dc/d11/f5c [0,4194304] 0 2024-09-10T11:29:34.632 INFO:tasks.workunit.client.0.smithi144.stdout:4/265: dread d14/d1e/f50 [0,4194304] 0 2024-09-10T11:29:34.632 INFO:tasks.workunit.client.0.smithi144.stdout:3/332: mkdir d4/dd/d22/d17/d18/d34/d65/d75 0 2024-09-10T11:29:34.632 INFO:tasks.workunit.client.0.smithi144.stdout:2/250: dread d7/de/f28 [0,4194304] 0 2024-09-10T11:29:34.632 INFO:tasks.workunit.client.1.smithi178.stdout:6/666: readlink d3/d7/d8/d72/l82 0 2024-09-10T11:29:34.633 INFO:tasks.workunit.client.1.smithi178.stdout:8/776: rename d5/d1d/d22/l94 to d5/db/dda/l104 0 2024-09-10T11:29:34.633 INFO:tasks.workunit.client.1.smithi178.stdout:0/655: write d0/dc/f2b [2724569,111438] 0 2024-09-10T11:29:34.633 INFO:tasks.workunit.client.0.smithi144.stdout:6/238: creat d9/dd/d3e/d47/d13/f5a x:0 0 0 2024-09-10T11:29:34.633 INFO:tasks.workunit.client.0.smithi144.stdout:1/392: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:34.633 INFO:tasks.workunit.client.1.smithi178.stdout:8/777: readlink d5/d1d/d22/d76/l102 0 2024-09-10T11:29:34.635 INFO:tasks.workunit.client.1.smithi178.stdout:4/559: symlink d0/d10/lc4 0 2024-09-10T11:29:34.636 INFO:tasks.workunit.client.1.smithi178.stdout:4/560: chown d0/d9d/da5/fb8 34 1 2024-09-10T11:29:34.636 INFO:tasks.workunit.client.0.smithi144.stdout:7/277: dread d5/dc/d11/d18/f1e [0,4194304] 0 2024-09-10T11:29:34.638 INFO:tasks.workunit.client.0.smithi144.stdout:9/343: write d11/d12/d1b/d41/f5c [629157,19063] 0 2024-09-10T11:29:34.639 INFO:tasks.workunit.client.1.smithi178.stdout:1/596: link d3/d9/d98/dac/d3a/f53 d3/d35/fd9 0 2024-09-10T11:29:34.639 INFO:tasks.workunit.client.0.smithi144.stdout:1/393: symlink d5/df/d4b/d5d/l7b 0 2024-09-10T11:29:34.640 INFO:tasks.workunit.client.0.smithi144.stdout:5/344: creat d4/d5/d32/d47/f71 x:0 0 0 2024-09-10T11:29:34.642 INFO:tasks.workunit.client.0.smithi144.stdout:9/344: chown d11/d36/d2d/l72 239 1 2024-09-10T11:29:34.642 INFO:tasks.workunit.client.0.smithi144.stdout:6/239: unlink d9/dd/d3e/d47/f25 0 2024-09-10T11:29:34.643 INFO:tasks.workunit.client.0.smithi144.stdout:9/345: dread - d11/d12/d1b/f77 zero size 2024-09-10T11:29:34.646 INFO:tasks.workunit.client.0.smithi144.stdout:4/266: dwrite f9 [4194304,4194304] 0 2024-09-10T11:29:34.647 INFO:tasks.workunit.client.0.smithi144.stdout:6/240: read - d9/d1f/d2c/d4f/f58 zero size 2024-09-10T11:29:34.648 INFO:tasks.workunit.client.0.smithi144.stdout:5/345: mkdir d4/d20/d22/d5c/d5d/d72 0 2024-09-10T11:29:34.649 INFO:tasks.workunit.client.0.smithi144.stdout:6/241: chown d9/dd/d11/d1b/f32 1925 1 2024-09-10T11:29:34.650 INFO:tasks.workunit.client.0.smithi144.stdout:4/267: truncate d14/d1e/d22/f31 322085 0 2024-09-10T11:29:34.650 INFO:tasks.workunit.client.0.smithi144.stdout:9/346: unlink d11/d36/d3e/l45 0 2024-09-10T11:29:34.654 INFO:tasks.workunit.client.0.smithi144.stdout:2/251: rename d7/da/f18 to d7/de/d36/f51 0 2024-09-10T11:29:34.655 INFO:tasks.workunit.client.0.smithi144.stdout:6/242: dread d9/dd/d3e/d47/f57 [0,4194304] 0 2024-09-10T11:29:34.655 INFO:tasks.workunit.client.0.smithi144.stdout:6/243: fsync d9/dd/d11/f46 0 2024-09-10T11:29:34.656 INFO:tasks.workunit.client.0.smithi144.stdout:6/244: truncate d9/dd/d3e/f40 719873 0 2024-09-10T11:29:34.658 INFO:tasks.workunit.client.1.smithi178.stdout:2/867: dread d1/d7/d24/d7d/d84/d4b/f67 [0,4194304] 0 2024-09-10T11:29:34.658 INFO:tasks.workunit.client.0.smithi144.stdout:6/245: chown d9/dd/d3e/d47/d13/f1e 74761 1 2024-09-10T11:29:34.658 INFO:tasks.workunit.client.1.smithi178.stdout:2/868: chown d1/d7/d24/d7d/d84 821011791 1 2024-09-10T11:29:34.659 INFO:tasks.workunit.client.1.smithi178.stdout:2/869: chown d1/d41/d14/l7b 102 1 2024-09-10T11:29:34.660 INFO:tasks.workunit.client.0.smithi144.stdout:3/333: write d4/dd/d22/fc [866891,64771] 0 2024-09-10T11:29:34.661 INFO:tasks.workunit.client.0.smithi144.stdout:4/268: mknod d14/d1e/d22/d34/d3e/c5d 0 2024-09-10T11:29:34.662 INFO:tasks.workunit.client.1.smithi178.stdout:2/870: read d1/d7/d24/d43/f86 [1108263,8692] 0 2024-09-10T11:29:34.663 INFO:tasks.workunit.client.1.smithi178.stdout:2/871: fsync d1/d7/d24/d7d/d84/f35 0 2024-09-10T11:29:34.667 INFO:tasks.workunit.client.0.smithi144.stdout:1/394: rename d5/d2a/d65/f72 to d5/df/d4b/d62/f7c 0 2024-09-10T11:29:34.667 INFO:tasks.workunit.client.0.smithi144.stdout:7/278: write d5/dc/d2a/f35 [1656606,1613] 0 2024-09-10T11:29:34.668 INFO:tasks.workunit.client.0.smithi144.stdout:1/395: write d5/fc [8024741,116596] 0 2024-09-10T11:29:34.671 INFO:tasks.workunit.client.0.smithi144.stdout:2/252: rmdir d7/de 39 2024-09-10T11:29:34.672 INFO:tasks.workunit.client.1.smithi178.stdout:2/872: dwrite d1/d7/d24/d7d/d84/f36 [4194304,4194304] 0 2024-09-10T11:29:34.674 INFO:tasks.workunit.client.0.smithi144.stdout:4/269: fdatasync d14/d1e/d22/d2b/d35/f38 0 2024-09-10T11:29:34.678 INFO:tasks.workunit.client.0.smithi144.stdout:5/346: rename d4/d20/d24/c2a to d4/d20/d24/d2d/d4b/c73 0 2024-09-10T11:29:34.678 INFO:tasks.workunit.client.1.smithi178.stdout:3/597: link db/d35/f5e db/d20/d3d/d63/da2/d98/fbc 0 2024-09-10T11:29:34.682 INFO:tasks.workunit.client.0.smithi144.stdout:3/334: dread d4/dd/f51 [0,4194304] 0 2024-09-10T11:29:34.682 INFO:tasks.workunit.client.0.smithi144.stdout:3/335: readlink d4/dd/l11 0 2024-09-10T11:29:34.684 INFO:tasks.workunit.client.0.smithi144.stdout:6/246: mkdir d9/dd/d3e/d47/d13/d4a/d5b 0 2024-09-10T11:29:34.686 INFO:tasks.workunit.client.0.smithi144.stdout:2/253: dread d7/f11 [0,4194304] 0 2024-09-10T11:29:34.686 INFO:tasks.workunit.client.0.smithi144.stdout:9/347: rmdir d11/d12/d1b/d7b 0 2024-09-10T11:29:34.687 INFO:tasks.workunit.client.1.smithi178.stdout:2/873: mkdir d1/d7/d24/d46/d88/d9d/d120 0 2024-09-10T11:29:34.688 INFO:tasks.workunit.client.1.smithi178.stdout:3/598: rmdir db/d9b 39 2024-09-10T11:29:34.690 INFO:tasks.workunit.client.0.smithi144.stdout:9/348: read d11/f17 [4325930,29546] 0 2024-09-10T11:29:34.690 INFO:tasks.workunit.client.1.smithi178.stdout:7/631: link d6/d10/d1e/d2d/d2b/da0/fb9 d6/d10/d1d/d22/d50/d7e/da7/feb 0 2024-09-10T11:29:34.691 INFO:tasks.workunit.client.1.smithi178.stdout:1/597: dwrite d3/d9/d21/d4b/f15 [0,4194304] 0 2024-09-10T11:29:34.696 INFO:tasks.workunit.client.0.smithi144.stdout:5/347: mkdir d4/d20/d24/d6f/d74 0 2024-09-10T11:29:34.697 INFO:tasks.workunit.client.1.smithi178.stdout:3/599: mknod db/d35/d3a/d42/d46/cbd 0 2024-09-10T11:29:34.698 INFO:tasks.workunit.client.0.smithi144.stdout:1/396: creat d5/d4a/f7d x:0 0 0 2024-09-10T11:29:34.698 INFO:tasks.workunit.client.1.smithi178.stdout:4/561: creat d0/d8/da/d1a/d51/d5d/fc5 x:0 0 0 2024-09-10T11:29:34.698 INFO:tasks.workunit.client.1.smithi178.stdout:6/667: dread d3/d6e/fba [0,4194304] 0 2024-09-10T11:29:34.699 INFO:tasks.workunit.client.0.smithi144.stdout:4/270: dwrite d14/d1e/f50 [0,4194304] 0 2024-09-10T11:29:34.700 INFO:tasks.workunit.client.1.smithi178.stdout:4/562: write d0/d8/df/f7c [400088,4775] 0 2024-09-10T11:29:34.701 INFO:tasks.workunit.client.1.smithi178.stdout:2/874: mknod d1/d41/c121 0 2024-09-10T11:29:34.701 INFO:tasks.workunit.client.0.smithi144.stdout:7/279: fsync d5/dc/d12/f1f 0 2024-09-10T11:29:34.702 INFO:tasks.workunit.client.1.smithi178.stdout:7/632: dread d6/d10/d1e/d2d/d2b/f39 [0,4194304] 0 2024-09-10T11:29:34.702 INFO:tasks.workunit.client.1.smithi178.stdout:6/668: fdatasync d3/d74/d91/daa/dc2/dcf/fe0 0 2024-09-10T11:29:34.702 INFO:tasks.workunit.client.0.smithi144.stdout:3/336: write d4/dd/d22/f3e [163227,123120] 0 2024-09-10T11:29:34.703 INFO:tasks.workunit.client.1.smithi178.stdout:1/598: mknod d3/d9/d98/dac/d3a/d50/d80/cda 0 2024-09-10T11:29:34.704 INFO:tasks.workunit.client.0.smithi144.stdout:4/271: symlink d14/d1e/d22/d2b/d35/l5e 0 2024-09-10T11:29:34.705 INFO:tasks.workunit.client.0.smithi144.stdout:5/348: write d4/d5/d18/d38/d6d/f70 [196083,17510] 0 2024-09-10T11:29:34.706 INFO:tasks.workunit.client.1.smithi178.stdout:3/600: readlink db/d20/d3d/d63/d6a/l75 0 2024-09-10T11:29:34.707 INFO:tasks.workunit.client.0.smithi144.stdout:9/349: rename lf to d11/d12/d1b/d41/l7e 0 2024-09-10T11:29:34.707 INFO:tasks.workunit.client.1.smithi178.stdout:3/601: readlink db/d20/d3e/l57 0 2024-09-10T11:29:34.707 INFO:tasks.workunit.client.1.smithi178.stdout:3/602: chown db/d20/d3d/d56/d64/l94 93141 1 2024-09-10T11:29:34.708 INFO:tasks.workunit.client.0.smithi144.stdout:5/349: truncate d4/d52/f69 804291 0 2024-09-10T11:29:34.710 INFO:tasks.workunit.client.0.smithi144.stdout:3/337: creat d4/d28/f76 x:0 0 0 2024-09-10T11:29:34.711 INFO:tasks.workunit.client.0.smithi144.stdout:5/350: write d4/d5/d18/d38/d6d/f70 [997329,106598] 0 2024-09-10T11:29:34.711 INFO:tasks.workunit.client.1.smithi178.stdout:6/669: dread d3/d74/dc9/d4b/f7b [0,4194304] 0 2024-09-10T11:29:34.712 INFO:tasks.workunit.client.0.smithi144.stdout:9/350: creat d11/d36/f7f x:0 0 0 2024-09-10T11:29:34.712 INFO:tasks.workunit.client.0.smithi144.stdout:6/247: link d9/dd/c10 d9/dd/d3e/d47/d13/d4a/d5b/c5c 0 2024-09-10T11:29:34.713 INFO:tasks.workunit.client.0.smithi144.stdout:3/338: dread - d4/d1a/f5f zero size 2024-09-10T11:29:34.713 INFO:tasks.workunit.client.1.smithi178.stdout:6/670: write d3/d7/d8/d15/d1c/f53 [1043698,102013] 0 2024-09-10T11:29:34.714 INFO:tasks.workunit.client.0.smithi144.stdout:1/397: creat d5/d2a/f7e x:0 0 0 2024-09-10T11:29:34.715 INFO:tasks.workunit.client.0.smithi144.stdout:6/248: dread - d9/dd/d11/f56 zero size 2024-09-10T11:29:34.715 INFO:tasks.workunit.client.1.smithi178.stdout:6/671: chown d3/d74/dc9/f93 3997393 1 2024-09-10T11:29:34.715 INFO:tasks.workunit.client.1.smithi178.stdout:2/875: mknod d1/d7/d24/d46/d88/dc6/c122 0 2024-09-10T11:29:34.715 INFO:tasks.workunit.client.0.smithi144.stdout:5/351: mkdir d4/d20/d22/d75 0 2024-09-10T11:29:34.717 INFO:tasks.workunit.client.1.smithi178.stdout:1/599: rename d3/d35/fd9 to d3/d9/d1a/fdb 0 2024-09-10T11:29:34.718 INFO:tasks.workunit.client.0.smithi144.stdout:1/398: symlink d5/dd/d64/d79/l7f 0 2024-09-10T11:29:34.718 INFO:tasks.workunit.client.0.smithi144.stdout:5/352: rename d4/d20 to d4/d20/d22/d5c/d5d/d64/d76 22 2024-09-10T11:29:34.719 INFO:tasks.workunit.client.1.smithi178.stdout:0/656: link d0/dc/dc4/cb3 d0/dc/d1f/ccf 0 2024-09-10T11:29:34.720 INFO:tasks.workunit.client.0.smithi144.stdout:1/399: chown d5/dd/c2b 887758370 1 2024-09-10T11:29:34.722 INFO:tasks.workunit.client.1.smithi178.stdout:7/633: dwrite d6/d10/d1d/d22/d50/d7e/da7/feb [0,4194304] 0 2024-09-10T11:29:34.722 INFO:tasks.workunit.client.1.smithi178.stdout:3/603: creat db/d20/d3d/d63/da2/d98/fbe x:0 0 0 2024-09-10T11:29:34.723 INFO:tasks.workunit.client.0.smithi144.stdout:6/249: dread d9/d1f/f43 [0,4194304] 0 2024-09-10T11:29:34.723 INFO:tasks.workunit.client.1.smithi178.stdout:3/604: readlink db/d36/l38 0 2024-09-10T11:29:34.724 INFO:tasks.workunit.client.0.smithi144.stdout:9/351: link d11/d12/d1b/d22/d53/d64/f68 d11/d36/f80 0 2024-09-10T11:29:34.724 INFO:tasks.workunit.client.1.smithi178.stdout:4/563: chown d0/d8/da/d1a/d51/d5d/d62/la7 0 1 2024-09-10T11:29:34.727 INFO:tasks.workunit.client.1.smithi178.stdout:6/672: symlink d3/d7/d8/d15/d1c/d2a/d18/d8e/d90/lf4 0 2024-09-10T11:29:34.727 INFO:tasks.workunit.client.0.smithi144.stdout:1/400: rename d5/df/d4b/d5d to d5/d4a/d44/d80 0 2024-09-10T11:29:34.727 INFO:tasks.workunit.client.0.smithi144.stdout:6/250: write d9/d1f/f37 [839857,90202] 0 2024-09-10T11:29:34.728 INFO:tasks.workunit.client.0.smithi144.stdout:1/401: fsync d5/f23 0 2024-09-10T11:29:34.729 INFO:tasks.workunit.client.1.smithi178.stdout:0/657: symlink d0/d12/ld0 0 2024-09-10T11:29:34.730 INFO:tasks.workunit.client.0.smithi144.stdout:3/339: dwrite d4/dd/d22/d1d/f2a [0,4194304] 0 2024-09-10T11:29:34.731 INFO:tasks.workunit.client.1.smithi178.stdout:7/634: creat d6/d10/d1d/d22/d93/db7/fec x:0 0 0 2024-09-10T11:29:34.732 INFO:tasks.workunit.client.0.smithi144.stdout:6/251: dread - d9/dd/d3e/d47/d13/f5a zero size 2024-09-10T11:29:34.733 INFO:tasks.workunit.client.1.smithi178.stdout:2/876: dwrite d1/d41/d8d/d9f/f110 [0,4194304] 0 2024-09-10T11:29:34.734 INFO:tasks.workunit.client.0.smithi144.stdout:9/352: truncate d11/d12/d1b/f77 793722 0 2024-09-10T11:29:34.734 INFO:tasks.workunit.client.0.smithi144.stdout:4/272: dread d14/d1e/d22/d2b/d35/f38 [0,4194304] 0 2024-09-10T11:29:34.734 INFO:tasks.workunit.client.0.smithi144.stdout:3/340: write d4/f58 [996120,96937] 0 2024-09-10T11:29:34.735 INFO:tasks.workunit.client.0.smithi144.stdout:1/402: chown d5/d2a/c30 152098 1 2024-09-10T11:29:34.735 INFO:tasks.workunit.client.0.smithi144.stdout:6/252: chown d9/dd/d3e/d47/d13/d4a/d5b 13 1 2024-09-10T11:29:34.736 INFO:tasks.workunit.client.1.smithi178.stdout:4/564: chown d0/d8/da/d1a/d51/d5d/d62/c82 498 1 2024-09-10T11:29:34.738 INFO:tasks.workunit.client.0.smithi144.stdout:9/353: chown d11/d36/d2d/d38/l39 1567 1 2024-09-10T11:29:34.740 INFO:tasks.workunit.client.0.smithi144.stdout:7/280: dread d5/dc/f20 [0,4194304] 0 2024-09-10T11:29:34.741 INFO:tasks.workunit.client.0.smithi144.stdout:7/281: write d5/dc/d11/f3d [525463,41336] 0 2024-09-10T11:29:34.742 INFO:tasks.workunit.client.0.smithi144.stdout:3/341: mknod d4/dd/d22/d17/d18/c77 0 2024-09-10T11:29:34.743 INFO:tasks.workunit.client.1.smithi178.stdout:2/877: creat d1/d7/d24/d46/d88/d9d/f123 x:0 0 0 2024-09-10T11:29:34.745 INFO:tasks.workunit.client.0.smithi144.stdout:5/353: write d4/d5/f28 [2068930,5065] 0 2024-09-10T11:29:34.746 INFO:tasks.workunit.client.1.smithi178.stdout:2/878: chown d1/d7/d24/d46/d88/d9d/dab/d5e/db3/lf5 454 1 2024-09-10T11:29:34.747 INFO:tasks.workunit.client.0.smithi144.stdout:5/354: truncate d4/d5/d32/d47/f71 106561 0 2024-09-10T11:29:34.748 INFO:tasks.workunit.client.1.smithi178.stdout:1/600: rename d3/d9/d21/d4b/d2b/d5f to d3/d9/d98/dd1/ddc 0 2024-09-10T11:29:34.748 INFO:tasks.workunit.client.0.smithi144.stdout:9/354: symlink d11/d36/d2d/d38/l81 0 2024-09-10T11:29:34.749 INFO:tasks.workunit.client.0.smithi144.stdout:7/282: stat l0 0 2024-09-10T11:29:34.750 INFO:tasks.workunit.client.1.smithi178.stdout:1/601: chown d3/d9/d21/d6f/ca1 10034 1 2024-09-10T11:29:34.751 INFO:tasks.workunit.client.1.smithi178.stdout:2/879: symlink d1/d41/d14/d11/d53/l124 0 2024-09-10T11:29:34.752 INFO:tasks.workunit.client.0.smithi144.stdout:3/342: mkdir d4/dd/d22/d17/d18/d34/d78 0 2024-09-10T11:29:34.753 INFO:tasks.workunit.client.1.smithi178.stdout:3/605: rename f8 to db/d20/d70/fbf 0 2024-09-10T11:29:34.755 INFO:tasks.workunit.client.0.smithi144.stdout:5/355: chown d4/d5/l48 1 1 2024-09-10T11:29:34.756 INFO:tasks.workunit.client.0.smithi144.stdout:9/355: creat d11/d36/d3e/f82 x:0 0 0 2024-09-10T11:29:34.758 INFO:tasks.workunit.client.0.smithi144.stdout:4/273: truncate d14/d1e/f2f 925168 0 2024-09-10T11:29:34.759 INFO:tasks.workunit.client.1.smithi178.stdout:2/880: mknod d1/d41/d8d/d9f/c125 0 2024-09-10T11:29:34.759 INFO:tasks.workunit.client.1.smithi178.stdout:6/673: dread d3/d7/d8/f58 [0,4194304] 0 2024-09-10T11:29:34.760 INFO:tasks.workunit.client.1.smithi178.stdout:9/705: write d2/d13/d40/d89/d93/d88/fb0 [344068,129613] 0 2024-09-10T11:29:34.760 INFO:tasks.workunit.client.0.smithi144.stdout:3/343: write d4/dd/d21/d2f/f3a [4882144,112710] 0 2024-09-10T11:29:34.760 INFO:tasks.workunit.client.0.smithi144.stdout:1/403: getdents d5/d2a/d65 0 2024-09-10T11:29:34.762 INFO:tasks.workunit.client.1.smithi178.stdout:0/658: rename d0/dc/dc4/d8c to d0/dc/d1f/d2d/dd1 0 2024-09-10T11:29:34.763 INFO:tasks.workunit.client.1.smithi178.stdout:1/602: truncate d3/d9/d1a/d48/fa6 1028346 0 2024-09-10T11:29:34.764 INFO:tasks.workunit.client.0.smithi144.stdout:4/274: chown d14/d1e/d22/f4a 52 1 2024-09-10T11:29:34.764 INFO:tasks.workunit.client.0.smithi144.stdout:6/253: truncate d9/d1f/d2c/f53 429316 0 2024-09-10T11:29:34.765 INFO:tasks.workunit.client.0.smithi144.stdout:3/344: dread - d4/dd/d22/d17/d18/f6d zero size 2024-09-10T11:29:34.768 INFO:tasks.workunit.client.1.smithi178.stdout:9/706: rmdir d2/d19 39 2024-09-10T11:29:34.768 INFO:tasks.workunit.client.1.smithi178.stdout:0/659: mknod d0/d2c/d95/cd2 0 2024-09-10T11:29:34.769 INFO:tasks.workunit.client.0.smithi144.stdout:7/283: dwrite d5/dc/d14/f44 [0,4194304] 0 2024-09-10T11:29:34.769 INFO:tasks.workunit.client.0.smithi144.stdout:1/404: symlink d5/df/d4b/d62/l81 0 2024-09-10T11:29:34.769 INFO:tasks.workunit.client.1.smithi178.stdout:1/603: mkdir d3/d35/ddd 0 2024-09-10T11:29:34.771 INFO:tasks.workunit.client.0.smithi144.stdout:4/275: chown d14/d1e/d22/f23 3043 1 2024-09-10T11:29:34.773 INFO:tasks.workunit.client.1.smithi178.stdout:8/778: truncate d5/d1d/f60 2162700 0 2024-09-10T11:29:34.774 INFO:tasks.workunit.client.1.smithi178.stdout:0/660: mknod d0/dc/d39/d7c/cd3 0 2024-09-10T11:29:34.775 INFO:tasks.workunit.client.0.smithi144.stdout:5/356: dwrite d4/d20/d24/d2d/d4b/f4e [4194304,4194304] 0 2024-09-10T11:29:34.775 INFO:tasks.workunit.client.0.smithi144.stdout:7/284: write d5/dc/d2a/f4b [1215829,29303] 0 2024-09-10T11:29:34.776 INFO:tasks.workunit.client.0.smithi144.stdout:3/345: symlink d4/dd/d22/d17/d18/l79 0 2024-09-10T11:29:34.777 INFO:tasks.workunit.client.1.smithi178.stdout:9/707: fdatasync d2/df/f3b 0 2024-09-10T11:29:34.779 INFO:tasks.workunit.client.0.smithi144.stdout:9/356: dwrite d11/d12/d1b/f61 [0,4194304] 0 2024-09-10T11:29:34.779 INFO:tasks.workunit.client.1.smithi178.stdout:1/604: creat d3/d9/d98/dac/d3a/d50/d6c/d78/fde x:0 0 0 2024-09-10T11:29:34.780 INFO:tasks.workunit.client.1.smithi178.stdout:8/779: symlink d5/db/d10/d70/dc4/l105 0 2024-09-10T11:29:34.780 INFO:tasks.workunit.client.1.smithi178.stdout:3/606: dread db/f1a [0,4194304] 0 2024-09-10T11:29:34.781 INFO:tasks.workunit.client.1.smithi178.stdout:0/661: rmdir d0/dc/d1f 39 2024-09-10T11:29:34.782 INFO:tasks.workunit.client.1.smithi178.stdout:2/881: dread d1/d7/d24/d43/f86 [0,4194304] 0 2024-09-10T11:29:34.784 INFO:tasks.workunit.client.0.smithi144.stdout:5/357: mknod d4/d52/c77 0 2024-09-10T11:29:34.786 INFO:tasks.workunit.client.0.smithi144.stdout:1/405: dread d5/df/f55 [0,4194304] 0 2024-09-10T11:29:34.786 INFO:tasks.workunit.client.0.smithi144.stdout:4/276: rename d14/d1e/f21 to d14/d1e/d22/d34/d3e/d40/d4b/d4c/f5f 0 2024-09-10T11:29:34.787 INFO:tasks.workunit.client.1.smithi178.stdout:1/605: symlink d3/d35/ldf 0 2024-09-10T11:29:34.787 INFO:tasks.workunit.client.1.smithi178.stdout:1/606: write d3/d35/f99 [204805,130092] 0 2024-09-10T11:29:34.787 INFO:tasks.workunit.client.1.smithi178.stdout:3/607: mkdir db/d20/d3e/d6b/dc0 0 2024-09-10T11:29:34.788 INFO:tasks.workunit.client.0.smithi144.stdout:3/346: dwrite d4/dd/d21/f62 [0,4194304] 0 2024-09-10T11:29:34.788 INFO:tasks.workunit.client.0.smithi144.stdout:4/277: chown f9 5 1 2024-09-10T11:29:34.790 INFO:tasks.workunit.client.1.smithi178.stdout:1/607: fsync d3/d9/d98/dd1/d7e/f8c 0 2024-09-10T11:29:34.795 INFO:tasks.workunit.client.0.smithi144.stdout:7/285: getdents d5/dc/d2a 0 2024-09-10T11:29:34.795 INFO:tasks.workunit.client.0.smithi144.stdout:1/406: mknod d5/df/d4b/d62/c82 0 2024-09-10T11:29:34.795 INFO:tasks.workunit.client.0.smithi144.stdout:7/286: fsync d5/f5a 0 2024-09-10T11:29:34.796 INFO:tasks.workunit.client.0.smithi144.stdout:7/287: stat f1 0 2024-09-10T11:29:34.797 INFO:tasks.workunit.client.1.smithi178.stdout:2/882: dread d1/d41/d14/d11/db9/fe3 [0,4194304] 0 2024-09-10T11:29:34.798 INFO:tasks.workunit.client.0.smithi144.stdout:1/407: chown d5/d4a/c6c 841 1 2024-09-10T11:29:34.799 INFO:tasks.workunit.client.0.smithi144.stdout:3/347: dwrite d4/dd/d22/d17/d18/d34/d4b/f53 [4194304,4194304] 0 2024-09-10T11:29:34.801 INFO:tasks.workunit.client.0.smithi144.stdout:4/278: rename f12 to d14/d1e/d22/d2b/d35/d4f/f60 0 2024-09-10T11:29:34.802 INFO:tasks.workunit.client.1.smithi178.stdout:9/708: dread d2/d8/f1a [0,4194304] 0 2024-09-10T11:29:34.805 INFO:tasks.workunit.client.1.smithi178.stdout:7/635: write d6/d10/d34/d6a/f7f [788198,129542] 0 2024-09-10T11:29:34.808 INFO:tasks.workunit.client.1.smithi178.stdout:4/565: write d0/d8/df/f20 [615328,48582] 0 2024-09-10T11:29:34.809 INFO:tasks.workunit.client.0.smithi144.stdout:3/348: mkdir d4/d1a/d7a 0 2024-09-10T11:29:34.809 INFO:tasks.workunit.client.0.smithi144.stdout:7/288: creat d5/dc/d11/d18/f5f x:0 0 0 2024-09-10T11:29:34.809 INFO:tasks.workunit.client.0.smithi144.stdout:9/357: write d11/f2b [3948469,78975] 0 2024-09-10T11:29:34.809 INFO:tasks.workunit.client.0.smithi144.stdout:1/408: dwrite d5/d4a/d44/d80/f66 [0,4194304] 0 2024-09-10T11:29:34.811 INFO:tasks.workunit.client.1.smithi178.stdout:9/709: creat d2/d51/fd5 x:0 0 0 2024-09-10T11:29:34.812 INFO:tasks.workunit.client.1.smithi178.stdout:3/608: getdents db/d35/d58 0 2024-09-10T11:29:34.814 INFO:tasks.workunit.client.1.smithi178.stdout:9/710: truncate d2/d8/fa 4573823 0 2024-09-10T11:29:34.814 INFO:tasks.workunit.client.1.smithi178.stdout:7/636: creat d6/d10/d1e/d2d/d2b/da0/d55/fed x:0 0 0 2024-09-10T11:29:34.815 INFO:tasks.workunit.client.0.smithi144.stdout:7/289: rmdir d5/dc/d12 39 2024-09-10T11:29:34.816 INFO:tasks.workunit.client.1.smithi178.stdout:2/883: mknod d1/d7/d8e/de9/c126 0 2024-09-10T11:29:34.816 INFO:tasks.workunit.client.1.smithi178.stdout:3/609: read db/d20/d3d/f53 [7809003,4491] 0 2024-09-10T11:29:34.817 INFO:tasks.workunit.client.1.smithi178.stdout:9/711: write d2/d13/d75/d60/f8d [2277512,128505] 0 2024-09-10T11:29:34.818 INFO:tasks.workunit.client.0.smithi144.stdout:1/409: rename d5/c11 to d5/df/d4b/d51/c83 0 2024-09-10T11:29:34.818 INFO:tasks.workunit.client.0.smithi144.stdout:4/279: mknod d14/d1e/d22/d34/c61 0 2024-09-10T11:29:34.819 INFO:tasks.workunit.client.1.smithi178.stdout:4/566: dwrite d0/f94 [0,4194304] 0 2024-09-10T11:29:34.819 INFO:tasks.workunit.client.1.smithi178.stdout:3/610: chown db/d35/d3a/f4a 0 1 2024-09-10T11:29:34.821 INFO:tasks.workunit.client.1.smithi178.stdout:3/611: chown db/d20/d3d/d56 259318 1 2024-09-10T11:29:34.824 INFO:tasks.workunit.client.0.smithi144.stdout:7/290: rmdir d5/dc/d12/d2f 39 2024-09-10T11:29:34.825 INFO:tasks.workunit.client.1.smithi178.stdout:9/712: write d2/d8/fa [1695076,110746] 0 2024-09-10T11:29:34.826 INFO:tasks.workunit.client.0.smithi144.stdout:6/254: dwrite d9/d1f/d2c/f53 [0,4194304] 0 2024-09-10T11:29:34.829 INFO:tasks.workunit.client.0.smithi144.stdout:3/349: dwrite d4/d28/f76 [0,4194304] 0 2024-09-10T11:29:34.829 INFO:tasks.workunit.client.0.smithi144.stdout:1/410: stat d5/df/d4b/c52 0 2024-09-10T11:29:34.829 INFO:tasks.workunit.client.1.smithi178.stdout:2/884: dwrite d1/d7/d24/d46/d88/d9d/dab/f15 [0,4194304] 0 2024-09-10T11:29:34.829 INFO:tasks.workunit.client.1.smithi178.stdout:2/885: readlink d1/d7/d24/d46/d88/d9d/dab/lc2 0 2024-09-10T11:29:34.830 INFO:tasks.workunit.client.0.smithi144.stdout:9/358: rename d11/d12/l5d to d11/d12/d1b/d22/l83 0 2024-09-10T11:29:34.831 INFO:tasks.workunit.client.1.smithi178.stdout:2/886: chown d1/d41/d14/d11/db9/ff6 2822 1 2024-09-10T11:29:34.832 INFO:tasks.workunit.client.1.smithi178.stdout:4/567: creat d0/d9/d3d/fc6 x:0 0 0 2024-09-10T11:29:34.833 INFO:tasks.workunit.client.1.smithi178.stdout:4/568: chown d0/d9/dc/d8e/dbf/l24 6 1 2024-09-10T11:29:34.838 INFO:tasks.workunit.client.1.smithi178.stdout:3/612: mknod db/d20/d3d/d63/cc1 0 2024-09-10T11:29:34.838 INFO:tasks.workunit.client.0.smithi144.stdout:6/255: creat d9/dd/d3a/f5d x:0 0 0 2024-09-10T11:29:34.840 INFO:tasks.workunit.client.0.smithi144.stdout:9/359: mkdir d11/d36/d3e/d84 0 2024-09-10T11:29:34.840 INFO:tasks.workunit.client.0.smithi144.stdout:3/350: creat d4/dd/d22/d17/d18/d34/d4b/d63/f7b x:0 0 0 2024-09-10T11:29:34.841 INFO:tasks.workunit.client.1.smithi178.stdout:9/713: mkdir d2/d13/d75/d60/dd0/dd6 0 2024-09-10T11:29:34.842 INFO:tasks.workunit.client.0.smithi144.stdout:1/411: mkdir d5/d4a/d63/d84 0 2024-09-10T11:29:34.843 INFO:tasks.workunit.client.1.smithi178.stdout:9/714: stat d2/df/d31/d4a/cb7 0 2024-09-10T11:29:34.844 INFO:tasks.workunit.client.0.smithi144.stdout:5/358: dwrite d4/f55 [0,4194304] 0 2024-09-10T11:29:34.845 INFO:tasks.workunit.client.1.smithi178.stdout:4/569: dwrite d0/d9/dc/d8e/dbf/f93 [0,4194304] 0 2024-09-10T11:29:34.846 INFO:tasks.workunit.client.1.smithi178.stdout:1/608: write d3/d9/d98/dd1/ddc/fd7 [1335090,108865] 0 2024-09-10T11:29:34.849 INFO:tasks.workunit.client.0.smithi144.stdout:9/360: symlink d11/d12/d1b/d22/d53/l85 0 2024-09-10T11:29:34.849 INFO:tasks.workunit.client.0.smithi144.stdout:4/280: dread d14/d1e/d22/f4d [0,4194304] 0 2024-09-10T11:29:34.850 INFO:tasks.workunit.client.1.smithi178.stdout:2/887: rename d1/d7/d8e/d10b to d1/d41/d127 0 2024-09-10T11:29:34.852 INFO:tasks.workunit.client.0.smithi144.stdout:1/412: mkdir d5/df/d4b/d85 0 2024-09-10T11:29:34.854 INFO:tasks.workunit.client.1.smithi178.stdout:1/609: write d3/d9/d98/dac/d3a/d50/d6c/fbb [715851,85399] 0 2024-09-10T11:29:34.854 INFO:tasks.workunit.client.0.smithi144.stdout:7/291: dwrite d5/dc/d12/f13 [0,4194304] 0 2024-09-10T11:29:34.855 INFO:tasks.workunit.client.0.smithi144.stdout:6/256: rename d9/dd/d3e/d47/d13/c15 to d9/d38/c5e 0 2024-09-10T11:29:34.856 INFO:tasks.workunit.client.1.smithi178.stdout:0/662: dwrite d0/d12/f34 [0,4194304] 0 2024-09-10T11:29:34.857 INFO:tasks.workunit.client.0.smithi144.stdout:4/281: write d14/d1e/d22/d2b/f2e [336982,89235] 0 2024-09-10T11:29:34.858 INFO:tasks.workunit.client.1.smithi178.stdout:9/715: dwrite d2/df/d31/d37/d3d/faf [0,4194304] 0 2024-09-10T11:29:34.859 INFO:tasks.workunit.client.1.smithi178.stdout:9/716: chown d2/d13/f15 3 1 2024-09-10T11:29:34.862 INFO:tasks.workunit.client.0.smithi144.stdout:7/292: mkdir d5/dc/d2a/d60 0 2024-09-10T11:29:34.862 INFO:tasks.workunit.client.0.smithi144.stdout:3/351: link d4/dd/d22/d17/d18/f6d d4/d28/d67/f7c 0 2024-09-10T11:29:34.864 INFO:tasks.workunit.client.0.smithi144.stdout:7/293: dread - d5/f37 zero size 2024-09-10T11:29:34.865 INFO:tasks.workunit.client.1.smithi178.stdout:3/613: getdents db/d20/d3e/d6b/dc0 0 2024-09-10T11:29:34.865 INFO:tasks.workunit.client.0.smithi144.stdout:3/352: write d4/d28/f38 [4185996,34008] 0 2024-09-10T11:29:34.868 INFO:tasks.workunit.client.0.smithi144.stdout:7/294: dread - d5/dc/d11/f17 zero size 2024-09-10T11:29:34.868 INFO:tasks.workunit.client.0.smithi144.stdout:9/361: dwrite d11/d12/d1b/f4d [0,4194304] 0 2024-09-10T11:29:34.869 INFO:tasks.workunit.client.1.smithi178.stdout:0/663: creat d0/dc/d39/d75/fd4 x:0 0 0 2024-09-10T11:29:34.869 INFO:tasks.workunit.client.0.smithi144.stdout:1/413: rename d5/df/d4b/l58 to d5/dd/d64/d79/l86 0 2024-09-10T11:29:34.869 INFO:tasks.workunit.client.0.smithi144.stdout:1/414: fdatasync d5/f7 0 2024-09-10T11:29:34.870 INFO:tasks.workunit.client.1.smithi178.stdout:0/664: chown d0/dc/d39/d7c/fbb 1890101721 1 2024-09-10T11:29:34.872 INFO:tasks.workunit.client.1.smithi178.stdout:0/665: fdatasync d0/dc/d39/d50/d5e/fca 0 2024-09-10T11:29:34.876 INFO:tasks.workunit.client.1.smithi178.stdout:2/888: creat d1/d41/f128 x:0 0 0 2024-09-10T11:29:34.876 INFO:tasks.workunit.client.0.smithi144.stdout:6/257: symlink d9/l5f 0 2024-09-10T11:29:34.877 INFO:tasks.workunit.client.1.smithi178.stdout:0/666: rename d0/dc/dc4/d90 to d0/d2c/d95/dbd/dd5 0 2024-09-10T11:29:34.878 INFO:tasks.workunit.client.0.smithi144.stdout:7/295: chown d5/dc/d14/d40/d5e 178 1 2024-09-10T11:29:34.879 INFO:tasks.workunit.client.1.smithi178.stdout:9/717: creat d2/df/fd7 x:0 0 0 2024-09-10T11:29:34.880 INFO:tasks.workunit.client.0.smithi144.stdout:3/353: mknod d4/d28/c7d 0 2024-09-10T11:29:34.880 INFO:tasks.workunit.client.0.smithi144.stdout:4/282: creat d14/d1e/d22/d34/d3e/d40/f62 x:0 0 0 2024-09-10T11:29:34.880 INFO:tasks.workunit.client.0.smithi144.stdout:1/415: mknod d5/dd/d64/c87 0 2024-09-10T11:29:34.882 INFO:tasks.workunit.client.0.smithi144.stdout:6/258: rmdir d9 39 2024-09-10T11:29:34.883 INFO:tasks.workunit.client.1.smithi178.stdout:3/614: creat db/d20/d3e/d6b/dc0/fc2 x:0 0 0 2024-09-10T11:29:34.883 INFO:tasks.workunit.client.0.smithi144.stdout:9/362: dwrite d11/f1c [4194304,4194304] 0 2024-09-10T11:29:34.884 INFO:tasks.workunit.client.1.smithi178.stdout:2/889: mknod d1/d41/d14/d11/d53/d97/c129 0 2024-09-10T11:29:34.884 INFO:tasks.workunit.client.0.smithi144.stdout:7/296: chown d5/dc/d12/d2f/l46 627099901 1 2024-09-10T11:29:34.887 INFO:tasks.workunit.client.1.smithi178.stdout:9/718: fdatasync d2/d13/d40/d89/d93/f72 0 2024-09-10T11:29:34.888 INFO:tasks.workunit.client.0.smithi144.stdout:6/259: fsync d9/f2a 0 2024-09-10T11:29:34.888 INFO:tasks.workunit.client.0.smithi144.stdout:7/297: fdatasync d5/dc/d11/d18/f24 0 2024-09-10T11:29:34.888 INFO:tasks.workunit.client.1.smithi178.stdout:9/719: chown d2/df/d31/d37/c71 268 1 2024-09-10T11:29:34.889 INFO:tasks.workunit.client.0.smithi144.stdout:3/354: symlink d4/dd/d22/d17/d18/d34/d78/l7e 0 2024-09-10T11:29:34.889 INFO:tasks.workunit.client.1.smithi178.stdout:1/610: link d3/d9/d21/d4b/d2b/c39 d3/d9/d98/dac/d3a/ce0 0 2024-09-10T11:29:34.890 INFO:tasks.workunit.client.0.smithi144.stdout:6/260: dread - d9/dd/d3a/f5d zero size 2024-09-10T11:29:34.890 INFO:tasks.workunit.client.0.smithi144.stdout:7/298: write d5/dc/d11/d18/f24 [1615695,54633] 0 2024-09-10T11:29:34.891 INFO:tasks.workunit.client.0.smithi144.stdout:6/261: chown d9/d38 1239558700 1 2024-09-10T11:29:34.892 INFO:tasks.workunit.client.0.smithi144.stdout:9/363: mknod d11/d36/d3e/c86 0 2024-09-10T11:29:34.893 INFO:tasks.workunit.client.1.smithi178.stdout:2/890: creat d1/d22/d4d/f12a x:0 0 0 2024-09-10T11:29:34.894 INFO:tasks.workunit.client.0.smithi144.stdout:1/416: dwrite d5/d2a/f2d [0,4194304] 0 2024-09-10T11:29:34.895 INFO:tasks.workunit.client.0.smithi144.stdout:6/262: write d9/dd/f1a [666567,17738] 0 2024-09-10T11:29:34.897 INFO:tasks.workunit.client.0.smithi144.stdout:7/299: fsync d5/dc/d12/f21 0 2024-09-10T11:29:34.898 INFO:tasks.workunit.client.0.smithi144.stdout:3/355: rename d4/dd/d22/d17/d18/c77 to d4/d1a/c7f 0 2024-09-10T11:29:34.899 INFO:tasks.workunit.client.0.smithi144.stdout:6/263: write d9/dd/d11/f21 [1608604,51215] 0 2024-09-10T11:29:34.900 INFO:tasks.workunit.client.0.smithi144.stdout:9/364: creat d11/d36/d3e/f87 x:0 0 0 2024-09-10T11:29:34.900 INFO:tasks.workunit.client.1.smithi178.stdout:4/570: dwrite d0/d10/f5a [0,4194304] 0 2024-09-10T11:29:34.900 INFO:tasks.workunit.client.1.smithi178.stdout:0/667: creat d0/dc/fd6 x:0 0 0 2024-09-10T11:29:34.908 INFO:tasks.workunit.client.0.smithi144.stdout:7/300: mknod d5/dc/d2a/d41/c61 0 2024-09-10T11:29:34.908 INFO:tasks.workunit.client.1.smithi178.stdout:9/720: rmdir d2/d13/d75/d60/dd0/dd6 0 2024-09-10T11:29:34.909 INFO:tasks.workunit.client.0.smithi144.stdout:7/301: chown d5/dc/d2a/d41/l5b 140383565 1 2024-09-10T11:29:34.910 INFO:tasks.workunit.client.0.smithi144.stdout:7/302: write d5/dc/d11/f26 [5070682,13811] 0 2024-09-10T11:29:34.912 INFO:tasks.workunit.client.1.smithi178.stdout:1/611: dwrite d3/f1c [4194304,4194304] 0 2024-09-10T11:29:34.914 INFO:tasks.workunit.client.0.smithi144.stdout:6/264: mkdir d9/dd/d3e/d47/d13/d60 0 2024-09-10T11:29:34.915 INFO:tasks.workunit.client.0.smithi144.stdout:6/265: write d9/d1f/f36 [544354,7912] 0 2024-09-10T11:29:34.919 INFO:tasks.workunit.client.1.smithi178.stdout:3/615: truncate db/d35/d58/f6e 561188 0 2024-09-10T11:29:34.921 INFO:tasks.workunit.client.1.smithi178.stdout:0/668: dwrite d0/dc/d39/d4c/f73 [0,4194304] 0 2024-09-10T11:29:34.922 INFO:tasks.workunit.client.1.smithi178.stdout:4/571: dwrite d0/d9/d3d/d4f/f97 [0,4194304] 0 2024-09-10T11:29:34.922 INFO:tasks.workunit.client.0.smithi144.stdout:7/303: mknod d5/dc/d12/d2f/c62 0 2024-09-10T11:29:34.923 INFO:tasks.workunit.client.0.smithi144.stdout:7/304: fsync d5/f7 0 2024-09-10T11:29:34.923 INFO:tasks.workunit.client.0.smithi144.stdout:1/417: getdents d5/d4a 0 2024-09-10T11:29:34.925 INFO:tasks.workunit.client.0.smithi144.stdout:7/305: write d5/dc/d12/d2f/f4d [881761,89642] 0 2024-09-10T11:29:34.927 INFO:tasks.workunit.client.1.smithi178.stdout:3/616: unlink db/d20/f34 0 2024-09-10T11:29:34.927 INFO:tasks.workunit.client.1.smithi178.stdout:2/891: dread d1/d7/d24/d7d/d84/d4b/f67 [0,4194304] 0 2024-09-10T11:29:34.928 INFO:tasks.workunit.client.0.smithi144.stdout:4/283: rmdir d14/d1e 39 2024-09-10T11:29:34.931 INFO:tasks.workunit.client.1.smithi178.stdout:1/612: symlink d3/d9/d98/dac/d3a/d6e/d8a/le1 0 2024-09-10T11:29:34.931 INFO:tasks.workunit.client.1.smithi178.stdout:9/721: dwrite d2/d13/d40/d89/fbd [0,4194304] 0 2024-09-10T11:29:34.932 INFO:tasks.workunit.client.0.smithi144.stdout:6/266: creat d9/dd/d3e/d47/d13/d4a/d5b/f61 x:0 0 0 2024-09-10T11:29:34.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:34 smithi144 ceph-mon[27942]: pgmap v24: 65 pgs: 65 active+clean; 2.0 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 67 MiB/s rd, 276 MiB/s wr, 517 op/s 2024-09-10T11:29:34.934 INFO:tasks.workunit.client.0.smithi144.stdout:7/306: write d5/dc/d2a/d41/f49 [2703244,91909] 0 2024-09-10T11:29:34.934 INFO:tasks.workunit.client.1.smithi178.stdout:9/722: readlink d2/df/d31/d37/d3d/la1 0 2024-09-10T11:29:34.936 INFO:tasks.workunit.client.1.smithi178.stdout:0/669: rmdir d0/dc/d39/d50 39 2024-09-10T11:29:34.937 INFO:tasks.workunit.client.0.smithi144.stdout:6/267: dread d9/d1f/f43 [0,4194304] 0 2024-09-10T11:29:34.937 INFO:tasks.workunit.client.0.smithi144.stdout:1/418: symlink d5/d2a/l88 0 2024-09-10T11:29:34.937 INFO:tasks.workunit.client.0.smithi144.stdout:4/284: write d14/d1e/d22/d34/d3e/d40/d4b/f52 [202912,99014] 0 2024-09-10T11:29:34.938 INFO:tasks.workunit.client.0.smithi144.stdout:4/285: chown d14/d1e/d22/f4d 30911 1 2024-09-10T11:29:34.938 INFO:tasks.workunit.client.0.smithi144.stdout:4/286: chown d14/d1e/f45 0 1 2024-09-10T11:29:34.940 INFO:tasks.workunit.client.1.smithi178.stdout:3/617: symlink db/d20/d3d/d63/lc3 0 2024-09-10T11:29:34.942 INFO:tasks.workunit.client.0.smithi144.stdout:6/268: unlink d9/dd/d11/f46 0 2024-09-10T11:29:34.943 INFO:tasks.workunit.client.1.smithi178.stdout:2/892: symlink d1/d7/d24/d46/d88/d9d/dab/d5b/d60/l12b 0 2024-09-10T11:29:34.943 INFO:tasks.workunit.client.1.smithi178.stdout:3/618: read db/f4f [962281,44686] 0 2024-09-10T11:29:34.943 INFO:tasks.workunit.client.0.smithi144.stdout:6/269: dread - d9/dd/d3e/d47/d13/d4a/d5b/f61 zero size 2024-09-10T11:29:34.943 INFO:tasks.workunit.client.1.smithi178.stdout:1/613: mkdir d3/d9/d98/dac/d3a/d50/d9e/dc2/de2 0 2024-09-10T11:29:34.944 INFO:tasks.workunit.client.0.smithi144.stdout:6/270: fdatasync d9/dd/d11/d1b/f32 0 2024-09-10T11:29:34.944 INFO:tasks.workunit.client.0.smithi144.stdout:3/356: rename d4/dd/d22 to d4/d28/d67/d80 0 2024-09-10T11:29:34.946 INFO:tasks.workunit.client.0.smithi144.stdout:6/271: write d9/dd/d3e/d47/d13/f5a [649907,10985] 0 2024-09-10T11:29:34.946 INFO:tasks.workunit.client.0.smithi144.stdout:7/307: mkdir d5/dc/d14/d40/d5e/d63 0 2024-09-10T11:29:34.947 INFO:tasks.workunit.client.1.smithi178.stdout:3/619: dread - db/d20/f8a zero size 2024-09-10T11:29:34.947 INFO:tasks.workunit.client.1.smithi178.stdout:9/723: creat d2/d13/d40/d56/fd8 x:0 0 0 2024-09-10T11:29:34.949 INFO:tasks.workunit.client.1.smithi178.stdout:1/614: fdatasync d3/d9/f42 0 2024-09-10T11:29:34.950 INFO:tasks.workunit.client.0.smithi144.stdout:4/287: dread d14/d1e/d22/f2d [0,4194304] 0 2024-09-10T11:29:34.950 INFO:tasks.workunit.client.0.smithi144.stdout:9/365: rename cb to d11/d12/d1b/d22/d53/d64/c88 0 2024-09-10T11:29:34.954 INFO:tasks.workunit.client.1.smithi178.stdout:0/670: mknod d0/d2c/cd7 0 2024-09-10T11:29:34.954 INFO:tasks.workunit.client.0.smithi144.stdout:7/308: dwrite d5/f38 [0,4194304] 0 2024-09-10T11:29:34.954 INFO:tasks.workunit.client.0.smithi144.stdout:6/272: mkdir d9/dd/d11/d1b/d62 0 2024-09-10T11:29:34.955 INFO:tasks.workunit.client.1.smithi178.stdout:4/572: getdents d0/d10 0 2024-09-10T11:29:34.956 INFO:tasks.workunit.client.0.smithi144.stdout:1/419: creat d5/d2a/f89 x:0 0 0 2024-09-10T11:29:34.957 INFO:tasks.workunit.client.0.smithi144.stdout:4/288: write d14/d1e/d22/d2b/d35/d4f/f60 [2363166,12802] 0 2024-09-10T11:29:34.958 INFO:tasks.workunit.client.1.smithi178.stdout:2/893: symlink d1/d7/d24/d7d/d84/d4b/d118/l12c 0 2024-09-10T11:29:34.960 INFO:tasks.workunit.client.0.smithi144.stdout:3/357: rename d4/d28/d67/d80/f3e to d4/d28/d67/d80/d17/f81 0 2024-09-10T11:29:34.961 INFO:tasks.workunit.client.1.smithi178.stdout:1/615: mkdir d3/d9/d98/dac/d3a/d50/db5/de3 0 2024-09-10T11:29:34.961 INFO:tasks.workunit.client.0.smithi144.stdout:9/366: dwrite d11/d12/f14 [0,4194304] 0 2024-09-10T11:29:34.963 INFO:tasks.workunit.client.1.smithi178.stdout:9/724: rename d2/d19/d1c/f24 to d2/d19/d1c/da5/fd9 0 2024-09-10T11:29:34.963 INFO:tasks.workunit.client.1.smithi178.stdout:2/894: truncate d1/d7/d24/d46/d88/d9d/dab/d5b/d60/dd7/d108/f111 66076 0 2024-09-10T11:29:34.965 INFO:tasks.workunit.client.0.smithi144.stdout:4/289: creat d14/d1e/d22/d2b/d35/d3c/f63 x:0 0 0 2024-09-10T11:29:34.965 INFO:tasks.workunit.client.1.smithi178.stdout:2/895: write d1/d7/d24/d7d/d84/f35 [3109621,105173] 0 2024-09-10T11:29:34.965 INFO:tasks.workunit.client.1.smithi178.stdout:0/671: unlink d0/dc/dc4/l46 0 2024-09-10T11:29:34.966 INFO:tasks.workunit.client.0.smithi144.stdout:1/420: write d5/df/f55 [391180,1084] 0 2024-09-10T11:29:34.967 INFO:tasks.workunit.client.1.smithi178.stdout:0/672: write d0/d12/f34 [651196,50176] 0 2024-09-10T11:29:34.968 INFO:tasks.workunit.client.1.smithi178.stdout:0/673: dread - d0/fbf zero size 2024-09-10T11:29:34.970 INFO:tasks.workunit.client.0.smithi144.stdout:3/358: write d4/d28/d67/d80/d17/d18/d34/d4b/f53 [4643547,19264] 0 2024-09-10T11:29:34.973 INFO:tasks.workunit.client.1.smithi178.stdout:3/620: write db/d35/d3a/d42/d46/f4e [7703,129528] 0 2024-09-10T11:29:34.973 INFO:tasks.workunit.client.0.smithi144.stdout:1/421: mknod d5/d4a/c8a 0 2024-09-10T11:29:34.975 INFO:tasks.workunit.client.1.smithi178.stdout:4/573: rename d0/d9/dc/l2e to d0/d8/df/lc7 0 2024-09-10T11:29:34.976 INFO:tasks.workunit.client.0.smithi144.stdout:6/273: dwrite d9/dd/d3e/d47/f28 [0,4194304] 0 2024-09-10T11:29:34.978 INFO:tasks.workunit.client.0.smithi144.stdout:1/422: read d5/f7 [3391027,108383] 0 2024-09-10T11:29:34.978 INFO:tasks.workunit.client.0.smithi144.stdout:3/359: fdatasync d4/dd/d21/f62 0 2024-09-10T11:29:34.978 INFO:tasks.workunit.client.1.smithi178.stdout:1/616: getdents d3/d9/d98/dac/d3a/d50/d6c 0 2024-09-10T11:29:34.980 INFO:tasks.workunit.client.0.smithi144.stdout:7/309: dwrite d5/dc/d12/d2f/f4a [0,4194304] 0 2024-09-10T11:29:34.980 INFO:tasks.workunit.client.1.smithi178.stdout:3/621: creat db/d20/d3e/d6b/fc4 x:0 0 0 2024-09-10T11:29:34.982 INFO:tasks.workunit.client.0.smithi144.stdout:3/360: chown d4/dd/d21/d2f 8350669 1 2024-09-10T11:29:34.984 INFO:tasks.workunit.client.1.smithi178.stdout:4/574: write d0/d9/dc/f8c [1961698,19199] 0 2024-09-10T11:29:34.986 INFO:tasks.workunit.client.0.smithi144.stdout:7/310: truncate d5/f8 5039484 0 2024-09-10T11:29:34.986 INFO:tasks.workunit.client.0.smithi144.stdout:9/367: dwrite d11/d36/d3e/f87 [0,4194304] 0 2024-09-10T11:29:34.987 INFO:tasks.workunit.client.0.smithi144.stdout:6/274: creat d9/d38/f63 x:0 0 0 2024-09-10T11:29:34.987 INFO:tasks.workunit.client.1.smithi178.stdout:9/725: write d2/d8/fc5 [2333241,89783] 0 2024-09-10T11:29:34.988 INFO:tasks.workunit.client.0.smithi144.stdout:3/361: write d4/dd/d21/f42 [193713,94882] 0 2024-09-10T11:29:34.988 INFO:tasks.workunit.client.1.smithi178.stdout:1/617: mknod d3/d9/ce4 0 2024-09-10T11:29:34.989 INFO:tasks.workunit.client.0.smithi144.stdout:1/423: read d5/d2a/f2d [1383496,79026] 0 2024-09-10T11:29:34.990 INFO:tasks.workunit.client.0.smithi144.stdout:4/290: creat d14/d1e/d22/d2b/f64 x:0 0 0 2024-09-10T11:29:34.992 INFO:tasks.workunit.client.1.smithi178.stdout:1/618: chown d3/d9/d98/dac/d3a/d50/db5/fc9 43021 1 2024-09-10T11:29:34.992 INFO:tasks.workunit.client.0.smithi144.stdout:6/275: creat d9/d1f/d2c/f64 x:0 0 0 2024-09-10T11:29:34.993 INFO:tasks.workunit.client.0.smithi144.stdout:8/204: sync 2024-09-10T11:29:34.993 INFO:tasks.workunit.client.0.smithi144.stdout:0/224: sync 2024-09-10T11:29:34.993 INFO:tasks.workunit.client.0.smithi144.stdout:2/254: sync 2024-09-10T11:29:34.995 INFO:tasks.workunit.client.0.smithi144.stdout:4/291: mkdir d14/d1e/d65 0 2024-09-10T11:29:34.995 INFO:tasks.workunit.client.1.smithi178.stdout:9/726: rename d2/d13/c38 to d2/df/d31/d4a/cda 0 2024-09-10T11:29:34.996 INFO:tasks.workunit.client.0.smithi144.stdout:4/292: fsync d14/d1e/d22/d34/d3e/d40/f62 0 2024-09-10T11:29:34.996 INFO:tasks.workunit.client.1.smithi178.stdout:2/896: truncate d1/d41/d5d/f116 37401 0 2024-09-10T11:29:34.999 INFO:tasks.workunit.client.1.smithi178.stdout:5/700: sync 2024-09-10T11:29:35.000 INFO:tasks.workunit.client.1.smithi178.stdout:1/619: mknod d3/d9/d98/dac/d3a/d50/d6c/d78/db8/ce5 0 2024-09-10T11:29:35.001 INFO:tasks.workunit.client.1.smithi178.stdout:2/897: write d1/d7/f16 [2977915,73019] 0 2024-09-10T11:29:35.001 INFO:tasks.workunit.client.0.smithi144.stdout:1/424: creat d5/df/d4b/d85/f8b x:0 0 0 2024-09-10T11:29:35.002 INFO:tasks.workunit.client.0.smithi144.stdout:0/225: rmdir da/d3a 39 2024-09-10T11:29:35.002 INFO:tasks.workunit.client.0.smithi144.stdout:4/293: dwrite f9 [8388608,4194304] 0 2024-09-10T11:29:35.003 INFO:tasks.workunit.client.0.smithi144.stdout:2/255: truncate d7/de/f2b 4754214 0 2024-09-10T11:29:35.003 INFO:tasks.workunit.client.1.smithi178.stdout:1/620: dread - d3/d9/d98/fca zero size 2024-09-10T11:29:35.005 INFO:tasks.workunit.client.1.smithi178.stdout:3/622: dwrite db/d20/d3d/d63/d6a/f84 [0,4194304] 0 2024-09-10T11:29:35.005 INFO:tasks.workunit.client.1.smithi178.stdout:5/701: dread - d9/dc/dd/d37/dbd/dda/db8/fe0 zero size 2024-09-10T11:29:35.005 INFO:tasks.workunit.client.0.smithi144.stdout:8/205: rename d0/d9/d1f/d34/l38 to d0/d9/d21/l46 0 2024-09-10T11:29:35.006 INFO:tasks.workunit.client.0.smithi144.stdout:7/311: link d5/dc/d11/d18/l23 d5/dc/d2a/l64 0 2024-09-10T11:29:35.008 INFO:tasks.workunit.client.1.smithi178.stdout:1/621: creat d3/d35/fe6 x:0 0 0 2024-09-10T11:29:35.009 INFO:tasks.workunit.client.1.smithi178.stdout:3/623: mkdir db/d20/d3e/dc5 0 2024-09-10T11:29:35.010 INFO:tasks.workunit.client.1.smithi178.stdout:0/674: dread d0/dc/f2b [0,4194304] 0 2024-09-10T11:29:35.013 INFO:tasks.workunit.client.0.smithi144.stdout:4/294: rmdir d14/d1e 39 2024-09-10T11:29:35.015 INFO:tasks.workunit.client.1.smithi178.stdout:1/622: creat d3/d9/d98/dac/d3a/d6e/fe7 x:0 0 0 2024-09-10T11:29:35.016 INFO:tasks.workunit.client.1.smithi178.stdout:9/727: write d2/df/f28 [3724218,118274] 0 2024-09-10T11:29:35.017 INFO:tasks.workunit.client.0.smithi144.stdout:7/312: unlink d5/dc/d12/f53 0 2024-09-10T11:29:35.017 INFO:tasks.workunit.client.0.smithi144.stdout:9/368: getdents d11/d12/d1b/d22/d53 0 2024-09-10T11:29:35.019 INFO:tasks.workunit.client.0.smithi144.stdout:3/362: getdents d4/d28/d67/d80 0 2024-09-10T11:29:35.019 INFO:tasks.workunit.client.1.smithi178.stdout:0/675: mknod d0/d2c/cd8 0 2024-09-10T11:29:35.019 INFO:tasks.workunit.client.1.smithi178.stdout:1/623: read d3/d9/d98/dd1/d7e/f8c [2287572,61644] 0 2024-09-10T11:29:35.019 INFO:tasks.workunit.client.1.smithi178.stdout:9/728: chown d2/c8b 404237 1 2024-09-10T11:29:35.020 INFO:tasks.workunit.client.0.smithi144.stdout:3/363: stat d4/dd/d21/l3c 0 2024-09-10T11:29:35.023 INFO:tasks.workunit.client.1.smithi178.stdout:5/702: dread d9/dc/d17/d1d/fb1 [0,4194304] 0 2024-09-10T11:29:35.025 INFO:tasks.workunit.client.1.smithi178.stdout:3/624: getdents db/d26 0 2024-09-10T11:29:35.027 INFO:tasks.workunit.client.1.smithi178.stdout:5/703: write d9/dc/dd/d37/d42/d62/f80 [358844,67159] 0 2024-09-10T11:29:35.028 INFO:tasks.workunit.client.1.smithi178.stdout:3/625: dread - db/d20/f8a zero size 2024-09-10T11:29:35.029 INFO:tasks.workunit.client.0.smithi144.stdout:7/313: rmdir d5/dc/d11 39 2024-09-10T11:29:35.030 INFO:tasks.workunit.client.0.smithi144.stdout:9/369: dwrite d11/d12/d1b/d2c/f31 [0,4194304] 0 2024-09-10T11:29:35.031 INFO:tasks.workunit.client.1.smithi178.stdout:1/624: link d3/d9/d98/dac/d3a/c3e d3/d9/d98/dac/d3a/dae/ce8 0 2024-09-10T11:29:35.032 INFO:tasks.workunit.client.0.smithi144.stdout:1/425: dwrite d5/d4a/d44/d80/f67 [0,4194304] 0 2024-09-10T11:29:35.034 INFO:tasks.workunit.client.0.smithi144.stdout:9/370: write d11/d12/d1b/d22/d53/f54 [406251,36763] 0 2024-09-10T11:29:35.034 INFO:tasks.workunit.client.0.smithi144.stdout:0/226: creat da/f41 x:0 0 0 2024-09-10T11:29:35.035 INFO:tasks.workunit.client.1.smithi178.stdout:3/626: mknod db/d26/d74/cc6 0 2024-09-10T11:29:35.035 INFO:tasks.workunit.client.0.smithi144.stdout:1/426: truncate d5/d2a/d65/f6b 817863 0 2024-09-10T11:29:35.035 INFO:tasks.workunit.client.0.smithi144.stdout:0/227: fsync da/f32 0 2024-09-10T11:29:35.037 INFO:tasks.workunit.client.0.smithi144.stdout:9/371: chown d11/d12/l5f 1086821 1 2024-09-10T11:29:35.037 INFO:tasks.workunit.client.0.smithi144.stdout:3/364: rmdir d4/d28/d67/d80/d17 39 2024-09-10T11:29:35.040 INFO:tasks.workunit.client.0.smithi144.stdout:9/372: fdatasync d11/d36/d3e/f82 0 2024-09-10T11:29:35.040 INFO:tasks.workunit.client.1.smithi178.stdout:2/898: dwrite d1/d22/f26 [0,4194304] 0 2024-09-10T11:29:35.041 INFO:tasks.workunit.client.0.smithi144.stdout:7/314: unlink d5/dc/d12/f1f 0 2024-09-10T11:29:35.042 INFO:tasks.workunit.client.0.smithi144.stdout:7/315: chown d5 177319481 1 2024-09-10T11:29:35.042 INFO:tasks.workunit.client.0.smithi144.stdout:8/206: dread d0/d9/ff [0,4194304] 0 2024-09-10T11:29:35.044 INFO:tasks.workunit.client.0.smithi144.stdout:1/427: symlink d5/df/d4b/l8c 0 2024-09-10T11:29:35.045 INFO:tasks.workunit.client.0.smithi144.stdout:1/428: fsync d5/d2a/f34 0 2024-09-10T11:29:35.045 INFO:tasks.workunit.client.0.smithi144.stdout:2/256: link d7/da/f12 d7/de/d3d/f52 0 2024-09-10T11:29:35.045 INFO:tasks.workunit.client.1.smithi178.stdout:3/627: mknod db/d20/cc7 0 2024-09-10T11:29:35.047 INFO:tasks.workunit.client.0.smithi144.stdout:1/429: readlink d5/df/d4b/d51/l5f 0 2024-09-10T11:29:35.048 INFO:tasks.workunit.client.0.smithi144.stdout:2/257: dread - d7/de/f3b zero size 2024-09-10T11:29:35.048 INFO:tasks.workunit.client.0.smithi144.stdout:1/430: fsync d5/d4a/d44/f75 0 2024-09-10T11:29:35.048 INFO:tasks.workunit.client.1.smithi178.stdout:3/628: chown la 2252097 1 2024-09-10T11:29:35.048 INFO:tasks.workunit.client.1.smithi178.stdout:1/625: dwrite d3/d35/fb4 [4194304,4194304] 0 2024-09-10T11:29:35.051 INFO:tasks.workunit.client.1.smithi178.stdout:5/704: dwrite d9/dc/dd/d3f/d5a/fd3 [0,4194304] 0 2024-09-10T11:29:35.052 INFO:tasks.workunit.client.0.smithi144.stdout:4/295: symlink d14/d1e/d22/d2b/l66 0 2024-09-10T11:29:35.053 INFO:tasks.workunit.client.0.smithi144.stdout:9/373: chown d11/d12/d1b/d22/l29 903362 1 2024-09-10T11:29:35.054 INFO:tasks.workunit.client.0.smithi144.stdout:0/228: symlink da/d3a/l42 0 2024-09-10T11:29:35.055 INFO:tasks.workunit.client.0.smithi144.stdout:8/207: mkdir d0/d1b/d47 0 2024-09-10T11:29:35.055 INFO:tasks.workunit.client.1.smithi178.stdout:3/629: write db/d36/f50 [4147355,58111] 0 2024-09-10T11:29:35.055 INFO:tasks.workunit.client.0.smithi144.stdout:7/316: chown d5/dc/d11/c3b 4225 1 2024-09-10T11:29:35.057 INFO:tasks.workunit.client.0.smithi144.stdout:0/229: fdatasync da/d3a/f3e 0 2024-09-10T11:29:35.058 INFO:tasks.workunit.client.0.smithi144.stdout:2/258: creat d7/da/d1e/d21/f53 x:0 0 0 2024-09-10T11:29:35.059 INFO:tasks.workunit.client.1.smithi178.stdout:5/705: truncate d9/d19/f66 977511 0 2024-09-10T11:29:35.059 INFO:tasks.workunit.client.1.smithi178.stdout:1/626: mkdir d3/d9/d98/dac/d3a/d50/d6c/de9 0 2024-09-10T11:29:35.060 INFO:tasks.workunit.client.1.smithi178.stdout:9/729: dread d2/f18 [0,4194304] 0 2024-09-10T11:29:35.060 INFO:tasks.workunit.client.1.smithi178.stdout:3/630: symlink db/d26/d40/lc8 0 2024-09-10T11:29:35.061 INFO:tasks.workunit.client.0.smithi144.stdout:8/208: write d0/d9/d23/d26/f36 [1403195,47374] 0 2024-09-10T11:29:35.064 INFO:tasks.workunit.client.1.smithi178.stdout:1/627: read - d3/d9/d98/dac/f7f zero size 2024-09-10T11:29:35.064 INFO:tasks.workunit.client.1.smithi178.stdout:3/631: symlink db/d20/lc9 0 2024-09-10T11:29:35.065 INFO:tasks.workunit.client.1.smithi178.stdout:5/706: creat d9/dc/d17/db0/feb x:0 0 0 2024-09-10T11:29:35.066 INFO:tasks.workunit.client.1.smithi178.stdout:9/730: chown d2/df/f3b 150212670 1 2024-09-10T11:29:35.066 INFO:tasks.workunit.client.0.smithi144.stdout:2/259: unlink d7/de/d36/c40 0 2024-09-10T11:29:35.066 INFO:tasks.workunit.client.0.smithi144.stdout:7/317: mknod d5/dc/d12/d2b/c65 0 2024-09-10T11:29:35.067 INFO:tasks.workunit.client.1.smithi178.stdout:2/899: write d1/d41/d14/d11/f2a [3816243,48402] 0 2024-09-10T11:29:35.067 INFO:tasks.workunit.client.1.smithi178.stdout:9/731: chown d2/df/c11 12 1 2024-09-10T11:29:35.068 INFO:tasks.workunit.client.0.smithi144.stdout:7/318: chown d5/dc 4 1 2024-09-10T11:29:35.068 INFO:tasks.workunit.client.1.smithi178.stdout:1/628: creat d3/d9/d98/dac/d3a/d9f/fea x:0 0 0 2024-09-10T11:29:35.070 INFO:tasks.workunit.client.0.smithi144.stdout:3/365: dwrite d4/d28/d67/d80/fc [0,4194304] 0 2024-09-10T11:29:35.070 INFO:tasks.workunit.client.1.smithi178.stdout:3/632: truncate db/d20/d3e/d6b/fa7 30634 0 2024-09-10T11:29:35.071 INFO:tasks.workunit.client.0.smithi144.stdout:2/260: mkdir d7/da/d1e/d21/d54 0 2024-09-10T11:29:35.072 INFO:tasks.workunit.client.0.smithi144.stdout:4/296: dwrite d14/d1e/d22/d2b/d35/d4f/f60 [0,4194304] 0 2024-09-10T11:29:35.072 INFO:tasks.workunit.client.1.smithi178.stdout:9/732: creat d2/d13/d40/d89/d93/d88/dae/fdb x:0 0 0 2024-09-10T11:29:35.074 INFO:tasks.workunit.client.0.smithi144.stdout:8/209: rmdir d0/d9/d23/d3d 0 2024-09-10T11:29:35.074 INFO:tasks.workunit.client.1.smithi178.stdout:1/629: rmdir d3/d9/d1a/d48 39 2024-09-10T11:29:35.076 INFO:tasks.workunit.client.0.smithi144.stdout:4/297: fdatasync d14/d1e/d22/f4a 0 2024-09-10T11:29:35.076 INFO:tasks.workunit.client.1.smithi178.stdout:3/633: symlink db/d20/d3d/d63/da2/d98/lca 0 2024-09-10T11:29:35.076 INFO:tasks.workunit.client.1.smithi178.stdout:1/630: write d3/d9/fa [2864526,98264] 0 2024-09-10T11:29:35.077 INFO:tasks.workunit.client.1.smithi178.stdout:9/733: symlink d2/d13/d40/d89/d93/ldc 0 2024-09-10T11:29:35.078 INFO:tasks.workunit.client.0.smithi144.stdout:7/319: rename d5/dc/f20 to d5/dc/d14/d40/d5e/d63/f66 0 2024-09-10T11:29:35.082 INFO:tasks.workunit.client.1.smithi178.stdout:9/734: fsync d2/d13/d40/d89/d93/d88/dae/fdb 0 2024-09-10T11:29:35.084 INFO:tasks.workunit.client.0.smithi144.stdout:3/366: dwrite d4/dd/d21/f5b [0,4194304] 0 2024-09-10T11:29:35.085 INFO:tasks.workunit.client.1.smithi178.stdout:1/631: mknod d3/d9/d98/dac/d8d/ceb 0 2024-09-10T11:29:35.086 INFO:tasks.workunit.client.1.smithi178.stdout:3/634: rename db/d20/d3e/d6b/d79 to db/d26/d40/d68/db6/dcb 0 2024-09-10T11:29:35.087 INFO:tasks.workunit.client.1.smithi178.stdout:9/735: write d2/df/d31/d37/d3d/fc7 [730992,11820] 0 2024-09-10T11:29:35.087 INFO:tasks.workunit.client.0.smithi144.stdout:7/320: creat d5/dc/d14/d40/f67 x:0 0 0 2024-09-10T11:29:35.088 INFO:tasks.workunit.client.1.smithi178.stdout:3/635: readlink db/l12 0 2024-09-10T11:29:35.088 INFO:tasks.workunit.client.0.smithi144.stdout:3/367: fsync d4/dd/d21/f5d 0 2024-09-10T11:29:35.089 INFO:tasks.workunit.client.1.smithi178.stdout:1/632: write d3/d9/d98/dac/d3a/d6e/fe7 [751348,63264] 0 2024-09-10T11:29:35.089 INFO:tasks.workunit.client.0.smithi144.stdout:8/210: dwrite d0/d9/d23/d26/f36 [0,4194304] 0 2024-09-10T11:29:35.090 INFO:tasks.workunit.client.1.smithi178.stdout:3/636: chown db/f16 57 1 2024-09-10T11:29:35.091 INFO:tasks.workunit.client.0.smithi144.stdout:0/230: dread da/f1c [0,4194304] 0 2024-09-10T11:29:35.094 INFO:tasks.workunit.client.1.smithi178.stdout:5/707: dread d9/dc/dd/d3f/f8b [0,4194304] 0 2024-09-10T11:29:35.094 INFO:tasks.workunit.client.1.smithi178.stdout:5/708: stat d9/d11/dc3 0 2024-09-10T11:29:35.094 INFO:tasks.workunit.client.0.smithi144.stdout:4/298: mkdir d14/d67 0 2024-09-10T11:29:35.103 INFO:tasks.workunit.client.0.smithi144.stdout:7/321: dwrite d5/dc/d12/f52 [0,4194304] 0 2024-09-10T11:29:35.103 INFO:tasks.workunit.client.0.smithi144.stdout:8/211: mkdir d0/d9/d23/d37/d48 0 2024-09-10T11:29:35.104 INFO:tasks.workunit.client.0.smithi144.stdout:4/299: truncate d14/d1e/d22/f4a 73603 0 2024-09-10T11:29:35.107 INFO:tasks.workunit.client.0.smithi144.stdout:3/368: truncate d4/d28/d67/d80/d17/d18/f6d 821738 0 2024-09-10T11:29:35.108 INFO:tasks.workunit.client.0.smithi144.stdout:9/374: dwrite d11/f15 [0,4194304] 0 2024-09-10T11:29:35.116 INFO:tasks.workunit.client.0.smithi144.stdout:7/322: rmdir d5/dc/d14/d40 39 2024-09-10T11:29:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:34 smithi178 ceph-mon[33514]: pgmap v24: 65 pgs: 65 active+clean; 2.0 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 67 MiB/s rd, 276 MiB/s wr, 517 op/s 2024-09-10T11:29:35.118 INFO:tasks.workunit.client.0.smithi144.stdout:1/431: dread d5/df/d4b/d51/f59 [0,4194304] 0 2024-09-10T11:29:35.120 INFO:tasks.workunit.client.0.smithi144.stdout:4/300: chown d14/c25 381541536 1 2024-09-10T11:29:35.122 INFO:tasks.workunit.client.0.smithi144.stdout:0/231: symlink da/l43 0 2024-09-10T11:29:35.127 INFO:tasks.workunit.client.0.smithi144.stdout:4/301: mknod d14/d1e/d22/d2b/d35/c68 0 2024-09-10T11:29:35.128 INFO:tasks.workunit.client.0.smithi144.stdout:9/375: dwrite d11/d12/d1b/f77 [0,4194304] 0 2024-09-10T11:29:35.131 INFO:tasks.workunit.client.0.smithi144.stdout:0/232: mkdir da/d44 0 2024-09-10T11:29:35.132 INFO:tasks.workunit.client.0.smithi144.stdout:8/212: link d0/d9/d1f/l1a d0/d1b/l49 0 2024-09-10T11:29:35.133 INFO:tasks.workunit.client.0.smithi144.stdout:7/323: mkdir d5/dc/d14/d40/d5e/d68 0 2024-09-10T11:29:35.135 INFO:tasks.workunit.client.0.smithi144.stdout:1/432: dwrite d5/fc [8388608,4194304] 0 2024-09-10T11:29:35.138 INFO:tasks.workunit.client.0.smithi144.stdout:3/369: dwrite d4/dd/d21/f41 [0,4194304] 0 2024-09-10T11:29:35.138 INFO:tasks.workunit.client.0.smithi144.stdout:4/302: mkdir d14/d67/d69 0 2024-09-10T11:29:35.145 INFO:tasks.workunit.client.0.smithi144.stdout:9/376: dwrite d11/d12/d1b/d22/f63 [0,4194304] 0 2024-09-10T11:29:35.146 INFO:tasks.workunit.client.0.smithi144.stdout:0/233: write f2 [195499,94726] 0 2024-09-10T11:29:35.147 INFO:tasks.workunit.client.0.smithi144.stdout:7/324: write f1 [218621,50339] 0 2024-09-10T11:29:35.153 INFO:tasks.workunit.client.0.smithi144.stdout:7/325: dread - d5/dc/d11/d18/f5f zero size 2024-09-10T11:29:35.156 INFO:tasks.workunit.client.0.smithi144.stdout:4/303: dread d14/d1e/d22/f37 [0,4194304] 0 2024-09-10T11:29:35.156 INFO:tasks.workunit.client.0.smithi144.stdout:3/370: truncate d4/d28/d67/d80/d17/d18/d34/d4b/d63/f74 457582 0 2024-09-10T11:29:35.156 INFO:tasks.workunit.client.0.smithi144.stdout:1/433: creat d5/dd/f8d x:0 0 0 2024-09-10T11:29:35.157 INFO:tasks.workunit.client.0.smithi144.stdout:4/304: write d14/f54 [677427,49931] 0 2024-09-10T11:29:35.163 INFO:tasks.workunit.client.1.smithi178.stdout:3/637: mkdir db/d26/d40/dcc 0 2024-09-10T11:29:35.163 INFO:tasks.workunit.client.0.smithi144.stdout:1/434: rename d5/df/f19 to d5/df/d4b/f8e 0 2024-09-10T11:29:35.165 INFO:tasks.workunit.client.0.smithi144.stdout:1/435: chown d5/f12 4 1 2024-09-10T11:29:35.170 INFO:tasks.workunit.client.0.smithi144.stdout:7/326: rename d5/c28 to d5/dc/d2a/d60/c69 0 2024-09-10T11:29:35.171 INFO:tasks.workunit.client.1.smithi178.stdout:3/638: dwrite db/d20/f8a [0,4194304] 0 2024-09-10T11:29:35.172 INFO:tasks.workunit.client.0.smithi144.stdout:8/213: dread d0/f10 [0,4194304] 0 2024-09-10T11:29:35.176 INFO:tasks.workunit.client.0.smithi144.stdout:4/305: dwrite d14/d1e/d22/d34/d3e/d40/d4b/f57 [0,4194304] 0 2024-09-10T11:29:35.180 INFO:tasks.workunit.client.0.smithi144.stdout:7/327: mkdir d5/dc/d14/d40/d58/d6a 0 2024-09-10T11:29:35.180 INFO:tasks.workunit.client.0.smithi144.stdout:8/214: readlink d0/d9/d21/l27 0 2024-09-10T11:29:35.180 INFO:tasks.workunit.client.0.smithi144.stdout:7/328: dread - d5/f5a zero size 2024-09-10T11:29:35.182 INFO:tasks.workunit.client.0.smithi144.stdout:8/215: write d0/d9/d1f/f1c [3093667,98863] 0 2024-09-10T11:29:35.182 INFO:tasks.workunit.client.0.smithi144.stdout:1/436: dread d5/d4a/d44/d80/f67 [0,4194304] 0 2024-09-10T11:29:35.184 INFO:tasks.workunit.client.0.smithi144.stdout:4/306: creat d14/d1e/d22/d34/d3e/d40/d4b/f6a x:0 0 0 2024-09-10T11:29:35.185 INFO:tasks.workunit.client.0.smithi144.stdout:4/307: chown d14/d67 300230 1 2024-09-10T11:29:35.187 INFO:tasks.workunit.client.1.smithi178.stdout:5/709: dread d9/dc/dd/d37/dbd/dda/fa9 [0,4194304] 0 2024-09-10T11:29:35.187 INFO:tasks.workunit.client.0.smithi144.stdout:9/377: dread d11/d12/d1b/d22/f3d [0,4194304] 0 2024-09-10T11:29:35.188 INFO:tasks.workunit.client.1.smithi178.stdout:3/639: rmdir db/d20/d3d/d63/da2/d98 39 2024-09-10T11:29:35.189 INFO:tasks.workunit.client.0.smithi144.stdout:3/371: truncate d4/dd/d21/f23 3096705 0 2024-09-10T11:29:35.190 INFO:tasks.workunit.client.0.smithi144.stdout:8/216: chown d0/d9/d21/l46 4190978 1 2024-09-10T11:29:35.192 INFO:tasks.workunit.client.0.smithi144.stdout:4/308: creat d14/d67/f6b x:0 0 0 2024-09-10T11:29:35.193 INFO:tasks.workunit.client.0.smithi144.stdout:0/234: dread f5 [0,4194304] 0 2024-09-10T11:29:35.194 INFO:tasks.workunit.client.0.smithi144.stdout:1/437: mkdir d5/dd/d8f 0 2024-09-10T11:29:35.195 INFO:tasks.workunit.client.1.smithi178.stdout:5/710: mknod d9/dc/dd/d3f/d5a/cec 0 2024-09-10T11:29:35.197 INFO:tasks.workunit.client.1.smithi178.stdout:5/711: write d9/dc/dd/d37/d42/d55/d6d/dcc/fd2 [7531460,125121] 0 2024-09-10T11:29:35.197 INFO:tasks.workunit.client.0.smithi144.stdout:9/378: rename d11/d12/d1b/d22/d53/d64/c69 to d11/d12/d1b/d2c/d78/c89 0 2024-09-10T11:29:35.197 INFO:tasks.workunit.client.1.smithi178.stdout:5/712: fsync d9/dc/d17/d1d/fce 0 2024-09-10T11:29:35.198 INFO:tasks.workunit.client.0.smithi144.stdout:7/329: creat d5/dc/d14/f6b x:0 0 0 2024-09-10T11:29:35.199 INFO:tasks.workunit.client.1.smithi178.stdout:5/713: stat d9/dc/dd/d37/d3e/l85 0 2024-09-10T11:29:35.200 INFO:tasks.workunit.client.1.smithi178.stdout:5/714: write d9/dc/dd/d37/dbd/dda/db8/fd4 [1226971,59777] 0 2024-09-10T11:29:35.203 INFO:tasks.workunit.client.0.smithi144.stdout:1/438: dread d5/d4a/f2f [0,4194304] 0 2024-09-10T11:29:35.204 INFO:tasks.workunit.client.0.smithi144.stdout:9/379: mkdir d11/d12/d1b/d2c/d8a 0 2024-09-10T11:29:35.205 INFO:tasks.workunit.client.0.smithi144.stdout:7/330: rmdir d5/dc/d2a/d60 39 2024-09-10T11:29:35.206 INFO:tasks.workunit.client.0.smithi144.stdout:8/217: dread d0/fc [0,4194304] 0 2024-09-10T11:29:35.207 INFO:tasks.workunit.client.0.smithi144.stdout:7/331: chown d5/dc/d14/d40/d5e 0 1 2024-09-10T11:29:35.208 INFO:tasks.workunit.client.0.smithi144.stdout:8/218: truncate d0/d9/d1f/f1c 5196708 0 2024-09-10T11:29:35.208 INFO:tasks.workunit.client.0.smithi144.stdout:7/332: read d5/dc/d11/f5c [3507927,43188] 0 2024-09-10T11:29:35.209 INFO:tasks.workunit.client.1.smithi178.stdout:5/715: getdents d9/dc/dd/d37/d42/d55 0 2024-09-10T11:29:35.209 INFO:tasks.workunit.client.0.smithi144.stdout:0/235: mknod da/d44/c45 0 2024-09-10T11:29:35.209 INFO:tasks.workunit.client.0.smithi144.stdout:8/219: chown d0/d9/f1d 176 1 2024-09-10T11:29:35.211 INFO:tasks.workunit.client.0.smithi144.stdout:1/439: mkdir d5/dd/d64/d79/d90 0 2024-09-10T11:29:35.212 INFO:tasks.workunit.client.0.smithi144.stdout:1/440: write d5/d2a/d65/d73/f24 [849552,31844] 0 2024-09-10T11:29:35.218 INFO:tasks.workunit.client.0.smithi144.stdout:8/220: fdatasync d0/d9/d21/d25/f29 0 2024-09-10T11:29:35.219 INFO:tasks.workunit.client.0.smithi144.stdout:4/309: dwrite d14/d1e/d22/f31 [0,4194304] 0 2024-09-10T11:29:35.220 INFO:tasks.workunit.client.0.smithi144.stdout:3/372: write d4/dd/d21/f23 [2140277,118586] 0 2024-09-10T11:29:35.220 INFO:tasks.workunit.client.1.smithi178.stdout:8/780: sync 2024-09-10T11:29:35.220 INFO:tasks.workunit.client.1.smithi178.stdout:7/637: sync 2024-09-10T11:29:35.220 INFO:tasks.workunit.client.1.smithi178.stdout:6/674: sync 2024-09-10T11:29:35.221 INFO:tasks.workunit.client.1.smithi178.stdout:5/716: dread d9/dc/dd/d37/f75 [0,4194304] 0 2024-09-10T11:29:35.226 INFO:tasks.workunit.client.1.smithi178.stdout:6/675: creat d3/d7/d8/d15/d1c/d2a/d18/d8e/ff5 x:0 0 0 2024-09-10T11:29:35.229 INFO:tasks.workunit.client.1.smithi178.stdout:6/676: read d3/fa0 [598005,49828] 0 2024-09-10T11:29:35.230 INFO:tasks.workunit.client.0.smithi144.stdout:9/380: truncate d11/d12/d1b/f20 621067 0 2024-09-10T11:29:35.231 INFO:tasks.workunit.client.1.smithi178.stdout:8/781: dwrite d5/db/d10/d53/f64 [0,4194304] 0 2024-09-10T11:29:35.231 INFO:tasks.workunit.client.0.smithi144.stdout:9/381: fdatasync d11/d12/d1b/d2c/f31 0 2024-09-10T11:29:35.237 INFO:tasks.workunit.client.1.smithi178.stdout:9/736: rename d2/df/f9c to d2/d13/d75/d60/fdd 0 2024-09-10T11:29:35.237 INFO:tasks.workunit.client.1.smithi178.stdout:8/782: dread d5/db/d10/d35/f100 [0,4194304] 0 2024-09-10T11:29:35.237 INFO:tasks.workunit.client.0.smithi144.stdout:0/236: rename da/f2b to da/f46 0 2024-09-10T11:29:35.239 INFO:tasks.workunit.client.0.smithi144.stdout:8/221: fsync d0/d9/d21/f2b 0 2024-09-10T11:29:35.240 INFO:tasks.workunit.client.1.smithi178.stdout:8/783: symlink d5/db/d10/d35/l106 0 2024-09-10T11:29:35.240 INFO:tasks.workunit.client.0.smithi144.stdout:1/441: symlink d5/d4a/d63/d84/l91 0 2024-09-10T11:29:35.242 INFO:tasks.workunit.client.0.smithi144.stdout:4/310: fdatasync d14/d1e/d22/f31 0 2024-09-10T11:29:35.242 INFO:tasks.workunit.client.1.smithi178.stdout:9/737: creat d2/d13/d75/d60/dd0/fde x:0 0 0 2024-09-10T11:29:35.242 INFO:tasks.workunit.client.1.smithi178.stdout:9/738: stat d2/c8b 0 2024-09-10T11:29:35.243 INFO:tasks.workunit.client.0.smithi144.stdout:3/373: rename d4/c57 to d4/dd/d21/c82 0 2024-09-10T11:29:35.244 INFO:tasks.workunit.client.0.smithi144.stdout:7/333: write d5/dc/d2a/f4b [570013,79923] 0 2024-09-10T11:29:35.244 INFO:tasks.workunit.client.1.smithi178.stdout:8/784: mknod d5/d30/d33/d8c/c107 0 2024-09-10T11:29:35.246 INFO:tasks.workunit.client.1.smithi178.stdout:9/739: mknod d2/dc/cdf 0 2024-09-10T11:29:35.248 INFO:tasks.workunit.client.1.smithi178.stdout:8/785: creat d5/db/d1e/f108 x:0 0 0 2024-09-10T11:29:35.249 INFO:tasks.workunit.client.1.smithi178.stdout:9/740: mknod d2/d19/d5e/ce0 0 2024-09-10T11:29:35.249 INFO:tasks.workunit.client.0.smithi144.stdout:4/311: dwrite d14/f3a [4194304,4194304] 0 2024-09-10T11:29:35.251 INFO:tasks.workunit.client.1.smithi178.stdout:8/786: fsync d5/db/d10/d35/f100 0 2024-09-10T11:29:35.251 INFO:tasks.workunit.client.0.smithi144.stdout:8/222: creat d0/d9/d23/d26/f4a x:0 0 0 2024-09-10T11:29:35.252 INFO:tasks.workunit.client.1.smithi178.stdout:9/741: mkdir d2/d13/d40/d56/de1 0 2024-09-10T11:29:35.254 INFO:tasks.workunit.client.1.smithi178.stdout:9/742: truncate d2/d19/d1c/d2d/f55 87267 0 2024-09-10T11:29:35.254 INFO:tasks.workunit.client.1.smithi178.stdout:8/787: mkdir d5/db/d10/d20/dc0/d109 0 2024-09-10T11:29:35.255 INFO:tasks.workunit.client.1.smithi178.stdout:3/640: mkdir db/d26/dcd 0 2024-09-10T11:29:35.257 INFO:tasks.workunit.client.0.smithi144.stdout:3/374: rmdir d4/dd 39 2024-09-10T11:29:35.258 INFO:tasks.workunit.client.1.smithi178.stdout:1/633: mknod d3/d9/d98/dac/d3a/d50/cec 0 2024-09-10T11:29:35.259 INFO:tasks.workunit.client.1.smithi178.stdout:5/717: dread d9/dc/dd/d16/f8e [0,4194304] 0 2024-09-10T11:29:35.259 INFO:tasks.workunit.client.1.smithi178.stdout:8/788: getdents d5/def 0 2024-09-10T11:29:35.260 INFO:tasks.workunit.client.0.smithi144.stdout:1/442: write d5/df/f33 [4473167,108190] 0 2024-09-10T11:29:35.260 INFO:tasks.workunit.client.0.smithi144.stdout:3/375: write d4/d28/d67/d80/d17/d18/d34/d4b/d63/f7b [52251,32482] 0 2024-09-10T11:29:35.260 INFO:tasks.workunit.client.1.smithi178.stdout:1/634: fsync d3/d9/d98/dd1/d7e/f8c 0 2024-09-10T11:29:35.261 INFO:tasks.workunit.client.0.smithi144.stdout:1/443: fdatasync d5/d2a/f7e 0 2024-09-10T11:29:35.261 INFO:tasks.workunit.client.1.smithi178.stdout:8/789: write d5/d1d/fdc [829465,31799] 0 2024-09-10T11:29:35.262 INFO:tasks.workunit.client.1.smithi178.stdout:9/743: symlink d2/d13/d40/d89/d93/dc2/le2 0 2024-09-10T11:29:35.263 INFO:tasks.workunit.client.1.smithi178.stdout:3/641: write db/d20/d3d/d56/d64/fae [1089977,21128] 0 2024-09-10T11:29:35.263 INFO:tasks.workunit.client.0.smithi144.stdout:7/334: rmdir d5 39 2024-09-10T11:29:35.265 INFO:tasks.workunit.client.1.smithi178.stdout:6/677: symlink d3/lf6 0 2024-09-10T11:29:35.267 INFO:tasks.workunit.client.1.smithi178.stdout:8/790: truncate d5/db/d3d/feb 395141 0 2024-09-10T11:29:35.269 INFO:tasks.workunit.client.0.smithi144.stdout:0/237: rename da/c2f to da/c47 0 2024-09-10T11:29:35.269 INFO:tasks.workunit.client.1.smithi178.stdout:9/744: creat d2/d13/fe3 x:0 0 0 2024-09-10T11:29:35.269 INFO:tasks.workunit.client.0.smithi144.stdout:0/238: stat da/c10 0 2024-09-10T11:29:35.271 INFO:tasks.workunit.client.0.smithi144.stdout:0/239: chown da/f39 13 1 2024-09-10T11:29:35.272 INFO:tasks.workunit.client.1.smithi178.stdout:5/718: dwrite d9/dc/dd/d3f/d5a/fd3 [0,4194304] 0 2024-09-10T11:29:35.272 INFO:tasks.workunit.client.0.smithi144.stdout:9/382: getdents d11/d12/d1b/d2c 0 2024-09-10T11:29:35.272 INFO:tasks.workunit.client.0.smithi144.stdout:0/240: stat da/l43 0 2024-09-10T11:29:35.273 INFO:tasks.workunit.client.1.smithi178.stdout:8/791: unlink d5/d30/d33/l101 0 2024-09-10T11:29:35.275 INFO:tasks.workunit.client.1.smithi178.stdout:9/745: chown d2/df/d31/d37/d3d/faf 1998 1 2024-09-10T11:29:35.275 INFO:tasks.workunit.client.1.smithi178.stdout:7/638: truncate d6/d10/d1d/d22/d3b/f65 2846864 0 2024-09-10T11:29:35.275 INFO:tasks.workunit.client.0.smithi144.stdout:3/376: creat d4/dd/d21/f83 x:0 0 0 2024-09-10T11:29:35.275 INFO:tasks.workunit.client.0.smithi144.stdout:4/312: dwrite d14/d1e/d22/f37 [0,4194304] 0 2024-09-10T11:29:35.276 INFO:tasks.workunit.client.0.smithi144.stdout:7/335: fdatasync d5/dc/d2a/d41/f49 0 2024-09-10T11:29:35.276 INFO:tasks.workunit.client.0.smithi144.stdout:5/359: sync 2024-09-10T11:29:35.279 INFO:tasks.workunit.client.1.smithi178.stdout:3/642: dwrite db/d20/d3d/d63/da2/d98/fbe [0,4194304] 0 2024-09-10T11:29:35.279 INFO:tasks.workunit.client.1.smithi178.stdout:7/639: write d6/d10/d1d/f73 [3177096,35160] 0 2024-09-10T11:29:35.282 INFO:tasks.workunit.client.1.smithi178.stdout:7/640: chown d6/d10/d1e 65288899 1 2024-09-10T11:29:35.284 INFO:tasks.workunit.client.1.smithi178.stdout:9/746: truncate d2/d8/f73 2109936 0 2024-09-10T11:29:35.285 INFO:tasks.workunit.client.1.smithi178.stdout:8/792: read d5/fae [575789,67519] 0 2024-09-10T11:29:35.287 INFO:tasks.workunit.client.0.smithi144.stdout:1/444: truncate d5/fc 8398686 0 2024-09-10T11:29:35.287 INFO:tasks.workunit.client.1.smithi178.stdout:7/641: symlink d6/d10/d1d/d22/d50/d98/da6/lee 0 2024-09-10T11:29:35.289 INFO:tasks.workunit.client.1.smithi178.stdout:7/642: read d6/d10/d1d/d22/d50/d98/da6/fb4 [644782,4051] 0 2024-09-10T11:29:35.290 INFO:tasks.workunit.client.1.smithi178.stdout:9/747: creat d2/d9b/fe4 x:0 0 0 2024-09-10T11:29:35.290 INFO:tasks.workunit.client.0.smithi144.stdout:9/383: rename d11/d36/f59 to d11/d12/d6f/f8b 0 2024-09-10T11:29:35.292 INFO:tasks.workunit.client.0.smithi144.stdout:8/223: getdents d0/d9/d23/d26 0 2024-09-10T11:29:35.292 INFO:tasks.workunit.client.1.smithi178.stdout:9/748: chown d2/df/d31/d37/d3d/fc7 0 1 2024-09-10T11:29:35.292 INFO:tasks.workunit.client.1.smithi178.stdout:8/793: read d5/db/d10/d35/f3c [3050685,102050] 0 2024-09-10T11:29:35.293 INFO:tasks.workunit.client.1.smithi178.stdout:8/794: fdatasync d5/db/d1e/f39 0 2024-09-10T11:29:35.294 INFO:tasks.workunit.client.0.smithi144.stdout:0/241: mkdir da/d48 0 2024-09-10T11:29:35.295 INFO:tasks.workunit.client.0.smithi144.stdout:0/242: chown da/d3a/f3b 827 1 2024-09-10T11:29:35.297 INFO:tasks.workunit.client.1.smithi178.stdout:3/643: rename db/d20/d70/c76 to db/d26/d40/cce 0 2024-09-10T11:29:35.297 INFO:tasks.workunit.client.0.smithi144.stdout:9/384: stat d11/d12/d1b/d22/d53/d64/l73 0 2024-09-10T11:29:35.298 INFO:tasks.workunit.client.0.smithi144.stdout:9/385: write d11/d12/d1b/f27 [1419380,43231] 0 2024-09-10T11:29:35.300 INFO:tasks.workunit.client.0.smithi144.stdout:4/313: creat d14/d67/d69/f6c x:0 0 0 2024-09-10T11:29:35.301 INFO:tasks.workunit.client.0.smithi144.stdout:4/314: chown d14/d1e/d22/d34/c61 29 1 2024-09-10T11:29:35.301 INFO:tasks.workunit.client.0.smithi144.stdout:7/336: symlink d5/dc/d11/l6c 0 2024-09-10T11:29:35.302 INFO:tasks.workunit.client.1.smithi178.stdout:1/635: dwrite d3/d9/f42 [4194304,4194304] 0 2024-09-10T11:29:35.302 INFO:tasks.workunit.client.1.smithi178.stdout:6/678: truncate d3/d7/d8/d15/d1c/d2a/d18/f70 6733749 0 2024-09-10T11:29:35.302 INFO:tasks.workunit.client.0.smithi144.stdout:4/315: write d14/d1e/d22/f47 [2183917,50313] 0 2024-09-10T11:29:35.303 INFO:tasks.workunit.client.0.smithi144.stdout:7/337: write d5/dc/d2a/d41/f49 [22950,71762] 0 2024-09-10T11:29:35.303 INFO:tasks.workunit.client.0.smithi144.stdout:3/377: rename d4/d28/d67/d80/d1d/f2a to d4/d28/d67/d80/d17/d18/d34/d65/d75/f84 0 2024-09-10T11:29:35.305 INFO:tasks.workunit.client.0.smithi144.stdout:9/386: unlink d11/l16 0 2024-09-10T11:29:35.306 INFO:tasks.workunit.client.0.smithi144.stdout:8/224: getdents d0/d9/d23/d26 0 2024-09-10T11:29:35.306 INFO:tasks.workunit.client.1.smithi178.stdout:8/795: creat d5/db/d10/d20/f10a x:0 0 0 2024-09-10T11:29:35.308 INFO:tasks.workunit.client.1.smithi178.stdout:8/796: chown d5/db/d3d/l6f 40 1 2024-09-10T11:29:35.310 INFO:tasks.workunit.client.0.smithi144.stdout:3/378: mknod d4/dd/d21/c85 0 2024-09-10T11:29:35.310 INFO:tasks.workunit.client.1.smithi178.stdout:6/679: symlink d3/d74/d91/lf7 0 2024-09-10T11:29:35.311 INFO:tasks.workunit.client.0.smithi144.stdout:5/360: getdents d4/d20/d22/d5c/d5d 0 2024-09-10T11:29:35.313 INFO:tasks.workunit.client.1.smithi178.stdout:9/749: dread d2/d19/d1c/f2c [0,4194304] 0 2024-09-10T11:29:35.313 INFO:tasks.workunit.client.1.smithi178.stdout:1/636: dread d3/da0/fc1 [0,4194304] 0 2024-09-10T11:29:35.314 INFO:tasks.workunit.client.0.smithi144.stdout:5/361: write d4/d5/d18/d67/f65 [749341,8546] 0 2024-09-10T11:29:35.314 INFO:tasks.workunit.client.0.smithi144.stdout:4/316: mkdir d14/d6d 0 2024-09-10T11:29:35.314 INFO:tasks.workunit.client.1.smithi178.stdout:3/644: truncate db/d35/d3a/f4b 27713 0 2024-09-10T11:29:35.315 INFO:tasks.workunit.client.0.smithi144.stdout:5/362: chown d4/d20/l2f 5830 1 2024-09-10T11:29:35.315 INFO:tasks.workunit.client.0.smithi144.stdout:4/317: read - d14/d1e/f20 zero size 2024-09-10T11:29:35.317 INFO:tasks.workunit.client.0.smithi144.stdout:9/387: creat d11/d12/d1b/f8c x:0 0 0 2024-09-10T11:29:35.319 INFO:tasks.workunit.client.1.smithi178.stdout:6/680: mkdir d3/d74/d91/daa/dc2/dcf/df8 0 2024-09-10T11:29:35.319 INFO:tasks.workunit.client.0.smithi144.stdout:4/318: mknod d14/d1e/d22/d2b/d35/d4f/c6e 0 2024-09-10T11:29:35.319 INFO:tasks.workunit.client.0.smithi144.stdout:9/388: truncate d11/d36/d3e/f82 124699 0 2024-09-10T11:29:35.325 INFO:tasks.workunit.client.1.smithi178.stdout:3/645: mkdir db/d20/d3d/d56/dcf 0 2024-09-10T11:29:35.326 INFO:tasks.workunit.client.1.smithi178.stdout:3/646: readlink db/d20/d3d/d63/d6a/lb0 0 2024-09-10T11:29:35.326 INFO:tasks.workunit.client.1.smithi178.stdout:6/681: symlink d3/d74/d91/daa/dc2/dcf/lf9 0 2024-09-10T11:29:35.327 INFO:tasks.workunit.client.1.smithi178.stdout:9/750: dwrite d2/f18 [0,4194304] 0 2024-09-10T11:29:35.327 INFO:tasks.workunit.client.0.smithi144.stdout:4/319: dwrite d14/d67/d69/f6c [0,4194304] 0 2024-09-10T11:29:35.331 INFO:tasks.workunit.client.1.smithi178.stdout:8/797: rmdir d5/def 0 2024-09-10T11:29:35.333 INFO:tasks.workunit.client.0.smithi144.stdout:4/320: mkdir d14/d6f 0 2024-09-10T11:29:35.335 INFO:tasks.workunit.client.1.smithi178.stdout:6/682: dwrite d3/d7/d8/d15/d1c/d2a/fa2 [0,4194304] 0 2024-09-10T11:29:35.337 INFO:tasks.workunit.client.1.smithi178.stdout:9/751: rmdir d2/df/d31/d4a 39 2024-09-10T11:29:35.338 INFO:tasks.workunit.client.0.smithi144.stdout:5/363: write d4/d5/f23 [1177317,72202] 0 2024-09-10T11:29:35.339 INFO:tasks.workunit.client.1.smithi178.stdout:8/798: symlink d5/d30/db8/l10b 0 2024-09-10T11:29:35.340 INFO:tasks.workunit.client.0.smithi144.stdout:4/321: write d14/d1e/d22/f2d [945693,105959] 0 2024-09-10T11:29:35.340 INFO:tasks.workunit.client.0.smithi144.stdout:3/379: dread d4/dd/f1b [0,4194304] 0 2024-09-10T11:29:35.342 INFO:tasks.workunit.client.1.smithi178.stdout:3/647: mkdir db/d26/d74/dd0 0 2024-09-10T11:29:35.342 INFO:tasks.workunit.client.0.smithi144.stdout:5/364: readlink d4/d5/d18/d67/l60 0 2024-09-10T11:29:35.343 INFO:tasks.workunit.client.1.smithi178.stdout:6/683: mknod d3/d7/d8/d15/d1c/d2a/d18/d8e/d90/cfa 0 2024-09-10T11:29:35.346 INFO:tasks.workunit.client.0.smithi144.stdout:8/225: dwrite d0/d9/fe [0,4194304] 0 2024-09-10T11:29:35.347 INFO:tasks.workunit.client.1.smithi178.stdout:9/752: dread d2/df/f3b [0,4194304] 0 2024-09-10T11:29:35.347 INFO:tasks.workunit.client.0.smithi144.stdout:5/365: readlink d4/d20/l29 0 2024-09-10T11:29:35.349 INFO:tasks.workunit.client.0.smithi144.stdout:8/226: write d0/d9/fe [3044897,51804] 0 2024-09-10T11:29:35.350 INFO:tasks.workunit.client.1.smithi178.stdout:9/753: write d2/d8/f1a [4545444,71325] 0 2024-09-10T11:29:35.351 INFO:tasks.workunit.client.0.smithi144.stdout:9/389: dwrite d11/d12/f26 [0,4194304] 0 2024-09-10T11:29:35.354 INFO:tasks.workunit.client.0.smithi144.stdout:5/366: mkdir d4/d20/d22/d5c/d5d/d78 0 2024-09-10T11:29:35.354 INFO:tasks.workunit.client.0.smithi144.stdout:4/322: getdents d14/d67/d69 0 2024-09-10T11:29:35.356 INFO:tasks.workunit.client.1.smithi178.stdout:1/637: dwrite d3/d9/d21/d4b/d2b/fb9 [0,4194304] 0 2024-09-10T11:29:35.356 INFO:tasks.workunit.client.0.smithi144.stdout:5/367: fdatasync d4/d5/d18/d38/d6d/f70 0 2024-09-10T11:29:35.359 INFO:tasks.workunit.client.0.smithi144.stdout:9/390: rename d11/d12/d1b/d22/d53/f54 to d11/d36/d2d/f8d 0 2024-09-10T11:29:35.361 INFO:tasks.workunit.client.0.smithi144.stdout:9/391: stat d11/d12/c4c 0 2024-09-10T11:29:35.365 INFO:tasks.workunit.client.0.smithi144.stdout:4/323: dwrite d14/d1e/f20 [0,4194304] 0 2024-09-10T11:29:35.365 INFO:tasks.workunit.client.0.smithi144.stdout:3/380: dread f3 [0,4194304] 0 2024-09-10T11:29:35.367 INFO:tasks.workunit.client.0.smithi144.stdout:4/324: write f5 [2034408,82902] 0 2024-09-10T11:29:35.371 INFO:tasks.workunit.client.1.smithi178.stdout:9/754: dwrite d2/d19/d1c/f2c [4194304,4194304] 0 2024-09-10T11:29:35.372 INFO:tasks.workunit.client.0.smithi144.stdout:4/325: chown d14/d1e/d22/d2b/d35/c68 495558 1 2024-09-10T11:29:35.374 INFO:tasks.workunit.client.1.smithi178.stdout:6/684: truncate d3/d7/d35/f6f 3679804 0 2024-09-10T11:29:35.374 INFO:tasks.workunit.client.1.smithi178.stdout:3/648: write db/d26/f32 [132567,95497] 0 2024-09-10T11:29:35.375 INFO:tasks.workunit.client.1.smithi178.stdout:3/649: chown db/d20 456 1 2024-09-10T11:29:35.377 INFO:tasks.workunit.client.1.smithi178.stdout:3/650: write db/d26/d40/f69 [891679,126170] 0 2024-09-10T11:29:35.378 INFO:tasks.workunit.client.0.smithi144.stdout:4/326: readlink d14/d1e/d22/l42 0 2024-09-10T11:29:35.380 INFO:tasks.workunit.client.0.smithi144.stdout:4/327: fdatasync f9 0 2024-09-10T11:29:35.380 INFO:tasks.workunit.client.0.smithi144.stdout:3/381: dread d4/dd/d21/f5b [0,4194304] 0 2024-09-10T11:29:35.381 INFO:tasks.workunit.client.1.smithi178.stdout:8/799: dread d5/db/d10/d53/f98 [0,4194304] 0 2024-09-10T11:29:35.382 INFO:tasks.workunit.client.1.smithi178.stdout:3/651: chown db/f1d 47 1 2024-09-10T11:29:35.382 INFO:tasks.workunit.client.0.smithi144.stdout:5/368: dread d4/d20/f43 [0,4194304] 0 2024-09-10T11:29:35.383 INFO:tasks.workunit.client.0.smithi144.stdout:3/382: chown d4/dd/f6f 186617123 1 2024-09-10T11:29:35.387 INFO:tasks.workunit.client.0.smithi144.stdout:3/383: fdatasync d4/dd/f1b 0 2024-09-10T11:29:35.389 INFO:tasks.workunit.client.0.smithi144.stdout:5/369: symlink d4/d5/d32/d47/l79 0 2024-09-10T11:29:35.390 INFO:tasks.workunit.client.1.smithi178.stdout:6/685: creat d3/d7/d8/d15/d1c/d2a/d18/d8e/d90/ffb x:0 0 0 2024-09-10T11:29:35.391 INFO:tasks.workunit.client.1.smithi178.stdout:9/755: dwrite d2/d13/d75/d60/d66/fd1 [0,4194304] 0 2024-09-10T11:29:35.391 INFO:tasks.workunit.client.0.smithi144.stdout:9/392: dwrite d11/f17 [0,4194304] 0 2024-09-10T11:29:35.392 INFO:tasks.workunit.client.1.smithi178.stdout:3/652: getdents db/d9b 0 2024-09-10T11:29:35.394 INFO:tasks.workunit.client.0.smithi144.stdout:8/227: dread d0/d9/f1d [0,4194304] 0 2024-09-10T11:29:35.394 INFO:tasks.workunit.client.0.smithi144.stdout:4/328: dwrite d14/d1e/d22/d34/f3b [0,4194304] 0 2024-09-10T11:29:35.394 INFO:tasks.workunit.client.1.smithi178.stdout:1/638: dwrite d3/d9/d98/dac/d3a/f53 [4194304,4194304] 0 2024-09-10T11:29:35.394 INFO:tasks.workunit.client.1.smithi178.stdout:6/686: chown d3/d74/d91/daa/dc2/dcf/lf9 9672017 1 2024-09-10T11:29:35.394 INFO:tasks.workunit.client.1.smithi178.stdout:3/653: dread - db/d20/fa5 zero size 2024-09-10T11:29:35.395 INFO:tasks.workunit.client.1.smithi178.stdout:8/800: dread d5/db/d10/d20/fb9 [0,4194304] 0 2024-09-10T11:29:35.395 INFO:tasks.workunit.client.0.smithi144.stdout:9/393: chown d11/d12/d1b/d2c 10770 1 2024-09-10T11:29:35.395 INFO:tasks.workunit.client.0.smithi144.stdout:5/370: write d4/d5/d18/d67/d59/f5e [1790294,38590] 0 2024-09-10T11:29:35.396 INFO:tasks.workunit.client.0.smithi144.stdout:4/329: chown d14/c25 0 1 2024-09-10T11:29:35.396 INFO:tasks.workunit.client.0.smithi144.stdout:8/228: chown d0/d9/d1f/d34 0 1 2024-09-10T11:29:35.400 INFO:tasks.workunit.client.0.smithi144.stdout:8/229: unlink d0/d9/d21/f2b 0 2024-09-10T11:29:35.405 INFO:tasks.workunit.client.1.smithi178.stdout:8/801: symlink d5/db/d10/d70/l10c 0 2024-09-10T11:29:35.406 INFO:tasks.workunit.client.1.smithi178.stdout:6/687: creat d3/d6e/de9/ffc x:0 0 0 2024-09-10T11:29:35.407 INFO:tasks.workunit.client.0.smithi144.stdout:4/330: symlink d14/d6f/l70 0 2024-09-10T11:29:35.414 INFO:tasks.workunit.client.0.smithi144.stdout:8/230: link d0/d9/d23/l44 d0/d1b/l4b 0 2024-09-10T11:29:35.415 INFO:tasks.workunit.client.1.smithi178.stdout:8/802: dwrite d5/d61/fa2 [0,4194304] 0 2024-09-10T11:29:35.417 INFO:tasks.workunit.client.1.smithi178.stdout:8/803: chown d5/db/d3d/f5f 12300680 1 2024-09-10T11:29:35.417 INFO:tasks.workunit.client.0.smithi144.stdout:9/394: getdents d11/d12/d1b/d22/d53 0 2024-09-10T11:29:35.417 INFO:tasks.workunit.client.0.smithi144.stdout:5/371: dwrite d4/d5/d18/d67/d59/f6c [0,4194304] 0 2024-09-10T11:29:35.418 INFO:tasks.workunit.client.0.smithi144.stdout:5/372: readlink d4/d5/d32/d47/l79 0 2024-09-10T11:29:35.418 INFO:tasks.workunit.client.1.smithi178.stdout:8/804: write d5/db/d3d/fbe [3611446,90284] 0 2024-09-10T11:29:35.419 INFO:tasks.workunit.client.0.smithi144.stdout:8/231: write d0/d9/ff [3096774,65972] 0 2024-09-10T11:29:35.420 INFO:tasks.workunit.client.1.smithi178.stdout:1/639: link d3/d9/d98/dac/d3a/f53 d3/d9/d98/dac/d3a/fed 0 2024-09-10T11:29:35.420 INFO:tasks.workunit.client.0.smithi144.stdout:9/395: write d11/d12/f26 [1967029,21223] 0 2024-09-10T11:29:35.422 INFO:tasks.workunit.client.1.smithi178.stdout:6/688: creat d3/d7/d8/d15/d1c/d2a/d18/ffd x:0 0 0 2024-09-10T11:29:35.423 INFO:tasks.workunit.client.0.smithi144.stdout:5/373: symlink d4/d5/d18/d67/l7a 0 2024-09-10T11:29:35.424 INFO:tasks.workunit.client.0.smithi144.stdout:9/396: read - d11/d36/d2d/d38/f76 zero size 2024-09-10T11:29:35.425 INFO:tasks.workunit.client.1.smithi178.stdout:1/640: unlink d3/d9/d98/dac/d3a/dae/fb3 0 2024-09-10T11:29:35.426 INFO:tasks.workunit.client.0.smithi144.stdout:4/331: rename d14/d1e/d22/l42 to d14/d1e/d22/d34/l71 0 2024-09-10T11:29:35.426 INFO:tasks.workunit.client.0.smithi144.stdout:5/374: fdatasync d4/d5/d32/f68 0 2024-09-10T11:29:35.426 INFO:tasks.workunit.client.0.smithi144.stdout:8/232: dread d0/d1b/f2f [0,4194304] 0 2024-09-10T11:29:35.428 INFO:tasks.workunit.client.0.smithi144.stdout:4/332: chown d14/d1e/d22/d2b/d35/f38 119526629 1 2024-09-10T11:29:35.428 INFO:tasks.workunit.client.0.smithi144.stdout:5/375: write d4/d5/d32/f68 [760048,110188] 0 2024-09-10T11:29:35.429 INFO:tasks.workunit.client.1.smithi178.stdout:6/689: mknod d3/d74/d91/daa/dc2/d57/dea/cfe 0 2024-09-10T11:29:35.431 INFO:tasks.workunit.client.0.smithi144.stdout:9/397: mkdir d11/d12/d1b/d41/d8e 0 2024-09-10T11:29:35.433 INFO:tasks.workunit.client.0.smithi144.stdout:4/333: chown d14/d1e/d22/d2b/d35/d3c/l59 5 1 2024-09-10T11:29:35.433 INFO:tasks.workunit.client.0.smithi144.stdout:8/233: dread d0/f10 [0,4194304] 0 2024-09-10T11:29:35.433 INFO:tasks.workunit.client.0.smithi144.stdout:5/376: symlink d4/d5/d56/l7b 0 2024-09-10T11:29:35.434 INFO:tasks.workunit.client.1.smithi178.stdout:6/690: write d3/d7/d8/d15/d1c/d2a/fa2 [1882374,83795] 0 2024-09-10T11:29:35.434 INFO:tasks.workunit.client.1.smithi178.stdout:1/641: dwrite d3/d35/f99 [0,4194304] 0 2024-09-10T11:29:35.435 INFO:tasks.workunit.client.0.smithi144.stdout:5/377: stat d4/d5/d18/d67/d44/l66 0 2024-09-10T11:29:35.437 INFO:tasks.workunit.client.0.smithi144.stdout:5/378: write d4/d5/d18/d67/d59/f6c [1979247,70469] 0 2024-09-10T11:29:35.437 INFO:tasks.workunit.client.0.smithi144.stdout:9/398: rename d11/d36/d3e to d11/d12/d1b/d41/d8f 0 2024-09-10T11:29:35.438 INFO:tasks.workunit.client.0.smithi144.stdout:5/379: fdatasync d4/d20/d24/d2d/f2e 0 2024-09-10T11:29:35.439 INFO:tasks.workunit.client.0.smithi144.stdout:8/234: creat d0/d1b/f4c x:0 0 0 2024-09-10T11:29:35.439 INFO:tasks.workunit.client.1.smithi178.stdout:1/642: unlink d3/d9/d98/dac/d2a/f34 0 2024-09-10T11:29:35.440 INFO:tasks.workunit.client.1.smithi178.stdout:9/756: truncate d2/df/f1b 6082377 0 2024-09-10T11:29:35.440 INFO:tasks.workunit.client.0.smithi144.stdout:4/334: creat d14/d1e/d22/d2b/d35/f72 x:0 0 0 2024-09-10T11:29:35.443 INFO:tasks.workunit.client.0.smithi144.stdout:4/335: fsync d14/d1e/d22/d2b/d35/d4f/f60 0 2024-09-10T11:29:35.444 INFO:tasks.workunit.client.0.smithi144.stdout:9/399: symlink d11/d12/l90 0 2024-09-10T11:29:35.445 INFO:tasks.workunit.client.1.smithi178.stdout:9/757: rename d2/d13/d40/d89/fc1 to d2/d13/d40/d89/fe5 0 2024-09-10T11:29:35.447 INFO:tasks.workunit.client.0.smithi144.stdout:9/400: fsync d11/d36/d2d/d38/f76 0 2024-09-10T11:29:35.448 INFO:tasks.workunit.client.0.smithi144.stdout:8/235: link d0/d9/d23/d26/c45 d0/d9/d23/c4d 0 2024-09-10T11:29:35.448 INFO:tasks.workunit.client.1.smithi178.stdout:1/643: rename d3/d9/d98/dac/f51 to d3/d9/d98/dac/d3a/fee 0 2024-09-10T11:29:35.448 INFO:tasks.workunit.client.0.smithi144.stdout:3/384: dread d4/f37 [0,4194304] 0 2024-09-10T11:29:35.449 INFO:tasks.workunit.client.1.smithi178.stdout:6/691: dread d3/d7/d8/d15/d1c/d2a/f5d [0,4194304] 0 2024-09-10T11:29:35.449 INFO:tasks.workunit.client.1.smithi178.stdout:9/758: mknod d2/d13/d40/d89/d93/d88/ce6 0 2024-09-10T11:29:35.450 INFO:tasks.workunit.client.0.smithi144.stdout:4/336: symlink d14/d1e/d22/d2b/d35/d4f/l73 0 2024-09-10T11:29:35.450 INFO:tasks.workunit.client.0.smithi144.stdout:9/401: fsync d11/d36/f52 0 2024-09-10T11:29:35.451 INFO:tasks.workunit.client.0.smithi144.stdout:5/380: dwrite d4/d20/f34 [0,4194304] 0 2024-09-10T11:29:35.451 INFO:tasks.workunit.client.1.smithi178.stdout:6/692: symlink d3/d7/d8/d15/lff 0 2024-09-10T11:29:35.452 INFO:tasks.workunit.client.0.smithi144.stdout:8/236: read d0/f13 [261108,75633] 0 2024-09-10T11:29:35.454 INFO:tasks.workunit.client.1.smithi178.stdout:9/759: rename d2/d19/d1c/d2d/l96 to d2/d13/d40/d89/d93/dc2/le7 0 2024-09-10T11:29:35.454 INFO:tasks.workunit.client.0.smithi144.stdout:8/237: write d0/d1b/f2a [3636306,37341] 0 2024-09-10T11:29:35.454 INFO:tasks.workunit.client.0.smithi144.stdout:8/238: chown d0 0 1 2024-09-10T11:29:35.455 INFO:tasks.workunit.client.0.smithi144.stdout:5/381: write d4/d5/d18/d38/d6d/f70 [414171,89955] 0 2024-09-10T11:29:35.457 INFO:tasks.workunit.client.0.smithi144.stdout:2/261: sync 2024-09-10T11:29:35.457 INFO:tasks.workunit.client.0.smithi144.stdout:6/276: sync 2024-09-10T11:29:35.460 INFO:tasks.workunit.client.0.smithi144.stdout:3/385: link d4/d28/d67/d80/d1d/f29 d4/dd/d21/f86 0 2024-09-10T11:29:35.460 INFO:tasks.workunit.client.1.smithi178.stdout:9/760: mknod d2/df/d31/ce8 0 2024-09-10T11:29:35.461 INFO:tasks.workunit.client.1.smithi178.stdout:6/693: dwrite d3/d7/d8/d15/d1c/d2a/d18/d8e/d90/fe5 [0,4194304] 0 2024-09-10T11:29:35.462 INFO:tasks.workunit.client.0.smithi144.stdout:3/386: chown d4/dd/l15 15462060 1 2024-09-10T11:29:35.463 INFO:tasks.workunit.client.0.smithi144.stdout:8/239: creat d0/d9/d23/d26/f4e x:0 0 0 2024-09-10T11:29:35.465 INFO:tasks.workunit.client.0.smithi144.stdout:5/382: mknod d4/d52/c7c 0 2024-09-10T11:29:35.466 INFO:tasks.workunit.client.1.smithi178.stdout:9/761: symlink d2/d19/d1c/le9 0 2024-09-10T11:29:35.467 INFO:tasks.workunit.client.0.smithi144.stdout:4/337: link d14/d6f/l70 d14/d6d/l74 0 2024-09-10T11:29:35.468 INFO:tasks.workunit.client.1.smithi178.stdout:3/654: dwrite db/d35/f77 [0,4194304] 0 2024-09-10T11:29:35.468 INFO:tasks.workunit.client.0.smithi144.stdout:3/387: fsync d4/d28/d67/d80/d17/d18/f4c 0 2024-09-10T11:29:35.471 INFO:tasks.workunit.client.0.smithi144.stdout:8/240: mkdir d0/d9/d21/d25/d4f 0 2024-09-10T11:29:35.471 INFO:tasks.workunit.client.0.smithi144.stdout:6/277: dwrite d9/dd/d11/f4c [0,4194304] 0 2024-09-10T11:29:35.471 INFO:tasks.workunit.client.0.smithi144.stdout:2/262: dwrite d7/da/f3c [0,4194304] 0 2024-09-10T11:29:35.471 INFO:tasks.workunit.client.1.smithi178.stdout:4/575: sync 2024-09-10T11:29:35.472 INFO:tasks.workunit.client.0.smithi144.stdout:5/383: write d4/d20/d24/f63 [3951388,85150] 0 2024-09-10T11:29:35.474 INFO:tasks.workunit.client.1.smithi178.stdout:8/805: truncate d5/d61/f8f 87239 0 2024-09-10T11:29:35.475 INFO:tasks.workunit.client.1.smithi178.stdout:9/762: symlink d2/d19/d1c/d2a/lea 0 2024-09-10T11:29:35.475 INFO:tasks.workunit.client.1.smithi178.stdout:6/694: getdents d3/d6e/d87 0 2024-09-10T11:29:35.477 INFO:tasks.workunit.client.1.smithi178.stdout:8/806: stat d5/db/d1e/f39 0 2024-09-10T11:29:35.477 INFO:tasks.workunit.client.0.smithi144.stdout:5/384: truncate d4/d5/d18/d67/f62 861689 0 2024-09-10T11:29:35.485 INFO:tasks.workunit.client.1.smithi178.stdout:9/763: read d2/d19/d1c/d2d/dab/fb1 [2783263,14349] 0 2024-09-10T11:29:35.486 INFO:tasks.workunit.client.0.smithi144.stdout:8/241: creat d0/d9/d23/f50 x:0 0 0 2024-09-10T11:29:35.487 INFO:tasks.workunit.client.0.smithi144.stdout:6/278: mknod d9/d1f/d2c/c65 0 2024-09-10T11:29:35.492 INFO:tasks.workunit.client.0.smithi144.stdout:3/388: link d4/dd/d21/f83 d4/d28/f87 0 2024-09-10T11:29:35.493 INFO:tasks.workunit.client.1.smithi178.stdout:4/576: rename d0/d8/df/l1f to d0/d8/da/d1a/d51/lc8 0 2024-09-10T11:29:35.496 INFO:tasks.workunit.client.0.smithi144.stdout:8/242: rename d0/d9/d23/d26/l32 to d0/d9/d1f/d34/l51 0 2024-09-10T11:29:35.497 INFO:tasks.workunit.client.0.smithi144.stdout:6/279: creat d9/dd/d11/d1b/d2f/f66 x:0 0 0 2024-09-10T11:29:35.499 INFO:tasks.workunit.client.0.smithi144.stdout:4/338: creat d14/d1e/d22/d34/f75 x:0 0 0 2024-09-10T11:29:35.500 INFO:tasks.workunit.client.0.smithi144.stdout:2/263: mknod d7/de/d36/d45/d47/c55 0 2024-09-10T11:29:35.501 INFO:tasks.workunit.client.1.smithi178.stdout:1/644: link d3/d9/d98/dac/d3a/fee d3/d9/d98/dac/d3a/d50/d80/fef 0 2024-09-10T11:29:35.502 INFO:tasks.workunit.client.0.smithi144.stdout:3/389: readlink d4/l10 0 2024-09-10T11:29:35.503 INFO:tasks.workunit.client.1.smithi178.stdout:1/645: creat d3/d9/d98/dac/ff0 x:0 0 0 2024-09-10T11:29:35.504 INFO:tasks.workunit.client.0.smithi144.stdout:8/243: write d0/f13 [4691858,65243] 0 2024-09-10T11:29:35.505 INFO:tasks.workunit.client.0.smithi144.stdout:3/390: chown d4/d28/d67/d80/d17/f30 201 1 2024-09-10T11:29:35.509 INFO:tasks.workunit.client.0.smithi144.stdout:2/264: dwrite d7/da/f44 [0,4194304] 0 2024-09-10T11:29:35.510 INFO:tasks.workunit.client.0.smithi144.stdout:6/280: creat d9/d1f/d2c/f67 x:0 0 0 2024-09-10T11:29:35.511 INFO:tasks.workunit.client.0.smithi144.stdout:3/391: readlink d4/d28/d67/d80/d17/d18/d34/d78/l7e 0 2024-09-10T11:29:35.512 INFO:tasks.workunit.client.0.smithi144.stdout:9/402: truncate d11/d12/d1b/d41/d8f/f82 107463 0 2024-09-10T11:29:35.514 INFO:tasks.workunit.client.1.smithi178.stdout:3/655: dread db/d20/d3d/d63/d6a/f6c [0,4194304] 0 2024-09-10T11:29:35.517 INFO:tasks.workunit.client.0.smithi144.stdout:5/385: dread d4/d5/d32/f5f [0,4194304] 0 2024-09-10T11:29:35.517 INFO:tasks.workunit.client.0.smithi144.stdout:9/403: dread d11/d12/d1b/d22/f2f [0,4194304] 0 2024-09-10T11:29:35.518 INFO:tasks.workunit.client.0.smithi144.stdout:5/386: write d4/d5/d32/d47/f71 [179455,119719] 0 2024-09-10T11:29:35.519 INFO:tasks.workunit.client.1.smithi178.stdout:6/695: dwrite d3/d7/d8/d15/fbc [0,4194304] 0 2024-09-10T11:29:35.519 INFO:tasks.workunit.client.1.smithi178.stdout:4/577: write d0/f1c [4716315,24489] 0 2024-09-10T11:29:35.520 INFO:tasks.workunit.client.1.smithi178.stdout:6/696: write d3/d7/d8/d15/d1c/f53 [311850,122320] 0 2024-09-10T11:29:35.521 INFO:tasks.workunit.client.0.smithi144.stdout:6/281: unlink d9/dd/d3e/d47/d13/l50 0 2024-09-10T11:29:35.522 INFO:tasks.workunit.client.0.smithi144.stdout:6/282: dread - d9/dd/f33 zero size 2024-09-10T11:29:35.524 INFO:tasks.workunit.client.0.smithi144.stdout:8/244: unlink d0/d9/f3b 0 2024-09-10T11:29:35.524 INFO:tasks.workunit.client.0.smithi144.stdout:6/283: chown d9/d1f/d2c/f67 87 1 2024-09-10T11:29:35.527 INFO:tasks.workunit.client.0.smithi144.stdout:2/265: symlink d7/de/d3d/d43/l56 0 2024-09-10T11:29:35.528 INFO:tasks.workunit.client.1.smithi178.stdout:6/697: symlink d3/d74/d91/l100 0 2024-09-10T11:29:35.529 INFO:tasks.workunit.client.1.smithi178.stdout:3/656: getdents db/d35/d58 0 2024-09-10T11:29:35.529 INFO:tasks.workunit.client.1.smithi178.stdout:9/764: dwrite d2/d8/f73 [0,4194304] 0 2024-09-10T11:29:35.533 INFO:tasks.workunit.client.1.smithi178.stdout:3/657: write db/d20/d3d/d63/f96 [865764,29704] 0 2024-09-10T11:29:35.537 INFO:tasks.workunit.client.1.smithi178.stdout:9/765: mkdir d2/d13/d40/d56/dbe/deb 0 2024-09-10T11:29:35.537 INFO:tasks.workunit.client.0.smithi144.stdout:4/339: link d14/f15 d14/d1e/d22/d34/d3e/d40/f76 0 2024-09-10T11:29:35.538 INFO:tasks.workunit.client.1.smithi178.stdout:3/658: mkdir db/d20/d3e/d6b/dd1 0 2024-09-10T11:29:35.539 INFO:tasks.workunit.client.0.smithi144.stdout:3/392: link d4/d28/d67/d80/d17/d18/f4c d4/d28/d67/d80/d17/d18/d34/d65/f88 0 2024-09-10T11:29:35.540 INFO:tasks.workunit.client.1.smithi178.stdout:4/578: dread d0/d9/d3d/f61 [0,4194304] 0 2024-09-10T11:29:35.540 INFO:tasks.workunit.client.1.smithi178.stdout:6/698: dread d3/d7/d8/d15/d1c/d2a/fa2 [0,4194304] 0 2024-09-10T11:29:35.543 INFO:tasks.workunit.client.0.smithi144.stdout:6/284: mkdir d9/dd/d11/d1b/d62/d68 0 2024-09-10T11:29:35.543 INFO:tasks.workunit.client.1.smithi178.stdout:3/659: creat db/d20/d3e/d6b/dd1/fd2 x:0 0 0 2024-09-10T11:29:35.544 INFO:tasks.workunit.client.0.smithi144.stdout:2/266: fdatasync d7/de/d3d/f52 0 2024-09-10T11:29:35.545 INFO:tasks.workunit.client.0.smithi144.stdout:4/340: readlink l13 0 2024-09-10T11:29:35.545 INFO:tasks.workunit.client.0.smithi144.stdout:8/245: rmdir d0/d9/d1f/d34 39 2024-09-10T11:29:35.546 INFO:tasks.workunit.client.1.smithi178.stdout:4/579: mkdir d0/d8/da/d1a/d51/d5d/dc9 0 2024-09-10T11:29:35.547 INFO:tasks.workunit.client.1.smithi178.stdout:6/699: fsync d3/d7/f9d 0 2024-09-10T11:29:35.548 INFO:tasks.workunit.client.0.smithi144.stdout:3/393: unlink d4/dd/f51 0 2024-09-10T11:29:35.548 INFO:tasks.workunit.client.1.smithi178.stdout:3/660: rename db/d9b/fba to db/d26/d74/fd3 0 2024-09-10T11:29:35.549 INFO:tasks.workunit.client.0.smithi144.stdout:5/387: link d4/d5/f25 d4/d5/f7d 0 2024-09-10T11:29:35.551 INFO:tasks.workunit.client.1.smithi178.stdout:6/700: symlink d3/d7/d8/d19/l101 0 2024-09-10T11:29:35.552 INFO:tasks.workunit.client.1.smithi178.stdout:3/661: fsync db/d35/d3a/f4a 0 2024-09-10T11:29:35.555 INFO:tasks.workunit.client.0.smithi144.stdout:8/246: write d0/d9/d23/d37/f3a [353569,23108] 0 2024-09-10T11:29:35.558 INFO:tasks.workunit.client.0.smithi144.stdout:3/394: creat d4/d1a/d7a/f89 x:0 0 0 2024-09-10T11:29:35.559 INFO:tasks.workunit.client.1.smithi178.stdout:6/701: creat d3/d7/d8/d15/d1c/f102 x:0 0 0 2024-09-10T11:29:35.559 INFO:tasks.workunit.client.0.smithi144.stdout:8/247: write d0/d1b/f2a [673215,78426] 0 2024-09-10T11:29:35.560 INFO:tasks.workunit.client.1.smithi178.stdout:1/646: write d3/d9/d21/d4b/f2d [179475,14890] 0 2024-09-10T11:29:35.560 INFO:tasks.workunit.client.0.smithi144.stdout:5/388: rename d4/d20/l29 to d4/d52/l7e 0 2024-09-10T11:29:35.561 INFO:tasks.workunit.client.1.smithi178.stdout:3/662: dwrite db/d20/f33 [0,4194304] 0 2024-09-10T11:29:35.565 INFO:tasks.workunit.client.0.smithi144.stdout:6/285: dread d9/dd/f1a [4194304,4194304] 0 2024-09-10T11:29:35.565 INFO:tasks.workunit.client.1.smithi178.stdout:6/702: truncate d3/d74/d91/daa/dc2/f66 3101827 0 2024-09-10T11:29:35.566 INFO:tasks.workunit.client.0.smithi144.stdout:8/248: chown d0/d9/d21/c3e 27 1 2024-09-10T11:29:35.566 INFO:tasks.workunit.client.0.smithi144.stdout:2/267: dwrite d7/de/d36/f51 [4194304,4194304] 0 2024-09-10T11:29:35.569 INFO:tasks.workunit.client.1.smithi178.stdout:9/766: dwrite d2/d19/d1c/d2d/dab/fb1 [0,4194304] 0 2024-09-10T11:29:35.569 INFO:tasks.workunit.client.0.smithi144.stdout:8/249: write d0/fc [4797422,92751] 0 2024-09-10T11:29:35.569 INFO:tasks.workunit.client.0.smithi144.stdout:6/286: write d9/dd/d3e/d47/f2e [741874,25564] 0 2024-09-10T11:29:35.569 INFO:tasks.workunit.client.0.smithi144.stdout:3/395: symlink d4/d28/d67/d80/d17/d18/d34/d4b/d63/l8a 0 2024-09-10T11:29:35.570 INFO:tasks.workunit.client.1.smithi178.stdout:3/663: creat db/d20/d3e/d6b/fd4 x:0 0 0 2024-09-10T11:29:35.575 INFO:tasks.workunit.client.0.smithi144.stdout:3/396: read f2 [2421688,83596] 0 2024-09-10T11:29:35.577 INFO:tasks.workunit.client.0.smithi144.stdout:2/268: creat d7/da/d1e/d21/d54/f57 x:0 0 0 2024-09-10T11:29:35.578 INFO:tasks.workunit.client.1.smithi178.stdout:1/647: dread d3/d9/d98/dac/d3a/d9f/fa2 [0,4194304] 0 2024-09-10T11:29:35.579 INFO:tasks.workunit.client.1.smithi178.stdout:6/703: mkdir d3/d74/d91/daa/d103 0 2024-09-10T11:29:35.579 INFO:tasks.workunit.client.0.smithi144.stdout:8/250: mknod d0/d9/c52 0 2024-09-10T11:29:35.582 INFO:tasks.workunit.client.0.smithi144.stdout:5/389: rmdir d4/d20/d22/d5c/d5d/d78 0 2024-09-10T11:29:35.582 INFO:tasks.workunit.client.0.smithi144.stdout:6/287: mknod d9/dd/c69 0 2024-09-10T11:29:35.583 INFO:tasks.workunit.client.1.smithi178.stdout:1/648: symlink d3/d9/d98/lf1 0 2024-09-10T11:29:35.585 INFO:tasks.workunit.client.0.smithi144.stdout:5/390: chown d4/d5/f23 214591193 1 2024-09-10T11:29:35.585 INFO:tasks.workunit.client.1.smithi178.stdout:4/580: truncate d0/d9/dc/d8e/dbf/f93 3132206 0 2024-09-10T11:29:35.585 INFO:tasks.workunit.client.0.smithi144.stdout:2/269: rename d7/de/d36/c3a to d7/de/d36/d45/c58 0 2024-09-10T11:29:35.586 INFO:tasks.workunit.client.1.smithi178.stdout:9/767: mknod d2/df/cec 0 2024-09-10T11:29:35.587 INFO:tasks.workunit.client.0.smithi144.stdout:8/251: chown d0/d1b/l4b 4443 1 2024-09-10T11:29:35.587 INFO:tasks.workunit.client.1.smithi178.stdout:6/704: creat d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/f104 x:0 0 0 2024-09-10T11:29:35.587 INFO:tasks.workunit.client.0.smithi144.stdout:5/391: truncate d4/d5/d32/d47/f6a 435078 0 2024-09-10T11:29:35.588 INFO:tasks.workunit.client.0.smithi144.stdout:6/288: symlink d9/d1f/d2c/d4f/l6a 0 2024-09-10T11:29:35.589 INFO:tasks.workunit.client.0.smithi144.stdout:5/392: fdatasync d4/d5/d18/d67/d59/f6c 0 2024-09-10T11:29:35.589 INFO:tasks.workunit.client.0.smithi144.stdout:6/289: chown d9/d1f/f26 6 1 2024-09-10T11:29:35.590 INFO:tasks.workunit.client.1.smithi178.stdout:1/649: unlink d3/d9/f59 0 2024-09-10T11:29:35.590 INFO:tasks.workunit.client.0.smithi144.stdout:2/270: unlink d7/de/f16 0 2024-09-10T11:29:35.591 INFO:tasks.workunit.client.0.smithi144.stdout:6/290: write d9/dd/d3e/d47/d13/f1e [352495,118388] 0 2024-09-10T11:29:35.592 INFO:tasks.workunit.client.1.smithi178.stdout:6/705: creat d3/d74/d91/daa/dc2/dcf/f105 x:0 0 0 2024-09-10T11:29:35.593 INFO:tasks.workunit.client.0.smithi144.stdout:5/393: truncate d4/d5/d18/d38/d6d/f70 1671000 0 2024-09-10T11:29:35.593 INFO:tasks.workunit.client.1.smithi178.stdout:3/664: link db/d20/d3d/d63/da2/d98/lca db/d35/ld5 0 2024-09-10T11:29:35.594 INFO:tasks.workunit.client.1.smithi178.stdout:6/706: write d3/d6e/ff1 [597310,12090] 0 2024-09-10T11:29:35.594 INFO:tasks.workunit.client.0.smithi144.stdout:5/394: write d4/d20/d24/d2d/d4b/f4e [9016026,115501] 0 2024-09-10T11:29:35.595 INFO:tasks.workunit.client.1.smithi178.stdout:4/581: mkdir d0/d9/dc/dca 0 2024-09-10T11:29:35.595 INFO:tasks.workunit.client.0.smithi144.stdout:4/341: truncate d14/d1e/f50 3021433 0 2024-09-10T11:29:35.597 INFO:tasks.workunit.client.0.smithi144.stdout:5/395: write d4/d20/d24/d2d/d4b/f4e [9292608,28708] 0 2024-09-10T11:29:35.597 INFO:tasks.workunit.client.0.smithi144.stdout:2/271: fdatasync d7/de/f28 0 2024-09-10T11:29:35.597 INFO:tasks.workunit.client.1.smithi178.stdout:3/665: fsync db/f1a 0 2024-09-10T11:29:35.598 INFO:tasks.workunit.client.1.smithi178.stdout:3/666: fsync db/d20/d3e/d6b/fa7 0 2024-09-10T11:29:35.600 INFO:tasks.workunit.client.0.smithi144.stdout:8/252: creat d0/d9/d1f/d34/f53 x:0 0 0 2024-09-10T11:29:35.600 INFO:tasks.workunit.client.0.smithi144.stdout:2/272: write d7/de/f3b [620952,42413] 0 2024-09-10T11:29:35.601 INFO:tasks.workunit.client.0.smithi144.stdout:6/291: symlink d9/d1f/d2c/l6b 0 2024-09-10T11:29:35.602 INFO:tasks.workunit.client.1.smithi178.stdout:3/667: rename db/c92 to db/d26/d74/cd6 0 2024-09-10T11:29:35.604 INFO:tasks.workunit.client.1.smithi178.stdout:4/582: rmdir d0/d9d/db3 0 2024-09-10T11:29:35.605 INFO:tasks.workunit.client.0.smithi144.stdout:4/342: creat d14/d1e/d22/d2b/d35/f77 x:0 0 0 2024-09-10T11:29:35.605 INFO:tasks.workunit.client.1.smithi178.stdout:3/668: read - db/d26/f80 zero size 2024-09-10T11:29:35.606 INFO:tasks.workunit.client.0.smithi144.stdout:8/253: symlink d0/d9/d21/d25/l54 0 2024-09-10T11:29:35.606 INFO:tasks.workunit.client.1.smithi178.stdout:3/669: truncate db/d20/f95 193020 0 2024-09-10T11:29:35.609 INFO:tasks.workunit.client.0.smithi144.stdout:8/254: write d0/d9/d23/d26/f36 [4103200,37150] 0 2024-09-10T11:29:35.609 INFO:tasks.workunit.client.0.smithi144.stdout:2/273: creat d7/d29/f59 x:0 0 0 2024-09-10T11:29:35.609 INFO:tasks.workunit.client.1.smithi178.stdout:4/583: creat d0/d9/d3d/fcb x:0 0 0 2024-09-10T11:29:35.610 INFO:tasks.workunit.client.0.smithi144.stdout:6/292: mkdir d9/d38/d6c 0 2024-09-10T11:29:35.611 INFO:tasks.workunit.client.1.smithi178.stdout:4/584: chown d0/d8/da/f2b 1798743 1 2024-09-10T11:29:35.614 INFO:tasks.workunit.client.0.smithi144.stdout:6/293: write d9/d1f/d2c/d4f/f58 [760145,73806] 0 2024-09-10T11:29:35.615 INFO:tasks.workunit.client.0.smithi144.stdout:2/274: rename d7/c9 to d7/de/d3d/d43/c5a 0 2024-09-10T11:29:35.618 INFO:tasks.workunit.client.0.smithi144.stdout:3/397: truncate f2 1156357 0 2024-09-10T11:29:35.618 INFO:tasks.workunit.client.0.smithi144.stdout:4/343: dwrite d14/d1e/d22/f37 [4194304,4194304] 0 2024-09-10T11:29:35.618 INFO:tasks.workunit.client.1.smithi178.stdout:4/585: dwrite d0/d8/da/f6b [0,4194304] 0 2024-09-10T11:29:35.618 INFO:tasks.workunit.client.1.smithi178.stdout:4/586: dread - d0/d63/f76 zero size 2024-09-10T11:29:35.621 INFO:tasks.workunit.client.0.smithi144.stdout:2/275: mkdir d7/da/d5b 0 2024-09-10T11:29:35.621 INFO:tasks.workunit.client.0.smithi144.stdout:3/398: write d4/d28/d67/d80/d17/d18/f43 [16950,51657] 0 2024-09-10T11:29:35.625 INFO:tasks.workunit.client.0.smithi144.stdout:5/396: dread d4/fa [0,4194304] 0 2024-09-10T11:29:35.627 INFO:tasks.workunit.client.1.smithi178.stdout:1/650: dwrite d3/d9/d98/dac/d3a/d6e/d8a/f43 [0,4194304] 0 2024-09-10T11:29:35.628 INFO:tasks.workunit.client.0.smithi144.stdout:2/276: mknod d7/de/d3d/c5c 0 2024-09-10T11:29:35.629 INFO:tasks.workunit.client.1.smithi178.stdout:9/768: write d2/df/d31/fc0 [2541300,57169] 0 2024-09-10T11:29:35.629 INFO:tasks.workunit.client.0.smithi144.stdout:2/277: fdatasync d7/da/f19 0 2024-09-10T11:29:35.631 INFO:tasks.workunit.client.0.smithi144.stdout:4/344: dwrite d14/f3a [4194304,4194304] 0 2024-09-10T11:29:35.633 INFO:tasks.workunit.client.1.smithi178.stdout:2/900: sync 2024-09-10T11:29:35.634 INFO:tasks.workunit.client.1.smithi178.stdout:0/676: sync 2024-09-10T11:29:35.635 INFO:tasks.workunit.client.0.smithi144.stdout:3/399: creat d4/dd/d21/f8b x:0 0 0 2024-09-10T11:29:35.638 INFO:tasks.workunit.client.1.smithi178.stdout:6/707: dread d3/d7/d8/d15/fa5 [0,4194304] 0 2024-09-10T11:29:35.639 INFO:tasks.workunit.client.0.smithi144.stdout:1/445: sync 2024-09-10T11:29:35.639 INFO:tasks.workunit.client.1.smithi178.stdout:3/670: truncate db/d35/f77 2518698 0 2024-09-10T11:29:35.642 INFO:tasks.workunit.client.0.smithi144.stdout:5/397: mknod d4/d20/d24/d2d/d4b/c7f 0 2024-09-10T11:29:35.643 INFO:tasks.workunit.client.1.smithi178.stdout:2/901: dread - d1/d41/d14/d11/db9/dc1/ffe zero size 2024-09-10T11:29:35.646 INFO:tasks.workunit.client.1.smithi178.stdout:4/587: write d0/d8/df/f29 [4664292,34617] 0 2024-09-10T11:29:35.647 INFO:tasks.workunit.client.0.smithi144.stdout:2/278: dwrite d7/da/f35 [0,4194304] 0 2024-09-10T11:29:35.647 INFO:tasks.workunit.client.0.smithi144.stdout:8/255: truncate d0/f10 1435532 0 2024-09-10T11:29:35.648 INFO:tasks.workunit.client.1.smithi178.stdout:4/588: readlink d0/d8/la8 0 2024-09-10T11:29:35.648 INFO:tasks.workunit.client.1.smithi178.stdout:4/589: write d0/d8/df/f7c [1378423,34360] 0 2024-09-10T11:29:35.650 INFO:tasks.workunit.client.0.smithi144.stdout:4/345: unlink d14/d67/f6b 0 2024-09-10T11:29:35.652 INFO:tasks.workunit.client.0.smithi144.stdout:1/446: write d5/dd/d64/f7a [763379,15010] 0 2024-09-10T11:29:35.653 INFO:tasks.workunit.client.1.smithi178.stdout:1/651: dwrite d3/d9/d98/dac/d3a/d6e/f7c [4194304,4194304] 0 2024-09-10T11:29:35.655 INFO:tasks.workunit.client.0.smithi144.stdout:8/256: symlink d0/d9/d21/l55 0 2024-09-10T11:29:35.655 INFO:tasks.workunit.client.0.smithi144.stdout:6/294: truncate d9/dd/f49 567943 0 2024-09-10T11:29:35.657 INFO:tasks.workunit.client.0.smithi144.stdout:5/398: link d4/d5/f23 d4/d20/d24/d2d/f80 0 2024-09-10T11:29:35.658 INFO:tasks.workunit.client.0.smithi144.stdout:4/346: rmdir d14/d67/d69 39 2024-09-10T11:29:35.658 INFO:tasks.workunit.client.1.smithi178.stdout:6/708: symlink d3/d74/d91/daa/dc2/l106 0 2024-09-10T11:29:35.660 INFO:tasks.workunit.client.1.smithi178.stdout:9/769: creat d2/df/d31/d37/d3d/fed x:0 0 0 2024-09-10T11:29:35.660 INFO:tasks.workunit.client.1.smithi178.stdout:0/677: read d0/dc/d39/f68 [528545,86307] 0 2024-09-10T11:29:35.661 INFO:tasks.workunit.client.1.smithi178.stdout:6/709: dread - d3/d7/d8/d15/d1c/d2a/d18/d8e/d90/ffb zero size 2024-09-10T11:29:35.661 INFO:tasks.workunit.client.1.smithi178.stdout:3/671: symlink db/d20/d3e/d6b/dc0/ld7 0 2024-09-10T11:29:35.662 INFO:tasks.workunit.client.0.smithi144.stdout:6/295: mknod d9/d38/d6c/c6d 0 2024-09-10T11:29:35.664 INFO:tasks.workunit.client.1.smithi178.stdout:9/770: rmdir d2/d19/d1c 39 2024-09-10T11:29:35.665 INFO:tasks.workunit.client.1.smithi178.stdout:0/678: rmdir d0/dc/d39/d7c/d85 39 2024-09-10T11:29:35.666 INFO:tasks.workunit.client.0.smithi144.stdout:4/347: write d14/d1e/d22/f4a [670223,16607] 0 2024-09-10T11:29:35.666 INFO:tasks.workunit.client.1.smithi178.stdout:2/902: symlink d1/d7/d24/l12d 0 2024-09-10T11:29:35.667 INFO:tasks.workunit.client.1.smithi178.stdout:3/672: fsync db/d20/f3c 0 2024-09-10T11:29:35.670 INFO:tasks.workunit.client.1.smithi178.stdout:9/771: dread - d2/d13/d40/d89/d93/d88/fa8 zero size 2024-09-10T11:29:35.670 INFO:tasks.workunit.client.1.smithi178.stdout:3/673: write db/f1a [1734542,30480] 0 2024-09-10T11:29:35.671 INFO:tasks.workunit.client.0.smithi144.stdout:1/447: dwrite d5/d4a/d44/d80/f67 [0,4194304] 0 2024-09-10T11:29:35.672 INFO:tasks.workunit.client.1.smithi178.stdout:6/710: dwrite d3/d74/dc9/da6/fdd [0,4194304] 0 2024-09-10T11:29:35.673 INFO:tasks.workunit.client.1.smithi178.stdout:4/590: read d0/d8/df/f7c [1210621,59615] 0 2024-09-10T11:29:35.673 INFO:tasks.workunit.client.0.smithi144.stdout:8/257: dwrite d0/d9/d23/d26/f30 [0,4194304] 0 2024-09-10T11:29:35.673 INFO:tasks.workunit.client.1.smithi178.stdout:1/652: getdents d3/d9/d98/dac/d8d 0 2024-09-10T11:29:35.674 INFO:tasks.workunit.client.1.smithi178.stdout:2/903: fdatasync d1/d7/d24/d46/d88/d9d/dab/d5b/d60/f64 0 2024-09-10T11:29:35.675 INFO:tasks.workunit.client.0.smithi144.stdout:6/296: rename d9/d1f/f43 to d9/dd/d3a/f6e 0 2024-09-10T11:29:35.675 INFO:tasks.workunit.client.1.smithi178.stdout:0/679: chown d0/dc/d1f/d2d/dd1/lb9 4 1 2024-09-10T11:29:35.676 INFO:tasks.workunit.client.1.smithi178.stdout:1/653: dread - d3/d9/d98/dac/d3a/fc7 zero size 2024-09-10T11:29:35.677 INFO:tasks.workunit.client.0.smithi144.stdout:4/348: chown d14/d1e/d22/d34/c43 142444 1 2024-09-10T11:29:35.677 INFO:tasks.workunit.client.0.smithi144.stdout:8/258: fsync d0/d9/fe 0 2024-09-10T11:29:35.678 INFO:tasks.workunit.client.1.smithi178.stdout:7/643: sync 2024-09-10T11:29:35.678 INFO:tasks.workunit.client.1.smithi178.stdout:5/719: sync 2024-09-10T11:29:35.678 INFO:tasks.workunit.client.1.smithi178.stdout:8/807: sync 2024-09-10T11:29:35.678 INFO:tasks.workunit.client.0.smithi144.stdout:1/448: truncate d5/df/d4b/d62/f7c 31676 0 2024-09-10T11:29:35.679 INFO:tasks.workunit.client.1.smithi178.stdout:2/904: write d1/d7/d24/d46/d88/d9d/dab/d5b/f10e [966291,30994] 0 2024-09-10T11:29:35.680 INFO:tasks.workunit.client.0.smithi144.stdout:1/449: stat d5/dd/f1d 0 2024-09-10T11:29:35.680 INFO:tasks.workunit.client.0.smithi144.stdout:6/297: creat d9/dd/d3a/f6f x:0 0 0 2024-09-10T11:29:35.682 INFO:tasks.workunit.client.0.smithi144.stdout:6/298: fsync d9/dd/d3e/d47/f16 0 2024-09-10T11:29:35.689 INFO:tasks.workunit.client.1.smithi178.stdout:9/772: rmdir d2/d13/d75 39 2024-09-10T11:29:35.689 INFO:tasks.workunit.client.1.smithi178.stdout:3/674: mknod db/d35/d3a/cd8 0 2024-09-10T11:29:35.689 INFO:tasks.workunit.client.1.smithi178.stdout:1/654: dwrite d3/d9/fc4 [4194304,4194304] 0 2024-09-10T11:29:35.691 INFO:tasks.workunit.client.0.smithi144.stdout:4/349: creat d14/d1e/d22/d34/d3e/d40/f78 x:0 0 0 2024-09-10T11:29:35.693 INFO:tasks.workunit.client.0.smithi144.stdout:5/399: dread d4/d20/d24/d2d/f80 [0,4194304] 0 2024-09-10T11:29:35.693 INFO:tasks.workunit.client.0.smithi144.stdout:3/400: dread d4/dd/d21/d2f/f3a [0,4194304] 0 2024-09-10T11:29:35.694 INFO:tasks.workunit.client.0.smithi144.stdout:3/401: chown d4/dd/d21/c47 55878 1 2024-09-10T11:29:35.694 INFO:tasks.workunit.client.0.smithi144.stdout:8/259: creat d0/d9/d21/d25/d4f/f56 x:0 0 0 2024-09-10T11:29:35.695 INFO:tasks.workunit.client.1.smithi178.stdout:9/773: dread - d2/df/d31/d37/d3d/fed zero size 2024-09-10T11:29:35.695 INFO:tasks.workunit.client.1.smithi178.stdout:5/720: creat d9/dc/dd/d37/dbd/dda/db8/fed x:0 0 0 2024-09-10T11:29:35.696 INFO:tasks.workunit.client.0.smithi144.stdout:3/402: write d4/dd/f1b [166670,51627] 0 2024-09-10T11:29:35.697 INFO:tasks.workunit.client.1.smithi178.stdout:5/721: write d9/dc/dd/d37/dbd/fc6 [615661,77065] 0 2024-09-10T11:29:35.697 INFO:tasks.workunit.client.1.smithi178.stdout:7/644: unlink d6/d10/d1e/d2d/d2b/f41 0 2024-09-10T11:29:35.699 INFO:tasks.workunit.client.1.smithi178.stdout:8/808: fdatasync d5/db/d10/d53/d82/fba 0 2024-09-10T11:29:35.699 INFO:tasks.workunit.client.0.smithi144.stdout:4/350: rename d14/d1e/d22/d34/d3e/d40/f62 to d14/d1e/d22/d34/d3e/d40/d4b/d4c/f79 0 2024-09-10T11:29:35.703 INFO:tasks.workunit.client.0.smithi144.stdout:1/450: rmdir d5/dd/d64/d79/d90 0 2024-09-10T11:29:35.703 INFO:tasks.workunit.client.0.smithi144.stdout:4/351: read d14/d1e/d22/f2d [1112113,43741] 0 2024-09-10T11:29:35.703 INFO:tasks.workunit.client.0.smithi144.stdout:8/260: chown d0/d9/d21/d25/c41 774 1 2024-09-10T11:29:35.704 INFO:tasks.workunit.client.0.smithi144.stdout:3/403: rmdir d4/d28/d67 39 2024-09-10T11:29:35.705 INFO:tasks.workunit.client.1.smithi178.stdout:3/675: mkdir db/d26/d74/dd9 0 2024-09-10T11:29:35.706 INFO:tasks.workunit.client.0.smithi144.stdout:1/451: creat d5/df/d4b/d51/f92 x:0 0 0 2024-09-10T11:29:35.707 INFO:tasks.workunit.client.1.smithi178.stdout:1/655: creat d3/d9/d98/dac/d3a/d50/d6c/ff2 x:0 0 0 2024-09-10T11:29:35.710 INFO:tasks.workunit.client.0.smithi144.stdout:3/404: write d4/d28/d67/d80/d17/d18/d34/f5c [2171938,62766] 0 2024-09-10T11:29:35.711 INFO:tasks.workunit.client.1.smithi178.stdout:7/645: mkdir d6/d10/d1d/d22/d50/def 0 2024-09-10T11:29:35.714 INFO:tasks.workunit.client.0.smithi144.stdout:1/452: creat d5/df/f93 x:0 0 0 2024-09-10T11:29:35.720 INFO:tasks.workunit.client.1.smithi178.stdout:5/722: mkdir d9/dc/dd/d37/dee 0 2024-09-10T11:29:35.720 INFO:tasks.workunit.client.0.smithi144.stdout:4/352: link d14/d1e/d22/f47 d14/d1e/d22/d34/f7a 0 2024-09-10T11:29:35.720 INFO:tasks.workunit.client.1.smithi178.stdout:1/656: dwrite d3/d9/d21/fb6 [0,4194304] 0 2024-09-10T11:29:35.721 INFO:tasks.workunit.client.1.smithi178.stdout:4/591: link d0/d10/c1b d0/d8/da/d1a/d51/d5d/d62/ccc 0 2024-09-10T11:29:35.722 INFO:tasks.workunit.client.1.smithi178.stdout:0/680: dread d0/dc/d39/d50/d5e/f7d [0,4194304] 0 2024-09-10T11:29:35.722 INFO:tasks.workunit.client.0.smithi144.stdout:1/453: write d5/d4a/f7d [989177,67117] 0 2024-09-10T11:29:35.725 INFO:tasks.workunit.client.1.smithi178.stdout:1/657: chown d3/d9/d98/dac/d3a/d50/cec 403 1 2024-09-10T11:29:35.727 INFO:tasks.workunit.client.1.smithi178.stdout:6/711: write d3/d7/d8/d15/d1c/f4d [1443659,14807] 0 2024-09-10T11:29:35.727 INFO:tasks.workunit.client.1.smithi178.stdout:2/905: getdents d1/d41/d8d/d9f 0 2024-09-10T11:29:35.730 INFO:tasks.workunit.client.1.smithi178.stdout:7/646: dwrite d6/d10/d1d/d22/d50/f59 [0,4194304] 0 2024-09-10T11:29:35.731 INFO:tasks.workunit.client.0.smithi144.stdout:4/353: symlink d14/d67/l7b 0 2024-09-10T11:29:35.731 INFO:tasks.workunit.client.1.smithi178.stdout:9/774: link d2/df/f28 d2/d13/d75/d60/d66/fee 0 2024-09-10T11:29:35.733 INFO:tasks.workunit.client.0.smithi144.stdout:3/405: getdents d4 0 2024-09-10T11:29:35.735 INFO:tasks.workunit.client.0.smithi144.stdout:6/299: dread d9/dd/f49 [0,4194304] 0 2024-09-10T11:29:35.735 INFO:tasks.workunit.client.0.smithi144.stdout:4/354: write d14/d1e/d22/d2b/d35/f5b [3705135,26350] 0 2024-09-10T11:29:35.736 INFO:tasks.workunit.client.1.smithi178.stdout:0/681: write d0/dc/d39/d4c/f73 [8780298,73870] 0 2024-09-10T11:29:35.738 INFO:tasks.workunit.client.0.smithi144.stdout:4/355: chown d14/l36 1647 1 2024-09-10T11:29:35.739 INFO:tasks.workunit.client.0.smithi144.stdout:6/300: unlink d9/dd/d3e/d47/le 0 2024-09-10T11:29:35.740 INFO:tasks.workunit.client.0.smithi144.stdout:4/356: write d14/d1e/d22/d34/f75 [393246,70587] 0 2024-09-10T11:29:35.740 INFO:tasks.workunit.client.1.smithi178.stdout:1/658: symlink d3/d9/d98/dac/d3a/d50/d6c/d78/lf3 0 2024-09-10T11:29:35.743 INFO:tasks.workunit.client.1.smithi178.stdout:5/723: fdatasync d9/dc/dd/d37/d42/d55/f6e 0 2024-09-10T11:29:35.746 INFO:tasks.workunit.client.0.smithi144.stdout:4/357: dread - d14/d1e/d22/d34/d3e/d40/d4b/d4c/f79 zero size 2024-09-10T11:29:35.746 INFO:tasks.workunit.client.1.smithi178.stdout:7/647: rmdir d6/d10/d1d/d4a/d79 39 2024-09-10T11:29:35.747 INFO:tasks.workunit.client.1.smithi178.stdout:6/712: write d3/d7/d8/d72/fd5 [76247,79370] 0 2024-09-10T11:29:35.747 INFO:tasks.workunit.client.0.smithi144.stdout:6/301: mknod d9/d1f/d2c/c70 0 2024-09-10T11:29:35.747 INFO:tasks.workunit.client.0.smithi144.stdout:3/406: dwrite d4/d28/d67/d80/d17/d18/d34/d4b/d63/f7b [0,4194304] 0 2024-09-10T11:29:35.748 INFO:tasks.workunit.client.1.smithi178.stdout:4/592: dread d0/d8/df/f7c [0,4194304] 0 2024-09-10T11:29:35.749 INFO:tasks.workunit.client.0.smithi144.stdout:1/454: dread d5/df/d4b/f54 [0,4194304] 0 2024-09-10T11:29:35.749 INFO:tasks.workunit.client.0.smithi144.stdout:1/455: stat d5/df/l3a 0 2024-09-10T11:29:35.750 INFO:tasks.workunit.client.0.smithi144.stdout:6/302: write d9/d1f/d2c/d4f/f58 [802078,115024] 0 2024-09-10T11:29:35.751 INFO:tasks.workunit.client.0.smithi144.stdout:4/358: creat d14/d1e/f7c x:0 0 0 2024-09-10T11:29:35.752 INFO:tasks.workunit.client.1.smithi178.stdout:2/906: write d1/d7/d24/d7d/d84/d55/f5c [1063577,127412] 0 2024-09-10T11:29:35.752 INFO:tasks.workunit.client.0.smithi144.stdout:6/303: dread d9/d1f/f26 [0,4194304] 0 2024-09-10T11:29:35.753 INFO:tasks.workunit.client.0.smithi144.stdout:3/407: creat d4/d1a/d7a/f8c x:0 0 0 2024-09-10T11:29:35.753 INFO:tasks.workunit.client.1.smithi178.stdout:0/682: truncate d0/dc/d39/d4c/f77 1071736 0 2024-09-10T11:29:35.753 INFO:tasks.workunit.client.1.smithi178.stdout:8/809: dwrite d5/d1d/f50 [0,4194304] 0 2024-09-10T11:29:35.754 INFO:tasks.workunit.client.0.smithi144.stdout:3/408: write d4/f58 [1734692,104412] 0 2024-09-10T11:29:35.756 INFO:tasks.workunit.client.0.smithi144.stdout:3/409: chown d4/dd/d21/f86 122780 1 2024-09-10T11:29:35.757 INFO:tasks.workunit.client.0.smithi144.stdout:3/410: write d4/d28/d67/d80/d17/d18/f48 [2115036,38828] 0 2024-09-10T11:29:35.758 INFO:tasks.workunit.client.0.smithi144.stdout:3/411: chown d4/d28/f38 382655 1 2024-09-10T11:29:35.759 INFO:tasks.workunit.client.1.smithi178.stdout:3/676: dwrite db/d20/fb9 [0,4194304] 0 2024-09-10T11:29:35.759 INFO:tasks.workunit.client.1.smithi178.stdout:6/713: readlink d3/d7/d8/d15/d1c/le3 0 2024-09-10T11:29:35.760 INFO:tasks.workunit.client.1.smithi178.stdout:9/775: link d2/df/d31/f47 d2/fef 0 2024-09-10T11:29:35.761 INFO:tasks.workunit.client.1.smithi178.stdout:9/776: fdatasync d2/d13/d75/d60/d66/fb9 0 2024-09-10T11:29:35.762 INFO:tasks.workunit.client.0.smithi144.stdout:1/456: getdents d5/d4a/d63/d84 0 2024-09-10T11:29:35.762 INFO:tasks.workunit.client.0.smithi144.stdout:3/412: rmdir d4 39 2024-09-10T11:29:35.763 INFO:tasks.workunit.client.0.smithi144.stdout:1/457: chown d5/dd/f8d 18485389 1 2024-09-10T11:29:35.764 INFO:tasks.workunit.client.0.smithi144.stdout:4/359: link d14/d1e/d22/d2b/l30 d14/d1e/d22/d34/d3e/d40/d4b/l7d 0 2024-09-10T11:29:35.765 INFO:tasks.workunit.client.1.smithi178.stdout:1/659: mkdir d3/d35/ddd/df4 0 2024-09-10T11:29:35.765 INFO:tasks.workunit.client.1.smithi178.stdout:7/648: chown d6/d10/d34/d6a/lde 3 1 2024-09-10T11:29:35.765 INFO:tasks.workunit.client.0.smithi144.stdout:4/360: fsync d14/d1e/d22/d34/f75 0 2024-09-10T11:29:35.766 INFO:tasks.workunit.client.1.smithi178.stdout:5/724: chown d9/dc/dd/d37/dbd/dda/fbb 3448003 1 2024-09-10T11:29:35.768 INFO:tasks.workunit.client.1.smithi178.stdout:9/777: chown d2/d19/d5e/f78 0 1 2024-09-10T11:29:35.768 INFO:tasks.workunit.client.0.smithi144.stdout:3/413: creat d4/dd/d21/d2f/f8d x:0 0 0 2024-09-10T11:29:35.769 INFO:tasks.workunit.client.1.smithi178.stdout:3/677: symlink db/d20/d3d/d63/da2/d98/lda 0 2024-09-10T11:29:35.770 INFO:tasks.workunit.client.0.smithi144.stdout:1/458: link d5/d2a/f89 d5/d4a/d44/f94 0 2024-09-10T11:29:35.771 INFO:tasks.workunit.client.1.smithi178.stdout:0/683: truncate d0/d2c/d95/dbd/dd5/fc7 819785 0 2024-09-10T11:29:35.773 INFO:tasks.workunit.client.1.smithi178.stdout:1/660: fsync d3/d9/d1a/fdb 0 2024-09-10T11:29:35.773 INFO:tasks.workunit.client.1.smithi178.stdout:6/714: write d3/d7/d8/d15/d1c/d2a/fc0 [1155408,54280] 0 2024-09-10T11:29:35.774 INFO:tasks.workunit.client.1.smithi178.stdout:3/678: rmdir db/d26/d40 39 2024-09-10T11:29:35.775 INFO:tasks.workunit.client.1.smithi178.stdout:6/715: stat d3/d6e/fba 0 2024-09-10T11:29:35.776 INFO:tasks.workunit.client.1.smithi178.stdout:3/679: write db/d20/d70/f8e [2117892,121122] 0 2024-09-10T11:29:35.777 INFO:tasks.workunit.client.0.smithi144.stdout:3/414: rename d4/d28/d67/d80/d17/d18/d34/d4b/d63 to d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e 0 2024-09-10T11:29:35.778 INFO:tasks.workunit.client.0.smithi144.stdout:1/459: unlink d5/df/d4b/c52 0 2024-09-10T11:29:35.779 INFO:tasks.workunit.client.1.smithi178.stdout:0/684: creat d0/dc/d1f/dae/fd9 x:0 0 0 2024-09-10T11:29:35.781 INFO:tasks.workunit.client.1.smithi178.stdout:4/593: rename d0/d8/da/d1a/d51/d5d/c5f to d0/d8/df/ccd 0 2024-09-10T11:29:35.781 INFO:tasks.workunit.client.0.smithi144.stdout:1/460: mkdir d5/d4a/d63/d84/d95 0 2024-09-10T11:29:35.785 INFO:tasks.workunit.client.0.smithi144.stdout:3/415: getdents d4/d1a/d7a 0 2024-09-10T11:29:35.786 INFO:tasks.workunit.client.1.smithi178.stdout:1/661: chown d3/d9/d98/dac/d3a/d50/cd6 1350 1 2024-09-10T11:29:35.787 INFO:tasks.workunit.client.1.smithi178.stdout:6/716: creat d3/d74/d91/daa/f107 x:0 0 0 2024-09-10T11:29:35.787 INFO:tasks.workunit.client.0.smithi144.stdout:6/304: truncate d9/d1f/d2c/f53 838867 0 2024-09-10T11:29:35.788 INFO:tasks.workunit.client.0.smithi144.stdout:6/305: dread - d9/dd/f33 zero size 2024-09-10T11:29:35.788 INFO:tasks.workunit.client.0.smithi144.stdout:3/416: symlink d4/dd/l8f 0 2024-09-10T11:29:35.789 INFO:tasks.workunit.client.1.smithi178.stdout:7/649: creat d6/d10/d1d/d4a/ff0 x:0 0 0 2024-09-10T11:29:35.789 INFO:tasks.workunit.client.1.smithi178.stdout:2/907: truncate d1/d22/d4d/d9b/f75 587326 0 2024-09-10T11:29:35.790 INFO:tasks.workunit.client.1.smithi178.stdout:6/717: write d3/d7/d8/d15/d1c/d2a/d18/d8e/d90/fe5 [4004852,45451] 0 2024-09-10T11:29:35.792 INFO:tasks.workunit.client.0.smithi144.stdout:1/461: link d5/d2a/d65/d73/f24 d5/d2a/f96 0 2024-09-10T11:29:35.793 INFO:tasks.workunit.client.1.smithi178.stdout:6/718: truncate d3/d7/d8/d72/fd5 1107884 0 2024-09-10T11:29:35.794 INFO:tasks.workunit.client.1.smithi178.stdout:5/725: dread d9/dc/dd/f7f [0,4194304] 0 2024-09-10T11:29:35.798 INFO:tasks.workunit.client.0.smithi144.stdout:0/243: sync 2024-09-10T11:29:35.800 INFO:tasks.workunit.client.0.smithi144.stdout:0/244: fdatasync da/f36 0 2024-09-10T11:29:35.801 INFO:tasks.workunit.client.0.smithi144.stdout:6/306: dwrite d9/f2a [0,4194304] 0 2024-09-10T11:29:35.802 INFO:tasks.workunit.client.1.smithi178.stdout:8/810: getdents d5/db/d10 0 2024-09-10T11:29:35.803 INFO:tasks.workunit.client.0.smithi144.stdout:1/462: creat d5/df/d4b/d85/f97 x:0 0 0 2024-09-10T11:29:35.805 INFO:tasks.workunit.client.1.smithi178.stdout:0/685: getdents d0/dc/d1f/d2d/dd1 0 2024-09-10T11:29:35.807 INFO:tasks.workunit.client.0.smithi144.stdout:0/245: mkdir da/d49 0 2024-09-10T11:29:35.807 INFO:tasks.workunit.client.1.smithi178.stdout:5/726: dread d9/f5e [0,4194304] 0 2024-09-10T11:29:35.808 INFO:tasks.workunit.client.1.smithi178.stdout:9/778: dwrite d2/d19/d1c/d2a/f67 [0,4194304] 0 2024-09-10T11:29:35.809 INFO:tasks.workunit.client.0.smithi144.stdout:1/463: read d5/df/d4b/f8e [912611,87291] 0 2024-09-10T11:29:35.809 INFO:tasks.workunit.client.0.smithi144.stdout:3/417: dwrite d4/d28/d67/d80/d17/f30 [4194304,4194304] 0 2024-09-10T11:29:35.810 INFO:tasks.workunit.client.1.smithi178.stdout:2/908: mkdir d1/d22/d4d/d78/d12e 0 2024-09-10T11:29:35.812 INFO:tasks.workunit.client.1.smithi178.stdout:6/719: creat d3/d74/dc9/da6/f108 x:0 0 0 2024-09-10T11:29:35.812 INFO:tasks.workunit.client.0.smithi144.stdout:6/307: dwrite d9/dd/f1d [0,4194304] 0 2024-09-10T11:29:35.814 INFO:tasks.workunit.client.1.smithi178.stdout:4/594: creat d0/d9/d3d/dc2/fce x:0 0 0 2024-09-10T11:29:35.816 INFO:tasks.workunit.client.1.smithi178.stdout:8/811: creat d5/d30/f10d x:0 0 0 2024-09-10T11:29:35.817 INFO:tasks.workunit.client.1.smithi178.stdout:8/812: chown d5/d1d/d22/d76/l102 9 1 2024-09-10T11:29:35.819 INFO:tasks.workunit.client.1.smithi178.stdout:8/813: dread - d5/db/d10/d20/dc0/fd7 zero size 2024-09-10T11:29:35.822 INFO:tasks.workunit.client.0.smithi144.stdout:3/418: mkdir d4/d28/d67/d80/d17/d18/d34/d65/d75/d90 0 2024-09-10T11:29:35.823 INFO:tasks.workunit.client.0.smithi144.stdout:1/464: mkdir d5/dd/d8f/d98 0 2024-09-10T11:29:35.824 INFO:tasks.workunit.client.1.smithi178.stdout:3/680: rmdir db/d26/d40/d68/db6 39 2024-09-10T11:29:35.825 INFO:tasks.workunit.client.1.smithi178.stdout:3/681: dread - db/d20/d3d/d63/da2/d98/fbc zero size 2024-09-10T11:29:35.826 INFO:tasks.workunit.client.0.smithi144.stdout:0/246: dread da/f21 [0,4194304] 0 2024-09-10T11:29:35.826 INFO:tasks.workunit.client.0.smithi144.stdout:0/247: chown da/f2a 450286 1 2024-09-10T11:29:35.827 INFO:tasks.workunit.client.1.smithi178.stdout:3/682: chown db/d20/d3d/d63/da2/d98/fbe 153384 1 2024-09-10T11:29:35.827 INFO:tasks.workunit.client.1.smithi178.stdout:7/650: rename d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d7c to d6/d10/d1d/d4a/d79/df1 0 2024-09-10T11:29:35.828 INFO:tasks.workunit.client.0.smithi144.stdout:0/248: dread - da/f1a zero size 2024-09-10T11:29:35.828 INFO:tasks.workunit.client.0.smithi144.stdout:6/308: getdents d9/dd/d11/d1b 0 2024-09-10T11:29:35.828 INFO:tasks.workunit.client.1.smithi178.stdout:7/651: chown d6/d10/d1e/ld3 4441186 1 2024-09-10T11:29:35.829 INFO:tasks.workunit.client.0.smithi144.stdout:3/419: mknod d4/d28/d67/d80/d17/d18/d34/d4b/c91 0 2024-09-10T11:29:35.829 INFO:tasks.workunit.client.0.smithi144.stdout:6/309: read d9/dd/f49 [282051,94652] 0 2024-09-10T11:29:35.829 INFO:tasks.workunit.client.1.smithi178.stdout:7/652: write d6/d10/d34/f44 [616213,26340] 0 2024-09-10T11:29:35.830 INFO:tasks.workunit.client.0.smithi144.stdout:3/420: fsync d4/dd/d21/f5d 0 2024-09-10T11:29:35.831 INFO:tasks.workunit.client.0.smithi144.stdout:0/249: creat da/d48/f4a x:0 0 0 2024-09-10T11:29:35.831 INFO:tasks.workunit.client.1.smithi178.stdout:2/909: creat d1/d7/d24/d46/d88/d9d/dab/d5e/f12f x:0 0 0 2024-09-10T11:29:35.831 INFO:tasks.workunit.client.0.smithi144.stdout:3/421: stat d4/dd/d21/f62 0 2024-09-10T11:29:35.832 INFO:tasks.workunit.client.1.smithi178.stdout:8/814: fsync d5/db/d1e/fa9 0 2024-09-10T11:29:35.835 INFO:tasks.workunit.client.0.smithi144.stdout:6/310: rename d9/dd/d3a/f5d to d9/f71 0 2024-09-10T11:29:35.837 INFO:tasks.workunit.client.0.smithi144.stdout:6/311: read d9/d1f/f36 [56615,7950] 0 2024-09-10T11:29:35.838 INFO:tasks.workunit.client.0.smithi144.stdout:3/422: rename d4/d1a/d7a to d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92 0 2024-09-10T11:29:35.841 INFO:tasks.workunit.client.1.smithi178.stdout:6/720: unlink d3/d6e/lc4 0 2024-09-10T11:29:35.841 INFO:tasks.workunit.client.0.smithi144.stdout:6/312: rename d9/dd/d3e/d47/d13/d4a/d5b to d9/dd/d3e/d47/d13/d4a/d5b/d72 22 2024-09-10T11:29:35.842 INFO:tasks.workunit.client.0.smithi144.stdout:3/423: truncate d4/d28/d67/d80/d17/d18/f43 832832 0 2024-09-10T11:29:35.842 INFO:tasks.workunit.client.1.smithi178.stdout:4/595: mkdir d0/d9/dcf 0 2024-09-10T11:29:35.843 INFO:tasks.workunit.client.0.smithi144.stdout:1/465: dread d5/d2a/f46 [0,4194304] 0 2024-09-10T11:29:35.844 INFO:tasks.workunit.client.0.smithi144.stdout:1/466: chown d5/d2a/f34 1118 1 2024-09-10T11:29:35.844 INFO:tasks.workunit.client.0.smithi144.stdout:6/313: mkdir d9/dd/d73 0 2024-09-10T11:29:35.846 INFO:tasks.workunit.client.1.smithi178.stdout:5/727: link d9/d11/c93 d9/dc/dd/d37/dbd/cef 0 2024-09-10T11:29:35.846 INFO:tasks.workunit.client.0.smithi144.stdout:1/467: write d5/f41 [1289486,8870] 0 2024-09-10T11:29:35.847 INFO:tasks.workunit.client.0.smithi144.stdout:1/468: chown d5/df/f55 3 1 2024-09-10T11:29:35.848 INFO:tasks.workunit.client.0.smithi144.stdout:3/424: mkdir d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93 0 2024-09-10T11:29:35.849 INFO:tasks.workunit.client.1.smithi178.stdout:2/910: rename d1/d7/d24/d46/d88/d9d/dab/d5b/d7c to d1/d7/d24/d7d/d84/d55/da1/d130 0 2024-09-10T11:29:35.851 INFO:tasks.workunit.client.1.smithi178.stdout:0/686: truncate d0/dc/dc4/f8a 3435279 0 2024-09-10T11:29:35.851 INFO:tasks.workunit.client.1.smithi178.stdout:0/687: readlink d0/d2c/d95/la2 0 2024-09-10T11:29:35.851 INFO:tasks.workunit.client.0.smithi144.stdout:6/314: mknod d9/dd/d3e/d47/c74 0 2024-09-10T11:29:35.852 INFO:tasks.workunit.client.0.smithi144.stdout:0/250: truncate da/f16 2810912 0 2024-09-10T11:29:35.853 INFO:tasks.workunit.client.0.smithi144.stdout:0/251: dread - da/f28 zero size 2024-09-10T11:29:35.854 INFO:tasks.workunit.client.1.smithi178.stdout:6/721: creat d3/d74/d91/daa/dc2/dcf/f109 x:0 0 0 2024-09-10T11:29:35.854 INFO:tasks.workunit.client.0.smithi144.stdout:1/469: creat d5/dd/f99 x:0 0 0 2024-09-10T11:29:35.854 INFO:tasks.workunit.client.0.smithi144.stdout:6/315: rmdir d9/dd/d3e 39 2024-09-10T11:29:35.855 INFO:tasks.workunit.client.0.smithi144.stdout:0/252: chown c6 6068863 1 2024-09-10T11:29:35.857 INFO:tasks.workunit.client.0.smithi144.stdout:3/425: dwrite d4/d1a/f5f [0,4194304] 0 2024-09-10T11:29:35.858 INFO:tasks.workunit.client.0.smithi144.stdout:6/316: write d9/dd/f33 [577246,101365] 0 2024-09-10T11:29:35.858 INFO:tasks.workunit.client.1.smithi178.stdout:8/815: write d5/db/d3d/de7/ffd [288024,7555] 0 2024-09-10T11:29:35.858 INFO:tasks.workunit.client.0.smithi144.stdout:0/253: mkdir da/d3a/d4b 0 2024-09-10T11:29:35.860 INFO:tasks.workunit.client.1.smithi178.stdout:5/728: unlink d9/dc/d17/d1d/fb1 0 2024-09-10T11:29:35.861 INFO:tasks.workunit.client.1.smithi178.stdout:4/596: dread d0/d8/d3e/f4c [4194304,4194304] 0 2024-09-10T11:29:35.861 INFO:tasks.workunit.client.1.smithi178.stdout:5/729: chown d9/dc/dd/d16/f41 11603902 1 2024-09-10T11:29:35.862 INFO:tasks.workunit.client.1.smithi178.stdout:2/911: unlink d1/d22/d4d/d9b/f92 0 2024-09-10T11:29:35.863 INFO:tasks.workunit.client.0.smithi144.stdout:3/426: symlink d4/d28/d67/d80/d17/d18/d34/l94 0 2024-09-10T11:29:35.864 INFO:tasks.workunit.client.0.smithi144.stdout:3/427: chown d4/c40 1747163 1 2024-09-10T11:29:35.866 INFO:tasks.workunit.client.0.smithi144.stdout:6/317: unlink d9/d1f/d2c/c3c 0 2024-09-10T11:29:35.866 INFO:tasks.workunit.client.0.smithi144.stdout:3/428: fdatasync d4/f32 0 2024-09-10T11:29:35.868 INFO:tasks.workunit.client.1.smithi178.stdout:8/816: write d5/fae [2073129,18486] 0 2024-09-10T11:29:35.868 INFO:tasks.workunit.client.0.smithi144.stdout:0/254: dwrite da/d3a/f3e [0,4194304] 0 2024-09-10T11:29:35.869 INFO:tasks.workunit.client.0.smithi144.stdout:1/470: link d5/df/d4b/d51/f92 d5/dd/d64/f9a 0 2024-09-10T11:29:35.871 INFO:tasks.workunit.client.0.smithi144.stdout:3/429: dread d4/d28/d67/f7c [0,4194304] 0 2024-09-10T11:29:35.873 INFO:tasks.workunit.client.1.smithi178.stdout:4/597: symlink d0/d9/ld0 0 2024-09-10T11:29:35.875 INFO:tasks.workunit.client.1.smithi178.stdout:3/683: dwrite db/f16 [0,4194304] 0 2024-09-10T11:29:35.880 INFO:tasks.workunit.client.1.smithi178.stdout:6/722: getdents d3/d7 0 2024-09-10T11:29:35.884 INFO:tasks.workunit.client.0.smithi144.stdout:0/255: rmdir da 39 2024-09-10T11:29:35.884 INFO:tasks.workunit.client.0.smithi144.stdout:1/471: mknod d5/df/d4b/d51/c9b 0 2024-09-10T11:29:35.885 INFO:tasks.workunit.client.1.smithi178.stdout:0/688: write d0/d12/f64 [3868465,91937] 0 2024-09-10T11:29:35.886 INFO:tasks.workunit.client.1.smithi178.stdout:5/730: write d9/dc/d17/d1d/f8c [3507415,128112] 0 2024-09-10T11:29:35.886 INFO:tasks.workunit.client.1.smithi178.stdout:2/912: write d1/d7/d24/d7d/d84/fb6 [2596043,12718] 0 2024-09-10T11:29:35.887 INFO:tasks.workunit.client.0.smithi144.stdout:3/430: symlink d4/d28/d67/d80/d17/d18/d34/d65/l95 0 2024-09-10T11:29:35.888 INFO:tasks.workunit.client.0.smithi144.stdout:3/431: fsync d4/f58 0 2024-09-10T11:29:35.889 INFO:tasks.workunit.client.1.smithi178.stdout:8/817: dwrite d5/db/d10/d20/f28 [0,4194304] 0 2024-09-10T11:29:35.890 INFO:tasks.workunit.client.1.smithi178.stdout:0/689: mknod d0/d12/d3f/cda 0 2024-09-10T11:29:35.892 INFO:tasks.workunit.client.0.smithi144.stdout:6/318: creat d9/dd/d11/d1b/d62/d68/f75 x:0 0 0 2024-09-10T11:29:35.894 INFO:tasks.workunit.client.1.smithi178.stdout:5/731: rename d9/d11 to d9/dc/dd/d37/d42/d62/daf/df0 0 2024-09-10T11:29:35.895 INFO:tasks.workunit.client.1.smithi178.stdout:8/818: symlink d5/d30/l10e 0 2024-09-10T11:29:35.897 INFO:tasks.workunit.client.1.smithi178.stdout:3/684: truncate db/d20/fb9 3421167 0 2024-09-10T11:29:35.899 INFO:tasks.workunit.client.0.smithi144.stdout:0/256: dwrite da/f30 [0,4194304] 0 2024-09-10T11:29:35.899 INFO:tasks.workunit.client.1.smithi178.stdout:2/913: creat d1/d7/d24/d46/d88/d9d/f131 x:0 0 0 2024-09-10T11:29:35.900 INFO:tasks.workunit.client.0.smithi144.stdout:1/472: rename d5/dd/d8f/d98 to d5/dd/d9c 0 2024-09-10T11:29:35.901 INFO:tasks.workunit.client.1.smithi178.stdout:0/690: unlink d0/dc/d1f/d2d/f42 0 2024-09-10T11:29:35.902 INFO:tasks.workunit.client.0.smithi144.stdout:1/473: write d5/d4a/d44/d80/f67 [2348991,9154] 0 2024-09-10T11:29:35.902 INFO:tasks.workunit.client.1.smithi178.stdout:0/691: chown d0/dc/d39/d50/c7e 40695932 1 2024-09-10T11:29:35.903 INFO:tasks.workunit.client.1.smithi178.stdout:8/819: mkdir d5/d1d/d22/d10f 0 2024-09-10T11:29:35.905 INFO:tasks.workunit.client.1.smithi178.stdout:8/820: write d5/db/dda/fe1 [325189,109935] 0 2024-09-10T11:29:35.905 INFO:tasks.workunit.client.0.smithi144.stdout:3/432: fsync f2 0 2024-09-10T11:29:35.906 INFO:tasks.workunit.client.0.smithi144.stdout:6/319: dread d9/dd/f1d [0,4194304] 0 2024-09-10T11:29:35.907 INFO:tasks.workunit.client.1.smithi178.stdout:5/732: rename d9/dc/dd/d37/d42/d62/daf/df0/f2f to d9/dc/d17/dcd/ff1 0 2024-09-10T11:29:35.907 INFO:tasks.workunit.client.0.smithi144.stdout:1/474: mknod d5/df/d4b/d62/c9d 0 2024-09-10T11:29:35.907 INFO:tasks.workunit.client.0.smithi144.stdout:1/475: stat d5/d2a/f96 0 2024-09-10T11:29:35.909 INFO:tasks.workunit.client.0.smithi144.stdout:1/476: chown d5/df/d4b/d51/l5f 138 1 2024-09-10T11:29:35.911 INFO:tasks.workunit.client.0.smithi144.stdout:1/477: write d5/f23 [4219588,1211] 0 2024-09-10T11:29:35.912 INFO:tasks.workunit.client.0.smithi144.stdout:3/433: fsync d4/d28/d67/d80/d17/d18/f6d 0 2024-09-10T11:29:35.912 INFO:tasks.workunit.client.0.smithi144.stdout:1/478: chown d5/df 29 1 2024-09-10T11:29:35.915 INFO:tasks.workunit.client.1.smithi178.stdout:6/723: dread d3/d74/dc9/f47 [0,4194304] 0 2024-09-10T11:29:35.915 INFO:tasks.workunit.client.1.smithi178.stdout:0/692: rename d0/d2c/f7b to d0/dc/d1f/dc0/fdb 0 2024-09-10T11:29:35.915 INFO:tasks.workunit.client.0.smithi144.stdout:0/257: rename da/f16 to da/d49/f4c 0 2024-09-10T11:29:35.915 INFO:tasks.workunit.client.1.smithi178.stdout:3/685: creat db/d20/d3d/d63/d6a/fdb x:0 0 0 2024-09-10T11:29:35.916 INFO:tasks.workunit.client.0.smithi144.stdout:0/258: truncate da/f2e 4369295 0 2024-09-10T11:29:35.916 INFO:tasks.workunit.client.0.smithi144.stdout:6/320: creat d9/dd/d3e/f76 x:0 0 0 2024-09-10T11:29:35.917 INFO:tasks.workunit.client.1.smithi178.stdout:0/693: read - d0/dc/d39/d50/fc3 zero size 2024-09-10T11:29:35.917 INFO:tasks.workunit.client.1.smithi178.stdout:3/686: chown db/d20/d70/fbf 38335 1 2024-09-10T11:29:35.917 INFO:tasks.workunit.client.1.smithi178.stdout:5/733: dwrite d9/dc/dd/d3f/d5a/dbc/fca [0,4194304] 0 2024-09-10T11:29:35.917 INFO:tasks.workunit.client.1.smithi178.stdout:6/724: read - d3/d7/d8/d15/d1c/d2a/d18/ffd zero size 2024-09-10T11:29:35.921 INFO:tasks.workunit.client.0.smithi144.stdout:1/479: unlink d5/d2a/c30 0 2024-09-10T11:29:35.922 INFO:tasks.workunit.client.1.smithi178.stdout:3/687: unlink db/d20/d3e/d6b/dc0/ld7 0 2024-09-10T11:29:35.923 INFO:tasks.workunit.client.0.smithi144.stdout:3/434: mkdir d4/d28/d67/d96 0 2024-09-10T11:29:35.923 INFO:tasks.workunit.client.1.smithi178.stdout:3/688: chown db/d20/d70 9227504 1 2024-09-10T11:29:35.924 INFO:tasks.workunit.client.0.smithi144.stdout:0/259: creat da/d3a/f4d x:0 0 0 2024-09-10T11:29:35.926 INFO:tasks.workunit.client.1.smithi178.stdout:3/689: dread - db/d35/f5e zero size 2024-09-10T11:29:35.926 INFO:tasks.workunit.client.0.smithi144.stdout:1/480: truncate d5/d4a/f2f 54915 0 2024-09-10T11:29:35.928 INFO:tasks.workunit.client.1.smithi178.stdout:3/690: truncate db/d35/d3a/d9f/fb8 211352 0 2024-09-10T11:29:35.929 INFO:tasks.workunit.client.0.smithi144.stdout:6/321: link d9/dd/d3e/d47/d13/d4a/d5b/f61 d9/d1f/f77 0 2024-09-10T11:29:35.929 INFO:tasks.workunit.client.1.smithi178.stdout:5/734: dread d9/dc/dd/d37/d42/d62/f80 [0,4194304] 0 2024-09-10T11:29:35.929 INFO:tasks.workunit.client.0.smithi144.stdout:6/322: chown d9/lb 711211 1 2024-09-10T11:29:35.930 INFO:tasks.workunit.client.1.smithi178.stdout:3/691: fsync db/d35/f5e 0 2024-09-10T11:29:35.931 INFO:tasks.workunit.client.1.smithi178.stdout:8/821: write d5/d30/d33/d75/f8b [798690,31912] 0 2024-09-10T11:29:35.931 INFO:tasks.workunit.client.0.smithi144.stdout:1/481: creat d5/d4a/d63/d84/f9e x:0 0 0 2024-09-10T11:29:35.932 INFO:tasks.workunit.client.1.smithi178.stdout:8/822: fsync d5/d1d/f25 0 2024-09-10T11:29:35.933 INFO:tasks.workunit.client.1.smithi178.stdout:5/735: creat d9/dc/dd/d37/d42/d62/daf/df0/ff2 x:0 0 0 2024-09-10T11:29:35.933 INFO:tasks.workunit.client.0.smithi144.stdout:6/323: getdents d9/dd/d73 0 2024-09-10T11:29:35.935 INFO:tasks.workunit.client.1.smithi178.stdout:0/694: rename d0/dc/d1f/dc0 to d0/dc/d39/da4/dce/ddc 0 2024-09-10T11:29:35.936 INFO:tasks.workunit.client.1.smithi178.stdout:3/692: mkdir db/d35/d58/ddc 0 2024-09-10T11:29:35.937 INFO:tasks.workunit.client.0.smithi144.stdout:6/324: unlink d9/dd/d3e/d47/f2e 0 2024-09-10T11:29:35.937 INFO:tasks.workunit.client.0.smithi144.stdout:6/325: readlink l2 0 2024-09-10T11:29:35.937 INFO:tasks.workunit.client.1.smithi178.stdout:3/693: write db/d20/d3e/d6b/dc0/fc2 [997345,33982] 0 2024-09-10T11:29:35.940 INFO:tasks.workunit.client.1.smithi178.stdout:6/725: write d3/d74/dc9/f80 [665519,37611] 0 2024-09-10T11:29:35.942 INFO:tasks.workunit.client.0.smithi144.stdout:6/326: dread d9/dd/d11/f4c [0,4194304] 0 2024-09-10T11:29:35.942 INFO:tasks.workunit.client.1.smithi178.stdout:8/823: getdents d5/d1d/d22/d10f 0 2024-09-10T11:29:35.943 INFO:tasks.workunit.client.1.smithi178.stdout:8/824: dread - d5/d30/db8/fe9 zero size 2024-09-10T11:29:35.944 INFO:tasks.workunit.client.0.smithi144.stdout:3/435: dread d4/d28/d67/d80/d17/d18/f48 [0,4194304] 0 2024-09-10T11:29:35.945 INFO:tasks.workunit.client.1.smithi178.stdout:3/694: mkdir db/ddd 0 2024-09-10T11:29:35.945 INFO:tasks.workunit.client.0.smithi144.stdout:6/327: fdatasync d9/d1f/f36 0 2024-09-10T11:29:35.945 INFO:tasks.workunit.client.0.smithi144.stdout:3/436: chown d4/d28/c7d 9992802 1 2024-09-10T11:29:35.946 INFO:tasks.workunit.client.0.smithi144.stdout:6/328: stat d9/dd/d3e/d47/d13/f19 0 2024-09-10T11:29:35.946 INFO:tasks.workunit.client.1.smithi178.stdout:6/726: fdatasync d3/d6e/fba 0 2024-09-10T11:29:35.947 INFO:tasks.workunit.client.0.smithi144.stdout:6/329: readlink d9/d1f/d2c/l6b 0 2024-09-10T11:29:35.948 INFO:tasks.workunit.client.1.smithi178.stdout:6/727: write d3/d7/d8/d15/fbc [2588478,102781] 0 2024-09-10T11:29:35.948 INFO:tasks.workunit.client.0.smithi144.stdout:3/437: chown d4/d28/d67/d80/d17/c72 0 1 2024-09-10T11:29:35.949 INFO:tasks.workunit.client.1.smithi178.stdout:5/736: rename d9/dc/dd/d37/d42/d62/cc2 to d9/cf3 0 2024-09-10T11:29:35.949 INFO:tasks.workunit.client.1.smithi178.stdout:3/695: rename db/d20/d3e/d6b/dc0 to db/d20/d3e/d6b/dc0/dde 22 2024-09-10T11:29:35.949 INFO:tasks.workunit.client.0.smithi144.stdout:0/260: truncate da/f30 2293547 0 2024-09-10T11:29:35.950 INFO:tasks.workunit.client.1.smithi178.stdout:5/737: chown d9/dc/dd/d37/d3e/l85 9 1 2024-09-10T11:29:35.951 INFO:tasks.workunit.client.1.smithi178.stdout:8/825: write d5/db/d10/d20/dc0/fdd [970124,35132] 0 2024-09-10T11:29:35.952 INFO:tasks.workunit.client.0.smithi144.stdout:1/482: truncate d5/d4a/d44/d80/f66 1567521 0 2024-09-10T11:29:35.953 INFO:tasks.workunit.client.1.smithi178.stdout:8/826: readlink d5/d1d/d22/d76/l7a 0 2024-09-10T11:29:35.953 INFO:tasks.workunit.client.0.smithi144.stdout:1/483: write d5/d4a/d44/d80/f76 [840630,40445] 0 2024-09-10T11:29:35.954 INFO:tasks.workunit.client.0.smithi144.stdout:1/484: chown d5/d4a 20783 1 2024-09-10T11:29:35.954 INFO:tasks.workunit.client.0.smithi144.stdout:7/338: sync 2024-09-10T11:29:35.954 INFO:tasks.workunit.client.0.smithi144.stdout:9/404: sync 2024-09-10T11:29:35.955 INFO:tasks.workunit.client.1.smithi178.stdout:5/738: dread d9/dc/dd/d3f/d5a/dbc/fc9 [0,4194304] 0 2024-09-10T11:29:35.956 INFO:tasks.workunit.client.0.smithi144.stdout:0/261: mkdir da/d3a/d4b/d4e 0 2024-09-10T11:29:35.956 INFO:tasks.workunit.client.0.smithi144.stdout:7/339: chown d5/dc/d12/d2f/f4a 5 1 2024-09-10T11:29:35.957 INFO:tasks.workunit.client.0.smithi144.stdout:9/405: truncate d11/d12/d1b/f71 157915 0 2024-09-10T11:29:35.958 INFO:tasks.workunit.client.1.smithi178.stdout:5/739: write d9/dc/d17/d1d/f8c [656487,126548] 0 2024-09-10T11:29:35.958 INFO:tasks.workunit.client.1.smithi178.stdout:8/827: mkdir d5/d30/d33/d3a/d110 0 2024-09-10T11:29:35.961 INFO:tasks.workunit.client.0.smithi144.stdout:0/262: mkdir da/d3a/d4f 0 2024-09-10T11:29:35.962 INFO:tasks.workunit.client.1.smithi178.stdout:6/728: dwrite d3/d7/d8/d15/fa5 [0,4194304] 0 2024-09-10T11:29:35.962 INFO:tasks.workunit.client.1.smithi178.stdout:3/696: rename db/d26/d40/d68/db6 to db/d20/d3e/d6b/dc0/ddf 0 2024-09-10T11:29:35.962 INFO:tasks.workunit.client.0.smithi144.stdout:3/438: link d4/d28/f87 d4/d28/d67/d80/d1d/f97 0 2024-09-10T11:29:35.963 INFO:tasks.workunit.client.1.smithi178.stdout:6/729: readlink d3/d7/d8/d15/l31 0 2024-09-10T11:29:35.967 INFO:tasks.workunit.client.0.smithi144.stdout:6/330: dread d9/dd/f1a [0,4194304] 0 2024-09-10T11:29:35.968 INFO:tasks.workunit.client.1.smithi178.stdout:3/697: fdatasync db/d26/f80 0 2024-09-10T11:29:35.969 INFO:tasks.workunit.client.1.smithi178.stdout:3/698: chown db/d20/d3d/d56/faf 2751 1 2024-09-10T11:29:35.970 INFO:tasks.workunit.client.0.smithi144.stdout:0/263: rename da/f15 to da/d3a/f50 0 2024-09-10T11:29:35.971 INFO:tasks.workunit.client.1.smithi178.stdout:5/740: creat d9/dc/ff4 x:0 0 0 2024-09-10T11:29:35.974 INFO:tasks.workunit.client.1.smithi178.stdout:8/828: dread d5/db/d1e/f62 [0,4194304] 0 2024-09-10T11:29:35.975 INFO:tasks.workunit.client.1.smithi178.stdout:6/730: link d3/d74/d91/daa/dc2/dcf/f109 d3/d74/dc9/f10a 0 2024-09-10T11:29:35.977 INFO:tasks.workunit.client.1.smithi178.stdout:6/731: readlink d3/d74/d91/daa/dc2/dcf/lf9 0 2024-09-10T11:29:35.977 INFO:tasks.workunit.client.1.smithi178.stdout:8/829: truncate d5/db/d1e/f5e 677445 0 2024-09-10T11:29:35.982 INFO:tasks.workunit.client.1.smithi178.stdout:8/830: creat d5/db/dda/dfa/f111 x:0 0 0 2024-09-10T11:29:35.982 INFO:tasks.workunit.client.1.smithi178.stdout:6/732: dread d3/d24/f81 [0,4194304] 0 2024-09-10T11:29:35.983 INFO:tasks.workunit.client.1.smithi178.stdout:8/831: chown d5/d1d/d22/d76/l7a 6111902 1 2024-09-10T11:29:35.983 INFO:tasks.workunit.client.1.smithi178.stdout:6/733: truncate f1 591013 0 2024-09-10T11:29:35.984 INFO:tasks.workunit.client.1.smithi178.stdout:8/832: mknod d5/db/d10/d53/c112 0 2024-09-10T11:29:35.987 INFO:tasks.workunit.client.1.smithi178.stdout:5/741: write d9/d19/f66 [1518135,97056] 0 2024-09-10T11:29:35.989 INFO:tasks.workunit.client.1.smithi178.stdout:8/833: rename d5/db/d10/d53/d82/db2/cf8 to d5/c113 0 2024-09-10T11:29:35.993 INFO:tasks.workunit.client.1.smithi178.stdout:6/734: dwrite d3/d7/d8/d15/d1c/f102 [0,4194304] 0 2024-09-10T11:29:35.993 INFO:tasks.workunit.client.1.smithi178.stdout:5/742: readlink d9/dc/dd/d16/l50 0 2024-09-10T11:29:35.993 INFO:tasks.workunit.client.1.smithi178.stdout:8/834: truncate d5/d30/f10d 194388 0 2024-09-10T11:29:35.995 INFO:tasks.workunit.client.1.smithi178.stdout:5/743: unlink d9/dc/dd/d16/l50 0 2024-09-10T11:29:35.998 INFO:tasks.workunit.client.1.smithi178.stdout:8/835: mknod d5/d1d/d22/d10f/c114 0 2024-09-10T11:29:35.998 INFO:tasks.workunit.client.1.smithi178.stdout:3/699: dread db/f1a [0,4194304] 0 2024-09-10T11:29:36.000 INFO:tasks.workunit.client.1.smithi178.stdout:5/744: creat d9/dc/dd/d3f/ff5 x:0 0 0 2024-09-10T11:29:36.001 INFO:tasks.workunit.client.1.smithi178.stdout:3/700: fsync db/d26/d40/f7a 0 2024-09-10T11:29:36.002 INFO:tasks.workunit.client.1.smithi178.stdout:8/836: creat d5/db/d10/d20/dc0/d109/f115 x:0 0 0 2024-09-10T11:29:36.011 INFO:tasks.workunit.client.1.smithi178.stdout:6/735: dwrite d3/d7/d35/f4c [0,4194304] 0 2024-09-10T11:29:36.014 INFO:tasks.workunit.client.1.smithi178.stdout:8/837: write d5/db/d1e/f45 [1130770,127521] 0 2024-09-10T11:29:36.023 INFO:tasks.workunit.client.1.smithi178.stdout:5/745: dwrite d9/dc/dd/d37/d42/d62/daf/fbf [4194304,4194304] 0 2024-09-10T11:29:36.027 INFO:tasks.workunit.client.1.smithi178.stdout:3/701: dwrite db/f27 [4194304,4194304] 0 2024-09-10T11:29:36.030 INFO:tasks.workunit.client.1.smithi178.stdout:6/736: dwrite d3/d7/d8/d19/f26 [0,4194304] 0 2024-09-10T11:29:36.030 INFO:tasks.workunit.client.0.smithi144.stdout:1/485: dread d5/f23 [0,4194304] 0 2024-09-10T11:29:36.030 INFO:tasks.workunit.client.0.smithi144.stdout:0/264: mkdir da/d3a/d51 0 2024-09-10T11:29:36.031 INFO:tasks.workunit.client.0.smithi144.stdout:7/340: truncate d5/f38 1500740 0 2024-09-10T11:29:36.032 INFO:tasks.workunit.client.0.smithi144.stdout:0/265: fdatasync da/d3a/f3e 0 2024-09-10T11:29:36.033 INFO:tasks.workunit.client.0.smithi144.stdout:6/331: dwrite d9/dd/d3e/f76 [0,4194304] 0 2024-09-10T11:29:36.033 INFO:tasks.workunit.client.1.smithi178.stdout:8/838: dread d5/db/d1e/f62 [0,4194304] 0 2024-09-10T11:29:36.036 INFO:tasks.workunit.client.1.smithi178.stdout:6/737: read d3/d7/d8/d72/fd5 [506048,34342] 0 2024-09-10T11:29:36.037 INFO:tasks.workunit.client.0.smithi144.stdout:9/406: dread d11/d12/d1b/d41/d8f/f87 [0,4194304] 0 2024-09-10T11:29:36.044 INFO:tasks.workunit.client.1.smithi178.stdout:5/746: mkdir d9/dc/df6 0 2024-09-10T11:29:36.049 INFO:tasks.workunit.client.1.smithi178.stdout:8/839: truncate d5/d30/d33/d8c/fcd 322946 0 2024-09-10T11:29:36.049 INFO:tasks.workunit.client.1.smithi178.stdout:6/738: link d3/d7/d8/d15/d1c/d2a/d18/f7e d3/d74/d91/daa/f10b 0 2024-09-10T11:29:36.054 INFO:tasks.workunit.client.1.smithi178.stdout:5/747: getdents d9/dc/dd/d37/dbd/dda/db8 0 2024-09-10T11:29:36.058 INFO:tasks.workunit.client.1.smithi178.stdout:5/748: rmdir d9/dc/df6 0 2024-09-10T11:29:36.061 INFO:tasks.workunit.client.1.smithi178.stdout:6/739: dwrite d3/d74/d91/daa/dc2/d57/d78/d7c/fd2 [0,4194304] 0 2024-09-10T11:29:36.065 INFO:tasks.workunit.client.1.smithi178.stdout:5/749: dread - d9/dc/dd/d37/dbd/dda/db8/fed zero size 2024-09-10T11:29:36.067 INFO:tasks.workunit.client.1.smithi178.stdout:3/702: dread db/f45 [0,4194304] 0 2024-09-10T11:29:36.067 INFO:tasks.workunit.client.1.smithi178.stdout:8/840: dread d5/d1d/f60 [0,4194304] 0 2024-09-10T11:29:36.067 INFO:tasks.workunit.client.1.smithi178.stdout:6/740: write d3/d74/dc9/d4b/f7b [1116451,35402] 0 2024-09-10T11:29:36.070 INFO:tasks.workunit.client.1.smithi178.stdout:6/741: chown d3/d7/d8/d15/d1c/d2a/d18/d8e/d90/cfa 21986262 1 2024-09-10T11:29:36.073 INFO:tasks.workunit.client.1.smithi178.stdout:3/703: creat db/d20/d3d/d56/fe0 x:0 0 0 2024-09-10T11:29:36.074 INFO:tasks.workunit.client.1.smithi178.stdout:8/841: symlink d5/db/d10/d53/d82/db2/l116 0 2024-09-10T11:29:36.075 INFO:tasks.workunit.client.1.smithi178.stdout:6/742: creat d3/d6e/d87/f10c x:0 0 0 2024-09-10T11:29:36.076 INFO:tasks.workunit.client.1.smithi178.stdout:5/750: truncate d9/dc/dd/d3f/d5a/f9c 3498777 0 2024-09-10T11:29:36.077 INFO:tasks.workunit.client.1.smithi178.stdout:8/842: mkdir d5/db/d10/d35/d117 0 2024-09-10T11:29:36.078 INFO:tasks.workunit.client.1.smithi178.stdout:6/743: symlink d3/d74/d91/daa/l10d 0 2024-09-10T11:29:36.080 INFO:tasks.workunit.client.1.smithi178.stdout:8/843: chown d5/d1d/d22/cf7 90 1 2024-09-10T11:29:36.081 INFO:tasks.workunit.client.1.smithi178.stdout:3/704: creat db/d35/d3a/d42/fe1 x:0 0 0 2024-09-10T11:29:36.083 INFO:tasks.workunit.client.1.smithi178.stdout:3/705: truncate db/d20/d3d/d63/fbb 1039761 0 2024-09-10T11:29:36.086 INFO:tasks.workunit.client.1.smithi178.stdout:3/706: unlink db/d20/c55 0 2024-09-10T11:29:36.087 INFO:tasks.workunit.client.1.smithi178.stdout:6/744: getdents d3/d74/dc9 0 2024-09-10T11:29:36.092 INFO:tasks.workunit.client.1.smithi178.stdout:3/707: fdatasync db/d35/d58/f6e 0 2024-09-10T11:29:36.094 INFO:tasks.workunit.client.1.smithi178.stdout:5/751: truncate d9/f86 656433 0 2024-09-10T11:29:36.095 INFO:tasks.workunit.client.1.smithi178.stdout:3/708: symlink db/d35/d3a/d42/d46/le2 0 2024-09-10T11:29:36.096 INFO:tasks.workunit.client.1.smithi178.stdout:6/745: getdents d3/d7/d8/d72 0 2024-09-10T11:29:36.098 INFO:tasks.workunit.client.1.smithi178.stdout:3/709: symlink db/d20/d3e/d6b/dc0/ddf/dcb/le3 0 2024-09-10T11:29:36.099 INFO:tasks.workunit.client.1.smithi178.stdout:6/746: symlink d3/d74/dc9/d4b/l10e 0 2024-09-10T11:29:36.104 INFO:tasks.workunit.client.1.smithi178.stdout:3/710: dwrite db/d20/f3c [0,4194304] 0 2024-09-10T11:29:36.106 INFO:tasks.workunit.client.1.smithi178.stdout:3/711: dread - db/d20/d3e/d6b/fd4 zero size 2024-09-10T11:29:36.107 INFO:tasks.workunit.client.1.smithi178.stdout:5/752: dwrite d9/dc/dd/d37/dbd/dda/fbb [0,4194304] 0 2024-09-10T11:29:36.108 INFO:tasks.workunit.client.1.smithi178.stdout:3/712: chown db/d20/d3d/d56/d64/f8f 65271 1 2024-09-10T11:29:36.115 INFO:tasks.workunit.client.1.smithi178.stdout:6/747: dread d3/d74/dc9/f93 [0,4194304] 0 2024-09-10T11:29:36.117 INFO:tasks.workunit.client.1.smithi178.stdout:6/748: dread - d3/d74/dc9/f10a zero size 2024-09-10T11:29:36.118 INFO:tasks.workunit.client.1.smithi178.stdout:3/713: dwrite db/d20/f95 [0,4194304] 0 2024-09-10T11:29:36.122 INFO:tasks.workunit.client.1.smithi178.stdout:6/749: creat d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/f10f x:0 0 0 2024-09-10T11:29:36.123 INFO:tasks.workunit.client.1.smithi178.stdout:3/714: creat db/d35/d3a/d9f/fe4 x:0 0 0 2024-09-10T11:29:36.124 INFO:tasks.workunit.client.1.smithi178.stdout:5/753: link d9/dc/dd/d37/d3e/l85 d9/dc/d17/db0/lf7 0 2024-09-10T11:29:36.125 INFO:tasks.workunit.client.1.smithi178.stdout:5/754: readlink d9/dc/dd/l83 0 2024-09-10T11:29:36.125 INFO:tasks.workunit.client.1.smithi178.stdout:6/750: creat d3/d7/d8/d72/f110 x:0 0 0 2024-09-10T11:29:36.126 INFO:tasks.workunit.client.1.smithi178.stdout:3/715: mkdir db/d20/d3e/d6b/dc0/de5 0 2024-09-10T11:29:36.127 INFO:tasks.workunit.client.1.smithi178.stdout:5/755: mknod d9/dc/d17/db0/cf8 0 2024-09-10T11:29:36.128 INFO:tasks.workunit.client.1.smithi178.stdout:3/716: write db/d20/d3d/d63/d6a/f84 [4155876,31785] 0 2024-09-10T11:29:36.129 INFO:tasks.workunit.client.0.smithi144.stdout:6/332: mknod d9/dd/d73/c78 0 2024-09-10T11:29:36.130 INFO:tasks.workunit.client.0.smithi144.stdout:0/266: fdatasync da/f30 0 2024-09-10T11:29:36.131 INFO:tasks.workunit.client.0.smithi144.stdout:1/486: dwrite d5/d4a/d44/d80/f76 [0,4194304] 0 2024-09-10T11:29:36.131 INFO:tasks.workunit.client.0.smithi144.stdout:9/407: dread d11/d36/f4e [0,4194304] 0 2024-09-10T11:29:36.132 INFO:tasks.workunit.client.1.smithi178.stdout:3/717: mknod db/d20/d3e/ce6 0 2024-09-10T11:29:36.133 INFO:tasks.workunit.client.0.smithi144.stdout:0/267: write da/f36 [872728,18856] 0 2024-09-10T11:29:36.135 INFO:tasks.workunit.client.0.smithi144.stdout:9/408: truncate d11/d12/f26 4357667 0 2024-09-10T11:29:36.135 INFO:tasks.workunit.client.0.smithi144.stdout:0/268: chown da 5 1 2024-09-10T11:29:36.135 INFO:tasks.workunit.client.0.smithi144.stdout:6/333: truncate d9/dd/d11/d1b/d62/d68/f75 436796 0 2024-09-10T11:29:36.136 INFO:tasks.workunit.client.0.smithi144.stdout:1/487: read d5/f6 [3085621,4915] 0 2024-09-10T11:29:36.136 INFO:tasks.workunit.client.1.smithi178.stdout:5/756: rename f4 to d9/dc/dd/ff9 0 2024-09-10T11:29:36.137 INFO:tasks.workunit.client.0.smithi144.stdout:1/488: fdatasync d5/df/f33 0 2024-09-10T11:29:36.138 INFO:tasks.workunit.client.0.smithi144.stdout:0/269: mknod da/d3a/c52 0 2024-09-10T11:29:36.139 INFO:tasks.workunit.client.1.smithi178.stdout:3/718: mknod db/d35/d58/ce7 0 2024-09-10T11:29:36.140 INFO:tasks.workunit.client.0.smithi144.stdout:9/409: mknod d11/d36/d2d/c91 0 2024-09-10T11:29:36.144 INFO:tasks.workunit.client.1.smithi178.stdout:3/719: mknod db/d20/d3e/d6b/dc0/ddf/ce8 0 2024-09-10T11:29:36.145 INFO:tasks.workunit.client.0.smithi144.stdout:9/410: creat d11/d12/d1b/d41/f92 x:0 0 0 2024-09-10T11:29:36.145 INFO:tasks.workunit.client.0.smithi144.stdout:1/489: dwrite d5/f12 [0,4194304] 0 2024-09-10T11:29:36.146 INFO:tasks.workunit.client.0.smithi144.stdout:0/270: symlink da/d3a/d4f/l53 0 2024-09-10T11:29:36.148 INFO:tasks.workunit.client.1.smithi178.stdout:3/720: rmdir db/d20/d3d/d56/d64 39 2024-09-10T11:29:36.148 INFO:tasks.workunit.client.0.smithi144.stdout:0/271: read - da/d48/f4a zero size 2024-09-10T11:29:36.148 INFO:tasks.workunit.client.0.smithi144.stdout:7/341: dread d5/f38 [0,4194304] 0 2024-09-10T11:29:36.149 INFO:tasks.workunit.client.1.smithi178.stdout:3/721: write db/d20/d3d/d63/f6f [1265244,12152] 0 2024-09-10T11:29:36.149 INFO:tasks.workunit.client.0.smithi144.stdout:1/490: creat d5/d4a/d63/f9f x:0 0 0 2024-09-10T11:29:36.152 INFO:tasks.workunit.client.0.smithi144.stdout:6/334: truncate d9/dd/d3e/f40 378383 0 2024-09-10T11:29:36.154 INFO:tasks.workunit.client.0.smithi144.stdout:1/491: getdents d5/d4a/d63/d84/d95 0 2024-09-10T11:29:36.154 INFO:tasks.workunit.client.0.smithi144.stdout:1/492: dread - d5/d4a/d63/d84/f9e zero size 2024-09-10T11:29:36.155 INFO:tasks.workunit.client.1.smithi178.stdout:3/722: creat db/d9b/fe9 x:0 0 0 2024-09-10T11:29:36.155 INFO:tasks.workunit.client.0.smithi144.stdout:6/335: symlink d9/dd/d11/d1b/d2f/l79 0 2024-09-10T11:29:36.157 INFO:tasks.workunit.client.1.smithi178.stdout:6/751: dwrite d3/d74/d91/daa/f10b [0,4194304] 0 2024-09-10T11:29:36.158 INFO:tasks.workunit.client.0.smithi144.stdout:6/336: mknod d9/dd/d3e/d47/d13/d4a/c7a 0 2024-09-10T11:29:36.160 INFO:tasks.workunit.client.0.smithi144.stdout:6/337: write d9/dd/f33 [1109051,119902] 0 2024-09-10T11:29:36.160 INFO:tasks.workunit.client.1.smithi178.stdout:5/757: dread d9/dc/dd/d37/d42/d55/d6d/dcc/fd2 [0,4194304] 0 2024-09-10T11:29:36.161 INFO:tasks.workunit.client.0.smithi144.stdout:6/338: read - d9/dd/d3e/f4d zero size 2024-09-10T11:29:36.161 INFO:tasks.workunit.client.1.smithi178.stdout:3/723: truncate db/d35/d3a/d9f/fb8 366579 0 2024-09-10T11:29:36.163 INFO:tasks.workunit.client.1.smithi178.stdout:3/724: fsync db/f45 0 2024-09-10T11:29:36.164 INFO:tasks.workunit.client.0.smithi144.stdout:6/339: read - d9/dd/d3a/f6f zero size 2024-09-10T11:29:36.165 INFO:tasks.workunit.client.1.smithi178.stdout:5/758: mknod d9/de4/cfa 0 2024-09-10T11:29:36.165 INFO:tasks.workunit.client.1.smithi178.stdout:6/752: link d3/d74/dc9/c6d d3/d7/d8/d15/d48/c111 0 2024-09-10T11:29:36.167 INFO:tasks.workunit.client.0.smithi144.stdout:6/340: symlink d9/d38/d6c/l7b 0 2024-09-10T11:29:36.168 INFO:tasks.workunit.client.0.smithi144.stdout:9/411: dwrite d11/d12/d1b/d41/d8f/f82 [0,4194304] 0 2024-09-10T11:29:36.168 INFO:tasks.workunit.client.1.smithi178.stdout:3/725: creat db/d35/d3a/d42/d46/fea x:0 0 0 2024-09-10T11:29:36.170 INFO:tasks.workunit.client.0.smithi144.stdout:1/493: dread d5/d4a/d44/d80/f66 [0,4194304] 0 2024-09-10T11:29:36.170 INFO:tasks.workunit.client.0.smithi144.stdout:6/341: symlink d9/dd/d3e/l7c 0 2024-09-10T11:29:36.172 INFO:tasks.workunit.client.1.smithi178.stdout:5/759: mknod d9/dc/dd/d3f/d5a/cfb 0 2024-09-10T11:29:36.173 INFO:tasks.workunit.client.1.smithi178.stdout:3/726: mknod db/d20/d3d/d63/da2/d98/ceb 0 2024-09-10T11:29:36.174 INFO:tasks.workunit.client.0.smithi144.stdout:6/342: creat d9/d1f/d2c/d4f/f7d x:0 0 0 2024-09-10T11:29:36.175 INFO:tasks.workunit.client.1.smithi178.stdout:5/760: mknod d9/dc/dd/d3f/d5a/cfc 0 2024-09-10T11:29:36.176 INFO:tasks.workunit.client.1.smithi178.stdout:3/727: symlink db/d20/d70/lec 0 2024-09-10T11:29:36.177 INFO:tasks.workunit.client.0.smithi144.stdout:1/494: mkdir d5/dd/da0 0 2024-09-10T11:29:36.177 INFO:tasks.workunit.client.0.smithi144.stdout:9/412: dwrite d11/d36/f4e [0,4194304] 0 2024-09-10T11:29:36.179 INFO:tasks.workunit.client.1.smithi178.stdout:5/761: dread d9/f5e [0,4194304] 0 2024-09-10T11:29:36.180 INFO:tasks.workunit.client.0.smithi144.stdout:6/343: mknod d9/d1f/d2c/d4f/c7e 0 2024-09-10T11:29:36.181 INFO:tasks.workunit.client.1.smithi178.stdout:3/728: mknod db/d26/d40/dcc/ced 0 2024-09-10T11:29:36.182 INFO:tasks.workunit.client.0.smithi144.stdout:1/495: mkdir d5/dd/da1 0 2024-09-10T11:29:36.184 INFO:tasks.workunit.client.1.smithi178.stdout:5/762: getdents d9/dc/dd/d37/d42/d55 0 2024-09-10T11:29:36.185 INFO:tasks.workunit.client.1.smithi178.stdout:3/729: write db/d20/d3d/d63/f6f [2375925,11384] 0 2024-09-10T11:29:36.186 INFO:tasks.workunit.client.0.smithi144.stdout:1/496: mkdir d5/d4a/d44/da2 0 2024-09-10T11:29:36.188 INFO:tasks.workunit.client.0.smithi144.stdout:5/400: sync 2024-09-10T11:29:36.189 INFO:tasks.workunit.client.1.smithi178.stdout:6/753: dwrite d3/d6e/fba [4194304,4194304] 0 2024-09-10T11:29:36.189 INFO:tasks.workunit.client.1.smithi178.stdout:3/730: mkdir db/d20/d3e/d6b/dee 0 2024-09-10T11:29:36.190 INFO:tasks.workunit.client.0.smithi144.stdout:6/344: getdents d9/d1f/d2c 0 2024-09-10T11:29:36.190 INFO:tasks.workunit.client.0.smithi144.stdout:2/279: sync 2024-09-10T11:29:36.190 INFO:tasks.workunit.client.0.smithi144.stdout:6/345: read d9/d1f/f26 [349908,71401] 0 2024-09-10T11:29:36.191 INFO:tasks.workunit.client.0.smithi144.stdout:1/497: unlink d5/d2a/d65/d73/c47 0 2024-09-10T11:29:36.193 INFO:tasks.workunit.client.1.smithi178.stdout:6/754: creat d3/d74/dc9/da6/f112 x:0 0 0 2024-09-10T11:29:36.193 INFO:tasks.workunit.client.1.smithi178.stdout:3/731: mknod db/d20/cef 0 2024-09-10T11:29:36.194 INFO:tasks.workunit.client.0.smithi144.stdout:5/401: mknod d4/d52/c81 0 2024-09-10T11:29:36.194 INFO:tasks.workunit.client.0.smithi144.stdout:9/413: fsync d11/d36/f4e 0 2024-09-10T11:29:36.196 INFO:tasks.workunit.client.1.smithi178.stdout:5/763: rename d9/dc/l72 to d9/dc/d17/lfd 0 2024-09-10T11:29:36.197 INFO:tasks.workunit.client.0.smithi144.stdout:6/346: rename d9/dd/d11/d1b/d62/d68 to d9/dd/d11/d1b/d2f/d7f 0 2024-09-10T11:29:36.199 INFO:tasks.workunit.client.0.smithi144.stdout:6/347: dread - d9/d1f/f77 zero size 2024-09-10T11:29:36.199 INFO:tasks.workunit.client.0.smithi144.stdout:2/280: creat d7/d42/f5d x:0 0 0 2024-09-10T11:29:36.200 INFO:tasks.workunit.client.1.smithi178.stdout:6/755: creat d3/d7/d8/d56/da3/f113 x:0 0 0 2024-09-10T11:29:36.201 INFO:tasks.workunit.client.0.smithi144.stdout:1/498: mknod d5/d2a/ca3 0 2024-09-10T11:29:36.202 INFO:tasks.workunit.client.0.smithi144.stdout:9/414: creat d11/d12/d1b/d41/d8f/f93 x:0 0 0 2024-09-10T11:29:36.203 INFO:tasks.workunit.client.1.smithi178.stdout:6/756: creat d3/d7/d35/f114 x:0 0 0 2024-09-10T11:29:36.203 INFO:tasks.workunit.client.0.smithi144.stdout:2/281: mkdir d7/de/d3d/d5e 0 2024-09-10T11:29:36.204 INFO:tasks.workunit.client.0.smithi144.stdout:6/348: fdatasync d9/dd/d3e/d47/d13/f20 0 2024-09-10T11:29:36.204 INFO:tasks.workunit.client.0.smithi144.stdout:5/402: dwrite d4/f55 [0,4194304] 0 2024-09-10T11:29:36.205 INFO:tasks.workunit.client.1.smithi178.stdout:5/764: creat d9/dc/dd/d37/d42/d55/d6d/dcc/de2/ffe x:0 0 0 2024-09-10T11:29:36.208 INFO:tasks.workunit.client.0.smithi144.stdout:9/415: rename d11/d12/d1b/d41/c56 to d11/d12/d1b/d41/c94 0 2024-09-10T11:29:36.209 INFO:tasks.workunit.client.1.smithi178.stdout:6/757: dwrite d3/d7/d8/d19/f26 [0,4194304] 0 2024-09-10T11:29:36.209 INFO:tasks.workunit.client.1.smithi178.stdout:5/765: creat d9/dc/dd/d3f/dd6/fff x:0 0 0 2024-09-10T11:29:36.210 INFO:tasks.workunit.client.0.smithi144.stdout:1/499: creat d5/d2a/fa4 x:0 0 0 2024-09-10T11:29:36.211 INFO:tasks.workunit.client.0.smithi144.stdout:5/403: chown d4/d20/d24/d2d/d4b/f4e 53947 1 2024-09-10T11:29:36.213 INFO:tasks.workunit.client.0.smithi144.stdout:9/416: readlink d11/d36/d2d/d38/l39 0 2024-09-10T11:29:36.215 INFO:tasks.workunit.client.0.smithi144.stdout:2/282: dwrite d7/da/f44 [0,4194304] 0 2024-09-10T11:29:36.215 INFO:tasks.workunit.client.1.smithi178.stdout:5/766: link d9/dc/dd/d16/fb3 d9/dc/dd/d37/d42/d55/d6d/dcc/dd8/f100 0 2024-09-10T11:29:36.216 INFO:tasks.workunit.client.0.smithi144.stdout:5/404: mkdir d4/d20/d22/d82 0 2024-09-10T11:29:36.217 INFO:tasks.workunit.client.1.smithi178.stdout:5/767: truncate d9/dc/dd/d3f/fe9 207964 0 2024-09-10T11:29:36.218 INFO:tasks.workunit.client.0.smithi144.stdout:6/349: dread d9/dd/f33 [0,4194304] 0 2024-09-10T11:29:36.219 INFO:tasks.workunit.client.1.smithi178.stdout:5/768: symlink d9/dc/dd/d3f/l101 0 2024-09-10T11:29:36.219 INFO:tasks.workunit.client.0.smithi144.stdout:9/417: creat d11/d12/d1b/f95 x:0 0 0 2024-09-10T11:29:36.221 INFO:tasks.workunit.client.0.smithi144.stdout:5/405: creat d4/d20/d24/f83 x:0 0 0 2024-09-10T11:29:36.221 INFO:tasks.workunit.client.0.smithi144.stdout:6/350: symlink d9/l80 0 2024-09-10T11:29:36.222 INFO:tasks.workunit.client.0.smithi144.stdout:9/418: write d11/d12/d1b/d41/d8f/f93 [312033,7639] 0 2024-09-10T11:29:36.222 INFO:tasks.workunit.client.0.smithi144.stdout:5/406: chown d4/d5/d18/d67/d44 17641 1 2024-09-10T11:29:36.222 INFO:tasks.workunit.client.1.smithi178.stdout:5/769: unlink d9/dc/c2a 0 2024-09-10T11:29:36.223 INFO:tasks.workunit.client.0.smithi144.stdout:2/283: getdents d7/de/d3d/d43 0 2024-09-10T11:29:36.223 INFO:tasks.workunit.client.1.smithi178.stdout:5/770: fdatasync d9/dc/dd/d37/d42/d55/d6d/fea 0 2024-09-10T11:29:36.224 INFO:tasks.workunit.client.0.smithi144.stdout:5/407: mkdir d4/d20/d24/d6f/d84 0 2024-09-10T11:29:36.225 INFO:tasks.workunit.client.0.smithi144.stdout:6/351: creat d9/f81 x:0 0 0 2024-09-10T11:29:36.225 INFO:tasks.workunit.client.0.smithi144.stdout:2/284: mkdir d7/de/d5f 0 2024-09-10T11:29:36.226 INFO:tasks.workunit.client.0.smithi144.stdout:5/408: dread d4/d52/f69 [0,4194304] 0 2024-09-10T11:29:36.227 INFO:tasks.workunit.client.0.smithi144.stdout:2/285: readlink d7/de/d3d/d43/l56 0 2024-09-10T11:29:36.244 INFO:tasks.workunit.client.1.smithi178.stdout:1/662: sync 2024-09-10T11:29:36.246 INFO:tasks.workunit.client.0.smithi144.stdout:6/352: rename d9/d38/d6c to d9/dd/d11/d1b/d2f/d7f/d82 0 2024-09-10T11:29:36.247 INFO:tasks.workunit.client.0.smithi144.stdout:5/409: creat d4/d20/d22/d82/f85 x:0 0 0 2024-09-10T11:29:36.249 INFO:tasks.workunit.client.0.smithi144.stdout:2/286: link d7/f8 d7/d29/f60 0 2024-09-10T11:29:36.249 INFO:tasks.workunit.client.1.smithi178.stdout:1/663: dwrite d3/d9/d98/dac/d3a/d50/d6c/d78/db8/fbf [0,4194304] 0 2024-09-10T11:29:36.252 INFO:tasks.workunit.client.0.smithi144.stdout:5/410: dwrite d4/d20/d24/f63 [0,4194304] 0 2024-09-10T11:29:36.252 INFO:tasks.workunit.client.1.smithi178.stdout:1/664: mknod d3/d9/d98/dac/d3a/d9f/cf5 0 2024-09-10T11:29:36.263 INFO:tasks.workunit.client.1.smithi178.stdout:1/665: dwrite d3/d9/d21/d4b/f17 [0,4194304] 0 2024-09-10T11:29:36.264 INFO:tasks.workunit.client.0.smithi144.stdout:5/411: dwrite d4/d5/d32/f5f [0,4194304] 0 2024-09-10T11:29:36.266 INFO:tasks.workunit.client.1.smithi178.stdout:1/666: write d3/d9/fa [31080,34696] 0 2024-09-10T11:29:36.270 INFO:tasks.workunit.client.0.smithi144.stdout:5/412: dread d4/d5/f23 [0,4194304] 0 2024-09-10T11:29:36.274 INFO:tasks.workunit.client.0.smithi144.stdout:5/413: unlink d4/d20/d24/d2d/c3c 0 2024-09-10T11:29:36.278 INFO:tasks.workunit.client.0.smithi144.stdout:5/414: mknod d4/d20/d24/c86 0 2024-09-10T11:29:36.279 INFO:tasks.workunit.client.1.smithi178.stdout:6/758: write d3/d74/dc9/d4b/f52 [533504,29415] 0 2024-09-10T11:29:36.282 INFO:tasks.workunit.client.1.smithi178.stdout:3/732: dwrite db/ff [0,4194304] 0 2024-09-10T11:29:36.282 INFO:tasks.workunit.client.1.smithi178.stdout:5/771: truncate d9/dc/dd/d3f/d5a/dbc/fca 1479973 0 2024-09-10T11:29:36.284 INFO:tasks.workunit.client.0.smithi144.stdout:5/415: dwrite d4/f6b [0,4194304] 0 2024-09-10T11:29:36.302 INFO:tasks.workunit.client.0.smithi144.stdout:9/419: dread d11/d12/d1b/d41/d8f/f93 [0,4194304] 0 2024-09-10T11:29:36.303 INFO:tasks.workunit.client.0.smithi144.stdout:5/416: dwrite d4/d5/d18/d38/d6d/f70 [0,4194304] 0 2024-09-10T11:29:36.305 INFO:tasks.workunit.client.0.smithi144.stdout:5/417: stat d4/d20/d24 0 2024-09-10T11:29:36.307 INFO:tasks.workunit.client.0.smithi144.stdout:9/420: symlink d11/d12/d1b/d41/l96 0 2024-09-10T11:29:36.308 INFO:tasks.workunit.client.1.smithi178.stdout:3/733: dread db/d20/d3d/d63/d6a/f84 [0,4194304] 0 2024-09-10T11:29:36.308 INFO:tasks.workunit.client.0.smithi144.stdout:1/500: dwrite d5/dd/f1d [0,4194304] 0 2024-09-10T11:29:36.312 INFO:tasks.workunit.client.1.smithi178.stdout:5/772: dwrite d9/dc/dd/d16/f70 [0,4194304] 0 2024-09-10T11:29:36.314 INFO:tasks.workunit.client.1.smithi178.stdout:5/773: write d9/fbe [1197964,93521] 0 2024-09-10T11:29:36.316 INFO:tasks.workunit.client.0.smithi144.stdout:9/421: unlink d11/d12/d1b/f6d 0 2024-09-10T11:29:36.319 INFO:tasks.workunit.client.1.smithi178.stdout:3/734: dwrite db/d20/f28 [0,4194304] 0 2024-09-10T11:29:36.322 INFO:tasks.workunit.client.0.smithi144.stdout:2/287: write d7/f15 [3463047,119314] 0 2024-09-10T11:29:36.324 INFO:tasks.workunit.client.1.smithi178.stdout:5/774: dwrite d9/dc/ff4 [0,4194304] 0 2024-09-10T11:29:36.325 INFO:tasks.workunit.client.0.smithi144.stdout:1/501: symlink d5/d4a/d63/d84/d95/la5 0 2024-09-10T11:29:36.328 INFO:tasks.workunit.client.0.smithi144.stdout:9/422: rename d11/d12/d1b/d22/l3b to d11/d12/d1b/d22/d53/l97 0 2024-09-10T11:29:36.328 INFO:tasks.workunit.client.1.smithi178.stdout:5/775: creat d9/dc/dd/d3f/dd6/f102 x:0 0 0 2024-09-10T11:29:36.329 INFO:tasks.workunit.client.0.smithi144.stdout:5/418: getdents d4/d20/d24/d6f 0 2024-09-10T11:29:36.329 INFO:tasks.workunit.client.0.smithi144.stdout:5/419: fsync d4/d20/d24/f83 0 2024-09-10T11:29:36.331 INFO:tasks.workunit.client.0.smithi144.stdout:2/288: mkdir d7/da/d1e/d21/d54/d61 0 2024-09-10T11:29:36.333 INFO:tasks.workunit.client.0.smithi144.stdout:9/423: symlink d11/d36/d2d/d38/l98 0 2024-09-10T11:29:36.333 INFO:tasks.workunit.client.1.smithi178.stdout:3/735: write db/d20/f52 [4821791,35341] 0 2024-09-10T11:29:36.334 INFO:tasks.workunit.client.1.smithi178.stdout:3/736: readlink l6 0 2024-09-10T11:29:36.334 INFO:tasks.workunit.client.1.smithi178.stdout:5/776: dread d9/dc/dd/d37/d42/d55/d6d/dcc/fd2 [4194304,4194304] 0 2024-09-10T11:29:36.335 INFO:tasks.workunit.client.0.smithi144.stdout:5/420: mkdir d4/d20/d87 0 2024-09-10T11:29:36.335 INFO:tasks.workunit.client.0.smithi144.stdout:5/421: chown d4/d20/d22/d82/f85 1241376 1 2024-09-10T11:29:36.336 INFO:tasks.workunit.client.0.smithi144.stdout:5/422: chown d4/d5/f7d 30473 1 2024-09-10T11:29:36.337 INFO:tasks.workunit.client.1.smithi178.stdout:5/777: dread - d9/dc/dd/d37/d42/d62/daf/df0/fe8 zero size 2024-09-10T11:29:36.337 INFO:tasks.workunit.client.1.smithi178.stdout:3/737: chown db/d35/f5b 1563 1 2024-09-10T11:29:36.337 INFO:tasks.workunit.client.0.smithi144.stdout:5/423: write d4/d5/d18/d67/d59/f5e [542365,23476] 0 2024-09-10T11:29:36.338 INFO:tasks.workunit.client.0.smithi144.stdout:5/424: readlink d4/l1a 0 2024-09-10T11:29:36.338 INFO:tasks.workunit.client.0.smithi144.stdout:1/502: mknod d5/ca6 0 2024-09-10T11:29:36.338 INFO:tasks.workunit.client.1.smithi178.stdout:3/738: read db/ff [3235763,89954] 0 2024-09-10T11:29:36.338 INFO:tasks.workunit.client.0.smithi144.stdout:9/424: rmdir d11/d12/d6f 39 2024-09-10T11:29:36.339 INFO:tasks.workunit.client.0.smithi144.stdout:9/425: truncate d11/d36/f7f 664132 0 2024-09-10T11:29:36.340 INFO:tasks.workunit.client.0.smithi144.stdout:1/503: fdatasync d5/d2a/d65/d73/f24 0 2024-09-10T11:29:36.341 INFO:tasks.workunit.client.0.smithi144.stdout:9/426: rmdir d11/d12/d1b/d22/d53 39 2024-09-10T11:29:36.343 INFO:tasks.workunit.client.1.smithi178.stdout:5/778: rmdir d9/dd5 0 2024-09-10T11:29:36.344 INFO:tasks.workunit.client.0.smithi144.stdout:1/504: unlink d5/d2a/d65/d73/c25 0 2024-09-10T11:29:36.344 INFO:tasks.workunit.client.1.smithi178.stdout:3/739: symlink db/d26/dcd/lf0 0 2024-09-10T11:29:36.345 INFO:tasks.workunit.client.1.smithi178.stdout:3/740: chown db/l12 19024 1 2024-09-10T11:29:36.345 INFO:tasks.workunit.client.0.smithi144.stdout:9/427: creat d11/d12/d1b/d41/d8e/f99 x:0 0 0 2024-09-10T11:29:36.347 INFO:tasks.workunit.client.0.smithi144.stdout:9/428: creat d11/d36/d2d/f9a x:0 0 0 2024-09-10T11:29:36.347 INFO:tasks.workunit.client.1.smithi178.stdout:5/779: mkdir d9/dc/dd/d37/d42/d55/d6d/dcc/dd8/d103 0 2024-09-10T11:29:36.351 INFO:tasks.workunit.client.0.smithi144.stdout:9/429: symlink d11/d12/d6f/l9b 0 2024-09-10T11:29:36.353 INFO:tasks.workunit.client.1.smithi178.stdout:3/741: dwrite db/d20/d3e/d6b/fa7 [0,4194304] 0 2024-09-10T11:29:36.354 INFO:tasks.workunit.client.0.smithi144.stdout:9/430: stat d11/d12/d1b/d2c/l65 0 2024-09-10T11:29:36.355 INFO:tasks.workunit.client.0.smithi144.stdout:2/289: write d7/d42/f48 [4393615,12471] 0 2024-09-10T11:29:36.357 INFO:tasks.workunit.client.1.smithi178.stdout:5/780: dwrite d9/dc/d17/db0/fd1 [0,4194304] 0 2024-09-10T11:29:36.357 INFO:tasks.workunit.client.0.smithi144.stdout:9/431: creat d11/d36/f9c x:0 0 0 2024-09-10T11:29:36.357 INFO:tasks.workunit.client.0.smithi144.stdout:2/290: write d7/d29/f3f [1942099,68931] 0 2024-09-10T11:29:36.358 INFO:tasks.workunit.client.1.smithi178.stdout:3/742: truncate db/d35/d58/f7b 1506395 0 2024-09-10T11:29:36.360 INFO:tasks.workunit.client.0.smithi144.stdout:1/505: dread d5/df/f16 [0,4194304] 0 2024-09-10T11:29:36.362 INFO:tasks.workunit.client.0.smithi144.stdout:9/432: creat d11/d12/d1b/d41/d8f/f9d x:0 0 0 2024-09-10T11:29:36.362 INFO:tasks.workunit.client.0.smithi144.stdout:2/291: symlink d7/de/d3d/d43/l62 0 2024-09-10T11:29:36.362 INFO:tasks.workunit.client.0.smithi144.stdout:1/506: truncate d5/d2a/fa4 555177 0 2024-09-10T11:29:36.363 INFO:tasks.workunit.client.1.smithi178.stdout:5/781: rename d9/dc/dd/ff9 to d9/dc/dd/d37/d42/d9f/f104 0 2024-09-10T11:29:36.364 INFO:tasks.workunit.client.0.smithi144.stdout:2/292: truncate d7/da/f19 4478310 0 2024-09-10T11:29:36.365 INFO:tasks.workunit.client.1.smithi178.stdout:5/782: read d9/f51 [2975840,99251] 0 2024-09-10T11:29:36.365 INFO:tasks.workunit.client.1.smithi178.stdout:3/743: dread db/d20/f3c [0,4194304] 0 2024-09-10T11:29:36.367 INFO:tasks.workunit.client.1.smithi178.stdout:3/744: write db/d35/d3a/d9f/fb8 [1354636,63790] 0 2024-09-10T11:29:36.369 INFO:tasks.workunit.client.1.smithi178.stdout:5/783: mkdir d9/dc/dd/d37/d42/d62/daf/df0/dc3/d105 0 2024-09-10T11:29:36.369 INFO:tasks.workunit.client.1.smithi178.stdout:3/745: fdatasync db/d35/d3a/d42/d46/f4e 0 2024-09-10T11:29:36.370 INFO:tasks.workunit.client.0.smithi144.stdout:9/433: rename d11/d12/c1e to d11/d36/d2d/c9e 0 2024-09-10T11:29:36.371 INFO:tasks.workunit.client.1.smithi178.stdout:3/746: symlink db/d35/d3a/d9f/lf1 0 2024-09-10T11:29:36.371 INFO:tasks.workunit.client.0.smithi144.stdout:2/293: creat d7/de/d3d/d5e/f63 x:0 0 0 2024-09-10T11:29:36.373 INFO:tasks.workunit.client.1.smithi178.stdout:3/747: mkdir db/d35/d58/df2 0 2024-09-10T11:29:36.373 INFO:tasks.workunit.client.0.smithi144.stdout:1/507: rename d5/df/d4b/d62 to d5/df/d4b/d51/da7 0 2024-09-10T11:29:36.376 INFO:tasks.workunit.client.0.smithi144.stdout:2/294: dread d7/da/f12 [0,4194304] 0 2024-09-10T11:29:36.377 INFO:tasks.workunit.client.1.smithi178.stdout:5/784: dwrite d9/dc/d17/dcd/ff1 [0,4194304] 0 2024-09-10T11:29:36.377 INFO:tasks.workunit.client.1.smithi178.stdout:3/748: link db/d36/f50 db/ddd/ff3 0 2024-09-10T11:29:36.379 INFO:tasks.workunit.client.1.smithi178.stdout:3/749: readlink db/d20/d3d/d63/l7f 0 2024-09-10T11:29:36.380 INFO:tasks.workunit.client.0.smithi144.stdout:2/295: mkdir d7/d42/d64 0 2024-09-10T11:29:36.380 INFO:tasks.workunit.client.1.smithi178.stdout:3/750: truncate db/d35/d3a/d42/d46/fea 741794 0 2024-09-10T11:29:36.381 INFO:tasks.workunit.client.1.smithi178.stdout:5/785: read d9/dc/dd/d37/d42/d62/daf/df0/faa [2333478,103840] 0 2024-09-10T11:29:36.381 INFO:tasks.workunit.client.1.smithi178.stdout:5/786: readlink d9/dc/dd/l7c 0 2024-09-10T11:29:36.384 INFO:tasks.workunit.client.0.smithi144.stdout:2/296: mknod d7/da/d5b/c65 0 2024-09-10T11:29:36.384 INFO:tasks.workunit.client.1.smithi178.stdout:5/787: mkdir d9/dc/d17/db0/d106 0 2024-09-10T11:29:36.385 INFO:tasks.workunit.client.0.smithi144.stdout:9/434: truncate d11/d12/d1b/f77 1400630 0 2024-09-10T11:29:36.388 INFO:tasks.workunit.client.0.smithi144.stdout:2/297: rename d7/d42/f5d to d7/da/d1e/f66 0 2024-09-10T11:29:36.388 INFO:tasks.workunit.client.1.smithi178.stdout:5/788: dread d9/dc/ff4 [0,4194304] 0 2024-09-10T11:29:36.391 INFO:tasks.workunit.client.1.smithi178.stdout:5/789: creat d9/dc/dd/d37/d42/d55/f107 x:0 0 0 2024-09-10T11:29:36.392 INFO:tasks.workunit.client.0.smithi144.stdout:1/508: dwrite d5/dd/d64/f9a [0,4194304] 0 2024-09-10T11:29:36.392 INFO:tasks.workunit.client.0.smithi144.stdout:9/435: rename d11/d36/d2d/c43 to d11/d12/d1b/d2c/c9f 0 2024-09-10T11:29:36.393 INFO:tasks.workunit.client.1.smithi178.stdout:5/790: dread - d9/dc/dd/d37/d42/d55/d6d/dcc/de2/ffe zero size 2024-09-10T11:29:36.394 INFO:tasks.workunit.client.0.smithi144.stdout:2/298: mknod d7/de/d3d/d4e/c67 0 2024-09-10T11:29:36.394 INFO:tasks.workunit.client.0.smithi144.stdout:1/509: truncate d5/dd/f48 9706 0 2024-09-10T11:29:36.395 INFO:tasks.workunit.client.0.smithi144.stdout:1/510: fdatasync d5/dd/f8d 0 2024-09-10T11:29:36.400 INFO:tasks.workunit.client.0.smithi144.stdout:1/511: write d5/df/d4b/f8e [4730516,87812] 0 2024-09-10T11:29:36.401 INFO:tasks.workunit.client.0.smithi144.stdout:9/436: dwrite d11/d36/d2d/d38/f76 [0,4194304] 0 2024-09-10T11:29:36.402 INFO:tasks.workunit.client.1.smithi178.stdout:5/791: write d9/dc/dd/d37/d42/d55/d6d/dcc/dd8/f100 [175310,2978] 0 2024-09-10T11:29:36.402 INFO:tasks.workunit.client.1.smithi178.stdout:3/751: dwrite db/d35/f37 [0,4194304] 0 2024-09-10T11:29:36.406 INFO:tasks.workunit.client.0.smithi144.stdout:1/512: mknod d5/dd/ca8 0 2024-09-10T11:29:36.407 INFO:tasks.workunit.client.0.smithi144.stdout:9/437: dread d11/f17 [0,4194304] 0 2024-09-10T11:29:36.408 INFO:tasks.workunit.client.1.smithi178.stdout:3/752: rename db/d20/ca0 to db/d20/d3e/d6b/dc0/ddf/dcb/d81/cf4 0 2024-09-10T11:29:36.410 INFO:tasks.workunit.client.0.smithi144.stdout:1/513: mknod d5/dd/d8f/ca9 0 2024-09-10T11:29:36.411 INFO:tasks.workunit.client.0.smithi144.stdout:2/299: write d7/da/f13 [1344090,62285] 0 2024-09-10T11:29:36.412 INFO:tasks.workunit.client.1.smithi178.stdout:5/792: dwrite d9/dc/dd/d37/dbd/dda/db8/fe0 [0,4194304] 0 2024-09-10T11:29:36.412 INFO:tasks.workunit.client.0.smithi144.stdout:2/300: write d7/da/fd [437418,6365] 0 2024-09-10T11:29:36.413 INFO:tasks.workunit.client.0.smithi144.stdout:9/438: creat d11/d12/d1b/d2c/d78/fa0 x:0 0 0 2024-09-10T11:29:36.414 INFO:tasks.workunit.client.0.smithi144.stdout:2/301: chown d7/de/d3d/d5e/f63 170 1 2024-09-10T11:29:36.414 INFO:tasks.workunit.client.1.smithi178.stdout:3/753: symlink db/d26/d40/d68/lf5 0 2024-09-10T11:29:36.415 INFO:tasks.workunit.client.0.smithi144.stdout:1/514: mknod d5/dd/d8f/caa 0 2024-09-10T11:29:36.416 INFO:tasks.workunit.client.0.smithi144.stdout:1/515: readlink d5/df/l36 0 2024-09-10T11:29:36.419 INFO:tasks.workunit.client.1.smithi178.stdout:3/754: truncate db/d20/d3d/d63/d6a/fdb 19122 0 2024-09-10T11:29:36.419 INFO:tasks.workunit.client.0.smithi144.stdout:2/302: mknod d7/d42/d64/c68 0 2024-09-10T11:29:36.420 INFO:tasks.workunit.client.1.smithi178.stdout:5/793: getdents d9/dc/dd/d3f/d5a/d7d 0 2024-09-10T11:29:36.421 INFO:tasks.workunit.client.0.smithi144.stdout:1/516: mkdir d5/d4a/d63/d84/dab 0 2024-09-10T11:29:36.421 INFO:tasks.workunit.client.0.smithi144.stdout:1/517: dread - d5/d2a/f5b zero size 2024-09-10T11:29:36.422 INFO:tasks.workunit.client.0.smithi144.stdout:9/439: creat d11/d12/fa1 x:0 0 0 2024-09-10T11:29:36.423 INFO:tasks.workunit.client.1.smithi178.stdout:5/794: creat d9/dc/dd/d3f/f108 x:0 0 0 2024-09-10T11:29:36.424 INFO:tasks.workunit.client.0.smithi144.stdout:2/303: creat d7/de/d36/d45/f69 x:0 0 0 2024-09-10T11:29:36.424 INFO:tasks.workunit.client.1.smithi178.stdout:3/755: getdents db/ddd 0 2024-09-10T11:29:36.424 INFO:tasks.workunit.client.0.smithi144.stdout:1/518: creat d5/d4a/d63/fac x:0 0 0 2024-09-10T11:29:36.425 INFO:tasks.workunit.client.1.smithi178.stdout:3/756: write db/d20/d3e/d6b/fd4 [980982,82076] 0 2024-09-10T11:29:36.426 INFO:tasks.workunit.client.1.smithi178.stdout:5/795: creat d9/dc/dd/d37/d42/d62/daf/df0/dc3/d105/f109 x:0 0 0 2024-09-10T11:29:36.427 INFO:tasks.workunit.client.0.smithi144.stdout:1/519: mkdir d5/dd/d64/dad 0 2024-09-10T11:29:36.428 INFO:tasks.workunit.client.1.smithi178.stdout:3/757: write db/d26/f2c [5068017,81336] 0 2024-09-10T11:29:36.429 INFO:tasks.workunit.client.0.smithi144.stdout:1/520: readlink d5/df/l3a 0 2024-09-10T11:29:36.429 INFO:tasks.workunit.client.1.smithi178.stdout:3/758: fsync db/f16 0 2024-09-10T11:29:36.431 INFO:tasks.workunit.client.0.smithi144.stdout:9/440: dwrite d11/d36/d2d/f9a [0,4194304] 0 2024-09-10T11:29:36.433 INFO:tasks.workunit.client.1.smithi178.stdout:3/759: symlink db/d35/d58/lf6 0 2024-09-10T11:29:36.436 INFO:tasks.workunit.client.0.smithi144.stdout:1/521: rename d5/df/f16 to d5/dd/d64/dad/fae 0 2024-09-10T11:29:36.437 INFO:tasks.workunit.client.0.smithi144.stdout:9/441: unlink fc 0 2024-09-10T11:29:36.438 INFO:tasks.workunit.client.1.smithi178.stdout:3/760: getdents db/d26/d74/dd0 0 2024-09-10T11:29:36.440 INFO:tasks.workunit.client.0.smithi144.stdout:1/522: creat d5/d2a/d65/faf x:0 0 0 2024-09-10T11:29:36.440 INFO:tasks.workunit.client.0.smithi144.stdout:2/304: dread d7/da/d1e/f25 [0,4194304] 0 2024-09-10T11:29:36.441 INFO:tasks.workunit.client.1.smithi178.stdout:3/761: creat db/d26/d74/dd0/ff7 x:0 0 0 2024-09-10T11:29:36.442 INFO:tasks.workunit.client.0.smithi144.stdout:2/305: write d7/de/d36/f4f [335331,37124] 0 2024-09-10T11:29:36.442 INFO:tasks.workunit.client.1.smithi178.stdout:5/796: dwrite d9/f86 [0,4194304] 0 2024-09-10T11:29:36.450 INFO:tasks.workunit.client.1.smithi178.stdout:3/762: link db/d36/l4d db/d20/d3d/d56/d64/lf8 0 2024-09-10T11:29:36.450 INFO:tasks.workunit.client.1.smithi178.stdout:5/797: dread d9/d19/f66 [0,4194304] 0 2024-09-10T11:29:36.451 INFO:tasks.workunit.client.1.smithi178.stdout:3/763: dread - db/d35/f5e zero size 2024-09-10T11:29:36.452 INFO:tasks.workunit.client.1.smithi178.stdout:5/798: read d9/dc/dd/d37/d42/d55/d6d/dcc/fd2 [6172685,66328] 0 2024-09-10T11:29:36.454 INFO:tasks.workunit.client.1.smithi178.stdout:5/799: mkdir d9/dc/dd/d3f/d5a/d10a 0 2024-09-10T11:29:36.455 INFO:tasks.workunit.client.1.smithi178.stdout:3/764: getdents db/d20/d3d/d63 0 2024-09-10T11:29:36.456 INFO:tasks.workunit.client.1.smithi178.stdout:3/765: dread - db/d20/d3d/d56/fe0 zero size 2024-09-10T11:29:36.456 INFO:tasks.workunit.client.1.smithi178.stdout:5/800: fdatasync d9/dc/dd/d3f/d5a/dbc/fca 0 2024-09-10T11:29:36.456 INFO:tasks.workunit.client.1.smithi178.stdout:3/766: stat db/d20/cef 0 2024-09-10T11:29:36.458 INFO:tasks.workunit.client.1.smithi178.stdout:3/767: write db/d20/d3d/d56/fe0 [325000,81628] 0 2024-09-10T11:29:36.460 INFO:tasks.workunit.client.1.smithi178.stdout:5/801: truncate d9/dc/dd/d3f/d5a/dbc/fc9 467219 0 2024-09-10T11:29:36.462 INFO:tasks.workunit.client.1.smithi178.stdout:5/802: creat d9/dc/dd/d37/d42/d55/d6d/f10b x:0 0 0 2024-09-10T11:29:36.463 INFO:tasks.workunit.client.1.smithi178.stdout:3/768: getdents db/d35/d3a/d42 0 2024-09-10T11:29:36.464 INFO:tasks.workunit.client.1.smithi178.stdout:5/803: creat d9/dc/dd/d37/d42/d62/daf/df0/dc3/d105/f10c x:0 0 0 2024-09-10T11:29:36.469 INFO:tasks.workunit.client.1.smithi178.stdout:5/804: rename d9/dc/dd/d37/d42/d62/daf/df0/c61 to d9/dc/dd/d37/d42/c10d 0 2024-09-10T11:29:36.469 INFO:tasks.workunit.client.1.smithi178.stdout:3/769: truncate db/d20/f28 442561 0 2024-09-10T11:29:36.470 INFO:tasks.workunit.client.1.smithi178.stdout:5/805: truncate d9/dc/dd/f7f 5054106 0 2024-09-10T11:29:36.472 INFO:tasks.workunit.client.1.smithi178.stdout:3/770: creat db/d26/ff9 x:0 0 0 2024-09-10T11:29:36.477 INFO:tasks.workunit.client.1.smithi178.stdout:3/771: dread db/d35/d3a/d9f/fb8 [0,4194304] 0 2024-09-10T11:29:36.481 INFO:tasks.workunit.client.1.smithi178.stdout:3/772: rename db/d20/d3d/d63/d6a to db/d20/d3e/d6b/dfa 0 2024-09-10T11:29:36.482 INFO:tasks.workunit.client.1.smithi178.stdout:5/806: dread d9/dc/d17/f3d [0,4194304] 0 2024-09-10T11:29:36.483 INFO:tasks.workunit.client.1.smithi178.stdout:3/773: unlink l0 0 2024-09-10T11:29:36.486 INFO:tasks.workunit.client.1.smithi178.stdout:5/807: rename d9/dc/dd/d37/d3e/c65 to d9/dc/dd/d37/d42/d55/d6d/c10e 0 2024-09-10T11:29:36.486 INFO:tasks.workunit.client.1.smithi178.stdout:3/774: fdatasync db/d35/f77 0 2024-09-10T11:29:36.487 INFO:tasks.workunit.client.1.smithi178.stdout:3/775: dread - db/d26/f80 zero size 2024-09-10T11:29:36.488 INFO:tasks.workunit.client.1.smithi178.stdout:3/776: chown db/d35/f5f 1261 1 2024-09-10T11:29:36.488 INFO:tasks.workunit.client.1.smithi178.stdout:3/777: fdatasync db/d20/d3d/d56/d64/f8f 0 2024-09-10T11:29:36.492 INFO:tasks.workunit.client.1.smithi178.stdout:5/808: truncate d9/dc/dd/d37/d42/f8a 3788162 0 2024-09-10T11:29:36.493 INFO:tasks.workunit.client.1.smithi178.stdout:5/809: fdatasync d9/fbe 0 2024-09-10T11:29:36.493 INFO:tasks.workunit.client.1.smithi178.stdout:5/810: chown l3 927939181 1 2024-09-10T11:29:36.494 INFO:tasks.workunit.client.1.smithi178.stdout:5/811: write d9/dc/dd/d37/d42/f4e [3401207,116270] 0 2024-09-10T11:29:36.496 INFO:tasks.workunit.client.1.smithi178.stdout:5/812: symlink d9/dc/d17/db0/l10f 0 2024-09-10T11:29:36.500 INFO:tasks.workunit.client.1.smithi178.stdout:3/778: dread db/d35/f77 [0,4194304] 0 2024-09-10T11:29:36.502 INFO:tasks.workunit.client.1.smithi178.stdout:3/779: mkdir db/d9b/dfb 0 2024-09-10T11:29:36.504 INFO:tasks.workunit.client.1.smithi178.stdout:3/780: creat db/d20/d3d/d56/dcf/ffc x:0 0 0 2024-09-10T11:29:36.506 INFO:tasks.workunit.client.0.smithi144.stdout:2/306: rename d7/f10 to d7/de/d3d/d4e/f6a 0 2024-09-10T11:29:36.506 INFO:tasks.workunit.client.1.smithi178.stdout:5/813: dwrite d9/dc/dd/d37/d3e/fa6 [0,4194304] 0 2024-09-10T11:29:36.508 INFO:tasks.workunit.client.0.smithi144.stdout:9/442: dwrite d11/d12/d6f/f8b [0,4194304] 0 2024-09-10T11:29:36.511 INFO:tasks.workunit.client.1.smithi178.stdout:3/781: dread db/d20/fb9 [0,4194304] 0 2024-09-10T11:29:36.511 INFO:tasks.workunit.client.1.smithi178.stdout:5/814: symlink d9/dc/dd/d3f/l110 0 2024-09-10T11:29:36.516 INFO:tasks.workunit.client.0.smithi144.stdout:2/307: dwrite d7/de/d36/f51 [4194304,4194304] 0 2024-09-10T11:29:36.518 INFO:tasks.workunit.client.0.smithi144.stdout:9/443: dwrite d11/d12/d1b/d41/f5c [0,4194304] 0 2024-09-10T11:29:36.518 INFO:tasks.workunit.client.1.smithi178.stdout:5/815: write d9/f5e [8787503,78493] 0 2024-09-10T11:29:36.519 INFO:tasks.workunit.client.1.smithi178.stdout:9/779: sync 2024-09-10T11:29:36.521 INFO:tasks.workunit.client.1.smithi178.stdout:3/782: dread db/d20/d3e/d6b/dc0/ddf/dcb/d81/fab [0,4194304] 0 2024-09-10T11:29:36.521 INFO:tasks.workunit.client.0.smithi144.stdout:2/308: mknod d7/de/d3d/d4e/c6b 0 2024-09-10T11:29:36.522 INFO:tasks.workunit.client.0.smithi144.stdout:2/309: chown d7/da/d1e/d21/d54/d61 62139735 1 2024-09-10T11:29:36.525 INFO:tasks.workunit.client.1.smithi178.stdout:9/780: creat d2/d8/ff0 x:0 0 0 2024-09-10T11:29:36.526 INFO:tasks.workunit.client.1.smithi178.stdout:5/816: chown d9/dc/dd/d37/d42/d62/daf/df0/c93 3848907 1 2024-09-10T11:29:36.528 INFO:tasks.workunit.client.1.smithi178.stdout:3/783: rename db/d35/d58/lf6 to db/d20/d3e/d6b/dc0/ddf/dcb/lfd 0 2024-09-10T11:29:36.531 INFO:tasks.workunit.client.1.smithi178.stdout:9/781: unlink d2/df/d31/l4c 0 2024-09-10T11:29:36.534 INFO:tasks.workunit.client.1.smithi178.stdout:5/817: mkdir d9/dc/dd/d37/d42/d111 0 2024-09-10T11:29:36.535 INFO:tasks.workunit.client.1.smithi178.stdout:5/818: chown d9/dc/dd/d37 20662 1 2024-09-10T11:29:36.535 INFO:tasks.workunit.client.1.smithi178.stdout:3/784: link db/d20/d3d/d56/d64/f8f db/d20/d3d/d63/ffe 0 2024-09-10T11:29:36.538 INFO:tasks.workunit.client.1.smithi178.stdout:9/782: getdents d2/d13/d75/d60/d66 0 2024-09-10T11:29:36.541 INFO:tasks.workunit.client.1.smithi178.stdout:9/783: mkdir d2/d13/d40/d56/dbe/df1 0 2024-09-10T11:29:36.541 INFO:tasks.workunit.client.1.smithi178.stdout:3/785: link l6 db/d20/d3d/lff 0 2024-09-10T11:29:36.542 INFO:tasks.workunit.client.1.smithi178.stdout:5/819: truncate d9/dc/dd/d37/f8d 5680009 0 2024-09-10T11:29:36.542 INFO:tasks.workunit.client.1.smithi178.stdout:9/784: dread - d2/d13/d40/d89/d93/d88/fa8 zero size 2024-09-10T11:29:36.542 INFO:tasks.workunit.client.1.smithi178.stdout:5/820: chown d9/f51 12 1 2024-09-10T11:29:36.543 INFO:tasks.workunit.client.1.smithi178.stdout:3/786: unlink db/d20/d3d/d63/da2/d98/lda 0 2024-09-10T11:29:36.544 INFO:tasks.workunit.client.1.smithi178.stdout:9/785: read d2/d8/f73 [3659663,75220] 0 2024-09-10T11:29:36.546 INFO:tasks.workunit.client.1.smithi178.stdout:5/821: rename d9/f5e to d9/dc/dd/d37/dee/f112 0 2024-09-10T11:29:36.547 INFO:tasks.workunit.client.1.smithi178.stdout:5/822: dread - d9/dc/dd/d3f/dd6/fff zero size 2024-09-10T11:29:36.547 INFO:tasks.workunit.client.1.smithi178.stdout:3/787: dread db/f27 [4194304,4194304] 0 2024-09-10T11:29:36.548 INFO:tasks.workunit.client.1.smithi178.stdout:5/823: read d9/dc/d17/f3d [2752093,18739] 0 2024-09-10T11:29:36.549 INFO:tasks.workunit.client.1.smithi178.stdout:9/786: rename d2/d19/d1c/d77/f90 to d2/d51/ff2 0 2024-09-10T11:29:36.549 INFO:tasks.workunit.client.1.smithi178.stdout:9/787: chown d2/d19/d1c/l65 178 1 2024-09-10T11:29:36.551 INFO:tasks.workunit.client.1.smithi178.stdout:3/788: symlink db/d20/d3d/d63/da2/d98/l100 0 2024-09-10T11:29:36.551 INFO:tasks.workunit.client.1.smithi178.stdout:5/824: mknod d9/dc/d17/db0/c113 0 2024-09-10T11:29:36.552 INFO:tasks.workunit.client.0.smithi144.stdout:2/310: write d7/de/d3d/f3e [1656166,9139] 0 2024-09-10T11:29:36.552 INFO:tasks.workunit.client.1.smithi178.stdout:9/788: fsync d2/d19/d1c/f79 0 2024-09-10T11:29:36.553 INFO:tasks.workunit.client.1.smithi178.stdout:3/789: readlink db/d26/d40/lac 0 2024-09-10T11:29:36.554 INFO:tasks.workunit.client.0.smithi144.stdout:2/311: mkdir d7/de/d36/d45/d47/d6c 0 2024-09-10T11:29:36.555 INFO:tasks.workunit.client.1.smithi178.stdout:9/789: symlink d2/d8/lf3 0 2024-09-10T11:29:36.555 INFO:tasks.workunit.client.0.smithi144.stdout:2/312: dread - d7/de/f37 zero size 2024-09-10T11:29:36.556 INFO:tasks.workunit.client.1.smithi178.stdout:5/825: truncate d9/dc/dd/d3f/f49 4601498 0 2024-09-10T11:29:36.557 INFO:tasks.workunit.client.0.smithi144.stdout:2/313: creat d7/de/d3d/d4e/f6d x:0 0 0 2024-09-10T11:29:36.558 INFO:tasks.workunit.client.1.smithi178.stdout:3/790: rename db/d20/d3d/d63/ffe to db/d20/d3d/d63/f101 0 2024-09-10T11:29:36.559 INFO:tasks.workunit.client.0.smithi144.stdout:2/314: creat d7/da/d5b/f6e x:0 0 0 2024-09-10T11:29:36.560 INFO:tasks.workunit.client.0.smithi144.stdout:2/315: write d7/da/f35 [3391470,86764] 0 2024-09-10T11:29:36.560 INFO:tasks.workunit.client.1.smithi178.stdout:9/790: dwrite d2/df/d31/d37/d3d/fc7 [0,4194304] 0 2024-09-10T11:29:36.567 INFO:tasks.workunit.client.1.smithi178.stdout:5/826: dwrite d9/dc/d17/d1d/f8c [0,4194304] 0 2024-09-10T11:29:36.570 INFO:tasks.workunit.client.1.smithi178.stdout:9/791: unlink d2/df/cb3 0 2024-09-10T11:29:36.570 INFO:tasks.workunit.client.1.smithi178.stdout:9/792: readlink d2/d13/d40/d89/d93/lcd 0 2024-09-10T11:29:36.571 INFO:tasks.workunit.client.1.smithi178.stdout:3/791: creat db/d26/d40/d68/f102 x:0 0 0 2024-09-10T11:29:36.574 INFO:tasks.workunit.client.1.smithi178.stdout:5/827: mkdir d9/dc/dd/d3f/d5a/d114 0 2024-09-10T11:29:36.577 INFO:tasks.workunit.client.1.smithi178.stdout:7/653: sync 2024-09-10T11:29:36.577 INFO:tasks.workunit.client.1.smithi178.stdout:4/598: sync 2024-09-10T11:29:36.580 INFO:tasks.workunit.client.1.smithi178.stdout:4/599: rename d0/d9/dc to d0/d9/dc/d8a/dd1 22 2024-09-10T11:29:36.581 INFO:tasks.workunit.client.1.smithi178.stdout:5/828: dread d9/f86 [0,4194304] 0 2024-09-10T11:29:36.582 INFO:tasks.workunit.client.1.smithi178.stdout:3/792: dwrite db/d26/d74/fd3 [0,4194304] 0 2024-09-10T11:29:36.583 INFO:tasks.workunit.client.1.smithi178.stdout:5/829: truncate d9/dc/dd/d37/d42/d55/fd0 327886 0 2024-09-10T11:29:36.584 INFO:tasks.workunit.client.1.smithi178.stdout:4/600: rmdir d0 39 2024-09-10T11:29:36.584 INFO:tasks.workunit.client.1.smithi178.stdout:5/830: read - d9/dc/dd/d3f/d5a/d7d/f9b zero size 2024-09-10T11:29:36.585 INFO:tasks.workunit.client.1.smithi178.stdout:5/831: stat d9/dc/dd/d3f/d5a/d114 0 2024-09-10T11:29:36.586 INFO:tasks.workunit.client.0.smithi144.stdout:2/316: dread d7/de/f1b [0,4194304] 0 2024-09-10T11:29:36.586 INFO:tasks.workunit.client.1.smithi178.stdout:7/654: symlink d6/d10/d1d/d22/d50/d7e/da7/lf2 0 2024-09-10T11:29:36.587 INFO:tasks.workunit.client.0.smithi144.stdout:2/317: write d7/f41 [683120,116941] 0 2024-09-10T11:29:36.589 INFO:tasks.workunit.client.0.smithi144.stdout:2/318: write d7/de/d3d/f3e [1628813,116737] 0 2024-09-10T11:29:36.589 INFO:tasks.workunit.client.1.smithi178.stdout:3/793: mkdir db/d26/d103 0 2024-09-10T11:29:36.591 INFO:tasks.workunit.client.1.smithi178.stdout:3/794: stat db/d20/d3e/d6b/dc0/ddf/dcb/d81/fab 0 2024-09-10T11:29:36.592 INFO:tasks.workunit.client.1.smithi178.stdout:5/832: read d9/dc/dd/d37/d42/d9f/fa5 [366605,63353] 0 2024-09-10T11:29:36.592 INFO:tasks.workunit.client.0.smithi144.stdout:2/319: mknod d7/de/d36/d45/d47/c6f 0 2024-09-10T11:29:36.593 INFO:tasks.workunit.client.0.smithi144.stdout:2/320: write d7/d29/f46 [535330,43170] 0 2024-09-10T11:29:36.594 INFO:tasks.workunit.client.1.smithi178.stdout:4/601: getdents d0/d8/da/d1a/d51/d5d/dc9 0 2024-09-10T11:29:36.595 INFO:tasks.workunit.client.0.smithi144.stdout:2/321: mkdir d7/de/d70 0 2024-09-10T11:29:36.596 INFO:tasks.workunit.client.1.smithi178.stdout:4/602: truncate d0/d9/d3d/fc6 591119 0 2024-09-10T11:29:36.596 INFO:tasks.workunit.client.1.smithi178.stdout:3/795: rmdir db/d20/d3e/d6b/dc0/ddf 39 2024-09-10T11:29:36.598 INFO:tasks.workunit.client.1.smithi178.stdout:5/833: truncate d9/dc/dd/d37/dbd/dda/fa9 4897088 0 2024-09-10T11:29:36.604 INFO:tasks.workunit.client.1.smithi178.stdout:9/793: chown d2/df/f1b 3 1 2024-09-10T11:29:36.605 INFO:tasks.workunit.client.0.smithi144.stdout:8/261: sync 2024-09-10T11:29:36.605 INFO:tasks.workunit.client.1.smithi178.stdout:9/794: unlink d2/f18 0 2024-09-10T11:29:36.606 INFO:tasks.workunit.client.0.smithi144.stdout:8/262: write d0/fc [3960840,80178] 0 2024-09-10T11:29:36.606 INFO:tasks.workunit.client.1.smithi178.stdout:3/796: getdents db/d20/d3e/d6b/dc0/ddf/dcb/d81 0 2024-09-10T11:29:36.607 INFO:tasks.workunit.client.1.smithi178.stdout:7/655: dwrite d6/d10/d1e/d2d/d2b/f39 [0,4194304] 0 2024-09-10T11:29:36.609 INFO:tasks.workunit.client.1.smithi178.stdout:4/603: dread d0/f1c [0,4194304] 0 2024-09-10T11:29:36.609 INFO:tasks.workunit.client.0.smithi144.stdout:8/263: rename d0/d9/d21/d25/c41 to d0/d1b/d47/c57 0 2024-09-10T11:29:36.610 INFO:tasks.workunit.client.1.smithi178.stdout:5/834: dwrite d9/dc/dd/d37/dbd/dda/db8/fc5 [0,4194304] 0 2024-09-10T11:29:36.612 INFO:tasks.workunit.client.1.smithi178.stdout:4/604: write d0/d10/f5a [3823070,57578] 0 2024-09-10T11:29:36.613 INFO:tasks.workunit.client.1.smithi178.stdout:7/656: truncate d6/d10/d1d/d22/d3b/f5a 1592347 0 2024-09-10T11:29:36.614 INFO:tasks.workunit.client.1.smithi178.stdout:1/667: sync 2024-09-10T11:29:36.614 INFO:tasks.workunit.client.1.smithi178.stdout:0/695: sync 2024-09-10T11:29:36.614 INFO:tasks.workunit.client.1.smithi178.stdout:2/914: sync 2024-09-10T11:29:36.614 INFO:tasks.workunit.client.1.smithi178.stdout:6/759: sync 2024-09-10T11:29:36.614 INFO:tasks.workunit.client.1.smithi178.stdout:8/844: sync 2024-09-10T11:29:36.615 INFO:tasks.workunit.client.1.smithi178.stdout:3/797: unlink db/d26/d40/c99 0 2024-09-10T11:29:36.615 INFO:tasks.workunit.client.1.smithi178.stdout:5/835: write d9/dc/dd/d37/d3e/fdb [467686,80922] 0 2024-09-10T11:29:36.616 INFO:tasks.workunit.client.0.smithi144.stdout:8/264: dwrite d0/d9/d23/f50 [0,4194304] 0 2024-09-10T11:29:36.617 INFO:tasks.workunit.client.0.smithi144.stdout:8/265: chown d0/d1b/l28 2922 1 2024-09-10T11:29:36.617 INFO:tasks.workunit.client.1.smithi178.stdout:9/795: write d2/d13/d75/d60/d66/fee [4754786,96305] 0 2024-09-10T11:29:36.619 INFO:tasks.workunit.client.0.smithi144.stdout:8/266: write d0/d9/d23/d26/f36 [2059601,121133] 0 2024-09-10T11:29:36.621 INFO:tasks.workunit.client.1.smithi178.stdout:7/657: fdatasync d6/d10/d1d/d4a/d79/df1/f8c 0 2024-09-10T11:29:36.625 INFO:tasks.workunit.client.1.smithi178.stdout:8/845: write d5/db/ff5 [119463,57787] 0 2024-09-10T11:29:36.626 INFO:tasks.workunit.client.1.smithi178.stdout:0/696: chown d0/dc/d39/d7c/lc8 1772407 1 2024-09-10T11:29:36.627 INFO:tasks.workunit.client.1.smithi178.stdout:6/760: symlink d3/d74/d91/daa/dc2/d97/l115 0 2024-09-10T11:29:36.627 INFO:tasks.workunit.client.0.smithi144.stdout:8/267: rename d0/d9/d1f/l1a to d0/d9/d21/d25/l58 0 2024-09-10T11:29:36.627 INFO:tasks.workunit.client.0.smithi144.stdout:8/268: stat d0/d9/d23/d26/f4e 0 2024-09-10T11:29:36.630 INFO:tasks.workunit.client.1.smithi178.stdout:9/796: dread d2/f3f [0,4194304] 0 2024-09-10T11:29:36.631 INFO:tasks.workunit.client.1.smithi178.stdout:0/697: creat d0/d12/d3f/fdd x:0 0 0 2024-09-10T11:29:36.632 INFO:tasks.workunit.client.1.smithi178.stdout:8/846: unlink d5/db/d10/d70/cf6 0 2024-09-10T11:29:36.633 INFO:tasks.workunit.client.1.smithi178.stdout:5/836: symlink d9/dc/dd/d37/d42/d62/daf/df0/dc3/d105/l115 0 2024-09-10T11:29:36.640 INFO:tasks.workunit.client.1.smithi178.stdout:4/605: dwrite d0/d9/d3d/fc6 [0,4194304] 0 2024-09-10T11:29:36.640 INFO:tasks.workunit.client.1.smithi178.stdout:3/798: rename db/d35/d3a/l44 to db/d20/d3d/d63/da2/d98/l104 0 2024-09-10T11:29:36.640 INFO:tasks.workunit.client.1.smithi178.stdout:6/761: chown d3/d7/d8/d56/da3/ce6 433136700 1 2024-09-10T11:29:36.642 INFO:tasks.workunit.client.1.smithi178.stdout:7/658: mkdir d6/d10/d1d/d22/d50/def/df3 0 2024-09-10T11:29:36.643 INFO:tasks.workunit.client.1.smithi178.stdout:3/799: readlink db/d20/d3d/d56/l8b 0 2024-09-10T11:29:36.644 INFO:tasks.workunit.client.1.smithi178.stdout:7/659: stat d6/d10/d1d/d22/d50/d98/da6/cc6 0 2024-09-10T11:29:36.647 INFO:tasks.workunit.client.1.smithi178.stdout:2/915: dwrite d1/d7/d24/d46/d88/d9d/dab/d5e/fdc [0,4194304] 0 2024-09-10T11:29:36.650 INFO:tasks.workunit.client.1.smithi178.stdout:2/916: read - d1/d7/d24/d46/d88/d9d/dab/d5e/f12f zero size 2024-09-10T11:29:36.663 INFO:tasks.workunit.client.1.smithi178.stdout:2/917: dread d1/d7/d24/d46/d88/d9d/dab/f17 [0,4194304] 0 2024-09-10T11:29:36.674 INFO:tasks.workunit.client.1.smithi178.stdout:0/698: fdatasync d0/f8 0 2024-09-10T11:29:36.676 INFO:tasks.workunit.client.1.smithi178.stdout:8/847: creat d5/db/d10/d53/d82/db2/f118 x:0 0 0 2024-09-10T11:29:36.677 INFO:tasks.workunit.client.1.smithi178.stdout:0/699: chown d0/dc/d39/d50/f25 27 1 2024-09-10T11:29:36.677 INFO:tasks.workunit.client.1.smithi178.stdout:1/668: rename d3/d9/d98/dac/d3a/d9f/lcd to d3/d9/d98/dac/d3a/d50/db5/lf6 0 2024-09-10T11:29:36.679 INFO:tasks.workunit.client.1.smithi178.stdout:1/669: stat d3/d9/d98/dd1/ddc/fd7 0 2024-09-10T11:29:36.683 INFO:tasks.workunit.client.1.smithi178.stdout:2/918: rmdir d1/d41 39 2024-09-10T11:29:36.685 INFO:tasks.workunit.client.1.smithi178.stdout:4/606: dread d0/f38 [0,4194304] 0 2024-09-10T11:29:36.685 INFO:tasks.workunit.client.1.smithi178.stdout:8/848: creat d5/d1d/f119 x:0 0 0 2024-09-10T11:29:36.686 INFO:tasks.workunit.client.1.smithi178.stdout:4/607: chown d0/d9/ld0 231 1 2024-09-10T11:29:36.687 INFO:tasks.workunit.client.1.smithi178.stdout:8/849: write d5/db/d10/d20/fcc [209801,36646] 0 2024-09-10T11:29:36.687 INFO:tasks.workunit.client.1.smithi178.stdout:0/700: symlink d0/dc/d1f/d2d/lde 0 2024-09-10T11:29:36.687 INFO:tasks.workunit.client.1.smithi178.stdout:4/608: write d0/d10/f2c [417644,6334] 0 2024-09-10T11:29:36.688 INFO:tasks.workunit.client.1.smithi178.stdout:8/850: stat d5/db/d10/d35 0 2024-09-10T11:29:36.689 INFO:tasks.workunit.client.1.smithi178.stdout:4/609: chown d0/d8/da/d1a/d51/fb0 0 1 2024-09-10T11:29:36.689 INFO:tasks.workunit.client.1.smithi178.stdout:8/851: chown d5/db/d1e/fa3 5640 1 2024-09-10T11:29:36.689 INFO:tasks.workunit.client.1.smithi178.stdout:3/800: link db/d20/f33 db/d26/d103/f105 0 2024-09-10T11:29:36.690 INFO:tasks.workunit.client.1.smithi178.stdout:4/610: truncate d0/d9/d3d/fcb 629158 0 2024-09-10T11:29:36.692 INFO:tasks.workunit.client.1.smithi178.stdout:7/660: symlink d6/d10/d1e/d2d/d2b/lf4 0 2024-09-10T11:29:36.695 INFO:tasks.workunit.client.1.smithi178.stdout:8/852: chown d5/db/d10/d20/f79 13022004 1 2024-09-10T11:29:36.696 INFO:tasks.workunit.client.1.smithi178.stdout:3/801: truncate db/d20/d70/f73 991480 0 2024-09-10T11:29:36.696 INFO:tasks.workunit.client.1.smithi178.stdout:3/802: dread - db/d26/d74/dd0/ff7 zero size 2024-09-10T11:29:36.697 INFO:tasks.workunit.client.1.smithi178.stdout:4/611: write d0/d8/da/d1a/d51/d5d/fa1 [212906,77725] 0 2024-09-10T11:29:36.697 INFO:tasks.workunit.client.1.smithi178.stdout:3/803: fdatasync db/d26/d74/fd3 0 2024-09-10T11:29:36.700 INFO:tasks.workunit.client.1.smithi178.stdout:3/804: dread db/d20/d3d/d63/fbb [0,4194304] 0 2024-09-10T11:29:36.701 INFO:tasks.workunit.client.1.smithi178.stdout:9/797: rename d2/df/d31/f76 to d2/d13/ff4 0 2024-09-10T11:29:36.701 INFO:tasks.workunit.client.1.smithi178.stdout:1/670: link d3/d9/d98/dac/d3a/d6e/lce d3/d9/d98/dd1/d7e/lf7 0 2024-09-10T11:29:36.702 INFO:tasks.workunit.client.1.smithi178.stdout:2/919: link d1/d7/d24/d7d/d84/d55/fb4 d1/d22/d4d/d9b/f132 0 2024-09-10T11:29:36.708 INFO:tasks.workunit.client.1.smithi178.stdout:1/671: symlink d3/d9/d98/dd1/ddc/lf8 0 2024-09-10T11:29:36.709 INFO:tasks.workunit.client.1.smithi178.stdout:2/920: creat d1/d7/d24/d7d/d84/d4b/d89/f133 x:0 0 0 2024-09-10T11:29:36.710 INFO:tasks.workunit.client.1.smithi178.stdout:7/661: truncate d6/d10/f5c 3389017 0 2024-09-10T11:29:36.711 INFO:tasks.workunit.client.1.smithi178.stdout:1/672: write d3/d9/d98/dd1/ddc/f86 [441918,56861] 0 2024-09-10T11:29:36.712 INFO:tasks.workunit.client.1.smithi178.stdout:6/762: dwrite d3/d74/dc9/f2d [0,4194304] 0 2024-09-10T11:29:36.713 INFO:tasks.workunit.client.1.smithi178.stdout:7/662: write d6/d10/d1d/d4a/d79/f85 [2681669,29203] 0 2024-09-10T11:29:36.714 INFO:tasks.workunit.client.1.smithi178.stdout:5/837: truncate d9/dc/dd/d37/d42/d55/d6d/dcc/dd8/f100 273247 0 2024-09-10T11:29:36.715 INFO:tasks.workunit.client.1.smithi178.stdout:9/798: creat d2/d13/d40/d56/de1/ff5 x:0 0 0 2024-09-10T11:29:36.722 INFO:tasks.workunit.client.1.smithi178.stdout:4/612: creat d0/d8/da/d1a/d51/d5d/fd2 x:0 0 0 2024-09-10T11:29:36.727 INFO:tasks.workunit.client.1.smithi178.stdout:0/701: rename d0/dc/d1f/l58 to d0/ldf 0 2024-09-10T11:29:36.728 INFO:tasks.workunit.client.1.smithi178.stdout:6/763: rmdir d3/d74/d91/daa/dc2/d57/d78/d7c 39 2024-09-10T11:29:36.729 INFO:tasks.workunit.client.1.smithi178.stdout:5/838: write d9/dc/ff4 [1559958,8717] 0 2024-09-10T11:29:36.729 INFO:tasks.workunit.client.1.smithi178.stdout:6/764: fsync d3/d7/d8/d15/d1c/d2a/d18/ffd 0 2024-09-10T11:29:36.731 INFO:tasks.workunit.client.1.smithi178.stdout:2/921: readlink d1/d41/d14/d11/db9/lf9 0 2024-09-10T11:29:36.732 INFO:tasks.workunit.client.1.smithi178.stdout:4/613: creat d0/d8/da/fd3 x:0 0 0 2024-09-10T11:29:36.733 INFO:tasks.workunit.client.1.smithi178.stdout:3/805: write db/d20/d70/f73 [213748,120943] 0 2024-09-10T11:29:36.735 INFO:tasks.workunit.client.1.smithi178.stdout:8/853: rename d5/d30/db8/l10b to d5/db/d10/d35/d117/l11a 0 2024-09-10T11:29:36.739 INFO:tasks.workunit.client.1.smithi178.stdout:1/673: write d3/d9/d98/dd1/ddc/f73 [243430,52376] 0 2024-09-10T11:29:36.739 INFO:tasks.workunit.client.0.smithi144.stdout:4/361: sync 2024-09-10T11:29:36.739 INFO:tasks.workunit.client.0.smithi144.stdout:4/362: chown d14/d1e/f32 8617683 1 2024-09-10T11:29:36.740 INFO:tasks.workunit.client.1.smithi178.stdout:6/765: dread d3/d7/d8/d15/d1c/f40 [0,4194304] 0 2024-09-10T11:29:36.741 INFO:tasks.workunit.client.1.smithi178.stdout:2/922: creat d1/d41/d14/d11/db9/dc1/f134 x:0 0 0 2024-09-10T11:29:36.741 INFO:tasks.workunit.client.0.smithi144.stdout:4/363: write d14/d1e/d22/d2b/d35/f38 [651566,79510] 0 2024-09-10T11:29:36.742 INFO:tasks.workunit.client.1.smithi178.stdout:4/614: fdatasync d0/d8/d3e/f4c 0 2024-09-10T11:29:36.742 INFO:tasks.workunit.client.1.smithi178.stdout:5/839: dwrite d9/dc/dd/d37/d42/d55/d6d/fea [0,4194304] 0 2024-09-10T11:29:36.744 INFO:tasks.workunit.client.0.smithi144.stdout:4/364: mkdir d14/d7e 0 2024-09-10T11:29:36.745 INFO:tasks.workunit.client.1.smithi178.stdout:9/799: truncate d2/d8/f9 2924539 0 2024-09-10T11:29:36.750 INFO:tasks.workunit.client.1.smithi178.stdout:9/800: chown d2/d13/d40/d89/d93/d88/dae/fdb 4 1 2024-09-10T11:29:36.751 INFO:tasks.workunit.client.1.smithi178.stdout:7/663: dwrite d6/fd [0,4194304] 0 2024-09-10T11:29:36.752 INFO:tasks.workunit.client.1.smithi178.stdout:6/766: mkdir d3/d74/d91/daa/dc2/d57/d116 0 2024-09-10T11:29:36.752 INFO:tasks.workunit.client.1.smithi178.stdout:4/615: symlink d0/d8/da/d1a/ld4 0 2024-09-10T11:29:36.752 INFO:tasks.workunit.client.1.smithi178.stdout:0/702: link d0/d2c/l54 d0/d12/le0 0 2024-09-10T11:29:36.756 INFO:tasks.workunit.client.1.smithi178.stdout:5/840: mknod d9/dc/c116 0 2024-09-10T11:29:36.758 INFO:tasks.workunit.client.1.smithi178.stdout:2/923: dwrite d1/d7/d24/d46/d88/d9d/dab/d5b/d60/dd7/d108/f111 [0,4194304] 0 2024-09-10T11:29:36.759 INFO:tasks.workunit.client.0.smithi144.stdout:4/365: dread d14/d1e/f50 [0,4194304] 0 2024-09-10T11:29:36.761 INFO:tasks.workunit.client.1.smithi178.stdout:4/616: mkdir d0/d9d/dd5 0 2024-09-10T11:29:36.761 INFO:tasks.workunit.client.1.smithi178.stdout:5/841: write d9/dc/dd/d3f/fe9 [814342,122919] 0 2024-09-10T11:29:36.762 INFO:tasks.workunit.client.1.smithi178.stdout:8/854: link d5/d30/d33/c89 d5/db/d10/d20/dc0/d109/c11b 0 2024-09-10T11:29:36.763 INFO:tasks.workunit.client.1.smithi178.stdout:8/855: dread - d5/db/dda/dfa/f111 zero size 2024-09-10T11:29:36.763 INFO:tasks.workunit.client.1.smithi178.stdout:3/806: truncate db/d20/d70/f8e 1353692 0 2024-09-10T11:29:36.764 INFO:tasks.workunit.client.0.smithi144.stdout:4/366: symlink d14/d1e/d22/l7f 0 2024-09-10T11:29:36.764 INFO:tasks.workunit.client.1.smithi178.stdout:8/856: fsync d5/d30/fb6 0 2024-09-10T11:29:36.764 INFO:tasks.workunit.client.1.smithi178.stdout:5/842: read d9/dc/dd/fc1 [200333,7076] 0 2024-09-10T11:29:36.765 INFO:tasks.workunit.client.1.smithi178.stdout:0/703: creat d0/dc/d39/da4/dce/fe1 x:0 0 0 2024-09-10T11:29:36.766 INFO:tasks.workunit.client.1.smithi178.stdout:5/843: fdatasync d9/dc/dd/d37/dbd/dda/db8/fe0 0 2024-09-10T11:29:36.769 INFO:tasks.workunit.client.1.smithi178.stdout:4/617: chown d0/d9/dc/cb1 5179 1 2024-09-10T11:29:36.769 INFO:tasks.workunit.client.0.smithi144.stdout:4/367: rename d14/d1e/d22/f4a to d14/d1e/d22/d34/d3e/d40/d4b/f80 0 2024-09-10T11:29:36.771 INFO:tasks.workunit.client.1.smithi178.stdout:4/618: write d0/d9/d3d/d4f/f52 [1862667,103310] 0 2024-09-10T11:29:36.771 INFO:tasks.workunit.client.1.smithi178.stdout:2/924: mkdir d1/d7/d24/d46/d88/d9d/dab/d5b/d60/dd7/d135 0 2024-09-10T11:29:36.771 INFO:tasks.workunit.client.1.smithi178.stdout:6/767: symlink d3/d74/d91/daa/d103/l117 0 2024-09-10T11:29:36.772 INFO:tasks.workunit.client.1.smithi178.stdout:0/704: read d0/dc/d39/d4c/f73 [3624672,57076] 0 2024-09-10T11:29:36.773 INFO:tasks.workunit.client.1.smithi178.stdout:3/807: creat db/d20/d3d/d56/dcf/f106 x:0 0 0 2024-09-10T11:29:36.773 INFO:tasks.workunit.client.1.smithi178.stdout:8/857: creat d5/d30/d33/d3a/f11c x:0 0 0 2024-09-10T11:29:36.774 INFO:tasks.workunit.client.1.smithi178.stdout:3/808: dread - db/d26/d40/f7a zero size 2024-09-10T11:29:36.775 INFO:tasks.workunit.client.1.smithi178.stdout:0/705: write d0/dc/d39/d50/f1d [3326754,93579] 0 2024-09-10T11:29:36.775 INFO:tasks.workunit.client.0.smithi144.stdout:4/368: link d14/d1e/d22/d2b/d35/d3c/l59 d14/d1e/l81 0 2024-09-10T11:29:36.777 INFO:tasks.workunit.client.1.smithi178.stdout:5/844: unlink d9/dc/dd/ce 0 2024-09-10T11:29:36.779 INFO:tasks.workunit.client.1.smithi178.stdout:0/706: chown d0/d2c/d89/lc6 0 1 2024-09-10T11:29:36.779 INFO:tasks.workunit.client.1.smithi178.stdout:5/845: fdatasync d9/dc/dd/d37/d3e/f78 0 2024-09-10T11:29:36.781 INFO:tasks.workunit.client.1.smithi178.stdout:6/768: fsync d3/d74/dc9/fb2 0 2024-09-10T11:29:36.782 INFO:tasks.workunit.client.1.smithi178.stdout:8/858: symlink d5/d30/d33/d75/l11d 0 2024-09-10T11:29:36.782 INFO:tasks.workunit.client.0.smithi144.stdout:4/369: link d14/d6f/l70 d14/d1e/d22/d2b/d35/d4f/l82 0 2024-09-10T11:29:36.782 INFO:tasks.workunit.client.0.smithi144.stdout:4/370: chown f5 251954289 1 2024-09-10T11:29:36.783 INFO:tasks.workunit.client.1.smithi178.stdout:7/664: truncate d6/d10/d1d/d4a/d79/df1/f8c 7211417 0 2024-09-10T11:29:36.783 INFO:tasks.workunit.client.1.smithi178.stdout:6/769: fsync d3/d7/d8/d56/fed 0 2024-09-10T11:29:36.785 INFO:tasks.workunit.client.1.smithi178.stdout:3/809: unlink db/d35/d3a/d42/d46/cbd 0 2024-09-10T11:29:36.787 INFO:tasks.workunit.client.1.smithi178.stdout:4/619: symlink d0/d8/df/ld6 0 2024-09-10T11:29:36.790 INFO:tasks.workunit.client.1.smithi178.stdout:9/801: dwrite d2/f26 [0,4194304] 0 2024-09-10T11:29:36.790 INFO:tasks.workunit.client.1.smithi178.stdout:0/707: readlink d0/dc/d39/d7c/d85/l8d 0 2024-09-10T11:29:36.793 INFO:tasks.workunit.client.1.smithi178.stdout:8/859: mknod d5/db/d10/c11e 0 2024-09-10T11:29:36.793 INFO:tasks.workunit.client.1.smithi178.stdout:0/708: truncate d0/d2c/d95/dbd/fcb 926742 0 2024-09-10T11:29:36.794 INFO:tasks.workunit.client.1.smithi178.stdout:6/770: mknod d3/d6e/d87/c118 0 2024-09-10T11:29:36.796 INFO:tasks.workunit.client.1.smithi178.stdout:8/860: dread d5/db/ff5 [0,4194304] 0 2024-09-10T11:29:36.799 INFO:tasks.workunit.client.1.smithi178.stdout:7/665: creat d6/d10/d1d/d22/d3b/ff5 x:0 0 0 2024-09-10T11:29:36.800 INFO:tasks.workunit.client.0.smithi144.stdout:4/371: dwrite d14/d1e/f50 [0,4194304] 0 2024-09-10T11:29:36.802 INFO:tasks.workunit.client.1.smithi178.stdout:2/925: dread d1/d7/d24/d7d/d84/f2d [0,4194304] 0 2024-09-10T11:29:36.804 INFO:tasks.workunit.client.1.smithi178.stdout:4/620: rename d0/d63/fa9 to d0/d8/da/d1a/d51/fd7 0 2024-09-10T11:29:36.809 INFO:tasks.workunit.client.0.smithi144.stdout:4/372: dwrite d14/d1e/d22/d2b/f2e [0,4194304] 0 2024-09-10T11:29:36.809 INFO:tasks.workunit.client.1.smithi178.stdout:6/771: mkdir d3/d74/d91/daa/dc2/d97/d119 0 2024-09-10T11:29:36.810 INFO:tasks.workunit.client.1.smithi178.stdout:2/926: dwrite d1/d41/f128 [0,4194304] 0 2024-09-10T11:29:36.814 INFO:tasks.workunit.client.1.smithi178.stdout:5/846: creat d9/dc/f117 x:0 0 0 2024-09-10T11:29:36.816 INFO:tasks.workunit.client.0.smithi144.stdout:4/373: dwrite d14/d1e/f7c [0,4194304] 0 2024-09-10T11:29:36.818 INFO:tasks.workunit.client.0.smithi144.stdout:4/374: mkdir d14/d1e/d83 0 2024-09-10T11:29:36.820 INFO:tasks.workunit.client.1.smithi178.stdout:6/772: write d3/d74/d91/daa/dc2/d57/d78/d7c/fd2 [2061124,102587] 0 2024-09-10T11:29:36.823 INFO:tasks.workunit.client.1.smithi178.stdout:9/802: link d2/d9b/lbf d2/d19/d1c/d77/lf6 0 2024-09-10T11:29:36.824 INFO:tasks.workunit.client.1.smithi178.stdout:2/927: dread d1/d7/d24/d43/f87 [0,4194304] 0 2024-09-10T11:29:36.825 INFO:tasks.workunit.client.1.smithi178.stdout:4/621: mknod d0/d8/cd8 0 2024-09-10T11:29:36.828 INFO:tasks.workunit.client.1.smithi178.stdout:7/666: getdents d6/d10/d1d/d22/d93 0 2024-09-10T11:29:36.828 INFO:tasks.workunit.client.0.smithi144.stdout:3/439: sync 2024-09-10T11:29:36.830 INFO:tasks.workunit.client.1.smithi178.stdout:3/810: link db/d35/d58/f6e db/d26/d9a/f107 0 2024-09-10T11:29:36.833 INFO:tasks.workunit.client.1.smithi178.stdout:9/803: creat d2/d51/ff7 x:0 0 0 2024-09-10T11:29:36.836 INFO:tasks.workunit.client.1.smithi178.stdout:9/804: chown d2/d51/f8f 452 1 2024-09-10T11:29:36.837 INFO:tasks.workunit.client.1.smithi178.stdout:8/861: mkdir d5/db/d10/d20/d52/d5c/d11f 0 2024-09-10T11:29:36.837 INFO:tasks.workunit.client.1.smithi178.stdout:3/811: chown l6 1 1 2024-09-10T11:29:36.839 INFO:tasks.workunit.client.1.smithi178.stdout:4/622: dwrite d0/d8/da/d1a/f4b [0,4194304] 0 2024-09-10T11:29:36.839 INFO:tasks.workunit.client.1.smithi178.stdout:2/928: fdatasync d1/d7/f103 0 2024-09-10T11:29:36.840 INFO:tasks.workunit.client.1.smithi178.stdout:7/667: mknod d6/d10/d1d/de6/cf6 0 2024-09-10T11:29:36.841 INFO:tasks.workunit.client.1.smithi178.stdout:0/709: dread d0/dc/d39/d4c/f57 [4194304,4194304] 0 2024-09-10T11:29:36.842 INFO:tasks.workunit.client.1.smithi178.stdout:4/623: chown d0/d10/fa2 646299301 1 2024-09-10T11:29:36.843 INFO:tasks.workunit.client.1.smithi178.stdout:3/812: write db/d35/d3a/d42/d46/f4e [80154,79493] 0 2024-09-10T11:29:36.845 INFO:tasks.workunit.client.1.smithi178.stdout:7/668: write d6/d10/d1d/d22/fbb [397964,43601] 0 2024-09-10T11:29:36.845 INFO:tasks.workunit.client.1.smithi178.stdout:9/805: rename d2/d19/d1c to d2/d9b/df8 0 2024-09-10T11:29:36.847 INFO:tasks.workunit.client.1.smithi178.stdout:7/669: rename d6 to d6/d10/d34/d6a/d6f/db1/df7 22 2024-09-10T11:29:36.847 INFO:tasks.workunit.client.1.smithi178.stdout:5/847: dread d9/dc/dd/d16/f27 [0,4194304] 0 2024-09-10T11:29:36.848 INFO:tasks.workunit.client.1.smithi178.stdout:7/670: dread - d6/d10/d1e/d2d/d2b/fda zero size 2024-09-10T11:29:36.849 INFO:tasks.workunit.client.0.smithi144.stdout:4/375: dwrite d14/d1e/f2f [0,4194304] 0 2024-09-10T11:29:36.849 INFO:tasks.workunit.client.0.smithi144.stdout:3/440: dwrite d4/dd/d21/f83 [0,4194304] 0 2024-09-10T11:29:36.850 INFO:tasks.workunit.client.0.smithi144.stdout:3/441: readlink d4/d1a/l6c 0 2024-09-10T11:29:36.851 INFO:tasks.workunit.client.1.smithi178.stdout:2/929: unlink d1/d7/d8e/fa8 0 2024-09-10T11:29:36.852 INFO:tasks.workunit.client.1.smithi178.stdout:4/624: symlink d0/d8/da/d1a/ld9 0 2024-09-10T11:29:36.854 INFO:tasks.workunit.client.1.smithi178.stdout:9/806: creat d2/dc/ff9 x:0 0 0 2024-09-10T11:29:36.856 INFO:tasks.workunit.client.0.smithi144.stdout:4/376: dwrite f5 [0,4194304] 0 2024-09-10T11:29:36.857 INFO:tasks.workunit.client.0.smithi144.stdout:3/442: mkdir d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93/d98 0 2024-09-10T11:29:36.858 INFO:tasks.workunit.client.0.smithi144.stdout:3/443: stat d4/dd/d21/d2f/f8d 0 2024-09-10T11:29:36.860 INFO:tasks.workunit.client.0.smithi144.stdout:4/377: truncate d14/d1e/d22/d34/d3e/d40/d4b/f57 4573297 0 2024-09-10T11:29:36.860 INFO:tasks.workunit.client.1.smithi178.stdout:7/671: mknod d6/d10/d1d/dca/cf8 0 2024-09-10T11:29:36.862 INFO:tasks.workunit.client.0.smithi144.stdout:3/444: symlink d4/dd/d21/d2f/l99 0 2024-09-10T11:29:36.864 INFO:tasks.workunit.client.1.smithi178.stdout:2/930: mknod d1/d7/d24/d46/c136 0 2024-09-10T11:29:36.864 INFO:tasks.workunit.client.1.smithi178.stdout:3/813: getdents db/d26/d74/dd9 0 2024-09-10T11:29:36.864 INFO:tasks.workunit.client.1.smithi178.stdout:5/848: creat d9/dc/dd/d3f/d5a/d10a/f118 x:0 0 0 2024-09-10T11:29:36.864 INFO:tasks.workunit.client.1.smithi178.stdout:8/862: link d5/d30/d33/f99 d5/db/d10/d35/d117/f120 0 2024-09-10T11:29:36.867 INFO:tasks.workunit.client.0.smithi144.stdout:4/378: dwrite d14/f54 [0,4194304] 0 2024-09-10T11:29:36.867 INFO:tasks.workunit.client.1.smithi178.stdout:4/625: dwrite d0/d8/da/f90 [0,4194304] 0 2024-09-10T11:29:36.869 INFO:tasks.workunit.client.0.smithi144.stdout:4/379: fsync d14/d1e/d22/f31 0 2024-09-10T11:29:36.870 INFO:tasks.workunit.client.1.smithi178.stdout:0/710: link d0/d12/d3f/c5c d0/dc/d1f/d2d/ce2 0 2024-09-10T11:29:36.870 INFO:tasks.workunit.client.1.smithi178.stdout:2/931: chown d1/d41/f44 70582 1 2024-09-10T11:29:36.871 INFO:tasks.workunit.client.1.smithi178.stdout:5/849: chown d9/dc/dd/d16/f70 59 1 2024-09-10T11:29:36.873 INFO:tasks.workunit.client.1.smithi178.stdout:0/711: chown d0/dc/d39/d75/fd4 105334 1 2024-09-10T11:29:36.874 INFO:tasks.workunit.client.0.smithi144.stdout:4/380: mkdir d14/d67/d84 0 2024-09-10T11:29:36.875 INFO:tasks.workunit.client.1.smithi178.stdout:7/672: write d6/d10/d1e/d2d/d2b/da0/dc0/fe8 [875029,62706] 0 2024-09-10T11:29:36.875 INFO:tasks.workunit.client.0.smithi144.stdout:3/445: dwrite d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/f74 [0,4194304] 0 2024-09-10T11:29:36.879 INFO:tasks.workunit.client.0.smithi144.stdout:3/446: unlink d4/d28/d67/d80/fc 0 2024-09-10T11:29:36.879 INFO:tasks.workunit.client.1.smithi178.stdout:4/626: symlink d0/d9/dc/d8e/lda 0 2024-09-10T11:29:36.880 INFO:tasks.workunit.client.1.smithi178.stdout:4/627: chown d0/d8/d3e/f4e 1 1 2024-09-10T11:29:36.881 INFO:tasks.workunit.client.1.smithi178.stdout:8/863: mkdir d5/db/d10/d70/d121 0 2024-09-10T11:29:36.882 INFO:tasks.workunit.client.0.smithi144.stdout:4/381: mkdir d14/d67/d69/d85 0 2024-09-10T11:29:36.882 INFO:tasks.workunit.client.1.smithi178.stdout:4/628: write d0/d9/dc/d8e/f9f [581123,125721] 0 2024-09-10T11:29:36.883 INFO:tasks.workunit.client.1.smithi178.stdout:3/814: mkdir db/d20/d3e/d6b/dc0/de5/d108 0 2024-09-10T11:29:36.883 INFO:tasks.workunit.client.1.smithi178.stdout:4/629: fdatasync d0/d9d/da5/fb8 0 2024-09-10T11:29:36.885 INFO:tasks.workunit.client.1.smithi178.stdout:3/815: dread - db/d20/d3d/f85 zero size 2024-09-10T11:29:36.888 INFO:tasks.workunit.client.1.smithi178.stdout:2/932: dread - d1/d7/d24/d46/d88/f95 zero size 2024-09-10T11:29:36.889 INFO:tasks.workunit.client.1.smithi178.stdout:5/850: truncate d9/dc/dd/d3f/d5a/dbc/fca 313348 0 2024-09-10T11:29:36.889 INFO:tasks.workunit.client.0.smithi144.stdout:4/382: mknod d14/c86 0 2024-09-10T11:29:36.891 INFO:tasks.workunit.client.0.smithi144.stdout:4/383: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/f58 [1042293,95781] 0 2024-09-10T11:29:36.892 INFO:tasks.workunit.client.1.smithi178.stdout:5/851: fdatasync d9/dc/d17/f3d 0 2024-09-10T11:29:36.893 INFO:tasks.workunit.client.1.smithi178.stdout:4/630: dwrite d0/d8/da/f6b [0,4194304] 0 2024-09-10T11:29:36.899 INFO:tasks.workunit.client.0.smithi144.stdout:4/384: rename d14/d1e/d22/d2b/d35/f72 to d14/d67/d69/d85/f87 0 2024-09-10T11:29:36.902 INFO:tasks.workunit.client.0.smithi144.stdout:4/385: write d14/d1e/d22/d34/d3e/d40/d4b/f80 [839098,4919] 0 2024-09-10T11:29:36.902 INFO:tasks.workunit.client.1.smithi178.stdout:9/807: getdents d2/d13/d40/d89/d93 0 2024-09-10T11:29:36.902 INFO:tasks.workunit.client.1.smithi178.stdout:7/673: rmdir d6/d10/d1d/d22/d93/db7 39 2024-09-10T11:29:36.904 INFO:tasks.workunit.client.1.smithi178.stdout:3/816: rename db/l24 to db/d35/d3a/d9f/l109 0 2024-09-10T11:29:36.904 INFO:tasks.workunit.client.1.smithi178.stdout:2/933: creat d1/d7/d24/d46/d88/d9d/f137 x:0 0 0 2024-09-10T11:29:36.905 INFO:tasks.workunit.client.1.smithi178.stdout:5/852: fdatasync d9/dc/dd/d37/dee/f112 0 2024-09-10T11:29:36.906 INFO:tasks.workunit.client.1.smithi178.stdout:7/674: read d6/d10/d1d/d22/f3a [2293261,63399] 0 2024-09-10T11:29:36.906 INFO:tasks.workunit.client.0.smithi144.stdout:4/386: truncate d14/d1e/d22/f31 3337309 0 2024-09-10T11:29:36.908 INFO:tasks.workunit.client.1.smithi178.stdout:9/808: symlink d2/d19/d5e/lfa 0 2024-09-10T11:29:36.909 INFO:tasks.workunit.client.1.smithi178.stdout:2/934: unlink d1/d7/l9c 0 2024-09-10T11:29:36.911 INFO:tasks.workunit.client.1.smithi178.stdout:2/935: chown d1/d41/d14/d11/f9 108799 1 2024-09-10T11:29:36.913 INFO:tasks.workunit.client.0.smithi144.stdout:3/447: dread d4/d28/f38 [0,4194304] 0 2024-09-10T11:29:36.914 INFO:tasks.workunit.client.0.smithi144.stdout:3/448: stat d4/d28/d67/d80/d17/c72 0 2024-09-10T11:29:36.914 INFO:tasks.workunit.client.1.smithi178.stdout:5/853: rename d9/dc/dd/d37/d3e/c7e to d9/dc/dd/d37/d42/d62/daf/df0/dc3/d105/c119 0 2024-09-10T11:29:36.915 INFO:tasks.workunit.client.0.smithi144.stdout:3/449: stat d4/dd/d21/d2f/f8d 0 2024-09-10T11:29:36.917 INFO:tasks.workunit.client.1.smithi178.stdout:9/809: chown d2/d9b/df8/d2a/f92 155092959 1 2024-09-10T11:29:36.918 INFO:tasks.workunit.client.1.smithi178.stdout:9/810: readlink d2/d13/l82 0 2024-09-10T11:29:36.919 INFO:tasks.workunit.client.0.smithi144.stdout:3/450: mkdir d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93/d98/d9a 0 2024-09-10T11:29:36.919 INFO:tasks.workunit.client.0.smithi144.stdout:3/451: dread - d4/dd/d21/f8b zero size 2024-09-10T11:29:36.919 INFO:tasks.workunit.client.1.smithi178.stdout:0/712: dread d0/d2c/f65 [0,4194304] 0 2024-09-10T11:29:36.920 INFO:tasks.workunit.client.1.smithi178.stdout:7/675: dwrite d6/fb [0,4194304] 0 2024-09-10T11:29:36.923 INFO:tasks.workunit.client.1.smithi178.stdout:7/676: stat d6/d10/d1d/d22/d50/d7e/lcb 0 2024-09-10T11:29:36.924 INFO:tasks.workunit.client.1.smithi178.stdout:0/713: fdatasync d0/d2c/d33/f81 0 2024-09-10T11:29:36.924 INFO:tasks.workunit.client.1.smithi178.stdout:5/854: truncate d9/dc/dd/f69 2582697 0 2024-09-10T11:29:36.926 INFO:tasks.workunit.client.1.smithi178.stdout:5/855: dread - d9/dc/dd/d37/d42/d55/f7a zero size 2024-09-10T11:29:36.927 INFO:tasks.workunit.client.0.smithi144.stdout:4/387: dread d14/d1e/d22/d2b/d35/f38 [0,4194304] 0 2024-09-10T11:29:36.928 INFO:tasks.workunit.client.0.smithi144.stdout:4/388: write d14/d67/d69/f6c [4493652,18616] 0 2024-09-10T11:29:36.929 INFO:tasks.workunit.client.0.smithi144.stdout:3/452: dread d4/d28/d67/d80/d17/d18/f2e [0,4194304] 0 2024-09-10T11:29:36.930 INFO:tasks.workunit.client.0.smithi144.stdout:3/453: read d4/d28/d67/d80/d1d/f97 [3430323,7683] 0 2024-09-10T11:29:36.931 INFO:tasks.workunit.client.1.smithi178.stdout:5/856: unlink d9/dc/dd/d37/d42/d55/d6d/fe1 0 2024-09-10T11:29:36.931 INFO:tasks.workunit.client.0.smithi144.stdout:4/389: write d14/d1e/d22/f4d [698484,57830] 0 2024-09-10T11:29:36.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:36 smithi144 ceph-mon[27942]: pgmap v25: 65 pgs: 65 active+clean; 2.0 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 65 MiB/s rd, 242 MiB/s wr, 408 op/s 2024-09-10T11:29:36.933 INFO:tasks.workunit.client.0.smithi144.stdout:3/454: creat d4/f9b x:0 0 0 2024-09-10T11:29:36.937 INFO:tasks.workunit.client.1.smithi178.stdout:3/817: dread db/d20/d3d/d56/f66 [0,4194304] 0 2024-09-10T11:29:36.939 INFO:tasks.workunit.client.1.smithi178.stdout:3/818: write db/d20/f3c [422722,34176] 0 2024-09-10T11:29:36.940 INFO:tasks.workunit.client.1.smithi178.stdout:5/857: dwrite d9/dc/dd/d3f/ff5 [0,4194304] 0 2024-09-10T11:29:36.942 INFO:tasks.workunit.client.1.smithi178.stdout:3/819: creat db/d20/f10a x:0 0 0 2024-09-10T11:29:36.945 INFO:tasks.workunit.client.1.smithi178.stdout:5/858: creat d9/dc/dd/d37/d42/d55/d6d/dcc/dd8/d103/f11a x:0 0 0 2024-09-10T11:29:36.949 INFO:tasks.workunit.client.1.smithi178.stdout:5/859: rename d9/ca3 to d9/dc/d17/c11b 0 2024-09-10T11:29:36.950 INFO:tasks.workunit.client.1.smithi178.stdout:5/860: dread - d9/dc/dd/d16/f41 zero size 2024-09-10T11:29:36.951 INFO:tasks.workunit.client.1.smithi178.stdout:5/861: chown d9/dc/dd/d3f/d5a/l81 58 1 2024-09-10T11:29:36.953 INFO:tasks.workunit.client.1.smithi178.stdout:5/862: unlink d9/dc/d17/d1d/f8c 0 2024-09-10T11:29:36.954 INFO:tasks.workunit.client.1.smithi178.stdout:3/820: dread db/d20/f95 [0,4194304] 0 2024-09-10T11:29:36.955 INFO:tasks.workunit.client.1.smithi178.stdout:5/863: symlink d9/dc/dd/d3f/d5a/dbc/l11c 0 2024-09-10T11:29:36.958 INFO:tasks.workunit.client.1.smithi178.stdout:3/821: dread db/d20/d3e/d6b/dfa/f84 [0,4194304] 0 2024-09-10T11:29:36.958 INFO:tasks.workunit.client.1.smithi178.stdout:5/864: unlink d9/dc/dd/d16/l53 0 2024-09-10T11:29:36.959 INFO:tasks.workunit.client.1.smithi178.stdout:3/822: symlink db/d26/d74/l10b 0 2024-09-10T11:29:36.960 INFO:tasks.workunit.client.1.smithi178.stdout:3/823: read - db/d26/d74/dd0/ff7 zero size 2024-09-10T11:29:36.960 INFO:tasks.workunit.client.1.smithi178.stdout:3/824: readlink l3 0 2024-09-10T11:29:36.960 INFO:tasks.workunit.client.1.smithi178.stdout:5/865: creat d9/dc/d17/f11d x:0 0 0 2024-09-10T11:29:36.961 INFO:tasks.workunit.client.1.smithi178.stdout:3/825: read db/d36/f50 [791815,108188] 0 2024-09-10T11:29:36.964 INFO:tasks.workunit.client.1.smithi178.stdout:3/826: mknod db/d20/d3e/c10c 0 2024-09-10T11:29:36.968 INFO:tasks.workunit.client.1.smithi178.stdout:5/866: dwrite d9/dc/dd/fc1 [0,4194304] 0 2024-09-10T11:29:36.971 INFO:tasks.workunit.client.1.smithi178.stdout:3/827: dread db/d35/d58/f6e [0,4194304] 0 2024-09-10T11:29:36.974 INFO:tasks.workunit.client.1.smithi178.stdout:6/773: sync 2024-09-10T11:29:36.974 INFO:tasks.workunit.client.1.smithi178.stdout:1/674: sync 2024-09-10T11:29:36.976 INFO:tasks.workunit.client.1.smithi178.stdout:1/675: fsync d3/d9/d98/dac/d3a/d6e/fe7 0 2024-09-10T11:29:36.977 INFO:tasks.workunit.client.1.smithi178.stdout:6/774: read - d3/d74/d91/daa/dc2/dcf/f109 zero size 2024-09-10T11:29:36.977 INFO:tasks.workunit.client.1.smithi178.stdout:3/828: mknod db/d26/d74/dd9/c10d 0 2024-09-10T11:29:36.978 INFO:tasks.workunit.client.1.smithi178.stdout:6/775: readlink d3/d74/lcd 0 2024-09-10T11:29:36.979 INFO:tasks.workunit.client.1.smithi178.stdout:5/867: link d9/dc/dd/d37/d42/d9f/ldc d9/dc/dd/d37/l11e 0 2024-09-10T11:29:36.982 INFO:tasks.workunit.client.1.smithi178.stdout:8/864: dwrite d5/db/f7d [0,4194304] 0 2024-09-10T11:29:36.985 INFO:tasks.workunit.client.0.smithi144.stdout:4/390: truncate d14/f3a 6306923 0 2024-09-10T11:29:36.986 INFO:tasks.workunit.client.1.smithi178.stdout:3/829: symlink db/d20/d3e/d6b/dfa/l10e 0 2024-09-10T11:29:36.988 INFO:tasks.workunit.client.1.smithi178.stdout:4/631: truncate d0/d9/dc/d8e/f9f 81631 0 2024-09-10T11:29:36.988 INFO:tasks.workunit.client.1.smithi178.stdout:8/865: dread - d5/d30/d33/d8c/da7/fe6 zero size 2024-09-10T11:29:36.990 INFO:tasks.workunit.client.1.smithi178.stdout:4/632: readlink d0/d8/da/d1a/ld9 0 2024-09-10T11:29:36.990 INFO:tasks.workunit.client.1.smithi178.stdout:8/866: readlink d5/db/d10/d35/l42 0 2024-09-10T11:29:36.990 INFO:tasks.workunit.client.1.smithi178.stdout:5/868: dwrite d9/dc/ff4 [0,4194304] 0 2024-09-10T11:29:36.991 INFO:tasks.workunit.client.1.smithi178.stdout:8/867: stat d5/db/d10/d20/f28 0 2024-09-10T11:29:36.991 INFO:tasks.workunit.client.1.smithi178.stdout:4/633: stat d0/d9/dc/d8e/lbe 0 2024-09-10T11:29:36.992 INFO:tasks.workunit.client.0.smithi144.stdout:3/455: creat d4/d28/d67/d80/d17/d18/d34/d4b/f9c x:0 0 0 2024-09-10T11:29:36.997 INFO:tasks.workunit.client.0.smithi144.stdout:3/456: rename d4/dd/d21/l3c to d4/dd/d21/d2f/l9d 0 2024-09-10T11:29:36.997 INFO:tasks.workunit.client.1.smithi178.stdout:6/776: rename d3/d7/d8/d19/l8d to d3/l11a 0 2024-09-10T11:29:36.997 INFO:tasks.workunit.client.1.smithi178.stdout:2/936: write d1/d7/f103 [4205100,50094] 0 2024-09-10T11:29:36.998 INFO:tasks.workunit.client.1.smithi178.stdout:8/868: symlink d5/d1d/l122 0 2024-09-10T11:29:37.000 INFO:tasks.workunit.client.1.smithi178.stdout:4/634: mkdir d0/d9/d3d/d4f/ddb 0 2024-09-10T11:29:37.002 INFO:tasks.workunit.client.1.smithi178.stdout:5/869: mknod d9/dc/dd/d37/d42/c11f 0 2024-09-10T11:29:37.002 INFO:tasks.workunit.client.1.smithi178.stdout:7/677: write f3 [4310691,36852] 0 2024-09-10T11:29:37.004 INFO:tasks.workunit.client.1.smithi178.stdout:6/777: write d3/d7/d8/fab [214545,111723] 0 2024-09-10T11:29:37.004 INFO:tasks.workunit.client.1.smithi178.stdout:4/635: dread - d0/d8/da/d1a/d51/fd7 zero size 2024-09-10T11:29:37.007 INFO:tasks.workunit.client.1.smithi178.stdout:9/811: dwrite d2/dc/f2f [0,4194304] 0 2024-09-10T11:29:37.008 INFO:tasks.workunit.client.1.smithi178.stdout:8/869: mkdir d5/d123 0 2024-09-10T11:29:37.009 INFO:tasks.workunit.client.0.smithi144.stdout:4/391: dread d14/d1e/d22/f48 [0,4194304] 0 2024-09-10T11:29:37.013 INFO:tasks.workunit.client.1.smithi178.stdout:5/870: chown d9/dc/dd/d37/d42/d55/d6d/dcc/fd2 1493298 1 2024-09-10T11:29:37.014 INFO:tasks.workunit.client.1.smithi178.stdout:7/678: fdatasync d6/d10/d1d/d22/d50/fe3 0 2024-09-10T11:29:37.014 INFO:tasks.workunit.client.1.smithi178.stdout:4/636: write d0/d9/dc/d8e/dbf/f37 [2479473,17005] 0 2024-09-10T11:29:37.014 INFO:tasks.workunit.client.0.smithi144.stdout:4/392: write d14/d1e/d22/d34/d3e/d40/d4b/f6a [23375,109438] 0 2024-09-10T11:29:37.014 INFO:tasks.workunit.client.0.smithi144.stdout:3/457: truncate d4/d28/f38 2017924 0 2024-09-10T11:29:37.015 INFO:tasks.workunit.client.1.smithi178.stdout:0/714: dwrite d0/dc/d39/d4c/f77 [0,4194304] 0 2024-09-10T11:29:37.018 INFO:tasks.workunit.client.0.smithi144.stdout:4/393: dwrite d14/d1e/d22/d34/d3e/d40/f78 [0,4194304] 0 2024-09-10T11:29:37.019 INFO:tasks.workunit.client.1.smithi178.stdout:9/812: dread - d2/d19/d5e/f9e zero size 2024-09-10T11:29:37.019 INFO:tasks.workunit.client.1.smithi178.stdout:3/830: write db/d20/f28 [1115114,130115] 0 2024-09-10T11:29:37.020 INFO:tasks.workunit.client.0.smithi144.stdout:0/272: sync 2024-09-10T11:29:37.020 INFO:tasks.workunit.client.0.smithi144.stdout:7/342: sync 2024-09-10T11:29:37.020 INFO:tasks.workunit.client.0.smithi144.stdout:3/458: getdents d4/dd 0 2024-09-10T11:29:37.021 INFO:tasks.workunit.client.0.smithi144.stdout:0/273: chown da/d48/f4a 48957 1 2024-09-10T11:29:37.023 INFO:tasks.workunit.client.1.smithi178.stdout:7/679: truncate d6/d10/d1e/d2d/d2b/fc1 124625 0 2024-09-10T11:29:37.023 INFO:tasks.workunit.client.0.smithi144.stdout:0/274: readlink da/l20 0 2024-09-10T11:29:37.027 INFO:tasks.workunit.client.1.smithi178.stdout:1/676: dwrite d3/d9/d21/d4b/d2b/f94 [0,4194304] 0 2024-09-10T11:29:37.027 INFO:tasks.workunit.client.1.smithi178.stdout:3/831: mknod db/d20/c10f 0 2024-09-10T11:29:37.030 INFO:tasks.workunit.client.1.smithi178.stdout:2/937: link d1/d7/f6c d1/d7/d24/d46/d88/d9d/dab/d5b/d60/f138 0 2024-09-10T11:29:37.030 INFO:tasks.workunit.client.1.smithi178.stdout:6/778: mkdir d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d11b 0 2024-09-10T11:29:37.034 INFO:tasks.workunit.client.1.smithi178.stdout:8/870: creat d5/db/d3d/de7/f124 x:0 0 0 2024-09-10T11:29:37.034 INFO:tasks.workunit.client.0.smithi144.stdout:7/343: dread f1 [0,4194304] 0 2024-09-10T11:29:37.036 INFO:tasks.workunit.client.0.smithi144.stdout:4/394: truncate d14/d1e/d22/d34/d3e/d40/d4b/d4c/f58 467993 0 2024-09-10T11:29:37.037 INFO:tasks.workunit.client.1.smithi178.stdout:4/637: creat d0/fdc x:0 0 0 2024-09-10T11:29:37.038 INFO:tasks.workunit.client.1.smithi178.stdout:3/832: rename db/d26/d40/d68 to db/d35/d3a/d42/d46/d110 0 2024-09-10T11:29:37.039 INFO:tasks.workunit.client.1.smithi178.stdout:5/871: write d9/dc/dd/d16/f27 [4535133,1386] 0 2024-09-10T11:29:37.039 INFO:tasks.workunit.client.0.smithi144.stdout:7/344: symlink d5/dc/d14/d40/d58/d6a/l6d 0 2024-09-10T11:29:37.041 INFO:tasks.workunit.client.1.smithi178.stdout:4/638: fsync d0/d8/da/d1a/d51/d5d/fbb 0 2024-09-10T11:29:37.042 INFO:tasks.workunit.client.1.smithi178.stdout:0/715: symlink d0/dc/d39/d75/d94/le3 0 2024-09-10T11:29:37.042 INFO:tasks.workunit.client.0.smithi144.stdout:7/345: mkdir d5/dc/d14/d40/d58/d6e 0 2024-09-10T11:29:37.043 INFO:tasks.workunit.client.1.smithi178.stdout:7/680: mknod d6/d10/d1d/d4a/cf9 0 2024-09-10T11:29:37.044 INFO:tasks.workunit.client.1.smithi178.stdout:8/871: creat d5/db/d10/d70/dc4/f125 x:0 0 0 2024-09-10T11:29:37.045 INFO:tasks.workunit.client.0.smithi144.stdout:7/346: creat d5/dc/d14/d40/d5e/f6f x:0 0 0 2024-09-10T11:29:37.045 INFO:tasks.workunit.client.1.smithi178.stdout:2/938: rename d1/d7/d24/d46/d88/d9d/dab/d5b/f106 to d1/d7/d8e/f139 0 2024-09-10T11:29:37.046 INFO:tasks.workunit.client.0.smithi144.stdout:7/347: truncate d5/fb 3658494 0 2024-09-10T11:29:37.048 INFO:tasks.workunit.client.1.smithi178.stdout:2/939: chown d1/feb 240187369 1 2024-09-10T11:29:37.049 INFO:tasks.workunit.client.0.smithi144.stdout:7/348: truncate d5/dc/d11/f4c 95061 0 2024-09-10T11:29:37.052 INFO:tasks.workunit.client.1.smithi178.stdout:0/716: read d0/dc/dc4/f5f [198601,52995] 0 2024-09-10T11:29:37.053 INFO:tasks.workunit.client.1.smithi178.stdout:4/639: dwrite d0/d9/d3d/d4f/f97 [0,4194304] 0 2024-09-10T11:29:37.053 INFO:tasks.workunit.client.1.smithi178.stdout:7/681: mknod d6/d10/d1d/dca/cfa 0 2024-09-10T11:29:37.054 INFO:tasks.workunit.client.1.smithi178.stdout:1/677: creat d3/d9/d98/dac/d3a/d50/ff9 x:0 0 0 2024-09-10T11:29:37.054 INFO:tasks.workunit.client.0.smithi144.stdout:7/349: mknod d5/dc/d12/d2f/c70 0 2024-09-10T11:29:37.055 INFO:tasks.workunit.client.1.smithi178.stdout:6/779: rename d3/d7/d8/d15/d1c/d2a/d18/d8e/ff5 to d3/d74/d91/daa/dc2/f11c 0 2024-09-10T11:29:37.057 INFO:tasks.workunit.client.1.smithi178.stdout:8/872: dwrite d5/d30/d33/d3a/f11c [0,4194304] 0 2024-09-10T11:29:37.057 INFO:tasks.workunit.client.1.smithi178.stdout:5/872: mkdir d9/dc/dd/d16/d120 0 2024-09-10T11:29:37.058 INFO:tasks.workunit.client.0.smithi144.stdout:7/350: symlink d5/dc/d14/d40/d58/l71 0 2024-09-10T11:29:37.058 INFO:tasks.workunit.client.0.smithi144.stdout:4/395: dwrite d14/f54 [4194304,4194304] 0 2024-09-10T11:29:37.063 INFO:tasks.workunit.client.0.smithi144.stdout:4/396: creat d14/d6f/f88 x:0 0 0 2024-09-10T11:29:37.064 INFO:tasks.workunit.client.1.smithi178.stdout:9/813: dread d2/df/f28 [0,4194304] 0 2024-09-10T11:29:37.065 INFO:tasks.workunit.client.0.smithi144.stdout:7/351: creat d5/dc/d14/d40/f72 x:0 0 0 2024-09-10T11:29:37.066 INFO:tasks.workunit.client.0.smithi144.stdout:7/352: chown d5/f7 2025 1 2024-09-10T11:29:37.066 INFO:tasks.workunit.client.0.smithi144.stdout:7/353: readlink d5/l6 0 2024-09-10T11:29:37.066 INFO:tasks.workunit.client.1.smithi178.stdout:2/940: symlink d1/d41/d14/d11/db9/def/l13a 0 2024-09-10T11:29:37.068 INFO:tasks.workunit.client.0.smithi144.stdout:4/397: rmdir d14/d1e/d22/d34/d3e/d40/d4b 39 2024-09-10T11:29:37.068 INFO:tasks.workunit.client.0.smithi144.stdout:4/398: chown d14/d1e/d22/d34/d3e 46124031 1 2024-09-10T11:29:37.069 INFO:tasks.workunit.client.1.smithi178.stdout:3/833: write db/d20/d3d/d63/f101 [678498,124853] 0 2024-09-10T11:29:37.071 INFO:tasks.workunit.client.0.smithi144.stdout:7/354: mknod d5/dc/d14/d40/d58/c73 0 2024-09-10T11:29:37.076 INFO:tasks.workunit.client.0.smithi144.stdout:7/355: creat d5/dc/d14/d40/d5e/d63/f74 x:0 0 0 2024-09-10T11:29:37.077 INFO:tasks.workunit.client.1.smithi178.stdout:6/780: rename d3/d7/d8/d56/de1/lef to d3/d74/d91/dee/l11d 0 2024-09-10T11:29:37.078 INFO:tasks.workunit.client.1.smithi178.stdout:4/640: rmdir d0/d9/dc/d8e/dbf 39 2024-09-10T11:29:37.080 INFO:tasks.workunit.client.1.smithi178.stdout:9/814: rmdir d2/dc 39 2024-09-10T11:29:37.080 INFO:tasks.workunit.client.1.smithi178.stdout:7/682: dwrite d6/d10/d1d/d22/d50/d7e/da7/feb [0,4194304] 0 2024-09-10T11:29:37.082 INFO:tasks.workunit.client.0.smithi144.stdout:7/356: creat d5/dc/d14/d40/d58/d6e/f75 x:0 0 0 2024-09-10T11:29:37.083 INFO:tasks.workunit.client.1.smithi178.stdout:3/834: read db/d20/d3e/d6b/dfa/f6c [3076888,34794] 0 2024-09-10T11:29:37.085 INFO:tasks.workunit.client.1.smithi178.stdout:1/678: mkdir d3/d9/d21/d4b/dfa 0 2024-09-10T11:29:37.085 INFO:tasks.workunit.client.0.smithi144.stdout:7/357: mknod d5/dc/d11/c76 0 2024-09-10T11:29:37.088 INFO:tasks.workunit.client.1.smithi178.stdout:8/873: dwrite d5/db/f88 [0,4194304] 0 2024-09-10T11:29:37.093 INFO:tasks.workunit.client.1.smithi178.stdout:1/679: truncate d3/d35/f74 1219517 0 2024-09-10T11:29:37.093 INFO:tasks.workunit.client.1.smithi178.stdout:5/873: dwrite d9/dc/dd/d3f/d5a/fd3 [0,4194304] 0 2024-09-10T11:29:37.094 INFO:tasks.workunit.client.0.smithi144.stdout:7/358: rename d5/dc/d11/f4c to d5/dc/d14/d40/d5e/d63/f77 0 2024-09-10T11:29:37.097 INFO:tasks.workunit.client.1.smithi178.stdout:1/680: fdatasync d3/d9/d21/fb6 0 2024-09-10T11:29:37.097 INFO:tasks.workunit.client.0.smithi144.stdout:7/359: mkdir d5/dc/d12/d2b/d78 0 2024-09-10T11:29:37.098 INFO:tasks.workunit.client.0.smithi144.stdout:4/399: dread d14/d1e/d22/f37 [4194304,4194304] 0 2024-09-10T11:29:37.103 INFO:tasks.workunit.client.1.smithi178.stdout:3/835: dwrite db/d26/ff9 [0,4194304] 0 2024-09-10T11:29:37.103 INFO:tasks.workunit.client.1.smithi178.stdout:7/683: truncate d6/d10/d1e/d2d/f21 2179394 0 2024-09-10T11:29:37.106 INFO:tasks.workunit.client.0.smithi144.stdout:7/360: symlink d5/dc/d2a/d60/l79 0 2024-09-10T11:29:37.107 INFO:tasks.workunit.client.1.smithi178.stdout:9/815: symlink d2/d9b/df8/d2a/lfb 0 2024-09-10T11:29:37.111 INFO:tasks.workunit.client.0.smithi144.stdout:7/361: dread d5/dc/d12/f22 [0,4194304] 0 2024-09-10T11:29:37.112 INFO:tasks.workunit.client.1.smithi178.stdout:0/717: dwrite d0/d12/d3f/d72/f82 [0,4194304] 0 2024-09-10T11:29:37.112 INFO:tasks.workunit.client.1.smithi178.stdout:8/874: rename d5/db/dda/dfa to d5/d1d/d22/d76/d126 0 2024-09-10T11:29:37.113 INFO:tasks.workunit.client.0.smithi144.stdout:7/362: truncate d5/dc/d14/d40/f67 869852 0 2024-09-10T11:29:37.114 INFO:tasks.workunit.client.1.smithi178.stdout:9/816: fdatasync d2/d8/fc5 0 2024-09-10T11:29:37.115 INFO:tasks.workunit.client.1.smithi178.stdout:7/684: dread - d6/d10/d1d/d22/d3b/fd9 zero size 2024-09-10T11:29:37.116 INFO:tasks.workunit.client.1.smithi178.stdout:4/641: dwrite d0/d9/d3d/f44 [0,4194304] 0 2024-09-10T11:29:37.116 INFO:tasks.workunit.client.0.smithi144.stdout:7/363: read d5/dc/f3c [1852229,21136] 0 2024-09-10T11:29:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:36 smithi178 ceph-mon[33514]: pgmap v25: 65 pgs: 65 active+clean; 2.0 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 65 MiB/s rd, 242 MiB/s wr, 408 op/s 2024-09-10T11:29:37.118 INFO:tasks.workunit.client.0.smithi144.stdout:7/364: chown d5/dc/d11/f3d 3128898 1 2024-09-10T11:29:37.118 INFO:tasks.workunit.client.1.smithi178.stdout:4/642: fsync d0/f6e 0 2024-09-10T11:29:37.119 INFO:tasks.workunit.client.1.smithi178.stdout:5/874: mknod d9/dc/dd/d37/d3e/c121 0 2024-09-10T11:29:37.122 INFO:tasks.workunit.client.1.smithi178.stdout:2/941: dread d1/d7/d24/d46/d88/d9d/dab/f15 [0,4194304] 0 2024-09-10T11:29:37.122 INFO:tasks.workunit.client.1.smithi178.stdout:4/643: fsync d0/d10/fa2 0 2024-09-10T11:29:37.122 INFO:tasks.workunit.client.1.smithi178.stdout:3/836: mknod db/d26/c111 0 2024-09-10T11:29:37.123 INFO:tasks.workunit.client.0.smithi144.stdout:7/365: mkdir d5/dc/d14/d40/d58/d6e/d7a 0 2024-09-10T11:29:37.125 INFO:tasks.workunit.client.1.smithi178.stdout:6/781: write d3/d7/d35/f6f [2555453,55075] 0 2024-09-10T11:29:37.125 INFO:tasks.workunit.client.1.smithi178.stdout:5/875: chown d9/dc/d17/db0/cf8 18 1 2024-09-10T11:29:37.126 INFO:tasks.workunit.client.1.smithi178.stdout:0/718: truncate d0/dc/d39/d4c/f77 4808515 0 2024-09-10T11:29:37.127 INFO:tasks.workunit.client.1.smithi178.stdout:0/719: readlink d0/dc/l10 0 2024-09-10T11:29:37.127 INFO:tasks.workunit.client.1.smithi178.stdout:3/837: fdatasync db/d20/d3d/d56/d64/fae 0 2024-09-10T11:29:37.130 INFO:tasks.workunit.client.0.smithi144.stdout:7/366: fdatasync d5/fb 0 2024-09-10T11:29:37.130 INFO:tasks.workunit.client.1.smithi178.stdout:7/685: truncate d6/d10/d1d/d22/d3b/fbe 532250 0 2024-09-10T11:29:37.130 INFO:tasks.workunit.client.1.smithi178.stdout:0/720: fdatasync d0/dc/d39/d7c/fa9 0 2024-09-10T11:29:37.131 INFO:tasks.workunit.client.0.smithi144.stdout:4/400: dwrite d14/d1e/d22/d2b/d35/f5b [0,4194304] 0 2024-09-10T11:29:37.131 INFO:tasks.workunit.client.1.smithi178.stdout:2/942: mknod d1/d7/d24/d46/d88/dc6/c13b 0 2024-09-10T11:29:37.131 INFO:tasks.workunit.client.0.smithi144.stdout:4/401: chown lf 564 1 2024-09-10T11:29:37.134 INFO:tasks.workunit.client.0.smithi144.stdout:4/402: mknod d14/d1e/d22/d2b/d35/c89 0 2024-09-10T11:29:37.134 INFO:tasks.workunit.client.1.smithi178.stdout:6/782: dread - d3/d74/d91/daa/dc2/d57/d78/d7c/f92 zero size 2024-09-10T11:29:37.135 INFO:tasks.workunit.client.0.smithi144.stdout:7/367: mkdir d5/dc/d7b 0 2024-09-10T11:29:37.135 INFO:tasks.workunit.client.0.smithi144.stdout:4/403: write d14/d1e/d22/d34/f3b [821838,121169] 0 2024-09-10T11:29:37.136 INFO:tasks.workunit.client.1.smithi178.stdout:8/875: link d5/d1d/d22/d76/f9e d5/d30/f127 0 2024-09-10T11:29:37.137 INFO:tasks.workunit.client.0.smithi144.stdout:4/404: write d14/d1e/f7c [4511730,2664] 0 2024-09-10T11:29:37.137 INFO:tasks.workunit.client.0.smithi144.stdout:7/368: stat d5/dc/d14/d40/d5e 0 2024-09-10T11:29:37.137 INFO:tasks.workunit.client.1.smithi178.stdout:0/721: chown d0/dc/d39/d50/d5e/l6a 800596218 1 2024-09-10T11:29:37.139 INFO:tasks.workunit.client.1.smithi178.stdout:4/644: dwrite d0/d8/d3e/f4c [4194304,4194304] 0 2024-09-10T11:29:37.140 INFO:tasks.workunit.client.1.smithi178.stdout:7/686: read d6/d10/d34/f74 [287076,57464] 0 2024-09-10T11:29:37.143 INFO:tasks.workunit.client.1.smithi178.stdout:1/681: write d3/d9/d98/dac/d3a/d50/d80/fef [5049850,103419] 0 2024-09-10T11:29:37.143 INFO:tasks.workunit.client.1.smithi178.stdout:2/943: write d1/d7/d24/d7d/d84/d4b/d89/fa3 [884850,85793] 0 2024-09-10T11:29:37.143 INFO:tasks.workunit.client.1.smithi178.stdout:3/838: creat db/d26/d74/dd0/f112 x:0 0 0 2024-09-10T11:29:37.144 INFO:tasks.workunit.client.0.smithi144.stdout:5/425: sync 2024-09-10T11:29:37.144 INFO:tasks.workunit.client.0.smithi144.stdout:6/353: sync 2024-09-10T11:29:37.144 INFO:tasks.workunit.client.0.smithi144.stdout:1/523: sync 2024-09-10T11:29:37.144 INFO:tasks.workunit.client.0.smithi144.stdout:1/524: readlink d5/df/l3a 0 2024-09-10T11:29:37.146 INFO:tasks.workunit.client.1.smithi178.stdout:6/783: rename d3/d74/dc9/d4b to d3/d74/dc9/da6/d11e 0 2024-09-10T11:29:37.146 INFO:tasks.workunit.client.1.smithi178.stdout:8/876: fsync d5/d30/db8/fe9 0 2024-09-10T11:29:37.147 INFO:tasks.workunit.client.0.smithi144.stdout:5/426: unlink d4/d5/l30 0 2024-09-10T11:29:37.149 INFO:tasks.workunit.client.0.smithi144.stdout:5/427: write d4/d5/d32/f68 [418360,119716] 0 2024-09-10T11:29:37.150 INFO:tasks.workunit.client.0.smithi144.stdout:5/428: stat d4/d5/d32/d47/f6a 0 2024-09-10T11:29:37.150 INFO:tasks.workunit.client.0.smithi144.stdout:6/354: rename d9/dd/d11/d1b/d62 to d9/dd/d3e/d83 0 2024-09-10T11:29:37.152 INFO:tasks.workunit.client.0.smithi144.stdout:1/525: mknod d5/cb0 0 2024-09-10T11:29:37.152 INFO:tasks.workunit.client.1.smithi178.stdout:7/687: dread d6/d10/d1e/d2d/d2b/f39 [0,4194304] 0 2024-09-10T11:29:37.152 INFO:tasks.workunit.client.0.smithi144.stdout:6/355: write d9/dd/d3e/f76 [4012595,54926] 0 2024-09-10T11:29:37.153 INFO:tasks.workunit.client.1.smithi178.stdout:1/682: symlink d3/d9/d98/dac/d3a/d50/d6c/d78/lfb 0 2024-09-10T11:29:37.156 INFO:tasks.workunit.client.1.smithi178.stdout:6/784: mknod d3/d7/d8/d15/d48/c11f 0 2024-09-10T11:29:37.157 INFO:tasks.workunit.client.0.smithi144.stdout:1/526: creat d5/d2a/d65/d73/fb1 x:0 0 0 2024-09-10T11:29:37.157 INFO:tasks.workunit.client.1.smithi178.stdout:5/876: link d9/dc/d17/c11b d9/dc/dd/d16/d120/c122 0 2024-09-10T11:29:37.158 INFO:tasks.workunit.client.0.smithi144.stdout:1/527: dread d5/dd/f48 [0,4194304] 0 2024-09-10T11:29:37.158 INFO:tasks.workunit.client.1.smithi178.stdout:4/645: rename d0/d8/df/l22 to d0/d9/dc/d9a/ldd 0 2024-09-10T11:29:37.159 INFO:tasks.workunit.client.1.smithi178.stdout:0/722: link d0/dc/d39/d50/d5e/l6a d0/dc/dc4/le4 0 2024-09-10T11:29:37.160 INFO:tasks.workunit.client.1.smithi178.stdout:0/723: fdatasync d0/dc/f21 0 2024-09-10T11:29:37.161 INFO:tasks.workunit.client.1.smithi178.stdout:6/785: write d3/d7/d8/d15/d1c/d2a/d18/ffd [377425,63937] 0 2024-09-10T11:29:37.162 INFO:tasks.workunit.client.0.smithi144.stdout:5/429: truncate d4/d5/f7d 967057 0 2024-09-10T11:29:37.162 INFO:tasks.workunit.client.1.smithi178.stdout:8/877: mknod d5/d30/d33/c128 0 2024-09-10T11:29:37.163 INFO:tasks.workunit.client.1.smithi178.stdout:6/786: chown d3/d7/d8/d56 0 1 2024-09-10T11:29:37.164 INFO:tasks.workunit.client.1.smithi178.stdout:3/839: dread db/ff [0,4194304] 0 2024-09-10T11:29:37.165 INFO:tasks.workunit.client.1.smithi178.stdout:0/724: fdatasync d0/dc/f2b 0 2024-09-10T11:29:37.166 INFO:tasks.workunit.client.0.smithi144.stdout:5/430: chown d4/d20/d24/d2d/f80 1 1 2024-09-10T11:29:37.166 INFO:tasks.workunit.client.0.smithi144.stdout:1/528: dread d5/d4a/f7d [0,4194304] 0 2024-09-10T11:29:37.166 INFO:tasks.workunit.client.0.smithi144.stdout:7/369: dwrite d5/dc/f10 [0,4194304] 0 2024-09-10T11:29:37.167 INFO:tasks.workunit.client.1.smithi178.stdout:5/877: symlink d9/dc/dd7/l123 0 2024-09-10T11:29:37.169 INFO:tasks.workunit.client.1.smithi178.stdout:4/646: mkdir d0/d8/df/dde 0 2024-09-10T11:29:37.169 INFO:tasks.workunit.client.0.smithi144.stdout:6/356: write d9/dd/d3e/d47/d13/d4a/d5b/f61 [346474,111398] 0 2024-09-10T11:29:37.170 INFO:tasks.workunit.client.0.smithi144.stdout:7/370: chown d5/dc/d14/d40/d58/d6e/d7a 0 1 2024-09-10T11:29:37.172 INFO:tasks.workunit.client.1.smithi178.stdout:6/787: dwrite d3/d7/d8/d15/d1c/d2a/d18/f7e [0,4194304] 0 2024-09-10T11:29:37.172 INFO:tasks.workunit.client.0.smithi144.stdout:4/405: dwrite d14/f15 [0,4194304] 0 2024-09-10T11:29:37.173 INFO:tasks.workunit.client.0.smithi144.stdout:5/431: fsync d4/fa 0 2024-09-10T11:29:37.179 INFO:tasks.workunit.client.0.smithi144.stdout:6/357: mknod d9/c84 0 2024-09-10T11:29:37.180 INFO:tasks.workunit.client.1.smithi178.stdout:8/878: truncate d5/db/d10/d20/f79 831001 0 2024-09-10T11:29:37.184 INFO:tasks.workunit.client.0.smithi144.stdout:7/371: dwrite d5/dc/d11/f26 [0,4194304] 0 2024-09-10T11:29:37.185 INFO:tasks.workunit.client.0.smithi144.stdout:4/406: dread d14/d1e/d22/d34/f75 [0,4194304] 0 2024-09-10T11:29:37.186 INFO:tasks.workunit.client.0.smithi144.stdout:1/529: dwrite d5/d4a/d44/d80/f76 [0,4194304] 0 2024-09-10T11:29:37.187 INFO:tasks.workunit.client.0.smithi144.stdout:4/407: chown fd 824957 1 2024-09-10T11:29:37.188 INFO:tasks.workunit.client.1.smithi178.stdout:1/683: getdents d3/d9/d98/dac/d3a/d50/d6c/d78 0 2024-09-10T11:29:37.192 INFO:tasks.workunit.client.1.smithi178.stdout:2/944: write d1/d7/d24/d7d/d84/d55/da1/d130/fa7 [588808,120244] 0 2024-09-10T11:29:37.192 INFO:tasks.workunit.client.0.smithi144.stdout:5/432: dread d4/d20/d24/f3e [0,4194304] 0 2024-09-10T11:29:37.195 INFO:tasks.workunit.client.1.smithi178.stdout:6/788: unlink d3/d74/dc9/da6/d11e/f7b 0 2024-09-10T11:29:37.195 INFO:tasks.workunit.client.0.smithi144.stdout:7/372: mkdir d5/dc/d12/d2f/d7c 0 2024-09-10T11:29:37.195 INFO:tasks.workunit.client.0.smithi144.stdout:4/408: rmdir d14/d1e/d22/d34/d3e 39 2024-09-10T11:29:37.200 INFO:tasks.workunit.client.0.smithi144.stdout:7/373: readlink d5/dc/d11/d18/l23 0 2024-09-10T11:29:37.202 INFO:tasks.workunit.client.0.smithi144.stdout:1/530: creat d5/d4a/d63/d84/dab/fb2 x:0 0 0 2024-09-10T11:29:37.202 INFO:tasks.workunit.client.0.smithi144.stdout:5/433: creat d4/d5/d18/d51/f88 x:0 0 0 2024-09-10T11:29:37.203 INFO:tasks.workunit.client.1.smithi178.stdout:0/725: creat d0/dc/d39/d7c/fe5 x:0 0 0 2024-09-10T11:29:37.203 INFO:tasks.workunit.client.1.smithi178.stdout:1/684: mknod d3/d9/d98/dac/d3a/d50/db5/dcc/cfc 0 2024-09-10T11:29:37.203 INFO:tasks.workunit.client.1.smithi178.stdout:5/878: dwrite d9/dc/dd/d37/d42/d62/f80 [0,4194304] 0 2024-09-10T11:29:37.204 INFO:tasks.workunit.client.0.smithi144.stdout:1/531: write d5/d2a/f4d [208857,39158] 0 2024-09-10T11:29:37.207 INFO:tasks.workunit.client.1.smithi178.stdout:6/789: creat d3/d74/d91/daa/dc2/d57/d78/d7c/f120 x:0 0 0 2024-09-10T11:29:37.208 INFO:tasks.workunit.client.1.smithi178.stdout:7/688: dread d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d70/f3c [0,4194304] 0 2024-09-10T11:29:37.209 INFO:tasks.workunit.client.0.smithi144.stdout:5/434: dwrite d4/f6b [0,4194304] 0 2024-09-10T11:29:37.211 INFO:tasks.workunit.client.0.smithi144.stdout:4/409: mkdir d14/d1e/d65/d8a 0 2024-09-10T11:29:37.211 INFO:tasks.workunit.client.0.smithi144.stdout:5/435: write d4/d20/d24/f46 [1899359,111789] 0 2024-09-10T11:29:37.211 INFO:tasks.workunit.client.1.smithi178.stdout:3/840: link db/d20/d3e/l6d db/d26/d40/l113 0 2024-09-10T11:29:37.216 INFO:tasks.workunit.client.0.smithi144.stdout:1/532: truncate d5/d2a/f96 4825492 0 2024-09-10T11:29:37.217 INFO:tasks.workunit.client.1.smithi178.stdout:2/945: rename d1/d7/d24/d46/ce0 to d1/d7/d24/d46/d88/d9d/dab/d5b/d60/c13c 0 2024-09-10T11:29:37.218 INFO:tasks.workunit.client.0.smithi144.stdout:1/533: readlink d5/d4a/d44/d80/l71 0 2024-09-10T11:29:37.218 INFO:tasks.workunit.client.0.smithi144.stdout:7/374: mknod d5/dc/d14/c7d 0 2024-09-10T11:29:37.220 INFO:tasks.workunit.client.0.smithi144.stdout:4/410: rename d14/d1e/d22/d2b/d35/d4f/c6e to d14/d67/d84/c8b 0 2024-09-10T11:29:37.223 INFO:tasks.workunit.client.1.smithi178.stdout:1/685: dwrite d3/d9/d98/dac/d3a/d50/ff9 [0,4194304] 0 2024-09-10T11:29:37.224 INFO:tasks.workunit.client.0.smithi144.stdout:5/436: creat d4/d20/d24/d6f/d84/f89 x:0 0 0 2024-09-10T11:29:37.224 INFO:tasks.workunit.client.1.smithi178.stdout:0/726: link d0/dc/d1f/d2d/fa8 d0/dc/fe6 0 2024-09-10T11:29:37.224 INFO:tasks.workunit.client.1.smithi178.stdout:8/879: write d5/db/d10/d35/f100 [175941,91916] 0 2024-09-10T11:29:37.225 INFO:tasks.workunit.client.0.smithi144.stdout:6/358: dread d9/f2a [0,4194304] 0 2024-09-10T11:29:37.226 INFO:tasks.workunit.client.0.smithi144.stdout:5/437: write d4/d20/d24/d6f/d84/f89 [63646,37278] 0 2024-09-10T11:29:37.227 INFO:tasks.workunit.client.1.smithi178.stdout:3/841: dread db/d20/d3e/d6b/dfa/f84 [0,4194304] 0 2024-09-10T11:29:37.227 INFO:tasks.workunit.client.0.smithi144.stdout:6/359: chown d9/d1f/d2c/l3d 12122 1 2024-09-10T11:29:37.228 INFO:tasks.workunit.client.1.smithi178.stdout:2/946: dread - d1/d22/d4d/f12a zero size 2024-09-10T11:29:37.229 INFO:tasks.workunit.client.0.smithi144.stdout:4/411: dread - d14/d1e/d22/d34/d3e/d40/d4b/d4c/f79 zero size 2024-09-10T11:29:37.231 INFO:tasks.workunit.client.0.smithi144.stdout:4/412: chown d14/d1e/d22/f31 4638 1 2024-09-10T11:29:37.231 INFO:tasks.workunit.client.0.smithi144.stdout:6/360: write d9/dd/d11/f4c [4610096,124299] 0 2024-09-10T11:29:37.231 INFO:tasks.workunit.client.0.smithi144.stdout:5/438: mkdir d4/d5/d56/d8a 0 2024-09-10T11:29:37.233 INFO:tasks.workunit.client.1.smithi178.stdout:5/879: rename d9/dc/dd/d37/d42/d55 to d9/dc/dd/d3f/dd6/d124 0 2024-09-10T11:29:37.233 INFO:tasks.workunit.client.1.smithi178.stdout:4/647: dread d0/d9/dc/f34 [0,4194304] 0 2024-09-10T11:29:37.233 INFO:tasks.workunit.client.0.smithi144.stdout:5/439: write d4/d20/d24/f83 [871328,44313] 0 2024-09-10T11:29:37.234 INFO:tasks.workunit.client.0.smithi144.stdout:5/440: chown d4/d5/d32/d47/f6a 49 1 2024-09-10T11:29:37.234 INFO:tasks.workunit.client.0.smithi144.stdout:5/441: chown d4/d5/d32/f68 1217734 1 2024-09-10T11:29:37.238 INFO:tasks.workunit.client.1.smithi178.stdout:8/880: rename d5/d30/d33/d8c/fcd to d5/d30/d33/d3a/f129 0 2024-09-10T11:29:37.239 INFO:tasks.workunit.client.0.smithi144.stdout:6/361: creat d9/d1f/d2c/f85 x:0 0 0 2024-09-10T11:29:37.239 INFO:tasks.workunit.client.1.smithi178.stdout:1/686: read d3/d9/d98/dac/d3a/d9f/fa2 [2605095,109566] 0 2024-09-10T11:29:37.240 INFO:tasks.workunit.client.0.smithi144.stdout:6/362: write d9/d1f/d2c/f39 [92835,29500] 0 2024-09-10T11:29:37.240 INFO:tasks.workunit.client.1.smithi178.stdout:4/648: fdatasync d0/f75 0 2024-09-10T11:29:37.241 INFO:tasks.workunit.client.0.smithi144.stdout:1/534: getdents d5/d4a/d63 0 2024-09-10T11:29:37.241 INFO:tasks.workunit.client.0.smithi144.stdout:1/535: stat d5/d4a/f7d 0 2024-09-10T11:29:37.242 INFO:tasks.workunit.client.1.smithi178.stdout:2/947: mkdir d1/d7/d24/d46/d88/d9d/dab/d5b/d60/dd7/d13d 0 2024-09-10T11:29:37.243 INFO:tasks.workunit.client.0.smithi144.stdout:5/442: mkdir d4/d5/d18/d38/d6d/d8b 0 2024-09-10T11:29:37.245 INFO:tasks.workunit.client.1.smithi178.stdout:7/689: link d6/l28 d6/d10/d1d/d22/d50/d98/ddc/lfb 0 2024-09-10T11:29:37.246 INFO:tasks.workunit.client.0.smithi144.stdout:6/363: mknod d9/dd/d11/d1b/d2f/c86 0 2024-09-10T11:29:37.246 INFO:tasks.workunit.client.1.smithi178.stdout:4/649: write d0/d8/df/f20 [1373484,73805] 0 2024-09-10T11:29:37.246 INFO:tasks.workunit.client.1.smithi178.stdout:5/880: dwrite d9/dc/dd/d37/dbd/fc6 [0,4194304] 0 2024-09-10T11:29:37.247 INFO:tasks.workunit.client.0.smithi144.stdout:4/413: link d14/d67/l7b d14/d1e/l8c 0 2024-09-10T11:29:37.247 INFO:tasks.workunit.client.0.smithi144.stdout:4/414: read d14/d1e/d22/f37 [6199032,58652] 0 2024-09-10T11:29:37.248 INFO:tasks.workunit.client.0.smithi144.stdout:5/443: mknod d4/d5/d18/d67/c8c 0 2024-09-10T11:29:37.250 INFO:tasks.workunit.client.1.smithi178.stdout:6/790: truncate d3/d7/d8/f58 3818287 0 2024-09-10T11:29:37.251 INFO:tasks.workunit.client.1.smithi178.stdout:3/842: creat db/d20/d3e/d6b/dfa/f114 x:0 0 0 2024-09-10T11:29:37.254 INFO:tasks.workunit.client.0.smithi144.stdout:4/415: dread d14/d1e/d22/d2b/d35/f5b [0,4194304] 0 2024-09-10T11:29:37.254 INFO:tasks.workunit.client.0.smithi144.stdout:5/444: mkdir d4/d5/d18/d38/d8d 0 2024-09-10T11:29:37.255 INFO:tasks.workunit.client.1.smithi178.stdout:0/727: mkdir d0/dc/d1f/d2d/dac/de7 0 2024-09-10T11:29:37.256 INFO:tasks.workunit.client.0.smithi144.stdout:6/364: truncate d9/dd/d3e/d47/d13/f31 3083938 0 2024-09-10T11:29:37.257 INFO:tasks.workunit.client.1.smithi178.stdout:7/690: dread d6/d10/d34/f44 [0,4194304] 0 2024-09-10T11:29:37.257 INFO:tasks.workunit.client.1.smithi178.stdout:3/843: write db/f39 [462088,31004] 0 2024-09-10T11:29:37.258 INFO:tasks.workunit.client.0.smithi144.stdout:6/365: write d9/d1f/d2c/f85 [501817,10696] 0 2024-09-10T11:29:37.259 INFO:tasks.workunit.client.0.smithi144.stdout:7/375: dwrite d5/dc/d11/d18/f34 [0,4194304] 0 2024-09-10T11:29:37.259 INFO:tasks.workunit.client.0.smithi144.stdout:4/416: symlink d14/l8d 0 2024-09-10T11:29:37.260 INFO:tasks.workunit.client.1.smithi178.stdout:2/948: dwrite d1/d41/fee [0,4194304] 0 2024-09-10T11:29:37.260 INFO:tasks.workunit.client.0.smithi144.stdout:6/366: fdatasync d9/dd/d3e/f4d 0 2024-09-10T11:29:37.261 INFO:tasks.workunit.client.0.smithi144.stdout:7/376: write d5/dc/d12/d2f/f4a [3213591,57472] 0 2024-09-10T11:29:37.263 INFO:tasks.workunit.client.1.smithi178.stdout:5/881: mknod d9/dc/dd/d3f/dd6/d124/d6d/dcc/de2/c125 0 2024-09-10T11:29:37.263 INFO:tasks.workunit.client.0.smithi144.stdout:5/445: dread d4/d5/d32/f68 [0,4194304] 0 2024-09-10T11:29:37.264 INFO:tasks.workunit.client.1.smithi178.stdout:0/728: read d0/dc/d39/d50/f1d [4059509,3601] 0 2024-09-10T11:29:37.265 INFO:tasks.workunit.client.1.smithi178.stdout:1/687: mkdir d3/d9/d21/d4b/d2b/dfd 0 2024-09-10T11:29:37.265 INFO:tasks.workunit.client.1.smithi178.stdout:2/949: stat d1/d7/d24/d46/d88/d9d/dab/d5e/f12f 0 2024-09-10T11:29:37.266 INFO:tasks.workunit.client.0.smithi144.stdout:4/417: symlink d14/l8e 0 2024-09-10T11:29:37.266 INFO:tasks.workunit.client.1.smithi178.stdout:6/791: dwrite d3/d24/f30 [8388608,4194304] 0 2024-09-10T11:29:37.268 INFO:tasks.workunit.client.0.smithi144.stdout:6/367: fsync d9/d1f/d2c/f53 0 2024-09-10T11:29:37.268 INFO:tasks.workunit.client.1.smithi178.stdout:4/650: mkdir d0/d9/d3d/d4f/ddf 0 2024-09-10T11:29:37.271 INFO:tasks.workunit.client.1.smithi178.stdout:8/881: link d5/d1d/f60 d5/d1d/d22/d76/d126/f12a 0 2024-09-10T11:29:37.274 INFO:tasks.workunit.client.0.smithi144.stdout:5/446: dread d4/d52/f69 [0,4194304] 0 2024-09-10T11:29:37.274 INFO:tasks.workunit.client.0.smithi144.stdout:6/368: unlink d9/dd/d3e/d47/lf 0 2024-09-10T11:29:37.276 INFO:tasks.workunit.client.1.smithi178.stdout:7/691: fsync d6/d10/d34/f9b 0 2024-09-10T11:29:37.276 INFO:tasks.workunit.client.0.smithi144.stdout:4/418: mkdir d14/d1e/d65/d8a/d8f 0 2024-09-10T11:29:37.277 INFO:tasks.workunit.client.1.smithi178.stdout:4/651: mkdir d0/d63/de0 0 2024-09-10T11:29:37.277 INFO:tasks.workunit.client.1.smithi178.stdout:6/792: unlink d3/d74/d91/daa/dc2/f11c 0 2024-09-10T11:29:37.278 INFO:tasks.workunit.client.0.smithi144.stdout:6/369: fsync d9/d1f/f26 0 2024-09-10T11:29:37.278 INFO:tasks.workunit.client.0.smithi144.stdout:4/419: write d14/f54 [7032201,102888] 0 2024-09-10T11:29:37.281 INFO:tasks.workunit.client.1.smithi178.stdout:3/844: mknod db/c115 0 2024-09-10T11:29:37.281 INFO:tasks.workunit.client.0.smithi144.stdout:5/447: unlink d4/l16 0 2024-09-10T11:29:37.283 INFO:tasks.workunit.client.0.smithi144.stdout:5/448: chown d4/d5/d32/d47/l79 57 1 2024-09-10T11:29:37.284 INFO:tasks.workunit.client.0.smithi144.stdout:6/370: write d9/f2a [3260581,52362] 0 2024-09-10T11:29:37.285 INFO:tasks.workunit.client.1.smithi178.stdout:6/793: unlink d3/d74/d91/daa/dc2/f61 0 2024-09-10T11:29:37.288 INFO:tasks.workunit.client.0.smithi144.stdout:6/371: write d9/dd/d3e/d47/d13/f1e [21339,53826] 0 2024-09-10T11:29:37.291 INFO:tasks.workunit.client.1.smithi178.stdout:1/688: creat d3/d9/d98/dac/d3a/d50/ffe x:0 0 0 2024-09-10T11:29:37.292 INFO:tasks.workunit.client.0.smithi144.stdout:5/449: dwrite d4/d20/d24/d2d/d4b/f4e [0,4194304] 0 2024-09-10T11:29:37.293 INFO:tasks.workunit.client.1.smithi178.stdout:7/692: creat d6/d10/d1d/d22/d50/d98/da6/dd5/ffc x:0 0 0 2024-09-10T11:29:37.294 INFO:tasks.workunit.client.1.smithi178.stdout:4/652: dwrite d0/d8/da/d1a/f69 [4194304,4194304] 0 2024-09-10T11:29:37.294 INFO:tasks.workunit.client.0.smithi144.stdout:6/372: mkdir d9/dd/d3e/d47/d13/d87 0 2024-09-10T11:29:37.294 INFO:tasks.workunit.client.0.smithi144.stdout:7/377: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:37.295 INFO:tasks.workunit.client.0.smithi144.stdout:5/450: write d4/d20/f34 [1803073,18097] 0 2024-09-10T11:29:37.296 INFO:tasks.workunit.client.1.smithi178.stdout:6/794: truncate d3/d74/dc9/f93 121114 0 2024-09-10T11:29:37.299 INFO:tasks.workunit.client.1.smithi178.stdout:2/950: rename d1/d41/d14/d11/db9/dc1/ld1 to d1/l13e 0 2024-09-10T11:29:37.300 INFO:tasks.workunit.client.0.smithi144.stdout:6/373: creat d9/dd/d3e/d47/f88 x:0 0 0 2024-09-10T11:29:37.302 INFO:tasks.workunit.client.0.smithi144.stdout:6/374: chown d9/dd/d11/d1b/d2f/d7f/d82/l7b 942825129 1 2024-09-10T11:29:37.303 INFO:tasks.workunit.client.0.smithi144.stdout:5/451: creat d4/d20/d22/d5c/d5d/d64/f8e x:0 0 0 2024-09-10T11:29:37.304 INFO:tasks.workunit.client.0.smithi144.stdout:6/375: read d9/d1f/d2c/f39 [685464,89585] 0 2024-09-10T11:29:37.306 INFO:tasks.workunit.client.1.smithi178.stdout:4/653: unlink d0/d10/f2c 0 2024-09-10T11:29:37.308 INFO:tasks.workunit.client.0.smithi144.stdout:5/452: unlink d4/d5/d32/c37 0 2024-09-10T11:29:37.309 INFO:tasks.workunit.client.0.smithi144.stdout:5/453: readlink d4/d5/d18/d67/l7a 0 2024-09-10T11:29:37.309 INFO:tasks.workunit.client.0.smithi144.stdout:6/376: mknod d9/dd/d11/d1b/d2f/d7f/c89 0 2024-09-10T11:29:37.310 INFO:tasks.workunit.client.0.smithi144.stdout:5/454: write d4/d20/d24/f46 [3057255,116840] 0 2024-09-10T11:29:37.310 INFO:tasks.workunit.client.1.smithi178.stdout:2/951: symlink d1/d7/d24/d43/l13f 0 2024-09-10T11:29:37.311 INFO:tasks.workunit.client.0.smithi144.stdout:6/377: chown d9/dd/d73/c78 509437885 1 2024-09-10T11:29:37.312 INFO:tasks.workunit.client.0.smithi144.stdout:5/455: fsync d4/d5/d18/d67/f62 0 2024-09-10T11:29:37.312 INFO:tasks.workunit.client.0.smithi144.stdout:5/456: readlink d4/d20/d22/l45 0 2024-09-10T11:29:37.313 INFO:tasks.workunit.client.1.smithi178.stdout:5/882: write d9/dc/dd/d37/d3e/f94 [319817,45603] 0 2024-09-10T11:29:37.314 INFO:tasks.workunit.client.1.smithi178.stdout:6/795: dread d3/d7/d8/d15/d1c/d2a/f5d [0,4194304] 0 2024-09-10T11:29:37.315 INFO:tasks.workunit.client.0.smithi144.stdout:5/457: fsync d4/d20/d24/d2d/f80 0 2024-09-10T11:29:37.317 INFO:tasks.workunit.client.0.smithi144.stdout:4/420: dread fa [0,4194304] 0 2024-09-10T11:29:37.317 INFO:tasks.workunit.client.1.smithi178.stdout:8/882: link d5/db/d10/d20/dc0/d109/c11b d5/d30/d33/c12b 0 2024-09-10T11:29:37.317 INFO:tasks.workunit.client.1.smithi178.stdout:8/883: chown d5/db/d10/d20/d52/d67/c7c 11 1 2024-09-10T11:29:37.319 INFO:tasks.workunit.client.0.smithi144.stdout:4/421: fsync fd 0 2024-09-10T11:29:37.320 INFO:tasks.workunit.client.0.smithi144.stdout:6/378: dwrite d9/dd/d3a/f51 [0,4194304] 0 2024-09-10T11:29:37.320 INFO:tasks.workunit.client.1.smithi178.stdout:5/883: creat d9/d19/f126 x:0 0 0 2024-09-10T11:29:37.321 INFO:tasks.workunit.client.0.smithi144.stdout:5/458: mkdir d4/d5/d18/d38/d6d/d8f 0 2024-09-10T11:29:37.322 INFO:tasks.workunit.client.0.smithi144.stdout:5/459: stat d4/d5/l27 0 2024-09-10T11:29:37.322 INFO:tasks.workunit.client.0.smithi144.stdout:6/379: read - d9/d1f/d2c/d4f/f7d zero size 2024-09-10T11:29:37.322 INFO:tasks.workunit.client.1.smithi178.stdout:6/796: rmdir d3/d74/d91/dee 39 2024-09-10T11:29:37.323 INFO:tasks.workunit.client.1.smithi178.stdout:0/729: truncate d0/dc/d1f/f26 3260049 0 2024-09-10T11:29:37.324 INFO:tasks.workunit.client.0.smithi144.stdout:6/380: read d9/d1f/f26 [9520,118292] 0 2024-09-10T11:29:37.329 INFO:tasks.workunit.client.1.smithi178.stdout:2/952: rename d1/d41/d5d/c70 to d1/d22/d4d/d78/c140 0 2024-09-10T11:29:37.332 INFO:tasks.workunit.client.0.smithi144.stdout:4/422: dwrite d14/d67/d69/d85/f87 [0,4194304] 0 2024-09-10T11:29:37.333 INFO:tasks.workunit.client.1.smithi178.stdout:1/689: dread d3/d35/f99 [0,4194304] 0 2024-09-10T11:29:37.334 INFO:tasks.workunit.client.1.smithi178.stdout:1/690: fsync d3/d35/fe6 0 2024-09-10T11:29:37.336 INFO:tasks.workunit.client.0.smithi144.stdout:6/381: creat d9/dd/d11/d1b/d2f/f8a x:0 0 0 2024-09-10T11:29:37.337 INFO:tasks.workunit.client.0.smithi144.stdout:4/423: getdents d14/d1e/d83 0 2024-09-10T11:29:37.338 INFO:tasks.workunit.client.1.smithi178.stdout:5/884: dread d9/dc/dd/d3f/dd6/d124/d6d/dcc/fd2 [0,4194304] 0 2024-09-10T11:29:37.339 INFO:tasks.workunit.client.0.smithi144.stdout:5/460: dwrite d4/d20/f43 [0,4194304] 0 2024-09-10T11:29:37.342 INFO:tasks.workunit.client.0.smithi144.stdout:4/424: mkdir d14/d67/d69/d90 0 2024-09-10T11:29:37.343 INFO:tasks.workunit.client.0.smithi144.stdout:4/425: chown d14/c25 55 1 2024-09-10T11:29:37.343 INFO:tasks.workunit.client.1.smithi178.stdout:8/884: dread d5/d30/d33/f49 [0,4194304] 0 2024-09-10T11:29:37.344 INFO:tasks.workunit.client.1.smithi178.stdout:4/654: write d0/d8/d3e/f85 [317065,27645] 0 2024-09-10T11:29:37.345 INFO:tasks.workunit.client.1.smithi178.stdout:6/797: getdents d3/d74/d91/daa/dc2/d57/d78 0 2024-09-10T11:29:37.345 INFO:tasks.workunit.client.1.smithi178.stdout:1/691: mknod d3/d9/d98/dac/d3a/d6e/d8a/cff 0 2024-09-10T11:29:37.346 INFO:tasks.workunit.client.1.smithi178.stdout:7/693: dwrite d6/f30 [0,4194304] 0 2024-09-10T11:29:37.347 INFO:tasks.workunit.client.1.smithi178.stdout:5/885: mkdir d9/dc/dd/d37/dbd/d127 0 2024-09-10T11:29:37.349 INFO:tasks.workunit.client.1.smithi178.stdout:5/886: dread - d9/dc/d17/f11d zero size 2024-09-10T11:29:37.350 INFO:tasks.workunit.client.1.smithi178.stdout:7/694: chown d6/d10/d1d/d22/d50/d7e/d82/db2/l47 5637739 1 2024-09-10T11:29:37.350 INFO:tasks.workunit.client.1.smithi178.stdout:1/692: truncate d3/d9/d98/dd1/ddc/f86 1112724 0 2024-09-10T11:29:37.352 INFO:tasks.workunit.client.0.smithi144.stdout:5/461: dwrite d4/d5/d32/f68 [0,4194304] 0 2024-09-10T11:29:37.353 INFO:tasks.workunit.client.0.smithi144.stdout:4/426: rename d14/d6d to d14/d1e/d22/d34/d91 0 2024-09-10T11:29:37.354 INFO:tasks.workunit.client.1.smithi178.stdout:1/693: dread - d3/d9/d98/dac/d3a/fc7 zero size 2024-09-10T11:29:37.354 INFO:tasks.workunit.client.0.smithi144.stdout:5/462: chown d4/d5/d32/d47 6638959 1 2024-09-10T11:29:37.357 INFO:tasks.workunit.client.0.smithi144.stdout:5/463: mknod d4/d20/d22/d5c/c90 0 2024-09-10T11:29:37.358 INFO:tasks.workunit.client.0.smithi144.stdout:4/427: mkdir d14/d67/d92 0 2024-09-10T11:29:37.358 INFO:tasks.workunit.client.0.smithi144.stdout:5/464: chown d4/d5/l27 48446944 1 2024-09-10T11:29:37.362 INFO:tasks.workunit.client.0.smithi144.stdout:5/465: fsync d4/d5/f25 0 2024-09-10T11:29:37.365 INFO:tasks.workunit.client.1.smithi178.stdout:0/730: dread d0/dc/d39/da4/dce/ddc/fdb [0,4194304] 0 2024-09-10T11:29:37.366 INFO:tasks.workunit.client.0.smithi144.stdout:4/428: dwrite d14/d1e/d22/d34/d3e/d40/f44 [0,4194304] 0 2024-09-10T11:29:37.366 INFO:tasks.workunit.client.1.smithi178.stdout:0/731: chown d0/d12/d3f/d72 711 1 2024-09-10T11:29:37.368 INFO:tasks.workunit.client.0.smithi144.stdout:5/466: link d4/d5/d32/d47/f6a d4/d5/d18/d38/d6d/d8b/f91 0 2024-09-10T11:29:37.368 INFO:tasks.workunit.client.0.smithi144.stdout:5/467: readlink d4/d20/d24/d2d/l57 0 2024-09-10T11:29:37.369 INFO:tasks.workunit.client.0.smithi144.stdout:4/429: rename d14/d1e/f45 to d14/d67/d69/d85/f93 0 2024-09-10T11:29:37.372 INFO:tasks.workunit.client.0.smithi144.stdout:4/430: mknod d14/d1e/d22/c94 0 2024-09-10T11:29:37.374 INFO:tasks.workunit.client.0.smithi144.stdout:5/468: getdents d4/d5/d18/d67/d44 0 2024-09-10T11:29:37.375 INFO:tasks.workunit.client.0.smithi144.stdout:4/431: mknod d14/d7e/c95 0 2024-09-10T11:29:37.378 INFO:tasks.workunit.client.0.smithi144.stdout:4/432: creat d14/d6f/f96 x:0 0 0 2024-09-10T11:29:37.380 INFO:tasks.workunit.client.0.smithi144.stdout:4/433: creat d14/d1e/d22/d34/d3e/f97 x:0 0 0 2024-09-10T11:29:37.381 INFO:tasks.workunit.client.0.smithi144.stdout:5/469: dwrite d4/d5/d18/f1f [0,4194304] 0 2024-09-10T11:29:37.385 INFO:tasks.workunit.client.0.smithi144.stdout:4/434: symlink d14/d67/d92/l98 0 2024-09-10T11:29:37.387 INFO:tasks.workunit.client.0.smithi144.stdout:4/435: unlink d14/l8e 0 2024-09-10T11:29:37.390 INFO:tasks.workunit.client.0.smithi144.stdout:5/470: dwrite d4/d5/d18/d67/f65 [0,4194304] 0 2024-09-10T11:29:37.390 INFO:tasks.workunit.client.0.smithi144.stdout:4/436: symlink d14/d1e/d65/l99 0 2024-09-10T11:29:37.393 INFO:tasks.workunit.client.0.smithi144.stdout:4/437: rename d14/l8d to d14/d7e/l9a 0 2024-09-10T11:29:37.393 INFO:tasks.workunit.client.0.smithi144.stdout:4/438: chown d14/d6f/f96 39443196 1 2024-09-10T11:29:37.395 INFO:tasks.workunit.client.0.smithi144.stdout:4/439: write d14/d1e/d22/f37 [9232779,78694] 0 2024-09-10T11:29:37.395 INFO:tasks.workunit.client.0.smithi144.stdout:4/440: chown d14/f15 44693 1 2024-09-10T11:29:37.397 INFO:tasks.workunit.client.0.smithi144.stdout:4/441: truncate d14/d1e/d22/d2b/d35/f77 1004031 0 2024-09-10T11:29:37.400 INFO:tasks.workunit.client.0.smithi144.stdout:4/442: symlink d14/d1e/d22/d2b/l9b 0 2024-09-10T11:29:37.401 INFO:tasks.workunit.client.0.smithi144.stdout:4/443: dread - d14/d1e/d22/d34/d3e/f97 zero size 2024-09-10T11:29:37.402 INFO:tasks.workunit.client.0.smithi144.stdout:5/471: dread d4/d5/f7d [0,4194304] 0 2024-09-10T11:29:37.404 INFO:tasks.workunit.client.0.smithi144.stdout:4/444: unlink d14/d1e/d22/d34/c61 0 2024-09-10T11:29:37.405 INFO:tasks.workunit.client.0.smithi144.stdout:4/445: fsync d14/d1e/d22/d2b/d35/f77 0 2024-09-10T11:29:37.410 INFO:tasks.workunit.client.0.smithi144.stdout:4/446: link d14/f3a d14/d6f/f9c 0 2024-09-10T11:29:37.411 INFO:tasks.workunit.client.0.smithi144.stdout:4/447: truncate d14/d1e/d22/f23 5231409 0 2024-09-10T11:29:37.417 INFO:tasks.workunit.client.0.smithi144.stdout:5/472: truncate d4/d20/d24/f83 428610 0 2024-09-10T11:29:37.422 INFO:tasks.workunit.client.0.smithi144.stdout:4/448: dwrite d14/d6f/f9c [0,4194304] 0 2024-09-10T11:29:37.423 INFO:tasks.workunit.client.0.smithi144.stdout:4/449: truncate d14/d67/d69/d85/f93 180544 0 2024-09-10T11:29:37.423 INFO:tasks.workunit.client.0.smithi144.stdout:5/473: dread d4/d5/d18/d67/d59/f5e [0,4194304] 0 2024-09-10T11:29:37.424 INFO:tasks.workunit.client.0.smithi144.stdout:5/474: stat d4/d20/d24/d2d/d4b/l4c 0 2024-09-10T11:29:37.427 INFO:tasks.workunit.client.0.smithi144.stdout:5/475: creat d4/d20/d24/f92 x:0 0 0 2024-09-10T11:29:37.427 INFO:tasks.workunit.client.0.smithi144.stdout:5/476: chown d4/d5/d18 1071441775 1 2024-09-10T11:29:37.428 INFO:tasks.workunit.client.0.smithi144.stdout:5/477: chown d4/d20/d22/c35 987092096 1 2024-09-10T11:29:37.431 INFO:tasks.workunit.client.0.smithi144.stdout:5/478: creat d4/d5/d18/d67/d44/f93 x:0 0 0 2024-09-10T11:29:37.431 INFO:tasks.workunit.client.1.smithi178.stdout:9/817: sync 2024-09-10T11:29:37.434 INFO:tasks.workunit.client.1.smithi178.stdout:6/798: unlink d3/d6e/fba 0 2024-09-10T11:29:37.434 INFO:tasks.workunit.client.0.smithi144.stdout:5/479: getdents d4/d5/d18/d38/d6d 0 2024-09-10T11:29:37.435 INFO:tasks.workunit.client.1.smithi178.stdout:4/655: mkdir d0/d9d/de1 0 2024-09-10T11:29:37.436 INFO:tasks.workunit.client.1.smithi178.stdout:8/885: chown d5/db/d10/d35/d117/l11a 10901489 1 2024-09-10T11:29:37.437 INFO:tasks.workunit.client.1.smithi178.stdout:7/695: chown d6/d10/d1e/d2d/d2b/da0/d55/dbd/c68 0 1 2024-09-10T11:29:37.438 INFO:tasks.workunit.client.1.smithi178.stdout:1/694: symlink d3/d9/d98/dac/d3a/d6e/l100 0 2024-09-10T11:29:37.438 INFO:tasks.workunit.client.0.smithi144.stdout:5/480: link d4/d20/l2f d4/d5/d18/d38/d6d/l94 0 2024-09-10T11:29:37.439 INFO:tasks.workunit.client.1.smithi178.stdout:0/732: mkdir d0/dc/dc4/de8 0 2024-09-10T11:29:37.441 INFO:tasks.workunit.client.0.smithi144.stdout:5/481: dread d4/d5/d18/d67/d59/f5e [0,4194304] 0 2024-09-10T11:29:37.442 INFO:tasks.workunit.client.1.smithi178.stdout:2/953: truncate d1/d7/d24/d7d/d84/f36 1648373 0 2024-09-10T11:29:37.442 INFO:tasks.workunit.client.1.smithi178.stdout:9/818: read d2/d9b/df8/da5/fd9 [717717,25148] 0 2024-09-10T11:29:37.443 INFO:tasks.workunit.client.0.smithi144.stdout:5/482: symlink d4/d5/d18/d38/l95 0 2024-09-10T11:29:37.443 INFO:tasks.workunit.client.1.smithi178.stdout:5/887: mkdir d9/dc/dd/d3f/d5a/d114/d128 0 2024-09-10T11:29:37.444 INFO:tasks.workunit.client.1.smithi178.stdout:5/888: readlink d9/dc/d17/db0/l10f 0 2024-09-10T11:29:37.444 INFO:tasks.workunit.client.1.smithi178.stdout:7/696: mkdir d6/d10/d1d/d22/d50/d98/da6/dfd 0 2024-09-10T11:29:37.445 INFO:tasks.workunit.client.0.smithi144.stdout:5/483: dread d4/d5/d18/d67/d59/f5e [0,4194304] 0 2024-09-10T11:29:37.445 INFO:tasks.workunit.client.1.smithi178.stdout:1/695: mknod d3/d9/d98/dac/d3a/d6e/c101 0 2024-09-10T11:29:37.447 INFO:tasks.workunit.client.1.smithi178.stdout:6/799: mkdir d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d11b/d121 0 2024-09-10T11:29:37.447 INFO:tasks.workunit.client.1.smithi178.stdout:4/656: mkdir d0/d9d/dd5/de2 0 2024-09-10T11:29:37.449 INFO:tasks.workunit.client.1.smithi178.stdout:6/800: chown d3/d4e 97860 1 2024-09-10T11:29:37.449 INFO:tasks.workunit.client.1.smithi178.stdout:2/954: mknod d1/d7/d24/d7d/d84/d55/da1/dd6/c141 0 2024-09-10T11:29:37.451 INFO:tasks.workunit.client.1.smithi178.stdout:1/696: symlink d3/d9/d98/l102 0 2024-09-10T11:29:37.452 INFO:tasks.workunit.client.1.smithi178.stdout:5/889: dwrite d9/d19/f126 [0,4194304] 0 2024-09-10T11:29:37.453 INFO:tasks.workunit.client.0.smithi144.stdout:5/484: dwrite d4/d5/d18/d67/d59/f5e [0,4194304] 0 2024-09-10T11:29:37.454 INFO:tasks.workunit.client.1.smithi178.stdout:2/955: chown d1/d7/d24/d7d/d84/d55/da1/lfa 50314033 1 2024-09-10T11:29:37.455 INFO:tasks.workunit.client.0.smithi144.stdout:5/485: write d4/d5/d18/d67/f62 [1398055,119398] 0 2024-09-10T11:29:37.456 INFO:tasks.workunit.client.1.smithi178.stdout:6/801: mkdir d3/d74/dc9/d122 0 2024-09-10T11:29:37.458 INFO:tasks.workunit.client.1.smithi178.stdout:4/657: dread d0/d9/d3d/f44 [0,4194304] 0 2024-09-10T11:29:37.462 INFO:tasks.workunit.client.1.smithi178.stdout:5/890: write d9/dc/d17/d1d/f54 [8419347,108082] 0 2024-09-10T11:29:37.463 INFO:tasks.workunit.client.1.smithi178.stdout:4/658: fdatasync d0/d8/da/d1a/d51/d5d/fa1 0 2024-09-10T11:29:37.465 INFO:tasks.workunit.client.0.smithi144.stdout:5/486: dwrite d4/d20/d24/f92 [0,4194304] 0 2024-09-10T11:29:37.466 INFO:tasks.workunit.client.1.smithi178.stdout:2/956: creat d1/d41/d127/f142 x:0 0 0 2024-09-10T11:29:37.467 INFO:tasks.workunit.client.1.smithi178.stdout:1/697: dread d3/d9/d98/dac/d3a/d50/ff9 [0,4194304] 0 2024-09-10T11:29:37.467 INFO:tasks.workunit.client.1.smithi178.stdout:5/891: write d9/dc/dd/fc1 [2811084,112853] 0 2024-09-10T11:29:37.470 INFO:tasks.workunit.client.0.smithi144.stdout:8/269: sync 2024-09-10T11:29:37.470 INFO:tasks.workunit.client.0.smithi144.stdout:3/459: sync 2024-09-10T11:29:37.470 INFO:tasks.workunit.client.0.smithi144.stdout:0/275: sync 2024-09-10T11:29:37.470 INFO:tasks.workunit.client.0.smithi144.stdout:2/322: sync 2024-09-10T11:29:37.471 INFO:tasks.workunit.client.0.smithi144.stdout:9/444: sync 2024-09-10T11:29:37.471 INFO:tasks.workunit.client.1.smithi178.stdout:0/733: dwrite d0/dc/d39/d4c/fb4 [0,4194304] 0 2024-09-10T11:29:37.472 INFO:tasks.workunit.client.1.smithi178.stdout:7/697: dread d6/ff [4194304,4194304] 0 2024-09-10T11:29:37.474 INFO:tasks.workunit.client.1.smithi178.stdout:0/734: write d0/dc/d39/da4/dce/fe1 [136107,10656] 0 2024-09-10T11:29:37.475 INFO:tasks.workunit.client.1.smithi178.stdout:0/735: fdatasync d0/fbf 0 2024-09-10T11:29:37.477 INFO:tasks.workunit.client.0.smithi144.stdout:2/323: rename d7/da/d5b/f6e to d7/d42/f71 0 2024-09-10T11:29:37.479 INFO:tasks.workunit.client.1.smithi178.stdout:5/892: fdatasync d9/dc/dd/d37/d42/d62/daf/df0/faa 0 2024-09-10T11:29:37.479 INFO:tasks.workunit.client.1.smithi178.stdout:2/957: mknod d1/d7/d24/d46/d88/d9d/dab/dbb/c143 0 2024-09-10T11:29:37.480 INFO:tasks.workunit.client.1.smithi178.stdout:5/893: readlink d9/dc/dd/d3f/d5a/l81 0 2024-09-10T11:29:37.481 INFO:tasks.workunit.client.0.smithi144.stdout:2/324: rmdir d7/da/d1e 39 2024-09-10T11:29:37.481 INFO:tasks.workunit.client.1.smithi178.stdout:6/802: rename d3/d7/d8/d15/d48 to d3/d123 0 2024-09-10T11:29:37.483 INFO:tasks.workunit.client.0.smithi144.stdout:2/325: read d7/da/f44 [3357642,8093] 0 2024-09-10T11:29:37.483 INFO:tasks.workunit.client.1.smithi178.stdout:1/698: mknod d3/d35/ddd/df4/c103 0 2024-09-10T11:29:37.483 INFO:tasks.workunit.client.0.smithi144.stdout:0/276: getdents da/d49 0 2024-09-10T11:29:37.484 INFO:tasks.workunit.client.1.smithi178.stdout:0/736: symlink d0/dc/d1f/d2d/dd1/le9 0 2024-09-10T11:29:37.485 INFO:tasks.workunit.client.1.smithi178.stdout:4/659: link d0/d63/f76 d0/d8/da/dae/fe3 0 2024-09-10T11:29:37.487 INFO:tasks.workunit.client.0.smithi144.stdout:9/445: dwrite d11/d12/d1b/d41/d8f/f55 [0,4194304] 0 2024-09-10T11:29:37.487 INFO:tasks.workunit.client.1.smithi178.stdout:7/698: dwrite d6/d10/d34/f44 [0,4194304] 0 2024-09-10T11:29:37.488 INFO:tasks.workunit.client.0.smithi144.stdout:0/277: creat da/d44/f54 x:0 0 0 2024-09-10T11:29:37.488 INFO:tasks.workunit.client.1.smithi178.stdout:0/737: chown d0/ca 15179 1 2024-09-10T11:29:37.491 INFO:tasks.workunit.client.1.smithi178.stdout:1/699: truncate d3/d9/d98/dd1/d7e/f8c 4974231 0 2024-09-10T11:29:37.494 INFO:tasks.workunit.client.0.smithi144.stdout:2/326: dwrite d7/f15 [0,4194304] 0 2024-09-10T11:29:37.494 INFO:tasks.workunit.client.1.smithi178.stdout:7/699: unlink d6/d10/fb3 0 2024-09-10T11:29:37.495 INFO:tasks.workunit.client.1.smithi178.stdout:6/803: rename d3/d74/d91/daa/l10d to d3/d74/d91/daa/dc2/d57/d116/l124 0 2024-09-10T11:29:37.497 INFO:tasks.workunit.client.1.smithi178.stdout:4/660: unlink d0/d9/dc/d8e/dbf/f77 0 2024-09-10T11:29:37.502 INFO:tasks.workunit.client.1.smithi178.stdout:0/738: mkdir d0/dc/d1f/dea 0 2024-09-10T11:29:37.502 INFO:tasks.workunit.client.1.smithi178.stdout:5/894: dwrite d9/dc/dd/d37/d42/d62/daf/df0/fe8 [0,4194304] 0 2024-09-10T11:29:37.508 INFO:tasks.workunit.client.1.smithi178.stdout:7/700: creat d6/d10/d1d/d22/d50/d98/da6/ffe x:0 0 0 2024-09-10T11:29:37.516 INFO:tasks.workunit.client.1.smithi178.stdout:1/700: dwrite d3/d35/fa4 [4194304,4194304] 0 2024-09-10T11:29:37.516 INFO:tasks.workunit.client.1.smithi178.stdout:7/701: dread - d6/d10/d1d/d22/d93/fe1 zero size 2024-09-10T11:29:37.516 INFO:tasks.workunit.client.1.smithi178.stdout:5/895: chown d9/dc/dd/d3f/dd6/fff 147 1 2024-09-10T11:29:37.519 INFO:tasks.workunit.client.1.smithi178.stdout:6/804: dwrite d3/d7/fb0 [0,4194304] 0 2024-09-10T11:29:37.524 INFO:tasks.workunit.client.1.smithi178.stdout:1/701: mkdir d3/d9/d21/d6f/d104 0 2024-09-10T11:29:37.524 INFO:tasks.workunit.client.1.smithi178.stdout:5/896: write d9/dc/dd/d3f/dd6/d124/f107 [521262,127132] 0 2024-09-10T11:29:37.525 INFO:tasks.workunit.client.1.smithi178.stdout:0/739: dread d0/d2c/d95/dbd/fcb [0,4194304] 0 2024-09-10T11:29:37.525 INFO:tasks.workunit.client.1.smithi178.stdout:7/702: readlink d6/d10/d1d/d22/d93/db7/ld1 0 2024-09-10T11:29:37.525 INFO:tasks.workunit.client.1.smithi178.stdout:4/661: dwrite d0/d8/df/f29 [0,4194304] 0 2024-09-10T11:29:37.531 INFO:tasks.workunit.client.1.smithi178.stdout:1/702: creat d3/d35/f105 x:0 0 0 2024-09-10T11:29:37.535 INFO:tasks.workunit.client.1.smithi178.stdout:5/897: unlink d9/dc/d17/d1d/c4a 0 2024-09-10T11:29:37.536 INFO:tasks.workunit.client.1.smithi178.stdout:0/740: creat d0/d12/feb x:0 0 0 2024-09-10T11:29:37.536 INFO:tasks.workunit.client.1.smithi178.stdout:5/898: fsync d9/dc/dd/d3f/d5a/d10a/f118 0 2024-09-10T11:29:37.537 INFO:tasks.workunit.client.1.smithi178.stdout:7/703: creat d6/d10/d1d/d4a/d79/df1/fff x:0 0 0 2024-09-10T11:29:37.537 INFO:tasks.workunit.client.1.smithi178.stdout:4/662: creat d0/d8/df/dde/fe4 x:0 0 0 2024-09-10T11:29:37.539 INFO:tasks.workunit.client.1.smithi178.stdout:5/899: dread - d9/dc/dd/d3f/d5a/d7d/f9b zero size 2024-09-10T11:29:37.541 INFO:tasks.workunit.client.1.smithi178.stdout:0/741: truncate d0/d2c/d33/f81 771578 0 2024-09-10T11:29:37.543 INFO:tasks.workunit.client.1.smithi178.stdout:0/742: write d0/dc/d39/d50/fc3 [974594,82805] 0 2024-09-10T11:29:37.545 INFO:tasks.workunit.client.1.smithi178.stdout:7/704: rename d6/d10/d1d/d22/d50/d7e/d82/lc4 to d6/d10/d1e/d2d/d2b/da0/d55/l100 0 2024-09-10T11:29:37.546 INFO:tasks.workunit.client.1.smithi178.stdout:4/663: rmdir d0/d9d/dd5/de2 0 2024-09-10T11:29:37.547 INFO:tasks.workunit.client.1.smithi178.stdout:4/664: chown d0/f1c 103782600 1 2024-09-10T11:29:37.547 INFO:tasks.workunit.client.0.smithi144.stdout:2/327: write d7/de/f37 [708016,88853] 0 2024-09-10T11:29:37.547 INFO:tasks.workunit.client.0.smithi144.stdout:0/278: truncate da/f19 4003844 0 2024-09-10T11:29:37.549 INFO:tasks.workunit.client.1.smithi178.stdout:5/900: dread d9/dc/dd/d3f/dd6/d124/d6d/dcc/dd8/f100 [0,4194304] 0 2024-09-10T11:29:37.550 INFO:tasks.workunit.client.1.smithi178.stdout:7/705: unlink d6/d10/d34/f8d 0 2024-09-10T11:29:37.553 INFO:tasks.workunit.client.1.smithi178.stdout:4/665: rmdir d0/d8/da/d1a 39 2024-09-10T11:29:37.553 INFO:tasks.workunit.client.0.smithi144.stdout:9/446: dwrite d11/d12/f14 [4194304,4194304] 0 2024-09-10T11:29:37.556 INFO:tasks.workunit.client.0.smithi144.stdout:2/328: getdents d7/de/d36/d45/d47/d6c 0 2024-09-10T11:29:37.556 INFO:tasks.workunit.client.1.smithi178.stdout:5/901: unlink d9/dc/dd/d3f/dd6/fff 0 2024-09-10T11:29:37.557 INFO:tasks.workunit.client.0.smithi144.stdout:3/460: dwrite d4/d28/d67/d80/d17/d18/f2e [0,4194304] 0 2024-09-10T11:29:37.557 INFO:tasks.workunit.client.1.smithi178.stdout:1/703: rmdir d3/d9 39 2024-09-10T11:29:37.559 INFO:tasks.workunit.client.0.smithi144.stdout:9/447: chown d11/d36/d2d/f70 1 1 2024-09-10T11:29:37.560 INFO:tasks.workunit.client.0.smithi144.stdout:2/329: stat d7/d29/f60 0 2024-09-10T11:29:37.561 INFO:tasks.workunit.client.0.smithi144.stdout:3/461: fsync d4/dd/d21/f62 0 2024-09-10T11:29:37.563 INFO:tasks.workunit.client.0.smithi144.stdout:0/279: dwrite da/f32 [0,4194304] 0 2024-09-10T11:29:37.564 INFO:tasks.workunit.client.0.smithi144.stdout:2/330: write d7/de/d3d/f52 [2499295,24636] 0 2024-09-10T11:29:37.565 INFO:tasks.workunit.client.0.smithi144.stdout:3/462: readlink d4/d28/d67/d80/d17/l52 0 2024-09-10T11:29:37.565 INFO:tasks.workunit.client.0.smithi144.stdout:0/280: truncate da/f28 982126 0 2024-09-10T11:29:37.567 INFO:tasks.workunit.client.1.smithi178.stdout:6/805: dwrite d3/d24/f81 [0,4194304] 0 2024-09-10T11:29:37.567 INFO:tasks.workunit.client.1.smithi178.stdout:5/902: creat d9/dc/d17/dcd/f129 x:0 0 0 2024-09-10T11:29:37.568 INFO:tasks.workunit.client.1.smithi178.stdout:1/704: symlink d3/d35/ddd/l106 0 2024-09-10T11:29:37.568 INFO:tasks.workunit.client.1.smithi178.stdout:7/706: fdatasync d6/d10/d1e/d2d/d2b/da0/d55/dbd/f77 0 2024-09-10T11:29:37.570 INFO:tasks.workunit.client.0.smithi144.stdout:2/331: creat d7/de/d3d/d4e/f72 x:0 0 0 2024-09-10T11:29:37.571 INFO:tasks.workunit.client.0.smithi144.stdout:2/332: chown d7/de/d3d/c4d 6 1 2024-09-10T11:29:37.572 INFO:tasks.workunit.client.0.smithi144.stdout:0/281: mknod da/d48/c55 0 2024-09-10T11:29:37.573 INFO:tasks.workunit.client.0.smithi144.stdout:3/463: mknod d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/c9e 0 2024-09-10T11:29:37.574 INFO:tasks.workunit.client.0.smithi144.stdout:0/282: write da/f1a [542826,97123] 0 2024-09-10T11:29:37.578 INFO:tasks.workunit.client.0.smithi144.stdout:3/464: chown d4/d28/d67/d80/d17/d18/d34/d65/l95 0 1 2024-09-10T11:29:37.579 INFO:tasks.workunit.client.1.smithi178.stdout:6/806: creat d3/d7/d8/d15/f125 x:0 0 0 2024-09-10T11:29:37.582 INFO:tasks.workunit.client.1.smithi178.stdout:3/845: sync 2024-09-10T11:29:37.583 INFO:tasks.workunit.client.0.smithi144.stdout:3/465: read d4/dd/d21/d2f/f3a [4128898,95921] 0 2024-09-10T11:29:37.583 INFO:tasks.workunit.client.0.smithi144.stdout:0/283: dwrite da/f36 [0,4194304] 0 2024-09-10T11:29:37.584 INFO:tasks.workunit.client.1.smithi178.stdout:4/666: rename d0/d8/da/d1a/d51/d5d/f80 to d0/d10/fe5 0 2024-09-10T11:29:37.584 INFO:tasks.workunit.client.1.smithi178.stdout:7/707: dwrite d6/d10/d1d/d4a/f57 [0,4194304] 0 2024-09-10T11:29:37.584 INFO:tasks.workunit.client.1.smithi178.stdout:3/846: dread - db/d26/d74/dd0/f112 zero size 2024-09-10T11:29:37.587 INFO:tasks.workunit.client.1.smithi178.stdout:7/708: chown d6/d10/d1e/d2d/d2b/f39 73192 1 2024-09-10T11:29:37.587 INFO:tasks.workunit.client.0.smithi144.stdout:3/466: mkdir d4/d28/d67/d9f 0 2024-09-10T11:29:37.592 INFO:tasks.workunit.client.0.smithi144.stdout:0/284: getdents da/d48 0 2024-09-10T11:29:37.595 INFO:tasks.workunit.client.0.smithi144.stdout:3/467: mknod d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/ca0 0 2024-09-10T11:29:37.595 INFO:tasks.workunit.client.1.smithi178.stdout:1/705: symlink d3/d9/d98/dac/d3a/d50/d6c/l107 0 2024-09-10T11:29:37.596 INFO:tasks.workunit.client.1.smithi178.stdout:0/743: write d0/dc/d1f/f74 [49147,84595] 0 2024-09-10T11:29:37.597 INFO:tasks.workunit.client.0.smithi144.stdout:9/448: dwrite d11/d12/d1b/f77 [0,4194304] 0 2024-09-10T11:29:37.599 INFO:tasks.workunit.client.1.smithi178.stdout:3/847: symlink db/d20/d3e/d6b/dc0/l116 0 2024-09-10T11:29:37.599 INFO:tasks.workunit.client.0.smithi144.stdout:2/333: dwrite d7/da/f44 [4194304,4194304] 0 2024-09-10T11:29:37.600 INFO:tasks.workunit.client.0.smithi144.stdout:3/468: chown d4/dd/d21/d2f/f3a 2813 1 2024-09-10T11:29:37.600 INFO:tasks.workunit.client.1.smithi178.stdout:3/848: write db/d20/f3c [5046772,35529] 0 2024-09-10T11:29:37.601 INFO:tasks.workunit.client.1.smithi178.stdout:3/849: fdatasync db/f45 0 2024-09-10T11:29:37.602 INFO:tasks.workunit.client.1.smithi178.stdout:0/744: mknod d0/dc/d39/da4/dce/cec 0 2024-09-10T11:29:37.602 INFO:tasks.workunit.client.1.smithi178.stdout:3/850: stat db/d20/d70/f73 0 2024-09-10T11:29:37.604 INFO:tasks.workunit.client.0.smithi144.stdout:3/469: chown d4/dd/f12 2150429 1 2024-09-10T11:29:37.604 INFO:tasks.workunit.client.0.smithi144.stdout:9/449: readlink d11/d36/d2d/d38/l81 0 2024-09-10T11:29:37.605 INFO:tasks.workunit.client.0.smithi144.stdout:2/334: write d7/f41 [1107408,74576] 0 2024-09-10T11:29:37.606 INFO:tasks.workunit.client.0.smithi144.stdout:0/285: dwrite da/f41 [0,4194304] 0 2024-09-10T11:29:37.606 INFO:tasks.workunit.client.0.smithi144.stdout:9/450: read d11/d36/f7f [614411,82689] 0 2024-09-10T11:29:37.607 INFO:tasks.workunit.client.0.smithi144.stdout:3/470: write d4/d1a/f3b [1233486,42678] 0 2024-09-10T11:29:37.608 INFO:tasks.workunit.client.0.smithi144.stdout:2/335: chown d7/da/d1e/d21/d54 308835 1 2024-09-10T11:29:37.609 INFO:tasks.workunit.client.0.smithi144.stdout:3/471: chown d4/d28/d67/d80/d17/d18/d34/d4b/f53 549 1 2024-09-10T11:29:37.616 INFO:tasks.workunit.client.1.smithi178.stdout:6/807: getdents d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d11b 0 2024-09-10T11:29:37.617 INFO:tasks.workunit.client.1.smithi178.stdout:6/808: chown d3/d74/d91/daa/dc2/c8f 2863 1 2024-09-10T11:29:37.618 INFO:tasks.workunit.client.0.smithi144.stdout:2/336: creat d7/de/d3d/d5e/f73 x:0 0 0 2024-09-10T11:29:37.618 INFO:tasks.workunit.client.0.smithi144.stdout:3/472: symlink d4/d28/d67/d80/d1d/la1 0 2024-09-10T11:29:37.624 INFO:tasks.workunit.client.0.smithi144.stdout:2/337: chown d7/da/d1e/l1f 7297087 1 2024-09-10T11:29:37.624 INFO:tasks.workunit.client.0.smithi144.stdout:9/451: creat d11/fa2 x:0 0 0 2024-09-10T11:29:37.625 INFO:tasks.workunit.client.1.smithi178.stdout:6/809: symlink d3/d74/dc9/da6/d11e/l126 0 2024-09-10T11:29:37.625 INFO:tasks.workunit.client.1.smithi178.stdout:6/810: dread - d3/d74/dc9/f10a zero size 2024-09-10T11:29:37.625 INFO:tasks.workunit.client.1.smithi178.stdout:0/745: truncate d0/dc/fe6 476527 0 2024-09-10T11:29:37.629 INFO:tasks.workunit.client.1.smithi178.stdout:6/811: creat d3/d74/dc9/f127 x:0 0 0 2024-09-10T11:29:37.631 INFO:tasks.workunit.client.0.smithi144.stdout:2/338: rename d7/da/f44 to d7/de/d36/f74 0 2024-09-10T11:29:37.632 INFO:tasks.workunit.client.1.smithi178.stdout:7/709: write d6/d10/d1d/d22/d50/d7e/d82/db2/f7b [686072,46912] 0 2024-09-10T11:29:37.633 INFO:tasks.workunit.client.0.smithi144.stdout:2/339: truncate d7/d29/f3f 4504777 0 2024-09-10T11:29:37.633 INFO:tasks.workunit.client.0.smithi144.stdout:9/452: dwrite d11/d36/f7f [0,4194304] 0 2024-09-10T11:29:37.634 INFO:tasks.workunit.client.1.smithi178.stdout:6/812: creat d3/d74/d91/daa/dc2/d57/d78/f128 x:0 0 0 2024-09-10T11:29:37.639 INFO:tasks.workunit.client.0.smithi144.stdout:9/453: symlink d11/d12/d1b/d2c/d78/la3 0 2024-09-10T11:29:37.641 INFO:tasks.workunit.client.0.smithi144.stdout:3/473: write d4/dd/d21/f83 [2187065,30622] 0 2024-09-10T11:29:37.642 INFO:tasks.workunit.client.0.smithi144.stdout:0/286: dread da/f30 [0,4194304] 0 2024-09-10T11:29:37.644 INFO:tasks.workunit.client.0.smithi144.stdout:3/474: write d4/dd/d21/d2f/f8d [699786,130392] 0 2024-09-10T11:29:37.645 INFO:tasks.workunit.client.0.smithi144.stdout:9/454: creat d11/d36/fa4 x:0 0 0 2024-09-10T11:29:37.647 INFO:tasks.workunit.client.1.smithi178.stdout:0/746: write d0/dc/f13 [7011545,31244] 0 2024-09-10T11:29:37.649 INFO:tasks.workunit.client.0.smithi144.stdout:0/287: fsync da/f19 0 2024-09-10T11:29:37.650 INFO:tasks.workunit.client.0.smithi144.stdout:3/475: rmdir d4/d28/d67/d80/d1d 39 2024-09-10T11:29:37.651 INFO:tasks.workunit.client.0.smithi144.stdout:2/340: dwrite d7/de/f22 [0,4194304] 0 2024-09-10T11:29:37.653 INFO:tasks.workunit.client.1.smithi178.stdout:3/851: dwrite db/d35/d58/f7b [0,4194304] 0 2024-09-10T11:29:37.654 INFO:tasks.workunit.client.0.smithi144.stdout:2/341: write d7/da/d1e/d21/f53 [712983,41732] 0 2024-09-10T11:29:37.655 INFO:tasks.workunit.client.0.smithi144.stdout:9/455: symlink d11/d12/d1b/d22/la5 0 2024-09-10T11:29:37.658 INFO:tasks.workunit.client.1.smithi178.stdout:1/706: dread d3/d9/d21/d4b/d2b/f88 [0,4194304] 0 2024-09-10T11:29:37.658 INFO:tasks.workunit.client.0.smithi144.stdout:2/342: symlink d7/d42/l75 0 2024-09-10T11:29:37.662 INFO:tasks.workunit.client.0.smithi144.stdout:3/476: dread d4/dd/d21/f86 [0,4194304] 0 2024-09-10T11:29:37.664 INFO:tasks.workunit.client.0.smithi144.stdout:3/477: chown d4/d28/d67/d80/d17/d18/d34 1 1 2024-09-10T11:29:37.667 INFO:tasks.workunit.client.1.smithi178.stdout:0/747: dread d0/dc/d39/d50/f25 [0,4194304] 0 2024-09-10T11:29:37.669 INFO:tasks.workunit.client.0.smithi144.stdout:3/478: creat d4/d28/d67/d80/fa2 x:0 0 0 2024-09-10T11:29:37.671 INFO:tasks.workunit.client.0.smithi144.stdout:3/479: mkdir d4/d28/d67/d80/da3 0 2024-09-10T11:29:37.677 INFO:tasks.workunit.client.0.smithi144.stdout:3/480: dwrite d4/dd/d21/f5b [0,4194304] 0 2024-09-10T11:29:37.677 INFO:tasks.workunit.client.1.smithi178.stdout:6/813: getdents d3/d7/d8/d15/d1c/d2a/d18 0 2024-09-10T11:29:37.680 INFO:tasks.workunit.client.1.smithi178.stdout:1/707: creat d3/d9/d98/dac/d2a/f108 x:0 0 0 2024-09-10T11:29:37.682 INFO:tasks.workunit.client.1.smithi178.stdout:0/748: chown d0/dc/l30 58 1 2024-09-10T11:29:37.684 INFO:tasks.workunit.client.1.smithi178.stdout:1/708: fsync d3/da0/fc1 0 2024-09-10T11:29:37.685 INFO:tasks.workunit.client.0.smithi144.stdout:0/288: dwrite da/f46 [0,4194304] 0 2024-09-10T11:29:37.686 INFO:tasks.workunit.client.1.smithi178.stdout:0/749: unlink d0/dc/d39/da4/dce/cec 0 2024-09-10T11:29:37.689 INFO:tasks.workunit.client.0.smithi144.stdout:0/289: dread - da/f39 zero size 2024-09-10T11:29:37.689 INFO:tasks.workunit.client.1.smithi178.stdout:3/852: getdents db/d20/d3d 0 2024-09-10T11:29:37.690 INFO:tasks.workunit.client.1.smithi178.stdout:3/853: fdatasync db/d20/d3d/d63/f101 0 2024-09-10T11:29:37.690 INFO:tasks.workunit.client.1.smithi178.stdout:0/750: dread - d0/dc/d1f/dae/fbc zero size 2024-09-10T11:29:37.693 INFO:tasks.workunit.client.0.smithi144.stdout:3/481: dwrite d4/d28/d67/d80/d17/d18/d34/d4b/f53 [4194304,4194304] 0 2024-09-10T11:29:37.693 INFO:tasks.workunit.client.0.smithi144.stdout:0/290: write da/f41 [1910644,122607] 0 2024-09-10T11:29:37.694 INFO:tasks.workunit.client.0.smithi144.stdout:2/343: dwrite d7/da/f17 [0,4194304] 0 2024-09-10T11:29:37.694 INFO:tasks.workunit.client.1.smithi178.stdout:3/854: creat db/d35/d3a/d9f/f117 x:0 0 0 2024-09-10T11:29:37.694 INFO:tasks.workunit.client.0.smithi144.stdout:3/482: fsync d4/dd/f12 0 2024-09-10T11:29:37.696 INFO:tasks.workunit.client.1.smithi178.stdout:3/855: mknod db/d20/d3d/d56/dcf/c118 0 2024-09-10T11:29:37.699 INFO:tasks.workunit.client.0.smithi144.stdout:2/344: mkdir d7/d42/d64/d76 0 2024-09-10T11:29:37.699 INFO:tasks.workunit.client.0.smithi144.stdout:0/291: mknod da/c56 0 2024-09-10T11:29:37.700 INFO:tasks.workunit.client.0.smithi144.stdout:3/483: mknod d4/ca4 0 2024-09-10T11:29:37.701 INFO:tasks.workunit.client.0.smithi144.stdout:2/345: fsync d7/de/d36/f74 0 2024-09-10T11:29:37.702 INFO:tasks.workunit.client.0.smithi144.stdout:0/292: fdatasync da/d3a/f4d 0 2024-09-10T11:29:37.703 INFO:tasks.workunit.client.1.smithi178.stdout:3/856: unlink db/f16 0 2024-09-10T11:29:37.706 INFO:tasks.workunit.client.0.smithi144.stdout:0/293: symlink da/d3a/d51/l57 0 2024-09-10T11:29:37.707 INFO:tasks.workunit.client.0.smithi144.stdout:3/484: symlink d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93/d98/la5 0 2024-09-10T11:29:37.708 INFO:tasks.workunit.client.1.smithi178.stdout:3/857: symlink db/d35/d3a/d42/d46/d110/l119 0 2024-09-10T11:29:37.709 INFO:tasks.workunit.client.0.smithi144.stdout:0/294: symlink da/l58 0 2024-09-10T11:29:37.710 INFO:tasks.workunit.client.0.smithi144.stdout:0/295: readlink da/l22 0 2024-09-10T11:29:37.712 INFO:tasks.workunit.client.0.smithi144.stdout:0/296: write da/d48/f4a [339402,67618] 0 2024-09-10T11:29:37.713 INFO:tasks.workunit.client.0.smithi144.stdout:2/346: dwrite d7/de/f1b [0,4194304] 0 2024-09-10T11:29:37.714 INFO:tasks.workunit.client.0.smithi144.stdout:3/485: creat d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93/fa6 x:0 0 0 2024-09-10T11:29:37.715 INFO:tasks.workunit.client.0.smithi144.stdout:2/347: getdents d7/de/d5f 0 2024-09-10T11:29:37.717 INFO:tasks.workunit.client.0.smithi144.stdout:3/486: stat d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/f89 0 2024-09-10T11:29:37.719 INFO:tasks.workunit.client.1.smithi178.stdout:6/814: write d3/d7/d8/d72/faf [405760,114210] 0 2024-09-10T11:29:37.721 INFO:tasks.workunit.client.0.smithi144.stdout:2/348: mknod d7/de/d36/d45/d47/c77 0 2024-09-10T11:29:37.722 INFO:tasks.workunit.client.0.smithi144.stdout:3/487: mknod d4/d28/d67/d96/ca7 0 2024-09-10T11:29:37.722 INFO:tasks.workunit.client.0.smithi144.stdout:2/349: chown d7/d42/l75 18657098 1 2024-09-10T11:29:37.722 INFO:tasks.workunit.client.0.smithi144.stdout:0/297: dwrite da/f1c [0,4194304] 0 2024-09-10T11:29:37.724 INFO:tasks.workunit.client.1.smithi178.stdout:6/815: truncate d3/d74/d91/daa/dc2/d57/d78/d7c/f92 13190 0 2024-09-10T11:29:37.724 INFO:tasks.workunit.client.1.smithi178.stdout:6/816: chown d3/d6e/fda 21 1 2024-09-10T11:29:37.727 INFO:tasks.workunit.client.0.smithi144.stdout:0/298: symlink da/d49/l59 0 2024-09-10T11:29:37.727 INFO:tasks.workunit.client.1.smithi178.stdout:6/817: fdatasync d3/d7/d8/d15/d1c/d2a/d18/d8e/fa4 0 2024-09-10T11:29:37.728 INFO:tasks.workunit.client.0.smithi144.stdout:3/488: mknod d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93/ca8 0 2024-09-10T11:29:37.730 INFO:tasks.workunit.client.0.smithi144.stdout:3/489: chown d4/ca4 766179459 1 2024-09-10T11:29:37.731 INFO:tasks.workunit.client.1.smithi178.stdout:6/818: creat d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d11b/f129 x:0 0 0 2024-09-10T11:29:37.734 INFO:tasks.workunit.client.0.smithi144.stdout:2/350: dwrite d7/de/f28 [0,4194304] 0 2024-09-10T11:29:37.736 INFO:tasks.workunit.client.0.smithi144.stdout:0/299: dwrite f5 [0,4194304] 0 2024-09-10T11:29:37.736 INFO:tasks.workunit.client.0.smithi144.stdout:3/490: stat d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/f8c 0 2024-09-10T11:29:37.737 INFO:tasks.workunit.client.1.smithi178.stdout:6/819: dwrite d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/f10f [0,4194304] 0 2024-09-10T11:29:37.739 INFO:tasks.workunit.client.0.smithi144.stdout:2/351: symlink d7/de/d3d/d4e/l78 0 2024-09-10T11:29:37.741 INFO:tasks.workunit.client.1.smithi178.stdout:6/820: unlink d3/d74/d91/daa/dc2/d57/fcb 0 2024-09-10T11:29:37.741 INFO:tasks.workunit.client.0.smithi144.stdout:0/300: unlink da/f39 0 2024-09-10T11:29:37.742 INFO:tasks.workunit.client.0.smithi144.stdout:0/301: chown da/c38 0 1 2024-09-10T11:29:37.744 INFO:tasks.workunit.client.0.smithi144.stdout:2/352: read d7/f8 [3665640,41913] 0 2024-09-10T11:29:37.744 INFO:tasks.workunit.client.0.smithi144.stdout:2/353: readlink d7/l34 0 2024-09-10T11:29:37.744 INFO:tasks.workunit.client.1.smithi178.stdout:6/821: mkdir d3/d74/dc9/d12a 0 2024-09-10T11:29:37.746 INFO:tasks.workunit.client.0.smithi144.stdout:2/354: read d7/da/d1e/d21/f53 [68552,120507] 0 2024-09-10T11:29:37.747 INFO:tasks.workunit.client.0.smithi144.stdout:3/491: unlink d4/d1a/c7f 0 2024-09-10T11:29:37.749 INFO:tasks.workunit.client.1.smithi178.stdout:6/822: creat d3/d74/dc9/ddc/f12b x:0 0 0 2024-09-10T11:29:37.750 INFO:tasks.workunit.client.1.smithi178.stdout:6/823: fdatasync d3/d7/d8/d15/f125 0 2024-09-10T11:29:37.752 INFO:tasks.workunit.client.1.smithi178.stdout:1/709: write d3/d9/d98/dd1/f6a [7283796,53335] 0 2024-09-10T11:29:37.755 INFO:tasks.workunit.client.0.smithi144.stdout:3/492: creat d4/dd/fa9 x:0 0 0 2024-09-10T11:29:37.756 INFO:tasks.workunit.client.1.smithi178.stdout:1/710: creat d3/d9/d21/f109 x:0 0 0 2024-09-10T11:29:37.758 INFO:tasks.workunit.client.0.smithi144.stdout:3/493: creat d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93/faa x:0 0 0 2024-09-10T11:29:37.758 INFO:tasks.workunit.client.1.smithi178.stdout:1/711: stat d3/d9/d98/dac/d3a/d6e/d8a/f2e 0 2024-09-10T11:29:37.761 INFO:tasks.workunit.client.1.smithi178.stdout:1/712: readlink d3/d35/l69 0 2024-09-10T11:29:37.761 INFO:tasks.workunit.client.1.smithi178.stdout:6/824: dwrite d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d11b/f129 [0,4194304] 0 2024-09-10T11:29:37.761 INFO:tasks.workunit.client.0.smithi144.stdout:3/494: mknod d4/d28/d67/d80/d17/d18/d34/d78/cab 0 2024-09-10T11:29:37.763 INFO:tasks.workunit.client.1.smithi178.stdout:3/858: truncate db/d20/d3d/d63/f96 348412 0 2024-09-10T11:29:37.764 INFO:tasks.workunit.client.1.smithi178.stdout:6/825: mkdir d3/d74/dc9/ddc/d12c 0 2024-09-10T11:29:37.764 INFO:tasks.workunit.client.1.smithi178.stdout:3/859: chown db/d20/d3e/d6b 33989004 1 2024-09-10T11:29:37.766 INFO:tasks.workunit.client.1.smithi178.stdout:1/713: rename d3/d9/d98/dac/d3a/d50/db5/fc9 to d3/d35/ddd/df4/f10a 0 2024-09-10T11:29:37.767 INFO:tasks.workunit.client.1.smithi178.stdout:3/860: rmdir db/d26/d9a 39 2024-09-10T11:29:37.768 INFO:tasks.workunit.client.1.smithi178.stdout:6/826: read - d3/d7/d8/d15/fbf zero size 2024-09-10T11:29:37.772 INFO:tasks.workunit.client.1.smithi178.stdout:1/714: creat d3/d9/d1a/d48/f10b x:0 0 0 2024-09-10T11:29:37.773 INFO:tasks.workunit.client.1.smithi178.stdout:3/861: symlink db/d9b/dfb/l11a 0 2024-09-10T11:29:37.774 INFO:tasks.workunit.client.1.smithi178.stdout:6/827: mkdir d3/d74/dc9/d12a/d12d 0 2024-09-10T11:29:37.775 INFO:tasks.workunit.client.1.smithi178.stdout:1/715: rmdir d3/d9/d98/dac/d3a/d50/d6c/d78/db8 39 2024-09-10T11:29:37.777 INFO:tasks.workunit.client.1.smithi178.stdout:6/828: dread d3/d7/d8/d72/faf [0,4194304] 0 2024-09-10T11:29:37.779 INFO:tasks.workunit.client.1.smithi178.stdout:1/716: rename d3/d9/d21/d4b/f2d to d3/d9/d98/dac/f10c 0 2024-09-10T11:29:37.781 INFO:tasks.workunit.client.1.smithi178.stdout:6/829: rmdir d3/d6e 39 2024-09-10T11:29:37.788 INFO:tasks.workunit.client.1.smithi178.stdout:3/862: truncate db/d20/f28 614173 0 2024-09-10T11:29:37.789 INFO:tasks.workunit.client.1.smithi178.stdout:3/863: dread - db/d20/d3d/d56/dcf/ffc zero size 2024-09-10T11:29:37.789 INFO:tasks.workunit.client.1.smithi178.stdout:3/864: stat db/d20/d3d/d56/d64/l94 0 2024-09-10T11:29:37.790 INFO:tasks.workunit.client.1.smithi178.stdout:3/865: read db/d36/f50 [1887963,5083] 0 2024-09-10T11:29:37.792 INFO:tasks.workunit.client.0.smithi144.stdout:1/536: sync 2024-09-10T11:29:37.792 INFO:tasks.workunit.client.0.smithi144.stdout:7/378: sync 2024-09-10T11:29:37.792 INFO:tasks.workunit.client.0.smithi144.stdout:6/382: sync 2024-09-10T11:29:37.795 INFO:tasks.workunit.client.1.smithi178.stdout:1/717: dwrite d3/d9/d21/d6f/f75 [0,4194304] 0 2024-09-10T11:29:37.798 INFO:tasks.workunit.client.1.smithi178.stdout:1/718: symlink d3/d35/ddd/l10d 0 2024-09-10T11:29:37.804 INFO:tasks.workunit.client.1.smithi178.stdout:3/866: dread db/d36/f50 [0,4194304] 0 2024-09-10T11:29:37.812 INFO:tasks.workunit.client.1.smithi178.stdout:1/719: dread d3/d9/d98/dac/d3a/fee [0,4194304] 0 2024-09-10T11:29:37.828 INFO:tasks.workunit.client.1.smithi178.stdout:8/886: sync 2024-09-10T11:29:37.829 INFO:tasks.workunit.client.1.smithi178.stdout:1/720: dread d3/d9/d98/dd1/d7e/f8c [0,4194304] 0 2024-09-10T11:29:37.829 INFO:tasks.workunit.client.0.smithi144.stdout:2/355: write d7/d29/f60 [5296776,49795] 0 2024-09-10T11:29:37.830 INFO:tasks.workunit.client.1.smithi178.stdout:8/887: fsync d5/db/d10/d35/d117/f120 0 2024-09-10T11:29:37.831 INFO:tasks.workunit.client.1.smithi178.stdout:1/721: mknod d3/d9/d98/dac/d3a/d6e/c10e 0 2024-09-10T11:29:37.831 INFO:tasks.workunit.client.1.smithi178.stdout:8/888: fsync d5/d1d/fdc 0 2024-09-10T11:29:37.833 INFO:tasks.workunit.client.0.smithi144.stdout:2/356: mknod d7/da/d1e/d21/d54/c79 0 2024-09-10T11:29:37.835 INFO:tasks.workunit.client.0.smithi144.stdout:3/495: write d4/d28/d67/d80/d17/d18/d34/d65/f88 [1018861,8812] 0 2024-09-10T11:29:37.836 INFO:tasks.workunit.client.1.smithi178.stdout:1/722: creat d3/d9/d98/dac/d3a/d50/d80/f10f x:0 0 0 2024-09-10T11:29:37.838 INFO:tasks.workunit.client.1.smithi178.stdout:1/723: chown d3/d9/d98/dac/d3a/d50/d80/fef 13411 1 2024-09-10T11:29:37.840 INFO:tasks.workunit.client.0.smithi144.stdout:7/379: dwrite d5/dc/d2a/d41/f49 [0,4194304] 0 2024-09-10T11:29:37.841 INFO:tasks.workunit.client.0.smithi144.stdout:6/383: dwrite d9/f71 [0,4194304] 0 2024-09-10T11:29:37.841 INFO:tasks.workunit.client.0.smithi144.stdout:2/357: creat d7/da/d1e/d21/f7a x:0 0 0 2024-09-10T11:29:37.843 INFO:tasks.workunit.client.0.smithi144.stdout:1/537: dwrite d5/df/d4b/d85/f97 [0,4194304] 0 2024-09-10T11:29:37.847 INFO:tasks.workunit.client.0.smithi144.stdout:2/358: fdatasync d7/da/f19 0 2024-09-10T11:29:37.848 INFO:tasks.workunit.client.0.smithi144.stdout:1/538: stat d5/df/d4b/d51/f92 0 2024-09-10T11:29:37.848 INFO:tasks.workunit.client.0.smithi144.stdout:2/359: readlink d7/da/l24 0 2024-09-10T11:29:37.851 INFO:tasks.workunit.client.0.smithi144.stdout:3/496: dwrite d4/dd/f1b [0,4194304] 0 2024-09-10T11:29:37.852 INFO:tasks.workunit.client.0.smithi144.stdout:7/380: fsync d5/dc/d14/d40/d5e/f6f 0 2024-09-10T11:29:37.852 INFO:tasks.workunit.client.0.smithi144.stdout:1/539: chown d5/dd/f13 796783851 1 2024-09-10T11:29:37.855 INFO:tasks.workunit.client.0.smithi144.stdout:3/497: chown d4/d28/d67/d80/d17/d18/d34/d4b 5666 1 2024-09-10T11:29:37.856 INFO:tasks.workunit.client.0.smithi144.stdout:1/540: write d5/dd/f1d [4092864,69179] 0 2024-09-10T11:29:37.857 INFO:tasks.workunit.client.0.smithi144.stdout:2/360: creat d7/de/d3d/d43/f7b x:0 0 0 2024-09-10T11:29:37.857 INFO:tasks.workunit.client.1.smithi178.stdout:8/889: dread d5/db/d10/d20/f26 [0,4194304] 0 2024-09-10T11:29:37.858 INFO:tasks.workunit.client.0.smithi144.stdout:7/381: fdatasync d5/dc/f3c 0 2024-09-10T11:29:37.859 INFO:tasks.workunit.client.0.smithi144.stdout:6/384: dwrite d9/dd/d3a/f51 [4194304,4194304] 0 2024-09-10T11:29:37.860 INFO:tasks.workunit.client.0.smithi144.stdout:1/541: chown d5/d2a/f7e 10032 1 2024-09-10T11:29:37.861 INFO:tasks.workunit.client.0.smithi144.stdout:3/498: dread - d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/f89 zero size 2024-09-10T11:29:37.866 INFO:tasks.workunit.client.1.smithi178.stdout:8/890: dwrite d5/d1d/f119 [0,4194304] 0 2024-09-10T11:29:37.868 INFO:tasks.workunit.client.0.smithi144.stdout:1/542: creat d5/dd/d64/d79/fb3 x:0 0 0 2024-09-10T11:29:37.868 INFO:tasks.workunit.client.0.smithi144.stdout:6/385: dread d9/d1f/f77 [0,4194304] 0 2024-09-10T11:29:37.868 INFO:tasks.workunit.client.0.smithi144.stdout:6/386: chown d9/f81 43125022 1 2024-09-10T11:29:37.868 INFO:tasks.workunit.client.0.smithi144.stdout:1/543: fdatasync d5/d4a/d63/f9f 0 2024-09-10T11:29:37.870 INFO:tasks.workunit.client.1.smithi178.stdout:8/891: mkdir d5/d30/d33/d8c/da7/d12c 0 2024-09-10T11:29:37.872 INFO:tasks.workunit.client.0.smithi144.stdout:1/544: creat d5/df/d4b/fb4 x:0 0 0 2024-09-10T11:29:37.874 INFO:tasks.workunit.client.1.smithi178.stdout:8/892: write d5/db/d10/d53/f98 [2603984,69392] 0 2024-09-10T11:29:37.874 INFO:tasks.workunit.client.0.smithi144.stdout:6/387: truncate d9/dd/d3e/f40 1295258 0 2024-09-10T11:29:37.877 INFO:tasks.workunit.client.0.smithi144.stdout:6/388: symlink d9/d1f/l8b 0 2024-09-10T11:29:37.877 INFO:tasks.workunit.client.0.smithi144.stdout:6/389: fsync d9/dd/d11/f56 0 2024-09-10T11:29:37.879 INFO:tasks.workunit.client.0.smithi144.stdout:6/390: fsync d9/d1f/d2c/f85 0 2024-09-10T11:29:37.880 INFO:tasks.workunit.client.0.smithi144.stdout:6/391: dread - d9/d1f/d2c/d4f/f7d zero size 2024-09-10T11:29:37.882 INFO:tasks.workunit.client.0.smithi144.stdout:6/392: creat d9/dd/d3e/f8c x:0 0 0 2024-09-10T11:29:37.883 INFO:tasks.workunit.client.1.smithi178.stdout:8/893: dwrite d5/db/d10/d70/dc4/f125 [0,4194304] 0 2024-09-10T11:29:37.887 INFO:tasks.workunit.client.0.smithi144.stdout:6/393: rename d9/d1f/f77 to d9/dd/d11/d1b/f8d 0 2024-09-10T11:29:37.888 INFO:tasks.workunit.client.1.smithi178.stdout:8/894: rename d5/db/d10/d53/d82/db2/l116 to d5/d30/db4/l12d 0 2024-09-10T11:29:37.889 INFO:tasks.workunit.client.0.smithi144.stdout:6/394: chown d9/dd/d3e/d47/f16 6 1 2024-09-10T11:29:37.889 INFO:tasks.workunit.client.0.smithi144.stdout:3/499: write d4/dd/d21/f86 [409720,108087] 0 2024-09-10T11:29:37.890 INFO:tasks.workunit.client.0.smithi144.stdout:2/361: truncate d7/da/f13 2386144 0 2024-09-10T11:29:37.890 INFO:tasks.workunit.client.0.smithi144.stdout:1/545: dread d5/d4a/d44/d80/f66 [0,4194304] 0 2024-09-10T11:29:37.891 INFO:tasks.workunit.client.0.smithi144.stdout:6/395: stat d9/dd/d11/d1b/d2f/c3f 0 2024-09-10T11:29:37.892 INFO:tasks.workunit.client.0.smithi144.stdout:1/546: write d5/dd/f13 [199341,106900] 0 2024-09-10T11:29:37.893 INFO:tasks.workunit.client.0.smithi144.stdout:6/396: read d9/dd/f49 [133660,80906] 0 2024-09-10T11:29:37.894 INFO:tasks.workunit.client.0.smithi144.stdout:3/500: unlink d4/d28/c2c 0 2024-09-10T11:29:37.895 INFO:tasks.workunit.client.0.smithi144.stdout:7/382: dread d5/dc/d12/d2f/f4a [0,4194304] 0 2024-09-10T11:29:37.895 INFO:tasks.workunit.client.1.smithi178.stdout:8/895: getdents d5/db/d10/d35/d117 0 2024-09-10T11:29:37.896 INFO:tasks.workunit.client.0.smithi144.stdout:2/362: mkdir d7/de/d36/d45/d47/d6c/d7c 0 2024-09-10T11:29:37.898 INFO:tasks.workunit.client.0.smithi144.stdout:6/397: symlink d9/dd/d3e/d47/d13/d4a/d5b/l8e 0 2024-09-10T11:29:37.898 INFO:tasks.workunit.client.1.smithi178.stdout:8/896: mkdir d5/db/d10/d20/d52/d5c/dad/d12e 0 2024-09-10T11:29:37.900 INFO:tasks.workunit.client.1.smithi178.stdout:8/897: creat d5/d30/f12f x:0 0 0 2024-09-10T11:29:37.901 INFO:tasks.workunit.client.0.smithi144.stdout:7/383: rename d5/dc/d11/c3b to d5/dc/c7e 0 2024-09-10T11:29:37.903 INFO:tasks.workunit.client.0.smithi144.stdout:1/547: dwrite d5/df/f33 [0,4194304] 0 2024-09-10T11:29:37.904 INFO:tasks.workunit.client.0.smithi144.stdout:3/501: dwrite d4/d28/f76 [0,4194304] 0 2024-09-10T11:29:37.905 INFO:tasks.workunit.client.0.smithi144.stdout:6/398: creat d9/dd/d11/d1b/d2f/d7f/d82/f8f x:0 0 0 2024-09-10T11:29:37.907 INFO:tasks.workunit.client.1.smithi178.stdout:8/898: dwrite d5/db/d3d/de7/f124 [0,4194304] 0 2024-09-10T11:29:37.909 INFO:tasks.workunit.client.0.smithi144.stdout:2/363: dwrite d7/de/d36/f4f [0,4194304] 0 2024-09-10T11:29:37.911 INFO:tasks.workunit.client.1.smithi178.stdout:8/899: unlink d5/d1d/f25 0 2024-09-10T11:29:37.912 INFO:tasks.workunit.client.0.smithi144.stdout:6/399: stat d9/d1f/d2c/l44 0 2024-09-10T11:29:37.916 INFO:tasks.workunit.client.0.smithi144.stdout:6/400: chown d9/dd/d3e/d47/d13/f20 2 1 2024-09-10T11:29:37.916 INFO:tasks.workunit.client.0.smithi144.stdout:2/364: unlink f6 0 2024-09-10T11:29:37.917 INFO:tasks.workunit.client.0.smithi144.stdout:2/365: stat d7/f11 0 2024-09-10T11:29:37.917 INFO:tasks.workunit.client.0.smithi144.stdout:1/548: dread d5/d2a/f34 [0,4194304] 0 2024-09-10T11:29:37.919 INFO:tasks.workunit.client.1.smithi178.stdout:8/900: dwrite d5/db/d10/d20/dc0/fe3 [4194304,4194304] 0 2024-09-10T11:29:37.921 INFO:tasks.workunit.client.0.smithi144.stdout:2/366: chown d7/da/d1e/l1f 117814905 1 2024-09-10T11:29:37.922 INFO:tasks.workunit.client.0.smithi144.stdout:2/367: read - d7/de/d3d/d4e/f72 zero size 2024-09-10T11:29:37.922 INFO:tasks.workunit.client.0.smithi144.stdout:7/384: dread d5/dc/d2a/f4b [0,4194304] 0 2024-09-10T11:29:37.923 INFO:tasks.workunit.client.0.smithi144.stdout:1/549: mknod d5/df/d4b/cb5 0 2024-09-10T11:29:37.927 INFO:tasks.workunit.client.0.smithi144.stdout:1/550: creat d5/df/fb6 x:0 0 0 2024-09-10T11:29:37.929 INFO:tasks.workunit.client.0.smithi144.stdout:7/385: symlink d5/dc/d12/d2f/d7c/l7f 0 2024-09-10T11:29:37.930 INFO:tasks.workunit.client.1.smithi178.stdout:8/901: rename d5/db/d3d/de7/l103 to d5/db/d10/d20/d52/l130 0 2024-09-10T11:29:37.931 INFO:tasks.workunit.client.1.smithi178.stdout:8/902: chown d5/f8 1 1 2024-09-10T11:29:37.931 INFO:tasks.workunit.client.0.smithi144.stdout:2/368: dwrite d7/d29/f46 [0,4194304] 0 2024-09-10T11:29:37.933 INFO:tasks.workunit.client.1.smithi178.stdout:8/903: getdents d5/d123 0 2024-09-10T11:29:37.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:37 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:29:37.935 INFO:tasks.workunit.client.0.smithi144.stdout:7/386: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:37.936 INFO:tasks.workunit.client.1.smithi178.stdout:8/904: mknod d5/db/d10/d70/c131 0 2024-09-10T11:29:37.936 INFO:tasks.workunit.client.0.smithi144.stdout:7/387: fsync d5/dc/d12/d2f/f4d 0 2024-09-10T11:29:37.938 INFO:tasks.workunit.client.1.smithi178.stdout:8/905: creat d5/d1d/f132 x:0 0 0 2024-09-10T11:29:37.941 INFO:tasks.workunit.client.0.smithi144.stdout:7/388: link d5/dc/d2a/d41/l5b d5/dc/d14/d40/d58/d6e/d7a/l80 0 2024-09-10T11:29:37.943 INFO:tasks.workunit.client.0.smithi144.stdout:7/389: truncate d5/dc/d2a/f55 183969 0 2024-09-10T11:29:37.943 INFO:tasks.workunit.client.0.smithi144.stdout:7/390: fdatasync d5/dc/d14/f6b 0 2024-09-10T11:29:37.947 INFO:tasks.workunit.client.0.smithi144.stdout:7/391: rmdir d5/dc/d14/d40/d5e/d63 39 2024-09-10T11:29:37.950 INFO:tasks.workunit.client.0.smithi144.stdout:7/392: write d5/dc/d14/d40/d5e/d63/f66 [231317,79569] 0 2024-09-10T11:29:37.952 INFO:tasks.workunit.client.1.smithi178.stdout:8/906: dread d5/db/d10/d53/f64 [0,4194304] 0 2024-09-10T11:29:37.954 INFO:tasks.workunit.client.0.smithi144.stdout:1/551: rmdir d5/df/d4b 39 2024-09-10T11:29:37.955 INFO:tasks.workunit.client.0.smithi144.stdout:6/401: dread d9/dd/d11/f1c [0,4194304] 0 2024-09-10T11:29:37.956 INFO:tasks.workunit.client.0.smithi144.stdout:1/552: fsync d5/df/d4b/fb4 0 2024-09-10T11:29:37.957 INFO:tasks.workunit.client.0.smithi144.stdout:6/402: write d9/dd/d3e/d47/f88 [425297,92176] 0 2024-09-10T11:29:37.958 INFO:tasks.workunit.client.0.smithi144.stdout:7/393: dwrite d5/dc/d14/d40/f57 [0,4194304] 0 2024-09-10T11:29:37.959 INFO:tasks.workunit.client.0.smithi144.stdout:6/403: chown c7 1 1 2024-09-10T11:29:37.959 INFO:tasks.workunit.client.0.smithi144.stdout:7/394: chown d5/fb 686 1 2024-09-10T11:29:37.959 INFO:tasks.workunit.client.0.smithi144.stdout:1/553: rmdir d5/d4a/d63/d84/dab 39 2024-09-10T11:29:37.965 INFO:tasks.workunit.client.0.smithi144.stdout:6/404: rename d9/dd/d3e/f8c to d9/dd/d11/f90 0 2024-09-10T11:29:37.968 INFO:tasks.workunit.client.0.smithi144.stdout:1/554: dwrite d5/df/d4b/fb4 [0,4194304] 0 2024-09-10T11:29:37.970 INFO:tasks.workunit.client.0.smithi144.stdout:6/405: rename d9/dd/d3a to d9/dd/d3e/d47/d13/d4a/d5b/d91 0 2024-09-10T11:29:37.970 INFO:tasks.workunit.client.0.smithi144.stdout:1/555: fsync d5/dd/f48 0 2024-09-10T11:29:37.972 INFO:tasks.workunit.client.0.smithi144.stdout:6/406: fdatasync d9/dd/d11/d1b/f8d 0 2024-09-10T11:29:37.972 INFO:tasks.workunit.client.0.smithi144.stdout:6/407: write d9/d1f/d2c/f64 [575252,77720] 0 2024-09-10T11:29:37.972 INFO:tasks.workunit.client.0.smithi144.stdout:1/556: symlink d5/d4a/d63/lb7 0 2024-09-10T11:29:37.974 INFO:tasks.workunit.client.0.smithi144.stdout:7/395: truncate d5/dc/d11/d18/f24 1845643 0 2024-09-10T11:29:37.978 INFO:tasks.workunit.client.0.smithi144.stdout:1/557: mknod d5/d4a/d63/d84/dab/cb8 0 2024-09-10T11:29:37.979 INFO:tasks.workunit.client.0.smithi144.stdout:6/408: link d9/dd/d11/f90 d9/dd/d11/d1b/d2f/d7f/f92 0 2024-09-10T11:29:37.981 INFO:tasks.workunit.client.0.smithi144.stdout:7/396: creat d5/dc/d11/f81 x:0 0 0 2024-09-10T11:29:37.982 INFO:tasks.workunit.client.0.smithi144.stdout:6/409: mknod d9/dd/d3e/d47/d13/d60/c93 0 2024-09-10T11:29:37.983 INFO:tasks.workunit.client.0.smithi144.stdout:7/397: write d5/dc/d12/f21 [3547432,118929] 0 2024-09-10T11:29:37.984 INFO:tasks.workunit.client.0.smithi144.stdout:7/398: dread - d5/dc/d11/f81 zero size 2024-09-10T11:29:37.986 INFO:tasks.workunit.client.0.smithi144.stdout:6/410: rmdir d9/dd/d3e/d47/d13/d4a/d5b/d91 39 2024-09-10T11:29:37.986 INFO:tasks.workunit.client.0.smithi144.stdout:1/558: dwrite d5/d4a/d44/d80/f76 [0,4194304] 0 2024-09-10T11:29:37.989 INFO:tasks.workunit.client.0.smithi144.stdout:7/399: mknod d5/dc/d14/d40/d5e/d63/c82 0 2024-09-10T11:29:37.989 INFO:tasks.workunit.client.0.smithi144.stdout:6/411: creat d9/dd/d11/d1b/f94 x:0 0 0 2024-09-10T11:29:37.990 INFO:tasks.workunit.client.0.smithi144.stdout:1/559: rename d5/d4a/d63/d84/l91 to d5/dd/da0/lb9 0 2024-09-10T11:29:37.990 INFO:tasks.workunit.client.0.smithi144.stdout:6/412: dread - d9/dd/d11/d1b/d2f/f8a zero size 2024-09-10T11:29:37.991 INFO:tasks.workunit.client.0.smithi144.stdout:7/400: mknod d5/dc/d14/d40/d5e/d68/c83 0 2024-09-10T11:29:37.994 INFO:tasks.workunit.client.0.smithi144.stdout:6/413: write d9/dd/d3e/d47/d13/d4a/d5b/d91/f51 [7635338,21833] 0 2024-09-10T11:29:37.997 INFO:tasks.workunit.client.0.smithi144.stdout:6/414: symlink d9/dd/d3e/d47/d13/d60/l95 0 2024-09-10T11:29:37.998 INFO:tasks.workunit.client.0.smithi144.stdout:7/401: link d5/dc/c3a d5/dc/d14/d40/d58/d6e/c84 0 2024-09-10T11:29:38.000 INFO:tasks.workunit.client.0.smithi144.stdout:6/415: symlink d9/dd/d11/d1b/d2f/d7f/l96 0 2024-09-10T11:29:38.004 INFO:tasks.workunit.client.0.smithi144.stdout:6/416: creat d9/dd/d3e/d47/d13/d87/f97 x:0 0 0 2024-09-10T11:29:38.007 INFO:tasks.workunit.client.0.smithi144.stdout:6/417: creat d9/dd/d3e/d47/f98 x:0 0 0 2024-09-10T11:29:38.009 INFO:tasks.workunit.client.0.smithi144.stdout:7/402: dwrite d5/dc/d2a/f4b [0,4194304] 0 2024-09-10T11:29:38.009 INFO:tasks.workunit.client.0.smithi144.stdout:1/560: dread d5/d2a/f2d [0,4194304] 0 2024-09-10T11:29:38.012 INFO:tasks.workunit.client.0.smithi144.stdout:1/561: unlink d5/d4a/d44/f75 0 2024-09-10T11:29:38.014 INFO:tasks.workunit.client.0.smithi144.stdout:7/403: creat d5/dc/d12/d2f/d7c/f85 x:0 0 0 2024-09-10T11:29:38.020 INFO:tasks.workunit.client.0.smithi144.stdout:7/404: mknod d5/dc/d14/d40/d58/d6e/c86 0 2024-09-10T11:29:38.021 INFO:tasks.workunit.client.0.smithi144.stdout:6/418: link d9/dd/d3e/d47/d13/f1e d9/dd/d3e/d47/d13/d4a/d5b/d91/f99 0 2024-09-10T11:29:38.027 INFO:tasks.workunit.client.0.smithi144.stdout:1/562: dwrite d5/df/d4b/d51/da7/f7c [0,4194304] 0 2024-09-10T11:29:38.033 INFO:tasks.workunit.client.0.smithi144.stdout:1/563: dread - d5/d4a/d63/fac zero size 2024-09-10T11:29:38.033 INFO:tasks.workunit.client.0.smithi144.stdout:7/405: dwrite d5/dc/f56 [0,4194304] 0 2024-09-10T11:29:38.033 INFO:tasks.workunit.client.0.smithi144.stdout:6/419: dread d9/dd/d3e/d47/d13/d4a/d5b/d91/f51 [4194304,4194304] 0 2024-09-10T11:29:38.034 INFO:tasks.workunit.client.0.smithi144.stdout:6/420: chown d9/d1f/d2c 1814164 1 2024-09-10T11:29:38.039 INFO:tasks.workunit.client.0.smithi144.stdout:1/564: link d5/d2a/d65/f6b d5/dd/d8f/fba 0 2024-09-10T11:29:38.049 INFO:tasks.workunit.client.0.smithi144.stdout:7/406: truncate d5/dc/d2a/f4b 13915 0 2024-09-10T11:29:38.050 INFO:tasks.workunit.client.0.smithi144.stdout:6/421: dread d9/dd/d3e/d47/f28 [0,4194304] 0 2024-09-10T11:29:38.055 INFO:tasks.workunit.client.0.smithi144.stdout:1/565: dread d5/f38 [0,4194304] 0 2024-09-10T11:29:38.058 INFO:tasks.workunit.client.1.smithi178.stdout:9/819: sync 2024-09-10T11:29:38.058 INFO:tasks.workunit.client.0.smithi144.stdout:1/566: rename d5/d2a/f4d to d5/d4a/d63/d84/dab/fbb 0 2024-09-10T11:29:38.061 INFO:tasks.workunit.client.0.smithi144.stdout:6/422: dwrite d9/d1f/f26 [0,4194304] 0 2024-09-10T11:29:38.062 INFO:tasks.workunit.client.0.smithi144.stdout:1/567: unlink d5/d2a/d65/f6b 0 2024-09-10T11:29:38.064 INFO:tasks.workunit.client.0.smithi144.stdout:6/423: symlink d9/dd/d11/d1b/d2f/d7f/l9a 0 2024-09-10T11:29:38.068 INFO:tasks.workunit.client.0.smithi144.stdout:6/424: truncate d9/dd/d3e/d47/d13/f20 2397009 0 2024-09-10T11:29:38.069 INFO:tasks.workunit.client.1.smithi178.stdout:9/820: truncate d2/dc/fcf 377071 0 2024-09-10T11:29:38.070 INFO:tasks.workunit.client.0.smithi144.stdout:6/425: creat d9/dd/d3e/f9b x:0 0 0 2024-09-10T11:29:38.071 INFO:tasks.workunit.client.1.smithi178.stdout:9/821: symlink d2/d9b/df8/d2d/dab/lfc 0 2024-09-10T11:29:38.072 INFO:tasks.workunit.client.0.smithi144.stdout:6/426: mknod d9/dd/d73/c9c 0 2024-09-10T11:29:38.073 INFO:tasks.workunit.client.0.smithi144.stdout:6/427: dread - d9/dd/d11/f56 zero size 2024-09-10T11:29:38.073 INFO:tasks.workunit.client.1.smithi178.stdout:9/822: chown d2/d13/f15 68 1 2024-09-10T11:29:38.073 INFO:tasks.workunit.client.1.smithi178.stdout:9/823: stat d2/d13/d40/d56/dbe/df1 0 2024-09-10T11:29:38.075 INFO:tasks.workunit.client.1.smithi178.stdout:9/824: mkdir d2/d19/dfd 0 2024-09-10T11:29:38.075 INFO:tasks.workunit.client.0.smithi144.stdout:7/407: dread d5/dc/d14/d40/d5e/d63/f66 [0,4194304] 0 2024-09-10T11:29:38.078 INFO:tasks.workunit.client.1.smithi178.stdout:9/825: getdents d2/d9b/df8/da5 0 2024-09-10T11:29:38.078 INFO:tasks.workunit.client.0.smithi144.stdout:7/408: unlink d5/dc/d2a/d41/c54 0 2024-09-10T11:29:38.080 INFO:tasks.workunit.client.0.smithi144.stdout:7/409: write d5/dc/d14/f44 [4092585,12364] 0 2024-09-10T11:29:38.080 INFO:tasks.workunit.client.0.smithi144.stdout:6/428: dread d9/dd/f33 [0,4194304] 0 2024-09-10T11:29:38.081 INFO:tasks.workunit.client.0.smithi144.stdout:6/429: dread - d9/dd/d11/d1b/d2f/d7f/d82/f8f zero size 2024-09-10T11:29:38.082 INFO:tasks.workunit.client.0.smithi144.stdout:6/430: chown d9/dd/d11/c12 7 1 2024-09-10T11:29:38.084 INFO:tasks.workunit.client.0.smithi144.stdout:7/410: creat d5/dc/d11/f87 x:0 0 0 2024-09-10T11:29:38.084 INFO:tasks.workunit.client.0.smithi144.stdout:1/568: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:38.084 INFO:tasks.workunit.client.0.smithi144.stdout:4/450: sync 2024-09-10T11:29:38.084 INFO:tasks.workunit.client.0.smithi144.stdout:5/487: sync 2024-09-10T11:29:38.086 INFO:tasks.workunit.client.0.smithi144.stdout:4/451: chown d14/d1e/d22/f4d 29 1 2024-09-10T11:29:38.086 INFO:tasks.workunit.client.0.smithi144.stdout:7/411: truncate d5/dc/d12/d2b/f43 1268887 0 2024-09-10T11:29:38.088 INFO:tasks.workunit.client.0.smithi144.stdout:5/488: creat d4/d20/d22/d5c/d5d/d64/f96 x:0 0 0 2024-09-10T11:29:38.088 INFO:tasks.workunit.client.0.smithi144.stdout:6/431: link d9/dd/d3e/d47/d13/f1e d9/dd/d11/f9d 0 2024-09-10T11:29:38.088 INFO:tasks.workunit.client.0.smithi144.stdout:7/412: chown d5/f8 457 1 2024-09-10T11:29:38.091 INFO:tasks.workunit.client.0.smithi144.stdout:4/452: creat d14/d67/d69/d90/f9d x:0 0 0 2024-09-10T11:29:38.091 INFO:tasks.workunit.client.0.smithi144.stdout:1/569: dwrite d5/df/fb6 [0,4194304] 0 2024-09-10T11:29:38.091 INFO:tasks.workunit.client.0.smithi144.stdout:6/432: mkdir d9/dd/d3e/d47/d13/d4a/d9e 0 2024-09-10T11:29:38.092 INFO:tasks.workunit.client.0.smithi144.stdout:5/489: mknod d4/d5/d18/d38/d8d/c97 0 2024-09-10T11:29:38.093 INFO:tasks.workunit.client.0.smithi144.stdout:1/570: readlink d5/l5a 0 2024-09-10T11:29:38.095 INFO:tasks.workunit.client.0.smithi144.stdout:6/433: dread - d9/dd/d3e/d47/d13/d87/f97 zero size 2024-09-10T11:29:38.096 INFO:tasks.workunit.client.0.smithi144.stdout:5/490: rmdir d4/d5/d32 39 2024-09-10T11:29:38.098 INFO:tasks.workunit.client.0.smithi144.stdout:1/571: symlink d5/df/d4b/d51/da7/lbc 0 2024-09-10T11:29:38.098 INFO:tasks.workunit.client.0.smithi144.stdout:7/413: dwrite d5/dc/d12/d2f/f4d [0,4194304] 0 2024-09-10T11:29:38.104 INFO:tasks.workunit.client.0.smithi144.stdout:4/453: dwrite d14/d1e/d22/d2b/f33 [0,4194304] 0 2024-09-10T11:29:38.106 INFO:tasks.workunit.client.0.smithi144.stdout:7/414: read d5/dc/d11/f5c [2636392,86915] 0 2024-09-10T11:29:38.106 INFO:tasks.workunit.client.0.smithi144.stdout:1/572: mkdir d5/d4a/d63/d84/dbd 0 2024-09-10T11:29:38.109 INFO:tasks.workunit.client.0.smithi144.stdout:6/434: creat d9/dd/d3e/d47/f9f x:0 0 0 2024-09-10T11:29:38.109 INFO:tasks.workunit.client.0.smithi144.stdout:4/454: mknod d14/d67/d92/c9e 0 2024-09-10T11:29:38.109 INFO:tasks.workunit.client.0.smithi144.stdout:7/415: rmdir d5/dc/d11/d18 39 2024-09-10T11:29:38.110 INFO:tasks.workunit.client.0.smithi144.stdout:5/491: dread d4/d5/d18/d67/f65 [0,4194304] 0 2024-09-10T11:29:38.111 INFO:tasks.workunit.client.0.smithi144.stdout:1/573: mkdir d5/d4a/d63/d84/dbd/dbe 0 2024-09-10T11:29:38.114 INFO:tasks.workunit.client.0.smithi144.stdout:4/455: link f9 d14/d7e/f9f 0 2024-09-10T11:29:38.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:37 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:29:38.121 INFO:tasks.workunit.client.1.smithi178.stdout:5/903: sync 2024-09-10T11:29:38.125 INFO:tasks.workunit.client.1.smithi178.stdout:9/826: rename d2/d13/d75/c80 to d2/d13/d75/d60/dd0/cfe 0 2024-09-10T11:29:38.127 INFO:tasks.workunit.client.1.smithi178.stdout:9/827: symlink d2/d9b/df8/lff 0 2024-09-10T11:29:38.129 INFO:tasks.workunit.client.1.smithi178.stdout:5/904: rename d9/dc/dd/d37/d3e/fa6 to d9/dc/d17/f12a 0 2024-09-10T11:29:38.130 INFO:tasks.workunit.client.1.smithi178.stdout:5/905: dread - d9/dc/dd/d16/f41 zero size 2024-09-10T11:29:38.131 INFO:tasks.workunit.client.1.smithi178.stdout:5/906: write d9/fbe [2476791,28508] 0 2024-09-10T11:29:38.133 INFO:tasks.workunit.client.1.smithi178.stdout:9/828: dwrite d2/d13/d40/d56/de1/ff5 [0,4194304] 0 2024-09-10T11:29:38.135 INFO:tasks.workunit.client.1.smithi178.stdout:5/907: write d9/dc/dd/d37/dbd/dda/db8/fd4 [1135140,52193] 0 2024-09-10T11:29:38.136 INFO:tasks.workunit.client.1.smithi178.stdout:5/908: stat d9/dc/dd/d3f/d5a/d7d/l87 0 2024-09-10T11:29:38.138 INFO:tasks.workunit.client.1.smithi178.stdout:9/829: symlink d2/d19/d5e/l100 0 2024-09-10T11:29:38.140 INFO:tasks.workunit.client.1.smithi178.stdout:5/909: mknod d9/dc/dd/d3f/d5a/d114/c12b 0 2024-09-10T11:29:38.142 INFO:tasks.workunit.client.1.smithi178.stdout:9/830: mknod d2/d19/c101 0 2024-09-10T11:29:38.145 INFO:tasks.workunit.client.1.smithi178.stdout:9/831: symlink d2/d19/l102 0 2024-09-10T11:29:38.147 INFO:tasks.workunit.client.1.smithi178.stdout:9/832: truncate d2/d13/d40/d89/d93/f72 836034 0 2024-09-10T11:29:38.153 INFO:tasks.workunit.client.1.smithi178.stdout:5/910: dwrite d9/dc/dd/d37/f8d [4194304,4194304] 0 2024-09-10T11:29:38.154 INFO:tasks.workunit.client.1.smithi178.stdout:5/911: dread - d9/dc/d17/f11d zero size 2024-09-10T11:29:38.160 INFO:tasks.workunit.client.1.smithi178.stdout:9/833: dread d2/d9b/df8/da5/fd9 [4194304,4194304] 0 2024-09-10T11:29:38.163 INFO:tasks.workunit.client.1.smithi178.stdout:9/834: symlink d2/d19/l103 0 2024-09-10T11:29:38.165 INFO:tasks.workunit.client.1.smithi178.stdout:9/835: creat d2/d13/d40/d56/dbe/f104 x:0 0 0 2024-09-10T11:29:38.168 INFO:tasks.workunit.client.1.smithi178.stdout:9/836: creat d2/d9b/df8/da5/f105 x:0 0 0 2024-09-10T11:29:38.169 INFO:tasks.workunit.client.1.smithi178.stdout:9/837: chown d2/d9b/df8/d77/l84 12611 1 2024-09-10T11:29:38.170 INFO:tasks.workunit.client.1.smithi178.stdout:9/838: mkdir d2/d106 0 2024-09-10T11:29:38.172 INFO:tasks.workunit.client.0.smithi144.stdout:9/456: sync 2024-09-10T11:29:38.172 INFO:tasks.workunit.client.0.smithi144.stdout:0/302: sync 2024-09-10T11:29:38.173 INFO:tasks.workunit.client.0.smithi144.stdout:8/270: sync 2024-09-10T11:29:38.177 INFO:tasks.workunit.client.1.smithi178.stdout:9/839: write d2/d9b/df8/d2a/fa9 [1041701,20852] 0 2024-09-10T11:29:38.178 INFO:tasks.workunit.client.1.smithi178.stdout:9/840: readlink d2/d9b/df8/d2a/lea 0 2024-09-10T11:29:38.179 INFO:tasks.workunit.client.0.smithi144.stdout:6/435: rename d9/dd/d11/d1b/d2f/d7f/d82/f8f to d9/fa0 0 2024-09-10T11:29:38.180 INFO:tasks.workunit.client.1.smithi178.stdout:5/912: dread d9/dc/dd/d37/dbd/dda/fbb [0,4194304] 0 2024-09-10T11:29:38.181 INFO:tasks.workunit.client.1.smithi178.stdout:9/841: mkdir d2/d51/d107 0 2024-09-10T11:29:38.183 INFO:tasks.workunit.client.1.smithi178.stdout:9/842: mknod d2/df/d31/d37/d3d/c108 0 2024-09-10T11:29:38.183 INFO:tasks.workunit.client.0.smithi144.stdout:5/492: dwrite d4/d5/d18/d67/f62 [0,4194304] 0 2024-09-10T11:29:38.184 INFO:tasks.workunit.client.1.smithi178.stdout:5/913: creat d9/dc/dd/d37/d42/d9f/f12c x:0 0 0 2024-09-10T11:29:38.184 INFO:tasks.workunit.client.1.smithi178.stdout:9/843: dread - d2/d13/d75/d60/dd0/fde zero size 2024-09-10T11:29:38.185 INFO:tasks.workunit.client.1.smithi178.stdout:9/844: stat d2/l9d 0 2024-09-10T11:29:38.187 INFO:tasks.workunit.client.0.smithi144.stdout:5/493: chown d4/d5/l27 99966759 1 2024-09-10T11:29:38.188 INFO:tasks.workunit.client.1.smithi178.stdout:5/914: dread d9/dc/dd/d37/dbd/fc6 [0,4194304] 0 2024-09-10T11:29:38.189 INFO:tasks.workunit.client.1.smithi178.stdout:5/915: chown d9/dc/dd/fc1 27292 1 2024-09-10T11:29:38.190 INFO:tasks.workunit.client.1.smithi178.stdout:5/916: chown d9/dc/dd/d37/dbd/dda/db8/le6 305 1 2024-09-10T11:29:38.191 INFO:tasks.workunit.client.1.smithi178.stdout:5/917: fdatasync d9/dc/dd/fad 0 2024-09-10T11:29:38.193 INFO:tasks.workunit.client.0.smithi144.stdout:8/271: creat d0/d9/d21/f59 x:0 0 0 2024-09-10T11:29:38.194 INFO:tasks.workunit.client.0.smithi144.stdout:6/436: symlink d9/dd/d3e/d47/d13/d4a/d5b/d91/la1 0 2024-09-10T11:29:38.194 INFO:tasks.workunit.client.0.smithi144.stdout:1/574: dwrite d5/f6 [0,4194304] 0 2024-09-10T11:29:38.194 INFO:tasks.workunit.client.0.smithi144.stdout:5/494: symlink d4/d5/d18/d51/l98 0 2024-09-10T11:29:38.195 INFO:tasks.workunit.client.0.smithi144.stdout:8/272: fsync d0/d9/d23/d37/f3a 0 2024-09-10T11:29:38.196 INFO:tasks.workunit.client.0.smithi144.stdout:9/457: creat d11/d12/fa6 x:0 0 0 2024-09-10T11:29:38.196 INFO:tasks.workunit.client.0.smithi144.stdout:4/456: dwrite d14/d1e/d22/d34/d3e/d40/d4b/f52 [0,4194304] 0 2024-09-10T11:29:38.199 INFO:tasks.workunit.client.0.smithi144.stdout:1/575: truncate d5/dd/f8d 364336 0 2024-09-10T11:29:38.199 INFO:tasks.workunit.client.0.smithi144.stdout:5/495: chown d4/d5/d18/d67/d59/f6c 8265 1 2024-09-10T11:29:38.199 INFO:tasks.workunit.client.1.smithi178.stdout:9/845: dwrite d2/d8/f50 [0,4194304] 0 2024-09-10T11:29:38.199 INFO:tasks.workunit.client.1.smithi178.stdout:5/918: rename d9/dc/dd/d37/d42/d9f/ldc to d9/dc/dd/l12d 0 2024-09-10T11:29:38.200 INFO:tasks.workunit.client.0.smithi144.stdout:6/437: symlink d9/dd/d11/la2 0 2024-09-10T11:29:38.201 INFO:tasks.workunit.client.0.smithi144.stdout:1/576: stat d5/df/d4b/d51/l60 0 2024-09-10T11:29:38.202 INFO:tasks.workunit.client.0.smithi144.stdout:8/273: fsync d0/d9/f1d 0 2024-09-10T11:29:38.202 INFO:tasks.workunit.client.0.smithi144.stdout:0/303: dread da/f26 [0,4194304] 0 2024-09-10T11:29:38.203 INFO:tasks.workunit.client.1.smithi178.stdout:9/846: write d2/df/f1b [2479483,107634] 0 2024-09-10T11:29:38.203 INFO:tasks.workunit.client.0.smithi144.stdout:6/438: chown d9/dd/d3e/f76 936001970 1 2024-09-10T11:29:38.204 INFO:tasks.workunit.client.0.smithi144.stdout:9/458: mknod d11/d12/ca7 0 2024-09-10T11:29:38.205 INFO:tasks.workunit.client.0.smithi144.stdout:0/304: write da/d3a/f4d [978550,59547] 0 2024-09-10T11:29:38.206 INFO:tasks.workunit.client.1.smithi178.stdout:5/919: creat d9/dc/dd/d3f/d5a/f12e x:0 0 0 2024-09-10T11:29:38.207 INFO:tasks.workunit.client.0.smithi144.stdout:8/274: mkdir d0/d1b/d5a 0 2024-09-10T11:29:38.210 INFO:tasks.workunit.client.1.smithi178.stdout:5/920: symlink d9/dc/dd/d37/d42/d62/l12f 0 2024-09-10T11:29:38.211 INFO:tasks.workunit.client.0.smithi144.stdout:1/577: link d5/dd/f99 d5/df/d4b/fbf 0 2024-09-10T11:29:38.212 INFO:tasks.workunit.client.0.smithi144.stdout:8/275: creat d0/d9/d23/d37/f5b x:0 0 0 2024-09-10T11:29:38.213 INFO:tasks.workunit.client.1.smithi178.stdout:5/921: dread d9/dc/dd/d3f/f98 [0,4194304] 0 2024-09-10T11:29:38.213 INFO:tasks.workunit.client.0.smithi144.stdout:9/459: symlink d11/d36/d2d/d38/la8 0 2024-09-10T11:29:38.213 INFO:tasks.workunit.client.0.smithi144.stdout:1/578: chown d5/d4a/d63/d84/f9e 0 1 2024-09-10T11:29:38.214 INFO:tasks.workunit.client.0.smithi144.stdout:5/496: dwrite d4/d5/d18/f1f [0,4194304] 0 2024-09-10T11:29:38.215 INFO:tasks.workunit.client.0.smithi144.stdout:1/579: fsync d5/d2a/f74 0 2024-09-10T11:29:38.216 INFO:tasks.workunit.client.0.smithi144.stdout:4/457: dread d14/d1e/d22/f2d [0,4194304] 0 2024-09-10T11:29:38.216 INFO:tasks.workunit.client.0.smithi144.stdout:6/439: mknod d9/dd/d3e/d47/d13/d4a/d9e/ca3 0 2024-09-10T11:29:38.219 INFO:tasks.workunit.client.0.smithi144.stdout:6/440: dread - d9/d38/f63 zero size 2024-09-10T11:29:38.219 INFO:tasks.workunit.client.1.smithi178.stdout:5/922: truncate d9/dc/dd/d37/dbd/dda/fa9 1746677 0 2024-09-10T11:29:38.223 INFO:tasks.workunit.client.0.smithi144.stdout:1/580: mkdir d5/d4a/dc0 0 2024-09-10T11:29:38.225 INFO:tasks.workunit.client.0.smithi144.stdout:6/441: chown d9/dd/d3e/d47/d13/f5a 63 1 2024-09-10T11:29:38.226 INFO:tasks.workunit.client.1.smithi178.stdout:5/923: unlink d9/dc/dd/d37/l11e 0 2024-09-10T11:29:38.226 INFO:tasks.workunit.client.1.smithi178.stdout:9/847: dwrite d2/df/d31/d37/d3d/fc7 [4194304,4194304] 0 2024-09-10T11:29:38.227 INFO:tasks.workunit.client.0.smithi144.stdout:4/458: link f9 d14/d1e/d22/d2b/fa0 0 2024-09-10T11:29:38.227 INFO:tasks.workunit.client.1.smithi178.stdout:5/924: readlink d9/dc/dd/d37/d42/d62/daf/df0/lb2 0 2024-09-10T11:29:38.230 INFO:tasks.workunit.client.0.smithi144.stdout:5/497: dwrite d4/d20/f34 [0,4194304] 0 2024-09-10T11:29:38.230 INFO:tasks.workunit.client.0.smithi144.stdout:8/276: dwrite d0/d9/fe [0,4194304] 0 2024-09-10T11:29:38.231 INFO:tasks.workunit.client.0.smithi144.stdout:6/442: write d9/dd/f33 [385063,27830] 0 2024-09-10T11:29:38.237 INFO:tasks.workunit.client.1.smithi178.stdout:9/848: dwrite d2/f26 [4194304,4194304] 0 2024-09-10T11:29:38.237 INFO:tasks.workunit.client.0.smithi144.stdout:9/460: dwrite d11/f17 [4194304,4194304] 0 2024-09-10T11:29:38.237 INFO:tasks.workunit.client.0.smithi144.stdout:8/277: write d0/d9/d23/d26/f4e [776895,61609] 0 2024-09-10T11:29:38.242 INFO:tasks.workunit.client.1.smithi178.stdout:9/849: symlink d2/d13/d75/l109 0 2024-09-10T11:29:38.243 INFO:tasks.workunit.client.1.smithi178.stdout:9/850: write d2/d13/d40/d89/fbd [593002,17974] 0 2024-09-10T11:29:38.245 INFO:tasks.workunit.client.0.smithi144.stdout:5/498: dwrite d4/d5/d18/d38/d6d/f70 [0,4194304] 0 2024-09-10T11:29:38.254 INFO:tasks.workunit.client.0.smithi144.stdout:9/461: creat d11/d12/d1b/d22/fa9 x:0 0 0 2024-09-10T11:29:38.256 INFO:tasks.workunit.client.0.smithi144.stdout:9/462: chown l7 0 1 2024-09-10T11:29:38.257 INFO:tasks.workunit.client.0.smithi144.stdout:5/499: mknod d4/d5/c99 0 2024-09-10T11:29:38.261 INFO:tasks.workunit.client.1.smithi178.stdout:9/851: write d2/d13/d40/d89/d93/d88/fa8 [651930,105977] 0 2024-09-10T11:29:38.261 INFO:tasks.workunit.client.1.smithi178.stdout:2/958: sync 2024-09-10T11:29:38.262 INFO:tasks.workunit.client.1.smithi178.stdout:9/852: stat d2/d13/d75/d60/d66/fa7 0 2024-09-10T11:29:38.264 INFO:tasks.workunit.client.0.smithi144.stdout:0/305: truncate da/f41 304904 0 2024-09-10T11:29:38.264 INFO:tasks.workunit.client.1.smithi178.stdout:9/853: dread - d2/d13/d75/d60/d66/fb9 zero size 2024-09-10T11:29:38.266 INFO:tasks.workunit.client.1.smithi178.stdout:9/854: chown d2/d13/fe3 5548 1 2024-09-10T11:29:38.266 INFO:tasks.workunit.client.0.smithi144.stdout:9/463: dwrite d11/d12/d1b/f61 [0,4194304] 0 2024-09-10T11:29:38.268 INFO:tasks.workunit.client.0.smithi144.stdout:1/581: truncate d5/dd/d64/f9a 2412842 0 2024-09-10T11:29:38.269 INFO:tasks.workunit.client.0.smithi144.stdout:1/582: dread - d5/d4a/d44/f69 zero size 2024-09-10T11:29:38.270 INFO:tasks.workunit.client.0.smithi144.stdout:5/500: getdents d4/d20/d22/d5c/d5d 0 2024-09-10T11:29:38.271 INFO:tasks.workunit.client.1.smithi178.stdout:2/959: dwrite d1/d7/d24/d7d/d84/d55/f10f [0,4194304] 0 2024-09-10T11:29:38.271 INFO:tasks.workunit.client.1.smithi178.stdout:9/855: symlink d2/df/d31/d37/l10a 0 2024-09-10T11:29:38.274 INFO:tasks.workunit.client.1.smithi178.stdout:2/960: stat d1/c83 0 2024-09-10T11:29:38.274 INFO:tasks.workunit.client.0.smithi144.stdout:5/501: mkdir d4/d20/d24/d6f/d9a 0 2024-09-10T11:29:38.275 INFO:tasks.workunit.client.1.smithi178.stdout:9/856: creat d2/d51/d107/f10b x:0 0 0 2024-09-10T11:29:38.275 INFO:tasks.workunit.client.0.smithi144.stdout:8/278: dread d0/d9/d21/d25/f29 [0,4194304] 0 2024-09-10T11:29:38.277 INFO:tasks.workunit.client.0.smithi144.stdout:4/459: dread d14/d1e/d22/d34/d3e/d40/d4b/d4c/f5f [0,4194304] 0 2024-09-10T11:29:38.277 INFO:tasks.workunit.client.1.smithi178.stdout:5/925: dread d9/dc/dd/d3f/dd6/d124/f6e [0,4194304] 0 2024-09-10T11:29:38.278 INFO:tasks.workunit.client.0.smithi144.stdout:5/502: dread d4/d5/d32/d47/f6a [0,4194304] 0 2024-09-10T11:29:38.281 INFO:tasks.workunit.client.0.smithi144.stdout:5/503: readlink d4/d5/d32/l39 0 2024-09-10T11:29:38.282 INFO:tasks.workunit.client.1.smithi178.stdout:2/961: creat d1/d7/d24/d7d/d84/d55/da1/dd6/f144 x:0 0 0 2024-09-10T11:29:38.285 INFO:tasks.workunit.client.0.smithi144.stdout:5/504: read d4/d5/d32/d47/f71 [21758,68961] 0 2024-09-10T11:29:38.285 INFO:tasks.workunit.client.0.smithi144.stdout:6/443: dread d9/dd/f1d [0,4194304] 0 2024-09-10T11:29:38.285 INFO:tasks.workunit.client.1.smithi178.stdout:5/926: dwrite d9/dc/dd/d16/f70 [0,4194304] 0 2024-09-10T11:29:38.287 INFO:tasks.workunit.client.0.smithi144.stdout:3/502: sync 2024-09-10T11:29:38.288 INFO:tasks.workunit.client.1.smithi178.stdout:2/962: rmdir d1/d7/d8e/de9 39 2024-09-10T11:29:38.288 INFO:tasks.workunit.client.0.smithi144.stdout:4/460: dwrite d14/d6f/f9c [0,4194304] 0 2024-09-10T11:29:38.289 INFO:tasks.workunit.client.0.smithi144.stdout:5/505: dread - d4/d20/d22/d5c/d5d/d64/f8e zero size 2024-09-10T11:29:38.295 INFO:tasks.workunit.client.1.smithi178.stdout:5/927: dwrite d9/dc/dd/d37/d3e/f94 [0,4194304] 0 2024-09-10T11:29:38.296 INFO:tasks.workunit.client.1.smithi178.stdout:5/928: stat d9/dc/d17/f3d 0 2024-09-10T11:29:38.300 INFO:tasks.workunit.client.1.smithi178.stdout:5/929: creat d9/dc/d17/f130 x:0 0 0 2024-09-10T11:29:38.301 INFO:tasks.workunit.client.0.smithi144.stdout:4/461: symlink d14/d1e/d65/la1 0 2024-09-10T11:29:38.302 INFO:tasks.workunit.client.0.smithi144.stdout:0/306: write da/f30 [2658349,126702] 0 2024-09-10T11:29:38.302 INFO:tasks.workunit.client.0.smithi144.stdout:4/462: chown d14/f54 853 1 2024-09-10T11:29:38.303 INFO:tasks.workunit.client.1.smithi178.stdout:5/930: dread - d9/dc/dd/d37/d42/d62/daf/df0/fcb zero size 2024-09-10T11:29:38.304 INFO:tasks.workunit.client.0.smithi144.stdout:9/464: write d11/d12/d1b/f20 [1481640,52085] 0 2024-09-10T11:29:38.307 INFO:tasks.workunit.client.0.smithi144.stdout:3/503: creat d4/d28/d67/d80/d1d/fac x:0 0 0 2024-09-10T11:29:38.307 INFO:tasks.workunit.client.0.smithi144.stdout:6/444: creat d9/dd/d3e/d47/d13/d4a/d5b/fa4 x:0 0 0 2024-09-10T11:29:38.308 INFO:tasks.workunit.client.1.smithi178.stdout:2/963: write d1/d7/d24/d7d/d84/f98 [228395,51172] 0 2024-09-10T11:29:38.309 INFO:tasks.workunit.client.0.smithi144.stdout:0/307: symlink da/d48/l5a 0 2024-09-10T11:29:38.311 INFO:tasks.workunit.client.0.smithi144.stdout:5/506: creat d4/d5/d18/d38/d6d/d8b/f9b x:0 0 0 2024-09-10T11:29:38.311 INFO:tasks.workunit.client.0.smithi144.stdout:4/463: rename d14/l36 to d14/d1e/d65/d8a/d8f/la2 0 2024-09-10T11:29:38.312 INFO:tasks.workunit.client.0.smithi144.stdout:0/308: readlink da/l25 0 2024-09-10T11:29:38.315 INFO:tasks.workunit.client.1.smithi178.stdout:2/964: mkdir d1/d41/d14/d11/db9/def/d145 0 2024-09-10T11:29:38.316 INFO:tasks.workunit.client.0.smithi144.stdout:5/507: write d4/d20/d24/f92 [4529915,97501] 0 2024-09-10T11:29:38.316 INFO:tasks.workunit.client.0.smithi144.stdout:0/309: write da/f46 [4679264,80461] 0 2024-09-10T11:29:38.320 INFO:tasks.workunit.client.0.smithi144.stdout:8/279: dwrite d0/d1b/f2f [0,4194304] 0 2024-09-10T11:29:38.320 INFO:tasks.workunit.client.0.smithi144.stdout:0/310: truncate da/d44/f54 737071 0 2024-09-10T11:29:38.320 INFO:tasks.workunit.client.1.smithi178.stdout:2/965: creat d1/d22/d4d/d9b/f146 x:0 0 0 2024-09-10T11:29:38.322 INFO:tasks.workunit.client.0.smithi144.stdout:8/280: dread - d0/d1b/f4c zero size 2024-09-10T11:29:38.323 INFO:tasks.workunit.client.1.smithi178.stdout:5/931: rmdir d9 39 2024-09-10T11:29:38.324 INFO:tasks.workunit.client.1.smithi178.stdout:2/966: getdents d1/d41/d8d/db1 0 2024-09-10T11:29:38.324 INFO:tasks.workunit.client.0.smithi144.stdout:8/281: fsync d0/d9/f1d 0 2024-09-10T11:29:38.325 INFO:tasks.workunit.client.0.smithi144.stdout:8/282: readlink d0/d9/d23/l3c 0 2024-09-10T11:29:38.325 INFO:tasks.workunit.client.1.smithi178.stdout:5/932: chown d9/dc/dd/d37 105453 1 2024-09-10T11:29:38.326 INFO:tasks.workunit.client.0.smithi144.stdout:4/464: dwrite d14/d1e/d22/d34/d3e/d40/f76 [0,4194304] 0 2024-09-10T11:29:38.327 INFO:tasks.workunit.client.1.smithi178.stdout:2/967: write d1/d7/d24/d46/d88/d9d/dab/f15 [2550990,108657] 0 2024-09-10T11:29:38.328 INFO:tasks.workunit.client.1.smithi178.stdout:5/933: unlink d9/dc/dd/fad 0 2024-09-10T11:29:38.329 INFO:tasks.workunit.client.0.smithi144.stdout:5/508: getdents d4/d5/d32 0 2024-09-10T11:29:38.329 INFO:tasks.workunit.client.1.smithi178.stdout:5/934: fsync d9/dc/ff4 0 2024-09-10T11:29:38.331 INFO:tasks.workunit.client.0.smithi144.stdout:4/465: symlink d14/d1e/d65/la3 0 2024-09-10T11:29:38.332 INFO:tasks.workunit.client.1.smithi178.stdout:2/968: creat d1/d41/d5d/f147 x:0 0 0 2024-09-10T11:29:38.332 INFO:tasks.workunit.client.0.smithi144.stdout:4/466: dread - d14/d6f/f96 zero size 2024-09-10T11:29:38.333 INFO:tasks.workunit.client.1.smithi178.stdout:2/969: read d1/d41/d14/d11/db9/fe3 [683664,21337] 0 2024-09-10T11:29:38.334 INFO:tasks.workunit.client.0.smithi144.stdout:4/467: dread - d14/d1e/d22/d34/d3e/d40/d4b/d4c/f79 zero size 2024-09-10T11:29:38.334 INFO:tasks.workunit.client.1.smithi178.stdout:5/935: chown d9/c77 952 1 2024-09-10T11:29:38.337 INFO:tasks.workunit.client.1.smithi178.stdout:2/970: creat d1/d7/d24/d43/f148 x:0 0 0 2024-09-10T11:29:38.337 INFO:tasks.workunit.client.1.smithi178.stdout:5/936: dread d9/dc/dd/d3f/f98 [0,4194304] 0 2024-09-10T11:29:38.339 INFO:tasks.workunit.client.0.smithi144.stdout:8/283: getdents d0/d9/d23 0 2024-09-10T11:29:38.340 INFO:tasks.workunit.client.1.smithi178.stdout:2/971: mknod d1/c149 0 2024-09-10T11:29:38.340 INFO:tasks.workunit.client.0.smithi144.stdout:4/468: mkdir d14/d1e/d83/da4 0 2024-09-10T11:29:38.341 INFO:tasks.workunit.client.1.smithi178.stdout:5/937: truncate d9/dc/dd/f69 2783844 0 2024-09-10T11:29:38.341 INFO:tasks.workunit.client.0.smithi144.stdout:8/284: readlink d0/d9/d23/l44 0 2024-09-10T11:29:38.342 INFO:tasks.workunit.client.0.smithi144.stdout:3/504: truncate d4/d28/d67/d80/d17/d18/d34/d65/d75/f84 3281641 0 2024-09-10T11:29:38.343 INFO:tasks.workunit.client.0.smithi144.stdout:9/465: dwrite d11/f21 [0,4194304] 0 2024-09-10T11:29:38.343 INFO:tasks.workunit.client.0.smithi144.stdout:5/509: write d4/d5/d32/d47/f6a [885579,85146] 0 2024-09-10T11:29:38.344 INFO:tasks.workunit.client.1.smithi178.stdout:2/972: symlink d1/d7/d24/d46/d88/d9d/dab/d5e/db3/l14a 0 2024-09-10T11:29:38.347 INFO:tasks.workunit.client.0.smithi144.stdout:9/466: chown d11/d12/c19 12 1 2024-09-10T11:29:38.347 INFO:tasks.workunit.client.1.smithi178.stdout:2/973: rmdir d1/d7/d24/d46/d88/d9d/dab 39 2024-09-10T11:29:38.349 INFO:tasks.workunit.client.0.smithi144.stdout:6/445: dwrite d9/dd/d3e/d47/d13/d4a/d5b/d91/f99 [0,4194304] 0 2024-09-10T11:29:38.350 INFO:tasks.workunit.client.0.smithi144.stdout:3/505: write d4/dd/d21/f62 [3747928,12009] 0 2024-09-10T11:29:38.350 INFO:tasks.workunit.client.0.smithi144.stdout:4/469: creat d14/d1e/d22/fa5 x:0 0 0 2024-09-10T11:29:38.355 INFO:tasks.workunit.client.1.smithi178.stdout:5/938: dread d9/dc/ff4 [0,4194304] 0 2024-09-10T11:29:38.359 INFO:tasks.workunit.client.0.smithi144.stdout:8/285: dwrite d0/fc [4194304,4194304] 0 2024-09-10T11:29:38.359 INFO:tasks.workunit.client.0.smithi144.stdout:6/446: creat d9/dd/d3e/d47/d13/d4a/fa5 x:0 0 0 2024-09-10T11:29:38.361 INFO:tasks.workunit.client.0.smithi144.stdout:3/506: symlink d4/d28/d67/lad 0 2024-09-10T11:29:38.363 INFO:tasks.workunit.client.0.smithi144.stdout:6/447: fdatasync d9/dd/f1a 0 2024-09-10T11:29:38.364 INFO:tasks.workunit.client.1.smithi178.stdout:5/939: dread d9/dc/dd/d16/f27 [4194304,4194304] 0 2024-09-10T11:29:38.366 INFO:tasks.workunit.client.1.smithi178.stdout:5/940: readlink d9/dc/dd/d3f/d5a/dbc/le5 0 2024-09-10T11:29:38.367 INFO:tasks.workunit.client.1.smithi178.stdout:5/941: truncate d9/dc/d17/db0/fd1 4749136 0 2024-09-10T11:29:38.375 INFO:tasks.workunit.client.1.smithi178.stdout:5/942: dwrite d9/dc/dd/d37/dbd/dda/db8/fed [0,4194304] 0 2024-09-10T11:29:38.377 INFO:tasks.workunit.client.0.smithi144.stdout:8/286: dwrite d0/d9/d21/d25/d4f/f56 [0,4194304] 0 2024-09-10T11:29:38.377 INFO:tasks.workunit.client.0.smithi144.stdout:5/510: dread d4/d20/d24/d2d/d4b/f4e [0,4194304] 0 2024-09-10T11:29:38.378 INFO:tasks.workunit.client.0.smithi144.stdout:6/448: dread d9/dd/d3e/d47/d13/d4a/d5b/d91/f6e [0,4194304] 0 2024-09-10T11:29:38.379 INFO:tasks.workunit.client.1.smithi178.stdout:5/943: rmdir d9/dc/dd/d3f/d5a/d7d 39 2024-09-10T11:29:38.380 INFO:tasks.workunit.client.0.smithi144.stdout:9/467: dread d11/d12/d1b/f27 [0,4194304] 0 2024-09-10T11:29:38.381 INFO:tasks.workunit.client.0.smithi144.stdout:4/470: dread d14/d1e/d22/d2b/fa0 [0,4194304] 0 2024-09-10T11:29:38.381 INFO:tasks.workunit.client.0.smithi144.stdout:8/287: dread - d0/d9/d23/d37/f5b zero size 2024-09-10T11:29:38.382 INFO:tasks.workunit.client.0.smithi144.stdout:8/288: stat d0/d9/d1f/d34 0 2024-09-10T11:29:38.384 INFO:tasks.workunit.client.1.smithi178.stdout:5/944: mkdir d9/dc/dd/d3f/dd6/d124/d6d/dcc/d131 0 2024-09-10T11:29:38.384 INFO:tasks.workunit.client.0.smithi144.stdout:6/449: rmdir d9/d1f/d2c 39 2024-09-10T11:29:38.384 INFO:tasks.workunit.client.0.smithi144.stdout:8/289: write d0/d9/fe [3443863,20838] 0 2024-09-10T11:29:38.384 INFO:tasks.workunit.client.0.smithi144.stdout:5/511: mknod d4/d5/d18/d38/c9c 0 2024-09-10T11:29:38.386 INFO:tasks.workunit.client.0.smithi144.stdout:9/468: mkdir d11/d12/d6f/daa 0 2024-09-10T11:29:38.387 INFO:tasks.workunit.client.1.smithi178.stdout:5/945: creat d9/dc/d17/d1d/f132 x:0 0 0 2024-09-10T11:29:38.390 INFO:tasks.workunit.client.0.smithi144.stdout:4/471: mknod d14/d1e/d22/d2b/d35/d3c/ca6 0 2024-09-10T11:29:38.391 INFO:tasks.workunit.client.0.smithi144.stdout:5/512: symlink d4/d5/d18/l9d 0 2024-09-10T11:29:38.392 INFO:tasks.workunit.client.0.smithi144.stdout:4/472: chown d14/d1e/d22/f2d 28879317 1 2024-09-10T11:29:38.392 INFO:tasks.workunit.client.0.smithi144.stdout:6/450: mknod d9/ca6 0 2024-09-10T11:29:38.396 INFO:tasks.workunit.client.1.smithi178.stdout:5/946: dwrite d9/dc/d17/d1d/fce [0,4194304] 0 2024-09-10T11:29:38.396 INFO:tasks.workunit.client.0.smithi144.stdout:5/513: creat d4/d5/d56/f9e x:0 0 0 2024-09-10T11:29:38.397 INFO:tasks.workunit.client.1.smithi178.stdout:5/947: fsync d9/dc/dd/d37/d42/f4e 0 2024-09-10T11:29:38.397 INFO:tasks.workunit.client.0.smithi144.stdout:8/290: dwrite d0/d9/d23/d26/f4a [0,4194304] 0 2024-09-10T11:29:38.400 INFO:tasks.workunit.client.1.smithi178.stdout:7/710: sync 2024-09-10T11:29:38.400 INFO:tasks.workunit.client.1.smithi178.stdout:4/667: sync 2024-09-10T11:29:38.400 INFO:tasks.workunit.client.0.smithi144.stdout:6/451: mknod d9/dd/ca7 0 2024-09-10T11:29:38.401 INFO:tasks.workunit.client.0.smithi144.stdout:5/514: readlink d4/d5/d18/l53 0 2024-09-10T11:29:38.402 INFO:tasks.workunit.client.0.smithi144.stdout:3/507: dwrite d4/d28/f38 [0,4194304] 0 2024-09-10T11:29:38.403 INFO:tasks.workunit.client.1.smithi178.stdout:5/948: link d9/dc/dd/d37/d3e/fdb d9/dc/dd/d3f/d5a/d114/f133 0 2024-09-10T11:29:38.404 INFO:tasks.workunit.client.1.smithi178.stdout:4/668: mknod d0/d9/d3d/d4f/ce6 0 2024-09-10T11:29:38.405 INFO:tasks.workunit.client.0.smithi144.stdout:9/469: truncate d11/d12/d1b/f20 404430 0 2024-09-10T11:29:38.406 INFO:tasks.workunit.client.1.smithi178.stdout:7/711: truncate d6/d10/d1d/fb8 579829 0 2024-09-10T11:29:38.406 INFO:tasks.workunit.client.0.smithi144.stdout:8/291: rename d0/d9/d21/d25/f29 to d0/d9/d21/f5c 0 2024-09-10T11:29:38.406 INFO:tasks.workunit.client.1.smithi178.stdout:4/669: chown d0/d8/da/d1a/d51/fb0 333421704 1 2024-09-10T11:29:38.406 INFO:tasks.workunit.client.0.smithi144.stdout:4/473: dread d14/d1e/d22/f4d [0,4194304] 0 2024-09-10T11:29:38.408 INFO:tasks.workunit.client.1.smithi178.stdout:5/949: symlink d9/dc/dd/d3f/dd6/d124/d6d/dcc/d131/l134 0 2024-09-10T11:29:38.409 INFO:tasks.workunit.client.0.smithi144.stdout:9/470: write d11/d12/d1b/f77 [1192256,42254] 0 2024-09-10T11:29:38.410 INFO:tasks.workunit.client.0.smithi144.stdout:5/515: dread d4/d5/f25 [0,4194304] 0 2024-09-10T11:29:38.410 INFO:tasks.workunit.client.1.smithi178.stdout:5/950: fsync d9/dc/d17/d1d/f54 0 2024-09-10T11:29:38.410 INFO:tasks.workunit.client.1.smithi178.stdout:7/712: mkdir d6/d10/d1e/d2d/d2b/da0/d101 0 2024-09-10T11:29:38.411 INFO:tasks.workunit.client.0.smithi144.stdout:5/516: chown d4/d20/d24 5893 1 2024-09-10T11:29:38.414 INFO:tasks.workunit.client.0.smithi144.stdout:4/474: read d14/d1e/d22/d2b/d35/f5b [1295213,9695] 0 2024-09-10T11:29:38.414 INFO:tasks.workunit.client.1.smithi178.stdout:4/670: fsync d0/d9/dc/d8e/dbf/f93 0 2024-09-10T11:29:38.418 INFO:tasks.workunit.client.0.smithi144.stdout:3/508: creat d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93/fae x:0 0 0 2024-09-10T11:29:38.419 INFO:tasks.workunit.client.1.smithi178.stdout:5/951: link d9/dc/dd/d16/ce7 d9/dc/dd/d3f/dd6/d124/d6d/dcc/c135 0 2024-09-10T11:29:38.420 INFO:tasks.workunit.client.0.smithi144.stdout:6/452: getdents d9/d1f/d2c/d4f 0 2024-09-10T11:29:38.421 INFO:tasks.workunit.client.1.smithi178.stdout:7/713: dwrite d6/d10/d34/f6e [0,4194304] 0 2024-09-10T11:29:38.422 INFO:tasks.workunit.client.0.smithi144.stdout:5/517: dwrite d4/d5/f23 [0,4194304] 0 2024-09-10T11:29:38.424 INFO:tasks.workunit.client.0.smithi144.stdout:4/475: read d14/d1e/d22/d34/d3e/d40/f44 [3912953,74828] 0 2024-09-10T11:29:38.426 INFO:tasks.workunit.client.0.smithi144.stdout:6/453: symlink d9/dd/d11/d1b/d2f/la8 0 2024-09-10T11:29:38.426 INFO:tasks.workunit.client.0.smithi144.stdout:9/471: dwrite d11/d12/d1b/f77 [0,4194304] 0 2024-09-10T11:29:38.428 INFO:tasks.workunit.client.0.smithi144.stdout:9/472: write d11/f21 [1648363,106877] 0 2024-09-10T11:29:38.428 INFO:tasks.workunit.client.0.smithi144.stdout:4/476: write d14/d1e/d22/d34/d3e/f97 [853167,12741] 0 2024-09-10T11:29:38.429 INFO:tasks.workunit.client.1.smithi178.stdout:5/952: dwrite d9/dc/d17/db0/fd1 [0,4194304] 0 2024-09-10T11:29:38.429 INFO:tasks.workunit.client.0.smithi144.stdout:3/509: mknod d4/d28/d67/d96/caf 0 2024-09-10T11:29:38.434 INFO:tasks.workunit.client.1.smithi178.stdout:5/953: creat d9/dc/dd/d37/dbd/f136 x:0 0 0 2024-09-10T11:29:38.437 INFO:tasks.workunit.client.0.smithi144.stdout:9/473: chown d11/d12/d1b/d41/c5e 42195 1 2024-09-10T11:29:38.438 INFO:tasks.workunit.client.0.smithi144.stdout:3/510: stat d4/d28/d67/d80/d1d/l3d 0 2024-09-10T11:29:38.440 INFO:tasks.workunit.client.1.smithi178.stdout:5/954: readlink d9/dc/d17/l24 0 2024-09-10T11:29:38.441 INFO:tasks.workunit.client.0.smithi144.stdout:6/454: mknod d9/dd/ca9 0 2024-09-10T11:29:38.441 INFO:tasks.workunit.client.0.smithi144.stdout:5/518: dwrite d4/d5/d56/f9e [0,4194304] 0 2024-09-10T11:29:38.441 INFO:tasks.workunit.client.0.smithi144.stdout:6/455: stat c4 0 2024-09-10T11:29:38.445 INFO:tasks.workunit.client.0.smithi144.stdout:6/456: dread d9/dd/d11/d1b/f8d [0,4194304] 0 2024-09-10T11:29:38.445 INFO:tasks.workunit.client.1.smithi178.stdout:5/955: mknod d9/dc/dd/d37/dbd/d127/c137 0 2024-09-10T11:29:38.448 INFO:tasks.workunit.client.1.smithi178.stdout:5/956: mkdir d9/dc/dd/d37/d42/d62/daf/d138 0 2024-09-10T11:29:38.449 INFO:tasks.workunit.client.0.smithi144.stdout:5/519: symlink d4/d20/d24/d6f/d74/l9f 0 2024-09-10T11:29:38.449 INFO:tasks.workunit.client.0.smithi144.stdout:5/520: chown d4/d5/f25 30537 1 2024-09-10T11:29:38.450 INFO:tasks.workunit.client.1.smithi178.stdout:5/957: chown d9/dc/d17/d1d/f132 11209006 1 2024-09-10T11:29:38.450 INFO:tasks.workunit.client.0.smithi144.stdout:6/457: symlink d9/dd/laa 0 2024-09-10T11:29:38.451 INFO:tasks.workunit.client.0.smithi144.stdout:5/521: stat d4/d20/d24/d2d/d4b/c73 0 2024-09-10T11:29:38.452 INFO:tasks.workunit.client.0.smithi144.stdout:7/416: sync 2024-09-10T11:29:38.452 INFO:tasks.workunit.client.0.smithi144.stdout:2/369: sync 2024-09-10T11:29:38.453 INFO:tasks.workunit.client.0.smithi144.stdout:6/458: mkdir d9/dd/d11/d1b/d2f/dab 0 2024-09-10T11:29:38.454 INFO:tasks.workunit.client.0.smithi144.stdout:7/417: fsync d5/dc/d11/f87 0 2024-09-10T11:29:38.454 INFO:tasks.workunit.client.0.smithi144.stdout:5/522: write d4/d20/d22/d82/f85 [657951,106574] 0 2024-09-10T11:29:38.454 INFO:tasks.workunit.client.1.smithi178.stdout:5/958: symlink d9/dc/dd/d37/d42/l139 0 2024-09-10T11:29:38.459 INFO:tasks.workunit.client.0.smithi144.stdout:5/523: truncate d4/d5/d18/d51/f88 145244 0 2024-09-10T11:29:38.460 INFO:tasks.workunit.client.0.smithi144.stdout:6/459: symlink d9/d1f/d2c/lac 0 2024-09-10T11:29:38.460 INFO:tasks.workunit.client.1.smithi178.stdout:4/671: dread d0/d8/da/d1a/f4b [0,4194304] 0 2024-09-10T11:29:38.462 INFO:tasks.workunit.client.0.smithi144.stdout:6/460: write d9/dd/d11/d1b/d2f/f66 [90710,39392] 0 2024-09-10T11:29:38.463 INFO:tasks.workunit.client.0.smithi144.stdout:7/418: creat d5/dc/d12/f88 x:0 0 0 2024-09-10T11:29:38.464 INFO:tasks.workunit.client.0.smithi144.stdout:2/370: getdents d7/de/d36/d45 0 2024-09-10T11:29:38.464 INFO:tasks.workunit.client.1.smithi178.stdout:7/714: dwrite d6/d10/d1d/d22/d3b/f65 [0,4194304] 0 2024-09-10T11:29:38.465 INFO:tasks.workunit.client.0.smithi144.stdout:2/371: readlink l0 0 2024-09-10T11:29:38.465 INFO:tasks.workunit.client.0.smithi144.stdout:4/477: dread d14/d1e/f2f [0,4194304] 0 2024-09-10T11:29:38.465 INFO:tasks.workunit.client.0.smithi144.stdout:5/524: fsync d4/d20/d24/f3e 0 2024-09-10T11:29:38.466 INFO:tasks.workunit.client.1.smithi178.stdout:7/715: dread - d6/d10/d1d/d4a/d79/d83/fcd zero size 2024-09-10T11:29:38.466 INFO:tasks.workunit.client.1.smithi178.stdout:4/672: creat d0/d8/da/fe7 x:0 0 0 2024-09-10T11:29:38.466 INFO:tasks.workunit.client.1.smithi178.stdout:5/959: write d9/dc/dd/d37/d3e/fdb [209574,74433] 0 2024-09-10T11:29:38.468 INFO:tasks.workunit.client.1.smithi178.stdout:5/960: write d9/dc/d17/f130 [179587,77193] 0 2024-09-10T11:29:38.469 INFO:tasks.workunit.client.1.smithi178.stdout:7/716: truncate d6/d10/d1d/d22/fbb 832212 0 2024-09-10T11:29:38.469 INFO:tasks.workunit.client.1.smithi178.stdout:4/673: write d0/d9/dc/f34 [5131309,26559] 0 2024-09-10T11:29:38.470 INFO:tasks.workunit.client.0.smithi144.stdout:4/478: write d14/d1e/d22/d34/d3e/d40/d4b/f57 [963187,47767] 0 2024-09-10T11:29:38.472 INFO:tasks.workunit.client.0.smithi144.stdout:2/372: mknod d7/de/d5f/c7d 0 2024-09-10T11:29:38.472 INFO:tasks.workunit.client.1.smithi178.stdout:4/674: truncate d0/d8/df/f20 1695184 0 2024-09-10T11:29:38.473 INFO:tasks.workunit.client.1.smithi178.stdout:4/675: fsync d0/d63/f78 0 2024-09-10T11:29:38.474 INFO:tasks.workunit.client.1.smithi178.stdout:5/961: dread d9/dc/dd/d37/d3e/f94 [0,4194304] 0 2024-09-10T11:29:38.476 INFO:tasks.workunit.client.1.smithi178.stdout:7/717: rmdir d6/d10/d1e/d2d/d2b/da0/dc0 39 2024-09-10T11:29:38.477 INFO:tasks.workunit.client.0.smithi144.stdout:2/373: truncate d7/de/f2b 5647573 0 2024-09-10T11:29:38.477 INFO:tasks.workunit.client.0.smithi144.stdout:3/511: dread d4/d28/d67/d80/d17/d18/d34/d65/f88 [0,4194304] 0 2024-09-10T11:29:38.478 INFO:tasks.workunit.client.0.smithi144.stdout:4/479: mkdir d14/d1e/d22/d2b/d35/d3c/da7 0 2024-09-10T11:29:38.478 INFO:tasks.workunit.client.0.smithi144.stdout:6/461: dwrite d9/dd/d3e/d47/d13/f5a [0,4194304] 0 2024-09-10T11:29:38.479 INFO:tasks.workunit.client.1.smithi178.stdout:5/962: fdatasync d9/dc/dd/d3f/dd6/d124/d6d/dcc/dd8/f100 0 2024-09-10T11:29:38.481 INFO:tasks.workunit.client.1.smithi178.stdout:4/676: dwrite d0/d8/da/fe7 [0,4194304] 0 2024-09-10T11:29:38.482 INFO:tasks.workunit.client.1.smithi178.stdout:9/857: sync 2024-09-10T11:29:38.482 INFO:tasks.workunit.client.1.smithi178.stdout:0/751: sync 2024-09-10T11:29:38.482 INFO:tasks.workunit.client.1.smithi178.stdout:6/830: sync 2024-09-10T11:29:38.482 INFO:tasks.workunit.client.1.smithi178.stdout:8/907: sync 2024-09-10T11:29:38.482 INFO:tasks.workunit.client.1.smithi178.stdout:1/724: sync 2024-09-10T11:29:38.483 INFO:tasks.workunit.client.1.smithi178.stdout:3/867: sync 2024-09-10T11:29:38.483 INFO:tasks.workunit.client.0.smithi144.stdout:2/374: write d7/d29/f59 [23129,28239] 0 2024-09-10T11:29:38.486 INFO:tasks.workunit.client.0.smithi144.stdout:5/525: dwrite d4/d20/d22/d82/f85 [0,4194304] 0 2024-09-10T11:29:38.486 INFO:tasks.workunit.client.1.smithi178.stdout:7/718: creat d6/d10/d34/d6a/d6f/db1/f102 x:0 0 0 2024-09-10T11:29:38.487 INFO:tasks.workunit.client.1.smithi178.stdout:8/908: stat d5/db/d10/d53/f64 0 2024-09-10T11:29:38.487 INFO:tasks.workunit.client.1.smithi178.stdout:1/725: write d3/d9/d98/dac/d3a/fc7 [972802,42421] 0 2024-09-10T11:29:38.490 INFO:tasks.workunit.client.0.smithi144.stdout:3/512: write d4/dd/d21/d2f/f60 [585834,118358] 0 2024-09-10T11:29:38.490 INFO:tasks.workunit.client.0.smithi144.stdout:6/462: mknod d9/d1f/d2c/cad 0 2024-09-10T11:29:38.491 INFO:tasks.workunit.client.1.smithi178.stdout:3/868: mkdir db/d9b/d11b 0 2024-09-10T11:29:38.492 INFO:tasks.workunit.client.0.smithi144.stdout:4/480: symlink d14/d1e/d22/d2b/d35/d3c/la8 0 2024-09-10T11:29:38.492 INFO:tasks.workunit.client.0.smithi144.stdout:5/526: fdatasync d4/d5/d32/f33 0 2024-09-10T11:29:38.492 INFO:tasks.workunit.client.1.smithi178.stdout:0/752: unlink d0/d12/feb 0 2024-09-10T11:29:38.493 INFO:tasks.workunit.client.1.smithi178.stdout:7/719: write d6/d10/d1d/d22/d50/d98/da6/dd5/ffc [899986,108165] 0 2024-09-10T11:29:38.494 INFO:tasks.workunit.client.0.smithi144.stdout:9/474: dwrite d11/d36/f80 [0,4194304] 0 2024-09-10T11:29:38.494 INFO:tasks.workunit.client.0.smithi144.stdout:3/513: chown d4/d28/d67/d80/d17/d18 10 1 2024-09-10T11:29:38.495 INFO:tasks.workunit.client.1.smithi178.stdout:9/858: rmdir d2/df/d31/d37/d3d 39 2024-09-10T11:29:38.496 INFO:tasks.workunit.client.0.smithi144.stdout:6/463: read d9/dd/d3e/d47/d13/f5a [2913834,49190] 0 2024-09-10T11:29:38.497 INFO:tasks.workunit.client.0.smithi144.stdout:2/375: truncate d7/da/f13 2602192 0 2024-09-10T11:29:38.498 INFO:tasks.workunit.client.0.smithi144.stdout:6/464: write d9/dd/d3e/d47/f9f [533427,1224] 0 2024-09-10T11:29:38.500 INFO:tasks.workunit.client.1.smithi178.stdout:0/753: symlink d0/dc/d1f/d2d/dd1/led 0 2024-09-10T11:29:38.504 INFO:tasks.workunit.client.0.smithi144.stdout:3/514: rmdir d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93 39 2024-09-10T11:29:38.505 INFO:tasks.workunit.client.0.smithi144.stdout:6/465: read - d9/dd/d3e/d47/d13/d4a/d5b/fa4 zero size 2024-09-10T11:29:38.506 INFO:tasks.workunit.client.1.smithi178.stdout:9/859: dwrite d2/d8/fa [0,4194304] 0 2024-09-10T11:29:38.508 INFO:tasks.workunit.client.0.smithi144.stdout:5/527: dwrite d4/d20/d22/d5c/d5d/d64/f8e [0,4194304] 0 2024-09-10T11:29:38.508 INFO:tasks.workunit.client.0.smithi144.stdout:9/475: rename d11/d12/d1b/d22/d53 to d11/d12/d1b/d41/d8f/dab 0 2024-09-10T11:29:38.508 INFO:tasks.workunit.client.0.smithi144.stdout:7/419: dread d5/dc/f10 [0,4194304] 0 2024-09-10T11:29:38.509 INFO:tasks.workunit.client.1.smithi178.stdout:2/974: sync 2024-09-10T11:29:38.510 INFO:tasks.workunit.client.0.smithi144.stdout:6/466: write d9/dd/d3e/d47/f88 [1188220,27115] 0 2024-09-10T11:29:38.510 INFO:tasks.workunit.client.0.smithi144.stdout:2/376: mkdir d7/da/d1e/d21/d7e 0 2024-09-10T11:29:38.516 INFO:tasks.workunit.client.1.smithi178.stdout:5/963: link d9/dc/d17/db0/l10f d9/dc/l13a 0 2024-09-10T11:29:38.517 INFO:tasks.workunit.client.1.smithi178.stdout:0/754: dwrite d0/dc/d39/d50/f71 [0,4194304] 0 2024-09-10T11:29:38.518 INFO:tasks.workunit.client.1.smithi178.stdout:6/831: link d3/d24/l64 d3/d74/dc9/d122/l12e 0 2024-09-10T11:29:38.521 INFO:tasks.workunit.client.1.smithi178.stdout:8/909: write d5/db/fe [37724,10876] 0 2024-09-10T11:29:38.523 INFO:tasks.workunit.client.0.smithi144.stdout:2/377: fdatasync d7/d42/f48 0 2024-09-10T11:29:38.525 INFO:tasks.workunit.client.0.smithi144.stdout:2/378: write d7/de/f2f [870390,27849] 0 2024-09-10T11:29:38.525 INFO:tasks.workunit.client.1.smithi178.stdout:1/726: creat d3/d9/d98/dd1/f110 x:0 0 0 2024-09-10T11:29:38.526 INFO:tasks.workunit.client.0.smithi144.stdout:7/420: rename d5/fb to d5/dc/d14/d40/d5e/d68/f89 0 2024-09-10T11:29:38.528 INFO:tasks.workunit.client.1.smithi178.stdout:4/677: link d0/d10/fe5 d0/d9d/dd5/fe8 0 2024-09-10T11:29:38.530 INFO:tasks.workunit.client.1.smithi178.stdout:0/755: stat d0/dc/d1f/d2d/dd1/led 0 2024-09-10T11:29:38.530 INFO:tasks.workunit.client.0.smithi144.stdout:4/481: getdents d14/d67/d92 0 2024-09-10T11:29:38.532 INFO:tasks.workunit.client.1.smithi178.stdout:8/910: creat d5/d30/d33/d75/f133 x:0 0 0 2024-09-10T11:29:38.532 INFO:tasks.workunit.client.0.smithi144.stdout:2/379: mkdir d7/de/d3d/d43/d7f 0 2024-09-10T11:29:38.535 INFO:tasks.workunit.client.1.smithi178.stdout:9/860: dwrite d2/d19/f8e [0,4194304] 0 2024-09-10T11:29:38.535 INFO:tasks.workunit.client.0.smithi144.stdout:2/380: write d7/de/d3d/f50 [3519632,48369] 0 2024-09-10T11:29:38.535 INFO:tasks.workunit.client.0.smithi144.stdout:7/421: symlink d5/dc/d14/d40/d5e/d68/l8a 0 2024-09-10T11:29:38.535 INFO:tasks.workunit.client.1.smithi178.stdout:3/869: getdents db/d35/d3a 0 2024-09-10T11:29:38.537 INFO:tasks.workunit.client.0.smithi144.stdout:4/482: chown d14/d67/d84/c8b 932412 1 2024-09-10T11:29:38.538 INFO:tasks.workunit.client.0.smithi144.stdout:6/467: dwrite d9/dd/d3e/d47/f28 [4194304,4194304] 0 2024-09-10T11:29:38.538 INFO:tasks.workunit.client.0.smithi144.stdout:2/381: write d7/de/f28 [353030,6652] 0 2024-09-10T11:29:38.540 INFO:tasks.workunit.client.1.smithi178.stdout:4/678: rmdir d0/d8/da/dae 39 2024-09-10T11:29:38.541 INFO:tasks.workunit.client.0.smithi144.stdout:5/528: getdents d4/d20/d22 0 2024-09-10T11:29:38.545 INFO:tasks.workunit.client.1.smithi178.stdout:7/720: dwrite d6/d10/d1e/d2d/f16 [4194304,4194304] 0 2024-09-10T11:29:38.546 INFO:tasks.workunit.client.0.smithi144.stdout:2/382: unlink d7/de/d36/d45/d47/c55 0 2024-09-10T11:29:38.548 INFO:tasks.workunit.client.0.smithi144.stdout:5/529: dwrite d4/d5/d32/f33 [4194304,4194304] 0 2024-09-10T11:29:38.550 INFO:tasks.workunit.client.0.smithi144.stdout:2/383: write d7/de/d3d/d43/f7b [493476,121902] 0 2024-09-10T11:29:38.550 INFO:tasks.workunit.client.0.smithi144.stdout:0/311: sync 2024-09-10T11:29:38.550 INFO:tasks.workunit.client.0.smithi144.stdout:1/583: sync 2024-09-10T11:29:38.552 INFO:tasks.workunit.client.0.smithi144.stdout:4/483: mkdir d14/d1e/d83/da4/da9 0 2024-09-10T11:29:38.553 INFO:tasks.workunit.client.0.smithi144.stdout:9/476: dread d11/d12/d1b/f4d [0,4194304] 0 2024-09-10T11:29:38.553 INFO:tasks.workunit.client.1.smithi178.stdout:5/964: symlink d9/dc/dd/d37/d42/d62/daf/df0/l13b 0 2024-09-10T11:29:38.555 INFO:tasks.workunit.client.0.smithi144.stdout:5/530: rmdir d4/d20/d22/d82 39 2024-09-10T11:29:38.556 INFO:tasks.workunit.client.0.smithi144.stdout:2/384: mkdir d7/da/d5b/d80 0 2024-09-10T11:29:38.558 INFO:tasks.workunit.client.1.smithi178.stdout:1/727: dwrite d3/d9/d1a/fdb [0,4194304] 0 2024-09-10T11:29:38.559 INFO:tasks.workunit.client.0.smithi144.stdout:0/312: symlink da/d3a/d4b/l5b 0 2024-09-10T11:29:38.560 INFO:tasks.workunit.client.0.smithi144.stdout:4/484: dread d14/d1e/d22/d34/d3e/d40/d4b/f6a [0,4194304] 0 2024-09-10T11:29:38.561 INFO:tasks.workunit.client.0.smithi144.stdout:7/422: getdents d5/dc/d14/d40/d58/d6a 0 2024-09-10T11:29:38.562 INFO:tasks.workunit.client.1.smithi178.stdout:7/721: truncate d6/d10/d1d/dca/fd4 457481 0 2024-09-10T11:29:38.563 INFO:tasks.workunit.client.1.smithi178.stdout:9/861: mknod d2/d13/d40/d56/dbe/c10c 0 2024-09-10T11:29:38.565 INFO:tasks.workunit.client.0.smithi144.stdout:3/515: truncate d4/d28/d67/d80/d17/f81 384313 0 2024-09-10T11:29:38.568 INFO:tasks.workunit.client.0.smithi144.stdout:9/477: unlink d11/d36/f58 0 2024-09-10T11:29:38.570 INFO:tasks.workunit.client.1.smithi178.stdout:8/911: dread d5/db/d10/d20/f26 [0,4194304] 0 2024-09-10T11:29:38.570 INFO:tasks.workunit.client.1.smithi178.stdout:0/756: dwrite d0/dc/d1f/dae/fd9 [0,4194304] 0 2024-09-10T11:29:38.570 INFO:tasks.workunit.client.1.smithi178.stdout:3/870: rename db/d35/d58/ce7 to db/d20/d3d/d63/da2/d98/c11c 0 2024-09-10T11:29:38.575 INFO:tasks.workunit.client.1.smithi178.stdout:2/975: dread d1/d7/d24/d46/fa4 [4194304,4194304] 0 2024-09-10T11:29:38.575 INFO:tasks.workunit.client.1.smithi178.stdout:4/679: dwrite d0/d9/d3d/f61 [0,4194304] 0 2024-09-10T11:29:38.576 INFO:tasks.workunit.client.0.smithi144.stdout:5/531: rename d4/d52/l6e to d4/d5/d18/d38/d6d/d8f/la0 0 2024-09-10T11:29:38.576 INFO:tasks.workunit.client.1.smithi178.stdout:1/728: readlink d3/d9/d98/dd1/ddc/lf8 0 2024-09-10T11:29:38.576 INFO:tasks.workunit.client.0.smithi144.stdout:2/385: creat d7/da/d5b/f81 x:0 0 0 2024-09-10T11:29:38.579 INFO:tasks.workunit.client.1.smithi178.stdout:8/912: symlink d5/db/d3d/de7/l134 0 2024-09-10T11:29:38.579 INFO:tasks.workunit.client.1.smithi178.stdout:3/871: fsync db/f4f 0 2024-09-10T11:29:38.580 INFO:tasks.workunit.client.0.smithi144.stdout:6/468: getdents d9/dd/d11/d1b 0 2024-09-10T11:29:38.581 INFO:tasks.workunit.client.0.smithi144.stdout:5/532: write d4/d20/d24/f46 [226755,61488] 0 2024-09-10T11:29:38.581 INFO:tasks.workunit.client.1.smithi178.stdout:8/913: dread - d5/db7/fc8 zero size 2024-09-10T11:29:38.582 INFO:tasks.workunit.client.1.smithi178.stdout:7/722: write d6/d10/d1d/d22/d50/d7e/da7/feb [4830775,1190] 0 2024-09-10T11:29:38.583 INFO:tasks.workunit.client.1.smithi178.stdout:5/965: rename d9/dc/dd/d37/c63 to d9/d19/c13c 0 2024-09-10T11:29:38.583 INFO:tasks.workunit.client.0.smithi144.stdout:7/423: rmdir d5/dc/d11 39 2024-09-10T11:29:38.585 INFO:tasks.workunit.client.1.smithi178.stdout:4/680: mknod d0/d10/ce9 0 2024-09-10T11:29:38.586 INFO:tasks.workunit.client.1.smithi178.stdout:9/862: symlink d2/d13/d40/d56/dbe/deb/l10d 0 2024-09-10T11:29:38.589 INFO:tasks.workunit.client.1.smithi178.stdout:8/914: creat d5/db/d10/d20/d52/d67/dbb/f135 x:0 0 0 2024-09-10T11:29:38.589 INFO:tasks.workunit.client.0.smithi144.stdout:5/533: dwrite d4/d5/f28 [0,4194304] 0 2024-09-10T11:29:38.592 INFO:tasks.workunit.client.0.smithi144.stdout:2/386: unlink d7/da/d1e/d21/d54/f57 0 2024-09-10T11:29:38.592 INFO:tasks.workunit.client.0.smithi144.stdout:0/313: dread da/f19 [0,4194304] 0 2024-09-10T11:29:38.595 INFO:tasks.workunit.client.0.smithi144.stdout:2/387: dread - d7/de/d3d/d5e/f63 zero size 2024-09-10T11:29:38.596 INFO:tasks.workunit.client.0.smithi144.stdout:2/388: stat d7/d29/f4b 0 2024-09-10T11:29:38.597 INFO:tasks.workunit.client.1.smithi178.stdout:9/863: mknod d2/d8/c10e 0 2024-09-10T11:29:38.597 INFO:tasks.workunit.client.1.smithi178.stdout:4/681: dwrite d0/d8/df/f20 [0,4194304] 0 2024-09-10T11:29:38.597 INFO:tasks.workunit.client.0.smithi144.stdout:4/485: link d14/d1e/d22/d2b/fa0 d14/d1e/d22/d2b/d35/faa 0 2024-09-10T11:29:38.597 INFO:tasks.workunit.client.1.smithi178.stdout:4/682: dread - d0/d9/fc1 zero size 2024-09-10T11:29:38.599 INFO:tasks.workunit.client.0.smithi144.stdout:1/584: link d5/dd/c17 d5/dd/da1/cc1 0 2024-09-10T11:29:38.601 INFO:tasks.workunit.client.1.smithi178.stdout:5/966: symlink d9/dc/dd/d3f/dd6/d124/d6d/dcc/dd8/l13d 0 2024-09-10T11:29:38.601 INFO:tasks.workunit.client.1.smithi178.stdout:7/723: creat d6/d10/d1d/d22/d50/d98/da6/f103 x:0 0 0 2024-09-10T11:29:38.601 INFO:tasks.workunit.client.1.smithi178.stdout:1/729: symlink d3/l111 0 2024-09-10T11:29:38.604 INFO:tasks.workunit.client.0.smithi144.stdout:9/478: rename d11/l25 to d11/d12/d1b/d2c/d78/lac 0 2024-09-10T11:29:38.607 INFO:tasks.workunit.client.1.smithi178.stdout:8/915: rename d5/db/d10/d70/dc4/f125 to d5/d61/f136 0 2024-09-10T11:29:38.607 INFO:tasks.workunit.client.0.smithi144.stdout:3/516: dread d4/f58 [0,4194304] 0 2024-09-10T11:29:38.611 INFO:tasks.workunit.client.1.smithi178.stdout:9/864: mkdir d2/d13/d40/d56/dbe/deb/d10f 0 2024-09-10T11:29:38.615 INFO:tasks.workunit.client.1.smithi178.stdout:4/683: mkdir d0/d8/d3e/dea 0 2024-09-10T11:29:38.617 INFO:tasks.workunit.client.0.smithi144.stdout:2/389: getdents d7/de/d36/d45 0 2024-09-10T11:29:38.617 INFO:tasks.workunit.client.0.smithi144.stdout:9/479: dwrite d11/d12/d1b/f61 [0,4194304] 0 2024-09-10T11:29:38.617 INFO:tasks.workunit.client.0.smithi144.stdout:1/585: dwrite d5/d2a/f5b [0,4194304] 0 2024-09-10T11:29:38.619 INFO:tasks.workunit.client.0.smithi144.stdout:1/586: chown d5/df/fb6 1158 1 2024-09-10T11:29:38.620 INFO:tasks.workunit.client.0.smithi144.stdout:7/424: link d5/dc/d11/c4e d5/dc/d12/d2f/c8b 0 2024-09-10T11:29:38.621 INFO:tasks.workunit.client.0.smithi144.stdout:9/480: write d11/d12/d1b/f77 [3015811,128965] 0 2024-09-10T11:29:38.622 INFO:tasks.workunit.client.1.smithi178.stdout:1/730: dwrite d3/d9/d98/dac/d3a/d50/d6c/ff2 [0,4194304] 0 2024-09-10T11:29:38.623 INFO:tasks.workunit.client.1.smithi178.stdout:8/916: read d5/f1f [2488113,51952] 0 2024-09-10T11:29:38.624 INFO:tasks.workunit.client.0.smithi144.stdout:5/534: getdents d4/d5/d18/d38 0 2024-09-10T11:29:38.625 INFO:tasks.workunit.client.1.smithi178.stdout:5/967: dwrite d9/dc/dd/d3f/d5a/f12e [0,4194304] 0 2024-09-10T11:29:38.625 INFO:tasks.workunit.client.0.smithi144.stdout:7/425: chown d5/dc/d7b 2 1 2024-09-10T11:29:38.625 INFO:tasks.workunit.client.0.smithi144.stdout:8/292: sync 2024-09-10T11:29:38.626 INFO:tasks.workunit.client.1.smithi178.stdout:6/832: dwrite d3/d74/d91/daa/dc2/f96 [0,4194304] 0 2024-09-10T11:29:38.627 INFO:tasks.workunit.client.0.smithi144.stdout:3/517: creat d4/d28/d67/d80/d17/d18/d34/d65/d75/fb0 x:0 0 0 2024-09-10T11:29:38.628 INFO:tasks.workunit.client.0.smithi144.stdout:2/390: unlink d7/d42/d64/c68 0 2024-09-10T11:29:38.631 INFO:tasks.workunit.client.0.smithi144.stdout:7/426: write d5/dc/d12/d2b/f43 [1088122,53679] 0 2024-09-10T11:29:38.636 INFO:tasks.workunit.client.1.smithi178.stdout:9/865: dwrite d2/df/fd7 [0,4194304] 0 2024-09-10T11:29:38.637 INFO:tasks.workunit.client.0.smithi144.stdout:1/587: symlink d5/d4a/d44/d80/lc2 0 2024-09-10T11:29:38.639 INFO:tasks.workunit.client.1.smithi178.stdout:5/968: stat d9/dc/dd/d37/dbd/dda/db8/fed 0 2024-09-10T11:29:38.639 INFO:tasks.workunit.client.1.smithi178.stdout:7/724: dread d6/d10/d1d/f73 [0,4194304] 0 2024-09-10T11:29:38.641 INFO:tasks.workunit.client.0.smithi144.stdout:0/314: write f0 [3941404,18004] 0 2024-09-10T11:29:38.641 INFO:tasks.workunit.client.0.smithi144.stdout:7/427: truncate d5/dc/d14/d40/d58/d6e/f75 74119 0 2024-09-10T11:29:38.644 INFO:tasks.workunit.client.0.smithi144.stdout:4/486: getdents d14/d1e/d83/da4 0 2024-09-10T11:29:38.645 INFO:tasks.workunit.client.0.smithi144.stdout:9/481: dwrite d11/d36/d2d/d38/f76 [0,4194304] 0 2024-09-10T11:29:38.645 INFO:tasks.workunit.client.1.smithi178.stdout:4/684: symlink d0/d8/d3e/dea/leb 0 2024-09-10T11:29:38.645 INFO:tasks.workunit.client.1.smithi178.stdout:1/731: write d3/d9/d98/fca [523770,84539] 0 2024-09-10T11:29:38.645 INFO:tasks.workunit.client.0.smithi144.stdout:6/469: dwrite d9/dd/d11/f90 [0,4194304] 0 2024-09-10T11:29:38.645 INFO:tasks.workunit.client.0.smithi144.stdout:6/470: chown d9/dd/d3e/d47/d13/d4a/c7a 37910403 1 2024-09-10T11:29:38.648 INFO:tasks.workunit.client.0.smithi144.stdout:9/482: readlink d11/d12/l90 0 2024-09-10T11:29:38.649 INFO:tasks.workunit.client.0.smithi144.stdout:8/293: read d0/d9/ff [4174673,45295] 0 2024-09-10T11:29:38.649 INFO:tasks.workunit.client.0.smithi144.stdout:5/535: rmdir d4/d20/d22/d5c 39 2024-09-10T11:29:38.654 INFO:tasks.workunit.client.1.smithi178.stdout:8/917: symlink d5/db/l137 0 2024-09-10T11:29:38.654 INFO:tasks.workunit.client.1.smithi178.stdout:1/732: write d3/d9/d21/d4b/f17 [929605,11769] 0 2024-09-10T11:29:38.654 INFO:tasks.workunit.client.1.smithi178.stdout:0/757: dwrite d0/dc/dc4/f8a [0,4194304] 0 2024-09-10T11:29:38.657 INFO:tasks.workunit.client.1.smithi178.stdout:7/725: unlink d6/d10/d34/d6a/lba 0 2024-09-10T11:29:38.658 INFO:tasks.workunit.client.0.smithi144.stdout:3/518: rmdir d4 39 2024-09-10T11:29:38.659 INFO:tasks.workunit.client.0.smithi144.stdout:4/487: mkdir d14/d1e/d22/d34/d3e/d40/d4b/d4c/dab 0 2024-09-10T11:29:38.660 INFO:tasks.workunit.client.0.smithi144.stdout:1/588: symlink d5/d4a/d44/d80/lc3 0 2024-09-10T11:29:38.661 INFO:tasks.workunit.client.1.smithi178.stdout:7/726: chown d6/d10/d1e/d2d/d2b/fc1 7825404 1 2024-09-10T11:29:38.664 INFO:tasks.workunit.client.1.smithi178.stdout:2/976: dwrite d1/d7/d24/d46/d88/d9d/dab/d5b/d60/fcc [0,4194304] 0 2024-09-10T11:29:38.664 INFO:tasks.workunit.client.0.smithi144.stdout:2/391: mkdir d7/de/d36/d45/d47/d6c/d7c/d82 0 2024-09-10T11:29:38.667 INFO:tasks.workunit.client.0.smithi144.stdout:7/428: dwrite d5/dc/d14/d40/d5e/d63/f74 [0,4194304] 0 2024-09-10T11:29:38.667 INFO:tasks.workunit.client.0.smithi144.stdout:1/589: chown d5/d4a/d44/f94 279223 1 2024-09-10T11:29:38.667 INFO:tasks.workunit.client.0.smithi144.stdout:4/488: truncate d14/d1e/d22/f48 4973882 0 2024-09-10T11:29:38.668 INFO:tasks.workunit.client.0.smithi144.stdout:8/294: dwrite d0/d9/d21/d25/d4f/f56 [0,4194304] 0 2024-09-10T11:29:38.670 INFO:tasks.workunit.client.0.smithi144.stdout:5/536: mknod d4/d20/d22/d82/ca1 0 2024-09-10T11:29:38.670 INFO:tasks.workunit.client.0.smithi144.stdout:8/295: readlink d0/d1b/l28 0 2024-09-10T11:29:38.672 INFO:tasks.workunit.client.1.smithi178.stdout:1/733: fdatasync d3/d9/d98/dac/d3a/d50/d6c/ff2 0 2024-09-10T11:29:38.674 INFO:tasks.workunit.client.1.smithi178.stdout:1/734: write d3/f1c [7853638,109115] 0 2024-09-10T11:29:38.675 INFO:tasks.workunit.client.1.smithi178.stdout:1/735: dread - d3/d9/d98/dac/d3a/d9f/fea zero size 2024-09-10T11:29:38.677 INFO:tasks.workunit.client.0.smithi144.stdout:9/483: link d11/d12/d1b/d41/d8f/dab/d64/l73 d11/d12/d1b/d2c/d8a/lad 0 2024-09-10T11:29:38.684 INFO:tasks.workunit.client.0.smithi144.stdout:8/296: symlink d0/d1b/d5a/l5d 0 2024-09-10T11:29:38.685 INFO:tasks.workunit.client.0.smithi144.stdout:4/489: creat d14/d1e/d22/d34/d3e/d40/fac x:0 0 0 2024-09-10T11:29:38.685 INFO:tasks.workunit.client.0.smithi144.stdout:7/429: dwrite d5/dc/d2a/f35 [0,4194304] 0 2024-09-10T11:29:38.686 INFO:tasks.workunit.client.0.smithi144.stdout:1/590: link d5/d2a/d65/c42 d5/d2a/d65/cc4 0 2024-09-10T11:29:38.687 INFO:tasks.workunit.client.0.smithi144.stdout:3/519: dwrite d4/d28/d67/d80/d17/d18/d34/d65/f88 [0,4194304] 0 2024-09-10T11:29:38.689 INFO:tasks.workunit.client.0.smithi144.stdout:9/484: rmdir d11/d12/d1b/d41/d8f/d84 0 2024-09-10T11:29:38.692 INFO:tasks.workunit.client.0.smithi144.stdout:5/537: rename d4/d20/d22/d5c/d5d/d72 to d4/d5/d18/d38/d6d/da2 0 2024-09-10T11:29:38.692 INFO:tasks.workunit.client.0.smithi144.stdout:8/297: read d0/d9/d23/d26/f30 [1855274,129832] 0 2024-09-10T11:29:38.693 INFO:tasks.workunit.client.0.smithi144.stdout:8/298: chown d0/d9/d23/d37/d48 446695180 1 2024-09-10T11:29:38.695 INFO:tasks.workunit.client.1.smithi178.stdout:5/969: chown d9/dc/dd/d37/d3e/l85 58639171 1 2024-09-10T11:29:38.696 INFO:tasks.workunit.client.0.smithi144.stdout:9/485: mknod d11/d12/d1b/d41/cae 0 2024-09-10T11:29:38.697 INFO:tasks.workunit.client.0.smithi144.stdout:1/591: getdents d5/d4a/d44/da2 0 2024-09-10T11:29:38.697 INFO:tasks.workunit.client.0.smithi144.stdout:5/538: truncate d4/fa 2226663 0 2024-09-10T11:29:38.700 INFO:tasks.workunit.client.0.smithi144.stdout:9/486: truncate d11/d12/fa1 401556 0 2024-09-10T11:29:38.701 INFO:tasks.workunit.client.0.smithi144.stdout:3/520: dwrite d4/dd/d21/f83 [0,4194304] 0 2024-09-10T11:29:38.701 INFO:tasks.workunit.client.0.smithi144.stdout:8/299: write d0/d9/ff [1550679,104388] 0 2024-09-10T11:29:38.707 INFO:tasks.workunit.client.0.smithi144.stdout:2/392: dread d7/da/f3c [0,4194304] 0 2024-09-10T11:29:38.714 INFO:tasks.workunit.client.0.smithi144.stdout:7/430: link d5/dc/d12/d2f/d7c/f85 d5/dc/d12/d2f/d7c/f8c 0 2024-09-10T11:29:38.715 INFO:tasks.workunit.client.0.smithi144.stdout:1/592: mknod d5/dd/d9c/cc5 0 2024-09-10T11:29:38.716 INFO:tasks.workunit.client.0.smithi144.stdout:5/539: readlink d4/d5/l61 0 2024-09-10T11:29:38.718 INFO:tasks.workunit.client.0.smithi144.stdout:8/300: mkdir d0/d1b/d47/d5e 0 2024-09-10T11:29:38.719 INFO:tasks.workunit.client.0.smithi144.stdout:3/521: dread - d4/dd/d21/f8b zero size 2024-09-10T11:29:38.720 INFO:tasks.workunit.client.0.smithi144.stdout:8/301: read d0/d9/f1d [4117495,41503] 0 2024-09-10T11:29:38.721 INFO:tasks.workunit.client.0.smithi144.stdout:8/302: fdatasync d0/d9/d23/d26/f36 0 2024-09-10T11:29:38.722 INFO:tasks.workunit.client.0.smithi144.stdout:1/593: unlink d5/d2a/f89 0 2024-09-10T11:29:38.722 INFO:tasks.workunit.client.0.smithi144.stdout:8/303: rename d0/d1b/d47 to d0/d1b/d47/d5e/d5f 22 2024-09-10T11:29:38.722 INFO:tasks.workunit.client.0.smithi144.stdout:4/490: link d14/l19 d14/d1e/d22/lad 0 2024-09-10T11:29:38.723 INFO:tasks.workunit.client.0.smithi144.stdout:6/471: write d9/dd/d3e/f40 [1292107,7328] 0 2024-09-10T11:29:38.725 INFO:tasks.workunit.client.0.smithi144.stdout:2/393: mknod d7/da/d1e/d21/d54/d61/c83 0 2024-09-10T11:29:38.727 INFO:tasks.workunit.client.0.smithi144.stdout:2/394: read d7/de/f2f [2035650,13551] 0 2024-09-10T11:29:38.728 INFO:tasks.workunit.client.0.smithi144.stdout:7/431: unlink d5/dc/d14/c27 0 2024-09-10T11:29:38.733 INFO:tasks.workunit.client.0.smithi144.stdout:8/304: dwrite d0/d9/d23/d37/f3a [0,4194304] 0 2024-09-10T11:29:38.734 INFO:tasks.workunit.client.0.smithi144.stdout:1/594: creat d5/d4a/dc0/fc6 x:0 0 0 2024-09-10T11:29:38.738 INFO:tasks.workunit.client.1.smithi178.stdout:4/685: unlink d0/d9/dc/f34 0 2024-09-10T11:29:38.739 INFO:tasks.workunit.client.1.smithi178.stdout:6/833: creat d3/d6e/d87/f12f x:0 0 0 2024-09-10T11:29:38.741 INFO:tasks.workunit.client.0.smithi144.stdout:8/305: dwrite d0/d1b/f2a [0,4194304] 0 2024-09-10T11:29:38.742 INFO:tasks.workunit.client.1.smithi178.stdout:9/866: rename d2/d9b/fe4 to d2/df/d31/d4a/f110 0 2024-09-10T11:29:38.744 INFO:tasks.workunit.client.1.smithi178.stdout:9/867: dread - d2/d13/d40/d56/fd8 zero size 2024-09-10T11:29:38.744 INFO:tasks.workunit.client.1.smithi178.stdout:8/918: creat d5/d30/d33/d8c/f138 x:0 0 0 2024-09-10T11:29:38.745 INFO:tasks.workunit.client.0.smithi144.stdout:2/395: creat d7/de/d36/d45/f84 x:0 0 0 2024-09-10T11:29:38.745 INFO:tasks.workunit.client.1.smithi178.stdout:9/868: chown d2/d13/d40/d56/de1 0 1 2024-09-10T11:29:38.745 INFO:tasks.workunit.client.0.smithi144.stdout:2/396: readlink l0 0 2024-09-10T11:29:38.746 INFO:tasks.workunit.client.0.smithi144.stdout:4/491: link d14/d6f/f96 d14/d1e/d22/d2b/d35/fae 0 2024-09-10T11:29:38.747 INFO:tasks.workunit.client.0.smithi144.stdout:2/397: fsync d7/de/d3d/d4e/f6d 0 2024-09-10T11:29:38.747 INFO:tasks.workunit.client.0.smithi144.stdout:3/522: truncate d4/d28/d67/d80/d17/d18/d34/d65/d75/f84 297123 0 2024-09-10T11:29:38.747 INFO:tasks.workunit.client.1.smithi178.stdout:8/919: read - d5/db/d10/d20/f10a zero size 2024-09-10T11:29:38.749 INFO:tasks.workunit.client.0.smithi144.stdout:7/432: dread d5/dc/d12/f13 [0,4194304] 0 2024-09-10T11:29:38.751 INFO:tasks.workunit.client.1.smithi178.stdout:5/970: creat d9/dc/dd/d37/d42/d62/f13e x:0 0 0 2024-09-10T11:29:38.752 INFO:tasks.workunit.client.0.smithi144.stdout:1/595: creat d5/d4a/d63/d84/d95/fc7 x:0 0 0 2024-09-10T11:29:38.753 INFO:tasks.workunit.client.0.smithi144.stdout:1/596: readlink d5/df/d4b/l8c 0 2024-09-10T11:29:38.753 INFO:tasks.workunit.client.1.smithi178.stdout:4/686: creat d0/d9/dc/d9a/fec x:0 0 0 2024-09-10T11:29:38.754 INFO:tasks.workunit.client.1.smithi178.stdout:4/687: chown d0/d8/da/d1a 1755 1 2024-09-10T11:29:38.755 INFO:tasks.workunit.client.1.smithi178.stdout:9/869: creat d2/d13/d40/d56/f111 x:0 0 0 2024-09-10T11:29:38.756 INFO:tasks.workunit.client.1.smithi178.stdout:8/920: creat d5/d1d/d22/f139 x:0 0 0 2024-09-10T11:29:38.756 INFO:tasks.workunit.client.0.smithi144.stdout:5/540: write d4/d5/f7d [898839,38131] 0 2024-09-10T11:29:38.757 INFO:tasks.workunit.client.1.smithi178.stdout:6/834: mknod d3/d74/dc9/d122/c130 0 2024-09-10T11:29:38.760 INFO:tasks.workunit.client.0.smithi144.stdout:4/492: symlink d14/d6f/laf 0 2024-09-10T11:29:38.761 INFO:tasks.workunit.client.0.smithi144.stdout:2/398: creat d7/da/d1e/f85 x:0 0 0 2024-09-10T11:29:38.761 INFO:tasks.workunit.client.1.smithi178.stdout:5/971: mkdir d9/dc/dd/d37/d13f 0 2024-09-10T11:29:38.762 INFO:tasks.workunit.client.0.smithi144.stdout:2/399: chown d7/da/d1e/d21/d7e 1689713 1 2024-09-10T11:29:38.763 INFO:tasks.workunit.client.1.smithi178.stdout:8/921: creat d5/db/d10/d35/f13a x:0 0 0 2024-09-10T11:29:38.763 INFO:tasks.workunit.client.0.smithi144.stdout:2/400: write d7/f41 [2006962,85978] 0 2024-09-10T11:29:38.764 INFO:tasks.workunit.client.0.smithi144.stdout:1/597: creat d5/df/d4b/fc8 x:0 0 0 2024-09-10T11:29:38.765 INFO:tasks.workunit.client.0.smithi144.stdout:6/472: getdents d9/dd/d3e/d47/d13 0 2024-09-10T11:29:38.766 INFO:tasks.workunit.client.1.smithi178.stdout:2/977: dread d1/d7/d24/d7d/d84/f36 [0,4194304] 0 2024-09-10T11:29:38.767 INFO:tasks.workunit.client.0.smithi144.stdout:5/541: mknod d4/d5/d18/ca3 0 2024-09-10T11:29:38.767 INFO:tasks.workunit.client.1.smithi178.stdout:9/870: creat d2/df/d31/d37/d3d/f112 x:0 0 0 2024-09-10T11:29:38.767 INFO:tasks.workunit.client.0.smithi144.stdout:6/473: readlink d9/d1f/l8b 0 2024-09-10T11:29:38.768 INFO:tasks.workunit.client.0.smithi144.stdout:1/598: write d5/dd/f5e [446185,276] 0 2024-09-10T11:29:38.769 INFO:tasks.workunit.client.1.smithi178.stdout:9/871: write d2/d13/d40/d89/d93/d88/dae/fdb [975668,462] 0 2024-09-10T11:29:38.769 INFO:tasks.workunit.client.1.smithi178.stdout:8/922: creat d5/d30/db4/f13b x:0 0 0 2024-09-10T11:29:38.770 INFO:tasks.workunit.client.1.smithi178.stdout:4/688: creat d0/d8/da/fed x:0 0 0 2024-09-10T11:29:38.770 INFO:tasks.workunit.client.0.smithi144.stdout:5/542: chown d4/d5/d18/d38/c4d 1271376 1 2024-09-10T11:29:38.771 INFO:tasks.workunit.client.0.smithi144.stdout:6/474: chown d9/dd/d11/d1b/d2f/d7f/l9a 3810684 1 2024-09-10T11:29:38.772 INFO:tasks.workunit.client.0.smithi144.stdout:1/599: dread d5/dd/f48 [0,4194304] 0 2024-09-10T11:29:38.772 INFO:tasks.workunit.client.1.smithi178.stdout:4/689: write d0/d9/d3d/d4f/f52 [1055613,68520] 0 2024-09-10T11:29:38.772 INFO:tasks.workunit.client.0.smithi144.stdout:7/433: dwrite f1 [0,4194304] 0 2024-09-10T11:29:38.773 INFO:tasks.workunit.client.0.smithi144.stdout:7/434: dread - d5/dc/d12/d2f/d7c/f85 zero size 2024-09-10T11:29:38.775 INFO:tasks.workunit.client.0.smithi144.stdout:3/523: mknod d4/d28/d67/d80/d17/d18/d34/d4b/cb1 0 2024-09-10T11:29:38.775 INFO:tasks.workunit.client.0.smithi144.stdout:1/600: stat d5/d4a/f7d 0 2024-09-10T11:29:38.777 INFO:tasks.workunit.client.1.smithi178.stdout:9/872: write d2/df/f28 [3357111,17770] 0 2024-09-10T11:29:38.778 INFO:tasks.workunit.client.0.smithi144.stdout:7/435: truncate d5/dc/d14/d40/d58/d6e/f75 733246 0 2024-09-10T11:29:38.778 INFO:tasks.workunit.client.0.smithi144.stdout:6/475: read d9/dd/d3e/f76 [2440099,89653] 0 2024-09-10T11:29:38.779 INFO:tasks.workunit.client.0.smithi144.stdout:7/436: readlink d5/dc/d14/d40/l42 0 2024-09-10T11:29:38.780 INFO:tasks.workunit.client.1.smithi178.stdout:8/923: truncate d5/db/d10/d35/ff1 73138 0 2024-09-10T11:29:38.781 INFO:tasks.workunit.client.1.smithi178.stdout:8/924: dread - d5/db/d1e/f108 zero size 2024-09-10T11:29:38.781 INFO:tasks.workunit.client.1.smithi178.stdout:6/835: link d3/d7/d8/d15/d1c/d2a/l5c d3/d6e/d87/l131 0 2024-09-10T11:29:38.785 INFO:tasks.workunit.client.0.smithi144.stdout:6/476: dwrite d9/f71 [0,4194304] 0 2024-09-10T11:29:38.786 INFO:tasks.workunit.client.1.smithi178.stdout:8/925: mknod d5/d30/db4/c13c 0 2024-09-10T11:29:38.787 INFO:tasks.workunit.client.0.smithi144.stdout:6/477: chown d9/dd/d11/d1b/d2f/d7f 0 1 2024-09-10T11:29:38.788 INFO:tasks.workunit.client.1.smithi178.stdout:8/926: write d5/db/d10/d53/d82/db2/f118 [395911,10558] 0 2024-09-10T11:29:38.789 INFO:tasks.workunit.client.1.smithi178.stdout:8/927: stat d5/db/d10/d20/l29 0 2024-09-10T11:29:38.790 INFO:tasks.workunit.client.0.smithi144.stdout:8/306: getdents d0 0 2024-09-10T11:29:38.790 INFO:tasks.workunit.client.1.smithi178.stdout:9/873: mkdir d2/d113 0 2024-09-10T11:29:38.791 INFO:tasks.workunit.client.1.smithi178.stdout:9/874: chown d2/c8b 481 1 2024-09-10T11:29:38.792 INFO:tasks.workunit.client.0.smithi144.stdout:2/401: mknod d7/da/c86 0 2024-09-10T11:29:38.793 INFO:tasks.workunit.client.0.smithi144.stdout:2/402: truncate d7/de/d36/f51 9405540 0 2024-09-10T11:29:38.794 INFO:tasks.workunit.client.1.smithi178.stdout:4/690: getdents d0/d8/da/d1a 0 2024-09-10T11:29:38.797 INFO:tasks.workunit.client.0.smithi144.stdout:4/493: creat d14/d1e/d22/d2b/fb0 x:0 0 0 2024-09-10T11:29:38.799 INFO:tasks.workunit.client.1.smithi178.stdout:4/691: mkdir d0/d8/da/d1a/d51/d5d/dc9/dee 0 2024-09-10T11:29:38.800 INFO:tasks.workunit.client.1.smithi178.stdout:9/875: dwrite d2/dc/ff9 [0,4194304] 0 2024-09-10T11:29:38.801 INFO:tasks.workunit.client.1.smithi178.stdout:4/692: dread - d0/d8/da/fd3 zero size 2024-09-10T11:29:38.804 INFO:tasks.workunit.client.1.smithi178.stdout:9/876: truncate d2/d13/d75/d60/d66/fb9 64631 0 2024-09-10T11:29:38.805 INFO:tasks.workunit.client.0.smithi144.stdout:2/403: rmdir d7/da 39 2024-09-10T11:29:38.805 INFO:tasks.workunit.client.0.smithi144.stdout:2/404: chown d7/de/d3d/d4e/f6a 21 1 2024-09-10T11:29:38.806 INFO:tasks.workunit.client.0.smithi144.stdout:7/437: mknod d5/dc/d2a/c8d 0 2024-09-10T11:29:38.807 INFO:tasks.workunit.client.0.smithi144.stdout:2/405: dread - d7/de/d3d/d4e/f6d zero size 2024-09-10T11:29:38.807 INFO:tasks.workunit.client.0.smithi144.stdout:2/406: readlink l0 0 2024-09-10T11:29:38.807 INFO:tasks.workunit.client.1.smithi178.stdout:7/727: write d6/d10/d1d/d22/d3b/f6d [211964,107227] 0 2024-09-10T11:29:38.808 INFO:tasks.workunit.client.0.smithi144.stdout:5/543: rmdir d4/d20/d87 0 2024-09-10T11:29:38.809 INFO:tasks.workunit.client.0.smithi144.stdout:5/544: write d4/d20/d24/f3e [69996,112352] 0 2024-09-10T11:29:38.810 INFO:tasks.workunit.client.0.smithi144.stdout:6/478: mkdir d9/d1f/dae 0 2024-09-10T11:29:38.811 INFO:tasks.workunit.client.1.smithi178.stdout:7/728: write d6/d10/d1e/d2d/f16 [7402979,4216] 0 2024-09-10T11:29:38.813 INFO:tasks.workunit.client.0.smithi144.stdout:1/601: rename d5/d4a/d44/d80 to d5/d2a/d65/dc9 0 2024-09-10T11:29:38.814 INFO:tasks.workunit.client.1.smithi178.stdout:4/693: dwrite d0/d8/da/fd3 [0,4194304] 0 2024-09-10T11:29:38.814 INFO:tasks.workunit.client.1.smithi178.stdout:8/928: getdents d5/d1d/d22 0 2024-09-10T11:29:38.815 INFO:tasks.workunit.client.0.smithi144.stdout:3/524: truncate d4/d28/f76 720189 0 2024-09-10T11:29:38.815 INFO:tasks.workunit.client.0.smithi144.stdout:1/602: write d5/d2a/f5b [1610213,103200] 0 2024-09-10T11:29:38.816 INFO:tasks.workunit.client.0.smithi144.stdout:3/525: chown d4/dd/d21/d2f 10231 1 2024-09-10T11:29:38.816 INFO:tasks.workunit.client.0.smithi144.stdout:5/545: dread d4/d5/d18/d38/d6d/d8b/f91 [0,4194304] 0 2024-09-10T11:29:38.817 INFO:tasks.workunit.client.0.smithi144.stdout:3/526: write d4/f58 [2011364,77483] 0 2024-09-10T11:29:38.818 INFO:tasks.workunit.client.0.smithi144.stdout:7/438: unlink d5/dc/d14/d40/d58/d6e/f75 0 2024-09-10T11:29:38.818 INFO:tasks.workunit.client.1.smithi178.stdout:0/758: dwrite d0/d2c/d33/f45 [0,4194304] 0 2024-09-10T11:29:38.819 INFO:tasks.workunit.client.0.smithi144.stdout:5/546: truncate d4/d5/f25 1065376 0 2024-09-10T11:29:38.822 INFO:tasks.workunit.client.1.smithi178.stdout:0/759: chown d0/dc/d39/d50/d5e/c6b 0 1 2024-09-10T11:29:38.823 INFO:tasks.workunit.client.1.smithi178.stdout:5/972: truncate d9/d19/f67 1610772 0 2024-09-10T11:29:38.826 INFO:tasks.workunit.client.0.smithi144.stdout:2/407: truncate d7/da/d1e/f85 577879 0 2024-09-10T11:29:38.828 INFO:tasks.workunit.client.1.smithi178.stdout:2/978: write d1/d7/f6c [1989850,10680] 0 2024-09-10T11:29:38.829 INFO:tasks.workunit.client.0.smithi144.stdout:8/307: getdents d0/d1b/d5a 0 2024-09-10T11:29:38.829 INFO:tasks.workunit.client.0.smithi144.stdout:5/547: write d4/d5/d18/d67/f65 [2777345,69623] 0 2024-09-10T11:29:38.830 INFO:tasks.workunit.client.1.smithi178.stdout:8/929: dwrite d5/db/d1e/f45 [0,4194304] 0 2024-09-10T11:29:38.831 INFO:tasks.workunit.client.0.smithi144.stdout:8/308: chown d0/d1b/d5a/l5d 128861691 1 2024-09-10T11:29:38.832 INFO:tasks.workunit.client.0.smithi144.stdout:1/603: mknod d5/d2a/cca 0 2024-09-10T11:29:38.835 INFO:tasks.workunit.client.0.smithi144.stdout:7/439: dwrite d5/dc/d12/d2f/f4d [4194304,4194304] 0 2024-09-10T11:29:38.835 INFO:tasks.workunit.client.1.smithi178.stdout:0/760: symlink d0/dc/d1f/dae/lee 0 2024-09-10T11:29:38.836 INFO:tasks.workunit.client.1.smithi178.stdout:7/729: dwrite d6/d10/d1d/d22/d50/d98/da6/f103 [0,4194304] 0 2024-09-10T11:29:38.836 INFO:tasks.workunit.client.0.smithi144.stdout:4/494: rename d14/d1e/f2f to d14/d67/fb1 0 2024-09-10T11:29:38.836 INFO:tasks.workunit.client.0.smithi144.stdout:1/604: readlink d5/d2a/d65/dc9/lc2 0 2024-09-10T11:29:38.838 INFO:tasks.workunit.client.0.smithi144.stdout:7/440: write d5/f7 [1451815,73120] 0 2024-09-10T11:29:38.843 INFO:tasks.workunit.client.0.smithi144.stdout:2/408: creat d7/de/d3d/d4e/f87 x:0 0 0 2024-09-10T11:29:38.844 INFO:tasks.workunit.client.1.smithi178.stdout:0/761: getdents d0/dc/d1f/d2d/dac/de7 0 2024-09-10T11:29:38.849 INFO:tasks.workunit.client.1.smithi178.stdout:5/973: truncate d9/dc/dd/d3f/d5a/fd3 4931114 0 2024-09-10T11:29:38.849 INFO:tasks.workunit.client.1.smithi178.stdout:6/836: dwrite d3/d7/d8/d72/fd5 [0,4194304] 0 2024-09-10T11:29:38.852 INFO:tasks.workunit.client.1.smithi178.stdout:4/694: dwrite d0/d9/dc/d8e/f9f [0,4194304] 0 2024-09-10T11:29:38.852 INFO:tasks.workunit.client.0.smithi144.stdout:8/309: creat d0/d9/d21/d25/f60 x:0 0 0 2024-09-10T11:29:38.853 INFO:tasks.workunit.client.0.smithi144.stdout:5/548: dwrite d4/d5/f25 [0,4194304] 0 2024-09-10T11:29:38.853 INFO:tasks.workunit.client.0.smithi144.stdout:6/479: rename d9/dd/d3e/d47/d13/d4a to d9/dd/d3e/d83/daf 0 2024-09-10T11:29:38.856 INFO:tasks.workunit.client.1.smithi178.stdout:2/979: dwrite d1/d7/d24/d46/d88/d9d/dab/f17 [0,4194304] 0 2024-09-10T11:29:38.856 INFO:tasks.workunit.client.1.smithi178.stdout:7/730: getdents d6/d10/d1e/d2d/d2b 0 2024-09-10T11:29:38.857 INFO:tasks.workunit.client.1.smithi178.stdout:0/762: creat d0/d12/d3f/d72/fef x:0 0 0 2024-09-10T11:29:38.859 INFO:tasks.workunit.client.1.smithi178.stdout:4/695: mknod d0/d9d/da5/cef 0 2024-09-10T11:29:38.859 INFO:tasks.workunit.client.0.smithi144.stdout:4/495: mknod d14/d1e/d22/d2b/d35/d3c/da7/cb2 0 2024-09-10T11:29:38.860 INFO:tasks.workunit.client.1.smithi178.stdout:0/763: chown d0/dc/dc4/de8 253453040 1 2024-09-10T11:29:38.861 INFO:tasks.workunit.client.1.smithi178.stdout:0/764: write d0/d12/f34 [2007218,67835] 0 2024-09-10T11:29:38.863 INFO:tasks.workunit.client.0.smithi144.stdout:6/480: fdatasync d9/dd/d3e/d47/d13/f31 0 2024-09-10T11:29:38.864 INFO:tasks.workunit.client.1.smithi178.stdout:5/974: creat d9/dc/dd/f140 x:0 0 0 2024-09-10T11:29:38.866 INFO:tasks.workunit.client.0.smithi144.stdout:6/481: chown d9/d38 23178067 1 2024-09-10T11:29:38.867 INFO:tasks.workunit.client.1.smithi178.stdout:6/837: mkdir d3/d74/d91/daa/dc2/d57/d78/d7c/d132 0 2024-09-10T11:29:38.868 INFO:tasks.workunit.client.0.smithi144.stdout:7/441: creat d5/dc/d12/f8e x:0 0 0 2024-09-10T11:29:38.868 INFO:tasks.workunit.client.0.smithi144.stdout:4/496: chown d14/d1e/d22/d34/d3e/d40/d4b/f52 4851668 1 2024-09-10T11:29:38.869 INFO:tasks.workunit.client.1.smithi178.stdout:5/975: dread d9/d19/f92 [0,4194304] 0 2024-09-10T11:29:38.872 INFO:tasks.workunit.client.1.smithi178.stdout:6/838: write d3/d7/d8/d19/f26 [1825605,27013] 0 2024-09-10T11:29:38.874 INFO:tasks.workunit.client.0.smithi144.stdout:3/527: dread d4/dd/d21/f23 [0,4194304] 0 2024-09-10T11:29:38.874 INFO:tasks.workunit.client.1.smithi178.stdout:8/930: write d5/db/d10/d53/d82/fd8 [587846,54290] 0 2024-09-10T11:29:38.875 INFO:tasks.workunit.client.1.smithi178.stdout:5/976: creat d9/dc/dd/d3f/d5a/dbc/f141 x:0 0 0 2024-09-10T11:29:38.875 INFO:tasks.workunit.client.0.smithi144.stdout:2/409: dwrite d7/de/d3d/d5e/f63 [0,4194304] 0 2024-09-10T11:29:38.875 INFO:tasks.workunit.client.0.smithi144.stdout:4/497: mkdir d14/d67/d84/db3 0 2024-09-10T11:29:38.875 INFO:tasks.workunit.client.1.smithi178.stdout:4/696: dread d0/d8/df/f29 [4194304,4194304] 0 2024-09-10T11:29:38.876 INFO:tasks.workunit.client.1.smithi178.stdout:5/977: fdatasync d9/dc/dd/d3f/dd6/d124/f107 0 2024-09-10T11:29:38.877 INFO:tasks.workunit.client.0.smithi144.stdout:7/442: chown d5/dc/d11/c76 607009 1 2024-09-10T11:29:38.878 INFO:tasks.workunit.client.0.smithi144.stdout:6/482: symlink d9/dd/d11/d1b/lb0 0 2024-09-10T11:29:38.879 INFO:tasks.workunit.client.0.smithi144.stdout:5/549: dwrite d4/d5/d18/d67/d59/f6c [0,4194304] 0 2024-09-10T11:29:38.879 INFO:tasks.workunit.client.1.smithi178.stdout:8/931: readlink d5/db/dda/l104 0 2024-09-10T11:29:38.882 INFO:tasks.workunit.client.1.smithi178.stdout:0/765: dread d0/dc/d39/da4/dce/fe1 [0,4194304] 0 2024-09-10T11:29:38.882 INFO:tasks.workunit.client.0.smithi144.stdout:8/310: dread d0/d9/d23/d37/f3a [0,4194304] 0 2024-09-10T11:29:38.884 INFO:tasks.workunit.client.0.smithi144.stdout:3/528: rmdir d4/d28/d67/d80/d17/d18/d34/d78 39 2024-09-10T11:29:38.884 INFO:tasks.workunit.client.0.smithi144.stdout:4/498: write d14/d1e/f7c [1056876,108196] 0 2024-09-10T11:29:38.885 INFO:tasks.workunit.client.1.smithi178.stdout:7/731: dread d6/d10/d1d/d22/d50/d7e/d82/db2/f36 [0,4194304] 0 2024-09-10T11:29:38.885 INFO:tasks.workunit.client.1.smithi178.stdout:8/932: creat d5/d1d/d22/d10f/f13d x:0 0 0 2024-09-10T11:29:38.886 INFO:tasks.workunit.client.0.smithi144.stdout:6/483: stat d9/dd/d3e/d83/daf/d5b/c5c 0 2024-09-10T11:29:38.887 INFO:tasks.workunit.client.0.smithi144.stdout:8/311: fsync d0/d9/d23/f50 0 2024-09-10T11:29:38.888 INFO:tasks.workunit.client.1.smithi178.stdout:4/697: creat d0/d8/da/d1a/d51/d5d/dc9/dee/ff0 x:0 0 0 2024-09-10T11:29:38.889 INFO:tasks.workunit.client.0.smithi144.stdout:7/443: rmdir d5/dc/d7b 0 2024-09-10T11:29:38.889 INFO:tasks.workunit.client.1.smithi178.stdout:0/766: mkdir d0/d12/d3f/d72/df0 0 2024-09-10T11:29:38.892 INFO:tasks.workunit.client.1.smithi178.stdout:7/732: write d6/d10/d1e/d2d/d2b/fc1 [435395,93453] 0 2024-09-10T11:29:38.895 INFO:tasks.workunit.client.0.smithi144.stdout:5/550: dwrite d4/d5/f23 [0,4194304] 0 2024-09-10T11:29:38.895 INFO:tasks.workunit.client.1.smithi178.stdout:6/839: dwrite d3/d74/d91/daa/dc2/d57/d78/d7c/f92 [0,4194304] 0 2024-09-10T11:29:38.897 INFO:tasks.workunit.client.0.smithi144.stdout:4/499: creat d14/d1e/d22/d2b/d35/d3c/da7/fb4 x:0 0 0 2024-09-10T11:29:38.899 INFO:tasks.workunit.client.0.smithi144.stdout:4/500: chown d14/d6f/laf 167920174 1 2024-09-10T11:29:38.901 INFO:tasks.workunit.client.1.smithi178.stdout:8/933: link d5/db/d1e/c43 d5/db7/c13e 0 2024-09-10T11:29:38.901 INFO:tasks.workunit.client.1.smithi178.stdout:4/698: mkdir d0/d9/dc/d8e/df1 0 2024-09-10T11:29:38.902 INFO:tasks.workunit.client.0.smithi144.stdout:3/529: dread d4/dd/f12 [0,4194304] 0 2024-09-10T11:29:38.903 INFO:tasks.workunit.client.0.smithi144.stdout:7/444: rename d5/dc/d2a/f35 to d5/dc/d14/d40/d5e/d68/f8f 0 2024-09-10T11:29:38.905 INFO:tasks.workunit.client.1.smithi178.stdout:5/978: dread d9/dc/dd/d37/dbd/dda/fa9 [0,4194304] 0 2024-09-10T11:29:38.909 INFO:tasks.workunit.client.0.smithi144.stdout:1/605: dread d5/d4a/d63/d84/dab/fbb [0,4194304] 0 2024-09-10T11:29:38.909 INFO:tasks.workunit.client.1.smithi178.stdout:8/934: mkdir d5/db/d10/d20/d52/d67/dbb/d13f 0 2024-09-10T11:29:38.910 INFO:tasks.workunit.client.0.smithi144.stdout:1/606: chown d5/dd/c2b 885261 1 2024-09-10T11:29:38.910 INFO:tasks.workunit.client.0.smithi144.stdout:1/607: chown d5/dd/c2b 1 1 2024-09-10T11:29:38.911 INFO:tasks.workunit.client.0.smithi144.stdout:4/501: creat d14/d67/d69/d90/fb5 x:0 0 0 2024-09-10T11:29:38.914 INFO:tasks.workunit.client.1.smithi178.stdout:7/733: dread d6/d10/d1d/daa/fd0 [0,4194304] 0 2024-09-10T11:29:38.916 INFO:tasks.workunit.client.0.smithi144.stdout:1/608: read d5/df/d4b/d51/da7/f7c [1932449,126834] 0 2024-09-10T11:29:38.917 INFO:tasks.workunit.client.0.smithi144.stdout:5/551: mkdir d4/d20/d24/da4 0 2024-09-10T11:29:38.918 INFO:tasks.workunit.client.1.smithi178.stdout:0/767: dread d0/dc/ff [0,4194304] 0 2024-09-10T11:29:38.921 INFO:tasks.workunit.client.0.smithi144.stdout:8/312: getdents d0/d9/d21/d25/d4f 0 2024-09-10T11:29:38.921 INFO:tasks.workunit.client.1.smithi178.stdout:2/980: dwrite d1/d41/d8d/fb7 [0,4194304] 0 2024-09-10T11:29:38.923 INFO:tasks.workunit.client.0.smithi144.stdout:4/502: creat d14/d1e/fb6 x:0 0 0 2024-09-10T11:29:38.924 INFO:tasks.workunit.client.1.smithi178.stdout:7/734: write d6/d10/d1e/d2d/d2b/fc1 [381420,96693] 0 2024-09-10T11:29:38.926 INFO:tasks.workunit.client.0.smithi144.stdout:1/609: dwrite d5/df/f93 [0,4194304] 0 2024-09-10T11:29:38.927 INFO:tasks.workunit.client.1.smithi178.stdout:3/872: sync 2024-09-10T11:29:38.927 INFO:tasks.workunit.client.0.smithi144.stdout:1/610: stat d5/df/d4b/cb5 0 2024-09-10T11:29:38.928 INFO:tasks.workunit.client.1.smithi178.stdout:8/935: dwrite d5/d30/d33/d3a/f129 [0,4194304] 0 2024-09-10T11:29:38.928 INFO:tasks.workunit.client.0.smithi144.stdout:5/552: dwrite d4/d5/d18/d67/f65 [0,4194304] 0 2024-09-10T11:29:38.929 INFO:tasks.workunit.client.1.smithi178.stdout:5/979: truncate d9/dc/dd/d37/f8d 3447571 0 2024-09-10T11:29:38.929 INFO:tasks.workunit.client.0.smithi144.stdout:5/553: readlink d4/d20/d22/l36 0 2024-09-10T11:29:38.929 INFO:tasks.workunit.client.1.smithi178.stdout:4/699: write d0/f75 [1544456,46243] 0 2024-09-10T11:29:38.932 INFO:tasks.workunit.client.0.smithi144.stdout:3/530: dread d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/f74 [0,4194304] 0 2024-09-10T11:29:38.933 INFO:tasks.workunit.client.1.smithi178.stdout:2/981: mkdir d1/d41/d5d/d14b 0 2024-09-10T11:29:38.933 INFO:tasks.workunit.client.0.smithi144.stdout:6/484: getdents d9/d38 0 2024-09-10T11:29:38.934 INFO:tasks.workunit.client.0.smithi144.stdout:3/531: dread - d4/dd/d21/f8b zero size 2024-09-10T11:29:38.936 INFO:tasks.workunit.client.0.smithi144.stdout:3/532: chown d4/d28/d67/d80/d17/d18/d34/l5e 173 1 2024-09-10T11:29:38.943 INFO:tasks.workunit.client.1.smithi178.stdout:0/768: creat d0/dc/d1f/d2d/dac/de7/ff1 x:0 0 0 2024-09-10T11:29:38.944 INFO:tasks.workunit.client.1.smithi178.stdout:9/877: sync 2024-09-10T11:29:38.944 INFO:tasks.workunit.client.1.smithi178.stdout:1/736: sync 2024-09-10T11:29:38.946 INFO:tasks.workunit.client.1.smithi178.stdout:0/769: write d0/dc/f2b [3436535,32618] 0 2024-09-10T11:29:38.947 INFO:tasks.workunit.client.1.smithi178.stdout:8/936: rmdir d5/db/d10/d53 39 2024-09-10T11:29:38.947 INFO:tasks.workunit.client.1.smithi178.stdout:6/840: dread d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/f10f [0,4194304] 0 2024-09-10T11:29:38.947 INFO:tasks.workunit.client.1.smithi178.stdout:7/735: symlink d6/d10/d1d/d4a/l104 0 2024-09-10T11:29:38.948 INFO:tasks.workunit.client.0.smithi144.stdout:8/313: rename d0/d1b/f2a to d0/d9/d1f/d34/f61 0 2024-09-10T11:29:38.948 INFO:tasks.workunit.client.1.smithi178.stdout:3/873: rmdir db/d20/d3d/d63/da2/d98 39 2024-09-10T11:29:38.950 INFO:tasks.workunit.client.1.smithi178.stdout:5/980: dwrite d9/dc/dd/d16/f27 [0,4194304] 0 2024-09-10T11:29:38.953 INFO:tasks.workunit.client.0.smithi144.stdout:3/533: chown d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93/d98/d9a 16407478 1 2024-09-10T11:29:38.953 INFO:tasks.workunit.client.0.smithi144.stdout:8/314: dread d0/d9/d1f/d34/f61 [0,4194304] 0 2024-09-10T11:29:38.956 INFO:tasks.workunit.client.1.smithi178.stdout:5/981: write d9/dc/dd/d37/d42/d62/f80 [3320523,48762] 0 2024-09-10T11:29:38.956 INFO:tasks.workunit.client.0.smithi144.stdout:1/611: mkdir d5/dd/dcb 0 2024-09-10T11:29:38.959 INFO:tasks.workunit.client.0.smithi144.stdout:5/554: mknod d4/d20/d24/ca5 0 2024-09-10T11:29:38.959 INFO:tasks.workunit.client.0.smithi144.stdout:7/445: link d5/dc/d12/d2f/c8b d5/dc/d12/d2f/d7c/c90 0 2024-09-10T11:29:38.959 INFO:tasks.workunit.client.0.smithi144.stdout:4/503: creat d14/d1e/d22/d2b/fb7 x:0 0 0 2024-09-10T11:29:38.961 INFO:tasks.workunit.client.0.smithi144.stdout:8/315: write d0/d9/d23/d37/f3a [1398929,73644] 0 2024-09-10T11:29:38.962 INFO:tasks.workunit.client.0.smithi144.stdout:4/504: chown d14/d1e/l2a 1134839 1 2024-09-10T11:29:38.962 INFO:tasks.workunit.client.0.smithi144.stdout:5/555: fdatasync d4/d5/d18/d38/d6d/d8b/f9b 0 2024-09-10T11:29:38.962 INFO:tasks.workunit.client.0.smithi144.stdout:8/316: chown d0/d1b/d5a/l5d 39 1 2024-09-10T11:29:38.963 INFO:tasks.workunit.client.1.smithi178.stdout:7/736: fdatasync d6/d10/d1d/f27 0 2024-09-10T11:29:38.963 INFO:tasks.workunit.client.1.smithi178.stdout:9/878: rename d2/df/d31/d4a to d2/df/d31/d37/d114 0 2024-09-10T11:29:38.964 INFO:tasks.workunit.client.1.smithi178.stdout:8/937: dwrite d5/d1d/f132 [0,4194304] 0 2024-09-10T11:29:38.965 INFO:tasks.workunit.client.0.smithi144.stdout:7/446: creat d5/dc/d2a/d60/f91 x:0 0 0 2024-09-10T11:29:38.968 INFO:tasks.workunit.client.0.smithi144.stdout:4/505: truncate fa 5689410 0 2024-09-10T11:29:38.968 INFO:tasks.workunit.client.0.smithi144.stdout:7/447: write d5/dc/d11/f17 [563837,80911] 0 2024-09-10T11:29:38.968 INFO:tasks.workunit.client.0.smithi144.stdout:1/612: symlink d5/d4a/lcc 0 2024-09-10T11:29:38.969 INFO:tasks.workunit.client.0.smithi144.stdout:8/317: fdatasync d0/d9/d23/d26/f30 0 2024-09-10T11:29:38.971 INFO:tasks.workunit.client.1.smithi178.stdout:9/879: truncate d2/d19/d5e/f9e 1020718 0 2024-09-10T11:29:38.971 INFO:tasks.workunit.client.1.smithi178.stdout:2/982: write d1/d7/d24/d46/f102 [306817,69924] 0 2024-09-10T11:29:38.971 INFO:tasks.workunit.client.0.smithi144.stdout:3/534: link d4/dd/d21/d2f/f3a d4/d28/d67/d80/d17/d18/d34/d78/fb2 0 2024-09-10T11:29:38.974 INFO:tasks.workunit.client.1.smithi178.stdout:0/770: rmdir d0 39 2024-09-10T11:29:38.974 INFO:tasks.workunit.client.1.smithi178.stdout:9/880: fsync d2/d9b/df8/da5/f105 0 2024-09-10T11:29:38.974 INFO:tasks.workunit.client.0.smithi144.stdout:1/613: creat d5/d2a/d65/fcd x:0 0 0 2024-09-10T11:29:38.974 INFO:tasks.workunit.client.0.smithi144.stdout:5/556: dwrite d4/d20/d22/d5c/d5d/d64/f96 [0,4194304] 0 2024-09-10T11:29:38.975 INFO:tasks.workunit.client.1.smithi178.stdout:5/982: truncate d9/dc/dd/d37/d3e/f94 878406 0 2024-09-10T11:29:38.976 INFO:tasks.workunit.client.1.smithi178.stdout:7/737: creat d6/d10/d1d/d22/d50/d98/da6/dd5/f105 x:0 0 0 2024-09-10T11:29:38.977 INFO:tasks.workunit.client.0.smithi144.stdout:1/614: dread - d5/d2a/d65/faf zero size 2024-09-10T11:29:38.977 INFO:tasks.workunit.client.1.smithi178.stdout:1/737: write d3/f40 [3765348,112315] 0 2024-09-10T11:29:38.982 INFO:tasks.workunit.client.0.smithi144.stdout:1/615: readlink d5/dd/d64/d79/l7f 0 2024-09-10T11:29:38.983 INFO:tasks.workunit.client.1.smithi178.stdout:6/841: rename d3/d74/d91/daa/dc2/d97/l9e to d3/d74/d91/daa/dc2/dcf/l133 0 2024-09-10T11:29:38.983 INFO:tasks.workunit.client.1.smithi178.stdout:4/700: truncate d0/d9/d3d/f44 3778873 0 2024-09-10T11:29:38.985 INFO:tasks.workunit.client.1.smithi178.stdout:4/701: dread - d0/d8/da/fed zero size 2024-09-10T11:29:38.986 INFO:tasks.workunit.client.1.smithi178.stdout:1/738: dwrite d3/d9/d98/dac/d3a/f53 [4194304,4194304] 0 2024-09-10T11:29:38.988 INFO:tasks.workunit.client.0.smithi144.stdout:7/448: dwrite d5/dc/d14/f50 [0,4194304] 0 2024-09-10T11:29:38.988 INFO:tasks.workunit.client.1.smithi178.stdout:3/874: getdents db/d20/d70 0 2024-09-10T11:29:38.990 INFO:tasks.workunit.client.0.smithi144.stdout:6/485: dread d9/dd/d3e/d47/d13/f19 [0,4194304] 0 2024-09-10T11:29:38.991 INFO:tasks.workunit.client.1.smithi178.stdout:3/875: truncate db/d35/d3a/d42/d46/d110/f102 946527 0 2024-09-10T11:29:38.991 INFO:tasks.workunit.client.1.smithi178.stdout:3/876: dread - db/d9b/fe9 zero size 2024-09-10T11:29:38.991 INFO:tasks.workunit.client.1.smithi178.stdout:2/983: unlink d1/d7/d24/d46/d88/d9d/dab/d5e/fdc 0 2024-09-10T11:29:38.992 INFO:tasks.workunit.client.0.smithi144.stdout:4/506: dwrite d14/d1e/d22/d2b/d35/f5b [0,4194304] 0 2024-09-10T11:29:38.997 INFO:tasks.workunit.client.1.smithi178.stdout:9/881: creat d2/d9b/df8/da5/f115 x:0 0 0 2024-09-10T11:29:38.999 INFO:tasks.workunit.client.0.smithi144.stdout:3/535: dwrite d4/d1a/f44 [0,4194304] 0 2024-09-10T11:29:38.999 INFO:tasks.workunit.client.1.smithi178.stdout:5/983: mkdir d9/dc/dd/d3f/d142 0 2024-09-10T11:29:38.999 INFO:tasks.workunit.client.1.smithi178.stdout:7/738: fsync d6/d10/d1d/d22/d93/fa8 0 2024-09-10T11:29:38.999 INFO:tasks.workunit.client.0.smithi144.stdout:6/486: truncate d9/dd/f1a 9057129 0 2024-09-10T11:29:38.999 INFO:tasks.workunit.client.0.smithi144.stdout:8/318: link d0/l2d d0/d1b/l62 0 2024-09-10T11:29:39.000 INFO:tasks.workunit.client.0.smithi144.stdout:7/449: read d5/dc/d11/f5c [3479047,112634] 0 2024-09-10T11:29:39.002 INFO:tasks.workunit.client.1.smithi178.stdout:7/739: dread - d6/d10/d1d/d22/d50/d98/da6/dd5/f105 zero size 2024-09-10T11:29:39.006 INFO:tasks.workunit.client.1.smithi178.stdout:6/842: creat d3/d6e/de9/f134 x:0 0 0 2024-09-10T11:29:39.007 INFO:tasks.workunit.client.1.smithi178.stdout:4/702: chown d0/d9/dc/l83 201499 1 2024-09-10T11:29:39.008 INFO:tasks.workunit.client.1.smithi178.stdout:8/938: chown d5/db/d10/d20/dc0/d109/c11b 92779526 1 2024-09-10T11:29:39.010 INFO:tasks.workunit.client.1.smithi178.stdout:3/877: mkdir db/d20/d3d/d63/d11d 0 2024-09-10T11:29:39.029 INFO:tasks.workunit.client.1.smithi178.stdout:5/984: symlink d9/dc/d17/dcd/l143 0 2024-09-10T11:29:39.030 INFO:tasks.workunit.client.0.smithi144.stdout:4/507: read d14/d1e/d22/f23 [5122873,106872] 0 2024-09-10T11:29:39.031 INFO:tasks.workunit.client.0.smithi144.stdout:5/557: truncate d4/d5/d18/d67/f62 549198 0 2024-09-10T11:29:39.032 INFO:tasks.workunit.client.0.smithi144.stdout:8/319: rmdir d0/d9/d1f 39 2024-09-10T11:29:39.033 INFO:tasks.workunit.client.0.smithi144.stdout:4/508: write f5 [3484434,109129] 0 2024-09-10T11:29:39.034 INFO:tasks.workunit.client.0.smithi144.stdout:6/487: write d9/dd/d11/f1c [4038013,61429] 0 2024-09-10T11:29:39.034 INFO:tasks.workunit.client.1.smithi178.stdout:7/740: creat d6/d10/d1d/d22/d50/d7e/d82/db2/d49/f106 x:0 0 0 2024-09-10T11:29:39.036 INFO:tasks.workunit.client.0.smithi144.stdout:3/536: dread f2 [0,4194304] 0 2024-09-10T11:29:39.038 INFO:tasks.workunit.client.1.smithi178.stdout:6/843: write d3/d74/dc9/f47 [6229132,41669] 0 2024-09-10T11:29:39.040 INFO:tasks.workunit.client.1.smithi178.stdout:4/703: chown d0/d63/cb4 251770 1 2024-09-10T11:29:39.041 INFO:tasks.workunit.client.1.smithi178.stdout:0/771: truncate d0/dc/fe6 1399042 0 2024-09-10T11:29:39.042 INFO:tasks.workunit.client.1.smithi178.stdout:3/878: mkdir db/d20/d3d/d56/dcf/d11e 0 2024-09-10T11:29:39.044 INFO:tasks.workunit.client.1.smithi178.stdout:5/985: symlink d9/d19/l144 0 2024-09-10T11:29:39.044 INFO:tasks.workunit.client.1.smithi178.stdout:7/741: mknod d6/d10/c107 0 2024-09-10T11:29:39.046 INFO:tasks.workunit.client.1.smithi178.stdout:2/984: write d1/d7/d24/d7d/d84/d55/da1/fff [92859,40175] 0 2024-09-10T11:29:39.046 INFO:tasks.workunit.client.1.smithi178.stdout:5/986: chown d9/dc/ff4 13 1 2024-09-10T11:29:39.046 INFO:tasks.workunit.client.1.smithi178.stdout:6/844: dread d3/d74/d91/daa/dc2/d57/f68 [0,4194304] 0 2024-09-10T11:29:39.047 INFO:tasks.workunit.client.0.smithi144.stdout:7/450: creat d5/dc/d14/d40/d58/d6a/f92 x:0 0 0 2024-09-10T11:29:39.047 INFO:tasks.workunit.client.0.smithi144.stdout:5/558: mkdir d4/d5/d18/d38/d6d/da6 0 2024-09-10T11:29:39.047 INFO:tasks.workunit.client.0.smithi144.stdout:4/509: read d14/f15 [3243149,49943] 0 2024-09-10T11:29:39.049 INFO:tasks.workunit.client.0.smithi144.stdout:6/488: creat d9/dd/d73/fb1 x:0 0 0 2024-09-10T11:29:39.049 INFO:tasks.workunit.client.1.smithi178.stdout:8/939: creat d5/f140 x:0 0 0 2024-09-10T11:29:39.049 INFO:tasks.workunit.client.1.smithi178.stdout:4/704: dwrite d0/d8/da/d1a/f69 [4194304,4194304] 0 2024-09-10T11:29:39.053 INFO:tasks.workunit.client.0.smithi144.stdout:4/510: write d14/d1e/f20 [583083,46457] 0 2024-09-10T11:29:39.053 INFO:tasks.workunit.client.1.smithi178.stdout:3/879: read db/d20/d3d/d63/da2/d98/fbe [728155,119087] 0 2024-09-10T11:29:39.053 INFO:tasks.workunit.client.1.smithi178.stdout:7/742: unlink d6/d10/d34/d6a/f7f 0 2024-09-10T11:29:39.054 INFO:tasks.workunit.client.1.smithi178.stdout:2/985: creat d1/d7/d24/d43/f14c x:0 0 0 2024-09-10T11:29:39.055 INFO:tasks.workunit.client.1.smithi178.stdout:1/739: truncate d3/d35/f74 447806 0 2024-09-10T11:29:39.058 INFO:tasks.workunit.client.1.smithi178.stdout:0/772: creat d0/d2c/ff2 x:0 0 0 2024-09-10T11:29:39.059 INFO:tasks.workunit.client.1.smithi178.stdout:0/773: chown d0/dc/d39/d75/fd4 7104207 1 2024-09-10T11:29:39.059 INFO:tasks.workunit.client.1.smithi178.stdout:8/940: chown d5/db/d10/d35/f3c 2 1 2024-09-10T11:29:39.060 INFO:tasks.workunit.client.0.smithi144.stdout:8/320: dwrite d0/d9/d23/d26/f30 [0,4194304] 0 2024-09-10T11:29:39.061 INFO:tasks.workunit.client.0.smithi144.stdout:5/559: getdents d4/d5/d18/d67/d44 0 2024-09-10T11:29:39.064 INFO:tasks.workunit.client.0.smithi144.stdout:3/537: dwrite d4/d28/d67/d80/fa2 [0,4194304] 0 2024-09-10T11:29:39.065 INFO:tasks.workunit.client.0.smithi144.stdout:6/489: link d9/dd/d11/d1b/d2f/c86 d9/d1f/dae/cb2 0 2024-09-10T11:29:39.066 INFO:tasks.workunit.client.1.smithi178.stdout:3/880: rmdir db/d9b/dfb 39 2024-09-10T11:29:39.067 INFO:tasks.workunit.client.1.smithi178.stdout:3/881: chown db/ff 26 1 2024-09-10T11:29:39.069 INFO:tasks.workunit.client.0.smithi144.stdout:5/560: chown d4/d20/d22/d5c/d5d/d64/f8e 32125 1 2024-09-10T11:29:39.070 INFO:tasks.workunit.client.0.smithi144.stdout:8/321: mkdir d0/d9/d1f/d63 0 2024-09-10T11:29:39.070 INFO:tasks.workunit.client.0.smithi144.stdout:7/451: dwrite d5/f37 [0,4194304] 0 2024-09-10T11:29:39.071 INFO:tasks.workunit.client.1.smithi178.stdout:1/740: dread d3/d9/d98/dac/d3a/d9f/fa2 [0,4194304] 0 2024-09-10T11:29:39.074 INFO:tasks.workunit.client.0.smithi144.stdout:5/561: creat d4/d20/d24/d2d/d4b/fa7 x:0 0 0 2024-09-10T11:29:39.074 INFO:tasks.workunit.client.0.smithi144.stdout:3/538: mkdir d4/d28/d67/d80/d17/d18/d34/db3 0 2024-09-10T11:29:39.074 INFO:tasks.workunit.client.0.smithi144.stdout:8/322: fsync d0/d9/d23/d26/f4e 0 2024-09-10T11:29:39.079 INFO:tasks.workunit.client.0.smithi144.stdout:5/562: fsync d4/d52/f69 0 2024-09-10T11:29:39.081 INFO:tasks.workunit.client.1.smithi178.stdout:6/845: link d3/d74/dc9/f127 d3/d7/f135 0 2024-09-10T11:29:39.082 INFO:tasks.workunit.client.1.smithi178.stdout:0/774: mknod d0/dc/d1f/d2d/dac/cf3 0 2024-09-10T11:29:39.083 INFO:tasks.workunit.client.0.smithi144.stdout:3/539: symlink d4/d28/d67/d80/d17/lb4 0 2024-09-10T11:29:39.083 INFO:tasks.workunit.client.1.smithi178.stdout:0/775: stat d0/d2c/c3d 0 2024-09-10T11:29:39.084 INFO:tasks.workunit.client.0.smithi144.stdout:5/563: creat d4/d5/d18/d38/d8d/fa8 x:0 0 0 2024-09-10T11:29:39.086 INFO:tasks.workunit.client.0.smithi144.stdout:7/452: getdents d5/dc/d14/d40/d5e/d68 0 2024-09-10T11:29:39.086 INFO:tasks.workunit.client.0.smithi144.stdout:8/323: dwrite d0/d1b/f2f [4194304,4194304] 0 2024-09-10T11:29:39.086 INFO:tasks.workunit.client.1.smithi178.stdout:5/987: link d9/dc/dd/d37/f75 d9/dc/dd/d37/d42/d62/daf/d138/f145 0 2024-09-10T11:29:39.087 INFO:tasks.workunit.client.0.smithi144.stdout:4/511: dread fd [0,4194304] 0 2024-09-10T11:29:39.087 INFO:tasks.workunit.client.1.smithi178.stdout:3/882: dread f1 [0,4194304] 0 2024-09-10T11:29:39.088 INFO:tasks.workunit.client.1.smithi178.stdout:5/988: chown d9/dc/dd/d3f/dd6/d124/fd0 0 1 2024-09-10T11:29:39.088 INFO:tasks.workunit.client.0.smithi144.stdout:5/564: chown d4/d5/d18/l9d 97 1 2024-09-10T11:29:39.089 INFO:tasks.workunit.client.1.smithi178.stdout:3/883: write db/d35/d58/f7b [1455267,18796] 0 2024-09-10T11:29:39.089 INFO:tasks.workunit.client.0.smithi144.stdout:4/512: truncate d14/d1e/f50 4817295 0 2024-09-10T11:29:39.090 INFO:tasks.workunit.client.1.smithi178.stdout:4/705: creat d0/d8/da/d1a/d51/d5d/d62/ff2 x:0 0 0 2024-09-10T11:29:39.091 INFO:tasks.workunit.client.0.smithi144.stdout:7/453: unlink d5/dc/d12/f88 0 2024-09-10T11:29:39.091 INFO:tasks.workunit.client.0.smithi144.stdout:6/490: dread d9/f2a [0,4194304] 0 2024-09-10T11:29:39.093 INFO:tasks.workunit.client.0.smithi144.stdout:6/491: stat d9/dd/d3e/l7c 0 2024-09-10T11:29:39.093 INFO:tasks.workunit.client.1.smithi178.stdout:7/743: creat d6/d10/d34/d6a/f108 x:0 0 0 2024-09-10T11:29:39.095 INFO:tasks.workunit.client.0.smithi144.stdout:7/454: mkdir d5/dc/d12/d2f/d93 0 2024-09-10T11:29:39.098 INFO:tasks.workunit.client.0.smithi144.stdout:7/455: stat d5/dc/d12/d2b/c65 0 2024-09-10T11:29:39.099 INFO:tasks.workunit.client.1.smithi178.stdout:3/884: fsync db/d20/f28 0 2024-09-10T11:29:39.101 INFO:tasks.workunit.client.1.smithi178.stdout:0/776: symlink d0/dc/d1f/dea/lf4 0 2024-09-10T11:29:39.102 INFO:tasks.workunit.client.1.smithi178.stdout:3/885: dread db/d20/f28 [0,4194304] 0 2024-09-10T11:29:39.102 INFO:tasks.workunit.client.1.smithi178.stdout:0/777: chown d0/dc/d39/d50 176321 1 2024-09-10T11:29:39.102 INFO:tasks.workunit.client.0.smithi144.stdout:4/513: link d14/d1e/d22/f48 d14/d1e/d22/d2b/d35/d3c/fb8 0 2024-09-10T11:29:39.103 INFO:tasks.workunit.client.0.smithi144.stdout:6/492: dwrite d9/dd/d3e/d47/d13/f1e [0,4194304] 0 2024-09-10T11:29:39.103 INFO:tasks.workunit.client.1.smithi178.stdout:7/744: creat d6/d10/f109 x:0 0 0 2024-09-10T11:29:39.104 INFO:tasks.workunit.client.1.smithi178.stdout:1/741: link d3/d9/d98/dd1/ddc/lb0 d3/d9/d98/dac/d3a/d6e/d8a/l112 0 2024-09-10T11:29:39.105 INFO:tasks.workunit.client.0.smithi144.stdout:7/456: symlink d5/dc/d14/d40/d58/d6a/l94 0 2024-09-10T11:29:39.106 INFO:tasks.workunit.client.1.smithi178.stdout:7/745: dread - d6/d10/d1d/d22/d50/d98/da6/dd5/f105 zero size 2024-09-10T11:29:39.107 INFO:tasks.workunit.client.1.smithi178.stdout:2/986: dread d1/d7/d24/d7d/d84/d4b/f66 [0,4194304] 0 2024-09-10T11:29:39.108 INFO:tasks.workunit.client.1.smithi178.stdout:3/886: dread db/d35/d58/f6e [0,4194304] 0 2024-09-10T11:29:39.108 INFO:tasks.workunit.client.0.smithi144.stdout:8/324: dread d0/f3f [0,4194304] 0 2024-09-10T11:29:39.108 INFO:tasks.workunit.client.0.smithi144.stdout:4/514: creat d14/d1e/d22/d2b/d35/fb9 x:0 0 0 2024-09-10T11:29:39.108 INFO:tasks.workunit.client.0.smithi144.stdout:8/325: chown d0/d9 2 1 2024-09-10T11:29:39.109 INFO:tasks.workunit.client.1.smithi178.stdout:2/987: chown d1/d7/d24/d43/c50 124 1 2024-09-10T11:29:39.109 INFO:tasks.workunit.client.0.smithi144.stdout:8/326: write d0/d9/d23/d26/f4a [1079865,117466] 0 2024-09-10T11:29:39.110 INFO:tasks.workunit.client.0.smithi144.stdout:6/493: rmdir d9/dd/d3e 39 2024-09-10T11:29:39.112 INFO:tasks.workunit.client.1.smithi178.stdout:4/706: mkdir d0/d8/da/d1a/d51/d5d/db6/df3 0 2024-09-10T11:29:39.112 INFO:tasks.workunit.client.1.smithi178.stdout:8/941: write d5/d61/f8f [5940,103463] 0 2024-09-10T11:29:39.116 INFO:tasks.workunit.client.0.smithi144.stdout:8/327: rmdir d0/d9/d21 39 2024-09-10T11:29:39.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:38 smithi178 ceph-mon[33514]: pgmap v26: 65 pgs: 65 active+clean; 2.0 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 65 MiB/s rd, 242 MiB/s wr, 408 op/s 2024-09-10T11:29:39.117 INFO:tasks.workunit.client.1.smithi178.stdout:8/942: read d5/db/d10/d35/f3c [3512216,4035] 0 2024-09-10T11:29:39.117 INFO:tasks.workunit.client.1.smithi178.stdout:3/887: mknod db/d35/d58/c11f 0 2024-09-10T11:29:39.118 INFO:tasks.workunit.client.0.smithi144.stdout:6/494: rmdir d9/d1f 39 2024-09-10T11:29:39.118 INFO:tasks.workunit.client.1.smithi178.stdout:4/707: dread d0/d10/fa2 [0,4194304] 0 2024-09-10T11:29:39.118 INFO:tasks.workunit.client.0.smithi144.stdout:8/328: write d0/d9/fe [2882661,9111] 0 2024-09-10T11:29:39.120 INFO:tasks.workunit.client.1.smithi178.stdout:8/943: write d5/d30/d33/d75/f133 [120505,96055] 0 2024-09-10T11:29:39.120 INFO:tasks.workunit.client.0.smithi144.stdout:5/565: dwrite d4/d5/d32/d47/f6a [0,4194304] 0 2024-09-10T11:29:39.120 INFO:tasks.workunit.client.0.smithi144.stdout:7/457: truncate d5/dc/d14/d40/d5e/d63/f77 665563 0 2024-09-10T11:29:39.121 INFO:tasks.workunit.client.1.smithi178.stdout:4/708: write d0/d8/da/d1a/d51/d5d/fa1 [964864,123520] 0 2024-09-10T11:29:39.123 INFO:tasks.workunit.client.1.smithi178.stdout:4/709: chown d0/d10/ce9 32031 1 2024-09-10T11:29:39.123 INFO:tasks.workunit.client.0.smithi144.stdout:5/566: write d4/d20/f43 [3021446,19681] 0 2024-09-10T11:29:39.123 INFO:tasks.workunit.client.0.smithi144.stdout:7/458: dread - d5/dc/d11/f81 zero size 2024-09-10T11:29:39.123 INFO:tasks.workunit.client.0.smithi144.stdout:8/329: write d0/d9/d1f/d34/f53 [908758,7130] 0 2024-09-10T11:29:39.124 INFO:tasks.workunit.client.1.smithi178.stdout:2/988: dread d1/d7/d24/d46/d88/dc6/ff2 [0,4194304] 0 2024-09-10T11:29:39.124 INFO:tasks.workunit.client.1.smithi178.stdout:3/888: rmdir db/d26/d40/dcc 39 2024-09-10T11:29:39.126 INFO:tasks.workunit.client.1.smithi178.stdout:8/944: symlink d5/db/d1e/l141 0 2024-09-10T11:29:39.127 INFO:tasks.workunit.client.0.smithi144.stdout:6/495: chown d9/d1f/d2c/l3d 0 1 2024-09-10T11:29:39.129 INFO:tasks.workunit.client.1.smithi178.stdout:1/742: creat d3/d9/d98/dac/d3a/d50/d6c/d78/f113 x:0 0 0 2024-09-10T11:29:39.129 INFO:tasks.workunit.client.1.smithi178.stdout:0/778: write d0/dc/d39/f5b [1037464,109097] 0 2024-09-10T11:29:39.131 INFO:tasks.workunit.client.1.smithi178.stdout:7/746: creat d6/d10/d1d/d22/d93/f10a x:0 0 0 2024-09-10T11:29:39.131 INFO:tasks.workunit.client.1.smithi178.stdout:4/710: mknod d0/d9/dc/d50/cf4 0 2024-09-10T11:29:39.131 INFO:tasks.workunit.client.1.smithi178.stdout:0/779: dread - d0/dc/fd6 zero size 2024-09-10T11:29:39.132 INFO:tasks.workunit.client.0.smithi144.stdout:8/330: creat d0/d1b/d47/d5e/f64 x:0 0 0 2024-09-10T11:29:39.133 INFO:tasks.workunit.client.0.smithi144.stdout:7/459: dwrite d5/dc/d14/d40/d58/d6a/f92 [0,4194304] 0 2024-09-10T11:29:39.133 INFO:tasks.workunit.client.1.smithi178.stdout:3/889: creat db/d35/d3a/d9f/f120 x:0 0 0 2024-09-10T11:29:39.135 INFO:tasks.workunit.client.0.smithi144.stdout:7/460: stat d5/dc/d12/f52 0 2024-09-10T11:29:39.135 INFO:tasks.workunit.client.0.smithi144.stdout:5/567: getdents d4/d52 0 2024-09-10T11:29:39.136 INFO:tasks.workunit.client.0.smithi144.stdout:6/496: creat d9/dd/d11/d1b/d2f/d7f/d82/fb3 x:0 0 0 2024-09-10T11:29:39.137 INFO:tasks.workunit.client.0.smithi144.stdout:7/461: stat d5/dc/d12/d2f/d93 0 2024-09-10T11:29:39.138 INFO:tasks.workunit.client.1.smithi178.stdout:8/945: mkdir d5/d142 0 2024-09-10T11:29:39.140 INFO:tasks.workunit.client.1.smithi178.stdout:8/946: fsync d5/db/d10/d20/fcc 0 2024-09-10T11:29:39.143 INFO:tasks.workunit.client.0.smithi144.stdout:8/331: dwrite d0/d9/d23/d26/f36 [4194304,4194304] 0 2024-09-10T11:29:39.143 INFO:tasks.workunit.client.1.smithi178.stdout:1/743: creat d3/d9/f114 x:0 0 0 2024-09-10T11:29:39.143 INFO:tasks.workunit.client.1.smithi178.stdout:7/747: dwrite d6/d10/d34/f6e [0,4194304] 0 2024-09-10T11:29:39.144 INFO:tasks.workunit.client.1.smithi178.stdout:1/744: fdatasync d3/d9/d1a/d48/f10b 0 2024-09-10T11:29:39.147 INFO:tasks.workunit.client.1.smithi178.stdout:8/947: mknod d5/d30/db8/c143 0 2024-09-10T11:29:39.147 INFO:tasks.workunit.client.0.smithi144.stdout:6/497: mknod d9/dd/d3e/d47/cb4 0 2024-09-10T11:29:39.147 INFO:tasks.workunit.client.1.smithi178.stdout:8/948: chown d5/db/d3d/de7 100750 1 2024-09-10T11:29:39.148 INFO:tasks.workunit.client.0.smithi144.stdout:5/568: dwrite d4/d5/d18/d67/d44/f93 [0,4194304] 0 2024-09-10T11:29:39.149 INFO:tasks.workunit.client.1.smithi178.stdout:7/748: readlink d6/d10/d1e/d2d/d2b/da0/d55/lae 0 2024-09-10T11:29:39.149 INFO:tasks.workunit.client.0.smithi144.stdout:7/462: creat d5/dc/d2a/f95 x:0 0 0 2024-09-10T11:29:39.150 INFO:tasks.workunit.client.1.smithi178.stdout:8/949: write d5/d61/fa2 [5021198,17647] 0 2024-09-10T11:29:39.151 INFO:tasks.workunit.client.0.smithi144.stdout:7/463: write d5/dc/d14/f50 [2210668,107112] 0 2024-09-10T11:29:39.152 INFO:tasks.workunit.client.1.smithi178.stdout:7/749: fsync d6/dbf/fcc 0 2024-09-10T11:29:39.152 INFO:tasks.workunit.client.0.smithi144.stdout:6/498: creat d9/dd/d3e/d83/daf/d5b/d91/fb5 x:0 0 0 2024-09-10T11:29:39.154 INFO:tasks.workunit.client.0.smithi144.stdout:8/332: creat d0/d9/d21/d25/d4f/f65 x:0 0 0 2024-09-10T11:29:39.155 INFO:tasks.workunit.client.1.smithi178.stdout:1/745: unlink d3/d9/d98/dac/d3a/c3e 0 2024-09-10T11:29:39.156 INFO:tasks.workunit.client.1.smithi178.stdout:7/750: mkdir d6/d10/d10b 0 2024-09-10T11:29:39.158 INFO:tasks.workunit.client.0.smithi144.stdout:7/464: write d5/dc/d14/d40/d5e/d68/f8f [977107,101964] 0 2024-09-10T11:29:39.159 INFO:tasks.workunit.client.0.smithi144.stdout:7/465: chown d5/dc/d14/d40/d5e 470683192 1 2024-09-10T11:29:39.159 INFO:tasks.workunit.client.1.smithi178.stdout:8/950: creat d5/db/d10/d20/d52/d67/dbb/d13f/f144 x:0 0 0 2024-09-10T11:29:39.159 INFO:tasks.workunit.client.1.smithi178.stdout:4/711: truncate d0/d8/d3e/f4c 3180742 0 2024-09-10T11:29:39.160 INFO:tasks.workunit.client.0.smithi144.stdout:8/333: mknod d0/d9/d23/d37/d48/c66 0 2024-09-10T11:29:39.161 INFO:tasks.workunit.client.1.smithi178.stdout:3/890: truncate db/d20/f3c 2743981 0 2024-09-10T11:29:39.162 INFO:tasks.workunit.client.1.smithi178.stdout:7/751: write d6/d10/d1d/d22/d3b/f5a [2270012,41279] 0 2024-09-10T11:29:39.164 INFO:tasks.workunit.client.1.smithi178.stdout:0/780: dwrite d0/d12/f2f [4194304,4194304] 0 2024-09-10T11:29:39.166 INFO:tasks.workunit.client.0.smithi144.stdout:7/466: symlink d5/dc/d14/d40/d58/d6e/l96 0 2024-09-10T11:29:39.166 INFO:tasks.workunit.client.1.smithi178.stdout:1/746: mknod d3/d9/d21/d6f/d104/c115 0 2024-09-10T11:29:39.167 INFO:tasks.workunit.client.1.smithi178.stdout:0/781: stat d0/dc/f13 0 2024-09-10T11:29:39.167 INFO:tasks.workunit.client.0.smithi144.stdout:6/499: write d9/d1f/f36 [1018990,109131] 0 2024-09-10T11:29:39.167 INFO:tasks.workunit.client.1.smithi178.stdout:4/712: symlink d0/d8/df/dde/lf5 0 2024-09-10T11:29:39.169 INFO:tasks.workunit.client.1.smithi178.stdout:4/713: chown d0/d9d/da5/cef 893 1 2024-09-10T11:29:39.172 INFO:tasks.workunit.client.1.smithi178.stdout:7/752: creat d6/d10/d1d/d22/d50/d7e/d82/db2/d49/f10c x:0 0 0 2024-09-10T11:29:39.172 INFO:tasks.workunit.client.1.smithi178.stdout:3/891: symlink db/d9b/l121 0 2024-09-10T11:29:39.173 INFO:tasks.workunit.client.1.smithi178.stdout:3/892: fsync db/d35/d3a/d42/d46/f4e 0 2024-09-10T11:29:39.174 INFO:tasks.workunit.client.1.smithi178.stdout:1/747: rename d3/d9/d98/dac/d8d/fbe to d3/d9/d21/d4b/d2b/dfd/f116 0 2024-09-10T11:29:39.175 INFO:tasks.workunit.client.0.smithi144.stdout:8/334: dwrite d0/f3f [0,4194304] 0 2024-09-10T11:29:39.175 INFO:tasks.workunit.client.0.smithi144.stdout:6/500: dread d9/dd/d3e/d83/daf/d5b/d91/f6e [0,4194304] 0 2024-09-10T11:29:39.177 INFO:tasks.workunit.client.0.smithi144.stdout:7/467: dwrite d5/dc/d14/f44 [0,4194304] 0 2024-09-10T11:29:39.178 INFO:tasks.workunit.client.1.smithi178.stdout:7/753: creat d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d70/f10d x:0 0 0 2024-09-10T11:29:39.179 INFO:tasks.workunit.client.0.smithi144.stdout:6/501: rmdir d9/d1f 39 2024-09-10T11:29:39.180 INFO:tasks.workunit.client.1.smithi178.stdout:4/714: creat d0/d9/ff6 x:0 0 0 2024-09-10T11:29:39.181 INFO:tasks.workunit.client.0.smithi144.stdout:8/335: creat d0/d9/d23/d26/f67 x:0 0 0 2024-09-10T11:29:39.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:38 smithi144 ceph-mon[27942]: pgmap v26: 65 pgs: 65 active+clean; 2.0 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 65 MiB/s rd, 242 MiB/s wr, 408 op/s 2024-09-10T11:29:39.184 INFO:tasks.workunit.client.1.smithi178.stdout:3/893: dwrite db/d26/d74/dd0/ff7 [0,4194304] 0 2024-09-10T11:29:39.188 INFO:tasks.workunit.client.1.smithi178.stdout:4/715: chown d0/d9d/dd5 207540829 1 2024-09-10T11:29:39.190 INFO:tasks.workunit.client.1.smithi178.stdout:1/748: getdents d3/d9/d98/dd1 0 2024-09-10T11:29:39.191 INFO:tasks.workunit.client.1.smithi178.stdout:3/894: dread db/d35/d3a/d42/d46/d110/f102 [0,4194304] 0 2024-09-10T11:29:39.193 INFO:tasks.workunit.client.1.smithi178.stdout:3/895: dread - db/d20/d3d/d63/da2/d98/fbc zero size 2024-09-10T11:29:39.194 INFO:tasks.workunit.client.0.smithi144.stdout:8/336: dwrite d0/d9/d23/d37/f5b [0,4194304] 0 2024-09-10T11:29:39.194 INFO:tasks.workunit.client.1.smithi178.stdout:3/896: stat db/d20/d3e/d6b/fd4 0 2024-09-10T11:29:39.196 INFO:tasks.workunit.client.0.smithi144.stdout:7/468: rmdir d5/dc/d12/d2b/d78 0 2024-09-10T11:29:39.198 INFO:tasks.workunit.client.0.smithi144.stdout:6/502: getdents d9/dd/d3e/d47/d13/d60 0 2024-09-10T11:29:39.201 INFO:tasks.workunit.client.1.smithi178.stdout:4/716: link d0/d8/da/fed d0/d8/da/d1a/ff7 0 2024-09-10T11:29:39.203 INFO:tasks.workunit.client.1.smithi178.stdout:1/749: dwrite d3/d9/d98/dac/d3a/d50/d6c/d78/f113 [0,4194304] 0 2024-09-10T11:29:39.204 INFO:tasks.workunit.client.0.smithi144.stdout:6/503: dwrite d9/dd/d3e/d47/f28 [4194304,4194304] 0 2024-09-10T11:29:39.205 INFO:tasks.workunit.client.0.smithi144.stdout:7/469: symlink d5/dc/d14/d40/d58/d6e/d7a/l97 0 2024-09-10T11:29:39.209 INFO:tasks.workunit.client.0.smithi144.stdout:7/470: mkdir d5/dc/d14/d40/d5e/d63/d98 0 2024-09-10T11:29:39.209 INFO:tasks.workunit.client.1.smithi178.stdout:4/717: fdatasync d0/d9/d3d/d4f/f97 0 2024-09-10T11:29:39.211 INFO:tasks.workunit.client.0.smithi144.stdout:8/337: rename d0/d9/d23/d37/d48 to d0/d9/d1f/d34/d68 0 2024-09-10T11:29:39.212 INFO:tasks.workunit.client.1.smithi178.stdout:1/750: chown d3/d9/d98/dd1/d7e/c8b 70 1 2024-09-10T11:29:39.213 INFO:tasks.workunit.client.0.smithi144.stdout:7/471: rmdir d5/dc/d12/d2b 39 2024-09-10T11:29:39.215 INFO:tasks.workunit.client.1.smithi178.stdout:7/754: dread d6/d10/d1d/d4a/f62 [0,4194304] 0 2024-09-10T11:29:39.216 INFO:tasks.workunit.client.1.smithi178.stdout:1/751: write d3/d35/f105 [745228,45267] 0 2024-09-10T11:29:39.217 INFO:tasks.workunit.client.0.smithi144.stdout:6/504: creat d9/d1f/fb6 x:0 0 0 2024-09-10T11:29:39.218 INFO:tasks.workunit.client.1.smithi178.stdout:8/951: dwrite d5/d30/f127 [4194304,4194304] 0 2024-09-10T11:29:39.219 INFO:tasks.workunit.client.1.smithi178.stdout:7/755: chown d6/d10/d1d/d22/d50/d98/da6 950 1 2024-09-10T11:29:39.220 INFO:tasks.workunit.client.0.smithi144.stdout:8/338: dread d0/d9/d23/d26/f4e [0,4194304] 0 2024-09-10T11:29:39.222 INFO:tasks.workunit.client.0.smithi144.stdout:8/339: read d0/d9/d23/f40 [2704730,92361] 0 2024-09-10T11:29:39.223 INFO:tasks.workunit.client.0.smithi144.stdout:8/340: write d0/d9/d21/d25/f60 [703948,92667] 0 2024-09-10T11:29:39.224 INFO:tasks.workunit.client.1.smithi178.stdout:4/718: creat d0/d63/de0/ff8 x:0 0 0 2024-09-10T11:29:39.225 INFO:tasks.workunit.client.0.smithi144.stdout:8/341: write d0/d9/d23/d37/f5b [4873186,4949] 0 2024-09-10T11:29:39.227 INFO:tasks.workunit.client.0.smithi144.stdout:7/472: rmdir d5/dc/d14/d40/d5e 39 2024-09-10T11:29:39.227 INFO:tasks.workunit.client.1.smithi178.stdout:3/897: dread db/ff [0,4194304] 0 2024-09-10T11:29:39.227 INFO:tasks.workunit.client.1.smithi178.stdout:4/719: write d0/d8/da/d1a/d51/d5d/fc5 [173119,121323] 0 2024-09-10T11:29:39.233 INFO:tasks.workunit.client.0.smithi144.stdout:8/342: truncate d0/d9/d23/f50 3448195 0 2024-09-10T11:29:39.233 INFO:tasks.workunit.client.1.smithi178.stdout:7/756: link d6/c6b d6/d10/d1d/d4a/d79/c10e 0 2024-09-10T11:29:39.235 INFO:tasks.workunit.client.0.smithi144.stdout:7/473: dwrite d5/dc/d2a/d60/f91 [0,4194304] 0 2024-09-10T11:29:39.237 INFO:tasks.workunit.client.0.smithi144.stdout:8/343: creat d0/d9/d23/d26/f69 x:0 0 0 2024-09-10T11:29:39.238 INFO:tasks.workunit.client.0.smithi144.stdout:8/344: chown d0/d9/d23/l3c 1828952 1 2024-09-10T11:29:39.241 INFO:tasks.workunit.client.1.smithi178.stdout:1/752: dread d3/d35/fb4 [0,4194304] 0 2024-09-10T11:29:39.241 INFO:tasks.workunit.client.1.smithi178.stdout:8/952: dread d5/db/d3d/fbe [0,4194304] 0 2024-09-10T11:29:39.242 INFO:tasks.workunit.client.0.smithi144.stdout:8/345: rename d0/d9/d23/d37/f5b to d0/d1b/d5a/f6a 0 2024-09-10T11:29:39.242 INFO:tasks.workunit.client.1.smithi178.stdout:7/757: dwrite d6/d10/d1d/d4a/d79/f85 [0,4194304] 0 2024-09-10T11:29:39.243 INFO:tasks.workunit.client.0.smithi144.stdout:7/474: link d5/dc/d11/d18/f5f d5/f99 0 2024-09-10T11:29:39.245 INFO:tasks.workunit.client.1.smithi178.stdout:7/758: write d6/d10/d1d/d22/d50/d98/da6/ffe [868878,3801] 0 2024-09-10T11:29:39.250 INFO:tasks.workunit.client.1.smithi178.stdout:1/753: dwrite d3/d9/d98/dd1/fcf [0,4194304] 0 2024-09-10T11:29:39.254 INFO:tasks.workunit.client.1.smithi178.stdout:1/754: creat d3/d35/ddd/f117 x:0 0 0 2024-09-10T11:29:39.255 INFO:tasks.workunit.client.1.smithi178.stdout:4/720: write d0/d9/f13 [3494893,89394] 0 2024-09-10T11:29:39.256 INFO:tasks.workunit.client.1.smithi178.stdout:7/759: dwrite d6/d10/d1d/d22/d50/d98/da6/ffe [0,4194304] 0 2024-09-10T11:29:39.262 INFO:tasks.workunit.client.0.smithi144.stdout:8/346: dwrite d0/f24 [0,4194304] 0 2024-09-10T11:29:39.264 INFO:tasks.workunit.client.1.smithi178.stdout:4/721: dwrite d0/d63/de0/ff8 [0,4194304] 0 2024-09-10T11:29:39.265 INFO:tasks.workunit.client.1.smithi178.stdout:1/755: rename d3/d9/d98/dac/d3a/d50/d6c/d78 to d3/d9/d98/dac/d3a/d50/d9e/dc2/de2/d118 0 2024-09-10T11:29:39.266 INFO:tasks.workunit.client.0.smithi144.stdout:7/475: dread d5/dc/d11/d18/f24 [0,4194304] 0 2024-09-10T11:29:39.267 INFO:tasks.workunit.client.0.smithi144.stdout:8/347: chown d0/d1b/d47/c57 528742769 1 2024-09-10T11:29:39.268 INFO:tasks.workunit.client.1.smithi178.stdout:4/722: creat d0/d10/ff9 x:0 0 0 2024-09-10T11:29:39.269 INFO:tasks.workunit.client.0.smithi144.stdout:7/476: creat d5/dc/d11/d18/f9a x:0 0 0 2024-09-10T11:29:39.270 INFO:tasks.workunit.client.1.smithi178.stdout:1/756: creat d3/d9/d21/d4b/d2b/dfd/f119 x:0 0 0 2024-09-10T11:29:39.273 INFO:tasks.workunit.client.1.smithi178.stdout:4/723: mknod d0/d8/da/d1a/d51/cfa 0 2024-09-10T11:29:39.273 INFO:tasks.workunit.client.0.smithi144.stdout:8/348: truncate d0/d9/d23/f50 3677803 0 2024-09-10T11:29:39.276 INFO:tasks.workunit.client.1.smithi178.stdout:7/760: truncate d6/d10/d1e/d2d/d2b/da0/f7a 1139835 0 2024-09-10T11:29:39.276 INFO:tasks.workunit.client.0.smithi144.stdout:8/349: symlink d0/d1b/l6b 0 2024-09-10T11:29:39.277 INFO:tasks.workunit.client.1.smithi178.stdout:1/757: getdents d3/d9/d98/dac/d3a/d9f 0 2024-09-10T11:29:39.279 INFO:tasks.workunit.client.0.smithi144.stdout:7/477: getdents d5/dc/d14/d40/d58/d6e 0 2024-09-10T11:29:39.279 INFO:tasks.workunit.client.1.smithi178.stdout:4/724: dread - d0/d10/fe5 zero size 2024-09-10T11:29:39.280 INFO:tasks.workunit.client.0.smithi144.stdout:7/478: dread - d5/dc/d12/d2f/d7c/f85 zero size 2024-09-10T11:29:39.280 INFO:tasks.workunit.client.0.smithi144.stdout:7/479: readlink d5/lf 0 2024-09-10T11:29:39.282 INFO:tasks.workunit.client.0.smithi144.stdout:8/350: getdents d0/d9/d21/d25 0 2024-09-10T11:29:39.282 INFO:tasks.workunit.client.1.smithi178.stdout:7/761: creat d6/d10/d1d/dca/f10f x:0 0 0 2024-09-10T11:29:39.285 INFO:tasks.workunit.client.1.smithi178.stdout:4/725: symlink d0/d9d/de1/lfb 0 2024-09-10T11:29:39.285 INFO:tasks.workunit.client.1.smithi178.stdout:7/762: creat d6/d10/d1d/d22/d50/d7e/d82/db2/d49/f110 x:0 0 0 2024-09-10T11:29:39.289 INFO:tasks.workunit.client.0.smithi144.stdout:7/480: rename d5/dc/d12/d2f/c62 to d5/dc/d14/d40/c9b 0 2024-09-10T11:29:39.289 INFO:tasks.workunit.client.1.smithi178.stdout:7/763: fsync d6/d10/d1d/d22/fbb 0 2024-09-10T11:29:39.290 INFO:tasks.workunit.client.1.smithi178.stdout:4/726: write d0/f1c [952353,111686] 0 2024-09-10T11:29:39.290 INFO:tasks.workunit.client.1.smithi178.stdout:7/764: dread - d6/d10/d1d/d4a/d79/df1/fff zero size 2024-09-10T11:29:39.290 INFO:tasks.workunit.client.0.smithi144.stdout:8/351: getdents d0/d9/d23 0 2024-09-10T11:29:39.290 INFO:tasks.workunit.client.0.smithi144.stdout:7/481: dread - d5/dc/d11/d18/f9a zero size 2024-09-10T11:29:39.290 INFO:tasks.workunit.client.1.smithi178.stdout:7/765: chown d6/d10/d1e/d2d/f8a 21769 1 2024-09-10T11:29:39.292 INFO:tasks.workunit.client.1.smithi178.stdout:7/766: chown d6/d10/d1d/d4a/l104 123153970 1 2024-09-10T11:29:39.293 INFO:tasks.workunit.client.1.smithi178.stdout:7/767: stat d6/d10/d1d/dca/cf8 0 2024-09-10T11:29:39.297 INFO:tasks.workunit.client.1.smithi178.stdout:4/727: getdents d0/d8/da/d1a/d51/d5d/db6 0 2024-09-10T11:29:39.297 INFO:tasks.workunit.client.0.smithi144.stdout:7/482: unlink d5/dc/d12/d2f/c8b 0 2024-09-10T11:29:39.298 INFO:tasks.workunit.client.1.smithi178.stdout:1/758: write d3/d35/ddd/df4/f10a [768512,61272] 0 2024-09-10T11:29:39.298 INFO:tasks.workunit.client.0.smithi144.stdout:7/483: readlink d5/dc/d14/d40/d58/d6e/l96 0 2024-09-10T11:29:39.299 INFO:tasks.workunit.client.0.smithi144.stdout:8/352: link d0/lb d0/d9/d1f/l6c 0 2024-09-10T11:29:39.300 INFO:tasks.workunit.client.1.smithi178.stdout:4/728: readlink d0/d9/dc/d8e/dbf/l24 0 2024-09-10T11:29:39.301 INFO:tasks.workunit.client.1.smithi178.stdout:7/768: dwrite f3 [0,4194304] 0 2024-09-10T11:29:39.301 INFO:tasks.workunit.client.1.smithi178.stdout:1/759: mkdir d3/d9/d21/d4b/d11a 0 2024-09-10T11:29:39.302 INFO:tasks.workunit.client.0.smithi144.stdout:7/484: unlink d5/dc/d11/f87 0 2024-09-10T11:29:39.304 INFO:tasks.workunit.client.1.smithi178.stdout:7/769: write d6/f30 [385960,95381] 0 2024-09-10T11:29:39.304 INFO:tasks.workunit.client.0.smithi144.stdout:8/353: symlink d0/d9/d21/l6d 0 2024-09-10T11:29:39.309 INFO:tasks.workunit.client.0.smithi144.stdout:7/485: symlink d5/dc/d14/d40/d5e/d63/d98/l9c 0 2024-09-10T11:29:39.309 INFO:tasks.workunit.client.1.smithi178.stdout:7/770: creat d6/d10/d1d/de6/f111 x:0 0 0 2024-09-10T11:29:39.310 INFO:tasks.workunit.client.0.smithi144.stdout:8/354: getdents d0/d9/d23/d37 0 2024-09-10T11:29:39.313 INFO:tasks.workunit.client.0.smithi144.stdout:8/355: symlink d0/d9/d21/d25/d4f/l6e 0 2024-09-10T11:29:39.313 INFO:tasks.workunit.client.1.smithi178.stdout:1/760: dwrite d3/d35/ddd/df4/f10a [0,4194304] 0 2024-09-10T11:29:39.315 INFO:tasks.workunit.client.1.smithi178.stdout:7/771: creat d6/d10/d1d/d22/d50/d7e/da7/f112 x:0 0 0 2024-09-10T11:29:39.320 INFO:tasks.workunit.client.1.smithi178.stdout:7/772: symlink d6/d10/d1d/d22/d93/db7/l113 0 2024-09-10T11:29:39.320 INFO:tasks.workunit.client.1.smithi178.stdout:4/729: dwrite d0/d63/f76 [0,4194304] 0 2024-09-10T11:29:39.324 INFO:tasks.workunit.client.0.smithi144.stdout:8/356: truncate d0/d1b/d5a/f6a 1450457 0 2024-09-10T11:29:39.325 INFO:tasks.workunit.client.1.smithi178.stdout:1/761: dwrite d3/d9/d21/f67 [0,4194304] 0 2024-09-10T11:29:39.326 INFO:tasks.workunit.client.1.smithi178.stdout:7/773: unlink d6/d10/d1e/d2d/f8a 0 2024-09-10T11:29:39.326 INFO:tasks.workunit.client.0.smithi144.stdout:7/486: dread d5/dc/d2a/d41/f49 [0,4194304] 0 2024-09-10T11:29:39.329 INFO:tasks.workunit.client.1.smithi178.stdout:1/762: truncate d3/d9/d98/dac/d3a/d50/db5/dcc/fd0 534087 0 2024-09-10T11:29:39.330 INFO:tasks.workunit.client.0.smithi144.stdout:7/487: mkdir d5/dc/d2a/d9d 0 2024-09-10T11:29:39.333 INFO:tasks.workunit.client.0.smithi144.stdout:8/357: dread d0/d9/d1f/d34/f53 [0,4194304] 0 2024-09-10T11:29:39.335 INFO:tasks.workunit.client.1.smithi178.stdout:4/730: creat d0/d9/dc/d8e/ffc x:0 0 0 2024-09-10T11:29:39.335 INFO:tasks.workunit.client.1.smithi178.stdout:7/774: creat d6/d10/d1d/d22/d93/db7/f114 x:0 0 0 2024-09-10T11:29:39.337 INFO:tasks.workunit.client.1.smithi178.stdout:7/775: fdatasync d6/d10/d1d/d22/d3b/f65 0 2024-09-10T11:29:39.337 INFO:tasks.workunit.client.1.smithi178.stdout:1/763: creat d3/d9/d98/dac/d3a/d50/db5/f11b x:0 0 0 2024-09-10T11:29:39.341 INFO:tasks.workunit.client.0.smithi144.stdout:7/488: link d5/dc/d14/d40/d5e/d63/d98/l9c d5/dc/d14/d40/l9e 0 2024-09-10T11:29:39.341 INFO:tasks.workunit.client.0.smithi144.stdout:8/358: dwrite d0/fc [0,4194304] 0 2024-09-10T11:29:39.341 INFO:tasks.workunit.client.0.smithi144.stdout:8/359: stat d0/d9/d23/d26/f4e 0 2024-09-10T11:29:39.341 INFO:tasks.workunit.client.0.smithi144.stdout:7/489: write d5/dc/d14/d40/d5e/d68/f8f [2044129,128983] 0 2024-09-10T11:29:39.343 INFO:tasks.workunit.client.1.smithi178.stdout:1/764: rmdir d3/d9/d1a/d48 39 2024-09-10T11:29:39.346 INFO:tasks.workunit.client.1.smithi178.stdout:4/731: fsync d0/d9/d3d/f44 0 2024-09-10T11:29:39.347 INFO:tasks.workunit.client.1.smithi178.stdout:4/732: fsync d0/d9/fc1 0 2024-09-10T11:29:39.349 INFO:tasks.workunit.client.0.smithi144.stdout:7/490: creat d5/dc/d14/d40/f9f x:0 0 0 2024-09-10T11:29:39.350 INFO:tasks.workunit.client.1.smithi178.stdout:1/765: getdents d3/d35/ddd 0 2024-09-10T11:29:39.353 INFO:tasks.workunit.client.1.smithi178.stdout:4/733: mkdir d0/d8/df/dfd 0 2024-09-10T11:29:39.353 INFO:tasks.workunit.client.0.smithi144.stdout:7/491: dread d5/dc/d14/d40/d5e/d63/f77 [0,4194304] 0 2024-09-10T11:29:39.354 INFO:tasks.workunit.client.1.smithi178.stdout:4/734: rmdir d0/d63 39 2024-09-10T11:29:39.356 INFO:tasks.workunit.client.0.smithi144.stdout:7/492: write d5/dc/d12/d2f/f4a [3645144,50265] 0 2024-09-10T11:29:39.357 INFO:tasks.workunit.client.1.smithi178.stdout:7/776: dread d6/f30 [0,4194304] 0 2024-09-10T11:29:39.359 INFO:tasks.workunit.client.0.smithi144.stdout:7/493: creat d5/dc/d14/d40/d58/d6a/fa0 x:0 0 0 2024-09-10T11:29:39.359 INFO:tasks.workunit.client.1.smithi178.stdout:4/735: rename d0/d9/dc/d8e/dbf/l24 to d0/d9/dc/d8e/dbf/lfe 0 2024-09-10T11:29:39.360 INFO:tasks.workunit.client.0.smithi144.stdout:7/494: chown d5/dc/d12/c1c 3 1 2024-09-10T11:29:39.361 INFO:tasks.workunit.client.1.smithi178.stdout:1/766: dread d3/d9/d21/d4b/d2b/f88 [0,4194304] 0 2024-09-10T11:29:39.363 INFO:tasks.workunit.client.1.smithi178.stdout:1/767: mkdir d3/d9/d98/d11c 0 2024-09-10T11:29:39.364 INFO:tasks.workunit.client.0.smithi144.stdout:7/495: rename d5/dc/c7e to d5/dc/d14/d40/d58/d6a/ca1 0 2024-09-10T11:29:39.364 INFO:tasks.workunit.client.1.smithi178.stdout:4/736: write d0/d8/da/f96 [375535,86385] 0 2024-09-10T11:29:39.365 INFO:tasks.workunit.client.1.smithi178.stdout:4/737: readlink d0/d8/da/d1a/ld4 0 2024-09-10T11:29:39.365 INFO:tasks.workunit.client.1.smithi178.stdout:1/768: dread - d3/d9/d21/d4b/d2b/dfd/f116 zero size 2024-09-10T11:29:39.366 INFO:tasks.workunit.client.0.smithi144.stdout:7/496: creat d5/dc/d14/d40/d58/d6a/fa2 x:0 0 0 2024-09-10T11:29:39.366 INFO:tasks.workunit.client.0.smithi144.stdout:7/497: fsync d5/dc/d2a/f55 0 2024-09-10T11:29:39.367 INFO:tasks.workunit.client.1.smithi178.stdout:4/738: mkdir d0/d8/da/d1a/dff 0 2024-09-10T11:29:39.368 INFO:tasks.workunit.client.0.smithi144.stdout:7/498: creat d5/dc/d14/d40/d58/fa3 x:0 0 0 2024-09-10T11:29:39.369 INFO:tasks.workunit.client.1.smithi178.stdout:1/769: symlink d3/d9/d98/dac/d3a/d50/d9e/dc2/de2/d118/db8/l11d 0 2024-09-10T11:29:39.370 INFO:tasks.workunit.client.1.smithi178.stdout:4/739: mknod d0/d9/d3d/d4f/ddf/c100 0 2024-09-10T11:29:39.370 INFO:tasks.workunit.client.1.smithi178.stdout:1/770: symlink d3/d9/d98/dd1/d7e/l11e 0 2024-09-10T11:29:39.373 INFO:tasks.workunit.client.1.smithi178.stdout:1/771: mkdir d3/d9/d98/dac/d3a/d50/d11f 0 2024-09-10T11:29:39.376 INFO:tasks.workunit.client.1.smithi178.stdout:1/772: rename d3/d9/d21/d6f to d3/d9/d98/dac/d3a/d9f/d120 0 2024-09-10T11:29:39.380 INFO:tasks.workunit.client.1.smithi178.stdout:1/773: getdents d3/d9/d98/dac/d3a/d6e/d8a 0 2024-09-10T11:29:39.381 INFO:tasks.workunit.client.1.smithi178.stdout:4/740: dread d0/d8/da/d1a/f69 [0,4194304] 0 2024-09-10T11:29:39.381 INFO:tasks.workunit.client.0.smithi144.stdout:7/499: dread d5/dc/d12/f21 [0,4194304] 0 2024-09-10T11:29:39.382 INFO:tasks.workunit.client.1.smithi178.stdout:4/741: chown d0/d8/da/d1a/d51/d5d/d62 62 1 2024-09-10T11:29:39.382 INFO:tasks.workunit.client.0.smithi144.stdout:7/500: dread - d5/dc/d12/f8e zero size 2024-09-10T11:29:39.382 INFO:tasks.workunit.client.1.smithi178.stdout:4/742: write d0/d9/f13 [1308663,63093] 0 2024-09-10T11:29:39.383 INFO:tasks.workunit.client.0.smithi144.stdout:7/501: write d5/dc/d2a/d60/f91 [4993293,16895] 0 2024-09-10T11:29:39.384 INFO:tasks.workunit.client.1.smithi178.stdout:4/743: chown d0/d9/dc/c14 252 1 2024-09-10T11:29:39.385 INFO:tasks.workunit.client.1.smithi178.stdout:4/744: rmdir d0/d9/dc/d8a 39 2024-09-10T11:29:39.387 INFO:tasks.workunit.client.0.smithi144.stdout:7/502: rename d5/dc/d14/l33 to d5/dc/d14/d40/d58/d6e/la4 0 2024-09-10T11:29:39.387 INFO:tasks.workunit.client.0.smithi144.stdout:7/503: fdatasync d5/dc/d14/d40/d58/d6a/fa0 0 2024-09-10T11:29:39.389 INFO:tasks.workunit.client.1.smithi178.stdout:4/745: creat d0/d8/da/f101 x:0 0 0 2024-09-10T11:29:39.392 INFO:tasks.workunit.client.0.smithi144.stdout:7/504: link d5/dc/d12/c1c d5/dc/d14/ca5 0 2024-09-10T11:29:39.392 INFO:tasks.workunit.client.1.smithi178.stdout:1/774: rename d3/d9/d98/dac/d3a/d9f to d3/d9/d98/dac/d3a/d50/d6c/de9/d121 0 2024-09-10T11:29:39.393 INFO:tasks.workunit.client.0.smithi144.stdout:7/505: rmdir d5/dc/d14/d40/d58/d6e/d7a 39 2024-09-10T11:29:39.394 INFO:tasks.workunit.client.1.smithi178.stdout:4/746: mknod d0/d63/de0/c102 0 2024-09-10T11:29:39.394 INFO:tasks.workunit.client.1.smithi178.stdout:1/775: mkdir d3/d9/d98/dac/d3a/d50/d6c/de9/d122 0 2024-09-10T11:29:39.395 INFO:tasks.workunit.client.1.smithi178.stdout:4/747: fdatasync d0/d8/da/d1a/f4b 0 2024-09-10T11:29:39.396 INFO:tasks.workunit.client.0.smithi144.stdout:7/506: mknod d5/dc/ca6 0 2024-09-10T11:29:39.397 INFO:tasks.workunit.client.1.smithi178.stdout:1/776: mkdir d3/d9/d98/dac/d3a/d50/d9e/dc2/d123 0 2024-09-10T11:29:39.398 INFO:tasks.workunit.client.1.smithi178.stdout:4/748: creat d0/d8/f103 x:0 0 0 2024-09-10T11:29:39.399 INFO:tasks.workunit.client.1.smithi178.stdout:1/777: getdents d3/d9/d98/dac/d3a/d50/db5/de3 0 2024-09-10T11:29:39.400 INFO:tasks.workunit.client.1.smithi178.stdout:4/749: mkdir d0/d8/da/d1a/d51/d5d/dc9/dee/d104 0 2024-09-10T11:29:39.402 INFO:tasks.workunit.client.0.smithi144.stdout:7/507: dwrite d5/dc/d14/f44 [0,4194304] 0 2024-09-10T11:29:39.406 INFO:tasks.workunit.client.1.smithi178.stdout:4/750: dwrite d0/d8/da/fd3 [0,4194304] 0 2024-09-10T11:29:39.407 INFO:tasks.workunit.client.0.smithi144.stdout:7/508: dread d5/dc/d2a/d60/f91 [0,4194304] 0 2024-09-10T11:29:39.411 INFO:tasks.workunit.client.1.smithi178.stdout:4/751: rename d0/d9/f81 to d0/d9/d3d/f105 0 2024-09-10T11:29:39.411 INFO:tasks.workunit.client.0.smithi144.stdout:7/509: symlink d5/dc/d2a/d9d/la7 0 2024-09-10T11:29:39.412 INFO:tasks.workunit.client.1.smithi178.stdout:1/778: write d3/d9/d21/d4b/f5a [2982206,129838] 0 2024-09-10T11:29:39.414 INFO:tasks.workunit.client.1.smithi178.stdout:1/779: fsync d3/d9/d98/dac/d3a/d50/d80/f10f 0 2024-09-10T11:29:39.414 INFO:tasks.workunit.client.1.smithi178.stdout:4/752: mkdir d0/d9/d3d/d4f/ddb/d106 0 2024-09-10T11:29:39.415 INFO:tasks.workunit.client.0.smithi144.stdout:7/510: creat d5/dc/d2a/fa8 x:0 0 0 2024-09-10T11:29:39.417 INFO:tasks.workunit.client.0.smithi144.stdout:7/511: fsync d5/dc/d14/d40/f67 0 2024-09-10T11:29:39.417 INFO:tasks.workunit.client.1.smithi178.stdout:1/780: creat d3/f124 x:0 0 0 2024-09-10T11:29:39.419 INFO:tasks.workunit.client.0.smithi144.stdout:7/512: chown d5/dc/d2a/f4b 0 1 2024-09-10T11:29:39.419 INFO:tasks.workunit.client.1.smithi178.stdout:4/753: creat d0/d8/f107 x:0 0 0 2024-09-10T11:29:39.420 INFO:tasks.workunit.client.1.smithi178.stdout:4/754: chown d0/d9/dc/d8e/dbf/l79 7657 1 2024-09-10T11:29:39.420 INFO:tasks.workunit.client.1.smithi178.stdout:4/755: chown d0/d8 23 1 2024-09-10T11:29:39.420 INFO:tasks.workunit.client.0.smithi144.stdout:7/513: truncate d5/dc/d12/f13 4377430 0 2024-09-10T11:29:39.421 INFO:tasks.workunit.client.1.smithi178.stdout:1/781: mknod d3/d9/d98/dac/d3a/d50/d9e/dc2/de2/d118/c125 0 2024-09-10T11:29:39.421 INFO:tasks.workunit.client.1.smithi178.stdout:4/756: fsync d0/d9/f13 0 2024-09-10T11:29:39.421 INFO:tasks.workunit.client.0.smithi144.stdout:7/514: read - d5/dc/d14/d40/f9f zero size 2024-09-10T11:29:39.421 INFO:tasks.workunit.client.1.smithi178.stdout:4/757: fdatasync d0/d8/da/fe7 0 2024-09-10T11:29:39.422 INFO:tasks.workunit.client.1.smithi178.stdout:4/758: chown d0/d8/df/ld6 19 1 2024-09-10T11:29:39.423 INFO:tasks.workunit.client.1.smithi178.stdout:1/782: mknod d3/d9/d98/dd1/d7e/c126 0 2024-09-10T11:29:39.423 INFO:tasks.workunit.client.0.smithi144.stdout:7/515: symlink d5/dc/la9 0 2024-09-10T11:29:39.424 INFO:tasks.workunit.client.0.smithi144.stdout:7/516: dread - d5/dc/d14/d40/f72 zero size 2024-09-10T11:29:39.424 INFO:tasks.workunit.client.1.smithi178.stdout:4/759: creat d0/d9/d3d/d4f/ddb/f108 x:0 0 0 2024-09-10T11:29:39.425 INFO:tasks.workunit.client.1.smithi178.stdout:1/783: creat d3/d9/d98/dac/d8d/f127 x:0 0 0 2024-09-10T11:29:39.425 INFO:tasks.workunit.client.0.smithi144.stdout:7/517: write d5/dc/d11/d18/f9a [354644,82448] 0 2024-09-10T11:29:39.426 INFO:tasks.workunit.client.0.smithi144.stdout:7/518: fsync d5/f7 0 2024-09-10T11:29:39.428 INFO:tasks.workunit.client.0.smithi144.stdout:7/519: rmdir d5/dc/d11/d18 39 2024-09-10T11:29:39.429 INFO:tasks.workunit.client.0.smithi144.stdout:7/520: read - d5/dc/d14/d40/d58/d6a/fa2 zero size 2024-09-10T11:29:39.432 INFO:tasks.workunit.client.0.smithi144.stdout:7/521: getdents d5/dc/d14/d40/d58/d6a 0 2024-09-10T11:29:39.434 INFO:tasks.workunit.client.0.smithi144.stdout:7/522: read d5/dc/d11/f17 [261347,18579] 0 2024-09-10T11:29:39.439 INFO:tasks.workunit.client.1.smithi178.stdout:1/784: dwrite d3/d9/d1a/d48/f66 [0,4194304] 0 2024-09-10T11:29:39.442 INFO:tasks.workunit.client.1.smithi178.stdout:1/785: mknod d3/d9/d21/d4b/d2b/dfd/c128 0 2024-09-10T11:29:39.442 INFO:tasks.workunit.client.1.smithi178.stdout:1/786: stat d3/d9/d21/fb7 0 2024-09-10T11:29:39.444 INFO:tasks.workunit.client.1.smithi178.stdout:1/787: write d3/d9/d98/dac/f10c [528984,92729] 0 2024-09-10T11:29:39.452 INFO:tasks.workunit.client.1.smithi178.stdout:4/760: dread d0/d8/df/f20 [0,4194304] 0 2024-09-10T11:29:39.454 INFO:tasks.workunit.client.1.smithi178.stdout:4/761: creat d0/f109 x:0 0 0 2024-09-10T11:29:39.454 INFO:tasks.workunit.client.1.smithi178.stdout:4/762: dread - d0/d8/da/d1a/ff7 zero size 2024-09-10T11:29:39.465 INFO:tasks.workunit.client.1.smithi178.stdout:9/882: sync 2024-09-10T11:29:39.468 INFO:tasks.workunit.client.1.smithi178.stdout:1/788: dread d3/d9/d98/dac/d3a/f4e [0,4194304] 0 2024-09-10T11:29:39.469 INFO:tasks.workunit.client.1.smithi178.stdout:1/789: write d3/d9/d98/dac/d3a/d50/db5/f11b [834332,123976] 0 2024-09-10T11:29:39.471 INFO:tasks.workunit.client.1.smithi178.stdout:1/790: write d3/d9/d98/dac/d3a/d50/d6c/de9/d121/d120/f75 [1111589,96036] 0 2024-09-10T11:29:39.541 INFO:tasks.workunit.client.1.smithi178.stdout:9/883: creat d2/d19/d5e/f116 x:0 0 0 2024-09-10T11:29:39.542 INFO:tasks.workunit.client.1.smithi178.stdout:1/791: mknod d3/d9/d98/c129 0 2024-09-10T11:29:39.547 INFO:tasks.workunit.client.1.smithi178.stdout:1/792: creat d3/d9/d21/d4b/dfa/f12a x:0 0 0 2024-09-10T11:29:39.547 INFO:tasks.workunit.client.1.smithi178.stdout:4/763: truncate d0/d8/da/f6b 2060816 0 2024-09-10T11:29:39.548 INFO:tasks.workunit.client.1.smithi178.stdout:1/793: write d3/d9/fc4 [3942983,95716] 0 2024-09-10T11:29:39.548 INFO:tasks.workunit.client.1.smithi178.stdout:5/989: sync 2024-09-10T11:29:39.548 INFO:tasks.workunit.client.1.smithi178.stdout:6/846: sync 2024-09-10T11:29:39.550 INFO:tasks.workunit.client.1.smithi178.stdout:1/794: dread - d3/d9/d98/dac/d3a/d6e/d8a/f7a zero size 2024-09-10T11:29:39.551 INFO:tasks.workunit.client.1.smithi178.stdout:9/884: write d2/d13/d40/d89/d93/f72 [535506,92360] 0 2024-09-10T11:29:39.552 INFO:tasks.workunit.client.1.smithi178.stdout:6/847: creat d3/d7/d8/d15/d1c/d2a/f136 x:0 0 0 2024-09-10T11:29:39.556 INFO:tasks.workunit.client.1.smithi178.stdout:6/848: mknod d3/d4e/c137 0 2024-09-10T11:29:39.556 INFO:tasks.workunit.client.1.smithi178.stdout:9/885: dread - d2/d13/d40/d56/dbe/f104 zero size 2024-09-10T11:29:39.557 INFO:tasks.workunit.client.1.smithi178.stdout:1/795: creat d3/d9/f12b x:0 0 0 2024-09-10T11:29:39.557 INFO:tasks.workunit.client.1.smithi178.stdout:4/764: dwrite d0/d10/ff9 [0,4194304] 0 2024-09-10T11:29:39.562 INFO:tasks.workunit.client.1.smithi178.stdout:5/990: dwrite d9/dc/dd/d3f/fe9 [0,4194304] 0 2024-09-10T11:29:39.565 INFO:tasks.workunit.client.1.smithi178.stdout:1/796: fsync d3/d35/f74 0 2024-09-10T11:29:39.565 INFO:tasks.workunit.client.1.smithi178.stdout:5/991: dread - d9/dc/dd/d37/d42/d9f/f12c zero size 2024-09-10T11:29:39.566 INFO:tasks.workunit.client.1.smithi178.stdout:5/992: readlink d9/lb5 0 2024-09-10T11:29:39.568 INFO:tasks.workunit.client.1.smithi178.stdout:4/765: read d0/d9/d3d/d4f/f52 [1752974,109622] 0 2024-09-10T11:29:39.568 INFO:tasks.workunit.client.1.smithi178.stdout:1/797: symlink d3/l12c 0 2024-09-10T11:29:39.615 INFO:tasks.workunit.client.1.smithi178.stdout:6/849: dwrite d3/d74/d91/daa/dc2/f96 [0,4194304] 0 2024-09-10T11:29:39.615 INFO:tasks.workunit.client.1.smithi178.stdout:9/886: truncate d2/d9b/df8/d2d/f95 431795 0 2024-09-10T11:29:39.615 INFO:tasks.workunit.client.1.smithi178.stdout:5/993: write d9/dc/dd/d37/dbd/dda/db8/fc5 [378462,23439] 0 2024-09-10T11:29:39.616 INFO:tasks.workunit.client.1.smithi178.stdout:6/850: chown d3/d74 38563 1 2024-09-10T11:29:39.616 INFO:tasks.workunit.client.1.smithi178.stdout:1/798: chown d3/d9/d1a 0 1 2024-09-10T11:29:39.619 INFO:tasks.workunit.client.1.smithi178.stdout:9/887: mkdir d2/d13/d40/d89/d93/d88/dae/d117 0 2024-09-10T11:29:39.619 INFO:tasks.workunit.client.1.smithi178.stdout:6/851: chown d3/d7/d8/d15/d1c/d2a/d18/d8e/d90/fe5 122499258 1 2024-09-10T11:29:39.624 INFO:tasks.workunit.client.1.smithi178.stdout:4/766: link d0/l4 d0/d9/dc/d8a/l10a 0 2024-09-10T11:29:39.625 INFO:tasks.workunit.client.1.smithi178.stdout:5/994: dwrite d9/dc/dd/f140 [0,4194304] 0 2024-09-10T11:29:39.646 INFO:tasks.workunit.client.1.smithi178.stdout:1/799: dread d3/d9/d98/dac/d2a/f92 [4194304,4194304] 0 2024-09-10T11:29:39.660 INFO:tasks.workunit.client.0.smithi144.stdout:0/315: sync 2024-09-10T11:29:39.660 INFO:tasks.workunit.client.0.smithi144.stdout:9/487: sync 2024-09-10T11:29:39.661 INFO:tasks.workunit.client.0.smithi144.stdout:0/316: write da/f32 [3798887,8660] 0 2024-09-10T11:29:39.662 INFO:tasks.workunit.client.1.smithi178.stdout:6/852: creat d3/d74/d91/daa/dc2/dcf/df8/f138 x:0 0 0 2024-09-10T11:29:39.663 INFO:tasks.workunit.client.0.smithi144.stdout:9/488: rmdir d11/d12/d1b/d41/d8f 39 2024-09-10T11:29:39.663 INFO:tasks.workunit.client.1.smithi178.stdout:6/853: chown d3/d7/c3e 1780819 1 2024-09-10T11:29:39.664 INFO:tasks.workunit.client.0.smithi144.stdout:0/317: readlink da/l1b 0 2024-09-10T11:29:39.665 INFO:tasks.workunit.client.1.smithi178.stdout:4/767: rename d0/d9/dc/d8e/dbf/f37 to d0/d9/dc/dca/f10b 0 2024-09-10T11:29:39.667 INFO:tasks.workunit.client.1.smithi178.stdout:9/888: write d2/fef [1567827,18588] 0 2024-09-10T11:29:39.668 INFO:tasks.workunit.client.1.smithi178.stdout:9/889: readlink d2/d9b/df8/lff 0 2024-09-10T11:29:39.669 INFO:tasks.workunit.client.0.smithi144.stdout:9/489: getdents d11 0 2024-09-10T11:29:39.670 INFO:tasks.workunit.client.1.smithi178.stdout:9/890: write d2/d13/d75/d60/dd0/fde [166322,14859] 0 2024-09-10T11:29:39.671 INFO:tasks.workunit.client.0.smithi144.stdout:0/318: dwrite da/f46 [0,4194304] 0 2024-09-10T11:29:39.673 INFO:tasks.workunit.client.1.smithi178.stdout:9/891: truncate d2/df/d31/d37/d3d/fc3 992137 0 2024-09-10T11:29:39.677 INFO:tasks.workunit.client.0.smithi144.stdout:9/490: dwrite d11/d36/fa4 [0,4194304] 0 2024-09-10T11:29:39.685 INFO:tasks.workunit.client.0.smithi144.stdout:9/491: read d11/d12/d1b/d41/d8f/f82 [3792746,90251] 0 2024-09-10T11:29:39.686 INFO:tasks.workunit.client.0.smithi144.stdout:0/319: dwrite da/f26 [0,4194304] 0 2024-09-10T11:29:39.688 INFO:tasks.workunit.client.1.smithi178.stdout:2/989: sync 2024-09-10T11:29:39.693 INFO:tasks.workunit.client.0.smithi144.stdout:9/492: symlink d11/d12/d1b/d22/laf 0 2024-09-10T11:29:39.694 INFO:tasks.workunit.client.1.smithi178.stdout:2/990: dread d1/d7/d24/d46/d88/d9d/dab/d5b/f10e [0,4194304] 0 2024-09-10T11:29:39.696 INFO:tasks.workunit.client.0.smithi144.stdout:9/493: read d11/d12/d1b/f20 [374076,4661] 0 2024-09-10T11:29:39.699 INFO:tasks.workunit.client.0.smithi144.stdout:9/494: chown d11/d12/d1b/d22/l83 15 1 2024-09-10T11:29:39.701 INFO:tasks.workunit.client.1.smithi178.stdout:5/995: symlink d9/dc/dd/d37/d42/d62/daf/d138/l146 0 2024-09-10T11:29:39.702 INFO:tasks.workunit.client.1.smithi178.stdout:6/854: write d3/d7/d8/d56/de1/ff2 [855973,48866] 0 2024-09-10T11:29:39.703 INFO:tasks.workunit.client.1.smithi178.stdout:5/996: write d9/dc/f117 [448358,56773] 0 2024-09-10T11:29:39.704 INFO:tasks.workunit.client.0.smithi144.stdout:0/320: dread da/f32 [0,4194304] 0 2024-09-10T11:29:39.704 INFO:tasks.workunit.client.0.smithi144.stdout:0/321: stat da 0 2024-09-10T11:29:39.705 INFO:tasks.workunit.client.0.smithi144.stdout:9/495: link d11/c5a d11/d12/d1b/d41/d8f/dab/cb0 0 2024-09-10T11:29:39.705 INFO:tasks.workunit.client.1.smithi178.stdout:1/800: creat d3/d9/d1a/f12d x:0 0 0 2024-09-10T11:29:39.706 INFO:tasks.workunit.client.0.smithi144.stdout:9/496: write d11/d36/fa4 [3038860,67036] 0 2024-09-10T11:29:39.707 INFO:tasks.workunit.client.1.smithi178.stdout:1/801: write d3/d9/d1a/fdb [9426187,108298] 0 2024-09-10T11:29:39.707 INFO:tasks.workunit.client.1.smithi178.stdout:9/892: unlink d2/df/c5c 0 2024-09-10T11:29:39.708 INFO:tasks.workunit.client.1.smithi178.stdout:9/893: stat d2/d8/c36 0 2024-09-10T11:29:39.713 INFO:tasks.workunit.client.0.smithi144.stdout:0/322: dwrite da/d3a/f4d [0,4194304] 0 2024-09-10T11:29:39.716 INFO:tasks.workunit.client.0.smithi144.stdout:0/323: stat da/d49 0 2024-09-10T11:29:39.718 INFO:tasks.workunit.client.0.smithi144.stdout:9/497: dwrite d11/d36/d2d/d38/f76 [4194304,4194304] 0 2024-09-10T11:29:39.719 INFO:tasks.workunit.client.1.smithi178.stdout:9/894: rename d2/d13/d40 to d2/d9b/df8/d2d/dab/d118 0 2024-09-10T11:29:39.719 INFO:tasks.workunit.client.0.smithi144.stdout:0/324: symlink da/d3a/d4f/l5c 0 2024-09-10T11:29:39.719 INFO:tasks.workunit.client.0.smithi144.stdout:0/325: stat da/d3a/c52 0 2024-09-10T11:29:39.721 INFO:tasks.workunit.client.0.smithi144.stdout:9/498: chown d11/d12/d1b/d2c/c9f 325372261 1 2024-09-10T11:29:39.723 INFO:tasks.workunit.client.0.smithi144.stdout:0/326: creat da/d3a/d4b/d4e/f5d x:0 0 0 2024-09-10T11:29:39.723 INFO:tasks.workunit.client.0.smithi144.stdout:9/499: mkdir d11/d36/db1 0 2024-09-10T11:29:39.724 INFO:tasks.workunit.client.1.smithi178.stdout:9/895: mknod d2/d9b/df8/d2d/dab/d118/d89/d93/dc2/c119 0 2024-09-10T11:29:39.724 INFO:tasks.workunit.client.0.smithi144.stdout:9/500: dread - d11/d12/d1b/d2c/d78/f7d zero size 2024-09-10T11:29:39.726 INFO:tasks.workunit.client.0.smithi144.stdout:9/501: truncate d11/d12/d1b/d22/f63 4976205 0 2024-09-10T11:29:39.727 INFO:tasks.workunit.client.1.smithi178.stdout:5/997: link d9/dc/dd/d37/d3e/f94 d9/dc/dd/d3f/dd6/f147 0 2024-09-10T11:29:39.729 INFO:tasks.workunit.client.0.smithi144.stdout:9/502: mknod d11/d12/d1b/d41/d8f/dab/d64/cb2 0 2024-09-10T11:29:39.729 INFO:tasks.workunit.client.1.smithi178.stdout:6/855: link d3/d7/d8/d19/f39 d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d11b/d121/f139 0 2024-09-10T11:29:39.733 INFO:tasks.workunit.client.1.smithi178.stdout:6/856: mknod d3/d7/d8/d19/c13a 0 2024-09-10T11:29:39.735 INFO:tasks.workunit.client.1.smithi178.stdout:5/998: truncate d9/dc/dd/d37/d42/d62/daf/df0/fa2 129441 0 2024-09-10T11:29:39.735 INFO:tasks.workunit.client.1.smithi178.stdout:1/802: dread d3/d9/d21/fb7 [0,4194304] 0 2024-09-10T11:29:39.736 INFO:tasks.workunit.client.1.smithi178.stdout:1/803: read - d3/d9/d98/dd1/f110 zero size 2024-09-10T11:29:39.737 INFO:tasks.workunit.client.1.smithi178.stdout:5/999: write d9/dc/dd/d16/f70 [2518100,63456] 0 2024-09-10T11:29:39.739 INFO:tasks.workunit.client.1.smithi178.stdout:2/991: write d1/d41/f44 [2175008,126425] 0 2024-09-10T11:29:39.742 INFO:tasks.workunit.client.1.smithi178.stdout:2/992: mkdir d1/d7/d24/d46/d88/d9d/dab/d5b/d14d 0 2024-09-10T11:29:39.744 INFO:tasks.workunit.client.1.smithi178.stdout:9/896: write d2/df/d31/d37/d3d/f83 [934434,4781] 0 2024-09-10T11:29:39.753 INFO:tasks.workunit.client.1.smithi178.stdout:6/857: dread d3/d24/fc7 [0,4194304] 0 2024-09-10T11:29:39.774 INFO:tasks.workunit.client.0.smithi144.stdout:1/616: sync 2024-09-10T11:29:39.775 INFO:tasks.workunit.client.0.smithi144.stdout:2/410: sync 2024-09-10T11:29:39.775 INFO:tasks.workunit.client.1.smithi178.stdout:9/897: chown d2/d9b/df8/d2d/dab/d118/d89 11259 1 2024-09-10T11:29:39.775 INFO:tasks.workunit.client.1.smithi178.stdout:6/858: chown d3/d74/dc9/l79 1239 1 2024-09-10T11:29:39.776 INFO:tasks.workunit.client.1.smithi178.stdout:2/993: link d1/d7/d24/d7d/d84/d55/da1/d130/fa7 d1/d7/d24/d46/d88/d9d/dab/d5b/d14d/f14e 0 2024-09-10T11:29:39.778 INFO:tasks.workunit.client.1.smithi178.stdout:6/859: mkdir d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d13b 0 2024-09-10T11:29:39.781 INFO:tasks.workunit.client.1.smithi178.stdout:9/898: rename d2/dc/f44 to d2/f11a 0 2024-09-10T11:29:39.783 INFO:tasks.workunit.client.1.smithi178.stdout:9/899: write d2/df/d31/d37/f94 [604907,112085] 0 2024-09-10T11:29:39.785 INFO:tasks.workunit.client.0.smithi144.stdout:0/327: truncate da/f32 3993391 0 2024-09-10T11:29:39.787 INFO:tasks.workunit.client.0.smithi144.stdout:0/328: rmdir da/d3a 39 2024-09-10T11:29:39.787 INFO:tasks.workunit.client.1.smithi178.stdout:2/994: rename d1/d7/d24/d7d/d84/d55/da1/dd6/f144 to d1/d7/d24/f14f 0 2024-09-10T11:29:39.790 INFO:tasks.workunit.client.0.smithi144.stdout:2/411: rename d7/d42/f71 to d7/de/d36/d45/d47/f88 0 2024-09-10T11:29:39.791 INFO:tasks.workunit.client.1.smithi178.stdout:6/860: rename d3/d7/d8/d56/fed to d3/d123/f13c 0 2024-09-10T11:29:39.794 INFO:tasks.workunit.client.0.smithi144.stdout:9/503: dwrite d11/d12/d1b/d41/d8f/f93 [0,4194304] 0 2024-09-10T11:29:39.796 INFO:tasks.workunit.client.0.smithi144.stdout:0/329: getdents da/d48 0 2024-09-10T11:29:39.797 INFO:tasks.workunit.client.1.smithi178.stdout:6/861: fsync d3/d7/d8/d15/f22 0 2024-09-10T11:29:39.799 INFO:tasks.workunit.client.0.smithi144.stdout:2/412: getdents d7/da/d1e/d21/d54 0 2024-09-10T11:29:39.803 INFO:tasks.workunit.client.1.smithi178.stdout:6/862: dread d3/d6e/ff1 [0,4194304] 0 2024-09-10T11:29:39.803 INFO:tasks.workunit.client.0.smithi144.stdout:9/504: dwrite d11/d12/f14 [0,4194304] 0 2024-09-10T11:29:39.805 INFO:tasks.workunit.client.1.smithi178.stdout:2/995: link d1/d22/d4d/d9b/l9e d1/d41/d14/d11/l150 0 2024-09-10T11:29:39.805 INFO:tasks.workunit.client.0.smithi144.stdout:0/330: link da/l43 da/d3a/d4b/d4e/l5e 0 2024-09-10T11:29:39.809 INFO:tasks.workunit.client.1.smithi178.stdout:9/900: dwrite d2/d8/f17 [0,4194304] 0 2024-09-10T11:29:39.810 INFO:tasks.workunit.client.0.smithi144.stdout:1/617: dread d5/d2a/f96 [0,4194304] 0 2024-09-10T11:29:39.811 INFO:tasks.workunit.client.0.smithi144.stdout:1/618: chown d5/d2a/d65/d73/fb1 266398311 1 2024-09-10T11:29:39.812 INFO:tasks.workunit.client.0.smithi144.stdout:0/331: dwrite f2 [0,4194304] 0 2024-09-10T11:29:39.813 INFO:tasks.workunit.client.0.smithi144.stdout:1/619: dread d5/dd/f48 [0,4194304] 0 2024-09-10T11:29:39.815 INFO:tasks.workunit.client.0.smithi144.stdout:0/332: rmdir da/d3a/d4b/d4e 39 2024-09-10T11:29:39.817 INFO:tasks.workunit.client.0.smithi144.stdout:9/505: truncate d11/d12/f26 1160993 0 2024-09-10T11:29:39.819 INFO:tasks.workunit.client.0.smithi144.stdout:9/506: write d11/d12/d1b/d41/d8f/dab/d64/f68 [22427,45326] 0 2024-09-10T11:29:39.821 INFO:tasks.workunit.client.0.smithi144.stdout:1/620: rename d5/d4a/d44/f69 to d5/dd/d8f/fce 0 2024-09-10T11:29:39.825 INFO:tasks.workunit.client.0.smithi144.stdout:0/333: rename da/d48 to da/d3a/d51/d5f 0 2024-09-10T11:29:39.825 INFO:tasks.workunit.client.0.smithi144.stdout:1/621: truncate d5/d4a/d63/d84/f9e 570132 0 2024-09-10T11:29:39.827 INFO:tasks.workunit.client.0.smithi144.stdout:9/507: dwrite d11/d36/f52 [0,4194304] 0 2024-09-10T11:29:39.830 INFO:tasks.workunit.client.0.smithi144.stdout:9/508: truncate d11/d12/d1b/d41/f92 901511 0 2024-09-10T11:29:39.834 INFO:tasks.workunit.client.0.smithi144.stdout:1/622: dwrite d5/d2a/d65/dc9/f6f [0,4194304] 0 2024-09-10T11:29:39.834 INFO:tasks.workunit.client.0.smithi144.stdout:9/509: symlink d11/d12/d6f/lb3 0 2024-09-10T11:29:39.837 INFO:tasks.workunit.client.0.smithi144.stdout:0/334: dread da/f41 [0,4194304] 0 2024-09-10T11:29:39.838 INFO:tasks.workunit.client.0.smithi144.stdout:1/623: mkdir d5/df/d4b/dcf 0 2024-09-10T11:29:39.839 INFO:tasks.workunit.client.0.smithi144.stdout:9/510: creat d11/d12/d1b/d41/d8f/fb4 x:0 0 0 2024-09-10T11:29:39.842 INFO:tasks.workunit.client.0.smithi144.stdout:1/624: mknod d5/cd0 0 2024-09-10T11:29:39.843 INFO:tasks.workunit.client.1.smithi178.stdout:9/901: symlink d2/d9b/df8/d2d/dab/d118/d56/l11b 0 2024-09-10T11:29:39.844 INFO:tasks.workunit.client.1.smithi178.stdout:2/996: symlink d1/d41/d5d/d14b/l151 0 2024-09-10T11:29:39.844 INFO:tasks.workunit.client.0.smithi144.stdout:0/335: creat da/d3a/d4b/d4e/f60 x:0 0 0 2024-09-10T11:29:39.846 INFO:tasks.workunit.client.1.smithi178.stdout:2/997: chown d1/d7/d24/d46/d88/d9d/dab/d5b/d60/dd7 120592 1 2024-09-10T11:29:39.848 INFO:tasks.workunit.client.1.smithi178.stdout:9/902: mknod d2/d9b/df8/d2d/dab/d118/d89/d93/dc2/c11c 0 2024-09-10T11:29:39.850 INFO:tasks.workunit.client.1.smithi178.stdout:2/998: mknod d1/d7/d24/d7d/d84/d4b/c152 0 2024-09-10T11:29:39.851 INFO:tasks.workunit.client.0.smithi144.stdout:9/511: fdatasync d11/d36/f52 0 2024-09-10T11:29:39.852 INFO:tasks.workunit.client.1.smithi178.stdout:9/903: symlink d2/d13/d75/d60/d66/l11d 0 2024-09-10T11:29:39.852 INFO:tasks.workunit.client.1.smithi178.stdout:2/999: read d1/d7/d24/d7d/d84/d55/da1/fff [76947,89998] 0 2024-09-10T11:29:39.855 INFO:tasks.workunit.client.1.smithi178.stdout:0/782: sync 2024-09-10T11:29:39.855 INFO:tasks.workunit.client.1.smithi178.stdout:9/904: truncate d2/d13/f43 2216862 0 2024-09-10T11:29:39.856 INFO:tasks.workunit.client.0.smithi144.stdout:0/336: write da/f36 [4805798,81251] 0 2024-09-10T11:29:39.859 INFO:tasks.workunit.client.1.smithi178.stdout:0/783: creat d0/dc/d39/da4/dce/ff5 x:0 0 0 2024-09-10T11:29:39.859 INFO:tasks.workunit.client.1.smithi178.stdout:6/863: dwrite d3/d7/d8/d19/f39 [0,4194304] 0 2024-09-10T11:29:39.860 INFO:tasks.workunit.client.1.smithi178.stdout:9/905: mknod d2/d13/d75/d60/d66/c11e 0 2024-09-10T11:29:39.861 INFO:tasks.workunit.client.1.smithi178.stdout:9/906: chown d2/df/d31/d37 202 1 2024-09-10T11:29:39.861 INFO:tasks.workunit.client.1.smithi178.stdout:6/864: stat d3/d7/d8/d15/d1c/d2a/d18/d8e/d90 0 2024-09-10T11:29:39.863 INFO:tasks.workunit.client.0.smithi144.stdout:9/512: dwrite d11/d12/d1b/d22/f4a [0,4194304] 0 2024-09-10T11:29:39.864 INFO:tasks.workunit.client.0.smithi144.stdout:1/625: dwrite d5/dd/d64/f9a [0,4194304] 0 2024-09-10T11:29:39.865 INFO:tasks.workunit.client.1.smithi178.stdout:6/865: readlink d3/d123/l65 0 2024-09-10T11:29:39.869 INFO:tasks.workunit.client.1.smithi178.stdout:0/784: dwrite d0/dc/d39/d50/f71 [0,4194304] 0 2024-09-10T11:29:39.869 INFO:tasks.workunit.client.1.smithi178.stdout:0/785: readlink d0/d2c/l54 0 2024-09-10T11:29:39.869 INFO:tasks.workunit.client.0.smithi144.stdout:1/626: write d5/dd/f5e [539545,9438] 0 2024-09-10T11:29:39.869 INFO:tasks.workunit.client.0.smithi144.stdout:0/337: dwrite da/f26 [0,4194304] 0 2024-09-10T11:29:39.871 INFO:tasks.workunit.client.0.smithi144.stdout:1/627: chown d5/d2a/d65/faf 1013 1 2024-09-10T11:29:39.872 INFO:tasks.workunit.client.0.smithi144.stdout:0/338: chown da/d3a/f4d 1587 1 2024-09-10T11:29:39.879 INFO:tasks.workunit.client.0.smithi144.stdout:9/513: getdents d11/d12/d1b/d2c/d8a 0 2024-09-10T11:29:39.879 INFO:tasks.workunit.client.1.smithi178.stdout:9/907: dread d2/d9b/df8/d2d/dab/d118/d89/fbd [0,4194304] 0 2024-09-10T11:29:39.881 INFO:tasks.workunit.client.0.smithi144.stdout:1/628: link d5/d2a/d65/dc9/f67 d5/d4a/d63/d84/d95/fd1 0 2024-09-10T11:29:39.882 INFO:tasks.workunit.client.1.smithi178.stdout:9/908: read d2/d19/d5e/f9e [758536,110803] 0 2024-09-10T11:29:39.883 INFO:tasks.workunit.client.0.smithi144.stdout:0/339: dread da/f32 [0,4194304] 0 2024-09-10T11:29:39.884 INFO:tasks.workunit.client.0.smithi144.stdout:0/340: truncate da/f36 5605396 0 2024-09-10T11:29:39.884 INFO:tasks.workunit.client.1.smithi178.stdout:9/909: mknod d2/dc/c11f 0 2024-09-10T11:29:39.885 INFO:tasks.workunit.client.0.smithi144.stdout:1/629: unlink d5/d4a/lcc 0 2024-09-10T11:29:39.890 INFO:tasks.workunit.client.0.smithi144.stdout:1/630: mkdir d5/d4a/d63/d84/d95/dd2 0 2024-09-10T11:29:39.891 INFO:tasks.workunit.client.0.smithi144.stdout:0/341: mkdir da/d3a/d4b/d61 0 2024-09-10T11:29:39.892 INFO:tasks.workunit.client.0.smithi144.stdout:9/514: link d11/d12/d1b/d2c/l65 d11/d12/d1b/d22/lb5 0 2024-09-10T11:29:39.893 INFO:tasks.workunit.client.1.smithi178.stdout:9/910: dread d2/d9b/df8/d2d/dab/d118/d89/d93/d88/fa8 [0,4194304] 0 2024-09-10T11:29:39.893 INFO:tasks.workunit.client.0.smithi144.stdout:4/515: sync 2024-09-10T11:29:39.893 INFO:tasks.workunit.client.0.smithi144.stdout:3/540: sync 2024-09-10T11:29:39.894 INFO:tasks.workunit.client.0.smithi144.stdout:0/342: mkdir da/d3a/d4b/d4e/d62 0 2024-09-10T11:29:39.895 INFO:tasks.workunit.client.1.smithi178.stdout:9/911: getdents d2/d113 0 2024-09-10T11:29:39.895 INFO:tasks.workunit.client.0.smithi144.stdout:3/541: chown d4/d28/d67/d80/d17/d18/d34/l5e 7 1 2024-09-10T11:29:39.895 INFO:tasks.workunit.client.0.smithi144.stdout:9/515: creat d11/d12/d1b/d41/d8e/fb6 x:0 0 0 2024-09-10T11:29:39.896 INFO:tasks.workunit.client.1.smithi178.stdout:9/912: write d2/df/fd7 [1828622,27020] 0 2024-09-10T11:29:39.896 INFO:tasks.workunit.client.1.smithi178.stdout:3/898: sync 2024-09-10T11:29:39.896 INFO:tasks.workunit.client.1.smithi178.stdout:7/777: sync 2024-09-10T11:29:39.896 INFO:tasks.workunit.client.1.smithi178.stdout:8/953: sync 2024-09-10T11:29:39.898 INFO:tasks.workunit.client.0.smithi144.stdout:9/516: dread d11/d12/fa1 [0,4194304] 0 2024-09-10T11:29:39.899 INFO:tasks.workunit.client.0.smithi144.stdout:0/343: creat da/d44/f63 x:0 0 0 2024-09-10T11:29:39.901 INFO:tasks.workunit.client.1.smithi178.stdout:8/954: rmdir d5/d1d/d22/d10f 39 2024-09-10T11:29:39.902 INFO:tasks.workunit.client.0.smithi144.stdout:0/344: write da/f46 [2753822,54524] 0 2024-09-10T11:29:39.902 INFO:tasks.workunit.client.1.smithi178.stdout:9/913: dread - d2/d9b/df8/d2d/dab/f87 zero size 2024-09-10T11:29:39.904 INFO:tasks.workunit.client.0.smithi144.stdout:4/516: dwrite d14/d1e/d22/d34/d3e/d40/d4b/f57 [0,4194304] 0 2024-09-10T11:29:39.905 INFO:tasks.workunit.client.1.smithi178.stdout:7/778: unlink d6/d10/d1e/d2d/d2b/cc3 0 2024-09-10T11:29:39.905 INFO:tasks.workunit.client.1.smithi178.stdout:3/899: dwrite db/d26/f32 [0,4194304] 0 2024-09-10T11:29:39.907 INFO:tasks.workunit.client.0.smithi144.stdout:3/542: link d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93/faa d4/d28/d67/d9f/fb5 0 2024-09-10T11:29:39.908 INFO:tasks.workunit.client.1.smithi178.stdout:9/914: mknod d2/df/d31/d37/d114/c120 0 2024-09-10T11:29:39.909 INFO:tasks.workunit.client.0.smithi144.stdout:0/345: mkdir da/d44/d64 0 2024-09-10T11:29:39.911 INFO:tasks.workunit.client.0.smithi144.stdout:4/517: creat d14/d1e/d22/d34/d3e/d40/d4b/d4c/fba x:0 0 0 2024-09-10T11:29:39.911 INFO:tasks.workunit.client.1.smithi178.stdout:6/866: write d3/d7/d8/d15/d1c/d2a/d18/f70 [2954202,12991] 0 2024-09-10T11:29:39.912 INFO:tasks.workunit.client.0.smithi144.stdout:0/346: write da/f46 [2171380,7007] 0 2024-09-10T11:29:39.915 INFO:tasks.workunit.client.1.smithi178.stdout:6/867: fdatasync d3/d74/d91/daa/dc2/d57/d78/f128 0 2024-09-10T11:29:39.916 INFO:tasks.workunit.client.1.smithi178.stdout:7/779: fsync d6/d10/d1d/d4a/f62 0 2024-09-10T11:29:39.917 INFO:tasks.workunit.client.1.smithi178.stdout:0/786: truncate d0/d2c/d33/f45 3742902 0 2024-09-10T11:29:39.919 INFO:tasks.workunit.client.0.smithi144.stdout:4/518: dwrite d14/d1e/d22/f31 [0,4194304] 0 2024-09-10T11:29:39.921 INFO:tasks.workunit.client.0.smithi144.stdout:0/347: creat da/d44/f65 x:0 0 0 2024-09-10T11:29:39.922 INFO:tasks.workunit.client.1.smithi178.stdout:9/915: dwrite d2/df/d31/d37/d3d/fc3 [0,4194304] 0 2024-09-10T11:29:39.922 INFO:tasks.workunit.client.1.smithi178.stdout:6/868: mkdir d3/d74/d91/daa/dc2/d13d 0 2024-09-10T11:29:39.924 INFO:tasks.workunit.client.0.smithi144.stdout:0/348: mknod da/d3a/d4f/c66 0 2024-09-10T11:29:39.931 INFO:tasks.workunit.client.1.smithi178.stdout:7/780: creat d6/d10/d1d/d22/d50/d7e/d82/f115 x:0 0 0 2024-09-10T11:29:39.932 INFO:tasks.workunit.client.1.smithi178.stdout:3/900: link db/d20/f3c db/d20/d3e/dc5/f122 0 2024-09-10T11:29:39.933 INFO:tasks.workunit.client.0.smithi144.stdout:5/569: sync 2024-09-10T11:29:39.933 INFO:tasks.workunit.client.1.smithi178.stdout:3/901: write db/d20/fa5 [236065,75739] 0 2024-09-10T11:29:39.934 INFO:tasks.workunit.client.1.smithi178.stdout:3/902: chown db/l2d 318493029 1 2024-09-10T11:29:39.937 INFO:tasks.workunit.client.1.smithi178.stdout:6/869: fdatasync d3/d7/f135 0 2024-09-10T11:29:39.938 INFO:tasks.workunit.client.0.smithi144.stdout:1/631: dwrite d5/d4a/d44/f94 [0,4194304] 0 2024-09-10T11:29:39.940 INFO:tasks.workunit.client.1.smithi178.stdout:9/916: rmdir d2 39 2024-09-10T11:29:39.940 INFO:tasks.workunit.client.1.smithi178.stdout:8/955: dwrite d5/db/d10/d20/f28 [0,4194304] 0 2024-09-10T11:29:39.941 INFO:tasks.workunit.client.0.smithi144.stdout:5/570: symlink d4/d20/d24/la9 0 2024-09-10T11:29:39.943 INFO:tasks.workunit.client.0.smithi144.stdout:3/543: dread d4/dd/f1b [0,4194304] 0 2024-09-10T11:29:39.943 INFO:tasks.workunit.client.0.smithi144.stdout:0/349: dread da/f36 [4194304,4194304] 0 2024-09-10T11:29:39.946 INFO:tasks.workunit.client.0.smithi144.stdout:9/517: dwrite d11/d12/d1b/f20 [0,4194304] 0 2024-09-10T11:29:39.948 INFO:tasks.workunit.client.0.smithi144.stdout:1/632: truncate d5/d4a/d63/fac 731066 0 2024-09-10T11:29:39.950 INFO:tasks.workunit.client.0.smithi144.stdout:1/633: chown d5/d4a/d63/c68 1516 1 2024-09-10T11:29:39.950 INFO:tasks.workunit.client.0.smithi144.stdout:5/571: mkdir d4/d5/d32/daa 0 2024-09-10T11:29:39.950 INFO:tasks.workunit.client.0.smithi144.stdout:3/544: stat d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93/d98 0 2024-09-10T11:29:39.953 INFO:tasks.workunit.client.1.smithi178.stdout:3/903: dread db/d36/f50 [0,4194304] 0 2024-09-10T11:29:39.955 INFO:tasks.workunit.client.0.smithi144.stdout:9/518: unlink d11/d12/d1b/l42 0 2024-09-10T11:29:39.958 INFO:tasks.workunit.client.0.smithi144.stdout:4/519: dread d14/d1e/d22/d2b/d35/d4f/f60 [0,4194304] 0 2024-09-10T11:29:39.958 INFO:tasks.workunit.client.0.smithi144.stdout:9/519: chown d11/d12/d1b/d41/d8f/dab/d64/cb2 1765 1 2024-09-10T11:29:39.959 INFO:tasks.workunit.client.0.smithi144.stdout:0/350: rename da/c24 to da/d3a/d51/c67 0 2024-09-10T11:29:39.962 INFO:tasks.workunit.client.0.smithi144.stdout:5/572: unlink d4/d20/d22/d5c/d5d/d64/f8e 0 2024-09-10T11:29:39.962 INFO:tasks.workunit.client.1.smithi178.stdout:3/904: mknod db/c123 0 2024-09-10T11:29:39.965 INFO:tasks.workunit.client.0.smithi144.stdout:9/520: symlink d11/d12/d1b/d41/d8f/lb7 0 2024-09-10T11:29:39.966 INFO:tasks.workunit.client.1.smithi178.stdout:9/917: creat d2/df/f121 x:0 0 0 2024-09-10T11:29:39.967 INFO:tasks.workunit.client.0.smithi144.stdout:5/573: creat d4/d20/d24/d6f/d84/fab x:0 0 0 2024-09-10T11:29:39.968 INFO:tasks.workunit.client.1.smithi178.stdout:6/870: write d3/d6e/ff1 [963809,30212] 0 2024-09-10T11:29:39.969 INFO:tasks.workunit.client.0.smithi144.stdout:0/351: creat da/d3a/d4b/d4e/d62/f68 x:0 0 0 2024-09-10T11:29:39.969 INFO:tasks.workunit.client.1.smithi178.stdout:3/905: dwrite db/d20/d3d/d56/d64/fae [0,4194304] 0 2024-09-10T11:29:39.970 INFO:tasks.workunit.client.1.smithi178.stdout:9/918: fdatasync d2/d9b/df8/d2d/dab/d118/d56/de1/ff5 0 2024-09-10T11:29:39.971 INFO:tasks.workunit.client.0.smithi144.stdout:1/634: creat d5/d4a/d44/fd3 x:0 0 0 2024-09-10T11:29:39.971 INFO:tasks.workunit.client.1.smithi178.stdout:7/781: truncate d6/d10/d34/f74 4083836 0 2024-09-10T11:29:39.972 INFO:tasks.workunit.client.1.smithi178.stdout:8/956: write d5/db/d10/d20/d52/d67/fdf [896854,51473] 0 2024-09-10T11:29:39.973 INFO:tasks.workunit.client.0.smithi144.stdout:1/635: write d5/d4a/d63/d84/dab/fb2 [198014,53932] 0 2024-09-10T11:29:39.974 INFO:tasks.workunit.client.0.smithi144.stdout:1/636: write d5/d4a/dc0/fc6 [812473,22248] 0 2024-09-10T11:29:39.976 INFO:tasks.workunit.client.1.smithi178.stdout:7/782: read - d6/d10/d1d/d22/d50/d7e/d82/db2/d49/f110 zero size 2024-09-10T11:29:39.977 INFO:tasks.workunit.client.1.smithi178.stdout:6/871: creat d3/d74/d91/dee/f13e x:0 0 0 2024-09-10T11:29:39.977 INFO:tasks.workunit.client.1.smithi178.stdout:9/919: read d2/d13/d75/f5d [645444,6253] 0 2024-09-10T11:29:39.978 INFO:tasks.workunit.client.1.smithi178.stdout:8/957: symlink d5/d30/l145 0 2024-09-10T11:29:39.979 INFO:tasks.workunit.client.0.smithi144.stdout:4/520: rename d14/d1e/d22/d2b/d35/d3c/f63 to d14/d1e/d22/d34/d3e/d40/fbb 0 2024-09-10T11:29:39.979 INFO:tasks.workunit.client.1.smithi178.stdout:8/958: fsync d5/db/d3d/de7/ffd 0 2024-09-10T11:29:39.980 INFO:tasks.workunit.client.0.smithi144.stdout:5/574: creat d4/d5/d18/d38/d8d/fac x:0 0 0 2024-09-10T11:29:39.980 INFO:tasks.workunit.client.1.smithi178.stdout:7/783: stat d6/ce 0 2024-09-10T11:29:39.981 INFO:tasks.workunit.client.0.smithi144.stdout:1/637: write d5/d4a/f7d [398972,12494] 0 2024-09-10T11:29:39.982 INFO:tasks.workunit.client.1.smithi178.stdout:3/906: dwrite db/d20/d3e/d6b/fd4 [0,4194304] 0 2024-09-10T11:29:39.983 INFO:tasks.workunit.client.1.smithi178.stdout:3/907: fsync db/d20/d3d/d56/dcf/f106 0 2024-09-10T11:29:39.983 INFO:tasks.workunit.client.0.smithi144.stdout:3/545: dread d4/d28/d67/d80/d17/d18/f48 [0,4194304] 0 2024-09-10T11:29:39.984 INFO:tasks.workunit.client.0.smithi144.stdout:3/546: fsync d4/dd/d21/d2f/f8d 0 2024-09-10T11:29:39.986 INFO:tasks.workunit.client.1.smithi178.stdout:7/784: chown d6/d10/f5c 253896234 1 2024-09-10T11:29:39.988 INFO:tasks.workunit.client.1.smithi178.stdout:3/908: fdatasync db/d26/d103/f105 0 2024-09-10T11:29:39.988 INFO:tasks.workunit.client.1.smithi178.stdout:8/959: truncate d5/d30/f4b 4894353 0 2024-09-10T11:29:39.990 INFO:tasks.workunit.client.0.smithi144.stdout:5/575: creat d4/d52/fad x:0 0 0 2024-09-10T11:29:39.991 INFO:tasks.workunit.client.0.smithi144.stdout:5/576: write d4/d20/d24/d2d/f80 [2421056,125795] 0 2024-09-10T11:29:39.991 INFO:tasks.workunit.client.1.smithi178.stdout:3/909: creat db/d35/f124 x:0 0 0 2024-09-10T11:29:39.992 INFO:tasks.workunit.client.0.smithi144.stdout:5/577: chown d4/d5/d18/d38/l95 1512 1 2024-09-10T11:29:39.994 INFO:tasks.workunit.client.0.smithi144.stdout:0/352: rmdir da/d44/d64 0 2024-09-10T11:29:39.995 INFO:tasks.workunit.client.0.smithi144.stdout:0/353: write da/d44/f54 [1641860,100242] 0 2024-09-10T11:29:39.996 INFO:tasks.workunit.client.1.smithi178.stdout:8/960: getdents d5/db/d10/d20 0 2024-09-10T11:29:39.997 INFO:tasks.workunit.client.0.smithi144.stdout:4/521: mkdir d14/d1e/d83/da4/da9/dbc 0 2024-09-10T11:29:39.998 INFO:tasks.workunit.client.1.smithi178.stdout:8/961: stat d5/db/d1e/f5e 0 2024-09-10T11:29:39.999 INFO:tasks.workunit.client.1.smithi178.stdout:8/962: readlink d5/db/d10/d35/la6 0 2024-09-10T11:29:40.000 INFO:tasks.workunit.client.0.smithi144.stdout:5/578: dwrite d4/d20/d24/f3e [0,4194304] 0 2024-09-10T11:29:40.002 INFO:tasks.workunit.client.0.smithi144.stdout:1/638: link d5/d2a/d65/fcd d5/dd/da1/fd4 0 2024-09-10T11:29:40.003 INFO:tasks.workunit.client.0.smithi144.stdout:1/639: truncate d5/dd/f5e 1141085 0 2024-09-10T11:29:40.004 INFO:tasks.workunit.client.0.smithi144.stdout:9/521: truncate d11/d36/f7f 2700346 0 2024-09-10T11:29:40.006 INFO:tasks.workunit.client.0.smithi144.stdout:5/579: symlink d4/d5/d18/d51/lae 0 2024-09-10T11:29:40.006 INFO:tasks.workunit.client.0.smithi144.stdout:9/522: truncate d11/d12/d1b/d41/d8f/f82 4301743 0 2024-09-10T11:29:40.007 INFO:tasks.workunit.client.0.smithi144.stdout:9/523: fdatasync d11/d12/d1b/f71 0 2024-09-10T11:29:40.008 INFO:tasks.workunit.client.0.smithi144.stdout:3/547: getdents d4/d28/d67/d80/d17/d18/d34/d65 0 2024-09-10T11:29:40.009 INFO:tasks.workunit.client.0.smithi144.stdout:5/580: dread d4/d52/f69 [0,4194304] 0 2024-09-10T11:29:40.010 INFO:tasks.workunit.client.0.smithi144.stdout:3/548: write d4/d28/d67/d80/d1d/f29 [152246,38604] 0 2024-09-10T11:29:40.010 INFO:tasks.workunit.client.0.smithi144.stdout:5/581: truncate d4/d5/d18/d51/f88 188209 0 2024-09-10T11:29:40.011 INFO:tasks.workunit.client.0.smithi144.stdout:9/524: creat d11/d12/d1b/d41/d8f/dab/fb8 x:0 0 0 2024-09-10T11:29:40.011 INFO:tasks.workunit.client.0.smithi144.stdout:1/640: getdents d5/d4a/dc0 0 2024-09-10T11:29:40.012 INFO:tasks.workunit.client.1.smithi178.stdout:3/910: dread db/d20/d3e/d6b/dfa/f9d [0,4194304] 0 2024-09-10T11:29:40.013 INFO:tasks.workunit.client.0.smithi144.stdout:1/641: stat d5/df/d4b/d51/f59 0 2024-09-10T11:29:40.014 INFO:tasks.workunit.client.0.smithi144.stdout:5/582: mknod d4/d52/caf 0 2024-09-10T11:29:40.015 INFO:tasks.workunit.client.0.smithi144.stdout:3/549: mkdir d4/d28/db6 0 2024-09-10T11:29:40.016 INFO:tasks.workunit.client.1.smithi178.stdout:3/911: creat db/d20/d3e/d6b/dfa/f125 x:0 0 0 2024-09-10T11:29:40.017 INFO:tasks.workunit.client.0.smithi144.stdout:9/525: rename d11/d12/d1b/d41/d8f/dab/d64/cb2 to d11/d12/d1b/d41/d8f/dab/d64/cb9 0 2024-09-10T11:29:40.017 INFO:tasks.workunit.client.1.smithi178.stdout:3/912: chown db/d20/d3e/d6b/fa7 4451322 1 2024-09-10T11:29:40.019 INFO:tasks.workunit.client.1.smithi178.stdout:3/913: mkdir db/d20/d3d/d126 0 2024-09-10T11:29:40.021 INFO:tasks.workunit.client.1.smithi178.stdout:3/914: getdents db/d20/d3e/d6b/dc0/ddf/dcb/d81 0 2024-09-10T11:29:40.022 INFO:tasks.workunit.client.1.smithi178.stdout:3/915: dread - db/d20/d3d/d56/dcf/ffc zero size 2024-09-10T11:29:40.025 INFO:tasks.workunit.client.1.smithi178.stdout:3/916: mkdir db/d35/d58/df2/d127 0 2024-09-10T11:29:40.025 INFO:tasks.workunit.client.0.smithi144.stdout:3/550: getdents d4/d1a 0 2024-09-10T11:29:40.026 INFO:tasks.workunit.client.1.smithi178.stdout:3/917: rename db/d20 to db/d20/d3d/d56/dcf/d11e/d128 22 2024-09-10T11:29:40.027 INFO:tasks.workunit.client.0.smithi144.stdout:9/526: rename d11/d12/d1b/d22/c37 to d11/d12/d1b/d22/cba 0 2024-09-10T11:29:40.028 INFO:tasks.workunit.client.0.smithi144.stdout:3/551: mknod d4/d1a/cb7 0 2024-09-10T11:29:40.030 INFO:tasks.workunit.client.1.smithi178.stdout:3/918: dread db/ff [0,4194304] 0 2024-09-10T11:29:40.030 INFO:tasks.workunit.client.0.smithi144.stdout:9/527: mkdir d11/d36/db1/dbb 0 2024-09-10T11:29:40.033 INFO:tasks.workunit.client.1.smithi178.stdout:3/919: creat db/d26/d74/dd0/f129 x:0 0 0 2024-09-10T11:29:40.034 INFO:tasks.workunit.client.0.smithi144.stdout:9/528: mknod d11/d12/d1b/d41/d8f/dab/cbc 0 2024-09-10T11:29:40.035 INFO:tasks.workunit.client.0.smithi144.stdout:3/552: dwrite d4/d28/d67/d80/d17/d18/d34/f5c [0,4194304] 0 2024-09-10T11:29:40.036 INFO:tasks.workunit.client.0.smithi144.stdout:3/553: readlink d4/d28/d67/d80/d17/d18/l79 0 2024-09-10T11:29:40.040 INFO:tasks.workunit.client.0.smithi144.stdout:9/529: rename d11/d12/d1b/f71 to d11/d12/d1b/d41/d8f/fbd 0 2024-09-10T11:29:40.040 INFO:tasks.workunit.client.0.smithi144.stdout:9/530: fsync d11/d12/d1b/f20 0 2024-09-10T11:29:40.042 INFO:tasks.workunit.client.0.smithi144.stdout:3/554: mknod d4/dd/cb8 0 2024-09-10T11:29:40.043 INFO:tasks.workunit.client.0.smithi144.stdout:3/555: readlink d4/d28/d67/d80/d17/d18/d34/d4b/l69 0 2024-09-10T11:29:40.046 INFO:tasks.workunit.client.1.smithi178.stdout:3/920: dread db/f45 [4194304,4194304] 0 2024-09-10T11:29:40.046 INFO:tasks.workunit.client.0.smithi144.stdout:9/531: rename d11/d12/c19 to d11/d12/d6f/cbe 0 2024-09-10T11:29:40.047 INFO:tasks.workunit.client.0.smithi144.stdout:1/642: truncate d5/d4a/dc0/fc6 282294 0 2024-09-10T11:29:40.048 INFO:tasks.workunit.client.0.smithi144.stdout:3/556: symlink d4/dd/d21/d2f/lb9 0 2024-09-10T11:29:40.049 INFO:tasks.workunit.client.1.smithi178.stdout:3/921: unlink db/d35/d3a/d9f/f120 0 2024-09-10T11:29:40.049 INFO:tasks.workunit.client.0.smithi144.stdout:9/532: mknod d11/d12/d1b/d2c/cbf 0 2024-09-10T11:29:40.050 INFO:tasks.workunit.client.1.smithi178.stdout:3/922: read db/ff [2928736,14691] 0 2024-09-10T11:29:40.051 INFO:tasks.workunit.client.0.smithi144.stdout:1/643: creat d5/d2a/d65/d73/fd5 x:0 0 0 2024-09-10T11:29:40.053 INFO:tasks.workunit.client.1.smithi178.stdout:3/923: creat db/d20/d3d/d63/da2/d98/f12a x:0 0 0 2024-09-10T11:29:40.053 INFO:tasks.workunit.client.0.smithi144.stdout:1/644: stat d5/f38 0 2024-09-10T11:29:40.053 INFO:tasks.workunit.client.1.smithi178.stdout:3/924: chown db/c115 107 1 2024-09-10T11:29:40.054 INFO:tasks.workunit.client.0.smithi144.stdout:9/533: mkdir d11/d12/d1b/d41/d8f/dc0 0 2024-09-10T11:29:40.055 INFO:tasks.workunit.client.0.smithi144.stdout:5/583: dwrite d4/fa [0,4194304] 0 2024-09-10T11:29:40.055 INFO:tasks.workunit.client.0.smithi144.stdout:1/645: readlink d5/d2a/d65/dc9/l71 0 2024-09-10T11:29:40.057 INFO:tasks.workunit.client.0.smithi144.stdout:9/534: fdatasync d11/d12/d1b/d41/d8f/dab/d64/f68 0 2024-09-10T11:29:40.058 INFO:tasks.workunit.client.0.smithi144.stdout:9/535: stat d11/d36/l46 0 2024-09-10T11:29:40.060 INFO:tasks.workunit.client.0.smithi144.stdout:1/646: write d5/df/d4b/d51/f59 [1812153,65409] 0 2024-09-10T11:29:40.064 INFO:tasks.workunit.client.0.smithi144.stdout:1/647: read d5/df/d4b/fb4 [3495601,111761] 0 2024-09-10T11:29:40.064 INFO:tasks.workunit.client.0.smithi144.stdout:9/536: creat d11/d12/d1b/d2c/d78/fc1 x:0 0 0 2024-09-10T11:29:40.065 INFO:tasks.workunit.client.0.smithi144.stdout:9/537: fsync d11/d12/d1b/d22/f4a 0 2024-09-10T11:29:40.067 INFO:tasks.workunit.client.0.smithi144.stdout:1/648: creat d5/dd/d64/fd6 x:0 0 0 2024-09-10T11:29:40.068 INFO:tasks.workunit.client.0.smithi144.stdout:3/557: dread d4/d1a/f5f [0,4194304] 0 2024-09-10T11:29:40.072 INFO:tasks.workunit.client.0.smithi144.stdout:9/538: creat d11/d36/db1/dbb/fc2 x:0 0 0 2024-09-10T11:29:40.073 INFO:tasks.workunit.client.0.smithi144.stdout:9/539: write d11/d12/d1b/d22/f3d [3633869,104638] 0 2024-09-10T11:29:40.073 INFO:tasks.workunit.client.0.smithi144.stdout:3/558: fsync d4/d28/f36 0 2024-09-10T11:29:40.073 INFO:tasks.workunit.client.0.smithi144.stdout:9/540: write d11/d36/d2d/f9a [2372762,128374] 0 2024-09-10T11:29:40.073 INFO:tasks.workunit.client.1.smithi178.stdout:3/925: dwrite db/ddd/ff3 [4194304,4194304] 0 2024-09-10T11:29:40.078 INFO:tasks.workunit.client.0.smithi144.stdout:1/649: creat d5/d4a/d44/da2/fd7 x:0 0 0 2024-09-10T11:29:40.079 INFO:tasks.workunit.client.0.smithi144.stdout:3/559: rmdir d4/d28/d67/d80/d17 39 2024-09-10T11:29:40.080 INFO:tasks.workunit.client.0.smithi144.stdout:9/541: mkdir d11/d36/d2d/d38/dc3 0 2024-09-10T11:29:40.081 INFO:tasks.workunit.client.0.smithi144.stdout:1/650: mkdir d5/d2a/d65/d73/dd8 0 2024-09-10T11:29:40.081 INFO:tasks.workunit.client.1.smithi178.stdout:1/804: sync 2024-09-10T11:29:40.081 INFO:tasks.workunit.client.1.smithi178.stdout:4/768: sync 2024-09-10T11:29:40.082 INFO:tasks.workunit.client.0.smithi144.stdout:5/584: dread d4/d5/d18/d67/d59/f5e [0,4194304] 0 2024-09-10T11:29:40.083 INFO:tasks.workunit.client.1.smithi178.stdout:3/926: truncate db/d20/d3d/d56/f66 3782979 0 2024-09-10T11:29:40.084 INFO:tasks.workunit.client.1.smithi178.stdout:4/769: symlink d0/d10/l10c 0 2024-09-10T11:29:40.085 INFO:tasks.workunit.client.0.smithi144.stdout:1/651: mknod d5/d4a/d63/d84/dbd/dbe/cd9 0 2024-09-10T11:29:40.086 INFO:tasks.workunit.client.1.smithi178.stdout:4/770: chown d0/d8/da/d1a/d51/d5d/fd2 23 1 2024-09-10T11:29:40.087 INFO:tasks.workunit.client.1.smithi178.stdout:4/771: dread - d0/d8/da/d1a/d51/d5d/fd2 zero size 2024-09-10T11:29:40.087 INFO:tasks.workunit.client.0.smithi144.stdout:3/560: symlink d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93/d98/d9a/lba 0 2024-09-10T11:29:40.088 INFO:tasks.workunit.client.0.smithi144.stdout:5/585: mkdir d4/d5/d18/d67/db0 0 2024-09-10T11:29:40.089 INFO:tasks.workunit.client.0.smithi144.stdout:3/561: write d4/d28/f87 [2568155,67406] 0 2024-09-10T11:29:40.089 INFO:tasks.workunit.client.1.smithi178.stdout:1/805: symlink d3/d9/d98/dac/d3a/dae/l12e 0 2024-09-10T11:29:40.090 INFO:tasks.workunit.client.0.smithi144.stdout:9/542: dwrite d11/f15 [0,4194304] 0 2024-09-10T11:29:40.093 INFO:tasks.workunit.client.1.smithi178.stdout:3/927: write db/d26/d40/f7a [879023,85847] 0 2024-09-10T11:29:40.094 INFO:tasks.workunit.client.0.smithi144.stdout:5/586: mkdir d4/d52/db1 0 2024-09-10T11:29:40.095 INFO:tasks.workunit.client.1.smithi178.stdout:1/806: dread d3/d9/fb [0,4194304] 0 2024-09-10T11:29:40.099 INFO:tasks.workunit.client.0.smithi144.stdout:3/562: dwrite d4/d28/d67/d80/d17/d18/f2e [0,4194304] 0 2024-09-10T11:29:40.101 INFO:tasks.workunit.client.1.smithi178.stdout:3/928: dwrite db/d26/d40/f69 [0,4194304] 0 2024-09-10T11:29:40.102 INFO:tasks.workunit.client.1.smithi178.stdout:3/929: chown db/d9b/d11b 931587154 1 2024-09-10T11:29:40.103 INFO:tasks.workunit.client.0.smithi144.stdout:5/587: creat d4/d5/d18/d38/d8d/fb2 x:0 0 0 2024-09-10T11:29:40.106 INFO:tasks.workunit.client.0.smithi144.stdout:5/588: dread - d4/d5/d18/d38/d8d/fb2 zero size 2024-09-10T11:29:40.108 INFO:tasks.workunit.client.0.smithi144.stdout:5/589: fsync d4/d5/d18/d38/d6d/d8b/f9b 0 2024-09-10T11:29:40.109 INFO:tasks.workunit.client.0.smithi144.stdout:3/563: creat d4/d28/d67/d9f/fbb x:0 0 0 2024-09-10T11:29:40.109 INFO:tasks.workunit.client.0.smithi144.stdout:1/652: truncate d5/f12 3082375 0 2024-09-10T11:29:40.111 INFO:tasks.workunit.client.0.smithi144.stdout:1/653: chown d5/d4a/d63/d84/dab/cb8 1941 1 2024-09-10T11:29:40.113 INFO:tasks.workunit.client.0.smithi144.stdout:9/543: truncate d11/d12/d1b/d41/d8f/fbd 135690 0 2024-09-10T11:29:40.116 INFO:tasks.workunit.client.0.smithi144.stdout:3/564: dread d4/d28/d67/d80/fa2 [0,4194304] 0 2024-09-10T11:29:40.117 INFO:tasks.workunit.client.0.smithi144.stdout:9/544: creat d11/d36/db1/fc4 x:0 0 0 2024-09-10T11:29:40.120 INFO:tasks.workunit.client.0.smithi144.stdout:5/590: truncate d4/d5/d32/d47/f71 749270 0 2024-09-10T11:29:40.121 INFO:tasks.workunit.client.0.smithi144.stdout:1/654: truncate d5/dd/f1d 2349188 0 2024-09-10T11:29:40.123 INFO:tasks.workunit.client.0.smithi144.stdout:3/565: rmdir d4/d28/d67/d80/da3 0 2024-09-10T11:29:40.126 INFO:tasks.workunit.client.0.smithi144.stdout:1/655: symlink d5/df/d4b/dcf/lda 0 2024-09-10T11:29:40.127 INFO:tasks.workunit.client.0.smithi144.stdout:9/545: dwrite d11/fa2 [0,4194304] 0 2024-09-10T11:29:40.129 INFO:tasks.workunit.client.0.smithi144.stdout:1/656: fdatasync d5/f38 0 2024-09-10T11:29:40.129 INFO:tasks.workunit.client.0.smithi144.stdout:5/591: truncate d4/d5/d18/d38/d8d/fac 787581 0 2024-09-10T11:29:40.129 INFO:tasks.workunit.client.0.smithi144.stdout:3/566: chown d4/d28/d67/d80/d17/d18/d34/d78/cab 577 1 2024-09-10T11:29:40.130 INFO:tasks.workunit.client.0.smithi144.stdout:9/546: write d11/d36/f9c [454610,89481] 0 2024-09-10T11:29:40.131 INFO:tasks.workunit.client.0.smithi144.stdout:5/592: chown d4/d20/d24/d6f/d74 776 1 2024-09-10T11:29:40.131 INFO:tasks.workunit.client.0.smithi144.stdout:5/593: fdatasync d4/d5/f23 0 2024-09-10T11:29:40.132 INFO:tasks.workunit.client.0.smithi144.stdout:9/547: stat d11/d12/d1b/d2c/d8a 0 2024-09-10T11:29:40.133 INFO:tasks.workunit.client.0.smithi144.stdout:5/594: write d4/d52/fad [332912,44982] 0 2024-09-10T11:29:40.134 INFO:tasks.workunit.client.0.smithi144.stdout:5/595: chown d4/d5/d18/d51/l98 67 1 2024-09-10T11:29:40.136 INFO:tasks.workunit.client.0.smithi144.stdout:9/548: creat d11/fc5 x:0 0 0 2024-09-10T11:29:40.137 INFO:tasks.workunit.client.0.smithi144.stdout:5/596: stat d4/d5/d18/l21 0 2024-09-10T11:29:40.139 INFO:tasks.workunit.client.0.smithi144.stdout:9/549: chown d11/d12/d1b/d41/d8f/dab/c60 11110669 1 2024-09-10T11:29:40.140 INFO:tasks.workunit.client.0.smithi144.stdout:1/657: dwrite d5/df/d4b/d51/f50 [0,4194304] 0 2024-09-10T11:29:40.141 INFO:tasks.workunit.client.0.smithi144.stdout:1/658: readlink d5/d2a/d65/l70 0 2024-09-10T11:29:40.144 INFO:tasks.workunit.client.0.smithi144.stdout:9/550: creat d11/d36/db1/fc6 x:0 0 0 2024-09-10T11:29:40.144 INFO:tasks.workunit.client.0.smithi144.stdout:8/360: sync 2024-09-10T11:29:40.144 INFO:tasks.workunit.client.0.smithi144.stdout:6/505: sync 2024-09-10T11:29:40.144 INFO:tasks.workunit.client.0.smithi144.stdout:7/523: sync 2024-09-10T11:29:40.145 INFO:tasks.workunit.client.0.smithi144.stdout:2/413: sync 2024-09-10T11:29:40.145 INFO:tasks.workunit.client.0.smithi144.stdout:5/597: symlink d4/d5/d18/d38/d6d/da6/lb3 0 2024-09-10T11:29:40.145 INFO:tasks.workunit.client.0.smithi144.stdout:1/659: symlink d5/d4a/d44/da2/ldb 0 2024-09-10T11:29:40.146 INFO:tasks.workunit.client.0.smithi144.stdout:9/551: dread - d11/d36/f79 zero size 2024-09-10T11:29:40.148 INFO:tasks.workunit.client.0.smithi144.stdout:6/506: fdatasync d9/d1f/fb6 0 2024-09-10T11:29:40.153 INFO:tasks.workunit.client.0.smithi144.stdout:9/552: symlink d11/d12/d1b/d2c/lc7 0 2024-09-10T11:29:40.157 INFO:tasks.workunit.client.0.smithi144.stdout:6/507: fsync d9/dd/d3e/d83/daf/d5b/d91/fb5 0 2024-09-10T11:29:40.162 INFO:tasks.workunit.client.0.smithi144.stdout:5/598: dwrite d4/d20/f43 [4194304,4194304] 0 2024-09-10T11:29:40.163 INFO:tasks.workunit.client.0.smithi144.stdout:9/553: read - d11/d12/d1b/d2c/d78/fc1 zero size 2024-09-10T11:29:40.165 INFO:tasks.workunit.client.0.smithi144.stdout:8/361: dwrite d0/d1b/d47/d5e/f64 [0,4194304] 0 2024-09-10T11:29:40.168 INFO:tasks.workunit.client.0.smithi144.stdout:1/660: dread d5/d4a/d44/f94 [0,4194304] 0 2024-09-10T11:29:40.169 INFO:tasks.workunit.client.0.smithi144.stdout:2/414: dwrite d7/da/d1e/f66 [0,4194304] 0 2024-09-10T11:29:40.169 INFO:tasks.workunit.client.0.smithi144.stdout:5/599: chown d4/d20/d24/f92 119 1 2024-09-10T11:29:40.175 INFO:tasks.workunit.client.0.smithi144.stdout:9/554: chown d11/d12/d1b/d41/d8e/f99 12295 1 2024-09-10T11:29:40.175 INFO:tasks.workunit.client.0.smithi144.stdout:1/661: dread - d5/dd/d64/d79/fb3 zero size 2024-09-10T11:29:40.175 INFO:tasks.workunit.client.0.smithi144.stdout:9/555: chown d11/d12/d1b/d22/f3d 6390052 1 2024-09-10T11:29:40.176 INFO:tasks.workunit.client.0.smithi144.stdout:8/362: creat d0/d9/d21/d25/d4f/f6f x:0 0 0 2024-09-10T11:29:40.179 INFO:tasks.workunit.client.0.smithi144.stdout:9/556: creat d11/d12/d1b/d41/d8f/fc8 x:0 0 0 2024-09-10T11:29:40.181 INFO:tasks.workunit.client.0.smithi144.stdout:0/354: sync 2024-09-10T11:29:40.184 INFO:tasks.workunit.client.1.smithi178.stdout:3/930: symlink db/d35/d58/df2/d127/l12b 0 2024-09-10T11:29:40.184 INFO:tasks.workunit.client.1.smithi178.stdout:3/931: readlink db/d26/dcd/lf0 0 2024-09-10T11:29:40.185 INFO:tasks.workunit.client.0.smithi144.stdout:0/355: fsync da/f30 0 2024-09-10T11:29:40.185 INFO:tasks.workunit.client.0.smithi144.stdout:9/557: dread - d11/d12/d1b/f95 zero size 2024-09-10T11:29:40.185 INFO:tasks.workunit.client.0.smithi144.stdout:2/415: dwrite d7/de/d3d/d43/f7b [0,4194304] 0 2024-09-10T11:29:40.185 INFO:tasks.workunit.client.1.smithi178.stdout:3/932: chown db/d20/d3d/d63/da2/d98/l100 40498178 1 2024-09-10T11:29:40.186 INFO:tasks.workunit.client.1.smithi178.stdout:4/772: link d0/d8/d3e/f4c d0/d8/d3e/f10d 0 2024-09-10T11:29:40.188 INFO:tasks.workunit.client.0.smithi144.stdout:9/558: readlink d11/d12/d1b/d41/l57 0 2024-09-10T11:29:40.193 INFO:tasks.workunit.client.1.smithi178.stdout:3/933: getdents db/d20/d3d/d63/da2 0 2024-09-10T11:29:40.194 INFO:tasks.workunit.client.0.smithi144.stdout:0/356: mkdir da/d3a/d4b/d4e/d62/d69 0 2024-09-10T11:29:40.194 INFO:tasks.workunit.client.0.smithi144.stdout:6/508: dwrite d9/dd/d3e/d83/daf/d5b/d91/f51 [4194304,4194304] 0 2024-09-10T11:29:40.197 INFO:tasks.workunit.client.1.smithi178.stdout:3/934: creat db/d20/d3d/d63/f12c x:0 0 0 2024-09-10T11:29:40.199 INFO:tasks.workunit.client.0.smithi144.stdout:2/416: mkdir d7/de/d5f/d89 0 2024-09-10T11:29:40.202 INFO:tasks.workunit.client.0.smithi144.stdout:8/363: dwrite d0/d9/d23/d37/f3a [0,4194304] 0 2024-09-10T11:29:40.209 INFO:tasks.workunit.client.0.smithi144.stdout:9/559: unlink d11/d12/d1b/d41/l75 0 2024-09-10T11:29:40.211 INFO:tasks.workunit.client.0.smithi144.stdout:2/417: creat d7/da/d1e/d21/d54/f8a x:0 0 0 2024-09-10T11:29:40.211 INFO:tasks.workunit.client.0.smithi144.stdout:5/600: dread d4/d20/d24/f63 [0,4194304] 0 2024-09-10T11:29:40.227 INFO:tasks.workunit.client.0.smithi144.stdout:2/418: creat d7/da/d1e/d21/d7e/f8b x:0 0 0 2024-09-10T11:29:40.228 INFO:tasks.workunit.client.0.smithi144.stdout:6/509: getdents d9/dd/d3e/d83/daf/d5b/d91 0 2024-09-10T11:29:40.229 INFO:tasks.workunit.client.0.smithi144.stdout:9/560: creat d11/d12/d1b/fc9 x:0 0 0 2024-09-10T11:29:40.234 INFO:tasks.workunit.client.0.smithi144.stdout:5/601: link d4/d5/d32/d47/l79 d4/lb4 0 2024-09-10T11:29:40.280 INFO:tasks.workunit.client.1.smithi178.stdout:4/773: dwrite d0/d10/fe5 [0,4194304] 0 2024-09-10T11:29:40.284 INFO:tasks.workunit.client.1.smithi178.stdout:4/774: unlink d0/d8/da/d1a/d51/d5d/fc5 0 2024-09-10T11:29:40.289 INFO:tasks.workunit.client.0.smithi144.stdout:6/510: rename d9/dd/d3e/d83/daf/d5b/d91/c55 to d9/dd/d11/cb7 0 2024-09-10T11:29:40.289 INFO:tasks.workunit.client.0.smithi144.stdout:7/524: truncate d5/f8 4263491 0 2024-09-10T11:29:40.290 INFO:tasks.workunit.client.1.smithi178.stdout:4/775: mkdir d0/d8/da/dae/d10e 0 2024-09-10T11:29:40.291 INFO:tasks.workunit.client.0.smithi144.stdout:1/662: truncate d5/df/d4b/f8e 413860 0 2024-09-10T11:29:40.292 INFO:tasks.workunit.client.1.smithi178.stdout:4/776: write d0/d9/d3d/fc6 [197025,93517] 0 2024-09-10T11:29:40.293 INFO:tasks.workunit.client.0.smithi144.stdout:3/567: dwrite d4/d28/d67/d9f/fb5 [0,4194304] 0 2024-09-10T11:29:40.294 INFO:tasks.workunit.client.0.smithi144.stdout:0/357: dread da/f2e [0,4194304] 0 2024-09-10T11:29:40.295 INFO:tasks.workunit.client.1.smithi178.stdout:4/777: mknod d0/d8/df/dfd/c10f 0 2024-09-10T11:29:40.298 INFO:tasks.workunit.client.1.smithi178.stdout:4/778: mknod d0/d9/d3d/c110 0 2024-09-10T11:29:40.299 INFO:tasks.workunit.client.0.smithi144.stdout:9/561: rename d11/d12/d1b/d41/d8f/dab/l85 to d11/d12/d1b/d41/d8f/lca 0 2024-09-10T11:29:40.299 INFO:tasks.workunit.client.0.smithi144.stdout:1/663: rmdir d5/dd/d8f 39 2024-09-10T11:29:40.300 INFO:tasks.workunit.client.1.smithi178.stdout:4/779: write d0/d63/de0/ff8 [317695,22092] 0 2024-09-10T11:29:40.300 INFO:tasks.workunit.client.0.smithi144.stdout:3/568: unlink d4/dd/l11 0 2024-09-10T11:29:40.301 INFO:tasks.workunit.client.0.smithi144.stdout:7/525: mknod d5/dc/d14/d40/d58/caa 0 2024-09-10T11:29:40.301 INFO:tasks.workunit.client.0.smithi144.stdout:5/602: creat d4/d5/d32/daa/fb5 x:0 0 0 2024-09-10T11:29:40.305 INFO:tasks.workunit.client.0.smithi144.stdout:9/562: readlink d11/d12/d1b/l6a 0 2024-09-10T11:29:40.309 INFO:tasks.workunit.client.0.smithi144.stdout:9/563: write d11/d36/d2d/f9a [2367971,126836] 0 2024-09-10T11:29:40.312 INFO:tasks.workunit.client.0.smithi144.stdout:8/364: dwrite d0/d9/d23/f50 [4194304,4194304] 0 2024-09-10T11:29:40.315 INFO:tasks.workunit.client.0.smithi144.stdout:6/511: dwrite d9/dd/d73/fb1 [0,4194304] 0 2024-09-10T11:29:40.317 INFO:tasks.workunit.client.0.smithi144.stdout:3/569: creat d4/d28/d67/d80/d17/d18/d34/fbc x:0 0 0 2024-09-10T11:29:40.318 INFO:tasks.workunit.client.0.smithi144.stdout:7/526: rename d5/dc/d2a/c51 to d5/dc/cab 0 2024-09-10T11:29:40.323 INFO:tasks.workunit.client.0.smithi144.stdout:9/564: rename d11/d12/d1b/d41/d8f/dab/c60 to d11/ccb 0 2024-09-10T11:29:40.326 INFO:tasks.workunit.client.0.smithi144.stdout:3/570: write d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/f7b [1425565,100881] 0 2024-09-10T11:29:40.328 INFO:tasks.workunit.client.0.smithi144.stdout:5/603: dwrite d4/d5/d18/d67/d59/f5e [0,4194304] 0 2024-09-10T11:29:40.330 INFO:tasks.workunit.client.0.smithi144.stdout:5/604: readlink d4/d52/l7e 0 2024-09-10T11:29:40.331 INFO:tasks.workunit.client.0.smithi144.stdout:7/527: unlink d5/dc/d2a/d41/l5b 0 2024-09-10T11:29:40.332 INFO:tasks.workunit.client.1.smithi178.stdout:0/787: sync 2024-09-10T11:29:40.336 INFO:tasks.workunit.client.1.smithi178.stdout:1/807: unlink d3/d9/d98/dac/d3a/d50/db5/f11b 0 2024-09-10T11:29:40.338 INFO:tasks.workunit.client.1.smithi178.stdout:1/808: fsync d3/d9/d21/d4b/d2b/f94 0 2024-09-10T11:29:40.339 INFO:tasks.workunit.client.0.smithi144.stdout:1/664: dwrite d5/f38 [0,4194304] 0 2024-09-10T11:29:40.340 INFO:tasks.workunit.client.0.smithi144.stdout:5/605: getdents d4/d20/d24/da4 0 2024-09-10T11:29:40.340 INFO:tasks.workunit.client.1.smithi178.stdout:3/935: rename db/d26/d40/dcc to db/d12d 0 2024-09-10T11:29:40.341 INFO:tasks.workunit.client.1.smithi178.stdout:0/788: dwrite d0/dc/f2b [0,4194304] 0 2024-09-10T11:29:40.342 INFO:tasks.workunit.client.1.smithi178.stdout:3/936: chown db/d20/d3d/d63/da2/d98/fbc 59344833 1 2024-09-10T11:29:40.342 INFO:tasks.workunit.client.0.smithi144.stdout:3/571: mknod d4/d28/d67/d80/d17/d18/d34/cbd 0 2024-09-10T11:29:40.346 INFO:tasks.workunit.client.1.smithi178.stdout:1/809: dread d3/d9/d98/dac/d3a/d50/d9e/dc2/de2/d118/f113 [0,4194304] 0 2024-09-10T11:29:40.348 INFO:tasks.workunit.client.1.smithi178.stdout:0/789: dwrite d0/d12/d3f/d72/fef [0,4194304] 0 2024-09-10T11:29:40.349 INFO:tasks.workunit.client.0.smithi144.stdout:3/572: truncate d4/d28/d67/d80/d17/d18/d34/f5c 4478749 0 2024-09-10T11:29:40.349 INFO:tasks.workunit.client.0.smithi144.stdout:5/606: mkdir d4/d20/d24/d2d/db6 0 2024-09-10T11:29:40.350 INFO:tasks.workunit.client.0.smithi144.stdout:9/565: rename d11/d12/d1b/d41/d8f/lb7 to d11/d12/d1b/d41/lcc 0 2024-09-10T11:29:40.350 INFO:tasks.workunit.client.1.smithi178.stdout:1/810: truncate d3/d9/d98/dac/d3a/d50/d80/f10f 475424 0 2024-09-10T11:29:40.350 INFO:tasks.workunit.client.0.smithi144.stdout:7/528: creat d5/dc/d2a/fac x:0 0 0 2024-09-10T11:29:40.351 INFO:tasks.workunit.client.1.smithi178.stdout:1/811: truncate d3/d35/fe6 84030 0 2024-09-10T11:29:40.352 INFO:tasks.workunit.client.1.smithi178.stdout:1/812: write d3/d9/d21/f109 [1018924,20993] 0 2024-09-10T11:29:40.355 INFO:tasks.workunit.client.0.smithi144.stdout:5/607: mknod d4/cb7 0 2024-09-10T11:29:40.356 INFO:tasks.workunit.client.0.smithi144.stdout:1/665: dwrite d5/df/d4b/d51/f59 [0,4194304] 0 2024-09-10T11:29:40.358 INFO:tasks.workunit.client.0.smithi144.stdout:7/529: truncate d5/dc/d14/d40/f67 1017667 0 2024-09-10T11:29:40.361 INFO:tasks.workunit.client.1.smithi178.stdout:4/780: dwrite d0/d8/da/d1a/d51/f7a [0,4194304] 0 2024-09-10T11:29:40.364 INFO:tasks.workunit.client.1.smithi178.stdout:3/937: getdents db/d20/d3d/d63 0 2024-09-10T11:29:40.365 INFO:tasks.workunit.client.0.smithi144.stdout:9/566: rmdir d11/d12/d1b/d41/d8f/dab/d64 39 2024-09-10T11:29:40.366 INFO:tasks.workunit.client.0.smithi144.stdout:7/530: rename d5/dc/d12/d2f/d93 to d5/dc/d14/d40/d58/d6a/dad 0 2024-09-10T11:29:40.368 INFO:tasks.workunit.client.1.smithi178.stdout:1/813: dwrite d3/d9/d21/d4b/d2b/dfd/f119 [0,4194304] 0 2024-09-10T11:29:40.370 INFO:tasks.workunit.client.0.smithi144.stdout:3/573: dread d4/d1a/f5f [0,4194304] 0 2024-09-10T11:29:40.370 INFO:tasks.workunit.client.1.smithi178.stdout:4/781: mknod d0/d9/dc/dca/c111 0 2024-09-10T11:29:40.371 INFO:tasks.workunit.client.0.smithi144.stdout:9/567: symlink d11/lcd 0 2024-09-10T11:29:40.374 INFO:tasks.workunit.client.1.smithi178.stdout:4/782: creat d0/d9d/f112 x:0 0 0 2024-09-10T11:29:40.375 INFO:tasks.workunit.client.0.smithi144.stdout:6/512: dread d9/dd/f1d [0,4194304] 0 2024-09-10T11:29:40.375 INFO:tasks.workunit.client.1.smithi178.stdout:4/783: stat d0/d8/df/f49 0 2024-09-10T11:29:40.377 INFO:tasks.workunit.client.0.smithi144.stdout:5/608: dwrite d4/d5/d32/f68 [0,4194304] 0 2024-09-10T11:29:40.377 INFO:tasks.workunit.client.1.smithi178.stdout:1/814: mkdir d3/d35/dc6/d12f 0 2024-09-10T11:29:40.378 INFO:tasks.workunit.client.0.smithi144.stdout:4/522: sync 2024-09-10T11:29:40.379 INFO:tasks.workunit.client.0.smithi144.stdout:6/513: fdatasync d9/dd/d3e/d47/d13/d87/f97 0 2024-09-10T11:29:40.381 INFO:tasks.workunit.client.0.smithi144.stdout:3/574: chown d4/d28/d67/d80/d17/d18/d34/f5c 748577285 1 2024-09-10T11:29:40.382 INFO:tasks.workunit.client.0.smithi144.stdout:9/568: write d11/d12/d1b/d2c/d78/fc1 [1021697,62039] 0 2024-09-10T11:29:40.382 INFO:tasks.workunit.client.1.smithi178.stdout:1/815: dread - d3/d9/d98/dd1/ddc/f95 zero size 2024-09-10T11:29:40.383 INFO:tasks.workunit.client.0.smithi144.stdout:4/523: read - d14/d1e/d22/d34/d3e/d40/fac zero size 2024-09-10T11:29:40.383 INFO:tasks.workunit.client.0.smithi144.stdout:8/365: truncate d0/d9/d1f/d34/f61 1045687 0 2024-09-10T11:29:40.383 INFO:tasks.workunit.client.0.smithi144.stdout:5/609: readlink d4/d5/d18/d51/l98 0 2024-09-10T11:29:40.385 INFO:tasks.workunit.client.0.smithi144.stdout:6/514: fdatasync d9/f2a 0 2024-09-10T11:29:40.386 INFO:tasks.workunit.client.0.smithi144.stdout:5/610: chown d4/d5/d18/d38/d6d/da2 32 1 2024-09-10T11:29:40.386 INFO:tasks.workunit.client.0.smithi144.stdout:6/515: chown d9/dd/d73/c9c 13537398 1 2024-09-10T11:29:40.387 INFO:tasks.workunit.client.1.smithi178.stdout:4/784: dread d0/d8/da/f60 [0,4194304] 0 2024-09-10T11:29:40.389 INFO:tasks.workunit.client.1.smithi178.stdout:0/790: dwrite d0/d12/d3f/f44 [0,4194304] 0 2024-09-10T11:29:40.389 INFO:tasks.workunit.client.0.smithi144.stdout:6/516: write d9/dd/d3e/d83/daf/d5b/f61 [1106002,53196] 0 2024-09-10T11:29:40.391 INFO:tasks.workunit.client.1.smithi178.stdout:4/785: stat d0/d8/da/d1a/d51/d5d 0 2024-09-10T11:29:40.391 INFO:tasks.workunit.client.1.smithi178.stdout:0/791: stat d0/dc/d39/d4c/f15 0 2024-09-10T11:29:40.394 INFO:tasks.workunit.client.1.smithi178.stdout:1/816: dwrite d3/d9/d98/dac/d3a/d6e/fe7 [0,4194304] 0 2024-09-10T11:29:40.395 INFO:tasks.workunit.client.0.smithi144.stdout:8/366: dwrite d0/d9/d23/d26/f30 [4194304,4194304] 0 2024-09-10T11:29:40.397 INFO:tasks.workunit.client.1.smithi178.stdout:0/792: creat d0/dc/dc4/de8/ff6 x:0 0 0 2024-09-10T11:29:40.398 INFO:tasks.workunit.client.1.smithi178.stdout:1/817: write d3/d9/d98/dac/d2a/f108 [959736,75053] 0 2024-09-10T11:29:40.400 INFO:tasks.workunit.client.1.smithi178.stdout:0/793: readlink d0/d12/d3f/l4f 0 2024-09-10T11:29:40.403 INFO:tasks.workunit.client.1.smithi178.stdout:0/794: unlink d0/dc/d39/d4c/fb4 0 2024-09-10T11:29:40.410 INFO:tasks.workunit.client.1.smithi178.stdout:0/795: creat d0/dc/d39/d50/d5e/ff7 x:0 0 0 2024-09-10T11:29:40.413 INFO:tasks.workunit.client.1.smithi178.stdout:0/796: creat d0/dc/d1f/d2d/dac/ff8 x:0 0 0 2024-09-10T11:29:40.414 INFO:tasks.workunit.client.1.smithi178.stdout:1/818: truncate d3/d9/d21/d4b/d2b/f94 2185809 0 2024-09-10T11:29:40.414 INFO:tasks.workunit.client.1.smithi178.stdout:1/819: fdatasync d3/d9/f114 0 2024-09-10T11:29:40.415 INFO:tasks.workunit.client.1.smithi178.stdout:0/797: write d0/dc/d39/d50/f25 [1418834,80872] 0 2024-09-10T11:29:40.415 INFO:tasks.workunit.client.1.smithi178.stdout:1/820: rmdir d3/d9/d98/dd1 39 2024-09-10T11:29:40.417 INFO:tasks.workunit.client.1.smithi178.stdout:1/821: stat d3/d9/d98/dac/d3a/d50/d6c/de9/d121/fc5 0 2024-09-10T11:29:40.419 INFO:tasks.workunit.client.1.smithi178.stdout:1/822: mknod d3/d35/dc6/d12f/c130 0 2024-09-10T11:29:40.421 INFO:tasks.workunit.client.1.smithi178.stdout:0/798: dwrite d0/dc/d1f/f74 [0,4194304] 0 2024-09-10T11:29:40.426 INFO:tasks.workunit.client.1.smithi178.stdout:1/823: rename d3/d9/d98/dac/d3a/d6e/d8a/f45 to d3/d9/d98/dac/d3a/d50/d6c/de9/d121/f131 0 2024-09-10T11:29:40.428 INFO:tasks.workunit.client.1.smithi178.stdout:1/824: symlink d3/d9/d21/d4b/d2b/dfd/l132 0 2024-09-10T11:29:40.430 INFO:tasks.workunit.client.1.smithi178.stdout:1/825: mkdir d3/d9/d21/d133 0 2024-09-10T11:29:40.438 INFO:tasks.workunit.client.1.smithi178.stdout:0/799: truncate d0/dc/d1f/f74 902251 0 2024-09-10T11:29:40.439 INFO:tasks.workunit.client.1.smithi178.stdout:0/800: mkdir d0/dc/d1f/d2d/df9 0 2024-09-10T11:29:40.442 INFO:tasks.workunit.client.1.smithi178.stdout:0/801: rename d0/db1/ccc to d0/dc/d39/da4/dce/cfa 0 2024-09-10T11:29:40.444 INFO:tasks.workunit.client.1.smithi178.stdout:0/802: creat d0/d2c/d89/da1/ffb x:0 0 0 2024-09-10T11:29:40.450 INFO:tasks.workunit.client.1.smithi178.stdout:1/826: dread d3/d9/d21/d4b/f17 [0,4194304] 0 2024-09-10T11:29:40.452 INFO:tasks.workunit.client.1.smithi178.stdout:1/827: truncate d3/d9/d98/dac/ff0 120900 0 2024-09-10T11:29:40.455 INFO:tasks.workunit.client.1.smithi178.stdout:1/828: symlink d3/d9/d98/dac/d3a/d50/d9e/dc2/de2/d118/db8/l134 0 2024-09-10T11:29:40.456 INFO:tasks.workunit.client.1.smithi178.stdout:0/803: dwrite d0/d12/f87 [0,4194304] 0 2024-09-10T11:29:40.457 INFO:tasks.workunit.client.1.smithi178.stdout:0/804: readlink d0/dc/d39/d50/l76 0 2024-09-10T11:29:40.459 INFO:tasks.workunit.client.1.smithi178.stdout:1/829: rmdir d3/d9/d98/dac/d3a/d50/d9e/dc2/de2/d118 39 2024-09-10T11:29:40.460 INFO:tasks.workunit.client.1.smithi178.stdout:1/830: dread - d3/d9/f12b zero size 2024-09-10T11:29:40.462 INFO:tasks.workunit.client.1.smithi178.stdout:0/805: unlink d0/d12/c18 0 2024-09-10T11:29:40.464 INFO:tasks.workunit.client.1.smithi178.stdout:1/831: rename d3/d9/d1a/d48/f66 to d3/d9/d98/dac/d3a/d6e/f135 0 2024-09-10T11:29:40.468 INFO:tasks.workunit.client.1.smithi178.stdout:1/832: creat d3/f136 x:0 0 0 2024-09-10T11:29:40.470 INFO:tasks.workunit.client.1.smithi178.stdout:0/806: dwrite d0/dc/d39/da4/dce/ddc/fdb [0,4194304] 0 2024-09-10T11:29:40.481 INFO:tasks.workunit.client.1.smithi178.stdout:1/833: rename d3/d9/d98/dac/d8d/f127 to d3/d9/d98/dd1/ddc/f137 0 2024-09-10T11:29:40.481 INFO:tasks.workunit.client.1.smithi178.stdout:0/807: mknod d0/dc/cfc 0 2024-09-10T11:29:40.481 INFO:tasks.workunit.client.1.smithi178.stdout:0/808: write d0/dc/dc4/de8/ff6 [703586,25913] 0 2024-09-10T11:29:40.486 INFO:tasks.workunit.client.0.smithi144.stdout:4/524: creat d14/d1e/d65/fbd x:0 0 0 2024-09-10T11:29:40.487 INFO:tasks.workunit.client.0.smithi144.stdout:4/525: dread - d14/d67/d69/d90/f9d zero size 2024-09-10T11:29:40.489 INFO:tasks.workunit.client.1.smithi178.stdout:1/834: write d3/d35/fa9 [988236,105644] 0 2024-09-10T11:29:40.490 INFO:tasks.workunit.client.1.smithi178.stdout:0/809: truncate d0/d12/d3f/f44 2255609 0 2024-09-10T11:29:40.492 INFO:tasks.workunit.client.0.smithi144.stdout:4/526: dread d14/d1e/d22/d34/d3e/d40/d4b/f57 [0,4194304] 0 2024-09-10T11:29:40.494 INFO:tasks.workunit.client.1.smithi178.stdout:1/835: rename d3/d35/ddd/f117 to d3/d9/d98/dac/d3a/d50/d9e/f138 0 2024-09-10T11:29:40.496 INFO:tasks.workunit.client.1.smithi178.stdout:0/810: dwrite d0/dc/d39/da4/dce/ff5 [0,4194304] 0 2024-09-10T11:29:40.500 INFO:tasks.workunit.client.1.smithi178.stdout:0/811: rename d0/d12/c4b to d0/dc/d1f/dae/cfd 0 2024-09-10T11:29:40.503 INFO:tasks.workunit.client.1.smithi178.stdout:0/812: symlink d0/dc/d1f/lfe 0 2024-09-10T11:29:40.510 INFO:tasks.workunit.client.1.smithi178.stdout:1/836: dread d3/d9/d21/d4b/f5a [0,4194304] 0 2024-09-10T11:29:40.511 INFO:tasks.workunit.client.1.smithi178.stdout:1/837: write d3/d9/d98/dac/d3a/fee [5556127,97945] 0 2024-09-10T11:29:40.516 INFO:tasks.workunit.client.1.smithi178.stdout:1/838: getdents d3/d9/d21/d4b/d2b/dfd 0 2024-09-10T11:29:40.516 INFO:tasks.workunit.client.1.smithi178.stdout:0/813: dwrite d0/dc/d39/d4c/f3a [0,4194304] 0 2024-09-10T11:29:40.517 INFO:tasks.workunit.client.1.smithi178.stdout:0/814: chown d0/dc/d1f/dae 2 1 2024-09-10T11:29:40.517 INFO:tasks.workunit.client.1.smithi178.stdout:1/839: write d3/d9/d98/dd1/f6a [20658,121499] 0 2024-09-10T11:29:40.521 INFO:tasks.workunit.client.1.smithi178.stdout:0/815: fdatasync d0/f2e 0 2024-09-10T11:29:40.525 INFO:tasks.workunit.client.1.smithi178.stdout:0/816: creat d0/dc/d39/d75/d94/fff x:0 0 0 2024-09-10T11:29:40.526 INFO:tasks.workunit.client.1.smithi178.stdout:1/840: dwrite d3/da0/fc1 [0,4194304] 0 2024-09-10T11:29:40.531 INFO:tasks.workunit.client.1.smithi178.stdout:0/817: rename d0/dc/d1f/dae/fbc to d0/d12/d3f/d72/df0/f100 0 2024-09-10T11:29:40.532 INFO:tasks.workunit.client.1.smithi178.stdout:6/872: sync 2024-09-10T11:29:40.535 INFO:tasks.workunit.client.1.smithi178.stdout:1/841: symlink d3/d9/d98/dac/d3a/d50/d9e/dc2/de2/d118/l139 0 2024-09-10T11:29:40.537 INFO:tasks.workunit.client.1.smithi178.stdout:0/818: dread - d0/d12/d3f/d72/df0/f100 zero size 2024-09-10T11:29:40.538 INFO:tasks.workunit.client.0.smithi144.stdout:6/517: truncate d9/dd/d3e/d47/d13/f31 3392841 0 2024-09-10T11:29:40.539 INFO:tasks.workunit.client.1.smithi178.stdout:6/873: mkdir d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d13f 0 2024-09-10T11:29:40.540 INFO:tasks.workunit.client.0.smithi144.stdout:8/367: unlink d0/d9/d21/d25/f60 0 2024-09-10T11:29:40.540 INFO:tasks.workunit.client.0.smithi144.stdout:7/531: write d5/dc/d12/f52 [8401,37717] 0 2024-09-10T11:29:40.542 INFO:tasks.workunit.client.0.smithi144.stdout:7/532: read d5/dc/d2a/d60/f91 [4977290,27082] 0 2024-09-10T11:29:40.543 INFO:tasks.workunit.client.1.smithi178.stdout:0/819: rename d0/d2c/cd7 to d0/dc/d39/da4/c101 0 2024-09-10T11:29:40.544 INFO:tasks.workunit.client.0.smithi144.stdout:7/533: chown d5/dc/d11/c76 676214 1 2024-09-10T11:29:40.545 INFO:tasks.workunit.client.1.smithi178.stdout:6/874: mknod d3/d6e/de9/c140 0 2024-09-10T11:29:40.545 INFO:tasks.workunit.client.0.smithi144.stdout:7/534: chown d5/dc/d12/d2f/f4d 1048651174 1 2024-09-10T11:29:40.545 INFO:tasks.workunit.client.0.smithi144.stdout:4/527: rename d14/d1e/d22/d2b to d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe 0 2024-09-10T11:29:40.546 INFO:tasks.workunit.client.1.smithi178.stdout:0/820: fdatasync d0/d12/d3f/fa5 0 2024-09-10T11:29:40.549 INFO:tasks.workunit.client.0.smithi144.stdout:5/611: mkdir d4/d5/d18/d67/db0/db8 0 2024-09-10T11:29:40.550 INFO:tasks.workunit.client.0.smithi144.stdout:3/575: dwrite f2 [0,4194304] 0 2024-09-10T11:29:40.555 INFO:tasks.workunit.client.0.smithi144.stdout:6/518: creat d9/dd/d11/d1b/d2f/fb8 x:0 0 0 2024-09-10T11:29:40.557 INFO:tasks.workunit.client.1.smithi178.stdout:1/842: write d3/d9/d21/d4b/d2b/f3d [1029729,94607] 0 2024-09-10T11:29:40.557 INFO:tasks.workunit.client.0.smithi144.stdout:5/612: dwrite d4/d5/d18/d67/d59/f6c [0,4194304] 0 2024-09-10T11:29:40.560 INFO:tasks.workunit.client.1.smithi178.stdout:6/875: dread d3/d74/dc9/da6/d11e/f52 [0,4194304] 0 2024-09-10T11:29:40.561 INFO:tasks.workunit.client.0.smithi144.stdout:4/528: rmdir d14/d67/d92 39 2024-09-10T11:29:40.562 INFO:tasks.workunit.client.0.smithi144.stdout:4/529: chown d14/d1e/d22/d34/d3e/f97 19 1 2024-09-10T11:29:40.562 INFO:tasks.workunit.client.1.smithi178.stdout:1/843: unlink d3/d9/d21/d4b/d2b/dfd/f119 0 2024-09-10T11:29:40.566 INFO:tasks.workunit.client.0.smithi144.stdout:9/569: truncate d11/d36/d2d/f8d 320989 0 2024-09-10T11:29:40.568 INFO:tasks.workunit.client.1.smithi178.stdout:0/821: dwrite d0/dc/d1f/fad [0,4194304] 0 2024-09-10T11:29:40.568 INFO:tasks.workunit.client.1.smithi178.stdout:6/876: mkdir d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d13f/d141 0 2024-09-10T11:29:40.569 INFO:tasks.workunit.client.0.smithi144.stdout:7/535: dwrite d5/dc/f10 [0,4194304] 0 2024-09-10T11:29:40.569 INFO:tasks.workunit.client.1.smithi178.stdout:0/822: chown d0/d2c/d33/l6d 85758989 1 2024-09-10T11:29:40.572 INFO:tasks.workunit.client.1.smithi178.stdout:1/844: dread d3/d35/f74 [0,4194304] 0 2024-09-10T11:29:40.572 INFO:tasks.workunit.client.0.smithi144.stdout:6/519: creat d9/dd/d3e/d83/daf/d5b/d91/fb9 x:0 0 0 2024-09-10T11:29:40.573 INFO:tasks.workunit.client.0.smithi144.stdout:2/419: sync 2024-09-10T11:29:40.573 INFO:tasks.workunit.client.0.smithi144.stdout:1/666: sync 2024-09-10T11:29:40.573 INFO:tasks.workunit.client.0.smithi144.stdout:0/358: sync 2024-09-10T11:29:40.578 INFO:tasks.workunit.client.1.smithi178.stdout:0/823: mknod d0/d12/d3f/d72/df0/c102 0 2024-09-10T11:29:40.578 INFO:tasks.workunit.client.0.smithi144.stdout:4/530: symlink d14/d67/d84/lbf 0 2024-09-10T11:29:40.579 INFO:tasks.workunit.client.0.smithi144.stdout:0/359: fsync da/d3a/d4b/d4e/f60 0 2024-09-10T11:29:40.579 INFO:tasks.workunit.client.0.smithi144.stdout:1/667: fdatasync d5/dd/d64/f9a 0 2024-09-10T11:29:40.579 INFO:tasks.workunit.client.0.smithi144.stdout:3/576: dread - d4/dd/f6f zero size 2024-09-10T11:29:40.580 INFO:tasks.workunit.client.1.smithi178.stdout:0/824: write d0/dc/d39/d7c/fa9 [1690675,112665] 0 2024-09-10T11:29:40.585 INFO:tasks.workunit.client.0.smithi144.stdout:6/520: creat d9/dd/d3e/d47/d13/d60/fba x:0 0 0 2024-09-10T11:29:40.585 INFO:tasks.workunit.client.0.smithi144.stdout:4/531: unlink fa 0 2024-09-10T11:29:40.587 INFO:tasks.workunit.client.1.smithi178.stdout:0/825: creat d0/dc/d39/d4c/f103 x:0 0 0 2024-09-10T11:29:40.587 INFO:tasks.workunit.client.0.smithi144.stdout:3/577: write d4/f58 [980546,24835] 0 2024-09-10T11:29:40.589 INFO:tasks.workunit.client.0.smithi144.stdout:0/360: truncate da/f32 2837939 0 2024-09-10T11:29:40.589 INFO:tasks.workunit.client.0.smithi144.stdout:9/570: fdatasync d11/d36/d2d/f8d 0 2024-09-10T11:29:40.590 INFO:tasks.workunit.client.0.smithi144.stdout:5/613: link d4/d5/d18/d51/lae d4/d5/d18/d38/d8d/lb9 0 2024-09-10T11:29:40.592 INFO:tasks.workunit.client.1.smithi178.stdout:0/826: link d0/d2c/d33/c48 d0/dc/d1f/dae/c104 0 2024-09-10T11:29:40.594 INFO:tasks.workunit.client.0.smithi144.stdout:7/536: creat d5/dc/fae x:0 0 0 2024-09-10T11:29:40.594 INFO:tasks.workunit.client.0.smithi144.stdout:2/420: dwrite d7/de/d36/d45/f69 [0,4194304] 0 2024-09-10T11:29:40.595 INFO:tasks.workunit.client.0.smithi144.stdout:4/532: rmdir d14/d1e/d65/d8a/d8f 39 2024-09-10T11:29:40.596 INFO:tasks.workunit.client.1.smithi178.stdout:0/827: creat d0/dc/d39/d50/f105 x:0 0 0 2024-09-10T11:29:40.598 INFO:tasks.workunit.client.1.smithi178.stdout:0/828: chown d0/d12/d3f/c5c 0 1 2024-09-10T11:29:40.599 INFO:tasks.workunit.client.1.smithi178.stdout:0/829: chown d0/d12/d3f/d72/c79 2599 1 2024-09-10T11:29:40.599 INFO:tasks.workunit.client.1.smithi178.stdout:6/877: truncate d3/d24/f30 4357396 0 2024-09-10T11:29:40.601 INFO:tasks.workunit.client.0.smithi144.stdout:3/578: dread - d4/d28/d67/d80/d17/d18/d34/d65/d75/fb0 zero size 2024-09-10T11:29:40.601 INFO:tasks.workunit.client.0.smithi144.stdout:8/368: truncate d0/d9/d23/d26/f4a 2151962 0 2024-09-10T11:29:40.601 INFO:tasks.workunit.client.0.smithi144.stdout:5/614: mknod d4/d20/d24/d6f/d84/cba 0 2024-09-10T11:29:40.602 INFO:tasks.workunit.client.0.smithi144.stdout:4/533: chown d14/d1e/d22/d34/d3e/d40/d4b/f57 60048 1 2024-09-10T11:29:40.602 INFO:tasks.workunit.client.0.smithi144.stdout:1/668: dwrite d5/f15 [0,4194304] 0 2024-09-10T11:29:40.605 INFO:tasks.workunit.client.0.smithi144.stdout:4/534: fdatasync d14/d1e/f50 0 2024-09-10T11:29:40.611 INFO:tasks.workunit.client.0.smithi144.stdout:6/521: dwrite d9/dd/d3e/d83/daf/d5b/d91/fb5 [0,4194304] 0 2024-09-10T11:29:40.614 INFO:tasks.workunit.client.0.smithi144.stdout:9/571: mknod d11/d12/d1b/d41/d8f/dc0/cce 0 2024-09-10T11:29:40.615 INFO:tasks.workunit.client.0.smithi144.stdout:3/579: write d4/dd/d21/f62 [1215484,36098] 0 2024-09-10T11:29:40.616 INFO:tasks.workunit.client.0.smithi144.stdout:8/369: creat d0/d9/d21/f70 x:0 0 0 2024-09-10T11:29:40.616 INFO:tasks.workunit.client.0.smithi144.stdout:3/580: readlink d4/dd/d21/d2f/l99 0 2024-09-10T11:29:40.616 INFO:tasks.workunit.client.0.smithi144.stdout:1/669: symlink d5/df/d4b/d51/ldc 0 2024-09-10T11:29:40.617 INFO:tasks.workunit.client.1.smithi178.stdout:0/830: dwrite d0/d12/d3f/d72/df0/f100 [0,4194304] 0 2024-09-10T11:29:40.617 INFO:tasks.workunit.client.0.smithi144.stdout:9/572: fdatasync d11/d12/d1b/d41/d8f/f9d 0 2024-09-10T11:29:40.619 INFO:tasks.workunit.client.0.smithi144.stdout:0/361: rename da/c38 to da/c6a 0 2024-09-10T11:29:40.619 INFO:tasks.workunit.client.1.smithi178.stdout:6/878: dwrite d3/d7/d8/d15/d1c/d2a/d18/d8e/fa4 [0,4194304] 0 2024-09-10T11:29:40.622 INFO:tasks.workunit.client.0.smithi144.stdout:2/421: link d7/de/f2b d7/d42/d64/d76/f8c 0 2024-09-10T11:29:40.632 INFO:tasks.workunit.client.0.smithi144.stdout:6/522: dread d9/dd/d3e/f40 [0,4194304] 0 2024-09-10T11:29:40.632 INFO:tasks.workunit.client.0.smithi144.stdout:6/523: chown d9/ca6 1021680144 1 2024-09-10T11:29:40.633 INFO:tasks.workunit.client.1.smithi178.stdout:6/879: dread d3/d7/d8/d72/fb3 [0,4194304] 0 2024-09-10T11:29:40.634 INFO:tasks.workunit.client.0.smithi144.stdout:6/524: chown d9/dd/d3e/d83/daf/d5b/f61 11066 1 2024-09-10T11:29:40.635 INFO:tasks.workunit.client.0.smithi144.stdout:9/573: mknod d11/d12/d1b/d2c/ccf 0 2024-09-10T11:29:40.637 INFO:tasks.workunit.client.0.smithi144.stdout:3/581: symlink d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/lbe 0 2024-09-10T11:29:40.638 INFO:tasks.workunit.client.0.smithi144.stdout:0/362: mkdir da/d3a/d51/d5f/d6b 0 2024-09-10T11:29:40.639 INFO:tasks.workunit.client.1.smithi178.stdout:6/880: truncate d3/d7/d8/d15/d1c/d2a/d18/f43 769004 0 2024-09-10T11:29:40.639 INFO:tasks.workunit.client.0.smithi144.stdout:2/422: mkdir d7/d42/d64/d8d 0 2024-09-10T11:29:40.641 INFO:tasks.workunit.client.0.smithi144.stdout:5/615: creat d4/d5/fbb x:0 0 0 2024-09-10T11:29:40.643 INFO:tasks.workunit.client.0.smithi144.stdout:7/537: getdents d5/dc/d2a 0 2024-09-10T11:29:40.644 INFO:tasks.workunit.client.1.smithi178.stdout:6/881: getdents d3/d74/d91/daa/dc2/dcf 0 2024-09-10T11:29:40.644 INFO:tasks.workunit.client.0.smithi144.stdout:6/525: truncate d9/d1f/f37 76980 0 2024-09-10T11:29:40.645 INFO:tasks.workunit.client.0.smithi144.stdout:7/538: stat d5/dc/d12/l32 0 2024-09-10T11:29:40.645 INFO:tasks.workunit.client.0.smithi144.stdout:8/370: dread d0/d9/d1f/d34/f61 [0,4194304] 0 2024-09-10T11:29:40.646 INFO:tasks.workunit.client.0.smithi144.stdout:9/574: chown d11/d12/d1b/d2c/d78/c89 2905 1 2024-09-10T11:29:40.646 INFO:tasks.workunit.client.0.smithi144.stdout:3/582: chown d4/dd/d21/f86 106 1 2024-09-10T11:29:40.647 INFO:tasks.workunit.client.1.smithi178.stdout:6/882: truncate d3/d74/d91/daa/dc2/d97/fb4 940998 0 2024-09-10T11:29:40.647 INFO:tasks.workunit.client.1.smithi178.stdout:6/883: chown d3/d74 939895604 1 2024-09-10T11:29:40.648 INFO:tasks.workunit.client.0.smithi144.stdout:2/423: unlink d7/de/f2b 0 2024-09-10T11:29:40.648 INFO:tasks.workunit.client.1.smithi178.stdout:0/831: dread d0/d12/f2f [0,4194304] 0 2024-09-10T11:29:40.651 INFO:tasks.workunit.client.0.smithi144.stdout:0/363: mkdir da/d3a/d4b/d61/d6c 0 2024-09-10T11:29:40.652 INFO:tasks.workunit.client.0.smithi144.stdout:0/364: stat da/d3a/f3b 0 2024-09-10T11:29:40.654 INFO:tasks.workunit.client.0.smithi144.stdout:8/371: creat d0/d9/d21/f71 x:0 0 0 2024-09-10T11:29:40.654 INFO:tasks.workunit.client.1.smithi178.stdout:6/884: link d3/d7/d8/d15/d1c/f1d d3/d7/d8/d72/f142 0 2024-09-10T11:29:40.656 INFO:tasks.workunit.client.0.smithi144.stdout:8/372: dread - d0/d9/d21/d25/d4f/f6f zero size 2024-09-10T11:29:40.656 INFO:tasks.workunit.client.0.smithi144.stdout:1/670: dread d5/dd/f1d [0,4194304] 0 2024-09-10T11:29:40.657 INFO:tasks.workunit.client.0.smithi144.stdout:3/583: creat d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93/fbf x:0 0 0 2024-09-10T11:29:40.659 INFO:tasks.workunit.client.0.smithi144.stdout:3/584: fdatasync d4/d28/d67/d80/d17/d18/f2e 0 2024-09-10T11:29:40.660 INFO:tasks.workunit.client.0.smithi144.stdout:7/539: symlink d5/dc/d11/d18/laf 0 2024-09-10T11:29:40.661 INFO:tasks.workunit.client.0.smithi144.stdout:7/540: write d5/f37 [4521998,45343] 0 2024-09-10T11:29:40.661 INFO:tasks.workunit.client.0.smithi144.stdout:0/365: rename da/d49/f4c to da/d3a/d51/d5f/f6d 0 2024-09-10T11:29:40.663 INFO:tasks.workunit.client.1.smithi178.stdout:6/885: chown d3/d74/dc9/f93 485347 1 2024-09-10T11:29:40.663 INFO:tasks.workunit.client.1.smithi178.stdout:0/832: dwrite d0/d12/d3f/d72/f82 [0,4194304] 0 2024-09-10T11:29:40.663 INFO:tasks.workunit.client.0.smithi144.stdout:8/373: mknod d0/d9/d23/d37/c72 0 2024-09-10T11:29:40.664 INFO:tasks.workunit.client.1.smithi178.stdout:6/886: readlink d3/d6e/d87/lae 0 2024-09-10T11:29:40.667 INFO:tasks.workunit.client.0.smithi144.stdout:1/671: write d5/d2a/f2d [681075,67817] 0 2024-09-10T11:29:40.672 INFO:tasks.workunit.client.0.smithi144.stdout:9/575: write d11/d12/d1b/d22/f74 [604243,130306] 0 2024-09-10T11:29:40.672 INFO:tasks.workunit.client.0.smithi144.stdout:5/616: truncate d4/d5/d18/d67/d44/f93 2912181 0 2024-09-10T11:29:40.673 INFO:tasks.workunit.client.0.smithi144.stdout:3/585: chown d4/dd/f1b 0 1 2024-09-10T11:29:40.676 INFO:tasks.workunit.client.0.smithi144.stdout:8/374: creat d0/d9/d21/f73 x:0 0 0 2024-09-10T11:29:40.678 INFO:tasks.workunit.client.0.smithi144.stdout:8/375: write d0/d9/d23/d26/f36 [7471843,50477] 0 2024-09-10T11:29:40.679 INFO:tasks.workunit.client.0.smithi144.stdout:6/526: getdents d9/d1f 0 2024-09-10T11:29:40.680 INFO:tasks.workunit.client.0.smithi144.stdout:2/424: truncate d7/d29/f3f 2300172 0 2024-09-10T11:29:40.680 INFO:tasks.workunit.client.0.smithi144.stdout:0/366: dread f5 [0,4194304] 0 2024-09-10T11:29:40.684 INFO:tasks.workunit.client.0.smithi144.stdout:5/617: mknod d4/d5/d18/d38/d6d/d8f/cbc 0 2024-09-10T11:29:40.685 INFO:tasks.workunit.client.0.smithi144.stdout:2/425: chown d7/de/d36/d45/d47/d6c/d7c 14055397 1 2024-09-10T11:29:40.685 INFO:tasks.workunit.client.0.smithi144.stdout:7/541: write d5/dc/d14/d40/d5e/d68/f89 [833422,18493] 0 2024-09-10T11:29:40.686 INFO:tasks.workunit.client.0.smithi144.stdout:0/367: symlink da/d3a/d4b/d61/l6e 0 2024-09-10T11:29:40.692 INFO:tasks.workunit.client.0.smithi144.stdout:5/618: mknod d4/d5/d56/cbd 0 2024-09-10T11:29:40.694 INFO:tasks.workunit.client.0.smithi144.stdout:2/426: rmdir d7/da/d5b 39 2024-09-10T11:29:40.697 INFO:tasks.workunit.client.0.smithi144.stdout:3/586: rename d4/d28/d67/d80/d17/d18/d34/cbd to d4/cc0 0 2024-09-10T11:29:40.698 INFO:tasks.workunit.client.0.smithi144.stdout:7/542: rmdir d5/dc/d14/d40/d5e 39 2024-09-10T11:29:40.699 INFO:tasks.workunit.client.0.smithi144.stdout:5/619: dwrite d4/d20/d24/f92 [4194304,4194304] 0 2024-09-10T11:29:40.699 INFO:tasks.workunit.client.1.smithi178.stdout:7/785: sync 2024-09-10T11:29:40.699 INFO:tasks.workunit.client.1.smithi178.stdout:9/920: sync 2024-09-10T11:29:40.700 INFO:tasks.workunit.client.1.smithi178.stdout:8/963: sync 2024-09-10T11:29:40.700 INFO:tasks.workunit.client.0.smithi144.stdout:7/543: write d5/dc/f56 [5232729,122706] 0 2024-09-10T11:29:40.700 INFO:tasks.workunit.client.0.smithi144.stdout:3/587: truncate d4/d28/d67/d9f/fbb 142081 0 2024-09-10T11:29:40.700 INFO:tasks.workunit.client.1.smithi178.stdout:8/964: fsync d5/d30/f127 0 2024-09-10T11:29:40.701 INFO:tasks.workunit.client.0.smithi144.stdout:8/376: link d0/d1b/d47/c57 d0/d9/d1f/d34/c74 0 2024-09-10T11:29:40.704 INFO:tasks.workunit.client.0.smithi144.stdout:2/427: mkdir d7/de/d36/d45/d8e 0 2024-09-10T11:29:40.705 INFO:tasks.workunit.client.0.smithi144.stdout:6/527: rename d9/dd/d3e/d47/d13/f1e to d9/d1f/fbb 0 2024-09-10T11:29:40.707 INFO:tasks.workunit.client.0.smithi144.stdout:0/368: mkdir da/d6f 0 2024-09-10T11:29:40.711 INFO:tasks.workunit.client.1.smithi178.stdout:9/921: dread d2/d9b/df8/d2d/dab/d118/d56/de1/ff5 [0,4194304] 0 2024-09-10T11:29:40.712 INFO:tasks.workunit.client.0.smithi144.stdout:2/428: mkdir d7/d29/d8f 0 2024-09-10T11:29:40.713 INFO:tasks.workunit.client.0.smithi144.stdout:6/528: mknod d9/dd/d11/d1b/d2f/d7f/d82/cbc 0 2024-09-10T11:29:40.714 INFO:tasks.workunit.client.0.smithi144.stdout:6/529: read - d9/dd/d3e/d83/daf/d5b/fa4 zero size 2024-09-10T11:29:40.714 INFO:tasks.workunit.client.0.smithi144.stdout:0/369: creat da/d3a/d4b/d4e/d62/f70 x:0 0 0 2024-09-10T11:29:40.715 INFO:tasks.workunit.client.0.smithi144.stdout:0/370: readlink da/d3a/d51/l57 0 2024-09-10T11:29:40.715 INFO:tasks.workunit.client.0.smithi144.stdout:3/588: unlink d4/dd/l8f 0 2024-09-10T11:29:40.716 INFO:tasks.workunit.client.0.smithi144.stdout:0/371: chown da/f21 0 1 2024-09-10T11:29:40.716 INFO:tasks.workunit.client.0.smithi144.stdout:0/372: chown f0 25 1 2024-09-10T11:29:40.718 INFO:tasks.workunit.client.0.smithi144.stdout:6/530: mknod d9/dd/d3e/d83/daf/d5b/d91/cbd 0 2024-09-10T11:29:40.719 INFO:tasks.workunit.client.0.smithi144.stdout:7/544: creat d5/dc/d14/d40/fb0 x:0 0 0 2024-09-10T11:29:40.726 INFO:tasks.workunit.client.1.smithi178.stdout:1/845: sync 2024-09-10T11:29:40.726 INFO:tasks.workunit.client.1.smithi178.stdout:4/786: sync 2024-09-10T11:29:40.726 INFO:tasks.workunit.client.1.smithi178.stdout:3/938: sync 2024-09-10T11:29:40.727 INFO:tasks.workunit.client.1.smithi178.stdout:1/846: write d3/f1c [9198919,128994] 0 2024-09-10T11:29:40.728 INFO:tasks.workunit.client.1.smithi178.stdout:1/847: write d3/d9/d1a/fdb [7822340,68371] 0 2024-09-10T11:29:40.729 INFO:tasks.workunit.client.0.smithi144.stdout:7/545: dwrite d5/dc/d14/d40/d58/d6a/fa2 [0,4194304] 0 2024-09-10T11:29:40.741 INFO:tasks.workunit.client.0.smithi144.stdout:2/429: dread d7/d29/f4b [0,4194304] 0 2024-09-10T11:29:40.742 INFO:tasks.workunit.client.0.smithi144.stdout:8/377: dwrite d0/f13 [0,4194304] 0 2024-09-10T11:29:40.742 INFO:tasks.workunit.client.0.smithi144.stdout:3/589: dwrite d4/d28/f3f [0,4194304] 0 2024-09-10T11:29:40.743 INFO:tasks.workunit.client.0.smithi144.stdout:2/430: chown d7/f8 2003 1 2024-09-10T11:29:40.746 INFO:tasks.workunit.client.0.smithi144.stdout:0/373: dwrite da/d3a/d51/d5f/f6d [0,4194304] 0 2024-09-10T11:29:40.746 INFO:tasks.workunit.client.0.smithi144.stdout:3/590: chown d4/d28/d67/d80/d17/d18/d34/db3 1 1 2024-09-10T11:29:40.754 INFO:tasks.workunit.client.0.smithi144.stdout:2/431: rmdir d7/de/d5f/d89 0 2024-09-10T11:29:40.756 INFO:tasks.workunit.client.0.smithi144.stdout:8/378: dwrite d0/d9/d21/f70 [0,4194304] 0 2024-09-10T11:29:40.758 INFO:tasks.workunit.client.0.smithi144.stdout:3/591: mkdir d4/d28/dc1 0 2024-09-10T11:29:40.758 INFO:tasks.workunit.client.0.smithi144.stdout:0/374: creat da/d3a/d51/d5f/d6b/f71 x:0 0 0 2024-09-10T11:29:40.765 INFO:tasks.workunit.client.0.smithi144.stdout:0/375: creat da/d3a/d4b/d61/f72 x:0 0 0 2024-09-10T11:29:40.768 INFO:tasks.workunit.client.0.smithi144.stdout:0/376: dread - da/d44/f65 zero size 2024-09-10T11:29:40.768 INFO:tasks.workunit.client.0.smithi144.stdout:2/432: dwrite d7/da/f17 [0,4194304] 0 2024-09-10T11:29:40.770 INFO:tasks.workunit.client.0.smithi144.stdout:3/592: creat d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93/d98/d9a/fc2 x:0 0 0 2024-09-10T11:29:40.771 INFO:tasks.workunit.client.0.smithi144.stdout:7/546: dread d5/dc/d11/f26 [0,4194304] 0 2024-09-10T11:29:40.771 INFO:tasks.workunit.client.1.smithi178.stdout:7/786: symlink d6/d10/d1d/dca/l116 0 2024-09-10T11:29:40.772 INFO:tasks.workunit.client.0.smithi144.stdout:8/379: dread d0/d9/d23/d37/f3a [0,4194304] 0 2024-09-10T11:29:40.773 INFO:tasks.workunit.client.1.smithi178.stdout:9/922: mknod d2/d9b/df8/d77/c122 0 2024-09-10T11:29:40.775 INFO:tasks.workunit.client.0.smithi144.stdout:8/380: truncate d0/fc 9258257 0 2024-09-10T11:29:40.776 INFO:tasks.workunit.client.1.smithi178.stdout:3/939: symlink db/d20/d3e/d6b/dc0/ddf/dcb/l12e 0 2024-09-10T11:29:40.780 INFO:tasks.workunit.client.0.smithi144.stdout:2/433: mknod d7/de/d3d/d43/c90 0 2024-09-10T11:29:40.780 INFO:tasks.workunit.client.0.smithi144.stdout:0/377: write da/f21 [186801,99106] 0 2024-09-10T11:29:40.780 INFO:tasks.workunit.client.0.smithi144.stdout:3/593: read d4/dd/d21/f41 [52525,10288] 0 2024-09-10T11:29:40.781 INFO:tasks.workunit.client.1.smithi178.stdout:4/787: creat d0/d8/da/d1a/dff/f113 x:0 0 0 2024-09-10T11:29:40.783 INFO:tasks.workunit.client.1.smithi178.stdout:3/940: mkdir db/d36/d12f 0 2024-09-10T11:29:40.784 INFO:tasks.workunit.client.1.smithi178.stdout:7/787: dwrite d6/f30 [0,4194304] 0 2024-09-10T11:29:40.786 INFO:tasks.workunit.client.1.smithi178.stdout:6/887: getdents d3/d7/d8/d19 0 2024-09-10T11:29:40.786 INFO:tasks.workunit.client.0.smithi144.stdout:8/381: symlink d0/d9/d21/d25/d4f/l75 0 2024-09-10T11:29:40.786 INFO:tasks.workunit.client.0.smithi144.stdout:7/547: dwrite d5/dc/d12/f52 [4194304,4194304] 0 2024-09-10T11:29:40.786 INFO:tasks.workunit.client.1.smithi178.stdout:3/941: fsync db/d35/d3a/d9f/fb8 0 2024-09-10T11:29:40.789 INFO:tasks.workunit.client.1.smithi178.stdout:4/788: creat d0/d8/f114 x:0 0 0 2024-09-10T11:29:40.789 INFO:tasks.workunit.client.0.smithi144.stdout:2/434: symlink d7/da/d1e/d21/d7e/l91 0 2024-09-10T11:29:40.790 INFO:tasks.workunit.client.0.smithi144.stdout:0/378: symlink da/d3a/d4b/d4e/l73 0 2024-09-10T11:29:40.790 INFO:tasks.workunit.client.1.smithi178.stdout:6/888: unlink d3/d7/c36 0 2024-09-10T11:29:40.792 INFO:tasks.workunit.client.0.smithi144.stdout:8/382: mkdir d0/d1b/d47/d5e/d76 0 2024-09-10T11:29:40.793 INFO:tasks.workunit.client.0.smithi144.stdout:0/379: truncate da/f41 1131017 0 2024-09-10T11:29:40.795 INFO:tasks.workunit.client.0.smithi144.stdout:8/383: chown d0/d9/l33 1841 1 2024-09-10T11:29:40.796 INFO:tasks.workunit.client.0.smithi144.stdout:7/548: mkdir d5/dc/d14/db1 0 2024-09-10T11:29:40.796 INFO:tasks.workunit.client.1.smithi178.stdout:7/788: rename d6/d10/d1d/d4a/d79/c10e to d6/d10/d1d/d22/d50/d98/da6/dd5/c117 0 2024-09-10T11:29:40.797 INFO:tasks.workunit.client.1.smithi178.stdout:4/789: creat d0/d9/d3d/d4f/ddb/d106/f115 x:0 0 0 2024-09-10T11:29:40.797 INFO:tasks.workunit.client.0.smithi144.stdout:0/380: symlink da/d3a/d51/d5f/d6b/l74 0 2024-09-10T11:29:40.798 INFO:tasks.workunit.client.1.smithi178.stdout:6/889: mknod d3/d74/d91/daa/dc2/d57/d78/d7c/c143 0 2024-09-10T11:29:40.798 INFO:tasks.workunit.client.0.smithi144.stdout:8/384: creat d0/d9/d21/f77 x:0 0 0 2024-09-10T11:29:40.799 INFO:tasks.workunit.client.1.smithi178.stdout:7/789: dread - d6/d10/d1d/d22/d50/d98/da6/dd5/f105 zero size 2024-09-10T11:29:40.800 INFO:tasks.workunit.client.1.smithi178.stdout:4/790: mknod d0/d9/c116 0 2024-09-10T11:29:40.800 INFO:tasks.workunit.client.0.smithi144.stdout:7/549: mkdir d5/dc/d14/d40/d58/d6a/db2 0 2024-09-10T11:29:40.802 INFO:tasks.workunit.client.1.smithi178.stdout:7/790: fsync d6/d10/d1d/d22/d50/d7e/d82/f92 0 2024-09-10T11:29:40.803 INFO:tasks.workunit.client.0.smithi144.stdout:0/381: creat da/d3a/d4b/f75 x:0 0 0 2024-09-10T11:29:40.803 INFO:tasks.workunit.client.1.smithi178.stdout:4/791: read - d0/f42 zero size 2024-09-10T11:29:40.804 INFO:tasks.workunit.client.0.smithi144.stdout:7/550: read d5/dc/d14/d40/d58/d6a/fa2 [1912378,60984] 0 2024-09-10T11:29:40.804 INFO:tasks.workunit.client.0.smithi144.stdout:0/382: write da/f1c [3494822,110993] 0 2024-09-10T11:29:40.805 INFO:tasks.workunit.client.1.smithi178.stdout:4/792: write d0/d8/da/fd3 [818821,107694] 0 2024-09-10T11:29:40.806 INFO:tasks.workunit.client.1.smithi178.stdout:7/791: creat d6/d10/d34/d6a/d6f/db1/f118 x:0 0 0 2024-09-10T11:29:40.808 INFO:tasks.workunit.client.0.smithi144.stdout:7/551: readlink d5/l6 0 2024-09-10T11:29:40.809 INFO:tasks.workunit.client.0.smithi144.stdout:0/383: dread - da/d3a/d4b/f75 zero size 2024-09-10T11:29:40.810 INFO:tasks.workunit.client.0.smithi144.stdout:3/594: write d4/d28/f76 [1029323,30820] 0 2024-09-10T11:29:40.810 INFO:tasks.workunit.client.0.smithi144.stdout:8/385: dwrite d0/d9/d23/d26/f4e [0,4194304] 0 2024-09-10T11:29:40.811 INFO:tasks.workunit.client.1.smithi178.stdout:7/792: chown d6/d10/d1d/d22/l4b 21 1 2024-09-10T11:29:40.814 INFO:tasks.workunit.client.0.smithi144.stdout:3/595: write d4/d28/d67/d80/d1d/fac [272055,63714] 0 2024-09-10T11:29:40.817 INFO:tasks.workunit.client.0.smithi144.stdout:0/384: creat da/d3a/f76 x:0 0 0 2024-09-10T11:29:40.818 INFO:tasks.workunit.client.0.smithi144.stdout:8/386: fsync d0/d9/d23/f40 0 2024-09-10T11:29:40.819 INFO:tasks.workunit.client.1.smithi178.stdout:8/965: dread d5/db/d10/d70/f78 [0,4194304] 0 2024-09-10T11:29:40.820 INFO:tasks.workunit.client.1.smithi178.stdout:7/793: rename d6/d10/d1e/d2d/d2b/fda to d6/d10/d1d/d22/d50/d98/da6/dfd/f119 0 2024-09-10T11:29:40.822 INFO:tasks.workunit.client.0.smithi144.stdout:0/385: write da/d3a/d4b/d4e/f5d [415329,49743] 0 2024-09-10T11:29:40.823 INFO:tasks.workunit.client.0.smithi144.stdout:7/552: rename d5/dc/d11/d18/f24 to d5/dc/d14/fb3 0 2024-09-10T11:29:40.825 INFO:tasks.workunit.client.1.smithi178.stdout:7/794: creat d6/d10/d34/d6a/d6f/db1/f11a x:0 0 0 2024-09-10T11:29:40.826 INFO:tasks.workunit.client.0.smithi144.stdout:8/387: link d0/d9/d1f/d34/d68/c66 d0/d9/d1f/d34/c78 0 2024-09-10T11:29:40.826 INFO:tasks.workunit.client.1.smithi178.stdout:8/966: dwrite d5/db/dda/fe1 [0,4194304] 0 2024-09-10T11:29:40.830 INFO:tasks.workunit.client.0.smithi144.stdout:0/386: dwrite da/f28 [0,4194304] 0 2024-09-10T11:29:40.835 INFO:tasks.workunit.client.0.smithi144.stdout:8/388: dwrite d0/fc [4194304,4194304] 0 2024-09-10T11:29:40.836 INFO:tasks.workunit.client.1.smithi178.stdout:8/967: link d5/db/d10/d35/f5a d5/db/d10/d20/d52/d5c/dad/f146 0 2024-09-10T11:29:40.837 INFO:tasks.workunit.client.1.smithi178.stdout:0/833: dwrite d0/dc/d39/d50/f9f [0,4194304] 0 2024-09-10T11:29:40.839 INFO:tasks.workunit.client.1.smithi178.stdout:9/923: write d2/d9b/df8/d77/fc9 [572393,103149] 0 2024-09-10T11:29:40.839 INFO:tasks.workunit.client.0.smithi144.stdout:7/553: truncate d5/dc/d14/fb3 130421 0 2024-09-10T11:29:40.841 INFO:tasks.workunit.client.0.smithi144.stdout:8/389: creat d0/d9/d23/f79 x:0 0 0 2024-09-10T11:29:40.842 INFO:tasks.workunit.client.0.smithi144.stdout:7/554: truncate d5/dc/d14/d40/f9f 559029 0 2024-09-10T11:29:40.844 INFO:tasks.workunit.client.1.smithi178.stdout:0/834: chown d0/d2c/d33/f81 552 1 2024-09-10T11:29:40.846 INFO:tasks.workunit.client.1.smithi178.stdout:9/924: dread d2/f3f [0,4194304] 0 2024-09-10T11:29:40.848 INFO:tasks.workunit.client.0.smithi144.stdout:7/555: dread d5/dc/d12/f22 [0,4194304] 0 2024-09-10T11:29:40.851 INFO:tasks.workunit.client.0.smithi144.stdout:0/387: getdents da 0 2024-09-10T11:29:40.851 INFO:tasks.workunit.client.0.smithi144.stdout:8/390: mknod d0/d9/d23/c7a 0 2024-09-10T11:29:40.853 INFO:tasks.workunit.client.1.smithi178.stdout:8/968: dwrite d5/d61/f136 [4194304,4194304] 0 2024-09-10T11:29:40.855 INFO:tasks.workunit.client.0.smithi144.stdout:8/391: creat d0/d1b/d47/f7b x:0 0 0 2024-09-10T11:29:40.857 INFO:tasks.workunit.client.0.smithi144.stdout:0/388: rename da/d3a/d4b/f75 to da/d3a/d4b/d61/f77 0 2024-09-10T11:29:40.858 INFO:tasks.workunit.client.1.smithi178.stdout:9/925: unlink d2/d51/ff2 0 2024-09-10T11:29:40.858 INFO:tasks.workunit.client.0.smithi144.stdout:7/556: getdents d5/dc/d12/d2f/d7c 0 2024-09-10T11:29:40.858 INFO:tasks.workunit.client.1.smithi178.stdout:3/942: dwrite db/d20/d3d/d63/f96 [0,4194304] 0 2024-09-10T11:29:40.860 INFO:tasks.workunit.client.0.smithi144.stdout:8/392: creat d0/d1b/d47/d5e/d76/f7c x:0 0 0 2024-09-10T11:29:40.861 INFO:tasks.workunit.client.1.smithi178.stdout:0/835: link d0/dc/d39/d50/f1d d0/dc/d1f/d2d/df9/f106 0 2024-09-10T11:29:40.862 INFO:tasks.workunit.client.0.smithi144.stdout:0/389: mkdir da/d3a/d4b/d4e/d62/d78 0 2024-09-10T11:29:40.862 INFO:tasks.workunit.client.0.smithi144.stdout:0/390: fdatasync f2 0 2024-09-10T11:29:40.865 INFO:tasks.workunit.client.1.smithi178.stdout:3/943: write db/d20/d3e/d6b/fa7 [414894,74438] 0 2024-09-10T11:29:40.866 INFO:tasks.workunit.client.0.smithi144.stdout:8/393: mkdir d0/d9/d23/d26/d7d 0 2024-09-10T11:29:40.868 INFO:tasks.workunit.client.0.smithi144.stdout:7/557: dwrite d5/dc/d14/d40/f9f [0,4194304] 0 2024-09-10T11:29:40.869 INFO:tasks.workunit.client.1.smithi178.stdout:7/795: dwrite d6/d10/d1d/d22/d93/fa8 [4194304,4194304] 0 2024-09-10T11:29:40.869 INFO:tasks.workunit.client.1.smithi178.stdout:9/926: mknod d2/d13/d75/d60/dd0/c123 0 2024-09-10T11:29:40.869 INFO:tasks.workunit.client.0.smithi144.stdout:8/394: write d0/d9/f1d [5223187,106353] 0 2024-09-10T11:29:40.870 INFO:tasks.workunit.client.1.smithi178.stdout:0/836: mkdir d0/d2c/d93/d107 0 2024-09-10T11:29:40.870 INFO:tasks.workunit.client.0.smithi144.stdout:0/391: creat da/d3a/d4b/d4e/d62/d69/f79 x:0 0 0 2024-09-10T11:29:40.872 INFO:tasks.workunit.client.1.smithi178.stdout:0/837: write d0/d12/f87 [2928817,3979] 0 2024-09-10T11:29:40.872 INFO:tasks.workunit.client.0.smithi144.stdout:8/395: mkdir d0/d9/d21/d7e 0 2024-09-10T11:29:40.874 INFO:tasks.workunit.client.1.smithi178.stdout:7/796: write d6/fb [4330087,65836] 0 2024-09-10T11:29:40.876 INFO:tasks.workunit.client.0.smithi144.stdout:0/392: dread - da/d3a/f3b zero size 2024-09-10T11:29:40.876 INFO:tasks.workunit.client.0.smithi144.stdout:0/393: fsync da/d3a/d4b/d4e/d62/f68 0 2024-09-10T11:29:40.878 INFO:tasks.workunit.client.0.smithi144.stdout:8/396: creat d0/d9/d21/d25/f7f x:0 0 0 2024-09-10T11:29:40.878 INFO:tasks.workunit.client.1.smithi178.stdout:6/890: dwrite d3/d24/fc7 [4194304,4194304] 0 2024-09-10T11:29:40.878 INFO:tasks.workunit.client.1.smithi178.stdout:7/797: chown d6/d10/d34/f6e 45746 1 2024-09-10T11:29:40.878 INFO:tasks.workunit.client.0.smithi144.stdout:8/397: write d0/d9/d21/f73 [488345,118443] 0 2024-09-10T11:29:40.879 INFO:tasks.workunit.client.0.smithi144.stdout:8/398: chown d0/d1b/l4b 122052752 1 2024-09-10T11:29:40.880 INFO:tasks.workunit.client.0.smithi144.stdout:7/558: creat d5/dc/d14/d40/fb4 x:0 0 0 2024-09-10T11:29:40.882 INFO:tasks.workunit.client.0.smithi144.stdout:0/394: creat da/d3a/d51/f7a x:0 0 0 2024-09-10T11:29:40.884 INFO:tasks.workunit.client.0.smithi144.stdout:0/395: fsync da/d3a/f76 0 2024-09-10T11:29:40.888 INFO:tasks.workunit.client.0.smithi144.stdout:8/399: dwrite d0/d1b/f2f [8388608,4194304] 0 2024-09-10T11:29:40.892 INFO:tasks.workunit.client.0.smithi144.stdout:7/559: mknod d5/dc/d14/d40/d58/cb5 0 2024-09-10T11:29:40.893 INFO:tasks.workunit.client.0.smithi144.stdout:0/396: symlink da/d44/l7b 0 2024-09-10T11:29:40.893 INFO:tasks.workunit.client.0.smithi144.stdout:7/560: write d5/dc/f10 [4672808,90167] 0 2024-09-10T11:29:40.893 INFO:tasks.workunit.client.0.smithi144.stdout:8/400: chown d0/d9/d21/l43 32515391 1 2024-09-10T11:29:40.894 INFO:tasks.workunit.client.1.smithi178.stdout:8/969: dread d5/d1d/f50 [0,4194304] 0 2024-09-10T11:29:40.898 INFO:tasks.workunit.client.0.smithi144.stdout:7/561: rmdir d5/dc/d2a/d60 39 2024-09-10T11:29:40.900 INFO:tasks.workunit.client.0.smithi144.stdout:7/562: truncate d5/dc/d14/d40/f72 240845 0 2024-09-10T11:29:40.901 INFO:tasks.workunit.client.0.smithi144.stdout:8/401: rename d0/d9/d21/d25/f7f to d0/d9/d23/d26/d7d/f80 0 2024-09-10T11:29:40.902 INFO:tasks.workunit.client.0.smithi144.stdout:8/402: chown d0/d1b/d47/d5e/f64 66620851 1 2024-09-10T11:29:40.903 INFO:tasks.workunit.client.0.smithi144.stdout:8/403: fsync d0/d9/d1f/d34/f53 0 2024-09-10T11:29:40.905 INFO:tasks.workunit.client.0.smithi144.stdout:8/404: creat d0/d9/d23/d26/d7d/f81 x:0 0 0 2024-09-10T11:29:40.909 INFO:tasks.workunit.client.0.smithi144.stdout:8/405: dread d0/d9/d1f/d34/f53 [0,4194304] 0 2024-09-10T11:29:40.910 INFO:tasks.workunit.client.0.smithi144.stdout:8/406: write d0/d9/d23/f79 [1026884,68806] 0 2024-09-10T11:29:40.911 INFO:tasks.workunit.client.0.smithi144.stdout:8/407: readlink d0/d1b/l62 0 2024-09-10T11:29:40.914 INFO:tasks.workunit.client.0.smithi144.stdout:0/397: write da/d3a/f50 [2971648,26942] 0 2024-09-10T11:29:40.916 INFO:tasks.workunit.client.0.smithi144.stdout:0/398: symlink da/d3a/d51/l7c 0 2024-09-10T11:29:40.918 INFO:tasks.workunit.client.0.smithi144.stdout:8/408: link d0/d9/d1f/d34/c74 d0/d9/d21/d7e/c82 0 2024-09-10T11:29:40.922 INFO:tasks.workunit.client.0.smithi144.stdout:7/563: dwrite d5/dc/d14/fb3 [0,4194304] 0 2024-09-10T11:29:40.922 INFO:tasks.workunit.client.0.smithi144.stdout:0/399: write da/f19 [2277069,98984] 0 2024-09-10T11:29:40.924 INFO:tasks.workunit.client.0.smithi144.stdout:8/409: mknod d0/d9/d21/d7e/c83 0 2024-09-10T11:29:40.927 INFO:tasks.workunit.client.0.smithi144.stdout:8/410: mkdir d0/d84 0 2024-09-10T11:29:40.928 INFO:tasks.workunit.client.0.smithi144.stdout:8/411: chown d0/d1b/f2f 31 1 2024-09-10T11:29:40.928 INFO:tasks.workunit.client.0.smithi144.stdout:8/412: dread - d0/d9/d21/f59 zero size 2024-09-10T11:29:40.929 INFO:tasks.workunit.client.0.smithi144.stdout:7/564: creat d5/dc/d14/d40/d5e/d68/fb6 x:0 0 0 2024-09-10T11:29:40.931 INFO:tasks.workunit.client.0.smithi144.stdout:0/400: creat da/d3a/d4b/f7d x:0 0 0 2024-09-10T11:29:40.933 INFO:tasks.workunit.client.0.smithi144.stdout:8/413: chown d0/d9/d1f/d34/c78 0 1 2024-09-10T11:29:40.933 INFO:tasks.workunit.client.0.smithi144.stdout:0/401: write da/f30 [2268579,118690] 0 2024-09-10T11:29:40.933 INFO:tasks.workunit.client.0.smithi144.stdout:7/565: symlink d5/dc/lb7 0 2024-09-10T11:29:40.937 INFO:tasks.workunit.client.0.smithi144.stdout:7/566: rmdir d5/dc/d14/db1 0 2024-09-10T11:29:40.938 INFO:tasks.workunit.client.1.smithi178.stdout:9/927: truncate d2/df/d31/d37/d3d/f6e 967132 0 2024-09-10T11:29:40.940 INFO:tasks.workunit.client.0.smithi144.stdout:7/567: write d5/dc/d14/d40/f72 [805526,110852] 0 2024-09-10T11:29:40.940 INFO:tasks.workunit.client.0.smithi144.stdout:8/414: dwrite d0/d9/f1d [0,4194304] 0 2024-09-10T11:29:40.942 INFO:tasks.workunit.client.0.smithi144.stdout:8/415: write d0/d9/f1d [6075954,15636] 0 2024-09-10T11:29:40.945 INFO:tasks.workunit.client.1.smithi178.stdout:7/798: dwrite d6/fc9 [0,4194304] 0 2024-09-10T11:29:40.946 INFO:tasks.workunit.client.0.smithi144.stdout:8/416: getdents d0/d84 0 2024-09-10T11:29:40.946 INFO:tasks.workunit.client.0.smithi144.stdout:0/402: dwrite da/f2e [0,4194304] 0 2024-09-10T11:29:40.947 INFO:tasks.workunit.client.0.smithi144.stdout:0/403: read da/f1a [272239,126393] 0 2024-09-10T11:29:40.952 INFO:tasks.workunit.client.0.smithi144.stdout:8/417: mkdir d0/d9/d21/d7e/d85 0 2024-09-10T11:29:40.954 INFO:tasks.workunit.client.0.smithi144.stdout:8/418: chown d0/d9/d1f/f1c 31 1 2024-09-10T11:29:40.954 INFO:tasks.workunit.client.0.smithi144.stdout:0/404: dwrite da/f1a [0,4194304] 0 2024-09-10T11:29:40.957 INFO:tasks.workunit.client.0.smithi144.stdout:0/405: stat da/d3a/d4b/d4e/d62/f70 0 2024-09-10T11:29:40.958 INFO:tasks.workunit.client.0.smithi144.stdout:0/406: chown da/d44/f65 1964414 1 2024-09-10T11:29:40.964 INFO:tasks.workunit.client.0.smithi144.stdout:7/568: truncate d5/dc/d14/fb3 3349549 0 2024-09-10T11:29:40.966 INFO:tasks.workunit.client.0.smithi144.stdout:0/407: dwrite da/d3a/f3e [4194304,4194304] 0 2024-09-10T11:29:40.967 INFO:tasks.workunit.client.1.smithi178.stdout:1/848: sync 2024-09-10T11:29:40.970 INFO:tasks.workunit.client.0.smithi144.stdout:0/408: mkdir da/d6f/d7e 0 2024-09-10T11:29:40.973 INFO:tasks.workunit.client.0.smithi144.stdout:0/409: rename da/d44/l7b to da/d6f/l7f 0 2024-09-10T11:29:40.974 INFO:tasks.workunit.client.0.smithi144.stdout:0/410: rmdir da/d49 39 2024-09-10T11:29:40.978 INFO:tasks.workunit.client.0.smithi144.stdout:8/419: dread d0/f24 [0,4194304] 0 2024-09-10T11:29:40.982 INFO:tasks.workunit.client.0.smithi144.stdout:8/420: link d0/d9/d23/d26/d7d/f80 d0/d9/d21/f86 0 2024-09-10T11:29:40.983 INFO:tasks.workunit.client.0.smithi144.stdout:8/421: chown d0/d9/d23/d26/d7d/f81 3 1 2024-09-10T11:29:40.985 INFO:tasks.workunit.client.1.smithi178.stdout:3/944: write db/d20/f8a [3579025,13266] 0 2024-09-10T11:29:40.988 INFO:tasks.workunit.client.1.smithi178.stdout:7/799: creat d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d70/da2/f11b x:0 0 0 2024-09-10T11:29:40.989 INFO:tasks.workunit.client.1.smithi178.stdout:7/800: chown d6/d10/d1d/l9d 40337019 1 2024-09-10T11:29:40.991 INFO:tasks.workunit.client.1.smithi178.stdout:0/838: write d0/d12/d3f/fa5 [833397,13910] 0 2024-09-10T11:29:40.992 INFO:tasks.workunit.client.1.smithi178.stdout:9/928: fsync d2/d9b/df8/d2d/f95 0 2024-09-10T11:29:40.994 INFO:tasks.workunit.client.1.smithi178.stdout:0/839: write d0/d12/d3f/d72/f82 [547853,78440] 0 2024-09-10T11:29:40.997 INFO:tasks.workunit.client.1.smithi178.stdout:7/801: mkdir d6/d10/d1d/daa/d11c 0 2024-09-10T11:29:40.998 INFO:tasks.workunit.client.1.smithi178.stdout:3/945: write db/d20/d3e/d6b/fd4 [4977691,40339] 0 2024-09-10T11:29:40.999 INFO:tasks.workunit.client.1.smithi178.stdout:6/891: dwrite d3/d7/d35/f5a [0,4194304] 0 2024-09-10T11:29:40.999 INFO:tasks.workunit.client.1.smithi178.stdout:9/929: fsync d2/d51/f8f 0 2024-09-10T11:29:41.002 INFO:tasks.workunit.client.1.smithi178.stdout:8/970: dwrite d5/d30/f87 [0,4194304] 0 2024-09-10T11:29:41.003 INFO:tasks.workunit.client.0.smithi144.stdout:8/422: dread d0/d9/d23/f40 [0,4194304] 0 2024-09-10T11:29:41.007 INFO:tasks.workunit.client.0.smithi144.stdout:8/423: creat d0/d9/d23/f87 x:0 0 0 2024-09-10T11:29:41.011 INFO:tasks.workunit.client.1.smithi178.stdout:7/802: dwrite d6/d10/d1d/dca/f10f [0,4194304] 0 2024-09-10T11:29:41.011 INFO:tasks.workunit.client.0.smithi144.stdout:8/424: getdents d0/d9/d21/d7e/d85 0 2024-09-10T11:29:41.013 INFO:tasks.workunit.client.1.smithi178.stdout:6/892: mkdir d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/d144 0 2024-09-10T11:29:41.014 INFO:tasks.workunit.client.1.smithi178.stdout:0/840: write d0/dc/d39/d50/d5e/f7d [29694,69650] 0 2024-09-10T11:29:41.014 INFO:tasks.workunit.client.1.smithi178.stdout:9/930: creat d2/d19/dfd/f124 x:0 0 0 2024-09-10T11:29:41.017 INFO:tasks.workunit.client.1.smithi178.stdout:8/971: rename d5/db/d10/d20/f28 to d5/d1d/d22/d76/d126/f147 0 2024-09-10T11:29:41.018 INFO:tasks.workunit.client.1.smithi178.stdout:7/803: rmdir d6/d10/d1d/d4a/d79/df1 39 2024-09-10T11:29:41.020 INFO:tasks.workunit.client.1.smithi178.stdout:7/804: fdatasync d6/d10/d1d/d22/d50/d7e/d82/db2/d49/f10c 0 2024-09-10T11:29:41.021 INFO:tasks.workunit.client.1.smithi178.stdout:0/841: fdatasync d0/dc/d39/d7c/fbb 0 2024-09-10T11:29:41.022 INFO:tasks.workunit.client.0.smithi144.stdout:8/425: dwrite d0/d9/d21/f86 [0,4194304] 0 2024-09-10T11:29:41.023 INFO:tasks.workunit.client.1.smithi178.stdout:6/893: chown d3/d74/d91/daa/dc2/f66 625 1 2024-09-10T11:29:41.025 INFO:tasks.workunit.client.1.smithi178.stdout:7/805: chown d6/d10/d1e/d2d/d2b/da0/d55/fed 21662 1 2024-09-10T11:29:41.026 INFO:tasks.workunit.client.1.smithi178.stdout:7/806: chown d6/d10/d1d/d4a/f57 1556092862 1 2024-09-10T11:29:41.027 INFO:tasks.workunit.client.1.smithi178.stdout:9/931: creat d2/d9b/df8/d2d/dab/d118/d56/dbe/deb/d10f/f125 x:0 0 0 2024-09-10T11:29:41.028 INFO:tasks.workunit.client.1.smithi178.stdout:0/842: symlink d0/dc/d1f/dae/l108 0 2024-09-10T11:29:41.033 INFO:tasks.workunit.client.1.smithi178.stdout:7/807: creat d6/d10/d1d/d22/d50/d7e/da7/f11d x:0 0 0 2024-09-10T11:29:41.035 INFO:tasks.workunit.client.1.smithi178.stdout:8/972: write d5/db/d10/d70/fcb [1815297,54168] 0 2024-09-10T11:29:41.035 INFO:tasks.workunit.client.1.smithi178.stdout:0/843: mknod d0/dc/d39/d50/d5e/c109 0 2024-09-10T11:29:41.038 INFO:tasks.workunit.client.1.smithi178.stdout:3/946: dwrite db/d20/d3d/d63/da2/d98/fbe [4194304,4194304] 0 2024-09-10T11:29:41.041 INFO:tasks.workunit.client.1.smithi178.stdout:8/973: chown d5/db/d3d/c6d 8251429 1 2024-09-10T11:29:41.041 INFO:tasks.workunit.client.1.smithi178.stdout:7/808: mknod d6/d10/d1d/d22/d50/c11e 0 2024-09-10T11:29:41.042 INFO:tasks.workunit.client.1.smithi178.stdout:9/932: dread d2/df/d31/d37/d3d/f83 [0,4194304] 0 2024-09-10T11:29:41.044 INFO:tasks.workunit.client.1.smithi178.stdout:0/844: creat d0/dc/d39/d75/f10a x:0 0 0 2024-09-10T11:29:41.045 INFO:tasks.workunit.client.1.smithi178.stdout:9/933: readlink d2/d9b/df8/d2a/lea 0 2024-09-10T11:29:41.045 INFO:tasks.workunit.client.1.smithi178.stdout:3/947: rmdir db/d20/d3d/d63 39 2024-09-10T11:29:41.048 INFO:tasks.workunit.client.1.smithi178.stdout:9/934: read d2/d8/f50 [667554,96380] 0 2024-09-10T11:29:41.048 INFO:tasks.workunit.client.1.smithi178.stdout:8/974: mknod d5/d30/d33/d8c/daa/c148 0 2024-09-10T11:29:41.054 INFO:tasks.workunit.client.1.smithi178.stdout:7/809: dwrite d6/d10/d1d/d22/d3b/f5a [0,4194304] 0 2024-09-10T11:29:41.055 INFO:tasks.workunit.client.1.smithi178.stdout:0/845: symlink d0/l10b 0 2024-09-10T11:29:41.055 INFO:tasks.workunit.client.1.smithi178.stdout:6/894: dread d3/d7/d8/d15/d1c/d2a/d18/f43 [0,4194304] 0 2024-09-10T11:29:41.056 INFO:tasks.workunit.client.1.smithi178.stdout:0/846: dread - d0/dc/d39/d7c/f9b zero size 2024-09-10T11:29:41.058 INFO:tasks.workunit.client.1.smithi178.stdout:8/975: fsync d5/d30/db4/fee 0 2024-09-10T11:29:41.060 INFO:tasks.workunit.client.1.smithi178.stdout:3/948: creat db/d35/d58/ddc/f130 x:0 0 0 2024-09-10T11:29:41.064 INFO:tasks.workunit.client.1.smithi178.stdout:9/935: creat d2/df/f126 x:0 0 0 2024-09-10T11:29:41.066 INFO:tasks.workunit.client.1.smithi178.stdout:9/936: chown d2/le 1531205 1 2024-09-10T11:29:41.069 INFO:tasks.workunit.client.1.smithi178.stdout:9/937: fdatasync d2/df/f126 0 2024-09-10T11:29:41.070 INFO:tasks.workunit.client.1.smithi178.stdout:0/847: dwrite d0/dc/f13 [0,4194304] 0 2024-09-10T11:29:41.074 INFO:tasks.workunit.client.1.smithi178.stdout:8/976: getdents d5/db/d10/d53/d82/db2 0 2024-09-10T11:29:41.078 INFO:tasks.workunit.client.1.smithi178.stdout:3/949: dwrite db/d20/f10a [0,4194304] 0 2024-09-10T11:29:41.080 INFO:tasks.workunit.client.0.smithi144.stdout:8/426: dwrite d0/d9/d21/f5c [4194304,4194304] 0 2024-09-10T11:29:41.081 INFO:tasks.workunit.client.1.smithi178.stdout:6/895: dwrite d3/d74/dc9/da6/d11e/f52 [0,4194304] 0 2024-09-10T11:29:41.081 INFO:tasks.workunit.client.0.smithi144.stdout:4/535: sync 2024-09-10T11:29:41.083 INFO:tasks.workunit.client.0.smithi144.stdout:4/536: write d14/d1e/d22/d34/d3e/f97 [20066,47759] 0 2024-09-10T11:29:41.083 INFO:tasks.workunit.client.1.smithi178.stdout:3/950: truncate db/d35/d3a/d9f/fb8 2133876 0 2024-09-10T11:29:41.088 INFO:tasks.workunit.client.1.smithi178.stdout:8/977: unlink d5/d30/db8/fe9 0 2024-09-10T11:29:41.090 INFO:tasks.workunit.client.0.smithi144.stdout:4/537: mknod d14/d1e/d83/da4/da9/dbc/cc0 0 2024-09-10T11:29:41.092 INFO:tasks.workunit.client.1.smithi178.stdout:4/793: sync 2024-09-10T11:29:41.097 INFO:tasks.workunit.client.0.smithi144.stdout:8/427: getdents d0/d9/d23/d26 0 2024-09-10T11:29:41.097 INFO:tasks.workunit.client.0.smithi144.stdout:8/428: write d0/d9/d23/d26/f30 [2557690,106441] 0 2024-09-10T11:29:41.097 INFO:tasks.workunit.client.0.smithi144.stdout:8/429: readlink d0/d1b/d5a/l5d 0 2024-09-10T11:29:41.097 INFO:tasks.workunit.client.1.smithi178.stdout:3/951: dwrite db/d35/d3a/d9f/fe4 [0,4194304] 0 2024-09-10T11:29:41.100 INFO:tasks.workunit.client.1.smithi178.stdout:9/938: dwrite d2/d13/f3c [0,4194304] 0 2024-09-10T11:29:41.103 INFO:tasks.workunit.client.1.smithi178.stdout:0/848: link d0/d12/d3f/d72/df0/f100 d0/dc/d39/f10c 0 2024-09-10T11:29:41.106 INFO:tasks.workunit.client.1.smithi178.stdout:8/978: dwrite d5/d30/d33/d75/f133 [0,4194304] 0 2024-09-10T11:29:41.108 INFO:tasks.workunit.client.0.smithi144.stdout:4/538: getdents d14/d67/d69 0 2024-09-10T11:29:41.108 INFO:tasks.workunit.client.1.smithi178.stdout:6/896: truncate d3/d7/d8/d72/f142 9305214 0 2024-09-10T11:29:41.110 INFO:tasks.workunit.client.0.smithi144.stdout:4/539: truncate d14/d67/d69/d85/f93 1176682 0 2024-09-10T11:29:41.115 INFO:tasks.workunit.client.0.smithi144.stdout:4/540: read - d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/f64 zero size 2024-09-10T11:29:41.115 INFO:tasks.workunit.client.1.smithi178.stdout:8/979: symlink d5/d30/d33/d3a/l149 0 2024-09-10T11:29:41.117 INFO:tasks.workunit.client.1.smithi178.stdout:4/794: creat d0/d8/da/d1a/d51/d5d/dc9/dee/d104/f117 x:0 0 0 2024-09-10T11:29:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:40 smithi178 ceph-mon[33514]: pgmap v27: 65 pgs: 65 active+clean; 2.7 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 126 MiB/s rd, 382 MiB/s wr, 694 op/s 2024-09-10T11:29:41.124 INFO:tasks.workunit.client.1.smithi178.stdout:9/939: dwrite d2/df/d31/d37/d3d/f112 [0,4194304] 0 2024-09-10T11:29:41.126 INFO:tasks.workunit.client.1.smithi178.stdout:8/980: write d5/db/d10/d53/f64 [3529332,64190] 0 2024-09-10T11:29:41.126 INFO:tasks.workunit.client.1.smithi178.stdout:4/795: dread - d0/d9d/da5/fb8 zero size 2024-09-10T11:29:41.126 INFO:tasks.workunit.client.1.smithi178.stdout:3/952: link db/d20/d3e/d6b/dfa/f6c db/d9b/dfb/f131 0 2024-09-10T11:29:41.127 INFO:tasks.workunit.client.0.smithi144.stdout:4/541: link d14/d1e/l8c d14/d67/d69/lc1 0 2024-09-10T11:29:41.131 INFO:tasks.workunit.client.1.smithi178.stdout:3/953: fdatasync db/d35/d58/f7b 0 2024-09-10T11:29:41.132 INFO:tasks.workunit.client.1.smithi178.stdout:9/940: mkdir d2/df/d127 0 2024-09-10T11:29:41.135 INFO:tasks.workunit.client.1.smithi178.stdout:6/897: dwrite d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d11b/d121/f139 [0,4194304] 0 2024-09-10T11:29:41.135 INFO:tasks.workunit.client.1.smithi178.stdout:8/981: chown d5/d1d/d22/d76/d126/f12a 683 1 2024-09-10T11:29:41.139 INFO:tasks.workunit.client.0.smithi144.stdout:4/542: dread d14/d1e/d22/d34/d3e/d40/d4b/f6a [0,4194304] 0 2024-09-10T11:29:41.139 INFO:tasks.workunit.client.0.smithi144.stdout:4/543: stat d14/d1e/d83 0 2024-09-10T11:29:41.141 INFO:tasks.workunit.client.1.smithi178.stdout:9/941: creat d2/d9b/df8/d2d/dab/d118/d56/f128 x:0 0 0 2024-09-10T11:29:41.143 INFO:tasks.workunit.client.1.smithi178.stdout:8/982: rmdir d5/d1d/d22/d76/d126 39 2024-09-10T11:29:41.144 INFO:tasks.workunit.client.0.smithi144.stdout:4/544: truncate d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/fb7 584067 0 2024-09-10T11:29:41.146 INFO:tasks.workunit.client.0.smithi144.stdout:4/545: dread d14/d1e/d22/d34/d3e/d40/d4b/f6a [0,4194304] 0 2024-09-10T11:29:41.149 INFO:tasks.workunit.client.0.smithi144.stdout:8/430: symlink d0/d9/d21/d25/l88 0 2024-09-10T11:29:41.151 INFO:tasks.workunit.client.0.smithi144.stdout:8/431: chown d0/d9/d21/d25/d4f/l75 374333 1 2024-09-10T11:29:41.153 INFO:tasks.workunit.client.1.smithi178.stdout:3/954: rename db/d20/d3e/d6b/dc0/ddf/dcb/le3 to db/d20/d3e/d6b/dfa/l132 0 2024-09-10T11:29:41.155 INFO:tasks.workunit.client.1.smithi178.stdout:9/942: creat d2/d9b/df8/d2d/dab/d118/d56/dbe/df1/f129 x:0 0 0 2024-09-10T11:29:41.157 INFO:tasks.workunit.client.0.smithi144.stdout:4/546: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/f38 [4194304,4194304] 0 2024-09-10T11:29:41.158 INFO:tasks.workunit.client.1.smithi178.stdout:3/955: rename db/d35/f5f to db/d36/d12f/f133 0 2024-09-10T11:29:41.158 INFO:tasks.workunit.client.0.smithi144.stdout:8/432: dread d0/f10 [0,4194304] 0 2024-09-10T11:29:41.160 INFO:tasks.workunit.client.0.smithi144.stdout:4/547: unlink d14/c86 0 2024-09-10T11:29:41.161 INFO:tasks.workunit.client.1.smithi178.stdout:9/943: mknod d2/d9b/df8/d2d/dab/c12a 0 2024-09-10T11:29:41.161 INFO:tasks.workunit.client.0.smithi144.stdout:4/548: dread - d14/d6f/f88 zero size 2024-09-10T11:29:41.164 INFO:tasks.workunit.client.1.smithi178.stdout:3/956: read db/d20/d3d/d63/da2/d98/fbe [6867769,7310] 0 2024-09-10T11:29:41.164 INFO:tasks.workunit.client.0.smithi144.stdout:4/549: rmdir d14/d67/d69/d90 39 2024-09-10T11:29:41.164 INFO:tasks.workunit.client.0.smithi144.stdout:4/550: chown d14/d1e/d83/da4 5163 1 2024-09-10T11:29:41.167 INFO:tasks.workunit.client.0.smithi144.stdout:4/551: creat d14/d67/d84/fc2 x:0 0 0 2024-09-10T11:29:41.168 INFO:tasks.workunit.client.0.smithi144.stdout:4/552: write d14/d1e/fb6 [923303,41953] 0 2024-09-10T11:29:41.169 INFO:tasks.workunit.client.0.smithi144.stdout:4/553: stat d14/d1e/d22/f47 0 2024-09-10T11:29:41.173 INFO:tasks.workunit.client.0.smithi144.stdout:4/554: creat d14/d67/d92/fc3 x:0 0 0 2024-09-10T11:29:41.176 INFO:tasks.workunit.client.0.smithi144.stdout:4/555: creat d14/d1e/d83/da4/fc4 x:0 0 0 2024-09-10T11:29:41.178 INFO:tasks.workunit.client.1.smithi178.stdout:6/898: truncate d3/d6e/f71 2355841 0 2024-09-10T11:29:41.178 INFO:tasks.workunit.client.1.smithi178.stdout:4/796: truncate d0/d8/df/f9e 2263065 0 2024-09-10T11:29:41.180 INFO:tasks.workunit.client.0.smithi144.stdout:4/556: creat d14/d1e/d22/d34/d3e/d40/d4b/d4c/dab/fc5 x:0 0 0 2024-09-10T11:29:41.183 INFO:tasks.workunit.client.1.smithi178.stdout:4/797: mknod d0/d9/dc/d50/c118 0 2024-09-10T11:29:41.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:40 smithi144 ceph-mon[27942]: pgmap v27: 65 pgs: 65 active+clean; 2.7 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 126 MiB/s rd, 382 MiB/s wr, 694 op/s 2024-09-10T11:29:41.186 INFO:tasks.workunit.client.1.smithi178.stdout:9/944: truncate d2/d8/f17 3142977 0 2024-09-10T11:29:41.188 INFO:tasks.workunit.client.0.smithi144.stdout:4/557: dwrite d14/d1e/d83/da4/fc4 [0,4194304] 0 2024-09-10T11:29:41.189 INFO:tasks.workunit.client.1.smithi178.stdout:4/798: mknod d0/d8/da/d1a/d51/d5d/dc9/dee/c119 0 2024-09-10T11:29:41.189 INFO:tasks.workunit.client.1.smithi178.stdout:3/957: getdents db/d35/d3a/d42/d46/d110 0 2024-09-10T11:29:41.191 INFO:tasks.workunit.client.0.smithi144.stdout:4/558: creat d14/d1e/d65/fc6 x:0 0 0 2024-09-10T11:29:41.192 INFO:tasks.workunit.client.1.smithi178.stdout:9/945: unlink d2/df/d31/d37/f94 0 2024-09-10T11:29:41.192 INFO:tasks.workunit.client.0.smithi144.stdout:8/433: dwrite d0/d9/d23/f40 [0,4194304] 0 2024-09-10T11:29:41.194 INFO:tasks.workunit.client.1.smithi178.stdout:4/799: mkdir d0/d63/d11a 0 2024-09-10T11:29:41.194 INFO:tasks.workunit.client.1.smithi178.stdout:4/800: stat d0/d9/d3d/dc2/fce 0 2024-09-10T11:29:41.196 INFO:tasks.workunit.client.1.smithi178.stdout:3/958: chown db/d20/d3d/d63/da2/d98/c11c 616336 1 2024-09-10T11:29:41.198 INFO:tasks.workunit.client.1.smithi178.stdout:9/946: truncate d2/d13/d75/d60/f8d 3478887 0 2024-09-10T11:29:41.201 INFO:tasks.workunit.client.1.smithi178.stdout:3/959: mkdir db/d20/d3d/d63/d134 0 2024-09-10T11:29:41.201 INFO:tasks.workunit.client.1.smithi178.stdout:6/899: dread d3/d74/d91/daa/dc2/d57/d78/d7c/fd2 [0,4194304] 0 2024-09-10T11:29:41.205 INFO:tasks.workunit.client.1.smithi178.stdout:4/801: dwrite d0/d9/d3d/fbd [0,4194304] 0 2024-09-10T11:29:41.211 INFO:tasks.workunit.client.1.smithi178.stdout:6/900: dwrite d3/d7/d35/f6f [0,4194304] 0 2024-09-10T11:29:41.213 INFO:tasks.workunit.client.0.smithi144.stdout:4/559: fsync d14/d1e/d22/d34/d3e/d40/d4b/f6a 0 2024-09-10T11:29:41.214 INFO:tasks.workunit.client.0.smithi144.stdout:8/434: link d0/d1b/d47/d5e/f64 d0/d9/d23/f89 0 2024-09-10T11:29:41.217 INFO:tasks.workunit.client.0.smithi144.stdout:4/560: unlink d14/c25 0 2024-09-10T11:29:41.221 INFO:tasks.workunit.client.0.smithi144.stdout:4/561: creat d14/d1e/d83/da4/da9/fc7 x:0 0 0 2024-09-10T11:29:41.223 INFO:tasks.workunit.client.0.smithi144.stdout:4/562: rmdir d14/d1e/d65/d8a 39 2024-09-10T11:29:41.225 INFO:tasks.workunit.client.1.smithi178.stdout:6/901: dwrite d3/d7/d8/d72/faf [0,4194304] 0 2024-09-10T11:29:41.228 INFO:tasks.workunit.client.1.smithi178.stdout:9/947: dread d2/d51/f8f [4194304,4194304] 0 2024-09-10T11:29:41.230 INFO:tasks.workunit.client.1.smithi178.stdout:3/960: truncate db/d35/d3a/d42/d46/fea 115814 0 2024-09-10T11:29:41.230 INFO:tasks.workunit.client.0.smithi144.stdout:4/563: mknod d14/d1e/d65/cc8 0 2024-09-10T11:29:41.231 INFO:tasks.workunit.client.1.smithi178.stdout:3/961: readlink db/d20/d3e/d6b/dc0/ddf/dcb/l12e 0 2024-09-10T11:29:41.231 INFO:tasks.workunit.client.1.smithi178.stdout:9/948: mknod d2/d13/d75/c12b 0 2024-09-10T11:29:41.234 INFO:tasks.workunit.client.1.smithi178.stdout:6/902: unlink d3/d74/dc9/da6/d11e/f52 0 2024-09-10T11:29:41.236 INFO:tasks.workunit.client.0.smithi144.stdout:4/564: read d14/d1e/d22/d34/d3e/f97 [362370,37255] 0 2024-09-10T11:29:41.240 INFO:tasks.workunit.client.1.smithi178.stdout:3/962: symlink db/l135 0 2024-09-10T11:29:41.241 INFO:tasks.workunit.client.1.smithi178.stdout:3/963: chown db/d20/d70/f8e 1 1 2024-09-10T11:29:41.242 INFO:tasks.workunit.client.1.smithi178.stdout:6/903: fdatasync d3/d74/d91/daa/dc2/d97/fb4 0 2024-09-10T11:29:41.243 INFO:tasks.workunit.client.0.smithi144.stdout:4/565: rename d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/l59 to d14/d67/d69/d85/lc9 0 2024-09-10T11:29:41.245 INFO:tasks.workunit.client.0.smithi144.stdout:8/435: dread d0/d9/d23/f89 [0,4194304] 0 2024-09-10T11:29:41.246 INFO:tasks.workunit.client.0.smithi144.stdout:4/566: mkdir d14/d67/d69/dca 0 2024-09-10T11:29:41.247 INFO:tasks.workunit.client.0.smithi144.stdout:8/436: mknod d0/d9/d21/d7e/d85/c8a 0 2024-09-10T11:29:41.247 INFO:tasks.workunit.client.0.smithi144.stdout:8/437: stat d0/d9/d21/d7e/d85/c8a 0 2024-09-10T11:29:41.248 INFO:tasks.workunit.client.1.smithi178.stdout:3/964: creat db/d35/d3a/d42/d46/d110/f136 x:0 0 0 2024-09-10T11:29:41.248 INFO:tasks.workunit.client.0.smithi144.stdout:4/567: mkdir d14/dcb 0 2024-09-10T11:29:41.249 INFO:tasks.workunit.client.1.smithi178.stdout:3/965: chown db/d20/d3d/d63/c88 590 1 2024-09-10T11:29:41.249 INFO:tasks.workunit.client.0.smithi144.stdout:8/438: creat d0/d1b/f8b x:0 0 0 2024-09-10T11:29:41.250 INFO:tasks.workunit.client.1.smithi178.stdout:6/904: mknod d3/d7/d8/d15/d1c/d2a/c145 0 2024-09-10T11:29:41.251 INFO:tasks.workunit.client.0.smithi144.stdout:4/568: symlink d14/d1e/d22/d34/d3e/lcc 0 2024-09-10T11:29:41.252 INFO:tasks.workunit.client.0.smithi144.stdout:8/439: symlink d0/d9/d1f/d63/l8c 0 2024-09-10T11:29:41.254 INFO:tasks.workunit.client.1.smithi178.stdout:6/905: truncate d3/d7/d8/d15/d1c/d2a/f5d 3109749 0 2024-09-10T11:29:41.254 INFO:tasks.workunit.client.0.smithi144.stdout:8/440: mkdir d0/d1b/d5a/d8d 0 2024-09-10T11:29:41.256 INFO:tasks.workunit.client.0.smithi144.stdout:8/441: rename d0/d9/d1f/l2e to d0/d84/l8e 0 2024-09-10T11:29:41.258 INFO:tasks.workunit.client.1.smithi178.stdout:3/966: link db/d9b/dfb/f131 db/d35/d58/df2/d127/f137 0 2024-09-10T11:29:41.260 INFO:tasks.workunit.client.1.smithi178.stdout:3/967: creat db/d35/d58/f138 x:0 0 0 2024-09-10T11:29:41.262 INFO:tasks.workunit.client.1.smithi178.stdout:3/968: mknod db/d20/d3d/d56/d64/c139 0 2024-09-10T11:29:41.263 INFO:tasks.workunit.client.0.smithi144.stdout:8/442: dwrite d0/d9/d23/d26/d7d/f81 [0,4194304] 0 2024-09-10T11:29:41.265 INFO:tasks.workunit.client.0.smithi144.stdout:8/443: dread - d0/d1b/f8b zero size 2024-09-10T11:29:41.268 INFO:tasks.workunit.client.0.smithi144.stdout:8/444: write d0/d9/fe [2208547,60711] 0 2024-09-10T11:29:41.269 INFO:tasks.workunit.client.0.smithi144.stdout:4/569: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/fb8 [0,4194304] 0 2024-09-10T11:29:41.270 INFO:tasks.workunit.client.0.smithi144.stdout:8/445: stat d0/d9/fe 0 2024-09-10T11:29:41.270 INFO:tasks.workunit.client.1.smithi178.stdout:3/969: creat db/d20/d3e/d6b/dc0/de5/d108/f13a x:0 0 0 2024-09-10T11:29:41.271 INFO:tasks.workunit.client.1.smithi178.stdout:6/906: write d3/d74/d91/daa/dc2/f66 [2390334,67625] 0 2024-09-10T11:29:41.272 INFO:tasks.workunit.client.1.smithi178.stdout:3/970: write db/d20/d3e/d6b/dfa/fdb [584035,106974] 0 2024-09-10T11:29:41.272 INFO:tasks.workunit.client.0.smithi144.stdout:4/570: creat d14/d1e/fcd x:0 0 0 2024-09-10T11:29:41.274 INFO:tasks.workunit.client.1.smithi178.stdout:6/907: read - d3/d74/d91/daa/dc2/dcf/fe0 zero size 2024-09-10T11:29:41.274 INFO:tasks.workunit.client.0.smithi144.stdout:8/446: creat d0/d9/d1f/f8f x:0 0 0 2024-09-10T11:29:41.275 INFO:tasks.workunit.client.1.smithi178.stdout:6/908: chown d3/d74/d91/dee 205 1 2024-09-10T11:29:41.275 INFO:tasks.workunit.client.0.smithi144.stdout:4/571: chown d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/c89 5019 1 2024-09-10T11:29:41.276 INFO:tasks.workunit.client.1.smithi178.stdout:3/971: creat db/d36/f13b x:0 0 0 2024-09-10T11:29:41.279 INFO:tasks.workunit.client.1.smithi178.stdout:3/972: symlink db/d20/d3d/d63/da2/l13c 0 2024-09-10T11:29:41.281 INFO:tasks.workunit.client.0.smithi144.stdout:8/447: getdents d0/d9/d21/d7e/d85 0 2024-09-10T11:29:41.281 INFO:tasks.workunit.client.1.smithi178.stdout:6/909: getdents d3/d74/dc9/ddc 0 2024-09-10T11:29:41.282 INFO:tasks.workunit.client.0.smithi144.stdout:4/572: dwrite f5 [0,4194304] 0 2024-09-10T11:29:41.282 INFO:tasks.workunit.client.1.smithi178.stdout:3/973: mknod db/d35/d3a/d9f/c13d 0 2024-09-10T11:29:41.284 INFO:tasks.workunit.client.0.smithi144.stdout:8/448: write d0/d9/d23/d26/f30 [5401834,60398] 0 2024-09-10T11:29:41.285 INFO:tasks.workunit.client.0.smithi144.stdout:4/573: write d14/d67/fb1 [572107,68226] 0 2024-09-10T11:29:41.285 INFO:tasks.workunit.client.1.smithi178.stdout:6/910: mknod d3/d74/d91/c146 0 2024-09-10T11:29:41.287 INFO:tasks.workunit.client.0.smithi144.stdout:8/449: mkdir d0/d1b/d5a/d90 0 2024-09-10T11:29:41.288 INFO:tasks.workunit.client.0.smithi144.stdout:8/450: dread - d0/d1b/f8b zero size 2024-09-10T11:29:41.288 INFO:tasks.workunit.client.1.smithi178.stdout:3/974: rename db/d20/d3d/d63/l7f to db/d20/d3d/d63/d134/l13e 0 2024-09-10T11:29:41.288 INFO:tasks.workunit.client.0.smithi144.stdout:4/574: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/f79 [77399,68469] 0 2024-09-10T11:29:41.289 INFO:tasks.workunit.client.1.smithi178.stdout:3/975: chown db/d20/d3e/d6b/dc0/ddf 8084421 1 2024-09-10T11:29:41.289 INFO:tasks.workunit.client.0.smithi144.stdout:5/620: sync 2024-09-10T11:29:41.290 INFO:tasks.workunit.client.0.smithi144.stdout:1/672: sync 2024-09-10T11:29:41.290 INFO:tasks.workunit.client.0.smithi144.stdout:9/576: sync 2024-09-10T11:29:41.290 INFO:tasks.workunit.client.1.smithi178.stdout:6/911: mkdir d3/d74/d91/dee/d147 0 2024-09-10T11:29:41.291 INFO:tasks.workunit.client.0.smithi144.stdout:4/575: write d14/d1e/d22/d34/d3e/d40/fac [876950,95826] 0 2024-09-10T11:29:41.292 INFO:tasks.workunit.client.0.smithi144.stdout:1/673: rmdir d5/d2a/d65 39 2024-09-10T11:29:41.294 INFO:tasks.workunit.client.0.smithi144.stdout:8/451: creat d0/d9/d1f/f91 x:0 0 0 2024-09-10T11:29:41.295 INFO:tasks.workunit.client.1.smithi178.stdout:3/976: read - db/d35/d3a/d42/d46/d110/f136 zero size 2024-09-10T11:29:41.296 INFO:tasks.workunit.client.0.smithi144.stdout:9/577: mkdir d11/d12/d1b/d41/d8f/dc0/dd0 0 2024-09-10T11:29:41.300 INFO:tasks.workunit.client.1.smithi178.stdout:6/912: dwrite d3/d74/dc9/da6/fdd [0,4194304] 0 2024-09-10T11:29:41.301 INFO:tasks.workunit.client.1.smithi178.stdout:3/977: rename db/d20/lc9 to db/d20/l13f 0 2024-09-10T11:29:41.302 INFO:tasks.workunit.client.0.smithi144.stdout:1/674: unlink d5/f38 0 2024-09-10T11:29:41.302 INFO:tasks.workunit.client.0.smithi144.stdout:1/675: fdatasync d5/d4a/d63/fac 0 2024-09-10T11:29:41.302 INFO:tasks.workunit.client.0.smithi144.stdout:5/621: unlink d4/d5/d32/d47/l79 0 2024-09-10T11:29:41.302 INFO:tasks.workunit.client.0.smithi144.stdout:4/576: creat d14/dcb/fce x:0 0 0 2024-09-10T11:29:41.302 INFO:tasks.workunit.client.0.smithi144.stdout:5/622: chown d4/d5/d18/d38/c42 49870 1 2024-09-10T11:29:41.303 INFO:tasks.workunit.client.1.smithi178.stdout:6/913: rmdir d3/d7/d8/d56/de1 39 2024-09-10T11:29:41.303 INFO:tasks.workunit.client.0.smithi144.stdout:9/578: unlink d11/d36/f79 0 2024-09-10T11:29:41.308 INFO:tasks.workunit.client.0.smithi144.stdout:5/623: mkdir d4/d20/d24/d6f/d74/dbe 0 2024-09-10T11:29:41.309 INFO:tasks.workunit.client.1.smithi178.stdout:6/914: creat d3/d74/d91/daa/dc2/d57/d78/d7c/f148 x:0 0 0 2024-09-10T11:29:41.310 INFO:tasks.workunit.client.0.smithi144.stdout:8/452: creat d0/d9/d1f/f92 x:0 0 0 2024-09-10T11:29:41.311 INFO:tasks.workunit.client.0.smithi144.stdout:8/453: write d0/d9/d21/f71 [482782,36448] 0 2024-09-10T11:29:41.312 INFO:tasks.workunit.client.1.smithi178.stdout:6/915: unlink d3/d74/dc9/da6/d11e/f55 0 2024-09-10T11:29:41.312 INFO:tasks.workunit.client.0.smithi144.stdout:8/454: dread - d0/d9/d21/f77 zero size 2024-09-10T11:29:41.313 INFO:tasks.workunit.client.1.smithi178.stdout:3/978: link db/d26/d40/c54 db/d20/d3e/d6b/dc0/de5/c140 0 2024-09-10T11:29:41.313 INFO:tasks.workunit.client.0.smithi144.stdout:8/455: write d0/d9/d23/f87 [956811,2745] 0 2024-09-10T11:29:41.314 INFO:tasks.workunit.client.1.smithi178.stdout:3/979: chown db/d20/f33 30665499 1 2024-09-10T11:29:41.314 INFO:tasks.workunit.client.0.smithi144.stdout:8/456: chown d0/d9/d1f/d34/d68/c66 0 1 2024-09-10T11:29:41.315 INFO:tasks.workunit.client.0.smithi144.stdout:8/457: fdatasync d0/d9/d23/d26/f30 0 2024-09-10T11:29:41.316 INFO:tasks.workunit.client.0.smithi144.stdout:7/569: sync 2024-09-10T11:29:41.316 INFO:tasks.workunit.client.0.smithi144.stdout:3/596: sync 2024-09-10T11:29:41.316 INFO:tasks.workunit.client.0.smithi144.stdout:2/435: sync 2024-09-10T11:29:41.316 INFO:tasks.workunit.client.0.smithi144.stdout:0/411: sync 2024-09-10T11:29:41.316 INFO:tasks.workunit.client.0.smithi144.stdout:6/531: sync 2024-09-10T11:29:41.317 INFO:tasks.workunit.client.1.smithi178.stdout:6/916: mknod d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/c149 0 2024-09-10T11:29:41.318 INFO:tasks.workunit.client.0.smithi144.stdout:6/532: chown d9/l5f 5250 1 2024-09-10T11:29:41.321 INFO:tasks.workunit.client.1.smithi178.stdout:3/980: dwrite db/d9b/fe9 [0,4194304] 0 2024-09-10T11:29:41.322 INFO:tasks.workunit.client.1.smithi178.stdout:6/917: creat d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d11b/f14a x:0 0 0 2024-09-10T11:29:41.325 INFO:tasks.workunit.client.1.smithi178.stdout:3/981: creat db/ddd/f141 x:0 0 0 2024-09-10T11:29:41.325 INFO:tasks.workunit.client.1.smithi178.stdout:6/918: dread - d3/d7/d8/d15/d1c/d2a/d18/d8e/d90/ffb zero size 2024-09-10T11:29:41.326 INFO:tasks.workunit.client.0.smithi144.stdout:3/597: dread d4/d1a/f5f [0,4194304] 0 2024-09-10T11:29:41.326 INFO:tasks.workunit.client.0.smithi144.stdout:8/458: dwrite d0/d9/d23/d26/d7d/f80 [0,4194304] 0 2024-09-10T11:29:41.328 INFO:tasks.workunit.client.1.smithi178.stdout:3/982: mknod db/d20/d3d/d126/c142 0 2024-09-10T11:29:41.329 INFO:tasks.workunit.client.0.smithi144.stdout:2/436: rename d7/de/d3d/d5e to d7/d29/d92 0 2024-09-10T11:29:41.330 INFO:tasks.workunit.client.1.smithi178.stdout:6/919: mkdir d3/d74/d91/daa/dc2/d97/d14b 0 2024-09-10T11:29:41.335 INFO:tasks.workunit.client.0.smithi144.stdout:1/676: dwrite d5/df/d4b/d51/da7/f7c [0,4194304] 0 2024-09-10T11:29:41.336 INFO:tasks.workunit.client.1.smithi178.stdout:1/849: sync 2024-09-10T11:29:41.336 INFO:tasks.workunit.client.1.smithi178.stdout:3/983: rename db/d20/d3d/d63/da2/d98/l100 to db/d26/d103/l143 0 2024-09-10T11:29:41.336 INFO:tasks.workunit.client.0.smithi144.stdout:9/579: write d11/d36/f7f [1618751,99940] 0 2024-09-10T11:29:41.337 INFO:tasks.workunit.client.1.smithi178.stdout:1/850: fdatasync d3/d9/d21/f67 0 2024-09-10T11:29:41.342 INFO:tasks.workunit.client.0.smithi144.stdout:4/577: dwrite d14/f15 [0,4194304] 0 2024-09-10T11:29:41.344 INFO:tasks.workunit.client.0.smithi144.stdout:0/412: dread da/f1a [0,4194304] 0 2024-09-10T11:29:41.344 INFO:tasks.workunit.client.0.smithi144.stdout:5/624: truncate d4/d5/f7d 2417990 0 2024-09-10T11:29:41.345 INFO:tasks.workunit.client.0.smithi144.stdout:8/459: mkdir d0/d9/d21/d25/d4f/d93 0 2024-09-10T11:29:41.346 INFO:tasks.workunit.client.0.smithi144.stdout:9/580: fdatasync d11/d36/f4e 0 2024-09-10T11:29:41.347 INFO:tasks.workunit.client.1.smithi178.stdout:1/851: mkdir d3/d9/d98/dac/d3a/d50/db5/de3/d13a 0 2024-09-10T11:29:41.353 INFO:tasks.workunit.client.0.smithi144.stdout:2/437: dwrite d7/da/d1e/f85 [0,4194304] 0 2024-09-10T11:29:41.356 INFO:tasks.workunit.client.0.smithi144.stdout:1/677: creat d5/fdd x:0 0 0 2024-09-10T11:29:41.358 INFO:tasks.workunit.client.0.smithi144.stdout:0/413: rmdir da/d3a/d4b/d4e/d62/d69 39 2024-09-10T11:29:41.359 INFO:tasks.workunit.client.0.smithi144.stdout:5/625: chown d4/d5/d18/d67/l60 0 1 2024-09-10T11:29:41.360 INFO:tasks.workunit.client.1.smithi178.stdout:6/920: write d3/d74/d91/daa/dc2/fb6 [3700578,42857] 0 2024-09-10T11:29:41.361 INFO:tasks.workunit.client.0.smithi144.stdout:6/533: getdents d9/d1f/dae 0 2024-09-10T11:29:41.362 INFO:tasks.workunit.client.1.smithi178.stdout:1/852: write d3/d9/d98/dac/f3b [4992994,10361] 0 2024-09-10T11:29:41.362 INFO:tasks.workunit.client.0.smithi144.stdout:8/460: dwrite d0/d9/f1d [0,4194304] 0 2024-09-10T11:29:41.363 INFO:tasks.workunit.client.0.smithi144.stdout:4/578: getdents d14/d1e/d83/da4 0 2024-09-10T11:29:41.364 INFO:tasks.workunit.client.0.smithi144.stdout:3/598: write d4/d28/d67/d80/d17/f30 [1791879,44646] 0 2024-09-10T11:29:41.365 INFO:tasks.workunit.client.0.smithi144.stdout:6/534: stat d9/dd/f1a 0 2024-09-10T11:29:41.367 INFO:tasks.workunit.client.0.smithi144.stdout:2/438: mknod d7/da/c93 0 2024-09-10T11:29:41.367 INFO:tasks.workunit.client.0.smithi144.stdout:7/570: dread d5/dc/d11/f3d [0,4194304] 0 2024-09-10T11:29:41.368 INFO:tasks.workunit.client.1.smithi178.stdout:3/984: dwrite db/d26/d40/f62 [0,4194304] 0 2024-09-10T11:29:41.368 INFO:tasks.workunit.client.0.smithi144.stdout:8/461: chown d0/d1b/f8b 27 1 2024-09-10T11:29:41.369 INFO:tasks.workunit.client.0.smithi144.stdout:0/414: write da/d44/f63 [952324,96509] 0 2024-09-10T11:29:41.369 INFO:tasks.workunit.client.1.smithi178.stdout:3/985: stat db/d26/f80 0 2024-09-10T11:29:41.370 INFO:tasks.workunit.client.0.smithi144.stdout:4/579: read d14/f54 [3681347,7752] 0 2024-09-10T11:29:41.371 INFO:tasks.workunit.client.1.smithi178.stdout:6/921: dread d3/d6e/ff1 [0,4194304] 0 2024-09-10T11:29:41.372 INFO:tasks.workunit.client.0.smithi144.stdout:2/439: write d7/f15 [5288484,87884] 0 2024-09-10T11:29:41.373 INFO:tasks.workunit.client.1.smithi178.stdout:1/853: rename d3/d9/d98/dac/d3a/d50/c64 to d3/d9/d21/d4b/d11a/c13b 0 2024-09-10T11:29:41.375 INFO:tasks.workunit.client.1.smithi178.stdout:3/986: chown db/d35/ld5 2079589077 1 2024-09-10T11:29:41.376 INFO:tasks.workunit.client.1.smithi178.stdout:6/922: rmdir d3/d74/dc9/ddc 39 2024-09-10T11:29:41.377 INFO:tasks.workunit.client.0.smithi144.stdout:8/462: dread - d0/d1b/d47/d5e/d76/f7c zero size 2024-09-10T11:29:41.379 INFO:tasks.workunit.client.1.smithi178.stdout:6/923: chown d3/d7/d8/d72/f110 110 1 2024-09-10T11:29:41.380 INFO:tasks.workunit.client.0.smithi144.stdout:7/571: dread d5/dc/d14/d40/f67 [0,4194304] 0 2024-09-10T11:29:41.381 INFO:tasks.workunit.client.0.smithi144.stdout:4/580: readlink d14/d1e/d22/lad 0 2024-09-10T11:29:41.382 INFO:tasks.workunit.client.1.smithi178.stdout:1/854: dwrite d3/d9/d98/dd1/ddc/f73 [0,4194304] 0 2024-09-10T11:29:41.383 INFO:tasks.workunit.client.0.smithi144.stdout:0/415: dread da/d44/f63 [0,4194304] 0 2024-09-10T11:29:41.383 INFO:tasks.workunit.client.0.smithi144.stdout:2/440: unlink d7/c23 0 2024-09-10T11:29:41.384 INFO:tasks.workunit.client.1.smithi178.stdout:3/987: dread db/d35/d58/f6e [0,4194304] 0 2024-09-10T11:29:41.384 INFO:tasks.workunit.client.1.smithi178.stdout:1/855: chown d3/d9/d98/dac/d3a/f53 945864038 1 2024-09-10T11:29:41.384 INFO:tasks.workunit.client.0.smithi144.stdout:3/599: rename d4/cc0 to d4/d28/cc3 0 2024-09-10T11:29:41.384 INFO:tasks.workunit.client.0.smithi144.stdout:7/572: chown d5/dc/d14/d40/d58/d6a/dad 3028 1 2024-09-10T11:29:41.384 INFO:tasks.workunit.client.1.smithi178.stdout:6/924: mkdir d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d11b/d14c 0 2024-09-10T11:29:41.385 INFO:tasks.workunit.client.0.smithi144.stdout:1/678: dwrite d5/d4a/d44/da2/fd7 [0,4194304] 0 2024-09-10T11:29:41.385 INFO:tasks.workunit.client.0.smithi144.stdout:8/463: write d0/d9/d23/d26/d7d/f80 [832838,61301] 0 2024-09-10T11:29:41.385 INFO:tasks.workunit.client.1.smithi178.stdout:6/925: readlink d3/d7/d35/l60 0 2024-09-10T11:29:41.390 INFO:tasks.workunit.client.0.smithi144.stdout:3/600: chown d4/d28/d67/d80/d17/c56 0 1 2024-09-10T11:29:41.390 INFO:tasks.workunit.client.0.smithi144.stdout:0/416: fsync da/d3a/f3e 0 2024-09-10T11:29:41.391 INFO:tasks.workunit.client.1.smithi178.stdout:6/926: creat d3/d7/d8/d15/d1c/d2a/d18/f14d x:0 0 0 2024-09-10T11:29:41.392 INFO:tasks.workunit.client.0.smithi144.stdout:4/581: rmdir d14/d1e/d22/d34/d3e/d40/d4b 39 2024-09-10T11:29:41.392 INFO:tasks.workunit.client.1.smithi178.stdout:3/988: getdents db/d35/d58/ddc 0 2024-09-10T11:29:41.393 INFO:tasks.workunit.client.0.smithi144.stdout:7/573: write d5/dc/d14/d40/d58/d6a/fa0 [413089,34852] 0 2024-09-10T11:29:41.395 INFO:tasks.workunit.client.1.smithi178.stdout:3/989: creat db/d26/dcd/f144 x:0 0 0 2024-09-10T11:29:41.398 INFO:tasks.workunit.client.1.smithi178.stdout:3/990: dread - db/d35/d3a/d42/d46/d110/f136 zero size 2024-09-10T11:29:41.399 INFO:tasks.workunit.client.1.smithi178.stdout:3/991: readlink db/d26/d40/lc8 0 2024-09-10T11:29:41.402 INFO:tasks.workunit.client.1.smithi178.stdout:3/992: creat db/d36/f145 x:0 0 0 2024-09-10T11:29:41.403 INFO:tasks.workunit.client.0.smithi144.stdout:9/581: dread d11/d36/f4e [0,4194304] 0 2024-09-10T11:29:41.403 INFO:tasks.workunit.client.0.smithi144.stdout:2/441: dwrite d7/de/f37 [0,4194304] 0 2024-09-10T11:29:41.403 INFO:tasks.workunit.client.0.smithi144.stdout:3/601: chown d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/f7b 211306 1 2024-09-10T11:29:41.403 INFO:tasks.workunit.client.0.smithi144.stdout:7/574: symlink d5/dc/d14/d40/d5e/d63/d98/lb8 0 2024-09-10T11:29:41.407 INFO:tasks.workunit.client.0.smithi144.stdout:0/417: truncate da/d3a/d4b/d4e/d62/f68 466158 0 2024-09-10T11:29:41.408 INFO:tasks.workunit.client.1.smithi178.stdout:6/927: rename d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d11b to d3/d74/d91/dee/d147/d14e 0 2024-09-10T11:29:41.408 INFO:tasks.workunit.client.1.smithi178.stdout:1/856: dread d3/d9/d98/dac/d3a/d6e/f77 [0,4194304] 0 2024-09-10T11:29:41.409 INFO:tasks.workunit.client.0.smithi144.stdout:8/464: truncate d0/d9/d23/d37/f3a 1634809 0 2024-09-10T11:29:41.409 INFO:tasks.workunit.client.0.smithi144.stdout:1/679: dwrite d5/df/d4b/d51/f59 [0,4194304] 0 2024-09-10T11:29:41.411 INFO:tasks.workunit.client.1.smithi178.stdout:6/928: symlink d3/d74/d91/daa/dc2/d57/d116/l14f 0 2024-09-10T11:29:41.412 INFO:tasks.workunit.client.0.smithi144.stdout:9/582: fsync d11/d12/d1b/d41/d8f/f87 0 2024-09-10T11:29:41.413 INFO:tasks.workunit.client.1.smithi178.stdout:6/929: truncate d3/d74/d91/daa/dc2/d57/d78/d7c/f148 706496 0 2024-09-10T11:29:41.415 INFO:tasks.workunit.client.0.smithi144.stdout:0/418: chown da/d3a/d51/l57 42181726 1 2024-09-10T11:29:41.415 INFO:tasks.workunit.client.0.smithi144.stdout:9/583: write d11/f15 [2489359,65255] 0 2024-09-10T11:29:41.415 INFO:tasks.workunit.client.0.smithi144.stdout:3/602: write d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93/fae [277957,36095] 0 2024-09-10T11:29:41.416 INFO:tasks.workunit.client.0.smithi144.stdout:2/442: symlink d7/de/d36/d45/d8e/l94 0 2024-09-10T11:29:41.417 INFO:tasks.workunit.client.1.smithi178.stdout:1/857: unlink d3/d9/d98/dd1/ddc/fd7 0 2024-09-10T11:29:41.417 INFO:tasks.workunit.client.0.smithi144.stdout:0/419: fsync da/d3a/f76 0 2024-09-10T11:29:41.418 INFO:tasks.workunit.client.1.smithi178.stdout:1/858: fdatasync d3/d9/d98/dac/d3a/fee 0 2024-09-10T11:29:41.419 INFO:tasks.workunit.client.0.smithi144.stdout:7/575: read d5/dc/d12/d2f/f47 [6462,45156] 0 2024-09-10T11:29:41.423 INFO:tasks.workunit.client.0.smithi144.stdout:4/582: dwrite d14/d7e/f9f [0,4194304] 0 2024-09-10T11:29:41.423 INFO:tasks.workunit.client.0.smithi144.stdout:2/443: write d7/da/f35 [3242240,4372] 0 2024-09-10T11:29:41.423 INFO:tasks.workunit.client.0.smithi144.stdout:0/420: symlink da/d3a/d51/d5f/l80 0 2024-09-10T11:29:41.424 INFO:tasks.workunit.client.1.smithi178.stdout:3/993: write db/d9b/dfb/f131 [2112169,119782] 0 2024-09-10T11:29:41.425 INFO:tasks.workunit.client.0.smithi144.stdout:3/603: stat d4/d28/d67/d80/d17/d18/d34/d65/f88 0 2024-09-10T11:29:41.426 INFO:tasks.workunit.client.0.smithi144.stdout:5/626: dread d4/d5/f7d [0,4194304] 0 2024-09-10T11:29:41.427 INFO:tasks.workunit.client.0.smithi144.stdout:1/680: creat d5/d4a/d63/d84/d95/dd2/fde x:0 0 0 2024-09-10T11:29:41.428 INFO:tasks.workunit.client.0.smithi144.stdout:4/583: fdatasync d14/d1e/d22/d34/d3e/d40/fac 0 2024-09-10T11:29:41.428 INFO:tasks.workunit.client.1.smithi178.stdout:3/994: creat db/d36/f146 x:0 0 0 2024-09-10T11:29:41.429 INFO:tasks.workunit.client.1.smithi178.stdout:1/859: link d3/d9/d98/dac/d3a/d50/db5/dcc/cfc d3/d9/d21/d133/c13c 0 2024-09-10T11:29:41.429 INFO:tasks.workunit.client.0.smithi144.stdout:5/627: chown d4/d5/d18/d67/d44 4565530 1 2024-09-10T11:29:41.429 INFO:tasks.workunit.client.0.smithi144.stdout:8/465: getdents d0/d9 0 2024-09-10T11:29:41.429 INFO:tasks.workunit.client.1.smithi178.stdout:3/995: chown db/d20/d3e/ce6 4016092 1 2024-09-10T11:29:41.430 INFO:tasks.workunit.client.0.smithi144.stdout:0/421: unlink da/d3a/d4b/d61/f72 0 2024-09-10T11:29:41.430 INFO:tasks.workunit.client.1.smithi178.stdout:1/860: write d3/d9/d1a/f12d [1046541,95278] 0 2024-09-10T11:29:41.431 INFO:tasks.workunit.client.0.smithi144.stdout:2/444: creat d7/d29/f95 x:0 0 0 2024-09-10T11:29:41.432 INFO:tasks.workunit.client.1.smithi178.stdout:6/930: dread d3/d7/d8/d15/d1c/f1d [0,4194304] 0 2024-09-10T11:29:41.433 INFO:tasks.workunit.client.1.smithi178.stdout:3/996: fdatasync db/d20/d3d/d63/f101 0 2024-09-10T11:29:41.434 INFO:tasks.workunit.client.1.smithi178.stdout:1/861: read d3/d9/d21/d4b/d2b/fb9 [1351318,28753] 0 2024-09-10T11:29:41.434 INFO:tasks.workunit.client.1.smithi178.stdout:6/931: chown d3/d4e/ce8 31099656 1 2024-09-10T11:29:41.436 INFO:tasks.workunit.client.1.smithi178.stdout:6/932: fsync d3/d7/d8/d19/f26 0 2024-09-10T11:29:41.438 INFO:tasks.workunit.client.0.smithi144.stdout:9/584: mknod d11/d12/d1b/d41/d8f/dc0/dd0/cd1 0 2024-09-10T11:29:41.439 INFO:tasks.workunit.client.1.smithi178.stdout:3/997: dread db/d20/f10a [0,4194304] 0 2024-09-10T11:29:41.439 INFO:tasks.workunit.client.0.smithi144.stdout:7/576: dread - d5/dc/d2a/fac zero size 2024-09-10T11:29:41.440 INFO:tasks.workunit.client.0.smithi144.stdout:7/577: chown d5/dc/d14/f44 13 1 2024-09-10T11:29:41.440 INFO:tasks.workunit.client.0.smithi144.stdout:4/584: symlink d14/d1e/d83/lcf 0 2024-09-10T11:29:41.441 INFO:tasks.workunit.client.0.smithi144.stdout:1/681: mknod d5/d4a/d63/d84/d95/cdf 0 2024-09-10T11:29:41.442 INFO:tasks.workunit.client.1.smithi178.stdout:1/862: rename d3/d9/d21/fb7 to d3/d9/d98/dac/d3a/d50/db5/de3/d13a/f13d 0 2024-09-10T11:29:41.444 INFO:tasks.workunit.client.1.smithi178.stdout:6/933: dread - d3/d74/d91/daa/dc2/dcf/f105 zero size 2024-09-10T11:29:41.444 INFO:tasks.workunit.client.0.smithi144.stdout:0/422: mkdir da/d3a/d4b/d4e/d81 0 2024-09-10T11:29:41.446 INFO:tasks.workunit.client.0.smithi144.stdout:5/628: mkdir d4/d20/d22/d82/dbf 0 2024-09-10T11:29:41.447 INFO:tasks.workunit.client.0.smithi144.stdout:5/629: chown d4/d5/d18/d38/d6d/d8b 22121696 1 2024-09-10T11:29:41.447 INFO:tasks.workunit.client.0.smithi144.stdout:9/585: mkdir d11/d12/d1b/d2c/d78/dd2 0 2024-09-10T11:29:41.448 INFO:tasks.workunit.client.1.smithi178.stdout:3/998: rename db/d20/d70/lec to db/d26/d74/dd9/l147 0 2024-09-10T11:29:41.449 INFO:tasks.workunit.client.0.smithi144.stdout:7/578: chown d5/dc/d2a/f55 1226574 1 2024-09-10T11:29:41.451 INFO:tasks.workunit.client.0.smithi144.stdout:4/585: rename d14/d1e/d22/f47 to d14/d1e/d22/d34/d3e/fd0 0 2024-09-10T11:29:41.452 INFO:tasks.workunit.client.0.smithi144.stdout:3/604: dread d4/dd/d21/f5b [0,4194304] 0 2024-09-10T11:29:41.453 INFO:tasks.workunit.client.0.smithi144.stdout:2/445: unlink d7/de/d36/d45/c58 0 2024-09-10T11:29:41.454 INFO:tasks.workunit.client.0.smithi144.stdout:8/466: write d0/d1b/d47/d5e/f64 [2390683,42213] 0 2024-09-10T11:29:41.457 INFO:tasks.workunit.client.1.smithi178.stdout:6/934: mknod d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/d144/c150 0 2024-09-10T11:29:41.459 INFO:tasks.workunit.client.0.smithi144.stdout:7/579: creat d5/dc/d2a/d60/fb9 x:0 0 0 2024-09-10T11:29:41.460 INFO:tasks.workunit.client.1.smithi178.stdout:1/863: mknod d3/d9/d98/dac/d3a/dae/c13e 0 2024-09-10T11:29:41.461 INFO:tasks.workunit.client.0.smithi144.stdout:1/682: creat d5/d2a/fe0 x:0 0 0 2024-09-10T11:29:41.461 INFO:tasks.workunit.client.0.smithi144.stdout:2/446: truncate d7/de/d36/d45/d47/f88 317343 0 2024-09-10T11:29:41.462 INFO:tasks.workunit.client.0.smithi144.stdout:0/423: rename da/d6f to da/d44/d82 0 2024-09-10T11:29:41.463 INFO:tasks.workunit.client.1.smithi178.stdout:6/935: truncate d3/d74/d91/daa/dc2/d57/f6a 301721 0 2024-09-10T11:29:41.463 INFO:tasks.workunit.client.0.smithi144.stdout:5/630: symlink d4/d52/db1/lc0 0 2024-09-10T11:29:41.466 INFO:tasks.workunit.client.1.smithi178.stdout:3/999: truncate db/d20/f8a 3631826 0 2024-09-10T11:29:41.468 INFO:tasks.workunit.client.0.smithi144.stdout:3/605: dwrite d4/d28/d67/d80/d17/d18/d34/d65/f88 [0,4194304] 0 2024-09-10T11:29:41.468 INFO:tasks.workunit.client.0.smithi144.stdout:9/586: dread d11/d12/d1b/f20 [0,4194304] 0 2024-09-10T11:29:41.468 INFO:tasks.workunit.client.0.smithi144.stdout:8/467: rename d0/d1b/d5a/d90 to d0/d1b/d47/d94 0 2024-09-10T11:29:41.473 INFO:tasks.workunit.client.0.smithi144.stdout:1/683: dread - d5/dd/d64/d79/fb3 zero size 2024-09-10T11:29:41.473 INFO:tasks.workunit.client.0.smithi144.stdout:3/606: chown d4/d28/d67/d80/d1d/fac 173 1 2024-09-10T11:29:41.475 INFO:tasks.workunit.client.0.smithi144.stdout:0/424: rename da/d3a/d4f to da/d3a/d51/d5f/d83 0 2024-09-10T11:29:41.477 INFO:tasks.workunit.client.1.smithi178.stdout:1/864: dread d3/d9/fa [0,4194304] 0 2024-09-10T11:29:41.478 INFO:tasks.workunit.client.0.smithi144.stdout:5/631: unlink d4/d5/d18/d67/d44/f93 0 2024-09-10T11:29:41.480 INFO:tasks.workunit.client.0.smithi144.stdout:9/587: chown d11/d36/d2d/c9e 146 1 2024-09-10T11:29:41.485 INFO:tasks.workunit.client.0.smithi144.stdout:1/684: creat d5/dd/da1/fe1 x:0 0 0 2024-09-10T11:29:41.486 INFO:tasks.workunit.client.0.smithi144.stdout:1/685: dread - d5/df/d4b/fc8 zero size 2024-09-10T11:29:41.487 INFO:tasks.workunit.client.0.smithi144.stdout:3/607: mkdir d4/d28/dc1/dc4 0 2024-09-10T11:29:41.488 INFO:tasks.workunit.client.1.smithi178.stdout:1/865: dwrite d3/d9/d98/dac/d3a/d50/db5/de3/d13a/f13d [0,4194304] 0 2024-09-10T11:29:41.491 INFO:tasks.workunit.client.0.smithi144.stdout:7/580: rename d5/dc/d14/d40/d58/d6e/d7a/l80 to d5/dc/d14/d40/d58/d6a/lba 0 2024-09-10T11:29:41.491 INFO:tasks.workunit.client.1.smithi178.stdout:1/866: write d3/d9/d98/dac/d3a/d6e/f135 [4085417,21794] 0 2024-09-10T11:29:41.491 INFO:tasks.workunit.client.0.smithi144.stdout:2/447: truncate d7/da/f3c 201682 0 2024-09-10T11:29:41.492 INFO:tasks.workunit.client.0.smithi144.stdout:8/468: creat d0/d9/d1f/f95 x:0 0 0 2024-09-10T11:29:41.493 INFO:tasks.workunit.client.0.smithi144.stdout:0/425: creat da/d3a/d4b/d4e/d81/f84 x:0 0 0 2024-09-10T11:29:41.495 INFO:tasks.workunit.client.0.smithi144.stdout:3/608: creat d4/d28/d67/d9f/fc5 x:0 0 0 2024-09-10T11:29:41.497 INFO:tasks.workunit.client.1.smithi178.stdout:1/867: creat d3/da0/f13f x:0 0 0 2024-09-10T11:29:41.498 INFO:tasks.workunit.client.0.smithi144.stdout:1/686: symlink d5/df/le2 0 2024-09-10T11:29:41.501 INFO:tasks.workunit.client.0.smithi144.stdout:0/426: mknod da/d44/d82/c85 0 2024-09-10T11:29:41.501 INFO:tasks.workunit.client.1.smithi178.stdout:1/868: symlink d3/d9/d21/l140 0 2024-09-10T11:29:41.501 INFO:tasks.workunit.client.0.smithi144.stdout:7/581: mknod d5/dc/d12/d2b/cbb 0 2024-09-10T11:29:41.502 INFO:tasks.workunit.client.0.smithi144.stdout:8/469: mknod d0/d1b/d5a/d8d/c96 0 2024-09-10T11:29:41.502 INFO:tasks.workunit.client.0.smithi144.stdout:5/632: dread d4/d20/d22/d5c/d5d/d64/f96 [0,4194304] 0 2024-09-10T11:29:41.503 INFO:tasks.workunit.client.0.smithi144.stdout:2/448: creat d7/da/d5b/f96 x:0 0 0 2024-09-10T11:29:41.503 INFO:tasks.workunit.client.0.smithi144.stdout:8/470: chown d0/d1b/l6b 430772760 1 2024-09-10T11:29:41.504 INFO:tasks.workunit.client.0.smithi144.stdout:9/588: getdents d11/d36/db1 0 2024-09-10T11:29:41.505 INFO:tasks.workunit.client.0.smithi144.stdout:1/687: rename d5/df/d4b/d51/f59 to d5/d4a/d44/da2/fe3 0 2024-09-10T11:29:41.505 INFO:tasks.workunit.client.0.smithi144.stdout:8/471: chown d0/d9/d21/d25/l88 23600 1 2024-09-10T11:29:41.506 INFO:tasks.workunit.client.0.smithi144.stdout:2/449: dread - d7/de/d36/d45/f84 zero size 2024-09-10T11:29:41.507 INFO:tasks.workunit.client.0.smithi144.stdout:0/427: rmdir da/d3a/d4b/d61 39 2024-09-10T11:29:41.508 INFO:tasks.workunit.client.1.smithi178.stdout:1/869: link d3/d9/d98/dac/d3a/d50/d80/ld4 d3/d9/d21/d4b/d11a/l141 0 2024-09-10T11:29:41.509 INFO:tasks.workunit.client.0.smithi144.stdout:7/582: unlink d5/f38 0 2024-09-10T11:29:41.510 INFO:tasks.workunit.client.0.smithi144.stdout:7/583: fsync d5/dc/d12/f52 0 2024-09-10T11:29:41.512 INFO:tasks.workunit.client.0.smithi144.stdout:5/633: symlink d4/d5/d18/d38/d6d/d8f/lc1 0 2024-09-10T11:29:41.514 INFO:tasks.workunit.client.0.smithi144.stdout:9/589: creat d11/d12/d6f/fd3 x:0 0 0 2024-09-10T11:29:41.514 INFO:tasks.workunit.client.0.smithi144.stdout:5/634: dread - d4/d5/fbb zero size 2024-09-10T11:29:41.514 INFO:tasks.workunit.client.0.smithi144.stdout:8/472: dwrite d0/d9/d21/f70 [4194304,4194304] 0 2024-09-10T11:29:41.523 INFO:tasks.workunit.client.0.smithi144.stdout:9/590: rmdir d11/d36 39 2024-09-10T11:29:41.524 INFO:tasks.workunit.client.0.smithi144.stdout:5/635: readlink d4/d5/d32/l39 0 2024-09-10T11:29:41.529 INFO:tasks.workunit.client.0.smithi144.stdout:3/609: rename d4/d28/d67/d80/d17/d18/d34/db3 to d4/d28/d67/d80/d17/d18/d34/d4b/dc6 0 2024-09-10T11:29:41.529 INFO:tasks.workunit.client.0.smithi144.stdout:1/688: rename d5/d4a/d44/da2 to d5/d4a/d44/da2/de4 22 2024-09-10T11:29:41.530 INFO:tasks.workunit.client.0.smithi144.stdout:5/636: mkdir d4/d20/d22/d5c/d5d/d64/dc2 0 2024-09-10T11:29:41.530 INFO:tasks.workunit.client.0.smithi144.stdout:1/689: chown d5/d4a/d63/d84/d95 334271119 1 2024-09-10T11:29:41.533 INFO:tasks.workunit.client.0.smithi144.stdout:3/610: creat d4/d1a/fc7 x:0 0 0 2024-09-10T11:29:41.534 INFO:tasks.workunit.client.0.smithi144.stdout:8/473: link d0/d9/c52 d0/d9/d23/d26/c97 0 2024-09-10T11:29:41.535 INFO:tasks.workunit.client.0.smithi144.stdout:1/690: symlink d5/d4a/d44/da2/le5 0 2024-09-10T11:29:41.537 INFO:tasks.workunit.client.0.smithi144.stdout:0/428: dwrite da/f41 [0,4194304] 0 2024-09-10T11:29:41.538 INFO:tasks.workunit.client.0.smithi144.stdout:7/584: getdents d5/dc/d14/d40/d58/d6e 0 2024-09-10T11:29:41.540 INFO:tasks.workunit.client.0.smithi144.stdout:7/585: readlink d5/dc/d12/l36 0 2024-09-10T11:29:41.541 INFO:tasks.workunit.client.0.smithi144.stdout:5/637: truncate d4/d20/d24/f83 718553 0 2024-09-10T11:29:41.541 INFO:tasks.workunit.client.0.smithi144.stdout:9/591: dread d11/d12/d1b/d41/d8f/f87 [0,4194304] 0 2024-09-10T11:29:41.546 INFO:tasks.workunit.client.0.smithi144.stdout:3/611: mknod d4/dd/d21/cc8 0 2024-09-10T11:29:41.546 INFO:tasks.workunit.client.0.smithi144.stdout:9/592: chown d11/c35 46 1 2024-09-10T11:29:41.549 INFO:tasks.workunit.client.0.smithi144.stdout:8/474: dread d0/d9/d23/f89 [0,4194304] 0 2024-09-10T11:29:41.550 INFO:tasks.workunit.client.0.smithi144.stdout:5/638: rename d4/d5/d32/f68 to d4/d20/d24/da4/fc3 0 2024-09-10T11:29:41.553 INFO:tasks.workunit.client.0.smithi144.stdout:0/429: dwrite da/f1a [0,4194304] 0 2024-09-10T11:29:41.553 INFO:tasks.workunit.client.0.smithi144.stdout:3/612: mknod d4/d28/d67/d80/d1d/cc9 0 2024-09-10T11:29:41.555 INFO:tasks.workunit.client.0.smithi144.stdout:8/475: creat d0/d1b/d5a/d8d/f98 x:0 0 0 2024-09-10T11:29:41.555 INFO:tasks.workunit.client.0.smithi144.stdout:9/593: rename d11/d12/d1b/d22/l83 to d11/d36/db1/ld4 0 2024-09-10T11:29:41.561 INFO:tasks.workunit.client.0.smithi144.stdout:8/476: unlink d0/d9/d23/d37/c72 0 2024-09-10T11:29:41.563 INFO:tasks.workunit.client.0.smithi144.stdout:8/477: chown d0/d9/ff 53626 1 2024-09-10T11:29:41.565 INFO:tasks.workunit.client.0.smithi144.stdout:9/594: dread - d11/d12/d1b/d41/d8f/dab/fb8 zero size 2024-09-10T11:29:41.567 INFO:tasks.workunit.client.0.smithi144.stdout:2/450: dwrite d7/da/f3c [0,4194304] 0 2024-09-10T11:29:41.568 INFO:tasks.workunit.client.0.smithi144.stdout:0/430: getdents da/d3a/d4b 0 2024-09-10T11:29:41.569 INFO:tasks.workunit.client.0.smithi144.stdout:5/639: dread d4/d20/d22/d5c/d5d/d64/f96 [0,4194304] 0 2024-09-10T11:29:41.569 INFO:tasks.workunit.client.0.smithi144.stdout:8/478: chown d0/d9/d21/f59 57094492 1 2024-09-10T11:29:41.569 INFO:tasks.workunit.client.0.smithi144.stdout:3/613: rmdir d4/d28/d67/d80/d17/d18/d34/d4b/dc6 0 2024-09-10T11:29:41.571 INFO:tasks.workunit.client.0.smithi144.stdout:2/451: mkdir d7/d29/d97 0 2024-09-10T11:29:41.571 INFO:tasks.workunit.client.0.smithi144.stdout:9/595: dread d11/d12/fa1 [0,4194304] 0 2024-09-10T11:29:41.573 INFO:tasks.workunit.client.0.smithi144.stdout:0/431: creat da/d3a/d4b/f86 x:0 0 0 2024-09-10T11:29:41.573 INFO:tasks.workunit.client.0.smithi144.stdout:7/586: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:41.575 INFO:tasks.workunit.client.0.smithi144.stdout:2/452: chown d7/de/d3d/d43/c5a 1726557169 1 2024-09-10T11:29:41.576 INFO:tasks.workunit.client.0.smithi144.stdout:8/479: rename d0/d9/d21/f86 to d0/d9/d1f/d34/f99 0 2024-09-10T11:29:41.577 INFO:tasks.workunit.client.0.smithi144.stdout:2/453: write d7/de/d36/f51 [4236880,31965] 0 2024-09-10T11:29:41.578 INFO:tasks.workunit.client.0.smithi144.stdout:2/454: chown d7/de/d3d/c4d 0 1 2024-09-10T11:29:41.583 INFO:tasks.workunit.client.0.smithi144.stdout:0/432: dread da/f1a [0,4194304] 0 2024-09-10T11:29:41.584 INFO:tasks.workunit.client.0.smithi144.stdout:5/640: creat d4/d5/d32/fc4 x:0 0 0 2024-09-10T11:29:41.586 INFO:tasks.workunit.client.0.smithi144.stdout:8/480: symlink d0/d9/d23/d37/l9a 0 2024-09-10T11:29:41.591 INFO:tasks.workunit.client.0.smithi144.stdout:1/691: dwrite d5/df/d4b/f8e [0,4194304] 0 2024-09-10T11:29:41.591 INFO:tasks.workunit.client.0.smithi144.stdout:0/433: chown da/l1b 229 1 2024-09-10T11:29:41.592 INFO:tasks.workunit.client.0.smithi144.stdout:7/587: dwrite d5/dc/d12/f22 [0,4194304] 0 2024-09-10T11:29:41.595 INFO:tasks.workunit.client.0.smithi144.stdout:9/596: dread d11/d12/d1b/d41/d8f/f93 [0,4194304] 0 2024-09-10T11:29:41.597 INFO:tasks.workunit.client.0.smithi144.stdout:2/455: creat d7/de/d36/d45/d47/d6c/d7c/d82/f98 x:0 0 0 2024-09-10T11:29:41.602 INFO:tasks.workunit.client.0.smithi144.stdout:8/481: creat d0/d84/f9b x:0 0 0 2024-09-10T11:29:41.602 INFO:tasks.workunit.client.0.smithi144.stdout:5/641: dread d4/d20/d22/d5c/d5d/d64/f96 [0,4194304] 0 2024-09-10T11:29:41.604 INFO:tasks.workunit.client.0.smithi144.stdout:1/692: write d5/d4a/d63/d84/f9e [481478,39822] 0 2024-09-10T11:29:41.607 INFO:tasks.workunit.client.0.smithi144.stdout:1/693: stat d5/dd/f1d 0 2024-09-10T11:29:41.611 INFO:tasks.workunit.client.0.smithi144.stdout:5/642: unlink d4/d20/d24/d2d/l57 0 2024-09-10T11:29:41.613 INFO:tasks.workunit.client.0.smithi144.stdout:8/482: rmdir d0/d9/d21/d25/d4f 39 2024-09-10T11:29:41.615 INFO:tasks.workunit.client.0.smithi144.stdout:1/694: dread d5/dd/d64/f7a [0,4194304] 0 2024-09-10T11:29:41.616 INFO:tasks.workunit.client.1.smithi178.stdout:8/983: sync 2024-09-10T11:29:41.616 INFO:tasks.workunit.client.1.smithi178.stdout:7/810: sync 2024-09-10T11:29:41.616 INFO:tasks.workunit.client.1.smithi178.stdout:4/802: sync 2024-09-10T11:29:41.616 INFO:tasks.workunit.client.1.smithi178.stdout:0/849: sync 2024-09-10T11:29:41.616 INFO:tasks.workunit.client.1.smithi178.stdout:9/949: sync 2024-09-10T11:29:41.617 INFO:tasks.workunit.client.0.smithi144.stdout:2/456: rename d7/da/d1e/d21/f7a to d7/de/d36/f99 0 2024-09-10T11:29:41.618 INFO:tasks.workunit.client.0.smithi144.stdout:8/483: mknod d0/c9c 0 2024-09-10T11:29:41.619 INFO:tasks.workunit.client.0.smithi144.stdout:5/643: creat d4/d5/d18/d38/fc5 x:0 0 0 2024-09-10T11:29:41.619 INFO:tasks.workunit.client.0.smithi144.stdout:9/597: getdents d11/d12/d6f 0 2024-09-10T11:29:41.621 INFO:tasks.workunit.client.0.smithi144.stdout:8/484: fdatasync d0/d9/ff 0 2024-09-10T11:29:41.623 INFO:tasks.workunit.client.1.smithi178.stdout:0/850: creat d0/dc/d39/d75/f10d x:0 0 0 2024-09-10T11:29:41.623 INFO:tasks.workunit.client.1.smithi178.stdout:0/851: chown d0/d12/c20 848037900 1 2024-09-10T11:29:41.628 INFO:tasks.workunit.client.1.smithi178.stdout:7/811: creat d6/d10/d1d/daa/d11c/f11f x:0 0 0 2024-09-10T11:29:41.629 INFO:tasks.workunit.client.1.smithi178.stdout:0/852: mknod d0/dc/d39/d75/c10e 0 2024-09-10T11:29:41.631 INFO:tasks.workunit.client.1.smithi178.stdout:0/853: fdatasync d0/d2c/d89/da1/ffb 0 2024-09-10T11:29:41.632 INFO:tasks.workunit.client.1.smithi178.stdout:9/950: dwrite d2/d19/d5e/f78 [0,4194304] 0 2024-09-10T11:29:41.632 INFO:tasks.workunit.client.0.smithi144.stdout:8/485: creat d0/d1b/d47/d94/f9d x:0 0 0 2024-09-10T11:29:41.633 INFO:tasks.workunit.client.1.smithi178.stdout:7/812: creat d6/d10/d1d/d22/d50/def/df3/f120 x:0 0 0 2024-09-10T11:29:41.634 INFO:tasks.workunit.client.1.smithi178.stdout:0/854: rmdir d0/d12/d3f 39 2024-09-10T11:29:41.634 INFO:tasks.workunit.client.0.smithi144.stdout:2/457: dwrite d7/da/d1e/f66 [0,4194304] 0 2024-09-10T11:29:41.635 INFO:tasks.workunit.client.1.smithi178.stdout:7/813: write d6/d10/d1d/d22/d50/def/df3/f120 [386818,70769] 0 2024-09-10T11:29:41.636 INFO:tasks.workunit.client.1.smithi178.stdout:0/855: read - d0/dc/d1f/d2d/fb5 zero size 2024-09-10T11:29:41.639 INFO:tasks.workunit.client.1.smithi178.stdout:7/814: dread - d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d70/da2/f11b zero size 2024-09-10T11:29:41.639 INFO:tasks.workunit.client.0.smithi144.stdout:5/644: write d4/d52/f69 [796467,124012] 0 2024-09-10T11:29:41.641 INFO:tasks.workunit.client.0.smithi144.stdout:7/588: dwrite d5/dc/d14/d40/f67 [0,4194304] 0 2024-09-10T11:29:41.641 INFO:tasks.workunit.client.1.smithi178.stdout:9/951: dwrite d2/d9b/df8/d2d/dab/d118/d56/de1/ff5 [0,4194304] 0 2024-09-10T11:29:41.642 INFO:tasks.workunit.client.0.smithi144.stdout:0/434: dread da/d3a/f50 [0,4194304] 0 2024-09-10T11:29:41.643 INFO:tasks.workunit.client.1.smithi178.stdout:9/952: chown d2/dc/l74 43 1 2024-09-10T11:29:41.644 INFO:tasks.workunit.client.1.smithi178.stdout:0/856: mknod d0/dc/d1f/d2d/dac/de7/c10f 0 2024-09-10T11:29:41.645 INFO:tasks.workunit.client.1.smithi178.stdout:7/815: dread - d6/d10/d1d/d22/d93/db7/fec zero size 2024-09-10T11:29:41.645 INFO:tasks.workunit.client.0.smithi144.stdout:2/458: rmdir d7/d29/d92 39 2024-09-10T11:29:41.647 INFO:tasks.workunit.client.1.smithi178.stdout:7/816: chown d6/d10/d1d/f73 30676 1 2024-09-10T11:29:41.650 INFO:tasks.workunit.client.0.smithi144.stdout:5/645: chown d4/l1a 11843 1 2024-09-10T11:29:41.650 INFO:tasks.workunit.client.1.smithi178.stdout:9/953: symlink d2/d9b/df8/d2d/dab/d118/d89/d93/dc2/l12c 0 2024-09-10T11:29:41.651 INFO:tasks.workunit.client.0.smithi144.stdout:8/486: dwrite d0/d9/d21/f5c [4194304,4194304] 0 2024-09-10T11:29:41.652 INFO:tasks.workunit.client.1.smithi178.stdout:0/857: fsync d0/dc/d1f/f26 0 2024-09-10T11:29:41.653 INFO:tasks.workunit.client.1.smithi178.stdout:4/803: dread d0/d8/da/fd3 [0,4194304] 0 2024-09-10T11:29:41.661 INFO:tasks.workunit.client.1.smithi178.stdout:4/804: truncate d0/d8/da/fed 871794 0 2024-09-10T11:29:41.662 INFO:tasks.workunit.client.1.smithi178.stdout:4/805: chown d0/d9/dc 128499 1 2024-09-10T11:29:41.663 INFO:tasks.workunit.client.1.smithi178.stdout:0/858: dwrite d0/d12/f2f [4194304,4194304] 0 2024-09-10T11:29:41.663 INFO:tasks.workunit.client.0.smithi144.stdout:1/695: dread d5/df/f55 [0,4194304] 0 2024-09-10T11:29:41.665 INFO:tasks.workunit.client.0.smithi144.stdout:2/459: rename d7/da/d1e/d21 to d7/de/d36/d45/d8e/d9a 0 2024-09-10T11:29:41.665 INFO:tasks.workunit.client.0.smithi144.stdout:9/598: dread d11/d12/d1b/d41/f5c [0,4194304] 0 2024-09-10T11:29:41.666 INFO:tasks.workunit.client.0.smithi144.stdout:5/646: dwrite d4/d20/f43 [0,4194304] 0 2024-09-10T11:29:41.668 INFO:tasks.workunit.client.0.smithi144.stdout:9/599: fsync d11/d36/f7f 0 2024-09-10T11:29:41.669 INFO:tasks.workunit.client.0.smithi144.stdout:0/435: dread da/f28 [0,4194304] 0 2024-09-10T11:29:41.670 INFO:tasks.workunit.client.1.smithi178.stdout:7/817: getdents d6/d10/d1d/d4a/d79 0 2024-09-10T11:29:41.671 INFO:tasks.workunit.client.0.smithi144.stdout:5/647: mknod d4/d20/d24/d6f/d84/cc6 0 2024-09-10T11:29:41.671 INFO:tasks.workunit.client.1.smithi178.stdout:9/954: getdents d2/d19 0 2024-09-10T11:29:41.673 INFO:tasks.workunit.client.0.smithi144.stdout:5/648: truncate d4/d5/d18/d38/d8d/fa8 1010196 0 2024-09-10T11:29:41.675 INFO:tasks.workunit.client.0.smithi144.stdout:9/600: mknod d11/d36/cd5 0 2024-09-10T11:29:41.675 INFO:tasks.workunit.client.0.smithi144.stdout:7/589: truncate d5/dc/d12/f52 1850022 0 2024-09-10T11:29:41.675 INFO:tasks.workunit.client.0.smithi144.stdout:0/436: fsync da/d44/f63 0 2024-09-10T11:29:41.676 INFO:tasks.workunit.client.0.smithi144.stdout:1/696: symlink d5/d2a/d65/le6 0 2024-09-10T11:29:41.679 INFO:tasks.workunit.client.0.smithi144.stdout:1/697: write d5/dd/d64/f9a [3927069,104832] 0 2024-09-10T11:29:41.680 INFO:tasks.workunit.client.0.smithi144.stdout:7/590: creat d5/dc/d14/d40/d5e/d68/fbc x:0 0 0 2024-09-10T11:29:41.681 INFO:tasks.workunit.client.0.smithi144.stdout:5/649: chown d4/d5/l19 14987 1 2024-09-10T11:29:41.681 INFO:tasks.workunit.client.0.smithi144.stdout:0/437: write da/f30 [357088,127718] 0 2024-09-10T11:29:41.682 INFO:tasks.workunit.client.0.smithi144.stdout:2/460: creat d7/de/d36/f9b x:0 0 0 2024-09-10T11:29:41.684 INFO:tasks.workunit.client.0.smithi144.stdout:5/650: readlink d4/d5/l11 0 2024-09-10T11:29:41.686 INFO:tasks.workunit.client.0.smithi144.stdout:8/487: truncate d0/d9/f1d 261848 0 2024-09-10T11:29:41.686 INFO:tasks.workunit.client.0.smithi144.stdout:1/698: mknod d5/d2a/d65/d73/ce7 0 2024-09-10T11:29:41.688 INFO:tasks.workunit.client.0.smithi144.stdout:7/591: symlink d5/dc/d14/d40/d58/d6e/lbd 0 2024-09-10T11:29:41.689 INFO:tasks.workunit.client.0.smithi144.stdout:0/438: write da/d3a/d4b/d4e/d81/f84 [281209,129534] 0 2024-09-10T11:29:41.689 INFO:tasks.workunit.client.0.smithi144.stdout:5/651: truncate d4/d20/d24/d2d/f2e 173533 0 2024-09-10T11:29:41.691 INFO:tasks.workunit.client.0.smithi144.stdout:1/699: creat d5/d4a/d63/d84/fe8 x:0 0 0 2024-09-10T11:29:41.691 INFO:tasks.workunit.client.0.smithi144.stdout:7/592: chown d5/dc/d11/c76 4370879 1 2024-09-10T11:29:41.692 INFO:tasks.workunit.client.0.smithi144.stdout:8/488: write d0/d9/d23/d26/f30 [1841801,28862] 0 2024-09-10T11:29:41.692 INFO:tasks.workunit.client.0.smithi144.stdout:2/461: dread d7/de/f3b [0,4194304] 0 2024-09-10T11:29:41.694 INFO:tasks.workunit.client.0.smithi144.stdout:8/489: dread - d0/d9/d23/d26/f69 zero size 2024-09-10T11:29:41.699 INFO:tasks.workunit.client.0.smithi144.stdout:2/462: creat d7/de/d36/d45/d8e/d9a/f9c x:0 0 0 2024-09-10T11:29:41.699 INFO:tasks.workunit.client.0.smithi144.stdout:2/463: chown d7/de/d3d/c4d 1308634267 1 2024-09-10T11:29:41.701 INFO:tasks.workunit.client.0.smithi144.stdout:8/490: rename d0/d9/d23/d26/f30 to d0/d9/d21/f9e 0 2024-09-10T11:29:41.701 INFO:tasks.workunit.client.0.smithi144.stdout:7/593: dread d5/dc/d11/f17 [0,4194304] 0 2024-09-10T11:29:41.702 INFO:tasks.workunit.client.0.smithi144.stdout:2/464: fdatasync d7/de/d36/d45/d8e/d9a/d54/f8a 0 2024-09-10T11:29:41.703 INFO:tasks.workunit.client.0.smithi144.stdout:8/491: write d0/f13 [5072811,21800] 0 2024-09-10T11:29:41.703 INFO:tasks.workunit.client.0.smithi144.stdout:7/594: dread - d5/dc/d14/d40/d5e/f6f zero size 2024-09-10T11:29:41.708 INFO:tasks.workunit.client.0.smithi144.stdout:8/492: write d0/f10 [1864716,84125] 0 2024-09-10T11:29:41.709 INFO:tasks.workunit.client.0.smithi144.stdout:0/439: dread da/f19 [0,4194304] 0 2024-09-10T11:29:41.710 INFO:tasks.workunit.client.0.smithi144.stdout:7/595: creat d5/dc/d14/d40/d58/d6a/fbe x:0 0 0 2024-09-10T11:29:41.712 INFO:tasks.workunit.client.0.smithi144.stdout:8/493: dread - d0/d1b/d47/d94/f9d zero size 2024-09-10T11:29:41.713 INFO:tasks.workunit.client.0.smithi144.stdout:2/465: dwrite d7/de/d36/d45/d8e/d9a/f9c [0,4194304] 0 2024-09-10T11:29:41.715 INFO:tasks.workunit.client.0.smithi144.stdout:8/494: truncate d0/d9/d1f/f95 684453 0 2024-09-10T11:29:41.716 INFO:tasks.workunit.client.0.smithi144.stdout:5/652: write d4/d20/d24/f83 [1705333,116775] 0 2024-09-10T11:29:41.717 INFO:tasks.workunit.client.0.smithi144.stdout:2/466: chown d7/d29/f95 24685 1 2024-09-10T11:29:41.718 INFO:tasks.workunit.client.0.smithi144.stdout:1/700: write d5/d4a/d44/f45 [4411539,9255] 0 2024-09-10T11:29:41.720 INFO:tasks.workunit.client.0.smithi144.stdout:1/701: dread d5/dd/f48 [0,4194304] 0 2024-09-10T11:29:41.725 INFO:tasks.workunit.client.0.smithi144.stdout:5/653: mkdir d4/d5/d56/d8a/dc7 0 2024-09-10T11:29:41.726 INFO:tasks.workunit.client.1.smithi178.stdout:0/859: creat d0/db1/f110 x:0 0 0 2024-09-10T11:29:41.726 INFO:tasks.workunit.client.0.smithi144.stdout:5/654: chown d4/d5/d18/d51/f88 465327424 1 2024-09-10T11:29:41.727 INFO:tasks.workunit.client.0.smithi144.stdout:5/655: dread - d4/d5/fbb zero size 2024-09-10T11:29:41.727 INFO:tasks.workunit.client.0.smithi144.stdout:2/467: getdents d7/de/d5f 0 2024-09-10T11:29:41.728 INFO:tasks.workunit.client.0.smithi144.stdout:8/495: truncate d0/f3f 2240872 0 2024-09-10T11:29:41.728 INFO:tasks.workunit.client.1.smithi178.stdout:4/806: symlink d0/d9/dc/d8e/l11b 0 2024-09-10T11:29:41.729 INFO:tasks.workunit.client.1.smithi178.stdout:0/860: dread - d0/dc/d39/f9c zero size 2024-09-10T11:29:41.730 INFO:tasks.workunit.client.0.smithi144.stdout:2/468: write d7/f15 [6018316,69074] 0 2024-09-10T11:29:41.731 INFO:tasks.workunit.client.1.smithi178.stdout:7/818: creat d6/d10/d1d/d22/f121 x:0 0 0 2024-09-10T11:29:41.731 INFO:tasks.workunit.client.0.smithi144.stdout:8/496: write d0/d9/d21/f71 [420604,33097] 0 2024-09-10T11:29:41.733 INFO:tasks.workunit.client.1.smithi178.stdout:7/819: truncate d6/d10/d1d/d22/fbb 1279315 0 2024-09-10T11:29:41.734 INFO:tasks.workunit.client.0.smithi144.stdout:2/469: mknod d7/de/d36/d45/d8e/c9d 0 2024-09-10T11:29:41.735 INFO:tasks.workunit.client.1.smithi178.stdout:0/861: creat d0/d2c/d93/f111 x:0 0 0 2024-09-10T11:29:41.735 INFO:tasks.workunit.client.1.smithi178.stdout:9/955: getdents d2/d13/d75/d60/d66 0 2024-09-10T11:29:41.735 INFO:tasks.workunit.client.0.smithi144.stdout:2/470: readlink d7/da/l24 0 2024-09-10T11:29:41.737 INFO:tasks.workunit.client.0.smithi144.stdout:0/440: dread da/d3a/f4d [0,4194304] 0 2024-09-10T11:29:41.737 INFO:tasks.workunit.client.0.smithi144.stdout:2/471: readlink d7/de/d3d/d4e/l78 0 2024-09-10T11:29:41.737 INFO:tasks.workunit.client.0.smithi144.stdout:8/497: dread d0/d9/d21/f71 [0,4194304] 0 2024-09-10T11:29:41.738 INFO:tasks.workunit.client.0.smithi144.stdout:0/441: readlink da/l1b 0 2024-09-10T11:29:41.738 INFO:tasks.workunit.client.1.smithi178.stdout:9/956: creat d2/d9b/df8/d2d/dab/d118/d89/f12d x:0 0 0 2024-09-10T11:29:41.739 INFO:tasks.workunit.client.0.smithi144.stdout:2/472: write d7/da/f3c [1804416,76004] 0 2024-09-10T11:29:41.740 INFO:tasks.workunit.client.0.smithi144.stdout:5/656: truncate d4/d5/d18/f1f 1462615 0 2024-09-10T11:29:41.743 INFO:tasks.workunit.client.0.smithi144.stdout:8/498: symlink d0/d1b/d5a/l9f 0 2024-09-10T11:29:41.744 INFO:tasks.workunit.client.0.smithi144.stdout:8/499: write d0/d9/d23/f87 [192454,120597] 0 2024-09-10T11:29:41.745 INFO:tasks.workunit.client.0.smithi144.stdout:0/442: fdatasync da/f36 0 2024-09-10T11:29:41.747 INFO:tasks.workunit.client.0.smithi144.stdout:2/473: mkdir d7/da/d9e 0 2024-09-10T11:29:41.749 INFO:tasks.workunit.client.0.smithi144.stdout:8/500: mkdir d0/d9/d1f/d34/da0 0 2024-09-10T11:29:41.751 INFO:tasks.workunit.client.0.smithi144.stdout:5/657: creat d4/d20/d24/fc8 x:0 0 0 2024-09-10T11:29:41.752 INFO:tasks.workunit.client.0.smithi144.stdout:5/658: fdatasync d4/d20/d24/fc8 0 2024-09-10T11:29:41.755 INFO:tasks.workunit.client.0.smithi144.stdout:5/659: creat d4/d20/d24/d2d/fc9 x:0 0 0 2024-09-10T11:29:41.756 INFO:tasks.workunit.client.0.smithi144.stdout:8/501: mknod d0/d9/d21/d25/d4f/d93/ca1 0 2024-09-10T11:29:41.756 INFO:tasks.workunit.client.0.smithi144.stdout:2/474: getdents d7/de/d36/d45/d8e/d9a/d54/d61 0 2024-09-10T11:29:41.757 INFO:tasks.workunit.client.1.smithi178.stdout:6/936: sync 2024-09-10T11:29:41.758 INFO:tasks.workunit.client.0.smithi144.stdout:5/660: rename d4/d20/d22/d82 to d4/d20/d22/d5c/d5d/d64/dc2/dca 0 2024-09-10T11:29:41.759 INFO:tasks.workunit.client.0.smithi144.stdout:2/475: unlink d7/d42/l75 0 2024-09-10T11:29:41.761 INFO:tasks.workunit.client.0.smithi144.stdout:5/661: read d4/d20/d24/f3e [2209238,71591] 0 2024-09-10T11:29:41.765 INFO:tasks.workunit.client.0.smithi144.stdout:8/502: dwrite d0/d9/d23/d26/d7d/f81 [0,4194304] 0 2024-09-10T11:29:41.766 INFO:tasks.workunit.client.0.smithi144.stdout:8/503: dread - d0/d9/d21/d25/d4f/f6f zero size 2024-09-10T11:29:41.767 INFO:tasks.workunit.client.1.smithi178.stdout:9/957: dwrite d2/d9b/df8/da5/f115 [0,4194304] 0 2024-09-10T11:29:41.770 INFO:tasks.workunit.client.1.smithi178.stdout:7/820: dread d6/d10/d1d/d22/d3b/f65 [0,4194304] 0 2024-09-10T11:29:41.770 INFO:tasks.workunit.client.0.smithi144.stdout:5/662: dwrite d4/d5/d32/d47/f6a [0,4194304] 0 2024-09-10T11:29:41.771 INFO:tasks.workunit.client.0.smithi144.stdout:8/504: mknod d0/d9/d21/ca2 0 2024-09-10T11:29:41.772 INFO:tasks.workunit.client.1.smithi178.stdout:9/958: read - d2/d8/ff0 zero size 2024-09-10T11:29:41.772 INFO:tasks.workunit.client.0.smithi144.stdout:2/476: write d7/d29/f3f [2347832,55220] 0 2024-09-10T11:29:41.774 INFO:tasks.workunit.client.0.smithi144.stdout:2/477: dread - d7/de/d36/f9b zero size 2024-09-10T11:29:41.774 INFO:tasks.workunit.client.1.smithi178.stdout:6/937: mkdir d3/d7/d8/d151 0 2024-09-10T11:29:41.774 INFO:tasks.workunit.client.1.smithi178.stdout:9/959: write d2/df/f121 [534091,29095] 0 2024-09-10T11:29:41.775 INFO:tasks.workunit.client.1.smithi178.stdout:4/807: dwrite d0/d9/dc/d8e/dbf/f93 [0,4194304] 0 2024-09-10T11:29:41.777 INFO:tasks.workunit.client.1.smithi178.stdout:7/821: creat d6/d10/d1e/d2d/d2b/da0/d55/dbd/f122 x:0 0 0 2024-09-10T11:29:41.778 INFO:tasks.workunit.client.0.smithi144.stdout:2/478: mknod d7/de/d36/d45/d8e/c9f 0 2024-09-10T11:29:41.778 INFO:tasks.workunit.client.1.smithi178.stdout:6/938: creat d3/d74/d91/daa/f152 x:0 0 0 2024-09-10T11:29:41.780 INFO:tasks.workunit.client.0.smithi144.stdout:5/663: link d4/d52/fad d4/d5/d18/d38/d6d/fcb 0 2024-09-10T11:29:41.781 INFO:tasks.workunit.client.1.smithi178.stdout:9/960: truncate d2/d9b/df8/d2a/f92 4680272 0 2024-09-10T11:29:41.781 INFO:tasks.workunit.client.1.smithi178.stdout:6/939: chown d3/d74/dc9/da6/d11e/l126 44697 1 2024-09-10T11:29:41.782 INFO:tasks.workunit.client.1.smithi178.stdout:7/822: rename d6/d10/d1d/dca to d6/d10/d1d/d22/d50/d7e/d123 0 2024-09-10T11:29:41.784 INFO:tasks.workunit.client.0.smithi144.stdout:2/479: mkdir d7/de/d36/d45/d47/da0 0 2024-09-10T11:29:41.786 INFO:tasks.workunit.client.0.smithi144.stdout:8/505: getdents d0/d9/d23/d37 0 2024-09-10T11:29:41.788 INFO:tasks.workunit.client.1.smithi178.stdout:7/823: mknod d6/d10/d1d/d22/d50/d98/da6/c124 0 2024-09-10T11:29:41.789 INFO:tasks.workunit.client.0.smithi144.stdout:5/664: dwrite d4/d20/f34 [0,4194304] 0 2024-09-10T11:29:41.790 INFO:tasks.workunit.client.1.smithi178.stdout:9/961: getdents d2/d9b/df8/d2d/dab/d118/d56/d9a 0 2024-09-10T11:29:41.791 INFO:tasks.workunit.client.1.smithi178.stdout:6/940: dwrite d3/d6e/ff1 [0,4194304] 0 2024-09-10T11:29:41.791 INFO:tasks.workunit.client.1.smithi178.stdout:7/824: creat d6/d10/d1d/daa/d11c/f125 x:0 0 0 2024-09-10T11:29:41.793 INFO:tasks.workunit.client.1.smithi178.stdout:6/941: write d3/d24/f81 [4098095,49541] 0 2024-09-10T11:29:41.795 INFO:tasks.workunit.client.1.smithi178.stdout:4/808: write d0/d8/da/d1a/d51/fb0 [896755,18864] 0 2024-09-10T11:29:41.801 INFO:tasks.workunit.client.0.smithi144.stdout:8/506: symlink d0/d1b/d47/d5e/la3 0 2024-09-10T11:29:41.801 INFO:tasks.workunit.client.0.smithi144.stdout:8/507: stat d0/d9/d21/d7e/d85 0 2024-09-10T11:29:41.803 INFO:tasks.workunit.client.1.smithi178.stdout:9/962: dwrite d2/d8/fc5 [0,4194304] 0 2024-09-10T11:29:41.805 INFO:tasks.workunit.client.0.smithi144.stdout:2/480: dread d7/de/d3d/f50 [0,4194304] 0 2024-09-10T11:29:41.806 INFO:tasks.workunit.client.0.smithi144.stdout:2/481: fdatasync d7/de/f22 0 2024-09-10T11:29:41.806 INFO:tasks.workunit.client.0.smithi144.stdout:8/508: mkdir d0/d9/d23/da4 0 2024-09-10T11:29:41.807 INFO:tasks.workunit.client.1.smithi178.stdout:9/963: rmdir d2/d9b/df8/d2d/dab/d118/d89/d93/d88 39 2024-09-10T11:29:41.813 INFO:tasks.workunit.client.0.smithi144.stdout:2/482: mknod d7/de/d36/d45/d8e/d9a/ca1 0 2024-09-10T11:29:41.813 INFO:tasks.workunit.client.0.smithi144.stdout:5/665: getdents d4/d5/d18/d38 0 2024-09-10T11:29:41.815 INFO:tasks.workunit.client.0.smithi144.stdout:8/509: dread d0/d9/d23/f40 [0,4194304] 0 2024-09-10T11:29:41.815 INFO:tasks.workunit.client.1.smithi178.stdout:7/825: write d6/d10/d1e/d2d/d2b/da0/d55/f91 [5016584,130714] 0 2024-09-10T11:29:41.816 INFO:tasks.workunit.client.0.smithi144.stdout:2/483: unlink d7/d29/f95 0 2024-09-10T11:29:41.816 INFO:tasks.workunit.client.0.smithi144.stdout:8/510: dread - d0/d9/d1f/f8f zero size 2024-09-10T11:29:41.816 INFO:tasks.workunit.client.1.smithi178.stdout:6/942: write d3/d74/dc9/da6/f108 [939525,114563] 0 2024-09-10T11:29:41.817 INFO:tasks.workunit.client.0.smithi144.stdout:5/666: creat d4/d5/d56/fcc x:0 0 0 2024-09-10T11:29:41.821 INFO:tasks.workunit.client.0.smithi144.stdout:8/511: symlink d0/d1b/d5a/la5 0 2024-09-10T11:29:41.821 INFO:tasks.workunit.client.1.smithi178.stdout:7/826: mkdir d6/d10/d34/d6a/d126 0 2024-09-10T11:29:41.822 INFO:tasks.workunit.client.0.smithi144.stdout:8/512: rename d0/d9/d23 to d0/d9/d23/d26/da6 22 2024-09-10T11:29:41.823 INFO:tasks.workunit.client.1.smithi178.stdout:9/964: write d2/dc/fcf [733567,71463] 0 2024-09-10T11:29:41.824 INFO:tasks.workunit.client.1.smithi178.stdout:4/809: dwrite d0/d9/dc/d8e/dbf/fb5 [0,4194304] 0 2024-09-10T11:29:41.825 INFO:tasks.workunit.client.0.smithi144.stdout:5/667: dwrite d4/d5/d18/d38/d6d/f70 [4194304,4194304] 0 2024-09-10T11:29:41.826 INFO:tasks.workunit.client.0.smithi144.stdout:8/513: symlink d0/d9/d21/la7 0 2024-09-10T11:29:41.826 INFO:tasks.workunit.client.1.smithi178.stdout:9/965: write d2/d51/d107/f10b [538649,121339] 0 2024-09-10T11:29:41.827 INFO:tasks.workunit.client.1.smithi178.stdout:6/943: dread d3/d7/d35/f5a [0,4194304] 0 2024-09-10T11:29:41.827 INFO:tasks.workunit.client.1.smithi178.stdout:4/810: dread - d0/d8/da/f101 zero size 2024-09-10T11:29:41.828 INFO:tasks.workunit.client.0.smithi144.stdout:8/514: write d0/d9/d23/d26/f67 [779341,106854] 0 2024-09-10T11:29:41.829 INFO:tasks.workunit.client.0.smithi144.stdout:8/515: readlink d0/d1b/l49 0 2024-09-10T11:29:41.830 INFO:tasks.workunit.client.1.smithi178.stdout:7/827: unlink d6/d10/d1d/d22/d50/d7e/d82/db2/d49/l63 0 2024-09-10T11:29:41.830 INFO:tasks.workunit.client.0.smithi144.stdout:5/668: rmdir d4/d20/d22/d5c/d5d/d64/dc2 39 2024-09-10T11:29:41.830 INFO:tasks.workunit.client.0.smithi144.stdout:5/669: readlink d4/l1a 0 2024-09-10T11:29:41.831 INFO:tasks.workunit.client.1.smithi178.stdout:6/944: creat d3/d4e/f153 x:0 0 0 2024-09-10T11:29:41.833 INFO:tasks.workunit.client.0.smithi144.stdout:8/516: creat d0/d9/d23/d26/d7d/fa8 x:0 0 0 2024-09-10T11:29:41.834 INFO:tasks.workunit.client.1.smithi178.stdout:9/966: mkdir d2/d9b/df8/d12e 0 2024-09-10T11:29:41.835 INFO:tasks.workunit.client.0.smithi144.stdout:8/517: dread - d0/d1b/d47/d5e/d76/f7c zero size 2024-09-10T11:29:41.836 INFO:tasks.workunit.client.0.smithi144.stdout:5/670: dread d4/d5/f25 [0,4194304] 0 2024-09-10T11:29:41.836 INFO:tasks.workunit.client.0.smithi144.stdout:2/484: dwrite d7/de/f2f [0,4194304] 0 2024-09-10T11:29:41.837 INFO:tasks.workunit.client.1.smithi178.stdout:4/811: symlink d0/d9/dc/d8e/l11c 0 2024-09-10T11:29:41.838 INFO:tasks.workunit.client.1.smithi178.stdout:7/828: creat d6/d10/d1e/d2d/d2b/da0/d101/f127 x:0 0 0 2024-09-10T11:29:41.839 INFO:tasks.workunit.client.1.smithi178.stdout:4/812: read d0/d63/de0/ff8 [1066705,78711] 0 2024-09-10T11:29:41.841 INFO:tasks.workunit.client.0.smithi144.stdout:2/485: dread - d7/d29/d92/f73 zero size 2024-09-10T11:29:41.841 INFO:tasks.workunit.client.1.smithi178.stdout:7/829: fdatasync d6/d10/d34/d6a/d6f/db1/f102 0 2024-09-10T11:29:41.842 INFO:tasks.workunit.client.0.smithi144.stdout:2/486: stat d7/de/d3d/d43/f7b 0 2024-09-10T11:29:41.843 INFO:tasks.workunit.client.1.smithi178.stdout:7/830: dread - d6/d10/d1d/d22/d50/d7e/d82/f115 zero size 2024-09-10T11:29:41.844 INFO:tasks.workunit.client.1.smithi178.stdout:6/945: link d3/d7/l17 d3/d7/d8/d15/d1c/d2a/d18/l154 0 2024-09-10T11:29:41.845 INFO:tasks.workunit.client.0.smithi144.stdout:8/518: truncate d0/d9/ff 1113786 0 2024-09-10T11:29:41.847 INFO:tasks.workunit.client.0.smithi144.stdout:2/487: link d7/de/f37 d7/de/d36/d45/d47/d6c/fa2 0 2024-09-10T11:29:41.849 INFO:tasks.workunit.client.1.smithi178.stdout:7/831: mknod d6/dbf/ddd/c128 0 2024-09-10T11:29:41.850 INFO:tasks.workunit.client.0.smithi144.stdout:2/488: creat d7/de/d3d/d4e/fa3 x:0 0 0 2024-09-10T11:29:41.850 INFO:tasks.workunit.client.0.smithi144.stdout:2/489: chown d7/d42/d64/d8d 102648 1 2024-09-10T11:29:41.852 INFO:tasks.workunit.client.0.smithi144.stdout:2/490: write d7/de/d36/d45/d47/d6c/d7c/d82/f98 [945858,110113] 0 2024-09-10T11:29:41.853 INFO:tasks.workunit.client.1.smithi178.stdout:7/832: rmdir d6/d10/d34/d6a 39 2024-09-10T11:29:41.853 INFO:tasks.workunit.client.1.smithi178.stdout:9/967: dread d2/d19/f8e [4194304,4194304] 0 2024-09-10T11:29:41.855 INFO:tasks.workunit.client.0.smithi144.stdout:2/491: mknod d7/d42/ca4 0 2024-09-10T11:29:41.857 INFO:tasks.workunit.client.1.smithi178.stdout:9/968: creat d2/d19/d5e/f12f x:0 0 0 2024-09-10T11:29:41.860 INFO:tasks.workunit.client.1.smithi178.stdout:4/813: dread d0/d9/dc/dca/f10b [0,4194304] 0 2024-09-10T11:29:41.861 INFO:tasks.workunit.client.1.smithi178.stdout:9/969: mkdir d2/d19/d5e/d130 0 2024-09-10T11:29:41.861 INFO:tasks.workunit.client.1.smithi178.stdout:4/814: chown d0/d8/da/dae/d10e 125 1 2024-09-10T11:29:41.863 INFO:tasks.workunit.client.1.smithi178.stdout:6/946: dread d3/d74/dc9/fb8 [0,4194304] 0 2024-09-10T11:29:41.864 INFO:tasks.workunit.client.0.smithi144.stdout:8/519: dread d0/d9/d23/d26/f36 [0,4194304] 0 2024-09-10T11:29:41.864 INFO:tasks.workunit.client.1.smithi178.stdout:4/815: creat d0/d9/d3d/d4f/ddb/f11d x:0 0 0 2024-09-10T11:29:41.865 INFO:tasks.workunit.client.1.smithi178.stdout:9/970: creat d2/d9b/df8/f131 x:0 0 0 2024-09-10T11:29:41.866 INFO:tasks.workunit.client.1.smithi178.stdout:9/971: read d2/d19/d5e/f9e [434037,32839] 0 2024-09-10T11:29:41.868 INFO:tasks.workunit.client.0.smithi144.stdout:8/520: rename d0/d1b/l49 to d0/d9/d1f/d34/d68/la9 0 2024-09-10T11:29:41.869 INFO:tasks.workunit.client.1.smithi178.stdout:4/816: rename d0/d8/l72 to d0/d8/da/d1a/d51/d5d/db6/df3/l11e 0 2024-09-10T11:29:41.869 INFO:tasks.workunit.client.0.smithi144.stdout:8/521: fdatasync d0/d84/f9b 0 2024-09-10T11:29:41.872 INFO:tasks.workunit.client.0.smithi144.stdout:2/492: dread d7/de/d3d/d43/f7b [0,4194304] 0 2024-09-10T11:29:41.872 INFO:tasks.workunit.client.0.smithi144.stdout:8/522: symlink d0/d9/laa 0 2024-09-10T11:29:41.873 INFO:tasks.workunit.client.1.smithi178.stdout:9/972: truncate d2/d13/f43 2596753 0 2024-09-10T11:29:41.876 INFO:tasks.workunit.client.1.smithi178.stdout:4/817: dwrite d0/d8/da/f96 [0,4194304] 0 2024-09-10T11:29:41.876 INFO:tasks.workunit.client.1.smithi178.stdout:9/973: write d2/df/f1b [5978133,31191] 0 2024-09-10T11:29:41.881 INFO:tasks.workunit.client.1.smithi178.stdout:9/974: creat d2/d9b/df8/da5/f132 x:0 0 0 2024-09-10T11:29:41.881 INFO:tasks.workunit.client.0.smithi144.stdout:2/493: dwrite d7/de/d3d/d4e/fa3 [0,4194304] 0 2024-09-10T11:29:41.883 INFO:tasks.workunit.client.0.smithi144.stdout:8/523: dread - d0/d9/d21/d25/d4f/f65 zero size 2024-09-10T11:29:41.883 INFO:tasks.workunit.client.0.smithi144.stdout:2/494: mknod d7/de/d3d/ca5 0 2024-09-10T11:29:41.885 INFO:tasks.workunit.client.1.smithi178.stdout:9/975: creat d2/dc/f133 x:0 0 0 2024-09-10T11:29:41.885 INFO:tasks.workunit.client.1.smithi178.stdout:4/818: creat d0/d8/da/d1a/d51/d5d/d62/f11f x:0 0 0 2024-09-10T11:29:41.886 INFO:tasks.workunit.client.0.smithi144.stdout:2/495: write d7/d29/f60 [4821376,125791] 0 2024-09-10T11:29:41.887 INFO:tasks.workunit.client.1.smithi178.stdout:7/833: dwrite d6/d10/d1d/d22/d93/f9e [0,4194304] 0 2024-09-10T11:29:41.890 INFO:tasks.workunit.client.1.smithi178.stdout:1/870: sync 2024-09-10T11:29:41.890 INFO:tasks.workunit.client.1.smithi178.stdout:8/984: sync 2024-09-10T11:29:41.890 INFO:tasks.workunit.client.1.smithi178.stdout:9/976: rmdir d2/d13/d75 39 2024-09-10T11:29:41.890 INFO:tasks.workunit.client.0.smithi144.stdout:2/496: symlink d7/d29/d97/la6 0 2024-09-10T11:29:41.891 INFO:tasks.workunit.client.0.smithi144.stdout:8/524: link d0/d9/d1f/d63/l8c d0/d84/lab 0 2024-09-10T11:29:41.893 INFO:tasks.workunit.client.0.smithi144.stdout:2/497: symlink d7/de/la7 0 2024-09-10T11:29:41.893 INFO:tasks.workunit.client.1.smithi178.stdout:6/947: dread d3/fa0 [0,4194304] 0 2024-09-10T11:29:41.894 INFO:tasks.workunit.client.1.smithi178.stdout:4/819: unlink d0/d9/dc/c14 0 2024-09-10T11:29:41.896 INFO:tasks.workunit.client.0.smithi144.stdout:2/498: rmdir d7/de/d36/d45/d47 39 2024-09-10T11:29:41.897 INFO:tasks.workunit.client.0.smithi144.stdout:2/499: chown d7/de/d36/d45/d8e 95338491 1 2024-09-10T11:29:41.898 INFO:tasks.workunit.client.0.smithi144.stdout:2/500: fdatasync d7/d29/f59 0 2024-09-10T11:29:41.898 INFO:tasks.workunit.client.1.smithi178.stdout:1/871: creat d3/d9/d21/d4b/d11a/f142 x:0 0 0 2024-09-10T11:29:41.899 INFO:tasks.workunit.client.0.smithi144.stdout:8/525: rmdir d0/d9/d1f 39 2024-09-10T11:29:41.901 INFO:tasks.workunit.client.1.smithi178.stdout:4/820: mkdir d0/d8/da/d1a/d51/d5d/db6/df3/d120 0 2024-09-10T11:29:41.902 INFO:tasks.workunit.client.0.smithi144.stdout:2/501: mkdir d7/da/da8 0 2024-09-10T11:29:41.903 INFO:tasks.workunit.client.1.smithi178.stdout:8/985: chown d5/db/d10/d20/d52/d67/dbb/f135 2 1 2024-09-10T11:29:41.904 INFO:tasks.workunit.client.1.smithi178.stdout:6/948: rename d3/f14 to d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d13f/f155 0 2024-09-10T11:29:41.906 INFO:tasks.workunit.client.1.smithi178.stdout:1/872: creat d3/d35/ddd/df4/f143 x:0 0 0 2024-09-10T11:29:41.906 INFO:tasks.workunit.client.0.smithi144.stdout:8/526: rename d0/l2d to d0/d9/d1f/d34/d68/lac 0 2024-09-10T11:29:41.907 INFO:tasks.workunit.client.0.smithi144.stdout:2/502: mknod d7/de/d36/d45/d8e/d9a/d54/d61/ca9 0 2024-09-10T11:29:41.909 INFO:tasks.workunit.client.0.smithi144.stdout:2/503: write d7/d29/f3f [606301,124816] 0 2024-09-10T11:29:41.909 INFO:tasks.workunit.client.0.smithi144.stdout:8/527: truncate d0/d9/d21/f71 1080083 0 2024-09-10T11:29:41.912 INFO:tasks.workunit.client.1.smithi178.stdout:8/986: dwrite d5/d30/f127 [0,4194304] 0 2024-09-10T11:29:41.913 INFO:tasks.workunit.client.1.smithi178.stdout:1/873: symlink d3/d9/d98/dac/d3a/d50/d11f/l144 0 2024-09-10T11:29:41.914 INFO:tasks.workunit.client.1.smithi178.stdout:8/987: write d5/d30/f12f [769128,27596] 0 2024-09-10T11:29:41.915 INFO:tasks.workunit.client.0.smithi144.stdout:2/504: link d7/d42/f48 d7/d42/d64/faa 0 2024-09-10T11:29:41.918 INFO:tasks.workunit.client.1.smithi178.stdout:4/821: dread d0/d8/da/d1a/d51/d5d/fa1 [0,4194304] 0 2024-09-10T11:29:41.918 INFO:tasks.workunit.client.0.smithi144.stdout:2/505: creat d7/de/d3d/fab x:0 0 0 2024-09-10T11:29:41.919 INFO:tasks.workunit.client.1.smithi178.stdout:8/988: read d5/db/d3d/de7/ffd [18336,66161] 0 2024-09-10T11:29:41.920 INFO:tasks.workunit.client.0.smithi144.stdout:8/528: write d0/d9/f1d [686840,11838] 0 2024-09-10T11:29:41.921 INFO:tasks.workunit.client.1.smithi178.stdout:1/874: creat d3/d9/d98/dac/d3a/d50/db5/de3/f145 x:0 0 0 2024-09-10T11:29:41.922 INFO:tasks.workunit.client.0.smithi144.stdout:8/529: dread - d0/d9/d21/f59 zero size 2024-09-10T11:29:41.922 INFO:tasks.workunit.client.1.smithi178.stdout:7/834: dwrite d6/d10/d1d/f20 [0,4194304] 0 2024-09-10T11:29:41.922 INFO:tasks.workunit.client.1.smithi178.stdout:4/822: rmdir d0/d8/dc0 39 2024-09-10T11:29:41.924 INFO:tasks.workunit.client.1.smithi178.stdout:9/977: dwrite d2/df/d31/fc0 [0,4194304] 0 2024-09-10T11:29:41.924 INFO:tasks.workunit.client.1.smithi178.stdout:6/949: dread d3/d7/d8/d56/de1/ff2 [0,4194304] 0 2024-09-10T11:29:41.924 INFO:tasks.workunit.client.0.smithi144.stdout:8/530: dread d0/d9/d1f/f95 [0,4194304] 0 2024-09-10T11:29:41.927 INFO:tasks.workunit.client.1.smithi178.stdout:4/823: symlink d0/d9/dc/d8e/dbf/l121 0 2024-09-10T11:29:41.929 INFO:tasks.workunit.client.0.smithi144.stdout:2/506: dwrite d7/de/d36/d45/d8e/d9a/d54/f8a [0,4194304] 0 2024-09-10T11:29:41.929 INFO:tasks.workunit.client.1.smithi178.stdout:9/978: unlink d2/df/d31/d37/d3d/la1 0 2024-09-10T11:29:41.930 INFO:tasks.workunit.client.1.smithi178.stdout:0/862: sync 2024-09-10T11:29:41.932 INFO:tasks.workunit.client.0.smithi144.stdout:2/507: mkdir d7/de/d3d/d4e/dac 0 2024-09-10T11:29:41.932 INFO:tasks.workunit.client.0.smithi144.stdout:8/531: creat d0/d9/d1f/d34/fad x:0 0 0 2024-09-10T11:29:41.933 INFO:tasks.workunit.client.1.smithi178.stdout:6/950: mknod d3/d74/d91/daa/dc2/c156 0 2024-09-10T11:29:41.938 INFO:tasks.workunit.client.0.smithi144.stdout:8/532: symlink d0/d9/d23/d37/lae 0 2024-09-10T11:29:41.939 INFO:tasks.workunit.client.0.smithi144.stdout:2/508: dread d7/de/d3d/d4e/fa3 [0,4194304] 0 2024-09-10T11:29:41.940 INFO:tasks.workunit.client.1.smithi178.stdout:8/989: rename d5/db/d10/d20/d52/d5c to d5/d1d/d22/d76/d126/d14a 0 2024-09-10T11:29:41.942 INFO:tasks.workunit.client.1.smithi178.stdout:8/990: truncate d5/d1d/d22/d76/d126/f111 810991 0 2024-09-10T11:29:41.943 INFO:tasks.workunit.client.1.smithi178.stdout:8/991: stat d5/db/f14 0 2024-09-10T11:29:41.943 INFO:tasks.workunit.client.1.smithi178.stdout:9/979: mkdir d2/df/d31/d37/d114/d134 0 2024-09-10T11:29:41.946 INFO:tasks.workunit.client.1.smithi178.stdout:6/951: symlink d3/d74/dc9/d122/l157 0 2024-09-10T11:29:41.947 INFO:tasks.workunit.client.1.smithi178.stdout:4/824: symlink d0/d8/da/d1a/d51/d5d/db6/df3/d120/l122 0 2024-09-10T11:29:41.948 INFO:tasks.workunit.client.1.smithi178.stdout:9/980: write d2/d9b/df8/d2d/dab/d118/d89/fe5 [740685,101769] 0 2024-09-10T11:29:41.949 INFO:tasks.workunit.client.1.smithi178.stdout:8/992: symlink d5/db/d10/d20/d52/d67/dbb/l14b 0 2024-09-10T11:29:41.950 INFO:tasks.workunit.client.1.smithi178.stdout:6/952: symlink d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/d144/l158 0 2024-09-10T11:29:41.953 INFO:tasks.workunit.client.1.smithi178.stdout:9/981: getdents d2/d9b/df8/d2d/dab 0 2024-09-10T11:29:41.953 INFO:tasks.workunit.client.1.smithi178.stdout:7/835: dwrite d6/d10/d1e/f35 [0,4194304] 0 2024-09-10T11:29:41.954 INFO:tasks.workunit.client.1.smithi178.stdout:9/982: fsync d2/f26 0 2024-09-10T11:29:41.954 INFO:tasks.workunit.client.1.smithi178.stdout:6/953: mknod d3/d7/d8/d15/d1c/d2a/c159 0 2024-09-10T11:29:41.956 INFO:tasks.workunit.client.1.smithi178.stdout:7/836: mknod d6/d10/d1d/d22/d3b/c129 0 2024-09-10T11:29:41.958 INFO:tasks.workunit.client.1.smithi178.stdout:9/983: truncate d2/df/d31/d37/d3d/fed 342211 0 2024-09-10T11:29:41.960 INFO:tasks.workunit.client.1.smithi178.stdout:7/837: creat d6/d10/d1d/de6/f12a x:0 0 0 2024-09-10T11:29:41.960 INFO:tasks.workunit.client.0.smithi144.stdout:8/533: write d0/d9/d23/d37/f3a [1605504,86860] 0 2024-09-10T11:29:41.966 INFO:tasks.workunit.client.1.smithi178.stdout:7/838: creat d6/d10/d1d/d22/d50/def/df3/f12b x:0 0 0 2024-09-10T11:29:41.967 INFO:tasks.workunit.client.1.smithi178.stdout:6/954: getdents d3/d74/d91/daa/d103 0 2024-09-10T11:29:41.968 INFO:tasks.workunit.client.1.smithi178.stdout:7/839: chown d6/d10/d1e/d2d/d2b/da0 0 1 2024-09-10T11:29:41.968 INFO:tasks.workunit.client.0.smithi144.stdout:8/534: dwrite d0/d1b/d5a/d8d/f98 [0,4194304] 0 2024-09-10T11:29:41.968 INFO:tasks.workunit.client.1.smithi178.stdout:9/984: mknod d2/d19/d5e/d130/c135 0 2024-09-10T11:29:41.970 INFO:tasks.workunit.client.0.smithi144.stdout:8/535: truncate d0/d1b/f4c 668585 0 2024-09-10T11:29:41.970 INFO:tasks.workunit.client.1.smithi178.stdout:6/955: chown d3/d7/d8/d15/cf0 75029600 1 2024-09-10T11:29:41.970 INFO:tasks.workunit.client.1.smithi178.stdout:1/875: dread d3/da0/fc1 [0,4194304] 0 2024-09-10T11:29:41.973 INFO:tasks.workunit.client.1.smithi178.stdout:7/840: rmdir d6/d10/d1d/d4a/d79/df1 39 2024-09-10T11:29:41.974 INFO:tasks.workunit.client.1.smithi178.stdout:1/876: chown d3/d9/d98/dac/d3a/d50/d80/fef 3 1 2024-09-10T11:29:41.975 INFO:tasks.workunit.client.1.smithi178.stdout:0/863: dwrite d0/dc/d39/d4c/f73 [8388608,4194304] 0 2024-09-10T11:29:41.975 INFO:tasks.workunit.client.1.smithi178.stdout:6/956: symlink d3/d74/d91/dee/d147/d14e/l15a 0 2024-09-10T11:29:41.975 INFO:tasks.workunit.client.0.smithi144.stdout:8/536: link d0/lb d0/d9/d21/laf 0 2024-09-10T11:29:41.975 INFO:tasks.workunit.client.0.smithi144.stdout:8/537: chown d0/fc 12 1 2024-09-10T11:29:41.977 INFO:tasks.workunit.client.0.smithi144.stdout:8/538: write d0/d9/d23/d26/d7d/fa8 [10694,68876] 0 2024-09-10T11:29:41.978 INFO:tasks.workunit.client.0.smithi144.stdout:8/539: chown d0/d1b/l6b 14 1 2024-09-10T11:29:41.978 INFO:tasks.workunit.client.1.smithi178.stdout:0/864: chown d0/l29 99 1 2024-09-10T11:29:41.979 INFO:tasks.workunit.client.1.smithi178.stdout:6/957: mkdir d3/d74/d15b 0 2024-09-10T11:29:41.980 INFO:tasks.workunit.client.1.smithi178.stdout:1/877: fdatasync d3/d9/d98/dac/d3a/d6e/f77 0 2024-09-10T11:29:41.981 INFO:tasks.workunit.client.1.smithi178.stdout:0/865: rmdir d0/dc/d39/d7c 39 2024-09-10T11:29:41.983 INFO:tasks.workunit.client.1.smithi178.stdout:6/958: truncate d3/d74/d91/daa/dc2/f96 4617864 0 2024-09-10T11:29:41.983 INFO:tasks.workunit.client.1.smithi178.stdout:7/841: link d6/d10/d1d/d22/d50/d7e/d82/db2/d49/f110 d6/d10/d1d/d22/d3b/da1/f12c 0 2024-09-10T11:29:41.986 INFO:tasks.workunit.client.1.smithi178.stdout:1/878: write d3/d9/d98/dac/d3a/d50/d6c/de9/d121/fa2 [254412,71070] 0 2024-09-10T11:29:41.987 INFO:tasks.workunit.client.1.smithi178.stdout:6/959: rename d3/d74/d91/c146 to d3/d74/c15c 0 2024-09-10T11:29:41.989 INFO:tasks.workunit.client.1.smithi178.stdout:0/866: truncate d0/dc/fe6 108543 0 2024-09-10T11:29:41.989 INFO:tasks.workunit.client.1.smithi178.stdout:7/842: truncate d6/d10/d1e/d2d/d2b/da0/dc0/fe8 1121429 0 2024-09-10T11:29:41.990 INFO:tasks.workunit.client.1.smithi178.stdout:7/843: chown d6/d10/d1e/l64 7307983 1 2024-09-10T11:29:41.991 INFO:tasks.workunit.client.1.smithi178.stdout:1/879: creat d3/d9/d98/dd1/ddc/f146 x:0 0 0 2024-09-10T11:29:41.991 INFO:tasks.workunit.client.1.smithi178.stdout:9/985: truncate d2/d9b/df8/d2a/fa9 386491 0 2024-09-10T11:29:41.993 INFO:tasks.workunit.client.1.smithi178.stdout:9/986: chown d2/df/d31/d37/d3d/fc3 601 1 2024-09-10T11:29:41.993 INFO:tasks.workunit.client.1.smithi178.stdout:0/867: chown d0/dc/d39/d50/cb2 109123 1 2024-09-10T11:29:41.993 INFO:tasks.workunit.client.1.smithi178.stdout:6/960: mknod d3/d74/d91/daa/dc2/d57/d78/c15d 0 2024-09-10T11:29:41.995 INFO:tasks.workunit.client.1.smithi178.stdout:7/844: link d6/d10/d1d/d22/d50/d7e/da7/f11d d6/d10/d1d/d22/d3b/f12d 0 2024-09-10T11:29:41.996 INFO:tasks.workunit.client.1.smithi178.stdout:9/987: chown d2/d9b/df8/d2d/dab/d118/d89/d93/d88/dae/d117 1616833 1 2024-09-10T11:29:41.997 INFO:tasks.workunit.client.1.smithi178.stdout:9/988: readlink d2/dc/l74 0 2024-09-10T11:29:41.998 INFO:tasks.workunit.client.1.smithi178.stdout:6/961: chown d3/d74/d91/daa/dc2/d57/d78/d7c/c143 2622 1 2024-09-10T11:29:42.001 INFO:tasks.workunit.client.1.smithi178.stdout:6/962: creat d3/d74/d91/dee/f15e x:0 0 0 2024-09-10T11:29:42.002 INFO:tasks.workunit.client.1.smithi178.stdout:0/868: dwrite d0/dc/d39/da4/dce/fe1 [0,4194304] 0 2024-09-10T11:29:42.004 INFO:tasks.workunit.client.1.smithi178.stdout:6/963: mknod d3/d6e/c15f 0 2024-09-10T11:29:42.006 INFO:tasks.workunit.client.0.smithi144.stdout:8/540: creat d0/d9/fb0 x:0 0 0 2024-09-10T11:29:42.008 INFO:tasks.workunit.client.1.smithi178.stdout:0/869: symlink d0/dc/d39/da4/dce/ddc/l112 0 2024-09-10T11:29:42.011 INFO:tasks.workunit.client.1.smithi178.stdout:1/880: dread d3/d9/d98/dac/d2a/f92 [4194304,4194304] 0 2024-09-10T11:29:42.012 INFO:tasks.workunit.client.0.smithi144.stdout:8/541: getdents d0/d1b 0 2024-09-10T11:29:42.013 INFO:tasks.workunit.client.1.smithi178.stdout:0/870: read - d0/dc/d39/d50/d5e/fcd zero size 2024-09-10T11:29:42.014 INFO:tasks.workunit.client.1.smithi178.stdout:1/881: readlink d3/d9/l52 0 2024-09-10T11:29:42.017 INFO:tasks.workunit.client.1.smithi178.stdout:0/871: creat d0/db1/f113 x:0 0 0 2024-09-10T11:29:42.018 INFO:tasks.workunit.client.1.smithi178.stdout:1/882: symlink d3/d35/ddd/df4/l147 0 2024-09-10T11:29:42.018 INFO:tasks.workunit.client.0.smithi144.stdout:8/542: dwrite d0/d9/d21/f70 [0,4194304] 0 2024-09-10T11:29:42.019 INFO:tasks.workunit.client.1.smithi178.stdout:0/872: readlink d0/dc/d39/d50/l76 0 2024-09-10T11:29:42.019 INFO:tasks.workunit.client.0.smithi144.stdout:8/543: stat d0/d1b/d5a/l5d 0 2024-09-10T11:29:42.021 INFO:tasks.workunit.client.1.smithi178.stdout:1/883: chown d3/d35/ddd/df4/f143 922622 1 2024-09-10T11:29:42.021 INFO:tasks.workunit.client.1.smithi178.stdout:0/873: stat d0/dc/d39/d75/fd4 0 2024-09-10T11:29:42.022 INFO:tasks.workunit.client.0.smithi144.stdout:8/544: write d0/d9/d23/d26/d7d/f80 [402599,96332] 0 2024-09-10T11:29:42.022 INFO:tasks.workunit.client.1.smithi178.stdout:9/989: dwrite d2/d19/f39 [12582912,4194304] 0 2024-09-10T11:29:42.022 INFO:tasks.workunit.client.0.smithi144.stdout:8/545: chown d0/d1b/d5a/d8d 49 1 2024-09-10T11:29:42.024 INFO:tasks.workunit.client.0.smithi144.stdout:8/546: creat d0/d9/d1f/d34/fb1 x:0 0 0 2024-09-10T11:29:42.032 INFO:tasks.workunit.client.0.smithi144.stdout:8/547: link d0/d84/f9b d0/d1b/d47/d5e/d76/fb2 0 2024-09-10T11:29:42.032 INFO:tasks.workunit.client.1.smithi178.stdout:6/964: dwrite d3/d74/dc9/da6/d11e/f9b [0,4194304] 0 2024-09-10T11:29:42.034 INFO:tasks.workunit.client.1.smithi178.stdout:1/884: dwrite d3/d35/fa9 [0,4194304] 0 2024-09-10T11:29:42.035 INFO:tasks.workunit.client.1.smithi178.stdout:1/885: fdatasync d3/d35/ddd/df4/f143 0 2024-09-10T11:29:42.037 INFO:tasks.workunit.client.1.smithi178.stdout:0/874: getdents d0/d2c/d95 0 2024-09-10T11:29:42.038 INFO:tasks.workunit.client.1.smithi178.stdout:1/886: readlink d3/d9/d98/dac/d3a/d50/d9e/dc2/de2/d118/db8/l11d 0 2024-09-10T11:29:42.039 INFO:tasks.workunit.client.0.smithi144.stdout:8/548: rename d0/d9/d23/da4 to d0/d9/d21/d25/db3 0 2024-09-10T11:29:42.040 INFO:tasks.workunit.client.0.smithi144.stdout:8/549: chown d0/d9/d21/d25/d4f/f56 7129 1 2024-09-10T11:29:42.044 INFO:tasks.workunit.client.1.smithi178.stdout:6/965: truncate d3/d7/d8/d15/fbf 453132 0 2024-09-10T11:29:42.044 INFO:tasks.workunit.client.0.smithi144.stdout:8/550: symlink d0/d9/d1f/d34/da0/lb4 0 2024-09-10T11:29:42.046 INFO:tasks.workunit.client.1.smithi178.stdout:0/875: mkdir d0/dc/d39/d50/d114 0 2024-09-10T11:29:42.047 INFO:tasks.workunit.client.0.smithi144.stdout:8/551: symlink d0/lb5 0 2024-09-10T11:29:42.047 INFO:tasks.workunit.client.1.smithi178.stdout:6/966: fdatasync d3/d6e/d87/f10c 0 2024-09-10T11:29:42.049 INFO:tasks.workunit.client.1.smithi178.stdout:0/876: dread - d0/fbf zero size 2024-09-10T11:29:42.051 INFO:tasks.workunit.client.1.smithi178.stdout:9/990: dwrite d2/f11a [0,4194304] 0 2024-09-10T11:29:42.052 INFO:tasks.workunit.client.1.smithi178.stdout:6/967: chown d3/d7/d8/d19/f26 54 1 2024-09-10T11:29:42.054 INFO:tasks.workunit.client.1.smithi178.stdout:0/877: write d0/dc/d39/d75/d94/fff [790804,26582] 0 2024-09-10T11:29:42.057 INFO:tasks.workunit.client.1.smithi178.stdout:0/878: chown d0/dc/d39/d75 24 1 2024-09-10T11:29:42.061 INFO:tasks.workunit.client.1.smithi178.stdout:1/887: dread d3/d9/d98/dd1/d7e/f8c [0,4194304] 0 2024-09-10T11:29:42.062 INFO:tasks.workunit.client.1.smithi178.stdout:6/968: write d3/d74/d91/daa/f107 [37794,102994] 0 2024-09-10T11:29:42.065 INFO:tasks.workunit.client.1.smithi178.stdout:9/991: dwrite d2/d9b/df8/d77/fc9 [0,4194304] 0 2024-09-10T11:29:42.069 INFO:tasks.workunit.client.1.smithi178.stdout:0/879: rename d0/dc/d1f/d2d/c31 to d0/d2c/d93/d107/c115 0 2024-09-10T11:29:42.070 INFO:tasks.workunit.client.1.smithi178.stdout:6/969: creat d3/d74/dc9/d122/f160 x:0 0 0 2024-09-10T11:29:42.075 INFO:tasks.workunit.client.1.smithi178.stdout:0/880: rename d0/dc/d1f/d2d/fb5 to d0/d2c/d95/f116 0 2024-09-10T11:29:42.078 INFO:tasks.workunit.client.1.smithi178.stdout:1/888: creat d3/d9/d98/dac/d3a/dae/f148 x:0 0 0 2024-09-10T11:29:42.080 INFO:tasks.workunit.client.1.smithi178.stdout:6/970: rename d3/d123/l94 to d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/d144/l161 0 2024-09-10T11:29:42.081 INFO:tasks.workunit.client.1.smithi178.stdout:0/881: mknod d0/dc/dc4/c117 0 2024-09-10T11:29:42.085 INFO:tasks.workunit.client.1.smithi178.stdout:6/971: fdatasync d3/d7/d8/d15/d1c/f1d 0 2024-09-10T11:29:42.088 INFO:tasks.workunit.client.1.smithi178.stdout:0/882: read d0/d12/d3f/d72/fef [3353295,69280] 0 2024-09-10T11:29:42.089 INFO:tasks.workunit.client.1.smithi178.stdout:6/972: readlink d3/d6e/d87/l131 0 2024-09-10T11:29:42.091 INFO:tasks.workunit.client.1.smithi178.stdout:9/992: dwrite d2/df/d31/d37/d3d/f6e [0,4194304] 0 2024-09-10T11:29:42.091 INFO:tasks.workunit.client.1.smithi178.stdout:6/973: write d3/d74/dc9/f47 [3024022,128322] 0 2024-09-10T11:29:42.099 INFO:tasks.workunit.client.1.smithi178.stdout:6/974: unlink d3/d6e/f71 0 2024-09-10T11:29:42.099 INFO:tasks.workunit.client.1.smithi178.stdout:9/993: chown d2/d13/d75/d60/d66/l11d 89027 1 2024-09-10T11:29:42.099 INFO:tasks.workunit.client.1.smithi178.stdout:0/883: write d0/f5a [812813,88572] 0 2024-09-10T11:29:42.101 INFO:tasks.workunit.client.1.smithi178.stdout:1/889: dwrite d3/d9/d21/d4b/d2b/f94 [0,4194304] 0 2024-09-10T11:29:42.104 INFO:tasks.workunit.client.1.smithi178.stdout:9/994: write d2/d19/d5e/f12f [826319,36979] 0 2024-09-10T11:29:42.109 INFO:tasks.workunit.client.1.smithi178.stdout:1/890: dwrite d3/d9/d98/dac/d3a/fed [0,4194304] 0 2024-09-10T11:29:42.109 INFO:tasks.workunit.client.1.smithi178.stdout:0/884: write d0/d2c/d95/dbd/fcb [1552409,117391] 0 2024-09-10T11:29:42.115 INFO:tasks.workunit.client.1.smithi178.stdout:0/885: write d0/db1/f113 [887498,129482] 0 2024-09-10T11:29:42.118 INFO:tasks.workunit.client.1.smithi178.stdout:6/975: dwrite d3/d74/dc9/f127 [0,4194304] 0 2024-09-10T11:29:42.118 INFO:tasks.workunit.client.1.smithi178.stdout:1/891: mkdir d3/d9/d98/dac/d3a/d50/d80/d149 0 2024-09-10T11:29:42.121 INFO:tasks.workunit.client.1.smithi178.stdout:6/976: truncate d3/d7/d8/d15/d1c/d2a/d18/f14d 759205 0 2024-09-10T11:29:42.123 INFO:tasks.workunit.client.1.smithi178.stdout:6/977: chown d3/d7/d8/d19/f8b 15 1 2024-09-10T11:29:42.123 INFO:tasks.workunit.client.1.smithi178.stdout:1/892: write d3/d9/d21/d4b/f17 [523871,69540] 0 2024-09-10T11:29:42.126 INFO:tasks.workunit.client.1.smithi178.stdout:1/893: mknod d3/d9/d98/dac/d2a/c14a 0 2024-09-10T11:29:42.127 INFO:tasks.workunit.client.1.smithi178.stdout:6/978: rename d3/d123/l6c to d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d13f/d141/l162 0 2024-09-10T11:29:42.128 INFO:tasks.workunit.client.1.smithi178.stdout:1/894: read d3/d9/d98/dac/d3a/d50/ff9 [2355916,51941] 0 2024-09-10T11:29:42.129 INFO:tasks.workunit.client.1.smithi178.stdout:6/979: write d3/d7/d35/f6f [560704,122209] 0 2024-09-10T11:29:42.132 INFO:tasks.workunit.client.1.smithi178.stdout:6/980: write d3/d74/d91/dee/d147/d14e/f129 [3401451,43758] 0 2024-09-10T11:29:42.154 INFO:tasks.workunit.client.0.smithi144.stdout:6/535: sync 2024-09-10T11:29:42.154 INFO:tasks.workunit.client.0.smithi144.stdout:4/586: sync 2024-09-10T11:29:42.157 INFO:tasks.workunit.client.1.smithi178.stdout:6/981: creat d3/d74/d91/daa/dc2/dcf/df8/f163 x:0 0 0 2024-09-10T11:29:42.157 INFO:tasks.workunit.client.0.smithi144.stdout:4/587: rmdir d14/d1e/d83/da4/da9/dbc 39 2024-09-10T11:29:42.157 INFO:tasks.workunit.client.1.smithi178.stdout:6/982: stat d3/d74 0 2024-09-10T11:29:42.158 INFO:tasks.workunit.client.1.smithi178.stdout:6/983: fsync d3/d74/dc9/f127 0 2024-09-10T11:29:42.158 INFO:tasks.workunit.client.1.smithi178.stdout:1/895: rename d3/d9/d98/dac/d3a/d50/d80/ld4 to d3/d9/d98/dac/d3a/d50/d9e/dc2/d123/l14b 0 2024-09-10T11:29:42.159 INFO:tasks.workunit.client.0.smithi144.stdout:6/536: symlink d9/dd/d3e/d83/daf/d9e/lbe 0 2024-09-10T11:29:42.161 INFO:tasks.workunit.client.1.smithi178.stdout:1/896: rename d3/d9/d21 to d3/d9/d21/d4b/d2b/d14c 22 2024-09-10T11:29:42.161 INFO:tasks.workunit.client.0.smithi144.stdout:4/588: unlink d14/d1e/l26 0 2024-09-10T11:29:42.163 INFO:tasks.workunit.client.0.smithi144.stdout:6/537: getdents d9/d38 0 2024-09-10T11:29:42.165 INFO:tasks.workunit.client.0.smithi144.stdout:6/538: mkdir d9/dd/d3e/dbf 0 2024-09-10T11:29:42.165 INFO:tasks.workunit.client.1.smithi178.stdout:0/886: dread d0/dc/f61 [0,4194304] 0 2024-09-10T11:29:42.165 INFO:tasks.workunit.client.1.smithi178.stdout:1/897: getdents d3/d35/ddd/df4 0 2024-09-10T11:29:42.166 INFO:tasks.workunit.client.0.smithi144.stdout:6/539: write d9/dd/f1a [1113400,63163] 0 2024-09-10T11:29:42.168 INFO:tasks.workunit.client.0.smithi144.stdout:6/540: fdatasync d9/dd/d3e/d83/daf/d5b/d91/fb5 0 2024-09-10T11:29:42.169 INFO:tasks.workunit.client.1.smithi178.stdout:0/887: unlink d0/dc/dc4/c70 0 2024-09-10T11:29:42.171 INFO:tasks.workunit.client.1.smithi178.stdout:0/888: fdatasync d0/dc/d39/d4c/f57 0 2024-09-10T11:29:42.172 INFO:tasks.workunit.client.1.smithi178.stdout:1/898: link d3/d9/d98/dac/d3a/d50/d6c/fbb d3/d9/d21/d4b/d11a/f14d 0 2024-09-10T11:29:42.172 INFO:tasks.workunit.client.1.smithi178.stdout:6/984: dread d3/d7/d8/d15/d1c/f40 [0,4194304] 0 2024-09-10T11:29:42.174 INFO:tasks.workunit.client.0.smithi144.stdout:6/541: link d9/d1f/d2c/lac d9/d1f/d2c/lc0 0 2024-09-10T11:29:42.174 INFO:tasks.workunit.client.1.smithi178.stdout:1/899: write d3/d9/d98/dac/d3a/d50/ffe [677606,95357] 0 2024-09-10T11:29:42.176 INFO:tasks.workunit.client.1.smithi178.stdout:1/900: symlink d3/d35/l14e 0 2024-09-10T11:29:42.177 INFO:tasks.workunit.client.0.smithi144.stdout:6/542: symlink d9/dd/d3e/d83/daf/d5b/d91/lc1 0 2024-09-10T11:29:42.180 INFO:tasks.workunit.client.1.smithi178.stdout:1/901: fsync d3/d9/d98/dac/d3a/d6e/d8a/f2e 0 2024-09-10T11:29:42.181 INFO:tasks.workunit.client.1.smithi178.stdout:6/985: dwrite d3/d7/d8/d15/d1c/d2a/d18/f70 [4194304,4194304] 0 2024-09-10T11:29:42.181 INFO:tasks.workunit.client.0.smithi144.stdout:6/543: rename d9/d38/c5e to d9/d1f/d2c/d4f/cc2 0 2024-09-10T11:29:42.182 INFO:tasks.workunit.client.0.smithi144.stdout:6/544: write d9/dd/f49 [2549584,118192] 0 2024-09-10T11:29:42.183 INFO:tasks.workunit.client.0.smithi144.stdout:6/545: read d9/dd/d3e/f40 [429068,99731] 0 2024-09-10T11:29:42.186 INFO:tasks.workunit.client.1.smithi178.stdout:6/986: readlink d3/d74/d91/dee/l11d 0 2024-09-10T11:29:42.188 INFO:tasks.workunit.client.1.smithi178.stdout:1/902: symlink d3/d9/d98/dac/d3a/d50/d9e/dc2/de2/l14f 0 2024-09-10T11:29:42.188 INFO:tasks.workunit.client.0.smithi144.stdout:6/546: dread - d9/d1f/d2c/f67 zero size 2024-09-10T11:29:42.188 INFO:tasks.workunit.client.1.smithi178.stdout:6/987: mknod d3/d74/d91/dee/c164 0 2024-09-10T11:29:42.189 INFO:tasks.workunit.client.0.smithi144.stdout:4/589: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/f64 [0,4194304] 0 2024-09-10T11:29:42.190 INFO:tasks.workunit.client.1.smithi178.stdout:6/988: write d3/d74/d91/daa/f107 [1099467,31599] 0 2024-09-10T11:29:42.191 INFO:tasks.workunit.client.1.smithi178.stdout:1/903: chown d3/d9/d98/dac/d3a/d6e/lce 4432844 1 2024-09-10T11:29:42.192 INFO:tasks.workunit.client.0.smithi144.stdout:6/547: fsync d9/dd/d3e/d47/d13/f19 0 2024-09-10T11:29:42.193 INFO:tasks.workunit.client.0.smithi144.stdout:4/590: creat d14/d67/d84/fd1 x:0 0 0 2024-09-10T11:29:42.195 INFO:tasks.workunit.client.1.smithi178.stdout:1/904: symlink d3/d9/d98/dac/d3a/d50/d6c/de9/d121/d120/l150 0 2024-09-10T11:29:42.197 INFO:tasks.workunit.client.0.smithi144.stdout:6/548: symlink d9/dd/d11/d1b/d2f/lc3 0 2024-09-10T11:29:42.197 INFO:tasks.workunit.client.0.smithi144.stdout:4/591: creat d14/d67/d69/d85/fd2 x:0 0 0 2024-09-10T11:29:42.198 INFO:tasks.workunit.client.1.smithi178.stdout:1/905: creat d3/d9/d98/dac/d3a/d50/d9e/dc2/d123/f151 x:0 0 0 2024-09-10T11:29:42.203 INFO:tasks.workunit.client.1.smithi178.stdout:6/989: dread d3/d7/d8/d15/d1c/f4d [0,4194304] 0 2024-09-10T11:29:42.205 INFO:tasks.workunit.client.0.smithi144.stdout:6/549: rename d9/dd/d3e/d83/daf to d9/dd/d11/d1b/d2f/d7f/dc4 0 2024-09-10T11:29:42.207 INFO:tasks.workunit.client.1.smithi178.stdout:1/906: dwrite d3/d9/d21/d4b/f5a [0,4194304] 0 2024-09-10T11:29:42.208 INFO:tasks.workunit.client.1.smithi178.stdout:1/907: dread - d3/d9/d1a/d48/f10b zero size 2024-09-10T11:29:42.209 INFO:tasks.workunit.client.1.smithi178.stdout:6/990: creat d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d13f/f165 x:0 0 0 2024-09-10T11:29:42.209 INFO:tasks.workunit.client.0.smithi144.stdout:4/592: write d14/d6f/f96 [815397,123127] 0 2024-09-10T11:29:42.210 INFO:tasks.workunit.client.1.smithi178.stdout:1/908: chown d3/d9/d98/dd1/ddc/f137 211 1 2024-09-10T11:29:42.210 INFO:tasks.workunit.client.1.smithi178.stdout:6/991: chown d3/d74/dc9/fb2 1 1 2024-09-10T11:29:42.213 INFO:tasks.workunit.client.1.smithi178.stdout:1/909: read d3/d9/d98/dac/d3a/d50/db5/de3/d13a/f13d [304504,51512] 0 2024-09-10T11:29:42.213 INFO:tasks.workunit.client.1.smithi178.stdout:6/992: fdatasync d3/d7/d35/f114 0 2024-09-10T11:29:42.218 INFO:tasks.workunit.client.1.smithi178.stdout:1/910: rmdir d3/d9/d98/dac/d3a/d50/d80/d149 0 2024-09-10T11:29:42.220 INFO:tasks.workunit.client.0.smithi144.stdout:6/550: symlink d9/dd/d3e/d47/d13/d60/lc5 0 2024-09-10T11:29:42.221 INFO:tasks.workunit.client.1.smithi178.stdout:6/993: dwrite d3/d7/d8/d19/f26 [0,4194304] 0 2024-09-10T11:29:42.223 INFO:tasks.workunit.client.0.smithi144.stdout:6/551: mkdir d9/dd/d11/d1b/d2f/dab/dc6 0 2024-09-10T11:29:42.225 INFO:tasks.workunit.client.1.smithi178.stdout:1/911: write d3/d9/d98/dac/d3a/d50/db5/de3/d13a/f13d [2545920,104545] 0 2024-09-10T11:29:42.226 INFO:tasks.workunit.client.1.smithi178.stdout:6/994: symlink d3/d74/d91/daa/dc2/dcf/df8/l166 0 2024-09-10T11:29:42.227 INFO:tasks.workunit.client.0.smithi144.stdout:4/593: link d14/d1e/l81 d14/dcb/ld3 0 2024-09-10T11:29:42.228 INFO:tasks.workunit.client.0.smithi144.stdout:6/552: truncate d9/d1f/f37 590635 0 2024-09-10T11:29:42.230 INFO:tasks.workunit.client.0.smithi144.stdout:4/594: read d14/d1e/d22/d34/d3e/d40/d4b/d4c/f5f [3995965,101858] 0 2024-09-10T11:29:42.232 INFO:tasks.workunit.client.0.smithi144.stdout:6/553: creat d9/dd/d11/d1b/d2f/d7f/fc7 x:0 0 0 2024-09-10T11:29:42.232 INFO:tasks.workunit.client.0.smithi144.stdout:4/595: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/fb0 [15651,95929] 0 2024-09-10T11:29:42.233 INFO:tasks.workunit.client.0.smithi144.stdout:4/596: truncate d14/dcb/fce 386555 0 2024-09-10T11:29:42.234 INFO:tasks.workunit.client.1.smithi178.stdout:6/995: dwrite d3/d7/d8/d15/d1c/f1d [0,4194304] 0 2024-09-10T11:29:42.237 INFO:tasks.workunit.client.0.smithi144.stdout:4/597: mkdir d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/dd4 0 2024-09-10T11:29:42.238 INFO:tasks.workunit.client.1.smithi178.stdout:1/912: truncate d3/d9/d98/dac/d3a/d6e/f135 3878603 0 2024-09-10T11:29:42.240 INFO:tasks.workunit.client.0.smithi144.stdout:4/598: mknod d14/d67/d69/d85/cd5 0 2024-09-10T11:29:42.241 INFO:tasks.workunit.client.1.smithi178.stdout:1/913: readlink d3/d9/d98/dac/d3a/d50/d9e/dc2/d123/l14b 0 2024-09-10T11:29:42.242 INFO:tasks.workunit.client.0.smithi144.stdout:4/599: dread - d14/d67/d69/d90/f9d zero size 2024-09-10T11:29:42.243 INFO:tasks.workunit.client.1.smithi178.stdout:1/914: mkdir d3/d9/d21/d4b/d152 0 2024-09-10T11:29:42.244 INFO:tasks.workunit.client.1.smithi178.stdout:6/996: write d3/d7/d8/d15/fbf [932831,41290] 0 2024-09-10T11:29:42.247 INFO:tasks.workunit.client.1.smithi178.stdout:1/915: mkdir d3/d9/d98/dac/d3a/d153 0 2024-09-10T11:29:42.248 INFO:tasks.workunit.client.1.smithi178.stdout:6/997: mknod d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d13f/d141/c167 0 2024-09-10T11:29:42.252 INFO:tasks.workunit.client.1.smithi178.stdout:6/998: link f1 d3/d74/d91/daa/dc2/d57/d78/d7c/dc5/df3/d13f/d141/f168 0 2024-09-10T11:29:42.252 INFO:tasks.workunit.client.1.smithi178.stdout:6/999: readlink d3/d74/dc9/d122/l157 0 2024-09-10T11:29:42.253 INFO:tasks.workunit.client.1.smithi178.stdout:1/916: write d3/d9/d98/dac/d3a/d50/d6c/fbb [376978,87026] 0 2024-09-10T11:29:42.254 INFO:tasks.workunit.client.1.smithi178.stdout:1/917: chown d3/d9/d98/dac/d3a/d50/d6c/fbb 196 1 2024-09-10T11:29:42.259 INFO:tasks.workunit.client.1.smithi178.stdout:1/918: rename d3/d35/f99 to d3/d9/d98/dac/d3a/d50/d9e/dc2/de2/d118/f154 0 2024-09-10T11:29:42.259 INFO:tasks.workunit.client.0.smithi144.stdout:4/600: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/f58 [0,4194304] 0 2024-09-10T11:29:42.262 INFO:tasks.workunit.client.1.smithi178.stdout:1/919: symlink d3/d9/d21/d4b/l155 0 2024-09-10T11:29:42.263 INFO:tasks.workunit.client.1.smithi178.stdout:1/920: write d3/d9/d21/d4b/f17 [173,47104] 0 2024-09-10T11:29:42.263 INFO:tasks.workunit.client.0.smithi144.stdout:4/601: creat d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/dd4/fd6 x:0 0 0 2024-09-10T11:29:42.266 INFO:tasks.workunit.client.1.smithi178.stdout:1/921: creat d3/d9/d98/dac/d3a/d6e/d8a/f156 x:0 0 0 2024-09-10T11:29:42.270 INFO:tasks.workunit.client.1.smithi178.stdout:1/922: creat d3/d9/d21/d4b/f157 x:0 0 0 2024-09-10T11:29:42.272 INFO:tasks.workunit.client.0.smithi144.stdout:4/602: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/fba [0,4194304] 0 2024-09-10T11:29:42.272 INFO:tasks.workunit.client.1.smithi178.stdout:1/923: rmdir d3/d9/d98/dac/d3a/d50/d6c/de9/d121/d120/d104 39 2024-09-10T11:29:42.273 INFO:tasks.workunit.client.0.smithi144.stdout:4/603: chown d14/d67/d69 431750 1 2024-09-10T11:29:42.277 INFO:tasks.workunit.client.1.smithi178.stdout:1/924: unlink d3/d9/d98/dac/d3a/d50/d9e/dc2/de2/d118/c125 0 2024-09-10T11:29:42.277 INFO:tasks.workunit.client.0.smithi144.stdout:4/604: getdents d14/d1e/d22/d34/d3e/d40/d4b/d4c 0 2024-09-10T11:29:42.280 INFO:tasks.workunit.client.0.smithi144.stdout:4/605: readlink d14/d1e/l8c 0 2024-09-10T11:29:42.282 INFO:tasks.workunit.client.1.smithi178.stdout:1/925: creat d3/d9/d21/d4b/f158 x:0 0 0 2024-09-10T11:29:42.286 INFO:tasks.workunit.client.0.smithi144.stdout:4/606: rename d14/d67/d69/d85 to d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7 0 2024-09-10T11:29:42.286 INFO:tasks.workunit.client.1.smithi178.stdout:1/926: dread d3/d9/d98/dac/d3a/d6e/f7c [4194304,4194304] 0 2024-09-10T11:29:42.288 INFO:tasks.workunit.client.1.smithi178.stdout:1/927: creat d3/d9/d98/dac/d2a/f159 x:0 0 0 2024-09-10T11:29:42.292 INFO:tasks.workunit.client.0.smithi144.stdout:4/607: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/f79 [0,4194304] 0 2024-09-10T11:29:42.292 INFO:tasks.workunit.client.1.smithi178.stdout:1/928: rename d3/d9/d1a/fdb to d3/d9/d98/dd1/f15a 0 2024-09-10T11:29:42.295 INFO:tasks.workunit.client.0.smithi144.stdout:4/608: symlink d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/ld8 0 2024-09-10T11:29:42.298 INFO:tasks.workunit.client.1.smithi178.stdout:1/929: dwrite d3/d9/d21/d4b/dfa/f12a [0,4194304] 0 2024-09-10T11:29:42.299 INFO:tasks.workunit.client.1.smithi178.stdout:1/930: chown d3/d35/c58 543529887 1 2024-09-10T11:29:42.301 INFO:tasks.workunit.client.0.smithi144.stdout:4/609: creat d14/d1e/d65/d8a/fd9 x:0 0 0 2024-09-10T11:29:42.301 INFO:tasks.workunit.client.1.smithi178.stdout:1/931: write d3/f1c [266303,100459] 0 2024-09-10T11:29:42.303 INFO:tasks.workunit.client.0.smithi144.stdout:4/610: mkdir d14/d67/d69/d90/dda 0 2024-09-10T11:29:42.305 INFO:tasks.workunit.client.0.smithi144.stdout:4/611: write d14/f54 [5833962,20593] 0 2024-09-10T11:29:42.305 INFO:tasks.workunit.client.1.smithi178.stdout:1/932: getdents d3/d9/d21/d4b/d152 0 2024-09-10T11:29:42.307 INFO:tasks.workunit.client.1.smithi178.stdout:8/993: sync 2024-09-10T11:29:42.307 INFO:tasks.workunit.client.1.smithi178.stdout:4/825: sync 2024-09-10T11:29:42.307 INFO:tasks.workunit.client.1.smithi178.stdout:7/845: sync 2024-09-10T11:29:42.307 INFO:tasks.workunit.client.1.smithi178.stdout:9/995: sync 2024-09-10T11:29:42.308 INFO:tasks.workunit.client.1.smithi178.stdout:8/994: fsync d5/db/fe 0 2024-09-10T11:29:42.308 INFO:tasks.workunit.client.0.smithi144.stdout:4/612: creat d14/d67/d69/dca/fdb x:0 0 0 2024-09-10T11:29:42.309 INFO:tasks.workunit.client.1.smithi178.stdout:9/996: write d2/d51/ff7 [922537,67113] 0 2024-09-10T11:29:42.312 INFO:tasks.workunit.client.1.smithi178.stdout:1/933: chown d3/d9/d98/dd1/f6a 6760 1 2024-09-10T11:29:42.313 INFO:tasks.workunit.client.0.smithi144.stdout:4/613: rename d14/d1e/d22/d34/d3e/c5d to d14/d1e/d22/d34/d3e/d40/d4b/d4c/dab/cdc 0 2024-09-10T11:29:42.314 INFO:tasks.workunit.client.1.smithi178.stdout:1/934: fsync d3/d9/d98/dac/d2a/f108 0 2024-09-10T11:29:42.316 INFO:tasks.workunit.client.1.smithi178.stdout:9/997: mknod d2/c136 0 2024-09-10T11:29:42.319 INFO:tasks.workunit.client.1.smithi178.stdout:4/826: write d0/d8/d3e/f4c [3230401,55109] 0 2024-09-10T11:29:42.321 INFO:tasks.workunit.client.1.smithi178.stdout:9/998: rename d2/d9b/df8/f2c to d2/d9b/df8/d2d/dab/d118/d56/dbe/f137 0 2024-09-10T11:29:42.322 INFO:tasks.workunit.client.1.smithi178.stdout:9/999: stat d2/d9b/df8/c7c 0 2024-09-10T11:29:42.323 INFO:tasks.workunit.client.1.smithi178.stdout:7/846: truncate d6/d10/d1d/d22/d50/d98/da6/f103 4056814 0 2024-09-10T11:29:42.324 INFO:tasks.workunit.client.1.smithi178.stdout:4/827: mkdir d0/d8/d3e/d123 0 2024-09-10T11:29:42.325 INFO:tasks.workunit.client.1.smithi178.stdout:1/935: write d3/d9/fd [809251,38285] 0 2024-09-10T11:29:42.325 INFO:tasks.workunit.client.1.smithi178.stdout:1/936: read - d3/d9/d21/d4b/f158 zero size 2024-09-10T11:29:42.328 INFO:tasks.workunit.client.0.smithi144.stdout:4/614: dread d14/d6f/f9c [0,4194304] 0 2024-09-10T11:29:42.328 INFO:tasks.workunit.client.1.smithi178.stdout:1/937: read d3/f40 [1316581,72234] 0 2024-09-10T11:29:42.328 INFO:tasks.workunit.client.1.smithi178.stdout:7/847: fdatasync d6/d10/d1e/d2d/d2b/da0/f7a 0 2024-09-10T11:29:42.331 INFO:tasks.workunit.client.1.smithi178.stdout:4/828: creat d0/f124 x:0 0 0 2024-09-10T11:29:42.331 INFO:tasks.workunit.client.1.smithi178.stdout:8/995: dread d5/db/d3d/de7/f124 [0,4194304] 0 2024-09-10T11:29:42.331 INFO:tasks.workunit.client.1.smithi178.stdout:7/848: readlink d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d70/da2/la4 0 2024-09-10T11:29:42.332 INFO:tasks.workunit.client.0.smithi144.stdout:4/615: rename d14/d1e/d22/d34/c55 to d14/d1e/d22/d34/d3e/d40/d4b/d4c/cdd 0 2024-09-10T11:29:42.333 INFO:tasks.workunit.client.1.smithi178.stdout:1/938: mkdir d3/d9/d98/dac/d3a/d50/d6c/de9/d121/d15b 0 2024-09-10T11:29:42.334 INFO:tasks.workunit.client.0.smithi144.stdout:4/616: mkdir d14/d1e/d22/d34/dde 0 2024-09-10T11:29:42.336 INFO:tasks.workunit.client.1.smithi178.stdout:7/849: mkdir d6/d10/d12e 0 2024-09-10T11:29:42.337 INFO:tasks.workunit.client.1.smithi178.stdout:8/996: dread d5/db/d10/d20/f7b [0,4194304] 0 2024-09-10T11:29:42.338 INFO:tasks.workunit.client.1.smithi178.stdout:1/939: creat d3/d35/ddd/df4/f15c x:0 0 0 2024-09-10T11:29:42.339 INFO:tasks.workunit.client.0.smithi144.stdout:4/617: link d14/d1e/d65/la1 d14/d1e/d83/da4/da9/dbc/ldf 0 2024-09-10T11:29:42.340 INFO:tasks.workunit.client.1.smithi178.stdout:4/829: link d0/d10/ff9 d0/d8/da/d1a/d51/d5d/db6/df3/f125 0 2024-09-10T11:29:42.341 INFO:tasks.workunit.client.1.smithi178.stdout:8/997: symlink d5/d30/db4/l14c 0 2024-09-10T11:29:42.342 INFO:tasks.workunit.client.1.smithi178.stdout:8/998: fdatasync d5/d30/fb6 0 2024-09-10T11:29:42.343 INFO:tasks.workunit.client.1.smithi178.stdout:1/940: creat d3/d9/d98/dac/f15d x:0 0 0 2024-09-10T11:29:42.344 INFO:tasks.workunit.client.1.smithi178.stdout:4/830: creat d0/d9/dc/d50/f126 x:0 0 0 2024-09-10T11:29:42.345 INFO:tasks.workunit.client.1.smithi178.stdout:4/831: fsync d0/d8/da/fe7 0 2024-09-10T11:29:42.345 INFO:tasks.workunit.client.0.smithi144.stdout:4/618: dwrite d14/d1e/d65/d8a/fd9 [0,4194304] 0 2024-09-10T11:29:42.346 INFO:tasks.workunit.client.1.smithi178.stdout:8/999: mkdir d5/d1d/d14d 0 2024-09-10T11:29:42.348 INFO:tasks.workunit.client.0.smithi144.stdout:4/619: stat d14/d1e/d22/d34/l51 0 2024-09-10T11:29:42.348 INFO:tasks.workunit.client.0.smithi144.stdout:4/620: stat l13 0 2024-09-10T11:29:42.349 INFO:tasks.workunit.client.1.smithi178.stdout:4/832: creat d0/d8/da/d1a/d51/f127 x:0 0 0 2024-09-10T11:29:42.353 INFO:tasks.workunit.client.1.smithi178.stdout:7/850: write d6/d10/d1d/d22/d50/f59 [2546564,84077] 0 2024-09-10T11:29:42.355 INFO:tasks.workunit.client.1.smithi178.stdout:1/941: rename d3/d9/d98/dac/d3a/d50/d11f/l144 to d3/d9/l15e 0 2024-09-10T11:29:42.355 INFO:tasks.workunit.client.1.smithi178.stdout:7/851: chown d6/d10/d1d/d22/f3a 28995596 1 2024-09-10T11:29:42.357 INFO:tasks.workunit.client.1.smithi178.stdout:0/889: sync 2024-09-10T11:29:42.358 INFO:tasks.workunit.client.0.smithi144.stdout:4/621: dread fd [0,4194304] 0 2024-09-10T11:29:42.358 INFO:tasks.workunit.client.0.smithi144.stdout:4/622: chown d14/d1e/d22/f4d 71750 1 2024-09-10T11:29:42.361 INFO:tasks.workunit.client.1.smithi178.stdout:1/942: rename d3/d9/d98/dac/d3a/d50/d9e/dc2/de2/d118/db8 to d3/d9/d98/dac/d3a/d50/d9e/dc2/d123/d15f 0 2024-09-10T11:29:42.361 INFO:tasks.workunit.client.0.smithi144.stdout:4/623: mkdir d14/d1e/d65/d8a/d8f/de0 0 2024-09-10T11:29:42.362 INFO:tasks.workunit.client.1.smithi178.stdout:1/943: fsync d3/f1c 0 2024-09-10T11:29:42.363 INFO:tasks.workunit.client.0.smithi144.stdout:4/624: fdatasync d14/d1e/d22/d34/d3e/d40/d4b/f80 0 2024-09-10T11:29:42.365 INFO:tasks.workunit.client.1.smithi178.stdout:4/833: dwrite d0/d8/da/d1a/f8b [0,4194304] 0 2024-09-10T11:29:42.366 INFO:tasks.workunit.client.1.smithi178.stdout:0/890: link d0/d12/d3f/d72/fef d0/dc/d39/d50/f118 0 2024-09-10T11:29:42.366 INFO:tasks.workunit.client.1.smithi178.stdout:7/852: dwrite d6/d10/d34/d6a/d6f/db1/f118 [0,4194304] 0 2024-09-10T11:29:42.369 INFO:tasks.workunit.client.0.smithi144.stdout:4/625: getdents d14/d1e/d22/d34/d91 0 2024-09-10T11:29:42.371 INFO:tasks.workunit.client.1.smithi178.stdout:1/944: unlink d3/d9/d98/dd1/lba 0 2024-09-10T11:29:42.372 INFO:tasks.workunit.client.1.smithi178.stdout:1/945: chown d3/d9/d98/dac/d3a/d50/d6c/de9/d121/d15b 1001 1 2024-09-10T11:29:42.373 INFO:tasks.workunit.client.1.smithi178.stdout:0/891: mknod d0/d12/d3f/d72/df0/c119 0 2024-09-10T11:29:42.375 INFO:tasks.workunit.client.1.smithi178.stdout:0/892: creat d0/d2c/d93/f11a x:0 0 0 2024-09-10T11:29:42.376 INFO:tasks.workunit.client.0.smithi144.stdout:4/626: creat d14/d1e/d22/fe1 x:0 0 0 2024-09-10T11:29:42.377 INFO:tasks.workunit.client.1.smithi178.stdout:7/853: mkdir d6/d10/d1d/d22/d93/d12f 0 2024-09-10T11:29:42.381 INFO:tasks.workunit.client.1.smithi178.stdout:4/834: write d0/d8/da/d1a/f57 [228376,118192] 0 2024-09-10T11:29:42.382 INFO:tasks.workunit.client.1.smithi178.stdout:1/946: getdents d3/d9/d98/dac/d3a/d50/db5/de3 0 2024-09-10T11:29:42.383 INFO:tasks.workunit.client.1.smithi178.stdout:4/835: fdatasync d0/d8/da/f60 0 2024-09-10T11:29:42.383 INFO:tasks.workunit.client.1.smithi178.stdout:1/947: write d3/d9/d21/d4b/f158 [729727,102455] 0 2024-09-10T11:29:42.384 INFO:tasks.workunit.client.0.smithi144.stdout:4/627: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/dab/fc5 [0,4194304] 0 2024-09-10T11:29:42.387 INFO:tasks.workunit.client.0.smithi144.stdout:4/628: creat d14/d1e/d22/d34/d3e/fe2 x:0 0 0 2024-09-10T11:29:42.390 INFO:tasks.workunit.client.1.smithi178.stdout:4/836: rename d0/d9/d3d/d4f/ddb/f108 to d0/d8/da/f128 0 2024-09-10T11:29:42.392 INFO:tasks.workunit.client.1.smithi178.stdout:4/837: truncate d0/d9/d3d/d4f/ddb/d106/f115 115488 0 2024-09-10T11:29:42.395 INFO:tasks.workunit.client.0.smithi144.stdout:4/629: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/f58 [4194304,4194304] 0 2024-09-10T11:29:42.396 INFO:tasks.workunit.client.1.smithi178.stdout:4/838: rename d0/f75 to d0/d8/da/d1a/dff/f129 0 2024-09-10T11:29:42.397 INFO:tasks.workunit.client.1.smithi178.stdout:0/893: write d0/f8 [5680188,43505] 0 2024-09-10T11:29:42.399 INFO:tasks.workunit.client.0.smithi144.stdout:4/630: creat d14/fe3 x:0 0 0 2024-09-10T11:29:42.402 INFO:tasks.workunit.client.1.smithi178.stdout:4/839: creat d0/d9/dc/d8a/f12a x:0 0 0 2024-09-10T11:29:42.402 INFO:tasks.workunit.client.1.smithi178.stdout:1/948: write d3/d9/d98/dac/d3a/f4e [2871414,33445] 0 2024-09-10T11:29:42.403 INFO:tasks.workunit.client.0.smithi144.stdout:4/631: mkdir d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/de4 0 2024-09-10T11:29:42.403 INFO:tasks.workunit.client.1.smithi178.stdout:0/894: creat d0/dc/d1f/d2d/f11b x:0 0 0 2024-09-10T11:29:42.404 INFO:tasks.workunit.client.1.smithi178.stdout:4/840: fdatasync d0/f38 0 2024-09-10T11:29:42.405 INFO:tasks.workunit.client.0.smithi144.stdout:4/632: fsync d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/fb4 0 2024-09-10T11:29:42.406 INFO:tasks.workunit.client.1.smithi178.stdout:4/841: chown d0/d9/dc/cbc 63879944 1 2024-09-10T11:29:42.407 INFO:tasks.workunit.client.1.smithi178.stdout:4/842: symlink d0/d63/l12b 0 2024-09-10T11:29:42.409 INFO:tasks.workunit.client.0.smithi144.stdout:4/633: creat d14/d1e/d65/d8a/fe5 x:0 0 0 2024-09-10T11:29:42.409 INFO:tasks.workunit.client.1.smithi178.stdout:1/949: getdents d3/d9/d98/dac/d3a/d50 0 2024-09-10T11:29:42.411 INFO:tasks.workunit.client.0.smithi144.stdout:4/634: fdatasync d14/d1e/d22/d34/d3e/d40/fbb 0 2024-09-10T11:29:42.411 INFO:tasks.workunit.client.1.smithi178.stdout:4/843: unlink d0/d8/df/ccd 0 2024-09-10T11:29:42.412 INFO:tasks.workunit.client.1.smithi178.stdout:4/844: write d0/f5 [321447,6979] 0 2024-09-10T11:29:42.415 INFO:tasks.workunit.client.1.smithi178.stdout:1/950: creat d3/d9/d21/d133/f160 x:0 0 0 2024-09-10T11:29:42.416 INFO:tasks.workunit.client.1.smithi178.stdout:4/845: dread d0/d8/da/d1a/f57 [0,4194304] 0 2024-09-10T11:29:42.417 INFO:tasks.workunit.client.0.smithi144.stdout:4/635: truncate d14/f54 3792176 0 2024-09-10T11:29:42.418 INFO:tasks.workunit.client.1.smithi178.stdout:4/846: chown d0/d9/dc/d50/c118 62235 1 2024-09-10T11:29:42.419 INFO:tasks.workunit.client.1.smithi178.stdout:1/951: creat d3/d9/d98/dac/d3a/d50/d6c/de9/d121/d120/d104/f161 x:0 0 0 2024-09-10T11:29:42.420 INFO:tasks.workunit.client.1.smithi178.stdout:4/847: symlink d0/d9/dc/dca/l12c 0 2024-09-10T11:29:42.421 INFO:tasks.workunit.client.1.smithi178.stdout:0/895: dwrite d0/dc/d39/d50/d5e/fca [0,4194304] 0 2024-09-10T11:29:42.422 INFO:tasks.workunit.client.1.smithi178.stdout:4/848: write d0/d8/da/f90 [81895,37139] 0 2024-09-10T11:29:42.423 INFO:tasks.workunit.client.1.smithi178.stdout:0/896: fdatasync d0/dc/d39/f5b 0 2024-09-10T11:29:42.425 INFO:tasks.workunit.client.0.smithi144.stdout:4/636: rename d14/f54 to d14/d1e/d65/fe6 0 2024-09-10T11:29:42.428 INFO:tasks.workunit.client.0.smithi144.stdout:4/637: mkdir d14/d67/d69/d90/dda/de7 0 2024-09-10T11:29:42.429 INFO:tasks.workunit.client.1.smithi178.stdout:1/952: creat d3/d9/d98/dac/d3a/d50/d9e/dc2/de2/f162 x:0 0 0 2024-09-10T11:29:42.431 INFO:tasks.workunit.client.0.smithi144.stdout:4/638: symlink d14/d1e/d22/d34/d91/le8 0 2024-09-10T11:29:42.431 INFO:tasks.workunit.client.1.smithi178.stdout:1/953: mkdir d3/d35/ddd/d163 0 2024-09-10T11:29:42.433 INFO:tasks.workunit.client.1.smithi178.stdout:1/954: creat d3/d9/d98/dac/d3a/d50/d9e/dc2/d123/d15f/f164 x:0 0 0 2024-09-10T11:29:42.434 INFO:tasks.workunit.client.1.smithi178.stdout:1/955: chown d3/d35/c58 288028 1 2024-09-10T11:29:42.434 INFO:tasks.workunit.client.0.smithi144.stdout:4/639: unlink d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/fb4 0 2024-09-10T11:29:42.435 INFO:tasks.workunit.client.1.smithi178.stdout:1/956: fdatasync d3/d9/d98/dd1/fcf 0 2024-09-10T11:29:42.444 INFO:tasks.workunit.client.0.smithi144.stdout:4/640: dwrite d14/d1e/d22/f23 [0,4194304] 0 2024-09-10T11:29:42.447 INFO:tasks.workunit.client.0.smithi144.stdout:4/641: symlink d14/d67/d69/d90/dda/le9 0 2024-09-10T11:29:42.448 INFO:tasks.workunit.client.1.smithi178.stdout:0/897: dread d0/d2c/d33/f45 [0,4194304] 0 2024-09-10T11:29:42.448 INFO:tasks.workunit.client.0.smithi144.stdout:4/642: dread - d14/d1e/fcd zero size 2024-09-10T11:29:42.448 INFO:tasks.workunit.client.0.smithi144.stdout:4/643: chown d14/d7e/c95 97845597 1 2024-09-10T11:29:42.451 INFO:tasks.workunit.client.0.smithi144.stdout:4/644: mknod d14/d1e/d83/da4/cea 0 2024-09-10T11:29:42.452 INFO:tasks.workunit.client.1.smithi178.stdout:0/898: link d0/d12/f2f d0/d2c/f11c 0 2024-09-10T11:29:42.453 INFO:tasks.workunit.client.1.smithi178.stdout:0/899: write d0/dc/d1f/d2d/dac/de7/ff1 [318802,86259] 0 2024-09-10T11:29:42.459 INFO:tasks.workunit.client.1.smithi178.stdout:0/900: dwrite d0/dc/d1f/d2d/dac/de7/ff1 [0,4194304] 0 2024-09-10T11:29:42.460 INFO:tasks.workunit.client.1.smithi178.stdout:0/901: write d0/dc/d39/f9d [3650971,32500] 0 2024-09-10T11:29:42.461 INFO:tasks.workunit.client.0.smithi144.stdout:4/645: dwrite d14/d1e/d22/d34/d3e/d40/f78 [4194304,4194304] 0 2024-09-10T11:29:42.462 INFO:tasks.workunit.client.1.smithi178.stdout:0/902: read d0/d12/f87 [2535159,31291] 0 2024-09-10T11:29:42.462 INFO:tasks.workunit.client.0.smithi144.stdout:4/646: write d14/d1e/d22/d34/d3e/d40/f76 [1602412,78789] 0 2024-09-10T11:29:42.464 INFO:tasks.workunit.client.0.smithi144.stdout:4/647: readlink d14/d1e/d65/la1 0 2024-09-10T11:29:42.466 INFO:tasks.workunit.client.1.smithi178.stdout:0/903: mkdir d0/d11d 0 2024-09-10T11:29:42.468 INFO:tasks.workunit.client.0.smithi144.stdout:4/648: mknod d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/dd4/ceb 0 2024-09-10T11:29:42.472 INFO:tasks.workunit.client.0.smithi144.stdout:4/649: creat d14/d1e/d22/d34/d91/fec x:0 0 0 2024-09-10T11:29:42.473 INFO:tasks.workunit.client.1.smithi178.stdout:0/904: dwrite d0/f5a [4194304,4194304] 0 2024-09-10T11:29:42.476 INFO:tasks.workunit.client.0.smithi144.stdout:4/650: creat d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/fed x:0 0 0 2024-09-10T11:29:42.479 INFO:tasks.workunit.client.0.smithi144.stdout:4/651: creat d14/d67/d84/db3/fee x:0 0 0 2024-09-10T11:29:42.480 INFO:tasks.workunit.client.0.smithi144.stdout:4/652: dread - d14/d1e/d22/fe1 zero size 2024-09-10T11:29:42.483 INFO:tasks.workunit.client.0.smithi144.stdout:4/653: symlink d14/d1e/d22/d34/d3e/d40/d4b/lef 0 2024-09-10T11:29:42.487 INFO:tasks.workunit.client.0.smithi144.stdout:6/554: sync 2024-09-10T11:29:42.487 INFO:tasks.workunit.client.0.smithi144.stdout:0/443: sync 2024-09-10T11:29:42.487 INFO:tasks.workunit.client.0.smithi144.stdout:9/601: sync 2024-09-10T11:29:42.487 INFO:tasks.workunit.client.0.smithi144.stdout:1/702: sync 2024-09-10T11:29:42.487 INFO:tasks.workunit.client.0.smithi144.stdout:3/614: sync 2024-09-10T11:29:42.487 INFO:tasks.workunit.client.0.smithi144.stdout:7/596: sync 2024-09-10T11:29:42.487 INFO:tasks.workunit.client.0.smithi144.stdout:2/509: sync 2024-09-10T11:29:42.487 INFO:tasks.workunit.client.0.smithi144.stdout:8/552: sync 2024-09-10T11:29:42.487 INFO:tasks.workunit.client.0.smithi144.stdout:5/671: sync 2024-09-10T11:29:42.490 INFO:tasks.workunit.client.0.smithi144.stdout:1/703: chown d5/df/d4b/d51/c9b 93388751 1 2024-09-10T11:29:42.490 INFO:tasks.workunit.client.0.smithi144.stdout:5/672: chown d4/d5/l13 125126914 1 2024-09-10T11:29:42.491 INFO:tasks.workunit.client.1.smithi178.stdout:0/905: dwrite d0/dc/d39/d50/f88 [0,4194304] 0 2024-09-10T11:29:42.492 INFO:tasks.workunit.client.0.smithi144.stdout:8/553: mknod d0/d1b/d5a/cb6 0 2024-09-10T11:29:42.493 INFO:tasks.workunit.client.0.smithi144.stdout:6/555: fdatasync d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/f6e 0 2024-09-10T11:29:42.494 INFO:tasks.workunit.client.0.smithi144.stdout:2/510: rename d7/de/d36/d45/l4a to d7/de/lad 0 2024-09-10T11:29:42.495 INFO:tasks.workunit.client.1.smithi178.stdout:0/906: mknod d0/dc/d39/d75/d94/c11e 0 2024-09-10T11:29:42.496 INFO:tasks.workunit.client.0.smithi144.stdout:1/704: mknod d5/df/d4b/ce9 0 2024-09-10T11:29:42.496 INFO:tasks.workunit.client.0.smithi144.stdout:9/602: symlink d11/d12/d1b/d2c/d78/dd2/ld6 0 2024-09-10T11:29:42.497 INFO:tasks.workunit.client.1.smithi178.stdout:0/907: creat d0/dc/d39/d4c/f11f x:0 0 0 2024-09-10T11:29:42.497 INFO:tasks.workunit.client.0.smithi144.stdout:3/615: symlink d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/lca 0 2024-09-10T11:29:42.498 INFO:tasks.workunit.client.1.smithi178.stdout:0/908: truncate d0/dc/d39/d4c/f11f 468802 0 2024-09-10T11:29:42.499 INFO:tasks.workunit.client.0.smithi144.stdout:5/673: rmdir d4/d5/d18/d38/d8d 39 2024-09-10T11:29:42.502 INFO:tasks.workunit.client.0.smithi144.stdout:0/444: dwrite da/f2e [0,4194304] 0 2024-09-10T11:29:42.502 INFO:tasks.workunit.client.0.smithi144.stdout:9/603: fsync d11/d12/d1b/d2c/d78/fc1 0 2024-09-10T11:29:42.502 INFO:tasks.workunit.client.0.smithi144.stdout:7/597: rename d5/dc/d12/d2b/c65 to d5/dc/d12/d2f/d7c/cbf 0 2024-09-10T11:29:42.504 INFO:tasks.workunit.client.0.smithi144.stdout:3/616: write d4/dd/d21/d2f/f3a [795716,64990] 0 2024-09-10T11:29:42.506 INFO:tasks.workunit.client.0.smithi144.stdout:2/511: creat d7/de/d3d/d4e/dac/fae x:0 0 0 2024-09-10T11:29:42.507 INFO:tasks.workunit.client.0.smithi144.stdout:8/554: dwrite d0/d9/fb0 [0,4194304] 0 2024-09-10T11:29:42.509 INFO:tasks.workunit.client.0.smithi144.stdout:0/445: chown da/l20 60 1 2024-09-10T11:29:42.510 INFO:tasks.workunit.client.0.smithi144.stdout:6/556: mkdir d9/dd/d11/d1b/d2f/d7f/dc4/d5b/dc8 0 2024-09-10T11:29:42.511 INFO:tasks.workunit.client.0.smithi144.stdout:0/446: write da/d44/f54 [2673587,75629] 0 2024-09-10T11:29:42.513 INFO:tasks.workunit.client.0.smithi144.stdout:1/705: creat d5/dd/d8f/fea x:0 0 0 2024-09-10T11:29:42.513 INFO:tasks.workunit.client.0.smithi144.stdout:5/674: mknod d4/d5/d32/ccd 0 2024-09-10T11:29:42.517 INFO:tasks.workunit.client.0.smithi144.stdout:7/598: creat d5/dc/d14/d40/d5e/d68/fc0 x:0 0 0 2024-09-10T11:29:42.518 INFO:tasks.workunit.client.0.smithi144.stdout:7/599: fdatasync d5/dc/d14/d40/f72 0 2024-09-10T11:29:42.519 INFO:tasks.workunit.client.0.smithi144.stdout:8/555: symlink d0/d1b/d5a/lb7 0 2024-09-10T11:29:42.521 INFO:tasks.workunit.client.0.smithi144.stdout:3/617: symlink d4/d28/d67/d80/d17/d18/d34/d65/d75/lcb 0 2024-09-10T11:29:42.524 INFO:tasks.workunit.client.0.smithi144.stdout:6/557: dread d9/dd/d11/d1b/d2f/d7f/f75 [0,4194304] 0 2024-09-10T11:29:42.524 INFO:tasks.workunit.client.0.smithi144.stdout:1/706: dread d5/d4a/f7d [0,4194304] 0 2024-09-10T11:29:42.526 INFO:tasks.workunit.client.0.smithi144.stdout:2/512: dread d7/d29/f3f [0,4194304] 0 2024-09-10T11:29:42.526 INFO:tasks.workunit.client.0.smithi144.stdout:7/600: symlink d5/dc/d14/d40/d58/d6a/lc1 0 2024-09-10T11:29:42.527 INFO:tasks.workunit.client.0.smithi144.stdout:8/556: mknod d0/d9/d23/cb8 0 2024-09-10T11:29:42.527 INFO:tasks.workunit.client.0.smithi144.stdout:5/675: mkdir d4/d20/d24/dce 0 2024-09-10T11:29:42.531 INFO:tasks.workunit.client.0.smithi144.stdout:6/558: chown d9/d1f/dae/cb2 84 1 2024-09-10T11:29:42.531 INFO:tasks.workunit.client.0.smithi144.stdout:0/447: truncate da/d44/f63 69656 0 2024-09-10T11:29:42.533 INFO:tasks.workunit.client.0.smithi144.stdout:3/618: dwrite d4/d1a/fc7 [0,4194304] 0 2024-09-10T11:29:42.533 INFO:tasks.workunit.client.0.smithi144.stdout:2/513: readlink d7/de/d36/d45/d8e/d9a/d7e/l91 0 2024-09-10T11:29:42.535 INFO:tasks.workunit.client.0.smithi144.stdout:7/601: write d5/dc/d14/d40/d5e/d63/f66 [4587780,121820] 0 2024-09-10T11:29:42.538 INFO:tasks.workunit.client.0.smithi144.stdout:5/676: write d4/d20/d24/d2d/d4b/fa7 [605298,124976] 0 2024-09-10T11:29:42.538 INFO:tasks.workunit.client.0.smithi144.stdout:0/448: fdatasync da/d3a/d51/d5f/d6b/f71 0 2024-09-10T11:29:42.541 INFO:tasks.workunit.client.0.smithi144.stdout:9/604: dwrite d11/d12/d1b/d2c/f31 [0,4194304] 0 2024-09-10T11:29:42.541 INFO:tasks.workunit.client.0.smithi144.stdout:3/619: chown d4/dd/d21/f5b 53026667 1 2024-09-10T11:29:42.543 INFO:tasks.workunit.client.0.smithi144.stdout:0/449: write da/d3a/d51/f7a [1038399,10573] 0 2024-09-10T11:29:42.544 INFO:tasks.workunit.client.0.smithi144.stdout:7/602: creat d5/dc/d12/d2b/fc2 x:0 0 0 2024-09-10T11:29:42.544 INFO:tasks.workunit.client.0.smithi144.stdout:5/677: symlink d4/d20/d22/d5c/lcf 0 2024-09-10T11:29:42.549 INFO:tasks.workunit.client.0.smithi144.stdout:2/514: link d7/de/f3b d7/de/d36/d45/d8e/d9a/d7e/faf 0 2024-09-10T11:29:42.549 INFO:tasks.workunit.client.0.smithi144.stdout:3/620: write d4/d1a/f5f [4651800,93522] 0 2024-09-10T11:29:42.549 INFO:tasks.workunit.client.0.smithi144.stdout:8/557: dwrite d0/d9/d21/d25/d4f/f56 [0,4194304] 0 2024-09-10T11:29:42.552 INFO:tasks.workunit.client.0.smithi144.stdout:5/678: symlink d4/d5/d18/ld0 0 2024-09-10T11:29:42.558 INFO:tasks.workunit.client.0.smithi144.stdout:5/679: creat d4/d20/d24/d2d/d4b/fd1 x:0 0 0 2024-09-10T11:29:42.564 INFO:tasks.workunit.client.0.smithi144.stdout:8/558: write d0/d9/d23/d26/f36 [9316131,51103] 0 2024-09-10T11:29:42.568 INFO:tasks.workunit.client.0.smithi144.stdout:3/621: dread f3 [0,4194304] 0 2024-09-10T11:29:42.570 INFO:tasks.workunit.client.0.smithi144.stdout:5/680: fsync d4/d5/d18/d67/f65 0 2024-09-10T11:29:42.571 INFO:tasks.workunit.client.0.smithi144.stdout:8/559: creat d0/d9/d21/d7e/d85/fb9 x:0 0 0 2024-09-10T11:29:42.571 INFO:tasks.workunit.client.0.smithi144.stdout:9/605: dwrite d11/d12/f14 [0,4194304] 0 2024-09-10T11:29:42.575 INFO:tasks.workunit.client.0.smithi144.stdout:1/707: dread d5/df/d4b/d51/f50 [0,4194304] 0 2024-09-10T11:29:42.575 INFO:tasks.workunit.client.0.smithi144.stdout:0/450: dwrite da/d3a/d4b/d4e/d62/f70 [0,4194304] 0 2024-09-10T11:29:42.578 INFO:tasks.workunit.client.0.smithi144.stdout:2/515: dwrite d7/da/d1e/f85 [0,4194304] 0 2024-09-10T11:29:42.578 INFO:tasks.workunit.client.0.smithi144.stdout:3/622: write d4/d28/d67/d80/d17/d18/d34/fbc [751700,45833] 0 2024-09-10T11:29:42.578 INFO:tasks.workunit.client.0.smithi144.stdout:9/606: unlink d11/d12/d1b/l6a 0 2024-09-10T11:29:42.579 INFO:tasks.workunit.client.0.smithi144.stdout:1/708: chown d5/d2a/cca 206358 1 2024-09-10T11:29:42.581 INFO:tasks.workunit.client.1.smithi178.stdout:7/854: sync 2024-09-10T11:29:42.584 INFO:tasks.workunit.client.1.smithi178.stdout:7/855: mkdir d6/d10/d34/d6a/d6f/db1/d130 0 2024-09-10T11:29:42.585 INFO:tasks.workunit.client.0.smithi144.stdout:0/451: rename da/d3a/d51/d5f/l5a to da/d3a/l87 0 2024-09-10T11:29:42.590 INFO:tasks.workunit.client.0.smithi144.stdout:8/560: getdents d0/d1b/d47/d5e/d76 0 2024-09-10T11:29:42.592 INFO:tasks.workunit.client.0.smithi144.stdout:1/709: creat d5/dd/dcb/feb x:0 0 0 2024-09-10T11:29:42.593 INFO:tasks.workunit.client.0.smithi144.stdout:7/603: dread d5/dc/d14/d40/d5e/d63/f74 [0,4194304] 0 2024-09-10T11:29:42.594 INFO:tasks.workunit.client.1.smithi178.stdout:7/856: dwrite d6/d10/d1d/d22/d50/fe3 [0,4194304] 0 2024-09-10T11:29:42.595 INFO:tasks.workunit.client.0.smithi144.stdout:7/604: truncate d5/dc/d11/f81 590476 0 2024-09-10T11:29:42.596 INFO:tasks.workunit.client.0.smithi144.stdout:1/710: symlink d5/dd/lec 0 2024-09-10T11:29:42.598 INFO:tasks.workunit.client.0.smithi144.stdout:7/605: read d5/dc/d11/f81 [210548,77674] 0 2024-09-10T11:29:42.602 INFO:tasks.workunit.client.0.smithi144.stdout:8/561: dwrite d0/d9/d21/d25/d4f/f65 [0,4194304] 0 2024-09-10T11:29:42.603 INFO:tasks.workunit.client.0.smithi144.stdout:7/606: symlink d5/dc/d12/d2b/lc3 0 2024-09-10T11:29:42.610 INFO:tasks.workunit.client.0.smithi144.stdout:8/562: link d0/d9/d21/d25/d4f/l75 d0/d9/d21/d25/db3/lba 0 2024-09-10T11:29:42.610 INFO:tasks.workunit.client.1.smithi178.stdout:7/857: write d6/d10/d1d/f27 [1964902,68309] 0 2024-09-10T11:29:42.612 INFO:tasks.workunit.client.0.smithi144.stdout:2/516: dread d7/f11 [0,4194304] 0 2024-09-10T11:29:42.612 INFO:tasks.workunit.client.0.smithi144.stdout:8/563: write d0/d1b/d47/f7b [741323,119146] 0 2024-09-10T11:29:42.613 INFO:tasks.workunit.client.0.smithi144.stdout:9/607: dread d11/d12/d1b/d41/d8f/f82 [0,4194304] 0 2024-09-10T11:29:42.615 INFO:tasks.workunit.client.0.smithi144.stdout:2/517: truncate d7/da/d5b/f96 664003 0 2024-09-10T11:29:42.616 INFO:tasks.workunit.client.0.smithi144.stdout:1/711: write d5/df/d4b/f54 [3782546,105277] 0 2024-09-10T11:29:42.617 INFO:tasks.workunit.client.1.smithi178.stdout:4/849: sync 2024-09-10T11:29:42.617 INFO:tasks.workunit.client.1.smithi178.stdout:1/957: sync 2024-09-10T11:29:42.617 INFO:tasks.workunit.client.1.smithi178.stdout:0/909: sync 2024-09-10T11:29:42.618 INFO:tasks.workunit.client.0.smithi144.stdout:7/607: truncate d5/dc/d14/d40/d5e/d68/f8f 1274134 0 2024-09-10T11:29:42.619 INFO:tasks.workunit.client.0.smithi144.stdout:9/608: write d11/d12/d1b/d41/d8f/dab/fb8 [886643,109596] 0 2024-09-10T11:29:42.621 INFO:tasks.workunit.client.1.smithi178.stdout:1/958: mkdir d3/d9/d165 0 2024-09-10T11:29:42.621 INFO:tasks.workunit.client.1.smithi178.stdout:7/858: dread d6/d10/d1e/d2d/f21 [0,4194304] 0 2024-09-10T11:29:42.621 INFO:tasks.workunit.client.0.smithi144.stdout:2/518: read d7/f1d [79114,21827] 0 2024-09-10T11:29:42.623 INFO:tasks.workunit.client.0.smithi144.stdout:3/623: dwrite d4/f32 [0,4194304] 0 2024-09-10T11:29:42.623 INFO:tasks.workunit.client.1.smithi178.stdout:4/850: symlink d0/d8/da/d1a/d51/d5d/dc9/dee/d104/l12d 0 2024-09-10T11:29:42.624 INFO:tasks.workunit.client.1.smithi178.stdout:4/851: fsync d0/fdc 0 2024-09-10T11:29:42.625 INFO:tasks.workunit.client.1.smithi178.stdout:7/859: symlink d6/d10/d1d/d22/d50/d98/l131 0 2024-09-10T11:29:42.626 INFO:tasks.workunit.client.0.smithi144.stdout:7/608: write d5/dc/d2a/fac [644436,113099] 0 2024-09-10T11:29:42.627 INFO:tasks.workunit.client.0.smithi144.stdout:1/712: fdatasync d5/d4a/d63/d84/dab/fbb 0 2024-09-10T11:29:42.631 INFO:tasks.workunit.client.1.smithi178.stdout:7/860: fsync d6/d10/d1d/d22/d50/d98/da6/f103 0 2024-09-10T11:29:42.632 INFO:tasks.workunit.client.0.smithi144.stdout:8/564: dwrite d0/d1b/d47/d94/f9d [0,4194304] 0 2024-09-10T11:29:42.633 INFO:tasks.workunit.client.0.smithi144.stdout:3/624: fsync d4/dd/f12 0 2024-09-10T11:29:42.634 INFO:tasks.workunit.client.1.smithi178.stdout:0/910: dwrite d0/dc/d39/d7c/f9b [0,4194304] 0 2024-09-10T11:29:42.635 INFO:tasks.workunit.client.0.smithi144.stdout:8/565: dread - d0/d9/d1f/f92 zero size 2024-09-10T11:29:42.636 INFO:tasks.workunit.client.0.smithi144.stdout:3/625: write d4/d28/f38 [3729379,76402] 0 2024-09-10T11:29:42.636 INFO:tasks.workunit.client.1.smithi178.stdout:7/861: write d6/d10/d1d/d22/d50/d98/da6/dd5/f105 [837162,37115] 0 2024-09-10T11:29:42.638 INFO:tasks.workunit.client.1.smithi178.stdout:0/911: mkdir d0/dc/d39/d75/d120 0 2024-09-10T11:29:42.639 INFO:tasks.workunit.client.1.smithi178.stdout:4/852: dread d0/d9/dc/d8e/f9f [0,4194304] 0 2024-09-10T11:29:42.639 INFO:tasks.workunit.client.0.smithi144.stdout:7/609: dwrite d5/dc/d14/f44 [0,4194304] 0 2024-09-10T11:29:42.640 INFO:tasks.workunit.client.0.smithi144.stdout:9/609: symlink d11/d12/d1b/d41/d8f/dab/ld7 0 2024-09-10T11:29:42.641 INFO:tasks.workunit.client.0.smithi144.stdout:9/610: fsync d11/d12/f14 0 2024-09-10T11:29:42.643 INFO:tasks.workunit.client.0.smithi144.stdout:9/611: write d11/d12/d1b/d41/d8f/fc8 [351895,10784] 0 2024-09-10T11:29:42.645 INFO:tasks.workunit.client.1.smithi178.stdout:0/912: unlink d0/dc/d1f/d2d/dac/de7/ff1 0 2024-09-10T11:29:42.645 INFO:tasks.workunit.client.1.smithi178.stdout:1/959: write d3/d9/f42 [3290679,130455] 0 2024-09-10T11:29:42.648 INFO:tasks.workunit.client.1.smithi178.stdout:1/960: chown d3/d9/d98/dac/d3a/d50/db5/de3 3596 1 2024-09-10T11:29:42.649 INFO:tasks.workunit.client.1.smithi178.stdout:1/961: fsync d3/d9/d1a/f12d 0 2024-09-10T11:29:42.650 INFO:tasks.workunit.client.1.smithi178.stdout:4/853: symlink d0/d9/dc/l12e 0 2024-09-10T11:29:42.651 INFO:tasks.workunit.client.1.smithi178.stdout:0/913: read d0/d2c/d95/dbd/dd5/fc7 [306104,102117] 0 2024-09-10T11:29:42.652 INFO:tasks.workunit.client.0.smithi144.stdout:2/519: write d7/d42/f48 [107523,44764] 0 2024-09-10T11:29:42.652 INFO:tasks.workunit.client.0.smithi144.stdout:8/566: mkdir d0/d9/d21/d25/db3/dbb 0 2024-09-10T11:29:42.652 INFO:tasks.workunit.client.0.smithi144.stdout:2/520: chown d7/de/d36 78972 1 2024-09-10T11:29:42.652 INFO:tasks.workunit.client.1.smithi178.stdout:1/962: fdatasync d3/d9/d98/dac/d3a/d50/d6c/de9/d121/d120/d104/f161 0 2024-09-10T11:29:42.653 INFO:tasks.workunit.client.1.smithi178.stdout:0/914: stat d0/d12/d3f/fa5 0 2024-09-10T11:29:42.653 INFO:tasks.workunit.client.1.smithi178.stdout:4/854: fsync d0/d9/d3d/f44 0 2024-09-10T11:29:42.654 INFO:tasks.workunit.client.1.smithi178.stdout:0/915: dread - d0/dc/d1f/d2d/dac/ff8 zero size 2024-09-10T11:29:42.655 INFO:tasks.workunit.client.0.smithi144.stdout:2/521: chown d7/d29/c2e 11243353 1 2024-09-10T11:29:42.655 INFO:tasks.workunit.client.0.smithi144.stdout:3/626: getdents d4/d28 0 2024-09-10T11:29:42.655 INFO:tasks.workunit.client.1.smithi178.stdout:1/963: creat d3/d9/d21/d4b/dfa/f166 x:0 0 0 2024-09-10T11:29:42.657 INFO:tasks.workunit.client.1.smithi178.stdout:0/916: creat d0/dc/d39/d75/f121 x:0 0 0 2024-09-10T11:29:42.658 INFO:tasks.workunit.client.1.smithi178.stdout:0/917: readlink d0/d2c/d33/l8f 0 2024-09-10T11:29:42.659 INFO:tasks.workunit.client.1.smithi178.stdout:1/964: mkdir d3/d9/d167 0 2024-09-10T11:29:42.660 INFO:tasks.workunit.client.0.smithi144.stdout:8/567: link d0/d1b/f8b d0/d9/d1f/d34/d68/fbc 0 2024-09-10T11:29:42.661 INFO:tasks.workunit.client.1.smithi178.stdout:1/965: fsync d3/d9/d98/fca 0 2024-09-10T11:29:42.663 INFO:tasks.workunit.client.0.smithi144.stdout:3/627: unlink d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/c9e 0 2024-09-10T11:29:42.664 INFO:tasks.workunit.client.0.smithi144.stdout:8/568: mknod d0/d9/d21/cbd 0 2024-09-10T11:29:42.665 INFO:tasks.workunit.client.0.smithi144.stdout:2/522: dwrite d7/da/f12 [0,4194304] 0 2024-09-10T11:29:42.667 INFO:tasks.workunit.client.1.smithi178.stdout:7/862: dwrite d6/d10/d1d/fb8 [0,4194304] 0 2024-09-10T11:29:42.667 INFO:tasks.workunit.client.0.smithi144.stdout:7/610: dread d5/dc/d12/d2f/f4a [0,4194304] 0 2024-09-10T11:29:42.667 INFO:tasks.workunit.client.0.smithi144.stdout:9/612: write d11/d12/d1b/d22/f7a [1812600,77379] 0 2024-09-10T11:29:42.668 INFO:tasks.workunit.client.1.smithi178.stdout:1/966: creat d3/d9/d98/dac/d3a/d50/d6c/de9/d121/f168 x:0 0 0 2024-09-10T11:29:42.668 INFO:tasks.workunit.client.1.smithi178.stdout:0/918: dwrite d0/dc/f21 [4194304,4194304] 0 2024-09-10T11:29:42.670 INFO:tasks.workunit.client.0.smithi144.stdout:2/523: write d7/de/f1b [3988864,112067] 0 2024-09-10T11:29:42.672 INFO:tasks.workunit.client.0.smithi144.stdout:2/524: chown d7/de/d3d/f52 5 1 2024-09-10T11:29:42.672 INFO:tasks.workunit.client.0.smithi144.stdout:1/713: dread d5/d2a/d65/dc9/f67 [0,4194304] 0 2024-09-10T11:29:42.674 INFO:tasks.workunit.client.0.smithi144.stdout:8/569: mknod d0/d9/d23/d26/cbe 0 2024-09-10T11:29:42.677 INFO:tasks.workunit.client.0.smithi144.stdout:2/525: creat d7/de/d3d/d43/fb0 x:0 0 0 2024-09-10T11:29:42.678 INFO:tasks.workunit.client.1.smithi178.stdout:7/863: stat d6/d10/d1d/d22/d3b/f6d 0 2024-09-10T11:29:42.679 INFO:tasks.workunit.client.1.smithi178.stdout:0/919: dread - d0/d2c/d89/da1/ffb zero size 2024-09-10T11:29:42.680 INFO:tasks.workunit.client.0.smithi144.stdout:7/611: mkdir d5/dc/d14/d40/d5e/dc4 0 2024-09-10T11:29:42.680 INFO:tasks.workunit.client.1.smithi178.stdout:0/920: chown d0/d12/d3f/d72/df0 16 1 2024-09-10T11:29:42.681 INFO:tasks.workunit.client.0.smithi144.stdout:9/613: creat d11/d12/d1b/d41/fd8 x:0 0 0 2024-09-10T11:29:42.685 INFO:tasks.workunit.client.0.smithi144.stdout:2/526: truncate d7/de/d36/d45/d47/d6c/d7c/d82/f98 1410518 0 2024-09-10T11:29:42.693 INFO:tasks.workunit.client.0.smithi144.stdout:3/628: dwrite d4/d28/d67/d80/d17/d18/d34/d65/d75/f84 [0,4194304] 0 2024-09-10T11:29:42.701 INFO:tasks.workunit.client.0.smithi144.stdout:1/714: dwrite d5/dd/f48 [0,4194304] 0 2024-09-10T11:29:42.703 INFO:tasks.workunit.client.0.smithi144.stdout:9/614: dread d11/d12/d1b/d22/f4a [0,4194304] 0 2024-09-10T11:29:42.704 INFO:tasks.workunit.client.0.smithi144.stdout:7/612: write d5/dc/d12/f52 [1761576,96821] 0 2024-09-10T11:29:42.704 INFO:tasks.workunit.client.0.smithi144.stdout:1/715: readlink d5/d4a/d44/da2/le5 0 2024-09-10T11:29:42.704 INFO:tasks.workunit.client.1.smithi178.stdout:4/855: dread d0/d9/d3d/d4f/f97 [0,4194304] 0 2024-09-10T11:29:42.706 INFO:tasks.workunit.client.0.smithi144.stdout:9/615: fdatasync d11/d36/d2d/f9a 0 2024-09-10T11:29:42.708 INFO:tasks.workunit.client.0.smithi144.stdout:1/716: write d5/d4a/d63/d84/dab/fbb [4512723,91969] 0 2024-09-10T11:29:42.711 INFO:tasks.workunit.client.0.smithi144.stdout:1/717: chown d5/d4a/d63/d84/d95/dd2/fde 158081 1 2024-09-10T11:29:42.711 INFO:tasks.workunit.client.0.smithi144.stdout:3/629: dwrite d4/d28/f36 [0,4194304] 0 2024-09-10T11:29:42.713 INFO:tasks.workunit.client.0.smithi144.stdout:3/630: dread f3 [0,4194304] 0 2024-09-10T11:29:42.713 INFO:tasks.workunit.client.0.smithi144.stdout:7/613: rename d5/c1b to d5/dc/d14/d40/d5e/d63/cc5 0 2024-09-10T11:29:42.716 INFO:tasks.workunit.client.0.smithi144.stdout:9/616: creat d11/d12/d1b/d41/fd9 x:0 0 0 2024-09-10T11:29:42.719 INFO:tasks.workunit.client.0.smithi144.stdout:7/614: creat d5/dc/d12/d2b/fc6 x:0 0 0 2024-09-10T11:29:42.721 INFO:tasks.workunit.client.0.smithi144.stdout:7/615: write d5/dc/d12/d2b/fc6 [118831,130767] 0 2024-09-10T11:29:42.723 INFO:tasks.workunit.client.0.smithi144.stdout:9/617: creat d11/d12/d1b/d41/d8f/dab/d64/fda x:0 0 0 2024-09-10T11:29:42.725 INFO:tasks.workunit.client.0.smithi144.stdout:9/618: readlink d11/d12/d1b/d2c/d78/la3 0 2024-09-10T11:29:42.729 INFO:tasks.workunit.client.1.smithi178.stdout:4/856: mkdir d0/d8/da/d1a/dff/d12f 0 2024-09-10T11:29:42.730 INFO:tasks.workunit.client.0.smithi144.stdout:1/718: dwrite d5/df/d4b/d85/f8b [0,4194304] 0 2024-09-10T11:29:42.731 INFO:tasks.workunit.client.1.smithi178.stdout:7/864: rename d6/d10/d1e/d2d/d2b/lf4 to d6/d10/d1d/d22/d50/d7e/d123/l132 0 2024-09-10T11:29:42.733 INFO:tasks.workunit.client.0.smithi144.stdout:9/619: dread d11/d12/d1b/d41/d8f/f82 [0,4194304] 0 2024-09-10T11:29:42.736 INFO:tasks.workunit.client.0.smithi144.stdout:9/620: chown d11/l62 12 1 2024-09-10T11:29:42.736 INFO:tasks.workunit.client.1.smithi178.stdout:1/967: link d3/d9/d21/f67 d3/d9/d98/dac/d3a/d50/db5/dcc/f169 0 2024-09-10T11:29:42.743 INFO:tasks.workunit.client.0.smithi144.stdout:1/719: dwrite d5/d2a/d65/d73/f24 [4194304,4194304] 0 2024-09-10T11:29:42.743 INFO:tasks.workunit.client.0.smithi144.stdout:3/631: dread d4/d28/d67/d80/d17/d18/f2e [0,4194304] 0 2024-09-10T11:29:42.745 INFO:tasks.workunit.client.0.smithi144.stdout:7/616: dread f1 [0,4194304] 0 2024-09-10T11:29:42.750 INFO:tasks.workunit.client.0.smithi144.stdout:1/720: rename d5/d4a/c6c to d5/d2a/d65/d73/dd8/ced 0 2024-09-10T11:29:42.751 INFO:tasks.workunit.client.0.smithi144.stdout:1/721: stat d5/f8 0 2024-09-10T11:29:42.754 INFO:tasks.workunit.client.0.smithi144.stdout:3/632: dwrite d4/dd/d21/d2f/f3a [0,4194304] 0 2024-09-10T11:29:42.755 INFO:tasks.workunit.client.0.smithi144.stdout:3/633: chown d4/dd/d21/d2f/l64 85036 1 2024-09-10T11:29:42.758 INFO:tasks.workunit.client.0.smithi144.stdout:1/722: dread d5/df/d4b/d85/f8b [0,4194304] 0 2024-09-10T11:29:42.758 INFO:tasks.workunit.client.0.smithi144.stdout:3/634: mkdir d4/d28/d67/d96/dcc 0 2024-09-10T11:29:42.763 INFO:tasks.workunit.client.0.smithi144.stdout:3/635: write d4/d28/d67/d80/fa2 [3839321,59102] 0 2024-09-10T11:29:42.768 INFO:tasks.workunit.client.0.smithi144.stdout:3/636: link d4/l10 d4/d28/d67/d80/d17/d18/d34/lcd 0 2024-09-10T11:29:42.769 INFO:tasks.workunit.client.1.smithi178.stdout:0/921: dwrite d0/f22 [0,4194304] 0 2024-09-10T11:29:42.773 INFO:tasks.workunit.client.1.smithi178.stdout:1/968: dread d3/d9/d98/dd1/d7e/fa7 [0,4194304] 0 2024-09-10T11:29:42.777 INFO:tasks.workunit.client.0.smithi144.stdout:1/723: truncate d5/d4a/d63/d84/d95/fd1 3008911 0 2024-09-10T11:29:42.779 INFO:tasks.workunit.client.0.smithi144.stdout:1/724: mkdir d5/dd/d64/d79/dee 0 2024-09-10T11:29:42.783 INFO:tasks.workunit.client.0.smithi144.stdout:7/617: dread d5/dc/d14/d40/d58/d6a/fa2 [0,4194304] 0 2024-09-10T11:29:42.785 INFO:tasks.workunit.client.1.smithi178.stdout:0/922: dwrite d0/d12/d3f/d72/f97 [0,4194304] 0 2024-09-10T11:29:42.785 INFO:tasks.workunit.client.0.smithi144.stdout:7/618: mknod d5/dc/d12/d2f/cc7 0 2024-09-10T11:29:42.787 INFO:tasks.workunit.client.1.smithi178.stdout:0/923: dread - d0/d12/d3f/fc9 zero size 2024-09-10T11:29:42.789 INFO:tasks.workunit.client.0.smithi144.stdout:7/619: rename d5/dc/d12/d2b/f43 to d5/dc/d14/d40/d58/d6a/db2/fc8 0 2024-09-10T11:29:42.790 INFO:tasks.workunit.client.1.smithi178.stdout:0/924: unlink d0/dc/d39/da4/dce/ddc/fdb 0 2024-09-10T11:29:42.791 INFO:tasks.workunit.client.1.smithi178.stdout:1/969: truncate d3/d35/fa4 5374893 0 2024-09-10T11:29:42.793 INFO:tasks.workunit.client.0.smithi144.stdout:7/620: creat d5/dc/d2a/d41/fc9 x:0 0 0 2024-09-10T11:29:42.795 INFO:tasks.workunit.client.1.smithi178.stdout:0/925: rename d0/d2c/f37 to d0/d2c/d95/f122 0 2024-09-10T11:29:42.796 INFO:tasks.workunit.client.1.smithi178.stdout:1/970: mknod d3/d9/d98/dac/d3a/d50/d6c/c16a 0 2024-09-10T11:29:42.797 INFO:tasks.workunit.client.1.smithi178.stdout:0/926: fsync d0/dc/f61 0 2024-09-10T11:29:42.800 INFO:tasks.workunit.client.0.smithi144.stdout:7/621: dwrite d5/dc/d2a/d60/f91 [0,4194304] 0 2024-09-10T11:29:42.802 INFO:tasks.workunit.client.0.smithi144.stdout:3/637: dread d4/dd/d21/f5d [0,4194304] 0 2024-09-10T11:29:42.804 INFO:tasks.workunit.client.1.smithi178.stdout:0/927: dwrite d0/dc/d39/d50/f105 [0,4194304] 0 2024-09-10T11:29:42.808 INFO:tasks.workunit.client.1.smithi178.stdout:0/928: creat d0/d2c/d93/f123 x:0 0 0 2024-09-10T11:29:42.809 INFO:tasks.workunit.client.1.smithi178.stdout:1/971: write d3/d9/d98/dac/f7f [642896,77381] 0 2024-09-10T11:29:42.810 INFO:tasks.workunit.client.0.smithi144.stdout:3/638: truncate d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/f8c 416463 0 2024-09-10T11:29:42.811 INFO:tasks.workunit.client.0.smithi144.stdout:7/622: dwrite d5/dc/d14/d40/f72 [0,4194304] 0 2024-09-10T11:29:42.815 INFO:tasks.workunit.client.0.smithi144.stdout:7/623: creat d5/dc/d14/d40/d5e/d63/d98/fca x:0 0 0 2024-09-10T11:29:42.818 INFO:tasks.workunit.client.0.smithi144.stdout:3/639: symlink d4/dd/d21/lce 0 2024-09-10T11:29:42.818 INFO:tasks.workunit.client.0.smithi144.stdout:7/624: mkdir d5/dc/d14/d40/d5e/d63/d98/dcb 0 2024-09-10T11:29:42.820 INFO:tasks.workunit.client.0.smithi144.stdout:7/625: write f1 [2243726,43768] 0 2024-09-10T11:29:42.828 INFO:tasks.workunit.client.1.smithi178.stdout:1/972: truncate d3/d35/f105 573953 0 2024-09-10T11:29:42.829 INFO:tasks.workunit.client.1.smithi178.stdout:1/973: fdatasync d3/d9/fc4 0 2024-09-10T11:29:42.830 INFO:tasks.workunit.client.1.smithi178.stdout:1/974: symlink d3/d35/dc6/l16b 0 2024-09-10T11:29:42.836 INFO:tasks.workunit.client.1.smithi178.stdout:1/975: unlink d3/d9/d98/dac/d3a/dae/ce8 0 2024-09-10T11:29:42.837 INFO:tasks.workunit.client.0.smithi144.stdout:7/626: truncate d5/dc/f56 519019 0 2024-09-10T11:29:42.839 INFO:tasks.workunit.client.0.smithi144.stdout:7/627: write d5/dc/d2a/fa8 [398779,60272] 0 2024-09-10T11:29:42.841 INFO:tasks.workunit.client.0.smithi144.stdout:7/628: write d5/dc/d2a/d41/f49 [2328844,110419] 0 2024-09-10T11:29:42.842 INFO:tasks.workunit.client.1.smithi178.stdout:0/929: dread d0/dc/f2b [0,4194304] 0 2024-09-10T11:29:42.843 INFO:tasks.workunit.client.1.smithi178.stdout:1/976: dwrite d3/d9/d21/f109 [0,4194304] 0 2024-09-10T11:29:42.844 INFO:tasks.workunit.client.1.smithi178.stdout:0/930: fsync d0/dc/d39/f9d 0 2024-09-10T11:29:42.844 INFO:tasks.workunit.client.0.smithi144.stdout:7/629: creat d5/dc/fcc x:0 0 0 2024-09-10T11:29:42.846 INFO:tasks.workunit.client.1.smithi178.stdout:1/977: write d3/d9/d98/dac/d3a/d50/ff9 [3478888,52999] 0 2024-09-10T11:29:42.847 INFO:tasks.workunit.client.0.smithi144.stdout:7/630: getdents d5/dc/d14/d40/d58/d6a/dad 0 2024-09-10T11:29:42.847 INFO:tasks.workunit.client.1.smithi178.stdout:0/931: symlink d0/dc/d39/d7c/l124 0 2024-09-10T11:29:42.848 INFO:tasks.workunit.client.1.smithi178.stdout:1/978: write d3/d9/d98/dac/d3a/d50/d6c/de9/d121/fa2 [3940240,60005] 0 2024-09-10T11:29:42.849 INFO:tasks.workunit.client.0.smithi144.stdout:7/631: write d5/dc/d12/d2b/fc6 [304162,19813] 0 2024-09-10T11:29:42.851 INFO:tasks.workunit.client.1.smithi178.stdout:0/932: creat d0/d2c/d95/f125 x:0 0 0 2024-09-10T11:29:42.852 INFO:tasks.workunit.client.1.smithi178.stdout:1/979: mknod d3/d9/d21/d133/c16c 0 2024-09-10T11:29:42.854 INFO:tasks.workunit.client.1.smithi178.stdout:1/980: dread - d3/d9/d98/dd1/f110 zero size 2024-09-10T11:29:42.857 INFO:tasks.workunit.client.0.smithi144.stdout:7/632: dwrite d5/dc/d12/f22 [0,4194304] 0 2024-09-10T11:29:42.858 INFO:tasks.workunit.client.0.smithi144.stdout:7/633: chown d5/dc/d11/f5c 10124 1 2024-09-10T11:29:42.859 INFO:tasks.workunit.client.0.smithi144.stdout:7/634: stat d5/dc/d14/ca5 0 2024-09-10T11:29:42.864 INFO:tasks.workunit.client.1.smithi178.stdout:0/933: write d0/dc/d39/d50/d5e/fcd [724316,17625] 0 2024-09-10T11:29:42.867 INFO:tasks.workunit.client.0.smithi144.stdout:7/635: rename d5/dc/d14/d40/d58/d6e/c84 to d5/dc/d14/d40/d5e/d63/ccd 0 2024-09-10T11:29:42.867 INFO:tasks.workunit.client.1.smithi178.stdout:0/934: read d0/dc/d39/d50/f71 [2613419,50780] 0 2024-09-10T11:29:42.870 INFO:tasks.workunit.client.1.smithi178.stdout:1/981: dwrite d3/d9/d1a/f6d [0,4194304] 0 2024-09-10T11:29:42.871 INFO:tasks.workunit.client.1.smithi178.stdout:0/935: mknod d0/dc/d1f/d2d/df9/c126 0 2024-09-10T11:29:42.874 INFO:tasks.workunit.client.1.smithi178.stdout:1/982: rename d3/d9/d21/d4b/f158 to d3/d9/d98/d11c/f16d 0 2024-09-10T11:29:42.877 INFO:tasks.workunit.client.1.smithi178.stdout:0/936: truncate d0/dc/d39/f10c 4107522 0 2024-09-10T11:29:42.877 INFO:tasks.workunit.client.1.smithi178.stdout:0/937: fdatasync d0/d2c/d93/f11a 0 2024-09-10T11:29:42.880 INFO:tasks.workunit.client.1.smithi178.stdout:0/938: unlink d0/d2c/d93/f11a 0 2024-09-10T11:29:42.883 INFO:tasks.workunit.client.1.smithi178.stdout:0/939: mkdir d0/d2c/d127 0 2024-09-10T11:29:42.884 INFO:tasks.workunit.client.0.smithi144.stdout:7/636: dwrite d5/f5a [0,4194304] 0 2024-09-10T11:29:42.885 INFO:tasks.workunit.client.1.smithi178.stdout:0/940: mkdir d0/d2c/d95/d128 0 2024-09-10T11:29:42.886 INFO:tasks.workunit.client.0.smithi144.stdout:7/637: write d5/dc/d14/d40/d58/fa3 [974377,3385] 0 2024-09-10T11:29:42.889 INFO:tasks.workunit.client.1.smithi178.stdout:0/941: truncate d0/dc/d39/d50/fc3 1618570 0 2024-09-10T11:29:42.891 INFO:tasks.workunit.client.0.smithi144.stdout:7/638: link d5/dc/d2a/fac d5/fce 0 2024-09-10T11:29:42.892 INFO:tasks.workunit.client.1.smithi178.stdout:0/942: mknod d0/dc/dc4/de8/c129 0 2024-09-10T11:29:42.897 INFO:tasks.workunit.client.1.smithi178.stdout:0/943: dwrite d0/dc/d39/d50/f88 [0,4194304] 0 2024-09-10T11:29:42.908 INFO:tasks.workunit.client.1.smithi178.stdout:0/944: dread d0/d12/f64 [0,4194304] 0 2024-09-10T11:29:42.917 INFO:tasks.workunit.client.1.smithi178.stdout:0/945: write d0/dc/fe6 [672008,63006] 0 2024-09-10T11:29:42.920 INFO:tasks.workunit.client.1.smithi178.stdout:0/946: rmdir d0/dc/d39/d50/d114 0 2024-09-10T11:29:42.920 INFO:tasks.workunit.client.1.smithi178.stdout:0/947: chown d0/dc/d39/d7c/fbb 114032 1 2024-09-10T11:29:42.922 INFO:tasks.workunit.client.1.smithi178.stdout:0/948: mknod d0/c12a 0 2024-09-10T11:29:42.923 INFO:tasks.workunit.client.1.smithi178.stdout:0/949: read d0/f8 [166896,47045] 0 2024-09-10T11:29:42.924 INFO:tasks.workunit.client.1.smithi178.stdout:0/950: mknod d0/db1/c12b 0 2024-09-10T11:29:42.925 INFO:tasks.workunit.client.1.smithi178.stdout:0/951: write d0/dc/d39/d7c/f9b [2049643,1292] 0 2024-09-10T11:29:42.935 INFO:tasks.workunit.client.1.smithi178.stdout:0/952: dread d0/dc/d1f/d2d/dd1/f98 [0,4194304] 0 2024-09-10T11:29:42.948 INFO:tasks.workunit.client.1.smithi178.stdout:0/953: dread d0/dc/d39/d4c/f73 [0,4194304] 0 2024-09-10T11:29:42.950 INFO:tasks.workunit.client.1.smithi178.stdout:0/954: dread d0/d2c/d33/f81 [0,4194304] 0 2024-09-10T11:29:42.953 INFO:tasks.workunit.client.1.smithi178.stdout:0/955: link d0/d2c/d33/l6d d0/d2c/d95/d128/l12c 0 2024-09-10T11:29:42.958 INFO:tasks.workunit.client.1.smithi178.stdout:0/956: dwrite d0/dc/d39/d4c/f103 [0,4194304] 0 2024-09-10T11:29:42.959 INFO:tasks.workunit.client.1.smithi178.stdout:0/957: fsync d0/dc/d39/f9d 0 2024-09-10T11:29:42.961 INFO:tasks.workunit.client.1.smithi178.stdout:0/958: mkdir d0/dc/d1f/d2d/dd1/d12d 0 2024-09-10T11:29:42.962 INFO:tasks.workunit.client.1.smithi178.stdout:0/959: chown d0/dc/d39/da4/caf 2456 1 2024-09-10T11:29:42.964 INFO:tasks.workunit.client.1.smithi178.stdout:0/960: mkdir d0/dc/d39/d75/d12e 0 2024-09-10T11:29:42.965 INFO:tasks.workunit.client.1.smithi178.stdout:0/961: read d0/d2c/f11c [3508732,3353] 0 2024-09-10T11:29:42.968 INFO:tasks.workunit.client.1.smithi178.stdout:0/962: rename d0/dc/d39/d7c to d0/d2c/d95/dbd/dd5/d12f 0 2024-09-10T11:29:42.969 INFO:tasks.workunit.client.1.smithi178.stdout:0/963: mknod d0/dc/c130 0 2024-09-10T11:29:42.971 INFO:tasks.workunit.client.1.smithi178.stdout:0/964: symlink d0/d2c/d95/dbd/l131 0 2024-09-10T11:29:42.976 INFO:tasks.workunit.client.1.smithi178.stdout:0/965: write d0/dc/d39/d75/fd4 [42309,5008] 0 2024-09-10T11:29:42.977 INFO:tasks.workunit.client.1.smithi178.stdout:0/966: truncate d0/d2c/d93/f111 730209 0 2024-09-10T11:29:42.978 INFO:tasks.workunit.client.1.smithi178.stdout:0/967: symlink d0/dc/d1f/l132 0 2024-09-10T11:29:42.980 INFO:tasks.workunit.client.1.smithi178.stdout:0/968: mkdir d0/dc/d39/d75/d120/d133 0 2024-09-10T11:29:42.987 INFO:tasks.workunit.client.1.smithi178.stdout:0/969: dwrite d0/dc/d1f/f26 [0,4194304] 0 2024-09-10T11:29:42.988 INFO:tasks.workunit.client.1.smithi178.stdout:0/970: chown d0/d12/d3f/d72/c79 1 1 2024-09-10T11:29:42.994 INFO:tasks.workunit.client.1.smithi178.stdout:7/865: sync 2024-09-10T11:29:42.994 INFO:tasks.workunit.client.1.smithi178.stdout:4/857: sync 2024-09-10T11:29:42.997 INFO:tasks.workunit.client.1.smithi178.stdout:7/866: truncate d6/d10/d1d/d22/d3b/fbe 149034 0 2024-09-10T11:29:42.998 INFO:tasks.workunit.client.1.smithi178.stdout:4/858: fdatasync d0/d8/da/f27 0 2024-09-10T11:29:42.999 INFO:tasks.workunit.client.1.smithi178.stdout:4/859: fdatasync d0/d8/da/d1a/d51/d5d/fa1 0 2024-09-10T11:29:43.000 INFO:tasks.workunit.client.1.smithi178.stdout:4/860: write d0/d9d/da5/fb8 [909014,90851] 0 2024-09-10T11:29:43.002 INFO:tasks.workunit.client.1.smithi178.stdout:4/861: creat d0/d8/d3e/dea/f130 x:0 0 0 2024-09-10T11:29:43.004 INFO:tasks.workunit.client.1.smithi178.stdout:4/862: symlink d0/d8/da/d1a/d51/d5d/dc9/dee/d104/l131 0 2024-09-10T11:29:43.013 INFO:tasks.workunit.client.1.smithi178.stdout:4/863: dread d0/d63/de0/ff8 [0,4194304] 0 2024-09-10T11:29:43.014 INFO:tasks.workunit.client.1.smithi178.stdout:4/864: dread - d0/d8/df/f59 zero size 2024-09-10T11:29:43.018 INFO:tasks.workunit.client.1.smithi178.stdout:4/865: write d0/f38 [5087902,43762] 0 2024-09-10T11:29:43.020 INFO:tasks.workunit.client.1.smithi178.stdout:4/866: mknod d0/d9/dc/d8e/c132 0 2024-09-10T11:29:43.026 INFO:tasks.workunit.client.1.smithi178.stdout:4/867: dread d0/d9/dc/dca/f10b [0,4194304] 0 2024-09-10T11:29:43.029 INFO:tasks.workunit.client.1.smithi178.stdout:4/868: dread d0/d8/da/d1a/f8b [0,4194304] 0 2024-09-10T11:29:43.029 INFO:tasks.workunit.client.1.smithi178.stdout:4/869: chown d0/d10 105218 1 2024-09-10T11:29:43.030 INFO:tasks.workunit.client.1.smithi178.stdout:4/870: readlink d0/d10/l7b 0 2024-09-10T11:29:43.035 INFO:tasks.workunit.client.1.smithi178.stdout:4/871: dwrite d0/d10/fa2 [0,4194304] 0 2024-09-10T11:29:43.037 INFO:tasks.workunit.client.1.smithi178.stdout:4/872: write d0/d8/da/f101 [737082,22832] 0 2024-09-10T11:29:43.040 INFO:tasks.workunit.client.1.smithi178.stdout:4/873: mkdir d0/d8/da/d1a/dff/d133 0 2024-09-10T11:29:43.042 INFO:tasks.workunit.client.1.smithi178.stdout:4/874: fdatasync d0/d8/da/d1a/d51/d5d/db6/df3/f125 0 2024-09-10T11:29:43.044 INFO:tasks.workunit.client.1.smithi178.stdout:4/875: write d0/d9/dc/d8e/ffc [372468,107266] 0 2024-09-10T11:29:43.045 INFO:tasks.workunit.client.1.smithi178.stdout:4/876: write d0/d9/d3d/d4f/f97 [5144580,56686] 0 2024-09-10T11:29:43.047 INFO:tasks.workunit.client.1.smithi178.stdout:4/877: mkdir d0/d9/dc/d8e/df1/d134 0 2024-09-10T11:29:43.048 INFO:tasks.workunit.client.1.smithi178.stdout:4/878: stat d0/d8/df/ca3 0 2024-09-10T11:29:43.052 INFO:tasks.workunit.client.1.smithi178.stdout:4/879: write d0/d8/da/f3f [286797,20849] 0 2024-09-10T11:29:43.053 INFO:tasks.workunit.client.1.smithi178.stdout:4/880: creat d0/d8/df/dfd/f135 x:0 0 0 2024-09-10T11:29:43.055 INFO:tasks.workunit.client.1.smithi178.stdout:4/881: creat d0/d9/d3d/f136 x:0 0 0 2024-09-10T11:29:43.055 INFO:tasks.workunit.client.1.smithi178.stdout:4/882: readlink d0/d10/l7b 0 2024-09-10T11:29:43.063 INFO:tasks.workunit.client.1.smithi178.stdout:4/883: dwrite d0/d8/df/f9e [0,4194304] 0 2024-09-10T11:29:43.063 INFO:tasks.workunit.client.1.smithi178.stdout:4/884: write d0/fdc [752437,71980] 0 2024-09-10T11:29:43.065 INFO:tasks.workunit.client.1.smithi178.stdout:4/885: creat d0/d9/d3d/d4f/ddb/f137 x:0 0 0 2024-09-10T11:29:43.068 INFO:tasks.workunit.client.1.smithi178.stdout:4/886: mkdir d0/d9/d138 0 2024-09-10T11:29:43.071 INFO:tasks.workunit.client.1.smithi178.stdout:4/887: rename d0/f5 to d0/d10/f139 0 2024-09-10T11:29:43.072 INFO:tasks.workunit.client.1.smithi178.stdout:4/888: unlink d0/d8/da/d1a/l41 0 2024-09-10T11:29:43.079 INFO:tasks.workunit.client.1.smithi178.stdout:4/889: truncate d0/d9/dc/d8e/f9f 1668857 0 2024-09-10T11:29:43.081 INFO:tasks.workunit.client.1.smithi178.stdout:4/890: getdents d0/d9d/dd5 0 2024-09-10T11:29:43.083 INFO:tasks.workunit.client.1.smithi178.stdout:4/891: creat d0/d8/d3e/d123/f13a x:0 0 0 2024-09-10T11:29:43.087 INFO:tasks.workunit.client.1.smithi178.stdout:4/892: rename d0/d8/da/d1a/d51 to d0/d8/df/d13b 0 2024-09-10T11:29:43.091 INFO:tasks.workunit.client.1.smithi178.stdout:1/983: sync 2024-09-10T11:29:43.096 INFO:tasks.workunit.client.1.smithi178.stdout:4/893: dwrite d0/f6e [0,4194304] 0 2024-09-10T11:29:43.099 INFO:tasks.workunit.client.1.smithi178.stdout:4/894: unlink d0/d8/df/d13b/fd7 0 2024-09-10T11:29:43.100 INFO:tasks.workunit.client.1.smithi178.stdout:1/984: dwrite d3/d9/d1a/d48/f90 [0,4194304] 0 2024-09-10T11:29:43.101 INFO:tasks.workunit.client.1.smithi178.stdout:4/895: symlink d0/d9/dcf/l13c 0 2024-09-10T11:29:43.106 INFO:tasks.workunit.client.1.smithi178.stdout:1/985: rename d3/d9/d98/fca to d3/d9/d98/dac/d3a/d50/db5/de3/f16e 0 2024-09-10T11:29:43.107 INFO:tasks.workunit.client.1.smithi178.stdout:1/986: fsync d3/d35/fe6 0 2024-09-10T11:29:43.114 INFO:tasks.workunit.client.1.smithi178.stdout:1/987: dwrite d3/d9/d98/dac/d3a/d50/d80/fef [4194304,4194304] 0 2024-09-10T11:29:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:42 smithi178 ceph-mon[33514]: pgmap v28: 65 pgs: 65 active+clean; 2.7 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 100 MiB/s rd, 279 MiB/s wr, 523 op/s 2024-09-10T11:29:43.119 INFO:tasks.workunit.client.1.smithi178.stdout:4/896: write d0/f94 [5152013,111601] 0 2024-09-10T11:29:43.120 INFO:tasks.workunit.client.1.smithi178.stdout:1/988: dread d3/d9/d98/dac/f7f [0,4194304] 0 2024-09-10T11:29:43.123 INFO:tasks.workunit.client.1.smithi178.stdout:4/897: mknod d0/d9/dc/d50/c13d 0 2024-09-10T11:29:43.124 INFO:tasks.workunit.client.1.smithi178.stdout:1/989: creat d3/d9/d98/dac/d3a/d153/f16f x:0 0 0 2024-09-10T11:29:43.124 INFO:tasks.workunit.client.1.smithi178.stdout:4/898: mknod d0/d9d/da5/c13e 0 2024-09-10T11:29:43.125 INFO:tasks.workunit.client.1.smithi178.stdout:4/899: chown d0/d8/da/f2b 3177 1 2024-09-10T11:29:43.125 INFO:tasks.workunit.client.1.smithi178.stdout:7/867: sync 2024-09-10T11:29:43.125 INFO:tasks.workunit.client.1.smithi178.stdout:0/971: sync 2024-09-10T11:29:43.126 INFO:tasks.workunit.client.1.smithi178.stdout:4/900: fdatasync d0/d8/f107 0 2024-09-10T11:29:43.127 INFO:tasks.workunit.client.1.smithi178.stdout:0/972: rmdir d0/d2c/d33 39 2024-09-10T11:29:43.137 INFO:tasks.workunit.client.1.smithi178.stdout:1/990: truncate d3/d9/d98/dac/d3a/d6e/fe7 545012 0 2024-09-10T11:29:43.138 INFO:tasks.workunit.client.1.smithi178.stdout:0/973: write d0/d2c/d33/f45 [802089,2531] 0 2024-09-10T11:29:43.138 INFO:tasks.workunit.client.1.smithi178.stdout:1/991: chown d3/d9/f12b 16559 1 2024-09-10T11:29:43.141 INFO:tasks.workunit.client.1.smithi178.stdout:1/992: creat d3/d9/d98/dac/f170 x:0 0 0 2024-09-10T11:29:43.142 INFO:tasks.workunit.client.1.smithi178.stdout:7/868: dread d6/d10/d1d/d4a/d79/df1/f8c [0,4194304] 0 2024-09-10T11:29:43.142 INFO:tasks.workunit.client.1.smithi178.stdout:0/974: unlink d0/d12/d3f/d72/df0/f100 0 2024-09-10T11:29:43.143 INFO:tasks.workunit.client.1.smithi178.stdout:4/901: dread d0/d8/da/f2b [0,4194304] 0 2024-09-10T11:29:43.144 INFO:tasks.workunit.client.1.smithi178.stdout:7/869: dread d6/d10/d34/f4e [0,4194304] 0 2024-09-10T11:29:43.146 INFO:tasks.workunit.client.1.smithi178.stdout:1/993: unlink d3/d9/d98/dd1/ddc/lb0 0 2024-09-10T11:29:43.148 INFO:tasks.workunit.client.1.smithi178.stdout:7/870: mknod d6/d10/d34/d6a/d126/c133 0 2024-09-10T11:29:43.149 INFO:tasks.workunit.client.1.smithi178.stdout:7/871: chown d6/d10/d1e/d2d/d2b/da0/c51 1 1 2024-09-10T11:29:43.149 INFO:tasks.workunit.client.1.smithi178.stdout:1/994: mknod d3/d35/dc6/c171 0 2024-09-10T11:29:43.152 INFO:tasks.workunit.client.1.smithi178.stdout:7/872: mkdir d6/d10/d1d/daa/d134 0 2024-09-10T11:29:43.154 INFO:tasks.workunit.client.1.smithi178.stdout:7/873: truncate d6/d10/d1d/d4a/d79/df1/fff 687868 0 2024-09-10T11:29:43.157 INFO:tasks.workunit.client.1.smithi178.stdout:1/995: rmdir d3/d9/d98/dd1/ddc 39 2024-09-10T11:29:43.158 INFO:tasks.workunit.client.1.smithi178.stdout:0/975: dwrite d0/d12/f27 [0,4194304] 0 2024-09-10T11:29:43.159 INFO:tasks.workunit.client.1.smithi178.stdout:1/996: read - d3/d9/d98/dac/d3a/d50/d9e/dc2/de2/f162 zero size 2024-09-10T11:29:43.161 INFO:tasks.workunit.client.1.smithi178.stdout:1/997: truncate d3/d35/fe6 631121 0 2024-09-10T11:29:43.161 INFO:tasks.workunit.client.1.smithi178.stdout:7/874: link d6/d10/d1d/d4a/d79/d83/fcd d6/d10/d1d/d22/d50/d7e/d82/db2/d49/f135 0 2024-09-10T11:29:43.165 INFO:tasks.workunit.client.1.smithi178.stdout:0/976: chown d0/dc/d39/da4/caf 17 1 2024-09-10T11:29:43.168 INFO:tasks.workunit.client.1.smithi178.stdout:1/998: mkdir d3/d9/d98/dac/d172 0 2024-09-10T11:29:43.170 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:42 smithi144 ceph-mon[27942]: pgmap v28: 65 pgs: 65 active+clean; 2.7 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 100 MiB/s rd, 279 MiB/s wr, 523 op/s 2024-09-10T11:29:43.170 INFO:tasks.workunit.client.0.smithi144.stdout:4/654: sync 2024-09-10T11:29:43.172 INFO:tasks.workunit.client.1.smithi178.stdout:1/999: mkdir d3/d9/d98/dac/d3a/dae/d173 0 2024-09-10T11:29:43.173 INFO:tasks.workunit.client.0.smithi144.stdout:4/655: fdatasync fd 0 2024-09-10T11:29:43.174 INFO:tasks.workunit.client.1.smithi178.stdout:0/977: getdents d0/d2c/d95/d128 0 2024-09-10T11:29:43.175 INFO:tasks.workunit.client.1.smithi178.stdout:0/978: rmdir d0/d2c/d93/d107 39 2024-09-10T11:29:43.177 INFO:tasks.workunit.client.0.smithi144.stdout:4/656: dread d14/d1e/d22/d34/d3e/d40/d4b/f6a [0,4194304] 0 2024-09-10T11:29:43.177 INFO:tasks.workunit.client.1.smithi178.stdout:0/979: fdatasync d0/f4e 0 2024-09-10T11:29:43.179 INFO:tasks.workunit.client.1.smithi178.stdout:7/875: dwrite d6/f14 [8388608,4194304] 0 2024-09-10T11:29:43.179 INFO:tasks.workunit.client.0.smithi144.stdout:4/657: rename d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/de4 to d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/de4/df0 22 2024-09-10T11:29:43.182 INFO:tasks.workunit.client.0.smithi144.stdout:4/658: rmdir d14/d67/d84/db3 39 2024-09-10T11:29:43.184 INFO:tasks.workunit.client.1.smithi178.stdout:0/980: creat d0/d2c/d95/dbd/dd5/d12f/d85/f134 x:0 0 0 2024-09-10T11:29:43.184 INFO:tasks.workunit.client.1.smithi178.stdout:0/981: readlink d0/d12/ld0 0 2024-09-10T11:29:43.184 INFO:tasks.workunit.client.0.smithi144.stdout:4/659: creat d14/d1e/d83/da4/da9/dbc/ff1 x:0 0 0 2024-09-10T11:29:43.186 INFO:tasks.workunit.client.1.smithi178.stdout:0/982: creat d0/d2c/d89/f135 x:0 0 0 2024-09-10T11:29:43.192 INFO:tasks.workunit.client.1.smithi178.stdout:0/983: link d0/dc/d39/f68 d0/dc/f136 0 2024-09-10T11:29:43.193 INFO:tasks.workunit.client.0.smithi144.stdout:4/660: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/dab/fc5 [4194304,4194304] 0 2024-09-10T11:29:43.195 INFO:tasks.workunit.client.0.smithi144.stdout:4/661: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/fd2 [314201,92564] 0 2024-09-10T11:29:43.195 INFO:tasks.workunit.client.1.smithi178.stdout:0/984: creat d0/dc/d1f/dea/f137 x:0 0 0 2024-09-10T11:29:43.198 INFO:tasks.workunit.client.0.smithi144.stdout:4/662: creat d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/ff2 x:0 0 0 2024-09-10T11:29:43.199 INFO:tasks.workunit.client.0.smithi144.stdout:4/663: readlink d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/l9b 0 2024-09-10T11:29:43.201 INFO:tasks.workunit.client.0.smithi144.stdout:4/664: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/f38 [4326959,40421] 0 2024-09-10T11:29:43.202 INFO:tasks.workunit.client.0.smithi144.stdout:4/665: write d14/d1e/d22/f31 [2348950,71014] 0 2024-09-10T11:29:43.210 INFO:tasks.workunit.client.0.smithi144.stdout:4/666: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/f5f [261206,127913] 0 2024-09-10T11:29:43.220 INFO:tasks.workunit.client.0.smithi144.stdout:0/452: sync 2024-09-10T11:29:43.220 INFO:tasks.workunit.client.0.smithi144.stdout:8/570: sync 2024-09-10T11:29:43.220 INFO:tasks.workunit.client.0.smithi144.stdout:1/725: sync 2024-09-10T11:29:43.220 INFO:tasks.workunit.client.0.smithi144.stdout:6/559: sync 2024-09-10T11:29:43.220 INFO:tasks.workunit.client.0.smithi144.stdout:7/639: sync 2024-09-10T11:29:43.220 INFO:tasks.workunit.client.0.smithi144.stdout:2/527: sync 2024-09-10T11:29:43.221 INFO:tasks.workunit.client.0.smithi144.stdout:5/681: sync 2024-09-10T11:29:43.221 INFO:tasks.workunit.client.0.smithi144.stdout:9/621: sync 2024-09-10T11:29:43.221 INFO:tasks.workunit.client.0.smithi144.stdout:3/640: sync 2024-09-10T11:29:43.223 INFO:tasks.workunit.client.0.smithi144.stdout:2/528: read d7/f8 [4562681,120389] 0 2024-09-10T11:29:43.224 INFO:tasks.workunit.client.0.smithi144.stdout:6/560: truncate d9/dd/d3e/d47/d13/d60/fba 668677 0 2024-09-10T11:29:43.230 INFO:tasks.workunit.client.0.smithi144.stdout:4/667: mkdir d14/d1e/d22/d34/d3e/d40/d4b/d4c/dab/df3 0 2024-09-10T11:29:43.230 INFO:tasks.workunit.client.0.smithi144.stdout:4/668: chown d14/d1e/d22/d34/d3e/fe2 127 1 2024-09-10T11:29:43.231 INFO:tasks.workunit.client.0.smithi144.stdout:9/622: unlink d11/d36/c34 0 2024-09-10T11:29:43.232 INFO:tasks.workunit.client.0.smithi144.stdout:0/453: creat da/d3a/d4b/d4e/f88 x:0 0 0 2024-09-10T11:29:43.233 INFO:tasks.workunit.client.0.smithi144.stdout:1/726: mkdir d5/d4a/d63/d84/dbd/dbe/def 0 2024-09-10T11:29:43.234 INFO:tasks.workunit.client.0.smithi144.stdout:3/641: fsync d4/d28/d67/d80/d17/d18/f43 0 2024-09-10T11:29:43.235 INFO:tasks.workunit.client.0.smithi144.stdout:1/727: dread - d5/d2a/d65/faf zero size 2024-09-10T11:29:43.236 INFO:tasks.workunit.client.0.smithi144.stdout:6/561: mkdir d9/dd/d3e/d47/d13/d87/dc9 0 2024-09-10T11:29:43.236 INFO:tasks.workunit.client.0.smithi144.stdout:2/529: write d7/da/f19 [3867295,4854] 0 2024-09-10T11:29:43.237 INFO:tasks.workunit.client.0.smithi144.stdout:4/669: mkdir d14/d1e/d83/da4/df4 0 2024-09-10T11:29:43.238 INFO:tasks.workunit.client.0.smithi144.stdout:8/571: getdents d0/d1b/d47/d5e/d76 0 2024-09-10T11:29:43.239 INFO:tasks.workunit.client.0.smithi144.stdout:4/670: stat d14/d7e/f9f 0 2024-09-10T11:29:43.240 INFO:tasks.workunit.client.0.smithi144.stdout:9/623: dwrite d11/d12/d6f/fd3 [0,4194304] 0 2024-09-10T11:29:43.240 INFO:tasks.workunit.client.0.smithi144.stdout:7/640: creat d5/dc/d14/d40/d5e/dc4/fcf x:0 0 0 2024-09-10T11:29:43.240 INFO:tasks.workunit.client.0.smithi144.stdout:0/454: readlink da/l3f 0 2024-09-10T11:29:43.241 INFO:tasks.workunit.client.1.smithi178.stdout:0/985: creat d0/dc/d39/d4c/f138 x:0 0 0 2024-09-10T11:29:43.241 INFO:tasks.workunit.client.1.smithi178.stdout:0/986: chown d0/d12/d3f/d72/f82 236 1 2024-09-10T11:29:43.245 INFO:tasks.workunit.client.0.smithi144.stdout:3/642: unlink d4/dd/d21/f62 0 2024-09-10T11:29:43.246 INFO:tasks.workunit.client.0.smithi144.stdout:1/728: creat d5/df/d4b/d85/ff0 x:0 0 0 2024-09-10T11:29:43.246 INFO:tasks.workunit.client.0.smithi144.stdout:4/671: write d14/d1e/d22/d34/d3e/d40/fac [1188717,168] 0 2024-09-10T11:29:43.246 INFO:tasks.workunit.client.0.smithi144.stdout:6/562: truncate d9/f81 211177 0 2024-09-10T11:29:43.247 INFO:tasks.workunit.client.0.smithi144.stdout:4/672: read - d14/d1e/d65/fc6 zero size 2024-09-10T11:29:43.247 INFO:tasks.workunit.client.1.smithi178.stdout:0/987: dread d0/dc/d39/f68 [0,4194304] 0 2024-09-10T11:29:43.248 INFO:tasks.workunit.client.0.smithi144.stdout:6/563: chown d9/dd/d11/d1b/d2f/dab/dc6 387501955 1 2024-09-10T11:29:43.248 INFO:tasks.workunit.client.0.smithi144.stdout:8/572: mknod d0/d9/d21/d25/d4f/cbf 0 2024-09-10T11:29:43.250 INFO:tasks.workunit.client.1.smithi178.stdout:0/988: mknod d0/dc/d1f/d2d/dd1/d12d/c139 0 2024-09-10T11:29:43.251 INFO:tasks.workunit.client.0.smithi144.stdout:0/455: mknod da/d3a/d4b/d4e/c89 0 2024-09-10T11:29:43.251 INFO:tasks.workunit.client.1.smithi178.stdout:0/989: fsync d0/dc/ff 0 2024-09-10T11:29:43.252 INFO:tasks.workunit.client.0.smithi144.stdout:3/643: mknod d4/d28/d67/d80/d17/d18/d34/d65/d75/ccf 0 2024-09-10T11:29:43.253 INFO:tasks.workunit.client.1.smithi178.stdout:0/990: fsync d0/dc/d39/d50/d5e/f7d 0 2024-09-10T11:29:43.255 INFO:tasks.workunit.client.0.smithi144.stdout:4/673: creat d14/d67/d69/d90/dda/ff5 x:0 0 0 2024-09-10T11:29:43.255 INFO:tasks.workunit.client.0.smithi144.stdout:6/564: dwrite d9/dd/d11/d1b/f8d [0,4194304] 0 2024-09-10T11:29:43.259 INFO:tasks.workunit.client.0.smithi144.stdout:9/624: rename d11/f21 to d11/d12/d1b/fdb 0 2024-09-10T11:29:43.262 INFO:tasks.workunit.client.0.smithi144.stdout:3/644: truncate d4/f37 4008945 0 2024-09-10T11:29:43.263 INFO:tasks.workunit.client.0.smithi144.stdout:2/530: dread d7/de/d36/d45/d47/d6c/fa2 [0,4194304] 0 2024-09-10T11:29:43.263 INFO:tasks.workunit.client.0.smithi144.stdout:4/674: creat d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/ff6 x:0 0 0 2024-09-10T11:29:43.264 INFO:tasks.workunit.client.0.smithi144.stdout:3/645: fdatasync d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93/fbf 0 2024-09-10T11:29:43.265 INFO:tasks.workunit.client.0.smithi144.stdout:7/641: rename d5/dc/d14/d40/d5e/d68/c83 to d5/dc/d11/d18/cd0 0 2024-09-10T11:29:43.265 INFO:tasks.workunit.client.0.smithi144.stdout:3/646: stat d4/dd/d21/f8b 0 2024-09-10T11:29:43.268 INFO:tasks.workunit.client.0.smithi144.stdout:2/531: mknod d7/d29/d92/cb1 0 2024-09-10T11:29:43.269 INFO:tasks.workunit.client.0.smithi144.stdout:1/729: rename d5/dd/d8f to d5/d4a/d63/d84/dbd/df1 0 2024-09-10T11:29:43.270 INFO:tasks.workunit.client.0.smithi144.stdout:4/675: dread d14/d1e/d22/d34/d3e/fd0 [0,4194304] 0 2024-09-10T11:29:43.271 INFO:tasks.workunit.client.0.smithi144.stdout:3/647: write d4/dd/f1b [2476221,105100] 0 2024-09-10T11:29:43.272 INFO:tasks.workunit.client.0.smithi144.stdout:7/642: rmdir d5/dc/d14/d40/d58/d6a 39 2024-09-10T11:29:43.276 INFO:tasks.workunit.client.0.smithi144.stdout:4/676: creat d14/d1e/d22/d34/d91/ff7 x:0 0 0 2024-09-10T11:29:43.277 INFO:tasks.workunit.client.0.smithi144.stdout:7/643: write d5/dc/d12/f52 [312939,128419] 0 2024-09-10T11:29:43.277 INFO:tasks.workunit.client.0.smithi144.stdout:2/532: dwrite d7/de/d36/d45/d8e/d9a/f9c [0,4194304] 0 2024-09-10T11:29:43.278 INFO:tasks.workunit.client.0.smithi144.stdout:8/573: rename d0/d84/lab to d0/d9/d21/d7e/d85/lc0 0 2024-09-10T11:29:43.279 INFO:tasks.workunit.client.0.smithi144.stdout:1/730: truncate d5/df/f33 3708384 0 2024-09-10T11:29:43.281 INFO:tasks.workunit.client.0.smithi144.stdout:0/456: rename da/l20 to da/d44/d82/l8a 0 2024-09-10T11:29:43.284 INFO:tasks.workunit.client.0.smithi144.stdout:7/644: rmdir d5/dc/d14/d40/d5e/d63/d98 39 2024-09-10T11:29:43.287 INFO:tasks.workunit.client.0.smithi144.stdout:2/533: unlink d7/de/d3d/d4e/f72 0 2024-09-10T11:29:43.287 INFO:tasks.workunit.client.0.smithi144.stdout:4/677: dwrite d14/d67/d69/dca/fdb [0,4194304] 0 2024-09-10T11:29:43.288 INFO:tasks.workunit.client.1.smithi178.stdout:4/902: sync 2024-09-10T11:29:43.288 INFO:tasks.workunit.client.0.smithi144.stdout:7/645: write d5/dc/d14/d40/f67 [4725989,12257] 0 2024-09-10T11:29:43.289 INFO:tasks.workunit.client.1.smithi178.stdout:4/903: write d0/d8/d3e/d123/f13a [1031415,81523] 0 2024-09-10T11:29:43.290 INFO:tasks.workunit.client.0.smithi144.stdout:4/678: chown d14/dcb 0 1 2024-09-10T11:29:43.290 INFO:tasks.workunit.client.0.smithi144.stdout:0/457: write da/d3a/f4d [3099675,43981] 0 2024-09-10T11:29:43.291 INFO:tasks.workunit.client.1.smithi178.stdout:4/904: write d0/d8/da/f101 [469850,54219] 0 2024-09-10T11:29:43.293 INFO:tasks.workunit.client.0.smithi144.stdout:8/574: dwrite d0/d9/d23/f40 [4194304,4194304] 0 2024-09-10T11:29:43.294 INFO:tasks.workunit.client.0.smithi144.stdout:7/646: write d5/dc/d14/d40/d5e/d68/fb6 [919067,110077] 0 2024-09-10T11:29:43.295 INFO:tasks.workunit.client.0.smithi144.stdout:8/575: fsync d0/d9/d21/d25/d4f/f56 0 2024-09-10T11:29:43.295 INFO:tasks.workunit.client.0.smithi144.stdout:7/647: read - d5/dc/d2a/d60/fb9 zero size 2024-09-10T11:29:43.297 INFO:tasks.workunit.client.0.smithi144.stdout:8/576: write d0/d1b/d5a/d8d/f98 [3538582,63919] 0 2024-09-10T11:29:43.300 INFO:tasks.workunit.client.0.smithi144.stdout:4/679: truncate d14/d67/d84/db3/fee 1040236 0 2024-09-10T11:29:43.301 INFO:tasks.workunit.client.0.smithi144.stdout:8/577: dread - d0/d9/d21/f77 zero size 2024-09-10T11:29:43.302 INFO:tasks.workunit.client.0.smithi144.stdout:4/680: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/fba [1000950,71752] 0 2024-09-10T11:29:43.306 INFO:tasks.workunit.client.0.smithi144.stdout:0/458: creat da/d44/d82/f8b x:0 0 0 2024-09-10T11:29:43.307 INFO:tasks.workunit.client.0.smithi144.stdout:5/682: truncate d4/d20/d24/d2d/d4b/f4e 2167322 0 2024-09-10T11:29:43.309 INFO:tasks.workunit.client.0.smithi144.stdout:5/683: read d4/d20/d24/d2d/d4b/fa7 [495937,4845] 0 2024-09-10T11:29:43.309 INFO:tasks.workunit.client.1.smithi178.stdout:4/905: dread d0/d8/df/f29 [0,4194304] 0 2024-09-10T11:29:43.313 INFO:tasks.workunit.client.1.smithi178.stdout:4/906: mknod d0/d8/df/d13b/d5d/db6/df3/c13f 0 2024-09-10T11:29:43.314 INFO:tasks.workunit.client.0.smithi144.stdout:6/565: rename d9/f71 to d9/dd/d11/d1b/d2f/d7f/dc4/d5b/fca 0 2024-09-10T11:29:43.315 INFO:tasks.workunit.client.0.smithi144.stdout:6/566: fsync d9/dd/d73/fb1 0 2024-09-10T11:29:43.315 INFO:tasks.workunit.client.1.smithi178.stdout:4/907: symlink d0/d9/d3d/l140 0 2024-09-10T11:29:43.316 INFO:tasks.workunit.client.0.smithi144.stdout:6/567: stat d9/dd/d11/d1b/f94 0 2024-09-10T11:29:43.317 INFO:tasks.workunit.client.0.smithi144.stdout:4/681: symlink d14/d1e/lf8 0 2024-09-10T11:29:43.318 INFO:tasks.workunit.client.1.smithi178.stdout:4/908: creat d0/d9d/f141 x:0 0 0 2024-09-10T11:29:43.318 INFO:tasks.workunit.client.0.smithi144.stdout:0/459: mkdir da/d44/d82/d8c 0 2024-09-10T11:29:43.321 INFO:tasks.workunit.client.1.smithi178.stdout:4/909: rename d0/f42 to d0/d9/d3d/f142 0 2024-09-10T11:29:43.322 INFO:tasks.workunit.client.0.smithi144.stdout:8/578: dread d0/d1b/f2f [0,4194304] 0 2024-09-10T11:29:43.323 INFO:tasks.workunit.client.0.smithi144.stdout:8/579: stat d0/d9/d21/d25/l54 0 2024-09-10T11:29:43.326 INFO:tasks.workunit.client.0.smithi144.stdout:9/625: write d11/d12/d1b/d22/f2f [858764,16679] 0 2024-09-10T11:29:43.330 INFO:tasks.workunit.client.0.smithi144.stdout:0/460: unlink da/d3a/d4b/d4e/d81/f84 0 2024-09-10T11:29:43.332 INFO:tasks.workunit.client.1.smithi178.stdout:4/910: write d0/d8/da/d1a/ff7 [832142,118541] 0 2024-09-10T11:29:43.333 INFO:tasks.workunit.client.1.smithi178.stdout:4/911: write d0/d8/da/dae/fe3 [1336094,73346] 0 2024-09-10T11:29:43.334 INFO:tasks.workunit.client.0.smithi144.stdout:8/580: symlink d0/d1b/d47/d5e/lc1 0 2024-09-10T11:29:43.336 INFO:tasks.workunit.client.0.smithi144.stdout:8/581: write d0/d1b/d47/d94/f9d [413040,93727] 0 2024-09-10T11:29:43.337 INFO:tasks.workunit.client.1.smithi178.stdout:4/912: dread d0/d8/da/f2b [0,4194304] 0 2024-09-10T11:29:43.338 INFO:tasks.workunit.client.0.smithi144.stdout:4/682: creat d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/de4/ff9 x:0 0 0 2024-09-10T11:29:43.339 INFO:tasks.workunit.client.1.smithi178.stdout:4/913: truncate d0/d8/df/d13b/d5d/d62/f11f 990958 0 2024-09-10T11:29:43.341 INFO:tasks.workunit.client.0.smithi144.stdout:7/648: rmdir d5/dc/d14/d40/d5e/d63/d98/dcb 0 2024-09-10T11:29:43.341 INFO:tasks.workunit.client.1.smithi178.stdout:4/914: fsync d0/d10/f5a 0 2024-09-10T11:29:43.342 INFO:tasks.workunit.client.1.smithi178.stdout:4/915: fdatasync d0/d8/f114 0 2024-09-10T11:29:43.344 INFO:tasks.workunit.client.0.smithi144.stdout:3/648: rename d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/d93 to d4/d28/d67/d80/d17/dd0 0 2024-09-10T11:29:43.345 INFO:tasks.workunit.client.0.smithi144.stdout:1/731: truncate d5/df/d4b/d51/f50 1647905 0 2024-09-10T11:29:43.348 INFO:tasks.workunit.client.0.smithi144.stdout:6/568: link d9/d1f/d2c/d4f/c7e d9/d1f/d2c/ccb 0 2024-09-10T11:29:43.349 INFO:tasks.workunit.client.0.smithi144.stdout:3/649: dread d4/d28/d67/d80/d17/d18/f2e [0,4194304] 0 2024-09-10T11:29:43.351 INFO:tasks.workunit.client.1.smithi178.stdout:4/916: dwrite d0/d10/ff9 [4194304,4194304] 0 2024-09-10T11:29:43.352 INFO:tasks.workunit.client.0.smithi144.stdout:3/650: truncate d4/d28/d67/d80/d17/f30 8680256 0 2024-09-10T11:29:43.352 INFO:tasks.workunit.client.0.smithi144.stdout:3/651: chown d4/d28/d67/d80/d17/d18 5938 1 2024-09-10T11:29:43.353 INFO:tasks.workunit.client.0.smithi144.stdout:4/683: mknod d14/d1e/d22/d34/d91/cfa 0 2024-09-10T11:29:43.353 INFO:tasks.workunit.client.0.smithi144.stdout:3/652: fsync d4/dd/d21/d2f/f3a 0 2024-09-10T11:29:43.355 INFO:tasks.workunit.client.0.smithi144.stdout:3/653: fsync d4/d28/d67/d80/d17/d18/d34/d78/fb2 0 2024-09-10T11:29:43.355 INFO:tasks.workunit.client.0.smithi144.stdout:3/654: stat d4/d28/d67/d96/ca7 0 2024-09-10T11:29:43.357 INFO:tasks.workunit.client.0.smithi144.stdout:0/461: mknod da/d44/d82/d7e/c8d 0 2024-09-10T11:29:43.358 INFO:tasks.workunit.client.0.smithi144.stdout:2/534: rename d7/de/d3d/d4e/c6b to d7/d29/d97/cb2 0 2024-09-10T11:29:43.360 INFO:tasks.workunit.client.0.smithi144.stdout:1/732: creat d5/df/d4b/d85/ff2 x:0 0 0 2024-09-10T11:29:43.360 INFO:tasks.workunit.client.1.smithi178.stdout:4/917: dwrite d0/d9/d3d/d4f/f97 [0,4194304] 0 2024-09-10T11:29:43.365 INFO:tasks.workunit.client.0.smithi144.stdout:3/655: read d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/f8c [195244,86451] 0 2024-09-10T11:29:43.366 INFO:tasks.workunit.client.0.smithi144.stdout:0/462: mkdir da/d3a/d8e 0 2024-09-10T11:29:43.368 INFO:tasks.workunit.client.1.smithi178.stdout:4/918: write d0/d8/df/dde/fe4 [164707,125241] 0 2024-09-10T11:29:43.368 INFO:tasks.workunit.client.0.smithi144.stdout:7/649: rename d5/dc/d12/d2f to d5/dc/d14/d40/d5e/d63/d98/dd1 0 2024-09-10T11:29:43.370 INFO:tasks.workunit.client.0.smithi144.stdout:3/656: write d4/dd/d21/f86 [1173825,30402] 0 2024-09-10T11:29:43.370 INFO:tasks.workunit.client.0.smithi144.stdout:1/733: dwrite d5/d4a/d63/fac [0,4194304] 0 2024-09-10T11:29:43.370 INFO:tasks.workunit.client.0.smithi144.stdout:2/535: creat d7/d29/d92/fb3 x:0 0 0 2024-09-10T11:29:43.372 INFO:tasks.workunit.client.0.smithi144.stdout:2/536: readlink d7/de/d3d/d43/l62 0 2024-09-10T11:29:43.372 INFO:tasks.workunit.client.1.smithi178.stdout:4/919: dread d0/d8/da/f60 [0,4194304] 0 2024-09-10T11:29:43.376 INFO:tasks.workunit.client.1.smithi178.stdout:4/920: mknod d0/d9/dc/d8e/dbf/c143 0 2024-09-10T11:29:43.376 INFO:tasks.workunit.client.0.smithi144.stdout:0/463: mkdir da/d3a/d51/d8f 0 2024-09-10T11:29:43.377 INFO:tasks.workunit.client.1.smithi178.stdout:4/921: stat d0/d8/df/d13b/d5d/dc9 0 2024-09-10T11:29:43.377 INFO:tasks.workunit.client.0.smithi144.stdout:0/464: chown da/d3a/d51/d8f 415774952 1 2024-09-10T11:29:43.380 INFO:tasks.workunit.client.0.smithi144.stdout:3/657: dread - d4/f9b zero size 2024-09-10T11:29:43.380 INFO:tasks.workunit.client.0.smithi144.stdout:2/537: dwrite d7/da/f19 [0,4194304] 0 2024-09-10T11:29:43.381 INFO:tasks.workunit.client.0.smithi144.stdout:3/658: chown d4/d28/f3f 15049 1 2024-09-10T11:29:43.384 INFO:tasks.workunit.client.0.smithi144.stdout:4/684: creat d14/d67/ffb x:0 0 0 2024-09-10T11:29:43.385 INFO:tasks.workunit.client.0.smithi144.stdout:4/685: readlink d14/d1e/d65/l99 0 2024-09-10T11:29:43.388 INFO:tasks.workunit.client.0.smithi144.stdout:8/582: write d0/d9/d1f/d34/d68/fbc [777932,124601] 0 2024-09-10T11:29:43.389 INFO:tasks.workunit.client.0.smithi144.stdout:0/465: write f5 [4641610,99685] 0 2024-09-10T11:29:43.394 INFO:tasks.workunit.client.1.smithi178.stdout:4/922: dread d0/d8/da/f96 [0,4194304] 0 2024-09-10T11:29:43.396 INFO:tasks.workunit.client.0.smithi144.stdout:3/659: dread d4/d28/d67/d80/d1d/f29 [0,4194304] 0 2024-09-10T11:29:43.397 INFO:tasks.workunit.client.0.smithi144.stdout:4/686: rename d14/d1e/d22/d34/d91/fec to d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/ffc 0 2024-09-10T11:29:43.398 INFO:tasks.workunit.client.0.smithi144.stdout:0/466: readlink da/d3a/l87 0 2024-09-10T11:29:43.399 INFO:tasks.workunit.client.0.smithi144.stdout:4/687: fsync d14/d1e/d22/d34/d3e/d40/d4b/d4c/fba 0 2024-09-10T11:29:43.399 INFO:tasks.workunit.client.0.smithi144.stdout:6/569: write d9/dd/d3e/d47/f57 [3397180,10028] 0 2024-09-10T11:29:43.400 INFO:tasks.workunit.client.0.smithi144.stdout:8/583: mkdir d0/d9/d21/d25/db3/dc2 0 2024-09-10T11:29:43.400 INFO:tasks.workunit.client.0.smithi144.stdout:4/688: write f5 [2427121,113414] 0 2024-09-10T11:29:43.401 INFO:tasks.workunit.client.0.smithi144.stdout:0/467: read da/f19 [3648025,112690] 0 2024-09-10T11:29:43.401 INFO:tasks.workunit.client.0.smithi144.stdout:8/584: chown d0/d9/d21/l6d 1874 1 2024-09-10T11:29:43.402 INFO:tasks.workunit.client.0.smithi144.stdout:4/689: stat d14/d1e/d22/d34/f3b 0 2024-09-10T11:29:43.404 INFO:tasks.workunit.client.1.smithi178.stdout:4/923: dwrite d0/d9/dc/d8e/f9f [0,4194304] 0 2024-09-10T11:29:43.406 INFO:tasks.workunit.client.0.smithi144.stdout:2/538: mknod d7/da/da8/cb4 0 2024-09-10T11:29:43.408 INFO:tasks.workunit.client.0.smithi144.stdout:3/660: mknod d4/d28/d67/d80/d17/dd0/cd1 0 2024-09-10T11:29:43.408 INFO:tasks.workunit.client.0.smithi144.stdout:0/468: dwrite da/d3a/f4d [4194304,4194304] 0 2024-09-10T11:29:43.410 INFO:tasks.workunit.client.0.smithi144.stdout:3/661: chown d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e 1572 1 2024-09-10T11:29:43.411 INFO:tasks.workunit.client.1.smithi178.stdout:4/924: rename d0/d8/df/d13b/d5d/d62/f11f to d0/d8/f144 0 2024-09-10T11:29:43.412 INFO:tasks.workunit.client.0.smithi144.stdout:6/570: mknod d9/dd/d11/d1b/ccc 0 2024-09-10T11:29:43.415 INFO:tasks.workunit.client.0.smithi144.stdout:8/585: mkdir d0/d9/d21/d25/d4f/dc3 0 2024-09-10T11:29:43.418 INFO:tasks.workunit.client.0.smithi144.stdout:1/734: getdents d5/d4a/d63/d84/d95 0 2024-09-10T11:29:43.419 INFO:tasks.workunit.client.0.smithi144.stdout:1/735: stat d5/d4a/d63/d84/d95/dd2/fde 0 2024-09-10T11:29:43.422 INFO:tasks.workunit.client.0.smithi144.stdout:0/469: rename da/d3a/d51/d5f/d6b to da/d3a/d4b/d4e/d90 0 2024-09-10T11:29:43.423 INFO:tasks.workunit.client.0.smithi144.stdout:7/650: getdents d5/dc/d14/d40/d58/d6a 0 2024-09-10T11:29:43.424 INFO:tasks.workunit.client.1.smithi178.stdout:7/876: sync 2024-09-10T11:29:43.425 INFO:tasks.workunit.client.1.smithi178.stdout:7/877: readlink d6/d10/l17 0 2024-09-10T11:29:43.425 INFO:tasks.workunit.client.0.smithi144.stdout:7/651: write d5/dc/d12/f22 [1046125,56274] 0 2024-09-10T11:29:43.425 INFO:tasks.workunit.client.0.smithi144.stdout:3/662: write d4/dd/d21/f5b [3006162,99011] 0 2024-09-10T11:29:43.425 INFO:tasks.workunit.client.1.smithi178.stdout:7/878: chown d6/d10/d1e/d2d/d2b/da0/f7a 376 1 2024-09-10T11:29:43.427 INFO:tasks.workunit.client.0.smithi144.stdout:8/586: dread d0/d9/d23/d37/f3a [0,4194304] 0 2024-09-10T11:29:43.427 INFO:tasks.workunit.client.0.smithi144.stdout:4/690: symlink d14/d1e/d83/da4/da9/lfd 0 2024-09-10T11:29:43.427 INFO:tasks.workunit.client.0.smithi144.stdout:7/652: write d5/dc/d14/f44 [3129939,129440] 0 2024-09-10T11:29:43.428 INFO:tasks.workunit.client.0.smithi144.stdout:3/663: stat d4/d28/d67/d80/d17/l52 0 2024-09-10T11:29:43.429 INFO:tasks.workunit.client.0.smithi144.stdout:1/736: truncate d5/df/d4b/fb4 2348188 0 2024-09-10T11:29:43.430 INFO:tasks.workunit.client.1.smithi178.stdout:7/879: rename d6/d10/d1d/d22/d50/d7e/da7/f112 to d6/d10/d1d/d22/d50/f136 0 2024-09-10T11:29:43.434 INFO:tasks.workunit.client.0.smithi144.stdout:3/664: write d4/dd/d21/d2f/f60 [1686620,75922] 0 2024-09-10T11:29:43.435 INFO:tasks.workunit.client.0.smithi144.stdout:1/737: truncate d5/d2a/f46 4742506 0 2024-09-10T11:29:43.436 INFO:tasks.workunit.client.0.smithi144.stdout:7/653: creat d5/dc/d14/d40/d5e/d68/fd2 x:0 0 0 2024-09-10T11:29:43.438 INFO:tasks.workunit.client.1.smithi178.stdout:7/880: write d6/d10/d1d/d4a/f7d [177099,126883] 0 2024-09-10T11:29:43.441 INFO:tasks.workunit.client.0.smithi144.stdout:0/470: dread da/f30 [0,4194304] 0 2024-09-10T11:29:43.441 INFO:tasks.workunit.client.0.smithi144.stdout:7/654: read d5/f7 [1098188,68400] 0 2024-09-10T11:29:43.443 INFO:tasks.workunit.client.0.smithi144.stdout:6/571: write d9/dd/d3e/d47/d13/f31 [3425523,56059] 0 2024-09-10T11:29:43.447 INFO:tasks.workunit.client.0.smithi144.stdout:2/539: dwrite d7/f1d [0,4194304] 0 2024-09-10T11:29:43.448 INFO:tasks.workunit.client.0.smithi144.stdout:4/691: link d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/fb7 d14/d1e/d83/ffe 0 2024-09-10T11:29:43.450 INFO:tasks.workunit.client.0.smithi144.stdout:1/738: write d5/d2a/f7e [508739,105008] 0 2024-09-10T11:29:43.450 INFO:tasks.workunit.client.1.smithi178.stdout:7/881: dread d6/d10/d1e/d2d/f5f [4194304,4194304] 0 2024-09-10T11:29:43.451 INFO:tasks.workunit.client.0.smithi144.stdout:4/692: write d14/fe3 [241137,100389] 0 2024-09-10T11:29:43.458 INFO:tasks.workunit.client.0.smithi144.stdout:0/471: dread da/d3a/d51/d5f/f4a [0,4194304] 0 2024-09-10T11:29:43.459 INFO:tasks.workunit.client.0.smithi144.stdout:8/587: dread d0/f3f [0,4194304] 0 2024-09-10T11:29:43.459 INFO:tasks.workunit.client.0.smithi144.stdout:3/665: dwrite d4/dd/d21/f5b [4194304,4194304] 0 2024-09-10T11:29:43.461 INFO:tasks.workunit.client.0.smithi144.stdout:7/655: dread d5/dc/d14/d40/d58/d6a/fa0 [0,4194304] 0 2024-09-10T11:29:43.463 INFO:tasks.workunit.client.1.smithi178.stdout:7/882: dread d6/d10/d1e/d2d/d2b/f39 [0,4194304] 0 2024-09-10T11:29:43.464 INFO:tasks.workunit.client.0.smithi144.stdout:6/572: creat d9/dd/d3e/d47/fcd x:0 0 0 2024-09-10T11:29:43.464 INFO:tasks.workunit.client.0.smithi144.stdout:1/739: read d5/fc [888136,109863] 0 2024-09-10T11:29:43.464 INFO:tasks.workunit.client.1.smithi178.stdout:7/883: write d6/d10/d1d/d22/d3b/f6d [1201432,5285] 0 2024-09-10T11:29:43.465 INFO:tasks.workunit.client.0.smithi144.stdout:0/472: rmdir da 39 2024-09-10T11:29:43.465 INFO:tasks.workunit.client.0.smithi144.stdout:5/684: sync 2024-09-10T11:29:43.466 INFO:tasks.workunit.client.1.smithi178.stdout:7/884: readlink d6/d10/d1d/d22/d50/d7e/da7/lf2 0 2024-09-10T11:29:43.469 INFO:tasks.workunit.client.1.smithi178.stdout:7/885: rmdir d6/d10/d1d/d22/d50/d7e/d82/db2 39 2024-09-10T11:29:43.470 INFO:tasks.workunit.client.0.smithi144.stdout:1/740: stat d5/d4a/d63/d84/fe8 0 2024-09-10T11:29:43.470 INFO:tasks.workunit.client.0.smithi144.stdout:2/540: dwrite d7/de/d3d/f3e [0,4194304] 0 2024-09-10T11:29:43.470 INFO:tasks.workunit.client.0.smithi144.stdout:8/588: write d0/fc [8644875,21512] 0 2024-09-10T11:29:43.470 INFO:tasks.workunit.client.1.smithi178.stdout:7/886: dread d6/d10/d1d/d4a/d79/df1/fff [0,4194304] 0 2024-09-10T11:29:43.471 INFO:tasks.workunit.client.0.smithi144.stdout:1/741: readlink d5/d2a/d65/dc9/lc3 0 2024-09-10T11:29:43.471 INFO:tasks.workunit.client.1.smithi178.stdout:7/887: write d6/d10/d1d/d22/d93/db7/f114 [10779,28232] 0 2024-09-10T11:29:43.474 INFO:tasks.workunit.client.1.smithi178.stdout:7/888: unlink f4 0 2024-09-10T11:29:43.475 INFO:tasks.workunit.client.1.smithi178.stdout:7/889: dread - d6/d10/d1d/d22/d3b/f12d zero size 2024-09-10T11:29:43.477 INFO:tasks.workunit.client.0.smithi144.stdout:8/589: write d0/d9/d23/d26/f69 [1036875,12853] 0 2024-09-10T11:29:43.477 INFO:tasks.workunit.client.0.smithi144.stdout:3/666: creat d4/d28/d67/d96/fd2 x:0 0 0 2024-09-10T11:29:43.478 INFO:tasks.workunit.client.0.smithi144.stdout:4/693: getdents d14/d1e/d65/d8a 0 2024-09-10T11:29:43.479 INFO:tasks.workunit.client.1.smithi178.stdout:7/890: mkdir d6/d10/d1d/daa/d137 0 2024-09-10T11:29:43.480 INFO:tasks.workunit.client.0.smithi144.stdout:5/685: rename d4/d20/d24/d6f to d4/d52/dd2 0 2024-09-10T11:29:43.481 INFO:tasks.workunit.client.1.smithi178.stdout:7/891: creat d6/d10/d1d/d22/d50/f138 x:0 0 0 2024-09-10T11:29:43.482 INFO:tasks.workunit.client.1.smithi178.stdout:7/892: chown d6/d10/d1d/d22/d93/db7/l113 8079 1 2024-09-10T11:29:43.483 INFO:tasks.workunit.client.0.smithi144.stdout:4/694: fsync d14/d1e/d22/f31 0 2024-09-10T11:29:43.485 INFO:tasks.workunit.client.0.smithi144.stdout:7/656: mknod d5/dc/d14/d40/d5e/d63/d98/dd1/d7c/cd3 0 2024-09-10T11:29:43.487 INFO:tasks.workunit.client.0.smithi144.stdout:5/686: read d4/d20/d24/f92 [5973342,38566] 0 2024-09-10T11:29:43.487 INFO:tasks.workunit.client.0.smithi144.stdout:8/590: mkdir d0/d9/d1f/d34/da0/dc4 0 2024-09-10T11:29:43.489 INFO:tasks.workunit.client.0.smithi144.stdout:1/742: truncate d5/df/d4b/d51/f50 1453977 0 2024-09-10T11:29:43.490 INFO:tasks.workunit.client.1.smithi178.stdout:7/893: write d6/d10/d1e/d2d/d2b/da0/f7a [1897015,12543] 0 2024-09-10T11:29:43.492 INFO:tasks.workunit.client.0.smithi144.stdout:4/695: write d14/d1e/d22/d34/d91/ff7 [701286,15620] 0 2024-09-10T11:29:43.492 INFO:tasks.workunit.client.0.smithi144.stdout:8/591: write d0/d1b/d5a/d8d/f98 [3085712,1624] 0 2024-09-10T11:29:43.493 INFO:tasks.workunit.client.0.smithi144.stdout:2/541: dread d7/de/d36/d45/d8e/d9a/f53 [0,4194304] 0 2024-09-10T11:29:43.493 INFO:tasks.workunit.client.0.smithi144.stdout:0/473: dwrite da/d3a/f3e [4194304,4194304] 0 2024-09-10T11:29:43.495 INFO:tasks.workunit.client.0.smithi144.stdout:1/743: mkdir d5/d2a/d65/d73/df3 0 2024-09-10T11:29:43.495 INFO:tasks.workunit.client.0.smithi144.stdout:3/667: write d4/d28/d67/d80/d17/d18/d34/d4b/f53 [6272952,59621] 0 2024-09-10T11:29:43.497 INFO:tasks.workunit.client.1.smithi178.stdout:7/894: dwrite d6/d10/d1e/d2d/f5f [0,4194304] 0 2024-09-10T11:29:43.500 INFO:tasks.workunit.client.0.smithi144.stdout:8/592: fdatasync d0/d9/d21/d25/d4f/f65 0 2024-09-10T11:29:43.502 INFO:tasks.workunit.client.1.smithi178.stdout:7/895: mknod d6/d10/d1d/d4a/d79/df1/c139 0 2024-09-10T11:29:43.503 INFO:tasks.workunit.client.0.smithi144.stdout:5/687: mknod d4/d52/dd2/d74/dbe/cd3 0 2024-09-10T11:29:43.504 INFO:tasks.workunit.client.1.smithi178.stdout:7/896: truncate d6/d10/d1d/f27 2050046 0 2024-09-10T11:29:43.504 INFO:tasks.workunit.client.0.smithi144.stdout:0/474: mknod da/d3a/d4b/c91 0 2024-09-10T11:29:43.506 INFO:tasks.workunit.client.0.smithi144.stdout:2/542: creat d7/de/d36/d45/d8e/d9a/d54/fb5 x:0 0 0 2024-09-10T11:29:43.508 INFO:tasks.workunit.client.1.smithi178.stdout:7/897: creat d6/d10/d1e/d2d/d2b/da0/d101/f13a x:0 0 0 2024-09-10T11:29:43.508 INFO:tasks.workunit.client.0.smithi144.stdout:2/543: fsync d7/de/d36/d45/d8e/d9a/f9c 0 2024-09-10T11:29:43.508 INFO:tasks.workunit.client.0.smithi144.stdout:1/744: symlink d5/d2a/lf4 0 2024-09-10T11:29:43.511 INFO:tasks.workunit.client.0.smithi144.stdout:4/696: unlink d14/d67/d69/d90/dda/ff5 0 2024-09-10T11:29:43.512 INFO:tasks.workunit.client.1.smithi178.stdout:7/898: truncate d6/d10/d1d/d22/d50/d7e/d82/db2/d49/f110 160136 0 2024-09-10T11:29:43.514 INFO:tasks.workunit.client.0.smithi144.stdout:6/573: link d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/fb5 d9/dd/d11/d1b/d2f/dab/fce 0 2024-09-10T11:29:43.514 INFO:tasks.workunit.client.0.smithi144.stdout:3/668: mknod d4/d28/d67/d80/cd3 0 2024-09-10T11:29:43.515 INFO:tasks.workunit.client.1.smithi178.stdout:7/899: creat d6/d10/d1d/daa/d137/f13b x:0 0 0 2024-09-10T11:29:43.516 INFO:tasks.workunit.client.0.smithi144.stdout:8/593: rename d0/d9/d1f/d34/da0/lb4 to d0/d1b/lc5 0 2024-09-10T11:29:43.516 INFO:tasks.workunit.client.1.smithi178.stdout:7/900: readlink d6/d10/d1d/d4a/d79/df1/l87 0 2024-09-10T11:29:43.517 INFO:tasks.workunit.client.1.smithi178.stdout:7/901: chown d6/d10/d1d/d22/d93 168 1 2024-09-10T11:29:43.517 INFO:tasks.workunit.client.0.smithi144.stdout:7/657: dwrite d5/dc/d11/d18/f34 [0,4194304] 0 2024-09-10T11:29:43.519 INFO:tasks.workunit.client.1.smithi178.stdout:7/902: mknod d6/d10/d1d/d22/d93/d12f/c13c 0 2024-09-10T11:29:43.520 INFO:tasks.workunit.client.0.smithi144.stdout:7/658: chown d5/dc/d11 515 1 2024-09-10T11:29:43.520 INFO:tasks.workunit.client.0.smithi144.stdout:2/544: dwrite d7/d42/d64/faa [4194304,4194304] 0 2024-09-10T11:29:43.521 INFO:tasks.workunit.client.0.smithi144.stdout:0/475: readlink da/d3a/d4b/d4e/l5e 0 2024-09-10T11:29:43.522 INFO:tasks.workunit.client.0.smithi144.stdout:6/574: rmdir d9/dd/d11 39 2024-09-10T11:29:43.523 INFO:tasks.workunit.client.0.smithi144.stdout:4/697: rmdir d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f 39 2024-09-10T11:29:43.523 INFO:tasks.workunit.client.1.smithi178.stdout:7/903: rename d6/d10/d1d/d22/d93/db7/l113 to d6/d10/d1d/daa/l13d 0 2024-09-10T11:29:43.527 INFO:tasks.workunit.client.0.smithi144.stdout:1/745: mknod d5/cf5 0 2024-09-10T11:29:43.527 INFO:tasks.workunit.client.0.smithi144.stdout:2/545: rmdir d7/de/d36/d45/d47 39 2024-09-10T11:29:43.529 INFO:tasks.workunit.client.0.smithi144.stdout:0/476: symlink da/d3a/d51/d5f/l92 0 2024-09-10T11:29:43.530 INFO:tasks.workunit.client.0.smithi144.stdout:4/698: rmdir d14/d1e/d22/d34/d3e/d40/d4b/d4c 39 2024-09-10T11:29:43.530 INFO:tasks.workunit.client.0.smithi144.stdout:3/669: symlink d4/d28/d67/d96/dcc/ld4 0 2024-09-10T11:29:43.530 INFO:tasks.workunit.client.0.smithi144.stdout:1/746: write d5/d4a/d44/f45 [2756836,77199] 0 2024-09-10T11:29:43.531 INFO:tasks.workunit.client.0.smithi144.stdout:0/477: chown da/d3a/f4d 345 1 2024-09-10T11:29:43.532 INFO:tasks.workunit.client.1.smithi178.stdout:7/904: dwrite d6/d10/d1e/d2d/d2b/da0/dc0/fe8 [0,4194304] 0 2024-09-10T11:29:43.533 INFO:tasks.workunit.client.0.smithi144.stdout:8/594: creat d0/d9/d1f/fc6 x:0 0 0 2024-09-10T11:29:43.535 INFO:tasks.workunit.client.0.smithi144.stdout:1/747: write d5/df/d4b/d85/f8b [2395674,91529] 0 2024-09-10T11:29:43.536 INFO:tasks.workunit.client.0.smithi144.stdout:9/626: sync 2024-09-10T11:29:43.537 INFO:tasks.workunit.client.0.smithi144.stdout:7/659: creat d5/dc/d14/d40/d58/d6e/fd4 x:0 0 0 2024-09-10T11:29:43.544 INFO:tasks.workunit.client.0.smithi144.stdout:8/595: mknod d0/d9/d21/d25/d4f/cc7 0 2024-09-10T11:29:43.544 INFO:tasks.workunit.client.0.smithi144.stdout:6/575: creat d9/dd/d3e/d47/d13/d87/dc9/fcf x:0 0 0 2024-09-10T11:29:43.548 INFO:tasks.workunit.client.0.smithi144.stdout:6/576: chown d9/dd/laa 2661273 1 2024-09-10T11:29:43.549 INFO:tasks.workunit.client.0.smithi144.stdout:2/546: creat d7/da/fb6 x:0 0 0 2024-09-10T11:29:43.550 INFO:tasks.workunit.client.0.smithi144.stdout:5/688: truncate d4/d5/d18/d38/d8d/fa8 542668 0 2024-09-10T11:29:43.554 INFO:tasks.workunit.client.0.smithi144.stdout:1/748: dwrite d5/df/d4b/d85/ff2 [0,4194304] 0 2024-09-10T11:29:43.555 INFO:tasks.workunit.client.0.smithi144.stdout:7/660: dwrite d5/dc/d2a/d41/f49 [4194304,4194304] 0 2024-09-10T11:29:43.558 INFO:tasks.workunit.client.0.smithi144.stdout:0/478: truncate da/f28 343447 0 2024-09-10T11:29:43.558 INFO:tasks.workunit.client.0.smithi144.stdout:3/670: link d4/dd/c73 d4/cd5 0 2024-09-10T11:29:43.558 INFO:tasks.workunit.client.0.smithi144.stdout:7/661: write d5/f5a [4795921,3451] 0 2024-09-10T11:29:43.559 INFO:tasks.workunit.client.0.smithi144.stdout:4/699: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/fb8 [0,4194304] 0 2024-09-10T11:29:43.560 INFO:tasks.workunit.client.0.smithi144.stdout:1/749: read d5/df/f55 [2503371,70060] 0 2024-09-10T11:29:43.560 INFO:tasks.workunit.client.0.smithi144.stdout:6/577: mkdir d9/dd/d11/d1b/d2f/dab/dd0 0 2024-09-10T11:29:43.564 INFO:tasks.workunit.client.0.smithi144.stdout:6/578: chown d9/dd/d3e/d47/d13/d60/lc5 23 1 2024-09-10T11:29:43.567 INFO:tasks.workunit.client.0.smithi144.stdout:8/596: getdents d0/d9/d1f/d34 0 2024-09-10T11:29:43.567 INFO:tasks.workunit.client.0.smithi144.stdout:0/479: rmdir da/d3a 39 2024-09-10T11:29:43.569 INFO:tasks.workunit.client.0.smithi144.stdout:3/671: fdatasync d4/dd/d21/f86 0 2024-09-10T11:29:43.569 INFO:tasks.workunit.client.0.smithi144.stdout:7/662: fdatasync d5/fa 0 2024-09-10T11:29:43.572 INFO:tasks.workunit.client.0.smithi144.stdout:5/689: creat d4/d20/d22/d5c/d5d/d64/fd4 x:0 0 0 2024-09-10T11:29:43.572 INFO:tasks.workunit.client.0.smithi144.stdout:7/663: fdatasync d5/dc/d14/d40/d5e/d68/fc0 0 2024-09-10T11:29:43.577 INFO:tasks.workunit.client.0.smithi144.stdout:8/597: creat d0/d9/d21/d25/d4f/fc8 x:0 0 0 2024-09-10T11:29:43.577 INFO:tasks.workunit.client.0.smithi144.stdout:0/480: creat da/d3a/f93 x:0 0 0 2024-09-10T11:29:43.579 INFO:tasks.workunit.client.0.smithi144.stdout:1/750: link d5/df/d4b/d85/ff2 d5/d4a/d44/ff6 0 2024-09-10T11:29:43.582 INFO:tasks.workunit.client.0.smithi144.stdout:8/598: truncate d0/d1b/d47/d5e/d76/f7c 972378 0 2024-09-10T11:29:43.585 INFO:tasks.workunit.client.0.smithi144.stdout:5/690: getdents d4 0 2024-09-10T11:29:43.586 INFO:tasks.workunit.client.0.smithi144.stdout:7/664: mkdir d5/dc/d14/d40/d58/dd5 0 2024-09-10T11:29:43.588 INFO:tasks.workunit.client.0.smithi144.stdout:8/599: symlink d0/d9/d23/lc9 0 2024-09-10T11:29:43.590 INFO:tasks.workunit.client.0.smithi144.stdout:3/672: dread d4/d28/f76 [0,4194304] 0 2024-09-10T11:29:43.591 INFO:tasks.workunit.client.0.smithi144.stdout:6/579: write d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/f6f [173038,81952] 0 2024-09-10T11:29:43.592 INFO:tasks.workunit.client.0.smithi144.stdout:4/700: dread d14/d1e/d22/f23 [0,4194304] 0 2024-09-10T11:29:43.595 INFO:tasks.workunit.client.0.smithi144.stdout:5/691: creat d4/d5/d18/d51/fd5 x:0 0 0 2024-09-10T11:29:43.596 INFO:tasks.workunit.client.0.smithi144.stdout:7/665: dread d5/dc/d2a/f4b [0,4194304] 0 2024-09-10T11:29:43.597 INFO:tasks.workunit.client.0.smithi144.stdout:0/481: dwrite da/d3a/d4b/f86 [0,4194304] 0 2024-09-10T11:29:43.599 INFO:tasks.workunit.client.0.smithi144.stdout:5/692: chown d4/d20/d22/d5c/d5d 220 1 2024-09-10T11:29:43.599 INFO:tasks.workunit.client.0.smithi144.stdout:6/580: read - d9/d1f/d2c/f67 zero size 2024-09-10T11:29:43.600 INFO:tasks.workunit.client.0.smithi144.stdout:3/673: dread - d4/d28/d67/d96/fd2 zero size 2024-09-10T11:29:43.600 INFO:tasks.workunit.client.0.smithi144.stdout:1/751: dwrite d5/df/d4b/d85/f8b [0,4194304] 0 2024-09-10T11:29:43.601 INFO:tasks.workunit.client.0.smithi144.stdout:8/600: truncate d0/d9/d23/d26/f4a 2714946 0 2024-09-10T11:29:43.601 INFO:tasks.workunit.client.0.smithi144.stdout:5/693: chown d4/d5/d18/d38/d8d 19838 1 2024-09-10T11:29:43.604 INFO:tasks.workunit.client.0.smithi144.stdout:4/701: unlink d14/d1e/d22/d34/d3e/d40/d4b/f6a 0 2024-09-10T11:29:43.609 INFO:tasks.workunit.client.0.smithi144.stdout:1/752: fsync d5/df/d4b/fb4 0 2024-09-10T11:29:43.610 INFO:tasks.workunit.client.0.smithi144.stdout:4/702: write d14/dcb/fce [1186000,103247] 0 2024-09-10T11:29:43.610 INFO:tasks.workunit.client.0.smithi144.stdout:6/581: mknod d9/dd/d11/d1b/d2f/d7f/dc4/d5b/dc8/cd1 0 2024-09-10T11:29:43.613 INFO:tasks.workunit.client.0.smithi144.stdout:6/582: readlink d9/l5f 0 2024-09-10T11:29:43.615 INFO:tasks.workunit.client.0.smithi144.stdout:3/674: symlink d4/d28/d67/d80/d17/ld6 0 2024-09-10T11:29:43.617 INFO:tasks.workunit.client.0.smithi144.stdout:8/601: dwrite d0/d1b/d47/d5e/d76/f7c [0,4194304] 0 2024-09-10T11:29:43.623 INFO:tasks.workunit.client.0.smithi144.stdout:5/694: getdents d4/d52/dd2/d74 0 2024-09-10T11:29:43.625 INFO:tasks.workunit.client.1.smithi178.stdout:0/991: sync 2024-09-10T11:29:43.625 INFO:tasks.workunit.client.0.smithi144.stdout:0/482: dwrite da/d3a/d4b/d4e/d62/d69/f79 [0,4194304] 0 2024-09-10T11:29:43.628 INFO:tasks.workunit.client.0.smithi144.stdout:5/695: chown d4/d5/d18/d67/l7a 0 1 2024-09-10T11:29:43.632 INFO:tasks.workunit.client.1.smithi178.stdout:0/992: dwrite d0/d2c/d89/f135 [0,4194304] 0 2024-09-10T11:29:43.634 INFO:tasks.workunit.client.0.smithi144.stdout:7/666: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:43.634 INFO:tasks.workunit.client.0.smithi144.stdout:6/583: rename d9/dd/d3e/d47/d13/d87/dc9/fcf to d9/dd/d11/d1b/fd2 0 2024-09-10T11:29:43.634 INFO:tasks.workunit.client.0.smithi144.stdout:0/483: mkdir da/d3a/d94 0 2024-09-10T11:29:43.635 INFO:tasks.workunit.client.0.smithi144.stdout:3/675: symlink d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/ld7 0 2024-09-10T11:29:43.635 INFO:tasks.workunit.client.0.smithi144.stdout:0/484: fsync f0 0 2024-09-10T11:29:43.639 INFO:tasks.workunit.client.1.smithi178.stdout:0/993: symlink d0/dc/d39/d75/d94/l13a 0 2024-09-10T11:29:43.640 INFO:tasks.workunit.client.1.smithi178.stdout:0/994: chown d0/dc/d39 157473 1 2024-09-10T11:29:43.640 INFO:tasks.workunit.client.0.smithi144.stdout:8/602: rename d0/d9/d23/l3c to d0/d9/d1f/d34/da0/dc4/lca 0 2024-09-10T11:29:43.641 INFO:tasks.workunit.client.0.smithi144.stdout:1/753: dwrite d5/d4a/d63/f9f [0,4194304] 0 2024-09-10T11:29:43.641 INFO:tasks.workunit.client.0.smithi144.stdout:5/696: fdatasync d4/d5/d18/d67/f62 0 2024-09-10T11:29:43.647 INFO:tasks.workunit.client.1.smithi178.stdout:0/995: link d0/dc/d39/d50/l76 d0/dc/d1f/dea/l13b 0 2024-09-10T11:29:43.652 INFO:tasks.workunit.client.1.smithi178.stdout:0/996: link d0/d2c/d95/dbd/l131 d0/dc/d39/d75/d120/l13c 0 2024-09-10T11:29:43.653 INFO:tasks.workunit.client.1.smithi178.stdout:4/925: sync 2024-09-10T11:29:43.653 INFO:tasks.workunit.client.1.smithi178.stdout:7/905: sync 2024-09-10T11:29:43.655 INFO:tasks.workunit.client.1.smithi178.stdout:7/906: dread - d6/d10/d1d/d22/d93/db7/fec zero size 2024-09-10T11:29:43.656 INFO:tasks.workunit.client.1.smithi178.stdout:0/997: getdents d0/d2c/d95 0 2024-09-10T11:29:43.657 INFO:tasks.workunit.client.1.smithi178.stdout:4/926: creat d0/d9/dc/d8e/df1/d134/f145 x:0 0 0 2024-09-10T11:29:43.659 INFO:tasks.workunit.client.1.smithi178.stdout:0/998: creat d0/d2c/d89/da1/f13d x:0 0 0 2024-09-10T11:29:43.663 INFO:tasks.workunit.client.1.smithi178.stdout:7/907: truncate d6/d10/d1e/d2d/d2b/da0/d55/dbd/f77 4419974 0 2024-09-10T11:29:43.664 INFO:tasks.workunit.client.1.smithi178.stdout:4/927: dwrite d0/d8/df/d13b/f7a [0,4194304] 0 2024-09-10T11:29:43.665 INFO:tasks.workunit.client.1.smithi178.stdout:4/928: dread - d0/d9/ff6 zero size 2024-09-10T11:29:43.666 INFO:tasks.workunit.client.1.smithi178.stdout:7/908: fdatasync d6/d10/d1e/d2d/d2b/da0/f7a 0 2024-09-10T11:29:43.668 INFO:tasks.workunit.client.1.smithi178.stdout:0/999: dwrite d0/dc/d39/d50/d5e/fcd [0,4194304] 0 2024-09-10T11:29:43.670 INFO:tasks.workunit.client.1.smithi178.stdout:7/909: creat d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d70/f13e x:0 0 0 2024-09-10T11:29:43.675 INFO:tasks.workunit.client.1.smithi178.stdout:4/929: dwrite d0/d8/f144 [0,4194304] 0 2024-09-10T11:29:43.676 INFO:tasks.workunit.client.1.smithi178.stdout:7/910: read d6/d10/d1d/d4a/f57 [3361236,78201] 0 2024-09-10T11:29:43.682 INFO:tasks.workunit.client.1.smithi178.stdout:4/930: mknod d0/d9/d3d/d4f/ddb/c146 0 2024-09-10T11:29:43.714 INFO:tasks.workunit.client.1.smithi178.stdout:7/911: mkdir d6/d10/d1d/d22/d50/d98/d13f 0 2024-09-10T11:29:43.717 INFO:tasks.workunit.client.1.smithi178.stdout:7/912: symlink d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d70/l140 0 2024-09-10T11:29:43.719 INFO:tasks.workunit.client.1.smithi178.stdout:7/913: creat d6/d10/d1d/d22/d50/d98/da6/dd5/f141 x:0 0 0 2024-09-10T11:29:43.720 INFO:tasks.workunit.client.0.smithi144.stdout:7/667: rename d5/dc/d14/d40/d5e/d63 to d5/dc/d2a/d60/dd6 0 2024-09-10T11:29:43.721 INFO:tasks.workunit.client.0.smithi144.stdout:8/603: symlink d0/d9/d21/d25/d4f/d93/lcb 0 2024-09-10T11:29:43.721 INFO:tasks.workunit.client.0.smithi144.stdout:0/485: mknod da/d3a/d4b/d4e/d62/c95 0 2024-09-10T11:29:43.723 INFO:tasks.workunit.client.0.smithi144.stdout:5/697: symlink d4/ld6 0 2024-09-10T11:29:43.724 INFO:tasks.workunit.client.0.smithi144.stdout:8/604: creat d0/d1b/d47/d94/fcc x:0 0 0 2024-09-10T11:29:43.724 INFO:tasks.workunit.client.1.smithi178.stdout:7/914: truncate d6/d10/d34/f74 3404110 0 2024-09-10T11:29:43.726 INFO:tasks.workunit.client.1.smithi178.stdout:7/915: symlink d6/d10/d1d/daa/d137/l142 0 2024-09-10T11:29:43.727 INFO:tasks.workunit.client.0.smithi144.stdout:0/486: mkdir da/d3a/d4b/d96 0 2024-09-10T11:29:43.727 INFO:tasks.workunit.client.0.smithi144.stdout:7/668: symlink d5/dc/d14/d40/d58/d6e/ld7 0 2024-09-10T11:29:43.728 INFO:tasks.workunit.client.0.smithi144.stdout:0/487: readlink da/d3a/d4b/d4e/d90/l74 0 2024-09-10T11:29:43.729 INFO:tasks.workunit.client.1.smithi178.stdout:7/916: mknod d6/d10/d10b/c143 0 2024-09-10T11:29:43.730 INFO:tasks.workunit.client.0.smithi144.stdout:8/605: rename d0/d84 to d0/d9/d21/d25/db3/dc2/dcd 0 2024-09-10T11:29:43.730 INFO:tasks.workunit.client.0.smithi144.stdout:1/754: getdents d5/d4a/d63 0 2024-09-10T11:29:43.730 INFO:tasks.workunit.client.0.smithi144.stdout:4/703: dread d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f/f60 [0,4194304] 0 2024-09-10T11:29:43.732 INFO:tasks.workunit.client.0.smithi144.stdout:0/488: stat da/l18 0 2024-09-10T11:29:43.733 INFO:tasks.workunit.client.0.smithi144.stdout:7/669: mkdir d5/dc/d14/d40/d5e/d68/dd8 0 2024-09-10T11:29:43.734 INFO:tasks.workunit.client.0.smithi144.stdout:8/606: truncate d0/d9/f1d 843959 0 2024-09-10T11:29:43.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.732+0000 7fb06fef7700 1 -- 172.21.15.144:0/383693945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb068071b40 msgr2=0x7fb068071f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:43.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.732+0000 7fb06fef7700 1 --2- 172.21.15.144:0/383693945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb068071b40 0x7fb068071f30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb058009f90 tx=0x7fb05800b330 comp rx=0 tx=0).stop 2024-09-10T11:29:43.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.732+0000 7fb06fef7700 1 -- 172.21.15.144:0/383693945 shutdown_connections 2024-09-10T11:29:43.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.732+0000 7fb06fef7700 1 --2- 172.21.15.144:0/383693945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb068072500 0x7fb068107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:43.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.732+0000 7fb06fef7700 1 --2- 172.21.15.144:0/383693945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb068071b40 0x7fb068071f30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:43.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.732+0000 7fb06fef7700 1 -- 172.21.15.144:0/383693945 >> 172.21.15.144:0/383693945 conn(0x7fb06806d050 msgr2=0x7fb06806f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:43.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.732+0000 7fb06fef7700 1 -- 172.21.15.144:0/383693945 shutdown_connections 2024-09-10T11:29:43.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.732+0000 7fb06fef7700 1 -- 172.21.15.144:0/383693945 wait complete. 2024-09-10T11:29:43.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.733+0000 7fb06fef7700 1 Processor -- start 2024-09-10T11:29:43.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.733+0000 7fb06fef7700 1 -- start start 2024-09-10T11:29:43.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.733+0000 7fb06fef7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb068071b40 0x7fb06819e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:43.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.733+0000 7fb06fef7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb068072500 0x7fb06819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:43.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.733+0000 7fb06fef7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06819ed10 con 0x7fb068071b40 2024-09-10T11:29:43.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.733+0000 7fb06fef7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb06819ee50 con 0x7fb068072500 2024-09-10T11:29:43.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.734+0000 7fb06d492700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb068072500 0x7fb06819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:43.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.734+0000 7fb06d492700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb068072500 0x7fb06819e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.144:35524/0 (socket says 172.21.15.144:35524) 2024-09-10T11:29:43.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.734+0000 7fb06d492700 1 -- 172.21.15.144:0/3342900955 learned_addr learned my addr 172.21.15.144:0/3342900955 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:29:43.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.734+0000 7fb06dc93700 1 --2- 172.21.15.144:0/3342900955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb068071b40 0x7fb06819e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:43.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.734+0000 7fb06d492700 1 -- 172.21.15.144:0/3342900955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb068071b40 msgr2=0x7fb06819e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:43.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.734+0000 7fb06d492700 1 --2- 172.21.15.144:0/3342900955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb068071b40 0x7fb06819e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:43.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.734+0000 7fb06d492700 1 -- 172.21.15.144:0/3342900955 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb058009c40 con 0x7fb068072500 2024-09-10T11:29:43.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.734+0000 7fb06d492700 1 --2- 172.21.15.144:0/3342900955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb068072500 0x7fb06819e6f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00e940 tx=0x7fb05c00ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:43.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.735+0000 7fb066ffd700 1 -- 172.21.15.144:0/3342900955 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb05c004a00 con 0x7fb068072500 2024-09-10T11:29:43.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.735+0000 7fb066ffd700 1 -- 172.21.15.144:0/3342900955 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb05c00cc70 con 0x7fb068072500 2024-09-10T11:29:43.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.735+0000 7fb066ffd700 1 -- 172.21.15.144:0/3342900955 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb05c0055b0 con 0x7fb068072500 2024-09-10T11:29:43.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.735+0000 7fb06fef7700 1 -- 172.21.15.144:0/3342900955 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb068108ec0 con 0x7fb068072500 2024-09-10T11:29:43.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.735+0000 7fb06fef7700 1 -- 172.21.15.144:0/3342900955 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb068109430 con 0x7fb068072500 2024-09-10T11:29:43.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.736+0000 7fb06fef7700 1 -- 172.21.15.144:0/3342900955 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb068111490 con 0x7fb068072500 2024-09-10T11:29:43.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.738+0000 7fb066ffd700 1 -- 172.21.15.144:0/3342900955 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) v1 ==== 91089+0+0 (secure 0 0 0) 0x7fb05c021440 con 0x7fb068072500 2024-09-10T11:29:43.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.738+0000 7fb066ffd700 1 --2- 172.21.15.144:0/3342900955 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7fb05006d450 0x7fb05006f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:43.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.738+0000 7fb066ffd700 1 -- 172.21.15.144:0/3342900955 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb05c014070 con 0x7fb068072500 2024-09-10T11:29:43.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.739+0000 7fb06dc93700 1 --2- 172.21.15.144:0/3342900955 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7fb05006d450 0x7fb05006f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:43.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.739+0000 7fb06dc93700 1 --2- 172.21.15.144:0/3342900955 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7fb05006d450 0x7fb05006f900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb0580060b0 tx=0x7fb058006020 comp rx=0 tx=0).ready entity=mgr.24329 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:43.742 INFO:tasks.workunit.client.0.smithi144.stdout:6/584: truncate d9/dd/d3e/d47/f28 7109811 0 2024-09-10T11:29:43.742 INFO:tasks.workunit.client.0.smithi144.stdout:1/755: dwrite d5/d4a/d44/da2/fd7 [0,4194304] 0 2024-09-10T11:29:43.746 INFO:tasks.workunit.client.0.smithi144.stdout:4/704: dwrite d14/d67/d84/fc2 [0,4194304] 0 2024-09-10T11:29:43.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:43.745+0000 7fb066ffd700 1 -- 172.21.15.144:0/3342900955 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7fb05c05a2d0 con 0x7fb068072500 2024-09-10T11:29:43.747 INFO:tasks.workunit.client.0.smithi144.stdout:8/607: read d0/d9/fe [574516,71254] 0 2024-09-10T11:29:43.748 INFO:tasks.workunit.client.0.smithi144.stdout:0/489: mknod da/d3a/d8e/c97 0 2024-09-10T11:29:43.750 INFO:tasks.workunit.client.0.smithi144.stdout:7/670: link d5/dc/d12/f52 d5/dc/d11/d18/fd9 0 2024-09-10T11:29:43.752 INFO:tasks.workunit.client.0.smithi144.stdout:7/671: chown d5/dc/d11/f26 79 1 2024-09-10T11:29:43.757 INFO:tasks.workunit.client.0.smithi144.stdout:1/756: rename d5/d4a/c8a to d5/dd/dcb/cf7 0 2024-09-10T11:29:43.757 INFO:tasks.workunit.client.0.smithi144.stdout:5/698: creat d4/d52/dd2/d74/dbe/fd7 x:0 0 0 2024-09-10T11:29:43.758 INFO:tasks.workunit.client.0.smithi144.stdout:4/705: unlink d14/d1e/d65/fc6 0 2024-09-10T11:29:43.762 INFO:tasks.workunit.client.0.smithi144.stdout:0/490: creat da/d3a/d4b/d4e/d81/f98 x:0 0 0 2024-09-10T11:29:43.762 INFO:tasks.workunit.client.0.smithi144.stdout:8/608: dwrite d0/d9/d21/d7e/d85/fb9 [0,4194304] 0 2024-09-10T11:29:43.764 INFO:tasks.workunit.client.0.smithi144.stdout:8/609: write d0/d9/d1f/f1c [2588411,96356] 0 2024-09-10T11:29:43.769 INFO:tasks.workunit.client.1.smithi178.stdout:7/917: write d6/d10/d1e/d2d/d2b/f39 [3546871,111488] 0 2024-09-10T11:29:43.770 INFO:tasks.workunit.client.0.smithi144.stdout:4/706: symlink d14/d1e/d83/da4/lff 0 2024-09-10T11:29:43.771 INFO:tasks.workunit.client.0.smithi144.stdout:7/672: truncate d5/dc/d14/d40/d5e/d68/fbc 999292 0 2024-09-10T11:29:43.772 INFO:tasks.workunit.client.0.smithi144.stdout:9/627: sync 2024-09-10T11:29:43.774 INFO:tasks.workunit.client.0.smithi144.stdout:0/491: dwrite da/f2e [0,4194304] 0 2024-09-10T11:29:43.778 INFO:tasks.workunit.client.1.smithi178.stdout:7/918: link d6/d10/d1e/d2d/d2b/da0/d55/f90 d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d70/da2/f144 0 2024-09-10T11:29:43.780 INFO:tasks.workunit.client.1.smithi178.stdout:7/919: dread - d6/d10/d1d/d22/d93/db7/fec zero size 2024-09-10T11:29:43.780 INFO:tasks.workunit.client.0.smithi144.stdout:0/492: chown da/d3a/d4b/f86 449608533 1 2024-09-10T11:29:43.780 INFO:tasks.workunit.client.0.smithi144.stdout:1/757: truncate d5/d2a/d65/d73/fb1 103365 0 2024-09-10T11:29:43.781 INFO:tasks.workunit.client.0.smithi144.stdout:9/628: creat d11/d12/d1b/d2c/d78/fdc x:0 0 0 2024-09-10T11:29:43.783 INFO:tasks.workunit.client.0.smithi144.stdout:5/699: rename d4/d20/d22/c3b to d4/d52/db1/cd8 0 2024-09-10T11:29:43.789 INFO:tasks.workunit.client.1.smithi178.stdout:7/920: dread d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d70/da2/f144 [0,4194304] 0 2024-09-10T11:29:43.790 INFO:tasks.workunit.client.0.smithi144.stdout:7/673: symlink d5/dc/d14/d40/d58/d6e/lda 0 2024-09-10T11:29:43.790 INFO:tasks.workunit.client.0.smithi144.stdout:9/629: mknod d11/d12/d1b/d2c/d78/dd2/cdd 0 2024-09-10T11:29:43.792 INFO:tasks.workunit.client.1.smithi178.stdout:7/921: symlink d6/d10/d1d/d22/d3b/da1/l145 0 2024-09-10T11:29:43.793 INFO:tasks.workunit.client.1.smithi178.stdout:7/922: chown d6/d10/d1d/d22/d50/d98/da6/dfd 4835 1 2024-09-10T11:29:43.794 INFO:tasks.workunit.client.0.smithi144.stdout:0/493: dwrite da/d3a/f50 [0,4194304] 0 2024-09-10T11:29:43.795 INFO:tasks.workunit.client.0.smithi144.stdout:7/674: truncate d5/fce 1496405 0 2024-09-10T11:29:43.795 INFO:tasks.workunit.client.0.smithi144.stdout:0/494: readlink l8 0 2024-09-10T11:29:43.797 INFO:tasks.workunit.client.0.smithi144.stdout:6/585: dread d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/f51 [4194304,4194304] 0 2024-09-10T11:29:43.800 INFO:tasks.workunit.client.0.smithi144.stdout:8/610: dwrite d0/d9/fe [0,4194304] 0 2024-09-10T11:29:43.801 INFO:tasks.workunit.client.0.smithi144.stdout:4/707: link d14/d67/d92/c9e d14/d1e/d22/d34/d3e/d40/d4b/c100 0 2024-09-10T11:29:43.801 INFO:tasks.workunit.client.1.smithi178.stdout:4/931: sync 2024-09-10T11:29:43.806 INFO:tasks.workunit.client.0.smithi144.stdout:0/495: mknod da/d44/d82/d7e/c99 0 2024-09-10T11:29:43.806 INFO:tasks.workunit.client.1.smithi178.stdout:4/932: creat d0/d9/dc/dca/f147 x:0 0 0 2024-09-10T11:29:43.808 INFO:tasks.workunit.client.0.smithi144.stdout:9/630: chown d11/d12/d1b/d41/d8f/dab/d64 286 1 2024-09-10T11:29:43.808 INFO:tasks.workunit.client.0.smithi144.stdout:5/700: rename d4/d20/d24/d2d/f2e to d4/d5/fd9 0 2024-09-10T11:29:43.810 INFO:tasks.workunit.client.1.smithi178.stdout:7/923: dwrite d6/d10/d34/f4e [0,4194304] 0 2024-09-10T11:29:43.811 INFO:tasks.workunit.client.0.smithi144.stdout:8/611: write d0/d9/d1f/d34/f53 [970894,20523] 0 2024-09-10T11:29:43.811 INFO:tasks.workunit.client.0.smithi144.stdout:7/675: write d5/dc/d11/d18/f5f [586104,130077] 0 2024-09-10T11:29:43.813 INFO:tasks.workunit.client.1.smithi178.stdout:4/933: creat d0/d9/dc/d8e/df1/f148 x:0 0 0 2024-09-10T11:29:43.815 INFO:tasks.workunit.client.0.smithi144.stdout:9/631: stat d11/d36/d2d/f9a 0 2024-09-10T11:29:43.816 INFO:tasks.workunit.client.1.smithi178.stdout:7/924: mkdir d6/d10/d1d/daa/d134/d146 0 2024-09-10T11:29:43.817 INFO:tasks.workunit.client.0.smithi144.stdout:4/708: rmdir d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/de4 39 2024-09-10T11:29:43.819 INFO:tasks.workunit.client.0.smithi144.stdout:1/758: dwrite d5/d2a/f46 [0,4194304] 0 2024-09-10T11:29:43.820 INFO:tasks.workunit.client.1.smithi178.stdout:7/925: creat d6/d10/d1d/d22/d50/d98/d13f/f147 x:0 0 0 2024-09-10T11:29:43.822 INFO:tasks.workunit.client.0.smithi144.stdout:6/586: dwrite d9/dd/f1a [8388608,4194304] 0 2024-09-10T11:29:43.822 INFO:tasks.workunit.client.0.smithi144.stdout:9/632: read d11/d36/f7f [542019,34225] 0 2024-09-10T11:29:43.823 INFO:tasks.workunit.client.0.smithi144.stdout:4/709: symlink d14/d67/d69/dca/l101 0 2024-09-10T11:29:43.825 INFO:tasks.workunit.client.0.smithi144.stdout:0/496: write da/d3a/d4b/d4e/f60 [14736,81806] 0 2024-09-10T11:29:43.826 INFO:tasks.workunit.client.0.smithi144.stdout:5/701: creat d4/d20/d22/d5c/d5d/d64/dc2/dca/fda x:0 0 0 2024-09-10T11:29:43.827 INFO:tasks.workunit.client.0.smithi144.stdout:8/612: rmdir d0/d9/d21/d25/db3/dbb 0 2024-09-10T11:29:43.828 INFO:tasks.workunit.client.0.smithi144.stdout:0/497: chown da/d44/d82/d7e 7314020 1 2024-09-10T11:29:43.830 INFO:tasks.workunit.client.1.smithi178.stdout:7/926: rename d6/d10/d34/d6a/d6f to d6/d10/d1d/d22/d50/d98/ddc/d148 0 2024-09-10T11:29:43.831 INFO:tasks.workunit.client.0.smithi144.stdout:2/547: sync 2024-09-10T11:29:43.832 INFO:tasks.workunit.client.0.smithi144.stdout:9/633: symlink d11/d12/d1b/d2c/lde 0 2024-09-10T11:29:43.834 INFO:tasks.workunit.client.1.smithi178.stdout:7/927: mknod d6/d10/d1d/d22/d50/d98/d13f/c149 0 2024-09-10T11:29:43.834 INFO:tasks.workunit.client.0.smithi144.stdout:0/498: write da/d3a/d4b/d4e/d62/f68 [1363122,10990] 0 2024-09-10T11:29:43.836 INFO:tasks.workunit.client.0.smithi144.stdout:7/676: read d5/f37 [1051030,129167] 0 2024-09-10T11:29:43.838 INFO:tasks.workunit.client.0.smithi144.stdout:2/548: mknod d7/de/d36/d45/d8e/d9a/cb7 0 2024-09-10T11:29:43.840 INFO:tasks.workunit.client.0.smithi144.stdout:7/677: symlink d5/dc/d12/d2b/ldb 0 2024-09-10T11:29:43.841 INFO:tasks.workunit.client.0.smithi144.stdout:8/613: creat d0/d9/d21/d25/db3/dc2/fce x:0 0 0 2024-09-10T11:29:43.843 INFO:tasks.workunit.client.1.smithi178.stdout:7/928: dread d6/d10/d1d/f27 [0,4194304] 0 2024-09-10T11:29:43.843 INFO:tasks.workunit.client.0.smithi144.stdout:4/710: dread d14/d6f/f96 [0,4194304] 0 2024-09-10T11:29:43.844 INFO:tasks.workunit.client.0.smithi144.stdout:9/634: getdents d11/d36/db1 0 2024-09-10T11:29:43.845 INFO:tasks.workunit.client.0.smithi144.stdout:7/678: write d5/dc/d14/d40/d58/d6a/f92 [4691890,112762] 0 2024-09-10T11:29:43.846 INFO:tasks.workunit.client.0.smithi144.stdout:8/614: symlink d0/d9/d21/d25/d4f/lcf 0 2024-09-10T11:29:43.849 INFO:tasks.workunit.client.0.smithi144.stdout:4/711: unlink d14/d1e/d22/d34/d3e/fd0 0 2024-09-10T11:29:43.851 INFO:tasks.workunit.client.0.smithi144.stdout:0/499: dread da/d3a/f50 [0,4194304] 0 2024-09-10T11:29:43.853 INFO:tasks.workunit.client.0.smithi144.stdout:9/635: mknod d11/d12/d1b/d41/d8f/dc0/dd0/cdf 0 2024-09-10T11:29:43.853 INFO:tasks.workunit.client.0.smithi144.stdout:2/549: dwrite d7/de/f22 [0,4194304] 0 2024-09-10T11:29:43.855 INFO:tasks.workunit.client.0.smithi144.stdout:1/759: write d5/d2a/d65/fcd [359529,101900] 0 2024-09-10T11:29:43.855 INFO:tasks.workunit.client.0.smithi144.stdout:5/702: truncate d4/fa 1044069 0 2024-09-10T11:29:43.856 INFO:tasks.workunit.client.0.smithi144.stdout:6/587: truncate d9/dd/d3e/d47/d13/f5a 1427635 0 2024-09-10T11:29:43.856 INFO:tasks.workunit.client.0.smithi144.stdout:2/550: readlink d7/de/d3d/d43/l62 0 2024-09-10T11:29:43.857 INFO:tasks.workunit.client.0.smithi144.stdout:4/712: mkdir d14/d67/d69/d90/d102 0 2024-09-10T11:29:43.860 INFO:tasks.workunit.client.0.smithi144.stdout:1/760: readlink d5/df/d4b/d51/da7/l81 0 2024-09-10T11:29:43.861 INFO:tasks.workunit.client.1.smithi178.stdout:7/929: dwrite d6/d10/d1d/d22/d50/d7e/d82/db2/f36 [0,4194304] 0 2024-09-10T11:29:43.863 INFO:tasks.workunit.client.1.smithi178.stdout:7/930: readlink d6/d10/d1d/d4a/d79/d83/le9 0 2024-09-10T11:29:43.863 INFO:tasks.workunit.client.0.smithi144.stdout:7/679: creat d5/dc/d14/d40/fdc x:0 0 0 2024-09-10T11:29:43.863 INFO:tasks.workunit.client.0.smithi144.stdout:7/680: stat d5/dc/lb7 0 2024-09-10T11:29:43.864 INFO:tasks.workunit.client.0.smithi144.stdout:8/615: dwrite d0/d1b/f2f [0,4194304] 0 2024-09-10T11:29:43.865 INFO:tasks.workunit.client.0.smithi144.stdout:9/636: creat d11/d12/d1b/d2c/d8a/fe0 x:0 0 0 2024-09-10T11:29:43.866 INFO:tasks.workunit.client.0.smithi144.stdout:0/500: creat da/d3a/d4b/d61/f9a x:0 0 0 2024-09-10T11:29:43.867 INFO:tasks.workunit.client.0.smithi144.stdout:2/551: symlink d7/de/d36/d45/d8e/lb8 0 2024-09-10T11:29:43.867 INFO:tasks.workunit.client.0.smithi144.stdout:0/501: write da/d3a/d51/f7a [1735348,99530] 0 2024-09-10T11:29:43.870 INFO:tasks.workunit.client.0.smithi144.stdout:1/761: mknod d5/d2a/d65/d73/cf8 0 2024-09-10T11:29:43.870 INFO:tasks.workunit.client.0.smithi144.stdout:0/502: chown da/f19 0 1 2024-09-10T11:29:43.870 INFO:tasks.workunit.client.1.smithi178.stdout:7/931: rename d6/d10/d1d/d22/d93/d12f/c13c to d6/d10/d1d/d22/d50/d7e/d123/c14a 0 2024-09-10T11:29:43.872 INFO:tasks.workunit.client.0.smithi144.stdout:4/713: mkdir d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f/d103 0 2024-09-10T11:29:43.873 INFO:tasks.workunit.client.0.smithi144.stdout:0/503: fsync da/d3a/d51/f7a 0 2024-09-10T11:29:43.874 INFO:tasks.workunit.client.1.smithi178.stdout:7/932: mkdir d6/d10/d34/d14b 0 2024-09-10T11:29:43.879 INFO:tasks.workunit.client.0.smithi144.stdout:9/637: truncate d11/d12/d1b/d2c/d78/fa0 566503 0 2024-09-10T11:29:43.880 INFO:tasks.workunit.client.0.smithi144.stdout:2/552: rmdir d7/de/d3d/d4e/dac 39 2024-09-10T11:29:43.884 INFO:tasks.workunit.client.0.smithi144.stdout:4/714: symlink d14/d1e/l104 0 2024-09-10T11:29:43.884 INFO:tasks.workunit.client.0.smithi144.stdout:7/681: fsync d5/dc/f56 0 2024-09-10T11:29:43.887 INFO:tasks.workunit.client.0.smithi144.stdout:9/638: creat d11/d36/db1/dbb/fe1 x:0 0 0 2024-09-10T11:29:43.888 INFO:tasks.workunit.client.0.smithi144.stdout:1/762: getdents d5/d4a/dc0 0 2024-09-10T11:29:43.890 INFO:tasks.workunit.client.0.smithi144.stdout:0/504: creat da/d3a/d4b/d61/d6c/f9b x:0 0 0 2024-09-10T11:29:43.891 INFO:tasks.workunit.client.0.smithi144.stdout:5/703: dread d4/fa [0,4194304] 0 2024-09-10T11:29:43.891 INFO:tasks.workunit.client.0.smithi144.stdout:6/588: dwrite d9/dd/d11/d1b/f32 [0,4194304] 0 2024-09-10T11:29:43.896 INFO:tasks.workunit.client.0.smithi144.stdout:5/704: write d4/d5/d18/d38/fc5 [289099,83047] 0 2024-09-10T11:29:43.897 INFO:tasks.workunit.client.0.smithi144.stdout:9/639: mknod d11/d12/d1b/d2c/ce2 0 2024-09-10T11:29:43.897 INFO:tasks.workunit.client.0.smithi144.stdout:4/715: chown d14/d1e/d65/d8a/d8f/la2 1059745746 1 2024-09-10T11:29:43.898 INFO:tasks.workunit.client.0.smithi144.stdout:0/505: creat da/d3a/d51/d5f/d83/f9c x:0 0 0 2024-09-10T11:29:43.900 INFO:tasks.workunit.client.0.smithi144.stdout:8/616: dwrite d0/f3f [0,4194304] 0 2024-09-10T11:29:43.901 INFO:tasks.workunit.client.1.smithi178.stdout:7/933: dread d6/d10/d1e/d2d/d2b/da0/fb9 [0,4194304] 0 2024-09-10T11:29:43.902 INFO:tasks.workunit.client.0.smithi144.stdout:9/640: write d11/d36/db1/dbb/fe1 [834570,62293] 0 2024-09-10T11:29:43.905 INFO:tasks.workunit.client.0.smithi144.stdout:5/705: mknod d4/d5/d32/d47/cdb 0 2024-09-10T11:29:43.905 INFO:tasks.workunit.client.0.smithi144.stdout:6/589: mknod d9/dd/d11/d1b/d2f/dab/dc6/cd3 0 2024-09-10T11:29:43.905 INFO:tasks.workunit.client.1.smithi178.stdout:7/934: creat d6/dbf/f14c x:0 0 0 2024-09-10T11:29:43.905 INFO:tasks.workunit.client.0.smithi144.stdout:1/763: write d5/f8 [5091863,53314] 0 2024-09-10T11:29:43.906 INFO:tasks.workunit.client.0.smithi144.stdout:8/617: symlink d0/d9/d23/d26/ld0 0 2024-09-10T11:29:43.907 INFO:tasks.workunit.client.0.smithi144.stdout:8/618: readlink d0/d1b/l28 0 2024-09-10T11:29:43.908 INFO:tasks.workunit.client.0.smithi144.stdout:7/682: write d5/dc/d14/fb3 [3902710,50689] 0 2024-09-10T11:29:43.912 INFO:tasks.workunit.client.0.smithi144.stdout:0/506: dread da/f30 [0,4194304] 0 2024-09-10T11:29:43.912 INFO:tasks.workunit.client.0.smithi144.stdout:5/706: write d4/d5/d18/d67/d59/f6c [2756985,83826] 0 2024-09-10T11:29:43.914 INFO:tasks.workunit.client.1.smithi178.stdout:7/935: truncate d6/ff 3218770 0 2024-09-10T11:29:43.915 INFO:tasks.workunit.client.1.smithi178.stdout:7/936: fsync d6/d10/d1e/d2d/f16 0 2024-09-10T11:29:43.918 INFO:tasks.workunit.client.0.smithi144.stdout:5/707: readlink d4/d5/d18/d38/l95 0 2024-09-10T11:29:43.918 INFO:tasks.workunit.client.0.smithi144.stdout:0/507: chown da/d3a/l87 91455324 1 2024-09-10T11:29:43.918 INFO:tasks.workunit.client.0.smithi144.stdout:8/619: mkdir d0/d9/d21/d25/db3/dc2/dd1 0 2024-09-10T11:29:43.919 INFO:tasks.workunit.client.0.smithi144.stdout:7/683: mkdir d5/dc/d14/d40/d5e/ddd 0 2024-09-10T11:29:43.919 INFO:tasks.workunit.client.1.smithi178.stdout:7/937: creat d6/d10/d34/d6a/f14d x:0 0 0 2024-09-10T11:29:43.921 INFO:tasks.workunit.client.0.smithi144.stdout:9/641: dwrite d11/d12/d1b/d41/d8f/dab/d64/fda [0,4194304] 0 2024-09-10T11:29:43.921 INFO:tasks.workunit.client.0.smithi144.stdout:0/508: stat da/f2e 0 2024-09-10T11:29:43.922 INFO:tasks.workunit.client.0.smithi144.stdout:4/716: link d14/d1e/d83/da4/da9/dbc/ldf d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/l105 0 2024-09-10T11:29:43.923 INFO:tasks.workunit.client.0.smithi144.stdout:7/684: write d5/dc/fae [983970,60165] 0 2024-09-10T11:29:43.924 INFO:tasks.workunit.client.1.smithi178.stdout:7/938: link d6/d10/d1d/d22/d50/l6c d6/d10/d1e/d2d/d2b/da0/d55/l14e 0 2024-09-10T11:29:43.925 INFO:tasks.workunit.client.0.smithi144.stdout:8/620: read d0/d9/d21/f73 [330932,72460] 0 2024-09-10T11:29:43.925 INFO:tasks.workunit.client.0.smithi144.stdout:1/764: dwrite d5/d4a/d63/d84/dab/fbb [0,4194304] 0 2024-09-10T11:29:43.927 INFO:tasks.workunit.client.0.smithi144.stdout:7/685: write d5/f99 [827158,54524] 0 2024-09-10T11:29:43.928 INFO:tasks.workunit.client.1.smithi178.stdout:7/939: symlink d6/d10/d34/d6a/d126/l14f 0 2024-09-10T11:29:43.932 INFO:tasks.workunit.client.0.smithi144.stdout:0/509: creat da/d3a/d4b/d61/d6c/f9d x:0 0 0 2024-09-10T11:29:43.935 INFO:tasks.workunit.client.0.smithi144.stdout:5/708: creat d4/d5/d32/fdc x:0 0 0 2024-09-10T11:29:43.935 INFO:tasks.workunit.client.0.smithi144.stdout:7/686: stat d5/dc/d2a/fa8 0 2024-09-10T11:29:43.935 INFO:tasks.workunit.client.1.smithi178.stdout:7/940: rename d6/d10/d1d/d22/d93/f99 to d6/d10/d1d/d22/d50/d98/ddc/f150 0 2024-09-10T11:29:43.936 INFO:tasks.workunit.client.0.smithi144.stdout:9/642: creat d11/d12/d1b/d22/fe3 x:0 0 0 2024-09-10T11:29:43.936 INFO:tasks.workunit.client.1.smithi178.stdout:7/941: dread - d6/d10/d1d/fea zero size 2024-09-10T11:29:43.937 INFO:tasks.workunit.client.0.smithi144.stdout:1/765: mkdir d5/d4a/d63/d84/df9 0 2024-09-10T11:29:43.937 INFO:tasks.workunit.client.1.smithi178.stdout:7/942: write d6/fc9 [4750851,17698] 0 2024-09-10T11:29:43.938 INFO:tasks.workunit.client.0.smithi144.stdout:4/717: truncate d14/d1e/d22/d34/d3e/d40/d4b/f80 1131452 0 2024-09-10T11:29:43.939 INFO:tasks.workunit.client.1.smithi178.stdout:7/943: chown d6/d10/d1d/d22/d50/def/df3/f12b 187 1 2024-09-10T11:29:43.939 INFO:tasks.workunit.client.0.smithi144.stdout:0/510: write da/f46 [890221,26100] 0 2024-09-10T11:29:43.941 INFO:tasks.workunit.client.1.smithi178.stdout:4/934: sync 2024-09-10T11:29:43.941 INFO:tasks.workunit.client.1.smithi178.stdout:7/944: symlink d6/d10/d1d/d4a/d79/d83/l151 0 2024-09-10T11:29:43.943 INFO:tasks.workunit.client.1.smithi178.stdout:7/945: read d6/d10/d1d/d4a/d79/df1/fff [545091,25441] 0 2024-09-10T11:29:43.944 INFO:tasks.workunit.client.0.smithi144.stdout:7/687: write d5/dc/d12/d2b/fc2 [266424,47880] 0 2024-09-10T11:29:43.944 INFO:tasks.workunit.client.0.smithi144.stdout:1/766: write d5/f8 [29335,13730] 0 2024-09-10T11:29:43.947 INFO:tasks.workunit.client.1.smithi178.stdout:7/946: creat d6/d10/d1e/d2d/d2b/da0/dc0/f152 x:0 0 0 2024-09-10T11:29:43.948 INFO:tasks.workunit.client.1.smithi178.stdout:4/935: dwrite d0/d9/dc/d8e/df1/f148 [0,4194304] 0 2024-09-10T11:29:43.948 INFO:tasks.workunit.client.0.smithi144.stdout:9/643: mkdir d11/d12/d1b/d2c/d78/dd2/de4 0 2024-09-10T11:29:43.948 INFO:tasks.workunit.client.0.smithi144.stdout:6/590: dwrite d9/dd/d11/d1b/f94 [0,4194304] 0 2024-09-10T11:29:43.949 INFO:tasks.workunit.client.0.smithi144.stdout:0/511: symlink da/d3a/d4b/d4e/d62/d78/l9e 0 2024-09-10T11:29:43.951 INFO:tasks.workunit.client.0.smithi144.stdout:8/621: dwrite d0/d9/d21/f77 [0,4194304] 0 2024-09-10T11:29:43.952 INFO:tasks.workunit.client.1.smithi178.stdout:4/936: creat d0/d8/d3e/d123/f149 x:0 0 0 2024-09-10T11:29:43.953 INFO:tasks.workunit.client.0.smithi144.stdout:1/767: chown d5/df/d4b/f8e 4 1 2024-09-10T11:29:43.954 INFO:tasks.workunit.client.0.smithi144.stdout:9/644: creat d11/d36/db1/dbb/fe5 x:0 0 0 2024-09-10T11:29:43.961 INFO:tasks.workunit.client.1.smithi178.stdout:4/937: creat d0/d8/da/d1a/dff/d133/f14a x:0 0 0 2024-09-10T11:29:43.961 INFO:tasks.workunit.client.0.smithi144.stdout:5/709: dwrite d4/d5/d18/d67/f65 [4194304,4194304] 0 2024-09-10T11:29:43.963 INFO:tasks.workunit.client.0.smithi144.stdout:4/718: mknod d14/d67/d69/d90/dda/de7/c106 0 2024-09-10T11:29:43.963 INFO:tasks.workunit.client.1.smithi178.stdout:4/938: rmdir d0/d8/df/d13b/d5d 39 2024-09-10T11:29:43.966 INFO:tasks.workunit.client.0.smithi144.stdout:6/591: creat d9/dd/d3e/d47/d13/d60/fd4 x:0 0 0 2024-09-10T11:29:43.968 INFO:tasks.workunit.client.0.smithi144.stdout:5/710: mkdir d4/d20/d22/d75/ddd 0 2024-09-10T11:29:43.968 INFO:tasks.workunit.client.0.smithi144.stdout:7/688: write d5/dc/d2a/d60/dd6/d98/dd1/f4a [1692331,25243] 0 2024-09-10T11:29:43.969 INFO:tasks.workunit.client.1.smithi178.stdout:4/939: chown d0/d9/dc/d8e/dbf/lfe 7 1 2024-09-10T11:29:43.970 INFO:tasks.workunit.client.1.smithi178.stdout:4/940: write d0/d9/dc/d8e/f9f [5236153,102692] 0 2024-09-10T11:29:43.973 INFO:tasks.workunit.client.0.smithi144.stdout:4/719: creat d14/d67/d69/d90/dda/f107 x:0 0 0 2024-09-10T11:29:43.973 INFO:tasks.workunit.client.0.smithi144.stdout:5/711: mkdir d4/d20/d22/d75/dde 0 2024-09-10T11:29:43.976 INFO:tasks.workunit.client.0.smithi144.stdout:9/645: rename d11/d12/d1b/l49 to d11/d12/d1b/d41/d8f/le6 0 2024-09-10T11:29:43.976 INFO:tasks.workunit.client.0.smithi144.stdout:8/622: dread d0/d1b/f4c [0,4194304] 0 2024-09-10T11:29:43.976 INFO:tasks.workunit.client.0.smithi144.stdout:4/720: dread d14/d6f/f96 [0,4194304] 0 2024-09-10T11:29:43.978 INFO:tasks.workunit.client.1.smithi178.stdout:4/941: creat d0/d9/dc/dca/f14b x:0 0 0 2024-09-10T11:29:43.978 INFO:tasks.workunit.client.0.smithi144.stdout:7/689: mkdir d5/dc/d12/d2b/dde 0 2024-09-10T11:29:43.979 INFO:tasks.workunit.client.1.smithi178.stdout:4/942: dread - d0/d9d/f112 zero size 2024-09-10T11:29:43.980 INFO:tasks.workunit.client.0.smithi144.stdout:8/623: chown d0/d9/d23/f87 3 1 2024-09-10T11:29:43.980 INFO:tasks.workunit.client.0.smithi144.stdout:1/768: dwrite d5/df/f55 [0,4194304] 0 2024-09-10T11:29:43.980 INFO:tasks.workunit.client.1.smithi178.stdout:7/947: write d6/d10/d1d/d22/d50/d98/da6/dd5/ffc [1536677,44238] 0 2024-09-10T11:29:43.981 INFO:tasks.workunit.client.0.smithi144.stdout:5/712: readlink d4/d5/d18/d51/lae 0 2024-09-10T11:29:43.981 INFO:tasks.workunit.client.1.smithi178.stdout:7/948: fsync d6/d10/d1d/d22/d3b/f65 0 2024-09-10T11:29:43.982 INFO:tasks.workunit.client.0.smithi144.stdout:4/721: write d14/d1e/d22/d34/d3e/f97 [1294736,11063] 0 2024-09-10T11:29:43.983 INFO:tasks.workunit.client.1.smithi178.stdout:7/949: fsync d6/d10/d1e/d2d/d2b/da0/f7a 0 2024-09-10T11:29:43.983 INFO:tasks.workunit.client.1.smithi178.stdout:4/943: dread d0/d8/df/f9e [0,4194304] 0 2024-09-10T11:29:43.985 INFO:tasks.workunit.client.0.smithi144.stdout:9/646: write d11/d36/d2d/f70 [111370,67795] 0 2024-09-10T11:29:43.985 INFO:tasks.workunit.client.0.smithi144.stdout:6/592: link d9/d1f/fbb d9/d38/fd5 0 2024-09-10T11:29:43.987 INFO:tasks.workunit.client.1.smithi178.stdout:7/950: creat d6/dbf/f153 x:0 0 0 2024-09-10T11:29:43.987 INFO:tasks.workunit.client.0.smithi144.stdout:1/769: creat d5/dd/d64/dad/ffa x:0 0 0 2024-09-10T11:29:43.987 INFO:tasks.workunit.client.0.smithi144.stdout:4/722: chown d14/d1e/d22/d34/d91 115 1 2024-09-10T11:29:43.988 INFO:tasks.workunit.client.1.smithi178.stdout:4/944: read d0/d8/d3e/f85 [204916,11905] 0 2024-09-10T11:29:43.988 INFO:tasks.workunit.client.0.smithi144.stdout:8/624: creat d0/d9/d1f/d34/da0/dc4/fd2 x:0 0 0 2024-09-10T11:29:43.990 INFO:tasks.workunit.client.0.smithi144.stdout:5/713: fsync d4/d20/d24/d2d/d4b/fa7 0 2024-09-10T11:29:43.990 INFO:tasks.workunit.client.0.smithi144.stdout:1/770: chown d5/d2a/f46 1 1 2024-09-10T11:29:43.990 INFO:tasks.workunit.client.0.smithi144.stdout:9/647: mkdir d11/d36/d2d/d38/de7 0 2024-09-10T11:29:43.992 INFO:tasks.workunit.client.0.smithi144.stdout:4/723: chown d14/d1e/d65/fe6 5361 1 2024-09-10T11:29:43.992 INFO:tasks.workunit.client.1.smithi178.stdout:7/951: unlink d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d70/f3c 0 2024-09-10T11:29:43.993 INFO:tasks.workunit.client.0.smithi144.stdout:8/625: fsync d0/d1b/d47/d5e/d76/fb2 0 2024-09-10T11:29:43.993 INFO:tasks.workunit.client.1.smithi178.stdout:4/945: creat d0/d8/da/d1a/dff/f14c x:0 0 0 2024-09-10T11:29:43.994 INFO:tasks.workunit.client.0.smithi144.stdout:9/648: rmdir d11/d12/d1b/d41/d8f/dc0 39 2024-09-10T11:29:43.994 INFO:tasks.workunit.client.0.smithi144.stdout:9/649: chown d11/d36/c2e 354 1 2024-09-10T11:29:43.995 INFO:tasks.workunit.client.1.smithi178.stdout:7/952: symlink d6/d10/d1e/d2d/d2b/da0/d55/l154 0 2024-09-10T11:29:43.998 INFO:tasks.workunit.client.1.smithi178.stdout:4/946: mknod d0/d9/dc/c14d 0 2024-09-10T11:29:44.000 INFO:tasks.workunit.client.1.smithi178.stdout:7/953: chown d6/d10/d1e/d2d/d2b/l2e 26762 1 2024-09-10T11:29:44.001 INFO:tasks.workunit.client.1.smithi178.stdout:7/954: chown d6/d10/d1e/d2d/d2b/da0/dc0/f152 41 1 2024-09-10T11:29:44.002 INFO:tasks.workunit.client.1.smithi178.stdout:7/955: chown d6/d10/d1e/d2d/c19 156 1 2024-09-10T11:29:44.003 INFO:tasks.workunit.client.0.smithi144.stdout:5/714: rename d4/d5/d18/d38/d6d/d8f to d4/d20/d22/d75/ddd/ddf 0 2024-09-10T11:29:44.003 INFO:tasks.workunit.client.1.smithi178.stdout:4/947: link d0/d8/df/dde/fe4 d0/d9/d3d/d4f/ddf/f14e 0 2024-09-10T11:29:44.005 INFO:tasks.workunit.client.0.smithi144.stdout:6/593: rmdir d9/dd/d3e/dbf 0 2024-09-10T11:29:44.006 INFO:tasks.workunit.client.1.smithi178.stdout:4/948: creat d0/d8/df/dfd/f14f x:0 0 0 2024-09-10T11:29:44.007 INFO:tasks.workunit.client.1.smithi178.stdout:7/956: getdents d6/d10/d1e/d2d/d2b/da0/dc0 0 2024-09-10T11:29:44.008 INFO:tasks.workunit.client.0.smithi144.stdout:9/650: symlink d11/d12/d1b/d2c/le8 0 2024-09-10T11:29:44.009 INFO:tasks.workunit.client.1.smithi178.stdout:4/949: mkdir d0/d8/da/d1a/dff/d133/d150 0 2024-09-10T11:29:44.010 INFO:tasks.workunit.client.0.smithi144.stdout:5/715: chown d4/d20/l2f 126889 1 2024-09-10T11:29:44.011 INFO:tasks.workunit.client.0.smithi144.stdout:4/724: rmdir d14 39 2024-09-10T11:29:44.013 INFO:tasks.workunit.client.1.smithi178.stdout:4/950: rename d0/d9/d3d/d4f/ddb/f11d to d0/d9d/f151 0 2024-09-10T11:29:44.015 INFO:tasks.workunit.client.0.smithi144.stdout:5/716: creat d4/d20/d22/d75/fe0 x:0 0 0 2024-09-10T11:29:44.017 INFO:tasks.workunit.client.0.smithi144.stdout:9/651: creat d11/d12/d1b/d41/d8f/dc0/dd0/fe9 x:0 0 0 2024-09-10T11:29:44.018 INFO:tasks.workunit.client.1.smithi178.stdout:4/951: creat d0/d9/dc/d8e/f152 x:0 0 0 2024-09-10T11:29:44.019 INFO:tasks.workunit.client.0.smithi144.stdout:5/717: creat d4/d5/d18/d38/d6d/d8b/fe1 x:0 0 0 2024-09-10T11:29:44.019 INFO:tasks.workunit.client.0.smithi144.stdout:7/690: write d5/dc/d11/f81 [18109,21615] 0 2024-09-10T11:29:44.019 INFO:tasks.workunit.client.0.smithi144.stdout:8/626: getdents d0/d9/d21/d25/db3/dc2 0 2024-09-10T11:29:44.020 INFO:tasks.workunit.client.1.smithi178.stdout:7/957: write d6/d10/d1e/d2d/f21 [2947944,52188] 0 2024-09-10T11:29:44.021 INFO:tasks.workunit.client.0.smithi144.stdout:6/594: dread d9/dd/d11/f90 [0,4194304] 0 2024-09-10T11:29:44.021 INFO:tasks.workunit.client.0.smithi144.stdout:7/691: chown d5/dc/d11/d18/laf 26264 1 2024-09-10T11:29:44.022 INFO:tasks.workunit.client.1.smithi178.stdout:4/952: dread d0/d8/d3e/f85 [0,4194304] 0 2024-09-10T11:29:44.023 INFO:tasks.workunit.client.0.smithi144.stdout:7/692: stat d5/dc/d2a/d41/c61 0 2024-09-10T11:29:44.023 INFO:tasks.workunit.client.1.smithi178.stdout:7/958: creat d6/dbf/ddd/f155 x:0 0 0 2024-09-10T11:29:44.024 INFO:tasks.workunit.client.1.smithi178.stdout:4/953: creat d0/d9d/f153 x:0 0 0 2024-09-10T11:29:44.025 INFO:tasks.workunit.client.0.smithi144.stdout:8/627: creat d0/d9/d1f/d34/da0/dc4/fd3 x:0 0 0 2024-09-10T11:29:44.025 INFO:tasks.workunit.client.0.smithi144.stdout:1/771: write d5/d2a/d65/dc9/f76 [4413192,84535] 0 2024-09-10T11:29:44.026 INFO:tasks.workunit.client.0.smithi144.stdout:5/718: mknod d4/d20/d22/ce2 0 2024-09-10T11:29:44.028 INFO:tasks.workunit.client.0.smithi144.stdout:5/719: read - d4/d20/d22/d75/fe0 zero size 2024-09-10T11:29:44.029 INFO:tasks.workunit.client.1.smithi178.stdout:4/954: link d0/d8/df/f3b d0/d9d/de1/f154 0 2024-09-10T11:29:44.030 INFO:tasks.workunit.client.0.smithi144.stdout:6/595: rename d9/dd/d3e/d47/d13 to d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6 0 2024-09-10T11:29:44.030 INFO:tasks.workunit.client.1.smithi178.stdout:4/955: chown d0/d8/da/dae 4694 1 2024-09-10T11:29:44.031 INFO:tasks.workunit.client.0.smithi144.stdout:6/596: dread - d9/dd/d11/d1b/d2f/d7f/d82/fb3 zero size 2024-09-10T11:29:44.033 INFO:tasks.workunit.client.0.smithi144.stdout:8/628: symlink d0/d1b/ld4 0 2024-09-10T11:29:44.034 INFO:tasks.workunit.client.1.smithi178.stdout:7/959: dwrite d6/d10/d1d/f73 [0,4194304] 0 2024-09-10T11:29:44.038 INFO:tasks.workunit.client.0.smithi144.stdout:6/597: unlink d9/d38/l59 0 2024-09-10T11:29:44.039 INFO:tasks.workunit.client.0.smithi144.stdout:9/652: write d11/d12/d1b/d2c/d78/fa0 [731397,35553] 0 2024-09-10T11:29:44.041 INFO:tasks.workunit.client.0.smithi144.stdout:7/693: link d5/dc/d14/d40/d58/d6a/fa0 d5/dc/d14/d40/d58/d6a/dad/fdf 0 2024-09-10T11:29:44.042 INFO:tasks.workunit.client.0.smithi144.stdout:9/653: truncate d11/d12/d1b/d22/f74 862434 0 2024-09-10T11:29:44.042 INFO:tasks.workunit.client.1.smithi178.stdout:7/960: write d6/d10/d1d/daa/fd0 [1232714,125700] 0 2024-09-10T11:29:44.045 INFO:tasks.workunit.client.0.smithi144.stdout:5/720: write d4/d5/d18/d67/f62 [648092,6259] 0 2024-09-10T11:29:44.046 INFO:tasks.workunit.client.1.smithi178.stdout:7/961: creat d6/d10/d1d/daa/d137/f156 x:0 0 0 2024-09-10T11:29:44.046 INFO:tasks.workunit.client.0.smithi144.stdout:7/694: fsync d5/dc/d12/f52 0 2024-09-10T11:29:44.047 INFO:tasks.workunit.client.0.smithi144.stdout:3/676: sync 2024-09-10T11:29:44.048 INFO:tasks.workunit.client.1.smithi178.stdout:7/962: stat d6/d10/d1d/daa/l13d 0 2024-09-10T11:29:44.048 INFO:tasks.workunit.client.0.smithi144.stdout:1/772: link d5/c4f d5/dd/d64/cfb 0 2024-09-10T11:29:44.050 INFO:tasks.workunit.client.0.smithi144.stdout:6/598: symlink d9/dd/d3e/d47/ld7 0 2024-09-10T11:29:44.050 INFO:tasks.workunit.client.0.smithi144.stdout:8/629: link d0/c9c d0/d1b/d5a/cd5 0 2024-09-10T11:29:44.051 INFO:tasks.workunit.client.0.smithi144.stdout:5/721: rmdir d4/d5/d18/d67/db0 39 2024-09-10T11:29:44.052 INFO:tasks.workunit.client.0.smithi144.stdout:9/654: dwrite d11/d36/db1/fc4 [0,4194304] 0 2024-09-10T11:29:44.053 INFO:tasks.workunit.client.0.smithi144.stdout:3/677: fdatasync d4/d28/d67/d80/d17/d18/d34/f5c 0 2024-09-10T11:29:44.053 INFO:tasks.workunit.client.0.smithi144.stdout:1/773: chown d5/df/d4b/d51/l60 2834067 1 2024-09-10T11:29:44.055 INFO:tasks.workunit.client.0.smithi144.stdout:9/655: write d11/d12/d1b/d22/f2f [968553,119220] 0 2024-09-10T11:29:44.058 INFO:tasks.workunit.client.0.smithi144.stdout:5/722: unlink d4/d5/d18/d67/d44/l66 0 2024-09-10T11:29:44.061 INFO:tasks.workunit.client.0.smithi144.stdout:7/695: mkdir d5/dc/d14/d40/d58/d6e/d7a/de0 0 2024-09-10T11:29:44.062 INFO:tasks.workunit.client.0.smithi144.stdout:3/678: rmdir d4/d28/d67/d80/d1d 39 2024-09-10T11:29:44.064 INFO:tasks.workunit.client.0.smithi144.stdout:9/656: symlink d11/d12/d1b/d41/d8f/dc0/dd0/lea 0 2024-09-10T11:29:44.064 INFO:tasks.workunit.client.0.smithi144.stdout:3/679: truncate d4/d28/d67/d9f/fb5 4536618 0 2024-09-10T11:29:44.065 INFO:tasks.workunit.client.0.smithi144.stdout:1/774: fdatasync d5/df/f33 0 2024-09-10T11:29:44.065 INFO:tasks.workunit.client.0.smithi144.stdout:6/599: symlink d9/dd/d11/d1b/d2f/d7f/dc4/d5b/ld8 0 2024-09-10T11:29:44.068 INFO:tasks.workunit.client.0.smithi144.stdout:7/696: mknod d5/dc/d14/d40/d58/d6e/ce1 0 2024-09-10T11:29:44.069 INFO:tasks.workunit.client.0.smithi144.stdout:1/775: write d5/d4a/f7d [277778,78332] 0 2024-09-10T11:29:44.070 INFO:tasks.workunit.client.0.smithi144.stdout:3/680: rename d4/d28/d67/lad to d4/d28/db6/ld8 0 2024-09-10T11:29:44.072 INFO:tasks.workunit.client.0.smithi144.stdout:9/657: symlink d11/d12/d1b/d41/d8f/leb 0 2024-09-10T11:29:44.073 INFO:tasks.workunit.client.0.smithi144.stdout:7/697: mkdir d5/dc/d11/de2 0 2024-09-10T11:29:44.076 INFO:tasks.workunit.client.0.smithi144.stdout:3/681: creat d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/fd9 x:0 0 0 2024-09-10T11:29:44.077 INFO:tasks.workunit.client.0.smithi144.stdout:6/600: dwrite d9/dd/f49 [4194304,4194304] 0 2024-09-10T11:29:44.080 INFO:tasks.workunit.client.0.smithi144.stdout:9/658: getdents d11/d36/d2d/d38 0 2024-09-10T11:29:44.081 INFO:tasks.workunit.client.0.smithi144.stdout:1/776: write d5/df/d4b/d85/ff2 [4804242,20] 0 2024-09-10T11:29:44.083 INFO:tasks.workunit.client.0.smithi144.stdout:5/723: dwrite d4/f55 [0,4194304] 0 2024-09-10T11:29:44.083 INFO:tasks.workunit.client.0.smithi144.stdout:1/777: chown d5/dd/d64/dad 4583 1 2024-09-10T11:29:44.086 INFO:tasks.workunit.client.0.smithi144.stdout:3/682: creat d4/d28/d67/d80/d17/d18/d34/d65/d75/fda x:0 0 0 2024-09-10T11:29:44.088 INFO:tasks.workunit.client.0.smithi144.stdout:6/601: symlink d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d60/ld9 0 2024-09-10T11:29:44.088 INFO:tasks.workunit.client.0.smithi144.stdout:9/659: mkdir d11/d12/d1b/d2c/d78/dd2/dec 0 2024-09-10T11:29:44.089 INFO:tasks.workunit.client.0.smithi144.stdout:7/698: dwrite d5/dc/d2a/d60/dd6/d98/dd1/f4d [4194304,4194304] 0 2024-09-10T11:29:44.095 INFO:tasks.workunit.client.0.smithi144.stdout:1/778: mknod d5/dd/d64/d79/dee/cfc 0 2024-09-10T11:29:44.100 INFO:tasks.workunit.client.0.smithi144.stdout:3/683: getdents d4/d28/d67/d80/d17/dd0/d98/d9a 0 2024-09-10T11:29:44.100 INFO:tasks.workunit.client.0.smithi144.stdout:1/779: symlink d5/d4a/d63/d84/d95/dd2/lfd 0 2024-09-10T11:29:44.103 INFO:tasks.workunit.client.0.smithi144.stdout:1/780: stat d5/ca6 0 2024-09-10T11:29:44.104 INFO:tasks.workunit.client.0.smithi144.stdout:5/724: dwrite d4/d5/d18/d38/d6d/d8b/f9b [0,4194304] 0 2024-09-10T11:29:44.105 INFO:tasks.workunit.client.0.smithi144.stdout:1/781: fdatasync d5/dd/d64/d79/fb3 0 2024-09-10T11:29:44.107 INFO:tasks.workunit.client.0.smithi144.stdout:3/684: unlink d4/d28/d67/d80/d17/d18/d34/fbc 0 2024-09-10T11:29:44.107 INFO:tasks.workunit.client.0.smithi144.stdout:5/725: read d4/d5/f28 [2262264,7312] 0 2024-09-10T11:29:44.109 INFO:tasks.workunit.client.0.smithi144.stdout:5/726: chown d4/f55 42465728 1 2024-09-10T11:29:44.109 INFO:tasks.workunit.client.0.smithi144.stdout:9/660: dread d11/d36/f4e [0,4194304] 0 2024-09-10T11:29:44.110 INFO:tasks.workunit.client.0.smithi144.stdout:5/727: chown d4/d20/d24/d2d/l3a 2 1 2024-09-10T11:29:44.112 INFO:tasks.workunit.client.0.smithi144.stdout:3/685: creat d4/d28/d67/d80/d17/dd0/d98/d9a/fdb x:0 0 0 2024-09-10T11:29:44.116 INFO:tasks.workunit.client.0.smithi144.stdout:6/602: write d9/d1f/d2c/f53 [1316731,86175] 0 2024-09-10T11:29:44.118 INFO:tasks.workunit.client.0.smithi144.stdout:3/686: mknod d4/d28/d67/d9f/cdc 0 2024-09-10T11:29:44.120 INFO:tasks.workunit.client.0.smithi144.stdout:9/661: dwrite d11/d12/d1b/f95 [0,4194304] 0 2024-09-10T11:29:44.120 INFO:tasks.workunit.client.0.smithi144.stdout:5/728: rename d4/d5/d32/daa/fb5 to d4/d20/d22/d75/ddd/fe3 0 2024-09-10T11:29:44.127 INFO:tasks.workunit.client.0.smithi144.stdout:5/729: unlink d4/d52/c81 0 2024-09-10T11:29:44.127 INFO:tasks.workunit.client.0.smithi144.stdout:7/699: dwrite d5/dc/d2a/d60/dd6/d98/dd1/f47 [0,4194304] 0 2024-09-10T11:29:44.128 INFO:tasks.workunit.client.0.smithi144.stdout:1/782: dread d5/f7 [0,4194304] 0 2024-09-10T11:29:44.132 INFO:tasks.workunit.client.0.smithi144.stdout:9/662: symlink d11/d12/d1b/d2c/d78/led 0 2024-09-10T11:29:44.135 INFO:tasks.workunit.client.0.smithi144.stdout:6/603: dwrite d9/dd/d11/f90 [0,4194304] 0 2024-09-10T11:29:44.137 INFO:tasks.workunit.client.0.smithi144.stdout:9/663: stat d11/d12/d1b/d41/d8f/f9d 0 2024-09-10T11:29:44.137 INFO:tasks.workunit.client.0.smithi144.stdout:7/700: write d5/dc/d14/d40/d5e/d68/fc0 [400520,90131] 0 2024-09-10T11:29:44.139 INFO:tasks.workunit.client.0.smithi144.stdout:9/664: fdatasync d11/d12/d1b/d41/d8f/fb4 0 2024-09-10T11:29:44.139 INFO:tasks.workunit.client.0.smithi144.stdout:5/730: symlink d4/d20/d22/le4 0 2024-09-10T11:29:44.143 INFO:tasks.workunit.client.0.smithi144.stdout:6/604: creat d9/dd/d11/d1b/d2f/dab/dc6/fda x:0 0 0 2024-09-10T11:29:44.144 INFO:tasks.workunit.client.0.smithi144.stdout:9/665: fsync d11/d12/d1b/d41/d8f/fb4 0 2024-09-10T11:29:44.146 INFO:tasks.workunit.client.0.smithi144.stdout:6/605: write d9/dd/d11/d1b/d2f/dab/dc6/fda [301104,125123] 0 2024-09-10T11:29:44.148 INFO:tasks.workunit.client.0.smithi144.stdout:9/666: rename d11/d12/d1b/d22 to d11/d12/d1b/d41/d8f/dc0/dd0/dee 0 2024-09-10T11:29:44.150 INFO:tasks.workunit.client.0.smithi144.stdout:9/667: chown d11/d12/d1b/d41/d8f/dab/cb0 6 1 2024-09-10T11:29:44.151 INFO:tasks.workunit.client.0.smithi144.stdout:6/606: dread d9/dd/d3e/d47/f9f [0,4194304] 0 2024-09-10T11:29:44.152 INFO:tasks.workunit.client.0.smithi144.stdout:9/668: rmdir d11/d36/db1/dbb 39 2024-09-10T11:29:44.152 INFO:tasks.workunit.client.0.smithi144.stdout:6/607: fsync d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/f31 0 2024-09-10T11:29:44.154 INFO:tasks.workunit.client.0.smithi144.stdout:6/608: dread - d9/dd/d3e/f9b zero size 2024-09-10T11:29:44.155 INFO:tasks.workunit.client.0.smithi144.stdout:9/669: symlink d11/d12/d6f/daa/lef 0 2024-09-10T11:29:44.156 INFO:tasks.workunit.client.0.smithi144.stdout:7/701: dread d5/dc/d14/d40/f9f [0,4194304] 0 2024-09-10T11:29:44.156 INFO:tasks.workunit.client.0.smithi144.stdout:7/702: fdatasync d5/f7 0 2024-09-10T11:29:44.157 INFO:tasks.workunit.client.0.smithi144.stdout:9/670: write d11/d12/d1b/d41/fd8 [471662,70305] 0 2024-09-10T11:29:44.159 INFO:tasks.workunit.client.0.smithi144.stdout:6/609: getdents d9/dd/d11/d1b/d2f/d7f/dc4/d5b/dc8 0 2024-09-10T11:29:44.160 INFO:tasks.workunit.client.0.smithi144.stdout:3/687: write d4/d28/d67/d80/d17/f81 [396377,30643] 0 2024-09-10T11:29:44.161 INFO:tasks.workunit.client.0.smithi144.stdout:9/671: write d11/d12/d1b/d41/d8f/dc0/dd0/dee/f7a [1430278,96003] 0 2024-09-10T11:29:44.162 INFO:tasks.workunit.client.0.smithi144.stdout:1/783: write d5/f41 [1757631,80543] 0 2024-09-10T11:29:44.162 INFO:tasks.workunit.client.0.smithi144.stdout:7/703: truncate d5/dc/d14/d40/fdc 872355 0 2024-09-10T11:29:44.163 INFO:tasks.workunit.client.0.smithi144.stdout:5/731: truncate d4/d20/d24/da4/fc3 3815953 0 2024-09-10T11:29:44.164 INFO:tasks.workunit.client.0.smithi144.stdout:1/784: readlink d5/d2a/lf4 0 2024-09-10T11:29:44.165 INFO:tasks.workunit.client.0.smithi144.stdout:9/672: chown d11/d12/d6f/f8b 7214481 1 2024-09-10T11:29:44.165 INFO:tasks.workunit.client.0.smithi144.stdout:6/610: chown d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d60/ld9 711 1 2024-09-10T11:29:44.166 INFO:tasks.workunit.client.0.smithi144.stdout:1/785: write d5/fdd [178514,93176] 0 2024-09-10T11:29:44.170 INFO:tasks.workunit.client.0.smithi144.stdout:6/611: chown d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d87 19 1 2024-09-10T11:29:44.172 INFO:tasks.workunit.client.0.smithi144.stdout:5/732: getdents d4/d20/d22/d5c/d5d/d64/dc2/dca 0 2024-09-10T11:29:44.172 INFO:tasks.workunit.client.0.smithi144.stdout:7/704: write d5/dc/f56 [10130,34647] 0 2024-09-10T11:29:44.173 INFO:tasks.workunit.client.0.smithi144.stdout:9/673: truncate d11/d12/d1b/d2c/d78/fdc 56463 0 2024-09-10T11:29:44.174 INFO:tasks.workunit.client.0.smithi144.stdout:5/733: readlink d4/d5/d18/l26 0 2024-09-10T11:29:44.176 INFO:tasks.workunit.client.0.smithi144.stdout:3/688: dwrite d4/d28/d67/d80/d17/d18/d34/f5c [4194304,4194304] 0 2024-09-10T11:29:44.178 INFO:tasks.workunit.client.0.smithi144.stdout:3/689: readlink d4/d28/d67/d96/dcc/ld4 0 2024-09-10T11:29:44.179 INFO:tasks.workunit.client.0.smithi144.stdout:7/705: creat d5/dc/d2a/d60/dd6/d98/fe3 x:0 0 0 2024-09-10T11:29:44.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.180+0000 7fb06fef7700 1 -- 172.21.15.144:0/3342900955 --> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb06810e7a0 con 0x7fb05006d450 2024-09-10T11:29:44.182 INFO:tasks.workunit.client.0.smithi144.stdout:9/674: rename d11/d12/d1b/d41 to d11/d36/db1/dbb/df0 0 2024-09-10T11:29:44.184 INFO:tasks.workunit.client.0.smithi144.stdout:6/612: creat d9/dd/d11/d1b/d2f/dab/dd0/fdb x:0 0 0 2024-09-10T11:29:44.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.183+0000 7fb066ffd700 1 -- 172.21.15.144:0/3342900955 <== mgr.24329 v2:172.21.15.178:6828/2974090415 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fb06810e7a0 con 0x7fb05006d450 2024-09-10T11:29:44.185 INFO:tasks.workunit.client.0.smithi144.stdout:9/675: dread - d11/d36/db1/dbb/df0/fd9 zero size 2024-09-10T11:29:44.187 INFO:tasks.workunit.client.0.smithi144.stdout:7/706: creat d5/dc/fe4 x:0 0 0 2024-09-10T11:29:44.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.187+0000 7fb06fef7700 1 -- 172.21.15.144:0/3342900955 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7fb05006d450 msgr2=0x7fb05006f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:44.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.187+0000 7fb06fef7700 1 --2- 172.21.15.144:0/3342900955 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7fb05006d450 0x7fb05006f900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb0580060b0 tx=0x7fb058006020 comp rx=0 tx=0).stop 2024-09-10T11:29:44.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.187+0000 7fb06fef7700 1 -- 172.21.15.144:0/3342900955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb068072500 msgr2=0x7fb06819e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:44.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.187+0000 7fb06fef7700 1 --2- 172.21.15.144:0/3342900955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb068072500 0x7fb06819e6f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fb05c00e940 tx=0x7fb05c00ec50 comp rx=0 tx=0).stop 2024-09-10T11:29:44.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.188+0000 7fb06fef7700 1 -- 172.21.15.144:0/3342900955 shutdown_connections 2024-09-10T11:29:44.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.188+0000 7fb06fef7700 1 --2- 172.21.15.144:0/3342900955 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb068071b40 0x7fb06819e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:44.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.188+0000 7fb06fef7700 1 --2- 172.21.15.144:0/3342900955 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7fb05006d450 0x7fb05006f900 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:44.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.188+0000 7fb06fef7700 1 --2- 172.21.15.144:0/3342900955 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb068072500 0x7fb06819e6f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:44.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.188+0000 7fb06fef7700 1 -- 172.21.15.144:0/3342900955 >> 172.21.15.144:0/3342900955 conn(0x7fb06806d050 msgr2=0x7fb06810d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:44.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.188+0000 7fb06fef7700 1 -- 172.21.15.144:0/3342900955 shutdown_connections 2024-09-10T11:29:44.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.188+0000 7fb06fef7700 1 -- 172.21.15.144:0/3342900955 wait complete. 2024-09-10T11:29:44.192 INFO:tasks.workunit.client.0.smithi144.stdout:6/613: getdents d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d60 0 2024-09-10T11:29:44.194 INFO:tasks.workunit.client.0.smithi144.stdout:7/707: symlink d5/dc/d11/de2/le5 0 2024-09-10T11:29:44.197 INFO:tasks.workunit.client.0.smithi144.stdout:6/614: mknod d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d60/cdc 0 2024-09-10T11:29:44.198 INFO:tasks.workunit.client.0.smithi144.stdout:5/734: dread d4/d20/d24/f63 [0,4194304] 0 2024-09-10T11:29:44.200 INFO:tasks.workunit.client.0.smithi144.stdout:1/786: dread d5/d4a/d44/f45 [0,4194304] 0 2024-09-10T11:29:44.201 INFO:tasks.workunit.client.0.smithi144.stdout:9/676: truncate d11/d36/d2d/d38/f76 7045070 0 2024-09-10T11:29:44.202 INFO:tasks.workunit.client.0.smithi144.stdout:3/690: dread d4/d28/f87 [0,4194304] 0 2024-09-10T11:29:44.203 INFO:tasks.workunit.client.0.smithi144.stdout:6/615: read d9/d1f/f37 [42991,85827] 0 2024-09-10T11:29:44.205 INFO:tasks.workunit.client.0.smithi144.stdout:9/677: creat d11/d12/d6f/daa/ff1 x:0 0 0 2024-09-10T11:29:44.206 INFO:tasks.workunit.client.0.smithi144.stdout:3/691: readlink d4/d28/d67/d80/d17/dd0/d98/la5 0 2024-09-10T11:29:44.209 INFO:tasks.workunit.client.0.smithi144.stdout:7/708: dwrite d5/dc/d14/fb3 [0,4194304] 0 2024-09-10T11:29:44.209 INFO:tasks.workunit.client.1.smithi178.stdout:4/956: sync 2024-09-10T11:29:44.213 INFO:tasks.workunit.client.1.smithi178.stdout:4/957: truncate d0/d9/d3d/f44 904201 0 2024-09-10T11:29:44.213 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:29:44.219 INFO:tasks.workunit.client.0.smithi144.stdout:3/692: write d4/d1a/f44 [4076752,95614] 0 2024-09-10T11:29:44.219 INFO:tasks.workunit.client.0.smithi144.stdout:6/616: dread d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/f6f [0,4194304] 0 2024-09-10T11:29:44.219 INFO:tasks.workunit.client.1.smithi178.stdout:4/958: rename d0/d9/d3d/dc2/fce to d0/d8/da/f155 0 2024-09-10T11:29:44.220 INFO:tasks.workunit.client.0.smithi144.stdout:9/678: creat d11/d12/d1b/d2c/d78/dd2/ff2 x:0 0 0 2024-09-10T11:29:44.224 INFO:tasks.workunit.client.0.smithi144.stdout:5/735: dread d4/d20/d22/d5c/d5d/d64/f96 [0,4194304] 0 2024-09-10T11:29:44.227 INFO:tasks.workunit.client.0.smithi144.stdout:1/787: dwrite d5/df/d4b/d85/f97 [0,4194304] 0 2024-09-10T11:29:44.229 INFO:tasks.workunit.client.0.smithi144.stdout:1/788: write d5/d4a/d63/fac [2589858,42031] 0 2024-09-10T11:29:44.230 INFO:tasks.workunit.client.1.smithi178.stdout:4/959: write d0/d9/d3d/d4f/ddf/f14e [1229416,10233] 0 2024-09-10T11:29:44.230 INFO:tasks.workunit.client.0.smithi144.stdout:1/789: write d5/d4a/d63/d84/d95/dd2/fde [90712,57698] 0 2024-09-10T11:29:44.236 INFO:tasks.workunit.client.1.smithi178.stdout:4/960: link d0/d9/dc/d8e/dbf/l98 d0/d8/da/dae/l156 0 2024-09-10T11:29:44.238 INFO:tasks.workunit.client.0.smithi144.stdout:3/693: write d4/dd/d21/f23 [2896249,121122] 0 2024-09-10T11:29:44.238 INFO:tasks.workunit.client.0.smithi144.stdout:7/709: truncate d5/dc/d12/f52 163933 0 2024-09-10T11:29:44.240 INFO:tasks.workunit.client.0.smithi144.stdout:3/694: write d4/f32 [1025422,124485] 0 2024-09-10T11:29:44.240 INFO:tasks.workunit.client.0.smithi144.stdout:6/617: mkdir d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d87/dc9/ddd 0 2024-09-10T11:29:44.241 INFO:tasks.workunit.client.0.smithi144.stdout:5/736: creat d4/d20/d22/d5c/d5d/fe5 x:0 0 0 2024-09-10T11:29:44.243 INFO:tasks.workunit.client.1.smithi178.stdout:4/961: rename d0/d8/df/d13b/d5d/db6 to d0/d63/d157 0 2024-09-10T11:29:44.245 INFO:tasks.workunit.client.0.smithi144.stdout:9/679: mknod d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/cf3 0 2024-09-10T11:29:44.246 INFO:tasks.workunit.client.0.smithi144.stdout:7/710: mknod d5/dc/d2a/d60/dd6/d98/dd1/d7c/ce6 0 2024-09-10T11:29:44.246 INFO:tasks.workunit.client.1.smithi178.stdout:4/962: creat d0/d8/da/d1a/f158 x:0 0 0 2024-09-10T11:29:44.248 INFO:tasks.workunit.client.1.smithi178.stdout:4/963: truncate d0/d10/f5a 1273355 0 2024-09-10T11:29:44.248 INFO:tasks.workunit.client.0.smithi144.stdout:1/790: rename d5/dd/d64/c78 to d5/df/d4b/d85/cfe 0 2024-09-10T11:29:44.250 INFO:tasks.workunit.client.0.smithi144.stdout:6/618: dwrite d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/f6e [0,4194304] 0 2024-09-10T11:29:44.250 INFO:tasks.workunit.client.1.smithi178.stdout:4/964: readlink d0/d9/dc/d8a/la0 0 2024-09-10T11:29:44.251 INFO:tasks.workunit.client.0.smithi144.stdout:7/711: symlink d5/dc/d2a/d60/dd6/d98/le7 0 2024-09-10T11:29:44.253 INFO:tasks.workunit.client.0.smithi144.stdout:7/712: write d5/dc/d11/d18/f9a [168568,90698] 0 2024-09-10T11:29:44.253 INFO:tasks.workunit.client.0.smithi144.stdout:9/680: truncate d11/d12/f26 127967 0 2024-09-10T11:29:44.255 INFO:tasks.workunit.client.1.smithi178.stdout:4/965: creat d0/d9/dc/d8e/f159 x:0 0 0 2024-09-10T11:29:44.255 INFO:tasks.workunit.client.0.smithi144.stdout:1/791: mkdir d5/d4a/d63/d84/dbd/dff 0 2024-09-10T11:29:44.255 INFO:tasks.workunit.client.0.smithi144.stdout:7/713: truncate d5/dc/d14/d40/d58/d6a/fbe 374103 0 2024-09-10T11:29:44.256 INFO:tasks.workunit.client.1.smithi178.stdout:4/966: write d0/f6e [3667168,76281] 0 2024-09-10T11:29:44.256 INFO:tasks.workunit.client.0.smithi144.stdout:6/619: truncate d9/dd/d11/f4c 1053412 0 2024-09-10T11:29:44.258 INFO:tasks.workunit.client.0.smithi144.stdout:6/620: fsync d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d60/fd4 0 2024-09-10T11:29:44.260 INFO:tasks.workunit.client.0.smithi144.stdout:3/695: rename d4/d28/db6/ld8 to d4/d28/d67/d9f/ldd 0 2024-09-10T11:29:44.299 INFO:tasks.workunit.client.0.smithi144.stdout:2/553: sync 2024-09-10T11:29:44.333 INFO:tasks.workunit.client.0.smithi144.stdout:5/737: rename d4/d20/d22/d5c/d5d/d64/dc2/dca to d4/d20/d22/d5c/d5d/de6 0 2024-09-10T11:29:44.333 INFO:tasks.workunit.client.0.smithi144.stdout:7/714: mknod d5/dc/d14/d40/d58/d6e/d7a/de0/ce8 0 2024-09-10T11:29:44.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.334+0000 7f8128878700 1 -- 172.21.15.144:0/1149658097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8124100190 msgr2=0x7f81241005a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:44.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.334+0000 7f8128878700 1 --2- 172.21.15.144:0/1149658097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8124100190 0x7f81241005a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8118009910 tx=0x7f8118009c20 comp rx=0 tx=0).stop 2024-09-10T11:29:44.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.334+0000 7f8128878700 1 -- 172.21.15.144:0/1149658097 shutdown_connections 2024-09-10T11:29:44.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.334+0000 7f8128878700 1 --2- 172.21.15.144:0/1149658097 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8124101390 0x7f81241017e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:44.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.334+0000 7f8128878700 1 --2- 172.21.15.144:0/1149658097 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8124100190 0x7f81241005a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:44.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.334+0000 7f8128878700 1 -- 172.21.15.144:0/1149658097 >> 172.21.15.144:0/1149658097 conn(0x7f81240fb720 msgr2=0x7f81240fdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:44.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.334+0000 7f8128878700 1 -- 172.21.15.144:0/1149658097 shutdown_connections 2024-09-10T11:29:44.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.334+0000 7f8128878700 1 -- 172.21.15.144:0/1149658097 wait complete. 2024-09-10T11:29:44.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.335+0000 7f8128878700 1 Processor -- start 2024-09-10T11:29:44.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.335+0000 7f8128878700 1 -- start start 2024-09-10T11:29:44.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.335+0000 7f8128878700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8124100190 0x7f8124074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:44.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.335+0000 7f8128878700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8124101390 0x7f8124073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:44.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.335+0000 7f8128878700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8124073680 con 0x7f8124101390 2024-09-10T11:29:44.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.335+0000 7f8128878700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81240737c0 con 0x7f8124100190 2024-09-10T11:29:44.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.335+0000 7f8121d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8124101390 0x7f8124073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:44.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.335+0000 7f8121d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8124101390 0x7f8124073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44616/0 (socket says 172.21.15.144:44616) 2024-09-10T11:29:44.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.335+0000 7f8121d9b700 1 -- 172.21.15.144:0/615475523 learned_addr learned my addr 172.21.15.144:0/615475523 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:29:44.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.335+0000 7f812259c700 1 --2- 172.21.15.144:0/615475523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8124100190 0x7f8124074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:44.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.335+0000 7f8121d9b700 1 -- 172.21.15.144:0/615475523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8124100190 msgr2=0x7f8124074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:44.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.335+0000 7f8121d9b700 1 --2- 172.21.15.144:0/615475523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8124100190 0x7f8124074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:44.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.335+0000 7f8121d9b700 1 -- 172.21.15.144:0/615475523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81180095f0 con 0x7f8124101390 2024-09-10T11:29:44.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.335+0000 7f812259c700 1 --2- 172.21.15.144:0/615475523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8124100190 0x7f8124074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:29:44.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.336+0000 7f8121d9b700 1 --2- 172.21.15.144:0/615475523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8124101390 0x7f8124073140 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f811400e920 tx=0x7f811400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:44.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.336+0000 7f81137fe700 1 -- 172.21.15.144:0/615475523 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8114009790 con 0x7f8124101390 2024-09-10T11:29:44.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.336+0000 7f8128878700 1 -- 172.21.15.144:0/615475523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8124073aa0 con 0x7f8124101390 2024-09-10T11:29:44.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.336+0000 7f81137fe700 1 -- 172.21.15.144:0/615475523 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f811400cba0 con 0x7f8124101390 2024-09-10T11:29:44.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.336+0000 7f81137fe700 1 -- 172.21.15.144:0/615475523 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8114018630 con 0x7f8124101390 2024-09-10T11:29:44.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.336+0000 7f8128878700 1 -- 172.21.15.144:0/615475523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812419d9f0 con 0x7f8124101390 2024-09-10T11:29:44.339 INFO:tasks.workunit.client.0.smithi144.stdout:3/696: unlink d4/d28/d67/d80/d1d/f29 0 2024-09-10T11:29:44.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.338+0000 7f8128878700 1 -- 172.21.15.144:0/615475523 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8124108ce0 con 0x7f8124101390 2024-09-10T11:29:44.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.338+0000 7f81137fe700 1 -- 172.21.15.144:0/615475523 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) v1 ==== 91089+0+0 (secure 0 0 0) 0x7f81140188f0 con 0x7f8124101390 2024-09-10T11:29:44.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.338+0000 7f81137fe700 1 --2- 172.21.15.144:0/615475523 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f810c06d3b0 0x7f810c06f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:44.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.338+0000 7f81137fe700 1 -- 172.21.15.144:0/615475523 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8114014070 con 0x7f8124101390 2024-09-10T11:29:44.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.338+0000 7f812259c700 1 --2- 172.21.15.144:0/615475523 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f810c06d3b0 0x7f810c06f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:44.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.339+0000 7f812259c700 1 --2- 172.21.15.144:0/615475523 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f810c06d3b0 0x7f810c06f860 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8118009fd0 tx=0x7f8118005fb0 comp rx=0 tx=0).ready entity=mgr.24329 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:44.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.343+0000 7f81137fe700 1 -- 172.21.15.144:0/615475523 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f81140539d0 con 0x7f8124101390 2024-09-10T11:29:44.379 INFO:tasks.workunit.client.1.smithi178.stdout:7/963: sync 2024-09-10T11:29:44.381 INFO:tasks.workunit.client.1.smithi178.stdout:7/964: symlink d6/d10/d1d/d22/d93/l157 0 2024-09-10T11:29:44.383 INFO:tasks.workunit.client.1.smithi178.stdout:7/965: unlink d6/d10/f5c 0 2024-09-10T11:29:44.386 INFO:tasks.workunit.client.1.smithi178.stdout:7/966: creat d6/d10/d1d/d22/d50/def/f158 x:0 0 0 2024-09-10T11:29:44.390 INFO:tasks.workunit.client.1.smithi178.stdout:7/967: dread d6/d10/d1d/d4a/d79/df1/f8c [0,4194304] 0 2024-09-10T11:29:44.391 INFO:tasks.workunit.client.1.smithi178.stdout:7/968: chown d6/d10/d1d/d22/d50/d7e/d82/db2/l47 29 1 2024-09-10T11:29:44.396 INFO:tasks.workunit.client.1.smithi178.stdout:7/969: dwrite d6/d10/d1d/d22/fbb [0,4194304] 0 2024-09-10T11:29:44.401 INFO:tasks.workunit.client.1.smithi178.stdout:7/970: link d6/d10/d1d/d22/d50/d7e/d82/db2/l47 d6/d10/d1d/d22/d50/d98/d13f/l159 0 2024-09-10T11:29:44.402 INFO:tasks.workunit.client.1.smithi178.stdout:7/971: write d6/d10/d1d/d22/fbb [1880695,85007] 0 2024-09-10T11:29:44.402 INFO:tasks.workunit.client.1.smithi178.stdout:4/967: sync 2024-09-10T11:29:44.407 INFO:tasks.workunit.client.1.smithi178.stdout:4/968: symlink d0/d8/da/d1a/dff/d12f/l15a 0 2024-09-10T11:29:44.407 INFO:tasks.workunit.client.1.smithi178.stdout:7/972: dread d6/d10/d1e/d2d/d2b/da0/d55/fb0 [0,4194304] 0 2024-09-10T11:29:44.408 INFO:tasks.workunit.client.1.smithi178.stdout:4/969: symlink d0/d8/d3e/d123/l15b 0 2024-09-10T11:29:44.425 INFO:tasks.workunit.client.0.smithi144.stdout:0/512: sync 2024-09-10T11:29:44.425 INFO:tasks.workunit.client.0.smithi144.stdout:4/725: sync 2024-09-10T11:29:44.426 INFO:tasks.workunit.client.1.smithi178.stdout:4/970: creat d0/d8/df/d13b/f15c x:0 0 0 2024-09-10T11:29:44.426 INFO:tasks.workunit.client.0.smithi144.stdout:5/738: chown d4/d5/d32/c49 267686 1 2024-09-10T11:29:44.427 INFO:tasks.workunit.client.0.smithi144.stdout:6/621: dwrite d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/f51 [0,4194304] 0 2024-09-10T11:29:44.428 INFO:tasks.workunit.client.0.smithi144.stdout:7/715: symlink d5/dc/d2a/le9 0 2024-09-10T11:29:44.429 INFO:tasks.workunit.client.0.smithi144.stdout:4/726: dread - d14/d6f/f88 zero size 2024-09-10T11:29:44.430 INFO:tasks.workunit.client.0.smithi144.stdout:2/554: rename d7/d29/d97/cb2 to d7/d42/d64/d8d/cb9 0 2024-09-10T11:29:44.432 INFO:tasks.workunit.client.0.smithi144.stdout:3/697: creat d4/d28/d67/d96/fde x:0 0 0 2024-09-10T11:29:44.432 INFO:tasks.workunit.client.0.smithi144.stdout:0/513: creat da/d3a/d51/f9f x:0 0 0 2024-09-10T11:29:44.432 INFO:tasks.workunit.client.0.smithi144.stdout:7/716: mknod d5/dc/d11/d18/cea 0 2024-09-10T11:29:44.433 INFO:tasks.workunit.client.1.smithi178.stdout:7/973: dwrite d6/d10/d1d/d4a/d79/d83/fcd [0,4194304] 0 2024-09-10T11:29:44.433 INFO:tasks.workunit.client.0.smithi144.stdout:2/555: dread - d7/de/d3d/d43/fb0 zero size 2024-09-10T11:29:44.434 INFO:tasks.workunit.client.0.smithi144.stdout:6/622: truncate d9/d1f/d2c/d4f/f7d 789593 0 2024-09-10T11:29:44.434 INFO:tasks.workunit.client.1.smithi178.stdout:4/971: dwrite d0/d9/dc/d8e/f159 [0,4194304] 0 2024-09-10T11:29:44.437 INFO:tasks.workunit.client.0.smithi144.stdout:4/727: symlink d14/d1e/d22/d34/l108 0 2024-09-10T11:29:44.439 INFO:tasks.workunit.client.0.smithi144.stdout:5/739: rename d4/d5/d18/d38/c4d to d4/d5/d18/d67/ce7 0 2024-09-10T11:29:44.440 INFO:tasks.workunit.client.0.smithi144.stdout:5/740: read - d4/d20/d24/fc8 zero size 2024-09-10T11:29:44.441 INFO:tasks.workunit.client.1.smithi178.stdout:7/974: truncate d6/d10/d1d/d4a/d79/df1/fa9 1108502 0 2024-09-10T11:29:44.442 INFO:tasks.workunit.client.0.smithi144.stdout:6/623: readlink d9/d1f/d2c/lac 0 2024-09-10T11:29:44.444 INFO:tasks.workunit.client.0.smithi144.stdout:0/514: mknod da/d3a/d51/ca0 0 2024-09-10T11:29:44.444 INFO:tasks.workunit.client.0.smithi144.stdout:5/741: chown d4/d5/d18/d38/c42 180 1 2024-09-10T11:29:44.444 INFO:tasks.workunit.client.0.smithi144.stdout:3/698: dwrite d4/dd/d21/f5b [0,4194304] 0 2024-09-10T11:29:44.446 INFO:tasks.workunit.client.0.smithi144.stdout:3/699: write d4/f58 [84138,65051] 0 2024-09-10T11:29:44.448 INFO:tasks.workunit.client.1.smithi178.stdout:7/975: rename d6/d10/d1d/d22/d50/d98/ddc/d148 to d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d70/da2/d15a 0 2024-09-10T11:29:44.449 INFO:tasks.workunit.client.0.smithi144.stdout:3/700: write d4/d28/f36 [3832363,75033] 0 2024-09-10T11:29:44.450 INFO:tasks.workunit.client.1.smithi178.stdout:7/976: stat d6/d10/d1d/d22/d50/d98/da6/f103 0 2024-09-10T11:29:44.452 INFO:tasks.workunit.client.0.smithi144.stdout:5/742: rename d4/d5/d18/l9d to d4/d5/d18/d51/le8 0 2024-09-10T11:29:44.454 INFO:tasks.workunit.client.1.smithi178.stdout:7/977: rmdir d6/d10/d1d/d22/d50/def/df3 39 2024-09-10T11:29:44.455 INFO:tasks.workunit.client.0.smithi144.stdout:4/728: dwrite d14/d1e/d22/f23 [0,4194304] 0 2024-09-10T11:29:44.455 INFO:tasks.workunit.client.0.smithi144.stdout:2/556: dread d7/f41 [0,4194304] 0 2024-09-10T11:29:44.456 INFO:tasks.workunit.client.1.smithi178.stdout:4/972: dwrite d0/d8/df/f59 [0,4194304] 0 2024-09-10T11:29:44.458 INFO:tasks.workunit.client.1.smithi178.stdout:4/973: rmdir d0/d9d 39 2024-09-10T11:29:44.460 INFO:tasks.workunit.client.0.smithi144.stdout:6/624: rmdir d9/dd/d3e/d83 0 2024-09-10T11:29:44.461 INFO:tasks.workunit.client.0.smithi144.stdout:6/625: stat d9/dd/d11/d1b/d2f/dab/fce 0 2024-09-10T11:29:44.463 INFO:tasks.workunit.client.0.smithi144.stdout:3/701: unlink d4/d28/d67/d80/d17/d18/d34/d65/f88 0 2024-09-10T11:29:44.464 INFO:tasks.workunit.client.1.smithi178.stdout:4/974: write d0/d8/da/f128 [326573,1923] 0 2024-09-10T11:29:44.466 INFO:tasks.workunit.client.1.smithi178.stdout:7/978: rename d6/d10/d1d/d22/d50/d98/da6/dfd/f119 to d6/d10/d1e/d2d/d2b/f15b 0 2024-09-10T11:29:44.466 INFO:tasks.workunit.client.1.smithi178.stdout:4/975: write d0/d8/df/d13b/f127 [1005286,46106] 0 2024-09-10T11:29:44.467 INFO:tasks.workunit.client.1.smithi178.stdout:4/976: stat d0/d9/dc/l12e 0 2024-09-10T11:29:44.469 INFO:tasks.workunit.client.1.smithi178.stdout:4/977: fdatasync d0/d8/da/d1a/f4b 0 2024-09-10T11:29:44.470 INFO:tasks.workunit.client.0.smithi144.stdout:7/717: write d5/f8 [1922742,95688] 0 2024-09-10T11:29:44.472 INFO:tasks.workunit.client.0.smithi144.stdout:7/718: chown d5/dc/d2a/le9 58354 1 2024-09-10T11:29:44.473 INFO:tasks.workunit.client.0.smithi144.stdout:4/729: rename d14/d67/d69/d90/f9d to d14/d7e/f109 0 2024-09-10T11:29:44.474 INFO:tasks.workunit.client.1.smithi178.stdout:7/979: link d6/d10/d1e/d2d/c19 d6/d10/d1d/daa/d137/c15c 0 2024-09-10T11:29:44.476 INFO:tasks.workunit.client.1.smithi178.stdout:7/980: creat d6/d10/d1d/daa/d11c/f15d x:0 0 0 2024-09-10T11:29:44.476 INFO:tasks.workunit.client.0.smithi144.stdout:6/626: mkdir d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/dde 0 2024-09-10T11:29:44.477 INFO:tasks.workunit.client.1.smithi178.stdout:7/981: dread - d6/d10/d1d/d22/f121 zero size 2024-09-10T11:29:44.477 INFO:tasks.workunit.client.0.smithi144.stdout:6/627: chown d9/f2a 31624056 1 2024-09-10T11:29:44.480 INFO:tasks.workunit.client.1.smithi178.stdout:7/982: rename d6/d10/d34/d6a/f108 to d6/d10/d34/d14b/f15e 0 2024-09-10T11:29:44.482 INFO:tasks.workunit.client.0.smithi144.stdout:0/515: dread da/f1c [0,4194304] 0 2024-09-10T11:29:44.484 INFO:tasks.workunit.client.0.smithi144.stdout:0/516: truncate da/d3a/d51/d5f/d83/f9c 50138 0 2024-09-10T11:29:44.485 INFO:tasks.workunit.client.0.smithi144.stdout:3/702: rmdir d4/d28/d67/d80/d17/d18/d34 39 2024-09-10T11:29:44.486 INFO:tasks.workunit.client.0.smithi144.stdout:5/743: truncate d4/d5/d18/d38/d8d/fa8 1402559 0 2024-09-10T11:29:44.486 INFO:tasks.workunit.client.1.smithi178.stdout:7/983: write d6/d10/d1d/d22/d93/db7/fec [962633,20965] 0 2024-09-10T11:29:44.488 INFO:tasks.workunit.client.1.smithi178.stdout:4/978: dwrite d0/d10/f5a [0,4194304] 0 2024-09-10T11:29:44.492 INFO:tasks.workunit.client.1.smithi178.stdout:7/984: mkdir d6/d10/d1d/d22/d50/d7e/d15f 0 2024-09-10T11:29:44.495 INFO:tasks.workunit.client.1.smithi178.stdout:7/985: truncate d6/d10/d1d/d22/d50/def/f158 894867 0 2024-09-10T11:29:44.495 INFO:tasks.workunit.client.0.smithi144.stdout:0/517: creat da/d3a/d4b/d61/d6c/fa1 x:0 0 0 2024-09-10T11:29:44.495 INFO:tasks.workunit.client.0.smithi144.stdout:2/557: write d7/de/f3b [1176250,27520] 0 2024-09-10T11:29:44.495 INFO:tasks.workunit.client.1.smithi178.stdout:4/979: creat d0/d9/dc/f15d x:0 0 0 2024-09-10T11:29:44.496 INFO:tasks.workunit.client.0.smithi144.stdout:0/518: write f0 [1802105,83053] 0 2024-09-10T11:29:44.497 INFO:tasks.workunit.client.1.smithi178.stdout:7/986: mkdir d6/d10/d12e/d160 0 2024-09-10T11:29:44.497 INFO:tasks.workunit.client.0.smithi144.stdout:4/730: write d14/d1e/d22/f2d [849004,16495] 0 2024-09-10T11:29:44.498 INFO:tasks.workunit.client.0.smithi144.stdout:6/628: write d9/dd/d11/f1c [1815483,6217] 0 2024-09-10T11:29:44.498 INFO:tasks.workunit.client.0.smithi144.stdout:3/703: chown d4/d28/d67/d80/d17/d18/d34/d65/d75/fda 2851902 1 2024-09-10T11:29:44.498 INFO:tasks.workunit.client.1.smithi178.stdout:4/980: creat d0/d8/da/d1a/dff/d12f/f15e x:0 0 0 2024-09-10T11:29:44.499 INFO:tasks.workunit.client.1.smithi178.stdout:4/981: readlink d0/d10/l35 0 2024-09-10T11:29:44.500 INFO:tasks.workunit.client.0.smithi144.stdout:2/558: dread - d7/de/d3d/d4e/f87 zero size 2024-09-10T11:29:44.500 INFO:tasks.workunit.client.0.smithi144.stdout:0/519: creat da/d44/d82/fa2 x:0 0 0 2024-09-10T11:29:44.500 INFO:tasks.workunit.client.1.smithi178.stdout:7/987: mknod d6/d10/d1d/d22/d50/d7e/d82/db2/c161 0 2024-09-10T11:29:44.500 INFO:tasks.workunit.client.1.smithi178.stdout:4/982: stat d0/d8/da/dae 0 2024-09-10T11:29:44.502 INFO:tasks.workunit.client.0.smithi144.stdout:2/559: fsync d7/de/d3d/d43/fb0 0 2024-09-10T11:29:44.504 INFO:tasks.workunit.client.1.smithi178.stdout:7/988: unlink d6/d10/d1d/f73 0 2024-09-10T11:29:44.504 INFO:tasks.workunit.client.0.smithi144.stdout:0/520: creat da/d3a/d4b/d96/fa3 x:0 0 0 2024-09-10T11:29:44.506 INFO:tasks.workunit.client.0.smithi144.stdout:2/560: stat d7/de/d36/d45/d8e/d9a/d54/c79 0 2024-09-10T11:29:44.506 INFO:tasks.workunit.client.0.smithi144.stdout:4/731: truncate d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/dd4/fd6 244276 0 2024-09-10T11:29:44.508 INFO:tasks.workunit.client.1.smithi178.stdout:7/989: rename d6/d10/d1d/d22/d50/def/f158 to d6/d10/f162 0 2024-09-10T11:29:44.509 INFO:tasks.workunit.client.0.smithi144.stdout:0/521: truncate da/d3a/d4b/d96/fa3 844320 0 2024-09-10T11:29:44.509 INFO:tasks.workunit.client.1.smithi178.stdout:4/983: dwrite d0/d8/df/f9e [0,4194304] 0 2024-09-10T11:29:44.510 INFO:tasks.workunit.client.0.smithi144.stdout:2/561: symlink d7/da/d5b/d80/lba 0 2024-09-10T11:29:44.511 INFO:tasks.workunit.client.1.smithi178.stdout:4/984: write d0/d8/da/d1a/dff/d133/f14a [1016443,97729] 0 2024-09-10T11:29:44.512 INFO:tasks.workunit.client.0.smithi144.stdout:4/732: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/f64 [855014,77719] 0 2024-09-10T11:29:44.515 INFO:tasks.workunit.client.0.smithi144.stdout:4/733: dread - d14/d67/ffb zero size 2024-09-10T11:29:44.515 INFO:tasks.workunit.client.0.smithi144.stdout:0/522: unlink da/f2a 0 2024-09-10T11:29:44.516 INFO:tasks.workunit.client.0.smithi144.stdout:6/629: dwrite d9/f81 [0,4194304] 0 2024-09-10T11:29:44.519 INFO:tasks.workunit.client.1.smithi178.stdout:4/985: dwrite d0/d8/d3e/f4e [0,4194304] 0 2024-09-10T11:29:44.519 INFO:tasks.workunit.client.0.smithi144.stdout:3/704: dread d4/d1a/f5f [0,4194304] 0 2024-09-10T11:29:44.520 INFO:tasks.workunit.client.0.smithi144.stdout:0/523: chown da/d3a/d4b/d61/d6c/f9b 2960543 1 2024-09-10T11:29:44.520 INFO:tasks.workunit.client.0.smithi144.stdout:4/734: write d14/d1e/fb6 [51538,33824] 0 2024-09-10T11:29:44.524 INFO:tasks.workunit.client.0.smithi144.stdout:4/735: write d14/d1e/d22/d34/d3e/d40/fbb [156927,111754] 0 2024-09-10T11:29:44.527 INFO:tasks.workunit.client.1.smithi178.stdout:4/986: fdatasync d0/d8/df/f49 0 2024-09-10T11:29:44.529 INFO:tasks.workunit.client.0.smithi144.stdout:0/524: truncate da/d3a/d51/d5f/f4a 315082 0 2024-09-10T11:29:44.530 INFO:tasks.workunit.client.1.smithi178.stdout:4/987: creat d0/d9/d3d/d4f/ddb/f15f x:0 0 0 2024-09-10T11:29:44.530 INFO:tasks.workunit.client.0.smithi144.stdout:2/562: dread d7/de/f37 [0,4194304] 0 2024-09-10T11:29:44.531 INFO:tasks.workunit.client.0.smithi144.stdout:3/705: truncate d4/d28/d67/d80/d17/d18/f6d 522923 0 2024-09-10T11:29:44.531 INFO:tasks.workunit.client.1.smithi178.stdout:4/988: write d0/d8/da/f90 [886037,98285] 0 2024-09-10T11:29:44.537 INFO:tasks.workunit.client.0.smithi144.stdout:8/630: sync 2024-09-10T11:29:44.538 INFO:tasks.workunit.client.0.smithi144.stdout:4/736: creat d14/d1e/d83/da4/da9/f10a x:0 0 0 2024-09-10T11:29:44.538 INFO:tasks.workunit.client.1.smithi178.stdout:4/989: rename d0/d9d/da5/c13e to d0/d8/da/c160 0 2024-09-10T11:29:44.539 INFO:tasks.workunit.client.0.smithi144.stdout:3/706: creat d4/d28/d67/fdf x:0 0 0 2024-09-10T11:29:44.542 INFO:tasks.workunit.client.0.smithi144.stdout:3/707: fsync d4/d28/d67/d80/fa2 0 2024-09-10T11:29:44.542 INFO:tasks.workunit.client.0.smithi144.stdout:8/631: mkdir d0/d9/d23/d26/dd6 0 2024-09-10T11:29:44.545 INFO:tasks.workunit.client.0.smithi144.stdout:8/632: fdatasync d0/d9/d1f/fc6 0 2024-09-10T11:29:44.545 INFO:tasks.workunit.client.0.smithi144.stdout:2/563: link d7/de/d3d/d4e/fa3 d7/d42/d64/d8d/fbb 0 2024-09-10T11:29:44.546 INFO:tasks.workunit.client.1.smithi178.stdout:4/990: dwrite d0/d8/da/d1a/f57 [0,4194304] 0 2024-09-10T11:29:44.549 INFO:tasks.workunit.client.0.smithi144.stdout:2/564: dread - d7/da/fb6 zero size 2024-09-10T11:29:44.550 INFO:tasks.workunit.client.0.smithi144.stdout:0/525: dwrite da/d3a/d4b/f7d [0,4194304] 0 2024-09-10T11:29:44.554 INFO:tasks.workunit.client.0.smithi144.stdout:4/737: read d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/f87 [575316,92682] 0 2024-09-10T11:29:44.554 INFO:tasks.workunit.client.0.smithi144.stdout:8/633: creat d0/d9/d1f/d34/d68/fd7 x:0 0 0 2024-09-10T11:29:44.554 INFO:tasks.workunit.client.0.smithi144.stdout:6/630: dwrite d9/d1f/d2c/f64 [0,4194304] 0 2024-09-10T11:29:44.560 INFO:tasks.workunit.client.0.smithi144.stdout:6/631: creat d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d60/fdf x:0 0 0 2024-09-10T11:29:44.560 INFO:tasks.workunit.client.0.smithi144.stdout:3/708: fdatasync d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/f89 0 2024-09-10T11:29:44.562 INFO:tasks.workunit.client.1.smithi178.stdout:4/991: dread d0/d9/f13 [0,4194304] 0 2024-09-10T11:29:44.566 INFO:tasks.workunit.client.0.smithi144.stdout:8/634: mkdir d0/d9/d21/d25/db3/dc2/dcd/dd8 0 2024-09-10T11:29:44.567 INFO:tasks.workunit.client.0.smithi144.stdout:6/632: symlink d9/dd/le0 0 2024-09-10T11:29:44.568 INFO:tasks.workunit.client.0.smithi144.stdout:8/635: chown d0/d9/d23/d26/f4a 1016368 1 2024-09-10T11:29:44.569 INFO:tasks.workunit.client.1.smithi178.stdout:4/992: creat d0/d9/d3d/d4f/f161 x:0 0 0 2024-09-10T11:29:44.571 INFO:tasks.workunit.client.0.smithi144.stdout:4/738: mkdir d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/d10b 0 2024-09-10T11:29:44.573 INFO:tasks.workunit.client.0.smithi144.stdout:0/526: dwrite da/d3a/d4b/d4e/d62/f68 [0,4194304] 0 2024-09-10T11:29:44.573 INFO:tasks.workunit.client.0.smithi144.stdout:4/739: chown d14/f15 5592058 1 2024-09-10T11:29:44.574 INFO:tasks.workunit.client.1.smithi178.stdout:4/993: dread d0/d8/da/f96 [0,4194304] 0 2024-09-10T11:29:44.574 INFO:tasks.workunit.client.0.smithi144.stdout:6/633: mkdir d9/dd/d11/d1b/d2f/dab/de1 0 2024-09-10T11:29:44.577 INFO:tasks.workunit.client.0.smithi144.stdout:4/740: chown d14/d67/d69/d90 65815679 1 2024-09-10T11:29:44.578 INFO:tasks.workunit.client.1.smithi178.stdout:4/994: mknod d0/d9/dc/c162 0 2024-09-10T11:29:44.578 INFO:tasks.workunit.client.0.smithi144.stdout:0/527: rmdir da/d3a/d51/d5f/d83 39 2024-09-10T11:29:44.579 INFO:tasks.workunit.client.0.smithi144.stdout:8/636: dwrite d0/d9/d1f/d34/d68/fd7 [0,4194304] 0 2024-09-10T11:29:44.581 INFO:tasks.workunit.client.0.smithi144.stdout:8/637: write d0/f3f [341253,6016] 0 2024-09-10T11:29:44.586 INFO:tasks.workunit.client.0.smithi144.stdout:0/528: truncate da/d3a/d51/d5f/d83/f9c 258141 0 2024-09-10T11:29:44.586 INFO:tasks.workunit.client.0.smithi144.stdout:6/634: truncate d9/dd/d11/d1b/d2f/d7f/f75 795060 0 2024-09-10T11:29:44.586 INFO:tasks.workunit.client.1.smithi178.stdout:4/995: dwrite d0/d8/f114 [0,4194304] 0 2024-09-10T11:29:44.591 INFO:tasks.workunit.client.0.smithi144.stdout:4/741: rename d14/d1e/f50 to d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f/d103/f10c 0 2024-09-10T11:29:44.592 INFO:tasks.workunit.client.0.smithi144.stdout:3/709: link d4/d1a/l6c d4/dd/d21/d2f/le0 0 2024-09-10T11:29:44.592 INFO:tasks.workunit.client.0.smithi144.stdout:0/529: readlink da/l43 0 2024-09-10T11:29:44.592 INFO:tasks.workunit.client.0.smithi144.stdout:8/638: creat d0/d1b/d5a/fd9 x:0 0 0 2024-09-10T11:29:44.595 INFO:tasks.workunit.client.1.smithi178.stdout:4/996: dwrite d0/d8/df/dfd/f14f [0,4194304] 0 2024-09-10T11:29:44.595 INFO:tasks.workunit.client.0.smithi144.stdout:6/635: creat d9/d1f/d2c/fe2 x:0 0 0 2024-09-10T11:29:44.597 INFO:tasks.workunit.client.1.smithi178.stdout:4/997: stat d0/d8/da/f2b 0 2024-09-10T11:29:44.599 INFO:tasks.workunit.client.0.smithi144.stdout:8/639: mkdir d0/d9/d23/d26/d7d/dda 0 2024-09-10T11:29:44.600 INFO:tasks.workunit.client.1.smithi178.stdout:4/998: creat d0/d9d/de1/f163 x:0 0 0 2024-09-10T11:29:44.601 INFO:tasks.workunit.client.0.smithi144.stdout:0/530: unlink da/c6a 0 2024-09-10T11:29:44.603 INFO:tasks.workunit.client.0.smithi144.stdout:3/710: creat d4/fe1 x:0 0 0 2024-09-10T11:29:44.604 INFO:tasks.workunit.client.1.smithi178.stdout:4/999: creat d0/d8/da/dae/d10e/f164 x:0 0 0 2024-09-10T11:29:44.605 INFO:tasks.workunit.client.0.smithi144.stdout:6/636: symlink d9/dd/d11/d1b/d2f/d7f/dc4/d5b/le3 0 2024-09-10T11:29:44.607 INFO:tasks.workunit.client.0.smithi144.stdout:2/565: truncate d7/d42/d64/faa 5335032 0 2024-09-10T11:29:44.609 INFO:tasks.workunit.client.0.smithi144.stdout:8/640: mknod d0/d9/d21/d25/db3/dc2/dd1/cdb 0 2024-09-10T11:29:44.609 INFO:tasks.workunit.client.0.smithi144.stdout:0/531: read da/f32 [1561158,80092] 0 2024-09-10T11:29:44.610 INFO:tasks.workunit.client.0.smithi144.stdout:8/641: chown d0/d9/d1f/d34/fad 37425 1 2024-09-10T11:29:44.611 INFO:tasks.workunit.client.0.smithi144.stdout:4/742: link d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/fb7 d14/d67/d69/d90/dda/de7/f10d 0 2024-09-10T11:29:44.615 INFO:tasks.workunit.client.0.smithi144.stdout:6/637: fdatasync d9/d1f/d2c/d4f/f7d 0 2024-09-10T11:29:44.616 INFO:tasks.workunit.client.0.smithi144.stdout:0/532: rmdir da/d3a/d4b 39 2024-09-10T11:29:44.617 INFO:tasks.workunit.client.0.smithi144.stdout:8/642: creat d0/d9/d21/d7e/d85/fdc x:0 0 0 2024-09-10T11:29:44.618 INFO:tasks.workunit.client.0.smithi144.stdout:0/533: write da/d3a/f76 [126827,92997] 0 2024-09-10T11:29:44.618 INFO:tasks.workunit.client.0.smithi144.stdout:4/743: mknod d14/d1e/d65/d8a/d8f/c10e 0 2024-09-10T11:29:44.623 INFO:tasks.workunit.client.0.smithi144.stdout:6/638: mknod d9/dd/d11/d1b/d2f/d7f/ce4 0 2024-09-10T11:29:44.624 INFO:tasks.workunit.client.0.smithi144.stdout:8/643: chown d0/d9/d1f/d34/d68/lac 32 1 2024-09-10T11:29:44.626 INFO:tasks.workunit.client.0.smithi144.stdout:3/711: link d4/d28/d67/d80/d17/c56 d4/dd/d21/ce2 0 2024-09-10T11:29:44.630 INFO:tasks.workunit.client.0.smithi144.stdout:0/534: symlink da/d3a/d94/la4 0 2024-09-10T11:29:44.633 INFO:tasks.workunit.client.0.smithi144.stdout:3/712: chown d4/dd/d21/f41 50977847 1 2024-09-10T11:29:44.633 INFO:tasks.workunit.client.0.smithi144.stdout:2/566: getdents d7/de/d36/d45/d8e/d9a/d7e 0 2024-09-10T11:29:44.638 INFO:tasks.workunit.client.0.smithi144.stdout:4/744: creat d14/d1e/d22/d34/d3e/d40/f10f x:0 0 0 2024-09-10T11:29:44.639 INFO:tasks.workunit.client.0.smithi144.stdout:3/713: mknod d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/ce3 0 2024-09-10T11:29:44.640 INFO:tasks.workunit.client.0.smithi144.stdout:6/639: link d9/dd/d11/d1b/d2f/c86 d9/d1f/d2c/ce5 0 2024-09-10T11:29:44.643 INFO:tasks.workunit.client.0.smithi144.stdout:0/535: unlink da/d3a/d4b/f7d 0 2024-09-10T11:29:44.644 INFO:tasks.workunit.client.0.smithi144.stdout:4/745: mkdir d14/d7e/d110 0 2024-09-10T11:29:44.647 INFO:tasks.workunit.client.0.smithi144.stdout:3/714: symlink d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/le4 0 2024-09-10T11:29:44.649 INFO:tasks.workunit.client.0.smithi144.stdout:0/536: dwrite da/d3a/d4b/d4e/d62/f68 [0,4194304] 0 2024-09-10T11:29:44.651 INFO:tasks.workunit.client.0.smithi144.stdout:4/746: unlink d14/d1e/d83/da4/da9/f10a 0 2024-09-10T11:29:44.654 INFO:tasks.workunit.client.0.smithi144.stdout:0/537: creat da/d3a/d4b/d4e/d62/d78/fa5 x:0 0 0 2024-09-10T11:29:44.655 INFO:tasks.workunit.client.0.smithi144.stdout:0/538: chown da/f32 4 1 2024-09-10T11:29:44.657 INFO:tasks.workunit.client.0.smithi144.stdout:4/747: unlink d14/d1e/d22/d34/d3e/d40/d4b/l5c 0 2024-09-10T11:29:44.658 INFO:tasks.workunit.client.0.smithi144.stdout:6/640: link d9/d1f/f37 d9/dd/d11/d1b/d2f/dab/de1/fe6 0 2024-09-10T11:29:44.658 INFO:tasks.workunit.client.0.smithi144.stdout:4/748: chown d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/f77 240552 1 2024-09-10T11:29:44.660 INFO:tasks.workunit.client.0.smithi144.stdout:0/539: creat da/d3a/d4b/d4e/d90/fa6 x:0 0 0 2024-09-10T11:29:44.661 INFO:tasks.workunit.client.1.smithi178.stdout:7/990: sync 2024-09-10T11:29:44.662 INFO:tasks.workunit.client.0.smithi144.stdout:6/641: mkdir d9/dd/d11/d1b/d2f/dab/de7 0 2024-09-10T11:29:44.664 INFO:tasks.workunit.client.1.smithi178.stdout:7/991: mkdir d6/d10/d1d/de6/d163 0 2024-09-10T11:29:44.664 INFO:tasks.workunit.client.0.smithi144.stdout:4/749: rename d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/f64 to d14/d1e/d83/f111 0 2024-09-10T11:29:44.665 INFO:tasks.workunit.client.1.smithi178.stdout:7/992: read d6/d10/d1d/d22/d3b/da1/f12c [101809,57606] 0 2024-09-10T11:29:44.667 INFO:tasks.workunit.client.0.smithi144.stdout:6/642: dread - d9/dd/d11/d1b/d2f/f8a zero size 2024-09-10T11:29:44.667 INFO:tasks.workunit.client.0.smithi144.stdout:0/540: dwrite da/d3a/f3e [0,4194304] 0 2024-09-10T11:29:44.671 INFO:tasks.workunit.client.0.smithi144.stdout:4/750: rename d14/d1e/l104 to d14/d1e/d22/d34/d3e/d40/d4b/d4c/dab/df3/l112 0 2024-09-10T11:29:44.671 INFO:tasks.workunit.client.0.smithi144.stdout:6/643: chown d9/dd/d3e/f9b 106527 1 2024-09-10T11:29:44.672 INFO:tasks.workunit.client.0.smithi144.stdout:2/567: truncate d7/da/d1e/f25 406572 0 2024-09-10T11:29:44.672 INFO:tasks.workunit.client.0.smithi144.stdout:0/541: fdatasync da/f19 0 2024-09-10T11:29:44.675 INFO:tasks.workunit.client.0.smithi144.stdout:0/542: read - da/d3a/d4b/d61/d6c/f9b zero size 2024-09-10T11:29:44.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.675+0000 7f8128878700 1 -- 172.21.15.144:0/615475523 --> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8124105ce0 con 0x7f810c06d3b0 2024-09-10T11:29:44.677 INFO:tasks.workunit.client.0.smithi144.stdout:4/751: getdents d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/d10b 0 2024-09-10T11:29:44.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.677+0000 7f81137fe700 1 -- 172.21.15.144:0/615475523 <== mgr.24329 v2:172.21.15.178:6828/2974090415 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f8124105ce0 con 0x7f810c06d3b0 2024-09-10T11:29:44.680 INFO:tasks.workunit.client.1.smithi178.stdout:7/993: dwrite d6/d10/d1e/d2d/d2b/da0/d55/fed [0,4194304] 0 2024-09-10T11:29:44.682 INFO:tasks.workunit.client.0.smithi144.stdout:2/568: truncate d7/de/d36/d45/d47/d6c/fa2 627630 0 2024-09-10T11:29:44.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.681+0000 7f8128878700 1 -- 172.21.15.144:0/615475523 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f810c06d3b0 msgr2=0x7f810c06f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:44.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.681+0000 7f8128878700 1 --2- 172.21.15.144:0/615475523 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f810c06d3b0 0x7f810c06f860 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8118009fd0 tx=0x7f8118005fb0 comp rx=0 tx=0).stop 2024-09-10T11:29:44.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.681+0000 7f8128878700 1 -- 172.21.15.144:0/615475523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8124101390 msgr2=0x7f8124073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:44.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.681+0000 7f8128878700 1 --2- 172.21.15.144:0/615475523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8124101390 0x7f8124073140 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f811400e920 tx=0x7f811400ece0 comp rx=0 tx=0).stop 2024-09-10T11:29:44.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.681+0000 7f8128878700 1 -- 172.21.15.144:0/615475523 shutdown_connections 2024-09-10T11:29:44.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.681+0000 7f8128878700 1 --2- 172.21.15.144:0/615475523 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f810c06d3b0 0x7f810c06f860 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:44.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.681+0000 7f8128878700 1 --2- 172.21.15.144:0/615475523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8124100190 0x7f8124074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:44.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.681+0000 7f8128878700 1 --2- 172.21.15.144:0/615475523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8124101390 0x7f8124073140 secure :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f811400e920 tx=0x7f811400ece0 comp rx=0 tx=0).stop 2024-09-10T11:29:44.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.682+0000 7f8128878700 1 -- 172.21.15.144:0/615475523 >> 172.21.15.144:0/615475523 conn(0x7f81240fb720 msgr2=0x7f81241045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:44.684 INFO:tasks.workunit.client.0.smithi144.stdout:6/644: link d9/d1f/f36 d9/dd/d11/d1b/d2f/dab/de7/fe8 0 2024-09-10T11:29:44.684 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.682+0000 7f8128878700 1 -- 172.21.15.144:0/615475523 shutdown_connections 2024-09-10T11:29:44.684 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.682+0000 7f8128878700 1 -- 172.21.15.144:0/615475523 wait complete. 2024-09-10T11:29:44.685 INFO:tasks.workunit.client.0.smithi144.stdout:3/715: dread d4/d28/d67/d80/d17/d18/d34/d78/fb2 [4194304,4194304] 0 2024-09-10T11:29:44.687 INFO:tasks.workunit.client.1.smithi178.stdout:7/994: mknod d6/d10/d1d/d22/d50/d98/da6/c164 0 2024-09-10T11:29:44.690 INFO:tasks.workunit.client.0.smithi144.stdout:0/543: dwrite da/d3a/f4d [0,4194304] 0 2024-09-10T11:29:44.694 INFO:tasks.workunit.client.0.smithi144.stdout:0/544: symlink da/d3a/d8e/la7 0 2024-09-10T11:29:44.696 INFO:tasks.workunit.client.0.smithi144.stdout:3/716: truncate d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/f89 625376 0 2024-09-10T11:29:44.697 INFO:tasks.workunit.client.0.smithi144.stdout:0/545: chown da/d3a/d51/l7c 7561 1 2024-09-10T11:29:44.697 INFO:tasks.workunit.client.0.smithi144.stdout:3/717: readlink d4/d28/d67/d80/d17/lb4 0 2024-09-10T11:29:44.699 INFO:tasks.workunit.client.0.smithi144.stdout:3/718: dread - d4/d28/d67/d96/fd2 zero size 2024-09-10T11:29:44.710 INFO:tasks.workunit.client.0.smithi144.stdout:3/719: chown d4/dd/d21/d2f/f8d 13345 1 2024-09-10T11:29:44.711 INFO:tasks.workunit.client.0.smithi144.stdout:4/752: dread d14/d1e/d22/f31 [0,4194304] 0 2024-09-10T11:29:44.711 INFO:tasks.workunit.client.0.smithi144.stdout:2/569: write d7/da/fd [2597739,52260] 0 2024-09-10T11:29:44.711 INFO:tasks.workunit.client.0.smithi144.stdout:3/720: getdents d4/dd/d21/d2f 0 2024-09-10T11:29:44.712 INFO:tasks.workunit.client.0.smithi144.stdout:0/546: dwrite da/d3a/d4b/d61/d6c/fa1 [0,4194304] 0 2024-09-10T11:29:44.712 INFO:tasks.workunit.client.0.smithi144.stdout:4/753: symlink d14/d67/d69/dca/l113 0 2024-09-10T11:29:44.717 INFO:tasks.workunit.client.0.smithi144.stdout:3/721: mknod d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/ce5 0 2024-09-10T11:29:44.719 INFO:tasks.workunit.client.0.smithi144.stdout:6/645: dread d9/dd/d11/d1b/d2f/d7f/dc4/d5b/fca [0,4194304] 0 2024-09-10T11:29:44.722 INFO:tasks.workunit.client.0.smithi144.stdout:4/754: dread d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/fb7 [0,4194304] 0 2024-09-10T11:29:44.725 INFO:tasks.workunit.client.0.smithi144.stdout:4/755: write d14/d67/d84/db3/fee [367475,3117] 0 2024-09-10T11:29:44.726 INFO:tasks.workunit.client.0.smithi144.stdout:0/547: dwrite da/d3a/d51/d5f/d83/f9c [0,4194304] 0 2024-09-10T11:29:44.729 INFO:tasks.workunit.client.0.smithi144.stdout:6/646: fsync d9/dd/d3e/f40 0 2024-09-10T11:29:44.730 INFO:tasks.workunit.client.0.smithi144.stdout:4/756: stat d14/d1e/d83/da4/da9/fc7 0 2024-09-10T11:29:44.730 INFO:tasks.workunit.client.0.smithi144.stdout:0/548: write da/d3a/d4b/d96/fa3 [876150,41827] 0 2024-09-10T11:29:44.731 INFO:tasks.workunit.client.0.smithi144.stdout:6/647: chown d9/dd/d11/d1b/l42 14 1 2024-09-10T11:29:44.731 INFO:tasks.workunit.client.0.smithi144.stdout:3/722: link d4/d28/f87 d4/d28/d67/d80/d17/fe6 0 2024-09-10T11:29:44.734 INFO:tasks.workunit.client.0.smithi144.stdout:3/723: readlink d4/d28/d67/d80/d1d/la1 0 2024-09-10T11:29:44.735 INFO:tasks.workunit.client.0.smithi144.stdout:2/570: dread d7/de/d3d/d43/f7b [0,4194304] 0 2024-09-10T11:29:44.736 INFO:tasks.workunit.client.0.smithi144.stdout:2/571: fdatasync d7/de/d3d/f52 0 2024-09-10T11:29:44.737 INFO:tasks.workunit.client.0.smithi144.stdout:6/648: rename d9/dd/le0 to d9/dd/d11/d1b/d2f/dab/dc6/le9 0 2024-09-10T11:29:44.742 INFO:tasks.workunit.client.0.smithi144.stdout:2/572: creat d7/de/d36/d45/d47/d6c/d7c/d82/fbc x:0 0 0 2024-09-10T11:29:44.748 INFO:tasks.workunit.client.0.smithi144.stdout:1/792: sync 2024-09-10T11:29:44.748 INFO:tasks.workunit.client.0.smithi144.stdout:9/681: sync 2024-09-10T11:29:44.750 INFO:tasks.workunit.client.0.smithi144.stdout:6/649: truncate d9/dd/d11/d1b/d2f/d7f/dc4/d5b/fca 746764 0 2024-09-10T11:29:44.751 INFO:tasks.workunit.client.0.smithi144.stdout:2/573: truncate d7/da/f3c 3939792 0 2024-09-10T11:29:44.752 INFO:tasks.workunit.client.0.smithi144.stdout:3/724: dwrite d4/dd/d21/f8b [0,4194304] 0 2024-09-10T11:29:44.753 INFO:tasks.workunit.client.0.smithi144.stdout:1/793: creat d5/d4a/d44/f100 x:0 0 0 2024-09-10T11:29:44.756 INFO:tasks.workunit.client.0.smithi144.stdout:2/574: rmdir d7/de/d3d 39 2024-09-10T11:29:44.760 INFO:tasks.workunit.client.0.smithi144.stdout:3/725: dread d4/d28/d67/d80/d17/d18/f6d [0,4194304] 0 2024-09-10T11:29:44.761 INFO:tasks.workunit.client.0.smithi144.stdout:1/794: mkdir d5/dd/d64/d79/d101 0 2024-09-10T11:29:44.762 INFO:tasks.workunit.client.0.smithi144.stdout:6/650: creat d9/fea x:0 0 0 2024-09-10T11:29:44.763 INFO:tasks.workunit.client.0.smithi144.stdout:1/795: fdatasync d5/d4a/d63/f9f 0 2024-09-10T11:29:44.763 INFO:tasks.workunit.client.0.smithi144.stdout:9/682: getdents d11/d36/db1/dbb/df0/d8f/dab 0 2024-09-10T11:29:44.763 INFO:tasks.workunit.client.0.smithi144.stdout:0/549: dread da/f41 [0,4194304] 0 2024-09-10T11:29:44.765 INFO:tasks.workunit.client.1.smithi178.stdout:7/995: rename d6/d10/d1d/d22/d50/d7e/d123/fd4 to d6/d10/d1d/d22/d50/d7e/d82/db2/f165 0 2024-09-10T11:29:44.766 INFO:tasks.workunit.client.0.smithi144.stdout:0/550: chown da/d44/d82/l8a 0 1 2024-09-10T11:29:44.769 INFO:tasks.workunit.client.0.smithi144.stdout:6/651: mkdir d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d87/dc9/deb 0 2024-09-10T11:29:44.769 INFO:tasks.workunit.client.1.smithi178.stdout:7/996: mkdir d6/d10/d1d/d22/d50/d7e/d82/db2/d49/d70/da2/d15a/db1/d130/d166 0 2024-09-10T11:29:44.771 INFO:tasks.workunit.client.0.smithi144.stdout:0/551: rename f2 to da/d3a/d51/d5f/d83/fa8 0 2024-09-10T11:29:44.772 INFO:tasks.workunit.client.0.smithi144.stdout:9/683: creat d11/d12/d1b/ff4 x:0 0 0 2024-09-10T11:29:44.772 INFO:tasks.workunit.client.0.smithi144.stdout:0/552: fdatasync da/d3a/d4b/d4e/d81/f98 0 2024-09-10T11:29:44.773 INFO:tasks.workunit.client.1.smithi178.stdout:7/997: truncate d6/d10/d1d/d22/d50/def/df3/f12b 778168 0 2024-09-10T11:29:44.773 INFO:tasks.workunit.client.0.smithi144.stdout:3/726: creat d4/d28/d67/d80/fe7 x:0 0 0 2024-09-10T11:29:44.775 INFO:tasks.workunit.client.0.smithi144.stdout:9/684: truncate d11/d12/d1b/f8c 420821 0 2024-09-10T11:29:44.776 INFO:tasks.workunit.client.0.smithi144.stdout:0/553: truncate da/d3a/d4b/d4e/d81/f98 250686 0 2024-09-10T11:29:44.778 INFO:tasks.workunit.client.0.smithi144.stdout:9/685: rmdir d11/d36/db1/dbb/df0/d8e 39 2024-09-10T11:29:44.781 INFO:tasks.workunit.client.0.smithi144.stdout:9/686: symlink d11/d36/lf5 0 2024-09-10T11:29:44.781 INFO:tasks.workunit.client.0.smithi144.stdout:9/687: chown d11/c35 338056 1 2024-09-10T11:29:44.782 INFO:tasks.workunit.client.0.smithi144.stdout:0/554: link da/d3a/d51/d5f/c55 da/d3a/d94/ca9 0 2024-09-10T11:29:44.783 INFO:tasks.workunit.client.0.smithi144.stdout:1/796: write d5/d2a/d65/dc9/f67 [2085766,58092] 0 2024-09-10T11:29:44.783 INFO:tasks.workunit.client.0.smithi144.stdout:9/688: mkdir d11/d36/d2d/df6 0 2024-09-10T11:29:44.785 INFO:tasks.workunit.client.0.smithi144.stdout:0/555: symlink da/d3a/d94/laa 0 2024-09-10T11:29:44.787 INFO:tasks.workunit.client.0.smithi144.stdout:2/575: truncate d7/de/d36/d45/d47/d6c/d7c/d82/f98 747835 0 2024-09-10T11:29:44.789 INFO:tasks.workunit.client.0.smithi144.stdout:9/689: mkdir d11/d36/db1/dbb/df0/d8f/dc0/dd0/df7 0 2024-09-10T11:29:44.790 INFO:tasks.workunit.client.0.smithi144.stdout:6/652: write d9/dd/f33 [1791755,108780] 0 2024-09-10T11:29:44.791 INFO:tasks.workunit.client.0.smithi144.stdout:3/727: write d4/d28/d67/f7c [1482535,57473] 0 2024-09-10T11:29:44.795 INFO:tasks.workunit.client.0.smithi144.stdout:1/797: rename d5/df/d4b/fbf to d5/d2a/f102 0 2024-09-10T11:29:44.796 INFO:tasks.workunit.client.0.smithi144.stdout:1/798: fdatasync d5/d2a/d65/dc9/f76 0 2024-09-10T11:29:44.797 INFO:tasks.workunit.client.0.smithi144.stdout:6/653: creat d9/dd/d11/d1b/d2f/d7f/fec x:0 0 0 2024-09-10T11:29:44.797 INFO:tasks.workunit.client.0.smithi144.stdout:1/799: readlink d5/df/d4b/l5c 0 2024-09-10T11:29:44.801 INFO:tasks.workunit.client.0.smithi144.stdout:2/576: dread d7/da/d1e/f66 [0,4194304] 0 2024-09-10T11:29:44.802 INFO:tasks.workunit.client.0.smithi144.stdout:0/556: write da/d44/f63 [271727,45153] 0 2024-09-10T11:29:44.803 INFO:tasks.workunit.client.0.smithi144.stdout:2/577: chown d7/de/d36/d45/d8e/d9a/f2a 23019 1 2024-09-10T11:29:44.807 INFO:tasks.workunit.client.0.smithi144.stdout:9/690: rename d11/d36/db1/dbb/df0/d8e to d11/d12/d1b/df8 0 2024-09-10T11:29:44.808 INFO:tasks.workunit.client.0.smithi144.stdout:0/557: rmdir da/d49 39 2024-09-10T11:29:44.811 INFO:tasks.workunit.client.0.smithi144.stdout:0/558: truncate da/d3a/f76 540736 0 2024-09-10T11:29:44.811 INFO:tasks.workunit.client.0.smithi144.stdout:1/800: write d5/d4a/dc0/fc6 [526271,49710] 0 2024-09-10T11:29:44.811 INFO:tasks.workunit.client.0.smithi144.stdout:6/654: truncate d9/dd/d11/d1b/f8d 1533206 0 2024-09-10T11:29:44.814 INFO:tasks.workunit.client.1.smithi178.stdout:7/998: write d6/d10/d1d/d4a/d79/df1/fa9 [1616414,130767] 0 2024-09-10T11:29:44.814 INFO:tasks.workunit.client.0.smithi144.stdout:0/559: mkdir da/d3a/d8e/dab 0 2024-09-10T11:29:44.814 INFO:tasks.workunit.client.0.smithi144.stdout:2/578: write d7/de/d3d/d4e/f87 [649680,22167] 0 2024-09-10T11:29:44.815 INFO:tasks.workunit.client.0.smithi144.stdout:9/691: write d11/d12/d1b/df8/f99 [260355,1700] 0 2024-09-10T11:29:44.815 INFO:tasks.workunit.client.0.smithi144.stdout:0/560: write da/d44/d82/fa2 [144819,107098] 0 2024-09-10T11:29:44.818 INFO:tasks.workunit.client.0.smithi144.stdout:1/801: mknod d5/d4a/d63/d84/dbd/dff/c103 0 2024-09-10T11:29:44.819 INFO:tasks.workunit.client.0.smithi144.stdout:0/561: creat da/d3a/d51/fac x:0 0 0 2024-09-10T11:29:44.820 INFO:tasks.workunit.client.1.smithi178.stdout:7/999: rename d6/dbf to d6/d10/d1e/d2d/d2b/d167 0 2024-09-10T11:29:44.821 INFO:tasks.workunit.client.0.smithi144.stdout:9/692: symlink d11/d12/d1b/d2c/d78/dd2/de4/lf9 0 2024-09-10T11:29:44.821 INFO:tasks.workunit.client.0.smithi144.stdout:1/802: rmdir d5/d2a/d65/d73/dd8 39 2024-09-10T11:29:44.823 INFO:tasks.workunit.client.0.smithi144.stdout:6/655: link d9/dd/d11/cb7 d9/dd/d11/d1b/ced 0 2024-09-10T11:29:44.823 INFO:tasks.workunit.client.1.smithi178.stderr:+ rm -rf -- ./tmp.KdfX4V8Omh 2024-09-10T11:29:44.824 INFO:tasks.workunit.client.0.smithi144.stdout:9/693: creat d11/d36/db1/ffa x:0 0 0 2024-09-10T11:29:44.824 INFO:tasks.workunit.client.0.smithi144.stdout:1/803: dread d5/d2a/d65/fcd [0,4194304] 0 2024-09-10T11:29:44.828 INFO:tasks.workunit.client.0.smithi144.stdout:6/656: unlink d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/f31 0 2024-09-10T11:29:44.829 INFO:tasks.workunit.client.0.smithi144.stdout:0/562: dwrite da/d3a/d4b/d61/d6c/f9b [0,4194304] 0 2024-09-10T11:29:44.833 INFO:tasks.workunit.client.0.smithi144.stdout:6/657: dread d9/f81 [0,4194304] 0 2024-09-10T11:29:44.838 INFO:tasks.workunit.client.0.smithi144.stdout:9/694: rename d11/d12/d6f/l9b to d11/d12/lfb 0 2024-09-10T11:29:44.840 INFO:tasks.workunit.client.0.smithi144.stdout:9/695: dread - d11/d12/d1b/fc9 zero size 2024-09-10T11:29:44.842 INFO:tasks.workunit.client.0.smithi144.stdout:6/658: mknod d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d60/cee 0 2024-09-10T11:29:44.846 INFO:tasks.workunit.client.0.smithi144.stdout:0/563: dwrite da/f26 [4194304,4194304] 0 2024-09-10T11:29:44.846 INFO:tasks.workunit.client.0.smithi144.stdout:6/659: stat d9/dd/d11/d1b/d2f/d7f/dc4 0 2024-09-10T11:29:44.846 INFO:tasks.workunit.client.0.smithi144.stdout:2/579: dread d7/da/f35 [0,4194304] 0 2024-09-10T11:29:44.846 INFO:tasks.workunit.client.0.smithi144.stdout:9/696: rmdir d11/d36/db1/dbb/df0/d8f/dc0/dd0/df7 0 2024-09-10T11:29:44.847 INFO:tasks.workunit.client.0.smithi144.stdout:2/580: stat d7/da/da8 0 2024-09-10T11:29:44.852 INFO:tasks.workunit.client.0.smithi144.stdout:0/564: unlink da/d3a/d4b/d4e/d62/d69/f79 0 2024-09-10T11:29:44.853 INFO:tasks.workunit.client.0.smithi144.stdout:9/697: unlink d11/d36/d2d/d38/l98 0 2024-09-10T11:29:44.854 INFO:tasks.workunit.client.0.smithi144.stdout:2/581: creat d7/de/d36/d45/d47/d6c/d7c/d82/fbd x:0 0 0 2024-09-10T11:29:44.856 INFO:tasks.workunit.client.0.smithi144.stdout:2/582: chown d7/da/d5b/d80/lba 1044405 1 2024-09-10T11:29:44.857 INFO:tasks.workunit.client.0.smithi144.stdout:0/565: getdents da/d3a/d4b/d4e/d62 0 2024-09-10T11:29:44.859 INFO:tasks.workunit.client.0.smithi144.stdout:2/583: creat d7/de/fbe x:0 0 0 2024-09-10T11:29:44.860 INFO:tasks.workunit.client.0.smithi144.stdout:2/584: fdatasync d7/f1d 0 2024-09-10T11:29:44.860 INFO:tasks.workunit.client.0.smithi144.stdout:0/566: creat da/d3a/d4b/d4e/d62/d78/fad x:0 0 0 2024-09-10T11:29:44.862 INFO:tasks.workunit.client.0.smithi144.stdout:2/585: mknod d7/de/d5f/cbf 0 2024-09-10T11:29:44.865 INFO:tasks.workunit.client.0.smithi144.stdout:2/586: creat d7/d29/d97/fc0 x:0 0 0 2024-09-10T11:29:44.867 INFO:tasks.workunit.client.0.smithi144.stdout:1/804: truncate d5/d2a/f7e 562049 0 2024-09-10T11:29:44.867 INFO:tasks.workunit.client.0.smithi144.stdout:2/587: rename d7 to d7/d29/d97/dc1 22 2024-09-10T11:29:44.868 INFO:tasks.workunit.client.0.smithi144.stdout:6/660: write d9/d1f/d2c/d4f/f7d [1590876,2314] 0 2024-09-10T11:29:44.870 INFO:tasks.workunit.client.0.smithi144.stdout:0/567: dwrite da/d44/f63 [0,4194304] 0 2024-09-10T11:29:44.873 INFO:tasks.workunit.client.0.smithi144.stdout:9/698: dread d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/f63 [0,4194304] 0 2024-09-10T11:29:44.873 INFO:tasks.workunit.client.0.smithi144.stdout:1/805: symlink d5/dd/d64/d79/l104 0 2024-09-10T11:29:44.874 INFO:tasks.workunit.client.0.smithi144.stdout:0/568: mkdir da/d3a/d4b/d4e/dae 0 2024-09-10T11:29:44.875 INFO:tasks.workunit.client.0.smithi144.stdout:9/699: chown d11/d36/db1/dbb/fc2 48 1 2024-09-10T11:29:44.877 INFO:tasks.workunit.client.0.smithi144.stdout:1/806: creat d5/dd/d64/dad/f105 x:0 0 0 2024-09-10T11:29:44.877 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.876+0000 7f94c95d1700 1 -- 172.21.15.144:0/1567876334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94c4101330 msgr2=0x7f94c41037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:44.878 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.876+0000 7f94c95d1700 1 --2- 172.21.15.144:0/1567876334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94c4101330 0x7f94c41037b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f94ac009910 tx=0x7f94ac009c20 comp rx=0 tx=0).stop 2024-09-10T11:29:44.878 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.876+0000 7f94c95d1700 1 -- 172.21.15.144:0/1567876334 shutdown_connections 2024-09-10T11:29:44.879 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.876+0000 7f94c95d1700 1 --2- 172.21.15.144:0/1567876334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94c4101330 0x7f94c41037b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:44.879 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.876+0000 7f94c95d1700 1 --2- 172.21.15.144:0/1567876334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94c40fe9d0 0x7f94c4100df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:44.879 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.876+0000 7f94c95d1700 1 -- 172.21.15.144:0/1567876334 >> 172.21.15.144:0/1567876334 conn(0x7f94c40fa5e0 msgr2=0x7f94c40fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:44.879 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.876+0000 7f94c95d1700 1 -- 172.21.15.144:0/1567876334 shutdown_connections 2024-09-10T11:29:44.879 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.876+0000 7f94c95d1700 1 -- 172.21.15.144:0/1567876334 wait complete. 2024-09-10T11:29:44.880 INFO:tasks.workunit.client.0.smithi144.stdout:0/569: fsync da/f30 0 2024-09-10T11:29:44.882 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.877+0000 7f94c95d1700 1 Processor -- start 2024-09-10T11:29:44.882 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.877+0000 7f94c95d1700 1 -- start start 2024-09-10T11:29:44.883 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.877+0000 7f94c95d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94c40fe9d0 0x7f94c4071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:44.883 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.877+0000 7f94c95d1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94c4101330 0x7f94c4072240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:44.883 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.877+0000 7f94c95d1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4072860 con 0x7f94c40fe9d0 2024-09-10T11:29:44.883 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.877+0000 7f94c95d1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c41a9d80 con 0x7f94c4101330 2024-09-10T11:29:44.883 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.878+0000 7f94c2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94c40fe9d0 0x7f94c4071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:44.883 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.878+0000 7f94ba7fc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94c4101330 0x7f94c4072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:44.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.878+0000 7f94c2ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94c40fe9d0 0x7f94c4071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44636/0 (socket says 172.21.15.144:44636) 2024-09-10T11:29:44.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.878+0000 7f94c2ffd700 1 -- 172.21.15.144:0/4204739296 learned_addr learned my addr 172.21.15.144:0/4204739296 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:29:44.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.878+0000 7f94c2ffd700 1 -- 172.21.15.144:0/4204739296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94c4101330 msgr2=0x7f94c4072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:44.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.878+0000 7f94c2ffd700 1 --2- 172.21.15.144:0/4204739296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94c4101330 0x7f94c4072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:44.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.878+0000 7f94c2ffd700 1 -- 172.21.15.144:0/4204739296 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac0095f0 con 0x7f94c40fe9d0 2024-09-10T11:29:44.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.878+0000 7f94ba7fc700 1 --2- 172.21.15.144:0/4204739296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94c4101330 0x7f94c4072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:29:44.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.879+0000 7f94c2ffd700 1 --2- 172.21.15.144:0/4204739296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94c40fe9d0 0x7f94c4071d00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f94b400ba50 tx=0x7f94b400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:44.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.879+0000 7f94c0ff9700 1 -- 172.21.15.144:0/4204739296 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94b400c900 con 0x7f94c40fe9d0 2024-09-10T11:29:44.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.879+0000 7f94c0ff9700 1 -- 172.21.15.144:0/4204739296 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f94b4014440 con 0x7f94c40fe9d0 2024-09-10T11:29:44.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.879+0000 7f94c0ff9700 1 -- 172.21.15.144:0/4204739296 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94b4007560 con 0x7f94c40fe9d0 2024-09-10T11:29:44.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.879+0000 7f94c95d1700 1 -- 172.21.15.144:0/4204739296 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c41a9f80 con 0x7f94c40fe9d0 2024-09-10T11:29:44.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.879+0000 7f94c95d1700 1 -- 172.21.15.144:0/4204739296 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c41aa420 con 0x7f94c40fe9d0 2024-09-10T11:29:44.886 INFO:tasks.workunit.client.0.smithi144.stdout:9/700: rename d11/d12/d6f/daa/lef to d11/d12/d1b/d2c/d78/lfc 0 2024-09-10T11:29:44.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.882+0000 7f94c95d1700 1 -- 172.21.15.144:0/4204739296 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a4005320 con 0x7f94c40fe9d0 2024-09-10T11:29:44.886 INFO:tasks.workunit.client.0.smithi144.stdout:0/570: symlink da/d3a/d4b/d61/laf 0 2024-09-10T11:29:44.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.883+0000 7f94c0ff9700 1 -- 172.21.15.144:0/4204739296 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) v1 ==== 91089+0+0 (secure 0 0 0) 0x7f94b4007780 con 0x7f94c40fe9d0 2024-09-10T11:29:44.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.883+0000 7f94c0ff9700 1 --2- 172.21.15.144:0/4204739296 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f94b006d3b0 0x7f94b006f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:44.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.884+0000 7f94c0ff9700 1 -- 172.21.15.144:0/4204739296 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f94b408c130 con 0x7f94c40fe9d0 2024-09-10T11:29:44.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.884+0000 7f94ba7fc700 1 --2- 172.21.15.144:0/4204739296 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f94b006d3b0 0x7f94b006f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:44.888 INFO:tasks.workunit.client.0.smithi144.stdout:2/588: dread d7/de/d36/d45/d8e/d9a/d7e/faf [0,4194304] 0 2024-09-10T11:29:44.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.884+0000 7f94ba7fc700 1 --2- 172.21.15.144:0/4204739296 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f94b006d3b0 0x7f94b006f860 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f94c4072df0 tx=0x7f94ac00b540 comp rx=0 tx=0).ready entity=mgr.24329 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:44.892 INFO:tasks.workunit.client.0.smithi144.stdout:9/701: dwrite d11/d36/db1/dbb/df0/d8f/fb4 [0,4194304] 0 2024-09-10T11:29:44.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:44.891+0000 7f94c0ff9700 1 -- 172.21.15.144:0/4204739296 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f94b4056c70 con 0x7f94c40fe9d0 2024-09-10T11:29:44.898 INFO:tasks.workunit.client.0.smithi144.stdout:2/589: write d7/da/d1e/f25 [1037885,76255] 0 2024-09-10T11:29:44.901 INFO:tasks.workunit.client.0.smithi144.stdout:9/702: creat d11/d12/d1b/ffd x:0 0 0 2024-09-10T11:29:44.901 INFO:tasks.workunit.client.0.smithi144.stdout:0/571: dwrite da/d3a/d51/f7a [0,4194304] 0 2024-09-10T11:29:44.905 INFO:tasks.workunit.client.0.smithi144.stdout:1/807: dwrite d5/d2a/f34 [0,4194304] 0 2024-09-10T11:29:44.907 INFO:tasks.workunit.client.0.smithi144.stdout:9/703: creat d11/d12/d6f/daa/ffe x:0 0 0 2024-09-10T11:29:44.911 INFO:tasks.workunit.client.0.smithi144.stdout:6/661: dread d9/dd/f1d [0,4194304] 0 2024-09-10T11:29:44.912 INFO:tasks.workunit.client.0.smithi144.stdout:9/704: truncate d11/d12/d1b/f27 789252 0 2024-09-10T11:29:44.913 INFO:tasks.workunit.client.0.smithi144.stdout:1/808: dread d5/df/f55 [0,4194304] 0 2024-09-10T11:29:44.914 INFO:tasks.workunit.client.0.smithi144.stdout:1/809: chown d5/df/d4b/d51/l5f 0 1 2024-09-10T11:29:44.914 INFO:tasks.workunit.client.0.smithi144.stdout:9/705: symlink d11/d12/lff 0 2024-09-10T11:29:44.915 INFO:tasks.workunit.client.0.smithi144.stdout:9/706: write d11/d36/fa4 [4324239,93738] 0 2024-09-10T11:29:44.917 INFO:tasks.workunit.client.0.smithi144.stdout:0/572: read da/f19 [3111144,126749] 0 2024-09-10T11:29:44.918 INFO:tasks.workunit.client.0.smithi144.stdout:0/573: chown da/d3a/d51/d8f 490 1 2024-09-10T11:29:44.918 INFO:tasks.workunit.client.0.smithi144.stdout:9/707: read d11/d36/f80 [3544002,51769] 0 2024-09-10T11:29:44.919 INFO:tasks.workunit.client.0.smithi144.stdout:1/810: rename d5/df/d4b/dcf/lda to d5/dd/l106 0 2024-09-10T11:29:44.920 INFO:tasks.workunit.client.0.smithi144.stdout:0/574: write da/f41 [2583601,56922] 0 2024-09-10T11:29:44.922 INFO:tasks.workunit.client.0.smithi144.stdout:1/811: write d5/dd/d64/d79/fb3 [914201,94681] 0 2024-09-10T11:29:44.922 INFO:tasks.workunit.client.0.smithi144.stdout:9/708: chown d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/f63 481281 1 2024-09-10T11:29:44.927 INFO:tasks.workunit.client.0.smithi144.stdout:1/812: symlink d5/d4a/d63/d84/dbd/dbe/def/l107 0 2024-09-10T11:29:44.930 INFO:tasks.workunit.client.0.smithi144.stdout:9/709: dwrite d11/d12/d1b/f95 [0,4194304] 0 2024-09-10T11:29:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:44 smithi144 ceph-mon[27942]: pgmap v29: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 150 MiB/s rd, 408 MiB/s wr, 781 op/s 2024-09-10T11:29:44.938 INFO:tasks.workunit.client.0.smithi144.stdout:1/813: rmdir d5 39 2024-09-10T11:29:44.944 INFO:tasks.workunit.client.0.smithi144.stdout:9/710: dread d11/d12/d1b/f27 [0,4194304] 0 2024-09-10T11:29:44.947 INFO:tasks.workunit.client.0.smithi144.stdout:9/711: creat d11/d36/d2d/df6/f100 x:0 0 0 2024-09-10T11:29:44.949 INFO:tasks.workunit.client.0.smithi144.stdout:9/712: symlink d11/d36/d2d/df6/l101 0 2024-09-10T11:29:44.952 INFO:tasks.workunit.client.0.smithi144.stdout:9/713: rename d11/d36/c2e to d11/d36/db1/dbb/df0/d8f/dc0/c102 0 2024-09-10T11:29:44.955 INFO:tasks.workunit.client.0.smithi144.stdout:9/714: getdents d11/d36/db1 0 2024-09-10T11:29:44.957 INFO:tasks.workunit.client.0.smithi144.stdout:9/715: write d11/d12/d1b/ff4 [712853,14560] 0 2024-09-10T11:29:44.957 INFO:tasks.workunit.client.0.smithi144.stdout:1/814: dread d5/df/d4b/f54 [0,4194304] 0 2024-09-10T11:29:44.960 INFO:tasks.workunit.client.0.smithi144.stdout:9/716: creat d11/d36/db1/dbb/df0/d8f/dc0/dd0/f103 x:0 0 0 2024-09-10T11:29:44.962 INFO:tasks.workunit.client.0.smithi144.stdout:9/717: chown d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/fa9 7 1 2024-09-10T11:29:44.964 INFO:tasks.workunit.client.0.smithi144.stdout:9/718: fdatasync f10 0 2024-09-10T11:29:44.966 INFO:tasks.workunit.client.0.smithi144.stdout:9/719: creat d11/d36/db1/dbb/f104 x:0 0 0 2024-09-10T11:29:44.971 INFO:tasks.workunit.client.0.smithi144.stdout:1/815: dread d5/d4a/d44/da2/fd7 [0,4194304] 0 2024-09-10T11:29:44.972 INFO:tasks.workunit.client.0.smithi144.stdout:9/720: truncate d11/f15 454426 0 2024-09-10T11:29:44.973 INFO:tasks.workunit.client.0.smithi144.stdout:1/816: write d5/d2a/d65/fcd [942030,40168] 0 2024-09-10T11:29:44.975 INFO:tasks.workunit.client.0.smithi144.stdout:9/721: dread d11/d12/f26 [0,4194304] 0 2024-09-10T11:29:44.975 INFO:tasks.workunit.client.0.smithi144.stdout:1/817: truncate d5/d4a/d63/d84/dbd/df1/fea 294386 0 2024-09-10T11:29:44.977 INFO:tasks.workunit.client.0.smithi144.stdout:9/722: truncate d11/d36/db1/dbb/df0/fd9 380520 0 2024-09-10T11:29:44.981 INFO:tasks.workunit.client.0.smithi144.stdout:1/818: dread d5/f15 [0,4194304] 0 2024-09-10T11:29:44.982 INFO:tasks.workunit.client.0.smithi144.stdout:1/819: truncate d5/dd/f13 288748 0 2024-09-10T11:29:44.984 INFO:tasks.workunit.client.0.smithi144.stdout:1/820: chown d5/d2a/d65/d73/fb1 1058 1 2024-09-10T11:29:44.985 INFO:tasks.workunit.client.0.smithi144.stdout:9/723: dwrite d11/d12/d1b/fc9 [0,4194304] 0 2024-09-10T11:29:44.990 INFO:tasks.workunit.client.0.smithi144.stdout:9/724: symlink d11/d12/d1b/l105 0 2024-09-10T11:29:44.992 INFO:tasks.workunit.client.0.smithi144.stdout:9/725: readlink d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/laf 0 2024-09-10T11:29:44.995 INFO:tasks.workunit.client.0.smithi144.stdout:1/821: write d5/df/d4b/d51/f50 [2356573,50450] 0 2024-09-10T11:29:44.999 INFO:tasks.workunit.client.0.smithi144.stdout:9/726: dread d11/d36/db1/dbb/fe1 [0,4194304] 0 2024-09-10T11:29:45.000 INFO:tasks.workunit.client.0.smithi144.stdout:1/822: creat d5/d4a/f108 x:0 0 0 2024-09-10T11:29:45.003 INFO:tasks.workunit.client.0.smithi144.stdout:9/727: fsync d11/d36/db1/dbb/df0/f5c 0 2024-09-10T11:29:45.012 INFO:tasks.workunit.client.0.smithi144.stdout:1/823: dread d5/f23 [0,4194304] 0 2024-09-10T11:29:45.016 INFO:tasks.workunit.client.0.smithi144.stdout:1/824: getdents d5/d2a/d65 0 2024-09-10T11:29:45.016 INFO:tasks.workunit.client.0.smithi144.stdout:7/719: sync 2024-09-10T11:29:45.016 INFO:tasks.workunit.client.0.smithi144.stdout:5/744: sync 2024-09-10T11:29:45.016 INFO:tasks.workunit.client.0.smithi144.stdout:9/728: write d11/d12/d1b/f27 [963666,10799] 0 2024-09-10T11:29:45.017 INFO:tasks.workunit.client.0.smithi144.stdout:1/825: chown d5/df/f33 236533 1 2024-09-10T11:29:45.020 INFO:tasks.workunit.client.0.smithi144.stdout:7/720: creat d5/dc/d12/d2b/dde/feb x:0 0 0 2024-09-10T11:29:45.021 INFO:tasks.workunit.client.0.smithi144.stdout:7/721: write d5/f5a [3196396,91827] 0 2024-09-10T11:29:45.022 INFO:tasks.workunit.client.0.smithi144.stdout:1/826: rename d5/d4a/d63/fac to d5/d4a/d63/d84/dbd/f109 0 2024-09-10T11:29:45.024 INFO:tasks.workunit.client.0.smithi144.stdout:7/722: stat d5/dc/d14/d40/d5e/d68/f8f 0 2024-09-10T11:29:45.026 INFO:tasks.workunit.client.0.smithi144.stdout:7/723: chown d5/dc/d11/f17 139292 1 2024-09-10T11:29:45.029 INFO:tasks.workunit.client.0.smithi144.stdout:7/724: creat d5/dc/fec x:0 0 0 2024-09-10T11:29:45.032 INFO:tasks.workunit.client.0.smithi144.stdout:7/725: mknod d5/dc/d14/d40/d5e/ced 0 2024-09-10T11:29:45.032 INFO:tasks.workunit.client.0.smithi144.stdout:9/729: dread d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/f4a [0,4194304] 0 2024-09-10T11:29:45.033 INFO:tasks.workunit.client.0.smithi144.stdout:5/745: dread d4/d20/f34 [0,4194304] 0 2024-09-10T11:29:45.036 INFO:tasks.workunit.client.0.smithi144.stdout:5/746: creat d4/d5/d18/d38/d6d/fe9 x:0 0 0 2024-09-10T11:29:45.037 INFO:tasks.workunit.client.0.smithi144.stdout:9/730: mkdir d11/d12/d1b/d2c/d78/dd2/d106 0 2024-09-10T11:29:45.039 INFO:tasks.workunit.client.0.smithi144.stdout:9/731: symlink d11/d36/d2d/df6/l107 0 2024-09-10T11:29:45.044 INFO:tasks.workunit.client.0.smithi144.stdout:5/747: dwrite d4/d5/d32/fc4 [0,4194304] 0 2024-09-10T11:29:45.045 INFO:tasks.workunit.client.0.smithi144.stdout:7/726: truncate d5/dc/d12/f13 3483171 0 2024-09-10T11:29:45.048 INFO:tasks.workunit.client.0.smithi144.stdout:5/748: symlink d4/d5/d56/d8a/lea 0 2024-09-10T11:29:45.050 INFO:tasks.workunit.client.0.smithi144.stdout:7/727: rename d5/dc/d14/d40/d58 to d5/dc/d14/d40/d58/d6a/dad/dee 22 2024-09-10T11:29:45.052 INFO:tasks.workunit.client.0.smithi144.stdout:9/732: dwrite d11/f15 [0,4194304] 0 2024-09-10T11:29:45.053 INFO:tasks.workunit.client.0.smithi144.stdout:5/749: mknod d4/d5/d56/ceb 0 2024-09-10T11:29:45.054 INFO:tasks.workunit.client.0.smithi144.stdout:9/733: write d11/d36/f80 [1596387,29015] 0 2024-09-10T11:29:45.055 INFO:tasks.workunit.client.0.smithi144.stdout:7/728: mkdir d5/dc/def 0 2024-09-10T11:29:45.059 INFO:tasks.workunit.client.0.smithi144.stdout:5/750: symlink d4/d5/d56/d8a/lec 0 2024-09-10T11:29:45.062 INFO:tasks.workunit.client.0.smithi144.stdout:9/734: unlink d11/d12/d1b/d2c/d78/dd2/ld6 0 2024-09-10T11:29:45.063 INFO:tasks.workunit.client.0.smithi144.stdout:5/751: unlink d4/d20/d22/d5c/d5d/d64/f96 0 2024-09-10T11:29:45.066 INFO:tasks.workunit.client.0.smithi144.stdout:9/735: creat d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/f108 x:0 0 0 2024-09-10T11:29:45.068 INFO:tasks.workunit.client.0.smithi144.stdout:9/736: write d11/f2b [3021848,76360] 0 2024-09-10T11:29:45.068 INFO:tasks.workunit.client.0.smithi144.stdout:9/737: stat l1 0 2024-09-10T11:29:45.072 INFO:tasks.workunit.client.0.smithi144.stdout:9/738: dread - d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/fe3 zero size 2024-09-10T11:29:45.073 INFO:tasks.workunit.client.0.smithi144.stdout:7/729: dwrite d5/dc/d14/d40/d58/d6a/fa0 [0,4194304] 0 2024-09-10T11:29:45.075 INFO:tasks.workunit.client.0.smithi144.stdout:7/730: chown d5/dc/d12/d2b/fc6 1028 1 2024-09-10T11:29:45.075 INFO:tasks.workunit.client.0.smithi144.stdout:7/731: readlink d5/dc/l39 0 2024-09-10T11:29:45.079 INFO:tasks.workunit.client.0.smithi144.stdout:7/732: mkdir d5/dc/d14/d40/d5e/d68/df0 0 2024-09-10T11:29:45.079 INFO:tasks.workunit.client.0.smithi144.stdout:9/739: rename d11/d36/db1/dbb/df0/c94 to d11/d12/d1b/d2c/d78/dd2/c109 0 2024-09-10T11:29:45.081 INFO:tasks.workunit.client.0.smithi144.stdout:9/740: write d11/d36/f4e [2485060,83417] 0 2024-09-10T11:29:45.081 INFO:tasks.workunit.client.0.smithi144.stdout:7/733: symlink d5/dc/d2a/d41/lf1 0 2024-09-10T11:29:45.087 INFO:tasks.workunit.client.0.smithi144.stdout:7/734: link d5/dc/d2a/d60/dd6/d98/dd1/d7c/l7f d5/dc/d12/lf2 0 2024-09-10T11:29:45.089 INFO:tasks.workunit.client.0.smithi144.stdout:7/735: mknod d5/dc/d14/d40/d5e/dc4/cf3 0 2024-09-10T11:29:45.097 INFO:tasks.workunit.client.0.smithi144.stdout:7/736: dwrite d5/dc/d2a/d60/dd6/f66 [0,4194304] 0 2024-09-10T11:29:45.097 INFO:tasks.workunit.client.0.smithi144.stdout:9/741: dwrite d11/d12/d1b/f77 [0,4194304] 0 2024-09-10T11:29:45.103 INFO:tasks.workunit.client.0.smithi144.stdout:9/742: write d11/d36/db1/dbb/df0/d8f/dc0/dd0/fe9 [420070,130523] 0 2024-09-10T11:29:45.103 INFO:tasks.workunit.client.0.smithi144.stdout:7/737: creat d5/dc/d14/d40/d58/d6a/db2/ff4 x:0 0 0 2024-09-10T11:29:45.110 INFO:tasks.workunit.client.0.smithi144.stdout:7/738: write d5/dc/d14/d40/d58/d6a/db2/fc8 [2115689,105735] 0 2024-09-10T11:29:45.114 INFO:tasks.workunit.client.0.smithi144.stdout:9/743: dwrite d11/d12/d1b/fc9 [0,4194304] 0 2024-09-10T11:29:45.117 INFO:tasks.workunit.client.0.smithi144.stdout:7/739: mkdir d5/dc/d2a/d60/dd6/d98/df5 0 2024-09-10T11:29:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:44 smithi178 ceph-mon[33514]: pgmap v29: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 150 MiB/s rd, 408 MiB/s wr, 781 op/s 2024-09-10T11:29:45.120 INFO:tasks.workunit.client.0.smithi144.stdout:7/740: creat d5/dc/d14/d40/d58/d6a/db2/ff6 x:0 0 0 2024-09-10T11:29:45.121 INFO:tasks.workunit.client.0.smithi144.stdout:9/744: dread d11/d12/d1b/f77 [0,4194304] 0 2024-09-10T11:29:45.121 INFO:tasks.workunit.client.0.smithi144.stdout:7/741: chown d5/dc/d11/d18/laf 0 1 2024-09-10T11:29:45.127 INFO:tasks.workunit.client.0.smithi144.stdout:9/745: mknod d11/d12/d1b/d2c/d78/dd2/dec/c10a 0 2024-09-10T11:29:45.129 INFO:tasks.workunit.client.0.smithi144.stdout:7/742: stat d5/dc/d2a/d60/dd6/d98/dd1/d7c/l7f 0 2024-09-10T11:29:45.132 INFO:tasks.workunit.client.0.smithi144.stdout:7/743: fsync d5/dc/d14/d40/d5e/d68/fc0 0 2024-09-10T11:29:45.133 INFO:tasks.workunit.client.0.smithi144.stdout:9/746: truncate d11/d36/db1/dbb/df0/d8f/fbd 608515 0 2024-09-10T11:29:45.135 INFO:tasks.workunit.client.0.smithi144.stdout:7/744: symlink d5/dc/d2a/d9d/lf7 0 2024-09-10T11:29:45.136 INFO:tasks.workunit.client.0.smithi144.stdout:9/747: truncate d11/d36/d2d/f8d 1261527 0 2024-09-10T11:29:45.139 INFO:tasks.workunit.client.0.smithi144.stdout:7/745: truncate d5/dc/d2a/d60/dd6/f77 565914 0 2024-09-10T11:29:45.144 INFO:tasks.workunit.client.0.smithi144.stdout:9/748: link d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/fa9 d11/d36/db1/f10b 0 2024-09-10T11:29:45.147 INFO:tasks.workunit.client.0.smithi144.stdout:9/749: getdents d11 0 2024-09-10T11:29:45.159 INFO:tasks.workunit.client.0.smithi144.stdout:7/746: truncate d5/f99 23178 0 2024-09-10T11:29:45.161 INFO:tasks.workunit.client.0.smithi144.stdout:9/750: write d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/f4a [3586532,116262] 0 2024-09-10T11:29:45.161 INFO:tasks.workunit.client.0.smithi144.stdout:7/747: creat d5/dc/d14/d40/d58/d6a/ff8 x:0 0 0 2024-09-10T11:29:45.162 INFO:tasks.workunit.client.0.smithi144.stdout:9/751: chown d11/d36/f4e 5857 1 2024-09-10T11:29:45.163 INFO:tasks.workunit.client.0.smithi144.stdout:7/748: stat d5/dc/d12/l48 0 2024-09-10T11:29:45.164 INFO:tasks.workunit.client.0.smithi144.stdout:7/749: dread - d5/dc/d2a/d60/dd6/d98/dd1/d7c/f8c zero size 2024-09-10T11:29:45.165 INFO:tasks.workunit.client.0.smithi144.stdout:9/752: truncate d11/d12/d1b/d2c/d78/fc1 1269125 0 2024-09-10T11:29:45.166 INFO:tasks.workunit.client.0.smithi144.stdout:7/750: truncate d5/dc/d12/f8e 187805 0 2024-09-10T11:29:45.167 INFO:tasks.workunit.client.0.smithi144.stdout:9/753: truncate d11/d12/d1b/ffd 669903 0 2024-09-10T11:29:45.170 INFO:tasks.workunit.client.0.smithi144.stdout:9/754: creat d11/f10c x:0 0 0 2024-09-10T11:29:45.173 INFO:tasks.workunit.client.0.smithi144.stdout:7/751: rename d5/dc/d14/d40/d58/d6a/lc1 to d5/dc/d14/lf9 0 2024-09-10T11:29:45.178 INFO:tasks.workunit.client.0.smithi144.stdout:7/752: creat d5/dc/d11/ffa x:0 0 0 2024-09-10T11:29:45.181 INFO:tasks.workunit.client.0.smithi144.stdout:7/753: symlink d5/dc/d14/d40/d5e/lfb 0 2024-09-10T11:29:45.184 INFO:tasks.workunit.client.0.smithi144.stdout:7/754: write d5/dc/d14/d40/f72 [5203480,19480] 0 2024-09-10T11:29:45.185 INFO:tasks.workunit.client.0.smithi144.stdout:9/755: chown d11/d12/d1b/d2c/d78/dd2/ff2 2669247 1 2024-09-10T11:29:45.189 INFO:tasks.workunit.client.0.smithi144.stdout:9/756: symlink d11/d36/d2d/d38/dc3/l10d 0 2024-09-10T11:29:45.203 INFO:tasks.workunit.client.0.smithi144.stdout:9/757: dread d11/d12/d6f/f8b [0,4194304] 0 2024-09-10T11:29:45.204 INFO:tasks.workunit.client.0.smithi144.stdout:9/758: readlink d11/d36/lf5 0 2024-09-10T11:29:45.207 INFO:tasks.workunit.client.0.smithi144.stdout:9/759: rmdir d11/d36/db1/dbb 39 2024-09-10T11:29:45.211 INFO:tasks.workunit.client.0.smithi144.stdout:7/755: dread d5/dc/d14/d40/f57 [0,4194304] 0 2024-09-10T11:29:45.227 INFO:tasks.workunit.client.0.smithi144.stdout:7/756: dwrite d5/dc/d12/f21 [4194304,4194304] 0 2024-09-10T11:29:45.228 INFO:tasks.workunit.client.0.smithi144.stdout:9/760: dread d11/d36/d2d/d38/f76 [4194304,4194304] 0 2024-09-10T11:29:45.238 INFO:tasks.workunit.client.0.smithi144.stdout:9/761: stat d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/f63 0 2024-09-10T11:29:45.239 INFO:tasks.workunit.client.0.smithi144.stdout:7/757: rename d5/dc/d14/d40/l42 to d5/dc/lfc 0 2024-09-10T11:29:45.242 INFO:tasks.workunit.client.0.smithi144.stdout:7/758: getdents d5/dc/d2a/d60/dd6 0 2024-09-10T11:29:45.245 INFO:tasks.workunit.client.0.smithi144.stdout:7/759: read d5/dc/d14/d40/d5e/d68/f89 [1708288,42779] 0 2024-09-10T11:29:45.247 INFO:tasks.workunit.client.0.smithi144.stdout:7/760: rmdir d5/dc/d11/de2 39 2024-09-10T11:29:45.248 INFO:tasks.workunit.client.0.smithi144.stdout:7/761: fdatasync d5/dc/d2a/d60/dd6/d98/dd1/f4d 0 2024-09-10T11:29:45.250 INFO:tasks.workunit.client.0.smithi144.stdout:7/762: write d5/dc/d2a/d60/dd6/d98/dd1/f4d [7461768,44051] 0 2024-09-10T11:29:45.256 INFO:tasks.workunit.client.0.smithi144.stdout:7/763: link d5/dc/d2a/d60/dd6/f66 d5/dc/d2a/d60/dd6/d98/dd1/ffd 0 2024-09-10T11:29:45.260 INFO:tasks.workunit.client.0.smithi144.stdout:7/764: rmdir d5/dc/d14/d40/d58/dd5 0 2024-09-10T11:29:45.261 INFO:tasks.workunit.client.0.smithi144.stdout:7/765: readlink d5/dc/d12/d2b/ldb 0 2024-09-10T11:29:45.262 INFO:tasks.workunit.client.0.smithi144.stdout:9/762: read - d11/d36/db1/fc6 zero size 2024-09-10T11:29:45.264 INFO:tasks.workunit.client.0.smithi144.stdout:7/766: creat d5/dc/d14/d40/d5e/dc4/ffe x:0 0 0 2024-09-10T11:29:45.265 INFO:tasks.workunit.client.0.smithi144.stdout:9/763: unlink d11/d12/d1b/f77 0 2024-09-10T11:29:45.267 INFO:tasks.workunit.client.0.smithi144.stdout:7/767: rename d5/dc/d2a/d9d/lf7 to d5/dc/d2a/d60/lff 0 2024-09-10T11:29:45.268 INFO:tasks.workunit.client.0.smithi144.stdout:7/768: chown d5/dc/d14/d40/d58/cb5 354754 1 2024-09-10T11:29:45.274 INFO:tasks.workunit.client.0.smithi144.stdout:7/769: dwrite d5/dc/d14/d40/f9f [0,4194304] 0 2024-09-10T11:29:45.284 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.283+0000 7f94c95d1700 1 -- 172.21.15.144:0/4204739296 --> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f94a4000bf0 con 0x7f94b006d3b0 2024-09-10T11:29:45.292 INFO:tasks.workunit.client.0.smithi144.stdout:7/770: truncate d5/dc/f10 4630741 0 2024-09-10T11:29:45.294 INFO:tasks.workunit.client.0.smithi144.stdout:7/771: unlink d5/dc/d14/d40/d58/caa 0 2024-09-10T11:29:45.297 INFO:tasks.workunit.client.0.smithi144.stdout:7/772: symlink d5/dc/d14/d40/d58/l100 0 2024-09-10T11:29:45.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.296+0000 7f94c0ff9700 1 -- 172.21.15.144:0/4204739296 <== mgr.24329 v2:172.21.15.178:6828/2974090415 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f94a4000bf0 con 0x7f94b006d3b0 2024-09-10T11:29:45.298 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:29:45.298 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (7m) 46s ago 8m 24.5M - 0.25.0 c8568f914cd2 b5c0bfae59de 2024-09-10T11:29:45.298 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (8m) 46s ago 8m 7680k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:29:45.298 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (7m) 39s ago 7m 7629k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:29:45.298 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (8m) 46s ago 8m 6665k - 18.2.0 dc2bc1663786 3311fb04aa0c 2024-09-10T11:29:45.298 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (7m) 39s ago 7m 6651k - 18.2.0 dc2bc1663786 6e408e8476d4 2024-09-10T11:29:45.298 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (6m) 46s ago 8m 88.6M - 9.4.7 954c08fa6188 e9a1aec955f6 2024-09-10T11:29:45.298 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (3m) 46s ago 3m 76.5M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:29:45.298 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (3m) 46s ago 3m 103M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:29:45.298 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (2m) 39s ago 2m 60.0M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:29:45.299 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (3m) 39s ago 3m 90.1M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:29:45.299 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:9283,8765,8443 running (9m) 46s ago 9m 171M - 18.2.0 dc2bc1663786 71a7e1c8a428 2024-09-10T11:29:45.299 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (66s) 39s ago 7m 480M - 19.1.1-305-g1cf819b7 41d671128357 859a740b57c8 2024-09-10T11:29:45.299 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (10m) 46s ago 10m 48.8M 2048M 18.2.0 dc2bc1663786 42addb654e4f 2024-09-10T11:29:45.299 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (7m) 39s ago 7m 40.4M 2048M 18.2.0 dc2bc1663786 3c1ba210ed3f 2024-09-10T11:29:45.299 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (8m) 46s ago 8m 16.3M - 1.5.0 0da6a335fe13 dfbf0c488d17 2024-09-10T11:29:45.299 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (7m) 39s ago 7m 15.2M - 1.5.0 0da6a335fe13 ff2cac118c8d 2024-09-10T11:29:45.299 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (6m) 46s ago 6m 78.9M 1216M 18.2.0 dc2bc1663786 2435febebdb5 2024-09-10T11:29:45.299 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (6m) 46s ago 6m 83.0M 1216M 18.2.0 dc2bc1663786 b55fe920ff9d 2024-09-10T11:29:45.299 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (5m) 46s ago 5m 70.1M 1216M 18.2.0 dc2bc1663786 3ed5618eac33 2024-09-10T11:29:45.299 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (5m) 39s ago 5m 76.3M 2239M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:29:45.299 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (4m) 39s ago 4m 70.5M 2239M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:29:45.299 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (4m) 39s ago 4m 68.7M 2239M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:29:45.300 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (6m) 46s ago 8m 65.9M - 2.43.0 a07b618ecd1d a463a804d232 2024-09-10T11:29:45.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.299+0000 7f94c95d1700 1 -- 172.21.15.144:0/4204739296 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f94b006d3b0 msgr2=0x7f94b006f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:45.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.299+0000 7f94c95d1700 1 --2- 172.21.15.144:0/4204739296 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f94b006d3b0 0x7f94b006f860 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f94c4072df0 tx=0x7f94ac00b540 comp rx=0 tx=0).stop 2024-09-10T11:29:45.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.299+0000 7f94c95d1700 1 -- 172.21.15.144:0/4204739296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94c40fe9d0 msgr2=0x7f94c4071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:45.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.299+0000 7f94c95d1700 1 --2- 172.21.15.144:0/4204739296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94c40fe9d0 0x7f94c4071d00 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f94b400ba50 tx=0x7f94b400bd60 comp rx=0 tx=0).stop 2024-09-10T11:29:45.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.299+0000 7f94c95d1700 1 -- 172.21.15.144:0/4204739296 shutdown_connections 2024-09-10T11:29:45.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.299+0000 7f94c95d1700 1 --2- 172.21.15.144:0/4204739296 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94c40fe9d0 0x7f94c4071d00 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:45.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.299+0000 7f94c95d1700 1 --2- 172.21.15.144:0/4204739296 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f94b006d3b0 0x7f94b006f860 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:45.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.299+0000 7f94c95d1700 1 --2- 172.21.15.144:0/4204739296 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94c4101330 0x7f94c4072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:45.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.299+0000 7f94c95d1700 1 -- 172.21.15.144:0/4204739296 >> 172.21.15.144:0/4204739296 conn(0x7f94c40fa5e0 msgr2=0x7f94c40fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:45.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.299+0000 7f94c95d1700 1 -- 172.21.15.144:0/4204739296 shutdown_connections 2024-09-10T11:29:45.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.299+0000 7f94c95d1700 1 -- 172.21.15.144:0/4204739296 wait complete. 2024-09-10T11:29:45.303 INFO:tasks.workunit.client.0.smithi144.stdout:7/773: write d5/dc/f3c [535647,68848] 0 2024-09-10T11:29:45.305 INFO:tasks.workunit.client.0.smithi144.stdout:7/774: fsync d5/dc/d11/f3d 0 2024-09-10T11:29:45.306 INFO:tasks.workunit.client.0.smithi144.stdout:7/775: stat d5/dc/d11/d18/f34 0 2024-09-10T11:29:45.307 INFO:tasks.workunit.client.0.smithi144.stdout:7/776: fdatasync d5/dc/d12/d2b/fc2 0 2024-09-10T11:29:45.310 INFO:tasks.workunit.client.0.smithi144.stdout:7/777: rmdir d5/dc/d11 39 2024-09-10T11:29:45.313 INFO:tasks.workunit.client.0.smithi144.stdout:7/778: unlink d5/dc/d12/f8e 0 2024-09-10T11:29:45.313 INFO:tasks.workunit.client.0.smithi144.stdout:7/779: fsync d5/dc/d14/d40/d5e/dc4/fcf 0 2024-09-10T11:29:45.318 INFO:tasks.workunit.client.0.smithi144.stdout:7/780: truncate d5/dc/d12/d2b/fc2 208778 0 2024-09-10T11:29:45.323 INFO:tasks.workunit.client.0.smithi144.stdout:7/781: rename d5/dc/d14/d40/d58/d6a/ca1 to d5/dc/d14/d40/d5e/d68/c101 0 2024-09-10T11:29:45.325 INFO:tasks.workunit.client.0.smithi144.stdout:7/782: write d5/dc/d2a/d41/f49 [3247371,24555] 0 2024-09-10T11:29:45.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.417+0000 7f954a159700 1 -- 172.21.15.144:0/2414613174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95440ff050 msgr2=0x7f95440ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:45.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.417+0000 7f954a159700 1 --2- 172.21.15.144:0/2414613174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95440ff050 0x7f95440ff460 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f9534009910 tx=0x7f9534009c20 comp rx=0 tx=0).stop 2024-09-10T11:29:45.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.418+0000 7f954a159700 1 -- 172.21.15.144:0/2414613174 shutdown_connections 2024-09-10T11:29:45.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.418+0000 7f954a159700 1 --2- 172.21.15.144:0/2414613174 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9544100250 0x7f95441006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:45.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.418+0000 7f954a159700 1 --2- 172.21.15.144:0/2414613174 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95440ff050 0x7f95440ff460 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:45.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.418+0000 7f954a159700 1 -- 172.21.15.144:0/2414613174 >> 172.21.15.144:0/2414613174 conn(0x7f95440fa600 msgr2=0x7f95440fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:45.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.418+0000 7f954a159700 1 -- 172.21.15.144:0/2414613174 shutdown_connections 2024-09-10T11:29:45.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.418+0000 7f954a159700 1 -- 172.21.15.144:0/2414613174 wait complete. 2024-09-10T11:29:45.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.418+0000 7f954a159700 1 Processor -- start 2024-09-10T11:29:45.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.419+0000 7f954a159700 1 -- start start 2024-09-10T11:29:45.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.419+0000 7f954a159700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95440ff050 0x7f9544195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:45.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.419+0000 7f954a159700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9544100250 0x7f9544195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:45.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.419+0000 7f954a159700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95441964d0 con 0x7f9544100250 2024-09-10T11:29:45.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.419+0000 7f954a159700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9544196610 con 0x7f95440ff050 2024-09-10T11:29:45.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.419+0000 7f95437fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95440ff050 0x7f9544195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:45.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.419+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9544100250 0x7f9544195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:45.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.419+0000 7f95437fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95440ff050 0x7f9544195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.144:35564/0 (socket says 172.21.15.144:35564) 2024-09-10T11:29:45.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.419+0000 7f95437fe700 1 -- 172.21.15.144:0/3828984594 learned_addr learned my addr 172.21.15.144:0/3828984594 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:29:45.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.419+0000 7f9542ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9544100250 0x7f9544195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44658/0 (socket says 172.21.15.144:44658) 2024-09-10T11:29:45.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.419+0000 7f95437fe700 1 -- 172.21.15.144:0/3828984594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9544100250 msgr2=0x7f9544195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:45.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.419+0000 7f95437fe700 1 --2- 172.21.15.144:0/3828984594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9544100250 0x7f9544195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:45.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.419+0000 7f95437fe700 1 -- 172.21.15.144:0/3828984594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95340095f0 con 0x7f95440ff050 2024-09-10T11:29:45.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.419+0000 7f95437fe700 1 --2- 172.21.15.144:0/3828984594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95440ff050 0x7f9544195970 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9534000c00 tx=0x7f953400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:45.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.420+0000 7f9540ff9700 1 -- 172.21.15.144:0/3828984594 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f953401d070 con 0x7f95440ff050 2024-09-10T11:29:45.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.420+0000 7f954a159700 1 -- 172.21.15.144:0/3828984594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9544101470 con 0x7f95440ff050 2024-09-10T11:29:45.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.420+0000 7f9540ff9700 1 -- 172.21.15.144:0/3828984594 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9534005dc0 con 0x7f95440ff050 2024-09-10T11:29:45.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.420+0000 7f9540ff9700 1 -- 172.21.15.144:0/3828984594 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9534022470 con 0x7f95440ff050 2024-09-10T11:29:45.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.420+0000 7f954a159700 1 -- 172.21.15.144:0/3828984594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9544101960 con 0x7f95440ff050 2024-09-10T11:29:45.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.421+0000 7f954a159700 1 -- 172.21.15.144:0/3828984594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9544107ba0 con 0x7f95440ff050 2024-09-10T11:29:45.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.421+0000 7f9540ff9700 1 -- 172.21.15.144:0/3828984594 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 25) v1 ==== 91089+0+0 (secure 0 0 0) 0x7f953400fa70 con 0x7f95440ff050 2024-09-10T11:29:45.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.422+0000 7f9540ff9700 1 --2- 172.21.15.144:0/3828984594 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f952c06d3c0 0x7f952c06f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:45.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.422+0000 7f9540ff9700 1 -- 172.21.15.144:0/3828984594 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f953408e040 con 0x7f95440ff050 2024-09-10T11:29:45.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.422+0000 7f9542ffd700 1 --2- 172.21.15.144:0/3828984594 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f952c06d3c0 0x7f952c06f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:45.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.422+0000 7f9542ffd700 1 --2- 172.21.15.144:0/3828984594 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f952c06d3c0 0x7f952c06f870 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9538010d80 tx=0x7f9538012040 comp rx=0 tx=0).ready entity=mgr.24329 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:45.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.426+0000 7f9540ff9700 1 -- 172.21.15.144:0/3828984594 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f953405b430 con 0x7f95440ff050 2024-09-10T11:29:45.588 INFO:tasks.workunit.client.0.smithi144.stdout:8/644: sync 2024-09-10T11:29:45.591 INFO:tasks.workunit.client.0.smithi144.stdout:8/645: mknod d0/d9/d21/d7e/cdd 0 2024-09-10T11:29:45.594 INFO:tasks.workunit.client.0.smithi144.stdout:8/646: creat d0/d9/d21/d25/db3/dc2/fde x:0 0 0 2024-09-10T11:29:45.596 INFO:tasks.workunit.client.0.smithi144.stdout:8/647: creat d0/d9/d21/fdf x:0 0 0 2024-09-10T11:29:45.598 INFO:tasks.workunit.client.0.smithi144.stdout:8/648: creat d0/d9/d23/fe0 x:0 0 0 2024-09-10T11:29:45.604 INFO:tasks.workunit.client.0.smithi144.stdout:8/649: dwrite d0/d9/d23/d26/f4e [0,4194304] 0 2024-09-10T11:29:45.606 INFO:tasks.workunit.client.0.smithi144.stdout:8/650: chown d0/d1b/d47/d94/f9d 1 1 2024-09-10T11:29:45.612 INFO:tasks.workunit.client.0.smithi144.stdout:8/651: truncate d0/d9/d21/d7e/d85/fb9 2083032 0 2024-09-10T11:29:45.615 INFO:tasks.workunit.client.0.smithi144.stdout:8/652: mkdir d0/d1b/d47/d5e/d76/de1 0 2024-09-10T11:29:45.616 INFO:tasks.workunit.client.0.smithi144.stdout:8/653: chown d0/d1b/d47/d5e/d76/f7c 803007251 1 2024-09-10T11:29:45.618 INFO:tasks.workunit.client.0.smithi144.stdout:8/654: truncate d0/d1b/d47/d5e/d76/fb2 1029890 0 2024-09-10T11:29:45.620 INFO:tasks.workunit.client.0.smithi144.stdout:8/655: symlink d0/d9/d21/d25/db3/dc2/dcd/dd8/le2 0 2024-09-10T11:29:45.623 INFO:tasks.workunit.client.0.smithi144.stdout:8/656: rename d0/d9/d21/d25/d4f/d93/lcb to d0/d1b/d47/le3 0 2024-09-10T11:29:45.624 INFO:tasks.workunit.client.0.smithi144.stdout:8/657: mknod d0/d9/d23/ce4 0 2024-09-10T11:29:45.626 INFO:tasks.workunit.client.0.smithi144.stdout:8/658: write d0/d9/d1f/d34/da0/dc4/fd3 [374430,2478] 0 2024-09-10T11:29:45.632 INFO:tasks.workunit.client.0.smithi144.stdout:8/659: dwrite d0/d9/d21/d25/d4f/f6f [0,4194304] 0 2024-09-10T11:29:45.635 INFO:tasks.workunit.client.0.smithi144.stdout:8/660: rmdir d0/d9/d21/d25/d4f/d93 39 2024-09-10T11:29:45.637 INFO:tasks.workunit.client.0.smithi144.stdout:8/661: truncate d0/d9/d21/fdf 314350 0 2024-09-10T11:29:45.639 INFO:tasks.workunit.client.0.smithi144.stdout:8/662: mkdir d0/d9/d21/d7e/de5 0 2024-09-10T11:29:45.646 INFO:tasks.workunit.client.0.smithi144.stdout:8/663: dwrite d0/d9/d1f/d34/da0/dc4/fd3 [0,4194304] 0 2024-09-10T11:29:45.649 INFO:tasks.workunit.client.0.smithi144.stdout:8/664: symlink d0/d9/d21/d25/d4f/le6 0 2024-09-10T11:29:45.652 INFO:tasks.workunit.client.0.smithi144.stdout:8/665: creat d0/d9/d21/d25/db3/fe7 x:0 0 0 2024-09-10T11:29:45.657 INFO:tasks.workunit.client.0.smithi144.stdout:8/666: getdents d0/d9/d23 0 2024-09-10T11:29:45.663 INFO:tasks.workunit.client.0.smithi144.stdout:8/667: dwrite d0/d9/d23/d26/d7d/f80 [0,4194304] 0 2024-09-10T11:29:45.664 INFO:tasks.workunit.client.0.smithi144.stdout:8/668: readlink d0/d9/d23/d37/lae 0 2024-09-10T11:29:45.666 INFO:tasks.workunit.client.0.smithi144.stdout:8/669: mkdir d0/d9/d21/d25/db3/de8 0 2024-09-10T11:29:45.707 INFO:tasks.workunit.client.0.smithi144.stdout:4/757: sync 2024-09-10T11:29:45.720 INFO:tasks.workunit.client.0.smithi144.stdout:4/758: dread d14/d1e/d22/d34/d3e/d40/d4b/d4c/f79 [0,4194304] 0 2024-09-10T11:29:45.724 INFO:tasks.workunit.client.0.smithi144.stdout:4/759: getdents d14/d1e/d83/da4/da9/dbc 0 2024-09-10T11:29:45.731 INFO:tasks.workunit.client.0.smithi144.stdout:4/760: dwrite d14/d1e/d22/d34/d3e/d40/f10f [0,4194304] 0 2024-09-10T11:29:45.736 INFO:tasks.workunit.client.0.smithi144.stdout:4/761: link d14/d67/d84/db3/fee d14/dcb/f114 0 2024-09-10T11:29:45.738 INFO:tasks.workunit.client.0.smithi144.stdout:4/762: dread - d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/fb9 zero size 2024-09-10T11:29:45.744 INFO:tasks.workunit.client.0.smithi144.stdout:4/763: dwrite d14/d1e/fb6 [0,4194304] 0 2024-09-10T11:29:45.748 INFO:tasks.workunit.client.0.smithi144.stdout:4/764: symlink d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/l115 0 2024-09-10T11:29:45.752 INFO:tasks.workunit.client.0.smithi144.stdout:4/765: unlink d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/f87 0 2024-09-10T11:29:45.756 INFO:tasks.workunit.client.0.smithi144.stdout:4/766: dread d14/d1e/d22/d34/d3e/d40/f10f [0,4194304] 0 2024-09-10T11:29:45.761 INFO:tasks.workunit.client.0.smithi144.stdout:4/767: mknod d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/d10b/c116 0 2024-09-10T11:29:45.768 INFO:tasks.workunit.client.0.smithi144.stdout:4/768: dwrite d14/d67/d92/fc3 [0,4194304] 0 2024-09-10T11:29:45.772 INFO:tasks.workunit.client.0.smithi144.stdout:4/769: link d14/d1e/f32 d14/dcb/f117 0 2024-09-10T11:29:45.775 INFO:tasks.workunit.client.0.smithi144.stdout:4/770: creat d14/d1e/d83/da4/f118 x:0 0 0 2024-09-10T11:29:45.780 INFO:tasks.workunit.client.0.smithi144.stdout:4/771: write d14/dcb/f114 [80533,105368] 0 2024-09-10T11:29:45.784 INFO:tasks.workunit.client.0.smithi144.stdout:4/772: rename d14/d1e/d22/d34/d3e/d40/d4b/l7d to d14/d67/d92/l119 0 2024-09-10T11:29:45.785 INFO:tasks.workunit.client.0.smithi144.stdout:4/773: chown d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/ff6 13980 1 2024-09-10T11:29:45.791 INFO:tasks.workunit.client.0.smithi144.stdout:4/774: truncate d14/d67/d69/dca/fdb 1683602 0 2024-09-10T11:29:45.794 INFO:tasks.workunit.client.0.smithi144.stdout:4/775: rename d14/d1e/d22/d34/d91/ff7 to d14/d1e/d22/f11a 0 2024-09-10T11:29:45.795 INFO:tasks.workunit.client.0.smithi144.stdout:4/776: write d14/d67/d69/d90/dda/f107 [593000,112941] 0 2024-09-10T11:29:45.798 INFO:tasks.workunit.client.0.smithi144.stdout:4/777: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/f93 [723173,104930] 0 2024-09-10T11:29:45.799 INFO:tasks.workunit.client.0.smithi144.stdout:4/778: dread d14/d1e/d83/ffe [0,4194304] 0 2024-09-10T11:29:45.803 INFO:tasks.workunit.client.0.smithi144.stdout:4/779: rename d14/d1e/d65/d8a/d8f to d14/d1e/d83/da4/d11b 0 2024-09-10T11:29:45.806 INFO:tasks.workunit.client.0.smithi144.stdout:4/780: truncate d14/d1e/d22/d34/d3e/d40/d4b/f80 486430 0 2024-09-10T11:29:45.808 INFO:tasks.workunit.client.0.smithi144.stdout:4/781: rename d14/d7e/l9a to d14/d67/d69/d90/d102/l11c 0 2024-09-10T11:29:45.810 INFO:tasks.workunit.client.0.smithi144.stdout:4/782: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/fb8 [270781,94496] 0 2024-09-10T11:29:45.813 INFO:tasks.workunit.client.0.smithi144.stdout:4/783: mknod d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/c11d 0 2024-09-10T11:29:45.816 INFO:tasks.workunit.client.0.smithi144.stdout:4/784: link d14/d1e/l8c d14/d1e/d83/l11e 0 2024-09-10T11:29:45.818 INFO:tasks.workunit.client.0.smithi144.stdout:4/785: truncate d14/d1e/d22/f4d 153450 0 2024-09-10T11:29:45.831 INFO:tasks.workunit.client.0.smithi144.stdout:4/786: truncate d14/d6f/f96 353330 0 2024-09-10T11:29:45.835 INFO:tasks.workunit.client.0.smithi144.stdout:4/787: creat d14/d67/f11f x:0 0 0 2024-09-10T11:29:45.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.835+0000 7f954a159700 1 -- 172.21.15.144:0/3828984594 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f95441025f0 con 0x7f95440ff050 2024-09-10T11:29:45.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.835+0000 7f9540ff9700 1 -- 172.21.15.144:0/3828984594 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+770 (secure 0 0 0) 0x7f953405afc0 con 0x7f95440ff050 2024-09-10T11:29:45.836 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:29:45.836 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:29:45.837 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-09-10T11:29:45.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:29:45.837 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:29:45.837 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:29:45.837 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 1 2024-09-10T11:29:45.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:29:45.837 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:29:45.837 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-09-10T11:29:45.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:29:45.837 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:29:45.837 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:29:45.837 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:29:45.837 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:29:45.838 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 13, 2024-09-10T11:29:45.838 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 1 2024-09-10T11:29:45.838 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:29:45.838 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:29:45.838 INFO:tasks.workunit.client.0.smithi144.stdout:4/788: mknod d14/d6f/c120 0 2024-09-10T11:29:45.838 INFO:tasks.workunit.client.0.smithi144.stdout:4/789: stat d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/ff6 0 2024-09-10T11:29:45.838 INFO:tasks.workunit.client.0.smithi144.stdout:4/790: fdatasync d14/d67/ffb 0 2024-09-10T11:29:45.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.838+0000 7f954a159700 1 -- 172.21.15.144:0/3828984594 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f952c06d3c0 msgr2=0x7f952c06f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:45.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.838+0000 7f954a159700 1 --2- 172.21.15.144:0/3828984594 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f952c06d3c0 0x7f952c06f870 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9538010d80 tx=0x7f9538012040 comp rx=0 tx=0).stop 2024-09-10T11:29:45.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.838+0000 7f954a159700 1 -- 172.21.15.144:0/3828984594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95440ff050 msgr2=0x7f9544195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:45.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.838+0000 7f954a159700 1 --2- 172.21.15.144:0/3828984594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95440ff050 0x7f9544195970 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9534000c00 tx=0x7f953400f690 comp rx=0 tx=0).stop 2024-09-10T11:29:45.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.838+0000 7f954a159700 1 -- 172.21.15.144:0/3828984594 shutdown_connections 2024-09-10T11:29:45.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.838+0000 7f954a159700 1 --2- 172.21.15.144:0/3828984594 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f952c06d3c0 0x7f952c06f870 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:45.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.838+0000 7f954a159700 1 --2- 172.21.15.144:0/3828984594 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95440ff050 0x7f9544195970 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:45.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.838+0000 7f954a159700 1 --2- 172.21.15.144:0/3828984594 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9544100250 0x7f9544195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:45.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.838+0000 7f954a159700 1 -- 172.21.15.144:0/3828984594 >> 172.21.15.144:0/3828984594 conn(0x7f95440fa600 msgr2=0x7f9544068c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:45.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.838+0000 7f954a159700 1 -- 172.21.15.144:0/3828984594 shutdown_connections 2024-09-10T11:29:45.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.839+0000 7f954a159700 1 -- 172.21.15.144:0/3828984594 wait complete. 2024-09-10T11:29:45.841 INFO:tasks.workunit.client.0.smithi144.stdout:4/791: fsync d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/fd2 0 2024-09-10T11:29:45.841 INFO:tasks.workunit.client.0.smithi144.stdout:4/792: truncate d14/d7e/f109 627433 0 2024-09-10T11:29:45.844 INFO:tasks.workunit.client.0.smithi144.stdout:4/793: creat d14/d67/f121 x:0 0 0 2024-09-10T11:29:45.850 INFO:tasks.workunit.client.0.smithi144.stdout:4/794: truncate d14/d6f/f96 1141254 0 2024-09-10T11:29:45.851 INFO:tasks.workunit.client.0.smithi144.stdout:4/795: write d14/d6f/f9c [6542721,78279] 0 2024-09-10T11:29:45.893 INFO:tasks.workunit.client.0.smithi144.stdout:2/590: sync 2024-09-10T11:29:45.893 INFO:tasks.workunit.client.0.smithi144.stdout:3/728: sync 2024-09-10T11:29:45.894 INFO:tasks.workunit.client.0.smithi144.stdout:0/575: sync 2024-09-10T11:29:45.910 INFO:tasks.workunit.client.0.smithi144.stdout:2/591: chown d7/d29/c2e 9434 1 2024-09-10T11:29:45.911 INFO:tasks.workunit.client.0.smithi144.stdout:4/796: dwrite d14/d67/d84/db3/fee [0,4194304] 0 2024-09-10T11:29:45.913 INFO:tasks.workunit.client.0.smithi144.stdout:0/576: creat da/d3a/d4b/d61/d6c/fb0 x:0 0 0 2024-09-10T11:29:45.915 INFO:tasks.workunit.client.0.smithi144.stdout:2/592: symlink d7/de/d36/d45/d47/d6c/d7c/d82/lc2 0 2024-09-10T11:29:45.916 INFO:tasks.workunit.client.0.smithi144.stdout:0/577: unlink da/f36 0 2024-09-10T11:29:45.918 INFO:tasks.workunit.client.0.smithi144.stdout:3/729: getdents d4/d28/d67/d80 0 2024-09-10T11:29:45.919 INFO:tasks.workunit.client.0.smithi144.stdout:0/578: mkdir da/d3a/d94/db1 0 2024-09-10T11:29:45.920 INFO:tasks.workunit.client.0.smithi144.stdout:0/579: dread - da/d3a/d4b/d4e/d90/fa6 zero size 2024-09-10T11:29:45.921 INFO:tasks.workunit.client.0.smithi144.stdout:3/730: truncate d4/d1a/f3b 262336 0 2024-09-10T11:29:45.923 INFO:tasks.workunit.client.0.smithi144.stdout:0/580: symlink da/d3a/d4b/d4e/d90/lb2 0 2024-09-10T11:29:45.924 INFO:tasks.workunit.client.0.smithi144.stdout:0/581: fdatasync da/d3a/d4b/d61/d6c/fa1 0 2024-09-10T11:29:45.924 INFO:tasks.workunit.client.0.smithi144.stdout:0/582: fdatasync da/f26 0 2024-09-10T11:29:45.925 INFO:tasks.workunit.client.0.smithi144.stdout:0/583: read - da/d3a/f93 zero size 2024-09-10T11:29:45.928 INFO:tasks.workunit.client.0.smithi144.stdout:3/731: truncate d4/d28/d67/d80/d17/f30 6863099 0 2024-09-10T11:29:45.929 INFO:tasks.workunit.client.0.smithi144.stdout:0/584: write da/d3a/d4b/d4e/d81/f98 [507131,69169] 0 2024-09-10T11:29:45.930 INFO:tasks.workunit.client.0.smithi144.stdout:0/585: write da/f26 [9037141,100598] 0 2024-09-10T11:29:45.933 INFO:tasks.workunit.client.0.smithi144.stdout:0/586: readlink da/l1f 0 2024-09-10T11:29:45.933 INFO:tasks.workunit.client.0.smithi144.stdout:2/593: write d7/de/f37 [1140844,38095] 0 2024-09-10T11:29:45.935 INFO:tasks.workunit.client.0.smithi144.stdout:2/594: dread - d7/de/d36/d45/f84 zero size 2024-09-10T11:29:45.937 INFO:tasks.workunit.client.0.smithi144.stdout:2/595: rmdir d7/de/d36/d45/d47 39 2024-09-10T11:29:45.939 INFO:tasks.workunit.client.0.smithi144.stdout:0/587: getdents da/d3a/d4b 0 2024-09-10T11:29:45.941 INFO:tasks.workunit.client.0.smithi144.stdout:0/588: write da/d3a/d4b/d61/d6c/fa1 [1838067,126760] 0 2024-09-10T11:29:45.943 INFO:tasks.workunit.client.0.smithi144.stdout:0/589: rmdir da/d3a/d94 39 2024-09-10T11:29:45.944 INFO:tasks.workunit.client.0.smithi144.stdout:2/596: getdents d7/de/d36/d45/d8e/d9a/d54/d61 0 2024-09-10T11:29:45.945 INFO:tasks.workunit.client.0.smithi144.stdout:0/590: chown da/d44/d82/l8a 48 1 2024-09-10T11:29:45.945 INFO:tasks.workunit.client.0.smithi144.stdout:2/597: stat d7/d42/d64/d8d 0 2024-09-10T11:29:45.945 INFO:tasks.workunit.client.0.smithi144.stdout:3/732: dread d4/d28/d67/d80/d17/d18/f4c [0,4194304] 0 2024-09-10T11:29:45.947 INFO:tasks.workunit.client.0.smithi144.stdout:0/591: mknod da/d44/cb3 0 2024-09-10T11:29:45.947 INFO:tasks.workunit.client.0.smithi144.stdout:2/598: rmdir d7/da/d5b/d80 39 2024-09-10T11:29:45.951 INFO:tasks.workunit.client.0.smithi144.stdout:3/733: rename d4/dd/d21/d2f/l9d to d4/d28/d67/d80/d17/d18/d34/d65/le8 0 2024-09-10T11:29:45.953 INFO:tasks.workunit.client.0.smithi144.stdout:0/592: getdents da/d3a/d4b/d4e/d81 0 2024-09-10T11:29:45.957 INFO:tasks.workunit.client.0.smithi144.stdout:2/599: dwrite d7/de/f2f [4194304,4194304] 0 2024-09-10T11:29:45.962 INFO:tasks.workunit.client.0.smithi144.stdout:2/600: symlink d7/d42/d64/d8d/lc3 0 2024-09-10T11:29:45.964 INFO:tasks.workunit.client.0.smithi144.stdout:3/734: dwrite d4/dd/d21/f5d [0,4194304] 0 2024-09-10T11:29:45.967 INFO:tasks.workunit.client.0.smithi144.stdout:2/601: chown d7/de/d36/d45/d8e/d9a/d54/fb5 6 1 2024-09-10T11:29:45.968 INFO:tasks.workunit.client.0.smithi144.stdout:0/593: dwrite da/d3a/d4b/d4e/d81/f98 [0,4194304] 0 2024-09-10T11:29:45.971 INFO:tasks.workunit.client.0.smithi144.stdout:3/735: rename d4/fe1 to d4/d28/d67/d96/dcc/fe9 0 2024-09-10T11:29:45.972 INFO:tasks.workunit.client.0.smithi144.stdout:0/594: symlink da/d3a/d8e/lb4 0 2024-09-10T11:29:45.973 INFO:tasks.workunit.client.0.smithi144.stdout:3/736: stat d4/c40 0 2024-09-10T11:29:45.973 INFO:tasks.workunit.client.0.smithi144.stdout:2/602: mkdir d7/de/d3d/d4e/dc4 0 2024-09-10T11:29:45.975 INFO:tasks.workunit.client.0.smithi144.stdout:3/737: readlink d4/d28/d67/d80/d1d/l3d 0 2024-09-10T11:29:45.979 INFO:tasks.workunit.client.0.smithi144.stdout:2/603: rename d7/de/lad to d7/d42/d64/d8d/lc5 0 2024-09-10T11:29:45.982 INFO:tasks.workunit.client.0.smithi144.stdout:2/604: creat d7/de/d3d/d43/d7f/fc6 x:0 0 0 2024-09-10T11:29:45.984 INFO:tasks.workunit.client.0.smithi144.stdout:3/738: link d4/d28/d67/d80/d17/d18/d34/d78/l7e d4/d28/d67/d80/d17/lea 0 2024-09-10T11:29:45.985 INFO:tasks.workunit.client.0.smithi144.stdout:3/739: chown d4/d28/d67/d80/d17/d18/f43 969833 1 2024-09-10T11:29:45.989 INFO:tasks.workunit.client.0.smithi144.stdout:2/605: dread d7/d29/f60 [4194304,4194304] 0 2024-09-10T11:29:45.992 INFO:tasks.workunit.client.0.smithi144.stdout:2/606: unlink d7/da/fd 0 2024-09-10T11:29:45.993 INFO:tasks.workunit.client.0.smithi144.stdout:0/595: dread da/f2e [0,4194304] 0 2024-09-10T11:29:45.993 INFO:tasks.workunit.client.0.smithi144.stdout:2/607: write d7/da/fb6 [633226,124203] 0 2024-09-10T11:29:45.995 INFO:tasks.workunit.client.0.smithi144.stdout:0/596: symlink da/d3a/d4b/d61/lb5 0 2024-09-10T11:29:45.998 INFO:tasks.workunit.client.0.smithi144.stdout:2/608: creat d7/de/d36/d45/d47/fc7 x:0 0 0 2024-09-10T11:29:45.998 INFO:tasks.workunit.client.0.smithi144.stdout:0/597: rmdir da/d3a/d4b 39 2024-09-10T11:29:45.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.997+0000 7f0afa576700 1 -- 172.21.15.144:0/1635141863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af41017c0 msgr2=0x7f0af4101bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:45.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.997+0000 7f0afa576700 1 --2- 172.21.15.144:0/1635141863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af41017c0 0x7f0af4101bd0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009910 tx=0x7f0ae4009c20 comp rx=0 tx=0).stop 2024-09-10T11:29:45.999 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:45 smithi144 ceph-mon[27942]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:45.999 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:45 smithi144 ceph-mon[27942]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:46.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.997+0000 7f0afa576700 1 -- 172.21.15.144:0/1635141863 shutdown_connections 2024-09-10T11:29:46.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.997+0000 7f0afa576700 1 --2- 172.21.15.144:0/1635141863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0af41021a0 0x7f0af40fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:46.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.997+0000 7f0afa576700 1 --2- 172.21.15.144:0/1635141863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af41017c0 0x7f0af4101bd0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:46.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.997+0000 7f0afa576700 1 -- 172.21.15.144:0/1635141863 >> 172.21.15.144:0/1635141863 conn(0x7f0af40f9dd0 msgr2=0x7f0af40fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:46.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.998+0000 7f0afa576700 1 -- 172.21.15.144:0/1635141863 shutdown_connections 2024-09-10T11:29:46.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.998+0000 7f0afa576700 1 -- 172.21.15.144:0/1635141863 wait complete. 2024-09-10T11:29:46.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.999+0000 7f0afa576700 1 Processor -- start 2024-09-10T11:29:46.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.999+0000 7f0afa576700 1 -- start start 2024-09-10T11:29:46.001 INFO:tasks.workunit.client.0.smithi144.stdout:2/609: readlink d7/d42/d64/d8d/lc3 0 2024-09-10T11:29:46.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.999+0000 7f0afa576700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0af41017c0 0x7f0af4195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:46.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.999+0000 7f0afa576700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af41021a0 0x7f0af4195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:46.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.999+0000 7f0afa576700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af4196590 con 0x7f0af41021a0 2024-09-10T11:29:46.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:45.999+0000 7f0afa576700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af41966d0 con 0x7f0af41017c0 2024-09-10T11:29:46.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.000+0000 7f0aebfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af41021a0 0x7f0af4195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:46.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.000+0000 7f0aebfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af41021a0 0x7f0af4195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44690/0 (socket says 172.21.15.144:44690) 2024-09-10T11:29:46.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.000+0000 7f0aebfff700 1 -- 172.21.15.144:0/4064374909 learned_addr learned my addr 172.21.15.144:0/4064374909 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:29:46.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.000+0000 7f0af3fff700 1 --2- 172.21.15.144:0/4064374909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0af41017c0 0x7f0af4195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:46.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.000+0000 7f0aebfff700 1 -- 172.21.15.144:0/4064374909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0af41017c0 msgr2=0x7f0af4195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:46.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.000+0000 7f0aebfff700 1 --2- 172.21.15.144:0/4064374909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0af41017c0 0x7f0af4195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:46.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.000+0000 7f0aebfff700 1 -- 172.21.15.144:0/4064374909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae40095f0 con 0x7f0af41021a0 2024-09-10T11:29:46.003 INFO:tasks.workunit.client.0.smithi144.stdout:0/598: unlink da/d3a/f76 0 2024-09-10T11:29:46.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.001+0000 7f0aebfff700 1 --2- 172.21.15.144:0/4064374909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af41021a0 0x7f0af4195f70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00e8f0 tx=0x7f0adc00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:46.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.001+0000 7f0af1ffb700 1 -- 172.21.15.144:0/4064374909 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0adc009790 con 0x7f0af41021a0 2024-09-10T11:29:46.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.001+0000 7f0af1ffb700 1 -- 172.21.15.144:0/4064374909 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0adc00c810 con 0x7f0af41021a0 2024-09-10T11:29:46.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.001+0000 7f0afa576700 1 -- 172.21.15.144:0/4064374909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af419b180 con 0x7f0af41021a0 2024-09-10T11:29:46.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.001+0000 7f0afa576700 1 -- 172.21.15.144:0/4064374909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af419b6d0 con 0x7f0af41021a0 2024-09-10T11:29:46.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.002+0000 7f0af1ffb700 1 -- 172.21.15.144:0/4064374909 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0adc0055b0 con 0x7f0af41021a0 2024-09-10T11:29:46.004 INFO:tasks.workunit.client.0.smithi144.stdout:2/610: mkdir d7/de/d3d/dc8 0 2024-09-10T11:29:46.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.003+0000 7f0afa576700 1 -- 172.21.15.144:0/4064374909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad4005320 con 0x7f0af41021a0 2024-09-10T11:29:46.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.004+0000 7f0af1ffb700 1 -- 172.21.15.144:0/4064374909 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 25) v1 ==== 91089+0+0 (secure 0 0 0) 0x7f0adc021440 con 0x7f0af41021a0 2024-09-10T11:29:46.006 INFO:tasks.workunit.client.0.smithi144.stdout:3/740: dread d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/f74 [0,4194304] 0 2024-09-10T11:29:46.006 INFO:tasks.workunit.client.0.smithi144.stdout:0/599: truncate da/d3a/d4b/d61/f9a 965892 0 2024-09-10T11:29:46.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.004+0000 7f0af1ffb700 1 --2- 172.21.15.144:0/4064374909 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f0ae006d410 0x7f0ae006f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:29:46.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.005+0000 7f0af1ffb700 1 -- 172.21.15.144:0/4064374909 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0adc014070 con 0x7f0af41021a0 2024-09-10T11:29:46.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.005+0000 7f0af3fff700 1 --2- 172.21.15.144:0/4064374909 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f0ae006d410 0x7f0ae006f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:29:46.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.006+0000 7f0af3fff700 1 --2- 172.21.15.144:0/4064374909 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f0ae006d410 0x7f0ae006f8c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009fd0 tx=0x7f0ae4005a90 comp rx=0 tx=0).ready entity=mgr.24329 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:29:46.008 INFO:tasks.workunit.client.0.smithi144.stdout:3/741: symlink d4/d28/d67/d96/leb 0 2024-09-10T11:29:46.009 INFO:tasks.workunit.client.0.smithi144.stdout:3/742: write f2 [1188409,122376] 0 2024-09-10T11:29:46.010 INFO:tasks.workunit.client.0.smithi144.stdout:0/600: symlink da/d49/lb6 0 2024-09-10T11:29:46.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.009+0000 7f0af1ffb700 1 -- 172.21.15.144:0/4064374909 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f0adc05a270 con 0x7f0af41021a0 2024-09-10T11:29:46.011 INFO:tasks.workunit.client.0.smithi144.stdout:0/601: chown da/d44/f54 796 1 2024-09-10T11:29:46.013 INFO:tasks.workunit.client.0.smithi144.stdout:0/602: write da/d3a/d4b/d61/d6c/f9d [746683,22581] 0 2024-09-10T11:29:46.015 INFO:tasks.workunit.client.0.smithi144.stdout:3/743: fdatasync d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/f74 0 2024-09-10T11:29:46.015 INFO:tasks.workunit.client.0.smithi144.stdout:0/603: read f0 [425245,126335] 0 2024-09-10T11:29:46.017 INFO:tasks.workunit.client.0.smithi144.stdout:0/604: write da/d3a/d4b/d4e/d90/fa6 [49171,65723] 0 2024-09-10T11:29:46.023 INFO:tasks.workunit.client.0.smithi144.stdout:0/605: dwrite da/d3a/f93 [0,4194304] 0 2024-09-10T11:29:46.029 INFO:tasks.workunit.client.0.smithi144.stdout:0/606: dwrite da/d3a/d4b/d61/d6c/f9b [0,4194304] 0 2024-09-10T11:29:46.036 INFO:tasks.workunit.client.0.smithi144.stdout:0/607: dwrite da/d3a/d51/d5f/f6d [0,4194304] 0 2024-09-10T11:29:46.039 INFO:tasks.workunit.client.0.smithi144.stdout:0/608: truncate da/d3a/d4b/d4e/f60 891558 0 2024-09-10T11:29:46.067 INFO:tasks.workunit.client.0.smithi144.stdout:3/744: chown d4/dd/l15 78497 1 2024-09-10T11:29:46.068 INFO:tasks.workunit.client.0.smithi144.stdout:3/745: write d4/d1a/fc7 [3097129,19950] 0 2024-09-10T11:29:46.070 INFO:tasks.workunit.client.0.smithi144.stdout:3/746: write d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/f7b [4248956,80857] 0 2024-09-10T11:29:46.111 INFO:tasks.workunit.client.0.smithi144.stdout:1/827: sync 2024-09-10T11:29:46.111 INFO:tasks.workunit.client.0.smithi144.stdout:6/662: sync 2024-09-10T11:29:46.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:45 smithi178 ceph-mon[33514]: from='client.24401 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:46.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:45 smithi178 ceph-mon[33514]: from='client.14640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:46.131 INFO:tasks.workunit.client.0.smithi144.stdout:0/609: creat da/d3a/d4b/fb7 x:0 0 0 2024-09-10T11:29:46.132 INFO:tasks.workunit.client.0.smithi144.stdout:1/828: unlink d5/dd/f48 0 2024-09-10T11:29:46.134 INFO:tasks.workunit.client.0.smithi144.stdout:3/747: dwrite d4/d28/f36 [0,4194304] 0 2024-09-10T11:29:46.136 INFO:tasks.workunit.client.0.smithi144.stdout:1/829: fsync d5/d2a/d65/d73/fb1 0 2024-09-10T11:29:46.137 INFO:tasks.workunit.client.0.smithi144.stdout:6/663: truncate d9/d1f/f36 2016473 0 2024-09-10T11:29:46.139 INFO:tasks.workunit.client.0.smithi144.stdout:1/830: truncate d5/d2a/f102 950103 0 2024-09-10T11:29:46.141 INFO:tasks.workunit.client.0.smithi144.stdout:1/831: dread - d5/dd/da1/fe1 zero size 2024-09-10T11:29:46.145 INFO:tasks.workunit.client.0.smithi144.stdout:1/832: chown d5/d2a/f7e 7193698 1 2024-09-10T11:29:46.146 INFO:tasks.workunit.client.0.smithi144.stdout:0/610: dread da/f32 [0,4194304] 0 2024-09-10T11:29:46.150 INFO:tasks.workunit.client.0.smithi144.stdout:3/748: dread d4/d28/d67/d80/d1d/f97 [0,4194304] 0 2024-09-10T11:29:46.151 INFO:tasks.workunit.client.0.smithi144.stdout:6/664: dwrite d9/dd/d11/d1b/fd2 [0,4194304] 0 2024-09-10T11:29:46.152 INFO:tasks.workunit.client.0.smithi144.stdout:1/833: rename d5/d4a/d63/d84 to d5/df/d4b/d51/d10a 0 2024-09-10T11:29:46.154 INFO:tasks.workunit.client.0.smithi144.stdout:0/611: symlink da/d3a/d4b/d4e/lb8 0 2024-09-10T11:29:46.161 INFO:tasks.workunit.client.0.smithi144.stdout:1/834: dread d5/dd/d64/f7a [0,4194304] 0 2024-09-10T11:29:46.163 INFO:tasks.workunit.client.0.smithi144.stdout:6/665: write d9/d1f/f37 [1043875,44131] 0 2024-09-10T11:29:46.163 INFO:tasks.workunit.client.0.smithi144.stdout:1/835: write d5/dd/dcb/feb [388186,7067] 0 2024-09-10T11:29:46.163 INFO:tasks.workunit.client.0.smithi144.stdout:0/612: write da/d3a/f3b [888243,8215] 0 2024-09-10T11:29:46.164 INFO:tasks.workunit.client.0.smithi144.stdout:6/666: readlink d9/dd/d3e/l7c 0 2024-09-10T11:29:46.167 INFO:tasks.workunit.client.0.smithi144.stdout:0/613: mknod da/d3a/d94/db1/cb9 0 2024-09-10T11:29:46.172 INFO:tasks.workunit.client.0.smithi144.stdout:6/667: write d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/f19 [85906,84933] 0 2024-09-10T11:29:46.172 INFO:tasks.workunit.client.0.smithi144.stdout:1/836: getdents d5/d2a/d65/d73/dd8 0 2024-09-10T11:29:46.172 INFO:tasks.workunit.client.0.smithi144.stdout:3/749: dread d4/d28/d67/d80/d17/dd0/faa [0,4194304] 0 2024-09-10T11:29:46.178 INFO:tasks.workunit.client.0.smithi144.stdout:3/750: write d4/d28/d67/d80/d17/d18/f48 [2070821,84949] 0 2024-09-10T11:29:46.179 INFO:tasks.workunit.client.0.smithi144.stdout:0/614: dwrite da/d3a/d4b/d61/f9a [0,4194304] 0 2024-09-10T11:29:46.179 INFO:tasks.workunit.client.0.smithi144.stdout:6/668: mkdir d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d60/def 0 2024-09-10T11:29:46.182 INFO:tasks.workunit.client.0.smithi144.stdout:1/837: dread d5/df/d4b/d51/d10a/dab/fb2 [0,4194304] 0 2024-09-10T11:29:46.185 INFO:tasks.workunit.client.0.smithi144.stdout:0/615: mknod da/d3a/d51/cba 0 2024-09-10T11:29:46.186 INFO:tasks.workunit.client.0.smithi144.stdout:6/669: mkdir d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/df0 0 2024-09-10T11:29:46.188 INFO:tasks.workunit.client.0.smithi144.stdout:6/670: write d9/dd/f49 [1851948,74083] 0 2024-09-10T11:29:46.189 INFO:tasks.workunit.client.0.smithi144.stdout:0/616: mkdir da/d3a/d4b/dbb 0 2024-09-10T11:29:46.190 INFO:tasks.workunit.client.0.smithi144.stdout:1/838: link d5/df/d4b/d51/d10a/dab/fb2 d5/df/d4b/dcf/f10b 0 2024-09-10T11:29:46.191 INFO:tasks.workunit.client.0.smithi144.stdout:1/839: fsync d5/df/d4b/d85/ff2 0 2024-09-10T11:29:46.194 INFO:tasks.workunit.client.0.smithi144.stdout:3/751: truncate d4/d28/d67/f7c 1137514 0 2024-09-10T11:29:46.194 INFO:tasks.workunit.client.0.smithi144.stdout:1/840: symlink d5/d2a/d65/l10c 0 2024-09-10T11:29:46.195 INFO:tasks.workunit.client.0.smithi144.stdout:0/617: creat da/d3a/d4b/d96/fbc x:0 0 0 2024-09-10T11:29:46.196 INFO:tasks.workunit.client.0.smithi144.stdout:6/671: read d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/f5a [462353,103486] 0 2024-09-10T11:29:46.196 INFO:tasks.workunit.client.0.smithi144.stdout:3/752: mkdir d4/d28/d67/d96/dcc/dec 0 2024-09-10T11:29:46.196 INFO:tasks.workunit.client.0.smithi144.stdout:1/841: readlink d5/df/d4b/d51/da7/l81 0 2024-09-10T11:29:46.197 INFO:tasks.workunit.client.0.smithi144.stdout:0/618: chown da/d3a/d4b/d4e/d90/f71 104478 1 2024-09-10T11:29:46.198 INFO:tasks.workunit.client.0.smithi144.stdout:6/672: readlink d9/dd/d11/d1b/d2f/d7f/dc4/d5b/ld8 0 2024-09-10T11:29:46.200 INFO:tasks.workunit.client.0.smithi144.stdout:1/842: dread d5/df/d4b/d51/d10a/dbd/df1/fea [0,4194304] 0 2024-09-10T11:29:46.202 INFO:tasks.workunit.client.0.smithi144.stdout:0/619: getdents da/d44 0 2024-09-10T11:29:46.203 INFO:tasks.workunit.client.0.smithi144.stdout:6/673: creat d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d87/ff1 x:0 0 0 2024-09-10T11:29:46.203 INFO:tasks.workunit.client.0.smithi144.stdout:1/843: fdatasync d5/d4a/d44/da2/fe3 0 2024-09-10T11:29:46.210 INFO:tasks.workunit.client.0.smithi144.stdout:0/620: getdents da/d3a/d94 0 2024-09-10T11:29:46.211 INFO:tasks.workunit.client.0.smithi144.stdout:3/753: dread d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/f7b [0,4194304] 0 2024-09-10T11:29:46.214 INFO:tasks.workunit.client.0.smithi144.stdout:6/674: rename d9/d1f/d2c/lac to d9/dd/d11/d1b/d2f/d7f/dc4/d5b/lf2 0 2024-09-10T11:29:46.217 INFO:tasks.workunit.client.0.smithi144.stdout:6/675: creat d9/d38/ff3 x:0 0 0 2024-09-10T11:29:46.217 INFO:tasks.workunit.client.0.smithi144.stdout:1/844: truncate d5/df/d4b/d51/d10a/d95/fd1 79413 0 2024-09-10T11:29:46.219 INFO:tasks.workunit.client.0.smithi144.stdout:0/621: fdatasync da/d3a/d51/d5f/f4a 0 2024-09-10T11:29:46.220 INFO:tasks.workunit.client.0.smithi144.stdout:0/622: stat da/d3a/f4d 0 2024-09-10T11:29:46.222 INFO:tasks.workunit.client.0.smithi144.stdout:6/676: creat d9/dd/d3e/d47/ff4 x:0 0 0 2024-09-10T11:29:46.223 INFO:tasks.workunit.client.0.smithi144.stdout:1/845: dread d5/d2a/d65/dc9/f66 [0,4194304] 0 2024-09-10T11:29:46.229 INFO:tasks.workunit.client.0.smithi144.stdout:1/846: dread - d5/df/d4b/d85/ff0 zero size 2024-09-10T11:29:46.230 INFO:tasks.workunit.client.0.smithi144.stdout:3/754: truncate d4/d28/d67/d80/d17/d18/d34/f5c 524203 0 2024-09-10T11:29:46.237 INFO:tasks.workunit.client.0.smithi144.stdout:1/847: dwrite d5/dd/da1/fd4 [0,4194304] 0 2024-09-10T11:29:46.240 INFO:tasks.workunit.client.0.smithi144.stdout:3/755: mkdir d4/d28/dc1/dc4/ded 0 2024-09-10T11:29:46.241 INFO:tasks.workunit.client.0.smithi144.stdout:0/623: dread da/f19 [0,4194304] 0 2024-09-10T11:29:46.242 INFO:tasks.workunit.client.0.smithi144.stdout:1/848: symlink d5/df/d4b/d51/d10a/dbd/df1/l10d 0 2024-09-10T11:29:46.242 INFO:tasks.workunit.client.0.smithi144.stdout:0/624: chown da/l43 0 1 2024-09-10T11:29:46.245 INFO:tasks.workunit.client.0.smithi144.stdout:3/756: mkdir d4/d28/d67/d80/d17/d18/d34/d65/dee 0 2024-09-10T11:29:46.248 INFO:tasks.workunit.client.0.smithi144.stdout:0/625: rename da/d3a/f3b to da/d3a/d4b/d4e/d81/fbd 0 2024-09-10T11:29:46.251 INFO:tasks.workunit.client.0.smithi144.stdout:0/626: fsync da/d3a/d4b/d96/fbc 0 2024-09-10T11:29:46.251 INFO:tasks.workunit.client.0.smithi144.stdout:0/627: chown da 21 1 2024-09-10T11:29:46.251 INFO:tasks.workunit.client.0.smithi144.stdout:1/849: dwrite d5/d4a/d63/f9f [4194304,4194304] 0 2024-09-10T11:29:46.252 INFO:tasks.workunit.client.0.smithi144.stdout:1/850: readlink d5/l5a 0 2024-09-10T11:29:46.254 INFO:tasks.workunit.client.0.smithi144.stdout:3/757: write d4/d28/d67/d80/d17/d18/d34/d65/d75/fb0 [12443,119210] 0 2024-09-10T11:29:46.255 INFO:tasks.workunit.client.0.smithi144.stdout:1/851: fdatasync d5/dd/f5e 0 2024-09-10T11:29:46.257 INFO:tasks.workunit.client.0.smithi144.stdout:6/677: truncate d9/dd/d11/d1b/d2f/d7f/dc4/d5b/fa4 446044 0 2024-09-10T11:29:46.260 INFO:tasks.workunit.client.0.smithi144.stdout:3/758: chown d4/d28/d67/d80/d17/d18/d34/d65/l59 1365364 1 2024-09-10T11:29:46.261 INFO:tasks.workunit.client.0.smithi144.stdout:1/852: creat d5/df/d4b/d51/d10a/df9/f10e x:0 0 0 2024-09-10T11:29:46.265 INFO:tasks.workunit.client.0.smithi144.stdout:1/853: mkdir d5/dd/d9c/d10f 0 2024-09-10T11:29:46.268 INFO:tasks.workunit.client.0.smithi144.stdout:6/678: dwrite d9/dd/d11/f90 [0,4194304] 0 2024-09-10T11:29:46.269 INFO:tasks.workunit.client.0.smithi144.stdout:0/628: dread f0 [0,4194304] 0 2024-09-10T11:29:46.270 INFO:tasks.workunit.client.0.smithi144.stdout:3/759: rename d4/d28/d67/d80/d1d/fac to d4/dd/fef 0 2024-09-10T11:29:46.272 INFO:tasks.workunit.client.0.smithi144.stdout:3/760: dread - d4/dd/f6f zero size 2024-09-10T11:29:46.272 INFO:tasks.workunit.client.0.smithi144.stdout:6/679: fdatasync d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d87/f97 0 2024-09-10T11:29:46.272 INFO:tasks.workunit.client.0.smithi144.stdout:0/629: write da/d3a/d4b/d4e/d62/f68 [554307,8413] 0 2024-09-10T11:29:46.279 INFO:tasks.workunit.client.0.smithi144.stdout:3/761: rename d4/dd/d21/c47 to d4/dd/cf0 0 2024-09-10T11:29:46.286 INFO:tasks.workunit.client.0.smithi144.stdout:6/680: dwrite d9/dd/d11/d1b/d2f/d7f/fc7 [0,4194304] 0 2024-09-10T11:29:46.287 INFO:tasks.workunit.client.0.smithi144.stdout:0/630: dwrite da/f1a [0,4194304] 0 2024-09-10T11:29:46.295 INFO:tasks.workunit.client.0.smithi144.stdout:1/854: dwrite d5/df/d4b/d51/d10a/dbd/df1/fea [0,4194304] 0 2024-09-10T11:29:46.297 INFO:tasks.workunit.client.0.smithi144.stdout:0/631: write da/d3a/d4b/d61/f77 [295830,17989] 0 2024-09-10T11:29:46.298 INFO:tasks.workunit.client.0.smithi144.stdout:9/764: sync 2024-09-10T11:29:46.299 INFO:tasks.workunit.client.0.smithi144.stdout:5/752: sync 2024-09-10T11:29:46.302 INFO:tasks.workunit.client.0.smithi144.stdout:6/681: dwrite d9/dd/d11/f1c [0,4194304] 0 2024-09-10T11:29:46.304 INFO:tasks.workunit.client.0.smithi144.stdout:3/762: dread d4/dd/d21/d2f/f60 [0,4194304] 0 2024-09-10T11:29:46.306 INFO:tasks.workunit.client.0.smithi144.stdout:1/855: mknod d5/dd/d64/dad/c110 0 2024-09-10T11:29:46.307 INFO:tasks.workunit.client.0.smithi144.stdout:9/765: readlink d11/d36/db1/dbb/df0/l6e 0 2024-09-10T11:29:46.307 INFO:tasks.workunit.client.0.smithi144.stdout:0/632: symlink da/lbe 0 2024-09-10T11:29:46.310 INFO:tasks.workunit.client.0.smithi144.stdout:0/633: symlink da/d3a/d4b/d4e/d62/d78/lbf 0 2024-09-10T11:29:46.311 INFO:tasks.workunit.client.0.smithi144.stdout:3/763: rmdir d4/d28/d67/d80/d17/d18/d34/d78 39 2024-09-10T11:29:46.312 INFO:tasks.workunit.client.0.smithi144.stdout:0/634: readlink da/d3a/d94/la4 0 2024-09-10T11:29:46.313 INFO:tasks.workunit.client.0.smithi144.stdout:9/766: chown d11/d12/d1b/d2c/d78/dd2/ff2 8144 1 2024-09-10T11:29:46.315 INFO:tasks.workunit.client.0.smithi144.stdout:5/753: symlink d4/d5/d18/d67/db0/db8/led 0 2024-09-10T11:29:46.316 INFO:tasks.workunit.client.0.smithi144.stdout:6/682: truncate d9/d38/f63 566109 0 2024-09-10T11:29:46.318 INFO:tasks.workunit.client.0.smithi144.stdout:9/767: stat d11/d36/db1/dbb/df0/d8f 0 2024-09-10T11:29:46.318 INFO:tasks.workunit.client.0.smithi144.stdout:5/754: chown d4/d5/d18/d67/f65 1529590 1 2024-09-10T11:29:46.322 INFO:tasks.workunit.client.0.smithi144.stdout:0/635: getdents da/d3a 0 2024-09-10T11:29:46.324 INFO:tasks.workunit.client.0.smithi144.stdout:0/636: stat da/d3a/l87 0 2024-09-10T11:29:46.326 INFO:tasks.workunit.client.0.smithi144.stdout:3/764: rename d4/dd/d21/c39 to d4/d28/d67/d80/d17/d18/d34/d4b/cf1 0 2024-09-10T11:29:46.327 INFO:tasks.workunit.client.0.smithi144.stdout:1/856: dread d5/dd/f13 [0,4194304] 0 2024-09-10T11:29:46.329 INFO:tasks.workunit.client.0.smithi144.stdout:0/637: creat da/d3a/d4b/fc0 x:0 0 0 2024-09-10T11:29:46.330 INFO:tasks.workunit.client.0.smithi144.stdout:1/857: fdatasync d5/d2a/f7e 0 2024-09-10T11:29:46.341 INFO:tasks.workunit.client.0.smithi144.stdout:3/765: dread d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/f7b [0,4194304] 0 2024-09-10T11:29:46.343 INFO:tasks.workunit.client.0.smithi144.stdout:9/768: dread d11/d12/d1b/f61 [0,4194304] 0 2024-09-10T11:29:46.345 INFO:tasks.workunit.client.0.smithi144.stdout:1/858: dread d5/f23 [0,4194304] 0 2024-09-10T11:29:46.345 INFO:tasks.workunit.client.0.smithi144.stdout:0/638: dwrite da/d3a/d4b/d61/d6c/fa1 [0,4194304] 0 2024-09-10T11:29:46.346 INFO:tasks.workunit.client.0.smithi144.stdout:3/766: stat d4/d28/d67/d80/l6a 0 2024-09-10T11:29:46.348 INFO:tasks.workunit.client.0.smithi144.stdout:5/755: dwrite d4/d5/d18/d38/d8d/fa8 [0,4194304] 0 2024-09-10T11:29:46.349 INFO:tasks.workunit.client.0.smithi144.stdout:7/783: sync 2024-09-10T11:29:46.349 INFO:tasks.workunit.client.0.smithi144.stdout:0/639: stat da/d3a/d4b/d4e/d81/fbd 0 2024-09-10T11:29:46.352 INFO:tasks.workunit.client.0.smithi144.stdout:6/683: dwrite d9/d1f/d2c/d4f/f58 [0,4194304] 0 2024-09-10T11:29:46.353 INFO:tasks.workunit.client.0.smithi144.stdout:9/769: chown d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/f63 42 1 2024-09-10T11:29:46.356 INFO:tasks.workunit.client.0.smithi144.stdout:9/770: chown d11/d12/d1b/d2c/d78/led 3285 1 2024-09-10T11:29:46.356 INFO:tasks.workunit.client.0.smithi144.stdout:1/859: symlink d5/df/d4b/d51/d10a/d95/l111 0 2024-09-10T11:29:46.359 INFO:tasks.workunit.client.0.smithi144.stdout:5/756: write d4/fa [612007,117061] 0 2024-09-10T11:29:46.362 INFO:tasks.workunit.client.0.smithi144.stdout:0/640: creat da/d3a/d4b/d4e/d62/fc1 x:0 0 0 2024-09-10T11:29:46.363 INFO:tasks.workunit.client.0.smithi144.stdout:5/757: fsync d4/d5/d18/d38/d6d/d8b/f91 0 2024-09-10T11:29:46.364 INFO:tasks.workunit.client.0.smithi144.stdout:0/641: fsync da/d3a/d4b/d96/fbc 0 2024-09-10T11:29:46.365 INFO:tasks.workunit.client.0.smithi144.stdout:5/758: stat d4/d20/d24/d2d/d4b/l4c 0 2024-09-10T11:29:46.366 INFO:tasks.workunit.client.0.smithi144.stdout:1/860: rename d5/df/f55 to d5/dd/dcb/f112 0 2024-09-10T11:29:46.366 INFO:tasks.workunit.client.0.smithi144.stdout:1/861: chown d5/dd/da0 117444 1 2024-09-10T11:29:46.372 INFO:tasks.workunit.client.0.smithi144.stdout:9/771: getdents d11/d12/d1b/d2c/d78 0 2024-09-10T11:29:46.373 INFO:tasks.workunit.client.0.smithi144.stdout:7/784: write d5/fce [1938365,60852] 0 2024-09-10T11:29:46.374 INFO:tasks.workunit.client.0.smithi144.stdout:5/759: getdents d4/d52/dd2 0 2024-09-10T11:29:46.375 INFO:tasks.workunit.client.0.smithi144.stdout:3/767: write d4/d28/d67/d80/f6b [783434,67681] 0 2024-09-10T11:29:46.380 INFO:tasks.workunit.client.0.smithi144.stdout:3/768: mkdir d4/d28/dc1/df2 0 2024-09-10T11:29:46.386 INFO:tasks.workunit.client.0.smithi144.stdout:1/862: dread d5/df/d4b/d51/d10a/dbd/df1/fba [0,4194304] 0 2024-09-10T11:29:46.386 INFO:tasks.workunit.client.0.smithi144.stdout:9/772: dwrite d11/f2b [0,4194304] 0 2024-09-10T11:29:46.393 INFO:tasks.workunit.client.0.smithi144.stdout:0/642: dwrite da/d3a/d51/d5f/f4a [0,4194304] 0 2024-09-10T11:29:46.395 INFO:tasks.workunit.client.0.smithi144.stdout:1/863: creat d5/df/d4b/d51/d10a/dbd/f113 x:0 0 0 2024-09-10T11:29:46.396 INFO:tasks.workunit.client.0.smithi144.stdout:9/773: write d11/fa2 [3024901,104065] 0 2024-09-10T11:29:46.397 INFO:tasks.workunit.client.0.smithi144.stdout:3/769: dwrite d4/dd/d21/f5b [0,4194304] 0 2024-09-10T11:29:46.399 INFO:tasks.workunit.client.0.smithi144.stdout:1/864: fsync d5/dd/d64/dad/ffa 0 2024-09-10T11:29:46.400 INFO:tasks.workunit.client.0.smithi144.stdout:0/643: mkdir da/d3a/d51/dc2 0 2024-09-10T11:29:46.402 INFO:tasks.workunit.client.0.smithi144.stdout:0/644: write da/d44/d82/f8b [1029229,15731] 0 2024-09-10T11:29:46.403 INFO:tasks.workunit.client.0.smithi144.stdout:9/774: chown d11/d36/db1/dbb/df0/f92 472656 1 2024-09-10T11:29:46.405 INFO:tasks.workunit.client.0.smithi144.stdout:3/770: unlink d4/d28/d67/d80/d17/d18/d34/l68 0 2024-09-10T11:29:46.406 INFO:tasks.workunit.client.0.smithi144.stdout:5/760: dread d4/d5/d18/d38/d6d/d8b/f9b [0,4194304] 0 2024-09-10T11:29:46.406 INFO:tasks.workunit.client.0.smithi144.stdout:1/865: unlink d5/df/d4b/l5c 0 2024-09-10T11:29:46.409 INFO:tasks.workunit.client.0.smithi144.stdout:9/775: dread - d11/d36/db1/ffa zero size 2024-09-10T11:29:46.411 INFO:tasks.workunit.client.0.smithi144.stdout:0/645: rename da/d3a/d51/d5f/f6d to da/d3a/d4b/d4e/dae/fc3 0 2024-09-10T11:29:46.411 INFO:tasks.workunit.client.0.smithi144.stdout:0/646: stat c6 0 2024-09-10T11:29:46.414 INFO:tasks.workunit.client.0.smithi144.stdout:3/771: dwrite d4/f32 [0,4194304] 0 2024-09-10T11:29:46.415 INFO:tasks.workunit.client.0.smithi144.stdout:9/776: write d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/f74 [1558943,60096] 0 2024-09-10T11:29:46.417 INFO:tasks.workunit.client.0.smithi144.stdout:0/647: symlink da/d3a/d4b/d4e/lc4 0 2024-09-10T11:29:46.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.424+0000 7f0afa576700 1 -- 172.21.15.144:0/4064374909 --> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ad4000bf0 con 0x7f0ae006d410 2024-09-10T11:29:46.426 INFO:tasks.workunit.client.0.smithi144.stdout:3/772: dwrite d4/d28/d67/d80/d17/d18/d34/d65/d75/fb0 [0,4194304] 0 2024-09-10T11:29:46.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.426+0000 7f0af1ffb700 1 -- 172.21.15.144:0/4064374909 <== mgr.24329 v2:172.21.15.178:6828/2974090415 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f0ad4000bf0 con 0x7f0ae006d410 2024-09-10T11:29:46.428 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:29:46.428 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:29:46.428 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:29:46.428 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T11:29:46.428 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [], 2024-09-10T11:29:46.428 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "1/2 daemons upgraded", 2024-09-10T11:29:46.428 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "", 2024-09-10T11:29:46.428 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:29:46.428 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:29:46.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.429+0000 7f0afa576700 1 -- 172.21.15.144:0/4064374909 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f0ae006d410 msgr2=0x7f0ae006f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:46.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.429+0000 7f0afa576700 1 --2- 172.21.15.144:0/4064374909 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f0ae006d410 0x7f0ae006f8c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0ae4009fd0 tx=0x7f0ae4005a90 comp rx=0 tx=0).stop 2024-09-10T11:29:46.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.429+0000 7f0afa576700 1 -- 172.21.15.144:0/4064374909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af41021a0 msgr2=0x7f0af4195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:29:46.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.429+0000 7f0afa576700 1 --2- 172.21.15.144:0/4064374909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af41021a0 0x7f0af4195f70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0adc00e8f0 tx=0x7f0adc00ec00 comp rx=0 tx=0).stop 2024-09-10T11:29:46.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.429+0000 7f0afa576700 1 -- 172.21.15.144:0/4064374909 shutdown_connections 2024-09-10T11:29:46.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.429+0000 7f0afa576700 1 --2- 172.21.15.144:0/4064374909 >> [v2:172.21.15.178:6828/2974090415,v1:172.21.15.178:6829/2974090415] conn(0x7f0ae006d410 0x7f0ae006f8c0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:46.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.429+0000 7f0afa576700 1 --2- 172.21.15.144:0/4064374909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0af41017c0 0x7f0af4195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:46.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.429+0000 7f0afa576700 1 --2- 172.21.15.144:0/4064374909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0af41021a0 0x7f0af4195f70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:29:46.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.429+0000 7f0afa576700 1 -- 172.21.15.144:0/4064374909 >> 172.21.15.144:0/4064374909 conn(0x7f0af40f9dd0 msgr2=0x7f0af4104790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:29:46.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.430+0000 7f0afa576700 1 -- 172.21.15.144:0/4064374909 shutdown_connections 2024-09-10T11:29:46.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:29:46.430+0000 7f0afa576700 1 -- 172.21.15.144:0/4064374909 wait complete. 2024-09-10T11:29:46.432 INFO:tasks.workunit.client.0.smithi144.stdout:3/773: truncate d4/dd/fef 213303 0 2024-09-10T11:29:46.432 INFO:tasks.workunit.client.0.smithi144.stdout:1/866: write d5/df/f33 [4733945,26354] 0 2024-09-10T11:29:46.437 INFO:tasks.workunit.client.0.smithi144.stdout:1/867: mknod d5/df/d4b/d51/c114 0 2024-09-10T11:29:46.437 INFO:tasks.workunit.client.0.smithi144.stdout:0/648: truncate da/d3a/f3e 4302745 0 2024-09-10T11:29:46.438 INFO:tasks.workunit.client.0.smithi144.stdout:9/777: truncate d11/d36/db1/dbb/df0/d8f/f82 1262589 0 2024-09-10T11:29:46.438 INFO:tasks.workunit.client.0.smithi144.stdout:1/868: read d5/d4a/d63/f9f [7492083,15372] 0 2024-09-10T11:29:46.440 INFO:tasks.workunit.client.0.smithi144.stdout:0/649: read da/d3a/d4b/d61/f77 [26762,59630] 0 2024-09-10T11:29:46.442 INFO:tasks.workunit.client.0.smithi144.stdout:1/869: dread d5/d2a/d65/d73/fb1 [0,4194304] 0 2024-09-10T11:29:46.443 INFO:tasks.workunit.client.0.smithi144.stdout:1/870: chown d5/dd/d64/f9a 0 1 2024-09-10T11:29:46.444 INFO:tasks.workunit.client.0.smithi144.stdout:9/778: creat d11/d12/d1b/d2c/d78/dd2/f10e x:0 0 0 2024-09-10T11:29:46.445 INFO:tasks.workunit.client.0.smithi144.stdout:0/650: mkdir da/d3a/d4b/d4e/d90/dc5 0 2024-09-10T11:29:46.449 INFO:tasks.workunit.client.0.smithi144.stdout:3/774: dwrite d4/d28/d67/d9f/fbb [0,4194304] 0 2024-09-10T11:29:46.452 INFO:tasks.workunit.client.0.smithi144.stdout:1/871: creat d5/df/d4b/d51/d10a/dab/f115 x:0 0 0 2024-09-10T11:29:46.452 INFO:tasks.workunit.client.0.smithi144.stdout:9/779: symlink d11/d36/d2d/d38/l10f 0 2024-09-10T11:29:46.454 INFO:tasks.workunit.client.0.smithi144.stdout:3/775: symlink d4/d28/d67/d9f/lf3 0 2024-09-10T11:29:46.456 INFO:tasks.workunit.client.0.smithi144.stdout:9/780: readlink d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/la5 0 2024-09-10T11:29:46.466 INFO:tasks.workunit.client.0.smithi144.stdout:0/651: dread da/d3a/d4b/f86 [0,4194304] 0 2024-09-10T11:29:46.468 INFO:tasks.workunit.client.0.smithi144.stdout:0/652: truncate da/d3a/d4b/fb7 987909 0 2024-09-10T11:29:46.473 INFO:tasks.workunit.client.0.smithi144.stdout:3/776: dread d4/d28/f38 [0,4194304] 0 2024-09-10T11:29:46.476 INFO:tasks.workunit.client.0.smithi144.stdout:1/872: dread d5/dd/dcb/f112 [0,4194304] 0 2024-09-10T11:29:46.476 INFO:tasks.workunit.client.0.smithi144.stdout:9/781: dread d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/f63 [0,4194304] 0 2024-09-10T11:29:46.478 INFO:tasks.workunit.client.0.smithi144.stdout:0/653: dwrite da/d3a/d4b/d4e/d90/fa6 [0,4194304] 0 2024-09-10T11:29:46.479 INFO:tasks.workunit.client.0.smithi144.stdout:1/873: write d5/df/d4b/d51/d10a/f9e [749060,90043] 0 2024-09-10T11:29:46.480 INFO:tasks.workunit.client.0.smithi144.stdout:9/782: mknod d11/d12/d6f/daa/c110 0 2024-09-10T11:29:46.480 INFO:tasks.workunit.client.0.smithi144.stdout:0/654: write da/d3a/d4b/d4e/d90/f71 [953691,88592] 0 2024-09-10T11:29:46.483 INFO:tasks.workunit.client.0.smithi144.stdout:1/874: mknod d5/c116 0 2024-09-10T11:29:46.485 INFO:tasks.workunit.client.0.smithi144.stdout:9/783: creat d11/d12/d1b/d2c/f111 x:0 0 0 2024-09-10T11:29:46.489 INFO:tasks.workunit.client.0.smithi144.stdout:0/655: creat da/d44/d82/d8c/fc6 x:0 0 0 2024-09-10T11:29:46.490 INFO:tasks.workunit.client.0.smithi144.stdout:9/784: creat d11/d36/db1/dbb/df0/d8f/dc0/dd0/f112 x:0 0 0 2024-09-10T11:29:46.493 INFO:tasks.workunit.client.0.smithi144.stdout:1/875: dwrite d5/df/d4b/d85/ff0 [0,4194304] 0 2024-09-10T11:29:46.496 INFO:tasks.workunit.client.0.smithi144.stdout:1/876: readlink d5/df/d4b/d51/d10a/dbd/df1/l10d 0 2024-09-10T11:29:46.496 INFO:tasks.workunit.client.0.smithi144.stdout:0/656: getdents da/d44 0 2024-09-10T11:29:46.498 INFO:tasks.workunit.client.0.smithi144.stdout:9/785: unlink d11/d12/d1b/d2c/d78/c89 0 2024-09-10T11:29:46.500 INFO:tasks.workunit.client.0.smithi144.stdout:3/777: dwrite d4/dd/fef [0,4194304] 0 2024-09-10T11:29:46.500 INFO:tasks.workunit.client.0.smithi144.stdout:1/877: write d5/df/d4b/d85/ff0 [3959117,117262] 0 2024-09-10T11:29:46.501 INFO:tasks.workunit.client.0.smithi144.stdout:0/657: chown da/d3a/d4b/d4e/d90/fa6 1 1 2024-09-10T11:29:46.505 INFO:tasks.workunit.client.0.smithi144.stdout:1/878: unlink d5/d2a/f102 0 2024-09-10T11:29:46.505 INFO:tasks.workunit.client.0.smithi144.stdout:9/786: read d11/d36/db1/dbb/df0/d8f/f82 [1085694,44349] 0 2024-09-10T11:29:46.506 INFO:tasks.workunit.client.0.smithi144.stdout:1/879: fsync d5/d2a/d65/fcd 0 2024-09-10T11:29:46.506 INFO:tasks.workunit.client.0.smithi144.stdout:3/778: dread - d4/d28/d67/d80/d17/d18/d34/d4b/f9c zero size 2024-09-10T11:29:46.509 INFO:tasks.workunit.client.0.smithi144.stdout:1/880: read d5/f8 [251617,52940] 0 2024-09-10T11:29:46.516 INFO:tasks.workunit.client.0.smithi144.stdout:1/881: rename d5/df/d4b/d51/d10a/dbd/df1/caa to d5/d4a/c117 0 2024-09-10T11:29:46.516 INFO:tasks.workunit.client.0.smithi144.stdout:1/882: readlink d5/d2a/d65/l70 0 2024-09-10T11:29:46.520 INFO:tasks.workunit.client.0.smithi144.stdout:1/883: creat d5/dd/d64/d79/f118 x:0 0 0 2024-09-10T11:29:46.524 INFO:tasks.workunit.client.0.smithi144.stdout:1/884: dread - d5/df/d4b/d51/d10a/fe8 zero size 2024-09-10T11:29:46.526 INFO:tasks.workunit.client.0.smithi144.stdout:9/787: dwrite d11/d12/d1b/df8/fb6 [0,4194304] 0 2024-09-10T11:29:46.526 INFO:tasks.workunit.client.0.smithi144.stdout:3/779: dwrite d4/d28/d67/d80/d17/d18/f6d [0,4194304] 0 2024-09-10T11:29:46.528 INFO:tasks.workunit.client.0.smithi144.stdout:0/658: dread da/f26 [4194304,4194304] 0 2024-09-10T11:29:46.530 INFO:tasks.workunit.client.0.smithi144.stdout:3/780: write d4/d28/d67/d80/d17/d18/f2e [4606149,31274] 0 2024-09-10T11:29:46.530 INFO:tasks.workunit.client.0.smithi144.stdout:1/885: dread d5/d2a/d65/d73/fb1 [0,4194304] 0 2024-09-10T11:29:46.530 INFO:tasks.workunit.client.0.smithi144.stdout:9/788: write d11/d36/db1/dbb/df0/d8f/fb4 [2690170,32503] 0 2024-09-10T11:29:46.536 INFO:tasks.workunit.client.0.smithi144.stdout:9/789: creat d11/d36/d2d/d38/dc3/f113 x:0 0 0 2024-09-10T11:29:46.536 INFO:tasks.workunit.client.0.smithi144.stdout:1/886: rmdir d5/df/d4b/d51/d10a/dbd/df1 39 2024-09-10T11:29:46.538 INFO:tasks.workunit.client.0.smithi144.stdout:0/659: write da/d44/f65 [589890,58452] 0 2024-09-10T11:29:46.540 INFO:tasks.workunit.client.0.smithi144.stdout:3/781: dwrite d4/d28/d67/d80/d17/d18/f6d [0,4194304] 0 2024-09-10T11:29:46.541 INFO:tasks.workunit.client.0.smithi144.stdout:1/887: rename d5/dd/d9c/d10f to d5/d2a/d65/d119 0 2024-09-10T11:29:46.542 INFO:tasks.workunit.client.0.smithi144.stdout:9/790: mknod d11/d36/db1/c114 0 2024-09-10T11:29:46.546 INFO:tasks.workunit.client.0.smithi144.stdout:3/782: mkdir d4/d28/d67/d80/d17/d18/df4 0 2024-09-10T11:29:46.549 INFO:tasks.workunit.client.0.smithi144.stdout:0/660: dwrite da/d3a/d4b/fc0 [0,4194304] 0 2024-09-10T11:29:46.550 INFO:tasks.workunit.client.0.smithi144.stdout:9/791: truncate d11/d36/db1/dbb/df0/f92 1663925 0 2024-09-10T11:29:46.552 INFO:tasks.workunit.client.0.smithi144.stdout:9/792: chown d11/d12/d1b/d2c/d8a 16974191 1 2024-09-10T11:29:46.554 INFO:tasks.workunit.client.0.smithi144.stdout:3/783: creat d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/ff5 x:0 0 0 2024-09-10T11:29:46.554 INFO:tasks.workunit.client.0.smithi144.stdout:0/661: mkdir da/d3a/d8e/dab/dc7 0 2024-09-10T11:29:46.558 INFO:tasks.workunit.client.0.smithi144.stdout:3/784: creat d4/d28/d67/d80/d17/d18/d34/d65/d75/ff6 x:0 0 0 2024-09-10T11:29:46.560 INFO:tasks.workunit.client.0.smithi144.stdout:9/793: rename d11/d36/db1/dbb/df0/l6e to d11/d12/d1b/d2c/d78/dd2/dec/l115 0 2024-09-10T11:29:46.560 INFO:tasks.workunit.client.0.smithi144.stdout:0/662: symlink da/d44/d82/d8c/lc8 0 2024-09-10T11:29:46.563 INFO:tasks.workunit.client.0.smithi144.stdout:0/663: creat da/d3a/d4b/d61/d6c/fc9 x:0 0 0 2024-09-10T11:29:46.565 INFO:tasks.workunit.client.0.smithi144.stdout:0/664: rename da/d3a/d51/l57 to da/d44/d82/d8c/lca 0 2024-09-10T11:29:46.567 INFO:tasks.workunit.client.0.smithi144.stdout:0/665: creat da/d3a/d94/db1/fcb x:0 0 0 2024-09-10T11:29:46.572 INFO:tasks.workunit.client.0.smithi144.stdout:9/794: getdents d11/d12/d1b/d2c/d78/dd2/dec 0 2024-09-10T11:29:46.572 INFO:tasks.workunit.client.0.smithi144.stdout:0/666: dwrite da/f46 [0,4194304] 0 2024-09-10T11:29:46.574 INFO:tasks.workunit.client.0.smithi144.stdout:9/795: mkdir d11/d36/d2d/d38/d116 0 2024-09-10T11:29:46.576 INFO:tasks.workunit.client.0.smithi144.stdout:0/667: rename da/d3a/d51/d5f/d83 to da/d3a/d94/db1/dcc 0 2024-09-10T11:29:46.577 INFO:tasks.workunit.client.0.smithi144.stdout:9/796: mknod d11/d36/d2d/d38/dc3/c117 0 2024-09-10T11:29:46.580 INFO:tasks.workunit.client.0.smithi144.stdout:9/797: creat d11/d36/db1/dbb/df0/d8f/dc0/f118 x:0 0 0 2024-09-10T11:29:46.582 INFO:tasks.workunit.client.0.smithi144.stdout:9/798: write d11/d36/db1/dbb/df0/f5c [1721411,107044] 0 2024-09-10T11:29:46.583 INFO:tasks.workunit.client.0.smithi144.stdout:0/668: dwrite da/f1a [4194304,4194304] 0 2024-09-10T11:29:46.585 INFO:tasks.workunit.client.0.smithi144.stdout:0/669: write da/d3a/d51/d5f/f4a [4349127,78219] 0 2024-09-10T11:29:46.587 INFO:tasks.workunit.client.0.smithi144.stdout:9/799: rename d11/d36/d2d/f9a to d11/d36/d2d/df6/f119 0 2024-09-10T11:29:46.588 INFO:tasks.workunit.client.0.smithi144.stdout:0/670: unlink da/d44/d82/d8c/fc6 0 2024-09-10T11:29:46.593 INFO:tasks.workunit.client.0.smithi144.stdout:0/671: write da/d3a/d4b/d4e/d62/d78/fad [167599,2672] 0 2024-09-10T11:29:46.595 INFO:tasks.workunit.client.0.smithi144.stdout:0/672: mknod da/d44/d82/ccd 0 2024-09-10T11:29:46.602 INFO:tasks.workunit.client.0.smithi144.stdout:0/673: dwrite da/d3a/d4b/f86 [0,4194304] 0 2024-09-10T11:29:46.604 INFO:tasks.workunit.client.0.smithi144.stdout:0/674: truncate da/d3a/d4b/d4e/f5d 490302 0 2024-09-10T11:29:46.639 INFO:tasks.workunit.client.0.smithi144.stdout:0/675: write da/d3a/d4b/d61/f77 [549230,99684] 0 2024-09-10T11:29:46.640 INFO:tasks.workunit.client.0.smithi144.stdout:0/676: chown da/d3a/d51/ca0 345958873 1 2024-09-10T11:29:46.642 INFO:tasks.workunit.client.0.smithi144.stdout:2/611: sync 2024-09-10T11:29:46.642 INFO:tasks.workunit.client.0.smithi144.stdout:8/670: sync 2024-09-10T11:29:46.642 INFO:tasks.workunit.client.0.smithi144.stdout:4/797: sync 2024-09-10T11:29:46.644 INFO:tasks.workunit.client.0.smithi144.stdout:8/671: read d0/f24 [2331996,35361] 0 2024-09-10T11:29:46.644 INFO:tasks.workunit.client.0.smithi144.stdout:4/798: creat d14/d1e/d83/da4/f122 x:0 0 0 2024-09-10T11:29:46.645 INFO:tasks.workunit.client.0.smithi144.stdout:2/612: truncate d7/d29/f60 4813138 0 2024-09-10T11:29:46.656 INFO:tasks.workunit.client.0.smithi144.stdout:4/799: truncate fd 4315562 0 2024-09-10T11:29:46.658 INFO:tasks.workunit.client.0.smithi144.stdout:2/613: stat d7/de/d36/d45/d8e/d9a/f9c 0 2024-09-10T11:29:46.660 INFO:tasks.workunit.client.0.smithi144.stdout:4/800: chown d14/d1e/d83/da4/da9/dbc/ldf 43561262 1 2024-09-10T11:29:46.663 INFO:tasks.workunit.client.0.smithi144.stdout:8/672: dwrite d0/d9/d21/d7e/d85/fdc [0,4194304] 0 2024-09-10T11:29:46.667 INFO:tasks.workunit.client.0.smithi144.stdout:4/801: mkdir d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f/d123 0 2024-09-10T11:29:46.668 INFO:tasks.workunit.client.0.smithi144.stdout:8/673: rename d0/d9/d21/fdf to d0/d9/d21/d7e/fe9 0 2024-09-10T11:29:46.673 INFO:tasks.workunit.client.0.smithi144.stdout:0/677: dread da/d3a/d51/f7a [0,4194304] 0 2024-09-10T11:29:46.674 INFO:tasks.workunit.client.0.smithi144.stdout:8/674: creat d0/d9/d23/d26/fea x:0 0 0 2024-09-10T11:29:46.675 INFO:tasks.workunit.client.0.smithi144.stdout:0/678: fdatasync da/d44/f65 0 2024-09-10T11:29:46.676 INFO:tasks.workunit.client.0.smithi144.stdout:2/614: dwrite d7/d42/d64/faa [0,4194304] 0 2024-09-10T11:29:46.679 INFO:tasks.workunit.client.0.smithi144.stdout:0/679: fdatasync da/d3a/d4b/d4e/d62/d78/fad 0 2024-09-10T11:29:46.680 INFO:tasks.workunit.client.0.smithi144.stdout:4/802: dwrite d14/d67/d84/fc2 [4194304,4194304] 0 2024-09-10T11:29:46.688 INFO:tasks.workunit.client.0.smithi144.stdout:2/615: getdents d7/d42/d64/d8d 0 2024-09-10T11:29:46.690 INFO:tasks.workunit.client.0.smithi144.stdout:8/675: dwrite d0/d9/d1f/d34/d68/fd7 [4194304,4194304] 0 2024-09-10T11:29:46.691 INFO:tasks.workunit.client.0.smithi144.stdout:2/616: dread - d7/d29/d97/fc0 zero size 2024-09-10T11:29:46.694 INFO:tasks.workunit.client.0.smithi144.stdout:8/676: stat d0/d1b/d47/d94/f9d 0 2024-09-10T11:29:46.694 INFO:tasks.workunit.client.0.smithi144.stdout:0/680: dwrite da/f30 [0,4194304] 0 2024-09-10T11:29:46.701 INFO:tasks.workunit.client.0.smithi144.stdout:8/677: rename d0/d1b/lc5 to d0/d1b/d5a/leb 0 2024-09-10T11:29:46.701 INFO:tasks.workunit.client.0.smithi144.stdout:8/678: chown d0/f13 13 1 2024-09-10T11:29:46.704 INFO:tasks.workunit.client.0.smithi144.stdout:0/681: rename da/d3a/d4b/d4e/d62/d78/l9e to da/d3a/d8e/dab/dc7/lce 0 2024-09-10T11:29:46.707 INFO:tasks.workunit.client.0.smithi144.stdout:0/682: symlink da/d3a/d4b/d61/lcf 0 2024-09-10T11:29:46.708 INFO:tasks.workunit.client.0.smithi144.stdout:0/683: chown da/l18 30185 1 2024-09-10T11:29:46.710 INFO:tasks.workunit.client.0.smithi144.stdout:8/679: link d0/d9/d1f/d34/l51 d0/d9/d21/d25/db3/dc2/dcd/lec 0 2024-09-10T11:29:46.714 INFO:tasks.workunit.client.0.smithi144.stdout:8/680: mknod d0/d9/d21/d25/db3/dc2/ced 0 2024-09-10T11:29:46.715 INFO:tasks.workunit.client.0.smithi144.stdout:2/617: dwrite d7/d42/d64/d8d/fbb [0,4194304] 0 2024-09-10T11:29:46.716 INFO:tasks.workunit.client.0.smithi144.stdout:2/618: dread - d7/d29/d92/f73 zero size 2024-09-10T11:29:46.718 INFO:tasks.workunit.client.0.smithi144.stdout:8/681: creat d0/d9/d1f/fee x:0 0 0 2024-09-10T11:29:46.719 INFO:tasks.workunit.client.0.smithi144.stdout:0/684: dread da/f2e [0,4194304] 0 2024-09-10T11:29:46.720 INFO:tasks.workunit.client.0.smithi144.stdout:0/685: chown da/d3a/l87 3 1 2024-09-10T11:29:46.721 INFO:tasks.workunit.client.0.smithi144.stdout:8/682: mknod d0/d9/d1f/d34/cef 0 2024-09-10T11:29:46.721 INFO:tasks.workunit.client.0.smithi144.stdout:0/686: chown da/d3a/d94/la4 2094 1 2024-09-10T11:29:46.729 INFO:tasks.workunit.client.0.smithi144.stdout:0/687: dwrite da/d3a/d4b/d4e/d90/fa6 [0,4194304] 0 2024-09-10T11:29:46.732 INFO:tasks.workunit.client.0.smithi144.stdout:0/688: fdatasync da/d3a/d4b/fb7 0 2024-09-10T11:29:46.734 INFO:tasks.workunit.client.0.smithi144.stdout:0/689: stat da/d3a/d4b/d4e/d62/d78/fad 0 2024-09-10T11:29:46.735 INFO:tasks.workunit.client.0.smithi144.stdout:0/690: chown da/d3a/d51/cba 5343280 1 2024-09-10T11:29:46.735 INFO:tasks.workunit.client.0.smithi144.stdout:4/803: dread d14/f15 [0,4194304] 0 2024-09-10T11:29:46.737 INFO:tasks.workunit.client.0.smithi144.stdout:4/804: fsync d14/d1e/fcd 0 2024-09-10T11:29:46.737 INFO:tasks.workunit.client.0.smithi144.stdout:0/691: write da/d3a/d4b/d4e/d81/f98 [402041,65313] 0 2024-09-10T11:29:46.742 INFO:tasks.workunit.client.0.smithi144.stdout:0/692: truncate da/d3a/d4b/d4e/d62/fc1 234000 0 2024-09-10T11:29:46.750 INFO:tasks.workunit.client.0.smithi144.stdout:4/805: dwrite d14/d1e/d22/d34/f3b [0,4194304] 0 2024-09-10T11:29:46.751 INFO:tasks.workunit.client.0.smithi144.stdout:0/693: dwrite da/d3a/d4b/d4e/d90/fa6 [0,4194304] 0 2024-09-10T11:29:46.754 INFO:tasks.workunit.client.0.smithi144.stdout:2/619: dread d7/de/f2f [0,4194304] 0 2024-09-10T11:29:46.755 INFO:tasks.workunit.client.0.smithi144.stdout:2/620: fsync d7/f1d 0 2024-09-10T11:29:46.761 INFO:tasks.workunit.client.0.smithi144.stdout:2/621: rmdir d7/de/d36/d45/d8e 39 2024-09-10T11:29:46.762 INFO:tasks.workunit.client.0.smithi144.stdout:4/806: write d14/d7e/f109 [1121246,70177] 0 2024-09-10T11:29:46.765 INFO:tasks.workunit.client.0.smithi144.stdout:8/683: dread d0/d9/d23/d26/d7d/f81 [0,4194304] 0 2024-09-10T11:29:46.766 INFO:tasks.workunit.client.0.smithi144.stdout:2/622: dread - d7/d29/d92/f73 zero size 2024-09-10T11:29:46.767 INFO:tasks.workunit.client.0.smithi144.stdout:4/807: fsync d14/d67/d92/fc3 0 2024-09-10T11:29:46.769 INFO:tasks.workunit.client.0.smithi144.stdout:0/694: dwrite da/d3a/f50 [0,4194304] 0 2024-09-10T11:29:46.769 INFO:tasks.workunit.client.0.smithi144.stdout:2/623: stat d7/da/da8 0 2024-09-10T11:29:46.814 INFO:tasks.workunit.client.0.smithi144.stdout:8/684: write d0/d9/d23/f89 [1031767,38866] 0 2024-09-10T11:29:46.815 INFO:tasks.workunit.client.0.smithi144.stdout:4/808: stat d14/d1e/d22/d34 0 2024-09-10T11:29:46.815 INFO:tasks.workunit.client.0.smithi144.stdout:2/624: readlink d7/d29/d97/la6 0 2024-09-10T11:29:46.816 INFO:tasks.workunit.client.0.smithi144.stdout:0/695: rename da/d44/d82/d8c to da/d44/d82/d8c/dd0 22 2024-09-10T11:29:46.817 INFO:tasks.workunit.client.0.smithi144.stdout:8/685: write d0/d1b/d5a/d8d/f98 [792067,121536] 0 2024-09-10T11:29:46.817 INFO:tasks.workunit.client.0.smithi144.stdout:2/625: write d7/da/f35 [4942533,113311] 0 2024-09-10T11:29:46.818 INFO:tasks.workunit.client.0.smithi144.stdout:4/809: creat d14/d1e/d22/d34/d3e/f124 x:0 0 0 2024-09-10T11:29:46.819 INFO:tasks.workunit.client.0.smithi144.stdout:8/686: fsync d0/d9/d1f/f1c 0 2024-09-10T11:29:46.828 INFO:tasks.workunit.client.0.smithi144.stdout:0/696: symlink da/d3a/d51/d8f/ld1 0 2024-09-10T11:29:46.829 INFO:tasks.workunit.client.0.smithi144.stdout:8/687: chown d0/d9/d1f/d34/f53 1 1 2024-09-10T11:29:46.830 INFO:tasks.workunit.client.0.smithi144.stdout:4/810: dread d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/dd4/fd6 [0,4194304] 0 2024-09-10T11:29:46.831 INFO:tasks.workunit.client.0.smithi144.stdout:0/697: mkdir da/d3a/d4b/d4e/d62/dd2 0 2024-09-10T11:29:46.834 INFO:tasks.workunit.client.0.smithi144.stdout:8/688: dwrite d0/d9/d21/f5c [4194304,4194304] 0 2024-09-10T11:29:46.835 INFO:tasks.workunit.client.0.smithi144.stdout:4/811: link d14/d1e/lf8 d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f/d123/l125 0 2024-09-10T11:29:46.836 INFO:tasks.workunit.client.0.smithi144.stdout:8/689: creat d0/d1b/d5a/d8d/ff0 x:0 0 0 2024-09-10T11:29:46.838 INFO:tasks.workunit.client.0.smithi144.stdout:8/690: chown d0/d9/d23/d26/d7d/f80 25 1 2024-09-10T11:29:46.838 INFO:tasks.workunit.client.0.smithi144.stdout:4/812: rmdir d14/d1e/d83/da4/da9 39 2024-09-10T11:29:46.839 INFO:tasks.workunit.client.0.smithi144.stdout:2/626: truncate d7/de/d36/d45/d47/f88 117571 0 2024-09-10T11:29:46.840 INFO:tasks.workunit.client.0.smithi144.stdout:8/691: chown d0/d1b/d5a/leb 813291468 1 2024-09-10T11:29:46.845 INFO:tasks.workunit.client.0.smithi144.stdout:2/627: dread d7/da/d1e/f25 [0,4194304] 0 2024-09-10T11:29:46.853 INFO:tasks.workunit.client.0.smithi144.stdout:2/628: unlink d7/de/d36/d45/d8e/lb8 0 2024-09-10T11:29:46.855 INFO:tasks.workunit.client.0.smithi144.stdout:2/629: dread d7/de/d36/f51 [8388608,4194304] 0 2024-09-10T11:29:46.856 INFO:tasks.workunit.client.0.smithi144.stdout:2/630: read - d7/d29/d92/f73 zero size 2024-09-10T11:29:46.858 INFO:tasks.workunit.client.0.smithi144.stdout:4/813: dwrite d14/d1e/d83/da4/fc4 [0,4194304] 0 2024-09-10T11:29:46.859 INFO:tasks.workunit.client.0.smithi144.stdout:2/631: mkdir d7/de/d36/d45/d8e/d9a/dc9 0 2024-09-10T11:29:46.863 INFO:tasks.workunit.client.0.smithi144.stdout:4/814: symlink d14/d1e/d22/d34/d3e/d40/d4b/l126 0 2024-09-10T11:29:46.864 INFO:tasks.workunit.client.0.smithi144.stdout:4/815: write d14/d1e/d22/f23 [3119962,59007] 0 2024-09-10T11:29:46.866 INFO:tasks.workunit.client.0.smithi144.stdout:4/816: fsync d14/d1e/d22/d34/d3e/d40/d4b/d4c/dab/fc5 0 2024-09-10T11:29:46.874 INFO:tasks.workunit.client.0.smithi144.stdout:2/632: dread d7/d29/f46 [0,4194304] 0 2024-09-10T11:29:46.880 INFO:tasks.workunit.client.0.smithi144.stdout:2/633: dread d7/da/fb6 [0,4194304] 0 2024-09-10T11:29:46.884 INFO:tasks.workunit.client.0.smithi144.stdout:4/817: dwrite d14/d1e/f20 [4194304,4194304] 0 2024-09-10T11:29:46.889 INFO:tasks.workunit.client.0.smithi144.stdout:4/818: symlink d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/l127 0 2024-09-10T11:29:46.890 INFO:tasks.workunit.client.0.smithi144.stdout:2/634: dwrite d7/de/d36/d45/d8e/d9a/d54/fb5 [0,4194304] 0 2024-09-10T11:29:46.893 INFO:tasks.workunit.client.0.smithi144.stdout:2/635: mknod d7/de/d5f/cca 0 2024-09-10T11:29:46.895 INFO:tasks.workunit.client.0.smithi144.stdout:4/819: symlink d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/l128 0 2024-09-10T11:29:46.896 INFO:tasks.workunit.client.0.smithi144.stdout:2/636: creat d7/de/d3d/d43/d7f/fcb x:0 0 0 2024-09-10T11:29:46.896 INFO:tasks.workunit.client.0.smithi144.stdout:2/637: fdatasync d7/de/f1b 0 2024-09-10T11:29:46.898 INFO:tasks.workunit.client.0.smithi144.stdout:2/638: chown d7/de/d36/d45/d8e/l94 116 1 2024-09-10T11:29:46.899 INFO:tasks.workunit.client.0.smithi144.stdout:4/820: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/ffc [1013456,80091] 0 2024-09-10T11:29:46.904 INFO:tasks.workunit.client.0.smithi144.stdout:4/821: rename d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f/l82 to d14/d67/d92/l129 0 2024-09-10T11:29:46.906 INFO:tasks.workunit.client.0.smithi144.stdout:4/822: creat d14/d1e/d83/da4/da9/dbc/f12a x:0 0 0 2024-09-10T11:29:46.910 INFO:tasks.workunit.client.0.smithi144.stdout:4/823: getdents d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/de4 0 2024-09-10T11:29:46.912 INFO:tasks.workunit.client.0.smithi144.stdout:4/824: chown d14/d67/d92/c9e 230462 1 2024-09-10T11:29:46.912 INFO:tasks.workunit.client.0.smithi144.stdout:4/825: chown d14/d1e 305 1 2024-09-10T11:29:46.913 INFO:tasks.workunit.client.0.smithi144.stdout:4/826: write d14/d1e/d22/f23 [1992481,79933] 0 2024-09-10T11:29:46.914 INFO:tasks.workunit.client.0.smithi144.stdout:4/827: chown d14/d67/d69/d90/dda 247064999 1 2024-09-10T11:29:46.915 INFO:tasks.workunit.client.0.smithi144.stdout:4/828: write d14/d67/f121 [433801,48747] 0 2024-09-10T11:29:46.919 INFO:tasks.workunit.client.0.smithi144.stdout:4/829: link d14/d1e/d83/da4/fc4 d14/d1e/d83/da4/f12b 0 2024-09-10T11:29:46.920 INFO:tasks.workunit.client.0.smithi144.stdout:4/830: rmdir d14/d1e/d83/da4/da9 39 2024-09-10T11:29:46.922 INFO:tasks.workunit.client.0.smithi144.stdout:4/831: unlink d14/d67/d84/lbf 0 2024-09-10T11:29:46.925 INFO:tasks.workunit.client.0.smithi144.stdout:4/832: dread d14/d1e/d22/f11a [0,4194304] 0 2024-09-10T11:29:46.928 INFO:tasks.workunit.client.0.smithi144.stdout:4/833: mkdir d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/d10b/d12c 0 2024-09-10T11:29:46.930 INFO:tasks.workunit.client.0.smithi144.stdout:4/834: stat d14/d1e/d83/da4/da9/dbc/cc0 0 2024-09-10T11:29:46.931 INFO:tasks.workunit.client.0.smithi144.stdout:4/835: dread - d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/ff2 zero size 2024-09-10T11:29:46.932 INFO:tasks.workunit.client.0.smithi144.stdout:4/836: chown d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/f38 34842 1 2024-09-10T11:29:46.934 INFO:tasks.workunit.client.0.smithi144.stdout:4/837: dread - d14/d1e/d83/da4/da9/fc7 zero size 2024-09-10T11:29:46.938 INFO:tasks.workunit.client.0.smithi144.stdout:4/838: rename d14/dcb/ld3 to d14/d67/d69/d90/dda/de7/l12d 0 2024-09-10T11:29:46.942 INFO:tasks.workunit.client.0.smithi144.stdout:4/839: rename d14/d1e/d22/d34/d3e/d40/d4b/d4c/dab/df3/l112 to d14/d1e/d83/da4/da9/dbc/l12e 0 2024-09-10T11:29:46.949 INFO:tasks.workunit.client.0.smithi144.stdout:4/840: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/fb0 [0,4194304] 0 2024-09-10T11:29:46.952 INFO:tasks.workunit.client.0.smithi144.stdout:4/841: mknod d14/d6f/c12f 0 2024-09-10T11:29:46.953 INFO:tasks.workunit.client.0.smithi144.stdout:4/842: chown d14/d67/d69 49451106 1 2024-09-10T11:29:46.956 INFO:tasks.workunit.client.0.smithi144.stdout:4/843: unlink d14/d67/d92/l98 0 2024-09-10T11:29:46.957 INFO:tasks.workunit.client.0.smithi144.stdout:4/844: write d14/d1e/d83/da4/f12b [2710230,40649] 0 2024-09-10T11:29:46.960 INFO:tasks.workunit.client.0.smithi144.stdout:4/845: symlink d14/d1e/d22/d34/d3e/d40/d4b/d4c/dab/l130 0 2024-09-10T11:29:46.962 INFO:tasks.workunit.client.0.smithi144.stdout:4/846: dread - d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/ff6 zero size 2024-09-10T11:29:46.966 INFO:tasks.workunit.client.0.smithi144.stdout:4/847: fsync d14/d1e/d83/da4/fc4 0 2024-09-10T11:29:46.971 INFO:tasks.workunit.client.0.smithi144.stdout:4/848: link d14/d1e/d83/da4/da9/fc7 d14/d1e/d22/f131 0 2024-09-10T11:29:46.974 INFO:tasks.workunit.client.0.smithi144.stdout:4/849: creat d14/d1e/d22/d34/d3e/d40/d4b/f132 x:0 0 0 2024-09-10T11:29:46.979 INFO:tasks.workunit.client.0.smithi144.stdout:4/850: write d14/d1e/d22/d34/d3e/d40/d4b/f80 [1234588,77816] 0 2024-09-10T11:29:46.981 INFO:tasks.workunit.client.0.smithi144.stdout:4/851: fsync d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/de4/ff9 0 2024-09-10T11:29:46.983 INFO:tasks.workunit.client.0.smithi144.stdout:4/852: creat d14/d67/d69/d90/dda/de7/f133 x:0 0 0 2024-09-10T11:29:46.987 INFO:tasks.workunit.client.0.smithi144.stdout:4/853: link d14/d1e/d22/f131 d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/f134 0 2024-09-10T11:29:46.993 INFO:tasks.workunit.client.0.smithi144.stdout:4/854: dwrite d14/d1e/d83/da4/f118 [0,4194304] 0 2024-09-10T11:29:46.996 INFO:tasks.workunit.client.0.smithi144.stdout:4/855: mknod d14/d67/d69/d90/dda/c135 0 2024-09-10T11:29:46.999 INFO:tasks.workunit.client.0.smithi144.stdout:4/856: truncate d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/ff2 293681 0 2024-09-10T11:29:47.002 INFO:tasks.workunit.client.0.smithi144.stdout:4/857: mknod d14/d67/d84/c136 0 2024-09-10T11:29:47.004 INFO:tasks.workunit.client.0.smithi144.stdout:4/858: symlink d14/d1e/d83/da4/d11b/de0/l137 0 2024-09-10T11:29:47.017 INFO:tasks.workunit.client.0.smithi144.stdout:4/859: dread d14/d1e/d22/d34/d3e/f3f [0,4194304] 0 2024-09-10T11:29:47.020 INFO:tasks.workunit.client.0.smithi144.stdout:4/860: symlink d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/d10b/d12c/l138 0 2024-09-10T11:29:47.024 INFO:tasks.workunit.client.0.smithi144.stdout:4/861: dread d14/d1e/d22/d34/d3e/f3f [0,4194304] 0 2024-09-10T11:29:47.071 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:46 smithi178 ceph-mon[33514]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:47.071 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:46 smithi178 ceph-mon[33514]: pgmap v30: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 111 MiB/s rd, 270 MiB/s wr, 544 op/s 2024-09-10T11:29:47.071 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:46 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/3828984594' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:29:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:46 smithi144 ceph-mon[27942]: from='client.14644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:46 smithi144 ceph-mon[27942]: pgmap v30: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 111 MiB/s rd, 270 MiB/s wr, 544 op/s 2024-09-10T11:29:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:46 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/3828984594' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:29:47.275 INFO:tasks.workunit.client.0.smithi144.stdout:6/684: sync 2024-09-10T11:29:47.279 INFO:tasks.workunit.client.0.smithi144.stdout:6/685: rename d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/fb9 to d9/d38/ff5 0 2024-09-10T11:29:47.279 INFO:tasks.workunit.client.0.smithi144.stdout:6/686: stat d9/f2a 0 2024-09-10T11:29:47.281 INFO:tasks.workunit.client.0.smithi144.stdout:6/687: creat d9/dd/d11/d1b/d2f/dab/dc6/ff6 x:0 0 0 2024-09-10T11:29:47.284 INFO:tasks.workunit.client.0.smithi144.stdout:6/688: symlink d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d87/lf7 0 2024-09-10T11:29:47.302 INFO:tasks.workunit.client.0.smithi144.stdout:6/689: truncate d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/f19 979147 0 2024-09-10T11:29:47.305 INFO:tasks.workunit.client.0.smithi144.stdout:6/690: getdents d9/dd/d3e 0 2024-09-10T11:29:47.306 INFO:tasks.workunit.client.0.smithi144.stdout:6/691: readlink d9/d1f/d2c/lc0 0 2024-09-10T11:29:47.307 INFO:tasks.workunit.client.0.smithi144.stdout:6/692: chown d9/d1f/f37 31654 1 2024-09-10T11:29:47.311 INFO:tasks.workunit.client.0.smithi144.stdout:6/693: write d9/dd/d11/f18 [4343834,51549] 0 2024-09-10T11:29:47.311 INFO:tasks.workunit.client.0.smithi144.stdout:6/694: fdatasync d9/d1f/d2c/fe2 0 2024-09-10T11:29:47.315 INFO:tasks.workunit.client.0.smithi144.stdout:6/695: rmdir d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d60/def 0 2024-09-10T11:29:47.318 INFO:tasks.workunit.client.0.smithi144.stdout:6/696: getdents d9/dd/d11/d1b/d2f/d7f/d82 0 2024-09-10T11:29:47.320 INFO:tasks.workunit.client.0.smithi144.stdout:6/697: chown d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/la1 3679003 1 2024-09-10T11:29:47.321 INFO:tasks.workunit.client.0.smithi144.stdout:6/698: write d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/f51 [2980929,86096] 0 2024-09-10T11:29:47.324 INFO:tasks.workunit.client.0.smithi144.stdout:6/699: creat d9/d1f/ff8 x:0 0 0 2024-09-10T11:29:47.326 INFO:tasks.workunit.client.0.smithi144.stdout:6/700: read d9/dd/d11/d1b/d2f/d7f/fc7 [3992764,79804] 0 2024-09-10T11:29:47.327 INFO:tasks.workunit.client.0.smithi144.stdout:6/701: write d9/dd/f1a [13402297,129029] 0 2024-09-10T11:29:47.334 INFO:tasks.workunit.client.0.smithi144.stdout:6/702: dread d9/dd/d11/d1b/d2f/dab/de1/fe6 [0,4194304] 0 2024-09-10T11:29:47.334 INFO:tasks.workunit.client.0.smithi144.stdout:6/703: stat d9/dd/d3e/d47/f98 0 2024-09-10T11:29:47.339 INFO:tasks.workunit.client.0.smithi144.stdout:6/704: dwrite d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/f51 [4194304,4194304] 0 2024-09-10T11:29:47.343 INFO:tasks.workunit.client.0.smithi144.stdout:6/705: mkdir d9/dd/d11/d1b/d2f/d7f/dc4/d5b/df9 0 2024-09-10T11:29:47.346 INFO:tasks.workunit.client.0.smithi144.stdout:6/706: chown d9/dd/d11/d1b/d2f/d7f/dc4/d5b/fca 0 1 2024-09-10T11:29:47.349 INFO:tasks.workunit.client.0.smithi144.stdout:6/707: creat d9/dd/d11/d1b/d2f/d7f/ffa x:0 0 0 2024-09-10T11:29:47.352 INFO:tasks.workunit.client.0.smithi144.stdout:6/708: unlink d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/f99 0 2024-09-10T11:29:47.383 INFO:tasks.workunit.client.0.smithi144.stdout:7/785: sync 2024-09-10T11:29:47.384 INFO:tasks.workunit.client.0.smithi144.stdout:7/786: chown d5/dc/d2a/d41 6574 1 2024-09-10T11:29:47.387 INFO:tasks.workunit.client.0.smithi144.stdout:7/787: mkdir d5/dc/d2a/d60/dd6/d98/d102 0 2024-09-10T11:29:47.390 INFO:tasks.workunit.client.0.smithi144.stdout:7/788: link d5/dc/d2a/le9 d5/dc/d12/l103 0 2024-09-10T11:29:47.392 INFO:tasks.workunit.client.0.smithi144.stdout:7/789: mknod d5/dc/d14/d40/d58/d6a/c104 0 2024-09-10T11:29:47.394 INFO:tasks.workunit.client.0.smithi144.stdout:7/790: truncate d5/dc/d12/f13 2942927 0 2024-09-10T11:29:47.394 INFO:tasks.workunit.client.0.smithi144.stdout:7/791: fdatasync d5/f7 0 2024-09-10T11:29:47.401 INFO:tasks.workunit.client.0.smithi144.stdout:7/792: dread d5/dc/d11/d18/f1e [0,4194304] 0 2024-09-10T11:29:47.408 INFO:tasks.workunit.client.0.smithi144.stdout:7/793: dwrite d5/dc/d14/d40/d5e/dc4/ffe [0,4194304] 0 2024-09-10T11:29:47.416 INFO:tasks.workunit.client.0.smithi144.stdout:7/794: fsync d5/fa 0 2024-09-10T11:29:47.418 INFO:tasks.workunit.client.0.smithi144.stdout:7/795: readlink d5/dc/d14/d40/d58/d6e/la4 0 2024-09-10T11:29:47.422 INFO:tasks.workunit.client.0.smithi144.stdout:7/796: getdents d5/dc/d2a/d60/dd6/d98/dd1/d7c 0 2024-09-10T11:29:47.424 INFO:tasks.workunit.client.0.smithi144.stdout:7/797: truncate d5/dc/d2a/f4b 376924 0 2024-09-10T11:29:47.475 INFO:tasks.workunit.client.0.smithi144.stdout:5/761: sync 2024-09-10T11:29:47.478 INFO:tasks.workunit.client.0.smithi144.stdout:5/762: creat d4/d20/d24/d2d/d4b/fee x:0 0 0 2024-09-10T11:29:47.479 INFO:tasks.workunit.client.0.smithi144.stdout:5/763: dread - d4/d52/dd2/d84/fab zero size 2024-09-10T11:29:47.481 INFO:tasks.workunit.client.0.smithi144.stdout:5/764: mknod d4/d52/dd2/d84/cef 0 2024-09-10T11:29:47.482 INFO:tasks.workunit.client.0.smithi144.stdout:5/765: chown d4/ld 1 1 2024-09-10T11:29:47.484 INFO:tasks.workunit.client.0.smithi144.stdout:5/766: mknod d4/d52/cf0 0 2024-09-10T11:29:47.486 INFO:tasks.workunit.client.0.smithi144.stdout:5/767: truncate d4/d5/d18/d67/d59/f5e 436657 0 2024-09-10T11:29:47.487 INFO:tasks.workunit.client.0.smithi144.stdout:5/768: stat d4/d52/c7c 0 2024-09-10T11:29:47.488 INFO:tasks.workunit.client.0.smithi144.stdout:5/769: write d4/d20/d24/d2d/d4b/fd1 [281201,95135] 0 2024-09-10T11:29:47.490 INFO:tasks.workunit.client.0.smithi144.stdout:5/770: unlink d4/d5/cf 0 2024-09-10T11:29:47.491 INFO:tasks.workunit.client.0.smithi144.stdout:5/771: chown d4/d20/d24/d2d 27 1 2024-09-10T11:29:47.498 INFO:tasks.workunit.client.0.smithi144.stdout:5/772: write d4/d20/d24/d2d/d4b/fa7 [1584405,84395] 0 2024-09-10T11:29:47.500 INFO:tasks.workunit.client.0.smithi144.stdout:5/773: unlink d4/d5/f7d 0 2024-09-10T11:29:47.505 INFO:tasks.workunit.client.0.smithi144.stdout:5/774: write d4/d20/d24/d2d/d4b/f4e [1482155,28877] 0 2024-09-10T11:29:47.505 INFO:tasks.workunit.client.0.smithi144.stdout:5/775: chown d4/d5/d32/l39 119099 1 2024-09-10T11:29:47.508 INFO:tasks.workunit.client.0.smithi144.stdout:5/776: creat d4/d5/d18/d38/d6d/ff1 x:0 0 0 2024-09-10T11:29:47.508 INFO:tasks.workunit.client.0.smithi144.stdout:5/777: chown d4/d20/d22/l45 191 1 2024-09-10T11:29:47.512 INFO:tasks.workunit.client.0.smithi144.stdout:5/778: dread d4/d5/d32/d47/f71 [0,4194304] 0 2024-09-10T11:29:47.513 INFO:tasks.workunit.client.0.smithi144.stdout:5/779: chown d4/d5/l11 6 1 2024-09-10T11:29:47.517 INFO:tasks.workunit.client.0.smithi144.stdout:5/780: write d4/d5/f23 [51157,57504] 0 2024-09-10T11:29:47.520 INFO:tasks.workunit.client.0.smithi144.stdout:5/781: write d4/d5/d18/d51/fd5 [338947,31323] 0 2024-09-10T11:29:47.526 INFO:tasks.workunit.client.0.smithi144.stdout:5/782: dwrite d4/d5/d32/f5f [0,4194304] 0 2024-09-10T11:29:47.528 INFO:tasks.workunit.client.0.smithi144.stdout:5/783: mkdir d4/d5/df2 0 2024-09-10T11:29:47.530 INFO:tasks.workunit.client.0.smithi144.stdout:5/784: write d4/d5/d18/d38/d6d/d8b/f9b [4132279,43947] 0 2024-09-10T11:29:47.534 INFO:tasks.workunit.client.0.smithi144.stdout:5/785: rename d4/d52/dd2 to d4/d5/d32/d47/df3 0 2024-09-10T11:29:47.538 INFO:tasks.workunit.client.0.smithi144.stdout:5/786: link d4/d20/d22/d5c/d5d/fe5 d4/d20/d22/d5c/d5d/de6/ff4 0 2024-09-10T11:29:47.541 INFO:tasks.workunit.client.0.smithi144.stdout:5/787: getdents d4/d5/d18/d67/d59 0 2024-09-10T11:29:47.541 INFO:tasks.workunit.client.0.smithi144.stdout:5/788: readlink d4/d5/d18/d38/d6d/da6/lb3 0 2024-09-10T11:29:47.857 INFO:tasks.workunit.client.0.smithi144.stdout:1/888: sync 2024-09-10T11:29:47.872 INFO:tasks.workunit.client.0.smithi144.stdout:1/889: dwrite d5/dd/d64/f9a [0,4194304] 0 2024-09-10T11:29:47.884 INFO:tasks.workunit.client.0.smithi144.stdout:1/890: dread d5/dd/d64/d79/fb3 [0,4194304] 0 2024-09-10T11:29:47.886 INFO:tasks.workunit.client.0.smithi144.stdout:1/891: mkdir d5/df/d4b/d85/d11a 0 2024-09-10T11:29:47.892 INFO:tasks.workunit.client.0.smithi144.stdout:1/892: rename d5/d2a/d65/faf to d5/d4a/f11b 0 2024-09-10T11:29:47.894 INFO:tasks.workunit.client.0.smithi144.stdout:1/893: mkdir d5/dd/da1/d11c 0 2024-09-10T11:29:47.895 INFO:tasks.workunit.client.0.smithi144.stdout:1/894: rmdir d5/d4a/d63 39 2024-09-10T11:29:47.906 INFO:tasks.workunit.client.0.smithi144.stdout:1/895: dwrite d5/df/d4b/fb4 [0,4194304] 0 2024-09-10T11:29:47.912 INFO:tasks.workunit.client.0.smithi144.stdout:1/896: truncate d5/d2a/f96 3967347 0 2024-09-10T11:29:47.915 INFO:tasks.workunit.client.0.smithi144.stdout:1/897: dread d5/f15 [0,4194304] 0 2024-09-10T11:29:47.916 INFO:tasks.workunit.client.0.smithi144.stdout:1/898: dread - d5/df/d4b/d51/d10a/dab/f115 zero size 2024-09-10T11:29:47.918 INFO:tasks.workunit.client.0.smithi144.stdout:1/899: fdatasync d5/dd/d64/d79/fb3 0 2024-09-10T11:29:47.929 INFO:tasks.workunit.client.0.smithi144.stdout:1/900: dwrite d5/df/d4b/d51/d10a/d95/fc7 [0,4194304] 0 2024-09-10T11:29:47.941 INFO:tasks.workunit.client.0.smithi144.stdout:1/901: dwrite d5/df/d4b/fc8 [0,4194304] 0 2024-09-10T11:29:47.944 INFO:tasks.workunit.client.0.smithi144.stdout:1/902: fdatasync d5/dd/f99 0 2024-09-10T11:29:47.945 INFO:tasks.workunit.client.0.smithi144.stdout:1/903: dread - d5/dd/da1/fe1 zero size 2024-09-10T11:29:47.947 INFO:tasks.workunit.client.0.smithi144.stdout:1/904: write d5/df/d4b/d51/f50 [392728,81088] 0 2024-09-10T11:29:47.948 INFO:tasks.workunit.client.0.smithi144.stdout:1/905: dread - d5/d4a/d44/f100 zero size 2024-09-10T11:29:47.951 INFO:tasks.workunit.client.0.smithi144.stdout:1/906: unlink d5/d2a/f7e 0 2024-09-10T11:29:47.954 INFO:tasks.workunit.client.0.smithi144.stdout:1/907: mkdir d5/d2a/d65/d73/d11d 0 2024-09-10T11:29:47.956 INFO:tasks.workunit.client.0.smithi144.stdout:1/908: symlink d5/d2a/l11e 0 2024-09-10T11:29:47.958 INFO:tasks.workunit.client.0.smithi144.stdout:1/909: creat d5/df/d4b/d51/d10a/d95/dd2/f11f x:0 0 0 2024-09-10T11:29:47.958 INFO:tasks.workunit.client.0.smithi144.stdout:1/910: chown d5/d4a/f108 66267689 1 2024-09-10T11:29:47.960 INFO:tasks.workunit.client.0.smithi144.stdout:1/911: unlink d5/df/d4b/d51/l60 0 2024-09-10T11:29:47.962 INFO:tasks.workunit.client.0.smithi144.stdout:1/912: symlink d5/df/d4b/d51/d10a/dab/l120 0 2024-09-10T11:29:47.965 INFO:tasks.workunit.client.0.smithi144.stdout:1/913: rename d5/d2a/d65/d73/dd8/ced to d5/dd/d9c/c121 0 2024-09-10T11:29:47.967 INFO:tasks.workunit.client.0.smithi144.stdout:1/914: rename d5/d2a/fa4 to d5/dd/da0/f122 0 2024-09-10T11:29:47.968 INFO:tasks.workunit.client.0.smithi144.stdout:1/915: readlink d5/d2a/d65/dc9/lc3 0 2024-09-10T11:29:47.968 INFO:tasks.workunit.client.0.smithi144.stdout:1/916: rename d5/df/d4b/d51/d10a to d5/df/d4b/d51/d10a/d95/d123 22 2024-09-10T11:29:47.977 INFO:tasks.workunit.client.0.smithi144.stdout:1/917: write d5/dd/da0/f122 [292802,71153] 0 2024-09-10T11:29:47.979 INFO:tasks.workunit.client.0.smithi144.stdout:1/918: truncate d5/dd/f8d 186305 0 2024-09-10T11:29:47.983 INFO:tasks.workunit.client.0.smithi144.stdout:1/919: rmdir d5/df/d4b/d51 39 2024-09-10T11:29:47.986 INFO:tasks.workunit.client.0.smithi144.stdout:1/920: write d5/df/d4b/f54 [1826941,116245] 0 2024-09-10T11:29:47.991 INFO:tasks.workunit.client.0.smithi144.stdout:1/921: dwrite d5/fdd [0,4194304] 0 2024-09-10T11:29:47.995 INFO:tasks.workunit.client.0.smithi144.stdout:1/922: rename d5/dd/da0 to d5/df/d4b/d51/d10a/d95/d124 0 2024-09-10T11:29:47.998 INFO:tasks.workunit.client.0.smithi144.stdout:1/923: unlink d5/df/d4b/d51/d10a/dbd/f113 0 2024-09-10T11:29:48.000 INFO:tasks.workunit.client.0.smithi144.stdout:1/924: mknod d5/df/d4b/d51/d10a/df9/c125 0 2024-09-10T11:29:48.008 INFO:tasks.workunit.client.0.smithi144.stdout:1/925: truncate d5/df/d4b/d51/d10a/dbd/f109 2789408 0 2024-09-10T11:29:48.010 INFO:tasks.workunit.client.0.smithi144.stdout:1/926: mknod d5/d2a/d65/d73/c126 0 2024-09-10T11:29:48.013 INFO:tasks.workunit.client.0.smithi144.stdout:1/927: truncate d5/d2a/d65/d73/f24 1565005 0 2024-09-10T11:29:48.014 INFO:tasks.workunit.client.0.smithi144.stdout:0/698: sync 2024-09-10T11:29:48.014 INFO:tasks.workunit.client.0.smithi144.stdout:2/639: sync 2024-09-10T11:29:48.015 INFO:tasks.workunit.client.0.smithi144.stdout:3/785: sync 2024-09-10T11:29:48.015 INFO:tasks.workunit.client.0.smithi144.stdout:6/709: sync 2024-09-10T11:29:48.015 INFO:tasks.workunit.client.0.smithi144.stdout:9/800: sync 2024-09-10T11:29:48.015 INFO:tasks.workunit.client.0.smithi144.stdout:8/692: sync 2024-09-10T11:29:48.015 INFO:tasks.workunit.client.0.smithi144.stdout:4/862: sync 2024-09-10T11:29:48.015 INFO:tasks.workunit.client.0.smithi144.stdout:7/798: sync 2024-09-10T11:29:48.015 INFO:tasks.workunit.client.0.smithi144.stdout:5/789: sync 2024-09-10T11:29:48.015 INFO:tasks.workunit.client.0.smithi144.stdout:1/928: truncate d5/d2a/d65/d73/fd5 665692 0 2024-09-10T11:29:48.017 INFO:tasks.workunit.client.0.smithi144.stdout:8/693: chown d0/d9/d21/l27 6 1 2024-09-10T11:29:48.017 INFO:tasks.workunit.client.0.smithi144.stdout:6/710: fdatasync d9/dd/d73/fb1 0 2024-09-10T11:29:48.018 INFO:tasks.workunit.client.0.smithi144.stdout:2/640: mkdir d7/dcc 0 2024-09-10T11:29:48.018 INFO:tasks.workunit.client.0.smithi144.stdout:5/790: dread - d4/d20/d22/d5c/d5d/fe5 zero size 2024-09-10T11:29:48.019 INFO:tasks.workunit.client.0.smithi144.stdout:0/699: creat da/d3a/d51/fd3 x:0 0 0 2024-09-10T11:29:48.020 INFO:tasks.workunit.client.0.smithi144.stdout:5/791: readlink d4/d20/l2b 0 2024-09-10T11:29:48.021 INFO:tasks.workunit.client.0.smithi144.stdout:8/694: dread - d0/d9/d1f/fc6 zero size 2024-09-10T11:29:48.023 INFO:tasks.workunit.client.0.smithi144.stdout:4/863: mkdir d14/d67/d69/d139 0 2024-09-10T11:29:48.025 INFO:tasks.workunit.client.0.smithi144.stdout:0/700: stat da/d3a/d8e/c97 0 2024-09-10T11:29:48.026 INFO:tasks.workunit.client.0.smithi144.stdout:1/929: chown d5/df/d4b/d51/d10a/dbd/df1/fea 114350 1 2024-09-10T11:29:48.029 INFO:tasks.workunit.client.0.smithi144.stdout:7/799: dwrite d5/fce [0,4194304] 0 2024-09-10T11:29:48.031 INFO:tasks.workunit.client.0.smithi144.stdout:1/930: fdatasync d5/d4a/f108 0 2024-09-10T11:29:48.033 INFO:tasks.workunit.client.0.smithi144.stdout:9/801: dwrite d11/d12/d1b/fc9 [0,4194304] 0 2024-09-10T11:29:48.033 INFO:tasks.workunit.client.0.smithi144.stdout:3/786: creat d4/d28/d67/d80/d17/ff7 x:0 0 0 2024-09-10T11:29:48.034 INFO:tasks.workunit.client.0.smithi144.stdout:2/641: rename d7/c32 to d7/de/d3d/d4e/dc4/ccd 0 2024-09-10T11:29:48.034 INFO:tasks.workunit.client.0.smithi144.stdout:1/931: write d5/fdd [279337,24944] 0 2024-09-10T11:29:48.037 INFO:tasks.workunit.client.0.smithi144.stdout:9/802: chown d11/d36/d2d/d38 6063805 1 2024-09-10T11:29:48.039 INFO:tasks.workunit.client.0.smithi144.stdout:8/695: symlink d0/d9/d21/d25/d4f/d93/lf1 0 2024-09-10T11:29:48.043 INFO:tasks.workunit.client.0.smithi144.stdout:3/787: mkdir d4/d1a/df8 0 2024-09-10T11:29:48.045 INFO:tasks.workunit.client.0.smithi144.stdout:4/864: dwrite d14/d1e/d22/d34/d3e/d40/f10f [0,4194304] 0 2024-09-10T11:29:48.048 INFO:tasks.workunit.client.0.smithi144.stdout:5/792: dwrite d4/d5/d18/d38/d6d/fcb [0,4194304] 0 2024-09-10T11:29:48.054 INFO:tasks.workunit.client.0.smithi144.stdout:7/800: creat d5/f105 x:0 0 0 2024-09-10T11:29:48.054 INFO:tasks.workunit.client.0.smithi144.stdout:8/696: mkdir d0/d9/d21/d25/db3/dc2/df2 0 2024-09-10T11:29:48.056 INFO:tasks.workunit.client.0.smithi144.stdout:6/711: link d9/dd/d11/d1b/ced d9/dd/d11/d1b/d2f/dab/de1/cfb 0 2024-09-10T11:29:48.056 INFO:tasks.workunit.client.0.smithi144.stdout:2/642: symlink d7/de/d36/d45/d8e/d9a/dc9/lce 0 2024-09-10T11:29:48.056 INFO:tasks.workunit.client.0.smithi144.stdout:3/788: truncate d4/d28/d67/d9f/fc5 180806 0 2024-09-10T11:29:48.058 INFO:tasks.workunit.client.0.smithi144.stdout:7/801: chown d5/dc/f56 266658 1 2024-09-10T11:29:48.058 INFO:tasks.workunit.client.0.smithi144.stdout:9/803: dwrite d11/f15 [4194304,4194304] 0 2024-09-10T11:29:48.059 INFO:tasks.workunit.client.0.smithi144.stdout:4/865: unlink d14/d1e/d22/d34/d3e/d40/f10f 0 2024-09-10T11:29:48.061 INFO:tasks.workunit.client.0.smithi144.stdout:3/789: write d4/d28/d67/d80/d17/ff7 [907271,108132] 0 2024-09-10T11:29:48.072 INFO:tasks.workunit.client.0.smithi144.stdout:1/932: getdents d5/d4a 0 2024-09-10T11:29:48.077 INFO:tasks.workunit.client.0.smithi144.stdout:5/793: rmdir d4/d5/d18/d67/d44 0 2024-09-10T11:29:48.079 INFO:tasks.workunit.client.0.smithi144.stdout:6/712: symlink d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d87/dc9/deb/lfc 0 2024-09-10T11:29:48.079 INFO:tasks.workunit.client.0.smithi144.stdout:9/804: dread d11/d36/fa4 [0,4194304] 0 2024-09-10T11:29:48.080 INFO:tasks.workunit.client.0.smithi144.stdout:6/713: stat d9/dd/d3e/d47/f98 0 2024-09-10T11:29:48.081 INFO:tasks.workunit.client.0.smithi144.stdout:1/933: creat d5/df/d4b/dcf/f127 x:0 0 0 2024-09-10T11:29:48.082 INFO:tasks.workunit.client.0.smithi144.stdout:8/697: link d0/d9/d21/d25/d4f/cc7 d0/d9/d1f/d63/cf3 0 2024-09-10T11:29:48.082 INFO:tasks.workunit.client.0.smithi144.stdout:7/802: write d5/dc/d14/d40/d5e/f6f [760661,13740] 0 2024-09-10T11:29:48.084 INFO:tasks.workunit.client.0.smithi144.stdout:2/643: link d7/de/d3d/d4e/f6d d7/da/d5b/fcf 0 2024-09-10T11:29:48.085 INFO:tasks.workunit.client.0.smithi144.stdout:5/794: dread d4/d5/d32/f5f [0,4194304] 0 2024-09-10T11:29:48.085 INFO:tasks.workunit.client.0.smithi144.stdout:6/714: fsync d9/dd/d3e/f76 0 2024-09-10T11:29:48.086 INFO:tasks.workunit.client.0.smithi144.stdout:8/698: mknod d0/d9/d23/d26/cf4 0 2024-09-10T11:29:48.089 INFO:tasks.workunit.client.0.smithi144.stdout:2/644: symlink d7/de/d5f/ld0 0 2024-09-10T11:29:48.090 INFO:tasks.workunit.client.0.smithi144.stdout:8/699: write d0/d9/d21/d25/d4f/f56 [4508250,123411] 0 2024-09-10T11:29:48.090 INFO:tasks.workunit.client.0.smithi144.stdout:2/645: chown d7/da/f17 253363 1 2024-09-10T11:29:48.091 INFO:tasks.workunit.client.0.smithi144.stdout:9/805: unlink d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/lb5 0 2024-09-10T11:29:48.093 INFO:tasks.workunit.client.0.smithi144.stdout:5/795: rename d4/d5/d18/d67/f65 to d4/d5/d32/daa/ff5 0 2024-09-10T11:29:48.093 INFO:tasks.workunit.client.0.smithi144.stdout:9/806: dread - d11/d36/db1/fc6 zero size 2024-09-10T11:29:48.093 INFO:tasks.workunit.client.0.smithi144.stdout:8/700: read d0/d9/d23/f40 [7995212,87764] 0 2024-09-10T11:29:48.094 INFO:tasks.workunit.client.0.smithi144.stdout:1/934: dwrite d5/d4a/d44/da2/fd7 [0,4194304] 0 2024-09-10T11:29:48.094 INFO:tasks.workunit.client.0.smithi144.stdout:6/715: creat d9/dd/d11/d1b/d2f/d7f/d82/ffd x:0 0 0 2024-09-10T11:29:48.099 INFO:tasks.workunit.client.0.smithi144.stdout:1/935: unlink d5/df/d4b/d51/d10a/d95/la5 0 2024-09-10T11:29:48.103 INFO:tasks.workunit.client.0.smithi144.stdout:5/796: dwrite d4/d20/f34 [0,4194304] 0 2024-09-10T11:29:48.105 INFO:tasks.workunit.client.0.smithi144.stdout:7/803: link d5/dc/d2a/d60/dd6/d98/dd1/d7c/ce6 d5/dc/d14/d40/d5e/d68/df0/c106 0 2024-09-10T11:29:48.105 INFO:tasks.workunit.client.0.smithi144.stdout:9/807: write d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/f63 [3072972,91996] 0 2024-09-10T11:29:48.105 INFO:tasks.workunit.client.0.smithi144.stdout:8/701: creat d0/d9/d21/d25/db3/de8/ff5 x:0 0 0 2024-09-10T11:29:48.106 INFO:tasks.workunit.client.0.smithi144.stdout:6/716: creat d9/dd/d11/d1b/d2f/dab/de1/ffe x:0 0 0 2024-09-10T11:29:48.108 INFO:tasks.workunit.client.0.smithi144.stdout:8/702: write d0/d9/f1d [825682,6659] 0 2024-09-10T11:29:48.109 INFO:tasks.workunit.client.0.smithi144.stdout:5/797: mknod d4/d20/d22/d75/dde/cf6 0 2024-09-10T11:29:48.113 INFO:tasks.workunit.client.0.smithi144.stdout:7/804: unlink d5/dc/d14/d40/d58/d6e/la4 0 2024-09-10T11:29:48.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:47 smithi178 ceph-mon[33514]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:48.117 INFO:tasks.workunit.client.0.smithi144.stdout:9/808: link d11/d36/fa4 d11/d36/d2d/d38/f11a 0 2024-09-10T11:29:48.118 INFO:tasks.workunit.client.0.smithi144.stdout:7/805: mkdir d5/dc/d14/d107 0 2024-09-10T11:29:48.120 INFO:tasks.workunit.client.0.smithi144.stdout:5/798: dwrite d4/d5/d18/d38/d6d/d8b/f91 [0,4194304] 0 2024-09-10T11:29:48.124 INFO:tasks.workunit.client.0.smithi144.stdout:5/799: chown d4/d20/d24/d2d/d4b/fee 3142 1 2024-09-10T11:29:48.124 INFO:tasks.workunit.client.0.smithi144.stdout:9/809: symlink d11/d36/db1/dbb/df0/l11b 0 2024-09-10T11:29:48.128 INFO:tasks.workunit.client.0.smithi144.stdout:9/810: mkdir d11/d11c 0 2024-09-10T11:29:48.129 INFO:tasks.workunit.client.0.smithi144.stdout:8/703: dread d0/d9/d23/d26/f36 [4194304,4194304] 0 2024-09-10T11:29:48.130 INFO:tasks.workunit.client.0.smithi144.stdout:2/646: dwrite d7/de/d36/d45/d47/d6c/d7c/d82/f98 [0,4194304] 0 2024-09-10T11:29:48.134 INFO:tasks.workunit.client.0.smithi144.stdout:9/811: chown d11/d12/d1b/f4d 5909314 1 2024-09-10T11:29:48.134 INFO:tasks.workunit.client.0.smithi144.stdout:2/647: write d7/da/d1e/f66 [1521482,10498] 0 2024-09-10T11:29:48.135 INFO:tasks.workunit.client.0.smithi144.stdout:2/648: chown d7/f11 9 1 2024-09-10T11:29:48.139 INFO:tasks.workunit.client.0.smithi144.stdout:8/704: link d0/d9/d23/lc9 d0/d9/d23/d37/lf6 0 2024-09-10T11:29:48.141 INFO:tasks.workunit.client.0.smithi144.stdout:2/649: symlink d7/da/d9e/ld1 0 2024-09-10T11:29:48.142 INFO:tasks.workunit.client.0.smithi144.stdout:9/812: mknod d11/d36/d2d/d38/de7/c11d 0 2024-09-10T11:29:48.143 INFO:tasks.workunit.client.0.smithi144.stdout:9/813: chown d11/d36/d2d/c91 11013 1 2024-09-10T11:29:48.144 INFO:tasks.workunit.client.0.smithi144.stdout:5/800: truncate d4/d20/d24/f92 5581027 0 2024-09-10T11:29:48.146 INFO:tasks.workunit.client.0.smithi144.stdout:9/814: rmdir d11 39 2024-09-10T11:29:48.146 INFO:tasks.workunit.client.0.smithi144.stdout:7/806: dread d5/dc/d14/d40/f57 [0,4194304] 0 2024-09-10T11:29:48.147 INFO:tasks.workunit.client.0.smithi144.stdout:5/801: write d4/d5/d18/d38/d6d/ff1 [170209,19521] 0 2024-09-10T11:29:48.147 INFO:tasks.workunit.client.0.smithi144.stdout:2/650: rename d7/de/d3d/d43 to d7/d29/d92/dd2 0 2024-09-10T11:29:48.149 INFO:tasks.workunit.client.0.smithi144.stdout:2/651: chown d7/de/f28 645510 1 2024-09-10T11:29:48.149 INFO:tasks.workunit.client.0.smithi144.stdout:5/802: mknod d4/d5/d18/cf7 0 2024-09-10T11:29:48.150 INFO:tasks.workunit.client.0.smithi144.stdout:2/652: dread - d7/d29/d97/fc0 zero size 2024-09-10T11:29:48.151 INFO:tasks.workunit.client.0.smithi144.stdout:7/807: rename l0 to d5/dc/d14/d40/d5e/dc4/l108 0 2024-09-10T11:29:48.152 INFO:tasks.workunit.client.0.smithi144.stdout:8/705: dread d0/d9/d23/d26/f4a [0,4194304] 0 2024-09-10T11:29:48.152 INFO:tasks.workunit.client.0.smithi144.stdout:7/808: chown d5/dc/d12/f22 1680801 1 2024-09-10T11:29:48.154 INFO:tasks.workunit.client.0.smithi144.stdout:5/803: creat d4/d5/d18/d38/d6d/da6/ff8 x:0 0 0 2024-09-10T11:29:48.156 INFO:tasks.workunit.client.0.smithi144.stdout:9/815: link d11/d12/d6f/fd3 d11/d12/d1b/df8/f11e 0 2024-09-10T11:29:48.157 INFO:tasks.workunit.client.0.smithi144.stdout:7/809: symlink d5/dc/d12/d2b/l109 0 2024-09-10T11:29:48.158 INFO:tasks.workunit.client.0.smithi144.stdout:8/706: symlink d0/d9/d21/d25/db3/dc2/dcd/lf7 0 2024-09-10T11:29:48.160 INFO:tasks.workunit.client.0.smithi144.stdout:2/653: mkdir d7/dcc/dd3 0 2024-09-10T11:29:48.161 INFO:tasks.workunit.client.0.smithi144.stdout:5/804: creat d4/d52/ff9 x:0 0 0 2024-09-10T11:29:48.162 INFO:tasks.workunit.client.0.smithi144.stdout:9/816: symlink d11/d36/d2d/d38/dc3/l11f 0 2024-09-10T11:29:48.165 INFO:tasks.workunit.client.0.smithi144.stdout:7/810: mkdir d5/dc/d12/d10a 0 2024-09-10T11:29:48.168 INFO:tasks.workunit.client.0.smithi144.stdout:9/817: rename d11/d12/d6f/daa/ff1 to d11/d36/d2d/d38/dc3/f120 0 2024-09-10T11:29:48.169 INFO:tasks.workunit.client.0.smithi144.stdout:8/707: dwrite d0/d1b/f2f [8388608,4194304] 0 2024-09-10T11:29:48.174 INFO:tasks.workunit.client.0.smithi144.stdout:7/811: write d5/dc/d14/d40/d58/d6a/f92 [3731185,42645] 0 2024-09-10T11:29:48.177 INFO:tasks.workunit.client.0.smithi144.stdout:9/818: write d11/d12/d6f/daa/ffe [384004,62758] 0 2024-09-10T11:29:48.177 INFO:tasks.workunit.client.0.smithi144.stdout:5/805: dwrite d4/d5/f23 [0,4194304] 0 2024-09-10T11:29:48.178 INFO:tasks.workunit.client.0.smithi144.stdout:2/654: dwrite d7/de/d36/f99 [0,4194304] 0 2024-09-10T11:29:48.183 INFO:tasks.workunit.client.0.smithi144.stdout:7/812: creat d5/dc/d12/d2b/dde/f10b x:0 0 0 2024-09-10T11:29:48.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:47 smithi144 ceph-mon[27942]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:29:48.184 INFO:tasks.workunit.client.0.smithi144.stdout:9/819: fsync d11/d36/db1/dbb/df0/d8f/dc0/dd0/f103 0 2024-09-10T11:29:48.187 INFO:tasks.workunit.client.0.smithi144.stdout:8/708: rename d0/d9/d21/f9e to d0/d9/d21/d25/ff8 0 2024-09-10T11:29:48.188 INFO:tasks.workunit.client.0.smithi144.stdout:2/655: mknod d7/de/d3d/dc8/cd4 0 2024-09-10T11:29:48.192 INFO:tasks.workunit.client.0.smithi144.stdout:5/806: dwrite d4/d5/d18/d67/d59/f6c [0,4194304] 0 2024-09-10T11:29:48.194 INFO:tasks.workunit.client.0.smithi144.stdout:8/709: mkdir d0/d1b/d47/d5e/df9 0 2024-09-10T11:29:48.195 INFO:tasks.workunit.client.0.smithi144.stdout:2/656: truncate d7/d29/f46 4234170 0 2024-09-10T11:29:48.197 INFO:tasks.workunit.client.0.smithi144.stdout:7/813: dwrite d5/dc/fe4 [0,4194304] 0 2024-09-10T11:29:48.198 INFO:tasks.workunit.client.0.smithi144.stdout:5/807: symlink d4/d5/d32/d47/df3/d9a/lfa 0 2024-09-10T11:29:48.200 INFO:tasks.workunit.client.0.smithi144.stdout:5/808: write d4/d5/d18/d38/d6d/d8b/f9b [4592174,128162] 0 2024-09-10T11:29:48.201 INFO:tasks.workunit.client.0.smithi144.stdout:2/657: mkdir d7/d42/d64/dd5 0 2024-09-10T11:29:48.201 INFO:tasks.workunit.client.0.smithi144.stdout:2/658: chown d7/de/l4c 891 1 2024-09-10T11:29:48.201 INFO:tasks.workunit.client.0.smithi144.stdout:2/659: chown d7/de/d5f 107921077 1 2024-09-10T11:29:48.202 INFO:tasks.workunit.client.0.smithi144.stdout:2/660: chown d7/dcc/dd3 50 1 2024-09-10T11:29:48.206 INFO:tasks.workunit.client.0.smithi144.stdout:2/661: creat d7/d42/d64/fd6 x:0 0 0 2024-09-10T11:29:48.207 INFO:tasks.workunit.client.0.smithi144.stdout:5/809: mknod d4/d5/d32/cfb 0 2024-09-10T11:29:48.207 INFO:tasks.workunit.client.0.smithi144.stdout:7/814: link d5/dc/d14/d40/d58/l71 d5/dc/d14/d40/d58/d6a/dad/l10c 0 2024-09-10T11:29:48.208 INFO:tasks.workunit.client.0.smithi144.stdout:5/810: chown d4/d5/d18/d38/d6d/da2 14109 1 2024-09-10T11:29:48.208 INFO:tasks.workunit.client.0.smithi144.stdout:2/662: fdatasync d7/f15 0 2024-09-10T11:29:48.281 INFO:tasks.workunit.client.0.smithi144.stdout:7/815: truncate d5/dc/d2a/d41/fc9 520496 0 2024-09-10T11:29:48.287 INFO:tasks.workunit.client.0.smithi144.stdout:5/811: mknod d4/d5/cfc 0 2024-09-10T11:29:48.332 INFO:tasks.workunit.client.0.smithi144.stdout:5/812: dread d4/d5/d32/d47/f71 [0,4194304] 0 2024-09-10T11:29:48.334 INFO:tasks.workunit.client.0.smithi144.stdout:9/820: dwrite d11/d36/db1/f10b [0,4194304] 0 2024-09-10T11:29:48.337 INFO:tasks.workunit.client.0.smithi144.stdout:5/813: truncate d4/d5/d18/d38/d6d/da6/ff8 446101 0 2024-09-10T11:29:48.337 INFO:tasks.workunit.client.0.smithi144.stdout:9/821: write d11/d36/db1/dbb/df0/d8f/dc0/dd0/f103 [852863,118670] 0 2024-09-10T11:29:48.338 INFO:tasks.workunit.client.0.smithi144.stdout:9/822: write d11/d36/db1/dbb/df0/f5c [1758147,122168] 0 2024-09-10T11:29:48.346 INFO:tasks.workunit.client.0.smithi144.stdout:9/823: link d11/d12/d6f/daa/c110 d11/d12/d1b/d2c/d8a/c121 0 2024-09-10T11:29:48.348 INFO:tasks.workunit.client.0.smithi144.stdout:9/824: dread - d11/d12/fa6 zero size 2024-09-10T11:29:48.350 INFO:tasks.workunit.client.0.smithi144.stdout:7/816: dread d5/dc/d14/d40/d58/d6a/f92 [0,4194304] 0 2024-09-10T11:29:48.353 INFO:tasks.workunit.client.0.smithi144.stdout:7/817: truncate d5/dc/d14/f6b 442593 0 2024-09-10T11:29:48.355 INFO:tasks.workunit.client.0.smithi144.stdout:7/818: mknod d5/dc/d2a/c10d 0 2024-09-10T11:29:48.357 INFO:tasks.workunit.client.0.smithi144.stdout:7/819: fdatasync d5/dc/d12/d2b/dde/f10b 0 2024-09-10T11:29:48.360 INFO:tasks.workunit.client.0.smithi144.stdout:7/820: read d5/dc/d11/f5c [1349236,121903] 0 2024-09-10T11:29:48.361 INFO:tasks.workunit.client.0.smithi144.stdout:9/825: dwrite d11/d12/fa6 [0,4194304] 0 2024-09-10T11:29:48.365 INFO:tasks.workunit.client.0.smithi144.stdout:9/826: read d11/d36/d2d/df6/f119 [1765940,68745] 0 2024-09-10T11:29:48.367 INFO:tasks.workunit.client.0.smithi144.stdout:7/821: link d5/dc/d14/d40/d58/cb5 d5/dc/d14/d40/d58/d6a/dad/c10e 0 2024-09-10T11:29:48.371 INFO:tasks.workunit.client.0.smithi144.stdout:9/827: link d11/d36/db1/dbb/df0/l96 d11/d36/db1/dbb/l122 0 2024-09-10T11:29:48.374 INFO:tasks.workunit.client.0.smithi144.stdout:7/822: getdents d5/dc 0 2024-09-10T11:29:48.375 INFO:tasks.workunit.client.0.smithi144.stdout:7/823: readlink d5/dc/d11/l6c 0 2024-09-10T11:29:48.381 INFO:tasks.workunit.client.0.smithi144.stdout:9/828: dwrite d11/d36/fa4 [4194304,4194304] 0 2024-09-10T11:29:48.381 INFO:tasks.workunit.client.0.smithi144.stdout:7/824: link d5/dc/d14/d40/d5e/d68/f8f d5/dc/d14/d40/d58/d6a/dad/f10f 0 2024-09-10T11:29:48.383 INFO:tasks.workunit.client.0.smithi144.stdout:7/825: chown d5/dc/d2a/le9 238 1 2024-09-10T11:29:48.384 INFO:tasks.workunit.client.0.smithi144.stdout:9/829: mkdir d11/d36/db1/dbb/df0/d123 0 2024-09-10T11:29:48.386 INFO:tasks.workunit.client.0.smithi144.stdout:7/826: mknod d5/dc/d14/d40/c110 0 2024-09-10T11:29:48.386 INFO:tasks.workunit.client.0.smithi144.stdout:9/830: mkdir d11/d36/db1/dbb/df0/d8f/dc0/d124 0 2024-09-10T11:29:48.390 INFO:tasks.workunit.client.0.smithi144.stdout:9/831: symlink d11/d36/d2d/d38/de7/l125 0 2024-09-10T11:29:48.395 INFO:tasks.workunit.client.0.smithi144.stdout:7/827: write d5/dc/d2a/d60/dd6/d98/dd1/d7c/f85 [845725,84641] 0 2024-09-10T11:29:48.398 INFO:tasks.workunit.client.0.smithi144.stdout:7/828: mknod d5/dc/d12/d10a/c111 0 2024-09-10T11:29:48.399 INFO:tasks.workunit.client.0.smithi144.stdout:9/832: dwrite d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/fe3 [0,4194304] 0 2024-09-10T11:29:48.401 INFO:tasks.workunit.client.0.smithi144.stdout:7/829: symlink d5/dc/d2a/d9d/l112 0 2024-09-10T11:29:48.403 INFO:tasks.workunit.client.0.smithi144.stdout:7/830: fdatasync d5/dc/d2a/d41/f49 0 2024-09-10T11:29:48.420 INFO:tasks.workunit.client.0.smithi144.stdout:7/831: dread d5/fa [0,4194304] 0 2024-09-10T11:29:48.422 INFO:tasks.workunit.client.0.smithi144.stdout:7/832: mknod d5/dc/d14/d40/d5e/ddd/c113 0 2024-09-10T11:29:48.423 INFO:tasks.workunit.client.0.smithi144.stdout:9/833: dread d11/d12/d1b/df8/fb6 [0,4194304] 0 2024-09-10T11:29:48.427 INFO:tasks.workunit.client.0.smithi144.stdout:9/834: rename d11/d12/d1b/d2c/lc7 to d11/d12/d1b/d2c/d78/dd2/dec/l126 0 2024-09-10T11:29:48.430 INFO:tasks.workunit.client.0.smithi144.stdout:7/833: dwrite d5/dc/d14/fb3 [0,4194304] 0 2024-09-10T11:29:48.433 INFO:tasks.workunit.client.0.smithi144.stdout:9/835: rmdir d11/d12/d1b/d2c/d78/dd2/d106 0 2024-09-10T11:29:48.435 INFO:tasks.workunit.client.0.smithi144.stdout:7/834: symlink d5/dc/def/l114 0 2024-09-10T11:29:48.439 INFO:tasks.workunit.client.0.smithi144.stdout:9/836: dread d11/d36/db1/dbb/df0/fd8 [0,4194304] 0 2024-09-10T11:29:48.440 INFO:tasks.workunit.client.0.smithi144.stdout:7/835: truncate d5/dc/d11/d18/f1e 1609903 0 2024-09-10T11:29:48.444 INFO:tasks.workunit.client.0.smithi144.stdout:7/836: getdents d5/dc/d14 0 2024-09-10T11:29:48.445 INFO:tasks.workunit.client.0.smithi144.stdout:9/837: dread d11/d36/d2d/d38/f11a [4194304,4194304] 0 2024-09-10T11:29:48.450 INFO:tasks.workunit.client.0.smithi144.stdout:9/838: rmdir d11/d36/db1/dbb/df0/d8f/dc0/d124 0 2024-09-10T11:29:48.453 INFO:tasks.workunit.client.0.smithi144.stdout:9/839: creat d11/d36/d2d/d38/dc3/f127 x:0 0 0 2024-09-10T11:29:48.454 INFO:tasks.workunit.client.0.smithi144.stdout:7/837: dwrite d5/dc/d2a/d60/dd6/d98/dd1/d7c/f85 [0,4194304] 0 2024-09-10T11:29:48.457 INFO:tasks.workunit.client.0.smithi144.stdout:7/838: rmdir d5/dc/d14/d40/d58/d6e/d7a 39 2024-09-10T11:29:48.461 INFO:tasks.workunit.client.0.smithi144.stdout:7/839: mkdir d5/dc/d2a/d60/dd6/d98/d102/d115 0 2024-09-10T11:29:48.463 INFO:tasks.workunit.client.0.smithi144.stdout:9/840: dwrite d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/fa9 [0,4194304] 0 2024-09-10T11:29:48.466 INFO:tasks.workunit.client.0.smithi144.stdout:7/840: fdatasync d5/dc/d14/d40/fb0 0 2024-09-10T11:29:48.467 INFO:tasks.workunit.client.0.smithi144.stdout:9/841: creat d11/d36/d2d/d38/d116/f128 x:0 0 0 2024-09-10T11:29:48.471 INFO:tasks.workunit.client.0.smithi144.stdout:9/842: readlink d11/d36/db1/dbb/df0/lcc 0 2024-09-10T11:29:48.472 INFO:tasks.workunit.client.0.smithi144.stdout:9/843: chown d11/d36/cd5 63925272 1 2024-09-10T11:29:48.473 INFO:tasks.workunit.client.0.smithi144.stdout:7/841: dwrite d5/dc/d11/f81 [0,4194304] 0 2024-09-10T11:29:48.474 INFO:tasks.workunit.client.0.smithi144.stdout:9/844: write d11/d36/d2d/d38/dc3/f127 [375506,25767] 0 2024-09-10T11:29:48.477 INFO:tasks.workunit.client.0.smithi144.stdout:9/845: write d11/d36/db1/dbb/df0/d8f/dc0/dd0/fe9 [1369587,40263] 0 2024-09-10T11:29:48.484 INFO:tasks.workunit.client.0.smithi144.stdout:9/846: dwrite d11/d36/d2d/d38/dc3/f127 [0,4194304] 0 2024-09-10T11:29:48.486 INFO:tasks.workunit.client.0.smithi144.stdout:9/847: write d11/d36/db1/dbb/df0/f5c [2901359,39817] 0 2024-09-10T11:29:48.488 INFO:tasks.workunit.client.0.smithi144.stdout:9/848: creat d11/d12/d1b/d2c/d78/dd2/f129 x:0 0 0 2024-09-10T11:29:48.499 INFO:tasks.workunit.client.0.smithi144.stdout:7/842: dwrite d5/dc/d2a/d60/dd6/d98/fca [0,4194304] 0 2024-09-10T11:29:48.500 INFO:tasks.workunit.client.0.smithi144.stdout:7/843: fsync d5/dc/d14/f44 0 2024-09-10T11:29:48.503 INFO:tasks.workunit.client.0.smithi144.stdout:7/844: mknod d5/dc/d12/d2b/c116 0 2024-09-10T11:29:48.505 INFO:tasks.workunit.client.0.smithi144.stdout:7/845: mknod d5/dc/d14/d40/d5e/ddd/c117 0 2024-09-10T11:29:48.509 INFO:tasks.workunit.client.0.smithi144.stdout:9/849: dread d11/d12/d6f/fd3 [0,4194304] 0 2024-09-10T11:29:48.511 INFO:tasks.workunit.client.0.smithi144.stdout:9/850: mknod d11/d11c/c12a 0 2024-09-10T11:29:48.512 INFO:tasks.workunit.client.0.smithi144.stdout:9/851: rmdir d11/d36/db1/dbb/df0 39 2024-09-10T11:29:48.526 INFO:tasks.workunit.client.0.smithi144.stdout:9/852: dread d11/d36/db1/fc4 [0,4194304] 0 2024-09-10T11:29:48.528 INFO:tasks.workunit.client.0.smithi144.stdout:9/853: mkdir d11/d12/d1b/d2c/d12b 0 2024-09-10T11:29:48.529 INFO:tasks.workunit.client.0.smithi144.stdout:9/854: mkdir d11/d36/d2d/d38/d12c 0 2024-09-10T11:29:48.533 INFO:tasks.workunit.client.0.smithi144.stdout:9/855: unlink d11/d36/db1/dbb/df0/d8f/le6 0 2024-09-10T11:29:48.538 INFO:tasks.workunit.client.0.smithi144.stdout:9/856: write d11/d36/d2d/d38/f76 [6782999,18995] 0 2024-09-10T11:29:48.896 INFO:tasks.workunit.client.0.smithi144.stdout:0/701: sync 2024-09-10T11:29:48.898 INFO:tasks.workunit.client.0.smithi144.stdout:0/702: mkdir da/d3a/d4b/dd4 0 2024-09-10T11:29:48.907 INFO:tasks.workunit.client.0.smithi144.stdout:0/703: dread da/d44/f54 [0,4194304] 0 2024-09-10T11:29:48.909 INFO:tasks.workunit.client.0.smithi144.stdout:0/704: symlink da/d3a/d51/ld5 0 2024-09-10T11:29:48.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:48 smithi144 ceph-mon[27942]: pgmap v31: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 111 MiB/s rd, 270 MiB/s wr, 544 op/s 2024-09-10T11:29:49.065 INFO:tasks.workunit.client.0.smithi144.stdout:4/866: sync 2024-09-10T11:29:49.069 INFO:tasks.workunit.client.0.smithi144.stdout:4/867: unlink d14/d1e/d22/d34/d91/l74 0 2024-09-10T11:29:49.072 INFO:tasks.workunit.client.0.smithi144.stdout:4/868: creat d14/d1e/d22/d34/d3e/d40/f13a x:0 0 0 2024-09-10T11:29:49.077 INFO:tasks.workunit.client.0.smithi144.stdout:4/869: dread d14/d7e/f109 [0,4194304] 0 2024-09-10T11:29:49.086 INFO:tasks.workunit.client.0.smithi144.stdout:4/870: truncate d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/fb0 3722027 0 2024-09-10T11:29:49.096 INFO:tasks.workunit.client.0.smithi144.stdout:4/871: dread d14/d1e/d22/d34/d3e/d40/d4b/f52 [0,4194304] 0 2024-09-10T11:29:49.096 INFO:tasks.workunit.client.0.smithi144.stdout:4/872: chown d14/d1e/d83/da4/df4 722615522 1 2024-09-10T11:29:49.100 INFO:tasks.workunit.client.0.smithi144.stdout:4/873: creat d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f/d103/f13b x:0 0 0 2024-09-10T11:29:49.102 INFO:tasks.workunit.client.0.smithi144.stdout:4/874: mkdir d14/d1e/d22/d34/d3e/d40/d4b/d4c/dab/d13c 0 2024-09-10T11:29:49.104 INFO:tasks.workunit.client.0.smithi144.stdout:4/875: read d14/d67/fb1 [3931345,20818] 0 2024-09-10T11:29:49.106 INFO:tasks.workunit.client.0.smithi144.stdout:4/876: creat d14/d1e/d22/f13d x:0 0 0 2024-09-10T11:29:49.107 INFO:tasks.workunit.client.0.smithi144.stdout:4/877: chown d14/d67/d69/d139 90318 1 2024-09-10T11:29:49.109 INFO:tasks.workunit.client.0.smithi144.stdout:4/878: mknod d14/d1e/d22/d34/d3e/c13e 0 2024-09-10T11:29:49.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:48 smithi178 ceph-mon[33514]: pgmap v31: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 111 MiB/s rd, 270 MiB/s wr, 544 op/s 2024-09-10T11:29:49.186 INFO:tasks.workunit.client.0.smithi144.stdout:4/879: getdents d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f/d103 0 2024-09-10T11:29:49.195 INFO:tasks.workunit.client.0.smithi144.stdout:4/880: dwrite d14/d67/d69/d90/fb5 [0,4194304] 0 2024-09-10T11:29:49.195 INFO:tasks.workunit.client.0.smithi144.stdout:4/881: chown d14/d6f/c120 1203462 1 2024-09-10T11:29:49.200 INFO:tasks.workunit.client.0.smithi144.stdout:4/882: rename d14/d1e/fb6 to d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/f13f 0 2024-09-10T11:29:49.202 INFO:tasks.workunit.client.0.smithi144.stdout:4/883: mkdir d14/d67/d69/d90/dda/de7/d140 0 2024-09-10T11:29:49.205 INFO:tasks.workunit.client.0.smithi144.stdout:4/884: mkdir d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/d10b/d141 0 2024-09-10T11:29:49.208 INFO:tasks.workunit.client.0.smithi144.stdout:4/885: getdents d14 0 2024-09-10T11:29:49.210 INFO:tasks.workunit.client.0.smithi144.stdout:4/886: mknod d14/d1e/d83/da4/c142 0 2024-09-10T11:29:49.213 INFO:tasks.workunit.client.0.smithi144.stdout:4/887: mkdir d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/d10b/d141/d143 0 2024-09-10T11:29:49.217 INFO:tasks.workunit.client.0.smithi144.stdout:4/888: link d14/d1e/d22/d34/c43 d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/c144 0 2024-09-10T11:29:49.223 INFO:tasks.workunit.client.0.smithi144.stdout:4/889: truncate d14/d7e/f109 536440 0 2024-09-10T11:29:49.225 INFO:tasks.workunit.client.0.smithi144.stdout:4/890: symlink d14/d67/d84/l145 0 2024-09-10T11:29:49.227 INFO:tasks.workunit.client.0.smithi144.stdout:4/891: mkdir d14/d1e/d22/d146 0 2024-09-10T11:29:49.232 INFO:tasks.workunit.client.0.smithi144.stdout:4/892: dread d14/d1e/d22/f2d [0,4194304] 0 2024-09-10T11:29:49.238 INFO:tasks.workunit.client.0.smithi144.stdout:4/893: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/faa [7043422,70307] 0 2024-09-10T11:29:49.249 INFO:tasks.workunit.client.0.smithi144.stdout:4/894: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/f134 [0,4194304] 0 2024-09-10T11:29:49.250 INFO:tasks.workunit.client.0.smithi144.stdout:4/895: write d14/d67/d69/d90/dda/de7/f133 [316398,110901] 0 2024-09-10T11:29:49.255 INFO:tasks.workunit.client.0.smithi144.stdout:4/896: creat d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/f147 x:0 0 0 2024-09-10T11:29:49.256 INFO:tasks.workunit.client.0.smithi144.stdout:1/936: sync 2024-09-10T11:29:49.256 INFO:tasks.workunit.client.0.smithi144.stdout:3/790: sync 2024-09-10T11:29:49.259 INFO:tasks.workunit.client.0.smithi144.stdout:1/937: creat d5/df/d4b/d51/f128 x:0 0 0 2024-09-10T11:29:49.259 INFO:tasks.workunit.client.0.smithi144.stdout:1/938: readlink d5/d2a/d65/dc9/l71 0 2024-09-10T11:29:49.260 INFO:tasks.workunit.client.0.smithi144.stdout:4/897: creat d14/d67/d84/f148 x:0 0 0 2024-09-10T11:29:49.261 INFO:tasks.workunit.client.0.smithi144.stdout:4/898: write d14/d1e/f20 [431639,104055] 0 2024-09-10T11:29:49.262 INFO:tasks.workunit.client.0.smithi144.stdout:1/939: symlink d5/dd/d64/dad/l129 0 2024-09-10T11:29:49.263 INFO:tasks.workunit.client.0.smithi144.stdout:3/791: creat d4/d28/d67/d80/d17/d18/d34/d4b/ff9 x:0 0 0 2024-09-10T11:29:49.264 INFO:tasks.workunit.client.0.smithi144.stdout:3/792: fdatasync d4/d28/d67/d80/f6b 0 2024-09-10T11:29:49.265 INFO:tasks.workunit.client.0.smithi144.stdout:4/899: dread d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/ff2 [0,4194304] 0 2024-09-10T11:29:49.268 INFO:tasks.workunit.client.0.smithi144.stdout:4/900: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/fb8 [118264,36944] 0 2024-09-10T11:29:49.276 INFO:tasks.workunit.client.0.smithi144.stdout:1/940: truncate d5/df/d4b/fb4 2945272 0 2024-09-10T11:29:49.279 INFO:tasks.workunit.client.0.smithi144.stdout:3/793: dread d4/dd/fef [0,4194304] 0 2024-09-10T11:29:49.280 INFO:tasks.workunit.client.0.smithi144.stdout:3/794: fdatasync d4/d28/d67/d80/f6b 0 2024-09-10T11:29:49.281 INFO:tasks.workunit.client.0.smithi144.stdout:4/901: dwrite d14/d1e/f7c [0,4194304] 0 2024-09-10T11:29:49.283 INFO:tasks.workunit.client.0.smithi144.stdout:3/795: mknod d4/cfa 0 2024-09-10T11:29:49.285 INFO:tasks.workunit.client.0.smithi144.stdout:3/796: chown d4/dd/d21/f86 2 1 2024-09-10T11:29:49.286 INFO:tasks.workunit.client.0.smithi144.stdout:1/941: truncate d5/df/d4b/d51/f50 1641031 0 2024-09-10T11:29:49.292 INFO:tasks.workunit.client.0.smithi144.stdout:4/902: dread d14/d1e/d22/f131 [0,4194304] 0 2024-09-10T11:29:49.305 INFO:tasks.workunit.client.0.smithi144.stdout:3/797: dread d4/d28/d67/d80/d17/d18/d34/d65/d75/fb0 [0,4194304] 0 2024-09-10T11:29:49.341 INFO:tasks.workunit.client.0.smithi144.stdout:1/942: creat d5/df/d4b/d51/da7/f12a x:0 0 0 2024-09-10T11:29:49.341 INFO:tasks.workunit.client.0.smithi144.stdout:4/903: truncate d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/fb7 657437 0 2024-09-10T11:29:49.342 INFO:tasks.workunit.client.0.smithi144.stdout:3/798: creat d4/d28/dc1/ffb x:0 0 0 2024-09-10T11:29:49.344 INFO:tasks.workunit.client.0.smithi144.stdout:1/943: mknod d5/df/d4b/d51/d10a/d95/d124/c12b 0 2024-09-10T11:29:49.346 INFO:tasks.workunit.client.0.smithi144.stdout:4/904: rename d14/f3a to d14/d67/d69/d90/d102/f149 0 2024-09-10T11:29:49.347 INFO:tasks.workunit.client.0.smithi144.stdout:3/799: symlink d4/d28/d67/d80/d17/dd0/lfc 0 2024-09-10T11:29:49.351 INFO:tasks.workunit.client.0.smithi144.stdout:3/800: mkdir d4/dd/dfd 0 2024-09-10T11:29:49.352 INFO:tasks.workunit.client.0.smithi144.stdout:3/801: fdatasync d4/dd/d21/f8b 0 2024-09-10T11:29:49.352 INFO:tasks.workunit.client.0.smithi144.stdout:1/944: creat d5/f12c x:0 0 0 2024-09-10T11:29:49.355 INFO:tasks.workunit.client.0.smithi144.stdout:1/945: creat d5/dd/d64/d79/f12d x:0 0 0 2024-09-10T11:29:49.356 INFO:tasks.workunit.client.0.smithi144.stdout:3/802: symlink d4/d28/d67/d96/dcc/dec/lfe 0 2024-09-10T11:29:49.357 INFO:tasks.workunit.client.0.smithi144.stdout:3/803: write d4/dd/d21/f5b [2013704,49685] 0 2024-09-10T11:29:49.358 INFO:tasks.workunit.client.0.smithi144.stdout:1/946: creat d5/d2a/d65/f12e x:0 0 0 2024-09-10T11:29:49.360 INFO:tasks.workunit.client.0.smithi144.stdout:1/947: fdatasync d5/f15 0 2024-09-10T11:29:49.361 INFO:tasks.workunit.client.0.smithi144.stdout:3/804: symlink d4/dd/lff 0 2024-09-10T11:29:49.362 INFO:tasks.workunit.client.0.smithi144.stdout:3/805: chown d4/dd/d21/d2f 19550 1 2024-09-10T11:29:49.365 INFO:tasks.workunit.client.0.smithi144.stdout:3/806: symlink d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/l100 0 2024-09-10T11:29:49.367 INFO:tasks.workunit.client.0.smithi144.stdout:3/807: write d4/d1a/f44 [3691969,19729] 0 2024-09-10T11:29:49.370 INFO:tasks.workunit.client.0.smithi144.stdout:3/808: mknod d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/c101 0 2024-09-10T11:29:49.374 INFO:tasks.workunit.client.0.smithi144.stdout:3/809: creat d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/f102 x:0 0 0 2024-09-10T11:29:49.376 INFO:tasks.workunit.client.0.smithi144.stdout:3/810: symlink d4/d28/d67/d80/d17/dd0/d98/l103 0 2024-09-10T11:29:49.383 INFO:tasks.workunit.client.0.smithi144.stdout:1/948: truncate d5/df/d4b/d51/d10a/d95/dd2/fde 517740 0 2024-09-10T11:29:49.387 INFO:tasks.workunit.client.0.smithi144.stdout:1/949: dread d5/d4a/d44/da2/fd7 [0,4194304] 0 2024-09-10T11:29:49.389 INFO:tasks.workunit.client.0.smithi144.stdout:1/950: creat d5/df/d4b/d51/d10a/d95/dd2/f12f x:0 0 0 2024-09-10T11:29:49.406 INFO:tasks.workunit.client.0.smithi144.stdout:8/710: sync 2024-09-10T11:29:49.406 INFO:tasks.workunit.client.0.smithi144.stdout:6/717: sync 2024-09-10T11:29:49.422 INFO:tasks.workunit.client.0.smithi144.stdout:4/905: rename d14/d1e/d22/d34/d3e/lcc to d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/dd4/l14a 0 2024-09-10T11:29:49.422 INFO:tasks.workunit.client.0.smithi144.stdout:1/951: getdents d5/df/d4b/d51/d10a/dab 0 2024-09-10T11:29:49.427 INFO:tasks.workunit.client.0.smithi144.stdout:8/711: write d0/d9/d21/f73 [164393,29700] 0 2024-09-10T11:29:49.427 INFO:tasks.workunit.client.0.smithi144.stdout:6/718: link d9/d1f/d2c/l44 d9/dd/d11/d1b/d2f/d7f/d82/lff 0 2024-09-10T11:29:49.428 INFO:tasks.workunit.client.0.smithi144.stdout:4/906: mknod d14/d67/d69/c14b 0 2024-09-10T11:29:49.431 INFO:tasks.workunit.client.0.smithi144.stdout:1/952: getdents d5/dd/d9c 0 2024-09-10T11:29:49.432 INFO:tasks.workunit.client.0.smithi144.stdout:1/953: truncate d5/d2a/f34 4908279 0 2024-09-10T11:29:49.433 INFO:tasks.workunit.client.0.smithi144.stdout:1/954: stat d5/df/d4b/d51/d10a/dbd 0 2024-09-10T11:29:49.434 INFO:tasks.workunit.client.0.smithi144.stdout:4/907: dread d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/fd2 [0,4194304] 0 2024-09-10T11:29:49.434 INFO:tasks.workunit.client.0.smithi144.stdout:6/719: link d9/dd/d11/d1b/d2f/dab/fce d9/f100 0 2024-09-10T11:29:49.434 INFO:tasks.workunit.client.0.smithi144.stdout:4/908: dread - d14/d1e/fcd zero size 2024-09-10T11:29:49.435 INFO:tasks.workunit.client.0.smithi144.stdout:1/955: symlink d5/d4a/d44/da2/l130 0 2024-09-10T11:29:49.436 INFO:tasks.workunit.client.0.smithi144.stdout:8/712: write d0/f24 [3152224,116231] 0 2024-09-10T11:29:49.437 INFO:tasks.workunit.client.0.smithi144.stdout:4/909: stat d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/d10b/c116 0 2024-09-10T11:29:49.438 INFO:tasks.workunit.client.0.smithi144.stdout:6/720: creat d9/dd/f101 x:0 0 0 2024-09-10T11:29:49.440 INFO:tasks.workunit.client.0.smithi144.stdout:1/956: fdatasync d5/d2a/d65/d73/f24 0 2024-09-10T11:29:49.442 INFO:tasks.workunit.client.0.smithi144.stdout:6/721: symlink d9/dd/d11/d1b/d2f/d7f/dc4/d5b/dc8/l102 0 2024-09-10T11:29:49.446 INFO:tasks.workunit.client.0.smithi144.stdout:8/713: dwrite d0/d9/d21/d7e/fe9 [0,4194304] 0 2024-09-10T11:29:49.447 INFO:tasks.workunit.client.0.smithi144.stdout:4/910: getdents d14/d1e/d22/d34/d3e/d40/d4b/d4c/dab/df3 0 2024-09-10T11:29:49.447 INFO:tasks.workunit.client.0.smithi144.stdout:6/722: read d9/dd/d11/d1b/f94 [995634,18172] 0 2024-09-10T11:29:49.448 INFO:tasks.workunit.client.0.smithi144.stdout:1/957: getdents d5/df/d4b/d85 0 2024-09-10T11:29:49.450 INFO:tasks.workunit.client.0.smithi144.stdout:8/714: mkdir d0/d1b/d47/dfa 0 2024-09-10T11:29:49.451 INFO:tasks.workunit.client.0.smithi144.stdout:8/715: stat d0/f3f 0 2024-09-10T11:29:49.453 INFO:tasks.workunit.client.0.smithi144.stdout:8/716: fdatasync d0/f13 0 2024-09-10T11:29:49.456 INFO:tasks.workunit.client.0.smithi144.stdout:6/723: symlink d9/dd/d3e/d47/l103 0 2024-09-10T11:29:49.462 INFO:tasks.workunit.client.0.smithi144.stdout:6/724: unlink d9/dd/d11/f9d 0 2024-09-10T11:29:49.462 INFO:tasks.workunit.client.0.smithi144.stdout:8/717: dwrite d0/d1b/d47/d5e/f64 [0,4194304] 0 2024-09-10T11:29:49.467 INFO:tasks.workunit.client.0.smithi144.stdout:6/725: truncate d9/f2a 2564637 0 2024-09-10T11:29:49.468 INFO:tasks.workunit.client.0.smithi144.stdout:4/911: dwrite d14/dcb/fce [0,4194304] 0 2024-09-10T11:29:49.469 INFO:tasks.workunit.client.0.smithi144.stdout:8/718: unlink d0/d9/d21/d25/d4f/cbf 0 2024-09-10T11:29:49.475 INFO:tasks.workunit.client.0.smithi144.stdout:1/958: dwrite d5/df/d4b/d51/d10a/d95/dd2/fde [0,4194304] 0 2024-09-10T11:29:49.477 INFO:tasks.workunit.client.0.smithi144.stdout:8/719: mknod d0/d1b/d47/d5e/d76/cfb 0 2024-09-10T11:29:49.477 INFO:tasks.workunit.client.0.smithi144.stdout:6/726: chown d9/dd/d11/d1b/d2f/d7f/d82/l7b 610 1 2024-09-10T11:29:49.480 INFO:tasks.workunit.client.0.smithi144.stdout:6/727: chown d9/dd/f49 31607 1 2024-09-10T11:29:49.480 INFO:tasks.workunit.client.0.smithi144.stdout:1/959: chown d5/df/d4b/d51/d10a/df9 25 1 2024-09-10T11:29:49.482 INFO:tasks.workunit.client.0.smithi144.stdout:1/960: readlink d5/d2a/d65/dc9/lc2 0 2024-09-10T11:29:49.486 INFO:tasks.workunit.client.0.smithi144.stdout:1/961: dread - d5/d2a/fe0 zero size 2024-09-10T11:29:49.486 INFO:tasks.workunit.client.0.smithi144.stdout:6/728: mknod d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/df0/c104 0 2024-09-10T11:29:49.489 INFO:tasks.workunit.client.0.smithi144.stdout:8/720: dwrite d0/d9/d21/d25/d4f/f56 [0,4194304] 0 2024-09-10T11:29:49.489 INFO:tasks.workunit.client.0.smithi144.stdout:1/962: chown d5/d2a/f74 102275010 1 2024-09-10T11:29:49.489 INFO:tasks.workunit.client.0.smithi144.stdout:6/729: write d9/d1f/d2c/d4f/f58 [30617,25446] 0 2024-09-10T11:29:49.493 INFO:tasks.workunit.client.0.smithi144.stdout:1/963: fdatasync d5/df/d4b/f8e 0 2024-09-10T11:29:49.495 INFO:tasks.workunit.client.0.smithi144.stdout:6/730: truncate d9/dd/d11/d1b/d2f/dab/de1/fe6 1244433 0 2024-09-10T11:29:49.498 INFO:tasks.workunit.client.0.smithi144.stdout:6/731: write d9/dd/d11/d1b/d2f/d7f/f92 [736546,76383] 0 2024-09-10T11:29:49.501 INFO:tasks.workunit.client.0.smithi144.stdout:4/912: dread d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f/f60 [0,4194304] 0 2024-09-10T11:29:49.511 INFO:tasks.workunit.client.0.smithi144.stdout:1/964: truncate d5/df/d4b/d51/d10a/d95/d124/f122 300532 0 2024-09-10T11:29:49.512 INFO:tasks.workunit.client.0.smithi144.stdout:4/913: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/f38 [7715684,77134] 0 2024-09-10T11:29:49.512 INFO:tasks.workunit.client.0.smithi144.stdout:6/732: truncate d9/dd/d11/d1b/d2f/d7f/fc7 1400760 0 2024-09-10T11:29:49.514 INFO:tasks.workunit.client.0.smithi144.stdout:8/721: dwrite d0/d9/fe [0,4194304] 0 2024-09-10T11:29:49.514 INFO:tasks.workunit.client.0.smithi144.stdout:1/965: fsync d5/df/d4b/d51/d10a/dab/f115 0 2024-09-10T11:29:49.515 INFO:tasks.workunit.client.0.smithi144.stdout:4/914: readlink d14/d1e/d83/lcf 0 2024-09-10T11:29:49.515 INFO:tasks.workunit.client.0.smithi144.stdout:8/722: chown d0/d9/d1f/d63 3047114 1 2024-09-10T11:29:49.516 INFO:tasks.workunit.client.0.smithi144.stdout:1/966: dread - d5/df/d4b/dcf/f127 zero size 2024-09-10T11:29:49.518 INFO:tasks.workunit.client.0.smithi144.stdout:6/733: mkdir d9/dd/d11/d1b/d2f/dab/de7/d105 0 2024-09-10T11:29:49.520 INFO:tasks.workunit.client.0.smithi144.stdout:4/915: creat d14/d1e/d22/d34/f14c x:0 0 0 2024-09-10T11:29:49.521 INFO:tasks.workunit.client.0.smithi144.stdout:0/705: sync 2024-09-10T11:29:49.521 INFO:tasks.workunit.client.0.smithi144.stdout:2/663: sync 2024-09-10T11:29:49.521 INFO:tasks.workunit.client.0.smithi144.stdout:9/857: sync 2024-09-10T11:29:49.521 INFO:tasks.workunit.client.0.smithi144.stdout:7/846: sync 2024-09-10T11:29:49.522 INFO:tasks.workunit.client.0.smithi144.stdout:5/814: sync 2024-09-10T11:29:49.525 INFO:tasks.workunit.client.0.smithi144.stdout:8/723: truncate d0/d9/d23/d26/f36 7316146 0 2024-09-10T11:29:49.526 INFO:tasks.workunit.client.0.smithi144.stdout:5/815: write d4/d5/d32/d47/f6a [3636790,55983] 0 2024-09-10T11:29:49.527 INFO:tasks.workunit.client.0.smithi144.stdout:0/706: write da/d3a/d4b/d4e/d81/f98 [4205705,81176] 0 2024-09-10T11:29:49.527 INFO:tasks.workunit.client.0.smithi144.stdout:2/664: creat d7/de/d5f/fd7 x:0 0 0 2024-09-10T11:29:49.529 INFO:tasks.workunit.client.0.smithi144.stdout:9/858: mkdir d11/d36/d2d/d38/de7/d12d 0 2024-09-10T11:29:49.531 INFO:tasks.workunit.client.0.smithi144.stdout:5/816: write d4/d5/d18/d38/d6d/d8b/f9b [1498571,112973] 0 2024-09-10T11:29:49.537 INFO:tasks.workunit.client.0.smithi144.stdout:1/967: getdents d5/dd/da1 0 2024-09-10T11:29:49.540 INFO:tasks.workunit.client.0.smithi144.stdout:4/916: creat d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/d10b/d141/d143/f14d x:0 0 0 2024-09-10T11:29:49.543 INFO:tasks.workunit.client.0.smithi144.stdout:2/665: creat d7/de/d3d/d4e/fd8 x:0 0 0 2024-09-10T11:29:49.545 INFO:tasks.workunit.client.0.smithi144.stdout:1/968: creat d5/dd/d64/d79/d101/f131 x:0 0 0 2024-09-10T11:29:49.549 INFO:tasks.workunit.client.0.smithi144.stdout:9/859: mkdir d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/d12e 0 2024-09-10T11:29:49.552 INFO:tasks.workunit.client.0.smithi144.stdout:5/817: dwrite d4/d5/d18/d38/d8d/fa8 [0,4194304] 0 2024-09-10T11:29:49.552 INFO:tasks.workunit.client.0.smithi144.stdout:0/707: dwrite da/d3a/d94/db1/fcb [0,4194304] 0 2024-09-10T11:29:49.557 INFO:tasks.workunit.client.0.smithi144.stdout:4/917: read - d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/ff6 zero size 2024-09-10T11:29:49.558 INFO:tasks.workunit.client.0.smithi144.stdout:6/734: dwrite d9/d1f/d2c/f67 [0,4194304] 0 2024-09-10T11:29:49.562 INFO:tasks.workunit.client.0.smithi144.stdout:0/708: truncate da/d3a/d4b/fb7 1841520 0 2024-09-10T11:29:49.563 INFO:tasks.workunit.client.0.smithi144.stdout:1/969: creat d5/df/d4b/f132 x:0 0 0 2024-09-10T11:29:49.564 INFO:tasks.workunit.client.0.smithi144.stdout:2/666: dread d7/de/f3b [0,4194304] 0 2024-09-10T11:29:49.566 INFO:tasks.workunit.client.0.smithi144.stdout:4/918: chown d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/ld8 21426759 1 2024-09-10T11:29:49.568 INFO:tasks.workunit.client.0.smithi144.stdout:6/735: fdatasync d9/dd/d3e/d47/fcd 0 2024-09-10T11:29:49.570 INFO:tasks.workunit.client.0.smithi144.stdout:7/847: dread d5/f37 [0,4194304] 0 2024-09-10T11:29:49.571 INFO:tasks.workunit.client.0.smithi144.stdout:2/667: truncate d7/de/d36/f74 1262909 0 2024-09-10T11:29:49.572 INFO:tasks.workunit.client.0.smithi144.stdout:0/709: dread da/d44/f54 [0,4194304] 0 2024-09-10T11:29:49.575 INFO:tasks.workunit.client.0.smithi144.stdout:4/919: rename d14/d1e/d22/d34/d3e/d40/d4b/d4c/f58 to d14/d67/d69/d90/d102/f14e 0 2024-09-10T11:29:49.576 INFO:tasks.workunit.client.0.smithi144.stdout:4/920: write d14/d1e/d22/d34/d3e/f124 [941888,121533] 0 2024-09-10T11:29:49.577 INFO:tasks.workunit.client.0.smithi144.stdout:6/736: creat d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/f106 x:0 0 0 2024-09-10T11:29:49.580 INFO:tasks.workunit.client.0.smithi144.stdout:7/848: mkdir d5/dc/d14/d40/d5e/dc4/d118 0 2024-09-10T11:29:49.582 INFO:tasks.workunit.client.0.smithi144.stdout:2/668: rmdir d7/de/d36/d45/d47/da0 0 2024-09-10T11:29:49.582 INFO:tasks.workunit.client.0.smithi144.stdout:0/710: getdents da/d44 0 2024-09-10T11:29:49.582 INFO:tasks.workunit.client.0.smithi144.stdout:9/860: dread d11/d36/f4e [0,4194304] 0 2024-09-10T11:29:49.583 INFO:tasks.workunit.client.0.smithi144.stdout:2/669: stat d7/d29/c2e 0 2024-09-10T11:29:49.588 INFO:tasks.workunit.client.0.smithi144.stdout:6/737: getdents d9/d1f 0 2024-09-10T11:29:49.589 INFO:tasks.workunit.client.0.smithi144.stdout:0/711: creat da/d44/d82/d7e/fd6 x:0 0 0 2024-09-10T11:29:49.592 INFO:tasks.workunit.client.0.smithi144.stdout:9/861: unlink d11/d36/db1/dbb/df0/d8f/c86 0 2024-09-10T11:29:49.592 INFO:tasks.workunit.client.0.smithi144.stdout:1/970: dwrite d5/dd/f13 [0,4194304] 0 2024-09-10T11:29:49.595 INFO:tasks.workunit.client.0.smithi144.stdout:9/862: write d11/d12/d1b/ffd [1347129,122313] 0 2024-09-10T11:29:49.595 INFO:tasks.workunit.client.0.smithi144.stdout:7/849: truncate d5/dc/d11/d18/f1e 723027 0 2024-09-10T11:29:49.596 INFO:tasks.workunit.client.0.smithi144.stdout:4/921: dread d14/d1e/d22/f31 [0,4194304] 0 2024-09-10T11:29:49.597 INFO:tasks.workunit.client.0.smithi144.stdout:0/712: mkdir da/d3a/d94/db1/dd7 0 2024-09-10T11:29:49.597 INFO:tasks.workunit.client.0.smithi144.stdout:2/670: creat d7/de/d36/d45/d8e/d9a/d7e/fd9 x:0 0 0 2024-09-10T11:29:49.598 INFO:tasks.workunit.client.0.smithi144.stdout:6/738: creat d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/df0/f107 x:0 0 0 2024-09-10T11:29:49.604 INFO:tasks.workunit.client.0.smithi144.stdout:1/971: dwrite d5/dd/da1/fd4 [4194304,4194304] 0 2024-09-10T11:29:49.604 INFO:tasks.workunit.client.0.smithi144.stdout:2/671: creat d7/de/d36/d45/d8e/d9a/d54/fda x:0 0 0 2024-09-10T11:29:49.606 INFO:tasks.workunit.client.0.smithi144.stdout:4/922: write d14/d6f/f9c [4451635,67340] 0 2024-09-10T11:29:49.606 INFO:tasks.workunit.client.0.smithi144.stdout:6/739: fdatasync d9/dd/d11/d1b/d2f/dab/de1/fe6 0 2024-09-10T11:29:49.614 INFO:tasks.workunit.client.0.smithi144.stdout:9/863: creat d11/d36/db1/f12f x:0 0 0 2024-09-10T11:29:49.616 INFO:tasks.workunit.client.0.smithi144.stdout:4/923: rmdir d14/d1e/d22/d34/d3e 39 2024-09-10T11:29:49.618 INFO:tasks.workunit.client.0.smithi144.stdout:0/713: rmdir da/d3a/d94/db1/dd7 0 2024-09-10T11:29:49.619 INFO:tasks.workunit.client.0.smithi144.stdout:9/864: dread d11/d12/d1b/df8/fb6 [0,4194304] 0 2024-09-10T11:29:49.619 INFO:tasks.workunit.client.0.smithi144.stdout:1/972: fdatasync d5/d2a/d65/d73/fd5 0 2024-09-10T11:29:49.622 INFO:tasks.workunit.client.0.smithi144.stdout:0/714: mkdir da/d44/dd8 0 2024-09-10T11:29:49.625 INFO:tasks.workunit.client.0.smithi144.stdout:2/672: dwrite d7/de/f22 [0,4194304] 0 2024-09-10T11:29:49.626 INFO:tasks.workunit.client.0.smithi144.stdout:6/740: creat d9/dd/d11/d1b/d2f/d7f/dc4/d5b/f108 x:0 0 0 2024-09-10T11:29:49.627 INFO:tasks.workunit.client.0.smithi144.stdout:9/865: write d11/d12/f14 [7522040,90025] 0 2024-09-10T11:29:49.629 INFO:tasks.workunit.client.0.smithi144.stdout:1/973: read d5/d4a/d63/f9f [3822218,73196] 0 2024-09-10T11:29:49.630 INFO:tasks.workunit.client.0.smithi144.stdout:2/673: unlink d7/da/f35 0 2024-09-10T11:29:49.633 INFO:tasks.workunit.client.0.smithi144.stdout:4/924: link d14/dcb/f114 d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f/f14f 0 2024-09-10T11:29:49.636 INFO:tasks.workunit.client.0.smithi144.stdout:0/715: symlink da/d3a/d4b/d4e/d62/ld9 0 2024-09-10T11:29:49.637 INFO:tasks.workunit.client.0.smithi144.stdout:1/974: truncate d5/d2a/f96 812029 0 2024-09-10T11:29:49.638 INFO:tasks.workunit.client.0.smithi144.stdout:6/741: truncate d9/f2a 369745 0 2024-09-10T11:29:49.642 INFO:tasks.workunit.client.0.smithi144.stdout:1/975: unlink d5/d4a/d44/ff6 0 2024-09-10T11:29:49.644 INFO:tasks.workunit.client.0.smithi144.stdout:2/674: rename d7/d29/d92/dd2 to d7/da/ddb 0 2024-09-10T11:29:49.644 INFO:tasks.workunit.client.0.smithi144.stdout:4/925: dwrite d14/d1e/d22/d34/d3e/d40/d4b/f132 [0,4194304] 0 2024-09-10T11:29:49.653 INFO:tasks.workunit.client.0.smithi144.stdout:0/716: dwrite da/d3a/f50 [0,4194304] 0 2024-09-10T11:29:49.655 INFO:tasks.workunit.client.0.smithi144.stdout:9/866: dwrite d11/d36/db1/fc6 [0,4194304] 0 2024-09-10T11:29:49.656 INFO:tasks.workunit.client.0.smithi144.stdout:4/926: chown d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/f13f 0 1 2024-09-10T11:29:49.659 INFO:tasks.workunit.client.0.smithi144.stdout:2/675: mkdir d7/d29/d8f/ddc 0 2024-09-10T11:29:49.659 INFO:tasks.workunit.client.0.smithi144.stdout:1/976: creat d5/f133 x:0 0 0 2024-09-10T11:29:49.661 INFO:tasks.workunit.client.0.smithi144.stdout:6/742: rename d9/dd/d11/d1b/d2f/dab/dc6/ff6 to d9/dd/d11/d1b/d2f/dab/de7/d105/f109 0 2024-09-10T11:29:49.662 INFO:tasks.workunit.client.0.smithi144.stdout:2/676: creat d7/dcc/fdd x:0 0 0 2024-09-10T11:29:49.662 INFO:tasks.workunit.client.0.smithi144.stdout:9/867: rmdir d11/d36/d2d 39 2024-09-10T11:29:49.664 INFO:tasks.workunit.client.0.smithi144.stdout:9/868: fdatasync d11/d36/db1/f12f 0 2024-09-10T11:29:49.665 INFO:tasks.workunit.client.0.smithi144.stdout:6/743: mkdir d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d10a 0 2024-09-10T11:29:49.667 INFO:tasks.workunit.client.0.smithi144.stdout:1/977: symlink d5/dd/da1/d11c/l134 0 2024-09-10T11:29:49.668 INFO:tasks.workunit.client.0.smithi144.stdout:0/717: dread da/f28 [0,4194304] 0 2024-09-10T11:29:49.669 INFO:tasks.workunit.client.0.smithi144.stdout:1/978: dread - d5/df/d4b/dcf/f127 zero size 2024-09-10T11:29:49.669 INFO:tasks.workunit.client.0.smithi144.stdout:4/927: dwrite d14/d1e/d83/da4/fc4 [0,4194304] 0 2024-09-10T11:29:49.669 INFO:tasks.workunit.client.0.smithi144.stdout:9/869: symlink d11/d12/d6f/l130 0 2024-09-10T11:29:49.670 INFO:tasks.workunit.client.0.smithi144.stdout:6/744: fdatasync d9/dd/d3e/d47/f98 0 2024-09-10T11:29:49.672 INFO:tasks.workunit.client.0.smithi144.stdout:4/928: dread - d14/d1e/d83/da4/f122 zero size 2024-09-10T11:29:49.674 INFO:tasks.workunit.client.0.smithi144.stdout:0/718: getdents da/d3a/d4b/d4e/d62/dd2 0 2024-09-10T11:29:49.675 INFO:tasks.workunit.client.0.smithi144.stdout:1/979: symlink d5/dd/d64/l135 0 2024-09-10T11:29:49.685 INFO:tasks.workunit.client.0.smithi144.stdout:3/811: sync 2024-09-10T11:29:49.686 INFO:tasks.workunit.client.0.smithi144.stdout:6/745: unlink d9/d1f/d2c/ccb 0 2024-09-10T11:29:49.689 INFO:tasks.workunit.client.0.smithi144.stdout:6/746: fdatasync d9/d1f/f37 0 2024-09-10T11:29:49.690 INFO:tasks.workunit.client.0.smithi144.stdout:0/719: rmdir da 39 2024-09-10T11:29:49.691 INFO:tasks.workunit.client.0.smithi144.stdout:3/812: readlink d4/d28/d67/d80/d17/d18/d34/l5e 0 2024-09-10T11:29:49.846 INFO:tasks.workunit.client.0.smithi144.stdout:1/980: creat d5/df/d4b/d85/f136 x:0 0 0 2024-09-10T11:29:49.847 INFO:tasks.workunit.client.0.smithi144.stdout:9/870: dwrite d11/d12/d1b/d2c/f31 [0,4194304] 0 2024-09-10T11:29:49.853 INFO:tasks.workunit.client.0.smithi144.stdout:1/981: creat d5/d4a/dc0/f137 x:0 0 0 2024-09-10T11:29:49.853 INFO:tasks.workunit.client.0.smithi144.stdout:2/677: dwrite d7/da/d5b/fcf [0,4194304] 0 2024-09-10T11:29:49.855 INFO:tasks.workunit.client.0.smithi144.stdout:6/747: creat d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d87/dc9/f10b x:0 0 0 2024-09-10T11:29:49.856 INFO:tasks.workunit.client.0.smithi144.stdout:0/720: chown da/d3a/d4b/d4e/d90/f71 456534614 1 2024-09-10T11:29:49.858 INFO:tasks.workunit.client.0.smithi144.stdout:1/982: symlink d5/d4a/dc0/l138 0 2024-09-10T11:29:49.860 INFO:tasks.workunit.client.0.smithi144.stdout:4/929: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/fed [0,4194304] 0 2024-09-10T11:29:49.862 INFO:tasks.workunit.client.0.smithi144.stdout:0/721: creat da/d44/fda x:0 0 0 2024-09-10T11:29:49.863 INFO:tasks.workunit.client.0.smithi144.stdout:2/678: chown d7/de/d36/d45/d8e/d9a/f2a 3074 1 2024-09-10T11:29:49.865 INFO:tasks.workunit.client.0.smithi144.stdout:9/871: dread d11/d12/d1b/d2c/d78/fc1 [0,4194304] 0 2024-09-10T11:29:49.870 INFO:tasks.workunit.client.0.smithi144.stdout:3/813: getdents d4/d28/d67/d80/d17/d18/d34 0 2024-09-10T11:29:49.870 INFO:tasks.workunit.client.0.smithi144.stdout:1/983: rmdir d5 39 2024-09-10T11:29:49.873 INFO:tasks.workunit.client.0.smithi144.stdout:0/722: unlink da/d3a/d8e/la7 0 2024-09-10T11:29:49.873 INFO:tasks.workunit.client.0.smithi144.stdout:6/748: write d9/d1f/fb6 [541682,70697] 0 2024-09-10T11:29:49.876 INFO:tasks.workunit.client.0.smithi144.stdout:6/749: read - d9/dd/d11/d1b/d2f/d7f/dc4/d5b/f108 zero size 2024-09-10T11:29:49.885 INFO:tasks.workunit.client.0.smithi144.stdout:2/679: link d7/de/d36/d45/d8e/d9a/d54/d61/ca9 d7/da/d5b/cde 0 2024-09-10T11:29:49.885 INFO:tasks.workunit.client.0.smithi144.stdout:3/814: link d4/d28/f3f d4/d28/d67/d80/d17/d18/f104 0 2024-09-10T11:29:49.886 INFO:tasks.workunit.client.0.smithi144.stdout:0/723: dread da/f2e [0,4194304] 0 2024-09-10T11:29:49.886 INFO:tasks.workunit.client.0.smithi144.stdout:2/680: creat d7/de/d36/d45/d8e/fdf x:0 0 0 2024-09-10T11:29:49.886 INFO:tasks.workunit.client.0.smithi144.stdout:9/872: write d11/d36/db1/dbb/df0/d8f/f82 [142026,7492] 0 2024-09-10T11:29:49.886 INFO:tasks.workunit.client.0.smithi144.stdout:4/930: write d14/d67/d69/d90/dda/de7/f10d [796593,58390] 0 2024-09-10T11:29:49.892 INFO:tasks.workunit.client.0.smithi144.stdout:6/750: write d9/dd/d11/d1b/d2f/dab/fce [267229,17884] 0 2024-09-10T11:29:49.894 INFO:tasks.workunit.client.0.smithi144.stdout:0/724: dread - da/d3a/d4b/d4e/f88 zero size 2024-09-10T11:29:49.897 INFO:tasks.workunit.client.0.smithi144.stdout:2/681: rename d7/da/da8 to d7/d42/d64/de0 0 2024-09-10T11:29:49.897 INFO:tasks.workunit.client.0.smithi144.stdout:1/984: dwrite d5/df/f93 [0,4194304] 0 2024-09-10T11:29:49.897 INFO:tasks.workunit.client.0.smithi144.stdout:3/815: write d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/fd9 [398249,74836] 0 2024-09-10T11:29:49.898 INFO:tasks.workunit.client.0.smithi144.stdout:9/873: dread d11/d36/db1/dbb/df0/d8f/fbd [0,4194304] 0 2024-09-10T11:29:49.898 INFO:tasks.workunit.client.0.smithi144.stdout:0/725: fdatasync da/d3a/d4b/fb7 0 2024-09-10T11:29:49.901 INFO:tasks.workunit.client.0.smithi144.stdout:2/682: write d7/da/d5b/fcf [4938241,69759] 0 2024-09-10T11:29:49.903 INFO:tasks.workunit.client.0.smithi144.stdout:9/874: chown d11/d12/d6f/lb3 22 1 2024-09-10T11:29:49.903 INFO:tasks.workunit.client.0.smithi144.stdout:4/931: dread d14/d1e/d83/da4/da9/fc7 [0,4194304] 0 2024-09-10T11:29:49.905 INFO:tasks.workunit.client.0.smithi144.stdout:3/816: dread - d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/f102 zero size 2024-09-10T11:29:49.908 INFO:tasks.workunit.client.0.smithi144.stdout:3/817: read d4/dd/fef [2594468,71575] 0 2024-09-10T11:29:49.908 INFO:tasks.workunit.client.0.smithi144.stdout:1/985: symlink d5/df/d4b/d51/d10a/dbd/dbe/l139 0 2024-09-10T11:29:49.909 INFO:tasks.workunit.client.0.smithi144.stdout:9/875: mkdir d11/d12/d1b/df8/d131 0 2024-09-10T11:29:49.910 INFO:tasks.workunit.client.0.smithi144.stdout:6/751: truncate d9/dd/d11/f4c 556178 0 2024-09-10T11:29:49.910 INFO:tasks.workunit.client.0.smithi144.stdout:4/932: mkdir d14/d1e/d150 0 2024-09-10T11:29:49.910 INFO:tasks.workunit.client.0.smithi144.stdout:9/876: readlink d11/d12/d1b/d2c/d78/lfc 0 2024-09-10T11:29:49.911 INFO:tasks.workunit.client.0.smithi144.stdout:6/752: chown d9/dd/ca7 6766184 1 2024-09-10T11:29:49.914 INFO:tasks.workunit.client.0.smithi144.stdout:9/877: write d11/d36/d2d/d38/dc3/f127 [3216919,93161] 0 2024-09-10T11:29:49.915 INFO:tasks.workunit.client.0.smithi144.stdout:6/753: unlink d9/dd/d11/d1b/d2f/lc3 0 2024-09-10T11:29:49.917 INFO:tasks.workunit.client.0.smithi144.stdout:2/683: dread d7/d29/f46 [0,4194304] 0 2024-09-10T11:29:49.920 INFO:tasks.workunit.client.0.smithi144.stdout:0/726: truncate da/d3a/d4b/d4e/d81/fbd 674796 0 2024-09-10T11:29:49.923 INFO:tasks.workunit.client.0.smithi144.stdout:3/818: link d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/f102 d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/f105 0 2024-09-10T11:29:49.926 INFO:tasks.workunit.client.0.smithi144.stdout:0/727: rename da/d3a/d4b/d61/d6c/f9b to da/d3a/d4b/d61/d6c/fdb 0 2024-09-10T11:29:49.927 INFO:tasks.workunit.client.0.smithi144.stdout:9/878: dread d11/d12/d1b/f61 [0,4194304] 0 2024-09-10T11:29:49.929 INFO:tasks.workunit.client.0.smithi144.stdout:3/819: symlink d4/d1a/df8/l106 0 2024-09-10T11:29:49.931 INFO:tasks.workunit.client.0.smithi144.stdout:9/879: creat d11/d12/d6f/daa/f132 x:0 0 0 2024-09-10T11:29:49.933 INFO:tasks.workunit.client.0.smithi144.stdout:0/728: creat da/d3a/d4b/d4e/d62/fdc x:0 0 0 2024-09-10T11:29:49.935 INFO:tasks.workunit.client.0.smithi144.stdout:9/880: chown d11/d12/d1b/d2c/d78/lac 19 1 2024-09-10T11:29:49.936 INFO:tasks.workunit.client.0.smithi144.stdout:6/754: dwrite d9/dd/d11/d1b/d2f/dab/dc6/fda [0,4194304] 0 2024-09-10T11:29:49.942 INFO:tasks.workunit.client.0.smithi144.stdout:3/820: write d4/d28/d67/d80/d17/d18/d34/d4b/f53 [5193193,107876] 0 2024-09-10T11:29:49.942 INFO:tasks.workunit.client.0.smithi144.stdout:6/755: mknod d9/dd/d3e/d47/c10c 0 2024-09-10T11:29:49.946 INFO:tasks.workunit.client.0.smithi144.stdout:0/729: truncate da/d3a/d94/db1/dcc/f9c 1497469 0 2024-09-10T11:29:49.947 INFO:tasks.workunit.client.0.smithi144.stdout:3/821: creat d4/d28/d67/d96/dcc/f107 x:0 0 0 2024-09-10T11:29:49.949 INFO:tasks.workunit.client.0.smithi144.stdout:6/756: getdents d9/d38 0 2024-09-10T11:29:49.950 INFO:tasks.workunit.client.0.smithi144.stdout:9/881: dwrite d11/d12/d1b/df8/fb6 [4194304,4194304] 0 2024-09-10T11:29:49.953 INFO:tasks.workunit.client.0.smithi144.stdout:0/730: rename da/f32 to da/d3a/d4b/d4e/fdd 0 2024-09-10T11:29:49.955 INFO:tasks.workunit.client.0.smithi144.stdout:0/731: symlink da/d3a/d8e/dab/lde 0 2024-09-10T11:29:49.955 INFO:tasks.workunit.client.0.smithi144.stdout:3/822: creat d4/d28/d67/d80/d17/d18/d34/d65/d75/f108 x:0 0 0 2024-09-10T11:29:49.958 INFO:tasks.workunit.client.0.smithi144.stdout:3/823: fdatasync d4/f9b 0 2024-09-10T11:29:49.960 INFO:tasks.workunit.client.0.smithi144.stdout:0/732: mkdir da/d3a/d4b/d61/ddf 0 2024-09-10T11:29:49.961 INFO:tasks.workunit.client.0.smithi144.stdout:6/757: link d9/dd/d11/d1b/d2f/dab/dc6/le9 d9/l10d 0 2024-09-10T11:29:49.963 INFO:tasks.workunit.client.0.smithi144.stdout:9/882: dread d11/d36/db1/dbb/df0/d8f/f82 [0,4194304] 0 2024-09-10T11:29:49.963 INFO:tasks.workunit.client.0.smithi144.stdout:6/758: write d9/d1f/d2c/fe2 [302394,109587] 0 2024-09-10T11:29:49.964 INFO:tasks.workunit.client.0.smithi144.stdout:9/883: chown d11/d36/fa4 158624 1 2024-09-10T11:29:49.965 INFO:tasks.workunit.client.0.smithi144.stdout:3/824: write d4/d28/f87 [2469391,49549] 0 2024-09-10T11:29:49.966 INFO:tasks.workunit.client.0.smithi144.stdout:0/733: creat da/d3a/d4b/d61/d6c/fe0 x:0 0 0 2024-09-10T11:29:49.971 INFO:tasks.workunit.client.0.smithi144.stdout:6/759: write d9/dd/d11/d1b/d2f/fb8 [769621,70087] 0 2024-09-10T11:29:49.976 INFO:tasks.workunit.client.0.smithi144.stdout:9/884: link d11/d36/db1/dbb/df0/d8f/dab/d64/c88 d11/d12/d1b/c133 0 2024-09-10T11:29:49.979 INFO:tasks.workunit.client.0.smithi144.stdout:0/734: dwrite da/f28 [0,4194304] 0 2024-09-10T11:29:49.980 INFO:tasks.workunit.client.0.smithi144.stdout:6/760: write d9/dd/d11/d1b/d2f/d7f/dc4/d5b/f61 [1718633,34574] 0 2024-09-10T11:29:49.981 INFO:tasks.workunit.client.0.smithi144.stdout:9/885: write d11/d36/d2d/d38/d116/f128 [17690,122045] 0 2024-09-10T11:29:49.983 INFO:tasks.workunit.client.0.smithi144.stdout:0/735: write da/d3a/d4b/d4e/d90/fa6 [16181,15527] 0 2024-09-10T11:29:49.986 INFO:tasks.workunit.client.0.smithi144.stdout:6/761: mkdir d9/d10e 0 2024-09-10T11:29:49.988 INFO:tasks.workunit.client.0.smithi144.stdout:9/886: creat d11/d36/db1/dbb/df0/d8f/dab/d64/f134 x:0 0 0 2024-09-10T11:29:49.989 INFO:tasks.workunit.client.0.smithi144.stdout:6/762: truncate d9/dd/d3e/f40 1535885 0 2024-09-10T11:29:49.989 INFO:tasks.workunit.client.0.smithi144.stdout:0/736: getdents da/d3a/d4b/dd4 0 2024-09-10T11:29:49.990 INFO:tasks.workunit.client.0.smithi144.stdout:3/825: dread d4/d28/d67/d80/fa2 [0,4194304] 0 2024-09-10T11:29:49.991 INFO:tasks.workunit.client.0.smithi144.stdout:9/887: creat d11/d36/d2d/d38/dc3/f135 x:0 0 0 2024-09-10T11:29:49.994 INFO:tasks.workunit.client.0.smithi144.stdout:0/737: stat da/d44/d82/l7f 0 2024-09-10T11:29:49.995 INFO:tasks.workunit.client.0.smithi144.stdout:3/826: truncate d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/f105 130946 0 2024-09-10T11:29:49.997 INFO:tasks.workunit.client.0.smithi144.stdout:6/763: truncate d9/f81 1902072 0 2024-09-10T11:29:49.999 INFO:tasks.workunit.client.0.smithi144.stdout:6/764: creat d9/dd/d73/f10f x:0 0 0 2024-09-10T11:29:50.000 INFO:tasks.workunit.client.0.smithi144.stdout:3/827: creat d4/d28/d67/d80/d17/f109 x:0 0 0 2024-09-10T11:29:50.001 INFO:tasks.workunit.client.0.smithi144.stdout:9/888: dwrite d11/d12/d1b/f95 [0,4194304] 0 2024-09-10T11:29:50.002 INFO:tasks.workunit.client.0.smithi144.stdout:6/765: creat d9/dd/d11/d1b/d2f/d7f/f110 x:0 0 0 2024-09-10T11:29:50.003 INFO:tasks.workunit.client.0.smithi144.stdout:9/889: chown d11/d36/db1/dbb/df0/d8f/f82 6123 1 2024-09-10T11:29:50.006 INFO:tasks.workunit.client.0.smithi144.stdout:3/828: getdents d4/d28/d67/d80/d17/d18/d34/d4b 0 2024-09-10T11:29:50.007 INFO:tasks.workunit.client.0.smithi144.stdout:3/829: chown d4/d1a/f44 1 1 2024-09-10T11:29:50.008 INFO:tasks.workunit.client.0.smithi144.stdout:6/766: mknod d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d87/dc9/ddd/c111 0 2024-09-10T11:29:50.013 INFO:tasks.workunit.client.0.smithi144.stdout:9/890: dwrite d11/d36/d2d/d38/d116/f128 [0,4194304] 0 2024-09-10T11:29:50.018 INFO:tasks.workunit.client.0.smithi144.stdout:9/891: creat d11/d36/d2d/df6/f136 x:0 0 0 2024-09-10T11:29:50.018 INFO:tasks.workunit.client.0.smithi144.stdout:3/830: link d4/d28/d67/d80/d17/d18/d34/d65/d75/f84 d4/d28/d67/d80/f10a 0 2024-09-10T11:29:50.019 INFO:tasks.workunit.client.0.smithi144.stdout:6/767: dwrite d9/dd/d73/f10f [0,4194304] 0 2024-09-10T11:29:50.022 INFO:tasks.workunit.client.0.smithi144.stdout:6/768: stat d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d60/fdf 0 2024-09-10T11:29:50.024 INFO:tasks.workunit.client.0.smithi144.stdout:9/892: rename d11/d36/d2d/d38/d116/f128 to d11/d36/db1/dbb/df0/d8f/dab/d64/f137 0 2024-09-10T11:29:50.027 INFO:tasks.workunit.client.0.smithi144.stdout:9/893: truncate f10 6963267 0 2024-09-10T11:29:50.031 INFO:tasks.workunit.client.0.smithi144.stdout:9/894: unlink d11/d12/d1b/d2c/d78/f7d 0 2024-09-10T11:29:50.032 INFO:tasks.workunit.client.0.smithi144.stdout:3/831: mknod d4/d28/d67/d80/d17/dd0/d98/c10b 0 2024-09-10T11:29:50.035 INFO:tasks.workunit.client.0.smithi144.stdout:3/832: creat d4/d28/d67/d80/d17/dd0/d98/f10c x:0 0 0 2024-09-10T11:29:50.038 INFO:tasks.workunit.client.0.smithi144.stdout:3/833: mknod d4/d28/dc1/dc4/c10d 0 2024-09-10T11:29:50.039 INFO:tasks.workunit.client.0.smithi144.stdout:9/895: dwrite d11/d36/f4e [0,4194304] 0 2024-09-10T11:29:50.041 INFO:tasks.workunit.client.0.smithi144.stdout:3/834: symlink d4/d28/d67/d96/dcc/dec/l10e 0 2024-09-10T11:29:50.043 INFO:tasks.workunit.client.0.smithi144.stdout:6/769: dread d9/dd/d11/d1b/d2f/d7f/f92 [0,4194304] 0 2024-09-10T11:29:50.045 INFO:tasks.workunit.client.0.smithi144.stdout:9/896: rename d11/d36/db1/f10b to d11/d12/d1b/d2c/d8a/f138 0 2024-09-10T11:29:50.047 INFO:tasks.workunit.client.0.smithi144.stdout:6/770: mknod d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d60/c112 0 2024-09-10T11:29:50.050 INFO:tasks.workunit.client.0.smithi144.stdout:9/897: rename d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/l33 to d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/d12e/l139 0 2024-09-10T11:29:50.052 INFO:tasks.workunit.client.0.smithi144.stdout:6/771: symlink d9/dd/d11/d1b/d2f/dab/l113 0 2024-09-10T11:29:50.055 INFO:tasks.workunit.client.0.smithi144.stdout:3/835: write f3 [583996,98145] 0 2024-09-10T11:29:50.060 INFO:tasks.workunit.client.0.smithi144.stdout:6/772: dwrite d9/d1f/f37 [0,4194304] 0 2024-09-10T11:29:50.060 INFO:tasks.workunit.client.0.smithi144.stdout:9/898: rmdir d11/d36/db1/dbb/df0/d8f 39 2024-09-10T11:29:50.064 INFO:tasks.workunit.client.0.smithi144.stdout:6/773: creat d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d87/f114 x:0 0 0 2024-09-10T11:29:50.064 INFO:tasks.workunit.client.0.smithi144.stdout:9/899: truncate d11/f1c 5827616 0 2024-09-10T11:29:50.068 INFO:tasks.workunit.client.0.smithi144.stdout:6/774: unlink d9/d1f/d2c/l44 0 2024-09-10T11:29:50.070 INFO:tasks.workunit.client.0.smithi144.stdout:6/775: chown d9/dd/f101 147 1 2024-09-10T11:29:50.073 INFO:tasks.workunit.client.0.smithi144.stdout:9/900: dwrite d11/d36/d2d/d38/dc3/f135 [0,4194304] 0 2024-09-10T11:29:50.076 INFO:tasks.workunit.client.0.smithi144.stdout:6/776: creat d9/dd/d11/d1b/d2f/dab/de7/f115 x:0 0 0 2024-09-10T11:29:50.078 INFO:tasks.workunit.client.0.smithi144.stdout:3/836: dread d4/dd/d21/f5b [4194304,4194304] 0 2024-09-10T11:29:50.079 INFO:tasks.workunit.client.0.smithi144.stdout:6/777: unlink d9/d1f/d2c/fe2 0 2024-09-10T11:29:50.081 INFO:tasks.workunit.client.0.smithi144.stdout:3/837: chown d4/d28/d67/d80/d17/d18/d34/d65/d75/fb0 5469788 1 2024-09-10T11:29:50.089 INFO:tasks.workunit.client.0.smithi144.stdout:9/901: dread d11/d36/f9c [0,4194304] 0 2024-09-10T11:29:50.089 INFO:tasks.workunit.client.0.smithi144.stdout:3/838: read - d4/d28/d67/d80/d17/dd0/fbf zero size 2024-09-10T11:29:50.089 INFO:tasks.workunit.client.0.smithi144.stdout:3/839: chown d4/dd/dfd 10 1 2024-09-10T11:29:50.089 INFO:tasks.workunit.client.0.smithi144.stdout:9/902: creat d11/d36/d2d/d38/de7/d12d/f13a x:0 0 0 2024-09-10T11:29:50.090 INFO:tasks.workunit.client.0.smithi144.stdout:3/840: symlink d4/d28/d67/d80/d17/d18/d34/d4b/l10f 0 2024-09-10T11:29:50.094 INFO:tasks.workunit.client.0.smithi144.stdout:6/778: write d9/dd/d3e/d47/f88 [771513,59675] 0 2024-09-10T11:29:50.095 INFO:tasks.workunit.client.0.smithi144.stdout:3/841: write d4/d28/d67/d80/d17/d18/f43 [799456,64704] 0 2024-09-10T11:29:50.096 INFO:tasks.workunit.client.0.smithi144.stdout:6/779: rmdir d9/d1f/dae 39 2024-09-10T11:29:50.098 INFO:tasks.workunit.client.0.smithi144.stdout:3/842: symlink d4/dd/d21/d2f/l110 0 2024-09-10T11:29:50.099 INFO:tasks.workunit.client.0.smithi144.stdout:6/780: fsync d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/f20 0 2024-09-10T11:29:50.100 INFO:tasks.workunit.client.0.smithi144.stdout:6/781: readlink d9/dd/d11/la2 0 2024-09-10T11:29:50.105 INFO:tasks.workunit.client.0.smithi144.stdout:6/782: symlink d9/dd/d11/l116 0 2024-09-10T11:29:50.106 INFO:tasks.workunit.client.0.smithi144.stdout:3/843: getdents d4/d28/d67/d80/d17/dd0/d98 0 2024-09-10T11:29:50.108 INFO:tasks.workunit.client.0.smithi144.stdout:3/844: dread - d4/d28/d67/d80/d17/dd0/fa6 zero size 2024-09-10T11:29:50.108 INFO:tasks.workunit.client.0.smithi144.stdout:3/845: chown d4 1638 1 2024-09-10T11:29:50.113 INFO:tasks.workunit.client.0.smithi144.stdout:3/846: dwrite d4/dd/d21/f5d [4194304,4194304] 0 2024-09-10T11:29:50.121 INFO:tasks.workunit.client.0.smithi144.stdout:3/847: write d4/dd/fa9 [1048057,89382] 0 2024-09-10T11:29:50.123 INFO:tasks.workunit.client.0.smithi144.stdout:3/848: dread - d4/d28/d67/d80/d17/dd0/fbf zero size 2024-09-10T11:29:50.128 INFO:tasks.workunit.client.0.smithi144.stdout:3/849: write d4/d28/d67/d80/d17/dd0/fbf [767939,102143] 0 2024-09-10T11:29:50.130 INFO:tasks.workunit.client.0.smithi144.stdout:3/850: creat d4/d28/dc1/dc4/f111 x:0 0 0 2024-09-10T11:29:50.136 INFO:tasks.workunit.client.0.smithi144.stdout:3/851: dwrite d4/d28/d67/d80/d17/d18/f6d [4194304,4194304] 0 2024-09-10T11:29:50.138 INFO:tasks.workunit.client.0.smithi144.stdout:3/852: symlink d4/d28/l112 0 2024-09-10T11:29:50.168 INFO:tasks.workunit.client.0.smithi144.stdout:8/724: sync 2024-09-10T11:29:50.177 INFO:tasks.workunit.client.0.smithi144.stdout:8/725: dwrite d0/d9/d23/d37/f3a [0,4194304] 0 2024-09-10T11:29:50.178 INFO:tasks.workunit.client.0.smithi144.stdout:8/726: dread - d0/d9/d1f/fee zero size 2024-09-10T11:29:50.181 INFO:tasks.workunit.client.0.smithi144.stdout:8/727: rmdir d0/d1b/d47/d5e/d76 39 2024-09-10T11:29:50.182 INFO:tasks.workunit.client.0.smithi144.stdout:8/728: write d0/d1b/f2f [1872118,26080] 0 2024-09-10T11:29:50.188 INFO:tasks.workunit.client.0.smithi144.stdout:8/729: creat d0/d1b/d47/d5e/ffc x:0 0 0 2024-09-10T11:29:50.190 INFO:tasks.workunit.client.0.smithi144.stdout:8/730: symlink d0/d9/d21/d25/db3/dc2/dcd/lfd 0 2024-09-10T11:29:50.193 INFO:tasks.workunit.client.0.smithi144.stdout:8/731: rmdir d0/d9/d23/d26/d7d 39 2024-09-10T11:29:50.199 INFO:tasks.workunit.client.0.smithi144.stdout:8/732: dwrite d0/d9/d1f/fee [0,4194304] 0 2024-09-10T11:29:50.200 INFO:tasks.workunit.client.0.smithi144.stdout:8/733: write d0/d9/d1f/fc6 [695545,4959] 0 2024-09-10T11:29:50.202 INFO:tasks.workunit.client.0.smithi144.stdout:8/734: truncate d0/d9/d23/d26/fea 970345 0 2024-09-10T11:29:50.207 INFO:tasks.workunit.client.0.smithi144.stdout:8/735: symlink d0/d9/d21/d25/db3/dc2/df2/lfe 0 2024-09-10T11:29:50.212 INFO:tasks.workunit.client.0.smithi144.stdout:8/736: link d0/d9/d21/d25/db3/dc2/dcd/lec d0/d9/d1f/d63/lff 0 2024-09-10T11:29:50.213 INFO:tasks.workunit.client.0.smithi144.stdout:8/737: chown d0/d1b 948536013 1 2024-09-10T11:29:50.215 INFO:tasks.workunit.client.0.smithi144.stdout:8/738: read d0/f24 [1665072,6508] 0 2024-09-10T11:29:50.216 INFO:tasks.workunit.client.0.smithi144.stdout:8/739: fdatasync d0/d9/f1d 0 2024-09-10T11:29:50.217 INFO:tasks.workunit.client.0.smithi144.stdout:8/740: dread - d0/d1b/d47/d94/fcc zero size 2024-09-10T11:29:50.218 INFO:tasks.workunit.client.0.smithi144.stdout:8/741: chown d0/d1b/d47/dfa 311 1 2024-09-10T11:29:50.224 INFO:tasks.workunit.client.0.smithi144.stdout:8/742: dwrite d0/d1b/d47/d5e/f64 [0,4194304] 0 2024-09-10T11:29:50.229 INFO:tasks.workunit.client.0.smithi144.stdout:8/743: link d0/d9/d21/f5c d0/d9/d21/d25/db3/f100 0 2024-09-10T11:29:50.230 INFO:tasks.workunit.client.0.smithi144.stdout:8/744: dread - d0/d9/d1f/f92 zero size 2024-09-10T11:29:50.232 INFO:tasks.workunit.client.0.smithi144.stdout:8/745: write d0/d9/d21/d25/ff8 [7798744,6157] 0 2024-09-10T11:29:50.234 INFO:tasks.workunit.client.0.smithi144.stdout:8/746: mkdir d0/d9/d1f/d34/d101 0 2024-09-10T11:29:50.239 INFO:tasks.workunit.client.0.smithi144.stdout:8/747: rename d0/f24 to d0/d9/d21/d25/d4f/d93/f102 0 2024-09-10T11:29:50.240 INFO:tasks.workunit.client.0.smithi144.stdout:8/748: readlink d0/d9/d21/d25/db3/dc2/dcd/l8e 0 2024-09-10T11:29:50.242 INFO:tasks.workunit.client.0.smithi144.stdout:8/749: creat d0/d9/d21/d25/db3/f103 x:0 0 0 2024-09-10T11:29:50.246 INFO:tasks.workunit.client.0.smithi144.stdout:8/750: rename d0/d1b/d47/d5e/df9 to d0/d9/d104 0 2024-09-10T11:29:50.247 INFO:tasks.workunit.client.0.smithi144.stdout:8/751: fsync d0/d1b/d47/f7b 0 2024-09-10T11:29:50.251 INFO:tasks.workunit.client.0.smithi144.stdout:8/752: link d0/d9/d21/d7e/d85/c8a d0/d9/d21/d25/d4f/d93/c105 0 2024-09-10T11:29:50.254 INFO:tasks.workunit.client.0.smithi144.stdout:8/753: rename d0/d9/d21/d25/d4f/dc3 to d0/d9/d21/d106 0 2024-09-10T11:29:50.261 INFO:tasks.workunit.client.0.smithi144.stdout:8/754: rmdir d0/d9/d21/d106 0 2024-09-10T11:29:50.262 INFO:tasks.workunit.client.0.smithi144.stdout:8/755: write d0/d9/d21/d7e/d85/fdc [4753829,37909] 0 2024-09-10T11:29:50.270 INFO:tasks.workunit.client.0.smithi144.stdout:8/756: dread d0/d9/d23/d26/d7d/f81 [0,4194304] 0 2024-09-10T11:29:50.279 INFO:tasks.workunit.client.0.smithi144.stdout:5/818: sync 2024-09-10T11:29:50.279 INFO:tasks.workunit.client.0.smithi144.stdout:7/850: sync 2024-09-10T11:29:50.283 INFO:tasks.workunit.client.0.smithi144.stdout:5/819: dread d4/d5/d32/d47/f71 [0,4194304] 0 2024-09-10T11:29:50.284 INFO:tasks.workunit.client.0.smithi144.stdout:7/851: write d5/dc/d2a/f4b [696621,91972] 0 2024-09-10T11:29:50.284 INFO:tasks.workunit.client.0.smithi144.stdout:8/757: dread d0/d9/d23/d26/d7d/f80 [0,4194304] 0 2024-09-10T11:29:50.285 INFO:tasks.workunit.client.0.smithi144.stdout:5/820: stat d4/d5/d32/d47/df3/d9a 0 2024-09-10T11:29:50.285 INFO:tasks.workunit.client.0.smithi144.stdout:8/758: stat d0/d1b/d5a/leb 0 2024-09-10T11:29:50.287 INFO:tasks.workunit.client.0.smithi144.stdout:5/821: creat d4/d20/d22/d75/ffd x:0 0 0 2024-09-10T11:29:50.287 INFO:tasks.workunit.client.0.smithi144.stdout:7/852: read d5/dc/d2a/d60/dd6/d98/fca [656372,116001] 0 2024-09-10T11:29:50.288 INFO:tasks.workunit.client.0.smithi144.stdout:8/759: creat d0/d9/d1f/d34/d68/f107 x:0 0 0 2024-09-10T11:29:50.289 INFO:tasks.workunit.client.0.smithi144.stdout:5/822: mknod d4/d20/d22/d5c/cfe 0 2024-09-10T11:29:50.290 INFO:tasks.workunit.client.0.smithi144.stdout:5/823: readlink d4/d5/d18/d38/l5b 0 2024-09-10T11:29:50.296 INFO:tasks.workunit.client.0.smithi144.stdout:7/853: truncate d5/dc/d2a/d60/dd6/f66 1973300 0 2024-09-10T11:29:50.299 INFO:tasks.workunit.client.0.smithi144.stdout:8/760: dwrite d0/d9/d23/d26/fea [0,4194304] 0 2024-09-10T11:29:50.324 INFO:tasks.workunit.client.0.smithi144.stdout:5/824: symlink d4/d5/d18/d38/d6d/da2/lff 0 2024-09-10T11:29:50.324 INFO:tasks.workunit.client.0.smithi144.stdout:5/825: readlink d4/ld 0 2024-09-10T11:29:50.327 INFO:tasks.workunit.client.0.smithi144.stdout:8/761: rename d0/f3f to d0/d9/d21/d25/db3/dc2/dcd/f108 0 2024-09-10T11:29:50.327 INFO:tasks.workunit.client.0.smithi144.stdout:8/762: chown d0/d9/d23/fe0 237 1 2024-09-10T11:29:50.333 INFO:tasks.workunit.client.0.smithi144.stdout:8/763: creat d0/d9/d21/d25/f109 x:0 0 0 2024-09-10T11:29:50.347 INFO:tasks.workunit.client.0.smithi144.stdout:7/854: write d5/dc/d14/d40/d5e/d68/f89 [4377407,38289] 0 2024-09-10T11:29:50.354 INFO:tasks.workunit.client.0.smithi144.stdout:5/826: dwrite d4/d5/d18/d38/d6d/f70 [0,4194304] 0 2024-09-10T11:29:50.356 INFO:tasks.workunit.client.0.smithi144.stdout:8/764: dwrite d0/d9/d23/d26/f4a [0,4194304] 0 2024-09-10T11:29:50.358 INFO:tasks.workunit.client.0.smithi144.stdout:5/827: getdents d4/d5/df2 0 2024-09-10T11:29:50.358 INFO:tasks.workunit.client.0.smithi144.stdout:7/855: dwrite d5/dc/d14/d40/f67 [0,4194304] 0 2024-09-10T11:29:50.362 INFO:tasks.workunit.client.0.smithi144.stdout:5/828: dread - d4/d20/d22/d5c/d5d/d64/fd4 zero size 2024-09-10T11:29:50.370 INFO:tasks.workunit.client.0.smithi144.stdout:7/856: dwrite d5/f105 [0,4194304] 0 2024-09-10T11:29:50.373 INFO:tasks.workunit.client.0.smithi144.stdout:5/829: creat d4/d20/d22/d5c/f100 x:0 0 0 2024-09-10T11:29:50.374 INFO:tasks.workunit.client.0.smithi144.stdout:7/857: mkdir d5/dc/d14/d40/d58/d6a/d119 0 2024-09-10T11:29:50.376 INFO:tasks.workunit.client.0.smithi144.stdout:5/830: creat d4/d20/d22/d5c/d5d/f101 x:0 0 0 2024-09-10T11:29:50.377 INFO:tasks.workunit.client.0.smithi144.stdout:7/858: mkdir d5/dc/d2a/d60/d11a 0 2024-09-10T11:29:50.378 INFO:tasks.workunit.client.0.smithi144.stdout:7/859: readlink d5/dc/d2a/d9d/la7 0 2024-09-10T11:29:50.380 INFO:tasks.workunit.client.0.smithi144.stdout:5/831: truncate d4/d5/d32/f33 9147093 0 2024-09-10T11:29:50.383 INFO:tasks.workunit.client.0.smithi144.stdout:5/832: creat d4/d52/f102 x:0 0 0 2024-09-10T11:29:50.383 INFO:tasks.workunit.client.0.smithi144.stdout:5/833: chown d4/d5/df2 19 1 2024-09-10T11:29:50.385 INFO:tasks.workunit.client.0.smithi144.stdout:7/860: mkdir d5/dc/d14/d40/d58/d6e/d7a/d11b 0 2024-09-10T11:29:50.385 INFO:tasks.workunit.client.0.smithi144.stdout:8/765: write d0/d1b/d47/d5e/d76/f7c [3220881,35830] 0 2024-09-10T11:29:50.390 INFO:tasks.workunit.client.0.smithi144.stdout:5/834: creat d4/d5/d18/d67/f103 x:0 0 0 2024-09-10T11:29:50.391 INFO:tasks.workunit.client.0.smithi144.stdout:8/766: unlink d0/d9/d1f/d34/da0/dc4/lca 0 2024-09-10T11:29:50.393 INFO:tasks.workunit.client.0.smithi144.stdout:8/767: symlink d0/d1b/d5a/l10a 0 2024-09-10T11:29:50.398 INFO:tasks.workunit.client.0.smithi144.stdout:5/835: getdents d4/d20/d22/d5c/d5d 0 2024-09-10T11:29:50.401 INFO:tasks.workunit.client.0.smithi144.stdout:5/836: rename d4/d5/d18/d38/l5b to d4/d5/d32/d47/df3/l104 0 2024-09-10T11:29:50.401 INFO:tasks.workunit.client.0.smithi144.stdout:7/861: dread d5/fce [0,4194304] 0 2024-09-10T11:29:50.401 INFO:tasks.workunit.client.0.smithi144.stdout:8/768: truncate d0/d9/d23/d37/f3a 334164 0 2024-09-10T11:29:50.405 INFO:tasks.workunit.client.0.smithi144.stdout:7/862: creat d5/dc/d12/f11c x:0 0 0 2024-09-10T11:29:50.408 INFO:tasks.workunit.client.0.smithi144.stdout:8/769: dread d0/d9/d23/f87 [0,4194304] 0 2024-09-10T11:29:50.411 INFO:tasks.workunit.client.0.smithi144.stdout:8/770: symlink d0/d9/d21/d25/db3/dc2/dcd/l10b 0 2024-09-10T11:29:50.412 INFO:tasks.workunit.client.0.smithi144.stdout:5/837: dwrite d4/d20/d22/d5c/d5d/d64/fd4 [0,4194304] 0 2024-09-10T11:29:50.415 INFO:tasks.workunit.client.0.smithi144.stdout:8/771: truncate d0/d9/d1f/d34/fb1 104624 0 2024-09-10T11:29:50.416 INFO:tasks.workunit.client.0.smithi144.stdout:5/838: mkdir d4/d5/d18/d38/d6d/d105 0 2024-09-10T11:29:50.426 INFO:tasks.workunit.client.0.smithi144.stdout:5/839: truncate d4/d20/d22/d5c/d5d/d64/fd4 2435495 0 2024-09-10T11:29:50.427 INFO:tasks.workunit.client.0.smithi144.stdout:8/772: dwrite d0/d9/d23/f40 [8388608,4194304] 0 2024-09-10T11:29:50.430 INFO:tasks.workunit.client.0.smithi144.stdout:8/773: symlink d0/d9/l10c 0 2024-09-10T11:29:50.431 INFO:tasks.workunit.client.0.smithi144.stdout:8/774: read d0/d9/d23/d26/d7d/f80 [2197766,83651] 0 2024-09-10T11:29:50.434 INFO:tasks.workunit.client.0.smithi144.stdout:5/840: dwrite d4/d5/d18/d38/d6d/d8b/f91 [0,4194304] 0 2024-09-10T11:29:50.437 INFO:tasks.workunit.client.0.smithi144.stdout:8/775: rename d0/d1b/d5a/d8d/ff0 to d0/d9/d1f/f10d 0 2024-09-10T11:29:50.438 INFO:tasks.workunit.client.0.smithi144.stdout:5/841: truncate d4/d20/d22/d75/ffd 957068 0 2024-09-10T11:29:50.442 INFO:tasks.workunit.client.0.smithi144.stdout:8/776: creat d0/d9/d23/d26/d7d/f10e x:0 0 0 2024-09-10T11:29:50.449 INFO:tasks.workunit.client.0.smithi144.stdout:8/777: dread d0/d1b/f8b [0,4194304] 0 2024-09-10T11:29:50.454 INFO:tasks.workunit.client.0.smithi144.stdout:5/842: dwrite d4/d20/f43 [8388608,4194304] 0 2024-09-10T11:29:50.455 INFO:tasks.workunit.client.0.smithi144.stdout:8/778: rename d0/d1b/d47/d5e/d76/fb2 to d0/d9/d21/f10f 0 2024-09-10T11:29:50.457 INFO:tasks.workunit.client.0.smithi144.stdout:5/843: mknod d4/d5/d18/d38/d6d/c106 0 2024-09-10T11:29:50.460 INFO:tasks.workunit.client.0.smithi144.stdout:5/844: symlink d4/d20/d24/d2d/l107 0 2024-09-10T11:29:50.464 INFO:tasks.workunit.client.0.smithi144.stdout:5/845: rename d4/d20/d24/ca5 to d4/d5/d18/d38/d8d/c108 0 2024-09-10T11:29:50.466 INFO:tasks.workunit.client.0.smithi144.stdout:5/846: symlink d4/d5/d32/d47/df3/d74/dbe/l109 0 2024-09-10T11:29:50.466 INFO:tasks.workunit.client.0.smithi144.stdout:8/779: dread d0/d1b/d47/f7b [0,4194304] 0 2024-09-10T11:29:50.467 INFO:tasks.workunit.client.0.smithi144.stdout:5/847: write d4/d52/f102 [368996,101208] 0 2024-09-10T11:29:50.468 INFO:tasks.workunit.client.0.smithi144.stdout:8/780: creat d0/d1b/d5a/f110 x:0 0 0 2024-09-10T11:29:50.469 INFO:tasks.workunit.client.0.smithi144.stdout:5/848: stat d4/d20/d22 0 2024-09-10T11:29:50.469 INFO:tasks.workunit.client.0.smithi144.stdout:8/781: fsync d0/d1b/d5a/d8d/f98 0 2024-09-10T11:29:50.471 INFO:tasks.workunit.client.0.smithi144.stdout:8/782: fsync d0/d9/d1f/d34/d68/f107 0 2024-09-10T11:29:50.473 INFO:tasks.workunit.client.0.smithi144.stdout:8/783: mkdir d0/d9/d23/d111 0 2024-09-10T11:29:50.475 INFO:tasks.workunit.client.0.smithi144.stdout:8/784: mkdir d0/d9/d1f/d34/da0/dc4/d112 0 2024-09-10T11:29:50.476 INFO:tasks.workunit.client.0.smithi144.stdout:8/785: dread - d0/d9/d1f/f8f zero size 2024-09-10T11:29:50.481 INFO:tasks.workunit.client.0.smithi144.stdout:8/786: write d0/d9/d1f/f19 [2656749,99092] 0 2024-09-10T11:29:50.483 INFO:tasks.workunit.client.0.smithi144.stdout:5/849: dread d4/d5/d18/d38/d6d/fcb [0,4194304] 0 2024-09-10T11:29:50.483 INFO:tasks.workunit.client.0.smithi144.stdout:8/787: creat d0/d9/d21/d25/db3/de8/f113 x:0 0 0 2024-09-10T11:29:50.487 INFO:tasks.workunit.client.0.smithi144.stdout:8/788: link d0/d9/d21/f71 d0/d1b/d47/d5e/d76/f114 0 2024-09-10T11:29:50.490 INFO:tasks.workunit.client.0.smithi144.stdout:5/850: dwrite d4/d20/d22/d5c/f100 [0,4194304] 0 2024-09-10T11:29:50.492 INFO:tasks.workunit.client.0.smithi144.stdout:8/789: unlink d0/d9/d21/d25/db3/dc2/dcd/lf7 0 2024-09-10T11:29:50.498 INFO:tasks.workunit.client.0.smithi144.stdout:8/790: rmdir d0/d9/d21/d7e/d85 39 2024-09-10T11:29:50.498 INFO:tasks.workunit.client.0.smithi144.stdout:5/851: truncate d4/f6b 1140551 0 2024-09-10T11:29:50.498 INFO:tasks.workunit.client.0.smithi144.stdout:8/791: mkdir d0/d1b/d5a/d8d/d115 0 2024-09-10T11:29:50.499 INFO:tasks.workunit.client.0.smithi144.stdout:5/852: symlink d4/d5/d32/daa/l10a 0 2024-09-10T11:29:50.500 INFO:tasks.workunit.client.0.smithi144.stdout:8/792: symlink d0/d9/d23/d111/l116 0 2024-09-10T11:29:50.501 INFO:tasks.workunit.client.0.smithi144.stdout:5/853: mknod d4/d5/df2/c10b 0 2024-09-10T11:29:50.506 INFO:tasks.workunit.client.0.smithi144.stdout:8/793: write d0/f10 [3267971,30171] 0 2024-09-10T11:29:50.508 INFO:tasks.workunit.client.0.smithi144.stdout:5/854: dwrite d4/d5/d32/f5f [4194304,4194304] 0 2024-09-10T11:29:50.510 INFO:tasks.workunit.client.0.smithi144.stdout:5/855: chown d4/d5/d18/d38/d6d/d8b/f91 10 1 2024-09-10T11:29:50.510 INFO:tasks.workunit.client.0.smithi144.stdout:8/794: fsync d0/d9/d1f/f1c 0 2024-09-10T11:29:50.512 INFO:tasks.workunit.client.0.smithi144.stdout:8/795: stat d0/d9/d21/la7 0 2024-09-10T11:29:50.515 INFO:tasks.workunit.client.0.smithi144.stdout:5/856: truncate d4/d5/d32/d47/df3/d84/fab 523735 0 2024-09-10T11:29:50.517 INFO:tasks.workunit.client.0.smithi144.stdout:5/857: symlink d4/d20/d22/d5c/d5d/l10c 0 2024-09-10T11:29:50.520 INFO:tasks.workunit.client.0.smithi144.stdout:5/858: chown d4/d5/d32/d47/df3/d84/fab 61422337 1 2024-09-10T11:29:50.520 INFO:tasks.workunit.client.0.smithi144.stdout:5/859: chown d4/d20/d22/d75/ddd 1 1 2024-09-10T11:29:50.522 INFO:tasks.workunit.client.0.smithi144.stdout:8/796: write d0/d9/d1f/f91 [835551,51915] 0 2024-09-10T11:29:50.522 INFO:tasks.workunit.client.0.smithi144.stdout:5/860: dread - d4/d5/d32/d47/df3/d74/dbe/fd7 zero size 2024-09-10T11:29:50.526 INFO:tasks.workunit.client.0.smithi144.stdout:8/797: link d0/d1b/f4c d0/d9/f117 0 2024-09-10T11:29:50.528 INFO:tasks.workunit.client.0.smithi144.stdout:5/861: dwrite d4/d5/d18/d38/d6d/ff1 [0,4194304] 0 2024-09-10T11:29:50.531 INFO:tasks.workunit.client.0.smithi144.stdout:5/862: creat d4/d5/d18/d38/d6d/f10d x:0 0 0 2024-09-10T11:29:50.538 INFO:tasks.workunit.client.0.smithi144.stdout:8/798: dwrite d0/d9/d23/d26/f36 [0,4194304] 0 2024-09-10T11:29:50.543 INFO:tasks.workunit.client.0.smithi144.stdout:5/863: truncate d4/d5/d18/d38/d6d/f70 4858605 0 2024-09-10T11:29:50.543 INFO:tasks.workunit.client.0.smithi144.stdout:5/864: chown d4/f55 3039889 1 2024-09-10T11:29:50.546 INFO:tasks.workunit.client.0.smithi144.stdout:8/799: write d0/d9/f117 [891835,12035] 0 2024-09-10T11:29:50.550 INFO:tasks.workunit.client.0.smithi144.stdout:8/800: mknod d0/d1b/d5a/c118 0 2024-09-10T11:29:50.554 INFO:tasks.workunit.client.0.smithi144.stdout:8/801: unlink d0/d9/d1f/d34/da0/dc4/fd3 0 2024-09-10T11:29:50.562 INFO:tasks.workunit.client.0.smithi144.stdout:8/802: dwrite d0/d9/d21/d25/d4f/d93/f102 [0,4194304] 0 2024-09-10T11:29:50.564 INFO:tasks.workunit.client.0.smithi144.stdout:5/865: dread d4/d20/d22/d5c/d5d/de6/f85 [0,4194304] 0 2024-09-10T11:29:50.580 INFO:tasks.workunit.client.0.smithi144.stdout:8/803: dwrite d0/d9/d23/d26/f69 [0,4194304] 0 2024-09-10T11:29:50.581 INFO:tasks.workunit.client.0.smithi144.stdout:5/866: dwrite d4/d20/d22/d5c/d5d/de6/ff4 [0,4194304] 0 2024-09-10T11:29:50.584 INFO:tasks.workunit.client.0.smithi144.stdout:5/867: chown d4/d5/d56/d8a/lea 1687756 1 2024-09-10T11:29:50.586 INFO:tasks.workunit.client.0.smithi144.stdout:8/804: mknod d0/d9/d23/d26/d7d/dda/c119 0 2024-09-10T11:29:50.593 INFO:tasks.workunit.client.0.smithi144.stdout:8/805: dwrite d0/d9/f117 [0,4194304] 0 2024-09-10T11:29:50.593 INFO:tasks.workunit.client.0.smithi144.stdout:5/868: write d4/d20/d22/d5c/d5d/de6/f85 [1530273,116547] 0 2024-09-10T11:29:50.598 INFO:tasks.workunit.client.0.smithi144.stdout:5/869: creat d4/d20/d24/d2d/d4b/f10e x:0 0 0 2024-09-10T11:29:50.601 INFO:tasks.workunit.client.0.smithi144.stdout:5/870: mknod d4/d20/d24/dce/c10f 0 2024-09-10T11:29:50.602 INFO:tasks.workunit.client.0.smithi144.stdout:8/806: dread d0/d9/d21/f71 [0,4194304] 0 2024-09-10T11:29:50.603 INFO:tasks.workunit.client.0.smithi144.stdout:5/871: dread - d4/d5/d18/d38/d6d/d8b/fe1 zero size 2024-09-10T11:29:50.607 INFO:tasks.workunit.client.0.smithi144.stdout:8/807: rename d0/d9/d21/d25/db3/dc2/dcd/dd8 to d0/d1b/d5a/d8d/d115/d11a 0 2024-09-10T11:29:50.609 INFO:tasks.workunit.client.0.smithi144.stdout:5/872: truncate d4/d5/d18/f1f 1185742 0 2024-09-10T11:29:50.610 INFO:tasks.workunit.client.0.smithi144.stdout:5/873: chown d4/d5/d18/ca3 2 1 2024-09-10T11:29:50.611 INFO:tasks.workunit.client.0.smithi144.stdout:5/874: fsync d4/d20/d22/d75/fe0 0 2024-09-10T11:29:50.614 INFO:tasks.workunit.client.0.smithi144.stdout:5/875: chown d4/d5/d32/d47/df3/d84/fab 127511837 1 2024-09-10T11:29:50.616 INFO:tasks.workunit.client.0.smithi144.stdout:5/876: rmdir d4/d20/d24/d2d/d4b 39 2024-09-10T11:29:50.621 INFO:tasks.workunit.client.0.smithi144.stdout:8/808: dwrite d0/d9/d1f/f8f [0,4194304] 0 2024-09-10T11:29:50.623 INFO:tasks.workunit.client.0.smithi144.stdout:8/809: dread - d0/d9/d21/d25/db3/dc2/fce zero size 2024-09-10T11:29:50.634 INFO:tasks.workunit.client.0.smithi144.stdout:1/986: sync 2024-09-10T11:29:50.637 INFO:tasks.workunit.client.0.smithi144.stdout:1/987: symlink d5/d2a/d65/dc9/l13a 0 2024-09-10T11:29:50.644 INFO:tasks.workunit.client.0.smithi144.stdout:8/810: dread d0/d9/d23/f40 [4194304,4194304] 0 2024-09-10T11:29:50.645 INFO:tasks.workunit.client.0.smithi144.stdout:1/988: dwrite d5/dd/d64/d79/fb3 [0,4194304] 0 2024-09-10T11:29:50.646 INFO:tasks.workunit.client.0.smithi144.stdout:8/811: stat d0/d9/d23/d26/d7d/dda 0 2024-09-10T11:29:50.646 INFO:tasks.workunit.client.0.smithi144.stdout:1/989: chown d5/df/d4b/d51/da7/lbc 22 1 2024-09-10T11:29:50.647 INFO:tasks.workunit.client.0.smithi144.stdout:1/990: chown d5/dd/d64/d79/d101/f131 23910737 1 2024-09-10T11:29:50.649 INFO:tasks.workunit.client.0.smithi144.stdout:1/991: write d5/d2a/d65/fcd [2926866,33425] 0 2024-09-10T11:29:50.652 INFO:tasks.workunit.client.0.smithi144.stdout:1/992: creat d5/d2a/d65/f13b x:0 0 0 2024-09-10T11:29:50.654 INFO:tasks.workunit.client.0.smithi144.stdout:8/812: dwrite d0/d9/d21/f5c [4194304,4194304] 0 2024-09-10T11:29:50.665 INFO:tasks.workunit.client.0.smithi144.stdout:8/813: dwrite d0/d9/d1f/f1c [4194304,4194304] 0 2024-09-10T11:29:50.667 INFO:tasks.workunit.client.0.smithi144.stdout:1/993: dwrite d5/dd/d64/dad/fae [0,4194304] 0 2024-09-10T11:29:50.674 INFO:tasks.workunit.client.0.smithi144.stdout:1/994: rename d5/d2a/d65/d73/d11d to d5/df/d4b/d51/d10a/dbd/dbe/d13c 0 2024-09-10T11:29:50.679 INFO:tasks.workunit.client.0.smithi144.stdout:1/995: getdents d5/d4a/d44 0 2024-09-10T11:29:50.681 INFO:tasks.workunit.client.0.smithi144.stdout:1/996: chown d5/d2a/d65/d73/f24 235 1 2024-09-10T11:29:50.696 INFO:tasks.workunit.client.0.smithi144.stdout:1/997: truncate d5/df/d4b/d51/d10a/d95/dd2/fde 1164779 0 2024-09-10T11:29:50.845 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:50 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:50.845 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:50 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:50.845 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:50 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:29:50.845 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:50 smithi144 ceph-mon[27942]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:29:50.845 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:50 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:29:50.845 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:50 smithi144 ceph-mon[27942]: Upgrade: Need to upgrade myself (mgr.smithi178.vmvxuf) 2024-09-10T11:29:50.845 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:50 smithi144 ceph-mon[27942]: Upgrade: Need to upgrade myself (mgr.smithi178.vmvxuf) 2024-09-10T11:29:50.845 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:50 smithi144 ceph-mon[27942]: pgmap v32: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 144 MiB/s rd, 340 MiB/s wr, 730 op/s 2024-09-10T11:29:50.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:50 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:50.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:50 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:50.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:50 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:29:50.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:50 smithi178 ceph-mon[33514]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:29:50.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:50 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:29:50.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:50 smithi178 ceph-mon[33514]: Upgrade: Need to upgrade myself (mgr.smithi178.vmvxuf) 2024-09-10T11:29:50.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:50 smithi178 ceph-mon[33514]: Upgrade: Need to upgrade myself (mgr.smithi178.vmvxuf) 2024-09-10T11:29:50.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:50 smithi178 ceph-mon[33514]: pgmap v32: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 144 MiB/s rd, 340 MiB/s wr, 730 op/s 2024-09-10T11:29:51.123 INFO:tasks.workunit.client.0.smithi144.stdout:4/933: sync 2024-09-10T11:29:51.174 INFO:tasks.workunit.client.0.smithi144.stdout:4/934: dwrite d14/d1e/d65/d8a/fe5 [0,4194304] 0 2024-09-10T11:29:51.178 INFO:tasks.workunit.client.0.smithi144.stdout:4/935: fsync d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/ff2 0 2024-09-10T11:29:51.291 INFO:tasks.workunit.client.0.smithi144.stdout:2/684: sync 2024-09-10T11:29:51.295 INFO:tasks.workunit.client.0.smithi144.stdout:2/685: rename d7/de/d36/f74 to d7/de/d36/d45/d8e/d9a/d7e/fe1 0 2024-09-10T11:29:51.298 INFO:tasks.workunit.client.0.smithi144.stdout:2/686: write d7/de/d3d/d4e/fa3 [3887701,71584] 0 2024-09-10T11:29:51.316 INFO:tasks.workunit.client.0.smithi144.stdout:2/687: creat d7/dcc/dd3/fe2 x:0 0 0 2024-09-10T11:29:51.318 INFO:tasks.workunit.client.0.smithi144.stdout:2/688: creat d7/de/d36/d45/d8e/d9a/d54/fe3 x:0 0 0 2024-09-10T11:29:51.320 INFO:tasks.workunit.client.0.smithi144.stdout:2/689: truncate d7/de/d36/f4f 4359323 0 2024-09-10T11:29:51.321 INFO:tasks.workunit.client.0.smithi144.stdout:2/690: write d7/da/d5b/fcf [620590,97879] 0 2024-09-10T11:29:51.323 INFO:tasks.workunit.client.0.smithi144.stdout:2/691: creat d7/d29/d97/fe4 x:0 0 0 2024-09-10T11:29:51.324 INFO:tasks.workunit.client.0.smithi144.stdout:2/692: write d7/d29/d97/fe4 [652800,123482] 0 2024-09-10T11:29:51.325 INFO:tasks.workunit.client.0.smithi144.stdout:2/693: dread - d7/de/d36/d45/d8e/d9a/d7e/f8b zero size 2024-09-10T11:29:51.327 INFO:tasks.workunit.client.0.smithi144.stdout:2/694: rmdir d7/de/d36/d45/d8e/d9a 39 2024-09-10T11:29:51.329 INFO:tasks.workunit.client.0.smithi144.stdout:2/695: rmdir d7/de/d36/d45/d47/d6c/d7c/d82 39 2024-09-10T11:29:51.335 INFO:tasks.workunit.client.0.smithi144.stdout:2/696: dwrite d7/d29/d92/f73 [0,4194304] 0 2024-09-10T11:29:51.506 INFO:tasks.workunit.client.0.smithi144.stdout:9/903: sync 2024-09-10T11:29:51.506 INFO:tasks.workunit.client.0.smithi144.stdout:3/853: sync 2024-09-10T11:29:51.506 INFO:tasks.workunit.client.0.smithi144.stdout:0/738: sync 2024-09-10T11:29:51.507 INFO:tasks.workunit.client.0.smithi144.stdout:0/739: dread - da/d3a/d4b/d4e/f88 zero size 2024-09-10T11:29:51.510 INFO:tasks.workunit.client.0.smithi144.stdout:0/740: creat da/d44/d82/fe1 x:0 0 0 2024-09-10T11:29:51.510 INFO:tasks.workunit.client.0.smithi144.stdout:3/854: write d4/f58 [341835,72608] 0 2024-09-10T11:29:51.512 INFO:tasks.workunit.client.0.smithi144.stdout:9/904: read d11/d36/db1/dbb/df0/d8f/dab/d64/fda [3592931,92811] 0 2024-09-10T11:29:51.513 INFO:tasks.workunit.client.0.smithi144.stdout:0/741: truncate da/d3a/d4b/d96/fa3 459851 0 2024-09-10T11:29:51.515 INFO:tasks.workunit.client.0.smithi144.stdout:3/855: unlink d4/dd/d21/f23 0 2024-09-10T11:29:51.515 INFO:tasks.workunit.client.0.smithi144.stdout:9/905: mknod d11/d36/d2d/df6/c13b 0 2024-09-10T11:29:51.516 INFO:tasks.workunit.client.0.smithi144.stdout:9/906: stat d11/d12/d1b/d2c/ce2 0 2024-09-10T11:29:51.519 INFO:tasks.workunit.client.0.smithi144.stdout:9/907: creat d11/d36/d2d/d38/f13c x:0 0 0 2024-09-10T11:29:51.520 INFO:tasks.workunit.client.0.smithi144.stdout:3/856: rename d4/d28/d67/d80/d17/d18/d34 to d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d113 22 2024-09-10T11:29:51.522 INFO:tasks.workunit.client.0.smithi144.stdout:0/742: dwrite da/f2e [0,4194304] 0 2024-09-10T11:29:51.524 INFO:tasks.workunit.client.0.smithi144.stdout:0/743: dread - da/d3a/d4b/d61/d6c/fe0 zero size 2024-09-10T11:29:51.525 INFO:tasks.workunit.client.0.smithi144.stdout:3/857: creat d4/d28/d67/d80/d17/d18/d34/d78/f114 x:0 0 0 2024-09-10T11:29:51.527 INFO:tasks.workunit.client.0.smithi144.stdout:3/858: fsync d4/dd/fef 0 2024-09-10T11:29:51.532 INFO:tasks.workunit.client.0.smithi144.stdout:0/744: rename da/d3a/d51/cba to da/d3a/d4b/d4e/d62/ce2 0 2024-09-10T11:29:51.537 INFO:tasks.workunit.client.0.smithi144.stdout:3/859: dwrite d4/d28/d67/d80/d17/f109 [0,4194304] 0 2024-09-10T11:29:51.541 INFO:tasks.workunit.client.0.smithi144.stdout:9/908: dwrite d11/d12/d6f/f8b [4194304,4194304] 0 2024-09-10T11:29:51.545 INFO:tasks.workunit.client.0.smithi144.stdout:0/745: dwrite da/d3a/d4b/d4e/d81/f98 [0,4194304] 0 2024-09-10T11:29:51.548 INFO:tasks.workunit.client.0.smithi144.stdout:9/909: mknod d11/d12/d1b/d2c/d78/dd2/c13d 0 2024-09-10T11:29:51.549 INFO:tasks.workunit.client.0.smithi144.stdout:0/746: truncate da/d3a/d4b/d4e/f60 1590966 0 2024-09-10T11:29:51.553 INFO:tasks.workunit.client.0.smithi144.stdout:3/860: write d4/d28/f3f [1236637,54605] 0 2024-09-10T11:29:51.555 INFO:tasks.workunit.client.0.smithi144.stdout:0/747: unlink da/d3a/d4b/d61/d6c/fb0 0 2024-09-10T11:29:51.558 INFO:tasks.workunit.client.0.smithi144.stdout:0/748: mknod da/d3a/d4b/ce3 0 2024-09-10T11:29:51.561 INFO:tasks.workunit.client.0.smithi144.stdout:9/910: dwrite d11/d36/d2d/d38/dc3/f120 [0,4194304] 0 2024-09-10T11:29:51.561 INFO:tasks.workunit.client.0.smithi144.stdout:0/749: write da/d3a/d4b/d61/d6c/fe0 [353633,23802] 0 2024-09-10T11:29:51.565 INFO:tasks.workunit.client.0.smithi144.stdout:0/750: creat da/d3a/d4b/d61/d6c/fe4 x:0 0 0 2024-09-10T11:29:51.568 INFO:tasks.workunit.client.0.smithi144.stdout:3/861: dread d4/dd/f12 [0,4194304] 0 2024-09-10T11:29:51.574 INFO:tasks.workunit.client.0.smithi144.stdout:6/783: sync 2024-09-10T11:29:51.575 INFO:tasks.workunit.client.0.smithi144.stdout:4/936: sync 2024-09-10T11:29:51.575 INFO:tasks.workunit.client.0.smithi144.stdout:7/863: sync 2024-09-10T11:29:51.575 INFO:tasks.workunit.client.0.smithi144.stdout:1/998: sync 2024-09-10T11:29:51.575 INFO:tasks.workunit.client.0.smithi144.stdout:8/814: sync 2024-09-10T11:29:51.575 INFO:tasks.workunit.client.0.smithi144.stdout:9/911: dwrite d11/d36/db1/fc6 [0,4194304] 0 2024-09-10T11:29:51.575 INFO:tasks.workunit.client.0.smithi144.stdout:5/877: sync 2024-09-10T11:29:51.580 INFO:tasks.workunit.client.0.smithi144.stdout:8/815: mkdir d0/d9/d21/d7e/d11b 0 2024-09-10T11:29:51.580 INFO:tasks.workunit.client.0.smithi144.stdout:1/999: mkdir d5/df/d4b/d51/d10a/dbd/d13d 0 2024-09-10T11:29:51.581 INFO:tasks.workunit.client.0.smithi144.stdout:7/864: dread - d5/dc/d12/f11c zero size 2024-09-10T11:29:51.582 INFO:tasks.workunit.client.0.smithi144.stdout:6/784: symlink d9/dd/d11/d1b/d2f/d7f/l117 0 2024-09-10T11:29:51.584 INFO:tasks.workunit.client.0.smithi144.stdout:5/878: rmdir d4/d5/df2 39 2024-09-10T11:29:51.587 INFO:tasks.workunit.client.0.smithi144.stdout:4/937: dwrite d14/d67/d84/f148 [0,4194304] 0 2024-09-10T11:29:51.590 INFO:tasks.workunit.client.0.smithi144.stdout:9/912: creat d11/d36/db1/dbb/df0/d123/f13e x:0 0 0 2024-09-10T11:29:51.590 INFO:tasks.workunit.client.0.smithi144.stdout:4/938: write d14/d1e/d22/d34/d3e/d40/d4b/f80 [207179,56537] 0 2024-09-10T11:29:51.592 INFO:tasks.workunit.client.0.smithi144.stdout:3/862: getdents d4/d28/d67/d80/d17/d18/d34/d65 0 2024-09-10T11:29:51.594 INFO:tasks.workunit.client.0.smithi144.stdout:3/863: fdatasync f3 0 2024-09-10T11:29:51.595 INFO:tasks.workunit.client.0.smithi144.stdout:6/785: read d9/d1f/d2c/d4f/f58 [3326047,74801] 0 2024-09-10T11:29:51.595 INFO:tasks.workunit.client.0.smithi144.stdout:7/865: truncate d5/dc/d11/d18/f1e 1512602 0 2024-09-10T11:29:51.596 INFO:tasks.workunit.client.0.smithi144.stdout:8/816: symlink d0/d9/d21/d7e/de5/l11c 0 2024-09-10T11:29:51.598 INFO:tasks.workunit.client.0.smithi144.stdout:6/786: dread - d9/dd/d11/d1b/d2f/dab/de7/f115 zero size 2024-09-10T11:29:51.600 INFO:tasks.workunit.client.0.smithi144.stdout:8/817: symlink d0/d1b/d5a/d8d/d115/l11d 0 2024-09-10T11:29:51.601 INFO:tasks.workunit.client.0.smithi144.stdout:7/866: read d5/dc/d2a/d60/dd6/d98/fca [1707410,31634] 0 2024-09-10T11:29:51.602 INFO:tasks.workunit.client.0.smithi144.stdout:7/867: chown d5/dc/d2a/d60/dd6/c82 179 1 2024-09-10T11:29:51.604 INFO:tasks.workunit.client.0.smithi144.stdout:9/913: getdents d11/d36/db1 0 2024-09-10T11:29:51.604 INFO:tasks.workunit.client.0.smithi144.stdout:8/818: mkdir d0/d9/d21/d25/db3/dc2/dcd/d11e 0 2024-09-10T11:29:51.606 INFO:tasks.workunit.client.0.smithi144.stdout:0/751: dread da/d44/f63 [0,4194304] 0 2024-09-10T11:29:51.608 INFO:tasks.workunit.client.0.smithi144.stdout:6/787: getdents d9/dd/d11/d1b/d2f/d7f 0 2024-09-10T11:29:51.610 INFO:tasks.workunit.client.0.smithi144.stdout:9/914: dread - d11/d36/db1/dbb/fc2 zero size 2024-09-10T11:29:51.610 INFO:tasks.workunit.client.0.smithi144.stdout:8/819: dread d0/d9/d21/f71 [0,4194304] 0 2024-09-10T11:29:51.611 INFO:tasks.workunit.client.0.smithi144.stdout:7/868: truncate d5/dc/d2a/d60/dd6/d98/dd1/f4d 6408076 0 2024-09-10T11:29:51.613 INFO:tasks.workunit.client.0.smithi144.stdout:5/879: dwrite d4/d5/d32/d47/f71 [0,4194304] 0 2024-09-10T11:29:51.619 INFO:tasks.workunit.client.0.smithi144.stdout:0/752: write da/d44/d82/fa2 [258464,39465] 0 2024-09-10T11:29:51.621 INFO:tasks.workunit.client.0.smithi144.stdout:8/820: creat d0/d9/d21/d25/db3/de8/f11f x:0 0 0 2024-09-10T11:29:51.623 INFO:tasks.workunit.client.0.smithi144.stdout:9/915: rename d11/d12/d1b/d2c/d78/fa0 to d11/d36/db1/f13f 0 2024-09-10T11:29:51.624 INFO:tasks.workunit.client.0.smithi144.stdout:0/753: read f0 [79394,53910] 0 2024-09-10T11:29:51.625 INFO:tasks.workunit.client.0.smithi144.stdout:6/788: symlink d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d10a/l118 0 2024-09-10T11:29:51.628 INFO:tasks.workunit.client.0.smithi144.stdout:5/880: dwrite d4/d5/d18/d67/d59/f6c [0,4194304] 0 2024-09-10T11:29:51.630 INFO:tasks.workunit.client.0.smithi144.stdout:9/916: creat d11/d12/d1b/f140 x:0 0 0 2024-09-10T11:29:51.633 INFO:tasks.workunit.client.0.smithi144.stdout:7/869: dwrite d5/dc/d14/d40/fb4 [0,4194304] 0 2024-09-10T11:29:51.635 INFO:tasks.workunit.client.0.smithi144.stdout:0/754: symlink da/d3a/d4b/d4e/d62/d78/le5 0 2024-09-10T11:29:51.636 INFO:tasks.workunit.client.0.smithi144.stdout:6/789: symlink d9/dd/d3e/d47/l119 0 2024-09-10T11:29:51.639 INFO:tasks.workunit.client.0.smithi144.stdout:5/881: dwrite d4/d5/d32/f5f [4194304,4194304] 0 2024-09-10T11:29:51.642 INFO:tasks.workunit.client.0.smithi144.stdout:7/870: write d5/dc/d14/d40/d58/d6a/f92 [5505274,99672] 0 2024-09-10T11:29:51.644 INFO:tasks.workunit.client.0.smithi144.stdout:7/871: chown d5/dc/d14/d40/d5e/d68/df0 111152232 1 2024-09-10T11:29:51.646 INFO:tasks.workunit.client.0.smithi144.stdout:0/755: mknod da/d3a/d4b/d96/ce6 0 2024-09-10T11:29:51.647 INFO:tasks.workunit.client.0.smithi144.stdout:5/882: rmdir d4/d5/d18/d67/db0 39 2024-09-10T11:29:51.652 INFO:tasks.workunit.client.0.smithi144.stdout:6/790: mkdir d9/dd/d11/d1b/d2f/d7f/dc4/d5b/df9/d11a 0 2024-09-10T11:29:51.652 INFO:tasks.workunit.client.0.smithi144.stdout:8/821: dread d0/d9/d21/d7e/fe9 [0,4194304] 0 2024-09-10T11:29:51.653 INFO:tasks.workunit.client.0.smithi144.stdout:7/872: rename d5/dc/d14/f44 to d5/dc/def/f11d 0 2024-09-10T11:29:51.654 INFO:tasks.workunit.client.0.smithi144.stdout:8/822: fsync d0/d9/f1d 0 2024-09-10T11:29:51.659 INFO:tasks.workunit.client.0.smithi144.stdout:8/823: mknod d0/d1b/d47/d5e/d76/c120 0 2024-09-10T11:29:51.660 INFO:tasks.workunit.client.0.smithi144.stdout:5/883: rename d4/d5/d18/d67/db0/db8/led to d4/d20/d22/d5c/d5d/de6/dbf/l110 0 2024-09-10T11:29:51.661 INFO:tasks.workunit.client.0.smithi144.stdout:8/824: creat d0/d9/d104/f121 x:0 0 0 2024-09-10T11:29:51.664 INFO:tasks.workunit.client.0.smithi144.stdout:6/791: dwrite d9/dd/d11/d1b/d2f/d7f/dc4/d5b/f61 [0,4194304] 0 2024-09-10T11:29:51.669 INFO:tasks.workunit.client.0.smithi144.stdout:5/884: getdents d4/d20/d24/d2d 0 2024-09-10T11:29:51.669 INFO:tasks.workunit.client.0.smithi144.stdout:9/917: write d11/d36/db1/dbb/df0/d8f/f93 [3525371,24640] 0 2024-09-10T11:29:51.673 INFO:tasks.workunit.client.0.smithi144.stdout:7/873: chown d5/dc/d14/d40/d58/d6a/db2/ff4 3932 1 2024-09-10T11:29:51.675 INFO:tasks.workunit.client.0.smithi144.stdout:5/885: creat d4/d20/d22/d75/ddd/ddf/f111 x:0 0 0 2024-09-10T11:29:51.677 INFO:tasks.workunit.client.0.smithi144.stdout:7/874: unlink d5/dc/fae 0 2024-09-10T11:29:51.678 INFO:tasks.workunit.client.0.smithi144.stdout:8/825: dwrite d0/d9/d21/d25/db3/f100 [8388608,4194304] 0 2024-09-10T11:29:51.681 INFO:tasks.workunit.client.0.smithi144.stdout:0/756: dread da/d3a/f50 [0,4194304] 0 2024-09-10T11:29:51.681 INFO:tasks.workunit.client.0.smithi144.stdout:5/886: unlink d4/d20/d22/d5c/c90 0 2024-09-10T11:29:51.684 INFO:tasks.workunit.client.0.smithi144.stdout:7/875: creat d5/dc/d2a/d60/dd6/d98/d102/f11e x:0 0 0 2024-09-10T11:29:51.686 INFO:tasks.workunit.client.0.smithi144.stdout:9/918: link d11/d12/d1b/d2c/d78/fdc d11/d36/d2d/d38/de7/d12d/f141 0 2024-09-10T11:29:51.687 INFO:tasks.workunit.client.0.smithi144.stdout:9/919: chown d11/d11c 3340802 1 2024-09-10T11:29:51.690 INFO:tasks.workunit.client.0.smithi144.stdout:7/876: unlink d5/dc/d2a/l3f 0 2024-09-10T11:29:51.692 INFO:tasks.workunit.client.0.smithi144.stdout:9/920: creat d11/d36/db1/dbb/df0/d8f/dc0/dd0/f142 x:0 0 0 2024-09-10T11:29:51.692 INFO:tasks.workunit.client.0.smithi144.stdout:7/877: stat d5/dc/d14/d40/c9b 0 2024-09-10T11:29:51.693 INFO:tasks.workunit.client.0.smithi144.stdout:0/757: link da/d3a/d51/d5f/l80 da/d3a/d51/d8f/le7 0 2024-09-10T11:29:51.696 INFO:tasks.workunit.client.0.smithi144.stdout:9/921: rename d11/fa2 to d11/d12/d6f/f143 0 2024-09-10T11:29:51.701 INFO:tasks.workunit.client.0.smithi144.stdout:9/922: mkdir d11/d36/db1/dbb/df0/d123/d144 0 2024-09-10T11:29:51.707 INFO:tasks.workunit.client.0.smithi144.stdout:7/878: link d5/dc/d14/d40/d58/d6a/dad/l10c d5/dc/d2a/l11f 0 2024-09-10T11:29:51.711 INFO:tasks.workunit.client.0.smithi144.stdout:9/923: rename d11/l62 to d11/d12/l145 0 2024-09-10T11:29:51.712 INFO:tasks.workunit.client.0.smithi144.stdout:0/758: write da/d3a/d4b/d4e/f5d [951526,124405] 0 2024-09-10T11:29:51.713 INFO:tasks.workunit.client.0.smithi144.stdout:7/879: symlink d5/dc/d14/d40/d5e/d68/df0/l120 0 2024-09-10T11:29:51.715 INFO:tasks.workunit.client.0.smithi144.stdout:5/887: dwrite d4/d20/d22/d75/fe0 [0,4194304] 0 2024-09-10T11:29:51.717 INFO:tasks.workunit.client.0.smithi144.stdout:7/880: chown d5/dc/d14/d40/d58/d6e/ld7 989747091 1 2024-09-10T11:29:51.722 INFO:tasks.workunit.client.0.smithi144.stdout:7/881: symlink d5/dc/d2a/d60/dd6/d98/d102/d115/l121 0 2024-09-10T11:29:51.723 INFO:tasks.workunit.client.0.smithi144.stdout:0/759: dwrite da/d44/d82/f8b [0,4194304] 0 2024-09-10T11:29:51.726 INFO:tasks.workunit.client.0.smithi144.stdout:7/882: mknod d5/dc/d2a/d60/d11a/c122 0 2024-09-10T11:29:51.729 INFO:tasks.workunit.client.0.smithi144.stdout:5/888: dwrite d4/d5/d18/d38/d6d/fcb [0,4194304] 0 2024-09-10T11:29:51.738 INFO:tasks.workunit.client.0.smithi144.stdout:5/889: link d4/d20/d24/d2d/d4b/l4c d4/d5/d18/d38/d6d/d8b/l112 0 2024-09-10T11:29:51.740 INFO:tasks.workunit.client.0.smithi144.stdout:5/890: fdatasync d4/d20/d22/d75/ddd/fe3 0 2024-09-10T11:29:51.742 INFO:tasks.workunit.client.0.smithi144.stdout:7/883: write d5/dc/d14/f50 [898127,83049] 0 2024-09-10T11:29:51.744 INFO:tasks.workunit.client.0.smithi144.stdout:5/891: rename d4/d5/d32/d47/df3/d9a to d4/d20/d22/d5c/d113 0 2024-09-10T11:29:51.745 INFO:tasks.workunit.client.0.smithi144.stdout:7/884: mknod d5/dc/d2a/d60/d11a/c123 0 2024-09-10T11:29:51.745 INFO:tasks.workunit.client.0.smithi144.stdout:5/892: fsync d4/d5/d32/daa/ff5 0 2024-09-10T11:29:51.749 INFO:tasks.workunit.client.0.smithi144.stdout:0/760: dread da/d3a/d94/db1/dcc/fa8 [0,4194304] 0 2024-09-10T11:29:51.750 INFO:tasks.workunit.client.0.smithi144.stdout:5/893: dread d4/d5/d32/f5f [4194304,4194304] 0 2024-09-10T11:29:51.751 INFO:tasks.workunit.client.0.smithi144.stdout:0/761: symlink da/d44/dd8/le8 0 2024-09-10T11:29:51.752 INFO:tasks.workunit.client.0.smithi144.stdout:5/894: mknod d4/d5/d32/d47/df3/d84/c114 0 2024-09-10T11:29:51.753 INFO:tasks.workunit.client.0.smithi144.stdout:2/697: sync 2024-09-10T11:29:51.753 INFO:tasks.workunit.client.0.smithi144.stdout:0/762: mknod da/d3a/d94/ce9 0 2024-09-10T11:29:51.755 INFO:tasks.workunit.client.0.smithi144.stdout:5/895: creat d4/d20/d22/d75/ddd/f115 x:0 0 0 2024-09-10T11:29:51.756 INFO:tasks.workunit.client.0.smithi144.stdout:5/896: readlink d4/d5/d32/daa/l10a 0 2024-09-10T11:29:51.758 INFO:tasks.workunit.client.0.smithi144.stdout:0/763: dread da/f2e [0,4194304] 0 2024-09-10T11:29:51.762 INFO:tasks.workunit.client.0.smithi144.stdout:0/764: getdents da/d44 0 2024-09-10T11:29:51.763 INFO:tasks.workunit.client.0.smithi144.stdout:2/698: write d7/da/ddb/f7b [1969369,90014] 0 2024-09-10T11:29:51.764 INFO:tasks.workunit.client.0.smithi144.stdout:5/897: dwrite d4/d5/d18/d67/d59/f6c [0,4194304] 0 2024-09-10T11:29:51.766 INFO:tasks.workunit.client.0.smithi144.stdout:0/765: dread - da/d3a/d4b/d4e/d62/d78/fa5 zero size 2024-09-10T11:29:51.770 INFO:tasks.workunit.client.0.smithi144.stdout:0/766: symlink da/d3a/d4b/d4e/d62/lea 0 2024-09-10T11:29:51.771 INFO:tasks.workunit.client.0.smithi144.stdout:2/699: dwrite d7/d42/d64/d8d/fbb [0,4194304] 0 2024-09-10T11:29:51.774 INFO:tasks.workunit.client.0.smithi144.stdout:0/767: mkdir da/d44/d82/d8c/deb 0 2024-09-10T11:29:51.777 INFO:tasks.workunit.client.0.smithi144.stdout:0/768: mkdir da/d3a/d51/d8f/dec 0 2024-09-10T11:29:51.779 INFO:tasks.workunit.client.0.smithi144.stdout:5/898: truncate d4/d5/d18/d38/d6d/d8b/f9b 312202 0 2024-09-10T11:29:51.780 INFO:tasks.workunit.client.0.smithi144.stdout:2/700: write d7/de/f28 [1346795,59860] 0 2024-09-10T11:29:51.781 INFO:tasks.workunit.client.0.smithi144.stdout:2/701: fdatasync d7/dcc/fdd 0 2024-09-10T11:29:51.785 INFO:tasks.workunit.client.0.smithi144.stdout:0/769: rename da/d3a/d51/d8f/ld1 to da/d3a/d4b/d4e/d62/led 0 2024-09-10T11:29:51.788 INFO:tasks.workunit.client.0.smithi144.stdout:2/702: link d7/de/d3d/fab d7/dcc/dd3/fe5 0 2024-09-10T11:29:51.788 INFO:tasks.workunit.client.0.smithi144.stdout:5/899: dwrite d4/fa [0,4194304] 0 2024-09-10T11:29:51.790 INFO:tasks.workunit.client.0.smithi144.stdout:0/770: rename da/d3a/f50 to da/d3a/d4b/d61/ddf/fee 0 2024-09-10T11:29:51.793 INFO:tasks.workunit.client.0.smithi144.stdout:5/900: symlink d4/d5/d32/d47/df3/d74/dbe/l116 0 2024-09-10T11:29:51.793 INFO:tasks.workunit.client.0.smithi144.stdout:2/703: unlink d7/de/d36/d45/d8e/d9a/d54/c79 0 2024-09-10T11:29:51.795 INFO:tasks.workunit.client.0.smithi144.stdout:0/771: symlink da/d44/d82/d8c/lef 0 2024-09-10T11:29:51.795 INFO:tasks.workunit.client.0.smithi144.stdout:2/704: fdatasync d7/da/ddb/f7b 0 2024-09-10T11:29:51.796 INFO:tasks.workunit.client.0.smithi144.stdout:2/705: chown d7/da/ddb/d7f 678 1 2024-09-10T11:29:51.815 INFO:tasks.workunit.client.0.smithi144.stdout:5/901: dwrite d4/d5/f28 [0,4194304] 0 2024-09-10T11:29:51.815 INFO:tasks.workunit.client.0.smithi144.stdout:2/706: dwrite d7/de/d36/d45/d47/fc7 [0,4194304] 0 2024-09-10T11:29:51.818 INFO:tasks.workunit.client.0.smithi144.stdout:2/707: write d7/de/d5f/fd7 [649928,7701] 0 2024-09-10T11:29:51.822 INFO:tasks.workunit.client.0.smithi144.stdout:0/772: dread da/d3a/d4b/f86 [0,4194304] 0 2024-09-10T11:29:51.823 INFO:tasks.workunit.client.0.smithi144.stdout:5/902: creat d4/d5/df2/f117 x:0 0 0 2024-09-10T11:29:51.824 INFO:tasks.workunit.client.0.smithi144.stdout:2/708: symlink d7/de/d36/d45/le6 0 2024-09-10T11:29:51.825 INFO:tasks.workunit.client.0.smithi144.stdout:2/709: stat d7/d29/d97/la6 0 2024-09-10T11:29:51.826 INFO:tasks.workunit.client.0.smithi144.stdout:5/903: creat d4/d5/d18/d67/db0/db8/f118 x:0 0 0 2024-09-10T11:29:51.826 INFO:tasks.workunit.client.0.smithi144.stdout:2/710: write d7/de/d3d/d4e/f6a [4443419,64335] 0 2024-09-10T11:29:51.831 INFO:tasks.workunit.client.0.smithi144.stdout:2/711: creat d7/de/d70/fe7 x:0 0 0 2024-09-10T11:29:51.834 INFO:tasks.workunit.client.0.smithi144.stdout:5/904: write d4/d20/d24/d2d/fc9 [322333,78949] 0 2024-09-10T11:29:51.835 INFO:tasks.workunit.client.0.smithi144.stdout:0/773: dwrite da/d3a/d94/db1/dcc/fa8 [4194304,4194304] 0 2024-09-10T11:29:51.836 INFO:tasks.workunit.client.0.smithi144.stdout:5/905: fsync d4/d20/d22/d75/ddd/f115 0 2024-09-10T11:29:51.842 INFO:tasks.workunit.client.0.smithi144.stdout:0/774: creat da/d44/d82/d8c/deb/ff0 x:0 0 0 2024-09-10T11:29:51.846 INFO:tasks.workunit.client.0.smithi144.stdout:0/775: truncate da/d3a/d4b/d61/f9a 5063843 0 2024-09-10T11:29:51.847 INFO:tasks.workunit.client.0.smithi144.stdout:0/776: chown da/lbe 5552791 1 2024-09-10T11:29:51.849 INFO:tasks.workunit.client.0.smithi144.stdout:0/777: read da/d3a/d51/f7a [1589790,124862] 0 2024-09-10T11:29:51.853 INFO:tasks.workunit.client.0.smithi144.stdout:2/712: dread d7/de/d36/d45/d8e/d9a/f2a [0,4194304] 0 2024-09-10T11:29:51.854 INFO:tasks.workunit.client.0.smithi144.stdout:2/713: stat d7/da/c93 0 2024-09-10T11:29:51.855 INFO:tasks.workunit.client.0.smithi144.stdout:0/778: dread da/d3a/d4b/f86 [0,4194304] 0 2024-09-10T11:29:51.862 INFO:tasks.workunit.client.0.smithi144.stdout:2/714: link d7/de/d36/d45/d47/fc7 d7/d42/fe8 0 2024-09-10T11:29:51.864 INFO:tasks.workunit.client.0.smithi144.stdout:0/779: dwrite da/d3a/d51/d5f/f4a [0,4194304] 0 2024-09-10T11:29:51.866 INFO:tasks.workunit.client.0.smithi144.stdout:2/715: dread - d7/de/d36/d45/f84 zero size 2024-09-10T11:29:51.869 INFO:tasks.workunit.client.0.smithi144.stdout:0/780: mknod da/d3a/d8e/dab/cf1 0 2024-09-10T11:29:51.888 INFO:tasks.workunit.client.0.smithi144.stdout:0/781: dwrite da/f41 [0,4194304] 0 2024-09-10T11:29:51.888 INFO:tasks.workunit.client.0.smithi144.stdout:2/716: dwrite d7/da/d5b/f96 [0,4194304] 0 2024-09-10T11:29:51.901 INFO:tasks.workunit.client.0.smithi144.stdout:2/717: rename d7/de/d36/d45/d8e/c9f to d7/d29/d97/ce9 0 2024-09-10T11:29:51.903 INFO:tasks.workunit.client.0.smithi144.stdout:2/718: write d7/da/ddb/d7f/fcb [725589,11461] 0 2024-09-10T11:29:51.906 INFO:tasks.workunit.client.0.smithi144.stdout:2/719: mkdir d7/de/dea 0 2024-09-10T11:29:51.915 INFO:tasks.workunit.client.0.smithi144.stdout:0/782: dwrite da/f21 [0,4194304] 0 2024-09-10T11:29:51.919 INFO:tasks.workunit.client.0.smithi144.stdout:0/783: chown da/d3a/d4b/d4e/d90/fa6 178157626 1 2024-09-10T11:29:51.926 INFO:tasks.workunit.client.0.smithi144.stdout:0/784: mkdir da/d3a/df2 0 2024-09-10T11:29:51.926 INFO:tasks.workunit.client.0.smithi144.stdout:2/720: dwrite d7/de/d36/d45/d47/d6c/d7c/d82/fbc [0,4194304] 0 2024-09-10T11:29:51.935 INFO:tasks.workunit.client.0.smithi144.stdout:0/785: rename da/d3a/d4b/d61/laf to da/d44/dd8/lf3 0 2024-09-10T11:29:51.937 INFO:tasks.workunit.client.0.smithi144.stdout:0/786: mknod da/d3a/d4b/d61/d6c/cf4 0 2024-09-10T11:29:51.938 INFO:tasks.workunit.client.0.smithi144.stdout:2/721: write d7/de/d36/d45/d8e/d9a/d7e/fe1 [536442,79405] 0 2024-09-10T11:29:51.939 INFO:tasks.workunit.client.0.smithi144.stdout:2/722: chown d7/de/f3b 1715856 1 2024-09-10T11:29:51.942 INFO:tasks.workunit.client.0.smithi144.stdout:0/787: rename da/d44/d82/l8a to da/d3a/d51/lf5 0 2024-09-10T11:29:51.942 INFO:tasks.workunit.client.0.smithi144.stdout:2/723: mknod d7/de/d36/d45/d8e/d9a/dc9/ceb 0 2024-09-10T11:29:51.942 INFO:tasks.workunit.client.0.smithi144.stdout:0/788: write da/d44/d82/fe1 [152273,32019] 0 2024-09-10T11:29:51.943 INFO:tasks.workunit.client.0.smithi144.stdout:0/789: stat da/f1c 0 2024-09-10T11:29:51.943 INFO:tasks.workunit.client.0.smithi144.stdout:0/790: fsync da/d44/fda 0 2024-09-10T11:29:51.944 INFO:tasks.workunit.client.0.smithi144.stdout:0/791: chown da/d44/d82/f8b 742 1 2024-09-10T11:29:51.945 INFO:tasks.workunit.client.0.smithi144.stdout:0/792: stat da/d3a/d94 0 2024-09-10T11:29:51.948 INFO:tasks.workunit.client.0.smithi144.stdout:0/793: creat da/d3a/d4b/ff6 x:0 0 0 2024-09-10T11:29:51.954 INFO:tasks.workunit.client.0.smithi144.stdout:0/794: write da/d3a/f93 [366390,30264] 0 2024-09-10T11:29:51.955 INFO:tasks.workunit.client.0.smithi144.stdout:2/724: dread d7/da/d1e/f66 [0,4194304] 0 2024-09-10T11:29:51.956 INFO:tasks.workunit.client.0.smithi144.stdout:2/725: fdatasync d7/f41 0 2024-09-10T11:29:51.957 INFO:tasks.workunit.client.0.smithi144.stdout:2/726: chown d7/de/f37 176 1 2024-09-10T11:29:51.962 INFO:tasks.workunit.client.0.smithi144.stdout:2/727: dwrite d7/d42/d64/faa [4194304,4194304] 0 2024-09-10T11:29:51.965 INFO:tasks.workunit.client.0.smithi144.stdout:2/728: getdents d7/de/d36/d45/d8e/d9a/d54/d61 0 2024-09-10T11:29:51.969 INFO:tasks.workunit.client.0.smithi144.stdout:2/729: link d7/da/l39 d7/de/lec 0 2024-09-10T11:29:51.972 INFO:tasks.workunit.client.0.smithi144.stdout:2/730: rename d7/de/d5f/fd7 to d7/d29/d92/fed 0 2024-09-10T11:29:51.974 INFO:tasks.workunit.client.0.smithi144.stdout:2/731: mknod d7/d29/d97/cee 0 2024-09-10T11:29:51.980 INFO:tasks.workunit.client.0.smithi144.stdout:2/732: truncate d7/de/f22 15848 0 2024-09-10T11:29:51.981 INFO:tasks.workunit.client.0.smithi144.stdout:2/733: write d7/d42/d64/fd6 [672672,63535] 0 2024-09-10T11:29:51.983 INFO:tasks.workunit.client.0.smithi144.stdout:2/734: mknod d7/d29/cef 0 2024-09-10T11:29:51.983 INFO:tasks.workunit.client.0.smithi144.stdout:2/735: write d7/da/ddb/d7f/fcb [112719,41383] 0 2024-09-10T11:29:51.984 INFO:tasks.workunit.client.0.smithi144.stdout:2/736: dread - d7/de/d36/d45/d8e/fdf zero size 2024-09-10T11:29:52.012 INFO:tasks.workunit.client.0.smithi144.stdout:2/737: write d7/d42/d64/d8d/fbb [4695877,51692] 0 2024-09-10T11:29:52.015 INFO:tasks.workunit.client.0.smithi144.stdout:2/738: mkdir d7/de/d36/d45/d47/d6c/d7c/d82/df0 0 2024-09-10T11:29:52.017 INFO:tasks.workunit.client.0.smithi144.stdout:2/739: mkdir d7/de/d70/df1 0 2024-09-10T11:29:52.018 INFO:tasks.workunit.client.0.smithi144.stdout:2/740: truncate d7/d29/f4b 531793 0 2024-09-10T11:29:52.019 INFO:tasks.workunit.client.0.smithi144.stdout:2/741: read d7/de/f3b [1126228,24462] 0 2024-09-10T11:29:52.020 INFO:tasks.workunit.client.0.smithi144.stdout:2/742: readlink d7/d42/d64/d8d/lc5 0 2024-09-10T11:29:52.020 INFO:tasks.workunit.client.0.smithi144.stdout:2/743: chown d7/de/d36/d45/d47/c77 24520301 1 2024-09-10T11:29:52.022 INFO:tasks.workunit.client.0.smithi144.stdout:2/744: mknod d7/de/d36/d45/d8e/cf2 0 2024-09-10T11:29:52.027 INFO:tasks.workunit.client.0.smithi144.stdout:2/745: rename d7/f11 to d7/d29/d8f/ddc/ff3 0 2024-09-10T11:29:52.042 INFO:tasks.workunit.client.0.smithi144.stdout:2/746: truncate d7/de/d36/f51 4957270 0 2024-09-10T11:29:52.058 INFO:tasks.workunit.client.0.smithi144.stdout:2/747: dread d7/de/d3d/d4e/f6d [0,4194304] 0 2024-09-10T11:29:52.062 INFO:tasks.workunit.client.0.smithi144.stdout:2/748: symlink d7/da/lf4 0 2024-09-10T11:29:52.064 INFO:tasks.workunit.client.0.smithi144.stdout:2/749: mkdir d7/de/d5f/df5 0 2024-09-10T11:29:52.066 INFO:tasks.workunit.client.0.smithi144.stdout:2/750: dread - d7/de/d3d/fab zero size 2024-09-10T11:29:52.071 INFO:tasks.workunit.client.0.smithi144.stdout:2/751: getdents d7/da/d1e 0 2024-09-10T11:29:52.076 INFO:tasks.workunit.client.0.smithi144.stdout:2/752: mknod d7/de/d5f/df5/cf6 0 2024-09-10T11:29:52.078 INFO:tasks.workunit.client.0.smithi144.stdout:2/753: symlink d7/d29/d92/lf7 0 2024-09-10T11:29:52.080 INFO:tasks.workunit.client.0.smithi144.stdout:2/754: fdatasync d7/de/d36/d45/d47/fc7 0 2024-09-10T11:29:52.082 INFO:tasks.workunit.client.0.smithi144.stdout:2/755: chown d7/de/d36/d45/d47/d6c/d7c/d82/fbd 138568497 1 2024-09-10T11:29:52.085 INFO:tasks.workunit.client.0.smithi144.stdout:2/756: truncate d7/f8 4794529 0 2024-09-10T11:29:52.091 INFO:tasks.workunit.client.0.smithi144.stdout:2/757: dread d7/de/d36/d45/d8e/d9a/f2a [0,4194304] 0 2024-09-10T11:29:52.108 INFO:tasks.workunit.client.0.smithi144.stdout:2/758: dread d7/de/d3d/f3e [0,4194304] 0 2024-09-10T11:29:52.109 INFO:tasks.workunit.client.0.smithi144.stdout:2/759: fsync d7/d29/d97/fc0 0 2024-09-10T11:29:52.113 INFO:tasks.workunit.client.0.smithi144.stdout:2/760: rename d7/d29/cef to d7/de/d5f/cf8 0 2024-09-10T11:29:52.114 INFO:tasks.workunit.client.0.smithi144.stdout:2/761: chown d7/de/d36/d45/d8e/d9a/dc9 314 1 2024-09-10T11:29:52.116 INFO:tasks.workunit.client.0.smithi144.stdout:2/762: fsync d7/de/d36/d45/f84 0 2024-09-10T11:29:52.124 INFO:tasks.workunit.client.0.smithi144.stdout:2/763: write d7/f8 [4305981,28469] 0 2024-09-10T11:29:52.133 INFO:tasks.workunit.client.0.smithi144.stdout:2/764: dwrite d7/de/d36/d45/d8e/d9a/f53 [0,4194304] 0 2024-09-10T11:29:52.136 INFO:tasks.workunit.client.0.smithi144.stdout:2/765: chown d7/d42/d64/d76 0 1 2024-09-10T11:29:52.140 INFO:tasks.workunit.client.0.smithi144.stdout:2/766: getdents d7/d29/d8f/ddc 0 2024-09-10T11:29:52.141 INFO:tasks.workunit.client.0.smithi144.stdout:2/767: stat d7/de/d36/d45/d8e/d9a/dc9/ceb 0 2024-09-10T11:29:52.146 INFO:tasks.workunit.client.0.smithi144.stdout:2/768: write d7/de/d3d/d4e/f87 [290224,115396] 0 2024-09-10T11:29:52.147 INFO:tasks.workunit.client.0.smithi144.stdout:2/769: stat d7/de/d3d/d4e/f6a 0 2024-09-10T11:29:52.148 INFO:tasks.workunit.client.0.smithi144.stdout:2/770: write d7/da/ddb/f7b [4060898,76726] 0 2024-09-10T11:29:52.150 INFO:tasks.workunit.client.0.smithi144.stdout:2/771: symlink d7/d42/d64/dd5/lf9 0 2024-09-10T11:29:52.158 INFO:tasks.workunit.client.0.smithi144.stdout:2/772: dwrite d7/dcc/dd3/fe5 [0,4194304] 0 2024-09-10T11:29:52.160 INFO:tasks.workunit.client.0.smithi144.stdout:2/773: rmdir d7/de/d36/d45/d8e/d9a/d7e 39 2024-09-10T11:29:52.174 INFO:tasks.workunit.client.0.smithi144.stdout:2/774: dread d7/de/f28 [0,4194304] 0 2024-09-10T11:29:52.180 INFO:tasks.workunit.client.0.smithi144.stdout:2/775: truncate d7/de/d36/d45/d47/d6c/d7c/d82/fbc 3437734 0 2024-09-10T11:29:52.184 INFO:tasks.workunit.client.0.smithi144.stdout:2/776: link d7/de/f22 d7/de/d36/d45/d8e/d9a/d54/d61/ffa 0 2024-09-10T11:29:52.186 INFO:tasks.workunit.client.0.smithi144.stdout:2/777: mknod d7/de/d36/d45/d47/d6c/d7c/d82/df0/cfb 0 2024-09-10T11:29:52.187 INFO:tasks.workunit.client.0.smithi144.stdout:2/778: creat d7/da/ddb/ffc x:0 0 0 2024-09-10T11:29:52.187 INFO:tasks.workunit.client.0.smithi144.stdout:2/779: chown d7/de/f3b 12 1 2024-09-10T11:29:52.188 INFO:tasks.workunit.client.0.smithi144.stdout:2/780: stat d7/dcc/dd3/fe5 0 2024-09-10T11:29:52.189 INFO:tasks.workunit.client.0.smithi144.stdout:2/781: mknod d7/de/dea/cfd 0 2024-09-10T11:29:52.192 INFO:tasks.workunit.client.0.smithi144.stdout:2/782: getdents d7/de/d3d/dc8 0 2024-09-10T11:29:52.193 INFO:tasks.workunit.client.0.smithi144.stdout:2/783: mknod d7/da/d1e/cfe 0 2024-09-10T11:29:52.194 INFO:tasks.workunit.client.0.smithi144.stdout:2/784: mkdir d7/de/d3d/dff 0 2024-09-10T11:29:52.197 INFO:tasks.workunit.client.0.smithi144.stdout:2/785: dread d7/d42/d64/d76/f8c [4194304,4194304] 0 2024-09-10T11:29:52.200 INFO:tasks.workunit.client.0.smithi144.stdout:2/786: getdents d7/de/d5f/df5 0 2024-09-10T11:29:52.205 INFO:tasks.workunit.client.0.smithi144.stdout:2/787: dwrite d7/de/d36/d45/d47/d6c/d7c/d82/f98 [0,4194304] 0 2024-09-10T11:29:52.209 INFO:tasks.workunit.client.0.smithi144.stdout:2/788: symlink d7/de/d36/d45/d8e/d9a/l100 0 2024-09-10T11:29:52.214 INFO:tasks.workunit.client.0.smithi144.stdout:2/789: write d7/de/f37 [326805,28218] 0 2024-09-10T11:29:52.218 INFO:tasks.workunit.client.0.smithi144.stdout:2/790: creat d7/de/d36/d45/d47/d6c/d7c/f101 x:0 0 0 2024-09-10T11:29:52.220 INFO:tasks.workunit.client.0.smithi144.stdout:2/791: creat d7/da/ddb/d7f/f102 x:0 0 0 2024-09-10T11:29:52.223 INFO:tasks.workunit.client.0.smithi144.stdout:2/792: creat d7/de/d36/d45/d47/d6c/d7c/d82/df0/f103 x:0 0 0 2024-09-10T11:29:52.225 INFO:tasks.workunit.client.0.smithi144.stdout:2/793: read d7/d29/d8f/ddc/ff3 [333166,128967] 0 2024-09-10T11:29:52.227 INFO:tasks.workunit.client.0.smithi144.stdout:2/794: symlink d7/da/ddb/d7f/l104 0 2024-09-10T11:29:52.228 INFO:tasks.workunit.client.0.smithi144.stdout:2/795: creat d7/de/d3d/f105 x:0 0 0 2024-09-10T11:29:52.233 INFO:tasks.workunit.client.0.smithi144.stdout:2/796: write d7/de/fbe [441997,66413] 0 2024-09-10T11:29:52.243 INFO:tasks.workunit.client.0.smithi144.stdout:2/797: dwrite d7/de/d3d/d4e/dac/fae [0,4194304] 0 2024-09-10T11:29:52.488 INFO:tasks.workunit.client.0.smithi144.stdout:4/939: sync 2024-09-10T11:29:52.494 INFO:tasks.workunit.client.0.smithi144.stdout:4/940: dread d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/f93 [0,4194304] 0 2024-09-10T11:29:52.498 INFO:tasks.workunit.client.0.smithi144.stdout:4/941: link d14/d1e/d83/da4/d11b/la2 d14/d1e/d83/da4/da9/l151 0 2024-09-10T11:29:52.500 INFO:tasks.workunit.client.0.smithi144.stdout:4/942: dread - d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f/d103/f13b zero size 2024-09-10T11:29:52.502 INFO:tasks.workunit.client.0.smithi144.stdout:4/943: symlink d14/d67/l152 0 2024-09-10T11:29:52.507 INFO:tasks.workunit.client.0.smithi144.stdout:4/944: dwrite d14/d6f/f9c [4194304,4194304] 0 2024-09-10T11:29:52.511 INFO:tasks.workunit.client.0.smithi144.stdout:4/945: truncate d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/dd4/fd6 964863 0 2024-09-10T11:29:52.514 INFO:tasks.workunit.client.0.smithi144.stdout:4/946: read d14/d1e/d22/f4d [10620,128342] 0 2024-09-10T11:29:52.515 INFO:tasks.workunit.client.0.smithi144.stdout:4/947: chown d14/d1e/d22/d34/d3e/d40/d4b/d4c/dab 1322364 1 2024-09-10T11:29:52.521 INFO:tasks.workunit.client.0.smithi144.stdout:4/948: dwrite d14/d7e/f9f [0,4194304] 0 2024-09-10T11:29:52.523 INFO:tasks.workunit.client.0.smithi144.stdout:4/949: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/f5b [1775346,101015] 0 2024-09-10T11:29:52.526 INFO:tasks.workunit.client.0.smithi144.stdout:4/950: truncate d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/f147 150205 0 2024-09-10T11:29:52.527 INFO:tasks.workunit.client.0.smithi144.stdout:4/951: stat d14/d1e/d22/d34/d3e/d40/d4b/d4c/dab/df3 0 2024-09-10T11:29:52.531 INFO:tasks.workunit.client.0.smithi144.stdout:4/952: mkdir d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/d10b/d153 0 2024-09-10T11:29:52.533 INFO:tasks.workunit.client.0.smithi144.stdout:4/953: chown d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/de4/ff9 2 1 2024-09-10T11:29:52.534 INFO:tasks.workunit.client.0.smithi144.stdout:4/954: fdatasync d14/d1e/d22/f131 0 2024-09-10T11:29:52.537 INFO:tasks.workunit.client.0.smithi144.stdout:4/955: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/dab/fc5 [3382958,57641] 0 2024-09-10T11:29:52.541 INFO:tasks.workunit.client.0.smithi144.stdout:4/956: link d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f/d103/f10c d14/dcb/f154 0 2024-09-10T11:29:52.542 INFO:tasks.workunit.client.0.smithi144.stdout:4/957: fsync d14/d67/d69/d90/dda/de7/f133 0 2024-09-10T11:29:52.544 INFO:tasks.workunit.client.0.smithi144.stdout:4/958: truncate d14/d1e/d22/d34/d3e/fe2 530287 0 2024-09-10T11:29:52.545 INFO:tasks.workunit.client.0.smithi144.stdout:4/959: chown d14/d1e/d22/d34/d91 59875846 1 2024-09-10T11:29:52.546 INFO:tasks.workunit.client.0.smithi144.stdout:4/960: write d14/d67/d69/d90/dda/de7/f133 [322199,90677] 0 2024-09-10T11:29:52.547 INFO:tasks.workunit.client.0.smithi144.stdout:4/961: chown d14/d67/d84/c136 5880 1 2024-09-10T11:29:52.562 INFO:tasks.workunit.client.0.smithi144.stdout:4/962: dread d14/d1e/d22/d34/d3e/d40/d4b/f57 [0,4194304] 0 2024-09-10T11:29:52.565 INFO:tasks.workunit.client.0.smithi144.stdout:4/963: rename d14/d1e/d22/d34/d3e/d40/d4b/f80 to d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f/f155 0 2024-09-10T11:29:52.572 INFO:tasks.workunit.client.0.smithi144.stdout:3/864: sync 2024-09-10T11:29:52.572 INFO:tasks.workunit.client.0.smithi144.stdout:3/865: readlink d4/d28/d67/d80/d17/l52 0 2024-09-10T11:29:52.578 INFO:tasks.workunit.client.0.smithi144.stdout:4/964: dread d14/d1e/f20 [4194304,4194304] 0 2024-09-10T11:29:52.581 INFO:tasks.workunit.client.0.smithi144.stdout:4/965: mkdir d14/d67/d69/dca/d156 0 2024-09-10T11:29:52.582 INFO:tasks.workunit.client.0.smithi144.stdout:3/866: write d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/f7b [1793163,8004] 0 2024-09-10T11:29:52.584 INFO:tasks.workunit.client.0.smithi144.stdout:3/867: symlink d4/d28/d67/d80/d17/d18/df4/l115 0 2024-09-10T11:29:52.587 INFO:tasks.workunit.client.0.smithi144.stdout:4/966: dwrite d14/d1e/d83/da4/f12b [0,4194304] 0 2024-09-10T11:29:52.588 INFO:tasks.workunit.client.0.smithi144.stdout:3/868: truncate d4/dd/d21/f86 979872 0 2024-09-10T11:29:52.603 INFO:tasks.workunit.client.0.smithi144.stdout:3/869: dread d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/f74 [0,4194304] 0 2024-09-10T11:29:52.610 INFO:tasks.workunit.client.0.smithi144.stdout:4/967: dread d14/d1e/d22/d34/d3e/d40/d4b/d4c/f5f [0,4194304] 0 2024-09-10T11:29:52.637 INFO:tasks.workunit.client.0.smithi144.stdout:4/968: dread d14/d1e/d83/da4/da9/fc7 [0,4194304] 0 2024-09-10T11:29:52.641 INFO:tasks.workunit.client.0.smithi144.stdout:4/969: creat d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/d10b/d141/f157 x:0 0 0 2024-09-10T11:29:52.642 INFO:tasks.workunit.client.0.smithi144.stdout:4/970: symlink d14/d67/d69/d139/l158 0 2024-09-10T11:29:52.644 INFO:tasks.workunit.client.0.smithi144.stdout:4/971: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/d10b/d141/d143/f14d [349135,72431] 0 2024-09-10T11:29:52.646 INFO:tasks.workunit.client.0.smithi144.stdout:4/972: mknod d14/d67/d69/dca/c159 0 2024-09-10T11:29:52.654 INFO:tasks.workunit.client.0.smithi144.stdout:4/973: write d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/fb0 [3491982,103764] 0 2024-09-10T11:29:52.663 INFO:tasks.workunit.client.0.smithi144.stdout:4/974: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/ffc [0,4194304] 0 2024-09-10T11:29:52.665 INFO:tasks.workunit.client.0.smithi144.stdout:4/975: write d14/dcb/fce [2641291,35971] 0 2024-09-10T11:29:52.671 INFO:tasks.workunit.client.0.smithi144.stdout:4/976: write d14/d1e/d22/d34/d3e/fe2 [1011350,53529] 0 2024-09-10T11:29:52.673 INFO:tasks.workunit.client.0.smithi144.stdout:4/977: stat d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/d10b/d141 0 2024-09-10T11:29:52.675 INFO:tasks.workunit.client.0.smithi144.stdout:4/978: mkdir d14/d7e/d15a 0 2024-09-10T11:29:52.683 INFO:tasks.workunit.client.0.smithi144.stdout:4/979: dwrite d14/d1e/d22/f37 [4194304,4194304] 0 2024-09-10T11:29:52.706 INFO:tasks.workunit.client.0.smithi144.stdout:4/980: dread f5 [0,4194304] 0 2024-09-10T11:29:52.710 INFO:tasks.workunit.client.0.smithi144.stdout:4/981: dread d14/d6f/f96 [0,4194304] 0 2024-09-10T11:29:52.711 INFO:tasks.workunit.client.0.smithi144.stdout:4/982: write d14/d1e/d22/d34/d3e/fe2 [306443,14394] 0 2024-09-10T11:29:52.714 INFO:tasks.workunit.client.0.smithi144.stdout:4/983: unlink d14/d1e/d83/da4/f12b 0 2024-09-10T11:29:52.716 INFO:tasks.workunit.client.0.smithi144.stdout:4/984: mknod d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f/c15b 0 2024-09-10T11:29:52.717 INFO:tasks.workunit.client.0.smithi144.stdout:4/985: creat d14/d6f/f15c x:0 0 0 2024-09-10T11:29:52.719 INFO:tasks.workunit.client.0.smithi144.stdout:4/986: mknod d14/d67/d69/d90/dda/c15d 0 2024-09-10T11:29:52.720 INFO:tasks.workunit.client.0.smithi144.stdout:4/987: symlink d14/d6f/l15e 0 2024-09-10T11:29:52.720 INFO:tasks.workunit.client.0.smithi144.stdout:4/988: chown d14/d1e/d83 1 1 2024-09-10T11:29:52.722 INFO:tasks.workunit.client.0.smithi144.stdout:4/989: mknod d14/d1e/d22/c15f 0 2024-09-10T11:29:52.724 INFO:tasks.workunit.client.0.smithi144.stdout:4/990: read d14/d67/d84/db3/fee [61089,60509] 0 2024-09-10T11:29:52.727 INFO:tasks.workunit.client.0.smithi144.stdout:4/991: mknod d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d3c/da7/dd7/de4/c160 0 2024-09-10T11:29:52.731 INFO:tasks.workunit.client.0.smithi144.stdout:4/992: write d14/d67/f11f [933112,43631] 0 2024-09-10T11:29:52.739 INFO:tasks.workunit.client.0.smithi144.stdout:4/993: dwrite d14/d1e/d65/d8a/fd9 [0,4194304] 0 2024-09-10T11:29:52.746 INFO:tasks.workunit.client.0.smithi144.stdout:0/795: sync 2024-09-10T11:29:52.746 INFO:tasks.workunit.client.0.smithi144.stdout:2/798: sync 2024-09-10T11:29:52.746 INFO:tasks.workunit.client.0.smithi144.stdout:5/906: sync 2024-09-10T11:29:52.746 INFO:tasks.workunit.client.0.smithi144.stdout:8/826: sync 2024-09-10T11:29:52.747 INFO:tasks.workunit.client.0.smithi144.stdout:2/799: chown d7/de/d70/df1 795650375 1 2024-09-10T11:29:52.747 INFO:tasks.workunit.client.0.smithi144.stdout:6/792: sync 2024-09-10T11:29:52.747 INFO:tasks.workunit.client.0.smithi144.stdout:9/924: sync 2024-09-10T11:29:52.747 INFO:tasks.workunit.client.0.smithi144.stdout:7/885: sync 2024-09-10T11:29:52.747 INFO:tasks.workunit.client.0.smithi144.stdout:0/796: read da/f21 [3198921,81589] 0 2024-09-10T11:29:52.749 INFO:tasks.workunit.client.0.smithi144.stdout:7/886: chown d5/f5a 48848977 1 2024-09-10T11:29:52.749 INFO:tasks.workunit.client.0.smithi144.stdout:2/800: write d7/dcc/dd3/fe2 [597818,113348] 0 2024-09-10T11:29:52.750 INFO:tasks.workunit.client.0.smithi144.stdout:5/907: readlink d4/d5/d18/d51/le8 0 2024-09-10T11:29:52.750 INFO:tasks.workunit.client.0.smithi144.stdout:6/793: read d9/dd/d11/f90 [2300786,102862] 0 2024-09-10T11:29:52.751 INFO:tasks.workunit.client.0.smithi144.stdout:9/925: fdatasync d11/f15 0 2024-09-10T11:29:52.753 INFO:tasks.workunit.client.0.smithi144.stdout:8/827: unlink d0/d9/fb0 0 2024-09-10T11:29:52.757 INFO:tasks.workunit.client.0.smithi144.stdout:7/887: truncate d5/dc/fcc 240155 0 2024-09-10T11:29:52.758 INFO:tasks.workunit.client.0.smithi144.stdout:9/926: rmdir d11/d12/d1b/d2c/d78/dd2 39 2024-09-10T11:29:52.763 INFO:tasks.workunit.client.0.smithi144.stdout:5/908: creat d4/d5/d18/d38/d6d/da6/f119 x:0 0 0 2024-09-10T11:29:52.763 INFO:tasks.workunit.client.0.smithi144.stdout:4/994: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/f77 [0,4194304] 0 2024-09-10T11:29:52.768 INFO:tasks.workunit.client.0.smithi144.stdout:2/801: creat d7/da/d5b/f106 x:0 0 0 2024-09-10T11:29:52.770 INFO:tasks.workunit.client.0.smithi144.stdout:4/995: creat d14/d67/d84/db3/f161 x:0 0 0 2024-09-10T11:29:52.770 INFO:tasks.workunit.client.0.smithi144.stdout:9/927: truncate d11/d36/d2d/f8d 95454 0 2024-09-10T11:29:52.771 INFO:tasks.workunit.client.0.smithi144.stdout:8/828: getdents d0/d9/d23/d111 0 2024-09-10T11:29:52.772 INFO:tasks.workunit.client.0.smithi144.stdout:7/888: creat d5/dc/d14/d40/f124 x:0 0 0 2024-09-10T11:29:52.777 INFO:tasks.workunit.client.0.smithi144.stdout:2/802: dread - d7/da/ddb/ffc zero size 2024-09-10T11:29:52.780 INFO:tasks.workunit.client.0.smithi144.stdout:8/829: symlink d0/d1b/d47/d5e/d76/l122 0 2024-09-10T11:29:52.780 INFO:tasks.workunit.client.0.smithi144.stdout:0/797: dwrite da/f26 [0,4194304] 0 2024-09-10T11:29:52.781 INFO:tasks.workunit.client.0.smithi144.stdout:3/870: sync 2024-09-10T11:29:52.784 INFO:tasks.workunit.client.0.smithi144.stdout:7/889: chown d5/dc/d14/d40/d58/d6a 139480213 1 2024-09-10T11:29:52.787 INFO:tasks.workunit.client.0.smithi144.stdout:0/798: write da/d3a/d51/d5f/f4a [4681660,44665] 0 2024-09-10T11:29:52.787 INFO:tasks.workunit.client.0.smithi144.stdout:3/871: mkdir d4/d28/d67/d96/dcc/d116 0 2024-09-10T11:29:52.787 INFO:tasks.workunit.client.0.smithi144.stdout:5/909: getdents d4/d5/d32/d47/df3/d74/dbe 0 2024-09-10T11:29:52.790 INFO:tasks.workunit.client.0.smithi144.stdout:8/830: chown d0/d9/d23/d37 9618 1 2024-09-10T11:29:52.790 INFO:tasks.workunit.client.0.smithi144.stdout:6/794: dwrite d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/f5a [0,4194304] 0 2024-09-10T11:29:52.793 INFO:tasks.workunit.client.0.smithi144.stdout:4/996: mkdir d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/d35/d4f/d162 0 2024-09-10T11:29:52.794 INFO:tasks.workunit.client.0.smithi144.stdout:9/928: getdents d11/d36/db1/dbb/df0/d123/d144 0 2024-09-10T11:29:52.795 INFO:tasks.workunit.client.0.smithi144.stdout:0/799: fsync da/f21 0 2024-09-10T11:29:52.797 INFO:tasks.workunit.client.0.smithi144.stdout:5/910: read - d4/d20/d22/d75/ddd/fe3 zero size 2024-09-10T11:29:52.798 INFO:tasks.workunit.client.0.smithi144.stdout:7/890: unlink d5/dc/c3a 0 2024-09-10T11:29:52.801 INFO:tasks.workunit.client.0.smithi144.stdout:9/929: symlink d11/d36/db1/dbb/df0/d8f/l146 0 2024-09-10T11:29:52.804 INFO:tasks.workunit.client.0.smithi144.stdout:2/803: dread d7/de/d3d/d4e/dac/fae [0,4194304] 0 2024-09-10T11:29:52.806 INFO:tasks.workunit.client.0.smithi144.stdout:7/891: chown d5/dc/d14/d40/d58/d6a/fa0 164114 1 2024-09-10T11:29:52.806 INFO:tasks.workunit.client.0.smithi144.stdout:6/795: truncate d9/d38/ff3 657218 0 2024-09-10T11:29:52.808 INFO:tasks.workunit.client.0.smithi144.stdout:8/831: dread d0/d9/d1f/fc6 [0,4194304] 0 2024-09-10T11:29:52.809 INFO:tasks.workunit.client.0.smithi144.stdout:7/892: write d5/dc/d14/f50 [3264667,97475] 0 2024-09-10T11:29:52.811 INFO:tasks.workunit.client.0.smithi144.stdout:5/911: dread d4/d5/d18/d38/d6d/da6/ff8 [0,4194304] 0 2024-09-10T11:29:52.811 INFO:tasks.workunit.client.0.smithi144.stdout:8/832: stat d0/d9/d1f/d34/da0/dc4 0 2024-09-10T11:29:52.813 INFO:tasks.workunit.client.0.smithi144.stdout:2/804: symlink d7/dcc/l107 0 2024-09-10T11:29:52.814 INFO:tasks.workunit.client.0.smithi144.stdout:9/930: unlink d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/cba 0 2024-09-10T11:29:52.816 INFO:tasks.workunit.client.0.smithi144.stdout:5/912: truncate d4/d5/d18/d67/f103 776643 0 2024-09-10T11:29:52.816 INFO:tasks.workunit.client.0.smithi144.stdout:2/805: read d7/de/d36/d45/d47/d6c/fa2 [918987,64868] 0 2024-09-10T11:29:52.816 INFO:tasks.workunit.client.0.smithi144.stdout:4/997: write d14/d1e/d22/d34/f3b [4577310,3315] 0 2024-09-10T11:29:52.817 INFO:tasks.workunit.client.0.smithi144.stdout:8/833: dread - d0/d9/d21/d25/d4f/fc8 zero size 2024-09-10T11:29:52.818 INFO:tasks.workunit.client.0.smithi144.stdout:9/931: chown d11/d36/db1/dbb/df0/d8f/dc0/cce 12864 1 2024-09-10T11:29:52.819 INFO:tasks.workunit.client.0.smithi144.stdout:7/893: symlink d5/dc/d12/d10a/l125 0 2024-09-10T11:29:52.820 INFO:tasks.workunit.client.0.smithi144.stdout:6/796: fsync d9/d38/ff3 0 2024-09-10T11:29:52.824 INFO:tasks.workunit.client.0.smithi144.stdout:5/913: mknod d4/d5/d18/d67/d59/c11a 0 2024-09-10T11:29:52.828 INFO:tasks.workunit.client.0.smithi144.stdout:8/834: creat d0/d9/d21/d25/db3/de8/f123 x:0 0 0 2024-09-10T11:29:52.831 INFO:tasks.workunit.client.0.smithi144.stdout:3/872: dwrite d4/d28/d67/d80/f10a [4194304,4194304] 0 2024-09-10T11:29:52.837 INFO:tasks.workunit.client.0.smithi144.stdout:4/998: dwrite d14/d1e/d22/d34/d3e/d40/d4b/d4c/dbe/fed [4194304,4194304] 0 2024-09-10T11:29:52.842 INFO:tasks.workunit.client.0.smithi144.stdout:9/932: rename d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/f4a to d11/d36/db1/dbb/df0/d8f/dc0/f147 0 2024-09-10T11:29:52.844 INFO:tasks.workunit.client.0.smithi144.stdout:0/800: dwrite da/d3a/d4b/d4e/f60 [0,4194304] 0 2024-09-10T11:29:52.848 INFO:tasks.workunit.client.0.smithi144.stdout:6/797: truncate d9/d1f/d2c/d4f/f58 3078535 0 2024-09-10T11:29:52.850 INFO:tasks.workunit.client.0.smithi144.stdout:2/806: getdents d7/de/d5f 0 2024-09-10T11:29:52.852 INFO:tasks.workunit.client.0.smithi144.stdout:3/873: symlink d4/dd/l117 0 2024-09-10T11:29:52.856 INFO:tasks.workunit.client.0.smithi144.stdout:4/999: rmdir d14/d6f 39 2024-09-10T11:29:52.856 INFO:tasks.workunit.client.0.smithi144.stdout:2/807: read d7/de/d3d/fab [4167032,110298] 0 2024-09-10T11:29:52.857 INFO:tasks.workunit.client.0.smithi144.stdout:7/894: dread d5/dc/d12/f13 [0,4194304] 0 2024-09-10T11:29:52.858 INFO:tasks.workunit.client.0.smithi144.stdout:8/835: dwrite d0/d9/d23/d26/fea [0,4194304] 0 2024-09-10T11:29:52.862 INFO:tasks.workunit.client.0.smithi144.stdout:5/914: dwrite d4/d5/d32/daa/ff5 [0,4194304] 0 2024-09-10T11:29:52.862 INFO:tasks.workunit.client.0.smithi144.stdout:6/798: mknod d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d60/c11b 0 2024-09-10T11:29:52.867 INFO:tasks.workunit.client.0.smithi144.stdout:7/895: write d5/dc/def/f11d [1420995,65336] 0 2024-09-10T11:29:52.867 INFO:tasks.workunit.client.0.smithi144.stdout:0/801: chown da/d3a/d8e/dab/dc7/lce 130 1 2024-09-10T11:29:52.869 INFO:tasks.workunit.client.0.smithi144.stdout:3/874: creat d4/d28/db6/f118 x:0 0 0 2024-09-10T11:29:52.875 INFO:tasks.workunit.client.0.smithi144.stdout:2/808: creat d7/de/d36/d45/d8e/d9a/f108 x:0 0 0 2024-09-10T11:29:52.876 INFO:tasks.workunit.client.0.smithi144.stdout:9/933: symlink d11/d36/db1/dbb/l148 0 2024-09-10T11:29:52.877 INFO:tasks.workunit.client.0.smithi144.stdout:8/836: dwrite d0/d9/fe [0,4194304] 0 2024-09-10T11:29:52.878 INFO:tasks.workunit.client.0.smithi144.stdout:5/915: link d4/d20/f43 d4/d20/d22/d5c/f11b 0 2024-09-10T11:29:52.899 INFO:tasks.workunit.client.0.smithi144.stdout:2/809: fsync d7/de/d36/d45/d8e/d9a/f9c 0 2024-09-10T11:29:52.900 INFO:tasks.workunit.client.0.smithi144.stdout:7/896: rename d5/dc/ca6 to d5/dc/d14/d40/d58/c126 0 2024-09-10T11:29:52.900 INFO:tasks.workunit.client.0.smithi144.stdout:8/837: mkdir d0/d9/d21/d25/db3/d124 0 2024-09-10T11:29:52.902 INFO:tasks.workunit.client.0.smithi144.stdout:3/875: read - d4/d28/d67/d96/fde zero size 2024-09-10T11:29:52.904 INFO:tasks.workunit.client.0.smithi144.stdout:0/802: getdents da/d49 0 2024-09-10T11:29:52.906 INFO:tasks.workunit.client.0.smithi144.stdout:9/934: truncate d11/d36/db1/dbb/df0/d8f/f87 493859 0 2024-09-10T11:29:52.906 INFO:tasks.workunit.client.0.smithi144.stdout:2/810: chown d7/d29/f4b 117209 1 2024-09-10T11:29:52.906 INFO:tasks.workunit.client.0.smithi144.stdout:8/838: creat d0/d9/d21/d25/db3/dc2/df2/f125 x:0 0 0 2024-09-10T11:29:52.907 INFO:tasks.workunit.client.0.smithi144.stdout:3/876: mknod d4/d28/d67/d80/d17/d18/c119 0 2024-09-10T11:29:52.908 INFO:tasks.workunit.client.0.smithi144.stdout:2/811: chown d7/da/ddb 611 1 2024-09-10T11:29:52.908 INFO:tasks.workunit.client.0.smithi144.stdout:7/897: getdents d5/dc/d14/d107 0 2024-09-10T11:29:52.912 INFO:tasks.workunit.client.0.smithi144.stdout:3/877: creat d4/d28/d67/d80/d17/d18/d34/d78/f11a x:0 0 0 2024-09-10T11:29:52.913 INFO:tasks.workunit.client.0.smithi144.stdout:2/812: fsync d7/de/f22 0 2024-09-10T11:29:52.916 INFO:tasks.workunit.client.0.smithi144.stdout:3/878: chown d4/dd/c73 448282401 1 2024-09-10T11:29:52.919 INFO:tasks.workunit.client.0.smithi144.stdout:7/898: rmdir d5/dc/d2a/d60/dd6/d98/d102/d115 39 2024-09-10T11:29:52.919 INFO:tasks.workunit.client.0.smithi144.stdout:8/839: dwrite d0/d9/d21/f73 [0,4194304] 0 2024-09-10T11:29:52.921 INFO:tasks.workunit.client.0.smithi144.stdout:3/879: dread - d4/d28/d67/d80/fe7 zero size 2024-09-10T11:29:52.922 INFO:tasks.workunit.client.0.smithi144.stdout:7/899: chown d5/dc/d14/d40/d58/d6e/lda 878455 1 2024-09-10T11:29:52.929 INFO:tasks.workunit.client.0.smithi144.stdout:2/813: dwrite d7/de/f2f [4194304,4194304] 0 2024-09-10T11:29:52.932 INFO:tasks.workunit.client.0.smithi144.stdout:3/880: creat d4/d28/d67/d80/d17/dd0/f11b x:0 0 0 2024-09-10T11:29:52.932 INFO:tasks.workunit.client.0.smithi144.stdout:8/840: dwrite d0/d9/d1f/f91 [0,4194304] 0 2024-09-10T11:29:52.935 INFO:tasks.workunit.client.0.smithi144.stdout:5/916: write d4/d20/d22/d5c/d5d/de6/fda [664170,5142] 0 2024-09-10T11:29:52.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:52 smithi144 ceph-mon[27942]: pgmap v33: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 84 MiB/s rd, 200 MiB/s wr, 443 op/s 2024-09-10T11:29:52.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:52 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:29:52.936 INFO:tasks.workunit.client.0.smithi144.stdout:8/841: fsync d0/f10 0 2024-09-10T11:29:52.941 INFO:tasks.workunit.client.0.smithi144.stdout:8/842: rmdir d0/d9/d21/d25/db3 39 2024-09-10T11:29:52.944 INFO:tasks.workunit.client.0.smithi144.stdout:5/917: creat d4/d5/d32/d47/f11c x:0 0 0 2024-09-10T11:29:52.945 INFO:tasks.workunit.client.0.smithi144.stdout:0/803: dwrite da/d3a/d4b/d61/d6c/f9d [0,4194304] 0 2024-09-10T11:29:52.950 INFO:tasks.workunit.client.0.smithi144.stdout:5/918: symlink d4/d5/d18/d38/d6d/d8b/l11d 0 2024-09-10T11:29:52.957 INFO:tasks.workunit.client.0.smithi144.stdout:0/804: link c6 da/d49/cf7 0 2024-09-10T11:29:52.957 INFO:tasks.workunit.client.0.smithi144.stdout:3/881: dwrite d4/d28/d67/d80/d17/dd0/d98/f10c [0,4194304] 0 2024-09-10T11:29:52.967 INFO:tasks.workunit.client.0.smithi144.stdout:9/935: dwrite d11/d12/d1b/d2c/d78/dd2/ff2 [0,4194304] 0 2024-09-10T11:29:52.970 INFO:tasks.workunit.client.0.smithi144.stdout:8/843: dread d0/d9/d1f/d34/f53 [0,4194304] 0 2024-09-10T11:29:52.974 INFO:tasks.workunit.client.0.smithi144.stdout:5/919: dwrite d4/d5/d18/d67/db0/db8/f118 [0,4194304] 0 2024-09-10T11:29:52.976 INFO:tasks.workunit.client.0.smithi144.stdout:0/805: symlink da/d3a/d4b/d4e/lf8 0 2024-09-10T11:29:52.978 INFO:tasks.workunit.client.0.smithi144.stdout:8/844: stat d0/d9/d21/d25/db3/dc2/dd1/cdb 0 2024-09-10T11:29:52.980 INFO:tasks.workunit.client.0.smithi144.stdout:5/920: readlink d4/d5/d18/d38/d6d/d8b/l112 0 2024-09-10T11:29:52.984 INFO:tasks.workunit.client.0.smithi144.stdout:3/882: write d4/d28/d67/d80/d17/f81 [1299618,128913] 0 2024-09-10T11:29:52.985 INFO:tasks.workunit.client.0.smithi144.stdout:8/845: creat d0/d9/d1f/d34/f126 x:0 0 0 2024-09-10T11:29:52.987 INFO:tasks.workunit.client.0.smithi144.stdout:9/936: rename d11/d12/d1b/df8/fb6 to d11/d36/db1/dbb/df0/f149 0 2024-09-10T11:29:52.987 INFO:tasks.workunit.client.0.smithi144.stdout:7/900: dread d5/dc/d2a/d60/dd6/f74 [0,4194304] 0 2024-09-10T11:29:52.990 INFO:tasks.workunit.client.0.smithi144.stdout:9/937: chown d11/fc5 233062 1 2024-09-10T11:29:52.996 INFO:tasks.workunit.client.0.smithi144.stdout:3/883: getdents d4 0 2024-09-10T11:29:53.001 INFO:tasks.workunit.client.0.smithi144.stdout:8/846: dwrite d0/d9/d23/d26/f69 [4194304,4194304] 0 2024-09-10T11:29:53.001 INFO:tasks.workunit.client.0.smithi144.stdout:9/938: rename d11/d36/db1/dbb/df0/d8f/dab to d11/d12/d1b/d2c/d12b/d14a 0 2024-09-10T11:29:53.007 INFO:tasks.workunit.client.0.smithi144.stdout:5/921: write d4/d20/d24/f92 [3736739,55272] 0 2024-09-10T11:29:53.007 INFO:tasks.workunit.client.0.smithi144.stdout:8/847: read d0/d9/d1f/d34/f53 [503573,49818] 0 2024-09-10T11:29:53.009 INFO:tasks.workunit.client.0.smithi144.stdout:7/901: rename d5/dc/d2a/d60/dd6/d98/d102/d115/l121 to d5/dc/d2a/d60/dd6/d98/dd1/l127 0 2024-09-10T11:29:53.010 INFO:tasks.workunit.client.0.smithi144.stdout:9/939: mkdir d11/d36/d2d/d38/de7/d12d/d14b 0 2024-09-10T11:29:53.011 INFO:tasks.workunit.client.0.smithi144.stdout:7/902: chown d5/dc/d12/l48 2 1 2024-09-10T11:29:53.013 INFO:tasks.workunit.client.0.smithi144.stdout:0/806: dwrite da/d44/f63 [0,4194304] 0 2024-09-10T11:29:53.013 INFO:tasks.workunit.client.0.smithi144.stdout:5/922: creat d4/d20/d22/d5c/d5d/de6/f11e x:0 0 0 2024-09-10T11:29:53.017 INFO:tasks.workunit.client.0.smithi144.stdout:8/848: creat d0/d9/d21/d7e/de5/f127 x:0 0 0 2024-09-10T11:29:53.019 INFO:tasks.workunit.client.0.smithi144.stdout:9/940: rename d11/d36/d2d/df6 to d11/d12/d1b/d2c/d14c 0 2024-09-10T11:29:53.022 INFO:tasks.workunit.client.0.smithi144.stdout:7/903: mkdir d5/dc/d11/d18/d128 0 2024-09-10T11:29:53.025 INFO:tasks.workunit.client.0.smithi144.stdout:3/884: dwrite d4/dd/fe [0,4194304] 0 2024-09-10T11:29:53.027 INFO:tasks.workunit.client.0.smithi144.stdout:5/923: symlink d4/d5/d18/d67/l11f 0 2024-09-10T11:29:53.027 INFO:tasks.workunit.client.0.smithi144.stdout:8/849: chown d0/d9/d23/d26/d7d/f80 1295 1 2024-09-10T11:29:53.028 INFO:tasks.workunit.client.0.smithi144.stdout:9/941: write d11/d36/db1/dbb/df0/d123/f13e [16991,41991] 0 2024-09-10T11:29:53.031 INFO:tasks.workunit.client.0.smithi144.stdout:7/904: rmdir d5/dc/d14/d40/d5e 39 2024-09-10T11:29:53.032 INFO:tasks.workunit.client.0.smithi144.stdout:8/850: chown d0/d9/d1f/f92 111174428 1 2024-09-10T11:29:53.035 INFO:tasks.workunit.client.0.smithi144.stdout:0/807: dwrite da/f2e [4194304,4194304] 0 2024-09-10T11:29:53.038 INFO:tasks.workunit.client.0.smithi144.stdout:5/924: symlink d4/d20/d22/d5c/d113/l120 0 2024-09-10T11:29:53.042 INFO:tasks.workunit.client.0.smithi144.stdout:5/925: stat d4/d5/d18/d38/d8d/c108 0 2024-09-10T11:29:53.045 INFO:tasks.workunit.client.0.smithi144.stdout:8/851: unlink d0/d9/d21/d25/db3/dc2/df2/f125 0 2024-09-10T11:29:53.047 INFO:tasks.workunit.client.0.smithi144.stdout:3/885: dwrite d4/dd/d21/f41 [4194304,4194304] 0 2024-09-10T11:29:53.048 INFO:tasks.workunit.client.0.smithi144.stdout:5/926: symlink d4/d5/d32/d47/df3/d84/l121 0 2024-09-10T11:29:53.049 INFO:tasks.workunit.client.0.smithi144.stdout:8/852: fsync d0/d9/d21/d25/f109 0 2024-09-10T11:29:53.052 INFO:tasks.workunit.client.0.smithi144.stdout:5/927: mkdir d4/d5/d32/daa/d122 0 2024-09-10T11:29:53.054 INFO:tasks.workunit.client.0.smithi144.stdout:8/853: creat d0/d9/d21/d25/db3/f128 x:0 0 0 2024-09-10T11:29:53.055 INFO:tasks.workunit.client.0.smithi144.stdout:9/942: write d11/d36/d2d/d38/dc3/f113 [382683,99097] 0 2024-09-10T11:29:53.060 INFO:tasks.workunit.client.0.smithi144.stdout:7/905: getdents d5/dc/d14/d40/d5e/dc4 0 2024-09-10T11:29:53.064 INFO:tasks.workunit.client.0.smithi144.stdout:3/886: getdents d4/dd/d21 0 2024-09-10T11:29:53.064 INFO:tasks.workunit.client.0.smithi144.stdout:9/943: mknod d11/d36/d2d/d38/d12c/c14d 0 2024-09-10T11:29:53.070 INFO:tasks.workunit.client.0.smithi144.stdout:7/906: mkdir d5/dc/d14/d40/d58/d6e/d7a/de0/d129 0 2024-09-10T11:29:53.070 INFO:tasks.workunit.client.0.smithi144.stdout:8/854: dwrite d0/d9/d1f/d34/d68/f107 [0,4194304] 0 2024-09-10T11:29:53.071 INFO:tasks.workunit.client.0.smithi144.stdout:9/944: read d11/d36/db1/dbb/df0/d123/f13e [35833,920] 0 2024-09-10T11:29:53.072 INFO:tasks.workunit.client.0.smithi144.stdout:0/808: dwrite da/d3a/d4b/d96/fbc [0,4194304] 0 2024-09-10T11:29:53.074 INFO:tasks.workunit.client.0.smithi144.stdout:3/887: creat d4/d28/d67/d96/f11c x:0 0 0 2024-09-10T11:29:53.078 INFO:tasks.workunit.client.0.smithi144.stdout:5/928: dwrite d4/d20/f43 [0,4194304] 0 2024-09-10T11:29:53.078 INFO:tasks.workunit.client.0.smithi144.stdout:8/855: symlink d0/d1b/d47/dfa/l129 0 2024-09-10T11:29:53.079 INFO:tasks.workunit.client.0.smithi144.stdout:0/809: truncate da/d3a/f93 4418438 0 2024-09-10T11:29:53.082 INFO:tasks.workunit.client.0.smithi144.stdout:7/907: rmdir d5/dc/d2a/d60/dd6/d98/df5 0 2024-09-10T11:29:53.085 INFO:tasks.workunit.client.0.smithi144.stdout:0/810: stat da/d3a/d4b/d61/d6c/fdb 0 2024-09-10T11:29:53.086 INFO:tasks.workunit.client.0.smithi144.stdout:5/929: mkdir d4/d5/d18/d67/db0/db8/d123 0 2024-09-10T11:29:53.087 INFO:tasks.workunit.client.0.smithi144.stdout:9/945: creat d11/d36/d2d/d38/de7/d12d/d14b/f14e x:0 0 0 2024-09-10T11:29:53.088 INFO:tasks.workunit.client.0.smithi144.stdout:8/856: creat d0/d9/d23/d26/d7d/dda/f12a x:0 0 0 2024-09-10T11:29:53.092 INFO:tasks.workunit.client.0.smithi144.stdout:3/888: rename d4/d28/d67/d80/d17/d18/d34/d65/d75/f84 to d4/d28/d67/d80/d17/f11d 0 2024-09-10T11:29:53.092 INFO:tasks.workunit.client.0.smithi144.stdout:3/889: stat d4/dd/d21/f8b 0 2024-09-10T11:29:53.094 INFO:tasks.workunit.client.0.smithi144.stdout:9/946: symlink d11/d36/db1/dbb/df0/d8f/l14f 0 2024-09-10T11:29:53.094 INFO:tasks.workunit.client.0.smithi144.stdout:8/857: creat d0/d9/d21/d25/db3/dc2/df2/f12b x:0 0 0 2024-09-10T11:29:53.096 INFO:tasks.workunit.client.0.smithi144.stdout:8/858: dread - d0/d9/d21/d25/db3/f128 zero size 2024-09-10T11:29:53.096 INFO:tasks.workunit.client.0.smithi144.stdout:3/890: creat d4/d28/d67/d80/d17/dd0/d98/f11e x:0 0 0 2024-09-10T11:29:53.097 INFO:tasks.workunit.client.0.smithi144.stdout:7/908: link d5/dc/d14/d40/d58/l71 d5/dc/d2a/d9d/l12a 0 2024-09-10T11:29:53.099 INFO:tasks.workunit.client.0.smithi144.stdout:9/947: creat d11/d36/f150 x:0 0 0 2024-09-10T11:29:53.101 INFO:tasks.workunit.client.0.smithi144.stdout:7/909: unlink d5/dc/d12/f22 0 2024-09-10T11:29:53.102 INFO:tasks.workunit.client.0.smithi144.stdout:3/891: symlink d4/dd/dfd/l11f 0 2024-09-10T11:29:53.103 INFO:tasks.workunit.client.0.smithi144.stdout:8/859: creat d0/d9/d21/d25/db3/d124/f12c x:0 0 0 2024-09-10T11:29:53.104 INFO:tasks.workunit.client.0.smithi144.stdout:0/811: dread f5 [0,4194304] 0 2024-09-10T11:29:53.105 INFO:tasks.workunit.client.0.smithi144.stdout:8/860: fsync d0/d9/d1f/f19 0 2024-09-10T11:29:53.106 INFO:tasks.workunit.client.0.smithi144.stdout:7/910: mknod d5/dc/d14/d40/d58/d6a/dad/c12b 0 2024-09-10T11:29:53.106 INFO:tasks.workunit.client.0.smithi144.stdout:3/892: symlink d4/dd/d21/d2f/l120 0 2024-09-10T11:29:53.107 INFO:tasks.workunit.client.0.smithi144.stdout:0/812: truncate da/d44/d82/fa2 756013 0 2024-09-10T11:29:53.108 INFO:tasks.workunit.client.0.smithi144.stdout:8/861: fsync d0/d9/d21/f77 0 2024-09-10T11:29:53.109 INFO:tasks.workunit.client.0.smithi144.stdout:3/893: rmdir d4/dd 39 2024-09-10T11:29:53.111 INFO:tasks.workunit.client.0.smithi144.stdout:5/930: dread d4/d20/d22/d5c/f100 [0,4194304] 0 2024-09-10T11:29:53.111 INFO:tasks.workunit.client.0.smithi144.stdout:0/813: symlink da/d3a/d94/db1/lf9 0 2024-09-10T11:29:53.113 INFO:tasks.workunit.client.0.smithi144.stdout:7/911: truncate d5/dc/d14/d40/d58/d6a/db2/ff4 344062 0 2024-09-10T11:29:53.114 INFO:tasks.workunit.client.0.smithi144.stdout:7/912: stat d5/dc/d2a/d60/dd6/d98/dd1/f4a 0 2024-09-10T11:29:53.115 INFO:tasks.workunit.client.0.smithi144.stdout:5/931: mkdir d4/d20/d24/d2d/d4b/d124 0 2024-09-10T11:29:53.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:52 smithi178 ceph-mon[33514]: pgmap v33: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 84 MiB/s rd, 200 MiB/s wr, 443 op/s 2024-09-10T11:29:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:52 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:29:53.119 INFO:tasks.workunit.client.0.smithi144.stdout:3/894: chown d4/d28/d67/d80/d1d/cc9 3880 1 2024-09-10T11:29:53.120 INFO:tasks.workunit.client.0.smithi144.stdout:3/895: write f3 [1566511,115888] 0 2024-09-10T11:29:53.122 INFO:tasks.workunit.client.0.smithi144.stdout:7/913: dwrite d5/dc/d12/d2b/dde/f10b [0,4194304] 0 2024-09-10T11:29:53.127 INFO:tasks.workunit.client.0.smithi144.stdout:8/862: truncate d0/d9/d21/f73 2035095 0 2024-09-10T11:29:53.129 INFO:tasks.workunit.client.0.smithi144.stdout:7/914: rename d5/dc/d11/d18/d128 to d5/dc/d2a/d60/dd6/d98/d12c 0 2024-09-10T11:29:53.131 INFO:tasks.workunit.client.0.smithi144.stdout:3/896: write d4/d28/d67/d9f/fc5 [87328,85712] 0 2024-09-10T11:29:53.133 INFO:tasks.workunit.client.0.smithi144.stdout:8/863: readlink d0/d9/l33 0 2024-09-10T11:29:53.134 INFO:tasks.workunit.client.0.smithi144.stdout:0/814: dwrite da/d3a/d4b/d4e/d62/d78/fad [0,4194304] 0 2024-09-10T11:29:53.138 INFO:tasks.workunit.client.0.smithi144.stdout:3/897: chown d4/dd/d21/c26 73 1 2024-09-10T11:29:53.139 INFO:tasks.workunit.client.0.smithi144.stdout:8/864: symlink d0/d1b/d47/d5e/d76/l12d 0 2024-09-10T11:29:53.140 INFO:tasks.workunit.client.0.smithi144.stdout:0/815: write da/d44/d82/d7e/fd6 [861576,117120] 0 2024-09-10T11:29:53.141 INFO:tasks.workunit.client.0.smithi144.stdout:7/915: truncate d5/dc/d12/f52 127165 0 2024-09-10T11:29:53.142 INFO:tasks.workunit.client.0.smithi144.stdout:5/932: dwrite d4/d5/fbb [0,4194304] 0 2024-09-10T11:29:53.142 INFO:tasks.workunit.client.0.smithi144.stdout:8/865: readlink d0/d9/d21/l46 0 2024-09-10T11:29:53.145 INFO:tasks.workunit.client.0.smithi144.stdout:0/816: symlink da/d3a/d94/lfa 0 2024-09-10T11:29:53.150 INFO:tasks.workunit.client.0.smithi144.stdout:5/933: link d4/d20/d22/d5c/f11b d4/d52/db1/f125 0 2024-09-10T11:29:53.150 INFO:tasks.workunit.client.0.smithi144.stdout:0/817: read da/f30 [2359056,102707] 0 2024-09-10T11:29:53.153 INFO:tasks.workunit.client.0.smithi144.stdout:7/916: write d5/dc/d14/d40/d58/fa3 [1244683,64498] 0 2024-09-10T11:29:53.154 INFO:tasks.workunit.client.0.smithi144.stdout:5/934: creat d4/d20/d22/d5c/d5d/de6/f126 x:0 0 0 2024-09-10T11:29:53.157 INFO:tasks.workunit.client.0.smithi144.stdout:0/818: rename da/d3a/d4b/d4e/f88 to da/ffb 0 2024-09-10T11:29:53.157 INFO:tasks.workunit.client.0.smithi144.stdout:8/866: write d0/d9/d1f/d34/f53 [6353,87756] 0 2024-09-10T11:29:53.158 INFO:tasks.workunit.client.0.smithi144.stdout:5/935: mknod d4/d5/d18/d38/d8d/c127 0 2024-09-10T11:29:53.160 INFO:tasks.workunit.client.0.smithi144.stdout:7/917: unlink d5/dc/d14/d40/d5e/dc4/l108 0 2024-09-10T11:29:53.164 INFO:tasks.workunit.client.0.smithi144.stdout:7/918: symlink d5/dc/d12/d2b/l12d 0 2024-09-10T11:29:53.165 INFO:tasks.workunit.client.0.smithi144.stdout:0/819: creat da/d3a/d4b/dbb/ffc x:0 0 0 2024-09-10T11:29:53.165 INFO:tasks.workunit.client.0.smithi144.stdout:3/898: dread d4/dd/d21/f83 [0,4194304] 0 2024-09-10T11:29:53.167 INFO:tasks.workunit.client.0.smithi144.stdout:0/820: symlink da/d3a/d51/lfd 0 2024-09-10T11:29:53.169 INFO:tasks.workunit.client.0.smithi144.stdout:5/936: rmdir d4/d20/d24/d2d/db6 0 2024-09-10T11:29:53.170 INFO:tasks.workunit.client.0.smithi144.stdout:0/821: dread f5 [0,4194304] 0 2024-09-10T11:29:53.171 INFO:tasks.workunit.client.0.smithi144.stdout:7/919: mknod d5/dc/d2a/d60/dd6/c12e 0 2024-09-10T11:29:53.174 INFO:tasks.workunit.client.0.smithi144.stdout:5/937: mknod d4/d20/d24/da4/c128 0 2024-09-10T11:29:53.175 INFO:tasks.workunit.client.0.smithi144.stdout:5/938: fdatasync d4/d20/d22/d5c/d5d/de6/fda 0 2024-09-10T11:29:53.178 INFO:tasks.workunit.client.0.smithi144.stdout:8/867: write d0/d1b/d5a/f6a [595260,35620] 0 2024-09-10T11:29:53.179 INFO:tasks.workunit.client.0.smithi144.stdout:8/868: write d0/d1b/d47/d5e/f64 [4004351,107355] 0 2024-09-10T11:29:53.186 INFO:tasks.workunit.client.0.smithi144.stdout:3/899: write d4/f37 [1678291,77333] 0 2024-09-10T11:29:53.189 INFO:tasks.workunit.client.0.smithi144.stdout:0/822: link da/d3a/d4b/d61/lb5 da/d3a/d8e/dab/lfe 0 2024-09-10T11:29:53.192 INFO:tasks.workunit.client.0.smithi144.stdout:5/939: dread d4/d5/f23 [0,4194304] 0 2024-09-10T11:29:53.192 INFO:tasks.workunit.client.0.smithi144.stdout:3/900: mkdir d4/d28/d67/d80/d1d/d121 0 2024-09-10T11:29:53.193 INFO:tasks.workunit.client.0.smithi144.stdout:8/869: mknod d0/d9/d21/d7e/d85/c12e 0 2024-09-10T11:29:53.194 INFO:tasks.workunit.client.0.smithi144.stdout:7/920: dwrite d5/dc/d14/d40/d5e/d68/fc0 [0,4194304] 0 2024-09-10T11:29:53.195 INFO:tasks.workunit.client.0.smithi144.stdout:0/823: truncate da/d3a/d51/fac 11114 0 2024-09-10T11:29:53.196 INFO:tasks.workunit.client.0.smithi144.stdout:0/824: stat da/d3a/d4b/d4e/lc4 0 2024-09-10T11:29:53.196 INFO:tasks.workunit.client.0.smithi144.stdout:3/901: chown d4/d28/d67/d80/d17/d18/f4c 57 1 2024-09-10T11:29:53.197 INFO:tasks.workunit.client.0.smithi144.stdout:5/940: creat d4/d5/d32/daa/d122/f129 x:0 0 0 2024-09-10T11:29:53.201 INFO:tasks.workunit.client.0.smithi144.stdout:0/825: creat da/d3a/d4b/d4e/d62/d78/fff x:0 0 0 2024-09-10T11:29:53.202 INFO:tasks.workunit.client.0.smithi144.stdout:5/941: write d4/d5/d18/d38/d6d/fcb [2870766,81477] 0 2024-09-10T11:29:53.207 INFO:tasks.workunit.client.0.smithi144.stdout:5/942: symlink d4/d20/d24/dce/l12a 0 2024-09-10T11:29:53.208 INFO:tasks.workunit.client.0.smithi144.stdout:7/921: getdents d5/dc/d2a/d60/dd6/d98/dd1 0 2024-09-10T11:29:53.209 INFO:tasks.workunit.client.0.smithi144.stdout:8/870: dwrite d0/d9/d21/f5c [8388608,4194304] 0 2024-09-10T11:29:53.209 INFO:tasks.workunit.client.0.smithi144.stdout:3/902: dwrite d4/f37 [0,4194304] 0 2024-09-10T11:29:53.212 INFO:tasks.workunit.client.0.smithi144.stdout:3/903: stat d4/d28/d67/d96/fd2 0 2024-09-10T11:29:53.212 INFO:tasks.workunit.client.0.smithi144.stdout:8/871: write d0/d9/d21/d25/f109 [416626,58050] 0 2024-09-10T11:29:53.217 INFO:tasks.workunit.client.0.smithi144.stdout:5/943: symlink d4/d5/d18/d38/d6d/l12b 0 2024-09-10T11:29:53.218 INFO:tasks.workunit.client.0.smithi144.stdout:0/826: dwrite da/d3a/d4b/d4e/f5d [0,4194304] 0 2024-09-10T11:29:53.219 INFO:tasks.workunit.client.0.smithi144.stdout:3/904: chown d4/d28/d67/d80/d17/d18/d34/lcd 6 1 2024-09-10T11:29:53.221 INFO:tasks.workunit.client.0.smithi144.stdout:0/827: fsync da/d3a/d4b/d4e/d62/fdc 0 2024-09-10T11:29:53.225 INFO:tasks.workunit.client.0.smithi144.stdout:8/872: mknod d0/d9/d21/d7e/c12f 0 2024-09-10T11:29:53.226 INFO:tasks.workunit.client.0.smithi144.stdout:0/828: symlink da/d3a/d4b/d96/l100 0 2024-09-10T11:29:53.227 INFO:tasks.workunit.client.0.smithi144.stdout:7/922: getdents d5/dc/d14/d40/d58/d6e/d7a/d11b 0 2024-09-10T11:29:53.230 INFO:tasks.workunit.client.0.smithi144.stdout:0/829: truncate da/f19 2108832 0 2024-09-10T11:29:53.232 INFO:tasks.workunit.client.0.smithi144.stdout:7/923: creat d5/dc/d14/d40/d58/d6a/db2/f12f x:0 0 0 2024-09-10T11:29:53.235 INFO:tasks.workunit.client.0.smithi144.stdout:0/830: unlink da/d3a/d4b/d61/d6c/fdb 0 2024-09-10T11:29:53.235 INFO:tasks.workunit.client.0.smithi144.stdout:7/924: fsync d5/dc/d14/d40/d58/d6a/f92 0 2024-09-10T11:29:53.236 INFO:tasks.workunit.client.0.smithi144.stdout:3/905: dwrite d4/d28/d67/d80/d17/d18/d34/d4b/f53 [0,4194304] 0 2024-09-10T11:29:53.238 INFO:tasks.workunit.client.0.smithi144.stdout:3/906: stat d4/dd/d21/d2f 0 2024-09-10T11:29:53.238 INFO:tasks.workunit.client.0.smithi144.stdout:7/925: chown d5/dc/d14/d40/c9b 52261 1 2024-09-10T11:29:53.241 INFO:tasks.workunit.client.0.smithi144.stdout:0/831: truncate da/d3a/d4b/d96/fa3 361220 0 2024-09-10T11:29:53.242 INFO:tasks.workunit.client.0.smithi144.stdout:3/907: symlink d4/d28/d67/d80/d17/l122 0 2024-09-10T11:29:53.243 INFO:tasks.workunit.client.0.smithi144.stdout:5/944: dread d4/d5/d56/f9e [0,4194304] 0 2024-09-10T11:29:53.243 INFO:tasks.workunit.client.0.smithi144.stdout:0/832: dread - da/d3a/d51/f9f zero size 2024-09-10T11:29:53.244 INFO:tasks.workunit.client.0.smithi144.stdout:7/926: truncate d5/dc/d2a/d60/dd6/d98/dd1/f4d 7154232 0 2024-09-10T11:29:53.245 INFO:tasks.workunit.client.0.smithi144.stdout:8/873: dwrite d0/d9/d23/f50 [4194304,4194304] 0 2024-09-10T11:29:53.247 INFO:tasks.workunit.client.0.smithi144.stdout:8/874: fdatasync d0/d9/d1f/f19 0 2024-09-10T11:29:53.253 INFO:tasks.workunit.client.0.smithi144.stdout:5/945: creat d4/d5/f12c x:0 0 0 2024-09-10T11:29:53.253 INFO:tasks.workunit.client.0.smithi144.stdout:8/875: write d0/d9/d1f/fc6 [1746868,39687] 0 2024-09-10T11:29:53.253 INFO:tasks.workunit.client.0.smithi144.stdout:3/908: creat d4/d28/d67/d80/d17/d18/d34/f123 x:0 0 0 2024-09-10T11:29:53.253 INFO:tasks.workunit.client.0.smithi144.stdout:5/946: write d4/d5/d18/d67/f103 [404730,22154] 0 2024-09-10T11:29:53.258 INFO:tasks.workunit.client.0.smithi144.stdout:0/833: dwrite da/d3a/d4b/d4e/d90/fa6 [0,4194304] 0 2024-09-10T11:29:53.261 INFO:tasks.workunit.client.0.smithi144.stdout:3/909: read d4/d28/d67/d80/d17/f81 [316940,109695] 0 2024-09-10T11:29:53.263 INFO:tasks.workunit.client.0.smithi144.stdout:5/947: rename d4/d20/d22/d5c/d5d/de6/f85 to d4/d5/d18/d67/db0/f12d 0 2024-09-10T11:29:53.263 INFO:tasks.workunit.client.0.smithi144.stdout:0/834: mknod da/d3a/d4b/c101 0 2024-09-10T11:29:53.265 INFO:tasks.workunit.client.0.smithi144.stdout:3/910: write d4/d28/d67/d80/f6b [684702,69436] 0 2024-09-10T11:29:53.270 INFO:tasks.workunit.client.0.smithi144.stdout:3/911: dread d4/dd/d21/f41 [4194304,4194304] 0 2024-09-10T11:29:53.273 INFO:tasks.workunit.client.0.smithi144.stdout:3/912: mknod d4/d28/d67/d80/d17/d18/d34/d65/dee/c124 0 2024-09-10T11:29:53.276 INFO:tasks.workunit.client.0.smithi144.stdout:3/913: unlink d4/d28/d67/d80/d17/d18/l79 0 2024-09-10T11:29:53.283 INFO:tasks.workunit.client.0.smithi144.stdout:3/914: dwrite d4/d28/f87 [0,4194304] 0 2024-09-10T11:29:53.283 INFO:tasks.workunit.client.0.smithi144.stdout:0/835: fsync da/f19 0 2024-09-10T11:29:53.286 INFO:tasks.workunit.client.0.smithi144.stdout:0/836: write da/d3a/d94/db1/dcc/fa8 [2374275,51654] 0 2024-09-10T11:29:53.294 INFO:tasks.workunit.client.0.smithi144.stdout:8/876: write d0/d9/d21/d25/db3/dc2/fde [570082,8355] 0 2024-09-10T11:29:53.294 INFO:tasks.workunit.client.0.smithi144.stdout:0/837: getdents da/d44/d82/d8c/deb 0 2024-09-10T11:29:53.297 INFO:tasks.workunit.client.0.smithi144.stdout:5/948: write d4/d5/d18/d67/d59/f5e [471941,42172] 0 2024-09-10T11:29:53.297 INFO:tasks.workunit.client.0.smithi144.stdout:8/877: read d0/fc [1775929,95323] 0 2024-09-10T11:29:53.300 INFO:tasks.workunit.client.0.smithi144.stdout:8/878: symlink d0/d9/d1f/d34/l130 0 2024-09-10T11:29:53.348 INFO:tasks.workunit.client.0.smithi144.stdout:5/949: symlink d4/d5/d32/d47/df3/l12e 0 2024-09-10T11:29:53.348 INFO:tasks.workunit.client.0.smithi144.stdout:0/838: dwrite da/f21 [0,4194304] 0 2024-09-10T11:29:53.396 INFO:tasks.workunit.client.0.smithi144.stdout:3/915: write d4/d28/d67/d80/d17/d18/d34/d4b/f9c [433317,5120] 0 2024-09-10T11:29:53.399 INFO:tasks.workunit.client.0.smithi144.stdout:8/879: write d0/d9/d21/d25/db3/dc2/df2/f12b [179278,43657] 0 2024-09-10T11:29:53.401 INFO:tasks.workunit.client.0.smithi144.stdout:5/950: write d4/d20/d24/d2d/d4b/f10e [41002,115636] 0 2024-09-10T11:29:53.402 INFO:tasks.workunit.client.0.smithi144.stdout:3/916: rmdir d4/d28/d67/d80/d17/d18/d34/d78 39 2024-09-10T11:29:53.406 INFO:tasks.workunit.client.0.smithi144.stdout:8/880: creat d0/d9/d21/d25/d4f/d93/f131 x:0 0 0 2024-09-10T11:29:53.410 INFO:tasks.workunit.client.0.smithi144.stdout:3/917: creat d4/d28/d67/d80/d17/d18/d34/d65/d75/d8e/f125 x:0 0 0 2024-09-10T11:29:53.414 INFO:tasks.workunit.client.0.smithi144.stdout:3/918: rename d4/d1a/df8/l106 to d4/d28/d67/l126 0 2024-09-10T11:29:53.417 INFO:tasks.workunit.client.0.smithi144.stdout:0/839: dread da/f26 [4194304,4194304] 0 2024-09-10T11:29:53.419 INFO:tasks.workunit.client.0.smithi144.stdout:0/840: chown da/d3a/c52 130009 1 2024-09-10T11:29:53.422 INFO:tasks.workunit.client.0.smithi144.stdout:3/919: rename d4/d28/d67/d80/d17/c72 to d4/d28/d67/d80/d17/d18/d34/d78/c127 0 2024-09-10T11:29:53.423 INFO:tasks.workunit.client.0.smithi144.stdout:0/841: creat da/d3a/d8e/dab/dc7/f102 x:0 0 0 2024-09-10T11:29:53.423 INFO:tasks.workunit.client.0.smithi144.stdout:8/881: dread d0/d9/d23/f89 [0,4194304] 0 2024-09-10T11:29:53.424 INFO:tasks.workunit.client.0.smithi144.stdout:3/920: readlink d4/d1a/l6c 0 2024-09-10T11:29:53.425 INFO:tasks.workunit.client.0.smithi144.stdout:8/882: fdatasync d0/d9/d1f/d34/f61 0 2024-09-10T11:29:53.430 INFO:tasks.workunit.client.0.smithi144.stdout:0/842: dwrite da/d3a/d4b/fb7 [0,4194304] 0 2024-09-10T11:29:53.431 INFO:tasks.workunit.client.0.smithi144.stdout:0/843: fdatasync da/f2e 0 2024-09-10T11:29:53.437 INFO:tasks.workunit.client.0.smithi144.stdout:0/844: mkdir da/d3a/d51/d103 0 2024-09-10T11:29:53.439 INFO:tasks.workunit.client.0.smithi144.stdout:5/951: dwrite d4/d5/d32/fc4 [0,4194304] 0 2024-09-10T11:29:53.440 INFO:tasks.workunit.client.0.smithi144.stdout:8/883: write d0/d9/d1f/d34/f61 [1361472,96793] 0 2024-09-10T11:29:53.441 INFO:tasks.workunit.client.0.smithi144.stdout:3/921: truncate d4/d28/d67/d80/f10a 7048220 0 2024-09-10T11:29:53.442 INFO:tasks.workunit.client.0.smithi144.stdout:0/845: truncate da/d3a/d4b/d61/d6c/fa1 3600637 0 2024-09-10T11:29:53.442 INFO:tasks.workunit.client.0.smithi144.stdout:3/922: stat d4/d28/d67/d96/dcc/dec 0 2024-09-10T11:29:53.444 INFO:tasks.workunit.client.0.smithi144.stdout:0/846: dread - da/d3a/d4b/d4e/d62/fdc zero size 2024-09-10T11:29:53.447 INFO:tasks.workunit.client.0.smithi144.stdout:3/923: symlink d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/l128 0 2024-09-10T11:29:53.449 INFO:tasks.workunit.client.0.smithi144.stdout:8/884: link d0/d9/d23/d26/ld0 d0/d1b/d47/d5e/d76/l132 0 2024-09-10T11:29:53.450 INFO:tasks.workunit.client.0.smithi144.stdout:3/924: creat d4/d28/d67/d96/dcc/f129 x:0 0 0 2024-09-10T11:29:53.453 INFO:tasks.workunit.client.0.smithi144.stdout:5/952: write d4/d20/d24/d2d/d4b/fee [322422,64270] 0 2024-09-10T11:29:53.453 INFO:tasks.workunit.client.0.smithi144.stdout:3/925: mknod d4/dd/d21/d2f/c12a 0 2024-09-10T11:29:53.457 INFO:tasks.workunit.client.0.smithi144.stdout:8/885: dwrite d0/d9/d21/d25/db3/f103 [0,4194304] 0 2024-09-10T11:29:53.457 INFO:tasks.workunit.client.0.smithi144.stdout:5/953: rmdir d4/d20/d22/d5c/d5d/d64 39 2024-09-10T11:29:53.457 INFO:tasks.workunit.client.0.smithi144.stdout:3/926: chown d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/f8c 328 1 2024-09-10T11:29:53.462 INFO:tasks.workunit.client.0.smithi144.stdout:5/954: symlink d4/l12f 0 2024-09-10T11:29:53.466 INFO:tasks.workunit.client.0.smithi144.stdout:8/886: dwrite d0/d9/d1f/d34/f53 [0,4194304] 0 2024-09-10T11:29:53.468 INFO:tasks.workunit.client.0.smithi144.stdout:3/927: getdents d4/d28/d67 0 2024-09-10T11:29:53.468 INFO:tasks.workunit.client.0.smithi144.stdout:5/955: fsync d4/d5/d32/f5f 0 2024-09-10T11:29:53.471 INFO:tasks.workunit.client.0.smithi144.stdout:8/887: creat d0/d9/d1f/d34/da0/dc4/f133 x:0 0 0 2024-09-10T11:29:53.472 INFO:tasks.workunit.client.0.smithi144.stdout:8/888: stat d0/d9/d23/d26/f4e 0 2024-09-10T11:29:53.479 INFO:tasks.workunit.client.0.smithi144.stdout:8/889: link d0/d9/d23/f50 d0/d9/d1f/d34/f134 0 2024-09-10T11:29:53.482 INFO:tasks.workunit.client.0.smithi144.stdout:5/956: dwrite d4/d20/d22/d5c/d5d/d64/fd4 [0,4194304] 0 2024-09-10T11:29:53.484 INFO:tasks.workunit.client.0.smithi144.stdout:8/890: creat d0/d9/d21/d25/f135 x:0 0 0 2024-09-10T11:29:53.487 INFO:tasks.workunit.client.0.smithi144.stdout:5/957: fdatasync d4/d52/ff9 0 2024-09-10T11:29:53.488 INFO:tasks.workunit.client.0.smithi144.stdout:3/928: dwrite d4/d28/d67/d80/d17/dd0/d98/d9a/fc2 [0,4194304] 0 2024-09-10T11:29:53.489 INFO:tasks.workunit.client.0.smithi144.stdout:3/929: chown d4/d28/d67/d96/dcc 56634 1 2024-09-10T11:29:53.490 INFO:tasks.workunit.client.0.smithi144.stdout:8/891: symlink d0/d9/d23/d26/d7d/dda/l136 0 2024-09-10T11:29:53.491 INFO:tasks.workunit.client.0.smithi144.stdout:5/958: write d4/d5/d32/d47/f11c [214747,82285] 0 2024-09-10T11:29:53.493 INFO:tasks.workunit.client.0.smithi144.stdout:8/892: unlink d0/d9/d23/l44 0 2024-09-10T11:29:53.495 INFO:tasks.workunit.client.0.smithi144.stdout:3/930: creat d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/f12b x:0 0 0 2024-09-10T11:29:53.496 INFO:tasks.workunit.client.0.smithi144.stdout:5/959: creat d4/d5/d56/d8a/dc7/f130 x:0 0 0 2024-09-10T11:29:53.497 INFO:tasks.workunit.client.0.smithi144.stdout:8/893: symlink d0/d9/d23/d37/l137 0 2024-09-10T11:29:53.501 INFO:tasks.workunit.client.0.smithi144.stdout:8/894: creat d0/d9/d21/d7e/d11b/f138 x:0 0 0 2024-09-10T11:29:53.502 INFO:tasks.workunit.client.0.smithi144.stdout:8/895: fdatasync d0/d9/d21/d25/db3/dc2/df2/f12b 0 2024-09-10T11:29:53.504 INFO:tasks.workunit.client.0.smithi144.stdout:8/896: fsync d0/d9/d21/d25/d4f/d93/f102 0 2024-09-10T11:29:53.517 INFO:tasks.workunit.client.0.smithi144.stdout:5/960: dread d4/d20/d24/d2d/d4b/f4e [0,4194304] 0 2024-09-10T11:29:53.519 INFO:tasks.workunit.client.0.smithi144.stdout:3/931: dwrite d4/d28/d67/d80/d17/d18/d34/d65/d75/fb0 [4194304,4194304] 0 2024-09-10T11:29:53.520 INFO:tasks.workunit.client.0.smithi144.stdout:3/932: readlink d4/d28/l112 0 2024-09-10T11:29:53.520 INFO:tasks.workunit.client.0.smithi144.stdout:8/897: dwrite d0/d9/d21/d25/db3/dc2/dcd/f108 [0,4194304] 0 2024-09-10T11:29:53.522 INFO:tasks.workunit.client.0.smithi144.stdout:5/961: creat d4/d5/d18/d38/d8d/f131 x:0 0 0 2024-09-10T11:29:53.526 INFO:tasks.workunit.client.0.smithi144.stdout:8/898: fdatasync d0/d9/d21/d7e/d85/fb9 0 2024-09-10T11:29:53.528 INFO:tasks.workunit.client.0.smithi144.stdout:8/899: truncate d0/d9/d1f/f10d 103069 0 2024-09-10T11:29:53.531 INFO:tasks.workunit.client.0.smithi144.stdout:8/900: readlink d0/d9/d21/d7e/de5/l11c 0 2024-09-10T11:29:53.532 INFO:tasks.workunit.client.0.smithi144.stdout:5/962: dwrite d4/d5/d18/d38/d6d/da6/f119 [0,4194304] 0 2024-09-10T11:29:53.534 INFO:tasks.workunit.client.0.smithi144.stdout:8/901: readlink d0/d9/d21/laf 0 2024-09-10T11:29:53.536 INFO:tasks.workunit.client.0.smithi144.stdout:5/963: creat d4/d5/d32/daa/f132 x:0 0 0 2024-09-10T11:29:53.539 INFO:tasks.workunit.client.0.smithi144.stdout:3/933: write d4/dd/d21/f86 [1728270,92570] 0 2024-09-10T11:29:53.542 INFO:tasks.workunit.client.0.smithi144.stdout:5/964: fdatasync d4/d20/d24/d2d/fc9 0 2024-09-10T11:29:53.544 INFO:tasks.workunit.client.0.smithi144.stdout:8/902: creat d0/d9/d1f/d34/da0/dc4/d112/f139 x:0 0 0 2024-09-10T11:29:53.544 INFO:tasks.workunit.client.0.smithi144.stdout:3/934: mknod d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/c12c 0 2024-09-10T11:29:53.545 INFO:tasks.workunit.client.0.smithi144.stdout:8/903: stat d0/d9/d21/d7e/c12f 0 2024-09-10T11:29:53.546 INFO:tasks.workunit.client.0.smithi144.stdout:5/965: creat d4/d5/d32/d47/f133 x:0 0 0 2024-09-10T11:29:53.547 INFO:tasks.workunit.client.0.smithi144.stdout:3/935: rmdir d4/d28/d67/d80/d17/d18/d34/d4b 39 2024-09-10T11:29:53.550 INFO:tasks.workunit.client.0.smithi144.stdout:5/966: creat d4/d20/d22/d5c/d5d/d64/dc2/f134 x:0 0 0 2024-09-10T11:29:53.552 INFO:tasks.workunit.client.0.smithi144.stdout:3/936: symlink d4/d28/d67/d80/d17/dd0/d98/l12d 0 2024-09-10T11:29:53.558 INFO:tasks.workunit.client.0.smithi144.stdout:8/904: write d0/d1b/d47/d5e/f64 [4343196,32308] 0 2024-09-10T11:29:53.558 INFO:tasks.workunit.client.0.smithi144.stdout:3/937: write d4/d28/d67/d80/d17/d18/f6d [3477702,106884] 0 2024-09-10T11:29:53.558 INFO:tasks.workunit.client.0.smithi144.stdout:5/967: dread d4/d20/d22/d5c/f11b [0,4194304] 0 2024-09-10T11:29:53.562 INFO:tasks.workunit.client.0.smithi144.stdout:8/905: getdents d0/d9/d1f/d34/d68 0 2024-09-10T11:29:53.569 INFO:tasks.workunit.client.0.smithi144.stdout:5/968: dwrite d4/fa [0,4194304] 0 2024-09-10T11:29:53.571 INFO:tasks.workunit.client.0.smithi144.stdout:5/969: readlink d4/d5/l19 0 2024-09-10T11:29:53.578 INFO:tasks.workunit.client.0.smithi144.stdout:3/938: dwrite d4/d28/d67/d80/d17/d18/d34/f5c [0,4194304] 0 2024-09-10T11:29:53.580 INFO:tasks.workunit.client.0.smithi144.stdout:8/906: dwrite d0/d9/d1f/d34/d68/fd7 [0,4194304] 0 2024-09-10T11:29:53.583 INFO:tasks.workunit.client.0.smithi144.stdout:8/907: write d0/d9/d21/d25/db3/dc2/fde [564452,125325] 0 2024-09-10T11:29:53.588 INFO:tasks.workunit.client.0.smithi144.stdout:3/939: dread d4/d28/d67/d80/d17/d18/f2e [4194304,4194304] 0 2024-09-10T11:29:53.590 INFO:tasks.workunit.client.0.smithi144.stdout:3/940: write d4/d28/d67/d80/d17/dd0/d98/d9a/fc2 [2148460,76192] 0 2024-09-10T11:29:53.595 INFO:tasks.workunit.client.0.smithi144.stdout:3/941: mkdir d4/d28/d67/d80/d17/d18/d34/d12e 0 2024-09-10T11:29:53.595 INFO:tasks.workunit.client.0.smithi144.stdout:3/942: stat f2 0 2024-09-10T11:29:53.599 INFO:tasks.workunit.client.0.smithi144.stdout:3/943: creat d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/f12f x:0 0 0 2024-09-10T11:29:53.603 INFO:tasks.workunit.client.0.smithi144.stdout:3/944: mknod d4/d28/d67/d96/dcc/d116/c130 0 2024-09-10T11:29:53.607 INFO:tasks.workunit.client.0.smithi144.stdout:8/908: dwrite d0/fc [4194304,4194304] 0 2024-09-10T11:29:53.608 INFO:tasks.workunit.client.0.smithi144.stdout:5/970: dread d4/d52/fad [0,4194304] 0 2024-09-10T11:29:53.615 INFO:tasks.workunit.client.0.smithi144.stdout:8/909: creat d0/d1b/d5a/d8d/d115/d11a/f13a x:0 0 0 2024-09-10T11:29:53.616 INFO:tasks.workunit.client.0.smithi144.stdout:8/910: write d0/d9/d21/d25/d4f/d93/f102 [2417577,58526] 0 2024-09-10T11:29:53.620 INFO:tasks.workunit.client.0.smithi144.stdout:3/945: dwrite d4/f58 [0,4194304] 0 2024-09-10T11:29:53.622 INFO:tasks.workunit.client.0.smithi144.stdout:8/911: mknod d0/d9/d1f/d34/da0/dc4/d112/c13b 0 2024-09-10T11:29:53.625 INFO:tasks.workunit.client.0.smithi144.stdout:8/912: readlink d0/d9/d21/laf 0 2024-09-10T11:29:53.626 INFO:tasks.workunit.client.0.smithi144.stdout:3/946: fdatasync d4/d28/d67/d80/d17/dd0/faa 0 2024-09-10T11:29:53.627 INFO:tasks.workunit.client.0.smithi144.stdout:3/947: symlink d4/d28/db6/l131 0 2024-09-10T11:29:53.631 INFO:tasks.workunit.client.0.smithi144.stdout:8/913: rename d0/d1b/d5a/l10a to d0/d1b/d5a/d8d/d115/l13c 0 2024-09-10T11:29:53.631 INFO:tasks.workunit.client.0.smithi144.stdout:5/971: dwrite d4/d5/d18/d38/fc5 [0,4194304] 0 2024-09-10T11:29:53.636 INFO:tasks.workunit.client.0.smithi144.stdout:5/972: symlink d4/d5/d32/daa/d122/l135 0 2024-09-10T11:29:53.639 INFO:tasks.workunit.client.0.smithi144.stdout:8/914: truncate d0/d9/d21/d25/d4f/fc8 129704 0 2024-09-10T11:29:53.640 INFO:tasks.workunit.client.0.smithi144.stdout:5/973: truncate d4/d20/d22/d75/ddd/fe3 332383 0 2024-09-10T11:29:53.648 INFO:tasks.workunit.client.0.smithi144.stdout:3/948: write d4/d28/d67/d80/d17/d18/d34/d65/d75/fda [339595,64851] 0 2024-09-10T11:29:53.650 INFO:tasks.workunit.client.0.smithi144.stdout:3/949: read d4/d28/d67/d80/d17/f109 [2915663,24481] 0 2024-09-10T11:29:53.650 INFO:tasks.workunit.client.0.smithi144.stdout:8/915: fsync d0/d9/d1f/f10d 0 2024-09-10T11:29:53.651 INFO:tasks.workunit.client.0.smithi144.stdout:5/974: truncate d4/d5/d18/d38/fc5 3551468 0 2024-09-10T11:29:53.653 INFO:tasks.workunit.client.0.smithi144.stdout:3/950: creat d4/d28/d67/d9f/f132 x:0 0 0 2024-09-10T11:29:53.655 INFO:tasks.workunit.client.0.smithi144.stdout:8/916: creat d0/d9/d21/d25/db3/de8/f13d x:0 0 0 2024-09-10T11:29:53.656 INFO:tasks.workunit.client.0.smithi144.stdout:3/951: write d4/d28/d67/d80/d17/d18/d34/d65/d75/fb0 [319052,50542] 0 2024-09-10T11:29:53.658 INFO:tasks.workunit.client.0.smithi144.stdout:3/952: fsync d4/d28/d67/d80/d17/d18/f104 0 2024-09-10T11:29:53.659 INFO:tasks.workunit.client.0.smithi144.stdout:8/917: creat d0/d9/d23/d111/f13e x:0 0 0 2024-09-10T11:29:53.664 INFO:tasks.workunit.client.0.smithi144.stdout:8/918: fdatasync d0/d9/d23/d37/f3a 0 2024-09-10T11:29:53.665 INFO:tasks.workunit.client.0.smithi144.stdout:5/975: dwrite d4/d52/f69 [0,4194304] 0 2024-09-10T11:29:53.665 INFO:tasks.workunit.client.0.smithi144.stdout:3/953: symlink d4/d28/dc1/df2/l133 0 2024-09-10T11:29:53.671 INFO:tasks.workunit.client.0.smithi144.stdout:8/919: creat d0/d1b/d47/d5e/d76/de1/f13f x:0 0 0 2024-09-10T11:29:53.674 INFO:tasks.workunit.client.0.smithi144.stdout:3/954: creat d4/d28/d67/d80/d17/d18/d34/d12e/f134 x:0 0 0 2024-09-10T11:29:53.677 INFO:tasks.workunit.client.0.smithi144.stdout:8/920: rename d0/d9/d1f/f95 to d0/d1b/d47/d94/f140 0 2024-09-10T11:29:53.678 INFO:tasks.workunit.client.0.smithi144.stdout:5/976: creat d4/d5/d18/d67/f136 x:0 0 0 2024-09-10T11:29:53.678 INFO:tasks.workunit.client.0.smithi144.stdout:8/921: stat d0/f10 0 2024-09-10T11:29:53.679 INFO:tasks.workunit.client.0.smithi144.stdout:5/977: readlink d4/d20/d24/d2d/d4b/l4c 0 2024-09-10T11:29:53.681 INFO:tasks.workunit.client.0.smithi144.stdout:5/978: creat d4/d20/d24/dce/f137 x:0 0 0 2024-09-10T11:29:53.683 INFO:tasks.workunit.client.0.smithi144.stdout:8/922: getdents d0/d9/d21/d25/db3/d124 0 2024-09-10T11:29:53.689 INFO:tasks.workunit.client.0.smithi144.stdout:8/923: link d0/d9/d21/d7e/de5/f127 d0/d9/d21/d25/db3/d124/f141 0 2024-09-10T11:29:53.691 INFO:tasks.workunit.client.0.smithi144.stdout:8/924: chown d0/d9/d21/d25/d4f/d93/f131 29509020 1 2024-09-10T11:29:53.691 INFO:tasks.workunit.client.0.smithi144.stdout:5/979: write d4/d5/d18/d38/d8d/fac [635989,113921] 0 2024-09-10T11:29:53.694 INFO:tasks.workunit.client.0.smithi144.stdout:8/925: fdatasync d0/d1b/d47/d94/f9d 0 2024-09-10T11:29:53.696 INFO:tasks.workunit.client.0.smithi144.stdout:3/955: dread d4/d28/d67/d80/d17/f11d [0,4194304] 0 2024-09-10T11:29:53.696 INFO:tasks.workunit.client.0.smithi144.stdout:8/926: chown d0/d9/d21/d25/db3/d124/f12c 181 1 2024-09-10T11:29:53.698 INFO:tasks.workunit.client.0.smithi144.stdout:5/980: link d4/d20/d24/c86 d4/d5/d18/d67/db0/c138 0 2024-09-10T11:29:53.699 INFO:tasks.workunit.client.0.smithi144.stdout:5/981: read - d4/d20/d22/d75/ddd/f115 zero size 2024-09-10T11:29:53.705 INFO:tasks.workunit.client.0.smithi144.stdout:3/956: getdents d4/d28/d67/d80/d17/d18/d34 0 2024-09-10T11:29:53.708 INFO:tasks.workunit.client.0.smithi144.stdout:5/982: dread d4/d20/d24/d2d/d4b/fa7 [0,4194304] 0 2024-09-10T11:29:53.709 INFO:tasks.workunit.client.0.smithi144.stdout:8/927: dwrite d0/d9/d21/d25/d4f/f6f [0,4194304] 0 2024-09-10T11:29:53.709 INFO:tasks.workunit.client.0.smithi144.stdout:5/983: chown d4/d5/d18/d67/d59 103410813 1 2024-09-10T11:29:53.712 INFO:tasks.workunit.client.0.smithi144.stdout:3/957: dread d4/d28/d67/d80/d17/dd0/fae [0,4194304] 0 2024-09-10T11:29:53.712 INFO:tasks.workunit.client.0.smithi144.stdout:5/984: write d4/d5/d32/d47/f71 [176657,59548] 0 2024-09-10T11:29:53.712 INFO:tasks.workunit.client.0.smithi144.stdout:8/928: write d0/d1b/d47/d5e/ffc [678492,80247] 0 2024-09-10T11:29:53.715 INFO:tasks.workunit.client.0.smithi144.stdout:5/985: creat d4/d5/d56/f139 x:0 0 0 2024-09-10T11:29:53.717 INFO:tasks.workunit.client.0.smithi144.stdout:3/958: unlink d4/d28/d67/d80/d17/d18/d34/d65/d75/ff6 0 2024-09-10T11:29:53.718 INFO:tasks.workunit.client.0.smithi144.stdout:8/929: mknod d0/d1b/d47/d5e/d76/de1/c142 0 2024-09-10T11:29:53.722 INFO:tasks.workunit.client.0.smithi144.stdout:3/959: symlink d4/d28/d67/d96/dcc/l135 0 2024-09-10T11:29:53.722 INFO:tasks.workunit.client.0.smithi144.stdout:8/930: creat d0/d9/d1f/d34/f143 x:0 0 0 2024-09-10T11:29:53.724 INFO:tasks.workunit.client.0.smithi144.stdout:3/960: mknod d4/d28/d67/d80/d17/d18/c136 0 2024-09-10T11:29:53.726 INFO:tasks.workunit.client.0.smithi144.stdout:3/961: write d4/dd/d21/f83 [3014219,130594] 0 2024-09-10T11:29:53.727 INFO:tasks.workunit.client.0.smithi144.stdout:3/962: fdatasync d4/d1a/f44 0 2024-09-10T11:29:53.728 INFO:tasks.workunit.client.0.smithi144.stdout:5/986: write d4/d5/d32/d47/df3/d84/fab [235535,51297] 0 2024-09-10T11:29:53.731 INFO:tasks.workunit.client.0.smithi144.stdout:3/963: dread d4/d28/d67/d80/f10a [4194304,4194304] 0 2024-09-10T11:29:53.732 INFO:tasks.workunit.client.0.smithi144.stdout:5/987: creat d4/d20/d22/d5c/d5d/d64/dc2/f13a x:0 0 0 2024-09-10T11:29:53.732 INFO:tasks.workunit.client.0.smithi144.stdout:3/964: readlink d4/d28/d67/d80/d17/l52 0 2024-09-10T11:29:53.733 INFO:tasks.workunit.client.0.smithi144.stdout:5/988: read - d4/d5/d18/d38/d8d/f131 zero size 2024-09-10T11:29:53.734 INFO:tasks.workunit.client.0.smithi144.stdout:3/965: dread - d4/d28/d67/d80/d17/d18/d34/d78/f11a zero size 2024-09-10T11:29:53.737 INFO:tasks.workunit.client.0.smithi144.stdout:5/989: unlink d4/d20/d22/d5c/d5d/d64/dc2/f134 0 2024-09-10T11:29:53.739 INFO:tasks.workunit.client.0.smithi144.stdout:5/990: write d4/d5/d32/d47/df3/d84/fab [1399633,86724] 0 2024-09-10T11:29:53.741 INFO:tasks.workunit.client.0.smithi144.stdout:3/966: dwrite d4/dd/fa9 [0,4194304] 0 2024-09-10T11:29:53.742 INFO:tasks.workunit.client.0.smithi144.stdout:8/931: dread d0/d9/d23/d26/d7d/f81 [0,4194304] 0 2024-09-10T11:29:53.745 INFO:tasks.workunit.client.0.smithi144.stdout:8/932: creat d0/d9/d21/d7e/de5/f144 x:0 0 0 2024-09-10T11:29:53.747 INFO:tasks.workunit.client.0.smithi144.stdout:8/933: chown d0/d9/d21/l55 22 1 2024-09-10T11:29:53.748 INFO:tasks.workunit.client.0.smithi144.stdout:3/967: rmdir d4/d28/dc1/dc4/ded 0 2024-09-10T11:29:53.748 INFO:tasks.workunit.client.0.smithi144.stdout:5/991: link d4/d5/d32/d47/df3/d84/l121 d4/d5/d18/d67/db0/l13b 0 2024-09-10T11:29:53.749 INFO:tasks.workunit.client.0.smithi144.stdout:3/968: rmdir d4/dd/d21/d2f 39 2024-09-10T11:29:53.762 INFO:tasks.workunit.client.0.smithi144.stdout:5/992: truncate d4/d52/f69 1911521 0 2024-09-10T11:29:53.762 INFO:tasks.workunit.client.0.smithi144.stdout:5/993: dread d4/d20/d24/d2d/d4b/fa7 [0,4194304] 0 2024-09-10T11:29:53.769 INFO:tasks.workunit.client.0.smithi144.stdout:8/934: dread d0/d9/d21/d25/ff8 [4194304,4194304] 0 2024-09-10T11:29:53.834 INFO:tasks.workunit.client.0.smithi144.stdout:5/994: dwrite d4/d5/d18/d51/fd5 [0,4194304] 0 2024-09-10T11:29:53.839 INFO:tasks.workunit.client.0.smithi144.stdout:8/935: rename d0/d1b/d47/d5e/d76/de1 to d0/d9/d21/d7e/d145 0 2024-09-10T11:29:53.841 INFO:tasks.workunit.client.0.smithi144.stdout:5/995: link d4/d20/f34 d4/d5/d32/daa/f13c 0 2024-09-10T11:29:53.849 INFO:tasks.workunit.client.0.smithi144.stdout:8/936: dread d0/d9/d23/d26/d7d/fa8 [0,4194304] 0 2024-09-10T11:29:53.849 INFO:tasks.workunit.client.0.smithi144.stdout:5/996: dwrite d4/fa [4194304,4194304] 0 2024-09-10T11:29:53.850 INFO:tasks.workunit.client.0.smithi144.stdout:5/997: stat d4/d5/d18/d67/d59/f5e 0 2024-09-10T11:29:53.850 INFO:tasks.workunit.client.0.smithi144.stdout:8/937: stat d0/d9/d21/ca2 0 2024-09-10T11:29:53.851 INFO:tasks.workunit.client.0.smithi144.stdout:5/998: chown d4/d20/d22/d75 1 1 2024-09-10T11:29:53.859 INFO:tasks.workunit.client.0.smithi144.stdout:8/938: dwrite d0/d9/d23/d26/f69 [0,4194304] 0 2024-09-10T11:29:53.869 INFO:tasks.workunit.client.0.smithi144.stdout:8/939: dwrite d0/d9/d21/d7e/de5/f144 [0,4194304] 0 2024-09-10T11:29:53.870 INFO:tasks.workunit.client.0.smithi144.stdout:8/940: chown d0/d1b/d47/dfa/l129 171517728 1 2024-09-10T11:29:53.873 INFO:tasks.workunit.client.0.smithi144.stdout:8/941: write d0/d9/d23/d26/d7d/f81 [758494,17795] 0 2024-09-10T11:29:53.899 INFO:tasks.workunit.client.0.smithi144.stdout:5/999: dread d4/d5/d18/d38/d8d/fa8 [0,4194304] 0 2024-09-10T11:29:53.908 INFO:tasks.workunit.client.0.smithi144.stdout:8/942: dread d0/d9/d23/d26/f4e [0,4194304] 0 2024-09-10T11:29:53.915 INFO:tasks.workunit.client.0.smithi144.stdout:8/943: dwrite d0/d9/d23/f89 [0,4194304] 0 2024-09-10T11:29:53.917 INFO:tasks.workunit.client.0.smithi144.stdout:8/944: dread - d0/d9/d1f/d34/f126 zero size 2024-09-10T11:29:53.920 INFO:tasks.workunit.client.0.smithi144.stdout:8/945: fdatasync d0/d9/d1f/f92 0 2024-09-10T11:29:53.921 INFO:tasks.workunit.client.0.smithi144.stdout:6/799: sync 2024-09-10T11:29:53.922 INFO:tasks.workunit.client.0.smithi144.stdout:6/800: stat d9/dd/f33 0 2024-09-10T11:29:53.924 INFO:tasks.workunit.client.0.smithi144.stdout:6/801: fsync d9/dd/d3e/d47/f16 0 2024-09-10T11:29:53.926 INFO:tasks.workunit.client.0.smithi144.stdout:8/946: fsync d0/d9/d21/d25/db3/dc2/dcd/f9b 0 2024-09-10T11:29:53.942 INFO:tasks.workunit.client.0.smithi144.stdout:6/802: dwrite d9/d1f/fbb [0,4194304] 0 2024-09-10T11:29:53.945 INFO:tasks.workunit.client.0.smithi144.stdout:6/803: dread - d9/dd/d11/d1b/d2f/d7f/d82/fb3 zero size 2024-09-10T11:29:53.947 INFO:tasks.workunit.client.0.smithi144.stdout:8/947: dread d0/d9/d1f/f8f [0,4194304] 0 2024-09-10T11:29:53.949 INFO:tasks.workunit.client.0.smithi144.stdout:6/804: creat d9/dd/d11/d1b/d2f/d7f/dc4/d9e/f11c x:0 0 0 2024-09-10T11:29:53.952 INFO:tasks.workunit.client.0.smithi144.stdout:6/805: mknod d9/dd/d11/d1b/d2f/d7f/dc4/d5b/df9/d11a/c11d 0 2024-09-10T11:29:53.953 INFO:tasks.workunit.client.0.smithi144.stdout:6/806: fdatasync d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d87/f114 0 2024-09-10T11:29:53.958 INFO:tasks.workunit.client.0.smithi144.stdout:8/948: truncate d0/d9/d21/d25/db3/f100 8353397 0 2024-09-10T11:29:53.959 INFO:tasks.workunit.client.0.smithi144.stdout:6/807: write d9/d38/f63 [338060,101395] 0 2024-09-10T11:29:53.959 INFO:tasks.workunit.client.0.smithi144.stdout:6/808: stat d9/dd/d11/d1b/d2f/d7f/dc4/d9e/lbe 0 2024-09-10T11:29:53.960 INFO:tasks.workunit.client.0.smithi144.stdout:8/949: creat d0/d1b/d5a/f146 x:0 0 0 2024-09-10T11:29:53.965 INFO:tasks.workunit.client.0.smithi144.stdout:6/809: rmdir d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/df0 39 2024-09-10T11:29:53.965 INFO:tasks.workunit.client.0.smithi144.stdout:8/950: mkdir d0/d1b/d5a/d8d/d115/d11a/d147 0 2024-09-10T11:29:53.967 INFO:tasks.workunit.client.0.smithi144.stdout:6/810: creat d9/d10e/f11e x:0 0 0 2024-09-10T11:29:53.969 INFO:tasks.workunit.client.0.smithi144.stdout:8/951: symlink d0/d1b/d5a/d8d/l148 0 2024-09-10T11:29:53.969 INFO:tasks.workunit.client.0.smithi144.stdout:6/811: chown d9/dd/d11/d1b/d2f/d7f/c89 13 1 2024-09-10T11:29:53.969 INFO:tasks.workunit.client.0.smithi144.stdout:2/814: sync 2024-09-10T11:29:53.970 INFO:tasks.workunit.client.0.smithi144.stdout:6/812: chown d9/dd/d11/d1b/d2f/d7f/dc4/d5b/df9 2920 1 2024-09-10T11:29:53.975 INFO:tasks.workunit.client.0.smithi144.stdout:6/813: rename d9/dd/d11/d1b/ced to d9/dd/d3e/d47/c11f 0 2024-09-10T11:29:53.976 INFO:tasks.workunit.client.0.smithi144.stdout:2/815: write d7/de/d36/d45/f69 [4836686,104429] 0 2024-09-10T11:29:53.978 INFO:tasks.workunit.client.0.smithi144.stdout:6/814: symlink d9/dd/d11/d1b/d2f/d7f/dc4/d5b/df9/l120 0 2024-09-10T11:29:53.979 INFO:tasks.workunit.client.0.smithi144.stdout:2/816: creat d7/de/d3d/dff/f109 x:0 0 0 2024-09-10T11:29:53.980 INFO:tasks.workunit.client.0.smithi144.stdout:6/815: read - d9/dd/d11/d1b/d2f/d7f/d82/fb3 zero size 2024-09-10T11:29:53.982 INFO:tasks.workunit.client.0.smithi144.stdout:8/952: dwrite d0/d9/d21/f71 [0,4194304] 0 2024-09-10T11:29:53.983 INFO:tasks.workunit.client.0.smithi144.stdout:2/817: rmdir d7/de/d36/d45/d47/d6c/d7c/d82/df0 39 2024-09-10T11:29:53.984 INFO:tasks.workunit.client.0.smithi144.stdout:8/953: write d0/d1b/d47/d5e/d76/f7c [1196300,38463] 0 2024-09-10T11:29:53.986 INFO:tasks.workunit.client.0.smithi144.stdout:8/954: symlink d0/d1b/d47/dfa/l149 0 2024-09-10T11:29:53.987 INFO:tasks.workunit.client.0.smithi144.stdout:2/818: truncate d7/d29/d92/fb3 871880 0 2024-09-10T11:29:53.993 INFO:tasks.workunit.client.0.smithi144.stdout:2/819: link d7/da/d9e/ld1 d7/de/d36/l10a 0 2024-09-10T11:29:53.994 INFO:tasks.workunit.client.0.smithi144.stdout:8/955: write d0/d9/d23/d26/f4e [2661296,127295] 0 2024-09-10T11:29:53.996 INFO:tasks.workunit.client.0.smithi144.stdout:8/956: write d0/d9/d23/d26/f36 [3594581,100665] 0 2024-09-10T11:29:53.996 INFO:tasks.workunit.client.0.smithi144.stdout:2/820: unlink d7/de/d5f/df5/cf6 0 2024-09-10T11:29:53.999 INFO:tasks.workunit.client.0.smithi144.stdout:8/957: chown d0/d9/d21/d25/d4f/d93/c105 5 1 2024-09-10T11:29:54.000 INFO:tasks.workunit.client.0.smithi144.stdout:8/958: write d0/d1b/d47/d5e/d76/f114 [1785372,107447] 0 2024-09-10T11:29:54.005 INFO:tasks.workunit.client.0.smithi144.stdout:8/959: truncate d0/d9/d21/d25/d4f/fc8 608334 0 2024-09-10T11:29:54.007 INFO:tasks.workunit.client.0.smithi144.stdout:8/960: dread d0/d9/d21/d25/db3/dc2/dcd/f9b [0,4194304] 0 2024-09-10T11:29:54.013 INFO:tasks.workunit.client.0.smithi144.stdout:2/821: dread d7/de/d36/d45/d8e/d9a/f53 [0,4194304] 0 2024-09-10T11:29:54.017 INFO:tasks.workunit.client.0.smithi144.stdout:2/822: creat d7/d42/d64/d76/f10b x:0 0 0 2024-09-10T11:29:54.019 INFO:tasks.workunit.client.0.smithi144.stdout:2/823: write d7/d29/d92/fed [63541,69366] 0 2024-09-10T11:29:54.021 INFO:tasks.workunit.client.0.smithi144.stdout:2/824: chown d7/de/d36/d45/d8e/d9a/d54/d61/c83 29010463 1 2024-09-10T11:29:54.023 INFO:tasks.workunit.client.0.smithi144.stdout:8/961: dread d0/d9/d1f/f91 [0,4194304] 0 2024-09-10T11:29:54.024 INFO:tasks.workunit.client.0.smithi144.stdout:2/825: rmdir d7/de/d36/d45/d8e/d9a/d7e 39 2024-09-10T11:29:54.025 INFO:tasks.workunit.client.0.smithi144.stdout:8/962: creat d0/d9/d21/d25/db3/dc2/dd1/f14a x:0 0 0 2024-09-10T11:29:54.027 INFO:tasks.workunit.client.0.smithi144.stdout:2/826: rename d7/f8 to d7/f10c 0 2024-09-10T11:29:54.031 INFO:tasks.workunit.client.0.smithi144.stdout:8/963: rename d0/d1b/d5a/cd5 to d0/d9/d1f/d34/da0/c14b 0 2024-09-10T11:29:54.033 INFO:tasks.workunit.client.0.smithi144.stdout:8/964: mkdir d0/d1b/d47/d5e/d76/d14c 0 2024-09-10T11:29:54.040 INFO:tasks.workunit.client.0.smithi144.stdout:2/827: dread d7/de/d3d/d4e/fa3 [0,4194304] 0 2024-09-10T11:29:54.041 INFO:tasks.workunit.client.0.smithi144.stdout:2/828: readlink d7/de/d36/d45/le6 0 2024-09-10T11:29:54.050 INFO:tasks.workunit.client.0.smithi144.stdout:2/829: dread d7/da/d5b/fcf [0,4194304] 0 2024-09-10T11:29:54.053 INFO:tasks.workunit.client.0.smithi144.stdout:2/830: mkdir d7/de/d36/d45/d47/d10d 0 2024-09-10T11:29:54.061 INFO:tasks.workunit.client.0.smithi144.stdout:2/831: dread d7/de/d36/d45/d8e/d9a/d7e/fe1 [0,4194304] 0 2024-09-10T11:29:54.065 INFO:tasks.workunit.client.0.smithi144.stdout:2/832: rename d7/l38 to d7/de/d36/d45/d8e/l10e 0 2024-09-10T11:29:54.066 INFO:tasks.workunit.client.0.smithi144.stdout:2/833: rmdir d7/de/d36/d45/d8e/d9a/d54/d61 39 2024-09-10T11:29:54.068 INFO:tasks.workunit.client.0.smithi144.stdout:2/834: symlink d7/da/ddb/d7f/l10f 0 2024-09-10T11:29:54.069 INFO:tasks.workunit.client.0.smithi144.stdout:2/835: chown d7/d42/d64/dd5/lf9 4806 1 2024-09-10T11:29:54.072 INFO:tasks.workunit.client.0.smithi144.stdout:2/836: truncate d7/de/d36/d45/d8e/d9a/d7e/f8b 292666 0 2024-09-10T11:29:54.072 INFO:tasks.workunit.client.0.smithi144.stdout:2/837: chown d7/da/ddb/c90 285185293 1 2024-09-10T11:29:54.074 INFO:tasks.workunit.client.0.smithi144.stdout:2/838: fsync d7/de/d36/d45/d8e/d9a/d54/f8a 0 2024-09-10T11:29:54.087 INFO:tasks.workunit.client.0.smithi144.stdout:2/839: dread d7/d29/d8f/ddc/ff3 [0,4194304] 0 2024-09-10T11:29:54.089 INFO:tasks.workunit.client.0.smithi144.stdout:2/840: creat d7/d29/d8f/ddc/f110 x:0 0 0 2024-09-10T11:29:54.093 INFO:tasks.workunit.client.0.smithi144.stdout:2/841: dread d7/d29/d97/fe4 [0,4194304] 0 2024-09-10T11:29:54.096 INFO:tasks.workunit.client.0.smithi144.stdout:2/842: fsync d7/de/d36/f51 0 2024-09-10T11:29:54.096 INFO:tasks.workunit.client.0.smithi144.stdout:2/843: chown d7/d29/d92 240273 1 2024-09-10T11:29:54.104 INFO:tasks.workunit.client.0.smithi144.stdout:2/844: dwrite d7/d42/d64/d8d/fbb [4194304,4194304] 0 2024-09-10T11:29:54.107 INFO:tasks.workunit.client.0.smithi144.stdout:2/845: creat d7/de/d36/d45/d8e/d9a/d7e/f111 x:0 0 0 2024-09-10T11:29:54.110 INFO:tasks.workunit.client.0.smithi144.stdout:2/846: mkdir d7/de/d36/d45/d8e/d112 0 2024-09-10T11:29:54.125 INFO:tasks.workunit.client.0.smithi144.stdout:2/847: write d7/f1d [3827847,126391] 0 2024-09-10T11:29:54.126 INFO:tasks.workunit.client.0.smithi144.stdout:2/848: readlink d7/da/ddb/d7f/l10f 0 2024-09-10T11:29:54.127 INFO:tasks.workunit.client.0.smithi144.stdout:2/849: fdatasync d7/de/d36/d45/d47/d6c/d7c/f101 0 2024-09-10T11:29:54.128 INFO:tasks.workunit.client.0.smithi144.stdout:2/850: write d7/de/d36/d45/d8e/d9a/d54/fe3 [4583,31718] 0 2024-09-10T11:29:54.130 INFO:tasks.workunit.client.0.smithi144.stdout:2/851: fdatasync d7/de/d36/d45/d8e/d9a/d54/fe3 0 2024-09-10T11:29:54.238 INFO:tasks.workunit.client.0.smithi144.stdout:3/969: sync 2024-09-10T11:29:54.238 INFO:tasks.workunit.client.0.smithi144.stdout:9/948: sync 2024-09-10T11:29:54.238 INFO:tasks.workunit.client.0.smithi144.stdout:7/927: sync 2024-09-10T11:29:54.238 INFO:tasks.workunit.client.0.smithi144.stdout:0/847: sync 2024-09-10T11:29:54.241 INFO:tasks.workunit.client.0.smithi144.stdout:9/949: write d11/d36/db1/dbb/df0/d8f/f82 [1580359,17955] 0 2024-09-10T11:29:54.243 INFO:tasks.workunit.client.0.smithi144.stdout:3/970: creat d4/d28/d67/d80/d17/d18/d34/d65/f137 x:0 0 0 2024-09-10T11:29:54.243 INFO:tasks.workunit.client.0.smithi144.stdout:9/950: chown d11/d12/d1b/d2c/d78/dd2/c13d 410 1 2024-09-10T11:29:54.243 INFO:tasks.workunit.client.0.smithi144.stdout:3/971: stat d4 0 2024-09-10T11:29:54.243 INFO:tasks.workunit.client.0.smithi144.stdout:7/928: dread d5/dc/d11/d18/fd9 [0,4194304] 0 2024-09-10T11:29:54.243 INFO:tasks.workunit.client.0.smithi144.stdout:0/848: truncate da/d3a/d51/f9f 469145 0 2024-09-10T11:29:54.246 INFO:tasks.workunit.client.0.smithi144.stdout:3/972: creat d4/d28/d67/d80/d17/dd0/d98/f138 x:0 0 0 2024-09-10T11:29:54.246 INFO:tasks.workunit.client.0.smithi144.stdout:9/951: rmdir d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/d12e 39 2024-09-10T11:29:54.247 INFO:tasks.workunit.client.0.smithi144.stdout:0/849: truncate da/d44/f65 1111678 0 2024-09-10T11:29:54.247 INFO:tasks.workunit.client.0.smithi144.stdout:3/973: stat d4/d28/d67/d80/d17/dd0/d98/d9a/lba 0 2024-09-10T11:29:54.248 INFO:tasks.workunit.client.0.smithi144.stdout:7/929: symlink d5/dc/d14/d40/d58/d6a/d119/l130 0 2024-09-10T11:29:54.250 INFO:tasks.workunit.client.0.smithi144.stdout:3/974: truncate d4/dd/f12 694098 0 2024-09-10T11:29:54.251 INFO:tasks.workunit.client.0.smithi144.stdout:3/975: write d4/d28/d67/d80/d17/dd0/d98/d9a/fc2 [3252720,8290] 0 2024-09-10T11:29:54.252 INFO:tasks.workunit.client.0.smithi144.stdout:7/930: truncate d5/dc/d11/f3d 1026951 0 2024-09-10T11:29:54.256 INFO:tasks.workunit.client.0.smithi144.stdout:3/976: creat d4/d28/d67/d96/f139 x:0 0 0 2024-09-10T11:29:54.257 INFO:tasks.workunit.client.0.smithi144.stdout:9/952: write d11/f10c [23732,84522] 0 2024-09-10T11:29:54.259 INFO:tasks.workunit.client.0.smithi144.stdout:0/850: write da/d3a/d94/db1/dcc/f9c [2381411,125339] 0 2024-09-10T11:29:54.259 INFO:tasks.workunit.client.0.smithi144.stdout:9/953: write d11/d36/d2d/d38/de7/d12d/d14b/f14e [537286,99319] 0 2024-09-10T11:29:54.262 INFO:tasks.workunit.client.0.smithi144.stdout:3/977: getdents d4 0 2024-09-10T11:29:54.264 INFO:tasks.workunit.client.0.smithi144.stdout:9/954: mknod d11/d12/d6f/c151 0 2024-09-10T11:29:54.266 INFO:tasks.workunit.client.0.smithi144.stdout:0/851: unlink da/c12 0 2024-09-10T11:29:54.267 INFO:tasks.workunit.client.0.smithi144.stdout:9/955: fsync d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee/f108 0 2024-09-10T11:29:54.268 INFO:tasks.workunit.client.0.smithi144.stdout:7/931: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:54.270 INFO:tasks.workunit.client.0.smithi144.stdout:0/852: creat da/d3a/d4b/d4e/d62/d78/f104 x:0 0 0 2024-09-10T11:29:54.271 INFO:tasks.workunit.client.0.smithi144.stdout:0/853: write da/d3a/d4b/d4e/f5d [3248969,6022] 0 2024-09-10T11:29:54.271 INFO:tasks.workunit.client.0.smithi144.stdout:9/956: mknod d11/d12/d1b/d2c/c152 0 2024-09-10T11:29:54.275 INFO:tasks.workunit.client.0.smithi144.stdout:9/957: creat d11/d12/d6f/f153 x:0 0 0 2024-09-10T11:29:54.275 INFO:tasks.workunit.client.0.smithi144.stdout:7/932: mkdir d5/dc/d2a/d60/dd6/d131 0 2024-09-10T11:29:54.279 INFO:tasks.workunit.client.0.smithi144.stdout:9/958: rmdir d11/d12/d1b/df8/d131 0 2024-09-10T11:29:54.281 INFO:tasks.workunit.client.0.smithi144.stdout:7/933: getdents d5/dc/d14/d40/d58/d6a/d119 0 2024-09-10T11:29:54.282 INFO:tasks.workunit.client.0.smithi144.stdout:9/959: creat d11/d36/d2d/d38/d12c/f154 x:0 0 0 2024-09-10T11:29:54.284 INFO:tasks.workunit.client.0.smithi144.stdout:9/960: readlink d11/d12/d1b/l23 0 2024-09-10T11:29:54.286 INFO:tasks.workunit.client.0.smithi144.stdout:9/961: creat d11/d36/d2d/d38/f155 x:0 0 0 2024-09-10T11:29:54.288 INFO:tasks.workunit.client.0.smithi144.stdout:9/962: mkdir d11/d36/d2d/d38/d116/d156 0 2024-09-10T11:29:54.291 INFO:tasks.workunit.client.0.smithi144.stdout:9/963: read d11/d36/db1/dbb/df0/d8f/dc0/dd0/fe9 [774345,29421] 0 2024-09-10T11:29:54.293 INFO:tasks.workunit.client.0.smithi144.stdout:7/934: dwrite d5/dc/d2a/fa8 [0,4194304] 0 2024-09-10T11:29:54.295 INFO:tasks.workunit.client.0.smithi144.stdout:9/964: rmdir d11/d36/db1/dbb/df0/d8f/dc0/dd0/dee 39 2024-09-10T11:29:54.296 INFO:tasks.workunit.client.0.smithi144.stdout:9/965: stat d11/d36/d2d/c9e 0 2024-09-10T11:29:54.299 INFO:tasks.workunit.client.0.smithi144.stdout:7/935: symlink d5/dc/def/l132 0 2024-09-10T11:29:54.302 INFO:tasks.workunit.client.0.smithi144.stdout:9/966: creat d11/d36/db1/dbb/df0/f157 x:0 0 0 2024-09-10T11:29:54.312 INFO:tasks.workunit.client.0.smithi144.stdout:9/967: dwrite d11/d12/d1b/f27 [0,4194304] 0 2024-09-10T11:29:54.316 INFO:tasks.workunit.client.0.smithi144.stdout:9/968: unlink d11/d36/db1/dbb/df0/f149 0 2024-09-10T11:29:54.316 INFO:tasks.workunit.client.0.smithi144.stdout:9/969: chown d11/d36/d2d/d38/dc3/f120 970 1 2024-09-10T11:29:54.318 INFO:tasks.workunit.client.0.smithi144.stdout:7/936: dread d5/dc/f10 [0,4194304] 0 2024-09-10T11:29:54.319 INFO:tasks.workunit.client.0.smithi144.stdout:9/970: mknod d11/d12/d1b/c158 0 2024-09-10T11:29:54.321 INFO:tasks.workunit.client.0.smithi144.stdout:7/937: creat d5/dc/d11/f133 x:0 0 0 2024-09-10T11:29:54.323 INFO:tasks.workunit.client.0.smithi144.stdout:9/971: link d11/d12/d6f/cbe d11/d12/d1b/d2c/d12b/d14a/d64/c159 0 2024-09-10T11:29:54.323 INFO:tasks.workunit.client.0.smithi144.stdout:7/938: chown d5/dc/d12/d2b/cbb 207612128 1 2024-09-10T11:29:54.325 INFO:tasks.workunit.client.0.smithi144.stdout:9/972: mknod d11/d36/db1/dbb/c15a 0 2024-09-10T11:29:54.326 INFO:tasks.workunit.client.0.smithi144.stdout:7/939: mknod d5/dc/d2a/d60/dd6/d131/c134 0 2024-09-10T11:29:54.328 INFO:tasks.workunit.client.0.smithi144.stdout:7/940: creat d5/dc/d2a/d60/dd6/d131/f135 x:0 0 0 2024-09-10T11:29:54.331 INFO:tasks.workunit.client.0.smithi144.stdout:7/941: creat d5/dc/d2a/d60/dd6/d98/dd1/f136 x:0 0 0 2024-09-10T11:29:54.332 INFO:tasks.workunit.client.0.smithi144.stdout:7/942: dread - d5/dc/d14/d40/d58/d6a/ff8 zero size 2024-09-10T11:29:54.337 INFO:tasks.workunit.client.0.smithi144.stdout:9/973: dread d11/d36/db1/dbb/df0/d8f/f93 [0,4194304] 0 2024-09-10T11:29:54.339 INFO:tasks.workunit.client.0.smithi144.stdout:9/974: creat d11/d12/d1b/d2c/d78/dd2/dec/f15b x:0 0 0 2024-09-10T11:29:54.340 INFO:tasks.workunit.client.0.smithi144.stdout:9/975: dread - d11/d12/d1b/d2c/d78/dd2/f10e zero size 2024-09-10T11:29:54.342 INFO:tasks.workunit.client.0.smithi144.stdout:9/976: creat d11/d12/d1b/df8/f15c x:0 0 0 2024-09-10T11:29:54.345 INFO:tasks.workunit.client.0.smithi144.stdout:7/943: dwrite d5/dc/d11/f3d [0,4194304] 0 2024-09-10T11:29:54.347 INFO:tasks.workunit.client.0.smithi144.stdout:9/977: rename d11/d36/db1/dbb/df0/f92 to d11/d36/d2d/d38/dc3/f15d 0 2024-09-10T11:29:54.354 INFO:tasks.workunit.client.0.smithi144.stdout:9/978: chown d11/d12/d1b/f27 6390570 1 2024-09-10T11:29:54.362 INFO:tasks.workunit.client.0.smithi144.stdout:7/944: creat d5/dc/d2a/d60/dd6/f137 x:0 0 0 2024-09-10T11:29:54.374 INFO:tasks.workunit.client.0.smithi144.stdout:9/979: truncate d11/d36/d2d/d38/dc3/f120 3422427 0 2024-09-10T11:29:54.374 INFO:tasks.workunit.client.0.smithi144.stdout:9/980: creat d11/d12/d1b/d2c/d12b/d14a/f15e x:0 0 0 2024-09-10T11:29:54.376 INFO:tasks.workunit.client.0.smithi144.stdout:7/945: dwrite d5/dc/d11/d18/fd9 [0,4194304] 0 2024-09-10T11:29:54.380 INFO:tasks.workunit.client.0.smithi144.stdout:9/981: getdents d11/d36/db1/dbb 0 2024-09-10T11:29:54.380 INFO:tasks.workunit.client.0.smithi144.stdout:7/946: write d5/fa [17355,13695] 0 2024-09-10T11:29:54.381 INFO:tasks.workunit.client.0.smithi144.stdout:7/947: fsync d5/dc/d14/d40/f67 0 2024-09-10T11:29:54.383 INFO:tasks.workunit.client.0.smithi144.stdout:9/982: mkdir d11/d36/d2d/d38/de7/d15f 0 2024-09-10T11:29:54.383 INFO:tasks.workunit.client.0.smithi144.stdout:9/983: chown d11/d36/d2d/d38/de7 2279729 1 2024-09-10T11:29:54.387 INFO:tasks.workunit.client.0.smithi144.stdout:9/984: rename d11/d36/db1/dbb/df0/d8f/dc0/dd0/cd1 to d11/d36/d2d/d38/d116/d156/c160 0 2024-09-10T11:29:54.389 INFO:tasks.workunit.client.0.smithi144.stdout:7/948: truncate d5/dc/d12/f52 3990688 0 2024-09-10T11:29:54.391 INFO:tasks.workunit.client.0.smithi144.stdout:7/949: unlink d5/dc/d2a/d60/dd6/f137 0 2024-09-10T11:29:54.404 INFO:tasks.workunit.client.0.smithi144.stdout:7/950: dread d5/dc/d12/d2b/dde/f10b [0,4194304] 0 2024-09-10T11:29:54.405 INFO:tasks.workunit.client.0.smithi144.stdout:9/985: dread d11/f17 [0,4194304] 0 2024-09-10T11:29:54.407 INFO:tasks.workunit.client.0.smithi144.stdout:7/951: mkdir d5/dc/d14/d40/d5e/dc4/d118/d138 0 2024-09-10T11:29:54.408 INFO:tasks.workunit.client.0.smithi144.stdout:9/986: fdatasync d11/d12/d1b/df8/f99 0 2024-09-10T11:29:54.410 INFO:tasks.workunit.client.0.smithi144.stdout:9/987: chown d11/d12/d6f/daa/ffe 16620709 1 2024-09-10T11:29:54.411 INFO:tasks.workunit.client.0.smithi144.stdout:7/952: truncate d5/dc/fe4 1301069 0 2024-09-10T11:29:54.412 INFO:tasks.workunit.client.0.smithi144.stdout:9/988: symlink d11/d36/db1/dbb/df0/d123/l161 0 2024-09-10T11:29:54.415 INFO:tasks.workunit.client.0.smithi144.stdout:7/953: rename d5/dc/d12/d2b/l12d to d5/dc/d14/d40/d58/d6a/db2/l139 0 2024-09-10T11:29:54.416 INFO:tasks.workunit.client.0.smithi144.stdout:7/954: write d5/dc/d11/f3d [286681,94972] 0 2024-09-10T11:29:54.420 INFO:tasks.workunit.client.0.smithi144.stdout:7/955: truncate d5/dc/d14/d40/f57 2193051 0 2024-09-10T11:29:54.426 INFO:tasks.workunit.client.0.smithi144.stdout:7/956: write d5/dc/d2a/d60/dd6/d98/fe3 [143784,38868] 0 2024-09-10T11:29:54.429 INFO:tasks.workunit.client.0.smithi144.stdout:9/989: dwrite d11/d12/d1b/d2c/d78/dd2/f129 [0,4194304] 0 2024-09-10T11:29:54.431 INFO:tasks.workunit.client.0.smithi144.stdout:7/957: fsync d5/dc/d14/d40/f72 0 2024-09-10T11:29:54.437 INFO:tasks.workunit.client.0.smithi144.stdout:9/990: dwrite d11/d36/d2d/d38/dc3/f113 [0,4194304] 0 2024-09-10T11:29:54.439 INFO:tasks.workunit.client.0.smithi144.stdout:7/958: write d5/dc/d14/d40/d58/d6a/db2/ff6 [933899,33291] 0 2024-09-10T11:29:54.441 INFO:tasks.workunit.client.0.smithi144.stdout:9/991: creat d11/d36/d2d/d38/de7/d15f/f162 x:0 0 0 2024-09-10T11:29:54.443 INFO:tasks.workunit.client.0.smithi144.stdout:7/959: mkdir d5/dc/d12/d2b/d13a 0 2024-09-10T11:29:54.449 INFO:tasks.workunit.client.0.smithi144.stdout:9/992: dwrite d11/d36/d2d/d38/f155 [0,4194304] 0 2024-09-10T11:29:54.452 INFO:tasks.workunit.client.0.smithi144.stdout:7/960: rename d5/dc/d14/d40/d58/d6a/dad/c10e to d5/dc/d14/d40/d5e/c13b 0 2024-09-10T11:29:54.453 INFO:tasks.workunit.client.0.smithi144.stdout:9/993: creat d11/d36/d2d/d38/de7/d12d/d14b/f163 x:0 0 0 2024-09-10T11:29:54.456 INFO:tasks.workunit.client.0.smithi144.stdout:9/994: mkdir d11/d36/d2d/d38/dc3/d164 0 2024-09-10T11:29:54.459 INFO:tasks.workunit.client.0.smithi144.stdout:9/995: mkdir d11/d12/d1b/d2c/d12b/d14a/d165 0 2024-09-10T11:29:54.463 INFO:tasks.workunit.client.0.smithi144.stdout:7/961: dread d5/dc/d14/d40/d58/d6a/dad/f10f [0,4194304] 0 2024-09-10T11:29:54.465 INFO:tasks.workunit.client.0.smithi144.stdout:9/996: dread d11/d36/db1/f13f [0,4194304] 0 2024-09-10T11:29:54.468 INFO:tasks.workunit.client.0.smithi144.stdout:7/962: rename d5/dc/d14/d40/d5e/d68/fd2 to d5/dc/d12/d10a/f13c 0 2024-09-10T11:29:54.472 INFO:tasks.workunit.client.0.smithi144.stdout:7/963: unlink d5/dc/d14/d40/d58/c126 0 2024-09-10T11:29:54.472 INFO:tasks.workunit.client.0.smithi144.stdout:9/997: write d11/d12/d1b/ff4 [54520,105039] 0 2024-09-10T11:29:54.474 INFO:tasks.workunit.client.0.smithi144.stdout:7/964: mknod d5/dc/d14/d40/d5e/dc4/c13d 0 2024-09-10T11:29:54.475 INFO:tasks.workunit.client.0.smithi144.stdout:7/965: write d5/dc/d14/d40/d58/d6a/f92 [2105039,90081] 0 2024-09-10T11:29:54.477 INFO:tasks.workunit.client.0.smithi144.stdout:9/998: rename d11/d36/d2d/d38/l39 to d11/d12/d1b/d2c/d12b/d14a/l166 0 2024-09-10T11:29:54.480 INFO:tasks.workunit.client.0.smithi144.stdout:7/966: dread d5/f8 [0,4194304] 0 2024-09-10T11:29:54.480 INFO:tasks.workunit.client.0.smithi144.stdout:7/967: stat d5/dc/d14/d40/d58/d6a/l6d 0 2024-09-10T11:29:54.481 INFO:tasks.workunit.client.0.smithi144.stdout:9/999: rename d11/f15 to d11/d11c/f167 0 2024-09-10T11:29:54.482 INFO:tasks.workunit.client.0.smithi144.stdout:7/968: chown d5/dc/d14/d40/d58/d6a/db2/ff4 206170 1 2024-09-10T11:29:54.484 INFO:tasks.workunit.client.0.smithi144.stdout:7/969: readlink d5/dc/d2a/l11f 0 2024-09-10T11:29:54.495 INFO:tasks.workunit.client.0.smithi144.stdout:7/970: dwrite d5/dc/d12/d10a/f13c [0,4194304] 0 2024-09-10T11:29:54.499 INFO:tasks.workunit.client.0.smithi144.stdout:7/971: dread d5/dc/d11/f3d [0,4194304] 0 2024-09-10T11:29:54.501 INFO:tasks.workunit.client.0.smithi144.stdout:7/972: write d5/dc/d2a/fa8 [4169655,59924] 0 2024-09-10T11:29:54.503 INFO:tasks.workunit.client.0.smithi144.stdout:7/973: mknod d5/dc/d2a/d41/c13e 0 2024-09-10T11:29:54.504 INFO:tasks.workunit.client.0.smithi144.stdout:7/974: readlink d5/dc/d11/l6c 0 2024-09-10T11:29:54.516 INFO:tasks.workunit.client.0.smithi144.stdout:7/975: dwrite d5/dc/d2a/d60/dd6/f66 [0,4194304] 0 2024-09-10T11:29:54.522 INFO:tasks.workunit.client.0.smithi144.stdout:7/976: rename d5/dc/d14/d40/d58/d6e/d7a/l97 to d5/dc/d14/d40/d58/d6e/d7a/d11b/l13f 0 2024-09-10T11:29:54.524 INFO:tasks.workunit.client.0.smithi144.stdout:7/977: mknod d5/dc/d2a/d60/c140 0 2024-09-10T11:29:54.529 INFO:tasks.workunit.client.0.smithi144.stdout:7/978: dwrite d5/f105 [0,4194304] 0 2024-09-10T11:29:54.545 INFO:tasks.workunit.client.0.smithi144.stdout:7/979: dwrite d5/dc/d14/d40/d5e/d68/fb6 [0,4194304] 0 2024-09-10T11:29:54.549 INFO:tasks.workunit.client.0.smithi144.stdout:7/980: mknod d5/dc/d14/d40/d58/d6a/dad/c141 0 2024-09-10T11:29:54.555 INFO:tasks.workunit.client.0.smithi144.stdout:7/981: dwrite d5/dc/d2a/d60/dd6/d98/fe3 [0,4194304] 0 2024-09-10T11:29:54.560 INFO:tasks.workunit.client.0.smithi144.stdout:7/982: creat d5/dc/d14/d40/d58/d6e/d7a/de0/d129/f142 x:0 0 0 2024-09-10T11:29:54.569 INFO:tasks.workunit.client.0.smithi144.stdout:7/983: getdents d5/dc/d14/d40/d58/d6a/dad 0 2024-09-10T11:29:54.578 INFO:tasks.workunit.client.0.smithi144.stdout:6/816: sync 2024-09-10T11:29:54.578 INFO:tasks.workunit.client.0.smithi144.stdout:7/984: dread d5/dc/fe4 [0,4194304] 0 2024-09-10T11:29:54.578 INFO:tasks.workunit.client.0.smithi144.stdout:7/985: chown d5/dc/d2a/f4b 256986101 1 2024-09-10T11:29:54.578 INFO:tasks.workunit.client.0.smithi144.stdout:6/817: symlink d9/dd/d11/d1b/d2f/dab/dd0/l121 0 2024-09-10T11:29:54.581 INFO:tasks.workunit.client.0.smithi144.stdout:6/818: creat d9/dd/d3e/d47/f122 x:0 0 0 2024-09-10T11:29:54.583 INFO:tasks.workunit.client.0.smithi144.stdout:6/819: read d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/f6f [178479,22161] 0 2024-09-10T11:29:54.587 INFO:tasks.workunit.client.0.smithi144.stdout:6/820: link d9/dd/d3e/f40 d9/dd/d11/d1b/d2f/dab/de1/f123 0 2024-09-10T11:29:54.590 INFO:tasks.workunit.client.0.smithi144.stdout:6/821: mkdir d9/dd/d11/d1b/d2f/d7f/dc4/d5b/df9/d11a/d124 0 2024-09-10T11:29:54.591 INFO:tasks.workunit.client.0.smithi144.stdout:7/986: dread d5/dc/d2a/d60/dd6/d98/fca [0,4194304] 0 2024-09-10T11:29:54.591 INFO:tasks.workunit.client.0.smithi144.stdout:6/822: readlink d9/dd/d11/d1b/d2f/d7f/d82/l7b 0 2024-09-10T11:29:54.592 INFO:tasks.workunit.client.0.smithi144.stdout:7/987: chown d5/dc/d12/d10a/l125 996619861 1 2024-09-10T11:29:54.592 INFO:tasks.workunit.client.0.smithi144.stdout:6/823: read - d9/dd/d11/d1b/d2f/d7f/f110 zero size 2024-09-10T11:29:54.594 INFO:tasks.workunit.client.0.smithi144.stdout:7/988: creat d5/dc/d14/d40/d58/d6a/db2/f143 x:0 0 0 2024-09-10T11:29:54.595 INFO:tasks.workunit.client.0.smithi144.stdout:7/989: write d5/dc/d2a/d60/dd6/d98/dd1/f136 [653514,81791] 0 2024-09-10T11:29:54.598 INFO:tasks.workunit.client.0.smithi144.stdout:7/990: rmdir d5/dc/d14/d107 0 2024-09-10T11:29:54.609 INFO:tasks.workunit.client.0.smithi144.stdout:7/991: dwrite d5/dc/fec [0,4194304] 0 2024-09-10T11:29:54.611 INFO:tasks.workunit.client.0.smithi144.stdout:7/992: mkdir d5/dc/d2a/d41/d144 0 2024-09-10T11:29:54.613 INFO:tasks.workunit.client.0.smithi144.stdout:7/993: rmdir d5/dc/d12 39 2024-09-10T11:29:54.616 INFO:tasks.workunit.client.0.smithi144.stdout:7/994: symlink d5/dc/d11/l145 0 2024-09-10T11:29:54.618 INFO:tasks.workunit.client.0.smithi144.stdout:7/995: symlink d5/dc/d2a/d41/d144/l146 0 2024-09-10T11:29:54.629 INFO:tasks.workunit.client.0.smithi144.stdout:7/996: dwrite d5/dc/d14/d40/d5e/f6f [0,4194304] 0 2024-09-10T11:29:54.632 INFO:tasks.workunit.client.0.smithi144.stdout:7/997: rmdir d5/dc/d14/d40/d5e/ddd 39 2024-09-10T11:29:54.634 INFO:tasks.workunit.client.0.smithi144.stdout:7/998: fdatasync d5/dc/d12/f11c 0 2024-09-10T11:29:54.635 INFO:tasks.workunit.client.0.smithi144.stdout:7/999: chown d5/dc/d14/d40/d58 244893976 1 2024-09-10T11:29:54.733 INFO:tasks.workunit.client.0.smithi144.stdout:8/965: sync 2024-09-10T11:29:54.734 INFO:tasks.workunit.client.0.smithi144.stdout:3/978: sync 2024-09-10T11:29:54.734 INFO:tasks.workunit.client.0.smithi144.stdout:0/854: sync 2024-09-10T11:29:54.734 INFO:tasks.workunit.client.0.smithi144.stdout:2/852: sync 2024-09-10T11:29:54.737 INFO:tasks.workunit.client.0.smithi144.stdout:8/966: creat d0/d1b/d5a/d8d/f14d x:0 0 0 2024-09-10T11:29:54.739 INFO:tasks.workunit.client.0.smithi144.stdout:2/853: mkdir d7/de/d36/d45/d47/d10d/d113 0 2024-09-10T11:29:54.741 INFO:tasks.workunit.client.0.smithi144.stdout:8/967: creat d0/d9/d21/d7e/d85/f14e x:0 0 0 2024-09-10T11:29:54.745 INFO:tasks.workunit.client.0.smithi144.stdout:3/979: dwrite d4/dd/f6f [0,4194304] 0 2024-09-10T11:29:54.749 INFO:tasks.workunit.client.0.smithi144.stdout:0/855: dwrite da/d3a/d51/f9f [0,4194304] 0 2024-09-10T11:29:54.749 INFO:tasks.workunit.client.0.smithi144.stdout:8/968: creat d0/d9/d21/d25/d4f/f14f x:0 0 0 2024-09-10T11:29:54.750 INFO:tasks.workunit.client.0.smithi144.stdout:2/854: dread d7/de/d3d/d4e/f6d [0,4194304] 0 2024-09-10T11:29:54.754 INFO:tasks.workunit.client.0.smithi144.stdout:8/969: unlink d0/d1b/d47/dfa/l149 0 2024-09-10T11:29:54.756 INFO:tasks.workunit.client.0.smithi144.stdout:0/856: mknod da/d3a/d4b/d4e/d62/c105 0 2024-09-10T11:29:54.759 INFO:tasks.workunit.client.0.smithi144.stdout:2/855: rename d7/de/d3d/f3e to d7/d29/d8f/ddc/f114 0 2024-09-10T11:29:54.762 INFO:tasks.workunit.client.0.smithi144.stdout:8/970: mkdir d0/d9/d1f/d34/da0/dc4/d112/d150 0 2024-09-10T11:29:54.764 INFO:tasks.workunit.client.0.smithi144.stdout:0/857: mkdir da/d3a/d4b/d4e/d62/d106 0 2024-09-10T11:29:54.765 INFO:tasks.workunit.client.0.smithi144.stdout:8/971: write d0/d1b/d5a/d8d/d115/d11a/f13a [1012067,27080] 0 2024-09-10T11:29:54.771 INFO:tasks.workunit.client.0.smithi144.stdout:2/856: dwrite d7/da/d5b/fcf [0,4194304] 0 2024-09-10T11:29:54.772 INFO:tasks.workunit.client.0.smithi144.stdout:8/972: getdents d0/d1b/d5a/d8d/d115/d11a 0 2024-09-10T11:29:54.773 INFO:tasks.workunit.client.0.smithi144.stdout:2/857: chown d7/d42/fe8 30815458 1 2024-09-10T11:29:54.774 INFO:tasks.workunit.client.0.smithi144.stdout:2/858: chown d7/da/d1e/l1f 23360351 1 2024-09-10T11:29:54.775 INFO:tasks.workunit.client.0.smithi144.stdout:0/858: dwrite da/d3a/d4b/d4e/dae/fc3 [0,4194304] 0 2024-09-10T11:29:54.777 INFO:tasks.workunit.client.0.smithi144.stdout:0/859: truncate da/d3a/d4b/ff6 1031594 0 2024-09-10T11:29:54.783 INFO:tasks.workunit.client.0.smithi144.stdout:3/980: truncate d4/d28/d67/d80/d17/dd0/fbf 321320 0 2024-09-10T11:29:54.783 INFO:tasks.workunit.client.0.smithi144.stdout:3/981: dread d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/f8c [0,4194304] 0 2024-09-10T11:29:54.784 INFO:tasks.workunit.client.0.smithi144.stdout:0/860: mknod da/d3a/d94/c107 0 2024-09-10T11:29:54.785 INFO:tasks.workunit.client.0.smithi144.stdout:6/824: sync 2024-09-10T11:29:54.785 INFO:tasks.workunit.client.0.smithi144.stdout:8/973: getdents d0/d9/d1f/d63 0 2024-09-10T11:29:54.788 INFO:tasks.workunit.client.0.smithi144.stdout:3/982: creat d4/d28/d67/d80/d17/d18/d34/d78/f13a x:0 0 0 2024-09-10T11:29:54.789 INFO:tasks.workunit.client.0.smithi144.stdout:6/825: dread - d9/dd/d11/d1b/d2f/dab/dd0/fdb zero size 2024-09-10T11:29:54.792 INFO:tasks.workunit.client.0.smithi144.stdout:2/859: write d7/d29/f4b [122307,100450] 0 2024-09-10T11:29:54.792 INFO:tasks.workunit.client.0.smithi144.stdout:8/974: getdents d0/d9/d1f/d63 0 2024-09-10T11:29:54.796 INFO:tasks.workunit.client.0.smithi144.stdout:6/826: fdatasync d9/dd/d11/d1b/d2f/dab/dd0/fdb 0 2024-09-10T11:29:54.796 INFO:tasks.workunit.client.0.smithi144.stdout:2/860: chown d7/de/d36/d45/d8e/d9a/f53 0 1 2024-09-10T11:29:54.796 INFO:tasks.workunit.client.0.smithi144.stdout:6/827: chown d9/dd/d3e/d47/fcd 0 1 2024-09-10T11:29:54.796 INFO:tasks.workunit.client.0.smithi144.stdout:2/861: chown d7/de/d36/f9b 58208760 1 2024-09-10T11:29:54.798 INFO:tasks.workunit.client.0.smithi144.stdout:8/975: mkdir d0/d9/d21/d25/d4f/d151 0 2024-09-10T11:29:54.800 INFO:tasks.workunit.client.0.smithi144.stdout:3/983: truncate d4/d28/d67/d80/d17/d18/f104 1691372 0 2024-09-10T11:29:54.801 INFO:tasks.workunit.client.0.smithi144.stdout:0/861: write da/ffb [167280,11768] 0 2024-09-10T11:29:54.802 INFO:tasks.workunit.client.0.smithi144.stdout:6/828: getdents d9/dd/d3e 0 2024-09-10T11:29:54.803 INFO:tasks.workunit.client.0.smithi144.stdout:8/976: mknod d0/d9/d21/d7e/c152 0 2024-09-10T11:29:54.805 INFO:tasks.workunit.client.0.smithi144.stdout:6/829: fdatasync d9/dd/d3e/d47/f88 0 2024-09-10T11:29:54.808 INFO:tasks.workunit.client.0.smithi144.stdout:8/977: write d0/d9/d23/d26/f69 [3931884,117018] 0 2024-09-10T11:29:54.809 INFO:tasks.workunit.client.0.smithi144.stdout:3/984: rename d4/d28/d67/d80/d17/d18/d34/d78 to d4/d1a/d13b 0 2024-09-10T11:29:54.810 INFO:tasks.workunit.client.0.smithi144.stdout:2/862: dwrite d7/da/d1e/f85 [0,4194304] 0 2024-09-10T11:29:54.811 INFO:tasks.workunit.client.0.smithi144.stdout:0/862: write da/d3a/d4b/d4e/d81/fbd [757321,76177] 0 2024-09-10T11:29:54.814 INFO:tasks.workunit.client.0.smithi144.stdout:6/830: rename d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d60/fba to d9/f125 0 2024-09-10T11:29:54.818 INFO:tasks.workunit.client.0.smithi144.stdout:6/831: write d9/dd/d73/f10f [2087646,46069] 0 2024-09-10T11:29:54.818 INFO:tasks.workunit.client.0.smithi144.stdout:8/978: rename d0/d9/d23/d26/f4e to d0/d1b/d5a/d8d/d115/d11a/f153 0 2024-09-10T11:29:54.820 INFO:tasks.workunit.client.0.smithi144.stdout:2/863: dwrite d7/da/d5b/f96 [0,4194304] 0 2024-09-10T11:29:54.821 INFO:tasks.workunit.client.0.smithi144.stdout:6/832: mkdir d9/d126 0 2024-09-10T11:29:54.823 INFO:tasks.workunit.client.0.smithi144.stdout:6/833: fsync d9/d38/ff5 0 2024-09-10T11:29:54.824 INFO:tasks.workunit.client.0.smithi144.stdout:2/864: rmdir d7/d42/d64/de0 39 2024-09-10T11:29:54.827 INFO:tasks.workunit.client.0.smithi144.stdout:2/865: symlink d7/d29/d8f/l115 0 2024-09-10T11:29:54.828 INFO:tasks.workunit.client.0.smithi144.stdout:2/866: dread - d7/de/d36/d45/d8e/d9a/f108 zero size 2024-09-10T11:29:54.829 INFO:tasks.workunit.client.0.smithi144.stdout:6/834: mknod d9/d126/c127 0 2024-09-10T11:29:54.830 INFO:tasks.workunit.client.0.smithi144.stdout:2/867: readlink d7/da/d5b/d80/lba 0 2024-09-10T11:29:54.833 INFO:tasks.workunit.client.0.smithi144.stdout:2/868: mkdir d7/de/d36/d45/d8e/d9a/dc9/d116 0 2024-09-10T11:29:54.836 INFO:tasks.workunit.client.0.smithi144.stdout:2/869: creat d7/d42/d64/d76/f117 x:0 0 0 2024-09-10T11:29:54.837 INFO:tasks.workunit.client.0.smithi144.stdout:6/835: dread d9/dd/d3e/d47/f88 [0,4194304] 0 2024-09-10T11:29:54.840 INFO:tasks.workunit.client.0.smithi144.stdout:2/870: getdents d7/de/d36/d45/d8e 0 2024-09-10T11:29:54.844 INFO:tasks.workunit.client.0.smithi144.stdout:2/871: truncate d7/de/d3d/f52 283298 0 2024-09-10T11:29:54.844 INFO:tasks.workunit.client.0.smithi144.stdout:6/836: dwrite d9/dd/d11/d1b/d2f/d7f/dc4/d9e/f11c [0,4194304] 0 2024-09-10T11:29:54.848 INFO:tasks.workunit.client.0.smithi144.stdout:2/872: dread d7/de/d36/d45/d47/f88 [0,4194304] 0 2024-09-10T11:29:54.858 INFO:tasks.workunit.client.0.smithi144.stdout:2/873: write d7/de/d36/d45/d47/d6c/d7c/d82/fbd [49120,20663] 0 2024-09-10T11:29:54.858 INFO:tasks.workunit.client.0.smithi144.stdout:6/837: read d9/dd/d11/f1c [1671616,54353] 0 2024-09-10T11:29:54.859 INFO:tasks.workunit.client.0.smithi144.stdout:2/874: fsync d7/f10c 0 2024-09-10T11:29:54.862 INFO:tasks.workunit.client.0.smithi144.stdout:2/875: mkdir d7/da/d1e/d118 0 2024-09-10T11:29:54.868 INFO:tasks.workunit.client.0.smithi144.stdout:2/876: creat d7/da/f119 x:0 0 0 2024-09-10T11:29:54.872 INFO:tasks.workunit.client.0.smithi144.stdout:2/877: creat d7/da/d1e/d118/f11a x:0 0 0 2024-09-10T11:29:54.872 INFO:tasks.workunit.client.0.smithi144.stdout:2/878: chown d7/de/d36/d45 2433 1 2024-09-10T11:29:54.876 INFO:tasks.workunit.client.0.smithi144.stdout:6/838: dread d9/f100 [0,4194304] 0 2024-09-10T11:29:54.882 INFO:tasks.workunit.client.0.smithi144.stdout:6/839: link d9/dd/d11/d1b/d2f/d7f/f75 d9/dd/d11/d1b/d2f/f128 0 2024-09-10T11:29:54.893 INFO:tasks.workunit.client.0.smithi144.stdout:6/840: write d9/dd/d11/d1b/fd2 [4410778,105706] 0 2024-09-10T11:29:54.899 INFO:tasks.workunit.client.0.smithi144.stdout:6/841: getdents d9/dd/d11/d1b/d2f/d7f/dc4/d5b/dc8 0 2024-09-10T11:29:54.903 INFO:tasks.workunit.client.0.smithi144.stdout:6/842: truncate d9/dd/d11/d1b/d2f/f128 1787902 0 2024-09-10T11:29:54.905 INFO:tasks.workunit.client.0.smithi144.stdout:6/843: dread - d9/dd/d11/f56 zero size 2024-09-10T11:29:54.911 INFO:tasks.workunit.client.0.smithi144.stdout:6/844: write d9/dd/d3e/f40 [551257,69230] 0 2024-09-10T11:29:54.922 INFO:tasks.workunit.client.0.smithi144.stdout:6/845: dread d9/dd/d11/f21 [0,4194304] 0 2024-09-10T11:29:54.927 INFO:tasks.workunit.client.0.smithi144.stdout:6/846: dread d9/dd/d11/d1b/d2f/dab/fce [0,4194304] 0 2024-09-10T11:29:54.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:54 smithi144 ceph-mon[27942]: pgmap v34: 65 pgs: 65 active+clean; 3.3 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 99 MiB/s rd, 242 MiB/s wr, 593 op/s 2024-09-10T11:29:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:54 smithi178 ceph-mon[33514]: pgmap v34: 65 pgs: 65 active+clean; 3.3 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 99 MiB/s rd, 242 MiB/s wr, 593 op/s 2024-09-10T11:29:55.364 INFO:tasks.workunit.client.0.smithi144.stdout:6/847: sync 2024-09-10T11:29:55.364 INFO:tasks.workunit.client.0.smithi144.stdout:8/979: sync 2024-09-10T11:29:55.364 INFO:tasks.workunit.client.0.smithi144.stdout:3/985: sync 2024-09-10T11:29:55.364 INFO:tasks.workunit.client.0.smithi144.stdout:2/879: sync 2024-09-10T11:29:55.364 INFO:tasks.workunit.client.0.smithi144.stdout:0/863: sync 2024-09-10T11:29:55.366 INFO:tasks.workunit.client.0.smithi144.stdout:0/864: read - da/d44/d82/d8c/deb/ff0 zero size 2024-09-10T11:29:55.366 INFO:tasks.workunit.client.0.smithi144.stdout:6/848: readlink d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/lc1 0 2024-09-10T11:29:55.367 INFO:tasks.workunit.client.0.smithi144.stdout:0/865: chown da/d44/d82/d8c/lc8 65495 1 2024-09-10T11:29:55.394 INFO:tasks.workunit.client.0.smithi144.stdout:6/849: dread d9/d38/ff3 [0,4194304] 0 2024-09-10T11:29:55.394 INFO:tasks.workunit.client.0.smithi144.stdout:0/866: creat da/d44/f108 x:0 0 0 2024-09-10T11:29:55.397 INFO:tasks.workunit.client.0.smithi144.stdout:0/867: rename l8 to da/d3a/d4b/d96/l109 0 2024-09-10T11:29:55.397 INFO:tasks.workunit.client.0.smithi144.stdout:8/980: link d0/d9/d21/d7e/d85/fb9 d0/d9/d23/d26/d7d/dda/f154 0 2024-09-10T11:29:55.398 INFO:tasks.workunit.client.0.smithi144.stdout:0/868: write da/d3a/d4b/d4e/f5d [1172662,31954] 0 2024-09-10T11:29:55.398 INFO:tasks.workunit.client.0.smithi144.stdout:8/981: dread - d0/d9/d21/d25/d4f/d93/f131 zero size 2024-09-10T11:29:55.404 INFO:tasks.workunit.client.0.smithi144.stdout:6/850: getdents d9/dd/d3e/d47 0 2024-09-10T11:29:55.408 INFO:tasks.workunit.client.0.smithi144.stdout:0/869: getdents da/d3a/d94 0 2024-09-10T11:29:55.410 INFO:tasks.workunit.client.0.smithi144.stdout:6/851: rename d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d87 to d9/dd/d129 0 2024-09-10T11:29:55.412 INFO:tasks.workunit.client.0.smithi144.stdout:0/870: write da/d3a/d4b/d61/ddf/fee [785643,93118] 0 2024-09-10T11:29:55.413 INFO:tasks.workunit.client.0.smithi144.stdout:6/852: read d9/d1f/d2c/f39 [736815,107521] 0 2024-09-10T11:29:55.421 INFO:tasks.workunit.client.0.smithi144.stdout:6/853: dread d9/dd/f33 [0,4194304] 0 2024-09-10T11:29:55.423 INFO:tasks.workunit.client.0.smithi144.stdout:6/854: fsync d9/dd/d3e/d47/ff4 0 2024-09-10T11:29:55.425 INFO:tasks.workunit.client.0.smithi144.stdout:2/880: write d7/de/d36/d45/d8e/d9a/f2a [5229709,110783] 0 2024-09-10T11:29:55.425 INFO:tasks.workunit.client.0.smithi144.stdout:3/986: write d4/d28/d67/d80/d17/dd0/faa [5479003,37845] 0 2024-09-10T11:29:55.429 INFO:tasks.workunit.client.0.smithi144.stdout:2/881: rmdir d7/da/d5b 39 2024-09-10T11:29:55.430 INFO:tasks.workunit.client.0.smithi144.stdout:3/987: symlink d4/d28/d67/d96/dcc/d116/l13c 0 2024-09-10T11:29:55.432 INFO:tasks.workunit.client.0.smithi144.stdout:3/988: write d4/d28/d67/d80/d17/dd0/f11b [401646,104965] 0 2024-09-10T11:29:55.437 INFO:tasks.workunit.client.0.smithi144.stdout:8/982: dwrite d0/d9/d21/f73 [0,4194304] 0 2024-09-10T11:29:55.437 INFO:tasks.workunit.client.0.smithi144.stdout:6/855: write d9/dd/d11/d1b/d2f/dab/de1/ffe [336147,100483] 0 2024-09-10T11:29:55.440 INFO:tasks.workunit.client.0.smithi144.stdout:2/882: write d7/d29/d8f/ddc/f114 [397890,11824] 0 2024-09-10T11:29:55.443 INFO:tasks.workunit.client.0.smithi144.stdout:2/883: mkdir d7/d42/d11b 0 2024-09-10T11:29:55.443 INFO:tasks.workunit.client.0.smithi144.stdout:0/871: dwrite f0 [0,4194304] 0 2024-09-10T11:29:55.445 INFO:tasks.workunit.client.0.smithi144.stdout:6/856: fdatasync d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/fb5 0 2024-09-10T11:29:55.448 INFO:tasks.workunit.client.0.smithi144.stdout:0/872: write da/d3a/d4b/d61/d6c/fe4 [961835,112935] 0 2024-09-10T11:29:55.448 INFO:tasks.workunit.client.0.smithi144.stdout:2/884: mkdir d7/d42/d64/dd5/d11c 0 2024-09-10T11:29:55.451 INFO:tasks.workunit.client.0.smithi144.stdout:3/989: dwrite d4/dd/d21/f5b [4194304,4194304] 0 2024-09-10T11:29:55.452 INFO:tasks.workunit.client.0.smithi144.stdout:2/885: readlink d7/da/d9e/ld1 0 2024-09-10T11:29:55.455 INFO:tasks.workunit.client.0.smithi144.stdout:8/983: dwrite d0/d9/d1f/d34/d68/fd7 [8388608,4194304] 0 2024-09-10T11:29:55.458 INFO:tasks.workunit.client.0.smithi144.stdout:0/873: truncate da/d3a/d4b/d4e/f5d 5134172 0 2024-09-10T11:29:55.459 INFO:tasks.workunit.client.0.smithi144.stdout:2/886: unlink d7/da/f17 0 2024-09-10T11:29:55.460 INFO:tasks.workunit.client.0.smithi144.stdout:0/874: readlink da/l31 0 2024-09-10T11:29:55.460 INFO:tasks.workunit.client.0.smithi144.stdout:2/887: stat d7/f15 0 2024-09-10T11:29:55.463 INFO:tasks.workunit.client.0.smithi144.stdout:2/888: truncate d7/d42/d64/fd6 1368073 0 2024-09-10T11:29:55.463 INFO:tasks.workunit.client.0.smithi144.stdout:8/984: unlink d0/d9/d1f/d63/lff 0 2024-09-10T11:29:55.464 INFO:tasks.workunit.client.0.smithi144.stdout:8/985: chown d0/d9/d21/d25/d4f/d93 55855 1 2024-09-10T11:29:55.464 INFO:tasks.workunit.client.0.smithi144.stdout:0/875: mknod da/d3a/d4b/d4e/d90/dc5/c10a 0 2024-09-10T11:29:55.469 INFO:tasks.workunit.client.0.smithi144.stdout:8/986: truncate d0/d9/d21/d25/d4f/f14f 884649 0 2024-09-10T11:29:55.469 INFO:tasks.workunit.client.0.smithi144.stdout:6/857: write d9/dd/d11/d1b/d2f/dab/de7/fe8 [1119331,124313] 0 2024-09-10T11:29:55.470 INFO:tasks.workunit.client.0.smithi144.stdout:2/889: rename d7/de/d36/d45/d8e/d9a/d7e/l91 to d7/d29/l11d 0 2024-09-10T11:29:55.474 INFO:tasks.workunit.client.0.smithi144.stdout:3/990: dwrite d4/dd/d21/f42 [0,4194304] 0 2024-09-10T11:29:55.474 INFO:tasks.workunit.client.0.smithi144.stdout:6/858: creat d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/dde/f12a x:0 0 0 2024-09-10T11:29:55.475 INFO:tasks.workunit.client.0.smithi144.stdout:3/991: chown d4/d28/d67/d96/ca7 0 1 2024-09-10T11:29:55.477 INFO:tasks.workunit.client.0.smithi144.stdout:6/859: mknod d9/d38/c12b 0 2024-09-10T11:29:55.478 INFO:tasks.workunit.client.0.smithi144.stdout:2/890: link d7/de/d5f/cbf d7/d42/d64/dd5/d11c/c11e 0 2024-09-10T11:29:55.478 INFO:tasks.workunit.client.0.smithi144.stdout:6/860: readlink d9/dd/d11/d1b/d2f/la8 0 2024-09-10T11:29:55.483 INFO:tasks.workunit.client.0.smithi144.stdout:6/861: creat d9/d1f/f12c x:0 0 0 2024-09-10T11:29:55.483 INFO:tasks.workunit.client.0.smithi144.stdout:2/891: getdents d7/de/dea 0 2024-09-10T11:29:55.484 INFO:tasks.workunit.client.0.smithi144.stdout:3/992: mknod d4/d28/d67/d80/d17/d18/d34/d65/d75/d90/d92/c13d 0 2024-09-10T11:29:55.486 INFO:tasks.workunit.client.0.smithi144.stdout:6/862: creat d9/d1f/d2c/f12d x:0 0 0 2024-09-10T11:29:55.491 INFO:tasks.workunit.client.0.smithi144.stdout:3/993: getdents d4/d28/d67/d80/d17/d18/df4 0 2024-09-10T11:29:55.493 INFO:tasks.workunit.client.0.smithi144.stdout:3/994: chown d4/d28/d67/f7c 896 1 2024-09-10T11:29:55.494 INFO:tasks.workunit.client.0.smithi144.stdout:2/892: dwrite d7/de/d36/d45/d47/d6c/d7c/d82/fbd [0,4194304] 0 2024-09-10T11:29:55.499 INFO:tasks.workunit.client.0.smithi144.stdout:2/893: creat d7/d42/d64/de0/f11f x:0 0 0 2024-09-10T11:29:55.502 INFO:tasks.workunit.client.0.smithi144.stdout:8/987: dread d0/d9/d21/d7e/fe9 [0,4194304] 0 2024-09-10T11:29:55.505 INFO:tasks.workunit.client.0.smithi144.stdout:8/988: unlink d0/d9/d1f/d34/f99 0 2024-09-10T11:29:55.506 INFO:tasks.workunit.client.0.smithi144.stdout:8/989: fsync d0/d9/d21/d7e/de5/f144 0 2024-09-10T11:29:55.507 INFO:tasks.workunit.client.0.smithi144.stdout:8/990: stat d0/d9/d21/d25/db3/dc2/dd1 0 2024-09-10T11:29:55.509 INFO:tasks.workunit.client.0.smithi144.stdout:8/991: rmdir d0/d9/d1f/d34/da0 39 2024-09-10T11:29:55.514 INFO:tasks.workunit.client.0.smithi144.stdout:2/894: dread d7/da/d1e/f66 [0,4194304] 0 2024-09-10T11:29:55.516 INFO:tasks.workunit.client.0.smithi144.stdout:2/895: chown d7/de/d36/d45/d8e/d9a/dc9/ceb 26989215 1 2024-09-10T11:29:55.516 INFO:tasks.workunit.client.0.smithi144.stdout:6/863: dread d9/d1f/d2c/f67 [0,4194304] 0 2024-09-10T11:29:55.517 INFO:tasks.workunit.client.0.smithi144.stdout:2/896: write d7/da/f12 [1022204,98680] 0 2024-09-10T11:29:55.520 INFO:tasks.workunit.client.0.smithi144.stdout:8/992: write d0/d9/d21/f10f [2044483,72192] 0 2024-09-10T11:29:55.521 INFO:tasks.workunit.client.0.smithi144.stdout:2/897: creat d7/da/d5b/d80/f120 x:0 0 0 2024-09-10T11:29:55.521 INFO:tasks.workunit.client.0.smithi144.stdout:6/864: getdents d9/dd/d3e 0 2024-09-10T11:29:55.522 INFO:tasks.workunit.client.0.smithi144.stdout:8/993: rmdir d0/d9/d1f/d34/d68 39 2024-09-10T11:29:55.525 INFO:tasks.workunit.client.0.smithi144.stdout:6/865: mknod d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/c12e 0 2024-09-10T11:29:55.527 INFO:tasks.workunit.client.0.smithi144.stdout:8/994: link d0/d9/d23/ce4 d0/d9/d21/d25/db3/dc2/dd1/c155 0 2024-09-10T11:29:55.531 INFO:tasks.workunit.client.0.smithi144.stdout:8/995: mkdir d0/d9/d21/d25/db3/dc2/d156 0 2024-09-10T11:29:55.532 INFO:tasks.workunit.client.0.smithi144.stdout:8/996: chown d0/d1b 23993 1 2024-09-10T11:29:55.532 INFO:tasks.workunit.client.0.smithi144.stdout:2/898: dwrite d7/da/f19 [0,4194304] 0 2024-09-10T11:29:55.533 INFO:tasks.workunit.client.0.smithi144.stdout:6/866: dwrite d9/d1f/fbb [0,4194304] 0 2024-09-10T11:29:55.535 INFO:tasks.workunit.client.0.smithi144.stdout:8/997: dread - d0/d9/d23/d26/d7d/dda/f12a zero size 2024-09-10T11:29:55.536 INFO:tasks.workunit.client.0.smithi144.stdout:2/899: chown d7/de/d3d/dc8/cd4 179 1 2024-09-10T11:29:55.537 INFO:tasks.workunit.client.0.smithi144.stdout:6/867: creat d9/dd/d73/f12f x:0 0 0 2024-09-10T11:29:55.537 INFO:tasks.workunit.client.0.smithi144.stdout:2/900: mkdir d7/d42/d11b/d121 0 2024-09-10T11:29:55.539 INFO:tasks.workunit.client.0.smithi144.stdout:8/998: truncate d0/d9/d21/d25/db3/de8/f113 552077 0 2024-09-10T11:29:55.540 INFO:tasks.workunit.client.0.smithi144.stdout:6/868: mknod d9/dd/d11/d1b/d2f/dab/de7/d105/c130 0 2024-09-10T11:29:55.546 INFO:tasks.workunit.client.0.smithi144.stdout:8/999: creat d0/d9/f157 x:0 0 0 2024-09-10T11:29:55.550 INFO:tasks.workunit.client.0.smithi144.stdout:6/869: link d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/la1 d9/dd/d11/d1b/d2f/d7f/dc4/d5b/dc8/l131 0 2024-09-10T11:29:55.552 INFO:tasks.workunit.client.0.smithi144.stdout:6/870: mknod d9/dd/d11/d1b/c132 0 2024-09-10T11:29:55.557 INFO:tasks.workunit.client.0.smithi144.stdout:2/901: truncate d7/de/d3d/d4e/fa3 5701472 0 2024-09-10T11:29:55.562 INFO:tasks.workunit.client.0.smithi144.stdout:2/902: getdents d7/de/d36/d45 0 2024-09-10T11:29:55.564 INFO:tasks.workunit.client.0.smithi144.stdout:6/871: dwrite d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/f51 [0,4194304] 0 2024-09-10T11:29:55.567 INFO:tasks.workunit.client.0.smithi144.stdout:6/872: read d9/dd/d11/d1b/f32 [1649599,116223] 0 2024-09-10T11:29:55.567 INFO:tasks.workunit.client.0.smithi144.stdout:2/903: link d7/de/d3d/c5c d7/de/d70/df1/c122 0 2024-09-10T11:29:55.569 INFO:tasks.workunit.client.0.smithi144.stdout:2/904: dread - d7/de/d36/d45/f84 zero size 2024-09-10T11:29:55.571 INFO:tasks.workunit.client.0.smithi144.stdout:6/873: rename d9/dd/c10 to d9/d1f/c133 0 2024-09-10T11:29:55.574 INFO:tasks.workunit.client.0.smithi144.stdout:6/874: creat d9/d1f/d2c/f134 x:0 0 0 2024-09-10T11:29:55.576 INFO:tasks.workunit.client.0.smithi144.stdout:2/905: dwrite d7/da/f119 [0,4194304] 0 2024-09-10T11:29:55.578 INFO:tasks.workunit.client.0.smithi144.stdout:2/906: read - d7/de/d3d/d4e/fd8 zero size 2024-09-10T11:29:55.580 INFO:tasks.workunit.client.0.smithi144.stdout:2/907: truncate d7/d42/f48 9187634 0 2024-09-10T11:29:55.581 INFO:tasks.workunit.client.0.smithi144.stdout:6/875: truncate d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/f19 1673200 0 2024-09-10T11:29:55.586 INFO:tasks.workunit.client.0.smithi144.stdout:2/908: creat d7/d42/d64/d76/f123 x:0 0 0 2024-09-10T11:29:55.587 INFO:tasks.workunit.client.0.smithi144.stdout:6/876: dread d9/dd/d11/d1b/d2f/d7f/dc4/d9e/f11c [0,4194304] 0 2024-09-10T11:29:55.592 INFO:tasks.workunit.client.0.smithi144.stdout:2/909: dwrite d7/da/d1e/f85 [0,4194304] 0 2024-09-10T11:29:55.593 INFO:tasks.workunit.client.0.smithi144.stdout:2/910: readlink d7/da/ddb/l62 0 2024-09-10T11:29:55.595 INFO:tasks.workunit.client.0.smithi144.stdout:2/911: symlink d7/d29/d8f/l124 0 2024-09-10T11:29:55.596 INFO:tasks.workunit.client.0.smithi144.stdout:6/877: write d9/dd/d3e/d47/f28 [1015540,79097] 0 2024-09-10T11:29:55.597 INFO:tasks.workunit.client.0.smithi144.stdout:2/912: fsync d7/de/f3b 0 2024-09-10T11:29:55.600 INFO:tasks.workunit.client.0.smithi144.stdout:6/878: mkdir d9/dd/d11/d135 0 2024-09-10T11:29:55.600 INFO:tasks.workunit.client.0.smithi144.stdout:6/879: readlink d9/dd/l34 0 2024-09-10T11:29:55.604 INFO:tasks.workunit.client.0.smithi144.stdout:6/880: rmdir d9/d1f/dae 39 2024-09-10T11:29:55.607 INFO:tasks.workunit.client.0.smithi144.stdout:2/913: write d7/de/d36/d45/d47/d6c/d7c/d82/fbc [1129734,116742] 0 2024-09-10T11:29:55.609 INFO:tasks.workunit.client.0.smithi144.stdout:2/914: write d7/de/d36/d45/d47/d6c/d7c/d82/fbd [3966633,423] 0 2024-09-10T11:29:55.610 INFO:tasks.workunit.client.0.smithi144.stdout:2/915: write d7/da/ddb/f7b [3514185,113793] 0 2024-09-10T11:29:55.614 INFO:tasks.workunit.client.0.smithi144.stdout:2/916: rename d7/da/d5b/f81 to d7/d42/d64/f125 0 2024-09-10T11:29:55.616 INFO:tasks.workunit.client.0.smithi144.stdout:2/917: creat d7/de/d36/d45/d47/d10d/f126 x:0 0 0 2024-09-10T11:29:55.618 INFO:tasks.workunit.client.0.smithi144.stdout:2/918: symlink d7/de/d36/d45/d47/d6c/l127 0 2024-09-10T11:29:55.618 INFO:tasks.workunit.client.0.smithi144.stdout:2/919: chown d7/d29/d92/cb1 7 1 2024-09-10T11:29:55.631 INFO:tasks.workunit.client.0.smithi144.stdout:2/920: dread d7/d42/f48 [0,4194304] 0 2024-09-10T11:29:55.633 INFO:tasks.workunit.client.0.smithi144.stdout:2/921: mkdir d7/d29/d92/d128 0 2024-09-10T11:29:55.635 INFO:tasks.workunit.client.0.smithi144.stdout:2/922: mknod d7/de/c129 0 2024-09-10T11:29:55.637 INFO:tasks.workunit.client.0.smithi144.stdout:2/923: mknod d7/d42/d64/de0/c12a 0 2024-09-10T11:29:55.637 INFO:tasks.workunit.client.0.smithi144.stdout:2/924: chown d7/de/d36/d45/d8e/d9a 29602542 1 2024-09-10T11:29:55.641 INFO:tasks.workunit.client.0.smithi144.stdout:2/925: rename d7/de/d36/d45/d8e/d9a/dc9/lce to d7/d42/d64/dd5/l12b 0 2024-09-10T11:29:55.641 INFO:tasks.workunit.client.0.smithi144.stdout:2/926: fdatasync d7/d42/d64/d76/f117 0 2024-09-10T11:29:55.644 INFO:tasks.workunit.client.0.smithi144.stdout:2/927: rename d7/da/ddb/fb0 to d7/de/d36/d45/d8e/d9a/d7e/f12c 0 2024-09-10T11:29:55.646 INFO:tasks.workunit.client.0.smithi144.stdout:2/928: mkdir d7/d42/d11b/d121/d12d 0 2024-09-10T11:29:55.648 INFO:tasks.workunit.client.0.smithi144.stdout:2/929: dread d7/de/d3d/d4e/fa3 [4194304,4194304] 0 2024-09-10T11:29:55.649 INFO:tasks.workunit.client.0.smithi144.stdout:2/930: write d7/da/ddb/f7b [2513633,25780] 0 2024-09-10T11:29:55.655 INFO:tasks.workunit.client.0.smithi144.stdout:2/931: dwrite d7/de/d3d/dff/f109 [0,4194304] 0 2024-09-10T11:29:55.666 INFO:tasks.workunit.client.0.smithi144.stdout:2/932: dwrite d7/d29/d92/fb3 [0,4194304] 0 2024-09-10T11:29:55.670 INFO:tasks.workunit.client.0.smithi144.stdout:2/933: dread d7/d42/d64/d76/f8c [4194304,4194304] 0 2024-09-10T11:29:55.671 INFO:tasks.workunit.client.0.smithi144.stdout:2/934: chown d7/de/d36/d45/d47/d6c/l127 8434234 1 2024-09-10T11:29:55.673 INFO:tasks.workunit.client.0.smithi144.stdout:2/935: mkdir d7/d42/d12e 0 2024-09-10T11:29:55.686 INFO:tasks.workunit.client.0.smithi144.stdout:2/936: dread d7/de/d36/d45/d47/d6c/fa2 [0,4194304] 0 2024-09-10T11:29:55.691 INFO:tasks.workunit.client.0.smithi144.stdout:2/937: truncate d7/de/d3d/d4e/f6a 1754052 0 2024-09-10T11:29:55.692 INFO:tasks.workunit.client.0.smithi144.stdout:2/938: chown d7/d29/c2e 1458086028 1 2024-09-10T11:29:55.695 INFO:tasks.workunit.client.0.smithi144.stdout:2/939: link d7/d42/d64/d8d/fbb d7/de/dea/f12f 0 2024-09-10T11:29:55.708 INFO:tasks.workunit.client.0.smithi144.stdout:2/940: dread d7/de/d3d/d4e/fa3 [0,4194304] 0 2024-09-10T11:29:55.710 INFO:tasks.workunit.client.0.smithi144.stdout:2/941: creat d7/d42/d11b/f130 x:0 0 0 2024-09-10T11:29:55.712 INFO:tasks.workunit.client.0.smithi144.stdout:2/942: truncate d7/da/d1e/f25 507337 0 2024-09-10T11:29:55.713 INFO:tasks.workunit.client.0.smithi144.stdout:2/943: chown d7/d29/d92/fb3 19623334 1 2024-09-10T11:29:55.719 INFO:tasks.workunit.client.0.smithi144.stdout:2/944: dwrite d7/dcc/dd3/fe2 [0,4194304] 0 2024-09-10T11:29:55.720 INFO:tasks.workunit.client.0.smithi144.stdout:2/945: write d7/de/d36/d45/d47/d6c/d7c/d82/fbc [2325300,3439] 0 2024-09-10T11:29:55.724 INFO:tasks.workunit.client.0.smithi144.stdout:2/946: creat d7/d42/d64/dd5/f131 x:0 0 0 2024-09-10T11:29:55.730 INFO:tasks.workunit.client.0.smithi144.stdout:2/947: dwrite d7/de/d36/d45/d47/d6c/d7c/d82/fbc [0,4194304] 0 2024-09-10T11:29:55.731 INFO:tasks.workunit.client.0.smithi144.stdout:2/948: chown d7/d29/f59 1712025 1 2024-09-10T11:29:55.733 INFO:tasks.workunit.client.0.smithi144.stdout:2/949: rmdir d7/d42/d64/dd5/d11c 39 2024-09-10T11:29:55.735 INFO:tasks.workunit.client.0.smithi144.stdout:2/950: symlink d7/de/d36/d45/d47/d6c/d7c/l132 0 2024-09-10T11:29:55.751 INFO:tasks.workunit.client.0.smithi144.stdout:2/951: dwrite d7/d29/d92/f63 [4194304,4194304] 0 2024-09-10T11:29:55.753 INFO:tasks.workunit.client.0.smithi144.stdout:0/876: sync 2024-09-10T11:29:55.753 INFO:tasks.workunit.client.0.smithi144.stdout:2/952: creat d7/d29/d8f/ddc/f133 x:0 0 0 2024-09-10T11:29:55.756 INFO:tasks.workunit.client.0.smithi144.stdout:2/953: mkdir d7/d29/d8f/d134 0 2024-09-10T11:29:55.758 INFO:tasks.workunit.client.0.smithi144.stdout:0/877: truncate da/d3a/d51/fd3 613708 0 2024-09-10T11:29:55.760 INFO:tasks.workunit.client.0.smithi144.stdout:0/878: creat da/d44/dd8/f10b x:0 0 0 2024-09-10T11:29:55.768 INFO:tasks.workunit.client.0.smithi144.stdout:2/954: write d7/f15 [346121,103641] 0 2024-09-10T11:29:55.771 INFO:tasks.workunit.client.0.smithi144.stdout:2/955: unlink d7/de/d36/d45/d8e/fdf 0 2024-09-10T11:29:55.776 INFO:tasks.workunit.client.0.smithi144.stdout:2/956: link d7/de/d36/d45/d8e/d9a/d7e/f8b d7/de/d3d/d4e/dac/f135 0 2024-09-10T11:29:55.776 INFO:tasks.workunit.client.0.smithi144.stdout:0/879: dread da/d3a/d4b/d4e/d62/f70 [0,4194304] 0 2024-09-10T11:29:55.777 INFO:tasks.workunit.client.0.smithi144.stdout:2/957: chown d7/d42/d64/dd5/f131 7304483 1 2024-09-10T11:29:55.777 INFO:tasks.workunit.client.0.smithi144.stdout:0/880: stat da/d44/f65 0 2024-09-10T11:29:55.779 INFO:tasks.workunit.client.0.smithi144.stdout:2/958: creat d7/de/d5f/df5/f136 x:0 0 0 2024-09-10T11:29:55.781 INFO:tasks.workunit.client.0.smithi144.stdout:2/959: chown d7/d42/d64/dd5/l12b 69 1 2024-09-10T11:29:55.783 INFO:tasks.workunit.client.0.smithi144.stdout:0/881: dwrite da/d3a/d4b/d4e/d62/d78/f104 [0,4194304] 0 2024-09-10T11:29:55.785 INFO:tasks.workunit.client.0.smithi144.stdout:2/960: truncate d7/d42/f48 1789156 0 2024-09-10T11:29:55.786 INFO:tasks.workunit.client.0.smithi144.stdout:2/961: write d7/de/d3d/d4e/f87 [615496,112235] 0 2024-09-10T11:29:55.787 INFO:tasks.workunit.client.0.smithi144.stdout:0/882: mknod da/d3a/d94/c10c 0 2024-09-10T11:29:55.789 INFO:tasks.workunit.client.0.smithi144.stdout:2/962: symlink d7/d29/d92/l137 0 2024-09-10T11:29:55.791 INFO:tasks.workunit.client.0.smithi144.stdout:0/883: mkdir da/d3a/d51/d10d 0 2024-09-10T11:29:55.794 INFO:tasks.workunit.client.0.smithi144.stdout:0/884: fdatasync da/d44/f54 0 2024-09-10T11:29:55.795 INFO:tasks.workunit.client.0.smithi144.stdout:2/963: creat d7/de/d36/d45/d8e/d9a/dc9/d116/f138 x:0 0 0 2024-09-10T11:29:55.796 INFO:tasks.workunit.client.0.smithi144.stdout:0/885: write da/d3a/d4b/ff6 [795363,44846] 0 2024-09-10T11:29:55.797 INFO:tasks.workunit.client.0.smithi144.stdout:6/881: sync 2024-09-10T11:29:55.797 INFO:tasks.workunit.client.0.smithi144.stdout:3/995: sync 2024-09-10T11:29:55.801 INFO:tasks.workunit.client.0.smithi144.stdout:2/964: mkdir d7/d29/d92/d139 0 2024-09-10T11:29:55.801 INFO:tasks.workunit.client.0.smithi144.stdout:0/886: rmdir da/d3a/d4b/d4e/d62/d78 39 2024-09-10T11:29:55.802 INFO:tasks.workunit.client.0.smithi144.stdout:3/996: mkdir d4/d28/d67/d80/d17/d18/d34/d65/d13e 0 2024-09-10T11:29:55.803 INFO:tasks.workunit.client.0.smithi144.stdout:6/882: creat d9/dd/d11/d1b/d2f/d7f/f136 x:0 0 0 2024-09-10T11:29:55.806 INFO:tasks.workunit.client.0.smithi144.stdout:3/997: creat d4/d28/d67/f13f x:0 0 0 2024-09-10T11:29:55.806 INFO:tasks.workunit.client.0.smithi144.stdout:6/883: symlink d9/dd/d3e/l137 0 2024-09-10T11:29:55.807 INFO:tasks.workunit.client.0.smithi144.stdout:2/965: dread d7/de/d36/d45/d8e/d9a/d7e/fe1 [0,4194304] 0 2024-09-10T11:29:55.809 INFO:tasks.workunit.client.0.smithi144.stdout:3/998: write d4/d28/d67/d80/d17/d18/d34/d65/d75/fda [1137130,109076] 0 2024-09-10T11:29:55.810 INFO:tasks.workunit.client.0.smithi144.stdout:2/966: mknod d7/d42/d11b/d121/c13a 0 2024-09-10T11:29:55.813 INFO:tasks.workunit.client.0.smithi144.stdout:2/967: creat d7/de/d3d/d4e/dc4/f13b x:0 0 0 2024-09-10T11:29:55.814 INFO:tasks.workunit.client.0.smithi144.stdout:2/968: dread - d7/de/d36/d45/d8e/d9a/dc9/d116/f138 zero size 2024-09-10T11:29:55.815 INFO:tasks.workunit.client.0.smithi144.stdout:3/999: mkdir d4/dd/d140 0 2024-09-10T11:29:55.816 INFO:tasks.workunit.client.0.smithi144.stdout:6/884: write d9/dd/f1a [6706837,9205] 0 2024-09-10T11:29:55.822 INFO:tasks.workunit.client.0.smithi144.stdout:0/887: dwrite da/d3a/d4b/d4e/d62/fc1 [0,4194304] 0 2024-09-10T11:29:55.823 INFO:tasks.workunit.client.0.smithi144.stdout:6/885: rename d9/dd/f101 to d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/f138 0 2024-09-10T11:29:55.825 INFO:tasks.workunit.client.0.smithi144.stdout:2/969: dwrite d7/de/d3d/f52 [0,4194304] 0 2024-09-10T11:29:55.828 INFO:tasks.workunit.client.0.smithi144.stdout:0/888: rename da/lbe to da/d3a/d51/d8f/dec/l10e 0 2024-09-10T11:29:55.830 INFO:tasks.workunit.client.0.smithi144.stdout:2/970: creat d7/de/d70/f13c x:0 0 0 2024-09-10T11:29:55.832 INFO:tasks.workunit.client.0.smithi144.stdout:0/889: chown da/l58 26148260 1 2024-09-10T11:29:55.834 INFO:tasks.workunit.client.0.smithi144.stdout:2/971: fdatasync d7/de/d36/d45/d8e/d9a/d54/f8a 0 2024-09-10T11:29:55.846 INFO:tasks.workunit.client.0.smithi144.stdout:6/886: write d9/dd/d3e/d47/f57 [2982540,8404] 0 2024-09-10T11:29:55.850 INFO:tasks.workunit.client.0.smithi144.stdout:6/887: creat d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d10a/f139 x:0 0 0 2024-09-10T11:29:55.851 INFO:tasks.workunit.client.0.smithi144.stdout:0/890: truncate da/f21 1161479 0 2024-09-10T11:29:55.854 INFO:tasks.workunit.client.0.smithi144.stdout:0/891: rmdir da/d3a/d51/d8f 39 2024-09-10T11:29:55.857 INFO:tasks.workunit.client.0.smithi144.stdout:0/892: mknod da/d3a/d51/d103/c10f 0 2024-09-10T11:29:55.858 INFO:tasks.workunit.client.0.smithi144.stdout:2/972: dwrite d7/de/dea/f12f [4194304,4194304] 0 2024-09-10T11:29:55.863 INFO:tasks.workunit.client.0.smithi144.stdout:0/893: unlink da/d3a/d4b/d96/fa3 0 2024-09-10T11:29:55.864 INFO:tasks.workunit.client.0.smithi144.stdout:0/894: chown da 26 1 2024-09-10T11:29:55.865 INFO:tasks.workunit.client.0.smithi144.stdout:6/888: write d9/f125 [1593474,78398] 0 2024-09-10T11:29:55.866 INFO:tasks.workunit.client.0.smithi144.stdout:2/973: dread d7/da/f119 [0,4194304] 0 2024-09-10T11:29:55.868 INFO:tasks.workunit.client.0.smithi144.stdout:6/889: chown d9/dd/d11/d1b/d2f/dab/dd0/fdb 3117 1 2024-09-10T11:29:55.870 INFO:tasks.workunit.client.0.smithi144.stdout:0/895: getdents da/d3a/d8e/dab 0 2024-09-10T11:29:55.872 INFO:tasks.workunit.client.0.smithi144.stdout:6/890: fsync d9/dd/d11/d1b/d2f/dab/de7/d105/f109 0 2024-09-10T11:29:55.874 INFO:tasks.workunit.client.0.smithi144.stdout:0/896: dread da/d3a/d51/fac [0,4194304] 0 2024-09-10T11:29:55.876 INFO:tasks.workunit.client.0.smithi144.stdout:2/974: dwrite d7/d29/d8f/ddc/f114 [0,4194304] 0 2024-09-10T11:29:55.876 INFO:tasks.workunit.client.0.smithi144.stdout:0/897: unlink da/d49/lb6 0 2024-09-10T11:29:55.877 INFO:tasks.workunit.client.0.smithi144.stdout:6/891: getdents d9/d126 0 2024-09-10T11:29:55.879 INFO:tasks.workunit.client.0.smithi144.stdout:6/892: read d9/f100 [3068898,68353] 0 2024-09-10T11:29:55.880 INFO:tasks.workunit.client.0.smithi144.stdout:6/893: rmdir d9 39 2024-09-10T11:29:55.881 INFO:tasks.workunit.client.0.smithi144.stdout:0/898: rmdir da/d3a/d51/d8f 39 2024-09-10T11:29:55.886 INFO:tasks.workunit.client.0.smithi144.stdout:0/899: rename da/d3a/d4b/d4e/dae/fc3 to da/d3a/d4b/d4e/f110 0 2024-09-10T11:29:55.887 INFO:tasks.workunit.client.0.smithi144.stdout:0/900: chown da/d3a/d4b/d4e/d62/d106 5355055 1 2024-09-10T11:29:55.892 INFO:tasks.workunit.client.0.smithi144.stdout:0/901: creat da/d3a/d51/d8f/dec/f111 x:0 0 0 2024-09-10T11:29:55.896 INFO:tasks.workunit.client.0.smithi144.stdout:6/894: write d9/dd/d11/f56 [861614,72281] 0 2024-09-10T11:29:55.904 INFO:tasks.workunit.client.0.smithi144.stdout:0/902: write da/d3a/d4b/d4e/d62/f68 [4888840,47507] 0 2024-09-10T11:29:55.907 INFO:tasks.workunit.client.0.smithi144.stdout:6/895: dwrite d9/fa0 [0,4194304] 0 2024-09-10T11:29:55.912 INFO:tasks.workunit.client.0.smithi144.stdout:6/896: dread - d9/dd/d11/d1b/d2f/d7f/ffa zero size 2024-09-10T11:29:55.917 INFO:tasks.workunit.client.0.smithi144.stdout:0/903: dwrite da/d3a/d4b/d4e/d62/fc1 [4194304,4194304] 0 2024-09-10T11:29:55.924 INFO:tasks.workunit.client.0.smithi144.stdout:6/897: rename d9/dd/d11/d1b/d2f/fb8 to d9/dd/d11/d1b/d2f/dab/de7/f13a 0 2024-09-10T11:29:55.926 INFO:tasks.workunit.client.0.smithi144.stdout:0/904: creat da/d3a/d51/f112 x:0 0 0 2024-09-10T11:29:55.947 INFO:tasks.workunit.client.0.smithi144.stdout:0/905: dwrite da/d3a/d4b/d4e/fdd [0,4194304] 0 2024-09-10T11:29:55.949 INFO:tasks.workunit.client.0.smithi144.stdout:0/906: fdatasync da/f1a 0 2024-09-10T11:29:55.952 INFO:tasks.workunit.client.0.smithi144.stdout:0/907: stat da/d3a/d51/d8f/le7 0 2024-09-10T11:29:55.953 INFO:tasks.workunit.client.0.smithi144.stdout:0/908: read da/d3a/d4b/d4e/fdd [1827697,126372] 0 2024-09-10T11:29:55.955 INFO:tasks.workunit.client.0.smithi144.stdout:0/909: read da/d3a/d51/f9f [2541508,124748] 0 2024-09-10T11:29:55.957 INFO:tasks.workunit.client.0.smithi144.stdout:0/910: rmdir da/d44/d82/d7e 39 2024-09-10T11:29:55.961 INFO:tasks.workunit.client.0.smithi144.stdout:0/911: mknod da/d3a/d4b/d4e/d62/dd2/c113 0 2024-09-10T11:29:55.970 INFO:tasks.workunit.client.0.smithi144.stdout:0/912: dwrite da/d3a/d4b/f86 [4194304,4194304] 0 2024-09-10T11:29:55.974 INFO:tasks.workunit.client.0.smithi144.stdout:0/913: symlink da/d49/l114 0 2024-09-10T11:29:55.975 INFO:tasks.workunit.client.0.smithi144.stdout:0/914: chown da/d3a/d4b/d4e/d62/lea 372 1 2024-09-10T11:29:55.986 INFO:tasks.workunit.client.0.smithi144.stdout:0/915: write da/f19 [724941,67560] 0 2024-09-10T11:29:55.990 INFO:tasks.workunit.client.0.smithi144.stdout:0/916: link da/d3a/d4b/d4e/d62/fc1 da/d3a/d4b/dd4/f115 0 2024-09-10T11:29:55.993 INFO:tasks.workunit.client.0.smithi144.stdout:0/917: creat da/d3a/d4b/d4e/d81/f116 x:0 0 0 2024-09-10T11:29:55.994 INFO:tasks.workunit.client.0.smithi144.stdout:0/918: rmdir da/d3a/d51/d8f 39 2024-09-10T11:29:56.001 INFO:tasks.workunit.client.0.smithi144.stdout:0/919: dwrite da/d3a/d4b/d4e/d81/f98 [0,4194304] 0 2024-09-10T11:29:56.005 INFO:tasks.workunit.client.0.smithi144.stdout:0/920: read da/d3a/f4d [5334774,55776] 0 2024-09-10T11:29:56.008 INFO:tasks.workunit.client.0.smithi144.stdout:0/921: rename da/d3a/d8e/dab/dc7/f102 to da/d49/f117 0 2024-09-10T11:29:56.011 INFO:tasks.workunit.client.0.smithi144.stdout:0/922: creat da/d44/d82/d8c/deb/f118 x:0 0 0 2024-09-10T11:29:56.012 INFO:tasks.workunit.client.0.smithi144.stdout:0/923: chown da/l22 70 1 2024-09-10T11:29:56.014 INFO:tasks.workunit.client.0.smithi144.stdout:0/924: readlink da/d3a/l3d 0 2024-09-10T11:29:56.020 INFO:tasks.workunit.client.0.smithi144.stdout:0/925: dwrite da/d3a/d4b/ff6 [0,4194304] 0 2024-09-10T11:29:56.025 INFO:tasks.workunit.client.0.smithi144.stdout:0/926: dread da/d3a/d4b/d4e/fdd [0,4194304] 0 2024-09-10T11:29:56.070 INFO:tasks.workunit.client.0.smithi144.stdout:0/927: creat da/d3a/d4b/d4e/dae/f119 x:0 0 0 2024-09-10T11:29:56.078 INFO:tasks.workunit.client.0.smithi144.stdout:0/928: dwrite da/d3a/d51/f7a [0,4194304] 0 2024-09-10T11:29:56.078 INFO:tasks.workunit.client.0.smithi144.stdout:0/929: stat da/f19 0 2024-09-10T11:29:56.134 INFO:tasks.workunit.client.0.smithi144.stdout:0/930: rename da/l58 to da/d3a/d4b/d4e/l11a 0 2024-09-10T11:29:56.138 INFO:tasks.workunit.client.0.smithi144.stdout:0/931: rename da/d3a/d51/d8f/dec to da/d44/dd8/d11b 0 2024-09-10T11:29:56.144 INFO:tasks.workunit.client.0.smithi144.stdout:0/932: dwrite da/d3a/d4b/d4e/d90/fa6 [0,4194304] 0 2024-09-10T11:29:56.147 INFO:tasks.workunit.client.0.smithi144.stdout:0/933: write da/d3a/d4b/d4e/d62/d78/f104 [1000204,72100] 0 2024-09-10T11:29:56.501 INFO:tasks.workunit.client.0.smithi144.stdout:2/975: sync 2024-09-10T11:29:56.504 INFO:tasks.workunit.client.0.smithi144.stdout:2/976: creat d7/d42/d64/f13d x:0 0 0 2024-09-10T11:29:56.504 INFO:tasks.workunit.client.0.smithi144.stdout:2/977: fdatasync d7/dcc/fdd 0 2024-09-10T11:29:56.506 INFO:tasks.workunit.client.0.smithi144.stdout:2/978: creat d7/d42/d64/f13e x:0 0 0 2024-09-10T11:29:56.656 INFO:tasks.workunit.client.0.smithi144.stdout:6/898: sync 2024-09-10T11:29:56.657 INFO:tasks.workunit.client.0.smithi144.stdout:0/934: sync 2024-09-10T11:29:56.659 INFO:tasks.workunit.client.0.smithi144.stdout:6/899: symlink d9/d1f/d2c/l13b 0 2024-09-10T11:29:56.659 INFO:tasks.workunit.client.0.smithi144.stdout:0/935: mknod da/d3a/d4b/d4e/d90/dc5/c11c 0 2024-09-10T11:29:56.662 INFO:tasks.workunit.client.0.smithi144.stdout:6/900: creat d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/f13c x:0 0 0 2024-09-10T11:29:56.663 INFO:tasks.workunit.client.0.smithi144.stdout:0/936: dread da/f46 [4194304,4194304] 0 2024-09-10T11:29:56.671 INFO:tasks.workunit.client.0.smithi144.stdout:0/937: dwrite da/d44/dd8/f10b [0,4194304] 0 2024-09-10T11:29:56.672 INFO:tasks.workunit.client.0.smithi144.stdout:6/901: dread d9/dd/d11/d1b/d2f/d7f/f92 [0,4194304] 0 2024-09-10T11:29:56.673 INFO:tasks.workunit.client.0.smithi144.stdout:6/902: stat d9/dd/d11/d1b/d2f/dab/de7 0 2024-09-10T11:29:56.674 INFO:tasks.workunit.client.0.smithi144.stdout:0/938: mknod da/d3a/d4b/d4e/dae/c11d 0 2024-09-10T11:29:56.676 INFO:tasks.workunit.client.0.smithi144.stdout:6/903: dread - d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d60/fdf zero size 2024-09-10T11:29:56.678 INFO:tasks.workunit.client.0.smithi144.stdout:0/939: symlink da/d3a/d4b/d4e/d62/d78/l11e 0 2024-09-10T11:29:56.679 INFO:tasks.workunit.client.0.smithi144.stdout:6/904: creat d9/dd/d11/d1b/f13d x:0 0 0 2024-09-10T11:29:56.679 INFO:tasks.workunit.client.0.smithi144.stdout:0/940: write da/d3a/d4b/d4e/d62/d78/fff [147006,68647] 0 2024-09-10T11:29:56.683 INFO:tasks.workunit.client.0.smithi144.stdout:0/941: symlink da/d44/d82/l11f 0 2024-09-10T11:29:56.683 INFO:tasks.workunit.client.0.smithi144.stdout:6/905: symlink d9/dd/d11/d1b/d2f/d7f/d82/l13e 0 2024-09-10T11:29:56.687 INFO:tasks.workunit.client.0.smithi144.stdout:6/906: creat d9/dd/f13f x:0 0 0 2024-09-10T11:29:56.688 INFO:tasks.workunit.client.0.smithi144.stdout:0/942: getdents da/d3a/d4b/d61/d6c 0 2024-09-10T11:29:56.688 INFO:tasks.workunit.client.0.smithi144.stdout:6/907: chown l2 127931236 1 2024-09-10T11:29:56.690 INFO:tasks.workunit.client.0.smithi144.stdout:6/908: truncate d9/dd/f49 620651 0 2024-09-10T11:29:56.694 INFO:tasks.workunit.client.0.smithi144.stdout:0/943: write da/f46 [4293706,99670] 0 2024-09-10T11:29:56.698 INFO:tasks.workunit.client.0.smithi144.stdout:0/944: truncate da/d3a/f3e 3998278 0 2024-09-10T11:29:56.699 INFO:tasks.workunit.client.0.smithi144.stdout:2/979: sync 2024-09-10T11:29:56.702 INFO:tasks.workunit.client.0.smithi144.stdout:6/909: dwrite d9/d1f/d2c/f64 [0,4194304] 0 2024-09-10T11:29:56.708 INFO:tasks.workunit.client.0.smithi144.stdout:0/945: dwrite da/f26 [0,4194304] 0 2024-09-10T11:29:56.712 INFO:tasks.workunit.client.0.smithi144.stdout:6/910: dwrite d9/f100 [0,4194304] 0 2024-09-10T11:29:56.714 INFO:tasks.workunit.client.0.smithi144.stdout:0/946: creat da/d44/d82/d8c/deb/f120 x:0 0 0 2024-09-10T11:29:56.718 INFO:tasks.workunit.client.0.smithi144.stdout:0/947: write da/d3a/d4b/d4e/d81/f98 [1187122,89738] 0 2024-09-10T11:29:56.720 INFO:tasks.workunit.client.0.smithi144.stdout:2/980: dread d7/da/ddb/f7b [0,4194304] 0 2024-09-10T11:29:56.721 INFO:tasks.workunit.client.0.smithi144.stdout:2/981: chown d7/d29/d8f/ddc/f110 1393235 1 2024-09-10T11:29:56.722 INFO:tasks.workunit.client.0.smithi144.stdout:0/948: read da/d3a/d4b/d4e/f60 [3796438,52230] 0 2024-09-10T11:29:56.723 INFO:tasks.workunit.client.0.smithi144.stdout:0/949: write da/f41 [228898,70412] 0 2024-09-10T11:29:56.726 INFO:tasks.workunit.client.0.smithi144.stdout:2/982: mkdir d7/da/ddb/d13f 0 2024-09-10T11:29:56.727 INFO:tasks.workunit.client.0.smithi144.stdout:6/911: dread d9/dd/d11/f56 [0,4194304] 0 2024-09-10T11:29:56.727 INFO:tasks.workunit.client.0.smithi144.stdout:0/950: mknod da/d3a/d4b/d4e/d62/d106/c121 0 2024-09-10T11:29:56.730 INFO:tasks.workunit.client.0.smithi144.stdout:6/912: mkdir d9/d126/d140 0 2024-09-10T11:29:56.732 INFO:tasks.workunit.client.0.smithi144.stdout:0/951: truncate da/d3a/d51/f9f 2980918 0 2024-09-10T11:29:56.733 INFO:tasks.workunit.client.0.smithi144.stdout:0/952: fdatasync da/d3a/d51/f7a 0 2024-09-10T11:29:56.735 INFO:tasks.workunit.client.0.smithi144.stdout:6/913: creat d9/dd/d11/d1b/d2f/d7f/dc4/d9e/f141 x:0 0 0 2024-09-10T11:29:56.737 INFO:tasks.workunit.client.0.smithi144.stdout:2/983: rename d7/de/d36/d45/d47/c6f to d7/de/d36/d45/d8e/d9a/c140 0 2024-09-10T11:29:56.738 INFO:tasks.workunit.client.0.smithi144.stdout:0/953: rmdir da/d3a/d4b/d4e/d90 39 2024-09-10T11:29:56.750 INFO:tasks.workunit.client.0.smithi144.stdout:6/914: truncate d9/f100 2160552 0 2024-09-10T11:29:56.751 INFO:tasks.workunit.client.0.smithi144.stdout:0/954: write da/f21 [2132806,101915] 0 2024-09-10T11:29:56.754 INFO:tasks.workunit.client.0.smithi144.stdout:2/984: dwrite d7/de/d36/f9b [0,4194304] 0 2024-09-10T11:29:56.757 INFO:tasks.workunit.client.0.smithi144.stdout:6/915: unlink d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d10a/l118 0 2024-09-10T11:29:56.760 INFO:tasks.workunit.client.0.smithi144.stdout:2/985: creat d7/de/d36/d45/d8e/d9a/d7e/f141 x:0 0 0 2024-09-10T11:29:56.761 INFO:tasks.workunit.client.0.smithi144.stdout:2/986: chown d7/da/d5b/d80 0 1 2024-09-10T11:29:56.763 INFO:tasks.workunit.client.0.smithi144.stdout:2/987: mknod d7/de/d36/d45/d8e/d112/c142 0 2024-09-10T11:29:56.765 INFO:tasks.workunit.client.0.smithi144.stdout:0/955: truncate da/d44/d82/f8b 3846534 0 2024-09-10T11:29:56.766 INFO:tasks.workunit.client.0.smithi144.stdout:0/956: rmdir da/d3a/d8e/dab 39 2024-09-10T11:29:56.767 INFO:tasks.workunit.client.0.smithi144.stdout:6/916: dread d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/f19 [0,4194304] 0 2024-09-10T11:29:56.768 INFO:tasks.workunit.client.0.smithi144.stdout:2/988: getdents d7/de/d70 0 2024-09-10T11:29:56.770 INFO:tasks.workunit.client.0.smithi144.stdout:6/917: unlink d9/dd/d73/c9c 0 2024-09-10T11:29:56.770 INFO:tasks.workunit.client.0.smithi144.stdout:0/957: creat da/d3a/d4b/d4e/d62/d69/f122 x:0 0 0 2024-09-10T11:29:56.775 INFO:tasks.workunit.client.0.smithi144.stdout:0/958: rename da/d49/f117 to da/d3a/d4b/d61/f123 0 2024-09-10T11:29:56.783 INFO:tasks.workunit.client.0.smithi144.stdout:0/959: dwrite da/d3a/d4b/d61/d6c/f9d [0,4194304] 0 2024-09-10T11:29:56.785 INFO:tasks.workunit.client.0.smithi144.stdout:2/989: dwrite d7/da/ddb/d7f/fc6 [0,4194304] 0 2024-09-10T11:29:56.787 INFO:tasks.workunit.client.0.smithi144.stdout:0/960: mkdir da/d3a/d51/d10d/d124 0 2024-09-10T11:29:56.787 INFO:tasks.workunit.client.0.smithi144.stdout:6/918: dread d9/d1f/f37 [0,4194304] 0 2024-09-10T11:29:56.788 INFO:tasks.workunit.client.0.smithi144.stdout:2/990: mknod d7/de/d36/d45/c143 0 2024-09-10T11:29:56.790 INFO:tasks.workunit.client.0.smithi144.stdout:0/961: fdatasync da/d3a/d4b/d4e/f60 0 2024-09-10T11:29:56.790 INFO:tasks.workunit.client.0.smithi144.stdout:2/991: fsync d7/d29/d97/fe4 0 2024-09-10T11:29:56.795 INFO:tasks.workunit.client.0.smithi144.stdout:6/919: dwrite d9/d1f/fbb [0,4194304] 0 2024-09-10T11:29:56.797 INFO:tasks.workunit.client.0.smithi144.stdout:0/962: mkdir da/d3a/d51/d8f/d125 0 2024-09-10T11:29:56.802 INFO:tasks.workunit.client.0.smithi144.stdout:6/920: link d9/fa0 d9/dd/d11/d1b/d2f/dab/dc6/f142 0 2024-09-10T11:29:56.803 INFO:tasks.workunit.client.0.smithi144.stdout:6/921: chown d9/c24 968807340 1 2024-09-10T11:29:56.806 INFO:tasks.workunit.client.0.smithi144.stdout:2/992: chown d7/de/d36/d45/d47/d6c/d7c/l132 2596 1 2024-09-10T11:29:56.806 INFO:tasks.workunit.client.0.smithi144.stdout:0/963: link da/d44/d82/d8c/lef da/d3a/d94/db1/dcc/l126 0 2024-09-10T11:29:56.807 INFO:tasks.workunit.client.0.smithi144.stdout:6/922: creat d9/d1f/dae/f143 x:0 0 0 2024-09-10T11:29:56.808 INFO:tasks.workunit.client.0.smithi144.stdout:0/964: write da/d3a/d4b/d4e/d62/d78/fff [141118,87131] 0 2024-09-10T11:29:56.808 INFO:tasks.workunit.client.0.smithi144.stdout:6/923: chown d9/dd/d3e/d47/f57 6 1 2024-09-10T11:29:56.809 INFO:tasks.workunit.client.0.smithi144.stdout:2/993: mkdir d7/de/d144 0 2024-09-10T11:29:56.809 INFO:tasks.workunit.client.0.smithi144.stdout:0/965: chown da/d44/f108 2060624 1 2024-09-10T11:29:56.811 INFO:tasks.workunit.client.0.smithi144.stdout:2/994: mknod d7/d29/d8f/ddc/c145 0 2024-09-10T11:29:56.812 INFO:tasks.workunit.client.0.smithi144.stdout:6/924: mknod d9/d1f/c144 0 2024-09-10T11:29:56.813 INFO:tasks.workunit.client.0.smithi144.stdout:2/995: read d7/f41 [1087630,80296] 0 2024-09-10T11:29:56.814 INFO:tasks.workunit.client.0.smithi144.stdout:0/966: rename da/d3a/d4b/d4e/d62/d78/fff to da/d3a/d94/db1/f127 0 2024-09-10T11:29:56.816 INFO:tasks.workunit.client.0.smithi144.stdout:6/925: dread - d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/f138 zero size 2024-09-10T11:29:56.821 INFO:tasks.workunit.client.0.smithi144.stdout:2/996: mknod d7/de/d36/d45/d47/d6c/d7c/d82/c146 0 2024-09-10T11:29:56.822 INFO:tasks.workunit.client.0.smithi144.stdout:0/967: link da/f1c da/d3a/d4b/d4e/d62/d78/f128 0 2024-09-10T11:29:56.824 INFO:tasks.workunit.client.0.smithi144.stdout:2/997: symlink d7/de/d36/d45/d8e/d112/l147 0 2024-09-10T11:29:56.825 INFO:tasks.workunit.client.0.smithi144.stdout:2/998: chown d7/d29/d92/lf7 105673333 1 2024-09-10T11:29:56.825 INFO:tasks.workunit.client.0.smithi144.stdout:2/999: chown d7/l34 445332669 1 2024-09-10T11:29:56.826 INFO:tasks.workunit.client.0.smithi144.stdout:6/926: rmdir d9/dd/d11/d1b/d2f/d7f/dc4/d5b/df9/d11a/d124 0 2024-09-10T11:29:56.827 INFO:tasks.workunit.client.0.smithi144.stdout:0/968: creat da/d3a/d94/db1/dcc/f129 x:0 0 0 2024-09-10T11:29:56.829 INFO:tasks.workunit.client.0.smithi144.stdout:0/969: write da/d44/d82/fe1 [977544,129035] 0 2024-09-10T11:29:56.830 INFO:tasks.workunit.client.0.smithi144.stdout:0/970: write da/d3a/d94/db1/dcc/fa8 [9165598,871] 0 2024-09-10T11:29:56.832 INFO:tasks.workunit.client.0.smithi144.stdout:0/971: write da/d3a/d4b/d4e/d62/d78/f104 [1284851,59863] 0 2024-09-10T11:29:56.834 INFO:tasks.workunit.client.0.smithi144.stdout:0/972: fsync da/d3a/d4b/d4e/d62/d78/f104 0 2024-09-10T11:29:56.841 INFO:tasks.workunit.client.0.smithi144.stdout:6/927: rename d9/dd/d11/d1b/d2f/d7f/dc4/d5b/df9/d11a to d9/dd/d11/d1b/d2f/d7f/dc4/d5b/dc8/d145 0 2024-09-10T11:29:56.844 INFO:tasks.workunit.client.0.smithi144.stdout:0/973: rename da/d3a/d51/d103 to da/d3a/d4b/d4e/d62/d106/d12a 0 2024-09-10T11:29:56.847 INFO:tasks.workunit.client.0.smithi144.stdout:6/928: rename d9/dd/d3e/d47/fcd to d9/d1f/dae/f146 0 2024-09-10T11:29:56.848 INFO:tasks.workunit.client.0.smithi144.stdout:6/929: stat d9/dd/d3e/l137 0 2024-09-10T11:29:56.848 INFO:tasks.workunit.client.0.smithi144.stdout:0/974: mkdir da/d3a/d51/d8f/d12b 0 2024-09-10T11:29:56.853 INFO:tasks.workunit.client.0.smithi144.stdout:6/930: truncate d9/d1f/d2c/f85 727556 0 2024-09-10T11:29:56.855 INFO:tasks.workunit.client.0.smithi144.stdout:6/931: fsync d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/f51 0 2024-09-10T11:29:56.855 INFO:tasks.workunit.client.0.smithi144.stdout:0/975: rename da/d3a/d4b/d61/ddf/fee to da/d3a/d8e/dab/f12c 0 2024-09-10T11:29:56.857 INFO:tasks.workunit.client.0.smithi144.stdout:0/976: write da/d3a/d4b/d61/d6c/fe4 [1252482,108324] 0 2024-09-10T11:29:56.857 INFO:tasks.workunit.client.0.smithi144.stdout:6/932: rmdir d9/d38 39 2024-09-10T11:29:56.873 INFO:tasks.workunit.client.0.smithi144.stdout:6/933: dread d9/dd/d11/f18 [0,4194304] 0 2024-09-10T11:29:56.877 INFO:tasks.workunit.client.0.smithi144.stdout:6/934: link d9/d1f/d2c/l3d d9/dd/d129/dc9/deb/l147 0 2024-09-10T11:29:56.878 INFO:tasks.workunit.client.0.smithi144.stdout:6/935: truncate d9/dd/d11/d1b/d2f/d7f/dc4/d9e/f141 347731 0 2024-09-10T11:29:56.911 INFO:tasks.workunit.client.0.smithi144.stdout:6/936: dwrite d9/d1f/d2c/d4f/f58 [0,4194304] 0 2024-09-10T11:29:56.913 INFO:tasks.workunit.client.0.smithi144.stdout:6/937: fdatasync d9/dd/d11/d1b/d2f/d7f/f110 0 2024-09-10T11:29:56.918 INFO:tasks.workunit.client.0.smithi144.stdout:6/938: getdents d9/dd/d11/d1b/d2f 0 2024-09-10T11:29:56.918 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:56 smithi178 ceph-mon[33514]: pgmap v35: 65 pgs: 65 active+clean; 3.3 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 49 MiB/s rd, 113 MiB/s wr, 335 op/s 2024-09-10T11:29:56.922 INFO:tasks.workunit.client.0.smithi144.stdout:6/939: creat d9/dd/d11/d1b/d2f/d7f/dc4/d5b/dc8/f148 x:0 0 0 2024-09-10T11:29:56.924 INFO:tasks.workunit.client.0.smithi144.stdout:6/940: chown d9/d38/ff5 121 1 2024-09-10T11:29:56.930 INFO:tasks.workunit.client.0.smithi144.stdout:6/941: link d9/dd/d11/d1b/d2f/dab/de1/fe6 d9/dd/d11/d1b/d2f/dab/de7/d105/f149 0 2024-09-10T11:29:56.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:56 smithi144 ceph-mon[27942]: pgmap v35: 65 pgs: 65 active+clean; 3.3 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 49 MiB/s rd, 113 MiB/s wr, 335 op/s 2024-09-10T11:29:56.935 INFO:tasks.workunit.client.0.smithi144.stdout:6/942: rename d9/d1f/c144 to d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/dde/c14a 0 2024-09-10T11:29:56.938 INFO:tasks.workunit.client.0.smithi144.stdout:6/943: creat d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d10a/f14b x:0 0 0 2024-09-10T11:29:56.948 INFO:tasks.workunit.client.0.smithi144.stdout:6/944: write d9/dd/d11/f21 [348388,15467] 0 2024-09-10T11:29:56.953 INFO:tasks.workunit.client.0.smithi144.stdout:6/945: truncate d9/dd/d11/d1b/d2f/dab/de7/f13a 1312615 0 2024-09-10T11:29:56.954 INFO:tasks.workunit.client.0.smithi144.stdout:6/946: read - d9/dd/d3e/f9b zero size 2024-09-10T11:29:56.955 INFO:tasks.workunit.client.0.smithi144.stdout:6/947: read - d9/dd/d11/d1b/d2f/d7f/fec zero size 2024-09-10T11:29:56.956 INFO:tasks.workunit.client.0.smithi144.stdout:6/948: chown d9/dd/d11/d1b/d2f/d7f/d82/l13e 6496767 1 2024-09-10T11:29:57.276 INFO:tasks.workunit.client.0.smithi144.stdout:0/977: sync 2024-09-10T11:29:57.277 INFO:tasks.workunit.client.0.smithi144.stdout:0/978: chown da/d3a/d51/f9f 34685459 1 2024-09-10T11:29:57.279 INFO:tasks.workunit.client.0.smithi144.stdout:0/979: mknod da/d3a/d4b/c12d 0 2024-09-10T11:29:57.281 INFO:tasks.workunit.client.0.smithi144.stdout:0/980: rename da/d3a/d94/db1/f127 to da/d44/dd8/f12e 0 2024-09-10T11:29:57.286 INFO:tasks.workunit.client.0.smithi144.stdout:0/981: dwrite da/f41 [0,4194304] 0 2024-09-10T11:29:57.307 INFO:tasks.workunit.client.0.smithi144.stdout:0/982: dread da/d44/f63 [0,4194304] 0 2024-09-10T11:29:57.308 INFO:tasks.workunit.client.0.smithi144.stdout:0/983: write da/d44/dd8/f10b [2813408,84742] 0 2024-09-10T11:29:57.313 INFO:tasks.workunit.client.0.smithi144.stdout:0/984: getdents da/d3a/d4b/d4e/d62/d69 0 2024-09-10T11:29:57.316 INFO:tasks.workunit.client.0.smithi144.stdout:0/985: creat da/d3a/d51/d8f/d12b/f12f x:0 0 0 2024-09-10T11:29:57.462 INFO:tasks.workunit.client.0.smithi144.stdout:6/949: sync 2024-09-10T11:29:57.466 INFO:tasks.workunit.client.0.smithi144.stdout:6/950: unlink d9/d1f/c133 0 2024-09-10T11:29:57.467 INFO:tasks.workunit.client.0.smithi144.stdout:6/951: write d9/dd/d3e/f40 [152193,103581] 0 2024-09-10T11:29:57.471 INFO:tasks.workunit.client.0.smithi144.stdout:6/952: getdents d9/dd/d11/d1b/d2f/dab/de7 0 2024-09-10T11:29:57.473 INFO:tasks.workunit.client.0.smithi144.stdout:6/953: mkdir d9/dd/d3e/d47/d14c 0 2024-09-10T11:29:57.473 INFO:tasks.workunit.client.0.smithi144.stdout:6/954: stat l2 0 2024-09-10T11:29:57.584 INFO:tasks.workunit.client.0.smithi144.stdout:0/986: sync 2024-09-10T11:29:57.586 INFO:tasks.workunit.client.0.smithi144.stdout:0/987: rename da/d3a/d4b/d4e/d62/d106/c121 to da/d3a/d4b/d61/c130 0 2024-09-10T11:29:57.604 INFO:tasks.workunit.client.0.smithi144.stdout:0/988: truncate da/f41 1243927 0 2024-09-10T11:29:57.607 INFO:tasks.workunit.client.0.smithi144.stdout:0/989: symlink da/d3a/d51/d8f/d12b/l131 0 2024-09-10T11:29:57.609 INFO:tasks.workunit.client.0.smithi144.stdout:0/990: symlink da/l132 0 2024-09-10T11:29:57.610 INFO:tasks.workunit.client.0.smithi144.stdout:0/991: read da/d3a/d51/fac [3184,25570] 0 2024-09-10T11:29:57.613 INFO:tasks.workunit.client.0.smithi144.stdout:0/992: dread da/d3a/d4b/d61/d6c/fe0 [0,4194304] 0 2024-09-10T11:29:57.616 INFO:tasks.workunit.client.0.smithi144.stdout:0/993: unlink da/d3a/d51/d5f/f4a 0 2024-09-10T11:29:57.619 INFO:tasks.workunit.client.0.smithi144.stdout:0/994: rename da/d3a/d8e/dab/dc7/lce to da/d3a/d4b/dbb/l133 0 2024-09-10T11:29:57.624 INFO:tasks.workunit.client.0.smithi144.stdout:0/995: dwrite da/ffb [0,4194304] 0 2024-09-10T11:29:57.625 INFO:tasks.workunit.client.0.smithi144.stdout:0/996: readlink da/d3a/d4b/d4e/d62/ld9 0 2024-09-10T11:29:57.632 INFO:tasks.workunit.client.0.smithi144.stdout:0/997: dwrite da/d44/d82/fa2 [0,4194304] 0 2024-09-10T11:29:57.640 INFO:tasks.workunit.client.0.smithi144.stdout:0/998: dwrite da/d3a/d4b/d4e/d62/d69/f122 [0,4194304] 0 2024-09-10T11:29:57.642 INFO:tasks.workunit.client.0.smithi144.stdout:0/999: chown da/d3a/d51/c67 7000 1 2024-09-10T11:29:57.682 INFO:tasks.workunit.client.0.smithi144.stdout:6/955: sync 2024-09-10T11:29:57.685 INFO:tasks.workunit.client.0.smithi144.stdout:6/956: truncate d9/dd/d11/d1b/f32 4846574 0 2024-09-10T11:29:57.688 INFO:tasks.workunit.client.0.smithi144.stdout:6/957: getdents d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6 0 2024-09-10T11:29:57.929 INFO:tasks.workunit.client.0.smithi144.stdout:6/958: sync 2024-09-10T11:29:58.172 INFO:tasks.workunit.client.0.smithi144.stdout:6/959: sync 2024-09-10T11:29:58.477 INFO:tasks.workunit.client.0.smithi144.stdout:6/960: sync 2024-09-10T11:29:58.508 INFO:tasks.workunit.client.0.smithi144.stdout:6/961: dwrite d9/dd/d3e/d47/f98 [0,4194304] 0 2024-09-10T11:29:58.510 INFO:tasks.workunit.client.0.smithi144.stdout:6/962: read - d9/dd/d11/d1b/d2f/d7f/dc4/fa5 zero size 2024-09-10T11:29:58.513 INFO:tasks.workunit.client.0.smithi144.stdout:6/963: rmdir d9/dd/d3e/d47/d14c 0 2024-09-10T11:29:58.516 INFO:tasks.workunit.client.0.smithi144.stdout:6/964: mknod d9/dd/d11/d1b/d2f/dab/de7/d105/c14d 0 2024-09-10T11:29:58.518 INFO:tasks.workunit.client.0.smithi144.stdout:6/965: truncate d9/dd/d73/fb1 3143173 0 2024-09-10T11:29:58.524 INFO:tasks.workunit.client.0.smithi144.stdout:6/966: write d9/f2a [172632,90392] 0 2024-09-10T11:29:58.535 INFO:tasks.workunit.client.0.smithi144.stdout:6/967: dread d9/dd/d3e/d47/f28 [4194304,4194304] 0 2024-09-10T11:29:58.537 INFO:tasks.workunit.client.0.smithi144.stdout:6/968: rmdir d9/dd/d11/d135 0 2024-09-10T11:29:58.543 INFO:tasks.workunit.client.0.smithi144.stdout:6/969: link d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/la1 d9/dd/d129/dc9/ddd/l14e 0 2024-09-10T11:29:58.545 INFO:tasks.workunit.client.0.smithi144.stdout:6/970: rmdir d9/dd/d11/d1b/d2f/dab 39 2024-09-10T11:29:58.546 INFO:tasks.workunit.client.0.smithi144.stdout:6/971: write d9/d1f/d2c/f134 [731052,99402] 0 2024-09-10T11:29:58.548 INFO:tasks.workunit.client.0.smithi144.stdout:6/972: symlink d9/dd/d3e/d47/l14f 0 2024-09-10T11:29:58.549 INFO:tasks.workunit.client.0.smithi144.stdout:6/973: chown d9/dd/d11/f18 0 1 2024-09-10T11:29:58.561 INFO:tasks.workunit.client.0.smithi144.stdout:6/974: dwrite d9/dd/d11/d1b/d2f/d7f/dc4/d5b/f61 [0,4194304] 0 2024-09-10T11:29:58.562 INFO:tasks.workunit.client.0.smithi144.stdout:6/975: chown d9/dd/d129/lf7 958898 1 2024-09-10T11:29:58.566 INFO:tasks.workunit.client.0.smithi144.stdout:6/976: write d9/dd/d129/f97 [500963,65427] 0 2024-09-10T11:29:58.578 INFO:tasks.workunit.client.0.smithi144.stdout:6/977: dwrite d9/dd/d11/d1b/d2f/dab/de1/fe6 [0,4194304] 0 2024-09-10T11:29:58.582 INFO:tasks.workunit.client.0.smithi144.stdout:6/978: rename d9/dd/d11/d1b/d2f/dab/de7 to d9/d1f/d2c/d4f/d150 0 2024-09-10T11:29:58.585 INFO:tasks.workunit.client.0.smithi144.stdout:6/979: creat d9/dd/d3e/d47/f151 x:0 0 0 2024-09-10T11:29:58.587 INFO:tasks.workunit.client.0.smithi144.stdout:6/980: read d9/dd/d11/d1b/d2f/d7f/f75 [1153544,43744] 0 2024-09-10T11:29:58.589 INFO:tasks.workunit.client.0.smithi144.stdout:6/981: symlink d9/dd/d129/dc9/ddd/l152 0 2024-09-10T11:29:58.591 INFO:tasks.workunit.client.0.smithi144.stdout:6/982: mkdir d9/dd/d73/d153 0 2024-09-10T11:29:58.593 INFO:tasks.workunit.client.0.smithi144.stdout:6/983: creat d9/dd/d129/dc9/deb/f154 x:0 0 0 2024-09-10T11:29:58.595 INFO:tasks.workunit.client.0.smithi144.stdout:6/984: write d9/dd/d11/d1b/d2f/d7f/dc4/d5b/d91/f51 [1548130,42221] 0 2024-09-10T11:29:58.601 INFO:tasks.workunit.client.0.smithi144.stdout:6/985: write d9/d38/ff3 [587325,57770] 0 2024-09-10T11:29:58.602 INFO:tasks.workunit.client.0.smithi144.stdout:6/986: chown d9/dd/d11/d1b/d2f/dab/dd0/l121 1383798 1 2024-09-10T11:29:58.603 INFO:tasks.workunit.client.0.smithi144.stdout:6/987: write d9/dd/d3e/d47/f88 [1897458,103748] 0 2024-09-10T11:29:58.604 INFO:tasks.workunit.client.0.smithi144.stdout:6/988: stat d9/dd/d11/d1b/d2f/d7f/fec 0 2024-09-10T11:29:58.607 INFO:tasks.workunit.client.0.smithi144.stdout:6/989: mkdir d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d155 0 2024-09-10T11:29:58.612 INFO:tasks.workunit.client.0.smithi144.stdout:6/990: rmdir d9/dd/d11/d1b 39 2024-09-10T11:29:58.616 INFO:tasks.workunit.client.0.smithi144.stdout:6/991: write d9/dd/d11/d1b/d2f/d7f/fc7 [917408,24867] 0 2024-09-10T11:29:58.677 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:58 smithi144 ceph-mon[27942]: pgmap v36: 65 pgs: 65 active+clean; 3.3 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 49 MiB/s rd, 113 MiB/s wr, 335 op/s 2024-09-10T11:29:58.677 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:58 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:58.677 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:58 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.bkogfs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:29:58.678 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:58 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.bkogfs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:29:58.678 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:58 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:29:58.678 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:58 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:29:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:58 smithi178 ceph-mon[33514]: pgmap v36: 65 pgs: 65 active+clean; 3.3 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 49 MiB/s rd, 113 MiB/s wr, 335 op/s 2024-09-10T11:29:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:58 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:29:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:58 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.bkogfs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:29:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:58 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.bkogfs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:29:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:58 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:29:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:58 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:29:59.144 INFO:tasks.workunit.client.0.smithi144.stdout:6/992: sync 2024-09-10T11:29:59.146 INFO:tasks.workunit.client.0.smithi144.stdout:6/993: mkdir d9/dd/d11/d156 0 2024-09-10T11:29:59.151 INFO:tasks.workunit.client.0.smithi144.stdout:6/994: write d9/dd/d11/d1b/d2f/d7f/dc4/d5b/fca [187615,124062] 0 2024-09-10T11:29:59.235 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T11:29:59.236 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-09-10T11:29:59.474 INFO:tasks.workunit.client.0.smithi144.stdout:6/995: sync 2024-09-10T11:29:59.506 INFO:tasks.workunit.client.0.smithi144.stdout:6/996: write d9/dd/d11/d1b/d2f/f66 [1095166,4742] 0 2024-09-10T11:29:59.510 INFO:tasks.workunit.client.0.smithi144.stdout:6/997: write d9/dd/d11/d1b/d2f/dab/fce [768994,131057] 0 2024-09-10T11:29:59.512 INFO:tasks.workunit.client.0.smithi144.stdout:6/998: write d9/dd/d11/d1b/d2f/d7f/dc4/d5b/fca [269114,10831] 0 2024-09-10T11:29:59.514 INFO:tasks.workunit.client.0.smithi144.stdout:6/999: symlink d9/dd/d11/d1b/d2f/d7f/dc4/d9e/dd6/d10a/l157 0 2024-09-10T11:29:59.517 INFO:tasks.workunit.client.0.smithi144.stderr:+ rm -rf -- ./tmp.hgnv0hcqfM 2024-09-10T11:29:59.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:59 smithi178 ceph-mon[33514]: Upgrade: Updating mgr.smithi144.bkogfs 2024-09-10T11:29:59.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:29:59 smithi178 ceph-mon[33514]: Deploying daemon mgr.smithi144.bkogfs on smithi144 2024-09-10T11:29:59.673 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:59 smithi144 ceph-mon[27942]: Upgrade: Updating mgr.smithi144.bkogfs 2024-09-10T11:29:59.673 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:29:59 smithi144 ceph-mon[27942]: Deploying daemon mgr.smithi144.bkogfs on smithi144 2024-09-10T11:30:00.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:00 smithi144 ceph-mon[27942]: pgmap v37: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 67 MiB/s rd, 156 MiB/s wr, 479 op/s 2024-09-10T11:30:00.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:00 smithi144 ceph-mon[27942]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T11:30:00.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:00 smithi144 ceph-mon[27942]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:30:00.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:00 smithi144 ceph-mon[27942]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:30:00.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:00 smithi178 ceph-mon[33514]: pgmap v37: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 67 MiB/s rd, 156 MiB/s wr, 479 op/s 2024-09-10T11:30:00.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:00 smithi178 ceph-mon[33514]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T11:30:00.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:00 smithi178 ceph-mon[33514]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:30:00.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:00 smithi178 ceph-mon[33514]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:30:02.646 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:02 smithi144 ceph-mon[27942]: pgmap v38: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 34 MiB/s rd, 86 MiB/s wr, 294 op/s 2024-09-10T11:30:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:02 smithi178 ceph-mon[33514]: pgmap v38: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 34 MiB/s rd, 86 MiB/s wr, 294 op/s 2024-09-10T11:30:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:04 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:04 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:04 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:30:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:04 smithi178 ceph-mon[33514]: pgmap v39: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 35 MiB/s rd, 89 MiB/s wr, 396 op/s 2024-09-10T11:30:04.411 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:04 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:04.411 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:04 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:04.411 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:04 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:30:04.411 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:04 smithi144 ceph-mon[27942]: pgmap v39: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 35 MiB/s rd, 89 MiB/s wr, 396 op/s 2024-09-10T11:30:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:06 smithi144 ceph-mon[27942]: pgmap v40: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 19 MiB/s rd, 46 MiB/s wr, 247 op/s 2024-09-10T11:30:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:06 smithi178 ceph-mon[33514]: pgmap v40: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 19 MiB/s rd, 46 MiB/s wr, 247 op/s 2024-09-10T11:30:07.901 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:07 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:30:08.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:07 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:30:08.807 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:08 smithi144 ceph-mon[27942]: pgmap v41: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 19 MiB/s rd, 46 MiB/s wr, 247 op/s 2024-09-10T11:30:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:08 smithi178 ceph-mon[33514]: pgmap v41: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 19 MiB/s rd, 46 MiB/s wr, 247 op/s 2024-09-10T11:30:10.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:09 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:10.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:09 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:10.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:09 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:10.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:09 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:10.130 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:09 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:10.130 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:09 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:10.130 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:09 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:10.130 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:09 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:10.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:10 smithi144 ceph-mon[27942]: pgmap v42: 65 pgs: 65 active+clean; 1.3 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 20 MiB/s rd, 47 MiB/s wr, 333 op/s 2024-09-10T11:30:10.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:10 smithi144 ceph-mon[27942]: Standby manager daemon smithi144.bkogfs restarted 2024-09-10T11:30:10.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:10 smithi144 ceph-mon[27942]: Standby manager daemon smithi144.bkogfs started 2024-09-10T11:30:10.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:10 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.bkogfs/crt"}]: dispatch 2024-09-10T11:30:10.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:10 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T11:30:10.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:10 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.bkogfs/key"}]: dispatch 2024-09-10T11:30:10.937 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:10 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T11:30:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:10 smithi178 ceph-mon[33514]: pgmap v42: 65 pgs: 65 active+clean; 1.3 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 20 MiB/s rd, 47 MiB/s wr, 333 op/s 2024-09-10T11:30:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:10 smithi178 ceph-mon[33514]: Standby manager daemon smithi144.bkogfs restarted 2024-09-10T11:30:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:10 smithi178 ceph-mon[33514]: Standby manager daemon smithi144.bkogfs started 2024-09-10T11:30:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:10 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.bkogfs/crt"}]: dispatch 2024-09-10T11:30:11.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:10 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T11:30:11.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:10 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi144.bkogfs/key"}]: dispatch 2024-09-10T11:30:11.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:10 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T11:30:11.331 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T11:30:11.332 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T11:30:11.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:11 smithi144 ceph-mon[27942]: mgrmap e26: smithi178.vmvxuf(active, since 79s), standbys: smithi144.bkogfs 2024-09-10T11:30:12.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:11 smithi178 ceph-mon[33514]: mgrmap e26: smithi178.vmvxuf(active, since 79s), standbys: smithi144.bkogfs 2024-09-10T11:30:13.096 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:12 smithi144 ceph-mon[27942]: pgmap v43: 65 pgs: 65 active+clean; 1.3 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 189 op/s 2024-09-10T11:30:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:12 smithi178 ceph-mon[33514]: pgmap v43: 65 pgs: 65 active+clean; 1.3 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 1.9 MiB/s rd, 3.8 MiB/s wr, 189 op/s 2024-09-10T11:30:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:30:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:30:14.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:14.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:30:14.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mgr fail", "who": "smithi178.vmvxuf"}]: dispatch 2024-09-10T11:30:14.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mgr fail", "who": "smithi178.vmvxuf"}]: dispatch 2024-09-10T11:30:14.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: osdmap e49: 6 total, 6 up, 6 in 2024-09-10T11:30:14.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd='[{"prefix": "mgr fail", "who": "smithi178.vmvxuf"}]': finished 2024-09-10T11:30:14.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: mgrmap e27: smithi144.bkogfs(active, starting, since 0.00883981s) 2024-09-10T11:30:14.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:30:14.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:30:14.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jaoyvs"}]: dispatch 2024-09-10T11:30:14.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:30:14.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.nmhrmh"}]: dispatch 2024-09-10T11:30:14.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.czjrsa"}]: dispatch 2024-09-10T11:30:14.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr metadata", "who": "smithi144.bkogfs", "id": "smithi144.bkogfs"}]: dispatch 2024-09-10T11:30:14.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:30:14.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:30:14.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:30:14.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:30:14.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:30:14.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:30:14.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T11:30:14.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T11:30:14.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T11:30:14.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: Manager daemon smithi144.bkogfs is now available 2024-09-10T11:30:14.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:30:14.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:30:14.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/mirror_snapshot_schedule"}]: dispatch 2024-09-10T11:30:14.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/mirror_snapshot_schedule"}]: dispatch 2024-09-10T11:30:14.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/trash_purge_schedule"}]: dispatch 2024-09-10T11:30:14.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:14 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/trash_purge_schedule"}]: dispatch 2024-09-10T11:30:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:14.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:14.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:30:14.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:30:14.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' 2024-09-10T11:30:14.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:30:14.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24329 172.21.15.178:0/3219641481' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mgr fail", "who": "smithi178.vmvxuf"}]: dispatch 2024-09-10T11:30:14.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "mgr fail", "who": "smithi178.vmvxuf"}]: dispatch 2024-09-10T11:30:14.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: osdmap e49: 6 total, 6 up, 6 in 2024-09-10T11:30:14.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24329 ' entity='mgr.smithi178.vmvxuf' cmd='[{"prefix": "mgr fail", "who": "smithi178.vmvxuf"}]': finished 2024-09-10T11:30:14.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: mgrmap e27: smithi144.bkogfs(active, starting, since 0.00883981s) 2024-09-10T11:30:14.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:30:14.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:30:14.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jaoyvs"}]: dispatch 2024-09-10T11:30:14.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:30:14.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.nmhrmh"}]: dispatch 2024-09-10T11:30:14.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.czjrsa"}]: dispatch 2024-09-10T11:30:14.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr metadata", "who": "smithi144.bkogfs", "id": "smithi144.bkogfs"}]: dispatch 2024-09-10T11:30:14.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:30:14.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:30:14.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:30:14.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:30:14.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:30:14.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:30:14.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T11:30:14.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T11:30:14.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T11:30:14.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: Manager daemon smithi144.bkogfs is now available 2024-09-10T11:30:14.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:30:14.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:30:14.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/mirror_snapshot_schedule"}]: dispatch 2024-09-10T11:30:14.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/mirror_snapshot_schedule"}]: dispatch 2024-09-10T11:30:14.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/trash_purge_schedule"}]: dispatch 2024-09-10T11:30:14.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:14 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/trash_purge_schedule"}]: dispatch 2024-09-10T11:30:15.605 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:15 smithi144 ceph-mon[27942]: mgrmap e28: smithi144.bkogfs(active, since 1.00898s) 2024-09-10T11:30:15.605 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:15 smithi144 ceph-mon[27942]: pgmap v3: 65 pgs: 65 active+clean; 294 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:30:15.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:15 smithi178 ceph-mon[33514]: mgrmap e28: smithi144.bkogfs(active, since 1.00898s) 2024-09-10T11:30:15.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:15 smithi178 ceph-mon[33514]: pgmap v3: 65 pgs: 65 active+clean; 294 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:30:16.579 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:16 smithi144 ceph-mon[27942]: [10/Sep/2024:11:30:15] ENGINE Bus STARTING 2024-09-10T11:30:16.579 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:16 smithi144 ceph-mon[27942]: [10/Sep/2024:11:30:15] ENGINE Serving on https://172.21.15.144:7150 2024-09-10T11:30:16.579 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:16 smithi144 ceph-mon[27942]: [10/Sep/2024:11:30:15] ENGINE Error in HTTPServer.serve 2024-09-10T11:30:16.579 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Traceback (most recent call last): 2024-09-10T11:30:16.580 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T11:30:16.580 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: self._connections.run(self.expiration_interval) 2024-09-10T11:30:16.580 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T11:30:16.580 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: self._run(expiration_interval) 2024-09-10T11:30:16.580 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T11:30:16.580 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T11:30:16.580 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T11:30:16.580 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T11:30:16.580 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T11:30:16.580 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: s = self.context.wrap_socket( 2024-09-10T11:30:16.580 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T11:30:16.580 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: return self.sslsocket_class._create( 2024-09-10T11:30:16.580 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T11:30:16.581 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: self.do_handshake() 2024-09-10T11:30:16.581 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T11:30:16.581 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: self._sslobj.do_handshake() 2024-09-10T11:30:16.581 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T11:30:16.581 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:16 smithi144 ceph-mon[27942]: [10/Sep/2024:11:30:15] ENGINE Serving on http://172.21.15.144:8765 2024-09-10T11:30:16.581 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:16 smithi144 ceph-mon[27942]: [10/Sep/2024:11:30:15] ENGINE Bus STARTED 2024-09-10T11:30:16.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.577+0000 7fde17425700 1 -- 172.21.15.144:0/1207150404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde10101350 msgr2=0x7fde101017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:16.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.577+0000 7fde17425700 1 --2- 172.21.15.144:0/1207150404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde10101350 0x7fde101017a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fde00009910 tx=0x7fde00009c20 comp rx=0 tx=0).stop 2024-09-10T11:30:16.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.577+0000 7fde17425700 1 -- 172.21.15.144:0/1207150404 shutdown_connections 2024-09-10T11:30:16.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.577+0000 7fde17425700 1 --2- 172.21.15.144:0/1207150404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde10101350 0x7fde101017a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:16.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.577+0000 7fde17425700 1 --2- 172.21.15.144:0/1207150404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde10100150 0x7fde10100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:16.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.577+0000 7fde17425700 1 -- 172.21.15.144:0/1207150404 >> 172.21.15.144:0/1207150404 conn(0x7fde100fb6e0 msgr2=0x7fde100fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:16.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.578+0000 7fde17425700 1 -- 172.21.15.144:0/1207150404 shutdown_connections 2024-09-10T11:30:16.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.578+0000 7fde17425700 1 -- 172.21.15.144:0/1207150404 wait complete. 2024-09-10T11:30:16.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.578+0000 7fde17425700 1 Processor -- start 2024-09-10T11:30:16.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.578+0000 7fde17425700 1 -- start start 2024-09-10T11:30:16.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.579+0000 7fde17425700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde10100150 0x7fde10074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.579+0000 7fde17425700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde10101350 0x7fde10073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.579+0000 7fde17425700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde10073680 con 0x7fde10101350 2024-09-10T11:30:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.579+0000 7fde17425700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde100737c0 con 0x7fde10100150 2024-09-10T11:30:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.579+0000 7fde149c0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde10101350 0x7fde10073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.579+0000 7fde149c0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde10101350 0x7fde10073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56726/0 (socket says 172.21.15.144:56726) 2024-09-10T11:30:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.579+0000 7fde149c0700 1 -- 172.21.15.144:0/1423883175 learned_addr learned my addr 172.21.15.144:0/1423883175 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:30:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.579+0000 7fde151c1700 1 --2- 172.21.15.144:0/1423883175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde10100150 0x7fde10074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.579+0000 7fde149c0700 1 -- 172.21.15.144:0/1423883175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde10100150 msgr2=0x7fde10074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.579+0000 7fde149c0700 1 --2- 172.21.15.144:0/1423883175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde10100150 0x7fde10074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.579+0000 7fde149c0700 1 -- 172.21.15.144:0/1423883175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde000095f0 con 0x7fde10101350 2024-09-10T11:30:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.579+0000 7fde149c0700 1 --2- 172.21.15.144:0/1423883175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde10101350 0x7fde10073140 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fde00000c00 tx=0x7fde0000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.579+0000 7fde067fc700 1 -- 172.21.15.144:0/1423883175 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde0001d070 con 0x7fde10101350 2024-09-10T11:30:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.580+0000 7fde17425700 1 -- 172.21.15.144:0/1423883175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde10073a40 con 0x7fde10101350 2024-09-10T11:30:16.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.580+0000 7fde067fc700 1 -- 172.21.15.144:0/1423883175 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fde00005dc0 con 0x7fde10101350 2024-09-10T11:30:16.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.580+0000 7fde067fc700 1 -- 172.21.15.144:0/1423883175 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde000226a0 con 0x7fde10101350 2024-09-10T11:30:16.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.580+0000 7fde17425700 1 -- 172.21.15.144:0/1423883175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde10073f30 con 0x7fde10101350 2024-09-10T11:30:16.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.581+0000 7fde067fc700 1 -- 172.21.15.144:0/1423883175 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 29) v1 ==== 46428+0+0 (secure 0 0 0) 0x7fde00022800 con 0x7fde10101350 2024-09-10T11:30:16.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.581+0000 7fde17425700 1 -- 172.21.15.144:0/1423883175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde10108ca0 con 0x7fde10101350 2024-09-10T11:30:16.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.581+0000 7fde067fc700 1 --2- 172.21.15.144:0/1423883175 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fddfc039360 0x7fddfc03b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:16.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.581+0000 7fde067fc700 1 -- 172.21.15.144:0/1423883175 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fde0004ec30 con 0x7fde10101350 2024-09-10T11:30:16.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.581+0000 7fde151c1700 1 --2- 172.21.15.144:0/1423883175 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fddfc039360 0x7fddfc03b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:16.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.582+0000 7fde151c1700 1 --2- 172.21.15.144:0/1423883175 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fddfc039360 0x7fddfc03b810 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fde101011b0 tx=0x7fde0c009260 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:16.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.587+0000 7fde067fc700 1 -- 172.21.15.144:0/1423883175 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7fde00055080 con 0x7fde10101350 2024-09-10T11:30:16.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:16 smithi178 ceph-mon[33514]: [10/Sep/2024:11:30:15] ENGINE Bus STARTING 2024-09-10T11:30:16.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:16 smithi178 ceph-mon[33514]: [10/Sep/2024:11:30:15] ENGINE Serving on https://172.21.15.144:7150 2024-09-10T11:30:16.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:16 smithi178 ceph-mon[33514]: [10/Sep/2024:11:30:15] ENGINE Error in HTTPServer.serve 2024-09-10T11:30:16.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Traceback (most recent call last): 2024-09-10T11:30:16.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T11:30:16.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: self._connections.run(self.expiration_interval) 2024-09-10T11:30:16.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T11:30:16.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: self._run(expiration_interval) 2024-09-10T11:30:16.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T11:30:16.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T11:30:16.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T11:30:16.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T11:30:16.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T11:30:16.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: s = self.context.wrap_socket( 2024-09-10T11:30:16.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T11:30:16.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: return self.sslsocket_class._create( 2024-09-10T11:30:16.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T11:30:16.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: self.do_handshake() 2024-09-10T11:30:16.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T11:30:16.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: self._sslobj.do_handshake() 2024-09-10T11:30:16.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T11:30:16.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:16 smithi178 ceph-mon[33514]: [10/Sep/2024:11:30:15] ENGINE Serving on http://172.21.15.144:8765 2024-09-10T11:30:16.619 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:16 smithi178 ceph-mon[33514]: [10/Sep/2024:11:30:15] ENGINE Bus STARTED 2024-09-10T11:30:16.927 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.925+0000 7fde17425700 1 -- 172.21.15.144:0/1423883175 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde10105ca0 con 0x7fddfc039360 2024-09-10T11:30:16.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.927+0000 7fde067fc700 1 -- 172.21.15.144:0/1423883175 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fde10105ca0 con 0x7fddfc039360 2024-09-10T11:30:16.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.929+0000 7fde17425700 1 -- 172.21.15.144:0/1423883175 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fddfc039360 msgr2=0x7fddfc03b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:16.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.929+0000 7fde17425700 1 --2- 172.21.15.144:0/1423883175 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fddfc039360 0x7fddfc03b810 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fde101011b0 tx=0x7fde0c009260 comp rx=0 tx=0).stop 2024-09-10T11:30:16.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.929+0000 7fde17425700 1 -- 172.21.15.144:0/1423883175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde10101350 msgr2=0x7fde10073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:16.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.929+0000 7fde17425700 1 --2- 172.21.15.144:0/1423883175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde10101350 0x7fde10073140 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fde00000c00 tx=0x7fde0000f690 comp rx=0 tx=0).stop 2024-09-10T11:30:16.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.929+0000 7fde17425700 1 -- 172.21.15.144:0/1423883175 shutdown_connections 2024-09-10T11:30:16.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.929+0000 7fde17425700 1 --2- 172.21.15.144:0/1423883175 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fddfc039360 0x7fddfc03b810 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:16.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.929+0000 7fde17425700 1 --2- 172.21.15.144:0/1423883175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde10100150 0x7fde10074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:16.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.929+0000 7fde17425700 1 --2- 172.21.15.144:0/1423883175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde10101350 0x7fde10073140 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:16.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.929+0000 7fde17425700 1 -- 172.21.15.144:0/1423883175 >> 172.21.15.144:0/1423883175 conn(0x7fde100fb6e0 msgr2=0x7fde10104580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:16.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.929+0000 7fde17425700 1 -- 172.21.15.144:0/1423883175 shutdown_connections 2024-09-10T11:30:16.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:16.929+0000 7fde17425700 1 -- 172.21.15.144:0/1423883175 wait complete. 2024-09-10T11:30:16.941 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:30:17.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.061+0000 7f8f61571700 1 -- 172.21.15.144:0/4083612435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f5c100250 msgr2=0x7f8f5c1006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:17.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.061+0000 7f8f61571700 1 --2- 172.21.15.144:0/4083612435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f5c100250 0x7f8f5c1006a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009910 tx=0x7f8f50009c20 comp rx=0 tx=0).stop 2024-09-10T11:30:17.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.061+0000 7f8f61571700 1 -- 172.21.15.144:0/4083612435 shutdown_connections 2024-09-10T11:30:17.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.061+0000 7f8f61571700 1 --2- 172.21.15.144:0/4083612435 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f5c100250 0x7f8f5c1006a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.061+0000 7f8f61571700 1 --2- 172.21.15.144:0/4083612435 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f5c0ff050 0x7f8f5c0ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.061+0000 7f8f61571700 1 -- 172.21.15.144:0/4083612435 >> 172.21.15.144:0/4083612435 conn(0x7f8f5c0fa600 msgr2=0x7f8f5c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:17.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.062+0000 7f8f61571700 1 -- 172.21.15.144:0/4083612435 shutdown_connections 2024-09-10T11:30:17.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.062+0000 7f8f61571700 1 -- 172.21.15.144:0/4083612435 wait complete. 2024-09-10T11:30:17.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.062+0000 7f8f61571700 1 Processor -- start 2024-09-10T11:30:17.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.063+0000 7f8f61571700 1 -- start start 2024-09-10T11:30:17.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.063+0000 7f8f61571700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f5c0ff050 0x7f8f5c194860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:17.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.063+0000 7f8f61571700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f5c100250 0x7f8f5c194da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:17.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.063+0000 7f8f61571700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c1953c0 con 0x7f8f5c0ff050 2024-09-10T11:30:17.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.063+0000 7f8f61571700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c195500 con 0x7f8f5c100250 2024-09-10T11:30:17.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.063+0000 7f8f5affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f5c0ff050 0x7f8f5c194860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:17.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.063+0000 7f8f5affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f5c0ff050 0x7f8f5c194860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56740/0 (socket says 172.21.15.144:56740) 2024-09-10T11:30:17.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.063+0000 7f8f5affd700 1 -- 172.21.15.144:0/1337832077 learned_addr learned my addr 172.21.15.144:0/1337832077 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:30:17.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.063+0000 7f8f5a7fc700 1 --2- 172.21.15.144:0/1337832077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f5c100250 0x7f8f5c194da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:17.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.063+0000 7f8f5affd700 1 -- 172.21.15.144:0/1337832077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f5c100250 msgr2=0x7f8f5c194da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:17.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.063+0000 7f8f5affd700 1 --2- 172.21.15.144:0/1337832077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f5c100250 0x7f8f5c194da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.063+0000 7f8f5affd700 1 -- 172.21.15.144:0/1337832077 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f500095f0 con 0x7f8f5c0ff050 2024-09-10T11:30:17.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.064+0000 7f8f5affd700 1 --2- 172.21.15.144:0/1337832077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f5c0ff050 0x7f8f5c194860 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00e8f0 tx=0x7f8f4c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:17.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.064+0000 7f8f43fff700 1 -- 172.21.15.144:0/1337832077 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f4c009790 con 0x7f8f5c0ff050 2024-09-10T11:30:17.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.064+0000 7f8f61571700 1 -- 172.21.15.144:0/1337832077 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5c068e00 con 0x7f8f5c0ff050 2024-09-10T11:30:17.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.064+0000 7f8f43fff700 1 -- 172.21.15.144:0/1337832077 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8f4c00c810 con 0x7f8f5c0ff050 2024-09-10T11:30:17.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.064+0000 7f8f43fff700 1 -- 172.21.15.144:0/1337832077 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f4c003d20 con 0x7f8f5c0ff050 2024-09-10T11:30:17.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.064+0000 7f8f61571700 1 -- 172.21.15.144:0/1337832077 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5c069350 con 0x7f8f5c0ff050 2024-09-10T11:30:17.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.065+0000 7f8f43fff700 1 -- 172.21.15.144:0/1337832077 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 29) v1 ==== 46428+0+0 (secure 0 0 0) 0x7f8f4c0041f0 con 0x7f8f5c0ff050 2024-09-10T11:30:17.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.065+0000 7f8f61571700 1 -- 172.21.15.144:0/1337832077 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f5c107ba0 con 0x7f8f5c0ff050 2024-09-10T11:30:17.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.065+0000 7f8f43fff700 1 --2- 172.21.15.144:0/1337832077 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f8f44039360 0x7f8f4403b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:17.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.066+0000 7f8f5a7fc700 1 --2- 172.21.15.144:0/1337832077 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f8f44039360 0x7f8f4403b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:17.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.066+0000 7f8f43fff700 1 -- 172.21.15.144:0/1337832077 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f8f4c014070 con 0x7f8f5c0ff050 2024-09-10T11:30:17.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.066+0000 7f8f5a7fc700 1 --2- 172.21.15.144:0/1337832077 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f8f44039360 0x7f8f4403b810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000b6f0 tx=0x7f8f50005fb0 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:17.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.072+0000 7f8f43fff700 1 -- 172.21.15.144:0/1337832077 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f8f4c01f340 con 0x7f8f5c0ff050 2024-09-10T11:30:17.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.344+0000 7f8f61571700 1 -- 172.21.15.144:0/1337832077 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f5c104ba0 con 0x7f8f44039360 2024-09-10T11:30:17.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.345+0000 7f8f43fff700 1 -- 172.21.15.144:0/1337832077 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f8f5c104ba0 con 0x7f8f44039360 2024-09-10T11:30:17.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.348+0000 7f8f61571700 1 -- 172.21.15.144:0/1337832077 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f8f44039360 msgr2=0x7f8f4403b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:17.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.348+0000 7f8f61571700 1 --2- 172.21.15.144:0/1337832077 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f8f44039360 0x7f8f4403b810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8f5000b6f0 tx=0x7f8f50005fb0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.348+0000 7f8f61571700 1 -- 172.21.15.144:0/1337832077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f5c0ff050 msgr2=0x7f8f5c194860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:17.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.348+0000 7f8f61571700 1 --2- 172.21.15.144:0/1337832077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f5c0ff050 0x7f8f5c194860 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8f4c00e8f0 tx=0x7f8f4c00ec00 comp rx=0 tx=0).stop 2024-09-10T11:30:17.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.348+0000 7f8f61571700 1 -- 172.21.15.144:0/1337832077 shutdown_connections 2024-09-10T11:30:17.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.348+0000 7f8f61571700 1 --2- 172.21.15.144:0/1337832077 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f8f44039360 0x7f8f4403b810 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.348+0000 7f8f61571700 1 --2- 172.21.15.144:0/1337832077 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f5c0ff050 0x7f8f5c194860 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.348+0000 7f8f61571700 1 --2- 172.21.15.144:0/1337832077 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f5c100250 0x7f8f5c194da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.348+0000 7f8f61571700 1 -- 172.21.15.144:0/1337832077 >> 172.21.15.144:0/1337832077 conn(0x7f8f5c0fa600 msgr2=0x7f8f5c103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:17.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.348+0000 7f8f61571700 1 -- 172.21.15.144:0/1337832077 shutdown_connections 2024-09-10T11:30:17.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.348+0000 7f8f61571700 1 -- 172.21.15.144:0/1337832077 wait complete. 2024-09-10T11:30:17.479 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.478+0000 7fc184621700 1 -- 172.21.15.144:0/4204433151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc17c100c80 msgr2=0x7fc17c103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:17.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.478+0000 7fc184621700 1 --2- 172.21.15.144:0/4204433151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc17c100c80 0x7fc17c103060 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc178009910 tx=0x7fc178009c20 comp rx=0 tx=0).stop 2024-09-10T11:30:17.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.478+0000 7fc184621700 1 -- 172.21.15.144:0/4204433151 shutdown_connections 2024-09-10T11:30:17.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.478+0000 7fc184621700 1 --2- 172.21.15.144:0/4204433151 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c1035a0 0x7fc17c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.478+0000 7fc184621700 1 --2- 172.21.15.144:0/4204433151 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc17c100c80 0x7fc17c103060 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.478+0000 7fc184621700 1 -- 172.21.15.144:0/4204433151 >> 172.21.15.144:0/4204433151 conn(0x7fc17c0fa5e0 msgr2=0x7fc17c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:17.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.478+0000 7fc184621700 1 -- 172.21.15.144:0/4204433151 shutdown_connections 2024-09-10T11:30:17.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.478+0000 7fc184621700 1 -- 172.21.15.144:0/4204433151 wait complete. 2024-09-10T11:30:17.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.478+0000 7fc184621700 1 Processor -- start 2024-09-10T11:30:17.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.478+0000 7fc184621700 1 -- start start 2024-09-10T11:30:17.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.479+0000 7fc184621700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc17c100c80 0x7fc17c1005e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:17.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.479+0000 7fc184621700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c1035a0 0x7fc17c0fec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:17.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.479+0000 7fc184621700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c0ff170 con 0x7fc17c100c80 2024-09-10T11:30:17.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.479+0000 7fc184621700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17c0ff2b0 con 0x7fc17c1035a0 2024-09-10T11:30:17.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.479+0000 7fc1823bd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc17c100c80 0x7fc17c1005e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:17.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.479+0000 7fc1823bd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc17c100c80 0x7fc17c1005e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56758/0 (socket says 172.21.15.144:56758) 2024-09-10T11:30:17.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.479+0000 7fc1823bd700 1 -- 172.21.15.144:0/3096709694 learned_addr learned my addr 172.21.15.144:0/3096709694 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:30:17.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.479+0000 7fc181bbc700 1 --2- 172.21.15.144:0/3096709694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c1035a0 0x7fc17c0fec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:17.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.479+0000 7fc1823bd700 1 -- 172.21.15.144:0/3096709694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c1035a0 msgr2=0x7fc17c0fec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:17.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.479+0000 7fc1823bd700 1 --2- 172.21.15.144:0/3096709694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c1035a0 0x7fc17c0fec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.479+0000 7fc1823bd700 1 -- 172.21.15.144:0/3096709694 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1780095f0 con 0x7fc17c100c80 2024-09-10T11:30:17.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.479+0000 7fc1823bd700 1 --2- 172.21.15.144:0/3096709694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc17c100c80 0x7fc17c1005e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc178000c00 tx=0x7fc17800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:17.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.479+0000 7fc1737fe700 1 -- 172.21.15.144:0/3096709694 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc17801d070 con 0x7fc17c100c80 2024-09-10T11:30:17.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.480+0000 7fc1737fe700 1 -- 172.21.15.144:0/3096709694 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc178005dc0 con 0x7fc17c100c80 2024-09-10T11:30:17.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.480+0000 7fc184621700 1 -- 172.21.15.144:0/3096709694 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc17c0ff530 con 0x7fc17c100c80 2024-09-10T11:30:17.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.480+0000 7fc1737fe700 1 -- 172.21.15.144:0/3096709694 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1780226a0 con 0x7fc17c100c80 2024-09-10T11:30:17.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.480+0000 7fc184621700 1 -- 172.21.15.144:0/3096709694 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc17c0ffa20 con 0x7fc17c100c80 2024-09-10T11:30:17.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.481+0000 7fc1737fe700 1 -- 172.21.15.144:0/3096709694 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 29) v1 ==== 46428+0+0 (secure 0 0 0) 0x7fc178022800 con 0x7fc17c100c80 2024-09-10T11:30:17.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.481+0000 7fc1737fe700 1 --2- 172.21.15.144:0/3096709694 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fc168039360 0x7fc16803b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:17.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.481+0000 7fc1737fe700 1 -- 172.21.15.144:0/3096709694 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fc17804ea40 con 0x7fc17c100c80 2024-09-10T11:30:17.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.481+0000 7fc181bbc700 1 --2- 172.21.15.144:0/3096709694 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fc168039360 0x7fc16803b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:17.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.481+0000 7fc184621700 1 -- 172.21.15.144:0/3096709694 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc17c18fb80 con 0x7fc17c100c80 2024-09-10T11:30:17.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.482+0000 7fc181bbc700 1 --2- 172.21.15.144:0/3096709694 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fc168039360 0x7fc16803b810 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009c60 tx=0x7fc16c009260 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:17.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.489+0000 7fc1737fe700 1 -- 172.21.15.144:0/3096709694 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7fc178020de0 con 0x7fc17c100c80 2024-09-10T11:30:17.610 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:17 smithi178 ceph-mon[33514]: mgrmap e29: smithi144.bkogfs(active, since 3s) 2024-09-10T11:30:17.610 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:17 smithi178 ceph-mon[33514]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:30:17.610 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:17 smithi178 ceph-mon[33514]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:17.661 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:17 smithi144 ceph-mon[27942]: mgrmap e29: smithi144.bkogfs(active, since 3s) 2024-09-10T11:30:17.661 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:17 smithi144 ceph-mon[27942]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:30:17.661 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:17 smithi144 ceph-mon[27942]: from='client.14688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:17.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.772+0000 7fc184621700 1 -- 172.21.15.144:0/3096709694 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc17c061190 con 0x7fc168039360 2024-09-10T11:30:17.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.790+0000 7fc1737fe700 1 -- 172.21.15.144:0/3096709694 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fc17c061190 con 0x7fc168039360 2024-09-10T11:30:17.792 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:30:17.792 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (7m) 8s ago 8m 24.6M - 0.25.0 c8568f914cd2 b5c0bfae59de 2024-09-10T11:30:17.792 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (9m) 8s ago 9m 7768k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:30:17.792 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (8m) 71s ago 8m 7629k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:30:17.792 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (9m) 8s ago 9m 6665k - 18.2.0 dc2bc1663786 3311fb04aa0c 2024-09-10T11:30:17.792 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (8m) 71s ago 8m 6651k - 18.2.0 dc2bc1663786 6e408e8476d4 2024-09-10T11:30:17.792 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (7m) 8s ago 8m 86.5M - 9.4.7 954c08fa6188 e9a1aec955f6 2024-09-10T11:30:17.792 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (3m) 8s ago 3m 148M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:30:17.792 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (3m) 8s ago 3m 166M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:30:17.792 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (3m) 71s ago 3m 60.0M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:30:17.793 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (3m) 71s ago 3m 90.1M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:30:17.793 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (15s) 8s ago 10m 64.2M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:30:17.793 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (99s) 71s ago 8m 480M - 19.1.1-305-g1cf819b7 41d671128357 859a740b57c8 2024-09-10T11:30:17.793 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (10m) 8s ago 10m 50.0M 2048M 18.2.0 dc2bc1663786 42addb654e4f 2024-09-10T11:30:17.793 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (8m) 71s ago 8m 40.4M 2048M 18.2.0 dc2bc1663786 3c1ba210ed3f 2024-09-10T11:30:17.793 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (9m) 8s ago 9m 16.0M - 1.5.0 0da6a335fe13 dfbf0c488d17 2024-09-10T11:30:17.793 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (8m) 71s ago 8m 15.2M - 1.5.0 0da6a335fe13 ff2cac118c8d 2024-09-10T11:30:17.793 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (7m) 8s ago 7m 190M 1216M 18.2.0 dc2bc1663786 2435febebdb5 2024-09-10T11:30:17.793 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (6m) 8s ago 6m 204M 1216M 18.2.0 dc2bc1663786 b55fe920ff9d 2024-09-10T11:30:17.793 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (6m) 8s ago 6m 157M 1216M 18.2.0 dc2bc1663786 3ed5618eac33 2024-09-10T11:30:17.793 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (5m) 71s ago 5m 76.3M 2239M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:30:17.793 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (5m) 71s ago 5m 70.5M 2239M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:30:17.793 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (5m) 71s ago 5m 68.7M 2239M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:30:17.794 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (33s) 8s ago 8m 51.6M - 2.43.0 a07b618ecd1d f2d2df915440 2024-09-10T11:30:17.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.794+0000 7fc184621700 1 -- 172.21.15.144:0/3096709694 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fc168039360 msgr2=0x7fc16803b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:17.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.794+0000 7fc184621700 1 --2- 172.21.15.144:0/3096709694 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fc168039360 0x7fc16803b810 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc16c009c60 tx=0x7fc16c009260 comp rx=0 tx=0).stop 2024-09-10T11:30:17.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.794+0000 7fc184621700 1 -- 172.21.15.144:0/3096709694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc17c100c80 msgr2=0x7fc17c1005e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:17.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.794+0000 7fc184621700 1 --2- 172.21.15.144:0/3096709694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc17c100c80 0x7fc17c1005e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc178000c00 tx=0x7fc17800f690 comp rx=0 tx=0).stop 2024-09-10T11:30:17.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.794+0000 7fc184621700 1 -- 172.21.15.144:0/3096709694 shutdown_connections 2024-09-10T11:30:17.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.794+0000 7fc184621700 1 --2- 172.21.15.144:0/3096709694 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fc168039360 0x7fc16803b810 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.794+0000 7fc184621700 1 --2- 172.21.15.144:0/3096709694 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc17c100c80 0x7fc17c1005e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.794+0000 7fc184621700 1 --2- 172.21.15.144:0/3096709694 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc17c1035a0 0x7fc17c0fec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.794+0000 7fc184621700 1 -- 172.21.15.144:0/3096709694 >> 172.21.15.144:0/3096709694 conn(0x7fc17c0fa5e0 msgr2=0x7fc17c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:17.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.794+0000 7fc184621700 1 -- 172.21.15.144:0/3096709694 shutdown_connections 2024-09-10T11:30:17.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.794+0000 7fc184621700 1 -- 172.21.15.144:0/3096709694 wait complete. 2024-09-10T11:30:17.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.939+0000 7f1b5c5ba700 1 -- 172.21.15.144:0/1186002573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b54100250 msgr2=0x7f1b541006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:17.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.939+0000 7f1b5c5ba700 1 --2- 172.21.15.144:0/1186002573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b54100250 0x7f1b541006a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009910 tx=0x7f1b44009c20 comp rx=0 tx=0).stop 2024-09-10T11:30:17.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.939+0000 7f1b5c5ba700 1 -- 172.21.15.144:0/1186002573 shutdown_connections 2024-09-10T11:30:17.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.939+0000 7f1b5c5ba700 1 --2- 172.21.15.144:0/1186002573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b54100250 0x7f1b541006a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.939+0000 7f1b5c5ba700 1 --2- 172.21.15.144:0/1186002573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b540ff050 0x7f1b540ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.939+0000 7f1b5c5ba700 1 -- 172.21.15.144:0/1186002573 >> 172.21.15.144:0/1186002573 conn(0x7f1b540fa600 msgr2=0x7f1b540fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:17.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.939+0000 7f1b5c5ba700 1 -- 172.21.15.144:0/1186002573 shutdown_connections 2024-09-10T11:30:17.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.939+0000 7f1b5c5ba700 1 -- 172.21.15.144:0/1186002573 wait complete. 2024-09-10T11:30:17.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.940+0000 7f1b5c5ba700 1 Processor -- start 2024-09-10T11:30:17.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.940+0000 7f1b5c5ba700 1 -- start start 2024-09-10T11:30:17.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.940+0000 7f1b5c5ba700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b540ff050 0x7f1b54197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:17.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.940+0000 7f1b5c5ba700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b54100250 0x7f1b54198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:17.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.940+0000 7f1b5c5ba700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b541986a0 con 0x7f1b54100250 2024-09-10T11:30:17.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.940+0000 7f1b5c5ba700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b541987e0 con 0x7f1b540ff050 2024-09-10T11:30:17.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.940+0000 7f1b59b55700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b54100250 0x7f1b54198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:17.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.940+0000 7f1b5a356700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b540ff050 0x7f1b54197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:17.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.940+0000 7f1b59b55700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b54100250 0x7f1b54198080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56778/0 (socket says 172.21.15.144:56778) 2024-09-10T11:30:17.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.940+0000 7f1b59b55700 1 -- 172.21.15.144:0/2113435023 learned_addr learned my addr 172.21.15.144:0/2113435023 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:30:17.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.941+0000 7f1b59b55700 1 -- 172.21.15.144:0/2113435023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b540ff050 msgr2=0x7f1b54197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:17.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.941+0000 7f1b59b55700 1 --2- 172.21.15.144:0/2113435023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b540ff050 0x7f1b54197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:17.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.941+0000 7f1b59b55700 1 -- 172.21.15.144:0/2113435023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b440095f0 con 0x7f1b54100250 2024-09-10T11:30:17.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.941+0000 7f1b59b55700 1 --2- 172.21.15.144:0/2113435023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b54100250 0x7f1b54198080 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400bd70 tx=0x7f1b4400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:17.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.941+0000 7f1b4b7fe700 1 -- 172.21.15.144:0/2113435023 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b4401d070 con 0x7f1b54100250 2024-09-10T11:30:17.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.941+0000 7f1b4b7fe700 1 -- 172.21.15.144:0/2113435023 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1b44005dc0 con 0x7f1b54100250 2024-09-10T11:30:17.945 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.941+0000 7f1b5c5ba700 1 -- 172.21.15.144:0/2113435023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b5419d230 con 0x7f1b54100250 2024-09-10T11:30:17.945 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.941+0000 7f1b4b7fe700 1 -- 172.21.15.144:0/2113435023 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b44021410 con 0x7f1b54100250 2024-09-10T11:30:17.945 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.941+0000 7f1b5c5ba700 1 -- 172.21.15.144:0/2113435023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b5419d720 con 0x7f1b54100250 2024-09-10T11:30:17.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.943+0000 7f1b4b7fe700 1 -- 172.21.15.144:0/2113435023 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 29) v1 ==== 46428+0+0 (secure 0 0 0) 0x7f1b4400f7d0 con 0x7f1b54100250 2024-09-10T11:30:17.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.943+0000 7f1b5c5ba700 1 -- 172.21.15.144:0/2113435023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b54107ba0 con 0x7f1b54100250 2024-09-10T11:30:17.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.943+0000 7f1b4b7fe700 1 --2- 172.21.15.144:0/2113435023 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1b40039360 0x7f1b4003b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:17.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.943+0000 7f1b4b7fe700 1 -- 172.21.15.144:0/2113435023 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f1b4404e760 con 0x7f1b54100250 2024-09-10T11:30:17.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.943+0000 7f1b5a356700 1 --2- 172.21.15.144:0/2113435023 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1b40039360 0x7f1b4003b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:17.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.944+0000 7f1b5a356700 1 --2- 172.21.15.144:0/2113435023 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1b40039360 0x7f1b4003b810 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1b541000b0 tx=0x7f1b50007400 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:17.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:17.949+0000 7f1b4b7fe700 1 -- 172.21.15.144:0/2113435023 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f1b44029910 con 0x7f1b54100250 2024-09-10T11:30:18.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.343+0000 7f1b5c5ba700 1 -- 172.21.15.144:0/2113435023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1b5419ddb0 con 0x7f1b54100250 2024-09-10T11:30:18.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.344+0000 7f1b4b7fe700 1 -- 172.21.15.144:0/2113435023 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f1b44026070 con 0x7f1b54100250 2024-09-10T11:30:18.345 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:30:18.346 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:30:18.346 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-09-10T11:30:18.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:30:18.346 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:30:18.346 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 1 2024-09-10T11:30:18.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:30:18.346 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:30:18.346 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-09-10T11:30:18.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:30:18.346 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:30:18.346 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:30:18.346 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:30:18.346 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:30:18.347 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-09-10T11:30:18.347 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 1 2024-09-10T11:30:18.347 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:30:18.347 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:30:18.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.347+0000 7f1b5c5ba700 1 -- 172.21.15.144:0/2113435023 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1b40039360 msgr2=0x7f1b4003b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:18.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.347+0000 7f1b5c5ba700 1 --2- 172.21.15.144:0/2113435023 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1b40039360 0x7f1b4003b810 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f1b541000b0 tx=0x7f1b50007400 comp rx=0 tx=0).stop 2024-09-10T11:30:18.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.347+0000 7f1b5c5ba700 1 -- 172.21.15.144:0/2113435023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b54100250 msgr2=0x7f1b54198080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:18.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.347+0000 7f1b5c5ba700 1 --2- 172.21.15.144:0/2113435023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b54100250 0x7f1b54198080 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f1b4400bd70 tx=0x7f1b4400bda0 comp rx=0 tx=0).stop 2024-09-10T11:30:18.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.347+0000 7f1b5c5ba700 1 -- 172.21.15.144:0/2113435023 shutdown_connections 2024-09-10T11:30:18.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.347+0000 7f1b5c5ba700 1 --2- 172.21.15.144:0/2113435023 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1b40039360 0x7f1b4003b810 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:18.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.347+0000 7f1b5c5ba700 1 --2- 172.21.15.144:0/2113435023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1b540ff050 0x7f1b54197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:18.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.347+0000 7f1b5c5ba700 1 --2- 172.21.15.144:0/2113435023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1b54100250 0x7f1b54198080 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:18.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.348+0000 7f1b5c5ba700 1 -- 172.21.15.144:0/2113435023 >> 172.21.15.144:0/2113435023 conn(0x7f1b540fa600 msgr2=0x7f1b54068c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:18.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.348+0000 7f1b5c5ba700 1 -- 172.21.15.144:0/2113435023 shutdown_connections 2024-09-10T11:30:18.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.348+0000 7f1b5c5ba700 1 -- 172.21.15.144:0/2113435023 wait complete. 2024-09-10T11:30:18.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.518+0000 7f655dddd700 1 -- 172.21.15.144:0/866355380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65580ff9d0 msgr2=0x7f65580ffe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:18.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.518+0000 7f655dddd700 1 --2- 172.21.15.144:0/866355380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65580ff9d0 0x7f65580ffe20 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6548009910 tx=0x7f6548009c20 comp rx=0 tx=0).stop 2024-09-10T11:30:18.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.519+0000 7f655dddd700 1 -- 172.21.15.144:0/866355380 shutdown_connections 2024-09-10T11:30:18.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.519+0000 7f655dddd700 1 --2- 172.21.15.144:0/866355380 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65580ff9d0 0x7f65580ffe20 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:18.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.519+0000 7f655dddd700 1 --2- 172.21.15.144:0/866355380 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65580fe7d0 0x7f65580febe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:18.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.519+0000 7f655dddd700 1 -- 172.21.15.144:0/866355380 >> 172.21.15.144:0/866355380 conn(0x7f65580f9d80 msgr2=0x7f65580fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:18.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.519+0000 7f655dddd700 1 -- 172.21.15.144:0/866355380 shutdown_connections 2024-09-10T11:30:18.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.519+0000 7f655dddd700 1 -- 172.21.15.144:0/866355380 wait complete. 2024-09-10T11:30:18.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.519+0000 7f655dddd700 1 Processor -- start 2024-09-10T11:30:18.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.520+0000 7f655dddd700 1 -- start start 2024-09-10T11:30:18.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.520+0000 7f655dddd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65580fe7d0 0x7f6558107890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:18.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.520+0000 7f655dddd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65580ff9d0 0x7f6558105ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:18.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.520+0000 7f655dddd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6558106420 con 0x7f65580fe7d0 2024-09-10T11:30:18.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.520+0000 7f655dddd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6558106560 con 0x7f65580ff9d0 2024-09-10T11:30:18.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.520+0000 7f65577fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65580fe7d0 0x7f6558107890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:18.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.520+0000 7f65577fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65580fe7d0 0x7f6558107890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56792/0 (socket says 172.21.15.144:56792) 2024-09-10T11:30:18.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.520+0000 7f65577fe700 1 -- 172.21.15.144:0/3067168270 learned_addr learned my addr 172.21.15.144:0/3067168270 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:30:18.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.520+0000 7f6556ffd700 1 --2- 172.21.15.144:0/3067168270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65580ff9d0 0x7f6558105ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:18.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.520+0000 7f65577fe700 1 -- 172.21.15.144:0/3067168270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65580ff9d0 msgr2=0x7f6558105ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:18.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.520+0000 7f65577fe700 1 --2- 172.21.15.144:0/3067168270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65580ff9d0 0x7f6558105ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:18.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.520+0000 7f65577fe700 1 -- 172.21.15.144:0/3067168270 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65480095f0 con 0x7f65580fe7d0 2024-09-10T11:30:18.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.521+0000 7f65577fe700 1 --2- 172.21.15.144:0/3067168270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65580fe7d0 0x7f6558107890 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f654000b4d0 tx=0x7f654000b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:18.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.521+0000 7f6554ff9700 1 -- 172.21.15.144:0/3067168270 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6540011800 con 0x7f65580fe7d0 2024-09-10T11:30:18.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.521+0000 7f655dddd700 1 -- 172.21.15.144:0/3067168270 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6558106840 con 0x7f65580fe7d0 2024-09-10T11:30:18.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.521+0000 7f655dddd700 1 -- 172.21.15.144:0/3067168270 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f655819da70 con 0x7f65580fe7d0 2024-09-10T11:30:18.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.521+0000 7f6554ff9700 1 -- 172.21.15.144:0/3067168270 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6540011e10 con 0x7f65580fe7d0 2024-09-10T11:30:18.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.522+0000 7f6554ff9700 1 -- 172.21.15.144:0/3067168270 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f654000f2e0 con 0x7f65580fe7d0 2024-09-10T11:30:18.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.522+0000 7f6554ff9700 1 -- 172.21.15.144:0/3067168270 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 29) v1 ==== 46428+0+0 (secure 0 0 0) 0x7f654000f500 con 0x7f65580fe7d0 2024-09-10T11:30:18.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.522+0000 7f6554ff9700 1 --2- 172.21.15.144:0/3067168270 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f6544039400 0x7f654403b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:18.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.522+0000 7f6554ff9700 1 -- 172.21.15.144:0/3067168270 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f654004e1f0 con 0x7f65580fe7d0 2024-09-10T11:30:18.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.522+0000 7f655dddd700 1 -- 172.21.15.144:0/3067168270 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6538005320 con 0x7f65580fe7d0 2024-09-10T11:30:18.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.522+0000 7f6556ffd700 1 --2- 172.21.15.144:0/3067168270 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f6544039400 0x7f654403b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:18.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.523+0000 7f6556ffd700 1 --2- 172.21.15.144:0/3067168270 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f6544039400 0x7f654403b8b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6548009fd0 tx=0x7f654800b540 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:18.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.529+0000 7f6554ff9700 1 -- 172.21.15.144:0/3067168270 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f65400178b0 con 0x7f65580fe7d0 2024-09-10T11:30:18.584 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:18 smithi178 ceph-mon[33514]: pgmap v5: 65 pgs: 65 active+clean; 294 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:30:18.584 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:18 smithi178 ceph-mon[33514]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:18.584 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:18 smithi178 ceph-mon[33514]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:18.584 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:18 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/2113435023' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:30:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:18 smithi144 ceph-mon[27942]: pgmap v5: 65 pgs: 65 active+clean; 294 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:30:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:18 smithi144 ceph-mon[27942]: from='client.14692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:18 smithi144 ceph-mon[27942]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:18 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2113435023' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:30:18.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.843+0000 7f655dddd700 1 -- 172.21.15.144:0/3067168270 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6538000bf0 con 0x7f6544039400 2024-09-10T11:30:18.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.844+0000 7f6554ff9700 1 -- 172.21.15.144:0/3067168270 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7f6538000bf0 con 0x7f6544039400 2024-09-10T11:30:18.845 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:30:18.845 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:30:18.845 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:30:18.845 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T11:30:18.846 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:30:18.846 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr" 2024-09-10T11:30:18.846 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:30:18.846 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "2/2 daemons upgraded", 2024-09-10T11:30:18.846 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "", 2024-09-10T11:30:18.846 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:30:18.846 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:30:18.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.846+0000 7f655dddd700 1 -- 172.21.15.144:0/3067168270 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f6544039400 msgr2=0x7f654403b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:18.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.846+0000 7f655dddd700 1 --2- 172.21.15.144:0/3067168270 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f6544039400 0x7f654403b8b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f6548009fd0 tx=0x7f654800b540 comp rx=0 tx=0).stop 2024-09-10T11:30:18.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.846+0000 7f655dddd700 1 -- 172.21.15.144:0/3067168270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65580fe7d0 msgr2=0x7f6558107890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:18.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.846+0000 7f655dddd700 1 --2- 172.21.15.144:0/3067168270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65580fe7d0 0x7f6558107890 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f654000b4d0 tx=0x7f654000b7e0 comp rx=0 tx=0).stop 2024-09-10T11:30:18.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.846+0000 7f655dddd700 1 -- 172.21.15.144:0/3067168270 shutdown_connections 2024-09-10T11:30:18.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.846+0000 7f655dddd700 1 --2- 172.21.15.144:0/3067168270 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f6544039400 0x7f654403b8b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:18.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.846+0000 7f655dddd700 1 --2- 172.21.15.144:0/3067168270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65580fe7d0 0x7f6558107890 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:18.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.846+0000 7f655dddd700 1 --2- 172.21.15.144:0/3067168270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65580ff9d0 0x7f6558105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:18.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.846+0000 7f655dddd700 1 -- 172.21.15.144:0/3067168270 >> 172.21.15.144:0/3067168270 conn(0x7f65580f9d80 msgr2=0x7f6558102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:18.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.846+0000 7f655dddd700 1 -- 172.21.15.144:0/3067168270 shutdown_connections 2024-09-10T11:30:18.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:18.846+0000 7f655dddd700 1 -- 172.21.15.144:0/3067168270 wait complete. 2024-09-10T11:30:19.548 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:19 smithi144 ceph-mon[27942]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:19.784 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:19 smithi178 ceph-mon[33514]: from='client.14704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:20.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:20 smithi144 ceph-mon[27942]: pgmap v6: 65 pgs: 65 active+clean; 299 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 68 op/s 2024-09-10T11:30:20.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:20 smithi144 ceph-mon[27942]: Standby manager daemon smithi178.vmvxuf started 2024-09-10T11:30:20.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:20 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/1690173664' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/crt"}]: dispatch 2024-09-10T11:30:20.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:20 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/1690173664' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T11:30:20.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:20 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/1690173664' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/key"}]: dispatch 2024-09-10T11:30:20.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:20 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/1690173664' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T11:30:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:20 smithi178 ceph-mon[33514]: pgmap v6: 65 pgs: 65 active+clean; 299 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 MiB/s rd, 1.3 MiB/s wr, 68 op/s 2024-09-10T11:30:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:20 smithi178 ceph-mon[33514]: Standby manager daemon smithi178.vmvxuf started 2024-09-10T11:30:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:20 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/1690173664' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/crt"}]: dispatch 2024-09-10T11:30:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:20 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/1690173664' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T11:30:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:20 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/1690173664' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/key"}]: dispatch 2024-09-10T11:30:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:20 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/1690173664' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T11:30:21.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:21 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr metadata", "who": "smithi178.vmvxuf", "id": "smithi178.vmvxuf"}]: dispatch 2024-09-10T11:30:21.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:21 smithi144 ceph-mon[27942]: mgrmap e30: smithi144.bkogfs(active, since 7s), standbys: smithi178.vmvxuf 2024-09-10T11:30:21.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:21 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr metadata", "who": "smithi178.vmvxuf", "id": "smithi178.vmvxuf"}]: dispatch 2024-09-10T11:30:21.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:21 smithi178 ceph-mon[33514]: mgrmap e30: smithi144.bkogfs(active, since 7s), standbys: smithi178.vmvxuf 2024-09-10T11:30:22.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:22 smithi144 ceph-mon[27942]: pgmap v7: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-09-10T11:30:22.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:22 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:22.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:22 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:22 smithi178 ceph-mon[33514]: pgmap v7: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.5 MiB/s wr, 106 op/s 2024-09-10T11:30:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:22 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:22 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:24.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:23 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:24.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:23 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:24.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:23 smithi178 ceph-mon[33514]: pgmap v8: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1006 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-09-10T11:30:24.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:23 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:24.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:23 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:24.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:23 smithi144 ceph-mon[27942]: pgmap v8: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1006 KiB/s rd, 1.2 MiB/s wr, 83 op/s 2024-09-10T11:30:26.630 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:26 smithi144 ceph-mon[27942]: pgmap v9: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-09-10T11:30:26.630 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:26 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:26.631 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:26 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:26.768 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:26 smithi178 ceph-mon[33514]: pgmap v9: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 142 op/s 2024-09-10T11:30:26.768 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:26 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:26.768 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:26 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:28.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:28 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:28.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:28 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:28.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:28 smithi178 ceph-mon[33514]: pgmap v10: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 130 op/s 2024-09-10T11:30:28.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:28 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:28.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:28 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:28.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:28 smithi144 ceph-mon[27942]: pgmap v10: 65 pgs: 65 active+clean; 300 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 MiB/s rd, 1.9 MiB/s wr, 130 op/s 2024-09-10T11:30:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:29 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:30:29.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:29 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:30:30.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:30 smithi178 ceph-mon[33514]: pgmap v11: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 167 op/s 2024-09-10T11:30:30.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:30 smithi144 ceph-mon[27942]: pgmap v11: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.7 MiB/s rd, 2.8 MiB/s wr, 167 op/s 2024-09-10T11:30:32.668 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:32 smithi178 ceph-mon[33514]: pgmap v12: 65 pgs: 65 active+clean; 308 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 174 op/s 2024-09-10T11:30:32.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:32 smithi144 ceph-mon[27942]: pgmap v12: 65 pgs: 65 active+clean; 308 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.4 MiB/s wr, 174 op/s 2024-09-10T11:30:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:34 smithi144 ceph-mon[27942]: pgmap v13: 65 pgs: 65 active+clean; 308 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 140 op/s 2024-09-10T11:30:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:34 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:34 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:34 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:30:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:34 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:30:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:34 smithi144 ceph-mon[27942]: Updating smithi144:/etc/ceph/ceph.conf 2024-09-10T11:30:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:34 smithi144 ceph-mon[27942]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T11:30:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:34 smithi178 ceph-mon[33514]: pgmap v13: 65 pgs: 65 active+clean; 308 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.1 MiB/s wr, 140 op/s 2024-09-10T11:30:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:34 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:34 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:34 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:30:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:34 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:30:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:34 smithi178 ceph-mon[33514]: Updating smithi144:/etc/ceph/ceph.conf 2024-09-10T11:30:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:34 smithi178 ceph-mon[33514]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T11:30:36.286 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:35 smithi144 ceph-mon[27942]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:30:36.287 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:35 smithi144 ceph-mon[27942]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:30:36.287 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:35 smithi144 ceph-mon[27942]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:30:36.287 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:35 smithi144 ceph-mon[27942]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:30:36.287 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:35 smithi144 ceph-mon[27942]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:30:36.287 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:35 smithi144 ceph-mon[27942]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:30:36.287 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:35 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:36.287 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:35 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:36.288 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:35 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:36.288 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:35 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:36.288 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:35 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:35 smithi178 ceph-mon[33514]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:30:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:35 smithi178 ceph-mon[33514]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:30:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:35 smithi178 ceph-mon[33514]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:30:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:35 smithi178 ceph-mon[33514]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:30:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:35 smithi178 ceph-mon[33514]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:30:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:35 smithi178 ceph-mon[33514]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:30:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:35 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:35 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:36.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:35 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:36.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:35 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:36.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:35 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:37.282 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:36 smithi144 ceph-mon[27942]: Reconfiguring prometheus.smithi144 (dependencies changed)... 2024-09-10T11:30:37.282 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:36 smithi144 ceph-mon[27942]: pgmap v14: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 217 op/s 2024-09-10T11:30:37.282 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:36 smithi144 ceph-mon[27942]: Reconfiguring daemon prometheus.smithi144 on smithi144 2024-09-10T11:30:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:36 smithi178 ceph-mon[33514]: Reconfiguring prometheus.smithi144 (dependencies changed)... 2024-09-10T11:30:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:36 smithi178 ceph-mon[33514]: pgmap v14: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 217 op/s 2024-09-10T11:30:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:36 smithi178 ceph-mon[33514]: Reconfiguring daemon prometheus.smithi144 on smithi144 2024-09-10T11:30:39.259 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:38 smithi144 ceph-mon[27942]: pgmap v15: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 148 op/s 2024-09-10T11:30:39.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:38 smithi178 ceph-mon[33514]: pgmap v15: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 2.1 MiB/s wr, 148 op/s 2024-09-10T11:30:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:40 smithi178 ceph-mon[33514]: pgmap v16: 65 pgs: 65 active+clean; 315 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 182 op/s 2024-09-10T11:30:41.397 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:40 smithi144 ceph-mon[27942]: pgmap v16: 65 pgs: 65 active+clean; 315 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.9 MiB/s rd, 2.9 MiB/s wr, 182 op/s 2024-09-10T11:30:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:42 smithi178 ceph-mon[33514]: pgmap v17: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 201 op/s 2024-09-10T11:30:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:42 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:42 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:42 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:30:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:42 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:30:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:42 smithi144 ceph-mon[27942]: pgmap v17: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 2.5 MiB/s wr, 201 op/s 2024-09-10T11:30:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:42 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:42 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:42 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:30:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:42 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:30:44.282 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:43 smithi178 ceph-mon[33514]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:30:44.283 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:43 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:30:44.283 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:43 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi178.vmvxuf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:30:44.283 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:43 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi178.vmvxuf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:30:44.283 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:43 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:30:44.283 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:43 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:30:44.323 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:43 smithi144 ceph-mon[27942]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:30:44.323 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:43 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:30:44.323 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:43 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi178.vmvxuf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:30:44.323 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:43 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi178.vmvxuf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:30:44.323 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:43 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:30:44.323 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:43 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:30:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:44 smithi178 ceph-mon[33514]: pgmap v18: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 166 op/s 2024-09-10T11:30:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:44 smithi178 ceph-mon[33514]: Upgrade: Updating mgr.smithi178.vmvxuf 2024-09-10T11:30:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:44 smithi178 ceph-mon[33514]: Deploying daemon mgr.smithi178.vmvxuf on smithi178 2024-09-10T11:30:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:44 smithi144 ceph-mon[27942]: pgmap v18: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 MiB/s rd, 2.2 MiB/s wr, 166 op/s 2024-09-10T11:30:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:44 smithi144 ceph-mon[27942]: Upgrade: Updating mgr.smithi178.vmvxuf 2024-09-10T11:30:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:44 smithi144 ceph-mon[27942]: Deploying daemon mgr.smithi178.vmvxuf on smithi178 2024-09-10T11:30:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:46 smithi178 ceph-mon[33514]: pgmap v19: 65 pgs: 65 active+clean; 314 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 273 op/s 2024-09-10T11:30:47.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:46 smithi144 ceph-mon[27942]: pgmap v19: 65 pgs: 65 active+clean; 314 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 273 op/s 2024-09-10T11:30:48.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.990+0000 7fedb1b5d700 1 -- 172.21.15.144:0/2330526620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedac103580 msgr2=0x7fedac105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:48.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.990+0000 7fedb1b5d700 1 --2- 172.21.15.144:0/2330526620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedac103580 0x7fedac105960 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7feda0009910 tx=0x7feda0009c20 comp rx=0 tx=0).stop 2024-09-10T11:30:48.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.990+0000 7fedb1b5d700 1 -- 172.21.15.144:0/2330526620 shutdown_connections 2024-09-10T11:30:48.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.990+0000 7fedb1b5d700 1 --2- 172.21.15.144:0/2330526620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedac103580 0x7fedac105960 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:48.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.990+0000 7fedb1b5d700 1 --2- 172.21.15.144:0/2330526620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedac100c60 0x7fedac103040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:48.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.990+0000 7fedb1b5d700 1 -- 172.21.15.144:0/2330526620 >> 172.21.15.144:0/2330526620 conn(0x7fedac0fa5e0 msgr2=0x7fedac0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:48.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.990+0000 7fedb1b5d700 1 -- 172.21.15.144:0/2330526620 shutdown_connections 2024-09-10T11:30:48.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.990+0000 7fedb1b5d700 1 -- 172.21.15.144:0/2330526620 wait complete. 2024-09-10T11:30:48.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.990+0000 7fedb1b5d700 1 Processor -- start 2024-09-10T11:30:48.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.991+0000 7fedb1b5d700 1 -- start start 2024-09-10T11:30:48.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.991+0000 7fedb1b5d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedac100c60 0x7fedac1005c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:48.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.991+0000 7fedb1b5d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedac103580 0x7fedac0fec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:48.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.991+0000 7fedb1b5d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0ff150 con 0x7fedac100c60 2024-09-10T11:30:48.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.991+0000 7fedb1b5d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedac0ff290 con 0x7fedac103580 2024-09-10T11:30:48.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.991+0000 7fedab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedac100c60 0x7fedac1005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:48.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.991+0000 7fedab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedac100c60 0x7fedac1005c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39770/0 (socket says 172.21.15.144:39770) 2024-09-10T11:30:48.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.991+0000 7fedab7fe700 1 -- 172.21.15.144:0/596019543 learned_addr learned my addr 172.21.15.144:0/596019543 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:30:48.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.991+0000 7fedaaffd700 1 --2- 172.21.15.144:0/596019543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedac103580 0x7fedac0fec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:48.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.991+0000 7fedab7fe700 1 -- 172.21.15.144:0/596019543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedac103580 msgr2=0x7fedac0fec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:48.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.991+0000 7fedab7fe700 1 --2- 172.21.15.144:0/596019543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedac103580 0x7fedac0fec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:48.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.991+0000 7fedab7fe700 1 -- 172.21.15.144:0/596019543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda00095f0 con 0x7fedac100c60 2024-09-10T11:30:48.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.992+0000 7fedab7fe700 1 --2- 172.21.15.144:0/596019543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedac100c60 0x7fedac1005c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00e920 tx=0x7fed9c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:48.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.992+0000 7feda8ff9700 1 -- 172.21.15.144:0/596019543 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed9c00c9d0 con 0x7fedac100c60 2024-09-10T11:30:48.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.992+0000 7fedb1b5d700 1 -- 172.21.15.144:0/596019543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedac0ff570 con 0x7fedac100c60 2024-09-10T11:30:48.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.992+0000 7feda8ff9700 1 -- 172.21.15.144:0/596019543 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fed9c010430 con 0x7fedac100c60 2024-09-10T11:30:48.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.992+0000 7fedb1b5d700 1 -- 172.21.15.144:0/596019543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedac19d9b0 con 0x7fedac100c60 2024-09-10T11:30:48.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.992+0000 7feda8ff9700 1 -- 172.21.15.144:0/596019543 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed9c018650 con 0x7fedac100c60 2024-09-10T11:30:48.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.993+0000 7fedb1b5d700 1 -- 172.21.15.144:0/596019543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedac18fb70 con 0x7fedac100c60 2024-09-10T11:30:48.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.994+0000 7feda8ff9700 1 -- 172.21.15.144:0/596019543 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 30) v1 ==== 92160+0+0 (secure 0 0 0) 0x7fed9c0105a0 con 0x7fedac100c60 2024-09-10T11:30:48.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.994+0000 7feda8ff9700 1 --2- 172.21.15.144:0/596019543 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fed9406dfc0 0x7fed94070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:48.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.994+0000 7feda8ff9700 1 -- 172.21.15.144:0/596019543 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fed9c014070 con 0x7fedac100c60 2024-09-10T11:30:48.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.994+0000 7fedaaffd700 1 --2- 172.21.15.144:0/596019543 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fed9406dfc0 0x7fed94070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:48.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.995+0000 7fedaaffd700 1 --2- 172.21.15.144:0/596019543 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fed9406dfc0 0x7fed94070470 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7feda0009fd0 tx=0x7feda0005940 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:49.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:48.999+0000 7feda8ff9700 1 -- 172.21.15.144:0/596019543 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7fed9c0539d0 con 0x7fedac100c60 2024-09-10T11:30:49.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.333+0000 7fedb1b5d700 1 -- 172.21.15.144:0/596019543 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fedac061190 con 0x7fed9406dfc0 2024-09-10T11:30:49.336 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:48 smithi144 ceph-mon[27942]: pgmap v20: 65 pgs: 65 active+clean; 314 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-09-10T11:30:49.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.334+0000 7feda8ff9700 1 -- 172.21.15.144:0/596019543 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fedac061190 con 0x7fed9406dfc0 2024-09-10T11:30:49.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.336+0000 7fedb1b5d700 1 -- 172.21.15.144:0/596019543 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fed9406dfc0 msgr2=0x7fed94070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:49.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.336+0000 7fedb1b5d700 1 --2- 172.21.15.144:0/596019543 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fed9406dfc0 0x7fed94070470 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7feda0009fd0 tx=0x7feda0005940 comp rx=0 tx=0).stop 2024-09-10T11:30:49.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.336+0000 7fedb1b5d700 1 -- 172.21.15.144:0/596019543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedac100c60 msgr2=0x7fedac1005c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:49.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.336+0000 7fedb1b5d700 1 --2- 172.21.15.144:0/596019543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedac100c60 0x7fedac1005c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fed9c00e920 tx=0x7fed9c00ec30 comp rx=0 tx=0).stop 2024-09-10T11:30:49.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.336+0000 7fedb1b5d700 1 -- 172.21.15.144:0/596019543 shutdown_connections 2024-09-10T11:30:49.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.336+0000 7fedb1b5d700 1 --2- 172.21.15.144:0/596019543 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fed9406dfc0 0x7fed94070470 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.336+0000 7fedb1b5d700 1 --2- 172.21.15.144:0/596019543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fedac100c60 0x7fedac1005c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.336+0000 7fedb1b5d700 1 --2- 172.21.15.144:0/596019543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fedac103580 0x7fedac0fec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.336+0000 7fedb1b5d700 1 -- 172.21.15.144:0/596019543 >> 172.21.15.144:0/596019543 conn(0x7fedac0fa5e0 msgr2=0x7fedac0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:49.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.337+0000 7fedb1b5d700 1 -- 172.21.15.144:0/596019543 shutdown_connections 2024-09-10T11:30:49.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.337+0000 7fedb1b5d700 1 -- 172.21.15.144:0/596019543 wait complete. 2024-09-10T11:30:49.349 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:30:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:48 smithi178 ceph-mon[33514]: pgmap v20: 65 pgs: 65 active+clean; 314 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 196 op/s 2024-09-10T11:30:49.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.493+0000 7f721862a700 1 -- 172.21.15.144:0/1273539189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72100ff070 msgr2=0x7f72100ff480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:49.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.493+0000 7f721862a700 1 --2- 172.21.15.144:0/1273539189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72100ff070 0x7f72100ff480 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f720c009910 tx=0x7f720c009c20 comp rx=0 tx=0).stop 2024-09-10T11:30:49.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.493+0000 7f721862a700 1 -- 172.21.15.144:0/1273539189 shutdown_connections 2024-09-10T11:30:49.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.493+0000 7f721862a700 1 --2- 172.21.15.144:0/1273539189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7210100270 0x7f72101006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.493+0000 7f721862a700 1 --2- 172.21.15.144:0/1273539189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72100ff070 0x7f72100ff480 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.493+0000 7f721862a700 1 -- 172.21.15.144:0/1273539189 >> 172.21.15.144:0/1273539189 conn(0x7f72100fa600 msgr2=0x7f72100fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:49.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.494+0000 7f721862a700 1 -- 172.21.15.144:0/1273539189 shutdown_connections 2024-09-10T11:30:49.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.494+0000 7f721862a700 1 -- 172.21.15.144:0/1273539189 wait complete. 2024-09-10T11:30:49.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.495+0000 7f721862a700 1 Processor -- start 2024-09-10T11:30:49.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.495+0000 7f721862a700 1 -- start start 2024-09-10T11:30:49.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.495+0000 7f721862a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72100ff070 0x7f7210071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:49.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.495+0000 7f721862a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7210100270 0x7f7210072240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:49.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.495+0000 7f721862a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7210072860 con 0x7f7210100270 2024-09-10T11:30:49.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.495+0000 7f721862a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72101a3430 con 0x7f72100ff070 2024-09-10T11:30:49.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.497+0000 7f72163c6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72100ff070 0x7f7210071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:49.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.497+0000 7f72163c6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72100ff070 0x7f7210071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.144:38684/0 (socket says 172.21.15.144:38684) 2024-09-10T11:30:49.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.497+0000 7f72163c6700 1 -- 172.21.15.144:0/3074978443 learned_addr learned my addr 172.21.15.144:0/3074978443 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:30:49.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.497+0000 7f7215bc5700 1 --2- 172.21.15.144:0/3074978443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7210100270 0x7f7210072240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:49.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.497+0000 7f72163c6700 1 -- 172.21.15.144:0/3074978443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7210100270 msgr2=0x7f7210072240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:49.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.497+0000 7f72163c6700 1 --2- 172.21.15.144:0/3074978443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7210100270 0x7f7210072240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.497+0000 7f72163c6700 1 -- 172.21.15.144:0/3074978443 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c0095f0 con 0x7f72100ff070 2024-09-10T11:30:49.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.498+0000 7f72163c6700 1 --2- 172.21.15.144:0/3074978443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72100ff070 0x7f7210071d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f720c00bd70 tx=0x7f720c00bda0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:49.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.498+0000 7f72077fe700 1 -- 172.21.15.144:0/3074978443 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f720c01d070 con 0x7f72100ff070 2024-09-10T11:30:49.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.498+0000 7f72077fe700 1 -- 172.21.15.144:0/3074978443 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f720c005a00 con 0x7f72100ff070 2024-09-10T11:30:49.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.498+0000 7f72077fe700 1 -- 172.21.15.144:0/3074978443 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f720c021410 con 0x7f72100ff070 2024-09-10T11:30:49.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.499+0000 7f721862a700 1 -- 172.21.15.144:0/3074978443 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72101a36b0 con 0x7f72100ff070 2024-09-10T11:30:49.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.499+0000 7f721862a700 1 -- 172.21.15.144:0/3074978443 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72101a3b50 con 0x7f72100ff070 2024-09-10T11:30:49.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.501+0000 7f721862a700 1 -- 172.21.15.144:0/3074978443 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7210108cc0 con 0x7f72100ff070 2024-09-10T11:30:49.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.502+0000 7f72077fe700 1 -- 172.21.15.144:0/3074978443 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 30) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f720c00f6f0 con 0x7f72100ff070 2024-09-10T11:30:49.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.502+0000 7f72077fe700 1 --2- 172.21.15.144:0/3074978443 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f71fc06e090 0x7f71fc070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:49.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.503+0000 7f72077fe700 1 -- 172.21.15.144:0/3074978443 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f720c08f0e0 con 0x7f72100ff070 2024-09-10T11:30:49.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.503+0000 7f7215bc5700 1 --2- 172.21.15.144:0/3074978443 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f71fc06e090 0x7f71fc070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:49.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.504+0000 7f7215bc5700 1 --2- 172.21.15.144:0/3074978443 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f71fc06e090 0x7f71fc070540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7210072df0 tx=0x7f720000b410 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:49.514 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.511+0000 7f72077fe700 1 -- 172.21.15.144:0/3074978443 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f720c058730 con 0x7f72100ff070 2024-09-10T11:30:49.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.801+0000 7f721862a700 1 -- 172.21.15.144:0/3074978443 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7210104bc0 con 0x7f71fc06e090 2024-09-10T11:30:49.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.802+0000 7f72077fe700 1 -- 172.21.15.144:0/3074978443 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f7210104bc0 con 0x7f71fc06e090 2024-09-10T11:30:49.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.806+0000 7f721862a700 1 -- 172.21.15.144:0/3074978443 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f71fc06e090 msgr2=0x7f71fc070540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:49.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.806+0000 7f721862a700 1 --2- 172.21.15.144:0/3074978443 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f71fc06e090 0x7f71fc070540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7210072df0 tx=0x7f720000b410 comp rx=0 tx=0).stop 2024-09-10T11:30:49.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.806+0000 7f721862a700 1 -- 172.21.15.144:0/3074978443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72100ff070 msgr2=0x7f7210071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:49.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.806+0000 7f721862a700 1 --2- 172.21.15.144:0/3074978443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72100ff070 0x7f7210071d00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f720c00bd70 tx=0x7f720c00bda0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.806+0000 7f721862a700 1 -- 172.21.15.144:0/3074978443 shutdown_connections 2024-09-10T11:30:49.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.806+0000 7f721862a700 1 --2- 172.21.15.144:0/3074978443 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f71fc06e090 0x7f71fc070540 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.806+0000 7f721862a700 1 --2- 172.21.15.144:0/3074978443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72100ff070 0x7f7210071d00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.806+0000 7f721862a700 1 --2- 172.21.15.144:0/3074978443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7210100270 0x7f7210072240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.806+0000 7f721862a700 1 -- 172.21.15.144:0/3074978443 >> 172.21.15.144:0/3074978443 conn(0x7f72100fa600 msgr2=0x7f72101034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:49.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.806+0000 7f721862a700 1 -- 172.21.15.144:0/3074978443 shutdown_connections 2024-09-10T11:30:49.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.806+0000 7f721862a700 1 -- 172.21.15.144:0/3074978443 wait complete. 2024-09-10T11:30:49.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.945+0000 7f1c8a103700 1 -- 172.21.15.144:0/55432659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c84100170 msgr2=0x7f1c84100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:49.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.945+0000 7f1c8a103700 1 --2- 172.21.15.144:0/55432659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c84100170 0x7f1c84100580 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00b0a0 tx=0x7f1c7c00b3b0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.945+0000 7f1c8a103700 1 -- 172.21.15.144:0/55432659 shutdown_connections 2024-09-10T11:30:49.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.945+0000 7f1c8a103700 1 --2- 172.21.15.144:0/55432659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c84101370 0x7f1c841017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.945+0000 7f1c8a103700 1 --2- 172.21.15.144:0/55432659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c84100170 0x7f1c84100580 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.945+0000 7f1c8a103700 1 -- 172.21.15.144:0/55432659 >> 172.21.15.144:0/55432659 conn(0x7f1c840fb720 msgr2=0x7f1c840fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:49.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.945+0000 7f1c8a103700 1 -- 172.21.15.144:0/55432659 shutdown_connections 2024-09-10T11:30:49.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.946+0000 7f1c8a103700 1 -- 172.21.15.144:0/55432659 wait complete. 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.946+0000 7f1c8a103700 1 Processor -- start 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.946+0000 7f1c8a103700 1 -- start start 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.946+0000 7f1c8a103700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c84100170 0x7f1c84197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.946+0000 7f1c8a103700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c84101370 0x7f1c841980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.946+0000 7f1c8a103700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c841986d0 con 0x7f1c84101370 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.947+0000 7f1c8a103700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c84198810 con 0x7f1c84100170 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.947+0000 7f1c837fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c84100170 0x7f1c84197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:49.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.947+0000 7f1c837fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c84100170 0x7f1c84197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.144:38700/0 (socket says 172.21.15.144:38700) 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.947+0000 7f1c837fe700 1 -- 172.21.15.144:0/1724291252 learned_addr learned my addr 172.21.15.144:0/1724291252 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.947+0000 7f1c82ffd700 1 --2- 172.21.15.144:0/1724291252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c84101370 0x7f1c841980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.947+0000 7f1c837fe700 1 -- 172.21.15.144:0/1724291252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c84101370 msgr2=0x7f1c841980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.947+0000 7f1c837fe700 1 --2- 172.21.15.144:0/1724291252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c84101370 0x7f1c841980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.947+0000 7f1c837fe700 1 -- 172.21.15.144:0/1724291252 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c7c009d00 con 0x7f1c84100170 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.947+0000 7f1c837fe700 1 --2- 172.21.15.144:0/1724291252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c84100170 0x7f1c84197b70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00b780 tx=0x7f1c7c003c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.947+0000 7f1c80ff9700 1 -- 172.21.15.144:0/1724291252 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c7c00e040 con 0x7f1c84100170 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.948+0000 7f1c8a103700 1 -- 172.21.15.144:0/1724291252 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c8419d260 con 0x7f1c84100170 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.948+0000 7f1c80ff9700 1 -- 172.21.15.144:0/1724291252 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1c7c00bdc0 con 0x7f1c84100170 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.948+0000 7f1c80ff9700 1 -- 172.21.15.144:0/1724291252 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1c7c01d8f0 con 0x7f1c84100170 2024-09-10T11:30:49.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.948+0000 7f1c8a103700 1 -- 172.21.15.144:0/1724291252 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c8419d7d0 con 0x7f1c84100170 2024-09-10T11:30:49.952 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.949+0000 7f1c8a103700 1 -- 172.21.15.144:0/1724291252 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c840733e0 con 0x7f1c84100170 2024-09-10T11:30:49.952 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.950+0000 7f1c80ff9700 1 -- 172.21.15.144:0/1724291252 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 30) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f1c7c01dbb0 con 0x7f1c84100170 2024-09-10T11:30:49.952 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.950+0000 7f1c80ff9700 1 --2- 172.21.15.144:0/1724291252 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1c6c06dfc0 0x7f1c6c070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:49.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.950+0000 7f1c80ff9700 1 -- 172.21.15.144:0/1724291252 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f1c7c08f7d0 con 0x7f1c84100170 2024-09-10T11:30:49.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.950+0000 7f1c82ffd700 1 --2- 172.21.15.144:0/1724291252 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1c6c06dfc0 0x7f1c6c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:49.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.951+0000 7f1c82ffd700 1 --2- 172.21.15.144:0/1724291252 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1c6c06dfc0 0x7f1c6c070470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009c60 tx=0x7f1c78009260 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:49.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:49.955+0000 7f1c80ff9700 1 -- 172.21.15.144:0/1724291252 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f1c7c05c9f0 con 0x7f1c84100170 2024-09-10T11:30:50.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.277+0000 7f1c8a103700 1 -- 172.21.15.144:0/1724291252 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c840fc670 con 0x7f1c6c06dfc0 2024-09-10T11:30:50.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.289+0000 7f1c80ff9700 1 -- 172.21.15.144:0/1724291252 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f1c840fc670 con 0x7f1c6c06dfc0 2024-09-10T11:30:50.291 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:30:50.291 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (8m) 28s ago 9m 24.6M - 0.25.0 c8568f914cd2 b5c0bfae59de 2024-09-10T11:30:50.291 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (9m) 28s ago 9m 7818k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:30:50.292 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (8m) 24s ago 8m 7750k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:30:50.292 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (9m) 28s ago 9m 6665k - 18.2.0 dc2bc1663786 3311fb04aa0c 2024-09-10T11:30:50.292 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (8m) 24s ago 8m 6651k - 18.2.0 dc2bc1663786 6e408e8476d4 2024-09-10T11:30:50.292 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (8m) 28s ago 9m 86.0M - 9.4.7 954c08fa6188 e9a1aec955f6 2024-09-10T11:30:50.292 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (4m) 28s ago 4m 146M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:30:50.292 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (4m) 28s ago 4m 164M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:30:50.292 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (4m) 24s ago 4m 152M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:30:50.292 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (4m) 24s ago 4m 176M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:30:50.292 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (47s) 28s ago 11m 476M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:30:50.292 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (2m) 24s ago 8m 105M - 19.1.1-305-g1cf819b7 41d671128357 859a740b57c8 2024-09-10T11:30:50.292 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (11m) 28s ago 11m 52.4M 2048M 18.2.0 dc2bc1663786 42addb654e4f 2024-09-10T11:30:50.292 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (8m) 24s ago 8m 43.2M 2048M 18.2.0 dc2bc1663786 3c1ba210ed3f 2024-09-10T11:30:50.292 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (9m) 28s ago 9m 16.2M - 1.5.0 0da6a335fe13 dfbf0c488d17 2024-09-10T11:30:50.293 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (8m) 24s ago 8m 15.0M - 1.5.0 0da6a335fe13 ff2cac118c8d 2024-09-10T11:30:50.293 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (7m) 28s ago 7m 181M 1216M 18.2.0 dc2bc1663786 2435febebdb5 2024-09-10T11:30:50.293 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (7m) 28s ago 7m 188M 1216M 18.2.0 dc2bc1663786 b55fe920ff9d 2024-09-10T11:30:50.293 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (6m) 28s ago 6m 144M 1216M 18.2.0 dc2bc1663786 3ed5618eac33 2024-09-10T11:30:50.293 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (6m) 24s ago 6m 217M 2239M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:30:50.293 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (6m) 24s ago 6m 188M 2239M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:30:50.293 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (5m) 24s ago 5m 179M 2239M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:30:50.293 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 starting - - - - 2024-09-10T11:30:50.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.292+0000 7f1c8a103700 1 -- 172.21.15.144:0/1724291252 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1c6c06dfc0 msgr2=0x7f1c6c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.292+0000 7f1c8a103700 1 --2- 172.21.15.144:0/1724291252 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1c6c06dfc0 0x7f1c6c070470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f1c78009c60 tx=0x7f1c78009260 comp rx=0 tx=0).stop 2024-09-10T11:30:50.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.292+0000 7f1c8a103700 1 -- 172.21.15.144:0/1724291252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c84100170 msgr2=0x7f1c84197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.292+0000 7f1c8a103700 1 --2- 172.21.15.144:0/1724291252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c84100170 0x7f1c84197b70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c00b780 tx=0x7f1c7c003c30 comp rx=0 tx=0).stop 2024-09-10T11:30:50.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.292+0000 7f1c8a103700 1 -- 172.21.15.144:0/1724291252 shutdown_connections 2024-09-10T11:30:50.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.292+0000 7f1c8a103700 1 --2- 172.21.15.144:0/1724291252 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1c6c06dfc0 0x7f1c6c070470 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.292+0000 7f1c8a103700 1 --2- 172.21.15.144:0/1724291252 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c84100170 0x7f1c84197b70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.292+0000 7f1c8a103700 1 --2- 172.21.15.144:0/1724291252 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c84101370 0x7f1c841980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.292+0000 7f1c8a103700 1 -- 172.21.15.144:0/1724291252 >> 172.21.15.144:0/1724291252 conn(0x7f1c840fb720 msgr2=0x7f1c840fbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:50.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.292+0000 7f1c8a103700 1 -- 172.21.15.144:0/1724291252 shutdown_connections 2024-09-10T11:30:50.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.292+0000 7f1c8a103700 1 -- 172.21.15.144:0/1724291252 wait complete. 2024-09-10T11:30:50.365 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-09-10T11:30:50.366 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-09-10T11:30:50.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.424+0000 7fa3649f0700 1 -- 172.21.15.144:0/199496246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa360071b40 msgr2=0x7fa360071f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.424+0000 7fa3649f0700 1 --2- 172.21.15.144:0/199496246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa360071b40 0x7fa360071f30 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa350009910 tx=0x7fa350009c20 comp rx=0 tx=0).stop 2024-09-10T11:30:50.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.424+0000 7fa3649f0700 1 -- 172.21.15.144:0/199496246 shutdown_connections 2024-09-10T11:30:50.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.424+0000 7fa3649f0700 1 --2- 172.21.15.144:0/199496246 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa360072500 0x7fa360107d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.424+0000 7fa3649f0700 1 --2- 172.21.15.144:0/199496246 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa360071b40 0x7fa360071f30 secure :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa350009910 tx=0x7fa350009c20 comp rx=0 tx=0).stop 2024-09-10T11:30:50.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.424+0000 7fa3649f0700 1 -- 172.21.15.144:0/199496246 >> 172.21.15.144:0/199496246 conn(0x7fa36006d050 msgr2=0x7fa36006f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:50.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.425+0000 7fa3649f0700 1 -- 172.21.15.144:0/199496246 shutdown_connections 2024-09-10T11:30:50.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.425+0000 7fa3649f0700 1 -- 172.21.15.144:0/199496246 wait complete. 2024-09-10T11:30:50.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.426+0000 7fa3649f0700 1 Processor -- start 2024-09-10T11:30:50.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.426+0000 7fa3649f0700 1 -- start start 2024-09-10T11:30:50.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.426+0000 7fa3649f0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa360072500 0x7fa3601a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.427+0000 7fa3649f0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3601a4ee0 0x7fa3601a9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.427+0000 7fa3649f0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3601a53e0 con 0x7fa360072500 2024-09-10T11:30:50.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.427+0000 7fa3649f0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3601a5550 con 0x7fa3601a4ee0 2024-09-10T11:30:50.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.427+0000 7fa355d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3601a4ee0 0x7fa3601a9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.427+0000 7fa355d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3601a4ee0 0x7fa3601a9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.144:38730/0 (socket says 172.21.15.144:38730) 2024-09-10T11:30:50.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.427+0000 7fa355d9b700 1 -- 172.21.15.144:0/1555463146 learned_addr learned my addr 172.21.15.144:0/1555463146 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:30:50.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.427+0000 7fa35e59c700 1 --2- 172.21.15.144:0/1555463146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa360072500 0x7fa3601a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.428+0000 7fa35e59c700 1 -- 172.21.15.144:0/1555463146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3601a4ee0 msgr2=0x7fa3601a9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.428+0000 7fa35e59c700 1 --2- 172.21.15.144:0/1555463146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3601a4ee0 0x7fa3601a9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.428+0000 7fa35e59c700 1 -- 172.21.15.144:0/1555463146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3500095f0 con 0x7fa360072500 2024-09-10T11:30:50.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.428+0000 7fa355d9b700 1 --2- 172.21.15.144:0/1555463146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3601a4ee0 0x7fa3601a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:30:50.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.428+0000 7fa35e59c700 1 --2- 172.21.15.144:0/1555463146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa360072500 0x7fa3601a49a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa350008000 tx=0x7fa35000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:50.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.429+0000 7fa357fff700 1 -- 172.21.15.144:0/1555463146 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa35001d070 con 0x7fa360072500 2024-09-10T11:30:50.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.429+0000 7fa357fff700 1 -- 172.21.15.144:0/1555463146 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa350005dc0 con 0x7fa360072500 2024-09-10T11:30:50.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.429+0000 7fa3649f0700 1 -- 172.21.15.144:0/1555463146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3601aa490 con 0x7fa360072500 2024-09-10T11:30:50.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.429+0000 7fa357fff700 1 -- 172.21.15.144:0/1555463146 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa350021410 con 0x7fa360072500 2024-09-10T11:30:50.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.431+0000 7fa3649f0700 1 -- 172.21.15.144:0/1555463146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3601aa950 con 0x7fa360072500 2024-09-10T11:30:50.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.432+0000 7fa3649f0700 1 -- 172.21.15.144:0/1555463146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa360109bd0 con 0x7fa360072500 2024-09-10T11:30:50.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.432+0000 7fa357fff700 1 -- 172.21.15.144:0/1555463146 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 30) v1 ==== 92160+0+0 (secure 0 0 0) 0x7fa350003a60 con 0x7fa360072500 2024-09-10T11:30:50.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.433+0000 7fa357fff700 1 --2- 172.21.15.144:0/1555463146 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fa34c06dff0 0x7fa34c0704a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.433+0000 7fa355d9b700 1 --2- 172.21.15.144:0/1555463146 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fa34c06dff0 0x7fa34c0704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.433+0000 7fa357fff700 1 -- 172.21.15.144:0/1555463146 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fa35000f680 con 0x7fa360072500 2024-09-10T11:30:50.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.434+0000 7fa355d9b700 1 --2- 172.21.15.144:0/1555463146 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fa34c06dff0 0x7fa34c0704a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa348005950 tx=0x7fa34800b500 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:50.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.443+0000 7fa357fff700 1 -- 172.21.15.144:0/1555463146 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7fa350094060 con 0x7fa360072500 2024-09-10T11:30:50.751 DEBUG:teuthology.parallel:result is None 2024-09-10T11:30:50.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.801+0000 7fa3649f0700 1 -- 172.21.15.144:0/1555463146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa360109d90 con 0x7fa360072500 2024-09-10T11:30:50.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.802+0000 7fa357fff700 1 -- 172.21.15.144:0/1555463146 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fa35005bde0 con 0x7fa360072500 2024-09-10T11:30:50.804 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:30:50.804 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:30:50.804 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-09-10T11:30:50.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:30:50.805 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:30:50.805 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:30:50.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:30:50.805 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:30:50.805 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-09-10T11:30:50.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:30:50.805 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:30:50.805 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:30:50.805 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:30:50.805 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:30:50.805 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-09-10T11:30:50.805 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:30:50.805 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:30:50.806 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:30:50.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.804+0000 7fa3649f0700 1 -- 172.21.15.144:0/1555463146 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fa34c06dff0 msgr2=0x7fa34c0704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.804+0000 7fa3649f0700 1 --2- 172.21.15.144:0/1555463146 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fa34c06dff0 0x7fa34c0704a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa348005950 tx=0x7fa34800b500 comp rx=0 tx=0).stop 2024-09-10T11:30:50.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.804+0000 7fa3649f0700 1 -- 172.21.15.144:0/1555463146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa360072500 msgr2=0x7fa3601a49a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.804+0000 7fa3649f0700 1 --2- 172.21.15.144:0/1555463146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa360072500 0x7fa3601a49a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa350008000 tx=0x7fa35000bd70 comp rx=0 tx=0).stop 2024-09-10T11:30:50.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.804+0000 7fa3649f0700 1 -- 172.21.15.144:0/1555463146 shutdown_connections 2024-09-10T11:30:50.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.804+0000 7fa3649f0700 1 --2- 172.21.15.144:0/1555463146 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fa34c06dff0 0x7fa34c0704a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.804+0000 7fa3649f0700 1 --2- 172.21.15.144:0/1555463146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa360072500 0x7fa3601a49a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.804+0000 7fa3649f0700 1 --2- 172.21.15.144:0/1555463146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3601a4ee0 0x7fa3601a9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.804+0000 7fa3649f0700 1 -- 172.21.15.144:0/1555463146 >> 172.21.15.144:0/1555463146 conn(0x7fa36006d050 msgr2=0x7fa3601126b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:50.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.805+0000 7fa3649f0700 1 -- 172.21.15.144:0/1555463146 shutdown_connections 2024-09-10T11:30:50.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.805+0000 7fa3649f0700 1 -- 172.21.15.144:0/1555463146 wait complete. 2024-09-10T11:30:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.922+0000 7fe0e3ad8700 1 -- 172.21.15.144:0/3515945778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc1013f0 msgr2=0x7fe0dc101840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.922+0000 7fe0e3ad8700 1 --2- 172.21.15.144:0/3515945778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc1013f0 0x7fe0dc101840 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009910 tx=0x7fe0cc009c20 comp rx=0 tx=0).stop 2024-09-10T11:30:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.922+0000 7fe0e3ad8700 1 -- 172.21.15.144:0/3515945778 shutdown_connections 2024-09-10T11:30:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.922+0000 7fe0e3ad8700 1 --2- 172.21.15.144:0/3515945778 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc1013f0 0x7fe0dc101840 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.922+0000 7fe0e3ad8700 1 --2- 172.21.15.144:0/3515945778 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0dc0fe410 0x7fe0dc0fe820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.922+0000 7fe0e3ad8700 1 -- 172.21.15.144:0/3515945778 >> 172.21.15.144:0/3515945778 conn(0x7fe0dc0f9d80 msgr2=0x7fe0dc0fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:50.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.923+0000 7fe0e3ad8700 1 -- 172.21.15.144:0/3515945778 shutdown_connections 2024-09-10T11:30:50.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.923+0000 7fe0e3ad8700 1 -- 172.21.15.144:0/3515945778 wait complete. 2024-09-10T11:30:50.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.923+0000 7fe0e3ad8700 1 Processor -- start 2024-09-10T11:30:50.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.923+0000 7fe0e3ad8700 1 -- start start 2024-09-10T11:30:50.927 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.923+0000 7fe0e3ad8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc0fe410 0x7fe0dc195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.927 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.923+0000 7fe0e3ad8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0dc1013f0 0x7fe0dc195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.927 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.923+0000 7fe0e3ad8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc1964b0 con 0x7fe0dc0fe410 2024-09-10T11:30:50.927 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.923+0000 7fe0e3ad8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0dc1965f0 con 0x7fe0dc1013f0 2024-09-10T11:30:50.927 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.923+0000 7fe0e1874700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc0fe410 0x7fe0dc195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.923+0000 7fe0e1874700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc0fe410 0x7fe0dc195950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39856/0 (socket says 172.21.15.144:39856) 2024-09-10T11:30:50.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.923+0000 7fe0e1874700 1 -- 172.21.15.144:0/2971119824 learned_addr learned my addr 172.21.15.144:0/2971119824 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:30:50.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.924+0000 7fe0e1073700 1 --2- 172.21.15.144:0/2971119824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0dc1013f0 0x7fe0dc195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.924+0000 7fe0e1874700 1 -- 172.21.15.144:0/2971119824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0dc1013f0 msgr2=0x7fe0dc195e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:50.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.924+0000 7fe0e1874700 1 --2- 172.21.15.144:0/2971119824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0dc1013f0 0x7fe0dc195e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:50.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.924+0000 7fe0e1874700 1 -- 172.21.15.144:0/2971119824 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0cc0095f0 con 0x7fe0dc0fe410 2024-09-10T11:30:50.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.924+0000 7fe0e1874700 1 --2- 172.21.15.144:0/2971119824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc0fe410 0x7fe0dc195950 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800e8f0 tx=0x7fe0d800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:50.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.924+0000 7fe0d2ffd700 1 -- 172.21.15.144:0/2971119824 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0d8009790 con 0x7fe0dc0fe410 2024-09-10T11:30:50.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.924+0000 7fe0e3ad8700 1 -- 172.21.15.144:0/2971119824 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0dc19b0a0 con 0x7fe0dc0fe410 2024-09-10T11:30:50.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.924+0000 7fe0d2ffd700 1 -- 172.21.15.144:0/2971119824 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe0d800c810 con 0x7fe0dc0fe410 2024-09-10T11:30:50.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.925+0000 7fe0d2ffd700 1 -- 172.21.15.144:0/2971119824 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0d8003d20 con 0x7fe0dc0fe410 2024-09-10T11:30:50.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.925+0000 7fe0e3ad8700 1 -- 172.21.15.144:0/2971119824 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0dc19b5f0 con 0x7fe0dc0fe410 2024-09-10T11:30:50.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.925+0000 7fe0e3ad8700 1 -- 172.21.15.144:0/2971119824 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0dc108c90 con 0x7fe0dc0fe410 2024-09-10T11:30:50.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.926+0000 7fe0d2ffd700 1 -- 172.21.15.144:0/2971119824 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 30) v1 ==== 92160+0+0 (secure 0 0 0) 0x7fe0d80041f0 con 0x7fe0dc0fe410 2024-09-10T11:30:50.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.926+0000 7fe0d2ffd700 1 --2- 172.21.15.144:0/2971119824 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fe0c806dfc0 0x7fe0c8070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:30:50.930 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.926+0000 7fe0d2ffd700 1 -- 172.21.15.144:0/2971119824 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fe0d8014070 con 0x7fe0dc0fe410 2024-09-10T11:30:50.930 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.926+0000 7fe0e1073700 1 --2- 172.21.15.144:0/2971119824 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fe0c806dfc0 0x7fe0c8070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:30:50.930 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.927+0000 7fe0e1073700 1 --2- 172.21.15.144:0/2971119824 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fe0c806dfc0 0x7fe0c8070470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009fd0 tx=0x7fe0cc005940 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:30:50.933 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:50.931+0000 7fe0d2ffd700 1 -- 172.21.15.144:0/2971119824 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7fe0d805b2e0 con 0x7fe0dc0fe410 2024-09-10T11:30:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:50 smithi144 ceph-mon[27942]: pgmap v21: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-09-10T11:30:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:50 smithi144 ceph-mon[27942]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:50 smithi144 ceph-mon[27942]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:50 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1555463146' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:30:51.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:51.183+0000 7fe0e3ad8700 1 -- 172.21.15.144:0/2971119824 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0dc105d40 con 0x7fe0c806dfc0 2024-09-10T11:30:51.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:51.185+0000 7fe0d2ffd700 1 -- 172.21.15.144:0/2971119824 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7fe0dc105d40 con 0x7fe0c806dfc0 2024-09-10T11:30:51.188 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:30:51.188 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:30:51.188 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:30:51.188 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T11:30:51.188 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:30:51.188 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr" 2024-09-10T11:30:51.188 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:30:51.188 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "2/2 daemons upgraded", 2024-09-10T11:30:51.188 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mgr daemons", 2024-09-10T11:30:51.188 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:30:51.188 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:30:51.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:51.189+0000 7fe0e3ad8700 1 -- 172.21.15.144:0/2971119824 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fe0c806dfc0 msgr2=0x7fe0c8070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:51.189+0000 7fe0e3ad8700 1 --2- 172.21.15.144:0/2971119824 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fe0c806dfc0 0x7fe0c8070470 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe0cc009fd0 tx=0x7fe0cc005940 comp rx=0 tx=0).stop 2024-09-10T11:30:51.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:51.189+0000 7fe0e3ad8700 1 -- 172.21.15.144:0/2971119824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc0fe410 msgr2=0x7fe0dc195950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:30:51.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:51.189+0000 7fe0e3ad8700 1 --2- 172.21.15.144:0/2971119824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc0fe410 0x7fe0dc195950 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe0d800e8f0 tx=0x7fe0d800ec00 comp rx=0 tx=0).stop 2024-09-10T11:30:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:51.189+0000 7fe0e3ad8700 1 -- 172.21.15.144:0/2971119824 shutdown_connections 2024-09-10T11:30:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:51.189+0000 7fe0e3ad8700 1 --2- 172.21.15.144:0/2971119824 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fe0c806dfc0 0x7fe0c8070470 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:51.189+0000 7fe0e3ad8700 1 --2- 172.21.15.144:0/2971119824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0dc0fe410 0x7fe0dc195950 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:51.189+0000 7fe0e3ad8700 1 --2- 172.21.15.144:0/2971119824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0dc1013f0 0x7fe0dc195e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:30:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:51.189+0000 7fe0e3ad8700 1 -- 172.21.15.144:0/2971119824 >> 172.21.15.144:0/2971119824 conn(0x7fe0dc0f9d80 msgr2=0x7fe0dc104620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:30:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:51.190+0000 7fe0e3ad8700 1 -- 172.21.15.144:0/2971119824 shutdown_connections 2024-09-10T11:30:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:30:51.190+0000 7fe0e3ad8700 1 -- 172.21.15.144:0/2971119824 wait complete. 2024-09-10T11:30:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:50 smithi178 ceph-mon[33514]: pgmap v21: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 218 op/s 2024-09-10T11:30:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:50 smithi178 ceph-mon[33514]: from='client.14708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:50 smithi178 ceph-mon[33514]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:50 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/1555463146' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:30:52.354 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:51 smithi178 ceph-mon[33514]: from='client.24473 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:52.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:51 smithi144 ceph-mon[27942]: from='client.24473 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:53.349 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:52 smithi178 ceph-mon[33514]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:53.349 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:52 smithi178 ceph-mon[33514]: pgmap v22: 65 pgs: 65 active+clean; 316 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 261 op/s 2024-09-10T11:30:53.349 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:52 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:53.349 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:52 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:53.349 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:52 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:30:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:52 smithi144 ceph-mon[27942]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:30:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:52 smithi144 ceph-mon[27942]: pgmap v22: 65 pgs: 65 active+clean; 316 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.8 MiB/s rd, 2.0 MiB/s wr, 261 op/s 2024-09-10T11:30:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:52 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:52 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:52 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:30:54.315 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:54 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:54.315 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:54 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:54.315 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:54 smithi178 ceph-mon[33514]: pgmap v23: 65 pgs: 65 active+clean; 316 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-09-10T11:30:54.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:54 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:54.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:54 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:54.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:54 smithi144 ceph-mon[27942]: pgmap v23: 65 pgs: 65 active+clean; 316 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.6 MiB/s wr, 205 op/s 2024-09-10T11:30:56.563 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:56 smithi178 ceph-mon[33514]: pgmap v24: 65 pgs: 65 active+clean; 315 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 294 op/s 2024-09-10T11:30:56.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:56 smithi144 ceph-mon[27942]: pgmap v24: 65 pgs: 65 active+clean; 315 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 2.1 MiB/s rd, 2.2 MiB/s wr, 294 op/s 2024-09-10T11:30:58.667 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:58 smithi178 ceph-mon[33514]: pgmap v25: 65 pgs: 65 active+clean; 315 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2024-09-10T11:30:58.667 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:58 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:58.667 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:58 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:58.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:58 smithi144 ceph-mon[27942]: pgmap v25: 65 pgs: 65 active+clean; 315 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 187 op/s 2024-09-10T11:30:58.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:58 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:58 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:59.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:59 smithi144 ceph-mon[27942]: Standby manager daemon smithi178.vmvxuf restarted 2024-09-10T11:30:59.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:59 smithi144 ceph-mon[27942]: Standby manager daemon smithi178.vmvxuf started 2024-09-10T11:30:59.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:59 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/3792157240' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/crt"}]: dispatch 2024-09-10T11:30:59.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:59 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/3792157240' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T11:30:59.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:59 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/3792157240' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/key"}]: dispatch 2024-09-10T11:30:59.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:59 smithi144 ceph-mon[27942]: from='mgr.? 172.21.15.178:0/3792157240' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T11:30:59.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:59 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:30:59.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:59 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:59.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:30:59 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:59.731 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:59 smithi178 ceph-mon[33514]: Standby manager daemon smithi178.vmvxuf restarted 2024-09-10T11:30:59.731 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:59 smithi178 ceph-mon[33514]: Standby manager daemon smithi178.vmvxuf started 2024-09-10T11:30:59.731 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:59 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/3792157240' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/crt"}]: dispatch 2024-09-10T11:30:59.731 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:59 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/3792157240' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T11:30:59.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:59 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/3792157240' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/key"}]: dispatch 2024-09-10T11:30:59.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:59 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/3792157240' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T11:30:59.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:59 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:30:59.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:59 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:30:59.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:30:59 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:00.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:00 smithi144 ceph-mon[27942]: pgmap v26: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 198 op/s 2024-09-10T11:31:00.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:00 smithi144 ceph-mon[27942]: mgrmap e31: smithi144.bkogfs(active, since 46s), standbys: smithi178.vmvxuf 2024-09-10T11:31:00.706 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:00 smithi178 ceph-mon[33514]: pgmap v26: 65 pgs: 65 active+clean; 313 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 198 op/s 2024-09-10T11:31:00.706 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:00 smithi178 ceph-mon[33514]: mgrmap e31: smithi144.bkogfs(active, since 46s), standbys: smithi178.vmvxuf 2024-09-10T11:31:02.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:02 smithi144 ceph-mon[27942]: pgmap v27: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 229 op/s 2024-09-10T11:31:02.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:02 smithi178 ceph-mon[33514]: pgmap v27: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 MiB/s rd, 1.7 MiB/s wr, 229 op/s 2024-09-10T11:31:04.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:04 smithi178 ceph-mon[33514]: pgmap v28: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 999 KiB/s wr, 152 op/s 2024-09-10T11:31:04.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:04 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:04.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:04 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:04.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:04 smithi144 ceph-mon[27942]: pgmap v28: 65 pgs: 65 active+clean; 312 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.1 MiB/s rd, 999 KiB/s wr, 152 op/s 2024-09-10T11:31:04.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:04 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:04.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:04 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:05.180 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-09-10T11:31:05.180 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T11:31:05.506 DEBUG:teuthology.parallel:result is None 2024-09-10T11:31:05.506 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T11:31:05.562 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T11:31:05.562 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-09-10T11:31:05.597 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-09-10T11:31:05.598 DEBUG:teuthology.parallel:result is None 2024-09-10T11:31:06.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:31:06.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:31:06.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:31:06.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:06.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: Upgrade: Setting container_image for all mgr 2024-09-10T11:31:06.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi144.bkogfs"}]: dispatch 2024-09-10T11:31:06.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi144.bkogfs"}]: dispatch 2024-09-10T11:31:06.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi144.bkogfs"}]': finished 2024-09-10T11:31:06.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi178.vmvxuf"}]: dispatch 2024-09-10T11:31:06.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi178.vmvxuf"}]: dispatch 2024-09-10T11:31:06.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi178.vmvxuf"}]': finished 2024-09-10T11:31:06.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:06.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: Upgrade: Setting container_image for all rgw 2024-09-10T11:31:06.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:06.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: Upgrade: Setting container_image for all rbd-mirror 2024-09-10T11:31:06.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:06.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: Upgrade: Setting container_image for all cephfs-mirror 2024-09-10T11:31:06.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:06.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: Upgrade: Setting container_image for all iscsi 2024-09-10T11:31:06.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:06.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: Upgrade: Setting container_image for all nfs 2024-09-10T11:31:06.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:06.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: Upgrade: Setting container_image for all nvmeof 2024-09-10T11:31:06.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:06 smithi178 ceph-mon[33514]: pgmap v29: 65 pgs: 65 active+clean; 314 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 234 op/s 2024-09-10T11:31:06.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:31:06.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:31:06.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:31:06.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:06.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: Upgrade: Setting container_image for all mgr 2024-09-10T11:31:06.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi144.bkogfs"}]: dispatch 2024-09-10T11:31:06.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi144.bkogfs"}]: dispatch 2024-09-10T11:31:06.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi144.bkogfs"}]': finished 2024-09-10T11:31:06.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi178.vmvxuf"}]: dispatch 2024-09-10T11:31:06.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi178.vmvxuf"}]: dispatch 2024-09-10T11:31:06.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi178.vmvxuf"}]': finished 2024-09-10T11:31:06.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:06.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: Upgrade: Setting container_image for all rgw 2024-09-10T11:31:06.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:06.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: Upgrade: Setting container_image for all rbd-mirror 2024-09-10T11:31:06.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:06.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: Upgrade: Setting container_image for all cephfs-mirror 2024-09-10T11:31:06.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:06.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: Upgrade: Setting container_image for all iscsi 2024-09-10T11:31:06.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:06.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: Upgrade: Setting container_image for all nfs 2024-09-10T11:31:06.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:06.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: Upgrade: Setting container_image for all nvmeof 2024-09-10T11:31:06.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:06.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:06 smithi144 ceph-mon[27942]: pgmap v29: 65 pgs: 65 active+clean; 314 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.4 MiB/s rd, 1.4 MiB/s wr, 234 op/s 2024-09-10T11:31:07.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:07 smithi144 ceph-mon[27942]: Upgrade: Updating node-exporter.smithi144 (1/2) 2024-09-10T11:31:07.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:07 smithi144 ceph-mon[27942]: Deploying daemon node-exporter.smithi144 on smithi144 2024-09-10T11:31:07.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:07 smithi178 ceph-mon[33514]: Upgrade: Updating node-exporter.smithi144 (1/2) 2024-09-10T11:31:07.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:07 smithi178 ceph-mon[33514]: Deploying daemon node-exporter.smithi144 on smithi144 2024-09-10T11:31:08.580 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:08 smithi144 ceph-mon[27942]: pgmap v30: 65 pgs: 65 active+clean; 314 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 751 KiB/s rd, 835 KiB/s wr, 145 op/s 2024-09-10T11:31:08.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:08 smithi178 ceph-mon[33514]: pgmap v30: 65 pgs: 65 active+clean; 314 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 751 KiB/s rd, 835 KiB/s wr, 145 op/s 2024-09-10T11:31:10.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:10 smithi144 ceph-mon[27942]: pgmap v31: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 773 KiB/s rd, 836 KiB/s wr, 158 op/s 2024-09-10T11:31:10.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:10 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:10.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:10 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:10.722 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:10 smithi178 ceph-mon[33514]: pgmap v31: 65 pgs: 65 active+clean; 311 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 773 KiB/s rd, 836 KiB/s wr, 158 op/s 2024-09-10T11:31:10.722 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:10 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:10.722 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:10 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:11.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:11 smithi144 ceph-mon[27942]: Upgrade: Updating node-exporter.smithi178 (2/2) 2024-09-10T11:31:11.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:11 smithi144 ceph-mon[27942]: Deploying daemon node-exporter.smithi178 on smithi178 2024-09-10T11:31:11.836 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:11 smithi178 ceph-mon[33514]: Upgrade: Updating node-exporter.smithi178 (2/2) 2024-09-10T11:31:11.836 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:11 smithi178 ceph-mon[33514]: Deploying daemon node-exporter.smithi178 on smithi178 2024-09-10T11:31:12.552 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:12 smithi178 ceph-mon[33514]: pgmap v32: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 965 KiB/s rd, 968 KiB/s wr, 173 op/s 2024-09-10T11:31:12.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:12 smithi144 ceph-mon[27942]: pgmap v32: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 965 KiB/s rd, 968 KiB/s wr, 173 op/s 2024-09-10T11:31:14.507 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:14 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:14.507 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:14 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:14.507 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:31:14.507 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:14 smithi144 ceph-mon[27942]: pgmap v33: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 565 KiB/s rd, 587 KiB/s wr, 120 op/s 2024-09-10T11:31:14.507 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:31:14.513 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:14 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:14.513 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:14 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:14.513 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:31:14.513 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:14 smithi178 ceph-mon[33514]: pgmap v33: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 565 KiB/s rd, 587 KiB/s wr, 120 op/s 2024-09-10T11:31:14.513 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:31:17.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:16 smithi144 ceph-mon[27942]: pgmap v34: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 581 KiB/s rd, 602 KiB/s wr, 123 op/s 2024-09-10T11:31:17.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:16 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:17.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:16 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:17.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:16 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:17.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:16 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:17.231 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:16 smithi178 ceph-mon[33514]: pgmap v34: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 581 KiB/s rd, 602 KiB/s wr, 123 op/s 2024-09-10T11:31:17.231 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:16 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:17.231 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:16 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:17.231 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:16 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:17.231 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:16 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:18 smithi178 ceph-mon[33514]: pgmap v35: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 230 KiB/s rd, 149 KiB/s wr, 42 op/s 2024-09-10T11:31:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:18 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:18 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:19.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:18 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:19.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:18 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:19.294 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:18 smithi144 ceph-mon[27942]: pgmap v35: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 230 KiB/s rd, 149 KiB/s wr, 42 op/s 2024-09-10T11:31:19.294 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:18 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:19.294 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:18 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:19.294 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:18 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:19.294 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:18 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:20.581 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:20 smithi144 ceph-mon[27942]: pgmap v36: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 231 KiB/s rd, 149 KiB/s wr, 44 op/s 2024-09-10T11:31:20.582 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:20 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:20.582 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:20 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:20 smithi178 ceph-mon[33514]: pgmap v36: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 231 KiB/s rd, 149 KiB/s wr, 44 op/s 2024-09-10T11:31:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:20 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:20 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:21.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.311+0000 7f70e9419700 1 -- 172.21.15.144:0/2088561102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e40ff050 msgr2=0x7f70e40ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:21.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.311+0000 7f70e9419700 1 --2- 172.21.15.144:0/2088561102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e40ff050 0x7f70e40ff460 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f70d4009910 tx=0x7f70d4009c20 comp rx=0 tx=0).stop 2024-09-10T11:31:21.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.311+0000 7f70e9419700 1 -- 172.21.15.144:0/2088561102 shutdown_connections 2024-09-10T11:31:21.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.311+0000 7f70e9419700 1 --2- 172.21.15.144:0/2088561102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70e4100250 0x7f70e41006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:21.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.311+0000 7f70e9419700 1 --2- 172.21.15.144:0/2088561102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e40ff050 0x7f70e40ff460 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:21.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.311+0000 7f70e9419700 1 -- 172.21.15.144:0/2088561102 >> 172.21.15.144:0/2088561102 conn(0x7f70e40fa5e0 msgr2=0x7f70e40fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:21.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.311+0000 7f70e9419700 1 -- 172.21.15.144:0/2088561102 shutdown_connections 2024-09-10T11:31:21.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.311+0000 7f70e9419700 1 -- 172.21.15.144:0/2088561102 wait complete. 2024-09-10T11:31:21.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.311+0000 7f70e9419700 1 Processor -- start 2024-09-10T11:31:21.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.312+0000 7f70e9419700 1 -- start start 2024-09-10T11:31:21.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.312+0000 7f70e9419700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70e40ff050 0x7f70e4195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:21.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.312+0000 7f70e9419700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e4100250 0x7f70e4195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:21.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.312+0000 7f70e9419700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e41964f0 con 0x7f70e4100250 2024-09-10T11:31:21.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.312+0000 7f70e9419700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e4196630 con 0x7f70e40ff050 2024-09-10T11:31:21.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.312+0000 7f70e27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e4100250 0x7f70e4195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:21.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.312+0000 7f70e27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e4100250 0x7f70e4195ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38076/0 (socket says 172.21.15.144:38076) 2024-09-10T11:31:21.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.312+0000 7f70e27fc700 1 -- 172.21.15.144:0/1910214656 learned_addr learned my addr 172.21.15.144:0/1910214656 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:31:21.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.312+0000 7f70e2ffd700 1 --2- 172.21.15.144:0/1910214656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70e40ff050 0x7f70e4195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:21.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.312+0000 7f70e27fc700 1 -- 172.21.15.144:0/1910214656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70e40ff050 msgr2=0x7f70e4195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:21.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.312+0000 7f70e27fc700 1 --2- 172.21.15.144:0/1910214656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70e40ff050 0x7f70e4195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:21.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.312+0000 7f70e27fc700 1 -- 172.21.15.144:0/1910214656 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d40095f0 con 0x7f70e4100250 2024-09-10T11:31:21.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.312+0000 7f70e27fc700 1 --2- 172.21.15.144:0/1910214656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e4100250 0x7f70e4195ed0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f70d800c8f0 tx=0x7f70d800cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:21.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.313+0000 7f70cbfff700 1 -- 172.21.15.144:0/1910214656 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70d8004a00 con 0x7f70e4100250 2024-09-10T11:31:21.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.313+0000 7f70e9419700 1 -- 172.21.15.144:0/1910214656 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e419b0e0 con 0x7f70e4100250 2024-09-10T11:31:21.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.313+0000 7f70cbfff700 1 -- 172.21.15.144:0/1910214656 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f70d80054d0 con 0x7f70e4100250 2024-09-10T11:31:21.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.313+0000 7f70cbfff700 1 -- 172.21.15.144:0/1910214656 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70d8007ce0 con 0x7f70e4100250 2024-09-10T11:31:21.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.313+0000 7f70e9419700 1 -- 172.21.15.144:0/1910214656 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e419b630 con 0x7f70e4100250 2024-09-10T11:31:21.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.314+0000 7f70e9419700 1 -- 172.21.15.144:0/1910214656 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e4108cc0 con 0x7f70e4100250 2024-09-10T11:31:21.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.314+0000 7f70cbfff700 1 -- 172.21.15.144:0/1910214656 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 31) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f70d8005020 con 0x7f70e4100250 2024-09-10T11:31:21.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.315+0000 7f70cbfff700 1 --2- 172.21.15.144:0/1910214656 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f70cc06dfc0 0x7f70cc070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:21.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.315+0000 7f70cbfff700 1 -- 172.21.15.144:0/1910214656 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f70d8013070 con 0x7f70e4100250 2024-09-10T11:31:21.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.315+0000 7f70e2ffd700 1 --2- 172.21.15.144:0/1910214656 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f70cc06dfc0 0x7f70cc070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:21.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.315+0000 7f70e2ffd700 1 --2- 172.21.15.144:0/1910214656 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f70cc06dfc0 0x7f70cc070470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f70d4006010 tx=0x7f70d400b540 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:21.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.320+0000 7f70cbfff700 1 -- 172.21.15.144:0/1910214656 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f70d805ad70 con 0x7f70e4100250 2024-09-10T11:31:21.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.546+0000 7f70e9419700 1 -- 172.21.15.144:0/1910214656 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f70e4104ba0 con 0x7f70cc06dfc0 2024-09-10T11:31:21.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.547+0000 7f70cbfff700 1 -- 172.21.15.144:0/1910214656 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f70e4104ba0 con 0x7f70cc06dfc0 2024-09-10T11:31:21.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.549+0000 7f70e9419700 1 -- 172.21.15.144:0/1910214656 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f70cc06dfc0 msgr2=0x7f70cc070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:21.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.549+0000 7f70e9419700 1 --2- 172.21.15.144:0/1910214656 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f70cc06dfc0 0x7f70cc070470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f70d4006010 tx=0x7f70d400b540 comp rx=0 tx=0).stop 2024-09-10T11:31:21.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.549+0000 7f70e9419700 1 -- 172.21.15.144:0/1910214656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e4100250 msgr2=0x7f70e4195ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:21.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.549+0000 7f70e9419700 1 --2- 172.21.15.144:0/1910214656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e4100250 0x7f70e4195ed0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f70d800c8f0 tx=0x7f70d800cc00 comp rx=0 tx=0).stop 2024-09-10T11:31:21.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.549+0000 7f70e9419700 1 -- 172.21.15.144:0/1910214656 shutdown_connections 2024-09-10T11:31:21.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.549+0000 7f70e9419700 1 --2- 172.21.15.144:0/1910214656 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f70cc06dfc0 0x7f70cc070470 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:21.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.549+0000 7f70e9419700 1 --2- 172.21.15.144:0/1910214656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f70e40ff050 0x7f70e4195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:21.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.549+0000 7f70e9419700 1 --2- 172.21.15.144:0/1910214656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f70e4100250 0x7f70e4195ed0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:21.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.549+0000 7f70e9419700 1 -- 172.21.15.144:0/1910214656 >> 172.21.15.144:0/1910214656 conn(0x7f70e40fa5e0 msgr2=0x7f70e4103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:21.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.549+0000 7f70e9419700 1 -- 172.21.15.144:0/1910214656 shutdown_connections 2024-09-10T11:31:21.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.549+0000 7f70e9419700 1 -- 172.21.15.144:0/1910214656 wait complete. 2024-09-10T11:31:21.561 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:31:21.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.656+0000 7f060f3e1700 1 -- 172.21.15.144:0/746267837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0608100660 msgr2=0x7f0608100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:21.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.656+0000 7f060f3e1700 1 --2- 172.21.15.144:0/746267837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0608100660 0x7f0608100a70 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f05f800b0a0 tx=0x7f05f800b3b0 comp rx=0 tx=0).stop 2024-09-10T11:31:21.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.656+0000 7f060f3e1700 1 -- 172.21.15.144:0/746267837 shutdown_connections 2024-09-10T11:31:21.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.656+0000 7f060f3e1700 1 --2- 172.21.15.144:0/746267837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06080ff2c0 0x7f06080ff730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:21.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.656+0000 7f060f3e1700 1 --2- 172.21.15.144:0/746267837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0608100660 0x7f0608100a70 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:21.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.656+0000 7f060f3e1700 1 -- 172.21.15.144:0/746267837 >> 172.21.15.144:0/746267837 conn(0x7f06080f9d80 msgr2=0x7f06080fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:21.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.656+0000 7f060f3e1700 1 -- 172.21.15.144:0/746267837 shutdown_connections 2024-09-10T11:31:21.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.656+0000 7f060f3e1700 1 -- 172.21.15.144:0/746267837 wait complete. 2024-09-10T11:31:21.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.657+0000 7f060f3e1700 1 Processor -- start 2024-09-10T11:31:21.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.657+0000 7f060f3e1700 1 -- start start 2024-09-10T11:31:21.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.657+0000 7f060f3e1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06080ff2c0 0x7f06080fed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:21.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.657+0000 7f060f3e1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0608100660 0x7f060819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:21.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.657+0000 7f060f3e1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060819adb0 con 0x7f0608100660 2024-09-10T11:31:21.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.657+0000 7f060f3e1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060819af20 con 0x7f06080ff2c0 2024-09-10T11:31:21.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.657+0000 7f060c97c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0608100660 0x7f060819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:21.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.657+0000 7f060c97c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0608100660 0x7f060819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38094/0 (socket says 172.21.15.144:38094) 2024-09-10T11:31:21.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.657+0000 7f060c97c700 1 -- 172.21.15.144:0/1379004323 learned_addr learned my addr 172.21.15.144:0/1379004323 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:31:21.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.657+0000 7f060d17d700 1 --2- 172.21.15.144:0/1379004323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06080ff2c0 0x7f06080fed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:21.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.657+0000 7f060c97c700 1 -- 172.21.15.144:0/1379004323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06080ff2c0 msgr2=0x7f06080fed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:21.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.657+0000 7f060c97c700 1 --2- 172.21.15.144:0/1379004323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06080ff2c0 0x7f06080fed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:21.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.657+0000 7f060c97c700 1 -- 172.21.15.144:0/1379004323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f8009d00 con 0x7f0608100660 2024-09-10T11:31:21.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.658+0000 7f060c97c700 1 --2- 172.21.15.144:0/1379004323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0608100660 0x7f060819a7e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00b4b0 tx=0x7f05fc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:21.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.658+0000 7f06067fc700 1 -- 172.21.15.144:0/1379004323 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05fc011800 con 0x7f0608100660 2024-09-10T11:31:21.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.658+0000 7f06067fc700 1 -- 172.21.15.144:0/1379004323 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f05fc011e10 con 0x7f0608100660 2024-09-10T11:31:21.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.658+0000 7f060f3e1700 1 -- 172.21.15.144:0/1379004323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060819b200 con 0x7f0608100660 2024-09-10T11:31:21.662 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.658+0000 7f06067fc700 1 -- 172.21.15.144:0/1379004323 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05fc00f2e0 con 0x7f0608100660 2024-09-10T11:31:21.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.658+0000 7f060f3e1700 1 -- 172.21.15.144:0/1379004323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060819b7d0 con 0x7f0608100660 2024-09-10T11:31:21.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.659+0000 7f060f3e1700 1 -- 172.21.15.144:0/1379004323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0608107c30 con 0x7f0608100660 2024-09-10T11:31:21.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.659+0000 7f06067fc700 1 -- 172.21.15.144:0/1379004323 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 31) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f05fc010380 con 0x7f0608100660 2024-09-10T11:31:21.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.660+0000 7f06067fc700 1 --2- 172.21.15.144:0/1379004323 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f05f006dfc0 0x7f05f0070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:21.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.660+0000 7f06067fc700 1 -- 172.21.15.144:0/1379004323 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f05fc08d860 con 0x7f0608100660 2024-09-10T11:31:21.665 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.660+0000 7f060d17d700 1 --2- 172.21.15.144:0/1379004323 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f05f006dfc0 0x7f05f0070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:21.665 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.660+0000 7f060d17d700 1 --2- 172.21.15.144:0/1379004323 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f05f006dfc0 0x7f05f0070470 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f05f8000f80 tx=0x7f05f800b7b0 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:21.665 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.664+0000 7f06067fc700 1 -- 172.21.15.144:0/1379004323 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f05fc05ab90 con 0x7f0608100660 2024-09-10T11:31:21.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.893+0000 7f060f3e1700 1 -- 172.21.15.144:0/1379004323 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0608104c20 con 0x7f05f006dfc0 2024-09-10T11:31:21.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.894+0000 7f06067fc700 1 -- 172.21.15.144:0/1379004323 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f0608104c20 con 0x7f05f006dfc0 2024-09-10T11:31:21.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.896+0000 7f060f3e1700 1 -- 172.21.15.144:0/1379004323 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f05f006dfc0 msgr2=0x7f05f0070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:21.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.896+0000 7f060f3e1700 1 --2- 172.21.15.144:0/1379004323 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f05f006dfc0 0x7f05f0070470 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f05f8000f80 tx=0x7f05f800b7b0 comp rx=0 tx=0).stop 2024-09-10T11:31:21.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.896+0000 7f060f3e1700 1 -- 172.21.15.144:0/1379004323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0608100660 msgr2=0x7f060819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:21.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.896+0000 7f060f3e1700 1 --2- 172.21.15.144:0/1379004323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0608100660 0x7f060819a7e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00b4b0 tx=0x7f05fc00b870 comp rx=0 tx=0).stop 2024-09-10T11:31:21.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.897+0000 7f060f3e1700 1 -- 172.21.15.144:0/1379004323 shutdown_connections 2024-09-10T11:31:21.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.897+0000 7f060f3e1700 1 --2- 172.21.15.144:0/1379004323 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f05f006dfc0 0x7f05f0070470 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:21.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.897+0000 7f060f3e1700 1 --2- 172.21.15.144:0/1379004323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06080ff2c0 0x7f06080fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:21.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.897+0000 7f060f3e1700 1 --2- 172.21.15.144:0/1379004323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0608100660 0x7f060819a7e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:21.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.897+0000 7f060f3e1700 1 -- 172.21.15.144:0/1379004323 >> 172.21.15.144:0/1379004323 conn(0x7f06080f9d80 msgr2=0x7f0608103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:21.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.897+0000 7f060f3e1700 1 -- 172.21.15.144:0/1379004323 shutdown_connections 2024-09-10T11:31:21.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:21.897+0000 7f060f3e1700 1 -- 172.21.15.144:0/1379004323 wait complete. 2024-09-10T11:31:22.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.004+0000 7f1755f1a700 1 -- 172.21.15.144:0/2520243660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1750100170 msgr2=0x7f1750100580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.004+0000 7f1755f1a700 1 --2- 172.21.15.144:0/2520243660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1750100170 0x7f1750100580 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f174000b0a0 tx=0x7f174000b3b0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.004+0000 7f1755f1a700 1 -- 172.21.15.144:0/2520243660 shutdown_connections 2024-09-10T11:31:22.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.004+0000 7f1755f1a700 1 --2- 172.21.15.144:0/2520243660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1750101370 0x7f17501017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.004+0000 7f1755f1a700 1 --2- 172.21.15.144:0/2520243660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1750100170 0x7f1750100580 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.004+0000 7f1755f1a700 1 -- 172.21.15.144:0/2520243660 >> 172.21.15.144:0/2520243660 conn(0x7f17500fb6e0 msgr2=0x7f17500fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:22.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.005+0000 7f1755f1a700 1 -- 172.21.15.144:0/2520243660 shutdown_connections 2024-09-10T11:31:22.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.005+0000 7f1755f1a700 1 -- 172.21.15.144:0/2520243660 wait complete. 2024-09-10T11:31:22.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.005+0000 7f1755f1a700 1 Processor -- start 2024-09-10T11:31:22.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.005+0000 7f1755f1a700 1 -- start start 2024-09-10T11:31:22.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.005+0000 7f1755f1a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1750100170 0x7f1750195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.005+0000 7f1755f1a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1750101370 0x7f1750195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.005+0000 7f1755f1a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1750196610 con 0x7f1750100170 2024-09-10T11:31:22.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.005+0000 7f1755f1a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1750196750 con 0x7f1750101370 2024-09-10T11:31:22.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.006+0000 7f174f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1750100170 0x7f1750195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.006+0000 7f174f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1750100170 0x7f1750195ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38116/0 (socket says 172.21.15.144:38116) 2024-09-10T11:31:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.006+0000 7f174f7fe700 1 -- 172.21.15.144:0/1600804273 learned_addr learned my addr 172.21.15.144:0/1600804273 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:31:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.006+0000 7f174effd700 1 --2- 172.21.15.144:0/1600804273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1750101370 0x7f1750195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.006+0000 7f174f7fe700 1 -- 172.21.15.144:0/1600804273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1750101370 msgr2=0x7f1750195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.006+0000 7f174f7fe700 1 --2- 172.21.15.144:0/1600804273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1750101370 0x7f1750195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.006+0000 7f174f7fe700 1 -- 172.21.15.144:0/1600804273 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1740009d00 con 0x7f1750100170 2024-09-10T11:31:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.006+0000 7f174effd700 1 --2- 172.21.15.144:0/1600804273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1750101370 0x7f1750195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:31:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.006+0000 7f174f7fe700 1 --2- 172.21.15.144:0/1600804273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1750100170 0x7f1750195ab0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f17400077b0 tx=0x7f1740003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.006+0000 7f174cff9700 1 -- 172.21.15.144:0/1600804273 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f174000e040 con 0x7f1750100170 2024-09-10T11:31:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.006+0000 7f174cff9700 1 -- 172.21.15.144:0/1600804273 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f174000ce60 con 0x7f1750100170 2024-09-10T11:31:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.006+0000 7f1755f1a700 1 -- 172.21.15.144:0/1600804273 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f175019b1a0 con 0x7f1750100170 2024-09-10T11:31:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.006+0000 7f174cff9700 1 -- 172.21.15.144:0/1600804273 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f174001db10 con 0x7f1750100170 2024-09-10T11:31:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.006+0000 7f1755f1a700 1 -- 172.21.15.144:0/1600804273 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f175019b710 con 0x7f1750100170 2024-09-10T11:31:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.007+0000 7f1755f1a700 1 -- 172.21.15.144:0/1600804273 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1750108d40 con 0x7f1750100170 2024-09-10T11:31:22.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.008+0000 7f174cff9700 1 -- 172.21.15.144:0/1600804273 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 31) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f174001dc90 con 0x7f1750100170 2024-09-10T11:31:22.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.008+0000 7f174cff9700 1 --2- 172.21.15.144:0/1600804273 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f173806dfc0 0x7f1738070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.008+0000 7f174cff9700 1 -- 172.21.15.144:0/1600804273 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f174008f720 con 0x7f1750100170 2024-09-10T11:31:22.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.008+0000 7f174effd700 1 --2- 172.21.15.144:0/1600804273 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f173806dfc0 0x7f1738070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.009+0000 7f174effd700 1 --2- 172.21.15.144:0/1600804273 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f173806dfc0 0x7f1738070470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f17440099d0 tx=0x7f1744008040 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:22.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.013+0000 7f174cff9700 1 -- 172.21.15.144:0/1600804273 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f174005c9c0 con 0x7f1750100170 2024-09-10T11:31:22.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.238+0000 7f1755f1a700 1 -- 172.21.15.144:0/1600804273 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1750105cc0 con 0x7f173806dfc0 2024-09-10T11:31:22.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.247+0000 7f174cff9700 1 -- 172.21.15.144:0/1600804273 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f1750105cc0 con 0x7f173806dfc0 2024-09-10T11:31:22.248 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:31:22.248 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (8m) 3s ago 10m 24.7M - 0.25.0 c8568f914cd2 b5c0bfae59de 2024-09-10T11:31:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (10m) 3s ago 10m 7914k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:31:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (9m) 6s ago 9m 7830k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:31:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (10m) 3s ago 10m 6669k - 18.2.0 dc2bc1663786 3311fb04aa0c 2024-09-10T11:31:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (9m) 6s ago 9m 6651k - 18.2.0 dc2bc1663786 6e408e8476d4 2024-09-10T11:31:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (8m) 3s ago 9m 86.0M - 9.4.7 954c08fa6188 e9a1aec955f6 2024-09-10T11:31:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (4m) 3s ago 4m 59.9M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:31:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (4m) 3s ago 4m 126M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:31:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (4m) 6s ago 4m 65.7M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:31:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (4m) 6s ago 4m 127M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:31:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (79s) 3s ago 11m 495M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:31:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (30s) 6s ago 9m 441M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:31:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (11m) 3s ago 11m 54.1M 2048M 18.2.0 dc2bc1663786 42addb654e4f 2024-09-10T11:31:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (9m) 6s ago 9m 43.5M 2048M 18.2.0 dc2bc1663786 3c1ba210ed3f 2024-09-10T11:31:22.249 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (13s) 3s ago 10m 5444k - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:31:22.250 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (9s) 6s ago 9m 3586k - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:31:22.250 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (8m) 3s ago 8m 205M 1216M 18.2.0 dc2bc1663786 2435febebdb5 2024-09-10T11:31:22.250 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (7m) 3s ago 7m 210M 1216M 18.2.0 dc2bc1663786 b55fe920ff9d 2024-09-10T11:31:22.250 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (7m) 3s ago 7m 168M 1216M 18.2.0 dc2bc1663786 3ed5618eac33 2024-09-10T11:31:22.250 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (7m) 6s ago 7m 252M 2239M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:31:22.250 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (6m) 6s ago 6m 217M 2239M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:31:22.250 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (6m) 6s ago 6m 195M 2239M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:31:22.250 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (40s) 3s ago 9m 60.7M - 2.43.0 a07b618ecd1d 0ee8f2562ab9 2024-09-10T11:31:22.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.250+0000 7f1755f1a700 1 -- 172.21.15.144:0/1600804273 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f173806dfc0 msgr2=0x7f1738070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.250+0000 7f1755f1a700 1 --2- 172.21.15.144:0/1600804273 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f173806dfc0 0x7f1738070470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f17440099d0 tx=0x7f1744008040 comp rx=0 tx=0).stop 2024-09-10T11:31:22.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.250+0000 7f1755f1a700 1 -- 172.21.15.144:0/1600804273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1750100170 msgr2=0x7f1750195ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.250+0000 7f1755f1a700 1 --2- 172.21.15.144:0/1600804273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1750100170 0x7f1750195ab0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f17400077b0 tx=0x7f1740003ce0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.250+0000 7f1755f1a700 1 -- 172.21.15.144:0/1600804273 shutdown_connections 2024-09-10T11:31:22.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.250+0000 7f1755f1a700 1 --2- 172.21.15.144:0/1600804273 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f173806dfc0 0x7f1738070470 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.250+0000 7f1755f1a700 1 --2- 172.21.15.144:0/1600804273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1750100170 0x7f1750195ab0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.250+0000 7f1755f1a700 1 --2- 172.21.15.144:0/1600804273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1750101370 0x7f1750195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.250+0000 7f1755f1a700 1 -- 172.21.15.144:0/1600804273 >> 172.21.15.144:0/1600804273 conn(0x7f17500fb6e0 msgr2=0x7f17501045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:22.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.250+0000 7f1755f1a700 1 -- 172.21.15.144:0/1600804273 shutdown_connections 2024-09-10T11:31:22.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.250+0000 7f1755f1a700 1 -- 172.21.15.144:0/1600804273 wait complete. 2024-09-10T11:31:22.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.358+0000 7f09e02af700 1 -- 172.21.15.144:0/1532825667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d8103560 msgr2=0x7f09d8105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.358+0000 7f09e02af700 1 --2- 172.21.15.144:0/1532825667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d8103560 0x7f09d8105940 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f09c8009910 tx=0x7f09c8009c20 comp rx=0 tx=0).stop 2024-09-10T11:31:22.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.359+0000 7f09e02af700 1 -- 172.21.15.144:0/1532825667 shutdown_connections 2024-09-10T11:31:22.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.359+0000 7f09e02af700 1 --2- 172.21.15.144:0/1532825667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d8103560 0x7f09d8105940 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.359+0000 7f09e02af700 1 --2- 172.21.15.144:0/1532825667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d8100c40 0x7f09d8103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.359+0000 7f09e02af700 1 -- 172.21.15.144:0/1532825667 >> 172.21.15.144:0/1532825667 conn(0x7f09d80fa5e0 msgr2=0x7f09d80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:22.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.359+0000 7f09e02af700 1 -- 172.21.15.144:0/1532825667 shutdown_connections 2024-09-10T11:31:22.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.359+0000 7f09e02af700 1 -- 172.21.15.144:0/1532825667 wait complete. 2024-09-10T11:31:22.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.359+0000 7f09e02af700 1 Processor -- start 2024-09-10T11:31:22.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.359+0000 7f09e02af700 1 -- start start 2024-09-10T11:31:22.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.359+0000 7f09e02af700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d8100c40 0x7f09d81005a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.359+0000 7f09e02af700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d8103560 0x7f09d80febf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.359+0000 7f09e02af700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d80ff130 con 0x7f09d8103560 2024-09-10T11:31:22.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.359+0000 7f09e02af700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d80ff270 con 0x7f09d8100c40 2024-09-10T11:31:22.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.360+0000 7f09dd84a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d8103560 0x7f09d80febf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.360+0000 7f09dd84a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d8103560 0x7f09d80febf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33448/0 (socket says 172.21.15.144:33448) 2024-09-10T11:31:22.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.360+0000 7f09dd84a700 1 -- 172.21.15.144:0/2097063242 learned_addr learned my addr 172.21.15.144:0/2097063242 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:31:22.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.360+0000 7f09de04b700 1 --2- 172.21.15.144:0/2097063242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d8100c40 0x7f09d81005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.360+0000 7f09dd84a700 1 -- 172.21.15.144:0/2097063242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d8100c40 msgr2=0x7f09d81005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.360+0000 7f09dd84a700 1 --2- 172.21.15.144:0/2097063242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d8100c40 0x7f09d81005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.360+0000 7f09dd84a700 1 -- 172.21.15.144:0/2097063242 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c80095f0 con 0x7f09d8103560 2024-09-10T11:31:22.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.360+0000 7f09dd84a700 1 --2- 172.21.15.144:0/2097063242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d8103560 0x7f09d80febf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f09c8006010 tx=0x7f09c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:22.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.360+0000 7f09cf7fe700 1 -- 172.21.15.144:0/2097063242 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09c801d070 con 0x7f09d8103560 2024-09-10T11:31:22.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.360+0000 7f09cf7fe700 1 -- 172.21.15.144:0/2097063242 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f09c800f460 con 0x7f09d8103560 2024-09-10T11:31:22.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.360+0000 7f09cf7fe700 1 -- 172.21.15.144:0/2097063242 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09c8017630 con 0x7f09d8103560 2024-09-10T11:31:22.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.360+0000 7f09e02af700 1 -- 172.21.15.144:0/2097063242 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09d80ff4f0 con 0x7f09d8103560 2024-09-10T11:31:22.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.360+0000 7f09e02af700 1 -- 172.21.15.144:0/2097063242 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09d80ff9e0 con 0x7f09d8103560 2024-09-10T11:31:22.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.361+0000 7f09e02af700 1 -- 172.21.15.144:0/2097063242 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d818fb70 con 0x7f09d8103560 2024-09-10T11:31:22.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.362+0000 7f09cf7fe700 1 -- 172.21.15.144:0/2097063242 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 31) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f09c8003a60 con 0x7f09d8103560 2024-09-10T11:31:22.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.362+0000 7f09cf7fe700 1 --2- 172.21.15.144:0/2097063242 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f09c406dfc0 0x7f09c4070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.362+0000 7f09cf7fe700 1 -- 172.21.15.144:0/2097063242 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f09c808f1b0 con 0x7f09d8103560 2024-09-10T11:31:22.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.362+0000 7f09de04b700 1 --2- 172.21.15.144:0/2097063242 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f09c406dfc0 0x7f09c4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.363+0000 7f09de04b700 1 --2- 172.21.15.144:0/2097063242 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f09c406dfc0 0x7f09c4070470 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f09d4005950 tx=0x7f09d40058e0 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:22.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.367+0000 7f09cf7fe700 1 -- 172.21.15.144:0/2097063242 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f09c805c450 con 0x7f09d8103560 2024-09-10T11:31:22.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: pgmap v37: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 209 KiB/s rd, 148 KiB/s wr, 31 op/s 2024-09-10T11:31:22.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:22.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:22.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:22.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:22.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:31:22.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:31:22.615 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:22.615 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:31:22.615 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:22.615 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:22.615 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:22.615 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:22.615 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:22.615 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:22.615 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:22.615 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:22 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:22.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.664+0000 7f09e02af700 1 -- 172.21.15.144:0/2097063242 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f09d802cc30 con 0x7f09d8103560 2024-09-10T11:31:22.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.665+0000 7f09cf7fe700 1 -- 172.21.15.144:0/2097063242 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f09c805bfe0 con 0x7f09d8103560 2024-09-10T11:31:22.666 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:31:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:31:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-09-10T11:31:22.666 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:31:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:31:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:31:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:31:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:31:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-09-10T11:31:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:31:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:31:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:31:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:31:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:31:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-09-10T11:31:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:31:22.667 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:31:22.668 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:31:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.667+0000 7f09e02af700 1 -- 172.21.15.144:0/2097063242 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f09c406dfc0 msgr2=0x7f09c4070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.667+0000 7f09e02af700 1 --2- 172.21.15.144:0/2097063242 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f09c406dfc0 0x7f09c4070470 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f09d4005950 tx=0x7f09d40058e0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.667+0000 7f09e02af700 1 -- 172.21.15.144:0/2097063242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d8103560 msgr2=0x7f09d80febf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.667+0000 7f09e02af700 1 --2- 172.21.15.144:0/2097063242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d8103560 0x7f09d80febf0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f09c8006010 tx=0x7f09c800be20 comp rx=0 tx=0).stop 2024-09-10T11:31:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.667+0000 7f09e02af700 1 -- 172.21.15.144:0/2097063242 shutdown_connections 2024-09-10T11:31:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.667+0000 7f09e02af700 1 --2- 172.21.15.144:0/2097063242 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f09c406dfc0 0x7f09c4070470 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.667+0000 7f09e02af700 1 --2- 172.21.15.144:0/2097063242 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09d8100c40 0x7f09d81005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.667+0000 7f09e02af700 1 --2- 172.21.15.144:0/2097063242 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09d8103560 0x7f09d80febf0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.667+0000 7f09e02af700 1 -- 172.21.15.144:0/2097063242 >> 172.21.15.144:0/2097063242 conn(0x7f09d80fa5e0 msgr2=0x7f09d80fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.668+0000 7f09e02af700 1 -- 172.21.15.144:0/2097063242 shutdown_connections 2024-09-10T11:31:22.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.668+0000 7f09e02af700 1 -- 172.21.15.144:0/2097063242 wait complete. 2024-09-10T11:31:22.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.776+0000 7f2c336ad700 1 -- 172.21.15.144:0/1138583687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c2c0ff050 msgr2=0x7f2c2c0ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.776+0000 7f2c336ad700 1 --2- 172.21.15.144:0/1138583687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c2c0ff050 0x7f2c2c0ff460 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c009910 tx=0x7f2c1c009c20 comp rx=0 tx=0).stop 2024-09-10T11:31:22.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.776+0000 7f2c336ad700 1 -- 172.21.15.144:0/1138583687 shutdown_connections 2024-09-10T11:31:22.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.776+0000 7f2c336ad700 1 --2- 172.21.15.144:0/1138583687 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c2c100250 0x7f2c2c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.776+0000 7f2c336ad700 1 --2- 172.21.15.144:0/1138583687 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c2c0ff050 0x7f2c2c0ff460 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.776+0000 7f2c336ad700 1 -- 172.21.15.144:0/1138583687 >> 172.21.15.144:0/1138583687 conn(0x7f2c2c0fa5e0 msgr2=0x7f2c2c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:22.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.776+0000 7f2c336ad700 1 -- 172.21.15.144:0/1138583687 shutdown_connections 2024-09-10T11:31:22.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.776+0000 7f2c336ad700 1 -- 172.21.15.144:0/1138583687 wait complete. 2024-09-10T11:31:22.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.777+0000 7f2c336ad700 1 Processor -- start 2024-09-10T11:31:22.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.777+0000 7f2c336ad700 1 -- start start 2024-09-10T11:31:22.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.777+0000 7f2c336ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c2c0ff050 0x7f2c2c195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.777+0000 7f2c336ad700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c2c100250 0x7f2c2c195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.777+0000 7f2c336ad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c196580 con 0x7f2c2c0ff050 2024-09-10T11:31:22.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.777+0000 7f2c336ad700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c2c1966c0 con 0x7f2c2c100250 2024-09-10T11:31:22.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.777+0000 7f2c31449700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c2c0ff050 0x7f2c2c195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.777+0000 7f2c31449700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c2c0ff050 0x7f2c2c195a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33476/0 (socket says 172.21.15.144:33476) 2024-09-10T11:31:22.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.777+0000 7f2c31449700 1 -- 172.21.15.144:0/2422387192 learned_addr learned my addr 172.21.15.144:0/2422387192 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:31:22.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.777+0000 7f2c30c48700 1 --2- 172.21.15.144:0/2422387192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c2c100250 0x7f2c2c195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.777+0000 7f2c31449700 1 -- 172.21.15.144:0/2422387192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c2c100250 msgr2=0x7f2c2c195f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:22.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.777+0000 7f2c31449700 1 --2- 172.21.15.144:0/2422387192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c2c100250 0x7f2c2c195f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:22.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.777+0000 7f2c31449700 1 -- 172.21.15.144:0/2422387192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c1c0095f0 con 0x7f2c2c0ff050 2024-09-10T11:31:22.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.777+0000 7f2c31449700 1 --2- 172.21.15.144:0/2422387192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c2c0ff050 0x7f2c2c195a20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00bda0 tx=0x7f2c1c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:22.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.778+0000 7f2c227fc700 1 -- 172.21.15.144:0/2422387192 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c1c01d070 con 0x7f2c2c0ff050 2024-09-10T11:31:22.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.778+0000 7f2c227fc700 1 -- 172.21.15.144:0/2422387192 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2c1c004c50 con 0x7f2c2c0ff050 2024-09-10T11:31:22.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.778+0000 7f2c336ad700 1 -- 172.21.15.144:0/2422387192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c2c19b110 con 0x7f2c2c0ff050 2024-09-10T11:31:22.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.778+0000 7f2c227fc700 1 -- 172.21.15.144:0/2422387192 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c1c00f690 con 0x7f2c2c0ff050 2024-09-10T11:31:22.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.778+0000 7f2c336ad700 1 -- 172.21.15.144:0/2422387192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c2c19b680 con 0x7f2c2c0ff050 2024-09-10T11:31:22.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.779+0000 7f2c336ad700 1 -- 172.21.15.144:0/2422387192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c2c107c20 con 0x7f2c2c0ff050 2024-09-10T11:31:22.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.779+0000 7f2c227fc700 1 -- 172.21.15.144:0/2422387192 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 31) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f2c1c0229b0 con 0x7f2c2c0ff050 2024-09-10T11:31:22.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.780+0000 7f2c227fc700 1 --2- 172.21.15.144:0/2422387192 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f2c1806dfc0 0x7f2c18070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:22.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.780+0000 7f2c227fc700 1 -- 172.21.15.144:0/2422387192 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f2c1c08f480 con 0x7f2c2c0ff050 2024-09-10T11:31:22.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.780+0000 7f2c30c48700 1 --2- 172.21.15.144:0/2422387192 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f2c1806dfc0 0x7f2c18070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:22.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.780+0000 7f2c30c48700 1 --2- 172.21.15.144:0/2422387192 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f2c1806dfc0 0x7f2c18070470 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2c28006fd0 tx=0x7f2c28008040 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:22.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:22.784+0000 7f2c227fc700 1 -- 172.21.15.144:0/2422387192 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f2c1c05c6f0 con 0x7f2c2c0ff050 2024-09-10T11:31:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: pgmap v37: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 209 KiB/s rd, 148 KiB/s wr, 31 op/s 2024-09-10T11:31:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='client.14736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:31:22.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:31:22.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:22.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:31:22.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:22.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:22.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:22.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:22.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:22.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:22.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:22.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:22 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:23.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:23.016+0000 7f2c336ad700 1 -- 172.21.15.144:0/2422387192 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c2c068ae0 con 0x7f2c1806dfc0 2024-09-10T11:31:23.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:23.017+0000 7f2c227fc700 1 -- 172.21.15.144:0/2422387192 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f2c2c068ae0 con 0x7f2c1806dfc0 2024-09-10T11:31:23.019 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:31:23.019 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:31:23.019 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:31:23.019 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T11:31:23.019 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:31:23.019 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr" 2024-09-10T11:31:23.019 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:31:23.019 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "2/2 daemons upgraded", 2024-09-10T11:31:23.019 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading prometheus daemons", 2024-09-10T11:31:23.019 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:31:23.019 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:31:23.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:23.020+0000 7f2c336ad700 1 -- 172.21.15.144:0/2422387192 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f2c1806dfc0 msgr2=0x7f2c18070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:23.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:23.020+0000 7f2c336ad700 1 --2- 172.21.15.144:0/2422387192 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f2c1806dfc0 0x7f2c18070470 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2c28006fd0 tx=0x7f2c28008040 comp rx=0 tx=0).stop 2024-09-10T11:31:23.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:23.020+0000 7f2c336ad700 1 -- 172.21.15.144:0/2422387192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c2c0ff050 msgr2=0x7f2c2c195a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:23.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:23.020+0000 7f2c336ad700 1 --2- 172.21.15.144:0/2422387192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c2c0ff050 0x7f2c2c195a20 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00bda0 tx=0x7f2c1c00be80 comp rx=0 tx=0).stop 2024-09-10T11:31:23.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:23.020+0000 7f2c336ad700 1 -- 172.21.15.144:0/2422387192 shutdown_connections 2024-09-10T11:31:23.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:23.020+0000 7f2c336ad700 1 --2- 172.21.15.144:0/2422387192 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f2c1806dfc0 0x7f2c18070470 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:23.020+0000 7f2c336ad700 1 --2- 172.21.15.144:0/2422387192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c2c0ff050 0x7f2c2c195a20 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:23.020+0000 7f2c336ad700 1 --2- 172.21.15.144:0/2422387192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c2c100250 0x7f2c2c195f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:23.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:23.020+0000 7f2c336ad700 1 -- 172.21.15.144:0/2422387192 >> 172.21.15.144:0/2422387192 conn(0x7f2c2c0fa5e0 msgr2=0x7f2c2c0683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:23.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:23.020+0000 7f2c336ad700 1 -- 172.21.15.144:0/2422387192 shutdown_connections 2024-09-10T11:31:23.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:23.020+0000 7f2c336ad700 1 -- 172.21.15.144:0/2422387192 wait complete. 2024-09-10T11:31:23.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:23 smithi178 ceph-mon[33514]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:23.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:23 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/2097063242' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:23.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:23 smithi144 ceph-mon[27942]: from='client.14740 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:23.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:23 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2097063242' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:24.660 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:24 smithi144 ceph-mon[27942]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:24.661 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:24 smithi144 ceph-mon[27942]: pgmap v38: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 15 KiB/s wr, 4 op/s 2024-09-10T11:31:24.661 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:24 smithi144 ceph-mon[27942]: Upgrade: Updating prometheus.smithi144 2024-09-10T11:31:24.661 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:24 smithi144 ceph-mon[27942]: Deploying daemon prometheus.smithi144 on smithi144 2024-09-10T11:31:24.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:24 smithi178 ceph-mon[33514]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:24.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:24 smithi178 ceph-mon[33514]: pgmap v38: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 15 KiB/s wr, 4 op/s 2024-09-10T11:31:24.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:24 smithi178 ceph-mon[33514]: Upgrade: Updating prometheus.smithi144 2024-09-10T11:31:24.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:24 smithi178 ceph-mon[33514]: Deploying daemon prometheus.smithi144 on smithi144 2024-09-10T11:31:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:26 smithi144 ceph-mon[27942]: pgmap v39: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-09-10T11:31:26.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:26 smithi178 ceph-mon[33514]: pgmap v39: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 18 KiB/s rd, 15 KiB/s wr, 7 op/s 2024-09-10T11:31:28.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:28 smithi178 ceph-mon[33514]: pgmap v40: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 4 op/s 2024-09-10T11:31:28.765 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:28 smithi144 ceph-mon[27942]: pgmap v40: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 170 B/s wr, 4 op/s 2024-09-10T11:31:29.811 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:29 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:31:29.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:29 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:31:31.225 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:30 smithi144 ceph-mon[27942]: pgmap v41: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-10T11:31:31.226 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:30 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:31.226 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:30 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:31.226 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:30 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:31:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:30 smithi178 ceph-mon[33514]: pgmap v41: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 170 B/s wr, 5 op/s 2024-09-10T11:31:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:30 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:30 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:30 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:31:33.254 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:32 smithi144 ceph-mon[27942]: pgmap v42: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:31:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:32 smithi178 ceph-mon[33514]: pgmap v42: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:31:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:34 smithi144 ceph-mon[27942]: pgmap v43: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-10T11:31:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:34 smithi178 ceph-mon[33514]: pgmap v43: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-10T11:31:36.512 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:36 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:36.513 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:36 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:36.513 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:36 smithi144 ceph-mon[27942]: pgmap v44: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-10T11:31:36.513 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:36 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:36.513 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:36 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:36.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:36 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:36.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:36 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:36.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:36 smithi178 ceph-mon[33514]: pgmap v44: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-10T11:31:36.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:36 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:36.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:36 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:38.539 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:38 smithi144 ceph-mon[27942]: pgmap v45: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-10T11:31:38.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:38 smithi178 ceph-mon[33514]: pgmap v45: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-10T11:31:40.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:40.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:40.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:31:40.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:31:40.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:40.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:31:40.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:31:40.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:31:40.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:39 smithi144 ceph-mon[27942]: pgmap v46: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 5 op/s 2024-09-10T11:31:40.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:40.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:40.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:31:40.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:31:40.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:40.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:31:40.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T11:31:40.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:31:40.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:40.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:39 smithi178 ceph-mon[33514]: pgmap v46: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 5 op/s 2024-09-10T11:31:41.128 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:40 smithi144 ceph-mon[27942]: Upgrade: Updating alertmanager.smithi144 2024-09-10T11:31:41.128 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:40 smithi144 ceph-mon[27942]: Deploying daemon alertmanager.smithi144 on smithi144 2024-09-10T11:31:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:40 smithi178 ceph-mon[33514]: Upgrade: Updating alertmanager.smithi144 2024-09-10T11:31:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:40 smithi178 ceph-mon[33514]: Deploying daemon alertmanager.smithi144 on smithi144 2024-09-10T11:31:42.131 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:42 smithi144 ceph-mon[27942]: pgmap v47: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:31:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:42 smithi178 ceph-mon[33514]: pgmap v47: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:31:44.610 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:44 smithi144 ceph-mon[27942]: pgmap v48: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-10T11:31:44.610 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:44 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:31:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:44 smithi178 ceph-mon[33514]: pgmap v48: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s 2024-09-10T11:31:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:44 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:31:45.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:45 smithi144 ceph-mon[27942]: mgrmap e32: smithi144.bkogfs(active, since 91s), standbys: smithi178.vmvxuf 2024-09-10T11:31:45.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:45 smithi178 ceph-mon[33514]: mgrmap e32: smithi144.bkogfs(active, since 91s), standbys: smithi178.vmvxuf 2024-09-10T11:31:46.828 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:46 smithi144 ceph-mon[27942]: pgmap v49: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T11:31:46.828 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:46 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:46.828 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:46 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:46.828 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:46 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:31:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:46 smithi178 ceph-mon[33514]: pgmap v49: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T11:31:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:46 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:46 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:46 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:31:48.871 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:48 smithi178 ceph-mon[33514]: pgmap v50: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T11:31:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:48 smithi144 ceph-mon[27942]: pgmap v50: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T11:31:50.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:50 smithi178 ceph-mon[33514]: pgmap v51: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T11:31:50.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:50 smithi144 ceph-mon[27942]: pgmap v51: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T11:31:51.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:51 smithi178 ceph-mon[33514]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:31:51.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:51 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:51.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:51 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:51.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:51 smithi144 ceph-mon[27942]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:31:51.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:51 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:51.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:51 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:53.115 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:52 smithi144 ceph-mon[27942]: pgmap v52: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T11:31:53.115 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:52 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:53.115 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:52 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:52 smithi178 ceph-mon[33514]: pgmap v52: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T11:31:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:52 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:52 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:53.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.131+0000 7f89b05f7700 1 -- 172.21.15.144:0/871053724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89a80ff060 msgr2=0x7f89a80ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:53.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.131+0000 7f89b05f7700 1 --2- 172.21.15.144:0/871053724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89a80ff060 0x7f89a80ff470 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f89a4009910 tx=0x7f89a4009c20 comp rx=0 tx=0).stop 2024-09-10T11:31:53.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.132+0000 7f89b05f7700 1 -- 172.21.15.144:0/871053724 shutdown_connections 2024-09-10T11:31:53.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.132+0000 7f89b05f7700 1 --2- 172.21.15.144:0/871053724 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89a8100260 0x7f89a81006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.132+0000 7f89b05f7700 1 --2- 172.21.15.144:0/871053724 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89a80ff060 0x7f89a80ff470 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.132+0000 7f89b05f7700 1 -- 172.21.15.144:0/871053724 >> 172.21.15.144:0/871053724 conn(0x7f89a80fa610 msgr2=0x7f89a80fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:53.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.132+0000 7f89b05f7700 1 -- 172.21.15.144:0/871053724 shutdown_connections 2024-09-10T11:31:53.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.132+0000 7f89b05f7700 1 -- 172.21.15.144:0/871053724 wait complete. 2024-09-10T11:31:53.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.132+0000 7f89b05f7700 1 Processor -- start 2024-09-10T11:31:53.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.132+0000 7f89b05f7700 1 -- start start 2024-09-10T11:31:53.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f89b05f7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89a80ff060 0x7f89a80698d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:53.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f89b05f7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89a8100260 0x7f89a819a790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:53.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f89b05f7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a819ad60 con 0x7f89a8100260 2024-09-10T11:31:53.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f89b05f7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89a819aea0 con 0x7f89a80ff060 2024-09-10T11:31:53.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f89adb92700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89a8100260 0x7f89a819a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:53.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f89adb92700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89a8100260 0x7f89a819a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54672/0 (socket says 172.21.15.144:54672) 2024-09-10T11:31:53.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f89adb92700 1 -- 172.21.15.144:0/416029942 learned_addr learned my addr 172.21.15.144:0/416029942 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:31:53.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f89ae393700 1 --2- 172.21.15.144:0/416029942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89a80ff060 0x7f89a80698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:53.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f89adb92700 1 -- 172.21.15.144:0/416029942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89a80ff060 msgr2=0x7f89a80698d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:53.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f89adb92700 1 --2- 172.21.15.144:0/416029942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89a80ff060 0x7f89a80698d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f89adb92700 1 -- 172.21.15.144:0/416029942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89a40095f0 con 0x7f89a8100260 2024-09-10T11:31:53.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f89adb92700 1 --2- 172.21.15.144:0/416029942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89a8100260 0x7f89a819a790 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f899800b4b0 tx=0x7f899800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:53.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f899f7fe700 1 -- 172.21.15.144:0/416029942 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8998011800 con 0x7f89a8100260 2024-09-10T11:31:53.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f899f7fe700 1 -- 172.21.15.144:0/416029942 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8998011e10 con 0x7f89a8100260 2024-09-10T11:31:53.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f89b05f7700 1 -- 172.21.15.144:0/416029942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89a819b180 con 0x7f89a8100260 2024-09-10T11:31:53.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f899f7fe700 1 -- 172.21.15.144:0/416029942 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f899800f2e0 con 0x7f89a8100260 2024-09-10T11:31:53.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.133+0000 7f89b05f7700 1 -- 172.21.15.144:0/416029942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89a819b750 con 0x7f89a8100260 2024-09-10T11:31:53.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.134+0000 7f89b05f7700 1 -- 172.21.15.144:0/416029942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89a8107c30 con 0x7f89a8100260 2024-09-10T11:31:53.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.135+0000 7f899f7fe700 1 -- 172.21.15.144:0/416029942 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f8998010380 con 0x7f89a8100260 2024-09-10T11:31:53.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.135+0000 7f899f7fe700 1 --2- 172.21.15.144:0/416029942 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f899406de80 0x7f8994070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:53.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.136+0000 7f89ae393700 1 --2- 172.21.15.144:0/416029942 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f899406de80 0x7f8994070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:53.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.136+0000 7f899f7fe700 1 -- 172.21.15.144:0/416029942 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f899808d590 con 0x7f89a8100260 2024-09-10T11:31:53.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.136+0000 7f89ae393700 1 --2- 172.21.15.144:0/416029942 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f899406de80 0x7f8994070330 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f89a4006010 tx=0x7f89a400b540 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:53.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.140+0000 7f899f7fe700 1 -- 172.21.15.144:0/416029942 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f899805aa50 con 0x7f89a8100260 2024-09-10T11:31:53.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.379+0000 7f89b05f7700 1 -- 172.21.15.144:0/416029942 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f89a8104bb0 con 0x7f899406de80 2024-09-10T11:31:53.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.380+0000 7f899f7fe700 1 -- 172.21.15.144:0/416029942 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f89a8104bb0 con 0x7f899406de80 2024-09-10T11:31:53.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.382+0000 7f89b05f7700 1 -- 172.21.15.144:0/416029942 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f899406de80 msgr2=0x7f8994070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:53.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.382+0000 7f89b05f7700 1 --2- 172.21.15.144:0/416029942 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f899406de80 0x7f8994070330 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f89a4006010 tx=0x7f89a400b540 comp rx=0 tx=0).stop 2024-09-10T11:31:53.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.382+0000 7f89b05f7700 1 -- 172.21.15.144:0/416029942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89a8100260 msgr2=0x7f89a819a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:53.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.382+0000 7f89b05f7700 1 --2- 172.21.15.144:0/416029942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89a8100260 0x7f89a819a790 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f899800b4b0 tx=0x7f899800b7c0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.382+0000 7f89b05f7700 1 -- 172.21.15.144:0/416029942 shutdown_connections 2024-09-10T11:31:53.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.382+0000 7f89b05f7700 1 --2- 172.21.15.144:0/416029942 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f899406de80 0x7f8994070330 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.382+0000 7f89b05f7700 1 --2- 172.21.15.144:0/416029942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89a80ff060 0x7f89a80698d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.382+0000 7f89b05f7700 1 --2- 172.21.15.144:0/416029942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89a8100260 0x7f89a819a790 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.382+0000 7f89b05f7700 1 -- 172.21.15.144:0/416029942 >> 172.21.15.144:0/416029942 conn(0x7f89a80fa610 msgr2=0x7f89a8103490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:53.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.383+0000 7f89b05f7700 1 -- 172.21.15.144:0/416029942 shutdown_connections 2024-09-10T11:31:53.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.383+0000 7f89b05f7700 1 -- 172.21.15.144:0/416029942 wait complete. 2024-09-10T11:31:53.394 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:31:53.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.491+0000 7f9d970d0700 1 -- 172.21.15.144:0/1211549859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d90100250 msgr2=0x7f9d901006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:53.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.491+0000 7f9d970d0700 1 --2- 172.21.15.144:0/1211549859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d90100250 0x7f9d901006a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000b0a0 tx=0x7f9d8000b3b0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.491+0000 7f9d970d0700 1 -- 172.21.15.144:0/1211549859 shutdown_connections 2024-09-10T11:31:53.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.491+0000 7f9d970d0700 1 --2- 172.21.15.144:0/1211549859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d90100250 0x7f9d901006a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.491+0000 7f9d970d0700 1 --2- 172.21.15.144:0/1211549859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d900ff050 0x7f9d900ff460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.491+0000 7f9d970d0700 1 -- 172.21.15.144:0/1211549859 >> 172.21.15.144:0/1211549859 conn(0x7f9d900fa5e0 msgr2=0x7f9d900fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.491+0000 7f9d970d0700 1 -- 172.21.15.144:0/1211549859 shutdown_connections 2024-09-10T11:31:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.491+0000 7f9d970d0700 1 -- 172.21.15.144:0/1211549859 wait complete. 2024-09-10T11:31:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.492+0000 7f9d970d0700 1 Processor -- start 2024-09-10T11:31:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.492+0000 7f9d970d0700 1 -- start start 2024-09-10T11:31:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.492+0000 7f9d970d0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d900ff050 0x7f9d90102fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.492+0000 7f9d970d0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d90100250 0x7f9d90101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.492+0000 7f9d970d0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d90101b70 con 0x7f9d900ff050 2024-09-10T11:31:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.492+0000 7f9d970d0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d90101cb0 con 0x7f9d90100250 2024-09-10T11:31:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.492+0000 7f9d94e6c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d900ff050 0x7f9d90102fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.492+0000 7f9d94e6c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d900ff050 0x7f9d90102fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54694/0 (socket says 172.21.15.144:54694) 2024-09-10T11:31:53.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.492+0000 7f9d94e6c700 1 -- 172.21.15.144:0/1738252205 learned_addr learned my addr 172.21.15.144:0/1738252205 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:31:53.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.492+0000 7f9d8ffff700 1 --2- 172.21.15.144:0/1738252205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d90100250 0x7f9d90101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:53.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.492+0000 7f9d94e6c700 1 -- 172.21.15.144:0/1738252205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d90100250 msgr2=0x7f9d90101630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:53.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.492+0000 7f9d94e6c700 1 --2- 172.21.15.144:0/1738252205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d90100250 0x7f9d90101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.492+0000 7f9d94e6c700 1 -- 172.21.15.144:0/1738252205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d80009d00 con 0x7f9d900ff050 2024-09-10T11:31:53.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.492+0000 7f9d94e6c700 1 --2- 172.21.15.144:0/1738252205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d900ff050 0x7f9d90102fe0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800d900 tx=0x7f9d8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:53.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.492+0000 7f9d8dffb700 1 -- 172.21.15.144:0/1738252205 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d880041f0 con 0x7f9d900ff050 2024-09-10T11:31:53.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.493+0000 7f9d8dffb700 1 -- 172.21.15.144:0/1738252205 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9d88004350 con 0x7f9d900ff050 2024-09-10T11:31:53.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.493+0000 7f9d8dffb700 1 -- 172.21.15.144:0/1738252205 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d88003da0 con 0x7f9d900ff050 2024-09-10T11:31:53.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.493+0000 7f9d970d0700 1 -- 172.21.15.144:0/1738252205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d90101f90 con 0x7f9d900ff050 2024-09-10T11:31:53.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.493+0000 7f9d970d0700 1 -- 172.21.15.144:0/1738252205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d9019d9f0 con 0x7f9d900ff050 2024-09-10T11:31:53.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.494+0000 7f9d970d0700 1 -- 172.21.15.144:0/1738252205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d90107ba0 con 0x7f9d900ff050 2024-09-10T11:31:53.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.494+0000 7f9d8dffb700 1 -- 172.21.15.144:0/1738252205 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f9d88009730 con 0x7f9d900ff050 2024-09-10T11:31:53.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.494+0000 7f9d8dffb700 1 --2- 172.21.15.144:0/1738252205 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f9d7c06de80 0x7f9d7c070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:53.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.495+0000 7f9d8dffb700 1 -- 172.21.15.144:0/1738252205 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f9d8808dc40 con 0x7f9d900ff050 2024-09-10T11:31:53.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.495+0000 7f9d8ffff700 1 --2- 172.21.15.144:0/1738252205 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f9d7c06de80 0x7f9d7c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:53.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.495+0000 7f9d8ffff700 1 --2- 172.21.15.144:0/1738252205 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f9d7c06de80 0x7f9d7c070330 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9d80009480 tx=0x7f9d80000f40 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:53.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.501+0000 7f9d8dffb700 1 -- 172.21.15.144:0/1738252205 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f9d8805b080 con 0x7f9d900ff050 2024-09-10T11:31:53.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.729+0000 7f9d970d0700 1 -- 172.21.15.144:0/1738252205 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d90068ae0 con 0x7f9d7c06de80 2024-09-10T11:31:53.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.730+0000 7f9d8dffb700 1 -- 172.21.15.144:0/1738252205 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f9d90068ae0 con 0x7f9d7c06de80 2024-09-10T11:31:53.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.733+0000 7f9d970d0700 1 -- 172.21.15.144:0/1738252205 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f9d7c06de80 msgr2=0x7f9d7c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:53.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.733+0000 7f9d970d0700 1 --2- 172.21.15.144:0/1738252205 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f9d7c06de80 0x7f9d7c070330 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9d80009480 tx=0x7f9d80000f40 comp rx=0 tx=0).stop 2024-09-10T11:31:53.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.733+0000 7f9d970d0700 1 -- 172.21.15.144:0/1738252205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d900ff050 msgr2=0x7f9d90102fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:53.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.733+0000 7f9d970d0700 1 --2- 172.21.15.144:0/1738252205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d900ff050 0x7f9d90102fe0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9d8800d900 tx=0x7f9d8800dcc0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.733+0000 7f9d970d0700 1 -- 172.21.15.144:0/1738252205 shutdown_connections 2024-09-10T11:31:53.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.733+0000 7f9d970d0700 1 --2- 172.21.15.144:0/1738252205 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f9d7c06de80 0x7f9d7c070330 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.733+0000 7f9d970d0700 1 --2- 172.21.15.144:0/1738252205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d900ff050 0x7f9d90102fe0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.733+0000 7f9d970d0700 1 --2- 172.21.15.144:0/1738252205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d90100250 0x7f9d90101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.733+0000 7f9d970d0700 1 -- 172.21.15.144:0/1738252205 >> 172.21.15.144:0/1738252205 conn(0x7f9d900fa5e0 msgr2=0x7f9d900683d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:53.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.733+0000 7f9d970d0700 1 -- 172.21.15.144:0/1738252205 shutdown_connections 2024-09-10T11:31:53.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.733+0000 7f9d970d0700 1 -- 172.21.15.144:0/1738252205 wait complete. 2024-09-10T11:31:53.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.841+0000 7fb3435f9700 1 -- 172.21.15.144:0/3204600572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb33c0ff050 msgr2=0x7fb33c0ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:53.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.841+0000 7fb3435f9700 1 --2- 172.21.15.144:0/3204600572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb33c0ff050 0x7fb33c0ff460 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fb338009910 tx=0x7fb338009c20 comp rx=0 tx=0).stop 2024-09-10T11:31:53.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.841+0000 7fb3435f9700 1 -- 172.21.15.144:0/3204600572 shutdown_connections 2024-09-10T11:31:53.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.841+0000 7fb3435f9700 1 --2- 172.21.15.144:0/3204600572 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb33c100250 0x7fb33c1006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.841+0000 7fb3435f9700 1 --2- 172.21.15.144:0/3204600572 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb33c0ff050 0x7fb33c0ff460 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.841+0000 7fb3435f9700 1 -- 172.21.15.144:0/3204600572 >> 172.21.15.144:0/3204600572 conn(0x7fb33c0fa600 msgr2=0x7fb33c0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:53.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.841+0000 7fb3435f9700 1 -- 172.21.15.144:0/3204600572 shutdown_connections 2024-09-10T11:31:53.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.841+0000 7fb3435f9700 1 -- 172.21.15.144:0/3204600572 wait complete. 2024-09-10T11:31:53.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.842+0000 7fb3435f9700 1 Processor -- start 2024-09-10T11:31:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.842+0000 7fb3435f9700 1 -- start start 2024-09-10T11:31:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.842+0000 7fb3435f9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb33c0ff050 0x7fb33c0697c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.842+0000 7fb3435f9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb33c100250 0x7fb33c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.842+0000 7fb3435f9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c19aca0 con 0x7fb33c100250 2024-09-10T11:31:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.842+0000 7fb3435f9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb33c19ade0 con 0x7fb33c0ff050 2024-09-10T11:31:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.842+0000 7fb340b94700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb33c100250 0x7fb33c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.842+0000 7fb340b94700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb33c100250 0x7fb33c19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54726/0 (socket says 172.21.15.144:54726) 2024-09-10T11:31:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.842+0000 7fb340b94700 1 -- 172.21.15.144:0/3056321328 learned_addr learned my addr 172.21.15.144:0/3056321328 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:31:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.842+0000 7fb340b94700 1 -- 172.21.15.144:0/3056321328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb33c0ff050 msgr2=0x7fb33c0697c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.842+0000 7fb340b94700 1 --2- 172.21.15.144:0/3056321328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb33c0ff050 0x7fb33c0697c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.842+0000 7fb340b94700 1 -- 172.21.15.144:0/3056321328 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3380095f0 con 0x7fb33c100250 2024-09-10T11:31:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.842+0000 7fb340b94700 1 --2- 172.21.15.144:0/3056321328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb33c100250 0x7fb33c19a6d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00d8d0 tx=0x7fb32c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:53.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.843+0000 7fb3327fc700 1 -- 172.21.15.144:0/3056321328 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb32c009940 con 0x7fb33c100250 2024-09-10T11:31:53.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.843+0000 7fb3327fc700 1 -- 172.21.15.144:0/3056321328 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb32c010460 con 0x7fb33c100250 2024-09-10T11:31:53.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.843+0000 7fb3435f9700 1 -- 172.21.15.144:0/3056321328 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb33c19b0c0 con 0x7fb33c100250 2024-09-10T11:31:53.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.843+0000 7fb3327fc700 1 -- 172.21.15.144:0/3056321328 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb32c00f5d0 con 0x7fb33c100250 2024-09-10T11:31:53.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.843+0000 7fb3435f9700 1 -- 172.21.15.144:0/3056321328 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb33c19b610 con 0x7fb33c100250 2024-09-10T11:31:53.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.844+0000 7fb3435f9700 1 -- 172.21.15.144:0/3056321328 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb33c107ba0 con 0x7fb33c100250 2024-09-10T11:31:53.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.844+0000 7fb3327fc700 1 -- 172.21.15.144:0/3056321328 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7fb32c0105d0 con 0x7fb33c100250 2024-09-10T11:31:53.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.845+0000 7fb3327fc700 1 --2- 172.21.15.144:0/3056321328 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fb32806de80 0x7fb328070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:53.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.845+0000 7fb3327fc700 1 -- 172.21.15.144:0/3056321328 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fb32c08da30 con 0x7fb33c100250 2024-09-10T11:31:53.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.845+0000 7fb341395700 1 --2- 172.21.15.144:0/3056321328 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fb32806de80 0x7fb328070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:53.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.846+0000 7fb341395700 1 --2- 172.21.15.144:0/3056321328 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fb32806de80 0x7fb328070330 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb33800b5c0 tx=0x7fb338005960 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:53.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:53.850+0000 7fb3327fc700 1 -- 172.21.15.144:0/3056321328 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7fb32c05ae50 con 0x7fb33c100250 2024-09-10T11:31:54.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.072+0000 7fb3435f9700 1 -- 172.21.15.144:0/3056321328 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb33c104ba0 con 0x7fb32806de80 2024-09-10T11:31:54.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.084+0000 7fb3327fc700 1 -- 172.21.15.144:0/3056321328 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fb33c104ba0 con 0x7fb32806de80 2024-09-10T11:31:54.086 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:31:54.086 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (8s) 2s ago 10m 15.9M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:31:54.086 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (10m) 2s ago 10m 7964k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:31:54.086 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (9m) 38s ago 9m 7830k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:31:54.086 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (10m) 2s ago 10m 6669k - 18.2.0 dc2bc1663786 3311fb04aa0c 2024-09-10T11:31:54.086 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (9m) 38s ago 9m 6651k - 18.2.0 dc2bc1663786 6e408e8476d4 2024-09-10T11:31:54.086 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (9m) 2s ago 10m 86.0M - 9.4.7 954c08fa6188 e9a1aec955f6 2024-09-10T11:31:54.086 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (5m) 2s ago 5m 59.9M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:31:54.086 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (5m) 2s ago 5m 127M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:31:54.086 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (5m) 38s ago 5m 65.7M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:31:54.087 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (5m) 38s ago 5m 127M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:31:54.087 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (111s) 2s ago 12m 498M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:31:54.087 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (62s) 38s ago 9m 441M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:31:54.087 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (12m) 2s ago 12m 56.0M 2048M 18.2.0 dc2bc1663786 42addb654e4f 2024-09-10T11:31:54.087 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (9m) 38s ago 9m 43.5M 2048M 18.2.0 dc2bc1663786 3c1ba210ed3f 2024-09-10T11:31:54.087 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (45s) 2s ago 10m 15.7M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:31:54.087 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (41s) 38s ago 9m 3586k - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:31:54.087 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (8m) 2s ago 8m 204M 1216M 18.2.0 dc2bc1663786 2435febebdb5 2024-09-10T11:31:54.087 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (8m) 2s ago 8m 211M 1216M 18.2.0 dc2bc1663786 b55fe920ff9d 2024-09-10T11:31:54.087 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (7m) 2s ago 7m 168M 1216M 18.2.0 dc2bc1663786 3ed5618eac33 2024-09-10T11:31:54.087 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (7m) 38s ago 7m 252M 2239M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:31:54.087 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (7m) 38s ago 7m 217M 2239M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:31:54.088 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (6m) 38s ago 6m 195M 2239M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:31:54.088 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (24s) 2s ago 10m 60.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:31:54.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.086+0000 7fb3435f9700 1 -- 172.21.15.144:0/3056321328 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fb32806de80 msgr2=0x7fb328070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.086+0000 7fb3435f9700 1 --2- 172.21.15.144:0/3056321328 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fb32806de80 0x7fb328070330 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb33800b5c0 tx=0x7fb338005960 comp rx=0 tx=0).stop 2024-09-10T11:31:54.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.086+0000 7fb3435f9700 1 -- 172.21.15.144:0/3056321328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb33c100250 msgr2=0x7fb33c19a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.086+0000 7fb3435f9700 1 --2- 172.21.15.144:0/3056321328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb33c100250 0x7fb33c19a6d0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00d8d0 tx=0x7fb32c00dc90 comp rx=0 tx=0).stop 2024-09-10T11:31:54.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.086+0000 7fb3435f9700 1 -- 172.21.15.144:0/3056321328 shutdown_connections 2024-09-10T11:31:54.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.086+0000 7fb3435f9700 1 --2- 172.21.15.144:0/3056321328 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fb32806de80 0x7fb328070330 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.086+0000 7fb3435f9700 1 --2- 172.21.15.144:0/3056321328 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb33c0ff050 0x7fb33c0697c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.086+0000 7fb3435f9700 1 --2- 172.21.15.144:0/3056321328 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb33c100250 0x7fb33c19a6d0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.086+0000 7fb3435f9700 1 -- 172.21.15.144:0/3056321328 >> 172.21.15.144:0/3056321328 conn(0x7fb33c0fa600 msgr2=0x7fb33c103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:54.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.086+0000 7fb3435f9700 1 -- 172.21.15.144:0/3056321328 shutdown_connections 2024-09-10T11:31:54.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.086+0000 7fb3435f9700 1 -- 172.21.15.144:0/3056321328 wait complete. 2024-09-10T11:31:54.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.196+0000 7feaf3d2f700 1 -- 172.21.15.144:0/3157861679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaec100620 msgr2=0x7feaec100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.196+0000 7feaf3d2f700 1 --2- 172.21.15.144:0/3157861679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaec100620 0x7feaec100a30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7feae8009910 tx=0x7feae8009c20 comp rx=0 tx=0).stop 2024-09-10T11:31:54.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.196+0000 7feaf3d2f700 1 -- 172.21.15.144:0/3157861679 shutdown_connections 2024-09-10T11:31:54.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.196+0000 7feaf3d2f700 1 --2- 172.21.15.144:0/3157861679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaec0ff280 0x7feaec0ff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.196+0000 7feaf3d2f700 1 --2- 172.21.15.144:0/3157861679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaec100620 0x7feaec100a30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.196+0000 7feaf3d2f700 1 -- 172.21.15.144:0/3157861679 >> 172.21.15.144:0/3157861679 conn(0x7feaec0f9d80 msgr2=0x7feaec0fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:54.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.196+0000 7feaf3d2f700 1 -- 172.21.15.144:0/3157861679 shutdown_connections 2024-09-10T11:31:54.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.196+0000 7feaf3d2f700 1 -- 172.21.15.144:0/3157861679 wait complete. 2024-09-10T11:31:54.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.197+0000 7feaf3d2f700 1 Processor -- start 2024-09-10T11:31:54.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.197+0000 7feaf3d2f700 1 -- start start 2024-09-10T11:31:54.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.197+0000 7feaf3d2f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaec0ff280 0x7feaec197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:54.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.197+0000 7feaf3d2f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaec100620 0x7feaec1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:54.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.197+0000 7feaf3d2f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec198710 con 0x7feaec100620 2024-09-10T11:31:54.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.197+0000 7feaf3d2f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaec198850 con 0x7feaec0ff280 2024-09-10T11:31:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.197+0000 7feaf12ca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaec100620 0x7feaec1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.197+0000 7feaf12ca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaec100620 0x7feaec1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54752/0 (socket says 172.21.15.144:54752) 2024-09-10T11:31:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.197+0000 7feaf12ca700 1 -- 172.21.15.144:0/303498771 learned_addr learned my addr 172.21.15.144:0/303498771 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:31:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.197+0000 7feaf1acb700 1 --2- 172.21.15.144:0/303498771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaec0ff280 0x7feaec197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.197+0000 7feaf12ca700 1 -- 172.21.15.144:0/303498771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaec0ff280 msgr2=0x7feaec197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.197+0000 7feaf12ca700 1 --2- 172.21.15.144:0/303498771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaec0ff280 0x7feaec197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.197+0000 7feaf12ca700 1 -- 172.21.15.144:0/303498771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae80095f0 con 0x7feaec100620 2024-09-10T11:31:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.197+0000 7feaf1acb700 1 --2- 172.21.15.144:0/303498771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaec0ff280 0x7feaec197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:31:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.198+0000 7feaf12ca700 1 --2- 172.21.15.144:0/303498771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaec100620 0x7feaec1980f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feadc00d900 tx=0x7feadc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.198+0000 7feae2ffd700 1 -- 172.21.15.144:0/303498771 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feadc0041f0 con 0x7feaec100620 2024-09-10T11:31:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.198+0000 7feae2ffd700 1 -- 172.21.15.144:0/303498771 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7feadc004350 con 0x7feaec100620 2024-09-10T11:31:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.198+0000 7feaf3d2f700 1 -- 172.21.15.144:0/303498771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaec19d300 con 0x7feaec100620 2024-09-10T11:31:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.198+0000 7feae2ffd700 1 -- 172.21.15.144:0/303498771 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feadc003da0 con 0x7feaec100620 2024-09-10T11:31:54.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.198+0000 7feaf3d2f700 1 -- 172.21.15.144:0/303498771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaec19d850 con 0x7feaec100620 2024-09-10T11:31:54.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.199+0000 7feaf3d2f700 1 -- 172.21.15.144:0/303498771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaec107bc0 con 0x7feaec100620 2024-09-10T11:31:54.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.200+0000 7feae2ffd700 1 -- 172.21.15.144:0/303498771 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7feadc009730 con 0x7feaec100620 2024-09-10T11:31:54.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.200+0000 7feae2ffd700 1 --2- 172.21.15.144:0/303498771 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fead806de80 0x7fead8070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:54.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.200+0000 7feae2ffd700 1 -- 172.21.15.144:0/303498771 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7feadc08dbf0 con 0x7feaec100620 2024-09-10T11:31:54.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.200+0000 7feaf1acb700 1 --2- 172.21.15.144:0/303498771 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fead806de80 0x7fead8070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:54.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.201+0000 7feaf1acb700 1 --2- 172.21.15.144:0/303498771 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fead806de80 0x7fead8070330 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feae800b5c0 tx=0x7feae8005e20 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:54.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.205+0000 7feae2ffd700 1 -- 172.21.15.144:0/303498771 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7feadc05b010 con 0x7feaec100620 2024-09-10T11:31:54.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.497+0000 7feaf3d2f700 1 -- 172.21.15.144:0/303498771 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feaec107db0 con 0x7feaec100620 2024-09-10T11:31:54.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.498+0000 7feae2ffd700 1 -- 172.21.15.144:0/303498771 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7feadc05aba0 con 0x7feaec100620 2024-09-10T11:31:54.499 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:31:54.499 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:31:54.499 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-09-10T11:31:54.499 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:31:54.500 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:31:54.500 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:31:54.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:31:54.500 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:31:54.500 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-09-10T11:31:54.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:31:54.500 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:31:54.500 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:31:54.500 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:31:54.500 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:31:54.500 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-09-10T11:31:54.500 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:31:54.500 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:31:54.500 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:31:54.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.500+0000 7feaf3d2f700 1 -- 172.21.15.144:0/303498771 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fead806de80 msgr2=0x7fead8070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.500+0000 7feaf3d2f700 1 --2- 172.21.15.144:0/303498771 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fead806de80 0x7fead8070330 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7feae800b5c0 tx=0x7feae8005e20 comp rx=0 tx=0).stop 2024-09-10T11:31:54.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.500+0000 7feaf3d2f700 1 -- 172.21.15.144:0/303498771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaec100620 msgr2=0x7feaec1980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.500+0000 7feaf3d2f700 1 --2- 172.21.15.144:0/303498771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaec100620 0x7feaec1980f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7feadc00d900 tx=0x7feadc00dcc0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.500+0000 7feaf3d2f700 1 -- 172.21.15.144:0/303498771 shutdown_connections 2024-09-10T11:31:54.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.500+0000 7feaf3d2f700 1 --2- 172.21.15.144:0/303498771 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fead806de80 0x7fead8070330 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.500+0000 7feaf3d2f700 1 --2- 172.21.15.144:0/303498771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feaec0ff280 0x7feaec197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.500+0000 7feaf3d2f700 1 --2- 172.21.15.144:0/303498771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feaec100620 0x7feaec1980f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.500+0000 7feaf3d2f700 1 -- 172.21.15.144:0/303498771 >> 172.21.15.144:0/303498771 conn(0x7feaec0f9d80 msgr2=0x7feaec0fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:54.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.500+0000 7feaf3d2f700 1 -- 172.21.15.144:0/303498771 shutdown_connections 2024-09-10T11:31:54.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.500+0000 7feaf3d2f700 1 -- 172.21.15.144:0/303498771 wait complete. 2024-09-10T11:31:54.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.606+0000 7ff12869b700 1 -- 172.21.15.144:0/3485279544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff120101370 msgr2=0x7ff1201017c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.606+0000 7ff12869b700 1 --2- 172.21.15.144:0/3485279544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff120101370 0x7ff1201017c0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7ff110009910 tx=0x7ff110009c20 comp rx=0 tx=0).stop 2024-09-10T11:31:54.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.606+0000 7ff12869b700 1 -- 172.21.15.144:0/3485279544 shutdown_connections 2024-09-10T11:31:54.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.606+0000 7ff12869b700 1 --2- 172.21.15.144:0/3485279544 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff120101370 0x7ff1201017c0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.606+0000 7ff12869b700 1 --2- 172.21.15.144:0/3485279544 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff120100170 0x7ff120100580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.606+0000 7ff12869b700 1 -- 172.21.15.144:0/3485279544 >> 172.21.15.144:0/3485279544 conn(0x7ff1200fb6e0 msgr2=0x7ff1200fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:54.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.606+0000 7ff12869b700 1 -- 172.21.15.144:0/3485279544 shutdown_connections 2024-09-10T11:31:54.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.606+0000 7ff12869b700 1 -- 172.21.15.144:0/3485279544 wait complete. 2024-09-10T11:31:54.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.607+0000 7ff12869b700 1 Processor -- start 2024-09-10T11:31:54.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.607+0000 7ff12869b700 1 -- start start 2024-09-10T11:31:54.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.607+0000 7ff12869b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff120100170 0x7ff120195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:54.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.607+0000 7ff12869b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff120101370 0x7ff120195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:54.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.607+0000 7ff12869b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff120196590 con 0x7ff120101370 2024-09-10T11:31:54.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.607+0000 7ff12869b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1201966d0 con 0x7ff120100170 2024-09-10T11:31:54.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.607+0000 7ff125c36700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff120101370 0x7ff120195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:54.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.607+0000 7ff125c36700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff120101370 0x7ff120195f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54772/0 (socket says 172.21.15.144:54772) 2024-09-10T11:31:54.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.607+0000 7ff125c36700 1 -- 172.21.15.144:0/965935260 learned_addr learned my addr 172.21.15.144:0/965935260 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:31:54.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.608+0000 7ff126437700 1 --2- 172.21.15.144:0/965935260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff120100170 0x7ff120195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:54.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.608+0000 7ff125c36700 1 -- 172.21.15.144:0/965935260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff120100170 msgr2=0x7ff120195a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.608+0000 7ff125c36700 1 --2- 172.21.15.144:0/965935260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff120100170 0x7ff120195a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.608+0000 7ff125c36700 1 -- 172.21.15.144:0/965935260 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1100095f0 con 0x7ff120101370 2024-09-10T11:31:54.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.608+0000 7ff125c36700 1 --2- 172.21.15.144:0/965935260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff120101370 0x7ff120195f70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff110006010 tx=0x7ff11000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:54.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.608+0000 7ff1177fe700 1 -- 172.21.15.144:0/965935260 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff11001d070 con 0x7ff120101370 2024-09-10T11:31:54.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.608+0000 7ff1177fe700 1 -- 172.21.15.144:0/965935260 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff11000f460 con 0x7ff120101370 2024-09-10T11:31:54.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.608+0000 7ff12869b700 1 -- 172.21.15.144:0/965935260 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff12019b120 con 0x7ff120101370 2024-09-10T11:31:54.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.608+0000 7ff1177fe700 1 -- 172.21.15.144:0/965935260 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff110021640 con 0x7ff120101370 2024-09-10T11:31:54.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.608+0000 7ff12869b700 1 -- 172.21.15.144:0/965935260 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff12019b690 con 0x7ff120101370 2024-09-10T11:31:54.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.609+0000 7ff12869b700 1 -- 172.21.15.144:0/965935260 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff120108d60 con 0x7ff120101370 2024-09-10T11:31:54.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.610+0000 7ff1177fe700 1 -- 172.21.15.144:0/965935260 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7ff11000fab0 con 0x7ff120101370 2024-09-10T11:31:54.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.610+0000 7ff1177fe700 1 --2- 172.21.15.144:0/965935260 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7ff10c06de80 0x7ff10c070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:31:54.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.610+0000 7ff1177fe700 1 -- 172.21.15.144:0/965935260 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7ff11008ef60 con 0x7ff120101370 2024-09-10T11:31:54.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.610+0000 7ff126437700 1 --2- 172.21.15.144:0/965935260 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7ff10c06de80 0x7ff10c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:31:54.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.611+0000 7ff126437700 1 --2- 172.21.15.144:0/965935260 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7ff10c06de80 0x7ff10c070330 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff11c006fd0 tx=0x7ff11c008040 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:31:54.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.615+0000 7ff1177fe700 1 -- 172.21.15.144:0/965935260 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7ff1100c39d0 con 0x7ff120101370 2024-09-10T11:31:54.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.843+0000 7ff12869b700 1 -- 172.21.15.144:0/965935260 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff120105cc0 con 0x7ff10c06de80 2024-09-10T11:31:54.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.845+0000 7ff1177fe700 1 -- 172.21.15.144:0/965935260 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7ff120105cc0 con 0x7ff10c06de80 2024-09-10T11:31:54.846 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:31:54.846 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:31:54.846 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:31:54.846 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-09-10T11:31:54.846 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:31:54.846 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr" 2024-09-10T11:31:54.846 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:31:54.846 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "2/2 daemons upgraded", 2024-09-10T11:31:54.847 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading alertmanager daemons", 2024-09-10T11:31:54.847 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:31:54.847 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:31:54.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.847+0000 7ff12869b700 1 -- 172.21.15.144:0/965935260 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7ff10c06de80 msgr2=0x7ff10c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.847+0000 7ff12869b700 1 --2- 172.21.15.144:0/965935260 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7ff10c06de80 0x7ff10c070330 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff11c006fd0 tx=0x7ff11c008040 comp rx=0 tx=0).stop 2024-09-10T11:31:54.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.847+0000 7ff12869b700 1 -- 172.21.15.144:0/965935260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff120101370 msgr2=0x7ff120195f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:31:54.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.847+0000 7ff12869b700 1 --2- 172.21.15.144:0/965935260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff120101370 0x7ff120195f70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff110006010 tx=0x7ff11000be20 comp rx=0 tx=0).stop 2024-09-10T11:31:54.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.847+0000 7ff12869b700 1 -- 172.21.15.144:0/965935260 shutdown_connections 2024-09-10T11:31:54.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.847+0000 7ff12869b700 1 --2- 172.21.15.144:0/965935260 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7ff10c06de80 0x7ff10c070330 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.847+0000 7ff12869b700 1 --2- 172.21.15.144:0/965935260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff120100170 0x7ff120195a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.847+0000 7ff12869b700 1 --2- 172.21.15.144:0/965935260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff120101370 0x7ff120195f70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:31:54.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.847+0000 7ff12869b700 1 -- 172.21.15.144:0/965935260 >> 172.21.15.144:0/965935260 conn(0x7ff1200fb6e0 msgr2=0x7ff1201045a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:31:54.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.847+0000 7ff12869b700 1 -- 172.21.15.144:0/965935260 shutdown_connections 2024-09-10T11:31:54.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:31:54.847+0000 7ff12869b700 1 -- 172.21.15.144:0/965935260 wait complete. 2024-09-10T11:31:55.080 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:54 smithi144 ceph-mon[27942]: pgmap v53: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T11:31:55.080 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:54 smithi144 ceph-mon[27942]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:55.080 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:54 smithi144 ceph-mon[27942]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:55.080 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:54 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/303498771' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:54 smithi178 ceph-mon[33514]: pgmap v53: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T11:31:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:54 smithi178 ceph-mon[33514]: from='client.14752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:54 smithi178 ceph-mon[33514]: from='client.14756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:54 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/303498771' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.110 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:56.110 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:56.110 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:56.110 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:56.110 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:31:56.110 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:31:56.111 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:56.111 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T11:31:56.111 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:31:56.111 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.111 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.111 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.111 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.111 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.111 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.111 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.111 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.111 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.111 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:55 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.283 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:56.284 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='client.14768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:31:56.284 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:56.284 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:56.284 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:31:56.284 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:31:56.284 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:31:56.284 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T11:31:56.284 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:31:56.284 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.284 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.284 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.285 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.285 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.285 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.285 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.285 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.285 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:56.285 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:55 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:31:57.232 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:56 smithi178 ceph-mon[33514]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T11:31:57.232 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:56 smithi178 ceph-mon[33514]: pgmap v54: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T11:31:57.355 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:56 smithi144 ceph-mon[27942]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T11:31:57.355 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:56 smithi144 ceph-mon[27942]: pgmap v54: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-09-10T11:31:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:57 smithi178 ceph-mon[33514]: Upgrade: Updating grafana.smithi144 2024-09-10T11:31:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:57 smithi178 ceph-mon[33514]: Deploying daemon grafana.smithi144 on smithi144 2024-09-10T11:31:58.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:57 smithi144 ceph-mon[27942]: Upgrade: Updating grafana.smithi144 2024-09-10T11:31:58.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:57 smithi144 ceph-mon[27942]: Deploying daemon grafana.smithi144 on smithi144 2024-09-10T11:31:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:58 smithi178 ceph-mon[33514]: pgmap v55: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:31:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:58 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:31:59.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:58 smithi144 ceph-mon[27942]: pgmap v55: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:31:59.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:58 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:32:00.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:31:59 smithi178 ceph-mon[33514]: pgmap v56: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-10T11:32:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:31:59 smithi144 ceph-mon[27942]: pgmap v56: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.6 KiB/s rd, 5 op/s 2024-09-10T11:32:02.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:02 smithi178 ceph-mon[33514]: pgmap v57: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:02.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:02 smithi144 ceph-mon[27942]: pgmap v57: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:04.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:04 smithi144 ceph-mon[27942]: pgmap v58: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:04.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:04 smithi178 ceph-mon[33514]: pgmap v58: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:06.539 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:06 smithi144 ceph-mon[27942]: pgmap v59: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:06.745 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:06 smithi178 ceph-mon[33514]: pgmap v59: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:08.747 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:08 smithi178 ceph-mon[33514]: pgmap v60: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:08.748 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:08 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:08.748 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:08 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:08.748 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:08 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:32:08.799 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:08 smithi144 ceph-mon[27942]: pgmap v60: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:08.799 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:08 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:08.799 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:08 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:08.799 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:08 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:32:09.783 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:09 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:09.783 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:09 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:09.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:09 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:09.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:09 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:10.544 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:10 smithi144 ceph-mon[27942]: pgmap v61: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:10.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:10 smithi178 ceph-mon[33514]: pgmap v61: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:12.748 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:12 smithi144 ceph-mon[27942]: pgmap v62: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:12 smithi178 ceph-mon[33514]: pgmap v62: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:14 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:14 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:14 smithi144 ceph-mon[27942]: pgmap v63: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:14 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:14 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:14 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:32:14.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:14 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:14.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:14 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:14.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:14 smithi178 ceph-mon[33514]: pgmap v63: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:14.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:14 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:14.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:14 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:14.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:14 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:32:16.606 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:16 smithi144 ceph-mon[27942]: pgmap v64: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:16.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:16 smithi178 ceph-mon[33514]: pgmap v64: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:18.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:18.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:18.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:32:18.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:32:18.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:18.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T11:32:18.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:32:18.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:18.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-10T11:32:18.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-10T11:32:18.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-10T11:32:18.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-10T11:32:18.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-10T11:32:18.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-10T11:32:18.371 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-10T11:32:18.371 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-10T11:32:18.371 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-10T11:32:18.371 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-10T11:32:18.371 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-10T11:32:18.371 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-10T11:32:18.371 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-10T11:32:18.371 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-10T11:32:18.371 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-10T11:32:18.371 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-10T11:32:18.371 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.371 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.371 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-10T11:32:18.372 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-10T11:32:18.372 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-10T11:32:18.372 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-10T11:32:18.372 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-10T11:32:18.372 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-10T11:32:18.372 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-10T11:32:18.372 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-10T11:32:18.372 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-10T11:32:18.372 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-10T11:32:18.372 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-10T11:32:18.372 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.373 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.373 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.373 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.373 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.373 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.373 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.373 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.373 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.373 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.373 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.373 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.373 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-10T11:32:18.373 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-10T11:32:18.373 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-10T11:32:18.374 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:32:18.374 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:32:18.374 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:32:18.374 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:18.374 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:32:18.374 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:32:18.374 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:32:18.374 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:17 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:18.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:18.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:18.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:32:18.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:32:18.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:18.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T11:32:18.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:32:18.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:18.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:18.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-10T11:32:18.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-10T11:32:18.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-10T11:32:18.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-10T11:32:18.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-10T11:32:18.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-10T11:32:18.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-10T11:32:18.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-10T11:32:18.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-10T11:32:18.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-10T11:32:18.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-10T11:32:18.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-10T11:32:18.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-10T11:32:18.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-10T11:32:18.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-10T11:32:18.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-10T11:32:18.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-10T11:32:18.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-10T11:32:18.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-10T11:32:18.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-10T11:32:18.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-10T11:32:18.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-10T11:32:18.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-10T11:32:18.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-10T11:32:18.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-10T11:32:18.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-10T11:32:18.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-10T11:32:18.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T11:32:18.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-10T11:32:18.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-10T11:32:18.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-10T11:32:18.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:32:18.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:32:18.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:32:18.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:18.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:32:18.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:32:18.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:32:18.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:17 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:18 smithi178 ceph-mon[33514]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T11:32:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:18 smithi178 ceph-mon[33514]: Upgrade: Finalizing container_image settings 2024-09-10T11:32:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:18 smithi178 ceph-mon[33514]: Upgrade: Complete! 2024-09-10T11:32:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:18 smithi178 ceph-mon[33514]: pgmap v65: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:18 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:19.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:18 smithi144 ceph-mon[27942]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T11:32:19.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:18 smithi144 ceph-mon[27942]: Upgrade: Finalizing container_image settings 2024-09-10T11:32:19.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:18 smithi144 ceph-mon[27942]: Upgrade: Complete! 2024-09-10T11:32:19.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:18 smithi144 ceph-mon[27942]: pgmap v65: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:19.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:18 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:20 smithi178 ceph-mon[33514]: pgmap v66: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:21.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:20 smithi144 ceph-mon[27942]: pgmap v66: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:23.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:22 smithi178 ceph-mon[33514]: pgmap v67: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:23.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:22 smithi144 ceph-mon[27942]: pgmap v67: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:24.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.967+0000 7f81639f1700 1 -- 172.21.15.144:0/3807783387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f815c074d80 msgr2=0x7f815c0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:24.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.967+0000 7f81639f1700 1 --2- 172.21.15.144:0/3807783387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f815c074d80 0x7f815c0731e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8158009910 tx=0x7f8158009c20 comp rx=0 tx=0).stop 2024-09-10T11:32:24.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.967+0000 7f81639f1700 1 -- 172.21.15.144:0/3807783387 shutdown_connections 2024-09-10T11:32:24.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.967+0000 7f81639f1700 1 --2- 172.21.15.144:0/3807783387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c0737b0 0x7f815c073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:24.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.967+0000 7f81639f1700 1 --2- 172.21.15.144:0/3807783387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f815c074d80 0x7f815c0731e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:24.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.967+0000 7f81639f1700 1 -- 172.21.15.144:0/3807783387 >> 172.21.15.144:0/3807783387 conn(0x7f815c0fb6e0 msgr2=0x7f815c0fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.967+0000 7f81639f1700 1 -- 172.21.15.144:0/3807783387 shutdown_connections 2024-09-10T11:32:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.967+0000 7f81639f1700 1 -- 172.21.15.144:0/3807783387 wait complete. 2024-09-10T11:32:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.967+0000 7f81639f1700 1 Processor -- start 2024-09-10T11:32:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.967+0000 7f81639f1700 1 -- start start 2024-09-10T11:32:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.968+0000 7f81639f1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c0737b0 0x7f815c19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.968+0000 7f81639f1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f815c074d80 0x7f815c19c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.968+0000 7f81639f1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c19cac0 con 0x7f815c074d80 2024-09-10T11:32:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.968+0000 7f81639f1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815c19cc00 con 0x7f815c0737b0 2024-09-10T11:32:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.968+0000 7f8160f8c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f815c074d80 0x7f815c19c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.968+0000 7f8160f8c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f815c074d80 0x7f815c19c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53594/0 (socket says 172.21.15.144:53594) 2024-09-10T11:32:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.968+0000 7f8160f8c700 1 -- 172.21.15.144:0/1966592580 learned_addr learned my addr 172.21.15.144:0/1966592580 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:32:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.968+0000 7f816178d700 1 --2- 172.21.15.144:0/1966592580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c0737b0 0x7f815c19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.968+0000 7f8160f8c700 1 -- 172.21.15.144:0/1966592580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c0737b0 msgr2=0x7f815c19bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.968+0000 7f8160f8c700 1 --2- 172.21.15.144:0/1966592580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c0737b0 0x7f815c19bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.968+0000 7f8160f8c700 1 -- 172.21.15.144:0/1966592580 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81580095f0 con 0x7f815c074d80 2024-09-10T11:32:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.968+0000 7f816178d700 1 --2- 172.21.15.144:0/1966592580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c0737b0 0x7f815c19bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.968+0000 7f8160f8c700 1 --2- 172.21.15.144:0/1966592580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f815c074d80 0x7f815c19c4a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f814c00d900 tx=0x7f814c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.968+0000 7f81527fc700 1 -- 172.21.15.144:0/1966592580 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f814c0041f0 con 0x7f815c074d80 2024-09-10T11:32:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.969+0000 7f81527fc700 1 -- 172.21.15.144:0/1966592580 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f814c004cf0 con 0x7f815c074d80 2024-09-10T11:32:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.969+0000 7f81639f1700 1 -- 172.21.15.144:0/1966592580 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815c1a16b0 con 0x7f815c074d80 2024-09-10T11:32:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.969+0000 7f81527fc700 1 -- 172.21.15.144:0/1966592580 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f814c00b750 con 0x7f815c074d80 2024-09-10T11:32:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.969+0000 7f81639f1700 1 -- 172.21.15.144:0/1966592580 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f815c1a1c00 con 0x7f815c074d80 2024-09-10T11:32:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.969+0000 7f81639f1700 1 -- 172.21.15.144:0/1966592580 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f815c10f2a0 con 0x7f815c074d80 2024-09-10T11:32:24.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.970+0000 7f81527fc700 1 -- 172.21.15.144:0/1966592580 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f814c004350 con 0x7f815c074d80 2024-09-10T11:32:24.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.971+0000 7f81527fc700 1 --2- 172.21.15.144:0/1966592580 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f814806de80 0x7f8148070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:24.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.971+0000 7f81527fc700 1 -- 172.21.15.144:0/1966592580 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f814c08daa0 con 0x7f815c074d80 2024-09-10T11:32:24.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.971+0000 7f816178d700 1 --2- 172.21.15.144:0/1966592580 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f814806de80 0x7f8148070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:24.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.971+0000 7f816178d700 1 --2- 172.21.15.144:0/1966592580 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f814806de80 0x7f8148070330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8158005b40 tx=0x7f8158005ab0 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:24.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:24.975+0000 7f81527fc700 1 -- 172.21.15.144:0/1966592580 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f814c05aee0 con 0x7f815c074d80 2024-09-10T11:32:25.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:25.196+0000 7f81639f1700 1 -- 172.21.15.144:0/1966592580 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f815c103220 con 0x7f814806de80 2024-09-10T11:32:25.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:25.197+0000 7f81527fc700 1 -- 172.21.15.144:0/1966592580 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f815c103220 con 0x7f814806de80 2024-09-10T11:32:25.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:25.199+0000 7f81639f1700 1 -- 172.21.15.144:0/1966592580 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f814806de80 msgr2=0x7f8148070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:25.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:25.199+0000 7f81639f1700 1 --2- 172.21.15.144:0/1966592580 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f814806de80 0x7f8148070330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8158005b40 tx=0x7f8158005ab0 comp rx=0 tx=0).stop 2024-09-10T11:32:25.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:25.199+0000 7f81639f1700 1 -- 172.21.15.144:0/1966592580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f815c074d80 msgr2=0x7f815c19c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:25.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:25.199+0000 7f81639f1700 1 --2- 172.21.15.144:0/1966592580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f815c074d80 0x7f815c19c4a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f814c00d900 tx=0x7f814c00dcc0 comp rx=0 tx=0).stop 2024-09-10T11:32:25.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:25.199+0000 7f81639f1700 1 -- 172.21.15.144:0/1966592580 shutdown_connections 2024-09-10T11:32:25.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:25.199+0000 7f81639f1700 1 --2- 172.21.15.144:0/1966592580 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f814806de80 0x7f8148070330 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:25.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:25.199+0000 7f81639f1700 1 --2- 172.21.15.144:0/1966592580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f815c0737b0 0x7f815c19bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:25.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:25.199+0000 7f81639f1700 1 --2- 172.21.15.144:0/1966592580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f815c074d80 0x7f815c19c4a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:25.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:25.199+0000 7f81639f1700 1 -- 172.21.15.144:0/1966592580 >> 172.21.15.144:0/1966592580 conn(0x7f815c0fb6e0 msgr2=0x7f815c101b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:25.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:25.200+0000 7f81639f1700 1 -- 172.21.15.144:0/1966592580 shutdown_connections 2024-09-10T11:32:25.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:25.200+0000 7f81639f1700 1 -- 172.21.15.144:0/1966592580 wait complete. 2024-09-10T11:32:25.212 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:24 smithi144 ceph-mon[27942]: pgmap v68: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:24 smithi178 ceph-mon[33514]: pgmap v68: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:25.925 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-09-10T11:32:26.172 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:32:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:26 smithi144 ceph-mon[27942]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:32:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:26 smithi144 ceph-mon[27942]: pgmap v69: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:26 smithi178 ceph-mon[33514]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:32:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:26 smithi178 ceph-mon[33514]: pgmap v69: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:27.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.789+0000 7fb230358700 1 -- 172.21.15.144:0/1195387585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb228100d70 msgr2=0x7fb228104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:27.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.789+0000 7fb230358700 1 --2- 172.21.15.144:0/1195387585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb228100d70 0x7fb228104d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb218009910 tx=0x7fb218009c20 comp rx=0 tx=0).stop 2024-09-10T11:32:27.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.789+0000 7fb230358700 1 -- 172.21.15.144:0/1195387585 shutdown_connections 2024-09-10T11:32:27.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.789+0000 7fb230358700 1 --2- 172.21.15.144:0/1195387585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb228100d70 0x7fb228104d40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:27.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.789+0000 7fb230358700 1 --2- 172.21.15.144:0/1195387585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2281003d0 0x7fb2281007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:27.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.789+0000 7fb230358700 1 -- 172.21.15.144:0/1195387585 >> 172.21.15.144:0/1195387585 conn(0x7fb2280fbc90 msgr2=0x7fb2280fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:27.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.789+0000 7fb230358700 1 -- 172.21.15.144:0/1195387585 shutdown_connections 2024-09-10T11:32:27.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.789+0000 7fb230358700 1 -- 172.21.15.144:0/1195387585 wait complete. 2024-09-10T11:32:27.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.789+0000 7fb230358700 1 Processor -- start 2024-09-10T11:32:27.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.790+0000 7fb230358700 1 -- start start 2024-09-10T11:32:27.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.790+0000 7fb230358700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2281003d0 0x7fb228073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:27.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.790+0000 7fb230358700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb228100d70 0x7fb228073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:27.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.790+0000 7fb230358700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb228073ad0 con 0x7fb2281003d0 2024-09-10T11:32:27.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.790+0000 7fb230358700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb228073c10 con 0x7fb228100d70 2024-09-10T11:32:27.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.790+0000 7fb22e0f4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2281003d0 0x7fb228073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:27.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.790+0000 7fb22e0f4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2281003d0 0x7fb228073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53598/0 (socket says 172.21.15.144:53598) 2024-09-10T11:32:27.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.790+0000 7fb22e0f4700 1 -- 172.21.15.144:0/2196082091 learned_addr learned my addr 172.21.15.144:0/2196082091 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:32:27.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.790+0000 7fb22d8f3700 1 --2- 172.21.15.144:0/2196082091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb228100d70 0x7fb228073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:27.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.790+0000 7fb22e0f4700 1 -- 172.21.15.144:0/2196082091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb228100d70 msgr2=0x7fb228073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:27.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.790+0000 7fb22e0f4700 1 --2- 172.21.15.144:0/2196082091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb228100d70 0x7fb228073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:27.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.790+0000 7fb22e0f4700 1 -- 172.21.15.144:0/2196082091 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180095f0 con 0x7fb2281003d0 2024-09-10T11:32:27.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.790+0000 7fb22e0f4700 1 --2- 172.21.15.144:0/2196082091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2281003d0 0x7fb228073050 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb22400b4b0 tx=0x7fb22400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:27.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.790+0000 7fb21f7fe700 1 -- 172.21.15.144:0/2196082091 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb224011800 con 0x7fb2281003d0 2024-09-10T11:32:27.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.791+0000 7fb21f7fe700 1 -- 172.21.15.144:0/2196082091 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb224011e20 con 0x7fb2281003d0 2024-09-10T11:32:27.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.791+0000 7fb230358700 1 -- 172.21.15.144:0/2196082091 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2281a2050 con 0x7fb2281003d0 2024-09-10T11:32:27.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.791+0000 7fb21f7fe700 1 -- 172.21.15.144:0/2196082091 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb22400f2e0 con 0x7fb2281003d0 2024-09-10T11:32:27.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.791+0000 7fb230358700 1 -- 172.21.15.144:0/2196082091 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2281a2460 con 0x7fb2281003d0 2024-09-10T11:32:27.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.792+0000 7fb230358700 1 -- 172.21.15.144:0/2196082091 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb228108670 con 0x7fb2281003d0 2024-09-10T11:32:27.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.792+0000 7fb21f7fe700 1 -- 172.21.15.144:0/2196082091 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7fb224010380 con 0x7fb2281003d0 2024-09-10T11:32:27.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.793+0000 7fb21f7fe700 1 --2- 172.21.15.144:0/2196082091 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fb21406de80 0x7fb214070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:27.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.793+0000 7fb21f7fe700 1 -- 172.21.15.144:0/2196082091 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fb22408d620 con 0x7fb2281003d0 2024-09-10T11:32:27.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.793+0000 7fb22d8f3700 1 --2- 172.21.15.144:0/2196082091 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fb21406de80 0x7fb214070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:27.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.793+0000 7fb22d8f3700 1 --2- 172.21.15.144:0/2196082091 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fb21406de80 0x7fb214070330 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb228074840 tx=0x7fb21801a040 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:27.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:27.797+0000 7fb21f7fe700 1 -- 172.21.15.144:0/2196082091 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7fb224056f30 con 0x7fb2281003d0 2024-09-10T11:32:28.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:28.088+0000 7fb230358700 1 -- 172.21.15.144:0/2196082091 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb22802cd90 con 0x7fb2281003d0 2024-09-10T11:32:28.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:28.088+0000 7fb21f7fe700 1 -- 172.21.15.144:0/2196082091 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fb22405a5f0 con 0x7fb2281003d0 2024-09-10T11:32:28.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:28.090+0000 7fb230358700 1 -- 172.21.15.144:0/2196082091 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fb21406de80 msgr2=0x7fb214070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:28.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:28.090+0000 7fb230358700 1 --2- 172.21.15.144:0/2196082091 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fb21406de80 0x7fb214070330 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb228074840 tx=0x7fb21801a040 comp rx=0 tx=0).stop 2024-09-10T11:32:28.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:28.090+0000 7fb230358700 1 -- 172.21.15.144:0/2196082091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2281003d0 msgr2=0x7fb228073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:28.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:28.090+0000 7fb230358700 1 --2- 172.21.15.144:0/2196082091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2281003d0 0x7fb228073050 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fb22400b4b0 tx=0x7fb22400b7c0 comp rx=0 tx=0).stop 2024-09-10T11:32:28.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:28.090+0000 7fb230358700 1 -- 172.21.15.144:0/2196082091 shutdown_connections 2024-09-10T11:32:28.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:28.090+0000 7fb230358700 1 --2- 172.21.15.144:0/2196082091 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fb21406de80 0x7fb214070330 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:28.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:28.090+0000 7fb230358700 1 --2- 172.21.15.144:0/2196082091 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2281003d0 0x7fb228073050 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:28.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:28.090+0000 7fb230358700 1 --2- 172.21.15.144:0/2196082091 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb228100d70 0x7fb228073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:28.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:28.090+0000 7fb230358700 1 -- 172.21.15.144:0/2196082091 >> 172.21.15.144:0/2196082091 conn(0x7fb2280fbc90 msgr2=0x7fb2280fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:28.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:28.091+0000 7fb230358700 1 -- 172.21.15.144:0/2196082091 shutdown_connections 2024-09-10T11:32:28.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:28.091+0000 7fb230358700 1 -- 172.21.15.144:0/2196082091 wait complete. 2024-09-10T11:32:28.102 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:32:29.031 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-09-10T11:32:29.237 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:32:29.357 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:28 smithi144 ceph-mon[27942]: pgmap v70: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:29.357 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:28 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2196082091' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:29.357 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:28 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:32:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:28 smithi178 ceph-mon[33514]: pgmap v70: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:28 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/2196082091' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:29.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:28 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:32:31.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.015+0000 7f00f10f0700 1 -- 172.21.15.144:0/1438711877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00ec0ffc70 msgr2=0x7f00ec103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.015+0000 7f00f10f0700 1 --2- 172.21.15.144:0/1438711877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00ec0ffc70 0x7f00ec103c40 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f00dc009910 tx=0x7f00dc009c20 comp rx=0 tx=0).stop 2024-09-10T11:32:31.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.015+0000 7f00f10f0700 1 -- 172.21.15.144:0/1438711877 shutdown_connections 2024-09-10T11:32:31.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.015+0000 7f00f10f0700 1 --2- 172.21.15.144:0/1438711877 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00ec0ffc70 0x7f00ec103c40 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.015+0000 7f00f10f0700 1 --2- 172.21.15.144:0/1438711877 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00ec0ff2d0 0x7f00ec0ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.015+0000 7f00f10f0700 1 -- 172.21.15.144:0/1438711877 >> 172.21.15.144:0/1438711877 conn(0x7f00ec0fabb0 msgr2=0x7f00ec0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:31.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.015+0000 7f00f10f0700 1 -- 172.21.15.144:0/1438711877 shutdown_connections 2024-09-10T11:32:31.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.015+0000 7f00f10f0700 1 -- 172.21.15.144:0/1438711877 wait complete. 2024-09-10T11:32:31.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.016+0000 7f00f10f0700 1 Processor -- start 2024-09-10T11:32:31.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.016+0000 7f00f10f0700 1 -- start start 2024-09-10T11:32:31.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00f10f0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00ec0ff2d0 0x7f00ec195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00f10f0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00ec0ffc70 0x7f00ec196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00f10f0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec196a60 con 0x7f00ec0ffc70 2024-09-10T11:32:31.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00f10f0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec19a7f0 con 0x7f00ec0ff2d0 2024-09-10T11:32:31.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00ea59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00ec0ffc70 0x7f00ec196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00ea59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00ec0ffc70 0x7f00ec196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53608/0 (socket says 172.21.15.144:53608) 2024-09-10T11:32:31.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00ea59c700 1 -- 172.21.15.144:0/1788368726 learned_addr learned my addr 172.21.15.144:0/1788368726 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:32:31.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00ead9d700 1 --2- 172.21.15.144:0/1788368726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00ec0ff2d0 0x7f00ec195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00ea59c700 1 -- 172.21.15.144:0/1788368726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00ec0ff2d0 msgr2=0x7f00ec195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00ea59c700 1 --2- 172.21.15.144:0/1788368726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00ec0ff2d0 0x7f00ec195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00ea59c700 1 -- 172.21.15.144:0/1788368726 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00dc0095f0 con 0x7f00ec0ffc70 2024-09-10T11:32:31.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00ea59c700 1 --2- 172.21.15.144:0/1788368726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00ec0ffc70 0x7f00ec196380 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0092e0 tx=0x7f00dc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:31.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00d3fff700 1 -- 172.21.15.144:0/1788368726 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00dc01d070 con 0x7f00ec0ffc70 2024-09-10T11:32:31.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00f10f0700 1 -- 172.21.15.144:0/1788368726 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ec19aa70 con 0x7f00ec0ffc70 2024-09-10T11:32:31.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00d3fff700 1 -- 172.21.15.144:0/1788368726 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00dc00f460 con 0x7f00ec0ffc70 2024-09-10T11:32:31.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.017+0000 7f00d3fff700 1 -- 172.21.15.144:0/1788368726 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00dc0175d0 con 0x7f00ec0ffc70 2024-09-10T11:32:31.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.018+0000 7f00f10f0700 1 -- 172.21.15.144:0/1788368726 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ec19af60 con 0x7f00ec0ffc70 2024-09-10T11:32:31.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.018+0000 7f00f10f0700 1 -- 172.21.15.144:0/1788368726 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ec04ea50 con 0x7f00ec0ffc70 2024-09-10T11:32:31.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.019+0000 7f00d3fff700 1 -- 172.21.15.144:0/1788368726 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f00dc00f5d0 con 0x7f00ec0ffc70 2024-09-10T11:32:31.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.019+0000 7f00d3fff700 1 --2- 172.21.15.144:0/1788368726 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f00d406de80 0x7f00d4070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:31.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.020+0000 7f00d3fff700 1 -- 172.21.15.144:0/1788368726 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f00dc08efe0 con 0x7f00ec0ffc70 2024-09-10T11:32:31.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.020+0000 7f00ead9d700 1 --2- 172.21.15.144:0/1788368726 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f00d406de80 0x7f00d4070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:31.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.020+0000 7f00ead9d700 1 --2- 172.21.15.144:0/1788368726 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f00d406de80 0x7f00d4070330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009a10 tx=0x7f00e0009190 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:31.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.024+0000 7f00d3fff700 1 -- 172.21.15.144:0/1788368726 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f00dc058890 con 0x7f00ec0ffc70 2024-09-10T11:32:31.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:30 smithi144 ceph-mon[27942]: pgmap v71: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:31.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.309+0000 7f00f10f0700 1 -- 172.21.15.144:0/1788368726 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f00ec19b310 con 0x7f00ec0ffc70 2024-09-10T11:32:31.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.310+0000 7f00d3fff700 1 -- 172.21.15.144:0/1788368726 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7f00dc05bf50 con 0x7f00ec0ffc70 2024-09-10T11:32:31.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.312+0000 7f00f10f0700 1 -- 172.21.15.144:0/1788368726 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f00d406de80 msgr2=0x7f00d4070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.312+0000 7f00f10f0700 1 --2- 172.21.15.144:0/1788368726 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f00d406de80 0x7f00d4070330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f00e0009a10 tx=0x7f00e0009190 comp rx=0 tx=0).stop 2024-09-10T11:32:31.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.312+0000 7f00f10f0700 1 -- 172.21.15.144:0/1788368726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00ec0ffc70 msgr2=0x7f00ec196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:31.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.312+0000 7f00f10f0700 1 --2- 172.21.15.144:0/1788368726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00ec0ffc70 0x7f00ec196380 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f00dc0092e0 tx=0x7f00dc00bd70 comp rx=0 tx=0).stop 2024-09-10T11:32:31.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.312+0000 7f00f10f0700 1 -- 172.21.15.144:0/1788368726 shutdown_connections 2024-09-10T11:32:31.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.312+0000 7f00f10f0700 1 --2- 172.21.15.144:0/1788368726 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f00d406de80 0x7f00d4070330 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.312+0000 7f00f10f0700 1 --2- 172.21.15.144:0/1788368726 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f00ec0ff2d0 0x7f00ec195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.312+0000 7f00f10f0700 1 --2- 172.21.15.144:0/1788368726 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f00ec0ffc70 0x7f00ec196380 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:31.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.312+0000 7f00f10f0700 1 -- 172.21.15.144:0/1788368726 >> 172.21.15.144:0/1788368726 conn(0x7f00ec0fabb0 msgr2=0x7f00ec1033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:31.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.312+0000 7f00f10f0700 1 -- 172.21.15.144:0/1788368726 shutdown_connections 2024-09-10T11:32:31.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:31.312+0000 7f00f10f0700 1 -- 172.21.15.144:0/1788368726 wait complete. 2024-09-10T11:32:31.324 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)" 2024-09-10T11:32:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:30 smithi178 ceph-mon[33514]: pgmap v71: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:31.990 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-09-10T11:32:32.192 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:32:32.312 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:31 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/1788368726' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:32.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:31 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/1788368726' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:32 smithi178 ceph-mon[33514]: pgmap v72: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:33.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:32 smithi144 ceph-mon[27942]: pgmap v72: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:33.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.950+0000 7faf0478d700 1 -- 172.21.15.144:0/1781501747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faefc0ffd30 msgr2=0x7faefc103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:33.955 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.950+0000 7faf0478d700 1 --2- 172.21.15.144:0/1781501747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faefc0ffd30 0x7faefc103ce0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faef8009910 tx=0x7faef8009c20 comp rx=0 tx=0).stop 2024-09-10T11:32:33.955 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.950+0000 7faf0478d700 1 -- 172.21.15.144:0/1781501747 shutdown_connections 2024-09-10T11:32:33.955 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.950+0000 7faf0478d700 1 --2- 172.21.15.144:0/1781501747 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faefc0ffd30 0x7faefc103ce0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:33.955 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.950+0000 7faf0478d700 1 --2- 172.21.15.144:0/1781501747 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faefc0ff390 0x7faefc0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:33.955 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.950+0000 7faf0478d700 1 -- 172.21.15.144:0/1781501747 >> 172.21.15.144:0/1781501747 conn(0x7faefc074f50 msgr2=0x7faefc075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:33.955 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.950+0000 7faf0478d700 1 -- 172.21.15.144:0/1781501747 shutdown_connections 2024-09-10T11:32:33.955 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.951+0000 7faf0478d700 1 -- 172.21.15.144:0/1781501747 wait complete. 2024-09-10T11:32:33.955 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.951+0000 7faf0478d700 1 Processor -- start 2024-09-10T11:32:33.955 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.951+0000 7faf0478d700 1 -- start start 2024-09-10T11:32:33.955 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.951+0000 7faf0478d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faefc0ff390 0x7faefc196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:33.955 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.951+0000 7faf0478d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faefc0ffd30 0x7faefc1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:33.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.951+0000 7faf0478d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc1976c0 con 0x7faefc0ff390 2024-09-10T11:32:33.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.951+0000 7faf0478d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc197830 con 0x7faefc0ffd30 2024-09-10T11:32:33.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.951+0000 7faf02529700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faefc0ff390 0x7faefc196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:33.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.952+0000 7faf02529700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faefc0ff390 0x7faefc196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34140/0 (socket says 172.21.15.144:34140) 2024-09-10T11:32:33.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.952+0000 7faf02529700 1 -- 172.21.15.144:0/523368210 learned_addr learned my addr 172.21.15.144:0/523368210 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:32:33.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.952+0000 7faf01d28700 1 --2- 172.21.15.144:0/523368210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faefc0ffd30 0x7faefc1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:33.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.952+0000 7faf02529700 1 -- 172.21.15.144:0/523368210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faefc0ffd30 msgr2=0x7faefc1970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:33.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.952+0000 7faf02529700 1 --2- 172.21.15.144:0/523368210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faefc0ffd30 0x7faefc1970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:33.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.952+0000 7faf02529700 1 -- 172.21.15.144:0/523368210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef80095f0 con 0x7faefc0ff390 2024-09-10T11:32:33.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.952+0000 7faf02529700 1 --2- 172.21.15.144:0/523368210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faefc0ff390 0x7faefc196b60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faeec00c8a0 tx=0x7faeec00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:33.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.952+0000 7faef37fe700 1 -- 172.21.15.144:0/523368210 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faeec0078e0 con 0x7faefc0ff390 2024-09-10T11:32:33.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.952+0000 7faef37fe700 1 -- 172.21.15.144:0/523368210 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faeec00f450 con 0x7faefc0ff390 2024-09-10T11:32:33.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.952+0000 7faf0478d700 1 -- 172.21.15.144:0/523368210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faefc19fbd0 con 0x7faefc0ff390 2024-09-10T11:32:33.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.952+0000 7faef37fe700 1 -- 172.21.15.144:0/523368210 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faeec00e5e0 con 0x7faefc0ff390 2024-09-10T11:32:33.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.952+0000 7faf0478d700 1 -- 172.21.15.144:0/523368210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faefc1a0090 con 0x7faefc0ff390 2024-09-10T11:32:33.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.953+0000 7faf0478d700 1 -- 172.21.15.144:0/523368210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faefc107600 con 0x7faefc0ff390 2024-09-10T11:32:33.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.954+0000 7faef37fe700 1 -- 172.21.15.144:0/523368210 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7faeec00f5c0 con 0x7faefc0ff390 2024-09-10T11:32:33.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.954+0000 7faef37fe700 1 --2- 172.21.15.144:0/523368210 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7faee806de80 0x7faee8070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:33.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.954+0000 7faef37fe700 1 -- 172.21.15.144:0/523368210 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7faeec08da70 con 0x7faefc0ff390 2024-09-10T11:32:33.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.954+0000 7faf01d28700 1 --2- 172.21.15.144:0/523368210 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7faee806de80 0x7faee8070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:33.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.955+0000 7faf01d28700 1 --2- 172.21.15.144:0/523368210 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7faee806de80 0x7faee8070330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faef8006010 tx=0x7faef801a040 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:33.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:33.959+0000 7faef37fe700 1 -- 172.21.15.144:0/523368210 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7faeec057380 con 0x7faefc0ff390 2024-09-10T11:32:34.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:34.250+0000 7faf0478d700 1 -- 172.21.15.144:0/523368210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faefc02cc30 con 0x7faefc0ff390 2024-09-10T11:32:34.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:34.250+0000 7faef37fe700 1 -- 172.21.15.144:0/523368210 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7faeec05aa40 con 0x7faefc0ff390 2024-09-10T11:32:34.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:34.252+0000 7faf0478d700 1 -- 172.21.15.144:0/523368210 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7faee806de80 msgr2=0x7faee8070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:34.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:34.252+0000 7faf0478d700 1 --2- 172.21.15.144:0/523368210 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7faee806de80 0x7faee8070330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7faef8006010 tx=0x7faef801a040 comp rx=0 tx=0).stop 2024-09-10T11:32:34.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:34.252+0000 7faf0478d700 1 -- 172.21.15.144:0/523368210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faefc0ff390 msgr2=0x7faefc196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:34.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:34.252+0000 7faf0478d700 1 --2- 172.21.15.144:0/523368210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faefc0ff390 0x7faefc196b60 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7faeec00c8a0 tx=0x7faeec00cbb0 comp rx=0 tx=0).stop 2024-09-10T11:32:34.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:34.252+0000 7faf0478d700 1 -- 172.21.15.144:0/523368210 shutdown_connections 2024-09-10T11:32:34.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:34.252+0000 7faf0478d700 1 --2- 172.21.15.144:0/523368210 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7faee806de80 0x7faee8070330 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:34.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:34.252+0000 7faf0478d700 1 --2- 172.21.15.144:0/523368210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faefc0ff390 0x7faefc196b60 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:34.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:34.253+0000 7faf0478d700 1 --2- 172.21.15.144:0/523368210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faefc0ffd30 0x7faefc1970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:34.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:34.253+0000 7faf0478d700 1 -- 172.21.15.144:0/523368210 >> 172.21.15.144:0/523368210 conn(0x7faefc074f50 msgr2=0x7faefc0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:34.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:34.253+0000 7faf0478d700 1 -- 172.21.15.144:0/523368210 shutdown_connections 2024-09-10T11:32:34.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:34.253+0000 7faf0478d700 1 -- 172.21.15.144:0/523368210 wait complete. 2024-09-10T11:32:34.265 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:32:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:34 smithi178 ceph-mon[33514]: pgmap v73: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:34 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/523368210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:35.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:34 smithi144 ceph-mon[27942]: pgmap v73: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:35.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:34 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/523368210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:32:37.305 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:36 smithi144 ceph-mon[27942]: pgmap v74: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:36 smithi178 ceph-mon[33514]: pgmap v74: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:37.706 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-09-10T11:32:37.914 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:32:39.332 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:38 smithi144 ceph-mon[27942]: pgmap v75: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:39.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:38 smithi178 ceph-mon[33514]: pgmap v75: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:39.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.567+0000 7f425455d700 1 -- 172.21.15.144:0/1766566930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f424c100d80 msgr2=0x7f424c104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:39.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.567+0000 7f425455d700 1 --2- 172.21.15.144:0/1766566930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f424c100d80 0x7f424c104d50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f423c009910 tx=0x7f423c009c20 comp rx=0 tx=0).stop 2024-09-10T11:32:39.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.567+0000 7f425455d700 1 -- 172.21.15.144:0/1766566930 shutdown_connections 2024-09-10T11:32:39.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.567+0000 7f425455d700 1 --2- 172.21.15.144:0/1766566930 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f424c100d80 0x7f424c104d50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:39.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.567+0000 7f425455d700 1 --2- 172.21.15.144:0/1766566930 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f424c1003e0 0x7f424c1007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:39.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.567+0000 7f425455d700 1 -- 172.21.15.144:0/1766566930 >> 172.21.15.144:0/1766566930 conn(0x7f424c0fbca0 msgr2=0x7f424c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:39.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.567+0000 7f425455d700 1 -- 172.21.15.144:0/1766566930 shutdown_connections 2024-09-10T11:32:39.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.567+0000 7f425455d700 1 -- 172.21.15.144:0/1766566930 wait complete. 2024-09-10T11:32:39.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.568+0000 7f425455d700 1 Processor -- start 2024-09-10T11:32:39.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.568+0000 7f425455d700 1 -- start start 2024-09-10T11:32:39.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.568+0000 7f425455d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f424c1003e0 0x7f424c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:39.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.568+0000 7f425455d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f424c100d80 0x7f424c1962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:39.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.568+0000 7f425455d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c1969d0 con 0x7f424c100d80 2024-09-10T11:32:39.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.568+0000 7f425455d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f424c19a760 con 0x7f424c1003e0 2024-09-10T11:32:39.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.568+0000 7f4251af8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f424c100d80 0x7f424c1962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:39.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.568+0000 7f4251af8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f424c100d80 0x7f424c1962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34158/0 (socket says 172.21.15.144:34158) 2024-09-10T11:32:39.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.568+0000 7f4251af8700 1 -- 172.21.15.144:0/2006313347 learned_addr learned my addr 172.21.15.144:0/2006313347 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:32:39.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.568+0000 7f42522f9700 1 --2- 172.21.15.144:0/2006313347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f424c1003e0 0x7f424c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:39.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.568+0000 7f4251af8700 1 -- 172.21.15.144:0/2006313347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f424c1003e0 msgr2=0x7f424c195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:39.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.568+0000 7f4251af8700 1 --2- 172.21.15.144:0/2006313347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f424c1003e0 0x7f424c195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:39.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.568+0000 7f4251af8700 1 -- 172.21.15.144:0/2006313347 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f423c0095f0 con 0x7f424c100d80 2024-09-10T11:32:39.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.569+0000 7f4251af8700 1 --2- 172.21.15.144:0/2006313347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f424c100d80 0x7f424c1962f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f423c0092e0 tx=0x7f423c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:39.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.569+0000 7f42437fe700 1 -- 172.21.15.144:0/2006313347 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f423c01d070 con 0x7f424c100d80 2024-09-10T11:32:39.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.569+0000 7f425455d700 1 -- 172.21.15.144:0/2006313347 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f424c19a9e0 con 0x7f424c100d80 2024-09-10T11:32:39.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.569+0000 7f42437fe700 1 -- 172.21.15.144:0/2006313347 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f423c00f460 con 0x7f424c100d80 2024-09-10T11:32:39.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.569+0000 7f42437fe700 1 -- 172.21.15.144:0/2006313347 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f423c0175f0 con 0x7f424c100d80 2024-09-10T11:32:39.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.569+0000 7f425455d700 1 -- 172.21.15.144:0/2006313347 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f424c19aed0 con 0x7f424c100d80 2024-09-10T11:32:39.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.570+0000 7f425455d700 1 -- 172.21.15.144:0/2006313347 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f424c04ea50 con 0x7f424c100d80 2024-09-10T11:32:39.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.570+0000 7f42437fe700 1 -- 172.21.15.144:0/2006313347 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f423c00f5d0 con 0x7f424c100d80 2024-09-10T11:32:39.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.571+0000 7f42437fe700 1 --2- 172.21.15.144:0/2006313347 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f423806de80 0x7f4238070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:39.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.571+0000 7f42437fe700 1 -- 172.21.15.144:0/2006313347 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f423c08ef70 con 0x7f424c100d80 2024-09-10T11:32:39.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.571+0000 7f42522f9700 1 --2- 172.21.15.144:0/2006313347 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f423806de80 0x7f4238070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:39.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.571+0000 7f42522f9700 1 --2- 172.21.15.144:0/2006313347 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f423806de80 0x7f4238070330 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f42480060b0 tx=0x7f4248007b80 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:39.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.575+0000 7f42437fe700 1 -- 172.21.15.144:0/2006313347 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f423c0c39d0 con 0x7f424c100d80 2024-09-10T11:32:39.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:39.798+0000 7f425455d700 1 -- 172.21.15.144:0/2006313347 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b", "target": ["mon-mgr", ""]}) v1 -- 0x7f424c19b2e0 con 0x7f423806de80 2024-09-10T11:32:41.067 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:40 smithi144 ceph-mon[27942]: pgmap v76: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:32:41.068 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:40 smithi144 ceph-mon[27942]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:32:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:40 smithi178 ceph-mon[33514]: pgmap v76: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:32:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:40 smithi178 ceph-mon[33514]: from='client.14788 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:32:41.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:41.668+0000 7f42437fe700 1 -- 172.21.15.144:0/2006313347 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5393 (secure 0 0 0) 0x7f424c19b2e0 con 0x7f423806de80 2024-09-10T11:32:41.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:41.671+0000 7f425455d700 1 -- 172.21.15.144:0/2006313347 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f423806de80 msgr2=0x7f4238070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:41.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:41.671+0000 7f425455d700 1 --2- 172.21.15.144:0/2006313347 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f423806de80 0x7f4238070330 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f42480060b0 tx=0x7f4248007b80 comp rx=0 tx=0).stop 2024-09-10T11:32:41.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:41.671+0000 7f425455d700 1 -- 172.21.15.144:0/2006313347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f424c100d80 msgr2=0x7f424c1962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:41.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:41.671+0000 7f425455d700 1 --2- 172.21.15.144:0/2006313347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f424c100d80 0x7f424c1962f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f423c0092e0 tx=0x7f423c00be20 comp rx=0 tx=0).stop 2024-09-10T11:32:41.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:41.671+0000 7f425455d700 1 -- 172.21.15.144:0/2006313347 shutdown_connections 2024-09-10T11:32:41.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:41.671+0000 7f425455d700 1 --2- 172.21.15.144:0/2006313347 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f423806de80 0x7f4238070330 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:41.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:41.671+0000 7f425455d700 1 --2- 172.21.15.144:0/2006313347 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f424c1003e0 0x7f424c195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:41.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:41.671+0000 7f425455d700 1 --2- 172.21.15.144:0/2006313347 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f424c100d80 0x7f424c1962f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:41.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:41.671+0000 7f425455d700 1 -- 172.21.15.144:0/2006313347 >> 172.21.15.144:0/2006313347 conn(0x7f424c0fbca0 msgr2=0x7f424c0fd240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:41.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:41.671+0000 7f425455d700 1 -- 172.21.15.144:0/2006313347 shutdown_connections 2024-09-10T11:32:41.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:41.671+0000 7f425455d700 1 -- 172.21.15.144:0/2006313347 wait complete. 2024-09-10T11:32:41.684 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:32:42.304 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:42 smithi144 ceph-mon[27942]: pgmap v77: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:42.326 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph orch ps' 2024-09-10T11:32:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:42 smithi178 ceph-mon[33514]: pgmap v77: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:42.528 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:32:44.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.217+0000 7f4d23d95700 1 -- 172.21.15.144:0/3621998089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ffd30 msgr2=0x7f4d1c103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:44.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.217+0000 7f4d23d95700 1 --2- 172.21.15.144:0/3621998089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ffd30 0x7f4d1c103ce0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4d18009910 tx=0x7f4d18009c20 comp rx=0 tx=0).stop 2024-09-10T11:32:44.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.217+0000 7f4d23d95700 1 -- 172.21.15.144:0/3621998089 shutdown_connections 2024-09-10T11:32:44.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.217+0000 7f4d23d95700 1 --2- 172.21.15.144:0/3621998089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ffd30 0x7f4d1c103ce0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:44.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.217+0000 7f4d23d95700 1 --2- 172.21.15.144:0/3621998089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c0ff390 0x7f4d1c0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:44.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.217+0000 7f4d23d95700 1 -- 172.21.15.144:0/3621998089 >> 172.21.15.144:0/3621998089 conn(0x7f4d1c074f50 msgr2=0x7f4d1c075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:44.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.217+0000 7f4d23d95700 1 -- 172.21.15.144:0/3621998089 shutdown_connections 2024-09-10T11:32:44.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.217+0000 7f4d23d95700 1 -- 172.21.15.144:0/3621998089 wait complete. 2024-09-10T11:32:44.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.218+0000 7f4d23d95700 1 Processor -- start 2024-09-10T11:32:44.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.218+0000 7f4d23d95700 1 -- start start 2024-09-10T11:32:44.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.218+0000 7f4d23d95700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ff390 0x7f4d1c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:44.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.218+0000 7f4d23d95700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c0ffd30 0x7f4d1c19af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:44.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.218+0000 7f4d23d95700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c19b5b0 con 0x7f4d1c0ff390 2024-09-10T11:32:44.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.218+0000 7f4d23d95700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c194a60 con 0x7f4d1c0ffd30 2024-09-10T11:32:44.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.218+0000 7f4d21b31700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ff390 0x7f4d1c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:44.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.218+0000 7f4d21b31700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ff390 0x7f4d1c19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34900/0 (socket says 172.21.15.144:34900) 2024-09-10T11:32:44.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.218+0000 7f4d21b31700 1 -- 172.21.15.144:0/4212022394 learned_addr learned my addr 172.21.15.144:0/4212022394 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:32:44.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.218+0000 7f4d21330700 1 --2- 172.21.15.144:0/4212022394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c0ffd30 0x7f4d1c19af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:44.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.219+0000 7f4d21b31700 1 -- 172.21.15.144:0/4212022394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c0ffd30 msgr2=0x7f4d1c19af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:44.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.219+0000 7f4d21b31700 1 --2- 172.21.15.144:0/4212022394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c0ffd30 0x7f4d1c19af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:44.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.219+0000 7f4d21b31700 1 -- 172.21.15.144:0/4212022394 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d180095f0 con 0x7f4d1c0ff390 2024-09-10T11:32:44.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.219+0000 7f4d21b31700 1 --2- 172.21.15.144:0/4212022394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ff390 0x7f4d1c19a9e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00d8d0 tx=0x7f4d0c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:44.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.219+0000 7f4d12ffd700 1 -- 172.21.15.144:0/4212022394 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d0c0098a0 con 0x7f4d1c0ff390 2024-09-10T11:32:44.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.219+0000 7f4d23d95700 1 -- 172.21.15.144:0/4212022394 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c194d40 con 0x7f4d1c0ff390 2024-09-10T11:32:44.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.219+0000 7f4d12ffd700 1 -- 172.21.15.144:0/4212022394 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d0c010460 con 0x7f4d1c0ff390 2024-09-10T11:32:44.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.219+0000 7f4d12ffd700 1 -- 172.21.15.144:0/4212022394 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d0c00f5f0 con 0x7f4d1c0ff390 2024-09-10T11:32:44.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.219+0000 7f4d23d95700 1 -- 172.21.15.144:0/4212022394 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c195290 con 0x7f4d1c0ff390 2024-09-10T11:32:44.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.220+0000 7f4d23d95700 1 -- 172.21.15.144:0/4212022394 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c107600 con 0x7f4d1c0ff390 2024-09-10T11:32:44.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.220+0000 7f4d12ffd700 1 -- 172.21.15.144:0/4212022394 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f4d0c009a00 con 0x7f4d1c0ff390 2024-09-10T11:32:44.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.221+0000 7f4d12ffd700 1 --2- 172.21.15.144:0/4212022394 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f4d0806de80 0x7f4d08070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:44.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.221+0000 7f4d12ffd700 1 -- 172.21.15.144:0/4212022394 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f4d0c08da30 con 0x7f4d1c0ff390 2024-09-10T11:32:44.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.221+0000 7f4d21330700 1 --2- 172.21.15.144:0/4212022394 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f4d0806de80 0x7f4d08070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:44.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.221+0000 7f4d21330700 1 --2- 172.21.15.144:0/4212022394 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f4d0806de80 0x7f4d08070330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c103e20 tx=0x7f4d1801a040 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:44.228 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.226+0000 7f4d12ffd700 1 -- 172.21.15.144:0/4212022394 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f4d0c05add0 con 0x7f4d1c0ff390 2024-09-10T11:32:44.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.436+0000 7f4d23d95700 1 -- 172.21.15.144:0/4212022394 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d1c195550 con 0x7f4d0806de80 2024-09-10T11:32:44.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.446+0000 7f4d12ffd700 1 -- 172.21.15.144:0/4212022394 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f4d1c195550 con 0x7f4d0806de80 2024-09-10T11:32:44.447 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (58s) 31s ago 11m 15.9M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (11m) 31s ago 11m 7990k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (10m) 88s ago 10m 7830k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (11m) 31s ago 11m 6669k - 18.2.0 dc2bc1663786 3311fb04aa0c 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (10m) 88s ago 10m 6651k - 18.2.0 dc2bc1663786 6e408e8476d4 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (36s) 31s ago 11m 46.3M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (6m) 31s ago 6m 59.9M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (5m) 31s ago 5m 127M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (5m) 88s ago 5m 65.7M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (6m) 88s ago 6m 127M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (2m) 31s ago 12m 505M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:32:44.448 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (112s) 88s ago 10m 441M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (13m) 31s ago 13m 56.5M 2048M 18.2.0 dc2bc1663786 42addb654e4f 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (10m) 88s ago 10m 43.5M 2048M 18.2.0 dc2bc1663786 3c1ba210ed3f 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (95s) 31s ago 11m 15.8M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (91s) 88s ago 10m 3586k - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (9m) 31s ago 9m 205M 1216M 18.2.0 dc2bc1663786 2435febebdb5 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (9m) 31s ago 9m 212M 1216M 18.2.0 dc2bc1663786 b55fe920ff9d 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (8m) 31s ago 8m 169M 1216M 18.2.0 dc2bc1663786 3ed5618eac33 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (8m) 88s ago 8m 252M 2239M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (7m) 88s ago 7m 217M 2239M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (7m) 88s ago 7m 195M 2239M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:32:44.449 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (75s) 31s ago 11m 63.9M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.449+0000 7f4d23d95700 1 -- 172.21.15.144:0/4212022394 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f4d0806de80 msgr2=0x7f4d08070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.449+0000 7f4d23d95700 1 --2- 172.21.15.144:0/4212022394 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f4d0806de80 0x7f4d08070330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c103e20 tx=0x7f4d1801a040 comp rx=0 tx=0).stop 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.449+0000 7f4d23d95700 1 -- 172.21.15.144:0/4212022394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ff390 msgr2=0x7f4d1c19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.449+0000 7f4d23d95700 1 --2- 172.21.15.144:0/4212022394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ff390 0x7f4d1c19a9e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c00d8d0 tx=0x7f4d0c00dbe0 comp rx=0 tx=0).stop 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.450+0000 7f4d23d95700 1 -- 172.21.15.144:0/4212022394 shutdown_connections 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.450+0000 7f4d23d95700 1 --2- 172.21.15.144:0/4212022394 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f4d0806de80 0x7f4d08070330 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:44.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.450+0000 7f4d23d95700 1 --2- 172.21.15.144:0/4212022394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d1c0ff390 0x7f4d1c19a9e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:44.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.450+0000 7f4d23d95700 1 --2- 172.21.15.144:0/4212022394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d1c0ffd30 0x7f4d1c19af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:44.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.450+0000 7f4d23d95700 1 -- 172.21.15.144:0/4212022394 >> 172.21.15.144:0/4212022394 conn(0x7f4d1c074f50 msgr2=0x7f4d1c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:44.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.450+0000 7f4d23d95700 1 -- 172.21.15.144:0/4212022394 shutdown_connections 2024-09-10T11:32:44.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:44.450+0000 7f4d23d95700 1 -- 172.21.15.144:0/4212022394 wait complete. 2024-09-10T11:32:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:44 smithi178 ceph-mon[33514]: pgmap v78: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:44 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:32:44.882 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:44 smithi144 ceph-mon[27942]: pgmap v78: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:44.882 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:44 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:32:45.134 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-10T11:32:45.135 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T11:32:45.135 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-09-10T11:32:45.336 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:32:45.716 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:45 smithi144 ceph-mon[27942]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:32:45.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:45 smithi178 ceph-mon[33514]: from='client.14792 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:32:46.713 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:46 smithi144 ceph-mon[27942]: pgmap v79: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:46 smithi178 ceph-mon[33514]: pgmap v79: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:46.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.986+0000 7f73a871d700 1 -- 172.21.15.144:0/3667526066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73a0073980 msgr2=0x7f73a010c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:46.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.986+0000 7f73a871d700 1 --2- 172.21.15.144:0/3667526066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73a0073980 0x7f73a010c4f0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7390009940 tx=0x7f7390009c50 comp rx=0 tx=0).stop 2024-09-10T11:32:46.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.986+0000 7f73a871d700 1 -- 172.21.15.144:0/3667526066 shutdown_connections 2024-09-10T11:32:46.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.986+0000 7f73a871d700 1 --2- 172.21.15.144:0/3667526066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73a0073980 0x7f73a010c4f0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:46.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.986+0000 7f73a871d700 1 --2- 172.21.15.144:0/3667526066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73a0073070 0x7f73a0073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:46.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.986+0000 7f73a871d700 1 -- 172.21.15.144:0/3667526066 >> 172.21.15.144:0/3667526066 conn(0x7f73a00fbc90 msgr2=0x7f73a00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:46.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.986+0000 7f73a871d700 1 -- 172.21.15.144:0/3667526066 shutdown_connections 2024-09-10T11:32:46.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.986+0000 7f73a871d700 1 -- 172.21.15.144:0/3667526066 wait complete. 2024-09-10T11:32:46.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.986+0000 7f73a871d700 1 Processor -- start 2024-09-10T11:32:46.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.986+0000 7f73a871d700 1 -- start start 2024-09-10T11:32:46.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73a871d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73a0073070 0x7f73a01980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:46.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73a871d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73a0073980 0x7f73a01985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:46.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73a871d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a0198cc0 con 0x7f73a0073980 2024-09-10T11:32:46.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73a871d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a019ca50 con 0x7f73a0073070 2024-09-10T11:32:46.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73a5cb8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73a0073980 0x7f73a01985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:46.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73a5cb8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73a0073980 0x7f73a01985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34916/0 (socket says 172.21.15.144:34916) 2024-09-10T11:32:46.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73a5cb8700 1 -- 172.21.15.144:0/3632143970 learned_addr learned my addr 172.21.15.144:0/3632143970 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:32:46.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73a64b9700 1 --2- 172.21.15.144:0/3632143970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73a0073070 0x7f73a01980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:46.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73a5cb8700 1 -- 172.21.15.144:0/3632143970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73a0073070 msgr2=0x7f73a01980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:46.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73a5cb8700 1 --2- 172.21.15.144:0/3632143970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73a0073070 0x7f73a01980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:46.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73a5cb8700 1 -- 172.21.15.144:0/3632143970 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73900095f0 con 0x7f73a0073980 2024-09-10T11:32:46.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73a5cb8700 1 --2- 172.21.15.144:0/3632143970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73a0073980 0x7f73a01985e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f73900055f0 tx=0x7f73900056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:46.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73977fe700 1 -- 172.21.15.144:0/3632143970 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f739001d070 con 0x7f73a0073980 2024-09-10T11:32:46.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73977fe700 1 -- 172.21.15.144:0/3632143970 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f739000f460 con 0x7f73a0073980 2024-09-10T11:32:46.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73a871d700 1 -- 172.21.15.144:0/3632143970 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a019ccd0 con 0x7f73a0073980 2024-09-10T11:32:46.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73977fe700 1 -- 172.21.15.144:0/3632143970 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7390021620 con 0x7f73a0073980 2024-09-10T11:32:46.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.987+0000 7f73a871d700 1 -- 172.21.15.144:0/3632143970 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a019d1c0 con 0x7f73a0073980 2024-09-10T11:32:46.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.988+0000 7f73a871d700 1 -- 172.21.15.144:0/3632143970 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a0109bf0 con 0x7f73a0073980 2024-09-10T11:32:46.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.989+0000 7f73977fe700 1 -- 172.21.15.144:0/3632143970 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f739000fa60 con 0x7f73a0073980 2024-09-10T11:32:46.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.989+0000 7f73977fe700 1 --2- 172.21.15.144:0/3632143970 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f738c06de80 0x7f738c070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:46.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.989+0000 7f73977fe700 1 -- 172.21.15.144:0/3632143970 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f739008ee30 con 0x7f73a0073980 2024-09-10T11:32:46.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.989+0000 7f73a64b9700 1 --2- 172.21.15.144:0/3632143970 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f738c06de80 0x7f738c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:46.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.990+0000 7f73a64b9700 1 --2- 172.21.15.144:0/3632143970 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f738c06de80 0x7f738c070330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f739c006fd0 tx=0x7f739c008040 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:46.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:46.994+0000 7f73977fe700 1 -- 172.21.15.144:0/3632143970 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f7390058740 con 0x7f73a0073980 2024-09-10T11:32:47.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:47.192+0000 7f73a871d700 1 -- 172.21.15.144:0/3632143970 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7f73a002cca0 con 0x7f73a0073980 2024-09-10T11:32:47.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:47.196+0000 7f73977fe700 1 -- 172.21.15.144:0/3632143970 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v45)=0 v45) v1 ==== 125+0+0 (secure 0 0 0) 0x7f739005be00 con 0x7f73a0073980 2024-09-10T11:32:47.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:47.199+0000 7f73a871d700 1 -- 172.21.15.144:0/3632143970 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f738c06de80 msgr2=0x7f738c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:47.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:47.199+0000 7f73a871d700 1 --2- 172.21.15.144:0/3632143970 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f738c06de80 0x7f738c070330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f739c006fd0 tx=0x7f739c008040 comp rx=0 tx=0).stop 2024-09-10T11:32:47.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:47.199+0000 7f73a871d700 1 -- 172.21.15.144:0/3632143970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73a0073980 msgr2=0x7f73a01985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:47.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:47.199+0000 7f73a871d700 1 --2- 172.21.15.144:0/3632143970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73a0073980 0x7f73a01985e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f73900055f0 tx=0x7f73900056d0 comp rx=0 tx=0).stop 2024-09-10T11:32:47.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:47.199+0000 7f73a871d700 1 -- 172.21.15.144:0/3632143970 shutdown_connections 2024-09-10T11:32:47.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:47.199+0000 7f73a871d700 1 --2- 172.21.15.144:0/3632143970 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f738c06de80 0x7f738c070330 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:47.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:47.199+0000 7f73a871d700 1 --2- 172.21.15.144:0/3632143970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73a0073070 0x7f73a01980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:47.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:47.199+0000 7f73a871d700 1 --2- 172.21.15.144:0/3632143970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73a0073980 0x7f73a01985e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:47.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:47.199+0000 7f73a871d700 1 -- 172.21.15.144:0/3632143970 >> 172.21.15.144:0/3632143970 conn(0x7f73a00fbc90 msgr2=0x7f73a0106d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:47.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:47.199+0000 7f73a871d700 1 -- 172.21.15.144:0/3632143970 shutdown_connections 2024-09-10T11:32:47.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:47.199+0000 7f73a871d700 1 -- 172.21.15.144:0/3632143970 wait complete. 2024-09-10T11:32:47.874 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-10T11:32:47.874 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T11:32:47.875 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-09-10T11:32:48.076 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:32:48.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:48 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/3632143970' entity='client.admin' 2024-09-10T11:32:48.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:48 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:32:48.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:48 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:32:48.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:48 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:32:48.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:48 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:48.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:48 smithi144 ceph-mon[27942]: pgmap v80: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:48 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/3632143970' entity='client.admin' 2024-09-10T11:32:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:48 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:32:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:48 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:32:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:48 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:32:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:48 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:48.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:48 smithi178 ceph-mon[33514]: pgmap v80: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:49.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.852+0000 7f47e2eb5700 1 -- 172.21.15.144:0/843240648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ffc60 msgr2=0x7f47dc103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:49.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.852+0000 7f47e2eb5700 1 --2- 172.21.15.144:0/843240648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ffc60 0x7f47dc103c30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f47c8009910 tx=0x7f47c8009c20 comp rx=0 tx=0).stop 2024-09-10T11:32:49.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.852+0000 7f47e2eb5700 1 -- 172.21.15.144:0/843240648 shutdown_connections 2024-09-10T11:32:49.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.852+0000 7f47e2eb5700 1 --2- 172.21.15.144:0/843240648 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ffc60 0x7f47dc103c30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:49.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.852+0000 7f47e2eb5700 1 --2- 172.21.15.144:0/843240648 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47dc0ff2c0 0x7f47dc0ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:49.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.852+0000 7f47e2eb5700 1 -- 172.21.15.144:0/843240648 >> 172.21.15.144:0/843240648 conn(0x7f47dc0fab80 msgr2=0x7f47dc0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:49.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.853+0000 7f47e2eb5700 1 -- 172.21.15.144:0/843240648 shutdown_connections 2024-09-10T11:32:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.853+0000 7f47e2eb5700 1 -- 172.21.15.144:0/843240648 wait complete. 2024-09-10T11:32:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.853+0000 7f47e2eb5700 1 Processor -- start 2024-09-10T11:32:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.853+0000 7f47e2eb5700 1 -- start start 2024-09-10T11:32:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.853+0000 7f47e2eb5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ff2c0 0x7f47dc195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.853+0000 7f47e2eb5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47dc0ffc60 0x7f47dc1962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.853+0000 7f47e2eb5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc196990 con 0x7f47dc0ff2c0 2024-09-10T11:32:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.853+0000 7f47e2eb5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc19a720 con 0x7f47dc0ffc60 2024-09-10T11:32:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.853+0000 7f47e0c51700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ff2c0 0x7f47dc195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.854+0000 7f47e0c51700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ff2c0 0x7f47dc195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34934/0 (socket says 172.21.15.144:34934) 2024-09-10T11:32:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.854+0000 7f47e0c51700 1 -- 172.21.15.144:0/2643018391 learned_addr learned my addr 172.21.15.144:0/2643018391 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:32:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.854+0000 7f47dbfff700 1 --2- 172.21.15.144:0/2643018391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47dc0ffc60 0x7f47dc1962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.854+0000 7f47e0c51700 1 -- 172.21.15.144:0/2643018391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47dc0ffc60 msgr2=0x7f47dc1962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.854+0000 7f47e0c51700 1 --2- 172.21.15.144:0/2643018391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47dc0ffc60 0x7f47dc1962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:49.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.854+0000 7f47e0c51700 1 -- 172.21.15.144:0/2643018391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c80095f0 con 0x7f47dc0ff2c0 2024-09-10T11:32:49.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.854+0000 7f47e0c51700 1 --2- 172.21.15.144:0/2643018391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ff2c0 0x7f47dc195d70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f47d000d8d0 tx=0x7f47d000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:49.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.854+0000 7f47d9ffb700 1 -- 172.21.15.144:0/2643018391 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d00098a0 con 0x7f47dc0ff2c0 2024-09-10T11:32:49.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.854+0000 7f47d9ffb700 1 -- 172.21.15.144:0/2643018391 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47d0010460 con 0x7f47dc0ff2c0 2024-09-10T11:32:49.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.854+0000 7f47e2eb5700 1 -- 172.21.15.144:0/2643018391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc19aa00 con 0x7f47dc0ff2c0 2024-09-10T11:32:49.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.854+0000 7f47d9ffb700 1 -- 172.21.15.144:0/2643018391 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47d000f5f0 con 0x7f47dc0ff2c0 2024-09-10T11:32:49.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.854+0000 7f47e2eb5700 1 -- 172.21.15.144:0/2643018391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc19af50 con 0x7f47dc0ff2c0 2024-09-10T11:32:49.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.855+0000 7f47e2eb5700 1 -- 172.21.15.144:0/2643018391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc108640 con 0x7f47dc0ff2c0 2024-09-10T11:32:49.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.856+0000 7f47d9ffb700 1 -- 172.21.15.144:0/2643018391 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f47d0009a00 con 0x7f47dc0ff2c0 2024-09-10T11:32:49.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.856+0000 7f47d9ffb700 1 --2- 172.21.15.144:0/2643018391 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f47cc06de80 0x7f47cc070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:49.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.856+0000 7f47d9ffb700 1 -- 172.21.15.144:0/2643018391 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f47d008d8e0 con 0x7f47dc0ff2c0 2024-09-10T11:32:49.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.856+0000 7f47dbfff700 1 --2- 172.21.15.144:0/2643018391 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f47cc06de80 0x7f47cc070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:49.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.857+0000 7f47dbfff700 1 --2- 172.21.15.144:0/2643018391 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f47cc06de80 0x7f47cc070330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f47dc197390 tx=0x7f47c801a040 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:49.862 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:49.861+0000 7f47d9ffb700 1 -- 172.21.15.144:0/2643018391 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f47d0057270 con 0x7f47dc0ff2c0 2024-09-10T11:32:50.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:50.063+0000 7f47e2eb5700 1 -- 172.21.15.144:0/2643018391 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f47dc04ea50 con 0x7f47dc0ff2c0 2024-09-10T11:32:50.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:50.063+0000 7f47d9ffb700 1 -- 172.21.15.144:0/2643018391 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v45)=0 v45) v1 ==== 155+0+0 (secure 0 0 0) 0x7f47d00160e0 con 0x7f47dc0ff2c0 2024-09-10T11:32:50.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:50.066+0000 7f47e2eb5700 1 -- 172.21.15.144:0/2643018391 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f47cc06de80 msgr2=0x7f47cc070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:50.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:50.066+0000 7f47e2eb5700 1 --2- 172.21.15.144:0/2643018391 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f47cc06de80 0x7f47cc070330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f47dc197390 tx=0x7f47c801a040 comp rx=0 tx=0).stop 2024-09-10T11:32:50.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:50.066+0000 7f47e2eb5700 1 -- 172.21.15.144:0/2643018391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ff2c0 msgr2=0x7f47dc195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:50.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:50.066+0000 7f47e2eb5700 1 --2- 172.21.15.144:0/2643018391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ff2c0 0x7f47dc195d70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f47d000d8d0 tx=0x7f47d000dbe0 comp rx=0 tx=0).stop 2024-09-10T11:32:50.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:50.066+0000 7f47e2eb5700 1 -- 172.21.15.144:0/2643018391 shutdown_connections 2024-09-10T11:32:50.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:50.066+0000 7f47e2eb5700 1 --2- 172.21.15.144:0/2643018391 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f47cc06de80 0x7f47cc070330 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:50.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:50.066+0000 7f47e2eb5700 1 --2- 172.21.15.144:0/2643018391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ff2c0 0x7f47dc195d70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:50.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:50.066+0000 7f47e2eb5700 1 --2- 172.21.15.144:0/2643018391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47dc0ffc60 0x7f47dc1962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:50.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:50.066+0000 7f47e2eb5700 1 -- 172.21.15.144:0/2643018391 >> 172.21.15.144:0/2643018391 conn(0x7f47dc0fab80 msgr2=0x7f47dc068fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:50.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:50.066+0000 7f47e2eb5700 1 -- 172.21.15.144:0/2643018391 shutdown_connections 2024-09-10T11:32:50.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:50.066+0000 7f47e2eb5700 1 -- 172.21.15.144:0/2643018391 wait complete. 2024-09-10T11:32:50.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:50 smithi144 ceph-mon[27942]: pgmap v81: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:50.831 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-09-10T11:32:50.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:50 smithi178 ceph-mon[33514]: pgmap v81: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:51.035 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:32:52.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:52 smithi144 ceph-mon[27942]: pgmap v82: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:32:52.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.718+0000 7f8c5c30a700 1 -- 172.21.15.144:0/2683944172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c541003a0 msgr2=0x7f8c54100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:52.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.718+0000 7f8c5c30a700 1 --2- 172.21.15.144:0/2683944172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c541003a0 0x7f8c54100770 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f8c44009940 tx=0x7f8c44009c50 comp rx=0 tx=0).stop 2024-09-10T11:32:52.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.718+0000 7f8c5c30a700 1 -- 172.21.15.144:0/2683944172 shutdown_connections 2024-09-10T11:32:52.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.718+0000 7f8c5c30a700 1 --2- 172.21.15.144:0/2683944172 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c54100cb0 0x7f8c54109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:52.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.718+0000 7f8c5c30a700 1 --2- 172.21.15.144:0/2683944172 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c541003a0 0x7f8c54100770 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:52.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.718+0000 7f8c5c30a700 1 -- 172.21.15.144:0/2683944172 >> 172.21.15.144:0/2683944172 conn(0x7f8c54074850 msgr2=0x7f8c54074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:52.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.718+0000 7f8c5c30a700 1 -- 172.21.15.144:0/2683944172 shutdown_connections 2024-09-10T11:32:52.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.718+0000 7f8c5c30a700 1 -- 172.21.15.144:0/2683944172 wait complete. 2024-09-10T11:32:52.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.718+0000 7f8c5c30a700 1 Processor -- start 2024-09-10T11:32:52.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.718+0000 7f8c5c30a700 1 -- start start 2024-09-10T11:32:52.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c5c30a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c541003a0 0x7f8c54196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:52.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c5c30a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c54100cb0 0x7f8c541971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:52.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c5c30a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c54197810 con 0x7f8c54100cb0 2024-09-10T11:32:52.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c5c30a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c54197980 con 0x7f8c541003a0 2024-09-10T11:32:52.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c598a5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c54100cb0 0x7f8c541971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:52.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c598a5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c54100cb0 0x7f8c541971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35408/0 (socket says 172.21.15.144:35408) 2024-09-10T11:32:52.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c598a5700 1 -- 172.21.15.144:0/3373107789 learned_addr learned my addr 172.21.15.144:0/3373107789 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:32:52.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c5a0a6700 1 --2- 172.21.15.144:0/3373107789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c541003a0 0x7f8c54196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:52.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c598a5700 1 -- 172.21.15.144:0/3373107789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c541003a0 msgr2=0x7f8c54196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:52.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c598a5700 1 --2- 172.21.15.144:0/3373107789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c541003a0 0x7f8c54196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:52.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c598a5700 1 -- 172.21.15.144:0/3373107789 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c440095f0 con 0x7f8c54100cb0 2024-09-10T11:32:52.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c5a0a6700 1 --2- 172.21.15.144:0/3373107789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c541003a0 0x7f8c54196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:32:52.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c598a5700 1 --2- 172.21.15.144:0/3373107789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c54100cb0 0x7f8c541971f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800b4b0 tx=0x7f8c4800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:52.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c537fe700 1 -- 172.21.15.144:0/3373107789 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c48011800 con 0x7f8c54100cb0 2024-09-10T11:32:52.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c537fe700 1 -- 172.21.15.144:0/3373107789 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c48011e20 con 0x7f8c54100cb0 2024-09-10T11:32:52.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c5c30a700 1 -- 172.21.15.144:0/3373107789 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5419fd20 con 0x7f8c54100cb0 2024-09-10T11:32:52.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.719+0000 7f8c537fe700 1 -- 172.21.15.144:0/3373107789 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c4800f2e0 con 0x7f8c54100cb0 2024-09-10T11:32:52.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.720+0000 7f8c5c30a700 1 -- 172.21.15.144:0/3373107789 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c541a0210 con 0x7f8c54100cb0 2024-09-10T11:32:52.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.721+0000 7f8c5c30a700 1 -- 172.21.15.144:0/3373107789 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c54106b10 con 0x7f8c54100cb0 2024-09-10T11:32:52.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.721+0000 7f8c537fe700 1 -- 172.21.15.144:0/3373107789 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f8c48010380 con 0x7f8c54100cb0 2024-09-10T11:32:52.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.722+0000 7f8c537fe700 1 --2- 172.21.15.144:0/3373107789 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f8c3c06de80 0x7f8c3c070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:52.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.722+0000 7f8c537fe700 1 -- 172.21.15.144:0/3373107789 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f8c4808d5c0 con 0x7f8c54100cb0 2024-09-10T11:32:52.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.722+0000 7f8c5a0a6700 1 --2- 172.21.15.144:0/3373107789 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f8c3c06de80 0x7f8c3c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:52.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.722+0000 7f8c5a0a6700 1 --2- 172.21.15.144:0/3373107789 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f8c3c06de80 0x7f8c3c070330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400b870 tx=0x7f8c44005c00 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:52.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.726+0000 7f8c537fe700 1 -- 172.21.15.144:0/3373107789 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f8c48056f50 con 0x7f8c54100cb0 2024-09-10T11:32:52.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:52 smithi178 ceph-mon[33514]: pgmap v82: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T11:32:52.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.924+0000 7f8c5c30a700 1 -- 172.21.15.144:0/3373107789 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f8c5402cc30 con 0x7f8c54100cb0 2024-09-10T11:32:52.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.925+0000 7f8c537fe700 1 -- 172.21.15.144:0/3373107789 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v45)=0 v45) v1 ==== 163+0+0 (secure 0 0 0) 0x7f8c4805a610 con 0x7f8c54100cb0 2024-09-10T11:32:52.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.927+0000 7f8c5c30a700 1 -- 172.21.15.144:0/3373107789 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f8c3c06de80 msgr2=0x7f8c3c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:52.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.927+0000 7f8c5c30a700 1 --2- 172.21.15.144:0/3373107789 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f8c3c06de80 0x7f8c3c070330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8c4400b870 tx=0x7f8c44005c00 comp rx=0 tx=0).stop 2024-09-10T11:32:52.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.927+0000 7f8c5c30a700 1 -- 172.21.15.144:0/3373107789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c54100cb0 msgr2=0x7f8c541971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:52.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.927+0000 7f8c5c30a700 1 --2- 172.21.15.144:0/3373107789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c54100cb0 0x7f8c541971f0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8c4800b4b0 tx=0x7f8c4800b870 comp rx=0 tx=0).stop 2024-09-10T11:32:52.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.927+0000 7f8c5c30a700 1 -- 172.21.15.144:0/3373107789 shutdown_connections 2024-09-10T11:32:52.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.927+0000 7f8c5c30a700 1 --2- 172.21.15.144:0/3373107789 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f8c3c06de80 0x7f8c3c070330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:52.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.927+0000 7f8c5c30a700 1 --2- 172.21.15.144:0/3373107789 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8c541003a0 0x7f8c54196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:52.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.927+0000 7f8c5c30a700 1 --2- 172.21.15.144:0/3373107789 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8c54100cb0 0x7f8c541971f0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:52.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.927+0000 7f8c5c30a700 1 -- 172.21.15.144:0/3373107789 >> 172.21.15.144:0/3373107789 conn(0x7f8c54074850 msgr2=0x7f8c54103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:52.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.927+0000 7f8c5c30a700 1 -- 172.21.15.144:0/3373107789 shutdown_connections 2024-09-10T11:32:52.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:52.927+0000 7f8c5c30a700 1 -- 172.21.15.144:0/3373107789 wait complete. 2024-09-10T11:32:53.587 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph config set global log_to_journald false --force' 2024-09-10T11:32:53.786 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:32:54.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:54 smithi178 ceph-mon[33514]: pgmap v83: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:54.881 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:54 smithi144 ceph-mon[27942]: pgmap v83: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:55.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.439+0000 7fc7ef65d700 1 -- 172.21.15.144:0/2075926601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7e8100d50 msgr2=0x7fc7e8104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:55.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.439+0000 7fc7ef65d700 1 --2- 172.21.15.144:0/2075926601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7e8100d50 0x7fc7e8104d20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc7d8009910 tx=0x7fc7d8009c20 comp rx=0 tx=0).stop 2024-09-10T11:32:55.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.439+0000 7fc7ef65d700 1 -- 172.21.15.144:0/2075926601 shutdown_connections 2024-09-10T11:32:55.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.439+0000 7fc7ef65d700 1 --2- 172.21.15.144:0/2075926601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7e8100d50 0x7fc7e8104d20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:55.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.439+0000 7fc7ef65d700 1 --2- 172.21.15.144:0/2075926601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7e81003b0 0x7fc7e8100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:55.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.439+0000 7fc7ef65d700 1 -- 172.21.15.144:0/2075926601 >> 172.21.15.144:0/2075926601 conn(0x7fc7e80fbc30 msgr2=0x7fc7e80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:55.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.439+0000 7fc7ef65d700 1 -- 172.21.15.144:0/2075926601 shutdown_connections 2024-09-10T11:32:55.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.440+0000 7fc7ef65d700 1 -- 172.21.15.144:0/2075926601 wait complete. 2024-09-10T11:32:55.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.440+0000 7fc7ef65d700 1 Processor -- start 2024-09-10T11:32:55.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.440+0000 7fc7ef65d700 1 -- start start 2024-09-10T11:32:55.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.440+0000 7fc7ef65d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7e81003b0 0x7fc7e8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:55.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.440+0000 7fc7ef65d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7e8100d50 0x7fc7e8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:55.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.440+0000 7fc7ef65d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e8073ad0 con 0x7fc7e81003b0 2024-09-10T11:32:55.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.440+0000 7fc7ef65d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7e8073c10 con 0x7fc7e8100d50 2024-09-10T11:32:55.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.440+0000 7fc7ed3f9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7e81003b0 0x7fc7e8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:55.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.441+0000 7fc7ed3f9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7e81003b0 0x7fc7e8073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35418/0 (socket says 172.21.15.144:35418) 2024-09-10T11:32:55.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.441+0000 7fc7ed3f9700 1 -- 172.21.15.144:0/2007959969 learned_addr learned my addr 172.21.15.144:0/2007959969 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:32:55.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.441+0000 7fc7ecbf8700 1 --2- 172.21.15.144:0/2007959969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7e8100d50 0x7fc7e8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:55.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.441+0000 7fc7ed3f9700 1 -- 172.21.15.144:0/2007959969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7e8100d50 msgr2=0x7fc7e8073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:55.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.441+0000 7fc7ed3f9700 1 --2- 172.21.15.144:0/2007959969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7e8100d50 0x7fc7e8073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:55.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.441+0000 7fc7ed3f9700 1 -- 172.21.15.144:0/2007959969 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d80095f0 con 0x7fc7e81003b0 2024-09-10T11:32:55.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.441+0000 7fc7ed3f9700 1 --2- 172.21.15.144:0/2007959969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7e81003b0 0x7fc7e8073050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400ba50 tx=0x7fc7e400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:55.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.441+0000 7fc7de7fc700 1 -- 172.21.15.144:0/2007959969 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7e400c900 con 0x7fc7e81003b0 2024-09-10T11:32:55.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.441+0000 7fc7ef65d700 1 -- 172.21.15.144:0/2007959969 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7e81a2090 con 0x7fc7e81003b0 2024-09-10T11:32:55.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.441+0000 7fc7de7fc700 1 -- 172.21.15.144:0/2007959969 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7e4014440 con 0x7fc7e81003b0 2024-09-10T11:32:55.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.441+0000 7fc7de7fc700 1 -- 172.21.15.144:0/2007959969 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7e40075a0 con 0x7fc7e81003b0 2024-09-10T11:32:55.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.441+0000 7fc7ef65d700 1 -- 172.21.15.144:0/2007959969 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e81a24a0 con 0x7fc7e81003b0 2024-09-10T11:32:55.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.443+0000 7fc7ef65d700 1 -- 172.21.15.144:0/2007959969 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7e8108670 con 0x7fc7e81003b0 2024-09-10T11:32:55.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.443+0000 7fc7de7fc700 1 -- 172.21.15.144:0/2007959969 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7fc7e4007810 con 0x7fc7e81003b0 2024-09-10T11:32:55.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.443+0000 7fc7de7fc700 1 --2- 172.21.15.144:0/2007959969 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fc7d406de80 0x7fc7d4070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:55.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.443+0000 7fc7de7fc700 1 -- 172.21.15.144:0/2007959969 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fc7e4006e40 con 0x7fc7e81003b0 2024-09-10T11:32:55.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.443+0000 7fc7ecbf8700 1 --2- 172.21.15.144:0/2007959969 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fc7d406de80 0x7fc7d4070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:55.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.444+0000 7fc7ecbf8700 1 --2- 172.21.15.144:0/2007959969 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fc7d406de80 0x7fc7d4070330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8074840 tx=0x7fc7d801a040 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:55.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.448+0000 7fc7de7fc700 1 -- 172.21.15.144:0/2007959969 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7fc7e4056bb0 con 0x7fc7e81003b0 2024-09-10T11:32:55.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.647+0000 7fc7ef65d700 1 -- 172.21.15.144:0/2007959969 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fc7e804ea50 con 0x7fc7e81003b0 2024-09-10T11:32:55.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.648+0000 7fc7de7fc700 1 -- 172.21.15.144:0/2007959969 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v45)=0 v45) v1 ==== 135+0+0 (secure 0 0 0) 0x7fc7e405a270 con 0x7fc7e81003b0 2024-09-10T11:32:55.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.650+0000 7fc7ef65d700 1 -- 172.21.15.144:0/2007959969 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fc7d406de80 msgr2=0x7fc7d4070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:55.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.650+0000 7fc7ef65d700 1 --2- 172.21.15.144:0/2007959969 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fc7d406de80 0x7fc7d4070330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8074840 tx=0x7fc7d801a040 comp rx=0 tx=0).stop 2024-09-10T11:32:55.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.650+0000 7fc7ef65d700 1 -- 172.21.15.144:0/2007959969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7e81003b0 msgr2=0x7fc7e8073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:55.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.650+0000 7fc7ef65d700 1 --2- 172.21.15.144:0/2007959969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7e81003b0 0x7fc7e8073050 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400ba50 tx=0x7fc7e400bd60 comp rx=0 tx=0).stop 2024-09-10T11:32:55.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.650+0000 7fc7ef65d700 1 -- 172.21.15.144:0/2007959969 shutdown_connections 2024-09-10T11:32:55.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.650+0000 7fc7ef65d700 1 --2- 172.21.15.144:0/2007959969 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fc7d406de80 0x7fc7d4070330 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:55.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.650+0000 7fc7ef65d700 1 --2- 172.21.15.144:0/2007959969 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc7e81003b0 0x7fc7e8073050 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:55.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.650+0000 7fc7ef65d700 1 --2- 172.21.15.144:0/2007959969 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc7e8100d50 0x7fc7e8073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:55.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.650+0000 7fc7ef65d700 1 -- 172.21.15.144:0/2007959969 >> 172.21.15.144:0/2007959969 conn(0x7fc7e80fbc30 msgr2=0x7fc7e80fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:55.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.650+0000 7fc7ef65d700 1 -- 172.21.15.144:0/2007959969 shutdown_connections 2024-09-10T11:32:55.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:55.650+0000 7fc7ef65d700 1 -- 172.21.15.144:0/2007959969 wait complete. 2024-09-10T11:32:56.345 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-09-10T11:32:56.545 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:32:56.665 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:56 smithi144 ceph-mon[27942]: pgmap v84: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:56.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:56 smithi178 ceph-mon[33514]: pgmap v84: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:32:58.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.205+0000 7f4d35cd4700 1 -- 172.21.15.144:0/1742354781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d30100400 msgr2=0x7f4d301007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:58.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.205+0000 7f4d35cd4700 1 --2- 172.21.15.144:0/1742354781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d30100400 0x7f4d301007d0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4d20009910 tx=0x7f4d20009c20 comp rx=0 tx=0).stop 2024-09-10T11:32:58.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.205+0000 7f4d35cd4700 1 -- 172.21.15.144:0/1742354781 shutdown_connections 2024-09-10T11:32:58.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.205+0000 7f4d35cd4700 1 --2- 172.21.15.144:0/1742354781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d30100da0 0x7f4d30104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:58.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.205+0000 7f4d35cd4700 1 --2- 172.21.15.144:0/1742354781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d30100400 0x7f4d301007d0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:58.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.205+0000 7f4d35cd4700 1 -- 172.21.15.144:0/1742354781 >> 172.21.15.144:0/1742354781 conn(0x7f4d300fbca0 msgr2=0x7f4d300fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:58.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.205+0000 7f4d35cd4700 1 -- 172.21.15.144:0/1742354781 shutdown_connections 2024-09-10T11:32:58.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.205+0000 7f4d35cd4700 1 -- 172.21.15.144:0/1742354781 wait complete. 2024-09-10T11:32:58.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.206+0000 7f4d35cd4700 1 Processor -- start 2024-09-10T11:32:58.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.206+0000 7f4d35cd4700 1 -- start start 2024-09-10T11:32:58.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.206+0000 7f4d35cd4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d30100400 0x7f4d30195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:58.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.206+0000 7f4d35cd4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d30100da0 0x7f4d30196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:58.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.206+0000 7f4d35cd4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d30196a60 con 0x7f4d30100400 2024-09-10T11:32:58.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.206+0000 7f4d35cd4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3019a7f0 con 0x7f4d30100da0 2024-09-10T11:32:58.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.206+0000 7f4d2f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d30100400 0x7f4d30195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:58.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.206+0000 7f4d2f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d30100400 0x7f4d30195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35442/0 (socket says 172.21.15.144:35442) 2024-09-10T11:32:58.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.206+0000 7f4d2f7fe700 1 -- 172.21.15.144:0/2500424560 learned_addr learned my addr 172.21.15.144:0/2500424560 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:32:58.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.207+0000 7f4d2f7fe700 1 -- 172.21.15.144:0/2500424560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d30100da0 msgr2=0x7f4d30196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:58.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.207+0000 7f4d2f7fe700 1 --2- 172.21.15.144:0/2500424560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d30100da0 0x7f4d30196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:58.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.207+0000 7f4d2f7fe700 1 -- 172.21.15.144:0/2500424560 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d200095f0 con 0x7f4d30100400 2024-09-10T11:32:58.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.207+0000 7f4d2f7fe700 1 --2- 172.21.15.144:0/2500424560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d30100400 0x7f4d30195e40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4d20006010 tx=0x7f4d2000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:58.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.207+0000 7f4d2cff9700 1 -- 172.21.15.144:0/2500424560 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2001d070 con 0x7f4d30100400 2024-09-10T11:32:58.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.207+0000 7f4d2cff9700 1 -- 172.21.15.144:0/2500424560 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d20004fb0 con 0x7f4d30100400 2024-09-10T11:32:58.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.207+0000 7f4d2cff9700 1 -- 172.21.15.144:0/2500424560 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d20022ea0 con 0x7f4d30100400 2024-09-10T11:32:58.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.207+0000 7f4d35cd4700 1 -- 172.21.15.144:0/2500424560 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3019aa70 con 0x7f4d30100400 2024-09-10T11:32:58.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.207+0000 7f4d35cd4700 1 -- 172.21.15.144:0/2500424560 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3019af60 con 0x7f4d30100400 2024-09-10T11:32:58.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.208+0000 7f4d35cd4700 1 -- 172.21.15.144:0/2500424560 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d30108710 con 0x7f4d30100400 2024-09-10T11:32:58.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.208+0000 7f4d2cff9700 1 -- 172.21.15.144:0/2500424560 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f4d20022470 con 0x7f4d30100400 2024-09-10T11:32:58.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.209+0000 7f4d2cff9700 1 --2- 172.21.15.144:0/2500424560 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f4d1806de80 0x7f4d18070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:32:58.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.209+0000 7f4d2cff9700 1 -- 172.21.15.144:0/2500424560 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f4d2008f050 con 0x7f4d30100400 2024-09-10T11:32:58.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.209+0000 7f4d2effd700 1 --2- 172.21.15.144:0/2500424560 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f4d1806de80 0x7f4d18070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:32:58.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.209+0000 7f4d2effd700 1 --2- 172.21.15.144:0/2500424560 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f4d1806de80 0x7f4d18070330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d30197460 tx=0x7f4d24006d20 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:32:58.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.214+0000 7f4d2cff9700 1 -- 172.21.15.144:0/2500424560 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f4d20058a10 con 0x7f4d30100400 2024-09-10T11:32:58.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.432+0000 7f4d35cd4700 1 -- 172.21.15.144:0/2500424560 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d3002cd20 con 0x7f4d1806de80 2024-09-10T11:32:58.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.439+0000 7f4d2cff9700 1 -- 172.21.15.144:0/2500424560 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f4d3002cd20 con 0x7f4d1806de80 2024-09-10T11:32:58.440 INFO:teuthology.orchestra.run.smithi144.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:32:58.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.441+0000 7f4d35cd4700 1 -- 172.21.15.144:0/2500424560 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f4d1806de80 msgr2=0x7f4d18070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:58.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.441+0000 7f4d35cd4700 1 --2- 172.21.15.144:0/2500424560 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f4d1806de80 0x7f4d18070330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4d30197460 tx=0x7f4d24006d20 comp rx=0 tx=0).stop 2024-09-10T11:32:58.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.441+0000 7f4d35cd4700 1 -- 172.21.15.144:0/2500424560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d30100400 msgr2=0x7f4d30195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:32:58.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.441+0000 7f4d35cd4700 1 --2- 172.21.15.144:0/2500424560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d30100400 0x7f4d30195e40 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4d20006010 tx=0x7f4d2000be20 comp rx=0 tx=0).stop 2024-09-10T11:32:58.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.441+0000 7f4d35cd4700 1 -- 172.21.15.144:0/2500424560 shutdown_connections 2024-09-10T11:32:58.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.441+0000 7f4d35cd4700 1 --2- 172.21.15.144:0/2500424560 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f4d1806de80 0x7f4d18070330 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:58.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.441+0000 7f4d35cd4700 1 --2- 172.21.15.144:0/2500424560 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d30100400 0x7f4d30195e40 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:58.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.441+0000 7f4d35cd4700 1 --2- 172.21.15.144:0/2500424560 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d30100da0 0x7f4d30196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:32:58.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.441+0000 7f4d35cd4700 1 -- 172.21.15.144:0/2500424560 >> 172.21.15.144:0/2500424560 conn(0x7f4d300fbca0 msgr2=0x7f4d300fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:32:58.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.441+0000 7f4d35cd4700 1 -- 172.21.15.144:0/2500424560 shutdown_connections 2024-09-10T11:32:58.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:32:58.441+0000 7f4d35cd4700 1 -- 172.21.15.144:0/2500424560 wait complete. 2024-09-10T11:32:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:58 smithi144 ceph-mon[27942]: pgmap v85: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:58 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:58 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:32:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:58 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:32:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:58 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:32:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:58 smithi178 ceph-mon[33514]: pgmap v85: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:32:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:58 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:58 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:32:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:58 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:32:58.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:58 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:32:59.168 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-09-10T11:32:59.168 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T11:32:59.169 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-09-10T11:32:59.394 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T11:32:59.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:59 smithi178 ceph-mon[33514]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:32:59.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:59 smithi178 ceph-mon[33514]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:32:59.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:59 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:59.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:59 smithi178 ceph-mon[33514]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:32:59.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:32:59 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:32:59.736 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:59 smithi144 ceph-mon[27942]: from='client.14812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:32:59.737 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:59 smithi144 ceph-mon[27942]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:32:59.737 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:59 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:32:59.737 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:59 smithi144 ceph-mon[27942]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:1cf819b7212a6048e92fc78c4eff963344f6d56b 2024-09-10T11:32:59.737 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:32:59 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:33:00.753 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:00 smithi144 ceph-mon[27942]: pgmap v86: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:00.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:00 smithi178 ceph-mon[33514]: pgmap v86: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:02.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.323+0000 7f2acbb78700 1 -- 172.21.15.144:0/3206539987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ac40ff390 msgr2=0x7f2ac40ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:02.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.323+0000 7f2acbb78700 1 --2- 172.21.15.144:0/3206539987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ac40ff390 0x7f2ac40ff760 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f2ac0009910 tx=0x7f2ac0009c20 comp rx=0 tx=0).stop 2024-09-10T11:33:02.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.323+0000 7f2acbb78700 1 -- 172.21.15.144:0/3206539987 shutdown_connections 2024-09-10T11:33:02.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.323+0000 7f2acbb78700 1 --2- 172.21.15.144:0/3206539987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ac40ffd30 0x7f2ac4103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:02.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.323+0000 7f2acbb78700 1 --2- 172.21.15.144:0/3206539987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ac40ff390 0x7f2ac40ff760 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:02.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.323+0000 7f2acbb78700 1 -- 172.21.15.144:0/3206539987 >> 172.21.15.144:0/3206539987 conn(0x7f2ac4074f50 msgr2=0x7f2ac4075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:02.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.323+0000 7f2acbb78700 1 -- 172.21.15.144:0/3206539987 shutdown_connections 2024-09-10T11:33:02.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.323+0000 7f2acbb78700 1 -- 172.21.15.144:0/3206539987 wait complete. 2024-09-10T11:33:02.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.323+0000 7f2acbb78700 1 Processor -- start 2024-09-10T11:33:02.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.323+0000 7f2acbb78700 1 -- start start 2024-09-10T11:33:02.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.324+0000 7f2acbb78700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ac40ff390 0x7f2ac4196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:02.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.324+0000 7f2acbb78700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ac40ffd30 0x7f2ac41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:02.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.324+0000 7f2acbb78700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac4197810 con 0x7f2ac40ffd30 2024-09-10T11:33:02.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.324+0000 7f2acbb78700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ac4197950 con 0x7f2ac40ff390 2024-09-10T11:33:02.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.324+0000 7f2ac9113700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ac40ffd30 0x7f2ac41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:02.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.324+0000 7f2ac9113700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ac40ffd30 0x7f2ac41971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44490/0 (socket says 172.21.15.144:44490) 2024-09-10T11:33:02.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.324+0000 7f2ac9113700 1 -- 172.21.15.144:0/3643708773 learned_addr learned my addr 172.21.15.144:0/3643708773 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:33:02.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.324+0000 7f2ac9914700 1 --2- 172.21.15.144:0/3643708773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ac40ff390 0x7f2ac4196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:02.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.324+0000 7f2ac9113700 1 -- 172.21.15.144:0/3643708773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ac40ff390 msgr2=0x7f2ac4196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:02.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.324+0000 7f2ac9113700 1 --2- 172.21.15.144:0/3643708773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ac40ff390 0x7f2ac4196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:02.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.324+0000 7f2ac9113700 1 -- 172.21.15.144:0/3643708773 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ac00095f0 con 0x7f2ac40ffd30 2024-09-10T11:33:02.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.324+0000 7f2ac9113700 1 --2- 172.21.15.144:0/3643708773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ac40ffd30 0x7f2ac41971f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400b480 tx=0x7f2ab400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:02.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.324+0000 7f2abaffd700 1 -- 172.21.15.144:0/3643708773 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ab40118c0 con 0x7f2ac40ffd30 2024-09-10T11:33:02.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.324+0000 7f2abaffd700 1 -- 172.21.15.144:0/3643708773 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ab4010380 con 0x7f2ac40ffd30 2024-09-10T11:33:02.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.324+0000 7f2acbb78700 1 -- 172.21.15.144:0/3643708773 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ac419fd20 con 0x7f2ac40ffd30 2024-09-10T11:33:02.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.325+0000 7f2abaffd700 1 -- 172.21.15.144:0/3643708773 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ab400f4f0 con 0x7f2ac40ffd30 2024-09-10T11:33:02.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.325+0000 7f2acbb78700 1 -- 172.21.15.144:0/3643708773 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ac41a01e0 con 0x7f2ac40ffd30 2024-09-10T11:33:02.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.325+0000 7f2acbb78700 1 -- 172.21.15.144:0/3643708773 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ac4107650 con 0x7f2ac40ffd30 2024-09-10T11:33:02.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.326+0000 7f2abaffd700 1 -- 172.21.15.144:0/3643708773 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f2ab40104f0 con 0x7f2ac40ffd30 2024-09-10T11:33:02.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.327+0000 7f2abaffd700 1 --2- 172.21.15.144:0/3643708773 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f2ab006de80 0x7f2ab0070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:02.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.327+0000 7f2abaffd700 1 -- 172.21.15.144:0/3643708773 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f2ab408d790 con 0x7f2ac40ffd30 2024-09-10T11:33:02.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.327+0000 7f2ac9914700 1 --2- 172.21.15.144:0/3643708773 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f2ab006de80 0x7f2ab0070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:02.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.327+0000 7f2ac9914700 1 --2- 172.21.15.144:0/3643708773 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f2ab006de80 0x7f2ab0070330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000b5c0 tx=0x7f2ac001a040 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:02.332 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.331+0000 7f2abaffd700 1 -- 172.21.15.144:0/3643708773 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f2ab4057000 con 0x7f2ac40ffd30 2024-09-10T11:33:02.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.564+0000 7f2acbb78700 1 -- 172.21.15.144:0/3643708773 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ac402cd10 con 0x7f2ab006de80 2024-09-10T11:33:02.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.565+0000 7f2abaffd700 1 -- 172.21.15.144:0/3643708773 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f2ac402cd10 con 0x7f2ab006de80 2024-09-10T11:33:02.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.567+0000 7f2acbb78700 1 -- 172.21.15.144:0/3643708773 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f2ab006de80 msgr2=0x7f2ab0070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:02.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.567+0000 7f2acbb78700 1 --2- 172.21.15.144:0/3643708773 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f2ab006de80 0x7f2ab0070330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000b5c0 tx=0x7f2ac001a040 comp rx=0 tx=0).stop 2024-09-10T11:33:02.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.567+0000 7f2acbb78700 1 -- 172.21.15.144:0/3643708773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ac40ffd30 msgr2=0x7f2ac41971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:02.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.567+0000 7f2acbb78700 1 --2- 172.21.15.144:0/3643708773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ac40ffd30 0x7f2ac41971f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2ab400b480 tx=0x7f2ab400b840 comp rx=0 tx=0).stop 2024-09-10T11:33:02.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.567+0000 7f2acbb78700 1 -- 172.21.15.144:0/3643708773 shutdown_connections 2024-09-10T11:33:02.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.567+0000 7f2acbb78700 1 --2- 172.21.15.144:0/3643708773 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f2ab006de80 0x7f2ab0070330 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:02.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.567+0000 7f2acbb78700 1 --2- 172.21.15.144:0/3643708773 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2ac40ff390 0x7f2ac4196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:02.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.567+0000 7f2acbb78700 1 --2- 172.21.15.144:0/3643708773 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2ac40ffd30 0x7f2ac41971f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:02.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.567+0000 7f2acbb78700 1 -- 172.21.15.144:0/3643708773 >> 172.21.15.144:0/3643708773 conn(0x7f2ac4074f50 msgr2=0x7f2ac40fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:02.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.568+0000 7f2acbb78700 1 -- 172.21.15.144:0/3643708773 shutdown_connections 2024-09-10T11:33:02.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.568+0000 7f2acbb78700 1 -- 172.21.15.144:0/3643708773 wait complete. 2024-09-10T11:33:02.580 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:33:02.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.677+0000 7febc6019700 1 -- 172.21.15.144:0/4127061958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc0100e00 msgr2=0x7febc0104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:02.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.677+0000 7febc6019700 1 --2- 172.21.15.144:0/4127061958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc0100e00 0x7febc0104dd0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7febb4009910 tx=0x7febb4009c20 comp rx=0 tx=0).stop 2024-09-10T11:33:02.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.677+0000 7febc6019700 1 -- 172.21.15.144:0/4127061958 shutdown_connections 2024-09-10T11:33:02.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.677+0000 7febc6019700 1 --2- 172.21.15.144:0/4127061958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc0100e00 0x7febc0104dd0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:02.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.677+0000 7febc6019700 1 --2- 172.21.15.144:0/4127061958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febc0100460 0x7febc0100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:02.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.677+0000 7febc6019700 1 -- 172.21.15.144:0/4127061958 >> 172.21.15.144:0/4127061958 conn(0x7febc00fbd00 msgr2=0x7febc00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:02.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.677+0000 7febc6019700 1 -- 172.21.15.144:0/4127061958 shutdown_connections 2024-09-10T11:33:02.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.677+0000 7febc6019700 1 -- 172.21.15.144:0/4127061958 wait complete. 2024-09-10T11:33:02.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.677+0000 7febc6019700 1 Processor -- start 2024-09-10T11:33:02.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.678+0000 7febc6019700 1 -- start start 2024-09-10T11:33:02.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.678+0000 7febc6019700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febc0100460 0x7febc0195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:02.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.678+0000 7febc6019700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc0100e00 0x7febc0196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:02.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.678+0000 7febc6019700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc0196ae0 con 0x7febc0100e00 2024-09-10T11:33:02.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.678+0000 7febc6019700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc019a870 con 0x7febc0100460 2024-09-10T11:33:02.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.678+0000 7febbeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc0100e00 0x7febc0196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:02.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.678+0000 7febbeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc0100e00 0x7febc0196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44508/0 (socket says 172.21.15.144:44508) 2024-09-10T11:33:02.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.678+0000 7febbeffd700 1 -- 172.21.15.144:0/2537887901 learned_addr learned my addr 172.21.15.144:0/2537887901 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:33:02.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.678+0000 7febbf7fe700 1 --2- 172.21.15.144:0/2537887901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febc0100460 0x7febc0195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:02.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.678+0000 7febbeffd700 1 -- 172.21.15.144:0/2537887901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febc0100460 msgr2=0x7febc0195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:02.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.678+0000 7febbeffd700 1 --2- 172.21.15.144:0/2537887901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febc0100460 0x7febc0195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:02.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.678+0000 7febbeffd700 1 -- 172.21.15.144:0/2537887901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febb40095f0 con 0x7febc0100e00 2024-09-10T11:33:02.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.678+0000 7febbf7fe700 1 --2- 172.21.15.144:0/2537887901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febc0100460 0x7febc0195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:33:02.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.679+0000 7febbeffd700 1 --2- 172.21.15.144:0/2537887901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc0100e00 0x7febc0196400 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7febb40092e0 tx=0x7febb400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:02.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.679+0000 7febbcff9700 1 -- 172.21.15.144:0/2537887901 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febb401d070 con 0x7febc0100e00 2024-09-10T11:33:02.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.679+0000 7febbcff9700 1 -- 172.21.15.144:0/2537887901 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7febb400f460 con 0x7febc0100e00 2024-09-10T11:33:02.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.679+0000 7febc6019700 1 -- 172.21.15.144:0/2537887901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc019aaf0 con 0x7febc0100e00 2024-09-10T11:33:02.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.679+0000 7febbcff9700 1 -- 172.21.15.144:0/2537887901 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febb40175f0 con 0x7febc0100e00 2024-09-10T11:33:02.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.679+0000 7febc6019700 1 -- 172.21.15.144:0/2537887901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc019afe0 con 0x7febc0100e00 2024-09-10T11:33:02.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.680+0000 7febc6019700 1 -- 172.21.15.144:0/2537887901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc0108720 con 0x7febc0100e00 2024-09-10T11:33:02.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.680+0000 7febbcff9700 1 -- 172.21.15.144:0/2537887901 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7febb400f5d0 con 0x7febc0100e00 2024-09-10T11:33:02.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.681+0000 7febbcff9700 1 --2- 172.21.15.144:0/2537887901 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7feba806de80 0x7feba8070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:02.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.681+0000 7febbcff9700 1 -- 172.21.15.144:0/2537887901 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7febb408efc0 con 0x7febc0100e00 2024-09-10T11:33:02.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.681+0000 7febbf7fe700 1 --2- 172.21.15.144:0/2537887901 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7feba806de80 0x7feba8070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:02.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.681+0000 7febbf7fe700 1 --2- 172.21.15.144:0/2537887901 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7feba806de80 0x7feba8070330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7febb0007900 tx=0x7febb0008040 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:02.686 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.686+0000 7febbcff9700 1 -- 172.21.15.144:0/2537887901 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7febb40c39d0 con 0x7febc0100e00 2024-09-10T11:33:02.764 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:02 smithi144 ceph-mon[27942]: pgmap v87: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:02.764 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:02 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:02.764 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:02 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:33:02.764 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:02 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:33:02.764 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:02 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:02.765 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:02 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-10T11:33:02.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:02 smithi178 ceph-mon[33514]: pgmap v87: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:02.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:02 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:02.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:02 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:33:02.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:02 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:33:02.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:02 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:02.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:02 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-10T11:33:02.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.914+0000 7febc6019700 1 -- 172.21.15.144:0/2537887901 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7febc002cca0 con 0x7feba806de80 2024-09-10T11:33:02.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.916+0000 7febbcff9700 1 -- 172.21.15.144:0/2537887901 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7febc002cca0 con 0x7feba806de80 2024-09-10T11:33:02.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.918+0000 7febc6019700 1 -- 172.21.15.144:0/2537887901 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7feba806de80 msgr2=0x7feba8070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:02.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.918+0000 7febc6019700 1 --2- 172.21.15.144:0/2537887901 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7feba806de80 0x7feba8070330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7febb0007900 tx=0x7febb0008040 comp rx=0 tx=0).stop 2024-09-10T11:33:02.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.918+0000 7febc6019700 1 -- 172.21.15.144:0/2537887901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc0100e00 msgr2=0x7febc0196400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:02.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.918+0000 7febc6019700 1 --2- 172.21.15.144:0/2537887901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc0100e00 0x7febc0196400 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7febb40092e0 tx=0x7febb400be20 comp rx=0 tx=0).stop 2024-09-10T11:33:02.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.918+0000 7febc6019700 1 -- 172.21.15.144:0/2537887901 shutdown_connections 2024-09-10T11:33:02.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.918+0000 7febc6019700 1 --2- 172.21.15.144:0/2537887901 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7feba806de80 0x7feba8070330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:02.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.918+0000 7febc6019700 1 --2- 172.21.15.144:0/2537887901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7febc0100460 0x7febc0195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:02.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.918+0000 7febc6019700 1 --2- 172.21.15.144:0/2537887901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7febc0100e00 0x7febc0196400 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:02.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.918+0000 7febc6019700 1 -- 172.21.15.144:0/2537887901 >> 172.21.15.144:0/2537887901 conn(0x7febc00fbd00 msgr2=0x7febc00fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:02.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.918+0000 7febc6019700 1 -- 172.21.15.144:0/2537887901 shutdown_connections 2024-09-10T11:33:02.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:02.918+0000 7febc6019700 1 -- 172.21.15.144:0/2537887901 wait complete. 2024-09-10T11:33:03.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.024+0000 7f5c41628700 1 -- 172.21.15.144:0/4177785209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0ffd30 msgr2=0x7f5c3c103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:03.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.024+0000 7f5c41628700 1 --2- 172.21.15.144:0/4177785209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0ffd30 0x7f5c3c103ce0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5c30009910 tx=0x7f5c30009c20 comp rx=0 tx=0).stop 2024-09-10T11:33:03.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.024+0000 7f5c41628700 1 -- 172.21.15.144:0/4177785209 shutdown_connections 2024-09-10T11:33:03.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.024+0000 7f5c41628700 1 --2- 172.21.15.144:0/4177785209 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0ffd30 0x7f5c3c103ce0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.024+0000 7f5c41628700 1 --2- 172.21.15.144:0/4177785209 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c3c0ff390 0x7f5c3c0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.024+0000 7f5c41628700 1 -- 172.21.15.144:0/4177785209 >> 172.21.15.144:0/4177785209 conn(0x7f5c3c074f50 msgr2=0x7f5c3c075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:03.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.024+0000 7f5c41628700 1 -- 172.21.15.144:0/4177785209 shutdown_connections 2024-09-10T11:33:03.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.025+0000 7f5c41628700 1 -- 172.21.15.144:0/4177785209 wait complete. 2024-09-10T11:33:03.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.025+0000 7f5c41628700 1 Processor -- start 2024-09-10T11:33:03.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.025+0000 7f5c41628700 1 -- start start 2024-09-10T11:33:03.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.026+0000 7f5c41628700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c3c0ff390 0x7f5c3c196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:03.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.026+0000 7f5c41628700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0ffd30 0x7f5c3c1970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:03.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.026+0000 7f5c41628700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c1976e0 con 0x7f5c3c0ffd30 2024-09-10T11:33:03.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.026+0000 7f5c41628700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c197820 con 0x7f5c3c0ff390 2024-09-10T11:33:03.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.026+0000 7f5c3a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0ffd30 0x7f5c3c1970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:03.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.026+0000 7f5c3a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0ffd30 0x7f5c3c1970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44522/0 (socket says 172.21.15.144:44522) 2024-09-10T11:33:03.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.026+0000 7f5c3a7fc700 1 -- 172.21.15.144:0/3524607496 learned_addr learned my addr 172.21.15.144:0/3524607496 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:33:03.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.026+0000 7f5c3affd700 1 --2- 172.21.15.144:0/3524607496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c3c0ff390 0x7f5c3c196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:03.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.026+0000 7f5c3a7fc700 1 -- 172.21.15.144:0/3524607496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c3c0ff390 msgr2=0x7f5c3c196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:03.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.026+0000 7f5c3a7fc700 1 --2- 172.21.15.144:0/3524607496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c3c0ff390 0x7f5c3c196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.026+0000 7f5c3a7fc700 1 -- 172.21.15.144:0/3524607496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c300095f0 con 0x7f5c3c0ffd30 2024-09-10T11:33:03.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.027+0000 7f5c3a7fc700 1 --2- 172.21.15.144:0/3524607496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0ffd30 0x7f5c3c1970c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5c300092e0 tx=0x7f5c3000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:03.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.027+0000 7f5c23fff700 1 -- 172.21.15.144:0/3524607496 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c3001d070 con 0x7f5c3c0ffd30 2024-09-10T11:33:03.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.027+0000 7f5c41628700 1 -- 172.21.15.144:0/3524607496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3c19fbf0 con 0x7f5c3c0ffd30 2024-09-10T11:33:03.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.027+0000 7f5c23fff700 1 -- 172.21.15.144:0/3524607496 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c3000f460 con 0x7f5c3c0ffd30 2024-09-10T11:33:03.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.027+0000 7f5c23fff700 1 -- 172.21.15.144:0/3524607496 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c300175d0 con 0x7f5c3c0ffd30 2024-09-10T11:33:03.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.027+0000 7f5c41628700 1 -- 172.21.15.144:0/3524607496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3c1a0000 con 0x7f5c3c0ffd30 2024-09-10T11:33:03.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.028+0000 7f5c41628700 1 -- 172.21.15.144:0/3524607496 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3c107600 con 0x7f5c3c0ffd30 2024-09-10T11:33:03.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.028+0000 7f5c23fff700 1 -- 172.21.15.144:0/3524607496 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f5c3000f5d0 con 0x7f5c3c0ffd30 2024-09-10T11:33:03.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.029+0000 7f5c23fff700 1 --2- 172.21.15.144:0/3524607496 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f5c2406de80 0x7f5c24070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:03.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.029+0000 7f5c23fff700 1 -- 172.21.15.144:0/3524607496 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f5c3008efb0 con 0x7f5c3c0ffd30 2024-09-10T11:33:03.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.029+0000 7f5c3affd700 1 --2- 172.21.15.144:0/3524607496 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f5c2406de80 0x7f5c24070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:03.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.029+0000 7f5c3affd700 1 --2- 172.21.15.144:0/3524607496 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f5c2406de80 0x7f5c24070330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c005950 tx=0x7f5c2c00a300 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:03.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.034+0000 7f5c23fff700 1 -- 172.21.15.144:0/3524607496 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f5c300588c0 con 0x7f5c3c0ffd30 2024-09-10T11:33:03.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.246+0000 7f5c41628700 1 -- 172.21.15.144:0/3524607496 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c3c02cd70 con 0x7f5c2406de80 2024-09-10T11:33:03.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.256+0000 7f5c23fff700 1 -- 172.21.15.144:0/3524607496 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f5c3c02cd70 con 0x7f5c2406de80 2024-09-10T11:33:03.257 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:33:03.257 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (77s) 50s ago 11m 15.9M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:33:03.257 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (12m) 50s ago 12m 7990k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:33:03.257 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (11m) 107s ago 11m 7830k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:33:03.257 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (12m) 50s ago 12m 6669k - 18.2.0 dc2bc1663786 3311fb04aa0c 2024-09-10T11:33:03.258 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (11m) 107s ago 11m 6651k - 18.2.0 dc2bc1663786 6e408e8476d4 2024-09-10T11:33:03.258 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (55s) 50s ago 11m 46.3M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:33:03.258 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (6m) 50s ago 6m 59.9M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:33:03.258 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (6m) 50s ago 6m 127M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:33:03.258 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (6m) 107s ago 6m 65.7M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:33:03.258 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (6m) 107s ago 6m 127M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:33:03.258 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (3m) 50s ago 13m 505M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:33:03.258 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (2m) 107s ago 10m 441M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:33:03.258 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (13m) 50s ago 13m 56.5M 2048M 18.2.0 dc2bc1663786 42addb654e4f 2024-09-10T11:33:03.258 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (10m) 107s ago 10m 43.5M 2048M 18.2.0 dc2bc1663786 3c1ba210ed3f 2024-09-10T11:33:03.258 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (114s) 50s ago 11m 15.8M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:33:03.258 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (110s) 107s ago 10m 3586k - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:33:03.258 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (10m) 50s ago 10m 205M 1216M 18.2.0 dc2bc1663786 2435febebdb5 2024-09-10T11:33:03.259 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (9m) 50s ago 9m 212M 1216M 18.2.0 dc2bc1663786 b55fe920ff9d 2024-09-10T11:33:03.259 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (9m) 50s ago 9m 169M 1216M 18.2.0 dc2bc1663786 3ed5618eac33 2024-09-10T11:33:03.259 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (8m) 107s ago 8m 252M 2239M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:33:03.259 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (8m) 107s ago 8m 217M 2239M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:33:03.259 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (7m) 107s ago 7m 195M 2239M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:33:03.259 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (94s) 50s ago 11m 63.9M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:33:03.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.258+0000 7f5c41628700 1 -- 172.21.15.144:0/3524607496 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f5c2406de80 msgr2=0x7f5c24070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:03.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.258+0000 7f5c41628700 1 --2- 172.21.15.144:0/3524607496 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f5c2406de80 0x7f5c24070330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c005950 tx=0x7f5c2c00a300 comp rx=0 tx=0).stop 2024-09-10T11:33:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.258+0000 7f5c41628700 1 -- 172.21.15.144:0/3524607496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0ffd30 msgr2=0x7f5c3c1970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.258+0000 7f5c41628700 1 --2- 172.21.15.144:0/3524607496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0ffd30 0x7f5c3c1970c0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5c300092e0 tx=0x7f5c3000bd70 comp rx=0 tx=0).stop 2024-09-10T11:33:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.258+0000 7f5c41628700 1 -- 172.21.15.144:0/3524607496 shutdown_connections 2024-09-10T11:33:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.258+0000 7f5c41628700 1 --2- 172.21.15.144:0/3524607496 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f5c2406de80 0x7f5c24070330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.258+0000 7f5c41628700 1 --2- 172.21.15.144:0/3524607496 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c3c0ff390 0x7f5c3c196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.258+0000 7f5c41628700 1 --2- 172.21.15.144:0/3524607496 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c3c0ffd30 0x7f5c3c1970c0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.258+0000 7f5c41628700 1 -- 172.21.15.144:0/3524607496 >> 172.21.15.144:0/3524607496 conn(0x7f5c3c074f50 msgr2=0x7f5c3c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.258+0000 7f5c41628700 1 -- 172.21.15.144:0/3524607496 shutdown_connections 2024-09-10T11:33:03.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.258+0000 7f5c41628700 1 -- 172.21.15.144:0/3524607496 wait complete. 2024-09-10T11:33:03.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.363+0000 7fd2e1261700 1 -- 172.21.15.144:0/1115661051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ffba0 msgr2=0x7fd2dc10a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:03.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.363+0000 7fd2e1261700 1 --2- 172.21.15.144:0/1115661051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ffba0 0x7fd2dc10a610 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fd2cc009910 tx=0x7fd2cc009c20 comp rx=0 tx=0).stop 2024-09-10T11:33:03.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.363+0000 7fd2e1261700 1 -- 172.21.15.144:0/1115661051 shutdown_connections 2024-09-10T11:33:03.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.363+0000 7fd2e1261700 1 --2- 172.21.15.144:0/1115661051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ffba0 0x7fd2dc10a610 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.363+0000 7fd2e1261700 1 --2- 172.21.15.144:0/1115661051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2dc0ff290 0x7fd2dc0ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.363+0000 7fd2e1261700 1 -- 172.21.15.144:0/1115661051 >> 172.21.15.144:0/1115661051 conn(0x7fd2dc074850 msgr2=0x7fd2dc074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:03.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.363+0000 7fd2e1261700 1 -- 172.21.15.144:0/1115661051 shutdown_connections 2024-09-10T11:33:03.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.363+0000 7fd2e1261700 1 -- 172.21.15.144:0/1115661051 wait complete. 2024-09-10T11:33:03.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.364+0000 7fd2e1261700 1 Processor -- start 2024-09-10T11:33:03.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.364+0000 7fd2e1261700 1 -- start start 2024-09-10T11:33:03.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.364+0000 7fd2e1261700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ff290 0x7fd2dc195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:03.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.364+0000 7fd2e1261700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2dc0ffba0 0x7fd2dc1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:03.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.364+0000 7fd2e1261700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc1969b0 con 0x7fd2dc0ff290 2024-09-10T11:33:03.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.364+0000 7fd2e1261700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2dc19a740 con 0x7fd2dc0ffba0 2024-09-10T11:33:03.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.364+0000 7fd2daffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ff290 0x7fd2dc195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:03.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.364+0000 7fd2daffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ff290 0x7fd2dc195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44544/0 (socket says 172.21.15.144:44544) 2024-09-10T11:33:03.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.364+0000 7fd2daffd700 1 -- 172.21.15.144:0/693271904 learned_addr learned my addr 172.21.15.144:0/693271904 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:33:03.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.364+0000 7fd2da7fc700 1 --2- 172.21.15.144:0/693271904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2dc0ffba0 0x7fd2dc1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:03.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.365+0000 7fd2daffd700 1 -- 172.21.15.144:0/693271904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2dc0ffba0 msgr2=0x7fd2dc1962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:03.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.365+0000 7fd2daffd700 1 --2- 172.21.15.144:0/693271904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2dc0ffba0 0x7fd2dc1962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.365+0000 7fd2daffd700 1 -- 172.21.15.144:0/693271904 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2cc0095f0 con 0x7fd2dc0ff290 2024-09-10T11:33:03.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.365+0000 7fd2daffd700 1 --2- 172.21.15.144:0/693271904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ff290 0x7fd2dc195d90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400d8d0 tx=0x7fd2c400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:03.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.365+0000 7fd2d3fff700 1 -- 172.21.15.144:0/693271904 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2c40098a0 con 0x7fd2dc0ff290 2024-09-10T11:33:03.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.365+0000 7fd2e1261700 1 -- 172.21.15.144:0/693271904 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2dc19aa20 con 0x7fd2dc0ff290 2024-09-10T11:33:03.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.365+0000 7fd2d3fff700 1 -- 172.21.15.144:0/693271904 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2c4010460 con 0x7fd2dc0ff290 2024-09-10T11:33:03.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.365+0000 7fd2d3fff700 1 -- 172.21.15.144:0/693271904 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2c400f5f0 con 0x7fd2dc0ff290 2024-09-10T11:33:03.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.365+0000 7fd2e1261700 1 -- 172.21.15.144:0/693271904 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2dc19af70 con 0x7fd2dc0ff290 2024-09-10T11:33:03.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.366+0000 7fd2e1261700 1 -- 172.21.15.144:0/693271904 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2dc107d10 con 0x7fd2dc0ff290 2024-09-10T11:33:03.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.367+0000 7fd2d3fff700 1 -- 172.21.15.144:0/693271904 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7fd2c4009a00 con 0x7fd2dc0ff290 2024-09-10T11:33:03.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.367+0000 7fd2d3fff700 1 --2- 172.21.15.144:0/693271904 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fd2c806de80 0x7fd2c8070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:03.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.367+0000 7fd2d3fff700 1 -- 172.21.15.144:0/693271904 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7fd2c408d9b0 con 0x7fd2dc0ff290 2024-09-10T11:33:03.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.367+0000 7fd2da7fc700 1 --2- 172.21.15.144:0/693271904 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fd2c806de80 0x7fd2c8070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:03.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.368+0000 7fd2da7fc700 1 --2- 172.21.15.144:0/693271904 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fd2c806de80 0x7fd2c8070330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc1973b0 tx=0x7fd2cc01a040 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:03.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.372+0000 7fd2d3fff700 1 -- 172.21.15.144:0/693271904 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7fd2c40572c0 con 0x7fd2dc0ff290 2024-09-10T11:33:03.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.676+0000 7fd2e1261700 1 -- 172.21.15.144:0/693271904 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd2dc19b250 con 0x7fd2dc0ff290 2024-09-10T11:33:03.678 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:03 smithi144 ceph-mon[27942]: Upgrade: Target is version 19.1.1-305-g1cf819b7 (squid) 2024-09-10T11:33:03.678 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:03 smithi144 ceph-mon[27942]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, digests ['quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559'] 2024-09-10T11:33:03.678 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:03 smithi144 ceph-mon[27942]: Upgrade: Setting container_image for all mgr 2024-09-10T11:33:03.678 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:03 smithi144 ceph-mon[27942]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:33:03.678 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:03 smithi144 ceph-mon[27942]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:33:03.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.677+0000 7fd2d3fff700 1 -- 172.21.15.144:0/693271904 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+679 (secure 0 0 0) 0x7fd2c405a980 con 0x7fd2dc0ff290 2024-09-10T11:33:03.679 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:33:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:33:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-09-10T11:33:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:33:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:33:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:33:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:33:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:33:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-09-10T11:33:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:33:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:33:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:33:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:33:03.679 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:33:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-09-10T11:33:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:33:03.680 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:33:03.680 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:33:03.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.679+0000 7fd2e1261700 1 -- 172.21.15.144:0/693271904 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fd2c806de80 msgr2=0x7fd2c8070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:03.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.679+0000 7fd2e1261700 1 --2- 172.21.15.144:0/693271904 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fd2c806de80 0x7fd2c8070330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2dc1973b0 tx=0x7fd2cc01a040 comp rx=0 tx=0).stop 2024-09-10T11:33:03.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.679+0000 7fd2e1261700 1 -- 172.21.15.144:0/693271904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ff290 msgr2=0x7fd2dc195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:03.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.679+0000 7fd2e1261700 1 --2- 172.21.15.144:0/693271904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ff290 0x7fd2dc195d90 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fd2c400d8d0 tx=0x7fd2c400dbe0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.679+0000 7fd2e1261700 1 -- 172.21.15.144:0/693271904 shutdown_connections 2024-09-10T11:33:03.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.679+0000 7fd2e1261700 1 --2- 172.21.15.144:0/693271904 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7fd2c806de80 0x7fd2c8070330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.679+0000 7fd2e1261700 1 --2- 172.21.15.144:0/693271904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2dc0ff290 0x7fd2dc195d90 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.679+0000 7fd2e1261700 1 --2- 172.21.15.144:0/693271904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2dc0ffba0 0x7fd2dc1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.679+0000 7fd2e1261700 1 -- 172.21.15.144:0/693271904 >> 172.21.15.144:0/693271904 conn(0x7fd2dc074850 msgr2=0x7fd2dc104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:03.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.679+0000 7fd2e1261700 1 -- 172.21.15.144:0/693271904 shutdown_connections 2024-09-10T11:33:03.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.679+0000 7fd2e1261700 1 -- 172.21.15.144:0/693271904 wait complete. 2024-09-10T11:33:03.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.790+0000 7f92eeeb6700 1 -- 172.21.15.144:0/1949490709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e80ff390 msgr2=0x7f92e80ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:03.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.790+0000 7f92eeeb6700 1 --2- 172.21.15.144:0/1949490709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e80ff390 0x7f92e80ff760 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f92dc009910 tx=0x7f92dc009c20 comp rx=0 tx=0).stop 2024-09-10T11:33:03.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.790+0000 7f92eeeb6700 1 -- 172.21.15.144:0/1949490709 shutdown_connections 2024-09-10T11:33:03.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.790+0000 7f92eeeb6700 1 --2- 172.21.15.144:0/1949490709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e80ffd30 0x7f92e8103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.790+0000 7f92eeeb6700 1 --2- 172.21.15.144:0/1949490709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e80ff390 0x7f92e80ff760 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.790+0000 7f92eeeb6700 1 -- 172.21.15.144:0/1949490709 >> 172.21.15.144:0/1949490709 conn(0x7f92e8074f50 msgr2=0x7f92e8075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:03.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.790+0000 7f92eeeb6700 1 -- 172.21.15.144:0/1949490709 shutdown_connections 2024-09-10T11:33:03.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.790+0000 7f92eeeb6700 1 -- 172.21.15.144:0/1949490709 wait complete. 2024-09-10T11:33:03.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.790+0000 7f92eeeb6700 1 Processor -- start 2024-09-10T11:33:03.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.790+0000 7f92eeeb6700 1 -- start start 2024-09-10T11:33:03.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.791+0000 7f92eeeb6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e80ff390 0x7f92e8100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:03.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.791+0000 7f92eeeb6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e80ffd30 0x7f92e81012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:03.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.791+0000 7f92eeeb6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8101800 con 0x7f92e80ff390 2024-09-10T11:33:03.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.791+0000 7f92eeeb6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8101940 con 0x7f92e80ffd30 2024-09-10T11:33:03.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.791+0000 7f92ecc52700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e80ff390 0x7f92e8100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:03.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.791+0000 7f92ecc52700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e80ff390 0x7f92e8100d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44562/0 (socket says 172.21.15.144:44562) 2024-09-10T11:33:03.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.791+0000 7f92ecc52700 1 -- 172.21.15.144:0/2101334707 learned_addr learned my addr 172.21.15.144:0/2101334707 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:33:03.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.791+0000 7f92e7fff700 1 --2- 172.21.15.144:0/2101334707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e80ffd30 0x7f92e81012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:03.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.791+0000 7f92ecc52700 1 -- 172.21.15.144:0/2101334707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e80ffd30 msgr2=0x7f92e81012c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:03.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.791+0000 7f92ecc52700 1 --2- 172.21.15.144:0/2101334707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e80ffd30 0x7f92e81012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:03.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.791+0000 7f92ecc52700 1 -- 172.21.15.144:0/2101334707 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92dc0095f0 con 0x7f92e80ff390 2024-09-10T11:33:03.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.791+0000 7f92ecc52700 1 --2- 172.21.15.144:0/2101334707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e80ff390 0x7f92e8100d80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00b5c0 tx=0x7f92dc00be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:03.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.791+0000 7f92e5ffb700 1 -- 172.21.15.144:0/2101334707 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92dc01d070 con 0x7f92e80ff390 2024-09-10T11:33:03.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.792+0000 7f92e5ffb700 1 -- 172.21.15.144:0/2101334707 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92dc00f460 con 0x7f92e80ff390 2024-09-10T11:33:03.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.792+0000 7f92eeeb6700 1 -- 172.21.15.144:0/2101334707 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e8101bc0 con 0x7f92e80ff390 2024-09-10T11:33:03.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.792+0000 7f92e5ffb700 1 -- 172.21.15.144:0/2101334707 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92dc017610 con 0x7f92e80ff390 2024-09-10T11:33:03.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.792+0000 7f92eeeb6700 1 -- 172.21.15.144:0/2101334707 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e81a2330 con 0x7f92e80ff390 2024-09-10T11:33:03.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.793+0000 7f92eeeb6700 1 -- 172.21.15.144:0/2101334707 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e8108660 con 0x7f92e80ff390 2024-09-10T11:33:03.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.793+0000 7f92e5ffb700 1 -- 172.21.15.144:0/2101334707 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f92dc00f5d0 con 0x7f92e80ff390 2024-09-10T11:33:03.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.794+0000 7f92e5ffb700 1 --2- 172.21.15.144:0/2101334707 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f92d806de80 0x7f92d8070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:03.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.794+0000 7f92e5ffb700 1 -- 172.21.15.144:0/2101334707 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f92dc08f0b0 con 0x7f92e80ff390 2024-09-10T11:33:03.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.794+0000 7f92e7fff700 1 --2- 172.21.15.144:0/2101334707 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f92d806de80 0x7f92d8070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:03.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.794+0000 7f92e7fff700 1 --2- 172.21.15.144:0/2101334707 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f92d806de80 0x7f92d8070330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f92d4006fd0 tx=0x7f92d4008040 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:03.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:03.798+0000 7f92e5ffb700 1 -- 172.21.15.144:0/2101334707 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f92dc0589c0 con 0x7f92e80ff390 2024-09-10T11:33:03.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:03 smithi178 ceph-mon[33514]: Upgrade: Target is version 19.1.1-305-g1cf819b7 (squid) 2024-09-10T11:33:03.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:03 smithi178 ceph-mon[33514]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, digests ['quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559'] 2024-09-10T11:33:03.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:03 smithi178 ceph-mon[33514]: Upgrade: Setting container_image for all mgr 2024-09-10T11:33:03.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:03 smithi178 ceph-mon[33514]: from='client.14816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:33:03.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:03 smithi178 ceph-mon[33514]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:33:04.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.076+0000 7f92eeeb6700 1 -- 172.21.15.144:0/2101334707 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f92e804ea50 con 0x7f92e80ff390 2024-09-10T11:33:04.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.077+0000 7f92e5ffb700 1 -- 172.21.15.144:0/2101334707 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 22 v22) v1 ==== 76+0+1871 (secure 0 0 0) 0x7f92dc05c080 con 0x7f92e80ff390 2024-09-10T11:33:04.078 INFO:teuthology.orchestra.run.smithi144.stdout:e22 2024-09-10T11:33:04.078 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:33:04.078 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:33:04.078 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:33:04.079 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:33:04.079 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:33:04.079 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:33:04.079 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 22 2024-09-10T11:33:04.079 INFO:teuthology.orchestra.run.smithi144.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T11:33:04.079 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:33:04.079 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:31:50.444180+0000 2024-09-10T11:33:04.079 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:33:04.079 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:33:04.079 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:33:04.079 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:33:04.079 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:33:04.079 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:33:04.079 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:33:04.080 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 47 2024-09-10T11:33:04.080 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:33:04.080 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:33:04.080 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:33:04.080 INFO:teuthology.orchestra.run.smithi144.stdout:up {0=14488,1=24267} 2024-09-10T11:33:04.080 INFO:teuthology.orchestra.run.smithi144.stdout:failed 2024-09-10T11:33:04.080 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:33:04.080 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:33:04.080 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:33:04.080 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:33:04.080 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:33:04.080 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:33:04.080 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:33:04.080 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:33:04.081 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{0:14488} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:04.081 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{0:14498} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/1752150537,v1:172.21.15.144:6827/1752150537] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:04.081 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{1:24267} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:04.081 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:04.081 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:33:04.081 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:33:04.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.080+0000 7f92eeeb6700 1 -- 172.21.15.144:0/2101334707 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f92d806de80 msgr2=0x7f92d8070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:04.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.080+0000 7f92eeeb6700 1 --2- 172.21.15.144:0/2101334707 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f92d806de80 0x7f92d8070330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f92d4006fd0 tx=0x7f92d4008040 comp rx=0 tx=0).stop 2024-09-10T11:33:04.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.080+0000 7f92eeeb6700 1 -- 172.21.15.144:0/2101334707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e80ff390 msgr2=0x7f92e8100d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:04.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.080+0000 7f92eeeb6700 1 --2- 172.21.15.144:0/2101334707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e80ff390 0x7f92e8100d80 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00b5c0 tx=0x7f92dc00be00 comp rx=0 tx=0).stop 2024-09-10T11:33:04.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.080+0000 7f92eeeb6700 1 -- 172.21.15.144:0/2101334707 shutdown_connections 2024-09-10T11:33:04.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.080+0000 7f92eeeb6700 1 --2- 172.21.15.144:0/2101334707 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f92d806de80 0x7f92d8070330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:04.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.080+0000 7f92eeeb6700 1 --2- 172.21.15.144:0/2101334707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92e80ff390 0x7f92e8100d80 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:04.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.080+0000 7f92eeeb6700 1 --2- 172.21.15.144:0/2101334707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92e80ffd30 0x7f92e81012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:04.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.080+0000 7f92eeeb6700 1 -- 172.21.15.144:0/2101334707 >> 172.21.15.144:0/2101334707 conn(0x7f92e8074f50 msgr2=0x7f92e80686d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:04.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.080+0000 7f92eeeb6700 1 -- 172.21.15.144:0/2101334707 shutdown_connections 2024-09-10T11:33:04.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.080+0000 7f92eeeb6700 1 -- 172.21.15.144:0/2101334707 wait complete. 2024-09-10T11:33:04.083 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 22 2024-09-10T11:33:04.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.189+0000 7f58c513e700 1 -- 172.21.15.144:0/332820616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c0100e00 msgr2=0x7f58c0104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:04.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.189+0000 7f58c513e700 1 --2- 172.21.15.144:0/332820616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c0100e00 0x7f58c0104dd0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f58b4009910 tx=0x7f58b4009c20 comp rx=0 tx=0).stop 2024-09-10T11:33:04.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.189+0000 7f58c513e700 1 -- 172.21.15.144:0/332820616 shutdown_connections 2024-09-10T11:33:04.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.189+0000 7f58c513e700 1 --2- 172.21.15.144:0/332820616 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c0100e00 0x7f58c0104dd0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:04.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.189+0000 7f58c513e700 1 --2- 172.21.15.144:0/332820616 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58c0100460 0x7f58c0100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:04.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.189+0000 7f58c513e700 1 -- 172.21.15.144:0/332820616 >> 172.21.15.144:0/332820616 conn(0x7f58c00fbd00 msgr2=0x7f58c00fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:04.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.190+0000 7f58c513e700 1 -- 172.21.15.144:0/332820616 shutdown_connections 2024-09-10T11:33:04.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.190+0000 7f58c513e700 1 -- 172.21.15.144:0/332820616 wait complete. 2024-09-10T11:33:04.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.191+0000 7f58c513e700 1 Processor -- start 2024-09-10T11:33:04.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.191+0000 7f58c513e700 1 -- start start 2024-09-10T11:33:04.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.191+0000 7f58c513e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58c0100460 0x7f58c0073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:04.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.191+0000 7f58c513e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c0100e00 0x7f58c0073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:04.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.191+0000 7f58c513e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c0073ad0 con 0x7f58c0100e00 2024-09-10T11:33:04.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.191+0000 7f58c513e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c0073c10 con 0x7f58c0100460 2024-09-10T11:33:04.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.191+0000 7f58be59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c0100e00 0x7f58c0073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:04.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.191+0000 7f58be59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c0100e00 0x7f58c0073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44576/0 (socket says 172.21.15.144:44576) 2024-09-10T11:33:04.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.191+0000 7f58be59c700 1 -- 172.21.15.144:0/64905784 learned_addr learned my addr 172.21.15.144:0/64905784 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:33:04.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.192+0000 7f58bed9d700 1 --2- 172.21.15.144:0/64905784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58c0100460 0x7f58c0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:04.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.192+0000 7f58be59c700 1 -- 172.21.15.144:0/64905784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58c0100460 msgr2=0x7f58c0073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:04.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.192+0000 7f58be59c700 1 --2- 172.21.15.144:0/64905784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58c0100460 0x7f58c0073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:04.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.192+0000 7f58be59c700 1 -- 172.21.15.144:0/64905784 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b40095f0 con 0x7f58c0100e00 2024-09-10T11:33:04.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.192+0000 7f58bed9d700 1 --2- 172.21.15.144:0/64905784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58c0100460 0x7f58c0073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:33:04.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.192+0000 7f58be59c700 1 --2- 172.21.15.144:0/64905784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c0100e00 0x7f58c0073590 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f58b40092e0 tx=0x7f58b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:04.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.192+0000 7f58a7fff700 1 -- 172.21.15.144:0/64905784 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58b401d070 con 0x7f58c0100e00 2024-09-10T11:33:04.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.192+0000 7f58a7fff700 1 -- 172.21.15.144:0/64905784 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58b400f460 con 0x7f58c0100e00 2024-09-10T11:33:04.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.192+0000 7f58c513e700 1 -- 172.21.15.144:0/64905784 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c0073e90 con 0x7f58c0100e00 2024-09-10T11:33:04.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.192+0000 7f58a7fff700 1 -- 172.21.15.144:0/64905784 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58b40175f0 con 0x7f58c0100e00 2024-09-10T11:33:04.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.192+0000 7f58c513e700 1 -- 172.21.15.144:0/64905784 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c01a23c0 con 0x7f58c0100e00 2024-09-10T11:33:04.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.193+0000 7f58c513e700 1 -- 172.21.15.144:0/64905784 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c0108720 con 0x7f58c0100e00 2024-09-10T11:33:04.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.194+0000 7f58a7fff700 1 -- 172.21.15.144:0/64905784 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f58b400f5d0 con 0x7f58c0100e00 2024-09-10T11:33:04.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.195+0000 7f58a7fff700 1 --2- 172.21.15.144:0/64905784 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f58a806de80 0x7f58a8070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:04.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.195+0000 7f58a7fff700 1 -- 172.21.15.144:0/64905784 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f58b408f070 con 0x7f58c0100e00 2024-09-10T11:33:04.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.195+0000 7f58bed9d700 1 --2- 172.21.15.144:0/64905784 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f58a806de80 0x7f58a8070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:04.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.195+0000 7f58bed9d700 1 --2- 172.21.15.144:0/64905784 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f58a806de80 0x7f58a8070330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58b00099d0 tx=0x7f58b0008040 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:04.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.199+0000 7f58a7fff700 1 -- 172.21.15.144:0/64905784 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f58b4058920 con 0x7f58c0100e00 2024-09-10T11:33:04.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.430+0000 7f58c513e700 1 -- 172.21.15.144:0/64905784 --> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f58c0061190 con 0x7f58a806de80 2024-09-10T11:33:04.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.432+0000 7f58a7fff700 1 -- 172.21.15.144:0/64905784 <== mgr.24417 v2:172.21.15.144:6800/1923800240 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f58c0061190 con 0x7f58a806de80 2024-09-10T11:33:04.433 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:33:04.433 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:33:04.433 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:33:04.433 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:33:04.433 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:33:04.433 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr" 2024-09-10T11:33:04.433 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:33:04.433 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "2/23 daemons upgraded", 2024-09-10T11:33:04.433 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mon daemons", 2024-09-10T11:33:04.434 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:33:04.434 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:33:04.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.434+0000 7f58c513e700 1 -- 172.21.15.144:0/64905784 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f58a806de80 msgr2=0x7f58a8070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:04.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.434+0000 7f58c513e700 1 --2- 172.21.15.144:0/64905784 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f58a806de80 0x7f58a8070330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f58b00099d0 tx=0x7f58b0008040 comp rx=0 tx=0).stop 2024-09-10T11:33:04.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.434+0000 7f58c513e700 1 -- 172.21.15.144:0/64905784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c0100e00 msgr2=0x7f58c0073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:04.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.434+0000 7f58c513e700 1 --2- 172.21.15.144:0/64905784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c0100e00 0x7f58c0073590 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f58b40092e0 tx=0x7f58b400be20 comp rx=0 tx=0).stop 2024-09-10T11:33:04.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.434+0000 7f58c513e700 1 -- 172.21.15.144:0/64905784 shutdown_connections 2024-09-10T11:33:04.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.434+0000 7f58c513e700 1 --2- 172.21.15.144:0/64905784 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f58a806de80 0x7f58a8070330 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:04.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.434+0000 7f58c513e700 1 --2- 172.21.15.144:0/64905784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58c0100460 0x7f58c0073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:04.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.434+0000 7f58c513e700 1 --2- 172.21.15.144:0/64905784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58c0100e00 0x7f58c0073590 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:04.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.434+0000 7f58c513e700 1 -- 172.21.15.144:0/64905784 >> 172.21.15.144:0/64905784 conn(0x7f58c00fbd00 msgr2=0x7f58c00fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:04.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.434+0000 7f58c513e700 1 -- 172.21.15.144:0/64905784 shutdown_connections 2024-09-10T11:33:04.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.434+0000 7f58c513e700 1 -- 172.21.15.144:0/64905784 wait complete. 2024-09-10T11:33:04.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.539+0000 7f1e89d08700 1 -- 172.21.15.144:0/1923158099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e840ff2d0 msgr2=0x7f1e840ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:04.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.539+0000 7f1e89d08700 1 --2- 172.21.15.144:0/1923158099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e840ff2d0 0x7f1e840ff6a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1e74009910 tx=0x7f1e74009c20 comp rx=0 tx=0).stop 2024-09-10T11:33:04.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.539+0000 7f1e89d08700 1 -- 172.21.15.144:0/1923158099 shutdown_connections 2024-09-10T11:33:04.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.539+0000 7f1e89d08700 1 --2- 172.21.15.144:0/1923158099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e840ffc70 0x7f1e84103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:04.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.539+0000 7f1e89d08700 1 --2- 172.21.15.144:0/1923158099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e840ff2d0 0x7f1e840ff6a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:04.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.539+0000 7f1e89d08700 1 -- 172.21.15.144:0/1923158099 >> 172.21.15.144:0/1923158099 conn(0x7f1e840fabb0 msgr2=0x7f1e840fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:04.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.539+0000 7f1e89d08700 1 -- 172.21.15.144:0/1923158099 shutdown_connections 2024-09-10T11:33:04.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.539+0000 7f1e89d08700 1 -- 172.21.15.144:0/1923158099 wait complete. 2024-09-10T11:33:04.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.540+0000 7f1e89d08700 1 Processor -- start 2024-09-10T11:33:04.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.540+0000 7f1e89d08700 1 -- start start 2024-09-10T11:33:04.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.540+0000 7f1e89d08700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e840ff2d0 0x7f1e84196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:04.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.540+0000 7f1e89d08700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e840ffc70 0x7f1e841971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:04.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.540+0000 7f1e89d08700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e84197810 con 0x7f1e840ffc70 2024-09-10T11:33:04.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.540+0000 7f1e89d08700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e84197950 con 0x7f1e840ff2d0 2024-09-10T11:33:04.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.540+0000 7f1e82ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e840ffc70 0x7f1e841971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:04.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.540+0000 7f1e82ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e840ffc70 0x7f1e841971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44596/0 (socket says 172.21.15.144:44596) 2024-09-10T11:33:04.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.540+0000 7f1e82ffd700 1 -- 172.21.15.144:0/3779714346 learned_addr learned my addr 172.21.15.144:0/3779714346 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:33:04.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.541+0000 7f1e837fe700 1 --2- 172.21.15.144:0/3779714346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e840ff2d0 0x7f1e84196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:04.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.541+0000 7f1e82ffd700 1 -- 172.21.15.144:0/3779714346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e840ff2d0 msgr2=0x7f1e84196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:04.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.541+0000 7f1e82ffd700 1 --2- 172.21.15.144:0/3779714346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e840ff2d0 0x7f1e84196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:04.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.541+0000 7f1e82ffd700 1 -- 172.21.15.144:0/3779714346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e740095f0 con 0x7f1e840ffc70 2024-09-10T11:33:04.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.541+0000 7f1e82ffd700 1 --2- 172.21.15.144:0/3779714346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e840ffc70 0x7f1e841971f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800b480 tx=0x7f1e7800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:04.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.541+0000 7f1e80ff9700 1 -- 172.21.15.144:0/3779714346 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e78011800 con 0x7f1e840ffc70 2024-09-10T11:33:04.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.541+0000 7f1e80ff9700 1 -- 172.21.15.144:0/3779714346 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1e78011e20 con 0x7f1e840ffc70 2024-09-10T11:33:04.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.541+0000 7f1e89d08700 1 -- 172.21.15.144:0/3779714346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8419fd20 con 0x7f1e840ffc70 2024-09-10T11:33:04.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.541+0000 7f1e80ff9700 1 -- 172.21.15.144:0/3779714346 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e78010ea0 con 0x7f1e840ffc70 2024-09-10T11:33:04.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.541+0000 7f1e89d08700 1 -- 172.21.15.144:0/3779714346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e841a01e0 con 0x7f1e840ffc70 2024-09-10T11:33:04.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.542+0000 7f1e89d08700 1 -- 172.21.15.144:0/3779714346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e841075e0 con 0x7f1e840ffc70 2024-09-10T11:33:04.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.543+0000 7f1e80ff9700 1 -- 172.21.15.144:0/3779714346 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 32) v1 ==== 92058+0+0 (secure 0 0 0) 0x7f1e78010380 con 0x7f1e840ffc70 2024-09-10T11:33:04.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.543+0000 7f1e80ff9700 1 --2- 172.21.15.144:0/3779714346 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1e6c06de80 0x7f1e6c070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:04.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.543+0000 7f1e80ff9700 1 -- 172.21.15.144:0/3779714346 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5898+0+0 (secure 0 0 0) 0x7f1e7808d520 con 0x7f1e840ffc70 2024-09-10T11:33:04.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.543+0000 7f1e837fe700 1 --2- 172.21.15.144:0/3779714346 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1e6c06de80 0x7f1e6c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:04.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.544+0000 7f1e837fe700 1 --2- 172.21.15.144:0/3779714346 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1e6c06de80 0x7f1e6c070330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1e74006010 tx=0x7f1e7401a040 comp rx=0 tx=0).ready entity=mgr.24417 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:04.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.548+0000 7f1e80ff9700 1 -- 172.21.15.144:0/3779714346 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186811 (secure 0 0 0) 0x7f1e78056eb0 con 0x7f1e840ffc70 2024-09-10T11:33:04.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.839+0000 7f1e89d08700 1 -- 172.21.15.144:0/3779714346 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1e8404ea50 con 0x7f1e840ffc70 2024-09-10T11:33:04.841 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:04 smithi144 ceph-mon[27942]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:33:04.841 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:04 smithi144 ceph-mon[27942]: pgmap v88: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:04.841 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:04 smithi144 ceph-mon[27942]: Upgrade: Updating mon.smithi144 2024-09-10T11:33:04.841 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:04 smithi144 ceph-mon[27942]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:04.841 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:04 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T11:33:04.841 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:04 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T11:33:04.841 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:04 smithi144 ceph-mon[27942]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:04.841 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:04 smithi144 ceph-mon[27942]: Deploying daemon mon.smithi144 on smithi144 2024-09-10T11:33:04.841 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:04 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/693271904' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:33:04.841 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:04 smithi144 ceph-mon[27942]: from='client.? 172.21.15.144:0/2101334707' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:33:04.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.839+0000 7f1e80ff9700 1 -- 172.21.15.144:0/3779714346 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f1e7805a570 con 0x7f1e840ffc70 2024-09-10T11:33:04.842 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T11:33:04.842 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:33:04.842 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:33:04.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.841+0000 7f1e89d08700 1 -- 172.21.15.144:0/3779714346 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1e6c06de80 msgr2=0x7f1e6c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:04.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.841+0000 7f1e89d08700 1 --2- 172.21.15.144:0/3779714346 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1e6c06de80 0x7f1e6c070330 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1e74006010 tx=0x7f1e7401a040 comp rx=0 tx=0).stop 2024-09-10T11:33:04.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.841+0000 7f1e89d08700 1 -- 172.21.15.144:0/3779714346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e840ffc70 msgr2=0x7f1e841971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:04.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.841+0000 7f1e89d08700 1 --2- 172.21.15.144:0/3779714346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e840ffc70 0x7f1e841971f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1e7800b480 tx=0x7f1e7800b790 comp rx=0 tx=0).stop 2024-09-10T11:33:04.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.841+0000 7f1e89d08700 1 -- 172.21.15.144:0/3779714346 shutdown_connections 2024-09-10T11:33:04.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.841+0000 7f1e89d08700 1 --2- 172.21.15.144:0/3779714346 >> [v2:172.21.15.144:6800/1923800240,v1:172.21.15.144:6801/1923800240] conn(0x7f1e6c06de80 0x7f1e6c070330 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:04.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.841+0000 7f1e89d08700 1 --2- 172.21.15.144:0/3779714346 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e840ff2d0 0x7f1e84196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:04.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.841+0000 7f1e89d08700 1 --2- 172.21.15.144:0/3779714346 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e840ffc70 0x7f1e841971f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:04.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.841+0000 7f1e89d08700 1 -- 172.21.15.144:0/3779714346 >> 172.21.15.144:0/3779714346 conn(0x7f1e840fabb0 msgr2=0x7f1e840fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:04.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.842+0000 7f1e89d08700 1 -- 172.21.15.144:0/3779714346 shutdown_connections 2024-09-10T11:33:04.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:04.842+0000 7f1e89d08700 1 -- 172.21.15.144:0/3779714346 wait complete. 2024-09-10T11:33:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:04 smithi178 ceph-mon[33514]: from='client.14824 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:33:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:04 smithi178 ceph-mon[33514]: pgmap v88: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:04 smithi178 ceph-mon[33514]: Upgrade: Updating mon.smithi144 2024-09-10T11:33:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:04 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:05.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:04 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T11:33:05.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:04 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T11:33:05.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:04 smithi178 ceph-mon[33514]: from='mgr.24417 172.21.15.144:0/2333652225' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:05.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:04 smithi178 ceph-mon[33514]: Deploying daemon mon.smithi144 on smithi144 2024-09-10T11:33:05.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:04 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/693271904' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:33:05.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:04 smithi178 ceph-mon[33514]: from='client.? 172.21.15.144:0/2101334707' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:33:05.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:05 smithi144 systemd[1]: Stopping Ceph mon.smithi144 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:33:05.892 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:05 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[27938]: 2024-09-10T11:33:05.580+0000 7f4ec2b84700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi144 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:33:05.892 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:05 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[27938]: 2024-09-10T11:33:05.580+0000 7f4ec2b84700 -1 mon.smithi144@0(leader) e2 *** Got Signal Terminated *** 2024-09-10T11:33:05.892 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:05 smithi144 podman[87279]: 2024-09-10 11:33:05.60195676 +0000 UTC m=+0.115781925 container died 42addb654e4f8b796385356b778525a269a7993950cca0146f524e6c0328c912 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T11:33:06.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:06 smithi144 podman[87279]: 2024-09-10 11:33:06.107001194 +0000 UTC m=+0.620826397 container cleanup 42addb654e4f8b796385356b778525a269a7993950cca0146f524e6c0328c912 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2) 2024-09-10T11:33:06.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:06 smithi144 bash[87279]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144 2024-09-10T11:33:06.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:06 smithi144 podman[87292]: 2024-09-10 11:33:06.308398079 +0000 UTC m=+0.698881628 container remove 42addb654e4f8b796385356b778525a269a7993950cca0146f524e6c0328c912 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, org.label-schema.build-date=20231212, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0) 2024-09-10T11:33:06.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:06 smithi144 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi144.service: Deactivated successfully. 2024-09-10T11:33:06.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:06 smithi144 systemd[1]: Stopped Ceph mon.smithi144 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:33:06.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:06 smithi144 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi144.service: Consumed 13.618s CPU time. 2024-09-10T11:33:07.068 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:06 smithi144 systemd[1]: Starting Ceph mon.smithi144 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:33:07.068 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:06 smithi144 podman[87418]: 2024-09-10 11:33:06.84140878 +0000 UTC m=+0.015849568 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:33:07.068 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:06 smithi144 podman[87418]: 2024-09-10 11:33:06.942558856 +0000 UTC m=+0.116999633 container create 0fc028b6cc37b8df63d191420f95d55ca82542a929a59d69fb10fd62a573b54c (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T11:33:07.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 podman[87418]: 2024-09-10 11:33:07.170732243 +0000 UTC m=+0.345173021 container init 0fc028b6cc37b8df63d191420f95d55ca82542a929a59d69fb10fd62a573b54c (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T11:33:07.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 podman[87418]: 2024-09-10 11:33:07.174185333 +0000 UTC m=+0.348626114 container start 0fc028b6cc37b8df63d191420f95d55ca82542a929a59d69fb10fd62a573b54c (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-10T11:33:07.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T11:33:07.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T11:33:07.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: pidfile_write: ignore empty --pid-file 2024-09-10T11:33:07.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: load: jerasure load: lrc 2024-09-10T11:33:07.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: RocksDB version: 7.9.2 2024-09-10T11:33:07.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Git sha 0 2024-09-10T11:33:07.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T11:33:07.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: DB SUMMARY 2024-09-10T11:33:07.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: DB Session ID: BW00JN3ZC5N8EYT63LH6 2024-09-10T11:33:07.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: CURRENT file: CURRENT 2024-09-10T11:33:07.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T11:33:07.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1508 Bytes 2024-09-10T11:33:07.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi144/store.db dir, Total Num: 1, files: 000032.sst 2024-09-10T11:33:07.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi144/store.db: 000030.log size: 3245130 ; 2024-09-10T11:33:07.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.error_if_exists: 0 2024-09-10T11:33:07.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.create_if_missing: 0 2024-09-10T11:33:07.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.paranoid_checks: 1 2024-09-10T11:33:07.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T11:33:07.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T11:33:07.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T11:33:07.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.env: 0x561d77bf9940 2024-09-10T11:33:07.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T11:33:07.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.info_log: 0x561d7a0218c0 2024-09-10T11:33:07.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T11:33:07.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.statistics: (nil) 2024-09-10T11:33:07.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.use_fsync: 0 2024-09-10T11:33:07.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_log_file_size: 0 2024-09-10T11:33:07.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T11:33:07.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T11:33:07.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T11:33:07.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T11:33:07.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.allow_fallocate: 1 2024-09-10T11:33:07.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T11:33:07.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T11:33:07.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.use_direct_reads: 0 2024-09-10T11:33:07.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T11:33:07.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T11:33:07.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.db_log_dir: 2024-09-10T11:33:07.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.wal_dir: 2024-09-10T11:33:07.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T11:33:07.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T11:33:07.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T11:33:07.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T11:33:07.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T11:33:07.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T11:33:07.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T11:33:07.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T11:33:07.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.write_buffer_manager: 0x561d7a030dc0 2024-09-10T11:33:07.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T11:33:07.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T11:33:07.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T11:33:07.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T11:33:07.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T11:33:07.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T11:33:07.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T11:33:07.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T11:33:07.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.unordered_write: 0 2024-09-10T11:33:07.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T11:33:07.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T11:33:07.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T11:33:07.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T11:33:07.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.row_cache: None 2024-09-10T11:33:07.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.wal_filter: None 2024-09-10T11:33:07.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T11:33:07.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T11:33:07.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.two_write_queues: 0 2024-09-10T11:33:07.440 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T11:33:07.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.wal_compression: 0 2024-09-10T11:33:07.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.atomic_flush: 0 2024-09-10T11:33:07.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T11:33:07.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T11:33:07.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T11:33:07.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.log_readahead_size: 0 2024-09-10T11:33:07.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T11:33:07.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T11:33:07.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T11:33:07.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T11:33:07.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T11:33:07.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T11:33:07.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T11:33:07.441 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_background_jobs: 2 2024-09-10T11:33:07.442 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_background_compactions: -1 2024-09-10T11:33:07.442 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_subcompactions: 1 2024-09-10T11:33:07.442 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T11:33:07.442 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T11:33:07.442 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T11:33:07.442 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T11:33:07.442 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T11:33:07.442 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T11:33:07.442 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T11:33:07.442 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T11:33:07.442 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_open_files: -1 2024-09-10T11:33:07.442 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T11:33:07.442 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T11:33:07.442 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T11:33:07.443 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T11:33:07.443 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_background_flushes: -1 2024-09-10T11:33:07.443 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Compression algorithms supported: 2024-09-10T11:33:07.443 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: kZSTD supported: 0 2024-09-10T11:33:07.443 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: kXpressCompression supported: 0 2024-09-10T11:33:07.443 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: kBZip2Compression supported: 0 2024-09-10T11:33:07.443 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T11:33:07.443 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: kLZ4Compression supported: 1 2024-09-10T11:33:07.443 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: kZlibCompression supported: 1 2024-09-10T11:33:07.443 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T11:33:07.443 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: kSnappyCompression supported: 1 2024-09-10T11:33:07.443 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T11:33:07.443 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T11:33:07.443 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi144/store.db/MANIFEST-000015 2024-09-10T11:33:07.443 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T11:33:07.444 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T11:33:07.444 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.merge_operator: 2024-09-10T11:33:07.444 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compaction_filter: None 2024-09-10T11:33:07.444 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T11:33:07.444 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T11:33:07.444 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T11:33:07.444 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T11:33:07.444 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x561d7a021520) 2024-09-10T11:33:07.444 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: cache_index_and_filter_blocks: 1 2024-09-10T11:33:07.444 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T11:33:07.444 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T11:33:07.444 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: pin_top_level_index_and_filter: 1 2024-09-10T11:33:07.444 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: index_type: 0 2024-09-10T11:33:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: data_block_index_type: 0 2024-09-10T11:33:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: index_shortening: 1 2024-09-10T11:33:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T11:33:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: checksum: 4 2024-09-10T11:33:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: no_block_cache: 0 2024-09-10T11:33:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache: 0x561d7a0151f0 2024-09-10T11:33:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache_name: BinnedLRUCache 2024-09-10T11:33:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache_options: 2024-09-10T11:33:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: capacity : 536870912 2024-09-10T11:33:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: num_shard_bits : 4 2024-09-10T11:33:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: strict_capacity_limit : 0 2024-09-10T11:33:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: high_pri_pool_ratio: 0.000 2024-09-10T11:33:07.445 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_cache_compressed: (nil) 2024-09-10T11:33:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: persistent_cache: (nil) 2024-09-10T11:33:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_size: 4096 2024-09-10T11:33:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_size_deviation: 10 2024-09-10T11:33:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_restart_interval: 16 2024-09-10T11:33:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: index_block_restart_interval: 1 2024-09-10T11:33:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: metadata_block_size: 4096 2024-09-10T11:33:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: partition_filters: 0 2024-09-10T11:33:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: use_delta_encoding: 1 2024-09-10T11:33:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: filter_policy: bloomfilter 2024-09-10T11:33:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: whole_key_filtering: 1 2024-09-10T11:33:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: verify_compression: 0 2024-09-10T11:33:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: read_amp_bytes_per_bit: 0 2024-09-10T11:33:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: format_version: 5 2024-09-10T11:33:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: enable_index_compression: 1 2024-09-10T11:33:07.446 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: block_align: 0 2024-09-10T11:33:07.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: max_auto_readahead_size: 262144 2024-09-10T11:33:07.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: prepopulate_block_cache: 0 2024-09-10T11:33:07.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: initial_auto_readahead_size: 8192 2024-09-10T11:33:07.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T11:33:07.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T11:33:07.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T11:33:07.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compression: NoCompression 2024-09-10T11:33:07.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T11:33:07.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T11:33:07.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T11:33:07.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.num_levels: 7 2024-09-10T11:33:07.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T11:33:07.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T11:33:07.447 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T11:33:07.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T11:33:07.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T11:33:07.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T11:33:07.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T11:33:07.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T11:33:07.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T11:33:07.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T11:33:07.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T11:33:07.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T11:33:07.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T11:33:07.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T11:33:07.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T11:33:07.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T11:33:07.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T11:33:07.448 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T11:33:07.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T11:33:07.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T11:33:07.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T11:33:07.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T11:33:07.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T11:33:07.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T11:33:07.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T11:33:07.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T11:33:07.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T11:33:07.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T11:33:07.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T11:33:07.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T11:33:07.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T11:33:07.449 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T11:33:07.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T11:33:07.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T11:33:07.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T11:33:07.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T11:33:07.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T11:33:07.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T11:33:07.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T11:33:07.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T11:33:07.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T11:33:07.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T11:33:07.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T11:33:07.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T11:33:07.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T11:33:07.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T11:33:07.450 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T11:33:07.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T11:33:07.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T11:33:07.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T11:33:07.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T11:33:07.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T11:33:07.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T11:33:07.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T11:33:07.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.inplace_update_support: 0 2024-09-10T11:33:07.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T11:33:07.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T11:33:07.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T11:33:07.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T11:33:07.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.bloom_locality: 0 2024-09-10T11:33:07.451 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.max_successive_merges: 0 2024-09-10T11:33:07.452 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T11:33:07.452 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T11:33:07.452 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T11:33:07.452 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T11:33:07.452 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.ttl: 2592000 2024-09-10T11:33:07.452 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T11:33:07.452 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T11:33:07.452 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T11:33:07.452 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.enable_blob_files: false 2024-09-10T11:33:07.452 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.min_blob_size: 0 2024-09-10T11:33:07.452 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T11:33:07.452 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T11:33:07.452 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T11:33:07.452 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T11:33:07.453 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T11:33:07.453 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T11:33:07.453 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T11:33:07.453 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T11:33:07.453 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-10T11:33:07.453 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi144/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12186, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-09-10T11:33:07.453 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-09-10T11:33:07.453 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: e484da36-1ce5-4d20-8c49-e5c565568da8 2024-09-10T11:33:07.453 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725967987200516, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-09-10T11:33:07.453 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-09-10T11:33:07.453 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725967987210042, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 2820148, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12187, "largest_seqno": 14875, "table_properties": {"data_size": 2809584, "index_size": 6317, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3205, "raw_key_size": 28037, "raw_average_key_size": 22, "raw_value_size": 2785111, "raw_average_value_size": 2206, "num_data_blocks": 287, "num_entries": 1262, "num_filter_entries": 1262, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725967987, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "e484da36-1ce5-4d20-8c49-e5c565568da8", "db_session_id": "BW00JN3ZC5N8EYT63LH6", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-09-10T11:33:07.453 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725967987210165, "job": 1, "event": "recovery_finished"} 2024-09-10T11:33:07.453 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-09-10T11:33:07.453 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-10T11:33:07.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi144/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T11:33:07.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561d7a042e00 2024-09-10T11:33:07.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: DB pointer 0x561d7a13e000 2024-09-10T11:33:07.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T11:33:07.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T11:33:07.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** DB Stats ** 2024-09-10T11:33:07.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T11:33:07.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T11:33:07.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T11:33:07.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T11:33:07.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T11:33:07.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T11:33:07.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T11:33:07.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-09-10T11:33:07.454 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** Compaction Stats [default] ** 2024-09-10T11:33:07.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T11:33:07.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T11:33:07.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: L0 1/0 2.69 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 611.4 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-10T11:33:07.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: L6 1/0 6.01 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-09-10T11:33:07.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Sum 2/0 8.70 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 611.4 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-10T11:33:07.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 611.4 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-10T11:33:07.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-09-10T11:33:07.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** Compaction Stats [default] ** 2024-09-10T11:33:07.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T11:33:07.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T11:33:07.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 611.4 0.00 0.00 1 0.004 0 0 0.0 0.0 2024-09-10T11:33:07.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-09-10T11:33:07.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T11:33:07.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-09-10T11:33:07.455 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T11:33:07.456 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-09-10T11:33:07.456 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T11:33:07.456 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T11:33:07.456 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T11:33:07.456 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T11:33:07.456 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Cumulative compaction: 0.00 GB write, 222.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T11:33:07.456 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Interval compaction: 0.00 GB write, 222.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T11:33:07.456 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T11:33:07.456 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Block cache BinnedLRUCache@0x561d7a0151f0#2 capacity: 512.00 MB usage: 9.86 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-10T11:33:07.456 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,3.28 KB,0.000625849%) IndexBlock(1,6.58 KB,0.00125468%) Misc(1,0.00 KB,0%) 2024-09-10T11:33:07.456 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-09-10T11:33:07.456 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T11:33:07.456 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: starting mon.smithi144 rank 0 at public addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] at bind addrs [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi144 fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:33:07.456 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: mon.smithi144@-1(???) e2 preinit fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:33:07.457 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: mon.smithi144@-1(???).mds e22 new map 2024-09-10T11:33:07.457 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: mon.smithi144@-1(???).mds e22 print_map 2024-09-10T11:33:07.457 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: e22 2024-09-10T11:33:07.457 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: btime 1970-01-01T00:00:00:000000+0000 2024-09-10T11:33:07.457 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:33:07.457 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:33:07.457 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: legacy client fscid: 1 2024-09-10T11:33:07.457 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-09-10T11:33:07.457 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Filesystem 'cephfs' (1) 2024-09-10T11:33:07.457 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: fs_name cephfs 2024-09-10T11:33:07.457 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: epoch 22 2024-09-10T11:33:07.457 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T11:33:07.457 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:33:07.457 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: modified 2024-09-10T11:31:50.444180+0000 2024-09-10T11:33:07.458 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: tableserver 0 2024-09-10T11:33:07.458 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: root 0 2024-09-10T11:33:07.458 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: session_timeout 60 2024-09-10T11:33:07.458 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: session_autoclose 300 2024-09-10T11:33:07.458 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: max_file_size 1099511627776 2024-09-10T11:33:07.458 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: max_xattr_size 65536 2024-09-10T11:33:07.458 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: required_client_features {} 2024-09-10T11:33:07.458 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: last_failure 0 2024-09-10T11:33:07.458 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: last_failure_osd_epoch 47 2024-09-10T11:33:07.458 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:33:07.458 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: max_mds 2 2024-09-10T11:33:07.458 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: in 0,1 2024-09-10T11:33:07.458 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: up {0=14488,1=24267} 2024-09-10T11:33:07.458 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: failed 2024-09-10T11:33:07.458 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: damaged 2024-09-10T11:33:07.459 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: stopped 2024-09-10T11:33:07.459 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: data_pools [3] 2024-09-10T11:33:07.459 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: metadata_pool 2 2024-09-10T11:33:07.459 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: inline_data enabled 2024-09-10T11:33:07.459 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: balancer 2024-09-10T11:33:07.459 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: bal_rank_mask -1 2024-09-10T11:33:07.459 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: standby_count_wanted 1 2024-09-10T11:33:07.459 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: qdb_cluster leader: 0 members: 2024-09-10T11:33:07.459 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: [mds.cephfs.smithi144.jaoyvs{0:14488} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:07.459 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: [mds.cephfs.smithi144.bfyvwv{0:14498} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/1752150537,v1:172.21.15.144:6827/1752150537] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:07.459 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: [mds.cephfs.smithi178.nmhrmh{1:24267} state up:active seq 4 export targets 0 join_fscid=1 addr [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:07.459 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: [mds.cephfs.smithi178.czjrsa{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:07.459 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-09-10T11:33:07.459 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: 2024-09-10T11:33:07.459 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: mon.smithi144@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-09-10T11:33:07.460 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: mon.smithi144@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-09-10T11:33:07.460 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: mon.smithi144@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-09-10T11:33:07.460 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: mon.smithi144@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-09-10T11:33:07.460 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 ceph-mon[87436]: mon.smithi144@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-09-10T11:33:07.460 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 bash[87418]: 0fc028b6cc37b8df63d191420f95d55ca82542a929a59d69fb10fd62a573b54c 2024-09-10T11:33:07.460 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:07 smithi144 systemd[1]: Started Ceph mon.smithi144 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:33:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: pgmap v89: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: pgmap v90: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: mon.smithi144 calling monitor election 2024-09-10T11:33:08.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: mon.smithi144 is new leader, mons smithi144,smithi178 in quorum (ranks 0,1) 2024-09-10T11:33:08.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: monmap epoch 2 2024-09-10T11:33:08.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:33:08.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: last_changed 2024-09-10T11:22:15.900195+0000 2024-09-10T11:33:08.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: created 2024-09-10T11:19:28.938528+0000 2024-09-10T11:33:08.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: min_mon_release 18 (reef) 2024-09-10T11:33:08.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: election_strategy: 1 2024-09-10T11:33:08.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: 0: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.smithi144 2024-09-10T11:33:08.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.smithi178 2024-09-10T11:33:08.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:33:08.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: osdmap e49: 6 total, 6 up, 6 in 2024-09-10T11:33:08.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: mgrmap e32: smithi144.bkogfs(active, since 2m), standbys: smithi178.vmvxuf 2024-09-10T11:33:08.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T11:33:08.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:33:08.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:33:08.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: from='mgr.24417 ' entity='' 2024-09-10T11:33:08.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:08 smithi144 ceph-mon[87436]: mgrmap e33: smithi144.bkogfs(active, since 2m), standbys: smithi178.vmvxuf 2024-09-10T11:33:08.991 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: pgmap v89: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:08.991 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: pgmap v90: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:08.991 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: mon.smithi144 calling monitor election 2024-09-10T11:33:08.991 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: mon.smithi144 is new leader, mons smithi144,smithi178 in quorum (ranks 0,1) 2024-09-10T11:33:08.992 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: monmap epoch 2 2024-09-10T11:33:08.992 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:33:08.992 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: last_changed 2024-09-10T11:22:15.900195+0000 2024-09-10T11:33:08.992 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: created 2024-09-10T11:19:28.938528+0000 2024-09-10T11:33:08.992 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: min_mon_release 18 (reef) 2024-09-10T11:33:08.992 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: election_strategy: 1 2024-09-10T11:33:08.992 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: 0: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.smithi144 2024-09-10T11:33:08.992 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.smithi178 2024-09-10T11:33:08.992 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:33:08.992 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: osdmap e49: 6 total, 6 up, 6 in 2024-09-10T11:33:08.992 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: mgrmap e32: smithi144.bkogfs(active, since 2m), standbys: smithi178.vmvxuf 2024-09-10T11:33:08.992 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T11:33:08.993 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:33:08.993 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:33:08.993 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: from='mgr.24417 ' entity='' 2024-09-10T11:33:08.993 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:08 smithi178 ceph-mon[33514]: mgrmap e33: smithi144.bkogfs(active, since 2m), standbys: smithi178.vmvxuf 2024-09-10T11:33:12.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:12 smithi144 ceph-mon[87436]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:33:12.348 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:12 smithi178 ceph-mon[33514]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:33:13.721 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:13 smithi178 ceph-mon[33514]: Standby manager daemon smithi178.vmvxuf restarted 2024-09-10T11:33:13.721 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:13 smithi178 ceph-mon[33514]: Standby manager daemon smithi178.vmvxuf started 2024-09-10T11:33:13.721 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:13 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/3552670824' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/crt"}]: dispatch 2024-09-10T11:33:13.721 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:13 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/3552670824' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T11:33:13.722 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:13 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/3552670824' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/key"}]: dispatch 2024-09-10T11:33:13.722 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:13 smithi178 ceph-mon[33514]: from='mgr.? 172.21.15.178:0/3552670824' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T11:33:13.727 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:13 smithi144 ceph-mon[87436]: Standby manager daemon smithi178.vmvxuf restarted 2024-09-10T11:33:13.727 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:13 smithi144 ceph-mon[87436]: Standby manager daemon smithi178.vmvxuf started 2024-09-10T11:33:13.727 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:13 smithi144 ceph-mon[87436]: from='mgr.? 172.21.15.178:0/3552670824' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/crt"}]: dispatch 2024-09-10T11:33:13.727 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:13 smithi144 ceph-mon[87436]: from='mgr.? 172.21.15.178:0/3552670824' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T11:33:13.727 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:13 smithi144 ceph-mon[87436]: from='mgr.? 172.21.15.178:0/3552670824' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi178.vmvxuf/key"}]: dispatch 2024-09-10T11:33:13.727 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:13 smithi144 ceph-mon[87436]: from='mgr.? 172.21.15.178:0/3552670824' entity='mgr.smithi178.vmvxuf' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T11:33:14.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: mgrmap e34: smithi144.bkogfs(active, since 3m), standbys: smithi178.vmvxuf 2024-09-10T11:33:14.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: Active manager daemon smithi144.bkogfs restarted 2024-09-10T11:33:14.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: Activating manager daemon smithi144.bkogfs 2024-09-10T11:33:14.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: osdmap e50: 6 total, 6 up, 6 in 2024-09-10T11:33:14.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: mgrmap e35: smithi144.bkogfs(active, starting, since 0.00548131s), standbys: smithi178.vmvxuf 2024-09-10T11:33:14.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:33:14.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:33:14.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jaoyvs"}]: dispatch 2024-09-10T11:33:14.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:33:14.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.nmhrmh"}]: dispatch 2024-09-10T11:33:14.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.czjrsa"}]: dispatch 2024-09-10T11:33:14.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr metadata", "who": "smithi144.bkogfs", "id": "smithi144.bkogfs"}]: dispatch 2024-09-10T11:33:14.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr metadata", "who": "smithi178.vmvxuf", "id": "smithi178.vmvxuf"}]: dispatch 2024-09-10T11:33:14.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:33:14.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:33:14.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:33:14.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:33:14.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:33:14.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:33:14.707 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T11:33:14.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T11:33:14.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T11:33:14.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: Manager daemon smithi144.bkogfs is now available 2024-09-10T11:33:14.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:33:14.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:33:14.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/mirror_snapshot_schedule"}]: dispatch 2024-09-10T11:33:14.708 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/trash_purge_schedule"}]: dispatch 2024-09-10T11:33:14.731 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: mgrmap e34: smithi144.bkogfs(active, since 3m), standbys: smithi178.vmvxuf 2024-09-10T11:33:14.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: Active manager daemon smithi144.bkogfs restarted 2024-09-10T11:33:14.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: Activating manager daemon smithi144.bkogfs 2024-09-10T11:33:14.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: osdmap e50: 6 total, 6 up, 6 in 2024-09-10T11:33:14.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: mgrmap e35: smithi144.bkogfs(active, starting, since 0.00548131s), standbys: smithi178.vmvxuf 2024-09-10T11:33:14.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:33:14.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:33:14.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jaoyvs"}]: dispatch 2024-09-10T11:33:14.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:33:14.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.nmhrmh"}]: dispatch 2024-09-10T11:33:14.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.czjrsa"}]: dispatch 2024-09-10T11:33:14.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr metadata", "who": "smithi144.bkogfs", "id": "smithi144.bkogfs"}]: dispatch 2024-09-10T11:33:14.732 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr metadata", "who": "smithi178.vmvxuf", "id": "smithi178.vmvxuf"}]: dispatch 2024-09-10T11:33:14.733 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:33:14.733 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:33:14.733 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:33:14.733 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:33:14.733 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:33:14.733 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:33:14.733 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T11:33:14.733 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T11:33:14.733 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T11:33:14.733 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: Manager daemon smithi144.bkogfs is now available 2024-09-10T11:33:14.733 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:33:14.733 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:33:14.733 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/mirror_snapshot_schedule"}]: dispatch 2024-09-10T11:33:14.733 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:14 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi144.bkogfs/trash_purge_schedule"}]: dispatch 2024-09-10T11:33:15.613 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:15 smithi178 ceph-mon[33514]: mgrmap e36: smithi144.bkogfs(active, since 1.00967s), standbys: smithi178.vmvxuf 2024-09-10T11:33:15.613 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:15 smithi178 ceph-mon[33514]: pgmap v3: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:33:15.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:15 smithi144 ceph-mon[87436]: mgrmap e36: smithi144.bkogfs(active, since 1.00967s), standbys: smithi178.vmvxuf 2024-09-10T11:33:15.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:15 smithi144 ceph-mon[87436]: pgmap v3: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:33:16.811 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:16 smithi144 ceph-mon[87436]: [10/Sep/2024:11:33:14] ENGINE Bus STARTING 2024-09-10T11:33:16.811 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:16 smithi144 ceph-mon[87436]: [10/Sep/2024:11:33:15] ENGINE Serving on https://172.21.15.144:7150 2024-09-10T11:33:16.811 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:16 smithi144 ceph-mon[87436]: [10/Sep/2024:11:33:15] ENGINE Error in HTTPServer.serve 2024-09-10T11:33:16.812 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: Traceback (most recent call last): 2024-09-10T11:33:16.812 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T11:33:16.812 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: self._connections.run(self.expiration_interval) 2024-09-10T11:33:16.812 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T11:33:16.812 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: self._run(expiration_interval) 2024-09-10T11:33:16.812 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T11:33:16.812 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T11:33:16.812 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T11:33:16.812 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T11:33:16.812 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T11:33:16.812 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: s = self.context.wrap_socket( 2024-09-10T11:33:16.812 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T11:33:16.812 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: return self.sslsocket_class._create( 2024-09-10T11:33:16.813 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T11:33:16.813 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: self.do_handshake() 2024-09-10T11:33:16.813 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T11:33:16.813 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: self._sslobj.do_handshake() 2024-09-10T11:33:16.813 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T11:33:16.813 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:16 smithi144 ceph-mon[87436]: [10/Sep/2024:11:33:15] ENGINE Serving on http://172.21.15.144:8765 2024-09-10T11:33:16.813 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:16 smithi144 ceph-mon[87436]: [10/Sep/2024:11:33:15] ENGINE Bus STARTED 2024-09-10T11:33:16.813 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:16 smithi144 ceph-mon[87436]: pgmap v4: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:33:16.813 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:16 smithi144 ceph-mon[87436]: mgrmap e37: smithi144.bkogfs(active, since 2s), standbys: smithi178.vmvxuf 2024-09-10T11:33:16.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:16 smithi178 ceph-mon[33514]: [10/Sep/2024:11:33:14] ENGINE Bus STARTING 2024-09-10T11:33:16.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:16 smithi178 ceph-mon[33514]: [10/Sep/2024:11:33:15] ENGINE Serving on https://172.21.15.144:7150 2024-09-10T11:33:16.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:16 smithi178 ceph-mon[33514]: [10/Sep/2024:11:33:15] ENGINE Error in HTTPServer.serve 2024-09-10T11:33:16.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Traceback (most recent call last): 2024-09-10T11:33:16.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T11:33:16.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: self._connections.run(self.expiration_interval) 2024-09-10T11:33:16.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T11:33:16.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: self._run(expiration_interval) 2024-09-10T11:33:16.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T11:33:16.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T11:33:16.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T11:33:16.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T11:33:16.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T11:33:16.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: s = self.context.wrap_socket( 2024-09-10T11:33:16.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T11:33:16.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: return self.sslsocket_class._create( 2024-09-10T11:33:16.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T11:33:16.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: self.do_handshake() 2024-09-10T11:33:16.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T11:33:16.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: self._sslobj.do_handshake() 2024-09-10T11:33:16.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T11:33:16.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:16 smithi178 ceph-mon[33514]: [10/Sep/2024:11:33:15] ENGINE Serving on http://172.21.15.144:8765 2024-09-10T11:33:16.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:16 smithi178 ceph-mon[33514]: [10/Sep/2024:11:33:15] ENGINE Bus STARTED 2024-09-10T11:33:16.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:16 smithi178 ceph-mon[33514]: pgmap v4: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:33:16.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:16 smithi178 ceph-mon[33514]: mgrmap e37: smithi144.bkogfs(active, since 2s), standbys: smithi178.vmvxuf 2024-09-10T11:33:17.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:17.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:17.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:17.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:17.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:17 smithi144 ceph-mon[87436]: pgmap v5: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:33:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:17 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:17 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:17 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:17 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:17 smithi178 ceph-mon[33514]: pgmap v5: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:33:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:19 smithi144 ceph-mon[87436]: mgrmap e38: smithi144.bkogfs(active, since 4s), standbys: smithi178.vmvxuf 2024-09-10T11:33:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:19 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:19.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:19 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:19.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:19 smithi178 ceph-mon[33514]: mgrmap e38: smithi144.bkogfs(active, since 4s), standbys: smithi178.vmvxuf 2024-09-10T11:33:19.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:19 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:19.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:19 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:20 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:20 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:20 smithi178 ceph-mon[33514]: pgmap v6: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:33:20.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:20 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:20.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:20 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:20.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:20 smithi144 ceph-mon[87436]: pgmap v6: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:33:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:22 smithi178 ceph-mon[33514]: pgmap v7: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-10T11:33:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:22 smithi144 ceph-mon[87436]: pgmap v7: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 31 KiB/s rd, 0 B/s wr, 19 op/s 2024-09-10T11:33:24.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: Detected new or changed devices on smithi144 2024-09-10T11:33:24.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:24.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:33:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: Updating smithi144:/etc/ceph/ceph.conf 2024-09-10T11:33:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T11:33:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:33:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:33:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:33:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:33:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:33:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:33:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: pgmap v8: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-10T11:33:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:24.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:24.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:23 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: Detected new or changed devices on smithi144 2024-09-10T11:33:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:33:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: Updating smithi144:/etc/ceph/ceph.conf 2024-09-10T11:33:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: Updating smithi178:/etc/ceph/ceph.conf 2024-09-10T11:33:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:33:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T11:33:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: Updating smithi144:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:33:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: Updating smithi178:/etc/ceph/ceph.client.admin.keyring 2024-09-10T11:33:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: Updating smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:33:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: Updating smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T11:33:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: pgmap v8: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 24 KiB/s rd, 0 B/s wr, 15 op/s 2024-09-10T11:33:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:24.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:24.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:24.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:24.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:24 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:33:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:24 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:33:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:24 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-10T11:33:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:24 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:33:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:24 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:33:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:24 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-09-10T11:33:26.570 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:26 smithi178 ceph-mon[33514]: Upgrade: Updating mon.smithi178 2024-09-10T11:33:26.570 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:26 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:26.570 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:26 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T11:33:26.570 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:26 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T11:33:26.570 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:26 smithi178 ceph-mon[33514]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:26.570 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:26 smithi178 ceph-mon[33514]: Deploying daemon mon.smithi178 on smithi178 2024-09-10T11:33:26.570 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:26 smithi178 ceph-mon[33514]: pgmap v9: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-10T11:33:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:26 smithi144 ceph-mon[87436]: Upgrade: Updating mon.smithi178 2024-09-10T11:33:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T11:33:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T11:33:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:26 smithi144 ceph-mon[87436]: Deploying daemon mon.smithi178 on smithi178 2024-09-10T11:33:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:26 smithi144 ceph-mon[87436]: pgmap v9: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 21 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-10T11:33:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:26 smithi178 systemd[1]: Stopping Ceph mon.smithi178 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:33:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:27 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178[33496]: 2024-09-10T11:33:27.114+0000 7fec3bcc6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi178 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:33:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:27 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178[33496]: 2024-09-10T11:33:27.114+0000 7fec3bcc6700 -1 mon.smithi178@1(peon) e2 *** Got Signal Terminated *** 2024-09-10T11:33:27.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:27 smithi178 podman[65779]: 2024-09-10 11:33:27.176406621 +0000 UTC m=+0.174580057 container died 3c1ba210ed3fb6c058a87286198a1cc8c1ce886d2ef5bce99e2c94cce123e6a2 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20231212, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-09-10T11:33:27.830 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:27 smithi178 podman[65779]: 2024-09-10 11:33:27.559341845 +0000 UTC m=+0.557515287 container cleanup 3c1ba210ed3fb6c058a87286198a1cc8c1ce886d2ef5bce99e2c94cce123e6a2 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=HEAD, maintainer=Guillaume Abrioux , ceph=True) 2024-09-10T11:33:27.831 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:27 smithi178 bash[65779]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178 2024-09-10T11:33:28.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:27 smithi178 podman[65792]: 2024-09-10 11:33:27.83021736 +0000 UTC m=+0.653716740 container remove 3c1ba210ed3fb6c058a87286198a1cc8c1ce886d2ef5bce99e2c94cce123e6a2 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, org.label-schema.vendor=CentOS) 2024-09-10T11:33:28.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:27 smithi178 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi178.service: Deactivated successfully. 2024-09-10T11:33:28.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:27 smithi178 systemd[1]: Stopped Ceph mon.smithi178 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:33:28.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:27 smithi178 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi178.service: Consumed 7.905s CPU time. 2024-09-10T11:33:28.397 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 systemd[1]: Starting Ceph mon.smithi178 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:33:28.741 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 podman[65920]: 2024-09-10 11:33:28.299465969 +0000 UTC m=+0.016023237 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:33:28.741 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 podman[65920]: 2024-09-10 11:33:28.537161997 +0000 UTC m=+0.253719251 container create a1b7b65c960cf7ec7ca9eddcf74c6a14ce0a9878a28924bd5e7af2fc170317c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True) 2024-09-10T11:33:28.773 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 podman[65920]: 2024-09-10 11:33:28.740336014 +0000 UTC m=+0.456893277 container init a1b7b65c960cf7ec7ca9eddcf74c6a14ce0a9878a28924bd5e7af2fc170317c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T11:33:28.773 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 podman[65920]: 2024-09-10 11:33:28.743945636 +0000 UTC m=+0.460502908 container start a1b7b65c960cf7ec7ca9eddcf74c6a14ce0a9878a28924bd5e7af2fc170317c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T11:33:28.773 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T11:33:28.773 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable), process ceph-mon, pid 2 2024-09-10T11:33:28.773 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: pidfile_write: ignore empty --pid-file 2024-09-10T11:33:28.773 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: load: jerasure load: lrc 2024-09-10T11:33:28.773 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: RocksDB version: 7.9.2 2024-09-10T11:33:28.773 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Git sha 0 2024-09-10T11:33:28.774 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Compile date 2024-09-09 20:43:47 2024-09-10T11:33:28.774 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: DB SUMMARY 2024-09-10T11:33:28.774 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: DB Session ID: SRPK1N57L4FAHTV40PH2 2024-09-10T11:33:28.774 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: CURRENT file: CURRENT 2024-09-10T11:33:28.774 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T11:33:28.774 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1402 Bytes 2024-09-10T11:33:28.774 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi178/store.db dir, Total Num: 1, files: 000027.sst 2024-09-10T11:33:28.774 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi178/store.db: 000025.log size: 7781677 ; 2024-09-10T11:33:28.774 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.error_if_exists: 0 2024-09-10T11:33:28.774 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.create_if_missing: 0 2024-09-10T11:33:28.774 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.paranoid_checks: 1 2024-09-10T11:33:28.774 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-10T11:33:28.774 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T11:33:28.774 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-10T11:33:28.775 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.env: 0x5591ff486940 2024-09-10T11:33:28.775 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.fs: PosixFileSystem 2024-09-10T11:33:28.775 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.info_log: 0x559200f058c0 2024-09-10T11:33:28.775 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T11:33:28.775 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.statistics: (nil) 2024-09-10T11:33:28.775 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.use_fsync: 0 2024-09-10T11:33:28.775 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_log_file_size: 0 2024-09-10T11:33:28.775 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T11:33:28.775 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T11:33:28.775 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T11:33:28.775 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T11:33:28.775 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.allow_fallocate: 1 2024-09-10T11:33:28.775 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T11:33:28.775 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T11:33:29.068 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.use_direct_reads: 0 2024-09-10T11:33:29.068 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T11:33:29.068 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T11:33:29.068 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.db_log_dir: 2024-09-10T11:33:29.068 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.wal_dir: 2024-09-10T11:33:29.068 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T11:33:29.068 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T11:33:29.068 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T11:33:29.068 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T11:33:29.069 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T11:33:29.069 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T11:33:29.069 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T11:33:29.069 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T11:33:29.069 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.write_buffer_manager: 0x559200f14dc0 2024-09-10T11:33:29.069 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T11:33:29.069 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T11:33:29.069 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T11:33:29.069 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T11:33:29.069 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T11:33:29.069 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T11:33:29.069 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T11:33:29.069 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T11:33:29.069 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.unordered_write: 0 2024-09-10T11:33:29.069 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T11:33:29.070 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T11:33:29.070 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T11:33:29.070 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T11:33:29.070 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.row_cache: None 2024-09-10T11:33:29.070 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.wal_filter: None 2024-09-10T11:33:29.070 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T11:33:29.070 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T11:33:29.070 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.two_write_queues: 0 2024-09-10T11:33:29.070 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T11:33:29.070 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.wal_compression: 0 2024-09-10T11:33:29.070 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.atomic_flush: 0 2024-09-10T11:33:29.070 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T11:33:29.070 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T11:33:29.070 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T11:33:29.071 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.log_readahead_size: 0 2024-09-10T11:33:29.072 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T11:33:29.072 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T11:33:29.072 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T11:33:29.072 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T11:33:29.072 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T11:33:29.072 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T11:33:29.072 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-10T11:33:29.072 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_background_jobs: 2 2024-09-10T11:33:29.072 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_background_compactions: -1 2024-09-10T11:33:29.072 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_subcompactions: 1 2024-09-10T11:33:29.073 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T11:33:29.073 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T11:33:29.073 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T11:33:29.073 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T11:33:29.073 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T11:33:29.073 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T11:33:29.073 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T11:33:29.073 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T11:33:29.073 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_open_files: -1 2024-09-10T11:33:29.073 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T11:33:29.073 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T11:33:29.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T11:33:29.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T11:33:29.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_background_flushes: -1 2024-09-10T11:33:29.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Compression algorithms supported: 2024-09-10T11:33:29.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: kZSTD supported: 0 2024-09-10T11:33:29.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: kXpressCompression supported: 0 2024-09-10T11:33:29.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: kBZip2Compression supported: 0 2024-09-10T11:33:29.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T11:33:29.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: kLZ4Compression supported: 1 2024-09-10T11:33:29.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: kZlibCompression supported: 1 2024-09-10T11:33:29.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T11:33:29.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: kSnappyCompression supported: 1 2024-09-10T11:33:29.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T11:33:29.075 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-10T11:33:29.075 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi178/store.db/MANIFEST-000010 2024-09-10T11:33:29.075 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-10T11:33:29.075 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T11:33:29.075 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.merge_operator: 2024-09-10T11:33:29.075 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compaction_filter: None 2024-09-10T11:33:29.075 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T11:33:29.075 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T11:33:29.075 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T11:33:29.075 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T11:33:29.075 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559200f05520) 2024-09-10T11:33:29.075 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: cache_index_and_filter_blocks: 1 2024-09-10T11:33:29.076 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T11:33:29.076 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T11:33:29.076 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: pin_top_level_index_and_filter: 1 2024-09-10T11:33:29.076 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: index_type: 0 2024-09-10T11:33:29.076 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: data_block_index_type: 0 2024-09-10T11:33:29.076 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: index_shortening: 1 2024-09-10T11:33:29.076 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T11:33:29.076 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: checksum: 4 2024-09-10T11:33:29.076 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: no_block_cache: 0 2024-09-10T11:33:29.076 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_cache: 0x559200ef91f0 2024-09-10T11:33:29.076 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_cache_name: BinnedLRUCache 2024-09-10T11:33:29.076 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_cache_options: 2024-09-10T11:33:29.076 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: capacity : 536870912 2024-09-10T11:33:29.076 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: num_shard_bits : 4 2024-09-10T11:33:29.077 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: strict_capacity_limit : 0 2024-09-10T11:33:29.077 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: high_pri_pool_ratio: 0.000 2024-09-10T11:33:29.077 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_cache_compressed: (nil) 2024-09-10T11:33:29.077 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: persistent_cache: (nil) 2024-09-10T11:33:29.077 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_size: 4096 2024-09-10T11:33:29.077 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_size_deviation: 10 2024-09-10T11:33:29.077 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_restart_interval: 16 2024-09-10T11:33:29.077 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: index_block_restart_interval: 1 2024-09-10T11:33:29.077 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: metadata_block_size: 4096 2024-09-10T11:33:29.077 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: partition_filters: 0 2024-09-10T11:33:29.077 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: use_delta_encoding: 1 2024-09-10T11:33:29.077 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: filter_policy: bloomfilter 2024-09-10T11:33:29.077 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: whole_key_filtering: 1 2024-09-10T11:33:29.077 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: verify_compression: 0 2024-09-10T11:33:29.077 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: read_amp_bytes_per_bit: 0 2024-09-10T11:33:29.078 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: format_version: 5 2024-09-10T11:33:29.078 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: enable_index_compression: 1 2024-09-10T11:33:29.078 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: block_align: 0 2024-09-10T11:33:29.078 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: max_auto_readahead_size: 262144 2024-09-10T11:33:29.078 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: prepopulate_block_cache: 0 2024-09-10T11:33:29.078 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: initial_auto_readahead_size: 8192 2024-09-10T11:33:29.078 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: num_file_reads_for_auto_readahead: 2 2024-09-10T11:33:29.078 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T11:33:29.078 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T11:33:29.078 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compression: NoCompression 2024-09-10T11:33:29.078 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T11:33:29.078 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T11:33:29.078 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T11:33:29.079 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.num_levels: 7 2024-09-10T11:33:29.079 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T11:33:29.079 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T11:33:29.079 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T11:33:29.079 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T11:33:29.079 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T11:33:29.079 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T11:33:29.079 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T11:33:29.079 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T11:33:29.079 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T11:33:29.079 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T11:33:29.079 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-10T11:33:29.079 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-10T11:33:29.079 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T11:33:29.079 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T11:33:29.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T11:33:29.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T11:33:29.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T11:33:29.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-10T11:33:29.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T11:33:29.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T11:33:29.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-10T11:33:29.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T11:33:29.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T11:33:29.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T11:33:29.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T11:33:29.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T11:33:29.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T11:33:29.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T11:33:29.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T11:33:29.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T11:33:29.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T11:33:29.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T11:33:29.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T11:33:29.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T11:33:29.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T11:33:29.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T11:33:29.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T11:33:29.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T11:33:29.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-10T11:33:29.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.arena_block_size: 1048576 2024-09-10T11:33:29.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T11:33:29.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T11:33:29.082 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T11:33:29.082 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T11:33:29.082 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T11:33:29.082 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T11:33:29.082 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T11:33:29.082 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T11:33:29.082 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T11:33:29.082 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T11:33:29.082 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T11:33:29.082 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T11:33:29.082 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T11:33:29.082 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T11:33:29.082 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.inplace_update_support: 0 2024-09-10T11:33:29.083 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T11:33:29.083 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T11:33:29.083 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T11:33:29.083 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T11:33:29.083 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.bloom_locality: 0 2024-09-10T11:33:29.083 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.max_successive_merges: 0 2024-09-10T11:33:29.083 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T11:33:29.083 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T11:33:29.083 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T11:33:29.083 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T11:33:29.083 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.ttl: 2592000 2024-09-10T11:33:29.083 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T11:33:29.083 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-10T11:33:29.083 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-10T11:33:29.083 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.enable_blob_files: false 2024-09-10T11:33:29.084 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.min_blob_size: 0 2024-09-10T11:33:29.084 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T11:33:29.084 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T11:33:29.084 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T11:33:29.084 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T11:33:29.084 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-10T11:33:29.084 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-10T11:33:29.084 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.blob_file_starting_level: 0 2024-09-10T11:33:29.084 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-10T11:33:29.084 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-10T11:33:29.085 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi178/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12284, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-09-10T11:33:29.085 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-09-10T11:33:29.085 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 97046055-eaba-4a4e-bd53-c95f3747ca37 2024-09-10T11:33:29.085 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725968008771050, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-09-10T11:33:29.085 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-09-10T11:33:29.085 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725968008791525, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 4651133, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12289, "largest_seqno": 15739, "table_properties": {"data_size": 4638599, "index_size": 7775, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3717, "raw_key_size": 32930, "raw_average_key_size": 22, "raw_value_size": 4609731, "raw_average_value_size": 3135, "num_data_blocks": 351, "num_entries": 1470, "num_filter_entries": 1470, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1725968008, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "97046055-eaba-4a4e-bd53-c95f3747ca37", "db_session_id": "SRPK1N57L4FAHTV40PH2", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-09-10T11:33:29.085 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725968008791665, "job": 1, "event": "recovery_finished"} 2024-09-10T11:33:29.085 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-09-10T11:33:29.085 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-09-10T11:33:29.085 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi178/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T11:33:29.085 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559200f26e00 2024-09-10T11:33:29.086 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: DB pointer 0x559201022000 2024-09-10T11:33:29.086 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-10T11:33:29.086 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-10T11:33:29.086 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: ** DB Stats ** 2024-09-10T11:33:29.086 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T11:33:29.086 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T11:33:29.086 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T11:33:29.088 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T11:33:29.088 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T11:33:29.088 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T11:33:29.089 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T11:33:29.089 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: 2024-09-10T11:33:29.089 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: ** Compaction Stats [default] ** 2024-09-10T11:33:29.089 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T11:33:29.089 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-10T11:33:29.089 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: L0 1/0 4.44 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 631.1 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-09-10T11:33:29.089 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: L6 1/0 6.01 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-09-10T11:33:29.089 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Sum 2/0 10.45 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 631.1 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-09-10T11:33:29.089 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 631.1 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-09-10T11:33:29.089 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: 2024-09-10T11:33:29.089 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: ** Compaction Stats [default] ** 2024-09-10T11:33:29.089 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-10T11:33:29.089 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T11:33:29.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 631.1 0.01 0.00 1 0.007 0 0 0.0 0.0 2024-09-10T11:33:29.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: 2024-09-10T11:33:29.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-10T11:33:29.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: 2024-09-10T11:33:29.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T11:33:29.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Flush(GB): cumulative 0.004, interval 0.004 2024-09-10T11:33:29.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T11:33:29.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T11:33:29.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T11:33:29.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T11:33:29.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Cumulative compaction: 0.00 GB write, 188.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T11:33:29.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Interval compaction: 0.00 GB write, 188.40 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T11:33:29.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T11:33:29.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Block cache BinnedLRUCache@0x559200ef91f0#2 capacity: 512.00 MB usage: 12.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-10T11:33:29.091 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,8.08 KB,0.00154078%) Misc(1,0.00 KB,0%) 2024-09-10T11:33:29.091 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: 2024-09-10T11:33:29.091 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T11:33:29.091 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: starting mon.smithi178 rank 1 at public addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] at bind addrs [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi178 fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:33:29.091 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: mon.smithi178@-1(???) e2 preinit fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:33:29.091 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: mon.smithi178@-1(???).mds e23 new map 2024-09-10T11:33:29.091 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: mon.smithi178@-1(???).mds e23 print_map 2024-09-10T11:33:29.091 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: e23 2024-09-10T11:33:29.091 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: btime 2024-09-10T11:33:11:119769+0000 2024-09-10T11:33:29.091 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:33:29.091 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:33:29.091 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: legacy client fscid: 1 2024-09-10T11:33:29.091 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: 2024-09-10T11:33:29.091 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: Filesystem 'cephfs' (1) 2024-09-10T11:33:29.092 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: fs_name cephfs 2024-09-10T11:33:29.092 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: epoch 23 2024-09-10T11:33:29.092 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T11:33:29.092 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:33:29.092 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: modified 2024-09-10T11:33:11.119766+0000 2024-09-10T11:33:29.092 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: tableserver 0 2024-09-10T11:33:29.092 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: root 0 2024-09-10T11:33:29.092 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: session_timeout 60 2024-09-10T11:33:29.092 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: session_autoclose 300 2024-09-10T11:33:29.093 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: max_file_size 1099511627776 2024-09-10T11:33:29.093 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: max_xattr_size 65536 2024-09-10T11:33:29.093 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: required_client_features {} 2024-09-10T11:33:29.093 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: last_failure 0 2024-09-10T11:33:29.093 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: last_failure_osd_epoch 47 2024-09-10T11:33:29.093 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:33:29.093 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: max_mds 2 2024-09-10T11:33:29.093 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: in 0,1 2024-09-10T11:33:29.093 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: up {0=14488,1=24267} 2024-09-10T11:33:29.093 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: failed 2024-09-10T11:33:29.093 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: damaged 2024-09-10T11:33:29.093 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: stopped 2024-09-10T11:33:29.093 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: data_pools [3] 2024-09-10T11:33:29.093 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: metadata_pool 2 2024-09-10T11:33:29.094 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: inline_data enabled 2024-09-10T11:33:29.094 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: balancer 2024-09-10T11:33:29.094 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: bal_rank_mask -1 2024-09-10T11:33:29.094 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: standby_count_wanted 1 2024-09-10T11:33:29.094 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: qdb_cluster leader: 14488 members: 14488,24267 2024-09-10T11:33:29.094 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: [mds.cephfs.smithi144.jaoyvs{0:14488} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:29.094 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: [mds.cephfs.smithi144.bfyvwv{0:14498} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/1752150537,v1:172.21.15.144:6827/1752150537] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:29.094 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: [mds.cephfs.smithi178.nmhrmh{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:29.094 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: [mds.cephfs.smithi178.czjrsa{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:29.094 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: 2024-09-10T11:33:29.094 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout: 2024-09-10T11:33:29.094 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: mon.smithi178@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-09-10T11:33:29.094 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: mon.smithi178@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-09-10T11:33:29.095 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: mon.smithi178@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-09-10T11:33:29.095 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: mon.smithi178@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-09-10T11:33:29.095 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 ceph-mon[65940]: mon.smithi178@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-09-10T11:33:29.095 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 bash[65920]: a1b7b65c960cf7ec7ca9eddcf74c6a14ce0a9878a28924bd5e7af2fc170317c1 2024-09-10T11:33:29.095 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:28 smithi178 systemd[1]: Started Ceph mon.smithi178 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:33:30.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:33:30.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: mon.smithi144 calling monitor election 2024-09-10T11:33:30.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: mon.smithi178 calling monitor election 2024-09-10T11:33:30.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: mon.smithi144 is new leader, mons smithi144,smithi178 in quorum (ranks 0,1) 2024-09-10T11:33:30.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: monmap epoch 3 2024-09-10T11:33:30.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:33:30.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: last_changed 2024-09-10T11:33:29.017586+0000 2024-09-10T11:33:30.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: created 2024-09-10T11:19:28.938528+0000 2024-09-10T11:33:30.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: min_mon_release 19 (squid) 2024-09-10T11:33:30.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: election_strategy: 1 2024-09-10T11:33:30.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: 0: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.smithi144 2024-09-10T11:33:30.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.smithi178 2024-09-10T11:33:30.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:33:30.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: osdmap e50: 6 total, 6 up, 6 in 2024-09-10T11:33:30.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: mgrmap e38: smithi144.bkogfs(active, since 15s), standbys: smithi178.vmvxuf 2024-09-10T11:33:30.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T11:33:30.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:33:30.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:33:30.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:33:30.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:30.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:30.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:33:30.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:30 smithi178 ceph-mon[65940]: pgmap v11: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-10T11:33:30.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi144"}]: dispatch 2024-09-10T11:33:30.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: mon.smithi144 calling monitor election 2024-09-10T11:33:30.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: mon.smithi178 calling monitor election 2024-09-10T11:33:30.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: mon.smithi144 is new leader, mons smithi144,smithi178 in quorum (ranks 0,1) 2024-09-10T11:33:30.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: monmap epoch 3 2024-09-10T11:33:30.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T11:33:30.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: last_changed 2024-09-10T11:33:29.017586+0000 2024-09-10T11:33:30.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: created 2024-09-10T11:19:28.938528+0000 2024-09-10T11:33:30.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: min_mon_release 19 (squid) 2024-09-10T11:33:30.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: election_strategy: 1 2024-09-10T11:33:30.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: 0: [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] mon.smithi144 2024-09-10T11:33:30.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: 1: [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] mon.smithi178 2024-09-10T11:33:30.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2 up:standby-replay 2024-09-10T11:33:30.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: osdmap e50: 6 total, 6 up, 6 in 2024-09-10T11:33:30.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: mgrmap e38: smithi144.bkogfs(active, since 15s), standbys: smithi178.vmvxuf 2024-09-10T11:33:30.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T11:33:30.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:33:30.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:33:30.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mon metadata", "id": "smithi178"}]: dispatch 2024-09-10T11:33:30.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:30.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:30.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:33:30.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:30 smithi144 ceph-mon[87436]: pgmap v11: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 12 op/s 2024-09-10T11:33:32.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:32 smithi178 ceph-mon[65940]: pgmap v12: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-10T11:33:32.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:32 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:32.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:32 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:32.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:32 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:32.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:32 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:32.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:32 smithi144 ceph-mon[87436]: pgmap v12: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 20 KiB/s rd, 0 B/s wr, 14 op/s 2024-09-10T11:33:32.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:32 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:32.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:32 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:32.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:32 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:32.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:32 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:34.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:34 smithi178 ceph-mon[65940]: pgmap v13: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:34.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:34 smithi144 ceph-mon[87436]: pgmap v13: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:34.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.961+0000 7f19d1ccd700 1 -- 172.21.15.144:0/3610907729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19cc1003d0 msgr2=0x7f19cc1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:34.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.961+0000 7f19d1ccd700 1 --2- 172.21.15.144:0/3610907729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19cc1003d0 0x7f19cc1007a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f19bc009910 tx=0x7f19bc009c20 comp rx=0 tx=0).stop 2024-09-10T11:33:34.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.961+0000 7f19d1ccd700 1 -- 172.21.15.144:0/3610907729 shutdown_connections 2024-09-10T11:33:34.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.961+0000 7f19d1ccd700 1 --2- 172.21.15.144:0/3610907729 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19cc100d70 0x7f19cc104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:34.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.961+0000 7f19d1ccd700 1 --2- 172.21.15.144:0/3610907729 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19cc1003d0 0x7f19cc1007a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:34.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.961+0000 7f19d1ccd700 1 -- 172.21.15.144:0/3610907729 >> 172.21.15.144:0/3610907729 conn(0x7f19cc0fbc50 msgr2=0x7f19cc0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:34.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.961+0000 7f19d1ccd700 1 -- 172.21.15.144:0/3610907729 shutdown_connections 2024-09-10T11:33:34.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.961+0000 7f19d1ccd700 1 -- 172.21.15.144:0/3610907729 wait complete. 2024-09-10T11:33:34.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.962+0000 7f19d1ccd700 1 Processor -- start 2024-09-10T11:33:34.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.962+0000 7f19d1ccd700 1 -- start start 2024-09-10T11:33:34.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.962+0000 7f19d1ccd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19cc1003d0 0x7f19cc197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:34.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.962+0000 7f19d1ccd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19cc100d70 0x7f19cc1984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:34.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.962+0000 7f19d1ccd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc198bd0 con 0x7f19cc1003d0 2024-09-10T11:33:34.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.962+0000 7f19d1ccd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19cc19c960 con 0x7f19cc100d70 2024-09-10T11:33:34.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.962+0000 7f19cb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19cc1003d0 0x7f19cc197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:34.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.962+0000 7f19cb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19cc1003d0 0x7f19cc197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54752/0 (socket says 172.21.15.144:54752) 2024-09-10T11:33:34.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.962+0000 7f19cb7fe700 1 -- 172.21.15.144:0/3717891383 learned_addr learned my addr 172.21.15.144:0/3717891383 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:33:34.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.962+0000 7f19c3fff700 1 --2- 172.21.15.144:0/3717891383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19cc100d70 0x7f19cc1984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:34.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.963+0000 7f19cb7fe700 1 -- 172.21.15.144:0/3717891383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19cc100d70 msgr2=0x7f19cc1984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:34.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.963+0000 7f19cb7fe700 1 --2- 172.21.15.144:0/3717891383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19cc100d70 0x7f19cc1984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:34.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.963+0000 7f19cb7fe700 1 -- 172.21.15.144:0/3717891383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19bc0095f0 con 0x7f19cc1003d0 2024-09-10T11:33:34.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.963+0000 7f19c3fff700 1 --2- 172.21.15.144:0/3717891383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19cc100d70 0x7f19cc1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:33:34.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.963+0000 7f19cb7fe700 1 --2- 172.21.15.144:0/3717891383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19cc1003d0 0x7f19cc197fb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0092e0 tx=0x7f19bc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:34.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.963+0000 7f19c97fa700 1 -- 172.21.15.144:0/3717891383 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19bc01d070 con 0x7f19cc1003d0 2024-09-10T11:33:34.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.963+0000 7f19d1ccd700 1 -- 172.21.15.144:0/3717891383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19cc19cbe0 con 0x7f19cc1003d0 2024-09-10T11:33:34.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.963+0000 7f19c97fa700 1 -- 172.21.15.144:0/3717891383 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19bc005e00 con 0x7f19cc1003d0 2024-09-10T11:33:34.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.963+0000 7f19c97fa700 1 -- 172.21.15.144:0/3717891383 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19bc017400 con 0x7f19cc1003d0 2024-09-10T11:33:34.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.963+0000 7f19d1ccd700 1 -- 172.21.15.144:0/3717891383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19cc19d0d0 con 0x7f19cc1003d0 2024-09-10T11:33:34.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.964+0000 7f19d1ccd700 1 -- 172.21.15.144:0/3717891383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19cc1044c0 con 0x7f19cc1003d0 2024-09-10T11:33:34.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.965+0000 7f19c97fa700 1 -- 172.21.15.144:0/3717891383 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f19bc00f6f0 con 0x7f19cc1003d0 2024-09-10T11:33:34.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.966+0000 7f19c97fa700 1 --2- 172.21.15.144:0/3717891383 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19b806e330 0x7f19b80707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:34.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.966+0000 7f19c97fa700 1 -- 172.21.15.144:0/3717891383 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f19bc08f8e0 con 0x7f19cc1003d0 2024-09-10T11:33:34.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.966+0000 7f19c3fff700 1 --2- 172.21.15.144:0/3717891383 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19b806e330 0x7f19b80707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:34.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.966+0000 7f19c3fff700 1 --2- 172.21.15.144:0/3717891383 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19b806e330 0x7f19b80707e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f19cc1995d0 tx=0x7f19b400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:34.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:34.970+0000 7f19c97fa700 1 -- 172.21.15.144:0/3717891383 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f19bc058bd0 con 0x7f19cc1003d0 2024-09-10T11:33:35.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.190+0000 7f19d1ccd700 1 -- 172.21.15.144:0/3717891383 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19cc19d420 con 0x7f19b806e330 2024-09-10T11:33:35.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.191+0000 7f19c97fa700 1 -- 172.21.15.144:0/3717891383 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f19cc19d420 con 0x7f19b806e330 2024-09-10T11:33:35.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.193+0000 7f19d1ccd700 1 -- 172.21.15.144:0/3717891383 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19b806e330 msgr2=0x7f19b80707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.193+0000 7f19d1ccd700 1 --2- 172.21.15.144:0/3717891383 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19b806e330 0x7f19b80707e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f19cc1995d0 tx=0x7f19b400b410 comp rx=0 tx=0).stop 2024-09-10T11:33:35.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.193+0000 7f19d1ccd700 1 -- 172.21.15.144:0/3717891383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19cc1003d0 msgr2=0x7f19cc197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.193+0000 7f19d1ccd700 1 --2- 172.21.15.144:0/3717891383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19cc1003d0 0x7f19cc197fb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f19bc0092e0 tx=0x7f19bc00bd70 comp rx=0 tx=0).stop 2024-09-10T11:33:35.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.193+0000 7f19d1ccd700 1 -- 172.21.15.144:0/3717891383 shutdown_connections 2024-09-10T11:33:35.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.193+0000 7f19d1ccd700 1 --2- 172.21.15.144:0/3717891383 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19b806e330 0x7f19b80707e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.193+0000 7f19d1ccd700 1 --2- 172.21.15.144:0/3717891383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19cc1003d0 0x7f19cc197fb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.193+0000 7f19d1ccd700 1 --2- 172.21.15.144:0/3717891383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19cc100d70 0x7f19cc1984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.193+0000 7f19d1ccd700 1 -- 172.21.15.144:0/3717891383 >> 172.21.15.144:0/3717891383 conn(0x7f19cc0fbc50 msgr2=0x7f19cc0fd1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:35.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.193+0000 7f19d1ccd700 1 -- 172.21.15.144:0/3717891383 shutdown_connections 2024-09-10T11:33:35.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.193+0000 7f19d1ccd700 1 -- 172.21.15.144:0/3717891383 wait complete. 2024-09-10T11:33:35.206 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:33:35.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.298+0000 7f522797f700 1 -- 172.21.15.144:0/779929459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52200ff390 msgr2=0x7f52200ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.298+0000 7f522797f700 1 --2- 172.21.15.144:0/779929459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52200ff390 0x7f52200ff760 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f521800b0a0 tx=0x7f521800b3b0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.298+0000 7f522797f700 1 -- 172.21.15.144:0/779929459 shutdown_connections 2024-09-10T11:33:35.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.298+0000 7f522797f700 1 --2- 172.21.15.144:0/779929459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52200ffd30 0x7f5220103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.298+0000 7f522797f700 1 --2- 172.21.15.144:0/779929459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52200ff390 0x7f52200ff760 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.298+0000 7f522797f700 1 -- 172.21.15.144:0/779929459 >> 172.21.15.144:0/779929459 conn(0x7f5220074f50 msgr2=0x7f5220075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:35.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.298+0000 7f522797f700 1 -- 172.21.15.144:0/779929459 shutdown_connections 2024-09-10T11:33:35.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.298+0000 7f522797f700 1 -- 172.21.15.144:0/779929459 wait complete. 2024-09-10T11:33:35.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.299+0000 7f522797f700 1 Processor -- start 2024-09-10T11:33:35.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.299+0000 7f522797f700 1 -- start start 2024-09-10T11:33:35.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.299+0000 7f522797f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52200ff390 0x7f5220196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:35.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.299+0000 7f522797f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52200ffd30 0x7f5220197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:35.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.299+0000 7f522797f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5220197860 con 0x7f52200ff390 2024-09-10T11:33:35.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.299+0000 7f522797f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52201979d0 con 0x7f52200ffd30 2024-09-10T11:33:35.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.299+0000 7f522571b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52200ff390 0x7f5220196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:35.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.299+0000 7f522571b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52200ff390 0x7f5220196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54772/0 (socket says 172.21.15.144:54772) 2024-09-10T11:33:35.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.299+0000 7f522571b700 1 -- 172.21.15.144:0/3963040235 learned_addr learned my addr 172.21.15.144:0/3963040235 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:33:35.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.299+0000 7f5224f1a700 1 --2- 172.21.15.144:0/3963040235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52200ffd30 0x7f5220197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:35.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.299+0000 7f522571b700 1 -- 172.21.15.144:0/3963040235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52200ffd30 msgr2=0x7f5220197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.299+0000 7f522571b700 1 --2- 172.21.15.144:0/3963040235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52200ffd30 0x7f5220197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.299+0000 7f522571b700 1 -- 172.21.15.144:0/3963040235 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5218009d00 con 0x7f52200ff390 2024-09-10T11:33:35.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.300+0000 7f522571b700 1 --2- 172.21.15.144:0/3963040235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52200ff390 0x7f5220196c40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5218000f80 tx=0x7f521800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:35.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.300+0000 7f521e7fc700 1 -- 172.21.15.144:0/3963040235 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f521800e040 con 0x7f52200ff390 2024-09-10T11:33:35.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.300+0000 7f521e7fc700 1 -- 172.21.15.144:0/3963040235 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5218003c10 con 0x7f52200ff390 2024-09-10T11:33:35.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.300+0000 7f521e7fc700 1 -- 172.21.15.144:0/3963040235 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f521801d6b0 con 0x7f52200ff390 2024-09-10T11:33:35.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.300+0000 7f522797f700 1 -- 172.21.15.144:0/3963040235 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f522019fcb0 con 0x7f52200ff390 2024-09-10T11:33:35.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.300+0000 7f522797f700 1 -- 172.21.15.144:0/3963040235 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52201a0120 con 0x7f52200ff390 2024-09-10T11:33:35.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.301+0000 7f522797f700 1 -- 172.21.15.144:0/3963040235 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5220107650 con 0x7f52200ff390 2024-09-10T11:33:35.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.302+0000 7f521e7fc700 1 -- 172.21.15.144:0/3963040235 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f5218004040 con 0x7f52200ff390 2024-09-10T11:33:35.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.302+0000 7f521e7fc700 1 --2- 172.21.15.144:0/3963040235 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f520806e340 0x7f52080707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:35.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.302+0000 7f521e7fc700 1 -- 172.21.15.144:0/3963040235 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f521808f970 con 0x7f52200ff390 2024-09-10T11:33:35.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.302+0000 7f5224f1a700 1 --2- 172.21.15.144:0/3963040235 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f520806e340 0x7f52080707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:35.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.303+0000 7f5224f1a700 1 --2- 172.21.15.144:0/3963040235 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f520806e340 0x7f52080707f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5220068c80 tx=0x7f521400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:35.309 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.307+0000 7f521e7fc700 1 -- 172.21.15.144:0/3963040235 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5218058c60 con 0x7f52200ff390 2024-09-10T11:33:35.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.527+0000 7f522797f700 1 -- 172.21.15.144:0/3963040235 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5220061190 con 0x7f520806e340 2024-09-10T11:33:35.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.528+0000 7f521e7fc700 1 -- 172.21.15.144:0/3963040235 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f5220061190 con 0x7f520806e340 2024-09-10T11:33:35.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.530+0000 7f522797f700 1 -- 172.21.15.144:0/3963040235 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f520806e340 msgr2=0x7f52080707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.530+0000 7f522797f700 1 --2- 172.21.15.144:0/3963040235 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f520806e340 0x7f52080707f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f5220068c80 tx=0x7f521400b410 comp rx=0 tx=0).stop 2024-09-10T11:33:35.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.530+0000 7f522797f700 1 -- 172.21.15.144:0/3963040235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52200ff390 msgr2=0x7f5220196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.530+0000 7f522797f700 1 --2- 172.21.15.144:0/3963040235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52200ff390 0x7f5220196c40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5218000f80 tx=0x7f521800cd20 comp rx=0 tx=0).stop 2024-09-10T11:33:35.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.531+0000 7f522797f700 1 -- 172.21.15.144:0/3963040235 shutdown_connections 2024-09-10T11:33:35.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.531+0000 7f522797f700 1 --2- 172.21.15.144:0/3963040235 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f520806e340 0x7f52080707f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.531+0000 7f522797f700 1 --2- 172.21.15.144:0/3963040235 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f52200ff390 0x7f5220196c40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.531+0000 7f522797f700 1 --2- 172.21.15.144:0/3963040235 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f52200ffd30 0x7f5220197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.531+0000 7f522797f700 1 -- 172.21.15.144:0/3963040235 >> 172.21.15.144:0/3963040235 conn(0x7f5220074f50 msgr2=0x7f52200fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:35.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.531+0000 7f522797f700 1 -- 172.21.15.144:0/3963040235 shutdown_connections 2024-09-10T11:33:35.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.531+0000 7f522797f700 1 -- 172.21.15.144:0/3963040235 wait complete. 2024-09-10T11:33:35.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.642+0000 7f6308e1e700 1 -- 172.21.15.144:0/3120258667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63040ff320 msgr2=0x7f63040ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.642+0000 7f6308e1e700 1 --2- 172.21.15.144:0/3120258667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63040ff320 0x7f63040ff6f0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f62f4009940 tx=0x7f62f4009c50 comp rx=0 tx=0).stop 2024-09-10T11:33:35.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.642+0000 7f6308e1e700 1 -- 172.21.15.144:0/3120258667 shutdown_connections 2024-09-10T11:33:35.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.642+0000 7f6308e1e700 1 --2- 172.21.15.144:0/3120258667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63040ffc30 0x7f630410a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.642+0000 7f6308e1e700 1 --2- 172.21.15.144:0/3120258667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63040ff320 0x7f63040ff6f0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.642+0000 7f6308e1e700 1 -- 172.21.15.144:0/3120258667 >> 172.21.15.144:0/3120258667 conn(0x7f63040748b0 msgr2=0x7f6304074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:35.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.642+0000 7f6308e1e700 1 -- 172.21.15.144:0/3120258667 shutdown_connections 2024-09-10T11:33:35.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.642+0000 7f6308e1e700 1 -- 172.21.15.144:0/3120258667 wait complete. 2024-09-10T11:33:35.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.643+0000 7f6308e1e700 1 Processor -- start 2024-09-10T11:33:35.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.643+0000 7f6308e1e700 1 -- start start 2024-09-10T11:33:35.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.644+0000 7f6308e1e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63040ff320 0x7f630419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:35.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.644+0000 7f6308e1e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63040ffc30 0x7f630419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:35.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.644+0000 7f6308e1e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630419c5f0 con 0x7f63040ffc30 2024-09-10T11:33:35.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.644+0000 7f6308e1e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6304195aa0 con 0x7f63040ff320 2024-09-10T11:33:35.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.644+0000 7f6301d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63040ffc30 0x7f630419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:35.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.644+0000 7f630259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63040ff320 0x7f630419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:35.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.644+0000 7f630259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63040ff320 0x7f630419ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.144:42188/0 (socket says 172.21.15.144:42188) 2024-09-10T11:33:35.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.644+0000 7f6301d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63040ffc30 0x7f630419bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54790/0 (socket says 172.21.15.144:54790) 2024-09-10T11:33:35.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.644+0000 7f630259c700 1 -- 172.21.15.144:0/4078434909 learned_addr learned my addr 172.21.15.144:0/4078434909 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:33:35.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.644+0000 7f630259c700 1 -- 172.21.15.144:0/4078434909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63040ffc30 msgr2=0x7f630419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.644+0000 7f630259c700 1 --2- 172.21.15.144:0/4078434909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63040ffc30 0x7f630419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.644+0000 7f630259c700 1 -- 172.21.15.144:0/4078434909 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f40095f0 con 0x7f63040ff320 2024-09-10T11:33:35.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.644+0000 7f630259c700 1 --2- 172.21.15.144:0/4078434909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63040ff320 0x7f630419ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f62f4005620 tx=0x7f62f4005650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:35.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.645+0000 7f62f37fe700 1 -- 172.21.15.144:0/4078434909 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62f401d070 con 0x7f63040ff320 2024-09-10T11:33:35.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.645+0000 7f62f37fe700 1 -- 172.21.15.144:0/4078434909 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f62f400f460 con 0x7f63040ff320 2024-09-10T11:33:35.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.645+0000 7f62f37fe700 1 -- 172.21.15.144:0/4078434909 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62f4021640 con 0x7f63040ff320 2024-09-10T11:33:35.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.645+0000 7f6308e1e700 1 -- 172.21.15.144:0/4078434909 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6304195d80 con 0x7f63040ff320 2024-09-10T11:33:35.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.645+0000 7f6308e1e700 1 -- 172.21.15.144:0/4078434909 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63041962d0 con 0x7f63040ff320 2024-09-10T11:33:35.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.646+0000 7f6308e1e700 1 -- 172.21.15.144:0/4078434909 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6304107da0 con 0x7f63040ff320 2024-09-10T11:33:35.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.646+0000 7f62f37fe700 1 -- 172.21.15.144:0/4078434909 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f62f4003a60 con 0x7f63040ff320 2024-09-10T11:33:35.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.647+0000 7f62f37fe700 1 --2- 172.21.15.144:0/4078434909 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f62ec06e330 0x7f62ec0707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:35.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.647+0000 7f62f37fe700 1 -- 172.21.15.144:0/4078434909 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f62f408f590 con 0x7f63040ff320 2024-09-10T11:33:35.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.647+0000 7f6301d9b700 1 --2- 172.21.15.144:0/4078434909 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f62ec06e330 0x7f62ec0707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:35.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.647+0000 7f6301d9b700 1 --2- 172.21.15.144:0/4078434909 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f62ec06e330 0x7f62ec0707e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f63040ff0e0 tx=0x7f62f800a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:35.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.651+0000 7f62f37fe700 1 -- 172.21.15.144:0/4078434909 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f62f405c310 con 0x7f63040ff320 2024-09-10T11:33:35.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.867+0000 7f6308e1e700 1 -- 172.21.15.144:0/4078434909 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6304196600 con 0x7f62ec06e330 2024-09-10T11:33:35.882 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.880+0000 7f62f37fe700 1 -- 172.21.15.144:0/4078434909 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6304196600 con 0x7f62ec06e330 2024-09-10T11:33:35.882 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:33:35.882 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (110s) 16s ago 12m 17.8M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:33:35.882 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (12m) 16s ago 12m 8074k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:33:35.882 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (11m) 4s ago 11m 8032k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:33:35.883 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (12m) 16s ago 12m 6669k - 18.2.0 dc2bc1663786 3311fb04aa0c 2024-09-10T11:33:35.883 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (11m) 4s ago 11m 6656k - 18.2.0 dc2bc1663786 6e408e8476d4 2024-09-10T11:33:35.883 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (88s) 16s ago 12m 75.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:33:35.883 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (6m) 16s ago 6m 59.6M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:33:35.883 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (6m) 16s ago 6m 125M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:33:35.883 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (6m) 4s ago 6m 61.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:33:35.883 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (6m) 4s ago 6m 127M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:33:35.883 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (3m) 16s ago 13m 472M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:33:35.883 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (2m) 4s ago 11m 438M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:33:35.883 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (29s) 16s ago 14m 31.2M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:33:35.883 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (7s) 4s ago 11m 24.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:33:35.883 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (2m) 16s ago 12m 16.3M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:33:35.883 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (2m) 4s ago 11m 16.0M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:33:35.883 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (10m) 16s ago 10m 206M 1216M 18.2.0 dc2bc1663786 2435febebdb5 2024-09-10T11:33:35.884 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (10m) 16s ago 10m 213M 1216M 18.2.0 dc2bc1663786 b55fe920ff9d 2024-09-10T11:33:35.884 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (9m) 16s ago 9m 170M 1216M 18.2.0 dc2bc1663786 3ed5618eac33 2024-09-10T11:33:35.884 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (9m) 4s ago 9m 255M 2239M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:33:35.884 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (8m) 4s ago 8m 219M 2239M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:33:35.884 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (8m) 4s ago 8m 198M 2239M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:33:35.884 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (2m) 16s ago 11m 77.1M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:33:35.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.883+0000 7f6308e1e700 1 -- 172.21.15.144:0/4078434909 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f62ec06e330 msgr2=0x7f62ec0707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.883+0000 7f6308e1e700 1 --2- 172.21.15.144:0/4078434909 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f62ec06e330 0x7f62ec0707e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f63040ff0e0 tx=0x7f62f800a300 comp rx=0 tx=0).stop 2024-09-10T11:33:35.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.883+0000 7f6308e1e700 1 -- 172.21.15.144:0/4078434909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63040ff320 msgr2=0x7f630419ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.883+0000 7f6308e1e700 1 --2- 172.21.15.144:0/4078434909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63040ff320 0x7f630419ba20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f62f4005620 tx=0x7f62f4005650 comp rx=0 tx=0).stop 2024-09-10T11:33:35.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.883+0000 7f6308e1e700 1 -- 172.21.15.144:0/4078434909 shutdown_connections 2024-09-10T11:33:35.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.883+0000 7f6308e1e700 1 --2- 172.21.15.144:0/4078434909 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f62ec06e330 0x7f62ec0707e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.883+0000 7f6308e1e700 1 --2- 172.21.15.144:0/4078434909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63040ff320 0x7f630419ba20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.883+0000 7f6308e1e700 1 --2- 172.21.15.144:0/4078434909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63040ffc30 0x7f630419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.883+0000 7f6308e1e700 1 -- 172.21.15.144:0/4078434909 >> 172.21.15.144:0/4078434909 conn(0x7f63040748b0 msgr2=0x7f6304104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:35.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.883+0000 7f6308e1e700 1 -- 172.21.15.144:0/4078434909 shutdown_connections 2024-09-10T11:33:35.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.883+0000 7f6308e1e700 1 -- 172.21.15.144:0/4078434909 wait complete. 2024-09-10T11:33:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.991+0000 7f03206af700 1 -- 172.21.15.144:0/607276886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03180ff390 msgr2=0x7f03180ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.991+0000 7f03206af700 1 --2- 172.21.15.144:0/607276886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03180ff390 0x7f03180ff760 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0314009910 tx=0x7f0314009c20 comp rx=0 tx=0).stop 2024-09-10T11:33:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.991+0000 7f03206af700 1 -- 172.21.15.144:0/607276886 shutdown_connections 2024-09-10T11:33:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.991+0000 7f03206af700 1 --2- 172.21.15.144:0/607276886 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03180ffd30 0x7f0318103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.991+0000 7f03206af700 1 --2- 172.21.15.144:0/607276886 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03180ff390 0x7f03180ff760 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.991+0000 7f03206af700 1 -- 172.21.15.144:0/607276886 >> 172.21.15.144:0/607276886 conn(0x7f0318074f50 msgr2=0x7f0318075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.991+0000 7f03206af700 1 -- 172.21.15.144:0/607276886 shutdown_connections 2024-09-10T11:33:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.991+0000 7f03206af700 1 -- 172.21.15.144:0/607276886 wait complete. 2024-09-10T11:33:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.992+0000 7f03206af700 1 Processor -- start 2024-09-10T11:33:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.992+0000 7f03206af700 1 -- start start 2024-09-10T11:33:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.992+0000 7f03206af700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03180ff390 0x7f0318196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.992+0000 7f03206af700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03180ffd30 0x7f0318197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.992+0000 7f03206af700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0318197840 con 0x7f03180ffd30 2024-09-10T11:33:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.992+0000 7f03206af700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03181979b0 con 0x7f03180ff390 2024-09-10T11:33:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.992+0000 7f031dc4a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03180ffd30 0x7f0318197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.992+0000 7f031dc4a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03180ffd30 0x7f0318197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54812/0 (socket says 172.21.15.144:54812) 2024-09-10T11:33:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.992+0000 7f031dc4a700 1 -- 172.21.15.144:0/3331978169 learned_addr learned my addr 172.21.15.144:0/3331978169 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:33:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.993+0000 7f031e44b700 1 --2- 172.21.15.144:0/3331978169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03180ff390 0x7f0318196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.993+0000 7f031dc4a700 1 -- 172.21.15.144:0/3331978169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03180ff390 msgr2=0x7f0318196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.993+0000 7f031dc4a700 1 --2- 172.21.15.144:0/3331978169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03180ff390 0x7f0318196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.993+0000 7f031dc4a700 1 -- 172.21.15.144:0/3331978169 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03140095f0 con 0x7f03180ffd30 2024-09-10T11:33:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.993+0000 7f031dc4a700 1 --2- 172.21.15.144:0/3331978169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03180ffd30 0x7f0318197160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f030800e8f0 tx=0x7f030800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.993+0000 7f030f7fe700 1 -- 172.21.15.144:0/3331978169 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0308009790 con 0x7f03180ffd30 2024-09-10T11:33:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.993+0000 7f030f7fe700 1 -- 172.21.15.144:0/3331978169 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f030800c810 con 0x7f03180ffd30 2024-09-10T11:33:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.993+0000 7f030f7fe700 1 -- 172.21.15.144:0/3331978169 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0308005580 con 0x7f03180ffd30 2024-09-10T11:33:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.993+0000 7f03206af700 1 -- 172.21.15.144:0/3331978169 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f031819fc90 con 0x7f03180ffd30 2024-09-10T11:33:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.993+0000 7f03206af700 1 -- 172.21.15.144:0/3331978169 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03181a01e0 con 0x7f03180ffd30 2024-09-10T11:33:35.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.994+0000 7f03206af700 1 -- 172.21.15.144:0/3331978169 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0318107650 con 0x7f03180ffd30 2024-09-10T11:33:35.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.995+0000 7f030f7fe700 1 -- 172.21.15.144:0/3331978169 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f030800c980 con 0x7f03180ffd30 2024-09-10T11:33:35.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.995+0000 7f030f7fe700 1 --2- 172.21.15.144:0/3331978169 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f030406e340 0x7f03040707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:35.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.995+0000 7f030f7fe700 1 -- 172.21.15.144:0/3331978169 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0308014070 con 0x7f03180ffd30 2024-09-10T11:33:35.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.995+0000 7f031e44b700 1 --2- 172.21.15.144:0/3331978169 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f030406e340 0x7f03040707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:35.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:35.996+0000 7f031e44b700 1 --2- 172.21.15.144:0/3331978169 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f030406e340 0x7f03040707f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f03140059d0 tx=0x7f0314005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:36.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.000+0000 7f030f7fe700 1 -- 172.21.15.144:0/3331978169 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0308057780 con 0x7f03180ffd30 2024-09-10T11:33:36.308 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.306+0000 7f03206af700 1 -- 172.21.15.144:0/3331978169 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f031804ea50 con 0x7f03180ffd30 2024-09-10T11:33:36.309 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.306+0000 7f030f7fe700 1 -- 172.21.15.144:0/3331978169 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+694 (secure 0 0 0) 0x7f030805ae40 con 0x7f03180ffd30 2024-09-10T11:33:36.309 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:33:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:33:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:33:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:33:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:33:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:33:36.309 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:33:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:33:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-09-10T11:33:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:33:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:33:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:33:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:33:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:33:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-09-10T11:33:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:33:36.310 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:33:36.310 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:33:36.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.309+0000 7f03206af700 1 -- 172.21.15.144:0/3331978169 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f030406e340 msgr2=0x7f03040707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.309+0000 7f03206af700 1 --2- 172.21.15.144:0/3331978169 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f030406e340 0x7f03040707f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f03140059d0 tx=0x7f0314005940 comp rx=0 tx=0).stop 2024-09-10T11:33:36.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.309+0000 7f03206af700 1 -- 172.21.15.144:0/3331978169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03180ffd30 msgr2=0x7f0318197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.309+0000 7f03206af700 1 --2- 172.21.15.144:0/3331978169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03180ffd30 0x7f0318197160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f030800e8f0 tx=0x7f030800ec00 comp rx=0 tx=0).stop 2024-09-10T11:33:36.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.309+0000 7f03206af700 1 -- 172.21.15.144:0/3331978169 shutdown_connections 2024-09-10T11:33:36.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.309+0000 7f03206af700 1 --2- 172.21.15.144:0/3331978169 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f030406e340 0x7f03040707f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.309+0000 7f03206af700 1 --2- 172.21.15.144:0/3331978169 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f03180ff390 0x7f0318196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.309+0000 7f03206af700 1 --2- 172.21.15.144:0/3331978169 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f03180ffd30 0x7f0318197160 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.309+0000 7f03206af700 1 -- 172.21.15.144:0/3331978169 >> 172.21.15.144:0/3331978169 conn(0x7f0318074f50 msgr2=0x7f03180fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:36.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.309+0000 7f03206af700 1 -- 172.21.15.144:0/3331978169 shutdown_connections 2024-09-10T11:33:36.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.309+0000 7f03206af700 1 -- 172.21.15.144:0/3331978169 wait complete. 2024-09-10T11:33:36.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.416+0000 7f1320846700 1 -- 172.21.15.144:0/3538873300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f131c0ffd30 msgr2=0x7f131c103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.416+0000 7f1320846700 1 --2- 172.21.15.144:0/3538873300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f131c0ffd30 0x7f131c103ce0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1304009910 tx=0x7f1304009c20 comp rx=0 tx=0).stop 2024-09-10T11:33:36.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.416+0000 7f1320846700 1 -- 172.21.15.144:0/3538873300 shutdown_connections 2024-09-10T11:33:36.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.416+0000 7f1320846700 1 --2- 172.21.15.144:0/3538873300 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f131c0ffd30 0x7f131c103ce0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.416+0000 7f1320846700 1 --2- 172.21.15.144:0/3538873300 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f131c0ff390 0x7f131c0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.416+0000 7f1320846700 1 -- 172.21.15.144:0/3538873300 >> 172.21.15.144:0/3538873300 conn(0x7f131c074f50 msgr2=0x7f131c075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:36.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.416+0000 7f1320846700 1 -- 172.21.15.144:0/3538873300 shutdown_connections 2024-09-10T11:33:36.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.416+0000 7f1320846700 1 -- 172.21.15.144:0/3538873300 wait complete. 2024-09-10T11:33:36.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.417+0000 7f1320846700 1 Processor -- start 2024-09-10T11:33:36.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.417+0000 7f1320846700 1 -- start start 2024-09-10T11:33:36.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.417+0000 7f1320846700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f131c0ff390 0x7f131c196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.417+0000 7f1320846700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f131c0ffd30 0x7f131c1970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.417+0000 7f1320846700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c1976c0 con 0x7f131c0ff390 2024-09-10T11:33:36.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.417+0000 7f1320846700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f131c197830 con 0x7f131c0ffd30 2024-09-10T11:33:36.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.417+0000 7f131a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f131c0ff390 0x7f131c196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.417+0000 7f131a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f131c0ff390 0x7f131c196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54826/0 (socket says 172.21.15.144:54826) 2024-09-10T11:33:36.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.417+0000 7f131a59c700 1 -- 172.21.15.144:0/569114052 learned_addr learned my addr 172.21.15.144:0/569114052 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:33:36.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.417+0000 7f1311d9b700 1 --2- 172.21.15.144:0/569114052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f131c0ffd30 0x7f131c1970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.418+0000 7f131a59c700 1 -- 172.21.15.144:0/569114052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f131c0ffd30 msgr2=0x7f131c1970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.418+0000 7f131a59c700 1 --2- 172.21.15.144:0/569114052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f131c0ffd30 0x7f131c1970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.418+0000 7f131a59c700 1 -- 172.21.15.144:0/569114052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13040095f0 con 0x7f131c0ff390 2024-09-10T11:33:36.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.418+0000 7f131a59c700 1 --2- 172.21.15.144:0/569114052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f131c0ff390 0x7f131c196b60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f130c00d900 tx=0x7f130c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:36.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.418+0000 7f1313fff700 1 -- 172.21.15.144:0/569114052 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130c0041f0 con 0x7f131c0ff390 2024-09-10T11:33:36.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.418+0000 7f1320846700 1 -- 172.21.15.144:0/569114052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f131c19fbd0 con 0x7f131c0ff390 2024-09-10T11:33:36.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.418+0000 7f1313fff700 1 -- 172.21.15.144:0/569114052 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f130c004cf0 con 0x7f131c0ff390 2024-09-10T11:33:36.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.418+0000 7f1313fff700 1 -- 172.21.15.144:0/569114052 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f130c00b750 con 0x7f131c0ff390 2024-09-10T11:33:36.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.418+0000 7f1320846700 1 -- 172.21.15.144:0/569114052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f131c1a0090 con 0x7f131c0ff390 2024-09-10T11:33:36.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.419+0000 7f1320846700 1 -- 172.21.15.144:0/569114052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f131c107600 con 0x7f131c0ff390 2024-09-10T11:33:36.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.420+0000 7f1313fff700 1 -- 172.21.15.144:0/569114052 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f130c004350 con 0x7f131c0ff390 2024-09-10T11:33:36.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.420+0000 7f1313fff700 1 --2- 172.21.15.144:0/569114052 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f130806e340 0x7f13080707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.420+0000 7f1311d9b700 1 --2- 172.21.15.144:0/569114052 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f130806e340 0x7f13080707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.420+0000 7f1313fff700 1 -- 172.21.15.144:0/569114052 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f130c08e3b0 con 0x7f131c0ff390 2024-09-10T11:33:36.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.421+0000 7f1311d9b700 1 --2- 172.21.15.144:0/569114052 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f130806e340 0x7f13080707f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1304006010 tx=0x7f130401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:36.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.425+0000 7f1313fff700 1 -- 172.21.15.144:0/569114052 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f130c057620 con 0x7f131c0ff390 2024-09-10T11:33:36.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.675+0000 7f1320846700 1 -- 172.21.15.144:0/569114052 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f131c02cc30 con 0x7f131c0ff390 2024-09-10T11:33:36.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.676+0000 7f1313fff700 1 -- 172.21.15.144:0/569114052 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1960 (secure 0 0 0) 0x7f130c05ace0 con 0x7f131c0ff390 2024-09-10T11:33:36.678 INFO:teuthology.orchestra.run.smithi144.stdout:e23 2024-09-10T11:33:36.679 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:33:11:119769+0000 2024-09-10T11:33:36.679 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:33:36.679 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:33:36.679 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:33:36.679 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:33:36.679 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:33:36.679 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:33:36.679 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 23 2024-09-10T11:33:36.679 INFO:teuthology.orchestra.run.smithi144.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T11:33:36.679 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:33:36.679 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:33:11.119766+0000 2024-09-10T11:33:36.679 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:33:36.679 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:33:36.680 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:33:36.680 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:33:36.680 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:33:36.680 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:33:36.680 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:33:36.680 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:33:36.680 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 47 2024-09-10T11:33:36.680 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:33:36.680 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:33:36.680 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:33:36.680 INFO:teuthology.orchestra.run.smithi144.stdout:up {0=14488,1=24267} 2024-09-10T11:33:36.680 INFO:teuthology.orchestra.run.smithi144.stdout:failed 2024-09-10T11:33:36.680 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:33:36.681 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:33:36.681 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:33:36.681 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:33:36.681 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:33:36.681 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:33:36.681 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:33:36.681 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:33:36.681 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 14488 members: 14488,24267 2024-09-10T11:33:36.681 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{0:14488} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:36.681 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{0:14498} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/1752150537,v1:172.21.15.144:6827/1752150537] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:36.681 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:36.681 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:33:36.681 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:33:36.681 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:33:36.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.678+0000 7f1320846700 1 -- 172.21.15.144:0/569114052 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f130806e340 msgr2=0x7f13080707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.678+0000 7f1320846700 1 --2- 172.21.15.144:0/569114052 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f130806e340 0x7f13080707f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1304006010 tx=0x7f130401a040 comp rx=0 tx=0).stop 2024-09-10T11:33:36.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.678+0000 7f1320846700 1 -- 172.21.15.144:0/569114052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f131c0ff390 msgr2=0x7f131c196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.678+0000 7f1320846700 1 --2- 172.21.15.144:0/569114052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f131c0ff390 0x7f131c196b60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f130c00d900 tx=0x7f130c00dcc0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.678+0000 7f1320846700 1 -- 172.21.15.144:0/569114052 shutdown_connections 2024-09-10T11:33:36.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.678+0000 7f1320846700 1 --2- 172.21.15.144:0/569114052 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f130806e340 0x7f13080707f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.678+0000 7f1320846700 1 --2- 172.21.15.144:0/569114052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f131c0ff390 0x7f131c196b60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.678+0000 7f1320846700 1 --2- 172.21.15.144:0/569114052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f131c0ffd30 0x7f131c1970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.678+0000 7f1320846700 1 -- 172.21.15.144:0/569114052 >> 172.21.15.144:0/569114052 conn(0x7f131c074f50 msgr2=0x7f131c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:36.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.678+0000 7f1320846700 1 -- 172.21.15.144:0/569114052 shutdown_connections 2024-09-10T11:33:36.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.678+0000 7f1320846700 1 -- 172.21.15.144:0/569114052 wait complete. 2024-09-10T11:33:36.683 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 23 2024-09-10T11:33:36.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.785+0000 7fa3d5431700 1 -- 172.21.15.144:0/2192442450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d00ff390 msgr2=0x7fa3d00ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.785+0000 7fa3d5431700 1 --2- 172.21.15.144:0/2192442450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d00ff390 0x7fa3d00ff760 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8009910 tx=0x7fa3c8009c20 comp rx=0 tx=0).stop 2024-09-10T11:33:36.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.785+0000 7fa3d5431700 1 -- 172.21.15.144:0/2192442450 shutdown_connections 2024-09-10T11:33:36.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.785+0000 7fa3d5431700 1 --2- 172.21.15.144:0/2192442450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3d00ffd30 0x7fa3d0103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.785+0000 7fa3d5431700 1 --2- 172.21.15.144:0/2192442450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d00ff390 0x7fa3d00ff760 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.785+0000 7fa3d5431700 1 -- 172.21.15.144:0/2192442450 >> 172.21.15.144:0/2192442450 conn(0x7fa3d0074f50 msgr2=0x7fa3d0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:36.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.785+0000 7fa3d5431700 1 -- 172.21.15.144:0/2192442450 shutdown_connections 2024-09-10T11:33:36.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.785+0000 7fa3d5431700 1 -- 172.21.15.144:0/2192442450 wait complete. 2024-09-10T11:33:36.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.786+0000 7fa3d5431700 1 Processor -- start 2024-09-10T11:33:36.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.786+0000 7fa3d5431700 1 -- start start 2024-09-10T11:33:36.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.786+0000 7fa3d5431700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d00ff390 0x7fa3d0195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.786+0000 7fa3d5431700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3d00ffd30 0x7fa3d0196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.786+0000 7fa3d5431700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d0196940 con 0x7fa3d00ff390 2024-09-10T11:33:36.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.786+0000 7fa3d5431700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d019a660 con 0x7fa3d00ffd30 2024-09-10T11:33:36.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.786+0000 7fa3ceffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d00ff390 0x7fa3d0195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.786+0000 7fa3ceffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d00ff390 0x7fa3d0195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54840/0 (socket says 172.21.15.144:54840) 2024-09-10T11:33:36.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.786+0000 7fa3ceffd700 1 -- 172.21.15.144:0/3030589772 learned_addr learned my addr 172.21.15.144:0/3030589772 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:33:36.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.787+0000 7fa3ceffd700 1 -- 172.21.15.144:0/3030589772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3d00ffd30 msgr2=0x7fa3d0196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:36.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.787+0000 7fa3ceffd700 1 --2- 172.21.15.144:0/3030589772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3d00ffd30 0x7fa3d0196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:36.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.787+0000 7fa3ceffd700 1 -- 172.21.15.144:0/3030589772 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c80095f0 con 0x7fa3d00ff390 2024-09-10T11:33:36.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.787+0000 7fa3ceffd700 1 --2- 172.21.15.144:0/3030589772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d00ff390 0x7fa3d0195d20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800f690 tx=0x7fa3c800f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:36.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.787+0000 7fa3c7fff700 1 -- 172.21.15.144:0/3030589772 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3c801d070 con 0x7fa3d00ff390 2024-09-10T11:33:36.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.787+0000 7fa3d5431700 1 -- 172.21.15.144:0/3030589772 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d019a8e0 con 0x7fa3d00ff390 2024-09-10T11:33:36.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.787+0000 7fa3c7fff700 1 -- 172.21.15.144:0/3030589772 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3c80050d0 con 0x7fa3d00ff390 2024-09-10T11:33:36.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.787+0000 7fa3c7fff700 1 -- 172.21.15.144:0/3030589772 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3c8022470 con 0x7fa3d00ff390 2024-09-10T11:33:36.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.787+0000 7fa3d5431700 1 -- 172.21.15.144:0/3030589772 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d019add0 con 0x7fa3d00ff390 2024-09-10T11:33:36.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.788+0000 7fa3d5431700 1 -- 172.21.15.144:0/3030589772 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d004ea50 con 0x7fa3d00ff390 2024-09-10T11:33:36.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.788+0000 7fa3c7fff700 1 -- 172.21.15.144:0/3030589772 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fa3c8003680 con 0x7fa3d00ff390 2024-09-10T11:33:36.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.789+0000 7fa3c7fff700 1 --2- 172.21.15.144:0/3030589772 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3bc06e330 0x7fa3bc0707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:36.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.789+0000 7fa3c7fff700 1 -- 172.21.15.144:0/3030589772 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa3c808f710 con 0x7fa3d00ff390 2024-09-10T11:33:36.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.789+0000 7fa3ce7fc700 1 --2- 172.21.15.144:0/3030589772 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3bc06e330 0x7fa3bc0707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:36.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.789+0000 7fa3ce7fc700 1 --2- 172.21.15.144:0/3030589772 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3bc06e330 0x7fa3bc0707e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005950 tx=0x7fa3b80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:36.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:36.793+0000 7fa3c7fff700 1 -- 172.21.15.144:0/3030589772 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa3c8058980 con 0x7fa3d00ff390 2024-09-10T11:33:36.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:36 smithi178 ceph-mon[65940]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:33:36.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:36 smithi178 ceph-mon[65940]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:33:36.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:36 smithi178 ceph-mon[65940]: Detected new or changed devices on smithi178 2024-09-10T11:33:36.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:36 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:36.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:36 smithi178 ceph-mon[65940]: pgmap v14: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:36.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:36 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:36.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:36 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:36.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:36 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:33:36.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:36 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:36.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:36 smithi178 ceph-mon[65940]: Reconfiguring mon.smithi144 (monmap changed)... 2024-09-10T11:33:36.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:36 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T11:33:36.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:36 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T11:33:36.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:36 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:36.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:36 smithi178 ceph-mon[65940]: Reconfiguring daemon mon.smithi144 on smithi144 2024-09-10T11:33:36.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:36 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3331978169' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:33:36.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:36 smithi144 ceph-mon[87436]: from='client.34130 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:33:36.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:36 smithi144 ceph-mon[87436]: from='client.34134 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:33:36.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:36 smithi144 ceph-mon[87436]: Detected new or changed devices on smithi178 2024-09-10T11:33:36.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:36 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:36.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:36 smithi144 ceph-mon[87436]: pgmap v14: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:36.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:36 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:36.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:36 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:36.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:36 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:33:36.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:36 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:36.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:36 smithi144 ceph-mon[87436]: Reconfiguring mon.smithi144 (monmap changed)... 2024-09-10T11:33:36.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:36 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T11:33:36.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:36 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T11:33:36.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:36 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:36.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:36 smithi144 ceph-mon[87436]: Reconfiguring daemon mon.smithi144 on smithi144 2024-09-10T11:33:36.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:36 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3331978169' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:33:37.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.021+0000 7fa3d5431700 1 -- 172.21.15.144:0/3030589772 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3d019b2e0 con 0x7fa3bc06e330 2024-09-10T11:33:37.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.022+0000 7fa3c7fff700 1 -- 172.21.15.144:0/3030589772 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fa3d019b2e0 con 0x7fa3bc06e330 2024-09-10T11:33:37.024 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:33:37.024 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:33:37.024 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:33:37.024 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:33:37.025 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:33:37.025 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:33:37.025 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:33:37.025 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:33:37.025 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "4/23 daemons upgraded", 2024-09-10T11:33:37.025 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mon daemons", 2024-09-10T11:33:37.025 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:33:37.025 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:33:37.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.024+0000 7fa3d5431700 1 -- 172.21.15.144:0/3030589772 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3bc06e330 msgr2=0x7fa3bc0707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.024+0000 7fa3d5431700 1 --2- 172.21.15.144:0/3030589772 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3bc06e330 0x7fa3bc0707e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005950 tx=0x7fa3b80058e0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.024+0000 7fa3d5431700 1 -- 172.21.15.144:0/3030589772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d00ff390 msgr2=0x7fa3d0195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.024+0000 7fa3d5431700 1 --2- 172.21.15.144:0/3030589772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d00ff390 0x7fa3d0195d20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa3c800f690 tx=0x7fa3c800f770 comp rx=0 tx=0).stop 2024-09-10T11:33:37.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.024+0000 7fa3d5431700 1 -- 172.21.15.144:0/3030589772 shutdown_connections 2024-09-10T11:33:37.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.024+0000 7fa3d5431700 1 --2- 172.21.15.144:0/3030589772 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3bc06e330 0x7fa3bc0707e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.024+0000 7fa3d5431700 1 --2- 172.21.15.144:0/3030589772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3d00ff390 0x7fa3d0195d20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.024+0000 7fa3d5431700 1 --2- 172.21.15.144:0/3030589772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3d00ffd30 0x7fa3d0196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.024+0000 7fa3d5431700 1 -- 172.21.15.144:0/3030589772 >> 172.21.15.144:0/3030589772 conn(0x7fa3d0074f50 msgr2=0x7fa3d0068450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:37.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.025+0000 7fa3d5431700 1 -- 172.21.15.144:0/3030589772 shutdown_connections 2024-09-10T11:33:37.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.025+0000 7fa3d5431700 1 -- 172.21.15.144:0/3030589772 wait complete. 2024-09-10T11:33:37.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.134+0000 7fe35c7f5700 1 -- 172.21.15.144:0/3512552872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3540ffd30 msgr2=0x7fe354103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.134+0000 7fe35c7f5700 1 --2- 172.21.15.144:0/3512552872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3540ffd30 0x7fe354103ce0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe344009910 tx=0x7fe344009c20 comp rx=0 tx=0).stop 2024-09-10T11:33:37.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.135+0000 7fe35c7f5700 1 -- 172.21.15.144:0/3512552872 shutdown_connections 2024-09-10T11:33:37.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.135+0000 7fe35c7f5700 1 --2- 172.21.15.144:0/3512552872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3540ffd30 0x7fe354103ce0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.135+0000 7fe35c7f5700 1 --2- 172.21.15.144:0/3512552872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3540ff390 0x7fe3540ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.135+0000 7fe35c7f5700 1 -- 172.21.15.144:0/3512552872 >> 172.21.15.144:0/3512552872 conn(0x7fe354074f50 msgr2=0x7fe354075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:37.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.135+0000 7fe35c7f5700 1 -- 172.21.15.144:0/3512552872 shutdown_connections 2024-09-10T11:33:37.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.135+0000 7fe35c7f5700 1 -- 172.21.15.144:0/3512552872 wait complete. 2024-09-10T11:33:37.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.135+0000 7fe35c7f5700 1 Processor -- start 2024-09-10T11:33:37.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.135+0000 7fe35c7f5700 1 -- start start 2024-09-10T11:33:37.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.135+0000 7fe35c7f5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3540ff390 0x7fe354196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.135+0000 7fe35c7f5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3540ffd30 0x7fe3541970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.135+0000 7fe35c7f5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3541976e0 con 0x7fe3540ff390 2024-09-10T11:33:37.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.135+0000 7fe35c7f5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe354197820 con 0x7fe3540ffd30 2024-09-10T11:33:37.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.136+0000 7fe35a591700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3540ff390 0x7fe354196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.136+0000 7fe35a591700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3540ff390 0x7fe354196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54860/0 (socket says 172.21.15.144:54860) 2024-09-10T11:33:37.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.136+0000 7fe35a591700 1 -- 172.21.15.144:0/661067072 learned_addr learned my addr 172.21.15.144:0/661067072 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:33:37.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.136+0000 7fe359d90700 1 --2- 172.21.15.144:0/661067072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3540ffd30 0x7fe3541970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.136+0000 7fe35a591700 1 -- 172.21.15.144:0/661067072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3540ffd30 msgr2=0x7fe3541970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.136+0000 7fe35a591700 1 --2- 172.21.15.144:0/661067072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3540ffd30 0x7fe3541970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.136+0000 7fe35a591700 1 -- 172.21.15.144:0/661067072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3440095f0 con 0x7fe3540ff390 2024-09-10T11:33:37.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.136+0000 7fe35a591700 1 --2- 172.21.15.144:0/661067072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3540ff390 0x7fe354196b80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe35000e920 tx=0x7fe35000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:37.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.136+0000 7fe34b7fe700 1 -- 172.21.15.144:0/661067072 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe35000c9d0 con 0x7fe3540ff390 2024-09-10T11:33:37.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.136+0000 7fe34b7fe700 1 -- 172.21.15.144:0/661067072 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe350010430 con 0x7fe3540ff390 2024-09-10T11:33:37.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.136+0000 7fe35c7f5700 1 -- 172.21.15.144:0/661067072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35419fbf0 con 0x7fe3540ff390 2024-09-10T11:33:37.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.137+0000 7fe35c7f5700 1 -- 172.21.15.144:0/661067072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3541a00b0 con 0x7fe3540ff390 2024-09-10T11:33:37.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.137+0000 7fe34b7fe700 1 -- 172.21.15.144:0/661067072 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe350018630 con 0x7fe3540ff390 2024-09-10T11:33:37.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.138+0000 7fe35c7f5700 1 -- 172.21.15.144:0/661067072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe354107600 con 0x7fe3540ff390 2024-09-10T11:33:37.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.138+0000 7fe34b7fe700 1 -- 172.21.15.144:0/661067072 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fe3500105a0 con 0x7fe3540ff390 2024-09-10T11:33:37.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.139+0000 7fe34b7fe700 1 --2- 172.21.15.144:0/661067072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe34006e340 0x7fe3400707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:33:37.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.139+0000 7fe34b7fe700 1 -- 172.21.15.144:0/661067072 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe350014070 con 0x7fe3540ff390 2024-09-10T11:33:37.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.139+0000 7fe359d90700 1 --2- 172.21.15.144:0/661067072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe34006e340 0x7fe3400707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:33:37.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.139+0000 7fe359d90700 1 --2- 172.21.15.144:0/661067072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe34006e340 0x7fe3400707f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe3440059d0 tx=0x7fe344005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:33:37.147 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.145+0000 7fe34b7fe700 1 -- 172.21.15.144:0/661067072 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe3500541b0 con 0x7fe3540ff390 2024-09-10T11:33:37.451 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T11:33:37.452 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:33:37.452 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:33:37.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.448+0000 7fe35c7f5700 1 -- 172.21.15.144:0/661067072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe3541979e0 con 0x7fe3540ff390 2024-09-10T11:33:37.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.449+0000 7fe34b7fe700 1 -- 172.21.15.144:0/661067072 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fe350057750 con 0x7fe3540ff390 2024-09-10T11:33:37.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.451+0000 7fe35c7f5700 1 -- 172.21.15.144:0/661067072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe34006e340 msgr2=0x7fe3400707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.451+0000 7fe35c7f5700 1 --2- 172.21.15.144:0/661067072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe34006e340 0x7fe3400707f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe3440059d0 tx=0x7fe344005940 comp rx=0 tx=0).stop 2024-09-10T11:33:37.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.451+0000 7fe35c7f5700 1 -- 172.21.15.144:0/661067072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3540ff390 msgr2=0x7fe354196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:33:37.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.451+0000 7fe35c7f5700 1 --2- 172.21.15.144:0/661067072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3540ff390 0x7fe354196b80 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe35000e920 tx=0x7fe35000ec30 comp rx=0 tx=0).stop 2024-09-10T11:33:37.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.451+0000 7fe35c7f5700 1 -- 172.21.15.144:0/661067072 shutdown_connections 2024-09-10T11:33:37.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.451+0000 7fe35c7f5700 1 --2- 172.21.15.144:0/661067072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe34006e340 0x7fe3400707f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.451+0000 7fe35c7f5700 1 --2- 172.21.15.144:0/661067072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3540ff390 0x7fe354196b80 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.451+0000 7fe35c7f5700 1 --2- 172.21.15.144:0/661067072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3540ffd30 0x7fe3541970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:33:37.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.451+0000 7fe35c7f5700 1 -- 172.21.15.144:0/661067072 >> 172.21.15.144:0/661067072 conn(0x7fe354074f50 msgr2=0x7fe3540fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:33:37.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.451+0000 7fe35c7f5700 1 -- 172.21.15.144:0/661067072 shutdown_connections 2024-09-10T11:33:37.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:33:37.451+0000 7fe35c7f5700 1 -- 172.21.15.144:0/661067072 wait complete. 2024-09-10T11:33:37.777 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:37 smithi144 ceph-mon[87436]: from='client.44107 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:33:37.777 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:37 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/569114052' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:33:37.777 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:37 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:37.777 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:37 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:37.777 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:37 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.bkogfs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:33:37.778 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:37 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:33:37.778 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:37 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:37.778 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:37 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/661067072' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:33:37.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:37 smithi178 ceph-mon[65940]: from='client.44107 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:33:37.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:37 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/569114052' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:33:37.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:37 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:37.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:37 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:37.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:37 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi144.bkogfs", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:33:37.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:37 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:33:37.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:37 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:37.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:37 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/661067072' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:33:38.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:38 smithi178 ceph-mon[65940]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:33:38.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:38 smithi178 ceph-mon[65940]: Reconfiguring mgr.smithi144.bkogfs (monmap changed)... 2024-09-10T11:33:38.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:38 smithi178 ceph-mon[65940]: Reconfiguring daemon mgr.smithi144.bkogfs on smithi144 2024-09-10T11:33:38.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:38 smithi178 ceph-mon[65940]: pgmap v15: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:38.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:38 smithi144 ceph-mon[87436]: from='client.34150 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:33:38.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:38 smithi144 ceph-mon[87436]: Reconfiguring mgr.smithi144.bkogfs (monmap changed)... 2024-09-10T11:33:38.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:38 smithi144 ceph-mon[87436]: Reconfiguring daemon mgr.smithi144.bkogfs on smithi144 2024-09-10T11:33:38.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:38 smithi144 ceph-mon[87436]: pgmap v15: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:39.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:39 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:39.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:39 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:39.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:39 smithi144 ceph-mon[87436]: Reconfiguring ceph-exporter.smithi144 (monmap changed)... 2024-09-10T11:33:39.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:39 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T11:33:39.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:39 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T11:33:39.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:39 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-10T11:33:39.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:39 smithi144 ceph-mon[87436]: Unable to update caps for client.ceph-exporter.smithi144 2024-09-10T11:33:39.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:39 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi144"}]: dispatch 2024-09-10T11:33:39.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:39 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:39.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:39 smithi144 ceph-mon[87436]: Reconfiguring daemon ceph-exporter.smithi144 on smithi144 2024-09-10T11:33:40.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:39 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:40.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:39 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:40.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:39 smithi178 ceph-mon[65940]: Reconfiguring ceph-exporter.smithi144 (monmap changed)... 2024-09-10T11:33:40.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:39 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T11:33:40.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:39 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T11:33:40.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:39 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-10T11:33:40.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:39 smithi178 ceph-mon[65940]: Unable to update caps for client.ceph-exporter.smithi144 2024-09-10T11:33:40.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:39 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi144"}]: dispatch 2024-09-10T11:33:40.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:39 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:40.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:39 smithi178 ceph-mon[65940]: Reconfiguring daemon ceph-exporter.smithi144 on smithi144 2024-09-10T11:33:40.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:40 smithi178 ceph-mon[65940]: pgmap v16: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:40.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:40 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:40.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:40 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:40.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:40 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T11:33:40.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:40 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:40.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:40 smithi144 ceph-mon[87436]: pgmap v16: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:40.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:40 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:40.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:40 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:40.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:40 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T11:33:40.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:40 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:41.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:41 smithi144 ceph-mon[87436]: Reconfiguring crash.smithi144 (monmap changed)... 2024-09-10T11:33:41.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:41 smithi144 ceph-mon[87436]: Reconfiguring daemon crash.smithi144 on smithi144 2024-09-10T11:33:42.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:41 smithi178 ceph-mon[65940]: Reconfiguring crash.smithi144 (monmap changed)... 2024-09-10T11:33:42.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:41 smithi178 ceph-mon[65940]: Reconfiguring daemon crash.smithi144 on smithi144 2024-09-10T11:33:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:42 smithi144 ceph-mon[87436]: pgmap v17: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:42 smithi144 ceph-mon[87436]: Reconfiguring osd.0 (monmap changed)... 2024-09-10T11:33:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T11:33:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:42 smithi144 ceph-mon[87436]: Reconfiguring daemon osd.0 on smithi144 2024-09-10T11:33:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:42 smithi178 ceph-mon[65940]: pgmap v17: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:42 smithi178 ceph-mon[65940]: Reconfiguring osd.0 (monmap changed)... 2024-09-10T11:33:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T11:33:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:43.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:42 smithi178 ceph-mon[65940]: Reconfiguring daemon osd.0 on smithi144 2024-09-10T11:33:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:44 smithi144 ceph-mon[87436]: Reconfiguring osd.1 (monmap changed)... 2024-09-10T11:33:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T11:33:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:44 smithi144 ceph-mon[87436]: Reconfiguring daemon osd.1 on smithi144 2024-09-10T11:33:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:44 smithi144 ceph-mon[87436]: pgmap v18: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:33:44.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:44.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:44.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:44 smithi178 ceph-mon[65940]: Reconfiguring osd.1 (monmap changed)... 2024-09-10T11:33:44.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T11:33:44.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:44.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:44 smithi178 ceph-mon[65940]: Reconfiguring daemon osd.1 on smithi144 2024-09-10T11:33:44.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:44 smithi178 ceph-mon[65940]: pgmap v18: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:44.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:33:45.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:45 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:45.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:45 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:45.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:45 smithi178 ceph-mon[65940]: Reconfiguring osd.2 (monmap changed)... 2024-09-10T11:33:45.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:45 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T11:33:45.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:45 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:45.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:45 smithi178 ceph-mon[65940]: Reconfiguring daemon osd.2 on smithi144 2024-09-10T11:33:45.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:45 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:45.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:45 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:45.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:45 smithi144 ceph-mon[87436]: Reconfiguring osd.2 (monmap changed)... 2024-09-10T11:33:45.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:45 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T11:33:45.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:45 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:45.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:45 smithi144 ceph-mon[87436]: Reconfiguring daemon osd.2 on smithi144 2024-09-10T11:33:46.806 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:46 smithi144 ceph-mon[87436]: pgmap v19: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:46.806 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:46 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:46.806 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:46 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:46.807 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:46 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.bfyvwv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:33:46.807 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:46 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:46 smithi178 ceph-mon[65940]: pgmap v19: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:46 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:46 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:46 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.bfyvwv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:33:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:46 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:47.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:47 smithi144 ceph-mon[87436]: Reconfiguring mds.cephfs.smithi144.bfyvwv (monmap changed)... 2024-09-10T11:33:47.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:47 smithi144 ceph-mon[87436]: Reconfiguring daemon mds.cephfs.smithi144.bfyvwv on smithi144 2024-09-10T11:33:48.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:47 smithi178 ceph-mon[65940]: Reconfiguring mds.cephfs.smithi144.bfyvwv (monmap changed)... 2024-09-10T11:33:48.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:47 smithi178 ceph-mon[65940]: Reconfiguring daemon mds.cephfs.smithi144.bfyvwv on smithi144 2024-09-10T11:33:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:48 smithi144 ceph-mon[87436]: pgmap v20: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:48 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:48 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:48 smithi144 ceph-mon[87436]: Reconfiguring mds.cephfs.smithi144.jaoyvs (monmap changed)... 2024-09-10T11:33:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:48 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jaoyvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:33:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:48 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:48.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:48 smithi144 ceph-mon[87436]: Reconfiguring daemon mds.cephfs.smithi144.jaoyvs on smithi144 2024-09-10T11:33:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:48 smithi178 ceph-mon[65940]: pgmap v20: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:48 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:48 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:48 smithi178 ceph-mon[65940]: Reconfiguring mds.cephfs.smithi144.jaoyvs (monmap changed)... 2024-09-10T11:33:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:48 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jaoyvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:33:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:48 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:48 smithi178 ceph-mon[65940]: Reconfiguring daemon mds.cephfs.smithi144.jaoyvs on smithi144 2024-09-10T11:33:50.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:50 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:50.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:50 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:50.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:50 smithi144 ceph-mon[87436]: Reconfiguring ceph-exporter.smithi178 (monmap changed)... 2024-09-10T11:33:50.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:50 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi178", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T11:33:50.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:50 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi178", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T11:33:50.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:50 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi178", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-10T11:33:50.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:50 smithi144 ceph-mon[87436]: Unable to update caps for client.ceph-exporter.smithi178 2024-09-10T11:33:50.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:50 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi178"}]: dispatch 2024-09-10T11:33:50.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:50 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:50.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:50 smithi144 ceph-mon[87436]: Reconfiguring daemon ceph-exporter.smithi178 on smithi178 2024-09-10T11:33:50.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:50 smithi144 ceph-mon[87436]: pgmap v21: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:50.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:50 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:50.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:50 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:50.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:50 smithi178 ceph-mon[65940]: Reconfiguring ceph-exporter.smithi178 (monmap changed)... 2024-09-10T11:33:50.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:50 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi178", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T11:33:50.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:50 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi178", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T11:33:50.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:50 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi178", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-09-10T11:33:50.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:50 smithi178 ceph-mon[65940]: Unable to update caps for client.ceph-exporter.smithi178 2024-09-10T11:33:50.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:50 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi178"}]: dispatch 2024-09-10T11:33:50.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:50 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:50.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:50 smithi178 ceph-mon[65940]: Reconfiguring daemon ceph-exporter.smithi178 on smithi178 2024-09-10T11:33:50.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:50 smithi178 ceph-mon[65940]: pgmap v21: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:51.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:51.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:51.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:51 smithi178 ceph-mon[65940]: Reconfiguring crash.smithi178 (monmap changed)... 2024-09-10T11:33:51.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi178", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T11:33:51.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:51.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:51 smithi178 ceph-mon[65940]: Reconfiguring daemon crash.smithi178 on smithi178 2024-09-10T11:33:51.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:51.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:51.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:51 smithi144 ceph-mon[87436]: Reconfiguring crash.smithi178 (monmap changed)... 2024-09-10T11:33:51.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi178", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T11:33:51.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:51.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:51 smithi144 ceph-mon[87436]: Reconfiguring daemon crash.smithi178 on smithi178 2024-09-10T11:33:52.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:52 smithi178 ceph-mon[65940]: pgmap v22: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:52.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:52.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:52.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi178.vmvxuf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:33:52.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:33:52.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:52.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:52 smithi144 ceph-mon[87436]: pgmap v22: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:52.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:52.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:52.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi178.vmvxuf", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T11:33:52.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T11:33:52.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:53.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:53 smithi178 ceph-mon[65940]: Reconfiguring mgr.smithi178.vmvxuf (monmap changed)... 2024-09-10T11:33:53.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:53 smithi178 ceph-mon[65940]: Reconfiguring daemon mgr.smithi178.vmvxuf on smithi178 2024-09-10T11:33:53.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:53.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:53.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T11:33:53.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T11:33:53.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:53.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:53 smithi144 ceph-mon[87436]: Reconfiguring mgr.smithi178.vmvxuf (monmap changed)... 2024-09-10T11:33:53.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:53 smithi144 ceph-mon[87436]: Reconfiguring daemon mgr.smithi178.vmvxuf on smithi178 2024-09-10T11:33:53.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:53.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:53.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T11:33:53.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T11:33:53.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:54.705 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:54 smithi178 ceph-mon[65940]: Reconfiguring mon.smithi178 (monmap changed)... 2024-09-10T11:33:54.705 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:54 smithi178 ceph-mon[65940]: Reconfiguring daemon mon.smithi178 on smithi178 2024-09-10T11:33:54.705 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:54 smithi178 ceph-mon[65940]: pgmap v23: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:54 smithi144 ceph-mon[87436]: Reconfiguring mon.smithi178 (monmap changed)... 2024-09-10T11:33:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:54 smithi144 ceph-mon[87436]: Reconfiguring daemon mon.smithi178 on smithi178 2024-09-10T11:33:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:54 smithi144 ceph-mon[87436]: pgmap v23: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:56.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:55 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:56.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:55 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:56.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:55 smithi178 ceph-mon[65940]: Reconfiguring osd.3 (monmap changed)... 2024-09-10T11:33:56.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:55 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T11:33:56.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:55 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:56.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:55 smithi178 ceph-mon[65940]: Reconfiguring daemon osd.3 on smithi178 2024-09-10T11:33:56.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:55 smithi178 ceph-mon[65940]: pgmap v24: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:55 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:55 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:55 smithi144 ceph-mon[87436]: Reconfiguring osd.3 (monmap changed)... 2024-09-10T11:33:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:55 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T11:33:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:55 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:55 smithi144 ceph-mon[87436]: Reconfiguring daemon osd.3 on smithi178 2024-09-10T11:33:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:55 smithi144 ceph-mon[87436]: pgmap v24: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:33:57.551 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:57 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:57.551 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:57 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:57.552 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:57 smithi178 ceph-mon[65940]: Reconfiguring osd.4 (monmap changed)... 2024-09-10T11:33:57.552 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:57 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T11:33:57.552 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:57 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:57.552 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:57 smithi178 ceph-mon[65940]: Reconfiguring daemon osd.4 on smithi178 2024-09-10T11:33:57.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:57 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:57.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:57 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:57.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:57 smithi144 ceph-mon[87436]: Reconfiguring osd.4 (monmap changed)... 2024-09-10T11:33:57.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:57 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T11:33:57.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:57 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:57.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:57 smithi144 ceph-mon[87436]: Reconfiguring daemon osd.4 on smithi178 2024-09-10T11:33:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:58 smithi178 ceph-mon[65940]: pgmap v25: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:58.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:58.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:58.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:58 smithi178 ceph-mon[65940]: Reconfiguring osd.5 (monmap changed)... 2024-09-10T11:33:58.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T11:33:58.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:58.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:58 smithi178 ceph-mon[65940]: Reconfiguring daemon osd.5 on smithi178 2024-09-10T11:33:58.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:33:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:33:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:58 smithi144 ceph-mon[87436]: pgmap v25: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:33:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:33:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:58 smithi144 ceph-mon[87436]: Reconfiguring osd.5 (monmap changed)... 2024-09-10T11:33:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T11:33:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:33:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:58 smithi144 ceph-mon[87436]: Reconfiguring daemon osd.5 on smithi178 2024-09-10T11:33:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:33:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:34:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:00 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:00 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:00 smithi144 ceph-mon[87436]: Reconfiguring mds.cephfs.smithi178.nmhrmh (monmap changed)... 2024-09-10T11:34:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:00 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.nmhrmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:34:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:00 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:00 smithi144 ceph-mon[87436]: Reconfiguring daemon mds.cephfs.smithi178.nmhrmh on smithi178 2024-09-10T11:34:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:00 smithi144 ceph-mon[87436]: pgmap v26: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:34:00.492 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:00 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:00.492 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:00 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:00.492 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:00 smithi178 ceph-mon[65940]: Reconfiguring mds.cephfs.smithi178.nmhrmh (monmap changed)... 2024-09-10T11:34:00.492 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:00 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.nmhrmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:34:00.492 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:00 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:00.492 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:00 smithi178 ceph-mon[65940]: Reconfiguring daemon mds.cephfs.smithi178.nmhrmh on smithi178 2024-09-10T11:34:00.492 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:00 smithi178 ceph-mon[65940]: pgmap v26: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:34:01.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:01 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:01.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:01 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:01.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:01 smithi144 ceph-mon[87436]: Reconfiguring mds.cephfs.smithi178.czjrsa (monmap changed)... 2024-09-10T11:34:01.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:01 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.czjrsa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:34:01.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:01 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:01.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:01 smithi144 ceph-mon[87436]: Reconfiguring daemon mds.cephfs.smithi178.czjrsa on smithi178 2024-09-10T11:34:01.856 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:01 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:01.857 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:01 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:01.857 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:01 smithi178 ceph-mon[65940]: Reconfiguring mds.cephfs.smithi178.czjrsa (monmap changed)... 2024-09-10T11:34:01.857 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:01 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.czjrsa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:34:01.857 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:01 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:01.857 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:01 smithi178 ceph-mon[65940]: Reconfiguring daemon mds.cephfs.smithi178.czjrsa on smithi178 2024-09-10T11:34:02.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:02 smithi144 ceph-mon[87436]: pgmap v27: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:02.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:02 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:02.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:02 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:02.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:02 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:34:02.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:02 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:02.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:02 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:02.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:02 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:02.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:02 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi144"}]: dispatch 2024-09-10T11:34:02.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:02 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi144"}]': finished 2024-09-10T11:34:02.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:02 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi178"}]: dispatch 2024-09-10T11:34:02.614 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:02 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi178"}]': finished 2024-09-10T11:34:02.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:02 smithi178 ceph-mon[65940]: pgmap v27: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:02.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:02 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:02.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:02 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:02.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:02 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:34:02.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:02 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:02.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:02 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:02.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:02 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:02.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:02 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi144"}]: dispatch 2024-09-10T11:34:02.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:02 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi144"}]': finished 2024-09-10T11:34:02.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:02 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi178"}]: dispatch 2024-09-10T11:34:02.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:02 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi178"}]': finished 2024-09-10T11:34:03.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:03 smithi144 ceph-mon[87436]: Upgrade: Setting container_image for all mon 2024-09-10T11:34:03.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:03 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:03.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:03 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T11:34:03.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:03 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:03.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:03 smithi178 ceph-mon[65940]: Upgrade: Setting container_image for all mon 2024-09-10T11:34:03.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:03 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:03.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:03 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi144", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T11:34:03.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:03 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:04.665 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:04 smithi144 ceph-mon[87436]: Upgrade: Updating crash.smithi144 (1/2) 2024-09-10T11:34:04.666 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:04 smithi144 ceph-mon[87436]: Deploying daemon crash.smithi144 on smithi144 2024-09-10T11:34:04.666 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:04 smithi144 ceph-mon[87436]: pgmap v28: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:04.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:04 smithi178 ceph-mon[65940]: Upgrade: Updating crash.smithi144 (1/2) 2024-09-10T11:34:04.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:04 smithi178 ceph-mon[65940]: Deploying daemon crash.smithi144 on smithi144 2024-09-10T11:34:04.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:04 smithi178 ceph-mon[65940]: pgmap v28: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:06.829 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:06 smithi144 ceph-mon[87436]: pgmap v29: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:34:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:06 smithi178 ceph-mon[65940]: pgmap v29: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:34:07.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.566+0000 7f2534f33700 1 -- 172.21.15.144:0/3971523122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2530100d70 msgr2=0x7f2530104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:07.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.566+0000 7f2534f33700 1 --2- 172.21.15.144:0/3971523122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2530100d70 0x7f2530104d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2520009910 tx=0x7f2520009c20 comp rx=0 tx=0).stop 2024-09-10T11:34:07.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.567+0000 7f2534f33700 1 -- 172.21.15.144:0/3971523122 shutdown_connections 2024-09-10T11:34:07.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.567+0000 7f2534f33700 1 --2- 172.21.15.144:0/3971523122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2530100d70 0x7f2530104d40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:07.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.567+0000 7f2534f33700 1 --2- 172.21.15.144:0/3971523122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25301003d0 0x7f25301007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:07.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.567+0000 7f2534f33700 1 -- 172.21.15.144:0/3971523122 >> 172.21.15.144:0/3971523122 conn(0x7f25300fbc90 msgr2=0x7f25300fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:07.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.567+0000 7f2534f33700 1 -- 172.21.15.144:0/3971523122 shutdown_connections 2024-09-10T11:34:07.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.567+0000 7f2534f33700 1 -- 172.21.15.144:0/3971523122 wait complete. 2024-09-10T11:34:07.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.567+0000 7f2534f33700 1 Processor -- start 2024-09-10T11:34:07.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.568+0000 7f2534f33700 1 -- start start 2024-09-10T11:34:07.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.568+0000 7f2534f33700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25301003d0 0x7f2530195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:07.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.568+0000 7f2534f33700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2530100d70 0x7f2530196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:07.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.568+0000 7f2534f33700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2530196960 con 0x7f25301003d0 2024-09-10T11:34:07.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.568+0000 7f2534f33700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253019a6f0 con 0x7f2530100d70 2024-09-10T11:34:07.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.568+0000 7f252e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25301003d0 0x7f2530195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:07.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.568+0000 7f252e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25301003d0 0x7f2530195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36846/0 (socket says 172.21.15.144:36846) 2024-09-10T11:34:07.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.568+0000 7f252e59c700 1 -- 172.21.15.144:0/1078504324 learned_addr learned my addr 172.21.15.144:0/1078504324 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:34:07.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.568+0000 7f252dd9b700 1 --2- 172.21.15.144:0/1078504324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2530100d70 0x7f2530196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:07.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.568+0000 7f252e59c700 1 -- 172.21.15.144:0/1078504324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2530100d70 msgr2=0x7f2530196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:07.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.568+0000 7f252e59c700 1 --2- 172.21.15.144:0/1078504324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2530100d70 0x7f2530196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:07.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.568+0000 7f252e59c700 1 -- 172.21.15.144:0/1078504324 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25200095f0 con 0x7f25301003d0 2024-09-10T11:34:07.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.568+0000 7f252dd9b700 1 --2- 172.21.15.144:0/1078504324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2530100d70 0x7f2530196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:07.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.568+0000 7f252e59c700 1 --2- 172.21.15.144:0/1078504324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25301003d0 0x7f2530195d40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f251800d8d0 tx=0x7f251800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:07.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.569+0000 7f25277fe700 1 -- 172.21.15.144:0/1078504324 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2518009940 con 0x7f25301003d0 2024-09-10T11:34:07.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.569+0000 7f2534f33700 1 -- 172.21.15.144:0/1078504324 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253019a9d0 con 0x7f25301003d0 2024-09-10T11:34:07.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.569+0000 7f25277fe700 1 -- 172.21.15.144:0/1078504324 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2518010460 con 0x7f25301003d0 2024-09-10T11:34:07.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.569+0000 7f25277fe700 1 -- 172.21.15.144:0/1078504324 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f251800f5f0 con 0x7f25301003d0 2024-09-10T11:34:07.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.569+0000 7f2534f33700 1 -- 172.21.15.144:0/1078504324 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253019af20 con 0x7f25301003d0 2024-09-10T11:34:07.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.570+0000 7f2534f33700 1 -- 172.21.15.144:0/1078504324 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2530108740 con 0x7f25301003d0 2024-09-10T11:34:07.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.570+0000 7f25277fe700 1 -- 172.21.15.144:0/1078504324 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f25180105d0 con 0x7f25301003d0 2024-09-10T11:34:07.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.570+0000 7f25277fe700 1 --2- 172.21.15.144:0/1078504324 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f251c06e340 0x7f251c0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:07.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.571+0000 7f25277fe700 1 -- 172.21.15.144:0/1078504324 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f251808e280 con 0x7f25301003d0 2024-09-10T11:34:07.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.571+0000 7f252dd9b700 1 --2- 172.21.15.144:0/1078504324 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f251c06e340 0x7f251c0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:07.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.571+0000 7f252dd9b700 1 --2- 172.21.15.144:0/1078504324 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f251c06e340 0x7f251c0707f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2530197360 tx=0x7f252001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:07.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.575+0000 7f25277fe700 1 -- 172.21.15.144:0/1078504324 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f25180574f0 con 0x7f25301003d0 2024-09-10T11:34:07.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.798+0000 7f2534f33700 1 -- 172.21.15.144:0/1078504324 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f253019b250 con 0x7f251c06e340 2024-09-10T11:34:07.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.799+0000 7f25277fe700 1 -- 172.21.15.144:0/1078504324 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f253019b250 con 0x7f251c06e340 2024-09-10T11:34:07.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.801+0000 7f2534f33700 1 -- 172.21.15.144:0/1078504324 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f251c06e340 msgr2=0x7f251c0707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:07.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.801+0000 7f2534f33700 1 --2- 172.21.15.144:0/1078504324 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f251c06e340 0x7f251c0707f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2530197360 tx=0x7f252001a040 comp rx=0 tx=0).stop 2024-09-10T11:34:07.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.801+0000 7f2534f33700 1 -- 172.21.15.144:0/1078504324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25301003d0 msgr2=0x7f2530195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:07.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.801+0000 7f2534f33700 1 --2- 172.21.15.144:0/1078504324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25301003d0 0x7f2530195d40 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f251800d8d0 tx=0x7f251800dc90 comp rx=0 tx=0).stop 2024-09-10T11:34:07.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.801+0000 7f2534f33700 1 -- 172.21.15.144:0/1078504324 shutdown_connections 2024-09-10T11:34:07.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.801+0000 7f2534f33700 1 --2- 172.21.15.144:0/1078504324 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f251c06e340 0x7f251c0707f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:07.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.801+0000 7f2534f33700 1 --2- 172.21.15.144:0/1078504324 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25301003d0 0x7f2530195d40 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:07.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.801+0000 7f2534f33700 1 --2- 172.21.15.144:0/1078504324 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2530100d70 0x7f2530196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:07.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.801+0000 7f2534f33700 1 -- 172.21.15.144:0/1078504324 >> 172.21.15.144:0/1078504324 conn(0x7f25300fbc90 msgr2=0x7f25300fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:07.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.801+0000 7f2534f33700 1 -- 172.21.15.144:0/1078504324 shutdown_connections 2024-09-10T11:34:07.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.801+0000 7f2534f33700 1 -- 172.21.15.144:0/1078504324 wait complete. 2024-09-10T11:34:07.813 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:34:07.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.907+0000 7f38e7af9700 1 -- 172.21.15.144:0/3789948489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e01003d0 msgr2=0x7f38e01007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.907+0000 7f38e7af9700 1 --2- 172.21.15.144:0/3789948489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e01003d0 0x7f38e01007a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009910 tx=0x7f38dc009c20 comp rx=0 tx=0).stop 2024-09-10T11:34:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.907+0000 7f38e7af9700 1 -- 172.21.15.144:0/3789948489 shutdown_connections 2024-09-10T11:34:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.907+0000 7f38e7af9700 1 --2- 172.21.15.144:0/3789948489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e0100d70 0x7f38e0104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.907+0000 7f38e7af9700 1 --2- 172.21.15.144:0/3789948489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e01003d0 0x7f38e01007a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.907+0000 7f38e7af9700 1 -- 172.21.15.144:0/3789948489 >> 172.21.15.144:0/3789948489 conn(0x7f38e00fbc90 msgr2=0x7f38e00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.907+0000 7f38e7af9700 1 -- 172.21.15.144:0/3789948489 shutdown_connections 2024-09-10T11:34:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.907+0000 7f38e7af9700 1 -- 172.21.15.144:0/3789948489 wait complete. 2024-09-10T11:34:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.908+0000 7f38e7af9700 1 Processor -- start 2024-09-10T11:34:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.908+0000 7f38e7af9700 1 -- start start 2024-09-10T11:34:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.908+0000 7f38e7af9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e01003d0 0x7f38e0195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.908+0000 7f38e7af9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0100d70 0x7f38e01962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.908+0000 7f38e7af9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e0196980 con 0x7f38e0100d70 2024-09-10T11:34:07.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.908+0000 7f38e7af9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e019a710 con 0x7f38e01003d0 2024-09-10T11:34:07.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.908+0000 7f38e5094700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0100d70 0x7f38e01962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:07.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.908+0000 7f38e5094700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0100d70 0x7f38e01962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36878/0 (socket says 172.21.15.144:36878) 2024-09-10T11:34:07.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.908+0000 7f38e5094700 1 -- 172.21.15.144:0/1716579266 learned_addr learned my addr 172.21.15.144:0/1716579266 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:34:07.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.908+0000 7f38e5895700 1 --2- 172.21.15.144:0/1716579266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e01003d0 0x7f38e0195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:07.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.909+0000 7f38e5094700 1 -- 172.21.15.144:0/1716579266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e01003d0 msgr2=0x7f38e0195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:07.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.909+0000 7f38e5094700 1 --2- 172.21.15.144:0/1716579266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e01003d0 0x7f38e0195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:07.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.909+0000 7f38e5094700 1 -- 172.21.15.144:0/1716579266 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc0095f0 con 0x7f38e0100d70 2024-09-10T11:34:07.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.909+0000 7f38e5895700 1 --2- 172.21.15.144:0/1716579266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e01003d0 0x7f38e0195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:34:07.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.909+0000 7f38e5094700 1 --2- 172.21.15.144:0/1716579266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0100d70 0x7f38e01962a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f38d000c8f0 tx=0x7f38d000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:07.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.909+0000 7f38d6ffd700 1 -- 172.21.15.144:0/1716579266 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38d0007a50 con 0x7f38e0100d70 2024-09-10T11:34:07.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.909+0000 7f38d6ffd700 1 -- 172.21.15.144:0/1716579266 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38d0007bb0 con 0x7f38e0100d70 2024-09-10T11:34:07.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.909+0000 7f38e7af9700 1 -- 172.21.15.144:0/1716579266 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e019a9f0 con 0x7f38e0100d70 2024-09-10T11:34:07.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.909+0000 7f38d6ffd700 1 -- 172.21.15.144:0/1716579266 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38d0018630 con 0x7f38e0100d70 2024-09-10T11:34:07.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.909+0000 7f38e7af9700 1 -- 172.21.15.144:0/1716579266 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e019af40 con 0x7f38e0100d70 2024-09-10T11:34:07.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.910+0000 7f38e7af9700 1 -- 172.21.15.144:0/1716579266 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e01086e0 con 0x7f38e0100d70 2024-09-10T11:34:07.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.911+0000 7f38d6ffd700 1 -- 172.21.15.144:0/1716579266 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f38d0007d20 con 0x7f38e0100d70 2024-09-10T11:34:07.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.911+0000 7f38d6ffd700 1 --2- 172.21.15.144:0/1716579266 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38cc06e330 0x7f38cc0707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:07.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.911+0000 7f38d6ffd700 1 -- 172.21.15.144:0/1716579266 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f38d008e250 con 0x7f38e0100d70 2024-09-10T11:34:07.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.911+0000 7f38e5895700 1 --2- 172.21.15.144:0/1716579266 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38cc06e330 0x7f38cc0707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:07.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.912+0000 7f38e5895700 1 --2- 172.21.15.144:0/1716579266 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38cc06e330 0x7f38cc0707e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38dc005100 tx=0x7f38dc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:07.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:07.915+0000 7f38d6ffd700 1 -- 172.21.15.144:0/1716579266 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f38d00574c0 con 0x7f38e0100d70 2024-09-10T11:34:08.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:07 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:08.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:07 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:08.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:07 smithi178 ceph-mon[65940]: pgmap v30: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:08.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.135+0000 7f38e7af9700 1 -- 172.21.15.144:0/1716579266 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f38e019b220 con 0x7f38cc06e330 2024-09-10T11:34:08.137 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:07 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:08.137 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:07 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:08.137 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:07 smithi144 ceph-mon[87436]: pgmap v30: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:08.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.136+0000 7f38d6ffd700 1 -- 172.21.15.144:0/1716579266 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7f38e019b220 con 0x7f38cc06e330 2024-09-10T11:34:08.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.138+0000 7f38e7af9700 1 -- 172.21.15.144:0/1716579266 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38cc06e330 msgr2=0x7f38cc0707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:08.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.138+0000 7f38e7af9700 1 --2- 172.21.15.144:0/1716579266 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38cc06e330 0x7f38cc0707e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f38dc005100 tx=0x7f38dc01a040 comp rx=0 tx=0).stop 2024-09-10T11:34:08.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.138+0000 7f38e7af9700 1 -- 172.21.15.144:0/1716579266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0100d70 msgr2=0x7f38e01962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:08.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.138+0000 7f38e7af9700 1 --2- 172.21.15.144:0/1716579266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0100d70 0x7f38e01962a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f38d000c8f0 tx=0x7f38d000ccb0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.139+0000 7f38e7af9700 1 -- 172.21.15.144:0/1716579266 shutdown_connections 2024-09-10T11:34:08.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.139+0000 7f38e7af9700 1 --2- 172.21.15.144:0/1716579266 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38cc06e330 0x7f38cc0707e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.139+0000 7f38e7af9700 1 --2- 172.21.15.144:0/1716579266 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e01003d0 0x7f38e0195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.139+0000 7f38e7af9700 1 --2- 172.21.15.144:0/1716579266 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0100d70 0x7f38e01962a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.139+0000 7f38e7af9700 1 -- 172.21.15.144:0/1716579266 >> 172.21.15.144:0/1716579266 conn(0x7f38e00fbc90 msgr2=0x7f38e00fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:08.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.139+0000 7f38e7af9700 1 -- 172.21.15.144:0/1716579266 shutdown_connections 2024-09-10T11:34:08.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.139+0000 7f38e7af9700 1 -- 172.21.15.144:0/1716579266 wait complete. 2024-09-10T11:34:08.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.245+0000 7fbc64cfe700 1 -- 172.21.15.144:0/4025891497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc60100da0 msgr2=0x7fbc60104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:08.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.245+0000 7fbc64cfe700 1 --2- 172.21.15.144:0/4025891497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc60100da0 0x7fbc60104d70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fbc54009910 tx=0x7fbc54009c20 comp rx=0 tx=0).stop 2024-09-10T11:34:08.249 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.245+0000 7fbc64cfe700 1 -- 172.21.15.144:0/4025891497 shutdown_connections 2024-09-10T11:34:08.249 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.245+0000 7fbc64cfe700 1 --2- 172.21.15.144:0/4025891497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc60100da0 0x7fbc60104d70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.249 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.245+0000 7fbc64cfe700 1 --2- 172.21.15.144:0/4025891497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc60100400 0x7fbc601007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.249 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.245+0000 7fbc64cfe700 1 -- 172.21.15.144:0/4025891497 >> 172.21.15.144:0/4025891497 conn(0x7fbc600fbca0 msgr2=0x7fbc600fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:08.249 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.245+0000 7fbc64cfe700 1 -- 172.21.15.144:0/4025891497 shutdown_connections 2024-09-10T11:34:08.249 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.245+0000 7fbc64cfe700 1 -- 172.21.15.144:0/4025891497 wait complete. 2024-09-10T11:34:08.249 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.246+0000 7fbc64cfe700 1 Processor -- start 2024-09-10T11:34:08.249 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.246+0000 7fbc64cfe700 1 -- start start 2024-09-10T11:34:08.249 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.246+0000 7fbc64cfe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc60100400 0x7fbc60195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:08.250 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.246+0000 7fbc64cfe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc60100da0 0x7fbc60196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:08.250 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.246+0000 7fbc64cfe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc60196970 con 0x7fbc60100400 2024-09-10T11:34:08.250 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.246+0000 7fbc64cfe700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6019a700 con 0x7fbc60100da0 2024-09-10T11:34:08.250 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.246+0000 7fbc5e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc60100400 0x7fbc60195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:08.250 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.246+0000 7fbc5e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc60100400 0x7fbc60195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36896/0 (socket says 172.21.15.144:36896) 2024-09-10T11:34:08.250 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.246+0000 7fbc5e59c700 1 -- 172.21.15.144:0/2962519498 learned_addr learned my addr 172.21.15.144:0/2962519498 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:34:08.250 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.246+0000 7fbc5dd9b700 1 --2- 172.21.15.144:0/2962519498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc60100da0 0x7fbc60196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:08.250 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.246+0000 7fbc5e59c700 1 -- 172.21.15.144:0/2962519498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc60100da0 msgr2=0x7fbc60196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:08.250 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.246+0000 7fbc5e59c700 1 --2- 172.21.15.144:0/2962519498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc60100da0 0x7fbc60196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.250 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.246+0000 7fbc5e59c700 1 -- 172.21.15.144:0/2962519498 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc540095f0 con 0x7fbc60100400 2024-09-10T11:34:08.250 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.247+0000 7fbc5e59c700 1 --2- 172.21.15.144:0/2962519498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc60100400 0x7fbc60195d50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000d8d0 tx=0x7fbc5000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:08.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.247+0000 7fbc4f7fe700 1 -- 172.21.15.144:0/2962519498 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc500098a0 con 0x7fbc60100400 2024-09-10T11:34:08.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.247+0000 7fbc4f7fe700 1 -- 172.21.15.144:0/2962519498 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc50010460 con 0x7fbc60100400 2024-09-10T11:34:08.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.247+0000 7fbc64cfe700 1 -- 172.21.15.144:0/2962519498 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6019a9e0 con 0x7fbc60100400 2024-09-10T11:34:08.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.247+0000 7fbc4f7fe700 1 -- 172.21.15.144:0/2962519498 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc5000f5f0 con 0x7fbc60100400 2024-09-10T11:34:08.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.247+0000 7fbc64cfe700 1 -- 172.21.15.144:0/2962519498 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6019af30 con 0x7fbc60100400 2024-09-10T11:34:08.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.248+0000 7fbc64cfe700 1 -- 172.21.15.144:0/2962519498 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc60108770 con 0x7fbc60100400 2024-09-10T11:34:08.253 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.248+0000 7fbc4f7fe700 1 -- 172.21.15.144:0/2962519498 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fbc50009a00 con 0x7fbc60100400 2024-09-10T11:34:08.253 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.249+0000 7fbc4f7fe700 1 --2- 172.21.15.144:0/2962519498 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc4806e340 0x7fbc480707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:08.253 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.249+0000 7fbc4f7fe700 1 -- 172.21.15.144:0/2962519498 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbc5008e120 con 0x7fbc60100400 2024-09-10T11:34:08.253 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.249+0000 7fbc5dd9b700 1 --2- 172.21.15.144:0/2962519498 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc4806e340 0x7fbc480707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:08.253 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.249+0000 7fbc5dd9b700 1 --2- 172.21.15.144:0/2962519498 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc4806e340 0x7fbc480707f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbc60197370 tx=0x7fbc5401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:08.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.253+0000 7fbc4f7fe700 1 -- 172.21.15.144:0/2962519498 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbc50057390 con 0x7fbc60100400 2024-09-10T11:34:08.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.484+0000 7fbc64cfe700 1 -- 172.21.15.144:0/2962519498 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc6019b260 con 0x7fbc4806e340 2024-09-10T11:34:08.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.494+0000 7fbc4f7fe700 1 -- 172.21.15.144:0/2962519498 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fbc6019b260 con 0x7fbc4806e340 2024-09-10T11:34:08.495 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:34:08.495 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (2m) 49s ago 12m 17.8M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:34:08.495 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (13m) 49s ago 13m 8074k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:34:08.495 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (12m) 36s ago 12m 8032k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:34:08.495 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 starting - - - - 2024-09-10T11:34:08.495 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (12m) 36s ago 12m 6656k - 18.2.0 dc2bc1663786 6e408e8476d4 2024-09-10T11:34:08.495 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (2m) 49s ago 12m 75.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:34:08.496 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (7m) 49s ago 7m 59.6M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:34:08.496 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (7m) 49s ago 7m 125M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:34:08.496 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (7m) 36s ago 7m 61.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:34:08.496 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (7m) 36s ago 7m 127M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:34:08.496 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (4m) 49s ago 14m 472M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:34:08.496 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (3m) 36s ago 11m 438M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:34:08.496 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (61s) 49s ago 14m 31.2M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:34:08.496 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (40s) 36s ago 11m 24.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:34:08.496 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (2m) 49s ago 13m 16.3M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:34:08.496 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (2m) 36s ago 12m 16.0M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:34:08.496 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (11m) 49s ago 11m 206M 1216M 18.2.0 dc2bc1663786 2435febebdb5 2024-09-10T11:34:08.496 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (10m) 49s ago 10m 213M 1216M 18.2.0 dc2bc1663786 b55fe920ff9d 2024-09-10T11:34:08.497 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (10m) 49s ago 10m 170M 1216M 18.2.0 dc2bc1663786 3ed5618eac33 2024-09-10T11:34:08.497 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (9m) 36s ago 9m 255M 2239M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:34:08.497 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (9m) 36s ago 9m 219M 2239M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:34:08.497 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (8m) 36s ago 8m 198M 2239M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:34:08.497 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (2m) 49s ago 12m 77.1M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:34:08.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.496+0000 7fbc64cfe700 1 -- 172.21.15.144:0/2962519498 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc4806e340 msgr2=0x7fbc480707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:08.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.496+0000 7fbc64cfe700 1 --2- 172.21.15.144:0/2962519498 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc4806e340 0x7fbc480707f0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbc60197370 tx=0x7fbc5401a040 comp rx=0 tx=0).stop 2024-09-10T11:34:08.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.496+0000 7fbc64cfe700 1 -- 172.21.15.144:0/2962519498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc60100400 msgr2=0x7fbc60195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:08.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.496+0000 7fbc64cfe700 1 --2- 172.21.15.144:0/2962519498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc60100400 0x7fbc60195d50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbc5000d8d0 tx=0x7fbc5000dbe0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.496+0000 7fbc64cfe700 1 -- 172.21.15.144:0/2962519498 shutdown_connections 2024-09-10T11:34:08.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.496+0000 7fbc64cfe700 1 --2- 172.21.15.144:0/2962519498 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc4806e340 0x7fbc480707f0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.496+0000 7fbc64cfe700 1 --2- 172.21.15.144:0/2962519498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc60100400 0x7fbc60195d50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.496+0000 7fbc64cfe700 1 --2- 172.21.15.144:0/2962519498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc60100da0 0x7fbc60196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.496+0000 7fbc64cfe700 1 -- 172.21.15.144:0/2962519498 >> 172.21.15.144:0/2962519498 conn(0x7fbc600fbca0 msgr2=0x7fbc600fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:08.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.496+0000 7fbc64cfe700 1 -- 172.21.15.144:0/2962519498 shutdown_connections 2024-09-10T11:34:08.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.496+0000 7fbc64cfe700 1 -- 172.21.15.144:0/2962519498 wait complete. 2024-09-10T11:34:08.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.599+0000 7faf08795700 1 -- 172.21.15.144:0/718174194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf000ff340 msgr2=0x7faf000ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:08.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.599+0000 7faf08795700 1 --2- 172.21.15.144:0/718174194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf000ff340 0x7faf000ff710 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7faef0009910 tx=0x7faef0009c20 comp rx=0 tx=0).stop 2024-09-10T11:34:08.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.599+0000 7faf08795700 1 -- 172.21.15.144:0/718174194 shutdown_connections 2024-09-10T11:34:08.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.599+0000 7faf08795700 1 --2- 172.21.15.144:0/718174194 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf000ffce0 0x7faf00103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.599+0000 7faf08795700 1 --2- 172.21.15.144:0/718174194 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf000ff340 0x7faf000ff710 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.599+0000 7faf08795700 1 -- 172.21.15.144:0/718174194 >> 172.21.15.144:0/718174194 conn(0x7faf000fac00 msgr2=0x7faf000fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:08.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.599+0000 7faf08795700 1 -- 172.21.15.144:0/718174194 shutdown_connections 2024-09-10T11:34:08.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.599+0000 7faf08795700 1 -- 172.21.15.144:0/718174194 wait complete. 2024-09-10T11:34:08.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.600+0000 7faf08795700 1 Processor -- start 2024-09-10T11:34:08.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.600+0000 7faf08795700 1 -- start start 2024-09-10T11:34:08.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.600+0000 7faf08795700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf000ff340 0x7faf00100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:08.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.600+0000 7faf08795700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf000ffce0 0x7faf00101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:08.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.600+0000 7faf08795700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf001017d0 con 0x7faf000ffce0 2024-09-10T11:34:08.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.600+0000 7faf08795700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf00101910 con 0x7faf000ff340 2024-09-10T11:34:08.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.600+0000 7faf05d30700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf000ffce0 0x7faf00101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:08.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.600+0000 7faf05d30700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf000ffce0 0x7faf00101290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36912/0 (socket says 172.21.15.144:36912) 2024-09-10T11:34:08.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.600+0000 7faf05d30700 1 -- 172.21.15.144:0/1537375753 learned_addr learned my addr 172.21.15.144:0/1537375753 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:34:08.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.600+0000 7faf06531700 1 --2- 172.21.15.144:0/1537375753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf000ff340 0x7faf00100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:08.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.601+0000 7faf05d30700 1 -- 172.21.15.144:0/1537375753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf000ff340 msgr2=0x7faf00100d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:08.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.601+0000 7faf05d30700 1 --2- 172.21.15.144:0/1537375753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf000ff340 0x7faf00100d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.601+0000 7faf05d30700 1 -- 172.21.15.144:0/1537375753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef00095f0 con 0x7faf000ffce0 2024-09-10T11:34:08.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.601+0000 7faf05d30700 1 --2- 172.21.15.144:0/1537375753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf000ffce0 0x7faf00101290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faefc00d900 tx=0x7faefc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:08.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.601+0000 7faef77fe700 1 -- 172.21.15.144:0/1537375753 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faefc009900 con 0x7faf000ffce0 2024-09-10T11:34:08.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.601+0000 7faef77fe700 1 -- 172.21.15.144:0/1537375753 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faefc010460 con 0x7faf000ffce0 2024-09-10T11:34:08.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.601+0000 7faef77fe700 1 -- 172.21.15.144:0/1537375753 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faefc00f5f0 con 0x7faf000ffce0 2024-09-10T11:34:08.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.601+0000 7faf08795700 1 -- 172.21.15.144:0/1537375753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf001a1f90 con 0x7faf000ffce0 2024-09-10T11:34:08.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.601+0000 7faf08795700 1 -- 172.21.15.144:0/1537375753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf001a23a0 con 0x7faf000ffce0 2024-09-10T11:34:08.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.602+0000 7faf08795700 1 -- 172.21.15.144:0/1537375753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0004ea50 con 0x7faf000ffce0 2024-09-10T11:34:08.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.603+0000 7faef77fe700 1 -- 172.21.15.144:0/1537375753 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7faefc0105d0 con 0x7faf000ffce0 2024-09-10T11:34:08.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.603+0000 7faef77fe700 1 --2- 172.21.15.144:0/1537375753 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faeec06e340 0x7faeec0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:08.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.603+0000 7faef77fe700 1 -- 172.21.15.144:0/1537375753 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7faefc08dd20 con 0x7faf000ffce0 2024-09-10T11:34:08.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.603+0000 7faf06531700 1 --2- 172.21.15.144:0/1537375753 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faeec06e340 0x7faeec0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:08.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.603+0000 7faf06531700 1 --2- 172.21.15.144:0/1537375753 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faeec06e340 0x7faeec0707f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faef0006010 tx=0x7faef001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:08.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.607+0000 7faef77fe700 1 -- 172.21.15.144:0/1537375753 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7faefc057010 con 0x7faf000ffce0 2024-09-10T11:34:08.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.896+0000 7faf08795700 1 -- 172.21.15.144:0/1537375753 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faf00102280 con 0x7faf000ffce0 2024-09-10T11:34:08.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.897+0000 7faef77fe700 1 -- 172.21.15.144:0/1537375753 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+694 (secure 0 0 0) 0x7faefc0172d0 con 0x7faf000ffce0 2024-09-10T11:34:08.899 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:34:08.899 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:34:08.899 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:34:08.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:34:08.899 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:34:08.899 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:34:08.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:34:08.899 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:34:08.899 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-09-10T11:34:08.899 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:34:08.899 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:34:08.899 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:34:08.900 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:34:08.900 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:34:08.900 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-09-10T11:34:08.900 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:34:08.900 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:34:08.900 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:34:08.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.901+0000 7faf08795700 1 -- 172.21.15.144:0/1537375753 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faeec06e340 msgr2=0x7faeec0707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:08.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.901+0000 7faf08795700 1 --2- 172.21.15.144:0/1537375753 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faeec06e340 0x7faeec0707f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7faef0006010 tx=0x7faef001a040 comp rx=0 tx=0).stop 2024-09-10T11:34:08.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.901+0000 7faf08795700 1 -- 172.21.15.144:0/1537375753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf000ffce0 msgr2=0x7faf00101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:08.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.901+0000 7faf08795700 1 --2- 172.21.15.144:0/1537375753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf000ffce0 0x7faf00101290 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7faefc00d900 tx=0x7faefc00dc10 comp rx=0 tx=0).stop 2024-09-10T11:34:08.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.901+0000 7faf08795700 1 -- 172.21.15.144:0/1537375753 shutdown_connections 2024-09-10T11:34:08.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.901+0000 7faf08795700 1 --2- 172.21.15.144:0/1537375753 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faeec06e340 0x7faeec0707f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.901+0000 7faf08795700 1 --2- 172.21.15.144:0/1537375753 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faf000ff340 0x7faf00100d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.901+0000 7faf08795700 1 --2- 172.21.15.144:0/1537375753 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faf000ffce0 0x7faf00101290 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:08.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.901+0000 7faf08795700 1 -- 172.21.15.144:0/1537375753 >> 172.21.15.144:0/1537375753 conn(0x7faf000fac00 msgr2=0x7faf00068db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:08.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.901+0000 7faf08795700 1 -- 172.21.15.144:0/1537375753 shutdown_connections 2024-09-10T11:34:08.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:08.901+0000 7faf08795700 1 -- 172.21.15.144:0/1537375753 wait complete. 2024-09-10T11:34:09.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.014+0000 7fa828045700 1 -- 172.21.15.144:0/2869664393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa820100d50 msgr2=0x7fa820104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:09.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.014+0000 7fa828045700 1 --2- 172.21.15.144:0/2869664393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa820100d50 0x7fa820104d20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa81c009910 tx=0x7fa81c009c20 comp rx=0 tx=0).stop 2024-09-10T11:34:09.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.014+0000 7fa828045700 1 -- 172.21.15.144:0/2869664393 shutdown_connections 2024-09-10T11:34:09.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.014+0000 7fa828045700 1 --2- 172.21.15.144:0/2869664393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa820100d50 0x7fa820104d20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:09.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.014+0000 7fa828045700 1 --2- 172.21.15.144:0/2869664393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8201003b0 0x7fa820100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:09.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.014+0000 7fa828045700 1 -- 172.21.15.144:0/2869664393 >> 172.21.15.144:0/2869664393 conn(0x7fa8200fbc30 msgr2=0x7fa8200fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:09.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.014+0000 7fa828045700 1 -- 172.21.15.144:0/2869664393 shutdown_connections 2024-09-10T11:34:09.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.014+0000 7fa828045700 1 -- 172.21.15.144:0/2869664393 wait complete. 2024-09-10T11:34:09.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.015+0000 7fa828045700 1 Processor -- start 2024-09-10T11:34:09.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.015+0000 7fa828045700 1 -- start start 2024-09-10T11:34:09.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.015+0000 7fa828045700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8201003b0 0x7fa820073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:09.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.015+0000 7fa828045700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa820100d50 0x7fa820073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:09.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.015+0000 7fa828045700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa820073ad0 con 0x7fa8201003b0 2024-09-10T11:34:09.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.015+0000 7fa828045700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa820073c10 con 0x7fa820100d50 2024-09-10T11:34:09.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.015+0000 7fa825de1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8201003b0 0x7fa820073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:09.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.015+0000 7fa825de1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8201003b0 0x7fa820073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36942/0 (socket says 172.21.15.144:36942) 2024-09-10T11:34:09.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.015+0000 7fa825de1700 1 -- 172.21.15.144:0/2142251286 learned_addr learned my addr 172.21.15.144:0/2142251286 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:34:09.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.015+0000 7fa8255e0700 1 --2- 172.21.15.144:0/2142251286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa820100d50 0x7fa820073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:09.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.015+0000 7fa8255e0700 1 -- 172.21.15.144:0/2142251286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8201003b0 msgr2=0x7fa820073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:09.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.015+0000 7fa8255e0700 1 --2- 172.21.15.144:0/2142251286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8201003b0 0x7fa820073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:09.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.015+0000 7fa8255e0700 1 -- 172.21.15.144:0/2142251286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa81c0095f0 con 0x7fa820100d50 2024-09-10T11:34:09.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.015+0000 7fa825de1700 1 --2- 172.21.15.144:0/2142251286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8201003b0 0x7fa820073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:34:09.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.016+0000 7fa8255e0700 1 --2- 172.21.15.144:0/2142251286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa820100d50 0x7fa820073590 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa82006a050 tx=0x7fa81c00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:09.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.016+0000 7fa816ffd700 1 -- 172.21.15.144:0/2142251286 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa81c01d070 con 0x7fa820100d50 2024-09-10T11:34:09.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.016+0000 7fa828045700 1 -- 172.21.15.144:0/2142251286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa820073e90 con 0x7fa820100d50 2024-09-10T11:34:09.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.016+0000 7fa816ffd700 1 -- 172.21.15.144:0/2142251286 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa81c005dc0 con 0x7fa820100d50 2024-09-10T11:34:09.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.017+0000 7fa816ffd700 1 -- 172.21.15.144:0/2142251286 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa81c00fe80 con 0x7fa820100d50 2024-09-10T11:34:09.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.017+0000 7fa828045700 1 -- 172.21.15.144:0/2142251286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8201a22c0 con 0x7fa820100d50 2024-09-10T11:34:09.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.018+0000 7fa816ffd700 1 -- 172.21.15.144:0/2142251286 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fa81c003a60 con 0x7fa820100d50 2024-09-10T11:34:09.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.018+0000 7fa828045700 1 -- 172.21.15.144:0/2142251286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa820108720 con 0x7fa820100d50 2024-09-10T11:34:09.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.018+0000 7fa816ffd700 1 --2- 172.21.15.144:0/2142251286 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa80c06e340 0x7fa80c0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:09.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.019+0000 7fa816ffd700 1 -- 172.21.15.144:0/2142251286 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa81c08f460 con 0x7fa820100d50 2024-09-10T11:34:09.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.019+0000 7fa825de1700 1 --2- 172.21.15.144:0/2142251286 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa80c06e340 0x7fa80c0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:09.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.019+0000 7fa825de1700 1 --2- 172.21.15.144:0/2142251286 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa80c06e340 0x7fa80c0707f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa810005950 tx=0x7fa81000b500 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:09.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.023+0000 7fa816ffd700 1 -- 172.21.15.144:0/2142251286 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa81c0586d0 con 0x7fa820100d50 2024-09-10T11:34:09.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.273+0000 7fa828045700 1 -- 172.21.15.144:0/2142251286 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa82004ea50 con 0x7fa820100d50 2024-09-10T11:34:09.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.274+0000 7fa816ffd700 1 -- 172.21.15.144:0/2142251286 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1960 (secure 0 0 0) 0x7fa81c05bd90 con 0x7fa820100d50 2024-09-10T11:34:09.276 INFO:teuthology.orchestra.run.smithi144.stdout:e23 2024-09-10T11:34:09.276 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:33:11:119769+0000 2024-09-10T11:34:09.276 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:34:09.276 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:34:09.276 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:34:09.276 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:34:09.276 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:34:09.277 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:34:09.277 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 23 2024-09-10T11:34:09.277 INFO:teuthology.orchestra.run.smithi144.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T11:34:09.277 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:34:09.277 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:33:11.119766+0000 2024-09-10T11:34:09.277 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:34:09.277 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:34:09.277 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:34:09.277 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:34:09.277 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:34:09.277 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:34:09.277 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:34:09.277 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:34:09.277 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 47 2024-09-10T11:34:09.278 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:34:09.278 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:34:09.278 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:34:09.278 INFO:teuthology.orchestra.run.smithi144.stdout:up {0=14488,1=24267} 2024-09-10T11:34:09.278 INFO:teuthology.orchestra.run.smithi144.stdout:failed 2024-09-10T11:34:09.278 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:34:09.278 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:34:09.278 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:34:09.278 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:34:09.278 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:34:09.278 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:34:09.278 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:34:09.278 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:34:09.279 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 14488 members: 14488,24267 2024-09-10T11:34:09.279 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{0:14488} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:34:09.279 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{0:14498} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/1752150537,v1:172.21.15.144:6827/1752150537] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:34:09.279 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:34:09.279 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:34:09.279 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:34:09.280 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:34:09.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.276+0000 7fa828045700 1 -- 172.21.15.144:0/2142251286 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa80c06e340 msgr2=0x7fa80c0707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:09.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.276+0000 7fa828045700 1 --2- 172.21.15.144:0/2142251286 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa80c06e340 0x7fa80c0707f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fa810005950 tx=0x7fa81000b500 comp rx=0 tx=0).stop 2024-09-10T11:34:09.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.276+0000 7fa828045700 1 -- 172.21.15.144:0/2142251286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa820100d50 msgr2=0x7fa820073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:09.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.276+0000 7fa828045700 1 --2- 172.21.15.144:0/2142251286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa820100d50 0x7fa820073590 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa82006a050 tx=0x7fa81c00bd70 comp rx=0 tx=0).stop 2024-09-10T11:34:09.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.276+0000 7fa828045700 1 -- 172.21.15.144:0/2142251286 shutdown_connections 2024-09-10T11:34:09.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.277+0000 7fa828045700 1 --2- 172.21.15.144:0/2142251286 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa80c06e340 0x7fa80c0707f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:09.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.277+0000 7fa828045700 1 --2- 172.21.15.144:0/2142251286 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8201003b0 0x7fa820073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:09.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.277+0000 7fa828045700 1 --2- 172.21.15.144:0/2142251286 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa820100d50 0x7fa820073590 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:09.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.277+0000 7fa828045700 1 -- 172.21.15.144:0/2142251286 >> 172.21.15.144:0/2142251286 conn(0x7fa8200fbc30 msgr2=0x7fa8200fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:09.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.277+0000 7fa828045700 1 -- 172.21.15.144:0/2142251286 shutdown_connections 2024-09-10T11:34:09.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.277+0000 7fa828045700 1 -- 172.21.15.144:0/2142251286 wait complete. 2024-09-10T11:34:09.281 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 23 2024-09-10T11:34:09.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.387+0000 7fe5ff185700 1 -- 172.21.15.144:0/1044180253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5f81003b0 msgr2=0x7fe5f8100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:09.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.387+0000 7fe5ff185700 1 --2- 172.21.15.144:0/1044180253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5f81003b0 0x7fe5f8100780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec009910 tx=0x7fe5ec009c20 comp rx=0 tx=0).stop 2024-09-10T11:34:09.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.387+0000 7fe5ff185700 1 -- 172.21.15.144:0/1044180253 shutdown_connections 2024-09-10T11:34:09.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.387+0000 7fe5ff185700 1 --2- 172.21.15.144:0/1044180253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5f8100d50 0x7fe5f8104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:09.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.387+0000 7fe5ff185700 1 --2- 172.21.15.144:0/1044180253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5f81003b0 0x7fe5f8100780 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:09.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.387+0000 7fe5ff185700 1 -- 172.21.15.144:0/1044180253 >> 172.21.15.144:0/1044180253 conn(0x7fe5f80fbc30 msgr2=0x7fe5f80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:09.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.387+0000 7fe5ff185700 1 -- 172.21.15.144:0/1044180253 shutdown_connections 2024-09-10T11:34:09.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.387+0000 7fe5ff185700 1 -- 172.21.15.144:0/1044180253 wait complete. 2024-09-10T11:34:09.389 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:09 smithi144 ceph-mon[87436]: from='client.34158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:09.389 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:09 smithi144 ceph-mon[87436]: from='client.34162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:09.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:09 smithi144 ceph-mon[87436]: Upgrade: Updating crash.smithi178 (2/2) 2024-09-10T11:34:09.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:09 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:09.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:09 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi178", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T11:34:09.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:09 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:09.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:09 smithi144 ceph-mon[87436]: Deploying daemon crash.smithi178 on smithi178 2024-09-10T11:34:09.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:09 smithi144 ceph-mon[87436]: from='client.34166 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:09.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:09 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1537375753' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:09.390 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:09 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2142251286' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:34:09.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.387+0000 7fe5ff185700 1 Processor -- start 2024-09-10T11:34:09.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.388+0000 7fe5ff185700 1 -- start start 2024-09-10T11:34:09.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.388+0000 7fe5ff185700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5f81003b0 0x7fe5f8195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:09.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.388+0000 7fe5ff185700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5f8100d50 0x7fe5f8196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:09.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.388+0000 7fe5ff185700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f8196930 con 0x7fe5f8100d50 2024-09-10T11:34:09.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.388+0000 7fe5ff185700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f819a6c0 con 0x7fe5f81003b0 2024-09-10T11:34:09.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.388+0000 7fe5f7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5f8100d50 0x7fe5f8196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:09.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.388+0000 7fe5f7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5f8100d50 0x7fe5f8196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36970/0 (socket says 172.21.15.144:36970) 2024-09-10T11:34:09.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.388+0000 7fe5f7fff700 1 -- 172.21.15.144:0/2498976213 learned_addr learned my addr 172.21.15.144:0/2498976213 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:34:09.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.388+0000 7fe5fcf21700 1 --2- 172.21.15.144:0/2498976213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5f81003b0 0x7fe5f8195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:09.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.388+0000 7fe5f7fff700 1 -- 172.21.15.144:0/2498976213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5f81003b0 msgr2=0x7fe5f8195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:09.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.388+0000 7fe5f7fff700 1 --2- 172.21.15.144:0/2498976213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5f81003b0 0x7fe5f8195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:09.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.388+0000 7fe5f7fff700 1 -- 172.21.15.144:0/2498976213 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec0095f0 con 0x7fe5f8100d50 2024-09-10T11:34:09.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.388+0000 7fe5fcf21700 1 --2- 172.21.15.144:0/2498976213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5f81003b0 0x7fe5f8195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:34:09.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.389+0000 7fe5f7fff700 1 --2- 172.21.15.144:0/2498976213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5f8100d50 0x7fe5f8196250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400d900 tx=0x7fe5e400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:09.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.389+0000 7fe5f5ffb700 1 -- 172.21.15.144:0/2498976213 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5e4009900 con 0x7fe5f8100d50 2024-09-10T11:34:09.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.389+0000 7fe5f5ffb700 1 -- 172.21.15.144:0/2498976213 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe5e4010460 con 0x7fe5f8100d50 2024-09-10T11:34:09.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.389+0000 7fe5ff185700 1 -- 172.21.15.144:0/2498976213 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f819a9a0 con 0x7fe5f8100d50 2024-09-10T11:34:09.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.389+0000 7fe5f5ffb700 1 -- 172.21.15.144:0/2498976213 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5e400f5f0 con 0x7fe5f8100d50 2024-09-10T11:34:09.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.389+0000 7fe5ff185700 1 -- 172.21.15.144:0/2498976213 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f819aef0 con 0x7fe5f8100d50 2024-09-10T11:34:09.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.391+0000 7fe5ff185700 1 -- 172.21.15.144:0/2498976213 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f81086c0 con 0x7fe5f8100d50 2024-09-10T11:34:09.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.391+0000 7fe5f5ffb700 1 -- 172.21.15.144:0/2498976213 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7fe5e40105d0 con 0x7fe5f8100d50 2024-09-10T11:34:09.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.391+0000 7fe5f5ffb700 1 --2- 172.21.15.144:0/2498976213 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe5e806e330 0x7fe5e80707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:09.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.391+0000 7fe5f5ffb700 1 -- 172.21.15.144:0/2498976213 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fe5e408dd20 con 0x7fe5f8100d50 2024-09-10T11:34:09.395 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.391+0000 7fe5fcf21700 1 --2- 172.21.15.144:0/2498976213 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe5e806e330 0x7fe5e80707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:09.395 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.392+0000 7fe5fcf21700 1 --2- 172.21.15.144:0/2498976213 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe5e806e330 0x7fe5e80707e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00b5c0 tx=0x7fe5ec01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:09.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.396+0000 7fe5f5ffb700 1 -- 172.21.15.144:0/2498976213 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe5e4057010 con 0x7fe5f8100d50 2024-09-10T11:34:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:09 smithi178 ceph-mon[65940]: from='client.34158 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:09 smithi178 ceph-mon[65940]: from='client.34162 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:09 smithi178 ceph-mon[65940]: Upgrade: Updating crash.smithi178 (2/2) 2024-09-10T11:34:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:09 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:09 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi178", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-09-10T11:34:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:09 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:09 smithi178 ceph-mon[65940]: Deploying daemon crash.smithi178 on smithi178 2024-09-10T11:34:09.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:09 smithi178 ceph-mon[65940]: from='client.34166 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:09.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:09 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1537375753' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:09.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:09 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2142251286' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:34:09.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.628+0000 7fe5ff185700 1 -- 172.21.15.144:0/2498976213 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5f819b220 con 0x7fe5e806e330 2024-09-10T11:34:09.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.630+0000 7fe5f5ffb700 1 -- 172.21.15.144:0/2498976213 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+385 (secure 0 0 0) 0x7fe5f819b220 con 0x7fe5e806e330 2024-09-10T11:34:09.631 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:34:09.631 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:34:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:34:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:34:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:34:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:34:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:34:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:34:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "4/23 daemons upgraded", 2024-09-10T11:34:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading crash daemons", 2024-09-10T11:34:09.632 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:34:09.632 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:34:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.633+0000 7fe5ff185700 1 -- 172.21.15.144:0/2498976213 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe5e806e330 msgr2=0x7fe5e80707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.633+0000 7fe5ff185700 1 --2- 172.21.15.144:0/2498976213 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe5e806e330 0x7fe5e80707e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec00b5c0 tx=0x7fe5ec01a040 comp rx=0 tx=0).stop 2024-09-10T11:34:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.633+0000 7fe5ff185700 1 -- 172.21.15.144:0/2498976213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5f8100d50 msgr2=0x7fe5f8196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.633+0000 7fe5ff185700 1 --2- 172.21.15.144:0/2498976213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5f8100d50 0x7fe5f8196250 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe5e400d900 tx=0x7fe5e400dc10 comp rx=0 tx=0).stop 2024-09-10T11:34:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.633+0000 7fe5ff185700 1 -- 172.21.15.144:0/2498976213 shutdown_connections 2024-09-10T11:34:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.633+0000 7fe5ff185700 1 --2- 172.21.15.144:0/2498976213 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe5e806e330 0x7fe5e80707e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.633+0000 7fe5ff185700 1 --2- 172.21.15.144:0/2498976213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe5f81003b0 0x7fe5f8195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.633+0000 7fe5ff185700 1 --2- 172.21.15.144:0/2498976213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe5f8100d50 0x7fe5f8196250 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.633+0000 7fe5ff185700 1 -- 172.21.15.144:0/2498976213 >> 172.21.15.144:0/2498976213 conn(0x7fe5f80fbc30 msgr2=0x7fe5f80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.633+0000 7fe5ff185700 1 -- 172.21.15.144:0/2498976213 shutdown_connections 2024-09-10T11:34:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.634+0000 7fe5ff185700 1 -- 172.21.15.144:0/2498976213 wait complete. 2024-09-10T11:34:09.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.744+0000 7f11dc111700 1 -- 172.21.15.144:0/4042962333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d40ff390 msgr2=0x7f11d40ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:09.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.744+0000 7f11dc111700 1 --2- 172.21.15.144:0/4042962333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d40ff390 0x7f11d40ff760 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f11c4009910 tx=0x7f11c4009c20 comp rx=0 tx=0).stop 2024-09-10T11:34:09.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.744+0000 7f11dc111700 1 -- 172.21.15.144:0/4042962333 shutdown_connections 2024-09-10T11:34:09.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.744+0000 7f11dc111700 1 --2- 172.21.15.144:0/4042962333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d40ffd30 0x7f11d4103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:09.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.744+0000 7f11dc111700 1 --2- 172.21.15.144:0/4042962333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d40ff390 0x7f11d40ff760 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:09.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.744+0000 7f11dc111700 1 -- 172.21.15.144:0/4042962333 >> 172.21.15.144:0/4042962333 conn(0x7f11d4074f50 msgr2=0x7f11d4075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:09.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.744+0000 7f11dc111700 1 -- 172.21.15.144:0/4042962333 shutdown_connections 2024-09-10T11:34:09.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.744+0000 7f11dc111700 1 -- 172.21.15.144:0/4042962333 wait complete. 2024-09-10T11:34:09.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.745+0000 7f11dc111700 1 Processor -- start 2024-09-10T11:34:09.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.745+0000 7f11dc111700 1 -- start start 2024-09-10T11:34:09.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.745+0000 7f11dc111700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d40ff390 0x7f11d4197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:09.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.745+0000 7f11dc111700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d40ffd30 0x7f11d4198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:09.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.745+0000 7f11dc111700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d4198b70 con 0x7f11d40ffd30 2024-09-10T11:34:09.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.745+0000 7f11dc111700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d419c900 con 0x7f11d40ff390 2024-09-10T11:34:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.745+0000 7f11d96ac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d40ffd30 0x7f11d4198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.745+0000 7f11d96ac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d40ffd30 0x7f11d4198490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36996/0 (socket says 172.21.15.144:36996) 2024-09-10T11:34:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.745+0000 7f11d96ac700 1 -- 172.21.15.144:0/1547609241 learned_addr learned my addr 172.21.15.144:0/1547609241 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:34:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.745+0000 7f11d9ead700 1 --2- 172.21.15.144:0/1547609241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d40ff390 0x7f11d4197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.745+0000 7f11d96ac700 1 -- 172.21.15.144:0/1547609241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d40ff390 msgr2=0x7f11d4197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.745+0000 7f11d96ac700 1 --2- 172.21.15.144:0/1547609241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d40ff390 0x7f11d4197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.745+0000 7f11d96ac700 1 -- 172.21.15.144:0/1547609241 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c40095f0 con 0x7f11d40ffd30 2024-09-10T11:34:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.745+0000 7f11d96ac700 1 --2- 172.21.15.144:0/1547609241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d40ffd30 0x7f11d4198490 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11d000d900 tx=0x7f11d000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.746+0000 7f11caffd700 1 -- 172.21.15.144:0/1547609241 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11d0009900 con 0x7f11d40ffd30 2024-09-10T11:34:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.746+0000 7f11caffd700 1 -- 172.21.15.144:0/1547609241 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11d0010460 con 0x7f11d40ffd30 2024-09-10T11:34:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.746+0000 7f11dc111700 1 -- 172.21.15.144:0/1547609241 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11d419cbe0 con 0x7f11d40ffd30 2024-09-10T11:34:09.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.746+0000 7f11caffd700 1 -- 172.21.15.144:0/1547609241 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11d000f5f0 con 0x7f11d40ffd30 2024-09-10T11:34:09.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.746+0000 7f11dc111700 1 -- 172.21.15.144:0/1547609241 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d419d130 con 0x7f11d40ffd30 2024-09-10T11:34:09.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.747+0000 7f11caffd700 1 -- 172.21.15.144:0/1547609241 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f11d000f870 con 0x7f11d40ffd30 2024-09-10T11:34:09.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.747+0000 7f11dc111700 1 -- 172.21.15.144:0/1547609241 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11d404ea50 con 0x7f11d40ffd30 2024-09-10T11:34:09.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.748+0000 7f11caffd700 1 --2- 172.21.15.144:0/1547609241 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f11c006e120 0x7f11c00705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:09.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.748+0000 7f11caffd700 1 -- 172.21.15.144:0/1547609241 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f11d008d330 con 0x7f11d40ffd30 2024-09-10T11:34:09.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.748+0000 7f11d9ead700 1 --2- 172.21.15.144:0/1547609241 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f11c006e120 0x7f11c00705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:09.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.748+0000 7f11d9ead700 1 --2- 172.21.15.144:0/1547609241 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f11c006e120 0x7f11c00705d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f11c400b5c0 tx=0x7f11c401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:09.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:09.752+0000 7f11caffd700 1 -- 172.21.15.144:0/1547609241 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f11d0092060 con 0x7f11d40ffd30 2024-09-10T11:34:10.054 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:10.052+0000 7f11dc111700 1 -- 172.21.15.144:0/1547609241 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f11d41992b0 con 0x7f11d40ffd30 2024-09-10T11:34:10.054 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:10.053+0000 7f11caffd700 1 -- 172.21.15.144:0/1547609241 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f11d0056510 con 0x7f11d40ffd30 2024-09-10T11:34:10.054 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T11:34:10.054 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:34:10.054 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:34:10.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:10.055+0000 7f11dc111700 1 -- 172.21.15.144:0/1547609241 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f11c006e120 msgr2=0x7f11c00705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:10.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:10.055+0000 7f11dc111700 1 --2- 172.21.15.144:0/1547609241 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f11c006e120 0x7f11c00705d0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f11c400b5c0 tx=0x7f11c401a040 comp rx=0 tx=0).stop 2024-09-10T11:34:10.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:10.055+0000 7f11dc111700 1 -- 172.21.15.144:0/1547609241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d40ffd30 msgr2=0x7f11d4198490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:10.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:10.055+0000 7f11dc111700 1 --2- 172.21.15.144:0/1547609241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d40ffd30 0x7f11d4198490 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f11d000d900 tx=0x7f11d000dc10 comp rx=0 tx=0).stop 2024-09-10T11:34:10.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:10.055+0000 7f11dc111700 1 -- 172.21.15.144:0/1547609241 shutdown_connections 2024-09-10T11:34:10.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:10.055+0000 7f11dc111700 1 --2- 172.21.15.144:0/1547609241 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f11c006e120 0x7f11c00705d0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:10.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:10.055+0000 7f11dc111700 1 --2- 172.21.15.144:0/1547609241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11d40ff390 0x7f11d4197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:10.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:10.055+0000 7f11dc111700 1 --2- 172.21.15.144:0/1547609241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11d40ffd30 0x7f11d4198490 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:10.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:10.055+0000 7f11dc111700 1 -- 172.21.15.144:0/1547609241 >> 172.21.15.144:0/1547609241 conn(0x7f11d4074f50 msgr2=0x7f11d4068d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:10.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:10.055+0000 7f11dc111700 1 -- 172.21.15.144:0/1547609241 shutdown_connections 2024-09-10T11:34:10.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:10.055+0000 7f11dc111700 1 -- 172.21.15.144:0/1547609241 wait complete. 2024-09-10T11:34:10.576 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:10 smithi178 ceph-mon[65940]: pgmap v31: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:34:10.577 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:10 smithi178 ceph-mon[65940]: from='client.34178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:10.577 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:10 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1547609241' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:34:10.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:10 smithi144 ceph-mon[87436]: pgmap v31: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:34:10.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:10 smithi144 ceph-mon[87436]: from='client.34178 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:10.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:10 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1547609241' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:34:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:12 smithi178 ceph-mon[65940]: pgmap v32: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:12 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:12 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:12.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:12 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:34:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:12 smithi144 ceph-mon[87436]: pgmap v32: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:12 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:12 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:12 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:34:13.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:34:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:34:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:14 smithi178 ceph-mon[65940]: pgmap v33: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:14 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:14 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:14.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:14 smithi144 ceph-mon[87436]: pgmap v33: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:14.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:14.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:16.400 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:16 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:16.400 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:16 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:16.401 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:16 smithi144 ceph-mon[87436]: pgmap v34: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:34:16.490 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:16 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:16.491 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:16 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:16.491 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:16 smithi178 ceph-mon[65940]: pgmap v34: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:34:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:17 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:17 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:17 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:17 smithi178 ceph-mon[65940]: pgmap v35: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:17 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:17 smithi144 ceph-mon[87436]: pgmap v35: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:19.302 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:19 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:19.302 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:19 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:19.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:19 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:19.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:19 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:20.408 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:20 smithi144 ceph-mon[87436]: pgmap v36: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:34:20.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:20 smithi178 ceph-mon[65940]: pgmap v36: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:34:21.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:21.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:21.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:21.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:34:21.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:21.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:34:21.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:21.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:21.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:21.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: Upgrade: Setting container_image for all crash 2024-09-10T11:34:21.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:21.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi144"}]: dispatch 2024-09-10T11:34:21.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi144"}]': finished 2024-09-10T11:34:21.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi178"}]: dispatch 2024-09-10T11:34:21.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi178"}]': finished 2024-09-10T11:34:21.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-10T11:34:21.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-10T11:34:21.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: Upgrade: osd.0 is safe to restart 2024-09-10T11:34:21.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:21 smithi144 ceph-mon[87436]: pgmap v37: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:22.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:22.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:22.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:22.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:34:22.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:22.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:34:22.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:22.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:22.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:22.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: Upgrade: Setting container_image for all crash 2024-09-10T11:34:22.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:22.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi144"}]: dispatch 2024-09-10T11:34:22.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi144"}]': finished 2024-09-10T11:34:22.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi178"}]: dispatch 2024-09-10T11:34:22.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi178"}]': finished 2024-09-10T11:34:22.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-10T11:34:22.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-09-10T11:34:22.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: Upgrade: osd.0 is safe to restart 2024-09-10T11:34:22.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:21 smithi178 ceph-mon[65940]: pgmap v37: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:23.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:23 smithi144 ceph-mon[87436]: Upgrade: Updating osd.0 2024-09-10T11:34:23.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:23.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T11:34:23.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:23.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:23 smithi144 ceph-mon[87436]: Deploying daemon osd.0 on smithi144 2024-09-10T11:34:23.859 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:23 smithi178 ceph-mon[65940]: Upgrade: Updating osd.0 2024-09-10T11:34:23.859 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:23 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:23.859 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:23 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T11:34:23.859 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:23 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:23.860 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:23 smithi178 ceph-mon[65940]: Deploying daemon osd.0 on smithi144 2024-09-10T11:34:24.495 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:24 smithi144 ceph-mon[87436]: pgmap v38: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:24.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:24 smithi178 ceph-mon[65940]: pgmap v38: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:34:25.796 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:25 smithi144 systemd[1]: Stopping Ceph osd.0 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:34:26.184 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:25 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0[42921]: 2024-09-10T11:34:25.794+0000 7f4da5b6e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:34:26.184 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:25 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0[42921]: 2024-09-10T11:34:25.794+0000 7f4da5b6e700 -1 osd.0 50 *** Got signal Terminated *** 2024-09-10T11:34:26.184 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:25 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0[42921]: 2024-09-10T11:34:25.794+0000 7f4da5b6e700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:34:26.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:26 smithi144 ceph-mon[87436]: pgmap v39: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:34:26.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:26 smithi144 ceph-mon[87436]: osd.0 marked itself down and dead 2024-09-10T11:34:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:26 smithi178 ceph-mon[65940]: pgmap v39: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:34:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:26 smithi178 ceph-mon[65940]: osd.0 marked itself down and dead 2024-09-10T11:34:27.246 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:26 smithi144 podman[98376]: 2024-09-10 11:34:26.959683227 +0000 UTC m=+1.279024025 container died 2435febebdb51b8eefd129faedf13ad81ef0606aa5d61c258670271ad494f889 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, RELEASE=HEAD) 2024-09-10T11:34:27.571 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:27 smithi144 podman[98376]: 2024-09-10 11:34:27.335810708 +0000 UTC m=+1.655151504 container cleanup 2435febebdb51b8eefd129faedf13ad81ef0606aa5d61c258670271ad494f889 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=-18.2.0, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=HEAD, GIT_BRANCH=HEAD) 2024-09-10T11:34:27.571 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:27 smithi144 bash[98376]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0 2024-09-10T11:34:27.906 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:27 smithi144 ceph-mon[87436]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T11:34:27.906 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:27 smithi144 ceph-mon[87436]: osdmap e51: 6 total, 5 up, 6 in 2024-09-10T11:34:27.906 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:27 smithi144 podman[98396]: 2024-09-10 11:34:27.570605998 +0000 UTC m=+0.606640566 container remove 2435febebdb51b8eefd129faedf13ad81ef0606aa5d61c258670271ad494f889 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, ceph=True, RELEASE=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, org.label-schema.vendor=CentOS) 2024-09-10T11:34:28.018 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:27 smithi178 ceph-mon[65940]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T11:34:28.019 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:27 smithi178 ceph-mon[65940]: osdmap e51: 6 total, 5 up, 6 in 2024-09-10T11:34:28.184 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:27 smithi144 podman[98484]: 2024-09-10 11:34:27.808637406 +0000 UTC m=+0.016336928 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:34:28.184 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:27 smithi144 podman[98484]: 2024-09-10 11:34:27.930011976 +0000 UTC m=+0.137711493 container create 3fac5e7109024aae42a458646902727c56edcb1c12aa3ce4db12440668a47f65 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:34:28.634 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:28 smithi144 podman[98484]: 2024-09-10 11:34:28.205621428 +0000 UTC m=+0.413320946 container init 3fac5e7109024aae42a458646902727c56edcb1c12aa3ce4db12440668a47f65 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731) 2024-09-10T11:34:28.634 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:28 smithi144 podman[98484]: 2024-09-10 11:34:28.209272275 +0000 UTC m=+0.416971785 container start 3fac5e7109024aae42a458646902727c56edcb1c12aa3ce4db12440668a47f65 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-10T11:34:28.634 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:28 smithi144 podman[98484]: 2024-09-10 11:34:28.291029894 +0000 UTC m=+0.498729405 container attach 3fac5e7109024aae42a458646902727c56edcb1c12aa3ce4db12440668a47f65 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:34:28.634 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:28 smithi144 podman[98484]: 2024-09-10 11:34:28.402446534 +0000 UTC m=+0.610146047 container died 3fac5e7109024aae42a458646902727c56edcb1c12aa3ce4db12440668a47f65 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-09-10T11:34:28.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:28 smithi144 ceph-mon[87436]: pgmap v41: 65 pgs: 9 stale+active+clean, 56 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T11:34:28.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:28 smithi144 ceph-mon[87436]: osdmap e52: 6 total, 5 up, 6 in 2024-09-10T11:34:28.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:28.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:34:29.034 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:28 smithi178 ceph-mon[65940]: pgmap v41: 65 pgs: 9 stale+active+clean, 56 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T11:34:29.034 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:28 smithi178 ceph-mon[65940]: osdmap e52: 6 total, 5 up, 6 in 2024-09-10T11:34:29.034 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:29.034 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:34:29.434 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:29 smithi144 podman[98508]: 2024-09-10 11:34:29.128416872 +0000 UTC m=+0.719797049 container remove 3fac5e7109024aae42a458646902727c56edcb1c12aa3ce4db12440668a47f65 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T11:34:29.434 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:29 smithi144 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.0.service: Deactivated successfully. 2024-09-10T11:34:29.434 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:29 smithi144 systemd[1]: Stopped Ceph osd.0 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:34:29.434 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:29 smithi144 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.0.service: Consumed 53.167s CPU time. 2024-09-10T11:34:29.434 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:29 smithi144 systemd[1]: Starting Ceph osd.0 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:34:29.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:29 smithi178 ceph-mon[65940]: Health check failed: Degraded data redundancy: 70/360 objects degraded (19.444%), 19 pgs degraded (PG_DEGRADED) 2024-09-10T11:34:29.908 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:29 smithi144 ceph-mon[87436]: Health check failed: Degraded data redundancy: 70/360 objects degraded (19.444%), 19 pgs degraded (PG_DEGRADED) 2024-09-10T11:34:29.908 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:29 smithi144 podman[98608]: 2024-09-10 11:34:29.482288569 +0000 UTC m=+0.015814128 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:34:29.908 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:29 smithi144 podman[98608]: 2024-09-10 11:34:29.588338696 +0000 UTC m=+0.121864252 container create 775fb00146dfa8efbdc1470dfa83c9e893f35c3853aea010c478ddac4fa0ab22 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-activate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T11:34:29.908 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:29 smithi144 podman[98608]: 2024-09-10 11:34:29.824358737 +0000 UTC m=+0.357884298 container init 775fb00146dfa8efbdc1470dfa83c9e893f35c3853aea010c478ddac4fa0ab22 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-activate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T11:34:29.908 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:29 smithi144 podman[98608]: 2024-09-10 11:34:29.827736026 +0000 UTC m=+0.361261597 container start 775fb00146dfa8efbdc1470dfa83c9e893f35c3853aea010c478ddac4fa0ab22 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-activate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T11:34:29.908 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:29 smithi144 podman[98608]: 2024-09-10 11:34:29.907569014 +0000 UTC m=+0.441094576 container attach 775fb00146dfa8efbdc1470dfa83c9e893f35c3853aea010c478ddac4fa0ab22 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-activate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T11:34:30.184 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:29 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-activate[98623]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:34:30.184 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:29 smithi144 bash[98608]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:34:30.184 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:29 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-activate[98623]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:34:30.184 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:29 smithi144 bash[98608]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:34:30.988 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:30 smithi144 ceph-mon[87436]: pgmap v43: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70/360 objects degraded (19.444%) 2024-09-10T11:34:30.988 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:30 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-activate[98623]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T11:34:30.988 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:30 smithi144 bash[98608]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T11:34:30.988 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:30 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-activate[98623]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-10T11:34:30.988 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:30 smithi144 bash[98608]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-10T11:34:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:30 smithi178 ceph-mon[65940]: pgmap v43: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70/360 objects degraded (19.444%) 2024-09-10T11:34:31.320 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:30 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-activate[98623]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-10T11:34:31.320 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:30 smithi144 bash[98608]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-10T11:34:31.321 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:30 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-activate[98623]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-10T11:34:31.321 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:30 smithi144 bash[98608]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-10T11:34:31.321 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:30 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-activate[98623]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-10T11:34:31.321 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:30 smithi144 bash[98608]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-09-10T11:34:31.321 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:30 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-activate[98623]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T11:34:31.321 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:30 smithi144 bash[98608]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-10T11:34:31.321 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:30 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-activate[98623]: --> ceph-volume raw activate successful for osd ID: 0 2024-09-10T11:34:31.321 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:30 smithi144 bash[98608]: --> ceph-volume raw activate successful for osd ID: 0 2024-09-10T11:34:31.321 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:31 smithi144 podman[98763]: 2024-09-10 11:34:31.042818363 +0000 UTC m=+0.016845011 container died 775fb00146dfa8efbdc1470dfa83c9e893f35c3853aea010c478ddac4fa0ab22 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-activate, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:34:32.037 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:31 smithi144 podman[98763]: 2024-09-10 11:34:31.703355739 +0000 UTC m=+0.677382379 container remove 775fb00146dfa8efbdc1470dfa83c9e893f35c3853aea010c478ddac4fa0ab22 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-activate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T11:34:32.037 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:31 smithi144 podman[98823]: 2024-09-10 11:34:31.810172122 +0000 UTC m=+0.015928149 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:34:32.037 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:31 smithi144 podman[98823]: 2024-09-10 11:34:31.929566408 +0000 UTC m=+0.135322442 container create b59b6d7c85be8431611e56a640620c4986cae932653a0e234da48bbd1911b09c (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T11:34:32.434 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:32 smithi144 podman[98823]: 2024-09-10 11:34:32.173543914 +0000 UTC m=+0.379299942 container init b59b6d7c85be8431611e56a640620c4986cae932653a0e234da48bbd1911b09c (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T11:34:32.434 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:32 smithi144 podman[98823]: 2024-09-10 11:34:32.176636674 +0000 UTC m=+0.382392701 container start b59b6d7c85be8431611e56a640620c4986cae932653a0e234da48bbd1911b09c (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD) 2024-09-10T11:34:32.434 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:32 smithi144 ceph-osd[98842]: -- 172.21.15.144:0/2853213024 <== mon.0 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556b2585e1e0 con 0x556b25846000 2024-09-10T11:34:32.434 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:32 smithi144 bash[98823]: b59b6d7c85be8431611e56a640620c4986cae932653a0e234da48bbd1911b09c 2024-09-10T11:34:32.434 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:32 smithi144 systemd[1]: Started Ceph osd.0 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:34:33.036 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:32 smithi144 ceph-mon[87436]: pgmap v44: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70/360 objects degraded (19.444%) 2024-09-10T11:34:33.036 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:32 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:33.036 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:32 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:33.036 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:32 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:34:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:32 smithi178 ceph-mon[65940]: pgmap v44: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70/360 objects degraded (19.444%) 2024-09-10T11:34:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:32 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:32 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:32 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:34:34.184 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:33 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0[98838]: 2024-09-10T11:34:33.820+0000 7fa4caf7b640 -1 Falling back to public interface 2024-09-10T11:34:34.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:34 smithi144 ceph-mon[87436]: pgmap v45: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70/360 objects degraded (19.444%) 2024-09-10T11:34:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:34 smithi178 ceph-mon[65940]: pgmap v45: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70/360 objects degraded (19.444%) 2024-09-10T11:34:36.896 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:36 smithi144 ceph-mon[87436]: pgmap v46: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70/360 objects degraded (19.444%) 2024-09-10T11:34:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:36 smithi178 ceph-mon[65940]: pgmap v46: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70/360 objects degraded (19.444%) 2024-09-10T11:34:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:38 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:38 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:38 smithi178 ceph-mon[65940]: pgmap v47: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70/360 objects degraded (19.444%) 2024-09-10T11:34:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:38 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:38 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:38.652 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:38 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:38.652 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:38 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:38.652 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:38 smithi144 ceph-mon[87436]: pgmap v47: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70/360 objects degraded (19.444%) 2024-09-10T11:34:38.652 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:38 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:38.652 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:38 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:39.620 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:39 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0[98838]: 2024-09-10T11:34:39.333+0000 7fa4caf7b640 -1 osd.0 0 read_superblock omap replica is missing. 2024-09-10T11:34:39.899 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:39 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0[98838]: 2024-09-10T11:34:39.618+0000 7fa4caf7b640 -1 osd.0 50 log_to_monitors true 2024-09-10T11:34:40.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.166+0000 7f5aecab7700 1 -- 172.21.15.144:0/1607462130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae80ffc30 msgr2=0x7f5ae810a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.166+0000 7f5aecab7700 1 --2- 172.21.15.144:0/1607462130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae80ffc30 0x7f5ae810a6a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f5adc009940 tx=0x7f5adc009c50 comp rx=0 tx=0).stop 2024-09-10T11:34:40.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.166+0000 7f5aecab7700 1 -- 172.21.15.144:0/1607462130 shutdown_connections 2024-09-10T11:34:40.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.166+0000 7f5aecab7700 1 --2- 172.21.15.144:0/1607462130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae80ffc30 0x7f5ae810a6a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.166+0000 7f5aecab7700 1 --2- 172.21.15.144:0/1607462130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ae80ff320 0x7f5ae80ff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.166+0000 7f5aecab7700 1 -- 172.21.15.144:0/1607462130 >> 172.21.15.144:0/1607462130 conn(0x7f5ae80748b0 msgr2=0x7f5ae8074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:40.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.166+0000 7f5aecab7700 1 -- 172.21.15.144:0/1607462130 shutdown_connections 2024-09-10T11:34:40.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.166+0000 7f5aecab7700 1 -- 172.21.15.144:0/1607462130 wait complete. 2024-09-10T11:34:40.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.167+0000 7f5aecab7700 1 Processor -- start 2024-09-10T11:34:40.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.167+0000 7f5aecab7700 1 -- start start 2024-09-10T11:34:40.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.167+0000 7f5aecab7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae80ff320 0x7f5ae819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.167+0000 7f5aecab7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ae80ffc30 0x7f5ae819bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.167+0000 7f5aecab7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae819c600 con 0x7f5ae80ff320 2024-09-10T11:34:40.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.167+0000 7f5aecab7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae8195ab0 con 0x7f5ae80ffc30 2024-09-10T11:34:40.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.168+0000 7f5ae659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae80ff320 0x7f5ae819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.168+0000 7f5ae659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae80ff320 0x7f5ae819ba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52502/0 (socket says 172.21.15.144:52502) 2024-09-10T11:34:40.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.168+0000 7f5ae659c700 1 -- 172.21.15.144:0/3934394489 learned_addr learned my addr 172.21.15.144:0/3934394489 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:34:40.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.168+0000 7f5ae5d9b700 1 --2- 172.21.15.144:0/3934394489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ae80ffc30 0x7f5ae819bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.168+0000 7f5ae659c700 1 -- 172.21.15.144:0/3934394489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ae80ffc30 msgr2=0x7f5ae819bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.168+0000 7f5ae659c700 1 --2- 172.21.15.144:0/3934394489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ae80ffc30 0x7f5ae819bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.168+0000 7f5ae659c700 1 -- 172.21.15.144:0/3934394489 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc0095f0 con 0x7f5ae80ff320 2024-09-10T11:34:40.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.168+0000 7f5ae659c700 1 --2- 172.21.15.144:0/3934394489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae80ff320 0x7f5ae819ba30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800d900 tx=0x7f5ad800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:40.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.168+0000 7f5ad77fe700 1 -- 172.21.15.144:0/3934394489 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ad80041f0 con 0x7f5ae80ff320 2024-09-10T11:34:40.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.168+0000 7f5ad77fe700 1 -- 172.21.15.144:0/3934394489 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ad8004350 con 0x7f5ae80ff320 2024-09-10T11:34:40.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.168+0000 7f5aecab7700 1 -- 172.21.15.144:0/3934394489 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae8195d90 con 0x7f5ae80ff320 2024-09-10T11:34:40.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.168+0000 7f5ad77fe700 1 -- 172.21.15.144:0/3934394489 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ad8003cf0 con 0x7f5ae80ff320 2024-09-10T11:34:40.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.168+0000 7f5aecab7700 1 -- 172.21.15.144:0/3934394489 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae81962b0 con 0x7f5ae80ff320 2024-09-10T11:34:40.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.169+0000 7f5aecab7700 1 -- 172.21.15.144:0/3934394489 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae8107da0 con 0x7f5ae80ff320 2024-09-10T11:34:40.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.170+0000 7f5ad77fe700 1 -- 172.21.15.144:0/3934394489 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f5ad8009730 con 0x7f5ae80ff320 2024-09-10T11:34:40.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.170+0000 7f5ad77fe700 1 --2- 172.21.15.144:0/3934394489 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5ad006e340 0x7f5ad00707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.171+0000 7f5ad77fe700 1 -- 172.21.15.144:0/3934394489 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5ad808e4b0 con 0x7f5ae80ff320 2024-09-10T11:34:40.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.171+0000 7f5ae5d9b700 1 --2- 172.21.15.144:0/3934394489 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5ad006e340 0x7f5ad00707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.171+0000 7f5ae5d9b700 1 --2- 172.21.15.144:0/3934394489 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5ad006e340 0x7f5ad00707f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5adc006010 tx=0x7f5adc005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:40.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.174+0000 7f5ad77fe700 1 -- 172.21.15.144:0/3934394489 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5ad805b370 con 0x7f5ae80ff320 2024-09-10T11:34:40.408 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.406+0000 7f5aecab7700 1 -- 172.21.15.144:0/3934394489 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ae8196600 con 0x7f5ad006e340 2024-09-10T11:34:40.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.408+0000 7f5ad77fe700 1 -- 172.21.15.144:0/3934394489 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5ae8196600 con 0x7f5ad006e340 2024-09-10T11:34:40.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.410+0000 7f5aecab7700 1 -- 172.21.15.144:0/3934394489 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5ad006e340 msgr2=0x7f5ad00707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.410+0000 7f5aecab7700 1 --2- 172.21.15.144:0/3934394489 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5ad006e340 0x7f5ad00707f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5adc006010 tx=0x7f5adc005a90 comp rx=0 tx=0).stop 2024-09-10T11:34:40.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.410+0000 7f5aecab7700 1 -- 172.21.15.144:0/3934394489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae80ff320 msgr2=0x7f5ae819ba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.410+0000 7f5aecab7700 1 --2- 172.21.15.144:0/3934394489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae80ff320 0x7f5ae819ba30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800d900 tx=0x7f5ad800dc10 comp rx=0 tx=0).stop 2024-09-10T11:34:40.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.410+0000 7f5aecab7700 1 -- 172.21.15.144:0/3934394489 shutdown_connections 2024-09-10T11:34:40.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.410+0000 7f5aecab7700 1 --2- 172.21.15.144:0/3934394489 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5ad006e340 0x7f5ad00707f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.410+0000 7f5aecab7700 1 --2- 172.21.15.144:0/3934394489 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ae80ff320 0x7f5ae819ba30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.410+0000 7f5aecab7700 1 --2- 172.21.15.144:0/3934394489 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ae80ffc30 0x7f5ae819bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.410+0000 7f5aecab7700 1 -- 172.21.15.144:0/3934394489 >> 172.21.15.144:0/3934394489 conn(0x7f5ae80748b0 msgr2=0x7f5ae8104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:40.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.410+0000 7f5aecab7700 1 -- 172.21.15.144:0/3934394489 shutdown_connections 2024-09-10T11:34:40.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.410+0000 7f5aecab7700 1 -- 172.21.15.144:0/3934394489 wait complete. 2024-09-10T11:34:40.423 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:34:40.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.527+0000 7f96b8298700 1 -- 172.21.15.144:0/3006598893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b0101340 msgr2=0x7f96b0101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.527+0000 7f96b8298700 1 --2- 172.21.15.144:0/3006598893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b0101340 0x7f96b0101710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f96a0009940 tx=0x7f96a0009c50 comp rx=0 tx=0).stop 2024-09-10T11:34:40.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.527+0000 7f96b8298700 1 -- 172.21.15.144:0/3006598893 shutdown_connections 2024-09-10T11:34:40.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.527+0000 7f96b8298700 1 --2- 172.21.15.144:0/3006598893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96b0101c50 0x7f96b010a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.527+0000 7f96b8298700 1 --2- 172.21.15.144:0/3006598893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b0101340 0x7f96b0101710 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.527+0000 7f96b8298700 1 -- 172.21.15.144:0/3006598893 >> 172.21.15.144:0/3006598893 conn(0x7f96b00fab80 msgr2=0x7f96b00fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:40.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.527+0000 7f96b8298700 1 -- 172.21.15.144:0/3006598893 shutdown_connections 2024-09-10T11:34:40.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.527+0000 7f96b8298700 1 -- 172.21.15.144:0/3006598893 wait complete. 2024-09-10T11:34:40.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.528+0000 7f96b8298700 1 Processor -- start 2024-09-10T11:34:40.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.528+0000 7f96b8298700 1 -- start start 2024-09-10T11:34:40.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.528+0000 7f96b8298700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96b0101340 0x7f96b0195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.528+0000 7f96b8298700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b0101c50 0x7f96b0196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.528+0000 7f96b8298700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b01968e0 con 0x7f96b0101c50 2024-09-10T11:34:40.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.528+0000 7f96b8298700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b01995c0 con 0x7f96b0101340 2024-09-10T11:34:40.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.528+0000 7f96b5833700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b0101c50 0x7f96b0196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.528+0000 7f96b5833700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b0101c50 0x7f96b0196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52524/0 (socket says 172.21.15.144:52524) 2024-09-10T11:34:40.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.528+0000 7f96b5833700 1 -- 172.21.15.144:0/3651671419 learned_addr learned my addr 172.21.15.144:0/3651671419 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:34:40.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.528+0000 7f96b6034700 1 --2- 172.21.15.144:0/3651671419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96b0101340 0x7f96b0195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.528+0000 7f96b5833700 1 -- 172.21.15.144:0/3651671419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96b0101340 msgr2=0x7f96b0195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.528+0000 7f96b5833700 1 --2- 172.21.15.144:0/3651671419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96b0101340 0x7f96b0195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.528+0000 7f96b5833700 1 -- 172.21.15.144:0/3651671419 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96a00095f0 con 0x7f96b0101c50 2024-09-10T11:34:40.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.529+0000 7f96b5833700 1 --2- 172.21.15.144:0/3651671419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b0101c50 0x7f96b0196250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00d8d0 tx=0x7f96ac00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:40.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.529+0000 7f96a77fe700 1 -- 172.21.15.144:0/3651671419 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96ac009940 con 0x7f96b0101c50 2024-09-10T11:34:40.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.529+0000 7f96a77fe700 1 -- 172.21.15.144:0/3651671419 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f96ac010460 con 0x7f96b0101c50 2024-09-10T11:34:40.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.529+0000 7f96a77fe700 1 -- 172.21.15.144:0/3651671419 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96ac00f5f0 con 0x7f96b0101c50 2024-09-10T11:34:40.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.529+0000 7f96b8298700 1 -- 172.21.15.144:0/3651671419 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b01998a0 con 0x7f96b0101c50 2024-09-10T11:34:40.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.529+0000 7f96b8298700 1 -- 172.21.15.144:0/3651671419 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b0199df0 con 0x7f96b0101c50 2024-09-10T11:34:40.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.530+0000 7f96b8298700 1 -- 172.21.15.144:0/3651671419 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b0107820 con 0x7f96b0101c50 2024-09-10T11:34:40.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.530+0000 7f96a77fe700 1 -- 172.21.15.144:0/3651671419 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f96ac0105d0 con 0x7f96b0101c50 2024-09-10T11:34:40.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.531+0000 7f96a77fe700 1 --2- 172.21.15.144:0/3651671419 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f969c06e330 0x7f969c0707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.531+0000 7f96a77fe700 1 -- 172.21.15.144:0/3651671419 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f96ac08e280 con 0x7f96b0101c50 2024-09-10T11:34:40.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.531+0000 7f96b6034700 1 --2- 172.21.15.144:0/3651671419 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f969c06e330 0x7f969c0707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.531+0000 7f96b6034700 1 --2- 172.21.15.144:0/3651671419 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f969c06e330 0x7f969c0707e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f96a000b870 tx=0x7f96a0005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:40.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.535+0000 7f96a77fe700 1 -- 172.21.15.144:0/3651671419 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f96ac0574f0 con 0x7f96b0101c50 2024-09-10T11:34:40.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:40 smithi144 ceph-mon[87436]: pgmap v48: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70/360 objects degraded (19.444%) 2024-09-10T11:34:40.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:40 smithi144 ceph-mon[87436]: from='osd.0 [v2:172.21.15.144:6802/3898822309,v1:172.21.15.144:6803/3898822309]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T11:34:40.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.843+0000 7f96b8298700 1 -- 172.21.15.144:0/3651671419 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f96b019a0d0 con 0x7f969c06e330 2024-09-10T11:34:40.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.844+0000 7f96a77fe700 1 -- 172.21.15.144:0/3651671419 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f96b019a0d0 con 0x7f969c06e330 2024-09-10T11:34:40.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.847+0000 7f96b8298700 1 -- 172.21.15.144:0/3651671419 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f969c06e330 msgr2=0x7f969c0707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.847+0000 7f96b8298700 1 --2- 172.21.15.144:0/3651671419 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f969c06e330 0x7f969c0707e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f96a000b870 tx=0x7f96a0005a90 comp rx=0 tx=0).stop 2024-09-10T11:34:40.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.847+0000 7f96b8298700 1 -- 172.21.15.144:0/3651671419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b0101c50 msgr2=0x7f96b0196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.847+0000 7f96b8298700 1 --2- 172.21.15.144:0/3651671419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b0101c50 0x7f96b0196250 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00d8d0 tx=0x7f96ac00dc90 comp rx=0 tx=0).stop 2024-09-10T11:34:40.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.847+0000 7f96b8298700 1 -- 172.21.15.144:0/3651671419 shutdown_connections 2024-09-10T11:34:40.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.847+0000 7f96b8298700 1 --2- 172.21.15.144:0/3651671419 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f969c06e330 0x7f969c0707e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.847+0000 7f96b8298700 1 --2- 172.21.15.144:0/3651671419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f96b0101340 0x7f96b0195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.847+0000 7f96b8298700 1 --2- 172.21.15.144:0/3651671419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f96b0101c50 0x7f96b0196250 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.847+0000 7f96b8298700 1 -- 172.21.15.144:0/3651671419 >> 172.21.15.144:0/3651671419 conn(0x7f96b00fab80 msgr2=0x7f96b0104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:40.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.847+0000 7f96b8298700 1 -- 172.21.15.144:0/3651671419 shutdown_connections 2024-09-10T11:34:40.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.848+0000 7f96b8298700 1 -- 172.21.15.144:0/3651671419 wait complete. 2024-09-10T11:34:40.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.971+0000 7f0a35f9d700 1 -- 172.21.15.144:0/1948735851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a30101c90 msgr2=0x7f0a3010a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.971+0000 7f0a35f9d700 1 --2- 172.21.15.144:0/1948735851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a30101c90 0x7f0a3010a190 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0a24009910 tx=0x7f0a24009c20 comp rx=0 tx=0).stop 2024-09-10T11:34:40.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.971+0000 7f0a35f9d700 1 -- 172.21.15.144:0/1948735851 shutdown_connections 2024-09-10T11:34:40.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.971+0000 7f0a35f9d700 1 --2- 172.21.15.144:0/1948735851 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a30101c90 0x7f0a3010a190 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.971+0000 7f0a35f9d700 1 --2- 172.21.15.144:0/1948735851 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a30101380 0x7f0a30101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.971+0000 7f0a35f9d700 1 -- 172.21.15.144:0/1948735851 >> 172.21.15.144:0/1948735851 conn(0x7f0a300fab80 msgr2=0x7f0a300fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:40.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.971+0000 7f0a35f9d700 1 -- 172.21.15.144:0/1948735851 shutdown_connections 2024-09-10T11:34:40.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.971+0000 7f0a35f9d700 1 -- 172.21.15.144:0/1948735851 wait complete. 2024-09-10T11:34:40.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.972+0000 7f0a35f9d700 1 Processor -- start 2024-09-10T11:34:40.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.972+0000 7f0a35f9d700 1 -- start start 2024-09-10T11:34:40.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.972+0000 7f0a35f9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a30101380 0x7f0a300fefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.972+0000 7f0a35f9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a30101c90 0x7f0a300ff520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.972+0000 7f0a35f9d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a300ffa60 con 0x7f0a30101c90 2024-09-10T11:34:40.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.972+0000 7f0a35f9d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a300ffba0 con 0x7f0a30101380 2024-09-10T11:34:40.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.973+0000 7f0a2effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a30101c90 0x7f0a300ff520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.973+0000 7f0a2f7fe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a30101380 0x7f0a300fefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.973+0000 7f0a2effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a30101c90 0x7f0a300ff520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52530/0 (socket says 172.21.15.144:52530) 2024-09-10T11:34:40.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.973+0000 7f0a2effd700 1 -- 172.21.15.144:0/646502739 learned_addr learned my addr 172.21.15.144:0/646502739 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:34:40.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.973+0000 7f0a2effd700 1 -- 172.21.15.144:0/646502739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a30101380 msgr2=0x7f0a300fefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:40.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.973+0000 7f0a2effd700 1 --2- 172.21.15.144:0/646502739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a30101380 0x7f0a300fefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:40.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.973+0000 7f0a2effd700 1 -- 172.21.15.144:0/646502739 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a240095f0 con 0x7f0a30101c90 2024-09-10T11:34:40.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.973+0000 7f0a2effd700 1 --2- 172.21.15.144:0/646502739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a30101c90 0x7f0a300ff520 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0a24006010 tx=0x7f0a2400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:40.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.973+0000 7f0a2cff9700 1 -- 172.21.15.144:0/646502739 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a2401d070 con 0x7f0a30101c90 2024-09-10T11:34:40.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.973+0000 7f0a35f9d700 1 -- 172.21.15.144:0/646502739 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a300ffe20 con 0x7f0a30101c90 2024-09-10T11:34:40.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.973+0000 7f0a2cff9700 1 -- 172.21.15.144:0/646502739 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a2400f460 con 0x7f0a30101c90 2024-09-10T11:34:40.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.973+0000 7f0a2cff9700 1 -- 172.21.15.144:0/646502739 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a24017630 con 0x7f0a30101c90 2024-09-10T11:34:40.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.973+0000 7f0a35f9d700 1 -- 172.21.15.144:0/646502739 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a301a21b0 con 0x7f0a30101c90 2024-09-10T11:34:40.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.975+0000 7f0a35f9d700 1 -- 172.21.15.144:0/646502739 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a30107800 con 0x7f0a30101c90 2024-09-10T11:34:40.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.975+0000 7f0a2cff9700 1 -- 172.21.15.144:0/646502739 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f0a2400f5d0 con 0x7f0a30101c90 2024-09-10T11:34:40.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.976+0000 7f0a2cff9700 1 --2- 172.21.15.144:0/646502739 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0a1806e340 0x7f0a180707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:40.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.976+0000 7f0a2cff9700 1 -- 172.21.15.144:0/646502739 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0a2408f790 con 0x7f0a30101c90 2024-09-10T11:34:40.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.976+0000 7f0a2f7fe700 1 --2- 172.21.15.144:0/646502739 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0a1806e340 0x7f0a180707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:40.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.976+0000 7f0a2f7fe700 1 --2- 172.21.15.144:0/646502739 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0a1806e340 0x7f0a180707f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000ac50 tx=0x7f0a2000a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:40.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:40.980+0000 7f0a2cff9700 1 -- 172.21.15.144:0/646502739 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0a24058a00 con 0x7f0a30101c90 2024-09-10T11:34:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:40 smithi178 ceph-mon[65940]: pgmap v48: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 70/360 objects degraded (19.444%) 2024-09-10T11:34:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:40 smithi178 ceph-mon[65940]: from='osd.0 [v2:172.21.15.144:6802/3898822309,v1:172.21.15.144:6803/3898822309]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T11:34:41.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.253+0000 7f0a35f9d700 1 -- 172.21.15.144:0/646502739 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0a3002cc80 con 0x7f0a1806e340 2024-09-10T11:34:41.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.271+0000 7f0a2cff9700 1 -- 172.21.15.144:0/646502739 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f0a3002cc80 con 0x7f0a1806e340 2024-09-10T11:34:41.272 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:34:41.272 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (2m) 3s ago 13m 25.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:34:41.273 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (13m) 3s ago 13m 8182k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:34:41.273 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (12m) 26s ago 12m 8090k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:34:41.273 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (34s) 3s ago 13m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:34:41.273 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (29s) 26s ago 12m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:34:41.273 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (2m) 3s ago 13m 79.5M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:34:41.273 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (8m) 3s ago 8m 60.1M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:34:41.273 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (7m) 3s ago 7m 126M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:34:41.273 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (7m) 26s ago 7m 62.2M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:34:41.273 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (8m) 26s ago 8m 127M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:34:41.273 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (4m) 3s ago 14m 494M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:34:41.273 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (3m) 26s ago 12m 439M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:34:41.273 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (94s) 3s ago 15m 46.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:34:41.273 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (72s) 26s ago 12m 35.5M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:34:41.274 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (3m) 3s ago 13m 15.9M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:34:41.274 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (3m) 26s ago 12m 15.8M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:34:41.274 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (9s) 3s ago 11m 11.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:34:41.274 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (11m) 3s ago 11m 214M 1216M 18.2.0 dc2bc1663786 b55fe920ff9d 2024-09-10T11:34:41.274 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (10m) 3s ago 10m 171M 1216M 18.2.0 dc2bc1663786 3ed5618eac33 2024-09-10T11:34:41.274 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (10m) 26s ago 10m 255M 2239M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:34:41.274 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (9m) 26s ago 9m 219M 2239M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:34:41.274 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (9m) 26s ago 9m 198M 2239M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:34:41.274 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (3m) 3s ago 13m 77.2M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:34:41.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.274+0000 7f0a35f9d700 1 -- 172.21.15.144:0/646502739 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0a1806e340 msgr2=0x7f0a180707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.274+0000 7f0a35f9d700 1 --2- 172.21.15.144:0/646502739 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0a1806e340 0x7f0a180707f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a2000ac50 tx=0x7f0a2000a380 comp rx=0 tx=0).stop 2024-09-10T11:34:41.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.274+0000 7f0a35f9d700 1 -- 172.21.15.144:0/646502739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a30101c90 msgr2=0x7f0a300ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.274+0000 7f0a35f9d700 1 --2- 172.21.15.144:0/646502739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a30101c90 0x7f0a300ff520 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0a24006010 tx=0x7f0a2400bd70 comp rx=0 tx=0).stop 2024-09-10T11:34:41.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.274+0000 7f0a35f9d700 1 -- 172.21.15.144:0/646502739 shutdown_connections 2024-09-10T11:34:41.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.274+0000 7f0a35f9d700 1 --2- 172.21.15.144:0/646502739 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0a1806e340 0x7f0a180707f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.274+0000 7f0a35f9d700 1 --2- 172.21.15.144:0/646502739 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0a30101380 0x7f0a300fefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.274+0000 7f0a35f9d700 1 --2- 172.21.15.144:0/646502739 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0a30101c90 0x7f0a300ff520 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.274+0000 7f0a35f9d700 1 -- 172.21.15.144:0/646502739 >> 172.21.15.144:0/646502739 conn(0x7f0a300fab80 msgr2=0x7f0a30104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:41.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.274+0000 7f0a35f9d700 1 -- 172.21.15.144:0/646502739 shutdown_connections 2024-09-10T11:34:41.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.274+0000 7f0a35f9d700 1 -- 172.21.15.144:0/646502739 wait complete. 2024-09-10T11:34:41.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.399+0000 7f62423a7700 1 -- 172.21.15.144:0/332281441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f623c0ff2e0 msgr2=0x7f623c0ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.399+0000 7f62423a7700 1 --2- 172.21.15.144:0/332281441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f623c0ff2e0 0x7f623c0ff6b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f622c009910 tx=0x7f622c009c20 comp rx=0 tx=0).stop 2024-09-10T11:34:41.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.400+0000 7f62423a7700 1 -- 172.21.15.144:0/332281441 shutdown_connections 2024-09-10T11:34:41.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.400+0000 7f62423a7700 1 --2- 172.21.15.144:0/332281441 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f623c0ffc80 0x7f623c103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.400+0000 7f62423a7700 1 --2- 172.21.15.144:0/332281441 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f623c0ff2e0 0x7f623c0ff6b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.400+0000 7f62423a7700 1 -- 172.21.15.144:0/332281441 >> 172.21.15.144:0/332281441 conn(0x7f623c0fab80 msgr2=0x7f623c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:41.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.400+0000 7f62423a7700 1 -- 172.21.15.144:0/332281441 shutdown_connections 2024-09-10T11:34:41.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.400+0000 7f62423a7700 1 -- 172.21.15.144:0/332281441 wait complete. 2024-09-10T11:34:41.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.400+0000 7f62423a7700 1 Processor -- start 2024-09-10T11:34:41.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.400+0000 7f62423a7700 1 -- start start 2024-09-10T11:34:41.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.401+0000 7f62423a7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f623c0ff2e0 0x7f623c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:41.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.401+0000 7f62423a7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f623c0ffc80 0x7f623c1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:41.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.401+0000 7f62423a7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c196980 con 0x7f623c0ffc80 2024-09-10T11:34:41.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.401+0000 7f62423a7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c19a710 con 0x7f623c0ff2e0 2024-09-10T11:34:41.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.401+0000 7f623b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f623c0ffc80 0x7f623c1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:41.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.401+0000 7f623b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f623c0ffc80 0x7f623c1962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52548/0 (socket says 172.21.15.144:52548) 2024-09-10T11:34:41.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.401+0000 7f623b7fe700 1 -- 172.21.15.144:0/2141036203 learned_addr learned my addr 172.21.15.144:0/2141036203 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:34:41.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.401+0000 7f623bfff700 1 --2- 172.21.15.144:0/2141036203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f623c0ff2e0 0x7f623c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:41.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.401+0000 7f623b7fe700 1 -- 172.21.15.144:0/2141036203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f623c0ff2e0 msgr2=0x7f623c195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.401+0000 7f623b7fe700 1 --2- 172.21.15.144:0/2141036203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f623c0ff2e0 0x7f623c195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.401+0000 7f623b7fe700 1 -- 172.21.15.144:0/2141036203 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c0095f0 con 0x7f623c0ffc80 2024-09-10T11:34:41.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.401+0000 7f623b7fe700 1 --2- 172.21.15.144:0/2141036203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f623c0ffc80 0x7f623c1962a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f623000d8d0 tx=0x7f623000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:41.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.402+0000 7f62397fa700 1 -- 172.21.15.144:0/2141036203 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62300041f0 con 0x7f623c0ffc80 2024-09-10T11:34:41.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.402+0000 7f62397fa700 1 -- 172.21.15.144:0/2141036203 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6230007620 con 0x7f623c0ffc80 2024-09-10T11:34:41.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.402+0000 7f62423a7700 1 -- 172.21.15.144:0/2141036203 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c19a9f0 con 0x7f623c0ffc80 2024-09-10T11:34:41.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.402+0000 7f62397fa700 1 -- 172.21.15.144:0/2141036203 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6230003c50 con 0x7f623c0ffc80 2024-09-10T11:34:41.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.402+0000 7f62423a7700 1 -- 172.21.15.144:0/2141036203 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c19af40 con 0x7f623c0ffc80 2024-09-10T11:34:41.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.403+0000 7f62423a7700 1 -- 172.21.15.144:0/2141036203 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c04ea50 con 0x7f623c0ffc80 2024-09-10T11:34:41.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.403+0000 7f62397fa700 1 -- 172.21.15.144:0/2141036203 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f6230004350 con 0x7f623c0ffc80 2024-09-10T11:34:41.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.404+0000 7f62397fa700 1 --2- 172.21.15.144:0/2141036203 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f622406e340 0x7f62240707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:41.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.404+0000 7f62397fa700 1 -- 172.21.15.144:0/2141036203 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f623008de20 con 0x7f623c0ffc80 2024-09-10T11:34:41.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.404+0000 7f623bfff700 1 --2- 172.21.15.144:0/2141036203 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f622406e340 0x7f62240707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:41.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.404+0000 7f623bfff700 1 --2- 172.21.15.144:0/2141036203 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f622406e340 0x7f62240707f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f622c00b5c0 tx=0x7f622c005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:41.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.412+0000 7f62397fa700 1 -- 172.21.15.144:0/2141036203 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6230057090 con 0x7f623c0ffc80 2024-09-10T11:34:41.756 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.755+0000 7f62423a7700 1 -- 172.21.15.144:0/2141036203 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f623c1970c0 con 0x7f623c0ffc80 2024-09-10T11:34:41.757 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 11:34:41 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0[98838]: 2024-09-10T11:34:41.478+0000 7fa4bed0d640 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:34:41.757 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:41 smithi144 ceph-mon[87436]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:41.757 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:41 smithi144 ceph-mon[87436]: from='osd.0 [v2:172.21.15.144:6802/3898822309,v1:172.21.15.144:6803/3898822309]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T11:34:41.757 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:41 smithi144 ceph-mon[87436]: osdmap e53: 6 total, 5 up, 6 in 2024-09-10T11:34:41.757 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:41 smithi144 ceph-mon[87436]: from='osd.0 [v2:172.21.15.144:6802/3898822309,v1:172.21.15.144:6803/3898822309]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T11:34:41.757 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:41 smithi144 ceph-mon[87436]: from='osd.0 [v2:172.21.15.144:6802/3898822309,v1:172.21.15.144:6803/3898822309]' entity='osd.0' 2024-09-10T11:34:41.757 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:41 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:41.757 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:41 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:41.758 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:41 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:41.758 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:41 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:34:41.758 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:41 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:41.758 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.755+0000 7f62397fa700 1 -- 172.21.15.144:0/2141036203 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+693 (secure 0 0 0) 0x7f6230003e80 con 0x7f623c0ffc80 2024-09-10T11:34:41.758 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:34:41.758 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:34:41.758 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:34:41.758 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:34:41.758 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:34:41.758 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:34:41.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:34:41.759 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:34:41.759 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5 2024-09-10T11:34:41.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:34:41.759 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:34:41.759 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:34:41.759 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:34:41.759 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:34:41.759 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 9, 2024-09-10T11:34:41.759 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:34:41.759 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:34:41.759 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:34:41.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.758+0000 7f62423a7700 1 -- 172.21.15.144:0/2141036203 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f622406e340 msgr2=0x7f62240707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.758+0000 7f62423a7700 1 --2- 172.21.15.144:0/2141036203 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f622406e340 0x7f62240707f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f622c00b5c0 tx=0x7f622c005960 comp rx=0 tx=0).stop 2024-09-10T11:34:41.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.758+0000 7f62423a7700 1 -- 172.21.15.144:0/2141036203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f623c0ffc80 msgr2=0x7f623c1962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.758+0000 7f62423a7700 1 --2- 172.21.15.144:0/2141036203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f623c0ffc80 0x7f623c1962a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f623000d8d0 tx=0x7f623000dc90 comp rx=0 tx=0).stop 2024-09-10T11:34:41.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.758+0000 7f62423a7700 1 -- 172.21.15.144:0/2141036203 shutdown_connections 2024-09-10T11:34:41.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.758+0000 7f62423a7700 1 --2- 172.21.15.144:0/2141036203 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f622406e340 0x7f62240707f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.758+0000 7f62423a7700 1 --2- 172.21.15.144:0/2141036203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f623c0ff2e0 0x7f623c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.760 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.758+0000 7f62423a7700 1 --2- 172.21.15.144:0/2141036203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f623c0ffc80 0x7f623c1962a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.758+0000 7f62423a7700 1 -- 172.21.15.144:0/2141036203 >> 172.21.15.144:0/2141036203 conn(0x7f623c0fab80 msgr2=0x7f623c068d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:41.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.758+0000 7f62423a7700 1 -- 172.21.15.144:0/2141036203 shutdown_connections 2024-09-10T11:34:41.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.758+0000 7f62423a7700 1 -- 172.21.15.144:0/2141036203 wait complete. 2024-09-10T11:34:41.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.871+0000 7f4174b79700 1 -- 172.21.15.144:0/468725234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4170101cf0 msgr2=0x7f417010a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.871+0000 7f4174b79700 1 --2- 172.21.15.144:0/468725234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4170101cf0 0x7f417010a1f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4164009940 tx=0x7f4164009c50 comp rx=0 tx=0).stop 2024-09-10T11:34:41.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.871+0000 7f4174b79700 1 -- 172.21.15.144:0/468725234 shutdown_connections 2024-09-10T11:34:41.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.871+0000 7f4174b79700 1 --2- 172.21.15.144:0/468725234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4170101cf0 0x7f417010a1f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.871+0000 7f4174b79700 1 --2- 172.21.15.144:0/468725234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41701013e0 0x7f41701017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.871+0000 7f4174b79700 1 -- 172.21.15.144:0/468725234 >> 172.21.15.144:0/468725234 conn(0x7f41700fac20 msgr2=0x7f41700fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:41.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.871+0000 7f4174b79700 1 -- 172.21.15.144:0/468725234 shutdown_connections 2024-09-10T11:34:41.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.871+0000 7f4174b79700 1 -- 172.21.15.144:0/468725234 wait complete. 2024-09-10T11:34:41.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.872+0000 7f4174b79700 1 Processor -- start 2024-09-10T11:34:41.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.872+0000 7f4174b79700 1 -- start start 2024-09-10T11:34:41.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.872+0000 7f4174b79700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41701013e0 0x7f4170195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:41.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.872+0000 7f4174b79700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4170101cf0 0x7f4170196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:41.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.872+0000 7f4174b79700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4170196930 con 0x7f4170101cf0 2024-09-10T11:34:41.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.872+0000 7f4174b79700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417019a6c0 con 0x7f41701013e0 2024-09-10T11:34:41.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.872+0000 7f416dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4170101cf0 0x7f4170196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:41.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.872+0000 7f416dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4170101cf0 0x7f4170196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52562/0 (socket says 172.21.15.144:52562) 2024-09-10T11:34:41.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.872+0000 7f416dd9b700 1 -- 172.21.15.144:0/2557001007 learned_addr learned my addr 172.21.15.144:0/2557001007 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:34:41.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.872+0000 7f416e59c700 1 --2- 172.21.15.144:0/2557001007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41701013e0 0x7f4170195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:41.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.872+0000 7f416dd9b700 1 -- 172.21.15.144:0/2557001007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41701013e0 msgr2=0x7f4170195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:41.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.872+0000 7f416dd9b700 1 --2- 172.21.15.144:0/2557001007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41701013e0 0x7f4170195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:41.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.872+0000 7f416dd9b700 1 -- 172.21.15.144:0/2557001007 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41640095f0 con 0x7f4170101cf0 2024-09-10T11:34:41.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.873+0000 7f416dd9b700 1 --2- 172.21.15.144:0/2557001007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4170101cf0 0x7f4170196250 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4164005620 tx=0x7f4164005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:41.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.873+0000 7f415f7fe700 1 -- 172.21.15.144:0/2557001007 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f416401d070 con 0x7f4170101cf0 2024-09-10T11:34:41.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.873+0000 7f415f7fe700 1 -- 172.21.15.144:0/2557001007 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f416400f460 con 0x7f4170101cf0 2024-09-10T11:34:41.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.873+0000 7f4174b79700 1 -- 172.21.15.144:0/2557001007 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f417019a940 con 0x7f4170101cf0 2024-09-10T11:34:41.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.873+0000 7f415f7fe700 1 -- 172.21.15.144:0/2557001007 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4164021640 con 0x7f4170101cf0 2024-09-10T11:34:41.877 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.873+0000 7f4174b79700 1 -- 172.21.15.144:0/2557001007 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417019ae30 con 0x7f4170101cf0 2024-09-10T11:34:41.877 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.874+0000 7f4174b79700 1 -- 172.21.15.144:0/2557001007 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4170107860 con 0x7f4170101cf0 2024-09-10T11:34:41.877 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.874+0000 7f415f7fe700 1 -- 172.21.15.144:0/2557001007 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f416400fa40 con 0x7f4170101cf0 2024-09-10T11:34:41.877 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.875+0000 7f415f7fe700 1 --2- 172.21.15.144:0/2557001007 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f415806e330 0x7f41580707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:41.878 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.875+0000 7f415f7fe700 1 -- 172.21.15.144:0/2557001007 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f416408f500 con 0x7f4170101cf0 2024-09-10T11:34:41.878 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.875+0000 7f416e59c700 1 --2- 172.21.15.144:0/2557001007 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f415806e330 0x7f41580707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:41.878 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.875+0000 7f416e59c700 1 --2- 172.21.15.144:0/2557001007 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f415806e330 0x7f41580707e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4160005fd0 tx=0x7f416000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:41.880 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:41.879+0000 7f415f7fe700 1 -- 172.21.15.144:0/2557001007 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f41640587f0 con 0x7f4170101cf0 2024-09-10T11:34:42.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:41 smithi178 ceph-mon[65940]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:42.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:41 smithi178 ceph-mon[65940]: from='osd.0 [v2:172.21.15.144:6802/3898822309,v1:172.21.15.144:6803/3898822309]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T11:34:42.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:41 smithi178 ceph-mon[65940]: osdmap e53: 6 total, 5 up, 6 in 2024-09-10T11:34:42.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:41 smithi178 ceph-mon[65940]: from='osd.0 [v2:172.21.15.144:6802/3898822309,v1:172.21.15.144:6803/3898822309]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T11:34:42.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:41 smithi178 ceph-mon[65940]: from='osd.0 [v2:172.21.15.144:6802/3898822309,v1:172.21.15.144:6803/3898822309]' entity='osd.0' 2024-09-10T11:34:42.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:41 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:42.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:41 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:42.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:41 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:42.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:41 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:34:42.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:41 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:42.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.125+0000 7f4174b79700 1 -- 172.21.15.144:0/2557001007 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f417004ea50 con 0x7f4170101cf0 2024-09-10T11:34:42.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.125+0000 7f415f7fe700 1 -- 172.21.15.144:0/2557001007 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1960 (secure 0 0 0) 0x7f416405beb0 con 0x7f4170101cf0 2024-09-10T11:34:42.127 INFO:teuthology.orchestra.run.smithi144.stdout:e23 2024-09-10T11:34:42.127 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:33:11:119769+0000 2024-09-10T11:34:42.127 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:34:42.127 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:34:42.127 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:34:42.127 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:34:42.127 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:34:42.128 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:34:42.128 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 23 2024-09-10T11:34:42.128 INFO:teuthology.orchestra.run.smithi144.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T11:34:42.128 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:34:42.128 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:33:11.119766+0000 2024-09-10T11:34:42.128 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:34:42.128 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:34:42.128 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:34:42.128 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:34:42.128 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:34:42.128 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:34:42.129 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:34:42.129 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:34:42.129 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 47 2024-09-10T11:34:42.129 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:34:42.129 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:34:42.129 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:34:42.129 INFO:teuthology.orchestra.run.smithi144.stdout:up {0=14488,1=24267} 2024-09-10T11:34:42.129 INFO:teuthology.orchestra.run.smithi144.stdout:failed 2024-09-10T11:34:42.129 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:34:42.129 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:34:42.129 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:34:42.130 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:34:42.130 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:34:42.130 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:34:42.130 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:34:42.130 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:34:42.130 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 14488 members: 14488,24267 2024-09-10T11:34:42.130 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{0:14488} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:34:42.130 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{0:14498} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/1752150537,v1:172.21.15.144:6827/1752150537] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:34:42.130 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:34:42.130 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:34:42.130 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:34:42.130 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:34:42.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.127+0000 7f4174b79700 1 -- 172.21.15.144:0/2557001007 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f415806e330 msgr2=0x7f41580707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.127+0000 7f4174b79700 1 --2- 172.21.15.144:0/2557001007 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f415806e330 0x7f41580707e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4160005fd0 tx=0x7f416000a560 comp rx=0 tx=0).stop 2024-09-10T11:34:42.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.127+0000 7f4174b79700 1 -- 172.21.15.144:0/2557001007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4170101cf0 msgr2=0x7f4170196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.127+0000 7f4174b79700 1 --2- 172.21.15.144:0/2557001007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4170101cf0 0x7f4170196250 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4164005620 tx=0x7f4164005700 comp rx=0 tx=0).stop 2024-09-10T11:34:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.127+0000 7f4174b79700 1 -- 172.21.15.144:0/2557001007 shutdown_connections 2024-09-10T11:34:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.127+0000 7f4174b79700 1 --2- 172.21.15.144:0/2557001007 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f415806e330 0x7f41580707e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.127+0000 7f4174b79700 1 --2- 172.21.15.144:0/2557001007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41701013e0 0x7f4170195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.127+0000 7f4174b79700 1 --2- 172.21.15.144:0/2557001007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4170101cf0 0x7f4170196250 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.128+0000 7f4174b79700 1 -- 172.21.15.144:0/2557001007 >> 172.21.15.144:0/2557001007 conn(0x7f41700fac20 msgr2=0x7f41701049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.128+0000 7f4174b79700 1 -- 172.21.15.144:0/2557001007 shutdown_connections 2024-09-10T11:34:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.128+0000 7f4174b79700 1 -- 172.21.15.144:0/2557001007 wait complete. 2024-09-10T11:34:42.132 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 23 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.234+0000 7f5e4e94d700 1 -- 172.21.15.144:0/1263697338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e480ffb30 msgr2=0x7f5e4810a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.234+0000 7f5e4e94d700 1 --2- 172.21.15.144:0/1263697338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e480ffb30 0x7f5e4810a5a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009940 tx=0x7f5e30009c50 comp rx=0 tx=0).stop 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.234+0000 7f5e4e94d700 1 -- 172.21.15.144:0/1263697338 shutdown_connections 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.234+0000 7f5e4e94d700 1 --2- 172.21.15.144:0/1263697338 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e480ffb30 0x7f5e4810a5a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.234+0000 7f5e4e94d700 1 --2- 172.21.15.144:0/1263697338 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e480ff220 0x7f5e480ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.234+0000 7f5e4e94d700 1 -- 172.21.15.144:0/1263697338 >> 172.21.15.144:0/1263697338 conn(0x7f5e480747e0 msgr2=0x7f5e48074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.234+0000 7f5e4e94d700 1 -- 172.21.15.144:0/1263697338 shutdown_connections 2024-09-10T11:34:42.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.234+0000 7f5e4e94d700 1 -- 172.21.15.144:0/1263697338 wait complete. 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.234+0000 7f5e4e94d700 1 Processor -- start 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.234+0000 7f5e4e94d700 1 -- start start 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e4e94d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e480ff220 0x7f5e48195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e4e94d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e480ffb30 0x7f5e48196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e4e94d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e481969e0 con 0x7f5e480ffb30 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e4e94d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4819a770 con 0x7f5e480ff220 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e3ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e480ffb30 0x7f5e48196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e3ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e480ffb30 0x7f5e48196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52574/0 (socket says 172.21.15.144:52574) 2024-09-10T11:34:42.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e3ffff700 1 -- 172.21.15.144:0/3132038749 learned_addr learned my addr 172.21.15.144:0/3132038749 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e47fff700 1 --2- 172.21.15.144:0/3132038749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e480ff220 0x7f5e48195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e3ffff700 1 -- 172.21.15.144:0/3132038749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e480ff220 msgr2=0x7f5e48195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e3ffff700 1 --2- 172.21.15.144:0/3132038749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e480ff220 0x7f5e48195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e3ffff700 1 -- 172.21.15.144:0/3132038749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e300095f0 con 0x7f5e480ffb30 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e3ffff700 1 --2- 172.21.15.144:0/3132038749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e480ffb30 0x7f5e48196300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5e30005620 tx=0x7f5e30005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e45ffb700 1 -- 172.21.15.144:0/3132038749 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e3001d070 con 0x7f5e480ffb30 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e45ffb700 1 -- 172.21.15.144:0/3132038749 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5e3000f460 con 0x7f5e480ffb30 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e4e94d700 1 -- 172.21.15.144:0/3132038749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4819a9f0 con 0x7f5e480ffb30 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e45ffb700 1 -- 172.21.15.144:0/3132038749 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e30021640 con 0x7f5e480ffb30 2024-09-10T11:34:42.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.235+0000 7f5e4e94d700 1 -- 172.21.15.144:0/3132038749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4819aee0 con 0x7f5e480ffb30 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.236+0000 7f5e4e94d700 1 -- 172.21.15.144:0/3132038749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e48107ca0 con 0x7f5e480ffb30 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.237+0000 7f5e45ffb700 1 -- 172.21.15.144:0/3132038749 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f5e3000fa40 con 0x7f5e480ffb30 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.237+0000 7f5e45ffb700 1 --2- 172.21.15.144:0/3132038749 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5e3406e330 0x7f5e340707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:42.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.237+0000 7f5e45ffb700 1 -- 172.21.15.144:0/3132038749 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5e3008f610 con 0x7f5e480ffb30 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.237+0000 7f5e47fff700 1 --2- 172.21.15.144:0/3132038749 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5e3406e330 0x7f5e340707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:42.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.238+0000 7f5e47fff700 1 --2- 172.21.15.144:0/3132038749 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5e3406e330 0x7f5e340707e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5e38006fd0 tx=0x7f5e38008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:42.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.242+0000 7f5e45ffb700 1 -- 172.21.15.144:0/3132038749 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5e30058880 con 0x7f5e480ffb30 2024-09-10T11:34:42.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.464+0000 7f5e4e94d700 1 -- 172.21.15.144:0/3132038749 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e4802cd10 con 0x7f5e3406e330 2024-09-10T11:34:42.467 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.465+0000 7f5e45ffb700 1 -- 172.21.15.144:0/3132038749 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f5e4802cd10 con 0x7f5e3406e330 2024-09-10T11:34:42.467 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:34:42.467 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:34:42.467 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:34:42.468 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:34:42.468 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:34:42.468 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:34:42.468 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:34:42.468 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:34:42.468 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:34:42.469 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "7/23 daemons upgraded", 2024-09-10T11:34:42.469 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading osd daemons", 2024-09-10T11:34:42.469 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:34:42.469 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:34:42.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.468+0000 7f5e4e94d700 1 -- 172.21.15.144:0/3132038749 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5e3406e330 msgr2=0x7f5e340707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.468+0000 7f5e4e94d700 1 --2- 172.21.15.144:0/3132038749 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5e3406e330 0x7f5e340707e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5e38006fd0 tx=0x7f5e38008040 comp rx=0 tx=0).stop 2024-09-10T11:34:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.468+0000 7f5e4e94d700 1 -- 172.21.15.144:0/3132038749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e480ffb30 msgr2=0x7f5e48196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.468+0000 7f5e4e94d700 1 --2- 172.21.15.144:0/3132038749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e480ffb30 0x7f5e48196300 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5e30005620 tx=0x7f5e30005700 comp rx=0 tx=0).stop 2024-09-10T11:34:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.468+0000 7f5e4e94d700 1 -- 172.21.15.144:0/3132038749 shutdown_connections 2024-09-10T11:34:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.468+0000 7f5e4e94d700 1 --2- 172.21.15.144:0/3132038749 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5e3406e330 0x7f5e340707e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.468+0000 7f5e4e94d700 1 --2- 172.21.15.144:0/3132038749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5e480ff220 0x7f5e48195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.468+0000 7f5e4e94d700 1 --2- 172.21.15.144:0/3132038749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5e480ffb30 0x7f5e48196300 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.468+0000 7f5e4e94d700 1 -- 172.21.15.144:0/3132038749 >> 172.21.15.144:0/3132038749 conn(0x7f5e480747e0 msgr2=0x7f5e48104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:42.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.468+0000 7f5e4e94d700 1 -- 172.21.15.144:0/3132038749 shutdown_connections 2024-09-10T11:34:42.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.468+0000 7f5e4e94d700 1 -- 172.21.15.144:0/3132038749 wait complete. 2024-09-10T11:34:42.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.586+0000 7f3f84267700 1 -- 172.21.15.144:0/3029095986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f7c0ffc60 msgr2=0x7f3f7c103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.586+0000 7f3f84267700 1 --2- 172.21.15.144:0/3029095986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f7c0ffc60 0x7f3f7c103c30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009910 tx=0x7f3f6c009c20 comp rx=0 tx=0).stop 2024-09-10T11:34:42.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.586+0000 7f3f84267700 1 -- 172.21.15.144:0/3029095986 shutdown_connections 2024-09-10T11:34:42.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.586+0000 7f3f84267700 1 --2- 172.21.15.144:0/3029095986 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f7c0ffc60 0x7f3f7c103c30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.586+0000 7f3f84267700 1 --2- 172.21.15.144:0/3029095986 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c0ff2c0 0x7f3f7c0ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.586+0000 7f3f84267700 1 -- 172.21.15.144:0/3029095986 >> 172.21.15.144:0/3029095986 conn(0x7f3f7c0fab60 msgr2=0x7f3f7c0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:42.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.586+0000 7f3f84267700 1 -- 172.21.15.144:0/3029095986 shutdown_connections 2024-09-10T11:34:42.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.586+0000 7f3f84267700 1 -- 172.21.15.144:0/3029095986 wait complete. 2024-09-10T11:34:42.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.587+0000 7f3f84267700 1 Processor -- start 2024-09-10T11:34:42.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.587+0000 7f3f84267700 1 -- start start 2024-09-10T11:34:42.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.587+0000 7f3f84267700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c0ff2c0 0x7f3f7c197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:42.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.587+0000 7f3f84267700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f7c0ffc60 0x7f3f7c198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:42.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.587+0000 7f3f84267700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c198c00 con 0x7f3f7c0ffc60 2024-09-10T11:34:42.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.587+0000 7f3f84267700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c19c990 con 0x7f3f7c0ff2c0 2024-09-10T11:34:42.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.587+0000 7f3f81802700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f7c0ffc60 0x7f3f7c198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:42.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.587+0000 7f3f81802700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f7c0ffc60 0x7f3f7c198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45648/0 (socket says 172.21.15.144:45648) 2024-09-10T11:34:42.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.587+0000 7f3f81802700 1 -- 172.21.15.144:0/3089779221 learned_addr learned my addr 172.21.15.144:0/3089779221 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:34:42.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.587+0000 7f3f82003700 1 --2- 172.21.15.144:0/3089779221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c0ff2c0 0x7f3f7c197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:42.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.587+0000 7f3f81802700 1 -- 172.21.15.144:0/3089779221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c0ff2c0 msgr2=0x7f3f7c197fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.587+0000 7f3f81802700 1 --2- 172.21.15.144:0/3089779221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c0ff2c0 0x7f3f7c197fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.587+0000 7f3f81802700 1 -- 172.21.15.144:0/3089779221 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c0095f0 con 0x7f3f7c0ffc60 2024-09-10T11:34:42.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.588+0000 7f3f81802700 1 --2- 172.21.15.144:0/3089779221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f7c0ffc60 0x7f3f7c198520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00bd70 tx=0x7f3f6c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:42.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.588+0000 7f3f737fe700 1 -- 172.21.15.144:0/3089779221 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f6c01d070 con 0x7f3f7c0ffc60 2024-09-10T11:34:42.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.588+0000 7f3f737fe700 1 -- 172.21.15.144:0/3089779221 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f6c005dc0 con 0x7f3f7c0ffc60 2024-09-10T11:34:42.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.588+0000 7f3f737fe700 1 -- 172.21.15.144:0/3089779221 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f6c017400 con 0x7f3f7c0ffc60 2024-09-10T11:34:42.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.588+0000 7f3f84267700 1 -- 172.21.15.144:0/3089779221 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7c19cc10 con 0x7f3f7c0ffc60 2024-09-10T11:34:42.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.588+0000 7f3f84267700 1 -- 172.21.15.144:0/3089779221 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7c19d100 con 0x7f3f7c0ffc60 2024-09-10T11:34:42.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.589+0000 7f3f84267700 1 -- 172.21.15.144:0/3089779221 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7c04ea50 con 0x7f3f7c0ffc60 2024-09-10T11:34:42.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.589+0000 7f3f737fe700 1 -- 172.21.15.144:0/3089779221 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 38) v1 ==== 92307+0+0 (secure 0 0 0) 0x7f3f6c00f460 con 0x7f3f7c0ffc60 2024-09-10T11:34:42.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.590+0000 7f3f737fe700 1 --2- 172.21.15.144:0/3089779221 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f6806e330 0x7f3f680707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:34:42.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.590+0000 7f3f82003700 1 --2- 172.21.15.144:0/3089779221 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f6806e330 0x7f3f680707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:34:42.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.590+0000 7f3f737fe700 1 -- 172.21.15.144:0/3089779221 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3f6c08f5e0 con 0x7f3f7c0ffc60 2024-09-10T11:34:42.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.590+0000 7f3f82003700 1 --2- 172.21.15.144:0/3089779221 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f6806e330 0x7f3f680707e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3f78005950 tx=0x7f3f7800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:34:42.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.594+0000 7f3f737fe700 1 -- 172.21.15.144:0/3089779221 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3f6c0588d0 con 0x7f3f7c0ffc60 2024-09-10T11:34:42.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.883+0000 7f3f84267700 1 -- 172.21.15.144:0/3089779221 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f3f7c19d5c0 con 0x7f3f7c0ffc60 2024-09-10T11:34:42.885 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:42.885 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:42.885 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: pgmap v50: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 70/360 objects degraded (19.444%) 2024-09-10T11:34:42.885 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:34:42.885 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:42.885 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:42.885 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:42.886 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-10T11:34:42.886 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-10T11:34:42.886 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-09-10T11:34:42.886 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2141036203' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:42.886 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2557001007' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:34:42.886 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T11:34:42.886 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: osd.0 [v2:172.21.15.144:6802/3898822309,v1:172.21.15.144:6803/3898822309] boot 2024-09-10T11:34:42.886 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: osdmap e54: 6 total, 6 up, 6 in 2024-09-10T11:34:42.886 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:34:42.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.884+0000 7f3f737fe700 1 -- 172.21.15.144:0/3089779221 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1429 (secure 0 0 0) 0x7f3f6c05bf90 con 0x7f3f7c0ffc60 2024-09-10T11:34:42.887 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; Degraded data redundancy: 70/360 objects degraded (19.444%), 19 pgs degraded 2024-09-10T11:34:42.887 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:34:42.887 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:34:42.887 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 70/360 objects degraded (19.444%), 19 pgs degraded 2024-09-10T11:34:42.887 INFO:teuthology.orchestra.run.smithi144.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-09-10T11:34:42.887 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-09-10T11:34:42.887 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-09-10T11:34:42.887 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-09-10T11:34:42.887 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-09-10T11:34:42.887 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-09-10T11:34:42.887 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-09-10T11:34:42.887 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-09-10T11:34:42.887 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-09-10T11:34:42.888 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-09-10T11:34:42.888 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-09-10T11:34:42.888 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-09-10T11:34:42.888 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-09-10T11:34:42.888 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-09-10T11:34:42.888 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-09-10T11:34:42.888 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-09-10T11:34:42.888 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-09-10T11:34:42.888 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-09-10T11:34:42.888 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-09-10T11:34:42.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.886+0000 7f3f84267700 1 -- 172.21.15.144:0/3089779221 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f6806e330 msgr2=0x7f3f680707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.886+0000 7f3f84267700 1 --2- 172.21.15.144:0/3089779221 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f6806e330 0x7f3f680707e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3f78005950 tx=0x7f3f7800b410 comp rx=0 tx=0).stop 2024-09-10T11:34:42.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.886+0000 7f3f84267700 1 -- 172.21.15.144:0/3089779221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f7c0ffc60 msgr2=0x7f3f7c198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:34:42.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.886+0000 7f3f84267700 1 --2- 172.21.15.144:0/3089779221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f7c0ffc60 0x7f3f7c198520 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c00bd70 tx=0x7f3f6c00bda0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.886+0000 7f3f84267700 1 -- 172.21.15.144:0/3089779221 shutdown_connections 2024-09-10T11:34:42.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.886+0000 7f3f84267700 1 --2- 172.21.15.144:0/3089779221 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f6806e330 0x7f3f680707e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.886+0000 7f3f84267700 1 --2- 172.21.15.144:0/3089779221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f7c0ff2c0 0x7f3f7c197fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.886+0000 7f3f84267700 1 --2- 172.21.15.144:0/3089779221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f7c0ffc60 0x7f3f7c198520 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:34:42.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.886+0000 7f3f84267700 1 -- 172.21.15.144:0/3089779221 >> 172.21.15.144:0/3089779221 conn(0x7f3f7c0fab60 msgr2=0x7f3f7c102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:34:42.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.886+0000 7f3f84267700 1 -- 172.21.15.144:0/3089779221 shutdown_connections 2024-09-10T11:34:42.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:34:42.886+0000 7f3f84267700 1 -- 172.21.15.144:0/3089779221 wait complete. 2024-09-10T11:34:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:43.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: pgmap v50: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 70/360 objects degraded (19.444%) 2024-09-10T11:34:43.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:34:43.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:43.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:43.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:43.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-10T11:34:43.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-10T11:34:43.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-09-10T11:34:43.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2141036203' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:34:43.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2557001007' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:34:43.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T11:34:43.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: osd.0 [v2:172.21.15.144:6802/3898822309,v1:172.21.15.144:6803/3898822309] boot 2024-09-10T11:34:43.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: osdmap e54: 6 total, 6 up, 6 in 2024-09-10T11:34:43.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T11:34:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:43 smithi144 ceph-mon[87436]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:43 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3089779221' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:34:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:43 smithi144 ceph-mon[87436]: osdmap e55: 6 total, 6 up, 6 in 2024-09-10T11:34:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:34:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:43 smithi178 ceph-mon[65940]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:34:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:43 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3089779221' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:34:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:43 smithi178 ceph-mon[65940]: osdmap e55: 6 total, 6 up, 6 in 2024-09-10T11:34:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:34:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:44 smithi144 ceph-mon[87436]: pgmap v53: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 70/360 objects degraded (19.444%) 2024-09-10T11:34:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:44 smithi178 ceph-mon[65940]: pgmap v53: 65 pgs: 15 active+undersized, 19 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 70/360 objects degraded (19.444%) 2024-09-10T11:34:45.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:45 smithi144 ceph-mon[87436]: Health check update: Degraded data redundancy: 14/360 objects degraded (3.889%), 3 pgs degraded (PG_DEGRADED) 2024-09-10T11:34:45.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:45 smithi144 ceph-mon[87436]: mgrmap e39: smithi144.bkogfs(active, since 92s), standbys: smithi178.vmvxuf 2024-09-10T11:34:46.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:45 smithi178 ceph-mon[65940]: Health check update: Degraded data redundancy: 14/360 objects degraded (3.889%), 3 pgs degraded (PG_DEGRADED) 2024-09-10T11:34:46.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:45 smithi178 ceph-mon[65940]: mgrmap e39: smithi144.bkogfs(active, since 92s), standbys: smithi178.vmvxuf 2024-09-10T11:34:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:46 smithi178 ceph-mon[65940]: pgmap v54: 65 pgs: 6 active+undersized, 3 active+undersized+degraded, 56 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s; 14/360 objects degraded (3.889%) 2024-09-10T11:34:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:46 smithi144 ceph-mon[87436]: pgmap v54: 65 pgs: 6 active+undersized, 3 active+undersized+degraded, 56 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 682 B/s wr, 3 op/s; 14/360 objects degraded (3.889%) 2024-09-10T11:34:48.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:47 smithi178 ceph-mon[65940]: pgmap v55: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 737 B/s wr, 1 op/s 2024-09-10T11:34:48.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:47 smithi178 ceph-mon[65940]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/360 objects degraded (3.889%), 3 pgs degraded) 2024-09-10T11:34:48.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:47 smithi144 ceph-mon[87436]: pgmap v55: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 737 B/s wr, 1 op/s 2024-09-10T11:34:48.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:47 smithi144 ceph-mon[87436]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 14/360 objects degraded (3.889%), 3 pgs degraded) 2024-09-10T11:34:50.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:50 smithi178 ceph-mon[65940]: pgmap v56: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 639 B/s wr, 3 op/s 2024-09-10T11:34:50.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:50 smithi144 ceph-mon[87436]: pgmap v56: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 639 B/s wr, 3 op/s 2024-09-10T11:34:52.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:52 smithi144 ceph-mon[87436]: pgmap v57: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 563 B/s wr, 3 op/s 2024-09-10T11:34:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:52 smithi178 ceph-mon[65940]: pgmap v57: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.4 KiB/s rd, 563 B/s wr, 3 op/s 2024-09-10T11:34:54.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:54 smithi178 ceph-mon[65940]: pgmap v58: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 507 B/s wr, 3 op/s 2024-09-10T11:34:54.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:54 smithi144 ceph-mon[87436]: pgmap v58: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 507 B/s wr, 3 op/s 2024-09-10T11:34:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:56.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:56 smithi178 ceph-mon[65940]: pgmap v59: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 426 B/s wr, 3 op/s 2024-09-10T11:34:56.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:56 smithi144 ceph-mon[87436]: pgmap v59: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.7 KiB/s rd, 426 B/s wr, 3 op/s 2024-09-10T11:34:57.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:57 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-10T11:34:57.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:57 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-10T11:34:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:58 smithi178 ceph-mon[65940]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-10T11:34:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:58 smithi178 ceph-mon[65940]: Upgrade: osd.1 is safe to restart 2024-09-10T11:34:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:58 smithi178 ceph-mon[65940]: pgmap v60: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-10T11:34:58.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:58.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T11:34:58.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:58.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:58.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:34:58.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:58 smithi144 ceph-mon[87436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-09-10T11:34:58.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:58 smithi144 ceph-mon[87436]: Upgrade: osd.1 is safe to restart 2024-09-10T11:34:58.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:58 smithi144 ceph-mon[87436]: pgmap v60: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-10T11:34:58.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:58.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T11:34:58.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:34:58.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:34:58.885 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:34:59.847 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:59 smithi144 ceph-mon[87436]: Upgrade: Updating osd.1 2024-09-10T11:34:59.848 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:34:59 smithi144 ceph-mon[87436]: Deploying daemon osd.1 on smithi144 2024-09-10T11:35:00.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:59 smithi178 ceph-mon[65940]: Upgrade: Updating osd.1 2024-09-10T11:35:00.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:34:59 smithi178 ceph-mon[65940]: Deploying daemon osd.1 on smithi144 2024-09-10T11:35:00.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:00 smithi178 ceph-mon[65940]: pgmap v61: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:35:00.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:00 smithi144 ceph-mon[87436]: pgmap v61: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:35:01.602 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:01 smithi144 systemd[1]: Stopping Ceph osd.1 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:35:01.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:01 smithi144 ceph-mon[87436]: osd.1 marked itself down and dead 2024-09-10T11:35:01.935 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:01 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1[48489]: 2024-09-10T11:35:01.600+0000 7fa59bce9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:35:01.935 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:01 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1[48489]: 2024-09-10T11:35:01.600+0000 7fa59bce9700 -1 osd.1 55 *** Got signal Terminated *** 2024-09-10T11:35:01.935 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:01 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1[48489]: 2024-09-10T11:35:01.600+0000 7fa59bce9700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:35:02.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:01 smithi178 ceph-mon[65940]: osd.1 marked itself down and dead 2024-09-10T11:35:02.434 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:02 smithi144 podman[103912]: 2024-09-10 11:35:02.192056003 +0000 UTC m=+0.697400807 container died b55fe920ff9d4c41347712d04e4e315ca73186d3b3230548fc21ddd9089fefce (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=HEAD, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0) 2024-09-10T11:35:02.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:02 smithi144 ceph-mon[87436]: pgmap v62: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:35:02.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:02 smithi144 ceph-mon[87436]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T11:35:02.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:02 smithi144 ceph-mon[87436]: osdmap e56: 6 total, 5 up, 6 in 2024-09-10T11:35:02.935 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:02 smithi144 podman[103912]: 2024-09-10 11:35:02.809264778 +0000 UTC m=+1.314609620 container cleanup b55fe920ff9d4c41347712d04e4e315ca73186d3b3230548fc21ddd9089fefce (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, RELEASE=HEAD, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-09-10T11:35:02.935 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:02 smithi144 bash[103912]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1 2024-09-10T11:35:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:02 smithi178 ceph-mon[65940]: pgmap v62: 65 pgs: 65 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:35:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:02 smithi178 ceph-mon[65940]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T11:35:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:02 smithi178 ceph-mon[65940]: osdmap e56: 6 total, 5 up, 6 in 2024-09-10T11:35:03.414 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:03 smithi144 podman[103931]: 2024-09-10 11:35:03.021163073 +0000 UTC m=+0.831174467 container remove b55fe920ff9d4c41347712d04e4e315ca73186d3b3230548fc21ddd9089fefce (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, org.label-schema.build-date=20231212, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-09-10T11:35:03.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:03 smithi144 ceph-mon[87436]: osdmap e57: 6 total, 5 up, 6 in 2024-09-10T11:35:03.684 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:03 smithi144 podman[104018]: 2024-09-10 11:35:03.413625519 +0000 UTC m=+0.108908559 container create 36a257cf177e94f0e22c123fc78165529cf3256a90c24c2013a8fe9ca7767832 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T11:35:03.684 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:03 smithi144 podman[104018]: 2024-09-10 11:35:03.320631898 +0000 UTC m=+0.015914939 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:35:03.684 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:03 smithi144 podman[104018]: 2024-09-10 11:35:03.683053247 +0000 UTC m=+0.378336294 container init 36a257cf177e94f0e22c123fc78165529cf3256a90c24c2013a8fe9ca7767832 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T11:35:03.934 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:03 smithi144 podman[104018]: 2024-09-10 11:35:03.686574193 +0000 UTC m=+0.381857241 container start 36a257cf177e94f0e22c123fc78165529cf3256a90c24c2013a8fe9ca7767832 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731) 2024-09-10T11:35:03.934 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:03 smithi144 podman[104018]: 2024-09-10 11:35:03.774597017 +0000 UTC m=+0.469880068 container attach 36a257cf177e94f0e22c123fc78165529cf3256a90c24c2013a8fe9ca7767832 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T11:35:03.934 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:03 smithi144 podman[104018]: 2024-09-10 11:35:03.882021066 +0000 UTC m=+0.577304114 container died 36a257cf177e94f0e22c123fc78165529cf3256a90c24c2013a8fe9ca7767832 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T11:35:04.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:03 smithi178 ceph-mon[65940]: osdmap e57: 6 total, 5 up, 6 in 2024-09-10T11:35:04.912 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:04 smithi144 ceph-mon[87436]: pgmap v65: 65 pgs: 12 stale+active+clean, 53 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s 2024-09-10T11:35:04.912 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:04 smithi144 podman[104040]: 2024-09-10 11:35:04.461839519 +0000 UTC m=+0.570139676 container remove 36a257cf177e94f0e22c123fc78165529cf3256a90c24c2013a8fe9ca7767832 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T11:35:04.913 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:04 smithi144 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.1.service: Deactivated successfully. 2024-09-10T11:35:04.913 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:04 smithi144 systemd[1]: Stopped Ceph osd.1 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:35:04.913 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:04 smithi144 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.1.service: Consumed 58.976s CPU time. 2024-09-10T11:35:04.913 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:04 smithi144 systemd[1]: Starting Ceph osd.1 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:35:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:04 smithi178 ceph-mon[65940]: pgmap v65: 65 pgs: 12 stale+active+clean, 53 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s 2024-09-10T11:35:05.165 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:04 smithi144 podman[104139]: 2024-09-10 11:35:04.813836573 +0000 UTC m=+0.016138580 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:35:05.166 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:04 smithi144 podman[104139]: 2024-09-10 11:35:04.921046057 +0000 UTC m=+0.123348054 container create f01f7e2d8cb02620bc479d7fd9d13069def7d360c6e90e536c791bc354eab463 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-activate, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-10T11:35:05.166 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:05 smithi144 podman[104139]: 2024-09-10 11:35:05.165236941 +0000 UTC m=+0.367538935 container init f01f7e2d8cb02620bc479d7fd9d13069def7d360c6e90e536c791bc354eab463 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-activate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T11:35:05.434 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:05 smithi144 podman[104139]: 2024-09-10 11:35:05.16863179 +0000 UTC m=+0.370933793 container start f01f7e2d8cb02620bc479d7fd9d13069def7d360c6e90e536c791bc354eab463 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-activate, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T11:35:05.434 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:05 smithi144 podman[104139]: 2024-09-10 11:35:05.265269953 +0000 UTC m=+0.467571965 container attach f01f7e2d8cb02620bc479d7fd9d13069def7d360c6e90e536c791bc354eab463 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-activate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-10T11:35:05.434 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:05 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-activate[104154]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:35:05.434 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:05 smithi144 bash[104139]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:35:05.434 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:05 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-activate[104154]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:35:05.434 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:05 smithi144 bash[104139]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:35:05.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:05 smithi178 ceph-mon[65940]: Health check failed: Degraded data redundancy: 49/360 objects degraded (13.611%), 14 pgs degraded (PG_DEGRADED) 2024-09-10T11:35:06.037 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:05 smithi144 ceph-mon[87436]: Health check failed: Degraded data redundancy: 49/360 objects degraded (13.611%), 14 pgs degraded (PG_DEGRADED) 2024-09-10T11:35:06.322 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-activate[104154]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T11:35:06.322 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 bash[104139]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T11:35:06.323 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-activate[104154]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-10T11:35:06.323 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 bash[104139]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-10T11:35:06.644 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-activate[104154]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-10T11:35:06.644 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 bash[104139]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-10T11:35:06.644 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-activate[104154]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-10T11:35:06.645 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 bash[104139]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-10T11:35:06.645 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-activate[104154]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-10T11:35:06.645 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 bash[104139]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-09-10T11:35:06.645 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-activate[104154]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T11:35:06.645 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 bash[104139]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-10T11:35:06.645 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-activate[104154]: --> ceph-volume raw activate successful for osd ID: 1 2024-09-10T11:35:06.645 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 bash[104139]: --> ceph-volume raw activate successful for osd ID: 1 2024-09-10T11:35:06.645 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 podman[104139]: 2024-09-10 11:35:06.351875131 +0000 UTC m=+1.554177126 container died f01f7e2d8cb02620bc479d7fd9d13069def7d360c6e90e536c791bc354eab463 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-activate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T11:35:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:06 smithi144 ceph-mon[87436]: pgmap v66: 65 pgs: 15 active+undersized, 3 stale+active+clean, 14 active+undersized+degraded, 33 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 49/360 objects degraded (13.611%) 2024-09-10T11:35:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:06 smithi178 ceph-mon[65940]: pgmap v66: 65 pgs: 15 active+undersized, 3 stale+active+clean, 14 active+undersized+degraded, 33 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 49/360 objects degraded (13.611%) 2024-09-10T11:35:07.310 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:06 smithi144 podman[104289]: 2024-09-10 11:35:06.960719532 +0000 UTC m=+0.597714601 container remove f01f7e2d8cb02620bc479d7fd9d13069def7d360c6e90e536c791bc354eab463 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-activate, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T11:35:07.310 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:07 smithi144 podman[104350]: 2024-09-10 11:35:07.068164069 +0000 UTC m=+0.016293206 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:35:07.310 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:07 smithi144 podman[104350]: 2024-09-10 11:35:07.186973161 +0000 UTC m=+0.135102283 container create e5d70992955a9ebbd82c7aaf32aa7ef4d5e1efe4be41f4ff6b8cf1f1cf120555 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T11:35:07.934 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:07 smithi144 podman[104350]: 2024-09-10 11:35:07.448018518 +0000 UTC m=+0.396147643 container init e5d70992955a9ebbd82c7aaf32aa7ef4d5e1efe4be41f4ff6b8cf1f1cf120555 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:35:07.934 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:07 smithi144 podman[104350]: 2024-09-10 11:35:07.451237702 +0000 UTC m=+0.399366826 container start e5d70992955a9ebbd82c7aaf32aa7ef4d5e1efe4be41f4ff6b8cf1f1cf120555 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T11:35:07.934 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:07 smithi144 ceph-osd[104370]: -- 172.21.15.144:0/3698009051 <== mon.0 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561efffae1e0 con 0x561eff15bc00 2024-09-10T11:35:07.934 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:07 smithi144 bash[104350]: e5d70992955a9ebbd82c7aaf32aa7ef4d5e1efe4be41f4ff6b8cf1f1cf120555 2024-09-10T11:35:07.934 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:07 smithi144 systemd[1]: Started Ceph osd.1 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:35:08.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:08 smithi178 ceph-mon[65940]: pgmap v67: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-09-10T11:35:08.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:08 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:08.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:08 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:08.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:08 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:35:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:08 smithi144 ceph-mon[87436]: pgmap v67: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-09-10T11:35:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:08 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:08 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:08 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:35:09.684 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:09 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1[104366]: 2024-09-10T11:35:09.352+0000 7fc05180f640 -1 Falling back to public interface 2024-09-10T11:35:10.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:10 smithi178 ceph-mon[65940]: pgmap v68: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-09-10T11:35:10.871 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:10 smithi144 ceph-mon[87436]: pgmap v68: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-09-10T11:35:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:12 smithi178 ceph-mon[65940]: pgmap v69: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-09-10T11:35:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:12 smithi178 ceph-mon[65940]: Health check update: Degraded data redundancy: 56/360 objects degraded (15.556%), 16 pgs degraded (PG_DEGRADED) 2024-09-10T11:35:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:12 smithi144 ceph-mon[87436]: pgmap v69: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-09-10T11:35:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:12 smithi144 ceph-mon[87436]: Health check update: Degraded data redundancy: 56/360 objects degraded (15.556%), 16 pgs degraded (PG_DEGRADED) 2024-09-10T11:35:13.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.000+0000 7fcbb06d0700 1 -- 172.21.15.144:0/2107485080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcba81003b0 msgr2=0x7fcba8100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.000+0000 7fcbb06d0700 1 --2- 172.21.15.144:0/2107485080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcba81003b0 0x7fcba8100780 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcba4009910 tx=0x7fcba4009c20 comp rx=0 tx=0).stop 2024-09-10T11:35:13.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.000+0000 7fcbb06d0700 1 -- 172.21.15.144:0/2107485080 shutdown_connections 2024-09-10T11:35:13.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.000+0000 7fcbb06d0700 1 --2- 172.21.15.144:0/2107485080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba8100d50 0x7fcba8104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.000+0000 7fcbb06d0700 1 --2- 172.21.15.144:0/2107485080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcba81003b0 0x7fcba8100780 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.000+0000 7fcbb06d0700 1 -- 172.21.15.144:0/2107485080 >> 172.21.15.144:0/2107485080 conn(0x7fcba80fbc30 msgr2=0x7fcba80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:13.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.000+0000 7fcbb06d0700 1 -- 172.21.15.144:0/2107485080 shutdown_connections 2024-09-10T11:35:13.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.000+0000 7fcbb06d0700 1 -- 172.21.15.144:0/2107485080 wait complete. 2024-09-10T11:35:13.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.001+0000 7fcbb06d0700 1 Processor -- start 2024-09-10T11:35:13.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.001+0000 7fcbb06d0700 1 -- start start 2024-09-10T11:35:13.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.001+0000 7fcbb06d0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcba81003b0 0x7fcba8195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.001+0000 7fcbb06d0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba8100d50 0x7fcba8196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.001+0000 7fcbb06d0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba8196930 con 0x7fcba81003b0 2024-09-10T11:35:13.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.001+0000 7fcbb06d0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba819a6c0 con 0x7fcba8100d50 2024-09-10T11:35:13.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.001+0000 7fcbae46c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcba81003b0 0x7fcba8195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.001+0000 7fcbae46c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcba81003b0 0x7fcba8195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:47324/0 (socket says 172.21.15.144:47324) 2024-09-10T11:35:13.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.001+0000 7fcbae46c700 1 -- 172.21.15.144:0/211580465 learned_addr learned my addr 172.21.15.144:0/211580465 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:35:13.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.001+0000 7fcbadc6b700 1 --2- 172.21.15.144:0/211580465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba8100d50 0x7fcba8196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.002+0000 7fcbae46c700 1 -- 172.21.15.144:0/211580465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba8100d50 msgr2=0x7fcba8196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.002+0000 7fcbae46c700 1 --2- 172.21.15.144:0/211580465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba8100d50 0x7fcba8196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.002+0000 7fcbae46c700 1 -- 172.21.15.144:0/211580465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba40095f0 con 0x7fcba81003b0 2024-09-10T11:35:13.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.002+0000 7fcbadc6b700 1 --2- 172.21.15.144:0/211580465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba8100d50 0x7fcba8196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:13.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.002+0000 7fcbae46c700 1 --2- 172.21.15.144:0/211580465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcba81003b0 0x7fcba8195d10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcba40092e0 tx=0x7fcba400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:13.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.002+0000 7fcb9f7fe700 1 -- 172.21.15.144:0/211580465 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcba401d070 con 0x7fcba81003b0 2024-09-10T11:35:13.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.002+0000 7fcb9f7fe700 1 -- 172.21.15.144:0/211580465 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcba4005e00 con 0x7fcba81003b0 2024-09-10T11:35:13.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.002+0000 7fcb9f7fe700 1 -- 172.21.15.144:0/211580465 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcba4017400 con 0x7fcba81003b0 2024-09-10T11:35:13.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.002+0000 7fcbb06d0700 1 -- 172.21.15.144:0/211580465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba819a940 con 0x7fcba81003b0 2024-09-10T11:35:13.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.002+0000 7fcbb06d0700 1 -- 172.21.15.144:0/211580465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba819ae30 con 0x7fcba81003b0 2024-09-10T11:35:13.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.003+0000 7fcbb06d0700 1 -- 172.21.15.144:0/211580465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba81086c0 con 0x7fcba81003b0 2024-09-10T11:35:13.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.004+0000 7fcb9f7fe700 1 -- 172.21.15.144:0/211580465 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fcba400f460 con 0x7fcba81003b0 2024-09-10T11:35:13.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.004+0000 7fcb9f7fe700 1 --2- 172.21.15.144:0/211580465 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb9406e200 0x7fcb940706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.004+0000 7fcb9f7fe700 1 -- 172.21.15.144:0/211580465 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fcba408f520 con 0x7fcba81003b0 2024-09-10T11:35:13.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.004+0000 7fcbadc6b700 1 --2- 172.21.15.144:0/211580465 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb9406e200 0x7fcb940706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.005+0000 7fcbadc6b700 1 --2- 172.21.15.144:0/211580465 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb9406e200 0x7fcb940706b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb98005950 tx=0x7fcb980058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:13.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.012+0000 7fcb9f7fe700 1 -- 172.21.15.144:0/211580465 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcba40589b0 con 0x7fcba81003b0 2024-09-10T11:35:13.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.254+0000 7fcbb06d0700 1 -- 172.21.15.144:0/211580465 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcba802ccf0 con 0x7fcb9406e200 2024-09-10T11:35:13.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.256+0000 7fcb9f7fe700 1 -- 172.21.15.144:0/211580465 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fcba802ccf0 con 0x7fcb9406e200 2024-09-10T11:35:13.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.258+0000 7fcbb06d0700 1 -- 172.21.15.144:0/211580465 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb9406e200 msgr2=0x7fcb940706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.258+0000 7fcbb06d0700 1 --2- 172.21.15.144:0/211580465 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb9406e200 0x7fcb940706b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcb98005950 tx=0x7fcb980058e0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.258+0000 7fcbb06d0700 1 -- 172.21.15.144:0/211580465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcba81003b0 msgr2=0x7fcba8195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.258+0000 7fcbb06d0700 1 --2- 172.21.15.144:0/211580465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcba81003b0 0x7fcba8195d10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcba40092e0 tx=0x7fcba400be20 comp rx=0 tx=0).stop 2024-09-10T11:35:13.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.258+0000 7fcbb06d0700 1 -- 172.21.15.144:0/211580465 shutdown_connections 2024-09-10T11:35:13.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.258+0000 7fcbb06d0700 1 --2- 172.21.15.144:0/211580465 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb9406e200 0x7fcb940706b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.258+0000 7fcbb06d0700 1 --2- 172.21.15.144:0/211580465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcba81003b0 0x7fcba8195d10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.258+0000 7fcbb06d0700 1 --2- 172.21.15.144:0/211580465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcba8100d50 0x7fcba8196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.258+0000 7fcbb06d0700 1 -- 172.21.15.144:0/211580465 >> 172.21.15.144:0/211580465 conn(0x7fcba80fbc30 msgr2=0x7fcba80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:13.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.258+0000 7fcbb06d0700 1 -- 172.21.15.144:0/211580465 shutdown_connections 2024-09-10T11:35:13.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.258+0000 7fcbb06d0700 1 -- 172.21.15.144:0/211580465 wait complete. 2024-09-10T11:35:13.271 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:35:13.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.380+0000 7fd488f0f700 1 -- 172.21.15.144:0/690268076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4840ff390 msgr2=0x7fd4840ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.380+0000 7fd488f0f700 1 --2- 172.21.15.144:0/690268076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4840ff390 0x7fd4840ff760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd474009910 tx=0x7fd474009c20 comp rx=0 tx=0).stop 2024-09-10T11:35:13.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.380+0000 7fd488f0f700 1 -- 172.21.15.144:0/690268076 shutdown_connections 2024-09-10T11:35:13.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.380+0000 7fd488f0f700 1 --2- 172.21.15.144:0/690268076 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4840ffd30 0x7fd484103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.380+0000 7fd488f0f700 1 --2- 172.21.15.144:0/690268076 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4840ff390 0x7fd4840ff760 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.380+0000 7fd488f0f700 1 -- 172.21.15.144:0/690268076 >> 172.21.15.144:0/690268076 conn(0x7fd484074f50 msgr2=0x7fd484075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:13.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.380+0000 7fd488f0f700 1 -- 172.21.15.144:0/690268076 shutdown_connections 2024-09-10T11:35:13.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.380+0000 7fd488f0f700 1 -- 172.21.15.144:0/690268076 wait complete. 2024-09-10T11:35:13.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.381+0000 7fd488f0f700 1 Processor -- start 2024-09-10T11:35:13.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.381+0000 7fd488f0f700 1 -- start start 2024-09-10T11:35:13.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.381+0000 7fd488f0f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4840ff390 0x7fd484196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.382+0000 7fd488f0f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4840ffd30 0x7fd484197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.382+0000 7fd488f0f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd484197860 con 0x7fd4840ff390 2024-09-10T11:35:13.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.382+0000 7fd488f0f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4841979d0 con 0x7fd4840ffd30 2024-09-10T11:35:13.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.382+0000 7fd48259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4840ff390 0x7fd484196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.382+0000 7fd48259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4840ff390 0x7fd484196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:47328/0 (socket says 172.21.15.144:47328) 2024-09-10T11:35:13.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.382+0000 7fd48259c700 1 -- 172.21.15.144:0/3029825541 learned_addr learned my addr 172.21.15.144:0/3029825541 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:35:13.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.382+0000 7fd481d9b700 1 --2- 172.21.15.144:0/3029825541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4840ffd30 0x7fd484197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.382+0000 7fd48259c700 1 -- 172.21.15.144:0/3029825541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4840ffd30 msgr2=0x7fd484197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.382+0000 7fd48259c700 1 --2- 172.21.15.144:0/3029825541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4840ffd30 0x7fd484197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.382+0000 7fd48259c700 1 -- 172.21.15.144:0/3029825541 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4740095f0 con 0x7fd4840ff390 2024-09-10T11:35:13.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.382+0000 7fd48259c700 1 --2- 172.21.15.144:0/3029825541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4840ff390 0x7fd484196c40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd474005f50 tx=0x7fd47400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:13.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.383+0000 7fd4737fe700 1 -- 172.21.15.144:0/3029825541 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd47401d070 con 0x7fd4840ff390 2024-09-10T11:35:13.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.383+0000 7fd4737fe700 1 -- 172.21.15.144:0/3029825541 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd47400b610 con 0x7fd4840ff390 2024-09-10T11:35:13.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.383+0000 7fd488f0f700 1 -- 172.21.15.144:0/3029825541 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd48419fcb0 con 0x7fd4840ff390 2024-09-10T11:35:13.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.383+0000 7fd4737fe700 1 -- 172.21.15.144:0/3029825541 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd474022470 con 0x7fd4840ff390 2024-09-10T11:35:13.386 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.383+0000 7fd488f0f700 1 -- 172.21.15.144:0/3029825541 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4841a0120 con 0x7fd4840ff390 2024-09-10T11:35:13.386 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.384+0000 7fd488f0f700 1 -- 172.21.15.144:0/3029825541 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd484107650 con 0x7fd4840ff390 2024-09-10T11:35:13.386 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.385+0000 7fd4737fe700 1 -- 172.21.15.144:0/3029825541 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd474003780 con 0x7fd4840ff390 2024-09-10T11:35:13.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.386+0000 7fd4737fe700 1 --2- 172.21.15.144:0/3029825541 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd46c06e200 0x7fd46c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.386+0000 7fd4737fe700 1 -- 172.21.15.144:0/3029825541 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd47408f4f0 con 0x7fd4840ff390 2024-09-10T11:35:13.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.386+0000 7fd481d9b700 1 --2- 172.21.15.144:0/3029825541 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd46c06e200 0x7fd46c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.386+0000 7fd481d9b700 1 --2- 172.21.15.144:0/3029825541 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd46c06e200 0x7fd46c0706b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd484068c80 tx=0x7fd478009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:13.392 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.390+0000 7fd4737fe700 1 -- 172.21.15.144:0/3029825541 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd474058b60 con 0x7fd4840ff390 2024-09-10T11:35:13.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.625+0000 7fd488f0f700 1 -- 172.21.15.144:0/3029825541 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd48402cca0 con 0x7fd46c06e200 2024-09-10T11:35:13.627 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.626+0000 7fd4737fe700 1 -- 172.21.15.144:0/3029825541 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fd48402cca0 con 0x7fd46c06e200 2024-09-10T11:35:13.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.629+0000 7fd488f0f700 1 -- 172.21.15.144:0/3029825541 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd46c06e200 msgr2=0x7fd46c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.629+0000 7fd488f0f700 1 --2- 172.21.15.144:0/3029825541 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd46c06e200 0x7fd46c0706b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fd484068c80 tx=0x7fd478009260 comp rx=0 tx=0).stop 2024-09-10T11:35:13.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.629+0000 7fd488f0f700 1 -- 172.21.15.144:0/3029825541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4840ff390 msgr2=0x7fd484196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.629+0000 7fd488f0f700 1 --2- 172.21.15.144:0/3029825541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4840ff390 0x7fd484196c40 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd474005f50 tx=0x7fd47400f690 comp rx=0 tx=0).stop 2024-09-10T11:35:13.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.629+0000 7fd488f0f700 1 -- 172.21.15.144:0/3029825541 shutdown_connections 2024-09-10T11:35:13.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.629+0000 7fd488f0f700 1 --2- 172.21.15.144:0/3029825541 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd46c06e200 0x7fd46c0706b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.629+0000 7fd488f0f700 1 --2- 172.21.15.144:0/3029825541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd4840ff390 0x7fd484196c40 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.629+0000 7fd488f0f700 1 --2- 172.21.15.144:0/3029825541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd4840ffd30 0x7fd484197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.629+0000 7fd488f0f700 1 -- 172.21.15.144:0/3029825541 >> 172.21.15.144:0/3029825541 conn(0x7fd484074f50 msgr2=0x7fd4840fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:13.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.629+0000 7fd488f0f700 1 -- 172.21.15.144:0/3029825541 shutdown_connections 2024-09-10T11:35:13.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.629+0000 7fd488f0f700 1 -- 172.21.15.144:0/3029825541 wait complete. 2024-09-10T11:35:13.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.745+0000 7f3c2d86c700 1 -- 172.21.15.144:0/1759554920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c280ffc60 msgr2=0x7f3c28103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.745+0000 7f3c2d86c700 1 --2- 172.21.15.144:0/1759554920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c280ffc60 0x7f3c28103c30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c009910 tx=0x7f3c1c009c20 comp rx=0 tx=0).stop 2024-09-10T11:35:13.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.745+0000 7f3c2d86c700 1 -- 172.21.15.144:0/1759554920 shutdown_connections 2024-09-10T11:35:13.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.745+0000 7f3c2d86c700 1 --2- 172.21.15.144:0/1759554920 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c280ffc60 0x7f3c28103c30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.745+0000 7f3c2d86c700 1 --2- 172.21.15.144:0/1759554920 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c280ff2c0 0x7f3c280ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.745+0000 7f3c2d86c700 1 -- 172.21.15.144:0/1759554920 >> 172.21.15.144:0/1759554920 conn(0x7f3c280fab60 msgr2=0x7f3c280fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:13.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.745+0000 7f3c2d86c700 1 -- 172.21.15.144:0/1759554920 shutdown_connections 2024-09-10T11:35:13.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.745+0000 7f3c2d86c700 1 -- 172.21.15.144:0/1759554920 wait complete. 2024-09-10T11:35:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.746+0000 7f3c2d86c700 1 Processor -- start 2024-09-10T11:35:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.746+0000 7f3c2d86c700 1 -- start start 2024-09-10T11:35:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.746+0000 7f3c2d86c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c280ff2c0 0x7f3c28194cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.746+0000 7f3c2d86c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c280ffc60 0x7f3c281951f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.746+0000 7f3c2d86c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c28195880 con 0x7f3c280ffc60 2024-09-10T11:35:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.746+0000 7f3c2d86c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c281985b0 con 0x7f3c280ff2c0 2024-09-10T11:35:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.746+0000 7f3c267fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c280ffc60 0x7f3c281951f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.746+0000 7f3c267fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c280ffc60 0x7f3c281951f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:47360/0 (socket says 172.21.15.144:47360) 2024-09-10T11:35:13.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.746+0000 7f3c267fc700 1 -- 172.21.15.144:0/1434479146 learned_addr learned my addr 172.21.15.144:0/1434479146 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:35:13.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.746+0000 7f3c26ffd700 1 --2- 172.21.15.144:0/1434479146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c280ff2c0 0x7f3c28194cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.747+0000 7f3c267fc700 1 -- 172.21.15.144:0/1434479146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c280ff2c0 msgr2=0x7f3c28194cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.747+0000 7f3c267fc700 1 --2- 172.21.15.144:0/1434479146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c280ff2c0 0x7f3c28194cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.747+0000 7f3c267fc700 1 -- 172.21.15.144:0/1434479146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1c0095f0 con 0x7f3c280ffc60 2024-09-10T11:35:13.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.747+0000 7f3c267fc700 1 --2- 172.21.15.144:0/1434479146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c280ffc60 0x7f3c281951f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00bd70 tx=0x7f3c1c00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:13.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.747+0000 7f3c2c86a700 1 -- 172.21.15.144:0/1434479146 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c1c01f070 con 0x7f3c280ffc60 2024-09-10T11:35:13.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.747+0000 7f3c2c86a700 1 -- 172.21.15.144:0/1434479146 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3c1c005dc0 con 0x7f3c280ffc60 2024-09-10T11:35:13.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.747+0000 7f3c2d86c700 1 -- 172.21.15.144:0/1434479146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c28198830 con 0x7f3c280ffc60 2024-09-10T11:35:13.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.747+0000 7f3c2c86a700 1 -- 172.21.15.144:0/1434479146 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c1c00f690 con 0x7f3c280ffc60 2024-09-10T11:35:13.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.747+0000 7f3c2d86c700 1 -- 172.21.15.144:0/1434479146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c28198d20 con 0x7f3c280ffc60 2024-09-10T11:35:13.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.748+0000 7f3c2d86c700 1 -- 172.21.15.144:0/1434479146 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2804ea50 con 0x7f3c280ffc60 2024-09-10T11:35:13.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.749+0000 7f3c2c86a700 1 -- 172.21.15.144:0/1434479146 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3c1c019960 con 0x7f3c280ffc60 2024-09-10T11:35:13.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.750+0000 7f3c2c86a700 1 --2- 172.21.15.144:0/1434479146 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3c1006e200 0x7f3c100706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:13.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.750+0000 7f3c26ffd700 1 --2- 172.21.15.144:0/1434479146 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3c1006e200 0x7f3c100706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:13.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.751+0000 7f3c2c86a700 1 -- 172.21.15.144:0/1434479146 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3c1c0917e0 con 0x7f3c280ffc60 2024-09-10T11:35:13.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.751+0000 7f3c26ffd700 1 --2- 172.21.15.144:0/1434479146 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3c1006e200 0x7f3c100706b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3c18005950 tx=0x7f3c1800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:13.756 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.755+0000 7f3c2c86a700 1 -- 172.21.15.144:0/1434479146 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3c1c05ac70 con 0x7f3c280ffc60 2024-09-10T11:35:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:13 smithi144 ceph-mon[87436]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:13 smithi144 ceph-mon[87436]: pgmap v70: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-09-10T11:35:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:35:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:13 smithi144 ceph-mon[87436]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:13.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.976+0000 7f3c2d86c700 1 -- 172.21.15.144:0/1434479146 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c281990b0 con 0x7f3c1006e200 2024-09-10T11:35:13.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.986+0000 7f3c2c86a700 1 -- 172.21.15.144:0/1434479146 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f3c281990b0 con 0x7f3c1006e200 2024-09-10T11:35:13.987 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:35:13.987 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (3m) 1s ago 13m 25.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:35:13.987 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (14m) 1s ago 14m 8418k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:35:13.988 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (13m) 59s ago 13m 8090k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:35:13.988 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (67s) 1s ago 14m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:35:13.988 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (62s) 59s ago 13m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:35:13.988 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (3m) 1s ago 13m 79.7M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:35:13.988 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (8m) 1s ago 8m 60.0M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:35:13.988 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (8m) 1s ago 8m 126M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:35:13.988 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (8m) 59s ago 8m 62.2M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:35:13.988 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (8m) 59s ago 8m 127M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:35:13.988 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (5m) 1s ago 15m 498M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:35:13.988 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (4m) 59s ago 13m 439M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:35:13.988 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (2m) 1s ago 15m 48.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:35:13.988 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (105s) 59s ago 12m 35.5M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:35:13.989 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (4m) 1s ago 14m 15.9M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:35:13.989 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (4m) 59s ago 13m 15.8M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:35:13.989 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (42s) 1s ago 12m 87.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:35:13.989 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (6s) 1s ago 11m 14.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:35:13.989 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (11m) 1s ago 11m 170M 1216M 18.2.0 dc2bc1663786 3ed5618eac33 2024-09-10T11:35:13.989 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (10m) 59s ago 10m 255M 2239M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:35:13.989 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (10m) 59s ago 10m 219M 2239M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:35:13.989 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (10m) 59s ago 10m 198M 2239M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:35:13.989 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (3m) 1s ago 13m 77.2M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:35:13.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.988+0000 7f3c2d86c700 1 -- 172.21.15.144:0/1434479146 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3c1006e200 msgr2=0x7f3c100706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.988+0000 7f3c2d86c700 1 --2- 172.21.15.144:0/1434479146 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3c1006e200 0x7f3c100706b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f3c18005950 tx=0x7f3c1800b410 comp rx=0 tx=0).stop 2024-09-10T11:35:13.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.988+0000 7f3c2d86c700 1 -- 172.21.15.144:0/1434479146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c280ffc60 msgr2=0x7f3c281951f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:13.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.988+0000 7f3c2d86c700 1 --2- 172.21.15.144:0/1434479146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c280ffc60 0x7f3c281951f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00bd70 tx=0x7f3c1c00bda0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.988+0000 7f3c2d86c700 1 -- 172.21.15.144:0/1434479146 shutdown_connections 2024-09-10T11:35:13.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.988+0000 7f3c2d86c700 1 --2- 172.21.15.144:0/1434479146 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3c1006e200 0x7f3c100706b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.988+0000 7f3c2d86c700 1 --2- 172.21.15.144:0/1434479146 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3c280ff2c0 0x7f3c28194cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.988+0000 7f3c2d86c700 1 --2- 172.21.15.144:0/1434479146 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3c280ffc60 0x7f3c281951f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:13.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.988+0000 7f3c2d86c700 1 -- 172.21.15.144:0/1434479146 >> 172.21.15.144:0/1434479146 conn(0x7f3c280fab60 msgr2=0x7f3c280fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:13.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.988+0000 7f3c2d86c700 1 -- 172.21.15.144:0/1434479146 shutdown_connections 2024-09-10T11:35:13.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:13.988+0000 7f3c2d86c700 1 -- 172.21.15.144:0/1434479146 wait complete. 2024-09-10T11:35:14.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.094+0000 7fa0c552b700 1 -- 172.21.15.144:0/1992155186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c0103450 msgr2=0x7fa0c01038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.094+0000 7fa0c552b700 1 --2- 172.21.15.144:0/1992155186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c0103450 0x7fa0c01038c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009910 tx=0x7fa0b4009c20 comp rx=0 tx=0).stop 2024-09-10T11:35:14.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.094+0000 7fa0c552b700 1 -- 172.21.15.144:0/1992155186 shutdown_connections 2024-09-10T11:35:14.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.094+0000 7fa0c552b700 1 --2- 172.21.15.144:0/1992155186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c0103450 0x7fa0c01038c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.094+0000 7fa0c552b700 1 --2- 172.21.15.144:0/1992155186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0c0104790 0x7fa0c0104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.094+0000 7fa0c552b700 1 -- 172.21.15.144:0/1992155186 >> 172.21.15.144:0/1992155186 conn(0x7fa0c0074850 msgr2=0x7fa0c0074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:14.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.094+0000 7fa0c552b700 1 -- 172.21.15.144:0/1992155186 shutdown_connections 2024-09-10T11:35:14.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.094+0000 7fa0c552b700 1 -- 172.21.15.144:0/1992155186 wait complete. 2024-09-10T11:35:14.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.094+0000 7fa0c552b700 1 Processor -- start 2024-09-10T11:35:14.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.094+0000 7fa0c552b700 1 -- start start 2024-09-10T11:35:14.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0c552b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c0103450 0x7fa0c0195310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:14.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0c552b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0c0104790 0x7fa0c0195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:14.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0c552b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c0195f30 con 0x7fa0c0103450 2024-09-10T11:35:14.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0c552b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c01995f0 con 0x7fa0c0104790 2024-09-10T11:35:14.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0beffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c0103450 0x7fa0c0195310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:14.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0beffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c0103450 0x7fa0c0195310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:47370/0 (socket says 172.21.15.144:47370) 2024-09-10T11:35:14.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0beffd700 1 -- 172.21.15.144:0/4108609483 learned_addr learned my addr 172.21.15.144:0/4108609483 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:35:14.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0be7fc700 1 --2- 172.21.15.144:0/4108609483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0c0104790 0x7fa0c0195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:14.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0beffd700 1 -- 172.21.15.144:0/4108609483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0c0104790 msgr2=0x7fa0c0195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0beffd700 1 --2- 172.21.15.144:0/4108609483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0c0104790 0x7fa0c0195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0beffd700 1 -- 172.21.15.144:0/4108609483 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b40095f0 con 0x7fa0c0103450 2024-09-10T11:35:14.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0beffd700 1 --2- 172.21.15.144:0/4108609483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c0103450 0x7fa0c0195310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c8a0 tx=0x7fa0b000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:14.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0a7fff700 1 -- 172.21.15.144:0/4108609483 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0b000cea0 con 0x7fa0c0103450 2024-09-10T11:35:14.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0a7fff700 1 -- 172.21.15.144:0/4108609483 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0b0004830 con 0x7fa0c0103450 2024-09-10T11:35:14.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0a7fff700 1 -- 172.21.15.144:0/4108609483 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0b00055b0 con 0x7fa0c0103450 2024-09-10T11:35:14.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.095+0000 7fa0c552b700 1 -- 172.21.15.144:0/4108609483 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c01998d0 con 0x7fa0c0103450 2024-09-10T11:35:14.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.096+0000 7fa0c552b700 1 -- 172.21.15.144:0/4108609483 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c0199e20 con 0x7fa0c0103450 2024-09-10T11:35:14.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.096+0000 7fa0c552b700 1 -- 172.21.15.144:0/4108609483 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c00fda10 con 0x7fa0c0103450 2024-09-10T11:35:14.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.097+0000 7fa0a7fff700 1 -- 172.21.15.144:0/4108609483 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa0b00049a0 con 0x7fa0c0103450 2024-09-10T11:35:14.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.097+0000 7fa0a7fff700 1 --2- 172.21.15.144:0/4108609483 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa0a806e200 0x7fa0a80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:14.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.097+0000 7fa0a7fff700 1 -- 172.21.15.144:0/4108609483 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa0b008dc90 con 0x7fa0c0103450 2024-09-10T11:35:14.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.098+0000 7fa0be7fc700 1 --2- 172.21.15.144:0/4108609483 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa0a806e200 0x7fa0a80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:14.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.098+0000 7fa0be7fc700 1 --2- 172.21.15.144:0/4108609483 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa0a806e200 0x7fa0a80706b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006010 tx=0x7fa0b400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:14.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.102+0000 7fa0a7fff700 1 -- 172.21.15.144:0/4108609483 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa0b0057120 con 0x7fa0c0103450 2024-09-10T11:35:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:13 smithi178 ceph-mon[65940]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:13 smithi178 ceph-mon[65940]: pgmap v70: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-09-10T11:35:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:14.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:35:14.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:13 smithi178 ceph-mon[65940]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:14.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.412+0000 7fa0c552b700 1 -- 172.21.15.144:0/4108609483 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa0c019a0a0 con 0x7fa0c0103450 2024-09-10T11:35:14.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.413+0000 7fa0a7fff700 1 -- 172.21.15.144:0/4108609483 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7fa0b001d330 con 0x7fa0c0103450 2024-09-10T11:35:14.415 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:35:14.415 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:35:14.416 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:35:14.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:35:14.416 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:35:14.416 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:35:14.416 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:35:14.417 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:35:14.417 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4, 2024-09-10T11:35:14.417 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 1 2024-09-10T11:35:14.417 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:35:14.417 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:35:14.417 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:35:14.418 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:35:14.418 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:35:14.418 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 8, 2024-09-10T11:35:14.418 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 5 2024-09-10T11:35:14.418 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:35:14.418 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:35:14.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.415+0000 7fa0c552b700 1 -- 172.21.15.144:0/4108609483 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa0a806e200 msgr2=0x7fa0a80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.415+0000 7fa0c552b700 1 --2- 172.21.15.144:0/4108609483 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa0a806e200 0x7fa0a80706b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4006010 tx=0x7fa0b400b540 comp rx=0 tx=0).stop 2024-09-10T11:35:14.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.415+0000 7fa0c552b700 1 -- 172.21.15.144:0/4108609483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c0103450 msgr2=0x7fa0c0195310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.415+0000 7fa0c552b700 1 --2- 172.21.15.144:0/4108609483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c0103450 0x7fa0c0195310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c8a0 tx=0x7fa0b000cc60 comp rx=0 tx=0).stop 2024-09-10T11:35:14.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.415+0000 7fa0c552b700 1 -- 172.21.15.144:0/4108609483 shutdown_connections 2024-09-10T11:35:14.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.415+0000 7fa0c552b700 1 --2- 172.21.15.144:0/4108609483 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa0a806e200 0x7fa0a80706b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.415+0000 7fa0c552b700 1 --2- 172.21.15.144:0/4108609483 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0c0103450 0x7fa0c0195310 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.415+0000 7fa0c552b700 1 --2- 172.21.15.144:0/4108609483 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0c0104790 0x7fa0c0195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.415+0000 7fa0c552b700 1 -- 172.21.15.144:0/4108609483 >> 172.21.15.144:0/4108609483 conn(0x7fa0c0074850 msgr2=0x7fa0c0102360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:14.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.416+0000 7fa0c552b700 1 -- 172.21.15.144:0/4108609483 shutdown_connections 2024-09-10T11:35:14.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.416+0000 7fa0c552b700 1 -- 172.21.15.144:0/4108609483 wait complete. 2024-09-10T11:35:14.528 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.524+0000 7f3d98ba9700 1 -- 172.21.15.144:0/2773516406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d94100d70 msgr2=0x7f3d94104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.528 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.524+0000 7f3d98ba9700 1 --2- 172.21.15.144:0/2773516406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d94100d70 0x7f3d94104d40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3d84009910 tx=0x7f3d84009c20 comp rx=0 tx=0).stop 2024-09-10T11:35:14.528 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.524+0000 7f3d98ba9700 1 -- 172.21.15.144:0/2773516406 shutdown_connections 2024-09-10T11:35:14.528 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.524+0000 7f3d98ba9700 1 --2- 172.21.15.144:0/2773516406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d94100d70 0x7f3d94104d40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.528 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.524+0000 7f3d98ba9700 1 --2- 172.21.15.144:0/2773516406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d941003d0 0x7f3d941007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.528 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.524+0000 7f3d98ba9700 1 -- 172.21.15.144:0/2773516406 >> 172.21.15.144:0/2773516406 conn(0x7f3d940fbc50 msgr2=0x7f3d940fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:14.528 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.524+0000 7f3d98ba9700 1 -- 172.21.15.144:0/2773516406 shutdown_connections 2024-09-10T11:35:14.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.524+0000 7f3d98ba9700 1 -- 172.21.15.144:0/2773516406 wait complete. 2024-09-10T11:35:14.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.525+0000 7f3d98ba9700 1 Processor -- start 2024-09-10T11:35:14.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.525+0000 7f3d98ba9700 1 -- start start 2024-09-10T11:35:14.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.525+0000 7f3d98ba9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d941003d0 0x7f3d94195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:14.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.525+0000 7f3d98ba9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d94100d70 0x7f3d94196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:14.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.525+0000 7f3d98ba9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d94196a20 con 0x7f3d941003d0 2024-09-10T11:35:14.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.525+0000 7f3d98ba9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d9419a7b0 con 0x7f3d94100d70 2024-09-10T11:35:14.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.525+0000 7f3d9259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d941003d0 0x7f3d94195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:14.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.525+0000 7f3d9259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d941003d0 0x7f3d94195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:47382/0 (socket says 172.21.15.144:47382) 2024-09-10T11:35:14.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.525+0000 7f3d9259c700 1 -- 172.21.15.144:0/1463528803 learned_addr learned my addr 172.21.15.144:0/1463528803 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:35:14.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.525+0000 7f3d91d9b700 1 --2- 172.21.15.144:0/1463528803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d94100d70 0x7f3d94196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:14.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.525+0000 7f3d9259c700 1 -- 172.21.15.144:0/1463528803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d94100d70 msgr2=0x7f3d94196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.525+0000 7f3d9259c700 1 --2- 172.21.15.144:0/1463528803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d94100d70 0x7f3d94196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.525+0000 7f3d9259c700 1 -- 172.21.15.144:0/1463528803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d840095f0 con 0x7f3d941003d0 2024-09-10T11:35:14.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.525+0000 7f3d9259c700 1 --2- 172.21.15.144:0/1463528803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d941003d0 0x7f3d94195e00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800e920 tx=0x7f3d8800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:14.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.526+0000 7f3d837fe700 1 -- 172.21.15.144:0/1463528803 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d8800c9d0 con 0x7f3d941003d0 2024-09-10T11:35:14.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.526+0000 7f3d98ba9700 1 -- 172.21.15.144:0/1463528803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9419aa90 con 0x7f3d941003d0 2024-09-10T11:35:14.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.526+0000 7f3d837fe700 1 -- 172.21.15.144:0/1463528803 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d88010430 con 0x7f3d941003d0 2024-09-10T11:35:14.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.526+0000 7f3d837fe700 1 -- 172.21.15.144:0/1463528803 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d88018630 con 0x7f3d941003d0 2024-09-10T11:35:14.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.526+0000 7f3d98ba9700 1 -- 172.21.15.144:0/1463528803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9419afb0 con 0x7f3d941003d0 2024-09-10T11:35:14.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.527+0000 7f3d98ba9700 1 -- 172.21.15.144:0/1463528803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d94108740 con 0x7f3d941003d0 2024-09-10T11:35:14.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.527+0000 7f3d837fe700 1 -- 172.21.15.144:0/1463528803 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3d880105a0 con 0x7f3d941003d0 2024-09-10T11:35:14.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.528+0000 7f3d837fe700 1 --2- 172.21.15.144:0/1463528803 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d7c06e200 0x7f3d7c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:14.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.528+0000 7f3d837fe700 1 -- 172.21.15.144:0/1463528803 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3d88014070 con 0x7f3d941003d0 2024-09-10T11:35:14.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.528+0000 7f3d91d9b700 1 --2- 172.21.15.144:0/1463528803 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d7c06e200 0x7f3d7c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:14.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.528+0000 7f3d91d9b700 1 --2- 172.21.15.144:0/1463528803 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d7c06e200 0x7f3d7c0706b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3d840059d0 tx=0x7f3d84005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:14.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.532+0000 7f3d837fe700 1 -- 172.21.15.144:0/1463528803 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3d880541b0 con 0x7f3d941003d0 2024-09-10T11:35:14.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.786+0000 7f3d98ba9700 1 -- 172.21.15.144:0/1463528803 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3d9419b220 con 0x7f3d941003d0 2024-09-10T11:35:14.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.786+0000 7f3d837fe700 1 -- 172.21.15.144:0/1463528803 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1960 (secure 0 0 0) 0x7f3d88057730 con 0x7f3d941003d0 2024-09-10T11:35:14.788 INFO:teuthology.orchestra.run.smithi144.stdout:e23 2024-09-10T11:35:14.788 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:33:11:119769+0000 2024-09-10T11:35:14.788 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:35:14.788 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:35:14.788 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:35:14.788 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:35:14.788 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:35:14.788 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:35:14.788 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 23 2024-09-10T11:35:14.788 INFO:teuthology.orchestra.run.smithi144.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T11:35:14.788 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:35:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:33:11.119766+0000 2024-09-10T11:35:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:35:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:35:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:35:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:35:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:35:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:35:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:35:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:35:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 47 2024-09-10T11:35:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:35:14.789 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:35:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:35:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:up {0=14488,1=24267} 2024-09-10T11:35:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:failed 2024-09-10T11:35:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:35:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:35:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:35:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:35:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:35:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:35:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:35:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:35:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 14488 members: 14488,24267 2024-09-10T11:35:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{0:14488} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:35:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{0:14498} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/1752150537,v1:172.21.15.144:6827/1752150537] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:35:14.790 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:35:14.791 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:35:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:35:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:35:14.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.789+0000 7f3d98ba9700 1 -- 172.21.15.144:0/1463528803 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d7c06e200 msgr2=0x7f3d7c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.789+0000 7f3d98ba9700 1 --2- 172.21.15.144:0/1463528803 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d7c06e200 0x7f3d7c0706b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3d840059d0 tx=0x7f3d84005940 comp rx=0 tx=0).stop 2024-09-10T11:35:14.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.789+0000 7f3d98ba9700 1 -- 172.21.15.144:0/1463528803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d941003d0 msgr2=0x7f3d94195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.789+0000 7f3d98ba9700 1 --2- 172.21.15.144:0/1463528803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d941003d0 0x7f3d94195e00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3d8800e920 tx=0x7f3d8800ec30 comp rx=0 tx=0).stop 2024-09-10T11:35:14.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.789+0000 7f3d98ba9700 1 -- 172.21.15.144:0/1463528803 shutdown_connections 2024-09-10T11:35:14.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.789+0000 7f3d98ba9700 1 --2- 172.21.15.144:0/1463528803 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d7c06e200 0x7f3d7c0706b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.789+0000 7f3d98ba9700 1 --2- 172.21.15.144:0/1463528803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d941003d0 0x7f3d94195e00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.789+0000 7f3d98ba9700 1 --2- 172.21.15.144:0/1463528803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d94100d70 0x7f3d94196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.789+0000 7f3d98ba9700 1 -- 172.21.15.144:0/1463528803 >> 172.21.15.144:0/1463528803 conn(0x7f3d940fbc50 msgr2=0x7f3d940fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:14.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.789+0000 7f3d98ba9700 1 -- 172.21.15.144:0/1463528803 shutdown_connections 2024-09-10T11:35:14.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.789+0000 7f3d98ba9700 1 -- 172.21.15.144:0/1463528803 wait complete. 2024-09-10T11:35:14.792 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 23 2024-09-10T11:35:14.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.900+0000 7fbc6fd38700 1 -- 172.21.15.144:0/2062581917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc680ff390 msgr2=0x7fbc680ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.900+0000 7fbc6fd38700 1 --2- 172.21.15.144:0/2062581917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc680ff390 0x7fbc680ff760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbc5800b0a0 tx=0x7fbc5800b3b0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.900+0000 7fbc6fd38700 1 -- 172.21.15.144:0/2062581917 shutdown_connections 2024-09-10T11:35:14.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.900+0000 7fbc6fd38700 1 --2- 172.21.15.144:0/2062581917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc680ffd30 0x7fbc68103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.900+0000 7fbc6fd38700 1 --2- 172.21.15.144:0/2062581917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc680ff390 0x7fbc680ff760 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.900+0000 7fbc6fd38700 1 -- 172.21.15.144:0/2062581917 >> 172.21.15.144:0/2062581917 conn(0x7fbc68074f50 msgr2=0x7fbc68075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:14.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.900+0000 7fbc6fd38700 1 -- 172.21.15.144:0/2062581917 shutdown_connections 2024-09-10T11:35:14.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.900+0000 7fbc6fd38700 1 -- 172.21.15.144:0/2062581917 wait complete. 2024-09-10T11:35:14.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.900+0000 7fbc6fd38700 1 Processor -- start 2024-09-10T11:35:14.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.901+0000 7fbc6fd38700 1 -- start start 2024-09-10T11:35:14.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.901+0000 7fbc6fd38700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc680ff390 0x7fbc68195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:14.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.901+0000 7fbc6fd38700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc680ffd30 0x7fbc68196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:14.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.901+0000 7fbc6fd38700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc681968b0 con 0x7fbc680ff390 2024-09-10T11:35:14.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.901+0000 7fbc6fd38700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc681969f0 con 0x7fbc680ffd30 2024-09-10T11:35:14.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.901+0000 7fbc6dad4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc680ff390 0x7fbc68195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:14.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.901+0000 7fbc6dad4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc680ff390 0x7fbc68195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:47408/0 (socket says 172.21.15.144:47408) 2024-09-10T11:35:14.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.901+0000 7fbc6dad4700 1 -- 172.21.15.144:0/842631910 learned_addr learned my addr 172.21.15.144:0/842631910 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:35:14.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.901+0000 7fbc6d2d3700 1 --2- 172.21.15.144:0/842631910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc680ffd30 0x7fbc68196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:14.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.901+0000 7fbc6dad4700 1 -- 172.21.15.144:0/842631910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc680ffd30 msgr2=0x7fbc68196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:14.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.901+0000 7fbc6dad4700 1 --2- 172.21.15.144:0/842631910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc680ffd30 0x7fbc68196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:14.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.901+0000 7fbc6dad4700 1 -- 172.21.15.144:0/842631910 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc58009d00 con 0x7fbc680ff390 2024-09-10T11:35:14.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.901+0000 7fbc6d2d3700 1 --2- 172.21.15.144:0/842631910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc680ffd30 0x7fbc68196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:14.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.901+0000 7fbc6dad4700 1 --2- 172.21.15.144:0/842631910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc680ff390 0x7fbc68195d20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc58000f80 tx=0x7fbc5800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:14.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.901+0000 7fbc66ffd700 1 -- 172.21.15.144:0/842631910 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc5800e040 con 0x7fbc680ff390 2024-09-10T11:35:14.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.902+0000 7fbc66ffd700 1 -- 172.21.15.144:0/842631910 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc58003e60 con 0x7fbc680ff390 2024-09-10T11:35:14.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.902+0000 7fbc6fd38700 1 -- 172.21.15.144:0/842631910 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6819a7e0 con 0x7fbc680ff390 2024-09-10T11:35:14.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.902+0000 7fbc66ffd700 1 -- 172.21.15.144:0/842631910 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc5801d910 con 0x7fbc680ff390 2024-09-10T11:35:14.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.902+0000 7fbc6fd38700 1 -- 172.21.15.144:0/842631910 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6819acd0 con 0x7fbc680ff390 2024-09-10T11:35:14.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.903+0000 7fbc6fd38700 1 -- 172.21.15.144:0/842631910 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6804ea50 con 0x7fbc680ff390 2024-09-10T11:35:14.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.903+0000 7fbc66ffd700 1 -- 172.21.15.144:0/842631910 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fbc58004040 con 0x7fbc680ff390 2024-09-10T11:35:14.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.903+0000 7fbc66ffd700 1 --2- 172.21.15.144:0/842631910 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc5006e200 0x7fbc500706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:14.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.904+0000 7fbc66ffd700 1 -- 172.21.15.144:0/842631910 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbc5808f8f0 con 0x7fbc680ff390 2024-09-10T11:35:14.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.904+0000 7fbc6d2d3700 1 --2- 172.21.15.144:0/842631910 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc5006e200 0x7fbc500706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:14.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.904+0000 7fbc6d2d3700 1 --2- 172.21.15.144:0/842631910 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc5006e200 0x7fbc500706b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c005fd0 tx=0x7fbc5c005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:14.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:14.908+0000 7fbc66ffd700 1 -- 172.21.15.144:0/842631910 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbc58058d80 con 0x7fbc680ff390 2024-09-10T11:35:14.996 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:14 smithi144 ceph-mon[87436]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:14.996 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:14 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/4108609483' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:14.997 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:14 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1[104366]: 2024-09-10T11:35:14.916+0000 7fc05180f640 -1 osd.1 0 read_superblock omap replica is missing. 2024-09-10T11:35:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:14 smithi178 ceph-mon[65940]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:14 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/4108609483' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:15.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.170+0000 7fbc6fd38700 1 -- 172.21.15.144:0/842631910 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc6819b250 con 0x7fbc5006e200 2024-09-10T11:35:15.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.171+0000 7fbc66ffd700 1 -- 172.21.15.144:0/842631910 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbc6819b250 con 0x7fbc5006e200 2024-09-10T11:35:15.172 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:35:15.173 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:35:15.173 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:35:15.173 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:35:15.173 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:35:15.173 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:35:15.173 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:35:15.173 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:35:15.173 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:35:15.173 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "8/23 daemons upgraded", 2024-09-10T11:35:15.173 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading osd daemons", 2024-09-10T11:35:15.173 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:35:15.173 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:35:15.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.174+0000 7fbc6fd38700 1 -- 172.21.15.144:0/842631910 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc5006e200 msgr2=0x7fbc500706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:15.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.174+0000 7fbc6fd38700 1 --2- 172.21.15.144:0/842631910 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc5006e200 0x7fbc500706b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbc5c005fd0 tx=0x7fbc5c005e30 comp rx=0 tx=0).stop 2024-09-10T11:35:15.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.174+0000 7fbc6fd38700 1 -- 172.21.15.144:0/842631910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc680ff390 msgr2=0x7fbc68195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:15.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.174+0000 7fbc6fd38700 1 --2- 172.21.15.144:0/842631910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc680ff390 0x7fbc68195d20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbc58000f80 tx=0x7fbc5800cdd0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.174+0000 7fbc6fd38700 1 -- 172.21.15.144:0/842631910 shutdown_connections 2024-09-10T11:35:15.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.174+0000 7fbc6fd38700 1 --2- 172.21.15.144:0/842631910 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc5006e200 0x7fbc500706b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.174+0000 7fbc6fd38700 1 --2- 172.21.15.144:0/842631910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc680ff390 0x7fbc68195d20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.174+0000 7fbc6fd38700 1 --2- 172.21.15.144:0/842631910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc680ffd30 0x7fbc68196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.174+0000 7fbc6fd38700 1 -- 172.21.15.144:0/842631910 >> 172.21.15.144:0/842631910 conn(0x7fbc68074f50 msgr2=0x7fbc68068450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:15.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.174+0000 7fbc6fd38700 1 -- 172.21.15.144:0/842631910 shutdown_connections 2024-09-10T11:35:15.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.174+0000 7fbc6fd38700 1 -- 172.21.15.144:0/842631910 wait complete. 2024-09-10T11:35:15.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.287+0000 7fec43d87700 1 -- 172.21.15.144:0/4077583620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec3c0ffc60 msgr2=0x7fec3c103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:15.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.287+0000 7fec43d87700 1 --2- 172.21.15.144:0/4077583620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec3c0ffc60 0x7fec3c103c30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009910 tx=0x7fec2c009c20 comp rx=0 tx=0).stop 2024-09-10T11:35:15.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.287+0000 7fec43d87700 1 -- 172.21.15.144:0/4077583620 shutdown_connections 2024-09-10T11:35:15.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.287+0000 7fec43d87700 1 --2- 172.21.15.144:0/4077583620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec3c0ffc60 0x7fec3c103c30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.287+0000 7fec43d87700 1 --2- 172.21.15.144:0/4077583620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec3c0ff2c0 0x7fec3c0ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.287+0000 7fec43d87700 1 -- 172.21.15.144:0/4077583620 >> 172.21.15.144:0/4077583620 conn(0x7fec3c0fab80 msgr2=0x7fec3c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:15.289 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:15 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1[104366]: 2024-09-10T11:35:15.227+0000 7fc05180f640 -1 osd.1 55 log_to_monitors true 2024-09-10T11:35:15.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.287+0000 7fec43d87700 1 -- 172.21.15.144:0/4077583620 shutdown_connections 2024-09-10T11:35:15.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.287+0000 7fec43d87700 1 -- 172.21.15.144:0/4077583620 wait complete. 2024-09-10T11:35:15.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.288+0000 7fec43d87700 1 Processor -- start 2024-09-10T11:35:15.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.288+0000 7fec43d87700 1 -- start start 2024-09-10T11:35:15.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.288+0000 7fec43d87700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec3c0ff2c0 0x7fec3c196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:15.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.288+0000 7fec43d87700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec3c0ffc60 0x7fec3c197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:15.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.288+0000 7fec43d87700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c197720 con 0x7fec3c0ffc60 2024-09-10T11:35:15.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.288+0000 7fec43d87700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c197860 con 0x7fec3c0ff2c0 2024-09-10T11:35:15.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.288+0000 7fec41322700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec3c0ffc60 0x7fec3c197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:15.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.288+0000 7fec41322700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec3c0ffc60 0x7fec3c197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:47438/0 (socket says 172.21.15.144:47438) 2024-09-10T11:35:15.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.288+0000 7fec41322700 1 -- 172.21.15.144:0/3651740044 learned_addr learned my addr 172.21.15.144:0/3651740044 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:35:15.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.289+0000 7fec41b23700 1 --2- 172.21.15.144:0/3651740044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec3c0ff2c0 0x7fec3c196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:15.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.289+0000 7fec41322700 1 -- 172.21.15.144:0/3651740044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec3c0ff2c0 msgr2=0x7fec3c196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:15.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.289+0000 7fec41322700 1 --2- 172.21.15.144:0/3651740044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec3c0ff2c0 0x7fec3c196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.289+0000 7fec41322700 1 -- 172.21.15.144:0/3651740044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec2c0095f0 con 0x7fec3c0ffc60 2024-09-10T11:35:15.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.289+0000 7fec41b23700 1 --2- 172.21.15.144:0/3651740044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec3c0ff2c0 0x7fec3c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:35:15.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.289+0000 7fec41322700 1 --2- 172.21.15.144:0/3651740044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec3c0ffc60 0x7fec3c197100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0092e0 tx=0x7fec2c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:15.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.289+0000 7fec32ffd700 1 -- 172.21.15.144:0/3651740044 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2c01d070 con 0x7fec3c0ffc60 2024-09-10T11:35:15.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.289+0000 7fec32ffd700 1 -- 172.21.15.144:0/3651740044 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec2c005dc0 con 0x7fec3c0ffc60 2024-09-10T11:35:15.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.289+0000 7fec43d87700 1 -- 172.21.15.144:0/3651740044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3c19fc30 con 0x7fec3c0ffc60 2024-09-10T11:35:15.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.289+0000 7fec32ffd700 1 -- 172.21.15.144:0/3651740044 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2c017400 con 0x7fec3c0ffc60 2024-09-10T11:35:15.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.289+0000 7fec43d87700 1 -- 172.21.15.144:0/3651740044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3c1a0040 con 0x7fec3c0ffc60 2024-09-10T11:35:15.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.290+0000 7fec43d87700 1 -- 172.21.15.144:0/3651740044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3c107630 con 0x7fec3c0ffc60 2024-09-10T11:35:15.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.291+0000 7fec32ffd700 1 -- 172.21.15.144:0/3651740044 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fec2c00f460 con 0x7fec3c0ffc60 2024-09-10T11:35:15.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.291+0000 7fec32ffd700 1 --2- 172.21.15.144:0/3651740044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec2806e200 0x7fec280706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:15.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.291+0000 7fec32ffd700 1 -- 172.21.15.144:0/3651740044 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fec2c08f4c0 con 0x7fec3c0ffc60 2024-09-10T11:35:15.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.291+0000 7fec41b23700 1 --2- 172.21.15.144:0/3651740044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec2806e200 0x7fec280706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:15.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.292+0000 7fec41b23700 1 --2- 172.21.15.144:0/3651740044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec2806e200 0x7fec280706b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fec38005950 tx=0x7fec380058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:15.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.295+0000 7fec32ffd700 1 -- 172.21.15.144:0/3651740044 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fec2c058950 con 0x7fec3c0ffc60 2024-09-10T11:35:15.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.594+0000 7fec43d87700 1 -- 172.21.15.144:0/3651740044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fec3c04ea50 con 0x7fec3c0ffc60 2024-09-10T11:35:15.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.595+0000 7fec32ffd700 1 -- 172.21.15.144:0/3651740044 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1350 (secure 0 0 0) 0x7fec2c05c010 con 0x7fec3c0ffc60 2024-09-10T11:35:15.596 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 56/360 objects degraded (15.556%), 16 pgs degraded 2024-09-10T11:35:15.597 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:35:15.597 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:35:15.597 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-10T11:35:15.597 INFO:teuthology.orchestra.run.smithi144.stdout: osd.1 (root=default,host=smithi144) is down 2024-09-10T11:35:15.597 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 56/360 objects degraded (15.556%), 16 pgs degraded 2024-09-10T11:35:15.597 INFO:teuthology.orchestra.run.smithi144.stdout: pg 1.0 is active+undersized+degraded, acting [3,0] 2024-09-10T11:35:15.597 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.0 is active+undersized+degraded, acting [3,0] 2024-09-10T11:35:15.597 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1 is active+undersized+degraded, acting [2,0] 2024-09-10T11:35:15.597 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.2 is active+undersized+degraded, acting [5,0] 2024-09-10T11:35:15.597 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.3 is active+undersized+degraded, acting [5,2] 2024-09-10T11:35:15.597 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.4 is active+undersized+degraded, acting [0,4] 2024-09-10T11:35:15.597 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.6 is active+undersized+degraded, acting [3,4] 2024-09-10T11:35:15.598 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.9 is active+undersized+degraded, acting [4,0] 2024-09-10T11:35:15.598 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.a is active+undersized+degraded, acting [4,3] 2024-09-10T11:35:15.598 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.d is active+undersized+degraded, acting [3,2] 2024-09-10T11:35:15.598 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.10 is active+undersized+degraded, acting [2,0] 2024-09-10T11:35:15.598 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.11 is active+undersized+degraded, acting [4,3] 2024-09-10T11:35:15.598 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.12 is active+undersized+degraded, acting [3,0] 2024-09-10T11:35:15.598 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.15 is active+undersized+degraded, acting [3,0] 2024-09-10T11:35:15.598 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.17 is active+undersized+degraded, acting [5,2] 2024-09-10T11:35:15.598 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1b is active+undersized+degraded, acting [0,5] 2024-09-10T11:35:15.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.597+0000 7fec43d87700 1 -- 172.21.15.144:0/3651740044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec2806e200 msgr2=0x7fec280706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:15.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.597+0000 7fec43d87700 1 --2- 172.21.15.144:0/3651740044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec2806e200 0x7fec280706b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fec38005950 tx=0x7fec380058e0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.597+0000 7fec43d87700 1 -- 172.21.15.144:0/3651740044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec3c0ffc60 msgr2=0x7fec3c197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:15.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.597+0000 7fec43d87700 1 --2- 172.21.15.144:0/3651740044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec3c0ffc60 0x7fec3c197100 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fec2c0092e0 tx=0x7fec2c00be20 comp rx=0 tx=0).stop 2024-09-10T11:35:15.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.598+0000 7fec43d87700 1 -- 172.21.15.144:0/3651740044 shutdown_connections 2024-09-10T11:35:15.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.598+0000 7fec43d87700 1 --2- 172.21.15.144:0/3651740044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec2806e200 0x7fec280706b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.598+0000 7fec43d87700 1 --2- 172.21.15.144:0/3651740044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec3c0ff2c0 0x7fec3c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.598+0000 7fec43d87700 1 --2- 172.21.15.144:0/3651740044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec3c0ffc60 0x7fec3c197100 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:15.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.598+0000 7fec43d87700 1 -- 172.21.15.144:0/3651740044 >> 172.21.15.144:0/3651740044 conn(0x7fec3c0fab80 msgr2=0x7fec3c0fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:15.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.598+0000 7fec43d87700 1 -- 172.21.15.144:0/3651740044 shutdown_connections 2024-09-10T11:35:15.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:15.598+0000 7fec43d87700 1 -- 172.21.15.144:0/3651740044 wait complete. 2024-09-10T11:35:15.934 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 11:35:15 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1[104366]: 2024-09-10T11:35:15.752+0000 7fc0495a9640 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:35:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:15 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1463528803' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:35:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:15 smithi144 ceph-mon[87436]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:15 smithi144 ceph-mon[87436]: from='osd.1 [v2:172.21.15.144:6810/1922853959,v1:172.21.15.144:6811/1922853959]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T11:35:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:15 smithi144 ceph-mon[87436]: pgmap v71: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-09-10T11:35:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:15 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3651740044' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:35:16.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:15 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1463528803' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:35:16.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:15 smithi178 ceph-mon[65940]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:16.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:15 smithi178 ceph-mon[65940]: from='osd.1 [v2:172.21.15.144:6810/1922853959,v1:172.21.15.144:6811/1922853959]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T11:35:16.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:15 smithi178 ceph-mon[65940]: pgmap v71: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 56/360 objects degraded (15.556%) 2024-09-10T11:35:16.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:15 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3651740044' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:35:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:16 smithi178 ceph-mon[65940]: from='osd.1 [v2:172.21.15.144:6810/1922853959,v1:172.21.15.144:6811/1922853959]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T11:35:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:16 smithi178 ceph-mon[65940]: osdmap e58: 6 total, 5 up, 6 in 2024-09-10T11:35:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:16 smithi178 ceph-mon[65940]: from='osd.1 [v2:172.21.15.144:6810/1922853959,v1:172.21.15.144:6811/1922853959]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T11:35:17.142 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:16 smithi144 ceph-mon[87436]: from='osd.1 [v2:172.21.15.144:6810/1922853959,v1:172.21.15.144:6811/1922853959]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T11:35:17.142 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:16 smithi144 ceph-mon[87436]: osdmap e58: 6 total, 5 up, 6 in 2024-09-10T11:35:17.142 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:16 smithi144 ceph-mon[87436]: from='osd.1 [v2:172.21.15.144:6810/1922853959,v1:172.21.15.144:6811/1922853959]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T11:35:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T11:35:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: osd.1 [v2:172.21.15.144:6810/1922853959,v1:172.21.15.144:6811/1922853959] boot 2024-09-10T11:35:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: osdmap e59: 6 total, 6 up, 6 in 2024-09-10T11:35:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:35:18.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:18.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:18.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:35:18.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:35:18.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:18.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:35:18.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:18.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:18.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:18.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-10T11:35:18.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-10T11:35:18.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-09-10T11:35:18.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: pgmap v74: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 56/360 objects degraded (15.556%) 2024-09-10T11:35:18.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:17 smithi178 ceph-mon[65940]: osdmap e60: 6 total, 6 up, 6 in 2024-09-10T11:35:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T11:35:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: osd.1 [v2:172.21.15.144:6810/1922853959,v1:172.21.15.144:6811/1922853959] boot 2024-09-10T11:35:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: osdmap e59: 6 total, 6 up, 6 in 2024-09-10T11:35:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T11:35:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:35:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:35:18.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:18.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:35:18.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:18.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:18.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:18.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-10T11:35:18.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-10T11:35:18.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-09-10T11:35:18.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: pgmap v74: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s; 56/360 objects degraded (15.556%) 2024-09-10T11:35:18.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:17 smithi144 ceph-mon[87436]: osdmap e60: 6 total, 6 up, 6 in 2024-09-10T11:35:19.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:19 smithi144 ceph-mon[87436]: Health check update: Degraded data redundancy: 23/360 objects degraded (6.389%), 7 pgs degraded (PG_DEGRADED) 2024-09-10T11:35:20.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:19 smithi178 ceph-mon[65940]: Health check update: Degraded data redundancy: 23/360 objects degraded (6.389%), 7 pgs degraded (PG_DEGRADED) 2024-09-10T11:35:20.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:20 smithi144 ceph-mon[87436]: pgmap v76: 65 pgs: 7 active+undersized, 7 active+undersized+degraded, 51 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 5 op/s; 23/360 objects degraded (6.389%) 2024-09-10T11:35:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:20 smithi178 ceph-mon[65940]: pgmap v76: 65 pgs: 7 active+undersized, 7 active+undersized+degraded, 51 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 3.2 KiB/s rd, 5 op/s; 23/360 objects degraded (6.389%) 2024-09-10T11:35:22.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:21 smithi178 ceph-mon[65940]: pgmap v77: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-09-10T11:35:22.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:21 smithi178 ceph-mon[65940]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/360 objects degraded (6.389%), 7 pgs degraded) 2024-09-10T11:35:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:21 smithi144 ceph-mon[87436]: pgmap v77: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 3 op/s 2024-09-10T11:35:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:21 smithi144 ceph-mon[87436]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 23/360 objects degraded (6.389%), 7 pgs degraded) 2024-09-10T11:35:24.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:24 smithi144 ceph-mon[87436]: pgmap v78: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-10T11:35:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:24 smithi178 ceph-mon[65940]: pgmap v78: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.1 KiB/s rd, 2 op/s 2024-09-10T11:35:26.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:26 smithi144 ceph-mon[87436]: pgmap v79: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-09-10T11:35:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:26 smithi178 ceph-mon[65940]: pgmap v79: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-09-10T11:35:28.919 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:28 smithi178 ceph-mon[65940]: pgmap v80: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T11:35:28.920 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:28.920 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:35:28.920 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:28.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:28 smithi144 ceph-mon[87436]: pgmap v80: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T11:35:28.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:28.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:35:28.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:30.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:30 smithi144 ceph-mon[87436]: pgmap v81: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:35:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:30 smithi178 ceph-mon[65940]: pgmap v81: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:35:32.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:32 smithi144 ceph-mon[87436]: pgmap v82: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:35:32.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:32 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-10T11:35:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:32 smithi178 ceph-mon[65940]: pgmap v82: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:35:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:32 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-10T11:35:33.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:33 smithi144 ceph-mon[87436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-10T11:35:33.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:33 smithi144 ceph-mon[87436]: Upgrade: osd.2 is safe to restart 2024-09-10T11:35:33.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:33.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T11:35:33.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:35:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:33 smithi178 ceph-mon[65940]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-09-10T11:35:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:33 smithi178 ceph-mon[65940]: Upgrade: osd.2 is safe to restart 2024-09-10T11:35:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T11:35:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:35:34.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:34 smithi144 ceph-mon[87436]: Upgrade: Updating osd.2 2024-09-10T11:35:34.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:34 smithi144 ceph-mon[87436]: Deploying daemon osd.2 on smithi144 2024-09-10T11:35:34.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:34 smithi144 ceph-mon[87436]: pgmap v83: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:35:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:34 smithi178 ceph-mon[65940]: Upgrade: Updating osd.2 2024-09-10T11:35:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:34 smithi178 ceph-mon[65940]: Deploying daemon osd.2 on smithi144 2024-09-10T11:35:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:34 smithi178 ceph-mon[65940]: pgmap v83: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:35:36.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:36 smithi144 ceph-mon[87436]: pgmap v84: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:35:36.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:36 smithi144 systemd[1]: Stopping Ceph osd.2 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:35:36.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:36 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2[53971]: 2024-09-10T11:35:36.645+0000 7f08208f3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:35:36.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:36 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2[53971]: 2024-09-10T11:35:36.645+0000 7f08208f3700 -1 osd.2 60 *** Got signal Terminated *** 2024-09-10T11:35:36.685 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:36 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2[53971]: 2024-09-10T11:35:36.645+0000 7f08208f3700 -1 osd.2 60 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:35:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:36 smithi178 ceph-mon[65940]: pgmap v84: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:35:37.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:37 smithi144 ceph-mon[87436]: osd.2 marked itself down and dead 2024-09-10T11:35:38.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:37 smithi178 ceph-mon[65940]: osd.2 marked itself down and dead 2024-09-10T11:35:38.390 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:38 smithi144 podman[109606]: 2024-09-10 11:35:38.096677752 +0000 UTC m=+1.543734150 container died 3ed5618eac331e1b088719a5ef704957ba5d9241300aefcdeb593f899711e0d5 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2, RELEASE=HEAD, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux ) 2024-09-10T11:35:38.652 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:38 smithi144 podman[109606]: 2024-09-10 11:35:38.545981789 +0000 UTC m=+1.993038112 container cleanup 3ed5618eac331e1b088719a5ef704957ba5d9241300aefcdeb593f899711e0d5 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=HEAD, org.label-schema.build-date=20231212) 2024-09-10T11:35:38.652 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:38 smithi144 bash[109606]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2 2024-09-10T11:35:38.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:38 smithi144 ceph-mon[87436]: pgmap v85: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:35:38.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:38 smithi144 ceph-mon[87436]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T11:35:38.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:38 smithi144 ceph-mon[87436]: osdmap e61: 6 total, 5 up, 6 in 2024-09-10T11:35:38.934 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:38 smithi144 podman[109624]: 2024-09-10 11:35:38.756363912 +0000 UTC m=+0.649163277 container remove 3ed5618eac331e1b088719a5ef704957ba5d9241300aefcdeb593f899711e0d5 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-09-10T11:35:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:38 smithi178 ceph-mon[65940]: pgmap v85: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:35:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:38 smithi178 ceph-mon[65940]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T11:35:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:38 smithi178 ceph-mon[65940]: osdmap e61: 6 total, 5 up, 6 in 2024-09-10T11:35:39.434 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:39 smithi144 podman[109711]: 2024-09-10 11:35:39.070546337 +0000 UTC m=+0.016105809 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:35:39.434 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:39 smithi144 podman[109711]: 2024-09-10 11:35:39.182098119 +0000 UTC m=+0.127657588 container create 7d073e3f41cb9399266e2951bb95a6e33b750ce663a9a29e2855cd6dc8b122c4 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True) 2024-09-10T11:35:39.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:39 smithi144 ceph-mon[87436]: osdmap e62: 6 total, 5 up, 6 in 2024-09-10T11:35:39.934 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:39 smithi144 podman[109711]: 2024-09-10 11:35:39.434899391 +0000 UTC m=+0.380458870 container init 7d073e3f41cb9399266e2951bb95a6e33b750ce663a9a29e2855cd6dc8b122c4 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:35:39.934 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:39 smithi144 podman[109711]: 2024-09-10 11:35:39.438400732 +0000 UTC m=+0.383960210 container start 7d073e3f41cb9399266e2951bb95a6e33b750ce663a9a29e2855cd6dc8b122c4 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T11:35:39.934 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:39 smithi144 podman[109711]: 2024-09-10 11:35:39.53496864 +0000 UTC m=+0.480528110 container attach 7d073e3f41cb9399266e2951bb95a6e33b750ce663a9a29e2855cd6dc8b122c4 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T11:35:39.935 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:39 smithi144 podman[109711]: 2024-09-10 11:35:39.629090879 +0000 UTC m=+0.574650349 container died 7d073e3f41cb9399266e2951bb95a6e33b750ce663a9a29e2855cd6dc8b122c4 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T11:35:40.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:39 smithi178 ceph-mon[65940]: osdmap e62: 6 total, 5 up, 6 in 2024-09-10T11:35:40.702 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:40 smithi144 ceph-mon[87436]: pgmap v88: 65 pgs: 7 active+undersized, 7 stale+active+clean, 5 active+undersized+degraded, 46 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 17/360 objects degraded (4.722%) 2024-09-10T11:35:40.703 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:40 smithi144 ceph-mon[87436]: Health check failed: Degraded data redundancy: 17/360 objects degraded (4.722%), 5 pgs degraded (PG_DEGRADED) 2024-09-10T11:35:40.703 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:40 smithi144 podman[109735]: 2024-09-10 11:35:40.447362193 +0000 UTC m=+0.812678864 container remove 7d073e3f41cb9399266e2951bb95a6e33b750ce663a9a29e2855cd6dc8b122c4 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:35:40.703 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:40 smithi144 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.2.service: Deactivated successfully. 2024-09-10T11:35:40.703 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:40 smithi144 systemd[1]: Stopped Ceph osd.2 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:35:40.703 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:40 smithi144 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.2.service: Consumed 45.467s CPU time. 2024-09-10T11:35:40.961 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:40 smithi144 systemd[1]: Starting Ceph osd.2 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:35:40.961 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:40 smithi144 podman[109833]: 2024-09-10 11:35:40.818556128 +0000 UTC m=+0.016026199 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:35:40.975 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:40 smithi178 ceph-mon[65940]: pgmap v88: 65 pgs: 7 active+undersized, 7 stale+active+clean, 5 active+undersized+degraded, 46 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 2 op/s; 17/360 objects degraded (4.722%) 2024-09-10T11:35:40.975 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:40 smithi178 ceph-mon[65940]: Health check failed: Degraded data redundancy: 17/360 objects degraded (4.722%), 5 pgs degraded (PG_DEGRADED) 2024-09-10T11:35:41.294 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:40 smithi144 podman[109833]: 2024-09-10 11:35:40.973770221 +0000 UTC m=+0.171240285 container create 650fe2add2c209f098fbebce8b1fb5422f9befbc81a6277c0b64eaa7ad1a054a (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-activate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T11:35:41.294 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:41 smithi144 podman[109833]: 2024-09-10 11:35:41.193375635 +0000 UTC m=+0.390845697 container init 650fe2add2c209f098fbebce8b1fb5422f9befbc81a6277c0b64eaa7ad1a054a (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-activate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T11:35:41.294 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:41 smithi144 podman[109833]: 2024-09-10 11:35:41.19693053 +0000 UTC m=+0.394400598 container start 650fe2add2c209f098fbebce8b1fb5422f9befbc81a6277c0b64eaa7ad1a054a (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-activate, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T11:35:41.294 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:41 smithi144 podman[109833]: 2024-09-10 11:35:41.293385478 +0000 UTC m=+0.490855542 container attach 650fe2add2c209f098fbebce8b1fb5422f9befbc81a6277c0b64eaa7ad1a054a (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-activate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T11:35:41.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:41 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-activate[109855]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:35:41.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:41 smithi144 bash[109833]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:35:41.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:41 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-activate[109855]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:35:41.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:41 smithi144 bash[109833]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:35:42.347 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-activate[109855]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T11:35:42.347 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 bash[109833]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T11:35:42.347 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-activate[109855]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-10T11:35:42.347 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 bash[109833]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-10T11:35:42.656 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-activate[109855]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-10T11:35:42.657 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 bash[109833]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-10T11:35:42.657 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-activate[109855]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-10T11:35:42.657 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 bash[109833]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-10T11:35:42.657 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-activate[109855]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-10T11:35:42.657 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 bash[109833]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-09-10T11:35:42.657 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-activate[109855]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T11:35:42.657 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 bash[109833]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-10T11:35:42.657 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-activate[109855]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-10T11:35:42.657 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 bash[109833]: --> ceph-volume raw activate successful for osd ID: 2 2024-09-10T11:35:42.657 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 podman[109833]: 2024-09-10 11:35:42.375024129 +0000 UTC m=+1.572494199 container died 650fe2add2c209f098fbebce8b1fb5422f9befbc81a6277c0b64eaa7ad1a054a (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-activate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T11:35:42.931 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:42 smithi144 ceph-mon[87436]: pgmap v89: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/360 objects degraded (12.500%) 2024-09-10T11:35:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:42 smithi178 ceph-mon[65940]: pgmap v89: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/360 objects degraded (12.500%) 2024-09-10T11:35:43.184 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:42 smithi144 podman[109992]: 2024-09-10 11:35:42.930734717 +0000 UTC m=+0.550052064 container remove 650fe2add2c209f098fbebce8b1fb5422f9befbc81a6277c0b64eaa7ad1a054a (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-activate, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T11:35:43.184 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:43 smithi144 podman[110052]: 2024-09-10 11:35:43.03665812 +0000 UTC m=+0.016015416 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:35:43.184 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:43 smithi144 podman[110052]: 2024-09-10 11:35:43.156639094 +0000 UTC m=+0.135996381 container create 8485d374ff4439a004c0723d8822bf6c528ecbbf1b3000a563e6f06465f02f4d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T11:35:43.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:43 smithi144 podman[110052]: 2024-09-10 11:35:43.384382143 +0000 UTC m=+0.363739430 container init 8485d374ff4439a004c0723d8822bf6c528ecbbf1b3000a563e6f06465f02f4d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-10T11:35:43.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:43 smithi144 podman[110052]: 2024-09-10 11:35:43.387635261 +0000 UTC m=+0.366992551 container start 8485d374ff4439a004c0723d8822bf6c528ecbbf1b3000a563e6f06465f02f4d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T11:35:43.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:43 smithi144 ceph-osd[110071]: -- 172.21.15.144:0/1309161807 <== mon.0 v2:172.21.15.144:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a04e661e0 con 0x560a04013c00 2024-09-10T11:35:43.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:43 smithi144 bash[110052]: 8485d374ff4439a004c0723d8822bf6c528ecbbf1b3000a563e6f06465f02f4d 2024-09-10T11:35:43.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:43 smithi144 systemd[1]: Started Ceph osd.2 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:35:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:35:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:44 smithi178 ceph-mon[65940]: pgmap v90: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/360 objects degraded (12.500%) 2024-09-10T11:35:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:35:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:35:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:44 smithi144 ceph-mon[87436]: pgmap v90: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/360 objects degraded (12.500%) 2024-09-10T11:35:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:35:45.434 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:45 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2[110067]: 2024-09-10T11:35:45.033+0000 7f11ebe43640 -1 Falling back to public interface 2024-09-10T11:35:45.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.711+0000 7f02b0ce6700 1 -- 172.21.15.144:0/1292555685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02ac0ff310 msgr2=0x7f02ac0ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:45.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.711+0000 7f02b0ce6700 1 --2- 172.21.15.144:0/1292555685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02ac0ff310 0x7f02ac0ff6e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f029c009910 tx=0x7f029c009c20 comp rx=0 tx=0).stop 2024-09-10T11:35:45.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.711+0000 7f02b0ce6700 1 -- 172.21.15.144:0/1292555685 shutdown_connections 2024-09-10T11:35:45.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.711+0000 7f02b0ce6700 1 --2- 172.21.15.144:0/1292555685 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02ac0ffcb0 0x7f02ac103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:45.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.711+0000 7f02b0ce6700 1 --2- 172.21.15.144:0/1292555685 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02ac0ff310 0x7f02ac0ff6e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:45.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.711+0000 7f02b0ce6700 1 -- 172.21.15.144:0/1292555685 >> 172.21.15.144:0/1292555685 conn(0x7f02ac0fabb0 msgr2=0x7f02ac0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:45.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.711+0000 7f02b0ce6700 1 -- 172.21.15.144:0/1292555685 shutdown_connections 2024-09-10T11:35:45.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.711+0000 7f02b0ce6700 1 -- 172.21.15.144:0/1292555685 wait complete. 2024-09-10T11:35:45.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.712+0000 7f02b0ce6700 1 Processor -- start 2024-09-10T11:35:45.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.712+0000 7f02b0ce6700 1 -- start start 2024-09-10T11:35:45.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.712+0000 7f02b0ce6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02ac0ff310 0x7f02ac100d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:45.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.712+0000 7f02b0ce6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02ac0ffcb0 0x7f02ac101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:45.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.712+0000 7f02b0ce6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac1017a0 con 0x7f02ac0ffcb0 2024-09-10T11:35:45.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.712+0000 7f02b0ce6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ac1018e0 con 0x7f02ac0ff310 2024-09-10T11:35:45.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.712+0000 7f02a9d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02ac0ffcb0 0x7f02ac101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:45.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.712+0000 7f02a9d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02ac0ffcb0 0x7f02ac101260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53768/0 (socket says 172.21.15.144:53768) 2024-09-10T11:35:45.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.712+0000 7f02a9d9b700 1 -- 172.21.15.144:0/1138616502 learned_addr learned my addr 172.21.15.144:0/1138616502 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:35:45.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.712+0000 7f02aa59c700 1 --2- 172.21.15.144:0/1138616502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02ac0ff310 0x7f02ac100d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:45.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.712+0000 7f02a9d9b700 1 -- 172.21.15.144:0/1138616502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02ac0ff310 msgr2=0x7f02ac100d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:45.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.712+0000 7f02a9d9b700 1 --2- 172.21.15.144:0/1138616502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02ac0ff310 0x7f02ac100d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:45.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.712+0000 7f02a9d9b700 1 -- 172.21.15.144:0/1138616502 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c0095f0 con 0x7f02ac0ffcb0 2024-09-10T11:35:45.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.712+0000 7f02aa59c700 1 --2- 172.21.15.144:0/1138616502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02ac0ff310 0x7f02ac100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:45.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.713+0000 7f02a9d9b700 1 --2- 172.21.15.144:0/1138616502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02ac0ffcb0 0x7f02ac101260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f02a000d900 tx=0x7f02a000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:45.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.713+0000 7f029b7fe700 1 -- 172.21.15.144:0/1138616502 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02a0009900 con 0x7f02ac0ffcb0 2024-09-10T11:35:45.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.713+0000 7f02b0ce6700 1 -- 172.21.15.144:0/1138616502 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ac1a1f50 con 0x7f02ac0ffcb0 2024-09-10T11:35:45.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.713+0000 7f029b7fe700 1 -- 172.21.15.144:0/1138616502 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02a0010460 con 0x7f02ac0ffcb0 2024-09-10T11:35:45.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.713+0000 7f029b7fe700 1 -- 172.21.15.144:0/1138616502 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f02a000f5f0 con 0x7f02ac0ffcb0 2024-09-10T11:35:45.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.713+0000 7f02b0ce6700 1 -- 172.21.15.144:0/1138616502 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ac1a2360 con 0x7f02ac0ffcb0 2024-09-10T11:35:45.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.714+0000 7f02b0ce6700 1 -- 172.21.15.144:0/1138616502 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02ac108630 con 0x7f02ac0ffcb0 2024-09-10T11:35:45.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.714+0000 7f029b7fe700 1 -- 172.21.15.144:0/1138616502 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f02a00105d0 con 0x7f02ac0ffcb0 2024-09-10T11:35:45.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.715+0000 7f029b7fe700 1 --2- 172.21.15.144:0/1138616502 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f029406e200 0x7f02940706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:45.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.715+0000 7f02aa59c700 1 --2- 172.21.15.144:0/1138616502 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f029406e200 0x7f02940706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:45.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.715+0000 7f029b7fe700 1 -- 172.21.15.144:0/1138616502 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f02a008da50 con 0x7f02ac0ffcb0 2024-09-10T11:35:45.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.716+0000 7f02aa59c700 1 --2- 172.21.15.144:0/1138616502 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f029406e200 0x7f02940706b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f029c006010 tx=0x7f029c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:45.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.720+0000 7f029b7fe700 1 -- 172.21.15.144:0/1138616502 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f02a0056f60 con 0x7f02ac0ffcb0 2024-09-10T11:35:45.952 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.951+0000 7f02b0ce6700 1 -- 172.21.15.144:0/1138616502 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f02ac02cd40 con 0x7f029406e200 2024-09-10T11:35:45.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.952+0000 7f029b7fe700 1 -- 172.21.15.144:0/1138616502 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f02ac02cd40 con 0x7f029406e200 2024-09-10T11:35:45.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.954+0000 7f02b0ce6700 1 -- 172.21.15.144:0/1138616502 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f029406e200 msgr2=0x7f02940706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:45.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.954+0000 7f02b0ce6700 1 --2- 172.21.15.144:0/1138616502 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f029406e200 0x7f02940706b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f029c006010 tx=0x7f029c00b540 comp rx=0 tx=0).stop 2024-09-10T11:35:45.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.954+0000 7f02b0ce6700 1 -- 172.21.15.144:0/1138616502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02ac0ffcb0 msgr2=0x7f02ac101260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:45.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.954+0000 7f02b0ce6700 1 --2- 172.21.15.144:0/1138616502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02ac0ffcb0 0x7f02ac101260 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f02a000d900 tx=0x7f02a000dcc0 comp rx=0 tx=0).stop 2024-09-10T11:35:45.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.954+0000 7f02b0ce6700 1 -- 172.21.15.144:0/1138616502 shutdown_connections 2024-09-10T11:35:45.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.954+0000 7f02b0ce6700 1 --2- 172.21.15.144:0/1138616502 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f029406e200 0x7f02940706b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:45.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.954+0000 7f02b0ce6700 1 --2- 172.21.15.144:0/1138616502 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f02ac0ff310 0x7f02ac100d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:45.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.954+0000 7f02b0ce6700 1 --2- 172.21.15.144:0/1138616502 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f02ac0ffcb0 0x7f02ac101260 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:45.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.954+0000 7f02b0ce6700 1 -- 172.21.15.144:0/1138616502 >> 172.21.15.144:0/1138616502 conn(0x7f02ac0fabb0 msgr2=0x7f02ac069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:45.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.954+0000 7f02b0ce6700 1 -- 172.21.15.144:0/1138616502 shutdown_connections 2024-09-10T11:35:45.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:45.954+0000 7f02b0ce6700 1 -- 172.21.15.144:0/1138616502 wait complete. 2024-09-10T11:35:45.966 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:35:46.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.061+0000 7fbf574eb700 1 -- 172.21.15.144:0/2277128725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf50100d70 msgr2=0x7fbf50104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:46.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.061+0000 7fbf574eb700 1 --2- 172.21.15.144:0/2277128725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf50100d70 0x7fbf50104d40 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009910 tx=0x7fbf44009c20 comp rx=0 tx=0).stop 2024-09-10T11:35:46.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.061+0000 7fbf574eb700 1 -- 172.21.15.144:0/2277128725 shutdown_connections 2024-09-10T11:35:46.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.061+0000 7fbf574eb700 1 --2- 172.21.15.144:0/2277128725 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf50100d70 0x7fbf50104d40 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:46.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.061+0000 7fbf574eb700 1 --2- 172.21.15.144:0/2277128725 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf501003d0 0x7fbf501007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:46.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.061+0000 7fbf574eb700 1 -- 172.21.15.144:0/2277128725 >> 172.21.15.144:0/2277128725 conn(0x7fbf500fbc90 msgr2=0x7fbf500fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:46.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.061+0000 7fbf574eb700 1 -- 172.21.15.144:0/2277128725 shutdown_connections 2024-09-10T11:35:46.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.061+0000 7fbf574eb700 1 -- 172.21.15.144:0/2277128725 wait complete. 2024-09-10T11:35:46.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.062+0000 7fbf574eb700 1 Processor -- start 2024-09-10T11:35:46.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.062+0000 7fbf574eb700 1 -- start start 2024-09-10T11:35:46.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.062+0000 7fbf574eb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf501003d0 0x7fbf50073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:46.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.062+0000 7fbf574eb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf50100d70 0x7fbf50073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:46.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.062+0000 7fbf574eb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf50073ad0 con 0x7fbf501003d0 2024-09-10T11:35:46.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.062+0000 7fbf574eb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf50073c10 con 0x7fbf50100d70 2024-09-10T11:35:46.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.062+0000 7fbf55287700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf501003d0 0x7fbf50073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:46.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.062+0000 7fbf55287700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf501003d0 0x7fbf50073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53798/0 (socket says 172.21.15.144:53798) 2024-09-10T11:35:46.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.062+0000 7fbf55287700 1 -- 172.21.15.144:0/1431027809 learned_addr learned my addr 172.21.15.144:0/1431027809 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:35:46.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.062+0000 7fbf54a86700 1 --2- 172.21.15.144:0/1431027809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf50100d70 0x7fbf50073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:46.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.062+0000 7fbf55287700 1 -- 172.21.15.144:0/1431027809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf50100d70 msgr2=0x7fbf50073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:46.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.062+0000 7fbf55287700 1 --2- 172.21.15.144:0/1431027809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf50100d70 0x7fbf50073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:46.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.062+0000 7fbf55287700 1 -- 172.21.15.144:0/1431027809 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf440095f0 con 0x7fbf501003d0 2024-09-10T11:35:46.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.063+0000 7fbf55287700 1 --2- 172.21.15.144:0/1431027809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf501003d0 0x7fbf50073050 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800e920 tx=0x7fbf4800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:46.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.063+0000 7fbf4e7fc700 1 -- 172.21.15.144:0/1431027809 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf4800c9d0 con 0x7fbf501003d0 2024-09-10T11:35:46.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.063+0000 7fbf4e7fc700 1 -- 172.21.15.144:0/1431027809 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf48010430 con 0x7fbf501003d0 2024-09-10T11:35:46.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.063+0000 7fbf574eb700 1 -- 172.21.15.144:0/1431027809 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf501a1f10 con 0x7fbf501003d0 2024-09-10T11:35:46.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.063+0000 7fbf4e7fc700 1 -- 172.21.15.144:0/1431027809 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf48018630 con 0x7fbf501003d0 2024-09-10T11:35:46.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.063+0000 7fbf574eb700 1 -- 172.21.15.144:0/1431027809 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf501a22c0 con 0x7fbf501003d0 2024-09-10T11:35:46.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.064+0000 7fbf574eb700 1 -- 172.21.15.144:0/1431027809 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf50108740 con 0x7fbf501003d0 2024-09-10T11:35:46.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.064+0000 7fbf4e7fc700 1 -- 172.21.15.144:0/1431027809 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fbf480105a0 con 0x7fbf501003d0 2024-09-10T11:35:46.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.065+0000 7fbf4e7fc700 1 --2- 172.21.15.144:0/1431027809 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf3806e200 0x7fbf380706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:46.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.065+0000 7fbf4e7fc700 1 -- 172.21.15.144:0/1431027809 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbf48014070 con 0x7fbf501003d0 2024-09-10T11:35:46.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.065+0000 7fbf54a86700 1 --2- 172.21.15.144:0/1431027809 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf3806e200 0x7fbf380706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:46.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.065+0000 7fbf54a86700 1 --2- 172.21.15.144:0/1431027809 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf3806e200 0x7fbf380706b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbf440059d0 tx=0x7fbf44005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:46.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.069+0000 7fbf4e7fc700 1 -- 172.21.15.144:0/1431027809 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbf480541b0 con 0x7fbf501003d0 2024-09-10T11:35:46.309 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.308+0000 7fbf574eb700 1 -- 172.21.15.144:0/1431027809 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf5002cca0 con 0x7fbf3806e200 2024-09-10T11:35:46.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.309+0000 7fbf4e7fc700 1 -- 172.21.15.144:0/1431027809 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbf5002cca0 con 0x7fbf3806e200 2024-09-10T11:35:46.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.312+0000 7fbf574eb700 1 -- 172.21.15.144:0/1431027809 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf3806e200 msgr2=0x7fbf380706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:46.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.312+0000 7fbf574eb700 1 --2- 172.21.15.144:0/1431027809 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf3806e200 0x7fbf380706b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fbf440059d0 tx=0x7fbf44005940 comp rx=0 tx=0).stop 2024-09-10T11:35:46.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.312+0000 7fbf574eb700 1 -- 172.21.15.144:0/1431027809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf501003d0 msgr2=0x7fbf50073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:46.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.312+0000 7fbf574eb700 1 --2- 172.21.15.144:0/1431027809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf501003d0 0x7fbf50073050 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbf4800e920 tx=0x7fbf4800ec30 comp rx=0 tx=0).stop 2024-09-10T11:35:46.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.312+0000 7fbf574eb700 1 -- 172.21.15.144:0/1431027809 shutdown_connections 2024-09-10T11:35:46.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.312+0000 7fbf574eb700 1 --2- 172.21.15.144:0/1431027809 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf3806e200 0x7fbf380706b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:46.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.312+0000 7fbf574eb700 1 --2- 172.21.15.144:0/1431027809 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf501003d0 0x7fbf50073050 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:46.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.312+0000 7fbf574eb700 1 --2- 172.21.15.144:0/1431027809 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf50100d70 0x7fbf50073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:46.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.312+0000 7fbf574eb700 1 -- 172.21.15.144:0/1431027809 >> 172.21.15.144:0/1431027809 conn(0x7fbf500fbc90 msgr2=0x7fbf500fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:46.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.312+0000 7fbf574eb700 1 -- 172.21.15.144:0/1431027809 shutdown_connections 2024-09-10T11:35:46.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.312+0000 7fbf574eb700 1 -- 172.21.15.144:0/1431027809 wait complete. 2024-09-10T11:35:46.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.423+0000 7f95ed05f700 1 -- 172.21.15.144:0/734674623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e8100400 msgr2=0x7f95e81007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:46.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.423+0000 7f95ed05f700 1 --2- 172.21.15.144:0/734674623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e8100400 0x7f95e81007d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f95d8009910 tx=0x7f95d8009c20 comp rx=0 tx=0).stop 2024-09-10T11:35:46.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.423+0000 7f95ed05f700 1 -- 172.21.15.144:0/734674623 shutdown_connections 2024-09-10T11:35:46.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.423+0000 7f95ed05f700 1 --2- 172.21.15.144:0/734674623 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e8100da0 0x7f95e8104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:46.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.423+0000 7f95ed05f700 1 --2- 172.21.15.144:0/734674623 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e8100400 0x7f95e81007d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:46.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.423+0000 7f95ed05f700 1 -- 172.21.15.144:0/734674623 >> 172.21.15.144:0/734674623 conn(0x7f95e80fbca0 msgr2=0x7f95e80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:46.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.423+0000 7f95ed05f700 1 -- 172.21.15.144:0/734674623 shutdown_connections 2024-09-10T11:35:46.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.423+0000 7f95ed05f700 1 -- 172.21.15.144:0/734674623 wait complete. 2024-09-10T11:35:46.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.424+0000 7f95ed05f700 1 Processor -- start 2024-09-10T11:35:46.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.424+0000 7f95ed05f700 1 -- start start 2024-09-10T11:35:46.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.424+0000 7f95ed05f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e8100400 0x7f95e8195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:46.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.424+0000 7f95ed05f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e8100da0 0x7f95e8196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:46.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.424+0000 7f95ed05f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e8196a60 con 0x7f95e8100400 2024-09-10T11:35:46.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.424+0000 7f95ed05f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e819a7f0 con 0x7f95e8100da0 2024-09-10T11:35:46.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.424+0000 7f95e6d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e8100400 0x7f95e8195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:46.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.424+0000 7f95e6d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e8100400 0x7f95e8195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53816/0 (socket says 172.21.15.144:53816) 2024-09-10T11:35:46.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.424+0000 7f95e6d9d700 1 -- 172.21.15.144:0/853754429 learned_addr learned my addr 172.21.15.144:0/853754429 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:35:46.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.424+0000 7f95e659c700 1 --2- 172.21.15.144:0/853754429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e8100da0 0x7f95e8196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:46.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.424+0000 7f95e6d9d700 1 -- 172.21.15.144:0/853754429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e8100da0 msgr2=0x7f95e8196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:46.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.424+0000 7f95e6d9d700 1 --2- 172.21.15.144:0/853754429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e8100da0 0x7f95e8196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:46.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.424+0000 7f95e6d9d700 1 -- 172.21.15.144:0/853754429 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d80095f0 con 0x7f95e8100400 2024-09-10T11:35:46.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.424+0000 7f95e659c700 1 --2- 172.21.15.144:0/853754429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e8100da0 0x7f95e8196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:35:46.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.425+0000 7f95e6d9d700 1 --2- 172.21.15.144:0/853754429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e8100400 0x7f95e8195e40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95d8006010 tx=0x7f95d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:46.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.425+0000 7f95cffff700 1 -- 172.21.15.144:0/853754429 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95d801d070 con 0x7f95e8100400 2024-09-10T11:35:46.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.425+0000 7f95cffff700 1 -- 172.21.15.144:0/853754429 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95d8004fb0 con 0x7f95e8100400 2024-09-10T11:35:46.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.425+0000 7f95cffff700 1 -- 172.21.15.144:0/853754429 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95d8022ea0 con 0x7f95e8100400 2024-09-10T11:35:46.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.425+0000 7f95ed05f700 1 -- 172.21.15.144:0/853754429 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e819aa70 con 0x7f95e8100400 2024-09-10T11:35:46.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.425+0000 7f95ed05f700 1 -- 172.21.15.144:0/853754429 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e819af60 con 0x7f95e8100400 2024-09-10T11:35:46.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.426+0000 7f95ed05f700 1 -- 172.21.15.144:0/853754429 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e8108710 con 0x7f95e8100400 2024-09-10T11:35:46.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.426+0000 7f95cffff700 1 -- 172.21.15.144:0/853754429 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f95d8022470 con 0x7f95e8100400 2024-09-10T11:35:46.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.427+0000 7f95cffff700 1 --2- 172.21.15.144:0/853754429 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f95d006e200 0x7f95d00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:46.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.427+0000 7f95cffff700 1 -- 172.21.15.144:0/853754429 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f95d808f560 con 0x7f95e8100400 2024-09-10T11:35:46.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.427+0000 7f95e659c700 1 --2- 172.21.15.144:0/853754429 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f95d006e200 0x7f95d00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:46.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.427+0000 7f95e659c700 1 --2- 172.21.15.144:0/853754429 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f95d006e200 0x7f95d00706b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f95e8197460 tx=0x7f95dc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:46.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.432+0000 7f95cffff700 1 -- 172.21.15.144:0/853754429 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f95d8058aa0 con 0x7f95e8100400 2024-09-10T11:35:46.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.652+0000 7f95ed05f700 1 -- 172.21.15.144:0/853754429 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f95e819b240 con 0x7f95d006e200 2024-09-10T11:35:46.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.663+0000 7f95cffff700 1 -- 172.21.15.144:0/853754429 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f95e819b240 con 0x7f95d006e200 2024-09-10T11:35:46.665 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:35:46.665 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (4m) 33s ago 14m 25.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:35:46.665 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (14m) 33s ago 14m 8418k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:35:46.665 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (13m) 92s ago 13m 8090k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:35:46.665 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (100s) 33s ago 14m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:35:46.665 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (95s) 92s ago 13m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:35:46.665 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (3m) 33s ago 14m 79.7M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:35:46.665 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (9m) 33s ago 9m 60.0M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:35:46.665 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (9m) 33s ago 9m 126M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:35:46.665 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (8m) 92s ago 8m 62.2M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:35:46.665 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (9m) 92s ago 9m 127M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:35:46.666 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (5m) 33s ago 15m 498M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:35:46.666 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (4m) 92s ago 13m 439M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:35:46.666 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (2m) 33s ago 16m 48.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:35:46.666 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (2m) 92s ago 13m 35.5M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:35:46.666 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (4m) 33s ago 14m 15.9M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:35:46.666 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (4m) 92s ago 13m 15.8M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:35:46.666 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (74s) 33s ago 12m 87.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:35:46.666 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (39s) 33s ago 12m 14.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:35:46.666 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 starting - - - 1216M 2024-09-10T11:35:46.666 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (11m) 92s ago 11m 255M 2239M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:35:46.666 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (11m) 92s ago 10m 219M 2239M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:35:46.666 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (10m) 92s ago 10m 198M 2239M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:35:46.666 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (4m) 33s ago 14m 77.2M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:35:46.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.665+0000 7f95ed05f700 1 -- 172.21.15.144:0/853754429 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f95d006e200 msgr2=0x7f95d00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:46.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.665+0000 7f95ed05f700 1 --2- 172.21.15.144:0/853754429 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f95d006e200 0x7f95d00706b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f95e8197460 tx=0x7f95dc008040 comp rx=0 tx=0).stop 2024-09-10T11:35:46.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.665+0000 7f95ed05f700 1 -- 172.21.15.144:0/853754429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e8100400 msgr2=0x7f95e8195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:46.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.665+0000 7f95ed05f700 1 --2- 172.21.15.144:0/853754429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e8100400 0x7f95e8195e40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f95d8006010 tx=0x7f95d800be20 comp rx=0 tx=0).stop 2024-09-10T11:35:46.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.666+0000 7f95ed05f700 1 -- 172.21.15.144:0/853754429 shutdown_connections 2024-09-10T11:35:46.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.666+0000 7f95ed05f700 1 --2- 172.21.15.144:0/853754429 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f95d006e200 0x7f95d00706b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:46.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.666+0000 7f95ed05f700 1 --2- 172.21.15.144:0/853754429 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95e8100400 0x7f95e8195e40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:46.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.666+0000 7f95ed05f700 1 --2- 172.21.15.144:0/853754429 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95e8100da0 0x7f95e8196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:46.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.666+0000 7f95ed05f700 1 -- 172.21.15.144:0/853754429 >> 172.21.15.144:0/853754429 conn(0x7f95e80fbca0 msgr2=0x7f95e80fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:46.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.666+0000 7f95ed05f700 1 -- 172.21.15.144:0/853754429 shutdown_connections 2024-09-10T11:35:46.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.666+0000 7f95ed05f700 1 -- 172.21.15.144:0/853754429 wait complete. 2024-09-10T11:35:46.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.771+0000 7fbfb9762700 1 -- 172.21.15.144:0/2726347543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfb41003d0 msgr2=0x7fbfb41007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:46.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.771+0000 7fbfb9762700 1 --2- 172.21.15.144:0/2726347543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfb41003d0 0x7fbfb41007a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009910 tx=0x7fbfa8009c20 comp rx=0 tx=0).stop 2024-09-10T11:35:46.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.771+0000 7fbfb9762700 1 -- 172.21.15.144:0/2726347543 shutdown_connections 2024-09-10T11:35:46.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.771+0000 7fbfb9762700 1 --2- 172.21.15.144:0/2726347543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb4100d70 0x7fbfb4104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:46.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.771+0000 7fbfb9762700 1 --2- 172.21.15.144:0/2726347543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfb41003d0 0x7fbfb41007a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:46.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.771+0000 7fbfb9762700 1 -- 172.21.15.144:0/2726347543 >> 172.21.15.144:0/2726347543 conn(0x7fbfb40fbc90 msgr2=0x7fbfb40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:46.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.771+0000 7fbfb9762700 1 -- 172.21.15.144:0/2726347543 shutdown_connections 2024-09-10T11:35:46.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.771+0000 7fbfb9762700 1 -- 172.21.15.144:0/2726347543 wait complete. 2024-09-10T11:35:46.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.772+0000 7fbfb9762700 1 Processor -- start 2024-09-10T11:35:46.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.772+0000 7fbfb9762700 1 -- start start 2024-09-10T11:35:46.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.772+0000 7fbfb9762700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb41003d0 0x7fbfb4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:46.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.772+0000 7fbfb9762700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfb4100d70 0x7fbfb4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:46.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.772+0000 7fbfb9762700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4073ad0 con 0x7fbfb4100d70 2024-09-10T11:35:46.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.772+0000 7fbfb9762700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb4073c10 con 0x7fbfb41003d0 2024-09-10T11:35:46.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.772+0000 7fbfb27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfb4100d70 0x7fbfb4073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:46.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.772+0000 7fbfb27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfb4100d70 0x7fbfb4073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53846/0 (socket says 172.21.15.144:53846) 2024-09-10T11:35:46.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.772+0000 7fbfb27fc700 1 -- 172.21.15.144:0/665875363 learned_addr learned my addr 172.21.15.144:0/665875363 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:35:46.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.772+0000 7fbfb2ffd700 1 --2- 172.21.15.144:0/665875363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb41003d0 0x7fbfb4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:46.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.773+0000 7fbfb27fc700 1 -- 172.21.15.144:0/665875363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb41003d0 msgr2=0x7fbfb4073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:46.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.773+0000 7fbfb27fc700 1 --2- 172.21.15.144:0/665875363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb41003d0 0x7fbfb4073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:46.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.773+0000 7fbfb27fc700 1 -- 172.21.15.144:0/665875363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa80095f0 con 0x7fbfb4100d70 2024-09-10T11:35:46.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.773+0000 7fbfb27fc700 1 --2- 172.21.15.144:0/665875363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfb4100d70 0x7fbfb4073590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400d900 tx=0x7fbfa400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:46.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.773+0000 7fbf9bfff700 1 -- 172.21.15.144:0/665875363 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfa4009900 con 0x7fbfb4100d70 2024-09-10T11:35:46.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.773+0000 7fbf9bfff700 1 -- 172.21.15.144:0/665875363 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfa4010460 con 0x7fbfb4100d70 2024-09-10T11:35:46.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.773+0000 7fbfb9762700 1 -- 172.21.15.144:0/665875363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb41a1f30 con 0x7fbfb4100d70 2024-09-10T11:35:46.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.773+0000 7fbf9bfff700 1 -- 172.21.15.144:0/665875363 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfa400f5f0 con 0x7fbfb4100d70 2024-09-10T11:35:46.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.773+0000 7fbfb9762700 1 -- 172.21.15.144:0/665875363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb41a2340 con 0x7fbfb4100d70 2024-09-10T11:35:46.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.774+0000 7fbfb9762700 1 -- 172.21.15.144:0/665875363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb41086e0 con 0x7fbfb4100d70 2024-09-10T11:35:46.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.775+0000 7fbf9bfff700 1 -- 172.21.15.144:0/665875363 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fbfa40105d0 con 0x7fbfb4100d70 2024-09-10T11:35:46.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.775+0000 7fbf9bfff700 1 --2- 172.21.15.144:0/665875363 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf9c06e200 0x7fbf9c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:46.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.775+0000 7fbf9bfff700 1 -- 172.21.15.144:0/665875363 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbfa408dbb0 con 0x7fbfb4100d70 2024-09-10T11:35:46.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.775+0000 7fbfb2ffd700 1 --2- 172.21.15.144:0/665875363 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf9c06e200 0x7fbf9c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:46.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.776+0000 7fbfb2ffd700 1 --2- 172.21.15.144:0/665875363 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf9c06e200 0x7fbf9c0706b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8006010 tx=0x7fbfa801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:46.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:46.779+0000 7fbf9bfff700 1 -- 172.21.15.144:0/665875363 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbfa4094080 con 0x7fbfb4100d70 2024-09-10T11:35:46.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:46 smithi144 ceph-mon[87436]: pgmap v91: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/360 objects degraded (12.500%) 2024-09-10T11:35:47.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.077+0000 7fbfb9762700 1 -- 172.21.15.144:0/665875363 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbfb404ea50 con 0x7fbfb4100d70 2024-09-10T11:35:47.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.078+0000 7fbf9bfff700 1 -- 172.21.15.144:0/665875363 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7fbfa4057040 con 0x7fbfb4100d70 2024-09-10T11:35:47.079 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:35:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:35:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:35:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:35:47.079 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:35:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:35:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:35:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:35:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-09-10T11:35:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:35:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:35:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:35:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:35:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:35:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:35:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 7, 2024-09-10T11:35:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:35:47.080 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:35:47.080 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:35:47.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.080+0000 7fbfb9762700 1 -- 172.21.15.144:0/665875363 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf9c06e200 msgr2=0x7fbf9c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:47.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.080+0000 7fbfb9762700 1 --2- 172.21.15.144:0/665875363 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf9c06e200 0x7fbf9c0706b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8006010 tx=0x7fbfa801a040 comp rx=0 tx=0).stop 2024-09-10T11:35:47.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.080+0000 7fbfb9762700 1 -- 172.21.15.144:0/665875363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfb4100d70 msgr2=0x7fbfb4073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:47.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.080+0000 7fbfb9762700 1 --2- 172.21.15.144:0/665875363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfb4100d70 0x7fbfb4073590 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fbfa400d900 tx=0x7fbfa400dc10 comp rx=0 tx=0).stop 2024-09-10T11:35:47.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.080+0000 7fbfb9762700 1 -- 172.21.15.144:0/665875363 shutdown_connections 2024-09-10T11:35:47.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.080+0000 7fbfb9762700 1 --2- 172.21.15.144:0/665875363 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf9c06e200 0x7fbf9c0706b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.080+0000 7fbfb9762700 1 --2- 172.21.15.144:0/665875363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbfb41003d0 0x7fbfb4073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.080+0000 7fbfb9762700 1 --2- 172.21.15.144:0/665875363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbfb4100d70 0x7fbfb4073590 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.080+0000 7fbfb9762700 1 -- 172.21.15.144:0/665875363 >> 172.21.15.144:0/665875363 conn(0x7fbfb40fbc90 msgr2=0x7fbfb40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:47.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.080+0000 7fbfb9762700 1 -- 172.21.15.144:0/665875363 shutdown_connections 2024-09-10T11:35:47.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.080+0000 7fbfb9762700 1 -- 172.21.15.144:0/665875363 wait complete. 2024-09-10T11:35:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:46 smithi178 ceph-mon[65940]: pgmap v91: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/360 objects degraded (12.500%) 2024-09-10T11:35:47.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.190+0000 7fad36126700 1 -- 172.21.15.144:0/2455810598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad300ff390 msgr2=0x7fad300ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:47.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.190+0000 7fad36126700 1 --2- 172.21.15.144:0/2455810598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad300ff390 0x7fad300ff760 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fad20009910 tx=0x7fad20009c20 comp rx=0 tx=0).stop 2024-09-10T11:35:47.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.190+0000 7fad36126700 1 -- 172.21.15.144:0/2455810598 shutdown_connections 2024-09-10T11:35:47.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.190+0000 7fad36126700 1 --2- 172.21.15.144:0/2455810598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad300ffd30 0x7fad30103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.190+0000 7fad36126700 1 --2- 172.21.15.144:0/2455810598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad300ff390 0x7fad300ff760 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.190+0000 7fad36126700 1 -- 172.21.15.144:0/2455810598 >> 172.21.15.144:0/2455810598 conn(0x7fad30074f50 msgr2=0x7fad30075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:47.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.191+0000 7fad36126700 1 -- 172.21.15.144:0/2455810598 shutdown_connections 2024-09-10T11:35:47.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.191+0000 7fad36126700 1 -- 172.21.15.144:0/2455810598 wait complete. 2024-09-10T11:35:47.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.191+0000 7fad36126700 1 Processor -- start 2024-09-10T11:35:47.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.191+0000 7fad36126700 1 -- start start 2024-09-10T11:35:47.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad36126700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad300ff390 0x7fad30195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:47.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad36126700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad300ffd30 0x7fad30196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:47.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad36126700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad30196950 con 0x7fad300ff390 2024-09-10T11:35:47.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad36126700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3019a6e0 con 0x7fad300ffd30 2024-09-10T11:35:47.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad2f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad300ff390 0x7fad30195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:47.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad2f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad300ff390 0x7fad30195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53854/0 (socket says 172.21.15.144:53854) 2024-09-10T11:35:47.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad2f7fe700 1 -- 172.21.15.144:0/3474919078 learned_addr learned my addr 172.21.15.144:0/3474919078 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:35:47.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad2effd700 1 --2- 172.21.15.144:0/3474919078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad300ffd30 0x7fad30196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:47.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad2f7fe700 1 -- 172.21.15.144:0/3474919078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad300ffd30 msgr2=0x7fad30196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:47.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad2f7fe700 1 --2- 172.21.15.144:0/3474919078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad300ffd30 0x7fad30196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad2f7fe700 1 -- 172.21.15.144:0/3474919078 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad200095f0 con 0x7fad300ff390 2024-09-10T11:35:47.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad2f7fe700 1 --2- 172.21.15.144:0/3474919078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad300ff390 0x7fad30195d30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fad20005f50 tx=0x7fad2000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:47.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad2cff9700 1 -- 172.21.15.144:0/3474919078 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad2001d070 con 0x7fad300ff390 2024-09-10T11:35:47.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad36126700 1 -- 172.21.15.144:0/3474919078 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3019a960 con 0x7fad300ff390 2024-09-10T11:35:47.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad2cff9700 1 -- 172.21.15.144:0/3474919078 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad2000b610 con 0x7fad300ff390 2024-09-10T11:35:47.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.192+0000 7fad2cff9700 1 -- 172.21.15.144:0/3474919078 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad20022470 con 0x7fad300ff390 2024-09-10T11:35:47.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.193+0000 7fad36126700 1 -- 172.21.15.144:0/3474919078 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3019ae50 con 0x7fad300ff390 2024-09-10T11:35:47.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.194+0000 7fad36126700 1 -- 172.21.15.144:0/3474919078 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad30108660 con 0x7fad300ff390 2024-09-10T11:35:47.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.195+0000 7fad2cff9700 1 -- 172.21.15.144:0/3474919078 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fad20003780 con 0x7fad300ff390 2024-09-10T11:35:47.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.196+0000 7fad2cff9700 1 --2- 172.21.15.144:0/3474919078 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fad1806e250 0x7fad18070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:47.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.196+0000 7fad2cff9700 1 -- 172.21.15.144:0/3474919078 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fad2008f390 con 0x7fad300ff390 2024-09-10T11:35:47.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.196+0000 7fad2effd700 1 --2- 172.21.15.144:0/3474919078 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fad1806e250 0x7fad18070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:47.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.196+0000 7fad2effd700 1 --2- 172.21.15.144:0/3474919078 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fad1806e250 0x7fad18070700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fad30197350 tx=0x7fad24009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:47.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.200+0000 7fad2cff9700 1 -- 172.21.15.144:0/3474919078 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fad20058800 con 0x7fad300ff390 2024-09-10T11:35:47.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.485+0000 7fad36126700 1 -- 172.21.15.144:0/3474919078 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fad3004ea50 con 0x7fad300ff390 2024-09-10T11:35:47.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.486+0000 7fad2cff9700 1 -- 172.21.15.144:0/3474919078 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1960 (secure 0 0 0) 0x7fad2005bec0 con 0x7fad300ff390 2024-09-10T11:35:47.487 INFO:teuthology.orchestra.run.smithi144.stdout:e23 2024-09-10T11:35:47.488 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:33:11:119769+0000 2024-09-10T11:35:47.488 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:35:47.488 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:35:47.488 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:35:47.488 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:35:47.488 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:35:47.488 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:35:47.488 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 23 2024-09-10T11:35:47.488 INFO:teuthology.orchestra.run.smithi144.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T11:35:47.488 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:35:47.488 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:33:11.119766+0000 2024-09-10T11:35:47.488 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:35:47.488 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:35:47.489 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:35:47.489 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:35:47.489 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:35:47.489 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:35:47.489 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:35:47.489 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:35:47.489 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 47 2024-09-10T11:35:47.489 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:35:47.489 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:35:47.489 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:35:47.489 INFO:teuthology.orchestra.run.smithi144.stdout:up {0=14488,1=24267} 2024-09-10T11:35:47.490 INFO:teuthology.orchestra.run.smithi144.stdout:failed 2024-09-10T11:35:47.490 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:35:47.490 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:35:47.490 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:35:47.490 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:35:47.490 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:35:47.490 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:35:47.490 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:35:47.490 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:35:47.490 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 14488 members: 14488,24267 2024-09-10T11:35:47.490 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{0:14488} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:35:47.491 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{0:14498} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/1752150537,v1:172.21.15.144:6827/1752150537] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:35:47.491 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:35:47.491 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:35:47.491 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:35:47.491 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:35:47.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.489+0000 7fad36126700 1 -- 172.21.15.144:0/3474919078 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fad1806e250 msgr2=0x7fad18070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:47.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.489+0000 7fad36126700 1 --2- 172.21.15.144:0/3474919078 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fad1806e250 0x7fad18070700 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fad30197350 tx=0x7fad24009260 comp rx=0 tx=0).stop 2024-09-10T11:35:47.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.489+0000 7fad36126700 1 -- 172.21.15.144:0/3474919078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad300ff390 msgr2=0x7fad30195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:47.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.489+0000 7fad36126700 1 --2- 172.21.15.144:0/3474919078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad300ff390 0x7fad30195d30 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fad20005f50 tx=0x7fad2000f690 comp rx=0 tx=0).stop 2024-09-10T11:35:47.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.489+0000 7fad36126700 1 -- 172.21.15.144:0/3474919078 shutdown_connections 2024-09-10T11:35:47.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.489+0000 7fad36126700 1 --2- 172.21.15.144:0/3474919078 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fad1806e250 0x7fad18070700 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.489+0000 7fad36126700 1 --2- 172.21.15.144:0/3474919078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad300ff390 0x7fad30195d30 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.489+0000 7fad36126700 1 --2- 172.21.15.144:0/3474919078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad300ffd30 0x7fad30196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.489+0000 7fad36126700 1 -- 172.21.15.144:0/3474919078 >> 172.21.15.144:0/3474919078 conn(0x7fad30074f50 msgr2=0x7fad30068630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:47.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.489+0000 7fad36126700 1 -- 172.21.15.144:0/3474919078 shutdown_connections 2024-09-10T11:35:47.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.489+0000 7fad36126700 1 -- 172.21.15.144:0/3474919078 wait complete. 2024-09-10T11:35:47.493 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 23 2024-09-10T11:35:47.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.611+0000 7f786425a700 1 -- 172.21.15.144:0/215189731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f785c100d40 msgr2=0x7f785c1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:47.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.611+0000 7f786425a700 1 --2- 172.21.15.144:0/215189731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f785c100d40 0x7f785c1094a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f784c009910 tx=0x7f784c009c20 comp rx=0 tx=0).stop 2024-09-10T11:35:47.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.611+0000 7f786425a700 1 -- 172.21.15.144:0/215189731 shutdown_connections 2024-09-10T11:35:47.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.611+0000 7f786425a700 1 --2- 172.21.15.144:0/215189731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f785c100d40 0x7f785c1094a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.611+0000 7f786425a700 1 --2- 172.21.15.144:0/215189731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f785c100430 0x7f785c100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.611+0000 7f786425a700 1 -- 172.21.15.144:0/215189731 >> 172.21.15.144:0/215189731 conn(0x7f785c0748b0 msgr2=0x7f785c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:47.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.611+0000 7f786425a700 1 -- 172.21.15.144:0/215189731 shutdown_connections 2024-09-10T11:35:47.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.611+0000 7f786425a700 1 -- 172.21.15.144:0/215189731 wait complete. 2024-09-10T11:35:47.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.612+0000 7f786425a700 1 Processor -- start 2024-09-10T11:35:47.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.612+0000 7f786425a700 1 -- start start 2024-09-10T11:35:47.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.612+0000 7f786425a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f785c100430 0x7f785c195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:47.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.612+0000 7f786425a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f785c100d40 0x7f785c1961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:47.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.612+0000 7f786425a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c196870 con 0x7f785c100430 2024-09-10T11:35:47.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.612+0000 7f786425a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f785c1995a0 con 0x7f785c100d40 2024-09-10T11:35:47.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.613+0000 7f7861ff6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f785c100430 0x7f785c195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:47.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.613+0000 7f7861ff6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f785c100430 0x7f785c195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53874/0 (socket says 172.21.15.144:53874) 2024-09-10T11:35:47.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.613+0000 7f7861ff6700 1 -- 172.21.15.144:0/2726686205 learned_addr learned my addr 172.21.15.144:0/2726686205 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:35:47.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.613+0000 7f78617f5700 1 --2- 172.21.15.144:0/2726686205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f785c100d40 0x7f785c1961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:47.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.613+0000 7f78617f5700 1 -- 172.21.15.144:0/2726686205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f785c100430 msgr2=0x7f785c195ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:47.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.613+0000 7f78617f5700 1 --2- 172.21.15.144:0/2726686205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f785c100430 0x7f785c195ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.613+0000 7f78617f5700 1 -- 172.21.15.144:0/2726686205 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f784c0095f0 con 0x7f785c100d40 2024-09-10T11:35:47.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.613+0000 7f78617f5700 1 --2- 172.21.15.144:0/2726686205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f785c100d40 0x7f785c1961e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f784c009fd0 tx=0x7f784c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:47.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.614+0000 7f7852ffd700 1 -- 172.21.15.144:0/2726686205 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f784c01d070 con 0x7f785c100d40 2024-09-10T11:35:47.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.614+0000 7f786425a700 1 -- 172.21.15.144:0/2726686205 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785c199820 con 0x7f785c100d40 2024-09-10T11:35:47.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.614+0000 7f7852ffd700 1 -- 172.21.15.144:0/2726686205 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f784c0050d0 con 0x7f785c100d40 2024-09-10T11:35:47.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.614+0000 7f786425a700 1 -- 172.21.15.144:0/2726686205 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f785c199df0 con 0x7f785c100d40 2024-09-10T11:35:47.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.614+0000 7f7852ffd700 1 -- 172.21.15.144:0/2726686205 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f784c003e90 con 0x7f785c100d40 2024-09-10T11:35:47.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.615+0000 7f786425a700 1 -- 172.21.15.144:0/2726686205 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f785c107e10 con 0x7f785c100d40 2024-09-10T11:35:47.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.615+0000 7f7852ffd700 1 -- 172.21.15.144:0/2726686205 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f784c003680 con 0x7f785c100d40 2024-09-10T11:35:47.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.616+0000 7f7852ffd700 1 --2- 172.21.15.144:0/2726686205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f784806e200 0x7f78480706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:47.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.616+0000 7f7852ffd700 1 -- 172.21.15.144:0/2726686205 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f784c08f510 con 0x7f785c100d40 2024-09-10T11:35:47.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.616+0000 7f7861ff6700 1 --2- 172.21.15.144:0/2726686205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f784806e200 0x7f78480706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:47.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.616+0000 7f7861ff6700 1 --2- 172.21.15.144:0/2726686205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f784806e200 0x7f78480706b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7858009d60 tx=0x7f7858009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:47.622 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.621+0000 7f7852ffd700 1 -- 172.21.15.144:0/2726686205 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f784c058900 con 0x7f785c100d40 2024-09-10T11:35:47.864 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.863+0000 7f786425a700 1 -- 172.21.15.144:0/2726686205 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f785c197030 con 0x7f784806e200 2024-09-10T11:35:47.865 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:47 smithi144 ceph-mon[87436]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:47.865 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:47 smithi144 ceph-mon[87436]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:47.865 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:47 smithi144 ceph-mon[87436]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:47.865 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:47 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/665875363' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:47.865 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:47 smithi144 ceph-mon[87436]: Health check update: Degraded data redundancy: 45/360 objects degraded (12.500%), 12 pgs degraded (PG_DEGRADED) 2024-09-10T11:35:47.865 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:47 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3474919078' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:35:47.865 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.864+0000 7f7852ffd700 1 -- 172.21.15.144:0/2726686205 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f785c197030 con 0x7f784806e200 2024-09-10T11:35:47.866 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:35:47.866 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:35:47.866 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:35:47.866 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:35:47.866 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:35:47.866 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:35:47.866 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:35:47.866 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:35:47.866 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:35:47.866 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "8/23 daemons upgraded", 2024-09-10T11:35:47.866 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading osd daemons", 2024-09-10T11:35:47.867 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:35:47.867 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:35:47.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.867+0000 7f786425a700 1 -- 172.21.15.144:0/2726686205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f784806e200 msgr2=0x7f78480706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:47.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.867+0000 7f786425a700 1 --2- 172.21.15.144:0/2726686205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f784806e200 0x7f78480706b0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7858009d60 tx=0x7f7858009260 comp rx=0 tx=0).stop 2024-09-10T11:35:47.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.867+0000 7f786425a700 1 -- 172.21.15.144:0/2726686205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f785c100d40 msgr2=0x7f785c1961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:47.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.867+0000 7f786425a700 1 --2- 172.21.15.144:0/2726686205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f785c100d40 0x7f785c1961e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f784c009fd0 tx=0x7f784c00f690 comp rx=0 tx=0).stop 2024-09-10T11:35:47.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.867+0000 7f786425a700 1 -- 172.21.15.144:0/2726686205 shutdown_connections 2024-09-10T11:35:47.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.867+0000 7f786425a700 1 --2- 172.21.15.144:0/2726686205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f784806e200 0x7f78480706b0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.867+0000 7f786425a700 1 --2- 172.21.15.144:0/2726686205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f785c100430 0x7f785c195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.867+0000 7f786425a700 1 --2- 172.21.15.144:0/2726686205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f785c100d40 0x7f785c1961e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.867+0000 7f786425a700 1 -- 172.21.15.144:0/2726686205 >> 172.21.15.144:0/2726686205 conn(0x7f785c0748b0 msgr2=0x7f785c0ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:47.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.867+0000 7f786425a700 1 -- 172.21.15.144:0/2726686205 shutdown_connections 2024-09-10T11:35:47.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.867+0000 7f786425a700 1 -- 172.21.15.144:0/2726686205 wait complete. 2024-09-10T11:35:47.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.978+0000 7fa492583700 1 -- 172.21.15.144:0/41116205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c068910 msgr2=0x7fa48c068d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:47.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.978+0000 7fa492583700 1 --2- 172.21.15.144:0/41116205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c068910 0x7fa48c068d80 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa480009910 tx=0x7fa480009c20 comp rx=0 tx=0).stop 2024-09-10T11:35:47.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.978+0000 7fa492583700 1 -- 172.21.15.144:0/41116205 shutdown_connections 2024-09-10T11:35:47.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.978+0000 7fa492583700 1 --2- 172.21.15.144:0/41116205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c068910 0x7fa48c068d80 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.978+0000 7fa492583700 1 --2- 172.21.15.144:0/41116205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa48c1050c0 0x7fa48c0683d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.978+0000 7fa492583700 1 -- 172.21.15.144:0/41116205 >> 172.21.15.144:0/41116205 conn(0x7fa48c074f50 msgr2=0x7fa48c075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:47.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.978+0000 7fa492583700 1 -- 172.21.15.144:0/41116205 shutdown_connections 2024-09-10T11:35:47.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.978+0000 7fa492583700 1 -- 172.21.15.144:0/41116205 wait complete. 2024-09-10T11:35:47.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.979+0000 7fa492583700 1 Processor -- start 2024-09-10T11:35:47.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.979+0000 7fa492583700 1 -- start start 2024-09-10T11:35:47.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.979+0000 7fa492583700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c068910 0x7fa48c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:47.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.979+0000 7fa492583700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa48c1050c0 0x7fa48c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:47.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.979+0000 7fa492583700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c19cf20 con 0x7fa48c068910 2024-09-10T11:35:47.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.979+0000 7fa492583700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa48c1a0cb0 con 0x7fa48c1050c0 2024-09-10T11:35:47.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.979+0000 7fa48bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c068910 0x7fa48c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:47.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.979+0000 7fa48bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c068910 0x7fa48c19c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53890/0 (socket says 172.21.15.144:53890) 2024-09-10T11:35:47.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.979+0000 7fa48bfff700 1 -- 172.21.15.144:0/1081457419 learned_addr learned my addr 172.21.15.144:0/1081457419 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:35:47.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.979+0000 7fa48b7fe700 1 --2- 172.21.15.144:0/1081457419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa48c1050c0 0x7fa48c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:47.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.979+0000 7fa48bfff700 1 -- 172.21.15.144:0/1081457419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa48c1050c0 msgr2=0x7fa48c19c840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:47.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.980+0000 7fa48bfff700 1 --2- 172.21.15.144:0/1081457419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa48c1050c0 0x7fa48c19c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:47.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.980+0000 7fa48bfff700 1 -- 172.21.15.144:0/1081457419 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4800095f0 con 0x7fa48c068910 2024-09-10T11:35:47.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.980+0000 7fa48bfff700 1 --2- 172.21.15.144:0/1081457419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c068910 0x7fa48c19c300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00d900 tx=0x7fa47c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:47.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.980+0000 7fa4897fa700 1 -- 172.21.15.144:0/1081457419 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa47c004a00 con 0x7fa48c068910 2024-09-10T11:35:47.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.980+0000 7fa4897fa700 1 -- 172.21.15.144:0/1081457419 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa47c0054e0 con 0x7fa48c068910 2024-09-10T11:35:47.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.980+0000 7fa492583700 1 -- 172.21.15.144:0/1081457419 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa48c1a0f90 con 0x7fa48c068910 2024-09-10T11:35:47.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.980+0000 7fa4897fa700 1 -- 172.21.15.144:0/1081457419 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa47c009d60 con 0x7fa48c068910 2024-09-10T11:35:47.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.980+0000 7fa492583700 1 -- 172.21.15.144:0/1081457419 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48c1a14e0 con 0x7fa48c068910 2024-09-10T11:35:47.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.981+0000 7fa492583700 1 -- 172.21.15.144:0/1081457419 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa48c10f700 con 0x7fa48c068910 2024-09-10T11:35:47.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.982+0000 7fa4897fa700 1 -- 172.21.15.144:0/1081457419 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa47c005020 con 0x7fa48c068910 2024-09-10T11:35:47.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.982+0000 7fa4897fa700 1 --2- 172.21.15.144:0/1081457419 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa47406e200 0x7fa4740706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:35:47.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.982+0000 7fa4897fa700 1 -- 172.21.15.144:0/1081457419 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa47c08e140 con 0x7fa48c068910 2024-09-10T11:35:47.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.982+0000 7fa48b7fe700 1 --2- 172.21.15.144:0/1081457419 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa47406e200 0x7fa4740706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:35:47.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.983+0000 7fa48b7fe700 1 --2- 172.21.15.144:0/1081457419 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa47406e200 0x7fa4740706b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa48000b5c0 tx=0x7fa480005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:35:47.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:47.987+0000 7fa4897fa700 1 -- 172.21.15.144:0/1081457419 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa47c057650 con 0x7fa48c068910 2024-09-10T11:35:48.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:47 smithi178 ceph-mon[65940]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:48.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:47 smithi178 ceph-mon[65940]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:48.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:47 smithi178 ceph-mon[65940]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:48.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:47 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/665875363' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:48.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:47 smithi178 ceph-mon[65940]: Health check update: Degraded data redundancy: 45/360 objects degraded (12.500%), 12 pgs degraded (PG_DEGRADED) 2024-09-10T11:35:48.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:47 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3474919078' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:35:48.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:48.289+0000 7fa492583700 1 -- 172.21.15.144:0/1081457419 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa48c04ea50 con 0x7fa48c068910 2024-09-10T11:35:48.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:48.290+0000 7fa4897fa700 1 -- 172.21.15.144:0/1081457419 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1131 (secure 0 0 0) 0x7fa47c05ad10 con 0x7fa48c068910 2024-09-10T11:35:48.291 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 45/360 objects degraded (12.500%), 12 pgs degraded 2024-09-10T11:35:48.291 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:35:48.291 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:35:48.291 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-10T11:35:48.291 INFO:teuthology.orchestra.run.smithi144.stdout: osd.2 (root=default,host=smithi144) is down 2024-09-10T11:35:48.292 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 45/360 objects degraded (12.500%), 12 pgs degraded 2024-09-10T11:35:48.292 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1 is active+undersized+degraded, acting [1,0] 2024-09-10T11:35:48.292 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.3 is active+undersized+degraded, acting [5,1] 2024-09-10T11:35:48.292 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.7 is active+undersized+degraded, acting [3,4] 2024-09-10T11:35:48.292 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.d is active+undersized+degraded, acting [1,3] 2024-09-10T11:35:48.292 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.e is active+undersized+degraded, acting [0,3] 2024-09-10T11:35:48.292 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.10 is active+undersized+degraded, acting [1,0] 2024-09-10T11:35:48.292 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.13 is active+undersized+degraded, acting [0,4] 2024-09-10T11:35:48.292 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.16 is active+undersized+degraded, acting [5,3] 2024-09-10T11:35:48.292 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.17 is active+undersized+degraded, acting [5,1] 2024-09-10T11:35:48.292 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.19 is active+undersized+degraded, acting [0,4] 2024-09-10T11:35:48.292 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1c is active+undersized+degraded, acting [4,5] 2024-09-10T11:35:48.292 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1e is active+undersized+degraded, acting [0,5] 2024-09-10T11:35:48.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:48.292+0000 7fa492583700 1 -- 172.21.15.144:0/1081457419 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa47406e200 msgr2=0x7fa4740706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:48.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:48.292+0000 7fa492583700 1 --2- 172.21.15.144:0/1081457419 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa47406e200 0x7fa4740706b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa48000b5c0 tx=0x7fa480005960 comp rx=0 tx=0).stop 2024-09-10T11:35:48.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:48.293+0000 7fa492583700 1 -- 172.21.15.144:0/1081457419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c068910 msgr2=0x7fa48c19c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:35:48.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:48.293+0000 7fa492583700 1 --2- 172.21.15.144:0/1081457419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c068910 0x7fa48c19c300 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa47c00d900 tx=0x7fa47c00dcc0 comp rx=0 tx=0).stop 2024-09-10T11:35:48.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:48.293+0000 7fa492583700 1 -- 172.21.15.144:0/1081457419 shutdown_connections 2024-09-10T11:35:48.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:48.293+0000 7fa492583700 1 --2- 172.21.15.144:0/1081457419 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa47406e200 0x7fa4740706b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:48.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:48.293+0000 7fa492583700 1 --2- 172.21.15.144:0/1081457419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa48c068910 0x7fa48c19c300 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:48.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:48.293+0000 7fa492583700 1 --2- 172.21.15.144:0/1081457419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa48c1050c0 0x7fa48c19c840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:35:48.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:48.293+0000 7fa492583700 1 -- 172.21.15.144:0/1081457419 >> 172.21.15.144:0/1081457419 conn(0x7fa48c074f50 msgr2=0x7fa48c0fccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:35:48.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:48.293+0000 7fa492583700 1 -- 172.21.15.144:0/1081457419 shutdown_connections 2024-09-10T11:35:48.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:35:48.293+0000 7fa492583700 1 -- 172.21.15.144:0/1081457419 wait complete. 2024-09-10T11:35:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:48 smithi144 ceph-mon[87436]: pgmap v92: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/360 objects degraded (12.500%) 2024-09-10T11:35:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:48 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1081457419' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:35:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:48 smithi178 ceph-mon[65940]: pgmap v92: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 45/360 objects degraded (12.500%) 2024-09-10T11:35:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:48 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1081457419' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:35:49.910 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:49 smithi144 ceph-mon[87436]: from='client.44215 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:49.910 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:49 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:49.910 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:49 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:49.910 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:49 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:49.910 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:49 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:49 smithi178 ceph-mon[65940]: from='client.44215 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:35:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:49 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:49 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:49 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:49 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:50.434 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:49 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2[110067]: 2024-09-10T11:35:49.975+0000 7f11ebe43640 -1 osd.2 0 read_superblock omap replica is missing. 2024-09-10T11:35:50.434 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:50 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2[110067]: 2024-09-10T11:35:50.223+0000 7f11ebe43640 -1 osd.2 60 log_to_monitors true 2024-09-10T11:35:50.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:50 smithi144 ceph-mon[87436]: pgmap v93: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 45/360 objects degraded (12.500%) 2024-09-10T11:35:50.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:50 smithi144 ceph-mon[87436]: from='osd.2 [v2:172.21.15.144:6818/2492794127,v1:172.21.15.144:6819/2492794127]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T11:35:51.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:50 smithi178 ceph-mon[65940]: pgmap v93: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 45/360 objects degraded (12.500%) 2024-09-10T11:35:51.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:50 smithi178 ceph-mon[65940]: from='osd.2 [v2:172.21.15.144:6818/2492794127,v1:172.21.15.144:6819/2492794127]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T11:35:51.666 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 11:35:51 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2[110067]: 2024-09-10T11:35:51.470+0000 7f11dfbd5640 -1 osd.2 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:35:51.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:51 smithi144 ceph-mon[87436]: from='osd.2 [v2:172.21.15.144:6818/2492794127,v1:172.21.15.144:6819/2492794127]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T11:35:51.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:51 smithi144 ceph-mon[87436]: osdmap e63: 6 total, 5 up, 6 in 2024-09-10T11:35:51.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:51 smithi144 ceph-mon[87436]: from='osd.2 [v2:172.21.15.144:6818/2492794127,v1:172.21.15.144:6819/2492794127]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T11:35:51.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:51 smithi144 ceph-mon[87436]: from='osd.2 [v2:172.21.15.144:6818/2492794127,v1:172.21.15.144:6819/2492794127]' entity='osd.2' 2024-09-10T11:35:52.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:51 smithi178 ceph-mon[65940]: from='osd.2 [v2:172.21.15.144:6818/2492794127,v1:172.21.15.144:6819/2492794127]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T11:35:52.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:51 smithi178 ceph-mon[65940]: osdmap e63: 6 total, 5 up, 6 in 2024-09-10T11:35:52.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:51 smithi178 ceph-mon[65940]: from='osd.2 [v2:172.21.15.144:6818/2492794127,v1:172.21.15.144:6819/2492794127]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi144", "root=default"]}]: dispatch 2024-09-10T11:35:52.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:51 smithi178 ceph-mon[65940]: from='osd.2 [v2:172.21.15.144:6818/2492794127,v1:172.21.15.144:6819/2492794127]' entity='osd.2' 2024-09-10T11:35:52.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:52 smithi144 ceph-mon[87436]: pgmap v95: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 45/360 objects degraded (12.500%) 2024-09-10T11:35:52.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:52 smithi144 ceph-mon[87436]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T11:35:52.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:52 smithi144 ceph-mon[87436]: osd.2 [v2:172.21.15.144:6818/2492794127,v1:172.21.15.144:6819/2492794127] boot 2024-09-10T11:35:52.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:52 smithi144 ceph-mon[87436]: osdmap e64: 6 total, 6 up, 6 in 2024-09-10T11:35:52.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:35:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:52 smithi178 ceph-mon[65940]: pgmap v95: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s; 45/360 objects degraded (12.500%) 2024-09-10T11:35:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:52 smithi178 ceph-mon[65940]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T11:35:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:52 smithi178 ceph-mon[65940]: osd.2 [v2:172.21.15.144:6818/2492794127,v1:172.21.15.144:6819/2492794127] boot 2024-09-10T11:35:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:52 smithi178 ceph-mon[65940]: osdmap e64: 6 total, 6 up, 6 in 2024-09-10T11:35:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T11:35:54.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:54.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:54.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:35:54.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:35:54.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:54.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:35:54.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:54.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:54.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:54.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-10T11:35:54.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:53 smithi178 ceph-mon[65940]: osdmap e65: 6 total, 6 up, 6 in 2024-09-10T11:35:54.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:54.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:54.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:35:54.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:35:54.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:54.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:35:54.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:54.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:54.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:35:54.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-10T11:35:54.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:53 smithi144 ceph-mon[87436]: osdmap e65: 6 total, 6 up, 6 in 2024-09-10T11:35:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:54 smithi178 ceph-mon[65940]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-10T11:35:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:54 smithi178 ceph-mon[65940]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-10T11:35:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:54 smithi178 ceph-mon[65940]: pgmap v98: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 45/360 objects degraded (12.500%) 2024-09-10T11:35:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:54 smithi144 ceph-mon[87436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-10T11:35:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:54 smithi144 ceph-mon[87436]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-10T11:35:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:54 smithi144 ceph-mon[87436]: pgmap v98: 65 pgs: 15 active+undersized, 12 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s; 45/360 objects degraded (12.500%) 2024-09-10T11:35:56.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:55 smithi178 ceph-mon[65940]: pgmap v99: 65 pgs: 10 active+undersized, 7 active+undersized+degraded, 48 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 28/360 objects degraded (7.778%) 2024-09-10T11:35:56.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:55 smithi178 ceph-mon[65940]: Health check update: Degraded data redundancy: 28/360 objects degraded (7.778%), 7 pgs degraded (PG_DEGRADED) 2024-09-10T11:35:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:55 smithi144 ceph-mon[87436]: pgmap v99: 65 pgs: 10 active+undersized, 7 active+undersized+degraded, 48 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s; 28/360 objects degraded (7.778%) 2024-09-10T11:35:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:55 smithi144 ceph-mon[87436]: Health check update: Degraded data redundancy: 28/360 objects degraded (7.778%), 7 pgs degraded (PG_DEGRADED) 2024-09-10T11:35:57.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:57 smithi144 ceph-mon[87436]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/360 objects degraded (7.778%), 7 pgs degraded) 2024-09-10T11:35:58.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:57 smithi178 ceph-mon[65940]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/360 objects degraded (7.778%), 7 pgs degraded) 2024-09-10T11:35:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:58 smithi178 ceph-mon[65940]: pgmap v100: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T11:35:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:35:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:35:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:58 smithi144 ceph-mon[87436]: pgmap v100: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-10T11:35:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:35:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:35:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:36:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:00 smithi178 ceph-mon[65940]: pgmap v101: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-09-10T11:36:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:00 smithi144 ceph-mon[87436]: pgmap v101: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-09-10T11:36:02.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:02 smithi144 ceph-mon[87436]: pgmap v102: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T11:36:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:02 smithi178 ceph-mon[65940]: pgmap v102: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T11:36:04.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:04 smithi144 ceph-mon[87436]: pgmap v103: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-10T11:36:04.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:04 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:04 smithi178 ceph-mon[65940]: pgmap v103: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-10T11:36:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:04 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:06 smithi144 ceph-mon[87436]: pgmap v104: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-10T11:36:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:06 smithi178 ceph-mon[65940]: pgmap v104: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-10T11:36:08.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:08 smithi178 ceph-mon[65940]: pgmap v105: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:36:08.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:08 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-10T11:36:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:08 smithi144 ceph-mon[87436]: pgmap v105: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:36:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:08 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-10T11:36:09.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:09 smithi178 ceph-mon[65940]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-10T11:36:09.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:09 smithi178 ceph-mon[65940]: Upgrade: osd.3 is safe to restart 2024-09-10T11:36:09.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:09 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:09.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:09 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T11:36:09.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:09 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:36:09.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:09 smithi144 ceph-mon[87436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-09-10T11:36:09.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:09 smithi144 ceph-mon[87436]: Upgrade: osd.3 is safe to restart 2024-09-10T11:36:09.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:09 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:09.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:09 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T11:36:09.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:09 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:36:10.898 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:10 smithi178 ceph-mon[65940]: Upgrade: Updating osd.3 2024-09-10T11:36:10.898 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:10 smithi178 ceph-mon[65940]: Deploying daemon osd.3 on smithi178 2024-09-10T11:36:10.898 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:10 smithi178 ceph-mon[65940]: pgmap v106: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:36:10.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:10 smithi144 ceph-mon[87436]: Upgrade: Updating osd.3 2024-09-10T11:36:10.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:10 smithi144 ceph-mon[87436]: Deploying daemon osd.3 on smithi178 2024-09-10T11:36:10.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:10 smithi144 ceph-mon[87436]: pgmap v106: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T11:36:12.686 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:12 smithi178 ceph-mon[65940]: pgmap v107: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:36:12.686 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:12 smithi178 systemd[1]: Stopping Ceph osd.3 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:36:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:12 smithi144 ceph-mon[87436]: pgmap v107: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:36:13.117 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:12 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3[39525]: 2024-09-10T11:36:12.684+0000 7f22e7daf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:36:13.117 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:12 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3[39525]: 2024-09-10T11:36:12.684+0000 7f22e7daf700 -1 osd.3 65 *** Got signal Terminated *** 2024-09-10T11:36:13.117 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:12 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3[39525]: 2024-09-10T11:36:12.684+0000 7f22e7daf700 -1 osd.3 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:36:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:13 smithi144 ceph-mon[87436]: osd.3 marked itself down and dead 2024-09-10T11:36:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:36:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:13 smithi178 ceph-mon[65940]: osd.3 marked itself down and dead 2024-09-10T11:36:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:36:14.379 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:14 smithi178 podman[74871]: 2024-09-10 11:36:14.127126659 +0000 UTC m=+1.548049902 container died f681e8a4fafba57db638b34770abb2ccdac714e125625dd0e2a1cf528613b059 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, RELEASE=HEAD, org.label-schema.build-date=20231212, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True) 2024-09-10T11:36:14.650 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:14 smithi178 podman[74871]: 2024-09-10 11:36:14.476569483 +0000 UTC m=+1.897492718 container cleanup f681e8a4fafba57db638b34770abb2ccdac714e125625dd0e2a1cf528613b059 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_BRANCH=HEAD) 2024-09-10T11:36:14.650 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:14 smithi178 bash[74871]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3 2024-09-10T11:36:14.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:14 smithi144 ceph-mon[87436]: pgmap v108: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:36:14.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:14 smithi144 ceph-mon[87436]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T11:36:14.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:14 smithi144 ceph-mon[87436]: osdmap e66: 6 total, 5 up, 6 in 2024-09-10T11:36:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:14 smithi178 ceph-mon[65940]: pgmap v108: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:36:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:14 smithi178 ceph-mon[65940]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T11:36:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:14 smithi178 ceph-mon[65940]: osdmap e66: 6 total, 5 up, 6 in 2024-09-10T11:36:15.117 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:14 smithi178 podman[74889]: 2024-09-10 11:36:14.811100223 +0000 UTC m=+0.680928209 container remove f681e8a4fafba57db638b34770abb2ccdac714e125625dd0e2a1cf528613b059 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-09-10T11:36:15.448 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:15 smithi178 podman[74980]: 2024-09-10 11:36:15.045288781 +0000 UTC m=+0.015741185 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:36:15.448 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:15 smithi178 podman[74980]: 2024-09-10 11:36:15.162006425 +0000 UTC m=+0.132458865 container create 0964e57c353a5da7a46adc329e643a1d1e758875e5bb4530ad1350dd9d415a5d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T11:36:15.448 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:15 smithi178 podman[74980]: 2024-09-10 11:36:15.355961086 +0000 UTC m=+0.326413480 container init 0964e57c353a5da7a46adc329e643a1d1e758875e5bb4530ad1350dd9d415a5d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:36:15.448 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:15 smithi178 podman[74980]: 2024-09-10 11:36:15.359886426 +0000 UTC m=+0.330338826 container start 0964e57c353a5da7a46adc329e643a1d1e758875e5bb4530ad1350dd9d415a5d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T11:36:15.448 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:15 smithi178 podman[74980]: 2024-09-10 11:36:15.447592232 +0000 UTC m=+0.418044640 container attach 0964e57c353a5da7a46adc329e643a1d1e758875e5bb4530ad1350dd9d415a5d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T11:36:15.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:15 smithi178 ceph-mon[65940]: osdmap e67: 6 total, 5 up, 6 in 2024-09-10T11:36:15.867 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:15 smithi178 podman[74980]: 2024-09-10 11:36:15.545585271 +0000 UTC m=+0.516037677 container died 0964e57c353a5da7a46adc329e643a1d1e758875e5bb4530ad1350dd9d415a5d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True) 2024-09-10T11:36:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:15 smithi144 ceph-mon[87436]: osdmap e67: 6 total, 5 up, 6 in 2024-09-10T11:36:16.608 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:16 smithi178 podman[75002]: 2024-09-10 11:36:16.168009704 +0000 UTC m=+0.612340395 container remove 0964e57c353a5da7a46adc329e643a1d1e758875e5bb4530ad1350dd9d415a5d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0) 2024-09-10T11:36:16.609 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:16 smithi178 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.3.service: Deactivated successfully. 2024-09-10T11:36:16.609 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:16 smithi178 systemd[1]: Stopped Ceph osd.3 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:36:16.609 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:16 smithi178 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.3.service: Consumed 1min 6.269s CPU time. 2024-09-10T11:36:16.609 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:16 smithi178 systemd[1]: Starting Ceph osd.3 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:36:16.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:16 smithi178 ceph-mon[65940]: pgmap v111: 65 pgs: 18 peering, 6 stale+active+clean, 41 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:36:16.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:16 smithi178 ceph-mon[65940]: Health check failed: Reduced data availability: 10 pgs peering (PG_AVAILABILITY) 2024-09-10T11:36:16.867 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:16 smithi178 podman[75101]: 2024-09-10 11:36:16.508131302 +0000 UTC m=+0.015986954 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:36:16.867 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:16 smithi178 podman[75101]: 2024-09-10 11:36:16.636760693 +0000 UTC m=+0.144616334 container create 301cf35da71c01f6295a2d7a2237ba9de34578134614262b7550a03c060bf8e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-activate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T11:36:16.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:16 smithi144 ceph-mon[87436]: pgmap v111: 65 pgs: 18 peering, 6 stale+active+clean, 41 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:36:16.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:16 smithi144 ceph-mon[87436]: Health check failed: Reduced data availability: 10 pgs peering (PG_AVAILABILITY) 2024-09-10T11:36:17.367 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:16 smithi178 podman[75101]: 2024-09-10 11:36:16.921615085 +0000 UTC m=+0.429470732 container init 301cf35da71c01f6295a2d7a2237ba9de34578134614262b7550a03c060bf8e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-activate, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T11:36:17.367 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:16 smithi178 podman[75101]: 2024-09-10 11:36:16.92516381 +0000 UTC m=+0.433019452 container start 301cf35da71c01f6295a2d7a2237ba9de34578134614262b7550a03c060bf8e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-activate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True) 2024-09-10T11:36:17.367 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:16 smithi178 podman[75101]: 2024-09-10 11:36:16.996494364 +0000 UTC m=+0.504350020 container attach 301cf35da71c01f6295a2d7a2237ba9de34578134614262b7550a03c060bf8e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-activate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True) 2024-09-10T11:36:17.367 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:17 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-activate[75117]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:36:17.367 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:17 smithi178 bash[75101]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:36:17.367 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:17 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-activate[75117]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:36:17.367 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:17 smithi178 bash[75101]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:36:18.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:17 smithi178 ceph-mon[65940]: Health check failed: Degraded data redundancy: 31/360 objects degraded (8.611%), 10 pgs degraded (PG_DEGRADED) 2024-09-10T11:36:18.074 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:17 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-activate[75117]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T11:36:18.074 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:17 smithi178 bash[75101]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T11:36:18.074 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:17 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-activate[75117]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-10T11:36:18.074 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:17 smithi178 bash[75101]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-09-10T11:36:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:17 smithi144 ceph-mon[87436]: Health check failed: Degraded data redundancy: 31/360 objects degraded (8.611%), 10 pgs degraded (PG_DEGRADED) 2024-09-10T11:36:18.367 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:18 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-activate[75117]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-10T11:36:18.367 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:18 smithi178 bash[75101]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-09-10T11:36:18.367 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:18 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-activate[75117]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-10T11:36:18.367 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:18 smithi178 bash[75101]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-09-10T11:36:18.367 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:18 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-activate[75117]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-09-10T11:36:18.367 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:18 smithi178 bash[75101]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-09-10T11:36:18.368 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:18 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-activate[75117]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T11:36:18.368 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:18 smithi178 bash[75101]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-10T11:36:18.368 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:18 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-activate[75117]: --> ceph-volume raw activate successful for osd ID: 3 2024-09-10T11:36:18.368 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:18 smithi178 bash[75101]: --> ceph-volume raw activate successful for osd ID: 3 2024-09-10T11:36:18.368 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:18 smithi178 podman[75101]: 2024-09-10 11:36:18.103184829 +0000 UTC m=+1.611040478 container died 301cf35da71c01f6295a2d7a2237ba9de34578134614262b7550a03c060bf8e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:36:18.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.407+0000 7fbc33980700 1 -- 172.21.15.144:0/4082282011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c0ffc70 msgr2=0x7fbc2c103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.407+0000 7fbc33980700 1 --2- 172.21.15.144:0/4082282011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c0ffc70 0x7fbc2c103c40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbc28009910 tx=0x7fbc28009c20 comp rx=0 tx=0).stop 2024-09-10T11:36:18.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.407+0000 7fbc33980700 1 -- 172.21.15.144:0/4082282011 shutdown_connections 2024-09-10T11:36:18.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.407+0000 7fbc33980700 1 --2- 172.21.15.144:0/4082282011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c0ffc70 0x7fbc2c103c40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.407+0000 7fbc33980700 1 --2- 172.21.15.144:0/4082282011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc2c0ff2d0 0x7fbc2c0ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.407+0000 7fbc33980700 1 -- 172.21.15.144:0/4082282011 >> 172.21.15.144:0/4082282011 conn(0x7fbc2c0fabb0 msgr2=0x7fbc2c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:18.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.408+0000 7fbc33980700 1 -- 172.21.15.144:0/4082282011 shutdown_connections 2024-09-10T11:36:18.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.408+0000 7fbc33980700 1 -- 172.21.15.144:0/4082282011 wait complete. 2024-09-10T11:36:18.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.408+0000 7fbc33980700 1 Processor -- start 2024-09-10T11:36:18.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.408+0000 7fbc33980700 1 -- start start 2024-09-10T11:36:18.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.408+0000 7fbc33980700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc2c0ff2d0 0x7fbc2c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.408+0000 7fbc33980700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c0ffc70 0x7fbc2c198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.408+0000 7fbc33980700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c198c10 con 0x7fbc2c0ffc70 2024-09-10T11:36:18.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.408+0000 7fbc33980700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc2c19c9a0 con 0x7fbc2c0ff2d0 2024-09-10T11:36:18.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.409+0000 7fbc30f1b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c0ffc70 0x7fbc2c198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.409+0000 7fbc30f1b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c0ffc70 0x7fbc2c198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41972/0 (socket says 172.21.15.144:41972) 2024-09-10T11:36:18.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.409+0000 7fbc3171c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc2c0ff2d0 0x7fbc2c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.409+0000 7fbc30f1b700 1 -- 172.21.15.144:0/2310105512 learned_addr learned my addr 172.21.15.144:0/2310105512 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:36:18.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.409+0000 7fbc30f1b700 1 -- 172.21.15.144:0/2310105512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc2c0ff2d0 msgr2=0x7fbc2c197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.409+0000 7fbc30f1b700 1 --2- 172.21.15.144:0/2310105512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc2c0ff2d0 0x7fbc2c197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.409+0000 7fbc30f1b700 1 -- 172.21.15.144:0/2310105512 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc280095f0 con 0x7fbc2c0ffc70 2024-09-10T11:36:18.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.409+0000 7fbc3171c700 1 --2- 172.21.15.144:0/2310105512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc2c0ff2d0 0x7fbc2c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:36:18.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.409+0000 7fbc30f1b700 1 --2- 172.21.15.144:0/2310105512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c0ffc70 0x7fbc2c198530 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbc280059d0 tx=0x7fbc2800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:18.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.409+0000 7fbc227fc700 1 -- 172.21.15.144:0/2310105512 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc2801d070 con 0x7fbc2c0ffc70 2024-09-10T11:36:18.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.409+0000 7fbc227fc700 1 -- 172.21.15.144:0/2310105512 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc2800b610 con 0x7fbc2c0ffc70 2024-09-10T11:36:18.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.409+0000 7fbc33980700 1 -- 172.21.15.144:0/2310105512 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc2c19cc20 con 0x7fbc2c0ffc70 2024-09-10T11:36:18.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.409+0000 7fbc227fc700 1 -- 172.21.15.144:0/2310105512 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc28022470 con 0x7fbc2c0ffc70 2024-09-10T11:36:18.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.409+0000 7fbc33980700 1 -- 172.21.15.144:0/2310105512 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc2c19d110 con 0x7fbc2c0ffc70 2024-09-10T11:36:18.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.410+0000 7fbc33980700 1 -- 172.21.15.144:0/2310105512 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc2c04ea50 con 0x7fbc2c0ffc70 2024-09-10T11:36:18.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.411+0000 7fbc227fc700 1 -- 172.21.15.144:0/2310105512 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fbc28003710 con 0x7fbc2c0ffc70 2024-09-10T11:36:18.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.411+0000 7fbc227fc700 1 --2- 172.21.15.144:0/2310105512 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc1806e200 0x7fbc180706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.411+0000 7fbc227fc700 1 -- 172.21.15.144:0/2310105512 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbc2808f3f0 con 0x7fbc2c0ffc70 2024-09-10T11:36:18.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.411+0000 7fbc3171c700 1 --2- 172.21.15.144:0/2310105512 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc1806e200 0x7fbc180706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.412+0000 7fbc3171c700 1 --2- 172.21.15.144:0/2310105512 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc1806e200 0x7fbc180706b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c011490 tx=0x7fbc1c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:18.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.416+0000 7fbc227fc700 1 -- 172.21.15.144:0/2310105512 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbc28058900 con 0x7fbc2c0ffc70 2024-09-10T11:36:18.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.637+0000 7fbc33980700 1 -- 172.21.15.144:0/2310105512 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc2c19d6c0 con 0x7fbc1806e200 2024-09-10T11:36:18.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.638+0000 7fbc227fc700 1 -- 172.21.15.144:0/2310105512 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fbc2c19d6c0 con 0x7fbc1806e200 2024-09-10T11:36:18.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.640+0000 7fbc33980700 1 -- 172.21.15.144:0/2310105512 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc1806e200 msgr2=0x7fbc180706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.640+0000 7fbc33980700 1 --2- 172.21.15.144:0/2310105512 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc1806e200 0x7fbc180706b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fbc1c011490 tx=0x7fbc1c009260 comp rx=0 tx=0).stop 2024-09-10T11:36:18.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.640+0000 7fbc33980700 1 -- 172.21.15.144:0/2310105512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c0ffc70 msgr2=0x7fbc2c198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.640+0000 7fbc33980700 1 --2- 172.21.15.144:0/2310105512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c0ffc70 0x7fbc2c198530 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbc280059d0 tx=0x7fbc2800f690 comp rx=0 tx=0).stop 2024-09-10T11:36:18.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.640+0000 7fbc33980700 1 -- 172.21.15.144:0/2310105512 shutdown_connections 2024-09-10T11:36:18.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.640+0000 7fbc33980700 1 --2- 172.21.15.144:0/2310105512 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc1806e200 0x7fbc180706b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.640+0000 7fbc33980700 1 --2- 172.21.15.144:0/2310105512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc2c0ff2d0 0x7fbc2c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.640+0000 7fbc33980700 1 --2- 172.21.15.144:0/2310105512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc2c0ffc70 0x7fbc2c198530 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.640+0000 7fbc33980700 1 -- 172.21.15.144:0/2310105512 >> 172.21.15.144:0/2310105512 conn(0x7fbc2c0fabb0 msgr2=0x7fbc2c102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:18.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.640+0000 7fbc33980700 1 -- 172.21.15.144:0/2310105512 shutdown_connections 2024-09-10T11:36:18.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.640+0000 7fbc33980700 1 -- 172.21.15.144:0/2310105512 wait complete. 2024-09-10T11:36:18.652 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:36:18.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.748+0000 7f9c7e52e700 1 -- 172.21.15.144:0/1359189074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c780ffba0 msgr2=0x7f9c7810a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.748+0000 7f9c7e52e700 1 --2- 172.21.15.144:0/1359189074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c780ffba0 0x7f9c7810a610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009910 tx=0x7f9c60009c20 comp rx=0 tx=0).stop 2024-09-10T11:36:18.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.748+0000 7f9c7e52e700 1 -- 172.21.15.144:0/1359189074 shutdown_connections 2024-09-10T11:36:18.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.748+0000 7f9c7e52e700 1 --2- 172.21.15.144:0/1359189074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c780ffba0 0x7f9c7810a610 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.748+0000 7f9c7e52e700 1 --2- 172.21.15.144:0/1359189074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c780ff290 0x7f9c780ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.748+0000 7f9c7e52e700 1 -- 172.21.15.144:0/1359189074 >> 172.21.15.144:0/1359189074 conn(0x7f9c78074850 msgr2=0x7f9c78074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:18.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.748+0000 7f9c7e52e700 1 -- 172.21.15.144:0/1359189074 shutdown_connections 2024-09-10T11:36:18.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.748+0000 7f9c7e52e700 1 -- 172.21.15.144:0/1359189074 wait complete. 2024-09-10T11:36:18.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.748+0000 7f9c7e52e700 1 Processor -- start 2024-09-10T11:36:18.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c7e52e700 1 -- start start 2024-09-10T11:36:18.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c7e52e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c780ff290 0x7f9c78197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c7e52e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c780ffba0 0x7f9c781984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c7e52e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c78198bb0 con 0x7f9c780ff290 2024-09-10T11:36:18.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c7e52e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c7819c940 con 0x7f9c780ffba0 2024-09-10T11:36:18.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c77fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c780ff290 0x7f9c78197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c77fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c780ff290 0x7f9c78197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41984/0 (socket says 172.21.15.144:41984) 2024-09-10T11:36:18.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c77fff700 1 -- 172.21.15.144:0/3447278952 learned_addr learned my addr 172.21.15.144:0/3447278952 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:36:18.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c6ffff700 1 --2- 172.21.15.144:0/3447278952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c780ffba0 0x7f9c781984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c77fff700 1 -- 172.21.15.144:0/3447278952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c780ffba0 msgr2=0x7f9c781984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c77fff700 1 --2- 172.21.15.144:0/3447278952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c780ffba0 0x7f9c781984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c77fff700 1 -- 172.21.15.144:0/3447278952 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c600095f0 con 0x7f9c780ff290 2024-09-10T11:36:18.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c77fff700 1 --2- 172.21.15.144:0/3447278952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c780ff290 0x7f9c78197f90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800d900 tx=0x7f9c6800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:18.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c75ffb700 1 -- 172.21.15.144:0/3447278952 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c68009900 con 0x7f9c780ff290 2024-09-10T11:36:18.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c7e52e700 1 -- 172.21.15.144:0/3447278952 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c7819cc20 con 0x7f9c780ff290 2024-09-10T11:36:18.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c75ffb700 1 -- 172.21.15.144:0/3447278952 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c68010460 con 0x7f9c780ff290 2024-09-10T11:36:18.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.749+0000 7f9c75ffb700 1 -- 172.21.15.144:0/3447278952 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c6800f5b0 con 0x7f9c780ff290 2024-09-10T11:36:18.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.750+0000 7f9c7e52e700 1 -- 172.21.15.144:0/3447278952 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c7819d170 con 0x7f9c780ff290 2024-09-10T11:36:18.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.750+0000 7f9c7e52e700 1 -- 172.21.15.144:0/3447278952 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c7810a170 con 0x7f9c780ff290 2024-09-10T11:36:18.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.751+0000 7f9c75ffb700 1 -- 172.21.15.144:0/3447278952 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9c680105d0 con 0x7f9c780ff290 2024-09-10T11:36:18.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.751+0000 7f9c75ffb700 1 --2- 172.21.15.144:0/3447278952 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9c6406e200 0x7f9c640706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:18.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.752+0000 7f9c6ffff700 1 --2- 172.21.15.144:0/3447278952 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9c6406e200 0x7f9c640706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:18.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.752+0000 7f9c75ffb700 1 -- 172.21.15.144:0/3447278952 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9c6808da90 con 0x7f9c780ff290 2024-09-10T11:36:18.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.752+0000 7f9c6ffff700 1 --2- 172.21.15.144:0/3447278952 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9c6406e200 0x7f9c640706b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9c781995b0 tx=0x7f9c6001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:18.757 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.756+0000 7f9c75ffb700 1 -- 172.21.15.144:0/3447278952 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9c68056fa0 con 0x7f9c780ff290 2024-09-10T11:36:18.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.976+0000 7f9c7e52e700 1 -- 172.21.15.144:0/3447278952 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c7819d4a0 con 0x7f9c6406e200 2024-09-10T11:36:18.978 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:18 smithi144 ceph-mon[87436]: pgmap v112: 65 pgs: 10 active+undersized, 18 peering, 10 active+undersized+degraded, 27 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31/360 objects degraded (8.611%) 2024-09-10T11:36:18.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.978+0000 7f9c75ffb700 1 -- 172.21.15.144:0/3447278952 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f9c7819d4a0 con 0x7f9c6406e200 2024-09-10T11:36:18.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.980+0000 7f9c7e52e700 1 -- 172.21.15.144:0/3447278952 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9c6406e200 msgr2=0x7f9c640706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.980+0000 7f9c7e52e700 1 --2- 172.21.15.144:0/3447278952 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9c6406e200 0x7f9c640706b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f9c781995b0 tx=0x7f9c6001a040 comp rx=0 tx=0).stop 2024-09-10T11:36:18.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.980+0000 7f9c7e52e700 1 -- 172.21.15.144:0/3447278952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c780ff290 msgr2=0x7f9c78197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:18.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.980+0000 7f9c7e52e700 1 --2- 172.21.15.144:0/3447278952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c780ff290 0x7f9c78197f90 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f9c6800d900 tx=0x7f9c6800dc10 comp rx=0 tx=0).stop 2024-09-10T11:36:18.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.980+0000 7f9c7e52e700 1 -- 172.21.15.144:0/3447278952 shutdown_connections 2024-09-10T11:36:18.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.980+0000 7f9c7e52e700 1 --2- 172.21.15.144:0/3447278952 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9c6406e200 0x7f9c640706b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.980+0000 7f9c7e52e700 1 --2- 172.21.15.144:0/3447278952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c780ff290 0x7f9c78197f90 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.980+0000 7f9c7e52e700 1 --2- 172.21.15.144:0/3447278952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c780ffba0 0x7f9c781984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:18.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.980+0000 7f9c7e52e700 1 -- 172.21.15.144:0/3447278952 >> 172.21.15.144:0/3447278952 conn(0x7f9c78074850 msgr2=0x7f9c78100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:18.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.980+0000 7f9c7e52e700 1 -- 172.21.15.144:0/3447278952 shutdown_connections 2024-09-10T11:36:18.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:18.980+0000 7f9c7e52e700 1 -- 172.21.15.144:0/3447278952 wait complete. 2024-09-10T11:36:18.992 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:18 smithi178 ceph-mon[65940]: pgmap v112: 65 pgs: 10 active+undersized, 18 peering, 10 active+undersized+degraded, 27 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 31/360 objects degraded (8.611%) 2024-09-10T11:36:18.992 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:18 smithi178 podman[75247]: 2024-09-10 11:36:18.784502896 +0000 UTC m=+0.670936698 container remove 301cf35da71c01f6295a2d7a2237ba9de34578134614262b7550a03c060bf8e6 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-activate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-10T11:36:19.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.085+0000 7f6d30798700 1 -- 172.21.15.144:0/3979668910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d280ff390 msgr2=0x7f6d280ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.085+0000 7f6d30798700 1 --2- 172.21.15.144:0/3979668910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d280ff390 0x7f6d280ff760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6d18009910 tx=0x7f6d18009c20 comp rx=0 tx=0).stop 2024-09-10T11:36:19.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.085+0000 7f6d30798700 1 -- 172.21.15.144:0/3979668910 shutdown_connections 2024-09-10T11:36:19.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.085+0000 7f6d30798700 1 --2- 172.21.15.144:0/3979668910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d280ffd30 0x7f6d28103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.085+0000 7f6d30798700 1 --2- 172.21.15.144:0/3979668910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d280ff390 0x7f6d280ff760 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.085+0000 7f6d30798700 1 -- 172.21.15.144:0/3979668910 >> 172.21.15.144:0/3979668910 conn(0x7f6d28074f50 msgr2=0x7f6d28075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:19.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.085+0000 7f6d30798700 1 -- 172.21.15.144:0/3979668910 shutdown_connections 2024-09-10T11:36:19.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.085+0000 7f6d30798700 1 -- 172.21.15.144:0/3979668910 wait complete. 2024-09-10T11:36:19.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.086+0000 7f6d30798700 1 Processor -- start 2024-09-10T11:36:19.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.086+0000 7f6d30798700 1 -- start start 2024-09-10T11:36:19.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.086+0000 7f6d30798700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d280ff390 0x7f6d28194a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.086+0000 7f6d30798700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d280ffd30 0x7f6d28194f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.086+0000 7f6d30798700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d281954b0 con 0x7f6d280ffd30 2024-09-10T11:36:19.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.086+0000 7f6d30798700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d28195620 con 0x7f6d280ff390 2024-09-10T11:36:19.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.086+0000 7f6d2dd33700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d280ffd30 0x7f6d28194f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.086+0000 7f6d2dd33700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d280ffd30 0x7f6d28194f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41998/0 (socket says 172.21.15.144:41998) 2024-09-10T11:36:19.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.086+0000 7f6d2dd33700 1 -- 172.21.15.144:0/3513340926 learned_addr learned my addr 172.21.15.144:0/3513340926 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:36:19.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.086+0000 7f6d2e534700 1 --2- 172.21.15.144:0/3513340926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d280ff390 0x7f6d28194a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.086+0000 7f6d2dd33700 1 -- 172.21.15.144:0/3513340926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d280ff390 msgr2=0x7f6d28194a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.086+0000 7f6d2dd33700 1 --2- 172.21.15.144:0/3513340926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d280ff390 0x7f6d28194a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.086+0000 7f6d2dd33700 1 -- 172.21.15.144:0/3513340926 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d180095f0 con 0x7f6d280ffd30 2024-09-10T11:36:19.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.087+0000 7f6d2dd33700 1 --2- 172.21.15.144:0/3513340926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d280ffd30 0x7f6d28194f70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400d900 tx=0x7f6d2400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:19.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.087+0000 7f6d1f7fe700 1 -- 172.21.15.144:0/3513340926 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d24009900 con 0x7f6d280ffd30 2024-09-10T11:36:19.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.087+0000 7f6d1f7fe700 1 -- 172.21.15.144:0/3513340926 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6d24010460 con 0x7f6d280ffd30 2024-09-10T11:36:19.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.087+0000 7f6d30798700 1 -- 172.21.15.144:0/3513340926 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d281a20a0 con 0x7f6d280ffd30 2024-09-10T11:36:19.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.087+0000 7f6d1f7fe700 1 -- 172.21.15.144:0/3513340926 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d2400f5f0 con 0x7f6d280ffd30 2024-09-10T11:36:19.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.087+0000 7f6d30798700 1 -- 172.21.15.144:0/3513340926 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d281a2400 con 0x7f6d280ffd30 2024-09-10T11:36:19.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.088+0000 7f6d30798700 1 -- 172.21.15.144:0/3513340926 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d28107650 con 0x7f6d280ffd30 2024-09-10T11:36:19.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.088+0000 7f6d1f7fe700 1 -- 172.21.15.144:0/3513340926 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6d240105d0 con 0x7f6d280ffd30 2024-09-10T11:36:19.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.089+0000 7f6d1f7fe700 1 --2- 172.21.15.144:0/3513340926 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6d1406e200 0x7f6d140706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.089+0000 7f6d1f7fe700 1 -- 172.21.15.144:0/3513340926 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6d2408d9e0 con 0x7f6d280ffd30 2024-09-10T11:36:19.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.089+0000 7f6d2e534700 1 --2- 172.21.15.144:0/3513340926 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6d1406e200 0x7f6d140706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.089+0000 7f6d2e534700 1 --2- 172.21.15.144:0/3513340926 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6d1406e200 0x7f6d140706b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d18006010 tx=0x7f6d1801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:19.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.093+0000 7f6d1f7fe700 1 -- 172.21.15.144:0/3513340926 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6d24056ef0 con 0x7f6d280ffd30 2024-09-10T11:36:19.290 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:18 smithi178 podman[75307]: 2024-09-10 11:36:18.894545197 +0000 UTC m=+0.015690077 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:36:19.290 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:19 smithi178 podman[75307]: 2024-09-10 11:36:19.001722354 +0000 UTC m=+0.122867226 container create 867cd5897d94f2f526c285cb7669bfd7292f94728510e828abd0bdfbf36c96bd (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True) 2024-09-10T11:36:19.290 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:19 smithi178 podman[75307]: 2024-09-10 11:36:19.289631695 +0000 UTC m=+0.410776573 container init 867cd5897d94f2f526c285cb7669bfd7292f94728510e828abd0bdfbf36c96bd (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T11:36:19.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.304+0000 7f6d30798700 1 -- 172.21.15.144:0/3513340926 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d28061190 con 0x7f6d1406e200 2024-09-10T11:36:19.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.314+0000 7f6d1f7fe700 1 -- 172.21.15.144:0/3513340926 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f6d28061190 con 0x7f6d1406e200 2024-09-10T11:36:19.315 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:36:19.315 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (4m) 30s ago 15m 25.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:36:19.315 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (15m) 30s ago 15m 8682k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:36:19.316 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (14m) 2m ago 14m 8090k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:36:19.316 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (2m) 30s ago 15m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:36:19.316 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (2m) 2m ago 14m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:36:19.316 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (4m) 30s ago 14m 80.6M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:36:19.316 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (9m) 30s ago 9m 59.9M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:36:19.316 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (9m) 30s ago 9m 126M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:36:19.316 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (9m) 2m ago 9m 62.2M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:36:19.316 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (9m) 2m ago 9m 127M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:36:19.316 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (6m) 30s ago 16m 499M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:36:19.316 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (5m) 2m ago 14m 439M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:36:19.316 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (3m) 30s ago 16m 50.2M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:36:19.316 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (2m) 2m ago 14m 35.5M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:36:19.316 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (5m) 30s ago 15m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:36:19.317 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (5m) 2m ago 14m 15.8M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:36:19.317 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (107s) 30s ago 13m 89.2M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:36:19.317 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (72s) 30s ago 12m 86.5M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:36:19.317 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (36s) 30s ago 12m 12.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:36:19.317 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (11m) 2m ago 11m 255M 2239M 18.2.0 dc2bc1663786 f681e8a4fafb 2024-09-10T11:36:19.317 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (11m) 2m ago 11m 219M 2239M 18.2.0 dc2bc1663786 c832b5dbf4b7 2024-09-10T11:36:19.317 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (11m) 2m ago 11m 198M 2239M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:36:19.317 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (4m) 30s ago 14m 77.2M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:36:19.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.317+0000 7f6d30798700 1 -- 172.21.15.144:0/3513340926 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6d1406e200 msgr2=0x7f6d140706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.317+0000 7f6d30798700 1 --2- 172.21.15.144:0/3513340926 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6d1406e200 0x7f6d140706b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f6d18006010 tx=0x7f6d1801a040 comp rx=0 tx=0).stop 2024-09-10T11:36:19.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.317+0000 7f6d30798700 1 -- 172.21.15.144:0/3513340926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d280ffd30 msgr2=0x7f6d28194f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.317+0000 7f6d30798700 1 --2- 172.21.15.144:0/3513340926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d280ffd30 0x7f6d28194f70 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6d2400d900 tx=0x7f6d2400dc10 comp rx=0 tx=0).stop 2024-09-10T11:36:19.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.317+0000 7f6d30798700 1 -- 172.21.15.144:0/3513340926 shutdown_connections 2024-09-10T11:36:19.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.317+0000 7f6d30798700 1 --2- 172.21.15.144:0/3513340926 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6d1406e200 0x7f6d140706b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.317+0000 7f6d30798700 1 --2- 172.21.15.144:0/3513340926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6d280ff390 0x7f6d28194a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.317+0000 7f6d30798700 1 --2- 172.21.15.144:0/3513340926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6d280ffd30 0x7f6d28194f70 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.317+0000 7f6d30798700 1 -- 172.21.15.144:0/3513340926 >> 172.21.15.144:0/3513340926 conn(0x7f6d28074f50 msgr2=0x7f6d280fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:19.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.317+0000 7f6d30798700 1 -- 172.21.15.144:0/3513340926 shutdown_connections 2024-09-10T11:36:19.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.317+0000 7f6d30798700 1 -- 172.21.15.144:0/3513340926 wait complete. 2024-09-10T11:36:19.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.421+0000 7f4278b94700 1 -- 172.21.15.144:0/2068491878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42740ffa70 msgr2=0x7f4274103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.421+0000 7f4278b94700 1 --2- 172.21.15.144:0/2068491878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42740ffa70 0x7f4274103f50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f4268009910 tx=0x7f4268009c20 comp rx=0 tx=0).stop 2024-09-10T11:36:19.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.421+0000 7f4278b94700 1 -- 172.21.15.144:0/2068491878 shutdown_connections 2024-09-10T11:36:19.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.421+0000 7f4278b94700 1 --2- 172.21.15.144:0/2068491878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42740ffa70 0x7f4274103f50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.421+0000 7f4278b94700 1 --2- 172.21.15.144:0/2068491878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42740ff160 0x7f42740ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.421+0000 7f4278b94700 1 -- 172.21.15.144:0/2068491878 >> 172.21.15.144:0/2068491878 conn(0x7f42740fabb0 msgr2=0x7f42740fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:19.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.421+0000 7f4278b94700 1 -- 172.21.15.144:0/2068491878 shutdown_connections 2024-09-10T11:36:19.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.421+0000 7f4278b94700 1 -- 172.21.15.144:0/2068491878 wait complete. 2024-09-10T11:36:19.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.422+0000 7f4278b94700 1 Processor -- start 2024-09-10T11:36:19.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.422+0000 7f4278b94700 1 -- start start 2024-09-10T11:36:19.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.422+0000 7f4278b94700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42740ff160 0x7f427419c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.422+0000 7f4278b94700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42740ffa70 0x7f427419c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.422+0000 7f4278b94700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f427419cf70 con 0x7f42740ffa70 2024-09-10T11:36:19.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.422+0000 7f4278b94700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42741a0d00 con 0x7f42740ff160 2024-09-10T11:36:19.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.422+0000 7f4271d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42740ffa70 0x7f427419c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.422+0000 7f4271d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42740ffa70 0x7f427419c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42004/0 (socket says 172.21.15.144:42004) 2024-09-10T11:36:19.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.422+0000 7f427259c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42740ff160 0x7f427419c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.422+0000 7f4271d9b700 1 -- 172.21.15.144:0/859782520 learned_addr learned my addr 172.21.15.144:0/859782520 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:36:19.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.422+0000 7f4271d9b700 1 -- 172.21.15.144:0/859782520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42740ff160 msgr2=0x7f427419c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.422+0000 7f4271d9b700 1 --2- 172.21.15.144:0/859782520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42740ff160 0x7f427419c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.422+0000 7f4271d9b700 1 -- 172.21.15.144:0/859782520 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42680095f0 con 0x7f42740ffa70 2024-09-10T11:36:19.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.422+0000 7f427259c700 1 --2- 172.21.15.144:0/859782520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42740ff160 0x7f427419c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:36:19.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.422+0000 7f4271d9b700 1 --2- 172.21.15.144:0/859782520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42740ffa70 0x7f427419c890 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4268005f50 tx=0x7f426800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:19.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.423+0000 7f42637fe700 1 -- 172.21.15.144:0/859782520 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f426801d070 con 0x7f42740ffa70 2024-09-10T11:36:19.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.423+0000 7f42637fe700 1 -- 172.21.15.144:0/859782520 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f426800b610 con 0x7f42740ffa70 2024-09-10T11:36:19.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.423+0000 7f4278b94700 1 -- 172.21.15.144:0/859782520 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42741a0f80 con 0x7f42740ffa70 2024-09-10T11:36:19.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.423+0000 7f42637fe700 1 -- 172.21.15.144:0/859782520 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4268022470 con 0x7f42740ffa70 2024-09-10T11:36:19.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.423+0000 7f4278b94700 1 -- 172.21.15.144:0/859782520 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42741a1470 con 0x7f42740ffa70 2024-09-10T11:36:19.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.424+0000 7f4278b94700 1 -- 172.21.15.144:0/859782520 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4274101620 con 0x7f42740ffa70 2024-09-10T11:36:19.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.424+0000 7f42637fe700 1 -- 172.21.15.144:0/859782520 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4268003780 con 0x7f42740ffa70 2024-09-10T11:36:19.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.425+0000 7f42637fe700 1 --2- 172.21.15.144:0/859782520 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f425c06e200 0x7f425c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.425+0000 7f42637fe700 1 -- 172.21.15.144:0/859782520 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f426808f2f0 con 0x7f42740ffa70 2024-09-10T11:36:19.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.425+0000 7f427259c700 1 --2- 172.21.15.144:0/859782520 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f425c06e200 0x7f425c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.425+0000 7f427259c700 1 --2- 172.21.15.144:0/859782520 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f425c06e200 0x7f425c0706b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4264011490 tx=0x7f4264009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:19.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.429+0000 7f42637fe700 1 -- 172.21.15.144:0/859782520 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4268058800 con 0x7f42740ffa70 2024-09-10T11:36:19.617 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:19 smithi178 podman[75307]: 2024-09-10 11:36:19.292979302 +0000 UTC m=+0.414124174 container start 867cd5897d94f2f526c285cb7669bfd7292f94728510e828abd0bdfbf36c96bd (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0) 2024-09-10T11:36:19.617 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:19 smithi178 ceph-osd[75326]: -- 172.21.15.178:0/514091918 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a560e9a1e0 con 0x55a560e98000 2024-09-10T11:36:19.617 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:19 smithi178 bash[75307]: 867cd5897d94f2f526c285cb7669bfd7292f94728510e828abd0bdfbf36c96bd 2024-09-10T11:36:19.617 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:19 smithi178 systemd[1]: Started Ceph osd.3 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:36:19.725 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.723+0000 7f4278b94700 1 -- 172.21.15.144:0/859782520 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f427404ea50 con 0x7f42740ffa70 2024-09-10T11:36:19.725 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.723+0000 7f42637fe700 1 -- 172.21.15.144:0/859782520 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7f426805bec0 con 0x7f42740ffa70 2024-09-10T11:36:19.725 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:36:19.725 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:36:19.725 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:36:19.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:36:19.725 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:36:19.725 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:36:19.725 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:36:19.725 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:36:19.725 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-09-10T11:36:19.726 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:36:19.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:36:19.726 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:36:19.726 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:36:19.726 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:36:19.726 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:36:19.726 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6, 2024-09-10T11:36:19.726 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 7 2024-09-10T11:36:19.726 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:36:19.726 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:36:19.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.726+0000 7f4278b94700 1 -- 172.21.15.144:0/859782520 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f425c06e200 msgr2=0x7f425c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.726+0000 7f4278b94700 1 --2- 172.21.15.144:0/859782520 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f425c06e200 0x7f425c0706b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4264011490 tx=0x7f4264009260 comp rx=0 tx=0).stop 2024-09-10T11:36:19.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.726+0000 7f4278b94700 1 -- 172.21.15.144:0/859782520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42740ffa70 msgr2=0x7f427419c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.726+0000 7f4278b94700 1 --2- 172.21.15.144:0/859782520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42740ffa70 0x7f427419c890 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f4268005f50 tx=0x7f426800f690 comp rx=0 tx=0).stop 2024-09-10T11:36:19.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.726+0000 7f4278b94700 1 -- 172.21.15.144:0/859782520 shutdown_connections 2024-09-10T11:36:19.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.726+0000 7f4278b94700 1 --2- 172.21.15.144:0/859782520 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f425c06e200 0x7f425c0706b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.726+0000 7f4278b94700 1 --2- 172.21.15.144:0/859782520 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42740ff160 0x7f427419c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.726+0000 7f4278b94700 1 --2- 172.21.15.144:0/859782520 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42740ffa70 0x7f427419c890 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.726+0000 7f4278b94700 1 -- 172.21.15.144:0/859782520 >> 172.21.15.144:0/859782520 conn(0x7f42740fabb0 msgr2=0x7f42741105d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:19.728 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.726+0000 7f4278b94700 1 -- 172.21.15.144:0/859782520 shutdown_connections 2024-09-10T11:36:19.728 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.726+0000 7f4278b94700 1 -- 172.21.15.144:0/859782520 wait complete. 2024-09-10T11:36:19.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.830+0000 7f2748271700 1 -- 172.21.15.144:0/409517974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27400ffd30 msgr2=0x7f2740103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.830+0000 7f2748271700 1 --2- 172.21.15.144:0/409517974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27400ffd30 0x7f2740103ce0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f273c009910 tx=0x7f273c009c20 comp rx=0 tx=0).stop 2024-09-10T11:36:19.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.830+0000 7f2748271700 1 -- 172.21.15.144:0/409517974 shutdown_connections 2024-09-10T11:36:19.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.830+0000 7f2748271700 1 --2- 172.21.15.144:0/409517974 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27400ffd30 0x7f2740103ce0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.830+0000 7f2748271700 1 --2- 172.21.15.144:0/409517974 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27400ff390 0x7f27400ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.830+0000 7f2748271700 1 -- 172.21.15.144:0/409517974 >> 172.21.15.144:0/409517974 conn(0x7f2740074f50 msgr2=0x7f2740075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:19.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.830+0000 7f2748271700 1 -- 172.21.15.144:0/409517974 shutdown_connections 2024-09-10T11:36:19.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.830+0000 7f2748271700 1 -- 172.21.15.144:0/409517974 wait complete. 2024-09-10T11:36:19.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.831+0000 7f2748271700 1 Processor -- start 2024-09-10T11:36:19.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.831+0000 7f2748271700 1 -- start start 2024-09-10T11:36:19.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.831+0000 7f2748271700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27400ff390 0x7f274019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.831+0000 7f2748271700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27400ffd30 0x7f274019af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.831+0000 7f2748271700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274019b5b0 con 0x7f27400ffd30 2024-09-10T11:36:19.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.831+0000 7f2748271700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2740194a60 con 0x7f27400ff390 2024-09-10T11:36:19.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.831+0000 7f274580c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27400ffd30 0x7f274019af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.831+0000 7f274580c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27400ffd30 0x7f274019af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42016/0 (socket says 172.21.15.144:42016) 2024-09-10T11:36:19.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.831+0000 7f274600d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27400ff390 0x7f274019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.831+0000 7f274580c700 1 -- 172.21.15.144:0/237195432 learned_addr learned my addr 172.21.15.144:0/237195432 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:36:19.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.831+0000 7f274580c700 1 -- 172.21.15.144:0/237195432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27400ff390 msgr2=0x7f274019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:19.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.831+0000 7f274580c700 1 --2- 172.21.15.144:0/237195432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27400ff390 0x7f274019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:19.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.831+0000 7f274580c700 1 -- 172.21.15.144:0/237195432 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f273c0095f0 con 0x7f27400ffd30 2024-09-10T11:36:19.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.832+0000 7f274580c700 1 --2- 172.21.15.144:0/237195432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27400ffd30 0x7f274019af20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f273c00bda0 tx=0x7f273c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:19.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.832+0000 7f27377fe700 1 -- 172.21.15.144:0/237195432 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f273c01f070 con 0x7f27400ffd30 2024-09-10T11:36:19.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.832+0000 7f2748271700 1 -- 172.21.15.144:0/237195432 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2740194ce0 con 0x7f27400ffd30 2024-09-10T11:36:19.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.832+0000 7f27377fe700 1 -- 172.21.15.144:0/237195432 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f273c005dc0 con 0x7f27400ffd30 2024-09-10T11:36:19.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.832+0000 7f27377fe700 1 -- 172.21.15.144:0/237195432 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f273c00f690 con 0x7f27400ffd30 2024-09-10T11:36:19.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.832+0000 7f2748271700 1 -- 172.21.15.144:0/237195432 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27401951d0 con 0x7f27400ffd30 2024-09-10T11:36:19.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.833+0000 7f2748271700 1 -- 172.21.15.144:0/237195432 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274004ea50 con 0x7f27400ffd30 2024-09-10T11:36:19.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.834+0000 7f27377fe700 1 -- 172.21.15.144:0/237195432 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f273c0199a0 con 0x7f27400ffd30 2024-09-10T11:36:19.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.834+0000 7f27377fe700 1 --2- 172.21.15.144:0/237195432 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f272c06e200 0x7f272c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:19.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.834+0000 7f27377fe700 1 -- 172.21.15.144:0/237195432 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f273c091640 con 0x7f27400ffd30 2024-09-10T11:36:19.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.834+0000 7f274600d700 1 --2- 172.21.15.144:0/237195432 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f272c06e200 0x7f272c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:19.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.834+0000 7f274600d700 1 --2- 172.21.15.144:0/237195432 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f272c06e200 0x7f272c0706b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2730005950 tx=0x7f273000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:19.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:19.838+0000 7f27377fe700 1 -- 172.21.15.144:0/237195432 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f273c05e600 con 0x7f27400ffd30 2024-09-10T11:36:19.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:19 smithi144 ceph-mon[87436]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:19.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:19 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:19.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:19 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:19.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:19 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:36:20.019 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:19 smithi178 ceph-mon[65940]: from='client.34282 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:20.019 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:19 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:20.019 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:19 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:20.019 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:19 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:36:20.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.082+0000 7f2748271700 1 -- 172.21.15.144:0/237195432 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2740195f70 con 0x7f27400ffd30 2024-09-10T11:36:20.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.082+0000 7f27377fe700 1 -- 172.21.15.144:0/237195432 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1960 (secure 0 0 0) 0x7f273c05e190 con 0x7f27400ffd30 2024-09-10T11:36:20.084 INFO:teuthology.orchestra.run.smithi144.stdout:e23 2024-09-10T11:36:20.084 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:33:11:119769+0000 2024-09-10T11:36:20.084 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:36:20.084 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:36:20.084 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:36:20.084 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:36:20.085 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:36:20.085 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:36:20.085 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 23 2024-09-10T11:36:20.085 INFO:teuthology.orchestra.run.smithi144.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T11:36:20.085 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:36:20.085 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:33:11.119766+0000 2024-09-10T11:36:20.085 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:36:20.085 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:36:20.085 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:36:20.085 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:36:20.085 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:36:20.085 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:36:20.085 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:36:20.085 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:36:20.086 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 47 2024-09-10T11:36:20.086 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:36:20.086 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:36:20.086 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:36:20.086 INFO:teuthology.orchestra.run.smithi144.stdout:up {0=14488,1=24267} 2024-09-10T11:36:20.086 INFO:teuthology.orchestra.run.smithi144.stdout:failed 2024-09-10T11:36:20.086 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:36:20.086 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:36:20.086 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:36:20.086 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:36:20.086 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:36:20.086 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:36:20.086 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:36:20.087 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:36:20.087 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 14488 members: 14488,24267 2024-09-10T11:36:20.087 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{0:14488} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:36:20.087 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{0:14498} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/1752150537,v1:172.21.15.144:6827/1752150537] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:36:20.087 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:36:20.087 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:36:20.087 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:36:20.087 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:36:20.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.085+0000 7f2748271700 1 -- 172.21.15.144:0/237195432 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f272c06e200 msgr2=0x7f272c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.085+0000 7f2748271700 1 --2- 172.21.15.144:0/237195432 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f272c06e200 0x7f272c0706b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f2730005950 tx=0x7f273000b410 comp rx=0 tx=0).stop 2024-09-10T11:36:20.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.085+0000 7f2748271700 1 -- 172.21.15.144:0/237195432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27400ffd30 msgr2=0x7f274019af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.085+0000 7f2748271700 1 --2- 172.21.15.144:0/237195432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27400ffd30 0x7f274019af20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f273c00bda0 tx=0x7f273c00bdd0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.085+0000 7f2748271700 1 -- 172.21.15.144:0/237195432 shutdown_connections 2024-09-10T11:36:20.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.085+0000 7f2748271700 1 --2- 172.21.15.144:0/237195432 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f272c06e200 0x7f272c0706b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.085+0000 7f2748271700 1 --2- 172.21.15.144:0/237195432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f27400ff390 0x7f274019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.085+0000 7f2748271700 1 --2- 172.21.15.144:0/237195432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f27400ffd30 0x7f274019af20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.085+0000 7f2748271700 1 -- 172.21.15.144:0/237195432 >> 172.21.15.144:0/237195432 conn(0x7f2740074f50 msgr2=0x7f27400fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:20.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.085+0000 7f2748271700 1 -- 172.21.15.144:0/237195432 shutdown_connections 2024-09-10T11:36:20.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.085+0000 7f2748271700 1 -- 172.21.15.144:0/237195432 wait complete. 2024-09-10T11:36:20.089 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 23 2024-09-10T11:36:20.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.192+0000 7f574a1b5700 1 -- 172.21.15.144:0/256180504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5744068630 msgr2=0x7f5744068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.192+0000 7f574a1b5700 1 --2- 172.21.15.144:0/256180504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5744068630 0x7f5744068a00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f5734009910 tx=0x7f5734009c20 comp rx=0 tx=0).stop 2024-09-10T11:36:20.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.192+0000 7f574a1b5700 1 -- 172.21.15.144:0/256180504 shutdown_connections 2024-09-10T11:36:20.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.192+0000 7f574a1b5700 1 --2- 172.21.15.144:0/256180504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5744068fd0 0x7f5744104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.192+0000 7f574a1b5700 1 --2- 172.21.15.144:0/256180504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5744068630 0x7f5744068a00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.192+0000 7f574a1b5700 1 -- 172.21.15.144:0/256180504 >> 172.21.15.144:0/256180504 conn(0x7f5744074f50 msgr2=0x7f5744075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:20.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.192+0000 7f574a1b5700 1 -- 172.21.15.144:0/256180504 shutdown_connections 2024-09-10T11:36:20.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.192+0000 7f574a1b5700 1 -- 172.21.15.144:0/256180504 wait complete. 2024-09-10T11:36:20.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.192+0000 7f574a1b5700 1 Processor -- start 2024-09-10T11:36:20.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.193+0000 7f574a1b5700 1 -- start start 2024-09-10T11:36:20.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.193+0000 7f574a1b5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5744068630 0x7f5744195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.193+0000 7f574a1b5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5744068fd0 0x7f5744196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.193+0000 7f574a1b5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5744196950 con 0x7f5744068fd0 2024-09-10T11:36:20.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.193+0000 7f574a1b5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574419a6e0 con 0x7f5744068630 2024-09-10T11:36:20.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.193+0000 7f573bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5744068fd0 0x7f5744196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.193+0000 7f573bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5744068fd0 0x7f5744196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42046/0 (socket says 172.21.15.144:42046) 2024-09-10T11:36:20.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.193+0000 7f573bfff700 1 -- 172.21.15.144:0/1045780285 learned_addr learned my addr 172.21.15.144:0/1045780285 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:36:20.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.193+0000 7f57437fe700 1 --2- 172.21.15.144:0/1045780285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5744068630 0x7f5744195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.193+0000 7f573bfff700 1 -- 172.21.15.144:0/1045780285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5744068630 msgr2=0x7f5744195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.193+0000 7f573bfff700 1 --2- 172.21.15.144:0/1045780285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5744068630 0x7f5744195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.193+0000 7f573bfff700 1 -- 172.21.15.144:0/1045780285 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57340095f0 con 0x7f5744068fd0 2024-09-10T11:36:20.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.194+0000 7f573bfff700 1 --2- 172.21.15.144:0/1045780285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5744068fd0 0x7f5744196270 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d8d0 tx=0x7f572c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:20.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.194+0000 7f57417fa700 1 -- 172.21.15.144:0/1045780285 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f572c0041f0 con 0x7f5744068fd0 2024-09-10T11:36:20.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.194+0000 7f574a1b5700 1 -- 172.21.15.144:0/1045780285 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574419a9c0 con 0x7f5744068fd0 2024-09-10T11:36:20.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.194+0000 7f574a1b5700 1 -- 172.21.15.144:0/1045780285 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574419af10 con 0x7f5744068fd0 2024-09-10T11:36:20.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.194+0000 7f57417fa700 1 -- 172.21.15.144:0/1045780285 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f572c007620 con 0x7f5744068fd0 2024-09-10T11:36:20.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.194+0000 7f57417fa700 1 -- 172.21.15.144:0/1045780285 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f572c0056b0 con 0x7f5744068fd0 2024-09-10T11:36:20.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.195+0000 7f574a1b5700 1 -- 172.21.15.144:0/1045780285 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57441086f0 con 0x7f5744068fd0 2024-09-10T11:36:20.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.196+0000 7f57417fa700 1 -- 172.21.15.144:0/1045780285 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f572c00f770 con 0x7f5744068fd0 2024-09-10T11:36:20.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.196+0000 7f57417fa700 1 --2- 172.21.15.144:0/1045780285 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f573006e200 0x7f57300706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.197+0000 7f57417fa700 1 -- 172.21.15.144:0/1045780285 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f572c08e350 con 0x7f5744068fd0 2024-09-10T11:36:20.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.197+0000 7f57437fe700 1 --2- 172.21.15.144:0/1045780285 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f573006e200 0x7f57300706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.197+0000 7f57437fe700 1 --2- 172.21.15.144:0/1045780285 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f573006e200 0x7f57300706b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5734006010 tx=0x7f573401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:20.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.203+0000 7f57417fa700 1 -- 172.21.15.144:0/1045780285 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f572c057800 con 0x7f5744068fd0 2024-09-10T11:36:20.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.427+0000 7f574a1b5700 1 -- 172.21.15.144:0/1045780285 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f574419b260 con 0x7f573006e200 2024-09-10T11:36:20.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.428+0000 7f57417fa700 1 -- 172.21.15.144:0/1045780285 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f574419b260 con 0x7f573006e200 2024-09-10T11:36:20.430 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:36:20.430 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:36:20.430 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:36:20.430 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:36:20.430 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:36:20.430 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:36:20.430 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:36:20.430 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:36:20.430 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:36:20.430 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "9/23 daemons upgraded", 2024-09-10T11:36:20.431 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading osd daemons", 2024-09-10T11:36:20.431 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:36:20.431 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:36:20.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.431+0000 7f574a1b5700 1 -- 172.21.15.144:0/1045780285 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f573006e200 msgr2=0x7f57300706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.431+0000 7f574a1b5700 1 --2- 172.21.15.144:0/1045780285 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f573006e200 0x7f57300706b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5734006010 tx=0x7f573401a040 comp rx=0 tx=0).stop 2024-09-10T11:36:20.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.431+0000 7f574a1b5700 1 -- 172.21.15.144:0/1045780285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5744068fd0 msgr2=0x7f5744196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.431+0000 7f574a1b5700 1 --2- 172.21.15.144:0/1045780285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5744068fd0 0x7f5744196270 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f572c00d8d0 tx=0x7f572c00dbe0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.431+0000 7f574a1b5700 1 -- 172.21.15.144:0/1045780285 shutdown_connections 2024-09-10T11:36:20.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.431+0000 7f574a1b5700 1 --2- 172.21.15.144:0/1045780285 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f573006e200 0x7f57300706b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.431+0000 7f574a1b5700 1 --2- 172.21.15.144:0/1045780285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5744068630 0x7f5744195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.431+0000 7f574a1b5700 1 --2- 172.21.15.144:0/1045780285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5744068fd0 0x7f5744196270 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.431+0000 7f574a1b5700 1 -- 172.21.15.144:0/1045780285 >> 172.21.15.144:0/1045780285 conn(0x7f5744074f50 msgr2=0x7f57440fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:20.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.431+0000 7f574a1b5700 1 -- 172.21.15.144:0/1045780285 shutdown_connections 2024-09-10T11:36:20.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.431+0000 7f574a1b5700 1 -- 172.21.15.144:0/1045780285 wait complete. 2024-09-10T11:36:20.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.536+0000 7fa0948e6700 1 -- 172.21.15.144:0/836288064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0900ffd30 msgr2=0x7fa090103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.536+0000 7fa0948e6700 1 --2- 172.21.15.144:0/836288064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0900ffd30 0x7fa090103ce0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa080009910 tx=0x7fa080009c20 comp rx=0 tx=0).stop 2024-09-10T11:36:20.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.536+0000 7fa0948e6700 1 -- 172.21.15.144:0/836288064 shutdown_connections 2024-09-10T11:36:20.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.536+0000 7fa0948e6700 1 --2- 172.21.15.144:0/836288064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0900ffd30 0x7fa090103ce0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.536+0000 7fa0948e6700 1 --2- 172.21.15.144:0/836288064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0900ff390 0x7fa0900ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.536+0000 7fa0948e6700 1 -- 172.21.15.144:0/836288064 >> 172.21.15.144:0/836288064 conn(0x7fa090074f50 msgr2=0x7fa090075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:20.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.536+0000 7fa0948e6700 1 -- 172.21.15.144:0/836288064 shutdown_connections 2024-09-10T11:36:20.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.536+0000 7fa0948e6700 1 -- 172.21.15.144:0/836288064 wait complete. 2024-09-10T11:36:20.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.537+0000 7fa0948e6700 1 Processor -- start 2024-09-10T11:36:20.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.537+0000 7fa0948e6700 1 -- start start 2024-09-10T11:36:20.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.537+0000 7fa0948e6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0900ff390 0x7fa090196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.537+0000 7fa0948e6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0900ffd30 0x7fa0901970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.537+0000 7fa0948e6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0901976c0 con 0x7fa0900ff390 2024-09-10T11:36:20.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.537+0000 7fa0948e6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa090197830 con 0x7fa0900ffd30 2024-09-10T11:36:20.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.537+0000 7fa08e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0900ff390 0x7fa090196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.537+0000 7fa08e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0900ff390 0x7fa090196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42066/0 (socket says 172.21.15.144:42066) 2024-09-10T11:36:20.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.537+0000 7fa08e59c700 1 -- 172.21.15.144:0/3461909613 learned_addr learned my addr 172.21.15.144:0/3461909613 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:36:20.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.537+0000 7fa08dd9b700 1 --2- 172.21.15.144:0/3461909613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0900ffd30 0x7fa0901970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.538+0000 7fa08e59c700 1 -- 172.21.15.144:0/3461909613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0900ffd30 msgr2=0x7fa0901970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.538+0000 7fa08e59c700 1 --2- 172.21.15.144:0/3461909613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0900ffd30 0x7fa0901970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.538+0000 7fa08e59c700 1 -- 172.21.15.144:0/3461909613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0800095f0 con 0x7fa0900ff390 2024-09-10T11:36:20.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.538+0000 7fa08e59c700 1 --2- 172.21.15.144:0/3461909613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0900ff390 0x7fa090196b60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa08400d8d0 tx=0x7fa08400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:20.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.538+0000 7fa07f7fe700 1 -- 172.21.15.144:0/3461909613 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0840098a0 con 0x7fa0900ff390 2024-09-10T11:36:20.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.538+0000 7fa07f7fe700 1 -- 172.21.15.144:0/3461909613 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa084010460 con 0x7fa0900ff390 2024-09-10T11:36:20.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.538+0000 7fa0948e6700 1 -- 172.21.15.144:0/3461909613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa09019fbd0 con 0x7fa0900ff390 2024-09-10T11:36:20.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.538+0000 7fa07f7fe700 1 -- 172.21.15.144:0/3461909613 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa08400f5f0 con 0x7fa0900ff390 2024-09-10T11:36:20.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.538+0000 7fa0948e6700 1 -- 172.21.15.144:0/3461909613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0901a0090 con 0x7fa0900ff390 2024-09-10T11:36:20.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.539+0000 7fa0948e6700 1 -- 172.21.15.144:0/3461909613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa090107600 con 0x7fa0900ff390 2024-09-10T11:36:20.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.540+0000 7fa07f7fe700 1 -- 172.21.15.144:0/3461909613 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa084009a00 con 0x7fa0900ff390 2024-09-10T11:36:20.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.540+0000 7fa07f7fe700 1 --2- 172.21.15.144:0/3461909613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa07806e200 0x7fa0780706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:20.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.540+0000 7fa07f7fe700 1 -- 172.21.15.144:0/3461909613 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa08408de80 con 0x7fa0900ff390 2024-09-10T11:36:20.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.540+0000 7fa08dd9b700 1 --2- 172.21.15.144:0/3461909613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa07806e200 0x7fa0780706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:20.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.540+0000 7fa08dd9b700 1 --2- 172.21.15.144:0/3461909613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa07806e200 0x7fa0780706b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa080006010 tx=0x7fa08001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:20.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.544+0000 7fa07f7fe700 1 -- 172.21.15.144:0/3461909613 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa084057310 con 0x7fa0900ff390 2024-09-10T11:36:20.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.830+0000 7fa0948e6700 1 -- 172.21.15.144:0/3461909613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa09004ea50 con 0x7fa0900ff390 2024-09-10T11:36:20.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.831+0000 7fa07f7fe700 1 -- 172.21.15.144:0/3461909613 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1518 (secure 0 0 0) 0x7fa08405a9d0 con 0x7fa0900ff390 2024-09-10T11:36:20.832 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 73/360 objects degraded (20.278%), 19 pgs degraded 2024-09-10T11:36:20.832 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:36:20.832 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:36:20.832 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-10T11:36:20.833 INFO:teuthology.orchestra.run.smithi144.stdout: osd.3 (root=default,host=smithi178) is down 2024-09-10T11:36:20.833 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 73/360 objects degraded (20.278%), 19 pgs degraded 2024-09-10T11:36:20.833 INFO:teuthology.orchestra.run.smithi144.stdout: pg 1.0 is active+undersized+degraded, acting [0,1] 2024-09-10T11:36:20.833 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.0 is active+undersized+degraded, acting [1,0] 2024-09-10T11:36:20.833 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.5 is active+undersized+degraded, acting [0,4] 2024-09-10T11:36:20.833 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.6 is active+undersized+degraded, acting [1,4] 2024-09-10T11:36:20.833 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.7 is active+undersized+degraded, acting [4,2] 2024-09-10T11:36:20.833 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.8 is active+undersized+degraded, acting [5,0] 2024-09-10T11:36:20.833 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.a is active+undersized+degraded, acting [1,4] 2024-09-10T11:36:20.833 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.b is active+undersized+degraded, acting [4,5] 2024-09-10T11:36:20.833 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.d is active+undersized+degraded, acting [1,2] 2024-09-10T11:36:20.833 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.e is active+undersized+degraded, acting [2,0] 2024-09-10T11:36:20.833 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.11 is active+undersized+degraded, acting [4,1] 2024-09-10T11:36:20.833 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.12 is active+undersized+degraded, acting [1,0] 2024-09-10T11:36:20.834 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.14 is active+undersized+degraded, acting [4,5] 2024-09-10T11:36:20.834 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.15 is active+undersized+degraded, acting [1,0] 2024-09-10T11:36:20.834 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.16 is active+undersized+degraded, acting [5,2] 2024-09-10T11:36:20.834 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.18 is active+undersized+degraded, acting [5,4] 2024-09-10T11:36:20.834 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1a is active+undersized+degraded, acting [4,5] 2024-09-10T11:36:20.834 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1d is active+undersized+degraded, acting [5,0] 2024-09-10T11:36:20.834 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1f is active+undersized+degraded, acting [0,4] 2024-09-10T11:36:20.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.833+0000 7fa0948e6700 1 -- 172.21.15.144:0/3461909613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa07806e200 msgr2=0x7fa0780706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.833+0000 7fa0948e6700 1 --2- 172.21.15.144:0/3461909613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa07806e200 0x7fa0780706b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa080006010 tx=0x7fa08001a040 comp rx=0 tx=0).stop 2024-09-10T11:36:20.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.833+0000 7fa0948e6700 1 -- 172.21.15.144:0/3461909613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0900ff390 msgr2=0x7fa090196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:20.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.833+0000 7fa0948e6700 1 --2- 172.21.15.144:0/3461909613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0900ff390 0x7fa090196b60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fa08400d8d0 tx=0x7fa08400dbe0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.833+0000 7fa0948e6700 1 -- 172.21.15.144:0/3461909613 shutdown_connections 2024-09-10T11:36:20.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.833+0000 7fa0948e6700 1 --2- 172.21.15.144:0/3461909613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa07806e200 0x7fa0780706b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.833+0000 7fa0948e6700 1 --2- 172.21.15.144:0/3461909613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0900ff390 0x7fa090196b60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.833+0000 7fa0948e6700 1 --2- 172.21.15.144:0/3461909613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0900ffd30 0x7fa0901970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:20.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.833+0000 7fa0948e6700 1 -- 172.21.15.144:0/3461909613 >> 172.21.15.144:0/3461909613 conn(0x7fa090074f50 msgr2=0x7fa0900fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:20.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.833+0000 7fa0948e6700 1 -- 172.21.15.144:0/3461909613 shutdown_connections 2024-09-10T11:36:20.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:20.833+0000 7fa0948e6700 1 -- 172.21.15.144:0/3461909613 wait complete. 2024-09-10T11:36:20.978 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:20 smithi178 ceph-mon[65940]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:20.978 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:20 smithi178 ceph-mon[65940]: from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:20.978 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:20 smithi178 ceph-mon[65940]: pgmap v113: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 73/360 objects degraded (20.278%) 2024-09-10T11:36:20.978 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:20 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/859782520' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:20.978 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:20 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/237195432' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:36:20.978 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:20 smithi178 ceph-mon[65940]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs peering) 2024-09-10T11:36:21.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:20 smithi144 ceph-mon[87436]: from='client.34286 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:21.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:20 smithi144 ceph-mon[87436]: from='client.34290 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:21.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:20 smithi144 ceph-mon[87436]: pgmap v113: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 73/360 objects degraded (20.278%) 2024-09-10T11:36:21.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:20 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/859782520' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:21.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:20 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/237195432' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:36:21.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:20 smithi144 ceph-mon[87436]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 10 pgs peering) 2024-09-10T11:36:21.367 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:21 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3[75322]: 2024-09-10T11:36:21.177+0000 7f211a0cc640 -1 Falling back to public interface 2024-09-10T11:36:22.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:21 smithi178 ceph-mon[65940]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:22.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:21 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3461909613' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:36:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:21 smithi144 ceph-mon[87436]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:21 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3461909613' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:36:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:22 smithi178 ceph-mon[65940]: pgmap v114: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 73/360 objects degraded (20.278%) 2024-09-10T11:36:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:23.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:22 smithi144 ceph-mon[87436]: pgmap v114: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 73/360 objects degraded (20.278%) 2024-09-10T11:36:23.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:23.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:24.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:23 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:24.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:23 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:24.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:23 smithi178 ceph-mon[65940]: pgmap v115: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 73/360 objects degraded (20.278%) 2024-09-10T11:36:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:23 smithi144 ceph-mon[87436]: pgmap v115: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 73/360 objects degraded (20.278%) 2024-09-10T11:36:26.804 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:26 smithi178 ceph-mon[65940]: pgmap v116: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 73/360 objects degraded (20.278%) 2024-09-10T11:36:26.804 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:26 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:26.804 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:26 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:26.804 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:26 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:36:26.804 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:26 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:36:26.804 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:26 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:26.804 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:26 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:36:26.804 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:26 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:26.804 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:26 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:26.804 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:26 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:26.804 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:26 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-10T11:36:27.117 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:26 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3[75322]: 2024-09-10T11:36:26.802+0000 7f211a0cc640 -1 osd.3 0 read_superblock omap replica is missing. 2024-09-10T11:36:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:26 smithi144 ceph-mon[87436]: pgmap v116: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 73/360 objects degraded (20.278%) 2024-09-10T11:36:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:36:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:36:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:36:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:27.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:27.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:27.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-10T11:36:27.617 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:27 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3[75322]: 2024-09-10T11:36:27.125+0000 7f211a0cc640 -1 osd.3 65 log_to_monitors true 2024-09-10T11:36:28.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:27 smithi178 ceph-mon[65940]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-10T11:36:28.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:27 smithi178 ceph-mon[65940]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-09-10T11:36:28.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:27 smithi178 ceph-mon[65940]: from='osd.3 [v2:172.21.15.178:6800/1154926806,v1:172.21.15.178:6801/1154926806]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T11:36:28.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:27 smithi178 ceph-mon[65940]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T11:36:28.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:27 smithi178 ceph-mon[65940]: Health check update: Degraded data redundancy: 73/360 objects degraded (20.278%), 19 pgs degraded (PG_DEGRADED) 2024-09-10T11:36:28.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:27 smithi144 ceph-mon[87436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-10T11:36:28.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:27 smithi144 ceph-mon[87436]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-09-10T11:36:28.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:27 smithi144 ceph-mon[87436]: from='osd.3 [v2:172.21.15.178:6800/1154926806,v1:172.21.15.178:6801/1154926806]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T11:36:28.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:27 smithi144 ceph-mon[87436]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T11:36:28.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:27 smithi144 ceph-mon[87436]: Health check update: Degraded data redundancy: 73/360 objects degraded (20.278%), 19 pgs degraded (PG_DEGRADED) 2024-09-10T11:36:28.701 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 11:36:28 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3[75322]: 2024-09-10T11:36:28.376+0000 7f210de5e640 -1 osd.3 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:36:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:28 smithi178 ceph-mon[65940]: pgmap v117: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 73/360 objects degraded (20.278%) 2024-09-10T11:36:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:28 smithi178 ceph-mon[65940]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T11:36:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:28 smithi178 ceph-mon[65940]: from='osd.3 [v2:172.21.15.178:6800/1154926806,v1:172.21.15.178:6801/1154926806]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:36:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:28 smithi178 ceph-mon[65940]: osdmap e68: 6 total, 5 up, 6 in 2024-09-10T11:36:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:28 smithi178 ceph-mon[65940]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:36:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:28 smithi178 ceph-mon[65940]: from='osd.3 ' entity='osd.3' 2024-09-10T11:36:29.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:29.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:36:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:28 smithi144 ceph-mon[87436]: pgmap v117: 65 pgs: 19 active+undersized, 19 active+undersized+degraded, 27 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 73/360 objects degraded (20.278%) 2024-09-10T11:36:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:28 smithi144 ceph-mon[87436]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T11:36:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:28 smithi144 ceph-mon[87436]: from='osd.3 [v2:172.21.15.178:6800/1154926806,v1:172.21.15.178:6801/1154926806]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:36:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:28 smithi144 ceph-mon[87436]: osdmap e68: 6 total, 5 up, 6 in 2024-09-10T11:36:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:28 smithi144 ceph-mon[87436]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:36:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:28 smithi144 ceph-mon[87436]: from='osd.3 ' entity='osd.3' 2024-09-10T11:36:29.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:29.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:36:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:29 smithi178 ceph-mon[65940]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T11:36:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:29 smithi178 ceph-mon[65940]: osd.3 [v2:172.21.15.178:6800/1154926806,v1:172.21.15.178:6801/1154926806] boot 2024-09-10T11:36:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:29 smithi178 ceph-mon[65940]: osdmap e69: 6 total, 6 up, 6 in 2024-09-10T11:36:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:29 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:36:30.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:29 smithi144 ceph-mon[87436]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T11:36:30.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:29 smithi144 ceph-mon[87436]: osd.3 [v2:172.21.15.178:6800/1154926806,v1:172.21.15.178:6801/1154926806] boot 2024-09-10T11:36:30.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:29 smithi144 ceph-mon[87436]: osdmap e69: 6 total, 6 up, 6 in 2024-09-10T11:36:30.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:29 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T11:36:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:30 smithi178 ceph-mon[65940]: pgmap v120: 65 pgs: 8 peering, 14 active+undersized, 16 active+undersized+degraded, 27 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 63/360 objects degraded (17.500%) 2024-09-10T11:36:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:30 smithi178 ceph-mon[65940]: osdmap e70: 6 total, 6 up, 6 in 2024-09-10T11:36:31.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:30 smithi144 ceph-mon[87436]: pgmap v120: 65 pgs: 8 peering, 14 active+undersized, 16 active+undersized+degraded, 27 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s; 63/360 objects degraded (17.500%) 2024-09-10T11:36:31.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:30 smithi144 ceph-mon[87436]: osdmap e70: 6 total, 6 up, 6 in 2024-09-10T11:36:32.986 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:32 smithi144 ceph-mon[87436]: pgmap v122: 65 pgs: 8 peering, 6 active+undersized, 10 active+undersized+degraded, 41 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 3 op/s; 44/360 objects degraded (12.222%) 2024-09-10T11:36:32.986 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:32 smithi144 ceph-mon[87436]: Health check update: Degraded data redundancy: 44/360 objects degraded (12.222%), 10 pgs degraded (PG_DEGRADED) 2024-09-10T11:36:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:32 smithi178 ceph-mon[65940]: pgmap v122: 65 pgs: 8 peering, 6 active+undersized, 10 active+undersized+degraded, 41 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 853 B/s rd, 3 op/s; 44/360 objects degraded (12.222%) 2024-09-10T11:36:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:32 smithi178 ceph-mon[65940]: Health check update: Degraded data redundancy: 44/360 objects degraded (12.222%), 10 pgs degraded (PG_DEGRADED) 2024-09-10T11:36:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:33 smithi178 ceph-mon[65940]: pgmap v123: 65 pgs: 8 peering, 57 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T11:36:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:33 smithi178 ceph-mon[65940]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 44/360 objects degraded (12.222%), 10 pgs degraded) 2024-09-10T11:36:34.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:33 smithi144 ceph-mon[87436]: pgmap v123: 65 pgs: 8 peering, 57 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T11:36:34.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:33 smithi144 ceph-mon[87436]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 44/360 objects degraded (12.222%), 10 pgs degraded) 2024-09-10T11:36:36.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:36 smithi178 ceph-mon[65940]: pgmap v124: 65 pgs: 65 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-10T11:36:36.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:36 smithi144 ceph-mon[87436]: pgmap v124: 65 pgs: 65 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-10T11:36:38.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:38 smithi144 ceph-mon[87436]: pgmap v125: 65 pgs: 65 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:36:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:38 smithi178 ceph-mon[65940]: pgmap v125: 65 pgs: 65 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:36:40.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:40 smithi178 ceph-mon[65940]: pgmap v126: 65 pgs: 65 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-10T11:36:40.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:40 smithi144 ceph-mon[87436]: pgmap v126: 65 pgs: 65 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-10T11:36:41.861 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:41 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-10T11:36:41.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:41 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-10T11:36:42.816 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:42 smithi178 ceph-mon[65940]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-10T11:36:42.816 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:42 smithi178 ceph-mon[65940]: Upgrade: osd.4 is safe to restart 2024-09-10T11:36:42.816 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:42 smithi178 ceph-mon[65940]: pgmap v127: 65 pgs: 65 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-09-10T11:36:42.816 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:42.817 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T11:36:42.817 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:36:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:42 smithi144 ceph-mon[87436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-09-10T11:36:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:42 smithi144 ceph-mon[87436]: Upgrade: osd.4 is safe to restart 2024-09-10T11:36:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:42 smithi144 ceph-mon[87436]: pgmap v127: 65 pgs: 65 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 4 op/s 2024-09-10T11:36:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T11:36:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:36:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:43 smithi144 ceph-mon[87436]: Upgrade: Updating osd.4 2024-09-10T11:36:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:43 smithi144 ceph-mon[87436]: Deploying daemon osd.4 on smithi178 2024-09-10T11:36:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:36:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:43.988 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:43 smithi178 ceph-mon[65940]: Upgrade: Updating osd.4 2024-09-10T11:36:43.988 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:43 smithi178 ceph-mon[65940]: Deploying daemon osd.4 on smithi178 2024-09-10T11:36:43.988 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:43.988 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:36:43.988 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:44.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:44 smithi144 ceph-mon[87436]: pgmap v128: 65 pgs: 65 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-09-10T11:36:45.045 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:44 smithi178 ceph-mon[65940]: pgmap v128: 65 pgs: 65 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s 2024-09-10T11:36:45.305 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:45 smithi178 systemd[1]: Stopping Ceph osd.4 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:36:45.617 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:45 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[44175]: 2024-09-10T11:36:45.344+0000 7f886e585700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:36:45.617 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:45 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[44175]: 2024-09-10T11:36:45.344+0000 7f886e585700 -1 osd.4 70 *** Got signal Terminated *** 2024-09-10T11:36:45.617 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:45 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[44175]: 2024-09-10T11:36:45.344+0000 7f886e585700 -1 osd.4 70 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:36:45.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:45 smithi144 ceph-mon[87436]: osd.4 marked itself down and dead 2024-09-10T11:36:46.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:45 smithi178 ceph-mon[65940]: osd.4 marked itself down and dead 2024-09-10T11:36:46.530 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:46 smithi178 podman[79775]: 2024-09-10 11:36:46.19344335 +0000 UTC m=+0.990321677 container died c832b5dbf4b73f78aedc29dcdb5818d82df17303949f1ed0f2fb9de288a575db (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20231212, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-09-10T11:36:46.530 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:46 smithi178 podman[79775]: 2024-09-10 11:36:46.529320015 +0000 UTC m=+1.326198330 container cleanup c832b5dbf4b73f78aedc29dcdb5818d82df17303949f1ed0f2fb9de288a575db (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20231212, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T11:36:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:46 smithi178 ceph-mon[65940]: pgmap v129: 65 pgs: 65 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:36:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:46 smithi178 ceph-mon[65940]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T11:36:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:46 smithi178 ceph-mon[65940]: osdmap e71: 6 total, 5 up, 6 in 2024-09-10T11:36:46.867 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:46 smithi178 bash[79775]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4 2024-09-10T11:36:46.867 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:46 smithi178 podman[79792]: 2024-09-10 11:36:46.713865785 +0000 UTC m=+0.520673184 container remove c832b5dbf4b73f78aedc29dcdb5818d82df17303949f1ed0f2fb9de288a575db (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-09-10T11:36:46.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:46 smithi144 ceph-mon[87436]: pgmap v129: 65 pgs: 65 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T11:36:46.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:46 smithi144 ceph-mon[87436]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T11:36:46.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:46 smithi144 ceph-mon[87436]: osdmap e71: 6 total, 5 up, 6 in 2024-09-10T11:36:47.609 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:47 smithi178 podman[79881]: 2024-09-10 11:36:47.041321643 +0000 UTC m=+0.015797060 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:36:47.609 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:47 smithi178 podman[79881]: 2024-09-10 11:36:47.156590366 +0000 UTC m=+0.131065770 container create 9f56e2c63ef5a2b3c479b0a5af1362594085197c8760ae5074674b69682e0b7d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T11:36:47.609 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:47 smithi178 podman[79881]: 2024-09-10 11:36:47.383911011 +0000 UTC m=+0.358386416 container init 9f56e2c63ef5a2b3c479b0a5af1362594085197c8760ae5074674b69682e0b7d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-10T11:36:47.609 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:47 smithi178 podman[79881]: 2024-09-10 11:36:47.387146676 +0000 UTC m=+0.361622089 container start 9f56e2c63ef5a2b3c479b0a5af1362594085197c8760ae5074674b69682e0b7d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T11:36:47.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:47 smithi178 ceph-mon[65940]: osdmap e72: 6 total, 5 up, 6 in 2024-09-10T11:36:47.867 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:47 smithi178 podman[79881]: 2024-09-10 11:36:47.667449286 +0000 UTC m=+0.641924690 container attach 9f56e2c63ef5a2b3c479b0a5af1362594085197c8760ae5074674b69682e0b7d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T11:36:47.867 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:47 smithi178 podman[79881]: 2024-09-10 11:36:47.66787853 +0000 UTC m=+0.642353944 container died 9f56e2c63ef5a2b3c479b0a5af1362594085197c8760ae5074674b69682e0b7d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-10T11:36:47.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:47 smithi144 ceph-mon[87436]: osdmap e72: 6 total, 5 up, 6 in 2024-09-10T11:36:48.617 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:48 smithi178 podman[79905]: 2024-09-10 11:36:48.271175593 +0000 UTC m=+0.657420212 container remove 9f56e2c63ef5a2b3c479b0a5af1362594085197c8760ae5074674b69682e0b7d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T11:36:48.617 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:48 smithi178 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.4.service: Deactivated successfully. 2024-09-10T11:36:48.617 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:48 smithi178 systemd[1]: Stopped Ceph osd.4 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:36:48.617 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:48 smithi178 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.4.service: Consumed 56.222s CPU time. 2024-09-10T11:36:48.617 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:48 smithi178 systemd[1]: Starting Ceph osd.4 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:36:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:48 smithi144 ceph-mon[87436]: pgmap v132: 65 pgs: 8 peering, 5 stale+active+clean, 52 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s 2024-09-10T11:36:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:48 smithi144 ceph-mon[87436]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T11:36:48.966 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:48 smithi178 ceph-mon[65940]: pgmap v132: 65 pgs: 8 peering, 5 stale+active+clean, 52 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 3 op/s 2024-09-10T11:36:48.967 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:48 smithi178 ceph-mon[65940]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T11:36:48.967 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:48 smithi178 podman[80003]: 2024-09-10 11:36:48.613274823 +0000 UTC m=+0.015709505 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:36:48.967 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:48 smithi178 podman[80003]: 2024-09-10 11:36:48.738933709 +0000 UTC m=+0.141368376 container create fac80702d990e7118ade68dab0d818eddcb31870b248349261320f86151e1e32 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-activate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux ) 2024-09-10T11:36:48.967 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:48 smithi178 podman[80003]: 2024-09-10 11:36:48.966317167 +0000 UTC m=+0.368751851 container init fac80702d990e7118ade68dab0d818eddcb31870b248349261320f86151e1e32 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-activate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T11:36:49.367 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:48 smithi178 podman[80003]: 2024-09-10 11:36:48.969982134 +0000 UTC m=+0.372416801 container start fac80702d990e7118ade68dab0d818eddcb31870b248349261320f86151e1e32 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-activate, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T11:36:49.367 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:49 smithi178 podman[80003]: 2024-09-10 11:36:49.057813683 +0000 UTC m=+0.460248351 container attach fac80702d990e7118ade68dab0d818eddcb31870b248349261320f86151e1e32 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-activate, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-10T11:36:49.367 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:49 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-activate[80018]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:36:49.367 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:49 smithi178 bash[80003]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:36:49.367 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:49 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-activate[80018]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:36:49.367 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:49 smithi178 bash[80003]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:36:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:49 smithi178 ceph-mon[65940]: pgmap v133: 65 pgs: 8 active+undersized, 8 peering, 3 stale+active+clean, 8 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 33/360 objects degraded (9.167%) 2024-09-10T11:36:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:49 smithi178 ceph-mon[65940]: Health check failed: Degraded data redundancy: 33/360 objects degraded (9.167%), 8 pgs degraded (PG_DEGRADED) 2024-09-10T11:36:50.117 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:49 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-activate[80018]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T11:36:50.117 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:49 smithi178 bash[80003]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T11:36:50.117 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:49 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-activate[80018]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-10T11:36:50.118 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:49 smithi178 bash[80003]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-09-10T11:36:50.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:49 smithi144 ceph-mon[87436]: pgmap v133: 65 pgs: 8 active+undersized, 8 peering, 3 stale+active+clean, 8 active+undersized+degraded, 38 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 33/360 objects degraded (9.167%) 2024-09-10T11:36:50.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:49 smithi144 ceph-mon[87436]: Health check failed: Degraded data redundancy: 33/360 objects degraded (9.167%), 8 pgs degraded (PG_DEGRADED) 2024-09-10T11:36:50.446 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:50 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-activate[80018]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-10T11:36:50.446 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:50 smithi178 bash[80003]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-09-10T11:36:50.446 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:50 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-activate[80018]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-10T11:36:50.446 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:50 smithi178 bash[80003]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-09-10T11:36:50.447 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:50 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-activate[80018]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-09-10T11:36:50.447 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:50 smithi178 bash[80003]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-09-10T11:36:50.447 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:50 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-activate[80018]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T11:36:50.447 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:50 smithi178 bash[80003]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-10T11:36:50.447 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:50 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-activate[80018]: --> ceph-volume raw activate successful for osd ID: 4 2024-09-10T11:36:50.447 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:50 smithi178 bash[80003]: --> ceph-volume raw activate successful for osd ID: 4 2024-09-10T11:36:50.447 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:50 smithi178 podman[80003]: 2024-09-10 11:36:50.169089559 +0000 UTC m=+1.571524233 container died fac80702d990e7118ade68dab0d818eddcb31870b248349261320f86151e1e32 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-activate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T11:36:50.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.952+0000 7face442b700 1 -- 172.21.15.144:0/3524595752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facdc1003d0 msgr2=0x7facdc1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:50.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.953+0000 7face442b700 1 --2- 172.21.15.144:0/3524595752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facdc1003d0 0x7facdc1007a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7facd8009910 tx=0x7facd8009c20 comp rx=0 tx=0).stop 2024-09-10T11:36:50.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.953+0000 7face442b700 1 -- 172.21.15.144:0/3524595752 shutdown_connections 2024-09-10T11:36:50.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.953+0000 7face442b700 1 --2- 172.21.15.144:0/3524595752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facdc100d70 0x7facdc104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:50.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.953+0000 7face442b700 1 --2- 172.21.15.144:0/3524595752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facdc1003d0 0x7facdc1007a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:50.956 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.953+0000 7face442b700 1 -- 172.21.15.144:0/3524595752 >> 172.21.15.144:0/3524595752 conn(0x7facdc0fbc90 msgr2=0x7facdc0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:50.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.953+0000 7face442b700 1 -- 172.21.15.144:0/3524595752 shutdown_connections 2024-09-10T11:36:50.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.953+0000 7face442b700 1 -- 172.21.15.144:0/3524595752 wait complete. 2024-09-10T11:36:50.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.953+0000 7face442b700 1 Processor -- start 2024-09-10T11:36:50.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.953+0000 7face442b700 1 -- start start 2024-09-10T11:36:50.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7face442b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facdc1003d0 0x7facdc195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:50.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7face442b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facdc100d70 0x7facdc1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:50.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7face442b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc196980 con 0x7facdc100d70 2024-09-10T11:36:50.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7face442b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facdc19a710 con 0x7facdc1003d0 2024-09-10T11:36:50.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7face19c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facdc100d70 0x7facdc1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:50.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7face19c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facdc100d70 0x7facdc1962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36136/0 (socket says 172.21.15.144:36136) 2024-09-10T11:36:50.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7face19c6700 1 -- 172.21.15.144:0/1090853351 learned_addr learned my addr 172.21.15.144:0/1090853351 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:36:50.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7face21c7700 1 --2- 172.21.15.144:0/1090853351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facdc1003d0 0x7facdc195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:50.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7face19c6700 1 -- 172.21.15.144:0/1090853351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facdc1003d0 msgr2=0x7facdc195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:50.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7face19c6700 1 --2- 172.21.15.144:0/1090853351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facdc1003d0 0x7facdc195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:50.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7face19c6700 1 -- 172.21.15.144:0/1090853351 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd80095f0 con 0x7facdc100d70 2024-09-10T11:36:50.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7face19c6700 1 --2- 172.21.15.144:0/1090853351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facdc100d70 0x7facdc1962a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faccc00d900 tx=0x7faccc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:50.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7facd37fe700 1 -- 172.21.15.144:0/1090853351 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faccc009900 con 0x7facdc100d70 2024-09-10T11:36:50.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7facd37fe700 1 -- 172.21.15.144:0/1090853351 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faccc010460 con 0x7facdc100d70 2024-09-10T11:36:50.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7face442b700 1 -- 172.21.15.144:0/1090853351 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facdc19a9f0 con 0x7facdc100d70 2024-09-10T11:36:50.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.954+0000 7facd37fe700 1 -- 172.21.15.144:0/1090853351 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faccc00f5f0 con 0x7facdc100d70 2024-09-10T11:36:50.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.955+0000 7face442b700 1 -- 172.21.15.144:0/1090853351 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facdc19af40 con 0x7facdc100d70 2024-09-10T11:36:50.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.956+0000 7face442b700 1 -- 172.21.15.144:0/1090853351 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facdc1086e0 con 0x7facdc100d70 2024-09-10T11:36:50.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.956+0000 7facd37fe700 1 -- 172.21.15.144:0/1090853351 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7faccc0105d0 con 0x7facdc100d70 2024-09-10T11:36:50.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.956+0000 7facd37fe700 1 --2- 172.21.15.144:0/1090853351 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7facc806e200 0x7facc80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:50.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.957+0000 7facd37fe700 1 -- 172.21.15.144:0/1090853351 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6156+0+0 (secure 0 0 0) 0x7faccc08de70 con 0x7facdc100d70 2024-09-10T11:36:50.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.957+0000 7face21c7700 1 --2- 172.21.15.144:0/1090853351 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7facc806e200 0x7facc80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:50.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.957+0000 7face21c7700 1 --2- 172.21.15.144:0/1090853351 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7facc806e200 0x7facc80706b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7facd8006010 tx=0x7facd801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:50.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:50.961+0000 7facd37fe700 1 -- 172.21.15.144:0/1090853351 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7faccc057300 con 0x7facdc100d70 2024-09-10T11:36:51.117 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:50 smithi178 podman[80149]: 2024-09-10 11:36:50.8033205 +0000 UTC m=+0.628095714 container remove fac80702d990e7118ade68dab0d818eddcb31870b248349261320f86151e1e32 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-activate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T11:36:51.117 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:51 smithi178 podman[80210]: 2024-09-10 11:36:50.908472599 +0000 UTC m=+0.015528456 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:36:51.117 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:51 smithi178 podman[80210]: 2024-09-10 11:36:51.0378863 +0000 UTC m=+0.144942193 container create 400e18bd887448c18a10b414078e80ccb46d00c8b5d1acb2b27a932014aec215 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T11:36:51.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.187+0000 7face442b700 1 -- 172.21.15.144:0/1090853351 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7facdc19b220 con 0x7facc806e200 2024-09-10T11:36:51.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.188+0000 7facd37fe700 1 -- 172.21.15.144:0/1090853351 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7facdc19b220 con 0x7facc806e200 2024-09-10T11:36:51.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.190+0000 7face442b700 1 -- 172.21.15.144:0/1090853351 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7facc806e200 msgr2=0x7facc80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.190+0000 7face442b700 1 --2- 172.21.15.144:0/1090853351 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7facc806e200 0x7facc80706b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7facd8006010 tx=0x7facd801a040 comp rx=0 tx=0).stop 2024-09-10T11:36:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.190+0000 7face442b700 1 -- 172.21.15.144:0/1090853351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facdc100d70 msgr2=0x7facdc1962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.190+0000 7face442b700 1 --2- 172.21.15.144:0/1090853351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facdc100d70 0x7facdc1962a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7faccc00d900 tx=0x7faccc00dcc0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.190+0000 7face442b700 1 -- 172.21.15.144:0/1090853351 shutdown_connections 2024-09-10T11:36:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.190+0000 7face442b700 1 --2- 172.21.15.144:0/1090853351 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7facc806e200 0x7facc80706b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.190+0000 7face442b700 1 --2- 172.21.15.144:0/1090853351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7facdc1003d0 0x7facdc195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.190+0000 7face442b700 1 --2- 172.21.15.144:0/1090853351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7facdc100d70 0x7facdc1962a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.190+0000 7face442b700 1 -- 172.21.15.144:0/1090853351 >> 172.21.15.144:0/1090853351 conn(0x7facdc0fbc90 msgr2=0x7facdc0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.190+0000 7face442b700 1 -- 172.21.15.144:0/1090853351 shutdown_connections 2024-09-10T11:36:51.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.190+0000 7face442b700 1 -- 172.21.15.144:0/1090853351 wait complete. 2024-09-10T11:36:51.202 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:36:51.298 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.297+0000 7fbc1d69d700 1 -- 172.21.15.144:0/3343424204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc181003d0 msgr2=0x7fbc181007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:51.298 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.297+0000 7fbc1d69d700 1 --2- 172.21.15.144:0/3343424204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc181003d0 0x7fbc181007a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fbc08009910 tx=0x7fbc08009c20 comp rx=0 tx=0).stop 2024-09-10T11:36:51.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.297+0000 7fbc1d69d700 1 -- 172.21.15.144:0/3343424204 shutdown_connections 2024-09-10T11:36:51.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.297+0000 7fbc1d69d700 1 --2- 172.21.15.144:0/3343424204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc18100d70 0x7fbc18104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.297+0000 7fbc1d69d700 1 --2- 172.21.15.144:0/3343424204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc181003d0 0x7fbc181007a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.297+0000 7fbc1d69d700 1 -- 172.21.15.144:0/3343424204 >> 172.21.15.144:0/3343424204 conn(0x7fbc180fbc90 msgr2=0x7fbc180fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:51.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.297+0000 7fbc1d69d700 1 -- 172.21.15.144:0/3343424204 shutdown_connections 2024-09-10T11:36:51.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.297+0000 7fbc1d69d700 1 -- 172.21.15.144:0/3343424204 wait complete. 2024-09-10T11:36:51.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.297+0000 7fbc1d69d700 1 Processor -- start 2024-09-10T11:36:51.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.298+0000 7fbc1d69d700 1 -- start start 2024-09-10T11:36:51.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.298+0000 7fbc1d69d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc181003d0 0x7fbc18195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:51.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.298+0000 7fbc1d69d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc18100d70 0x7fbc181962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:51.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.298+0000 7fbc1d69d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc18196980 con 0x7fbc18100d70 2024-09-10T11:36:51.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.298+0000 7fbc1d69d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1819a710 con 0x7fbc181003d0 2024-09-10T11:36:51.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.298+0000 7fbc167fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc18100d70 0x7fbc181962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:51.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.298+0000 7fbc167fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc18100d70 0x7fbc181962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36150/0 (socket says 172.21.15.144:36150) 2024-09-10T11:36:51.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.298+0000 7fbc167fc700 1 -- 172.21.15.144:0/3503139683 learned_addr learned my addr 172.21.15.144:0/3503139683 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:36:51.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.298+0000 7fbc16ffd700 1 --2- 172.21.15.144:0/3503139683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc181003d0 0x7fbc18195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:51.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.298+0000 7fbc167fc700 1 -- 172.21.15.144:0/3503139683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc181003d0 msgr2=0x7fbc18195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:51.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.298+0000 7fbc167fc700 1 --2- 172.21.15.144:0/3503139683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc181003d0 0x7fbc18195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.298+0000 7fbc167fc700 1 -- 172.21.15.144:0/3503139683 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc080095f0 con 0x7fbc18100d70 2024-09-10T11:36:51.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.299+0000 7fbc167fc700 1 --2- 172.21.15.144:0/3503139683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc18100d70 0x7fbc181962a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000d900 tx=0x7fbc0000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:51.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.299+0000 7fbc0ffff700 1 -- 172.21.15.144:0/3503139683 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc00009900 con 0x7fbc18100d70 2024-09-10T11:36:51.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.299+0000 7fbc0ffff700 1 -- 172.21.15.144:0/3503139683 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc00010460 con 0x7fbc18100d70 2024-09-10T11:36:51.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.299+0000 7fbc1d69d700 1 -- 172.21.15.144:0/3503139683 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1819a9f0 con 0x7fbc18100d70 2024-09-10T11:36:51.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.299+0000 7fbc0ffff700 1 -- 172.21.15.144:0/3503139683 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc0000f5f0 con 0x7fbc18100d70 2024-09-10T11:36:51.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.299+0000 7fbc1d69d700 1 -- 172.21.15.144:0/3503139683 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1819af40 con 0x7fbc18100d70 2024-09-10T11:36:51.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.300+0000 7fbc1d69d700 1 -- 172.21.15.144:0/3503139683 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc181086e0 con 0x7fbc18100d70 2024-09-10T11:36:51.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.301+0000 7fbc0ffff700 1 -- 172.21.15.144:0/3503139683 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fbc000105d0 con 0x7fbc18100d70 2024-09-10T11:36:51.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.301+0000 7fbc0ffff700 1 --2- 172.21.15.144:0/3503139683 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc0406e200 0x7fbc040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:51.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.301+0000 7fbc0ffff700 1 -- 172.21.15.144:0/3503139683 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbc0008dbb0 con 0x7fbc18100d70 2024-09-10T11:36:51.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.301+0000 7fbc16ffd700 1 --2- 172.21.15.144:0/3503139683 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc0406e200 0x7fbc040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:51.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.302+0000 7fbc16ffd700 1 --2- 172.21.15.144:0/3503139683 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc0406e200 0x7fbc040706b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbc08006010 tx=0x7fbc0801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:51.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.305+0000 7fbc0ffff700 1 -- 172.21.15.144:0/3503139683 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbc00094080 con 0x7fbc18100d70 2024-09-10T11:36:51.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.537+0000 7fbc1d69d700 1 -- 172.21.15.144:0/3503139683 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc1819b290 con 0x7fbc0406e200 2024-09-10T11:36:51.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.539+0000 7fbc0ffff700 1 -- 172.21.15.144:0/3503139683 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fbc1819b290 con 0x7fbc0406e200 2024-09-10T11:36:51.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.541+0000 7fbc1d69d700 1 -- 172.21.15.144:0/3503139683 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc0406e200 msgr2=0x7fbc040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:51.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.541+0000 7fbc1d69d700 1 --2- 172.21.15.144:0/3503139683 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc0406e200 0x7fbc040706b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbc08006010 tx=0x7fbc0801a040 comp rx=0 tx=0).stop 2024-09-10T11:36:51.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.541+0000 7fbc1d69d700 1 -- 172.21.15.144:0/3503139683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc18100d70 msgr2=0x7fbc181962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:51.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.541+0000 7fbc1d69d700 1 --2- 172.21.15.144:0/3503139683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc18100d70 0x7fbc181962a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000d900 tx=0x7fbc0000dc10 comp rx=0 tx=0).stop 2024-09-10T11:36:51.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.541+0000 7fbc1d69d700 1 -- 172.21.15.144:0/3503139683 shutdown_connections 2024-09-10T11:36:51.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.541+0000 7fbc1d69d700 1 --2- 172.21.15.144:0/3503139683 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc0406e200 0x7fbc040706b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.541+0000 7fbc1d69d700 1 --2- 172.21.15.144:0/3503139683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc181003d0 0x7fbc18195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.541+0000 7fbc1d69d700 1 --2- 172.21.15.144:0/3503139683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc18100d70 0x7fbc181962a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.541+0000 7fbc1d69d700 1 -- 172.21.15.144:0/3503139683 >> 172.21.15.144:0/3503139683 conn(0x7fbc180fbc90 msgr2=0x7fbc180fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:51.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.541+0000 7fbc1d69d700 1 -- 172.21.15.144:0/3503139683 shutdown_connections 2024-09-10T11:36:51.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.541+0000 7fbc1d69d700 1 -- 172.21.15.144:0/3503139683 wait complete. 2024-09-10T11:36:51.617 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:51 smithi178 podman[80210]: 2024-09-10 11:36:51.298247075 +0000 UTC m=+0.405302928 container init 400e18bd887448c18a10b414078e80ccb46d00c8b5d1acb2b27a932014aec215 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T11:36:51.617 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:51 smithi178 podman[80210]: 2024-09-10 11:36:51.301503958 +0000 UTC m=+0.408559806 container start 400e18bd887448c18a10b414078e80ccb46d00c8b5d1acb2b27a932014aec215 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, ceph=True) 2024-09-10T11:36:51.617 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:51 smithi178 ceph-osd[80229]: -- 172.21.15.178:0/7417375 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff64c1c1e0 con 0x55ff63dc5c00 2024-09-10T11:36:51.617 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:51 smithi178 bash[80210]: 400e18bd887448c18a10b414078e80ccb46d00c8b5d1acb2b27a932014aec215 2024-09-10T11:36:51.617 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:51 smithi178 systemd[1]: Started Ceph osd.4 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:36:51.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.650+0000 7f9739395700 1 -- 172.21.15.144:0/1723000478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9734100c80 msgr2=0x7f97341093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:51.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.650+0000 7f9739395700 1 --2- 172.21.15.144:0/1723000478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9734100c80 0x7f97341093e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9724009910 tx=0x7f9724009c20 comp rx=0 tx=0).stop 2024-09-10T11:36:51.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.650+0000 7f9739395700 1 -- 172.21.15.144:0/1723000478 shutdown_connections 2024-09-10T11:36:51.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.650+0000 7f9739395700 1 --2- 172.21.15.144:0/1723000478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9734100c80 0x7f97341093e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.650+0000 7f9739395700 1 --2- 172.21.15.144:0/1723000478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9734100370 0x7f9734100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.650+0000 7f9739395700 1 -- 172.21.15.144:0/1723000478 >> 172.21.15.144:0/1723000478 conn(0x7f97340747e0 msgr2=0x7f9734074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:51.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.650+0000 7f9739395700 1 -- 172.21.15.144:0/1723000478 shutdown_connections 2024-09-10T11:36:51.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.650+0000 7f9739395700 1 -- 172.21.15.144:0/1723000478 wait complete. 2024-09-10T11:36:51.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.651+0000 7f9739395700 1 Processor -- start 2024-09-10T11:36:51.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.651+0000 7f9739395700 1 -- start start 2024-09-10T11:36:51.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.651+0000 7f9739395700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9734100370 0x7f973419ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:51.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.651+0000 7f9739395700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9734100c80 0x7f973419bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:51.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.651+0000 7f9739395700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973419c630 con 0x7f9734100c80 2024-09-10T11:36:51.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.651+0000 7f9739395700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9734195ae0 con 0x7f9734100370 2024-09-10T11:36:51.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.652+0000 7f97327fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9734100c80 0x7f973419bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:51.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.652+0000 7f9732ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9734100370 0x7f973419ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:51.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.652+0000 7f97327fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9734100c80 0x7f973419bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36156/0 (socket says 172.21.15.144:36156) 2024-09-10T11:36:51.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.652+0000 7f97327fc700 1 -- 172.21.15.144:0/2168373760 learned_addr learned my addr 172.21.15.144:0/2168373760 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:36:51.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.652+0000 7f97327fc700 1 -- 172.21.15.144:0/2168373760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9734100370 msgr2=0x7f973419ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:51.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.652+0000 7f97327fc700 1 --2- 172.21.15.144:0/2168373760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9734100370 0x7f973419ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.652+0000 7f97327fc700 1 -- 172.21.15.144:0/2168373760 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97240095f0 con 0x7f9734100c80 2024-09-10T11:36:51.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.652+0000 7f9732ffd700 1 --2- 172.21.15.144:0/2168373760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9734100370 0x7f973419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:36:51.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.652+0000 7f97327fc700 1 --2- 172.21.15.144:0/2168373760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9734100c80 0x7f973419bfa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9724006010 tx=0x7f972400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:51.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.652+0000 7f972bfff700 1 -- 172.21.15.144:0/2168373760 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f972401d070 con 0x7f9734100c80 2024-09-10T11:36:51.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.652+0000 7f9739395700 1 -- 172.21.15.144:0/2168373760 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9734195d60 con 0x7f9734100c80 2024-09-10T11:36:51.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.652+0000 7f972bfff700 1 -- 172.21.15.144:0/2168373760 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9724005dc0 con 0x7f9734100c80 2024-09-10T11:36:51.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.653+0000 7f972bfff700 1 -- 172.21.15.144:0/2168373760 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9724017400 con 0x7f9734100c80 2024-09-10T11:36:51.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.653+0000 7f9739395700 1 -- 172.21.15.144:0/2168373760 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97341962d0 con 0x7f9734100c80 2024-09-10T11:36:51.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.654+0000 7f972bfff700 1 -- 172.21.15.144:0/2168373760 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f972400f460 con 0x7f9734100c80 2024-09-10T11:36:51.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.654+0000 7f9739395700 1 -- 172.21.15.144:0/2168373760 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9734106d40 con 0x7f9734100c80 2024-09-10T11:36:51.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.654+0000 7f972bfff700 1 --2- 172.21.15.144:0/2168373760 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f972006dff0 0x7f97200704a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:51.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.655+0000 7f972bfff700 1 -- 172.21.15.144:0/2168373760 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f972408f300 con 0x7f9734100c80 2024-09-10T11:36:51.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.655+0000 7f9732ffd700 1 --2- 172.21.15.144:0/2168373760 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f972006dff0 0x7f97200704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:51.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.655+0000 7f9732ffd700 1 --2- 172.21.15.144:0/2168373760 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f972006dff0 0x7f97200704a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f971c005950 tx=0x7f971c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:51.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.659+0000 7f972bfff700 1 -- 172.21.15.144:0/2168373760 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f972405c340 con 0x7f9734100c80 2024-09-10T11:36:51.883 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.881+0000 7f9739395700 1 -- 172.21.15.144:0/2168373760 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9734196510 con 0x7f972006dff0 2024-09-10T11:36:51.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.891+0000 7f972bfff700 1 -- 172.21.15.144:0/2168373760 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f9734196510 con 0x7f972006dff0 2024-09-10T11:36:51.892 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:36:51.892 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (5m) 63s ago 15m 25.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:36:51.892 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (15m) 63s ago 15m 8682k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:36:51.892 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (14m) 29s ago 14m 8258k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:36:51.892 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (2m) 63s ago 15m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:36:51.892 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (2m) 29s ago 14m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:36:51.892 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (4m) 63s ago 15m 80.6M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:36:51.892 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (10m) 63s ago 10m 59.9M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:36:51.893 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (10m) 63s ago 10m 126M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:36:51.893 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (10m) 29s ago 10m 62.1M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:36:51.893 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (10m) 29s ago 10m 126M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:36:51.893 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (6m) 63s ago 17m 499M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:36:51.893 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (5m) 29s ago 14m 441M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:36:51.893 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (3m) 63s ago 17m 50.2M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:36:51.893 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (3m) 29s ago 14m 40.0M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:36:51.893 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (5m) 63s ago 15m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:36:51.893 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (5m) 29s ago 14m 15.6M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:36:51.893 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (2m) 63s ago 13m 89.2M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:36:51.893 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (104s) 63s ago 13m 86.5M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:36:51.893 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (68s) 63s ago 12m 12.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:36:51.894 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (32s) 29s ago 12m 13.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:36:51.894 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 starting - - - 2239M 2024-09-10T11:36:51.894 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (11m) 29s ago 11m 199M 2239M 18.2.0 dc2bc1663786 c331ffed30cd 2024-09-10T11:36:51.894 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (5m) 63s ago 15m 77.2M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:36:51.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.893+0000 7f9739395700 1 -- 172.21.15.144:0/2168373760 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f972006dff0 msgr2=0x7f97200704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:51.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.893+0000 7f9739395700 1 --2- 172.21.15.144:0/2168373760 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f972006dff0 0x7f97200704a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f971c005950 tx=0x7f971c00b410 comp rx=0 tx=0).stop 2024-09-10T11:36:51.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.893+0000 7f9739395700 1 -- 172.21.15.144:0/2168373760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9734100c80 msgr2=0x7f973419bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:51.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.893+0000 7f9739395700 1 --2- 172.21.15.144:0/2168373760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9734100c80 0x7f973419bfa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9724006010 tx=0x7f972400bd70 comp rx=0 tx=0).stop 2024-09-10T11:36:51.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.893+0000 7f9739395700 1 -- 172.21.15.144:0/2168373760 shutdown_connections 2024-09-10T11:36:51.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.893+0000 7f9739395700 1 --2- 172.21.15.144:0/2168373760 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f972006dff0 0x7f97200704a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.893+0000 7f9739395700 1 --2- 172.21.15.144:0/2168373760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9734100370 0x7f973419ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.893+0000 7f9739395700 1 --2- 172.21.15.144:0/2168373760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9734100c80 0x7f973419bfa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:51.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.893+0000 7f9739395700 1 -- 172.21.15.144:0/2168373760 >> 172.21.15.144:0/2168373760 conn(0x7f97340747e0 msgr2=0x7f97340fe670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:51.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.893+0000 7f9739395700 1 -- 172.21.15.144:0/2168373760 shutdown_connections 2024-09-10T11:36:51.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.893+0000 7f9739395700 1 -- 172.21.15.144:0/2168373760 wait complete. 2024-09-10T11:36:51.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.998+0000 7faa538a0700 1 -- 172.21.15.144:0/1993361193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4c0ffd30 msgr2=0x7faa4c103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:52.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.998+0000 7faa538a0700 1 --2- 172.21.15.144:0/1993361193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4c0ffd30 0x7faa4c103ce0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7faa48009910 tx=0x7faa48009c20 comp rx=0 tx=0).stop 2024-09-10T11:36:52.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.998+0000 7faa538a0700 1 -- 172.21.15.144:0/1993361193 shutdown_connections 2024-09-10T11:36:52.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.998+0000 7faa538a0700 1 --2- 172.21.15.144:0/1993361193 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4c0ffd30 0x7faa4c103ce0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.998+0000 7faa538a0700 1 --2- 172.21.15.144:0/1993361193 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa4c0ff390 0x7faa4c0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.998+0000 7faa538a0700 1 -- 172.21.15.144:0/1993361193 >> 172.21.15.144:0/1993361193 conn(0x7faa4c074f50 msgr2=0x7faa4c075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:52.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.998+0000 7faa538a0700 1 -- 172.21.15.144:0/1993361193 shutdown_connections 2024-09-10T11:36:52.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.998+0000 7faa538a0700 1 -- 172.21.15.144:0/1993361193 wait complete. 2024-09-10T11:36:52.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.999+0000 7faa538a0700 1 Processor -- start 2024-09-10T11:36:52.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.999+0000 7faa538a0700 1 -- start start 2024-09-10T11:36:52.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.999+0000 7faa538a0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4c0ff390 0x7faa4c195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:52.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.999+0000 7faa538a0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa4c0ffd30 0x7faa4c1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:52.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.999+0000 7faa538a0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c1969b0 con 0x7faa4c0ff390 2024-09-10T11:36:52.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.999+0000 7faa538a0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4c19a740 con 0x7faa4c0ffd30 2024-09-10T11:36:52.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.999+0000 7faa5163c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4c0ff390 0x7faa4c195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:52.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.999+0000 7faa5163c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4c0ff390 0x7faa4c195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36160/0 (socket says 172.21.15.144:36160) 2024-09-10T11:36:52.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.999+0000 7faa5163c700 1 -- 172.21.15.144:0/426497901 learned_addr learned my addr 172.21.15.144:0/426497901 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:36:52.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.999+0000 7faa50e3b700 1 --2- 172.21.15.144:0/426497901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa4c0ffd30 0x7faa4c1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:52.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.999+0000 7faa5163c700 1 -- 172.21.15.144:0/426497901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa4c0ffd30 msgr2=0x7faa4c1962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:52.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.999+0000 7faa5163c700 1 --2- 172.21.15.144:0/426497901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa4c0ffd30 0x7faa4c1962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.999+0000 7faa5163c700 1 -- 172.21.15.144:0/426497901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa480095f0 con 0x7faa4c0ff390 2024-09-10T11:36:52.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:51.999+0000 7faa5163c700 1 --2- 172.21.15.144:0/426497901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4c0ff390 0x7faa4c195d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00d8d0 tx=0x7faa3c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:52.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.000+0000 7faa427fc700 1 -- 172.21.15.144:0/426497901 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa3c0098a0 con 0x7faa4c0ff390 2024-09-10T11:36:52.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.000+0000 7faa427fc700 1 -- 172.21.15.144:0/426497901 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faa3c010460 con 0x7faa4c0ff390 2024-09-10T11:36:52.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.000+0000 7faa538a0700 1 -- 172.21.15.144:0/426497901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa4c19aa20 con 0x7faa4c0ff390 2024-09-10T11:36:52.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.000+0000 7faa427fc700 1 -- 172.21.15.144:0/426497901 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa3c00f5f0 con 0x7faa4c0ff390 2024-09-10T11:36:52.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.000+0000 7faa538a0700 1 -- 172.21.15.144:0/426497901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa4c19af70 con 0x7faa4c0ff390 2024-09-10T11:36:52.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.001+0000 7faa538a0700 1 -- 172.21.15.144:0/426497901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa4c04ea50 con 0x7faa4c0ff390 2024-09-10T11:36:52.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.001+0000 7faa427fc700 1 -- 172.21.15.144:0/426497901 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7faa3c009a00 con 0x7faa4c0ff390 2024-09-10T11:36:52.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.002+0000 7faa427fc700 1 --2- 172.21.15.144:0/426497901 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faa3806e200 0x7faa380706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:52.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.002+0000 7faa427fc700 1 -- 172.21.15.144:0/426497901 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6156+0+0 (secure 0 0 0) 0x7faa3c08dec0 con 0x7faa4c0ff390 2024-09-10T11:36:52.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.002+0000 7faa50e3b700 1 --2- 172.21.15.144:0/426497901 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faa3806e200 0x7faa380706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:52.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.002+0000 7faa50e3b700 1 --2- 172.21.15.144:0/426497901 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faa3806e200 0x7faa380706b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faa4c1973b0 tx=0x7faa4801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:52.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.006+0000 7faa427fc700 1 -- 172.21.15.144:0/426497901 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7faa3c057350 con 0x7faa4c0ff390 2024-09-10T11:36:52.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.292+0000 7faa538a0700 1 -- 172.21.15.144:0/426497901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faa4c19abb0 con 0x7faa4c0ff390 2024-09-10T11:36:52.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.293+0000 7faa427fc700 1 -- 172.21.15.144:0/426497901 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+799 (secure 0 0 0) 0x7faa4c19abb0 con 0x7faa4c0ff390 2024-09-10T11:36:52.294 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:36:52.294 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:36:52.294 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:36:52.294 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:36:52.294 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:36:52.295 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:36:52.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:36:52.295 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:36:52.295 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:36:52.295 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:36:52.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:36:52.295 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:36:52.295 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:36:52.295 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:36:52.295 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:36:52.295 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5, 2024-09-10T11:36:52.295 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 8 2024-09-10T11:36:52.295 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:36:52.295 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:36:52.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.295+0000 7faa538a0700 1 -- 172.21.15.144:0/426497901 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faa3806e200 msgr2=0x7faa380706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:52.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.295+0000 7faa538a0700 1 --2- 172.21.15.144:0/426497901 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faa3806e200 0x7faa380706b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7faa4c1973b0 tx=0x7faa4801a040 comp rx=0 tx=0).stop 2024-09-10T11:36:52.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.295+0000 7faa538a0700 1 -- 172.21.15.144:0/426497901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4c0ff390 msgr2=0x7faa4c195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:52.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.295+0000 7faa538a0700 1 --2- 172.21.15.144:0/426497901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4c0ff390 0x7faa4c195d90 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00d8d0 tx=0x7faa3c00dbe0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.295+0000 7faa538a0700 1 -- 172.21.15.144:0/426497901 shutdown_connections 2024-09-10T11:36:52.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.295+0000 7faa538a0700 1 --2- 172.21.15.144:0/426497901 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faa3806e200 0x7faa380706b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.295+0000 7faa538a0700 1 --2- 172.21.15.144:0/426497901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faa4c0ff390 0x7faa4c195d90 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.295+0000 7faa538a0700 1 --2- 172.21.15.144:0/426497901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faa4c0ffd30 0x7faa4c1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.295+0000 7faa538a0700 1 -- 172.21.15.144:0/426497901 >> 172.21.15.144:0/426497901 conn(0x7faa4c074f50 msgr2=0x7faa4c103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:52.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.295+0000 7faa538a0700 1 -- 172.21.15.144:0/426497901 shutdown_connections 2024-09-10T11:36:52.298 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.295+0000 7faa538a0700 1 -- 172.21.15.144:0/426497901 wait complete. 2024-09-10T11:36:52.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.401+0000 7fad6a671700 1 -- 172.21.15.144:0/1487474893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad640ffca0 msgr2=0x7fad64103c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:52.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.401+0000 7fad6a671700 1 --2- 172.21.15.144:0/1487474893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad640ffca0 0x7fad64103c70 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fad58009910 tx=0x7fad58009c20 comp rx=0 tx=0).stop 2024-09-10T11:36:52.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.401+0000 7fad6a671700 1 -- 172.21.15.144:0/1487474893 shutdown_connections 2024-09-10T11:36:52.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.401+0000 7fad6a671700 1 --2- 172.21.15.144:0/1487474893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad640ffca0 0x7fad64103c70 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.401+0000 7fad6a671700 1 --2- 172.21.15.144:0/1487474893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad640ff300 0x7fad640ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.401+0000 7fad6a671700 1 -- 172.21.15.144:0/1487474893 >> 172.21.15.144:0/1487474893 conn(0x7fad640faba0 msgr2=0x7fad640fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:52.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.401+0000 7fad6a671700 1 -- 172.21.15.144:0/1487474893 shutdown_connections 2024-09-10T11:36:52.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.401+0000 7fad6a671700 1 -- 172.21.15.144:0/1487474893 wait complete. 2024-09-10T11:36:52.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.401+0000 7fad6a671700 1 Processor -- start 2024-09-10T11:36:52.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.401+0000 7fad6a671700 1 -- start start 2024-09-10T11:36:52.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.402+0000 7fad6a671700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad640ff300 0x7fad64196c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:52.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.402+0000 7fad6a671700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad640ffca0 0x7fad641971a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:52.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.402+0000 7fad6a671700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad641977c0 con 0x7fad640ff300 2024-09-10T11:36:52.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.402+0000 7fad6a671700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad64197900 con 0x7fad640ffca0 2024-09-10T11:36:52.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.402+0000 7fad63fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad640ff300 0x7fad64196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:52.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.402+0000 7fad63fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad640ff300 0x7fad64196c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36382/0 (socket says 172.21.15.144:36382) 2024-09-10T11:36:52.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.402+0000 7fad63fff700 1 -- 172.21.15.144:0/4293727459 learned_addr learned my addr 172.21.15.144:0/4293727459 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:36:52.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.402+0000 7fad637fe700 1 --2- 172.21.15.144:0/4293727459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad640ffca0 0x7fad641971a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:52.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.402+0000 7fad63fff700 1 -- 172.21.15.144:0/4293727459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad640ffca0 msgr2=0x7fad641971a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:52.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.402+0000 7fad63fff700 1 --2- 172.21.15.144:0/4293727459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad640ffca0 0x7fad641971a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.402+0000 7fad63fff700 1 -- 172.21.15.144:0/4293727459 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad580095f0 con 0x7fad640ff300 2024-09-10T11:36:52.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.402+0000 7fad63fff700 1 --2- 172.21.15.144:0/4293727459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad640ff300 0x7fad64196c60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fad5400d8d0 tx=0x7fad5400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:52.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.403+0000 7fad617fa700 1 -- 172.21.15.144:0/4293727459 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad540098a0 con 0x7fad640ff300 2024-09-10T11:36:52.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.403+0000 7fad617fa700 1 -- 172.21.15.144:0/4293727459 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fad54010460 con 0x7fad640ff300 2024-09-10T11:36:52.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.403+0000 7fad6a671700 1 -- 172.21.15.144:0/4293727459 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad6419fcd0 con 0x7fad640ff300 2024-09-10T11:36:52.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.403+0000 7fad617fa700 1 -- 172.21.15.144:0/4293727459 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad5400f5f0 con 0x7fad640ff300 2024-09-10T11:36:52.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.403+0000 7fad6a671700 1 -- 172.21.15.144:0/4293727459 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad641a0190 con 0x7fad640ff300 2024-09-10T11:36:52.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.404+0000 7fad6a671700 1 -- 172.21.15.144:0/4293727459 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad6404ea50 con 0x7fad640ff300 2024-09-10T11:36:52.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.404+0000 7fad617fa700 1 -- 172.21.15.144:0/4293727459 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fad54009a00 con 0x7fad640ff300 2024-09-10T11:36:52.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.405+0000 7fad617fa700 1 --2- 172.21.15.144:0/4293727459 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fad4c06e200 0x7fad4c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:52.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.405+0000 7fad617fa700 1 -- 172.21.15.144:0/4293727459 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fad5408ddc0 con 0x7fad640ff300 2024-09-10T11:36:52.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.405+0000 7fad637fe700 1 --2- 172.21.15.144:0/4293727459 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fad4c06e200 0x7fad4c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:52.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.405+0000 7fad637fe700 1 --2- 172.21.15.144:0/4293727459 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fad4c06e200 0x7fad4c0706b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fad58009fd0 tx=0x7fad5801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:52.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.409+0000 7fad617fa700 1 -- 172.21.15.144:0/4293727459 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fad540572d0 con 0x7fad640ff300 2024-09-10T11:36:52.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:52 smithi178 ceph-mon[65940]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:52.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:52.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:52.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:36:52.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:52 smithi178 ceph-mon[65940]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:52.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:52 smithi178 ceph-mon[65940]: pgmap v134: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 61/360 objects degraded (16.944%) 2024-09-10T11:36:52.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:52 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/426497901' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:52.654 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:52 smithi144 ceph-mon[87436]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:52.655 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:52.655 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:52.655 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:36:52.655 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:52 smithi144 ceph-mon[87436]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:52.655 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:52 smithi144 ceph-mon[87436]: pgmap v134: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 61/360 objects degraded (16.944%) 2024-09-10T11:36:52.655 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:52 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/426497901' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:52.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.653+0000 7fad6a671700 1 -- 172.21.15.144:0/4293727459 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fad64069320 con 0x7fad640ff300 2024-09-10T11:36:52.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.653+0000 7fad617fa700 1 -- 172.21.15.144:0/4293727459 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1960 (secure 0 0 0) 0x7fad540207c0 con 0x7fad640ff300 2024-09-10T11:36:52.656 INFO:teuthology.orchestra.run.smithi144.stdout:e23 2024-09-10T11:36:52.656 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:33:11:119769+0000 2024-09-10T11:36:52.656 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:36:52.656 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:36:52.656 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:36:52.656 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:36:52.656 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:36:52.656 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:36:52.656 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 23 2024-09-10T11:36:52.656 INFO:teuthology.orchestra.run.smithi144.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T11:36:52.656 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:36:52.656 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:33:11.119766+0000 2024-09-10T11:36:52.657 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:36:52.657 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:36:52.657 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:36:52.657 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:36:52.657 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:36:52.657 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:36:52.657 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:36:52.657 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:36:52.657 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 47 2024-09-10T11:36:52.657 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:36:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:36:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:36:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:up {0=14488,1=24267} 2024-09-10T11:36:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:failed 2024-09-10T11:36:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:36:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:36:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:36:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:36:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:36:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:36:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:36:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:36:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 14488 members: 14488,24267 2024-09-10T11:36:52.658 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{0:14488} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:36:52.659 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{0:14498} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/1752150537,v1:172.21.15.144:6827/1752150537] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:36:52.659 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:36:52.659 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:36:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:36:52.659 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:36:52.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.656+0000 7fad6a671700 1 -- 172.21.15.144:0/4293727459 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fad4c06e200 msgr2=0x7fad4c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:52.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.656+0000 7fad6a671700 1 --2- 172.21.15.144:0/4293727459 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fad4c06e200 0x7fad4c0706b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fad58009fd0 tx=0x7fad5801a040 comp rx=0 tx=0).stop 2024-09-10T11:36:52.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.656+0000 7fad6a671700 1 -- 172.21.15.144:0/4293727459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad640ff300 msgr2=0x7fad64196c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:52.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.656+0000 7fad6a671700 1 --2- 172.21.15.144:0/4293727459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad640ff300 0x7fad64196c60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fad5400d8d0 tx=0x7fad5400dbe0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.656+0000 7fad6a671700 1 -- 172.21.15.144:0/4293727459 shutdown_connections 2024-09-10T11:36:52.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.656+0000 7fad6a671700 1 --2- 172.21.15.144:0/4293727459 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fad4c06e200 0x7fad4c0706b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.656+0000 7fad6a671700 1 --2- 172.21.15.144:0/4293727459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fad640ff300 0x7fad64196c60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.656+0000 7fad6a671700 1 --2- 172.21.15.144:0/4293727459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fad640ffca0 0x7fad641971a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.656+0000 7fad6a671700 1 -- 172.21.15.144:0/4293727459 >> 172.21.15.144:0/4293727459 conn(0x7fad640faba0 msgr2=0x7fad640fc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:52.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.656+0000 7fad6a671700 1 -- 172.21.15.144:0/4293727459 shutdown_connections 2024-09-10T11:36:52.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.656+0000 7fad6a671700 1 -- 172.21.15.144:0/4293727459 wait complete. 2024-09-10T11:36:52.661 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 23 2024-09-10T11:36:52.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.763+0000 7f751a251700 1 -- 172.21.15.144:0/148815625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75140ffc70 msgr2=0x7f7514103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:52.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.763+0000 7f751a251700 1 --2- 172.21.15.144:0/148815625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75140ffc70 0x7f7514103c40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7504009910 tx=0x7f7504009c20 comp rx=0 tx=0).stop 2024-09-10T11:36:52.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.763+0000 7f751a251700 1 -- 172.21.15.144:0/148815625 shutdown_connections 2024-09-10T11:36:52.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.763+0000 7f751a251700 1 --2- 172.21.15.144:0/148815625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75140ffc70 0x7f7514103c40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.763+0000 7f751a251700 1 --2- 172.21.15.144:0/148815625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75140ff2d0 0x7f75140ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.763+0000 7f751a251700 1 -- 172.21.15.144:0/148815625 >> 172.21.15.144:0/148815625 conn(0x7f75140fabb0 msgr2=0x7f75140fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:52.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.763+0000 7f751a251700 1 -- 172.21.15.144:0/148815625 shutdown_connections 2024-09-10T11:36:52.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.763+0000 7f751a251700 1 -- 172.21.15.144:0/148815625 wait complete. 2024-09-10T11:36:52.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.763+0000 7f751a251700 1 Processor -- start 2024-09-10T11:36:52.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.764+0000 7f751a251700 1 -- start start 2024-09-10T11:36:52.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.764+0000 7f751a251700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75140ff2d0 0x7f751419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:52.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.764+0000 7f751a251700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75140ffc70 0x7f751419af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:52.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.764+0000 7f751a251700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f751419b600 con 0x7f75140ffc70 2024-09-10T11:36:52.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.764+0000 7f751a251700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7514194ab0 con 0x7f75140ff2d0 2024-09-10T11:36:52.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.764+0000 7f7512ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75140ffc70 0x7f751419af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:52.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.764+0000 7f7512ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75140ffc70 0x7f751419af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36390/0 (socket says 172.21.15.144:36390) 2024-09-10T11:36:52.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.764+0000 7f7512ffd700 1 -- 172.21.15.144:0/1331676538 learned_addr learned my addr 172.21.15.144:0/1331676538 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:36:52.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.764+0000 7f75137fe700 1 --2- 172.21.15.144:0/1331676538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75140ff2d0 0x7f751419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:52.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.764+0000 7f7512ffd700 1 -- 172.21.15.144:0/1331676538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75140ff2d0 msgr2=0x7f751419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:52.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.764+0000 7f7512ffd700 1 --2- 172.21.15.144:0/1331676538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75140ff2d0 0x7f751419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:52.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.764+0000 7f7512ffd700 1 -- 172.21.15.144:0/1331676538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75040095f0 con 0x7f75140ffc70 2024-09-10T11:36:52.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.764+0000 7f7512ffd700 1 --2- 172.21.15.144:0/1331676538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75140ffc70 0x7f751419af70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f75040092e0 tx=0x7f750400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:52.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.764+0000 7f7510ff9700 1 -- 172.21.15.144:0/1331676538 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f750401d070 con 0x7f75140ffc70 2024-09-10T11:36:52.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.765+0000 7f7510ff9700 1 -- 172.21.15.144:0/1331676538 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7504005dc0 con 0x7f75140ffc70 2024-09-10T11:36:52.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.765+0000 7f751a251700 1 -- 172.21.15.144:0/1331676538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7514194d30 con 0x7f75140ffc70 2024-09-10T11:36:52.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.765+0000 7f7510ff9700 1 -- 172.21.15.144:0/1331676538 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7504017400 con 0x7f75140ffc70 2024-09-10T11:36:52.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.765+0000 7f751a251700 1 -- 172.21.15.144:0/1331676538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7514195220 con 0x7f75140ffc70 2024-09-10T11:36:52.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.765+0000 7f751a251700 1 -- 172.21.15.144:0/1331676538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751404ea50 con 0x7f75140ffc70 2024-09-10T11:36:52.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.766+0000 7f7510ff9700 1 -- 172.21.15.144:0/1331676538 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f750400f460 con 0x7f75140ffc70 2024-09-10T11:36:52.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.766+0000 7f7510ff9700 1 --2- 172.21.15.144:0/1331676538 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f74fc06e200 0x7f74fc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:52.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.767+0000 7f7510ff9700 1 -- 172.21.15.144:0/1331676538 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f750408f4e0 con 0x7f75140ffc70 2024-09-10T11:36:52.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.767+0000 7f75137fe700 1 --2- 172.21.15.144:0/1331676538 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f74fc06e200 0x7f74fc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:52.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.767+0000 7f75137fe700 1 --2- 172.21.15.144:0/1331676538 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f74fc06e200 0x7f74fc0706b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7508005950 tx=0x7f75080058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:52.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.771+0000 7f7510ff9700 1 -- 172.21.15.144:0/1331676538 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f750405c4a0 con 0x7f75140ffc70 2024-09-10T11:36:52.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.997+0000 7f751a251700 1 -- 172.21.15.144:0/1331676538 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f75141956e0 con 0x7f74fc06e200 2024-09-10T11:36:52.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:52.998+0000 7f7510ff9700 1 -- 172.21.15.144:0/1331676538 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f75141956e0 con 0x7f74fc06e200 2024-09-10T11:36:52.999 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:36:52.999 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:36:53.000 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:36:53.000 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:36:53.000 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:36:53.000 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:36:53.000 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:36:53.000 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:36:53.000 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:36:53.000 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "10/23 daemons upgraded", 2024-09-10T11:36:53.000 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading osd daemons", 2024-09-10T11:36:53.000 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:36:53.000 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:36:53.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.000+0000 7f751a251700 1 -- 172.21.15.144:0/1331676538 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f74fc06e200 msgr2=0x7f74fc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:53.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.000+0000 7f751a251700 1 --2- 172.21.15.144:0/1331676538 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f74fc06e200 0x7f74fc0706b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7508005950 tx=0x7f75080058e0 comp rx=0 tx=0).stop 2024-09-10T11:36:53.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.000+0000 7f751a251700 1 -- 172.21.15.144:0/1331676538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75140ffc70 msgr2=0x7f751419af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:53.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.000+0000 7f751a251700 1 --2- 172.21.15.144:0/1331676538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75140ffc70 0x7f751419af70 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f75040092e0 tx=0x7f750400be20 comp rx=0 tx=0).stop 2024-09-10T11:36:53.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.000+0000 7f751a251700 1 -- 172.21.15.144:0/1331676538 shutdown_connections 2024-09-10T11:36:53.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.000+0000 7f751a251700 1 --2- 172.21.15.144:0/1331676538 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f74fc06e200 0x7f74fc0706b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:53.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.000+0000 7f751a251700 1 --2- 172.21.15.144:0/1331676538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75140ff2d0 0x7f751419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:53.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.000+0000 7f751a251700 1 --2- 172.21.15.144:0/1331676538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75140ffc70 0x7f751419af70 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:53.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.000+0000 7f751a251700 1 -- 172.21.15.144:0/1331676538 >> 172.21.15.144:0/1331676538 conn(0x7f75140fabb0 msgr2=0x7f75140fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:53.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.000+0000 7f751a251700 1 -- 172.21.15.144:0/1331676538 shutdown_connections 2024-09-10T11:36:53.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.001+0000 7f751a251700 1 -- 172.21.15.144:0/1331676538 wait complete. 2024-09-10T11:36:53.107 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.106+0000 7fc9ef2b3700 1 -- 172.21.15.144:0/3332399404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e8100d80 msgr2=0x7fc9e8104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:53.107 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.106+0000 7fc9ef2b3700 1 --2- 172.21.15.144:0/3332399404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e8100d80 0x7fc9e8104d50 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009910 tx=0x7fc9dc009c20 comp rx=0 tx=0).stop 2024-09-10T11:36:53.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.106+0000 7fc9ef2b3700 1 -- 172.21.15.144:0/3332399404 shutdown_connections 2024-09-10T11:36:53.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.106+0000 7fc9ef2b3700 1 --2- 172.21.15.144:0/3332399404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e8100d80 0x7fc9e8104d50 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:53.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.106+0000 7fc9ef2b3700 1 --2- 172.21.15.144:0/3332399404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9e81003e0 0x7fc9e81007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:53.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.106+0000 7fc9ef2b3700 1 -- 172.21.15.144:0/3332399404 >> 172.21.15.144:0/3332399404 conn(0x7fc9e80fbca0 msgr2=0x7fc9e80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:53.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.106+0000 7fc9ef2b3700 1 -- 172.21.15.144:0/3332399404 shutdown_connections 2024-09-10T11:36:53.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.106+0000 7fc9ef2b3700 1 -- 172.21.15.144:0/3332399404 wait complete. 2024-09-10T11:36:53.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.107+0000 7fc9ef2b3700 1 Processor -- start 2024-09-10T11:36:53.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.107+0000 7fc9ef2b3700 1 -- start start 2024-09-10T11:36:53.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.107+0000 7fc9ef2b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e81003e0 0x7fc9e8195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:53.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.107+0000 7fc9ef2b3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9e8100d80 0x7fc9e8196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:53.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.107+0000 7fc9ef2b3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e81969f0 con 0x7fc9e81003e0 2024-09-10T11:36:53.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.107+0000 7fc9ef2b3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e819a780 con 0x7fc9e8100d80 2024-09-10T11:36:53.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.107+0000 7fc9ed04f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e81003e0 0x7fc9e8195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.107+0000 7fc9ed04f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e81003e0 0x7fc9e8195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36410/0 (socket says 172.21.15.144:36410) 2024-09-10T11:36:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.107+0000 7fc9ed04f700 1 -- 172.21.15.144:0/113214142 learned_addr learned my addr 172.21.15.144:0/113214142 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:36:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.107+0000 7fc9ec84e700 1 --2- 172.21.15.144:0/113214142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9e8100d80 0x7fc9e8196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.107+0000 7fc9ed04f700 1 -- 172.21.15.144:0/113214142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9e8100d80 msgr2=0x7fc9e8196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.107+0000 7fc9ed04f700 1 --2- 172.21.15.144:0/113214142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9e8100d80 0x7fc9e8196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.107+0000 7fc9ed04f700 1 -- 172.21.15.144:0/113214142 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc0095f0 con 0x7fc9e81003e0 2024-09-10T11:36:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.107+0000 7fc9ed04f700 1 --2- 172.21.15.144:0/113214142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e81003e0 0x7fc9e8195dd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000ada0 tx=0x7fc9e000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:53.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.108+0000 7fc9e67fc700 1 -- 172.21.15.144:0/113214142 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9e000f070 con 0x7fc9e81003e0 2024-09-10T11:36:53.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.108+0000 7fc9e67fc700 1 -- 172.21.15.144:0/113214142 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc9e0007bf0 con 0x7fc9e81003e0 2024-09-10T11:36:53.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.108+0000 7fc9ef2b3700 1 -- 172.21.15.144:0/113214142 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e819aa60 con 0x7fc9e81003e0 2024-09-10T11:36:53.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.108+0000 7fc9e67fc700 1 -- 172.21.15.144:0/113214142 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9e0005690 con 0x7fc9e81003e0 2024-09-10T11:36:53.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.108+0000 7fc9ef2b3700 1 -- 172.21.15.144:0/113214142 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e819afb0 con 0x7fc9e81003e0 2024-09-10T11:36:53.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.109+0000 7fc9ef2b3700 1 -- 172.21.15.144:0/113214142 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e804ea50 con 0x7fc9e81003e0 2024-09-10T11:36:53.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.109+0000 7fc9e67fc700 1 -- 172.21.15.144:0/113214142 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc9e0005870 con 0x7fc9e81003e0 2024-09-10T11:36:53.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.110+0000 7fc9e67fc700 1 --2- 172.21.15.144:0/113214142 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc9d006e200 0x7fc9d00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:36:53.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.110+0000 7fc9ec84e700 1 --2- 172.21.15.144:0/113214142 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc9d006e200 0x7fc9d00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:36:53.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.110+0000 7fc9e67fc700 1 -- 172.21.15.144:0/113214142 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc9e008d910 con 0x7fc9e81003e0 2024-09-10T11:36:53.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.110+0000 7fc9ec84e700 1 --2- 172.21.15.144:0/113214142 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc9d006e200 0x7fc9d00706b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc9e81973f0 tx=0x7fc9dc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:36:53.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.114+0000 7fc9e67fc700 1 -- 172.21.15.144:0/113214142 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc9e0056e20 con 0x7fc9e81003e0 2024-09-10T11:36:53.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.414+0000 7fc9ef2b3700 1 -- 172.21.15.144:0/113214142 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc9e8197130 con 0x7fc9e81003e0 2024-09-10T11:36:53.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.414+0000 7fc9e67fc700 1 -- 172.21.15.144:0/113214142 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1352 (secure 0 0 0) 0x7fc9e005a4e0 con 0x7fc9e81003e0 2024-09-10T11:36:53.416 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; Degraded data redundancy: 61/360 objects degraded (16.944%), 16 pgs degraded 2024-09-10T11:36:53.416 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:36:53.416 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:36:53.416 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-10T11:36:53.416 INFO:teuthology.orchestra.run.smithi144.stdout: osd.4 (root=default,host=smithi178) is down 2024-09-10T11:36:53.416 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 61/360 objects degraded (16.944%), 16 pgs degraded 2024-09-10T11:36:53.416 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.4 is active+undersized+degraded, acting [1,0] 2024-09-10T11:36:53.416 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.5 is active+undersized+degraded, acting [3,0] 2024-09-10T11:36:53.416 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.6 is active+undersized+degraded, acting [1,3] 2024-09-10T11:36:53.416 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.7 is active+undersized+degraded, acting [3,2] 2024-09-10T11:36:53.417 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.9 is active+undersized+degraded, acting [1,0] 2024-09-10T11:36:53.417 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.a is active+undersized+degraded, acting [1,3] 2024-09-10T11:36:53.417 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.b is active+undersized+degraded, acting [3,5] 2024-09-10T11:36:53.417 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.f is active+undersized+degraded, acting [0,5] 2024-09-10T11:36:53.417 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.11 is active+undersized+degraded, acting [3,1] 2024-09-10T11:36:53.417 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.13 is active+undersized+degraded, acting [0,2] 2024-09-10T11:36:53.417 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.14 is active+undersized+degraded, acting [3,5] 2024-09-10T11:36:53.417 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.18 is active+undersized+degraded, acting [5,3] 2024-09-10T11:36:53.417 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.19 is active+undersized+degraded, acting [0,2] 2024-09-10T11:36:53.417 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1a is active+undersized+degraded, acting [3,5] 2024-09-10T11:36:53.417 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1c is active+undersized+degraded, acting [5,2] 2024-09-10T11:36:53.418 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1f is active+undersized+degraded, acting [0,3] 2024-09-10T11:36:53.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.416+0000 7fc9ef2b3700 1 -- 172.21.15.144:0/113214142 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc9d006e200 msgr2=0x7fc9d00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:53.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.416+0000 7fc9ef2b3700 1 --2- 172.21.15.144:0/113214142 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc9d006e200 0x7fc9d00706b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc9e81973f0 tx=0x7fc9dc01a040 comp rx=0 tx=0).stop 2024-09-10T11:36:53.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.416+0000 7fc9ef2b3700 1 -- 172.21.15.144:0/113214142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e81003e0 msgr2=0x7fc9e8195dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:36:53.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.416+0000 7fc9ef2b3700 1 --2- 172.21.15.144:0/113214142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e81003e0 0x7fc9e8195dd0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000ada0 tx=0x7fc9e000c330 comp rx=0 tx=0).stop 2024-09-10T11:36:53.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.416+0000 7fc9ef2b3700 1 -- 172.21.15.144:0/113214142 shutdown_connections 2024-09-10T11:36:53.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.416+0000 7fc9ef2b3700 1 --2- 172.21.15.144:0/113214142 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc9d006e200 0x7fc9d00706b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:53.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.416+0000 7fc9ef2b3700 1 --2- 172.21.15.144:0/113214142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc9e81003e0 0x7fc9e8195dd0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:53.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.416+0000 7fc9ef2b3700 1 --2- 172.21.15.144:0/113214142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc9e8100d80 0x7fc9e8196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:36:53.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.416+0000 7fc9ef2b3700 1 -- 172.21.15.144:0/113214142 >> 172.21.15.144:0/113214142 conn(0x7fc9e80fbca0 msgr2=0x7fc9e80fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:36:53.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.417+0000 7fc9ef2b3700 1 -- 172.21.15.144:0/113214142 shutdown_connections 2024-09-10T11:36:53.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:36:53.417+0000 7fc9ef2b3700 1 -- 172.21.15.144:0/113214142 wait complete. 2024-09-10T11:36:53.441 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:53 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[80225]: 2024-09-10T11:36:53.172+0000 7fd5848a2640 -1 Falling back to public interface 2024-09-10T11:36:53.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:53 smithi144 ceph-mon[87436]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:53.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:53 smithi144 ceph-mon[87436]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T11:36:53.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:53 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/4293727459' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:36:53.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:53 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/113214142' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:36:53.724 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:53 smithi178 ceph-mon[65940]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:53.724 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:53 smithi178 ceph-mon[65940]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T11:36:53.724 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:53 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/4293727459' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:36:53.724 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:53 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/113214142' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:36:54.785 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:54 smithi178 ceph-mon[65940]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:54.785 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:54 smithi178 ceph-mon[65940]: pgmap v135: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 61/360 objects degraded (16.944%) 2024-09-10T11:36:54.785 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:54.785 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:54 smithi144 ceph-mon[87436]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:36:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:54 smithi144 ceph-mon[87436]: pgmap v135: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 61/360 objects degraded (16.944%) 2024-09-10T11:36:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:56.317 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:55 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:56.317 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:55 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:56.317 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:55 smithi178 ceph-mon[65940]: pgmap v136: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 61/360 objects degraded (16.944%) 2024-09-10T11:36:56.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:55 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:56.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:55 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:56.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:55 smithi144 ceph-mon[87436]: pgmap v136: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 61/360 objects degraded (16.944%) 2024-09-10T11:36:57.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:57 smithi178 ceph-mon[65940]: Health check update: Degraded data redundancy: 61/360 objects degraded (16.944%), 16 pgs degraded (PG_DEGRADED) 2024-09-10T11:36:57.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:57 smithi144 ceph-mon[87436]: Health check update: Degraded data redundancy: 61/360 objects degraded (16.944%), 16 pgs degraded (PG_DEGRADED) 2024-09-10T11:36:58.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:58 smithi178 ceph-mon[65940]: pgmap v137: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 61/360 objects degraded (16.944%) 2024-09-10T11:36:58.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:58.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:58.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:36:58.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:36:58.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:58.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:36:58.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:58.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:58.619 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:58.619 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-10T11:36:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:58 smithi144 ceph-mon[87436]: pgmap v137: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 0 op/s; 61/360 objects degraded (16.944%) 2024-09-10T11:36:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:36:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:36:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:36:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:58.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:36:58.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-10T11:36:59.058 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:58 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[80225]: 2024-09-10T11:36:58.742+0000 7fd5848a2640 -1 osd.4 0 read_superblock omap replica is missing. 2024-09-10T11:36:59.058 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:59 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[80225]: 2024-09-10T11:36:59.057+0000 7fd5848a2640 -1 osd.4 70 log_to_monitors true 2024-09-10T11:36:59.867 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 11:36:59 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[80225]: 2024-09-10T11:36:59.623+0000 7fd57c63c640 -1 osd.4 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:36:59.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:59 smithi178 ceph-mon[65940]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-10T11:36:59.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:59 smithi178 ceph-mon[65940]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-10T11:36:59.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:59 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:59.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:59 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:36:59.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:59 smithi178 ceph-mon[65940]: from='osd.4 [v2:172.21.15.178:6808/2974862475,v1:172.21.15.178:6809/2974862475]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T11:36:59.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:36:59 smithi178 ceph-mon[65940]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T11:36:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:59 smithi144 ceph-mon[87436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-10T11:36:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:59 smithi144 ceph-mon[87436]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-09-10T11:36:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:59 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:36:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:59 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:36:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:59 smithi144 ceph-mon[87436]: from='osd.4 [v2:172.21.15.178:6808/2974862475,v1:172.21.15.178:6809/2974862475]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T11:36:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:36:59 smithi144 ceph-mon[87436]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T11:37:00.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:00 smithi178 ceph-mon[65940]: pgmap v138: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 61/360 objects degraded (16.944%) 2024-09-10T11:37:00.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:00 smithi178 ceph-mon[65940]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T11:37:00.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:00 smithi178 ceph-mon[65940]: osdmap e73: 6 total, 5 up, 6 in 2024-09-10T11:37:00.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:00 smithi178 ceph-mon[65940]: from='osd.4 [v2:172.21.15.178:6808/2974862475,v1:172.21.15.178:6809/2974862475]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:37:00.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:00 smithi178 ceph-mon[65940]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:37:00.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:00 smithi144 ceph-mon[87436]: pgmap v138: 65 pgs: 18 active+undersized, 16 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 0 B/s rd, 1 op/s; 61/360 objects degraded (16.944%) 2024-09-10T11:37:00.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:00 smithi144 ceph-mon[87436]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T11:37:00.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:00 smithi144 ceph-mon[87436]: osdmap e73: 6 total, 5 up, 6 in 2024-09-10T11:37:00.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:00 smithi144 ceph-mon[87436]: from='osd.4 [v2:172.21.15.178:6808/2974862475,v1:172.21.15.178:6809/2974862475]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:37:00.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:00 smithi144 ceph-mon[87436]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:37:01.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:01 smithi144 ceph-mon[87436]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T11:37:01.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:01 smithi144 ceph-mon[87436]: osd.4 [v2:172.21.15.178:6808/2974862475,v1:172.21.15.178:6809/2974862475] boot 2024-09-10T11:37:01.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:01 smithi144 ceph-mon[87436]: osdmap e74: 6 total, 6 up, 6 in 2024-09-10T11:37:01.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:01 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:37:02.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:01 smithi178 ceph-mon[65940]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T11:37:02.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:01 smithi178 ceph-mon[65940]: osd.4 [v2:172.21.15.178:6808/2974862475,v1:172.21.15.178:6809/2974862475] boot 2024-09-10T11:37:02.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:01 smithi178 ceph-mon[65940]: osdmap e74: 6 total, 6 up, 6 in 2024-09-10T11:37:02.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:01 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T11:37:02.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:02 smithi144 ceph-mon[87436]: pgmap v141: 65 pgs: 14 peering, 11 active+undersized, 9 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s; 35/360 objects degraded (9.722%) 2024-09-10T11:37:02.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:02 smithi144 ceph-mon[87436]: osdmap e75: 6 total, 6 up, 6 in 2024-09-10T11:37:02.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:02 smithi144 ceph-mon[87436]: Health check update: Degraded data redundancy: 35/360 objects degraded (9.722%), 9 pgs degraded (PG_DEGRADED) 2024-09-10T11:37:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:02 smithi178 ceph-mon[65940]: pgmap v141: 65 pgs: 14 peering, 11 active+undersized, 9 active+undersized+degraded, 31 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 4 op/s; 35/360 objects degraded (9.722%) 2024-09-10T11:37:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:02 smithi178 ceph-mon[65940]: osdmap e75: 6 total, 6 up, 6 in 2024-09-10T11:37:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:02 smithi178 ceph-mon[65940]: Health check update: Degraded data redundancy: 35/360 objects degraded (9.722%), 9 pgs degraded (PG_DEGRADED) 2024-09-10T11:37:04.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:04 smithi144 ceph-mon[87436]: pgmap v143: 65 pgs: 14 peering, 6 active+undersized, 4 active+undersized+degraded, 41 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s; 10/360 objects degraded (2.778%) 2024-09-10T11:37:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:04 smithi178 ceph-mon[65940]: pgmap v143: 65 pgs: 14 peering, 6 active+undersized, 4 active+undersized+degraded, 41 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 3 op/s; 10/360 objects degraded (2.778%) 2024-09-10T11:37:05.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:05 smithi144 ceph-mon[87436]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 10/360 objects degraded (2.778%), 4 pgs degraded) 2024-09-10T11:37:06.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:05 smithi178 ceph-mon[65940]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 10/360 objects degraded (2.778%), 4 pgs degraded) 2024-09-10T11:37:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:06 smithi144 ceph-mon[87436]: pgmap v144: 65 pgs: 14 peering, 51 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-09-10T11:37:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:06 smithi178 ceph-mon[65940]: pgmap v144: 65 pgs: 14 peering, 51 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.3 KiB/s rd, 2 op/s 2024-09-10T11:37:08.946 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:08 smithi144 ceph-mon[87436]: pgmap v145: 65 pgs: 65 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1 KiB/s rd, 2 op/s 2024-09-10T11:37:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:08 smithi178 ceph-mon[65940]: pgmap v145: 65 pgs: 65 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1 KiB/s rd, 2 op/s 2024-09-10T11:37:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:10 smithi178 ceph-mon[65940]: pgmap v146: 65 pgs: 65 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2 KiB/s rd, 4 op/s 2024-09-10T11:37:11.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:10 smithi144 ceph-mon[87436]: pgmap v146: 65 pgs: 65 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2 KiB/s rd, 4 op/s 2024-09-10T11:37:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:12 smithi144 ceph-mon[87436]: pgmap v147: 65 pgs: 65 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-10T11:37:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:12 smithi178 ceph-mon[65940]: pgmap v147: 65 pgs: 65 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.8 KiB/s rd, 3 op/s 2024-09-10T11:37:13.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-10T11:37:13.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:13.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:37:13.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-10T11:37:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:37:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:14 smithi178 ceph-mon[65940]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-10T11:37:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:14 smithi178 ceph-mon[65940]: Upgrade: osd.5 is safe to restart 2024-09-10T11:37:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:14 smithi178 ceph-mon[65940]: pgmap v148: 65 pgs: 65 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T11:37:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:14 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:14 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T11:37:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:14 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:37:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:14 smithi144 ceph-mon[87436]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-09-10T11:37:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:14 smithi144 ceph-mon[87436]: Upgrade: osd.5 is safe to restart 2024-09-10T11:37:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:14 smithi144 ceph-mon[87436]: pgmap v148: 65 pgs: 65 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T11:37:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T11:37:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:37:15.927 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:15 smithi178 ceph-mon[65940]: Upgrade: Updating osd.5 2024-09-10T11:37:15.928 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:15 smithi178 ceph-mon[65940]: Deploying daemon osd.5 on smithi178 2024-09-10T11:37:16.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:15 smithi144 ceph-mon[87436]: Upgrade: Updating osd.5 2024-09-10T11:37:16.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:15 smithi144 ceph-mon[87436]: Deploying daemon osd.5 on smithi178 2024-09-10T11:37:16.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:16 smithi178 ceph-mon[65940]: pgmap v149: 65 pgs: 65 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T11:37:17.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:16 smithi144 ceph-mon[87436]: pgmap v149: 65 pgs: 65 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 2.0 KiB/s rd, 3 op/s 2024-09-10T11:37:17.951 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:17 smithi178 systemd[1]: Stopping Ceph osd.5 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:37:18.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:17 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[48821]: 2024-09-10T11:37:17.949+0000 7f1d320d8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T11:37:18.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:17 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[48821]: 2024-09-10T11:37:17.949+0000 7f1d320d8700 -1 osd.5 75 *** Got signal Terminated *** 2024-09-10T11:37:18.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:17 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[48821]: 2024-09-10T11:37:17.949+0000 7f1d320d8700 -1 osd.5 75 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T11:37:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:18 smithi144 ceph-mon[87436]: pgmap v150: 65 pgs: 65 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:37:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:18 smithi144 ceph-mon[87436]: osd.5 marked itself down and dead 2024-09-10T11:37:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:18 smithi178 ceph-mon[65940]: pgmap v150: 65 pgs: 65 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T11:37:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:18 smithi178 ceph-mon[65940]: osd.5 marked itself down and dead 2024-09-10T11:37:19.559 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:19 smithi178 podman[84827]: 2024-09-10 11:37:19.262407173 +0000 UTC m=+1.411348816 container died c331ffed30cda2f97a301e0592d9268fb49eff588b7c226e2d0f0843174486d8 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-10T11:37:19.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:19 smithi178 ceph-mon[65940]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T11:37:19.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:19 smithi178 ceph-mon[65940]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-09-10T11:37:19.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:19 smithi178 ceph-mon[65940]: osdmap e76: 6 total, 5 up, 6 in 2024-09-10T11:37:19.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:19 smithi178 podman[84827]: 2024-09-10 11:37:19.658253105 +0000 UTC m=+1.807194737 container cleanup c331ffed30cda2f97a301e0592d9268fb49eff588b7c226e2d0f0843174486d8 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20231212, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-09-10T11:37:19.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:19 smithi178 bash[84827]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5 2024-09-10T11:37:19.982 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:19 smithi144 ceph-mon[87436]: Health check failed: 1 osds down (OSD_DOWN) 2024-09-10T11:37:19.982 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:19 smithi144 ceph-mon[87436]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-09-10T11:37:19.982 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:19 smithi144 ceph-mon[87436]: osdmap e76: 6 total, 5 up, 6 in 2024-09-10T11:37:20.242 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:19 smithi178 podman[84854]: 2024-09-10 11:37:19.910160381 +0000 UTC m=+0.644930628 container remove c331ffed30cda2f97a301e0592d9268fb49eff588b7c226e2d0f0843174486d8 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20231212, ceph=True, CEPH_POINT_RELEASE=-18.2.0, io.buildah.version=1.29.1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T11:37:20.506 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:20 smithi178 podman[84944]: 2024-09-10 11:37:20.144513733 +0000 UTC m=+0.015644733 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:37:20.506 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:20 smithi178 podman[84944]: 2024-09-10 11:37:20.252950954 +0000 UTC m=+0.124081947 container create 38e1127865c907df7dfe3bc613af9e2c3b869c87a65dc6d34155200687a51d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-10T11:37:20.506 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:20 smithi178 podman[84944]: 2024-09-10 11:37:20.505369099 +0000 UTC m=+0.376500093 container init 38e1127865c907df7dfe3bc613af9e2c3b869c87a65dc6d34155200687a51d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T11:37:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:20 smithi178 ceph-mon[65940]: pgmap v152: 65 pgs: 13 stale+active+clean, 52 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-09-10T11:37:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:20 smithi178 ceph-mon[65940]: osdmap e77: 6 total, 5 up, 6 in 2024-09-10T11:37:20.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:20 smithi178 podman[84944]: 2024-09-10 11:37:20.508758204 +0000 UTC m=+0.379889207 container start 38e1127865c907df7dfe3bc613af9e2c3b869c87a65dc6d34155200687a51d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:37:20.868 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:20 smithi178 podman[84944]: 2024-09-10 11:37:20.629882882 +0000 UTC m=+0.501013877 container attach 38e1127865c907df7dfe3bc613af9e2c3b869c87a65dc6d34155200687a51d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T11:37:20.868 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:20 smithi178 podman[84944]: 2024-09-10 11:37:20.69169301 +0000 UTC m=+0.562824011 container died 38e1127865c907df7dfe3bc613af9e2c3b869c87a65dc6d34155200687a51d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T11:37:21.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:20 smithi144 ceph-mon[87436]: pgmap v152: 65 pgs: 13 stale+active+clean, 52 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 1.5 KiB/s rd, 2 op/s 2024-09-10T11:37:21.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:20 smithi144 ceph-mon[87436]: osdmap e77: 6 total, 5 up, 6 in 2024-09-10T11:37:21.617 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:21 smithi178 podman[84967]: 2024-09-10 11:37:21.257157758 +0000 UTC m=+0.560416836 container remove 38e1127865c907df7dfe3bc613af9e2c3b869c87a65dc6d34155200687a51d43 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T11:37:21.617 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:21 smithi178 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.5.service: Deactivated successfully. 2024-09-10T11:37:21.617 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:21 smithi178 systemd[1]: Stopped Ceph osd.5 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:37:21.617 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:21 smithi178 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.5.service: Consumed 55.037s CPU time. 2024-09-10T11:37:21.617 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:21 smithi178 systemd[1]: Starting Ceph osd.5 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T11:37:21.925 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:21 smithi178 ceph-mon[65940]: pgmap v154: 65 pgs: 6 active+undersized, 7 peering, 3 stale+active+clean, 5 active+undersized+degraded, 44 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 16/360 objects degraded (4.444%) 2024-09-10T11:37:21.925 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:21 smithi178 ceph-mon[65940]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-09-10T11:37:21.925 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:21 smithi178 ceph-mon[65940]: Health check failed: Degraded data redundancy: 16/360 objects degraded (4.444%), 5 pgs degraded (PG_DEGRADED) 2024-09-10T11:37:21.925 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:21 smithi178 podman[85065]: 2024-09-10 11:37:21.660992051 +0000 UTC m=+0.095070701 container create 34bf27908b7c0c08b7ba04caaf153bd73d2c21e7e6f905b0bc438ba194403071 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T11:37:21.925 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:21 smithi178 podman[85065]: 2024-09-10 11:37:21.581389149 +0000 UTC m=+0.015467753 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:37:21.925 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:21 smithi178 podman[85065]: 2024-09-10 11:37:21.924866976 +0000 UTC m=+0.358945581 container init 34bf27908b7c0c08b7ba04caaf153bd73d2c21e7e6f905b0bc438ba194403071 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-activate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-10T11:37:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:21 smithi144 ceph-mon[87436]: pgmap v154: 65 pgs: 6 active+undersized, 7 peering, 3 stale+active+clean, 5 active+undersized+degraded, 44 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 16/360 objects degraded (4.444%) 2024-09-10T11:37:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:21 smithi144 ceph-mon[87436]: Health check failed: Reduced data availability: 4 pgs peering (PG_AVAILABILITY) 2024-09-10T11:37:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:21 smithi144 ceph-mon[87436]: Health check failed: Degraded data redundancy: 16/360 objects degraded (4.444%), 5 pgs degraded (PG_DEGRADED) 2024-09-10T11:37:22.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:21 smithi178 podman[85065]: 2024-09-10 11:37:21.928248037 +0000 UTC m=+0.362326650 container start 34bf27908b7c0c08b7ba04caaf153bd73d2c21e7e6f905b0bc438ba194403071 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-10T11:37:22.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:22 smithi178 podman[85065]: 2024-09-10 11:37:22.016563454 +0000 UTC m=+0.450642073 container attach 34bf27908b7c0c08b7ba04caaf153bd73d2c21e7e6f905b0bc438ba194403071 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-activate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T11:37:22.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:22 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-activate[85080]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:37:22.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:22 smithi178 bash[85065]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:37:22.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:22 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-activate[85080]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:37:22.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:22 smithi178 bash[85065]: Running command: /usr/bin/ceph-authtool --gen-print-key 2024-09-10T11:37:23.084 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:22 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-activate[85080]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T11:37:23.084 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:22 smithi178 bash[85065]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T11:37:23.084 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:22 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-activate[85080]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-10T11:37:23.084 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:22 smithi178 bash[85065]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-09-10T11:37:23.346 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:23 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-activate[85080]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-10T11:37:23.346 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:23 smithi178 bash[85065]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-09-10T11:37:23.346 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:23 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-activate[85080]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-10T11:37:23.346 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:23 smithi178 bash[85065]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-09-10T11:37:23.346 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:23 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-activate[85080]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-09-10T11:37:23.346 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:23 smithi178 bash[85065]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-09-10T11:37:23.346 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:23 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-activate[85080]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T11:37:23.346 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:23 smithi178 bash[85065]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-10T11:37:23.346 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:23 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-activate[85080]: --> ceph-volume raw activate successful for osd ID: 5 2024-09-10T11:37:23.347 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:23 smithi178 bash[85065]: --> ceph-volume raw activate successful for osd ID: 5 2024-09-10T11:37:23.347 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:23 smithi178 podman[85065]: 2024-09-10 11:37:23.110745042 +0000 UTC m=+1.544823647 container died 34bf27908b7c0c08b7ba04caaf153bd73d2c21e7e6f905b0bc438ba194403071 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-activate, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-10T11:37:23.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.537+0000 7f4d11476700 1 -- 172.21.15.144:0/606460505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0c1003d0 msgr2=0x7f4d0c1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:23.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.537+0000 7f4d11476700 1 --2- 172.21.15.144:0/606460505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0c1003d0 0x7f4d0c1007a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc009910 tx=0x7f4cfc009c20 comp rx=0 tx=0).stop 2024-09-10T11:37:23.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.537+0000 7f4d11476700 1 -- 172.21.15.144:0/606460505 shutdown_connections 2024-09-10T11:37:23.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.537+0000 7f4d11476700 1 --2- 172.21.15.144:0/606460505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d0c100d70 0x7f4d0c104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:23.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.537+0000 7f4d11476700 1 --2- 172.21.15.144:0/606460505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0c1003d0 0x7f4d0c1007a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:23.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.538+0000 7f4d11476700 1 -- 172.21.15.144:0/606460505 >> 172.21.15.144:0/606460505 conn(0x7f4d0c0fbc90 msgr2=0x7f4d0c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:23.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.538+0000 7f4d11476700 1 -- 172.21.15.144:0/606460505 shutdown_connections 2024-09-10T11:37:23.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.538+0000 7f4d11476700 1 -- 172.21.15.144:0/606460505 wait complete. 2024-09-10T11:37:23.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.538+0000 7f4d11476700 1 Processor -- start 2024-09-10T11:37:23.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.538+0000 7f4d11476700 1 -- start start 2024-09-10T11:37:23.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.539+0000 7f4d11476700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d0c1003d0 0x7f4d0c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:23.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.539+0000 7f4d11476700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0c100d70 0x7f4d0c1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:23.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.539+0000 7f4d11476700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c196980 con 0x7f4d0c100d70 2024-09-10T11:37:23.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.539+0000 7f4d11476700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0c19a710 con 0x7f4d0c1003d0 2024-09-10T11:37:23.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.539+0000 7f4d0affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d0c1003d0 0x7f4d0c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:23.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.539+0000 7f4d0affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d0c1003d0 0x7f4d0c195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.144:47602/0 (socket says 172.21.15.144:47602) 2024-09-10T11:37:23.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.539+0000 7f4d0affd700 1 -- 172.21.15.144:0/3748337426 learned_addr learned my addr 172.21.15.144:0/3748337426 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:37:23.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.539+0000 7f4d0a7fc700 1 --2- 172.21.15.144:0/3748337426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0c100d70 0x7f4d0c1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:23.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.539+0000 7f4d0affd700 1 -- 172.21.15.144:0/3748337426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0c100d70 msgr2=0x7f4d0c1962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:23.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.539+0000 7f4d0affd700 1 --2- 172.21.15.144:0/3748337426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0c100d70 0x7f4d0c1962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:23.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.539+0000 7f4d0affd700 1 -- 172.21.15.144:0/3748337426 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cfc0095f0 con 0x7f4d0c1003d0 2024-09-10T11:37:23.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.539+0000 7f4d0affd700 1 --2- 172.21.15.144:0/3748337426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d0c1003d0 0x7f4d0c195d60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005f50 tx=0x7f4cfc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:23.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.540+0000 7f4d03fff700 1 -- 172.21.15.144:0/3748337426 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cfc01d070 con 0x7f4d0c1003d0 2024-09-10T11:37:23.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.540+0000 7f4d03fff700 1 -- 172.21.15.144:0/3748337426 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cfc00b610 con 0x7f4d0c1003d0 2024-09-10T11:37:23.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.540+0000 7f4d03fff700 1 -- 172.21.15.144:0/3748337426 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cfc022470 con 0x7f4d0c1003d0 2024-09-10T11:37:23.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.540+0000 7f4d11476700 1 -- 172.21.15.144:0/3748337426 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d0c19a990 con 0x7f4d0c1003d0 2024-09-10T11:37:23.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.540+0000 7f4d11476700 1 -- 172.21.15.144:0/3748337426 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d0c19ae80 con 0x7f4d0c1003d0 2024-09-10T11:37:23.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.541+0000 7f4d11476700 1 -- 172.21.15.144:0/3748337426 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0c1086e0 con 0x7f4d0c1003d0 2024-09-10T11:37:23.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.541+0000 7f4d03fff700 1 -- 172.21.15.144:0/3748337426 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4cfc003780 con 0x7f4d0c1003d0 2024-09-10T11:37:23.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.542+0000 7f4d03fff700 1 --2- 172.21.15.144:0/3748337426 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cf806e200 0x7f4cf80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:23.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.542+0000 7f4d03fff700 1 -- 172.21.15.144:0/3748337426 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4cfc08f3b0 con 0x7f4d0c1003d0 2024-09-10T11:37:23.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.542+0000 7f4d0a7fc700 1 --2- 172.21.15.144:0/3748337426 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cf806e200 0x7f4cf80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:23.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.542+0000 7f4d0a7fc700 1 --2- 172.21.15.144:0/3748337426 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cf806e200 0x7f4cf80706b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c197380 tx=0x7f4cf4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:23.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.548+0000 7f4d03fff700 1 -- 172.21.15.144:0/3748337426 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4cfc058840 con 0x7f4d0c1003d0 2024-09-10T11:37:23.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.779+0000 7f4d11476700 1 -- 172.21.15.144:0/3748337426 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d0c02ccf0 con 0x7f4cf806e200 2024-09-10T11:37:23.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.780+0000 7f4d03fff700 1 -- 172.21.15.144:0/3748337426 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f4d0c02ccf0 con 0x7f4cf806e200 2024-09-10T11:37:23.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.782+0000 7f4d11476700 1 -- 172.21.15.144:0/3748337426 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cf806e200 msgr2=0x7f4cf80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:23.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.782+0000 7f4d11476700 1 --2- 172.21.15.144:0/3748337426 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cf806e200 0x7f4cf80706b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c197380 tx=0x7f4cf4009260 comp rx=0 tx=0).stop 2024-09-10T11:37:23.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.782+0000 7f4d11476700 1 -- 172.21.15.144:0/3748337426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d0c1003d0 msgr2=0x7f4d0c195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:23.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.782+0000 7f4d11476700 1 --2- 172.21.15.144:0/3748337426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d0c1003d0 0x7f4d0c195d60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f4cfc005f50 tx=0x7f4cfc00f690 comp rx=0 tx=0).stop 2024-09-10T11:37:23.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.782+0000 7f4d11476700 1 -- 172.21.15.144:0/3748337426 shutdown_connections 2024-09-10T11:37:23.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.782+0000 7f4d11476700 1 --2- 172.21.15.144:0/3748337426 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cf806e200 0x7f4cf80706b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:23.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.782+0000 7f4d11476700 1 --2- 172.21.15.144:0/3748337426 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d0c1003d0 0x7f4d0c195d60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:23.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.782+0000 7f4d11476700 1 --2- 172.21.15.144:0/3748337426 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d0c100d70 0x7f4d0c1962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:23.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.782+0000 7f4d11476700 1 -- 172.21.15.144:0/3748337426 >> 172.21.15.144:0/3748337426 conn(0x7f4d0c0fbc90 msgr2=0x7f4d0c0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:23.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.782+0000 7f4d11476700 1 -- 172.21.15.144:0/3748337426 shutdown_connections 2024-09-10T11:37:23.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.782+0000 7f4d11476700 1 -- 172.21.15.144:0/3748337426 wait complete. 2024-09-10T11:37:23.795 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:37:23.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.889+0000 7f4d3f24f700 1 -- 172.21.15.144:0/3398206005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d380ffd20 msgr2=0x7f4d38103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:23.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.889+0000 7f4d3f24f700 1 --2- 172.21.15.144:0/3398206005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d380ffd20 0x7f4d38103cf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f4d24009910 tx=0x7f4d24009c20 comp rx=0 tx=0).stop 2024-09-10T11:37:23.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.889+0000 7f4d3f24f700 1 -- 172.21.15.144:0/3398206005 shutdown_connections 2024-09-10T11:37:23.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.889+0000 7f4d3f24f700 1 --2- 172.21.15.144:0/3398206005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d380ffd20 0x7f4d38103cf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:23.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.889+0000 7f4d3f24f700 1 --2- 172.21.15.144:0/3398206005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d380ff380 0x7f4d380ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:23.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.889+0000 7f4d3f24f700 1 -- 172.21.15.144:0/3398206005 >> 172.21.15.144:0/3398206005 conn(0x7f4d380fac20 msgr2=0x7f4d380fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:23.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.889+0000 7f4d3f24f700 1 -- 172.21.15.144:0/3398206005 shutdown_connections 2024-09-10T11:37:23.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.889+0000 7f4d3f24f700 1 -- 172.21.15.144:0/3398206005 wait complete. 2024-09-10T11:37:23.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.889+0000 7f4d3f24f700 1 Processor -- start 2024-09-10T11:37:23.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.889+0000 7f4d3f24f700 1 -- start start 2024-09-10T11:37:23.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d3f24f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d380ff380 0x7f4d38100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:23.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d3f24f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d380ffd20 0x7f4d381012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:23.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d3f24f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d38101810 con 0x7f4d380ffd20 2024-09-10T11:37:23.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d3f24f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d38101950 con 0x7f4d380ff380 2024-09-10T11:37:23.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d37fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d380ffd20 0x7f4d381012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:23.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d37fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d380ffd20 0x7f4d381012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37018/0 (socket says 172.21.15.144:37018) 2024-09-10T11:37:23.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d3cfeb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d380ff380 0x7f4d38100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:23.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d37fff700 1 -- 172.21.15.144:0/533175349 learned_addr learned my addr 172.21.15.144:0/533175349 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:37:23.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d37fff700 1 -- 172.21.15.144:0/533175349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d380ff380 msgr2=0x7f4d38100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:23.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d37fff700 1 --2- 172.21.15.144:0/533175349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d380ff380 0x7f4d38100d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:23.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d37fff700 1 -- 172.21.15.144:0/533175349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d240095f0 con 0x7f4d380ffd20 2024-09-10T11:37:23.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d37fff700 1 --2- 172.21.15.144:0/533175349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d380ffd20 0x7f4d381012d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d240059d0 tx=0x7f4d2400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:23.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d35ffb700 1 -- 172.21.15.144:0/533175349 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2401d070 con 0x7f4d380ffd20 2024-09-10T11:37:23.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d3f24f700 1 -- 172.21.15.144:0/533175349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d38101bd0 con 0x7f4d380ffd20 2024-09-10T11:37:23.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d35ffb700 1 -- 172.21.15.144:0/533175349 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4d2400b610 con 0x7f4d380ffd20 2024-09-10T11:37:23.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d35ffb700 1 -- 172.21.15.144:0/533175349 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d24022470 con 0x7f4d380ffd20 2024-09-10T11:37:23.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.890+0000 7f4d3f24f700 1 -- 172.21.15.144:0/533175349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d381a2280 con 0x7f4d380ffd20 2024-09-10T11:37:23.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.891+0000 7f4d3f24f700 1 -- 172.21.15.144:0/533175349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3804ea50 con 0x7f4d380ffd20 2024-09-10T11:37:23.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.892+0000 7f4d35ffb700 1 -- 172.21.15.144:0/533175349 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4d24003710 con 0x7f4d380ffd20 2024-09-10T11:37:23.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.892+0000 7f4d35ffb700 1 --2- 172.21.15.144:0/533175349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4d2806e200 0x7f4d280706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:23.896 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.892+0000 7f4d35ffb700 1 -- 172.21.15.144:0/533175349 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f4d2408f440 con 0x7f4d380ffd20 2024-09-10T11:37:23.896 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.892+0000 7f4d3cfeb700 1 --2- 172.21.15.144:0/533175349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4d2806e200 0x7f4d280706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:23.896 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.893+0000 7f4d3cfeb700 1 --2- 172.21.15.144:0/533175349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4d2806e200 0x7f4d280706b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009c60 tx=0x7f4d2c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:23.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:23.897+0000 7f4d35ffb700 1 -- 172.21.15.144:0/533175349 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4d240588d0 con 0x7f4d380ffd20 2024-09-10T11:37:23.994 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:23 smithi178 podman[85226]: 2024-09-10 11:37:23.661767604 +0000 UTC m=+0.544815634 container remove 34bf27908b7c0c08b7ba04caaf153bd73d2c21e7e6f905b0bc438ba194403071 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-activate, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-10T11:37:23.994 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:23 smithi178 podman[85286]: 2024-09-10 11:37:23.768268348 +0000 UTC m=+0.015772246 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T11:37:23.994 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:23 smithi178 podman[85286]: 2024-09-10 11:37:23.879804822 +0000 UTC m=+0.127308713 container create 4a6125da2507a4954980f0c0d476ac03f0a6927b76a293ae79279fa9f2991c7a (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True) 2024-09-10T11:37:24.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.119+0000 7f4d3f24f700 1 -- 172.21.15.144:0/533175349 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d380695a0 con 0x7f4d2806e200 2024-09-10T11:37:24.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.121+0000 7f4d35ffb700 1 -- 172.21.15.144:0/533175349 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f4d380695a0 con 0x7f4d2806e200 2024-09-10T11:37:24.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.124+0000 7f4d3f24f700 1 -- 172.21.15.144:0/533175349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4d2806e200 msgr2=0x7f4d280706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:24.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.124+0000 7f4d3f24f700 1 --2- 172.21.15.144:0/533175349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4d2806e200 0x7f4d280706b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c009c60 tx=0x7f4d2c009260 comp rx=0 tx=0).stop 2024-09-10T11:37:24.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.124+0000 7f4d3f24f700 1 -- 172.21.15.144:0/533175349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d380ffd20 msgr2=0x7f4d381012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:24.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.124+0000 7f4d3f24f700 1 --2- 172.21.15.144:0/533175349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d380ffd20 0x7f4d381012d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f4d240059d0 tx=0x7f4d2400f690 comp rx=0 tx=0).stop 2024-09-10T11:37:24.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.124+0000 7f4d3f24f700 1 -- 172.21.15.144:0/533175349 shutdown_connections 2024-09-10T11:37:24.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.124+0000 7f4d3f24f700 1 --2- 172.21.15.144:0/533175349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4d2806e200 0x7f4d280706b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.124+0000 7f4d3f24f700 1 --2- 172.21.15.144:0/533175349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4d380ff380 0x7f4d38100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.124+0000 7f4d3f24f700 1 --2- 172.21.15.144:0/533175349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4d380ffd20 0x7f4d381012d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.124+0000 7f4d3f24f700 1 -- 172.21.15.144:0/533175349 >> 172.21.15.144:0/533175349 conn(0x7f4d380fac20 msgr2=0x7f4d38068e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:24.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.124+0000 7f4d3f24f700 1 -- 172.21.15.144:0/533175349 shutdown_connections 2024-09-10T11:37:24.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.124+0000 7f4d3f24f700 1 -- 172.21.15.144:0/533175349 wait complete. 2024-09-10T11:37:24.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.228+0000 7efe1fd18700 1 -- 172.21.15.144:0/3661205981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe180ffd30 msgr2=0x7efe18103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:24.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.228+0000 7efe1fd18700 1 --2- 172.21.15.144:0/3661205981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe180ffd30 0x7efe18103ce0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7efe14009910 tx=0x7efe14009c20 comp rx=0 tx=0).stop 2024-09-10T11:37:24.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.228+0000 7efe1fd18700 1 -- 172.21.15.144:0/3661205981 shutdown_connections 2024-09-10T11:37:24.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.228+0000 7efe1fd18700 1 --2- 172.21.15.144:0/3661205981 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe180ffd30 0x7efe18103ce0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.228+0000 7efe1fd18700 1 --2- 172.21.15.144:0/3661205981 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe180ff390 0x7efe180ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.228+0000 7efe1fd18700 1 -- 172.21.15.144:0/3661205981 >> 172.21.15.144:0/3661205981 conn(0x7efe18074f50 msgr2=0x7efe18075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:24.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.228+0000 7efe1fd18700 1 -- 172.21.15.144:0/3661205981 shutdown_connections 2024-09-10T11:37:24.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.228+0000 7efe1fd18700 1 -- 172.21.15.144:0/3661205981 wait complete. 2024-09-10T11:37:24.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.229+0000 7efe1fd18700 1 Processor -- start 2024-09-10T11:37:24.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.229+0000 7efe1fd18700 1 -- start start 2024-09-10T11:37:24.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.229+0000 7efe1fd18700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe180ff390 0x7efe18195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:24.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.229+0000 7efe1fd18700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe180ffd30 0x7efe181962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:24.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.229+0000 7efe1fd18700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe181969b0 con 0x7efe180ffd30 2024-09-10T11:37:24.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.229+0000 7efe1fd18700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe1819a740 con 0x7efe180ff390 2024-09-10T11:37:24.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.229+0000 7efe1d2b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe180ffd30 0x7efe181962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:24.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.229+0000 7efe1d2b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe180ffd30 0x7efe181962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37030/0 (socket says 172.21.15.144:37030) 2024-09-10T11:37:24.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.229+0000 7efe1d2b3700 1 -- 172.21.15.144:0/3334305395 learned_addr learned my addr 172.21.15.144:0/3334305395 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:37:24.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.229+0000 7efe1dab4700 1 --2- 172.21.15.144:0/3334305395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe180ff390 0x7efe18195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:24.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.229+0000 7efe1d2b3700 1 -- 172.21.15.144:0/3334305395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe180ff390 msgr2=0x7efe18195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:24.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.229+0000 7efe1d2b3700 1 --2- 172.21.15.144:0/3334305395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe180ff390 0x7efe18195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.230+0000 7efe1d2b3700 1 -- 172.21.15.144:0/3334305395 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe140095f0 con 0x7efe180ffd30 2024-09-10T11:37:24.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.230+0000 7efe1dab4700 1 --2- 172.21.15.144:0/3334305395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe180ff390 0x7efe18195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:37:24.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.230+0000 7efe1d2b3700 1 --2- 172.21.15.144:0/3334305395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe180ffd30 0x7efe181962d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efe14000c00 tx=0x7efe1400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:24.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.230+0000 7efe0effd700 1 -- 172.21.15.144:0/3334305395 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe1401d070 con 0x7efe180ffd30 2024-09-10T11:37:24.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.230+0000 7efe0effd700 1 -- 172.21.15.144:0/3334305395 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe14005dc0 con 0x7efe180ffd30 2024-09-10T11:37:24.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.230+0000 7efe0effd700 1 -- 172.21.15.144:0/3334305395 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe14017400 con 0x7efe180ffd30 2024-09-10T11:37:24.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.230+0000 7efe1fd18700 1 -- 172.21.15.144:0/3334305395 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe1819a9c0 con 0x7efe180ffd30 2024-09-10T11:37:24.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.230+0000 7efe1fd18700 1 -- 172.21.15.144:0/3334305395 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe1819aeb0 con 0x7efe180ffd30 2024-09-10T11:37:24.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.231+0000 7efe1fd18700 1 -- 172.21.15.144:0/3334305395 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe1804ea50 con 0x7efe180ffd30 2024-09-10T11:37:24.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.231+0000 7efe0effd700 1 -- 172.21.15.144:0/3334305395 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7efe1400f460 con 0x7efe180ffd30 2024-09-10T11:37:24.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.232+0000 7efe0effd700 1 --2- 172.21.15.144:0/3334305395 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7efe0406e200 0x7efe040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:24.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.232+0000 7efe0effd700 1 -- 172.21.15.144:0/3334305395 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6156+0+0 (secure 0 0 0) 0x7efe1408f370 con 0x7efe180ffd30 2024-09-10T11:37:24.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.232+0000 7efe1dab4700 1 --2- 172.21.15.144:0/3334305395 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7efe0406e200 0x7efe040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:24.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.232+0000 7efe1dab4700 1 --2- 172.21.15.144:0/3334305395 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7efe0406e200 0x7efe040706b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe08005950 tx=0x7efe0800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:24.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.236+0000 7efe0effd700 1 -- 172.21.15.144:0/3334305395 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7efe14058880 con 0x7efe180ffd30 2024-09-10T11:37:24.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:24 smithi178 podman[85286]: 2024-09-10 11:37:24.132835928 +0000 UTC m=+0.380339828 container init 4a6125da2507a4954980f0c0d476ac03f0a6927b76a293ae79279fa9f2991c7a (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-09-10T11:37:24.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:24 smithi178 podman[85286]: 2024-09-10 11:37:24.136159609 +0000 UTC m=+0.383663494 container start 4a6125da2507a4954980f0c0d476ac03f0a6927b76a293ae79279fa9f2991c7a (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-10T11:37:24.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:24 smithi178 ceph-osd[85305]: -- 172.21.15.178:0/3843051704 <== mon.1 v2:172.21.15.178:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f81f3ac1e0 con 0x55f81e559c00 2024-09-10T11:37:24.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:24 smithi178 bash[85286]: 4a6125da2507a4954980f0c0d476ac03f0a6927b76a293ae79279fa9f2991c7a 2024-09-10T11:37:24.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:24 smithi178 systemd[1]: Started Ceph osd.5 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T11:37:24.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.449+0000 7efe1fd18700 1 -- 172.21.15.144:0/3334305395 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efe1819b3e0 con 0x7efe0406e200 2024-09-10T11:37:24.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.458+0000 7efe0effd700 1 -- 172.21.15.144:0/3334305395 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7efe1819b3e0 con 0x7efe0406e200 2024-09-10T11:37:24.460 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:37:24.460 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (5m) 95s ago 16m 25.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:37:24.461 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (16m) 95s ago 16m 8682k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:37:24.461 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (15m) 30s ago 15m 8442k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:37:24.461 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (3m) 95s ago 16m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:37:24.461 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (3m) 30s ago 15m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:37:24.461 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (5m) 95s ago 15m 80.6M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:37:24.461 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (10m) 95s ago 10m 59.9M - 18.2.0 dc2bc1663786 ec9bcb23a4c2 2024-09-10T11:37:24.461 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (10m) 95s ago 10m 126M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:37:24.461 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (10m) 30s ago 10m 61.8M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:37:24.461 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (10m) 30s ago 10m 126M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:37:24.461 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (7m) 95s ago 17m 499M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:37:24.461 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (6m) 30s ago 15m 441M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:37:24.461 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (4m) 95s ago 17m 50.2M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:37:24.461 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (3m) 30s ago 15m 38.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:37:24.462 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (6m) 95s ago 16m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:37:24.462 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (6m) 30s ago 15m 15.8M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:37:24.462 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (2m) 95s ago 14m 89.2M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:37:24.462 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (2m) 95s ago 13m 86.5M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:37:24.462 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (101s) 95s ago 13m 12.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:37:24.462 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (65s) 30s ago 13m 100M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:37:24.462 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (33s) 30s ago 12m 14.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:37:24.462 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 starting - - - 2239M 2024-09-10T11:37:24.462 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (5m) 95s ago 15m 77.2M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:37:24.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.461+0000 7efe1fd18700 1 -- 172.21.15.144:0/3334305395 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7efe0406e200 msgr2=0x7efe040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:24.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.461+0000 7efe1fd18700 1 --2- 172.21.15.144:0/3334305395 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7efe0406e200 0x7efe040706b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe08005950 tx=0x7efe0800b410 comp rx=0 tx=0).stop 2024-09-10T11:37:24.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.461+0000 7efe1fd18700 1 -- 172.21.15.144:0/3334305395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe180ffd30 msgr2=0x7efe181962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:24.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.461+0000 7efe1fd18700 1 --2- 172.21.15.144:0/3334305395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe180ffd30 0x7efe181962d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7efe14000c00 tx=0x7efe1400bd70 comp rx=0 tx=0).stop 2024-09-10T11:37:24.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.461+0000 7efe1fd18700 1 -- 172.21.15.144:0/3334305395 shutdown_connections 2024-09-10T11:37:24.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.461+0000 7efe1fd18700 1 --2- 172.21.15.144:0/3334305395 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7efe0406e200 0x7efe040706b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.461+0000 7efe1fd18700 1 --2- 172.21.15.144:0/3334305395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efe180ff390 0x7efe18195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.461+0000 7efe1fd18700 1 --2- 172.21.15.144:0/3334305395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efe180ffd30 0x7efe181962d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.461+0000 7efe1fd18700 1 -- 172.21.15.144:0/3334305395 >> 172.21.15.144:0/3334305395 conn(0x7efe18074f50 msgr2=0x7efe18103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:24.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.461+0000 7efe1fd18700 1 -- 172.21.15.144:0/3334305395 shutdown_connections 2024-09-10T11:37:24.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.461+0000 7efe1fd18700 1 -- 172.21.15.144:0/3334305395 wait complete. 2024-09-10T11:37:24.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.565+0000 7fa11dfc4700 1 -- 172.21.15.144:0/249338444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1180ff290 msgr2=0x7fa1180ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:24.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.565+0000 7fa11dfc4700 1 --2- 172.21.15.144:0/249338444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1180ff290 0x7fa1180ff660 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa10800b0a0 tx=0x7fa10800b3b0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.565+0000 7fa11dfc4700 1 -- 172.21.15.144:0/249338444 shutdown_connections 2024-09-10T11:37:24.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.565+0000 7fa11dfc4700 1 --2- 172.21.15.144:0/249338444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1180ffba0 0x7fa11810a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.565+0000 7fa11dfc4700 1 --2- 172.21.15.144:0/249338444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1180ff290 0x7fa1180ff660 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.565+0000 7fa11dfc4700 1 -- 172.21.15.144:0/249338444 >> 172.21.15.144:0/249338444 conn(0x7fa118074850 msgr2=0x7fa118074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:24.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.565+0000 7fa11dfc4700 1 -- 172.21.15.144:0/249338444 shutdown_connections 2024-09-10T11:37:24.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.565+0000 7fa11dfc4700 1 -- 172.21.15.144:0/249338444 wait complete. 2024-09-10T11:37:24.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.565+0000 7fa11dfc4700 1 Processor -- start 2024-09-10T11:37:24.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.566+0000 7fa11dfc4700 1 -- start start 2024-09-10T11:37:24.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.566+0000 7fa11dfc4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1180ff290 0x7fa118195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:24.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.566+0000 7fa11dfc4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1180ffba0 0x7fa1181962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:24.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.566+0000 7fa11dfc4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa118196990 con 0x7fa1180ffba0 2024-09-10T11:37:24.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.566+0000 7fa11dfc4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11819a720 con 0x7fa1180ff290 2024-09-10T11:37:24.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.566+0000 7fa116ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1180ffba0 0x7fa1181962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:24.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.566+0000 7fa116ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1180ffba0 0x7fa1181962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37048/0 (socket says 172.21.15.144:37048) 2024-09-10T11:37:24.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.566+0000 7fa116ffd700 1 -- 172.21.15.144:0/3727502785 learned_addr learned my addr 172.21.15.144:0/3727502785 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:37:24.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.566+0000 7fa1177fe700 1 --2- 172.21.15.144:0/3727502785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1180ff290 0x7fa118195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:24.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.566+0000 7fa116ffd700 1 -- 172.21.15.144:0/3727502785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1180ff290 msgr2=0x7fa118195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:24.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.566+0000 7fa116ffd700 1 --2- 172.21.15.144:0/3727502785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1180ff290 0x7fa118195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.566+0000 7fa116ffd700 1 -- 172.21.15.144:0/3727502785 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa108009d00 con 0x7fa1180ffba0 2024-09-10T11:37:24.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.566+0000 7fa1177fe700 1 --2- 172.21.15.144:0/3727502785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1180ff290 0x7fa118195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:37:24.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.566+0000 7fa116ffd700 1 --2- 172.21.15.144:0/3727502785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1180ffba0 0x7fa1181962b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa10000e920 tx=0x7fa10000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:24.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.567+0000 7fa114ff9700 1 -- 172.21.15.144:0/3727502785 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa10000c9d0 con 0x7fa1180ffba0 2024-09-10T11:37:24.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.567+0000 7fa11dfc4700 1 -- 172.21.15.144:0/3727502785 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11819aa00 con 0x7fa1180ffba0 2024-09-10T11:37:24.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.567+0000 7fa114ff9700 1 -- 172.21.15.144:0/3727502785 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa100010430 con 0x7fa1180ffba0 2024-09-10T11:37:24.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.567+0000 7fa114ff9700 1 -- 172.21.15.144:0/3727502785 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa100018630 con 0x7fa1180ffba0 2024-09-10T11:37:24.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.567+0000 7fa11dfc4700 1 -- 172.21.15.144:0/3727502785 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11819af50 con 0x7fa1180ffba0 2024-09-10T11:37:24.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.568+0000 7fa11dfc4700 1 -- 172.21.15.144:0/3727502785 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa118107d10 con 0x7fa1180ffba0 2024-09-10T11:37:24.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.568+0000 7fa114ff9700 1 -- 172.21.15.144:0/3727502785 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa1000105a0 con 0x7fa1180ffba0 2024-09-10T11:37:24.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.569+0000 7fa114ff9700 1 --2- 172.21.15.144:0/3727502785 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa10406e200 0x7fa1040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:24.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.569+0000 7fa114ff9700 1 -- 172.21.15.144:0/3727502785 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa100014070 con 0x7fa1180ffba0 2024-09-10T11:37:24.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.569+0000 7fa1177fe700 1 --2- 172.21.15.144:0/3727502785 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa10406e200 0x7fa1040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:24.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.569+0000 7fa1177fe700 1 --2- 172.21.15.144:0/3727502785 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa10406e200 0x7fa1040706b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa108009fd0 tx=0x7fa1080060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:24.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.573+0000 7fa114ff9700 1 -- 172.21.15.144:0/3727502785 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa1000541b0 con 0x7fa1180ffba0 2024-09-10T11:37:24.864 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.859+0000 7fa11dfc4700 1 -- 172.21.15.144:0/3727502785 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa11804ea50 con 0x7fa1180ffba0 2024-09-10T11:37:24.865 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:24 smithi144 ceph-mon[87436]: pgmap v155: 65 pgs: 10 active+undersized, 7 peering, 11 active+undersized+degraded, 37 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 48/360 objects degraded (13.333%) 2024-09-10T11:37:24.865 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:24 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:24.865 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:24 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:24.865 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:24 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:37:24.865 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.860+0000 7fa114ff9700 1 -- 172.21.15.144:0/3727502785 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+708 (secure 0 0 0) 0x7fa100057730 con 0x7fa1180ffba0 2024-09-10T11:37:24.866 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:37:24.866 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:37:24.866 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:37:24.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:37:24.866 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:37:24.866 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:37:24.866 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:37:24.866 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:37:24.867 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 5 2024-09-10T11:37:24.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:37:24.867 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:37:24.867 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-09-10T11:37:24.867 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:37:24.867 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:37:24.867 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4, 2024-09-10T11:37:24.867 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 9 2024-09-10T11:37:24.867 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:37:24.868 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:37:24.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.862+0000 7fa11dfc4700 1 -- 172.21.15.144:0/3727502785 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa10406e200 msgr2=0x7fa1040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:24.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.862+0000 7fa11dfc4700 1 --2- 172.21.15.144:0/3727502785 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa10406e200 0x7fa1040706b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa108009fd0 tx=0x7fa1080060c0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.862+0000 7fa11dfc4700 1 -- 172.21.15.144:0/3727502785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1180ffba0 msgr2=0x7fa1181962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:24.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.862+0000 7fa11dfc4700 1 --2- 172.21.15.144:0/3727502785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1180ffba0 0x7fa1181962b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fa10000e920 tx=0x7fa10000ec30 comp rx=0 tx=0).stop 2024-09-10T11:37:24.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.862+0000 7fa11dfc4700 1 -- 172.21.15.144:0/3727502785 shutdown_connections 2024-09-10T11:37:24.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.862+0000 7fa11dfc4700 1 --2- 172.21.15.144:0/3727502785 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa10406e200 0x7fa1040706b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.862+0000 7fa11dfc4700 1 --2- 172.21.15.144:0/3727502785 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1180ff290 0x7fa118195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.862+0000 7fa11dfc4700 1 --2- 172.21.15.144:0/3727502785 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1180ffba0 0x7fa1181962b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.862+0000 7fa11dfc4700 1 -- 172.21.15.144:0/3727502785 >> 172.21.15.144:0/3727502785 conn(0x7fa118074850 msgr2=0x7fa118104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:24.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.862+0000 7fa11dfc4700 1 -- 172.21.15.144:0/3727502785 shutdown_connections 2024-09-10T11:37:24.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.862+0000 7fa11dfc4700 1 -- 172.21.15.144:0/3727502785 wait complete. 2024-09-10T11:37:24.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.967+0000 7f7c633e1700 1 -- 172.21.15.144:0/1819092529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c5c1003e0 msgr2=0x7f7c5c1007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:24.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.967+0000 7f7c633e1700 1 --2- 172.21.15.144:0/1819092529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c5c1003e0 0x7f7c5c1007b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c00b0a0 tx=0x7f7c4c00b3b0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.967+0000 7f7c633e1700 1 -- 172.21.15.144:0/1819092529 shutdown_connections 2024-09-10T11:37:24.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.967+0000 7f7c633e1700 1 --2- 172.21.15.144:0/1819092529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c5c100d80 0x7f7c5c104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.967+0000 7f7c633e1700 1 --2- 172.21.15.144:0/1819092529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c5c1003e0 0x7f7c5c1007b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.967+0000 7f7c633e1700 1 -- 172.21.15.144:0/1819092529 >> 172.21.15.144:0/1819092529 conn(0x7f7c5c0fbca0 msgr2=0x7f7c5c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:24.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.967+0000 7f7c633e1700 1 -- 172.21.15.144:0/1819092529 shutdown_connections 2024-09-10T11:37:24.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.967+0000 7f7c633e1700 1 -- 172.21.15.144:0/1819092529 wait complete. 2024-09-10T11:37:24.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.968+0000 7f7c633e1700 1 Processor -- start 2024-09-10T11:37:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.968+0000 7f7c633e1700 1 -- start start 2024-09-10T11:37:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.968+0000 7f7c633e1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c5c1003e0 0x7f7c5c195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.968+0000 7f7c633e1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c5c100d80 0x7f7c5c196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.968+0000 7f7c633e1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c196940 con 0x7f7c5c100d80 2024-09-10T11:37:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.968+0000 7f7c633e1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c5c19a6d0 con 0x7f7c5c1003e0 2024-09-10T11:37:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.968+0000 7f7c6097c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c5c100d80 0x7f7c5c196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.968+0000 7f7c6097c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c5c100d80 0x7f7c5c196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37068/0 (socket says 172.21.15.144:37068) 2024-09-10T11:37:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.968+0000 7f7c6097c700 1 -- 172.21.15.144:0/4222838004 learned_addr learned my addr 172.21.15.144:0/4222838004 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:37:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.968+0000 7f7c6117d700 1 --2- 172.21.15.144:0/4222838004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c5c1003e0 0x7f7c5c195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.968+0000 7f7c6097c700 1 -- 172.21.15.144:0/4222838004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c5c1003e0 msgr2=0x7f7c5c195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.968+0000 7f7c6097c700 1 --2- 172.21.15.144:0/4222838004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c5c1003e0 0x7f7c5c195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:24.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.968+0000 7f7c6097c700 1 -- 172.21.15.144:0/4222838004 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c4c009d00 con 0x7f7c5c100d80 2024-09-10T11:37:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.968+0000 7f7c6097c700 1 --2- 172.21.15.144:0/4222838004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c5c100d80 0x7f7c5c196260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000d8d0 tx=0x7f7c5000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.969+0000 7f7c5a7fc700 1 -- 172.21.15.144:0/4222838004 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c500041f0 con 0x7f7c5c100d80 2024-09-10T11:37:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.969+0000 7f7c5a7fc700 1 -- 172.21.15.144:0/4222838004 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c50007620 con 0x7f7c5c100d80 2024-09-10T11:37:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.969+0000 7f7c633e1700 1 -- 172.21.15.144:0/4222838004 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c5c19a9b0 con 0x7f7c5c100d80 2024-09-10T11:37:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.969+0000 7f7c5a7fc700 1 -- 172.21.15.144:0/4222838004 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c500056b0 con 0x7f7c5c100d80 2024-09-10T11:37:24.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.969+0000 7f7c633e1700 1 -- 172.21.15.144:0/4222838004 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c5c19af00 con 0x7f7c5c100d80 2024-09-10T11:37:24.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.970+0000 7f7c633e1700 1 -- 172.21.15.144:0/4222838004 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c5c1086f0 con 0x7f7c5c100d80 2024-09-10T11:37:24.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.970+0000 7f7c5a7fc700 1 -- 172.21.15.144:0/4222838004 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7c5000f770 con 0x7f7c5c100d80 2024-09-10T11:37:24.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.971+0000 7f7c5a7fc700 1 --2- 172.21.15.144:0/4222838004 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c4406e200 0x7f7c440706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:24.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.971+0000 7f7c5a7fc700 1 -- 172.21.15.144:0/4222838004 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f7c5008ddf0 con 0x7f7c5c100d80 2024-09-10T11:37:24.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.971+0000 7f7c6117d700 1 --2- 172.21.15.144:0/4222838004 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c4406e200 0x7f7c440706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:24.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.971+0000 7f7c6117d700 1 --2- 172.21.15.144:0/4222838004 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c4406e200 0x7f7c440706b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c000f80 tx=0x7f7c4c0060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:24.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:24.975+0000 7f7c5a7fc700 1 -- 172.21.15.144:0/4222838004 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7c50057280 con 0x7f7c5c100d80 2024-09-10T11:37:25.020 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:24 smithi178 ceph-mon[65940]: pgmap v155: 65 pgs: 10 active+undersized, 7 peering, 11 active+undersized+degraded, 37 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 48/360 objects degraded (13.333%) 2024-09-10T11:37:25.020 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:24 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:25.021 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:24 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:25.021 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:24 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:37:25.225 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.223+0000 7f7c633e1700 1 -- 172.21.15.144:0/4222838004 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7c5c04ea50 con 0x7f7c5c100d80 2024-09-10T11:37:25.226 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.224+0000 7f7c5a7fc700 1 -- 172.21.15.144:0/4222838004 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+1960 (secure 0 0 0) 0x7f7c500167c0 con 0x7f7c5c100d80 2024-09-10T11:37:25.226 INFO:teuthology.orchestra.run.smithi144.stdout:e23 2024-09-10T11:37:25.226 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:33:11:119769+0000 2024-09-10T11:37:25.226 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:37:25.226 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:37:25.226 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:37:25.226 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:37:25.227 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:37:25.227 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:37:25.227 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 23 2024-09-10T11:37:25.227 INFO:teuthology.orchestra.run.smithi144.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T11:37:25.227 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:37:25.227 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:33:11.119766+0000 2024-09-10T11:37:25.227 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:37:25.227 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:37:25.227 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:37:25.227 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:37:25.227 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:37:25.227 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:37:25.227 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:37:25.227 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:37:25.228 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 47 2024-09-10T11:37:25.228 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:37:25.228 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:37:25.228 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:37:25.228 INFO:teuthology.orchestra.run.smithi144.stdout:up {0=14488,1=24267} 2024-09-10T11:37:25.228 INFO:teuthology.orchestra.run.smithi144.stdout:failed 2024-09-10T11:37:25.228 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:37:25.228 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:37:25.228 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:37:25.228 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:37:25.228 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:37:25.228 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:37:25.228 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:37:25.229 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:37:25.229 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 14488 members: 14488,24267 2024-09-10T11:37:25.229 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{0:14488} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.144:6828/2138997251,v1:172.21.15.144:6829/2138997251] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:37:25.229 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{0:14498} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/1752150537,v1:172.21.15.144:6827/1752150537] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:37:25.229 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{1:24267} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.178:6824/2986758450,v1:172.21.15.178:6825/2986758450] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:37:25.229 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{1:24273} state up:standby-replay seq 2 join_fscid=1 addr [v2:172.21.15.178:6826/3262518638,v1:172.21.15.178:6827/3262518638] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:37:25.229 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:37:25.229 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:37:25.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.227+0000 7f7c633e1700 1 -- 172.21.15.144:0/4222838004 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c4406e200 msgr2=0x7f7c440706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:25.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.227+0000 7f7c633e1700 1 --2- 172.21.15.144:0/4222838004 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c4406e200 0x7f7c440706b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7c4c000f80 tx=0x7f7c4c0060c0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.227+0000 7f7c633e1700 1 -- 172.21.15.144:0/4222838004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c5c100d80 msgr2=0x7f7c5c196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:25.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.227+0000 7f7c633e1700 1 --2- 172.21.15.144:0/4222838004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c5c100d80 0x7f7c5c196260 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000d8d0 tx=0x7f7c5000dbe0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.227+0000 7f7c633e1700 1 -- 172.21.15.144:0/4222838004 shutdown_connections 2024-09-10T11:37:25.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.227+0000 7f7c633e1700 1 --2- 172.21.15.144:0/4222838004 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c4406e200 0x7f7c440706b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.227+0000 7f7c633e1700 1 --2- 172.21.15.144:0/4222838004 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c5c1003e0 0x7f7c5c195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.227+0000 7f7c633e1700 1 --2- 172.21.15.144:0/4222838004 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c5c100d80 0x7f7c5c196260 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.227+0000 7f7c633e1700 1 -- 172.21.15.144:0/4222838004 >> 172.21.15.144:0/4222838004 conn(0x7f7c5c0fbca0 msgr2=0x7f7c5c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:25.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.227+0000 7f7c633e1700 1 -- 172.21.15.144:0/4222838004 shutdown_connections 2024-09-10T11:37:25.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.227+0000 7f7c633e1700 1 -- 172.21.15.144:0/4222838004 wait complete. 2024-09-10T11:37:25.231 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 23 2024-09-10T11:37:25.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.332+0000 7ff1cf8e3700 1 -- 172.21.15.144:0/690335033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c80ff2c0 msgr2=0x7ff1c80ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:25.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.332+0000 7ff1cf8e3700 1 --2- 172.21.15.144:0/690335033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c80ff2c0 0x7ff1c80ff690 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009910 tx=0x7ff1c4009c20 comp rx=0 tx=0).stop 2024-09-10T11:37:25.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.332+0000 7ff1cf8e3700 1 -- 172.21.15.144:0/690335033 shutdown_connections 2024-09-10T11:37:25.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.332+0000 7ff1cf8e3700 1 --2- 172.21.15.144:0/690335033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1c80ffc60 0x7ff1c8103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.333+0000 7ff1cf8e3700 1 --2- 172.21.15.144:0/690335033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c80ff2c0 0x7ff1c80ff690 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.333+0000 7ff1cf8e3700 1 -- 172.21.15.144:0/690335033 >> 172.21.15.144:0/690335033 conn(0x7ff1c80fab80 msgr2=0x7ff1c80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:25.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.333+0000 7ff1cf8e3700 1 -- 172.21.15.144:0/690335033 shutdown_connections 2024-09-10T11:37:25.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.333+0000 7ff1cf8e3700 1 -- 172.21.15.144:0/690335033 wait complete. 2024-09-10T11:37:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.333+0000 7ff1cf8e3700 1 Processor -- start 2024-09-10T11:37:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.333+0000 7ff1cf8e3700 1 -- start start 2024-09-10T11:37:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.333+0000 7ff1cf8e3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1c80ff2c0 0x7ff1c8195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.333+0000 7ff1cf8e3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c80ffc60 0x7ff1c8196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.333+0000 7ff1cf8e3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c81969e0 con 0x7ff1c80ffc60 2024-09-10T11:37:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.333+0000 7ff1cf8e3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1c819a770 con 0x7ff1c80ff2c0 2024-09-10T11:37:25.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.334+0000 7ff1cce7e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c80ffc60 0x7ff1c8196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.334+0000 7ff1cce7e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c80ffc60 0x7ff1c8196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37084/0 (socket says 172.21.15.144:37084) 2024-09-10T11:37:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.334+0000 7ff1cce7e700 1 -- 172.21.15.144:0/3122461387 learned_addr learned my addr 172.21.15.144:0/3122461387 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:37:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.334+0000 7ff1cd67f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1c80ff2c0 0x7ff1c8195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.334+0000 7ff1cce7e700 1 -- 172.21.15.144:0/3122461387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1c80ff2c0 msgr2=0x7ff1c8195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.334+0000 7ff1cce7e700 1 --2- 172.21.15.144:0/3122461387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1c80ff2c0 0x7ff1c8195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.334+0000 7ff1cce7e700 1 -- 172.21.15.144:0/3122461387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c40095f0 con 0x7ff1c80ffc60 2024-09-10T11:37:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.334+0000 7ff1cce7e700 1 --2- 172.21.15.144:0/3122461387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c80ffc60 0x7ff1c8196300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800e8f0 tx=0x7ff1b800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.334+0000 7ff1be7fc700 1 -- 172.21.15.144:0/3122461387 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1b8009790 con 0x7ff1c80ffc60 2024-09-10T11:37:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.334+0000 7ff1be7fc700 1 -- 172.21.15.144:0/3122461387 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1b800c810 con 0x7ff1c80ffc60 2024-09-10T11:37:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.334+0000 7ff1cf8e3700 1 -- 172.21.15.144:0/3122461387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1c819aa50 con 0x7ff1c80ffc60 2024-09-10T11:37:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.334+0000 7ff1be7fc700 1 -- 172.21.15.144:0/3122461387 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1b8005580 con 0x7ff1c80ffc60 2024-09-10T11:37:25.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.334+0000 7ff1cf8e3700 1 -- 172.21.15.144:0/3122461387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c819afa0 con 0x7ff1c80ffc60 2024-09-10T11:37:25.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.335+0000 7ff1cf8e3700 1 -- 172.21.15.144:0/3122461387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1c804ea50 con 0x7ff1c80ffc60 2024-09-10T11:37:25.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.336+0000 7ff1be7fc700 1 -- 172.21.15.144:0/3122461387 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff1b800c980 con 0x7ff1c80ffc60 2024-09-10T11:37:25.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.336+0000 7ff1be7fc700 1 --2- 172.21.15.144:0/3122461387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff1b406e200 0x7ff1b40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:25.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.336+0000 7ff1be7fc700 1 -- 172.21.15.144:0/3122461387 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff1b8014070 con 0x7ff1c80ffc60 2024-09-10T11:37:25.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.336+0000 7ff1cd67f700 1 --2- 172.21.15.144:0/3122461387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff1b406e200 0x7ff1b40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:25.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.337+0000 7ff1cd67f700 1 --2- 172.21.15.144:0/3122461387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff1b406e200 0x7ff1b40706b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40059d0 tx=0x7ff1c4005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:25.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.341+0000 7ff1be7fc700 1 -- 172.21.15.144:0/3122461387 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff1b80576c0 con 0x7ff1c80ffc60 2024-09-10T11:37:25.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.562+0000 7ff1cf8e3700 1 -- 172.21.15.144:0/3122461387 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1c819b480 con 0x7ff1b406e200 2024-09-10T11:37:25.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.563+0000 7ff1be7fc700 1 -- 172.21.15.144:0/3122461387 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7ff1c819b480 con 0x7ff1b406e200 2024-09-10T11:37:25.565 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:37:25.565 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:37:25.566 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:37:25.566 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:37:25.566 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:37:25.566 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:37:25.566 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:37:25.566 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:37:25.566 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:37:25.566 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "11/23 daemons upgraded", 2024-09-10T11:37:25.566 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading osd daemons", 2024-09-10T11:37:25.566 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:37:25.566 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:37:25.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.566+0000 7ff1cf8e3700 1 -- 172.21.15.144:0/3122461387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff1b406e200 msgr2=0x7ff1b40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:25.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.566+0000 7ff1cf8e3700 1 --2- 172.21.15.144:0/3122461387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff1b406e200 0x7ff1b40706b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff1c40059d0 tx=0x7ff1c4005940 comp rx=0 tx=0).stop 2024-09-10T11:37:25.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.566+0000 7ff1cf8e3700 1 -- 172.21.15.144:0/3122461387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c80ffc60 msgr2=0x7ff1c8196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:25.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.566+0000 7ff1cf8e3700 1 --2- 172.21.15.144:0/3122461387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c80ffc60 0x7ff1c8196300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff1b800e8f0 tx=0x7ff1b800ec00 comp rx=0 tx=0).stop 2024-09-10T11:37:25.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.566+0000 7ff1cf8e3700 1 -- 172.21.15.144:0/3122461387 shutdown_connections 2024-09-10T11:37:25.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.566+0000 7ff1cf8e3700 1 --2- 172.21.15.144:0/3122461387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff1b406e200 0x7ff1b40706b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.566+0000 7ff1cf8e3700 1 --2- 172.21.15.144:0/3122461387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1c80ff2c0 0x7ff1c8195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.566+0000 7ff1cf8e3700 1 --2- 172.21.15.144:0/3122461387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1c80ffc60 0x7ff1c8196300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.566+0000 7ff1cf8e3700 1 -- 172.21.15.144:0/3122461387 >> 172.21.15.144:0/3122461387 conn(0x7ff1c80fab80 msgr2=0x7ff1c8068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:25.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.566+0000 7ff1cf8e3700 1 -- 172.21.15.144:0/3122461387 shutdown_connections 2024-09-10T11:37:25.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.566+0000 7ff1cf8e3700 1 -- 172.21.15.144:0/3122461387 wait complete. 2024-09-10T11:37:25.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.674+0000 7f89d9ba5700 1 -- 172.21.15.144:0/769037223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89d40ff2c0 msgr2=0x7f89d40ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:25.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.674+0000 7f89d9ba5700 1 --2- 172.21.15.144:0/769037223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89d40ff2c0 0x7f89d40ff690 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f89c4009910 tx=0x7f89c4009c20 comp rx=0 tx=0).stop 2024-09-10T11:37:25.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.674+0000 7f89d9ba5700 1 -- 172.21.15.144:0/769037223 shutdown_connections 2024-09-10T11:37:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.674+0000 7f89d9ba5700 1 --2- 172.21.15.144:0/769037223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89d40ffc60 0x7f89d4103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.674+0000 7f89d9ba5700 1 --2- 172.21.15.144:0/769037223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89d40ff2c0 0x7f89d40ff690 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.674+0000 7f89d9ba5700 1 -- 172.21.15.144:0/769037223 >> 172.21.15.144:0/769037223 conn(0x7f89d40fab80 msgr2=0x7f89d40fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.674+0000 7f89d9ba5700 1 -- 172.21.15.144:0/769037223 shutdown_connections 2024-09-10T11:37:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.674+0000 7f89d9ba5700 1 -- 172.21.15.144:0/769037223 wait complete. 2024-09-10T11:37:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.674+0000 7f89d9ba5700 1 Processor -- start 2024-09-10T11:37:25.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.675+0000 7f89d9ba5700 1 -- start start 2024-09-10T11:37:25.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.675+0000 7f89d9ba5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89d40ff2c0 0x7f89d4195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:25.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.675+0000 7f89d9ba5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89d40ffc60 0x7f89d4196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:25.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.675+0000 7f89d9ba5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d41969e0 con 0x7f89d40ff2c0 2024-09-10T11:37:25.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.675+0000 7f89d9ba5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89d419a770 con 0x7f89d40ffc60 2024-09-10T11:37:25.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.675+0000 7f89d37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89d40ff2c0 0x7f89d4195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:25.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.675+0000 7f89d37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89d40ff2c0 0x7f89d4195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37112/0 (socket says 172.21.15.144:37112) 2024-09-10T11:37:25.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.675+0000 7f89d37fe700 1 -- 172.21.15.144:0/1952671021 learned_addr learned my addr 172.21.15.144:0/1952671021 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:37:25.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.675+0000 7f89d2ffd700 1 --2- 172.21.15.144:0/1952671021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89d40ffc60 0x7f89d4196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:25.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.675+0000 7f89d37fe700 1 -- 172.21.15.144:0/1952671021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89d40ffc60 msgr2=0x7f89d4196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:25.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.675+0000 7f89d37fe700 1 --2- 172.21.15.144:0/1952671021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89d40ffc60 0x7f89d4196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.684 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.675+0000 7f89d37fe700 1 -- 172.21.15.144:0/1952671021 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c40095f0 con 0x7f89d40ff2c0 2024-09-10T11:37:25.684 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.675+0000 7f89d37fe700 1 --2- 172.21.15.144:0/1952671021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89d40ff2c0 0x7f89d4195dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f89c400b5c0 tx=0x7f89c400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:25.684 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.675+0000 7f89d0ff9700 1 -- 172.21.15.144:0/1952671021 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89c401d070 con 0x7f89d40ff2c0 2024-09-10T11:37:25.684 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.676+0000 7f89d9ba5700 1 -- 172.21.15.144:0/1952671021 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89d419a9f0 con 0x7f89d40ff2c0 2024-09-10T11:37:25.684 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.676+0000 7f89d0ff9700 1 -- 172.21.15.144:0/1952671021 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f89c4005e00 con 0x7f89d40ff2c0 2024-09-10T11:37:25.684 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.676+0000 7f89d0ff9700 1 -- 172.21.15.144:0/1952671021 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f89c4017400 con 0x7f89d40ff2c0 2024-09-10T11:37:25.685 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.676+0000 7f89d9ba5700 1 -- 172.21.15.144:0/1952671021 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89d419aee0 con 0x7f89d40ff2c0 2024-09-10T11:37:25.685 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.677+0000 7f89d9ba5700 1 -- 172.21.15.144:0/1952671021 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89d404ea50 con 0x7f89d40ff2c0 2024-09-10T11:37:25.685 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.677+0000 7f89d0ff9700 1 -- 172.21.15.144:0/1952671021 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f89c400f6f0 con 0x7f89d40ff2c0 2024-09-10T11:37:25.685 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.678+0000 7f89d0ff9700 1 --2- 172.21.15.144:0/1952671021 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f89bc06e200 0x7f89bc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:25.685 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.678+0000 7f89d0ff9700 1 -- 172.21.15.144:0/1952671021 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f89c408f490 con 0x7f89d40ff2c0 2024-09-10T11:37:25.686 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.678+0000 7f89d2ffd700 1 --2- 172.21.15.144:0/1952671021 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f89bc06e200 0x7f89bc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:25.686 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.678+0000 7f89d2ffd700 1 --2- 172.21.15.144:0/1952671021 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f89bc06e200 0x7f89bc0706b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89d41973e0 tx=0x7f89c80073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:25.687 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.682+0000 7f89d0ff9700 1 -- 172.21.15.144:0/1952671021 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f89c4058920 con 0x7f89d40ff2c0 2024-09-10T11:37:25.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:25 smithi178 ceph-mon[65940]: from='client.44283 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:25.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:25 smithi178 ceph-mon[65940]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:25.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:25 smithi178 ceph-mon[65940]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:25.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:25 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3727502785' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:25.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:25 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/4222838004' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:37:25.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:25 smithi144 ceph-mon[87436]: from='client.44283 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:25.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:25 smithi144 ceph-mon[87436]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:25.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:25 smithi144 ceph-mon[87436]: from='client.34354 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:25.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:25 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3727502785' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:25.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:25 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/4222838004' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:37:25.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.966+0000 7f89d9ba5700 1 -- 172.21.15.144:0/1952671021 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f89d419b340 con 0x7f89d40ff2c0 2024-09-10T11:37:25.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.967+0000 7f89d0ff9700 1 -- 172.21.15.144:0/1952671021 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1480 (secure 0 0 0) 0x7f89c405bfe0 con 0x7f89d40ff2c0 2024-09-10T11:37:25.969 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data; 1 osds down; all OSDs are running squid or later but require_osd_release < squid; Degraded data redundancy: 55/360 objects degraded (15.278%), 14 pgs degraded 2024-09-10T11:37:25.969 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:37:25.969 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:37:25.969 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] OSD_DOWN: 1 osds down 2024-09-10T11:37:25.969 INFO:teuthology.orchestra.run.smithi144.stdout: osd.5 (root=default,host=smithi178) is down 2024-09-10T11:37:25.969 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] OSD_UPGRADE_FINISHED: all OSDs are running squid or later but require_osd_release < squid 2024-09-10T11:37:25.969 INFO:teuthology.orchestra.run.smithi144.stdout: all OSDs are running squid or later but require_osd_release < squid 2024-09-10T11:37:25.969 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 55/360 objects degraded (15.278%), 14 pgs degraded 2024-09-10T11:37:25.970 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.2 is active+undersized+degraded, acting [1,0] 2024-09-10T11:37:25.970 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.3 is active+undersized+degraded, acting [2,1] 2024-09-10T11:37:25.970 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.8 is active+undersized+degraded, acting [3,0] 2024-09-10T11:37:25.970 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.b is active+undersized+degraded, acting [3,4] 2024-09-10T11:37:25.970 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.f is active+undersized+degraded, acting [4,0] 2024-09-10T11:37:25.970 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.14 is active+undersized+degraded, acting [3,4] 2024-09-10T11:37:25.970 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.16 is active+undersized+degraded, acting [3,2] 2024-09-10T11:37:25.970 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.17 is active+undersized+degraded, acting [2,1] 2024-09-10T11:37:25.970 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.18 is active+undersized+degraded, acting [4,3] 2024-09-10T11:37:25.970 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1a is active+undersized+degraded, acting [3,4] 2024-09-10T11:37:25.970 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1b is active+undersized+degraded, acting [1,0] 2024-09-10T11:37:25.970 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1c is active+undersized+degraded, acting [4,2] 2024-09-10T11:37:25.970 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1d is active+undersized+degraded, acting [3,0] 2024-09-10T11:37:25.970 INFO:teuthology.orchestra.run.smithi144.stdout: pg 2.1e is active+undersized+degraded, acting [2,0] 2024-09-10T11:37:25.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.969+0000 7f89d9ba5700 1 -- 172.21.15.144:0/1952671021 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f89bc06e200 msgr2=0x7f89bc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:25.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.969+0000 7f89d9ba5700 1 --2- 172.21.15.144:0/1952671021 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f89bc06e200 0x7f89bc0706b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f89d41973e0 tx=0x7f89c80073d0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.969+0000 7f89d9ba5700 1 -- 172.21.15.144:0/1952671021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89d40ff2c0 msgr2=0x7f89d4195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:25.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.969+0000 7f89d9ba5700 1 --2- 172.21.15.144:0/1952671021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89d40ff2c0 0x7f89d4195dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f89c400b5c0 tx=0x7f89c400bd70 comp rx=0 tx=0).stop 2024-09-10T11:37:25.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.969+0000 7f89d9ba5700 1 -- 172.21.15.144:0/1952671021 shutdown_connections 2024-09-10T11:37:25.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.969+0000 7f89d9ba5700 1 --2- 172.21.15.144:0/1952671021 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f89bc06e200 0x7f89bc0706b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.969+0000 7f89d9ba5700 1 --2- 172.21.15.144:0/1952671021 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89d40ff2c0 0x7f89d4195dc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.969+0000 7f89d9ba5700 1 --2- 172.21.15.144:0/1952671021 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89d40ffc60 0x7f89d4196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:25.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.969+0000 7f89d9ba5700 1 -- 172.21.15.144:0/1952671021 >> 172.21.15.144:0/1952671021 conn(0x7f89d40fab80 msgr2=0x7f89d4068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:25.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.969+0000 7f89d9ba5700 1 -- 172.21.15.144:0/1952671021 shutdown_connections 2024-09-10T11:37:25.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:25.969+0000 7f89d9ba5700 1 -- 172.21.15.144:0/1952671021 wait complete. 2024-09-10T11:37:26.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:26 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T11:37:26.049+0000 7f82f8c25640 -1 Falling back to public interface 2024-09-10T11:37:26.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:26 smithi178 ceph-mon[65940]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:26.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:26 smithi178 ceph-mon[65940]: pgmap v156: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/360 objects degraded (15.278%) 2024-09-10T11:37:26.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:26 smithi178 ceph-mon[65940]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-09-10T11:37:26.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:26 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1952671021' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:37:26.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:26 smithi144 ceph-mon[87436]: from='client.34366 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:26.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:26 smithi144 ceph-mon[87436]: pgmap v156: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/360 objects degraded (15.278%) 2024-09-10T11:37:26.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:26 smithi144 ceph-mon[87436]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 4 pgs peering) 2024-09-10T11:37:26.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:26 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1952671021' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:37:28.268 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:28.268 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:28.269 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:28 smithi178 ceph-mon[65940]: Health check update: Degraded data redundancy: 55/360 objects degraded (15.278%), 14 pgs degraded (PG_DEGRADED) 2024-09-10T11:37:28.269 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:28 smithi178 ceph-mon[65940]: pgmap v157: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/360 objects degraded (15.278%) 2024-09-10T11:37:28.269 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:28.269 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:28.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:28.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:28.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:28 smithi144 ceph-mon[87436]: Health check update: Degraded data redundancy: 55/360 objects degraded (15.278%), 14 pgs degraded (PG_DEGRADED) 2024-09-10T11:37:28.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:28 smithi144 ceph-mon[87436]: pgmap v157: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/360 objects degraded (15.278%) 2024-09-10T11:37:28.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:28.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:29.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:29 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:29.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:29 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:37:29.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:29 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:29.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:29 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:37:30.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:30 smithi178 ceph-mon[65940]: pgmap v158: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/360 objects degraded (15.278%) 2024-09-10T11:37:30.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:30 smithi144 ceph-mon[87436]: pgmap v158: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/360 objects degraded (15.278%) 2024-09-10T11:37:31.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T11:37:31.047+0000 7f82f8c25640 -1 osd.5 0 read_superblock omap replica is missing. 2024-09-10T11:37:31.367 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T11:37:31.291+0000 7f82f8c25640 -1 osd.5 75 log_to_monitors true 2024-09-10T11:37:32.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:32.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:32.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:37:32.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:37:32.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:32.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:37:32.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:32.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:32.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:32.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:32.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:32.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-09-10T11:37:32.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-09-10T11:37:32.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-09-10T11:37:32.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-09-10T11:37:32.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-09-10T11:37:32.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-09-10T11:37:32.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-09-10T11:37:32.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-09-10T11:37:32.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-09-10T11:37:32.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-09-10T11:37:32.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-09-10T11:37:32.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-09-10T11:37:32.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-09-10T11:37:32.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='osd.5 [v2:172.21.15.178:6816/4171640630,v1:172.21.15.178:6817/4171640630]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T11:37:32.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:31 smithi178 ceph-mon[65940]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T11:37:32.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:32.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:32.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:37:32.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:37:32.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:32.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:37:32.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:32.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:32.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:32.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:32.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:32.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-09-10T11:37:32.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-09-10T11:37:32.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-09-10T11:37:32.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-09-10T11:37:32.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-09-10T11:37:32.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-09-10T11:37:32.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-09-10T11:37:32.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-09-10T11:37:32.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-09-10T11:37:32.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-09-10T11:37:32.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-09-10T11:37:32.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-09-10T11:37:32.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-09-10T11:37:32.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='osd.5 [v2:172.21.15.178:6816/4171640630,v1:172.21.15.178:6817/4171640630]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T11:37:32.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:31 smithi144 ceph-mon[87436]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T11:37:32.947 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 11:37:32 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T11:37:32.660+0000 7f82ec9b7640 -1 osd.5 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T11:37:32.947 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:32 smithi178 ceph-mon[65940]: Upgrade: Setting container_image for all osd 2024-09-10T11:37:32.947 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:32 smithi178 ceph-mon[65940]: Upgrade: Setting require_osd_release to 19 squid 2024-09-10T11:37:32.947 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:32 smithi178 ceph-mon[65940]: pgmap v159: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/360 objects degraded (15.278%) 2024-09-10T11:37:32.947 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:32 smithi178 ceph-mon[65940]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-09-10T11:37:32.947 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:32 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-09-10T11:37:32.947 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:32 smithi178 ceph-mon[65940]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T11:37:32.948 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:32 smithi178 ceph-mon[65940]: from='osd.5 [v2:172.21.15.178:6816/4171640630,v1:172.21.15.178:6817/4171640630]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:37:32.948 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:32 smithi178 ceph-mon[65940]: osdmap e78: 6 total, 5 up, 6 in 2024-09-10T11:37:32.948 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:32 smithi178 ceph-mon[65940]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:37:32.948 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:32 smithi178 ceph-mon[65940]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-09-10T11:37:32.948 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:32 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:32.948 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:32 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-09-10T11:37:32.948 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:32 smithi178 ceph-mon[65940]: from='osd.5 ' entity='osd.5' 2024-09-10T11:37:32.948 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:32 smithi178 ceph-mon[65940]: osdmap e79: 6 total, 5 up, 6 in 2024-09-10T11:37:33.048 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:32 smithi144 ceph-mon[87436]: Upgrade: Setting container_image for all osd 2024-09-10T11:37:33.048 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:32 smithi144 ceph-mon[87436]: Upgrade: Setting require_osd_release to 19 squid 2024-09-10T11:37:33.048 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:32 smithi144 ceph-mon[87436]: pgmap v159: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 309 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 55/360 objects degraded (15.278%) 2024-09-10T11:37:33.049 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:32 smithi144 ceph-mon[87436]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-09-10T11:37:33.049 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:32 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-09-10T11:37:33.049 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:32 smithi144 ceph-mon[87436]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T11:37:33.049 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:32 smithi144 ceph-mon[87436]: from='osd.5 [v2:172.21.15.178:6816/4171640630,v1:172.21.15.178:6817/4171640630]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:37:33.049 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:32 smithi144 ceph-mon[87436]: osdmap e78: 6 total, 5 up, 6 in 2024-09-10T11:37:33.050 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:32 smithi144 ceph-mon[87436]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi178", "root=default"]}]: dispatch 2024-09-10T11:37:33.050 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:32 smithi144 ceph-mon[87436]: Upgrade: Disabling standby-replay for filesystem cephfs 2024-09-10T11:37:33.050 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:32 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:33.050 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:32 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]: dispatch 2024-09-10T11:37:33.050 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:32 smithi144 ceph-mon[87436]: from='osd.5 ' entity='osd.5' 2024-09-10T11:37:33.050 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:32 smithi144 ceph-mon[87436]: osdmap e79: 6 total, 5 up, 6 in 2024-09-10T11:37:33.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:37:33.047+0000 7f80c2950640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:37:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-10T11:37:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-09-10T11:37:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2024-09-10T11:37:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:37:34.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:37:34.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:37:34.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:34.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:37:34.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:34.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:34.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:34.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:34.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-09-10T11:37:34.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T11:37:34.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T11:37:34.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T11:37:34.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:37:34.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-10T11:37:34.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T11:37:34.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: osd.5 [v2:172.21.15.178:6816/4171640630,v1:172.21.15.178:6817/4171640630] boot 2024-09-10T11:37:34.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: osdmap e80: 6 total, 6 up, 6 in 2024-09-10T11:37:34.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] up:boot 2024-09-10T11:37:34.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.144:6826/4013687114,v1:172.21.15.144:6827/4013687114] up:boot 2024-09-10T11:37:34.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-09-10T11:37:34.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:37:34.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.czjrsa"}]: dispatch 2024-09-10T11:37:34.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:37:34.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:33 smithi178 ceph-mon[65940]: pgmap v163: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 55/360 objects degraded (15.278%) 2024-09-10T11:37:34.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-09-10T11:37:34.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "0"}]': finished 2024-09-10T11:37:34.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: fsmap cephfs:2 {0=cephfs.smithi144.jaoyvs=up:active,1=cephfs.smithi178.nmhrmh=up:active} 2024-09-10T11:37:34.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:37:34.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:37:34.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:37:34.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:34.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:37:34.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:34.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:34.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:34.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:34.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-09-10T11:37:34.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-10T11:37:34.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-09-10T11:37:34.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-10T11:37:34.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-10T11:37:34.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-09-10T11:37:34.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-10T11:37:34.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: osd.5 [v2:172.21.15.178:6816/4171640630,v1:172.21.15.178:6817/4171640630] boot 2024-09-10T11:37:34.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: osdmap e80: 6 total, 6 up, 6 in 2024-09-10T11:37:34.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] up:boot 2024-09-10T11:37:34.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.144:6826/4013687114,v1:172.21.15.144:6827/4013687114] up:boot 2024-09-10T11:37:34.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-09-10T11:37:34.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:37:34.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.czjrsa"}]: dispatch 2024-09-10T11:37:34.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T11:37:34.186 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:33 smithi144 ceph-mon[87436]: pgmap v163: 65 pgs: 14 active+undersized, 14 active+undersized+degraded, 37 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s; 55/360 objects degraded (15.278%) 2024-09-10T11:37:35.163 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:35 smithi144 ceph-mon[87436]: osdmap e81: 6 total, 6 up, 6 in 2024-09-10T11:37:35.163 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:35 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.178:6824/559040332,v1:172.21.15.178:6825/559040332] up:boot 2024-09-10T11:37:35.163 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:35 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.144:6828/1022937641,v1:172.21.15.144:6829/1022937641] up:boot 2024-09-10T11:37:35.164 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:35 smithi144 ceph-mon[87436]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T11:37:35.164 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:35 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jaoyvs"}]: dispatch 2024-09-10T11:37:35.164 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:35 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.nmhrmh"}]: dispatch 2024-09-10T11:37:35.164 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:35 smithi144 ceph-mon[87436]: Upgrade: Updating mds.cephfs.smithi144.bfyvwv 2024-09-10T11:37:35.164 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:35 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:35.164 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:35 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.bfyvwv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:37:35.164 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:35 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:37:35.164 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:35 smithi144 ceph-mon[87436]: Deploying daemon mds.cephfs.smithi144.bfyvwv on smithi144 2024-09-10T11:37:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:35 smithi178 ceph-mon[65940]: osdmap e81: 6 total, 6 up, 6 in 2024-09-10T11:37:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:35 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.178:6824/559040332,v1:172.21.15.178:6825/559040332] up:boot 2024-09-10T11:37:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:35 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.144:6828/1022937641,v1:172.21.15.144:6829/1022937641] up:boot 2024-09-10T11:37:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:35 smithi178 ceph-mon[65940]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T11:37:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:35 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jaoyvs"}]: dispatch 2024-09-10T11:37:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:35 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.nmhrmh"}]: dispatch 2024-09-10T11:37:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:35 smithi178 ceph-mon[65940]: Upgrade: Updating mds.cephfs.smithi144.bfyvwv 2024-09-10T11:37:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:35 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:35 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.bfyvwv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:37:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:35 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:37:35.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:35 smithi178 ceph-mon[65940]: Deploying daemon mds.cephfs.smithi144.bfyvwv on smithi144 2024-09-10T11:37:36.157 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:36 smithi144 ceph-mon[87436]: pgmap v165: 65 pgs: 13 active+undersized, 12 active+undersized+degraded, 40 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 51/360 objects degraded (14.167%) 2024-09-10T11:37:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:36 smithi178 ceph-mon[65940]: pgmap v165: 65 pgs: 13 active+undersized, 12 active+undersized+degraded, 40 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 1 op/s; 51/360 objects degraded (14.167%) 2024-09-10T11:37:37.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:37 smithi178 ceph-mon[65940]: Health check update: Degraded data redundancy: 51/360 objects degraded (14.167%), 12 pgs degraded (PG_DEGRADED) 2024-09-10T11:37:37.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:37 smithi144 ceph-mon[87436]: Health check update: Degraded data redundancy: 51/360 objects degraded (14.167%), 12 pgs degraded (PG_DEGRADED) 2024-09-10T11:37:38.327 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:38 smithi178 ceph-mon[65940]: osdmap e82: 6 total, 6 up, 6 in 2024-09-10T11:37:38.327 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:38 smithi178 ceph-mon[65940]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T11:37:38.327 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:38 smithi178 ceph-mon[65940]: pgmap v167: 65 pgs: 2 active+undersized, 5 active+undersized+degraded, 58 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 28/360 objects degraded (7.778%) 2024-09-10T11:37:38.328 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:38 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:37:38.335 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:38 smithi144 ceph-mon[87436]: osdmap e82: 6 total, 6 up, 6 in 2024-09-10T11:37:38.335 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:38 smithi144 ceph-mon[87436]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T11:37:38.335 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:38 smithi144 ceph-mon[87436]: pgmap v167: 65 pgs: 2 active+undersized, 5 active+undersized+degraded, 58 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 28/360 objects degraded (7.778%) 2024-09-10T11:37:38.335 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:38 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:37:39.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:39 smithi144 ceph-mon[87436]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/360 objects degraded (7.778%), 5 pgs degraded) 2024-09-10T11:37:40.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:39 smithi178 ceph-mon[65940]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 28/360 objects degraded (7.778%), 5 pgs degraded) 2024-09-10T11:37:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:40 smithi178 ceph-mon[65940]: pgmap v168: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:41.132 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:40 smithi144 ceph-mon[87436]: pgmap v168: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:42.981 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:42 smithi144 ceph-mon[87436]: pgmap v169: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:42.982 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:42.982 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:42.982 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:42 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:37:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:42 smithi178 ceph-mon[65940]: pgmap v169: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:42 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:37:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:43 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] up:boot 2024-09-10T11:37:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:43 smithi144 ceph-mon[87436]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T11:37:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:37:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:37:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:43 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] up:boot 2024-09-10T11:37:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:43 smithi178 ceph-mon[65940]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T11:37:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T11:37:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:37:45.016 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:44 smithi144 ceph-mon[87436]: pgmap v170: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:44 smithi178 ceph-mon[65940]: pgmap v170: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:46 smithi178 ceph-mon[65940]: pgmap v171: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:47.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:46 smithi144 ceph-mon[87436]: pgmap v171: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:48.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:48 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:48.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:48 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:48.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:48 smithi178 ceph-mon[65940]: pgmap v172: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:48.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:48 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:48.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:48 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:48.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:48 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:48.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:48 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:48.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:48 smithi144 ceph-mon[87436]: pgmap v172: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:48.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:48 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:48.394 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:48 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:49.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:49 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:49 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:50.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:50 smithi144 ceph-mon[87436]: pgmap v173: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:50 smithi178 ceph-mon[65940]: pgmap v173: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:52.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:52.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:52.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:37:52.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:37:52.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:52.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:37:52.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:52.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:52.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:52.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:52.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:52 smithi144 ceph-mon[87436]: pgmap v174: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:52.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:52.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:52.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:37:52.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:37:52.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:52.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:37:52.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:52.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:52.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:52.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:52.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:52 smithi178 ceph-mon[65940]: pgmap v174: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:53.899 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:53 smithi144 ceph-mon[87436]: Upgrade: Updating mds.cephfs.smithi144.jaoyvs 2024-09-10T11:37:53.900 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:53.900 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jaoyvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:37:53.900 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:37:53.900 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:53 smithi144 ceph-mon[87436]: Deploying daemon mds.cephfs.smithi144.jaoyvs on smithi144 2024-09-10T11:37:54.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:53 smithi178 ceph-mon[65940]: Upgrade: Updating mds.cephfs.smithi144.jaoyvs 2024-09-10T11:37:54.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:54.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi144.jaoyvs", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:37:54.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:37:54.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:53 smithi178 ceph-mon[65940]: Deploying daemon mds.cephfs.smithi144.jaoyvs on smithi144 2024-09-10T11:37:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:54 smithi144 ceph-mon[87436]: pgmap v175: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:54 smithi178 ceph-mon[65940]: pgmap v175: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:55.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:55 smithi144 ceph-mon[87436]: osdmap e83: 6 total, 6 up, 6 in 2024-09-10T11:37:55.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:55 smithi144 ceph-mon[87436]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T11:37:56.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.091+0000 7f49ce67b700 1 -- 172.21.15.144:0/1482555130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49c81003d0 msgr2=0x7f49c81007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:56.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.091+0000 7f49ce67b700 1 --2- 172.21.15.144:0/1482555130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49c81003d0 0x7f49c81007a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009910 tx=0x7f49b8009c20 comp rx=0 tx=0).stop 2024-09-10T11:37:56.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.091+0000 7f49ce67b700 1 -- 172.21.15.144:0/1482555130 shutdown_connections 2024-09-10T11:37:56.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.091+0000 7f49ce67b700 1 --2- 172.21.15.144:0/1482555130 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49c8100d70 0x7f49c8104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:56.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.091+0000 7f49ce67b700 1 --2- 172.21.15.144:0/1482555130 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49c81003d0 0x7f49c81007a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:56.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.091+0000 7f49ce67b700 1 -- 172.21.15.144:0/1482555130 >> 172.21.15.144:0/1482555130 conn(0x7f49c80fbc90 msgr2=0x7f49c80fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:56.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.091+0000 7f49ce67b700 1 -- 172.21.15.144:0/1482555130 shutdown_connections 2024-09-10T11:37:56.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.091+0000 7f49ce67b700 1 -- 172.21.15.144:0/1482555130 wait complete. 2024-09-10T11:37:56.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.092+0000 7f49ce67b700 1 Processor -- start 2024-09-10T11:37:56.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.092+0000 7f49ce67b700 1 -- start start 2024-09-10T11:37:56.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.092+0000 7f49ce67b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49c81003d0 0x7f49c8195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:56.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.092+0000 7f49ce67b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49c8100d70 0x7f49c81962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:56.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.092+0000 7f49ce67b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c81968f0 con 0x7f49c8100d70 2024-09-10T11:37:56.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.092+0000 7f49ce67b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c8196a30 con 0x7f49c81003d0 2024-09-10T11:37:56.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.093+0000 7f49c77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49c8100d70 0x7f49c81962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:56.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.093+0000 7f49c77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49c8100d70 0x7f49c81962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59226/0 (socket says 172.21.15.144:59226) 2024-09-10T11:37:56.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.093+0000 7f49c77fe700 1 -- 172.21.15.144:0/733300260 learned_addr learned my addr 172.21.15.144:0/733300260 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:37:56.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.093+0000 7f49c7fff700 1 --2- 172.21.15.144:0/733300260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49c81003d0 0x7f49c8195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:56.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.093+0000 7f49c77fe700 1 -- 172.21.15.144:0/733300260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49c81003d0 msgr2=0x7f49c8195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:56.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.093+0000 7f49c77fe700 1 --2- 172.21.15.144:0/733300260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49c81003d0 0x7f49c8195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:56.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.093+0000 7f49c77fe700 1 -- 172.21.15.144:0/733300260 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b80095f0 con 0x7f49c8100d70 2024-09-10T11:37:56.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.093+0000 7f49c77fe700 1 --2- 172.21.15.144:0/733300260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49c8100d70 0x7f49c81962a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00c960 tx=0x7f49bc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:56.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.093+0000 7f49c57fa700 1 -- 172.21.15.144:0/733300260 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49bc007a70 con 0x7f49c8100d70 2024-09-10T11:37:56.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.093+0000 7f49ce67b700 1 -- 172.21.15.144:0/733300260 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c819a880 con 0x7f49c8100d70 2024-09-10T11:37:56.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.093+0000 7f49c57fa700 1 -- 172.21.15.144:0/733300260 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f49bc007bd0 con 0x7f49c8100d70 2024-09-10T11:37:56.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.093+0000 7f49c57fa700 1 -- 172.21.15.144:0/733300260 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49bc018630 con 0x7f49c8100d70 2024-09-10T11:37:56.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.093+0000 7f49ce67b700 1 -- 172.21.15.144:0/733300260 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c819add0 con 0x7f49c8100d70 2024-09-10T11:37:56.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.094+0000 7f49ce67b700 1 -- 172.21.15.144:0/733300260 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c81086e0 con 0x7f49c8100d70 2024-09-10T11:37:56.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.095+0000 7f49c57fa700 1 -- 172.21.15.144:0/733300260 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f49bc00f450 con 0x7f49c8100d70 2024-09-10T11:37:56.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.095+0000 7f49c57fa700 1 --2- 172.21.15.144:0/733300260 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f49b006e200 0x7f49b00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:56.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.096+0000 7f49c57fa700 1 -- 172.21.15.144:0/733300260 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f49bc08e000 con 0x7f49c8100d70 2024-09-10T11:37:56.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.096+0000 7f49c7fff700 1 --2- 172.21.15.144:0/733300260 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f49b006e200 0x7f49b00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:56.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.096+0000 7f49c7fff700 1 --2- 172.21.15.144:0/733300260 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f49b006e200 0x7f49b00706b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f49b8006010 tx=0x7f49b801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:56.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.100+0000 7f49c57fa700 1 -- 172.21.15.144:0/733300260 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f49bc0573e0 con 0x7f49c8100d70 2024-09-10T11:37:56.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:55 smithi178 ceph-mon[65940]: osdmap e83: 6 total, 6 up, 6 in 2024-09-10T11:37:56.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:55 smithi178 ceph-mon[65940]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T11:37:56.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.324+0000 7f49ce67b700 1 -- 172.21.15.144:0/733300260 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f49c802cd40 con 0x7f49b006e200 2024-09-10T11:37:56.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.326+0000 7f49c57fa700 1 -- 172.21.15.144:0/733300260 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f49c802cd40 con 0x7f49b006e200 2024-09-10T11:37:56.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.328+0000 7f49ce67b700 1 -- 172.21.15.144:0/733300260 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f49b006e200 msgr2=0x7f49b00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:56.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.328+0000 7f49ce67b700 1 --2- 172.21.15.144:0/733300260 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f49b006e200 0x7f49b00706b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f49b8006010 tx=0x7f49b801a040 comp rx=0 tx=0).stop 2024-09-10T11:37:56.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.328+0000 7f49ce67b700 1 -- 172.21.15.144:0/733300260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49c8100d70 msgr2=0x7f49c81962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:56.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.328+0000 7f49ce67b700 1 --2- 172.21.15.144:0/733300260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49c8100d70 0x7f49c81962a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00c960 tx=0x7f49bc00cc70 comp rx=0 tx=0).stop 2024-09-10T11:37:56.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.328+0000 7f49ce67b700 1 -- 172.21.15.144:0/733300260 shutdown_connections 2024-09-10T11:37:56.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.328+0000 7f49ce67b700 1 --2- 172.21.15.144:0/733300260 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f49b006e200 0x7f49b00706b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:56.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.328+0000 7f49ce67b700 1 --2- 172.21.15.144:0/733300260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49c81003d0 0x7f49c8195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:56.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.328+0000 7f49ce67b700 1 --2- 172.21.15.144:0/733300260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49c8100d70 0x7f49c81962a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:56.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.328+0000 7f49ce67b700 1 -- 172.21.15.144:0/733300260 >> 172.21.15.144:0/733300260 conn(0x7f49c80fbc90 msgr2=0x7f49c80fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:56.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.328+0000 7f49ce67b700 1 -- 172.21.15.144:0/733300260 shutdown_connections 2024-09-10T11:37:56.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.328+0000 7f49ce67b700 1 -- 172.21.15.144:0/733300260 wait complete. 2024-09-10T11:37:56.340 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:37:56.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.435+0000 7fddcfbfc700 1 -- 172.21.15.144:0/1119230100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc80ffd30 msgr2=0x7fddc8103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:56.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.435+0000 7fddcfbfc700 1 --2- 172.21.15.144:0/1119230100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc80ffd30 0x7fddc8103ce0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fddb8009910 tx=0x7fddb8009c20 comp rx=0 tx=0).stop 2024-09-10T11:37:56.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.435+0000 7fddcfbfc700 1 -- 172.21.15.144:0/1119230100 shutdown_connections 2024-09-10T11:37:56.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.435+0000 7fddcfbfc700 1 --2- 172.21.15.144:0/1119230100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc80ffd30 0x7fddc8103ce0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:56.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.435+0000 7fddcfbfc700 1 --2- 172.21.15.144:0/1119230100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddc80ff390 0x7fddc80ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:56.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.435+0000 7fddcfbfc700 1 -- 172.21.15.144:0/1119230100 >> 172.21.15.144:0/1119230100 conn(0x7fddc8074f50 msgr2=0x7fddc8075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:56.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.435+0000 7fddcfbfc700 1 -- 172.21.15.144:0/1119230100 shutdown_connections 2024-09-10T11:37:56.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.435+0000 7fddcfbfc700 1 -- 172.21.15.144:0/1119230100 wait complete. 2024-09-10T11:37:56.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.436+0000 7fddcfbfc700 1 Processor -- start 2024-09-10T11:37:56.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.436+0000 7fddcfbfc700 1 -- start start 2024-09-10T11:37:56.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.436+0000 7fddcfbfc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddc80ff390 0x7fddc8196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:56.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.436+0000 7fddcfbfc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc80ffd30 0x7fddc8197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:56.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.436+0000 7fddcfbfc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc8197740 con 0x7fddc80ffd30 2024-09-10T11:37:56.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.436+0000 7fddcfbfc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddc8197880 con 0x7fddc80ff390 2024-09-10T11:37:56.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.436+0000 7fddcd197700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc80ffd30 0x7fddc8197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:56.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.436+0000 7fddcd197700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc80ffd30 0x7fddc8197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59240/0 (socket says 172.21.15.144:59240) 2024-09-10T11:37:56.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.436+0000 7fddcd197700 1 -- 172.21.15.144:0/3741785704 learned_addr learned my addr 172.21.15.144:0/3741785704 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:37:56.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.436+0000 7fddcd998700 1 --2- 172.21.15.144:0/3741785704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddc80ff390 0x7fddc8196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:56.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.437+0000 7fddcd197700 1 -- 172.21.15.144:0/3741785704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddc80ff390 msgr2=0x7fddc8196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:56.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.437+0000 7fddcd197700 1 --2- 172.21.15.144:0/3741785704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddc80ff390 0x7fddc8196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:56.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.437+0000 7fddcd197700 1 -- 172.21.15.144:0/3741785704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb80095f0 con 0x7fddc80ffd30 2024-09-10T11:37:56.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.437+0000 7fddcd197700 1 --2- 172.21.15.144:0/3741785704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc80ffd30 0x7fddc8197060 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000c00 tx=0x7fddb800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:56.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.437+0000 7fddbeffd700 1 -- 172.21.15.144:0/3741785704 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddb801d070 con 0x7fddc80ffd30 2024-09-10T11:37:56.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.437+0000 7fddcfbfc700 1 -- 172.21.15.144:0/3741785704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddc819fb90 con 0x7fddc80ffd30 2024-09-10T11:37:56.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.437+0000 7fddbeffd700 1 -- 172.21.15.144:0/3741785704 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddb8005dc0 con 0x7fddc80ffd30 2024-09-10T11:37:56.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.437+0000 7fddbeffd700 1 -- 172.21.15.144:0/3741785704 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddb8017400 con 0x7fddc80ffd30 2024-09-10T11:37:56.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.437+0000 7fddcfbfc700 1 -- 172.21.15.144:0/3741785704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddc81a0000 con 0x7fddc80ffd30 2024-09-10T11:37:56.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.438+0000 7fddcfbfc700 1 -- 172.21.15.144:0/3741785704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddc8107600 con 0x7fddc80ffd30 2024-09-10T11:37:56.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.439+0000 7fddbeffd700 1 -- 172.21.15.144:0/3741785704 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fddb800f460 con 0x7fddc80ffd30 2024-09-10T11:37:56.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.439+0000 7fddbeffd700 1 --2- 172.21.15.144:0/3741785704 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fddb406e200 0x7fddb40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:56.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.439+0000 7fddbeffd700 1 -- 172.21.15.144:0/3741785704 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fddb808f6b0 con 0x7fddc80ffd30 2024-09-10T11:37:56.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.439+0000 7fddcd998700 1 --2- 172.21.15.144:0/3741785704 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fddb406e200 0x7fddb40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:56.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.440+0000 7fddcd998700 1 --2- 172.21.15.144:0/3741785704 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fddb406e200 0x7fddb40706b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fddc4005950 tx=0x7fddc400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:56.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.443+0000 7fddbeffd700 1 -- 172.21.15.144:0/3741785704 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fddb8058b10 con 0x7fddc80ffd30 2024-09-10T11:37:56.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.669+0000 7fddcfbfc700 1 -- 172.21.15.144:0/3741785704 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fddc802cd40 con 0x7fddb406e200 2024-09-10T11:37:56.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.670+0000 7fddbeffd700 1 -- 172.21.15.144:0/3741785704 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fddc802cd40 con 0x7fddb406e200 2024-09-10T11:37:56.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.672+0000 7fddcfbfc700 1 -- 172.21.15.144:0/3741785704 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fddb406e200 msgr2=0x7fddb40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:56.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.672+0000 7fddcfbfc700 1 --2- 172.21.15.144:0/3741785704 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fddb406e200 0x7fddb40706b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fddc4005950 tx=0x7fddc400b410 comp rx=0 tx=0).stop 2024-09-10T11:37:56.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.672+0000 7fddcfbfc700 1 -- 172.21.15.144:0/3741785704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc80ffd30 msgr2=0x7fddc8197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:56.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.672+0000 7fddcfbfc700 1 --2- 172.21.15.144:0/3741785704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc80ffd30 0x7fddc8197060 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fddb8000c00 tx=0x7fddb800bd70 comp rx=0 tx=0).stop 2024-09-10T11:37:56.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.672+0000 7fddcfbfc700 1 -- 172.21.15.144:0/3741785704 shutdown_connections 2024-09-10T11:37:56.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.672+0000 7fddcfbfc700 1 --2- 172.21.15.144:0/3741785704 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fddb406e200 0x7fddb40706b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:56.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.672+0000 7fddcfbfc700 1 --2- 172.21.15.144:0/3741785704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddc80ff390 0x7fddc8196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:56.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.672+0000 7fddcfbfc700 1 --2- 172.21.15.144:0/3741785704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddc80ffd30 0x7fddc8197060 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:56.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.672+0000 7fddcfbfc700 1 -- 172.21.15.144:0/3741785704 >> 172.21.15.144:0/3741785704 conn(0x7fddc8074f50 msgr2=0x7fddc80fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:56.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.673+0000 7fddcfbfc700 1 -- 172.21.15.144:0/3741785704 shutdown_connections 2024-09-10T11:37:56.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.673+0000 7fddcfbfc700 1 -- 172.21.15.144:0/3741785704 wait complete. 2024-09-10T11:37:56.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.779+0000 7fe74c9ba700 1 -- 172.21.15.144:0/1457755257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7480ffc60 msgr2=0x7fe748103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:56.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.779+0000 7fe74c9ba700 1 --2- 172.21.15.144:0/1457755257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7480ffc60 0x7fe748103c30 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe730009910 tx=0x7fe730009c20 comp rx=0 tx=0).stop 2024-09-10T11:37:56.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.779+0000 7fe74c9ba700 1 -- 172.21.15.144:0/1457755257 shutdown_connections 2024-09-10T11:37:56.781 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:56 smithi144 ceph-mon[87436]: pgmap v177: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:56.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.779+0000 7fe74c9ba700 1 --2- 172.21.15.144:0/1457755257 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7480ffc60 0x7fe748103c30 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:56.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.779+0000 7fe74c9ba700 1 --2- 172.21.15.144:0/1457755257 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7480ff2c0 0x7fe7480ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:56.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.779+0000 7fe74c9ba700 1 -- 172.21.15.144:0/1457755257 >> 172.21.15.144:0/1457755257 conn(0x7fe7480fab80 msgr2=0x7fe7480fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:56.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.779+0000 7fe74c9ba700 1 -- 172.21.15.144:0/1457755257 shutdown_connections 2024-09-10T11:37:56.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.779+0000 7fe74c9ba700 1 -- 172.21.15.144:0/1457755257 wait complete. 2024-09-10T11:37:56.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.780+0000 7fe74c9ba700 1 Processor -- start 2024-09-10T11:37:56.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.780+0000 7fe74c9ba700 1 -- start start 2024-09-10T11:37:56.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.780+0000 7fe74c9ba700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7480ff2c0 0x7fe748195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:56.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.780+0000 7fe74c9ba700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7480ffc60 0x7fe748196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:56.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.780+0000 7fe74c9ba700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7481968e0 con 0x7fe7480ff2c0 2024-09-10T11:37:56.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.780+0000 7fe74c9ba700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe748196a20 con 0x7fe7480ffc60 2024-09-10T11:37:56.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.780+0000 7fe74659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7480ff2c0 0x7fe748195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:56.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.780+0000 7fe74659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7480ff2c0 0x7fe748195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59266/0 (socket says 172.21.15.144:59266) 2024-09-10T11:37:56.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.780+0000 7fe74659c700 1 -- 172.21.15.144:0/2819449835 learned_addr learned my addr 172.21.15.144:0/2819449835 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:37:56.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.780+0000 7fe73dd9b700 1 --2- 172.21.15.144:0/2819449835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7480ffc60 0x7fe748196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:56.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.781+0000 7fe74659c700 1 -- 172.21.15.144:0/2819449835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7480ffc60 msgr2=0x7fe748196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:56.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.781+0000 7fe74659c700 1 --2- 172.21.15.144:0/2819449835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7480ffc60 0x7fe748196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:56.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.781+0000 7fe74659c700 1 -- 172.21.15.144:0/2819449835 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7300095f0 con 0x7fe7480ff2c0 2024-09-10T11:37:56.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.781+0000 7fe74659c700 1 --2- 172.21.15.144:0/2819449835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7480ff2c0 0x7fe748195d50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe73800c930 tx=0x7fe73800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:56.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.781+0000 7fe73ffff700 1 -- 172.21.15.144:0/2819449835 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe738007ad0 con 0x7fe7480ff2c0 2024-09-10T11:37:56.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.781+0000 7fe74c9ba700 1 -- 172.21.15.144:0/2819449835 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74819a870 con 0x7fe7480ff2c0 2024-09-10T11:37:56.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.781+0000 7fe73ffff700 1 -- 172.21.15.144:0/2819449835 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe738007c30 con 0x7fe7480ff2c0 2024-09-10T11:37:56.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.781+0000 7fe73ffff700 1 -- 172.21.15.144:0/2819449835 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe738018610 con 0x7fe7480ff2c0 2024-09-10T11:37:56.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.781+0000 7fe74c9ba700 1 -- 172.21.15.144:0/2819449835 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe74819adc0 con 0x7fe7480ff2c0 2024-09-10T11:37:56.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.782+0000 7fe74c9ba700 1 -- 172.21.15.144:0/2819449835 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74804ea50 con 0x7fe7480ff2c0 2024-09-10T11:37:56.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.783+0000 7fe73ffff700 1 -- 172.21.15.144:0/2819449835 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe73800f450 con 0x7fe7480ff2c0 2024-09-10T11:37:56.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.783+0000 7fe73ffff700 1 --2- 172.21.15.144:0/2819449835 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe73406e200 0x7fe7340706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:56.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.783+0000 7fe73ffff700 1 -- 172.21.15.144:0/2819449835 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe73808e2c0 con 0x7fe7480ff2c0 2024-09-10T11:37:56.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.783+0000 7fe73dd9b700 1 --2- 172.21.15.144:0/2819449835 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe73406e200 0x7fe7340706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:56.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.784+0000 7fe73dd9b700 1 --2- 172.21.15.144:0/2819449835 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe73406e200 0x7fe7340706b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe748197370 tx=0x7fe73001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:56.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.787+0000 7fe73ffff700 1 -- 172.21.15.144:0/2819449835 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe7380576a0 con 0x7fe7480ff2c0 2024-09-10T11:37:57.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:56.999+0000 7fe74c9ba700 1 -- 172.21.15.144:0/2819449835 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe74819b320 con 0x7fe73406e200 2024-09-10T11:37:57.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.009+0000 7fe73ffff700 1 -- 172.21.15.144:0/2819449835 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fe74819b320 con 0x7fe73406e200 2024-09-10T11:37:57.010 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:37:57.010 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (6m) 9s ago 16m 24.9M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:37:57.010 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (16m) 9s ago 16m 9172k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:37:57.010 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (15m) 29s ago 15m 8715k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:37:57.010 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (3m) 9s ago 16m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:37:57.010 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (3m) 29s ago 15m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:37:57.010 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (5m) 9s ago 16m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:37:57.011 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (15s) 9s ago 11m 13.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:37:57.011 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (11m) 9s ago 11m 11.4M - 18.2.0 dc2bc1663786 7b60f404e74e 2024-09-10T11:37:57.011 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (11m) 29s ago 11m 62.1M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:37:57.011 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (11m) 29s ago 11m 126M - 18.2.0 dc2bc1663786 5b21527f43f7 2024-09-10T11:37:57.011 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (7m) 9s ago 18m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:37:57.011 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (7m) 29s ago 15m 441M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:37:57.011 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (4m) 9s ago 18m 55.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:37:57.011 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (4m) 29s ago 15m 42.0M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:37:57.011 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (6m) 9s ago 16m 16.4M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:37:57.011 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (6m) 29s ago 15m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:37:57.011 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (3m) 9s ago 14m 91.9M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:37:57.011 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (2m) 9s ago 14m 90.1M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:37:57.011 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (2m) 9s ago 13m 82.2M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:37:57.012 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (98s) 29s ago 13m 102M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:37:57.012 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (66s) 29s ago 13m 87.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:37:57.012 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (33s) 29s ago 12m 12.1M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:37:57.012 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (6m) 9s ago 16m 77.9M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:37:57.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.011+0000 7fe74c9ba700 1 -- 172.21.15.144:0/2819449835 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe73406e200 msgr2=0x7fe7340706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:57.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.011+0000 7fe74c9ba700 1 --2- 172.21.15.144:0/2819449835 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe73406e200 0x7fe7340706b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe748197370 tx=0x7fe73001a040 comp rx=0 tx=0).stop 2024-09-10T11:37:57.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.011+0000 7fe74c9ba700 1 -- 172.21.15.144:0/2819449835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7480ff2c0 msgr2=0x7fe748195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:57.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.011+0000 7fe74c9ba700 1 --2- 172.21.15.144:0/2819449835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7480ff2c0 0x7fe748195d50 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe73800c930 tx=0x7fe73800ccf0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.011+0000 7fe74c9ba700 1 -- 172.21.15.144:0/2819449835 shutdown_connections 2024-09-10T11:37:57.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.011+0000 7fe74c9ba700 1 --2- 172.21.15.144:0/2819449835 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe73406e200 0x7fe7340706b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.011+0000 7fe74c9ba700 1 --2- 172.21.15.144:0/2819449835 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe7480ff2c0 0x7fe748195d50 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.011+0000 7fe74c9ba700 1 --2- 172.21.15.144:0/2819449835 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe7480ffc60 0x7fe748196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.011+0000 7fe74c9ba700 1 -- 172.21.15.144:0/2819449835 >> 172.21.15.144:0/2819449835 conn(0x7fe7480fab80 msgr2=0x7fe748068ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:57.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.011+0000 7fe74c9ba700 1 -- 172.21.15.144:0/2819449835 shutdown_connections 2024-09-10T11:37:57.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.011+0000 7fe74c9ba700 1 -- 172.21.15.144:0/2819449835 wait complete. 2024-09-10T11:37:57.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.115+0000 7f8d243cd700 1 -- 172.21.15.144:0/2048874523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c100d40 msgr2=0x7f8d1c1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:57.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.115+0000 7f8d243cd700 1 --2- 172.21.15.144:0/2048874523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c100d40 0x7f8d1c1094a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009910 tx=0x7f8d0c009c20 comp rx=0 tx=0).stop 2024-09-10T11:37:57.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.116+0000 7f8d243cd700 1 -- 172.21.15.144:0/2048874523 shutdown_connections 2024-09-10T11:37:57.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.116+0000 7f8d243cd700 1 --2- 172.21.15.144:0/2048874523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c100d40 0x7f8d1c1094a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.116+0000 7f8d243cd700 1 --2- 172.21.15.144:0/2048874523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d1c100430 0x7f8d1c100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.116+0000 7f8d243cd700 1 -- 172.21.15.144:0/2048874523 >> 172.21.15.144:0/2048874523 conn(0x7f8d1c0748b0 msgr2=0x7f8d1c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:57.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.116+0000 7f8d243cd700 1 -- 172.21.15.144:0/2048874523 shutdown_connections 2024-09-10T11:37:57.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.116+0000 7f8d243cd700 1 -- 172.21.15.144:0/2048874523 wait complete. 2024-09-10T11:37:57.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.116+0000 7f8d243cd700 1 Processor -- start 2024-09-10T11:37:57.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.116+0000 7f8d243cd700 1 -- start start 2024-09-10T11:37:57.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:56 smithi178 ceph-mon[65940]: pgmap v177: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:57.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.117+0000 7f8d243cd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d1c100430 0x7f8d1c19b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:57.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.117+0000 7f8d243cd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c100d40 0x7f8d1c19bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:57.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.117+0000 7f8d243cd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c19c580 con 0x7f8d1c100d40 2024-09-10T11:37:57.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.117+0000 7f8d243cd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d1c195a30 con 0x7f8d1c100430 2024-09-10T11:37:57.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.117+0000 7f8d21968700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c100d40 0x7f8d1c19bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:57.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.117+0000 7f8d21968700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c100d40 0x7f8d1c19bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59284/0 (socket says 172.21.15.144:59284) 2024-09-10T11:37:57.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.117+0000 7f8d21968700 1 -- 172.21.15.144:0/3912272937 learned_addr learned my addr 172.21.15.144:0/3912272937 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:37:57.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.117+0000 7f8d22169700 1 --2- 172.21.15.144:0/3912272937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d1c100430 0x7f8d1c19b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:57.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.117+0000 7f8d21968700 1 -- 172.21.15.144:0/3912272937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d1c100430 msgr2=0x7f8d1c19b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:57.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.117+0000 7f8d21968700 1 --2- 172.21.15.144:0/3912272937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d1c100430 0x7f8d1c19b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.117+0000 7f8d21968700 1 -- 172.21.15.144:0/3912272937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c0095f0 con 0x7f8d1c100d40 2024-09-10T11:37:57.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.117+0000 7f8d22169700 1 --2- 172.21.15.144:0/3912272937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d1c100430 0x7f8d1c19b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:37:57.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.118+0000 7f8d21968700 1 --2- 172.21.15.144:0/3912272937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c100d40 0x7f8d1c19bef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0092e0 tx=0x7f8d0c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:57.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.118+0000 7f8d137fe700 1 -- 172.21.15.144:0/3912272937 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d0c01d070 con 0x7f8d1c100d40 2024-09-10T11:37:57.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.118+0000 7f8d137fe700 1 -- 172.21.15.144:0/3912272937 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d0c005e00 con 0x7f8d1c100d40 2024-09-10T11:37:57.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.118+0000 7f8d243cd700 1 -- 172.21.15.144:0/3912272937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1c195d10 con 0x7f8d1c100d40 2024-09-10T11:37:57.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.118+0000 7f8d137fe700 1 -- 172.21.15.144:0/3912272937 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d0c017400 con 0x7f8d1c100d40 2024-09-10T11:37:57.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.118+0000 7f8d243cd700 1 -- 172.21.15.144:0/3912272937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1c1962e0 con 0x7f8d1c100d40 2024-09-10T11:37:57.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.119+0000 7f8d243cd700 1 -- 172.21.15.144:0/3912272937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d1c107e10 con 0x7f8d1c100d40 2024-09-10T11:37:57.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.119+0000 7f8d137fe700 1 -- 172.21.15.144:0/3912272937 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8d0c003a60 con 0x7f8d1c100d40 2024-09-10T11:37:57.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.120+0000 7f8d137fe700 1 --2- 172.21.15.144:0/3912272937 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8d0806e200 0x7f8d080706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:57.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.120+0000 7f8d137fe700 1 -- 172.21.15.144:0/3912272937 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8d0c08f4f0 con 0x7f8d1c100d40 2024-09-10T11:37:57.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.120+0000 7f8d22169700 1 --2- 172.21.15.144:0/3912272937 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8d0806e200 0x7f8d080706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:57.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.121+0000 7f8d22169700 1 --2- 172.21.15.144:0/3912272937 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8d0806e200 0x7f8d080706b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8d18005950 tx=0x7f8d1800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:57.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.126+0000 7f8d137fe700 1 -- 172.21.15.144:0/3912272937 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8d0c05c400 con 0x7f8d1c100d40 2024-09-10T11:37:57.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.420+0000 7f8d243cd700 1 -- 172.21.15.144:0/3912272937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8d1c04ea50 con 0x7f8d1c100d40 2024-09-10T11:37:57.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.421+0000 7f8d137fe700 1 -- 172.21.15.144:0/3912272937 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f8d0c05bf90 con 0x7f8d1c100d40 2024-09-10T11:37:57.422 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:37:57.422 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:37:57.422 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:37:57.422 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:37:57.423 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:37:57.423 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:37:57.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:37:57.423 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:37:57.423 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:37:57.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:37:57.423 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:37:57.423 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-09-10T11:37:57.423 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 1 2024-09-10T11:37:57.423 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:37:57.423 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:37:57.423 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-09-10T11:37:57.423 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 11 2024-09-10T11:37:57.424 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:37:57.424 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:37:57.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.424+0000 7f8d243cd700 1 -- 172.21.15.144:0/3912272937 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8d0806e200 msgr2=0x7f8d080706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:57.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.424+0000 7f8d243cd700 1 --2- 172.21.15.144:0/3912272937 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8d0806e200 0x7f8d080706b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8d18005950 tx=0x7f8d1800b410 comp rx=0 tx=0).stop 2024-09-10T11:37:57.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.424+0000 7f8d243cd700 1 -- 172.21.15.144:0/3912272937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c100d40 msgr2=0x7f8d1c19bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:57.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.424+0000 7f8d243cd700 1 --2- 172.21.15.144:0/3912272937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c100d40 0x7f8d1c19bef0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c0092e0 tx=0x7f8d0c00bd70 comp rx=0 tx=0).stop 2024-09-10T11:37:57.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.424+0000 7f8d243cd700 1 -- 172.21.15.144:0/3912272937 shutdown_connections 2024-09-10T11:37:57.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.424+0000 7f8d243cd700 1 --2- 172.21.15.144:0/3912272937 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8d0806e200 0x7f8d080706b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.424+0000 7f8d243cd700 1 --2- 172.21.15.144:0/3912272937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8d1c100430 0x7f8d1c19b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.424+0000 7f8d243cd700 1 --2- 172.21.15.144:0/3912272937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8d1c100d40 0x7f8d1c19bef0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.424+0000 7f8d243cd700 1 -- 172.21.15.144:0/3912272937 >> 172.21.15.144:0/3912272937 conn(0x7f8d1c0748b0 msgr2=0x7f8d1c0ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:57.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.424+0000 7f8d243cd700 1 -- 172.21.15.144:0/3912272937 shutdown_connections 2024-09-10T11:37:57.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.424+0000 7f8d243cd700 1 -- 172.21.15.144:0/3912272937 wait complete. 2024-09-10T11:37:57.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.530+0000 7f159e786700 1 -- 172.21.15.144:0/583845432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598101c40 msgr2=0x7f159810a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:57.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.530+0000 7f159e786700 1 --2- 172.21.15.144:0/583845432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598101c40 0x7f159810a140 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f158c009940 tx=0x7f158c009c50 comp rx=0 tx=0).stop 2024-09-10T11:37:57.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.531+0000 7f159e786700 1 -- 172.21.15.144:0/583845432 shutdown_connections 2024-09-10T11:37:57.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.531+0000 7f159e786700 1 --2- 172.21.15.144:0/583845432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598101c40 0x7f159810a140 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.531+0000 7f159e786700 1 --2- 172.21.15.144:0/583845432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1598101330 0x7f1598101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.531+0000 7f159e786700 1 -- 172.21.15.144:0/583845432 >> 172.21.15.144:0/583845432 conn(0x7f15980fabb0 msgr2=0x7f15980fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:57.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.531+0000 7f159e786700 1 -- 172.21.15.144:0/583845432 shutdown_connections 2024-09-10T11:37:57.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.531+0000 7f159e786700 1 -- 172.21.15.144:0/583845432 wait complete. 2024-09-10T11:37:57.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.531+0000 7f159e786700 1 Processor -- start 2024-09-10T11:37:57.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.531+0000 7f159e786700 1 -- start start 2024-09-10T11:37:57.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f159e786700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1598101330 0x7f159819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:57.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f159e786700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598101c40 0x7f159819c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:57.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f159e786700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f159819c690 con 0x7f1598101c40 2024-09-10T11:37:57.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f159e786700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1598195b40 con 0x7f1598101330 2024-09-10T11:37:57.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f15977fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598101c40 0x7f159819c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:57.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f15977fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598101c40 0x7f159819c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59296/0 (socket says 172.21.15.144:59296) 2024-09-10T11:37:57.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f15977fe700 1 -- 172.21.15.144:0/2533996037 learned_addr learned my addr 172.21.15.144:0/2533996037 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:37:57.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f1597fff700 1 --2- 172.21.15.144:0/2533996037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1598101330 0x7f159819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:57.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f15977fe700 1 -- 172.21.15.144:0/2533996037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1598101330 msgr2=0x7f159819bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:57.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f15977fe700 1 --2- 172.21.15.144:0/2533996037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1598101330 0x7f159819bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f15977fe700 1 -- 172.21.15.144:0/2533996037 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f158c0095f0 con 0x7f1598101c40 2024-09-10T11:37:57.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f15977fe700 1 --2- 172.21.15.144:0/2533996037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598101c40 0x7f159819c000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f158c0055f0 tx=0x7f158c0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:57.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f15957fa700 1 -- 172.21.15.144:0/2533996037 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f158c01d070 con 0x7f1598101c40 2024-09-10T11:37:57.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f159e786700 1 -- 172.21.15.144:0/2533996037 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1598195dc0 con 0x7f1598101c40 2024-09-10T11:37:57.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f15957fa700 1 -- 172.21.15.144:0/2533996037 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f158c00f460 con 0x7f1598101c40 2024-09-10T11:37:57.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.532+0000 7f15957fa700 1 -- 172.21.15.144:0/2533996037 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f158c021620 con 0x7f1598101c40 2024-09-10T11:37:57.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.533+0000 7f159e786700 1 -- 172.21.15.144:0/2533996037 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15981962b0 con 0x7f1598101c40 2024-09-10T11:37:57.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.533+0000 7f159e786700 1 -- 172.21.15.144:0/2533996037 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15981077b0 con 0x7f1598101c40 2024-09-10T11:37:57.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.534+0000 7f15957fa700 1 -- 172.21.15.144:0/2533996037 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f158c00fa60 con 0x7f1598101c40 2024-09-10T11:37:57.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.535+0000 7f15957fa700 1 --2- 172.21.15.144:0/2533996037 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f158006e200 0x7f15800706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:57.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.535+0000 7f15957fa700 1 -- 172.21.15.144:0/2533996037 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f158c08f490 con 0x7f1598101c40 2024-09-10T11:37:57.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.535+0000 7f1597fff700 1 --2- 172.21.15.144:0/2533996037 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f158006e200 0x7f15800706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:57.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.535+0000 7f1597fff700 1 --2- 172.21.15.144:0/2533996037 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f158006e200 0x7f15800706b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1588005fd0 tx=0x7f158800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:57.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.539+0000 7f15957fa700 1 -- 172.21.15.144:0/2533996037 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f158c05c300 con 0x7f1598101c40 2024-09-10T11:37:57.672 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:57 smithi144 ceph-mon[87436]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:57.672 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:57 smithi144 ceph-mon[87436]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:57.672 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:57 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3912272937' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:57.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.798+0000 7f159e786700 1 -- 172.21.15.144:0/2533996037 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f159804ea50 con 0x7f1598101c40 2024-09-10T11:37:57.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.798+0000 7f15957fa700 1 -- 172.21.15.144:0/2533996037 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1735 (secure 0 0 0) 0x7f158c05be90 con 0x7f1598101c40 2024-09-10T11:37:57.800 INFO:teuthology.orchestra.run.smithi144.stdout:e29 2024-09-10T11:37:57.800 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:37:54:629649+0000 2024-09-10T11:37:57.800 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:37:57.800 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:37:57.800 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:37:57.800 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:37:57.800 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:37:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:37:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:37:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:37:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:37:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:37:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:37:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:37:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:37:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:37:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:37:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:37:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:37:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:37:57.801 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:37:57.802 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:37:57.802 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:37:57.802 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:37:57.802 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:37:57.802 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:37:57.802 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:37:57.802 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:37:57.802 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:37:57.802 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:37:57.802 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:37:57.802 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:37:57.802 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:37:57.802 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:37:57.802 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:37:57.802 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:37:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:37:57.803 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:37:57.803 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:37:57.803 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:37:57.803 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:37:57.803 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44325} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/559040332,v1:172.21.15.178:6825/559040332] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:37:57.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.801+0000 7f159e786700 1 -- 172.21.15.144:0/2533996037 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f158006e200 msgr2=0x7f15800706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:57.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.801+0000 7f159e786700 1 --2- 172.21.15.144:0/2533996037 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f158006e200 0x7f15800706b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1588005fd0 tx=0x7f158800a560 comp rx=0 tx=0).stop 2024-09-10T11:37:57.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.801+0000 7f159e786700 1 -- 172.21.15.144:0/2533996037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598101c40 msgr2=0x7f159819c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:57.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.801+0000 7f159e786700 1 --2- 172.21.15.144:0/2533996037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598101c40 0x7f159819c000 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f158c0055f0 tx=0x7f158c0056d0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.801+0000 7f159e786700 1 -- 172.21.15.144:0/2533996037 shutdown_connections 2024-09-10T11:37:57.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.801+0000 7f159e786700 1 --2- 172.21.15.144:0/2533996037 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f158006e200 0x7f15800706b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.801+0000 7f159e786700 1 --2- 172.21.15.144:0/2533996037 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1598101330 0x7f159819bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.801+0000 7f159e786700 1 --2- 172.21.15.144:0/2533996037 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1598101c40 0x7f159819c000 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.801+0000 7f159e786700 1 -- 172.21.15.144:0/2533996037 >> 172.21.15.144:0/2533996037 conn(0x7f15980fabb0 msgr2=0x7f15981048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:57.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.801+0000 7f159e786700 1 -- 172.21.15.144:0/2533996037 shutdown_connections 2024-09-10T11:37:57.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.801+0000 7f159e786700 1 -- 172.21.15.144:0/2533996037 wait complete. 2024-09-10T11:37:57.805 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 29 2024-09-10T11:37:57.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:57 smithi178 ceph-mon[65940]: from='client.34394 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:57.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:57 smithi178 ceph-mon[65940]: from='client.34398 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:57.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:57 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3912272937' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:37:57.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.907+0000 7fbef60d4700 1 -- 172.21.15.144:0/788374585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef0100d80 msgr2=0x7fbef0104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:57.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.907+0000 7fbef60d4700 1 --2- 172.21.15.144:0/788374585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef0100d80 0x7fbef0104d50 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbee4009910 tx=0x7fbee4009c20 comp rx=0 tx=0).stop 2024-09-10T11:37:57.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.907+0000 7fbef60d4700 1 -- 172.21.15.144:0/788374585 shutdown_connections 2024-09-10T11:37:57.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.907+0000 7fbef60d4700 1 --2- 172.21.15.144:0/788374585 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef0100d80 0x7fbef0104d50 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.907+0000 7fbef60d4700 1 --2- 172.21.15.144:0/788374585 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef01003e0 0x7fbef01007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.907+0000 7fbef60d4700 1 -- 172.21.15.144:0/788374585 >> 172.21.15.144:0/788374585 conn(0x7fbef00fbca0 msgr2=0x7fbef00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:57.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.907+0000 7fbef60d4700 1 -- 172.21.15.144:0/788374585 shutdown_connections 2024-09-10T11:37:57.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.907+0000 7fbef60d4700 1 -- 172.21.15.144:0/788374585 wait complete. 2024-09-10T11:37:57.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.908+0000 7fbef60d4700 1 Processor -- start 2024-09-10T11:37:57.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.908+0000 7fbef60d4700 1 -- start start 2024-09-10T11:37:57.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.908+0000 7fbef60d4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef01003e0 0x7fbef0073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:57.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.908+0000 7fbef60d4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef0100d80 0x7fbef0073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:57.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.908+0000 7fbef60d4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef0073ad0 con 0x7fbef01003e0 2024-09-10T11:37:57.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.908+0000 7fbef60d4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef0073c10 con 0x7fbef0100d80 2024-09-10T11:37:57.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.908+0000 7fbeef7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef01003e0 0x7fbef0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:57.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.909+0000 7fbeef7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef01003e0 0x7fbef0073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59310/0 (socket says 172.21.15.144:59310) 2024-09-10T11:37:57.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.909+0000 7fbeef7fe700 1 -- 172.21.15.144:0/1665735808 learned_addr learned my addr 172.21.15.144:0/1665735808 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:37:57.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.909+0000 7fbeeeffd700 1 --2- 172.21.15.144:0/1665735808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef0100d80 0x7fbef0073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:57.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.909+0000 7fbeef7fe700 1 -- 172.21.15.144:0/1665735808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef0100d80 msgr2=0x7fbef0073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:57.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.909+0000 7fbeef7fe700 1 --2- 172.21.15.144:0/1665735808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef0100d80 0x7fbef0073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:57.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.909+0000 7fbeef7fe700 1 -- 172.21.15.144:0/1665735808 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee40095f0 con 0x7fbef01003e0 2024-09-10T11:37:57.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.909+0000 7fbeef7fe700 1 --2- 172.21.15.144:0/1665735808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef01003e0 0x7fbef0073050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbee000c930 tx=0x7fbee000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:57.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.909+0000 7fbeecff9700 1 -- 172.21.15.144:0/1665735808 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee0007ad0 con 0x7fbef01003e0 2024-09-10T11:37:57.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.909+0000 7fbeecff9700 1 -- 172.21.15.144:0/1665735808 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbee000ce80 con 0x7fbef01003e0 2024-09-10T11:37:57.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.909+0000 7fbef60d4700 1 -- 172.21.15.144:0/1665735808 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef01a1f80 con 0x7fbef01003e0 2024-09-10T11:37:57.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.909+0000 7fbeecff9700 1 -- 172.21.15.144:0/1665735808 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee0018630 con 0x7fbef01003e0 2024-09-10T11:37:57.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.909+0000 7fbef60d4700 1 -- 172.21.15.144:0/1665735808 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef01a22e0 con 0x7fbef01003e0 2024-09-10T11:37:57.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.910+0000 7fbef60d4700 1 -- 172.21.15.144:0/1665735808 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef004ea50 con 0x7fbef01003e0 2024-09-10T11:37:57.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.911+0000 7fbeecff9700 1 -- 172.21.15.144:0/1665735808 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fbee0007c30 con 0x7fbef01003e0 2024-09-10T11:37:57.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.911+0000 7fbeecff9700 1 --2- 172.21.15.144:0/1665735808 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbed806e200 0x7fbed80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:57.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.911+0000 7fbeeeffd700 1 --2- 172.21.15.144:0/1665735808 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbed806e200 0x7fbed80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:57.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.911+0000 7fbeecff9700 1 -- 172.21.15.144:0/1665735808 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbee008df80 con 0x7fbef01003e0 2024-09-10T11:37:57.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.912+0000 7fbeeeffd700 1 --2- 172.21.15.144:0/1665735808 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbed806e200 0x7fbed80706b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbef0074840 tx=0x7fbee401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:57.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:57.916+0000 7fbeecff9700 1 -- 172.21.15.144:0/1665735808 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbee00573e0 con 0x7fbef01003e0 2024-09-10T11:37:58.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.191+0000 7fbef60d4700 1 -- 172.21.15.144:0/1665735808 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbef00fdfc0 con 0x7fbed806e200 2024-09-10T11:37:58.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.192+0000 7fbeecff9700 1 -- 172.21.15.144:0/1665735808 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbef00fdfc0 con 0x7fbed806e200 2024-09-10T11:37:58.194 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:37:58.194 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:37:58.194 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:37:58.194 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:37:58.194 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:37:58.194 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:37:58.194 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:37:58.194 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:37:58.194 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:37:58.194 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:37:58.194 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "13/23 daemons upgraded", 2024-09-10T11:37:58.194 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:37:58.194 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:37:58.195 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:37:58.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.195+0000 7fbef60d4700 1 -- 172.21.15.144:0/1665735808 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbed806e200 msgr2=0x7fbed80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:58.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.195+0000 7fbef60d4700 1 --2- 172.21.15.144:0/1665735808 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbed806e200 0x7fbed80706b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbef0074840 tx=0x7fbee401a040 comp rx=0 tx=0).stop 2024-09-10T11:37:58.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.195+0000 7fbef60d4700 1 -- 172.21.15.144:0/1665735808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef01003e0 msgr2=0x7fbef0073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:58.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.195+0000 7fbef60d4700 1 --2- 172.21.15.144:0/1665735808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef01003e0 0x7fbef0073050 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbee000c930 tx=0x7fbee000cc40 comp rx=0 tx=0).stop 2024-09-10T11:37:58.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.195+0000 7fbef60d4700 1 -- 172.21.15.144:0/1665735808 shutdown_connections 2024-09-10T11:37:58.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.195+0000 7fbef60d4700 1 --2- 172.21.15.144:0/1665735808 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbed806e200 0x7fbed80706b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:58.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.195+0000 7fbef60d4700 1 --2- 172.21.15.144:0/1665735808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbef01003e0 0x7fbef0073050 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:58.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.195+0000 7fbef60d4700 1 --2- 172.21.15.144:0/1665735808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbef0100d80 0x7fbef0073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:58.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.195+0000 7fbef60d4700 1 -- 172.21.15.144:0/1665735808 >> 172.21.15.144:0/1665735808 conn(0x7fbef00fbca0 msgr2=0x7fbef00fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:58.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.195+0000 7fbef60d4700 1 -- 172.21.15.144:0/1665735808 shutdown_connections 2024-09-10T11:37:58.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.195+0000 7fbef60d4700 1 -- 172.21.15.144:0/1665735808 wait complete. 2024-09-10T11:37:58.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.314+0000 7f0ad8e7e700 1 -- 172.21.15.144:0/431732548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad40ff390 msgr2=0x7f0ad40ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:58.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.314+0000 7f0ad8e7e700 1 --2- 172.21.15.144:0/431732548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad40ff390 0x7f0ad40ff760 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0ac4009910 tx=0x7f0ac4009c20 comp rx=0 tx=0).stop 2024-09-10T11:37:58.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.314+0000 7f0ad8e7e700 1 -- 172.21.15.144:0/431732548 shutdown_connections 2024-09-10T11:37:58.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.314+0000 7f0ad8e7e700 1 --2- 172.21.15.144:0/431732548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ad40ffd30 0x7f0ad4103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:58.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.314+0000 7f0ad8e7e700 1 --2- 172.21.15.144:0/431732548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad40ff390 0x7f0ad40ff760 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:58.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.314+0000 7f0ad8e7e700 1 -- 172.21.15.144:0/431732548 >> 172.21.15.144:0/431732548 conn(0x7f0ad4074f50 msgr2=0x7f0ad4075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:58.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.314+0000 7f0ad8e7e700 1 -- 172.21.15.144:0/431732548 shutdown_connections 2024-09-10T11:37:58.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.314+0000 7f0ad8e7e700 1 -- 172.21.15.144:0/431732548 wait complete. 2024-09-10T11:37:58.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.315+0000 7f0ad8e7e700 1 Processor -- start 2024-09-10T11:37:58.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.315+0000 7f0ad8e7e700 1 -- start start 2024-09-10T11:37:58.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.315+0000 7f0ad8e7e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad40ff390 0x7f0ad4195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:58.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.315+0000 7f0ad8e7e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ad40ffd30 0x7f0ad4196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:58.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.315+0000 7f0ad8e7e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad4196960 con 0x7f0ad40ff390 2024-09-10T11:37:58.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.315+0000 7f0ad8e7e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad419a6f0 con 0x7f0ad40ffd30 2024-09-10T11:37:58.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.315+0000 7f0ad259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad40ff390 0x7f0ad4195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:58.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.315+0000 7f0ad259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad40ff390 0x7f0ad4195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59338/0 (socket says 172.21.15.144:59338) 2024-09-10T11:37:58.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.315+0000 7f0ad259c700 1 -- 172.21.15.144:0/977423184 learned_addr learned my addr 172.21.15.144:0/977423184 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:37:58.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.315+0000 7f0ad1d9b700 1 --2- 172.21.15.144:0/977423184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ad40ffd30 0x7f0ad4196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:58.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.315+0000 7f0ad259c700 1 -- 172.21.15.144:0/977423184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ad40ffd30 msgr2=0x7f0ad4196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:58.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.315+0000 7f0ad259c700 1 --2- 172.21.15.144:0/977423184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ad40ffd30 0x7f0ad4196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:58.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.315+0000 7f0ad259c700 1 -- 172.21.15.144:0/977423184 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac40095f0 con 0x7f0ad40ff390 2024-09-10T11:37:58.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.316+0000 7f0ad259c700 1 --2- 172.21.15.144:0/977423184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad40ff390 0x7f0ad4195d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40092e0 tx=0x7f0ac400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:58.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.316+0000 7f0ac37fe700 1 -- 172.21.15.144:0/977423184 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ac401d070 con 0x7f0ad40ff390 2024-09-10T11:37:58.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.316+0000 7f0ac37fe700 1 -- 172.21.15.144:0/977423184 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0ac4005e00 con 0x7f0ad40ff390 2024-09-10T11:37:58.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.316+0000 7f0ad8e7e700 1 -- 172.21.15.144:0/977423184 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad419a970 con 0x7f0ad40ff390 2024-09-10T11:37:58.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.316+0000 7f0ac37fe700 1 -- 172.21.15.144:0/977423184 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ac4017400 con 0x7f0ad40ff390 2024-09-10T11:37:58.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.316+0000 7f0ad8e7e700 1 -- 172.21.15.144:0/977423184 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad419ae60 con 0x7f0ad40ff390 2024-09-10T11:37:58.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.317+0000 7f0ad8e7e700 1 -- 172.21.15.144:0/977423184 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad404ea50 con 0x7f0ad40ff390 2024-09-10T11:37:58.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.317+0000 7f0ac37fe700 1 -- 172.21.15.144:0/977423184 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0ac400f460 con 0x7f0ad40ff390 2024-09-10T11:37:58.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.318+0000 7f0ac37fe700 1 --2- 172.21.15.144:0/977423184 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0abc06e200 0x7f0abc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:37:58.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.318+0000 7f0ac37fe700 1 -- 172.21.15.144:0/977423184 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0ac408f4b0 con 0x7f0ad40ff390 2024-09-10T11:37:58.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.318+0000 7f0ad1d9b700 1 --2- 172.21.15.144:0/977423184 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0abc06e200 0x7f0abc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:37:58.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.319+0000 7f0ad1d9b700 1 --2- 172.21.15.144:0/977423184 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0abc06e200 0x7f0abc0706b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8005950 tx=0x7f0ac80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:37:58.323 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.322+0000 7f0ac37fe700 1 -- 172.21.15.144:0/977423184 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0ac4058910 con 0x7f0ad40ff390 2024-09-10T11:37:58.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.648+0000 7f0ad8e7e700 1 -- 172.21.15.144:0/977423184 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0ad419b300 con 0x7f0ad40ff390 2024-09-10T11:37:58.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.648+0000 7f0ac37fe700 1 -- 172.21.15.144:0/977423184 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f0ac405bfd0 con 0x7f0ad40ff390 2024-09-10T11:37:58.650 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:37:58.650 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:37:58.651 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:37:58.651 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:37:58.651 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:37:58.651 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:37:58.651 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:37:58.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.650+0000 7f0ad8e7e700 1 -- 172.21.15.144:0/977423184 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0abc06e200 msgr2=0x7f0abc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:58.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.650+0000 7f0ad8e7e700 1 --2- 172.21.15.144:0/977423184 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0abc06e200 0x7f0abc0706b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ac8005950 tx=0x7f0ac80058e0 comp rx=0 tx=0).stop 2024-09-10T11:37:58.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.651+0000 7f0ad8e7e700 1 -- 172.21.15.144:0/977423184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad40ff390 msgr2=0x7f0ad4195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:37:58.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.651+0000 7f0ad8e7e700 1 --2- 172.21.15.144:0/977423184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad40ff390 0x7f0ad4195d40 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f0ac40092e0 tx=0x7f0ac400be20 comp rx=0 tx=0).stop 2024-09-10T11:37:58.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.651+0000 7f0ad8e7e700 1 -- 172.21.15.144:0/977423184 shutdown_connections 2024-09-10T11:37:58.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.651+0000 7f0ad8e7e700 1 --2- 172.21.15.144:0/977423184 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0abc06e200 0x7f0abc0706b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:58.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.651+0000 7f0ad8e7e700 1 --2- 172.21.15.144:0/977423184 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0ad40ff390 0x7f0ad4195d40 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:58.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.651+0000 7f0ad8e7e700 1 --2- 172.21.15.144:0/977423184 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0ad40ffd30 0x7f0ad4196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:37:58.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.651+0000 7f0ad8e7e700 1 -- 172.21.15.144:0/977423184 >> 172.21.15.144:0/977423184 conn(0x7f0ad4074f50 msgr2=0x7f0ad4068410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:37:58.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.651+0000 7f0ad8e7e700 1 -- 172.21.15.144:0/977423184 shutdown_connections 2024-09-10T11:37:58.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:37:58.651+0000 7f0ad8e7e700 1 -- 172.21.15.144:0/977423184 wait complete. 2024-09-10T11:37:58.873 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:58 smithi144 ceph-mon[87436]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:58.874 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:58 smithi144 ceph-mon[87436]: pgmap v178: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:58.874 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:58 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2533996037' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:37:58.874 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:37:59.032 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:58 smithi178 ceph-mon[65940]: from='client.34402 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:59.032 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:58 smithi178 ceph-mon[65940]: pgmap v178: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:37:59.033 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:58 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2533996037' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:37:59.033 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:37:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:59 smithi144 ceph-mon[87436]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:59 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/977423184' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:37:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:59 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:59 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:37:59 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:37:59.997 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:59 smithi178 ceph-mon[65940]: from='client.34414 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:37:59.997 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:59 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/977423184' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:37:59.997 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:59 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:59.997 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:59 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:37:59.998 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:37:59 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:38:00.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:00 smithi144 ceph-mon[87436]: pgmap v179: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:00.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:00 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] up:boot 2024-09-10T11:38:00.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:00 smithi144 ceph-mon[87436]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T11:38:00.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:00 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jaoyvs"}]: dispatch 2024-09-10T11:38:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:00 smithi178 ceph-mon[65940]: pgmap v179: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:00 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] up:boot 2024-09-10T11:38:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:00 smithi178 ceph-mon[65940]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T11:38:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:00 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi144.jaoyvs"}]: dispatch 2024-09-10T11:38:02.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:02 smithi178 ceph-mon[65940]: pgmap v180: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:02.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:02 smithi144 ceph-mon[87436]: pgmap v180: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:04.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:04 smithi144 ceph-mon[87436]: pgmap v181: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:04.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:04 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:04.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:04 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:04.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:04 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:04.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:04 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:04 smithi178 ceph-mon[65940]: pgmap v181: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:04 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:04 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:04 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:04 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:06 smithi144 ceph-mon[87436]: pgmap v182: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:06 smithi178 ceph-mon[65940]: pgmap v182: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:08.753 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:08 smithi178 ceph-mon[65940]: pgmap v183: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:08.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:08 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:08.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:08 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:08.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:08 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:38:08.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:08 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:38:08.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:08 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:08.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:08 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:38:08.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:08 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:38:08.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:08 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:38:08.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:08 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:38:08.754 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:08 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:38:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:08 smithi144 ceph-mon[87436]: pgmap v183: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:08 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:08 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:08 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:38:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:08 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:38:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:08 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:08 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:38:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:08 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:38:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:08 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:38:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:08 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:38:08.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:08 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:38:10.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:10 smithi144 ceph-mon[87436]: Upgrade: Updating mds.cephfs.smithi178.nmhrmh 2024-09-10T11:38:10.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:10 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:10.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:10 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.nmhrmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:38:10.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:10 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:38:10.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:10 smithi144 ceph-mon[87436]: Deploying daemon mds.cephfs.smithi178.nmhrmh on smithi178 2024-09-10T11:38:10.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:10 smithi144 ceph-mon[87436]: pgmap v184: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:10.750 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:10 smithi178 ceph-mon[65940]: Upgrade: Updating mds.cephfs.smithi178.nmhrmh 2024-09-10T11:38:10.750 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:10 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:10.751 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:10 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.nmhrmh", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:38:10.751 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:10 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:38:10.751 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:10 smithi178 ceph-mon[65940]: Deploying daemon mds.cephfs.smithi178.nmhrmh on smithi178 2024-09-10T11:38:10.751 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:10 smithi178 ceph-mon[65940]: pgmap v184: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:12 smithi178 ceph-mon[65940]: osdmap e84: 6 total, 6 up, 6 in 2024-09-10T11:38:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:12 smithi178 ceph-mon[65940]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T11:38:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:12 smithi178 ceph-mon[65940]: pgmap v186: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:12 smithi144 ceph-mon[87436]: osdmap e84: 6 total, 6 up, 6 in 2024-09-10T11:38:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:12 smithi144 ceph-mon[87436]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T11:38:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:12 smithi144 ceph-mon[87436]: pgmap v186: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:13.912 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:38:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:38:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:14 smithi178 ceph-mon[65940]: pgmap v187: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:14.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:14 smithi144 ceph-mon[87436]: pgmap v187: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:15 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] up:boot 2024-09-10T11:38:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:15 smithi144 ceph-mon[87436]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T11:38:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:15 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.nmhrmh"}]: dispatch 2024-09-10T11:38:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:15 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:15 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:15 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:38:16.073 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:15 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] up:boot 2024-09-10T11:38:16.073 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:15 smithi178 ceph-mon[65940]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T11:38:16.073 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:15 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.nmhrmh"}]: dispatch 2024-09-10T11:38:16.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:15 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:16.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:15 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:16.074 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:15 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:38:16.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:16 smithi144 ceph-mon[87436]: pgmap v188: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:16.985 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:16 smithi178 ceph-mon[65940]: pgmap v188: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:18.863 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:18 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:18.863 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:18 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:18.863 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:18 smithi178 ceph-mon[65940]: pgmap v189: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:18.863 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:18 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:18.863 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:18 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:18 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:18 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:18 smithi144 ceph-mon[87436]: pgmap v189: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:18 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:18 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:38:20.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:20 smithi144 ceph-mon[87436]: pgmap v190: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:20.979 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:20 smithi178 ceph-mon[65940]: pgmap v190: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:22 smithi144 ceph-mon[87436]: pgmap v191: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:23.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:22 smithi178 ceph-mon[65940]: pgmap v191: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:24.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:24 smithi144 ceph-mon[87436]: pgmap v192: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:24 smithi178 ceph-mon[65940]: pgmap v192: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:26.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:26 smithi144 ceph-mon[87436]: pgmap v193: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:26 smithi178 ceph-mon[65940]: pgmap v193: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:28.771 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.768+0000 7f6f3ce18700 1 -- 172.21.15.144:0/3351517038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f38100d70 msgr2=0x7f6f38104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.771 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.768+0000 7f6f3ce18700 1 --2- 172.21.15.144:0/3351517038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f38100d70 0x7f6f38104d40 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c009910 tx=0x7f6f2c009c20 comp rx=0 tx=0).stop 2024-09-10T11:38:28.771 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.768+0000 7f6f3ce18700 1 -- 172.21.15.144:0/3351517038 shutdown_connections 2024-09-10T11:38:28.771 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.768+0000 7f6f3ce18700 1 --2- 172.21.15.144:0/3351517038 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f38100d70 0x7f6f38104d40 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.771 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.768+0000 7f6f3ce18700 1 --2- 172.21.15.144:0/3351517038 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f381003d0 0x7f6f381007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.768+0000 7f6f3ce18700 1 -- 172.21.15.144:0/3351517038 >> 172.21.15.144:0/3351517038 conn(0x7f6f380fbc90 msgr2=0x7f6f380fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:28.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.768+0000 7f6f3ce18700 1 -- 172.21.15.144:0/3351517038 shutdown_connections 2024-09-10T11:38:28.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.768+0000 7f6f3ce18700 1 -- 172.21.15.144:0/3351517038 wait complete. 2024-09-10T11:38:28.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.768+0000 7f6f3ce18700 1 Processor -- start 2024-09-10T11:38:28.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.769+0000 7f6f3ce18700 1 -- start start 2024-09-10T11:38:28.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.769+0000 7f6f3ce18700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f381003d0 0x7f6f38073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.769+0000 7f6f3ce18700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f38100d70 0x7f6f38073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.769+0000 7f6f3ce18700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f38073ad0 con 0x7f6f381003d0 2024-09-10T11:38:28.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.769+0000 7f6f3ce18700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f38073c10 con 0x7f6f38100d70 2024-09-10T11:38:28.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.769+0000 7f6f3659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f381003d0 0x7f6f38073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.769+0000 7f6f3659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f381003d0 0x7f6f38073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50476/0 (socket says 172.21.15.144:50476) 2024-09-10T11:38:28.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.769+0000 7f6f3659c700 1 -- 172.21.15.144:0/1683799417 learned_addr learned my addr 172.21.15.144:0/1683799417 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.769+0000 7f6f35d9b700 1 --2- 172.21.15.144:0/1683799417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f38100d70 0x7f6f38073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.769+0000 7f6f3659c700 1 -- 172.21.15.144:0/1683799417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f38100d70 msgr2=0x7f6f38073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.769+0000 7f6f3659c700 1 --2- 172.21.15.144:0/1683799417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f38100d70 0x7f6f38073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.769+0000 7f6f3659c700 1 -- 172.21.15.144:0/1683799417 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f2c0095f0 con 0x7f6f381003d0 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.769+0000 7f6f3659c700 1 --2- 172.21.15.144:0/1683799417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f381003d0 0x7f6f38073050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800d900 tx=0x7f6f2800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.770+0000 7f6f277fe700 1 -- 172.21.15.144:0/1683799417 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f280041f0 con 0x7f6f381003d0 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.770+0000 7f6f277fe700 1 -- 172.21.15.144:0/1683799417 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f28004cf0 con 0x7f6f381003d0 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.770+0000 7f6f3ce18700 1 -- 172.21.15.144:0/1683799417 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f381a1f10 con 0x7f6f381003d0 2024-09-10T11:38:28.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.770+0000 7f6f277fe700 1 -- 172.21.15.144:0/1683799417 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f2800b750 con 0x7f6f381003d0 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.770+0000 7f6f3ce18700 1 -- 172.21.15.144:0/1683799417 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f381a2320 con 0x7f6f381003d0 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.771+0000 7f6f277fe700 1 -- 172.21.15.144:0/1683799417 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6f28004750 con 0x7f6f381003d0 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.771+0000 7f6f3ce18700 1 -- 172.21.15.144:0/1683799417 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f38108740 con 0x7f6f381003d0 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.772+0000 7f6f277fe700 1 --2- 172.21.15.144:0/1683799417 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6f2006e200 0x7f6f200706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:28.774 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.772+0000 7f6f277fe700 1 -- 172.21.15.144:0/1683799417 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6f2808dd40 con 0x7f6f381003d0 2024-09-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.772+0000 7f6f35d9b700 1 --2- 172.21.15.144:0/1683799417 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6f2006e200 0x7f6f200706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:28.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.772+0000 7f6f35d9b700 1 --2- 172.21.15.144:0/1683799417 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6f2006e200 0x7f6f200706b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6f38074840 tx=0x7f6f2c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:28.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.776+0000 7f6f277fe700 1 -- 172.21.15.144:0/1683799417 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6f28057120 con 0x7f6f381003d0 2024-09-10T11:38:28.887 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:28 smithi178 ceph-mon[65940]: pgmap v194: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:28.887 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:38:28.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:28 smithi144 ceph-mon[87436]: pgmap v194: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:28.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:38:29.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:28.999+0000 7f6f3ce18700 1 -- 172.21.15.144:0/1683799417 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f3802ccf0 con 0x7f6f2006e200 2024-09-10T11:38:29.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.000+0000 7f6f277fe700 1 -- 172.21.15.144:0/1683799417 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6f3802ccf0 con 0x7f6f2006e200 2024-09-10T11:38:29.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.002+0000 7f6f3ce18700 1 -- 172.21.15.144:0/1683799417 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6f2006e200 msgr2=0x7f6f200706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.002+0000 7f6f3ce18700 1 --2- 172.21.15.144:0/1683799417 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6f2006e200 0x7f6f200706b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6f38074840 tx=0x7f6f2c01a040 comp rx=0 tx=0).stop 2024-09-10T11:38:29.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.002+0000 7f6f3ce18700 1 -- 172.21.15.144:0/1683799417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f381003d0 msgr2=0x7f6f38073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.002+0000 7f6f3ce18700 1 --2- 172.21.15.144:0/1683799417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f381003d0 0x7f6f38073050 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6f2800d900 tx=0x7f6f2800dc10 comp rx=0 tx=0).stop 2024-09-10T11:38:29.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.002+0000 7f6f3ce18700 1 -- 172.21.15.144:0/1683799417 shutdown_connections 2024-09-10T11:38:29.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.002+0000 7f6f3ce18700 1 --2- 172.21.15.144:0/1683799417 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6f2006e200 0x7f6f200706b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.002+0000 7f6f3ce18700 1 --2- 172.21.15.144:0/1683799417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6f381003d0 0x7f6f38073050 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.002+0000 7f6f3ce18700 1 --2- 172.21.15.144:0/1683799417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6f38100d70 0x7f6f38073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.002+0000 7f6f3ce18700 1 -- 172.21.15.144:0/1683799417 >> 172.21.15.144:0/1683799417 conn(0x7f6f380fbc90 msgr2=0x7f6f380fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:29.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.002+0000 7f6f3ce18700 1 -- 172.21.15.144:0/1683799417 shutdown_connections 2024-09-10T11:38:29.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.002+0000 7f6f3ce18700 1 -- 172.21.15.144:0/1683799417 wait complete. 2024-09-10T11:38:29.014 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.108+0000 7ffabc821700 1 -- 172.21.15.144:0/2079148078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffab80ff270 msgr2=0x7ffab80ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.108+0000 7ffabc821700 1 --2- 172.21.15.144:0/2079148078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffab80ff270 0x7ffab80ff6e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ffaac00aa20 tx=0x7ffaac00ad30 comp rx=0 tx=0).stop 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.108+0000 7ffabc821700 1 -- 172.21.15.144:0/2079148078 shutdown_connections 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.108+0000 7ffabc821700 1 --2- 172.21.15.144:0/2079148078 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffab80ff270 0x7ffab80ff6e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.108+0000 7ffabc821700 1 --2- 172.21.15.144:0/2079148078 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffab81049a0 0x7ffab8104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.109+0000 7ffabc821700 1 -- 172.21.15.144:0/2079148078 >> 172.21.15.144:0/2079148078 conn(0x7ffab8074850 msgr2=0x7ffab8074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:29.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.109+0000 7ffabc821700 1 -- 172.21.15.144:0/2079148078 shutdown_connections 2024-09-10T11:38:29.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.109+0000 7ffabc821700 1 -- 172.21.15.144:0/2079148078 wait complete. 2024-09-10T11:38:29.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.109+0000 7ffabc821700 1 Processor -- start 2024-09-10T11:38:29.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.109+0000 7ffabc821700 1 -- start start 2024-09-10T11:38:29.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.110+0000 7ffabc821700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffab80ff270 0x7ffab8195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.110+0000 7ffabc821700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffab81049a0 0x7ffab8196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.110+0000 7ffabc821700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab81969f0 con 0x7ffab80ff270 2024-09-10T11:38:29.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.110+0000 7ffabc821700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab819a780 con 0x7ffab81049a0 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.110+0000 7ffab659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffab80ff270 0x7ffab8195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.110+0000 7ffab5d9b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffab81049a0 0x7ffab8196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.110+0000 7ffab659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffab80ff270 0x7ffab8195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50502/0 (socket says 172.21.15.144:50502) 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.110+0000 7ffab659c700 1 -- 172.21.15.144:0/1744445758 learned_addr learned my addr 172.21.15.144:0/1744445758 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.110+0000 7ffab5d9b700 1 -- 172.21.15.144:0/1744445758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffab80ff270 msgr2=0x7ffab8195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.110+0000 7ffab5d9b700 1 --2- 172.21.15.144:0/1744445758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffab80ff270 0x7ffab8195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.110+0000 7ffab5d9b700 1 -- 172.21.15.144:0/1744445758 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac00a700 con 0x7ffab81049a0 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.110+0000 7ffab659c700 1 --2- 172.21.15.144:0/1744445758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffab80ff270 0x7ffab8195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.110+0000 7ffab5d9b700 1 --2- 172.21.15.144:0/1744445758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffab81049a0 0x7ffab8196310 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006010 tx=0x7ffaac00cd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.111+0000 7ffaa77fe700 1 -- 172.21.15.144:0/1744445758 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaac004af0 con 0x7ffab81049a0 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.111+0000 7ffabc821700 1 -- 172.21.15.144:0/1744445758 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab819aa00 con 0x7ffab81049a0 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.111+0000 7ffaa77fe700 1 -- 172.21.15.144:0/1744445758 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffaac004c50 con 0x7ffab81049a0 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.111+0000 7ffaa77fe700 1 -- 172.21.15.144:0/1744445758 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffaac0167c0 con 0x7ffab81049a0 2024-09-10T11:38:29.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.111+0000 7ffabc821700 1 -- 172.21.15.144:0/1744445758 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab819aef0 con 0x7ffab81049a0 2024-09-10T11:38:29.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.111+0000 7ffabc821700 1 -- 172.21.15.144:0/1744445758 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab8101ba0 con 0x7ffab81049a0 2024-09-10T11:38:29.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.112+0000 7ffaa77fe700 1 -- 172.21.15.144:0/1744445758 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ffaac00eb30 con 0x7ffab81049a0 2024-09-10T11:38:29.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.113+0000 7ffaa77fe700 1 --2- 172.21.15.144:0/1744445758 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffaa006e200 0x7ffaa00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.113+0000 7ffaa77fe700 1 -- 172.21.15.144:0/1744445758 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ffaac012070 con 0x7ffab81049a0 2024-09-10T11:38:29.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.113+0000 7ffab659c700 1 --2- 172.21.15.144:0/1744445758 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffaa006e200 0x7ffaa00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.113+0000 7ffab659c700 1 --2- 172.21.15.144:0/1744445758 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffaa006e200 0x7ffaa00706b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80104e0 tx=0x7ffaa8007480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:29.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.121+0000 7ffaa77fe700 1 -- 172.21.15.144:0/1744445758 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ffaac0599d0 con 0x7ffab81049a0 2024-09-10T11:38:29.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.347+0000 7ffabc821700 1 -- 172.21.15.144:0/1744445758 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffab819b1d0 con 0x7ffaa006e200 2024-09-10T11:38:29.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.348+0000 7ffaa77fe700 1 -- 172.21.15.144:0/1744445758 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ffab819b1d0 con 0x7ffaa006e200 2024-09-10T11:38:29.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.351+0000 7ffabc821700 1 -- 172.21.15.144:0/1744445758 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffaa006e200 msgr2=0x7ffaa00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.351+0000 7ffabc821700 1 --2- 172.21.15.144:0/1744445758 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffaa006e200 0x7ffaa00706b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ffaa80104e0 tx=0x7ffaa8007480 comp rx=0 tx=0).stop 2024-09-10T11:38:29.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.351+0000 7ffabc821700 1 -- 172.21.15.144:0/1744445758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffab81049a0 msgr2=0x7ffab8196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.351+0000 7ffabc821700 1 --2- 172.21.15.144:0/1744445758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffab81049a0 0x7ffab8196310 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006010 tx=0x7ffaac00cd20 comp rx=0 tx=0).stop 2024-09-10T11:38:29.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.351+0000 7ffabc821700 1 -- 172.21.15.144:0/1744445758 shutdown_connections 2024-09-10T11:38:29.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.351+0000 7ffabc821700 1 --2- 172.21.15.144:0/1744445758 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffaa006e200 0x7ffaa00706b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.351+0000 7ffabc821700 1 --2- 172.21.15.144:0/1744445758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffab80ff270 0x7ffab8195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.351+0000 7ffabc821700 1 --2- 172.21.15.144:0/1744445758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffab81049a0 0x7ffab8196310 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.351+0000 7ffabc821700 1 -- 172.21.15.144:0/1744445758 >> 172.21.15.144:0/1744445758 conn(0x7ffab8074850 msgr2=0x7ffab80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:29.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.351+0000 7ffabc821700 1 -- 172.21.15.144:0/1744445758 shutdown_connections 2024-09-10T11:38:29.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.351+0000 7ffabc821700 1 -- 172.21.15.144:0/1744445758 wait complete. 2024-09-10T11:38:29.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.456+0000 7f259f4b9700 1 -- 172.21.15.144:0/3267787679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2598100d70 msgr2=0x7f2598104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.456+0000 7f259f4b9700 1 --2- 172.21.15.144:0/3267787679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2598100d70 0x7f2598104d40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2588009910 tx=0x7f2588009c20 comp rx=0 tx=0).stop 2024-09-10T11:38:29.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.456+0000 7f259f4b9700 1 -- 172.21.15.144:0/3267787679 shutdown_connections 2024-09-10T11:38:29.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.456+0000 7f259f4b9700 1 --2- 172.21.15.144:0/3267787679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2598100d70 0x7f2598104d40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.456+0000 7f259f4b9700 1 --2- 172.21.15.144:0/3267787679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25981003d0 0x7f25981007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.456+0000 7f259f4b9700 1 -- 172.21.15.144:0/3267787679 >> 172.21.15.144:0/3267787679 conn(0x7f25980fbc90 msgr2=0x7f25980fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:29.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.456+0000 7f259f4b9700 1 -- 172.21.15.144:0/3267787679 shutdown_connections 2024-09-10T11:38:29.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.456+0000 7f259f4b9700 1 -- 172.21.15.144:0/3267787679 wait complete. 2024-09-10T11:38:29.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.457+0000 7f259f4b9700 1 Processor -- start 2024-09-10T11:38:29.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.457+0000 7f259f4b9700 1 -- start start 2024-09-10T11:38:29.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.457+0000 7f259f4b9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25981003d0 0x7f2598195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.457+0000 7f259f4b9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2598100d70 0x7f2598196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.457+0000 7f259f4b9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2598196960 con 0x7f25981003d0 2024-09-10T11:38:29.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.457+0000 7f259f4b9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259819a6f0 con 0x7f2598100d70 2024-09-10T11:38:29.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.457+0000 7f259d255700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25981003d0 0x7f2598195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.457+0000 7f259d255700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25981003d0 0x7f2598195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50518/0 (socket says 172.21.15.144:50518) 2024-09-10T11:38:29.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.457+0000 7f259d255700 1 -- 172.21.15.144:0/3220584653 learned_addr learned my addr 172.21.15.144:0/3220584653 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:38:29.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.457+0000 7f259ca54700 1 --2- 172.21.15.144:0/3220584653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2598100d70 0x7f2598196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.457+0000 7f259d255700 1 -- 172.21.15.144:0/3220584653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2598100d70 msgr2=0x7f2598196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.457+0000 7f259d255700 1 --2- 172.21.15.144:0/3220584653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2598100d70 0x7f2598196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.457+0000 7f259d255700 1 -- 172.21.15.144:0/3220584653 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25880095f0 con 0x7f25981003d0 2024-09-10T11:38:29.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.457+0000 7f259d255700 1 --2- 172.21.15.144:0/3220584653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25981003d0 0x7f2598195d40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f259400e920 tx=0x7f259400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:29.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.458+0000 7f258e7fc700 1 -- 172.21.15.144:0/3220584653 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f259400c9d0 con 0x7f25981003d0 2024-09-10T11:38:29.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.458+0000 7f259f4b9700 1 -- 172.21.15.144:0/3220584653 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259819a9d0 con 0x7f25981003d0 2024-09-10T11:38:29.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.458+0000 7f258e7fc700 1 -- 172.21.15.144:0/3220584653 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2594010430 con 0x7f25981003d0 2024-09-10T11:38:29.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.458+0000 7f258e7fc700 1 -- 172.21.15.144:0/3220584653 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2594018630 con 0x7f25981003d0 2024-09-10T11:38:29.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.458+0000 7f259f4b9700 1 -- 172.21.15.144:0/3220584653 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259819aef0 con 0x7f25981003d0 2024-09-10T11:38:29.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.458+0000 7f259f4b9700 1 -- 172.21.15.144:0/3220584653 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2598108740 con 0x7f25981003d0 2024-09-10T11:38:29.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.459+0000 7f258e7fc700 1 -- 172.21.15.144:0/3220584653 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f25940105a0 con 0x7f25981003d0 2024-09-10T11:38:29.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.460+0000 7f258e7fc700 1 --2- 172.21.15.144:0/3220584653 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f258406e200 0x7f25840706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.460+0000 7f258e7fc700 1 -- 172.21.15.144:0/3220584653 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2594014070 con 0x7f25981003d0 2024-09-10T11:38:29.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.460+0000 7f259ca54700 1 --2- 172.21.15.144:0/3220584653 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f258406e200 0x7f25840706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.460+0000 7f259ca54700 1 --2- 172.21.15.144:0/3220584653 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f258406e200 0x7f25840706b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f25880059d0 tx=0x7f2588005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:29.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.464+0000 7f258e7fc700 1 -- 172.21.15.144:0/3220584653 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f25940541b0 con 0x7f25981003d0 2024-09-10T11:38:29.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.675+0000 7f259f4b9700 1 -- 172.21.15.144:0/3220584653 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f259819b220 con 0x7f258406e200 2024-09-10T11:38:29.686 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.685+0000 7f258e7fc700 1 -- 172.21.15.144:0/3220584653 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f259819b220 con 0x7f258406e200 2024-09-10T11:38:29.686 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:38:29.686 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (6m) 25s ago 17m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:38:29.686 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (17m) 25s ago 17m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:38:29.686 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (16m) 12s ago 16m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:38:29.687 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (4m) 25s ago 17m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:38:29.687 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (4m) 12s ago 16m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:38:29.687 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (6m) 25s ago 17m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:38:29.688 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (48s) 25s ago 11m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:38:29.688 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (31s) 25s ago 11m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:38:29.688 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (11m) 12s ago 11m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:38:29.688 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (15s) 12s ago 11m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:38:29.688 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (8m) 25s ago 18m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:38:29.688 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (7m) 12s ago 16m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:38:29.688 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (5m) 25s ago 18m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:38:29.688 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (5m) 12s ago 16m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:38:29.689 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (7m) 25s ago 17m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:38:29.689 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (7m) 12s ago 16m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:38:29.689 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (3m) 25s ago 15m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:38:29.689 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (3m) 25s ago 15m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:38:29.689 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (2m) 25s ago 14m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:38:29.689 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (2m) 12s ago 14m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:38:29.689 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (98s) 12s ago 13m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:38:29.689 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (65s) 12s ago 13m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:38:29.689 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (7m) 25s ago 16m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:38:29.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.687+0000 7f259f4b9700 1 -- 172.21.15.144:0/3220584653 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f258406e200 msgr2=0x7f25840706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.687+0000 7f259f4b9700 1 --2- 172.21.15.144:0/3220584653 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f258406e200 0x7f25840706b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f25880059d0 tx=0x7f2588005940 comp rx=0 tx=0).stop 2024-09-10T11:38:29.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.687+0000 7f259f4b9700 1 -- 172.21.15.144:0/3220584653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25981003d0 msgr2=0x7f2598195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.687+0000 7f259f4b9700 1 --2- 172.21.15.144:0/3220584653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25981003d0 0x7f2598195d40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f259400e920 tx=0x7f259400ec30 comp rx=0 tx=0).stop 2024-09-10T11:38:29.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.687+0000 7f259f4b9700 1 -- 172.21.15.144:0/3220584653 shutdown_connections 2024-09-10T11:38:29.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.687+0000 7f259f4b9700 1 --2- 172.21.15.144:0/3220584653 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f258406e200 0x7f25840706b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.687+0000 7f259f4b9700 1 --2- 172.21.15.144:0/3220584653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25981003d0 0x7f2598195d40 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.687+0000 7f259f4b9700 1 --2- 172.21.15.144:0/3220584653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2598100d70 0x7f2598196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.687+0000 7f259f4b9700 1 -- 172.21.15.144:0/3220584653 >> 172.21.15.144:0/3220584653 conn(0x7f25980fbc90 msgr2=0x7f25980fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:29.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.687+0000 7f259f4b9700 1 -- 172.21.15.144:0/3220584653 shutdown_connections 2024-09-10T11:38:29.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.687+0000 7f259f4b9700 1 -- 172.21.15.144:0/3220584653 wait complete. 2024-09-10T11:38:29.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.791+0000 7f99e468a700 1 -- 172.21.15.144:0/423257255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99dc0ffc70 msgr2=0x7f99dc103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.791+0000 7f99e468a700 1 --2- 172.21.15.144:0/423257255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99dc0ffc70 0x7f99dc103c40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f99d0009910 tx=0x7f99d0009c20 comp rx=0 tx=0).stop 2024-09-10T11:38:29.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.791+0000 7f99e468a700 1 -- 172.21.15.144:0/423257255 shutdown_connections 2024-09-10T11:38:29.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.791+0000 7f99e468a700 1 --2- 172.21.15.144:0/423257255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99dc0ffc70 0x7f99dc103c40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.791+0000 7f99e468a700 1 --2- 172.21.15.144:0/423257255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99dc0ff2d0 0x7f99dc0ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.791+0000 7f99e468a700 1 -- 172.21.15.144:0/423257255 >> 172.21.15.144:0/423257255 conn(0x7f99dc0fabb0 msgr2=0x7f99dc0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:29.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.791+0000 7f99e468a700 1 -- 172.21.15.144:0/423257255 shutdown_connections 2024-09-10T11:38:29.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.791+0000 7f99e468a700 1 -- 172.21.15.144:0/423257255 wait complete. 2024-09-10T11:38:29.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.792+0000 7f99e468a700 1 Processor -- start 2024-09-10T11:38:29.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.792+0000 7f99e468a700 1 -- start start 2024-09-10T11:38:29.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.792+0000 7f99e468a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99dc0ff2d0 0x7f99dc195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.792+0000 7f99e468a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99dc0ffc70 0x7f99dc1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.792+0000 7f99e468a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc1969c0 con 0x7f99dc0ffc70 2024-09-10T11:38:29.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.792+0000 7f99e468a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99dc19a750 con 0x7f99dc0ff2d0 2024-09-10T11:38:29.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.792+0000 7f99e1c25700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99dc0ffc70 0x7f99dc1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.792+0000 7f99e1c25700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99dc0ffc70 0x7f99dc1962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50534/0 (socket says 172.21.15.144:50534) 2024-09-10T11:38:29.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.792+0000 7f99e1c25700 1 -- 172.21.15.144:0/2103690277 learned_addr learned my addr 172.21.15.144:0/2103690277 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:38:29.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.792+0000 7f99e2426700 1 --2- 172.21.15.144:0/2103690277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99dc0ff2d0 0x7f99dc195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.792+0000 7f99e1c25700 1 -- 172.21.15.144:0/2103690277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99dc0ff2d0 msgr2=0x7f99dc195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:29.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.792+0000 7f99e1c25700 1 --2- 172.21.15.144:0/2103690277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99dc0ff2d0 0x7f99dc195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:29.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.792+0000 7f99e1c25700 1 -- 172.21.15.144:0/2103690277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99d00095f0 con 0x7f99dc0ffc70 2024-09-10T11:38:29.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.792+0000 7f99e1c25700 1 --2- 172.21.15.144:0/2103690277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99dc0ffc70 0x7f99dc1962e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f99d00059d0 tx=0x7f99d000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:29.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.793+0000 7f99db7fe700 1 -- 172.21.15.144:0/2103690277 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99d001d070 con 0x7f99dc0ffc70 2024-09-10T11:38:29.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.793+0000 7f99e468a700 1 -- 172.21.15.144:0/2103690277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99dc19a9d0 con 0x7f99dc0ffc70 2024-09-10T11:38:29.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.793+0000 7f99db7fe700 1 -- 172.21.15.144:0/2103690277 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f99d000b610 con 0x7f99dc0ffc70 2024-09-10T11:38:29.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.793+0000 7f99db7fe700 1 -- 172.21.15.144:0/2103690277 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99d0022470 con 0x7f99dc0ffc70 2024-09-10T11:38:29.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.793+0000 7f99e468a700 1 -- 172.21.15.144:0/2103690277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99dc19aec0 con 0x7f99dc0ffc70 2024-09-10T11:38:29.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.794+0000 7f99e468a700 1 -- 172.21.15.144:0/2103690277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99dc108550 con 0x7f99dc0ffc70 2024-09-10T11:38:29.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.794+0000 7f99db7fe700 1 -- 172.21.15.144:0/2103690277 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f99d0003710 con 0x7f99dc0ffc70 2024-09-10T11:38:29.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.795+0000 7f99db7fe700 1 --2- 172.21.15.144:0/2103690277 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f99c406e200 0x7f99c40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:29.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.795+0000 7f99e2426700 1 --2- 172.21.15.144:0/2103690277 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f99c406e200 0x7f99c40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:29.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.795+0000 7f99db7fe700 1 -- 172.21.15.144:0/2103690277 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f99d008f570 con 0x7f99dc0ffc70 2024-09-10T11:38:29.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.795+0000 7f99e2426700 1 --2- 172.21.15.144:0/2103690277 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f99c406e200 0x7f99c40706b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f99d400f540 tx=0x7f99d400b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:29.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:29.799+0000 7f99db7fe700 1 -- 172.21.15.144:0/2103690277 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f99d0058950 con 0x7f99dc0ffc70 2024-09-10T11:38:30.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.094+0000 7f99e468a700 1 -- 172.21.15.144:0/2103690277 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f99dc04ea50 con 0x7f99dc0ffc70 2024-09-10T11:38:30.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.094+0000 7f99db7fe700 1 -- 172.21.15.144:0/2103690277 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f99d005c010 con 0x7f99dc0ffc70 2024-09-10T11:38:30.096 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:38:30.096 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:38:30.096 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:38:30.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:38:30.096 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:38:30.096 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:38:30.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:38:30.096 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:38:30.096 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:38:30.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:38:30.096 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:38:30.097 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:38:30.097 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:38:30.097 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:38:30.097 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:38:30.097 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:38:30.097 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:38:30.097 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:38:30.097 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:38:30.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.097+0000 7f99e468a700 1 -- 172.21.15.144:0/2103690277 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f99c406e200 msgr2=0x7f99c40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.097+0000 7f99e468a700 1 --2- 172.21.15.144:0/2103690277 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f99c406e200 0x7f99c40706b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f99d400f540 tx=0x7f99d400b040 comp rx=0 tx=0).stop 2024-09-10T11:38:30.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.097+0000 7f99e468a700 1 -- 172.21.15.144:0/2103690277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99dc0ffc70 msgr2=0x7f99dc1962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.097+0000 7f99e468a700 1 --2- 172.21.15.144:0/2103690277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99dc0ffc70 0x7f99dc1962e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f99d00059d0 tx=0x7f99d000f690 comp rx=0 tx=0).stop 2024-09-10T11:38:30.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.097+0000 7f99e468a700 1 -- 172.21.15.144:0/2103690277 shutdown_connections 2024-09-10T11:38:30.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.097+0000 7f99e468a700 1 --2- 172.21.15.144:0/2103690277 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f99c406e200 0x7f99c40706b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.097+0000 7f99e468a700 1 --2- 172.21.15.144:0/2103690277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f99dc0ff2d0 0x7f99dc195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.097+0000 7f99e468a700 1 --2- 172.21.15.144:0/2103690277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f99dc0ffc70 0x7f99dc1962e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.097+0000 7f99e468a700 1 -- 172.21.15.144:0/2103690277 >> 172.21.15.144:0/2103690277 conn(0x7f99dc0fabb0 msgr2=0x7f99dc0fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:30.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.097+0000 7f99e468a700 1 -- 172.21.15.144:0/2103690277 shutdown_connections 2024-09-10T11:38:30.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.097+0000 7f99e468a700 1 -- 172.21.15.144:0/2103690277 wait complete. 2024-09-10T11:38:30.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.202+0000 7f9ae788e700 1 -- 172.21.15.144:0/2301996998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae00ff390 msgr2=0x7f9ae00ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.202+0000 7f9ae788e700 1 --2- 172.21.15.144:0/2301996998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae00ff390 0x7f9ae00ff760 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009910 tx=0x7f9ad0009c20 comp rx=0 tx=0).stop 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.202+0000 7f9ae788e700 1 -- 172.21.15.144:0/2301996998 shutdown_connections 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.202+0000 7f9ae788e700 1 --2- 172.21.15.144:0/2301996998 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ae00ffd30 0x7f9ae0103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.202+0000 7f9ae788e700 1 --2- 172.21.15.144:0/2301996998 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae00ff390 0x7f9ae00ff760 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.202+0000 7f9ae788e700 1 -- 172.21.15.144:0/2301996998 >> 172.21.15.144:0/2301996998 conn(0x7f9ae0074f50 msgr2=0x7f9ae0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.202+0000 7f9ae788e700 1 -- 172.21.15.144:0/2301996998 shutdown_connections 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.202+0000 7f9ae788e700 1 -- 172.21.15.144:0/2301996998 wait complete. 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.203+0000 7f9ae788e700 1 Processor -- start 2024-09-10T11:38:30.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.203+0000 7f9ae788e700 1 -- start start 2024-09-10T11:38:30.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.203+0000 7f9ae788e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ae00ff390 0x7f9ae0195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.203+0000 7f9ae788e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae00ffd30 0x7f9ae0196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.203+0000 7f9ae788e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae0196940 con 0x7f9ae00ffd30 2024-09-10T11:38:30.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.203+0000 7f9ae788e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae019a6d0 con 0x7f9ae00ff390 2024-09-10T11:38:30.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.203+0000 7f9ae4e29700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae00ffd30 0x7f9ae0196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.203+0000 7f9ae4e29700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae00ffd30 0x7f9ae0196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50548/0 (socket says 172.21.15.144:50548) 2024-09-10T11:38:30.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.203+0000 7f9ae4e29700 1 -- 172.21.15.144:0/212543541 learned_addr learned my addr 172.21.15.144:0/212543541 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:38:30.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.203+0000 7f9ae562a700 1 --2- 172.21.15.144:0/212543541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ae00ff390 0x7f9ae0195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.203+0000 7f9ae4e29700 1 -- 172.21.15.144:0/212543541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ae00ff390 msgr2=0x7f9ae0195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.203+0000 7f9ae4e29700 1 --2- 172.21.15.144:0/212543541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ae00ff390 0x7f9ae0195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.203+0000 7f9ae4e29700 1 -- 172.21.15.144:0/212543541 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad00095f0 con 0x7f9ae00ffd30 2024-09-10T11:38:30.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.203+0000 7f9ae4e29700 1 --2- 172.21.15.144:0/212543541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae00ffd30 0x7f9ae0196260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00c8f0 tx=0x7f9adc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.204+0000 7f9ad67fc700 1 -- 172.21.15.144:0/212543541 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9adc007a50 con 0x7f9ae00ffd30 2024-09-10T11:38:30.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.204+0000 7f9ad67fc700 1 -- 172.21.15.144:0/212543541 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9adc007bb0 con 0x7f9ae00ffd30 2024-09-10T11:38:30.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.204+0000 7f9ae788e700 1 -- 172.21.15.144:0/212543541 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae019a9b0 con 0x7f9ae00ffd30 2024-09-10T11:38:30.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.204+0000 7f9ad67fc700 1 -- 172.21.15.144:0/212543541 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9adc018630 con 0x7f9ae00ffd30 2024-09-10T11:38:30.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.204+0000 7f9ae788e700 1 -- 172.21.15.144:0/212543541 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae019af00 con 0x7f9ae00ffd30 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.205+0000 7f9ae788e700 1 -- 172.21.15.144:0/212543541 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae004ea50 con 0x7f9ae00ffd30 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.205+0000 7f9ad67fc700 1 -- 172.21.15.144:0/212543541 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9adc007d20 con 0x7f9ae00ffd30 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.206+0000 7f9ad67fc700 1 --2- 172.21.15.144:0/212543541 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9acc06e200 0x7f9acc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.206+0000 7f9ad67fc700 1 -- 172.21.15.144:0/212543541 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9adc08df70 con 0x7f9ae00ffd30 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.206+0000 7f9ae562a700 1 --2- 172.21.15.144:0/212543541 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9acc06e200 0x7f9acc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.206+0000 7f9ae562a700 1 --2- 172.21.15.144:0/212543541 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9acc06e200 0x7f9acc0706b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000b5c0 tx=0x7f9ad001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.210+0000 7f9ad67fc700 1 -- 172.21.15.144:0/212543541 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9adc0c31b0 con 0x7f9ae00ffd30 2024-09-10T11:38:30.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.462+0000 7f9ae788e700 1 -- 172.21.15.144:0/212543541 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9ae0197080 con 0x7f9ae00ffd30 2024-09-10T11:38:30.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.462+0000 7f9ad67fc700 1 -- 172.21.15.144:0/212543541 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f9adc0573d0 con 0x7f9ae00ffd30 2024-09-10T11:38:30.464 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:38:30.464 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:38:30.464 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:38:30.464 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:38:30.464 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:38:30.464 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:38:30.464 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:38:30.464 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:38:30.464 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:38:30.464 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:38:30.464 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:38:30.464 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:38:30.465 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:38:30.465 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:38:30.465 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:38:30.465 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:38:30.465 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:38:30.465 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:38:30.465 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:38:30.465 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:38:30.465 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:38:30.465 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:38:30.465 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:38:30.465 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:38:30.465 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:38:30.466 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:38:30.466 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:38:30.466 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:38:30.466 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:38:30.466 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:38:30.466 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:38:30.466 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:38:30.466 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:38:30.466 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:38:30.466 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:38:30.466 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:38:30.466 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:38:30.466 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:38:30.466 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:38:30.467 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:38:30.467 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:38:30.467 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:38:30.467 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:38:30.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.464+0000 7f9ae788e700 1 -- 172.21.15.144:0/212543541 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9acc06e200 msgr2=0x7f9acc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.464+0000 7f9ae788e700 1 --2- 172.21.15.144:0/212543541 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9acc06e200 0x7f9acc0706b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f9ad000b5c0 tx=0x7f9ad001a040 comp rx=0 tx=0).stop 2024-09-10T11:38:30.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.464+0000 7f9ae788e700 1 -- 172.21.15.144:0/212543541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae00ffd30 msgr2=0x7f9ae0196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.464+0000 7f9ae788e700 1 --2- 172.21.15.144:0/212543541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae00ffd30 0x7f9ae0196260 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9adc00c8f0 tx=0x7f9adc00ccb0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.464+0000 7f9ae788e700 1 -- 172.21.15.144:0/212543541 shutdown_connections 2024-09-10T11:38:30.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.464+0000 7f9ae788e700 1 --2- 172.21.15.144:0/212543541 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9acc06e200 0x7f9acc0706b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.464+0000 7f9ae788e700 1 --2- 172.21.15.144:0/212543541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ae00ff390 0x7f9ae0195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.464+0000 7f9ae788e700 1 --2- 172.21.15.144:0/212543541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ae00ffd30 0x7f9ae0196260 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.464+0000 7f9ae788e700 1 -- 172.21.15.144:0/212543541 >> 172.21.15.144:0/212543541 conn(0x7f9ae0074f50 msgr2=0x7f9ae0103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:30.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.465+0000 7f9ae788e700 1 -- 172.21.15.144:0/212543541 shutdown_connections 2024-09-10T11:38:30.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.465+0000 7f9ae788e700 1 -- 172.21.15.144:0/212543541 wait complete. 2024-09-10T11:38:30.469 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:38:30.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.569+0000 7fe840dc9700 1 -- 172.21.15.144:0/18914625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe83c0ffc70 msgr2=0x7fe83c103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.569+0000 7fe840dc9700 1 --2- 172.21.15.144:0/18914625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe83c0ffc70 0x7fe83c103c40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe830009910 tx=0x7fe830009c20 comp rx=0 tx=0).stop 2024-09-10T11:38:30.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.569+0000 7fe840dc9700 1 -- 172.21.15.144:0/18914625 shutdown_connections 2024-09-10T11:38:30.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.569+0000 7fe840dc9700 1 --2- 172.21.15.144:0/18914625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe83c0ffc70 0x7fe83c103c40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.569+0000 7fe840dc9700 1 --2- 172.21.15.144:0/18914625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe83c0ff2d0 0x7fe83c0ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.569+0000 7fe840dc9700 1 -- 172.21.15.144:0/18914625 >> 172.21.15.144:0/18914625 conn(0x7fe83c0fabb0 msgr2=0x7fe83c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:30.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.570+0000 7fe840dc9700 1 -- 172.21.15.144:0/18914625 shutdown_connections 2024-09-10T11:38:30.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.570+0000 7fe840dc9700 1 -- 172.21.15.144:0/18914625 wait complete. 2024-09-10T11:38:30.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.570+0000 7fe840dc9700 1 Processor -- start 2024-09-10T11:38:30.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.570+0000 7fe840dc9700 1 -- start start 2024-09-10T11:38:30.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.570+0000 7fe840dc9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe83c0ff2d0 0x7fe83c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.570+0000 7fe840dc9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe83c0ffc70 0x7fe83c19af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.570+0000 7fe840dc9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c19b600 con 0x7fe83c0ffc70 2024-09-10T11:38:30.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.570+0000 7fe840dc9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c194ab0 con 0x7fe83c0ff2d0 2024-09-10T11:38:30.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.571+0000 7fe839d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe83c0ffc70 0x7fe83c19af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.571+0000 7fe839d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe83c0ffc70 0x7fe83c19af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50554/0 (socket says 172.21.15.144:50554) 2024-09-10T11:38:30.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.571+0000 7fe839d9b700 1 -- 172.21.15.144:0/3878597250 learned_addr learned my addr 172.21.15.144:0/3878597250 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:38:30.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.571+0000 7fe83a59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe83c0ff2d0 0x7fe83c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.571+0000 7fe839d9b700 1 -- 172.21.15.144:0/3878597250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe83c0ff2d0 msgr2=0x7fe83c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.571+0000 7fe839d9b700 1 --2- 172.21.15.144:0/3878597250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe83c0ff2d0 0x7fe83c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.571+0000 7fe839d9b700 1 -- 172.21.15.144:0/3878597250 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8300095f0 con 0x7fe83c0ffc70 2024-09-10T11:38:30.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.571+0000 7fe839d9b700 1 --2- 172.21.15.144:0/3878597250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe83c0ffc70 0x7fe83c19af70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe83000bd70 tx=0x7fe83000bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.571+0000 7fe82b7fe700 1 -- 172.21.15.144:0/3878597250 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe83001d070 con 0x7fe83c0ffc70 2024-09-10T11:38:30.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.571+0000 7fe82b7fe700 1 -- 172.21.15.144:0/3878597250 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe830005dc0 con 0x7fe83c0ffc70 2024-09-10T11:38:30.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.571+0000 7fe840dc9700 1 -- 172.21.15.144:0/3878597250 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c194d30 con 0x7fe83c0ffc70 2024-09-10T11:38:30.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.571+0000 7fe82b7fe700 1 -- 172.21.15.144:0/3878597250 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe830017400 con 0x7fe83c0ffc70 2024-09-10T11:38:30.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.571+0000 7fe840dc9700 1 -- 172.21.15.144:0/3878597250 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c195220 con 0x7fe83c0ffc70 2024-09-10T11:38:30.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.572+0000 7fe840dc9700 1 -- 172.21.15.144:0/3878597250 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe83c04ea50 con 0x7fe83c0ffc70 2024-09-10T11:38:30.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.573+0000 7fe82b7fe700 1 -- 172.21.15.144:0/3878597250 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe83000f460 con 0x7fe83c0ffc70 2024-09-10T11:38:30.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.573+0000 7fe82b7fe700 1 --2- 172.21.15.144:0/3878597250 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe82406e200 0x7fe8240706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.573+0000 7fe82b7fe700 1 -- 172.21.15.144:0/3878597250 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe83008f420 con 0x7fe83c0ffc70 2024-09-10T11:38:30.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.573+0000 7fe83a59c700 1 --2- 172.21.15.144:0/3878597250 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe82406e200 0x7fe8240706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.577 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.574+0000 7fe83a59c700 1 --2- 172.21.15.144:0/3878597250 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe82406e200 0x7fe8240706b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe82c005950 tx=0x7fe82c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.579 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.578+0000 7fe82b7fe700 1 -- 172.21.15.144:0/3878597250 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe83005c3b0 con 0x7fe83c0ffc70 2024-09-10T11:38:30.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.800+0000 7fe840dc9700 1 -- 172.21.15.144:0/3878597250 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe83c195650 con 0x7fe82406e200 2024-09-10T11:38:30.801 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:30 smithi144 ceph-mon[87436]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:38:30.801 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:30 smithi144 ceph-mon[87436]: from='client.44363 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:38:30.801 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:30 smithi144 ceph-mon[87436]: pgmap v195: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:30.801 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:30 smithi144 ceph-mon[87436]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:38:30.801 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:30 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2103690277' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:38:30.801 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:30 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/212543541' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:38:30.802 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.801+0000 7fe82b7fe700 1 -- 172.21.15.144:0/3878597250 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe83c195650 con 0x7fe82406e200 2024-09-10T11:38:30.802 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:38:30.802 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:38:30.802 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:38:30.802 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:38:30.802 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:38:30.802 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:38:30.802 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:38:30.803 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:38:30.803 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:38:30.803 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:38:30.803 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:38:30.803 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:38:30.803 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:38:30.803 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:38:30.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.803+0000 7fe840dc9700 1 -- 172.21.15.144:0/3878597250 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe82406e200 msgr2=0x7fe8240706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.803+0000 7fe840dc9700 1 --2- 172.21.15.144:0/3878597250 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe82406e200 0x7fe8240706b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe82c005950 tx=0x7fe82c00b410 comp rx=0 tx=0).stop 2024-09-10T11:38:30.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.803+0000 7fe840dc9700 1 -- 172.21.15.144:0/3878597250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe83c0ffc70 msgr2=0x7fe83c19af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.803+0000 7fe840dc9700 1 --2- 172.21.15.144:0/3878597250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe83c0ffc70 0x7fe83c19af70 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe83000bd70 tx=0x7fe83000bda0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.804+0000 7fe840dc9700 1 -- 172.21.15.144:0/3878597250 shutdown_connections 2024-09-10T11:38:30.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.804+0000 7fe840dc9700 1 --2- 172.21.15.144:0/3878597250 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe82406e200 0x7fe8240706b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.804+0000 7fe840dc9700 1 --2- 172.21.15.144:0/3878597250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe83c0ff2d0 0x7fe83c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.804+0000 7fe840dc9700 1 --2- 172.21.15.144:0/3878597250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe83c0ffc70 0x7fe83c19af70 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.804+0000 7fe840dc9700 1 -- 172.21.15.144:0/3878597250 >> 172.21.15.144:0/3878597250 conn(0x7fe83c0fabb0 msgr2=0x7fe83c0fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:30.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.804+0000 7fe840dc9700 1 -- 172.21.15.144:0/3878597250 shutdown_connections 2024-09-10T11:38:30.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.804+0000 7fe840dc9700 1 -- 172.21.15.144:0/3878597250 wait complete. 2024-09-10T11:38:30.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.908+0000 7f4b03d84700 1 -- 172.21.15.144:0/4133241546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4afc0ff220 msgr2=0x7f4afc0ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.908+0000 7f4b03d84700 1 --2- 172.21.15.144:0/4133241546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4afc0ff220 0x7f4afc0ff5f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009940 tx=0x7f4aec009c50 comp rx=0 tx=0).stop 2024-09-10T11:38:30.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.908+0000 7f4b03d84700 1 -- 172.21.15.144:0/4133241546 shutdown_connections 2024-09-10T11:38:30.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.908+0000 7f4b03d84700 1 --2- 172.21.15.144:0/4133241546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4afc0ffb30 0x7f4afc10a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.908+0000 7f4b03d84700 1 --2- 172.21.15.144:0/4133241546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4afc0ff220 0x7f4afc0ff5f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.908+0000 7f4b03d84700 1 -- 172.21.15.144:0/4133241546 >> 172.21.15.144:0/4133241546 conn(0x7f4afc0747e0 msgr2=0x7f4afc074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:30.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.908+0000 7f4b03d84700 1 -- 172.21.15.144:0/4133241546 shutdown_connections 2024-09-10T11:38:30.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.908+0000 7f4b03d84700 1 -- 172.21.15.144:0/4133241546 wait complete. 2024-09-10T11:38:30.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.908+0000 7f4b03d84700 1 Processor -- start 2024-09-10T11:38:30.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4b03d84700 1 -- start start 2024-09-10T11:38:30.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4b03d84700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4afc0ff220 0x7f4afc19ba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4b03d84700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4afc0ffb30 0x7f4afc19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4b03d84700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc19c620 con 0x7f4afc0ffb30 2024-09-10T11:38:30.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4b03d84700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4afc195ad0 con 0x7f4afc0ff220 2024-09-10T11:38:30.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4b0131f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4afc0ffb30 0x7f4afc19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4b0131f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4afc0ffb30 0x7f4afc19bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50564/0 (socket says 172.21.15.144:50564) 2024-09-10T11:38:30.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4b0131f700 1 -- 172.21.15.144:0/1591601963 learned_addr learned my addr 172.21.15.144:0/1591601963 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:38:30.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4b01b20700 1 --2- 172.21.15.144:0/1591601963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4afc0ff220 0x7f4afc19ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4b0131f700 1 -- 172.21.15.144:0/1591601963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4afc0ff220 msgr2=0x7f4afc19ba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:30.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4b0131f700 1 --2- 172.21.15.144:0/1591601963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4afc0ff220 0x7f4afc19ba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:30.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4b0131f700 1 -- 172.21.15.144:0/1591601963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aec0095f0 con 0x7f4afc0ffb30 2024-09-10T11:38:30.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4b01b20700 1 --2- 172.21.15.144:0/1591601963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4afc0ff220 0x7f4afc19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:38:30.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4b0131f700 1 --2- 172.21.15.144:0/1591601963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4afc0ffb30 0x7f4afc19bf90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4af800d8d0 tx=0x7f4af800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4af2ffd700 1 -- 172.21.15.144:0/1591601963 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4af8009940 con 0x7f4afc0ffb30 2024-09-10T11:38:30.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4b03d84700 1 -- 172.21.15.144:0/1591601963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4afc195db0 con 0x7f4afc0ffb30 2024-09-10T11:38:30.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4af2ffd700 1 -- 172.21.15.144:0/1591601963 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4af8010460 con 0x7f4afc0ffb30 2024-09-10T11:38:30.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.909+0000 7f4af2ffd700 1 -- 172.21.15.144:0/1591601963 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4af800f5f0 con 0x7f4afc0ffb30 2024-09-10T11:38:30.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.910+0000 7f4b03d84700 1 -- 172.21.15.144:0/1591601963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4afc196300 con 0x7f4afc0ffb30 2024-09-10T11:38:30.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.910+0000 7f4b03d84700 1 -- 172.21.15.144:0/1591601963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4afc107ca0 con 0x7f4afc0ffb30 2024-09-10T11:38:30.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.911+0000 7f4af2ffd700 1 -- 172.21.15.144:0/1591601963 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4af80105d0 con 0x7f4afc0ffb30 2024-09-10T11:38:30.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.911+0000 7f4af2ffd700 1 --2- 172.21.15.144:0/1591601963 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ae806e200 0x7f4ae80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:38:30.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.912+0000 7f4b01b20700 1 --2- 172.21.15.144:0/1591601963 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ae806e200 0x7f4ae80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:38:30.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.912+0000 7f4af2ffd700 1 -- 172.21.15.144:0/1591601963 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4af808e0a0 con 0x7f4afc0ffb30 2024-09-10T11:38:30.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.912+0000 7f4b01b20700 1 --2- 172.21.15.144:0/1591601963 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ae806e200 0x7f4ae80706b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4aec005b40 tx=0x7f4aec005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:38:30.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:30.916+0000 7f4af2ffd700 1 -- 172.21.15.144:0/1591601963 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4af805afb0 con 0x7f4afc0ffb30 2024-09-10T11:38:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:30 smithi178 ceph-mon[65940]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:38:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:30 smithi178 ceph-mon[65940]: from='client.44363 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:38:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:30 smithi178 ceph-mon[65940]: pgmap v195: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:30 smithi178 ceph-mon[65940]: from='client.34436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:38:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:30 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2103690277' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:38:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:30 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/212543541' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:38:31.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:31.203+0000 7f4b03d84700 1 -- 172.21.15.144:0/1591601963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4afc04ea50 con 0x7f4afc0ffb30 2024-09-10T11:38:31.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:31.204+0000 7f4af2ffd700 1 -- 172.21.15.144:0/1591601963 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f4af805ab40 con 0x7f4afc0ffb30 2024-09-10T11:38:31.205 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:38:31.205 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:38:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:38:31.205 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:38:31.205 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:38:31.205 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:38:31.206 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:38:31.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:31.206+0000 7f4b03d84700 1 -- 172.21.15.144:0/1591601963 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ae806e200 msgr2=0x7f4ae80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:31.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:31.206+0000 7f4b03d84700 1 --2- 172.21.15.144:0/1591601963 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ae806e200 0x7f4ae80706b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4aec005b40 tx=0x7f4aec005a90 comp rx=0 tx=0).stop 2024-09-10T11:38:31.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:31.206+0000 7f4b03d84700 1 -- 172.21.15.144:0/1591601963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4afc0ffb30 msgr2=0x7f4afc19bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:38:31.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:31.206+0000 7f4b03d84700 1 --2- 172.21.15.144:0/1591601963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4afc0ffb30 0x7f4afc19bf90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f4af800d8d0 tx=0x7f4af800dc90 comp rx=0 tx=0).stop 2024-09-10T11:38:31.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:31.206+0000 7f4b03d84700 1 -- 172.21.15.144:0/1591601963 shutdown_connections 2024-09-10T11:38:31.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:31.206+0000 7f4b03d84700 1 --2- 172.21.15.144:0/1591601963 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ae806e200 0x7f4ae80706b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:31.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:31.206+0000 7f4b03d84700 1 --2- 172.21.15.144:0/1591601963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4afc0ff220 0x7f4afc19ba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:31.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:31.206+0000 7f4b03d84700 1 --2- 172.21.15.144:0/1591601963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4afc0ffb30 0x7f4afc19bf90 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:38:31.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:31.206+0000 7f4b03d84700 1 -- 172.21.15.144:0/1591601963 >> 172.21.15.144:0/1591601963 conn(0x7f4afc0747e0 msgr2=0x7f4afc104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:38:31.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:31.206+0000 7f4b03d84700 1 -- 172.21.15.144:0/1591601963 shutdown_connections 2024-09-10T11:38:31.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:38:31.206+0000 7f4b03d84700 1 -- 172.21.15.144:0/1591601963 wait complete. 2024-09-10T11:38:32.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:31 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1591601963' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:38:32.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:31 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1591601963' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:38:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:32 smithi144 ceph-mon[87436]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:38:33.115 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:32 smithi144 ceph-mon[87436]: pgmap v196: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:33.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:32 smithi178 ceph-mon[65940]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:38:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:32 smithi178 ceph-mon[65940]: pgmap v196: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:34 smithi178 ceph-mon[65940]: pgmap v197: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:34 smithi144 ceph-mon[87436]: pgmap v197: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:36 smithi178 ceph-mon[65940]: pgmap v198: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:37.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:36 smithi144 ceph-mon[87436]: pgmap v198: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:38.985 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:38 smithi144 ceph-mon[87436]: pgmap v199: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:38 smithi178 ceph-mon[65940]: pgmap v199: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:40 smithi178 ceph-mon[65940]: pgmap v200: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:41.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:40 smithi144 ceph-mon[87436]: pgmap v200: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:43.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:42 smithi144 ceph-mon[87436]: pgmap v201: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:43.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:42 smithi178 ceph-mon[65940]: pgmap v201: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:38:44.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:38:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:44 smithi178 ceph-mon[65940]: pgmap v202: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:44 smithi144 ceph-mon[87436]: pgmap v202: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:46 smithi178 ceph-mon[65940]: pgmap v203: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:46 smithi144 ceph-mon[87436]: pgmap v203: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:48 smithi178 ceph-mon[65940]: pgmap v204: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:49.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:48 smithi144 ceph-mon[87436]: pgmap v204: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:50 smithi178 ceph-mon[65940]: pgmap v205: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:50 smithi144 ceph-mon[87436]: pgmap v205: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:53.000 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:52 smithi178 ceph-mon[65940]: pgmap v206: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:53.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:52 smithi144 ceph-mon[87436]: pgmap v206: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:54.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:54 smithi144 ceph-mon[87436]: pgmap v207: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:54 smithi178 ceph-mon[65940]: pgmap v207: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:56 smithi178 ceph-mon[65940]: pgmap v208: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:56 smithi144 ceph-mon[87436]: pgmap v208: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:58 smithi178 ceph-mon[65940]: pgmap v209: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:38:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:38:59.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:58 smithi144 ceph-mon[87436]: pgmap v209: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:38:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:38:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:39:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:00 smithi178 ceph-mon[65940]: pgmap v210: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:01.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:00 smithi144 ceph-mon[87436]: pgmap v210: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:01.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.324+0000 7f936dd15700 1 -- 172.21.15.144:0/1461447939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93680ff390 msgr2=0x7f93680ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:01.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.324+0000 7f936dd15700 1 --2- 172.21.15.144:0/1461447939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93680ff390 0x7f93680ff760 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f935800b0a0 tx=0x7f935800b3b0 comp rx=0 tx=0).stop 2024-09-10T11:39:01.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.324+0000 7f936dd15700 1 -- 172.21.15.144:0/1461447939 shutdown_connections 2024-09-10T11:39:01.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.324+0000 7f936dd15700 1 --2- 172.21.15.144:0/1461447939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93680ffd30 0x7f9368103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:01.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.324+0000 7f936dd15700 1 --2- 172.21.15.144:0/1461447939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93680ff390 0x7f93680ff760 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:01.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.324+0000 7f936dd15700 1 -- 172.21.15.144:0/1461447939 >> 172.21.15.144:0/1461447939 conn(0x7f9368074f50 msgr2=0x7f9368075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:01.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.324+0000 7f936dd15700 1 -- 172.21.15.144:0/1461447939 shutdown_connections 2024-09-10T11:39:01.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.324+0000 7f936dd15700 1 -- 172.21.15.144:0/1461447939 wait complete. 2024-09-10T11:39:01.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.325+0000 7f936dd15700 1 Processor -- start 2024-09-10T11:39:01.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.325+0000 7f936dd15700 1 -- start start 2024-09-10T11:39:01.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.325+0000 7f936dd15700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93680ff390 0x7f9368196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:01.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.325+0000 7f936dd15700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93680ffd30 0x7f9368197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:01.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.325+0000 7f936dd15700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9368197860 con 0x7f93680ff390 2024-09-10T11:39:01.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.325+0000 7f936dd15700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93681979d0 con 0x7f93680ffd30 2024-09-10T11:39:01.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.325+0000 7f93677fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93680ff390 0x7f9368196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:01.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.325+0000 7f93677fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93680ff390 0x7f9368196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33770/0 (socket says 172.21.15.144:33770) 2024-09-10T11:39:01.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.325+0000 7f93677fe700 1 -- 172.21.15.144:0/3285918442 learned_addr learned my addr 172.21.15.144:0/3285918442 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:39:01.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.326+0000 7f9366ffd700 1 --2- 172.21.15.144:0/3285918442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93680ffd30 0x7f9368197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:01.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.326+0000 7f93677fe700 1 -- 172.21.15.144:0/3285918442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93680ffd30 msgr2=0x7f9368197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:01.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.326+0000 7f93677fe700 1 --2- 172.21.15.144:0/3285918442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93680ffd30 0x7f9368197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:01.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.326+0000 7f93677fe700 1 -- 172.21.15.144:0/3285918442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9358009d00 con 0x7f93680ff390 2024-09-10T11:39:01.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.326+0000 7f93677fe700 1 --2- 172.21.15.144:0/3285918442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93680ff390 0x7f9368196c40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9358000f80 tx=0x7f935800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:01.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.326+0000 7f9364ff9700 1 -- 172.21.15.144:0/3285918442 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f935800e040 con 0x7f93680ff390 2024-09-10T11:39:01.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.326+0000 7f9364ff9700 1 -- 172.21.15.144:0/3285918442 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9358003c10 con 0x7f93680ff390 2024-09-10T11:39:01.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.326+0000 7f936dd15700 1 -- 172.21.15.144:0/3285918442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f936819fcb0 con 0x7f93680ff390 2024-09-10T11:39:01.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.326+0000 7f9364ff9700 1 -- 172.21.15.144:0/3285918442 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f935801d6b0 con 0x7f93680ff390 2024-09-10T11:39:01.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.326+0000 7f936dd15700 1 -- 172.21.15.144:0/3285918442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93681a0120 con 0x7f93680ff390 2024-09-10T11:39:01.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.327+0000 7f936dd15700 1 -- 172.21.15.144:0/3285918442 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9368107650 con 0x7f93680ff390 2024-09-10T11:39:01.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.328+0000 7f9364ff9700 1 -- 172.21.15.144:0/3285918442 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9358004040 con 0x7f93680ff390 2024-09-10T11:39:01.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.328+0000 7f9364ff9700 1 --2- 172.21.15.144:0/3285918442 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f935406e200 0x7f93540706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:01.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.329+0000 7f9366ffd700 1 --2- 172.21.15.144:0/3285918442 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f935406e200 0x7f93540706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:01.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.329+0000 7f9364ff9700 1 -- 172.21.15.144:0/3285918442 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f935808f7f0 con 0x7f93680ff390 2024-09-10T11:39:01.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.329+0000 7f9366ffd700 1 --2- 172.21.15.144:0/3285918442 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f935406e200 0x7f93540706b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9368068c80 tx=0x7f935000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:01.334 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.333+0000 7f9364ff9700 1 -- 172.21.15.144:0/3285918442 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9358058bd0 con 0x7f93680ff390 2024-09-10T11:39:01.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.552+0000 7f936dd15700 1 -- 172.21.15.144:0/3285918442 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f936802cca0 con 0x7f935406e200 2024-09-10T11:39:01.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.554+0000 7f9364ff9700 1 -- 172.21.15.144:0/3285918442 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f936802cca0 con 0x7f935406e200 2024-09-10T11:39:01.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.556+0000 7f936dd15700 1 -- 172.21.15.144:0/3285918442 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f935406e200 msgr2=0x7f93540706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:01.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.556+0000 7f936dd15700 1 --2- 172.21.15.144:0/3285918442 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f935406e200 0x7f93540706b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f9368068c80 tx=0x7f935000b410 comp rx=0 tx=0).stop 2024-09-10T11:39:01.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.556+0000 7f936dd15700 1 -- 172.21.15.144:0/3285918442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93680ff390 msgr2=0x7f9368196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:01.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.556+0000 7f936dd15700 1 --2- 172.21.15.144:0/3285918442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93680ff390 0x7f9368196c40 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f9358000f80 tx=0x7f935800cd20 comp rx=0 tx=0).stop 2024-09-10T11:39:01.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.556+0000 7f936dd15700 1 -- 172.21.15.144:0/3285918442 shutdown_connections 2024-09-10T11:39:01.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.556+0000 7f936dd15700 1 --2- 172.21.15.144:0/3285918442 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f935406e200 0x7f93540706b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:01.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.556+0000 7f936dd15700 1 --2- 172.21.15.144:0/3285918442 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93680ff390 0x7f9368196c40 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:01.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.556+0000 7f936dd15700 1 --2- 172.21.15.144:0/3285918442 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93680ffd30 0x7f9368197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:01.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.556+0000 7f936dd15700 1 -- 172.21.15.144:0/3285918442 >> 172.21.15.144:0/3285918442 conn(0x7f9368074f50 msgr2=0x7f93680fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:01.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.556+0000 7f936dd15700 1 -- 172.21.15.144:0/3285918442 shutdown_connections 2024-09-10T11:39:01.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.556+0000 7f936dd15700 1 -- 172.21.15.144:0/3285918442 wait complete. 2024-09-10T11:39:01.568 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:39:01.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.665+0000 7f0f4533a700 1 -- 172.21.15.144:0/945288548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f400ffcb0 msgr2=0x7f0f40103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:01.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.665+0000 7f0f4533a700 1 --2- 172.21.15.144:0/945288548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f400ffcb0 0x7f0f40103c80 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0f28009910 tx=0x7f0f28009c20 comp rx=0 tx=0).stop 2024-09-10T11:39:01.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.665+0000 7f0f4533a700 1 -- 172.21.15.144:0/945288548 shutdown_connections 2024-09-10T11:39:01.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.665+0000 7f0f4533a700 1 --2- 172.21.15.144:0/945288548 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f400ffcb0 0x7f0f40103c80 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:01.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.665+0000 7f0f4533a700 1 --2- 172.21.15.144:0/945288548 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f400ff310 0x7f0f400ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:01.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.665+0000 7f0f4533a700 1 -- 172.21.15.144:0/945288548 >> 172.21.15.144:0/945288548 conn(0x7f0f400fabb0 msgr2=0x7f0f400fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:01.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.665+0000 7f0f4533a700 1 -- 172.21.15.144:0/945288548 shutdown_connections 2024-09-10T11:39:01.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.665+0000 7f0f4533a700 1 -- 172.21.15.144:0/945288548 wait complete. 2024-09-10T11:39:01.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.666+0000 7f0f4533a700 1 Processor -- start 2024-09-10T11:39:01.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.666+0000 7f0f4533a700 1 -- start start 2024-09-10T11:39:01.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.666+0000 7f0f4533a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f400ff310 0x7f0f40196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:01.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.666+0000 7f0f4533a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f400ffcb0 0x7f0f401970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:01.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.666+0000 7f0f4533a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f40197700 con 0x7f0f400ffcb0 2024-09-10T11:39:01.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.666+0000 7f0f4533a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f40197870 con 0x7f0f400ff310 2024-09-10T11:39:01.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.666+0000 7f0f37fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f400ffcb0 0x7f0f401970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:01.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.666+0000 7f0f37fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f400ffcb0 0x7f0f401970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33786/0 (socket says 172.21.15.144:33786) 2024-09-10T11:39:01.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.666+0000 7f0f37fff700 1 -- 172.21.15.144:0/741275281 learned_addr learned my addr 172.21.15.144:0/741275281 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:39:01.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.667+0000 7f0f3effd700 1 --2- 172.21.15.144:0/741275281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f400ff310 0x7f0f40196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:01.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.667+0000 7f0f37fff700 1 -- 172.21.15.144:0/741275281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f400ff310 msgr2=0x7f0f40196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:01.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.667+0000 7f0f37fff700 1 --2- 172.21.15.144:0/741275281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f400ff310 0x7f0f40196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:01.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.667+0000 7f0f37fff700 1 -- 172.21.15.144:0/741275281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f280095f0 con 0x7f0f400ffcb0 2024-09-10T11:39:01.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.667+0000 7f0f37fff700 1 --2- 172.21.15.144:0/741275281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f400ffcb0 0x7f0f401970e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f2800bd70 tx=0x7f0f2800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:01.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.667+0000 7f0f3cff9700 1 -- 172.21.15.144:0/741275281 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f2801d070 con 0x7f0f400ffcb0 2024-09-10T11:39:01.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.667+0000 7f0f3cff9700 1 -- 172.21.15.144:0/741275281 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f28005dc0 con 0x7f0f400ffcb0 2024-09-10T11:39:01.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.667+0000 7f0f3cff9700 1 -- 172.21.15.144:0/741275281 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f28017400 con 0x7f0f400ffcb0 2024-09-10T11:39:01.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.667+0000 7f0f4533a700 1 -- 172.21.15.144:0/741275281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4019fc10 con 0x7f0f400ffcb0 2024-09-10T11:39:01.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.667+0000 7f0f4533a700 1 -- 172.21.15.144:0/741275281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f401a0080 con 0x7f0f400ffcb0 2024-09-10T11:39:01.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.668+0000 7f0f4533a700 1 -- 172.21.15.144:0/741275281 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f40107590 con 0x7f0f400ffcb0 2024-09-10T11:39:01.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.669+0000 7f0f3cff9700 1 -- 172.21.15.144:0/741275281 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0f2800f7f0 con 0x7f0f400ffcb0 2024-09-10T11:39:01.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.669+0000 7f0f3cff9700 1 --2- 172.21.15.144:0/741275281 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0f2c06e200 0x7f0f2c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:01.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.669+0000 7f0f3cff9700 1 -- 172.21.15.144:0/741275281 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0f2808f5b0 con 0x7f0f400ffcb0 2024-09-10T11:39:01.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.669+0000 7f0f3effd700 1 --2- 172.21.15.144:0/741275281 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0f2c06e200 0x7f0f2c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:01.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.670+0000 7f0f3effd700 1 --2- 172.21.15.144:0/741275281 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0f2c06e200 0x7f0f2c0706b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f30005950 tx=0x7f0f300058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:01.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.674+0000 7f0f3cff9700 1 -- 172.21.15.144:0/741275281 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0f28058990 con 0x7f0f400ffcb0 2024-09-10T11:39:01.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.898+0000 7f0f4533a700 1 -- 172.21.15.144:0/741275281 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f4002cca0 con 0x7f0f2c06e200 2024-09-10T11:39:01.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.899+0000 7f0f3cff9700 1 -- 172.21.15.144:0/741275281 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0f4002cca0 con 0x7f0f2c06e200 2024-09-10T11:39:01.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.901+0000 7f0f4533a700 1 -- 172.21.15.144:0/741275281 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0f2c06e200 msgr2=0x7f0f2c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:01.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.901+0000 7f0f4533a700 1 --2- 172.21.15.144:0/741275281 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0f2c06e200 0x7f0f2c0706b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f30005950 tx=0x7f0f300058e0 comp rx=0 tx=0).stop 2024-09-10T11:39:01.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.901+0000 7f0f4533a700 1 -- 172.21.15.144:0/741275281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f400ffcb0 msgr2=0x7f0f401970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:01.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.901+0000 7f0f4533a700 1 --2- 172.21.15.144:0/741275281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f400ffcb0 0x7f0f401970e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0f2800bd70 tx=0x7f0f2800be50 comp rx=0 tx=0).stop 2024-09-10T11:39:01.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.901+0000 7f0f4533a700 1 -- 172.21.15.144:0/741275281 shutdown_connections 2024-09-10T11:39:01.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.901+0000 7f0f4533a700 1 --2- 172.21.15.144:0/741275281 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0f2c06e200 0x7f0f2c0706b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:01.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.901+0000 7f0f4533a700 1 --2- 172.21.15.144:0/741275281 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0f400ff310 0x7f0f40196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:01.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.901+0000 7f0f4533a700 1 --2- 172.21.15.144:0/741275281 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0f400ffcb0 0x7f0f401970e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:01.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.901+0000 7f0f4533a700 1 -- 172.21.15.144:0/741275281 >> 172.21.15.144:0/741275281 conn(0x7f0f400fabb0 msgr2=0x7f0f401034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:01.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.901+0000 7f0f4533a700 1 -- 172.21.15.144:0/741275281 shutdown_connections 2024-09-10T11:39:01.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:01.901+0000 7f0f4533a700 1 -- 172.21.15.144:0/741275281 wait complete. 2024-09-10T11:39:02.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.005+0000 7fd2bf6ab700 1 -- 172.21.15.144:0/3601703260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b8100d70 msgr2=0x7fd2b8104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:02.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.005+0000 7fd2bf6ab700 1 --2- 172.21.15.144:0/3601703260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b8100d70 0x7fd2b8104d40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fd2b4009910 tx=0x7fd2b4009c20 comp rx=0 tx=0).stop 2024-09-10T11:39:02.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.005+0000 7fd2bf6ab700 1 -- 172.21.15.144:0/3601703260 shutdown_connections 2024-09-10T11:39:02.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.005+0000 7fd2bf6ab700 1 --2- 172.21.15.144:0/3601703260 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b8100d70 0x7fd2b8104d40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.005+0000 7fd2bf6ab700 1 --2- 172.21.15.144:0/3601703260 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2b81003d0 0x7fd2b81007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.005+0000 7fd2bf6ab700 1 -- 172.21.15.144:0/3601703260 >> 172.21.15.144:0/3601703260 conn(0x7fd2b80fbc90 msgr2=0x7fd2b80fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:02.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.005+0000 7fd2bf6ab700 1 -- 172.21.15.144:0/3601703260 shutdown_connections 2024-09-10T11:39:02.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.005+0000 7fd2bf6ab700 1 -- 172.21.15.144:0/3601703260 wait complete. 2024-09-10T11:39:02.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.006+0000 7fd2bf6ab700 1 Processor -- start 2024-09-10T11:39:02.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.006+0000 7fd2bf6ab700 1 -- start start 2024-09-10T11:39:02.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.006+0000 7fd2bf6ab700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2b81003d0 0x7fd2b8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:02.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.006+0000 7fd2bf6ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b8100d70 0x7fd2b8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:02.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.006+0000 7fd2bf6ab700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b8073ad0 con 0x7fd2b8100d70 2024-09-10T11:39:02.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.006+0000 7fd2bf6ab700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b8073c10 con 0x7fd2b81003d0 2024-09-10T11:39:02.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.006+0000 7fd2bcc46700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b8100d70 0x7fd2b8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:02.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.006+0000 7fd2bcc46700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b8100d70 0x7fd2b8073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33792/0 (socket says 172.21.15.144:33792) 2024-09-10T11:39:02.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.006+0000 7fd2bcc46700 1 -- 172.21.15.144:0/170151975 learned_addr learned my addr 172.21.15.144:0/170151975 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:39:02.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.006+0000 7fd2bd447700 1 --2- 172.21.15.144:0/170151975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2b81003d0 0x7fd2b8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:02.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.007+0000 7fd2bcc46700 1 -- 172.21.15.144:0/170151975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2b81003d0 msgr2=0x7fd2b8073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:02.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.007+0000 7fd2bcc46700 1 --2- 172.21.15.144:0/170151975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2b81003d0 0x7fd2b8073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.007+0000 7fd2bcc46700 1 -- 172.21.15.144:0/170151975 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b40095f0 con 0x7fd2b8100d70 2024-09-10T11:39:02.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.007+0000 7fd2bcc46700 1 --2- 172.21.15.144:0/170151975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b8100d70 0x7fd2b8073590 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40092e0 tx=0x7fd2b400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:02.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.007+0000 7fd2ae7fc700 1 -- 172.21.15.144:0/170151975 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2b401d070 con 0x7fd2b8100d70 2024-09-10T11:39:02.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.007+0000 7fd2ae7fc700 1 -- 172.21.15.144:0/170151975 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2b400f460 con 0x7fd2b8100d70 2024-09-10T11:39:02.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.007+0000 7fd2bf6ab700 1 -- 172.21.15.144:0/170151975 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b8073e90 con 0x7fd2b8100d70 2024-09-10T11:39:02.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.007+0000 7fd2ae7fc700 1 -- 172.21.15.144:0/170151975 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2b40175f0 con 0x7fd2b8100d70 2024-09-10T11:39:02.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.007+0000 7fd2bf6ab700 1 -- 172.21.15.144:0/170151975 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b81a2380 con 0x7fd2b8100d70 2024-09-10T11:39:02.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.008+0000 7fd2bf6ab700 1 -- 172.21.15.144:0/170151975 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b8108690 con 0x7fd2b8100d70 2024-09-10T11:39:02.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.009+0000 7fd2ae7fc700 1 -- 172.21.15.144:0/170151975 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd2b400f5d0 con 0x7fd2b8100d70 2024-09-10T11:39:02.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.009+0000 7fd2ae7fc700 1 --2- 172.21.15.144:0/170151975 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd2a406e200 0x7fd2a40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:02.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.009+0000 7fd2ae7fc700 1 -- 172.21.15.144:0/170151975 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd2b408f4f0 con 0x7fd2b8100d70 2024-09-10T11:39:02.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.009+0000 7fd2bd447700 1 --2- 172.21.15.144:0/170151975 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd2a406e200 0x7fd2a40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:02.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.010+0000 7fd2bd447700 1 --2- 172.21.15.144:0/170151975 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd2a406e200 0x7fd2a40706b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8006fd0 tx=0x7fd2a8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:02.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.013+0000 7fd2ae7fc700 1 -- 172.21.15.144:0/170151975 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd2b4058950 con 0x7fd2b8100d70 2024-09-10T11:39:02.228 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.226+0000 7fd2bf6ab700 1 -- 172.21.15.144:0/170151975 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2b8061190 con 0x7fd2a406e200 2024-09-10T11:39:02.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.236+0000 7fd2ae7fc700 1 -- 172.21.15.144:0/170151975 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fd2b8061190 con 0x7fd2a406e200 2024-09-10T11:39:02.237 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:39:02.237 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (7m) 58s ago 17m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:39:02.237 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (18m) 58s ago 18m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:39:02.237 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (17m) 44s ago 17m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:39:02.237 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (4m) 58s ago 17m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:39:02.238 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (4m) 44s ago 17m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:39:02.238 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (6m) 58s ago 17m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:39:02.238 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (80s) 58s ago 12m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:39:02.238 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (63s) 58s ago 12m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:39:02.238 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (12m) 44s ago 12m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:39:02.238 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (47s) 44s ago 12m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:39:02.238 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (8m) 58s ago 19m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:39:02.238 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (8m) 44s ago 16m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:39:02.238 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (5m) 58s ago 19m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:39:02.238 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (5m) 44s ago 16m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:39:02.238 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (7m) 58s ago 17m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:39:02.238 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (7m) 44s ago 16m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:39:02.238 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (4m) 58s ago 16m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:39:02.238 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (3m) 58s ago 15m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:39:02.239 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (3m) 58s ago 15m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:39:02.239 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (2m) 44s ago 14m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:39:02.239 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (2m) 44s ago 14m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:39:02.239 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (98s) 44s ago 13m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:39:02.239 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (7m) 58s ago 17m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:39:02.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.238+0000 7fd2bf6ab700 1 -- 172.21.15.144:0/170151975 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd2a406e200 msgr2=0x7fd2a40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:02.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.238+0000 7fd2bf6ab700 1 --2- 172.21.15.144:0/170151975 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd2a406e200 0x7fd2a40706b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8006fd0 tx=0x7fd2a8008040 comp rx=0 tx=0).stop 2024-09-10T11:39:02.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.238+0000 7fd2bf6ab700 1 -- 172.21.15.144:0/170151975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b8100d70 msgr2=0x7fd2b8073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:02.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.238+0000 7fd2bf6ab700 1 --2- 172.21.15.144:0/170151975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b8100d70 0x7fd2b8073590 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40092e0 tx=0x7fd2b400be20 comp rx=0 tx=0).stop 2024-09-10T11:39:02.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.238+0000 7fd2bf6ab700 1 -- 172.21.15.144:0/170151975 shutdown_connections 2024-09-10T11:39:02.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.238+0000 7fd2bf6ab700 1 --2- 172.21.15.144:0/170151975 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd2a406e200 0x7fd2a40706b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.238+0000 7fd2bf6ab700 1 --2- 172.21.15.144:0/170151975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2b81003d0 0x7fd2b8073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.238+0000 7fd2bf6ab700 1 --2- 172.21.15.144:0/170151975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2b8100d70 0x7fd2b8073590 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.238+0000 7fd2bf6ab700 1 -- 172.21.15.144:0/170151975 >> 172.21.15.144:0/170151975 conn(0x7fd2b80fbc90 msgr2=0x7fd2b80fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:02.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.238+0000 7fd2bf6ab700 1 -- 172.21.15.144:0/170151975 shutdown_connections 2024-09-10T11:39:02.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.238+0000 7fd2bf6ab700 1 -- 172.21.15.144:0/170151975 wait complete. 2024-09-10T11:39:02.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.343+0000 7f361d51f700 1 -- 172.21.15.144:0/4028624954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3618068fd0 msgr2=0x7f3618104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:02.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.343+0000 7f361d51f700 1 --2- 172.21.15.144:0/4028624954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3618068fd0 0x7f3618104df0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3600009910 tx=0x7f3600009c20 comp rx=0 tx=0).stop 2024-09-10T11:39:02.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.343+0000 7f361d51f700 1 -- 172.21.15.144:0/4028624954 shutdown_connections 2024-09-10T11:39:02.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.343+0000 7f361d51f700 1 --2- 172.21.15.144:0/4028624954 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3618068fd0 0x7f3618104df0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.343+0000 7f361d51f700 1 --2- 172.21.15.144:0/4028624954 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3618068630 0x7f3618068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.343+0000 7f361d51f700 1 -- 172.21.15.144:0/4028624954 >> 172.21.15.144:0/4028624954 conn(0x7f3618074f50 msgr2=0x7f3618075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:02.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.343+0000 7f361d51f700 1 -- 172.21.15.144:0/4028624954 shutdown_connections 2024-09-10T11:39:02.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.343+0000 7f361d51f700 1 -- 172.21.15.144:0/4028624954 wait complete. 2024-09-10T11:39:02.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.344+0000 7f361d51f700 1 Processor -- start 2024-09-10T11:39:02.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.344+0000 7f361d51f700 1 -- start start 2024-09-10T11:39:02.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.344+0000 7f361d51f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3618068630 0x7f36180ff200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:02.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.344+0000 7f361d51f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3618068fd0 0x7f36180ff740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:02.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.344+0000 7f361d51f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36180ffc80 con 0x7f3618068630 2024-09-10T11:39:02.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.344+0000 7f361d51f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36180ffdc0 con 0x7f3618068fd0 2024-09-10T11:39:02.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.344+0000 7f3616ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3618068630 0x7f36180ff200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:02.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.344+0000 7f3616ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3618068630 0x7f36180ff200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53474/0 (socket says 172.21.15.144:53474) 2024-09-10T11:39:02.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.344+0000 7f3616ffd700 1 -- 172.21.15.144:0/1692284781 learned_addr learned my addr 172.21.15.144:0/1692284781 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:39:02.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.344+0000 7f36167fc700 1 --2- 172.21.15.144:0/1692284781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3618068fd0 0x7f36180ff740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:02.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.345+0000 7f3616ffd700 1 -- 172.21.15.144:0/1692284781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3618068fd0 msgr2=0x7f36180ff740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:02.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.345+0000 7f3616ffd700 1 --2- 172.21.15.144:0/1692284781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3618068fd0 0x7f36180ff740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.345+0000 7f3616ffd700 1 -- 172.21.15.144:0/1692284781 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36000095f0 con 0x7f3618068630 2024-09-10T11:39:02.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.345+0000 7f3616ffd700 1 --2- 172.21.15.144:0/1692284781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3618068630 0x7f36180ff200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f36100060b0 tx=0x7f361000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:02.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.345+0000 7f360ffff700 1 -- 172.21.15.144:0/1692284781 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f361000dda0 con 0x7f3618068630 2024-09-10T11:39:02.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.345+0000 7f360ffff700 1 -- 172.21.15.144:0/1692284781 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f361000f040 con 0x7f3618068630 2024-09-10T11:39:02.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.345+0000 7f360ffff700 1 -- 172.21.15.144:0/1692284781 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36100146f0 con 0x7f3618068630 2024-09-10T11:39:02.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.345+0000 7f361d51f700 1 -- 172.21.15.144:0/1692284781 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36181a1ff0 con 0x7f3618068630 2024-09-10T11:39:02.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.345+0000 7f361d51f700 1 -- 172.21.15.144:0/1692284781 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36181a2400 con 0x7f3618068630 2024-09-10T11:39:02.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.346+0000 7f361d51f700 1 -- 172.21.15.144:0/1692284781 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36181086d0 con 0x7f3618068630 2024-09-10T11:39:02.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.347+0000 7f360ffff700 1 -- 172.21.15.144:0/1692284781 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3610015400 con 0x7f3618068630 2024-09-10T11:39:02.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.347+0000 7f360ffff700 1 --2- 172.21.15.144:0/1692284781 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f360406e200 0x7f36040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:02.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.347+0000 7f360ffff700 1 -- 172.21.15.144:0/1692284781 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f361008da20 con 0x7f3618068630 2024-09-10T11:39:02.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.348+0000 7f36167fc700 1 --2- 172.21.15.144:0/1692284781 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f360406e200 0x7f36040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:02.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.348+0000 7f36167fc700 1 --2- 172.21.15.144:0/1692284781 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f360406e200 0x7f36040706b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f36181009f0 tx=0x7f360001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:02.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.352+0000 7f360ffff700 1 -- 172.21.15.144:0/1692284781 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3610056eb0 con 0x7f3618068630 2024-09-10T11:39:02.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.640+0000 7f361d51f700 1 -- 172.21.15.144:0/1692284781 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f36181080b0 con 0x7f3618068630 2024-09-10T11:39:02.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.640+0000 7f360ffff700 1 -- 172.21.15.144:0/1692284781 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f361005a570 con 0x7f3618068630 2024-09-10T11:39:02.642 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:39:02.642 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:39:02.642 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:39:02.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:39:02.642 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:39:02.642 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:39:02.642 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:39:02.642 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:39:02.642 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:39:02.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:39:02.643 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:39:02.643 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:39:02.643 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:39:02.643 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:39:02.643 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:39:02.643 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:39:02.643 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:39:02.643 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:39:02.643 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:39:02.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.643+0000 7f361d51f700 1 -- 172.21.15.144:0/1692284781 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f360406e200 msgr2=0x7f36040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:02.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.643+0000 7f361d51f700 1 --2- 172.21.15.144:0/1692284781 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f360406e200 0x7f36040706b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f36181009f0 tx=0x7f360001a040 comp rx=0 tx=0).stop 2024-09-10T11:39:02.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.643+0000 7f361d51f700 1 -- 172.21.15.144:0/1692284781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3618068630 msgr2=0x7f36180ff200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:02.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.643+0000 7f361d51f700 1 --2- 172.21.15.144:0/1692284781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3618068630 0x7f36180ff200 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f36100060b0 tx=0x7f361000d3b0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.643+0000 7f361d51f700 1 -- 172.21.15.144:0/1692284781 shutdown_connections 2024-09-10T11:39:02.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.643+0000 7f361d51f700 1 --2- 172.21.15.144:0/1692284781 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f360406e200 0x7f36040706b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.643+0000 7f361d51f700 1 --2- 172.21.15.144:0/1692284781 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3618068630 0x7f36180ff200 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.643+0000 7f361d51f700 1 --2- 172.21.15.144:0/1692284781 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3618068fd0 0x7f36180ff740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.643+0000 7f361d51f700 1 -- 172.21.15.144:0/1692284781 >> 172.21.15.144:0/1692284781 conn(0x7f3618074f50 msgr2=0x7f36180fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:02.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.643+0000 7f361d51f700 1 -- 172.21.15.144:0/1692284781 shutdown_connections 2024-09-10T11:39:02.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.643+0000 7f361d51f700 1 -- 172.21.15.144:0/1692284781 wait complete. 2024-09-10T11:39:02.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.747+0000 7f5aa183d700 1 -- 172.21.15.144:0/3452993221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c1003e0 msgr2=0x7f5a9c1007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:02.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.747+0000 7f5aa183d700 1 --2- 172.21.15.144:0/3452993221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c1003e0 0x7f5a9c1007b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c009910 tx=0x7f5a8c009c20 comp rx=0 tx=0).stop 2024-09-10T11:39:02.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.747+0000 7f5aa183d700 1 -- 172.21.15.144:0/3452993221 shutdown_connections 2024-09-10T11:39:02.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.747+0000 7f5aa183d700 1 --2- 172.21.15.144:0/3452993221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a9c100d80 0x7f5a9c104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.747+0000 7f5aa183d700 1 --2- 172.21.15.144:0/3452993221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c1003e0 0x7f5a9c1007b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.747+0000 7f5aa183d700 1 -- 172.21.15.144:0/3452993221 >> 172.21.15.144:0/3452993221 conn(0x7f5a9c0fbca0 msgr2=0x7f5a9c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:02.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.747+0000 7f5aa183d700 1 -- 172.21.15.144:0/3452993221 shutdown_connections 2024-09-10T11:39:02.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.747+0000 7f5aa183d700 1 -- 172.21.15.144:0/3452993221 wait complete. 2024-09-10T11:39:02.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.748+0000 7f5aa183d700 1 Processor -- start 2024-09-10T11:39:02.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.748+0000 7f5aa183d700 1 -- start start 2024-09-10T11:39:02.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.748+0000 7f5aa183d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a9c1003e0 0x7f5a9c195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:02.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.748+0000 7f5aa183d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c100d80 0x7f5a9c1962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:02.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.748+0000 7f5aa183d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c1969a0 con 0x7f5a9c100d80 2024-09-10T11:39:02.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.748+0000 7f5aa183d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9c19a730 con 0x7f5a9c1003e0 2024-09-10T11:39:02.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.748+0000 7f5a93fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c100d80 0x7f5a9c1962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:02.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.748+0000 7f5a93fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c100d80 0x7f5a9c1962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53486/0 (socket says 172.21.15.144:53486) 2024-09-10T11:39:02.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.748+0000 7f5a93fff700 1 -- 172.21.15.144:0/51871628 learned_addr learned my addr 172.21.15.144:0/51871628 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:39:02.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.748+0000 7f5a9affd700 1 --2- 172.21.15.144:0/51871628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a9c1003e0 0x7f5a9c195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:02.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.748+0000 7f5a93fff700 1 -- 172.21.15.144:0/51871628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a9c1003e0 msgr2=0x7f5a9c195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:02.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.748+0000 7f5a93fff700 1 --2- 172.21.15.144:0/51871628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a9c1003e0 0x7f5a9c195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:02.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.748+0000 7f5a93fff700 1 -- 172.21.15.144:0/51871628 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a8c0095f0 con 0x7f5a9c100d80 2024-09-10T11:39:02.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.748+0000 7f5a93fff700 1 --2- 172.21.15.144:0/51871628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c100d80 0x7f5a9c1962c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400e8f0 tx=0x7f5a8400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:02.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.749+0000 7f5aa083b700 1 -- 172.21.15.144:0/51871628 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a84009790 con 0x7f5a9c100d80 2024-09-10T11:39:02.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.749+0000 7f5aa183d700 1 -- 172.21.15.144:0/51871628 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9c19aa10 con 0x7f5a9c100d80 2024-09-10T11:39:02.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.749+0000 7f5aa083b700 1 -- 172.21.15.144:0/51871628 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a84004cf0 con 0x7f5a9c100d80 2024-09-10T11:39:02.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.749+0000 7f5aa083b700 1 -- 172.21.15.144:0/51871628 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a84005690 con 0x7f5a9c100d80 2024-09-10T11:39:02.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.749+0000 7f5aa183d700 1 -- 172.21.15.144:0/51871628 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9c19af60 con 0x7f5a9c100d80 2024-09-10T11:39:02.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.750+0000 7f5aa183d700 1 -- 172.21.15.144:0/51871628 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9c1086f0 con 0x7f5a9c100d80 2024-09-10T11:39:02.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.750+0000 7f5aa083b700 1 -- 172.21.15.144:0/51871628 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5a84004750 con 0x7f5a9c100d80 2024-09-10T11:39:02.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.751+0000 7f5aa083b700 1 --2- 172.21.15.144:0/51871628 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a8806e200 0x7f5a880706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:02.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.751+0000 7f5aa083b700 1 -- 172.21.15.144:0/51871628 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5a84014070 con 0x7f5a9c100d80 2024-09-10T11:39:02.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.751+0000 7f5a9affd700 1 --2- 172.21.15.144:0/51871628 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a8806e200 0x7f5a880706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:02.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.751+0000 7f5a9affd700 1 --2- 172.21.15.144:0/51871628 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a8806e200 0x7f5a880706b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00b760 tx=0x7f5a8c009f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:02.756 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:02.755+0000 7f5aa083b700 1 -- 172.21.15.144:0/51871628 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5a840578b0 con 0x7f5a9c100d80 2024-09-10T11:39:02.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:02 smithi144 ceph-mon[87436]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:02.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:02 smithi144 ceph-mon[87436]: pgmap v211: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:02.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:02 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1692284781' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:39:03.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.000+0000 7f5aa183d700 1 -- 172.21.15.144:0/51871628 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5a9c04ea50 con 0x7f5a9c100d80 2024-09-10T11:39:03.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.000+0000 7f5aa083b700 1 -- 172.21.15.144:0/51871628 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f5a8405af70 con 0x7f5a9c100d80 2024-09-10T11:39:03.002 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:39:03.002 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:39:03.002 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:39:03.002 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:39:03.002 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:39:03.002 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:39:03.002 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:39:03.002 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:39:03.002 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:39:03.002 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:39:03.003 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:39:03.003 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:39:03.003 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:39:03.003 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:39:03.003 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:39:03.003 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:39:03.003 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:39:03.003 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:39:03.003 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:39:03.003 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:39:03.003 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:39:03.003 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:39:03.003 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:39:03.003 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:39:03.004 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:39:03.004 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:39:03.004 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:39:03.004 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:39:03.004 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:39:03.004 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:39:03.004 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:39:03.004 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:39:03.004 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:39:03.004 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:39:03.004 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:39:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:39:03.004 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:39:03.004 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:39:03.005 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:39:03.005 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:39:03.005 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:39:03.005 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:39:03.005 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:39:03.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.003+0000 7f5aa183d700 1 -- 172.21.15.144:0/51871628 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a8806e200 msgr2=0x7f5a880706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:03.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.003+0000 7f5aa183d700 1 --2- 172.21.15.144:0/51871628 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a8806e200 0x7f5a880706b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f5a8c00b760 tx=0x7f5a8c009f90 comp rx=0 tx=0).stop 2024-09-10T11:39:03.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.003+0000 7f5aa183d700 1 -- 172.21.15.144:0/51871628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c100d80 msgr2=0x7f5a9c1962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:03.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.003+0000 7f5aa183d700 1 --2- 172.21.15.144:0/51871628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c100d80 0x7f5a9c1962c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5a8400e8f0 tx=0x7f5a8400ecb0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.003+0000 7f5aa183d700 1 -- 172.21.15.144:0/51871628 shutdown_connections 2024-09-10T11:39:03.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.003+0000 7f5aa183d700 1 --2- 172.21.15.144:0/51871628 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a8806e200 0x7f5a880706b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.003+0000 7f5aa183d700 1 --2- 172.21.15.144:0/51871628 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a9c1003e0 0x7f5a9c195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.003+0000 7f5aa183d700 1 --2- 172.21.15.144:0/51871628 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a9c100d80 0x7f5a9c1962c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.003+0000 7f5aa183d700 1 -- 172.21.15.144:0/51871628 >> 172.21.15.144:0/51871628 conn(0x7f5a9c0fbca0 msgr2=0x7f5a9c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:03.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.003+0000 7f5aa183d700 1 -- 172.21.15.144:0/51871628 shutdown_connections 2024-09-10T11:39:03.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.003+0000 7f5aa183d700 1 -- 172.21.15.144:0/51871628 wait complete. 2024-09-10T11:39:03.007 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:39:03.102 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:02 smithi178 ceph-mon[65940]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:03.102 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:02 smithi178 ceph-mon[65940]: pgmap v211: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:03.103 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:02 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1692284781' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:39:03.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.110+0000 7fb06524f700 1 -- 172.21.15.144:0/2895779313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0600ffc30 msgr2=0x7fb060103c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:03.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.110+0000 7fb06524f700 1 --2- 172.21.15.144:0/2895779313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0600ffc30 0x7fb060103c00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fb048009910 tx=0x7fb048009c20 comp rx=0 tx=0).stop 2024-09-10T11:39:03.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.110+0000 7fb06524f700 1 -- 172.21.15.144:0/2895779313 shutdown_connections 2024-09-10T11:39:03.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.110+0000 7fb06524f700 1 --2- 172.21.15.144:0/2895779313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0600ffc30 0x7fb060103c00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.110+0000 7fb06524f700 1 --2- 172.21.15.144:0/2895779313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0600ff290 0x7fb0600ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.110+0000 7fb06524f700 1 -- 172.21.15.144:0/2895779313 >> 172.21.15.144:0/2895779313 conn(0x7fb0600fabb0 msgr2=0x7fb0600fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:03.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.110+0000 7fb06524f700 1 -- 172.21.15.144:0/2895779313 shutdown_connections 2024-09-10T11:39:03.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.110+0000 7fb06524f700 1 -- 172.21.15.144:0/2895779313 wait complete. 2024-09-10T11:39:03.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.111+0000 7fb06524f700 1 Processor -- start 2024-09-10T11:39:03.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.111+0000 7fb06524f700 1 -- start start 2024-09-10T11:39:03.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.111+0000 7fb06524f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0600ff290 0x7fb060100ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:03.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.111+0000 7fb06524f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0600ffc30 0x7fb0601011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:03.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.111+0000 7fb06524f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb060101720 con 0x7fb0600ff290 2024-09-10T11:39:03.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.111+0000 7fb06524f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb060101860 con 0x7fb0600ffc30 2024-09-10T11:39:03.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.111+0000 7fb05ed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0600ff290 0x7fb060100ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:03.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.111+0000 7fb05ed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0600ff290 0x7fb060100ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53506/0 (socket says 172.21.15.144:53506) 2024-09-10T11:39:03.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.111+0000 7fb05ed9d700 1 -- 172.21.15.144:0/3828223936 learned_addr learned my addr 172.21.15.144:0/3828223936 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:39:03.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.111+0000 7fb057fff700 1 --2- 172.21.15.144:0/3828223936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0600ffc30 0x7fb0601011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:03.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.111+0000 7fb05ed9d700 1 -- 172.21.15.144:0/3828223936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0600ffc30 msgr2=0x7fb0601011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:03.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.111+0000 7fb05ed9d700 1 --2- 172.21.15.144:0/3828223936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0600ffc30 0x7fb0601011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.111+0000 7fb05ed9d700 1 -- 172.21.15.144:0/3828223936 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0480095f0 con 0x7fb0600ff290 2024-09-10T11:39:03.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.111+0000 7fb057fff700 1 --2- 172.21.15.144:0/3828223936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0600ffc30 0x7fb0601011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:03.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.112+0000 7fb05ed9d700 1 --2- 172.21.15.144:0/3828223936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0600ff290 0x7fb060100ca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb05000e920 tx=0x7fb05000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:03.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.112+0000 7fb05cd99700 1 -- 172.21.15.144:0/3828223936 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb050009790 con 0x7fb0600ff290 2024-09-10T11:39:03.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.112+0000 7fb05cd99700 1 -- 172.21.15.144:0/3828223936 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb05000cba0 con 0x7fb0600ff290 2024-09-10T11:39:03.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.112+0000 7fb06524f700 1 -- 172.21.15.144:0/3828223936 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0601a1ec0 con 0x7fb0600ff290 2024-09-10T11:39:03.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.112+0000 7fb05cd99700 1 -- 172.21.15.144:0/3828223936 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb050018650 con 0x7fb0600ff290 2024-09-10T11:39:03.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.112+0000 7fb06524f700 1 -- 172.21.15.144:0/3828223936 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0601a22d0 con 0x7fb0600ff290 2024-09-10T11:39:03.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.113+0000 7fb06524f700 1 -- 172.21.15.144:0/3828223936 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb060108610 con 0x7fb0600ff290 2024-09-10T11:39:03.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.113+0000 7fb05cd99700 1 -- 172.21.15.144:0/3828223936 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb05000cd10 con 0x7fb0600ff290 2024-09-10T11:39:03.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.114+0000 7fb05cd99700 1 --2- 172.21.15.144:0/3828223936 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb04c06e200 0x7fb04c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:03.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.114+0000 7fb05cd99700 1 -- 172.21.15.144:0/3828223936 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb050014070 con 0x7fb0600ff290 2024-09-10T11:39:03.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.114+0000 7fb057fff700 1 --2- 172.21.15.144:0/3828223936 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb04c06e200 0x7fb04c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:03.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.115+0000 7fb057fff700 1 --2- 172.21.15.144:0/3828223936 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb04c06e200 0x7fb04c0706b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb048005b40 tx=0x7fb048005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:03.119 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.118+0000 7fb05cd99700 1 -- 172.21.15.144:0/3828223936 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb0500541b0 con 0x7fb0600ff290 2024-09-10T11:39:03.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.340+0000 7fb06524f700 1 -- 172.21.15.144:0/3828223936 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb060101a20 con 0x7fb04c06e200 2024-09-10T11:39:03.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.341+0000 7fb05cd99700 1 -- 172.21.15.144:0/3828223936 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb060101a20 con 0x7fb04c06e200 2024-09-10T11:39:03.343 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:39:03.343 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:39:03.343 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:39:03.343 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:39:03.343 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:39:03.343 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:39:03.343 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:39:03.343 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:39:03.343 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:39:03.343 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:39:03.343 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:39:03.343 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:39:03.343 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:39:03.344 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:39:03.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.344+0000 7fb06524f700 1 -- 172.21.15.144:0/3828223936 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb04c06e200 msgr2=0x7fb04c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:03.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.344+0000 7fb06524f700 1 --2- 172.21.15.144:0/3828223936 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb04c06e200 0x7fb04c0706b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb048005b40 tx=0x7fb048005a90 comp rx=0 tx=0).stop 2024-09-10T11:39:03.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.344+0000 7fb06524f700 1 -- 172.21.15.144:0/3828223936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0600ff290 msgr2=0x7fb060100ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:03.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.344+0000 7fb06524f700 1 --2- 172.21.15.144:0/3828223936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0600ff290 0x7fb060100ca0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb05000e920 tx=0x7fb05000ece0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.344+0000 7fb06524f700 1 -- 172.21.15.144:0/3828223936 shutdown_connections 2024-09-10T11:39:03.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.344+0000 7fb06524f700 1 --2- 172.21.15.144:0/3828223936 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb04c06e200 0x7fb04c0706b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.344+0000 7fb06524f700 1 --2- 172.21.15.144:0/3828223936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0600ff290 0x7fb060100ca0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.344+0000 7fb06524f700 1 --2- 172.21.15.144:0/3828223936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0600ffc30 0x7fb0601011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.344+0000 7fb06524f700 1 -- 172.21.15.144:0/3828223936 >> 172.21.15.144:0/3828223936 conn(0x7fb0600fabb0 msgr2=0x7fb060068ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:03.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.344+0000 7fb06524f700 1 -- 172.21.15.144:0/3828223936 shutdown_connections 2024-09-10T11:39:03.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.344+0000 7fb06524f700 1 -- 172.21.15.144:0/3828223936 wait complete. 2024-09-10T11:39:03.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.449+0000 7f769a521700 1 -- 172.21.15.144:0/2441266126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76941003b0 msgr2=0x7f7694100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:03.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.449+0000 7f769a521700 1 --2- 172.21.15.144:0/2441266126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76941003b0 0x7f7694100780 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7684009910 tx=0x7f7684009c20 comp rx=0 tx=0).stop 2024-09-10T11:39:03.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.449+0000 7f769a521700 1 -- 172.21.15.144:0/2441266126 shutdown_connections 2024-09-10T11:39:03.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.449+0000 7f769a521700 1 --2- 172.21.15.144:0/2441266126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7694100d50 0x7f7694104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.449+0000 7f769a521700 1 --2- 172.21.15.144:0/2441266126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f76941003b0 0x7f7694100780 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.449+0000 7f769a521700 1 -- 172.21.15.144:0/2441266126 >> 172.21.15.144:0/2441266126 conn(0x7f76940fbc30 msgr2=0x7f76940fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:03.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.449+0000 7f769a521700 1 -- 172.21.15.144:0/2441266126 shutdown_connections 2024-09-10T11:39:03.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.449+0000 7f769a521700 1 -- 172.21.15.144:0/2441266126 wait complete. 2024-09-10T11:39:03.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.449+0000 7f769a521700 1 Processor -- start 2024-09-10T11:39:03.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.450+0000 7f769a521700 1 -- start start 2024-09-10T11:39:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.450+0000 7f769a521700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76941003b0 0x7f7694195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.450+0000 7f769a521700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7694100d50 0x7f7694196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.450+0000 7f769a521700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7694196a40 con 0x7f7694100d50 2024-09-10T11:39:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.450+0000 7f769a521700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769419a760 con 0x7f76941003b0 2024-09-10T11:39:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.450+0000 7f76937fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7694100d50 0x7f7694196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.450+0000 7f76937fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7694100d50 0x7f7694196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53522/0 (socket says 172.21.15.144:53522) 2024-09-10T11:39:03.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.450+0000 7f76937fe700 1 -- 172.21.15.144:0/597402668 learned_addr learned my addr 172.21.15.144:0/597402668 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:39:03.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.450+0000 7f76937fe700 1 -- 172.21.15.144:0/597402668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76941003b0 msgr2=0x7f7694195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:03.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.450+0000 7f7693fff700 1 --2- 172.21.15.144:0/597402668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76941003b0 0x7f7694195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:03.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.450+0000 7f76937fe700 1 --2- 172.21.15.144:0/597402668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76941003b0 0x7f7694195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.450+0000 7f76937fe700 1 -- 172.21.15.144:0/597402668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76840095f0 con 0x7f7694100d50 2024-09-10T11:39:03.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.451+0000 7f76937fe700 1 --2- 172.21.15.144:0/597402668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7694100d50 0x7f7694196360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f768800b480 tx=0x7f768800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:03.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.451+0000 7f76917fa700 1 -- 172.21.15.144:0/597402668 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76880118c0 con 0x7f7694100d50 2024-09-10T11:39:03.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.451+0000 7f76917fa700 1 -- 172.21.15.144:0/597402668 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7688010380 con 0x7f7694100d50 2024-09-10T11:39:03.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.451+0000 7f769a521700 1 -- 172.21.15.144:0/597402668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769419aa40 con 0x7f7694100d50 2024-09-10T11:39:03.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.451+0000 7f76917fa700 1 -- 172.21.15.144:0/597402668 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f768800f4f0 con 0x7f7694100d50 2024-09-10T11:39:03.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.451+0000 7f769a521700 1 -- 172.21.15.144:0/597402668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769419af90 con 0x7f7694100d50 2024-09-10T11:39:03.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.452+0000 7f769a521700 1 -- 172.21.15.144:0/597402668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76941086c0 con 0x7f7694100d50 2024-09-10T11:39:03.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.452+0000 7f76917fa700 1 -- 172.21.15.144:0/597402668 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f76880104f0 con 0x7f7694100d50 2024-09-10T11:39:03.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.453+0000 7f76917fa700 1 --2- 172.21.15.144:0/597402668 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f767c06e200 0x7f767c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:03.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.453+0000 7f76917fa700 1 -- 172.21.15.144:0/597402668 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f768808dc20 con 0x7f7694100d50 2024-09-10T11:39:03.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.453+0000 7f7693fff700 1 --2- 172.21.15.144:0/597402668 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f767c06e200 0x7f767c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:03.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.453+0000 7f7693fff700 1 --2- 172.21.15.144:0/597402668 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f767c06e200 0x7f767c0706b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f768400b5c0 tx=0x7f7684005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:03.458 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.457+0000 7f76917fa700 1 -- 172.21.15.144:0/597402668 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7688057080 con 0x7f7694100d50 2024-09-10T11:39:03.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.744+0000 7f769a521700 1 -- 172.21.15.144:0/597402668 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f769404ea50 con 0x7f7694100d50 2024-09-10T11:39:03.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.746+0000 7f76917fa700 1 -- 172.21.15.144:0/597402668 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f76880151c0 con 0x7f7694100d50 2024-09-10T11:39:03.748 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:39:03.748 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:39:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:39:03.748 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:39:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:39:03.748 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:39:03.748 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:39:03.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.748+0000 7f769a521700 1 -- 172.21.15.144:0/597402668 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f767c06e200 msgr2=0x7f767c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:03.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.748+0000 7f769a521700 1 --2- 172.21.15.144:0/597402668 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f767c06e200 0x7f767c0706b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f768400b5c0 tx=0x7f7684005960 comp rx=0 tx=0).stop 2024-09-10T11:39:03.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.749+0000 7f769a521700 1 -- 172.21.15.144:0/597402668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7694100d50 msgr2=0x7f7694196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:03.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.749+0000 7f769a521700 1 --2- 172.21.15.144:0/597402668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7694100d50 0x7f7694196360 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f768800b480 tx=0x7f768800b840 comp rx=0 tx=0).stop 2024-09-10T11:39:03.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.749+0000 7f769a521700 1 -- 172.21.15.144:0/597402668 shutdown_connections 2024-09-10T11:39:03.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.749+0000 7f769a521700 1 --2- 172.21.15.144:0/597402668 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f767c06e200 0x7f767c0706b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.749+0000 7f769a521700 1 --2- 172.21.15.144:0/597402668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f76941003b0 0x7f7694195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.749+0000 7f769a521700 1 --2- 172.21.15.144:0/597402668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7694100d50 0x7f7694196360 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:03.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.749+0000 7f769a521700 1 -- 172.21.15.144:0/597402668 >> 172.21.15.144:0/597402668 conn(0x7f76940fbc30 msgr2=0x7f76940fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:03.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.749+0000 7f769a521700 1 -- 172.21.15.144:0/597402668 shutdown_connections 2024-09-10T11:39:03.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:03.749+0000 7f769a521700 1 -- 172.21.15.144:0/597402668 wait complete. 2024-09-10T11:39:04.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:03 smithi178 ceph-mon[65940]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:04.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:03 smithi178 ceph-mon[65940]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:04.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:03 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/51871628' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:39:04.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:03 smithi144 ceph-mon[87436]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:04.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:03 smithi144 ceph-mon[87436]: from='client.34464 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:04.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:03 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/51871628' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:39:04.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:04 smithi144 ceph-mon[87436]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:04.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:04 smithi144 ceph-mon[87436]: pgmap v212: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:04.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:04 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/597402668' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:39:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:04 smithi178 ceph-mon[65940]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:04 smithi178 ceph-mon[65940]: pgmap v212: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:04 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/597402668' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:39:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:06 smithi178 ceph-mon[65940]: pgmap v213: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:06 smithi144 ceph-mon[87436]: pgmap v213: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:09.103 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:08 smithi178 ceph-mon[65940]: pgmap v214: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:08 smithi144 ceph-mon[87436]: pgmap v214: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:10 smithi178 ceph-mon[65940]: pgmap v215: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:11.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:10 smithi144 ceph-mon[87436]: pgmap v215: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:12 smithi144 ceph-mon[87436]: pgmap v216: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:13.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:12 smithi178 ceph-mon[65940]: pgmap v216: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:39:14.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:39:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:14 smithi178 ceph-mon[65940]: pgmap v217: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:14 smithi144 ceph-mon[87436]: pgmap v217: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:16.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:15 smithi178 ceph-mon[65940]: pgmap v218: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:16.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:15 smithi144 ceph-mon[87436]: pgmap v218: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:18 smithi178 ceph-mon[65940]: pgmap v219: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:18 smithi144 ceph-mon[87436]: pgmap v219: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:20 smithi178 ceph-mon[65940]: pgmap v220: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:21.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:20 smithi144 ceph-mon[87436]: pgmap v220: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:23.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:22 smithi144 ceph-mon[87436]: pgmap v221: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:23.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:22 smithi178 ceph-mon[65940]: pgmap v221: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:24 smithi178 ceph-mon[65940]: pgmap v222: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:24 smithi144 ceph-mon[87436]: pgmap v222: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:26 smithi178 ceph-mon[65940]: pgmap v223: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:27.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:26 smithi144 ceph-mon[87436]: pgmap v223: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:28 smithi178 ceph-mon[65940]: pgmap v224: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:39:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:28 smithi144 ceph-mon[87436]: pgmap v224: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:39:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:30 smithi178 ceph-mon[65940]: pgmap v225: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:31.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:30 smithi144 ceph-mon[87436]: pgmap v225: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:32 smithi144 ceph-mon[87436]: pgmap v226: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:33.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:32 smithi178 ceph-mon[65940]: pgmap v226: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:33.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.868+0000 7fba13076700 1 -- 172.21.15.144:0/3658215058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba0c1003d0 msgr2=0x7fba0c1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:33.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.868+0000 7fba13076700 1 --2- 172.21.15.144:0/3658215058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba0c1003d0 0x7fba0c1007a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fba00009910 tx=0x7fba00009c20 comp rx=0 tx=0).stop 2024-09-10T11:39:33.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.868+0000 7fba13076700 1 -- 172.21.15.144:0/3658215058 shutdown_connections 2024-09-10T11:39:33.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.868+0000 7fba13076700 1 --2- 172.21.15.144:0/3658215058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba0c100d70 0x7fba0c104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.868+0000 7fba13076700 1 --2- 172.21.15.144:0/3658215058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba0c1003d0 0x7fba0c1007a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.868+0000 7fba13076700 1 -- 172.21.15.144:0/3658215058 >> 172.21.15.144:0/3658215058 conn(0x7fba0c0fbc90 msgr2=0x7fba0c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:33.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.868+0000 7fba13076700 1 -- 172.21.15.144:0/3658215058 shutdown_connections 2024-09-10T11:39:33.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.868+0000 7fba13076700 1 -- 172.21.15.144:0/3658215058 wait complete. 2024-09-10T11:39:33.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.868+0000 7fba13076700 1 Processor -- start 2024-09-10T11:39:33.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.868+0000 7fba13076700 1 -- start start 2024-09-10T11:39:33.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.869+0000 7fba13076700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba0c1003d0 0x7fba0c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:33.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.869+0000 7fba13076700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba0c100d70 0x7fba0c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:33.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.869+0000 7fba13076700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c073ad0 con 0x7fba0c100d70 2024-09-10T11:39:33.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.869+0000 7fba13076700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c073c10 con 0x7fba0c1003d0 2024-09-10T11:39:33.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.869+0000 7fba0bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba0c100d70 0x7fba0c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:33.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.869+0000 7fba0bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba0c100d70 0x7fba0c073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56492/0 (socket says 172.21.15.144:56492) 2024-09-10T11:39:33.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.869+0000 7fba0bfff700 1 -- 172.21.15.144:0/2678242009 learned_addr learned my addr 172.21.15.144:0/2678242009 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:39:33.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.869+0000 7fba10e12700 1 --2- 172.21.15.144:0/2678242009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba0c1003d0 0x7fba0c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:33.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.869+0000 7fba0bfff700 1 -- 172.21.15.144:0/2678242009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba0c1003d0 msgr2=0x7fba0c073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:33.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.869+0000 7fba0bfff700 1 --2- 172.21.15.144:0/2678242009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba0c1003d0 0x7fba0c073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:33.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.869+0000 7fba0bfff700 1 -- 172.21.15.144:0/2678242009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba000095f0 con 0x7fba0c100d70 2024-09-10T11:39:33.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.869+0000 7fba0bfff700 1 --2- 172.21.15.144:0/2678242009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba0c100d70 0x7fba0c073590 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800c8f0 tx=0x7fb9f800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:33.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.869+0000 7fba09ffb700 1 -- 172.21.15.144:0/2678242009 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9f8007a50 con 0x7fba0c100d70 2024-09-10T11:39:33.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.870+0000 7fba09ffb700 1 -- 172.21.15.144:0/2678242009 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9f8007bb0 con 0x7fba0c100d70 2024-09-10T11:39:33.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.870+0000 7fba13076700 1 -- 172.21.15.144:0/2678242009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0c1a1f30 con 0x7fba0c100d70 2024-09-10T11:39:33.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.870+0000 7fba09ffb700 1 -- 172.21.15.144:0/2678242009 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9f8018630 con 0x7fba0c100d70 2024-09-10T11:39:33.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.870+0000 7fba13076700 1 -- 172.21.15.144:0/2678242009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0c1a2340 con 0x7fba0c100d70 2024-09-10T11:39:33.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.871+0000 7fba13076700 1 -- 172.21.15.144:0/2678242009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba0c1086e0 con 0x7fba0c100d70 2024-09-10T11:39:33.874 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.871+0000 7fba09ffb700 1 -- 172.21.15.144:0/2678242009 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb9f8007d20 con 0x7fba0c100d70 2024-09-10T11:39:33.874 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.872+0000 7fba09ffb700 1 --2- 172.21.15.144:0/2678242009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb9fc06e200 0x7fb9fc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:33.874 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.872+0000 7fba09ffb700 1 -- 172.21.15.144:0/2678242009 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb9f808dfa0 con 0x7fba0c100d70 2024-09-10T11:39:33.874 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.872+0000 7fba10e12700 1 --2- 172.21.15.144:0/2678242009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb9fc06e200 0x7fb9fc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:33.874 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.872+0000 7fba10e12700 1 --2- 172.21.15.144:0/2678242009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb9fc06e200 0x7fb9fc0706b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fba0000b5c0 tx=0x7fba0001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:33.877 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:33.876+0000 7fba09ffb700 1 -- 172.21.15.144:0/2678242009 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb9f8057400 con 0x7fba0c100d70 2024-09-10T11:39:34.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.098+0000 7fba13076700 1 -- 172.21.15.144:0/2678242009 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fba0c02cd40 con 0x7fb9fc06e200 2024-09-10T11:39:34.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.099+0000 7fba09ffb700 1 -- 172.21.15.144:0/2678242009 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fba0c02cd40 con 0x7fb9fc06e200 2024-09-10T11:39:34.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.101+0000 7fba13076700 1 -- 172.21.15.144:0/2678242009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb9fc06e200 msgr2=0x7fb9fc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.101+0000 7fba13076700 1 --2- 172.21.15.144:0/2678242009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb9fc06e200 0x7fb9fc0706b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fba0000b5c0 tx=0x7fba0001a040 comp rx=0 tx=0).stop 2024-09-10T11:39:34.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.101+0000 7fba13076700 1 -- 172.21.15.144:0/2678242009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba0c100d70 msgr2=0x7fba0c073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.101+0000 7fba13076700 1 --2- 172.21.15.144:0/2678242009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba0c100d70 0x7fba0c073590 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800c8f0 tx=0x7fb9f800ccb0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.101+0000 7fba13076700 1 -- 172.21.15.144:0/2678242009 shutdown_connections 2024-09-10T11:39:34.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.101+0000 7fba13076700 1 --2- 172.21.15.144:0/2678242009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb9fc06e200 0x7fb9fc0706b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.101+0000 7fba13076700 1 --2- 172.21.15.144:0/2678242009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fba0c1003d0 0x7fba0c073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.101+0000 7fba13076700 1 --2- 172.21.15.144:0/2678242009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fba0c100d70 0x7fba0c073590 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.101+0000 7fba13076700 1 -- 172.21.15.144:0/2678242009 >> 172.21.15.144:0/2678242009 conn(0x7fba0c0fbc90 msgr2=0x7fba0c0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:34.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.101+0000 7fba13076700 1 -- 172.21.15.144:0/2678242009 shutdown_connections 2024-09-10T11:39:34.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.101+0000 7fba13076700 1 -- 172.21.15.144:0/2678242009 wait complete. 2024-09-10T11:39:34.113 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:39:34.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.208+0000 7fb7f5256700 1 -- 172.21.15.144:0/548240291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f00ff390 msgr2=0x7fb7f00ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.208+0000 7fb7f5256700 1 --2- 172.21.15.144:0/548240291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f00ff390 0x7fb7f00ff760 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8009910 tx=0x7fb7d8009c20 comp rx=0 tx=0).stop 2024-09-10T11:39:34.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.208+0000 7fb7f5256700 1 -- 172.21.15.144:0/548240291 shutdown_connections 2024-09-10T11:39:34.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.208+0000 7fb7f5256700 1 --2- 172.21.15.144:0/548240291 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7f00ffd30 0x7fb7f0103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.208+0000 7fb7f5256700 1 --2- 172.21.15.144:0/548240291 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f00ff390 0x7fb7f00ff760 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.208+0000 7fb7f5256700 1 -- 172.21.15.144:0/548240291 >> 172.21.15.144:0/548240291 conn(0x7fb7f0074f50 msgr2=0x7fb7f0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:34.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.208+0000 7fb7f5256700 1 -- 172.21.15.144:0/548240291 shutdown_connections 2024-09-10T11:39:34.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.208+0000 7fb7f5256700 1 -- 172.21.15.144:0/548240291 wait complete. 2024-09-10T11:39:34.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.209+0000 7fb7f5256700 1 Processor -- start 2024-09-10T11:39:34.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.209+0000 7fb7f5256700 1 -- start start 2024-09-10T11:39:34.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.209+0000 7fb7f5256700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f00ff390 0x7fb7f0195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.209+0000 7fb7f5256700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7f00ffd30 0x7fb7f0196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.209+0000 7fb7f5256700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f01969e0 con 0x7fb7f00ff390 2024-09-10T11:39:34.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.209+0000 7fb7f5256700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7f019a770 con 0x7fb7f00ffd30 2024-09-10T11:39:34.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.209+0000 7fb7eed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f00ff390 0x7fb7f0195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.209+0000 7fb7eed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f00ff390 0x7fb7f0195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56512/0 (socket says 172.21.15.144:56512) 2024-09-10T11:39:34.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.209+0000 7fb7eed9d700 1 -- 172.21.15.144:0/1310561072 learned_addr learned my addr 172.21.15.144:0/1310561072 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:39:34.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.210+0000 7fb7eed9d700 1 -- 172.21.15.144:0/1310561072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7f00ffd30 msgr2=0x7fb7f0196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.210+0000 7fb7ee59c700 1 --2- 172.21.15.144:0/1310561072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7f00ffd30 0x7fb7f0196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.210+0000 7fb7eed9d700 1 --2- 172.21.15.144:0/1310561072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7f00ffd30 0x7fb7f0196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.210+0000 7fb7eed9d700 1 -- 172.21.15.144:0/1310561072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d80095f0 con 0x7fb7f00ff390 2024-09-10T11:39:34.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.210+0000 7fb7ee59c700 1 --2- 172.21.15.144:0/1310561072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7f00ffd30 0x7fb7f0196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:39:34.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.210+0000 7fb7eed9d700 1 --2- 172.21.15.144:0/1310561072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f00ff390 0x7fb7f0195dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8006010 tx=0x7fb7d800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:34.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.210+0000 7fb7e7fff700 1 -- 172.21.15.144:0/1310561072 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7d801d070 con 0x7fb7f00ff390 2024-09-10T11:39:34.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.210+0000 7fb7e7fff700 1 -- 172.21.15.144:0/1310561072 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7d8004fb0 con 0x7fb7f00ff390 2024-09-10T11:39:34.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.210+0000 7fb7f5256700 1 -- 172.21.15.144:0/1310561072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7f019a9f0 con 0x7fb7f00ff390 2024-09-10T11:39:34.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.210+0000 7fb7e7fff700 1 -- 172.21.15.144:0/1310561072 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7d800f690 con 0x7fb7f00ff390 2024-09-10T11:39:34.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.210+0000 7fb7f5256700 1 -- 172.21.15.144:0/1310561072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7f019aee0 con 0x7fb7f00ff390 2024-09-10T11:39:34.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.211+0000 7fb7f5256700 1 -- 172.21.15.144:0/1310561072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7f004ea50 con 0x7fb7f00ff390 2024-09-10T11:39:34.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.211+0000 7fb7e7fff700 1 -- 172.21.15.144:0/1310561072 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb7d800f7f0 con 0x7fb7f00ff390 2024-09-10T11:39:34.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.212+0000 7fb7e7fff700 1 --2- 172.21.15.144:0/1310561072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb7dc06e200 0x7fb7dc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.212+0000 7fb7e7fff700 1 -- 172.21.15.144:0/1310561072 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb7d808f870 con 0x7fb7f00ff390 2024-09-10T11:39:34.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.212+0000 7fb7ee59c700 1 --2- 172.21.15.144:0/1310561072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb7dc06e200 0x7fb7dc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.212+0000 7fb7ee59c700 1 --2- 172.21.15.144:0/1310561072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb7dc06e200 0x7fb7dc0706b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0005fd0 tx=0x7fb7e0005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:34.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.216+0000 7fb7e7fff700 1 -- 172.21.15.144:0/1310561072 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb7d8058e20 con 0x7fb7f00ff390 2024-09-10T11:39:34.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.438+0000 7fb7f5256700 1 -- 172.21.15.144:0/1310561072 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7f019b320 con 0x7fb7dc06e200 2024-09-10T11:39:34.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.439+0000 7fb7e7fff700 1 -- 172.21.15.144:0/1310561072 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb7f019b320 con 0x7fb7dc06e200 2024-09-10T11:39:34.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.441+0000 7fb7f5256700 1 -- 172.21.15.144:0/1310561072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb7dc06e200 msgr2=0x7fb7dc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.441+0000 7fb7f5256700 1 --2- 172.21.15.144:0/1310561072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb7dc06e200 0x7fb7dc0706b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fb7e0005fd0 tx=0x7fb7e0005dc0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.441+0000 7fb7f5256700 1 -- 172.21.15.144:0/1310561072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f00ff390 msgr2=0x7fb7f0195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.441+0000 7fb7f5256700 1 --2- 172.21.15.144:0/1310561072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f00ff390 0x7fb7f0195dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8006010 tx=0x7fb7d800be20 comp rx=0 tx=0).stop 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.441+0000 7fb7f5256700 1 -- 172.21.15.144:0/1310561072 shutdown_connections 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.441+0000 7fb7f5256700 1 --2- 172.21.15.144:0/1310561072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb7dc06e200 0x7fb7dc0706b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.441+0000 7fb7f5256700 1 --2- 172.21.15.144:0/1310561072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7f00ff390 0x7fb7f0195dc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.441+0000 7fb7f5256700 1 --2- 172.21.15.144:0/1310561072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7f00ffd30 0x7fb7f0196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.441+0000 7fb7f5256700 1 -- 172.21.15.144:0/1310561072 >> 172.21.15.144:0/1310561072 conn(0x7fb7f0074f50 msgr2=0x7fb7f00fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.441+0000 7fb7f5256700 1 -- 172.21.15.144:0/1310561072 shutdown_connections 2024-09-10T11:39:34.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.441+0000 7fb7f5256700 1 -- 172.21.15.144:0/1310561072 wait complete. 2024-09-10T11:39:34.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.546+0000 7f1a07820700 1 -- 172.21.15.144:0/2924025701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a000ff2c0 msgr2=0x7f1a000ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.546+0000 7f1a07820700 1 --2- 172.21.15.144:0/2924025701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a000ff2c0 0x7f1a000ff690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f19f0009910 tx=0x7f19f0009c20 comp rx=0 tx=0).stop 2024-09-10T11:39:34.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.546+0000 7f1a07820700 1 -- 172.21.15.144:0/2924025701 shutdown_connections 2024-09-10T11:39:34.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.546+0000 7f1a07820700 1 --2- 172.21.15.144:0/2924025701 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a000ffc60 0x7f1a00103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.546+0000 7f1a07820700 1 --2- 172.21.15.144:0/2924025701 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a000ff2c0 0x7f1a000ff690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.546+0000 7f1a07820700 1 -- 172.21.15.144:0/2924025701 >> 172.21.15.144:0/2924025701 conn(0x7f1a000fab80 msgr2=0x7f1a000fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:34.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.546+0000 7f1a07820700 1 -- 172.21.15.144:0/2924025701 shutdown_connections 2024-09-10T11:39:34.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.546+0000 7f1a07820700 1 -- 172.21.15.144:0/2924025701 wait complete. 2024-09-10T11:39:34.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.547+0000 7f1a07820700 1 Processor -- start 2024-09-10T11:39:34.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.547+0000 7f1a07820700 1 -- start start 2024-09-10T11:39:34.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.547+0000 7f1a07820700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a000ff2c0 0x7f1a00196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.547+0000 7f1a07820700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a000ffc60 0x7f1a001970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.547+0000 7f1a07820700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a001976f0 con 0x7f1a000ffc60 2024-09-10T11:39:34.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.547+0000 7f1a07820700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a00197830 con 0x7f1a000ff2c0 2024-09-10T11:39:34.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.547+0000 7f1a04dbb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a000ffc60 0x7f1a001970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.547+0000 7f1a04dbb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a000ffc60 0x7f1a001970d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56522/0 (socket says 172.21.15.144:56522) 2024-09-10T11:39:34.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.547+0000 7f1a04dbb700 1 -- 172.21.15.144:0/713369120 learned_addr learned my addr 172.21.15.144:0/713369120 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:39:34.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.547+0000 7f1a055bc700 1 --2- 172.21.15.144:0/713369120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a000ff2c0 0x7f1a00196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.547+0000 7f1a04dbb700 1 -- 172.21.15.144:0/713369120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a000ff2c0 msgr2=0x7f1a00196b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.547+0000 7f1a04dbb700 1 --2- 172.21.15.144:0/713369120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a000ff2c0 0x7f1a00196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.547+0000 7f1a04dbb700 1 -- 172.21.15.144:0/713369120 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f00095f0 con 0x7f1a000ffc60 2024-09-10T11:39:34.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.548+0000 7f1a04dbb700 1 --2- 172.21.15.144:0/713369120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a000ffc60 0x7f1a001970d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00c8f0 tx=0x7f19fc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:34.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.548+0000 7f19f67fc700 1 -- 172.21.15.144:0/713369120 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19fc007a50 con 0x7f1a000ffc60 2024-09-10T11:39:34.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.548+0000 7f19f67fc700 1 -- 172.21.15.144:0/713369120 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19fc007bb0 con 0x7f1a000ffc60 2024-09-10T11:39:34.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.548+0000 7f19f67fc700 1 -- 172.21.15.144:0/713369120 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19fc018630 con 0x7f1a000ffc60 2024-09-10T11:39:34.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.548+0000 7f1a07820700 1 -- 172.21.15.144:0/713369120 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0019fc00 con 0x7f1a000ffc60 2024-09-10T11:39:34.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.548+0000 7f1a07820700 1 -- 172.21.15.144:0/713369120 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a001a00c0 con 0x7f1a000ffc60 2024-09-10T11:39:34.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.549+0000 7f1a07820700 1 -- 172.21.15.144:0/713369120 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a001075d0 con 0x7f1a000ffc60 2024-09-10T11:39:34.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.549+0000 7f19f67fc700 1 -- 172.21.15.144:0/713369120 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f19fc007d20 con 0x7f1a000ffc60 2024-09-10T11:39:34.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.550+0000 7f19f67fc700 1 --2- 172.21.15.144:0/713369120 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19ec06e200 0x7f19ec0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.550+0000 7f19f67fc700 1 -- 172.21.15.144:0/713369120 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f19fc08e060 con 0x7f1a000ffc60 2024-09-10T11:39:34.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.550+0000 7f1a055bc700 1 --2- 172.21.15.144:0/713369120 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19ec06e200 0x7f19ec0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.551+0000 7f1a055bc700 1 --2- 172.21.15.144:0/713369120 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19ec06e200 0x7f19ec0706b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f19f000b5c0 tx=0x7f19f0005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:34.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.555+0000 7f19f67fc700 1 -- 172.21.15.144:0/713369120 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f19fc057440 con 0x7f1a000ffc60 2024-09-10T11:39:34.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.767+0000 7f1a07820700 1 -- 172.21.15.144:0/713369120 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a0002cd40 con 0x7f19ec06e200 2024-09-10T11:39:34.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.777+0000 7f19f67fc700 1 -- 172.21.15.144:0/713369120 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1a0002cd40 con 0x7f19ec06e200 2024-09-10T11:39:34.778 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:39:34.778 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (7m) 90s ago 18m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:39:34.778 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (18m) 90s ago 18m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:39:34.778 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (17m) 77s ago 17m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:39:34.778 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (5m) 90s ago 18m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:39:34.778 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (5m) 77s ago 17m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:39:34.778 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (7m) 90s ago 18m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:39:34.778 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (113s) 90s ago 12m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:39:34.778 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (96s) 90s ago 12m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:39:34.779 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (12m) 77s ago 12m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:39:34.779 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (80s) 77s ago 12m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:39:34.779 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (9m) 90s ago 19m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:39:34.779 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (8m) 77s ago 17m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:39:34.779 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (6m) 90s ago 20m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:39:34.779 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (6m) 77s ago 17m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:39:34.779 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (8m) 90s ago 18m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:39:34.779 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (8m) 77s ago 17m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:39:34.779 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (5m) 90s ago 16m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:39:34.779 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (4m) 90s ago 16m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:39:34.779 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (3m) 90s ago 15m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:39:34.779 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (3m) 77s ago 15m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:39:34.779 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (2m) 77s ago 14m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:39:34.779 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (2m) 77s ago 14m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:39:34.780 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (8m) 90s ago 17m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:39:34.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.779+0000 7f1a07820700 1 -- 172.21.15.144:0/713369120 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19ec06e200 msgr2=0x7f19ec0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.779+0000 7f1a07820700 1 --2- 172.21.15.144:0/713369120 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19ec06e200 0x7f19ec0706b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f19f000b5c0 tx=0x7f19f0005960 comp rx=0 tx=0).stop 2024-09-10T11:39:34.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.779+0000 7f1a07820700 1 -- 172.21.15.144:0/713369120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a000ffc60 msgr2=0x7f1a001970d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.779+0000 7f1a07820700 1 --2- 172.21.15.144:0/713369120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a000ffc60 0x7f1a001970d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f19fc00c8f0 tx=0x7f19fc00ccb0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.779+0000 7f1a07820700 1 -- 172.21.15.144:0/713369120 shutdown_connections 2024-09-10T11:39:34.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.779+0000 7f1a07820700 1 --2- 172.21.15.144:0/713369120 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19ec06e200 0x7f19ec0706b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.779+0000 7f1a07820700 1 --2- 172.21.15.144:0/713369120 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a000ff2c0 0x7f1a00196b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.779+0000 7f1a07820700 1 --2- 172.21.15.144:0/713369120 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a000ffc60 0x7f1a001970d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.779+0000 7f1a07820700 1 -- 172.21.15.144:0/713369120 >> 172.21.15.144:0/713369120 conn(0x7f1a000fab80 msgr2=0x7f1a000fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:34.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.779+0000 7f1a07820700 1 -- 172.21.15.144:0/713369120 shutdown_connections 2024-09-10T11:39:34.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.779+0000 7f1a07820700 1 -- 172.21.15.144:0/713369120 wait complete. 2024-09-10T11:39:34.883 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.882+0000 7f84ef491700 1 -- 172.21.15.144:0/3136970952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84e80ffc60 msgr2=0x7f84e8103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.882+0000 7f84ef491700 1 --2- 172.21.15.144:0/3136970952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84e80ffc60 0x7f84e8103c30 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f84dc009910 tx=0x7f84dc009c20 comp rx=0 tx=0).stop 2024-09-10T11:39:34.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.882+0000 7f84ef491700 1 -- 172.21.15.144:0/3136970952 shutdown_connections 2024-09-10T11:39:34.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.882+0000 7f84ef491700 1 --2- 172.21.15.144:0/3136970952 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84e80ffc60 0x7f84e8103c30 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.882+0000 7f84ef491700 1 --2- 172.21.15.144:0/3136970952 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e80ff2c0 0x7f84e80ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.882+0000 7f84ef491700 1 -- 172.21.15.144:0/3136970952 >> 172.21.15.144:0/3136970952 conn(0x7f84e80fab80 msgr2=0x7f84e80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:34.884 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:34 smithi144 ceph-mon[87436]: pgmap v227: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:34.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.882+0000 7f84ef491700 1 -- 172.21.15.144:0/3136970952 shutdown_connections 2024-09-10T11:39:34.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.882+0000 7f84ef491700 1 -- 172.21.15.144:0/3136970952 wait complete. 2024-09-10T11:39:34.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.883+0000 7f84ef491700 1 Processor -- start 2024-09-10T11:39:34.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.883+0000 7f84ef491700 1 -- start start 2024-09-10T11:39:34.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.883+0000 7f84ef491700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84e80ff2c0 0x7f84e8195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.883+0000 7f84ef491700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e80ffc60 0x7f84e8196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.883+0000 7f84ef491700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e8196960 con 0x7f84e80ff2c0 2024-09-10T11:39:34.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.883+0000 7f84ef491700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84e819a6f0 con 0x7f84e80ffc60 2024-09-10T11:39:34.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.883+0000 7f84ed22d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84e80ff2c0 0x7f84e8195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.883+0000 7f84ed22d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84e80ff2c0 0x7f84e8195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56548/0 (socket says 172.21.15.144:56548) 2024-09-10T11:39:34.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.884+0000 7f84ed22d700 1 -- 172.21.15.144:0/2598222931 learned_addr learned my addr 172.21.15.144:0/2598222931 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:39:34.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.884+0000 7f84eca2c700 1 --2- 172.21.15.144:0/2598222931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e80ffc60 0x7f84e8196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.884+0000 7f84ed22d700 1 -- 172.21.15.144:0/2598222931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e80ffc60 msgr2=0x7f84e8196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:34.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.884+0000 7f84ed22d700 1 --2- 172.21.15.144:0/2598222931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e80ffc60 0x7f84e8196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:34.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.884+0000 7f84ed22d700 1 -- 172.21.15.144:0/2598222931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84dc0095f0 con 0x7f84e80ff2c0 2024-09-10T11:39:34.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.884+0000 7f84ed22d700 1 --2- 172.21.15.144:0/2598222931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84e80ff2c0 0x7f84e8195d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f84e400c8a0 tx=0x7f84e400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:34.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.884+0000 7f84da7fc700 1 -- 172.21.15.144:0/2598222931 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84e400cea0 con 0x7f84e80ff2c0 2024-09-10T11:39:34.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.884+0000 7f84da7fc700 1 -- 172.21.15.144:0/2598222931 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84e4004830 con 0x7f84e80ff2c0 2024-09-10T11:39:34.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.884+0000 7f84ef491700 1 -- 172.21.15.144:0/2598222931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84e819a9d0 con 0x7f84e80ff2c0 2024-09-10T11:39:34.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.884+0000 7f84da7fc700 1 -- 172.21.15.144:0/2598222931 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84e40055b0 con 0x7f84e80ff2c0 2024-09-10T11:39:34.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.884+0000 7f84ef491700 1 -- 172.21.15.144:0/2598222931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84e819af20 con 0x7f84e80ff2c0 2024-09-10T11:39:34.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.885+0000 7f84ef491700 1 -- 172.21.15.144:0/2598222931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84e804ea50 con 0x7f84e80ff2c0 2024-09-10T11:39:34.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.886+0000 7f84da7fc700 1 -- 172.21.15.144:0/2598222931 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f84e40049a0 con 0x7f84e80ff2c0 2024-09-10T11:39:34.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.886+0000 7f84da7fc700 1 --2- 172.21.15.144:0/2598222931 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f84d406e200 0x7f84d40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:34.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.886+0000 7f84da7fc700 1 -- 172.21.15.144:0/2598222931 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f84e408dc40 con 0x7f84e80ff2c0 2024-09-10T11:39:34.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.886+0000 7f84eca2c700 1 --2- 172.21.15.144:0/2598222931 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f84d406e200 0x7f84d40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:34.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.887+0000 7f84eca2c700 1 --2- 172.21.15.144:0/2598222931 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f84d406e200 0x7f84d40706b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f84e8197360 tx=0x7f84dc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:34.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:34.891+0000 7f84da7fc700 1 -- 172.21.15.144:0/2598222931 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f84e40570a0 con 0x7f84e80ff2c0 2024-09-10T11:39:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:34 smithi178 ceph-mon[65940]: pgmap v227: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:35.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.182+0000 7f84ef491700 1 -- 172.21.15.144:0/2598222931 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f84e819b460 con 0x7f84e80ff2c0 2024-09-10T11:39:35.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.182+0000 7f84da7fc700 1 -- 172.21.15.144:0/2598222931 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f84e401d070 con 0x7f84e80ff2c0 2024-09-10T11:39:35.184 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:39:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:39:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:39:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:39:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:39:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:39:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:39:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:39:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:39:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:39:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:39:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:39:35.184 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:39:35.185 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:39:35.185 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:39:35.185 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:39:35.185 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:39:35.185 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:39:35.185 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:39:35.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.185+0000 7f84ef491700 1 -- 172.21.15.144:0/2598222931 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f84d406e200 msgr2=0x7f84d40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.185+0000 7f84ef491700 1 --2- 172.21.15.144:0/2598222931 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f84d406e200 0x7f84d40706b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f84e8197360 tx=0x7f84dc01a040 comp rx=0 tx=0).stop 2024-09-10T11:39:35.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.185+0000 7f84ef491700 1 -- 172.21.15.144:0/2598222931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84e80ff2c0 msgr2=0x7f84e8195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.185+0000 7f84ef491700 1 --2- 172.21.15.144:0/2598222931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84e80ff2c0 0x7f84e8195d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f84e400c8a0 tx=0x7f84e400cc60 comp rx=0 tx=0).stop 2024-09-10T11:39:35.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.185+0000 7f84ef491700 1 -- 172.21.15.144:0/2598222931 shutdown_connections 2024-09-10T11:39:35.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.185+0000 7f84ef491700 1 --2- 172.21.15.144:0/2598222931 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f84d406e200 0x7f84d40706b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.185+0000 7f84ef491700 1 --2- 172.21.15.144:0/2598222931 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84e80ff2c0 0x7f84e8195d40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.185+0000 7f84ef491700 1 --2- 172.21.15.144:0/2598222931 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84e80ffc60 0x7f84e8196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.185+0000 7f84ef491700 1 -- 172.21.15.144:0/2598222931 >> 172.21.15.144:0/2598222931 conn(0x7f84e80fab80 msgr2=0x7f84e80fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:35.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.185+0000 7f84ef491700 1 -- 172.21.15.144:0/2598222931 shutdown_connections 2024-09-10T11:39:35.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.185+0000 7f84ef491700 1 -- 172.21.15.144:0/2598222931 wait complete. 2024-09-10T11:39:35.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.288+0000 7fc5651e1700 1 -- 172.21.15.144:0/2003752861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc560100370 msgr2=0x7fc560100740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.288+0000 7fc5651e1700 1 --2- 172.21.15.144:0/2003752861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc560100370 0x7fc560100740 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc554009910 tx=0x7fc554009c20 comp rx=0 tx=0).stop 2024-09-10T11:39:35.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.289+0000 7fc5651e1700 1 -- 172.21.15.144:0/2003752861 shutdown_connections 2024-09-10T11:39:35.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.289+0000 7fc5651e1700 1 --2- 172.21.15.144:0/2003752861 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc560100c80 0x7fc5601093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.289+0000 7fc5651e1700 1 --2- 172.21.15.144:0/2003752861 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc560100370 0x7fc560100740 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.289+0000 7fc5651e1700 1 -- 172.21.15.144:0/2003752861 >> 172.21.15.144:0/2003752861 conn(0x7fc5600747e0 msgr2=0x7fc560074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:35.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.289+0000 7fc5651e1700 1 -- 172.21.15.144:0/2003752861 shutdown_connections 2024-09-10T11:39:35.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.289+0000 7fc5651e1700 1 -- 172.21.15.144:0/2003752861 wait complete. 2024-09-10T11:39:35.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.289+0000 7fc5651e1700 1 Processor -- start 2024-09-10T11:39:35.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.290+0000 7fc5651e1700 1 -- start start 2024-09-10T11:39:35.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.290+0000 7fc5651e1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc560100370 0x7fc560101ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.290+0000 7fc5651e1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc560100c80 0x7fc560102220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.290+0000 7fc5651e1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc560102760 con 0x7fc560100c80 2024-09-10T11:39:35.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.290+0000 7fc5651e1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5601028a0 con 0x7fc560100370 2024-09-10T11:39:35.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.290+0000 7fc55e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc560100c80 0x7fc560102220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.290+0000 7fc55e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc560100c80 0x7fc560102220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56564/0 (socket says 172.21.15.144:56564) 2024-09-10T11:39:35.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.290+0000 7fc55e59c700 1 -- 172.21.15.144:0/3487328741 learned_addr learned my addr 172.21.15.144:0/3487328741 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:39:35.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.290+0000 7fc55ed9d700 1 --2- 172.21.15.144:0/3487328741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc560100370 0x7fc560101ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.290+0000 7fc55e59c700 1 -- 172.21.15.144:0/3487328741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc560100370 msgr2=0x7fc560101ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.290+0000 7fc55e59c700 1 --2- 172.21.15.144:0/3487328741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc560100370 0x7fc560101ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.290+0000 7fc55e59c700 1 -- 172.21.15.144:0/3487328741 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5540095f0 con 0x7fc560100c80 2024-09-10T11:39:35.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.290+0000 7fc55ed9d700 1 --2- 172.21.15.144:0/3487328741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc560100370 0x7fc560101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:39:35.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.290+0000 7fc55e59c700 1 --2- 172.21.15.144:0/3487328741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc560100c80 0x7fc560102220 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc55000c8a0 tx=0x7fc55000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:35.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.291+0000 7fc547fff700 1 -- 172.21.15.144:0/3487328741 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc55000cea0 con 0x7fc560100c80 2024-09-10T11:39:35.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.291+0000 7fc547fff700 1 -- 172.21.15.144:0/3487328741 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc550004830 con 0x7fc560100c80 2024-09-10T11:39:35.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.291+0000 7fc547fff700 1 -- 172.21.15.144:0/3487328741 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5500055b0 con 0x7fc560100c80 2024-09-10T11:39:35.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.291+0000 7fc5651e1700 1 -- 172.21.15.144:0/3487328741 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5601a1fb0 con 0x7fc560100c80 2024-09-10T11:39:35.294 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.291+0000 7fc5651e1700 1 -- 172.21.15.144:0/3487328741 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5601a23c0 con 0x7fc560100c80 2024-09-10T11:39:35.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.292+0000 7fc5651e1700 1 -- 172.21.15.144:0/3487328741 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc56004ea50 con 0x7fc560100c80 2024-09-10T11:39:35.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.292+0000 7fc547fff700 1 -- 172.21.15.144:0/3487328741 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc55000f450 con 0x7fc560100c80 2024-09-10T11:39:35.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.293+0000 7fc547fff700 1 --2- 172.21.15.144:0/3487328741 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc54806e200 0x7fc5480706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.293+0000 7fc547fff700 1 -- 172.21.15.144:0/3487328741 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc55008dd50 con 0x7fc560100c80 2024-09-10T11:39:35.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.293+0000 7fc55ed9d700 1 --2- 172.21.15.144:0/3487328741 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc54806e200 0x7fc5480706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.295 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.293+0000 7fc55ed9d700 1 --2- 172.21.15.144:0/3487328741 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc54806e200 0x7fc5480706b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc554005b40 tx=0x7fc554005ad0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:35.298 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.297+0000 7fc547fff700 1 -- 172.21.15.144:0/3487328741 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc550057130 con 0x7fc560100c80 2024-09-10T11:39:35.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.541+0000 7fc5651e1700 1 -- 172.21.15.144:0/3487328741 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc5601a2740 con 0x7fc560100c80 2024-09-10T11:39:35.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.541+0000 7fc547fff700 1 -- 172.21.15.144:0/3487328741 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7fc55001d070 con 0x7fc560100c80 2024-09-10T11:39:35.542 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:39:35.542 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:39:35.543 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:39:35.543 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:39:35.543 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:39:35.543 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:39:35.543 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:39:35.543 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:39:35.543 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:39:35.543 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:39:35.543 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:39:35.543 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:39:35.543 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:39:35.543 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:39:35.543 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:39:35.544 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:39:35.544 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:39:35.544 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:39:35.544 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:39:35.544 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:39:35.544 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:39:35.544 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:39:35.544 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:39:35.544 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:39:35.544 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:39:35.544 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:39:35.544 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:39:35.544 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:39:35.545 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:39:35.545 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:39:35.545 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:39:35.545 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:39:35.545 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:39:35.545 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:39:35.545 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:39:35.545 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:39:35.545 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:39:35.545 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:39:35.545 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:39:35.545 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:39:35.545 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:39:35.546 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:39:35.546 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:39:35.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.543+0000 7fc5651e1700 1 -- 172.21.15.144:0/3487328741 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc54806e200 msgr2=0x7fc5480706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.543+0000 7fc5651e1700 1 --2- 172.21.15.144:0/3487328741 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc54806e200 0x7fc5480706b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fc554005b40 tx=0x7fc554005ad0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.543+0000 7fc5651e1700 1 -- 172.21.15.144:0/3487328741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc560100c80 msgr2=0x7fc560102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.543+0000 7fc5651e1700 1 --2- 172.21.15.144:0/3487328741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc560100c80 0x7fc560102220 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fc55000c8a0 tx=0x7fc55000cc60 comp rx=0 tx=0).stop 2024-09-10T11:39:35.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.544+0000 7fc5651e1700 1 -- 172.21.15.144:0/3487328741 shutdown_connections 2024-09-10T11:39:35.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.544+0000 7fc5651e1700 1 --2- 172.21.15.144:0/3487328741 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc54806e200 0x7fc5480706b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.544+0000 7fc5651e1700 1 --2- 172.21.15.144:0/3487328741 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc560100370 0x7fc560101ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.544+0000 7fc5651e1700 1 --2- 172.21.15.144:0/3487328741 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc560100c80 0x7fc560102220 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.544+0000 7fc5651e1700 1 -- 172.21.15.144:0/3487328741 >> 172.21.15.144:0/3487328741 conn(0x7fc5600747e0 msgr2=0x7fc5600febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:35.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.544+0000 7fc5651e1700 1 -- 172.21.15.144:0/3487328741 shutdown_connections 2024-09-10T11:39:35.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.544+0000 7fc5651e1700 1 -- 172.21.15.144:0/3487328741 wait complete. 2024-09-10T11:39:35.547 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:39:35.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.649+0000 7f0773c0d700 1 -- 172.21.15.144:0/370694762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c101cc0 msgr2=0x7f076c10a1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.649+0000 7f0773c0d700 1 --2- 172.21.15.144:0/370694762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c101cc0 0x7f076c10a1c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f075c009910 tx=0x7f075c009c20 comp rx=0 tx=0).stop 2024-09-10T11:39:35.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.649+0000 7f0773c0d700 1 -- 172.21.15.144:0/370694762 shutdown_connections 2024-09-10T11:39:35.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.649+0000 7f0773c0d700 1 --2- 172.21.15.144:0/370694762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c101cc0 0x7f076c10a1c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.649+0000 7f0773c0d700 1 --2- 172.21.15.144:0/370694762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f076c1013b0 0x7f076c101780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.649+0000 7f0773c0d700 1 -- 172.21.15.144:0/370694762 >> 172.21.15.144:0/370694762 conn(0x7f076c0fabb0 msgr2=0x7f076c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:35.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.650+0000 7f0773c0d700 1 -- 172.21.15.144:0/370694762 shutdown_connections 2024-09-10T11:39:35.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.650+0000 7f0773c0d700 1 -- 172.21.15.144:0/370694762 wait complete. 2024-09-10T11:39:35.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.650+0000 7f0773c0d700 1 Processor -- start 2024-09-10T11:39:35.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.650+0000 7f0773c0d700 1 -- start start 2024-09-10T11:39:35.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.650+0000 7f0773c0d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c1013b0 0x7f076c195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.650+0000 7f0773c0d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f076c101cc0 0x7f076c1963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.650+0000 7f0773c0d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c196ab0 con 0x7f076c1013b0 2024-09-10T11:39:35.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.650+0000 7f0773c0d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076c19a840 con 0x7f076c101cc0 2024-09-10T11:39:35.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.650+0000 7f07719a9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c1013b0 0x7f076c195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.651+0000 7f07719a9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c1013b0 0x7f076c195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56594/0 (socket says 172.21.15.144:56594) 2024-09-10T11:39:35.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.651+0000 7f07719a9700 1 -- 172.21.15.144:0/1215068830 learned_addr learned my addr 172.21.15.144:0/1215068830 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:39:35.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.651+0000 7f07711a8700 1 --2- 172.21.15.144:0/1215068830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f076c101cc0 0x7f076c1963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.651+0000 7f07719a9700 1 -- 172.21.15.144:0/1215068830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f076c101cc0 msgr2=0x7f076c1963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.651+0000 7f07719a9700 1 --2- 172.21.15.144:0/1215068830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f076c101cc0 0x7f076c1963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.651+0000 7f07719a9700 1 -- 172.21.15.144:0/1215068830 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c0095f0 con 0x7f076c1013b0 2024-09-10T11:39:35.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.651+0000 7f07711a8700 1 --2- 172.21.15.144:0/1215068830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f076c101cc0 0x7f076c1963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:39:35.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.651+0000 7f07719a9700 1 --2- 172.21.15.144:0/1215068830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c1013b0 0x7f076c195e90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f076800b4b0 tx=0x7f076800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:35.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.651+0000 7f0762ffd700 1 -- 172.21.15.144:0/1215068830 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0768011800 con 0x7f076c1013b0 2024-09-10T11:39:35.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.651+0000 7f0762ffd700 1 -- 172.21.15.144:0/1215068830 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0768011e20 con 0x7f076c1013b0 2024-09-10T11:39:35.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.651+0000 7f0762ffd700 1 -- 172.21.15.144:0/1215068830 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f076800f2e0 con 0x7f076c1013b0 2024-09-10T11:39:35.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.651+0000 7f0773c0d700 1 -- 172.21.15.144:0/1215068830 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076c19ab20 con 0x7f076c1013b0 2024-09-10T11:39:35.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.651+0000 7f0773c0d700 1 -- 172.21.15.144:0/1215068830 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f076c19b070 con 0x7f076c1013b0 2024-09-10T11:39:35.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.652+0000 7f0773c0d700 1 -- 172.21.15.144:0/1215068830 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f076c107830 con 0x7f076c1013b0 2024-09-10T11:39:35.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.653+0000 7f0762ffd700 1 -- 172.21.15.144:0/1215068830 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0768010380 con 0x7f076c1013b0 2024-09-10T11:39:35.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.653+0000 7f0762ffd700 1 --2- 172.21.15.144:0/1215068830 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f075806e200 0x7f07580706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.654+0000 7f0762ffd700 1 -- 172.21.15.144:0/1215068830 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f076808df80 con 0x7f076c1013b0 2024-09-10T11:39:35.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.654+0000 7f07711a8700 1 --2- 172.21.15.144:0/1215068830 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f075806e200 0x7f07580706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.654+0000 7f07711a8700 1 --2- 172.21.15.144:0/1215068830 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f075806e200 0x7f07580706b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f075c00b5c0 tx=0x7f075c005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:35.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.657+0000 7f0762ffd700 1 -- 172.21.15.144:0/1215068830 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0768057480 con 0x7f076c1013b0 2024-09-10T11:39:35.883 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.882+0000 7f0773c0d700 1 -- 172.21.15.144:0/1215068830 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f076c19b3b0 con 0x7f075806e200 2024-09-10T11:39:35.887 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:35 smithi144 ceph-mon[87436]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:35.887 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:35 smithi144 ceph-mon[87436]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:35.887 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:35 smithi144 ceph-mon[87436]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:35.887 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:35 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2598222931' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:39:35.887 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:35 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3487328741' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:39:35.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.883+0000 7f0762ffd700 1 -- 172.21.15.144:0/1215068830 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f076c19b3b0 con 0x7f075806e200 2024-09-10T11:39:35.888 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:39:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:39:35.888 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:39:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:39:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:39:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:39:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:39:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:39:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:39:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:39:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:39:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:39:35.889 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:39:35.889 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:39:35.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.886+0000 7f0773c0d700 1 -- 172.21.15.144:0/1215068830 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f075806e200 msgr2=0x7f07580706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.886+0000 7f0773c0d700 1 --2- 172.21.15.144:0/1215068830 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f075806e200 0x7f07580706b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f075c00b5c0 tx=0x7f075c005480 comp rx=0 tx=0).stop 2024-09-10T11:39:35.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.886+0000 7f0773c0d700 1 -- 172.21.15.144:0/1215068830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c1013b0 msgr2=0x7f076c195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.886+0000 7f0773c0d700 1 --2- 172.21.15.144:0/1215068830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c1013b0 0x7f076c195e90 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f076800b4b0 tx=0x7f076800b870 comp rx=0 tx=0).stop 2024-09-10T11:39:35.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.887+0000 7f0773c0d700 1 -- 172.21.15.144:0/1215068830 shutdown_connections 2024-09-10T11:39:35.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.887+0000 7f0773c0d700 1 --2- 172.21.15.144:0/1215068830 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f075806e200 0x7f07580706b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.887+0000 7f0773c0d700 1 --2- 172.21.15.144:0/1215068830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f076c1013b0 0x7f076c195e90 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.887+0000 7f0773c0d700 1 --2- 172.21.15.144:0/1215068830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f076c101cc0 0x7f076c1963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.887+0000 7f0773c0d700 1 -- 172.21.15.144:0/1215068830 >> 172.21.15.144:0/1215068830 conn(0x7f076c0fabb0 msgr2=0x7f076c104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:35.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.887+0000 7f0773c0d700 1 -- 172.21.15.144:0/1215068830 shutdown_connections 2024-09-10T11:39:35.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.887+0000 7f0773c0d700 1 -- 172.21.15.144:0/1215068830 wait complete. 2024-09-10T11:39:35.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.991+0000 7f25dfcb8700 1 -- 172.21.15.144:0/2581939013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d8100db0 msgr2=0x7f25d8104d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.991+0000 7f25dfcb8700 1 --2- 172.21.15.144:0/2581939013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d8100db0 0x7f25d8104d80 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f25cc009910 tx=0x7f25cc009c20 comp rx=0 tx=0).stop 2024-09-10T11:39:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.991+0000 7f25dfcb8700 1 -- 172.21.15.144:0/2581939013 shutdown_connections 2024-09-10T11:39:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.991+0000 7f25dfcb8700 1 --2- 172.21.15.144:0/2581939013 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d8100db0 0x7f25d8104d80 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.991+0000 7f25dfcb8700 1 --2- 172.21.15.144:0/2581939013 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d8100410 0x7f25d81007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.991+0000 7f25dfcb8700 1 -- 172.21.15.144:0/2581939013 >> 172.21.15.144:0/2581939013 conn(0x7f25d80fbcf0 msgr2=0x7f25d80fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.991+0000 7f25dfcb8700 1 -- 172.21.15.144:0/2581939013 shutdown_connections 2024-09-10T11:39:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.991+0000 7f25dfcb8700 1 -- 172.21.15.144:0/2581939013 wait complete. 2024-09-10T11:39:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.992+0000 7f25dfcb8700 1 Processor -- start 2024-09-10T11:39:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.992+0000 7f25dfcb8700 1 -- start start 2024-09-10T11:39:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.992+0000 7f25dfcb8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d8100410 0x7f25d8195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.992+0000 7f25dfcb8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d8100db0 0x7f25d81962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.992+0000 7f25dfcb8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d81969c0 con 0x7f25d8100db0 2024-09-10T11:39:35.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.992+0000 7f25dfcb8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d819a750 con 0x7f25d8100410 2024-09-10T11:39:35.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.992+0000 7f25dd253700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d8100db0 0x7f25d81962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.992+0000 7f25dd253700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d8100db0 0x7f25d81962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56608/0 (socket says 172.21.15.144:56608) 2024-09-10T11:39:35.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.992+0000 7f25dd253700 1 -- 172.21.15.144:0/3781608128 learned_addr learned my addr 172.21.15.144:0/3781608128 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:39:35.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.992+0000 7f25dda54700 1 --2- 172.21.15.144:0/3781608128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d8100410 0x7f25d8195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.993+0000 7f25dd253700 1 -- 172.21.15.144:0/3781608128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d8100410 msgr2=0x7f25d8195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:35.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.993+0000 7f25dd253700 1 --2- 172.21.15.144:0/3781608128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d8100410 0x7f25d8195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:35.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.993+0000 7f25dd253700 1 -- 172.21.15.144:0/3781608128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc0095f0 con 0x7f25d8100db0 2024-09-10T11:39:35.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.993+0000 7f25dd253700 1 --2- 172.21.15.144:0/3781608128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d8100db0 0x7f25d81962e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00bd70 tx=0x7f25cc00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:35.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.993+0000 7f25d6ffd700 1 -- 172.21.15.144:0/3781608128 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25cc01d070 con 0x7f25d8100db0 2024-09-10T11:39:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.993+0000 7f25d6ffd700 1 -- 172.21.15.144:0/3781608128 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25cc005dc0 con 0x7f25d8100db0 2024-09-10T11:39:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.993+0000 7f25dfcb8700 1 -- 172.21.15.144:0/3781608128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d819a9d0 con 0x7f25d8100db0 2024-09-10T11:39:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.993+0000 7f25d6ffd700 1 -- 172.21.15.144:0/3781608128 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25cc017400 con 0x7f25d8100db0 2024-09-10T11:39:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.993+0000 7f25dfcb8700 1 -- 172.21.15.144:0/3781608128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d819aec0 con 0x7f25d8100db0 2024-09-10T11:39:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.994+0000 7f25dfcb8700 1 -- 172.21.15.144:0/3781608128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d804ea50 con 0x7f25d8100db0 2024-09-10T11:39:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.994+0000 7f25d6ffd700 1 -- 172.21.15.144:0/3781608128 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f25cc00f460 con 0x7f25d8100db0 2024-09-10T11:39:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.995+0000 7f25d6ffd700 1 --2- 172.21.15.144:0/3781608128 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25c006e200 0x7f25c00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:39:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.995+0000 7f25dda54700 1 --2- 172.21.15.144:0/3781608128 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25c006e200 0x7f25c00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:39:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.995+0000 7f25d6ffd700 1 -- 172.21.15.144:0/3781608128 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f25cc08f4f0 con 0x7f25d8100db0 2024-09-10T11:39:35.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.995+0000 7f25dda54700 1 --2- 172.21.15.144:0/3781608128 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25c006e200 0x7f25c00706b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009550 tx=0x7f25d00091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:39:36.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:35.999+0000 7f25d6ffd700 1 -- 172.21.15.144:0/3781608128 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f25cc0588d0 con 0x7f25d8100db0 2024-09-10T11:39:36.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:35 smithi178 ceph-mon[65940]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:36.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:35 smithi178 ceph-mon[65940]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:36.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:35 smithi178 ceph-mon[65940]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:36.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:35 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2598222931' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:39:36.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:35 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3487328741' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:36.283+0000 7f25dfcb8700 1 -- 172.21.15.144:0/3781608128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f25d8197100 con 0x7f25d8100db0 2024-09-10T11:39:36.284 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:36.283+0000 7f25d6ffd700 1 -- 172.21.15.144:0/3781608128 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f25cc05bf90 con 0x7f25d8100db0 2024-09-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:39:36.285 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:36.285+0000 7f25dfcb8700 1 -- 172.21.15.144:0/3781608128 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25c006e200 msgr2=0x7f25c00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:36.285+0000 7f25dfcb8700 1 --2- 172.21.15.144:0/3781608128 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25c006e200 0x7f25c00706b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009550 tx=0x7f25d00091b0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:36.285+0000 7f25dfcb8700 1 -- 172.21.15.144:0/3781608128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d8100db0 msgr2=0x7f25d81962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:36.285+0000 7f25dfcb8700 1 --2- 172.21.15.144:0/3781608128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d8100db0 0x7f25d81962e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00bd70 tx=0x7f25cc00bda0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:36.285+0000 7f25dfcb8700 1 -- 172.21.15.144:0/3781608128 shutdown_connections 2024-09-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:36.285+0000 7f25dfcb8700 1 --2- 172.21.15.144:0/3781608128 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25c006e200 0x7f25c00706b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:36.285+0000 7f25dfcb8700 1 --2- 172.21.15.144:0/3781608128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25d8100410 0x7f25d8195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:36.285+0000 7f25dfcb8700 1 --2- 172.21.15.144:0/3781608128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25d8100db0 0x7f25d81962e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:36.286+0000 7f25dfcb8700 1 -- 172.21.15.144:0/3781608128 >> 172.21.15.144:0/3781608128 conn(0x7f25d80fbcf0 msgr2=0x7f25d80fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:39:36.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:36.286+0000 7f25dfcb8700 1 -- 172.21.15.144:0/3781608128 shutdown_connections 2024-09-10T11:39:36.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:39:36.286+0000 7f25dfcb8700 1 -- 172.21.15.144:0/3781608128 wait complete. 2024-09-10T11:39:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:36 smithi178 ceph-mon[65940]: pgmap v228: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:36 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3781608128' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:39:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:36 smithi144 ceph-mon[87436]: pgmap v228: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:36 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3781608128' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:39:38.085 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:37 smithi144 ceph-mon[87436]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:38.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:37 smithi178 ceph-mon[65940]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:39:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:38 smithi178 ceph-mon[65940]: pgmap v229: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:38 smithi144 ceph-mon[87436]: pgmap v229: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:40 smithi178 ceph-mon[65940]: pgmap v230: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:40 smithi144 ceph-mon[87436]: pgmap v230: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:43.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:42 smithi144 ceph-mon[87436]: pgmap v231: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:43.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:42 smithi178 ceph-mon[65940]: pgmap v231: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:43.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:39:44.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:39:45.021 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:44 smithi144 ceph-mon[87436]: pgmap v232: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:44 smithi178 ceph-mon[65940]: pgmap v232: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:46 smithi178 ceph-mon[65940]: pgmap v233: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:46 smithi144 ceph-mon[87436]: pgmap v233: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:48.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:48 smithi178 ceph-mon[65940]: pgmap v234: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:49.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:48 smithi144 ceph-mon[87436]: pgmap v234: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:50 smithi178 ceph-mon[65940]: pgmap v235: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:50 smithi144 ceph-mon[87436]: pgmap v235: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:53.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:52 smithi144 ceph-mon[87436]: pgmap v236: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:52 smithi178 ceph-mon[65940]: pgmap v236: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:54 smithi178 ceph-mon[65940]: pgmap v237: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:55.126 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:54 smithi144 ceph-mon[87436]: pgmap v237: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:56 smithi178 ceph-mon[65940]: pgmap v238: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:56 smithi144 ceph-mon[87436]: pgmap v238: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:58.919 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:58 smithi178 ceph-mon[65940]: pgmap v239: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:58.919 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:39:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:39:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:58 smithi144 ceph-mon[87436]: pgmap v239: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:39:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:39:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:40:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:39:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:40:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:39:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:40:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:39:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-09-10T11:40:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:39:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:40:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:39:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:40:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:39:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:40:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:39:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-09-10T11:40:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:00 smithi178 ceph-mon[65940]: pgmap v240: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:00 smithi178 ceph-mon[65940]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:40:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:00 smithi178 ceph-mon[65940]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:40:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:00 smithi178 ceph-mon[65940]: fs cephfs is degraded 2024-09-10T11:40:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:00 smithi178 ceph-mon[65940]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:40:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:00 smithi178 ceph-mon[65940]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:40:01.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:00 smithi178 ceph-mon[65940]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:40:01.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:00 smithi178 ceph-mon[65940]: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:40:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:00 smithi144 ceph-mon[87436]: pgmap v240: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:00 smithi144 ceph-mon[87436]: Health detail: HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:40:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:00 smithi144 ceph-mon[87436]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:40:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:00 smithi144 ceph-mon[87436]: fs cephfs is degraded 2024-09-10T11:40:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:00 smithi144 ceph-mon[87436]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:40:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:00 smithi144 ceph-mon[87436]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:40:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:00 smithi144 ceph-mon[87436]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:40:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:00 smithi144 ceph-mon[87436]: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:40:03.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:02 smithi144 ceph-mon[87436]: pgmap v241: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:03.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:02 smithi178 ceph-mon[65940]: pgmap v241: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:04 smithi178 ceph-mon[65940]: pgmap v242: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:05.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:04 smithi144 ceph-mon[87436]: pgmap v242: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:06.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.404+0000 7fa9c9a0e700 1 -- 172.21.15.144:0/1208771934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9c40ffc30 msgr2=0x7fa9c410a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.404+0000 7fa9c9a0e700 1 --2- 172.21.15.144:0/1208771934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9c40ffc30 0x7fa9c410a6a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009940 tx=0x7fa9ac009c50 comp rx=0 tx=0).stop 2024-09-10T11:40:06.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.404+0000 7fa9c9a0e700 1 -- 172.21.15.144:0/1208771934 shutdown_connections 2024-09-10T11:40:06.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.404+0000 7fa9c9a0e700 1 --2- 172.21.15.144:0/1208771934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9c40ffc30 0x7fa9c410a6a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.404+0000 7fa9c9a0e700 1 --2- 172.21.15.144:0/1208771934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9c40ff320 0x7fa9c40ff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.404+0000 7fa9c9a0e700 1 -- 172.21.15.144:0/1208771934 >> 172.21.15.144:0/1208771934 conn(0x7fa9c40748b0 msgr2=0x7fa9c4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:06.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.404+0000 7fa9c9a0e700 1 -- 172.21.15.144:0/1208771934 shutdown_connections 2024-09-10T11:40:06.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.404+0000 7fa9c9a0e700 1 -- 172.21.15.144:0/1208771934 wait complete. 2024-09-10T11:40:06.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.405+0000 7fa9c9a0e700 1 Processor -- start 2024-09-10T11:40:06.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.405+0000 7fa9c9a0e700 1 -- start start 2024-09-10T11:40:06.408 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.405+0000 7fa9c9a0e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9c40ff320 0x7fa9c4100500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.408 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.405+0000 7fa9c9a0e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9c40ffc30 0x7fa9c4100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.408 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.405+0000 7fa9c9a0e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c4100f80 con 0x7fa9c40ffc30 2024-09-10T11:40:06.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.405+0000 7fa9c9a0e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c41010c0 con 0x7fa9c40ff320 2024-09-10T11:40:06.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.406+0000 7fa9ba7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9c40ffc30 0x7fa9c4100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.406+0000 7fa9ba7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9c40ffc30 0x7fa9c4100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50702/0 (socket says 172.21.15.144:50702) 2024-09-10T11:40:06.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.406+0000 7fa9ba7fc700 1 -- 172.21.15.144:0/2923039185 learned_addr learned my addr 172.21.15.144:0/2923039185 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:40:06.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.406+0000 7fa9c2ffd700 1 --2- 172.21.15.144:0/2923039185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9c40ff320 0x7fa9c4100500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.406+0000 7fa9ba7fc700 1 -- 172.21.15.144:0/2923039185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9c40ff320 msgr2=0x7fa9c4100500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.406+0000 7fa9ba7fc700 1 --2- 172.21.15.144:0/2923039185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9c40ff320 0x7fa9c4100500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.406+0000 7fa9ba7fc700 1 -- 172.21.15.144:0/2923039185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9ac0095f0 con 0x7fa9c40ffc30 2024-09-10T11:40:06.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.406+0000 7fa9ba7fc700 1 --2- 172.21.15.144:0/2923039185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9c40ffc30 0x7fa9c4100a40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0055f0 tx=0x7fa9ac005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:06.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.406+0000 7fa9c8a0c700 1 -- 172.21.15.144:0/2923039185 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9ac01d070 con 0x7fa9c40ffc30 2024-09-10T11:40:06.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.406+0000 7fa9c8a0c700 1 -- 172.21.15.144:0/2923039185 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9ac005dc0 con 0x7fa9c40ffc30 2024-09-10T11:40:06.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.406+0000 7fa9c8a0c700 1 -- 172.21.15.144:0/2923039185 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9ac021410 con 0x7fa9c40ffc30 2024-09-10T11:40:06.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.407+0000 7fa9c9a0e700 1 -- 172.21.15.144:0/2923039185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c41a1e10 con 0x7fa9c40ffc30 2024-09-10T11:40:06.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.407+0000 7fa9c9a0e700 1 -- 172.21.15.144:0/2923039185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c41a2220 con 0x7fa9c40ffc30 2024-09-10T11:40:06.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.408+0000 7fa9c9a0e700 1 -- 172.21.15.144:0/2923039185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9c4107da0 con 0x7fa9c40ffc30 2024-09-10T11:40:06.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.408+0000 7fa9c8a0c700 1 -- 172.21.15.144:0/2923039185 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa9ac00f6f0 con 0x7fa9c40ffc30 2024-09-10T11:40:06.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.409+0000 7fa9c8a0c700 1 --2- 172.21.15.144:0/2923039185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa9b006e200 0x7fa9b00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.409+0000 7fa9c8a0c700 1 -- 172.21.15.144:0/2923039185 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa9ac08f390 con 0x7fa9c40ffc30 2024-09-10T11:40:06.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.409+0000 7fa9c2ffd700 1 --2- 172.21.15.144:0/2923039185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa9b006e200 0x7fa9b00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.409+0000 7fa9c2ffd700 1 --2- 172.21.15.144:0/2923039185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa9b006e200 0x7fa9b00706b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4005950 tx=0x7fa9b400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:06.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.413+0000 7fa9c8a0c700 1 -- 172.21.15.144:0/2923039185 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa9ac058830 con 0x7fa9c40ffc30 2024-09-10T11:40:06.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.638+0000 7fa9c9a0e700 1 -- 172.21.15.144:0/2923039185 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa9c4101280 con 0x7fa9b006e200 2024-09-10T11:40:06.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.639+0000 7fa9c8a0c700 1 -- 172.21.15.144:0/2923039185 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa9c4101280 con 0x7fa9b006e200 2024-09-10T11:40:06.643 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.641+0000 7fa9c9a0e700 1 -- 172.21.15.144:0/2923039185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa9b006e200 msgr2=0x7fa9b00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.643 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.641+0000 7fa9c9a0e700 1 --2- 172.21.15.144:0/2923039185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa9b006e200 0x7fa9b00706b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4005950 tx=0x7fa9b400b410 comp rx=0 tx=0).stop 2024-09-10T11:40:06.643 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.641+0000 7fa9c9a0e700 1 -- 172.21.15.144:0/2923039185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9c40ffc30 msgr2=0x7fa9c4100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.643 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.641+0000 7fa9c9a0e700 1 --2- 172.21.15.144:0/2923039185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9c40ffc30 0x7fa9c4100a40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac0055f0 tx=0x7fa9ac005620 comp rx=0 tx=0).stop 2024-09-10T11:40:06.643 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.642+0000 7fa9c9a0e700 1 -- 172.21.15.144:0/2923039185 shutdown_connections 2024-09-10T11:40:06.643 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.642+0000 7fa9c9a0e700 1 --2- 172.21.15.144:0/2923039185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa9b006e200 0x7fa9b00706b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.643 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.642+0000 7fa9c9a0e700 1 --2- 172.21.15.144:0/2923039185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9c40ff320 0x7fa9c4100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.643 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.642+0000 7fa9c9a0e700 1 --2- 172.21.15.144:0/2923039185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9c40ffc30 0x7fa9c4100a40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.643 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.642+0000 7fa9c9a0e700 1 -- 172.21.15.144:0/2923039185 >> 172.21.15.144:0/2923039185 conn(0x7fa9c40748b0 msgr2=0x7fa9c4104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:06.643 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.642+0000 7fa9c9a0e700 1 -- 172.21.15.144:0/2923039185 shutdown_connections 2024-09-10T11:40:06.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.642+0000 7fa9c9a0e700 1 -- 172.21.15.144:0/2923039185 wait complete. 2024-09-10T11:40:06.653 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:40:06.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.749+0000 7f4a7cf8a700 1 -- 172.21.15.144:0/371448667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a78100d70 msgr2=0x7f4a78104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.749+0000 7f4a7cf8a700 1 --2- 172.21.15.144:0/371448667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a78100d70 0x7f4a78104d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009910 tx=0x7f4a6c009c20 comp rx=0 tx=0).stop 2024-09-10T11:40:06.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.750+0000 7f4a7cf8a700 1 -- 172.21.15.144:0/371448667 shutdown_connections 2024-09-10T11:40:06.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.750+0000 7f4a7cf8a700 1 --2- 172.21.15.144:0/371448667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a78100d70 0x7f4a78104d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.750+0000 7f4a7cf8a700 1 --2- 172.21.15.144:0/371448667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a781003d0 0x7f4a781007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.750+0000 7f4a7cf8a700 1 -- 172.21.15.144:0/371448667 >> 172.21.15.144:0/371448667 conn(0x7f4a780fbc90 msgr2=0x7f4a780fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:06.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.750+0000 7f4a7cf8a700 1 -- 172.21.15.144:0/371448667 shutdown_connections 2024-09-10T11:40:06.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.750+0000 7f4a7cf8a700 1 -- 172.21.15.144:0/371448667 wait complete. 2024-09-10T11:40:06.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.750+0000 7f4a7cf8a700 1 Processor -- start 2024-09-10T11:40:06.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.750+0000 7f4a7cf8a700 1 -- start start 2024-09-10T11:40:06.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a7cf8a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a781003d0 0x7f4a78073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a7cf8a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a78100d70 0x7f4a78073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a7cf8a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a78073ad0 con 0x7f4a781003d0 2024-09-10T11:40:06.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a7cf8a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a78073c10 con 0x7f4a78100d70 2024-09-10T11:40:06.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a7659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a781003d0 0x7f4a78073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a7659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a781003d0 0x7f4a78073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50734/0 (socket says 172.21.15.144:50734) 2024-09-10T11:40:06.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a7659c700 1 -- 172.21.15.144:0/4191555840 learned_addr learned my addr 172.21.15.144:0/4191555840 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:40:06.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a75d9b700 1 --2- 172.21.15.144:0/4191555840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a78100d70 0x7f4a78073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a7659c700 1 -- 172.21.15.144:0/4191555840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a78100d70 msgr2=0x7f4a78073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a7659c700 1 --2- 172.21.15.144:0/4191555840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a78100d70 0x7f4a78073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a7659c700 1 -- 172.21.15.144:0/4191555840 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a6c0095f0 con 0x7f4a781003d0 2024-09-10T11:40:06.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a7659c700 1 --2- 172.21.15.144:0/4191555840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a781003d0 0x7f4a78073050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800c8a0 tx=0x7f4a6800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:06.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a677fe700 1 -- 172.21.15.144:0/4191555840 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a680078e0 con 0x7f4a781003d0 2024-09-10T11:40:06.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a677fe700 1 -- 172.21.15.144:0/4191555840 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a6800f450 con 0x7f4a781003d0 2024-09-10T11:40:06.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a7cf8a700 1 -- 172.21.15.144:0/4191555840 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a781a1f10 con 0x7f4a781003d0 2024-09-10T11:40:06.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a677fe700 1 -- 172.21.15.144:0/4191555840 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a6800e5e0 con 0x7f4a781003d0 2024-09-10T11:40:06.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.751+0000 7f4a7cf8a700 1 -- 172.21.15.144:0/4191555840 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a781a2320 con 0x7f4a781003d0 2024-09-10T11:40:06.755 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.752+0000 7f4a7cf8a700 1 -- 172.21.15.144:0/4191555840 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a78108740 con 0x7f4a781003d0 2024-09-10T11:40:06.755 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.753+0000 7f4a677fe700 1 -- 172.21.15.144:0/4191555840 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4a6800f5c0 con 0x7f4a781003d0 2024-09-10T11:40:06.755 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.753+0000 7f4a677fe700 1 --2- 172.21.15.144:0/4191555840 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4a6006e200 0x7f4a600706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:06.755 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.754+0000 7f4a677fe700 1 -- 172.21.15.144:0/4191555840 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4a6808df10 con 0x7f4a781003d0 2024-09-10T11:40:06.756 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.754+0000 7f4a75d9b700 1 --2- 172.21.15.144:0/4191555840 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4a6006e200 0x7f4a600706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:06.756 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.754+0000 7f4a75d9b700 1 --2- 172.21.15.144:0/4191555840 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4a6006e200 0x7f4a600706b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4a78074840 tx=0x7f4a6c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:06.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.758+0000 7f4a677fe700 1 -- 172.21.15.144:0/4191555840 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4a68057370 con 0x7f4a781003d0 2024-09-10T11:40:06.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.979+0000 7f4a7cf8a700 1 -- 172.21.15.144:0/4191555840 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a7802cd90 con 0x7f4a6006e200 2024-09-10T11:40:06.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.980+0000 7f4a677fe700 1 -- 172.21.15.144:0/4191555840 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4a7802cd90 con 0x7f4a6006e200 2024-09-10T11:40:06.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.982+0000 7f4a7cf8a700 1 -- 172.21.15.144:0/4191555840 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4a6006e200 msgr2=0x7f4a600706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.982+0000 7f4a7cf8a700 1 --2- 172.21.15.144:0/4191555840 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4a6006e200 0x7f4a600706b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4a78074840 tx=0x7f4a6c01a040 comp rx=0 tx=0).stop 2024-09-10T11:40:06.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.982+0000 7f4a7cf8a700 1 -- 172.21.15.144:0/4191555840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a781003d0 msgr2=0x7f4a78073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:06.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.982+0000 7f4a7cf8a700 1 --2- 172.21.15.144:0/4191555840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a781003d0 0x7f4a78073050 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f4a6800c8a0 tx=0x7f4a6800cbb0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.982+0000 7f4a7cf8a700 1 -- 172.21.15.144:0/4191555840 shutdown_connections 2024-09-10T11:40:06.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.982+0000 7f4a7cf8a700 1 --2- 172.21.15.144:0/4191555840 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4a6006e200 0x7f4a600706b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.982+0000 7f4a7cf8a700 1 --2- 172.21.15.144:0/4191555840 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a781003d0 0x7f4a78073050 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.982+0000 7f4a7cf8a700 1 --2- 172.21.15.144:0/4191555840 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a78100d70 0x7f4a78073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:06.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.982+0000 7f4a7cf8a700 1 -- 172.21.15.144:0/4191555840 >> 172.21.15.144:0/4191555840 conn(0x7f4a780fbc90 msgr2=0x7f4a780fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:06.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.983+0000 7f4a7cf8a700 1 -- 172.21.15.144:0/4191555840 shutdown_connections 2024-09-10T11:40:06.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:06.983+0000 7f4a7cf8a700 1 -- 172.21.15.144:0/4191555840 wait complete. 2024-09-10T11:40:07.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.087+0000 7fc6267ab700 1 -- 172.21.15.144:0/1983573413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6200ff270 msgr2=0x7fc6200ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.087+0000 7fc6267ab700 1 --2- 172.21.15.144:0/1983573413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6200ff270 0x7fc6200ff640 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc610009910 tx=0x7fc610009c20 comp rx=0 tx=0).stop 2024-09-10T11:40:07.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.087+0000 7fc6267ab700 1 -- 172.21.15.144:0/1983573413 shutdown_connections 2024-09-10T11:40:07.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.087+0000 7fc6267ab700 1 --2- 172.21.15.144:0/1983573413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6200ffb80 0x7fc62010a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.088+0000 7fc6267ab700 1 --2- 172.21.15.144:0/1983573413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6200ff270 0x7fc6200ff640 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.088+0000 7fc6267ab700 1 -- 172.21.15.144:0/1983573413 >> 172.21.15.144:0/1983573413 conn(0x7fc620074850 msgr2=0x7fc620074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:07.089 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:06 smithi144 ceph-mon[87436]: pgmap v243: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:07.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.088+0000 7fc6267ab700 1 -- 172.21.15.144:0/1983573413 shutdown_connections 2024-09-10T11:40:07.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.088+0000 7fc6267ab700 1 -- 172.21.15.144:0/1983573413 wait complete. 2024-09-10T11:40:07.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.088+0000 7fc6267ab700 1 Processor -- start 2024-09-10T11:40:07.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.088+0000 7fc6267ab700 1 -- start start 2024-09-10T11:40:07.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.089+0000 7fc6267ab700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6200ff270 0x7fc620195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.089+0000 7fc6267ab700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6200ffb80 0x7fc620196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.089+0000 7fc6267ab700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc620196a20 con 0x7fc6200ff270 2024-09-10T11:40:07.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.089+0000 7fc6267ab700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62019a7b0 con 0x7fc6200ffb80 2024-09-10T11:40:07.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.089+0000 7fc61ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6200ff270 0x7fc620195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.089+0000 7fc61ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6200ff270 0x7fc620195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50750/0 (socket says 172.21.15.144:50750) 2024-09-10T11:40:07.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.089+0000 7fc61ffff700 1 -- 172.21.15.144:0/2730124933 learned_addr learned my addr 172.21.15.144:0/2730124933 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:40:07.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.089+0000 7fc61f7fe700 1 --2- 172.21.15.144:0/2730124933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6200ffb80 0x7fc620196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.089+0000 7fc61ffff700 1 -- 172.21.15.144:0/2730124933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6200ffb80 msgr2=0x7fc620196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.089+0000 7fc61ffff700 1 --2- 172.21.15.144:0/2730124933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6200ffb80 0x7fc620196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.089+0000 7fc61ffff700 1 -- 172.21.15.144:0/2730124933 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6100095f0 con 0x7fc6200ff270 2024-09-10T11:40:07.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.089+0000 7fc61ffff700 1 --2- 172.21.15.144:0/2730124933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6200ff270 0x7fc620195e00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc610006010 tx=0x7fc61000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:07.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.090+0000 7fc61d7fa700 1 -- 172.21.15.144:0/2730124933 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc61001d070 con 0x7fc6200ff270 2024-09-10T11:40:07.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.090+0000 7fc61d7fa700 1 -- 172.21.15.144:0/2730124933 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc61000f460 con 0x7fc6200ff270 2024-09-10T11:40:07.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.090+0000 7fc61d7fa700 1 -- 172.21.15.144:0/2730124933 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc610017610 con 0x7fc6200ff270 2024-09-10T11:40:07.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.090+0000 7fc6267ab700 1 -- 172.21.15.144:0/2730124933 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc62019aa30 con 0x7fc6200ff270 2024-09-10T11:40:07.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.090+0000 7fc6267ab700 1 -- 172.21.15.144:0/2730124933 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62019afa0 con 0x7fc6200ff270 2024-09-10T11:40:07.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.091+0000 7fc6267ab700 1 -- 172.21.15.144:0/2730124933 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc620107cf0 con 0x7fc6200ff270 2024-09-10T11:40:07.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.092+0000 7fc61d7fa700 1 -- 172.21.15.144:0/2730124933 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc61000f5d0 con 0x7fc6200ff270 2024-09-10T11:40:07.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.092+0000 7fc61d7fa700 1 --2- 172.21.15.144:0/2730124933 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc60806e200 0x7fc6080706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.092+0000 7fc61d7fa700 1 -- 172.21.15.144:0/2730124933 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc61008f610 con 0x7fc6200ff270 2024-09-10T11:40:07.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.092+0000 7fc61f7fe700 1 --2- 172.21.15.144:0/2730124933 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc60806e200 0x7fc6080706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.093+0000 7fc61f7fe700 1 --2- 172.21.15.144:0/2730124933 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc60806e200 0x7fc6080706b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc620197420 tx=0x7fc614009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:07.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.097+0000 7fc61d7fa700 1 -- 172.21.15.144:0/2730124933 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc6100589f0 con 0x7fc6200ff270 2024-09-10T11:40:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:06 smithi178 ceph-mon[65940]: pgmap v243: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:07.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.304+0000 7fc6267ab700 1 -- 172.21.15.144:0/2730124933 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc62019b280 con 0x7fc60806e200 2024-09-10T11:40:07.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.313+0000 7fc61d7fa700 1 -- 172.21.15.144:0/2730124933 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc62019b280 con 0x7fc60806e200 2024-09-10T11:40:07.315 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:40:07.315 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (8m) 2m ago 18m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:40:07.315 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (19m) 2m ago 19m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:40:07.315 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (18m) 109s ago 18m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:40:07.315 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (6m) 2m ago 19m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:40:07.315 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (5m) 109s ago 18m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:40:07.315 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (7m) 2m ago 18m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:40:07.315 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (2m) 2m ago 13m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:40:07.315 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (2m) 2m ago 13m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:40:07.315 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (13m) 109s ago 13m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:40:07.316 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (113s) 109s ago 13m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:40:07.316 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (10m) 2m ago 20m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:40:07.316 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (9m) 109s ago 17m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:40:07.316 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (7m) 2m ago 20m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:40:07.316 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (6m) 109s ago 17m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:40:07.316 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (8m) 2m ago 19m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:40:07.316 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (8m) 109s ago 18m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:40:07.316 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (5m) 2m ago 17m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:40:07.316 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (5m) 2m ago 16m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:40:07.316 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (4m) 2m ago 16m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:40:07.316 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (3m) 109s ago 15m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:40:07.316 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (3m) 109s ago 15m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:40:07.316 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (2m) 109s ago 14m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:40:07.316 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (8m) 2m ago 18m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:40:07.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.315+0000 7fc6267ab700 1 -- 172.21.15.144:0/2730124933 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc60806e200 msgr2=0x7fc6080706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.315+0000 7fc6267ab700 1 --2- 172.21.15.144:0/2730124933 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc60806e200 0x7fc6080706b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc620197420 tx=0x7fc614009190 comp rx=0 tx=0).stop 2024-09-10T11:40:07.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.315+0000 7fc6267ab700 1 -- 172.21.15.144:0/2730124933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6200ff270 msgr2=0x7fc620195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.315+0000 7fc6267ab700 1 --2- 172.21.15.144:0/2730124933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6200ff270 0x7fc620195e00 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc610006010 tx=0x7fc61000bd70 comp rx=0 tx=0).stop 2024-09-10T11:40:07.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.316+0000 7fc6267ab700 1 -- 172.21.15.144:0/2730124933 shutdown_connections 2024-09-10T11:40:07.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.316+0000 7fc6267ab700 1 --2- 172.21.15.144:0/2730124933 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc60806e200 0x7fc6080706b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.316+0000 7fc6267ab700 1 --2- 172.21.15.144:0/2730124933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc6200ff270 0x7fc620195e00 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.316+0000 7fc6267ab700 1 --2- 172.21.15.144:0/2730124933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc6200ffb80 0x7fc620196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.316+0000 7fc6267ab700 1 -- 172.21.15.144:0/2730124933 >> 172.21.15.144:0/2730124933 conn(0x7fc620074850 msgr2=0x7fc620104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:07.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.316+0000 7fc6267ab700 1 -- 172.21.15.144:0/2730124933 shutdown_connections 2024-09-10T11:40:07.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.316+0000 7fc6267ab700 1 -- 172.21.15.144:0/2730124933 wait complete. 2024-09-10T11:40:07.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.421+0000 7ffa50d22700 1 -- 172.21.15.144:0/464871133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1003e0 msgr2=0x7ffa4c1007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.421+0000 7ffa50d22700 1 --2- 172.21.15.144:0/464871133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1003e0 0x7ffa4c1007b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c009910 tx=0x7ffa3c009c20 comp rx=0 tx=0).stop 2024-09-10T11:40:07.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.421+0000 7ffa50d22700 1 -- 172.21.15.144:0/464871133 shutdown_connections 2024-09-10T11:40:07.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.421+0000 7ffa50d22700 1 --2- 172.21.15.144:0/464871133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa4c100d80 0x7ffa4c104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.421+0000 7ffa50d22700 1 --2- 172.21.15.144:0/464871133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1003e0 0x7ffa4c1007b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.421+0000 7ffa50d22700 1 -- 172.21.15.144:0/464871133 >> 172.21.15.144:0/464871133 conn(0x7ffa4c0fbca0 msgr2=0x7ffa4c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:07.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.421+0000 7ffa50d22700 1 -- 172.21.15.144:0/464871133 shutdown_connections 2024-09-10T11:40:07.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.421+0000 7ffa50d22700 1 -- 172.21.15.144:0/464871133 wait complete. 2024-09-10T11:40:07.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.422+0000 7ffa50d22700 1 Processor -- start 2024-09-10T11:40:07.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.422+0000 7ffa50d22700 1 -- start start 2024-09-10T11:40:07.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.422+0000 7ffa50d22700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1003e0 0x7ffa4c195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.422+0000 7ffa50d22700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa4c100d80 0x7ffa4c1962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.422+0000 7ffa50d22700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c1969a0 con 0x7ffa4c1003e0 2024-09-10T11:40:07.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.422+0000 7ffa50d22700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c19a730 con 0x7ffa4c100d80 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.422+0000 7ffa4a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1003e0 0x7ffa4c195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.422+0000 7ffa4a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1003e0 0x7ffa4c195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50756/0 (socket says 172.21.15.144:50756) 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.422+0000 7ffa4a59c700 1 -- 172.21.15.144:0/3165454944 learned_addr learned my addr 172.21.15.144:0/3165454944 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.422+0000 7ffa41d9b700 1 --2- 172.21.15.144:0/3165454944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa4c100d80 0x7ffa4c1962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.422+0000 7ffa4a59c700 1 -- 172.21.15.144:0/3165454944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa4c100d80 msgr2=0x7ffa4c1962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.422+0000 7ffa4a59c700 1 --2- 172.21.15.144:0/3165454944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa4c100d80 0x7ffa4c1962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.422+0000 7ffa4a59c700 1 -- 172.21.15.144:0/3165454944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3c0095f0 con 0x7ffa4c1003e0 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.423+0000 7ffa4a59c700 1 --2- 172.21.15.144:0/3165454944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1003e0 0x7ffa4c195d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0092e0 tx=0x7ffa3c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.423+0000 7ffa43fff700 1 -- 172.21.15.144:0/3165454944 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa3c01d070 con 0x7ffa4c1003e0 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.423+0000 7ffa43fff700 1 -- 172.21.15.144:0/3165454944 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa3c005e00 con 0x7ffa4c1003e0 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.423+0000 7ffa43fff700 1 -- 172.21.15.144:0/3165454944 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa3c017400 con 0x7ffa4c1003e0 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.423+0000 7ffa50d22700 1 -- 172.21.15.144:0/3165454944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c19a9b0 con 0x7ffa4c1003e0 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.423+0000 7ffa50d22700 1 -- 172.21.15.144:0/3165454944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c19aea0 con 0x7ffa4c1003e0 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.424+0000 7ffa50d22700 1 -- 172.21.15.144:0/3165454944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c1086f0 con 0x7ffa4c1003e0 2024-09-10T11:40:07.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.424+0000 7ffa43fff700 1 -- 172.21.15.144:0/3165454944 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ffa3c00f460 con 0x7ffa4c1003e0 2024-09-10T11:40:07.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.425+0000 7ffa43fff700 1 --2- 172.21.15.144:0/3165454944 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffa3806e200 0x7ffa380706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.425+0000 7ffa43fff700 1 -- 172.21.15.144:0/3165454944 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ffa3c08f670 con 0x7ffa4c1003e0 2024-09-10T11:40:07.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.425+0000 7ffa41d9b700 1 --2- 172.21.15.144:0/3165454944 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffa3806e200 0x7ffa380706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.426+0000 7ffa41d9b700 1 --2- 172.21.15.144:0/3165454944 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffa3806e200 0x7ffa380706b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffa34005950 tx=0x7ffa340058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:07.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.430+0000 7ffa43fff700 1 -- 172.21.15.144:0/3165454944 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ffa3c0589b0 con 0x7ffa4c1003e0 2024-09-10T11:40:07.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.718+0000 7ffa50d22700 1 -- 172.21.15.144:0/3165454944 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffa4c02ccf0 con 0x7ffa4c1003e0 2024-09-10T11:40:07.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.718+0000 7ffa43fff700 1 -- 172.21.15.144:0/3165454944 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7ffa3c05c070 con 0x7ffa4c1003e0 2024-09-10T11:40:07.720 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:40:07.720 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:40:07.720 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:40:07.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:40:07.720 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:40:07.720 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:40:07.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:40:07.720 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:40:07.720 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:40:07.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:40:07.720 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:40:07.721 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:40:07.721 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:40:07.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:40:07.721 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:40:07.721 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:40:07.721 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:40:07.721 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:40:07.721 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:40:07.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.721+0000 7ffa50d22700 1 -- 172.21.15.144:0/3165454944 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffa3806e200 msgr2=0x7ffa380706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.721+0000 7ffa50d22700 1 --2- 172.21.15.144:0/3165454944 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffa3806e200 0x7ffa380706b0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ffa34005950 tx=0x7ffa340058e0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.721+0000 7ffa50d22700 1 -- 172.21.15.144:0/3165454944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1003e0 msgr2=0x7ffa4c195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.721+0000 7ffa50d22700 1 --2- 172.21.15.144:0/3165454944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1003e0 0x7ffa4c195d80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c0092e0 tx=0x7ffa3c00be20 comp rx=0 tx=0).stop 2024-09-10T11:40:07.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.721+0000 7ffa50d22700 1 -- 172.21.15.144:0/3165454944 shutdown_connections 2024-09-10T11:40:07.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.721+0000 7ffa50d22700 1 --2- 172.21.15.144:0/3165454944 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffa3806e200 0x7ffa380706b0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.721+0000 7ffa50d22700 1 --2- 172.21.15.144:0/3165454944 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffa4c1003e0 0x7ffa4c195d80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.721+0000 7ffa50d22700 1 --2- 172.21.15.144:0/3165454944 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffa4c100d80 0x7ffa4c1962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.721+0000 7ffa50d22700 1 -- 172.21.15.144:0/3165454944 >> 172.21.15.144:0/3165454944 conn(0x7ffa4c0fbca0 msgr2=0x7ffa4c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:07.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.721+0000 7ffa50d22700 1 -- 172.21.15.144:0/3165454944 shutdown_connections 2024-09-10T11:40:07.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.721+0000 7ffa50d22700 1 -- 172.21.15.144:0/3165454944 wait complete. 2024-09-10T11:40:07.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.827+0000 7f98acc9e700 1 -- 172.21.15.144:0/1863320499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98a80ffc30 msgr2=0x7f98a810a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.827+0000 7f98acc9e700 1 --2- 172.21.15.144:0/1863320499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98a80ffc30 0x7f98a810a6a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f9890009940 tx=0x7f9890009c50 comp rx=0 tx=0).stop 2024-09-10T11:40:07.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.827+0000 7f98acc9e700 1 -- 172.21.15.144:0/1863320499 shutdown_connections 2024-09-10T11:40:07.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.827+0000 7f98acc9e700 1 --2- 172.21.15.144:0/1863320499 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98a80ffc30 0x7f98a810a6a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.827+0000 7f98acc9e700 1 --2- 172.21.15.144:0/1863320499 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a80ff320 0x7f98a80ff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.827+0000 7f98acc9e700 1 -- 172.21.15.144:0/1863320499 >> 172.21.15.144:0/1863320499 conn(0x7f98a80748b0 msgr2=0x7f98a8074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:07.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.827+0000 7f98acc9e700 1 -- 172.21.15.144:0/1863320499 shutdown_connections 2024-09-10T11:40:07.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.827+0000 7f98acc9e700 1 -- 172.21.15.144:0/1863320499 wait complete. 2024-09-10T11:40:07.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.828+0000 7f98acc9e700 1 Processor -- start 2024-09-10T11:40:07.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.828+0000 7f98acc9e700 1 -- start start 2024-09-10T11:40:07.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.828+0000 7f98acc9e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98a80ff320 0x7f98a819ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.828+0000 7f98acc9e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a80ffc30 0x7f98a819bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.828+0000 7f98acc9e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a819c5e0 con 0x7f98a80ff320 2024-09-10T11:40:07.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.828+0000 7f98acc9e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a8195a90 con 0x7f98a80ffc30 2024-09-10T11:40:07.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.828+0000 7f98a659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98a80ff320 0x7f98a819ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.828+0000 7f98a659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98a80ff320 0x7f98a819ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50780/0 (socket says 172.21.15.144:50780) 2024-09-10T11:40:07.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.828+0000 7f98a659c700 1 -- 172.21.15.144:0/1447803540 learned_addr learned my addr 172.21.15.144:0/1447803540 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:40:07.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.829+0000 7f98a5d9b700 1 --2- 172.21.15.144:0/1447803540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a80ffc30 0x7f98a819bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.829+0000 7f98a659c700 1 -- 172.21.15.144:0/1447803540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a80ffc30 msgr2=0x7f98a819bf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:07.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.829+0000 7f98a659c700 1 --2- 172.21.15.144:0/1447803540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a80ffc30 0x7f98a819bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:07.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.829+0000 7f98a659c700 1 -- 172.21.15.144:0/1447803540 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98900095f0 con 0x7f98a80ff320 2024-09-10T11:40:07.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.829+0000 7f98a659c700 1 --2- 172.21.15.144:0/1447803540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98a80ff320 0x7f98a819ba10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f98a000ef10 tx=0x7f98a000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:07.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.829+0000 7f989f7fe700 1 -- 172.21.15.144:0/1447803540 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98a000ca50 con 0x7f98a80ff320 2024-09-10T11:40:07.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.829+0000 7f989f7fe700 1 -- 172.21.15.144:0/1447803540 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98a000f040 con 0x7f98a80ff320 2024-09-10T11:40:07.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.829+0000 7f98acc9e700 1 -- 172.21.15.144:0/1447803540 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a8195d10 con 0x7f98a80ff320 2024-09-10T11:40:07.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.829+0000 7f989f7fe700 1 -- 172.21.15.144:0/1447803540 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98a00146b0 con 0x7f98a80ff320 2024-09-10T11:40:07.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.829+0000 7f98acc9e700 1 -- 172.21.15.144:0/1447803540 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a8196260 con 0x7f98a80ff320 2024-09-10T11:40:07.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.830+0000 7f98acc9e700 1 -- 172.21.15.144:0/1447803540 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a8107da0 con 0x7f98a80ff320 2024-09-10T11:40:07.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.831+0000 7f989f7fe700 1 -- 172.21.15.144:0/1447803540 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f98a000cbb0 con 0x7f98a80ff320 2024-09-10T11:40:07.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.831+0000 7f989f7fe700 1 --2- 172.21.15.144:0/1447803540 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f989406e200 0x7f98940706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:07.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.831+0000 7f98a5d9b700 1 --2- 172.21.15.144:0/1447803540 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f989406e200 0x7f98940706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:07.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.831+0000 7f989f7fe700 1 -- 172.21.15.144:0/1447803540 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f98a008e490 con 0x7f98a80ff320 2024-09-10T11:40:07.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.832+0000 7f98a5d9b700 1 --2- 172.21.15.144:0/1447803540 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f989406e200 0x7f98940706b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f98a80fdff0 tx=0x7f9890005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:07.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:07.836+0000 7f989f7fe700 1 -- 172.21.15.144:0/1447803540 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f98a005b3a0 con 0x7f98a80ff320 2024-09-10T11:40:08.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.078+0000 7f98acc9e700 1 -- 172.21.15.144:0/1447803540 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f98a802ccf0 con 0x7f98a80ff320 2024-09-10T11:40:08.080 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:07 smithi144 ceph-mon[87436]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:08.080 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:07 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3165454944' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:40:08.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.078+0000 7f989f7fe700 1 -- 172.21.15.144:0/1447803540 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f98a005af30 con 0x7f98a80ff320 2024-09-10T11:40:08.081 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:40:08.081 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:40:08.081 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:40:08.081 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:40:08.081 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:40:08.081 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:40:08.081 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:40:08.081 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:40:08.081 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:40:08.082 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:40:08.082 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:40:08.082 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:40:08.082 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:40:08.082 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:40:08.082 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:40:08.082 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:40:08.082 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:40:08.082 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:40:08.082 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:40:08.082 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:40:08.082 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:40:08.082 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:40:08.082 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:40:08.083 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:40:08.083 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:40:08.083 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:40:08.083 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:40:08.083 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:40:08.083 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:40:08.083 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:40:08.083 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:40:08.083 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:40:08.083 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:40:08.083 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:40:08.083 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:40:08.083 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:40:08.083 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:40:08.083 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:40:08.084 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:40:08.084 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:40:08.084 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:40:08.084 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:40:08.084 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:40:08.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.082+0000 7f98acc9e700 1 -- 172.21.15.144:0/1447803540 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f989406e200 msgr2=0x7f98940706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.082+0000 7f98acc9e700 1 --2- 172.21.15.144:0/1447803540 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f989406e200 0x7f98940706b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f98a80fdff0 tx=0x7f9890005fd0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.082+0000 7f98acc9e700 1 -- 172.21.15.144:0/1447803540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98a80ff320 msgr2=0x7f98a819ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.082+0000 7f98acc9e700 1 --2- 172.21.15.144:0/1447803540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98a80ff320 0x7f98a819ba10 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f98a000ef10 tx=0x7f98a000c3a0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.082+0000 7f98acc9e700 1 -- 172.21.15.144:0/1447803540 shutdown_connections 2024-09-10T11:40:08.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.082+0000 7f98acc9e700 1 --2- 172.21.15.144:0/1447803540 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f989406e200 0x7f98940706b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.082+0000 7f98acc9e700 1 --2- 172.21.15.144:0/1447803540 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98a80ff320 0x7f98a819ba10 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.082+0000 7f98acc9e700 1 --2- 172.21.15.144:0/1447803540 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98a80ffc30 0x7f98a819bf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.082+0000 7f98acc9e700 1 -- 172.21.15.144:0/1447803540 >> 172.21.15.144:0/1447803540 conn(0x7f98a80748b0 msgr2=0x7f98a8104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:08.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.082+0000 7f98acc9e700 1 -- 172.21.15.144:0/1447803540 shutdown_connections 2024-09-10T11:40:08.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.082+0000 7f98acc9e700 1 -- 172.21.15.144:0/1447803540 wait complete. 2024-09-10T11:40:08.086 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:40:08.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:07 smithi178 ceph-mon[65940]: from='client.34512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:08.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:07 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3165454944' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:40:08.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.190+0000 7f7b0b194700 1 -- 172.21.15.144:0/924346284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b041003e0 msgr2=0x7f7b041007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.190+0000 7f7b0b194700 1 --2- 172.21.15.144:0/924346284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b041003e0 0x7f7b041007b0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7af8009910 tx=0x7f7af8009c20 comp rx=0 tx=0).stop 2024-09-10T11:40:08.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.190+0000 7f7b0b194700 1 -- 172.21.15.144:0/924346284 shutdown_connections 2024-09-10T11:40:08.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.190+0000 7f7b0b194700 1 --2- 172.21.15.144:0/924346284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b04100d80 0x7f7b04104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.190+0000 7f7b0b194700 1 --2- 172.21.15.144:0/924346284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b041003e0 0x7f7b041007b0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.190+0000 7f7b0b194700 1 -- 172.21.15.144:0/924346284 >> 172.21.15.144:0/924346284 conn(0x7f7b040fbca0 msgr2=0x7f7b040fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:08.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.190+0000 7f7b0b194700 1 -- 172.21.15.144:0/924346284 shutdown_connections 2024-09-10T11:40:08.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.190+0000 7f7b0b194700 1 -- 172.21.15.144:0/924346284 wait complete. 2024-09-10T11:40:08.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.191+0000 7f7b0b194700 1 Processor -- start 2024-09-10T11:40:08.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.191+0000 7f7b0b194700 1 -- start start 2024-09-10T11:40:08.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.191+0000 7f7b0b194700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b041003e0 0x7f7b04195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:08.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.191+0000 7f7b0b194700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b04100d80 0x7f7b04196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:08.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.191+0000 7f7b0b194700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b04196940 con 0x7f7b041003e0 2024-09-10T11:40:08.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.191+0000 7f7b0b194700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0419a6d0 con 0x7f7b04100d80 2024-09-10T11:40:08.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.191+0000 7f7b08f30700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b041003e0 0x7f7b04195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:08.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.191+0000 7f7b08f30700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b041003e0 0x7f7b04195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50802/0 (socket says 172.21.15.144:50802) 2024-09-10T11:40:08.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.191+0000 7f7b08f30700 1 -- 172.21.15.144:0/494761010 learned_addr learned my addr 172.21.15.144:0/494761010 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:40:08.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.191+0000 7f7b03fff700 1 --2- 172.21.15.144:0/494761010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b04100d80 0x7f7b04196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:08.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.191+0000 7f7b08f30700 1 -- 172.21.15.144:0/494761010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b04100d80 msgr2=0x7f7b04196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.191+0000 7f7b08f30700 1 --2- 172.21.15.144:0/494761010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b04100d80 0x7f7b04196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.191+0000 7f7b08f30700 1 -- 172.21.15.144:0/494761010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af80095f0 con 0x7f7b041003e0 2024-09-10T11:40:08.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.191+0000 7f7b08f30700 1 --2- 172.21.15.144:0/494761010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b041003e0 0x7f7b04195d20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7af8005f50 tx=0x7f7af800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:08.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.192+0000 7f7b01ffb700 1 -- 172.21.15.144:0/494761010 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7af801d070 con 0x7f7b041003e0 2024-09-10T11:40:08.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.192+0000 7f7b0b194700 1 -- 172.21.15.144:0/494761010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b0419a950 con 0x7f7b041003e0 2024-09-10T11:40:08.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.192+0000 7f7b01ffb700 1 -- 172.21.15.144:0/494761010 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7af800b610 con 0x7f7b041003e0 2024-09-10T11:40:08.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.192+0000 7f7b01ffb700 1 -- 172.21.15.144:0/494761010 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7af8022470 con 0x7f7b041003e0 2024-09-10T11:40:08.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.192+0000 7f7b0b194700 1 -- 172.21.15.144:0/494761010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b0419ae40 con 0x7f7b041003e0 2024-09-10T11:40:08.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.193+0000 7f7b0b194700 1 -- 172.21.15.144:0/494761010 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b041086f0 con 0x7f7b041003e0 2024-09-10T11:40:08.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.193+0000 7f7b01ffb700 1 -- 172.21.15.144:0/494761010 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7af800fca0 con 0x7f7b041003e0 2024-09-10T11:40:08.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.194+0000 7f7b01ffb700 1 --2- 172.21.15.144:0/494761010 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7af406e200 0x7f7af40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:08.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.194+0000 7f7b01ffb700 1 -- 172.21.15.144:0/494761010 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7af808f540 con 0x7f7b041003e0 2024-09-10T11:40:08.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.194+0000 7f7b03fff700 1 --2- 172.21.15.144:0/494761010 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7af406e200 0x7f7af40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:08.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.194+0000 7f7b03fff700 1 --2- 172.21.15.144:0/494761010 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7af406e200 0x7f7af40706b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b04197340 tx=0x7f7af0009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:08.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.198+0000 7f7b01ffb700 1 -- 172.21.15.144:0/494761010 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7af8058920 con 0x7f7b041003e0 2024-09-10T11:40:08.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.418+0000 7f7b0b194700 1 -- 172.21.15.144:0/494761010 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b0402ccf0 con 0x7f7af406e200 2024-09-10T11:40:08.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.419+0000 7f7b01ffb700 1 -- 172.21.15.144:0/494761010 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7b0402ccf0 con 0x7f7af406e200 2024-09-10T11:40:08.421 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:40:08.421 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:40:08.421 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:40:08.421 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:40:08.421 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:40:08.421 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:40:08.421 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:40:08.421 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:40:08.421 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:40:08.421 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:40:08.422 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:40:08.422 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:40:08.422 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:40:08.422 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:40:08.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.423+0000 7f7b0b194700 1 -- 172.21.15.144:0/494761010 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7af406e200 msgr2=0x7f7af40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.423+0000 7f7b0b194700 1 --2- 172.21.15.144:0/494761010 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7af406e200 0x7f7af40706b0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7b04197340 tx=0x7f7af0009260 comp rx=0 tx=0).stop 2024-09-10T11:40:08.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.423+0000 7f7b0b194700 1 -- 172.21.15.144:0/494761010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b041003e0 msgr2=0x7f7b04195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.423+0000 7f7b0b194700 1 --2- 172.21.15.144:0/494761010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b041003e0 0x7f7b04195d20 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7af8005f50 tx=0x7f7af800f740 comp rx=0 tx=0).stop 2024-09-10T11:40:08.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.423+0000 7f7b0b194700 1 -- 172.21.15.144:0/494761010 shutdown_connections 2024-09-10T11:40:08.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.423+0000 7f7b0b194700 1 --2- 172.21.15.144:0/494761010 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7af406e200 0x7f7af40706b0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.423+0000 7f7b0b194700 1 --2- 172.21.15.144:0/494761010 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7b041003e0 0x7f7b04195d20 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.423+0000 7f7b0b194700 1 --2- 172.21.15.144:0/494761010 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7b04100d80 0x7f7b04196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.423+0000 7f7b0b194700 1 -- 172.21.15.144:0/494761010 >> 172.21.15.144:0/494761010 conn(0x7f7b040fbca0 msgr2=0x7f7b040fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:08.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.423+0000 7f7b0b194700 1 -- 172.21.15.144:0/494761010 shutdown_connections 2024-09-10T11:40:08.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.423+0000 7f7b0b194700 1 -- 172.21.15.144:0/494761010 wait complete. 2024-09-10T11:40:08.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.529+0000 7f0e1ac52700 1 -- 172.21.15.144:0/2643797938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e140ffc70 msgr2=0x7f0e14103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.529+0000 7f0e1ac52700 1 --2- 172.21.15.144:0/2643797938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e140ffc70 0x7f0e14103c40 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f0e00009910 tx=0x7f0e00009c20 comp rx=0 tx=0).stop 2024-09-10T11:40:08.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.529+0000 7f0e1ac52700 1 -- 172.21.15.144:0/2643797938 shutdown_connections 2024-09-10T11:40:08.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.529+0000 7f0e1ac52700 1 --2- 172.21.15.144:0/2643797938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e140ffc70 0x7f0e14103c40 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.529+0000 7f0e1ac52700 1 --2- 172.21.15.144:0/2643797938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e140ff2d0 0x7f0e140ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.529+0000 7f0e1ac52700 1 -- 172.21.15.144:0/2643797938 >> 172.21.15.144:0/2643797938 conn(0x7f0e140fabb0 msgr2=0x7f0e140fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:08.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.529+0000 7f0e1ac52700 1 -- 172.21.15.144:0/2643797938 shutdown_connections 2024-09-10T11:40:08.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.529+0000 7f0e1ac52700 1 -- 172.21.15.144:0/2643797938 wait complete. 2024-09-10T11:40:08.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.529+0000 7f0e1ac52700 1 Processor -- start 2024-09-10T11:40:08.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.529+0000 7f0e1ac52700 1 -- start start 2024-09-10T11:40:08.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.530+0000 7f0e1ac52700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e140ff2d0 0x7f0e14109fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:08.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.530+0000 7f0e1ac52700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e140ffc70 0x7f0e1410a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:08.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.530+0000 7f0e1ac52700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1410aa30 con 0x7f0e140ffc70 2024-09-10T11:40:08.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.530+0000 7f0e1ac52700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1410ab70 con 0x7f0e140ff2d0 2024-09-10T11:40:08.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.530+0000 7f0e13fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e140ffc70 0x7f0e1410a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:08.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.530+0000 7f0e13fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e140ffc70 0x7f0e1410a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50824/0 (socket says 172.21.15.144:50824) 2024-09-10T11:40:08.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.530+0000 7f0e13fff700 1 -- 172.21.15.144:0/1061369135 learned_addr learned my addr 172.21.15.144:0/1061369135 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:40:08.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.530+0000 7f0e189ee700 1 --2- 172.21.15.144:0/1061369135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e140ff2d0 0x7f0e14109fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:08.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.530+0000 7f0e13fff700 1 -- 172.21.15.144:0/1061369135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e140ff2d0 msgr2=0x7f0e14109fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.530+0000 7f0e13fff700 1 --2- 172.21.15.144:0/1061369135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e140ff2d0 0x7f0e14109fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.530+0000 7f0e13fff700 1 -- 172.21.15.144:0/1061369135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e000095f0 con 0x7f0e140ffc70 2024-09-10T11:40:08.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.530+0000 7f0e189ee700 1 --2- 172.21.15.144:0/1061369135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e140ff2d0 0x7f0e14109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:08.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.530+0000 7f0e13fff700 1 --2- 172.21.15.144:0/1061369135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e140ffc70 0x7f0e1410a4f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000bd70 tx=0x7f0e0000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:08.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.530+0000 7f0e11ffb700 1 -- 172.21.15.144:0/1061369135 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0001d070 con 0x7f0e140ffc70 2024-09-10T11:40:08.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.530+0000 7f0e1ac52700 1 -- 172.21.15.144:0/1061369135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1410adf0 con 0x7f0e140ffc70 2024-09-10T11:40:08.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.531+0000 7f0e11ffb700 1 -- 172.21.15.144:0/1061369135 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e00005dc0 con 0x7f0e140ffc70 2024-09-10T11:40:08.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.531+0000 7f0e11ffb700 1 -- 172.21.15.144:0/1061369135 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e00017400 con 0x7f0e140ffc70 2024-09-10T11:40:08.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.531+0000 7f0e1ac52700 1 -- 172.21.15.144:0/1061369135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e141a2290 con 0x7f0e140ffc70 2024-09-10T11:40:08.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.532+0000 7f0e1ac52700 1 -- 172.21.15.144:0/1061369135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e14107640 con 0x7f0e140ffc70 2024-09-10T11:40:08.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.532+0000 7f0e11ffb700 1 -- 172.21.15.144:0/1061369135 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0e0000f7f0 con 0x7f0e140ffc70 2024-09-10T11:40:08.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.533+0000 7f0e11ffb700 1 --2- 172.21.15.144:0/1061369135 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0e0406e200 0x7f0e040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:08.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.533+0000 7f0e189ee700 1 --2- 172.21.15.144:0/1061369135 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0e0406e200 0x7f0e040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:08.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.533+0000 7f0e11ffb700 1 -- 172.21.15.144:0/1061369135 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0e0008f590 con 0x7f0e140ffc70 2024-09-10T11:40:08.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.533+0000 7f0e189ee700 1 --2- 172.21.15.144:0/1061369135 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0e0406e200 0x7f0e040706b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0e08005950 tx=0x7f0e080058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:08.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.537+0000 7f0e11ffb700 1 -- 172.21.15.144:0/1061369135 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0e00058970 con 0x7f0e140ffc70 2024-09-10T11:40:08.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.823+0000 7f0e1ac52700 1 -- 172.21.15.144:0/1061369135 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0e14107020 con 0x7f0e140ffc70 2024-09-10T11:40:08.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.824+0000 7f0e11ffb700 1 -- 172.21.15.144:0/1061369135 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f0e0005c030 con 0x7f0e140ffc70 2024-09-10T11:40:08.825 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:40:08.825 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:40:08.825 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:40:08.825 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:40:08.825 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:40:08.825 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:40:08.825 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:40:08.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.826+0000 7f0e1ac52700 1 -- 172.21.15.144:0/1061369135 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0e0406e200 msgr2=0x7f0e040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.826+0000 7f0e1ac52700 1 --2- 172.21.15.144:0/1061369135 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0e0406e200 0x7f0e040706b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f0e08005950 tx=0x7f0e080058e0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.826+0000 7f0e1ac52700 1 -- 172.21.15.144:0/1061369135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e140ffc70 msgr2=0x7f0e1410a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:08.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.826+0000 7f0e1ac52700 1 --2- 172.21.15.144:0/1061369135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e140ffc70 0x7f0e1410a4f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000bd70 tx=0x7f0e0000be50 comp rx=0 tx=0).stop 2024-09-10T11:40:08.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.826+0000 7f0e1ac52700 1 -- 172.21.15.144:0/1061369135 shutdown_connections 2024-09-10T11:40:08.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.826+0000 7f0e1ac52700 1 --2- 172.21.15.144:0/1061369135 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0e0406e200 0x7f0e040706b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.826+0000 7f0e1ac52700 1 --2- 172.21.15.144:0/1061369135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e140ff2d0 0x7f0e14109fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.826+0000 7f0e1ac52700 1 --2- 172.21.15.144:0/1061369135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e140ffc70 0x7f0e1410a4f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:08.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.826+0000 7f0e1ac52700 1 -- 172.21.15.144:0/1061369135 >> 172.21.15.144:0/1061369135 conn(0x7f0e140fabb0 msgr2=0x7f0e140fc150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:08.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.826+0000 7f0e1ac52700 1 -- 172.21.15.144:0/1061369135 shutdown_connections 2024-09-10T11:40:08.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:08.826+0000 7f0e1ac52700 1 -- 172.21.15.144:0/1061369135 wait complete. 2024-09-10T11:40:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:08 smithi178 ceph-mon[65940]: from='client.34516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:08 smithi178 ceph-mon[65940]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:08 smithi178 ceph-mon[65940]: pgmap v244: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:08 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1447803540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:40:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:08 smithi178 ceph-mon[65940]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:08 smithi144 ceph-mon[87436]: from='client.34516 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:08 smithi144 ceph-mon[87436]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:08 smithi144 ceph-mon[87436]: pgmap v244: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:08 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1447803540' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:40:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:08 smithi144 ceph-mon[87436]: from='client.34532 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:10.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:09 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1061369135' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:40:10.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:09 smithi178 ceph-mon[65940]: pgmap v245: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:10.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:09 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1061369135' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:40:10.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:09 smithi144 ceph-mon[87436]: pgmap v245: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:12 smithi144 ceph-mon[87436]: pgmap v246: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:12 smithi178 ceph-mon[65940]: pgmap v246: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:40:14.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:40:15.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:14 smithi178 ceph-mon[65940]: pgmap v247: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:14 smithi144 ceph-mon[87436]: pgmap v247: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:16 smithi178 ceph-mon[65940]: pgmap v248: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:17.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:16 smithi144 ceph-mon[87436]: pgmap v248: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:18 smithi178 ceph-mon[65940]: pgmap v249: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:19.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:18 smithi144 ceph-mon[87436]: pgmap v249: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:20 smithi178 ceph-mon[65940]: pgmap v250: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:21.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:20 smithi144 ceph-mon[87436]: pgmap v250: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:23.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:22 smithi144 ceph-mon[87436]: pgmap v251: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:23.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:22 smithi178 ceph-mon[65940]: pgmap v251: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:24 smithi178 ceph-mon[65940]: pgmap v252: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:25.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:24 smithi144 ceph-mon[87436]: pgmap v252: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:26 smithi178 ceph-mon[65940]: pgmap v253: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:26 smithi144 ceph-mon[87436]: pgmap v253: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:28 smithi178 ceph-mon[65940]: pgmap v254: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:40:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:28 smithi144 ceph-mon[87436]: pgmap v254: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:40:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:30 smithi178 ceph-mon[65940]: pgmap v255: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:31.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:30 smithi144 ceph-mon[87436]: pgmap v255: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:32 smithi144 ceph-mon[87436]: pgmap v256: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:33.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:32 smithi178 ceph-mon[65940]: pgmap v256: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:34 smithi178 ceph-mon[65940]: pgmap v257: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:34 smithi144 ceph-mon[87436]: pgmap v257: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:36 smithi178 ceph-mon[65940]: pgmap v258: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:37.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:36 smithi144 ceph-mon[87436]: pgmap v258: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:38.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.944+0000 7f9f2ba69700 1 -- 172.21.15.144:0/3430061576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f241003d0 msgr2=0x7f9f241007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:38.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.944+0000 7f9f2ba69700 1 --2- 172.21.15.144:0/3430061576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f241003d0 0x7f9f241007a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f9f14009910 tx=0x7f9f14009c20 comp rx=0 tx=0).stop 2024-09-10T11:40:38.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.944+0000 7f9f2ba69700 1 -- 172.21.15.144:0/3430061576 shutdown_connections 2024-09-10T11:40:38.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.944+0000 7f9f2ba69700 1 --2- 172.21.15.144:0/3430061576 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f24100d70 0x7f9f24104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:38.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.944+0000 7f9f2ba69700 1 --2- 172.21.15.144:0/3430061576 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f241003d0 0x7f9f241007a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:38.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.944+0000 7f9f2ba69700 1 -- 172.21.15.144:0/3430061576 >> 172.21.15.144:0/3430061576 conn(0x7f9f240fbc90 msgr2=0x7f9f240fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:38.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.944+0000 7f9f2ba69700 1 -- 172.21.15.144:0/3430061576 shutdown_connections 2024-09-10T11:40:38.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.944+0000 7f9f2ba69700 1 -- 172.21.15.144:0/3430061576 wait complete. 2024-09-10T11:40:38.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.945+0000 7f9f2ba69700 1 Processor -- start 2024-09-10T11:40:38.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.945+0000 7f9f2ba69700 1 -- start start 2024-09-10T11:40:38.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f2ba69700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f241003d0 0x7f9f24195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:38.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f2ba69700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f24100d70 0x7f9f24196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:38.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f2ba69700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f24196b40 con 0x7f9f24100d70 2024-09-10T11:40:38.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f2ba69700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2419a830 con 0x7f9f241003d0 2024-09-10T11:40:38.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f29004700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f24100d70 0x7f9f24196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:38.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f29004700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f24100d70 0x7f9f24196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34588/0 (socket says 172.21.15.144:34588) 2024-09-10T11:40:38.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f29004700 1 -- 172.21.15.144:0/2084401128 learned_addr learned my addr 172.21.15.144:0/2084401128 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:40:38.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f29805700 1 --2- 172.21.15.144:0/2084401128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f241003d0 0x7f9f24195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:38.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f29004700 1 -- 172.21.15.144:0/2084401128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f241003d0 msgr2=0x7f9f24195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:38.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f29004700 1 --2- 172.21.15.144:0/2084401128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f241003d0 0x7f9f24195f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:38.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f29004700 1 -- 172.21.15.144:0/2084401128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f140095f0 con 0x7f9f24100d70 2024-09-10T11:40:38.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f29805700 1 --2- 172.21.15.144:0/2084401128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f241003d0 0x7f9f24195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:40:38.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f29004700 1 --2- 172.21.15.144:0/2084401128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f24100d70 0x7f9f24196460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000b480 tx=0x7f9f2000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:38.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f1affd700 1 -- 172.21.15.144:0/2084401128 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f200118c0 con 0x7f9f24100d70 2024-09-10T11:40:38.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f2ba69700 1 -- 172.21.15.144:0/2084401128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2419ab10 con 0x7f9f24100d70 2024-09-10T11:40:38.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f1affd700 1 -- 172.21.15.144:0/2084401128 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f20010380 con 0x7f9f24100d70 2024-09-10T11:40:38.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.946+0000 7f9f1affd700 1 -- 172.21.15.144:0/2084401128 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f2000f4f0 con 0x7f9f24100d70 2024-09-10T11:40:38.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.947+0000 7f9f2ba69700 1 -- 172.21.15.144:0/2084401128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f2419b060 con 0x7f9f24100d70 2024-09-10T11:40:38.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.947+0000 7f9f2ba69700 1 -- 172.21.15.144:0/2084401128 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f241086e0 con 0x7f9f24100d70 2024-09-10T11:40:38.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.948+0000 7f9f1affd700 1 -- 172.21.15.144:0/2084401128 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9f200104f0 con 0x7f9f24100d70 2024-09-10T11:40:38.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.948+0000 7f9f1affd700 1 --2- 172.21.15.144:0/2084401128 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f1006e200 0x7f9f100706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:38.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.949+0000 7f9f1affd700 1 -- 172.21.15.144:0/2084401128 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9f2008dc20 con 0x7f9f24100d70 2024-09-10T11:40:38.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.949+0000 7f9f29805700 1 --2- 172.21.15.144:0/2084401128 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f1006e200 0x7f9f100706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:38.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.949+0000 7f9f29805700 1 --2- 172.21.15.144:0/2084401128 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f1006e200 0x7f9f100706b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f14005100 tx=0x7f9f1401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:38.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:38.953+0000 7f9f1affd700 1 -- 172.21.15.144:0/2084401128 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9f20057080 con 0x7f9f24100d70 2024-09-10T11:40:39.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:38 smithi178 ceph-mon[65940]: pgmap v259: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:39.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:38 smithi144 ceph-mon[87436]: pgmap v259: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:39.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.185+0000 7f9f2ba69700 1 -- 172.21.15.144:0/2084401128 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f2419b400 con 0x7f9f1006e200 2024-09-10T11:40:39.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.187+0000 7f9f1affd700 1 -- 172.21.15.144:0/2084401128 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9f2419b400 con 0x7f9f1006e200 2024-09-10T11:40:39.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.189+0000 7f9f2ba69700 1 -- 172.21.15.144:0/2084401128 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f1006e200 msgr2=0x7f9f100706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:39.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.189+0000 7f9f2ba69700 1 --2- 172.21.15.144:0/2084401128 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f1006e200 0x7f9f100706b0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f14005100 tx=0x7f9f1401a040 comp rx=0 tx=0).stop 2024-09-10T11:40:39.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.189+0000 7f9f2ba69700 1 -- 172.21.15.144:0/2084401128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f24100d70 msgr2=0x7f9f24196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:39.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.189+0000 7f9f2ba69700 1 --2- 172.21.15.144:0/2084401128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f24100d70 0x7f9f24196460 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f9f2000b480 tx=0x7f9f2000b840 comp rx=0 tx=0).stop 2024-09-10T11:40:39.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.189+0000 7f9f2ba69700 1 -- 172.21.15.144:0/2084401128 shutdown_connections 2024-09-10T11:40:39.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.189+0000 7f9f2ba69700 1 --2- 172.21.15.144:0/2084401128 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f1006e200 0x7f9f100706b0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.189+0000 7f9f2ba69700 1 --2- 172.21.15.144:0/2084401128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f241003d0 0x7f9f24195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.189+0000 7f9f2ba69700 1 --2- 172.21.15.144:0/2084401128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f24100d70 0x7f9f24196460 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.189+0000 7f9f2ba69700 1 -- 172.21.15.144:0/2084401128 >> 172.21.15.144:0/2084401128 conn(0x7f9f240fbc90 msgr2=0x7f9f240fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:39.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.189+0000 7f9f2ba69700 1 -- 172.21.15.144:0/2084401128 shutdown_connections 2024-09-10T11:40:39.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.189+0000 7f9f2ba69700 1 -- 172.21.15.144:0/2084401128 wait complete. 2024-09-10T11:40:39.201 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:40:39.298 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.295+0000 7f6691b5c700 1 -- 172.21.15.144:0/2766543135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f668c100d70 msgr2=0x7f668c104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:39.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.295+0000 7f6691b5c700 1 --2- 172.21.15.144:0/2766543135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f668c100d70 0x7f668c104d40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f6680009910 tx=0x7f6680009c20 comp rx=0 tx=0).stop 2024-09-10T11:40:39.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.296+0000 7f6691b5c700 1 -- 172.21.15.144:0/2766543135 shutdown_connections 2024-09-10T11:40:39.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.296+0000 7f6691b5c700 1 --2- 172.21.15.144:0/2766543135 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f668c100d70 0x7f668c104d40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.296+0000 7f6691b5c700 1 --2- 172.21.15.144:0/2766543135 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f668c1003d0 0x7f668c1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.296+0000 7f6691b5c700 1 -- 172.21.15.144:0/2766543135 >> 172.21.15.144:0/2766543135 conn(0x7f668c0fbc90 msgr2=0x7f668c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:39.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.296+0000 7f6691b5c700 1 -- 172.21.15.144:0/2766543135 shutdown_connections 2024-09-10T11:40:39.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.296+0000 7f6691b5c700 1 -- 172.21.15.144:0/2766543135 wait complete. 2024-09-10T11:40:39.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.296+0000 7f6691b5c700 1 Processor -- start 2024-09-10T11:40:39.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.296+0000 7f6691b5c700 1 -- start start 2024-09-10T11:40:39.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f6691b5c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f668c1003d0 0x7f668c195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:39.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f6691b5c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f668c100d70 0x7f668c196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:39.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f6691b5c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c196960 con 0x7f668c1003d0 2024-09-10T11:40:39.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f6691b5c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c19a6f0 con 0x7f668c100d70 2024-09-10T11:40:39.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f668b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f668c1003d0 0x7f668c195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:39.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f668b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f668c1003d0 0x7f668c195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34610/0 (socket says 172.21.15.144:34610) 2024-09-10T11:40:39.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f668b7fe700 1 -- 172.21.15.144:0/1714652205 learned_addr learned my addr 172.21.15.144:0/1714652205 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:40:39.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f668affd700 1 --2- 172.21.15.144:0/1714652205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f668c100d70 0x7f668c196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:39.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f668b7fe700 1 -- 172.21.15.144:0/1714652205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f668c100d70 msgr2=0x7f668c196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:39.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f668b7fe700 1 --2- 172.21.15.144:0/1714652205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f668c100d70 0x7f668c196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f668b7fe700 1 -- 172.21.15.144:0/1714652205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66800095f0 con 0x7f668c1003d0 2024-09-10T11:40:39.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f668b7fe700 1 --2- 172.21.15.144:0/1714652205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f668c1003d0 0x7f668c195d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f667c00c8a0 tx=0x7f667c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:39.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f6688ff9700 1 -- 172.21.15.144:0/1714652205 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f667c0078e0 con 0x7f668c1003d0 2024-09-10T11:40:39.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f6688ff9700 1 -- 172.21.15.144:0/1714652205 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f667c00f450 con 0x7f668c1003d0 2024-09-10T11:40:39.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f6691b5c700 1 -- 172.21.15.144:0/1714652205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668c19a9d0 con 0x7f668c1003d0 2024-09-10T11:40:39.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f6688ff9700 1 -- 172.21.15.144:0/1714652205 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f667c00e5e0 con 0x7f668c1003d0 2024-09-10T11:40:39.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.297+0000 7f6691b5c700 1 -- 172.21.15.144:0/1714652205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668c19af20 con 0x7f668c1003d0 2024-09-10T11:40:39.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.298+0000 7f6691b5c700 1 -- 172.21.15.144:0/1714652205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668c108740 con 0x7f668c1003d0 2024-09-10T11:40:39.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.299+0000 7f6688ff9700 1 -- 172.21.15.144:0/1714652205 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f667c00f5c0 con 0x7f668c1003d0 2024-09-10T11:40:39.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.299+0000 7f6688ff9700 1 --2- 172.21.15.144:0/1714652205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f667406e200 0x7f66740706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:39.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.299+0000 7f6688ff9700 1 -- 172.21.15.144:0/1714652205 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f667c08df10 con 0x7f668c1003d0 2024-09-10T11:40:39.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.299+0000 7f668affd700 1 --2- 172.21.15.144:0/1714652205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f667406e200 0x7f66740706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:39.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.300+0000 7f668affd700 1 --2- 172.21.15.144:0/1714652205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f667406e200 0x7f66740706b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f668c197360 tx=0x7f668001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:39.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.304+0000 7f6688ff9700 1 -- 172.21.15.144:0/1714652205 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f667c057370 con 0x7f668c1003d0 2024-09-10T11:40:39.527 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.526+0000 7f6691b5c700 1 -- 172.21.15.144:0/1714652205 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f668c19b300 con 0x7f667406e200 2024-09-10T11:40:39.528 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.527+0000 7f6688ff9700 1 -- 172.21.15.144:0/1714652205 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f668c19b300 con 0x7f667406e200 2024-09-10T11:40:39.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.529+0000 7f6691b5c700 1 -- 172.21.15.144:0/1714652205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f667406e200 msgr2=0x7f66740706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:39.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.529+0000 7f6691b5c700 1 --2- 172.21.15.144:0/1714652205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f667406e200 0x7f66740706b0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f668c197360 tx=0x7f668001a040 comp rx=0 tx=0).stop 2024-09-10T11:40:39.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.529+0000 7f6691b5c700 1 -- 172.21.15.144:0/1714652205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f668c1003d0 msgr2=0x7f668c195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:39.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.529+0000 7f6691b5c700 1 --2- 172.21.15.144:0/1714652205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f668c1003d0 0x7f668c195d40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f667c00c8a0 tx=0x7f667c00cbb0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.529+0000 7f6691b5c700 1 -- 172.21.15.144:0/1714652205 shutdown_connections 2024-09-10T11:40:39.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.529+0000 7f6691b5c700 1 --2- 172.21.15.144:0/1714652205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f667406e200 0x7f66740706b0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.529+0000 7f6691b5c700 1 --2- 172.21.15.144:0/1714652205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f668c1003d0 0x7f668c195d40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.529+0000 7f6691b5c700 1 --2- 172.21.15.144:0/1714652205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f668c100d70 0x7f668c196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.529+0000 7f6691b5c700 1 -- 172.21.15.144:0/1714652205 >> 172.21.15.144:0/1714652205 conn(0x7f668c0fbc90 msgr2=0x7f668c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:39.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.530+0000 7f6691b5c700 1 -- 172.21.15.144:0/1714652205 shutdown_connections 2024-09-10T11:40:39.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.530+0000 7f6691b5c700 1 -- 172.21.15.144:0/1714652205 wait complete. 2024-09-10T11:40:39.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.635+0000 7f86dd59b700 1 -- 172.21.15.144:0/3217212712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86d8100d80 msgr2=0x7f86d8104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:39.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.635+0000 7f86dd59b700 1 --2- 172.21.15.144:0/3217212712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86d8100d80 0x7f86d8104d50 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f86c0009910 tx=0x7f86c0009c20 comp rx=0 tx=0).stop 2024-09-10T11:40:39.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.635+0000 7f86dd59b700 1 -- 172.21.15.144:0/3217212712 shutdown_connections 2024-09-10T11:40:39.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.635+0000 7f86dd59b700 1 --2- 172.21.15.144:0/3217212712 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86d8100d80 0x7f86d8104d50 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.635+0000 7f86dd59b700 1 --2- 172.21.15.144:0/3217212712 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86d81003e0 0x7f86d81007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.635+0000 7f86dd59b700 1 -- 172.21.15.144:0/3217212712 >> 172.21.15.144:0/3217212712 conn(0x7f86d80fbca0 msgr2=0x7f86d80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:39.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.635+0000 7f86dd59b700 1 -- 172.21.15.144:0/3217212712 shutdown_connections 2024-09-10T11:40:39.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.635+0000 7f86dd59b700 1 -- 172.21.15.144:0/3217212712 wait complete. 2024-09-10T11:40:39.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.635+0000 7f86dd59b700 1 Processor -- start 2024-09-10T11:40:39.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.636+0000 7f86dd59b700 1 -- start start 2024-09-10T11:40:39.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.636+0000 7f86dd59b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86d81003e0 0x7f86d8197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:39.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.636+0000 7f86dd59b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86d8100d80 0x7f86d81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:39.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.636+0000 7f86dd59b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d8198b90 con 0x7f86d81003e0 2024-09-10T11:40:39.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.636+0000 7f86dd59b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d819c920 con 0x7f86d8100d80 2024-09-10T11:40:39.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.636+0000 7f86d6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86d81003e0 0x7f86d8197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:39.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.636+0000 7f86d6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86d81003e0 0x7f86d8197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34628/0 (socket says 172.21.15.144:34628) 2024-09-10T11:40:39.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.636+0000 7f86d6ffd700 1 -- 172.21.15.144:0/520211024 learned_addr learned my addr 172.21.15.144:0/520211024 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:40:39.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.636+0000 7f86d6ffd700 1 -- 172.21.15.144:0/520211024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86d8100d80 msgr2=0x7f86d81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:39.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.636+0000 7f86ce7fc700 1 --2- 172.21.15.144:0/520211024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86d8100d80 0x7f86d81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:39.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.636+0000 7f86d6ffd700 1 --2- 172.21.15.144:0/520211024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86d8100d80 0x7f86d81984b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.636+0000 7f86d6ffd700 1 -- 172.21.15.144:0/520211024 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c00095f0 con 0x7f86d81003e0 2024-09-10T11:40:39.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.637+0000 7f86d6ffd700 1 --2- 172.21.15.144:0/520211024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86d81003e0 0x7f86d8197f70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f86c800d8d0 tx=0x7f86c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:39.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.637+0000 7f86d4ff9700 1 -- 172.21.15.144:0/520211024 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86c8009940 con 0x7f86d81003e0 2024-09-10T11:40:39.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.637+0000 7f86d4ff9700 1 -- 172.21.15.144:0/520211024 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f86c8010460 con 0x7f86d81003e0 2024-09-10T11:40:39.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.637+0000 7f86dd59b700 1 -- 172.21.15.144:0/520211024 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d819cc00 con 0x7f86d81003e0 2024-09-10T11:40:39.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.637+0000 7f86d4ff9700 1 -- 172.21.15.144:0/520211024 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f86c800f5f0 con 0x7f86d81003e0 2024-09-10T11:40:39.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.637+0000 7f86dd59b700 1 -- 172.21.15.144:0/520211024 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d819d150 con 0x7f86d81003e0 2024-09-10T11:40:39.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.638+0000 7f86dd59b700 1 -- 172.21.15.144:0/520211024 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d81044d0 con 0x7f86d81003e0 2024-09-10T11:40:39.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.638+0000 7f86d4ff9700 1 -- 172.21.15.144:0/520211024 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f86c80105d0 con 0x7f86d81003e0 2024-09-10T11:40:39.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.639+0000 7f86d4ff9700 1 --2- 172.21.15.144:0/520211024 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f86c406e200 0x7f86c40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:39.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.639+0000 7f86d4ff9700 1 -- 172.21.15.144:0/520211024 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f86c808e0a0 con 0x7f86d81003e0 2024-09-10T11:40:39.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.639+0000 7f86ce7fc700 1 --2- 172.21.15.144:0/520211024 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f86c406e200 0x7f86c40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:39.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.639+0000 7f86ce7fc700 1 --2- 172.21.15.144:0/520211024 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f86c406e200 0x7f86c40706b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f86c00059d0 tx=0x7f86c0005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:39.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.643+0000 7f86d4ff9700 1 -- 172.21.15.144:0/520211024 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f86c8057480 con 0x7f86d81003e0 2024-09-10T11:40:39.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.853+0000 7f86dd59b700 1 -- 172.21.15.144:0/520211024 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f86d819d450 con 0x7f86c406e200 2024-09-10T11:40:39.864 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.863+0000 7f86d4ff9700 1 -- 172.21.15.144:0/520211024 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f86d819d450 con 0x7f86c406e200 2024-09-10T11:40:39.865 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:40:39.865 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (8m) 2m ago 19m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:40:39.865 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (19m) 2m ago 19m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:40:39.865 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (18m) 2m ago 18m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:40:39.865 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (6m) 2m ago 19m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:40:39.865 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (6m) 2m ago 18m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:40:39.865 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (8m) 2m ago 19m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:40:39.865 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (2m) 2m ago 14m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:40:39.865 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (2m) 2m ago 13m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:40:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (13m) 2m ago 13m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:40:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (2m) 2m ago 13m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:40:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (10m) 2m ago 20m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:40:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (9m) 2m ago 18m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:40:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (7m) 2m ago 21m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:40:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (7m) 2m ago 18m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:40:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (9m) 2m ago 19m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:40:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (9m) 2m ago 18m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:40:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (6m) 2m ago 17m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:40:39.866 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (5m) 2m ago 17m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:40:39.867 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (4m) 2m ago 16m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:40:39.867 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (4m) 2m ago 16m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:40:39.867 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (3m) 2m ago 15m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:40:39.867 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (3m) 2m ago 15m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:40:39.867 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (9m) 2m ago 18m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:40:39.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.865+0000 7f86dd59b700 1 -- 172.21.15.144:0/520211024 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f86c406e200 msgr2=0x7f86c40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:39.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.865+0000 7f86dd59b700 1 --2- 172.21.15.144:0/520211024 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f86c406e200 0x7f86c40706b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f86c00059d0 tx=0x7f86c0005960 comp rx=0 tx=0).stop 2024-09-10T11:40:39.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.865+0000 7f86dd59b700 1 -- 172.21.15.144:0/520211024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86d81003e0 msgr2=0x7f86d8197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:39.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.865+0000 7f86dd59b700 1 --2- 172.21.15.144:0/520211024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86d81003e0 0x7f86d8197f70 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f86c800d8d0 tx=0x7f86c800dc90 comp rx=0 tx=0).stop 2024-09-10T11:40:39.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.865+0000 7f86dd59b700 1 -- 172.21.15.144:0/520211024 shutdown_connections 2024-09-10T11:40:39.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.865+0000 7f86dd59b700 1 --2- 172.21.15.144:0/520211024 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f86c406e200 0x7f86c40706b0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.865+0000 7f86dd59b700 1 --2- 172.21.15.144:0/520211024 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f86d81003e0 0x7f86d8197f70 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.865+0000 7f86dd59b700 1 --2- 172.21.15.144:0/520211024 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f86d8100d80 0x7f86d81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.865+0000 7f86dd59b700 1 -- 172.21.15.144:0/520211024 >> 172.21.15.144:0/520211024 conn(0x7f86d80fbca0 msgr2=0x7f86d80fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:39.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.866+0000 7f86dd59b700 1 -- 172.21.15.144:0/520211024 shutdown_connections 2024-09-10T11:40:39.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.866+0000 7f86dd59b700 1 -- 172.21.15.144:0/520211024 wait complete. 2024-09-10T11:40:39.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.970+0000 7fcad5415700 1 -- 172.21.15.144:0/1089217709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad0068630 msgr2=0x7fcad0068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:39.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.970+0000 7fcad5415700 1 --2- 172.21.15.144:0/1089217709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad0068630 0x7fcad0068a00 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcab8009910 tx=0x7fcab8009c20 comp rx=0 tx=0).stop 2024-09-10T11:40:39.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.970+0000 7fcad5415700 1 -- 172.21.15.144:0/1089217709 shutdown_connections 2024-09-10T11:40:39.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.970+0000 7fcad5415700 1 --2- 172.21.15.144:0/1089217709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcad0068fd0 0x7fcad0104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.970+0000 7fcad5415700 1 --2- 172.21.15.144:0/1089217709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad0068630 0x7fcad0068a00 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.970+0000 7fcad5415700 1 -- 172.21.15.144:0/1089217709 >> 172.21.15.144:0/1089217709 conn(0x7fcad0074f50 msgr2=0x7fcad0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:39.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.970+0000 7fcad5415700 1 -- 172.21.15.144:0/1089217709 shutdown_connections 2024-09-10T11:40:39.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.970+0000 7fcad5415700 1 -- 172.21.15.144:0/1089217709 wait complete. 2024-09-10T11:40:39.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.970+0000 7fcad5415700 1 Processor -- start 2024-09-10T11:40:39.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.971+0000 7fcad5415700 1 -- start start 2024-09-10T11:40:39.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.971+0000 7fcad5415700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad0068630 0x7fcad0195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:39.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.971+0000 7fcad5415700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcad0068fd0 0x7fcad0196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:39.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.971+0000 7fcad5415700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad0196b40 con 0x7fcad0068630 2024-09-10T11:40:39.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.971+0000 7fcad5415700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcad019a8d0 con 0x7fcad0068fd0 2024-09-10T11:40:39.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.971+0000 7fcaceffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad0068630 0x7fcad0195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:39.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.971+0000 7fcaceffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad0068630 0x7fcad0195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34644/0 (socket says 172.21.15.144:34644) 2024-09-10T11:40:39.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.971+0000 7fcaceffd700 1 -- 172.21.15.144:0/3040512610 learned_addr learned my addr 172.21.15.144:0/3040512610 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:40:39.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.971+0000 7fcaceffd700 1 -- 172.21.15.144:0/3040512610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcad0068fd0 msgr2=0x7fcad0196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:39.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.971+0000 7fcaceffd700 1 --2- 172.21.15.144:0/3040512610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcad0068fd0 0x7fcad0196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:39.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.971+0000 7fcaceffd700 1 -- 172.21.15.144:0/3040512610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab80095f0 con 0x7fcad0068630 2024-09-10T11:40:39.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.971+0000 7fcaceffd700 1 --2- 172.21.15.144:0/3040512610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad0068630 0x7fcad0195f20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcab800b5c0 tx=0x7fcab800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:39.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.972+0000 7fcac7fff700 1 -- 172.21.15.144:0/3040512610 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcab801d070 con 0x7fcad0068630 2024-09-10T11:40:39.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.972+0000 7fcac7fff700 1 -- 172.21.15.144:0/3040512610 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcab800f460 con 0x7fcad0068630 2024-09-10T11:40:39.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.972+0000 7fcad5415700 1 -- 172.21.15.144:0/3040512610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcad019ab50 con 0x7fcad0068630 2024-09-10T11:40:39.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.972+0000 7fcac7fff700 1 -- 172.21.15.144:0/3040512610 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcab8017610 con 0x7fcad0068630 2024-09-10T11:40:39.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.972+0000 7fcad5415700 1 -- 172.21.15.144:0/3040512610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcad019b040 con 0x7fcad0068630 2024-09-10T11:40:39.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.973+0000 7fcad5415700 1 -- 172.21.15.144:0/3040512610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcad01086d0 con 0x7fcad0068630 2024-09-10T11:40:39.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.973+0000 7fcac7fff700 1 -- 172.21.15.144:0/3040512610 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fcab800f5d0 con 0x7fcad0068630 2024-09-10T11:40:39.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.974+0000 7fcac7fff700 1 --2- 172.21.15.144:0/3040512610 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcabc06e200 0x7fcabc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:39.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.974+0000 7fcac7fff700 1 -- 172.21.15.144:0/3040512610 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcab808f6e0 con 0x7fcad0068630 2024-09-10T11:40:39.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.974+0000 7fcace7fc700 1 --2- 172.21.15.144:0/3040512610 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcabc06e200 0x7fcabc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:39.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.974+0000 7fcace7fc700 1 --2- 172.21.15.144:0/3040512610 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcabc06e200 0x7fcabc0706b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcad0197540 tx=0x7fcac0006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:39.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:39.978+0000 7fcac7fff700 1 -- 172.21.15.144:0/3040512610 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcab8058ac0 con 0x7fcad0068630 2024-09-10T11:40:40.266 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.264+0000 7fcad5415700 1 -- 172.21.15.144:0/3040512610 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcad002cc30 con 0x7fcad0068630 2024-09-10T11:40:40.266 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.265+0000 7fcac7fff700 1 -- 172.21.15.144:0/3040512610 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fcab805c180 con 0x7fcad0068630 2024-09-10T11:40:40.266 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:40:40.267 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:40:40.267 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:40:40.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:40:40.267 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:40:40.267 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:40:40.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:40:40.267 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:40:40.267 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:40:40.267 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:40:40.267 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:40:40.267 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:40:40.267 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:40:40.268 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:40:40.268 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:40:40.268 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:40:40.268 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:40:40.268 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:40:40.268 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:40:40.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.267+0000 7fcad5415700 1 -- 172.21.15.144:0/3040512610 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcabc06e200 msgr2=0x7fcabc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:40.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.267+0000 7fcad5415700 1 --2- 172.21.15.144:0/3040512610 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcabc06e200 0x7fcabc0706b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcad0197540 tx=0x7fcac0006d20 comp rx=0 tx=0).stop 2024-09-10T11:40:40.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.267+0000 7fcad5415700 1 -- 172.21.15.144:0/3040512610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad0068630 msgr2=0x7fcad0195f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:40.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.267+0000 7fcad5415700 1 --2- 172.21.15.144:0/3040512610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad0068630 0x7fcad0195f20 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcab800b5c0 tx=0x7fcab800be00 comp rx=0 tx=0).stop 2024-09-10T11:40:40.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.267+0000 7fcad5415700 1 -- 172.21.15.144:0/3040512610 shutdown_connections 2024-09-10T11:40:40.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.267+0000 7fcad5415700 1 --2- 172.21.15.144:0/3040512610 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcabc06e200 0x7fcabc0706b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:40.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.267+0000 7fcad5415700 1 --2- 172.21.15.144:0/3040512610 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcad0068630 0x7fcad0195f20 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:40.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.267+0000 7fcad5415700 1 --2- 172.21.15.144:0/3040512610 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcad0068fd0 0x7fcad0196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:40.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.267+0000 7fcad5415700 1 -- 172.21.15.144:0/3040512610 >> 172.21.15.144:0/3040512610 conn(0x7fcad0074f50 msgr2=0x7fcad00fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:40.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.267+0000 7fcad5415700 1 -- 172.21.15.144:0/3040512610 shutdown_connections 2024-09-10T11:40:40.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.267+0000 7fcad5415700 1 -- 172.21.15.144:0/3040512610 wait complete. 2024-09-10T11:40:40.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.372+0000 7fed0657b700 1 -- 172.21.15.144:0/1570874177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed001003e0 msgr2=0x7fed001007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:40.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.372+0000 7fed0657b700 1 --2- 172.21.15.144:0/1570874177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed001003e0 0x7fed001007b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fecf0009910 tx=0x7fecf0009c20 comp rx=0 tx=0).stop 2024-09-10T11:40:40.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.373+0000 7fed0657b700 1 -- 172.21.15.144:0/1570874177 shutdown_connections 2024-09-10T11:40:40.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.373+0000 7fed0657b700 1 --2- 172.21.15.144:0/1570874177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed00100d80 0x7fed00104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:40.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.373+0000 7fed0657b700 1 --2- 172.21.15.144:0/1570874177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed001003e0 0x7fed001007b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:40.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.373+0000 7fed0657b700 1 -- 172.21.15.144:0/1570874177 >> 172.21.15.144:0/1570874177 conn(0x7fed000fbca0 msgr2=0x7fed000fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:40.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.373+0000 7fed0657b700 1 -- 172.21.15.144:0/1570874177 shutdown_connections 2024-09-10T11:40:40.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.373+0000 7fed0657b700 1 -- 172.21.15.144:0/1570874177 wait complete. 2024-09-10T11:40:40.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.373+0000 7fed0657b700 1 Processor -- start 2024-09-10T11:40:40.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.373+0000 7fed0657b700 1 -- start start 2024-09-10T11:40:40.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fed0657b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed001003e0 0x7fed00195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:40.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fed0657b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed00100d80 0x7fed001962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:40.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fed0657b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed001969a0 con 0x7fed001003e0 2024-09-10T11:40:40.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fed0657b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed0019a730 con 0x7fed00100d80 2024-09-10T11:40:40.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fecfffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed001003e0 0x7fed00195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:40.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fecfffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed001003e0 0x7fed00195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34658/0 (socket says 172.21.15.144:34658) 2024-09-10T11:40:40.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fecfffff700 1 -- 172.21.15.144:0/228300536 learned_addr learned my addr 172.21.15.144:0/228300536 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:40:40.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fecf7fff700 1 --2- 172.21.15.144:0/228300536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed00100d80 0x7fed001962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:40.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fecfffff700 1 -- 172.21.15.144:0/228300536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed00100d80 msgr2=0x7fed001962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:40.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fecfffff700 1 --2- 172.21.15.144:0/228300536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed00100d80 0x7fed001962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:40.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fecfffff700 1 -- 172.21.15.144:0/228300536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf00095f0 con 0x7fed001003e0 2024-09-10T11:40:40.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fecfffff700 1 --2- 172.21.15.144:0/228300536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed001003e0 0x7fed00195d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fecf0006010 tx=0x7fecf000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:40.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fecfdffb700 1 -- 172.21.15.144:0/228300536 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecf001d070 con 0x7fed001003e0 2024-09-10T11:40:40.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fed0657b700 1 -- 172.21.15.144:0/228300536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed0019a9b0 con 0x7fed001003e0 2024-09-10T11:40:40.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fecfdffb700 1 -- 172.21.15.144:0/228300536 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fecf0004fb0 con 0x7fed001003e0 2024-09-10T11:40:40.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fecfdffb700 1 -- 172.21.15.144:0/228300536 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fecf0022ea0 con 0x7fed001003e0 2024-09-10T11:40:40.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.374+0000 7fed0657b700 1 -- 172.21.15.144:0/228300536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed0019aea0 con 0x7fed001003e0 2024-09-10T11:40:40.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.375+0000 7fed0657b700 1 -- 172.21.15.144:0/228300536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed001086f0 con 0x7fed001003e0 2024-09-10T11:40:40.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.376+0000 7fecfdffb700 1 -- 172.21.15.144:0/228300536 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fecf0022470 con 0x7fed001003e0 2024-09-10T11:40:40.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.376+0000 7fecfdffb700 1 --2- 172.21.15.144:0/228300536 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fecec06e200 0x7fecec0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:40.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.376+0000 7fecfdffb700 1 -- 172.21.15.144:0/228300536 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fecf008f610 con 0x7fed001003e0 2024-09-10T11:40:40.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.376+0000 7fecf7fff700 1 --2- 172.21.15.144:0/228300536 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fecec06e200 0x7fecec0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:40.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.377+0000 7fecf7fff700 1 --2- 172.21.15.144:0/228300536 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fecec06e200 0x7fecec0706b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fece8005fd0 tx=0x7fece800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:40.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.381+0000 7fecfdffb700 1 -- 172.21.15.144:0/228300536 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fecf0058aa0 con 0x7fed001003e0 2024-09-10T11:40:40.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.628+0000 7fed0657b700 1 -- 172.21.15.144:0/228300536 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fed0002ccf0 con 0x7fed001003e0 2024-09-10T11:40:40.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.628+0000 7fecfdffb700 1 -- 172.21.15.144:0/228300536 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7fecf0022720 con 0x7fed001003e0 2024-09-10T11:40:40.630 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:40:40.630 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:40:40.630 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:40:40.630 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:40:40.630 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:40:40.630 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:40:40.630 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:40:40.630 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:40:40.630 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:40:40.630 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:40:40.630 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:40:40.630 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:40:40.631 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:40:40.631 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:40:40.631 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:40:40.631 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:40:40.631 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:40:40.631 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:40:40.631 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:40:40.631 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:40:40.631 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:40:40.631 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:40:40.631 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:40:40.631 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:40:40.631 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:40:40.632 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:40:40.632 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:40:40.632 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:40:40.632 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:40:40.632 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:40:40.632 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:40:40.632 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:40:40.632 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:40:40.632 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:40:40.632 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:40:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:40:40.632 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:40:40.632 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:40:40.633 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:40:40.633 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:40:40.633 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:40:40.633 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:40:40.633 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:40:40.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.630+0000 7fed0657b700 1 -- 172.21.15.144:0/228300536 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fecec06e200 msgr2=0x7fecec0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:40.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.630+0000 7fed0657b700 1 --2- 172.21.15.144:0/228300536 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fecec06e200 0x7fecec0706b0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fece8005fd0 tx=0x7fece800a560 comp rx=0 tx=0).stop 2024-09-10T11:40:40.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.630+0000 7fed0657b700 1 -- 172.21.15.144:0/228300536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed001003e0 msgr2=0x7fed00195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:40.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.630+0000 7fed0657b700 1 --2- 172.21.15.144:0/228300536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed001003e0 0x7fed00195d80 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fecf0006010 tx=0x7fecf000be20 comp rx=0 tx=0).stop 2024-09-10T11:40:40.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.631+0000 7fed0657b700 1 -- 172.21.15.144:0/228300536 shutdown_connections 2024-09-10T11:40:40.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.631+0000 7fed0657b700 1 --2- 172.21.15.144:0/228300536 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fecec06e200 0x7fecec0706b0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:40.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.631+0000 7fed0657b700 1 --2- 172.21.15.144:0/228300536 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fed001003e0 0x7fed00195d80 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:40.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.631+0000 7fed0657b700 1 --2- 172.21.15.144:0/228300536 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fed00100d80 0x7fed001962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:40.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.631+0000 7fed0657b700 1 -- 172.21.15.144:0/228300536 >> 172.21.15.144:0/228300536 conn(0x7fed000fbca0 msgr2=0x7fed000fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:40.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.631+0000 7fed0657b700 1 -- 172.21.15.144:0/228300536 shutdown_connections 2024-09-10T11:40:40.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.631+0000 7fed0657b700 1 -- 172.21.15.144:0/228300536 wait complete. 2024-09-10T11:40:40.634 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:40:40.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.736+0000 7fefd25fb700 1 -- 172.21.15.144:0/4211904111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0ff290 msgr2=0x7fefcc0ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:40.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.736+0000 7fefd25fb700 1 --2- 172.21.15.144:0/4211904111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0ff290 0x7fefcc0ff660 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fefbc009910 tx=0x7fefbc009c20 comp rx=0 tx=0).stop 2024-09-10T11:40:40.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.736+0000 7fefd25fb700 1 -- 172.21.15.144:0/4211904111 shutdown_connections 2024-09-10T11:40:40.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.736+0000 7fefd25fb700 1 --2- 172.21.15.144:0/4211904111 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefcc0ffba0 0x7fefcc10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:40.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.736+0000 7fefd25fb700 1 --2- 172.21.15.144:0/4211904111 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0ff290 0x7fefcc0ff660 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:40.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.736+0000 7fefd25fb700 1 -- 172.21.15.144:0/4211904111 >> 172.21.15.144:0/4211904111 conn(0x7fefcc074850 msgr2=0x7fefcc074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:40.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.736+0000 7fefd25fb700 1 -- 172.21.15.144:0/4211904111 shutdown_connections 2024-09-10T11:40:40.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.736+0000 7fefd25fb700 1 -- 172.21.15.144:0/4211904111 wait complete. 2024-09-10T11:40:40.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.737+0000 7fefd25fb700 1 Processor -- start 2024-09-10T11:40:40.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.737+0000 7fefd25fb700 1 -- start start 2024-09-10T11:40:40.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.737+0000 7fefd25fb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0ff290 0x7fefcc19bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:40.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.737+0000 7fefd25fb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefcc0ffba0 0x7fefcc19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:40.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.737+0000 7fefd25fb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc19c690 con 0x7fefcc0ff290 2024-09-10T11:40:40.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.737+0000 7fefd25fb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc195b40 con 0x7fefcc0ffba0 2024-09-10T11:40:40.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.738+0000 7fefcbfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0ff290 0x7fefcc19bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:40.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.738+0000 7fefcbfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0ff290 0x7fefcc19bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34684/0 (socket says 172.21.15.144:34684) 2024-09-10T11:40:40.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.738+0000 7fefcbfff700 1 -- 172.21.15.144:0/628916360 learned_addr learned my addr 172.21.15.144:0/628916360 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:40:40.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.738+0000 7fefcb7fe700 1 --2- 172.21.15.144:0/628916360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefcc0ffba0 0x7fefcc19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:40.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.738+0000 7fefcbfff700 1 -- 172.21.15.144:0/628916360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefcc0ffba0 msgr2=0x7fefcc19c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:40.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.738+0000 7fefcbfff700 1 --2- 172.21.15.144:0/628916360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefcc0ffba0 0x7fefcc19c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:40.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.738+0000 7fefcbfff700 1 -- 172.21.15.144:0/628916360 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefbc0095f0 con 0x7fefcc0ff290 2024-09-10T11:40:40.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.738+0000 7fefcbfff700 1 --2- 172.21.15.144:0/628916360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0ff290 0x7fefcc19bac0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0092e0 tx=0x7fefbc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:40.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.738+0000 7fefc97fa700 1 -- 172.21.15.144:0/628916360 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefbc01d070 con 0x7fefcc0ff290 2024-09-10T11:40:40.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.738+0000 7fefd25fb700 1 -- 172.21.15.144:0/628916360 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefcc195dc0 con 0x7fefcc0ff290 2024-09-10T11:40:40.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.738+0000 7fefd25fb700 1 -- 172.21.15.144:0/628916360 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefcc196330 con 0x7fefcc0ff290 2024-09-10T11:40:40.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.738+0000 7fefc97fa700 1 -- 172.21.15.144:0/628916360 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fefbc005dc0 con 0x7fefcc0ff290 2024-09-10T11:40:40.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.738+0000 7fefc97fa700 1 -- 172.21.15.144:0/628916360 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefbc017400 con 0x7fefcc0ff290 2024-09-10T11:40:40.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.739+0000 7fefd25fb700 1 -- 172.21.15.144:0/628916360 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefcc107d10 con 0x7fefcc0ff290 2024-09-10T11:40:40.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.740+0000 7fefc97fa700 1 -- 172.21.15.144:0/628916360 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fefbc00f6f0 con 0x7fefcc0ff290 2024-09-10T11:40:40.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.740+0000 7fefc97fa700 1 --2- 172.21.15.144:0/628916360 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fefb406e200 0x7fefb40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:40.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.740+0000 7fefc97fa700 1 -- 172.21.15.144:0/628916360 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fefbc08f3a0 con 0x7fefcc0ff290 2024-09-10T11:40:40.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.740+0000 7fefcb7fe700 1 --2- 172.21.15.144:0/628916360 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fefb406e200 0x7fefb40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:40.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.741+0000 7fefcb7fe700 1 --2- 172.21.15.144:0/628916360 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fefb406e200 0x7fefb40706b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0fdd20 tx=0x7fefc000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:40.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.745+0000 7fefc97fa700 1 -- 172.21.15.144:0/628916360 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fefbc05c330 con 0x7fefcc0ff290 2024-09-10T11:40:40.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:40 smithi144 ceph-mon[87436]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:40.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:40 smithi144 ceph-mon[87436]: from='client.34544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:40.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:40 smithi144 ceph-mon[87436]: pgmap v260: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:40.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:40 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3040512610' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:40:40.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:40 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/228300536' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:40:40.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.970+0000 7fefd25fb700 1 -- 172.21.15.144:0/628916360 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fefcc1965f0 con 0x7fefb406e200 2024-09-10T11:40:40.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.972+0000 7fefc97fa700 1 -- 172.21.15.144:0/628916360 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fefcc1965f0 con 0x7fefb406e200 2024-09-10T11:40:40.973 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:40:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:40:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:40:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:40:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:40:40.973 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:40:40.974 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:40:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:40:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:40:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:40:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:40:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:40:40.976 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:40:40.977 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:40:40.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.974+0000 7fefd25fb700 1 -- 172.21.15.144:0/628916360 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fefb406e200 msgr2=0x7fefb40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:40.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.974+0000 7fefd25fb700 1 --2- 172.21.15.144:0/628916360 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fefb406e200 0x7fefb40706b0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fefcc0fdd20 tx=0x7fefc000b410 comp rx=0 tx=0).stop 2024-09-10T11:40:40.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.974+0000 7fefd25fb700 1 -- 172.21.15.144:0/628916360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0ff290 msgr2=0x7fefcc19bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:40.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.974+0000 7fefd25fb700 1 --2- 172.21.15.144:0/628916360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0ff290 0x7fefcc19bac0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fefbc0092e0 tx=0x7fefbc00bd70 comp rx=0 tx=0).stop 2024-09-10T11:40:40.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.974+0000 7fefd25fb700 1 -- 172.21.15.144:0/628916360 shutdown_connections 2024-09-10T11:40:40.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.974+0000 7fefd25fb700 1 --2- 172.21.15.144:0/628916360 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fefb406e200 0x7fefb40706b0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:40.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.974+0000 7fefd25fb700 1 --2- 172.21.15.144:0/628916360 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fefcc0ff290 0x7fefcc19bac0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:40.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.974+0000 7fefd25fb700 1 --2- 172.21.15.144:0/628916360 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fefcc0ffba0 0x7fefcc19c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:40.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.974+0000 7fefd25fb700 1 -- 172.21.15.144:0/628916360 >> 172.21.15.144:0/628916360 conn(0x7fefcc074850 msgr2=0x7fefcc104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:40.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.974+0000 7fefd25fb700 1 -- 172.21.15.144:0/628916360 shutdown_connections 2024-09-10T11:40:40.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:40.974+0000 7fefd25fb700 1 -- 172.21.15.144:0/628916360 wait complete. 2024-09-10T11:40:41.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.079+0000 7f4758f82700 1 -- 172.21.15.144:0/315654646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47540ffd30 msgr2=0x7f4754103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:41.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.079+0000 7f4758f82700 1 --2- 172.21.15.144:0/315654646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47540ffd30 0x7f4754103ce0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4748009910 tx=0x7f4748009c20 comp rx=0 tx=0).stop 2024-09-10T11:40:41.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.079+0000 7f4758f82700 1 -- 172.21.15.144:0/315654646 shutdown_connections 2024-09-10T11:40:41.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.079+0000 7f4758f82700 1 --2- 172.21.15.144:0/315654646 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47540ffd30 0x7f4754103ce0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:41.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.079+0000 7f4758f82700 1 --2- 172.21.15.144:0/315654646 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47540ff390 0x7f47540ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:41.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.079+0000 7f4758f82700 1 -- 172.21.15.144:0/315654646 >> 172.21.15.144:0/315654646 conn(0x7f4754074f50 msgr2=0x7f4754075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:41.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.079+0000 7f4758f82700 1 -- 172.21.15.144:0/315654646 shutdown_connections 2024-09-10T11:40:41.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.079+0000 7f4758f82700 1 -- 172.21.15.144:0/315654646 wait complete. 2024-09-10T11:40:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.080+0000 7f4758f82700 1 Processor -- start 2024-09-10T11:40:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.080+0000 7f4758f82700 1 -- start start 2024-09-10T11:40:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.080+0000 7f4758f82700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47540ff390 0x7f4754197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.080+0000 7f4758f82700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47540ffd30 0x7f47541984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.080+0000 7f4758f82700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4754198b30 con 0x7f47540ffd30 2024-09-10T11:40:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.080+0000 7f4758f82700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4754198c70 con 0x7f47540ff390 2024-09-10T11:40:41.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.080+0000 7f4751d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47540ffd30 0x7f47541984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.080+0000 7f4751d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47540ffd30 0x7f47541984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34708/0 (socket says 172.21.15.144:34708) 2024-09-10T11:40:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.080+0000 7f4751d9b700 1 -- 172.21.15.144:0/1964334745 learned_addr learned my addr 172.21.15.144:0/1964334745 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:40:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.080+0000 7f475259c700 1 --2- 172.21.15.144:0/1964334745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47540ff390 0x7f4754197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.081+0000 7f4751d9b700 1 -- 172.21.15.144:0/1964334745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47540ff390 msgr2=0x7f4754197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.081+0000 7f4751d9b700 1 --2- 172.21.15.144:0/1964334745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47540ff390 0x7f4754197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.081+0000 7f4751d9b700 1 -- 172.21.15.144:0/1964334745 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47480095f0 con 0x7f47540ffd30 2024-09-10T11:40:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.081+0000 7f475259c700 1 --2- 172.21.15.144:0/1964334745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47540ff390 0x7f4754197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:40:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.081+0000 7f4751d9b700 1 --2- 172.21.15.144:0/1964334745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47540ffd30 0x7f47541984e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f474800bda0 tx=0x7f474800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.081+0000 7f47437fe700 1 -- 172.21.15.144:0/1964334745 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f474801d070 con 0x7f47540ffd30 2024-09-10T11:40:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.081+0000 7f47437fe700 1 -- 172.21.15.144:0/1964334745 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4748004fb0 con 0x7f47540ffd30 2024-09-10T11:40:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.081+0000 7f47437fe700 1 -- 172.21.15.144:0/1964334745 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f474800f480 con 0x7f47540ffd30 2024-09-10T11:40:41.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.081+0000 7f4758f82700 1 -- 172.21.15.144:0/1964334745 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f475419ca60 con 0x7f47540ffd30 2024-09-10T11:40:41.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.081+0000 7f4758f82700 1 -- 172.21.15.144:0/1964334745 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f475419cf00 con 0x7f47540ffd30 2024-09-10T11:40:41.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.082+0000 7f4758f82700 1 -- 172.21.15.144:0/1964334745 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f475404ea50 con 0x7f47540ffd30 2024-09-10T11:40:41.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.082+0000 7f47437fe700 1 -- 172.21.15.144:0/1964334745 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4748022870 con 0x7f47540ffd30 2024-09-10T11:40:41.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.083+0000 7f47437fe700 1 --2- 172.21.15.144:0/1964334745 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f473c06e200 0x7f473c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:40:41.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.083+0000 7f47437fe700 1 -- 172.21.15.144:0/1964334745 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f474808f5f0 con 0x7f47540ffd30 2024-09-10T11:40:41.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.083+0000 7f475259c700 1 --2- 172.21.15.144:0/1964334745 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f473c06e200 0x7f473c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:40:41.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.083+0000 7f475259c700 1 --2- 172.21.15.144:0/1964334745 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f473c06e200 0x7f473c0706b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4744005fd0 tx=0x7f4744005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:40:41.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.087+0000 7f47437fe700 1 -- 172.21.15.144:0/1964334745 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4748058a80 con 0x7f47540ffd30 2024-09-10T11:40:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:40 smithi178 ceph-mon[65940]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:40 smithi178 ceph-mon[65940]: from='client.34544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:40 smithi178 ceph-mon[65940]: pgmap v260: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:40 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3040512610' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:40:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:40 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/228300536' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:40:41.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.369+0000 7f4758f82700 1 -- 172.21.15.144:0/1964334745 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f475419d3c0 con 0x7f47540ffd30 2024-09-10T11:40:41.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.369+0000 7f47437fe700 1 -- 172.21.15.144:0/1964334745 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f474805c140 con 0x7f47540ffd30 2024-09-10T11:40:41.371 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:40:41.371 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:40:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:40:41.371 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:40:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:40:41.371 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:40:41.371 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:40:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.372+0000 7f4758f82700 1 -- 172.21.15.144:0/1964334745 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f473c06e200 msgr2=0x7f473c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.372+0000 7f4758f82700 1 --2- 172.21.15.144:0/1964334745 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f473c06e200 0x7f473c0706b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4744005fd0 tx=0x7f4744005e20 comp rx=0 tx=0).stop 2024-09-10T11:40:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.372+0000 7f4758f82700 1 -- 172.21.15.144:0/1964334745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47540ffd30 msgr2=0x7f47541984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:40:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.372+0000 7f4758f82700 1 --2- 172.21.15.144:0/1964334745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47540ffd30 0x7f47541984e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f474800bda0 tx=0x7f474800be80 comp rx=0 tx=0).stop 2024-09-10T11:40:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.372+0000 7f4758f82700 1 -- 172.21.15.144:0/1964334745 shutdown_connections 2024-09-10T11:40:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.372+0000 7f4758f82700 1 --2- 172.21.15.144:0/1964334745 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f473c06e200 0x7f473c0706b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.372+0000 7f4758f82700 1 --2- 172.21.15.144:0/1964334745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47540ff390 0x7f4754197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:41.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.372+0000 7f4758f82700 1 --2- 172.21.15.144:0/1964334745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47540ffd30 0x7f47541984e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:40:41.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.372+0000 7f4758f82700 1 -- 172.21.15.144:0/1964334745 >> 172.21.15.144:0/1964334745 conn(0x7f4754074f50 msgr2=0x7f47540fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:40:41.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.372+0000 7f4758f82700 1 -- 172.21.15.144:0/1964334745 shutdown_connections 2024-09-10T11:40:41.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:40:41.372+0000 7f4758f82700 1 -- 172.21.15.144:0/1964334745 wait complete. 2024-09-10T11:40:42.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:41 smithi178 ceph-mon[65940]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:42.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:41 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1964334745' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:40:42.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:41 smithi144 ceph-mon[87436]: from='client.34548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:42.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:41 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1964334745' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:40:43.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:42 smithi144 ceph-mon[87436]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:43.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:42 smithi144 ceph-mon[87436]: pgmap v261: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:42 smithi178 ceph-mon[65940]: from='client.34560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:40:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:42 smithi178 ceph-mon[65940]: pgmap v261: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:40:44.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:40:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:44 smithi178 ceph-mon[65940]: pgmap v262: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:45.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:44 smithi144 ceph-mon[87436]: pgmap v262: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:46 smithi178 ceph-mon[65940]: pgmap v263: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:46 smithi144 ceph-mon[87436]: pgmap v263: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:48 smithi178 ceph-mon[65940]: pgmap v264: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:49.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:48 smithi144 ceph-mon[87436]: pgmap v264: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:50 smithi178 ceph-mon[65940]: pgmap v265: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:50 smithi144 ceph-mon[87436]: pgmap v265: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:53.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:52 smithi144 ceph-mon[87436]: pgmap v266: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:52 smithi178 ceph-mon[65940]: pgmap v266: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:54 smithi178 ceph-mon[65940]: pgmap v267: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:55.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:54 smithi144 ceph-mon[87436]: pgmap v267: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:56 smithi178 ceph-mon[65940]: pgmap v268: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:56 smithi144 ceph-mon[87436]: pgmap v268: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:58.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:57 smithi178 ceph-mon[65940]: pgmap v269: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:58.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:57 smithi144 ceph-mon[87436]: pgmap v269: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:40:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:40:59.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:41:00.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:40:59 smithi178 ceph-mon[65940]: pgmap v270: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:00.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:40:59 smithi144 ceph-mon[87436]: pgmap v270: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:03.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:02 smithi144 ceph-mon[87436]: pgmap v271: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:03.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:02 smithi178 ceph-mon[65940]: pgmap v271: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:04 smithi178 ceph-mon[65940]: pgmap v272: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:05.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:04 smithi144 ceph-mon[87436]: pgmap v272: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:06 smithi178 ceph-mon[65940]: pgmap v273: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:06 smithi144 ceph-mon[87436]: pgmap v273: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:08.986 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:08 smithi144 ceph-mon[87436]: pgmap v274: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:08 smithi178 ceph-mon[65940]: pgmap v274: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:10 smithi178 ceph-mon[65940]: pgmap v275: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:11.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:10 smithi144 ceph-mon[87436]: pgmap v275: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:11.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.489+0000 7f6ae0a45700 1 -- 172.21.15.144:0/849159282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc0ff380 msgr2=0x7f6adc0ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:11.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.489+0000 7f6ae0a45700 1 --2- 172.21.15.144:0/849159282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc0ff380 0x7f6adc0ff750 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6acc009910 tx=0x7f6acc009c20 comp rx=0 tx=0).stop 2024-09-10T11:41:11.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.489+0000 7f6ae0a45700 1 -- 172.21.15.144:0/849159282 shutdown_connections 2024-09-10T11:41:11.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.489+0000 7f6ae0a45700 1 --2- 172.21.15.144:0/849159282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6adc0ffd20 0x7f6adc103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:11.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.489+0000 7f6ae0a45700 1 --2- 172.21.15.144:0/849159282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc0ff380 0x7f6adc0ff750 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:11.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.489+0000 7f6ae0a45700 1 -- 172.21.15.144:0/849159282 >> 172.21.15.144:0/849159282 conn(0x7f6adc0fac20 msgr2=0x7f6adc0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:11.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.489+0000 7f6ae0a45700 1 -- 172.21.15.144:0/849159282 shutdown_connections 2024-09-10T11:41:11.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.489+0000 7f6ae0a45700 1 -- 172.21.15.144:0/849159282 wait complete. 2024-09-10T11:41:11.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.490+0000 7f6ae0a45700 1 Processor -- start 2024-09-10T11:41:11.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.490+0000 7f6ae0a45700 1 -- start start 2024-09-10T11:41:11.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.490+0000 7f6ae0a45700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6adc0ff380 0x7f6adc19aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:11.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.490+0000 7f6ae0a45700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc0ffd20 0x7f6adc19b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:11.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.490+0000 7f6ae0a45700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc19b690 con 0x7f6adc0ffd20 2024-09-10T11:41:11.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.490+0000 7f6ae0a45700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc194b40 con 0x7f6adc0ff380 2024-09-10T11:41:11.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.491+0000 7f6ad1d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc0ffd20 0x7f6adc19b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:11.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.491+0000 7f6ad1d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc0ffd20 0x7f6adc19b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52814/0 (socket says 172.21.15.144:52814) 2024-09-10T11:41:11.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.491+0000 7f6ada59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6adc0ff380 0x7f6adc19aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:11.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.491+0000 7f6ad1d9b700 1 -- 172.21.15.144:0/627148053 learned_addr learned my addr 172.21.15.144:0/627148053 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:41:11.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.491+0000 7f6ad1d9b700 1 -- 172.21.15.144:0/627148053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6adc0ff380 msgr2=0x7f6adc19aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:11.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.491+0000 7f6ad1d9b700 1 --2- 172.21.15.144:0/627148053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6adc0ff380 0x7f6adc19aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:11.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.491+0000 7f6ad1d9b700 1 -- 172.21.15.144:0/627148053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc0095f0 con 0x7f6adc0ffd20 2024-09-10T11:41:11.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.491+0000 7f6ada59c700 1 --2- 172.21.15.144:0/627148053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6adc0ff380 0x7f6adc19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:41:11.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.491+0000 7f6ad1d9b700 1 --2- 172.21.15.144:0/627148053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc0ffd20 0x7f6adc19b000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40098f0 tx=0x7f6ac4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:11.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.491+0000 7f6ad3fff700 1 -- 172.21.15.144:0/627148053 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ac4012070 con 0x7f6adc0ffd20 2024-09-10T11:41:11.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.491+0000 7f6ad3fff700 1 -- 172.21.15.144:0/627148053 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6ac400d1f0 con 0x7f6adc0ffd20 2024-09-10T11:41:11.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.491+0000 7f6ae0a45700 1 -- 172.21.15.144:0/627148053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc194dc0 con 0x7f6adc0ffd20 2024-09-10T11:41:11.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.491+0000 7f6ad3fff700 1 -- 172.21.15.144:0/627148053 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ac400c8f0 con 0x7f6adc0ffd20 2024-09-10T11:41:11.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.491+0000 7f6ae0a45700 1 -- 172.21.15.144:0/627148053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc195310 con 0x7f6adc0ffd20 2024-09-10T11:41:11.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.493+0000 7f6ae0a45700 1 -- 172.21.15.144:0/627148053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6adc04ea50 con 0x7f6adc0ffd20 2024-09-10T11:41:11.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.493+0000 7f6ad3fff700 1 -- 172.21.15.144:0/627148053 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6ac4016070 con 0x7f6adc0ffd20 2024-09-10T11:41:11.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.493+0000 7f6ad3fff700 1 --2- 172.21.15.144:0/627148053 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6ac806e200 0x7f6ac80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:11.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.493+0000 7f6ad3fff700 1 -- 172.21.15.144:0/627148053 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6ac408f260 con 0x7f6adc0ffd20 2024-09-10T11:41:11.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.494+0000 7f6ada59c700 1 --2- 172.21.15.144:0/627148053 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6ac806e200 0x7f6ac80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:11.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.494+0000 7f6ada59c700 1 --2- 172.21.15.144:0/627148053 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6ac806e200 0x7f6ac80706b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6acc006010 tx=0x7f6acc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:11.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.498+0000 7f6ad3fff700 1 -- 172.21.15.144:0/627148053 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6ac405c1e0 con 0x7f6adc0ffd20 2024-09-10T11:41:11.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.720+0000 7f6ae0a45700 1 -- 172.21.15.144:0/627148053 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6adc195700 con 0x7f6ac806e200 2024-09-10T11:41:11.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.721+0000 7f6ad3fff700 1 -- 172.21.15.144:0/627148053 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6adc195700 con 0x7f6ac806e200 2024-09-10T11:41:11.728 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.724+0000 7f6ae0a45700 1 -- 172.21.15.144:0/627148053 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6ac806e200 msgr2=0x7f6ac80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:11.728 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.724+0000 7f6ae0a45700 1 --2- 172.21.15.144:0/627148053 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6ac806e200 0x7f6ac80706b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6acc006010 tx=0x7f6acc01a040 comp rx=0 tx=0).stop 2024-09-10T11:41:11.728 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.724+0000 7f6ae0a45700 1 -- 172.21.15.144:0/627148053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc0ffd20 msgr2=0x7f6adc19b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:11.728 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.724+0000 7f6ae0a45700 1 --2- 172.21.15.144:0/627148053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc0ffd20 0x7f6adc19b000 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f6ac40098f0 tx=0x7f6ac4009c00 comp rx=0 tx=0).stop 2024-09-10T11:41:11.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.724+0000 7f6ae0a45700 1 -- 172.21.15.144:0/627148053 shutdown_connections 2024-09-10T11:41:11.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.724+0000 7f6ae0a45700 1 --2- 172.21.15.144:0/627148053 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6ac806e200 0x7f6ac80706b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:11.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.724+0000 7f6ae0a45700 1 --2- 172.21.15.144:0/627148053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6adc0ff380 0x7f6adc19aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:11.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.724+0000 7f6ae0a45700 1 --2- 172.21.15.144:0/627148053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6adc0ffd20 0x7f6adc19b000 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:11.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.724+0000 7f6ae0a45700 1 -- 172.21.15.144:0/627148053 >> 172.21.15.144:0/627148053 conn(0x7f6adc0fac20 msgr2=0x7f6adc0fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:11.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.724+0000 7f6ae0a45700 1 -- 172.21.15.144:0/627148053 shutdown_connections 2024-09-10T11:41:11.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.724+0000 7f6ae0a45700 1 -- 172.21.15.144:0/627148053 wait complete. 2024-09-10T11:41:11.737 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:41:11.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.829+0000 7f901357c700 1 -- 172.21.15.144:0/739501577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900c0ffc70 msgr2=0x7f900c103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:11.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.829+0000 7f901357c700 1 --2- 172.21.15.144:0/739501577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900c0ffc70 0x7f900c103c40 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9000009910 tx=0x7f9000009c20 comp rx=0 tx=0).stop 2024-09-10T11:41:11.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.829+0000 7f901357c700 1 -- 172.21.15.144:0/739501577 shutdown_connections 2024-09-10T11:41:11.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.829+0000 7f901357c700 1 --2- 172.21.15.144:0/739501577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900c0ffc70 0x7f900c103c40 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:11.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.829+0000 7f901357c700 1 --2- 172.21.15.144:0/739501577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f900c0ff2d0 0x7f900c0ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:11.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.829+0000 7f901357c700 1 -- 172.21.15.144:0/739501577 >> 172.21.15.144:0/739501577 conn(0x7f900c0fabb0 msgr2=0x7f900c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:11.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.830+0000 7f901357c700 1 -- 172.21.15.144:0/739501577 shutdown_connections 2024-09-10T11:41:11.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.830+0000 7f901357c700 1 -- 172.21.15.144:0/739501577 wait complete. 2024-09-10T11:41:11.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.830+0000 7f901357c700 1 Processor -- start 2024-09-10T11:41:11.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.830+0000 7f901357c700 1 -- start start 2024-09-10T11:41:11.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.830+0000 7f901357c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f900c0ff2d0 0x7f900c195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:11.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.830+0000 7f901357c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900c0ffc70 0x7f900c196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:11.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.830+0000 7f901357c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c1968c0 con 0x7f900c0ffc70 2024-09-10T11:41:11.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.830+0000 7f901357c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c196a00 con 0x7f900c0ff2d0 2024-09-10T11:41:11.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.831+0000 7f9010b17700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900c0ffc70 0x7f900c196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:11.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.831+0000 7f9010b17700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900c0ffc70 0x7f900c196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52838/0 (socket says 172.21.15.144:52838) 2024-09-10T11:41:11.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.831+0000 7f9010b17700 1 -- 172.21.15.144:0/2241176461 learned_addr learned my addr 172.21.15.144:0/2241176461 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:41:11.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.831+0000 7f9011318700 1 --2- 172.21.15.144:0/2241176461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f900c0ff2d0 0x7f900c195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:11.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.831+0000 7f9010b17700 1 -- 172.21.15.144:0/2241176461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f900c0ff2d0 msgr2=0x7f900c195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:11.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.831+0000 7f9010b17700 1 --2- 172.21.15.144:0/2241176461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f900c0ff2d0 0x7f900c195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:11.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.831+0000 7f9010b17700 1 -- 172.21.15.144:0/2241176461 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90000095f0 con 0x7f900c0ffc70 2024-09-10T11:41:11.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.831+0000 7f9010b17700 1 --2- 172.21.15.144:0/2241176461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900c0ffc70 0x7f900c196270 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f900000bda0 tx=0x7f900000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.831+0000 7f900a7fc700 1 -- 172.21.15.144:0/2241176461 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f900001d070 con 0x7f900c0ffc70 2024-09-10T11:41:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.831+0000 7f900a7fc700 1 -- 172.21.15.144:0/2241176461 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9000004fb0 con 0x7f900c0ffc70 2024-09-10T11:41:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.831+0000 7f901357c700 1 -- 172.21.15.144:0/2241176461 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c19a7f0 con 0x7f900c0ffc70 2024-09-10T11:41:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.831+0000 7f900a7fc700 1 -- 172.21.15.144:0/2241176461 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f900000f480 con 0x7f900c0ffc70 2024-09-10T11:41:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.831+0000 7f901357c700 1 -- 172.21.15.144:0/2241176461 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c19ace0 con 0x7f900c0ffc70 2024-09-10T11:41:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.832+0000 7f901357c700 1 -- 172.21.15.144:0/2241176461 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900c108650 con 0x7f900c0ffc70 2024-09-10T11:41:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.833+0000 7f900a7fc700 1 -- 172.21.15.144:0/2241176461 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9000022870 con 0x7f900c0ffc70 2024-09-10T11:41:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.833+0000 7f900a7fc700 1 --2- 172.21.15.144:0/2241176461 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8ff406e200 0x7f8ff40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.833+0000 7f900a7fc700 1 -- 172.21.15.144:0/2241176461 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f900008f500 con 0x7f900c0ffc70 2024-09-10T11:41:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.833+0000 7f9011318700 1 --2- 172.21.15.144:0/2241176461 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8ff406e200 0x7f8ff40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.834+0000 7f9011318700 1 --2- 172.21.15.144:0/2241176461 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8ff406e200 0x7f8ff40706b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f900400e270 tx=0x7f9004009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:11.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:11.839+0000 7f900a7fc700 1 -- 172.21.15.144:0/2241176461 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9000058a10 con 0x7f900c0ffc70 2024-09-10T11:41:12.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.061+0000 7f901357c700 1 -- 172.21.15.144:0/2241176461 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f900c02ccf0 con 0x7f8ff406e200 2024-09-10T11:41:12.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.063+0000 7f900a7fc700 1 -- 172.21.15.144:0/2241176461 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f900c02ccf0 con 0x7f8ff406e200 2024-09-10T11:41:12.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.065+0000 7f901357c700 1 -- 172.21.15.144:0/2241176461 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8ff406e200 msgr2=0x7f8ff40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:12.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.065+0000 7f901357c700 1 --2- 172.21.15.144:0/2241176461 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8ff406e200 0x7f8ff40706b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f900400e270 tx=0x7f9004009040 comp rx=0 tx=0).stop 2024-09-10T11:41:12.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.065+0000 7f901357c700 1 -- 172.21.15.144:0/2241176461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900c0ffc70 msgr2=0x7f900c196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:12.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.065+0000 7f901357c700 1 --2- 172.21.15.144:0/2241176461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900c0ffc70 0x7f900c196270 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f900000bda0 tx=0x7f900000be80 comp rx=0 tx=0).stop 2024-09-10T11:41:12.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.065+0000 7f901357c700 1 -- 172.21.15.144:0/2241176461 shutdown_connections 2024-09-10T11:41:12.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.065+0000 7f901357c700 1 --2- 172.21.15.144:0/2241176461 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8ff406e200 0x7f8ff40706b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.065+0000 7f901357c700 1 --2- 172.21.15.144:0/2241176461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f900c0ff2d0 0x7f900c195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.065+0000 7f901357c700 1 --2- 172.21.15.144:0/2241176461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f900c0ffc70 0x7f900c196270 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.065+0000 7f901357c700 1 -- 172.21.15.144:0/2241176461 >> 172.21.15.144:0/2241176461 conn(0x7f900c0fabb0 msgr2=0x7f900c068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:12.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.065+0000 7f901357c700 1 -- 172.21.15.144:0/2241176461 shutdown_connections 2024-09-10T11:41:12.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.065+0000 7f901357c700 1 -- 172.21.15.144:0/2241176461 wait complete. 2024-09-10T11:41:12.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.171+0000 7f089d73b700 1 -- 172.21.15.144:0/2008522671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0898101c30 msgr2=0x7f089810a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:12.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.171+0000 7f089d73b700 1 --2- 172.21.15.144:0/2008522671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0898101c30 0x7f089810a130 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0880009910 tx=0x7f0880009c20 comp rx=0 tx=0).stop 2024-09-10T11:41:12.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.171+0000 7f089d73b700 1 -- 172.21.15.144:0/2008522671 shutdown_connections 2024-09-10T11:41:12.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.171+0000 7f089d73b700 1 --2- 172.21.15.144:0/2008522671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0898101c30 0x7f089810a130 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.171+0000 7f089d73b700 1 --2- 172.21.15.144:0/2008522671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0898101320 0x7f08981016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.171+0000 7f089d73b700 1 -- 172.21.15.144:0/2008522671 >> 172.21.15.144:0/2008522671 conn(0x7f08980fab80 msgr2=0x7f08980fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:12.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.171+0000 7f089d73b700 1 -- 172.21.15.144:0/2008522671 shutdown_connections 2024-09-10T11:41:12.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.171+0000 7f089d73b700 1 -- 172.21.15.144:0/2008522671 wait complete. 2024-09-10T11:41:12.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.172+0000 7f089d73b700 1 Processor -- start 2024-09-10T11:41:12.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.172+0000 7f089d73b700 1 -- start start 2024-09-10T11:41:12.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.172+0000 7f089d73b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0898101320 0x7f08980fef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:12.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.172+0000 7f089d73b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0898101c30 0x7f08980ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:12.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.172+0000 7f089d73b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08980ffa00 con 0x7f0898101320 2024-09-10T11:41:12.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.172+0000 7f089d73b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08980ffb40 con 0x7f0898101c30 2024-09-10T11:41:12.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.172+0000 7f0896ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0898101320 0x7f08980fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:12.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.172+0000 7f0896ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0898101320 0x7f08980fef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52856/0 (socket says 172.21.15.144:52856) 2024-09-10T11:41:12.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.172+0000 7f0896ffd700 1 -- 172.21.15.144:0/3104774053 learned_addr learned my addr 172.21.15.144:0/3104774053 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:41:12.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.173+0000 7f08967fc700 1 --2- 172.21.15.144:0/3104774053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0898101c30 0x7f08980ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:12.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.173+0000 7f0896ffd700 1 -- 172.21.15.144:0/3104774053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0898101c30 msgr2=0x7f08980ff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:12.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.173+0000 7f0896ffd700 1 --2- 172.21.15.144:0/3104774053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0898101c30 0x7f08980ff4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.173+0000 7f0896ffd700 1 -- 172.21.15.144:0/3104774053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800095f0 con 0x7f0898101320 2024-09-10T11:41:12.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.173+0000 7f08967fc700 1 --2- 172.21.15.144:0/3104774053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0898101c30 0x7f08980ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:41:12.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.173+0000 7f0896ffd700 1 --2- 172.21.15.144:0/3104774053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0898101320 0x7f08980fef80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f088800ef10 tx=0x7f088800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:12.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.173+0000 7f088ffff700 1 -- 172.21.15.144:0/3104774053 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f088800ca50 con 0x7f0898101320 2024-09-10T11:41:12.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.173+0000 7f088ffff700 1 -- 172.21.15.144:0/3104774053 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f088800f040 con 0x7f0898101320 2024-09-10T11:41:12.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.173+0000 7f089d73b700 1 -- 172.21.15.144:0/3104774053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08980ffdc0 con 0x7f0898101320 2024-09-10T11:41:12.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.173+0000 7f088ffff700 1 -- 172.21.15.144:0/3104774053 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08880146b0 con 0x7f0898101320 2024-09-10T11:41:12.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.173+0000 7f089d73b700 1 -- 172.21.15.144:0/3104774053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08981a2240 con 0x7f0898101320 2024-09-10T11:41:12.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.174+0000 7f089d73b700 1 -- 172.21.15.144:0/3104774053 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08981077a0 con 0x7f0898101320 2024-09-10T11:41:12.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.175+0000 7f088ffff700 1 -- 172.21.15.144:0/3104774053 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f088800cbb0 con 0x7f0898101320 2024-09-10T11:41:12.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.175+0000 7f088ffff700 1 --2- 172.21.15.144:0/3104774053 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f088406e200 0x7f08840706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:12.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.175+0000 7f088ffff700 1 -- 172.21.15.144:0/3104774053 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f088808e480 con 0x7f0898101320 2024-09-10T11:41:12.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.175+0000 7f08967fc700 1 --2- 172.21.15.144:0/3104774053 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f088406e200 0x7f08840706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:12.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.175+0000 7f08967fc700 1 --2- 172.21.15.144:0/3104774053 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f088406e200 0x7f08840706b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f088000b760 tx=0x7f0880009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:12.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.179+0000 7f088ffff700 1 -- 172.21.15.144:0/3104774053 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0888057860 con 0x7f0898101320 2024-09-10T11:41:12.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.388+0000 7f089d73b700 1 -- 172.21.15.144:0/3104774053 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0898061190 con 0x7f088406e200 2024-09-10T11:41:12.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.398+0000 7f088ffff700 1 -- 172.21.15.144:0/3104774053 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f0898061190 con 0x7f088406e200 2024-09-10T11:41:12.400 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:41:12.400 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (9m) 3m ago 19m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:41:12.400 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (20m) 3m ago 20m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:41:12.401 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (19m) 2m ago 19m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:41:12.401 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (7m) 3m ago 20m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:41:12.401 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (7m) 2m ago 19m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:41:12.401 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (9m) 3m ago 19m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:41:12.401 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (3m) 3m ago 14m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:41:12.401 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (3m) 3m ago 14m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:41:12.401 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (14m) 2m ago 14m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:41:12.401 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (2m) 2m ago 14m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:41:12.401 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (11m) 3m ago 21m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:41:12.401 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (10m) 2m ago 19m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:41:12.401 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (8m) 3m ago 21m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:41:12.401 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (7m) 2m ago 18m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:41:12.401 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (10m) 3m ago 20m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:41:12.402 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (9m) 2m ago 19m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:41:12.402 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (6m) 3m ago 18m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:41:12.402 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (6m) 3m ago 17m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:41:12.402 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (5m) 3m ago 17m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:41:12.402 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (4m) 2m ago 16m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:41:12.402 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (4m) 2m ago 16m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:41:12.402 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (3m) 2m ago 16m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:41:12.402 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (9m) 3m ago 19m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:41:12.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.400+0000 7f089d73b700 1 -- 172.21.15.144:0/3104774053 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f088406e200 msgr2=0x7f08840706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:12.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.400+0000 7f089d73b700 1 --2- 172.21.15.144:0/3104774053 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f088406e200 0x7f08840706b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f088000b760 tx=0x7f0880009f70 comp rx=0 tx=0).stop 2024-09-10T11:41:12.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.400+0000 7f089d73b700 1 -- 172.21.15.144:0/3104774053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0898101320 msgr2=0x7f08980fef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:12.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.400+0000 7f089d73b700 1 --2- 172.21.15.144:0/3104774053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0898101320 0x7f08980fef80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f088800ef10 tx=0x7f088800c3a0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.400+0000 7f089d73b700 1 -- 172.21.15.144:0/3104774053 shutdown_connections 2024-09-10T11:41:12.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.400+0000 7f089d73b700 1 --2- 172.21.15.144:0/3104774053 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f088406e200 0x7f08840706b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.400+0000 7f089d73b700 1 --2- 172.21.15.144:0/3104774053 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0898101320 0x7f08980fef80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.400+0000 7f089d73b700 1 --2- 172.21.15.144:0/3104774053 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0898101c30 0x7f08980ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.400+0000 7f089d73b700 1 -- 172.21.15.144:0/3104774053 >> 172.21.15.144:0/3104774053 conn(0x7f08980fab80 msgr2=0x7f08981048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:12.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.400+0000 7f089d73b700 1 -- 172.21.15.144:0/3104774053 shutdown_connections 2024-09-10T11:41:12.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.400+0000 7f089d73b700 1 -- 172.21.15.144:0/3104774053 wait complete. 2024-09-10T11:41:12.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.504+0000 7fcf44b6e700 1 -- 172.21.15.144:0/71810113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf400ff2d0 msgr2=0x7fcf400ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:12.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.504+0000 7fcf44b6e700 1 --2- 172.21.15.144:0/71810113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf400ff2d0 0x7fcf400ff6a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fcf30009910 tx=0x7fcf30009c20 comp rx=0 tx=0).stop 2024-09-10T11:41:12.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.504+0000 7fcf44b6e700 1 -- 172.21.15.144:0/71810113 shutdown_connections 2024-09-10T11:41:12.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.504+0000 7fcf44b6e700 1 --2- 172.21.15.144:0/71810113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf400ffc70 0x7fcf40103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.504+0000 7fcf44b6e700 1 --2- 172.21.15.144:0/71810113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf400ff2d0 0x7fcf400ff6a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.504+0000 7fcf44b6e700 1 -- 172.21.15.144:0/71810113 >> 172.21.15.144:0/71810113 conn(0x7fcf400fabb0 msgr2=0x7fcf400fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:12.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.504+0000 7fcf44b6e700 1 -- 172.21.15.144:0/71810113 shutdown_connections 2024-09-10T11:41:12.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.504+0000 7fcf44b6e700 1 -- 172.21.15.144:0/71810113 wait complete. 2024-09-10T11:41:12.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.505+0000 7fcf44b6e700 1 Processor -- start 2024-09-10T11:41:12.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.505+0000 7fcf44b6e700 1 -- start start 2024-09-10T11:41:12.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.505+0000 7fcf44b6e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf400ff2d0 0x7fcf40100ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:12.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.505+0000 7fcf44b6e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf400ffc70 0x7fcf40101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:12.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.505+0000 7fcf44b6e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf40101760 con 0x7fcf400ffc70 2024-09-10T11:41:12.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.505+0000 7fcf44b6e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf401018a0 con 0x7fcf400ff2d0 2024-09-10T11:41:12.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.505+0000 7fcf3dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf400ffc70 0x7fcf40101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:12.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.505+0000 7fcf3dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf400ffc70 0x7fcf40101220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57756/0 (socket says 172.21.15.144:57756) 2024-09-10T11:41:12.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.505+0000 7fcf3dd9b700 1 -- 172.21.15.144:0/2177921061 learned_addr learned my addr 172.21.15.144:0/2177921061 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:41:12.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.505+0000 7fcf3e59c700 1 --2- 172.21.15.144:0/2177921061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf400ff2d0 0x7fcf40100ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:12.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.506+0000 7fcf3dd9b700 1 -- 172.21.15.144:0/2177921061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf400ff2d0 msgr2=0x7fcf40100ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:12.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.506+0000 7fcf3dd9b700 1 --2- 172.21.15.144:0/2177921061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf400ff2d0 0x7fcf40100ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.506+0000 7fcf3dd9b700 1 -- 172.21.15.144:0/2177921061 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf300095f0 con 0x7fcf400ffc70 2024-09-10T11:41:12.510 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.506+0000 7fcf3dd9b700 1 --2- 172.21.15.144:0/2177921061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf400ffc70 0x7fcf40101220 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800d900 tx=0x7fcf2800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:12.510 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.506+0000 7fcf377fe700 1 -- 172.21.15.144:0/2177921061 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf28009900 con 0x7fcf400ffc70 2024-09-10T11:41:12.510 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.506+0000 7fcf377fe700 1 -- 172.21.15.144:0/2177921061 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf28010460 con 0x7fcf400ffc70 2024-09-10T11:41:12.510 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.506+0000 7fcf44b6e700 1 -- 172.21.15.144:0/2177921061 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf401a1f30 con 0x7fcf400ffc70 2024-09-10T11:41:12.510 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.506+0000 7fcf377fe700 1 -- 172.21.15.144:0/2177921061 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf2800f5f0 con 0x7fcf400ffc70 2024-09-10T11:41:12.510 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.506+0000 7fcf44b6e700 1 -- 172.21.15.144:0/2177921061 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf401a2340 con 0x7fcf400ffc70 2024-09-10T11:41:12.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.507+0000 7fcf44b6e700 1 -- 172.21.15.144:0/2177921061 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf4004ea50 con 0x7fcf400ffc70 2024-09-10T11:41:12.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.508+0000 7fcf377fe700 1 -- 172.21.15.144:0/2177921061 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fcf280105d0 con 0x7fcf400ffc70 2024-09-10T11:41:12.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.508+0000 7fcf377fe700 1 --2- 172.21.15.144:0/2177921061 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcf2c06e200 0x7fcf2c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:12.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.508+0000 7fcf377fe700 1 -- 172.21.15.144:0/2177921061 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcf2808dc20 con 0x7fcf400ffc70 2024-09-10T11:41:12.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.508+0000 7fcf3e59c700 1 --2- 172.21.15.144:0/2177921061 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcf2c06e200 0x7fcf2c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:12.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.508+0000 7fcf3e59c700 1 --2- 172.21.15.144:0/2177921061 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcf2c06e200 0x7fcf2c0706b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000b5c0 tx=0x7fcf3001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:12.514 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.512+0000 7fcf377fe700 1 -- 172.21.15.144:0/2177921061 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcf28057000 con 0x7fcf400ffc70 2024-09-10T11:41:12.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.797+0000 7fcf44b6e700 1 -- 172.21.15.144:0/2177921061 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcf40101a60 con 0x7fcf400ffc70 2024-09-10T11:41:12.800 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:12 smithi144 ceph-mon[87436]: pgmap v276: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:12.800 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:12 smithi144 ceph-mon[87436]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:12.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.798+0000 7fcf377fe700 1 -- 172.21.15.144:0/2177921061 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fcf40101a60 con 0x7fcf400ffc70 2024-09-10T11:41:12.800 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:41:12.801 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:41:12.801 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:41:12.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:41:12.801 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:41:12.801 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:41:12.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:41:12.801 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:41:12.801 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:41:12.801 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:41:12.801 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:41:12.801 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:41:12.802 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:41:12.802 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:41:12.802 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:41:12.802 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:41:12.802 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:41:12.802 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:41:12.802 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:41:12.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.800+0000 7fcf44b6e700 1 -- 172.21.15.144:0/2177921061 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcf2c06e200 msgr2=0x7fcf2c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:12.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.800+0000 7fcf44b6e700 1 --2- 172.21.15.144:0/2177921061 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcf2c06e200 0x7fcf2c0706b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcf3000b5c0 tx=0x7fcf3001a040 comp rx=0 tx=0).stop 2024-09-10T11:41:12.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.800+0000 7fcf44b6e700 1 -- 172.21.15.144:0/2177921061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf400ffc70 msgr2=0x7fcf40101220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:12.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.800+0000 7fcf44b6e700 1 --2- 172.21.15.144:0/2177921061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf400ffc70 0x7fcf40101220 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fcf2800d900 tx=0x7fcf2800dc10 comp rx=0 tx=0).stop 2024-09-10T11:41:12.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.800+0000 7fcf44b6e700 1 -- 172.21.15.144:0/2177921061 shutdown_connections 2024-09-10T11:41:12.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.800+0000 7fcf44b6e700 1 --2- 172.21.15.144:0/2177921061 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcf2c06e200 0x7fcf2c0706b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.800+0000 7fcf44b6e700 1 --2- 172.21.15.144:0/2177921061 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf400ff2d0 0x7fcf40100ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.800+0000 7fcf44b6e700 1 --2- 172.21.15.144:0/2177921061 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf400ffc70 0x7fcf40101220 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.800+0000 7fcf44b6e700 1 -- 172.21.15.144:0/2177921061 >> 172.21.15.144:0/2177921061 conn(0x7fcf400fabb0 msgr2=0x7fcf40103390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:12.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.800+0000 7fcf44b6e700 1 -- 172.21.15.144:0/2177921061 shutdown_connections 2024-09-10T11:41:12.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.800+0000 7fcf44b6e700 1 -- 172.21.15.144:0/2177921061 wait complete. 2024-09-10T11:41:12.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.906+0000 7f4ee3a32700 1 -- 172.21.15.144:0/2786808791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc0ffcb0 msgr2=0x7f4edc103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:12.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.906+0000 7f4ee3a32700 1 --2- 172.21.15.144:0/2786808791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc0ffcb0 0x7f4edc103c80 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009910 tx=0x7f4ed0009c20 comp rx=0 tx=0).stop 2024-09-10T11:41:12.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.906+0000 7f4ee3a32700 1 -- 172.21.15.144:0/2786808791 shutdown_connections 2024-09-10T11:41:12.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.906+0000 7f4ee3a32700 1 --2- 172.21.15.144:0/2786808791 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc0ffcb0 0x7f4edc103c80 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.906+0000 7f4ee3a32700 1 --2- 172.21.15.144:0/2786808791 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4edc0ff310 0x7f4edc0ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.906+0000 7f4ee3a32700 1 -- 172.21.15.144:0/2786808791 >> 172.21.15.144:0/2786808791 conn(0x7f4edc0fabb0 msgr2=0x7f4edc0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:12.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.906+0000 7f4ee3a32700 1 -- 172.21.15.144:0/2786808791 shutdown_connections 2024-09-10T11:41:12.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.906+0000 7f4ee3a32700 1 -- 172.21.15.144:0/2786808791 wait complete. 2024-09-10T11:41:12.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.906+0000 7f4ee3a32700 1 Processor -- start 2024-09-10T11:41:12.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.907+0000 7f4ee3a32700 1 -- start start 2024-09-10T11:41:12.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.907+0000 7f4ee3a32700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc0ff310 0x7f4edc196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:12.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.907+0000 7f4ee3a32700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4edc0ffcb0 0x7f4edc197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:12.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.907+0000 7f4ee3a32700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc197750 con 0x7f4edc0ff310 2024-09-10T11:41:12.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.907+0000 7f4ee3a32700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc197890 con 0x7f4edc0ffcb0 2024-09-10T11:41:12.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.907+0000 7f4ee17ce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc0ff310 0x7f4edc196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:12.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.907+0000 7f4ee17ce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc0ff310 0x7f4edc196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57782/0 (socket says 172.21.15.144:57782) 2024-09-10T11:41:12.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.907+0000 7f4ee17ce700 1 -- 172.21.15.144:0/4169687064 learned_addr learned my addr 172.21.15.144:0/4169687064 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:41:12.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.907+0000 7f4ee0fcd700 1 --2- 172.21.15.144:0/4169687064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4edc0ffcb0 0x7f4edc197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:12.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.907+0000 7f4ee17ce700 1 -- 172.21.15.144:0/4169687064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4edc0ffcb0 msgr2=0x7f4edc197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:12.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.907+0000 7f4ee17ce700 1 --2- 172.21.15.144:0/4169687064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4edc0ffcb0 0x7f4edc197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:12.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.907+0000 7f4ee17ce700 1 -- 172.21.15.144:0/4169687064 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed00095f0 con 0x7f4edc0ff310 2024-09-10T11:41:12.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.907+0000 7f4ee17ce700 1 --2- 172.21.15.144:0/4169687064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc0ff310 0x7f4edc196bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400d900 tx=0x7f4ed400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:12.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.907+0000 7f4eda7fc700 1 -- 172.21.15.144:0/4169687064 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ed40041f0 con 0x7f4edc0ff310 2024-09-10T11:41:12.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.908+0000 7f4ee3a32700 1 -- 172.21.15.144:0/4169687064 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc19fc60 con 0x7f4edc0ff310 2024-09-10T11:41:12.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.908+0000 7f4eda7fc700 1 -- 172.21.15.144:0/4169687064 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ed4004cf0 con 0x7f4edc0ff310 2024-09-10T11:41:12.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.908+0000 7f4eda7fc700 1 -- 172.21.15.144:0/4169687064 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ed400b750 con 0x7f4edc0ff310 2024-09-10T11:41:12.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.908+0000 7f4ee3a32700 1 -- 172.21.15.144:0/4169687064 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc1a0120 con 0x7f4edc0ff310 2024-09-10T11:41:12.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.909+0000 7f4ee3a32700 1 -- 172.21.15.144:0/4169687064 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4edc107680 con 0x7f4edc0ff310 2024-09-10T11:41:12.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.909+0000 7f4eda7fc700 1 -- 172.21.15.144:0/4169687064 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4ed4004350 con 0x7f4edc0ff310 2024-09-10T11:41:12.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.910+0000 7f4eda7fc700 1 --2- 172.21.15.144:0/4169687064 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ec406e200 0x7f4ec40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:12.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.910+0000 7f4ee0fcd700 1 --2- 172.21.15.144:0/4169687064 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ec406e200 0x7f4ec40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:12.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.910+0000 7f4eda7fc700 1 -- 172.21.15.144:0/4169687064 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4ed408e1a0 con 0x7f4edc0ff310 2024-09-10T11:41:12.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.910+0000 7f4ee0fcd700 1 --2- 172.21.15.144:0/4169687064 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ec406e200 0x7f4ec40706b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006010 tx=0x7f4ed001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:12.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:12.914+0000 7f4eda7fc700 1 -- 172.21.15.144:0/4169687064 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4ed4057580 con 0x7f4edc0ff310 2024-09-10T11:41:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:12 smithi178 ceph-mon[65940]: pgmap v276: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:12 smithi178 ceph-mon[65940]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:13.161 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.157+0000 7f4ee3a32700 1 -- 172.21.15.144:0/4169687064 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4edc02ccd0 con 0x7f4edc0ff310 2024-09-10T11:41:13.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.157+0000 7f4eda7fc700 1 -- 172.21.15.144:0/4169687064 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f4ed405ac40 con 0x7f4edc0ff310 2024-09-10T11:41:13.162 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:41:13.162 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:41:13.162 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:41:13.163 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:41:13.163 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:41:13.163 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:41:13.163 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:41:13.163 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:41:13.163 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:41:13.164 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:41:13.164 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:41:13.164 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:41:13.164 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:41:13.164 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:41:13.165 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:41:13.165 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:41:13.165 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:41:13.165 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:41:13.165 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:41:13.165 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:41:13.166 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:41:13.166 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:41:13.166 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:41:13.166 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:41:13.166 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:41:13.167 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:41:13.167 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:41:13.167 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:41:13.167 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:41:13.167 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:41:13.167 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:41:13.168 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:41:13.168 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:41:13.168 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:41:13.168 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:41:13.168 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:41:13.168 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:41:13.169 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:41:13.169 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:41:13.169 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:41:13.169 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:41:13.169 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:41:13.170 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:41:13.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.160+0000 7f4ee3a32700 1 -- 172.21.15.144:0/4169687064 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ec406e200 msgr2=0x7f4ec40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:13.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.160+0000 7f4ee3a32700 1 --2- 172.21.15.144:0/4169687064 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ec406e200 0x7f4ec40706b0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0006010 tx=0x7f4ed001a040 comp rx=0 tx=0).stop 2024-09-10T11:41:13.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.160+0000 7f4ee3a32700 1 -- 172.21.15.144:0/4169687064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc0ff310 msgr2=0x7f4edc196bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:13.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.160+0000 7f4ee3a32700 1 --2- 172.21.15.144:0/4169687064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc0ff310 0x7f4edc196bf0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400d900 tx=0x7f4ed400dcc0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.160+0000 7f4ee3a32700 1 -- 172.21.15.144:0/4169687064 shutdown_connections 2024-09-10T11:41:13.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.160+0000 7f4ee3a32700 1 --2- 172.21.15.144:0/4169687064 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ec406e200 0x7f4ec40706b0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.160+0000 7f4ee3a32700 1 --2- 172.21.15.144:0/4169687064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4edc0ff310 0x7f4edc196bf0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.160+0000 7f4ee3a32700 1 --2- 172.21.15.144:0/4169687064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4edc0ffcb0 0x7f4edc197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.160+0000 7f4ee3a32700 1 -- 172.21.15.144:0/4169687064 >> 172.21.15.144:0/4169687064 conn(0x7f4edc0fabb0 msgr2=0x7f4edc0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:13.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.160+0000 7f4ee3a32700 1 -- 172.21.15.144:0/4169687064 shutdown_connections 2024-09-10T11:41:13.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.160+0000 7f4ee3a32700 1 -- 172.21.15.144:0/4169687064 wait complete. 2024-09-10T11:41:13.173 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:41:13.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.265+0000 7f4a672e1700 1 -- 172.21.15.144:0/2945903771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a600ff270 msgr2=0x7f4a600ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:13.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.265+0000 7f4a672e1700 1 --2- 172.21.15.144:0/2945903771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a600ff270 0x7f4a600ff6e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4a50009910 tx=0x7f4a50009c20 comp rx=0 tx=0).stop 2024-09-10T11:41:13.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.265+0000 7f4a672e1700 1 -- 172.21.15.144:0/2945903771 shutdown_connections 2024-09-10T11:41:13.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.265+0000 7f4a672e1700 1 --2- 172.21.15.144:0/2945903771 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a600ff270 0x7f4a600ff6e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.266+0000 7f4a672e1700 1 --2- 172.21.15.144:0/2945903771 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a601049a0 0x7f4a60104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.266+0000 7f4a672e1700 1 -- 172.21.15.144:0/2945903771 >> 172.21.15.144:0/2945903771 conn(0x7f4a60074850 msgr2=0x7f4a60074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:13.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.266+0000 7f4a672e1700 1 -- 172.21.15.144:0/2945903771 shutdown_connections 2024-09-10T11:41:13.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.266+0000 7f4a672e1700 1 -- 172.21.15.144:0/2945903771 wait complete. 2024-09-10T11:41:13.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.266+0000 7f4a672e1700 1 Processor -- start 2024-09-10T11:41:13.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.266+0000 7f4a672e1700 1 -- start start 2024-09-10T11:41:13.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.266+0000 7f4a672e1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a600ff270 0x7f4a60196b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:13.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.266+0000 7f4a672e1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a601049a0 0x7f4a601970d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:13.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.266+0000 7f4a672e1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a601976f0 con 0x7f4a600ff270 2024-09-10T11:41:13.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.266+0000 7f4a672e1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a60197830 con 0x7f4a601049a0 2024-09-10T11:41:13.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.266+0000 7f4a6507d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a600ff270 0x7f4a60196b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:13.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.267+0000 7f4a6507d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a600ff270 0x7f4a60196b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57786/0 (socket says 172.21.15.144:57786) 2024-09-10T11:41:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.267+0000 7f4a6507d700 1 -- 172.21.15.144:0/3812218171 learned_addr learned my addr 172.21.15.144:0/3812218171 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:41:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.267+0000 7f4a6487c700 1 --2- 172.21.15.144:0/3812218171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a601049a0 0x7f4a601970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.267+0000 7f4a6507d700 1 -- 172.21.15.144:0/3812218171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a601049a0 msgr2=0x7f4a601970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.267+0000 7f4a6507d700 1 --2- 172.21.15.144:0/3812218171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a601049a0 0x7f4a601970d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.267+0000 7f4a6507d700 1 -- 172.21.15.144:0/3812218171 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a500095f0 con 0x7f4a600ff270 2024-09-10T11:41:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.267+0000 7f4a6487c700 1 --2- 172.21.15.144:0/3812218171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a601049a0 0x7f4a601970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:41:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.267+0000 7f4a6507d700 1 --2- 172.21.15.144:0/3812218171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a600ff270 0x7f4a60196b90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00c8a0 tx=0x7f4a5c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.267+0000 7f4a567fc700 1 -- 172.21.15.144:0/3812218171 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a5c00cea0 con 0x7f4a600ff270 2024-09-10T11:41:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.267+0000 7f4a672e1700 1 -- 172.21.15.144:0/3812218171 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a6019fc00 con 0x7f4a600ff270 2024-09-10T11:41:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.267+0000 7f4a567fc700 1 -- 172.21.15.144:0/3812218171 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4a5c004830 con 0x7f4a600ff270 2024-09-10T11:41:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.267+0000 7f4a567fc700 1 -- 172.21.15.144:0/3812218171 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a5c0055b0 con 0x7f4a600ff270 2024-09-10T11:41:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.267+0000 7f4a672e1700 1 -- 172.21.15.144:0/3812218171 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a601a00c0 con 0x7f4a600ff270 2024-09-10T11:41:13.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.268+0000 7f4a672e1700 1 -- 172.21.15.144:0/3812218171 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a6004ea50 con 0x7f4a600ff270 2024-09-10T11:41:13.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.269+0000 7f4a567fc700 1 -- 172.21.15.144:0/3812218171 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4a5c0049a0 con 0x7f4a600ff270 2024-09-10T11:41:13.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.269+0000 7f4a567fc700 1 --2- 172.21.15.144:0/3812218171 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4a4c06e200 0x7f4a4c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:13.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.269+0000 7f4a567fc700 1 -- 172.21.15.144:0/3812218171 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4a5c08dd40 con 0x7f4a600ff270 2024-09-10T11:41:13.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.269+0000 7f4a6487c700 1 --2- 172.21.15.144:0/3812218171 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4a4c06e200 0x7f4a4c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:13.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.270+0000 7f4a6487c700 1 --2- 172.21.15.144:0/3812218171 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4a4c06e200 0x7f4a4c0706b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4a600fe7d0 tx=0x7f4a5001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:13.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.273+0000 7f4a567fc700 1 -- 172.21.15.144:0/3812218171 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4a5c057120 con 0x7f4a600ff270 2024-09-10T11:41:13.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.494+0000 7f4a672e1700 1 -- 172.21.15.144:0/3812218171 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a60108720 con 0x7f4a4c06e200 2024-09-10T11:41:13.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.495+0000 7f4a567fc700 1 -- 172.21.15.144:0/3812218171 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4a60108720 con 0x7f4a4c06e200 2024-09-10T11:41:13.497 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:41:13.497 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:41:13.497 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:41:13.497 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:41:13.497 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:41:13.498 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:41:13.498 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:41:13.498 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:41:13.498 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:41:13.498 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:41:13.498 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:41:13.498 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:41:13.498 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:41:13.498 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:41:13.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.497+0000 7f4a672e1700 1 -- 172.21.15.144:0/3812218171 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4a4c06e200 msgr2=0x7f4a4c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:13.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.497+0000 7f4a672e1700 1 --2- 172.21.15.144:0/3812218171 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4a4c06e200 0x7f4a4c0706b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4a600fe7d0 tx=0x7f4a5001a040 comp rx=0 tx=0).stop 2024-09-10T11:41:13.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.497+0000 7f4a672e1700 1 -- 172.21.15.144:0/3812218171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a600ff270 msgr2=0x7f4a60196b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:13.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.497+0000 7f4a672e1700 1 --2- 172.21.15.144:0/3812218171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a600ff270 0x7f4a60196b90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f4a5c00c8a0 tx=0x7f4a5c00cc60 comp rx=0 tx=0).stop 2024-09-10T11:41:13.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.497+0000 7f4a672e1700 1 -- 172.21.15.144:0/3812218171 shutdown_connections 2024-09-10T11:41:13.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.497+0000 7f4a672e1700 1 --2- 172.21.15.144:0/3812218171 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4a4c06e200 0x7f4a4c0706b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.497+0000 7f4a672e1700 1 --2- 172.21.15.144:0/3812218171 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4a600ff270 0x7f4a60196b90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.497+0000 7f4a672e1700 1 --2- 172.21.15.144:0/3812218171 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4a601049a0 0x7f4a601970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.497+0000 7f4a672e1700 1 -- 172.21.15.144:0/3812218171 >> 172.21.15.144:0/3812218171 conn(0x7f4a60074850 msgr2=0x7f4a60108010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:13.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.497+0000 7f4a672e1700 1 -- 172.21.15.144:0/3812218171 shutdown_connections 2024-09-10T11:41:13.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.497+0000 7f4a672e1700 1 -- 172.21.15.144:0/3812218171 wait complete. 2024-09-10T11:41:13.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.602+0000 7f9c9df6c700 1 -- 172.21.15.144:0/1204471994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c98100d70 msgr2=0x7f9c98104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:13.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.602+0000 7f9c9df6c700 1 --2- 172.21.15.144:0/1204471994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c98100d70 0x7f9c98104d40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c009910 tx=0x7f9c8c009c20 comp rx=0 tx=0).stop 2024-09-10T11:41:13.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.603+0000 7f9c9df6c700 1 -- 172.21.15.144:0/1204471994 shutdown_connections 2024-09-10T11:41:13.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.603+0000 7f9c9df6c700 1 --2- 172.21.15.144:0/1204471994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c98100d70 0x7f9c98104d40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.603+0000 7f9c9df6c700 1 --2- 172.21.15.144:0/1204471994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c981003d0 0x7f9c981007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.603+0000 7f9c9df6c700 1 -- 172.21.15.144:0/1204471994 >> 172.21.15.144:0/1204471994 conn(0x7f9c980fbc90 msgr2=0x7f9c980fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:13.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.603+0000 7f9c9df6c700 1 -- 172.21.15.144:0/1204471994 shutdown_connections 2024-09-10T11:41:13.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.603+0000 7f9c9df6c700 1 -- 172.21.15.144:0/1204471994 wait complete. 2024-09-10T11:41:13.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.603+0000 7f9c9df6c700 1 Processor -- start 2024-09-10T11:41:13.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.603+0000 7f9c9df6c700 1 -- start start 2024-09-10T11:41:13.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.604+0000 7f9c9df6c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c981003d0 0x7f9c98195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:13.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.604+0000 7f9c9df6c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c98100d70 0x7f9c981963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:13.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.604+0000 7f9c9df6c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c98196aa0 con 0x7f9c98100d70 2024-09-10T11:41:13.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.604+0000 7f9c9df6c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c9819a830 con 0x7f9c981003d0 2024-09-10T11:41:13.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.604+0000 7f9c96ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c98100d70 0x7f9c981963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:13.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.604+0000 7f9c96ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c98100d70 0x7f9c981963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57804/0 (socket says 172.21.15.144:57804) 2024-09-10T11:41:13.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.604+0000 7f9c96ffd700 1 -- 172.21.15.144:0/1122682156 learned_addr learned my addr 172.21.15.144:0/1122682156 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:41:13.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.604+0000 7f9c977fe700 1 --2- 172.21.15.144:0/1122682156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c981003d0 0x7f9c98195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:13.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.604+0000 7f9c96ffd700 1 -- 172.21.15.144:0/1122682156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c981003d0 msgr2=0x7f9c98195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:13.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.604+0000 7f9c96ffd700 1 --2- 172.21.15.144:0/1122682156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c981003d0 0x7f9c98195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.604+0000 7f9c96ffd700 1 -- 172.21.15.144:0/1122682156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c8c0095f0 con 0x7f9c98100d70 2024-09-10T11:41:13.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.604+0000 7f9c96ffd700 1 --2- 172.21.15.144:0/1122682156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c98100d70 0x7f9c981963c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00bda0 tx=0x7f9c8c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:13.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.604+0000 7f9c94ff9700 1 -- 172.21.15.144:0/1122682156 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c8c01d070 con 0x7f9c98100d70 2024-09-10T11:41:13.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.605+0000 7f9c9df6c700 1 -- 172.21.15.144:0/1122682156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c9819aab0 con 0x7f9c98100d70 2024-09-10T11:41:13.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.605+0000 7f9c94ff9700 1 -- 172.21.15.144:0/1122682156 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9c8c00f460 con 0x7f9c98100d70 2024-09-10T11:41:13.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.605+0000 7f9c94ff9700 1 -- 172.21.15.144:0/1122682156 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c8c017630 con 0x7f9c98100d70 2024-09-10T11:41:13.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.605+0000 7f9c9df6c700 1 -- 172.21.15.144:0/1122682156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c9819afa0 con 0x7f9c98100d70 2024-09-10T11:41:13.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.606+0000 7f9c9df6c700 1 -- 172.21.15.144:0/1122682156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c98108690 con 0x7f9c98100d70 2024-09-10T11:41:13.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.606+0000 7f9c94ff9700 1 -- 172.21.15.144:0/1122682156 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9c8c00fab0 con 0x7f9c98100d70 2024-09-10T11:41:13.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.607+0000 7f9c94ff9700 1 --2- 172.21.15.144:0/1122682156 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9c8006e200 0x7f9c800706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:13.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.607+0000 7f9c94ff9700 1 -- 172.21.15.144:0/1122682156 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9c8c08f550 con 0x7f9c98100d70 2024-09-10T11:41:13.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.607+0000 7f9c977fe700 1 --2- 172.21.15.144:0/1122682156 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9c8006e200 0x7f9c800706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:13.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.607+0000 7f9c977fe700 1 --2- 172.21.15.144:0/1122682156 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9c8006e200 0x7f9c800706b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009a10 tx=0x7f9c88009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:13.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.611+0000 7f9c94ff9700 1 -- 172.21.15.144:0/1122682156 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9c8c058930 con 0x7f9c98100d70 2024-09-10T11:41:13.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.905+0000 7f9c9df6c700 1 -- 172.21.15.144:0/1122682156 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f9c9802cd20 con 0x7f9c98100d70 2024-09-10T11:41:13.907 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:13 smithi144 ceph-mon[87436]: from='client.34572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:13.908 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:13 smithi144 ceph-mon[87436]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:13.908 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:13 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2177921061' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:41:13.908 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:13 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/4169687064' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:41:13.908 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:41:13.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.905+0000 7f9c94ff9700 1 -- 172.21.15.144:0/1122682156 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f9c8c05bff0 con 0x7f9c98100d70 2024-09-10T11:41:13.908 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:41:13.909 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:41:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:41:13.909 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:41:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:41:13.909 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:41:13.909 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:41:13.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.907+0000 7f9c9df6c700 1 -- 172.21.15.144:0/1122682156 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9c8006e200 msgr2=0x7f9c800706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:13.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.907+0000 7f9c9df6c700 1 --2- 172.21.15.144:0/1122682156 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9c8006e200 0x7f9c800706b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9c88009a10 tx=0x7f9c88009190 comp rx=0 tx=0).stop 2024-09-10T11:41:13.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.907+0000 7f9c9df6c700 1 -- 172.21.15.144:0/1122682156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c98100d70 msgr2=0x7f9c981963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:13.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.907+0000 7f9c9df6c700 1 --2- 172.21.15.144:0/1122682156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c98100d70 0x7f9c981963c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f9c8c00bda0 tx=0x7f9c8c00bdd0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.908+0000 7f9c9df6c700 1 -- 172.21.15.144:0/1122682156 shutdown_connections 2024-09-10T11:41:13.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.908+0000 7f9c9df6c700 1 --2- 172.21.15.144:0/1122682156 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9c8006e200 0x7f9c800706b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.908+0000 7f9c9df6c700 1 --2- 172.21.15.144:0/1122682156 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c981003d0 0x7f9c98195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.908+0000 7f9c9df6c700 1 --2- 172.21.15.144:0/1122682156 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c98100d70 0x7f9c981963c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:13.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.908+0000 7f9c9df6c700 1 -- 172.21.15.144:0/1122682156 >> 172.21.15.144:0/1122682156 conn(0x7f9c980fbc90 msgr2=0x7f9c980fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:13.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.908+0000 7f9c9df6c700 1 -- 172.21.15.144:0/1122682156 shutdown_connections 2024-09-10T11:41:13.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:13.908+0000 7f9c9df6c700 1 -- 172.21.15.144:0/1122682156 wait complete. 2024-09-10T11:41:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:13 smithi178 ceph-mon[65940]: from='client.34572 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:13 smithi178 ceph-mon[65940]: from='client.34576 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:13 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2177921061' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:41:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:13 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/4169687064' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:41:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:41:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:14 smithi178 ceph-mon[65940]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:14 smithi178 ceph-mon[65940]: pgmap v277: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:14 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1122682156' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:41:15.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:14 smithi144 ceph-mon[87436]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:14 smithi144 ceph-mon[87436]: pgmap v277: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:14 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1122682156' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:41:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:16 smithi178 ceph-mon[65940]: pgmap v278: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:17.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:16 smithi144 ceph-mon[87436]: pgmap v278: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:18 smithi178 ceph-mon[65940]: pgmap v279: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:18 smithi144 ceph-mon[87436]: pgmap v279: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:20 smithi178 ceph-mon[65940]: pgmap v280: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:21.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:20 smithi144 ceph-mon[87436]: pgmap v280: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:23.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:22 smithi144 ceph-mon[87436]: pgmap v281: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:23.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:22 smithi178 ceph-mon[65940]: pgmap v281: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:24.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:24 smithi178 ceph-mon[65940]: pgmap v282: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:24 smithi144 ceph-mon[87436]: pgmap v282: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:26 smithi178 ceph-mon[65940]: pgmap v283: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:27.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:26 smithi144 ceph-mon[87436]: pgmap v283: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:28 smithi178 ceph-mon[65940]: pgmap v284: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:41:29.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:28 smithi144 ceph-mon[87436]: pgmap v284: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:41:31.102 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:30 smithi144 ceph-mon[87436]: pgmap v285: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:30 smithi178 ceph-mon[65940]: pgmap v285: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:32 smithi144 ceph-mon[87436]: pgmap v286: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:33.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:32 smithi178 ceph-mon[65940]: pgmap v286: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:34 smithi178 ceph-mon[65940]: pgmap v287: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:34 smithi144 ceph-mon[87436]: pgmap v287: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:36 smithi178 ceph-mon[65940]: pgmap v288: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:36 smithi144 ceph-mon[87436]: pgmap v288: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:38 smithi178 ceph-mon[65940]: pgmap v289: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:38 smithi144 ceph-mon[87436]: pgmap v289: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:40 smithi178 ceph-mon[65940]: pgmap v290: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:41.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:40 smithi144 ceph-mon[87436]: pgmap v290: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:43.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:42 smithi144 ceph-mon[87436]: pgmap v291: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:43.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:42 smithi178 ceph-mon[65940]: pgmap v291: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:44.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.028+0000 7fa7deb50700 1 -- 172.21.15.144:0/426593230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d80ffbe0 msgr2=0x7fa7d810a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:44.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.028+0000 7fa7deb50700 1 --2- 172.21.15.144:0/426593230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d80ffbe0 0x7fa7d810a650 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4009940 tx=0x7fa7c4009c50 comp rx=0 tx=0).stop 2024-09-10T11:41:44.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.028+0000 7fa7deb50700 1 -- 172.21.15.144:0/426593230 shutdown_connections 2024-09-10T11:41:44.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.028+0000 7fa7deb50700 1 --2- 172.21.15.144:0/426593230 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d80ffbe0 0x7fa7d810a650 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.028+0000 7fa7deb50700 1 --2- 172.21.15.144:0/426593230 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7d80ff2d0 0x7fa7d80ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.028+0000 7fa7deb50700 1 -- 172.21.15.144:0/426593230 >> 172.21.15.144:0/426593230 conn(0x7fa7d80747e0 msgr2=0x7fa7d8074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:44.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.028+0000 7fa7deb50700 1 -- 172.21.15.144:0/426593230 shutdown_connections 2024-09-10T11:41:44.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.028+0000 7fa7deb50700 1 -- 172.21.15.144:0/426593230 wait complete. 2024-09-10T11:41:44.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.028+0000 7fa7deb50700 1 Processor -- start 2024-09-10T11:41:44.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.028+0000 7fa7deb50700 1 -- start start 2024-09-10T11:41:44.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.029+0000 7fa7deb50700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d80ff2d0 0x7fa7d819ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:44.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.029+0000 7fa7deb50700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7d80ffbe0 0x7fa7d819bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:44.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.029+0000 7fa7deb50700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d819c5d0 con 0x7fa7d80ff2d0 2024-09-10T11:41:44.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.029+0000 7fa7deb50700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d8195a80 con 0x7fa7d80ffbe0 2024-09-10T11:41:44.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.029+0000 7fa7dc8ec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d80ff2d0 0x7fa7d819ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:44.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.029+0000 7fa7dc8ec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d80ff2d0 0x7fa7d819ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38084/0 (socket says 172.21.15.144:38084) 2024-09-10T11:41:44.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.029+0000 7fa7dc8ec700 1 -- 172.21.15.144:0/3149795836 learned_addr learned my addr 172.21.15.144:0/3149795836 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:41:44.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.029+0000 7fa7d7fff700 1 --2- 172.21.15.144:0/3149795836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7d80ffbe0 0x7fa7d819bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:44.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.029+0000 7fa7dc8ec700 1 -- 172.21.15.144:0/3149795836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7d80ffbe0 msgr2=0x7fa7d819bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:44.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.029+0000 7fa7dc8ec700 1 --2- 172.21.15.144:0/3149795836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7d80ffbe0 0x7fa7d819bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.029+0000 7fa7dc8ec700 1 -- 172.21.15.144:0/3149795836 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c40095f0 con 0x7fa7d80ff2d0 2024-09-10T11:41:44.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.029+0000 7fa7dc8ec700 1 --2- 172.21.15.144:0/3149795836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d80ff2d0 0x7fa7d819ba00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00d900 tx=0x7fa7cc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:44.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.029+0000 7fa7d5ffb700 1 -- 172.21.15.144:0/3149795836 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7cc004a00 con 0x7fa7d80ff2d0 2024-09-10T11:41:44.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.030+0000 7fa7d5ffb700 1 -- 172.21.15.144:0/3149795836 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7cc0054e0 con 0x7fa7d80ff2d0 2024-09-10T11:41:44.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.030+0000 7fa7deb50700 1 -- 172.21.15.144:0/3149795836 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d8195d60 con 0x7fa7d80ff2d0 2024-09-10T11:41:44.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.030+0000 7fa7d5ffb700 1 -- 172.21.15.144:0/3149795836 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7cc009d00 con 0x7fa7d80ff2d0 2024-09-10T11:41:44.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.030+0000 7fa7deb50700 1 -- 172.21.15.144:0/3149795836 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d8196300 con 0x7fa7d80ff2d0 2024-09-10T11:41:44.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.031+0000 7fa7deb50700 1 -- 172.21.15.144:0/3149795836 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d8107d50 con 0x7fa7d80ff2d0 2024-09-10T11:41:44.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.031+0000 7fa7d5ffb700 1 -- 172.21.15.144:0/3149795836 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa7cc005020 con 0x7fa7d80ff2d0 2024-09-10T11:41:44.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.031+0000 7fa7d5ffb700 1 --2- 172.21.15.144:0/3149795836 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa7c806e200 0x7fa7c80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:44.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.032+0000 7fa7d5ffb700 1 -- 172.21.15.144:0/3149795836 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa7cc08df40 con 0x7fa7d80ff2d0 2024-09-10T11:41:44.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.032+0000 7fa7d7fff700 1 --2- 172.21.15.144:0/3149795836 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa7c806e200 0x7fa7c80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:44.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.032+0000 7fa7d7fff700 1 --2- 172.21.15.144:0/3149795836 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa7c806e200 0x7fa7c80706b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4006010 tx=0x7fa7c4005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:44.037 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.036+0000 7fa7d5ffb700 1 -- 172.21.15.144:0/3149795836 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa7cc0c31b0 con 0x7fa7d80ff2d0 2024-09-10T11:41:44.037 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:41:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:41:44.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.258+0000 7fa7deb50700 1 -- 172.21.15.144:0/3149795836 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7d8196570 con 0x7fa7c806e200 2024-09-10T11:41:44.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.259+0000 7fa7d5ffb700 1 -- 172.21.15.144:0/3149795836 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa7d8196570 con 0x7fa7c806e200 2024-09-10T11:41:44.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.261+0000 7fa7deb50700 1 -- 172.21.15.144:0/3149795836 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa7c806e200 msgr2=0x7fa7c80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:44.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.261+0000 7fa7deb50700 1 --2- 172.21.15.144:0/3149795836 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa7c806e200 0x7fa7c80706b0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa7c4006010 tx=0x7fa7c4005a90 comp rx=0 tx=0).stop 2024-09-10T11:41:44.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.261+0000 7fa7deb50700 1 -- 172.21.15.144:0/3149795836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d80ff2d0 msgr2=0x7fa7d819ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:44.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.261+0000 7fa7deb50700 1 --2- 172.21.15.144:0/3149795836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d80ff2d0 0x7fa7d819ba00 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc00d900 tx=0x7fa7cc00dc10 comp rx=0 tx=0).stop 2024-09-10T11:41:44.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.262+0000 7fa7deb50700 1 -- 172.21.15.144:0/3149795836 shutdown_connections 2024-09-10T11:41:44.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.262+0000 7fa7deb50700 1 --2- 172.21.15.144:0/3149795836 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa7c806e200 0x7fa7c80706b0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.262+0000 7fa7deb50700 1 --2- 172.21.15.144:0/3149795836 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7d80ff2d0 0x7fa7d819ba00 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.262+0000 7fa7deb50700 1 --2- 172.21.15.144:0/3149795836 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7d80ffbe0 0x7fa7d819bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.262+0000 7fa7deb50700 1 -- 172.21.15.144:0/3149795836 >> 172.21.15.144:0/3149795836 conn(0x7fa7d80747e0 msgr2=0x7fa7d8104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:44.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.262+0000 7fa7deb50700 1 -- 172.21.15.144:0/3149795836 shutdown_connections 2024-09-10T11:41:44.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.262+0000 7fa7deb50700 1 -- 172.21.15.144:0/3149795836 wait complete. 2024-09-10T11:41:44.273 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:41:44.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.367+0000 7ff805b97700 1 -- 172.21.15.144:0/3941146829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8001013d0 msgr2=0x7ff8001017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:44.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.367+0000 7ff805b97700 1 --2- 172.21.15.144:0/3941146829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8001013d0 0x7ff8001017a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7ff7f0009910 tx=0x7ff7f0009c20 comp rx=0 tx=0).stop 2024-09-10T11:41:44.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.367+0000 7ff805b97700 1 -- 172.21.15.144:0/3941146829 shutdown_connections 2024-09-10T11:41:44.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.367+0000 7ff805b97700 1 --2- 172.21.15.144:0/3941146829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff800101ce0 0x7ff80010a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.367+0000 7ff805b97700 1 --2- 172.21.15.144:0/3941146829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8001013d0 0x7ff8001017a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.367+0000 7ff805b97700 1 -- 172.21.15.144:0/3941146829 >> 172.21.15.144:0/3941146829 conn(0x7ff8000fabd0 msgr2=0x7ff8000fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:44.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.367+0000 7ff805b97700 1 -- 172.21.15.144:0/3941146829 shutdown_connections 2024-09-10T11:41:44.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.367+0000 7ff805b97700 1 -- 172.21.15.144:0/3941146829 wait complete. 2024-09-10T11:41:44.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.368+0000 7ff805b97700 1 Processor -- start 2024-09-10T11:41:44.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.368+0000 7ff805b97700 1 -- start start 2024-09-10T11:41:44.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.368+0000 7ff805b97700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8001013d0 0x7ff8000ff030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:44.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.368+0000 7ff805b97700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff800101ce0 0x7ff8000ff570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:44.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.368+0000 7ff805b97700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000ffab0 con 0x7ff8001013d0 2024-09-10T11:41:44.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.368+0000 7ff805b97700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8000ffbf0 con 0x7ff800101ce0 2024-09-10T11:41:44.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.369+0000 7ff7ff7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8001013d0 0x7ff8000ff030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:44.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.369+0000 7ff7ff7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8001013d0 0x7ff8000ff030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38096/0 (socket says 172.21.15.144:38096) 2024-09-10T11:41:44.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.369+0000 7ff7ff7fe700 1 -- 172.21.15.144:0/2695610101 learned_addr learned my addr 172.21.15.144:0/2695610101 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:41:44.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.369+0000 7ff7f7fff700 1 --2- 172.21.15.144:0/2695610101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff800101ce0 0x7ff8000ff570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:44.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.369+0000 7ff7ff7fe700 1 -- 172.21.15.144:0/2695610101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff800101ce0 msgr2=0x7ff8000ff570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:44.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.369+0000 7ff7ff7fe700 1 --2- 172.21.15.144:0/2695610101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff800101ce0 0x7ff8000ff570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.369+0000 7ff7ff7fe700 1 -- 172.21.15.144:0/2695610101 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f00095f0 con 0x7ff8001013d0 2024-09-10T11:41:44.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.369+0000 7ff7ff7fe700 1 --2- 172.21.15.144:0/2695610101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8001013d0 0x7ff8000ff030 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00092e0 tx=0x7ff7f000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:44.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.369+0000 7ff7fd7fa700 1 -- 172.21.15.144:0/2695610101 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7f001d070 con 0x7ff8001013d0 2024-09-10T11:41:44.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.369+0000 7ff805b97700 1 -- 172.21.15.144:0/2695610101 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8000ffe70 con 0x7ff8001013d0 2024-09-10T11:41:44.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.369+0000 7ff7fd7fa700 1 -- 172.21.15.144:0/2695610101 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff7f0005e00 con 0x7ff8001013d0 2024-09-10T11:41:44.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.369+0000 7ff7fd7fa700 1 -- 172.21.15.144:0/2695610101 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7f0017400 con 0x7ff8001013d0 2024-09-10T11:41:44.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.369+0000 7ff805b97700 1 -- 172.21.15.144:0/2695610101 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8001a2200 con 0x7ff8001013d0 2024-09-10T11:41:44.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.370+0000 7ff805b97700 1 -- 172.21.15.144:0/2695610101 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8001078b0 con 0x7ff8001013d0 2024-09-10T11:41:44.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.371+0000 7ff7fd7fa700 1 -- 172.21.15.144:0/2695610101 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff7f000f6f0 con 0x7ff8001013d0 2024-09-10T11:41:44.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.372+0000 7ff7fd7fa700 1 --2- 172.21.15.144:0/2695610101 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff7ec06e200 0x7ff7ec0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:44.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.372+0000 7ff7fd7fa700 1 -- 172.21.15.144:0/2695610101 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff7f008f500 con 0x7ff8001013d0 2024-09-10T11:41:44.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.372+0000 7ff7f7fff700 1 --2- 172.21.15.144:0/2695610101 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff7ec06e200 0x7ff7ec0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:44.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.372+0000 7ff7f7fff700 1 --2- 172.21.15.144:0/2695610101 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff7ec06e200 0x7ff7ec0706b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff800100820 tx=0x7ff7e800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:44.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.376+0000 7ff7fd7fa700 1 -- 172.21.15.144:0/2695610101 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff7f00588e0 con 0x7ff8001013d0 2024-09-10T11:41:44.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.595+0000 7ff805b97700 1 -- 172.21.15.144:0/2695610101 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff80002cd40 con 0x7ff7ec06e200 2024-09-10T11:41:44.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.596+0000 7ff7fd7fa700 1 -- 172.21.15.144:0/2695610101 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff80002cd40 con 0x7ff7ec06e200 2024-09-10T11:41:44.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.598+0000 7ff805b97700 1 -- 172.21.15.144:0/2695610101 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff7ec06e200 msgr2=0x7ff7ec0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:44.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.598+0000 7ff805b97700 1 --2- 172.21.15.144:0/2695610101 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff7ec06e200 0x7ff7ec0706b0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ff800100820 tx=0x7ff7e800b410 comp rx=0 tx=0).stop 2024-09-10T11:41:44.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.598+0000 7ff805b97700 1 -- 172.21.15.144:0/2695610101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8001013d0 msgr2=0x7ff8000ff030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:44.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.598+0000 7ff805b97700 1 --2- 172.21.15.144:0/2695610101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8001013d0 0x7ff8000ff030 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7ff7f00092e0 tx=0x7ff7f000bd70 comp rx=0 tx=0).stop 2024-09-10T11:41:44.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.598+0000 7ff805b97700 1 -- 172.21.15.144:0/2695610101 shutdown_connections 2024-09-10T11:41:44.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.598+0000 7ff805b97700 1 --2- 172.21.15.144:0/2695610101 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff7ec06e200 0x7ff7ec0706b0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.598+0000 7ff805b97700 1 --2- 172.21.15.144:0/2695610101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff8001013d0 0x7ff8000ff030 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.598+0000 7ff805b97700 1 --2- 172.21.15.144:0/2695610101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff800101ce0 0x7ff8000ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.598+0000 7ff805b97700 1 -- 172.21.15.144:0/2695610101 >> 172.21.15.144:0/2695610101 conn(0x7ff8000fabd0 msgr2=0x7ff8001049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:44.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.599+0000 7ff805b97700 1 -- 172.21.15.144:0/2695610101 shutdown_connections 2024-09-10T11:41:44.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.599+0000 7ff805b97700 1 -- 172.21.15.144:0/2695610101 wait complete. 2024-09-10T11:41:44.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.705+0000 7f6996c61700 1 -- 172.21.15.144:0/944693643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69900ffd30 msgr2=0x7f6990103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:44.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.705+0000 7f6996c61700 1 --2- 172.21.15.144:0/944693643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69900ffd30 0x7f6990103ce0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6980009910 tx=0x7f6980009c20 comp rx=0 tx=0).stop 2024-09-10T11:41:44.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.705+0000 7f6996c61700 1 -- 172.21.15.144:0/944693643 shutdown_connections 2024-09-10T11:41:44.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.705+0000 7f6996c61700 1 --2- 172.21.15.144:0/944693643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69900ffd30 0x7f6990103ce0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.705+0000 7f6996c61700 1 --2- 172.21.15.144:0/944693643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69900ff390 0x7f69900ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.705+0000 7f6996c61700 1 -- 172.21.15.144:0/944693643 >> 172.21.15.144:0/944693643 conn(0x7f6990074f50 msgr2=0x7f6990075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:44.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.705+0000 7f6996c61700 1 -- 172.21.15.144:0/944693643 shutdown_connections 2024-09-10T11:41:44.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.705+0000 7f6996c61700 1 -- 172.21.15.144:0/944693643 wait complete. 2024-09-10T11:41:44.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.706+0000 7f6996c61700 1 Processor -- start 2024-09-10T11:41:44.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.706+0000 7f6996c61700 1 -- start start 2024-09-10T11:41:44.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.706+0000 7f6996c61700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69900ff390 0x7f6990195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:44.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.706+0000 7f6996c61700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69900ffd30 0x7f69901962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:44.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.706+0000 7f6996c61700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69901969b0 con 0x7f69900ff390 2024-09-10T11:41:44.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.706+0000 7f6996c61700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699019a740 con 0x7f69900ffd30 2024-09-10T11:41:44.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.706+0000 7f69949fd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69900ff390 0x7f6990195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:44.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.706+0000 7f69949fd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69900ff390 0x7f6990195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38102/0 (socket says 172.21.15.144:38102) 2024-09-10T11:41:44.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.706+0000 7f69949fd700 1 -- 172.21.15.144:0/3734494207 learned_addr learned my addr 172.21.15.144:0/3734494207 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:41:44.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.706+0000 7f6987fff700 1 --2- 172.21.15.144:0/3734494207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69900ffd30 0x7f69901962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:44.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.706+0000 7f69949fd700 1 -- 172.21.15.144:0/3734494207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69900ffd30 msgr2=0x7f69901962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:44.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.706+0000 7f69949fd700 1 --2- 172.21.15.144:0/3734494207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69900ffd30 0x7f69901962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.706+0000 7f69949fd700 1 -- 172.21.15.144:0/3734494207 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69800095f0 con 0x7f69900ff390 2024-09-10T11:41:44.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.706+0000 7f69949fd700 1 --2- 172.21.15.144:0/3734494207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69900ff390 0x7f6990195d90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f698c00ada0 tx=0x7f698c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:44.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.707+0000 7f6985ffb700 1 -- 172.21.15.144:0/3734494207 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f698c00f070 con 0x7f69900ff390 2024-09-10T11:41:44.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.707+0000 7f6985ffb700 1 -- 172.21.15.144:0/3734494207 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f698c007bf0 con 0x7f69900ff390 2024-09-10T11:41:44.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.707+0000 7f6996c61700 1 -- 172.21.15.144:0/3734494207 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699019aa20 con 0x7f69900ff390 2024-09-10T11:41:44.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.707+0000 7f6985ffb700 1 -- 172.21.15.144:0/3734494207 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f698c005690 con 0x7f69900ff390 2024-09-10T11:41:44.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.707+0000 7f6996c61700 1 -- 172.21.15.144:0/3734494207 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699019af70 con 0x7f69900ff390 2024-09-10T11:41:44.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.708+0000 7f6996c61700 1 -- 172.21.15.144:0/3734494207 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f699004ea50 con 0x7f69900ff390 2024-09-10T11:41:44.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.708+0000 7f6985ffb700 1 -- 172.21.15.144:0/3734494207 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f698c005900 con 0x7f69900ff390 2024-09-10T11:41:44.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.709+0000 7f6985ffb700 1 --2- 172.21.15.144:0/3734494207 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f697806e200 0x7f69780706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:44.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.709+0000 7f6987fff700 1 --2- 172.21.15.144:0/3734494207 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f697806e200 0x7f69780706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:44.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.709+0000 7f6985ffb700 1 -- 172.21.15.144:0/3734494207 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f698c08dac0 con 0x7f69900ff390 2024-09-10T11:41:44.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.709+0000 7f6987fff700 1 --2- 172.21.15.144:0/3734494207 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f697806e200 0x7f69780706b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f69800059d0 tx=0x7f6980005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:44.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.713+0000 7f6985ffb700 1 -- 172.21.15.144:0/3734494207 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f698c056ea0 con 0x7f69900ff390 2024-09-10T11:41:44.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.926+0000 7f6996c61700 1 -- 172.21.15.144:0/3734494207 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f699019b400 con 0x7f697806e200 2024-09-10T11:41:44.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.936+0000 7f6985ffb700 1 -- 172.21.15.144:0/3734494207 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f699019b400 con 0x7f697806e200 2024-09-10T11:41:44.938 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:41:44.938 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (9m) 3m ago 20m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:41:44.938 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (20m) 3m ago 20m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:41:44.938 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (19m) 3m ago 19m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:41:44.938 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (7m) 3m ago 20m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:41:44.938 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (7m) 3m ago 19m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:41:44.938 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (9m) 3m ago 20m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:41:44.938 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (4m) 3m ago 15m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:41:44.938 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (3m) 3m ago 14m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:41:44.938 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (14m) 3m ago 14m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:41:44.938 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (3m) 3m ago 15m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:41:44.939 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (11m) 3m ago 21m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:41:44.939 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (10m) 3m ago 19m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:41:44.939 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (8m) 3m ago 22m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:41:44.939 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (8m) 3m ago 19m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:41:44.939 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (10m) 3m ago 20m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:41:44.939 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (10m) 3m ago 19m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:41:44.939 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (7m) 3m ago 18m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:41:44.939 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (6m) 3m ago 18m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:41:44.939 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (6m) 3m ago 17m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:41:44.939 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (5m) 3m ago 17m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:41:44.939 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (4m) 3m ago 16m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:41:44.940 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (4m) 3m ago 16m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:41:44.940 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (10m) 3m ago 20m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:41:44.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.939+0000 7f6996c61700 1 -- 172.21.15.144:0/3734494207 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f697806e200 msgr2=0x7f69780706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:44.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.939+0000 7f6996c61700 1 --2- 172.21.15.144:0/3734494207 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f697806e200 0x7f69780706b0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f69800059d0 tx=0x7f6980005960 comp rx=0 tx=0).stop 2024-09-10T11:41:44.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.939+0000 7f6996c61700 1 -- 172.21.15.144:0/3734494207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69900ff390 msgr2=0x7f6990195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:44.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.939+0000 7f6996c61700 1 --2- 172.21.15.144:0/3734494207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69900ff390 0x7f6990195d90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f698c00ada0 tx=0x7f698c00c330 comp rx=0 tx=0).stop 2024-09-10T11:41:44.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.939+0000 7f6996c61700 1 -- 172.21.15.144:0/3734494207 shutdown_connections 2024-09-10T11:41:44.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.939+0000 7f6996c61700 1 --2- 172.21.15.144:0/3734494207 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f697806e200 0x7f69780706b0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.939+0000 7f6996c61700 1 --2- 172.21.15.144:0/3734494207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69900ff390 0x7f6990195d90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.939+0000 7f6996c61700 1 --2- 172.21.15.144:0/3734494207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69900ffd30 0x7f69901962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:44.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.939+0000 7f6996c61700 1 -- 172.21.15.144:0/3734494207 >> 172.21.15.144:0/3734494207 conn(0x7f6990074f50 msgr2=0x7f6990103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:44.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.939+0000 7f6996c61700 1 -- 172.21.15.144:0/3734494207 shutdown_connections 2024-09-10T11:41:44.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:44.939+0000 7f6996c61700 1 -- 172.21.15.144:0/3734494207 wait complete. 2024-09-10T11:41:45.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.044+0000 7f2d1be4a700 1 -- 172.21.15.144:0/354389513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d140ffc80 msgr2=0x7f2d14103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:45.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.044+0000 7f2d1be4a700 1 --2- 172.21.15.144:0/354389513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d140ffc80 0x7f2d14103c50 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009910 tx=0x7f2d08009c20 comp rx=0 tx=0).stop 2024-09-10T11:41:45.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.044+0000 7f2d1be4a700 1 -- 172.21.15.144:0/354389513 shutdown_connections 2024-09-10T11:41:45.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.044+0000 7f2d1be4a700 1 --2- 172.21.15.144:0/354389513 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d140ffc80 0x7f2d14103c50 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:45.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.044+0000 7f2d1be4a700 1 --2- 172.21.15.144:0/354389513 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d140ff2e0 0x7f2d140ff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:45.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.044+0000 7f2d1be4a700 1 -- 172.21.15.144:0/354389513 >> 172.21.15.144:0/354389513 conn(0x7f2d140fab80 msgr2=0x7f2d140fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:45.046 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:44 smithi144 ceph-mon[87436]: pgmap v292: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:45.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.044+0000 7f2d1be4a700 1 -- 172.21.15.144:0/354389513 shutdown_connections 2024-09-10T11:41:45.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.044+0000 7f2d1be4a700 1 -- 172.21.15.144:0/354389513 wait complete. 2024-09-10T11:41:45.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.045+0000 7f2d1be4a700 1 Processor -- start 2024-09-10T11:41:45.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.045+0000 7f2d1be4a700 1 -- start start 2024-09-10T11:41:45.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.045+0000 7f2d1be4a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d140ff2e0 0x7f2d14194cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:45.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.045+0000 7f2d1be4a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d140ffc80 0x7f2d14195210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:45.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.045+0000 7f2d1be4a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d141958a0 con 0x7f2d140ffc80 2024-09-10T11:41:45.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.045+0000 7f2d1be4a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d141985d0 con 0x7f2d140ff2e0 2024-09-10T11:41:45.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.045+0000 7f2d193e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d140ffc80 0x7f2d14195210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:45.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.045+0000 7f2d193e5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d140ffc80 0x7f2d14195210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38122/0 (socket says 172.21.15.144:38122) 2024-09-10T11:41:45.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.045+0000 7f2d193e5700 1 -- 172.21.15.144:0/3905899285 learned_addr learned my addr 172.21.15.144:0/3905899285 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:41:45.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.045+0000 7f2d19be6700 1 --2- 172.21.15.144:0/3905899285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d140ff2e0 0x7f2d14194cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:45.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.046+0000 7f2d193e5700 1 -- 172.21.15.144:0/3905899285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d140ff2e0 msgr2=0x7f2d14194cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:45.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.046+0000 7f2d193e5700 1 --2- 172.21.15.144:0/3905899285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d140ff2e0 0x7f2d14194cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:45.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.046+0000 7f2d193e5700 1 -- 172.21.15.144:0/3905899285 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080095f0 con 0x7f2d140ffc80 2024-09-10T11:41:45.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.046+0000 7f2d193e5700 1 --2- 172.21.15.144:0/3905899285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d140ffc80 0x7f2d14195210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2d080092e0 tx=0x7f2d0800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:45.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.046+0000 7f2d12ffd700 1 -- 172.21.15.144:0/3905899285 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d0801d070 con 0x7f2d140ffc80 2024-09-10T11:41:45.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.046+0000 7f2d12ffd700 1 -- 172.21.15.144:0/3905899285 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2d08005dc0 con 0x7f2d140ffc80 2024-09-10T11:41:45.050 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.046+0000 7f2d1be4a700 1 -- 172.21.15.144:0/3905899285 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d14198770 con 0x7f2d140ffc80 2024-09-10T11:41:45.050 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.046+0000 7f2d12ffd700 1 -- 172.21.15.144:0/3905899285 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d08017400 con 0x7f2d140ffc80 2024-09-10T11:41:45.050 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.046+0000 7f2d1be4a700 1 -- 172.21.15.144:0/3905899285 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d14198c60 con 0x7f2d140ffc80 2024-09-10T11:41:45.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.047+0000 7f2d1be4a700 1 -- 172.21.15.144:0/3905899285 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d1404ea50 con 0x7f2d140ffc80 2024-09-10T11:41:45.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.048+0000 7f2d12ffd700 1 -- 172.21.15.144:0/3905899285 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2d0800f6f0 con 0x7f2d140ffc80 2024-09-10T11:41:45.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.048+0000 7f2d12ffd700 1 --2- 172.21.15.144:0/3905899285 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2cfc06e200 0x7f2cfc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:45.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.048+0000 7f2d12ffd700 1 -- 172.21.15.144:0/3905899285 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2d0808f470 con 0x7f2d140ffc80 2024-09-10T11:41:45.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.048+0000 7f2d19be6700 1 --2- 172.21.15.144:0/3905899285 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2cfc06e200 0x7f2cfc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:45.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.049+0000 7f2d19be6700 1 --2- 172.21.15.144:0/3905899285 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2cfc06e200 0x7f2cfc0706b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c005950 tx=0x7f2d0c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:45.053 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.052+0000 7f2d12ffd700 1 -- 172.21.15.144:0/3905899285 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2d08058850 con 0x7f2d140ffc80 2024-09-10T11:41:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:44 smithi178 ceph-mon[65940]: pgmap v292: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:45.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.340+0000 7f2d1be4a700 1 -- 172.21.15.144:0/3905899285 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2d141990d0 con 0x7f2d140ffc80 2024-09-10T11:41:45.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.341+0000 7f2d12ffd700 1 -- 172.21.15.144:0/3905899285 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f2d0805bf10 con 0x7f2d140ffc80 2024-09-10T11:41:45.342 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:41:45.342 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:41:45.342 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:41:45.342 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:41:45.343 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:41:45.343 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:41:45.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:41:45.343 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:41:45.343 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:41:45.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:41:45.343 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:41:45.343 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:41:45.343 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:41:45.343 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:41:45.343 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:41:45.343 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:41:45.343 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:41:45.343 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:41:45.344 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:41:45.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.343+0000 7f2d1be4a700 1 -- 172.21.15.144:0/3905899285 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2cfc06e200 msgr2=0x7f2cfc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:45.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.343+0000 7f2d1be4a700 1 --2- 172.21.15.144:0/3905899285 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2cfc06e200 0x7f2cfc0706b0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c005950 tx=0x7f2d0c00b410 comp rx=0 tx=0).stop 2024-09-10T11:41:45.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.343+0000 7f2d1be4a700 1 -- 172.21.15.144:0/3905899285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d140ffc80 msgr2=0x7f2d14195210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:45.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.343+0000 7f2d1be4a700 1 --2- 172.21.15.144:0/3905899285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d140ffc80 0x7f2d14195210 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2d080092e0 tx=0x7f2d0800bd70 comp rx=0 tx=0).stop 2024-09-10T11:41:45.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.343+0000 7f2d1be4a700 1 -- 172.21.15.144:0/3905899285 shutdown_connections 2024-09-10T11:41:45.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.343+0000 7f2d1be4a700 1 --2- 172.21.15.144:0/3905899285 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2cfc06e200 0x7f2cfc0706b0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:45.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.343+0000 7f2d1be4a700 1 --2- 172.21.15.144:0/3905899285 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2d140ff2e0 0x7f2d14194cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:45.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.343+0000 7f2d1be4a700 1 --2- 172.21.15.144:0/3905899285 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2d140ffc80 0x7f2d14195210 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:45.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.343+0000 7f2d1be4a700 1 -- 172.21.15.144:0/3905899285 >> 172.21.15.144:0/3905899285 conn(0x7f2d140fab80 msgr2=0x7f2d140fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:45.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.343+0000 7f2d1be4a700 1 -- 172.21.15.144:0/3905899285 shutdown_connections 2024-09-10T11:41:45.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.343+0000 7f2d1be4a700 1 -- 172.21.15.144:0/3905899285 wait complete. 2024-09-10T11:41:45.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.447+0000 7f7416ae6700 1 -- 172.21.15.144:0/3397760272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74100ffd30 msgr2=0x7f7410103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:45.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.447+0000 7f7416ae6700 1 --2- 172.21.15.144:0/3397760272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74100ffd30 0x7f7410103ce0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7400009910 tx=0x7f7400009c20 comp rx=0 tx=0).stop 2024-09-10T11:41:45.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.447+0000 7f7416ae6700 1 -- 172.21.15.144:0/3397760272 shutdown_connections 2024-09-10T11:41:45.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.447+0000 7f7416ae6700 1 --2- 172.21.15.144:0/3397760272 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74100ffd30 0x7f7410103ce0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:45.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.447+0000 7f7416ae6700 1 --2- 172.21.15.144:0/3397760272 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74100ff390 0x7f74100ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:45.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.447+0000 7f7416ae6700 1 -- 172.21.15.144:0/3397760272 >> 172.21.15.144:0/3397760272 conn(0x7f7410074f50 msgr2=0x7f7410075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:45.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.447+0000 7f7416ae6700 1 -- 172.21.15.144:0/3397760272 shutdown_connections 2024-09-10T11:41:45.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.447+0000 7f7416ae6700 1 -- 172.21.15.144:0/3397760272 wait complete. 2024-09-10T11:41:45.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.447+0000 7f7416ae6700 1 Processor -- start 2024-09-10T11:41:45.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.448+0000 7f7416ae6700 1 -- start start 2024-09-10T11:41:45.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.448+0000 7f7416ae6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74100ff390 0x7f7410195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:45.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.448+0000 7f7416ae6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74100ffd30 0x7f7410196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:45.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.448+0000 7f7416ae6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7410196a60 con 0x7f74100ff390 2024-09-10T11:41:45.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.448+0000 7f7416ae6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f741019a7f0 con 0x7f74100ffd30 2024-09-10T11:41:45.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.448+0000 7f7414882700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74100ff390 0x7f7410195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:45.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.448+0000 7f7414882700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74100ff390 0x7f7410195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38138/0 (socket says 172.21.15.144:38138) 2024-09-10T11:41:45.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.448+0000 7f7414882700 1 -- 172.21.15.144:0/3815662866 learned_addr learned my addr 172.21.15.144:0/3815662866 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:41:45.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.448+0000 7f7407fff700 1 --2- 172.21.15.144:0/3815662866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74100ffd30 0x7f7410196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:45.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.448+0000 7f7414882700 1 -- 172.21.15.144:0/3815662866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74100ffd30 msgr2=0x7f7410196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:45.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.448+0000 7f7414882700 1 --2- 172.21.15.144:0/3815662866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74100ffd30 0x7f7410196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:45.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.448+0000 7f7414882700 1 -- 172.21.15.144:0/3815662866 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74000095f0 con 0x7f74100ff390 2024-09-10T11:41:45.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.448+0000 7f7414882700 1 --2- 172.21.15.144:0/3815662866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74100ff390 0x7f7410195e40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f740c00ba50 tx=0x7f740c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:45.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.448+0000 7f7405ffb700 1 -- 172.21.15.144:0/3815662866 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f740c00ca00 con 0x7f74100ff390 2024-09-10T11:41:45.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.448+0000 7f7405ffb700 1 -- 172.21.15.144:0/3815662866 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f740c00cb60 con 0x7f74100ff390 2024-09-10T11:41:45.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.449+0000 7f7416ae6700 1 -- 172.21.15.144:0/3815662866 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f741019aad0 con 0x7f74100ff390 2024-09-10T11:41:45.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.449+0000 7f7405ffb700 1 -- 172.21.15.144:0/3815662866 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f740c0075a0 con 0x7f74100ff390 2024-09-10T11:41:45.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.449+0000 7f7416ae6700 1 -- 172.21.15.144:0/3815662866 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f741019b020 con 0x7f74100ff390 2024-09-10T11:41:45.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.450+0000 7f7416ae6700 1 -- 172.21.15.144:0/3815662866 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f741004ea50 con 0x7f74100ff390 2024-09-10T11:41:45.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.450+0000 7f7405ffb700 1 -- 172.21.15.144:0/3815662866 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f740c00ccd0 con 0x7f74100ff390 2024-09-10T11:41:45.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.450+0000 7f7405ffb700 1 --2- 172.21.15.144:0/3815662866 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f73f806e200 0x7f73f80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:45.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.451+0000 7f7405ffb700 1 -- 172.21.15.144:0/3815662866 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f740c006e40 con 0x7f74100ff390 2024-09-10T11:41:45.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.451+0000 7f7407fff700 1 --2- 172.21.15.144:0/3815662866 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f73f806e200 0x7f73f80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:45.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.451+0000 7f7407fff700 1 --2- 172.21.15.144:0/3815662866 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f73f806e200 0x7f73f80706b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7410197460 tx=0x7f740001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:45.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.455+0000 7f7405ffb700 1 -- 172.21.15.144:0/3815662866 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f740c056ce0 con 0x7f74100ff390 2024-09-10T11:41:45.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.702+0000 7f7416ae6700 1 -- 172.21.15.144:0/3815662866 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f741019b4a0 con 0x7f74100ff390 2024-09-10T11:41:45.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.703+0000 7f7405ffb700 1 -- 172.21.15.144:0/3815662866 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f740c05a3a0 con 0x7f74100ff390 2024-09-10T11:41:45.704 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:41:45.705 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:41:45.705 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:41:45.705 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:41:45.705 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:41:45.705 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:41:45.705 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:41:45.705 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:41:45.705 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:41:45.705 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:41:45.706 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:41:45.706 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:41:45.706 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:41:45.706 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:41:45.706 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:41:45.706 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:41:45.706 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:41:45.706 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:41:45.706 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:41:45.706 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:41:45.706 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:41:45.707 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:41:45.707 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:41:45.707 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:41:45.707 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:41:45.707 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:41:45.707 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:41:45.707 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:41:45.707 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:41:45.707 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:41:45.707 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:41:45.707 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:41:45.707 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:41:45.707 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:41:45.708 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:41:45.708 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:41:45.708 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:41:45.708 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:41:45.708 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:41:45.708 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:41:45.708 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:41:45.708 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:41:45.708 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:41:45.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.705+0000 7f7416ae6700 1 -- 172.21.15.144:0/3815662866 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f73f806e200 msgr2=0x7f73f80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:45.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.705+0000 7f7416ae6700 1 --2- 172.21.15.144:0/3815662866 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f73f806e200 0x7f73f80706b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f7410197460 tx=0x7f740001a040 comp rx=0 tx=0).stop 2024-09-10T11:41:45.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.705+0000 7f7416ae6700 1 -- 172.21.15.144:0/3815662866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74100ff390 msgr2=0x7f7410195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:45.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.705+0000 7f7416ae6700 1 --2- 172.21.15.144:0/3815662866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74100ff390 0x7f7410195e40 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f740c00ba50 tx=0x7f740c00be10 comp rx=0 tx=0).stop 2024-09-10T11:41:45.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.705+0000 7f7416ae6700 1 -- 172.21.15.144:0/3815662866 shutdown_connections 2024-09-10T11:41:45.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.705+0000 7f7416ae6700 1 --2- 172.21.15.144:0/3815662866 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f73f806e200 0x7f73f80706b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:45.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.705+0000 7f7416ae6700 1 --2- 172.21.15.144:0/3815662866 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74100ff390 0x7f7410195e40 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:45.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.705+0000 7f7416ae6700 1 --2- 172.21.15.144:0/3815662866 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74100ffd30 0x7f7410196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:45.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.705+0000 7f7416ae6700 1 -- 172.21.15.144:0/3815662866 >> 172.21.15.144:0/3815662866 conn(0x7f7410074f50 msgr2=0x7f7410103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:45.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.705+0000 7f7416ae6700 1 -- 172.21.15.144:0/3815662866 shutdown_connections 2024-09-10T11:41:45.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.705+0000 7f7416ae6700 1 -- 172.21.15.144:0/3815662866 wait complete. 2024-09-10T11:41:45.710 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:41:45.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.811+0000 7fd24ac73700 1 -- 172.21.15.144:0/331068005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2440ff2c0 msgr2=0x7fd2440ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:45.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.811+0000 7fd24ac73700 1 --2- 172.21.15.144:0/331068005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2440ff2c0 0x7fd2440ff690 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd238009910 tx=0x7fd238009c20 comp rx=0 tx=0).stop 2024-09-10T11:41:45.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.811+0000 7fd24ac73700 1 -- 172.21.15.144:0/331068005 shutdown_connections 2024-09-10T11:41:45.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.811+0000 7fd24ac73700 1 --2- 172.21.15.144:0/331068005 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2440ffc60 0x7fd244103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.811+0000 7fd24ac73700 1 --2- 172.21.15.144:0/331068005 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2440ff2c0 0x7fd2440ff690 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.812+0000 7fd24ac73700 1 -- 172.21.15.144:0/331068005 >> 172.21.15.144:0/331068005 conn(0x7fd2440fab80 msgr2=0x7fd2440fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.812+0000 7fd24ac73700 1 -- 172.21.15.144:0/331068005 shutdown_connections 2024-09-10T11:41:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.812+0000 7fd24ac73700 1 -- 172.21.15.144:0/331068005 wait complete. 2024-09-10T11:41:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.812+0000 7fd24ac73700 1 Processor -- start 2024-09-10T11:41:45.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.812+0000 7fd24ac73700 1 -- start start 2024-09-10T11:41:45.819 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.812+0000 7fd24ac73700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2440ff2c0 0x7fd244196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:45.819 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.812+0000 7fd24ac73700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2440ffc60 0x7fd2441971b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:45.819 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.812+0000 7fd24ac73700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd244197740 con 0x7fd2440ff2c0 2024-09-10T11:41:45.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.812+0000 7fd24ac73700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2441978b0 con 0x7fd2440ffc60 2024-09-10T11:41:45.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.813+0000 7fd248a0f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2440ff2c0 0x7fd244196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:45.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.813+0000 7fd248a0f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2440ff2c0 0x7fd244196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38152/0 (socket says 172.21.15.144:38152) 2024-09-10T11:41:45.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.813+0000 7fd248a0f700 1 -- 172.21.15.144:0/113515867 learned_addr learned my addr 172.21.15.144:0/113515867 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:41:45.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.813+0000 7fd243fff700 1 --2- 172.21.15.144:0/113515867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2440ffc60 0x7fd2441971b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:45.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.813+0000 7fd248a0f700 1 -- 172.21.15.144:0/113515867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2440ffc60 msgr2=0x7fd2441971b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:45.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.813+0000 7fd248a0f700 1 --2- 172.21.15.144:0/113515867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2440ffc60 0x7fd2441971b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:45.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.813+0000 7fd248a0f700 1 -- 172.21.15.144:0/113515867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2380095f0 con 0x7fd2440ff2c0 2024-09-10T11:41:45.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.813+0000 7fd248a0f700 1 --2- 172.21.15.144:0/113515867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2440ff2c0 0x7fd244196c70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd238006010 tx=0x7fd23800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:45.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.813+0000 7fd241ffb700 1 -- 172.21.15.144:0/113515867 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd23801d070 con 0x7fd2440ff2c0 2024-09-10T11:41:45.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.813+0000 7fd241ffb700 1 -- 172.21.15.144:0/113515867 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd238004fb0 con 0x7fd2440ff2c0 2024-09-10T11:41:45.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.813+0000 7fd241ffb700 1 -- 172.21.15.144:0/113515867 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd238022ea0 con 0x7fd2440ff2c0 2024-09-10T11:41:45.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.813+0000 7fd24ac73700 1 -- 172.21.15.144:0/113515867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24419fce0 con 0x7fd2440ff2c0 2024-09-10T11:41:45.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.813+0000 7fd24ac73700 1 -- 172.21.15.144:0/113515867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2441a0040 con 0x7fd2440ff2c0 2024-09-10T11:41:45.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.814+0000 7fd24ac73700 1 -- 172.21.15.144:0/113515867 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2441075d0 con 0x7fd2440ff2c0 2024-09-10T11:41:45.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.815+0000 7fd241ffb700 1 -- 172.21.15.144:0/113515867 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd238022470 con 0x7fd2440ff2c0 2024-09-10T11:41:45.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.815+0000 7fd241ffb700 1 --2- 172.21.15.144:0/113515867 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd23406e200 0x7fd2340706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:45.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.815+0000 7fd241ffb700 1 -- 172.21.15.144:0/113515867 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd23808f5f0 con 0x7fd2440ff2c0 2024-09-10T11:41:45.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.815+0000 7fd243fff700 1 --2- 172.21.15.144:0/113515867 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd23406e200 0x7fd2340706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:45.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.816+0000 7fd243fff700 1 --2- 172.21.15.144:0/113515867 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd23406e200 0x7fd2340706b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd244069640 tx=0x7fd230005f50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:45.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:45.820+0000 7fd241ffb700 1 -- 172.21.15.144:0/113515867 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd238058a80 con 0x7fd2440ff2c0 2024-09-10T11:41:46.041 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.040+0000 7fd24ac73700 1 -- 172.21.15.144:0/113515867 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd24402cca0 con 0x7fd23406e200 2024-09-10T11:41:46.041 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:45 smithi144 ceph-mon[87436]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:46.041 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:45 smithi144 ceph-mon[87436]: from='client.34600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:46.042 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:45 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3905899285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:41:46.042 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:45 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3815662866' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:41:46.042 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.041+0000 7fd241ffb700 1 -- 172.21.15.144:0/113515867 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd24402cca0 con 0x7fd23406e200 2024-09-10T11:41:46.042 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:41:46.042 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:41:46.042 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:41:46.043 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:41:46.043 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:41:46.043 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:41:46.043 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:41:46.043 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:41:46.043 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:41:46.043 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:41:46.043 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:41:46.043 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:41:46.043 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:41:46.043 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:41:46.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.044+0000 7fd24ac73700 1 -- 172.21.15.144:0/113515867 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd23406e200 msgr2=0x7fd2340706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:46.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.044+0000 7fd24ac73700 1 --2- 172.21.15.144:0/113515867 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd23406e200 0x7fd2340706b0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd244069640 tx=0x7fd230005f50 comp rx=0 tx=0).stop 2024-09-10T11:41:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.044+0000 7fd24ac73700 1 -- 172.21.15.144:0/113515867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2440ff2c0 msgr2=0x7fd244196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.044+0000 7fd24ac73700 1 --2- 172.21.15.144:0/113515867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2440ff2c0 0x7fd244196c70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd238006010 tx=0x7fd23800be20 comp rx=0 tx=0).stop 2024-09-10T11:41:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.044+0000 7fd24ac73700 1 -- 172.21.15.144:0/113515867 shutdown_connections 2024-09-10T11:41:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.044+0000 7fd24ac73700 1 --2- 172.21.15.144:0/113515867 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd23406e200 0x7fd2340706b0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.044+0000 7fd24ac73700 1 --2- 172.21.15.144:0/113515867 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2440ff2c0 0x7fd244196c70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.044+0000 7fd24ac73700 1 --2- 172.21.15.144:0/113515867 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2440ffc60 0x7fd2441971b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.044+0000 7fd24ac73700 1 -- 172.21.15.144:0/113515867 >> 172.21.15.144:0/113515867 conn(0x7fd2440fab80 msgr2=0x7fd2440fc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.044+0000 7fd24ac73700 1 -- 172.21.15.144:0/113515867 shutdown_connections 2024-09-10T11:41:46.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.044+0000 7fd24ac73700 1 -- 172.21.15.144:0/113515867 wait complete. 2024-09-10T11:41:46.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:45 smithi178 ceph-mon[65940]: from='client.34596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:46.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:45 smithi178 ceph-mon[65940]: from='client.34600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:46.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:45 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3905899285' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:41:46.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:45 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3815662866' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:41:46.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.150+0000 7f37da813700 1 -- 172.21.15.144:0/3397025469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d4100d70 msgr2=0x7f37d4104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:46.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.150+0000 7f37da813700 1 --2- 172.21.15.144:0/3397025469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d4100d70 0x7f37d4104d40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f37c8009910 tx=0x7f37c8009c20 comp rx=0 tx=0).stop 2024-09-10T11:41:46.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.150+0000 7f37da813700 1 -- 172.21.15.144:0/3397025469 shutdown_connections 2024-09-10T11:41:46.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.150+0000 7f37da813700 1 --2- 172.21.15.144:0/3397025469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d4100d70 0x7f37d4104d40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:46.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.150+0000 7f37da813700 1 --2- 172.21.15.144:0/3397025469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d41003d0 0x7f37d41007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:46.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.150+0000 7f37da813700 1 -- 172.21.15.144:0/3397025469 >> 172.21.15.144:0/3397025469 conn(0x7f37d40fbc90 msgr2=0x7f37d40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:46.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.150+0000 7f37da813700 1 -- 172.21.15.144:0/3397025469 shutdown_connections 2024-09-10T11:41:46.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.150+0000 7f37da813700 1 -- 172.21.15.144:0/3397025469 wait complete. 2024-09-10T11:41:46.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.151+0000 7f37da813700 1 Processor -- start 2024-09-10T11:41:46.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.151+0000 7f37da813700 1 -- start start 2024-09-10T11:41:46.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.151+0000 7f37da813700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d41003d0 0x7f37d4195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:46.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.151+0000 7f37da813700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d4100d70 0x7f37d41963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:46.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.151+0000 7f37da813700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d4196aa0 con 0x7f37d4100d70 2024-09-10T11:41:46.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.151+0000 7f37da813700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d419a830 con 0x7f37d41003d0 2024-09-10T11:41:46.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.151+0000 7f37d37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d4100d70 0x7f37d41963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:46.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.151+0000 7f37d37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d4100d70 0x7f37d41963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38164/0 (socket says 172.21.15.144:38164) 2024-09-10T11:41:46.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.151+0000 7f37d37fe700 1 -- 172.21.15.144:0/1544559590 learned_addr learned my addr 172.21.15.144:0/1544559590 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:41:46.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.152+0000 7f37d3fff700 1 --2- 172.21.15.144:0/1544559590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d41003d0 0x7f37d4195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:46.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.152+0000 7f37d37fe700 1 -- 172.21.15.144:0/1544559590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d41003d0 msgr2=0x7f37d4195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:46.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.152+0000 7f37d37fe700 1 --2- 172.21.15.144:0/1544559590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d41003d0 0x7f37d4195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:46.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.152+0000 7f37d37fe700 1 -- 172.21.15.144:0/1544559590 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c80095f0 con 0x7f37d4100d70 2024-09-10T11:41:46.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.152+0000 7f37d37fe700 1 --2- 172.21.15.144:0/1544559590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d4100d70 0x7f37d41963c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f37c80092e0 tx=0x7f37c800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:46.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.152+0000 7f37d17fa700 1 -- 172.21.15.144:0/1544559590 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37c801d070 con 0x7f37d4100d70 2024-09-10T11:41:46.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.152+0000 7f37d17fa700 1 -- 172.21.15.144:0/1544559590 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f37c800f460 con 0x7f37d4100d70 2024-09-10T11:41:46.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.152+0000 7f37da813700 1 -- 172.21.15.144:0/1544559590 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d419aab0 con 0x7f37d4100d70 2024-09-10T11:41:46.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.152+0000 7f37d17fa700 1 -- 172.21.15.144:0/1544559590 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37c80175f0 con 0x7f37d4100d70 2024-09-10T11:41:46.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.152+0000 7f37da813700 1 -- 172.21.15.144:0/1544559590 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d419afa0 con 0x7f37d4100d70 2024-09-10T11:41:46.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.153+0000 7f37da813700 1 -- 172.21.15.144:0/1544559590 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d4108690 con 0x7f37d4100d70 2024-09-10T11:41:46.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.154+0000 7f37d17fa700 1 -- 172.21.15.144:0/1544559590 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f37c8003a60 con 0x7f37d4100d70 2024-09-10T11:41:46.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.154+0000 7f37d17fa700 1 --2- 172.21.15.144:0/1544559590 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f37bc06e200 0x7f37bc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:41:46.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.154+0000 7f37d17fa700 1 -- 172.21.15.144:0/1544559590 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f37c808f530 con 0x7f37d4100d70 2024-09-10T11:41:46.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.154+0000 7f37d3fff700 1 --2- 172.21.15.144:0/1544559590 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f37bc06e200 0x7f37bc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:41:46.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.155+0000 7f37d3fff700 1 --2- 172.21.15.144:0/1544559590 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f37bc06e200 0x7f37bc0706b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f37c4006fd0 tx=0x7f37c4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:41:46.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.159+0000 7f37d17fa700 1 -- 172.21.15.144:0/1544559590 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f37c8058990 con 0x7f37d4100d70 2024-09-10T11:41:46.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.461+0000 7f37da813700 1 -- 172.21.15.144:0/1544559590 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f37d402cd20 con 0x7f37d4100d70 2024-09-10T11:41:46.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.462+0000 7f37d17fa700 1 -- 172.21.15.144:0/1544559590 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f37c805c050 con 0x7f37d4100d70 2024-09-10T11:41:46.463 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:41:46.463 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:41:46.463 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:41:46.463 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:41:46.463 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:41:46.463 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:41:46.464 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:41:46.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.464+0000 7f37da813700 1 -- 172.21.15.144:0/1544559590 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f37bc06e200 msgr2=0x7f37bc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:46.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.464+0000 7f37da813700 1 --2- 172.21.15.144:0/1544559590 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f37bc06e200 0x7f37bc0706b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f37c4006fd0 tx=0x7f37c4008040 comp rx=0 tx=0).stop 2024-09-10T11:41:46.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.464+0000 7f37da813700 1 -- 172.21.15.144:0/1544559590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d4100d70 msgr2=0x7f37d41963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:41:46.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.464+0000 7f37da813700 1 --2- 172.21.15.144:0/1544559590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d4100d70 0x7f37d41963c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f37c80092e0 tx=0x7f37c800be20 comp rx=0 tx=0).stop 2024-09-10T11:41:46.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.464+0000 7f37da813700 1 -- 172.21.15.144:0/1544559590 shutdown_connections 2024-09-10T11:41:46.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.464+0000 7f37da813700 1 --2- 172.21.15.144:0/1544559590 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f37bc06e200 0x7f37bc0706b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:46.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.464+0000 7f37da813700 1 --2- 172.21.15.144:0/1544559590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37d41003d0 0x7f37d4195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:46.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.464+0000 7f37da813700 1 --2- 172.21.15.144:0/1544559590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37d4100d70 0x7f37d41963c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:41:46.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.464+0000 7f37da813700 1 -- 172.21.15.144:0/1544559590 >> 172.21.15.144:0/1544559590 conn(0x7f37d40fbc90 msgr2=0x7f37d40fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:41:46.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.464+0000 7f37da813700 1 -- 172.21.15.144:0/1544559590 shutdown_connections 2024-09-10T11:41:46.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:41:46.464+0000 7f37da813700 1 -- 172.21.15.144:0/1544559590 wait complete. 2024-09-10T11:41:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:46 smithi178 ceph-mon[65940]: from='client.34604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:46 smithi178 ceph-mon[65940]: pgmap v293: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:46 smithi178 ceph-mon[65940]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:46 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1544559590' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:41:47.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:46 smithi144 ceph-mon[87436]: from='client.34604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:46 smithi144 ceph-mon[87436]: pgmap v293: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:46 smithi144 ceph-mon[87436]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:41:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:46 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1544559590' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:41:48.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:47 smithi178 ceph-mon[65940]: pgmap v294: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:48.135 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:47 smithi144 ceph-mon[87436]: pgmap v294: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:50 smithi178 ceph-mon[65940]: pgmap v295: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:51.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:50 smithi144 ceph-mon[87436]: pgmap v295: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:53.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:52 smithi144 ceph-mon[87436]: pgmap v296: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:52 smithi178 ceph-mon[65940]: pgmap v296: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:54 smithi178 ceph-mon[65940]: pgmap v297: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:55.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:54 smithi144 ceph-mon[87436]: pgmap v297: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:56 smithi178 ceph-mon[65940]: pgmap v298: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:56 smithi144 ceph-mon[87436]: pgmap v298: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:58 smithi178 ceph-mon[65940]: pgmap v299: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:41:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:41:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:58 smithi144 ceph-mon[87436]: pgmap v299: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:41:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:41:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:42:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:00 smithi178 ceph-mon[65940]: pgmap v300: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:01.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:00 smithi144 ceph-mon[87436]: pgmap v300: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:03.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:02 smithi144 ceph-mon[87436]: pgmap v301: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:03.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:02 smithi178 ceph-mon[65940]: pgmap v301: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:04 smithi178 ceph-mon[65940]: pgmap v302: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:05.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:04 smithi144 ceph-mon[87436]: pgmap v302: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:06 smithi178 ceph-mon[65940]: pgmap v303: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:07.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:06 smithi144 ceph-mon[87436]: pgmap v303: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:08 smithi178 ceph-mon[65940]: pgmap v304: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:09.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:08 smithi144 ceph-mon[87436]: pgmap v304: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:10 smithi178 ceph-mon[65940]: pgmap v305: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:11.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:10 smithi144 ceph-mon[87436]: pgmap v305: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:12 smithi144 ceph-mon[87436]: pgmap v306: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:13.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:12 smithi178 ceph-mon[65940]: pgmap v306: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:42:14.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:42:15.056 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:14 smithi178 ceph-mon[65940]: pgmap v307: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:15.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:14 smithi144 ceph-mon[87436]: pgmap v307: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.583+0000 7f06e1f75700 1 -- 172.21.15.144:0/2529617478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc0ffd30 msgr2=0x7f06dc103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.583+0000 7f06e1f75700 1 --2- 172.21.15.144:0/2529617478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc0ffd30 0x7f06dc103ce0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009910 tx=0x7f06d0009c20 comp rx=0 tx=0).stop 2024-09-10T11:42:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.583+0000 7f06e1f75700 1 -- 172.21.15.144:0/2529617478 shutdown_connections 2024-09-10T11:42:16.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.583+0000 7f06e1f75700 1 --2- 172.21.15.144:0/2529617478 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc0ffd30 0x7f06dc103ce0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:16.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.583+0000 7f06e1f75700 1 --2- 172.21.15.144:0/2529617478 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06dc0ff390 0x7f06dc0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:16.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.583+0000 7f06e1f75700 1 -- 172.21.15.144:0/2529617478 >> 172.21.15.144:0/2529617478 conn(0x7f06dc074f50 msgr2=0x7f06dc075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:16.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.584+0000 7f06e1f75700 1 -- 172.21.15.144:0/2529617478 shutdown_connections 2024-09-10T11:42:16.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.584+0000 7f06e1f75700 1 -- 172.21.15.144:0/2529617478 wait complete. 2024-09-10T11:42:16.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.584+0000 7f06e1f75700 1 Processor -- start 2024-09-10T11:42:16.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.584+0000 7f06e1f75700 1 -- start start 2024-09-10T11:42:16.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.585+0000 7f06e1f75700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06dc0ff390 0x7f06dc196b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:16.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.585+0000 7f06e1f75700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc0ffd30 0x7f06dc197060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:16.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.585+0000 7f06e1f75700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc197740 con 0x7f06dc0ffd30 2024-09-10T11:42:16.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.585+0000 7f06e1f75700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06dc197880 con 0x7f06dc0ff390 2024-09-10T11:42:16.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.585+0000 7f06daffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc0ffd30 0x7f06dc197060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:16.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.585+0000 7f06daffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc0ffd30 0x7f06dc197060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52752/0 (socket says 172.21.15.144:52752) 2024-09-10T11:42:16.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.585+0000 7f06daffd700 1 -- 172.21.15.144:0/4162519431 learned_addr learned my addr 172.21.15.144:0/4162519431 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:42:16.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.585+0000 7f06db7fe700 1 --2- 172.21.15.144:0/4162519431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06dc0ff390 0x7f06dc196b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:16.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.585+0000 7f06daffd700 1 -- 172.21.15.144:0/4162519431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06dc0ff390 msgr2=0x7f06dc196b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:16.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.585+0000 7f06daffd700 1 --2- 172.21.15.144:0/4162519431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06dc0ff390 0x7f06dc196b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:16.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.585+0000 7f06daffd700 1 -- 172.21.15.144:0/4162519431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d00095f0 con 0x7f06dc0ffd30 2024-09-10T11:42:16.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.585+0000 7f06daffd700 1 --2- 172.21.15.144:0/4162519431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc0ffd30 0x7f06dc197060 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f06d000bd70 tx=0x7f06d000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:16.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.586+0000 7f06d8ff9700 1 -- 172.21.15.144:0/4162519431 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06d001d070 con 0x7f06dc0ffd30 2024-09-10T11:42:16.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.586+0000 7f06d8ff9700 1 -- 172.21.15.144:0/4162519431 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f06d0005dc0 con 0x7f06dc0ffd30 2024-09-10T11:42:16.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.586+0000 7f06e1f75700 1 -- 172.21.15.144:0/4162519431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06dc19fb90 con 0x7f06dc0ffd30 2024-09-10T11:42:16.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.586+0000 7f06d8ff9700 1 -- 172.21.15.144:0/4162519431 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06d0017400 con 0x7f06dc0ffd30 2024-09-10T11:42:16.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.586+0000 7f06e1f75700 1 -- 172.21.15.144:0/4162519431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06dc1a0000 con 0x7f06dc0ffd30 2024-09-10T11:42:16.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.587+0000 7f06e1f75700 1 -- 172.21.15.144:0/4162519431 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06dc107600 con 0x7f06dc0ffd30 2024-09-10T11:42:16.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.588+0000 7f06d8ff9700 1 -- 172.21.15.144:0/4162519431 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f06d0003a60 con 0x7f06dc0ffd30 2024-09-10T11:42:16.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.589+0000 7f06d8ff9700 1 --2- 172.21.15.144:0/4162519431 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f06c406e200 0x7f06c40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:16.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.589+0000 7f06d8ff9700 1 -- 172.21.15.144:0/4162519431 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f06d008f570 con 0x7f06dc0ffd30 2024-09-10T11:42:16.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.589+0000 7f06db7fe700 1 --2- 172.21.15.144:0/4162519431 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f06c406e200 0x7f06c40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:16.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.589+0000 7f06db7fe700 1 --2- 172.21.15.144:0/4162519431 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f06c406e200 0x7f06c40706b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f06cc005950 tx=0x7f06cc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:16.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.596+0000 7f06d8ff9700 1 -- 172.21.15.144:0/4162519431 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f06d0058950 con 0x7f06dc0ffd30 2024-09-10T11:42:16.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.824+0000 7f06e1f75700 1 -- 172.21.15.144:0/4162519431 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06dc02ccf0 con 0x7f06c406e200 2024-09-10T11:42:16.825 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:16 smithi144 ceph-mon[87436]: pgmap v308: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:16.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.826+0000 7f06d8ff9700 1 -- 172.21.15.144:0/4162519431 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f06dc02ccf0 con 0x7f06c406e200 2024-09-10T11:42:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.828+0000 7f06e1f75700 1 -- 172.21.15.144:0/4162519431 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f06c406e200 msgr2=0x7f06c40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.828+0000 7f06e1f75700 1 --2- 172.21.15.144:0/4162519431 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f06c406e200 0x7f06c40706b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f06cc005950 tx=0x7f06cc0058e0 comp rx=0 tx=0).stop 2024-09-10T11:42:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.828+0000 7f06e1f75700 1 -- 172.21.15.144:0/4162519431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc0ffd30 msgr2=0x7f06dc197060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.828+0000 7f06e1f75700 1 --2- 172.21.15.144:0/4162519431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc0ffd30 0x7f06dc197060 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f06d000bd70 tx=0x7f06d000be50 comp rx=0 tx=0).stop 2024-09-10T11:42:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.828+0000 7f06e1f75700 1 -- 172.21.15.144:0/4162519431 shutdown_connections 2024-09-10T11:42:16.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.828+0000 7f06e1f75700 1 --2- 172.21.15.144:0/4162519431 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f06c406e200 0x7f06c40706b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:16.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.828+0000 7f06e1f75700 1 --2- 172.21.15.144:0/4162519431 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06dc0ff390 0x7f06dc196b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:16.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.828+0000 7f06e1f75700 1 --2- 172.21.15.144:0/4162519431 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06dc0ffd30 0x7f06dc197060 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:16.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.828+0000 7f06e1f75700 1 -- 172.21.15.144:0/4162519431 >> 172.21.15.144:0/4162519431 conn(0x7f06dc074f50 msgr2=0x7f06dc0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:16.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.828+0000 7f06e1f75700 1 -- 172.21.15.144:0/4162519431 shutdown_connections 2024-09-10T11:42:16.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.828+0000 7f06e1f75700 1 -- 172.21.15.144:0/4162519431 wait complete. 2024-09-10T11:42:16.840 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:42:16.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.934+0000 7fe054bbe700 1 -- 172.21.15.144:0/3782317717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe050100d70 msgr2=0x7fe050104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:16.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.934+0000 7fe054bbe700 1 --2- 172.21.15.144:0/3782317717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe050100d70 0x7fe050104d40 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe040009910 tx=0x7fe040009c20 comp rx=0 tx=0).stop 2024-09-10T11:42:16.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.934+0000 7fe054bbe700 1 -- 172.21.15.144:0/3782317717 shutdown_connections 2024-09-10T11:42:16.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.934+0000 7fe054bbe700 1 --2- 172.21.15.144:0/3782317717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe050100d70 0x7fe050104d40 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:16.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.934+0000 7fe054bbe700 1 --2- 172.21.15.144:0/3782317717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0501003d0 0x7fe0501007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:16.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.934+0000 7fe054bbe700 1 -- 172.21.15.144:0/3782317717 >> 172.21.15.144:0/3782317717 conn(0x7fe0500fbc90 msgr2=0x7fe0500fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:16.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.934+0000 7fe054bbe700 1 -- 172.21.15.144:0/3782317717 shutdown_connections 2024-09-10T11:42:16.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.934+0000 7fe054bbe700 1 -- 172.21.15.144:0/3782317717 wait complete. 2024-09-10T11:42:16.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.934+0000 7fe054bbe700 1 Processor -- start 2024-09-10T11:42:16.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.934+0000 7fe054bbe700 1 -- start start 2024-09-10T11:42:16.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe054bbe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0501003d0 0x7fe050073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:16.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe054bbe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050100d70 0x7fe050073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe054bbe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe050073ad0 con 0x7fe0501003d0 2024-09-10T11:42:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe054bbe700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe050073c10 con 0x7fe050100d70 2024-09-10T11:42:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe04e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0501003d0 0x7fe050073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe04e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0501003d0 0x7fe050073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52766/0 (socket says 172.21.15.144:52766) 2024-09-10T11:42:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe04e59c700 1 -- 172.21.15.144:0/2945965779 learned_addr learned my addr 172.21.15.144:0/2945965779 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:42:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe04dd9b700 1 --2- 172.21.15.144:0/2945965779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050100d70 0x7fe050073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe04e59c700 1 -- 172.21.15.144:0/2945965779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050100d70 msgr2=0x7fe050073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe04e59c700 1 --2- 172.21.15.144:0/2945965779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050100d70 0x7fe050073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe04e59c700 1 -- 172.21.15.144:0/2945965779 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0400095f0 con 0x7fe0501003d0 2024-09-10T11:42:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe04dd9b700 1 --2- 172.21.15.144:0/2945965779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050100d70 0x7fe050073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:42:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe04e59c700 1 --2- 172.21.15.144:0/2945965779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0501003d0 0x7fe050073050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe04400c8a0 tx=0x7fe04400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe03f7fe700 1 -- 172.21.15.144:0/2945965779 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe04400cea0 con 0x7fe0501003d0 2024-09-10T11:42:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe03f7fe700 1 -- 172.21.15.144:0/2945965779 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe044004830 con 0x7fe0501003d0 2024-09-10T11:42:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe054bbe700 1 -- 172.21.15.144:0/2945965779 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0501a1f10 con 0x7fe0501003d0 2024-09-10T11:42:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe03f7fe700 1 -- 172.21.15.144:0/2945965779 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0440055b0 con 0x7fe0501003d0 2024-09-10T11:42:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.935+0000 7fe054bbe700 1 -- 172.21.15.144:0/2945965779 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0501a2320 con 0x7fe0501003d0 2024-09-10T11:42:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.936+0000 7fe054bbe700 1 -- 172.21.15.144:0/2945965779 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe050108740 con 0x7fe0501003d0 2024-09-10T11:42:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.937+0000 7fe03f7fe700 1 -- 172.21.15.144:0/2945965779 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe0440049a0 con 0x7fe0501003d0 2024-09-10T11:42:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.937+0000 7fe03f7fe700 1 --2- 172.21.15.144:0/2945965779 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe03806e200 0x7fe0380706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.937+0000 7fe03f7fe700 1 -- 172.21.15.144:0/2945965779 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe04408dd40 con 0x7fe0501003d0 2024-09-10T11:42:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.937+0000 7fe04dd9b700 1 --2- 172.21.15.144:0/2945965779 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe03806e200 0x7fe0380706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:16.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.938+0000 7fe04dd9b700 1 --2- 172.21.15.144:0/2945965779 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe03806e200 0x7fe0380706b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe050074840 tx=0x7fe04001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:16.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:16.942+0000 7fe03f7fe700 1 -- 172.21.15.144:0/2945965779 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe044057120 con 0x7fe0501003d0 2024-09-10T11:42:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:16 smithi178 ceph-mon[65940]: pgmap v308: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:17.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.161+0000 7fe054bbe700 1 -- 172.21.15.144:0/2945965779 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe05002cc80 con 0x7fe03806e200 2024-09-10T11:42:17.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.162+0000 7fe03f7fe700 1 -- 172.21.15.144:0/2945965779 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe05002cc80 con 0x7fe03806e200 2024-09-10T11:42:17.165 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.164+0000 7fe054bbe700 1 -- 172.21.15.144:0/2945965779 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe03806e200 msgr2=0x7fe0380706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:17.165 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.164+0000 7fe054bbe700 1 --2- 172.21.15.144:0/2945965779 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe03806e200 0x7fe0380706b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fe050074840 tx=0x7fe04001a040 comp rx=0 tx=0).stop 2024-09-10T11:42:17.165 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.164+0000 7fe054bbe700 1 -- 172.21.15.144:0/2945965779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0501003d0 msgr2=0x7fe050073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:17.165 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.164+0000 7fe054bbe700 1 --2- 172.21.15.144:0/2945965779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0501003d0 0x7fe050073050 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fe04400c8a0 tx=0x7fe04400cc60 comp rx=0 tx=0).stop 2024-09-10T11:42:17.165 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.164+0000 7fe054bbe700 1 -- 172.21.15.144:0/2945965779 shutdown_connections 2024-09-10T11:42:17.165 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.164+0000 7fe054bbe700 1 --2- 172.21.15.144:0/2945965779 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe03806e200 0x7fe0380706b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.166 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.164+0000 7fe054bbe700 1 --2- 172.21.15.144:0/2945965779 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0501003d0 0x7fe050073050 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.166 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.164+0000 7fe054bbe700 1 --2- 172.21.15.144:0/2945965779 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe050100d70 0x7fe050073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.166 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.164+0000 7fe054bbe700 1 -- 172.21.15.144:0/2945965779 >> 172.21.15.144:0/2945965779 conn(0x7fe0500fbc90 msgr2=0x7fe0500fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:17.166 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.164+0000 7fe054bbe700 1 -- 172.21.15.144:0/2945965779 shutdown_connections 2024-09-10T11:42:17.166 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.164+0000 7fe054bbe700 1 -- 172.21.15.144:0/2945965779 wait complete. 2024-09-10T11:42:17.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.269+0000 7fccab040700 1 -- 172.21.15.144:0/3247111643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca40ffd30 msgr2=0x7fcca4103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:17.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.269+0000 7fccab040700 1 --2- 172.21.15.144:0/3247111643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca40ffd30 0x7fcca4103ce0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009910 tx=0x7fcc90009c20 comp rx=0 tx=0).stop 2024-09-10T11:42:17.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.269+0000 7fccab040700 1 -- 172.21.15.144:0/3247111643 shutdown_connections 2024-09-10T11:42:17.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.269+0000 7fccab040700 1 --2- 172.21.15.144:0/3247111643 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca40ffd30 0x7fcca4103ce0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.269+0000 7fccab040700 1 --2- 172.21.15.144:0/3247111643 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcca40ff390 0x7fcca40ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.269+0000 7fccab040700 1 -- 172.21.15.144:0/3247111643 >> 172.21.15.144:0/3247111643 conn(0x7fcca4074f50 msgr2=0x7fcca4075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:17.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.269+0000 7fccab040700 1 -- 172.21.15.144:0/3247111643 shutdown_connections 2024-09-10T11:42:17.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.269+0000 7fccab040700 1 -- 172.21.15.144:0/3247111643 wait complete. 2024-09-10T11:42:17.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.270+0000 7fccab040700 1 Processor -- start 2024-09-10T11:42:17.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.270+0000 7fccab040700 1 -- start start 2024-09-10T11:42:17.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.270+0000 7fccab040700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcca40ff390 0x7fcca4197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:17.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.270+0000 7fccab040700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca40ffd30 0x7fcca4198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:17.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.270+0000 7fccab040700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca4198bf0 con 0x7fcca40ffd30 2024-09-10T11:42:17.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.270+0000 7fccab040700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcca419c980 con 0x7fcca40ff390 2024-09-10T11:42:17.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.270+0000 7fcca3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca40ffd30 0x7fcca4198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:17.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.270+0000 7fcca3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca40ffd30 0x7fcca4198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52782/0 (socket says 172.21.15.144:52782) 2024-09-10T11:42:17.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.270+0000 7fcca3fff700 1 -- 172.21.15.144:0/263224711 learned_addr learned my addr 172.21.15.144:0/263224711 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:42:17.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.270+0000 7fcca8ddc700 1 --2- 172.21.15.144:0/263224711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcca40ff390 0x7fcca4197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:17.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.271+0000 7fcca3fff700 1 -- 172.21.15.144:0/263224711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcca40ff390 msgr2=0x7fcca4197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:17.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.271+0000 7fcca3fff700 1 --2- 172.21.15.144:0/263224711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcca40ff390 0x7fcca4197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.271+0000 7fcca3fff700 1 -- 172.21.15.144:0/263224711 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc900095f0 con 0x7fcca40ffd30 2024-09-10T11:42:17.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.271+0000 7fcca3fff700 1 --2- 172.21.15.144:0/263224711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca40ffd30 0x7fcca4198510 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc900092e0 tx=0x7fcc9000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:17.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.271+0000 7fcca1ffb700 1 -- 172.21.15.144:0/263224711 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc9001d070 con 0x7fcca40ffd30 2024-09-10T11:42:17.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.271+0000 7fcca1ffb700 1 -- 172.21.15.144:0/263224711 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcc90005dc0 con 0x7fcca40ffd30 2024-09-10T11:42:17.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.271+0000 7fccab040700 1 -- 172.21.15.144:0/263224711 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcca419cc00 con 0x7fcca40ffd30 2024-09-10T11:42:17.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.271+0000 7fcca1ffb700 1 -- 172.21.15.144:0/263224711 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcc90017400 con 0x7fcca40ffd30 2024-09-10T11:42:17.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.271+0000 7fccab040700 1 -- 172.21.15.144:0/263224711 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcca419d0f0 con 0x7fcca40ffd30 2024-09-10T11:42:17.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.272+0000 7fccab040700 1 -- 172.21.15.144:0/263224711 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcca404ea50 con 0x7fcca40ffd30 2024-09-10T11:42:17.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.273+0000 7fcca1ffb700 1 -- 172.21.15.144:0/263224711 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fcc9000f460 con 0x7fcca40ffd30 2024-09-10T11:42:17.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.273+0000 7fcca1ffb700 1 --2- 172.21.15.144:0/263224711 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcc9406e200 0x7fcc940706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:17.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.273+0000 7fcca1ffb700 1 -- 172.21.15.144:0/263224711 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcc9008f590 con 0x7fcca40ffd30 2024-09-10T11:42:17.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.273+0000 7fcca8ddc700 1 --2- 172.21.15.144:0/263224711 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcc9406e200 0x7fcc940706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:17.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.274+0000 7fcca8ddc700 1 --2- 172.21.15.144:0/263224711 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcc9406e200 0x7fcc940706b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc98005950 tx=0x7fcc980058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:17.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.278+0000 7fcca1ffb700 1 -- 172.21.15.144:0/263224711 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcc90058970 con 0x7fcca40ffd30 2024-09-10T11:42:17.489 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.488+0000 7fccab040700 1 -- 172.21.15.144:0/263224711 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcca419d5d0 con 0x7fcc9406e200 2024-09-10T11:42:17.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.498+0000 7fcca1ffb700 1 -- 172.21.15.144:0/263224711 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fcca419d5d0 con 0x7fcc9406e200 2024-09-10T11:42:17.499 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:42:17.499 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (10m) 4m ago 20m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:42:17.499 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (21m) 4m ago 21m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:42:17.499 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (20m) 3m ago 20m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:42:17.499 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (8m) 4m ago 21m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:42:17.499 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (8m) 3m ago 20m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:42:17.499 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (10m) 4m ago 20m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:42:17.499 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (4m) 4m ago 15m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:42:17.500 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (4m) 4m ago 15m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:42:17.500 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (15m) 3m ago 15m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:42:17.500 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (4m) 3m ago 15m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:42:17.500 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (12m) 4m ago 22m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:42:17.500 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (11m) 3m ago 20m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:42:17.500 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (9m) 4m ago 22m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:42:17.500 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (8m) 3m ago 20m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:42:17.500 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (11m) 4m ago 21m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:42:17.500 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (11m) 3m ago 20m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:42:17.500 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (7m) 4m ago 19m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:42:17.500 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (7m) 4m ago 18m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:42:17.500 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (6m) 4m ago 18m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:42:17.500 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (5m) 3m ago 17m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:42:17.501 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (5m) 3m ago 17m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:42:17.501 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (4m) 3m ago 17m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:42:17.501 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (10m) 4m ago 20m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:42:17.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.500+0000 7fccab040700 1 -- 172.21.15.144:0/263224711 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcc9406e200 msgr2=0x7fcc940706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:17.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.500+0000 7fccab040700 1 --2- 172.21.15.144:0/263224711 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcc9406e200 0x7fcc940706b0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc98005950 tx=0x7fcc980058e0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.500+0000 7fccab040700 1 -- 172.21.15.144:0/263224711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca40ffd30 msgr2=0x7fcca4198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:17.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.500+0000 7fccab040700 1 --2- 172.21.15.144:0/263224711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca40ffd30 0x7fcca4198510 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fcc900092e0 tx=0x7fcc9000be20 comp rx=0 tx=0).stop 2024-09-10T11:42:17.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.500+0000 7fccab040700 1 -- 172.21.15.144:0/263224711 shutdown_connections 2024-09-10T11:42:17.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.500+0000 7fccab040700 1 --2- 172.21.15.144:0/263224711 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcc9406e200 0x7fcc940706b0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.500+0000 7fccab040700 1 --2- 172.21.15.144:0/263224711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcca40ff390 0x7fcca4197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.500+0000 7fccab040700 1 --2- 172.21.15.144:0/263224711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcca40ffd30 0x7fcca4198510 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.500+0000 7fccab040700 1 -- 172.21.15.144:0/263224711 >> 172.21.15.144:0/263224711 conn(0x7fcca4074f50 msgr2=0x7fcca4068cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:17.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.500+0000 7fccab040700 1 -- 172.21.15.144:0/263224711 shutdown_connections 2024-09-10T11:42:17.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.500+0000 7fccab040700 1 -- 172.21.15.144:0/263224711 wait complete. 2024-09-10T11:42:17.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.604+0000 7fd38c21b700 1 -- 172.21.15.144:0/1363122596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3841003d0 msgr2=0x7fd3841007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:17.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.604+0000 7fd38c21b700 1 --2- 172.21.15.144:0/1363122596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3841003d0 0x7fd3841007a0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fd380009910 tx=0x7fd380009c20 comp rx=0 tx=0).stop 2024-09-10T11:42:17.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.604+0000 7fd38c21b700 1 -- 172.21.15.144:0/1363122596 shutdown_connections 2024-09-10T11:42:17.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.604+0000 7fd38c21b700 1 --2- 172.21.15.144:0/1363122596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd384100d70 0x7fd384104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.604+0000 7fd38c21b700 1 --2- 172.21.15.144:0/1363122596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3841003d0 0x7fd3841007a0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.604+0000 7fd38c21b700 1 -- 172.21.15.144:0/1363122596 >> 172.21.15.144:0/1363122596 conn(0x7fd3840fbc90 msgr2=0x7fd3840fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:17.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.605+0000 7fd38c21b700 1 -- 172.21.15.144:0/1363122596 shutdown_connections 2024-09-10T11:42:17.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.605+0000 7fd38c21b700 1 -- 172.21.15.144:0/1363122596 wait complete. 2024-09-10T11:42:17.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.605+0000 7fd38c21b700 1 Processor -- start 2024-09-10T11:42:17.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.605+0000 7fd38c21b700 1 -- start start 2024-09-10T11:42:17.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.605+0000 7fd38c21b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3841003d0 0x7fd384195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:17.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.605+0000 7fd38c21b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd384100d70 0x7fd3841962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:17.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.605+0000 7fd38c21b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd384196980 con 0x7fd3841003d0 2024-09-10T11:42:17.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.605+0000 7fd38c21b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38419a710 con 0x7fd384100d70 2024-09-10T11:42:17.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.606+0000 7fd389fb7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3841003d0 0x7fd384195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:17.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.606+0000 7fd389fb7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3841003d0 0x7fd384195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52806/0 (socket says 172.21.15.144:52806) 2024-09-10T11:42:17.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.606+0000 7fd389fb7700 1 -- 172.21.15.144:0/1639465477 learned_addr learned my addr 172.21.15.144:0/1639465477 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:42:17.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.606+0000 7fd3897b6700 1 --2- 172.21.15.144:0/1639465477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd384100d70 0x7fd3841962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:17.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.606+0000 7fd389fb7700 1 -- 172.21.15.144:0/1639465477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd384100d70 msgr2=0x7fd3841962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:17.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.606+0000 7fd389fb7700 1 --2- 172.21.15.144:0/1639465477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd384100d70 0x7fd3841962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.606+0000 7fd389fb7700 1 -- 172.21.15.144:0/1639465477 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3800095f0 con 0x7fd3841003d0 2024-09-10T11:42:17.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.606+0000 7fd389fb7700 1 --2- 172.21.15.144:0/1639465477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3841003d0 0x7fd384195d60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd380006010 tx=0x7fd38000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:17.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.606+0000 7fd37affd700 1 -- 172.21.15.144:0/1639465477 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd38001d070 con 0x7fd3841003d0 2024-09-10T11:42:17.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.606+0000 7fd37affd700 1 -- 172.21.15.144:0/1639465477 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd380005dc0 con 0x7fd3841003d0 2024-09-10T11:42:17.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.606+0000 7fd38c21b700 1 -- 172.21.15.144:0/1639465477 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38419a990 con 0x7fd3841003d0 2024-09-10T11:42:17.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.606+0000 7fd37affd700 1 -- 172.21.15.144:0/1639465477 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd380017400 con 0x7fd3841003d0 2024-09-10T11:42:17.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.606+0000 7fd38c21b700 1 -- 172.21.15.144:0/1639465477 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38419ae80 con 0x7fd3841003d0 2024-09-10T11:42:17.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.607+0000 7fd38c21b700 1 -- 172.21.15.144:0/1639465477 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3841086e0 con 0x7fd3841003d0 2024-09-10T11:42:17.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.608+0000 7fd37affd700 1 -- 172.21.15.144:0/1639465477 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd380003a60 con 0x7fd3841003d0 2024-09-10T11:42:17.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.608+0000 7fd37affd700 1 --2- 172.21.15.144:0/1639465477 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd37006e200 0x7fd3700706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:17.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.608+0000 7fd37affd700 1 -- 172.21.15.144:0/1639465477 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd38008f450 con 0x7fd3841003d0 2024-09-10T11:42:17.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.608+0000 7fd3897b6700 1 --2- 172.21.15.144:0/1639465477 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd37006e200 0x7fd3700706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:17.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.609+0000 7fd3897b6700 1 --2- 172.21.15.144:0/1639465477 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd37006e200 0x7fd3700706b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd384197380 tx=0x7fd37400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:17.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.613+0000 7fd37affd700 1 -- 172.21.15.144:0/1639465477 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd3800588b0 con 0x7fd3841003d0 2024-09-10T11:42:17.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.901+0000 7fd38c21b700 1 -- 172.21.15.144:0/1639465477 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd38419b1b0 con 0x7fd3841003d0 2024-09-10T11:42:17.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.902+0000 7fd37affd700 1 -- 172.21.15.144:0/1639465477 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fd38005bf70 con 0x7fd3841003d0 2024-09-10T11:42:17.903 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:42:17.903 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:42:17.903 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:42:17.903 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:42:17.903 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:42:17.903 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:42:17.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:42:17.904 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:42:17.904 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:42:17.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:42:17.904 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:42:17.904 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:42:17.904 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:42:17.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:42:17.904 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:42:17.904 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:42:17.904 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:42:17.904 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:42:17.904 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:42:17.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.904+0000 7fd38c21b700 1 -- 172.21.15.144:0/1639465477 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd37006e200 msgr2=0x7fd3700706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:17.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.904+0000 7fd38c21b700 1 --2- 172.21.15.144:0/1639465477 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd37006e200 0x7fd3700706b0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fd384197380 tx=0x7fd37400b410 comp rx=0 tx=0).stop 2024-09-10T11:42:17.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.904+0000 7fd38c21b700 1 -- 172.21.15.144:0/1639465477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3841003d0 msgr2=0x7fd384195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:17.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.904+0000 7fd38c21b700 1 --2- 172.21.15.144:0/1639465477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3841003d0 0x7fd384195d60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd380006010 tx=0x7fd38000bd70 comp rx=0 tx=0).stop 2024-09-10T11:42:17.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.904+0000 7fd38c21b700 1 -- 172.21.15.144:0/1639465477 shutdown_connections 2024-09-10T11:42:17.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.904+0000 7fd38c21b700 1 --2- 172.21.15.144:0/1639465477 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd37006e200 0x7fd3700706b0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.904+0000 7fd38c21b700 1 --2- 172.21.15.144:0/1639465477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3841003d0 0x7fd384195d60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.904+0000 7fd38c21b700 1 --2- 172.21.15.144:0/1639465477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd384100d70 0x7fd3841962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:17.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.904+0000 7fd38c21b700 1 -- 172.21.15.144:0/1639465477 >> 172.21.15.144:0/1639465477 conn(0x7fd3840fbc90 msgr2=0x7fd3840fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:17.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.904+0000 7fd38c21b700 1 -- 172.21.15.144:0/1639465477 shutdown_connections 2024-09-10T11:42:17.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:17.904+0000 7fd38c21b700 1 -- 172.21.15.144:0/1639465477 wait complete. 2024-09-10T11:42:18.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.009+0000 7f75eefdc700 1 -- 172.21.15.144:0/4069037437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e80ff2d0 msgr2=0x7f75e80ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:18.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.009+0000 7f75eefdc700 1 --2- 172.21.15.144:0/4069037437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e80ff2d0 0x7f75e80ff6a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f75dc009910 tx=0x7f75dc009c20 comp rx=0 tx=0).stop 2024-09-10T11:42:18.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.009+0000 7f75eefdc700 1 -- 172.21.15.144:0/4069037437 shutdown_connections 2024-09-10T11:42:18.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.009+0000 7f75eefdc700 1 --2- 172.21.15.144:0/4069037437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75e80ffc70 0x7f75e8103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.009+0000 7f75eefdc700 1 --2- 172.21.15.144:0/4069037437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e80ff2d0 0x7f75e80ff6a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.009+0000 7f75eefdc700 1 -- 172.21.15.144:0/4069037437 >> 172.21.15.144:0/4069037437 conn(0x7f75e80fabb0 msgr2=0x7f75e80fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:18.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.009+0000 7f75eefdc700 1 -- 172.21.15.144:0/4069037437 shutdown_connections 2024-09-10T11:42:18.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.009+0000 7f75eefdc700 1 -- 172.21.15.144:0/4069037437 wait complete. 2024-09-10T11:42:18.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.010+0000 7f75eefdc700 1 Processor -- start 2024-09-10T11:42:18.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.010+0000 7f75eefdc700 1 -- start start 2024-09-10T11:42:18.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.010+0000 7f75eefdc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e80ff2d0 0x7f75e8195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:18.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.010+0000 7f75eefdc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75e80ffc70 0x7f75e81962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:18.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.010+0000 7f75eefdc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e81969d0 con 0x7f75e80ff2d0 2024-09-10T11:42:18.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.010+0000 7f75eefdc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e819a760 con 0x7f75e80ffc70 2024-09-10T11:42:18.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.010+0000 7f75ecd78700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e80ff2d0 0x7f75e8195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:18.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.010+0000 7f75ecd78700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e80ff2d0 0x7f75e8195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52820/0 (socket says 172.21.15.144:52820) 2024-09-10T11:42:18.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.010+0000 7f75ecd78700 1 -- 172.21.15.144:0/588319017 learned_addr learned my addr 172.21.15.144:0/588319017 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:42:18.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.010+0000 7f75e7fff700 1 --2- 172.21.15.144:0/588319017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75e80ffc70 0x7f75e81962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:18.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.011+0000 7f75ecd78700 1 -- 172.21.15.144:0/588319017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75e80ffc70 msgr2=0x7f75e81962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:18.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.011+0000 7f75ecd78700 1 --2- 172.21.15.144:0/588319017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75e80ffc70 0x7f75e81962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.011+0000 7f75ecd78700 1 -- 172.21.15.144:0/588319017 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc0095f0 con 0x7f75e80ff2d0 2024-09-10T11:42:18.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.011+0000 7f75ecd78700 1 --2- 172.21.15.144:0/588319017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e80ff2d0 0x7f75e8195db0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f75dc006010 tx=0x7f75dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:18.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.011+0000 7f75e5ffb700 1 -- 172.21.15.144:0/588319017 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75dc01d070 con 0x7f75e80ff2d0 2024-09-10T11:42:18.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.011+0000 7f75e5ffb700 1 -- 172.21.15.144:0/588319017 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75dc00f460 con 0x7f75e80ff2d0 2024-09-10T11:42:18.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.011+0000 7f75eefdc700 1 -- 172.21.15.144:0/588319017 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e819a9e0 con 0x7f75e80ff2d0 2024-09-10T11:42:18.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.011+0000 7f75e5ffb700 1 -- 172.21.15.144:0/588319017 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75dc017630 con 0x7f75e80ff2d0 2024-09-10T11:42:18.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.011+0000 7f75eefdc700 1 -- 172.21.15.144:0/588319017 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e819aed0 con 0x7f75e80ff2d0 2024-09-10T11:42:18.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.012+0000 7f75eefdc700 1 -- 172.21.15.144:0/588319017 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e804ea50 con 0x7f75e80ff2d0 2024-09-10T11:42:18.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.013+0000 7f75e5ffb700 1 -- 172.21.15.144:0/588319017 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f75dc00f5d0 con 0x7f75e80ff2d0 2024-09-10T11:42:18.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.013+0000 7f75e5ffb700 1 --2- 172.21.15.144:0/588319017 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f75d806e200 0x7f75d80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:18.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.013+0000 7f75e5ffb700 1 -- 172.21.15.144:0/588319017 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f75dc08f590 con 0x7f75e80ff2d0 2024-09-10T11:42:18.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.013+0000 7f75e7fff700 1 --2- 172.21.15.144:0/588319017 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f75d806e200 0x7f75d80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:18.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.013+0000 7f75e7fff700 1 --2- 172.21.15.144:0/588319017 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f75d806e200 0x7f75d80706b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f75d4005950 tx=0x7f75d40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:18.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.017+0000 7f75e5ffb700 1 -- 172.21.15.144:0/588319017 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f75dc058950 con 0x7f75e80ff2d0 2024-09-10T11:42:18.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.263+0000 7f75eefdc700 1 -- 172.21.15.144:0/588319017 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f75e819b3e0 con 0x7f75e80ff2d0 2024-09-10T11:42:18.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.264+0000 7f75e5ffb700 1 -- 172.21.15.144:0/588319017 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f75dc05c010 con 0x7f75e80ff2d0 2024-09-10T11:42:18.265 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:42:18.265 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:42:18.265 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:42:18.266 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:42:18.266 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:42:18.266 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:42:18.266 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:42:18.266 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:42:18.266 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:42:18.266 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:42:18.266 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:42:18.266 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:42:18.266 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:42:18.266 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:42:18.266 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:42:18.266 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:42:18.267 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:42:18.267 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:42:18.267 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:42:18.267 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:42:18.267 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:42:18.267 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:42:18.267 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:42:18.267 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:42:18.267 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:42:18.267 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:42:18.267 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:42:18.267 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:42:18.267 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:42:18.268 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:42:18.268 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:42:18.268 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:42:18.268 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:42:18.268 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:42:18.268 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:42:18.268 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:42:18.268 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:42:18.268 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:42:18.268 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:42:18.268 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:42:18.268 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:42:18.268 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:42:18.269 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:42:18.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.267+0000 7f75eefdc700 1 -- 172.21.15.144:0/588319017 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f75d806e200 msgr2=0x7f75d80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:18.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.267+0000 7f75eefdc700 1 --2- 172.21.15.144:0/588319017 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f75d806e200 0x7f75d80706b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f75d4005950 tx=0x7f75d40058e0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.267+0000 7f75eefdc700 1 -- 172.21.15.144:0/588319017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e80ff2d0 msgr2=0x7f75e8195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:18.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.267+0000 7f75eefdc700 1 --2- 172.21.15.144:0/588319017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e80ff2d0 0x7f75e8195db0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f75dc006010 tx=0x7f75dc00be20 comp rx=0 tx=0).stop 2024-09-10T11:42:18.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.267+0000 7f75eefdc700 1 -- 172.21.15.144:0/588319017 shutdown_connections 2024-09-10T11:42:18.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.267+0000 7f75eefdc700 1 --2- 172.21.15.144:0/588319017 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f75d806e200 0x7f75d80706b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.267+0000 7f75eefdc700 1 --2- 172.21.15.144:0/588319017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75e80ff2d0 0x7f75e8195db0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.267+0000 7f75eefdc700 1 --2- 172.21.15.144:0/588319017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75e80ffc70 0x7f75e81962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.267+0000 7f75eefdc700 1 -- 172.21.15.144:0/588319017 >> 172.21.15.144:0/588319017 conn(0x7f75e80fabb0 msgr2=0x7f75e8068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:18.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.267+0000 7f75eefdc700 1 -- 172.21.15.144:0/588319017 shutdown_connections 2024-09-10T11:42:18.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.267+0000 7f75eefdc700 1 -- 172.21.15.144:0/588319017 wait complete. 2024-09-10T11:42:18.270 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:42:18.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.373+0000 7f6535d56700 1 -- 172.21.15.144:0/3161783140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6530101380 msgr2=0x7f6530101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:18.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.373+0000 7f6535d56700 1 --2- 172.21.15.144:0/3161783140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6530101380 0x7f6530101750 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f6520009910 tx=0x7f6520009c20 comp rx=0 tx=0).stop 2024-09-10T11:42:18.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.373+0000 7f6535d56700 1 -- 172.21.15.144:0/3161783140 shutdown_connections 2024-09-10T11:42:18.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.373+0000 7f6535d56700 1 --2- 172.21.15.144:0/3161783140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530101c90 0x7f653010a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.373+0000 7f6535d56700 1 --2- 172.21.15.144:0/3161783140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6530101380 0x7f6530101750 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.373+0000 7f6535d56700 1 -- 172.21.15.144:0/3161783140 >> 172.21.15.144:0/3161783140 conn(0x7f65300fab80 msgr2=0x7f65300fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:18.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.373+0000 7f6535d56700 1 -- 172.21.15.144:0/3161783140 shutdown_connections 2024-09-10T11:42:18.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.373+0000 7f6535d56700 1 -- 172.21.15.144:0/3161783140 wait complete. 2024-09-10T11:42:18.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.374+0000 7f6535d56700 1 Processor -- start 2024-09-10T11:42:18.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.374+0000 7f6535d56700 1 -- start start 2024-09-10T11:42:18.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.374+0000 7f6535d56700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530101380 0x7f6530195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:18.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.374+0000 7f6535d56700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6530101c90 0x7f65301962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:18.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.374+0000 7f6535d56700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6530196990 con 0x7f6530101c90 2024-09-10T11:42:18.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.374+0000 7f6535d56700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f653019a6d0 con 0x7f6530101380 2024-09-10T11:42:18.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.374+0000 7f6527fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6530101c90 0x7f65301962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:18.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.374+0000 7f6527fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6530101c90 0x7f65301962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52840/0 (socket says 172.21.15.144:52840) 2024-09-10T11:42:18.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.374+0000 7f6527fff700 1 -- 172.21.15.144:0/2005179059 learned_addr learned my addr 172.21.15.144:0/2005179059 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:42:18.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.375+0000 7f652f7fe700 1 --2- 172.21.15.144:0/2005179059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530101380 0x7f6530195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:18.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.375+0000 7f6527fff700 1 -- 172.21.15.144:0/2005179059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530101380 msgr2=0x7f6530195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:18.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.375+0000 7f6527fff700 1 --2- 172.21.15.144:0/2005179059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530101380 0x7f6530195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.375+0000 7f6527fff700 1 -- 172.21.15.144:0/2005179059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65200095f0 con 0x7f6530101c90 2024-09-10T11:42:18.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.375+0000 7f6527fff700 1 --2- 172.21.15.144:0/2005179059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6530101c90 0x7f65301962b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f651800e920 tx=0x7f651800ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:18.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.375+0000 7f652d7fa700 1 -- 172.21.15.144:0/2005179059 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6518009790 con 0x7f6530101c90 2024-09-10T11:42:18.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.375+0000 7f652d7fa700 1 -- 172.21.15.144:0/2005179059 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f651800cba0 con 0x7f6530101c90 2024-09-10T11:42:18.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.375+0000 7f6535d56700 1 -- 172.21.15.144:0/2005179059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f653019a9b0 con 0x7f6530101c90 2024-09-10T11:42:18.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.375+0000 7f652d7fa700 1 -- 172.21.15.144:0/2005179059 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6518018650 con 0x7f6530101c90 2024-09-10T11:42:18.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.375+0000 7f6535d56700 1 -- 172.21.15.144:0/2005179059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f653019af00 con 0x7f6530101c90 2024-09-10T11:42:18.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.376+0000 7f6535d56700 1 -- 172.21.15.144:0/2005179059 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6530107860 con 0x7f6530101c90 2024-09-10T11:42:18.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.377+0000 7f652d7fa700 1 -- 172.21.15.144:0/2005179059 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f651800cd10 con 0x7f6530101c90 2024-09-10T11:42:18.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.377+0000 7f652d7fa700 1 --2- 172.21.15.144:0/2005179059 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f651c06e200 0x7f651c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:18.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.377+0000 7f652d7fa700 1 -- 172.21.15.144:0/2005179059 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6518014070 con 0x7f6530101c90 2024-09-10T11:42:18.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.377+0000 7f652f7fe700 1 --2- 172.21.15.144:0/2005179059 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f651c06e200 0x7f651c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:18.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.378+0000 7f652f7fe700 1 --2- 172.21.15.144:0/2005179059 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f651c06e200 0x7f651c0706b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f652000b760 tx=0x7f6520009f90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:18.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.381+0000 7f652d7fa700 1 -- 172.21.15.144:0/2005179059 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f65180541b0 con 0x7f6530101c90 2024-09-10T11:42:18.602 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.601+0000 7f6535d56700 1 -- 172.21.15.144:0/2005179059 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f653019b260 con 0x7f651c06e200 2024-09-10T11:42:18.603 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.602+0000 7f652d7fa700 1 -- 172.21.15.144:0/2005179059 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f653019b260 con 0x7f651c06e200 2024-09-10T11:42:18.603 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:42:18.603 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:42:18.603 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:42:18.604 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:42:18.604 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:42:18.604 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:42:18.604 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:42:18.604 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:42:18.604 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:42:18.604 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:42:18.604 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:42:18.604 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:42:18.604 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:42:18.604 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:42:18.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.605+0000 7f6535d56700 1 -- 172.21.15.144:0/2005179059 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f651c06e200 msgr2=0x7f651c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:18.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.605+0000 7f6535d56700 1 --2- 172.21.15.144:0/2005179059 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f651c06e200 0x7f651c0706b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f652000b760 tx=0x7f6520009f90 comp rx=0 tx=0).stop 2024-09-10T11:42:18.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.605+0000 7f6535d56700 1 -- 172.21.15.144:0/2005179059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6530101c90 msgr2=0x7f65301962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:18.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.605+0000 7f6535d56700 1 --2- 172.21.15.144:0/2005179059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6530101c90 0x7f65301962b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f651800e920 tx=0x7f651800ece0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.605+0000 7f6535d56700 1 -- 172.21.15.144:0/2005179059 shutdown_connections 2024-09-10T11:42:18.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.605+0000 7f6535d56700 1 --2- 172.21.15.144:0/2005179059 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f651c06e200 0x7f651c0706b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.605+0000 7f6535d56700 1 --2- 172.21.15.144:0/2005179059 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6530101380 0x7f6530195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.605+0000 7f6535d56700 1 --2- 172.21.15.144:0/2005179059 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6530101c90 0x7f65301962b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.605+0000 7f6535d56700 1 -- 172.21.15.144:0/2005179059 >> 172.21.15.144:0/2005179059 conn(0x7f65300fab80 msgr2=0x7f65301049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:18.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.605+0000 7f6535d56700 1 -- 172.21.15.144:0/2005179059 shutdown_connections 2024-09-10T11:42:18.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.605+0000 7f6535d56700 1 -- 172.21.15.144:0/2005179059 wait complete. 2024-09-10T11:42:18.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.711+0000 7f11bd91c700 1 -- 172.21.15.144:0/2966262786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b80ff120 msgr2=0x7f11b80ff4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:18.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.711+0000 7f11bd91c700 1 --2- 172.21.15.144:0/2966262786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b80ff120 0x7f11b80ff4f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f11a8009910 tx=0x7f11a8009c20 comp rx=0 tx=0).stop 2024-09-10T11:42:18.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.711+0000 7f11bd91c700 1 -- 172.21.15.144:0/2966262786 shutdown_connections 2024-09-10T11:42:18.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.711+0000 7f11bd91c700 1 --2- 172.21.15.144:0/2966262786 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11b80ffa30 0x7f11b8103f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.711+0000 7f11bd91c700 1 --2- 172.21.15.144:0/2966262786 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b80ff120 0x7f11b80ff4f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.711+0000 7f11bd91c700 1 -- 172.21.15.144:0/2966262786 >> 172.21.15.144:0/2966262786 conn(0x7f11b80fabb0 msgr2=0x7f11b80fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:18.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.711+0000 7f11bd91c700 1 -- 172.21.15.144:0/2966262786 shutdown_connections 2024-09-10T11:42:18.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.711+0000 7f11bd91c700 1 -- 172.21.15.144:0/2966262786 wait complete. 2024-09-10T11:42:18.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.711+0000 7f11bd91c700 1 Processor -- start 2024-09-10T11:42:18.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.712+0000 7f11bd91c700 1 -- start start 2024-09-10T11:42:18.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.712+0000 7f11bd91c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b80ff120 0x7f11b819c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:18.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.712+0000 7f11bd91c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11b80ffa30 0x7f11b819c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:18.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.712+0000 7f11bd91c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b819cf30 con 0x7f11b80ff120 2024-09-10T11:42:18.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.712+0000 7f11bd91c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b81a0cc0 con 0x7f11b80ffa30 2024-09-10T11:42:18.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.712+0000 7f11b6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b80ff120 0x7f11b819c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:18.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.712+0000 7f11b6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b80ff120 0x7f11b819c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52862/0 (socket says 172.21.15.144:52862) 2024-09-10T11:42:18.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.712+0000 7f11b6ffd700 1 -- 172.21.15.144:0/2309199683 learned_addr learned my addr 172.21.15.144:0/2309199683 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:42:18.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.712+0000 7f11affff700 1 --2- 172.21.15.144:0/2309199683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11b80ffa30 0x7f11b819c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:18.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.712+0000 7f11b6ffd700 1 -- 172.21.15.144:0/2309199683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11b80ffa30 msgr2=0x7f11b819c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:18.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.712+0000 7f11b6ffd700 1 --2- 172.21.15.144:0/2309199683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11b80ffa30 0x7f11b819c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:18.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.712+0000 7f11b6ffd700 1 -- 172.21.15.144:0/2309199683 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a80095f0 con 0x7f11b80ff120 2024-09-10T11:42:18.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.712+0000 7f11b6ffd700 1 --2- 172.21.15.144:0/2309199683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b80ff120 0x7f11b819c310 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f11a8006010 tx=0x7f11a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:18.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.712+0000 7f11bc91a700 1 -- 172.21.15.144:0/2309199683 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11a801d070 con 0x7f11b80ff120 2024-09-10T11:42:18.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.712+0000 7f11bc91a700 1 -- 172.21.15.144:0/2309199683 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f11a8004fb0 con 0x7f11b80ff120 2024-09-10T11:42:18.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.713+0000 7f11bc91a700 1 -- 172.21.15.144:0/2309199683 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11a8022ea0 con 0x7f11b80ff120 2024-09-10T11:42:18.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.713+0000 7f11bd91c700 1 -- 172.21.15.144:0/2309199683 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b81a0f40 con 0x7f11b80ff120 2024-09-10T11:42:18.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.713+0000 7f11bd91c700 1 -- 172.21.15.144:0/2309199683 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b81a1430 con 0x7f11b80ff120 2024-09-10T11:42:18.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.714+0000 7f11bd91c700 1 -- 172.21.15.144:0/2309199683 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b8101640 con 0x7f11b80ff120 2024-09-10T11:42:18.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.714+0000 7f11bc91a700 1 -- 172.21.15.144:0/2309199683 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f11a8022470 con 0x7f11b80ff120 2024-09-10T11:42:18.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.715+0000 7f11bc91a700 1 --2- 172.21.15.144:0/2309199683 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f11a406e200 0x7f11a40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:18.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.715+0000 7f11affff700 1 --2- 172.21.15.144:0/2309199683 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f11a406e200 0x7f11a40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:18.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.715+0000 7f11bc91a700 1 -- 172.21.15.144:0/2309199683 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f11a808f540 con 0x7f11b80ff120 2024-09-10T11:42:18.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.715+0000 7f11affff700 1 --2- 172.21.15.144:0/2309199683 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f11a406e200 0x7f11a40706b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f11a0005fd0 tx=0x7f11a000a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:18.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:18.719+0000 7f11bc91a700 1 -- 172.21.15.144:0/2309199683 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f11a8058a50 con 0x7f11b80ff120 2024-09-10T11:42:18.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:18 smithi144 ceph-mon[87436]: from='client.34624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:18 smithi144 ceph-mon[87436]: from='client.34628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:18 smithi144 ceph-mon[87436]: from='client.34632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:18 smithi144 ceph-mon[87436]: pgmap v309: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:18 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1639465477' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:42:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:18 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/588319017' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:42:19.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:19.005+0000 7f11bd91c700 1 -- 172.21.15.144:0/2309199683 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f11b802ccf0 con 0x7f11b80ff120 2024-09-10T11:42:19.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:19.006+0000 7f11bc91a700 1 -- 172.21.15.144:0/2309199683 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f11a8022720 con 0x7f11b80ff120 2024-09-10T11:42:19.007 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:42:19.007 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:42:19.007 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:42:19.007 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:42:19.007 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:42:19.008 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:42:19.008 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:42:19.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:19.008+0000 7f11bd91c700 1 -- 172.21.15.144:0/2309199683 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f11a406e200 msgr2=0x7f11a40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:19.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:19.008+0000 7f11bd91c700 1 --2- 172.21.15.144:0/2309199683 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f11a406e200 0x7f11a40706b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f11a0005fd0 tx=0x7f11a000a560 comp rx=0 tx=0).stop 2024-09-10T11:42:19.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:19.008+0000 7f11bd91c700 1 -- 172.21.15.144:0/2309199683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b80ff120 msgr2=0x7f11b819c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:19.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:19.008+0000 7f11bd91c700 1 --2- 172.21.15.144:0/2309199683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b80ff120 0x7f11b819c310 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f11a8006010 tx=0x7f11a800be20 comp rx=0 tx=0).stop 2024-09-10T11:42:19.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:19.008+0000 7f11bd91c700 1 -- 172.21.15.144:0/2309199683 shutdown_connections 2024-09-10T11:42:19.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:19.008+0000 7f11bd91c700 1 --2- 172.21.15.144:0/2309199683 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f11a406e200 0x7f11a40706b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:19.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:19.008+0000 7f11bd91c700 1 --2- 172.21.15.144:0/2309199683 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11b80ff120 0x7f11b819c310 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:19.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:19.008+0000 7f11bd91c700 1 --2- 172.21.15.144:0/2309199683 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11b80ffa30 0x7f11b819c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:19.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:19.008+0000 7f11bd91c700 1 -- 172.21.15.144:0/2309199683 >> 172.21.15.144:0/2309199683 conn(0x7f11b80fabb0 msgr2=0x7f11b8110590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:19.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:19.008+0000 7f11bd91c700 1 -- 172.21.15.144:0/2309199683 shutdown_connections 2024-09-10T11:42:19.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:19.008+0000 7f11bd91c700 1 -- 172.21.15.144:0/2309199683 wait complete. 2024-09-10T11:42:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:18 smithi178 ceph-mon[65940]: from='client.34624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:18 smithi178 ceph-mon[65940]: from='client.34628 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:18 smithi178 ceph-mon[65940]: from='client.34632 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:18 smithi178 ceph-mon[65940]: pgmap v309: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:18 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1639465477' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:42:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:18 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/588319017' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:42:20.108 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:19 smithi178 ceph-mon[65940]: from='client.34644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:20.108 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:19 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2309199683' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:42:20.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:19 smithi144 ceph-mon[87436]: from='client.34644 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:20.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:19 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2309199683' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:42:21.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:20 smithi178 ceph-mon[65940]: pgmap v310: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:21.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:20 smithi144 ceph-mon[87436]: pgmap v310: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:23.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:22 smithi144 ceph-mon[87436]: pgmap v311: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:23.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:22 smithi178 ceph-mon[65940]: pgmap v311: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:24 smithi178 ceph-mon[65940]: pgmap v312: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:25.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:24 smithi144 ceph-mon[87436]: pgmap v312: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:26.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:26 smithi144 ceph-mon[87436]: pgmap v313: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:26 smithi178 ceph-mon[65940]: pgmap v313: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:28 smithi178 ceph-mon[65940]: pgmap v314: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:42:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:28 smithi144 ceph-mon[87436]: pgmap v314: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:42:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:30 smithi178 ceph-mon[65940]: pgmap v315: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:31.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:30 smithi144 ceph-mon[87436]: pgmap v315: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:32 smithi144 ceph-mon[87436]: pgmap v316: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:33.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:32 smithi178 ceph-mon[65940]: pgmap v316: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:33 smithi178 ceph-mon[65940]: pgmap v317: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:34.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:33 smithi144 ceph-mon[87436]: pgmap v317: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:36 smithi178 ceph-mon[65940]: pgmap v318: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:36 smithi144 ceph-mon[87436]: pgmap v318: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:38 smithi178 ceph-mon[65940]: pgmap v319: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:39.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:38 smithi144 ceph-mon[87436]: pgmap v319: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:40 smithi178 ceph-mon[65940]: pgmap v320: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:41.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:40 smithi144 ceph-mon[87436]: pgmap v320: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:43.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:42 smithi144 ceph-mon[87436]: pgmap v321: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:43.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:42 smithi178 ceph-mon[65940]: pgmap v321: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:42:44.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:42:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:44 smithi178 ceph-mon[65940]: pgmap v322: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:44 smithi144 ceph-mon[87436]: pgmap v322: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:46 smithi178 ceph-mon[65940]: pgmap v323: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:46 smithi144 ceph-mon[87436]: pgmap v323: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:48 smithi178 ceph-mon[65940]: pgmap v324: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:49.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.127+0000 7f932688a700 1 -- 172.21.15.144:0/3787866990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93200ffc60 msgr2=0x7f9320103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:49.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.127+0000 7f932688a700 1 --2- 172.21.15.144:0/3787866990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93200ffc60 0x7f9320103c30 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9314009910 tx=0x7f9314009c20 comp rx=0 tx=0).stop 2024-09-10T11:42:49.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.127+0000 7f932688a700 1 -- 172.21.15.144:0/3787866990 shutdown_connections 2024-09-10T11:42:49.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.127+0000 7f932688a700 1 --2- 172.21.15.144:0/3787866990 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93200ffc60 0x7f9320103c30 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:49.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.127+0000 7f932688a700 1 --2- 172.21.15.144:0/3787866990 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93200ff2c0 0x7f93200ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:49.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.127+0000 7f932688a700 1 -- 172.21.15.144:0/3787866990 >> 172.21.15.144:0/3787866990 conn(0x7f93200fab80 msgr2=0x7f93200fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:49.129 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:48 smithi144 ceph-mon[87436]: pgmap v324: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:49.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.127+0000 7f932688a700 1 -- 172.21.15.144:0/3787866990 shutdown_connections 2024-09-10T11:42:49.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.127+0000 7f932688a700 1 -- 172.21.15.144:0/3787866990 wait complete. 2024-09-10T11:42:49.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.127+0000 7f932688a700 1 Processor -- start 2024-09-10T11:42:49.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.128+0000 7f932688a700 1 -- start start 2024-09-10T11:42:49.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.128+0000 7f932688a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93200ff2c0 0x7f9320197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:49.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.128+0000 7f932688a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93200ffc60 0x7f9320198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:49.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.128+0000 7f932688a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9320198b50 con 0x7f93200ff2c0 2024-09-10T11:42:49.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.128+0000 7f932688a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f932019c8e0 con 0x7f93200ffc60 2024-09-10T11:42:49.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.128+0000 7f931ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93200ff2c0 0x7f9320197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:49.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.128+0000 7f931ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93200ff2c0 0x7f9320197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36094/0 (socket says 172.21.15.144:36094) 2024-09-10T11:42:49.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.128+0000 7f931ffff700 1 -- 172.21.15.144:0/1835728039 learned_addr learned my addr 172.21.15.144:0/1835728039 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:42:49.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.128+0000 7f931f7fe700 1 --2- 172.21.15.144:0/1835728039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93200ffc60 0x7f9320198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:49.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.128+0000 7f931ffff700 1 -- 172.21.15.144:0/1835728039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93200ffc60 msgr2=0x7f9320198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:49.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.128+0000 7f931ffff700 1 --2- 172.21.15.144:0/1835728039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93200ffc60 0x7f9320198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:49.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.128+0000 7f931ffff700 1 -- 172.21.15.144:0/1835728039 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93140095f0 con 0x7f93200ff2c0 2024-09-10T11:42:49.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.128+0000 7f931f7fe700 1 --2- 172.21.15.144:0/1835728039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93200ffc60 0x7f9320198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:42:49.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.129+0000 7f931ffff700 1 --2- 172.21.15.144:0/1835728039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93200ff2c0 0x7f9320197f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f931000d8d0 tx=0x7f931000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:49.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.129+0000 7f931d7fa700 1 -- 172.21.15.144:0/1835728039 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9310009940 con 0x7f93200ff2c0 2024-09-10T11:42:49.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.129+0000 7f931d7fa700 1 -- 172.21.15.144:0/1835728039 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9310010460 con 0x7f93200ff2c0 2024-09-10T11:42:49.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.129+0000 7f932688a700 1 -- 172.21.15.144:0/1835728039 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f932019cbc0 con 0x7f93200ff2c0 2024-09-10T11:42:49.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.129+0000 7f931d7fa700 1 -- 172.21.15.144:0/1835728039 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f931000f5f0 con 0x7f93200ff2c0 2024-09-10T11:42:49.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.129+0000 7f932688a700 1 -- 172.21.15.144:0/1835728039 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f932019d110 con 0x7f93200ff2c0 2024-09-10T11:42:49.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.130+0000 7f932688a700 1 -- 172.21.15.144:0/1835728039 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f932004ea50 con 0x7f93200ff2c0 2024-09-10T11:42:49.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.131+0000 7f931d7fa700 1 -- 172.21.15.144:0/1835728039 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f93100105d0 con 0x7f93200ff2c0 2024-09-10T11:42:49.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.131+0000 7f931d7fa700 1 --2- 172.21.15.144:0/1835728039 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f930806e200 0x7f93080706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:49.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.131+0000 7f931d7fa700 1 -- 172.21.15.144:0/1835728039 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f931008df90 con 0x7f93200ff2c0 2024-09-10T11:42:49.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.131+0000 7f931f7fe700 1 --2- 172.21.15.144:0/1835728039 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f930806e200 0x7f93080706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:49.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.132+0000 7f931f7fe700 1 --2- 172.21.15.144:0/1835728039 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f930806e200 0x7f93080706b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9314006010 tx=0x7f931400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:49.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.136+0000 7f931d7fa700 1 -- 172.21.15.144:0/1835728039 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f93100573f0 con 0x7f93200ff2c0 2024-09-10T11:42:49.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.354+0000 7f932688a700 1 -- 172.21.15.144:0/1835728039 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93200008d0 con 0x7f930806e200 2024-09-10T11:42:49.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.356+0000 7f931d7fa700 1 -- 172.21.15.144:0/1835728039 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f93200008d0 con 0x7f930806e200 2024-09-10T11:42:49.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.358+0000 7f932688a700 1 -- 172.21.15.144:0/1835728039 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f930806e200 msgr2=0x7f93080706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:49.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.358+0000 7f932688a700 1 --2- 172.21.15.144:0/1835728039 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f930806e200 0x7f93080706b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9314006010 tx=0x7f931400b540 comp rx=0 tx=0).stop 2024-09-10T11:42:49.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.358+0000 7f932688a700 1 -- 172.21.15.144:0/1835728039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93200ff2c0 msgr2=0x7f9320197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:49.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.358+0000 7f932688a700 1 --2- 172.21.15.144:0/1835728039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93200ff2c0 0x7f9320197f30 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f931000d8d0 tx=0x7f931000dc90 comp rx=0 tx=0).stop 2024-09-10T11:42:49.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.358+0000 7f932688a700 1 -- 172.21.15.144:0/1835728039 shutdown_connections 2024-09-10T11:42:49.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.358+0000 7f932688a700 1 --2- 172.21.15.144:0/1835728039 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f930806e200 0x7f93080706b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:49.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.358+0000 7f932688a700 1 --2- 172.21.15.144:0/1835728039 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93200ff2c0 0x7f9320197f30 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:49.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.358+0000 7f932688a700 1 --2- 172.21.15.144:0/1835728039 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93200ffc60 0x7f9320198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:49.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.358+0000 7f932688a700 1 -- 172.21.15.144:0/1835728039 >> 172.21.15.144:0/1835728039 conn(0x7f93200fab80 msgr2=0x7f9320102c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:49.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.358+0000 7f932688a700 1 -- 172.21.15.144:0/1835728039 shutdown_connections 2024-09-10T11:42:49.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.358+0000 7f932688a700 1 -- 172.21.15.144:0/1835728039 wait complete. 2024-09-10T11:42:49.369 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:42:49.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.466+0000 7f2490b9e700 1 -- 172.21.15.144:0/1461374063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c101aa0 msgr2=0x7f248c109fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:49.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.466+0000 7f2490b9e700 1 --2- 172.21.15.144:0/1461374063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c101aa0 0x7f248c109fa0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f2474009910 tx=0x7f2474009c20 comp rx=0 tx=0).stop 2024-09-10T11:42:49.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.466+0000 7f2490b9e700 1 -- 172.21.15.144:0/1461374063 shutdown_connections 2024-09-10T11:42:49.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.466+0000 7f2490b9e700 1 --2- 172.21.15.144:0/1461374063 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c101aa0 0x7f248c109fa0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:49.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.466+0000 7f2490b9e700 1 --2- 172.21.15.144:0/1461374063 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f248c101190 0x7f248c101560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:49.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.466+0000 7f2490b9e700 1 -- 172.21.15.144:0/1461374063 >> 172.21.15.144:0/1461374063 conn(0x7f248c0fa9d0 msgr2=0x7f248c0fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.466+0000 7f2490b9e700 1 -- 172.21.15.144:0/1461374063 shutdown_connections 2024-09-10T11:42:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.466+0000 7f2490b9e700 1 -- 172.21.15.144:0/1461374063 wait complete. 2024-09-10T11:42:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.467+0000 7f2490b9e700 1 Processor -- start 2024-09-10T11:42:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.467+0000 7f2490b9e700 1 -- start start 2024-09-10T11:42:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.467+0000 7f2490b9e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f248c101190 0x7f248c19b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.467+0000 7f2490b9e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c101aa0 0x7f248c19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.467+0000 7f2490b9e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c19c4f0 con 0x7f248c101aa0 2024-09-10T11:42:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.467+0000 7f2490b9e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c1959a0 con 0x7f248c101190 2024-09-10T11:42:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.467+0000 7f248affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c101aa0 0x7f248c19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:49.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.467+0000 7f248affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c101aa0 0x7f248c19be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36116/0 (socket says 172.21.15.144:36116) 2024-09-10T11:42:49.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.467+0000 7f248affd700 1 -- 172.21.15.144:0/3025635337 learned_addr learned my addr 172.21.15.144:0/3025635337 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:42:49.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.467+0000 7f248affd700 1 -- 172.21.15.144:0/3025635337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f248c101190 msgr2=0x7f248c19b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:49.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.467+0000 7f248affd700 1 --2- 172.21.15.144:0/3025635337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f248c101190 0x7f248c19b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:49.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.467+0000 7f248affd700 1 -- 172.21.15.144:0/3025635337 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24740095f0 con 0x7f248c101aa0 2024-09-10T11:42:49.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.468+0000 7f248affd700 1 --2- 172.21.15.144:0/3025635337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c101aa0 0x7f248c19be60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2474006010 tx=0x7f247400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:49.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.468+0000 7f2488ff9700 1 -- 172.21.15.144:0/3025635337 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f247401d070 con 0x7f248c101aa0 2024-09-10T11:42:49.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.468+0000 7f2488ff9700 1 -- 172.21.15.144:0/3025635337 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2474004fb0 con 0x7f248c101aa0 2024-09-10T11:42:49.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.468+0000 7f2490b9e700 1 -- 172.21.15.144:0/3025635337 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248c195c20 con 0x7f248c101aa0 2024-09-10T11:42:49.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.468+0000 7f2488ff9700 1 -- 172.21.15.144:0/3025635337 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f247400f690 con 0x7f248c101aa0 2024-09-10T11:42:49.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.468+0000 7f2490b9e700 1 -- 172.21.15.144:0/3025635337 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248c196110 con 0x7f248c101aa0 2024-09-10T11:42:49.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.469+0000 7f2490b9e700 1 -- 172.21.15.144:0/3025635337 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248c107610 con 0x7f248c101aa0 2024-09-10T11:42:49.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.469+0000 7f2488ff9700 1 -- 172.21.15.144:0/3025635337 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f24740228a0 con 0x7f248c101aa0 2024-09-10T11:42:49.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.470+0000 7f2488ff9700 1 --2- 172.21.15.144:0/3025635337 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f247806e200 0x7f24780706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:49.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.470+0000 7f2488ff9700 1 -- 172.21.15.144:0/3025635337 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f247408f8f0 con 0x7f248c101aa0 2024-09-10T11:42:49.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.470+0000 7f248b7fe700 1 --2- 172.21.15.144:0/3025635337 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f247806e200 0x7f24780706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:49.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.470+0000 7f248b7fe700 1 --2- 172.21.15.144:0/3025635337 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f247806e200 0x7f24780706b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f247c00a990 tx=0x7f247c005c80 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:49.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.474+0000 7f2488ff9700 1 -- 172.21.15.144:0/3025635337 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f247405c800 con 0x7f248c101aa0 2024-09-10T11:42:49.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.697+0000 7f2490b9e700 1 -- 172.21.15.144:0/3025635337 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f248c02ccf0 con 0x7f247806e200 2024-09-10T11:42:49.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.698+0000 7f2488ff9700 1 -- 172.21.15.144:0/3025635337 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f248c02ccf0 con 0x7f247806e200 2024-09-10T11:42:49.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.700+0000 7f2490b9e700 1 -- 172.21.15.144:0/3025635337 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f247806e200 msgr2=0x7f24780706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:49.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.700+0000 7f2490b9e700 1 --2- 172.21.15.144:0/3025635337 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f247806e200 0x7f24780706b0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f247c00a990 tx=0x7f247c005c80 comp rx=0 tx=0).stop 2024-09-10T11:42:49.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.700+0000 7f2490b9e700 1 -- 172.21.15.144:0/3025635337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c101aa0 msgr2=0x7f248c19be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:49.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.700+0000 7f2490b9e700 1 --2- 172.21.15.144:0/3025635337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c101aa0 0x7f248c19be60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f2474006010 tx=0x7f247400be20 comp rx=0 tx=0).stop 2024-09-10T11:42:49.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.700+0000 7f2490b9e700 1 -- 172.21.15.144:0/3025635337 shutdown_connections 2024-09-10T11:42:49.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.700+0000 7f2490b9e700 1 --2- 172.21.15.144:0/3025635337 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f247806e200 0x7f24780706b0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:49.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.700+0000 7f2490b9e700 1 --2- 172.21.15.144:0/3025635337 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f248c101190 0x7f248c19b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:49.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.700+0000 7f2490b9e700 1 --2- 172.21.15.144:0/3025635337 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f248c101aa0 0x7f248c19be60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:49.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.700+0000 7f2490b9e700 1 -- 172.21.15.144:0/3025635337 >> 172.21.15.144:0/3025635337 conn(0x7f248c0fa9d0 msgr2=0x7f248c104750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:49.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.700+0000 7f2490b9e700 1 -- 172.21.15.144:0/3025635337 shutdown_connections 2024-09-10T11:42:49.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.700+0000 7f2490b9e700 1 -- 172.21.15.144:0/3025635337 wait complete. 2024-09-10T11:42:49.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.806+0000 7fdb1ccd7700 1 -- 172.21.15.144:0/2855296025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18101cd0 msgr2=0x7fdb1810a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:49.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.806+0000 7fdb1ccd7700 1 --2- 172.21.15.144:0/2855296025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18101cd0 0x7fdb1810a1d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fdb00009910 tx=0x7fdb00009c20 comp rx=0 tx=0).stop 2024-09-10T11:42:49.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.806+0000 7fdb1ccd7700 1 -- 172.21.15.144:0/2855296025 shutdown_connections 2024-09-10T11:42:49.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.806+0000 7fdb1ccd7700 1 --2- 172.21.15.144:0/2855296025 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18101cd0 0x7fdb1810a1d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:49.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.806+0000 7fdb1ccd7700 1 --2- 172.21.15.144:0/2855296025 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb181013c0 0x7fdb18101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:49.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.806+0000 7fdb1ccd7700 1 -- 172.21.15.144:0/2855296025 >> 172.21.15.144:0/2855296025 conn(0x7fdb180fab80 msgr2=0x7fdb180fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:49.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.806+0000 7fdb1ccd7700 1 -- 172.21.15.144:0/2855296025 shutdown_connections 2024-09-10T11:42:49.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.806+0000 7fdb1ccd7700 1 -- 172.21.15.144:0/2855296025 wait complete. 2024-09-10T11:42:49.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.807+0000 7fdb1ccd7700 1 Processor -- start 2024-09-10T11:42:49.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.807+0000 7fdb1ccd7700 1 -- start start 2024-09-10T11:42:49.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.807+0000 7fdb1ccd7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb181013c0 0x7fdb18195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:49.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.807+0000 7fdb1ccd7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18101cd0 0x7fdb181963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:49.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.807+0000 7fdb1ccd7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb18196aa0 con 0x7fdb18101cd0 2024-09-10T11:42:49.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.807+0000 7fdb1ccd7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb1819a830 con 0x7fdb181013c0 2024-09-10T11:42:49.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.807+0000 7fdb15d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18101cd0 0x7fdb181963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:49.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.807+0000 7fdb15d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18101cd0 0x7fdb181963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36134/0 (socket says 172.21.15.144:36134) 2024-09-10T11:42:49.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.807+0000 7fdb15d9b700 1 -- 172.21.15.144:0/2907366750 learned_addr learned my addr 172.21.15.144:0/2907366750 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:42:49.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.807+0000 7fdb1659c700 1 --2- 172.21.15.144:0/2907366750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb181013c0 0x7fdb18195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:49.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.807+0000 7fdb15d9b700 1 -- 172.21.15.144:0/2907366750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb181013c0 msgr2=0x7fdb18195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:49.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.807+0000 7fdb15d9b700 1 --2- 172.21.15.144:0/2907366750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb181013c0 0x7fdb18195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:49.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.807+0000 7fdb15d9b700 1 -- 172.21.15.144:0/2907366750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb000095f0 con 0x7fdb18101cd0 2024-09-10T11:42:49.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.808+0000 7fdb15d9b700 1 --2- 172.21.15.144:0/2907366750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18101cd0 0x7fdb181963c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdb00006010 tx=0x7fdb0000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:49.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.808+0000 7fdb0f7fe700 1 -- 172.21.15.144:0/2907366750 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb0001d070 con 0x7fdb18101cd0 2024-09-10T11:42:49.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.808+0000 7fdb0f7fe700 1 -- 172.21.15.144:0/2907366750 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb00004fb0 con 0x7fdb18101cd0 2024-09-10T11:42:49.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.808+0000 7fdb1ccd7700 1 -- 172.21.15.144:0/2907366750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb1819aab0 con 0x7fdb18101cd0 2024-09-10T11:42:49.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.808+0000 7fdb0f7fe700 1 -- 172.21.15.144:0/2907366750 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb00022ea0 con 0x7fdb18101cd0 2024-09-10T11:42:49.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.808+0000 7fdb1ccd7700 1 -- 172.21.15.144:0/2907366750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb1819afa0 con 0x7fdb18101cd0 2024-09-10T11:42:49.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.809+0000 7fdb1ccd7700 1 -- 172.21.15.144:0/2907366750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb18107840 con 0x7fdb18101cd0 2024-09-10T11:42:49.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.810+0000 7fdb0f7fe700 1 -- 172.21.15.144:0/2907366750 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fdb00022470 con 0x7fdb18101cd0 2024-09-10T11:42:49.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.810+0000 7fdb0f7fe700 1 --2- 172.21.15.144:0/2907366750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdb0406e200 0x7fdb040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:49.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.810+0000 7fdb0f7fe700 1 -- 172.21.15.144:0/2907366750 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdb0008f610 con 0x7fdb18101cd0 2024-09-10T11:42:49.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.810+0000 7fdb1659c700 1 --2- 172.21.15.144:0/2907366750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdb0406e200 0x7fdb040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:49.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.811+0000 7fdb1659c700 1 --2- 172.21.15.144:0/2907366750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdb0406e200 0x7fdb040706b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdb10007dc0 tx=0x7fdb100072f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:49.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:49.815+0000 7fdb0f7fe700 1 -- 172.21.15.144:0/2907366750 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdb00058aa0 con 0x7fdb18101cd0 2024-09-10T11:42:50.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.025+0000 7fdb1ccd7700 1 -- 172.21.15.144:0/2907366750 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb1819b2f0 con 0x7fdb0406e200 2024-09-10T11:42:50.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.034+0000 7fdb0f7fe700 1 -- 172.21.15.144:0/2907366750 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fdb1819b2f0 con 0x7fdb0406e200 2024-09-10T11:42:50.035 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:42:50.036 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (11m) 4m ago 21m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:42:50.036 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (21m) 4m ago 21m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:42:50.036 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (20m) 4m ago 20m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:42:50.036 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (8m) 4m ago 21m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:42:50.036 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (8m) 4m ago 20m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:42:50.036 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (10m) 4m ago 21m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:42:50.036 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (5m) 4m ago 16m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:42:50.036 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (4m) 4m ago 16m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:42:50.036 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (16m) 4m ago 16m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:42:50.036 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (4m) 4m ago 16m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:42:50.036 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (12m) 4m ago 23m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:42:50.036 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (11m) 4m ago 20m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:42:50.036 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (9m) 4m ago 23m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:42:50.037 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (9m) 4m ago 20m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:42:50.037 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (11m) 4m ago 21m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:42:50.037 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (11m) 4m ago 20m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:42:50.037 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (8m) 4m ago 19m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:42:50.037 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (7m) 4m ago 19m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:42:50.037 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (7m) 4m ago 18m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:42:50.037 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (6m) 4m ago 18m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:42:50.037 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (5m) 4m ago 18m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:42:50.037 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (5m) 4m ago 17m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:42:50.037 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (11m) 4m ago 21m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:42:50.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.036+0000 7fdb1ccd7700 1 -- 172.21.15.144:0/2907366750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdb0406e200 msgr2=0x7fdb040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:50.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.036+0000 7fdb1ccd7700 1 --2- 172.21.15.144:0/2907366750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdb0406e200 0x7fdb040706b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdb10007dc0 tx=0x7fdb100072f0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.036+0000 7fdb1ccd7700 1 -- 172.21.15.144:0/2907366750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18101cd0 msgr2=0x7fdb181963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:50.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.036+0000 7fdb1ccd7700 1 --2- 172.21.15.144:0/2907366750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18101cd0 0x7fdb181963c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fdb00006010 tx=0x7fdb0000be20 comp rx=0 tx=0).stop 2024-09-10T11:42:50.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.036+0000 7fdb1ccd7700 1 -- 172.21.15.144:0/2907366750 shutdown_connections 2024-09-10T11:42:50.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.036+0000 7fdb1ccd7700 1 --2- 172.21.15.144:0/2907366750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdb0406e200 0x7fdb040706b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.036+0000 7fdb1ccd7700 1 --2- 172.21.15.144:0/2907366750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdb181013c0 0x7fdb18195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.036+0000 7fdb1ccd7700 1 --2- 172.21.15.144:0/2907366750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdb18101cd0 0x7fdb181963c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.036+0000 7fdb1ccd7700 1 -- 172.21.15.144:0/2907366750 >> 172.21.15.144:0/2907366750 conn(0x7fdb180fab80 msgr2=0x7fdb18104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:50.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.036+0000 7fdb1ccd7700 1 -- 172.21.15.144:0/2907366750 shutdown_connections 2024-09-10T11:42:50.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.037+0000 7fdb1ccd7700 1 -- 172.21.15.144:0/2907366750 wait complete. 2024-09-10T11:42:50.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.139+0000 7fa877eee700 1 -- 172.21.15.144:0/1210246938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa870100d80 msgr2=0x7fa870104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:50.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.139+0000 7fa877eee700 1 --2- 172.21.15.144:0/1210246938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa870100d80 0x7fa870104d50 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fa860009910 tx=0x7fa860009c20 comp rx=0 tx=0).stop 2024-09-10T11:42:50.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.140+0000 7fa877eee700 1 -- 172.21.15.144:0/1210246938 shutdown_connections 2024-09-10T11:42:50.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.140+0000 7fa877eee700 1 --2- 172.21.15.144:0/1210246938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa870100d80 0x7fa870104d50 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.140+0000 7fa877eee700 1 --2- 172.21.15.144:0/1210246938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8701003e0 0x7fa8701007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.140+0000 7fa877eee700 1 -- 172.21.15.144:0/1210246938 >> 172.21.15.144:0/1210246938 conn(0x7fa8700fbca0 msgr2=0x7fa8700fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:50.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.140+0000 7fa877eee700 1 -- 172.21.15.144:0/1210246938 shutdown_connections 2024-09-10T11:42:50.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.140+0000 7fa877eee700 1 -- 172.21.15.144:0/1210246938 wait complete. 2024-09-10T11:42:50.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.140+0000 7fa877eee700 1 Processor -- start 2024-09-10T11:42:50.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.141+0000 7fa877eee700 1 -- start start 2024-09-10T11:42:50.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.141+0000 7fa877eee700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8701003e0 0x7fa870195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:50.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.141+0000 7fa877eee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa870100d80 0x7fa870196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:50.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.141+0000 7fa877eee700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8701969f0 con 0x7fa870100d80 2024-09-10T11:42:50.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.141+0000 7fa877eee700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87019a780 con 0x7fa8701003e0 2024-09-10T11:42:50.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.141+0000 7fa875489700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa870100d80 0x7fa870196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:50.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.141+0000 7fa875489700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa870100d80 0x7fa870196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36150/0 (socket says 172.21.15.144:36150) 2024-09-10T11:42:50.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.141+0000 7fa875489700 1 -- 172.21.15.144:0/1599909234 learned_addr learned my addr 172.21.15.144:0/1599909234 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:42:50.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.141+0000 7fa875c8a700 1 --2- 172.21.15.144:0/1599909234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8701003e0 0x7fa870195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:50.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.141+0000 7fa875489700 1 -- 172.21.15.144:0/1599909234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8701003e0 msgr2=0x7fa870195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:50.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.141+0000 7fa875489700 1 --2- 172.21.15.144:0/1599909234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8701003e0 0x7fa870195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.141+0000 7fa875489700 1 -- 172.21.15.144:0/1599909234 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8600095f0 con 0x7fa870100d80 2024-09-10T11:42:50.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.142+0000 7fa875489700 1 --2- 172.21.15.144:0/1599909234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa870100d80 0x7fa870196310 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa8600092e0 tx=0x7fa86000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:50.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.142+0000 7fa866ffd700 1 -- 172.21.15.144:0/1599909234 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa86001d070 con 0x7fa870100d80 2024-09-10T11:42:50.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.142+0000 7fa866ffd700 1 -- 172.21.15.144:0/1599909234 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa860005dc0 con 0x7fa870100d80 2024-09-10T11:42:50.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.142+0000 7fa877eee700 1 -- 172.21.15.144:0/1599909234 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa87019aa00 con 0x7fa870100d80 2024-09-10T11:42:50.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.142+0000 7fa866ffd700 1 -- 172.21.15.144:0/1599909234 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa860017400 con 0x7fa870100d80 2024-09-10T11:42:50.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.142+0000 7fa877eee700 1 -- 172.21.15.144:0/1599909234 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa87019ae30 con 0x7fa870100d80 2024-09-10T11:42:50.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.143+0000 7fa877eee700 1 -- 172.21.15.144:0/1599909234 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87004ea50 con 0x7fa870100d80 2024-09-10T11:42:50.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.143+0000 7fa866ffd700 1 -- 172.21.15.144:0/1599909234 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa86000f6f0 con 0x7fa870100d80 2024-09-10T11:42:50.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.144+0000 7fa866ffd700 1 --2- 172.21.15.144:0/1599909234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa85c06e200 0x7fa85c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:50.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.144+0000 7fa866ffd700 1 -- 172.21.15.144:0/1599909234 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa86008f3a0 con 0x7fa870100d80 2024-09-10T11:42:50.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.144+0000 7fa875c8a700 1 --2- 172.21.15.144:0/1599909234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa85c06e200 0x7fa85c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:50.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.144+0000 7fa875c8a700 1 --2- 172.21.15.144:0/1599909234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa85c06e200 0x7fa85c0706b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa86c005950 tx=0x7fa86c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:50.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.148+0000 7fa866ffd700 1 -- 172.21.15.144:0/1599909234 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa860058800 con 0x7fa870100d80 2024-09-10T11:42:50.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.431+0000 7fa877eee700 1 -- 172.21.15.144:0/1599909234 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa87019b280 con 0x7fa870100d80 2024-09-10T11:42:50.433 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.432+0000 7fa866ffd700 1 -- 172.21.15.144:0/1599909234 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fa86005bec0 con 0x7fa870100d80 2024-09-10T11:42:50.433 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:42:50.433 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:42:50.433 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:42:50.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:42:50.434 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:42:50.434 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:42:50.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:42:50.434 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:42:50.434 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:42:50.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:42:50.434 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:42:50.434 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:42:50.434 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:42:50.434 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:42:50.434 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:42:50.434 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:42:50.435 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:42:50.435 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:42:50.435 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:42:50.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.435+0000 7fa877eee700 1 -- 172.21.15.144:0/1599909234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa85c06e200 msgr2=0x7fa85c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:50.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.435+0000 7fa877eee700 1 --2- 172.21.15.144:0/1599909234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa85c06e200 0x7fa85c0706b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa86c005950 tx=0x7fa86c00b410 comp rx=0 tx=0).stop 2024-09-10T11:42:50.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.435+0000 7fa877eee700 1 -- 172.21.15.144:0/1599909234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa870100d80 msgr2=0x7fa870196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:50.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.435+0000 7fa877eee700 1 --2- 172.21.15.144:0/1599909234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa870100d80 0x7fa870196310 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa8600092e0 tx=0x7fa86000bd70 comp rx=0 tx=0).stop 2024-09-10T11:42:50.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.435+0000 7fa877eee700 1 -- 172.21.15.144:0/1599909234 shutdown_connections 2024-09-10T11:42:50.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.435+0000 7fa877eee700 1 --2- 172.21.15.144:0/1599909234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa85c06e200 0x7fa85c0706b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.435+0000 7fa877eee700 1 --2- 172.21.15.144:0/1599909234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8701003e0 0x7fa870195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.435+0000 7fa877eee700 1 --2- 172.21.15.144:0/1599909234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa870100d80 0x7fa870196310 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.435+0000 7fa877eee700 1 -- 172.21.15.144:0/1599909234 >> 172.21.15.144:0/1599909234 conn(0x7fa8700fbca0 msgr2=0x7fa8700fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:50.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.435+0000 7fa877eee700 1 -- 172.21.15.144:0/1599909234 shutdown_connections 2024-09-10T11:42:50.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.435+0000 7fa877eee700 1 -- 172.21.15.144:0/1599909234 wait complete. 2024-09-10T11:42:50.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.539+0000 7f6c1bae7700 1 -- 172.21.15.144:0/2280649600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c140ff310 msgr2=0x7f6c140ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:50.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.539+0000 7f6c1bae7700 1 --2- 172.21.15.144:0/2280649600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c140ff310 0x7f6c140ff6e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6c10009910 tx=0x7f6c10009c20 comp rx=0 tx=0).stop 2024-09-10T11:42:50.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.539+0000 7f6c1bae7700 1 -- 172.21.15.144:0/2280649600 shutdown_connections 2024-09-10T11:42:50.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.539+0000 7f6c1bae7700 1 --2- 172.21.15.144:0/2280649600 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c140ffcb0 0x7f6c14103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.540+0000 7f6c1bae7700 1 --2- 172.21.15.144:0/2280649600 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c140ff310 0x7f6c140ff6e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.540+0000 7f6c1bae7700 1 -- 172.21.15.144:0/2280649600 >> 172.21.15.144:0/2280649600 conn(0x7f6c140fabb0 msgr2=0x7f6c140fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:50.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.540+0000 7f6c1bae7700 1 -- 172.21.15.144:0/2280649600 shutdown_connections 2024-09-10T11:42:50.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.540+0000 7f6c1bae7700 1 -- 172.21.15.144:0/2280649600 wait complete. 2024-09-10T11:42:50.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.540+0000 7f6c1bae7700 1 Processor -- start 2024-09-10T11:42:50.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.540+0000 7f6c1bae7700 1 -- start start 2024-09-10T11:42:50.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.540+0000 7f6c1bae7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c140ff310 0x7f6c14195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:50.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.540+0000 7f6c1bae7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c140ffcb0 0x7f6c141962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:50.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.540+0000 7f6c1bae7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c141969a0 con 0x7f6c140ffcb0 2024-09-10T11:42:50.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.540+0000 7f6c1bae7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c1419a730 con 0x7f6c140ff310 2024-09-10T11:42:50.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.541+0000 7f6c19082700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c140ffcb0 0x7f6c141962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:50.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.541+0000 7f6c19082700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c140ffcb0 0x7f6c141962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36172/0 (socket says 172.21.15.144:36172) 2024-09-10T11:42:50.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.541+0000 7f6c19082700 1 -- 172.21.15.144:0/3651936269 learned_addr learned my addr 172.21.15.144:0/3651936269 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:42:50.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.541+0000 7f6c19883700 1 --2- 172.21.15.144:0/3651936269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c140ff310 0x7f6c14195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:50.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.541+0000 7f6c19082700 1 -- 172.21.15.144:0/3651936269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c140ff310 msgr2=0x7f6c14195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:50.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.541+0000 7f6c19082700 1 --2- 172.21.15.144:0/3651936269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c140ff310 0x7f6c14195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.541+0000 7f6c19082700 1 -- 172.21.15.144:0/3651936269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c100095f0 con 0x7f6c140ffcb0 2024-09-10T11:42:50.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.541+0000 7f6c19082700 1 --2- 172.21.15.144:0/3651936269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c140ffcb0 0x7f6c141962c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400d8d0 tx=0x7f6c0400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:50.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.541+0000 7f6c0affd700 1 -- 172.21.15.144:0/3651936269 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c040041f0 con 0x7f6c140ffcb0 2024-09-10T11:42:50.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.541+0000 7f6c1bae7700 1 -- 172.21.15.144:0/3651936269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c1419aa10 con 0x7f6c140ffcb0 2024-09-10T11:42:50.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.541+0000 7f6c0affd700 1 -- 172.21.15.144:0/3651936269 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c04007620 con 0x7f6c140ffcb0 2024-09-10T11:42:50.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.541+0000 7f6c0affd700 1 -- 172.21.15.144:0/3651936269 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c04003c50 con 0x7f6c140ffcb0 2024-09-10T11:42:50.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.541+0000 7f6c1bae7700 1 -- 172.21.15.144:0/3651936269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c1419af60 con 0x7f6c140ffcb0 2024-09-10T11:42:50.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.542+0000 7f6c1bae7700 1 -- 172.21.15.144:0/3651936269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c14108630 con 0x7f6c140ffcb0 2024-09-10T11:42:50.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.543+0000 7f6c0affd700 1 -- 172.21.15.144:0/3651936269 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6c04004350 con 0x7f6c140ffcb0 2024-09-10T11:42:50.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.543+0000 7f6c0affd700 1 --2- 172.21.15.144:0/3651936269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6c0006e200 0x7f6c000706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:50.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.543+0000 7f6c0affd700 1 -- 172.21.15.144:0/3651936269 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6c0408dc30 con 0x7f6c140ffcb0 2024-09-10T11:42:50.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.543+0000 7f6c19883700 1 --2- 172.21.15.144:0/3651936269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6c0006e200 0x7f6c000706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:50.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.544+0000 7f6c19883700 1 --2- 172.21.15.144:0/3651936269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6c0006e200 0x7f6c000706b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6c10006010 tx=0x7f6c1001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:50.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.547+0000 7f6c0affd700 1 -- 172.21.15.144:0/3651936269 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6c04057010 con 0x7f6c140ffcb0 2024-09-10T11:42:50.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.795+0000 7f6c1bae7700 1 -- 172.21.15.144:0/3651936269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6c1404ea50 con 0x7f6c140ffcb0 2024-09-10T11:42:50.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.795+0000 7f6c0affd700 1 -- 172.21.15.144:0/3651936269 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f6c0401f790 con 0x7f6c140ffcb0 2024-09-10T11:42:50.797 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:42:50.797 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:42:50.797 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:42:50.797 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:42:50.797 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:42:50.797 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:42:50.797 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:42:50.797 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:42:50.797 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:42:50.797 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:42:50.797 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:42:50.798 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:42:50.798 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:42:50.798 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:42:50.798 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:42:50.798 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:42:50.798 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:42:50.798 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:42:50.798 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:42:50.798 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:42:50.798 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:42:50.798 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:42:50.798 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:42:50.798 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:42:50.798 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:42:50.799 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:42:50.799 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:42:50.799 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:42:50.799 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:42:50.799 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:42:50.799 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:42:50.799 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:42:50.799 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:42:50.799 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:42:50.799 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:42:50.799 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:42:50.799 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:42:50.799 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:42:50.799 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:42:50.800 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:42:50.800 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:42:50.800 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:42:50.800 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:42:50.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.797+0000 7f6c1bae7700 1 -- 172.21.15.144:0/3651936269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6c0006e200 msgr2=0x7f6c000706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:50.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.797+0000 7f6c1bae7700 1 --2- 172.21.15.144:0/3651936269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6c0006e200 0x7f6c000706b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6c10006010 tx=0x7f6c1001a040 comp rx=0 tx=0).stop 2024-09-10T11:42:50.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.797+0000 7f6c1bae7700 1 -- 172.21.15.144:0/3651936269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c140ffcb0 msgr2=0x7f6c141962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:50.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.798+0000 7f6c1bae7700 1 --2- 172.21.15.144:0/3651936269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c140ffcb0 0x7f6c141962c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f6c0400d8d0 tx=0x7f6c0400dc90 comp rx=0 tx=0).stop 2024-09-10T11:42:50.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.798+0000 7f6c1bae7700 1 -- 172.21.15.144:0/3651936269 shutdown_connections 2024-09-10T11:42:50.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.798+0000 7f6c1bae7700 1 --2- 172.21.15.144:0/3651936269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6c0006e200 0x7f6c000706b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.798+0000 7f6c1bae7700 1 --2- 172.21.15.144:0/3651936269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c140ff310 0x7f6c14195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.798+0000 7f6c1bae7700 1 --2- 172.21.15.144:0/3651936269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c140ffcb0 0x7f6c141962c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.798+0000 7f6c1bae7700 1 -- 172.21.15.144:0/3651936269 >> 172.21.15.144:0/3651936269 conn(0x7f6c140fabb0 msgr2=0x7f6c14069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:50.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.798+0000 7f6c1bae7700 1 -- 172.21.15.144:0/3651936269 shutdown_connections 2024-09-10T11:42:50.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.798+0000 7f6c1bae7700 1 -- 172.21.15.144:0/3651936269 wait complete. 2024-09-10T11:42:50.801 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:42:50.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.903+0000 7f759d9bb700 1 -- 172.21.15.144:0/1290983088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7598100d80 msgr2=0x7f7598104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:50.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.903+0000 7f759d9bb700 1 --2- 172.21.15.144:0/1290983088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7598100d80 0x7f7598104d50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f7580009910 tx=0x7f7580009c20 comp rx=0 tx=0).stop 2024-09-10T11:42:50.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.903+0000 7f759d9bb700 1 -- 172.21.15.144:0/1290983088 shutdown_connections 2024-09-10T11:42:50.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.903+0000 7f759d9bb700 1 --2- 172.21.15.144:0/1290983088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7598100d80 0x7f7598104d50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.903+0000 7f759d9bb700 1 --2- 172.21.15.144:0/1290983088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75981003e0 0x7f75981007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.903+0000 7f759d9bb700 1 -- 172.21.15.144:0/1290983088 >> 172.21.15.144:0/1290983088 conn(0x7f75980fbca0 msgr2=0x7f75980fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:50.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.903+0000 7f759d9bb700 1 -- 172.21.15.144:0/1290983088 shutdown_connections 2024-09-10T11:42:50.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.903+0000 7f759d9bb700 1 -- 172.21.15.144:0/1290983088 wait complete. 2024-09-10T11:42:50.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.904+0000 7f759d9bb700 1 Processor -- start 2024-09-10T11:42:50.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.904+0000 7f759d9bb700 1 -- start start 2024-09-10T11:42:50.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.904+0000 7f759d9bb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75981003e0 0x7f7598195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:50.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.904+0000 7f759d9bb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7598100d80 0x7f7598196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:50.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.904+0000 7f759d9bb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7598196970 con 0x7f7598100d80 2024-09-10T11:42:50.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.904+0000 7f759d9bb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759819a700 con 0x7f75981003e0 2024-09-10T11:42:50.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.904+0000 7f758ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7598100d80 0x7f7598196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:50.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.904+0000 7f758ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7598100d80 0x7f7598196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36190/0 (socket says 172.21.15.144:36190) 2024-09-10T11:42:50.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.904+0000 7f758ffff700 1 -- 172.21.15.144:0/3533401497 learned_addr learned my addr 172.21.15.144:0/3533401497 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:42:50.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.904+0000 7f7596ffd700 1 --2- 172.21.15.144:0/3533401497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75981003e0 0x7f7598195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:50.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.904+0000 7f758ffff700 1 -- 172.21.15.144:0/3533401497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75981003e0 msgr2=0x7f7598195d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:50.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.904+0000 7f758ffff700 1 --2- 172.21.15.144:0/3533401497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75981003e0 0x7f7598195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:50.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.904+0000 7f758ffff700 1 -- 172.21.15.144:0/3533401497 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75800095f0 con 0x7f7598100d80 2024-09-10T11:42:50.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.905+0000 7f758ffff700 1 --2- 172.21.15.144:0/3533401497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7598100d80 0x7f7598196290 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7580005f50 tx=0x7f758000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:50.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.905+0000 7f759c9b9700 1 -- 172.21.15.144:0/3533401497 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f758001d070 con 0x7f7598100d80 2024-09-10T11:42:50.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.905+0000 7f759c9b9700 1 -- 172.21.15.144:0/3533401497 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f758000f460 con 0x7f7598100d80 2024-09-10T11:42:50.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.905+0000 7f759c9b9700 1 -- 172.21.15.144:0/3533401497 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7580017630 con 0x7f7598100d80 2024-09-10T11:42:50.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.905+0000 7f759d9bb700 1 -- 172.21.15.144:0/3533401497 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759819a980 con 0x7f7598100d80 2024-09-10T11:42:50.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.905+0000 7f759d9bb700 1 -- 172.21.15.144:0/3533401497 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759819ae70 con 0x7f7598100d80 2024-09-10T11:42:50.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.906+0000 7f759d9bb700 1 -- 172.21.15.144:0/3533401497 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7598108750 con 0x7f7598100d80 2024-09-10T11:42:50.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.906+0000 7f759c9b9700 1 -- 172.21.15.144:0/3533401497 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f758000f5d0 con 0x7f7598100d80 2024-09-10T11:42:50.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.907+0000 7f759c9b9700 1 --2- 172.21.15.144:0/3533401497 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f758406e200 0x7f75840706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:50.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.907+0000 7f759c9b9700 1 -- 172.21.15.144:0/3533401497 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f758008f5f0 con 0x7f7598100d80 2024-09-10T11:42:50.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.907+0000 7f7596ffd700 1 --2- 172.21.15.144:0/3533401497 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f758406e200 0x7f75840706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:50.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.907+0000 7f7596ffd700 1 --2- 172.21.15.144:0/3533401497 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f758406e200 0x7f75840706b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7588005950 tx=0x7f7588005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:50.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:50.911+0000 7f759c9b9700 1 -- 172.21.15.144:0/3533401497 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7580058a50 con 0x7f7598100d80 2024-09-10T11:42:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:50 smithi178 ceph-mon[65940]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:50 smithi178 ceph-mon[65940]: pgmap v325: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:50 smithi178 ceph-mon[65940]: from='client.34656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:50 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1599909234' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:42:51.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.133+0000 7f759d9bb700 1 -- 172.21.15.144:0/3533401497 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f759802cd20 con 0x7f758406e200 2024-09-10T11:42:51.135 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:50 smithi144 ceph-mon[87436]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:51.135 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:50 smithi144 ceph-mon[87436]: pgmap v325: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:51.135 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:50 smithi144 ceph-mon[87436]: from='client.34656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:51.135 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:50 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1599909234' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:42:51.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.135+0000 7f759c9b9700 1 -- 172.21.15.144:0/3533401497 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f759802cd20 con 0x7f758406e200 2024-09-10T11:42:51.136 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:42:51.136 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:42:51.136 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:42:51.136 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:42:51.136 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:42:51.136 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:42:51.136 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:42:51.136 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:42:51.137 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:42:51.137 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:42:51.137 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:42:51.137 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:42:51.137 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:42:51.137 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:42:51.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.137+0000 7f759d9bb700 1 -- 172.21.15.144:0/3533401497 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f758406e200 msgr2=0x7f75840706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:51.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.137+0000 7f759d9bb700 1 --2- 172.21.15.144:0/3533401497 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f758406e200 0x7f75840706b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f7588005950 tx=0x7f7588005cf0 comp rx=0 tx=0).stop 2024-09-10T11:42:51.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.137+0000 7f759d9bb700 1 -- 172.21.15.144:0/3533401497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7598100d80 msgr2=0x7f7598196290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:51.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.137+0000 7f759d9bb700 1 --2- 172.21.15.144:0/3533401497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7598100d80 0x7f7598196290 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7580005f50 tx=0x7f758000be20 comp rx=0 tx=0).stop 2024-09-10T11:42:51.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.137+0000 7f759d9bb700 1 -- 172.21.15.144:0/3533401497 shutdown_connections 2024-09-10T11:42:51.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.137+0000 7f759d9bb700 1 --2- 172.21.15.144:0/3533401497 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f758406e200 0x7f75840706b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:51.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.137+0000 7f759d9bb700 1 --2- 172.21.15.144:0/3533401497 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75981003e0 0x7f7598195d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:51.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.137+0000 7f759d9bb700 1 --2- 172.21.15.144:0/3533401497 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7598100d80 0x7f7598196290 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:51.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.137+0000 7f759d9bb700 1 -- 172.21.15.144:0/3533401497 >> 172.21.15.144:0/3533401497 conn(0x7f75980fbca0 msgr2=0x7f75980fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:51.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.137+0000 7f759d9bb700 1 -- 172.21.15.144:0/3533401497 shutdown_connections 2024-09-10T11:42:51.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.137+0000 7f759d9bb700 1 -- 172.21.15.144:0/3533401497 wait complete. 2024-09-10T11:42:51.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.242+0000 7f5a60efa700 1 -- 172.21.15.144:0/640232750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a5c0ffc80 msgr2=0x7f5a5c103c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:51.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.242+0000 7f5a60efa700 1 --2- 172.21.15.144:0/640232750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a5c0ffc80 0x7f5a5c103c50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5a50009910 tx=0x7f5a50009c20 comp rx=0 tx=0).stop 2024-09-10T11:42:51.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.243+0000 7f5a60efa700 1 -- 172.21.15.144:0/640232750 shutdown_connections 2024-09-10T11:42:51.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.243+0000 7f5a60efa700 1 --2- 172.21.15.144:0/640232750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a5c0ffc80 0x7f5a5c103c50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:51.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.243+0000 7f5a60efa700 1 --2- 172.21.15.144:0/640232750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a5c0ff2e0 0x7f5a5c0ff6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:51.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.243+0000 7f5a60efa700 1 -- 172.21.15.144:0/640232750 >> 172.21.15.144:0/640232750 conn(0x7f5a5c0fab80 msgr2=0x7f5a5c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:51.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.243+0000 7f5a60efa700 1 -- 172.21.15.144:0/640232750 shutdown_connections 2024-09-10T11:42:51.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.243+0000 7f5a60efa700 1 -- 172.21.15.144:0/640232750 wait complete. 2024-09-10T11:42:51.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.243+0000 7f5a60efa700 1 Processor -- start 2024-09-10T11:42:51.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.244+0000 7f5a60efa700 1 -- start start 2024-09-10T11:42:51.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.244+0000 7f5a60efa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a5c0ff2e0 0x7f5a5c196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:51.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.244+0000 7f5a60efa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a5c0ffc80 0x7f5a5c197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:51.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.244+0000 7f5a60efa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c197720 con 0x7f5a5c0ffc80 2024-09-10T11:42:51.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.244+0000 7f5a60efa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a5c197890 con 0x7f5a5c0ff2e0 2024-09-10T11:42:51.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.244+0000 7f5a59d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a5c0ffc80 0x7f5a5c197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:51.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.244+0000 7f5a59d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a5c0ffc80 0x7f5a5c197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36212/0 (socket says 172.21.15.144:36212) 2024-09-10T11:42:51.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.244+0000 7f5a59d9b700 1 -- 172.21.15.144:0/3410111351 learned_addr learned my addr 172.21.15.144:0/3410111351 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:42:51.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.244+0000 7f5a5a59c700 1 --2- 172.21.15.144:0/3410111351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a5c0ff2e0 0x7f5a5c196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:51.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.244+0000 7f5a59d9b700 1 -- 172.21.15.144:0/3410111351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a5c0ff2e0 msgr2=0x7f5a5c196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:51.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.244+0000 7f5a59d9b700 1 --2- 172.21.15.144:0/3410111351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a5c0ff2e0 0x7f5a5c196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:51.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.244+0000 7f5a59d9b700 1 -- 172.21.15.144:0/3410111351 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a500095f0 con 0x7f5a5c0ffc80 2024-09-10T11:42:51.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.244+0000 7f5a59d9b700 1 --2- 172.21.15.144:0/3410111351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a5c0ffc80 0x7f5a5c197100 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5a500092e0 tx=0x7f5a5000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:51.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.244+0000 7f5a4b7fe700 1 -- 172.21.15.144:0/3410111351 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a5001d070 con 0x7f5a5c0ffc80 2024-09-10T11:42:51.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.245+0000 7f5a4b7fe700 1 -- 172.21.15.144:0/3410111351 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a50005dc0 con 0x7f5a5c0ffc80 2024-09-10T11:42:51.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.245+0000 7f5a60efa700 1 -- 172.21.15.144:0/3410111351 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a5c19fc30 con 0x7f5a5c0ffc80 2024-09-10T11:42:51.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.245+0000 7f5a4b7fe700 1 -- 172.21.15.144:0/3410111351 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a50017400 con 0x7f5a5c0ffc80 2024-09-10T11:42:51.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.245+0000 7f5a60efa700 1 -- 172.21.15.144:0/3410111351 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a5c1a00a0 con 0x7f5a5c0ffc80 2024-09-10T11:42:51.248 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.246+0000 7f5a60efa700 1 -- 172.21.15.144:0/3410111351 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a5c107560 con 0x7f5a5c0ffc80 2024-09-10T11:42:51.249 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.246+0000 7f5a4b7fe700 1 -- 172.21.15.144:0/3410111351 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5a5000f6f0 con 0x7f5a5c0ffc80 2024-09-10T11:42:51.249 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.247+0000 7f5a4b7fe700 1 --2- 172.21.15.144:0/3410111351 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a4406e200 0x7f5a440706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:42:51.249 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.247+0000 7f5a5a59c700 1 --2- 172.21.15.144:0/3410111351 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a4406e200 0x7f5a440706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:42:51.249 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.247+0000 7f5a4b7fe700 1 -- 172.21.15.144:0/3410111351 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5a5008f3a0 con 0x7f5a5c0ffc80 2024-09-10T11:42:51.249 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.247+0000 7f5a5a59c700 1 --2- 172.21.15.144:0/3410111351 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a4406e200 0x7f5a440706b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c004630 tx=0x7f5a4c00b3c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:42:51.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.251+0000 7f5a4b7fe700 1 -- 172.21.15.144:0/3410111351 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5a50058800 con 0x7f5a5c0ffc80 2024-09-10T11:42:51.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.534+0000 7f5a60efa700 1 -- 172.21.15.144:0/3410111351 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5a5c04ea50 con 0x7f5a5c0ffc80 2024-09-10T11:42:51.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.535+0000 7f5a4b7fe700 1 -- 172.21.15.144:0/3410111351 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f5a5005bec0 con 0x7f5a5c0ffc80 2024-09-10T11:42:51.536 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:42:51.536 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:42:51.536 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:42:51.536 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:42:51.536 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:42:51.536 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:42:51.536 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:42:51.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.537+0000 7f5a60efa700 1 -- 172.21.15.144:0/3410111351 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a4406e200 msgr2=0x7f5a440706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:51.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.537+0000 7f5a60efa700 1 --2- 172.21.15.144:0/3410111351 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a4406e200 0x7f5a440706b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c004630 tx=0x7f5a4c00b3c0 comp rx=0 tx=0).stop 2024-09-10T11:42:51.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.537+0000 7f5a60efa700 1 -- 172.21.15.144:0/3410111351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a5c0ffc80 msgr2=0x7f5a5c197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:42:51.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.537+0000 7f5a60efa700 1 --2- 172.21.15.144:0/3410111351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a5c0ffc80 0x7f5a5c197100 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5a500092e0 tx=0x7f5a5000bd70 comp rx=0 tx=0).stop 2024-09-10T11:42:51.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.537+0000 7f5a60efa700 1 -- 172.21.15.144:0/3410111351 shutdown_connections 2024-09-10T11:42:51.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.537+0000 7f5a60efa700 1 --2- 172.21.15.144:0/3410111351 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a4406e200 0x7f5a440706b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:51.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.537+0000 7f5a60efa700 1 --2- 172.21.15.144:0/3410111351 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a5c0ff2e0 0x7f5a5c196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:51.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.537+0000 7f5a60efa700 1 --2- 172.21.15.144:0/3410111351 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a5c0ffc80 0x7f5a5c197100 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:42:51.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.537+0000 7f5a60efa700 1 -- 172.21.15.144:0/3410111351 >> 172.21.15.144:0/3410111351 conn(0x7f5a5c0fab80 msgr2=0x7f5a5c1034c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:42:51.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.537+0000 7f5a60efa700 1 -- 172.21.15.144:0/3410111351 shutdown_connections 2024-09-10T11:42:51.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:42:51.537+0000 7f5a60efa700 1 -- 172.21.15.144:0/3410111351 wait complete. 2024-09-10T11:42:51.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:51 smithi144 ceph-mon[87436]: from='client.34660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:51.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:51 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3651936269' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:42:51.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:51 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3410111351' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:42:52.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:51 smithi178 ceph-mon[65940]: from='client.34660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:52.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:51 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3651936269' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:42:52.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:51 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3410111351' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:42:53.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:52 smithi144 ceph-mon[87436]: from='client.34672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:53.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:52 smithi144 ceph-mon[87436]: pgmap v326: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:53.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:52 smithi178 ceph-mon[65940]: from='client.34672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:42:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:52 smithi178 ceph-mon[65940]: pgmap v326: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:54 smithi178 ceph-mon[65940]: pgmap v327: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:54 smithi144 ceph-mon[87436]: pgmap v327: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:56 smithi178 ceph-mon[65940]: pgmap v328: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:57.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:56 smithi144 ceph-mon[87436]: pgmap v328: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:58 smithi178 ceph-mon[65940]: pgmap v329: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:42:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:42:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:58 smithi144 ceph-mon[87436]: pgmap v329: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:42:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:42:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:43:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:00 smithi178 ceph-mon[65940]: pgmap v330: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:01.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:00 smithi144 ceph-mon[87436]: pgmap v330: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:03.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:02 smithi144 ceph-mon[87436]: pgmap v331: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:02 smithi178 ceph-mon[65940]: pgmap v331: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:04 smithi178 ceph-mon[65940]: pgmap v332: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:05.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:04 smithi144 ceph-mon[87436]: pgmap v332: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:07.098 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:06 smithi144 ceph-mon[87436]: pgmap v333: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:06 smithi178 ceph-mon[65940]: pgmap v333: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:08 smithi178 ceph-mon[65940]: pgmap v334: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:09.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:08 smithi144 ceph-mon[87436]: pgmap v334: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:10 smithi178 ceph-mon[65940]: pgmap v335: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:11.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:10 smithi144 ceph-mon[87436]: pgmap v335: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:12 smithi144 ceph-mon[87436]: pgmap v336: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:13.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:12 smithi178 ceph-mon[65940]: pgmap v336: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:43:14.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:43:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:14 smithi178 ceph-mon[65940]: pgmap v337: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:14 smithi144 ceph-mon[87436]: pgmap v337: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:16.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:15 smithi178 ceph-mon[65940]: pgmap v338: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:16.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:15 smithi144 ceph-mon[87436]: pgmap v338: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:18 smithi178 ceph-mon[65940]: pgmap v339: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:18 smithi144 ceph-mon[87436]: pgmap v339: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:20 smithi178 ceph-mon[65940]: pgmap v340: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:21.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:20 smithi144 ceph-mon[87436]: pgmap v340: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:21.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.655+0000 7f898b074700 1 -- 172.21.15.144:0/3675515579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89840ff380 msgr2=0x7f89840ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:21.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.655+0000 7f898b074700 1 --2- 172.21.15.144:0/3675515579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89840ff380 0x7f89840ff750 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f8978009910 tx=0x7f8978009c20 comp rx=0 tx=0).stop 2024-09-10T11:43:21.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.655+0000 7f898b074700 1 -- 172.21.15.144:0/3675515579 shutdown_connections 2024-09-10T11:43:21.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.655+0000 7f898b074700 1 --2- 172.21.15.144:0/3675515579 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89840ffd20 0x7f8984103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:21.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.655+0000 7f898b074700 1 --2- 172.21.15.144:0/3675515579 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89840ff380 0x7f89840ff750 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:21.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.655+0000 7f898b074700 1 -- 172.21.15.144:0/3675515579 >> 172.21.15.144:0/3675515579 conn(0x7f89840fac20 msgr2=0x7f89840fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:21.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.655+0000 7f898b074700 1 -- 172.21.15.144:0/3675515579 shutdown_connections 2024-09-10T11:43:21.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.655+0000 7f898b074700 1 -- 172.21.15.144:0/3675515579 wait complete. 2024-09-10T11:43:21.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.656+0000 7f898b074700 1 Processor -- start 2024-09-10T11:43:21.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.656+0000 7f898b074700 1 -- start start 2024-09-10T11:43:21.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.656+0000 7f898b074700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89840ff380 0x7f8984100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:21.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.656+0000 7f898b074700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89840ffd20 0x7f89841012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:21.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.656+0000 7f898b074700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8984101810 con 0x7f89840ffd20 2024-09-10T11:43:21.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.656+0000 7f898b074700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8984101950 con 0x7f89840ff380 2024-09-10T11:43:21.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.656+0000 7f8983fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89840ffd20 0x7f89841012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:21.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.656+0000 7f8983fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89840ffd20 0x7f89841012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:49854/0 (socket says 172.21.15.144:49854) 2024-09-10T11:43:21.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.656+0000 7f8983fff700 1 -- 172.21.15.144:0/2225529941 learned_addr learned my addr 172.21.15.144:0/2225529941 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:43:21.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.657+0000 7f8988e10700 1 --2- 172.21.15.144:0/2225529941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89840ff380 0x7f8984100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:21.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.657+0000 7f8983fff700 1 -- 172.21.15.144:0/2225529941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89840ff380 msgr2=0x7f8984100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:21.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.657+0000 7f8983fff700 1 --2- 172.21.15.144:0/2225529941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89840ff380 0x7f8984100d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:21.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.657+0000 7f8983fff700 1 -- 172.21.15.144:0/2225529941 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89780095f0 con 0x7f89840ffd20 2024-09-10T11:43:21.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.657+0000 7f8983fff700 1 --2- 172.21.15.144:0/2225529941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89840ffd20 0x7f89841012d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f897000e8f0 tx=0x7f897000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:21.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.657+0000 7f8981ffb700 1 -- 172.21.15.144:0/2225529941 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8970009790 con 0x7f89840ffd20 2024-09-10T11:43:21.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.657+0000 7f8981ffb700 1 -- 172.21.15.144:0/2225529941 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f897000c810 con 0x7f89840ffd20 2024-09-10T11:43:21.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.657+0000 7f8981ffb700 1 -- 172.21.15.144:0/2225529941 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8970005580 con 0x7f89840ffd20 2024-09-10T11:43:21.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.657+0000 7f898b074700 1 -- 172.21.15.144:0/2225529941 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89841a1f30 con 0x7f89840ffd20 2024-09-10T11:43:21.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.657+0000 7f898b074700 1 -- 172.21.15.144:0/2225529941 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89841a2340 con 0x7f89840ffd20 2024-09-10T11:43:21.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.658+0000 7f898b074700 1 -- 172.21.15.144:0/2225529941 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f898404ea50 con 0x7f89840ffd20 2024-09-10T11:43:21.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.659+0000 7f8981ffb700 1 -- 172.21.15.144:0/2225529941 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f897000c980 con 0x7f89840ffd20 2024-09-10T11:43:21.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.659+0000 7f8981ffb700 1 --2- 172.21.15.144:0/2225529941 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f897406e200 0x7f89740706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:21.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.659+0000 7f8981ffb700 1 -- 172.21.15.144:0/2225529941 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8970014070 con 0x7f89840ffd20 2024-09-10T11:43:21.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.659+0000 7f8988e10700 1 --2- 172.21.15.144:0/2225529941 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f897406e200 0x7f89740706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:21.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.660+0000 7f8988e10700 1 --2- 172.21.15.144:0/2225529941 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f897406e200 0x7f89740706b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f897800b760 tx=0x7f8978005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:21.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.664+0000 7f8981ffb700 1 -- 172.21.15.144:0/2225529941 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f89700576a0 con 0x7f89840ffd20 2024-09-10T11:43:21.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.887+0000 7f898b074700 1 -- 172.21.15.144:0/2225529941 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f89840695e0 con 0x7f897406e200 2024-09-10T11:43:21.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.889+0000 7f8981ffb700 1 -- 172.21.15.144:0/2225529941 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f89840695e0 con 0x7f897406e200 2024-09-10T11:43:21.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.891+0000 7f898b074700 1 -- 172.21.15.144:0/2225529941 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f897406e200 msgr2=0x7f89740706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:21.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.891+0000 7f898b074700 1 --2- 172.21.15.144:0/2225529941 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f897406e200 0x7f89740706b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f897800b760 tx=0x7f8978005a90 comp rx=0 tx=0).stop 2024-09-10T11:43:21.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.891+0000 7f898b074700 1 -- 172.21.15.144:0/2225529941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89840ffd20 msgr2=0x7f89841012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:21.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.891+0000 7f898b074700 1 --2- 172.21.15.144:0/2225529941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89840ffd20 0x7f89841012d0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f897000e8f0 tx=0x7f897000ec00 comp rx=0 tx=0).stop 2024-09-10T11:43:21.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.891+0000 7f898b074700 1 -- 172.21.15.144:0/2225529941 shutdown_connections 2024-09-10T11:43:21.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.891+0000 7f898b074700 1 --2- 172.21.15.144:0/2225529941 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f897406e200 0x7f89740706b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:21.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.891+0000 7f898b074700 1 --2- 172.21.15.144:0/2225529941 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f89840ff380 0x7f8984100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:21.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.891+0000 7f898b074700 1 --2- 172.21.15.144:0/2225529941 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f89840ffd20 0x7f89841012d0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:21.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.891+0000 7f898b074700 1 -- 172.21.15.144:0/2225529941 >> 172.21.15.144:0/2225529941 conn(0x7f89840fac20 msgr2=0x7f8984068ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:21.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.891+0000 7f898b074700 1 -- 172.21.15.144:0/2225529941 shutdown_connections 2024-09-10T11:43:21.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.891+0000 7f898b074700 1 -- 172.21.15.144:0/2225529941 wait complete. 2024-09-10T11:43:21.903 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:43:21.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.998+0000 7f3118ab9700 1 -- 172.21.15.144:0/1296366671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31140ffcb0 msgr2=0x7f3114103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:21.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.998+0000 7f3118ab9700 1 --2- 172.21.15.144:0/1296366671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31140ffcb0 0x7f3114103c80 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3108009910 tx=0x7f3108009c20 comp rx=0 tx=0).stop 2024-09-10T11:43:21.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.998+0000 7f3118ab9700 1 -- 172.21.15.144:0/1296366671 shutdown_connections 2024-09-10T11:43:21.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.998+0000 7f3118ab9700 1 --2- 172.21.15.144:0/1296366671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31140ffcb0 0x7f3114103c80 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:21.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.998+0000 7f3118ab9700 1 --2- 172.21.15.144:0/1296366671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31140ff310 0x7f31140ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:21.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.998+0000 7f3118ab9700 1 -- 172.21.15.144:0/1296366671 >> 172.21.15.144:0/1296366671 conn(0x7f31140fabb0 msgr2=0x7f31140fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:22.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.998+0000 7f3118ab9700 1 -- 172.21.15.144:0/1296366671 shutdown_connections 2024-09-10T11:43:22.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.998+0000 7f3118ab9700 1 -- 172.21.15.144:0/1296366671 wait complete. 2024-09-10T11:43:22.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.998+0000 7f3118ab9700 1 Processor -- start 2024-09-10T11:43:22.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.999+0000 7f3118ab9700 1 -- start start 2024-09-10T11:43:22.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.999+0000 7f3118ab9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31140ff310 0x7f3114100d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.999+0000 7f3118ab9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31140ffcb0 0x7f3114101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.999+0000 7f3118ab9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31141017a0 con 0x7f31140ff310 2024-09-10T11:43:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.999+0000 7f3118ab9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31141018e0 con 0x7f31140ffcb0 2024-09-10T11:43:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.999+0000 7f311259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31140ff310 0x7f3114100d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.999+0000 7f311259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31140ff310 0x7f3114100d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:49870/0 (socket says 172.21.15.144:49870) 2024-09-10T11:43:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.999+0000 7f311259c700 1 -- 172.21.15.144:0/1315649406 learned_addr learned my addr 172.21.15.144:0/1315649406 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:43:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.999+0000 7f3111d9b700 1 --2- 172.21.15.144:0/1315649406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31140ffcb0 0x7f3114101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.999+0000 7f311259c700 1 -- 172.21.15.144:0/1315649406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31140ffcb0 msgr2=0x7f3114101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.999+0000 7f311259c700 1 --2- 172.21.15.144:0/1315649406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31140ffcb0 0x7f3114101260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.999+0000 7f311259c700 1 -- 172.21.15.144:0/1315649406 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31080095f0 con 0x7f31140ff310 2024-09-10T11:43:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.999+0000 7f311259c700 1 --2- 172.21.15.144:0/1315649406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31140ff310 0x7f3114100d20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f310400d8d0 tx=0x7f310400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:21.999+0000 7f31037fe700 1 -- 172.21.15.144:0/1315649406 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31040098a0 con 0x7f31140ff310 2024-09-10T11:43:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.000+0000 7f31037fe700 1 -- 172.21.15.144:0/1315649406 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3104010460 con 0x7f31140ff310 2024-09-10T11:43:22.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.000+0000 7f31037fe700 1 -- 172.21.15.144:0/1315649406 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f310400f5f0 con 0x7f31140ff310 2024-09-10T11:43:22.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.000+0000 7f3118ab9700 1 -- 172.21.15.144:0/1315649406 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31141a1f30 con 0x7f31140ff310 2024-09-10T11:43:22.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.000+0000 7f3118ab9700 1 -- 172.21.15.144:0/1315649406 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31141a2340 con 0x7f31140ff310 2024-09-10T11:43:22.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.000+0000 7f3118ab9700 1 -- 172.21.15.144:0/1315649406 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311404ea50 con 0x7f31140ff310 2024-09-10T11:43:22.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.001+0000 7f31037fe700 1 -- 172.21.15.144:0/1315649406 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3104009a00 con 0x7f31140ff310 2024-09-10T11:43:22.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.002+0000 7f31037fe700 1 --2- 172.21.15.144:0/1315649406 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f30fc06e200 0x7f30fc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:22.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.002+0000 7f31037fe700 1 -- 172.21.15.144:0/1315649406 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f310408df70 con 0x7f31140ff310 2024-09-10T11:43:22.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.002+0000 7f3111d9b700 1 --2- 172.21.15.144:0/1315649406 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f30fc06e200 0x7f30fc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:22.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.002+0000 7f3111d9b700 1 --2- 172.21.15.144:0/1315649406 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f30fc06e200 0x7f30fc0706b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3114102510 tx=0x7f310801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:22.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.006+0000 7f31037fe700 1 -- 172.21.15.144:0/1315649406 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f31040c31b0 con 0x7f31140ff310 2024-09-10T11:43:22.226 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.225+0000 7f3118ab9700 1 -- 172.21.15.144:0/1315649406 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3114101aa0 con 0x7f30fc06e200 2024-09-10T11:43:22.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.226+0000 7f31037fe700 1 -- 172.21.15.144:0/1315649406 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3114101aa0 con 0x7f30fc06e200 2024-09-10T11:43:22.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.228+0000 7f3118ab9700 1 -- 172.21.15.144:0/1315649406 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f30fc06e200 msgr2=0x7f30fc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:22.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.228+0000 7f3118ab9700 1 --2- 172.21.15.144:0/1315649406 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f30fc06e200 0x7f30fc0706b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3114102510 tx=0x7f310801a040 comp rx=0 tx=0).stop 2024-09-10T11:43:22.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.228+0000 7f3118ab9700 1 -- 172.21.15.144:0/1315649406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31140ff310 msgr2=0x7f3114100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:22.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.228+0000 7f3118ab9700 1 --2- 172.21.15.144:0/1315649406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31140ff310 0x7f3114100d20 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f310400d8d0 tx=0x7f310400dbe0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.228+0000 7f3118ab9700 1 -- 172.21.15.144:0/1315649406 shutdown_connections 2024-09-10T11:43:22.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.228+0000 7f3118ab9700 1 --2- 172.21.15.144:0/1315649406 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f30fc06e200 0x7f30fc0706b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.228+0000 7f3118ab9700 1 --2- 172.21.15.144:0/1315649406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31140ff310 0x7f3114100d20 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.228+0000 7f3118ab9700 1 --2- 172.21.15.144:0/1315649406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31140ffcb0 0x7f3114101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.228+0000 7f3118ab9700 1 -- 172.21.15.144:0/1315649406 >> 172.21.15.144:0/1315649406 conn(0x7f31140fabb0 msgr2=0x7f3114068e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:22.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.228+0000 7f3118ab9700 1 -- 172.21.15.144:0/1315649406 shutdown_connections 2024-09-10T11:43:22.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.228+0000 7f3118ab9700 1 -- 172.21.15.144:0/1315649406 wait complete. 2024-09-10T11:43:22.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.340+0000 7f8bf0072700 1 -- 172.21.15.144:0/3469856795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8be81003a0 msgr2=0x7f8be8100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:22.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.340+0000 7f8bf0072700 1 --2- 172.21.15.144:0/3469856795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8be81003a0 0x7f8be8100770 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8009940 tx=0x7f8bd8009c50 comp rx=0 tx=0).stop 2024-09-10T11:43:22.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.340+0000 7f8bf0072700 1 -- 172.21.15.144:0/3469856795 shutdown_connections 2024-09-10T11:43:22.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.340+0000 7f8bf0072700 1 --2- 172.21.15.144:0/3469856795 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8be8100cb0 0x7f8be8109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.340+0000 7f8bf0072700 1 --2- 172.21.15.144:0/3469856795 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8be81003a0 0x7f8be8100770 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.340+0000 7f8bf0072700 1 -- 172.21.15.144:0/3469856795 >> 172.21.15.144:0/3469856795 conn(0x7f8be8074850 msgr2=0x7f8be8074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:22.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.341+0000 7f8bf0072700 1 -- 172.21.15.144:0/3469856795 shutdown_connections 2024-09-10T11:43:22.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.341+0000 7f8bf0072700 1 -- 172.21.15.144:0/3469856795 wait complete. 2024-09-10T11:43:22.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.341+0000 7f8bf0072700 1 Processor -- start 2024-09-10T11:43:22.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.341+0000 7f8bf0072700 1 -- start start 2024-09-10T11:43:22.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8bf0072700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8be81003a0 0x7f8be819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:22.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8bf0072700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8be8100cb0 0x7f8be819af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:22.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8bf0072700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be819b610 con 0x7f8be8100cb0 2024-09-10T11:43:22.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8bf0072700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be8194ac0 con 0x7f8be81003a0 2024-09-10T11:43:22.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8bed60d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8be8100cb0 0x7f8be819af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:22.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8bed60d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8be8100cb0 0x7f8be819af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52686/0 (socket says 172.21.15.144:52686) 2024-09-10T11:43:22.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8bed60d700 1 -- 172.21.15.144:0/1147140832 learned_addr learned my addr 172.21.15.144:0/1147140832 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:43:22.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8bede0e700 1 --2- 172.21.15.144:0/1147140832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8be81003a0 0x7f8be819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:22.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8bed60d700 1 -- 172.21.15.144:0/1147140832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8be81003a0 msgr2=0x7f8be819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:22.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8bed60d700 1 --2- 172.21.15.144:0/1147140832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8be81003a0 0x7f8be819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8bed60d700 1 -- 172.21.15.144:0/1147140832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd80095f0 con 0x7f8be8100cb0 2024-09-10T11:43:22.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8bede0e700 1 --2- 172.21.15.144:0/1147140832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8be81003a0 0x7f8be819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:43:22.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8bed60d700 1 --2- 172.21.15.144:0/1147140832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8be8100cb0 0x7f8be819af80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00d8d0 tx=0x7f8bdc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:22.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8be6ffd700 1 -- 172.21.15.144:0/1147140832 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bdc0098a0 con 0x7f8be8100cb0 2024-09-10T11:43:22.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8be6ffd700 1 -- 172.21.15.144:0/1147140832 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bdc010460 con 0x7f8be8100cb0 2024-09-10T11:43:22.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8bf0072700 1 -- 172.21.15.144:0/1147140832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be8194da0 con 0x7f8be8100cb0 2024-09-10T11:43:22.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8be6ffd700 1 -- 172.21.15.144:0/1147140832 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bdc00f5f0 con 0x7f8be8100cb0 2024-09-10T11:43:22.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.342+0000 7f8bf0072700 1 -- 172.21.15.144:0/1147140832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be81952f0 con 0x7f8be8100cb0 2024-09-10T11:43:22.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.343+0000 7f8bf0072700 1 -- 172.21.15.144:0/1147140832 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be8106b10 con 0x7f8be8100cb0 2024-09-10T11:43:22.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.344+0000 7f8be6ffd700 1 -- 172.21.15.144:0/1147140832 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8bdc009a00 con 0x7f8be8100cb0 2024-09-10T11:43:22.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.344+0000 7f8be6ffd700 1 --2- 172.21.15.144:0/1147140832 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8bd006e200 0x7f8bd00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:22.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.344+0000 7f8be6ffd700 1 -- 172.21.15.144:0/1147140832 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8bdc08de70 con 0x7f8be8100cb0 2024-09-10T11:43:22.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.345+0000 7f8bede0e700 1 --2- 172.21.15.144:0/1147140832 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8bd006e200 0x7f8bd00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:22.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.345+0000 7f8bede0e700 1 --2- 172.21.15.144:0/1147140832 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8bd006e200 0x7f8bd00706b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8006010 tx=0x7f8bd8005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:22.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.349+0000 7f8be6ffd700 1 -- 172.21.15.144:0/1147140832 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8bdc05ae00 con 0x7f8be8100cb0 2024-09-10T11:43:22.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.557+0000 7f8bf0072700 1 -- 172.21.15.144:0/1147140832 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8be8195620 con 0x7f8bd006e200 2024-09-10T11:43:22.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.567+0000 7f8be6ffd700 1 -- 172.21.15.144:0/1147140832 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f8be8195620 con 0x7f8bd006e200 2024-09-10T11:43:22.568 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:43:22.568 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (11m) 5m ago 22m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:43:22.568 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (22m) 5m ago 22m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:43:22.568 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (21m) 5m ago 21m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:43:22.568 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (9m) 5m ago 22m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:43:22.568 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (9m) 5m ago 21m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:43:22.568 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (11m) 5m ago 21m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:43:22.569 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (5m) 5m ago 16m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:43:22.569 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (5m) 5m ago 16m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:43:22.569 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (16m) 5m ago 16m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:43:22.569 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (5m) 5m ago 16m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:43:22.569 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (13m) 5m ago 23m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:43:22.569 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (12m) 5m ago 21m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:43:22.569 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (10m) 5m ago 23m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:43:22.569 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (9m) 5m ago 21m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:43:22.569 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (12m) 5m ago 22m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:43:22.569 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (12m) 5m ago 21m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:43:22.569 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (8m) 5m ago 20m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:43:22.569 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (8m) 5m ago 19m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:43:22.569 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (7m) 5m ago 19m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:43:22.569 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (7m) 5m ago 19m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:43:22.570 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (6m) 5m ago 18m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:43:22.570 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (5m) 5m ago 18m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:43:22.570 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (11m) 5m ago 21m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:43:22.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.569+0000 7f8bf0072700 1 -- 172.21.15.144:0/1147140832 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8bd006e200 msgr2=0x7f8bd00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:22.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.569+0000 7f8bf0072700 1 --2- 172.21.15.144:0/1147140832 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8bd006e200 0x7f8bd00706b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8006010 tx=0x7f8bd8005a90 comp rx=0 tx=0).stop 2024-09-10T11:43:22.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.569+0000 7f8bf0072700 1 -- 172.21.15.144:0/1147140832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8be8100cb0 msgr2=0x7f8be819af80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:22.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.569+0000 7f8bf0072700 1 --2- 172.21.15.144:0/1147140832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8be8100cb0 0x7f8be819af80 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8bdc00d8d0 tx=0x7f8bdc00dbe0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.569+0000 7f8bf0072700 1 -- 172.21.15.144:0/1147140832 shutdown_connections 2024-09-10T11:43:22.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.569+0000 7f8bf0072700 1 --2- 172.21.15.144:0/1147140832 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8bd006e200 0x7f8bd00706b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.569+0000 7f8bf0072700 1 --2- 172.21.15.144:0/1147140832 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8be81003a0 0x7f8be819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.569+0000 7f8bf0072700 1 --2- 172.21.15.144:0/1147140832 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8be8100cb0 0x7f8be819af80 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.569+0000 7f8bf0072700 1 -- 172.21.15.144:0/1147140832 >> 172.21.15.144:0/1147140832 conn(0x7f8be8074850 msgr2=0x7f8be8103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:22.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.569+0000 7f8bf0072700 1 -- 172.21.15.144:0/1147140832 shutdown_connections 2024-09-10T11:43:22.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.569+0000 7f8bf0072700 1 -- 172.21.15.144:0/1147140832 wait complete. 2024-09-10T11:43:22.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.672+0000 7f9970a5f700 1 -- 172.21.15.144:0/2030271772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c101cd0 msgr2=0x7f996c10a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:22.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.672+0000 7f9970a5f700 1 --2- 172.21.15.144:0/2030271772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c101cd0 0x7f996c10a1d0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f995c009910 tx=0x7f995c009c20 comp rx=0 tx=0).stop 2024-09-10T11:43:22.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.672+0000 7f9970a5f700 1 -- 172.21.15.144:0/2030271772 shutdown_connections 2024-09-10T11:43:22.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.672+0000 7f9970a5f700 1 --2- 172.21.15.144:0/2030271772 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c101cd0 0x7f996c10a1d0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.672+0000 7f9970a5f700 1 --2- 172.21.15.144:0/2030271772 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f996c1013c0 0x7f996c101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.672+0000 7f9970a5f700 1 -- 172.21.15.144:0/2030271772 >> 172.21.15.144:0/2030271772 conn(0x7f996c0fab80 msgr2=0x7f996c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:22.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.672+0000 7f9970a5f700 1 -- 172.21.15.144:0/2030271772 shutdown_connections 2024-09-10T11:43:22.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.673+0000 7f9970a5f700 1 -- 172.21.15.144:0/2030271772 wait complete. 2024-09-10T11:43:22.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.673+0000 7f9970a5f700 1 Processor -- start 2024-09-10T11:43:22.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.673+0000 7f9970a5f700 1 -- start start 2024-09-10T11:43:22.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.673+0000 7f9970a5f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f996c1013c0 0x7f996c195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:22.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.673+0000 7f9970a5f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c101cd0 0x7f996c1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:22.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.673+0000 7f9970a5f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c196ac0 con 0x7f996c101cd0 2024-09-10T11:43:22.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.673+0000 7f9970a5f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f996c19a850 con 0x7f996c1013c0 2024-09-10T11:43:22.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.674+0000 7f9969d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c101cd0 0x7f996c1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:22.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.674+0000 7f9969d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c101cd0 0x7f996c1963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52710/0 (socket says 172.21.15.144:52710) 2024-09-10T11:43:22.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.674+0000 7f996a59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f996c1013c0 0x7f996c195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.674+0000 7f9969d9b700 1 -- 172.21.15.144:0/244560807 learned_addr learned my addr 172.21.15.144:0/244560807 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:43:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.674+0000 7f9969d9b700 1 -- 172.21.15.144:0/244560807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f996c1013c0 msgr2=0x7f996c195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.674+0000 7f9969d9b700 1 --2- 172.21.15.144:0/244560807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f996c1013c0 0x7f996c195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.674+0000 7f9969d9b700 1 -- 172.21.15.144:0/244560807 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995c0095f0 con 0x7f996c101cd0 2024-09-10T11:43:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.674+0000 7f9969d9b700 1 --2- 172.21.15.144:0/244560807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c101cd0 0x7f996c1963e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f995c006010 tx=0x7f995c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.674+0000 7f99637fe700 1 -- 172.21.15.144:0/244560807 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f995c01d070 con 0x7f996c101cd0 2024-09-10T11:43:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.674+0000 7f99637fe700 1 -- 172.21.15.144:0/244560807 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f995c00f460 con 0x7f996c101cd0 2024-09-10T11:43:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.674+0000 7f9970a5f700 1 -- 172.21.15.144:0/244560807 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996c19aad0 con 0x7f996c101cd0 2024-09-10T11:43:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.674+0000 7f99637fe700 1 -- 172.21.15.144:0/244560807 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f995c017610 con 0x7f996c101cd0 2024-09-10T11:43:22.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.674+0000 7f9970a5f700 1 -- 172.21.15.144:0/244560807 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f996c19afc0 con 0x7f996c101cd0 2024-09-10T11:43:22.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.675+0000 7f9970a5f700 1 -- 172.21.15.144:0/244560807 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f996c107840 con 0x7f996c101cd0 2024-09-10T11:43:22.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.676+0000 7f99637fe700 1 -- 172.21.15.144:0/244560807 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f995c00f5d0 con 0x7f996c101cd0 2024-09-10T11:43:22.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.676+0000 7f99637fe700 1 --2- 172.21.15.144:0/244560807 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f995806e200 0x7f99580706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:22.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.677+0000 7f99637fe700 1 -- 172.21.15.144:0/244560807 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f995c08f510 con 0x7f996c101cd0 2024-09-10T11:43:22.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.677+0000 7f996a59c700 1 --2- 172.21.15.144:0/244560807 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f995806e200 0x7f99580706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:22.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.677+0000 7f996a59c700 1 --2- 172.21.15.144:0/244560807 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f995806e200 0x7f99580706b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9954009a10 tx=0x7f9954009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:22.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.681+0000 7f99637fe700 1 -- 172.21.15.144:0/244560807 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f995c058970 con 0x7f996c101cd0 2024-09-10T11:43:22.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:22 smithi144 ceph-mon[87436]: pgmap v341: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:22.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.967+0000 7f9970a5f700 1 -- 172.21.15.144:0/244560807 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f996c19b2f0 con 0x7f996c101cd0 2024-09-10T11:43:22.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.967+0000 7f99637fe700 1 -- 172.21.15.144:0/244560807 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f995c05c030 con 0x7f996c101cd0 2024-09-10T11:43:22.968 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:43:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:43:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:43:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:43:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:43:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:43:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:43:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:43:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:43:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:43:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:43:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:43:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:43:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:43:22.969 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:43:22.970 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:43:22.970 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:43:22.970 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:43:22.970 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:43:22.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.969+0000 7f9970a5f700 1 -- 172.21.15.144:0/244560807 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f995806e200 msgr2=0x7f99580706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:22.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.969+0000 7f9970a5f700 1 --2- 172.21.15.144:0/244560807 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f995806e200 0x7f99580706b0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9954009a10 tx=0x7f9954009190 comp rx=0 tx=0).stop 2024-09-10T11:43:22.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.969+0000 7f9970a5f700 1 -- 172.21.15.144:0/244560807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c101cd0 msgr2=0x7f996c1963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:22.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.969+0000 7f9970a5f700 1 --2- 172.21.15.144:0/244560807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c101cd0 0x7f996c1963e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f995c006010 tx=0x7f995c00bd70 comp rx=0 tx=0).stop 2024-09-10T11:43:22.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.969+0000 7f9970a5f700 1 -- 172.21.15.144:0/244560807 shutdown_connections 2024-09-10T11:43:22.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.969+0000 7f9970a5f700 1 --2- 172.21.15.144:0/244560807 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f995806e200 0x7f99580706b0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.969+0000 7f9970a5f700 1 --2- 172.21.15.144:0/244560807 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f996c1013c0 0x7f996c195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.969+0000 7f9970a5f700 1 --2- 172.21.15.144:0/244560807 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f996c101cd0 0x7f996c1963e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:22.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.969+0000 7f9970a5f700 1 -- 172.21.15.144:0/244560807 >> 172.21.15.144:0/244560807 conn(0x7f996c0fab80 msgr2=0x7f996c104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:22.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.970+0000 7f9970a5f700 1 -- 172.21.15.144:0/244560807 shutdown_connections 2024-09-10T11:43:22.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:22.970+0000 7f9970a5f700 1 -- 172.21.15.144:0/244560807 wait complete. 2024-09-10T11:43:23.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.073+0000 7fc31c611700 1 -- 172.21.15.144:0/1065213842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314101c30 msgr2=0x7fc31410a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:23.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.073+0000 7fc31c611700 1 --2- 172.21.15.144:0/1065213842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314101c30 0x7fc31410a130 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc308009910 tx=0x7fc308009c20 comp rx=0 tx=0).stop 2024-09-10T11:43:23.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.073+0000 7fc31c611700 1 -- 172.21.15.144:0/1065213842 shutdown_connections 2024-09-10T11:43:23.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.073+0000 7fc31c611700 1 --2- 172.21.15.144:0/1065213842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314101c30 0x7fc31410a130 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:23.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.073+0000 7fc31c611700 1 --2- 172.21.15.144:0/1065213842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc314101320 0x7fc3141016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:23.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.073+0000 7fc31c611700 1 -- 172.21.15.144:0/1065213842 >> 172.21.15.144:0/1065213842 conn(0x7fc3140fab80 msgr2=0x7fc3140fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:23.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.074+0000 7fc31c611700 1 -- 172.21.15.144:0/1065213842 shutdown_connections 2024-09-10T11:43:23.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.074+0000 7fc31c611700 1 -- 172.21.15.144:0/1065213842 wait complete. 2024-09-10T11:43:23.076 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.074+0000 7fc31c611700 1 Processor -- start 2024-09-10T11:43:23.076 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.074+0000 7fc31c611700 1 -- start start 2024-09-10T11:43:23.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.074+0000 7fc31c611700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314101320 0x7fc3140fef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:23.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.074+0000 7fc31c611700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc314101c30 0x7fc3140ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:23.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.074+0000 7fc31c611700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3140ffa00 con 0x7fc314101320 2024-09-10T11:43:23.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.074+0000 7fc31c611700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3140ffb40 con 0x7fc314101c30 2024-09-10T11:43:23.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.074+0000 7fc31a3ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314101320 0x7fc3140fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:23.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.075+0000 7fc31a3ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314101320 0x7fc3140fef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52718/0 (socket says 172.21.15.144:52718) 2024-09-10T11:43:23.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.075+0000 7fc31a3ad700 1 -- 172.21.15.144:0/1005344533 learned_addr learned my addr 172.21.15.144:0/1005344533 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:43:23.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.075+0000 7fc319bac700 1 --2- 172.21.15.144:0/1005344533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc314101c30 0x7fc3140ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:23.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.075+0000 7fc31a3ad700 1 -- 172.21.15.144:0/1005344533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc314101c30 msgr2=0x7fc3140ff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:23.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.075+0000 7fc31a3ad700 1 --2- 172.21.15.144:0/1005344533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc314101c30 0x7fc3140ff4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:23.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.075+0000 7fc31a3ad700 1 -- 172.21.15.144:0/1005344533 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3080095f0 con 0x7fc314101320 2024-09-10T11:43:23.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.075+0000 7fc319bac700 1 --2- 172.21.15.144:0/1005344533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc314101c30 0x7fc3140ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:43:23.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.075+0000 7fc31a3ad700 1 --2- 172.21.15.144:0/1005344533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314101320 0x7fc3140fef80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00ba50 tx=0x7fc30c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:23.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.075+0000 7fc3137fe700 1 -- 172.21.15.144:0/1005344533 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc30c00ca00 con 0x7fc314101320 2024-09-10T11:43:23.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.075+0000 7fc3137fe700 1 -- 172.21.15.144:0/1005344533 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc30c00cb60 con 0x7fc314101320 2024-09-10T11:43:23.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.075+0000 7fc3137fe700 1 -- 172.21.15.144:0/1005344533 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc30c0075a0 con 0x7fc314101320 2024-09-10T11:43:23.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.075+0000 7fc31c611700 1 -- 172.21.15.144:0/1005344533 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3141a2020 con 0x7fc314101320 2024-09-10T11:43:23.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.075+0000 7fc31c611700 1 -- 172.21.15.144:0/1005344533 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3141a2430 con 0x7fc314101320 2024-09-10T11:43:23.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.076+0000 7fc31c611700 1 -- 172.21.15.144:0/1005344533 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3141077a0 con 0x7fc314101320 2024-09-10T11:43:23.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.077+0000 7fc3137fe700 1 -- 172.21.15.144:0/1005344533 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc30c00ccd0 con 0x7fc314101320 2024-09-10T11:43:23.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.077+0000 7fc3137fe700 1 --2- 172.21.15.144:0/1005344533 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc2fc06e200 0x7fc2fc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:23.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.077+0000 7fc3137fe700 1 -- 172.21.15.144:0/1005344533 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc30c006e40 con 0x7fc314101320 2024-09-10T11:43:23.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.077+0000 7fc319bac700 1 --2- 172.21.15.144:0/1005344533 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc2fc06e200 0x7fc2fc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:23.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.078+0000 7fc319bac700 1 --2- 172.21.15.144:0/1005344533 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc2fc06e200 0x7fc2fc0706b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc30800b5c0 tx=0x7fc308005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:23.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.081+0000 7fc3137fe700 1 -- 172.21.15.144:0/1005344533 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc30c056c90 con 0x7fc314101320 2024-09-10T11:43:23.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:22 smithi178 ceph-mon[65940]: pgmap v341: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:23.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.325+0000 7fc31c611700 1 -- 172.21.15.144:0/1005344533 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc31404ea50 con 0x7fc314101320 2024-09-10T11:43:23.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.325+0000 7fc3137fe700 1 -- 172.21.15.144:0/1005344533 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7fc30c08be60 con 0x7fc314101320 2024-09-10T11:43:23.327 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:43:23.327 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:43:23.327 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:43:23.327 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:43:23.327 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:43:23.327 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:43:23.327 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:43:23.327 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:43:23.327 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:43:23.327 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:43:23.327 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:43:23.327 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:43:23.328 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:43:23.328 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:43:23.328 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:43:23.328 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:43:23.328 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:43:23.328 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:43:23.328 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:43:23.328 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:43:23.328 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:43:23.328 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:43:23.328 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:43:23.328 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:43:23.328 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:43:23.328 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:43:23.329 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:43:23.329 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:43:23.329 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:43:23.329 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:43:23.329 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:43:23.329 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:43:23.329 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:43:23.329 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:43:23.329 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:43:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:43:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:43:23.329 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:43:23.329 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:43:23.329 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:43:23.330 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:43:23.330 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:43:23.330 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:43:23.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.328+0000 7fc31c611700 1 -- 172.21.15.144:0/1005344533 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc2fc06e200 msgr2=0x7fc2fc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:23.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.328+0000 7fc31c611700 1 --2- 172.21.15.144:0/1005344533 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc2fc06e200 0x7fc2fc0706b0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fc30800b5c0 tx=0x7fc308005480 comp rx=0 tx=0).stop 2024-09-10T11:43:23.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.328+0000 7fc31c611700 1 -- 172.21.15.144:0/1005344533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314101320 msgr2=0x7fc3140fef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:23.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.328+0000 7fc31c611700 1 --2- 172.21.15.144:0/1005344533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314101320 0x7fc3140fef80 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00ba50 tx=0x7fc30c00be10 comp rx=0 tx=0).stop 2024-09-10T11:43:23.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.328+0000 7fc31c611700 1 -- 172.21.15.144:0/1005344533 shutdown_connections 2024-09-10T11:43:23.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.328+0000 7fc31c611700 1 --2- 172.21.15.144:0/1005344533 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc2fc06e200 0x7fc2fc0706b0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:23.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.328+0000 7fc31c611700 1 --2- 172.21.15.144:0/1005344533 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc314101320 0x7fc3140fef80 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:23.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.328+0000 7fc31c611700 1 --2- 172.21.15.144:0/1005344533 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc314101c30 0x7fc3140ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:23.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.328+0000 7fc31c611700 1 -- 172.21.15.144:0/1005344533 >> 172.21.15.144:0/1005344533 conn(0x7fc3140fab80 msgr2=0x7fc3141048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:23.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.328+0000 7fc31c611700 1 -- 172.21.15.144:0/1005344533 shutdown_connections 2024-09-10T11:43:23.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.328+0000 7fc31c611700 1 -- 172.21.15.144:0/1005344533 wait complete. 2024-09-10T11:43:23.331 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:43:23.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.434+0000 7fe77e318700 1 -- 172.21.15.144:0/389045987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101370 msgr2=0x7fe778101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:23.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.434+0000 7fe77e318700 1 --2- 172.21.15.144:0/389045987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101370 0x7fe778101740 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fe768009910 tx=0x7fe768009c20 comp rx=0 tx=0).stop 2024-09-10T11:43:23.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.434+0000 7fe77e318700 1 -- 172.21.15.144:0/389045987 shutdown_connections 2024-09-10T11:43:23.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.434+0000 7fe77e318700 1 --2- 172.21.15.144:0/389045987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe778101c80 0x7fe77810a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:23.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.434+0000 7fe77e318700 1 --2- 172.21.15.144:0/389045987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101370 0x7fe778101740 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:23.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.434+0000 7fe77e318700 1 -- 172.21.15.144:0/389045987 >> 172.21.15.144:0/389045987 conn(0x7fe7780fabb0 msgr2=0x7fe7780fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:23.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.434+0000 7fe77e318700 1 -- 172.21.15.144:0/389045987 shutdown_connections 2024-09-10T11:43:23.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.434+0000 7fe77e318700 1 -- 172.21.15.144:0/389045987 wait complete. 2024-09-10T11:43:23.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.434+0000 7fe77e318700 1 Processor -- start 2024-09-10T11:43:23.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.435+0000 7fe77e318700 1 -- start start 2024-09-10T11:43:23.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.435+0000 7fe77e318700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe778101370 0x7fe778195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:23.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.435+0000 7fe77e318700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101c80 0x7fe778196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:23.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.435+0000 7fe77e318700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe778196960 con 0x7fe778101c80 2024-09-10T11:43:23.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.435+0000 7fe77e318700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe77819a6f0 con 0x7fe778101370 2024-09-10T11:43:23.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.435+0000 7fe7777fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101c80 0x7fe778196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:23.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.435+0000 7fe7777fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101c80 0x7fe778196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52740/0 (socket says 172.21.15.144:52740) 2024-09-10T11:43:23.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.435+0000 7fe7777fe700 1 -- 172.21.15.144:0/3816819749 learned_addr learned my addr 172.21.15.144:0/3816819749 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:43:23.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.435+0000 7fe777fff700 1 --2- 172.21.15.144:0/3816819749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe778101370 0x7fe778195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:23.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.435+0000 7fe7777fe700 1 -- 172.21.15.144:0/3816819749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe778101370 msgr2=0x7fe778195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:23.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.435+0000 7fe7777fe700 1 --2- 172.21.15.144:0/3816819749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe778101370 0x7fe778195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:23.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.435+0000 7fe7777fe700 1 -- 172.21.15.144:0/3816819749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7680095f0 con 0x7fe778101c80 2024-09-10T11:43:23.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.435+0000 7fe777fff700 1 --2- 172.21.15.144:0/3816819749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe778101370 0x7fe778195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:43:23.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.435+0000 7fe7777fe700 1 --2- 172.21.15.144:0/3816819749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101c80 0x7fe778196280 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00d8d0 tx=0x7fe76c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:23.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.436+0000 7fe7757fa700 1 -- 172.21.15.144:0/3816819749 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe76c009940 con 0x7fe778101c80 2024-09-10T11:43:23.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.436+0000 7fe77e318700 1 -- 172.21.15.144:0/3816819749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe77819a9d0 con 0x7fe778101c80 2024-09-10T11:43:23.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.436+0000 7fe7757fa700 1 -- 172.21.15.144:0/3816819749 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe76c010460 con 0x7fe778101c80 2024-09-10T11:43:23.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.436+0000 7fe7757fa700 1 -- 172.21.15.144:0/3816819749 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe76c00f5f0 con 0x7fe778101c80 2024-09-10T11:43:23.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.436+0000 7fe77e318700 1 -- 172.21.15.144:0/3816819749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe77819af20 con 0x7fe778101c80 2024-09-10T11:43:23.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.437+0000 7fe77e318700 1 -- 172.21.15.144:0/3816819749 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe778107850 con 0x7fe778101c80 2024-09-10T11:43:23.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.437+0000 7fe7757fa700 1 -- 172.21.15.144:0/3816819749 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe76c0105d0 con 0x7fe778101c80 2024-09-10T11:43:23.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.438+0000 7fe7757fa700 1 --2- 172.21.15.144:0/3816819749 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe76006e200 0x7fe7600706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:23.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.438+0000 7fe7757fa700 1 -- 172.21.15.144:0/3816819749 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe76c08e110 con 0x7fe778101c80 2024-09-10T11:43:23.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.438+0000 7fe777fff700 1 --2- 172.21.15.144:0/3816819749 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe76006e200 0x7fe7600706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:23.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.438+0000 7fe777fff700 1 --2- 172.21.15.144:0/3816819749 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe76006e200 0x7fe7600706b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe76800b5c0 tx=0x7fe768005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:23.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.441+0000 7fe7757fa700 1 -- 172.21.15.144:0/3816819749 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe76c057580 con 0x7fe778101c80 2024-09-10T11:43:23.665 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.664+0000 7fe77e318700 1 -- 172.21.15.144:0/3816819749 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe77819b310 con 0x7fe76006e200 2024-09-10T11:43:23.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.665+0000 7fe7757fa700 1 -- 172.21.15.144:0/3816819749 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe77819b310 con 0x7fe76006e200 2024-09-10T11:43:23.666 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:43:23.666 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:43:23.666 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:43:23.667 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:43:23.667 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:43:23.667 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:43:23.667 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:43:23.667 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:43:23.667 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:43:23.667 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:43:23.667 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:43:23.667 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:43:23.667 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:43:23.667 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:43:23.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.668+0000 7fe77e318700 1 -- 172.21.15.144:0/3816819749 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe76006e200 msgr2=0x7fe7600706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:23.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.668+0000 7fe77e318700 1 --2- 172.21.15.144:0/3816819749 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe76006e200 0x7fe7600706b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe76800b5c0 tx=0x7fe768005960 comp rx=0 tx=0).stop 2024-09-10T11:43:23.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.668+0000 7fe77e318700 1 -- 172.21.15.144:0/3816819749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101c80 msgr2=0x7fe778196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:23.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.668+0000 7fe77e318700 1 --2- 172.21.15.144:0/3816819749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101c80 0x7fe778196280 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fe76c00d8d0 tx=0x7fe76c00dc90 comp rx=0 tx=0).stop 2024-09-10T11:43:23.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.668+0000 7fe77e318700 1 -- 172.21.15.144:0/3816819749 shutdown_connections 2024-09-10T11:43:23.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.668+0000 7fe77e318700 1 --2- 172.21.15.144:0/3816819749 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe76006e200 0x7fe7600706b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:23.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.668+0000 7fe77e318700 1 --2- 172.21.15.144:0/3816819749 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe778101370 0x7fe778195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:23.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.668+0000 7fe77e318700 1 --2- 172.21.15.144:0/3816819749 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe778101c80 0x7fe778196280 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:23.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.668+0000 7fe77e318700 1 -- 172.21.15.144:0/3816819749 >> 172.21.15.144:0/3816819749 conn(0x7fe7780fabb0 msgr2=0x7fe778104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:23.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.668+0000 7fe77e318700 1 -- 172.21.15.144:0/3816819749 shutdown_connections 2024-09-10T11:43:23.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.668+0000 7fe77e318700 1 -- 172.21.15.144:0/3816819749 wait complete. 2024-09-10T11:43:23.774 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.773+0000 7fc2ce5de700 1 -- 172.21.15.144:0/4067980684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2c81003e0 msgr2=0x7fc2c81007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:23.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.773+0000 7fc2ce5de700 1 --2- 172.21.15.144:0/4067980684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2c81003e0 0x7fc2c81007b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009910 tx=0x7fc2b8009c20 comp rx=0 tx=0).stop 2024-09-10T11:43:23.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.773+0000 7fc2ce5de700 1 -- 172.21.15.144:0/4067980684 shutdown_connections 2024-09-10T11:43:23.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.773+0000 7fc2ce5de700 1 --2- 172.21.15.144:0/4067980684 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2c8100d80 0x7fc2c8104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:23.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.773+0000 7fc2ce5de700 1 --2- 172.21.15.144:0/4067980684 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2c81003e0 0x7fc2c81007b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:23.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.773+0000 7fc2ce5de700 1 -- 172.21.15.144:0/4067980684 >> 172.21.15.144:0/4067980684 conn(0x7fc2c80fbca0 msgr2=0x7fc2c80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:23.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.773+0000 7fc2ce5de700 1 -- 172.21.15.144:0/4067980684 shutdown_connections 2024-09-10T11:43:23.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.773+0000 7fc2ce5de700 1 -- 172.21.15.144:0/4067980684 wait complete. 2024-09-10T11:43:23.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.774+0000 7fc2ce5de700 1 Processor -- start 2024-09-10T11:43:23.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.774+0000 7fc2ce5de700 1 -- start start 2024-09-10T11:43:23.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.774+0000 7fc2ce5de700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2c81003e0 0x7fc2c8195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:23.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.774+0000 7fc2ce5de700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2c8100d80 0x7fc2c8196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:23.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.774+0000 7fc2ce5de700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c8196940 con 0x7fc2c8100d80 2024-09-10T11:43:23.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.775+0000 7fc2ce5de700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c819a6d0 con 0x7fc2c81003e0 2024-09-10T11:43:23.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.775+0000 7fc2c77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2c8100d80 0x7fc2c8196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:23.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.775+0000 7fc2c77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2c8100d80 0x7fc2c8196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52764/0 (socket says 172.21.15.144:52764) 2024-09-10T11:43:23.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.775+0000 7fc2c77fe700 1 -- 172.21.15.144:0/702885357 learned_addr learned my addr 172.21.15.144:0/702885357 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:43:23.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.775+0000 7fc2c7fff700 1 --2- 172.21.15.144:0/702885357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2c81003e0 0x7fc2c8195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:23.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.775+0000 7fc2c77fe700 1 -- 172.21.15.144:0/702885357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2c81003e0 msgr2=0x7fc2c8195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:23.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.775+0000 7fc2c77fe700 1 --2- 172.21.15.144:0/702885357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2c81003e0 0x7fc2c8195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:23.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.775+0000 7fc2c77fe700 1 -- 172.21.15.144:0/702885357 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b80095f0 con 0x7fc2c8100d80 2024-09-10T11:43:23.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.775+0000 7fc2c77fe700 1 --2- 172.21.15.144:0/702885357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2c8100d80 0x7fc2c8196260 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00d900 tx=0x7fc2bc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:23.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.775+0000 7fc2c57fa700 1 -- 172.21.15.144:0/702885357 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2bc009900 con 0x7fc2c8100d80 2024-09-10T11:43:23.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.775+0000 7fc2ce5de700 1 -- 172.21.15.144:0/702885357 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c819a9b0 con 0x7fc2c8100d80 2024-09-10T11:43:23.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.775+0000 7fc2c57fa700 1 -- 172.21.15.144:0/702885357 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc2bc010460 con 0x7fc2c8100d80 2024-09-10T11:43:23.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.775+0000 7fc2c57fa700 1 -- 172.21.15.144:0/702885357 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2bc00f5f0 con 0x7fc2c8100d80 2024-09-10T11:43:23.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.775+0000 7fc2ce5de700 1 -- 172.21.15.144:0/702885357 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c819af00 con 0x7fc2c8100d80 2024-09-10T11:43:23.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.777+0000 7fc2ce5de700 1 -- 172.21.15.144:0/702885357 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c81086f0 con 0x7fc2c8100d80 2024-09-10T11:43:23.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.777+0000 7fc2c57fa700 1 -- 172.21.15.144:0/702885357 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc2bc0105d0 con 0x7fc2c8100d80 2024-09-10T11:43:23.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.777+0000 7fc2c57fa700 1 --2- 172.21.15.144:0/702885357 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc2b006e200 0x7fc2b00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:23.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.777+0000 7fc2c57fa700 1 -- 172.21.15.144:0/702885357 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc2bc08dc60 con 0x7fc2c8100d80 2024-09-10T11:43:23.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.777+0000 7fc2c7fff700 1 --2- 172.21.15.144:0/702885357 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc2b006e200 0x7fc2b00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:23.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.778+0000 7fc2c7fff700 1 --2- 172.21.15.144:0/702885357 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc2b006e200 0x7fc2b00706b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8006010 tx=0x7fc2b801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:23.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:23.782+0000 7fc2c57fa700 1 -- 172.21.15.144:0/702885357 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc2bc057040 con 0x7fc2c8100d80 2024-09-10T11:43:23.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:23 smithi144 ceph-mon[87436]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:23.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:23 smithi144 ceph-mon[87436]: from='client.34684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:23.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:23 smithi144 ceph-mon[87436]: from='client.34688 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:23.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:23 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/244560807' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:43:23.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:23 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1005344533' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:43:24.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:24.076+0000 7fc2ce5de700 1 -- 172.21.15.144:0/702885357 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc2c819b1e0 con 0x7fc2c8100d80 2024-09-10T11:43:24.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:24.076+0000 7fc2c57fa700 1 -- 172.21.15.144:0/702885357 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fc2bc0171c0 con 0x7fc2c8100d80 2024-09-10T11:43:24.078 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:43:24.078 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:43:24.078 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:43:24.078 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:43:24.078 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:43:24.078 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:43:24.078 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:43:24.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:24.079+0000 7fc2ce5de700 1 -- 172.21.15.144:0/702885357 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc2b006e200 msgr2=0x7fc2b00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:24.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:24.079+0000 7fc2ce5de700 1 --2- 172.21.15.144:0/702885357 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc2b006e200 0x7fc2b00706b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8006010 tx=0x7fc2b801a040 comp rx=0 tx=0).stop 2024-09-10T11:43:24.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:24.079+0000 7fc2ce5de700 1 -- 172.21.15.144:0/702885357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2c8100d80 msgr2=0x7fc2c8196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:24.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:24.079+0000 7fc2ce5de700 1 --2- 172.21.15.144:0/702885357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2c8100d80 0x7fc2c8196260 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00d900 tx=0x7fc2bc00dc10 comp rx=0 tx=0).stop 2024-09-10T11:43:24.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:24.079+0000 7fc2ce5de700 1 -- 172.21.15.144:0/702885357 shutdown_connections 2024-09-10T11:43:24.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:24.079+0000 7fc2ce5de700 1 --2- 172.21.15.144:0/702885357 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc2b006e200 0x7fc2b00706b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:24.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:24.079+0000 7fc2ce5de700 1 --2- 172.21.15.144:0/702885357 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2c81003e0 0x7fc2c8195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:24.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:24.079+0000 7fc2ce5de700 1 --2- 172.21.15.144:0/702885357 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2c8100d80 0x7fc2c8196260 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:24.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:24.079+0000 7fc2ce5de700 1 -- 172.21.15.144:0/702885357 >> 172.21.15.144:0/702885357 conn(0x7fc2c80fbca0 msgr2=0x7fc2c80fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:24.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:24.079+0000 7fc2ce5de700 1 -- 172.21.15.144:0/702885357 shutdown_connections 2024-09-10T11:43:24.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:24.079+0000 7fc2ce5de700 1 -- 172.21.15.144:0/702885357 wait complete. 2024-09-10T11:43:24.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:23 smithi178 ceph-mon[65940]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:24.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:23 smithi178 ceph-mon[65940]: from='client.34684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:24.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:23 smithi178 ceph-mon[65940]: from='client.34688 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:24.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:23 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/244560807' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:43:24.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:23 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1005344533' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:43:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:24 smithi178 ceph-mon[65940]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:24 smithi178 ceph-mon[65940]: pgmap v342: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:24 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/702885357' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:43:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:24 smithi144 ceph-mon[87436]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:24 smithi144 ceph-mon[87436]: pgmap v342: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:24 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/702885357' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:43:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:26 smithi178 ceph-mon[65940]: pgmap v343: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:27.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:26 smithi144 ceph-mon[87436]: pgmap v343: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:28 smithi178 ceph-mon[65940]: pgmap v344: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:43:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:28 smithi144 ceph-mon[87436]: pgmap v344: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:43:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:30 smithi178 ceph-mon[65940]: pgmap v345: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:31.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:30 smithi144 ceph-mon[87436]: pgmap v345: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:33.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:32 smithi144 ceph-mon[87436]: pgmap v346: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:33.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:32 smithi178 ceph-mon[65940]: pgmap v346: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:34 smithi178 ceph-mon[65940]: pgmap v347: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:34 smithi144 ceph-mon[87436]: pgmap v347: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:36 smithi178 ceph-mon[65940]: pgmap v348: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:37.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:36 smithi144 ceph-mon[87436]: pgmap v348: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:38 smithi178 ceph-mon[65940]: pgmap v349: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:38.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:38 smithi144 ceph-mon[87436]: pgmap v349: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:40 smithi178 ceph-mon[65940]: pgmap v350: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:41.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:40 smithi144 ceph-mon[87436]: pgmap v350: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:43.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:42 smithi144 ceph-mon[87436]: pgmap v351: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:43.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:42 smithi178 ceph-mon[65940]: pgmap v351: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:43:44.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:43:45.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:44 smithi178 ceph-mon[65940]: pgmap v352: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:44 smithi144 ceph-mon[87436]: pgmap v352: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:46 smithi178 ceph-mon[65940]: pgmap v353: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:47.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:46 smithi144 ceph-mon[87436]: pgmap v353: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:48 smithi178 ceph-mon[65940]: pgmap v354: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:49.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:48 smithi144 ceph-mon[87436]: pgmap v354: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:51.029 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:50 smithi178 ceph-mon[65940]: pgmap v355: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:50 smithi144 ceph-mon[87436]: pgmap v355: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:52.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:52 smithi144 ceph-mon[87436]: pgmap v356: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:52 smithi178 ceph-mon[65940]: pgmap v356: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:54.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.193+0000 7fa424a10700 1 -- 172.21.15.144:0/1426883703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4201003d0 msgr2=0x7fa4201007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:54.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.193+0000 7fa424a10700 1 --2- 172.21.15.144:0/1426883703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4201003d0 0x7fa4201007a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fa41000b0a0 tx=0x7fa41000b3b0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.193+0000 7fa424a10700 1 -- 172.21.15.144:0/1426883703 shutdown_connections 2024-09-10T11:43:54.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.193+0000 7fa424a10700 1 --2- 172.21.15.144:0/1426883703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa420100d70 0x7fa420104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.193+0000 7fa424a10700 1 --2- 172.21.15.144:0/1426883703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4201003d0 0x7fa4201007a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.193+0000 7fa424a10700 1 -- 172.21.15.144:0/1426883703 >> 172.21.15.144:0/1426883703 conn(0x7fa4200fbc50 msgr2=0x7fa4200fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:54.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.193+0000 7fa424a10700 1 -- 172.21.15.144:0/1426883703 shutdown_connections 2024-09-10T11:43:54.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.193+0000 7fa424a10700 1 -- 172.21.15.144:0/1426883703 wait complete. 2024-09-10T11:43:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.194+0000 7fa424a10700 1 Processor -- start 2024-09-10T11:43:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.194+0000 7fa424a10700 1 -- start start 2024-09-10T11:43:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.194+0000 7fa424a10700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4201003d0 0x7fa420195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.194+0000 7fa424a10700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa420100d70 0x7fa4201962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.194+0000 7fa424a10700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4201969d0 con 0x7fa420100d70 2024-09-10T11:43:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.194+0000 7fa424a10700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa42019a760 con 0x7fa4201003d0 2024-09-10T11:43:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.194+0000 7fa41dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa420100d70 0x7fa4201962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.194+0000 7fa41dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa420100d70 0x7fa4201962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56100/0 (socket says 172.21.15.144:56100) 2024-09-10T11:43:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.194+0000 7fa41dd9b700 1 -- 172.21.15.144:0/1302502891 learned_addr learned my addr 172.21.15.144:0/1302502891 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:43:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.194+0000 7fa41e59c700 1 --2- 172.21.15.144:0/1302502891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4201003d0 0x7fa420195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:54.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.194+0000 7fa41dd9b700 1 -- 172.21.15.144:0/1302502891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4201003d0 msgr2=0x7fa420195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:54.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.194+0000 7fa41dd9b700 1 --2- 172.21.15.144:0/1302502891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4201003d0 0x7fa420195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.194+0000 7fa41dd9b700 1 -- 172.21.15.144:0/1302502891 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa410009d00 con 0x7fa420100d70 2024-09-10T11:43:54.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.195+0000 7fa41dd9b700 1 --2- 172.21.15.144:0/1302502891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa420100d70 0x7fa4201962f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa41400c8f0 tx=0x7fa41400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:54.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.195+0000 7fa40f7fe700 1 -- 172.21.15.144:0/1302502891 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4140078a0 con 0x7fa420100d70 2024-09-10T11:43:54.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.195+0000 7fa424a10700 1 -- 172.21.15.144:0/1302502891 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42019aa40 con 0x7fa420100d70 2024-09-10T11:43:54.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.195+0000 7fa40f7fe700 1 -- 172.21.15.144:0/1302502891 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa41400f450 con 0x7fa420100d70 2024-09-10T11:43:54.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.195+0000 7fa40f7fe700 1 -- 172.21.15.144:0/1302502891 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa414018610 con 0x7fa420100d70 2024-09-10T11:43:54.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.195+0000 7fa424a10700 1 -- 172.21.15.144:0/1302502891 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42019af90 con 0x7fa420100d70 2024-09-10T11:43:54.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.196+0000 7fa424a10700 1 -- 172.21.15.144:0/1302502891 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4201086e0 con 0x7fa420100d70 2024-09-10T11:43:54.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.196+0000 7fa40f7fe700 1 -- 172.21.15.144:0/1302502891 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa414007a00 con 0x7fa420100d70 2024-09-10T11:43:54.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.197+0000 7fa40f7fe700 1 --2- 172.21.15.144:0/1302502891 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa40806e200 0x7fa4080706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:54.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.197+0000 7fa40f7fe700 1 -- 172.21.15.144:0/1302502891 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa41408dff0 con 0x7fa420100d70 2024-09-10T11:43:54.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.197+0000 7fa41e59c700 1 --2- 172.21.15.144:0/1302502891 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa40806e200 0x7fa4080706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:54.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.197+0000 7fa41e59c700 1 --2- 172.21.15.144:0/1302502891 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa40806e200 0x7fa4080706b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa410000f80 tx=0x7fa4100060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:54.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.201+0000 7fa40f7fe700 1 -- 172.21.15.144:0/1302502891 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa4140573d0 con 0x7fa420100d70 2024-09-10T11:43:54.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.423+0000 7fa424a10700 1 -- 172.21.15.144:0/1302502891 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa42019b2d0 con 0x7fa40806e200 2024-09-10T11:43:54.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.424+0000 7fa40f7fe700 1 -- 172.21.15.144:0/1302502891 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa42019b2d0 con 0x7fa40806e200 2024-09-10T11:43:54.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.427+0000 7fa424a10700 1 -- 172.21.15.144:0/1302502891 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa40806e200 msgr2=0x7fa4080706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:54.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.427+0000 7fa424a10700 1 --2- 172.21.15.144:0/1302502891 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa40806e200 0x7fa4080706b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa410000f80 tx=0x7fa4100060c0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.427+0000 7fa424a10700 1 -- 172.21.15.144:0/1302502891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa420100d70 msgr2=0x7fa4201962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:54.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.427+0000 7fa424a10700 1 --2- 172.21.15.144:0/1302502891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa420100d70 0x7fa4201962f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa41400c8f0 tx=0x7fa41400cc00 comp rx=0 tx=0).stop 2024-09-10T11:43:54.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.427+0000 7fa424a10700 1 -- 172.21.15.144:0/1302502891 shutdown_connections 2024-09-10T11:43:54.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.427+0000 7fa424a10700 1 --2- 172.21.15.144:0/1302502891 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa40806e200 0x7fa4080706b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.427+0000 7fa424a10700 1 --2- 172.21.15.144:0/1302502891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4201003d0 0x7fa420195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.427+0000 7fa424a10700 1 --2- 172.21.15.144:0/1302502891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa420100d70 0x7fa4201962f0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.427+0000 7fa424a10700 1 -- 172.21.15.144:0/1302502891 >> 172.21.15.144:0/1302502891 conn(0x7fa4200fbc50 msgr2=0x7fa4200fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:54.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.427+0000 7fa424a10700 1 -- 172.21.15.144:0/1302502891 shutdown_connections 2024-09-10T11:43:54.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.427+0000 7fa424a10700 1 -- 172.21.15.144:0/1302502891 wait complete. 2024-09-10T11:43:54.440 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:43:54.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.533+0000 7fb175018700 1 -- 172.21.15.144:0/2623364007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb170100d70 msgr2=0x7fb170104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:54.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.533+0000 7fb175018700 1 --2- 172.21.15.144:0/2623364007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb170100d70 0x7fb170104d40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb158009910 tx=0x7fb158009c20 comp rx=0 tx=0).stop 2024-09-10T11:43:54.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.533+0000 7fb175018700 1 -- 172.21.15.144:0/2623364007 shutdown_connections 2024-09-10T11:43:54.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.533+0000 7fb175018700 1 --2- 172.21.15.144:0/2623364007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb170100d70 0x7fb170104d40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.533+0000 7fb175018700 1 --2- 172.21.15.144:0/2623364007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1701003d0 0x7fb1701007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.533+0000 7fb175018700 1 -- 172.21.15.144:0/2623364007 >> 172.21.15.144:0/2623364007 conn(0x7fb1700fbc50 msgr2=0x7fb1700fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:54.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.533+0000 7fb175018700 1 -- 172.21.15.144:0/2623364007 shutdown_connections 2024-09-10T11:43:54.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.533+0000 7fb175018700 1 -- 172.21.15.144:0/2623364007 wait complete. 2024-09-10T11:43:54.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.534+0000 7fb175018700 1 Processor -- start 2024-09-10T11:43:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.534+0000 7fb175018700 1 -- start start 2024-09-10T11:43:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.534+0000 7fb175018700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1701003d0 0x7fb170195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.534+0000 7fb175018700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb170100d70 0x7fb170196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.534+0000 7fb175018700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb170196a20 con 0x7fb1701003d0 2024-09-10T11:43:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.534+0000 7fb175018700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb17019a740 con 0x7fb170100d70 2024-09-10T11:43:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.534+0000 7fb16ed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1701003d0 0x7fb170195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.534+0000 7fb16ed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1701003d0 0x7fb170195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56116/0 (socket says 172.21.15.144:56116) 2024-09-10T11:43:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.534+0000 7fb16ed9d700 1 -- 172.21.15.144:0/403802480 learned_addr learned my addr 172.21.15.144:0/403802480 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:43:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.535+0000 7fb16e59c700 1 --2- 172.21.15.144:0/403802480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb170100d70 0x7fb170196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.535+0000 7fb16ed9d700 1 -- 172.21.15.144:0/403802480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb170100d70 msgr2=0x7fb170196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.535+0000 7fb16ed9d700 1 --2- 172.21.15.144:0/403802480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb170100d70 0x7fb170196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.535+0000 7fb16ed9d700 1 -- 172.21.15.144:0/403802480 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1580095f0 con 0x7fb1701003d0 2024-09-10T11:43:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.535+0000 7fb16ed9d700 1 --2- 172.21.15.144:0/403802480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1701003d0 0x7fb170195e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb16800ef10 tx=0x7fb16800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.535+0000 7fb167fff700 1 -- 172.21.15.144:0/403802480 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb16800c9f0 con 0x7fb1701003d0 2024-09-10T11:43:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.535+0000 7fb167fff700 1 -- 172.21.15.144:0/403802480 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb16800f040 con 0x7fb1701003d0 2024-09-10T11:43:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.535+0000 7fb175018700 1 -- 172.21.15.144:0/403802480 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb17019a9c0 con 0x7fb1701003d0 2024-09-10T11:43:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.535+0000 7fb167fff700 1 -- 172.21.15.144:0/403802480 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1680146b0 con 0x7fb1701003d0 2024-09-10T11:43:54.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.535+0000 7fb175018700 1 -- 172.21.15.144:0/403802480 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb17019aee0 con 0x7fb1701003d0 2024-09-10T11:43:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.536+0000 7fb175018700 1 -- 172.21.15.144:0/403802480 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb170108740 con 0x7fb1701003d0 2024-09-10T11:43:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.537+0000 7fb167fff700 1 -- 172.21.15.144:0/403802480 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb16800cb50 con 0x7fb1701003d0 2024-09-10T11:43:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.537+0000 7fb167fff700 1 --2- 172.21.15.144:0/403802480 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb15c06e200 0x7fb15c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.537+0000 7fb167fff700 1 -- 172.21.15.144:0/403802480 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb16808e460 con 0x7fb1701003d0 2024-09-10T11:43:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.537+0000 7fb16e59c700 1 --2- 172.21.15.144:0/403802480 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb15c06e200 0x7fb15c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:54.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.538+0000 7fb16e59c700 1 --2- 172.21.15.144:0/403802480 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb15c06e200 0x7fb15c0706b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb158005b40 tx=0x7fb158005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:54.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.542+0000 7fb167fff700 1 -- 172.21.15.144:0/403802480 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb1680577e0 con 0x7fb1701003d0 2024-09-10T11:43:54.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.763+0000 7fb175018700 1 -- 172.21.15.144:0/403802480 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb17019b1c0 con 0x7fb15c06e200 2024-09-10T11:43:54.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.764+0000 7fb167fff700 1 -- 172.21.15.144:0/403802480 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb17019b1c0 con 0x7fb15c06e200 2024-09-10T11:43:54.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.766+0000 7fb175018700 1 -- 172.21.15.144:0/403802480 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb15c06e200 msgr2=0x7fb15c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:54.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.766+0000 7fb175018700 1 --2- 172.21.15.144:0/403802480 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb15c06e200 0x7fb15c0706b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb158005b40 tx=0x7fb158005a90 comp rx=0 tx=0).stop 2024-09-10T11:43:54.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.766+0000 7fb175018700 1 -- 172.21.15.144:0/403802480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1701003d0 msgr2=0x7fb170195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:54.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.766+0000 7fb175018700 1 --2- 172.21.15.144:0/403802480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1701003d0 0x7fb170195e00 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb16800ef10 tx=0x7fb16800c3a0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.767+0000 7fb175018700 1 -- 172.21.15.144:0/403802480 shutdown_connections 2024-09-10T11:43:54.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.767+0000 7fb175018700 1 --2- 172.21.15.144:0/403802480 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb15c06e200 0x7fb15c0706b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.767+0000 7fb175018700 1 --2- 172.21.15.144:0/403802480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1701003d0 0x7fb170195e00 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.767+0000 7fb175018700 1 --2- 172.21.15.144:0/403802480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb170100d70 0x7fb170196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.767+0000 7fb175018700 1 -- 172.21.15.144:0/403802480 >> 172.21.15.144:0/403802480 conn(0x7fb1700fbc50 msgr2=0x7fb1700fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:54.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.767+0000 7fb175018700 1 -- 172.21.15.144:0/403802480 shutdown_connections 2024-09-10T11:43:54.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.767+0000 7fb175018700 1 -- 172.21.15.144:0/403802480 wait complete. 2024-09-10T11:43:54.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.871+0000 7f6812db1700 1 -- 172.21.15.144:0/3464035325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f680c1003d0 msgr2=0x7f680c1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:54.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.871+0000 7f6812db1700 1 --2- 172.21.15.144:0/3464035325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f680c1003d0 0x7f680c1007a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6800009910 tx=0x7f6800009c20 comp rx=0 tx=0).stop 2024-09-10T11:43:54.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.871+0000 7f6812db1700 1 -- 172.21.15.144:0/3464035325 shutdown_connections 2024-09-10T11:43:54.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.871+0000 7f6812db1700 1 --2- 172.21.15.144:0/3464035325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f680c100d70 0x7f680c104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.871+0000 7f6812db1700 1 --2- 172.21.15.144:0/3464035325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f680c1003d0 0x7f680c1007a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.874 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.871+0000 7f6812db1700 1 -- 172.21.15.144:0/3464035325 >> 172.21.15.144:0/3464035325 conn(0x7f680c0fbc90 msgr2=0x7f680c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:54.874 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:54 smithi144 ceph-mon[87436]: pgmap v357: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:54.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.871+0000 7f6812db1700 1 -- 172.21.15.144:0/3464035325 shutdown_connections 2024-09-10T11:43:54.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.871+0000 7f6812db1700 1 -- 172.21.15.144:0/3464035325 wait complete. 2024-09-10T11:43:54.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.872+0000 7f6812db1700 1 Processor -- start 2024-09-10T11:43:54.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.872+0000 7f6812db1700 1 -- start start 2024-09-10T11:43:54.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.872+0000 7f6812db1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f680c1003d0 0x7f680c195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:54.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.872+0000 7f6812db1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f680c100d70 0x7f680c1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:54.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.872+0000 7f6812db1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c1969b0 con 0x7f680c1003d0 2024-09-10T11:43:54.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.872+0000 7f6812db1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f680c19a740 con 0x7f680c100d70 2024-09-10T11:43:54.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.872+0000 7f6810b4d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f680c1003d0 0x7f680c195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:54.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.872+0000 7f6810b4d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f680c1003d0 0x7f680c195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56134/0 (socket says 172.21.15.144:56134) 2024-09-10T11:43:54.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.872+0000 7f6810b4d700 1 -- 172.21.15.144:0/565061210 learned_addr learned my addr 172.21.15.144:0/565061210 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:43:54.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.872+0000 7f680bfff700 1 --2- 172.21.15.144:0/565061210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f680c100d70 0x7f680c1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:54.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.872+0000 7f6810b4d700 1 -- 172.21.15.144:0/565061210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f680c100d70 msgr2=0x7f680c1962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:54.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.872+0000 7f6810b4d700 1 --2- 172.21.15.144:0/565061210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f680c100d70 0x7f680c1962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:54.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.872+0000 7f6810b4d700 1 -- 172.21.15.144:0/565061210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68000095f0 con 0x7f680c1003d0 2024-09-10T11:43:54.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.873+0000 7f6810b4d700 1 --2- 172.21.15.144:0/565061210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f680c1003d0 0x7f680c195d90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6800006010 tx=0x7f680000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:54.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.873+0000 7f6809ffb700 1 -- 172.21.15.144:0/565061210 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f680001d070 con 0x7f680c1003d0 2024-09-10T11:43:54.877 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.873+0000 7f6809ffb700 1 -- 172.21.15.144:0/565061210 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6800005dc0 con 0x7f680c1003d0 2024-09-10T11:43:54.877 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.873+0000 7f6812db1700 1 -- 172.21.15.144:0/565061210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f680c19a9c0 con 0x7f680c1003d0 2024-09-10T11:43:54.877 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.873+0000 7f6809ffb700 1 -- 172.21.15.144:0/565061210 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6800017400 con 0x7f680c1003d0 2024-09-10T11:43:54.877 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.873+0000 7f6812db1700 1 -- 172.21.15.144:0/565061210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f680c19aeb0 con 0x7f680c1003d0 2024-09-10T11:43:54.878 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.874+0000 7f6812db1700 1 -- 172.21.15.144:0/565061210 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f680c1086e0 con 0x7f680c1003d0 2024-09-10T11:43:54.878 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.875+0000 7f6809ffb700 1 -- 172.21.15.144:0/565061210 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6800003a60 con 0x7f680c1003d0 2024-09-10T11:43:54.878 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.875+0000 7f6809ffb700 1 --2- 172.21.15.144:0/565061210 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67fc06e200 0x7f67fc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:54.878 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.875+0000 7f6809ffb700 1 -- 172.21.15.144:0/565061210 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f680008f500 con 0x7f680c1003d0 2024-09-10T11:43:54.878 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.875+0000 7f680bfff700 1 --2- 172.21.15.144:0/565061210 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67fc06e200 0x7f67fc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:54.878 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.876+0000 7f680bfff700 1 --2- 172.21.15.144:0/565061210 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67fc06e200 0x7f67fc0706b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f680c1973b0 tx=0x7f67f800b3c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:54.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:54.880+0000 7f6809ffb700 1 -- 172.21.15.144:0/565061210 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f68000588e0 con 0x7f680c1003d0 2024-09-10T11:43:55.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.087+0000 7f6812db1700 1 -- 172.21.15.144:0/565061210 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f680c19b190 con 0x7f67fc06e200 2024-09-10T11:43:55.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.096+0000 7f6809ffb700 1 -- 172.21.15.144:0/565061210 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f680c19b190 con 0x7f67fc06e200 2024-09-10T11:43:55.098 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:43:55.098 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (12m) 5m ago 22m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:43:55.098 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (22m) 5m ago 22m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:43:55.098 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (21m) 5m ago 21m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:43:55.098 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (9m) 5m ago 22m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:43:55.098 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (9m) 5m ago 21m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:43:55.098 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (11m) 5m ago 22m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:43:55.098 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (6m) 5m ago 17m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:43:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (5m) 5m ago 17m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:43:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (17m) 5m ago 17m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:43:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (5m) 5m ago 17m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:43:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (13m) 5m ago 24m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:43:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (13m) 5m ago 21m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:43:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (10m) 5m ago 24m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:43:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (10m) 5m ago 21m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:43:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (12m) 5m ago 22m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:43:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (12m) 5m ago 21m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:43:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (9m) 5m ago 20m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:43:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (8m) 5m ago 20m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:43:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (8m) 5m ago 19m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:43:55.099 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (7m) 5m ago 19m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:43:55.100 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (7m) 5m ago 19m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:43:55.100 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (6m) 5m ago 18m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:43:55.100 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (12m) 5m ago 22m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:43:55.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.098+0000 7f6812db1700 1 -- 172.21.15.144:0/565061210 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67fc06e200 msgr2=0x7f67fc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:55.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.098+0000 7f6812db1700 1 --2- 172.21.15.144:0/565061210 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67fc06e200 0x7f67fc0706b0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f680c1973b0 tx=0x7f67f800b3c0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.098+0000 7f6812db1700 1 -- 172.21.15.144:0/565061210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f680c1003d0 msgr2=0x7f680c195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.098+0000 7f6812db1700 1 --2- 172.21.15.144:0/565061210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f680c1003d0 0x7f680c195d90 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6800006010 tx=0x7f680000bd70 comp rx=0 tx=0).stop 2024-09-10T11:43:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.098+0000 7f6812db1700 1 -- 172.21.15.144:0/565061210 shutdown_connections 2024-09-10T11:43:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.098+0000 7f6812db1700 1 --2- 172.21.15.144:0/565061210 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67fc06e200 0x7f67fc0706b0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.098+0000 7f6812db1700 1 --2- 172.21.15.144:0/565061210 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f680c1003d0 0x7f680c195d90 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.098+0000 7f6812db1700 1 --2- 172.21.15.144:0/565061210 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f680c100d70 0x7f680c1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.098+0000 7f6812db1700 1 -- 172.21.15.144:0/565061210 >> 172.21.15.144:0/565061210 conn(0x7f680c0fbc90 msgr2=0x7f680c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.098+0000 7f6812db1700 1 -- 172.21.15.144:0/565061210 shutdown_connections 2024-09-10T11:43:55.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.098+0000 7f6812db1700 1 -- 172.21.15.144:0/565061210 wait complete. 2024-09-10T11:43:55.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:54 smithi178 ceph-mon[65940]: pgmap v357: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.201+0000 7f8297306700 1 -- 172.21.15.144:0/1004790359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82900730f0 msgr2=0x7f82900734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.201+0000 7f8297306700 1 --2- 172.21.15.144:0/1004790359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82900730f0 0x7f82900734c0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8288009910 tx=0x7f8288009c20 comp rx=0 tx=0).stop 2024-09-10T11:43:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.201+0000 7f8297306700 1 -- 172.21.15.144:0/1004790359 shutdown_connections 2024-09-10T11:43:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.201+0000 7f8297306700 1 --2- 172.21.15.144:0/1004790359 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8290073a00 0x7f8290110c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.201+0000 7f8297306700 1 --2- 172.21.15.144:0/1004790359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f82900730f0 0x7f82900734c0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.201+0000 7f8297306700 1 -- 172.21.15.144:0/1004790359 >> 172.21.15.144:0/1004790359 conn(0x7f82900fbc90 msgr2=0x7f82900fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:55.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.201+0000 7f8297306700 1 -- 172.21.15.144:0/1004790359 shutdown_connections 2024-09-10T11:43:55.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.201+0000 7f8297306700 1 -- 172.21.15.144:0/1004790359 wait complete. 2024-09-10T11:43:55.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.202+0000 7f8297306700 1 Processor -- start 2024-09-10T11:43:55.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.202+0000 7f8297306700 1 -- start start 2024-09-10T11:43:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.202+0000 7f8297306700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82900730f0 0x7f829019c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.202+0000 7f8297306700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8290073a00 0x7f829019c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.202+0000 7f8297306700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829019cef0 con 0x7f8290073a00 2024-09-10T11:43:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.202+0000 7f8297306700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82901a0c80 con 0x7f82900730f0 2024-09-10T11:43:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.202+0000 7f82948a1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8290073a00 0x7f829019c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.202+0000 7f82948a1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8290073a00 0x7f829019c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56156/0 (socket says 172.21.15.144:56156) 2024-09-10T11:43:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.203+0000 7f82948a1700 1 -- 172.21.15.144:0/1420390459 learned_addr learned my addr 172.21.15.144:0/1420390459 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:43:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.203+0000 7f82950a2700 1 --2- 172.21.15.144:0/1420390459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82900730f0 0x7f829019c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.203+0000 7f82948a1700 1 -- 172.21.15.144:0/1420390459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82900730f0 msgr2=0x7f829019c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:55.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.203+0000 7f82948a1700 1 --2- 172.21.15.144:0/1420390459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82900730f0 0x7f829019c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.203+0000 7f82948a1700 1 -- 172.21.15.144:0/1420390459 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82880095f0 con 0x7f8290073a00 2024-09-10T11:43:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.203+0000 7f82948a1700 1 --2- 172.21.15.144:0/1420390459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8290073a00 0x7f829019c810 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f828400d8d0 tx=0x7f828400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.203+0000 7f828e7fc700 1 -- 172.21.15.144:0/1420390459 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82840098a0 con 0x7f8290073a00 2024-09-10T11:43:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.203+0000 7f828e7fc700 1 -- 172.21.15.144:0/1420390459 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8284010460 con 0x7f8290073a00 2024-09-10T11:43:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.203+0000 7f8297306700 1 -- 172.21.15.144:0/1420390459 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82901a0f60 con 0x7f8290073a00 2024-09-10T11:43:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.203+0000 7f828e7fc700 1 -- 172.21.15.144:0/1420390459 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f828400f5f0 con 0x7f8290073a00 2024-09-10T11:43:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.203+0000 7f8297306700 1 -- 172.21.15.144:0/1420390459 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82901a14b0 con 0x7f8290073a00 2024-09-10T11:43:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.204+0000 7f8297306700 1 -- 172.21.15.144:0/1420390459 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f829010e3c0 con 0x7f8290073a00 2024-09-10T11:43:55.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.205+0000 7f828e7fc700 1 -- 172.21.15.144:0/1420390459 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8284009a00 con 0x7f8290073a00 2024-09-10T11:43:55.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.205+0000 7f828e7fc700 1 --2- 172.21.15.144:0/1420390459 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f827806e200 0x7f82780706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:55.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.205+0000 7f828e7fc700 1 -- 172.21.15.144:0/1420390459 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f828408de70 con 0x7f8290073a00 2024-09-10T11:43:55.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.205+0000 7f82950a2700 1 --2- 172.21.15.144:0/1420390459 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f827806e200 0x7f82780706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:55.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.206+0000 7f82950a2700 1 --2- 172.21.15.144:0/1420390459 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f827806e200 0x7f82780706b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8288006010 tx=0x7f828801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:55.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.210+0000 7f828e7fc700 1 -- 172.21.15.144:0/1420390459 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f82840572d0 con 0x7f8290073a00 2024-09-10T11:43:55.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.493+0000 7f8297306700 1 -- 172.21.15.144:0/1420390459 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f829004ea50 con 0x7f8290073a00 2024-09-10T11:43:55.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.493+0000 7f828e7fc700 1 -- 172.21.15.144:0/1420390459 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f8284016340 con 0x7f8290073a00 2024-09-10T11:43:55.495 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:43:55.495 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:43:55.496 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:43:55.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:43:55.496 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:43:55.496 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:43:55.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:43:55.496 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:43:55.496 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:43:55.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:43:55.496 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:43:55.496 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:43:55.496 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:43:55.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:43:55.496 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:43:55.496 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:43:55.497 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:43:55.497 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:43:55.497 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:43:55.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.495+0000 7f8297306700 1 -- 172.21.15.144:0/1420390459 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f827806e200 msgr2=0x7f82780706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.495+0000 7f8297306700 1 --2- 172.21.15.144:0/1420390459 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f827806e200 0x7f82780706b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8288006010 tx=0x7f828801a040 comp rx=0 tx=0).stop 2024-09-10T11:43:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.495+0000 7f8297306700 1 -- 172.21.15.144:0/1420390459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8290073a00 msgr2=0x7f829019c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.495+0000 7f8297306700 1 --2- 172.21.15.144:0/1420390459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8290073a00 0x7f829019c810 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f828400d8d0 tx=0x7f828400dbe0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.496+0000 7f8297306700 1 -- 172.21.15.144:0/1420390459 shutdown_connections 2024-09-10T11:43:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.496+0000 7f8297306700 1 --2- 172.21.15.144:0/1420390459 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f827806e200 0x7f82780706b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.496+0000 7f8297306700 1 --2- 172.21.15.144:0/1420390459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f82900730f0 0x7f829019c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.496+0000 7f8297306700 1 --2- 172.21.15.144:0/1420390459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8290073a00 0x7f829019c810 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.496+0000 7f8297306700 1 -- 172.21.15.144:0/1420390459 >> 172.21.15.144:0/1420390459 conn(0x7f82900fbc90 msgr2=0x7f8290102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.496+0000 7f8297306700 1 -- 172.21.15.144:0/1420390459 shutdown_connections 2024-09-10T11:43:55.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.496+0000 7f8297306700 1 -- 172.21.15.144:0/1420390459 wait complete. 2024-09-10T11:43:55.602 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.600+0000 7f215b053700 1 -- 172.21.15.144:0/136313408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2154068fd0 msgr2=0x7f2154104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:55.603 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.600+0000 7f215b053700 1 --2- 172.21.15.144:0/136313408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2154068fd0 0x7f2154104df0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f214400b0a0 tx=0x7f214400b3b0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.603 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.600+0000 7f215b053700 1 -- 172.21.15.144:0/136313408 shutdown_connections 2024-09-10T11:43:55.603 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.600+0000 7f215b053700 1 --2- 172.21.15.144:0/136313408 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2154068fd0 0x7f2154104df0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.603 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.600+0000 7f215b053700 1 --2- 172.21.15.144:0/136313408 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2154068630 0x7f2154068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.603 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.600+0000 7f215b053700 1 -- 172.21.15.144:0/136313408 >> 172.21.15.144:0/136313408 conn(0x7f2154074f50 msgr2=0x7f2154075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:55.603 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.601+0000 7f215b053700 1 -- 172.21.15.144:0/136313408 shutdown_connections 2024-09-10T11:43:55.603 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.601+0000 7f215b053700 1 -- 172.21.15.144:0/136313408 wait complete. 2024-09-10T11:43:55.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.601+0000 7f215b053700 1 Processor -- start 2024-09-10T11:43:55.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.601+0000 7f215b053700 1 -- start start 2024-09-10T11:43:55.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.601+0000 7f215b053700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2154068630 0x7f2154195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:55.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.601+0000 7f215b053700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2154068fd0 0x7f2154196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:55.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.601+0000 7f215b053700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2154196a60 con 0x7f2154068630 2024-09-10T11:43:55.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.601+0000 7f215b053700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215419a7f0 con 0x7f2154068fd0 2024-09-10T11:43:55.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.601+0000 7f2158def700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2154068630 0x7f2154195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:55.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.601+0000 7f2158def700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2154068630 0x7f2154195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56170/0 (socket says 172.21.15.144:56170) 2024-09-10T11:43:55.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.602+0000 7f2158def700 1 -- 172.21.15.144:0/1607335323 learned_addr learned my addr 172.21.15.144:0/1607335323 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:43:55.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.602+0000 7f2153fff700 1 --2- 172.21.15.144:0/1607335323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2154068fd0 0x7f2154196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:55.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.602+0000 7f2158def700 1 -- 172.21.15.144:0/1607335323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2154068fd0 msgr2=0x7f2154196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:55.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.602+0000 7f2158def700 1 --2- 172.21.15.144:0/1607335323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2154068fd0 0x7f2154196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.602+0000 7f2158def700 1 -- 172.21.15.144:0/1607335323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2144009d00 con 0x7f2154068630 2024-09-10T11:43:55.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.602+0000 7f2158def700 1 --2- 172.21.15.144:0/1607335323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2154068630 0x7f2154195e40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f214c00d8d0 tx=0x7f214c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:55.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.602+0000 7f2151ffb700 1 -- 172.21.15.144:0/1607335323 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f214c0098a0 con 0x7f2154068630 2024-09-10T11:43:55.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.602+0000 7f2151ffb700 1 -- 172.21.15.144:0/1607335323 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f214c010460 con 0x7f2154068630 2024-09-10T11:43:55.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.602+0000 7f215b053700 1 -- 172.21.15.144:0/1607335323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215419aad0 con 0x7f2154068630 2024-09-10T11:43:55.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.602+0000 7f2151ffb700 1 -- 172.21.15.144:0/1607335323 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f214c00f5f0 con 0x7f2154068630 2024-09-10T11:43:55.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.602+0000 7f215b053700 1 -- 172.21.15.144:0/1607335323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215419aff0 con 0x7f2154068630 2024-09-10T11:43:55.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.603+0000 7f215b053700 1 -- 172.21.15.144:0/1607335323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21541086d0 con 0x7f2154068630 2024-09-10T11:43:55.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.604+0000 7f2151ffb700 1 -- 172.21.15.144:0/1607335323 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f214c009a00 con 0x7f2154068630 2024-09-10T11:43:55.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.604+0000 7f2151ffb700 1 --2- 172.21.15.144:0/1607335323 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f214006e200 0x7f21400706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:55.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.604+0000 7f2151ffb700 1 -- 172.21.15.144:0/1607335323 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f214c08df10 con 0x7f2154068630 2024-09-10T11:43:55.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.604+0000 7f2153fff700 1 --2- 172.21.15.144:0/1607335323 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f214006e200 0x7f21400706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:55.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.605+0000 7f2153fff700 1 --2- 172.21.15.144:0/1607335323 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f214006e200 0x7f21400706b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2144009fd0 tx=0x7f21440060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:55.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.608+0000 7f2151ffb700 1 -- 172.21.15.144:0/1607335323 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f214c0572d0 con 0x7f2154068630 2024-09-10T11:43:55.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.857+0000 7f215b053700 1 -- 172.21.15.144:0/1607335323 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f215419b260 con 0x7f2154068630 2024-09-10T11:43:55.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.858+0000 7f2151ffb700 1 -- 172.21.15.144:0/1607335323 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f214c020790 con 0x7f2154068630 2024-09-10T11:43:55.860 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:43:55.860 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:43:55.860 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:43:55.860 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:43:55.860 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:43:55.860 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:43:55.860 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:43:55.861 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:43:55.861 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:43:55.861 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:43:55.861 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:43:55.861 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:43:55.861 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:43:55.861 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:43:55.861 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:43:55.861 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:43:55.861 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:43:55.861 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:43:55.861 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:43:55.861 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:43:55.862 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:43:55.862 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:43:55.862 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:43:55.862 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:43:55.862 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:43:55.862 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:43:55.862 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:43:55.862 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:43:55.862 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:43:55.862 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:43:55.862 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:43:55.862 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:43:55.862 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:43:55.862 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:43:55.862 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:43:55.863 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:43:55.863 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:43:55.863 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:43:55.863 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:43:55.863 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:43:55.863 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:43:55.863 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:43:55.863 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:43:55.864 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.860+0000 7f215b053700 1 -- 172.21.15.144:0/1607335323 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f214006e200 msgr2=0x7f21400706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:55.864 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.860+0000 7f215b053700 1 --2- 172.21.15.144:0/1607335323 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f214006e200 0x7f21400706b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2144009fd0 tx=0x7f21440060c0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.864 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.860+0000 7f215b053700 1 -- 172.21.15.144:0/1607335323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2154068630 msgr2=0x7f2154195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:55.864 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.860+0000 7f215b053700 1 --2- 172.21.15.144:0/1607335323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2154068630 0x7f2154195e40 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f214c00d8d0 tx=0x7f214c00dbe0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.864 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.860+0000 7f215b053700 1 -- 172.21.15.144:0/1607335323 shutdown_connections 2024-09-10T11:43:55.864 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.860+0000 7f215b053700 1 --2- 172.21.15.144:0/1607335323 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f214006e200 0x7f21400706b0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.864 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.860+0000 7f215b053700 1 --2- 172.21.15.144:0/1607335323 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2154068630 0x7f2154195e40 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.864 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.860+0000 7f215b053700 1 --2- 172.21.15.144:0/1607335323 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2154068fd0 0x7f2154196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.864 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.860+0000 7f215b053700 1 -- 172.21.15.144:0/1607335323 >> 172.21.15.144:0/1607335323 conn(0x7f2154074f50 msgr2=0x7f21540fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:55.865 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.860+0000 7f215b053700 1 -- 172.21.15.144:0/1607335323 shutdown_connections 2024-09-10T11:43:55.865 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.860+0000 7f215b053700 1 -- 172.21.15.144:0/1607335323 wait complete. 2024-09-10T11:43:55.865 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:43:55.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.965+0000 7f4f711a2700 1 -- 172.21.15.144:0/187806923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c0ffa60 msgr2=0x7f4f6c10a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:55.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.965+0000 7f4f711a2700 1 --2- 172.21.15.144:0/187806923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c0ffa60 0x7f4f6c10a4d0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4f60009910 tx=0x7f4f60009c20 comp rx=0 tx=0).stop 2024-09-10T11:43:55.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.965+0000 7f4f711a2700 1 -- 172.21.15.144:0/187806923 shutdown_connections 2024-09-10T11:43:55.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.965+0000 7f4f711a2700 1 --2- 172.21.15.144:0/187806923 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c0ffa60 0x7f4f6c10a4d0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.965+0000 7f4f711a2700 1 --2- 172.21.15.144:0/187806923 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f6c0ff150 0x7f4f6c0ff520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.965+0000 7f4f711a2700 1 -- 172.21.15.144:0/187806923 >> 172.21.15.144:0/187806923 conn(0x7f4f6c074690 msgr2=0x7f4f6c074a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:55.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.965+0000 7f4f711a2700 1 -- 172.21.15.144:0/187806923 shutdown_connections 2024-09-10T11:43:55.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.965+0000 7f4f711a2700 1 -- 172.21.15.144:0/187806923 wait complete. 2024-09-10T11:43:55.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.966+0000 7f4f711a2700 1 Processor -- start 2024-09-10T11:43:55.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.966+0000 7f4f711a2700 1 -- start start 2024-09-10T11:43:55.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.966+0000 7f4f711a2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c0ff150 0x7f4f6c19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:55.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.966+0000 7f4f711a2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f6c0ffa60 0x7f4f6c19bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:55.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.966+0000 7f4f711a2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c19c570 con 0x7f4f6c0ff150 2024-09-10T11:43:55.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.966+0000 7f4f711a2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f6c195a20 con 0x7f4f6c0ffa60 2024-09-10T11:43:55.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.967+0000 7f4f6bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c0ff150 0x7f4f6c19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:55.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.967+0000 7f4f6bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c0ff150 0x7f4f6c19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56192/0 (socket says 172.21.15.144:56192) 2024-09-10T11:43:55.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.967+0000 7f4f6bfff700 1 -- 172.21.15.144:0/2164276069 learned_addr learned my addr 172.21.15.144:0/2164276069 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:43:55.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.967+0000 7f4f6b7fe700 1 --2- 172.21.15.144:0/2164276069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f6c0ffa60 0x7f4f6c19bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:55.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.967+0000 7f4f6bfff700 1 -- 172.21.15.144:0/2164276069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f6c0ffa60 msgr2=0x7f4f6c19bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:55.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.967+0000 7f4f6bfff700 1 --2- 172.21.15.144:0/2164276069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f6c0ffa60 0x7f4f6c19bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:55.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.967+0000 7f4f6bfff700 1 -- 172.21.15.144:0/2164276069 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f600095f0 con 0x7f4f6c0ff150 2024-09-10T11:43:55.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.967+0000 7f4f6bfff700 1 --2- 172.21.15.144:0/2164276069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c0ff150 0x7f4f6c19b9a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00d8d0 tx=0x7f4f5c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:55.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.967+0000 7f4f697fa700 1 -- 172.21.15.144:0/2164276069 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f5c0098a0 con 0x7f4f6c0ff150 2024-09-10T11:43:55.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.967+0000 7f4f711a2700 1 -- 172.21.15.144:0/2164276069 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6c195d00 con 0x7f4f6c0ff150 2024-09-10T11:43:55.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.967+0000 7f4f697fa700 1 -- 172.21.15.144:0/2164276069 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f5c010460 con 0x7f4f6c0ff150 2024-09-10T11:43:55.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.967+0000 7f4f697fa700 1 -- 172.21.15.144:0/2164276069 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f5c00f5f0 con 0x7f4f6c0ff150 2024-09-10T11:43:55.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.967+0000 7f4f711a2700 1 -- 172.21.15.144:0/2164276069 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6c196220 con 0x7f4f6c0ff150 2024-09-10T11:43:55.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.968+0000 7f4f711a2700 1 -- 172.21.15.144:0/2164276069 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f6c107bd0 con 0x7f4f6c0ff150 2024-09-10T11:43:55.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.969+0000 7f4f697fa700 1 -- 172.21.15.144:0/2164276069 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4f5c009a00 con 0x7f4f6c0ff150 2024-09-10T11:43:55.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.969+0000 7f4f697fa700 1 --2- 172.21.15.144:0/2164276069 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4f5406e200 0x7f4f540706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:55.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.969+0000 7f4f6b7fe700 1 --2- 172.21.15.144:0/2164276069 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4f5406e200 0x7f4f540706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:55.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.969+0000 7f4f697fa700 1 -- 172.21.15.144:0/2164276069 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4f5c08df70 con 0x7f4f6c0ff150 2024-09-10T11:43:55.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.970+0000 7f4f6b7fe700 1 --2- 172.21.15.144:0/2164276069 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4f5406e200 0x7f4f540706b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000b5c0 tx=0x7f4f6001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:55.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:55.974+0000 7f4f697fa700 1 -- 172.21.15.144:0/2164276069 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4f5c0c31b0 con 0x7f4f6c0ff150 2024-09-10T11:43:56.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:55 smithi178 ceph-mon[65940]: from='client.34708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:56.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:55 smithi178 ceph-mon[65940]: from='client.34712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:56.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:55 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1420390459' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:43:56.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:55 smithi144 ceph-mon[87436]: from='client.34708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:55 smithi144 ceph-mon[87436]: from='client.34712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:55 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1420390459' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:43:56.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.196+0000 7f4f711a2700 1 -- 172.21.15.144:0/2164276069 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4f6c196500 con 0x7f4f5406e200 2024-09-10T11:43:56.199 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.197+0000 7f4f697fa700 1 -- 172.21.15.144:0/2164276069 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4f6c196500 con 0x7f4f5406e200 2024-09-10T11:43:56.199 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:43:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:43:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:43:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:43:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:43:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:43:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:43:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:43:56.199 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:43:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:43:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:43:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:43:56.200 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:43:56.200 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:43:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.199+0000 7f4f711a2700 1 -- 172.21.15.144:0/2164276069 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4f5406e200 msgr2=0x7f4f540706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.199+0000 7f4f711a2700 1 --2- 172.21.15.144:0/2164276069 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4f5406e200 0x7f4f540706b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4f6000b5c0 tx=0x7f4f6001a040 comp rx=0 tx=0).stop 2024-09-10T11:43:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.199+0000 7f4f711a2700 1 -- 172.21.15.144:0/2164276069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c0ff150 msgr2=0x7f4f6c19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.199+0000 7f4f711a2700 1 --2- 172.21.15.144:0/2164276069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c0ff150 0x7f4f6c19b9a0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4f5c00d8d0 tx=0x7f4f5c00dbe0 comp rx=0 tx=0).stop 2024-09-10T11:43:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.199+0000 7f4f711a2700 1 -- 172.21.15.144:0/2164276069 shutdown_connections 2024-09-10T11:43:56.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.199+0000 7f4f711a2700 1 --2- 172.21.15.144:0/2164276069 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4f5406e200 0x7f4f540706b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:56.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.199+0000 7f4f711a2700 1 --2- 172.21.15.144:0/2164276069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4f6c0ff150 0x7f4f6c19b9a0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:56.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.199+0000 7f4f711a2700 1 --2- 172.21.15.144:0/2164276069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4f6c0ffa60 0x7f4f6c19bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:56.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.199+0000 7f4f711a2700 1 -- 172.21.15.144:0/2164276069 >> 172.21.15.144:0/2164276069 conn(0x7f4f6c074690 msgr2=0x7f4f6c104d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:56.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.199+0000 7f4f711a2700 1 -- 172.21.15.144:0/2164276069 shutdown_connections 2024-09-10T11:43:56.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.199+0000 7f4f711a2700 1 -- 172.21.15.144:0/2164276069 wait complete. 2024-09-10T11:43:56.308 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.303+0000 7fd564987700 1 -- 172.21.15.144:0/3114628428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ff310 msgr2=0x7fd5600ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:56.308 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.303+0000 7fd564987700 1 --2- 172.21.15.144:0/3114628428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ff310 0x7fd5600ff6e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd550009910 tx=0x7fd550009c20 comp rx=0 tx=0).stop 2024-09-10T11:43:56.308 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.304+0000 7fd564987700 1 -- 172.21.15.144:0/3114628428 shutdown_connections 2024-09-10T11:43:56.308 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.304+0000 7fd564987700 1 --2- 172.21.15.144:0/3114628428 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5600ffcb0 0x7fd560103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:56.308 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.304+0000 7fd564987700 1 --2- 172.21.15.144:0/3114628428 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ff310 0x7fd5600ff6e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:56.308 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.304+0000 7fd564987700 1 -- 172.21.15.144:0/3114628428 >> 172.21.15.144:0/3114628428 conn(0x7fd5600fabb0 msgr2=0x7fd5600fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:56.308 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.304+0000 7fd564987700 1 -- 172.21.15.144:0/3114628428 shutdown_connections 2024-09-10T11:43:56.309 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.304+0000 7fd564987700 1 -- 172.21.15.144:0/3114628428 wait complete. 2024-09-10T11:43:56.309 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.304+0000 7fd564987700 1 Processor -- start 2024-09-10T11:43:56.309 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.304+0000 7fd564987700 1 -- start start 2024-09-10T11:43:56.309 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd564987700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ff310 0x7fd560195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:56.309 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd564987700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5600ffcb0 0x7fd560196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:56.309 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd564987700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd560196a70 con 0x7fd5600ff310 2024-09-10T11:43:56.309 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd564987700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56019a800 con 0x7fd5600ffcb0 2024-09-10T11:43:56.309 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd55e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ff310 0x7fd560195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:56.309 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd55e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ff310 0x7fd560195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56218/0 (socket says 172.21.15.144:56218) 2024-09-10T11:43:56.310 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd55e59c700 1 -- 172.21.15.144:0/1768012632 learned_addr learned my addr 172.21.15.144:0/1768012632 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:43:56.310 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd55dd9b700 1 --2- 172.21.15.144:0/1768012632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5600ffcb0 0x7fd560196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:56.310 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd55e59c700 1 -- 172.21.15.144:0/1768012632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5600ffcb0 msgr2=0x7fd560196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:56.310 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd55e59c700 1 --2- 172.21.15.144:0/1768012632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5600ffcb0 0x7fd560196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:56.310 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd55e59c700 1 -- 172.21.15.144:0/1768012632 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5500095f0 con 0x7fd5600ff310 2024-09-10T11:43:56.310 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd55e59c700 1 --2- 172.21.15.144:0/1768012632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ff310 0x7fd560195e50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd550006010 tx=0x7fd55000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:56.310 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd54f7fe700 1 -- 172.21.15.144:0/1768012632 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd55001d070 con 0x7fd5600ff310 2024-09-10T11:43:56.310 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd54f7fe700 1 -- 172.21.15.144:0/1768012632 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd55000f460 con 0x7fd5600ff310 2024-09-10T11:43:56.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd564987700 1 -- 172.21.15.144:0/1768012632 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56019aa80 con 0x7fd5600ff310 2024-09-10T11:43:56.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.305+0000 7fd54f7fe700 1 -- 172.21.15.144:0/1768012632 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd550017610 con 0x7fd5600ff310 2024-09-10T11:43:56.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.306+0000 7fd564987700 1 -- 172.21.15.144:0/1768012632 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56019af70 con 0x7fd5600ff310 2024-09-10T11:43:56.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.306+0000 7fd564987700 1 -- 172.21.15.144:0/1768012632 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56004ea50 con 0x7fd5600ff310 2024-09-10T11:43:56.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.307+0000 7fd54f7fe700 1 -- 172.21.15.144:0/1768012632 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd55000f5d0 con 0x7fd5600ff310 2024-09-10T11:43:56.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.307+0000 7fd54f7fe700 1 --2- 172.21.15.144:0/1768012632 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd54806e200 0x7fd5480706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:43:56.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.308+0000 7fd54f7fe700 1 -- 172.21.15.144:0/1768012632 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd55008f600 con 0x7fd5600ff310 2024-09-10T11:43:56.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.308+0000 7fd55dd9b700 1 --2- 172.21.15.144:0/1768012632 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd54806e200 0x7fd5480706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:43:56.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.308+0000 7fd55dd9b700 1 --2- 172.21.15.144:0/1768012632 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd54806e200 0x7fd5480706b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd560197470 tx=0x7fd554009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:43:56.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.312+0000 7fd54f7fe700 1 -- 172.21.15.144:0/1768012632 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd5500589e0 con 0x7fd5600ff310 2024-09-10T11:43:56.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.594+0000 7fd564987700 1 -- 172.21.15.144:0/1768012632 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd5601971b0 con 0x7fd5600ff310 2024-09-10T11:43:56.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.594+0000 7fd54f7fe700 1 -- 172.21.15.144:0/1768012632 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fd55005c0a0 con 0x7fd5600ff310 2024-09-10T11:43:56.597 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:43:56.597 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:43:56.597 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:43:56.597 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:43:56.597 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:43:56.597 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:43:56.597 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:43:56.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.597+0000 7fd564987700 1 -- 172.21.15.144:0/1768012632 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd54806e200 msgr2=0x7fd5480706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:56.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.597+0000 7fd564987700 1 --2- 172.21.15.144:0/1768012632 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd54806e200 0x7fd5480706b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd560197470 tx=0x7fd554009190 comp rx=0 tx=0).stop 2024-09-10T11:43:56.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.597+0000 7fd564987700 1 -- 172.21.15.144:0/1768012632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ff310 msgr2=0x7fd560195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:43:56.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.597+0000 7fd564987700 1 --2- 172.21.15.144:0/1768012632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ff310 0x7fd560195e50 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd550006010 tx=0x7fd55000bd70 comp rx=0 tx=0).stop 2024-09-10T11:43:56.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.597+0000 7fd564987700 1 -- 172.21.15.144:0/1768012632 shutdown_connections 2024-09-10T11:43:56.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.597+0000 7fd564987700 1 --2- 172.21.15.144:0/1768012632 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd54806e200 0x7fd5480706b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:56.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.597+0000 7fd564987700 1 --2- 172.21.15.144:0/1768012632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ff310 0x7fd560195e50 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:56.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.597+0000 7fd564987700 1 --2- 172.21.15.144:0/1768012632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5600ffcb0 0x7fd560196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:43:56.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.597+0000 7fd564987700 1 -- 172.21.15.144:0/1768012632 >> 172.21.15.144:0/1768012632 conn(0x7fd5600fabb0 msgr2=0x7fd5601033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:43:56.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.597+0000 7fd564987700 1 -- 172.21.15.144:0/1768012632 shutdown_connections 2024-09-10T11:43:56.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:43:56.597+0000 7fd564987700 1 -- 172.21.15.144:0/1768012632 wait complete. 2024-09-10T11:43:56.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:56 smithi144 ceph-mon[87436]: from='client.34716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:56.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:56 smithi144 ceph-mon[87436]: pgmap v358: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:56.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:56 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1607335323' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:43:56.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:56 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1768012632' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:43:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:56 smithi178 ceph-mon[65940]: from='client.34716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:56 smithi178 ceph-mon[65940]: pgmap v358: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:56 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1607335323' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:43:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:56 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1768012632' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:43:57.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:57 smithi144 ceph-mon[87436]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:58.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:57 smithi178 ceph-mon[65940]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:43:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:58 smithi178 ceph-mon[65940]: pgmap v359: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:43:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:43:59.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:58 smithi144 ceph-mon[87436]: pgmap v359: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:43:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:43:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:44:01.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:00 smithi178 ceph-mon[65940]: pgmap v360: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:01.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:00 smithi144 ceph-mon[87436]: pgmap v360: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:02.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:02 smithi144 ceph-mon[87436]: pgmap v361: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:02 smithi178 ceph-mon[65940]: pgmap v361: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:04 smithi178 ceph-mon[65940]: pgmap v362: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:05.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:04 smithi144 ceph-mon[87436]: pgmap v362: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:06 smithi178 ceph-mon[65940]: pgmap v363: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:06 smithi144 ceph-mon[87436]: pgmap v363: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:08 smithi178 ceph-mon[65940]: pgmap v364: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:09.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:08 smithi144 ceph-mon[87436]: pgmap v364: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:10 smithi178 ceph-mon[65940]: pgmap v365: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:11.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:10 smithi144 ceph-mon[87436]: pgmap v365: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:12 smithi144 ceph-mon[87436]: pgmap v366: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:12 smithi178 ceph-mon[65940]: pgmap v366: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:44:14.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:44:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:14 smithi178 ceph-mon[65940]: pgmap v367: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:15.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:14 smithi144 ceph-mon[87436]: pgmap v367: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:16.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:15 smithi178 ceph-mon[65940]: pgmap v368: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:16.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:15 smithi144 ceph-mon[87436]: pgmap v368: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:18 smithi178 ceph-mon[65940]: pgmap v369: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:19.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:18 smithi144 ceph-mon[87436]: pgmap v369: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:20 smithi178 ceph-mon[65940]: pgmap v370: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:21.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:20 smithi144 ceph-mon[87436]: pgmap v370: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:23.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:22 smithi144 ceph-mon[87436]: pgmap v371: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:23.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:22 smithi178 ceph-mon[65940]: pgmap v371: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:24 smithi178 ceph-mon[65940]: pgmap v372: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:25.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:24 smithi144 ceph-mon[87436]: pgmap v372: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:26.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.717+0000 7f472adb5700 1 -- 172.21.15.144:0/3053674505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47240ff390 msgr2=0x7f47240ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:26.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.717+0000 7f472adb5700 1 --2- 172.21.15.144:0/3053674505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47240ff390 0x7f47240ff760 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f471400b0a0 tx=0x7f471400b3b0 comp rx=0 tx=0).stop 2024-09-10T11:44:26.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.717+0000 7f472adb5700 1 -- 172.21.15.144:0/3053674505 shutdown_connections 2024-09-10T11:44:26.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.717+0000 7f472adb5700 1 --2- 172.21.15.144:0/3053674505 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47240ffd30 0x7f4724103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:26.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.717+0000 7f472adb5700 1 --2- 172.21.15.144:0/3053674505 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47240ff390 0x7f47240ff760 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:26.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.717+0000 7f472adb5700 1 -- 172.21.15.144:0/3053674505 >> 172.21.15.144:0/3053674505 conn(0x7f4724074f50 msgr2=0x7f4724075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:26.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.717+0000 7f472adb5700 1 -- 172.21.15.144:0/3053674505 shutdown_connections 2024-09-10T11:44:26.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.717+0000 7f472adb5700 1 -- 172.21.15.144:0/3053674505 wait complete. 2024-09-10T11:44:26.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.718+0000 7f472adb5700 1 Processor -- start 2024-09-10T11:44:26.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.718+0000 7f472adb5700 1 -- start start 2024-09-10T11:44:26.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.718+0000 7f472adb5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47240ff390 0x7f4724100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:26.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.718+0000 7f472adb5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47240ffd30 0x7f47241012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:26.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.718+0000 7f472adb5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4724101800 con 0x7f47240ffd30 2024-09-10T11:44:26.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.718+0000 7f472adb5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4724101940 con 0x7f47240ff390 2024-09-10T11:44:26.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.718+0000 7f4723fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47240ffd30 0x7f47241012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:26.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.718+0000 7f4723fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47240ffd30 0x7f47241012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37204/0 (socket says 172.21.15.144:37204) 2024-09-10T11:44:26.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.718+0000 7f4723fff700 1 -- 172.21.15.144:0/1493900588 learned_addr learned my addr 172.21.15.144:0/1493900588 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:44:26.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.719+0000 7f4728b51700 1 --2- 172.21.15.144:0/1493900588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47240ff390 0x7f4724100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:26.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.719+0000 7f4723fff700 1 -- 172.21.15.144:0/1493900588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47240ff390 msgr2=0x7f4724100d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:26.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.719+0000 7f4723fff700 1 --2- 172.21.15.144:0/1493900588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47240ff390 0x7f4724100d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:26.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.719+0000 7f4723fff700 1 -- 172.21.15.144:0/1493900588 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4714009d00 con 0x7f47240ffd30 2024-09-10T11:44:26.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.719+0000 7f4723fff700 1 --2- 172.21.15.144:0/1493900588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47240ffd30 0x7f47241012c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f470c00d900 tx=0x7f470c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:26.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.719+0000 7f4721ffb700 1 -- 172.21.15.144:0/1493900588 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f470c009900 con 0x7f47240ffd30 2024-09-10T11:44:26.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.719+0000 7f4721ffb700 1 -- 172.21.15.144:0/1493900588 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f470c010460 con 0x7f47240ffd30 2024-09-10T11:44:26.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.719+0000 7f472adb5700 1 -- 172.21.15.144:0/1493900588 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47241a1ef0 con 0x7f47240ffd30 2024-09-10T11:44:26.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.719+0000 7f4721ffb700 1 -- 172.21.15.144:0/1493900588 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f470c00f5f0 con 0x7f47240ffd30 2024-09-10T11:44:26.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.719+0000 7f472adb5700 1 -- 172.21.15.144:0/1493900588 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47241a2300 con 0x7f47240ffd30 2024-09-10T11:44:26.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.720+0000 7f472adb5700 1 -- 172.21.15.144:0/1493900588 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f472404ea50 con 0x7f47240ffd30 2024-09-10T11:44:26.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.721+0000 7f4721ffb700 1 -- 172.21.15.144:0/1493900588 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f470c0105d0 con 0x7f47240ffd30 2024-09-10T11:44:26.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.721+0000 7f4721ffb700 1 --2- 172.21.15.144:0/1493900588 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f471006e200 0x7f47100706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:26.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.721+0000 7f4721ffb700 1 -- 172.21.15.144:0/1493900588 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f470c08de60 con 0x7f47240ffd30 2024-09-10T11:44:26.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.721+0000 7f4728b51700 1 --2- 172.21.15.144:0/1493900588 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f471006e200 0x7f47100706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:26.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.722+0000 7f4728b51700 1 --2- 172.21.15.144:0/1493900588 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f471006e200 0x7f47100706b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f471400b830 tx=0x7f47140061f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:26.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.726+0000 7f4721ffb700 1 -- 172.21.15.144:0/1493900588 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f470c0572c0 con 0x7f47240ffd30 2024-09-10T11:44:26.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.948+0000 7f472adb5700 1 -- 172.21.15.144:0/1493900588 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4724068b60 con 0x7f471006e200 2024-09-10T11:44:26.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.949+0000 7f4721ffb700 1 -- 172.21.15.144:0/1493900588 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4724068b60 con 0x7f471006e200 2024-09-10T11:44:26.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.952+0000 7f472adb5700 1 -- 172.21.15.144:0/1493900588 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f471006e200 msgr2=0x7f47100706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:26.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.952+0000 7f472adb5700 1 --2- 172.21.15.144:0/1493900588 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f471006e200 0x7f47100706b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f471400b830 tx=0x7f47140061f0 comp rx=0 tx=0).stop 2024-09-10T11:44:26.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.952+0000 7f472adb5700 1 -- 172.21.15.144:0/1493900588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47240ffd30 msgr2=0x7f47241012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:26.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.952+0000 7f472adb5700 1 --2- 172.21.15.144:0/1493900588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47240ffd30 0x7f47241012c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f470c00d900 tx=0x7f470c00dcc0 comp rx=0 tx=0).stop 2024-09-10T11:44:26.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.952+0000 7f472adb5700 1 -- 172.21.15.144:0/1493900588 shutdown_connections 2024-09-10T11:44:26.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.952+0000 7f472adb5700 1 --2- 172.21.15.144:0/1493900588 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f471006e200 0x7f47100706b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:26.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.952+0000 7f472adb5700 1 --2- 172.21.15.144:0/1493900588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47240ff390 0x7f4724100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:26.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.952+0000 7f472adb5700 1 --2- 172.21.15.144:0/1493900588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47240ffd30 0x7f47241012c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:26.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.952+0000 7f472adb5700 1 -- 172.21.15.144:0/1493900588 >> 172.21.15.144:0/1493900588 conn(0x7f4724074f50 msgr2=0x7f4724068450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:26.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.952+0000 7f472adb5700 1 -- 172.21.15.144:0/1493900588 shutdown_connections 2024-09-10T11:44:26.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:26.952+0000 7f472adb5700 1 -- 172.21.15.144:0/1493900588 wait complete. 2024-09-10T11:44:26.963 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:44:27.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.060+0000 7f1c12130700 1 -- 172.21.15.144:0/1310825649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c0c0ff2c0 msgr2=0x7f1c0c0ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:27.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.060+0000 7f1c12130700 1 --2- 172.21.15.144:0/1310825649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c0c0ff2c0 0x7f1c0c0ff690 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b0a0 tx=0x7f1bfc00b3b0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.060+0000 7f1c12130700 1 -- 172.21.15.144:0/1310825649 shutdown_connections 2024-09-10T11:44:27.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.060+0000 7f1c12130700 1 --2- 172.21.15.144:0/1310825649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c0c0ffc60 0x7f1c0c103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.060+0000 7f1c12130700 1 --2- 172.21.15.144:0/1310825649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c0c0ff2c0 0x7f1c0c0ff690 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.060+0000 7f1c12130700 1 -- 172.21.15.144:0/1310825649 >> 172.21.15.144:0/1310825649 conn(0x7f1c0c0fab80 msgr2=0x7f1c0c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:27.062 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:26 smithi144 ceph-mon[87436]: pgmap v373: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:27.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.060+0000 7f1c12130700 1 -- 172.21.15.144:0/1310825649 shutdown_connections 2024-09-10T11:44:27.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.060+0000 7f1c12130700 1 -- 172.21.15.144:0/1310825649 wait complete. 2024-09-10T11:44:27.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.060+0000 7f1c12130700 1 Processor -- start 2024-09-10T11:44:27.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.061+0000 7f1c12130700 1 -- start start 2024-09-10T11:44:27.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.061+0000 7f1c12130700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c0c0ff2c0 0x7f1c0c196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:27.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.061+0000 7f1c12130700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c0c0ffc60 0x7f1c0c1971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:27.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.061+0000 7f1c12130700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c1977f0 con 0x7f1c0c0ff2c0 2024-09-10T11:44:27.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.061+0000 7f1c12130700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0c197960 con 0x7f1c0c0ffc60 2024-09-10T11:44:27.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.061+0000 7f1c0b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c0c0ff2c0 0x7f1c0c196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:27.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.061+0000 7f1c0b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c0c0ff2c0 0x7f1c0c196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37222/0 (socket says 172.21.15.144:37222) 2024-09-10T11:44:27.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.061+0000 7f1c0b7fe700 1 -- 172.21.15.144:0/2310975714 learned_addr learned my addr 172.21.15.144:0/2310975714 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:44:27.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.061+0000 7f1c0affd700 1 --2- 172.21.15.144:0/2310975714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c0c0ffc60 0x7f1c0c1971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:27.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.061+0000 7f1c0b7fe700 1 -- 172.21.15.144:0/2310975714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c0c0ffc60 msgr2=0x7f1c0c1971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:27.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.061+0000 7f1c0b7fe700 1 --2- 172.21.15.144:0/2310975714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c0c0ffc60 0x7f1c0c1971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.061+0000 7f1c0b7fe700 1 -- 172.21.15.144:0/2310975714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc009d00 con 0x7f1c0c0ff2c0 2024-09-10T11:44:27.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.061+0000 7f1c0affd700 1 --2- 172.21.15.144:0/2310975714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c0c0ffc60 0x7f1c0c1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:44:27.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.061+0000 7f1c0b7fe700 1 --2- 172.21.15.144:0/2310975714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c0c0ff2c0 0x7f1c0c196c90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b070 tx=0x7f1bfc00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:27.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.062+0000 7f1c08ff9700 1 -- 172.21.15.144:0/2310975714 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bfc00e040 con 0x7f1c0c0ff2c0 2024-09-10T11:44:27.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.062+0000 7f1c08ff9700 1 -- 172.21.15.144:0/2310975714 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bfc003de0 con 0x7f1c0c0ff2c0 2024-09-10T11:44:27.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.062+0000 7f1c12130700 1 -- 172.21.15.144:0/2310975714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0c19fd00 con 0x7f1c0c0ff2c0 2024-09-10T11:44:27.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.062+0000 7f1c08ff9700 1 -- 172.21.15.144:0/2310975714 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bfc01d8e0 con 0x7f1c0c0ff2c0 2024-09-10T11:44:27.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.062+0000 7f1c12130700 1 -- 172.21.15.144:0/2310975714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0c1a0170 con 0x7f1c0c0ff2c0 2024-09-10T11:44:27.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.063+0000 7f1c12130700 1 -- 172.21.15.144:0/2310975714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0c1075d0 con 0x7f1c0c0ff2c0 2024-09-10T11:44:27.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.063+0000 7f1c08ff9700 1 -- 172.21.15.144:0/2310975714 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1bfc004040 con 0x7f1c0c0ff2c0 2024-09-10T11:44:27.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.064+0000 7f1c08ff9700 1 --2- 172.21.15.144:0/2310975714 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1bf406e200 0x7f1bf40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:27.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.064+0000 7f1c0affd700 1 --2- 172.21.15.144:0/2310975714 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1bf406e200 0x7f1bf40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:27.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.064+0000 7f1c08ff9700 1 -- 172.21.15.144:0/2310975714 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1bfc08f8d0 con 0x7f1c0c0ff2c0 2024-09-10T11:44:27.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.064+0000 7f1c0affd700 1 --2- 172.21.15.144:0/2310975714 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1bf406e200 0x7f1bf40706b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c069660 tx=0x7f1c000092a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:27.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.068+0000 7f1c08ff9700 1 -- 172.21.15.144:0/2310975714 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1bfc058cb0 con 0x7f1c0c0ff2c0 2024-09-10T11:44:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:26 smithi178 ceph-mon[65940]: pgmap v373: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:27.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.287+0000 7f1c12130700 1 -- 172.21.15.144:0/2310975714 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c0c061190 con 0x7f1bf406e200 2024-09-10T11:44:27.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.288+0000 7f1c08ff9700 1 -- 172.21.15.144:0/2310975714 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1c0c061190 con 0x7f1bf406e200 2024-09-10T11:44:27.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.290+0000 7f1c12130700 1 -- 172.21.15.144:0/2310975714 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1bf406e200 msgr2=0x7f1bf40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:27.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.290+0000 7f1c12130700 1 --2- 172.21.15.144:0/2310975714 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1bf406e200 0x7f1bf40706b0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f1c0c069660 tx=0x7f1c000092a0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.290+0000 7f1c12130700 1 -- 172.21.15.144:0/2310975714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c0c0ff2c0 msgr2=0x7f1c0c196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:27.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.290+0000 7f1c12130700 1 --2- 172.21.15.144:0/2310975714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c0c0ff2c0 0x7f1c0c196c90 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc00b070 tx=0x7f1bfc00cdd0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.291+0000 7f1c12130700 1 -- 172.21.15.144:0/2310975714 shutdown_connections 2024-09-10T11:44:27.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.291+0000 7f1c12130700 1 --2- 172.21.15.144:0/2310975714 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1bf406e200 0x7f1bf40706b0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.291+0000 7f1c12130700 1 --2- 172.21.15.144:0/2310975714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c0c0ff2c0 0x7f1c0c196c90 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.291+0000 7f1c12130700 1 --2- 172.21.15.144:0/2310975714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c0c0ffc60 0x7f1c0c1971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.291+0000 7f1c12130700 1 -- 172.21.15.144:0/2310975714 >> 172.21.15.144:0/2310975714 conn(0x7f1c0c0fab80 msgr2=0x7f1c0c0fc1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:27.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.291+0000 7f1c12130700 1 -- 172.21.15.144:0/2310975714 shutdown_connections 2024-09-10T11:44:27.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.291+0000 7f1c12130700 1 -- 172.21.15.144:0/2310975714 wait complete. 2024-09-10T11:44:27.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.397+0000 7f5d34e2b700 1 -- 172.21.15.144:0/1820092395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff290 msgr2=0x7f5d300ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:27.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.397+0000 7f5d34e2b700 1 --2- 172.21.15.144:0/1820092395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff290 0x7f5d300ff660 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5d24009910 tx=0x7f5d24009c20 comp rx=0 tx=0).stop 2024-09-10T11:44:27.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.397+0000 7f5d34e2b700 1 -- 172.21.15.144:0/1820092395 shutdown_connections 2024-09-10T11:44:27.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.397+0000 7f5d34e2b700 1 --2- 172.21.15.144:0/1820092395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d300ffba0 0x7f5d3010a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.397+0000 7f5d34e2b700 1 --2- 172.21.15.144:0/1820092395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff290 0x7f5d300ff660 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.397+0000 7f5d34e2b700 1 -- 172.21.15.144:0/1820092395 >> 172.21.15.144:0/1820092395 conn(0x7f5d30074850 msgr2=0x7f5d30074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:27.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.397+0000 7f5d34e2b700 1 -- 172.21.15.144:0/1820092395 shutdown_connections 2024-09-10T11:44:27.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.397+0000 7f5d34e2b700 1 -- 172.21.15.144:0/1820092395 wait complete. 2024-09-10T11:44:27.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.398+0000 7f5d34e2b700 1 Processor -- start 2024-09-10T11:44:27.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.398+0000 7f5d34e2b700 1 -- start start 2024-09-10T11:44:27.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.398+0000 7f5d34e2b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff290 0x7f5d30195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:27.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.398+0000 7f5d34e2b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d300ffba0 0x7f5d301962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:27.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.398+0000 7f5d34e2b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d30196990 con 0x7f5d300ff290 2024-09-10T11:44:27.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.398+0000 7f5d34e2b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d3019a720 con 0x7f5d300ffba0 2024-09-10T11:44:27.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.398+0000 7f5d2e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff290 0x7f5d30195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:27.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.398+0000 7f5d2e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff290 0x7f5d30195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37252/0 (socket says 172.21.15.144:37252) 2024-09-10T11:44:27.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.398+0000 7f5d2e59c700 1 -- 172.21.15.144:0/2608387006 learned_addr learned my addr 172.21.15.144:0/2608387006 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:44:27.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.398+0000 7f5d2dd9b700 1 --2- 172.21.15.144:0/2608387006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d300ffba0 0x7f5d301962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:27.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.398+0000 7f5d2e59c700 1 -- 172.21.15.144:0/2608387006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d300ffba0 msgr2=0x7f5d301962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:27.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.398+0000 7f5d2e59c700 1 --2- 172.21.15.144:0/2608387006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d300ffba0 0x7f5d301962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.399+0000 7f5d2e59c700 1 -- 172.21.15.144:0/2608387006 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d240095f0 con 0x7f5d300ff290 2024-09-10T11:44:27.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.399+0000 7f5d2dd9b700 1 --2- 172.21.15.144:0/2608387006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d300ffba0 0x7f5d301962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:44:27.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.399+0000 7f5d2e59c700 1 --2- 172.21.15.144:0/2608387006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff290 0x7f5d30195d70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5d240092e0 tx=0x7f5d2400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:27.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.399+0000 7f5d1f7fe700 1 -- 172.21.15.144:0/2608387006 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d2401d070 con 0x7f5d300ff290 2024-09-10T11:44:27.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.399+0000 7f5d1f7fe700 1 -- 172.21.15.144:0/2608387006 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5d24005dc0 con 0x7f5d300ff290 2024-09-10T11:44:27.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.399+0000 7f5d34e2b700 1 -- 172.21.15.144:0/2608387006 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d3019a9a0 con 0x7f5d300ff290 2024-09-10T11:44:27.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.399+0000 7f5d1f7fe700 1 -- 172.21.15.144:0/2608387006 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d24017400 con 0x7f5d300ff290 2024-09-10T11:44:27.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.399+0000 7f5d34e2b700 1 -- 172.21.15.144:0/2608387006 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d3019ae90 con 0x7f5d300ff290 2024-09-10T11:44:27.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.400+0000 7f5d34e2b700 1 -- 172.21.15.144:0/2608387006 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d30107d10 con 0x7f5d300ff290 2024-09-10T11:44:27.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.401+0000 7f5d1f7fe700 1 -- 172.21.15.144:0/2608387006 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5d2400f460 con 0x7f5d300ff290 2024-09-10T11:44:27.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.401+0000 7f5d1f7fe700 1 --2- 172.21.15.144:0/2608387006 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5d1806e200 0x7f5d180706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:27.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.401+0000 7f5d1f7fe700 1 -- 172.21.15.144:0/2608387006 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5d2408f590 con 0x7f5d300ff290 2024-09-10T11:44:27.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.401+0000 7f5d2dd9b700 1 --2- 172.21.15.144:0/2608387006 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5d1806e200 0x7f5d180706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:27.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.402+0000 7f5d2dd9b700 1 --2- 172.21.15.144:0/2608387006 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5d1806e200 0x7f5d180706b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5d20005fd0 tx=0x7f5d20005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:27.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.406+0000 7f5d1f7fe700 1 -- 172.21.15.144:0/2608387006 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5d24058950 con 0x7f5d300ff290 2024-09-10T11:44:27.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.615+0000 7f5d34e2b700 1 -- 172.21.15.144:0/2608387006 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d3002cd20 con 0x7f5d1806e200 2024-09-10T11:44:27.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.624+0000 7f5d1f7fe700 1 -- 172.21.15.144:0/2608387006 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f5d3002cd20 con 0x7f5d1806e200 2024-09-10T11:44:27.626 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:44:27.626 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (12m) 6m ago 23m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:44:27.626 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (23m) 6m ago 23m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:44:27.626 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (22m) 6m ago 22m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:44:27.626 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (10m) 6m ago 23m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:44:27.626 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (10m) 6m ago 22m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:44:27.626 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (12m) 6m ago 22m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:44:27.626 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (6m) 6m ago 17m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:44:27.626 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (6m) 6m ago 17m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:44:27.626 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (17m) 6m ago 17m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:44:27.626 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (6m) 6m ago 17m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:44:27.626 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (14m) 6m ago 24m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:44:27.627 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (13m) 6m ago 22m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:44:27.627 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (11m) 6m ago 24m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:44:27.627 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (10m) 6m ago 22m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:44:27.627 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (13m) 6m ago 23m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:44:27.627 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (13m) 6m ago 22m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:44:27.627 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (9m) 6m ago 21m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:44:27.627 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (9m) 6m ago 20m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:44:27.627 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (8m) 6m ago 20m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:44:27.627 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (8m) 6m ago 20m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:44:27.627 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (7m) 6m ago 19m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:44:27.627 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (7m) 6m ago 19m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:44:27.627 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (12m) 6m ago 22m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:44:27.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.626+0000 7f5d34e2b700 1 -- 172.21.15.144:0/2608387006 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5d1806e200 msgr2=0x7f5d180706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:27.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.626+0000 7f5d34e2b700 1 --2- 172.21.15.144:0/2608387006 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5d1806e200 0x7f5d180706b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f5d20005fd0 tx=0x7f5d20005ee0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.626+0000 7f5d34e2b700 1 -- 172.21.15.144:0/2608387006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff290 msgr2=0x7f5d30195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:27.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.626+0000 7f5d34e2b700 1 --2- 172.21.15.144:0/2608387006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff290 0x7f5d30195d70 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f5d240092e0 tx=0x7f5d2400be20 comp rx=0 tx=0).stop 2024-09-10T11:44:27.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.627+0000 7f5d34e2b700 1 -- 172.21.15.144:0/2608387006 shutdown_connections 2024-09-10T11:44:27.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.627+0000 7f5d34e2b700 1 --2- 172.21.15.144:0/2608387006 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5d1806e200 0x7f5d180706b0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.627+0000 7f5d34e2b700 1 --2- 172.21.15.144:0/2608387006 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5d300ff290 0x7f5d30195d70 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.627+0000 7f5d34e2b700 1 --2- 172.21.15.144:0/2608387006 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5d300ffba0 0x7f5d301962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.627+0000 7f5d34e2b700 1 -- 172.21.15.144:0/2608387006 >> 172.21.15.144:0/2608387006 conn(0x7f5d30074850 msgr2=0x7f5d30104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:27.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.627+0000 7f5d34e2b700 1 -- 172.21.15.144:0/2608387006 shutdown_connections 2024-09-10T11:44:27.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.627+0000 7f5d34e2b700 1 -- 172.21.15.144:0/2608387006 wait complete. 2024-09-10T11:44:27.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.731+0000 7f87a1725700 1 -- 172.21.15.144:0/1054924967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f879c100d50 msgr2=0x7f879c104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:27.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.731+0000 7f87a1725700 1 --2- 172.21.15.144:0/1054924967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f879c100d50 0x7f879c104d20 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f8784009910 tx=0x7f8784009c20 comp rx=0 tx=0).stop 2024-09-10T11:44:27.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.731+0000 7f87a1725700 1 -- 172.21.15.144:0/1054924967 shutdown_connections 2024-09-10T11:44:27.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.731+0000 7f87a1725700 1 --2- 172.21.15.144:0/1054924967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f879c100d50 0x7f879c104d20 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.731+0000 7f87a1725700 1 --2- 172.21.15.144:0/1054924967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f879c1003b0 0x7f879c100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.731+0000 7f87a1725700 1 -- 172.21.15.144:0/1054924967 >> 172.21.15.144:0/1054924967 conn(0x7f879c0fbc30 msgr2=0x7f879c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:27.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.731+0000 7f87a1725700 1 -- 172.21.15.144:0/1054924967 shutdown_connections 2024-09-10T11:44:27.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.731+0000 7f87a1725700 1 -- 172.21.15.144:0/1054924967 wait complete. 2024-09-10T11:44:27.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.732+0000 7f87a1725700 1 Processor -- start 2024-09-10T11:44:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.732+0000 7f87a1725700 1 -- start start 2024-09-10T11:44:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.732+0000 7f87a1725700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f879c1003b0 0x7f879c195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.732+0000 7f87a1725700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f879c100d50 0x7f879c196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.732+0000 7f87a1725700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c1969e0 con 0x7f879c1003b0 2024-09-10T11:44:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.732+0000 7f87a1725700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c19a770 con 0x7f879c100d50 2024-09-10T11:44:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.732+0000 7f879affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f879c1003b0 0x7f879c195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.732+0000 7f879affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f879c1003b0 0x7f879c195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37266/0 (socket says 172.21.15.144:37266) 2024-09-10T11:44:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.732+0000 7f879affd700 1 -- 172.21.15.144:0/1725703236 learned_addr learned my addr 172.21.15.144:0/1725703236 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:44:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.732+0000 7f8793fff700 1 --2- 172.21.15.144:0/1725703236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f879c100d50 0x7f879c196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.733+0000 7f879affd700 1 -- 172.21.15.144:0/1725703236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f879c100d50 msgr2=0x7f879c196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.733+0000 7f879affd700 1 --2- 172.21.15.144:0/1725703236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f879c100d50 0x7f879c196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.733+0000 7f879affd700 1 -- 172.21.15.144:0/1725703236 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87840095f0 con 0x7f879c1003b0 2024-09-10T11:44:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.733+0000 7f8793fff700 1 --2- 172.21.15.144:0/1725703236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f879c100d50 0x7f879c196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:44:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.733+0000 7f879affd700 1 --2- 172.21.15.144:0/1725703236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f879c1003b0 0x7f879c195dc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f878c00e920 tx=0x7f878c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.733+0000 7f8798ff9700 1 -- 172.21.15.144:0/1725703236 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f878c009790 con 0x7f879c1003b0 2024-09-10T11:44:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.733+0000 7f87a1725700 1 -- 172.21.15.144:0/1725703236 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879c19aa50 con 0x7f879c1003b0 2024-09-10T11:44:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.733+0000 7f8798ff9700 1 -- 172.21.15.144:0/1725703236 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f878c00cba0 con 0x7f879c1003b0 2024-09-10T11:44:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.733+0000 7f8798ff9700 1 -- 172.21.15.144:0/1725703236 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f878c018650 con 0x7f879c1003b0 2024-09-10T11:44:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.733+0000 7f87a1725700 1 -- 172.21.15.144:0/1725703236 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879c19afa0 con 0x7f879c1003b0 2024-09-10T11:44:27.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.734+0000 7f87a1725700 1 -- 172.21.15.144:0/1725703236 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879c108720 con 0x7f879c1003b0 2024-09-10T11:44:27.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.734+0000 7f8798ff9700 1 -- 172.21.15.144:0/1725703236 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f878c00cd10 con 0x7f879c1003b0 2024-09-10T11:44:27.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.735+0000 7f8798ff9700 1 --2- 172.21.15.144:0/1725703236 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f878806e200 0x7f87880706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:27.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.735+0000 7f8798ff9700 1 -- 172.21.15.144:0/1725703236 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f878c014070 con 0x7f879c1003b0 2024-09-10T11:44:27.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.735+0000 7f8793fff700 1 --2- 172.21.15.144:0/1725703236 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f878806e200 0x7f87880706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:27.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.735+0000 7f8793fff700 1 --2- 172.21.15.144:0/1725703236 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f878806e200 0x7f87880706b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8784005b40 tx=0x7f8784005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:27.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:27.739+0000 7f8798ff9700 1 -- 172.21.15.144:0/1725703236 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f878c0541b0 con 0x7f879c1003b0 2024-09-10T11:44:28.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.026+0000 7f87a1725700 1 -- 172.21.15.144:0/1725703236 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f879c19b1c0 con 0x7f879c1003b0 2024-09-10T11:44:28.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.027+0000 7f8798ff9700 1 -- 172.21.15.144:0/1725703236 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f878c07b070 con 0x7f879c1003b0 2024-09-10T11:44:28.028 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:44:28.028 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:44:28.028 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:44:28.028 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:44:28.029 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:44:28.029 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:44:28.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:44:28.029 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:44:28.029 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:44:28.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:44:28.029 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:44:28.029 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:44:28.029 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:44:28.029 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:44:28.029 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:44:28.029 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:44:28.029 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:44:28.029 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:44:28.030 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:44:28.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.029+0000 7f87a1725700 1 -- 172.21.15.144:0/1725703236 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f878806e200 msgr2=0x7f87880706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:28.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.029+0000 7f87a1725700 1 --2- 172.21.15.144:0/1725703236 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f878806e200 0x7f87880706b0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8784005b40 tx=0x7f8784005a90 comp rx=0 tx=0).stop 2024-09-10T11:44:28.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.029+0000 7f87a1725700 1 -- 172.21.15.144:0/1725703236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f879c1003b0 msgr2=0x7f879c195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:28.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.029+0000 7f87a1725700 1 --2- 172.21.15.144:0/1725703236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f879c1003b0 0x7f879c195dc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f878c00e920 tx=0x7f878c00ece0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.029+0000 7f87a1725700 1 -- 172.21.15.144:0/1725703236 shutdown_connections 2024-09-10T11:44:28.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.029+0000 7f87a1725700 1 --2- 172.21.15.144:0/1725703236 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f878806e200 0x7f87880706b0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.029+0000 7f87a1725700 1 --2- 172.21.15.144:0/1725703236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f879c1003b0 0x7f879c195dc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.029+0000 7f87a1725700 1 --2- 172.21.15.144:0/1725703236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f879c100d50 0x7f879c196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.029+0000 7f87a1725700 1 -- 172.21.15.144:0/1725703236 >> 172.21.15.144:0/1725703236 conn(0x7f879c0fbc30 msgr2=0x7f879c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:28.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.030+0000 7f87a1725700 1 -- 172.21.15.144:0/1725703236 shutdown_connections 2024-09-10T11:44:28.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.030+0000 7f87a1725700 1 -- 172.21.15.144:0/1725703236 wait complete. 2024-09-10T11:44:28.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.133+0000 7f67edf87700 1 -- 172.21.15.144:0/1630876601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e8101310 msgr2=0x7f67e81016e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:28.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.133+0000 7f67edf87700 1 --2- 172.21.15.144:0/1630876601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e8101310 0x7f67e81016e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f67dc009910 tx=0x7f67dc009c20 comp rx=0 tx=0).stop 2024-09-10T11:44:28.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.133+0000 7f67edf87700 1 -- 172.21.15.144:0/1630876601 shutdown_connections 2024-09-10T11:44:28.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.133+0000 7f67edf87700 1 --2- 172.21.15.144:0/1630876601 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67e8101c20 0x7f67e810a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.133+0000 7f67edf87700 1 --2- 172.21.15.144:0/1630876601 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e8101310 0x7f67e81016e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.133+0000 7f67edf87700 1 -- 172.21.15.144:0/1630876601 >> 172.21.15.144:0/1630876601 conn(0x7f67e80fab50 msgr2=0x7f67e80fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:28.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.134+0000 7f67edf87700 1 -- 172.21.15.144:0/1630876601 shutdown_connections 2024-09-10T11:44:28.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.134+0000 7f67edf87700 1 -- 172.21.15.144:0/1630876601 wait complete. 2024-09-10T11:44:28.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.134+0000 7f67edf87700 1 Processor -- start 2024-09-10T11:44:28.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.134+0000 7f67edf87700 1 -- start start 2024-09-10T11:44:28.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.134+0000 7f67edf87700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e8101310 0x7f67e819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:28.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.134+0000 7f67edf87700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67e8101c20 0x7f67e819c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:28.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.135+0000 7f67edf87700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e819c690 con 0x7f67e8101310 2024-09-10T11:44:28.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.135+0000 7f67edf87700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e8195b90 con 0x7f67e8101c20 2024-09-10T11:44:28.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.135+0000 7f67e77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e8101310 0x7f67e819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:28.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.135+0000 7f67e77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e8101310 0x7f67e819bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37288/0 (socket says 172.21.15.144:37288) 2024-09-10T11:44:28.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.135+0000 7f67e77fe700 1 -- 172.21.15.144:0/579194370 learned_addr learned my addr 172.21.15.144:0/579194370 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:44:28.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.135+0000 7f67e6ffd700 1 --2- 172.21.15.144:0/579194370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67e8101c20 0x7f67e819c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:28.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.135+0000 7f67e77fe700 1 -- 172.21.15.144:0/579194370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67e8101c20 msgr2=0x7f67e819c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:28.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.135+0000 7f67e77fe700 1 --2- 172.21.15.144:0/579194370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67e8101c20 0x7f67e819c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.135+0000 7f67e77fe700 1 -- 172.21.15.144:0/579194370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc0095f0 con 0x7f67e8101310 2024-09-10T11:44:28.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.135+0000 7f67e77fe700 1 --2- 172.21.15.144:0/579194370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e8101310 0x7f67e819bac0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f67dc006010 tx=0x7f67dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:28.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.135+0000 7f67e4ff9700 1 -- 172.21.15.144:0/579194370 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67dc01d070 con 0x7f67e8101310 2024-09-10T11:44:28.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.135+0000 7f67e4ff9700 1 -- 172.21.15.144:0/579194370 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67dc00f460 con 0x7f67e8101310 2024-09-10T11:44:28.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.135+0000 7f67e4ff9700 1 -- 172.21.15.144:0/579194370 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67dc017630 con 0x7f67e8101310 2024-09-10T11:44:28.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.135+0000 7f67edf87700 1 -- 172.21.15.144:0/579194370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e8195e10 con 0x7f67e8101310 2024-09-10T11:44:28.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.135+0000 7f67edf87700 1 -- 172.21.15.144:0/579194370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e8196300 con 0x7f67e8101310 2024-09-10T11:44:28.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.136+0000 7f67edf87700 1 -- 172.21.15.144:0/579194370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e81077f0 con 0x7f67e8101310 2024-09-10T11:44:28.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.137+0000 7f67e4ff9700 1 -- 172.21.15.144:0/579194370 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f67dc00f5d0 con 0x7f67e8101310 2024-09-10T11:44:28.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.137+0000 7f67e4ff9700 1 --2- 172.21.15.144:0/579194370 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67d006e200 0x7f67d00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:28.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.137+0000 7f67e4ff9700 1 -- 172.21.15.144:0/579194370 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f67dc08f4d0 con 0x7f67e8101310 2024-09-10T11:44:28.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.137+0000 7f67e6ffd700 1 --2- 172.21.15.144:0/579194370 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67d006e200 0x7f67d00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:28.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.138+0000 7f67e6ffd700 1 --2- 172.21.15.144:0/579194370 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67d006e200 0x7f67d00706b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f67d8005950 tx=0x7f67d80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:28.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.142+0000 7f67e4ff9700 1 -- 172.21.15.144:0/579194370 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f67dc05c460 con 0x7f67e8101310 2024-09-10T11:44:28.392 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.391+0000 7f67edf87700 1 -- 172.21.15.144:0/579194370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f67e804ea50 con 0x7f67e8101310 2024-09-10T11:44:28.392 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.391+0000 7f67e4ff9700 1 -- 172.21.15.144:0/579194370 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f67dc05bff0 con 0x7f67e8101310 2024-09-10T11:44:28.392 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:44:28.393 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:44:28.393 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:44:28.393 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:44:28.393 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:44:28.393 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:44:28.393 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:44:28.393 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:44:28.393 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:44:28.393 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:44:28.393 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:44:28.394 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:44:28.394 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:44:28.394 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:44:28.394 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:44:28.394 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:44:28.394 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:44:28.394 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:44:28.394 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:44:28.394 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:44:28.394 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:44:28.394 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:44:28.395 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:44:28.395 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:44:28.395 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:44:28.395 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:44:28.395 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:44:28.395 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:44:28.395 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:44:28.395 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:44:28.395 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:44:28.396 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:44:28.396 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:44:28.396 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:44:28.396 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:44:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:44:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:44:28.396 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:44:28.396 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:44:28.396 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:44:28.396 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:44:28.397 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:44:28.397 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:44:28.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.393+0000 7f67edf87700 1 -- 172.21.15.144:0/579194370 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67d006e200 msgr2=0x7f67d00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:28.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.393+0000 7f67edf87700 1 --2- 172.21.15.144:0/579194370 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67d006e200 0x7f67d00706b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f67d8005950 tx=0x7f67d80058e0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.393+0000 7f67edf87700 1 -- 172.21.15.144:0/579194370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e8101310 msgr2=0x7f67e819bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:28.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.393+0000 7f67edf87700 1 --2- 172.21.15.144:0/579194370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e8101310 0x7f67e819bac0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f67dc006010 tx=0x7f67dc00be20 comp rx=0 tx=0).stop 2024-09-10T11:44:28.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.393+0000 7f67edf87700 1 -- 172.21.15.144:0/579194370 shutdown_connections 2024-09-10T11:44:28.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.393+0000 7f67edf87700 1 --2- 172.21.15.144:0/579194370 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67d006e200 0x7f67d00706b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.393+0000 7f67edf87700 1 --2- 172.21.15.144:0/579194370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67e8101310 0x7f67e819bac0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.393+0000 7f67edf87700 1 --2- 172.21.15.144:0/579194370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67e8101c20 0x7f67e819c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.393+0000 7f67edf87700 1 -- 172.21.15.144:0/579194370 >> 172.21.15.144:0/579194370 conn(0x7f67e80fab50 msgr2=0x7f67e8104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:28.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.393+0000 7f67edf87700 1 -- 172.21.15.144:0/579194370 shutdown_connections 2024-09-10T11:44:28.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.393+0000 7f67edf87700 1 -- 172.21.15.144:0/579194370 wait complete. 2024-09-10T11:44:28.399 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:44:28.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.499+0000 7fd09299f700 1 -- 172.21.15.144:0/477292512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd08c0fefa0 msgr2=0x7fd08c0ff370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:28.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.499+0000 7fd09299f700 1 --2- 172.21.15.144:0/477292512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd08c0fefa0 0x7fd08c0ff370 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd088009910 tx=0x7fd088009c20 comp rx=0 tx=0).stop 2024-09-10T11:44:28.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.499+0000 7fd09299f700 1 -- 172.21.15.144:0/477292512 shutdown_connections 2024-09-10T11:44:28.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.499+0000 7fd09299f700 1 --2- 172.21.15.144:0/477292512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08c0ff8b0 0x7fd08c103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.499+0000 7fd09299f700 1 --2- 172.21.15.144:0/477292512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd08c0fefa0 0x7fd08c0ff370 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.499+0000 7fd09299f700 1 -- 172.21.15.144:0/477292512 >> 172.21.15.144:0/477292512 conn(0x7fd08c0faa10 msgr2=0x7fd08c0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:28.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.499+0000 7fd09299f700 1 -- 172.21.15.144:0/477292512 shutdown_connections 2024-09-10T11:44:28.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.499+0000 7fd09299f700 1 -- 172.21.15.144:0/477292512 wait complete. 2024-09-10T11:44:28.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.500+0000 7fd09299f700 1 Processor -- start 2024-09-10T11:44:28.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.500+0000 7fd09299f700 1 -- start start 2024-09-10T11:44:28.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.500+0000 7fd09299f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd08c0fefa0 0x7fd08c19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:28.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.500+0000 7fd09299f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08c0ff8b0 0x7fd08c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:28.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.500+0000 7fd09299f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c19cdb0 con 0x7fd08c0fefa0 2024-09-10T11:44:28.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.500+0000 7fd09299f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd08c1a0b40 con 0x7fd08c0ff8b0 2024-09-10T11:44:28.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.500+0000 7fd09199d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd08c0fefa0 0x7fd08c19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:28.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.500+0000 7fd09199d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd08c0fefa0 0x7fd08c19c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37306/0 (socket says 172.21.15.144:37306) 2024-09-10T11:44:28.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.500+0000 7fd09199d700 1 -- 172.21.15.144:0/2146148396 learned_addr learned my addr 172.21.15.144:0/2146148396 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:44:28.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.500+0000 7fd09119c700 1 --2- 172.21.15.144:0/2146148396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08c0ff8b0 0x7fd08c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:28.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.500+0000 7fd09199d700 1 -- 172.21.15.144:0/2146148396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08c0ff8b0 msgr2=0x7fd08c19c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:28.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.500+0000 7fd09199d700 1 --2- 172.21.15.144:0/2146148396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08c0ff8b0 0x7fd08c19c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.500+0000 7fd09199d700 1 -- 172.21.15.144:0/2146148396 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0880095f0 con 0x7fd08c0fefa0 2024-09-10T11:44:28.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.500+0000 7fd09119c700 1 --2- 172.21.15.144:0/2146148396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08c0ff8b0 0x7fd08c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:44:28.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.501+0000 7fd09199d700 1 --2- 172.21.15.144:0/2146148396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd08c0fefa0 0x7fd08c19c190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd088006010 tx=0x7fd08800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:28.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.501+0000 7fd082ffd700 1 -- 172.21.15.144:0/2146148396 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd08801d070 con 0x7fd08c0fefa0 2024-09-10T11:44:28.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.501+0000 7fd082ffd700 1 -- 172.21.15.144:0/2146148396 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd088004fb0 con 0x7fd08c0fefa0 2024-09-10T11:44:28.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.501+0000 7fd09299f700 1 -- 172.21.15.144:0/2146148396 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd08c1a0dc0 con 0x7fd08c0fefa0 2024-09-10T11:44:28.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.501+0000 7fd082ffd700 1 -- 172.21.15.144:0/2146148396 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd088022ea0 con 0x7fd08c0fefa0 2024-09-10T11:44:28.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.501+0000 7fd09299f700 1 -- 172.21.15.144:0/2146148396 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd08c1a12b0 con 0x7fd08c0fefa0 2024-09-10T11:44:28.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.502+0000 7fd09299f700 1 -- 172.21.15.144:0/2146148396 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd08c1014c0 con 0x7fd08c0fefa0 2024-09-10T11:44:28.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.502+0000 7fd082ffd700 1 -- 172.21.15.144:0/2146148396 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd088022470 con 0x7fd08c0fefa0 2024-09-10T11:44:28.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.503+0000 7fd082ffd700 1 --2- 172.21.15.144:0/2146148396 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd07806e200 0x7fd0780706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:28.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.503+0000 7fd082ffd700 1 -- 172.21.15.144:0/2146148396 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd08808f510 con 0x7fd08c0fefa0 2024-09-10T11:44:28.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.503+0000 7fd09119c700 1 --2- 172.21.15.144:0/2146148396 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd07806e200 0x7fd0780706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:28.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.503+0000 7fd09119c700 1 --2- 172.21.15.144:0/2146148396 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd07806e200 0x7fd0780706b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd07c005fd0 tx=0x7fd07c005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:28.508 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.507+0000 7fd082ffd700 1 -- 172.21.15.144:0/2146148396 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd088058b40 con 0x7fd08c0fefa0 2024-09-10T11:44:28.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.729+0000 7fd09299f700 1 -- 172.21.15.144:0/2146148396 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd08c1a1650 con 0x7fd07806e200 2024-09-10T11:44:28.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.731+0000 7fd082ffd700 1 -- 172.21.15.144:0/2146148396 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd08c1a1650 con 0x7fd07806e200 2024-09-10T11:44:28.732 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:44:28.732 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:44:28.733 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:44:28.733 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:44:28.733 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:44:28.734 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:44:28.734 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:44:28.734 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:44:28.734 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:44:28.734 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:44:28.734 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:44:28.734 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:44:28.735 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:44:28.735 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:44:28.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.733+0000 7fd09299f700 1 -- 172.21.15.144:0/2146148396 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd07806e200 msgr2=0x7fd0780706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:28.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.733+0000 7fd09299f700 1 --2- 172.21.15.144:0/2146148396 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd07806e200 0x7fd0780706b0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd07c005fd0 tx=0x7fd07c005dc0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.733+0000 7fd09299f700 1 -- 172.21.15.144:0/2146148396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd08c0fefa0 msgr2=0x7fd08c19c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:28.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.733+0000 7fd09299f700 1 --2- 172.21.15.144:0/2146148396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd08c0fefa0 0x7fd08c19c190 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd088006010 tx=0x7fd08800be20 comp rx=0 tx=0).stop 2024-09-10T11:44:28.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.733+0000 7fd09299f700 1 -- 172.21.15.144:0/2146148396 shutdown_connections 2024-09-10T11:44:28.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.733+0000 7fd09299f700 1 --2- 172.21.15.144:0/2146148396 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd07806e200 0x7fd0780706b0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.733+0000 7fd09299f700 1 --2- 172.21.15.144:0/2146148396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd08c0fefa0 0x7fd08c19c190 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.733+0000 7fd09299f700 1 --2- 172.21.15.144:0/2146148396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd08c0ff8b0 0x7fd08c19c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.733+0000 7fd09299f700 1 -- 172.21.15.144:0/2146148396 >> 172.21.15.144:0/2146148396 conn(0x7fd08c0faa10 msgr2=0x7fd08c110410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:28.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.733+0000 7fd09299f700 1 -- 172.21.15.144:0/2146148396 shutdown_connections 2024-09-10T11:44:28.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.733+0000 7fd09299f700 1 -- 172.21.15.144:0/2146148396 wait complete. 2024-09-10T11:44:28.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.839+0000 7fea6081a700 1 -- 172.21.15.144:0/4270138459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea5c0ff2d0 msgr2=0x7fea5c0ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:28.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.839+0000 7fea6081a700 1 --2- 172.21.15.144:0/4270138459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea5c0ff2d0 0x7fea5c0ff6a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fea50009910 tx=0x7fea50009c20 comp rx=0 tx=0).stop 2024-09-10T11:44:28.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.839+0000 7fea6081a700 1 -- 172.21.15.144:0/4270138459 shutdown_connections 2024-09-10T11:44:28.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.839+0000 7fea6081a700 1 --2- 172.21.15.144:0/4270138459 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c0ffc70 0x7fea5c103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.839+0000 7fea6081a700 1 --2- 172.21.15.144:0/4270138459 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea5c0ff2d0 0x7fea5c0ff6a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.839+0000 7fea6081a700 1 -- 172.21.15.144:0/4270138459 >> 172.21.15.144:0/4270138459 conn(0x7fea5c0fabb0 msgr2=0x7fea5c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:28.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.839+0000 7fea6081a700 1 -- 172.21.15.144:0/4270138459 shutdown_connections 2024-09-10T11:44:28.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.839+0000 7fea6081a700 1 -- 172.21.15.144:0/4270138459 wait complete. 2024-09-10T11:44:28.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.840+0000 7fea6081a700 1 Processor -- start 2024-09-10T11:44:28.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.840+0000 7fea6081a700 1 -- start start 2024-09-10T11:44:28.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.840+0000 7fea6081a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c0ff2d0 0x7fea5c195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:28.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.840+0000 7fea6081a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea5c0ffc70 0x7fea5c196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:28.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.840+0000 7fea6081a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c1969e0 con 0x7fea5c0ffc70 2024-09-10T11:44:28.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.840+0000 7fea6081a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5c19a770 con 0x7fea5c0ff2d0 2024-09-10T11:44:28.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.840+0000 7fea59d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea5c0ffc70 0x7fea5c196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:28.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.840+0000 7fea59d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea5c0ffc70 0x7fea5c196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37324/0 (socket says 172.21.15.144:37324) 2024-09-10T11:44:28.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.841+0000 7fea59d9b700 1 -- 172.21.15.144:0/2282418249 learned_addr learned my addr 172.21.15.144:0/2282418249 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:44:28.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.841+0000 7fea5a59c700 1 --2- 172.21.15.144:0/2282418249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c0ff2d0 0x7fea5c195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:28.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.841+0000 7fea59d9b700 1 -- 172.21.15.144:0/2282418249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c0ff2d0 msgr2=0x7fea5c195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:28.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.841+0000 7fea59d9b700 1 --2- 172.21.15.144:0/2282418249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c0ff2d0 0x7fea5c195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:28.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.841+0000 7fea59d9b700 1 -- 172.21.15.144:0/2282418249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea500095f0 con 0x7fea5c0ffc70 2024-09-10T11:44:28.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.841+0000 7fea59d9b700 1 --2- 172.21.15.144:0/2282418249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea5c0ffc70 0x7fea5c196300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00e970 tx=0x7fea4c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:28.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.841+0000 7fea4b7fe700 1 -- 172.21.15.144:0/2282418249 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea4c00c9d0 con 0x7fea5c0ffc70 2024-09-10T11:44:28.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.841+0000 7fea4b7fe700 1 -- 172.21.15.144:0/2282418249 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fea4c004830 con 0x7fea5c0ffc70 2024-09-10T11:44:28.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.841+0000 7fea6081a700 1 -- 172.21.15.144:0/2282418249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5c19aa50 con 0x7fea5c0ffc70 2024-09-10T11:44:28.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.841+0000 7fea4b7fe700 1 -- 172.21.15.144:0/2282418249 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea4c005660 con 0x7fea5c0ffc70 2024-09-10T11:44:28.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.841+0000 7fea6081a700 1 -- 172.21.15.144:0/2282418249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5c19afa0 con 0x7fea5c0ffc70 2024-09-10T11:44:28.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.842+0000 7fea6081a700 1 -- 172.21.15.144:0/2282418249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea5c1085f0 con 0x7fea5c0ffc70 2024-09-10T11:44:28.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.843+0000 7fea4b7fe700 1 -- 172.21.15.144:0/2282418249 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fea4c00cb30 con 0x7fea5c0ffc70 2024-09-10T11:44:28.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.843+0000 7fea4b7fe700 1 --2- 172.21.15.144:0/2282418249 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fea4406e200 0x7fea440706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:28.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.843+0000 7fea4b7fe700 1 -- 172.21.15.144:0/2282418249 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fea4c014070 con 0x7fea5c0ffc70 2024-09-10T11:44:28.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.843+0000 7fea5a59c700 1 --2- 172.21.15.144:0/2282418249 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fea4406e200 0x7fea440706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:28.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.844+0000 7fea5a59c700 1 --2- 172.21.15.144:0/2282418249 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fea4406e200 0x7fea440706b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fea500059d0 tx=0x7fea50005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:28.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:28.848+0000 7fea4b7fe700 1 -- 172.21.15.144:0/2282418249 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fea4c057720 con 0x7fea5c0ffc70 2024-09-10T11:44:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:28 smithi178 ceph-mon[65940]: from='client.34736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:44:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:28 smithi178 ceph-mon[65940]: from='client.34740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:44:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:28 smithi178 ceph-mon[65940]: from='client.34744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:44:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:28 smithi178 ceph-mon[65940]: pgmap v374: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:28 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1725703236' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:44:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:28 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/579194370' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:44:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:44:29.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:29.131+0000 7fea6081a700 1 -- 172.21.15.144:0/2282418249 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fea5c19b2d0 con 0x7fea5c0ffc70 2024-09-10T11:44:29.132 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:28 smithi144 ceph-mon[87436]: from='client.34736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:44:29.132 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:28 smithi144 ceph-mon[87436]: from='client.34740 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:44:29.132 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:28 smithi144 ceph-mon[87436]: from='client.34744 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:44:29.133 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:28 smithi144 ceph-mon[87436]: pgmap v374: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:29.133 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:28 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1725703236' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:44:29.133 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:28 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/579194370' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:44:29.133 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:44:29.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:29.131+0000 7fea4b7fe700 1 -- 172.21.15.144:0/2282418249 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fea4c05ade0 con 0x7fea5c0ffc70 2024-09-10T11:44:29.133 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:44:29.133 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:44:29.134 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:44:29.134 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:44:29.134 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:44:29.134 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:44:29.134 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:44:29.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:29.133+0000 7fea6081a700 1 -- 172.21.15.144:0/2282418249 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fea4406e200 msgr2=0x7fea440706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:29.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:29.133+0000 7fea6081a700 1 --2- 172.21.15.144:0/2282418249 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fea4406e200 0x7fea440706b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fea500059d0 tx=0x7fea50005940 comp rx=0 tx=0).stop 2024-09-10T11:44:29.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:29.133+0000 7fea6081a700 1 -- 172.21.15.144:0/2282418249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea5c0ffc70 msgr2=0x7fea5c196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:29.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:29.133+0000 7fea6081a700 1 --2- 172.21.15.144:0/2282418249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea5c0ffc70 0x7fea5c196300 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fea4c00e970 tx=0x7fea4c00ec80 comp rx=0 tx=0).stop 2024-09-10T11:44:29.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:29.133+0000 7fea6081a700 1 -- 172.21.15.144:0/2282418249 shutdown_connections 2024-09-10T11:44:29.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:29.133+0000 7fea6081a700 1 --2- 172.21.15.144:0/2282418249 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fea4406e200 0x7fea440706b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:29.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:29.133+0000 7fea6081a700 1 --2- 172.21.15.144:0/2282418249 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fea5c0ff2d0 0x7fea5c195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:29.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:29.133+0000 7fea6081a700 1 --2- 172.21.15.144:0/2282418249 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fea5c0ffc70 0x7fea5c196300 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:29.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:29.134+0000 7fea6081a700 1 -- 172.21.15.144:0/2282418249 >> 172.21.15.144:0/2282418249 conn(0x7fea5c0fabb0 msgr2=0x7fea5c068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:29.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:29.134+0000 7fea6081a700 1 -- 172.21.15.144:0/2282418249 shutdown_connections 2024-09-10T11:44:29.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:29.134+0000 7fea6081a700 1 -- 172.21.15.144:0/2282418249 wait complete. 2024-09-10T11:44:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:29 smithi178 ceph-mon[65940]: from='client.34756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:44:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:29 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2282418249' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:44:30.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:29 smithi144 ceph-mon[87436]: from='client.34756 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:44:30.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:29 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2282418249' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:44:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:30 smithi178 ceph-mon[65940]: pgmap v375: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:31.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:30 smithi144 ceph-mon[87436]: pgmap v375: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:33.116 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:32 smithi144 ceph-mon[87436]: pgmap v376: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:32 smithi178 ceph-mon[65940]: pgmap v376: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:34 smithi178 ceph-mon[65940]: pgmap v377: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:35.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:34 smithi144 ceph-mon[87436]: pgmap v377: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:36 smithi178 ceph-mon[65940]: pgmap v378: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:36 smithi144 ceph-mon[87436]: pgmap v378: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:38 smithi178 ceph-mon[65940]: pgmap v379: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:38 smithi144 ceph-mon[87436]: pgmap v379: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:40 smithi178 ceph-mon[65940]: pgmap v380: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:41.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:40 smithi144 ceph-mon[87436]: pgmap v380: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:43.069 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:42 smithi144 ceph-mon[87436]: pgmap v381: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:42 smithi178 ceph-mon[65940]: pgmap v381: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:44:44.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:44:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:44 smithi178 ceph-mon[65940]: pgmap v382: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:44 smithi144 ceph-mon[87436]: pgmap v382: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:46 smithi178 ceph-mon[65940]: pgmap v383: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:47.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:46 smithi144 ceph-mon[87436]: pgmap v383: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:48 smithi178 ceph-mon[65940]: pgmap v384: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:49.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:48 smithi144 ceph-mon[87436]: pgmap v384: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:50 smithi178 ceph-mon[65940]: pgmap v385: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:51.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:50 smithi144 ceph-mon[87436]: pgmap v385: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:52.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:51 smithi178 ceph-mon[65940]: pgmap v386: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:52.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:51 smithi144 ceph-mon[87436]: pgmap v386: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:54 smithi178 ceph-mon[65940]: pgmap v387: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:55.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:54 smithi144 ceph-mon[87436]: pgmap v387: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:56 smithi178 ceph-mon[65940]: pgmap v388: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:57.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:56 smithi144 ceph-mon[87436]: pgmap v388: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:58 smithi178 ceph-mon[65940]: pgmap v389: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:44:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:44:59.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:58 smithi144 ceph-mon[87436]: pgmap v389: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:44:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:44:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:44:59.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.252+0000 7fe26b7e9700 1 -- 172.21.15.144:0/955349002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2640ffd30 msgr2=0x7fe264103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:59.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.252+0000 7fe26b7e9700 1 --2- 172.21.15.144:0/955349002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2640ffd30 0x7fe264103ce0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe258009910 tx=0x7fe258009c20 comp rx=0 tx=0).stop 2024-09-10T11:44:59.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.252+0000 7fe26b7e9700 1 -- 172.21.15.144:0/955349002 shutdown_connections 2024-09-10T11:44:59.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.252+0000 7fe26b7e9700 1 --2- 172.21.15.144:0/955349002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2640ffd30 0x7fe264103ce0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.252+0000 7fe26b7e9700 1 --2- 172.21.15.144:0/955349002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2640ff390 0x7fe2640ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.252+0000 7fe26b7e9700 1 -- 172.21.15.144:0/955349002 >> 172.21.15.144:0/955349002 conn(0x7fe264074f50 msgr2=0x7fe264075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:59.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.252+0000 7fe26b7e9700 1 -- 172.21.15.144:0/955349002 shutdown_connections 2024-09-10T11:44:59.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.252+0000 7fe26b7e9700 1 -- 172.21.15.144:0/955349002 wait complete. 2024-09-10T11:44:59.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.253+0000 7fe26b7e9700 1 Processor -- start 2024-09-10T11:44:59.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.253+0000 7fe26b7e9700 1 -- start start 2024-09-10T11:44:59.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.253+0000 7fe26b7e9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2640ff390 0x7fe264100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:59.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.253+0000 7fe26b7e9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2640ffd30 0x7fe2641012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:59.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.253+0000 7fe26b7e9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe264101800 con 0x7fe2640ffd30 2024-09-10T11:44:59.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.253+0000 7fe26b7e9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe264101940 con 0x7fe2640ff390 2024-09-10T11:44:59.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.253+0000 7fe268d84700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2640ffd30 0x7fe2641012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:59.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.253+0000 7fe268d84700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2640ffd30 0x7fe2641012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59110/0 (socket says 172.21.15.144:59110) 2024-09-10T11:44:59.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.253+0000 7fe268d84700 1 -- 172.21.15.144:0/4051096762 learned_addr learned my addr 172.21.15.144:0/4051096762 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:44:59.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.254+0000 7fe269585700 1 --2- 172.21.15.144:0/4051096762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2640ff390 0x7fe264100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:59.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.254+0000 7fe268d84700 1 -- 172.21.15.144:0/4051096762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2640ff390 msgr2=0x7fe264100d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:59.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.254+0000 7fe268d84700 1 --2- 172.21.15.144:0/4051096762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2640ff390 0x7fe264100d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.254+0000 7fe268d84700 1 -- 172.21.15.144:0/4051096762 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2580095f0 con 0x7fe2640ffd30 2024-09-10T11:44:59.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.254+0000 7fe269585700 1 --2- 172.21.15.144:0/4051096762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2640ff390 0x7fe264100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:44:59.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.254+0000 7fe268d84700 1 --2- 172.21.15.144:0/4051096762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2640ffd30 0x7fe2641012c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe2580092e0 tx=0x7fe25800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:59.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.254+0000 7fe2567fc700 1 -- 172.21.15.144:0/4051096762 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe25801d070 con 0x7fe2640ffd30 2024-09-10T11:44:59.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.254+0000 7fe2567fc700 1 -- 172.21.15.144:0/4051096762 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe25800f460 con 0x7fe2640ffd30 2024-09-10T11:44:59.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.254+0000 7fe26b7e9700 1 -- 172.21.15.144:0/4051096762 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe264101bc0 con 0x7fe2640ffd30 2024-09-10T11:44:59.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.254+0000 7fe2567fc700 1 -- 172.21.15.144:0/4051096762 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2580175f0 con 0x7fe2640ffd30 2024-09-10T11:44:59.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.254+0000 7fe26b7e9700 1 -- 172.21.15.144:0/4051096762 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2641a2290 con 0x7fe2640ffd30 2024-09-10T11:44:59.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.255+0000 7fe26b7e9700 1 -- 172.21.15.144:0/4051096762 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe26404ea50 con 0x7fe2640ffd30 2024-09-10T11:44:59.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.256+0000 7fe2567fc700 1 -- 172.21.15.144:0/4051096762 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe25800f5d0 con 0x7fe2640ffd30 2024-09-10T11:44:59.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.256+0000 7fe2567fc700 1 --2- 172.21.15.144:0/4051096762 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe25006e200 0x7fe2500706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:59.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.256+0000 7fe269585700 1 --2- 172.21.15.144:0/4051096762 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe25006e200 0x7fe2500706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:59.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.256+0000 7fe2567fc700 1 -- 172.21.15.144:0/4051096762 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe25808f4d0 con 0x7fe2640ffd30 2024-09-10T11:44:59.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.257+0000 7fe269585700 1 --2- 172.21.15.144:0/4051096762 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe25006e200 0x7fe2500706b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe260005fd0 tx=0x7fe260005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:59.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.261+0000 7fe2567fc700 1 -- 172.21.15.144:0/4051096762 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe258058930 con 0x7fe2640ffd30 2024-09-10T11:44:59.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.486+0000 7fe26b7e9700 1 -- 172.21.15.144:0/4051096762 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2640fcc20 con 0x7fe25006e200 2024-09-10T11:44:59.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.487+0000 7fe2567fc700 1 -- 172.21.15.144:0/4051096762 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe2640fcc20 con 0x7fe25006e200 2024-09-10T11:44:59.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.489+0000 7fe26b7e9700 1 -- 172.21.15.144:0/4051096762 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe25006e200 msgr2=0x7fe2500706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:59.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.489+0000 7fe26b7e9700 1 --2- 172.21.15.144:0/4051096762 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe25006e200 0x7fe2500706b0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe260005fd0 tx=0x7fe260005dc0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.489+0000 7fe26b7e9700 1 -- 172.21.15.144:0/4051096762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2640ffd30 msgr2=0x7fe2641012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:59.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.489+0000 7fe26b7e9700 1 --2- 172.21.15.144:0/4051096762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2640ffd30 0x7fe2641012c0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe2580092e0 tx=0x7fe25800be20 comp rx=0 tx=0).stop 2024-09-10T11:44:59.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.489+0000 7fe26b7e9700 1 -- 172.21.15.144:0/4051096762 shutdown_connections 2024-09-10T11:44:59.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.489+0000 7fe26b7e9700 1 --2- 172.21.15.144:0/4051096762 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe25006e200 0x7fe2500706b0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.489+0000 7fe26b7e9700 1 --2- 172.21.15.144:0/4051096762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2640ff390 0x7fe264100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.489+0000 7fe26b7e9700 1 --2- 172.21.15.144:0/4051096762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2640ffd30 0x7fe2641012c0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.489+0000 7fe26b7e9700 1 -- 172.21.15.144:0/4051096762 >> 172.21.15.144:0/4051096762 conn(0x7fe264074f50 msgr2=0x7fe264103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:59.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.490+0000 7fe26b7e9700 1 -- 172.21.15.144:0/4051096762 shutdown_connections 2024-09-10T11:44:59.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.490+0000 7fe26b7e9700 1 -- 172.21.15.144:0/4051096762 wait complete. 2024-09-10T11:44:59.501 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:44:59.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.595+0000 7f67c6644700 1 -- 172.21.15.144:0/2089928189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67c00ffd30 msgr2=0x7f67c0103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:59.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.595+0000 7f67c6644700 1 --2- 172.21.15.144:0/2089928189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67c00ffd30 0x7f67c0103ce0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f67b4009910 tx=0x7f67b4009c20 comp rx=0 tx=0).stop 2024-09-10T11:44:59.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.595+0000 7f67c6644700 1 -- 172.21.15.144:0/2089928189 shutdown_connections 2024-09-10T11:44:59.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.595+0000 7f67c6644700 1 --2- 172.21.15.144:0/2089928189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67c00ffd30 0x7f67c0103ce0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.595+0000 7f67c6644700 1 --2- 172.21.15.144:0/2089928189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67c00ff390 0x7f67c00ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.595+0000 7f67c6644700 1 -- 172.21.15.144:0/2089928189 >> 172.21.15.144:0/2089928189 conn(0x7f67c0074f50 msgr2=0x7f67c0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:59.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.595+0000 7f67c6644700 1 -- 172.21.15.144:0/2089928189 shutdown_connections 2024-09-10T11:44:59.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.595+0000 7f67c6644700 1 -- 172.21.15.144:0/2089928189 wait complete. 2024-09-10T11:44:59.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.596+0000 7f67c6644700 1 Processor -- start 2024-09-10T11:44:59.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.596+0000 7f67c6644700 1 -- start start 2024-09-10T11:44:59.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.596+0000 7f67c6644700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67c00ff390 0x7f67c0195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:59.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.596+0000 7f67c6644700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67c00ffd30 0x7f67c01962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:59.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.596+0000 7f67c6644700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c01969b0 con 0x7f67c00ff390 2024-09-10T11:44:59.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.596+0000 7f67c6644700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c019a740 con 0x7f67c00ffd30 2024-09-10T11:44:59.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.596+0000 7f67bffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67c00ff390 0x7f67c0195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:59.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.596+0000 7f67bffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67c00ff390 0x7f67c0195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59116/0 (socket says 172.21.15.144:59116) 2024-09-10T11:44:59.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.596+0000 7f67bffff700 1 -- 172.21.15.144:0/2223403189 learned_addr learned my addr 172.21.15.144:0/2223403189 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:44:59.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.597+0000 7f67bf7fe700 1 --2- 172.21.15.144:0/2223403189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67c00ffd30 0x7f67c01962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:59.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.597+0000 7f67bffff700 1 -- 172.21.15.144:0/2223403189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67c00ffd30 msgr2=0x7f67c01962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:59.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.597+0000 7f67bffff700 1 --2- 172.21.15.144:0/2223403189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67c00ffd30 0x7f67c01962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.597+0000 7f67bffff700 1 -- 172.21.15.144:0/2223403189 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67b40095f0 con 0x7f67c00ff390 2024-09-10T11:44:59.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.597+0000 7f67bffff700 1 --2- 172.21.15.144:0/2223403189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67c00ff390 0x7f67c0195d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f67b000d8d0 tx=0x7f67b000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:59.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.597+0000 7f67bd7fa700 1 -- 172.21.15.144:0/2223403189 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67b0009940 con 0x7f67c00ff390 2024-09-10T11:44:59.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.597+0000 7f67c6644700 1 -- 172.21.15.144:0/2223403189 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67c019aa20 con 0x7f67c00ff390 2024-09-10T11:44:59.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.597+0000 7f67bd7fa700 1 -- 172.21.15.144:0/2223403189 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f67b0010460 con 0x7f67c00ff390 2024-09-10T11:44:59.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.597+0000 7f67bd7fa700 1 -- 172.21.15.144:0/2223403189 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67b000f5f0 con 0x7f67c00ff390 2024-09-10T11:44:59.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.597+0000 7f67c6644700 1 -- 172.21.15.144:0/2223403189 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67c019af70 con 0x7f67c00ff390 2024-09-10T11:44:59.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.598+0000 7f67c6644700 1 -- 172.21.15.144:0/2223403189 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67c004ea50 con 0x7f67c00ff390 2024-09-10T11:44:59.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.599+0000 7f67bd7fa700 1 -- 172.21.15.144:0/2223403189 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f67b00105d0 con 0x7f67c00ff390 2024-09-10T11:44:59.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.599+0000 7f67bd7fa700 1 --2- 172.21.15.144:0/2223403189 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67a806e200 0x7f67a80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:59.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.599+0000 7f67bd7fa700 1 -- 172.21.15.144:0/2223403189 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f67b008e070 con 0x7f67c00ff390 2024-09-10T11:44:59.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.599+0000 7f67bf7fe700 1 --2- 172.21.15.144:0/2223403189 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67a806e200 0x7f67a80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:59.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.600+0000 7f67bf7fe700 1 --2- 172.21.15.144:0/2223403189 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67a806e200 0x7f67a80706b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f67c01973b0 tx=0x7f67b401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:59.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.603+0000 7f67bd7fa700 1 -- 172.21.15.144:0/2223403189 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f67b0057450 con 0x7f67c00ff390 2024-09-10T11:44:59.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.825+0000 7f67c6644700 1 -- 172.21.15.144:0/2223403189 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f67c019b350 con 0x7f67a806e200 2024-09-10T11:44:59.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.826+0000 7f67bd7fa700 1 -- 172.21.15.144:0/2223403189 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f67c019b350 con 0x7f67a806e200 2024-09-10T11:44:59.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.828+0000 7f67c6644700 1 -- 172.21.15.144:0/2223403189 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67a806e200 msgr2=0x7f67a80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:59.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.828+0000 7f67c6644700 1 --2- 172.21.15.144:0/2223403189 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67a806e200 0x7f67a80706b0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f67c01973b0 tx=0x7f67b401a040 comp rx=0 tx=0).stop 2024-09-10T11:44:59.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.828+0000 7f67c6644700 1 -- 172.21.15.144:0/2223403189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67c00ff390 msgr2=0x7f67c0195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:59.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.828+0000 7f67c6644700 1 --2- 172.21.15.144:0/2223403189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67c00ff390 0x7f67c0195d90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f67b000d8d0 tx=0x7f67b000dc90 comp rx=0 tx=0).stop 2024-09-10T11:44:59.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.829+0000 7f67c6644700 1 -- 172.21.15.144:0/2223403189 shutdown_connections 2024-09-10T11:44:59.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.829+0000 7f67c6644700 1 --2- 172.21.15.144:0/2223403189 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f67a806e200 0x7f67a80706b0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.829+0000 7f67c6644700 1 --2- 172.21.15.144:0/2223403189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f67c00ff390 0x7f67c0195d90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.829+0000 7f67c6644700 1 --2- 172.21.15.144:0/2223403189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f67c00ffd30 0x7f67c01962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.829+0000 7f67c6644700 1 -- 172.21.15.144:0/2223403189 >> 172.21.15.144:0/2223403189 conn(0x7f67c0074f50 msgr2=0x7f67c0103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:59.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.829+0000 7f67c6644700 1 -- 172.21.15.144:0/2223403189 shutdown_connections 2024-09-10T11:44:59.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.829+0000 7f67c6644700 1 -- 172.21.15.144:0/2223403189 wait complete. 2024-09-10T11:44:59.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.934+0000 7ff7a674d700 1 -- 172.21.15.144:0/1358417691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7a00ff290 msgr2=0x7ff7a00ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:59.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.934+0000 7ff7a674d700 1 --2- 172.21.15.144:0/1358417691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7a00ff290 0x7ff7a00ff660 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ff790009910 tx=0x7ff790009c20 comp rx=0 tx=0).stop 2024-09-10T11:44:59.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.934+0000 7ff7a674d700 1 -- 172.21.15.144:0/1358417691 shutdown_connections 2024-09-10T11:44:59.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.934+0000 7ff7a674d700 1 --2- 172.21.15.144:0/1358417691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a00ffba0 0x7ff7a010a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.934+0000 7ff7a674d700 1 --2- 172.21.15.144:0/1358417691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7a00ff290 0x7ff7a00ff660 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.934+0000 7ff7a674d700 1 -- 172.21.15.144:0/1358417691 >> 172.21.15.144:0/1358417691 conn(0x7ff7a0074850 msgr2=0x7ff7a0074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:44:59.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.934+0000 7ff7a674d700 1 -- 172.21.15.144:0/1358417691 shutdown_connections 2024-09-10T11:44:59.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.934+0000 7ff7a674d700 1 -- 172.21.15.144:0/1358417691 wait complete. 2024-09-10T11:44:59.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.934+0000 7ff7a674d700 1 Processor -- start 2024-09-10T11:44:59.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.934+0000 7ff7a674d700 1 -- start start 2024-09-10T11:44:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.935+0000 7ff7a674d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7a00ff290 0x7ff7a0195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.935+0000 7ff7a674d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a00ffba0 0x7ff7a01962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.935+0000 7ff7a674d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a0196990 con 0x7ff7a00ff290 2024-09-10T11:44:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.935+0000 7ff7a674d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a019a720 con 0x7ff7a00ffba0 2024-09-10T11:44:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.935+0000 7ff79ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7a00ff290 0x7ff7a0195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.935+0000 7ff79ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7a00ff290 0x7ff7a0195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59140/0 (socket says 172.21.15.144:59140) 2024-09-10T11:44:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.935+0000 7ff79ffff700 1 -- 172.21.15.144:0/1657491325 learned_addr learned my addr 172.21.15.144:0/1657491325 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:44:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.935+0000 7ff79f7fe700 1 --2- 172.21.15.144:0/1657491325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a00ffba0 0x7ff7a01962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:59.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.935+0000 7ff79ffff700 1 -- 172.21.15.144:0/1657491325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a00ffba0 msgr2=0x7ff7a01962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:44:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.935+0000 7ff79ffff700 1 --2- 172.21.15.144:0/1657491325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a00ffba0 0x7ff7a01962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:44:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.935+0000 7ff79ffff700 1 -- 172.21.15.144:0/1657491325 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7900095f0 con 0x7ff7a00ff290 2024-09-10T11:44:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.935+0000 7ff79ffff700 1 --2- 172.21.15.144:0/1657491325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7a00ff290 0x7ff7a0195d70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff7900092e0 tx=0x7ff79000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.936+0000 7ff79d7fa700 1 -- 172.21.15.144:0/1657491325 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff79001d070 con 0x7ff7a00ff290 2024-09-10T11:44:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.936+0000 7ff79d7fa700 1 -- 172.21.15.144:0/1657491325 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff79000f460 con 0x7ff7a00ff290 2024-09-10T11:44:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.936+0000 7ff7a674d700 1 -- 172.21.15.144:0/1657491325 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a019a9a0 con 0x7ff7a00ff290 2024-09-10T11:44:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.936+0000 7ff79d7fa700 1 -- 172.21.15.144:0/1657491325 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7900175d0 con 0x7ff7a00ff290 2024-09-10T11:44:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.936+0000 7ff7a674d700 1 -- 172.21.15.144:0/1657491325 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a019ae90 con 0x7ff7a00ff290 2024-09-10T11:44:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.937+0000 7ff7a674d700 1 -- 172.21.15.144:0/1657491325 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a0107d10 con 0x7ff7a00ff290 2024-09-10T11:44:59.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.937+0000 7ff79d7fa700 1 -- 172.21.15.144:0/1657491325 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff79000f5d0 con 0x7ff7a00ff290 2024-09-10T11:44:59.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.937+0000 7ff79d7fa700 1 --2- 172.21.15.144:0/1657491325 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff78806e200 0x7ff7880706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:44:59.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.938+0000 7ff79d7fa700 1 -- 172.21.15.144:0/1657491325 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff79008f590 con 0x7ff7a00ff290 2024-09-10T11:44:59.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.938+0000 7ff79f7fe700 1 --2- 172.21.15.144:0/1657491325 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff78806e200 0x7ff7880706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:44:59.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.938+0000 7ff79f7fe700 1 --2- 172.21.15.144:0/1657491325 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff78806e200 0x7ff7880706b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0197390 tx=0x7ff79400a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:44:59.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:44:59.942+0000 7ff79d7fa700 1 -- 172.21.15.144:0/1657491325 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff790058970 con 0x7ff7a00ff290 2024-09-10T11:45:00.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.153+0000 7ff7a674d700 1 -- 172.21.15.144:0/1657491325 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7a002ccf0 con 0x7ff78806e200 2024-09-10T11:45:00.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.163+0000 7ff79d7fa700 1 -- 172.21.15.144:0/1657491325 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7ff7a002ccf0 con 0x7ff78806e200 2024-09-10T11:45:00.164 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:45:00.164 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (13m) 6m ago 23m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:45:00.164 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (24m) 6m ago 24m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:45:00.164 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (23m) 6m ago 23m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:45:00.165 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (10m) 6m ago 23m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:45:00.165 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (10m) 6m ago 22m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:45:00.165 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (12m) 6m ago 23m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:45:00.165 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (7m) 6m ago 18m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:45:00.165 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (7m) 6m ago 18m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:45:00.165 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (18m) 6m ago 18m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:45:00.165 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (6m) 6m ago 18m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:45:00.165 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (14m) 6m ago 25m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:45:00.165 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (14m) 6m ago 22m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:45:00.165 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (11m) 6m ago 25m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:45:00.165 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (11m) 6m ago 22m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:45:00.165 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (13m) 6m ago 23m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:45:00.165 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (13m) 6m ago 22m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:45:00.165 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (10m) 6m ago 21m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:45:00.166 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (9m) 6m ago 21m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:45:00.166 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (9m) 6m ago 21m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:45:00.166 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (8m) 6m ago 20m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:45:00.166 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (8m) 6m ago 20m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:45:00.166 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (7m) 6m ago 19m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:45:00.166 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (13m) 6m ago 23m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:45:00.166 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.165+0000 7ff7a674d700 1 -- 172.21.15.144:0/1657491325 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff78806e200 msgr2=0x7ff7880706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:00.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.165+0000 7ff7a674d700 1 --2- 172.21.15.144:0/1657491325 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff78806e200 0x7ff7880706b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff7a0197390 tx=0x7ff79400a300 comp rx=0 tx=0).stop 2024-09-10T11:45:00.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.165+0000 7ff7a674d700 1 -- 172.21.15.144:0/1657491325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7a00ff290 msgr2=0x7ff7a0195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:00.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.165+0000 7ff7a674d700 1 --2- 172.21.15.144:0/1657491325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7a00ff290 0x7ff7a0195d70 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7ff7900092e0 tx=0x7ff79000bd70 comp rx=0 tx=0).stop 2024-09-10T11:45:00.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.165+0000 7ff7a674d700 1 -- 172.21.15.144:0/1657491325 shutdown_connections 2024-09-10T11:45:00.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.165+0000 7ff7a674d700 1 --2- 172.21.15.144:0/1657491325 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff78806e200 0x7ff7880706b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.165+0000 7ff7a674d700 1 --2- 172.21.15.144:0/1657491325 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7a00ff290 0x7ff7a0195d70 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.165+0000 7ff7a674d700 1 --2- 172.21.15.144:0/1657491325 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7a00ffba0 0x7ff7a01962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.165+0000 7ff7a674d700 1 -- 172.21.15.144:0/1657491325 >> 172.21.15.144:0/1657491325 conn(0x7ff7a0074850 msgr2=0x7ff7a0104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:00.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.165+0000 7ff7a674d700 1 -- 172.21.15.144:0/1657491325 shutdown_connections 2024-09-10T11:45:00.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.165+0000 7ff7a674d700 1 -- 172.21.15.144:0/1657491325 wait complete. 2024-09-10T11:45:00.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.268+0000 7fdc0b953700 1 -- 172.21.15.144:0/4195493554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc041013e0 msgr2=0x7fdc041017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:00.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.268+0000 7fdc0b953700 1 --2- 172.21.15.144:0/4195493554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc041013e0 0x7fdc041017b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4009910 tx=0x7fdbf4009c20 comp rx=0 tx=0).stop 2024-09-10T11:45:00.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.268+0000 7fdc0b953700 1 -- 172.21.15.144:0/4195493554 shutdown_connections 2024-09-10T11:45:00.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.268+0000 7fdc0b953700 1 --2- 172.21.15.144:0/4195493554 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc04101cf0 0x7fdc0410a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.268+0000 7fdc0b953700 1 --2- 172.21.15.144:0/4195493554 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc041013e0 0x7fdc041017b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.268+0000 7fdc0b953700 1 -- 172.21.15.144:0/4195493554 >> 172.21.15.144:0/4195493554 conn(0x7fdc040fabe0 msgr2=0x7fdc040fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:00.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.268+0000 7fdc0b953700 1 -- 172.21.15.144:0/4195493554 shutdown_connections 2024-09-10T11:45:00.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.268+0000 7fdc0b953700 1 -- 172.21.15.144:0/4195493554 wait complete. 2024-09-10T11:45:00.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.269+0000 7fdc0b953700 1 Processor -- start 2024-09-10T11:45:00.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.269+0000 7fdc0b953700 1 -- start start 2024-09-10T11:45:00.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.269+0000 7fdc0b953700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc041013e0 0x7fdc04195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:00.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.269+0000 7fdc0b953700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc04101cf0 0x7fdc041962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:00.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.269+0000 7fdc0b953700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc04196990 con 0x7fdc04101cf0 2024-09-10T11:45:00.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.269+0000 7fdc0b953700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0419a6d0 con 0x7fdc041013e0 2024-09-10T11:45:00.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.269+0000 7fdc08eee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc04101cf0 0x7fdc041962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:00.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.270+0000 7fdc08eee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc04101cf0 0x7fdc041962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59164/0 (socket says 172.21.15.144:59164) 2024-09-10T11:45:00.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.270+0000 7fdc08eee700 1 -- 172.21.15.144:0/3468043824 learned_addr learned my addr 172.21.15.144:0/3468043824 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:45:00.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.270+0000 7fdc096ef700 1 --2- 172.21.15.144:0/3468043824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc041013e0 0x7fdc04195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:00.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.270+0000 7fdc08eee700 1 -- 172.21.15.144:0/3468043824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc041013e0 msgr2=0x7fdc04195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:00.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.270+0000 7fdc08eee700 1 --2- 172.21.15.144:0/3468043824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc041013e0 0x7fdc04195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.270+0000 7fdc08eee700 1 -- 172.21.15.144:0/3468043824 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf40095f0 con 0x7fdc04101cf0 2024-09-10T11:45:00.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.270+0000 7fdc08eee700 1 --2- 172.21.15.144:0/3468043824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc04101cf0 0x7fdc041962b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000d8d0 tx=0x7fdc0000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:00.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.270+0000 7fdbfa7fc700 1 -- 172.21.15.144:0/3468043824 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc000098a0 con 0x7fdc04101cf0 2024-09-10T11:45:00.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.270+0000 7fdc0b953700 1 -- 172.21.15.144:0/3468043824 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0419a9b0 con 0x7fdc04101cf0 2024-09-10T11:45:00.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.270+0000 7fdbfa7fc700 1 -- 172.21.15.144:0/3468043824 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc00010460 con 0x7fdc04101cf0 2024-09-10T11:45:00.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.270+0000 7fdbfa7fc700 1 -- 172.21.15.144:0/3468043824 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc0000f5f0 con 0x7fdc04101cf0 2024-09-10T11:45:00.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.270+0000 7fdc0b953700 1 -- 172.21.15.144:0/3468043824 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0419af00 con 0x7fdc04101cf0 2024-09-10T11:45:00.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.271+0000 7fdc0b953700 1 -- 172.21.15.144:0/3468043824 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc041078c0 con 0x7fdc04101cf0 2024-09-10T11:45:00.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.272+0000 7fdbfa7fc700 1 -- 172.21.15.144:0/3468043824 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fdc00009a00 con 0x7fdc04101cf0 2024-09-10T11:45:00.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.273+0000 7fdbfa7fc700 1 --2- 172.21.15.144:0/3468043824 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdbf006e200 0x7fdbf00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:00.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.273+0000 7fdbfa7fc700 1 -- 172.21.15.144:0/3468043824 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdc0008df70 con 0x7fdc04101cf0 2024-09-10T11:45:00.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.273+0000 7fdc096ef700 1 --2- 172.21.15.144:0/3468043824 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdbf006e200 0x7fdbf00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:00.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.273+0000 7fdc096ef700 1 --2- 172.21.15.144:0/3468043824 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdbf006e200 0x7fdbf00706b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4006010 tx=0x7fdbf401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:00.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.277+0000 7fdbfa7fc700 1 -- 172.21.15.144:0/3468043824 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdc000c31b0 con 0x7fdc04101cf0 2024-09-10T11:45:00.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.558+0000 7fdc0b953700 1 -- 172.21.15.144:0/3468043824 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdc0404ea50 con 0x7fdc04101cf0 2024-09-10T11:45:00.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.558+0000 7fdbfa7fc700 1 -- 172.21.15.144:0/3468043824 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fdc00057350 con 0x7fdc04101cf0 2024-09-10T11:45:00.560 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:45:00.560 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:45:00.560 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:45:00.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:45:00.560 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:45:00.560 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:45:00.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:45:00.560 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:45:00.560 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:45:00.560 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:45:00.560 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:45:00.561 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:45:00.561 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:45:00.561 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:45:00.561 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:45:00.561 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:45:00.561 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:45:00.561 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:45:00.561 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:45:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.561+0000 7fdc0b953700 1 -- 172.21.15.144:0/3468043824 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdbf006e200 msgr2=0x7fdbf00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.561+0000 7fdc0b953700 1 --2- 172.21.15.144:0/3468043824 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdbf006e200 0x7fdbf00706b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4006010 tx=0x7fdbf401a040 comp rx=0 tx=0).stop 2024-09-10T11:45:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.561+0000 7fdc0b953700 1 -- 172.21.15.144:0/3468043824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc04101cf0 msgr2=0x7fdc041962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.561+0000 7fdc0b953700 1 --2- 172.21.15.144:0/3468043824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc04101cf0 0x7fdc041962b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fdc0000d8d0 tx=0x7fdc0000dbe0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.561+0000 7fdc0b953700 1 -- 172.21.15.144:0/3468043824 shutdown_connections 2024-09-10T11:45:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.561+0000 7fdc0b953700 1 --2- 172.21.15.144:0/3468043824 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdbf006e200 0x7fdbf00706b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.561+0000 7fdc0b953700 1 --2- 172.21.15.144:0/3468043824 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc041013e0 0x7fdc04195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.561+0000 7fdc0b953700 1 --2- 172.21.15.144:0/3468043824 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc04101cf0 0x7fdc041962b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.561+0000 7fdc0b953700 1 -- 172.21.15.144:0/3468043824 >> 172.21.15.144:0/3468043824 conn(0x7fdc040fabe0 msgr2=0x7fdc04104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.561+0000 7fdc0b953700 1 -- 172.21.15.144:0/3468043824 shutdown_connections 2024-09-10T11:45:00.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.561+0000 7fdc0b953700 1 -- 172.21.15.144:0/3468043824 wait complete. 2024-09-10T11:45:00.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.665+0000 7f54415b3700 1 -- 172.21.15.144:0/1594047443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f543c0ffc60 msgr2=0x7f543c103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:00.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.665+0000 7f54415b3700 1 --2- 172.21.15.144:0/1594047443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f543c0ffc60 0x7f543c103c30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5424009910 tx=0x7f5424009c20 comp rx=0 tx=0).stop 2024-09-10T11:45:00.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.665+0000 7f54415b3700 1 -- 172.21.15.144:0/1594047443 shutdown_connections 2024-09-10T11:45:00.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.665+0000 7f54415b3700 1 --2- 172.21.15.144:0/1594047443 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f543c0ffc60 0x7f543c103c30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.665+0000 7f54415b3700 1 --2- 172.21.15.144:0/1594047443 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f543c0ff2c0 0x7f543c0ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.665+0000 7f54415b3700 1 -- 172.21.15.144:0/1594047443 >> 172.21.15.144:0/1594047443 conn(0x7f543c0fab60 msgr2=0x7f543c0fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:00.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.665+0000 7f54415b3700 1 -- 172.21.15.144:0/1594047443 shutdown_connections 2024-09-10T11:45:00.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.665+0000 7f54415b3700 1 -- 172.21.15.144:0/1594047443 wait complete. 2024-09-10T11:45:00.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.666+0000 7f54415b3700 1 Processor -- start 2024-09-10T11:45:00.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.666+0000 7f54415b3700 1 -- start start 2024-09-10T11:45:00.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.666+0000 7f54415b3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f543c0ff2c0 0x7f543c196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:00.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.666+0000 7f54415b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f543c0ffc60 0x7f543c197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:00.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.666+0000 7f54415b3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c1976f0 con 0x7f543c0ffc60 2024-09-10T11:45:00.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.666+0000 7f54415b3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f543c197830 con 0x7f543c0ff2c0 2024-09-10T11:45:00.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.666+0000 7f5433fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f543c0ffc60 0x7f543c197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:00.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.666+0000 7f5433fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f543c0ffc60 0x7f543c197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59186/0 (socket says 172.21.15.144:59186) 2024-09-10T11:45:00.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.666+0000 7f5433fff700 1 -- 172.21.15.144:0/4260945263 learned_addr learned my addr 172.21.15.144:0/4260945263 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:45:00.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.666+0000 7f543affd700 1 --2- 172.21.15.144:0/4260945263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f543c0ff2c0 0x7f543c196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:00.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.666+0000 7f5433fff700 1 -- 172.21.15.144:0/4260945263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f543c0ff2c0 msgr2=0x7f543c196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:00.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.666+0000 7f5433fff700 1 --2- 172.21.15.144:0/4260945263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f543c0ff2c0 0x7f543c196c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.666+0000 7f5433fff700 1 -- 172.21.15.144:0/4260945263 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54240095f0 con 0x7f543c0ffc60 2024-09-10T11:45:00.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.667+0000 7f5433fff700 1 --2- 172.21.15.144:0/4260945263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f543c0ffc60 0x7f543c197160 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f542400bda0 tx=0x7f542400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:00.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.667+0000 7f5438ff9700 1 -- 172.21.15.144:0/4260945263 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f542401d070 con 0x7f543c0ffc60 2024-09-10T11:45:00.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.667+0000 7f54415b3700 1 -- 172.21.15.144:0/4260945263 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f543c19fc90 con 0x7f543c0ffc60 2024-09-10T11:45:00.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.667+0000 7f5438ff9700 1 -- 172.21.15.144:0/4260945263 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5424004fb0 con 0x7f543c0ffc60 2024-09-10T11:45:00.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.667+0000 7f5438ff9700 1 -- 172.21.15.144:0/4260945263 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f542400f480 con 0x7f543c0ffc60 2024-09-10T11:45:00.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.667+0000 7f54415b3700 1 -- 172.21.15.144:0/4260945263 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f543c1a0040 con 0x7f543c0ffc60 2024-09-10T11:45:00.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.668+0000 7f54415b3700 1 -- 172.21.15.144:0/4260945263 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f543c04ea50 con 0x7f543c0ffc60 2024-09-10T11:45:00.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.668+0000 7f5438ff9700 1 -- 172.21.15.144:0/4260945263 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5424022870 con 0x7f543c0ffc60 2024-09-10T11:45:00.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.669+0000 7f5438ff9700 1 --2- 172.21.15.144:0/4260945263 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f542806e200 0x7f54280706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:00.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.669+0000 7f5438ff9700 1 -- 172.21.15.144:0/4260945263 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f542408f5c0 con 0x7f543c0ffc60 2024-09-10T11:45:00.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.669+0000 7f543affd700 1 --2- 172.21.15.144:0/4260945263 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f542806e200 0x7f54280706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:00.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.669+0000 7f543affd700 1 --2- 172.21.15.144:0/4260945263 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f542806e200 0x7f54280706b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f542c00ba60 tx=0x7f542c005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:00.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.673+0000 7f5438ff9700 1 -- 172.21.15.144:0/4260945263 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5424058a50 con 0x7f543c0ffc60 2024-09-10T11:45:00.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.918+0000 7f54415b3700 1 -- 172.21.15.144:0/4260945263 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f543c1a0390 con 0x7f543c0ffc60 2024-09-10T11:45:00.919 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:00 smithi144 ceph-mon[87436]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:00.919 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:00 smithi144 ceph-mon[87436]: pgmap v390: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:00.919 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:00 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3468043824' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:45:00.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.918+0000 7f5438ff9700 1 -- 172.21.15.144:0/4260945263 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f542405c110 con 0x7f543c0ffc60 2024-09-10T11:45:00.920 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:45:00.920 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:45:00.920 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:45:00.920 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:45:00.920 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:45:00.920 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:45:00.921 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:45:00.921 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:45:00.921 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:45:00.921 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:45:00.921 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:45:00.921 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:45:00.921 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:45:00.921 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:45:00.921 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:45:00.921 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:45:00.921 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:45:00.921 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:45:00.921 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:45:00.921 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:45:00.922 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:45:00.922 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:45:00.922 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:45:00.922 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:45:00.922 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:45:00.922 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:45:00.922 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:45:00.922 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:45:00.922 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:45:00.922 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:45:00.922 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:45:00.922 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:45:00.922 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:45:00.923 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:45:00.923 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:45:00.923 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:45:00.923 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:45:00.923 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:45:00.923 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:45:00.923 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:45:00.923 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:45:00.923 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:45:00.923 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:45:00.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.921+0000 7f54415b3700 1 -- 172.21.15.144:0/4260945263 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f542806e200 msgr2=0x7f54280706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:00.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.921+0000 7f54415b3700 1 --2- 172.21.15.144:0/4260945263 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f542806e200 0x7f54280706b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f542c00ba60 tx=0x7f542c005d50 comp rx=0 tx=0).stop 2024-09-10T11:45:00.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.921+0000 7f54415b3700 1 -- 172.21.15.144:0/4260945263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f543c0ffc60 msgr2=0x7f543c197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:00.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.921+0000 7f54415b3700 1 --2- 172.21.15.144:0/4260945263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f543c0ffc60 0x7f543c197160 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f542400bda0 tx=0x7f542400be80 comp rx=0 tx=0).stop 2024-09-10T11:45:00.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.922+0000 7f54415b3700 1 -- 172.21.15.144:0/4260945263 shutdown_connections 2024-09-10T11:45:00.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.922+0000 7f54415b3700 1 --2- 172.21.15.144:0/4260945263 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f542806e200 0x7f54280706b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.922+0000 7f54415b3700 1 --2- 172.21.15.144:0/4260945263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f543c0ff2c0 0x7f543c196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.922+0000 7f54415b3700 1 --2- 172.21.15.144:0/4260945263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f543c0ffc60 0x7f543c197160 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:00.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.922+0000 7f54415b3700 1 -- 172.21.15.144:0/4260945263 >> 172.21.15.144:0/4260945263 conn(0x7f543c0fab60 msgr2=0x7f543c0fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:00.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.922+0000 7f54415b3700 1 -- 172.21.15.144:0/4260945263 shutdown_connections 2024-09-10T11:45:00.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:00.922+0000 7f54415b3700 1 -- 172.21.15.144:0/4260945263 wait complete. 2024-09-10T11:45:00.925 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:45:01.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.028+0000 7f15644c3700 1 -- 172.21.15.144:0/3648681277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f155c101ca0 msgr2=0x7f155c10a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:01.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.028+0000 7f15644c3700 1 --2- 172.21.15.144:0/3648681277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f155c101ca0 0x7f155c10a1a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f154c009910 tx=0x7f154c009c20 comp rx=0 tx=0).stop 2024-09-10T11:45:01.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.028+0000 7f15644c3700 1 -- 172.21.15.144:0/3648681277 shutdown_connections 2024-09-10T11:45:01.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.028+0000 7f15644c3700 1 --2- 172.21.15.144:0/3648681277 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f155c101ca0 0x7f155c10a1a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:01.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.028+0000 7f15644c3700 1 --2- 172.21.15.144:0/3648681277 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f155c101390 0x7f155c101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:01.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.028+0000 7f15644c3700 1 -- 172.21.15.144:0/3648681277 >> 172.21.15.144:0/3648681277 conn(0x7f155c0fabb0 msgr2=0x7f155c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:01.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.028+0000 7f15644c3700 1 -- 172.21.15.144:0/3648681277 shutdown_connections 2024-09-10T11:45:01.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.028+0000 7f15644c3700 1 -- 172.21.15.144:0/3648681277 wait complete. 2024-09-10T11:45:01.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.028+0000 7f15644c3700 1 Processor -- start 2024-09-10T11:45:01.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.028+0000 7f15644c3700 1 -- start start 2024-09-10T11:45:01.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f15644c3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f155c101390 0x7f155c195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:01.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f15644c3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f155c101ca0 0x7f155c196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:01.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f15644c3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c196930 con 0x7f155c101ca0 2024-09-10T11:45:01.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f15644c3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155c19a6c0 con 0x7f155c101390 2024-09-10T11:45:01.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f1561a5e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f155c101ca0 0x7f155c196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:01.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f1561a5e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f155c101ca0 0x7f155c196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59214/0 (socket says 172.21.15.144:59214) 2024-09-10T11:45:01.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f1561a5e700 1 -- 172.21.15.144:0/3924110649 learned_addr learned my addr 172.21.15.144:0/3924110649 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:45:01.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f156225f700 1 --2- 172.21.15.144:0/3924110649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f155c101390 0x7f155c195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:01.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f1561a5e700 1 -- 172.21.15.144:0/3924110649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f155c101390 msgr2=0x7f155c195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:01.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f1561a5e700 1 --2- 172.21.15.144:0/3924110649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f155c101390 0x7f155c195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:01.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f1561a5e700 1 -- 172.21.15.144:0/3924110649 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f154c0095f0 con 0x7f155c101ca0 2024-09-10T11:45:01.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f1561a5e700 1 --2- 172.21.15.144:0/3924110649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f155c101ca0 0x7f155c196250 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f154c0092e0 tx=0x7f154c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:01.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f15537fe700 1 -- 172.21.15.144:0/3924110649 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f154c01d070 con 0x7f155c101ca0 2024-09-10T11:45:01.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f15537fe700 1 -- 172.21.15.144:0/3924110649 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f154c005e00 con 0x7f155c101ca0 2024-09-10T11:45:01.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f15644c3700 1 -- 172.21.15.144:0/3924110649 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155c19a940 con 0x7f155c101ca0 2024-09-10T11:45:01.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f15537fe700 1 -- 172.21.15.144:0/3924110649 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f154c017400 con 0x7f155c101ca0 2024-09-10T11:45:01.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.029+0000 7f15644c3700 1 -- 172.21.15.144:0/3924110649 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155c19ae30 con 0x7f155c101ca0 2024-09-10T11:45:01.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.030+0000 7f15644c3700 1 -- 172.21.15.144:0/3924110649 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155c107810 con 0x7f155c101ca0 2024-09-10T11:45:01.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.031+0000 7f15537fe700 1 -- 172.21.15.144:0/3924110649 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f154c00f460 con 0x7f155c101ca0 2024-09-10T11:45:01.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.031+0000 7f15537fe700 1 --2- 172.21.15.144:0/3924110649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f154806e200 0x7f15480706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:01.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.031+0000 7f15537fe700 1 -- 172.21.15.144:0/3924110649 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f154c08f5d0 con 0x7f155c101ca0 2024-09-10T11:45:01.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.031+0000 7f156225f700 1 --2- 172.21.15.144:0/3924110649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f154806e200 0x7f15480706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:01.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.032+0000 7f156225f700 1 --2- 172.21.15.144:0/3924110649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f154806e200 0x7f15480706b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1558005950 tx=0x7f15580058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:01.037 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.036+0000 7f15537fe700 1 -- 172.21.15.144:0/3924110649 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f154c0589b0 con 0x7f155c101ca0 2024-09-10T11:45:01.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:00 smithi178 ceph-mon[65940]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:00 smithi178 ceph-mon[65940]: pgmap v390: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:00 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3468043824' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:45:01.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.257+0000 7f15644c3700 1 -- 172.21.15.144:0/3924110649 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f155c02cd70 con 0x7f154806e200 2024-09-10T11:45:01.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.258+0000 7f15537fe700 1 -- 172.21.15.144:0/3924110649 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f155c02cd70 con 0x7f154806e200 2024-09-10T11:45:01.259 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:45:01.260 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:45:01.260 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:45:01.260 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:45:01.260 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:45:01.260 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:45:01.260 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:45:01.260 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:45:01.260 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:45:01.261 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:45:01.261 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:45:01.261 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:45:01.261 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:45:01.261 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:45:01.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.260+0000 7f15644c3700 1 -- 172.21.15.144:0/3924110649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f154806e200 msgr2=0x7f15480706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:01.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.260+0000 7f15644c3700 1 --2- 172.21.15.144:0/3924110649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f154806e200 0x7f15480706b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1558005950 tx=0x7f15580058e0 comp rx=0 tx=0).stop 2024-09-10T11:45:01.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.260+0000 7f15644c3700 1 -- 172.21.15.144:0/3924110649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f155c101ca0 msgr2=0x7f155c196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:01.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.260+0000 7f15644c3700 1 --2- 172.21.15.144:0/3924110649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f155c101ca0 0x7f155c196250 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f154c0092e0 tx=0x7f154c00be20 comp rx=0 tx=0).stop 2024-09-10T11:45:01.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.260+0000 7f15644c3700 1 -- 172.21.15.144:0/3924110649 shutdown_connections 2024-09-10T11:45:01.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.260+0000 7f15644c3700 1 --2- 172.21.15.144:0/3924110649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f154806e200 0x7f15480706b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:01.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.260+0000 7f15644c3700 1 --2- 172.21.15.144:0/3924110649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f155c101390 0x7f155c195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:01.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.260+0000 7f15644c3700 1 --2- 172.21.15.144:0/3924110649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f155c101ca0 0x7f155c196250 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:01.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.260+0000 7f15644c3700 1 -- 172.21.15.144:0/3924110649 >> 172.21.15.144:0/3924110649 conn(0x7f155c0fabb0 msgr2=0x7f155c104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:01.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.261+0000 7f15644c3700 1 -- 172.21.15.144:0/3924110649 shutdown_connections 2024-09-10T11:45:01.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.261+0000 7f15644c3700 1 -- 172.21.15.144:0/3924110649 wait complete. 2024-09-10T11:45:01.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.366+0000 7fe4008e9700 1 -- 172.21.15.144:0/1131869299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3fc1003b0 msgr2=0x7fe3fc100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:01.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.366+0000 7fe4008e9700 1 --2- 172.21.15.144:0/1131869299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3fc1003b0 0x7fe3fc100780 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec009910 tx=0x7fe3ec009c20 comp rx=0 tx=0).stop 2024-09-10T11:45:01.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.366+0000 7fe4008e9700 1 -- 172.21.15.144:0/1131869299 shutdown_connections 2024-09-10T11:45:01.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.366+0000 7fe4008e9700 1 --2- 172.21.15.144:0/1131869299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3fc100d50 0x7fe3fc104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:01.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.366+0000 7fe4008e9700 1 --2- 172.21.15.144:0/1131869299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3fc1003b0 0x7fe3fc100780 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:01.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.366+0000 7fe4008e9700 1 -- 172.21.15.144:0/1131869299 >> 172.21.15.144:0/1131869299 conn(0x7fe3fc0fbc30 msgr2=0x7fe3fc0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:01.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.366+0000 7fe4008e9700 1 -- 172.21.15.144:0/1131869299 shutdown_connections 2024-09-10T11:45:01.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.366+0000 7fe4008e9700 1 -- 172.21.15.144:0/1131869299 wait complete. 2024-09-10T11:45:01.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.366+0000 7fe4008e9700 1 Processor -- start 2024-09-10T11:45:01.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.367+0000 7fe4008e9700 1 -- start start 2024-09-10T11:45:01.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.367+0000 7fe4008e9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3fc1003b0 0x7fe3fc073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:01.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.367+0000 7fe4008e9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3fc100d50 0x7fe3fc073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:01.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.367+0000 7fe4008e9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc073ad0 con 0x7fe3fc100d50 2024-09-10T11:45:01.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.367+0000 7fe4008e9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3fc073c10 con 0x7fe3fc1003b0 2024-09-10T11:45:01.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.367+0000 7fe3f1d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3fc100d50 0x7fe3fc073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:01.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.367+0000 7fe3f1d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3fc100d50 0x7fe3fc073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:59232/0 (socket says 172.21.15.144:59232) 2024-09-10T11:45:01.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.367+0000 7fe3f1d9b700 1 -- 172.21.15.144:0/2237469349 learned_addr learned my addr 172.21.15.144:0/2237469349 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:45:01.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.367+0000 7fe3fa59c700 1 --2- 172.21.15.144:0/2237469349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3fc1003b0 0x7fe3fc073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:01.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.367+0000 7fe3f1d9b700 1 -- 172.21.15.144:0/2237469349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3fc1003b0 msgr2=0x7fe3fc073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:01.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.367+0000 7fe3f1d9b700 1 --2- 172.21.15.144:0/2237469349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3fc1003b0 0x7fe3fc073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:01.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.367+0000 7fe3f1d9b700 1 -- 172.21.15.144:0/2237469349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3ec0095f0 con 0x7fe3fc100d50 2024-09-10T11:45:01.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.368+0000 7fe3f1d9b700 1 --2- 172.21.15.144:0/2237469349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3fc100d50 0x7fe3fc073590 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400d8d0 tx=0x7fe3e400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:01.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.368+0000 7fe3f3fff700 1 -- 172.21.15.144:0/2237469349 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3e40041f0 con 0x7fe3fc100d50 2024-09-10T11:45:01.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.368+0000 7fe3f3fff700 1 -- 172.21.15.144:0/2237469349 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3e4007620 con 0x7fe3fc100d50 2024-09-10T11:45:01.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.368+0000 7fe4008e9700 1 -- 172.21.15.144:0/2237469349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc1a1ee0 con 0x7fe3fc100d50 2024-09-10T11:45:01.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.368+0000 7fe3f3fff700 1 -- 172.21.15.144:0/2237469349 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3e40056b0 con 0x7fe3fc100d50 2024-09-10T11:45:01.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.368+0000 7fe4008e9700 1 -- 172.21.15.144:0/2237469349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3fc1a22f0 con 0x7fe3fc100d50 2024-09-10T11:45:01.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.369+0000 7fe4008e9700 1 -- 172.21.15.144:0/2237469349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3fc1086c0 con 0x7fe3fc100d50 2024-09-10T11:45:01.374 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.369+0000 7fe3f3fff700 1 -- 172.21.15.144:0/2237469349 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe3e400f770 con 0x7fe3fc100d50 2024-09-10T11:45:01.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.370+0000 7fe3f3fff700 1 --2- 172.21.15.144:0/2237469349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe3e806e200 0x7fe3e80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:01.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.370+0000 7fe3fa59c700 1 --2- 172.21.15.144:0/2237469349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe3e806e200 0x7fe3e80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:01.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.370+0000 7fe3f3fff700 1 -- 172.21.15.144:0/2237469349 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe3e408dea0 con 0x7fe3fc100d50 2024-09-10T11:45:01.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.370+0000 7fe3fa59c700 1 --2- 172.21.15.144:0/2237469349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe3e806e200 0x7fe3e80706b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00b5c0 tx=0x7fe3ec01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:01.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.374+0000 7fe3f3fff700 1 -- 172.21.15.144:0/2237469349 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe3e4057280 con 0x7fe3fc100d50 2024-09-10T11:45:01.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.657+0000 7fe4008e9700 1 -- 172.21.15.144:0/2237469349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe3fc073dd0 con 0x7fe3fc100d50 2024-09-10T11:45:01.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.657+0000 7fe3f3fff700 1 -- 172.21.15.144:0/2237469349 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fe3e40161c0 con 0x7fe3fc100d50 2024-09-10T11:45:01.659 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:45:01.659 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:45:01.659 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:45:01.659 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:45:01.659 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:45:01.659 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:45:01.659 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:45:01.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.659+0000 7fe4008e9700 1 -- 172.21.15.144:0/2237469349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe3e806e200 msgr2=0x7fe3e80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:01.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.659+0000 7fe4008e9700 1 --2- 172.21.15.144:0/2237469349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe3e806e200 0x7fe3e80706b0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe3ec00b5c0 tx=0x7fe3ec01a040 comp rx=0 tx=0).stop 2024-09-10T11:45:01.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.659+0000 7fe4008e9700 1 -- 172.21.15.144:0/2237469349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3fc100d50 msgr2=0x7fe3fc073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:01.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.659+0000 7fe4008e9700 1 --2- 172.21.15.144:0/2237469349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3fc100d50 0x7fe3fc073590 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fe3e400d8d0 tx=0x7fe3e400dbe0 comp rx=0 tx=0).stop 2024-09-10T11:45:01.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.659+0000 7fe4008e9700 1 -- 172.21.15.144:0/2237469349 shutdown_connections 2024-09-10T11:45:01.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.659+0000 7fe4008e9700 1 --2- 172.21.15.144:0/2237469349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe3e806e200 0x7fe3e80706b0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:01.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.659+0000 7fe4008e9700 1 --2- 172.21.15.144:0/2237469349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3fc1003b0 0x7fe3fc073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:01.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.659+0000 7fe4008e9700 1 --2- 172.21.15.144:0/2237469349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3fc100d50 0x7fe3fc073590 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:01.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.659+0000 7fe4008e9700 1 -- 172.21.15.144:0/2237469349 >> 172.21.15.144:0/2237469349 conn(0x7fe3fc0fbc30 msgr2=0x7fe3fc0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:01.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.660+0000 7fe4008e9700 1 -- 172.21.15.144:0/2237469349 shutdown_connections 2024-09-10T11:45:01.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:01.660+0000 7fe4008e9700 1 -- 172.21.15.144:0/2237469349 wait complete. 2024-09-10T11:45:01.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:01 smithi144 ceph-mon[87436]: from='client.34768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:01.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:01 smithi144 ceph-mon[87436]: from='client.34772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:01.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:01 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/4260945263' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:45:01.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:01 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2237469349' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:45:02.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:01 smithi178 ceph-mon[65940]: from='client.34768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:02.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:01 smithi178 ceph-mon[65940]: from='client.34772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:02.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:01 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/4260945263' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:45:02.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:01 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2237469349' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:45:03.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:02 smithi144 ceph-mon[87436]: from='client.34784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:03.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:02 smithi144 ceph-mon[87436]: pgmap v391: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:03.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:02 smithi178 ceph-mon[65940]: from='client.34784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:03.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:02 smithi178 ceph-mon[65940]: pgmap v391: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:04 smithi178 ceph-mon[65940]: pgmap v392: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:05.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:04 smithi144 ceph-mon[87436]: pgmap v392: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:06 smithi178 ceph-mon[65940]: pgmap v393: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:07.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:06 smithi144 ceph-mon[87436]: pgmap v393: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:08.919 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:08 smithi178 ceph-mon[65940]: pgmap v394: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:08 smithi144 ceph-mon[87436]: pgmap v394: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:10 smithi178 ceph-mon[65940]: pgmap v395: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:11.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:10 smithi144 ceph-mon[87436]: pgmap v395: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:13.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:12 smithi144 ceph-mon[87436]: pgmap v396: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:13.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:12 smithi178 ceph-mon[65940]: pgmap v396: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:45:14.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:45:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:14 smithi178 ceph-mon[65940]: pgmap v397: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:15.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:14 smithi144 ceph-mon[87436]: pgmap v397: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:16 smithi178 ceph-mon[65940]: pgmap v398: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:17.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:16 smithi144 ceph-mon[87436]: pgmap v398: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:18 smithi178 ceph-mon[65940]: pgmap v399: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:18 smithi144 ceph-mon[87436]: pgmap v399: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:20 smithi178 ceph-mon[65940]: pgmap v400: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:21.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:20 smithi144 ceph-mon[87436]: pgmap v400: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:23.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:22 smithi144 ceph-mon[87436]: pgmap v401: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:23.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:22 smithi178 ceph-mon[65940]: pgmap v401: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:24.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:23 smithi144 ceph-mon[87436]: pgmap v402: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:24.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:23 smithi178 ceph-mon[65940]: pgmap v402: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:26 smithi178 ceph-mon[65940]: pgmap v403: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:27.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:26 smithi144 ceph-mon[87436]: pgmap v403: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:28 smithi178 ceph-mon[65940]: pgmap v404: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:45:29.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:28 smithi144 ceph-mon[87436]: pgmap v404: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:45:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:30 smithi178 ceph-mon[65940]: pgmap v405: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:31.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:30 smithi144 ceph-mon[87436]: pgmap v405: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:31.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.777+0000 7f3e093a7700 1 -- 172.21.15.144:0/2340036727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e04100e40 msgr2=0x7f3e04104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:31.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.777+0000 7f3e093a7700 1 --2- 172.21.15.144:0/2340036727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e04100e40 0x7f3e04104e10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3df8009910 tx=0x7f3df8009c20 comp rx=0 tx=0).stop 2024-09-10T11:45:31.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.777+0000 7f3e093a7700 1 -- 172.21.15.144:0/2340036727 shutdown_connections 2024-09-10T11:45:31.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.777+0000 7f3e093a7700 1 --2- 172.21.15.144:0/2340036727 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e04100e40 0x7f3e04104e10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:31.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.777+0000 7f3e093a7700 1 --2- 172.21.15.144:0/2340036727 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e041004a0 0x7f3e04100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:31.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.777+0000 7f3e093a7700 1 -- 172.21.15.144:0/2340036727 >> 172.21.15.144:0/2340036727 conn(0x7f3e040fbd60 msgr2=0x7f3e040fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:31.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.777+0000 7f3e093a7700 1 -- 172.21.15.144:0/2340036727 shutdown_connections 2024-09-10T11:45:31.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.777+0000 7f3e093a7700 1 -- 172.21.15.144:0/2340036727 wait complete. 2024-09-10T11:45:31.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.778+0000 7f3e093a7700 1 Processor -- start 2024-09-10T11:45:31.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.778+0000 7f3e093a7700 1 -- start start 2024-09-10T11:45:31.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.778+0000 7f3e093a7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e041004a0 0x7f3e04195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:31.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.778+0000 7f3e093a7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e04100e40 0x7f3e04196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:31.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.778+0000 7f3e093a7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e04196960 con 0x7f3e041004a0 2024-09-10T11:45:31.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.778+0000 7f3e093a7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e0419a6f0 con 0x7f3e04100e40 2024-09-10T11:45:31.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.779+0000 7f3e02ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e041004a0 0x7f3e04195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:31.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.779+0000 7f3e02ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e041004a0 0x7f3e04195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40152/0 (socket says 172.21.15.144:40152) 2024-09-10T11:45:31.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.779+0000 7f3e02ffd700 1 -- 172.21.15.144:0/3787750465 learned_addr learned my addr 172.21.15.144:0/3787750465 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:45:31.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.779+0000 7f3e027fc700 1 --2- 172.21.15.144:0/3787750465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e04100e40 0x7f3e04196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:31.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.779+0000 7f3e02ffd700 1 -- 172.21.15.144:0/3787750465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e04100e40 msgr2=0x7f3e04196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:31.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.779+0000 7f3e02ffd700 1 --2- 172.21.15.144:0/3787750465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e04100e40 0x7f3e04196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:31.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.779+0000 7f3e02ffd700 1 -- 172.21.15.144:0/3787750465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df80095f0 con 0x7f3e041004a0 2024-09-10T11:45:31.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.779+0000 7f3e02ffd700 1 --2- 172.21.15.144:0/3787750465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e041004a0 0x7f3e04195d40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3df400d8d0 tx=0x7f3df400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:31.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.779+0000 7f3debfff700 1 -- 172.21.15.144:0/3787750465 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3df40098a0 con 0x7f3e041004a0 2024-09-10T11:45:31.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.779+0000 7f3e093a7700 1 -- 172.21.15.144:0/3787750465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0419a9d0 con 0x7f3e041004a0 2024-09-10T11:45:31.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.779+0000 7f3debfff700 1 -- 172.21.15.144:0/3787750465 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3df4010460 con 0x7f3e041004a0 2024-09-10T11:45:31.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.779+0000 7f3debfff700 1 -- 172.21.15.144:0/3787750465 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3df400f5f0 con 0x7f3e041004a0 2024-09-10T11:45:31.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.779+0000 7f3e093a7700 1 -- 172.21.15.144:0/3787750465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0419af20 con 0x7f3e041004a0 2024-09-10T11:45:31.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.780+0000 7f3e093a7700 1 -- 172.21.15.144:0/3787750465 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e04108810 con 0x7f3e041004a0 2024-09-10T11:45:31.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.781+0000 7f3debfff700 1 -- 172.21.15.144:0/3787750465 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3df4009a00 con 0x7f3e041004a0 2024-09-10T11:45:31.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.781+0000 7f3debfff700 1 --2- 172.21.15.144:0/3787750465 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3dec06e200 0x7f3dec0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:31.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.781+0000 7f3debfff700 1 -- 172.21.15.144:0/3787750465 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3df408df70 con 0x7f3e041004a0 2024-09-10T11:45:31.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.781+0000 7f3e027fc700 1 --2- 172.21.15.144:0/3787750465 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3dec06e200 0x7f3dec0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:31.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.782+0000 7f3e027fc700 1 --2- 172.21.15.144:0/3787750465 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3dec06e200 0x7f3dec0706b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3e04197360 tx=0x7f3df801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:31.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:31.786+0000 7f3debfff700 1 -- 172.21.15.144:0/3787750465 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3df40c31b0 con 0x7f3e041004a0 2024-09-10T11:45:32.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.006+0000 7f3e093a7700 1 -- 172.21.15.144:0/3787750465 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e0419b250 con 0x7f3dec06e200 2024-09-10T11:45:32.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.007+0000 7f3debfff700 1 -- 172.21.15.144:0/3787750465 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3e0419b250 con 0x7f3dec06e200 2024-09-10T11:45:32.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.010+0000 7f3e093a7700 1 -- 172.21.15.144:0/3787750465 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3dec06e200 msgr2=0x7f3dec0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:32.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.010+0000 7f3e093a7700 1 --2- 172.21.15.144:0/3787750465 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3dec06e200 0x7f3dec0706b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3e04197360 tx=0x7f3df801a040 comp rx=0 tx=0).stop 2024-09-10T11:45:32.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.010+0000 7f3e093a7700 1 -- 172.21.15.144:0/3787750465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e041004a0 msgr2=0x7f3e04195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:32.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.010+0000 7f3e093a7700 1 --2- 172.21.15.144:0/3787750465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e041004a0 0x7f3e04195d40 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f3df400d8d0 tx=0x7f3df400dbe0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.010+0000 7f3e093a7700 1 -- 172.21.15.144:0/3787750465 shutdown_connections 2024-09-10T11:45:32.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.010+0000 7f3e093a7700 1 --2- 172.21.15.144:0/3787750465 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3dec06e200 0x7f3dec0706b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.010+0000 7f3e093a7700 1 --2- 172.21.15.144:0/3787750465 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3e041004a0 0x7f3e04195d40 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.010+0000 7f3e093a7700 1 --2- 172.21.15.144:0/3787750465 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3e04100e40 0x7f3e04196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.010+0000 7f3e093a7700 1 -- 172.21.15.144:0/3787750465 >> 172.21.15.144:0/3787750465 conn(0x7f3e040fbd60 msgr2=0x7f3e040fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:32.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.010+0000 7f3e093a7700 1 -- 172.21.15.144:0/3787750465 shutdown_connections 2024-09-10T11:45:32.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.010+0000 7f3e093a7700 1 -- 172.21.15.144:0/3787750465 wait complete. 2024-09-10T11:45:32.021 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:45:32.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.116+0000 7fe6656de700 1 -- 172.21.15.144:0/4280965364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff260 msgr2=0x7fe6600ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:32.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.116+0000 7fe6656de700 1 --2- 172.21.15.144:0/4280965364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff260 0x7fe6600ff630 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fe654009910 tx=0x7fe654009c20 comp rx=0 tx=0).stop 2024-09-10T11:45:32.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.116+0000 7fe6656de700 1 -- 172.21.15.144:0/4280965364 shutdown_connections 2024-09-10T11:45:32.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.116+0000 7fe6656de700 1 --2- 172.21.15.144:0/4280965364 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600ffb70 0x7fe66010a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.116+0000 7fe6656de700 1 --2- 172.21.15.144:0/4280965364 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff260 0x7fe6600ff630 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.116+0000 7fe6656de700 1 -- 172.21.15.144:0/4280965364 >> 172.21.15.144:0/4280965364 conn(0x7fe6600747e0 msgr2=0x7fe660074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:32.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.116+0000 7fe6656de700 1 -- 172.21.15.144:0/4280965364 shutdown_connections 2024-09-10T11:45:32.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.116+0000 7fe6656de700 1 -- 172.21.15.144:0/4280965364 wait complete. 2024-09-10T11:45:32.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.117+0000 7fe6656de700 1 Processor -- start 2024-09-10T11:45:32.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.117+0000 7fe6656de700 1 -- start start 2024-09-10T11:45:32.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.117+0000 7fe6656de700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff260 0x7fe660195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:32.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.117+0000 7fe6656de700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600ffb70 0x7fe660196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:32.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.117+0000 7fe6656de700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe660196ae0 con 0x7fe6600ff260 2024-09-10T11:45:32.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.117+0000 7fe6656de700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66019a870 con 0x7fe6600ffb70 2024-09-10T11:45:32.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.118+0000 7fe65effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff260 0x7fe660195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:32.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.118+0000 7fe65effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff260 0x7fe660195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40176/0 (socket says 172.21.15.144:40176) 2024-09-10T11:45:32.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.118+0000 7fe65effd700 1 -- 172.21.15.144:0/90741026 learned_addr learned my addr 172.21.15.144:0/90741026 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:45:32.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.118+0000 7fe65e7fc700 1 --2- 172.21.15.144:0/90741026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600ffb70 0x7fe660196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:32.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.118+0000 7fe65effd700 1 -- 172.21.15.144:0/90741026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600ffb70 msgr2=0x7fe660196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:32.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.118+0000 7fe65effd700 1 --2- 172.21.15.144:0/90741026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600ffb70 0x7fe660196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.118+0000 7fe65effd700 1 -- 172.21.15.144:0/90741026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6540095f0 con 0x7fe6600ff260 2024-09-10T11:45:32.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.118+0000 7fe65effd700 1 --2- 172.21.15.144:0/90741026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff260 0x7fe660195ec0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe654006010 tx=0x7fe65400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:32.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.118+0000 7fe647fff700 1 -- 172.21.15.144:0/90741026 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe65401d070 con 0x7fe6600ff260 2024-09-10T11:45:32.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.118+0000 7fe6656de700 1 -- 172.21.15.144:0/90741026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66019aaf0 con 0x7fe6600ff260 2024-09-10T11:45:32.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.118+0000 7fe647fff700 1 -- 172.21.15.144:0/90741026 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe65400f460 con 0x7fe6600ff260 2024-09-10T11:45:32.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.118+0000 7fe647fff700 1 -- 172.21.15.144:0/90741026 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe654017610 con 0x7fe6600ff260 2024-09-10T11:45:32.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.118+0000 7fe6656de700 1 -- 172.21.15.144:0/90741026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66019afe0 con 0x7fe6600ff260 2024-09-10T11:45:32.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.119+0000 7fe6656de700 1 -- 172.21.15.144:0/90741026 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe660107ce0 con 0x7fe6600ff260 2024-09-10T11:45:32.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.120+0000 7fe647fff700 1 -- 172.21.15.144:0/90741026 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe65400f5d0 con 0x7fe6600ff260 2024-09-10T11:45:32.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.120+0000 7fe647fff700 1 --2- 172.21.15.144:0/90741026 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe64806e200 0x7fe6480706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:32.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.120+0000 7fe647fff700 1 -- 172.21.15.144:0/90741026 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe65408f630 con 0x7fe6600ff260 2024-09-10T11:45:32.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.120+0000 7fe65e7fc700 1 --2- 172.21.15.144:0/90741026 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe64806e200 0x7fe6480706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:32.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.121+0000 7fe65e7fc700 1 --2- 172.21.15.144:0/90741026 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe64806e200 0x7fe6480706b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe6601974e0 tx=0x7fe650009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:32.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.125+0000 7fe647fff700 1 -- 172.21.15.144:0/90741026 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe654058a10 con 0x7fe6600ff260 2024-09-10T11:45:32.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.342+0000 7fe6656de700 1 -- 172.21.15.144:0/90741026 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe66019b2c0 con 0x7fe64806e200 2024-09-10T11:45:32.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.343+0000 7fe647fff700 1 -- 172.21.15.144:0/90741026 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fe66019b2c0 con 0x7fe64806e200 2024-09-10T11:45:32.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.346+0000 7fe6656de700 1 -- 172.21.15.144:0/90741026 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe64806e200 msgr2=0x7fe6480706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:32.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.346+0000 7fe6656de700 1 --2- 172.21.15.144:0/90741026 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe64806e200 0x7fe6480706b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe6601974e0 tx=0x7fe650009190 comp rx=0 tx=0).stop 2024-09-10T11:45:32.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.346+0000 7fe6656de700 1 -- 172.21.15.144:0/90741026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff260 msgr2=0x7fe660195ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:32.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.346+0000 7fe6656de700 1 --2- 172.21.15.144:0/90741026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff260 0x7fe660195ec0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fe654006010 tx=0x7fe65400bd70 comp rx=0 tx=0).stop 2024-09-10T11:45:32.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.346+0000 7fe6656de700 1 -- 172.21.15.144:0/90741026 shutdown_connections 2024-09-10T11:45:32.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.346+0000 7fe6656de700 1 --2- 172.21.15.144:0/90741026 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe64806e200 0x7fe6480706b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.346+0000 7fe6656de700 1 --2- 172.21.15.144:0/90741026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff260 0x7fe660195ec0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.346+0000 7fe6656de700 1 --2- 172.21.15.144:0/90741026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600ffb70 0x7fe660196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.346+0000 7fe6656de700 1 -- 172.21.15.144:0/90741026 >> 172.21.15.144:0/90741026 conn(0x7fe6600747e0 msgr2=0x7fe660104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:32.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.346+0000 7fe6656de700 1 -- 172.21.15.144:0/90741026 shutdown_connections 2024-09-10T11:45:32.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.346+0000 7fe6656de700 1 -- 172.21.15.144:0/90741026 wait complete. 2024-09-10T11:45:32.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.451+0000 7fd28a10f700 1 -- 172.21.15.144:0/3880590762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2840ffd30 msgr2=0x7fd284103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:32.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.451+0000 7fd28a10f700 1 --2- 172.21.15.144:0/3880590762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2840ffd30 0x7fd284103ce0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fd26c009910 tx=0x7fd26c009c20 comp rx=0 tx=0).stop 2024-09-10T11:45:32.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.451+0000 7fd28a10f700 1 -- 172.21.15.144:0/3880590762 shutdown_connections 2024-09-10T11:45:32.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.451+0000 7fd28a10f700 1 --2- 172.21.15.144:0/3880590762 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2840ffd30 0x7fd284103ce0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.451+0000 7fd28a10f700 1 --2- 172.21.15.144:0/3880590762 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2840ff390 0x7fd2840ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.451+0000 7fd28a10f700 1 -- 172.21.15.144:0/3880590762 >> 172.21.15.144:0/3880590762 conn(0x7fd284074f50 msgr2=0x7fd284075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:32.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.451+0000 7fd28a10f700 1 -- 172.21.15.144:0/3880590762 shutdown_connections 2024-09-10T11:45:32.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.451+0000 7fd28a10f700 1 -- 172.21.15.144:0/3880590762 wait complete. 2024-09-10T11:45:32.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.451+0000 7fd28a10f700 1 Processor -- start 2024-09-10T11:45:32.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.452+0000 7fd28a10f700 1 -- start start 2024-09-10T11:45:32.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.452+0000 7fd28a10f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2840ff390 0x7fd28419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:32.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.452+0000 7fd28a10f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2840ffd30 0x7fd28419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:32.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.452+0000 7fd28a10f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28419b520 con 0x7fd2840ffd30 2024-09-10T11:45:32.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.452+0000 7fd28a10f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd284194a60 con 0x7fd2840ff390 2024-09-10T11:45:32.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.452+0000 7fd282ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2840ffd30 0x7fd28419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:32.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.452+0000 7fd282ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2840ffd30 0x7fd28419af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43186/0 (socket says 172.21.15.144:43186) 2024-09-10T11:45:32.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.452+0000 7fd282ffd700 1 -- 172.21.15.144:0/1490352949 learned_addr learned my addr 172.21.15.144:0/1490352949 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:45:32.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.452+0000 7fd2837fe700 1 --2- 172.21.15.144:0/1490352949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2840ff390 0x7fd28419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:32.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.452+0000 7fd282ffd700 1 -- 172.21.15.144:0/1490352949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2840ff390 msgr2=0x7fd28419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:32.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.452+0000 7fd282ffd700 1 --2- 172.21.15.144:0/1490352949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2840ff390 0x7fd28419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.452+0000 7fd282ffd700 1 -- 172.21.15.144:0/1490352949 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd26c0095f0 con 0x7fd2840ffd30 2024-09-10T11:45:32.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.453+0000 7fd282ffd700 1 --2- 172.21.15.144:0/1490352949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2840ffd30 0x7fd28419af20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00bda0 tx=0x7fd26c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:32.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.453+0000 7fd280ff9700 1 -- 172.21.15.144:0/1490352949 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd26c01d070 con 0x7fd2840ffd30 2024-09-10T11:45:32.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.453+0000 7fd280ff9700 1 -- 172.21.15.144:0/1490352949 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd26c00f460 con 0x7fd2840ffd30 2024-09-10T11:45:32.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.453+0000 7fd28a10f700 1 -- 172.21.15.144:0/1490352949 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd284194ce0 con 0x7fd2840ffd30 2024-09-10T11:45:32.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.453+0000 7fd280ff9700 1 -- 172.21.15.144:0/1490352949 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd26c017630 con 0x7fd2840ffd30 2024-09-10T11:45:32.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.453+0000 7fd28a10f700 1 -- 172.21.15.144:0/1490352949 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2841951d0 con 0x7fd2840ffd30 2024-09-10T11:45:32.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.454+0000 7fd28a10f700 1 -- 172.21.15.144:0/1490352949 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd284107600 con 0x7fd2840ffd30 2024-09-10T11:45:32.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.454+0000 7fd280ff9700 1 -- 172.21.15.144:0/1490352949 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd26c00fab0 con 0x7fd2840ffd30 2024-09-10T11:45:32.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.455+0000 7fd280ff9700 1 --2- 172.21.15.144:0/1490352949 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd27006e200 0x7fd2700706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:32.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.455+0000 7fd280ff9700 1 -- 172.21.15.144:0/1490352949 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd26c08f510 con 0x7fd2840ffd30 2024-09-10T11:45:32.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.455+0000 7fd2837fe700 1 --2- 172.21.15.144:0/1490352949 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd27006e200 0x7fd2700706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:32.458 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.455+0000 7fd2837fe700 1 --2- 172.21.15.144:0/1490352949 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd27006e200 0x7fd2700706b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00f440 tx=0x7fd27c0079e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:32.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.459+0000 7fd280ff9700 1 -- 172.21.15.144:0/1490352949 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd26c05c4a0 con 0x7fd2840ffd30 2024-09-10T11:45:32.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.668+0000 7fd28a10f700 1 -- 172.21.15.144:0/1490352949 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2841953f0 con 0x7fd27006e200 2024-09-10T11:45:32.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.677+0000 7fd280ff9700 1 -- 172.21.15.144:0/1490352949 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fd2841953f0 con 0x7fd27006e200 2024-09-10T11:45:32.678 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:45:32.678 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (13m) 7m ago 24m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:45:32.678 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (24m) 7m ago 24m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:45:32.678 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (23m) 7m ago 23m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:45:32.679 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (11m) 7m ago 24m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:45:32.679 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (11m) 7m ago 23m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:45:32.679 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (13m) 7m ago 24m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:45:32.679 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (7m) 7m ago 18m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:45:32.679 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (7m) 7m ago 18m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:45:32.679 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (18m) 7m ago 18m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:45:32.679 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (7m) 7m ago 18m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:45:32.679 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (15m) 7m ago 25m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:45:32.679 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (14m) 7m ago 23m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:45:32.679 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (12m) 7m ago 26m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:45:32.679 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (12m) 7m ago 23m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:45:32.679 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (14m) 7m ago 24m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:45:32.679 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (14m) 7m ago 23m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:45:32.679 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (11m) 7m ago 22m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:45:32.680 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (10m) 7m ago 22m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:45:32.680 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (9m) 7m ago 21m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:45:32.680 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (9m) 7m ago 21m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:45:32.680 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (8m) 7m ago 20m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:45:32.680 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (8m) 7m ago 20m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:45:32.680 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (14m) 7m ago 23m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:45:32.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.679+0000 7fd28a10f700 1 -- 172.21.15.144:0/1490352949 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd27006e200 msgr2=0x7fd2700706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:32.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.679+0000 7fd28a10f700 1 --2- 172.21.15.144:0/1490352949 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd27006e200 0x7fd2700706b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd27c00f440 tx=0x7fd27c0079e0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.679+0000 7fd28a10f700 1 -- 172.21.15.144:0/1490352949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2840ffd30 msgr2=0x7fd28419af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:32.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.679+0000 7fd28a10f700 1 --2- 172.21.15.144:0/1490352949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2840ffd30 0x7fd28419af20 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00bda0 tx=0x7fd26c00bdd0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.679+0000 7fd28a10f700 1 -- 172.21.15.144:0/1490352949 shutdown_connections 2024-09-10T11:45:32.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.679+0000 7fd28a10f700 1 --2- 172.21.15.144:0/1490352949 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd27006e200 0x7fd2700706b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.679+0000 7fd28a10f700 1 --2- 172.21.15.144:0/1490352949 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2840ff390 0x7fd28419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.679+0000 7fd28a10f700 1 --2- 172.21.15.144:0/1490352949 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2840ffd30 0x7fd28419af20 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.679+0000 7fd28a10f700 1 -- 172.21.15.144:0/1490352949 >> 172.21.15.144:0/1490352949 conn(0x7fd284074f50 msgr2=0x7fd2840fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:32.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.679+0000 7fd28a10f700 1 -- 172.21.15.144:0/1490352949 shutdown_connections 2024-09-10T11:45:32.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.679+0000 7fd28a10f700 1 -- 172.21.15.144:0/1490352949 wait complete. 2024-09-10T11:45:32.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.783+0000 7f38ccceb700 1 -- 172.21.15.144:0/4177061320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c80ffd30 msgr2=0x7f38c8103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:32.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.783+0000 7f38ccceb700 1 --2- 172.21.15.144:0/4177061320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c80ffd30 0x7f38c8103ce0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009910 tx=0x7f38b8009c20 comp rx=0 tx=0).stop 2024-09-10T11:45:32.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.783+0000 7f38ccceb700 1 -- 172.21.15.144:0/4177061320 shutdown_connections 2024-09-10T11:45:32.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.783+0000 7f38ccceb700 1 --2- 172.21.15.144:0/4177061320 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c80ffd30 0x7f38c8103ce0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.783+0000 7f38ccceb700 1 --2- 172.21.15.144:0/4177061320 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c80ff390 0x7f38c80ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.783+0000 7f38ccceb700 1 -- 172.21.15.144:0/4177061320 >> 172.21.15.144:0/4177061320 conn(0x7f38c8074f50 msgr2=0x7f38c8075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:32.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.783+0000 7f38ccceb700 1 -- 172.21.15.144:0/4177061320 shutdown_connections 2024-09-10T11:45:32.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.783+0000 7f38ccceb700 1 -- 172.21.15.144:0/4177061320 wait complete. 2024-09-10T11:45:32.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.784+0000 7f38ccceb700 1 Processor -- start 2024-09-10T11:45:32.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.784+0000 7f38ccceb700 1 -- start start 2024-09-10T11:45:32.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.784+0000 7f38ccceb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c80ff390 0x7f38c819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:32.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.784+0000 7f38ccceb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c80ffd30 0x7f38c819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:32.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.784+0000 7f38ccceb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c819b5b0 con 0x7f38c80ff390 2024-09-10T11:45:32.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.784+0000 7f38ccceb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c8194a60 con 0x7f38c80ffd30 2024-09-10T11:45:32.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.785+0000 7f38c659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c80ff390 0x7f38c819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:32.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.785+0000 7f38c659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c80ff390 0x7f38c819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43192/0 (socket says 172.21.15.144:43192) 2024-09-10T11:45:32.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.785+0000 7f38c659c700 1 -- 172.21.15.144:0/3182932009 learned_addr learned my addr 172.21.15.144:0/3182932009 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:45:32.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.785+0000 7f38c5d9b700 1 --2- 172.21.15.144:0/3182932009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c80ffd30 0x7f38c819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:32.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.785+0000 7f38c659c700 1 -- 172.21.15.144:0/3182932009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c80ffd30 msgr2=0x7f38c819af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:32.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.785+0000 7f38c659c700 1 --2- 172.21.15.144:0/3182932009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c80ffd30 0x7f38c819af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:32.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.785+0000 7f38c659c700 1 -- 172.21.15.144:0/3182932009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b80095f0 con 0x7f38c80ff390 2024-09-10T11:45:32.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.785+0000 7f38c659c700 1 --2- 172.21.15.144:0/3182932009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c80ff390 0x7f38c819a9e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009970 tx=0x7f38b0009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:32.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.785+0000 7f38bf7fe700 1 -- 172.21.15.144:0/3182932009 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38b0012070 con 0x7f38c80ff390 2024-09-10T11:45:32.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.785+0000 7f38bf7fe700 1 -- 172.21.15.144:0/3182932009 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38b0007af0 con 0x7f38c80ff390 2024-09-10T11:45:32.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.785+0000 7f38ccceb700 1 -- 172.21.15.144:0/3182932009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c8194ce0 con 0x7f38c80ff390 2024-09-10T11:45:32.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.785+0000 7f38bf7fe700 1 -- 172.21.15.144:0/3182932009 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38b0017bf0 con 0x7f38c80ff390 2024-09-10T11:45:32.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.785+0000 7f38ccceb700 1 -- 172.21.15.144:0/3182932009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c8195200 con 0x7f38c80ff390 2024-09-10T11:45:32.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.786+0000 7f38ccceb700 1 -- 172.21.15.144:0/3182932009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c804ea50 con 0x7f38c80ff390 2024-09-10T11:45:32.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.787+0000 7f38bf7fe700 1 -- 172.21.15.144:0/3182932009 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f38b0007c60 con 0x7f38c80ff390 2024-09-10T11:45:32.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.787+0000 7f38bf7fe700 1 --2- 172.21.15.144:0/3182932009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38b406e200 0x7f38b40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:32.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.787+0000 7f38bf7fe700 1 -- 172.21.15.144:0/3182932009 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f38b008f5a0 con 0x7f38c80ff390 2024-09-10T11:45:32.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.787+0000 7f38c5d9b700 1 --2- 172.21.15.144:0/3182932009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38b406e200 0x7f38b40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:32.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.788+0000 7f38c5d9b700 1 --2- 172.21.15.144:0/3182932009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38b406e200 0x7f38b40706b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38b8006010 tx=0x7f38b801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:32.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:32.792+0000 7f38bf7fe700 1 -- 172.21.15.144:0/3182932009 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f38b005c560 con 0x7f38c80ff390 2024-09-10T11:45:33.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.084+0000 7f38ccceb700 1 -- 172.21.15.144:0/3182932009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f38c81956d0 con 0x7f38c80ff390 2024-09-10T11:45:33.086 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:32 smithi144 ceph-mon[87436]: pgmap v406: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:33.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.084+0000 7f38bf7fe700 1 -- 172.21.15.144:0/3182932009 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f38b005c0f0 con 0x7f38c80ff390 2024-09-10T11:45:33.086 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:45:33.086 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:45:33.086 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:45:33.086 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:45:33.086 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:45:33.087 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:45:33.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:45:33.087 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:45:33.087 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:45:33.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:45:33.087 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:45:33.087 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:45:33.087 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:45:33.087 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:45:33.087 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:45:33.087 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:45:33.087 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:45:33.087 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:45:33.087 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:45:33.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.087+0000 7f38ccceb700 1 -- 172.21.15.144:0/3182932009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38b406e200 msgr2=0x7f38b40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:33.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.087+0000 7f38ccceb700 1 --2- 172.21.15.144:0/3182932009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38b406e200 0x7f38b40706b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f38b8006010 tx=0x7f38b801a040 comp rx=0 tx=0).stop 2024-09-10T11:45:33.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.087+0000 7f38ccceb700 1 -- 172.21.15.144:0/3182932009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c80ff390 msgr2=0x7f38c819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:33.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.087+0000 7f38ccceb700 1 --2- 172.21.15.144:0/3182932009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c80ff390 0x7f38c819a9e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f38b0009970 tx=0x7f38b0009c80 comp rx=0 tx=0).stop 2024-09-10T11:45:33.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.087+0000 7f38ccceb700 1 -- 172.21.15.144:0/3182932009 shutdown_connections 2024-09-10T11:45:33.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.087+0000 7f38ccceb700 1 --2- 172.21.15.144:0/3182932009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38b406e200 0x7f38b40706b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.087+0000 7f38ccceb700 1 --2- 172.21.15.144:0/3182932009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38c80ff390 0x7f38c819a9e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.087+0000 7f38ccceb700 1 --2- 172.21.15.144:0/3182932009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38c80ffd30 0x7f38c819af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.087+0000 7f38ccceb700 1 -- 172.21.15.144:0/3182932009 >> 172.21.15.144:0/3182932009 conn(0x7f38c8074f50 msgr2=0x7f38c80fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:33.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.087+0000 7f38ccceb700 1 -- 172.21.15.144:0/3182932009 shutdown_connections 2024-09-10T11:45:33.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.087+0000 7f38ccceb700 1 -- 172.21.15.144:0/3182932009 wait complete. 2024-09-10T11:45:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:32 smithi178 ceph-mon[65940]: pgmap v406: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:33.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.192+0000 7f5c59af2700 1 -- 172.21.15.144:0/782819598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c54101cb0 msgr2=0x7f5c5410a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:33.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.192+0000 7f5c59af2700 1 --2- 172.21.15.144:0/782819598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c54101cb0 0x7f5c5410a1b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5c48009910 tx=0x7f5c48009c20 comp rx=0 tx=0).stop 2024-09-10T11:45:33.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.192+0000 7f5c59af2700 1 -- 172.21.15.144:0/782819598 shutdown_connections 2024-09-10T11:45:33.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.192+0000 7f5c59af2700 1 --2- 172.21.15.144:0/782819598 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c54101cb0 0x7f5c5410a1b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.192+0000 7f5c59af2700 1 --2- 172.21.15.144:0/782819598 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c541013a0 0x7f5c54101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.192+0000 7f5c59af2700 1 -- 172.21.15.144:0/782819598 >> 172.21.15.144:0/782819598 conn(0x7f5c540fac00 msgr2=0x7f5c540fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:33.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.192+0000 7f5c59af2700 1 -- 172.21.15.144:0/782819598 shutdown_connections 2024-09-10T11:45:33.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.192+0000 7f5c59af2700 1 -- 172.21.15.144:0/782819598 wait complete. 2024-09-10T11:45:33.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.193+0000 7f5c59af2700 1 Processor -- start 2024-09-10T11:45:33.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.193+0000 7f5c59af2700 1 -- start start 2024-09-10T11:45:33.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.193+0000 7f5c59af2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c541013a0 0x7f5c54195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:33.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.193+0000 7f5c59af2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c54101cb0 0x7f5c54196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:33.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.193+0000 7f5c59af2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c54196960 con 0x7f5c54101cb0 2024-09-10T11:45:33.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.193+0000 7f5c59af2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c5419a6f0 con 0x7f5c541013a0 2024-09-10T11:45:33.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.193+0000 7f5c52ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c54101cb0 0x7f5c54196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:33.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.193+0000 7f5c52ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c54101cb0 0x7f5c54196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43216/0 (socket says 172.21.15.144:43216) 2024-09-10T11:45:33.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.193+0000 7f5c52ffd700 1 -- 172.21.15.144:0/24316316 learned_addr learned my addr 172.21.15.144:0/24316316 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:45:33.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.193+0000 7f5c537fe700 1 --2- 172.21.15.144:0/24316316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c541013a0 0x7f5c54195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:33.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.193+0000 7f5c52ffd700 1 -- 172.21.15.144:0/24316316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c541013a0 msgr2=0x7f5c54195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:33.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.193+0000 7f5c52ffd700 1 --2- 172.21.15.144:0/24316316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c541013a0 0x7f5c54195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.193+0000 7f5c52ffd700 1 -- 172.21.15.144:0/24316316 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c480095f0 con 0x7f5c54101cb0 2024-09-10T11:45:33.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.194+0000 7f5c52ffd700 1 --2- 172.21.15.144:0/24316316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c54101cb0 0x7f5c54196280 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006010 tx=0x7f5c4800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:33.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.194+0000 7f5c50ff9700 1 -- 172.21.15.144:0/24316316 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c4801d070 con 0x7f5c54101cb0 2024-09-10T11:45:33.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.194+0000 7f5c50ff9700 1 -- 172.21.15.144:0/24316316 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c48005dc0 con 0x7f5c54101cb0 2024-09-10T11:45:33.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.194+0000 7f5c59af2700 1 -- 172.21.15.144:0/24316316 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c5419a970 con 0x7f5c54101cb0 2024-09-10T11:45:33.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.194+0000 7f5c50ff9700 1 -- 172.21.15.144:0/24316316 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c48017400 con 0x7f5c54101cb0 2024-09-10T11:45:33.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.194+0000 7f5c59af2700 1 -- 172.21.15.144:0/24316316 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c5419ae60 con 0x7f5c54101cb0 2024-09-10T11:45:33.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.195+0000 7f5c59af2700 1 -- 172.21.15.144:0/24316316 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c54107820 con 0x7f5c54101cb0 2024-09-10T11:45:33.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.195+0000 7f5c50ff9700 1 -- 172.21.15.144:0/24316316 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5c48003a60 con 0x7f5c54101cb0 2024-09-10T11:45:33.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.196+0000 7f5c50ff9700 1 --2- 172.21.15.144:0/24316316 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5c3c06e200 0x7f5c3c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:33.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.196+0000 7f5c50ff9700 1 -- 172.21.15.144:0/24316316 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5c4808f500 con 0x7f5c54101cb0 2024-09-10T11:45:33.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.196+0000 7f5c537fe700 1 --2- 172.21.15.144:0/24316316 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5c3c06e200 0x7f5c3c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:33.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.196+0000 7f5c537fe700 1 --2- 172.21.15.144:0/24316316 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5c3c06e200 0x7f5c3c0706b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009550 tx=0x7f5c440091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:33.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.200+0000 7f5c50ff9700 1 -- 172.21.15.144:0/24316316 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5c480588e0 con 0x7f5c54101cb0 2024-09-10T11:45:33.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.449+0000 7f5c59af2700 1 -- 172.21.15.144:0/24316316 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5c5419b140 con 0x7f5c54101cb0 2024-09-10T11:45:33.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.449+0000 7f5c50ff9700 1 -- 172.21.15.144:0/24316316 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f5c4805bfa0 con 0x7f5c54101cb0 2024-09-10T11:45:33.451 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:45:33.451 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:45:33.451 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:45:33.451 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:45:33.451 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:45:33.451 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:45:33.451 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:45:33.451 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:45:33.451 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:45:33.452 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:45:33.452 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:45:33.452 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:45:33.452 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:45:33.452 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:45:33.452 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:45:33.452 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:45:33.452 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:45:33.452 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:45:33.452 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:45:33.452 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:45:33.452 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:45:33.453 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:45:33.453 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:45:33.453 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:45:33.453 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:45:33.453 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:45:33.453 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:45:33.453 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:45:33.453 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:45:33.453 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:45:33.453 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:45:33.453 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:45:33.453 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:45:33.454 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:45:33.454 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:45:33.454 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:45:33.454 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:45:33.454 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:45:33.454 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:45:33.454 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:45:33.454 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:45:33.454 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:45:33.454 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:45:33.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.452+0000 7f5c59af2700 1 -- 172.21.15.144:0/24316316 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5c3c06e200 msgr2=0x7f5c3c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:33.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.452+0000 7f5c59af2700 1 --2- 172.21.15.144:0/24316316 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5c3c06e200 0x7f5c3c0706b0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f5c44009550 tx=0x7f5c440091b0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.452+0000 7f5c59af2700 1 -- 172.21.15.144:0/24316316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c54101cb0 msgr2=0x7f5c54196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:33.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.452+0000 7f5c59af2700 1 --2- 172.21.15.144:0/24316316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c54101cb0 0x7f5c54196280 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5c48006010 tx=0x7f5c4800bd70 comp rx=0 tx=0).stop 2024-09-10T11:45:33.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.452+0000 7f5c59af2700 1 -- 172.21.15.144:0/24316316 shutdown_connections 2024-09-10T11:45:33.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.452+0000 7f5c59af2700 1 --2- 172.21.15.144:0/24316316 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5c3c06e200 0x7f5c3c0706b0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.452+0000 7f5c59af2700 1 --2- 172.21.15.144:0/24316316 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5c541013a0 0x7f5c54195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.452+0000 7f5c59af2700 1 --2- 172.21.15.144:0/24316316 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5c54101cb0 0x7f5c54196280 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.452+0000 7f5c59af2700 1 -- 172.21.15.144:0/24316316 >> 172.21.15.144:0/24316316 conn(0x7f5c540fac00 msgr2=0x7f5c54104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:33.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.452+0000 7f5c59af2700 1 -- 172.21.15.144:0/24316316 shutdown_connections 2024-09-10T11:45:33.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.452+0000 7f5c59af2700 1 -- 172.21.15.144:0/24316316 wait complete. 2024-09-10T11:45:33.456 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:45:33.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.558+0000 7ff66759e700 1 -- 172.21.15.144:0/4158627079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff668100250 msgr2=0x7ff668100620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:33.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.558+0000 7ff66759e700 1 --2- 172.21.15.144:0/4158627079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff668100250 0x7ff668100620 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7ff65c009910 tx=0x7ff65c009c20 comp rx=0 tx=0).stop 2024-09-10T11:45:33.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.558+0000 7ff66759e700 1 -- 172.21.15.144:0/4158627079 shutdown_connections 2024-09-10T11:45:33.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.558+0000 7ff66759e700 1 --2- 172.21.15.144:0/4158627079 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff668100bf0 0x7ff668104bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.558+0000 7ff66759e700 1 --2- 172.21.15.144:0/4158627079 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff668100250 0x7ff668100620 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.558+0000 7ff66759e700 1 -- 172.21.15.144:0/4158627079 >> 172.21.15.144:0/4158627079 conn(0x7ff6680fbb10 msgr2=0x7ff6680fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:33.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.558+0000 7ff66759e700 1 -- 172.21.15.144:0/4158627079 shutdown_connections 2024-09-10T11:45:33.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.558+0000 7ff66759e700 1 -- 172.21.15.144:0/4158627079 wait complete. 2024-09-10T11:45:33.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.559+0000 7ff66759e700 1 Processor -- start 2024-09-10T11:45:33.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.559+0000 7ff66759e700 1 -- start start 2024-09-10T11:45:33.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.559+0000 7ff66759e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff668100250 0x7ff66819ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:33.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.559+0000 7ff66759e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff668100bf0 0x7ff66819bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:33.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.559+0000 7ff66759e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66819c5d0 con 0x7ff668100bf0 2024-09-10T11:45:33.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.559+0000 7ff66759e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff668195a80 con 0x7ff668100250 2024-09-10T11:45:33.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.560+0000 7ff665d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff668100bf0 0x7ff66819bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:33.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.560+0000 7ff665d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff668100bf0 0x7ff66819bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43232/0 (socket says 172.21.15.144:43232) 2024-09-10T11:45:33.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.560+0000 7ff665d9b700 1 -- 172.21.15.144:0/1251349604 learned_addr learned my addr 172.21.15.144:0/1251349604 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:45:33.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.560+0000 7ff66659c700 1 --2- 172.21.15.144:0/1251349604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff668100250 0x7ff66819ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:33.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.560+0000 7ff665d9b700 1 -- 172.21.15.144:0/1251349604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff668100250 msgr2=0x7ff66819ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:33.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.560+0000 7ff665d9b700 1 --2- 172.21.15.144:0/1251349604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff668100250 0x7ff66819ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.560+0000 7ff665d9b700 1 -- 172.21.15.144:0/1251349604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff65c0095f0 con 0x7ff668100bf0 2024-09-10T11:45:33.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.560+0000 7ff665d9b700 1 --2- 172.21.15.144:0/1251349604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff668100bf0 0x7ff66819bf40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff65800c8f0 tx=0x7ff65800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:33.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.560+0000 7ff6577fe700 1 -- 172.21.15.144:0/1251349604 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff658007a50 con 0x7ff668100bf0 2024-09-10T11:45:33.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.560+0000 7ff6577fe700 1 -- 172.21.15.144:0/1251349604 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff658007bb0 con 0x7ff668100bf0 2024-09-10T11:45:33.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.560+0000 7ff66759e700 1 -- 172.21.15.144:0/1251349604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff668195d60 con 0x7ff668100bf0 2024-09-10T11:45:33.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.560+0000 7ff6577fe700 1 -- 172.21.15.144:0/1251349604 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff658018630 con 0x7ff668100bf0 2024-09-10T11:45:33.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.560+0000 7ff66759e700 1 -- 172.21.15.144:0/1251349604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6681962b0 con 0x7ff668100bf0 2024-09-10T11:45:33.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.561+0000 7ff66759e700 1 -- 172.21.15.144:0/1251349604 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff668108560 con 0x7ff668100bf0 2024-09-10T11:45:33.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.562+0000 7ff6577fe700 1 -- 172.21.15.144:0/1251349604 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff658007d20 con 0x7ff668100bf0 2024-09-10T11:45:33.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.562+0000 7ff6577fe700 1 --2- 172.21.15.144:0/1251349604 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff65006e200 0x7ff6500706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:33.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.562+0000 7ff6577fe700 1 -- 172.21.15.144:0/1251349604 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff65808e060 con 0x7ff668100bf0 2024-09-10T11:45:33.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.562+0000 7ff66659c700 1 --2- 172.21.15.144:0/1251349604 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff65006e200 0x7ff6500706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:33.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.563+0000 7ff66659c700 1 --2- 172.21.15.144:0/1251349604 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff65006e200 0x7ff6500706b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00b5c0 tx=0x7ff65c005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:33.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.567+0000 7ff6577fe700 1 -- 172.21.15.144:0/1251349604 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff65805af70 con 0x7ff668100bf0 2024-09-10T11:45:33.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.797+0000 7ff66759e700 1 -- 172.21.15.144:0/1251349604 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6681965a0 con 0x7ff65006e200 2024-09-10T11:45:33.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.799+0000 7ff6577fe700 1 -- 172.21.15.144:0/1251349604 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff6681965a0 con 0x7ff65006e200 2024-09-10T11:45:33.800 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:45:33.800 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:45:33.800 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:45:33.800 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:45:33.800 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:45:33.800 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:45:33.800 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:45:33.801 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:45:33.801 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:45:33.801 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:45:33.801 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:45:33.801 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:45:33.801 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:45:33.801 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:45:33.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.801+0000 7ff66759e700 1 -- 172.21.15.144:0/1251349604 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff65006e200 msgr2=0x7ff6500706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:33.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.801+0000 7ff66759e700 1 --2- 172.21.15.144:0/1251349604 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff65006e200 0x7ff6500706b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff65c00b5c0 tx=0x7ff65c005960 comp rx=0 tx=0).stop 2024-09-10T11:45:33.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.801+0000 7ff66759e700 1 -- 172.21.15.144:0/1251349604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff668100bf0 msgr2=0x7ff66819bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:33.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.801+0000 7ff66759e700 1 --2- 172.21.15.144:0/1251349604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff668100bf0 0x7ff66819bf40 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7ff65800c8f0 tx=0x7ff65800ccb0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.801+0000 7ff66759e700 1 -- 172.21.15.144:0/1251349604 shutdown_connections 2024-09-10T11:45:33.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.801+0000 7ff66759e700 1 --2- 172.21.15.144:0/1251349604 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff65006e200 0x7ff6500706b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.801+0000 7ff66759e700 1 --2- 172.21.15.144:0/1251349604 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff668100250 0x7ff66819ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.801+0000 7ff66759e700 1 --2- 172.21.15.144:0/1251349604 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff668100bf0 0x7ff66819bf40 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.802+0000 7ff66759e700 1 -- 172.21.15.144:0/1251349604 >> 172.21.15.144:0/1251349604 conn(0x7ff6680fbb10 msgr2=0x7ff6680fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:33.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.802+0000 7ff66759e700 1 -- 172.21.15.144:0/1251349604 shutdown_connections 2024-09-10T11:45:33.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.802+0000 7ff66759e700 1 -- 172.21.15.144:0/1251349604 wait complete. 2024-09-10T11:45:33.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.905+0000 7fca20da1700 1 -- 172.21.15.144:0/2833445692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca1c0683d0 msgr2=0x7fca1c068840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:33.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.905+0000 7fca20da1700 1 --2- 172.21.15.144:0/2833445692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca1c0683d0 0x7fca1c068840 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fca04009910 tx=0x7fca04009c20 comp rx=0 tx=0).stop 2024-09-10T11:45:33.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.905+0000 7fca20da1700 1 -- 172.21.15.144:0/2833445692 shutdown_connections 2024-09-10T11:45:33.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.905+0000 7fca20da1700 1 --2- 172.21.15.144:0/2833445692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca1c0683d0 0x7fca1c068840 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.905+0000 7fca20da1700 1 --2- 172.21.15.144:0/2833445692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca1c105f90 0x7fca1c106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.905+0000 7fca20da1700 1 -- 172.21.15.144:0/2833445692 >> 172.21.15.144:0/2833445692 conn(0x7fca1c074d00 msgr2=0x7fca1c075100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:33.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.905+0000 7fca20da1700 1 -- 172.21.15.144:0/2833445692 shutdown_connections 2024-09-10T11:45:33.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.905+0000 7fca20da1700 1 -- 172.21.15.144:0/2833445692 wait complete. 2024-09-10T11:45:33.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.906+0000 7fca20da1700 1 Processor -- start 2024-09-10T11:45:33.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.906+0000 7fca20da1700 1 -- start start 2024-09-10T11:45:33.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.906+0000 7fca20da1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca1c0683d0 0x7fca1c0ff040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:33.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.906+0000 7fca20da1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca1c105f90 0x7fca1c0ff580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:33.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.906+0000 7fca20da1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c0ffac0 con 0x7fca1c0683d0 2024-09-10T11:45:33.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.906+0000 7fca20da1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca1c0ffc00 con 0x7fca1c105f90 2024-09-10T11:45:33.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.906+0000 7fca1b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca1c0683d0 0x7fca1c0ff040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:33.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.906+0000 7fca1b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca1c0683d0 0x7fca1c0ff040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:43250/0 (socket says 172.21.15.144:43250) 2024-09-10T11:45:33.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.906+0000 7fca1b7fe700 1 -- 172.21.15.144:0/765749147 learned_addr learned my addr 172.21.15.144:0/765749147 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:45:33.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.906+0000 7fca1affd700 1 --2- 172.21.15.144:0/765749147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca1c105f90 0x7fca1c0ff580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:33.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.906+0000 7fca1b7fe700 1 -- 172.21.15.144:0/765749147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca1c105f90 msgr2=0x7fca1c0ff580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:33.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.906+0000 7fca1b7fe700 1 --2- 172.21.15.144:0/765749147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca1c105f90 0x7fca1c0ff580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:33.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.906+0000 7fca1b7fe700 1 -- 172.21.15.144:0/765749147 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca040095f0 con 0x7fca1c0683d0 2024-09-10T11:45:33.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.907+0000 7fca1b7fe700 1 --2- 172.21.15.144:0/765749147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca1c0683d0 0x7fca1c0ff040 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fca1400ba50 tx=0x7fca1400be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:33.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.907+0000 7fca18ff9700 1 -- 172.21.15.144:0/765749147 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca1400ca00 con 0x7fca1c0683d0 2024-09-10T11:45:33.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.907+0000 7fca20da1700 1 -- 172.21.15.144:0/765749147 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca1c1a1e30 con 0x7fca1c0683d0 2024-09-10T11:45:33.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.907+0000 7fca18ff9700 1 -- 172.21.15.144:0/765749147 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fca1400cb60 con 0x7fca1c0683d0 2024-09-10T11:45:33.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.907+0000 7fca18ff9700 1 -- 172.21.15.144:0/765749147 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca140075a0 con 0x7fca1c0683d0 2024-09-10T11:45:33.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.907+0000 7fca20da1700 1 -- 172.21.15.144:0/765749147 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca1c1a2240 con 0x7fca1c0683d0 2024-09-10T11:45:33.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.908+0000 7fca20da1700 1 -- 172.21.15.144:0/765749147 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca1c108490 con 0x7fca1c0683d0 2024-09-10T11:45:33.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.908+0000 7fca18ff9700 1 -- 172.21.15.144:0/765749147 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fca1400ccd0 con 0x7fca1c0683d0 2024-09-10T11:45:33.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.909+0000 7fca18ff9700 1 --2- 172.21.15.144:0/765749147 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fca0806e200 0x7fca080706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:45:33.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.909+0000 7fca18ff9700 1 -- 172.21.15.144:0/765749147 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fca14006e40 con 0x7fca1c0683d0 2024-09-10T11:45:33.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.909+0000 7fca1affd700 1 --2- 172.21.15.144:0/765749147 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fca0806e200 0x7fca080706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:45:33.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.909+0000 7fca1affd700 1 --2- 172.21.15.144:0/765749147 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fca0806e200 0x7fca080706b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fca04005f50 tx=0x7fca0401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:45:33.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:33.913+0000 7fca18ff9700 1 -- 172.21.15.144:0/765749147 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fca14056cc0 con 0x7fca1c0683d0 2024-09-10T11:45:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:33 smithi178 ceph-mon[65940]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:33 smithi178 ceph-mon[65940]: from='client.34796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:33 smithi178 ceph-mon[65940]: from='client.34800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:33 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3182932009' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:45:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:33 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/24316316' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:45:34.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:33 smithi144 ceph-mon[87436]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:34.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:33 smithi144 ceph-mon[87436]: from='client.34796 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:34.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:33 smithi144 ceph-mon[87436]: from='client.34800 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:34.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:33 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3182932009' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:45:34.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:33 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/24316316' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:45:34.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:34.199+0000 7fca20da1700 1 -- 172.21.15.144:0/765749147 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fca1c04f2a0 con 0x7fca1c0683d0 2024-09-10T11:45:34.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:34.200+0000 7fca18ff9700 1 -- 172.21.15.144:0/765749147 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fca1405a380 con 0x7fca1c0683d0 2024-09-10T11:45:34.201 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:45:34.201 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:45:34.201 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:45:34.201 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:45:34.201 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:45:34.201 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:45:34.201 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:45:34.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:34.202+0000 7fca20da1700 1 -- 172.21.15.144:0/765749147 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fca0806e200 msgr2=0x7fca080706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:34.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:34.202+0000 7fca20da1700 1 --2- 172.21.15.144:0/765749147 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fca0806e200 0x7fca080706b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fca04005f50 tx=0x7fca0401a040 comp rx=0 tx=0).stop 2024-09-10T11:45:34.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:34.202+0000 7fca20da1700 1 -- 172.21.15.144:0/765749147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca1c0683d0 msgr2=0x7fca1c0ff040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:45:34.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:34.202+0000 7fca20da1700 1 --2- 172.21.15.144:0/765749147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca1c0683d0 0x7fca1c0ff040 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fca1400ba50 tx=0x7fca1400be10 comp rx=0 tx=0).stop 2024-09-10T11:45:34.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:34.202+0000 7fca20da1700 1 -- 172.21.15.144:0/765749147 shutdown_connections 2024-09-10T11:45:34.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:34.202+0000 7fca20da1700 1 --2- 172.21.15.144:0/765749147 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fca0806e200 0x7fca080706b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:34.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:34.202+0000 7fca20da1700 1 --2- 172.21.15.144:0/765749147 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fca1c0683d0 0x7fca1c0ff040 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:34.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:34.202+0000 7fca20da1700 1 --2- 172.21.15.144:0/765749147 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fca1c105f90 0x7fca1c0ff580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:45:34.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:34.202+0000 7fca20da1700 1 -- 172.21.15.144:0/765749147 >> 172.21.15.144:0/765749147 conn(0x7fca1c074d00 msgr2=0x7fca1c0fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:45:34.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:34.202+0000 7fca20da1700 1 -- 172.21.15.144:0/765749147 shutdown_connections 2024-09-10T11:45:34.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:45:34.202+0000 7fca20da1700 1 -- 172.21.15.144:0/765749147 wait complete. 2024-09-10T11:45:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:34 smithi178 ceph-mon[65940]: pgmap v407: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:34 smithi178 ceph-mon[65940]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:34 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/765749147' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:45:35.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:34 smithi144 ceph-mon[87436]: pgmap v407: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:34 smithi144 ceph-mon[87436]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:45:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:34 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/765749147' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:45:37.105 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:36 smithi178 ceph-mon[65940]: pgmap v408: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:36 smithi144 ceph-mon[87436]: pgmap v408: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:38.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:38 smithi144 ceph-mon[87436]: pgmap v409: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:38 smithi178 ceph-mon[65940]: pgmap v409: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:40 smithi178 ceph-mon[65940]: pgmap v410: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:40 smithi144 ceph-mon[87436]: pgmap v410: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:43.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:42 smithi144 ceph-mon[87436]: pgmap v411: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:43.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:42 smithi178 ceph-mon[65940]: pgmap v411: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:45:44.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:45:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:44 smithi178 ceph-mon[65940]: pgmap v412: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:45.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:44 smithi144 ceph-mon[87436]: pgmap v412: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:46 smithi178 ceph-mon[65940]: pgmap v413: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:46 smithi144 ceph-mon[87436]: pgmap v413: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:48 smithi178 ceph-mon[65940]: pgmap v414: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:49.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:48 smithi144 ceph-mon[87436]: pgmap v414: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:50 smithi178 ceph-mon[65940]: pgmap v415: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:51.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:50 smithi144 ceph-mon[87436]: pgmap v415: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:53.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:52 smithi144 ceph-mon[87436]: pgmap v416: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:53.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:52 smithi178 ceph-mon[65940]: pgmap v416: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:54.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:53 smithi178 ceph-mon[65940]: pgmap v417: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:54.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:53 smithi144 ceph-mon[87436]: pgmap v417: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:56 smithi178 ceph-mon[65940]: pgmap v418: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:56 smithi144 ceph-mon[87436]: pgmap v418: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:58 smithi178 ceph-mon[65940]: pgmap v419: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:45:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:45:59.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:58 smithi144 ceph-mon[87436]: pgmap v419: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:45:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:45:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:46:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:00 smithi178 ceph-mon[65940]: pgmap v420: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:01.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:00 smithi144 ceph-mon[87436]: pgmap v420: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:03.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:02 smithi144 ceph-mon[87436]: pgmap v421: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:03.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:02 smithi178 ceph-mon[65940]: pgmap v421: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:04.324 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.321+0000 7f2f0bd86700 1 -- 172.21.15.144:0/2454625874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f041003d0 msgr2=0x7f2f041007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:04.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.321+0000 7f2f0bd86700 1 --2- 172.21.15.144:0/2454625874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f041003d0 0x7f2f041007a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2f00009910 tx=0x7f2f00009c20 comp rx=0 tx=0).stop 2024-09-10T11:46:04.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.321+0000 7f2f0bd86700 1 -- 172.21.15.144:0/2454625874 shutdown_connections 2024-09-10T11:46:04.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.321+0000 7f2f0bd86700 1 --2- 172.21.15.144:0/2454625874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f04100d70 0x7f2f04104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:04.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.321+0000 7f2f0bd86700 1 --2- 172.21.15.144:0/2454625874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f041003d0 0x7f2f041007a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:04.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.321+0000 7f2f0bd86700 1 -- 172.21.15.144:0/2454625874 >> 172.21.15.144:0/2454625874 conn(0x7f2f040fbc90 msgr2=0x7f2f040fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:04.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.321+0000 7f2f0bd86700 1 -- 172.21.15.144:0/2454625874 shutdown_connections 2024-09-10T11:46:04.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.321+0000 7f2f0bd86700 1 -- 172.21.15.144:0/2454625874 wait complete. 2024-09-10T11:46:04.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.322+0000 7f2f0bd86700 1 Processor -- start 2024-09-10T11:46:04.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.322+0000 7f2f0bd86700 1 -- start start 2024-09-10T11:46:04.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.322+0000 7f2f0bd86700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f041003d0 0x7f2f04195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:04.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.322+0000 7f2f0bd86700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f04100d70 0x7f2f041962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:04.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.322+0000 7f2f0bd86700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f04196980 con 0x7f2f04100d70 2024-09-10T11:46:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.322+0000 7f2f0bd86700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0419a710 con 0x7f2f041003d0 2024-09-10T11:46:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.322+0000 7f2f09321700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f04100d70 0x7f2f041962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.322+0000 7f2f09321700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f04100d70 0x7f2f041962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:32864/0 (socket says 172.21.15.144:32864) 2024-09-10T11:46:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.322+0000 7f2f09321700 1 -- 172.21.15.144:0/1795257271 learned_addr learned my addr 172.21.15.144:0/1795257271 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:46:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.323+0000 7f2f09b22700 1 --2- 172.21.15.144:0/1795257271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f041003d0 0x7f2f04195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.323+0000 7f2f09321700 1 -- 172.21.15.144:0/1795257271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f041003d0 msgr2=0x7f2f04195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.323+0000 7f2f09321700 1 --2- 172.21.15.144:0/1795257271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f041003d0 0x7f2f04195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.323+0000 7f2f09321700 1 -- 172.21.15.144:0/1795257271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f000095f0 con 0x7f2f04100d70 2024-09-10T11:46:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.323+0000 7f2f09b22700 1 --2- 172.21.15.144:0/1795257271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f041003d0 0x7f2f04195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:46:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.323+0000 7f2f09321700 1 --2- 172.21.15.144:0/1795257271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f04100d70 0x7f2f041962a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400c8f0 tx=0x7f2ef400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.323+0000 7f2efaffd700 1 -- 172.21.15.144:0/1795257271 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ef4007a50 con 0x7f2f04100d70 2024-09-10T11:46:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.323+0000 7f2efaffd700 1 -- 172.21.15.144:0/1795257271 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ef4007bb0 con 0x7f2f04100d70 2024-09-10T11:46:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.323+0000 7f2f0bd86700 1 -- 172.21.15.144:0/1795257271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f0419a9f0 con 0x7f2f04100d70 2024-09-10T11:46:04.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.323+0000 7f2efaffd700 1 -- 172.21.15.144:0/1795257271 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ef4018630 con 0x7f2f04100d70 2024-09-10T11:46:04.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.323+0000 7f2f0bd86700 1 -- 172.21.15.144:0/1795257271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0419af40 con 0x7f2f04100d70 2024-09-10T11:46:04.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.324+0000 7f2f0bd86700 1 -- 172.21.15.144:0/1795257271 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f041086e0 con 0x7f2f04100d70 2024-09-10T11:46:04.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.325+0000 7f2efaffd700 1 -- 172.21.15.144:0/1795257271 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2ef4007d20 con 0x7f2f04100d70 2024-09-10T11:46:04.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.325+0000 7f2efaffd700 1 --2- 172.21.15.144:0/1795257271 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2ef006e200 0x7f2ef00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:04.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.325+0000 7f2efaffd700 1 -- 172.21.15.144:0/1795257271 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2ef408e060 con 0x7f2f04100d70 2024-09-10T11:46:04.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.325+0000 7f2f09b22700 1 --2- 172.21.15.144:0/1795257271 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2ef006e200 0x7f2ef00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:04.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.326+0000 7f2f09b22700 1 --2- 172.21.15.144:0/1795257271 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2ef006e200 0x7f2ef00706b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2f000059d0 tx=0x7f2f0001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:04.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.330+0000 7f2efaffd700 1 -- 172.21.15.144:0/1795257271 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2ef4057440 con 0x7f2f04100d70 2024-09-10T11:46:04.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.552+0000 7f2f0bd86700 1 -- 172.21.15.144:0/1795257271 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f0419b2d0 con 0x7f2ef006e200 2024-09-10T11:46:04.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.553+0000 7f2efaffd700 1 -- 172.21.15.144:0/1795257271 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2f0419b2d0 con 0x7f2ef006e200 2024-09-10T11:46:04.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.555+0000 7f2f0bd86700 1 -- 172.21.15.144:0/1795257271 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2ef006e200 msgr2=0x7f2ef00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:04.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.555+0000 7f2f0bd86700 1 --2- 172.21.15.144:0/1795257271 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2ef006e200 0x7f2ef00706b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2f000059d0 tx=0x7f2f0001a040 comp rx=0 tx=0).stop 2024-09-10T11:46:04.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.555+0000 7f2f0bd86700 1 -- 172.21.15.144:0/1795257271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f04100d70 msgr2=0x7f2f041962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:04.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.555+0000 7f2f0bd86700 1 --2- 172.21.15.144:0/1795257271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f04100d70 0x7f2f041962a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2ef400c8f0 tx=0x7f2ef400ccb0 comp rx=0 tx=0).stop 2024-09-10T11:46:04.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.555+0000 7f2f0bd86700 1 -- 172.21.15.144:0/1795257271 shutdown_connections 2024-09-10T11:46:04.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.555+0000 7f2f0bd86700 1 --2- 172.21.15.144:0/1795257271 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2ef006e200 0x7f2ef00706b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:04.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.555+0000 7f2f0bd86700 1 --2- 172.21.15.144:0/1795257271 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2f041003d0 0x7f2f04195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:04.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.555+0000 7f2f0bd86700 1 --2- 172.21.15.144:0/1795257271 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2f04100d70 0x7f2f041962a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:04.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.555+0000 7f2f0bd86700 1 -- 172.21.15.144:0/1795257271 >> 172.21.15.144:0/1795257271 conn(0x7f2f040fbc90 msgr2=0x7f2f040fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:04.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.556+0000 7f2f0bd86700 1 -- 172.21.15.144:0/1795257271 shutdown_connections 2024-09-10T11:46:04.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.556+0000 7f2f0bd86700 1 -- 172.21.15.144:0/1795257271 wait complete. 2024-09-10T11:46:04.567 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:46:04.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.662+0000 7fd317dbf700 1 -- 172.21.15.144:0/3297400270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3101003d0 msgr2=0x7fd3101007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:04.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.662+0000 7fd317dbf700 1 --2- 172.21.15.144:0/3297400270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3101003d0 0x7fd3101007a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fd304009910 tx=0x7fd304009c20 comp rx=0 tx=0).stop 2024-09-10T11:46:04.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.662+0000 7fd317dbf700 1 -- 172.21.15.144:0/3297400270 shutdown_connections 2024-09-10T11:46:04.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.662+0000 7fd317dbf700 1 --2- 172.21.15.144:0/3297400270 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd310100d70 0x7fd310104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:04.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.662+0000 7fd317dbf700 1 --2- 172.21.15.144:0/3297400270 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3101003d0 0x7fd3101007a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:04.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.662+0000 7fd317dbf700 1 -- 172.21.15.144:0/3297400270 >> 172.21.15.144:0/3297400270 conn(0x7fd3100fbc90 msgr2=0x7fd3100fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:04.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.662+0000 7fd317dbf700 1 -- 172.21.15.144:0/3297400270 shutdown_connections 2024-09-10T11:46:04.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.662+0000 7fd317dbf700 1 -- 172.21.15.144:0/3297400270 wait complete. 2024-09-10T11:46:04.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.663+0000 7fd317dbf700 1 Processor -- start 2024-09-10T11:46:04.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.663+0000 7fd317dbf700 1 -- start start 2024-09-10T11:46:04.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.663+0000 7fd317dbf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3101003d0 0x7fd310195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:04.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.663+0000 7fd317dbf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd310100d70 0x7fd3101962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:04.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.663+0000 7fd317dbf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd310196980 con 0x7fd3101003d0 2024-09-10T11:46:04.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.663+0000 7fd317dbf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd31019a710 con 0x7fd310100d70 2024-09-10T11:46:04.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.663+0000 7fd315b5b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3101003d0 0x7fd310195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:04.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.663+0000 7fd315b5b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3101003d0 0x7fd310195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:32884/0 (socket says 172.21.15.144:32884) 2024-09-10T11:46:04.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.663+0000 7fd315b5b700 1 -- 172.21.15.144:0/4097675649 learned_addr learned my addr 172.21.15.144:0/4097675649 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:46:04.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.663+0000 7fd31535a700 1 --2- 172.21.15.144:0/4097675649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd310100d70 0x7fd3101962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:04.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.663+0000 7fd315b5b700 1 -- 172.21.15.144:0/4097675649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd310100d70 msgr2=0x7fd3101962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:04.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.663+0000 7fd315b5b700 1 --2- 172.21.15.144:0/4097675649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd310100d70 0x7fd3101962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:04.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.663+0000 7fd315b5b700 1 -- 172.21.15.144:0/4097675649 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3040095f0 con 0x7fd3101003d0 2024-09-10T11:46:04.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.664+0000 7fd315b5b700 1 --2- 172.21.15.144:0/4097675649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3101003d0 0x7fd310195d60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd30400b5c0 tx=0x7fd30400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:04.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.664+0000 7fd302ffd700 1 -- 172.21.15.144:0/4097675649 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd30401d070 con 0x7fd3101003d0 2024-09-10T11:46:04.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.664+0000 7fd317dbf700 1 -- 172.21.15.144:0/4097675649 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd31019a990 con 0x7fd3101003d0 2024-09-10T11:46:04.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.664+0000 7fd302ffd700 1 -- 172.21.15.144:0/4097675649 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd30400f460 con 0x7fd3101003d0 2024-09-10T11:46:04.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.664+0000 7fd302ffd700 1 -- 172.21.15.144:0/4097675649 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd304017610 con 0x7fd3101003d0 2024-09-10T11:46:04.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.664+0000 7fd317dbf700 1 -- 172.21.15.144:0/4097675649 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd31019ae80 con 0x7fd3101003d0 2024-09-10T11:46:04.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.665+0000 7fd317dbf700 1 -- 172.21.15.144:0/4097675649 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3101086e0 con 0x7fd3101003d0 2024-09-10T11:46:04.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.666+0000 7fd302ffd700 1 -- 172.21.15.144:0/4097675649 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd30400f5d0 con 0x7fd3101003d0 2024-09-10T11:46:04.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.666+0000 7fd302ffd700 1 --2- 172.21.15.144:0/4097675649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd2fc06e200 0x7fd2fc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:04.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.666+0000 7fd302ffd700 1 -- 172.21.15.144:0/4097675649 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd30408f6e0 con 0x7fd3101003d0 2024-09-10T11:46:04.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.666+0000 7fd31535a700 1 --2- 172.21.15.144:0/4097675649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd2fc06e200 0x7fd2fc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:04.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.667+0000 7fd31535a700 1 --2- 172.21.15.144:0/4097675649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd2fc06e200 0x7fd2fc0706b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd30c005fd0 tx=0x7fd30c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:04.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.671+0000 7fd302ffd700 1 -- 172.21.15.144:0/4097675649 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd304058ac0 con 0x7fd3101003d0 2024-09-10T11:46:04.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.892+0000 7fd317dbf700 1 -- 172.21.15.144:0/4097675649 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd31002ccf0 con 0x7fd2fc06e200 2024-09-10T11:46:04.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.893+0000 7fd302ffd700 1 -- 172.21.15.144:0/4097675649 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd31002ccf0 con 0x7fd2fc06e200 2024-09-10T11:46:04.896 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.895+0000 7fd317dbf700 1 -- 172.21.15.144:0/4097675649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd2fc06e200 msgr2=0x7fd2fc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:04.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.895+0000 7fd317dbf700 1 --2- 172.21.15.144:0/4097675649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd2fc06e200 0x7fd2fc0706b0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd30c005fd0 tx=0x7fd30c00a560 comp rx=0 tx=0).stop 2024-09-10T11:46:04.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.895+0000 7fd317dbf700 1 -- 172.21.15.144:0/4097675649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3101003d0 msgr2=0x7fd310195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:04.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.895+0000 7fd317dbf700 1 --2- 172.21.15.144:0/4097675649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3101003d0 0x7fd310195d60 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fd30400b5c0 tx=0x7fd30400be00 comp rx=0 tx=0).stop 2024-09-10T11:46:04.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.895+0000 7fd317dbf700 1 -- 172.21.15.144:0/4097675649 shutdown_connections 2024-09-10T11:46:04.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.895+0000 7fd317dbf700 1 --2- 172.21.15.144:0/4097675649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd2fc06e200 0x7fd2fc0706b0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:04.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.895+0000 7fd317dbf700 1 --2- 172.21.15.144:0/4097675649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd3101003d0 0x7fd310195d60 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:04.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.895+0000 7fd317dbf700 1 --2- 172.21.15.144:0/4097675649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd310100d70 0x7fd3101962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:04.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.895+0000 7fd317dbf700 1 -- 172.21.15.144:0/4097675649 >> 172.21.15.144:0/4097675649 conn(0x7fd3100fbc90 msgr2=0x7fd3100fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:04.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.895+0000 7fd317dbf700 1 -- 172.21.15.144:0/4097675649 shutdown_connections 2024-09-10T11:46:04.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:04.895+0000 7fd317dbf700 1 -- 172.21.15.144:0/4097675649 wait complete. 2024-09-10T11:46:04.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:04 smithi144 ceph-mon[87436]: pgmap v422: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:05.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.000+0000 7fcfa4370700 1 -- 172.21.15.144:0/511931341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf9c0ffce0 msgr2=0x7fcf9c103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:05.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.000+0000 7fcfa4370700 1 --2- 172.21.15.144:0/511931341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf9c0ffce0 0x7fcf9c103cb0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c009910 tx=0x7fcf8c009c20 comp rx=0 tx=0).stop 2024-09-10T11:46:05.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.001+0000 7fcfa4370700 1 -- 172.21.15.144:0/511931341 shutdown_connections 2024-09-10T11:46:05.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.001+0000 7fcfa4370700 1 --2- 172.21.15.144:0/511931341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf9c0ffce0 0x7fcf9c103cb0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.001+0000 7fcfa4370700 1 --2- 172.21.15.144:0/511931341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf9c0ff340 0x7fcf9c0ff710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.001+0000 7fcfa4370700 1 -- 172.21.15.144:0/511931341 >> 172.21.15.144:0/511931341 conn(0x7fcf9c0fabe0 msgr2=0x7fcf9c0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:05.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.001+0000 7fcfa4370700 1 -- 172.21.15.144:0/511931341 shutdown_connections 2024-09-10T11:46:05.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.001+0000 7fcfa4370700 1 -- 172.21.15.144:0/511931341 wait complete. 2024-09-10T11:46:05.004 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.001+0000 7fcfa4370700 1 Processor -- start 2024-09-10T11:46:05.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.001+0000 7fcfa4370700 1 -- start start 2024-09-10T11:46:05.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcfa4370700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf9c0ff340 0x7fcf9c196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:05.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcfa4370700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf9c0ffce0 0x7fcf9c1970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:05.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcfa4370700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c197710 con 0x7fcf9c0ffce0 2024-09-10T11:46:05.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcfa4370700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf9c197880 con 0x7fcf9c0ff340 2024-09-10T11:46:05.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcfa190b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf9c0ffce0 0x7fcf9c1970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:05.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcfa190b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf9c0ffce0 0x7fcf9c1970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:32900/0 (socket says 172.21.15.144:32900) 2024-09-10T11:46:05.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcfa190b700 1 -- 172.21.15.144:0/648512583 learned_addr learned my addr 172.21.15.144:0/648512583 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:46:05.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcfa210c700 1 --2- 172.21.15.144:0/648512583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf9c0ff340 0x7fcf9c196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:05.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcfa190b700 1 -- 172.21.15.144:0/648512583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf9c0ff340 msgr2=0x7fcf9c196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:05.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcfa190b700 1 --2- 172.21.15.144:0/648512583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf9c0ff340 0x7fcf9c196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcfa190b700 1 -- 172.21.15.144:0/648512583 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c0095f0 con 0x7fcf9c0ffce0 2024-09-10T11:46:05.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcfa210c700 1 --2- 172.21.15.144:0/648512583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf9c0ff340 0x7fcf9c196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:46:05.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcfa190b700 1 --2- 172.21.15.144:0/648512583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf9c0ffce0 0x7fcf9c1970f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0059d0 tx=0x7fcf8c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:05.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcf937fe700 1 -- 172.21.15.144:0/648512583 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf8c01d070 con 0x7fcf9c0ffce0 2024-09-10T11:46:05.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcf937fe700 1 -- 172.21.15.144:0/648512583 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcf8c00b610 con 0x7fcf9c0ffce0 2024-09-10T11:46:05.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcfa4370700 1 -- 172.21.15.144:0/648512583 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9c19fc20 con 0x7fcf9c0ffce0 2024-09-10T11:46:05.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.002+0000 7fcf937fe700 1 -- 172.21.15.144:0/648512583 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf8c022470 con 0x7fcf9c0ffce0 2024-09-10T11:46:05.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.003+0000 7fcfa4370700 1 -- 172.21.15.144:0/648512583 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf9c1a0090 con 0x7fcf9c0ffce0 2024-09-10T11:46:05.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.004+0000 7fcfa4370700 1 -- 172.21.15.144:0/648512583 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf9c1076b0 con 0x7fcf9c0ffce0 2024-09-10T11:46:05.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.004+0000 7fcf937fe700 1 -- 172.21.15.144:0/648512583 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fcf8c00fc70 con 0x7fcf9c0ffce0 2024-09-10T11:46:05.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.004+0000 7fcf937fe700 1 --2- 172.21.15.144:0/648512583 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcf8806e200 0x7fcf880706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:05.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.005+0000 7fcf937fe700 1 -- 172.21.15.144:0/648512583 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcf8c08f440 con 0x7fcf9c0ffce0 2024-09-10T11:46:05.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.005+0000 7fcfa210c700 1 --2- 172.21.15.144:0/648512583 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcf8806e200 0x7fcf880706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:05.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.005+0000 7fcfa210c700 1 --2- 172.21.15.144:0/648512583 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcf8806e200 0x7fcf880706b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009cb0 tx=0x7fcf98009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:05.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.009+0000 7fcf937fe700 1 -- 172.21.15.144:0/648512583 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcf8c0588a0 con 0x7fcf9c0ffce0 2024-09-10T11:46:05.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:04 smithi178 ceph-mon[65940]: pgmap v422: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:05.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.219+0000 7fcfa4370700 1 -- 172.21.15.144:0/648512583 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf9c02ccf0 con 0x7fcf8806e200 2024-09-10T11:46:05.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.229+0000 7fcf937fe700 1 -- 172.21.15.144:0/648512583 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fcf9c02ccf0 con 0x7fcf8806e200 2024-09-10T11:46:05.230 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:46:05.230 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (14m) 8m ago 24m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:46:05.230 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (25m) 8m ago 25m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:46:05.230 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (24m) 7m ago 24m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:46:05.230 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (11m) 8m ago 25m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:46:05.230 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (11m) 7m ago 24m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:46:05.230 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (13m) 8m ago 24m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:46:05.231 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (8m) 8m ago 19m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:46:05.231 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (8m) 8m ago 19m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:46:05.231 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (19m) 7m ago 19m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:46:05.231 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (7m) 7m ago 19m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:46:05.231 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (16m) 8m ago 26m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:46:05.231 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (15m) 7m ago 23m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:46:05.231 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (12m) 8m ago 26m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:46:05.231 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (12m) 7m ago 23m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:46:05.231 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (14m) 8m ago 24m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:46:05.231 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (14m) 7m ago 23m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:46:05.231 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (11m) 8m ago 23m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:46:05.231 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (10m) 8m ago 22m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:46:05.231 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (10m) 8m ago 22m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:46:05.231 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (9m) 7m ago 21m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:46:05.232 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (9m) 7m ago 21m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:46:05.232 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (8m) 7m ago 20m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:46:05.232 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (14m) 8m ago 24m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:46:05.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.231+0000 7fcfa4370700 1 -- 172.21.15.144:0/648512583 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcf8806e200 msgr2=0x7fcf880706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:05.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.231+0000 7fcfa4370700 1 --2- 172.21.15.144:0/648512583 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcf8806e200 0x7fcf880706b0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fcf98009cb0 tx=0x7fcf98009260 comp rx=0 tx=0).stop 2024-09-10T11:46:05.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.231+0000 7fcfa4370700 1 -- 172.21.15.144:0/648512583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf9c0ffce0 msgr2=0x7fcf9c1970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:05.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.231+0000 7fcfa4370700 1 --2- 172.21.15.144:0/648512583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf9c0ffce0 0x7fcf9c1970f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fcf8c0059d0 tx=0x7fcf8c00f740 comp rx=0 tx=0).stop 2024-09-10T11:46:05.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.231+0000 7fcfa4370700 1 -- 172.21.15.144:0/648512583 shutdown_connections 2024-09-10T11:46:05.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.231+0000 7fcfa4370700 1 --2- 172.21.15.144:0/648512583 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcf8806e200 0x7fcf880706b0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.231+0000 7fcfa4370700 1 --2- 172.21.15.144:0/648512583 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcf9c0ff340 0x7fcf9c196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.231+0000 7fcfa4370700 1 --2- 172.21.15.144:0/648512583 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcf9c0ffce0 0x7fcf9c1970f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.231+0000 7fcfa4370700 1 -- 172.21.15.144:0/648512583 >> 172.21.15.144:0/648512583 conn(0x7fcf9c0fabe0 msgr2=0x7fcf9c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:05.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.232+0000 7fcfa4370700 1 -- 172.21.15.144:0/648512583 shutdown_connections 2024-09-10T11:46:05.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.232+0000 7fcfa4370700 1 -- 172.21.15.144:0/648512583 wait complete. 2024-09-10T11:46:05.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.335+0000 7f3eb7968700 1 -- 172.21.15.144:0/2328614305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00ff310 msgr2=0x7f3eb00ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:05.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.335+0000 7f3eb7968700 1 --2- 172.21.15.144:0/2328614305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00ff310 0x7f3eb00ff6e0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00b0a0 tx=0x7f3eac00b3b0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.335+0000 7f3eb7968700 1 -- 172.21.15.144:0/2328614305 shutdown_connections 2024-09-10T11:46:05.336 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.335+0000 7f3eb7968700 1 --2- 172.21.15.144:0/2328614305 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eb00ffcb0 0x7f3eb0103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.335+0000 7f3eb7968700 1 --2- 172.21.15.144:0/2328614305 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00ff310 0x7f3eb00ff6e0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.335+0000 7f3eb7968700 1 -- 172.21.15.144:0/2328614305 >> 172.21.15.144:0/2328614305 conn(0x7f3eb00fabb0 msgr2=0x7f3eb00fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:05.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.335+0000 7f3eb7968700 1 -- 172.21.15.144:0/2328614305 shutdown_connections 2024-09-10T11:46:05.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.335+0000 7f3eb7968700 1 -- 172.21.15.144:0/2328614305 wait complete. 2024-09-10T11:46:05.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.336+0000 7f3eb7968700 1 Processor -- start 2024-09-10T11:46:05.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.336+0000 7f3eb7968700 1 -- start start 2024-09-10T11:46:05.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.336+0000 7f3eb7968700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eb00ff310 0x7f3eb0196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:05.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.336+0000 7f3eb7968700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00ffcb0 0x7f3eb0197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:05.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.336+0000 7f3eb7968700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb0197750 con 0x7f3eb00ffcb0 2024-09-10T11:46:05.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.336+0000 7f3eb7968700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb0197890 con 0x7f3eb00ff310 2024-09-10T11:46:05.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.336+0000 7f3eb4f03700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00ffcb0 0x7f3eb0197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:05.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.336+0000 7f3eb4f03700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00ffcb0 0x7f3eb0197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:32924/0 (socket says 172.21.15.144:32924) 2024-09-10T11:46:05.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.336+0000 7f3eb4f03700 1 -- 172.21.15.144:0/2752644256 learned_addr learned my addr 172.21.15.144:0/2752644256 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:46:05.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.336+0000 7f3eb5704700 1 --2- 172.21.15.144:0/2752644256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eb00ff310 0x7f3eb0196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:05.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.337+0000 7f3eb4f03700 1 -- 172.21.15.144:0/2752644256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eb00ff310 msgr2=0x7f3eb0196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:05.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.337+0000 7f3eb4f03700 1 --2- 172.21.15.144:0/2752644256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eb00ff310 0x7f3eb0196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.337+0000 7f3eb4f03700 1 -- 172.21.15.144:0/2752644256 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac009d00 con 0x7f3eb00ffcb0 2024-09-10T11:46:05.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.337+0000 7f3eb4f03700 1 --2- 172.21.15.144:0/2752644256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00ffcb0 0x7f3eb0197130 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000d900 tx=0x7f3ea000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:05.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.337+0000 7f3ea67fc700 1 -- 172.21.15.144:0/2752644256 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ea0009900 con 0x7f3eb00ffcb0 2024-09-10T11:46:05.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.337+0000 7f3ea67fc700 1 -- 172.21.15.144:0/2752644256 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ea0010460 con 0x7f3eb00ffcb0 2024-09-10T11:46:05.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.337+0000 7f3eb7968700 1 -- 172.21.15.144:0/2752644256 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb019fc60 con 0x7f3eb00ffcb0 2024-09-10T11:46:05.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.337+0000 7f3ea67fc700 1 -- 172.21.15.144:0/2752644256 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ea000f5f0 con 0x7f3eb00ffcb0 2024-09-10T11:46:05.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.337+0000 7f3eb7968700 1 -- 172.21.15.144:0/2752644256 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb01a0120 con 0x7f3eb00ffcb0 2024-09-10T11:46:05.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.338+0000 7f3eb7968700 1 -- 172.21.15.144:0/2752644256 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb0107620 con 0x7f3eb00ffcb0 2024-09-10T11:46:05.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.339+0000 7f3ea67fc700 1 -- 172.21.15.144:0/2752644256 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3ea00105d0 con 0x7f3eb00ffcb0 2024-09-10T11:46:05.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.339+0000 7f3ea67fc700 1 --2- 172.21.15.144:0/2752644256 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3e9c06e200 0x7f3e9c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:05.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.339+0000 7f3ea67fc700 1 -- 172.21.15.144:0/2752644256 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3ea008db00 con 0x7f3eb00ffcb0 2024-09-10T11:46:05.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.339+0000 7f3eb5704700 1 --2- 172.21.15.144:0/2752644256 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3e9c06e200 0x7f3e9c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:05.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.340+0000 7f3eb5704700 1 --2- 172.21.15.144:0/2752644256 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3e9c06e200 0x7f3e9c0706b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3eac000f80 tx=0x7f3eac0061f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:05.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.344+0000 7f3ea67fc700 1 -- 172.21.15.144:0/2752644256 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3ea0056f60 con 0x7f3eb00ffcb0 2024-09-10T11:46:05.627 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.626+0000 7f3eb7968700 1 -- 172.21.15.144:0/2752644256 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3eb004ea50 con 0x7f3eb00ffcb0 2024-09-10T11:46:05.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.627+0000 7f3ea67fc700 1 -- 172.21.15.144:0/2752644256 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f3ea0017340 con 0x7f3eb00ffcb0 2024-09-10T11:46:05.628 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:46:05.628 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:46:05.628 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:46:05.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:46:05.628 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:46:05.628 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:46:05.628 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:46:05.628 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:46:05.629 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:46:05.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:46:05.629 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:46:05.629 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:46:05.629 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:46:05.629 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:46:05.629 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:46:05.629 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:46:05.629 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:46:05.629 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:46:05.629 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:46:05.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.629+0000 7f3eb7968700 1 -- 172.21.15.144:0/2752644256 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3e9c06e200 msgr2=0x7f3e9c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:05.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.629+0000 7f3eb7968700 1 --2- 172.21.15.144:0/2752644256 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3e9c06e200 0x7f3e9c0706b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3eac000f80 tx=0x7f3eac0061f0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.629+0000 7f3eb7968700 1 -- 172.21.15.144:0/2752644256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00ffcb0 msgr2=0x7f3eb0197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:05.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.629+0000 7f3eb7968700 1 --2- 172.21.15.144:0/2752644256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00ffcb0 0x7f3eb0197130 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000d900 tx=0x7f3ea000dc10 comp rx=0 tx=0).stop 2024-09-10T11:46:05.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.629+0000 7f3eb7968700 1 -- 172.21.15.144:0/2752644256 shutdown_connections 2024-09-10T11:46:05.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.629+0000 7f3eb7968700 1 --2- 172.21.15.144:0/2752644256 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3e9c06e200 0x7f3e9c0706b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.629+0000 7f3eb7968700 1 --2- 172.21.15.144:0/2752644256 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eb00ff310 0x7f3eb0196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.629+0000 7f3eb7968700 1 --2- 172.21.15.144:0/2752644256 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb00ffcb0 0x7f3eb0197130 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.629+0000 7f3eb7968700 1 -- 172.21.15.144:0/2752644256 >> 172.21.15.144:0/2752644256 conn(0x7f3eb00fabb0 msgr2=0x7f3eb00fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:05.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.629+0000 7f3eb7968700 1 -- 172.21.15.144:0/2752644256 shutdown_connections 2024-09-10T11:46:05.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.629+0000 7f3eb7968700 1 -- 172.21.15.144:0/2752644256 wait complete. 2024-09-10T11:46:05.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.735+0000 7f403f566700 1 -- 172.21.15.144:0/2265349678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40380ff390 msgr2=0x7f40380ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:05.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.735+0000 7f403f566700 1 --2- 172.21.15.144:0/2265349678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40380ff390 0x7f40380ff760 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f403000b0a0 tx=0x7f403000b3b0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.735+0000 7f403f566700 1 -- 172.21.15.144:0/2265349678 shutdown_connections 2024-09-10T11:46:05.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.735+0000 7f403f566700 1 --2- 172.21.15.144:0/2265349678 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40380ffd30 0x7f4038103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.735+0000 7f403f566700 1 --2- 172.21.15.144:0/2265349678 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40380ff390 0x7f40380ff760 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.735+0000 7f403f566700 1 -- 172.21.15.144:0/2265349678 >> 172.21.15.144:0/2265349678 conn(0x7f4038074f50 msgr2=0x7f4038075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:05.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.735+0000 7f403f566700 1 -- 172.21.15.144:0/2265349678 shutdown_connections 2024-09-10T11:46:05.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.735+0000 7f403f566700 1 -- 172.21.15.144:0/2265349678 wait complete. 2024-09-10T11:46:05.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.736+0000 7f403f566700 1 Processor -- start 2024-09-10T11:46:05.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.736+0000 7f403f566700 1 -- start start 2024-09-10T11:46:05.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.736+0000 7f403f566700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40380ff390 0x7f4038100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:05.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.736+0000 7f403f566700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40380ffd30 0x7f40381012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:05.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.736+0000 7f403f566700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4038101800 con 0x7f40380ffd30 2024-09-10T11:46:05.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.736+0000 7f403f566700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4038101940 con 0x7f40380ff390 2024-09-10T11:46:05.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.736+0000 7f403cb01700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40380ffd30 0x7f40381012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:05.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.736+0000 7f403cb01700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40380ffd30 0x7f40381012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:32942/0 (socket says 172.21.15.144:32942) 2024-09-10T11:46:05.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.736+0000 7f403cb01700 1 -- 172.21.15.144:0/1613039967 learned_addr learned my addr 172.21.15.144:0/1613039967 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:46:05.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.736+0000 7f403d302700 1 --2- 172.21.15.144:0/1613039967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40380ff390 0x7f4038100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:05.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.736+0000 7f403cb01700 1 -- 172.21.15.144:0/1613039967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40380ff390 msgr2=0x7f4038100d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:05.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.736+0000 7f403cb01700 1 --2- 172.21.15.144:0/1613039967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40380ff390 0x7f4038100d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:05.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.736+0000 7f403cb01700 1 -- 172.21.15.144:0/1613039967 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4030009d00 con 0x7f40380ffd30 2024-09-10T11:46:05.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.736+0000 7f403d302700 1 --2- 172.21.15.144:0/1613039967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40380ff390 0x7f4038100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:46:05.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.737+0000 7f403cb01700 1 --2- 172.21.15.144:0/1613039967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40380ffd30 0x7f40381012c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f402c00c960 tx=0x7f402c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:05.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.737+0000 7f40367fc700 1 -- 172.21.15.144:0/1613039967 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f402c007a70 con 0x7f40380ffd30 2024-09-10T11:46:05.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.737+0000 7f40367fc700 1 -- 172.21.15.144:0/1613039967 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f402c007bd0 con 0x7f40380ffd30 2024-09-10T11:46:05.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.737+0000 7f403f566700 1 -- 172.21.15.144:0/1613039967 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40381a1f00 con 0x7f40380ffd30 2024-09-10T11:46:05.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.737+0000 7f40367fc700 1 -- 172.21.15.144:0/1613039967 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f402c0186c0 con 0x7f40380ffd30 2024-09-10T11:46:05.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.737+0000 7f403f566700 1 -- 172.21.15.144:0/1613039967 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40381a2260 con 0x7f40380ffd30 2024-09-10T11:46:05.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.738+0000 7f403f566700 1 -- 172.21.15.144:0/1613039967 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4038108660 con 0x7f40380ffd30 2024-09-10T11:46:05.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.738+0000 7f40367fc700 1 -- 172.21.15.144:0/1613039967 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f402c00faa0 con 0x7f40380ffd30 2024-09-10T11:46:05.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.739+0000 7f40367fc700 1 --2- 172.21.15.144:0/1613039967 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f402006e200 0x7f40200706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:05.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.739+0000 7f403d302700 1 --2- 172.21.15.144:0/1613039967 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f402006e200 0x7f40200706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:05.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.739+0000 7f40367fc700 1 -- 172.21.15.144:0/1613039967 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f402c08e0d0 con 0x7f40380ffd30 2024-09-10T11:46:05.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.739+0000 7f403d302700 1 --2- 172.21.15.144:0/1613039967 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f402006e200 0x7f40200706b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f40300062a0 tx=0x7f4030006210 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:05.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.743+0000 7f40367fc700 1 -- 172.21.15.144:0/1613039967 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f402c05afe0 con 0x7f40380ffd30 2024-09-10T11:46:05.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:05 smithi144 ceph-mon[87436]: from='client.34820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:05.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:05 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2752644256' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:46:05.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.994+0000 7f403f566700 1 -- 172.21.15.144:0/1613039967 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4038101b00 con 0x7f40380ffd30 2024-09-10T11:46:05.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.995+0000 7f40367fc700 1 -- 172.21.15.144:0/1613039967 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f402c05ab70 con 0x7f40380ffd30 2024-09-10T11:46:05.996 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:46:05.996 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:46:05.996 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:46:05.997 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:46:05.997 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:46:05.997 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:46:05.997 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:46:05.997 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:46:05.997 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:46:05.997 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:46:05.997 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:46:05.997 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:46:05.997 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:46:05.997 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:46:05.997 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:46:05.997 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:46:05.997 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:46:05.998 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:46:05.998 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:46:05.998 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:46:05.998 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:46:05.998 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:46:05.998 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:46:05.998 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:46:05.998 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:46:05.998 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:46:05.998 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:46:05.998 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:46:05.998 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:46:05.998 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:46:05.999 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:46:05.999 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:46:05.999 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:46:05.999 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:46:05.999 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:46:05.999 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:46:05.999 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:46:05.999 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:46:05.999 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:46:05.999 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:46:05.999 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:46:05.999 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:46:05.999 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:46:06.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.997+0000 7f403f566700 1 -- 172.21.15.144:0/1613039967 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f402006e200 msgr2=0x7f40200706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:06.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.997+0000 7f403f566700 1 --2- 172.21.15.144:0/1613039967 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f402006e200 0x7f40200706b0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f40300062a0 tx=0x7f4030006210 comp rx=0 tx=0).stop 2024-09-10T11:46:06.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.997+0000 7f403f566700 1 -- 172.21.15.144:0/1613039967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40380ffd30 msgr2=0x7f40381012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:06.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.997+0000 7f403f566700 1 --2- 172.21.15.144:0/1613039967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40380ffd30 0x7f40381012c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f402c00c960 tx=0x7f402c00cd20 comp rx=0 tx=0).stop 2024-09-10T11:46:06.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.997+0000 7f403f566700 1 -- 172.21.15.144:0/1613039967 shutdown_connections 2024-09-10T11:46:06.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.997+0000 7f403f566700 1 --2- 172.21.15.144:0/1613039967 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f402006e200 0x7f40200706b0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.997+0000 7f403f566700 1 --2- 172.21.15.144:0/1613039967 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f40380ff390 0x7f4038100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.997+0000 7f403f566700 1 --2- 172.21.15.144:0/1613039967 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f40380ffd30 0x7f40381012c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.997+0000 7f403f566700 1 -- 172.21.15.144:0/1613039967 >> 172.21.15.144:0/1613039967 conn(0x7f4038074f50 msgr2=0x7f4038068630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:06.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.998+0000 7f403f566700 1 -- 172.21.15.144:0/1613039967 shutdown_connections 2024-09-10T11:46:06.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:05.998+0000 7f403f566700 1 -- 172.21.15.144:0/1613039967 wait complete. 2024-09-10T11:46:06.001 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:46:06.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.105+0000 7feb56b44700 1 -- 172.21.15.144:0/2287655494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb500ff390 msgr2=0x7feb500ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:06.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.105+0000 7feb56b44700 1 --2- 172.21.15.144:0/2287655494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb500ff390 0x7feb500ff760 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009910 tx=0x7feb4c009c20 comp rx=0 tx=0).stop 2024-09-10T11:46:06.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.105+0000 7feb56b44700 1 -- 172.21.15.144:0/2287655494 shutdown_connections 2024-09-10T11:46:06.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.105+0000 7feb56b44700 1 --2- 172.21.15.144:0/2287655494 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb500ffd30 0x7feb50103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.105+0000 7feb56b44700 1 --2- 172.21.15.144:0/2287655494 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb500ff390 0x7feb500ff760 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.105+0000 7feb56b44700 1 -- 172.21.15.144:0/2287655494 >> 172.21.15.144:0/2287655494 conn(0x7feb50074f50 msgr2=0x7feb50075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:06.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.105+0000 7feb56b44700 1 -- 172.21.15.144:0/2287655494 shutdown_connections 2024-09-10T11:46:06.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.105+0000 7feb56b44700 1 -- 172.21.15.144:0/2287655494 wait complete. 2024-09-10T11:46:06.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.106+0000 7feb56b44700 1 Processor -- start 2024-09-10T11:46:06.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.106+0000 7feb56b44700 1 -- start start 2024-09-10T11:46:06.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.106+0000 7feb56b44700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb500ff390 0x7feb50194a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:06.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.106+0000 7feb56b44700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb500ffd30 0x7feb50194f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:06.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.106+0000 7feb56b44700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb501954b0 con 0x7feb500ff390 2024-09-10T11:46:06.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.106+0000 7feb56b44700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb50195620 con 0x7feb500ffd30 2024-09-10T11:46:06.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.106+0000 7feb548e0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb500ff390 0x7feb50194a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:06.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.106+0000 7feb548e0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb500ff390 0x7feb50194a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:32972/0 (socket says 172.21.15.144:32972) 2024-09-10T11:46:06.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.106+0000 7feb548e0700 1 -- 172.21.15.144:0/776270009 learned_addr learned my addr 172.21.15.144:0/776270009 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:46:06.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.106+0000 7feb47fff700 1 --2- 172.21.15.144:0/776270009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb500ffd30 0x7feb50194f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:06.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.106+0000 7feb548e0700 1 -- 172.21.15.144:0/776270009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb500ffd30 msgr2=0x7feb50194f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:06.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.106+0000 7feb548e0700 1 --2- 172.21.15.144:0/776270009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb500ffd30 0x7feb50194f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.106+0000 7feb548e0700 1 -- 172.21.15.144:0/776270009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4c0095f0 con 0x7feb500ff390 2024-09-10T11:46:06.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.107+0000 7feb548e0700 1 --2- 172.21.15.144:0/776270009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb500ff390 0x7feb50194a30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7feb4c006010 tx=0x7feb4c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:06.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.107+0000 7feb45ffb700 1 -- 172.21.15.144:0/776270009 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb4c01d070 con 0x7feb500ff390 2024-09-10T11:46:06.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.107+0000 7feb45ffb700 1 -- 172.21.15.144:0/776270009 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb4c00f460 con 0x7feb500ff390 2024-09-10T11:46:06.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.107+0000 7feb56b44700 1 -- 172.21.15.144:0/776270009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb501958a0 con 0x7feb500ff390 2024-09-10T11:46:06.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.107+0000 7feb45ffb700 1 -- 172.21.15.144:0/776270009 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb4c017630 con 0x7feb500ff390 2024-09-10T11:46:06.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.107+0000 7feb56b44700 1 -- 172.21.15.144:0/776270009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb501a22d0 con 0x7feb500ff390 2024-09-10T11:46:06.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.108+0000 7feb56b44700 1 -- 172.21.15.144:0/776270009 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb50107650 con 0x7feb500ff390 2024-09-10T11:46:06.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.108+0000 7feb45ffb700 1 -- 172.21.15.144:0/776270009 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7feb4c00f5d0 con 0x7feb500ff390 2024-09-10T11:46:06.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.109+0000 7feb45ffb700 1 --2- 172.21.15.144:0/776270009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feb3806e200 0x7feb380706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:06.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.109+0000 7feb45ffb700 1 -- 172.21.15.144:0/776270009 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7feb4c08f390 con 0x7feb500ff390 2024-09-10T11:46:06.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.109+0000 7feb47fff700 1 --2- 172.21.15.144:0/776270009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feb3806e200 0x7feb380706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:06.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.109+0000 7feb47fff700 1 --2- 172.21.15.144:0/776270009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feb3806e200 0x7feb380706b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feb40005950 tx=0x7feb400058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:06.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.113+0000 7feb45ffb700 1 -- 172.21.15.144:0/776270009 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7feb4c058770 con 0x7feb500ff390 2024-09-10T11:46:06.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:05 smithi178 ceph-mon[65940]: from='client.34820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:06.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:05 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2752644256' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:46:06.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.333+0000 7feb56b44700 1 -- 172.21.15.144:0/776270009 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7feb501958a0 con 0x7feb3806e200 2024-09-10T11:46:06.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.334+0000 7feb45ffb700 1 -- 172.21.15.144:0/776270009 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7feb501958a0 con 0x7feb3806e200 2024-09-10T11:46:06.344 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:46:06.344 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:46:06.344 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:46:06.344 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:46:06.344 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:46:06.344 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:46:06.345 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:46:06.345 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:46:06.345 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:46:06.345 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:46:06.345 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:46:06.345 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:46:06.345 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:46:06.345 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:46:06.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.337+0000 7feb56b44700 1 -- 172.21.15.144:0/776270009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feb3806e200 msgr2=0x7feb380706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:06.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.337+0000 7feb56b44700 1 --2- 172.21.15.144:0/776270009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feb3806e200 0x7feb380706b0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7feb40005950 tx=0x7feb400058e0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.337+0000 7feb56b44700 1 -- 172.21.15.144:0/776270009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb500ff390 msgr2=0x7feb50194a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:06.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.337+0000 7feb56b44700 1 --2- 172.21.15.144:0/776270009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb500ff390 0x7feb50194a30 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7feb4c006010 tx=0x7feb4c00be20 comp rx=0 tx=0).stop 2024-09-10T11:46:06.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.337+0000 7feb56b44700 1 -- 172.21.15.144:0/776270009 shutdown_connections 2024-09-10T11:46:06.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.337+0000 7feb56b44700 1 --2- 172.21.15.144:0/776270009 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feb3806e200 0x7feb380706b0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.337+0000 7feb56b44700 1 --2- 172.21.15.144:0/776270009 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb500ff390 0x7feb50194a30 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.337+0000 7feb56b44700 1 --2- 172.21.15.144:0/776270009 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb500ffd30 0x7feb50194f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.337+0000 7feb56b44700 1 -- 172.21.15.144:0/776270009 >> 172.21.15.144:0/776270009 conn(0x7feb50074f50 msgr2=0x7feb500fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:06.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.337+0000 7feb56b44700 1 -- 172.21.15.144:0/776270009 shutdown_connections 2024-09-10T11:46:06.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.337+0000 7feb56b44700 1 -- 172.21.15.144:0/776270009 wait complete. 2024-09-10T11:46:06.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.443+0000 7fa741efb700 1 -- 172.21.15.144:0/3349326926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa73c0ffc70 msgr2=0x7fa73c103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:06.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.443+0000 7fa741efb700 1 --2- 172.21.15.144:0/3349326926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa73c0ffc70 0x7fa73c103c40 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fa724009910 tx=0x7fa724009c20 comp rx=0 tx=0).stop 2024-09-10T11:46:06.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.443+0000 7fa741efb700 1 -- 172.21.15.144:0/3349326926 shutdown_connections 2024-09-10T11:46:06.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.443+0000 7fa741efb700 1 --2- 172.21.15.144:0/3349326926 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa73c0ffc70 0x7fa73c103c40 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.443+0000 7fa741efb700 1 --2- 172.21.15.144:0/3349326926 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa73c0ff2d0 0x7fa73c0ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.443+0000 7fa741efb700 1 -- 172.21.15.144:0/3349326926 >> 172.21.15.144:0/3349326926 conn(0x7fa73c0fabb0 msgr2=0x7fa73c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:06.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.443+0000 7fa741efb700 1 -- 172.21.15.144:0/3349326926 shutdown_connections 2024-09-10T11:46:06.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.443+0000 7fa741efb700 1 -- 172.21.15.144:0/3349326926 wait complete. 2024-09-10T11:46:06.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.443+0000 7fa741efb700 1 Processor -- start 2024-09-10T11:46:06.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.443+0000 7fa741efb700 1 -- start start 2024-09-10T11:46:06.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.444+0000 7fa741efb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa73c0ff2d0 0x7fa73c195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:06.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.444+0000 7fa741efb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa73c0ffc70 0x7fa73c196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:06.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.444+0000 7fa741efb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c196950 con 0x7fa73c0ff2d0 2024-09-10T11:46:06.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.444+0000 7fa741efb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c19a6e0 con 0x7fa73c0ffc70 2024-09-10T11:46:06.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.444+0000 7fa73b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa73c0ff2d0 0x7fa73c195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:06.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.444+0000 7fa73b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa73c0ff2d0 0x7fa73c195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:32994/0 (socket says 172.21.15.144:32994) 2024-09-10T11:46:06.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.444+0000 7fa73b7fe700 1 -- 172.21.15.144:0/2076713451 learned_addr learned my addr 172.21.15.144:0/2076713451 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:46:06.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.444+0000 7fa733fff700 1 --2- 172.21.15.144:0/2076713451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa73c0ffc70 0x7fa73c196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:06.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.444+0000 7fa73b7fe700 1 -- 172.21.15.144:0/2076713451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa73c0ffc70 msgr2=0x7fa73c196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:06.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.444+0000 7fa73b7fe700 1 --2- 172.21.15.144:0/2076713451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa73c0ffc70 0x7fa73c196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.444+0000 7fa73b7fe700 1 -- 172.21.15.144:0/2076713451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7240095f0 con 0x7fa73c0ff2d0 2024-09-10T11:46:06.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.444+0000 7fa73b7fe700 1 --2- 172.21.15.144:0/2076713451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa73c0ff2d0 0x7fa73c195d30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00d8d0 tx=0x7fa72c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:06.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.444+0000 7fa7397fa700 1 -- 172.21.15.144:0/2076713451 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa72c0098a0 con 0x7fa73c0ff2d0 2024-09-10T11:46:06.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.444+0000 7fa7397fa700 1 -- 172.21.15.144:0/2076713451 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa72c010460 con 0x7fa73c0ff2d0 2024-09-10T11:46:06.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.445+0000 7fa741efb700 1 -- 172.21.15.144:0/2076713451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c19a9c0 con 0x7fa73c0ff2d0 2024-09-10T11:46:06.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.445+0000 7fa7397fa700 1 -- 172.21.15.144:0/2076713451 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa72c00f5f0 con 0x7fa73c0ff2d0 2024-09-10T11:46:06.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.445+0000 7fa741efb700 1 -- 172.21.15.144:0/2076713451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c19af10 con 0x7fa73c0ff2d0 2024-09-10T11:46:06.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.445+0000 7fa741efb700 1 -- 172.21.15.144:0/2076713451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c108650 con 0x7fa73c0ff2d0 2024-09-10T11:46:06.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.446+0000 7fa7397fa700 1 -- 172.21.15.144:0/2076713451 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa72c009a00 con 0x7fa73c0ff2d0 2024-09-10T11:46:06.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.446+0000 7fa7397fa700 1 --2- 172.21.15.144:0/2076713451 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa72806e200 0x7fa7280706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:06.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.447+0000 7fa7397fa700 1 -- 172.21.15.144:0/2076713451 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa72c08df70 con 0x7fa73c0ff2d0 2024-09-10T11:46:06.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.447+0000 7fa733fff700 1 --2- 172.21.15.144:0/2076713451 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa72806e200 0x7fa7280706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:06.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.447+0000 7fa733fff700 1 --2- 172.21.15.144:0/2076713451 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa72806e200 0x7fa7280706b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa73c197350 tx=0x7fa72401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:06.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.451+0000 7fa7397fa700 1 -- 172.21.15.144:0/2076713451 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa72c0c31b0 con 0x7fa73c0ff2d0 2024-09-10T11:46:06.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.734+0000 7fa741efb700 1 -- 172.21.15.144:0/2076713451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa73c197090 con 0x7fa73c0ff2d0 2024-09-10T11:46:06.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.734+0000 7fa7397fa700 1 -- 172.21.15.144:0/2076713451 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fa73c197090 con 0x7fa73c0ff2d0 2024-09-10T11:46:06.736 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:46:06.736 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:46:06.737 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:46:06.737 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:46:06.737 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:46:06.737 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:46:06.737 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:46:06.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.737+0000 7fa741efb700 1 -- 172.21.15.144:0/2076713451 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa72806e200 msgr2=0x7fa7280706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:06.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.737+0000 7fa741efb700 1 --2- 172.21.15.144:0/2076713451 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa72806e200 0x7fa7280706b0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fa73c197350 tx=0x7fa72401a040 comp rx=0 tx=0).stop 2024-09-10T11:46:06.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.737+0000 7fa741efb700 1 -- 172.21.15.144:0/2076713451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa73c0ff2d0 msgr2=0x7fa73c195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:06.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.737+0000 7fa741efb700 1 --2- 172.21.15.144:0/2076713451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa73c0ff2d0 0x7fa73c195d30 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fa72c00d8d0 tx=0x7fa72c00dbe0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.737+0000 7fa741efb700 1 -- 172.21.15.144:0/2076713451 shutdown_connections 2024-09-10T11:46:06.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.737+0000 7fa741efb700 1 --2- 172.21.15.144:0/2076713451 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa72806e200 0x7fa7280706b0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.737+0000 7fa741efb700 1 --2- 172.21.15.144:0/2076713451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa73c0ff2d0 0x7fa73c195d30 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.737+0000 7fa741efb700 1 --2- 172.21.15.144:0/2076713451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa73c0ffc70 0x7fa73c196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:06.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.737+0000 7fa741efb700 1 -- 172.21.15.144:0/2076713451 >> 172.21.15.144:0/2076713451 conn(0x7fa73c0fabb0 msgr2=0x7fa73c068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:06.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.737+0000 7fa741efb700 1 -- 172.21.15.144:0/2076713451 shutdown_connections 2024-09-10T11:46:06.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:06.737+0000 7fa741efb700 1 -- 172.21.15.144:0/2076713451 wait complete. 2024-09-10T11:46:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:06 smithi178 ceph-mon[65940]: from='client.34824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:06 smithi178 ceph-mon[65940]: from='client.34828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:06 smithi178 ceph-mon[65940]: pgmap v423: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:06 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1613039967' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:46:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:06 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2076713451' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:46:07.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:06 smithi144 ceph-mon[87436]: from='client.34824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:06 smithi144 ceph-mon[87436]: from='client.34828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:06 smithi144 ceph-mon[87436]: pgmap v423: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:06 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1613039967' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:46:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:06 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2076713451' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:46:08.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:07 smithi178 ceph-mon[65940]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:08.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:07 smithi144 ceph-mon[87436]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:08 smithi178 ceph-mon[65940]: pgmap v424: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:09.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:08 smithi144 ceph-mon[87436]: pgmap v424: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:10 smithi178 ceph-mon[65940]: pgmap v425: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:11.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:10 smithi144 ceph-mon[87436]: pgmap v425: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:12 smithi144 ceph-mon[87436]: pgmap v426: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:13.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:12 smithi178 ceph-mon[65940]: pgmap v426: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:46:14.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:46:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:14 smithi178 ceph-mon[65940]: pgmap v427: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:15.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:14 smithi144 ceph-mon[87436]: pgmap v427: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:16 smithi178 ceph-mon[65940]: pgmap v428: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:17.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:16 smithi144 ceph-mon[87436]: pgmap v428: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:18 smithi178 ceph-mon[65940]: pgmap v429: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:19.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:18 smithi144 ceph-mon[87436]: pgmap v429: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:20.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:19 smithi144 ceph-mon[87436]: pgmap v430: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:20.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:19 smithi178 ceph-mon[65940]: pgmap v430: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:22 smithi178 ceph-mon[65940]: pgmap v431: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:23.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:22 smithi144 ceph-mon[87436]: pgmap v431: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:24 smithi178 ceph-mon[65940]: pgmap v432: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:25.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:24 smithi144 ceph-mon[87436]: pgmap v432: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:26 smithi178 ceph-mon[65940]: pgmap v433: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:26 smithi144 ceph-mon[87436]: pgmap v433: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:28 smithi178 ceph-mon[65940]: pgmap v434: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:46:29.145 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:28 smithi144 ceph-mon[87436]: pgmap v434: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:29.145 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:46:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:30 smithi178 ceph-mon[65940]: pgmap v435: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:31.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:30 smithi144 ceph-mon[87436]: pgmap v435: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:32 smithi144 ceph-mon[87436]: pgmap v436: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:33.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:32 smithi178 ceph-mon[65940]: pgmap v436: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:34 smithi178 ceph-mon[65940]: pgmap v437: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:34 smithi144 ceph-mon[87436]: pgmap v437: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:36.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.853+0000 7f2b64f2e700 1 -- 172.21.15.144:0/3513607935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b600ffba0 msgr2=0x7f2b6010a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:36.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.853+0000 7f2b64f2e700 1 --2- 172.21.15.144:0/3513607935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b600ffba0 0x7f2b6010a610 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009910 tx=0x7f2b54009c20 comp rx=0 tx=0).stop 2024-09-10T11:46:36.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.853+0000 7f2b64f2e700 1 -- 172.21.15.144:0/3513607935 shutdown_connections 2024-09-10T11:46:36.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.853+0000 7f2b64f2e700 1 --2- 172.21.15.144:0/3513607935 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b600ffba0 0x7f2b6010a610 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:36.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.853+0000 7f2b64f2e700 1 --2- 172.21.15.144:0/3513607935 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b600ff290 0x7f2b600ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:36.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.853+0000 7f2b64f2e700 1 -- 172.21.15.144:0/3513607935 >> 172.21.15.144:0/3513607935 conn(0x7f2b60074850 msgr2=0x7f2b60074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:36.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.853+0000 7f2b64f2e700 1 -- 172.21.15.144:0/3513607935 shutdown_connections 2024-09-10T11:46:36.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.853+0000 7f2b64f2e700 1 -- 172.21.15.144:0/3513607935 wait complete. 2024-09-10T11:46:36.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.854+0000 7f2b64f2e700 1 Processor -- start 2024-09-10T11:46:36.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.854+0000 7f2b64f2e700 1 -- start start 2024-09-10T11:46:36.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.854+0000 7f2b64f2e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b600ff290 0x7f2b60195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:36.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.854+0000 7f2b64f2e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b600ffba0 0x7f2b601963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:36.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.854+0000 7f2b64f2e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b60196a90 con 0x7f2b600ffba0 2024-09-10T11:46:36.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.854+0000 7f2b64f2e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b6019a820 con 0x7f2b600ff290 2024-09-10T11:46:36.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.854+0000 7f2b5dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b600ffba0 0x7f2b601963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:36.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.854+0000 7f2b5dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b600ffba0 0x7f2b601963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37174/0 (socket says 172.21.15.144:37174) 2024-09-10T11:46:36.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.854+0000 7f2b5dd9b700 1 -- 172.21.15.144:0/708947934 learned_addr learned my addr 172.21.15.144:0/708947934 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:46:36.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.855+0000 7f2b5e59c700 1 --2- 172.21.15.144:0/708947934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b600ff290 0x7f2b60195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:36.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.855+0000 7f2b5dd9b700 1 -- 172.21.15.144:0/708947934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b600ff290 msgr2=0x7f2b60195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:36.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.855+0000 7f2b5dd9b700 1 --2- 172.21.15.144:0/708947934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b600ff290 0x7f2b60195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:36.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.855+0000 7f2b5dd9b700 1 -- 172.21.15.144:0/708947934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b540095f0 con 0x7f2b600ffba0 2024-09-10T11:46:36.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.855+0000 7f2b5dd9b700 1 --2- 172.21.15.144:0/708947934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b600ffba0 0x7f2b601963b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2b54006010 tx=0x7f2b5400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:36.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.855+0000 7f2b4f7fe700 1 -- 172.21.15.144:0/708947934 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b5401d070 con 0x7f2b600ffba0 2024-09-10T11:46:36.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.855+0000 7f2b4f7fe700 1 -- 172.21.15.144:0/708947934 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b54004fb0 con 0x7f2b600ffba0 2024-09-10T11:46:36.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.855+0000 7f2b64f2e700 1 -- 172.21.15.144:0/708947934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6019aaa0 con 0x7f2b600ffba0 2024-09-10T11:46:36.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.855+0000 7f2b4f7fe700 1 -- 172.21.15.144:0/708947934 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b5400f690 con 0x7f2b600ffba0 2024-09-10T11:46:36.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.855+0000 7f2b64f2e700 1 -- 172.21.15.144:0/708947934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6019af90 con 0x7f2b600ffba0 2024-09-10T11:46:36.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.856+0000 7f2b64f2e700 1 -- 172.21.15.144:0/708947934 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b60107d10 con 0x7f2b600ffba0 2024-09-10T11:46:36.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.857+0000 7f2b4f7fe700 1 -- 172.21.15.144:0/708947934 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2b540228a0 con 0x7f2b600ffba0 2024-09-10T11:46:36.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.857+0000 7f2b4f7fe700 1 --2- 172.21.15.144:0/708947934 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2b4806e200 0x7f2b480706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:36.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.857+0000 7f2b4f7fe700 1 -- 172.21.15.144:0/708947934 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2b5408f8f0 con 0x7f2b600ffba0 2024-09-10T11:46:36.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.857+0000 7f2b5e59c700 1 --2- 172.21.15.144:0/708947934 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2b4806e200 0x7f2b480706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:36.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.858+0000 7f2b5e59c700 1 --2- 172.21.15.144:0/708947934 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2b4806e200 0x7f2b480706b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2b6006a5c0 tx=0x7f2b500094f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:36.864 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:36.862+0000 7f2b4f7fe700 1 -- 172.21.15.144:0/708947934 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2b54058cd0 con 0x7f2b600ffba0 2024-09-10T11:46:37.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.082+0000 7f2b64f2e700 1 -- 172.21.15.144:0/708947934 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2b6019b380 con 0x7f2b4806e200 2024-09-10T11:46:37.085 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:36 smithi144 ceph-mon[87436]: pgmap v438: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:37.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.084+0000 7f2b4f7fe700 1 -- 172.21.15.144:0/708947934 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f2b6019b380 con 0x7f2b4806e200 2024-09-10T11:46:37.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.086+0000 7f2b64f2e700 1 -- 172.21.15.144:0/708947934 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2b4806e200 msgr2=0x7f2b480706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:37.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.086+0000 7f2b64f2e700 1 --2- 172.21.15.144:0/708947934 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2b4806e200 0x7f2b480706b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f2b6006a5c0 tx=0x7f2b500094f0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.086+0000 7f2b64f2e700 1 -- 172.21.15.144:0/708947934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b600ffba0 msgr2=0x7f2b601963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:37.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.086+0000 7f2b64f2e700 1 --2- 172.21.15.144:0/708947934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b600ffba0 0x7f2b601963b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f2b54006010 tx=0x7f2b5400be20 comp rx=0 tx=0).stop 2024-09-10T11:46:37.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.086+0000 7f2b64f2e700 1 -- 172.21.15.144:0/708947934 shutdown_connections 2024-09-10T11:46:37.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.086+0000 7f2b64f2e700 1 --2- 172.21.15.144:0/708947934 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2b4806e200 0x7f2b480706b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.086+0000 7f2b64f2e700 1 --2- 172.21.15.144:0/708947934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2b600ff290 0x7f2b60195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.086+0000 7f2b64f2e700 1 --2- 172.21.15.144:0/708947934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2b600ffba0 0x7f2b601963b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.086+0000 7f2b64f2e700 1 -- 172.21.15.144:0/708947934 >> 172.21.15.144:0/708947934 conn(0x7f2b60074850 msgr2=0x7f2b60104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:37.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.086+0000 7f2b64f2e700 1 -- 172.21.15.144:0/708947934 shutdown_connections 2024-09-10T11:46:37.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.086+0000 7f2b64f2e700 1 -- 172.21.15.144:0/708947934 wait complete. 2024-09-10T11:46:37.098 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:46:37.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:36 smithi178 ceph-mon[65940]: pgmap v438: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:37.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.192+0000 7ff17cc13700 1 -- 172.21.15.144:0/2547843845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1780ff390 msgr2=0x7ff1780ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:37.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.192+0000 7ff17cc13700 1 --2- 172.21.15.144:0/2547843845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1780ff390 0x7ff1780ff760 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff17000b0a0 tx=0x7ff17000b3b0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.192+0000 7ff17cc13700 1 -- 172.21.15.144:0/2547843845 shutdown_connections 2024-09-10T11:46:37.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.192+0000 7ff17cc13700 1 --2- 172.21.15.144:0/2547843845 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1780ffd30 0x7ff178103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.192+0000 7ff17cc13700 1 --2- 172.21.15.144:0/2547843845 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1780ff390 0x7ff1780ff760 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.192+0000 7ff17cc13700 1 -- 172.21.15.144:0/2547843845 >> 172.21.15.144:0/2547843845 conn(0x7ff178074f50 msgr2=0x7ff178075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:37.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.192+0000 7ff17cc13700 1 -- 172.21.15.144:0/2547843845 shutdown_connections 2024-09-10T11:46:37.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.192+0000 7ff17cc13700 1 -- 172.21.15.144:0/2547843845 wait complete. 2024-09-10T11:46:37.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.193+0000 7ff17cc13700 1 Processor -- start 2024-09-10T11:46:37.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.193+0000 7ff17cc13700 1 -- start start 2024-09-10T11:46:37.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.193+0000 7ff17cc13700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1780ff390 0x7ff17819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:37.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.193+0000 7ff17cc13700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1780ffd30 0x7ff17819af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:37.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.193+0000 7ff17cc13700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17819b640 con 0x7ff1780ff390 2024-09-10T11:46:37.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.193+0000 7ff17cc13700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff178194b00 con 0x7ff1780ffd30 2024-09-10T11:46:37.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.193+0000 7ff17659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1780ff390 0x7ff17819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:37.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.193+0000 7ff17659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1780ff390 0x7ff17819aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37182/0 (socket says 172.21.15.144:37182) 2024-09-10T11:46:37.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.193+0000 7ff17659c700 1 -- 172.21.15.144:0/2857391717 learned_addr learned my addr 172.21.15.144:0/2857391717 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:46:37.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.193+0000 7ff17659c700 1 -- 172.21.15.144:0/2857391717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1780ffd30 msgr2=0x7ff17819af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:37.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.193+0000 7ff17659c700 1 --2- 172.21.15.144:0/2857391717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1780ffd30 0x7ff17819af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.193+0000 7ff17659c700 1 -- 172.21.15.144:0/2857391717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff170009d00 con 0x7ff1780ff390 2024-09-10T11:46:37.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.194+0000 7ff17659c700 1 --2- 172.21.15.144:0/2857391717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1780ff390 0x7ff17819aa20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff170000f80 tx=0x7ff17000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:37.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.194+0000 7ff16f7fe700 1 -- 172.21.15.144:0/2857391717 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff17000e040 con 0x7ff1780ff390 2024-09-10T11:46:37.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.194+0000 7ff16f7fe700 1 -- 172.21.15.144:0/2857391717 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff170003de0 con 0x7ff1780ff390 2024-09-10T11:46:37.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.194+0000 7ff16f7fe700 1 -- 172.21.15.144:0/2857391717 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff17001d790 con 0x7ff1780ff390 2024-09-10T11:46:37.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.194+0000 7ff17cc13700 1 -- 172.21.15.144:0/2857391717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff178194d80 con 0x7ff1780ff390 2024-09-10T11:46:37.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.194+0000 7ff17cc13700 1 -- 172.21.15.144:0/2857391717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff178195270 con 0x7ff1780ff390 2024-09-10T11:46:37.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.195+0000 7ff17cc13700 1 -- 172.21.15.144:0/2857391717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff178107650 con 0x7ff1780ff390 2024-09-10T11:46:37.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.195+0000 7ff16f7fe700 1 -- 172.21.15.144:0/2857391717 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff170004040 con 0x7ff1780ff390 2024-09-10T11:46:37.199 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.196+0000 7ff16f7fe700 1 --2- 172.21.15.144:0/2857391717 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff16406e200 0x7ff1640706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:37.199 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.196+0000 7ff16f7fe700 1 -- 172.21.15.144:0/2857391717 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff17008f890 con 0x7ff1780ff390 2024-09-10T11:46:37.199 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.196+0000 7ff175d9b700 1 --2- 172.21.15.144:0/2857391717 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff16406e200 0x7ff1640706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:37.199 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.196+0000 7ff175d9b700 1 --2- 172.21.15.144:0/2857391717 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff16406e200 0x7ff1640706b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff160005950 tx=0x7ff1600058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:37.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.200+0000 7ff16f7fe700 1 -- 172.21.15.144:0/2857391717 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff17005c7a0 con 0x7ff1780ff390 2024-09-10T11:46:37.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.423+0000 7ff17cc13700 1 -- 172.21.15.144:0/2857391717 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff17802cca0 con 0x7ff16406e200 2024-09-10T11:46:37.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.425+0000 7ff16f7fe700 1 -- 172.21.15.144:0/2857391717 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff17802cca0 con 0x7ff16406e200 2024-09-10T11:46:37.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.427+0000 7ff17cc13700 1 -- 172.21.15.144:0/2857391717 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff16406e200 msgr2=0x7ff1640706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:37.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.427+0000 7ff17cc13700 1 --2- 172.21.15.144:0/2857391717 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff16406e200 0x7ff1640706b0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff160005950 tx=0x7ff1600058e0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.427+0000 7ff17cc13700 1 -- 172.21.15.144:0/2857391717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1780ff390 msgr2=0x7ff17819aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:37.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.427+0000 7ff17cc13700 1 --2- 172.21.15.144:0/2857391717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1780ff390 0x7ff17819aa20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff170000f80 tx=0x7ff17000cdd0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.427+0000 7ff17cc13700 1 -- 172.21.15.144:0/2857391717 shutdown_connections 2024-09-10T11:46:37.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.427+0000 7ff17cc13700 1 --2- 172.21.15.144:0/2857391717 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff16406e200 0x7ff1640706b0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.427+0000 7ff17cc13700 1 --2- 172.21.15.144:0/2857391717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff1780ff390 0x7ff17819aa20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.427+0000 7ff17cc13700 1 --2- 172.21.15.144:0/2857391717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff1780ffd30 0x7ff17819af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.427+0000 7ff17cc13700 1 -- 172.21.15.144:0/2857391717 >> 172.21.15.144:0/2857391717 conn(0x7ff178074f50 msgr2=0x7ff1780fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:37.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.427+0000 7ff17cc13700 1 -- 172.21.15.144:0/2857391717 shutdown_connections 2024-09-10T11:46:37.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.427+0000 7ff17cc13700 1 -- 172.21.15.144:0/2857391717 wait complete. 2024-09-10T11:46:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.531+0000 7f6721a19700 1 -- 172.21.15.144:0/1973517152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c0ff320 msgr2=0x7f671c0ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.531+0000 7f6721a19700 1 --2- 172.21.15.144:0/1973517152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c0ff320 0x7f671c0ff6f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f671400b0a0 tx=0x7f671400b3b0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.531+0000 7f6721a19700 1 -- 172.21.15.144:0/1973517152 shutdown_connections 2024-09-10T11:46:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.531+0000 7f6721a19700 1 --2- 172.21.15.144:0/1973517152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f671c0ffc30 0x7f671c10a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.531+0000 7f6721a19700 1 --2- 172.21.15.144:0/1973517152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c0ff320 0x7f671c0ff6f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.531+0000 7f6721a19700 1 -- 172.21.15.144:0/1973517152 >> 172.21.15.144:0/1973517152 conn(0x7f671c0748b0 msgr2=0x7f671c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.531+0000 7f6721a19700 1 -- 172.21.15.144:0/1973517152 shutdown_connections 2024-09-10T11:46:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.531+0000 7f6721a19700 1 -- 172.21.15.144:0/1973517152 wait complete. 2024-09-10T11:46:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.532+0000 7f6721a19700 1 Processor -- start 2024-09-10T11:46:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.532+0000 7f6721a19700 1 -- start start 2024-09-10T11:46:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.532+0000 7f6721a19700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f671c0ff320 0x7f671c19b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.532+0000 7f6721a19700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c0ffc30 0x7f671c19bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.532+0000 7f6721a19700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c19c5c0 con 0x7f671c0ffc30 2024-09-10T11:46:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.532+0000 7f6721a19700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f671c195a70 con 0x7f671c0ff320 2024-09-10T11:46:37.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.532+0000 7f671a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c0ffc30 0x7f671c19bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:37.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.532+0000 7f671a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c0ffc30 0x7f671c19bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37200/0 (socket says 172.21.15.144:37200) 2024-09-10T11:46:37.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.532+0000 7f671a7fc700 1 -- 172.21.15.144:0/4258833134 learned_addr learned my addr 172.21.15.144:0/4258833134 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:46:37.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.532+0000 7f671a7fc700 1 -- 172.21.15.144:0/4258833134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f671c0ff320 msgr2=0x7f671c19b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:37.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.533+0000 7f671a7fc700 1 --2- 172.21.15.144:0/4258833134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f671c0ff320 0x7f671c19b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.533+0000 7f671a7fc700 1 -- 172.21.15.144:0/4258833134 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6714009d00 con 0x7f671c0ffc30 2024-09-10T11:46:37.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.533+0000 7f671a7fc700 1 --2- 172.21.15.144:0/4258833134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c0ffc30 0x7f671c19bf30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f670400e920 tx=0x7f670400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:37.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.533+0000 7f6720a17700 1 -- 172.21.15.144:0/4258833134 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6704009790 con 0x7f671c0ffc30 2024-09-10T11:46:37.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.533+0000 7f6720a17700 1 -- 172.21.15.144:0/4258833134 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f670400cba0 con 0x7f671c0ffc30 2024-09-10T11:46:37.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.533+0000 7f6721a19700 1 -- 172.21.15.144:0/4258833134 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f671c195d50 con 0x7f671c0ffc30 2024-09-10T11:46:37.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.533+0000 7f6720a17700 1 -- 172.21.15.144:0/4258833134 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6704018650 con 0x7f671c0ffc30 2024-09-10T11:46:37.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.533+0000 7f6721a19700 1 -- 172.21.15.144:0/4258833134 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671c196320 con 0x7f671c0ffc30 2024-09-10T11:46:37.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.534+0000 7f6721a19700 1 -- 172.21.15.144:0/4258833134 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f671c107da0 con 0x7f671c0ffc30 2024-09-10T11:46:37.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.534+0000 7f6720a17700 1 -- 172.21.15.144:0/4258833134 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f670400cd10 con 0x7f671c0ffc30 2024-09-10T11:46:37.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.535+0000 7f6720a17700 1 --2- 172.21.15.144:0/4258833134 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f670806e200 0x7f67080706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:37.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.535+0000 7f6720a17700 1 -- 172.21.15.144:0/4258833134 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6704014070 con 0x7f671c0ffc30 2024-09-10T11:46:37.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.535+0000 7f671affd700 1 --2- 172.21.15.144:0/4258833134 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f670806e200 0x7f67080706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:37.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.535+0000 7f671affd700 1 --2- 172.21.15.144:0/4258833134 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f670806e200 0x7f67080706b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6714009480 tx=0x7f67140060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:37.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.539+0000 7f6720a17700 1 -- 172.21.15.144:0/4258833134 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f67040541b0 con 0x7f671c0ffc30 2024-09-10T11:46:37.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.750+0000 7f6721a19700 1 -- 172.21.15.144:0/4258833134 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f671c196590 con 0x7f670806e200 2024-09-10T11:46:37.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.760+0000 7f6720a17700 1 -- 172.21.15.144:0/4258833134 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f671c196590 con 0x7f670806e200 2024-09-10T11:46:37.762 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:46:37.762 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (14m) 8m ago 25m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:46:37.762 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (25m) 8m ago 25m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:46:37.762 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (24m) 8m ago 24m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:46:37.762 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (12m) 8m ago 25m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:46:37.762 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (12m) 8m ago 24m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:46:37.762 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (14m) 8m ago 25m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:46:37.763 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (8m) 8m ago 19m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:46:37.763 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (8m) 8m ago 19m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:46:37.763 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (19m) 8m ago 19m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:46:37.763 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (8m) 8m ago 19m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:46:37.763 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (16m) 8m ago 26m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:46:37.763 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (15m) 8m ago 24m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:46:37.763 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (13m) 8m ago 27m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:46:37.763 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (13m) 8m ago 24m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:46:37.763 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (15m) 8m ago 25m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:46:37.763 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (15m) 8m ago 24m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:46:37.763 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (12m) 8m ago 23m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:46:37.763 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (11m) 8m ago 23m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:46:37.763 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (10m) 8m ago 22m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:46:37.763 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (10m) 8m ago 22m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:46:37.763 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (9m) 8m ago 21m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:46:37.764 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (9m) 8m ago 21m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:46:37.764 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (15m) 8m ago 24m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:46:37.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.762+0000 7f6721a19700 1 -- 172.21.15.144:0/4258833134 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f670806e200 msgr2=0x7f67080706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:37.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.762+0000 7f6721a19700 1 --2- 172.21.15.144:0/4258833134 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f670806e200 0x7f67080706b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6714009480 tx=0x7f67140060c0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.762+0000 7f6721a19700 1 -- 172.21.15.144:0/4258833134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c0ffc30 msgr2=0x7f671c19bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:37.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.762+0000 7f6721a19700 1 --2- 172.21.15.144:0/4258833134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c0ffc30 0x7f671c19bf30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f670400e920 tx=0x7f670400ece0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.762+0000 7f6721a19700 1 -- 172.21.15.144:0/4258833134 shutdown_connections 2024-09-10T11:46:37.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.762+0000 7f6721a19700 1 --2- 172.21.15.144:0/4258833134 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f670806e200 0x7f67080706b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.762+0000 7f6721a19700 1 --2- 172.21.15.144:0/4258833134 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f671c0ff320 0x7f671c19b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.762+0000 7f6721a19700 1 --2- 172.21.15.144:0/4258833134 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f671c0ffc30 0x7f671c19bf30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.762+0000 7f6721a19700 1 -- 172.21.15.144:0/4258833134 >> 172.21.15.144:0/4258833134 conn(0x7f671c0748b0 msgr2=0x7f671c104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:37.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.762+0000 7f6721a19700 1 -- 172.21.15.144:0/4258833134 shutdown_connections 2024-09-10T11:46:37.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.762+0000 7f6721a19700 1 -- 172.21.15.144:0/4258833134 wait complete. 2024-09-10T11:46:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.868+0000 7fa169bef700 1 -- 172.21.15.144:0/759543469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1640ffce0 msgr2=0x7fa164103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.868+0000 7fa169bef700 1 --2- 172.21.15.144:0/759543469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1640ffce0 0x7fa164103cb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa158009910 tx=0x7fa158009c20 comp rx=0 tx=0).stop 2024-09-10T11:46:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.868+0000 7fa169bef700 1 -- 172.21.15.144:0/759543469 shutdown_connections 2024-09-10T11:46:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.868+0000 7fa169bef700 1 --2- 172.21.15.144:0/759543469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1640ffce0 0x7fa164103cb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.868+0000 7fa169bef700 1 --2- 172.21.15.144:0/759543469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1640ff340 0x7fa1640ff710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.868+0000 7fa169bef700 1 -- 172.21.15.144:0/759543469 >> 172.21.15.144:0/759543469 conn(0x7fa1640fabe0 msgr2=0x7fa1640fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:37.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.868+0000 7fa169bef700 1 -- 172.21.15.144:0/759543469 shutdown_connections 2024-09-10T11:46:37.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.868+0000 7fa169bef700 1 -- 172.21.15.144:0/759543469 wait complete. 2024-09-10T11:46:37.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.868+0000 7fa169bef700 1 Processor -- start 2024-09-10T11:46:37.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.869+0000 7fa169bef700 1 -- start start 2024-09-10T11:46:37.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.869+0000 7fa169bef700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1640ff340 0x7fa164195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:37.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.869+0000 7fa169bef700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1640ffce0 0x7fa1641962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:37.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.869+0000 7fa169bef700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa164196990 con 0x7fa1640ffce0 2024-09-10T11:46:37.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.869+0000 7fa169bef700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16419a720 con 0x7fa1640ff340 2024-09-10T11:46:37.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.869+0000 7fa162ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1640ffce0 0x7fa1641962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:37.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.869+0000 7fa162ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1640ffce0 0x7fa1641962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37218/0 (socket says 172.21.15.144:37218) 2024-09-10T11:46:37.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.869+0000 7fa162ffd700 1 -- 172.21.15.144:0/222624454 learned_addr learned my addr 172.21.15.144:0/222624454 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:46:37.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.869+0000 7fa1637fe700 1 --2- 172.21.15.144:0/222624454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1640ff340 0x7fa164195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:37.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.869+0000 7fa162ffd700 1 -- 172.21.15.144:0/222624454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1640ff340 msgr2=0x7fa164195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:37.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.869+0000 7fa162ffd700 1 --2- 172.21.15.144:0/222624454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1640ff340 0x7fa164195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:37.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.869+0000 7fa162ffd700 1 -- 172.21.15.144:0/222624454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1580095f0 con 0x7fa1640ffce0 2024-09-10T11:46:37.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.869+0000 7fa162ffd700 1 --2- 172.21.15.144:0/222624454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1640ffce0 0x7fa1641962b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa15800bd70 tx=0x7fa15800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:37.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.870+0000 7fa160ff9700 1 -- 172.21.15.144:0/222624454 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa15801d070 con 0x7fa1640ffce0 2024-09-10T11:46:37.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.870+0000 7fa160ff9700 1 -- 172.21.15.144:0/222624454 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa158005dc0 con 0x7fa1640ffce0 2024-09-10T11:46:37.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.870+0000 7fa169bef700 1 -- 172.21.15.144:0/222624454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa16419a9a0 con 0x7fa1640ffce0 2024-09-10T11:46:37.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.870+0000 7fa160ff9700 1 -- 172.21.15.144:0/222624454 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa158017400 con 0x7fa1640ffce0 2024-09-10T11:46:37.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.870+0000 7fa169bef700 1 -- 172.21.15.144:0/222624454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa16419ae90 con 0x7fa1640ffce0 2024-09-10T11:46:37.874 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.871+0000 7fa169bef700 1 -- 172.21.15.144:0/222624454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa16404ea50 con 0x7fa1640ffce0 2024-09-10T11:46:37.874 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.871+0000 7fa160ff9700 1 -- 172.21.15.144:0/222624454 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa15800f7f0 con 0x7fa1640ffce0 2024-09-10T11:46:37.874 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.871+0000 7fa160ff9700 1 --2- 172.21.15.144:0/222624454 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa14c06e200 0x7fa14c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:37.874 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.872+0000 7fa160ff9700 1 -- 172.21.15.144:0/222624454 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa15808f570 con 0x7fa1640ffce0 2024-09-10T11:46:37.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.872+0000 7fa1637fe700 1 --2- 172.21.15.144:0/222624454 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa14c06e200 0x7fa14c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:37.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.872+0000 7fa1637fe700 1 --2- 172.21.15.144:0/222624454 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa14c06e200 0x7fa14c0706b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa154009d10 tx=0x7fa154009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:37.878 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:37.876+0000 7fa160ff9700 1 -- 172.21.15.144:0/222624454 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa158058950 con 0x7fa1640ffce0 2024-09-10T11:46:38.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.165+0000 7fa169bef700 1 -- 172.21.15.144:0/222624454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa16419b240 con 0x7fa1640ffce0 2024-09-10T11:46:38.168 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.166+0000 7fa160ff9700 1 -- 172.21.15.144:0/222624454 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fa15805c010 con 0x7fa1640ffce0 2024-09-10T11:46:38.168 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:46:38.168 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:46:38.168 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:46:38.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:46:38.168 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:46:38.168 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:46:38.168 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:46:38.168 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:46:38.169 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:46:38.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:46:38.169 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:46:38.169 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:46:38.169 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:46:38.169 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:46:38.169 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:46:38.169 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:46:38.169 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:46:38.169 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:46:38.169 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:46:38.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.168+0000 7fa169bef700 1 -- 172.21.15.144:0/222624454 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa14c06e200 msgr2=0x7fa14c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:38.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.168+0000 7fa169bef700 1 --2- 172.21.15.144:0/222624454 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa14c06e200 0x7fa14c0706b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa154009d10 tx=0x7fa154009040 comp rx=0 tx=0).stop 2024-09-10T11:46:38.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.168+0000 7fa169bef700 1 -- 172.21.15.144:0/222624454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1640ffce0 msgr2=0x7fa1641962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:38.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.168+0000 7fa169bef700 1 --2- 172.21.15.144:0/222624454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1640ffce0 0x7fa1641962b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fa15800bd70 tx=0x7fa15800be50 comp rx=0 tx=0).stop 2024-09-10T11:46:38.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.168+0000 7fa169bef700 1 -- 172.21.15.144:0/222624454 shutdown_connections 2024-09-10T11:46:38.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.168+0000 7fa169bef700 1 --2- 172.21.15.144:0/222624454 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa14c06e200 0x7fa14c0706b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.168+0000 7fa169bef700 1 --2- 172.21.15.144:0/222624454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1640ff340 0x7fa164195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.168+0000 7fa169bef700 1 --2- 172.21.15.144:0/222624454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa1640ffce0 0x7fa1641962b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.168+0000 7fa169bef700 1 -- 172.21.15.144:0/222624454 >> 172.21.15.144:0/222624454 conn(0x7fa1640fabe0 msgr2=0x7fa1640fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:38.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.169+0000 7fa169bef700 1 -- 172.21.15.144:0/222624454 shutdown_connections 2024-09-10T11:46:38.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.169+0000 7fa169bef700 1 -- 172.21.15.144:0/222624454 wait complete. 2024-09-10T11:46:38.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.274+0000 7f1d0b3b6700 1 -- 172.21.15.144:0/1298527523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d040ff2d0 msgr2=0x7f1d040ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:38.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.274+0000 7f1d0b3b6700 1 --2- 172.21.15.144:0/1298527523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d040ff2d0 0x7f1d040ff6a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f1d00009910 tx=0x7f1d00009c20 comp rx=0 tx=0).stop 2024-09-10T11:46:38.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.274+0000 7f1d0b3b6700 1 -- 172.21.15.144:0/1298527523 shutdown_connections 2024-09-10T11:46:38.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.274+0000 7f1d0b3b6700 1 --2- 172.21.15.144:0/1298527523 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d040ffc70 0x7f1d04103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.274+0000 7f1d0b3b6700 1 --2- 172.21.15.144:0/1298527523 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d040ff2d0 0x7f1d040ff6a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.274+0000 7f1d0b3b6700 1 -- 172.21.15.144:0/1298527523 >> 172.21.15.144:0/1298527523 conn(0x7f1d040fabb0 msgr2=0x7f1d040fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:38.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.274+0000 7f1d0b3b6700 1 -- 172.21.15.144:0/1298527523 shutdown_connections 2024-09-10T11:46:38.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.274+0000 7f1d0b3b6700 1 -- 172.21.15.144:0/1298527523 wait complete. 2024-09-10T11:46:38.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.274+0000 7f1d0b3b6700 1 Processor -- start 2024-09-10T11:46:38.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1d0b3b6700 1 -- start start 2024-09-10T11:46:38.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1d0b3b6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d040ff2d0 0x7f1d04100ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:38.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1d0b3b6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d040ffc70 0x7f1d04101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:38.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1d0b3b6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d04101760 con 0x7f1d040ff2d0 2024-09-10T11:46:38.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1d0b3b6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d041018a0 con 0x7f1d040ffc70 2024-09-10T11:46:38.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1d09152700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d040ff2d0 0x7f1d04100ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:38.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1d09152700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d040ff2d0 0x7f1d04100ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37244/0 (socket says 172.21.15.144:37244) 2024-09-10T11:46:38.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1d09152700 1 -- 172.21.15.144:0/1728090099 learned_addr learned my addr 172.21.15.144:0/1728090099 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:46:38.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1d09152700 1 -- 172.21.15.144:0/1728090099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d040ffc70 msgr2=0x7f1d04101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:38.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1d09152700 1 --2- 172.21.15.144:0/1728090099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d040ffc70 0x7f1d04101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1d09152700 1 -- 172.21.15.144:0/1728090099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d000095f0 con 0x7f1d040ff2d0 2024-09-10T11:46:38.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1d09152700 1 --2- 172.21.15.144:0/1728090099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d040ff2d0 0x7f1d04100ce0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1d00006010 tx=0x7f1d0000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:38.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1cfa7fc700 1 -- 172.21.15.144:0/1728090099 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d0001d070 con 0x7f1d040ff2d0 2024-09-10T11:46:38.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1d0b3b6700 1 -- 172.21.15.144:0/1728090099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d04101b20 con 0x7f1d040ff2d0 2024-09-10T11:46:38.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1cfa7fc700 1 -- 172.21.15.144:0/1728090099 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d0000f460 con 0x7f1d040ff2d0 2024-09-10T11:46:38.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.275+0000 7f1cfa7fc700 1 -- 172.21.15.144:0/1728090099 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d00017630 con 0x7f1d040ff2d0 2024-09-10T11:46:38.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.276+0000 7f1d0b3b6700 1 -- 172.21.15.144:0/1728090099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d041a2280 con 0x7f1d040ff2d0 2024-09-10T11:46:38.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.276+0000 7f1d0b3b6700 1 -- 172.21.15.144:0/1728090099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d0404ea50 con 0x7f1d040ff2d0 2024-09-10T11:46:38.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.277+0000 7f1cfa7fc700 1 -- 172.21.15.144:0/1728090099 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1d0000f5d0 con 0x7f1d040ff2d0 2024-09-10T11:46:38.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.277+0000 7f1cfa7fc700 1 --2- 172.21.15.144:0/1728090099 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1cf006e200 0x7f1cf00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:38.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.278+0000 7f1cfa7fc700 1 -- 172.21.15.144:0/1728090099 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1d0008f540 con 0x7f1d040ff2d0 2024-09-10T11:46:38.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.278+0000 7f1d08951700 1 --2- 172.21.15.144:0/1728090099 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1cf006e200 0x7f1cf00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:38.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.278+0000 7f1d08951700 1 --2- 172.21.15.144:0/1728090099 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1cf006e200 0x7f1cf00706b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4005950 tx=0x7f1cf40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:38.283 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.282+0000 7f1cfa7fc700 1 -- 172.21.15.144:0/1728090099 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1d00058920 con 0x7f1d040ff2d0 2024-09-10T11:46:38.527 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.525+0000 7f1d0b3b6700 1 -- 172.21.15.144:0/1728090099 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1d04102210 con 0x7f1d040ff2d0 2024-09-10T11:46:38.528 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.525+0000 7f1cfa7fc700 1 -- 172.21.15.144:0/1728090099 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f1d0005bfe0 con 0x7f1d040ff2d0 2024-09-10T11:46:38.528 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:46:38.528 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:46:38.528 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:46:38.528 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:46:38.528 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:46:38.528 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:46:38.528 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:46:38.529 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:46:38.529 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:46:38.529 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:46:38.529 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:46:38.529 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:46:38.529 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:46:38.529 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:46:38.529 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:46:38.529 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:46:38.529 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:46:38.529 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:46:38.529 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:46:38.529 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:46:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:46:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:46:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:46:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:46:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:46:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:46:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:46:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:46:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:46:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:46:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:46:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:46:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:46:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:46:38.530 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:46:38.531 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:46:38.531 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:46:38.531 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:46:38.531 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:46:38.531 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:46:38.531 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:46:38.531 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:46:38.531 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:46:38.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.528+0000 7f1d0b3b6700 1 -- 172.21.15.144:0/1728090099 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1cf006e200 msgr2=0x7f1cf00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:38.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.528+0000 7f1d0b3b6700 1 --2- 172.21.15.144:0/1728090099 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1cf006e200 0x7f1cf00706b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4005950 tx=0x7f1cf40058e0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.528+0000 7f1d0b3b6700 1 -- 172.21.15.144:0/1728090099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d040ff2d0 msgr2=0x7f1d04100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:38.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.528+0000 7f1d0b3b6700 1 --2- 172.21.15.144:0/1728090099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d040ff2d0 0x7f1d04100ce0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f1d00006010 tx=0x7f1d0000be20 comp rx=0 tx=0).stop 2024-09-10T11:46:38.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.529+0000 7f1d0b3b6700 1 -- 172.21.15.144:0/1728090099 shutdown_connections 2024-09-10T11:46:38.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.529+0000 7f1d0b3b6700 1 --2- 172.21.15.144:0/1728090099 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1cf006e200 0x7f1cf00706b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.529+0000 7f1d0b3b6700 1 --2- 172.21.15.144:0/1728090099 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d040ff2d0 0x7f1d04100ce0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.529+0000 7f1d0b3b6700 1 --2- 172.21.15.144:0/1728090099 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d040ffc70 0x7f1d04101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.529+0000 7f1d0b3b6700 1 -- 172.21.15.144:0/1728090099 >> 172.21.15.144:0/1728090099 conn(0x7f1d040fabb0 msgr2=0x7f1d04068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:38.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.529+0000 7f1d0b3b6700 1 -- 172.21.15.144:0/1728090099 shutdown_connections 2024-09-10T11:46:38.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.529+0000 7f1d0b3b6700 1 -- 172.21.15.144:0/1728090099 wait complete. 2024-09-10T11:46:38.533 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:46:38.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.634+0000 7fbf4c0f2700 1 -- 172.21.15.144:0/1914923453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf440ffa30 msgr2=0x7fbf44103f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:38.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.634+0000 7fbf4c0f2700 1 --2- 172.21.15.144:0/1914923453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf440ffa30 0x7fbf44103f10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fbf34009910 tx=0x7fbf34009c20 comp rx=0 tx=0).stop 2024-09-10T11:46:38.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.634+0000 7fbf4c0f2700 1 -- 172.21.15.144:0/1914923453 shutdown_connections 2024-09-10T11:46:38.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.634+0000 7fbf4c0f2700 1 --2- 172.21.15.144:0/1914923453 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf440ffa30 0x7fbf44103f10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.634+0000 7fbf4c0f2700 1 --2- 172.21.15.144:0/1914923453 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf440ff120 0x7fbf440ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.634+0000 7fbf4c0f2700 1 -- 172.21.15.144:0/1914923453 >> 172.21.15.144:0/1914923453 conn(0x7fbf440fabb0 msgr2=0x7fbf440fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:38.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.634+0000 7fbf4c0f2700 1 -- 172.21.15.144:0/1914923453 shutdown_connections 2024-09-10T11:46:38.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.634+0000 7fbf4c0f2700 1 -- 172.21.15.144:0/1914923453 wait complete. 2024-09-10T11:46:38.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf4c0f2700 1 Processor -- start 2024-09-10T11:46:38.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf4c0f2700 1 -- start start 2024-09-10T11:46:38.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf4c0f2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf440ff120 0x7fbf4419c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:38.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf4c0f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf440ffa30 0x7fbf4419c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:38.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf4c0f2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4419cf50 con 0x7fbf440ffa30 2024-09-10T11:46:38.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf4c0f2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf441a0ce0 con 0x7fbf440ff120 2024-09-10T11:46:38.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf4968d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf440ffa30 0x7fbf4419c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:38.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf4968d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf440ffa30 0x7fbf4419c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37256/0 (socket says 172.21.15.144:37256) 2024-09-10T11:46:38.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf4968d700 1 -- 172.21.15.144:0/893513407 learned_addr learned my addr 172.21.15.144:0/893513407 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:46:38.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf49e8e700 1 --2- 172.21.15.144:0/893513407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf440ff120 0x7fbf4419c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:38.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf4968d700 1 -- 172.21.15.144:0/893513407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf440ff120 msgr2=0x7fbf4419c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:38.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf4968d700 1 --2- 172.21.15.144:0/893513407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf440ff120 0x7fbf4419c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf4968d700 1 -- 172.21.15.144:0/893513407 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf340095f0 con 0x7fbf440ffa30 2024-09-10T11:46:38.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf49e8e700 1 --2- 172.21.15.144:0/893513407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf440ff120 0x7fbf4419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:46:38.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf4968d700 1 --2- 172.21.15.144:0/893513407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf440ffa30 0x7fbf4419c870 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbf34006010 tx=0x7fbf3400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:38.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.635+0000 7fbf3affd700 1 -- 172.21.15.144:0/893513407 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf3401d070 con 0x7fbf440ffa30 2024-09-10T11:46:38.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.636+0000 7fbf3affd700 1 -- 172.21.15.144:0/893513407 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbf3400f460 con 0x7fbf440ffa30 2024-09-10T11:46:38.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.636+0000 7fbf3affd700 1 -- 172.21.15.144:0/893513407 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf34017630 con 0x7fbf440ffa30 2024-09-10T11:46:38.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.636+0000 7fbf4c0f2700 1 -- 172.21.15.144:0/893513407 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf441a0f60 con 0x7fbf440ffa30 2024-09-10T11:46:38.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.636+0000 7fbf4c0f2700 1 -- 172.21.15.144:0/893513407 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf441a1450 con 0x7fbf440ffa30 2024-09-10T11:46:38.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.636+0000 7fbf4c0f2700 1 -- 172.21.15.144:0/893513407 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf441015e0 con 0x7fbf440ffa30 2024-09-10T11:46:38.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.637+0000 7fbf3affd700 1 -- 172.21.15.144:0/893513407 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fbf3400f5d0 con 0x7fbf440ffa30 2024-09-10T11:46:38.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.638+0000 7fbf3affd700 1 --2- 172.21.15.144:0/893513407 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf3006e200 0x7fbf300706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:38.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.638+0000 7fbf3affd700 1 -- 172.21.15.144:0/893513407 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbf3408f570 con 0x7fbf440ffa30 2024-09-10T11:46:38.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.638+0000 7fbf49e8e700 1 --2- 172.21.15.144:0/893513407 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf3006e200 0x7fbf300706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:38.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.638+0000 7fbf49e8e700 1 --2- 172.21.15.144:0/893513407 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf3006e200 0x7fbf300706b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbf40005950 tx=0x7fbf400058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:38.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.642+0000 7fbf3affd700 1 -- 172.21.15.144:0/893513407 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbf34058950 con 0x7fbf440ffa30 2024-09-10T11:46:38.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.867+0000 7fbf4c0f2700 1 -- 172.21.15.144:0/893513407 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf4402cd10 con 0x7fbf3006e200 2024-09-10T11:46:38.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.868+0000 7fbf3affd700 1 -- 172.21.15.144:0/893513407 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbf4402cd10 con 0x7fbf3006e200 2024-09-10T11:46:38.870 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:46:38.870 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:46:38.870 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:46:38.870 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:46:38.870 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:46:38.870 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:46:38.870 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:46:38.870 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:46:38.871 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:46:38.871 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:46:38.871 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:46:38.871 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:46:38.871 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:46:38.871 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:46:38.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.870+0000 7fbf4c0f2700 1 -- 172.21.15.144:0/893513407 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf3006e200 msgr2=0x7fbf300706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:38.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.870+0000 7fbf4c0f2700 1 --2- 172.21.15.144:0/893513407 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf3006e200 0x7fbf300706b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fbf40005950 tx=0x7fbf400058e0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.870+0000 7fbf4c0f2700 1 -- 172.21.15.144:0/893513407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf440ffa30 msgr2=0x7fbf4419c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:38.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.870+0000 7fbf4c0f2700 1 --2- 172.21.15.144:0/893513407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf440ffa30 0x7fbf4419c870 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbf34006010 tx=0x7fbf3400be20 comp rx=0 tx=0).stop 2024-09-10T11:46:38.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.870+0000 7fbf4c0f2700 1 -- 172.21.15.144:0/893513407 shutdown_connections 2024-09-10T11:46:38.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.870+0000 7fbf4c0f2700 1 --2- 172.21.15.144:0/893513407 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbf3006e200 0x7fbf300706b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.870+0000 7fbf4c0f2700 1 --2- 172.21.15.144:0/893513407 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbf440ff120 0x7fbf4419c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.870+0000 7fbf4c0f2700 1 --2- 172.21.15.144:0/893513407 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbf440ffa30 0x7fbf4419c870 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.870+0000 7fbf4c0f2700 1 -- 172.21.15.144:0/893513407 >> 172.21.15.144:0/893513407 conn(0x7fbf440fabb0 msgr2=0x7fbf44110590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:38.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.871+0000 7fbf4c0f2700 1 -- 172.21.15.144:0/893513407 shutdown_connections 2024-09-10T11:46:38.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.871+0000 7fbf4c0f2700 1 -- 172.21.15.144:0/893513407 wait complete. 2024-09-10T11:46:38.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.975+0000 7f91230b1700 1 -- 172.21.15.144:0/3658362914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f911c104790 msgr2=0x7f911c104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:38.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.975+0000 7f91230b1700 1 --2- 172.21.15.144:0/3658362914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f911c104790 0x7f911c104b60 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f911000aa20 tx=0x7f911000ad30 comp rx=0 tx=0).stop 2024-09-10T11:46:38.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.976+0000 7f91230b1700 1 -- 172.21.15.144:0/3658362914 shutdown_connections 2024-09-10T11:46:38.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.976+0000 7f91230b1700 1 --2- 172.21.15.144:0/3658362914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f911c103450 0x7f911c1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.976+0000 7f91230b1700 1 --2- 172.21.15.144:0/3658362914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f911c104790 0x7f911c104b60 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.976+0000 7f91230b1700 1 -- 172.21.15.144:0/3658362914 >> 172.21.15.144:0/3658362914 conn(0x7f911c074850 msgr2=0x7f911c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:38.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.976+0000 7f91230b1700 1 -- 172.21.15.144:0/3658362914 shutdown_connections 2024-09-10T11:46:38.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.976+0000 7f91230b1700 1 -- 172.21.15.144:0/3658362914 wait complete. 2024-09-10T11:46:38.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.976+0000 7f91230b1700 1 Processor -- start 2024-09-10T11:46:38.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.976+0000 7f91230b1700 1 -- start start 2024-09-10T11:46:38.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.976+0000 7f91230b1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f911c103450 0x7f911c19bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:38.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.976+0000 7f91230b1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f911c104790 0x7f911c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:38.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.976+0000 7f91230b1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c19c680 con 0x7f911c103450 2024-09-10T11:46:38.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.976+0000 7f91230b1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c195b30 con 0x7f911c104790 2024-09-10T11:46:38.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.977+0000 7f9120e4d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f911c103450 0x7f911c19bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:38.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.977+0000 7f9120e4d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f911c103450 0x7f911c19bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37274/0 (socket says 172.21.15.144:37274) 2024-09-10T11:46:38.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.977+0000 7f9120e4d700 1 -- 172.21.15.144:0/2255058372 learned_addr learned my addr 172.21.15.144:0/2255058372 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:46:38.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.977+0000 7f911bfff700 1 --2- 172.21.15.144:0/2255058372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f911c104790 0x7f911c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:38.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.977+0000 7f9120e4d700 1 -- 172.21.15.144:0/2255058372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f911c104790 msgr2=0x7f911c19bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:38.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.977+0000 7f9120e4d700 1 --2- 172.21.15.144:0/2255058372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f911c104790 0x7f911c19bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:38.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.977+0000 7f9120e4d700 1 -- 172.21.15.144:0/2255058372 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911000a700 con 0x7f911c103450 2024-09-10T11:46:38.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.977+0000 7f9120e4d700 1 --2- 172.21.15.144:0/2255058372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f911c103450 0x7f911c19bab0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9110006010 tx=0x7f911000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:38.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.977+0000 7f9119ffb700 1 -- 172.21.15.144:0/2255058372 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9110004af0 con 0x7f911c103450 2024-09-10T11:46:38.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.977+0000 7f91230b1700 1 -- 172.21.15.144:0/2255058372 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911c195db0 con 0x7f911c103450 2024-09-10T11:46:38.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.977+0000 7f9119ffb700 1 -- 172.21.15.144:0/2255058372 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9110004c50 con 0x7f911c103450 2024-09-10T11:46:38.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.977+0000 7f9119ffb700 1 -- 172.21.15.144:0/2255058372 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9110016660 con 0x7f911c103450 2024-09-10T11:46:38.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.977+0000 7f91230b1700 1 -- 172.21.15.144:0/2255058372 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911c1962a0 con 0x7f911c103450 2024-09-10T11:46:38.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.978+0000 7f91230b1700 1 -- 172.21.15.144:0/2255058372 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911c0fda10 con 0x7f911c103450 2024-09-10T11:46:38.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.979+0000 7f9119ffb700 1 -- 172.21.15.144:0/2255058372 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f911000e450 con 0x7f911c103450 2024-09-10T11:46:38.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.979+0000 7f9119ffb700 1 --2- 172.21.15.144:0/2255058372 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f910c06e200 0x7f910c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:46:38.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.979+0000 7f9119ffb700 1 -- 172.21.15.144:0/2255058372 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9110012070 con 0x7f911c103450 2024-09-10T11:46:38.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.979+0000 7f911bfff700 1 --2- 172.21.15.144:0/2255058372 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f910c06e200 0x7f910c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:46:38.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.980+0000 7f911bfff700 1 --2- 172.21.15.144:0/2255058372 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f910c06e200 0x7f910c0706b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f911c074dd0 tx=0x7f910800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:46:38.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:38.984+0000 7f9119ffb700 1 -- 172.21.15.144:0/2255058372 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f911005d060 con 0x7f911c103450 2024-09-10T11:46:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:38 smithi178 ceph-mon[65940]: from='client.34848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:38 smithi178 ceph-mon[65940]: from='client.34852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:38 smithi178 ceph-mon[65940]: pgmap v439: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:38 smithi178 ceph-mon[65940]: from='client.34856 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:38 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/222624454' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:46:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:38 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1728090099' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:46:39.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:38 smithi144 ceph-mon[87436]: from='client.34848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:38 smithi144 ceph-mon[87436]: from='client.34852 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:38 smithi144 ceph-mon[87436]: pgmap v439: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:38 smithi144 ceph-mon[87436]: from='client.34856 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:38 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/222624454' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:46:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:38 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1728090099' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:46:39.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:39.272+0000 7f91230b1700 1 -- 172.21.15.144:0/2255058372 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f911c196580 con 0x7f911c103450 2024-09-10T11:46:39.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:39.272+0000 7f9119ffb700 1 -- 172.21.15.144:0/2255058372 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f911005cbf0 con 0x7f911c103450 2024-09-10T11:46:39.275 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:46:39.275 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:46:39.275 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:46:39.275 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:46:39.275 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:46:39.275 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:46:39.275 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:46:39.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:39.275+0000 7f91230b1700 1 -- 172.21.15.144:0/2255058372 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f910c06e200 msgr2=0x7f910c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:39.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:39.275+0000 7f91230b1700 1 --2- 172.21.15.144:0/2255058372 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f910c06e200 0x7f910c0706b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f911c074dd0 tx=0x7f910800b410 comp rx=0 tx=0).stop 2024-09-10T11:46:39.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:39.275+0000 7f91230b1700 1 -- 172.21.15.144:0/2255058372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f911c103450 msgr2=0x7f911c19bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:46:39.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:39.275+0000 7f91230b1700 1 --2- 172.21.15.144:0/2255058372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f911c103450 0x7f911c19bab0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9110006010 tx=0x7f911000cd20 comp rx=0 tx=0).stop 2024-09-10T11:46:39.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:39.275+0000 7f91230b1700 1 -- 172.21.15.144:0/2255058372 shutdown_connections 2024-09-10T11:46:39.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:39.275+0000 7f91230b1700 1 --2- 172.21.15.144:0/2255058372 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f910c06e200 0x7f910c0706b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:39.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:39.275+0000 7f91230b1700 1 --2- 172.21.15.144:0/2255058372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f911c103450 0x7f911c19bab0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:39.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:39.275+0000 7f91230b1700 1 --2- 172.21.15.144:0/2255058372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f911c104790 0x7f911c19bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:46:39.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:39.275+0000 7f91230b1700 1 -- 172.21.15.144:0/2255058372 >> 172.21.15.144:0/2255058372 conn(0x7f911c074850 msgr2=0x7f911c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:46:39.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:39.275+0000 7f91230b1700 1 -- 172.21.15.144:0/2255058372 shutdown_connections 2024-09-10T11:46:39.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:46:39.275+0000 7f91230b1700 1 -- 172.21.15.144:0/2255058372 wait complete. 2024-09-10T11:46:40.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:39 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2255058372' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:46:40.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:39 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2255058372' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:46:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:40 smithi178 ceph-mon[65940]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:40 smithi178 ceph-mon[65940]: pgmap v440: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:41.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:40 smithi144 ceph-mon[87436]: from='client.34868 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:46:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:40 smithi144 ceph-mon[87436]: pgmap v440: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:43.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:42 smithi144 ceph-mon[87436]: pgmap v441: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:43.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:42 smithi178 ceph-mon[65940]: pgmap v441: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:46:44.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:46:45.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:44 smithi178 ceph-mon[65940]: pgmap v442: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:45.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:44 smithi144 ceph-mon[87436]: pgmap v442: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:46.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:45 smithi144 ceph-mon[87436]: pgmap v443: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:46.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:45 smithi178 ceph-mon[65940]: pgmap v443: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:48 smithi178 ceph-mon[65940]: pgmap v444: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:49.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:48 smithi144 ceph-mon[87436]: pgmap v444: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:50 smithi178 ceph-mon[65940]: pgmap v445: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:51.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:50 smithi144 ceph-mon[87436]: pgmap v445: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:53.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:52 smithi144 ceph-mon[87436]: pgmap v446: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:52 smithi178 ceph-mon[65940]: pgmap v446: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:54 smithi178 ceph-mon[65940]: pgmap v447: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:55.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:54 smithi144 ceph-mon[87436]: pgmap v447: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:56 smithi178 ceph-mon[65940]: pgmap v448: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:57.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:56 smithi144 ceph-mon[87436]: pgmap v448: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:59.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:58 smithi178 ceph-mon[65940]: pgmap v449: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:46:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:46:59.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:58 smithi144 ceph-mon[87436]: pgmap v449: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:46:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:46:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:47:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:00 smithi178 ceph-mon[65940]: pgmap v450: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:01.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:00 smithi144 ceph-mon[87436]: pgmap v450: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:03.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:02 smithi144 ceph-mon[87436]: pgmap v451: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:03.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:02 smithi178 ceph-mon[65940]: pgmap v451: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:04 smithi178 ceph-mon[65940]: pgmap v452: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:05.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:04 smithi144 ceph-mon[87436]: pgmap v452: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:06 smithi178 ceph-mon[65940]: pgmap v453: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:07.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:06 smithi144 ceph-mon[87436]: pgmap v453: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:08 smithi178 ceph-mon[65940]: pgmap v454: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:08 smithi144 ceph-mon[87436]: pgmap v454: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:09.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.397+0000 7f9b1d926700 1 -- 172.21.15.144:0/2755168971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b180ff2c0 msgr2=0x7f9b180ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:09.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.397+0000 7f9b1d926700 1 --2- 172.21.15.144:0/2755168971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b180ff2c0 0x7f9b180ff690 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f9b0800b0a0 tx=0x7f9b0800b3b0 comp rx=0 tx=0).stop 2024-09-10T11:47:09.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.397+0000 7f9b1d926700 1 -- 172.21.15.144:0/2755168971 shutdown_connections 2024-09-10T11:47:09.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.397+0000 7f9b1d926700 1 --2- 172.21.15.144:0/2755168971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b180ffc60 0x7f9b18103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:09.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.397+0000 7f9b1d926700 1 --2- 172.21.15.144:0/2755168971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b180ff2c0 0x7f9b180ff690 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:09.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.397+0000 7f9b1d926700 1 -- 172.21.15.144:0/2755168971 >> 172.21.15.144:0/2755168971 conn(0x7f9b180fab80 msgr2=0x7f9b180fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:09.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.397+0000 7f9b1d926700 1 -- 172.21.15.144:0/2755168971 shutdown_connections 2024-09-10T11:47:09.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.397+0000 7f9b1d926700 1 -- 172.21.15.144:0/2755168971 wait complete. 2024-09-10T11:47:09.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.398+0000 7f9b1d926700 1 Processor -- start 2024-09-10T11:47:09.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.398+0000 7f9b1d926700 1 -- start start 2024-09-10T11:47:09.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b1d926700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b180ff2c0 0x7f9b18196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:09.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b1d926700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b180ffc60 0x7f9b181970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:09.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b1d926700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b181976d0 con 0x7f9b180ffc60 2024-09-10T11:47:09.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b1d926700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b18197840 con 0x7f9b180ff2c0 2024-09-10T11:47:09.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b167fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b180ffc60 0x7f9b181970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:09.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b167fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b180ffc60 0x7f9b181970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33972/0 (socket says 172.21.15.144:33972) 2024-09-10T11:47:09.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b167fc700 1 -- 172.21.15.144:0/2124657144 learned_addr learned my addr 172.21.15.144:0/2124657144 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:47:09.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b16ffd700 1 --2- 172.21.15.144:0/2124657144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b180ff2c0 0x7f9b18196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:09.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b167fc700 1 -- 172.21.15.144:0/2124657144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b180ff2c0 msgr2=0x7f9b18196b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:09.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b167fc700 1 --2- 172.21.15.144:0/2124657144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b180ff2c0 0x7f9b18196b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:09.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b167fc700 1 -- 172.21.15.144:0/2124657144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b08009d00 con 0x7f9b180ffc60 2024-09-10T11:47:09.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b167fc700 1 --2- 172.21.15.144:0/2124657144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b180ffc60 0x7f9b181970b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000d900 tx=0x7f9b0000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:09.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b1c924700 1 -- 172.21.15.144:0/2124657144 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b00009900 con 0x7f9b180ffc60 2024-09-10T11:47:09.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b1d926700 1 -- 172.21.15.144:0/2124657144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b1819fbe0 con 0x7f9b180ffc60 2024-09-10T11:47:09.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b1c924700 1 -- 172.21.15.144:0/2124657144 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b00010460 con 0x7f9b180ffc60 2024-09-10T11:47:09.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b1c924700 1 -- 172.21.15.144:0/2124657144 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b0000f5f0 con 0x7f9b180ffc60 2024-09-10T11:47:09.408 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.399+0000 7f9b1d926700 1 -- 172.21.15.144:0/2124657144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b181a00d0 con 0x7f9b180ffc60 2024-09-10T11:47:09.408 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.400+0000 7f9b1d926700 1 -- 172.21.15.144:0/2124657144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b181075d0 con 0x7f9b180ffc60 2024-09-10T11:47:09.408 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.401+0000 7f9b1c924700 1 -- 172.21.15.144:0/2124657144 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9b000105d0 con 0x7f9b180ffc60 2024-09-10T11:47:09.408 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.401+0000 7f9b1c924700 1 --2- 172.21.15.144:0/2124657144 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9b0406e200 0x7f9b040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:09.408 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.402+0000 7f9b1c924700 1 -- 172.21.15.144:0/2124657144 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9b0008dc50 con 0x7f9b180ffc60 2024-09-10T11:47:09.408 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.402+0000 7f9b16ffd700 1 --2- 172.21.15.144:0/2124657144 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9b0406e200 0x7f9b040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:09.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.402+0000 7f9b16ffd700 1 --2- 172.21.15.144:0/2124657144 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9b0406e200 0x7f9b040706b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9b08000f80 tx=0x7f9b080060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:09.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.406+0000 7f9b1c924700 1 -- 172.21.15.144:0/2124657144 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9b00057030 con 0x7f9b180ffc60 2024-09-10T11:47:09.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.629+0000 7f9b1d926700 1 -- 172.21.15.144:0/2124657144 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b1802cd10 con 0x7f9b0406e200 2024-09-10T11:47:09.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.630+0000 7f9b1c924700 1 -- 172.21.15.144:0/2124657144 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9b1802cd10 con 0x7f9b0406e200 2024-09-10T11:47:09.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.632+0000 7f9b1d926700 1 -- 172.21.15.144:0/2124657144 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9b0406e200 msgr2=0x7f9b040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:09.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.632+0000 7f9b1d926700 1 --2- 172.21.15.144:0/2124657144 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9b0406e200 0x7f9b040706b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f9b08000f80 tx=0x7f9b080060c0 comp rx=0 tx=0).stop 2024-09-10T11:47:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.633+0000 7f9b1d926700 1 -- 172.21.15.144:0/2124657144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b180ffc60 msgr2=0x7f9b181970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.633+0000 7f9b1d926700 1 --2- 172.21.15.144:0/2124657144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b180ffc60 0x7f9b181970b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000d900 tx=0x7f9b0000dc10 comp rx=0 tx=0).stop 2024-09-10T11:47:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.633+0000 7f9b1d926700 1 -- 172.21.15.144:0/2124657144 shutdown_connections 2024-09-10T11:47:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.633+0000 7f9b1d926700 1 --2- 172.21.15.144:0/2124657144 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9b0406e200 0x7f9b040706b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:09.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.633+0000 7f9b1d926700 1 --2- 172.21.15.144:0/2124657144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b180ff2c0 0x7f9b18196b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:09.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.633+0000 7f9b1d926700 1 --2- 172.21.15.144:0/2124657144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b180ffc60 0x7f9b181970b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:09.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.633+0000 7f9b1d926700 1 -- 172.21.15.144:0/2124657144 >> 172.21.15.144:0/2124657144 conn(0x7f9b180fab80 msgr2=0x7f9b180fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:09.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.633+0000 7f9b1d926700 1 -- 172.21.15.144:0/2124657144 shutdown_connections 2024-09-10T11:47:09.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.633+0000 7f9b1d926700 1 -- 172.21.15.144:0/2124657144 wait complete. 2024-09-10T11:47:09.645 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:47:09.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.740+0000 7f1dcbafc700 1 -- 172.21.15.144:0/1361916713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc40ff270 msgr2=0x7f1dc40ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:09.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.740+0000 7f1dcbafc700 1 --2- 172.21.15.144:0/1361916713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc40ff270 0x7f1dc40ff640 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0009910 tx=0x7f1dc0009c20 comp rx=0 tx=0).stop 2024-09-10T11:47:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.741+0000 7f1dcbafc700 1 -- 172.21.15.144:0/1361916713 shutdown_connections 2024-09-10T11:47:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.741+0000 7f1dcbafc700 1 --2- 172.21.15.144:0/1361916713 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dc40ffb80 0x7f1dc410a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.741+0000 7f1dcbafc700 1 --2- 172.21.15.144:0/1361916713 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc40ff270 0x7f1dc40ff640 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.741+0000 7f1dcbafc700 1 -- 172.21.15.144:0/1361916713 >> 172.21.15.144:0/1361916713 conn(0x7f1dc4074850 msgr2=0x7f1dc4074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:09.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.741+0000 7f1dcbafc700 1 -- 172.21.15.144:0/1361916713 shutdown_connections 2024-09-10T11:47:09.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.741+0000 7f1dcbafc700 1 -- 172.21.15.144:0/1361916713 wait complete. 2024-09-10T11:47:09.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.741+0000 7f1dcbafc700 1 Processor -- start 2024-09-10T11:47:09.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.741+0000 7f1dcbafc700 1 -- start start 2024-09-10T11:47:09.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.741+0000 7f1dcbafc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc40ff270 0x7f1dc419ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:09.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.741+0000 7f1dcbafc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dc40ffb80 0x7f1dc419bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:09.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.741+0000 7f1dcbafc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc419c5f0 con 0x7f1dc40ff270 2024-09-10T11:47:09.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.741+0000 7f1dcbafc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dc4195aa0 con 0x7f1dc40ffb80 2024-09-10T11:47:09.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.742+0000 7f1dc9898700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc40ff270 0x7f1dc419ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:09.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.742+0000 7f1dc9898700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc40ff270 0x7f1dc419ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33982/0 (socket says 172.21.15.144:33982) 2024-09-10T11:47:09.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.742+0000 7f1dc9898700 1 -- 172.21.15.144:0/468054699 learned_addr learned my addr 172.21.15.144:0/468054699 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:47:09.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.742+0000 7f1dc9097700 1 --2- 172.21.15.144:0/468054699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dc40ffb80 0x7f1dc419bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:09.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.742+0000 7f1dc9898700 1 -- 172.21.15.144:0/468054699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dc40ffb80 msgr2=0x7f1dc419bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:09.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.742+0000 7f1dc9898700 1 --2- 172.21.15.144:0/468054699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dc40ffb80 0x7f1dc419bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:09.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.742+0000 7f1dc9898700 1 -- 172.21.15.144:0/468054699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dc00095f0 con 0x7f1dc40ff270 2024-09-10T11:47:09.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.742+0000 7f1dc9097700 1 --2- 172.21.15.144:0/468054699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dc40ffb80 0x7f1dc419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:47:09.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.742+0000 7f1dc9898700 1 --2- 172.21.15.144:0/468054699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc40ff270 0x7f1dc419ba20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006010 tx=0x7f1dc000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:09.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.742+0000 7f1db6ffd700 1 -- 172.21.15.144:0/468054699 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dc001d070 con 0x7f1dc40ff270 2024-09-10T11:47:09.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.742+0000 7f1db6ffd700 1 -- 172.21.15.144:0/468054699 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1dc000f460 con 0x7f1dc40ff270 2024-09-10T11:47:09.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.742+0000 7f1dcbafc700 1 -- 172.21.15.144:0/468054699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dc4195d80 con 0x7f1dc40ff270 2024-09-10T11:47:09.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.742+0000 7f1db6ffd700 1 -- 172.21.15.144:0/468054699 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dc0017630 con 0x7f1dc40ff270 2024-09-10T11:47:09.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.742+0000 7f1dcbafc700 1 -- 172.21.15.144:0/468054699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dc4196350 con 0x7f1dc40ff270 2024-09-10T11:47:09.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.743+0000 7f1dcbafc700 1 -- 172.21.15.144:0/468054699 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc4107cf0 con 0x7f1dc40ff270 2024-09-10T11:47:09.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.744+0000 7f1db6ffd700 1 -- 172.21.15.144:0/468054699 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1dc000f5d0 con 0x7f1dc40ff270 2024-09-10T11:47:09.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.744+0000 7f1db6ffd700 1 --2- 172.21.15.144:0/468054699 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1db006e200 0x7f1db00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:09.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.745+0000 7f1db6ffd700 1 -- 172.21.15.144:0/468054699 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1dc008f4a0 con 0x7f1dc40ff270 2024-09-10T11:47:09.755 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.745+0000 7f1dc9097700 1 --2- 172.21.15.144:0/468054699 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1db006e200 0x7f1db00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:09.755 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.745+0000 7f1dc9097700 1 --2- 172.21.15.144:0/468054699 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1db006e200 0x7f1db00706b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1db8005950 tx=0x7f1db80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:09.756 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.749+0000 7f1db6ffd700 1 -- 172.21.15.144:0/468054699 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1dc005c3b0 con 0x7f1dc40ff270 2024-09-10T11:47:09.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.973+0000 7f1dcbafc700 1 -- 172.21.15.144:0/468054699 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1dc4196610 con 0x7f1db006e200 2024-09-10T11:47:09.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.975+0000 7f1db6ffd700 1 -- 172.21.15.144:0/468054699 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f1dc4196610 con 0x7f1db006e200 2024-09-10T11:47:09.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.977+0000 7f1dcbafc700 1 -- 172.21.15.144:0/468054699 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1db006e200 msgr2=0x7f1db00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:09.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.977+0000 7f1dcbafc700 1 --2- 172.21.15.144:0/468054699 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1db006e200 0x7f1db00706b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1db8005950 tx=0x7f1db80058e0 comp rx=0 tx=0).stop 2024-09-10T11:47:09.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.977+0000 7f1dcbafc700 1 -- 172.21.15.144:0/468054699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc40ff270 msgr2=0x7f1dc419ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:09.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.977+0000 7f1dcbafc700 1 --2- 172.21.15.144:0/468054699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc40ff270 0x7f1dc419ba20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006010 tx=0x7f1dc000be20 comp rx=0 tx=0).stop 2024-09-10T11:47:09.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.977+0000 7f1dcbafc700 1 -- 172.21.15.144:0/468054699 shutdown_connections 2024-09-10T11:47:09.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.977+0000 7f1dcbafc700 1 --2- 172.21.15.144:0/468054699 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1db006e200 0x7f1db00706b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:09.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.977+0000 7f1dcbafc700 1 --2- 172.21.15.144:0/468054699 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1dc40ff270 0x7f1dc419ba20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:09.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.977+0000 7f1dcbafc700 1 --2- 172.21.15.144:0/468054699 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1dc40ffb80 0x7f1dc419bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:09.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.977+0000 7f1dcbafc700 1 -- 172.21.15.144:0/468054699 >> 172.21.15.144:0/468054699 conn(0x7f1dc4074850 msgr2=0x7f1dc4104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:09.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.977+0000 7f1dcbafc700 1 -- 172.21.15.144:0/468054699 shutdown_connections 2024-09-10T11:47:09.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:09.977+0000 7f1dcbafc700 1 -- 172.21.15.144:0/468054699 wait complete. 2024-09-10T11:47:10.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.082+0000 7fadefdb3700 1 -- 172.21.15.144:0/3981050959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade80ffd30 msgr2=0x7fade8103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:10.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.082+0000 7fadefdb3700 1 --2- 172.21.15.144:0/3981050959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade80ffd30 0x7fade8103ce0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fade4009910 tx=0x7fade4009c20 comp rx=0 tx=0).stop 2024-09-10T11:47:10.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.082+0000 7fadefdb3700 1 -- 172.21.15.144:0/3981050959 shutdown_connections 2024-09-10T11:47:10.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.082+0000 7fadefdb3700 1 --2- 172.21.15.144:0/3981050959 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade80ffd30 0x7fade8103ce0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.082+0000 7fadefdb3700 1 --2- 172.21.15.144:0/3981050959 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fade80ff390 0x7fade80ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.082+0000 7fadefdb3700 1 -- 172.21.15.144:0/3981050959 >> 172.21.15.144:0/3981050959 conn(0x7fade8074f50 msgr2=0x7fade8075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:10.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.082+0000 7fadefdb3700 1 -- 172.21.15.144:0/3981050959 shutdown_connections 2024-09-10T11:47:10.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.082+0000 7fadefdb3700 1 -- 172.21.15.144:0/3981050959 wait complete. 2024-09-10T11:47:10.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.083+0000 7fadefdb3700 1 Processor -- start 2024-09-10T11:47:10.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.083+0000 7fadefdb3700 1 -- start start 2024-09-10T11:47:10.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.083+0000 7fadefdb3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fade80ff390 0x7fade8100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:10.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.083+0000 7fadefdb3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade80ffd30 0x7fade81012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:10.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.083+0000 7fadefdb3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade8101800 con 0x7fade80ffd30 2024-09-10T11:47:10.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.083+0000 7fadefdb3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade8101940 con 0x7fade80ff390 2024-09-10T11:47:10.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.083+0000 7faded34e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade80ffd30 0x7fade81012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:10.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.083+0000 7faded34e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade80ffd30 0x7fade81012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34012/0 (socket says 172.21.15.144:34012) 2024-09-10T11:47:10.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.083+0000 7faded34e700 1 -- 172.21.15.144:0/1036756044 learned_addr learned my addr 172.21.15.144:0/1036756044 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:47:10.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.083+0000 7fadedb4f700 1 --2- 172.21.15.144:0/1036756044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fade80ff390 0x7fade8100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:10.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.084+0000 7faded34e700 1 -- 172.21.15.144:0/1036756044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fade80ff390 msgr2=0x7fade8100d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:10.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.084+0000 7faded34e700 1 --2- 172.21.15.144:0/1036756044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fade80ff390 0x7fade8100d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.084+0000 7faded34e700 1 -- 172.21.15.144:0/1036756044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade40095f0 con 0x7fade80ffd30 2024-09-10T11:47:10.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.084+0000 7faded34e700 1 --2- 172.21.15.144:0/1036756044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade80ffd30 0x7fade81012c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fade40092e0 tx=0x7fade400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:10.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.084+0000 7faddeffd700 1 -- 172.21.15.144:0/1036756044 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fade401d070 con 0x7fade80ffd30 2024-09-10T11:47:10.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.084+0000 7fadefdb3700 1 -- 172.21.15.144:0/1036756044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade8101bc0 con 0x7fade80ffd30 2024-09-10T11:47:10.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.084+0000 7faddeffd700 1 -- 172.21.15.144:0/1036756044 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fade4005dc0 con 0x7fade80ffd30 2024-09-10T11:47:10.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.084+0000 7fadefdb3700 1 -- 172.21.15.144:0/1036756044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade81a2290 con 0x7fade80ffd30 2024-09-10T11:47:10.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.084+0000 7faddeffd700 1 -- 172.21.15.144:0/1036756044 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fade4017400 con 0x7fade80ffd30 2024-09-10T11:47:10.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.085+0000 7fadefdb3700 1 -- 172.21.15.144:0/1036756044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadcc0052f0 con 0x7fade80ffd30 2024-09-10T11:47:10.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.086+0000 7faddeffd700 1 -- 172.21.15.144:0/1036756044 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fade400f460 con 0x7fade80ffd30 2024-09-10T11:47:10.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.086+0000 7faddeffd700 1 --2- 172.21.15.144:0/1036756044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fadd406e200 0x7fadd40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:10.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.086+0000 7faddeffd700 1 -- 172.21.15.144:0/1036756044 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fade408f590 con 0x7fade80ffd30 2024-09-10T11:47:10.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.087+0000 7fadedb4f700 1 --2- 172.21.15.144:0/1036756044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fadd406e200 0x7fadd40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:10.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.088+0000 7fadedb4f700 1 --2- 172.21.15.144:0/1036756044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fadd406e200 0x7fadd40706b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fadd8005950 tx=0x7fadd80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:10.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.091+0000 7faddeffd700 1 -- 172.21.15.144:0/1036756044 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fade40588d0 con 0x7fade80ffd30 2024-09-10T11:47:10.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.301+0000 7fadefdb3700 1 -- 172.21.15.144:0/1036756044 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fadcc000bc0 con 0x7fadd406e200 2024-09-10T11:47:10.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.310+0000 7faddeffd700 1 -- 172.21.15.144:0/1036756044 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fadcc000bc0 con 0x7fadd406e200 2024-09-10T11:47:10.311 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:47:10.312 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (15m) 9m ago 25m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:47:10.312 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (26m) 9m ago 26m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:47:10.312 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (25m) 8m ago 25m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:47:10.312 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (13m) 9m ago 26m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:47:10.312 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (12m) 8m ago 25m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:47:10.312 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (15m) 9m ago 25m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:47:10.312 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (9m) 9m ago 20m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:47:10.312 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (9m) 9m ago 20m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:47:10.312 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (20m) 8m ago 20m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:47:10.312 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (8m) 8m ago 20m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:47:10.312 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (17m) 9m ago 27m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:47:10.312 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (16m) 8m ago 25m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:47:10.313 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (14m) 9m ago 27m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:47:10.313 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (13m) 8m ago 24m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:47:10.313 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (16m) 9m ago 26m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:47:10.313 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (15m) 8m ago 25m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:47:10.313 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (12m) 9m ago 24m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:47:10.313 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (12m) 9m ago 23m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:47:10.313 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (11m) 9m ago 23m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:47:10.313 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (10m) 8m ago 22m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:47:10.313 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (10m) 8m ago 22m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:47:10.313 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (9m) 8m ago 21m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:47:10.313 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (15m) 9m ago 25m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:47:10.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.312+0000 7fadefdb3700 1 -- 172.21.15.144:0/1036756044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fadd406e200 msgr2=0x7fadd40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:10.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.312+0000 7fadefdb3700 1 --2- 172.21.15.144:0/1036756044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fadd406e200 0x7fadd40706b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fadd8005950 tx=0x7fadd80058e0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.312+0000 7fadefdb3700 1 -- 172.21.15.144:0/1036756044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade80ffd30 msgr2=0x7fade81012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:10.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.312+0000 7fadefdb3700 1 --2- 172.21.15.144:0/1036756044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade80ffd30 0x7fade81012c0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fade40092e0 tx=0x7fade400be20 comp rx=0 tx=0).stop 2024-09-10T11:47:10.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.312+0000 7fadefdb3700 1 -- 172.21.15.144:0/1036756044 shutdown_connections 2024-09-10T11:47:10.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.312+0000 7fadefdb3700 1 --2- 172.21.15.144:0/1036756044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fadd406e200 0x7fadd40706b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.312+0000 7fadefdb3700 1 --2- 172.21.15.144:0/1036756044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fade80ff390 0x7fade8100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.312+0000 7fadefdb3700 1 --2- 172.21.15.144:0/1036756044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fade80ffd30 0x7fade81012c0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.312+0000 7fadefdb3700 1 -- 172.21.15.144:0/1036756044 >> 172.21.15.144:0/1036756044 conn(0x7fade8074f50 msgr2=0x7fade8103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:10.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.313+0000 7fadefdb3700 1 -- 172.21.15.144:0/1036756044 shutdown_connections 2024-09-10T11:47:10.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.313+0000 7fadefdb3700 1 -- 172.21.15.144:0/1036756044 wait complete. 2024-09-10T11:47:10.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.416+0000 7f3ebfd7c700 1 -- 172.21.15.144:0/1141406253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb80ffd20 msgr2=0x7f3eb8103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:10.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.416+0000 7f3ebfd7c700 1 --2- 172.21.15.144:0/1141406253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb80ffd20 0x7f3eb8103cf0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f3eac009910 tx=0x7f3eac009c20 comp rx=0 tx=0).stop 2024-09-10T11:47:10.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.416+0000 7f3ebfd7c700 1 -- 172.21.15.144:0/1141406253 shutdown_connections 2024-09-10T11:47:10.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.416+0000 7f3ebfd7c700 1 --2- 172.21.15.144:0/1141406253 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb80ffd20 0x7f3eb8103cf0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.416+0000 7f3ebfd7c700 1 --2- 172.21.15.144:0/1141406253 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eb80ff380 0x7f3eb80ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.416+0000 7f3ebfd7c700 1 -- 172.21.15.144:0/1141406253 >> 172.21.15.144:0/1141406253 conn(0x7f3eb80fac20 msgr2=0x7f3eb80fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:10.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.416+0000 7f3ebfd7c700 1 -- 172.21.15.144:0/1141406253 shutdown_connections 2024-09-10T11:47:10.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.416+0000 7f3ebfd7c700 1 -- 172.21.15.144:0/1141406253 wait complete. 2024-09-10T11:47:10.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.417+0000 7f3ebfd7c700 1 Processor -- start 2024-09-10T11:47:10.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.417+0000 7f3ebfd7c700 1 -- start start 2024-09-10T11:47:10.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.417+0000 7f3ebfd7c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eb80ff380 0x7f3eb8100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:10.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.417+0000 7f3ebfd7c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb80ffd20 0x7f3eb81012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:10.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.417+0000 7f3ebfd7c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb8101810 con 0x7f3eb80ffd20 2024-09-10T11:47:10.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.417+0000 7f3ebfd7c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb8101950 con 0x7f3eb80ff380 2024-09-10T11:47:10.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.417+0000 7f3ebd317700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb80ffd20 0x7f3eb81012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:10.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.417+0000 7f3ebd317700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb80ffd20 0x7f3eb81012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34028/0 (socket says 172.21.15.144:34028) 2024-09-10T11:47:10.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.417+0000 7f3ebd317700 1 -- 172.21.15.144:0/42646195 learned_addr learned my addr 172.21.15.144:0/42646195 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:47:10.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.418+0000 7f3ebdb18700 1 --2- 172.21.15.144:0/42646195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eb80ff380 0x7f3eb8100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:10.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.418+0000 7f3ebd317700 1 -- 172.21.15.144:0/42646195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eb80ff380 msgr2=0x7f3eb8100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:10.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.418+0000 7f3ebd317700 1 --2- 172.21.15.144:0/42646195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eb80ff380 0x7f3eb8100d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.418+0000 7f3ebd317700 1 -- 172.21.15.144:0/42646195 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eac0095f0 con 0x7f3eb80ffd20 2024-09-10T11:47:10.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.418+0000 7f3ebd317700 1 --2- 172.21.15.144:0/42646195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb80ffd20 0x7f3eb81012d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0092e0 tx=0x7f3eac00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:10.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.418+0000 7f3eb6ffd700 1 -- 172.21.15.144:0/42646195 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eac01d070 con 0x7f3eb80ffd20 2024-09-10T11:47:10.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.418+0000 7f3eb6ffd700 1 -- 172.21.15.144:0/42646195 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3eac005dc0 con 0x7f3eb80ffd20 2024-09-10T11:47:10.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.418+0000 7f3ebfd7c700 1 -- 172.21.15.144:0/42646195 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb8101bd0 con 0x7f3eb80ffd20 2024-09-10T11:47:10.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.418+0000 7f3eb6ffd700 1 -- 172.21.15.144:0/42646195 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eac017400 con 0x7f3eb80ffd20 2024-09-10T11:47:10.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.419+0000 7f3ebfd7c700 1 -- 172.21.15.144:0/42646195 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb81a2280 con 0x7f3eb80ffd20 2024-09-10T11:47:10.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.419+0000 7f3ebfd7c700 1 -- 172.21.15.144:0/42646195 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb804ea50 con 0x7f3eb80ffd20 2024-09-10T11:47:10.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.420+0000 7f3eb6ffd700 1 -- 172.21.15.144:0/42646195 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3eac003a60 con 0x7f3eb80ffd20 2024-09-10T11:47:10.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.421+0000 7f3eb6ffd700 1 --2- 172.21.15.144:0/42646195 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3ea006e250 0x7f3ea0070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:10.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.421+0000 7f3eb6ffd700 1 -- 172.21.15.144:0/42646195 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3eac08f530 con 0x7f3eb80ffd20 2024-09-10T11:47:10.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.421+0000 7f3ebdb18700 1 --2- 172.21.15.144:0/42646195 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3ea006e250 0x7f3ea0070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:10.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.421+0000 7f3ebdb18700 1 --2- 172.21.15.144:0/42646195 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3ea006e250 0x7f3ea0070700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009d10 tx=0x7f3ea8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.425+0000 7f3eb6ffd700 1 -- 172.21.15.144:0/42646195 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3eac058a30 con 0x7f3eb80ffd20 2024-09-10T11:47:10.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.711+0000 7f3ebfd7c700 1 -- 172.21.15.144:0/42646195 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3eb81a2690 con 0x7f3eb80ffd20 2024-09-10T11:47:10.713 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.711+0000 7f3eb6ffd700 1 -- 172.21.15.144:0/42646195 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f3eac05c0f0 con 0x7f3eb80ffd20 2024-09-10T11:47:10.713 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:47:10.713 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:47:10.713 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:47:10.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:47:10.713 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:47:10.713 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:47:10.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:47:10.713 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:47:10.713 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:47:10.713 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:47:10.714 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:47:10.714 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:47:10.714 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:47:10.714 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:47:10.714 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:47:10.714 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:47:10.714 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:47:10.714 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:47:10.714 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:47:10.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.714+0000 7f3ebfd7c700 1 -- 172.21.15.144:0/42646195 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3ea006e250 msgr2=0x7f3ea0070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:10.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.714+0000 7f3ebfd7c700 1 --2- 172.21.15.144:0/42646195 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3ea006e250 0x7f3ea0070700 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8009d10 tx=0x7f3ea8009040 comp rx=0 tx=0).stop 2024-09-10T11:47:10.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.714+0000 7f3ebfd7c700 1 -- 172.21.15.144:0/42646195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb80ffd20 msgr2=0x7f3eb81012d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:10.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.714+0000 7f3ebfd7c700 1 --2- 172.21.15.144:0/42646195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb80ffd20 0x7f3eb81012d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0092e0 tx=0x7f3eac00be20 comp rx=0 tx=0).stop 2024-09-10T11:47:10.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.714+0000 7f3ebfd7c700 1 -- 172.21.15.144:0/42646195 shutdown_connections 2024-09-10T11:47:10.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.714+0000 7f3ebfd7c700 1 --2- 172.21.15.144:0/42646195 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3ea006e250 0x7f3ea0070700 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.714+0000 7f3ebfd7c700 1 --2- 172.21.15.144:0/42646195 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3eb80ff380 0x7f3eb8100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.714+0000 7f3ebfd7c700 1 --2- 172.21.15.144:0/42646195 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3eb80ffd20 0x7f3eb81012d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.714+0000 7f3ebfd7c700 1 -- 172.21.15.144:0/42646195 >> 172.21.15.144:0/42646195 conn(0x7f3eb80fac20 msgr2=0x7f3eb8068ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:10.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.714+0000 7f3ebfd7c700 1 -- 172.21.15.144:0/42646195 shutdown_connections 2024-09-10T11:47:10.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.714+0000 7f3ebfd7c700 1 -- 172.21.15.144:0/42646195 wait complete. 2024-09-10T11:47:10.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.819+0000 7f238298d700 1 -- 172.21.15.144:0/2978648011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f237c0ffcb0 msgr2=0x7f237c103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:10.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.819+0000 7f238298d700 1 --2- 172.21.15.144:0/2978648011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f237c0ffcb0 0x7f237c103c80 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2364009910 tx=0x7f2364009c20 comp rx=0 tx=0).stop 2024-09-10T11:47:10.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.819+0000 7f238298d700 1 -- 172.21.15.144:0/2978648011 shutdown_connections 2024-09-10T11:47:10.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.819+0000 7f238298d700 1 --2- 172.21.15.144:0/2978648011 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f237c0ffcb0 0x7f237c103c80 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.819+0000 7f238298d700 1 --2- 172.21.15.144:0/2978648011 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f237c0ff310 0x7f237c0ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.819+0000 7f238298d700 1 -- 172.21.15.144:0/2978648011 >> 172.21.15.144:0/2978648011 conn(0x7f237c0fabd0 msgr2=0x7f237c0fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:10.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.819+0000 7f238298d700 1 -- 172.21.15.144:0/2978648011 shutdown_connections 2024-09-10T11:47:10.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.819+0000 7f238298d700 1 -- 172.21.15.144:0/2978648011 wait complete. 2024-09-10T11:47:10.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.820+0000 7f238298d700 1 Processor -- start 2024-09-10T11:47:10.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.820+0000 7f238298d700 1 -- start start 2024-09-10T11:47:10.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.820+0000 7f238298d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f237c0ff310 0x7f237c194cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:10.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.820+0000 7f238298d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f237c0ffcb0 0x7f237c195200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:10.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.820+0000 7f238298d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c195890 con 0x7f237c0ff310 2024-09-10T11:47:10.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.820+0000 7f238298d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c1985c0 con 0x7f237c0ffcb0 2024-09-10T11:47:10.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.820+0000 7f237bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f237c0ff310 0x7f237c194cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:10.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.820+0000 7f237bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f237c0ff310 0x7f237c194cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34046/0 (socket says 172.21.15.144:34046) 2024-09-10T11:47:10.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.820+0000 7f237bfff700 1 -- 172.21.15.144:0/2174128122 learned_addr learned my addr 172.21.15.144:0/2174128122 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:47:10.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.821+0000 7f237b7fe700 1 --2- 172.21.15.144:0/2174128122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f237c0ffcb0 0x7f237c195200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:10.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.821+0000 7f237bfff700 1 -- 172.21.15.144:0/2174128122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f237c0ffcb0 msgr2=0x7f237c195200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:10.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.821+0000 7f237bfff700 1 --2- 172.21.15.144:0/2174128122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f237c0ffcb0 0x7f237c195200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:10.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.821+0000 7f237bfff700 1 -- 172.21.15.144:0/2174128122 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23640095f0 con 0x7f237c0ff310 2024-09-10T11:47:10.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.821+0000 7f237bfff700 1 --2- 172.21.15.144:0/2174128122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f237c0ff310 0x7f237c194cc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f236c009970 tx=0x7f236c009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:10.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.821+0000 7f23797fa700 1 -- 172.21.15.144:0/2174128122 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f236c012070 con 0x7f237c0ff310 2024-09-10T11:47:10.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.821+0000 7f23797fa700 1 -- 172.21.15.144:0/2174128122 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f236c007af0 con 0x7f237c0ff310 2024-09-10T11:47:10.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.821+0000 7f238298d700 1 -- 172.21.15.144:0/2174128122 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237c198840 con 0x7f237c0ff310 2024-09-10T11:47:10.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.821+0000 7f23797fa700 1 -- 172.21.15.144:0/2174128122 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f236c017c60 con 0x7f237c0ff310 2024-09-10T11:47:10.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.821+0000 7f238298d700 1 -- 172.21.15.144:0/2174128122 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237c198d90 con 0x7f237c0ff310 2024-09-10T11:47:10.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.822+0000 7f238298d700 1 -- 172.21.15.144:0/2174128122 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237c04ea50 con 0x7f237c0ff310 2024-09-10T11:47:10.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.823+0000 7f23797fa700 1 -- 172.21.15.144:0/2174128122 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f236c007c60 con 0x7f237c0ff310 2024-09-10T11:47:10.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.823+0000 7f23797fa700 1 --2- 172.21.15.144:0/2174128122 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f236806e200 0x7f23680706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:10.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.823+0000 7f23797fa700 1 -- 172.21.15.144:0/2174128122 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f236c08f5a0 con 0x7f237c0ff310 2024-09-10T11:47:10.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.823+0000 7f237b7fe700 1 --2- 172.21.15.144:0/2174128122 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f236806e200 0x7f23680706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:10.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.824+0000 7f237b7fe700 1 --2- 172.21.15.144:0/2174128122 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f236806e200 0x7f23680706b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2364006010 tx=0x7f236400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:10.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:10.828+0000 7f23797fa700 1 -- 172.21.15.144:0/2174128122 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f236c058ab0 con 0x7f237c0ff310 2024-09-10T11:47:11.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.073+0000 7f238298d700 1 -- 172.21.15.144:0/2174128122 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f237c1991a0 con 0x7f237c0ff310 2024-09-10T11:47:11.075 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:10 smithi144 ceph-mon[87436]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:11.075 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:10 smithi144 ceph-mon[87436]: pgmap v455: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:11.075 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:10 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/42646195' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:47:11.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.074+0000 7f23797fa700 1 -- 172.21.15.144:0/2174128122 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f236c05c170 con 0x7f237c0ff310 2024-09-10T11:47:11.076 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:47:11.076 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:47:11.076 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:47:11.076 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:47:11.076 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:47:11.076 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:47:11.076 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:47:11.076 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:47:11.076 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:47:11.076 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:47:11.076 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:47:11.076 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:47:11.076 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:47:11.077 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:47:11.077 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:47:11.077 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:47:11.077 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:47:11.077 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:47:11.077 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:47:11.077 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:47:11.077 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:47:11.077 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:47:11.077 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:47:11.077 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:47:11.077 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:47:11.078 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:47:11.078 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:47:11.078 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:47:11.078 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:47:11.078 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:47:11.078 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:47:11.078 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:47:11.078 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:47:11.078 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:47:11.078 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:47:11.078 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:47:11.078 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:47:11.078 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:47:11.078 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:47:11.079 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:47:11.079 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:47:11.079 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:47:11.079 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:47:11.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.076+0000 7f238298d700 1 -- 172.21.15.144:0/2174128122 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f236806e200 msgr2=0x7f23680706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:11.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.076+0000 7f238298d700 1 --2- 172.21.15.144:0/2174128122 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f236806e200 0x7f23680706b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2364006010 tx=0x7f236400b540 comp rx=0 tx=0).stop 2024-09-10T11:47:11.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.076+0000 7f238298d700 1 -- 172.21.15.144:0/2174128122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f237c0ff310 msgr2=0x7f237c194cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:11.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.076+0000 7f238298d700 1 --2- 172.21.15.144:0/2174128122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f237c0ff310 0x7f237c194cc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f236c009970 tx=0x7f236c009d30 comp rx=0 tx=0).stop 2024-09-10T11:47:11.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.076+0000 7f238298d700 1 -- 172.21.15.144:0/2174128122 shutdown_connections 2024-09-10T11:47:11.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.076+0000 7f238298d700 1 --2- 172.21.15.144:0/2174128122 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f236806e200 0x7f23680706b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.076+0000 7f238298d700 1 --2- 172.21.15.144:0/2174128122 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f237c0ff310 0x7f237c194cc0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.076+0000 7f238298d700 1 --2- 172.21.15.144:0/2174128122 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f237c0ffcb0 0x7f237c195200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.076+0000 7f238298d700 1 -- 172.21.15.144:0/2174128122 >> 172.21.15.144:0/2174128122 conn(0x7f237c0fabd0 msgr2=0x7f237c0fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:11.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.076+0000 7f238298d700 1 -- 172.21.15.144:0/2174128122 shutdown_connections 2024-09-10T11:47:11.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.077+0000 7f238298d700 1 -- 172.21.15.144:0/2174128122 wait complete. 2024-09-10T11:47:11.080 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:47:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:10 smithi178 ceph-mon[65940]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:10 smithi178 ceph-mon[65940]: pgmap v455: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:10 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/42646195' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:47:11.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.182+0000 7fdd24aa0700 1 -- 172.21.15.144:0/814695095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd200ffc60 msgr2=0x7fdd20103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:11.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.182+0000 7fdd24aa0700 1 --2- 172.21.15.144:0/814695095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd200ffc60 0x7fdd20103c30 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fdd10009910 tx=0x7fdd10009c20 comp rx=0 tx=0).stop 2024-09-10T11:47:11.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.182+0000 7fdd24aa0700 1 -- 172.21.15.144:0/814695095 shutdown_connections 2024-09-10T11:47:11.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.182+0000 7fdd24aa0700 1 --2- 172.21.15.144:0/814695095 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd200ffc60 0x7fdd20103c30 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.182+0000 7fdd24aa0700 1 --2- 172.21.15.144:0/814695095 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd200ff2c0 0x7fdd200ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.182+0000 7fdd24aa0700 1 -- 172.21.15.144:0/814695095 >> 172.21.15.144:0/814695095 conn(0x7fdd200fab80 msgr2=0x7fdd200fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:11.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.182+0000 7fdd24aa0700 1 -- 172.21.15.144:0/814695095 shutdown_connections 2024-09-10T11:47:11.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.182+0000 7fdd24aa0700 1 -- 172.21.15.144:0/814695095 wait complete. 2024-09-10T11:47:11.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.183+0000 7fdd24aa0700 1 Processor -- start 2024-09-10T11:47:11.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.183+0000 7fdd24aa0700 1 -- start start 2024-09-10T11:47:11.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.183+0000 7fdd24aa0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd200ff2c0 0x7fdd20195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:11.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.183+0000 7fdd24aa0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd200ffc60 0x7fdd20196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:11.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.183+0000 7fdd24aa0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd20196a70 con 0x7fdd200ff2c0 2024-09-10T11:47:11.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.183+0000 7fdd24aa0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd2019a790 con 0x7fdd200ffc60 2024-09-10T11:47:11.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.184+0000 7fdd1e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd200ff2c0 0x7fdd20195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:11.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.184+0000 7fdd1e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd200ff2c0 0x7fdd20195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34072/0 (socket says 172.21.15.144:34072) 2024-09-10T11:47:11.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.184+0000 7fdd1e59c700 1 -- 172.21.15.144:0/364457393 learned_addr learned my addr 172.21.15.144:0/364457393 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:47:11.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.184+0000 7fdd1dd9b700 1 --2- 172.21.15.144:0/364457393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd200ffc60 0x7fdd20196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:11.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.184+0000 7fdd1e59c700 1 -- 172.21.15.144:0/364457393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd200ffc60 msgr2=0x7fdd20196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:11.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.184+0000 7fdd1e59c700 1 --2- 172.21.15.144:0/364457393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd200ffc60 0x7fdd20196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.184+0000 7fdd1e59c700 1 -- 172.21.15.144:0/364457393 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd100095f0 con 0x7fdd200ff2c0 2024-09-10T11:47:11.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.184+0000 7fdd1e59c700 1 --2- 172.21.15.144:0/364457393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd200ff2c0 0x7fdd20195e50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400b4b0 tx=0x7fdd1400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:11.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.184+0000 7fdd0f7fe700 1 -- 172.21.15.144:0/364457393 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd14011800 con 0x7fdd200ff2c0 2024-09-10T11:47:11.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.184+0000 7fdd0f7fe700 1 -- 172.21.15.144:0/364457393 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd14011e20 con 0x7fdd200ff2c0 2024-09-10T11:47:11.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.184+0000 7fdd0f7fe700 1 -- 172.21.15.144:0/364457393 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd1400f2e0 con 0x7fdd200ff2c0 2024-09-10T11:47:11.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.184+0000 7fdd24aa0700 1 -- 172.21.15.144:0/364457393 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd2019aa70 con 0x7fdd200ff2c0 2024-09-10T11:47:11.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.184+0000 7fdd24aa0700 1 -- 172.21.15.144:0/364457393 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd2019af90 con 0x7fdd200ff2c0 2024-09-10T11:47:11.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.185+0000 7fdd24aa0700 1 -- 172.21.15.144:0/364457393 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd2004ea50 con 0x7fdd200ff2c0 2024-09-10T11:47:11.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.186+0000 7fdd0f7fe700 1 -- 172.21.15.144:0/364457393 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fdd14010380 con 0x7fdd200ff2c0 2024-09-10T11:47:11.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.186+0000 7fdd0f7fe700 1 --2- 172.21.15.144:0/364457393 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdd0806e200 0x7fdd080706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:11.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.187+0000 7fdd0f7fe700 1 -- 172.21.15.144:0/364457393 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdd1408dab0 con 0x7fdd200ff2c0 2024-09-10T11:47:11.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.187+0000 7fdd1dd9b700 1 --2- 172.21.15.144:0/364457393 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdd0806e200 0x7fdd080706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:11.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.187+0000 7fdd1dd9b700 1 --2- 172.21.15.144:0/364457393 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdd0806e200 0x7fdd080706b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdd10006010 tx=0x7fdd1001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:11.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.191+0000 7fdd0f7fe700 1 -- 172.21.15.144:0/364457393 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdd14056f10 con 0x7fdd200ff2c0 2024-09-10T11:47:11.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.412+0000 7fdd24aa0700 1 -- 172.21.15.144:0/364457393 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd2019b370 con 0x7fdd0806e200 2024-09-10T11:47:11.414 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.413+0000 7fdd0f7fe700 1 -- 172.21.15.144:0/364457393 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fdd2019b370 con 0x7fdd0806e200 2024-09-10T11:47:11.414 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:47:11.414 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:47:11.414 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:47:11.414 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:47:11.414 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:47:11.415 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:47:11.415 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:47:11.415 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:47:11.415 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:47:11.415 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:47:11.415 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:47:11.415 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:47:11.415 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:47:11.415 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:47:11.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.415+0000 7fdd24aa0700 1 -- 172.21.15.144:0/364457393 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdd0806e200 msgr2=0x7fdd080706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:11.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.415+0000 7fdd24aa0700 1 --2- 172.21.15.144:0/364457393 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdd0806e200 0x7fdd080706b0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fdd10006010 tx=0x7fdd1001a040 comp rx=0 tx=0).stop 2024-09-10T11:47:11.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.415+0000 7fdd24aa0700 1 -- 172.21.15.144:0/364457393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd200ff2c0 msgr2=0x7fdd20195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:11.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.415+0000 7fdd24aa0700 1 --2- 172.21.15.144:0/364457393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd200ff2c0 0x7fdd20195e50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400b4b0 tx=0x7fdd1400b7c0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.415+0000 7fdd24aa0700 1 -- 172.21.15.144:0/364457393 shutdown_connections 2024-09-10T11:47:11.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.415+0000 7fdd24aa0700 1 --2- 172.21.15.144:0/364457393 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdd0806e200 0x7fdd080706b0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.415+0000 7fdd24aa0700 1 --2- 172.21.15.144:0/364457393 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd200ff2c0 0x7fdd20195e50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.415+0000 7fdd24aa0700 1 --2- 172.21.15.144:0/364457393 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd200ffc60 0x7fdd20196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.415+0000 7fdd24aa0700 1 -- 172.21.15.144:0/364457393 >> 172.21.15.144:0/364457393 conn(0x7fdd200fab80 msgr2=0x7fdd201033e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:11.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.415+0000 7fdd24aa0700 1 -- 172.21.15.144:0/364457393 shutdown_connections 2024-09-10T11:47:11.417 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.415+0000 7fdd24aa0700 1 -- 172.21.15.144:0/364457393 wait complete. 2024-09-10T11:47:11.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.520+0000 7fe3f2dc7700 1 -- 172.21.15.144:0/2459360588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ec1003d0 msgr2=0x7fe3ec1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:11.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.520+0000 7fe3f2dc7700 1 --2- 172.21.15.144:0/2459360588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ec1003d0 0x7fe3ec1007a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0009910 tx=0x7fe3e0009c20 comp rx=0 tx=0).stop 2024-09-10T11:47:11.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.520+0000 7fe3f2dc7700 1 -- 172.21.15.144:0/2459360588 shutdown_connections 2024-09-10T11:47:11.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.520+0000 7fe3f2dc7700 1 --2- 172.21.15.144:0/2459360588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3ec100d70 0x7fe3ec104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.520+0000 7fe3f2dc7700 1 --2- 172.21.15.144:0/2459360588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ec1003d0 0x7fe3ec1007a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.520+0000 7fe3f2dc7700 1 -- 172.21.15.144:0/2459360588 >> 172.21.15.144:0/2459360588 conn(0x7fe3ec0fbc90 msgr2=0x7fe3ec0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:11.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.520+0000 7fe3f2dc7700 1 -- 172.21.15.144:0/2459360588 shutdown_connections 2024-09-10T11:47:11.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.521+0000 7fe3f2dc7700 1 -- 172.21.15.144:0/2459360588 wait complete. 2024-09-10T11:47:11.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.521+0000 7fe3f2dc7700 1 Processor -- start 2024-09-10T11:47:11.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.521+0000 7fe3f2dc7700 1 -- start start 2024-09-10T11:47:11.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.521+0000 7fe3f2dc7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3ec1003d0 0x7fe3ec195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:11.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.521+0000 7fe3f2dc7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ec100d70 0x7fe3ec1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:11.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.521+0000 7fe3f2dc7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec196980 con 0x7fe3ec100d70 2024-09-10T11:47:11.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.521+0000 7fe3f2dc7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3ec19a710 con 0x7fe3ec1003d0 2024-09-10T11:47:11.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.521+0000 7fe3ebfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ec100d70 0x7fe3ec1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:11.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.521+0000 7fe3ebfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ec100d70 0x7fe3ec1962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34088/0 (socket says 172.21.15.144:34088) 2024-09-10T11:47:11.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.521+0000 7fe3ebfff700 1 -- 172.21.15.144:0/3006093166 learned_addr learned my addr 172.21.15.144:0/3006093166 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:47:11.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.522+0000 7fe3f0b63700 1 --2- 172.21.15.144:0/3006093166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3ec1003d0 0x7fe3ec195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:11.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.522+0000 7fe3ebfff700 1 -- 172.21.15.144:0/3006093166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3ec1003d0 msgr2=0x7fe3ec195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:11.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.522+0000 7fe3ebfff700 1 --2- 172.21.15.144:0/3006093166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3ec1003d0 0x7fe3ec195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.522+0000 7fe3ebfff700 1 -- 172.21.15.144:0/3006093166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e00095f0 con 0x7fe3ec100d70 2024-09-10T11:47:11.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.522+0000 7fe3ebfff700 1 --2- 172.21.15.144:0/3006093166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ec100d70 0x7fe3ec1962a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800d900 tx=0x7fe3d800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:11.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.522+0000 7fe3e9ffb700 1 -- 172.21.15.144:0/3006093166 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3d8009900 con 0x7fe3ec100d70 2024-09-10T11:47:11.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.522+0000 7fe3e9ffb700 1 -- 172.21.15.144:0/3006093166 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe3d8010460 con 0x7fe3ec100d70 2024-09-10T11:47:11.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.522+0000 7fe3f2dc7700 1 -- 172.21.15.144:0/3006093166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3ec19a9f0 con 0x7fe3ec100d70 2024-09-10T11:47:11.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.522+0000 7fe3e9ffb700 1 -- 172.21.15.144:0/3006093166 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3d800f5f0 con 0x7fe3ec100d70 2024-09-10T11:47:11.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.522+0000 7fe3f2dc7700 1 -- 172.21.15.144:0/3006093166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3ec19af40 con 0x7fe3ec100d70 2024-09-10T11:47:11.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.523+0000 7fe3f2dc7700 1 -- 172.21.15.144:0/3006093166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3ec1086e0 con 0x7fe3ec100d70 2024-09-10T11:47:11.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.524+0000 7fe3e9ffb700 1 -- 172.21.15.144:0/3006093166 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe3d80105d0 con 0x7fe3ec100d70 2024-09-10T11:47:11.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.524+0000 7fe3e9ffb700 1 --2- 172.21.15.144:0/3006093166 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe3dc06e200 0x7fe3dc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:11.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.524+0000 7fe3e9ffb700 1 -- 172.21.15.144:0/3006093166 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe3d808de60 con 0x7fe3ec100d70 2024-09-10T11:47:11.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.524+0000 7fe3f0b63700 1 --2- 172.21.15.144:0/3006093166 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe3dc06e200 0x7fe3dc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:11.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.525+0000 7fe3f0b63700 1 --2- 172.21.15.144:0/3006093166 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe3dc06e200 0x7fe3dc0706b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000b5c0 tx=0x7fe3e0005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:11.529 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.529+0000 7fe3e9ffb700 1 -- 172.21.15.144:0/3006093166 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe3d80572c0 con 0x7fe3ec100d70 2024-09-10T11:47:11.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.813+0000 7fe3f2dc7700 1 -- 172.21.15.144:0/3006093166 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe3ec04ea50 con 0x7fe3ec100d70 2024-09-10T11:47:11.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.813+0000 7fe3e9ffb700 1 -- 172.21.15.144:0/3006093166 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fe3d80201c0 con 0x7fe3ec100d70 2024-09-10T11:47:11.814 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:47:11.814 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:47:11.815 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:47:11.815 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:47:11.815 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:47:11.815 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:47:11.815 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:47:11.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.815+0000 7fe3f2dc7700 1 -- 172.21.15.144:0/3006093166 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe3dc06e200 msgr2=0x7fe3dc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:11.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.815+0000 7fe3f2dc7700 1 --2- 172.21.15.144:0/3006093166 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe3dc06e200 0x7fe3dc0706b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe3e000b5c0 tx=0x7fe3e0005960 comp rx=0 tx=0).stop 2024-09-10T11:47:11.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.815+0000 7fe3f2dc7700 1 -- 172.21.15.144:0/3006093166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ec100d70 msgr2=0x7fe3ec1962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:11.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.815+0000 7fe3f2dc7700 1 --2- 172.21.15.144:0/3006093166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ec100d70 0x7fe3ec1962a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800d900 tx=0x7fe3d800dcc0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.815+0000 7fe3f2dc7700 1 -- 172.21.15.144:0/3006093166 shutdown_connections 2024-09-10T11:47:11.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.815+0000 7fe3f2dc7700 1 --2- 172.21.15.144:0/3006093166 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe3dc06e200 0x7fe3dc0706b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.815+0000 7fe3f2dc7700 1 --2- 172.21.15.144:0/3006093166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe3ec1003d0 0x7fe3ec195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.815+0000 7fe3f2dc7700 1 --2- 172.21.15.144:0/3006093166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe3ec100d70 0x7fe3ec1962a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:11.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.815+0000 7fe3f2dc7700 1 -- 172.21.15.144:0/3006093166 >> 172.21.15.144:0/3006093166 conn(0x7fe3ec0fbc90 msgr2=0x7fe3ec0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:11.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.816+0000 7fe3f2dc7700 1 -- 172.21.15.144:0/3006093166 shutdown_connections 2024-09-10T11:47:11.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:11.816+0000 7fe3f2dc7700 1 -- 172.21.15.144:0/3006093166 wait complete. 2024-09-10T11:47:12.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:11 smithi178 ceph-mon[65940]: from='client.34880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:12.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:11 smithi178 ceph-mon[65940]: from='client.34884 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:12.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:11 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2174128122' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:47:12.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:11 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3006093166' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:47:12.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:11 smithi144 ceph-mon[87436]: from='client.34880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:12.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:11 smithi144 ceph-mon[87436]: from='client.34884 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:12.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:11 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2174128122' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:47:12.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:11 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3006093166' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:47:13.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:12 smithi178 ceph-mon[65940]: from='client.34896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:13.090 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:12 smithi178 ceph-mon[65940]: pgmap v456: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:12 smithi144 ceph-mon[87436]: from='client.34896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:12 smithi144 ceph-mon[87436]: pgmap v456: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:47:14.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:47:14.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:14 smithi144 ceph-mon[87436]: pgmap v457: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:15.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:14 smithi178 ceph-mon[65940]: pgmap v457: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:16.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:15 smithi178 ceph-mon[65940]: pgmap v458: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:16.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:15 smithi144 ceph-mon[87436]: pgmap v458: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:18 smithi178 ceph-mon[65940]: pgmap v459: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:19.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:18 smithi144 ceph-mon[87436]: pgmap v459: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:20 smithi178 ceph-mon[65940]: pgmap v460: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:21.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:20 smithi144 ceph-mon[87436]: pgmap v460: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:23.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:22 smithi144 ceph-mon[87436]: pgmap v461: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:23.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:22 smithi178 ceph-mon[65940]: pgmap v461: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:24 smithi178 ceph-mon[65940]: pgmap v462: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:24 smithi144 ceph-mon[87436]: pgmap v462: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:26 smithi178 ceph-mon[65940]: pgmap v463: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:27.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:26 smithi144 ceph-mon[87436]: pgmap v463: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:28 smithi178 ceph-mon[65940]: pgmap v464: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:47:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:28 smithi144 ceph-mon[87436]: pgmap v464: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:47:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:30 smithi178 ceph-mon[65940]: pgmap v465: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:31.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:30 smithi144 ceph-mon[87436]: pgmap v465: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:32 smithi144 ceph-mon[87436]: pgmap v466: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:33.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:32 smithi178 ceph-mon[65940]: pgmap v466: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:34 smithi178 ceph-mon[65940]: pgmap v467: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:34 smithi144 ceph-mon[87436]: pgmap v467: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:36 smithi178 ceph-mon[65940]: pgmap v468: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:36 smithi144 ceph-mon[87436]: pgmap v468: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:38 smithi178 ceph-mon[65940]: pgmap v469: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:38.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:38 smithi144 ceph-mon[87436]: pgmap v469: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:40 smithi178 ceph-mon[65940]: pgmap v470: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:41.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:40 smithi144 ceph-mon[87436]: pgmap v470: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.933+0000 7fb1ebda9700 1 -- 172.21.15.144:0/2989624152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e41003d0 msgr2=0x7fb1e41007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.933+0000 7fb1ebda9700 1 --2- 172.21.15.144:0/2989624152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e41003d0 0x7fb1e41007a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8009910 tx=0x7fb1d8009c20 comp rx=0 tx=0).stop 2024-09-10T11:47:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.934+0000 7fb1ebda9700 1 -- 172.21.15.144:0/2989624152 shutdown_connections 2024-09-10T11:47:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.934+0000 7fb1ebda9700 1 --2- 172.21.15.144:0/2989624152 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1e4100d70 0x7fb1e4104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.934+0000 7fb1ebda9700 1 --2- 172.21.15.144:0/2989624152 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e41003d0 0x7fb1e41007a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.934+0000 7fb1ebda9700 1 -- 172.21.15.144:0/2989624152 >> 172.21.15.144:0/2989624152 conn(0x7fb1e40fbc90 msgr2=0x7fb1e40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.934+0000 7fb1ebda9700 1 -- 172.21.15.144:0/2989624152 shutdown_connections 2024-09-10T11:47:41.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.934+0000 7fb1ebda9700 1 -- 172.21.15.144:0/2989624152 wait complete. 2024-09-10T11:47:41.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.934+0000 7fb1ebda9700 1 Processor -- start 2024-09-10T11:47:41.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.934+0000 7fb1ebda9700 1 -- start start 2024-09-10T11:47:41.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.934+0000 7fb1ebda9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1e41003d0 0x7fb1e4195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:41.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.934+0000 7fb1ebda9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e4100d70 0x7fb1e41962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.934+0000 7fb1ebda9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e41968f0 con 0x7fb1e4100d70 2024-09-10T11:47:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.935+0000 7fb1ebda9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e4196a30 con 0x7fb1e41003d0 2024-09-10T11:47:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.935+0000 7fb1e9344700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e4100d70 0x7fb1e41962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.935+0000 7fb1e9344700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e4100d70 0x7fb1e41962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56460/0 (socket says 172.21.15.144:56460) 2024-09-10T11:47:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.935+0000 7fb1e9344700 1 -- 172.21.15.144:0/3279392698 learned_addr learned my addr 172.21.15.144:0/3279392698 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:47:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.935+0000 7fb1e9b45700 1 --2- 172.21.15.144:0/3279392698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1e41003d0 0x7fb1e4195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.935+0000 7fb1e9344700 1 -- 172.21.15.144:0/3279392698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1e41003d0 msgr2=0x7fb1e4195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.935+0000 7fb1e9344700 1 --2- 172.21.15.144:0/3279392698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1e41003d0 0x7fb1e4195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.935+0000 7fb1e9344700 1 -- 172.21.15.144:0/3279392698 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d80095f0 con 0x7fb1e4100d70 2024-09-10T11:47:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.935+0000 7fb1e9344700 1 --2- 172.21.15.144:0/3279392698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e4100d70 0x7fb1e41962a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000c960 tx=0x7fb1e000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.935+0000 7fb1d6ffd700 1 -- 172.21.15.144:0/3279392698 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1e0007a70 con 0x7fb1e4100d70 2024-09-10T11:47:41.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.935+0000 7fb1d6ffd700 1 -- 172.21.15.144:0/3279392698 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1e0007bd0 con 0x7fb1e4100d70 2024-09-10T11:47:41.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.935+0000 7fb1ebda9700 1 -- 172.21.15.144:0/3279392698 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e419a880 con 0x7fb1e4100d70 2024-09-10T11:47:41.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.935+0000 7fb1d6ffd700 1 -- 172.21.15.144:0/3279392698 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1e0018630 con 0x7fb1e4100d70 2024-09-10T11:47:41.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.935+0000 7fb1ebda9700 1 -- 172.21.15.144:0/3279392698 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e419add0 con 0x7fb1e4100d70 2024-09-10T11:47:41.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.936+0000 7fb1ebda9700 1 -- 172.21.15.144:0/3279392698 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e41086e0 con 0x7fb1e4100d70 2024-09-10T11:47:41.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.937+0000 7fb1d6ffd700 1 -- 172.21.15.144:0/3279392698 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb1e000f450 con 0x7fb1e4100d70 2024-09-10T11:47:41.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.938+0000 7fb1d6ffd700 1 --2- 172.21.15.144:0/3279392698 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb1d006e200 0x7fb1d00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:41.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.938+0000 7fb1d6ffd700 1 -- 172.21.15.144:0/3279392698 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb1e008df60 con 0x7fb1e4100d70 2024-09-10T11:47:41.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.938+0000 7fb1e9b45700 1 --2- 172.21.15.144:0/3279392698 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb1d006e200 0x7fb1d00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:41.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.938+0000 7fb1e9b45700 1 --2- 172.21.15.144:0/3279392698 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb1d006e200 0x7fb1d00706b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8006010 tx=0x7fb1d801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:41.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:41.942+0000 7fb1d6ffd700 1 -- 172.21.15.144:0/3279392698 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb1e00c31b0 con 0x7fb1e4100d70 2024-09-10T11:47:42.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.171+0000 7fb1ebda9700 1 -- 172.21.15.144:0/3279392698 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1e402ccf0 con 0x7fb1d006e200 2024-09-10T11:47:42.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.173+0000 7fb1d6ffd700 1 -- 172.21.15.144:0/3279392698 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fb1e402ccf0 con 0x7fb1d006e200 2024-09-10T11:47:42.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.175+0000 7fb1ebda9700 1 -- 172.21.15.144:0/3279392698 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb1d006e200 msgr2=0x7fb1d00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:42.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.175+0000 7fb1ebda9700 1 --2- 172.21.15.144:0/3279392698 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb1d006e200 0x7fb1d00706b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb1d8006010 tx=0x7fb1d801a040 comp rx=0 tx=0).stop 2024-09-10T11:47:42.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.175+0000 7fb1ebda9700 1 -- 172.21.15.144:0/3279392698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e4100d70 msgr2=0x7fb1e41962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:42.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.175+0000 7fb1ebda9700 1 --2- 172.21.15.144:0/3279392698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e4100d70 0x7fb1e41962a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fb1e000c960 tx=0x7fb1e000cc70 comp rx=0 tx=0).stop 2024-09-10T11:47:42.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.175+0000 7fb1ebda9700 1 -- 172.21.15.144:0/3279392698 shutdown_connections 2024-09-10T11:47:42.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.175+0000 7fb1ebda9700 1 --2- 172.21.15.144:0/3279392698 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb1d006e200 0x7fb1d00706b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.175+0000 7fb1ebda9700 1 --2- 172.21.15.144:0/3279392698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1e41003d0 0x7fb1e4195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.175+0000 7fb1ebda9700 1 --2- 172.21.15.144:0/3279392698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1e4100d70 0x7fb1e41962a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.175+0000 7fb1ebda9700 1 -- 172.21.15.144:0/3279392698 >> 172.21.15.144:0/3279392698 conn(0x7fb1e40fbc90 msgr2=0x7fb1e40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:42.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.175+0000 7fb1ebda9700 1 -- 172.21.15.144:0/3279392698 shutdown_connections 2024-09-10T11:47:42.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.175+0000 7fb1ebda9700 1 -- 172.21.15.144:0/3279392698 wait complete. 2024-09-10T11:47:42.187 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:47:42.284 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.280+0000 7f498ad41700 1 -- 172.21.15.144:0/1138453072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4984100410 msgr2=0x7f49841007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:42.284 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.280+0000 7f498ad41700 1 --2- 172.21.15.144:0/1138453072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4984100410 0x7f49841007e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f4978009910 tx=0x7f4978009c20 comp rx=0 tx=0).stop 2024-09-10T11:47:42.284 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.281+0000 7f498ad41700 1 -- 172.21.15.144:0/1138453072 shutdown_connections 2024-09-10T11:47:42.284 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.281+0000 7f498ad41700 1 --2- 172.21.15.144:0/1138453072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4984100db0 0x7f4984104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.284 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.281+0000 7f498ad41700 1 --2- 172.21.15.144:0/1138453072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4984100410 0x7f49841007e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.284 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.281+0000 7f498ad41700 1 -- 172.21.15.144:0/1138453072 >> 172.21.15.144:0/1138453072 conn(0x7f49840fbcf0 msgr2=0x7f49840fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:42.284 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.281+0000 7f498ad41700 1 -- 172.21.15.144:0/1138453072 shutdown_connections 2024-09-10T11:47:42.284 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.281+0000 7f498ad41700 1 -- 172.21.15.144:0/1138453072 wait complete. 2024-09-10T11:47:42.284 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.281+0000 7f498ad41700 1 Processor -- start 2024-09-10T11:47:42.285 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.281+0000 7f498ad41700 1 -- start start 2024-09-10T11:47:42.285 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f498ad41700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4984100410 0x7f4984195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:42.285 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f498ad41700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4984100db0 0x7f4984196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:42.285 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f498ad41700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4984196a00 con 0x7f4984100db0 2024-09-10T11:47:42.285 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f498ad41700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498419a790 con 0x7f4984100410 2024-09-10T11:47:42.285 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f4983fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4984100db0 0x7f4984196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:42.285 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f4983fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4984100db0 0x7f4984196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56488/0 (socket says 172.21.15.144:56488) 2024-09-10T11:47:42.285 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f4983fff700 1 -- 172.21.15.144:0/1586675662 learned_addr learned my addr 172.21.15.144:0/1586675662 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:47:42.285 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f4988add700 1 --2- 172.21.15.144:0/1586675662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4984100410 0x7f4984195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:42.286 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f4983fff700 1 -- 172.21.15.144:0/1586675662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4984100410 msgr2=0x7f4984195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:42.286 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f4983fff700 1 --2- 172.21.15.144:0/1586675662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4984100410 0x7f4984195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.286 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f4983fff700 1 -- 172.21.15.144:0/1586675662 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49780095f0 con 0x7f4984100db0 2024-09-10T11:47:42.286 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f4988add700 1 --2- 172.21.15.144:0/1586675662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4984100410 0x7f4984195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:47:42.286 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f4983fff700 1 --2- 172.21.15.144:0/1586675662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4984100db0 0x7f4984196320 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f497000c8f0 tx=0x7f497000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:42.286 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f4981ffb700 1 -- 172.21.15.144:0/1586675662 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4970007a50 con 0x7f4984100db0 2024-09-10T11:47:42.286 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f4981ffb700 1 -- 172.21.15.144:0/1586675662 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4970007bb0 con 0x7f4984100db0 2024-09-10T11:47:42.286 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f498ad41700 1 -- 172.21.15.144:0/1586675662 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498419aa70 con 0x7f4984100db0 2024-09-10T11:47:42.286 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.282+0000 7f4981ffb700 1 -- 172.21.15.144:0/1586675662 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4970018630 con 0x7f4984100db0 2024-09-10T11:47:42.286 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.283+0000 7f498ad41700 1 -- 172.21.15.144:0/1586675662 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498419afc0 con 0x7f4984100db0 2024-09-10T11:47:42.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.284+0000 7f498ad41700 1 -- 172.21.15.144:0/1586675662 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4984108720 con 0x7f4984100db0 2024-09-10T11:47:42.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.284+0000 7f4981ffb700 1 -- 172.21.15.144:0/1586675662 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4970007d20 con 0x7f4984100db0 2024-09-10T11:47:42.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.284+0000 7f4981ffb700 1 --2- 172.21.15.144:0/1586675662 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f497406e200 0x7f49740706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:42.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.285+0000 7f4981ffb700 1 -- 172.21.15.144:0/1586675662 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f497008e060 con 0x7f4984100db0 2024-09-10T11:47:42.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.285+0000 7f4988add700 1 --2- 172.21.15.144:0/1586675662 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f497406e200 0x7f49740706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:42.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.285+0000 7f4988add700 1 --2- 172.21.15.144:0/1586675662 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f497406e200 0x7f49740706b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f49780059d0 tx=0x7f497801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:42.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.289+0000 7f4981ffb700 1 -- 172.21.15.144:0/1586675662 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4970057440 con 0x7f4984100db0 2024-09-10T11:47:42.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.510+0000 7f498ad41700 1 -- 172.21.15.144:0/1586675662 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f498419b2b0 con 0x7f497406e200 2024-09-10T11:47:42.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.511+0000 7f4981ffb700 1 -- 172.21.15.144:0/1586675662 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f498419b2b0 con 0x7f497406e200 2024-09-10T11:47:42.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.514+0000 7f498ad41700 1 -- 172.21.15.144:0/1586675662 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f497406e200 msgr2=0x7f49740706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:42.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.514+0000 7f498ad41700 1 --2- 172.21.15.144:0/1586675662 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f497406e200 0x7f49740706b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f49780059d0 tx=0x7f497801a040 comp rx=0 tx=0).stop 2024-09-10T11:47:42.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.514+0000 7f498ad41700 1 -- 172.21.15.144:0/1586675662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4984100db0 msgr2=0x7f4984196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:42.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.514+0000 7f498ad41700 1 --2- 172.21.15.144:0/1586675662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4984100db0 0x7f4984196320 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f497000c8f0 tx=0x7f497000ccb0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.514+0000 7f498ad41700 1 -- 172.21.15.144:0/1586675662 shutdown_connections 2024-09-10T11:47:42.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.514+0000 7f498ad41700 1 --2- 172.21.15.144:0/1586675662 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f497406e200 0x7f49740706b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.514+0000 7f498ad41700 1 --2- 172.21.15.144:0/1586675662 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4984100410 0x7f4984195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.514+0000 7f498ad41700 1 --2- 172.21.15.144:0/1586675662 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4984100db0 0x7f4984196320 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.514+0000 7f498ad41700 1 -- 172.21.15.144:0/1586675662 >> 172.21.15.144:0/1586675662 conn(0x7f49840fbcf0 msgr2=0x7f49840fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:42.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.514+0000 7f498ad41700 1 -- 172.21.15.144:0/1586675662 shutdown_connections 2024-09-10T11:47:42.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.514+0000 7f498ad41700 1 -- 172.21.15.144:0/1586675662 wait complete. 2024-09-10T11:47:42.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.618+0000 7ffaec6fa700 1 -- 172.21.15.144:0/2910805565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffae40ff300 msgr2=0x7ffae40ff6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:42.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.618+0000 7ffaec6fa700 1 --2- 172.21.15.144:0/2910805565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffae40ff300 0x7ffae40ff6d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7ffad8009910 tx=0x7ffad8009c20 comp rx=0 tx=0).stop 2024-09-10T11:47:42.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.618+0000 7ffaec6fa700 1 -- 172.21.15.144:0/2910805565 shutdown_connections 2024-09-10T11:47:42.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.618+0000 7ffaec6fa700 1 --2- 172.21.15.144:0/2910805565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffae40ffca0 0x7ffae4103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.618+0000 7ffaec6fa700 1 --2- 172.21.15.144:0/2910805565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffae40ff300 0x7ffae40ff6d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.618+0000 7ffaec6fa700 1 -- 172.21.15.144:0/2910805565 >> 172.21.15.144:0/2910805565 conn(0x7ffae40faba0 msgr2=0x7ffae40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:42.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.618+0000 7ffaec6fa700 1 -- 172.21.15.144:0/2910805565 shutdown_connections 2024-09-10T11:47:42.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.619+0000 7ffaec6fa700 1 -- 172.21.15.144:0/2910805565 wait complete. 2024-09-10T11:47:42.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.619+0000 7ffaec6fa700 1 Processor -- start 2024-09-10T11:47:42.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.619+0000 7ffaec6fa700 1 -- start start 2024-09-10T11:47:42.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.619+0000 7ffaec6fa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffae40ff300 0x7ffae4196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:42.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.619+0000 7ffaec6fa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffae40ffca0 0x7ffae41971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:42.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.619+0000 7ffaec6fa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae41977f0 con 0x7ffae40ffca0 2024-09-10T11:47:42.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.619+0000 7ffaec6fa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae4197930 con 0x7ffae40ff300 2024-09-10T11:47:42.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.620+0000 7ffae9c95700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffae40ffca0 0x7ffae41971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:42.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.620+0000 7ffae9c95700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffae40ffca0 0x7ffae41971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33598/0 (socket says 172.21.15.144:33598) 2024-09-10T11:47:42.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.620+0000 7ffae9c95700 1 -- 172.21.15.144:0/1024912748 learned_addr learned my addr 172.21.15.144:0/1024912748 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:47:42.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.620+0000 7ffaea496700 1 --2- 172.21.15.144:0/1024912748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffae40ff300 0x7ffae4196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:42.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.620+0000 7ffae9c95700 1 -- 172.21.15.144:0/1024912748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffae40ff300 msgr2=0x7ffae4196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:42.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.620+0000 7ffae9c95700 1 --2- 172.21.15.144:0/1024912748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffae40ff300 0x7ffae4196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.620+0000 7ffae9c95700 1 -- 172.21.15.144:0/1024912748 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad80095f0 con 0x7ffae40ffca0 2024-09-10T11:47:42.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.620+0000 7ffaea496700 1 --2- 172.21.15.144:0/1024912748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffae40ff300 0x7ffae4196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:47:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.620+0000 7ffae9c95700 1 --2- 172.21.15.144:0/1024912748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffae40ffca0 0x7ffae41971d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffae000d8d0 tx=0x7ffae000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.620+0000 7ffad77fe700 1 -- 172.21.15.144:0/1024912748 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffae0004a00 con 0x7ffae40ffca0 2024-09-10T11:47:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.620+0000 7ffad77fe700 1 -- 172.21.15.144:0/1024912748 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffae00054e0 con 0x7ffae40ffca0 2024-09-10T11:47:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.620+0000 7ffaec6fa700 1 -- 172.21.15.144:0/1024912748 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae419fd00 con 0x7ffae40ffca0 2024-09-10T11:47:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.620+0000 7ffad77fe700 1 -- 172.21.15.144:0/1024912748 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffae0009d40 con 0x7ffae40ffca0 2024-09-10T11:47:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.620+0000 7ffaec6fa700 1 -- 172.21.15.144:0/1024912748 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae41a01c0 con 0x7ffae40ffca0 2024-09-10T11:47:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.621+0000 7ffaec6fa700 1 -- 172.21.15.144:0/1024912748 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae4107610 con 0x7ffae40ffca0 2024-09-10T11:47:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.622+0000 7ffad77fe700 1 -- 172.21.15.144:0/1024912748 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ffae0005020 con 0x7ffae40ffca0 2024-09-10T11:47:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.622+0000 7ffad77fe700 1 --2- 172.21.15.144:0/1024912748 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffad006e200 0x7ffad00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.622+0000 7ffaea496700 1 --2- 172.21.15.144:0/1024912748 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffad006e200 0x7ffad00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.622+0000 7ffad77fe700 1 -- 172.21.15.144:0/1024912748 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ffae008d9b0 con 0x7ffae40ffca0 2024-09-10T11:47:42.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.623+0000 7ffaea496700 1 --2- 172.21.15.144:0/1024912748 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffad006e200 0x7ffad00706b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ffad800b5c0 tx=0x7ffad80053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:42.628 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.627+0000 7ffad77fe700 1 -- 172.21.15.144:0/1024912748 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ffae0056d90 con 0x7ffae40ffca0 2024-09-10T11:47:42.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.842+0000 7ffaec6fa700 1 -- 172.21.15.144:0/1024912748 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffae402cca0 con 0x7ffad006e200 2024-09-10T11:47:42.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.852+0000 7ffad77fe700 1 -- 172.21.15.144:0/1024912748 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7ffae402cca0 con 0x7ffad006e200 2024-09-10T11:47:42.853 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:47:42.853 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (15m) 9m ago 26m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:47:42.853 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (26m) 9m ago 26m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:47:42.853 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (25m) 9m ago 25m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:47:42.854 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (13m) 9m ago 26m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:47:42.854 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (13m) 9m ago 25m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:47:42.854 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (15m) 9m ago 26m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:47:42.854 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (10m) 9m ago 21m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:47:42.854 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (9m) 9m ago 20m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:47:42.854 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (20m) 9m ago 20m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:47:42.854 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (9m) 9m ago 21m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:47:42.854 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (17m) 9m ago 27m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:47:42.854 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (16m) 9m ago 25m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:47:42.854 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (14m) 9m ago 28m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:47:42.854 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (14m) 9m ago 25m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:47:42.854 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (16m) 9m ago 26m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:47:42.854 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (16m) 9m ago 25m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:47:42.854 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (13m) 9m ago 24m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:47:42.855 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (12m) 9m ago 24m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:47:42.855 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (11m) 9m ago 23m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:47:42.855 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (11m) 9m ago 23m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:47:42.855 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (10m) 9m ago 22m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:47:42.855 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (10m) 9m ago 22m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:47:42.855 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (16m) 9m ago 26m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:47:42.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.854+0000 7ffaec6fa700 1 -- 172.21.15.144:0/1024912748 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffad006e200 msgr2=0x7ffad00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.854+0000 7ffaec6fa700 1 --2- 172.21.15.144:0/1024912748 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffad006e200 0x7ffad00706b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7ffad800b5c0 tx=0x7ffad80053a0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.854+0000 7ffaec6fa700 1 -- 172.21.15.144:0/1024912748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffae40ffca0 msgr2=0x7ffae41971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.854+0000 7ffaec6fa700 1 --2- 172.21.15.144:0/1024912748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffae40ffca0 0x7ffae41971d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7ffae000d8d0 tx=0x7ffae000dc90 comp rx=0 tx=0).stop 2024-09-10T11:47:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.854+0000 7ffaec6fa700 1 -- 172.21.15.144:0/1024912748 shutdown_connections 2024-09-10T11:47:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.854+0000 7ffaec6fa700 1 --2- 172.21.15.144:0/1024912748 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ffad006e200 0x7ffad00706b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.854+0000 7ffaec6fa700 1 --2- 172.21.15.144:0/1024912748 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ffae40ff300 0x7ffae4196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.854+0000 7ffaec6fa700 1 --2- 172.21.15.144:0/1024912748 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ffae40ffca0 0x7ffae41971d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.854+0000 7ffaec6fa700 1 -- 172.21.15.144:0/1024912748 >> 172.21.15.144:0/1024912748 conn(0x7ffae40faba0 msgr2=0x7ffae40fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.854+0000 7ffaec6fa700 1 -- 172.21.15.144:0/1024912748 shutdown_connections 2024-09-10T11:47:42.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.854+0000 7ffaec6fa700 1 -- 172.21.15.144:0/1024912748 wait complete. 2024-09-10T11:47:42.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.958+0000 7f9cc617d700 1 -- 172.21.15.144:0/4111029432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cc00ffc70 msgr2=0x7f9cc0103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:42.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.958+0000 7f9cc617d700 1 --2- 172.21.15.144:0/4111029432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cc00ffc70 0x7f9cc0103c40 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9cb4009910 tx=0x7f9cb4009c20 comp rx=0 tx=0).stop 2024-09-10T11:47:42.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.958+0000 7f9cc617d700 1 -- 172.21.15.144:0/4111029432 shutdown_connections 2024-09-10T11:47:42.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.958+0000 7f9cc617d700 1 --2- 172.21.15.144:0/4111029432 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cc00ffc70 0x7f9cc0103c40 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.958+0000 7f9cc617d700 1 --2- 172.21.15.144:0/4111029432 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cc00ff2d0 0x7f9cc00ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.958+0000 7f9cc617d700 1 -- 172.21.15.144:0/4111029432 >> 172.21.15.144:0/4111029432 conn(0x7f9cc00fabb0 msgr2=0x7f9cc00fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:42.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.958+0000 7f9cc617d700 1 -- 172.21.15.144:0/4111029432 shutdown_connections 2024-09-10T11:47:42.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.958+0000 7f9cc617d700 1 -- 172.21.15.144:0/4111029432 wait complete. 2024-09-10T11:47:42.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.959+0000 7f9cc617d700 1 Processor -- start 2024-09-10T11:47:42.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.959+0000 7f9cc617d700 1 -- start start 2024-09-10T11:47:42.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.959+0000 7f9cc617d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cc00ff2d0 0x7f9cc0195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:42.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.959+0000 7f9cc617d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cc00ffc70 0x7f9cc0196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:42.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.959+0000 7f9cc617d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc0196a60 con 0x7f9cc00ff2d0 2024-09-10T11:47:42.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.959+0000 7f9cc617d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc019a7f0 con 0x7f9cc00ffc70 2024-09-10T11:47:42.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.959+0000 7f9cbf7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cc00ff2d0 0x7f9cc0195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:42.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.959+0000 7f9cbf7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cc00ff2d0 0x7f9cc0195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33618/0 (socket says 172.21.15.144:33618) 2024-09-10T11:47:42.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.960+0000 7f9cbf7fe700 1 -- 172.21.15.144:0/494752716 learned_addr learned my addr 172.21.15.144:0/494752716 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:47:42.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.960+0000 7f9cbeffd700 1 --2- 172.21.15.144:0/494752716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cc00ffc70 0x7f9cc0196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.960+0000 7f9cbf7fe700 1 -- 172.21.15.144:0/494752716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cc00ffc70 msgr2=0x7f9cc0196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.960+0000 7f9cbf7fe700 1 --2- 172.21.15.144:0/494752716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cc00ffc70 0x7f9cc0196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.960+0000 7f9cbf7fe700 1 -- 172.21.15.144:0/494752716 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb40095f0 con 0x7f9cc00ff2d0 2024-09-10T11:47:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.960+0000 7f9cbf7fe700 1 --2- 172.21.15.144:0/494752716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cc00ff2d0 0x7f9cc0195e40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000b4b0 tx=0x7f9cb000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.960+0000 7f9cbcff9700 1 -- 172.21.15.144:0/494752716 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cb0011800 con 0x7f9cc00ff2d0 2024-09-10T11:47:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.960+0000 7f9cc617d700 1 -- 172.21.15.144:0/494752716 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc019aad0 con 0x7f9cc00ff2d0 2024-09-10T11:47:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.960+0000 7f9cbcff9700 1 -- 172.21.15.144:0/494752716 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cb0011e20 con 0x7f9cc00ff2d0 2024-09-10T11:47:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.960+0000 7f9cbcff9700 1 -- 172.21.15.144:0/494752716 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cb000f2e0 con 0x7f9cc00ff2d0 2024-09-10T11:47:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.960+0000 7f9cc617d700 1 -- 172.21.15.144:0/494752716 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc019b020 con 0x7f9cc00ff2d0 2024-09-10T11:47:42.964 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.961+0000 7f9cc617d700 1 -- 172.21.15.144:0/494752716 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc004ea50 con 0x7f9cc00ff2d0 2024-09-10T11:47:42.964 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.962+0000 7f9cbcff9700 1 -- 172.21.15.144:0/494752716 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9cb0010380 con 0x7f9cc00ff2d0 2024-09-10T11:47:42.964 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.962+0000 7f9cbcff9700 1 --2- 172.21.15.144:0/494752716 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9ca806e200 0x7f9ca80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:42.964 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.962+0000 7f9cbcff9700 1 -- 172.21.15.144:0/494752716 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9cb008dc50 con 0x7f9cc00ff2d0 2024-09-10T11:47:42.965 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.962+0000 7f9cbeffd700 1 --2- 172.21.15.144:0/494752716 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9ca806e200 0x7f9ca80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:42.965 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.963+0000 7f9cbeffd700 1 --2- 172.21.15.144:0/494752716 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9ca806e200 0x7f9ca80706b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0197460 tx=0x7f9cb401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:42.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:42.967+0000 7f9cbcff9700 1 -- 172.21.15.144:0/494752716 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9cb0057030 con 0x7f9cc00ff2d0 2024-09-10T11:47:43.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:42 smithi144 ceph-mon[87436]: pgmap v471: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:43.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:42 smithi178 ceph-mon[65940]: pgmap v471: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:43.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.255+0000 7f9cc617d700 1 -- 172.21.15.144:0/494752716 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9cc019b4b0 con 0x7f9cc00ff2d0 2024-09-10T11:47:43.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.255+0000 7f9cbcff9700 1 -- 172.21.15.144:0/494752716 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f9cb005a6f0 con 0x7f9cc00ff2d0 2024-09-10T11:47:43.257 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:47:43.257 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:47:43.257 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:47:43.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:47:43.258 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:47:43.258 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:47:43.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:47:43.258 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:47:43.258 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:47:43.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:47:43.258 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:47:43.258 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:47:43.258 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:47:43.259 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:47:43.259 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:47:43.259 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:47:43.259 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:47:43.259 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:47:43.259 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:47:43.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.257+0000 7f9cc617d700 1 -- 172.21.15.144:0/494752716 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9ca806e200 msgr2=0x7f9ca80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:43.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.257+0000 7f9cc617d700 1 --2- 172.21.15.144:0/494752716 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9ca806e200 0x7f9ca80706b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0197460 tx=0x7f9cb401a040 comp rx=0 tx=0).stop 2024-09-10T11:47:43.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.257+0000 7f9cc617d700 1 -- 172.21.15.144:0/494752716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cc00ff2d0 msgr2=0x7f9cc0195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:43.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.257+0000 7f9cc617d700 1 --2- 172.21.15.144:0/494752716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cc00ff2d0 0x7f9cc0195e40 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9cb000b4b0 tx=0x7f9cb000b870 comp rx=0 tx=0).stop 2024-09-10T11:47:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.257+0000 7f9cc617d700 1 -- 172.21.15.144:0/494752716 shutdown_connections 2024-09-10T11:47:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.257+0000 7f9cc617d700 1 --2- 172.21.15.144:0/494752716 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9ca806e200 0x7f9ca80706b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.257+0000 7f9cc617d700 1 --2- 172.21.15.144:0/494752716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cc00ff2d0 0x7f9cc0195e40 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.257+0000 7f9cc617d700 1 --2- 172.21.15.144:0/494752716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cc00ffc70 0x7f9cc0196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.257+0000 7f9cc617d700 1 -- 172.21.15.144:0/494752716 >> 172.21.15.144:0/494752716 conn(0x7f9cc00fabb0 msgr2=0x7f9cc01033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.258+0000 7f9cc617d700 1 -- 172.21.15.144:0/494752716 shutdown_connections 2024-09-10T11:47:43.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.258+0000 7f9cc617d700 1 -- 172.21.15.144:0/494752716 wait complete. 2024-09-10T11:47:43.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.362+0000 7fc3d84c5700 1 -- 172.21.15.144:0/1373829486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ffd30 msgr2=0x7fc3d0103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:43.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.362+0000 7fc3d84c5700 1 --2- 172.21.15.144:0/1373829486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ffd30 0x7fc3d0103ce0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc009910 tx=0x7fc3cc009c20 comp rx=0 tx=0).stop 2024-09-10T11:47:43.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.362+0000 7fc3d84c5700 1 -- 172.21.15.144:0/1373829486 shutdown_connections 2024-09-10T11:47:43.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.362+0000 7fc3d84c5700 1 --2- 172.21.15.144:0/1373829486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ffd30 0x7fc3d0103ce0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.362+0000 7fc3d84c5700 1 --2- 172.21.15.144:0/1373829486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3d00ff390 0x7fc3d00ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.362+0000 7fc3d84c5700 1 -- 172.21.15.144:0/1373829486 >> 172.21.15.144:0/1373829486 conn(0x7fc3d0074f50 msgr2=0x7fc3d0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:43.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.362+0000 7fc3d84c5700 1 -- 172.21.15.144:0/1373829486 shutdown_connections 2024-09-10T11:47:43.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.362+0000 7fc3d84c5700 1 -- 172.21.15.144:0/1373829486 wait complete. 2024-09-10T11:47:43.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.363+0000 7fc3d84c5700 1 Processor -- start 2024-09-10T11:47:43.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.363+0000 7fc3d84c5700 1 -- start start 2024-09-10T11:47:43.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.363+0000 7fc3d84c5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3d00ff390 0x7fc3d0195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:43.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.363+0000 7fc3d84c5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ffd30 0x7fc3d01962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:43.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.363+0000 7fc3d84c5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d01969b0 con 0x7fc3d00ffd30 2024-09-10T11:47:43.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.363+0000 7fc3d84c5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d019a740 con 0x7fc3d00ff390 2024-09-10T11:47:43.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.363+0000 7fc3d5a60700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ffd30 0x7fc3d01962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:43.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.363+0000 7fc3d5a60700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ffd30 0x7fc3d01962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33648/0 (socket says 172.21.15.144:33648) 2024-09-10T11:47:43.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.363+0000 7fc3d5a60700 1 -- 172.21.15.144:0/2310664617 learned_addr learned my addr 172.21.15.144:0/2310664617 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:47:43.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.363+0000 7fc3d6261700 1 --2- 172.21.15.144:0/2310664617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3d00ff390 0x7fc3d0195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:43.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.363+0000 7fc3d5a60700 1 -- 172.21.15.144:0/2310664617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3d00ff390 msgr2=0x7fc3d0195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:43.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.363+0000 7fc3d5a60700 1 --2- 172.21.15.144:0/2310664617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3d00ff390 0x7fc3d0195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.363+0000 7fc3d5a60700 1 -- 172.21.15.144:0/2310664617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3cc0095f0 con 0x7fc3d00ffd30 2024-09-10T11:47:43.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.364+0000 7fc3d5a60700 1 --2- 172.21.15.144:0/2310664617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ffd30 0x7fc3d01962d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0092e0 tx=0x7fc3cc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:43.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.364+0000 7fc3c77fe700 1 -- 172.21.15.144:0/2310664617 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3cc01d070 con 0x7fc3d00ffd30 2024-09-10T11:47:43.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.364+0000 7fc3c77fe700 1 -- 172.21.15.144:0/2310664617 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3cc005dc0 con 0x7fc3d00ffd30 2024-09-10T11:47:43.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.364+0000 7fc3d84c5700 1 -- 172.21.15.144:0/2310664617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d019a9c0 con 0x7fc3d00ffd30 2024-09-10T11:47:43.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.364+0000 7fc3c77fe700 1 -- 172.21.15.144:0/2310664617 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3cc017400 con 0x7fc3d00ffd30 2024-09-10T11:47:43.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.364+0000 7fc3d84c5700 1 -- 172.21.15.144:0/2310664617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d019aeb0 con 0x7fc3d00ffd30 2024-09-10T11:47:43.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.365+0000 7fc3d84c5700 1 -- 172.21.15.144:0/2310664617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d004ea50 con 0x7fc3d00ffd30 2024-09-10T11:47:43.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.365+0000 7fc3c77fe700 1 -- 172.21.15.144:0/2310664617 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc3cc00f6f0 con 0x7fc3d00ffd30 2024-09-10T11:47:43.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.366+0000 7fc3c77fe700 1 --2- 172.21.15.144:0/2310664617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc3bc06e200 0x7fc3bc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:43.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.366+0000 7fc3d6261700 1 --2- 172.21.15.144:0/2310664617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc3bc06e200 0x7fc3bc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:43.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.366+0000 7fc3c77fe700 1 -- 172.21.15.144:0/2310664617 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc3cc08f3a0 con 0x7fc3d00ffd30 2024-09-10T11:47:43.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.367+0000 7fc3d6261700 1 --2- 172.21.15.144:0/2310664617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc3bc06e200 0x7fc3bc0706b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0005950 tx=0x7fc3c000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:43.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.371+0000 7fc3c77fe700 1 -- 172.21.15.144:0/2310664617 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc3cc058800 con 0x7fc3d00ffd30 2024-09-10T11:47:43.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.618+0000 7fc3d84c5700 1 -- 172.21.15.144:0/2310664617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc3d019b370 con 0x7fc3d00ffd30 2024-09-10T11:47:43.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.619+0000 7fc3c77fe700 1 -- 172.21.15.144:0/2310664617 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7fc3cc05bec0 con 0x7fc3d00ffd30 2024-09-10T11:47:43.620 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:47:43.620 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:47:43.620 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:47:43.620 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:47:43.621 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:47:43.621 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:47:43.621 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:47:43.621 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:47:43.621 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:47:43.621 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:47:43.621 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:47:43.621 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:47:43.621 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:47:43.621 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:47:43.621 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:47:43.621 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:47:43.621 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:47:43.622 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:47:43.622 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:47:43.622 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:47:43.622 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:47:43.622 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:47:43.622 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:47:43.622 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:47:43.622 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:47:43.622 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:47:43.622 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:47:43.622 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:47:43.622 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:47:43.622 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:47:43.622 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:47:43.623 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:47:43.623 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:47:43.623 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:47:43.623 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:47:43.623 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:47:43.623 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:47:43.623 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:47:43.623 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:47:43.623 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:47:43.623 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:47:43.623 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:47:43.623 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:47:43.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.621+0000 7fc3d84c5700 1 -- 172.21.15.144:0/2310664617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc3bc06e200 msgr2=0x7fc3bc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:43.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.621+0000 7fc3d84c5700 1 --2- 172.21.15.144:0/2310664617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc3bc06e200 0x7fc3bc0706b0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc3c0005950 tx=0x7fc3c000b410 comp rx=0 tx=0).stop 2024-09-10T11:47:43.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.621+0000 7fc3d84c5700 1 -- 172.21.15.144:0/2310664617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ffd30 msgr2=0x7fc3d01962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:43.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.621+0000 7fc3d84c5700 1 --2- 172.21.15.144:0/2310664617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ffd30 0x7fc3d01962d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fc3cc0092e0 tx=0x7fc3cc00bd70 comp rx=0 tx=0).stop 2024-09-10T11:47:43.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.621+0000 7fc3d84c5700 1 -- 172.21.15.144:0/2310664617 shutdown_connections 2024-09-10T11:47:43.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.621+0000 7fc3d84c5700 1 --2- 172.21.15.144:0/2310664617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc3bc06e200 0x7fc3bc0706b0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.621+0000 7fc3d84c5700 1 --2- 172.21.15.144:0/2310664617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc3d00ff390 0x7fc3d0195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.621+0000 7fc3d84c5700 1 --2- 172.21.15.144:0/2310664617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc3d00ffd30 0x7fc3d01962d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.621+0000 7fc3d84c5700 1 -- 172.21.15.144:0/2310664617 >> 172.21.15.144:0/2310664617 conn(0x7fc3d0074f50 msgr2=0x7fc3d0103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:43.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.622+0000 7fc3d84c5700 1 -- 172.21.15.144:0/2310664617 shutdown_connections 2024-09-10T11:47:43.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.622+0000 7fc3d84c5700 1 -- 172.21.15.144:0/2310664617 wait complete. 2024-09-10T11:47:43.625 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:47:43.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.728+0000 7f9565cfb700 1 -- 172.21.15.144:0/1755265603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95600ff310 msgr2=0x7f95600ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:43.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.728+0000 7f9565cfb700 1 --2- 172.21.15.144:0/1755265603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95600ff310 0x7f95600ff6e0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f955000b0a0 tx=0x7f955000b3b0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.728+0000 7f9565cfb700 1 -- 172.21.15.144:0/1755265603 shutdown_connections 2024-09-10T11:47:43.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.728+0000 7f9565cfb700 1 --2- 172.21.15.144:0/1755265603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95600ffcb0 0x7f9560103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.728+0000 7f9565cfb700 1 --2- 172.21.15.144:0/1755265603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95600ff310 0x7f95600ff6e0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.728+0000 7f9565cfb700 1 -- 172.21.15.144:0/1755265603 >> 172.21.15.144:0/1755265603 conn(0x7f95600fabd0 msgr2=0x7f95600fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:43.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.728+0000 7f9565cfb700 1 -- 172.21.15.144:0/1755265603 shutdown_connections 2024-09-10T11:47:43.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.728+0000 7f9565cfb700 1 -- 172.21.15.144:0/1755265603 wait complete. 2024-09-10T11:47:43.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.728+0000 7f9565cfb700 1 Processor -- start 2024-09-10T11:47:43.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.729+0000 7f9565cfb700 1 -- start start 2024-09-10T11:47:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.729+0000 7f9565cfb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95600ff310 0x7f9560194b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.729+0000 7f9565cfb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95600ffcb0 0x7f9560195060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.729+0000 7f9565cfb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95601955a0 con 0x7f95600ff310 2024-09-10T11:47:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.729+0000 7f9565cfb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9560195710 con 0x7f95600ffcb0 2024-09-10T11:47:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.729+0000 7f955f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95600ff310 0x7f9560194b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.729+0000 7f955f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95600ff310 0x7f9560194b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33672/0 (socket says 172.21.15.144:33672) 2024-09-10T11:47:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.729+0000 7f955f7fe700 1 -- 172.21.15.144:0/2107677113 learned_addr learned my addr 172.21.15.144:0/2107677113 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:47:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.729+0000 7f955effd700 1 --2- 172.21.15.144:0/2107677113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95600ffcb0 0x7f9560195060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.729+0000 7f955f7fe700 1 -- 172.21.15.144:0/2107677113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95600ffcb0 msgr2=0x7f9560195060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.729+0000 7f955f7fe700 1 --2- 172.21.15.144:0/2107677113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95600ffcb0 0x7f9560195060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.729+0000 7f955f7fe700 1 -- 172.21.15.144:0/2107677113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9550009d00 con 0x7f95600ff310 2024-09-10T11:47:43.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.729+0000 7f955effd700 1 --2- 172.21.15.144:0/2107677113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95600ffcb0 0x7f9560195060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:47:43.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.729+0000 7f955f7fe700 1 --2- 172.21.15.144:0/2107677113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95600ff310 0x7f9560194b20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f955000b070 tx=0x7f955000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:43.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.729+0000 7f955cff9700 1 -- 172.21.15.144:0/2107677113 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f955000e040 con 0x7f95600ff310 2024-09-10T11:47:43.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.730+0000 7f955cff9700 1 -- 172.21.15.144:0/2107677113 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9550003e60 con 0x7f95600ff310 2024-09-10T11:47:43.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.730+0000 7f9565cfb700 1 -- 172.21.15.144:0/2107677113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9560195990 con 0x7f95600ff310 2024-09-10T11:47:43.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.730+0000 7f955cff9700 1 -- 172.21.15.144:0/2107677113 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f955001d910 con 0x7f95600ff310 2024-09-10T11:47:43.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.730+0000 7f9565cfb700 1 -- 172.21.15.144:0/2107677113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95601a2410 con 0x7f95600ff310 2024-09-10T11:47:43.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.731+0000 7f9565cfb700 1 -- 172.21.15.144:0/2107677113 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9560107620 con 0x7f95600ff310 2024-09-10T11:47:43.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.731+0000 7f955cff9700 1 -- 172.21.15.144:0/2107677113 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9550004820 con 0x7f95600ff310 2024-09-10T11:47:43.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.731+0000 7f955cff9700 1 --2- 172.21.15.144:0/2107677113 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f954806e200 0x7f95480706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:43.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.732+0000 7f955cff9700 1 -- 172.21.15.144:0/2107677113 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f955008f750 con 0x7f95600ff310 2024-09-10T11:47:43.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.732+0000 7f955effd700 1 --2- 172.21.15.144:0/2107677113 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f954806e200 0x7f95480706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:43.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.732+0000 7f955effd700 1 --2- 172.21.15.144:0/2107677113 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f954806e200 0x7f95480706b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f95600696d0 tx=0x7f9554008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:43.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.736+0000 7f955cff9700 1 -- 172.21.15.144:0/2107677113 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f955005c6e0 con 0x7f95600ff310 2024-09-10T11:47:43.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:43 smithi144 ceph-mon[87436]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:43 smithi144 ceph-mon[87436]: from='client.34908 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:43 smithi144 ceph-mon[87436]: from='client.34912 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:43 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/494752716' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:47:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:47:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:43 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2310664617' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:47:43.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.958+0000 7f9565cfb700 1 -- 172.21.15.144:0/2107677113 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9560195990 con 0x7f954806e200 2024-09-10T11:47:43.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.960+0000 7f955cff9700 1 -- 172.21.15.144:0/2107677113 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f9560195990 con 0x7f954806e200 2024-09-10T11:47:43.961 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:47:43.961 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:47:43.961 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:47:43.961 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:47:43.961 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:47:43.961 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:47:43.961 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:47:43.961 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:47:43.961 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:47:43.961 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:47:43.961 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:47:43.962 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:47:43.962 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:47:43.962 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:47:43.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.962+0000 7f9565cfb700 1 -- 172.21.15.144:0/2107677113 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f954806e200 msgr2=0x7f95480706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:43.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.962+0000 7f9565cfb700 1 --2- 172.21.15.144:0/2107677113 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f954806e200 0x7f95480706b0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f95600696d0 tx=0x7f9554008040 comp rx=0 tx=0).stop 2024-09-10T11:47:43.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.962+0000 7f9565cfb700 1 -- 172.21.15.144:0/2107677113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95600ff310 msgr2=0x7f9560194b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:43.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.962+0000 7f9565cfb700 1 --2- 172.21.15.144:0/2107677113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95600ff310 0x7f9560194b20 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f955000b070 tx=0x7f955000cdd0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.962+0000 7f9565cfb700 1 -- 172.21.15.144:0/2107677113 shutdown_connections 2024-09-10T11:47:43.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.962+0000 7f9565cfb700 1 --2- 172.21.15.144:0/2107677113 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f954806e200 0x7f95480706b0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.962+0000 7f9565cfb700 1 --2- 172.21.15.144:0/2107677113 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f95600ff310 0x7f9560194b20 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.962+0000 7f9565cfb700 1 --2- 172.21.15.144:0/2107677113 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f95600ffcb0 0x7f9560195060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:43.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.962+0000 7f9565cfb700 1 -- 172.21.15.144:0/2107677113 >> 172.21.15.144:0/2107677113 conn(0x7f95600fabd0 msgr2=0x7f95600fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:43.964 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.962+0000 7f9565cfb700 1 -- 172.21.15.144:0/2107677113 shutdown_connections 2024-09-10T11:47:43.964 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:43.962+0000 7f9565cfb700 1 -- 172.21.15.144:0/2107677113 wait complete. 2024-09-10T11:47:44.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.067+0000 7fa3ca601700 1 -- 172.21.15.144:0/3755041510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3c40ffc70 msgr2=0x7fa3c4103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:44.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.067+0000 7fa3ca601700 1 --2- 172.21.15.144:0/3755041510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3c40ffc70 0x7fa3c4103c40 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009910 tx=0x7fa3b8009c20 comp rx=0 tx=0).stop 2024-09-10T11:47:44.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.067+0000 7fa3ca601700 1 -- 172.21.15.144:0/3755041510 shutdown_connections 2024-09-10T11:47:44.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.067+0000 7fa3ca601700 1 --2- 172.21.15.144:0/3755041510 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3c40ffc70 0x7fa3c4103c40 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:44.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.067+0000 7fa3ca601700 1 --2- 172.21.15.144:0/3755041510 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3c40ff2d0 0x7fa3c40ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:44.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.067+0000 7fa3ca601700 1 -- 172.21.15.144:0/3755041510 >> 172.21.15.144:0/3755041510 conn(0x7fa3c40fabb0 msgr2=0x7fa3c40fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:44.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.067+0000 7fa3ca601700 1 -- 172.21.15.144:0/3755041510 shutdown_connections 2024-09-10T11:47:44.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.067+0000 7fa3ca601700 1 -- 172.21.15.144:0/3755041510 wait complete. 2024-09-10T11:47:44.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.068+0000 7fa3ca601700 1 Processor -- start 2024-09-10T11:47:44.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.068+0000 7fa3ca601700 1 -- start start 2024-09-10T11:47:44.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3ca601700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3c40ff2d0 0x7fa3c4196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:44.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3ca601700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3c40ffc70 0x7fa3c4197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:44.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3ca601700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c4197820 con 0x7fa3c40ff2d0 2024-09-10T11:47:44.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3ca601700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c4197960 con 0x7fa3c40ffc70 2024-09-10T11:47:44.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3c3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3c40ff2d0 0x7fa3c4196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:44.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3c3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3c40ff2d0 0x7fa3c4196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:33686/0 (socket says 172.21.15.144:33686) 2024-09-10T11:47:44.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3c3fff700 1 -- 172.21.15.144:0/2092575735 learned_addr learned my addr 172.21.15.144:0/2092575735 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:47:44.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3c37fe700 1 --2- 172.21.15.144:0/2092575735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3c40ffc70 0x7fa3c4197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:44.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3c3fff700 1 -- 172.21.15.144:0/2092575735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3c40ffc70 msgr2=0x7fa3c4197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:44.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3c3fff700 1 --2- 172.21.15.144:0/2092575735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3c40ffc70 0x7fa3c4197200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:44.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3c3fff700 1 -- 172.21.15.144:0/2092575735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b80095f0 con 0x7fa3c40ff2d0 2024-09-10T11:47:44.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3c37fe700 1 --2- 172.21.15.144:0/2092575735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3c40ffc70 0x7fa3c4197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:47:44.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3c3fff700 1 --2- 172.21.15.144:0/2092575735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3c40ff2d0 0x7fa3c4196cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400b4b0 tx=0x7fa3b400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:44.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3c17fa700 1 -- 172.21.15.144:0/2092575735 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3b4011800 con 0x7fa3c40ff2d0 2024-09-10T11:47:44.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3c17fa700 1 -- 172.21.15.144:0/2092575735 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3b4011e20 con 0x7fa3c40ff2d0 2024-09-10T11:47:44.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3ca601700 1 -- 172.21.15.144:0/2092575735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3c419fd30 con 0x7fa3c40ff2d0 2024-09-10T11:47:44.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3c17fa700 1 -- 172.21.15.144:0/2092575735 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3b400f2e0 con 0x7fa3c40ff2d0 2024-09-10T11:47:44.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.069+0000 7fa3ca601700 1 -- 172.21.15.144:0/2092575735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c41a01f0 con 0x7fa3c40ff2d0 2024-09-10T11:47:44.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.070+0000 7fa3ca601700 1 -- 172.21.15.144:0/2092575735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c4107570 con 0x7fa3c40ff2d0 2024-09-10T11:47:44.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.071+0000 7fa3c17fa700 1 -- 172.21.15.144:0/2092575735 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa3b4011960 con 0x7fa3c40ff2d0 2024-09-10T11:47:44.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.071+0000 7fa3c17fa700 1 --2- 172.21.15.144:0/2092575735 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3ac06e200 0x7fa3ac0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:47:44.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.071+0000 7fa3c17fa700 1 -- 172.21.15.144:0/2092575735 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa3b408db50 con 0x7fa3c40ff2d0 2024-09-10T11:47:44.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.071+0000 7fa3c37fe700 1 --2- 172.21.15.144:0/2092575735 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3ac06e200 0x7fa3ac0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:47:44.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.072+0000 7fa3c37fe700 1 --2- 172.21.15.144:0/2092575735 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3ac06e200 0x7fa3ac0706b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4069600 tx=0x7fa3b800b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:47:44.076 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.076+0000 7fa3c17fa700 1 -- 172.21.15.144:0/2092575735 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa3b4056fb0 con 0x7fa3c40ff2d0 2024-09-10T11:47:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:43 smithi178 ceph-mon[65940]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:43 smithi178 ceph-mon[65940]: from='client.34908 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:43 smithi178 ceph-mon[65940]: from='client.34912 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:43 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/494752716' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:47:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:47:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:43 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2310664617' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:47:44.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.357+0000 7fa3ca601700 1 -- 172.21.15.144:0/2092575735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa3c404ea50 con 0x7fa3c40ff2d0 2024-09-10T11:47:44.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.358+0000 7fa3c17fa700 1 -- 172.21.15.144:0/2092575735 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fa3b40151c0 con 0x7fa3c40ff2d0 2024-09-10T11:47:44.361 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:47:44.361 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:47:44.361 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:47:44.361 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:47:44.361 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:47:44.362 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:47:44.362 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:47:44.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.360+0000 7fa3ca601700 1 -- 172.21.15.144:0/2092575735 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3ac06e200 msgr2=0x7fa3ac0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:44.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.360+0000 7fa3ca601700 1 --2- 172.21.15.144:0/2092575735 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3ac06e200 0x7fa3ac0706b0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa3c4069600 tx=0x7fa3b800b540 comp rx=0 tx=0).stop 2024-09-10T11:47:44.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.360+0000 7fa3ca601700 1 -- 172.21.15.144:0/2092575735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3c40ff2d0 msgr2=0x7fa3c4196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:47:44.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.360+0000 7fa3ca601700 1 --2- 172.21.15.144:0/2092575735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3c40ff2d0 0x7fa3c4196cc0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400b4b0 tx=0x7fa3b400b870 comp rx=0 tx=0).stop 2024-09-10T11:47:44.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.360+0000 7fa3ca601700 1 -- 172.21.15.144:0/2092575735 shutdown_connections 2024-09-10T11:47:44.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.360+0000 7fa3ca601700 1 --2- 172.21.15.144:0/2092575735 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3ac06e200 0x7fa3ac0706b0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:44.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.360+0000 7fa3ca601700 1 --2- 172.21.15.144:0/2092575735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3c40ff2d0 0x7fa3c4196cc0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:44.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.360+0000 7fa3ca601700 1 --2- 172.21.15.144:0/2092575735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3c40ffc70 0x7fa3c4197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:47:44.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.360+0000 7fa3ca601700 1 -- 172.21.15.144:0/2092575735 >> 172.21.15.144:0/2092575735 conn(0x7fa3c40fabb0 msgr2=0x7fa3c40fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:47:44.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.360+0000 7fa3ca601700 1 -- 172.21.15.144:0/2092575735 shutdown_connections 2024-09-10T11:47:44.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:47:44.360+0000 7fa3ca601700 1 -- 172.21.15.144:0/2092575735 wait complete. 2024-09-10T11:47:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:44 smithi178 ceph-mon[65940]: pgmap v472: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:44 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2092575735' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:47:45.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:44 smithi144 ceph-mon[87436]: pgmap v472: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:44 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2092575735' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:47:46.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:45 smithi178 ceph-mon[65940]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:46.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:45 smithi144 ceph-mon[87436]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:47:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:46 smithi178 ceph-mon[65940]: pgmap v473: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:47.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:46 smithi144 ceph-mon[87436]: pgmap v473: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:48 smithi178 ceph-mon[65940]: pgmap v474: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:49.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:48 smithi144 ceph-mon[87436]: pgmap v474: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:50 smithi178 ceph-mon[65940]: pgmap v475: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:50 smithi144 ceph-mon[87436]: pgmap v475: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:53.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:52 smithi144 ceph-mon[87436]: pgmap v476: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:52 smithi178 ceph-mon[65940]: pgmap v476: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:54 smithi178 ceph-mon[65940]: pgmap v477: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:55.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:54 smithi144 ceph-mon[87436]: pgmap v477: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:56 smithi178 ceph-mon[65940]: pgmap v478: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:57.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:56 smithi144 ceph-mon[87436]: pgmap v478: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:58 smithi178 ceph-mon[65940]: pgmap v479: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:47:59.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:58 smithi144 ceph-mon[87436]: pgmap v479: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:47:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:48:00.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:47:59 smithi144 ceph-mon[87436]: pgmap v480: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:00.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:47:59 smithi178 ceph-mon[65940]: pgmap v480: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:03.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:02 smithi144 ceph-mon[87436]: pgmap v481: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:03.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:02 smithi178 ceph-mon[65940]: pgmap v481: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:04 smithi178 ceph-mon[65940]: pgmap v482: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:05.117 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:04 smithi144 ceph-mon[87436]: pgmap v482: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:06 smithi178 ceph-mon[65940]: pgmap v483: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:06 smithi144 ceph-mon[87436]: pgmap v483: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:08 smithi178 ceph-mon[65940]: pgmap v484: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:09.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:08 smithi144 ceph-mon[87436]: pgmap v484: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:10 smithi178 ceph-mon[65940]: pgmap v485: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:11.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:10 smithi144 ceph-mon[87436]: pgmap v485: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:12 smithi144 ceph-mon[87436]: pgmap v486: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:13.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:12 smithi178 ceph-mon[65940]: pgmap v486: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:48:14.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:48:14.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.480+0000 7fec4b16a700 1 -- 172.21.15.144:0/2576096115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec44100d80 msgr2=0x7fec44104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:14.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.480+0000 7fec4b16a700 1 --2- 172.21.15.144:0/2576096115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec44100d80 0x7fec44104d50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fec3400b0a0 tx=0x7fec3400b3b0 comp rx=0 tx=0).stop 2024-09-10T11:48:14.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.480+0000 7fec4b16a700 1 -- 172.21.15.144:0/2576096115 shutdown_connections 2024-09-10T11:48:14.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.480+0000 7fec4b16a700 1 --2- 172.21.15.144:0/2576096115 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec44100d80 0x7fec44104d50 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:14.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.480+0000 7fec4b16a700 1 --2- 172.21.15.144:0/2576096115 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec441003e0 0x7fec441007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:14.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.480+0000 7fec4b16a700 1 -- 172.21.15.144:0/2576096115 >> 172.21.15.144:0/2576096115 conn(0x7fec440fbca0 msgr2=0x7fec440fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:14.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.480+0000 7fec4b16a700 1 -- 172.21.15.144:0/2576096115 shutdown_connections 2024-09-10T11:48:14.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.480+0000 7fec4b16a700 1 -- 172.21.15.144:0/2576096115 wait complete. 2024-09-10T11:48:14.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.481+0000 7fec4b16a700 1 Processor -- start 2024-09-10T11:48:14.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.481+0000 7fec4b16a700 1 -- start start 2024-09-10T11:48:14.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.481+0000 7fec4b16a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec441003e0 0x7fec44195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:14.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.481+0000 7fec4b16a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec44100d80 0x7fec44196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:14.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.481+0000 7fec4b16a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec44196970 con 0x7fec441003e0 2024-09-10T11:48:14.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.481+0000 7fec4b16a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec4419a700 con 0x7fec44100d80 2024-09-10T11:48:14.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.481+0000 7fec48f06700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec441003e0 0x7fec44195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:14.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.482+0000 7fec48f06700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec441003e0 0x7fec44195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42170/0 (socket says 172.21.15.144:42170) 2024-09-10T11:48:14.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.482+0000 7fec48f06700 1 -- 172.21.15.144:0/1435141473 learned_addr learned my addr 172.21.15.144:0/1435141473 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:48:14.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.482+0000 7fec43fff700 1 --2- 172.21.15.144:0/1435141473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec44100d80 0x7fec44196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:14.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.482+0000 7fec48f06700 1 -- 172.21.15.144:0/1435141473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec44100d80 msgr2=0x7fec44196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:14.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.482+0000 7fec48f06700 1 --2- 172.21.15.144:0/1435141473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec44100d80 0x7fec44196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:14.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.482+0000 7fec48f06700 1 -- 172.21.15.144:0/1435141473 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec34009d00 con 0x7fec441003e0 2024-09-10T11:48:14.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.482+0000 7fec48f06700 1 --2- 172.21.15.144:0/1435141473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec441003e0 0x7fec44195d50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00d900 tx=0x7fec3c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:14.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.482+0000 7fec41ffb700 1 -- 172.21.15.144:0/1435141473 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec3c004a00 con 0x7fec441003e0 2024-09-10T11:48:14.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.482+0000 7fec41ffb700 1 -- 172.21.15.144:0/1435141473 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec3c0054e0 con 0x7fec441003e0 2024-09-10T11:48:14.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.482+0000 7fec4b16a700 1 -- 172.21.15.144:0/1435141473 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec4419a9e0 con 0x7fec441003e0 2024-09-10T11:48:14.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.482+0000 7fec41ffb700 1 -- 172.21.15.144:0/1435141473 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec3c009d00 con 0x7fec441003e0 2024-09-10T11:48:14.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.482+0000 7fec4b16a700 1 -- 172.21.15.144:0/1435141473 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4419af00 con 0x7fec441003e0 2024-09-10T11:48:14.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.483+0000 7fec4b16a700 1 -- 172.21.15.144:0/1435141473 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec44108660 con 0x7fec441003e0 2024-09-10T11:48:14.487 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.484+0000 7fec41ffb700 1 -- 172.21.15.144:0/1435141473 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fec3c005020 con 0x7fec441003e0 2024-09-10T11:48:14.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.484+0000 7fec41ffb700 1 --2- 172.21.15.144:0/1435141473 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec3006e200 0x7fec300706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:14.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.485+0000 7fec41ffb700 1 -- 172.21.15.144:0/1435141473 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fec3c08df40 con 0x7fec441003e0 2024-09-10T11:48:14.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.485+0000 7fec43fff700 1 --2- 172.21.15.144:0/1435141473 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec3006e200 0x7fec300706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:14.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.485+0000 7fec43fff700 1 --2- 172.21.15.144:0/1435141473 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec3006e200 0x7fec300706b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fec34009fd0 tx=0x7fec340060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:14.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.489+0000 7fec41ffb700 1 -- 172.21.15.144:0/1435141473 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fec3c0c31b0 con 0x7fec441003e0 2024-09-10T11:48:14.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.713+0000 7fec4b16a700 1 -- 172.21.15.144:0/1435141473 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec4419b1e0 con 0x7fec3006e200 2024-09-10T11:48:14.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.714+0000 7fec41ffb700 1 -- 172.21.15.144:0/1435141473 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fec4419b1e0 con 0x7fec3006e200 2024-09-10T11:48:14.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.716+0000 7fec4b16a700 1 -- 172.21.15.144:0/1435141473 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec3006e200 msgr2=0x7fec300706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:14.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.716+0000 7fec4b16a700 1 --2- 172.21.15.144:0/1435141473 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec3006e200 0x7fec300706b0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fec34009fd0 tx=0x7fec340060c0 comp rx=0 tx=0).stop 2024-09-10T11:48:14.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.716+0000 7fec4b16a700 1 -- 172.21.15.144:0/1435141473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec441003e0 msgr2=0x7fec44195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:14.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.716+0000 7fec4b16a700 1 --2- 172.21.15.144:0/1435141473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec441003e0 0x7fec44195d50 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00d900 tx=0x7fec3c00dc10 comp rx=0 tx=0).stop 2024-09-10T11:48:14.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.716+0000 7fec4b16a700 1 -- 172.21.15.144:0/1435141473 shutdown_connections 2024-09-10T11:48:14.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.716+0000 7fec4b16a700 1 --2- 172.21.15.144:0/1435141473 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec3006e200 0x7fec300706b0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:14.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.716+0000 7fec4b16a700 1 --2- 172.21.15.144:0/1435141473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec441003e0 0x7fec44195d50 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:14.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.717+0000 7fec4b16a700 1 --2- 172.21.15.144:0/1435141473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec44100d80 0x7fec44196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:14.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.717+0000 7fec4b16a700 1 -- 172.21.15.144:0/1435141473 >> 172.21.15.144:0/1435141473 conn(0x7fec440fbca0 msgr2=0x7fec441045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:14.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.717+0000 7fec4b16a700 1 -- 172.21.15.144:0/1435141473 shutdown_connections 2024-09-10T11:48:14.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.717+0000 7fec4b16a700 1 -- 172.21.15.144:0/1435141473 wait complete. 2024-09-10T11:48:14.728 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:48:14.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.822+0000 7f63924aa700 1 -- 172.21.15.144:0/41397854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f638c100d70 msgr2=0x7f638c104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:14.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.822+0000 7f63924aa700 1 --2- 172.21.15.144:0/41397854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f638c100d70 0x7f638c104d40 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6380009910 tx=0x7f6380009c20 comp rx=0 tx=0).stop 2024-09-10T11:48:14.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.822+0000 7f63924aa700 1 -- 172.21.15.144:0/41397854 shutdown_connections 2024-09-10T11:48:14.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.822+0000 7f63924aa700 1 --2- 172.21.15.144:0/41397854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f638c100d70 0x7f638c104d40 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:14.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.822+0000 7f63924aa700 1 --2- 172.21.15.144:0/41397854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f638c1003d0 0x7f638c1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:14.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.822+0000 7f63924aa700 1 -- 172.21.15.144:0/41397854 >> 172.21.15.144:0/41397854 conn(0x7f638c0fbc90 msgr2=0x7f638c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:14.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.822+0000 7f63924aa700 1 -- 172.21.15.144:0/41397854 shutdown_connections 2024-09-10T11:48:14.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.822+0000 7f63924aa700 1 -- 172.21.15.144:0/41397854 wait complete. 2024-09-10T11:48:14.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.822+0000 7f63924aa700 1 Processor -- start 2024-09-10T11:48:14.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.822+0000 7f63924aa700 1 -- start start 2024-09-10T11:48:14.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.823+0000 7f63924aa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f638c1003d0 0x7f638c195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:14.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.823+0000 7f63924aa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f638c100d70 0x7f638c1962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:14.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.823+0000 7f63924aa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c1969a0 con 0x7f638c1003d0 2024-09-10T11:48:14.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.823+0000 7f63924aa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f638c19a730 con 0x7f638c100d70 2024-09-10T11:48:14.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.823+0000 7f638bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f638c1003d0 0x7f638c195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:14.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.823+0000 7f638bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f638c1003d0 0x7f638c195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42188/0 (socket says 172.21.15.144:42188) 2024-09-10T11:48:14.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.823+0000 7f638bfff700 1 -- 172.21.15.144:0/641329467 learned_addr learned my addr 172.21.15.144:0/641329467 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:48:14.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.823+0000 7f638b7fe700 1 --2- 172.21.15.144:0/641329467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f638c100d70 0x7f638c1962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:14.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.823+0000 7f638bfff700 1 -- 172.21.15.144:0/641329467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f638c100d70 msgr2=0x7f638c1962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:14.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.823+0000 7f638bfff700 1 --2- 172.21.15.144:0/641329467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f638c100d70 0x7f638c1962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:14.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.823+0000 7f638bfff700 1 -- 172.21.15.144:0/641329467 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63800095f0 con 0x7f638c1003d0 2024-09-10T11:48:14.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.824+0000 7f638bfff700 1 --2- 172.21.15.144:0/641329467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f638c1003d0 0x7f638c195d80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f637c00d900 tx=0x7f637c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:14.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.824+0000 7f63897fa700 1 -- 172.21.15.144:0/641329467 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f637c004a00 con 0x7f638c1003d0 2024-09-10T11:48:14.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.824+0000 7f63924aa700 1 -- 172.21.15.144:0/641329467 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f638c19aa10 con 0x7f638c1003d0 2024-09-10T11:48:14.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.824+0000 7f63897fa700 1 -- 172.21.15.144:0/641329467 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f637c0054e0 con 0x7f638c1003d0 2024-09-10T11:48:14.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.824+0000 7f63897fa700 1 -- 172.21.15.144:0/641329467 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f637c009d00 con 0x7f638c1003d0 2024-09-10T11:48:14.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.824+0000 7f63924aa700 1 -- 172.21.15.144:0/641329467 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f638c19af60 con 0x7f638c1003d0 2024-09-10T11:48:14.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.825+0000 7f63924aa700 1 -- 172.21.15.144:0/641329467 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f638c108650 con 0x7f638c1003d0 2024-09-10T11:48:14.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.825+0000 7f63897fa700 1 -- 172.21.15.144:0/641329467 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f637c010460 con 0x7f638c1003d0 2024-09-10T11:48:14.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.826+0000 7f63897fa700 1 --2- 172.21.15.144:0/641329467 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f637406e200 0x7f63740706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:14.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.826+0000 7f63897fa700 1 -- 172.21.15.144:0/641329467 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f637c08dff0 con 0x7f638c1003d0 2024-09-10T11:48:14.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.826+0000 7f638b7fe700 1 --2- 172.21.15.144:0/641329467 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f637406e200 0x7f63740706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:14.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.826+0000 7f638b7fe700 1 --2- 172.21.15.144:0/641329467 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f637406e200 0x7f63740706b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f638c1973a0 tx=0x7f638001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:14.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:14.831+0000 7f63897fa700 1 -- 172.21.15.144:0/641329467 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f637c057440 con 0x7f638c1003d0 2024-09-10T11:48:15.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.062+0000 7f63924aa700 1 -- 172.21.15.144:0/641329467 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f638c19b240 con 0x7f637406e200 2024-09-10T11:48:15.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.063+0000 7f63897fa700 1 -- 172.21.15.144:0/641329467 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f638c19b240 con 0x7f637406e200 2024-09-10T11:48:15.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.065+0000 7f63924aa700 1 -- 172.21.15.144:0/641329467 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f637406e200 msgr2=0x7f63740706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:15.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.065+0000 7f63924aa700 1 --2- 172.21.15.144:0/641329467 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f637406e200 0x7f63740706b0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f638c1973a0 tx=0x7f638001a040 comp rx=0 tx=0).stop 2024-09-10T11:48:15.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.065+0000 7f63924aa700 1 -- 172.21.15.144:0/641329467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f638c1003d0 msgr2=0x7f638c195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:15.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.065+0000 7f63924aa700 1 --2- 172.21.15.144:0/641329467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f638c1003d0 0x7f638c195d80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f637c00d900 tx=0x7f637c00dc10 comp rx=0 tx=0).stop 2024-09-10T11:48:15.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.066+0000 7f63924aa700 1 -- 172.21.15.144:0/641329467 shutdown_connections 2024-09-10T11:48:15.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.066+0000 7f63924aa700 1 --2- 172.21.15.144:0/641329467 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f637406e200 0x7f63740706b0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.066+0000 7f63924aa700 1 --2- 172.21.15.144:0/641329467 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f638c1003d0 0x7f638c195d80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.066+0000 7f63924aa700 1 --2- 172.21.15.144:0/641329467 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f638c100d70 0x7f638c1962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.066+0000 7f63924aa700 1 -- 172.21.15.144:0/641329467 >> 172.21.15.144:0/641329467 conn(0x7f638c0fbc90 msgr2=0x7f638c1045b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:15.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.066+0000 7f63924aa700 1 -- 172.21.15.144:0/641329467 shutdown_connections 2024-09-10T11:48:15.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.066+0000 7f63924aa700 1 -- 172.21.15.144:0/641329467 wait complete. 2024-09-10T11:48:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:14 smithi178 ceph-mon[65940]: pgmap v487: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:15.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.171+0000 7f6c6450b700 1 -- 172.21.15.144:0/3530884552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c0ffc30 msgr2=0x7f6c5c10a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:15.172 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.171+0000 7f6c6450b700 1 --2- 172.21.15.144:0/3530884552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c0ffc30 0x7f6c5c10a6a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009910 tx=0x7f6c58009c20 comp rx=0 tx=0).stop 2024-09-10T11:48:15.172 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:14 smithi144 ceph-mon[87436]: pgmap v487: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:15.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.171+0000 7f6c6450b700 1 -- 172.21.15.144:0/3530884552 shutdown_connections 2024-09-10T11:48:15.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.171+0000 7f6c6450b700 1 --2- 172.21.15.144:0/3530884552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c0ffc30 0x7f6c5c10a6a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.171+0000 7f6c6450b700 1 --2- 172.21.15.144:0/3530884552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c5c0ff320 0x7f6c5c0ff6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.171+0000 7f6c6450b700 1 -- 172.21.15.144:0/3530884552 >> 172.21.15.144:0/3530884552 conn(0x7f6c5c0748b0 msgr2=0x7f6c5c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:15.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.171+0000 7f6c6450b700 1 -- 172.21.15.144:0/3530884552 shutdown_connections 2024-09-10T11:48:15.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.171+0000 7f6c6450b700 1 -- 172.21.15.144:0/3530884552 wait complete. 2024-09-10T11:48:15.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.172+0000 7f6c6450b700 1 Processor -- start 2024-09-10T11:48:15.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.172+0000 7f6c6450b700 1 -- start start 2024-09-10T11:48:15.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.172+0000 7f6c6450b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c5c0ff320 0x7f6c5c19ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:15.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.172+0000 7f6c6450b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c0ffc30 0x7f6c5c19bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:15.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.172+0000 7f6c6450b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c19c5f0 con 0x7f6c5c0ffc30 2024-09-10T11:48:15.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.172+0000 7f6c6450b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c195aa0 con 0x7f6c5c0ff320 2024-09-10T11:48:15.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.172+0000 7f6c61aa6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c0ffc30 0x7f6c5c19bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:15.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.172+0000 7f6c61aa6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c0ffc30 0x7f6c5c19bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42206/0 (socket says 172.21.15.144:42206) 2024-09-10T11:48:15.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.172+0000 7f6c61aa6700 1 -- 172.21.15.144:0/506305799 learned_addr learned my addr 172.21.15.144:0/506305799 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:48:15.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.172+0000 7f6c622a7700 1 --2- 172.21.15.144:0/506305799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c5c0ff320 0x7f6c5c19ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:15.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.173+0000 7f6c61aa6700 1 -- 172.21.15.144:0/506305799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c5c0ff320 msgr2=0x7f6c5c19ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:15.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.173+0000 7f6c61aa6700 1 --2- 172.21.15.144:0/506305799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c5c0ff320 0x7f6c5c19ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.173+0000 7f6c61aa6700 1 -- 172.21.15.144:0/506305799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c580095f0 con 0x7f6c5c0ffc30 2024-09-10T11:48:15.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.173+0000 7f6c61aa6700 1 --2- 172.21.15.144:0/506305799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c0ffc30 0x7f6c5c19bf60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6c58006010 tx=0x7f6c5800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:15.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.173+0000 7f6c537fe700 1 -- 172.21.15.144:0/506305799 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c5801d070 con 0x7f6c5c0ffc30 2024-09-10T11:48:15.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.173+0000 7f6c6450b700 1 -- 172.21.15.144:0/506305799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5c195d80 con 0x7f6c5c0ffc30 2024-09-10T11:48:15.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.173+0000 7f6c6450b700 1 -- 172.21.15.144:0/506305799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5c196350 con 0x7f6c5c0ffc30 2024-09-10T11:48:15.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.173+0000 7f6c537fe700 1 -- 172.21.15.144:0/506305799 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6c5800f460 con 0x7f6c5c0ffc30 2024-09-10T11:48:15.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.173+0000 7f6c537fe700 1 -- 172.21.15.144:0/506305799 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c58005de0 con 0x7f6c5c0ffc30 2024-09-10T11:48:15.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.174+0000 7f6c6450b700 1 -- 172.21.15.144:0/506305799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c40005320 con 0x7f6c5c0ffc30 2024-09-10T11:48:15.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.175+0000 7f6c537fe700 1 -- 172.21.15.144:0/506305799 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6c58003a60 con 0x7f6c5c0ffc30 2024-09-10T11:48:15.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.176+0000 7f6c537fe700 1 --2- 172.21.15.144:0/506305799 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6c4806e200 0x7f6c480706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:15.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.176+0000 7f6c537fe700 1 -- 172.21.15.144:0/506305799 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6c5808f5f0 con 0x7f6c5c0ffc30 2024-09-10T11:48:15.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.176+0000 7f6c622a7700 1 --2- 172.21.15.144:0/506305799 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6c4806e200 0x7f6c480706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:15.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.176+0000 7f6c622a7700 1 --2- 172.21.15.144:0/506305799 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6c4806e200 0x7f6c480706b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c005950 tx=0x7f6c4c00a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:15.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.179+0000 7f6c537fe700 1 -- 172.21.15.144:0/506305799 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6c5805c4e0 con 0x7f6c5c0ffc30 2024-09-10T11:48:15.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.414+0000 7f6c6450b700 1 -- 172.21.15.144:0/506305799 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c40000bf0 con 0x7f6c4806e200 2024-09-10T11:48:15.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.424+0000 7f6c537fe700 1 -- 172.21.15.144:0/506305799 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f6c40000bf0 con 0x7f6c4806e200 2024-09-10T11:48:15.501 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:48:15.501 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (16m) 10m ago 26m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:48:15.501 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (27m) 10m ago 27m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:48:15.501 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (26m) 9m ago 26m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:48:15.502 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (14m) 10m ago 27m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:48:15.502 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (14m) 9m ago 26m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:48:15.502 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (16m) 10m ago 26m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:48:15.502 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (10m) 10m ago 21m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:48:15.502 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (10m) 10m ago 21m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:48:15.502 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (21m) 9m ago 21m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:48:15.502 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (10m) 9m ago 21m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:48:15.502 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (18m) 10m ago 28m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:48:15.502 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (17m) 9m ago 26m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:48:15.502 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (15m) 10m ago 28m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:48:15.502 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (14m) 9m ago 25m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:48:15.502 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (17m) 10m ago 27m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:48:15.502 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (17m) 9m ago 26m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:48:15.503 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (13m) 10m ago 25m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:48:15.503 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (13m) 10m ago 24m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:48:15.503 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (12m) 10m ago 24m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:48:15.503 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (11m) 9m ago 23m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:48:15.503 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (11m) 9m ago 23m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:48:15.503 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (10m) 9m ago 23m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:48:15.503 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (16m) 10m ago 26m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:48:15.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.426+0000 7f6c6450b700 1 -- 172.21.15.144:0/506305799 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6c4806e200 msgr2=0x7f6c480706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:15.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.426+0000 7f6c6450b700 1 --2- 172.21.15.144:0/506305799 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6c4806e200 0x7f6c480706b0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c005950 tx=0x7f6c4c00a300 comp rx=0 tx=0).stop 2024-09-10T11:48:15.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.426+0000 7f6c6450b700 1 -- 172.21.15.144:0/506305799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c0ffc30 msgr2=0x7f6c5c19bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:15.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.426+0000 7f6c6450b700 1 --2- 172.21.15.144:0/506305799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c0ffc30 0x7f6c5c19bf60 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f6c58006010 tx=0x7f6c5800bd70 comp rx=0 tx=0).stop 2024-09-10T11:48:15.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.426+0000 7f6c6450b700 1 -- 172.21.15.144:0/506305799 shutdown_connections 2024-09-10T11:48:15.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.426+0000 7f6c6450b700 1 --2- 172.21.15.144:0/506305799 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6c4806e200 0x7f6c480706b0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.426+0000 7f6c6450b700 1 --2- 172.21.15.144:0/506305799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c5c0ff320 0x7f6c5c19ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.426+0000 7f6c6450b700 1 --2- 172.21.15.144:0/506305799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c5c0ffc30 0x7f6c5c19bf60 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.426+0000 7f6c6450b700 1 -- 172.21.15.144:0/506305799 >> 172.21.15.144:0/506305799 conn(0x7f6c5c0748b0 msgr2=0x7f6c5c104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:15.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.426+0000 7f6c6450b700 1 -- 172.21.15.144:0/506305799 shutdown_connections 2024-09-10T11:48:15.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.426+0000 7f6c6450b700 1 -- 172.21.15.144:0/506305799 wait complete. 2024-09-10T11:48:15.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.529+0000 7f591ddfe700 1 -- 172.21.15.144:0/4268398668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59180ff2c0 msgr2=0x7f59180ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:15.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.529+0000 7f591ddfe700 1 --2- 172.21.15.144:0/4268398668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59180ff2c0 0x7f59180ff690 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5908009910 tx=0x7f5908009c20 comp rx=0 tx=0).stop 2024-09-10T11:48:15.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.529+0000 7f591ddfe700 1 -- 172.21.15.144:0/4268398668 shutdown_connections 2024-09-10T11:48:15.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.529+0000 7f591ddfe700 1 --2- 172.21.15.144:0/4268398668 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59180ffc60 0x7f5918103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.529+0000 7f591ddfe700 1 --2- 172.21.15.144:0/4268398668 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59180ff2c0 0x7f59180ff690 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.529+0000 7f591ddfe700 1 -- 172.21.15.144:0/4268398668 >> 172.21.15.144:0/4268398668 conn(0x7f59180fab80 msgr2=0x7f59180fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:15.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.529+0000 7f591ddfe700 1 -- 172.21.15.144:0/4268398668 shutdown_connections 2024-09-10T11:48:15.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.529+0000 7f591ddfe700 1 -- 172.21.15.144:0/4268398668 wait complete. 2024-09-10T11:48:15.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.530+0000 7f591ddfe700 1 Processor -- start 2024-09-10T11:48:15.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.530+0000 7f591ddfe700 1 -- start start 2024-09-10T11:48:15.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.530+0000 7f591ddfe700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59180ff2c0 0x7f591819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:15.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.530+0000 7f591ddfe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59180ffc60 0x7f591819af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:15.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.530+0000 7f591ddfe700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591819b5c0 con 0x7f59180ffc60 2024-09-10T11:48:15.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.530+0000 7f591ddfe700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5918194a70 con 0x7f59180ff2c0 2024-09-10T11:48:15.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.530+0000 7f590effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59180ffc60 0x7f591819af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:15.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.530+0000 7f590effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59180ffc60 0x7f591819af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42228/0 (socket says 172.21.15.144:42228) 2024-09-10T11:48:15.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.530+0000 7f590effd700 1 -- 172.21.15.144:0/3224990897 learned_addr learned my addr 172.21.15.144:0/3224990897 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:48:15.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.531+0000 7f59177fe700 1 --2- 172.21.15.144:0/3224990897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59180ff2c0 0x7f591819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:15.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.531+0000 7f590effd700 1 -- 172.21.15.144:0/3224990897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59180ff2c0 msgr2=0x7f591819a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:15.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.531+0000 7f590effd700 1 --2- 172.21.15.144:0/3224990897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59180ff2c0 0x7f591819a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.531+0000 7f590effd700 1 -- 172.21.15.144:0/3224990897 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59080095f0 con 0x7f59180ffc60 2024-09-10T11:48:15.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.531+0000 7f590effd700 1 --2- 172.21.15.144:0/3224990897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59180ffc60 0x7f591819af30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f590000d900 tx=0x7f590000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:15.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.531+0000 7f59157fa700 1 -- 172.21.15.144:0/3224990897 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5900009900 con 0x7f59180ffc60 2024-09-10T11:48:15.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.531+0000 7f59157fa700 1 -- 172.21.15.144:0/3224990897 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5900010460 con 0x7f59180ffc60 2024-09-10T11:48:15.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.531+0000 7f59157fa700 1 -- 172.21.15.144:0/3224990897 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f590000f5f0 con 0x7f59180ffc60 2024-09-10T11:48:15.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.531+0000 7f591ddfe700 1 -- 172.21.15.144:0/3224990897 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5918194d50 con 0x7f59180ffc60 2024-09-10T11:48:15.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.531+0000 7f591ddfe700 1 -- 172.21.15.144:0/3224990897 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59181952a0 con 0x7f59180ffc60 2024-09-10T11:48:15.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.532+0000 7f591ddfe700 1 -- 172.21.15.144:0/3224990897 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591804ea50 con 0x7f59180ffc60 2024-09-10T11:48:15.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.533+0000 7f59157fa700 1 -- 172.21.15.144:0/3224990897 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f59000105d0 con 0x7f59180ffc60 2024-09-10T11:48:15.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.533+0000 7f59157fa700 1 --2- 172.21.15.144:0/3224990897 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f590406e200 0x7f59040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:15.535 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.533+0000 7f59157fa700 1 -- 172.21.15.144:0/3224990897 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f590008df20 con 0x7f59180ffc60 2024-09-10T11:48:15.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.533+0000 7f59177fe700 1 --2- 172.21.15.144:0/3224990897 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f590406e200 0x7f59040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:15.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.534+0000 7f59177fe700 1 --2- 172.21.15.144:0/3224990897 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f590406e200 0x7f59040706b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f590800b5c0 tx=0x7f590801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:15.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.537+0000 7f59157fa700 1 -- 172.21.15.144:0/3224990897 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f59000c31b0 con 0x7f59180ffc60 2024-09-10T11:48:15.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.826+0000 7f591ddfe700 1 -- 172.21.15.144:0/3224990897 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5918195f80 con 0x7f59180ffc60 2024-09-10T11:48:15.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.827+0000 7f59157fa700 1 -- 172.21.15.144:0/3224990897 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f590005ae30 con 0x7f59180ffc60 2024-09-10T11:48:15.828 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:48:15.828 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:48:15.828 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:48:15.828 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:48:15.829 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:48:15.829 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:48:15.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:48:15.829 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:48:15.829 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:48:15.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:48:15.829 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:48:15.829 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:48:15.829 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:48:15.829 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:48:15.829 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:48:15.829 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:48:15.829 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:48:15.829 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:48:15.830 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:48:15.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.830+0000 7f591ddfe700 1 -- 172.21.15.144:0/3224990897 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f590406e200 msgr2=0x7f59040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:15.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.830+0000 7f591ddfe700 1 --2- 172.21.15.144:0/3224990897 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f590406e200 0x7f59040706b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f590800b5c0 tx=0x7f590801a040 comp rx=0 tx=0).stop 2024-09-10T11:48:15.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.830+0000 7f591ddfe700 1 -- 172.21.15.144:0/3224990897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59180ffc60 msgr2=0x7f591819af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:15.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.830+0000 7f591ddfe700 1 --2- 172.21.15.144:0/3224990897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59180ffc60 0x7f591819af30 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f590000d900 tx=0x7f590000dcc0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.830+0000 7f591ddfe700 1 -- 172.21.15.144:0/3224990897 shutdown_connections 2024-09-10T11:48:15.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.830+0000 7f591ddfe700 1 --2- 172.21.15.144:0/3224990897 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f590406e200 0x7f59040706b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.830+0000 7f591ddfe700 1 --2- 172.21.15.144:0/3224990897 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f59180ff2c0 0x7f591819a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.830+0000 7f591ddfe700 1 --2- 172.21.15.144:0/3224990897 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f59180ffc60 0x7f591819af30 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.830+0000 7f591ddfe700 1 -- 172.21.15.144:0/3224990897 >> 172.21.15.144:0/3224990897 conn(0x7f59180fab80 msgr2=0x7f59180fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:15.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.830+0000 7f591ddfe700 1 -- 172.21.15.144:0/3224990897 shutdown_connections 2024-09-10T11:48:15.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.830+0000 7f591ddfe700 1 -- 172.21.15.144:0/3224990897 wait complete. 2024-09-10T11:48:15.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.935+0000 7f4ea769f700 1 -- 172.21.15.144:0/3031069101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00ffd30 msgr2=0x7f4ea0103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:15.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.935+0000 7f4ea769f700 1 --2- 172.21.15.144:0/3031069101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00ffd30 0x7f4ea0103ce0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009910 tx=0x7f4e9c009c20 comp rx=0 tx=0).stop 2024-09-10T11:48:15.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.935+0000 7f4ea769f700 1 -- 172.21.15.144:0/3031069101 shutdown_connections 2024-09-10T11:48:15.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.935+0000 7f4ea769f700 1 --2- 172.21.15.144:0/3031069101 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00ffd30 0x7f4ea0103ce0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.935+0000 7f4ea769f700 1 --2- 172.21.15.144:0/3031069101 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea00ff390 0x7f4ea00ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.935+0000 7f4ea769f700 1 -- 172.21.15.144:0/3031069101 >> 172.21.15.144:0/3031069101 conn(0x7f4ea0074f50 msgr2=0x7f4ea0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:15.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.935+0000 7f4ea769f700 1 -- 172.21.15.144:0/3031069101 shutdown_connections 2024-09-10T11:48:15.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.935+0000 7f4ea769f700 1 -- 172.21.15.144:0/3031069101 wait complete. 2024-09-10T11:48:15.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.936+0000 7f4ea769f700 1 Processor -- start 2024-09-10T11:48:15.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.936+0000 7f4ea769f700 1 -- start start 2024-09-10T11:48:15.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.936+0000 7f4ea769f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea00ff390 0x7f4ea0195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:15.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.936+0000 7f4ea769f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00ffd30 0x7f4ea0196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:15.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.936+0000 7f4ea769f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea0196a60 con 0x7f4ea00ffd30 2024-09-10T11:48:15.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.936+0000 7f4ea769f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea019a7f0 con 0x7f4ea00ff390 2024-09-10T11:48:15.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.936+0000 7f4ea4c3a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00ffd30 0x7f4ea0196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:15.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.936+0000 7f4ea4c3a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00ffd30 0x7f4ea0196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42248/0 (socket says 172.21.15.144:42248) 2024-09-10T11:48:15.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.936+0000 7f4ea4c3a700 1 -- 172.21.15.144:0/2936205991 learned_addr learned my addr 172.21.15.144:0/2936205991 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:48:15.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.936+0000 7f4ea4c3a700 1 -- 172.21.15.144:0/2936205991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea00ff390 msgr2=0x7f4ea0195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:15.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.936+0000 7f4ea543b700 1 --2- 172.21.15.144:0/2936205991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea00ff390 0x7f4ea0195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:15.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.936+0000 7f4ea4c3a700 1 --2- 172.21.15.144:0/2936205991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea00ff390 0x7f4ea0195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:15.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.936+0000 7f4ea4c3a700 1 -- 172.21.15.144:0/2936205991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c0095f0 con 0x7f4ea00ffd30 2024-09-10T11:48:15.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.936+0000 7f4ea543b700 1 --2- 172.21.15.144:0/2936205991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea00ff390 0x7f4ea0195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:48:15.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.936+0000 7f4ea4c3a700 1 --2- 172.21.15.144:0/2936205991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00ffd30 0x7f4ea0196380 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00bda0 tx=0x7f4e9c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:15.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.937+0000 7f4e967fc700 1 -- 172.21.15.144:0/2936205991 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e9c01d070 con 0x7f4ea00ffd30 2024-09-10T11:48:15.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.937+0000 7f4e967fc700 1 -- 172.21.15.144:0/2936205991 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4e9c004fb0 con 0x7f4ea00ffd30 2024-09-10T11:48:15.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.937+0000 7f4ea769f700 1 -- 172.21.15.144:0/2936205991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ea019aa70 con 0x7f4ea00ffd30 2024-09-10T11:48:15.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.937+0000 7f4e967fc700 1 -- 172.21.15.144:0/2936205991 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e9c00f480 con 0x7f4ea00ffd30 2024-09-10T11:48:15.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.937+0000 7f4ea769f700 1 -- 172.21.15.144:0/2936205991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ea019af60 con 0x7f4ea00ffd30 2024-09-10T11:48:15.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.938+0000 7f4ea769f700 1 -- 172.21.15.144:0/2936205991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ea004ea50 con 0x7f4ea00ffd30 2024-09-10T11:48:15.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.938+0000 7f4e967fc700 1 -- 172.21.15.144:0/2936205991 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4e9c022870 con 0x7f4ea00ffd30 2024-09-10T11:48:15.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.938+0000 7f4e967fc700 1 --2- 172.21.15.144:0/2936205991 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4e8c06e200 0x7f4e8c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:15.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.939+0000 7f4e967fc700 1 -- 172.21.15.144:0/2936205991 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4e9c08f500 con 0x7f4ea00ffd30 2024-09-10T11:48:15.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.939+0000 7f4ea543b700 1 --2- 172.21.15.144:0/2936205991 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4e8c06e200 0x7f4e8c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:15.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.939+0000 7f4ea543b700 1 --2- 172.21.15.144:0/2936205991 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4e8c06e200 0x7f4e8c0706b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4e90007900 tx=0x7f4e90008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:15.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:15.943+0000 7f4e967fc700 1 -- 172.21.15.144:0/2936205991 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4e9c058a10 con 0x7f4ea00ffd30 2024-09-10T11:48:16.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:15 smithi178 ceph-mon[65940]: from='client.34932 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:16.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:15 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3224990897' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:48:16.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:15 smithi144 ceph-mon[87436]: from='client.34932 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:16.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:15 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3224990897' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:48:16.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.190+0000 7f4ea769f700 1 -- 172.21.15.144:0/2936205991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4ea019b360 con 0x7f4ea00ffd30 2024-09-10T11:48:16.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.190+0000 7f4e967fc700 1 -- 172.21.15.144:0/2936205991 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f4e9c05c0d0 con 0x7f4ea00ffd30 2024-09-10T11:48:16.191 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:48:16.192 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:48:16.192 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:48:16.192 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:48:16.192 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:48:16.192 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:48:16.192 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:48:16.192 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:48:16.192 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:48:16.192 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:48:16.192 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:48:16.193 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:48:16.193 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:48:16.193 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:48:16.193 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:48:16.193 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:48:16.193 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:48:16.193 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:48:16.193 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:48:16.193 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:48:16.193 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:48:16.193 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:48:16.193 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:48:16.193 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:48:16.194 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:48:16.194 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:48:16.194 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:48:16.194 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:48:16.194 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:48:16.194 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:48:16.194 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:48:16.194 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:48:16.194 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:48:16.194 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:48:16.194 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:48:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:48:16.194 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:48:16.195 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:48:16.195 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:48:16.195 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:48:16.195 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:48:16.195 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:48:16.195 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:48:16.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.192+0000 7f4ea769f700 1 -- 172.21.15.144:0/2936205991 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4e8c06e200 msgr2=0x7f4e8c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:16.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.192+0000 7f4ea769f700 1 --2- 172.21.15.144:0/2936205991 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4e8c06e200 0x7f4e8c0706b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4e90007900 tx=0x7f4e90008040 comp rx=0 tx=0).stop 2024-09-10T11:48:16.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.192+0000 7f4ea769f700 1 -- 172.21.15.144:0/2936205991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00ffd30 msgr2=0x7f4ea0196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:16.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.192+0000 7f4ea769f700 1 --2- 172.21.15.144:0/2936205991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00ffd30 0x7f4ea0196380 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c00bda0 tx=0x7f4e9c00be80 comp rx=0 tx=0).stop 2024-09-10T11:48:16.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.193+0000 7f4ea769f700 1 -- 172.21.15.144:0/2936205991 shutdown_connections 2024-09-10T11:48:16.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.193+0000 7f4ea769f700 1 --2- 172.21.15.144:0/2936205991 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4e8c06e200 0x7f4e8c0706b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.193+0000 7f4ea769f700 1 --2- 172.21.15.144:0/2936205991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ea00ff390 0x7f4ea0195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.193+0000 7f4ea769f700 1 --2- 172.21.15.144:0/2936205991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ea00ffd30 0x7f4ea0196380 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.193+0000 7f4ea769f700 1 -- 172.21.15.144:0/2936205991 >> 172.21.15.144:0/2936205991 conn(0x7f4ea0074f50 msgr2=0x7f4ea0103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:16.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.193+0000 7f4ea769f700 1 -- 172.21.15.144:0/2936205991 shutdown_connections 2024-09-10T11:48:16.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.193+0000 7f4ea769f700 1 -- 172.21.15.144:0/2936205991 wait complete. 2024-09-10T11:48:16.196 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:48:16.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.297+0000 7f94e7504700 1 -- 172.21.15.144:0/4161634611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e0100d70 msgr2=0x7f94e0104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:16.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.297+0000 7f94e7504700 1 --2- 172.21.15.144:0/4161634611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e0100d70 0x7f94e0104d40 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f94d0009910 tx=0x7f94d0009c20 comp rx=0 tx=0).stop 2024-09-10T11:48:16.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.297+0000 7f94e7504700 1 -- 172.21.15.144:0/4161634611 shutdown_connections 2024-09-10T11:48:16.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.298+0000 7f94e7504700 1 --2- 172.21.15.144:0/4161634611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e0100d70 0x7f94e0104d40 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.298+0000 7f94e7504700 1 --2- 172.21.15.144:0/4161634611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e01003d0 0x7f94e01007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.298+0000 7f94e7504700 1 -- 172.21.15.144:0/4161634611 >> 172.21.15.144:0/4161634611 conn(0x7f94e00fbc90 msgr2=0x7f94e00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:16.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.298+0000 7f94e7504700 1 -- 172.21.15.144:0/4161634611 shutdown_connections 2024-09-10T11:48:16.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.298+0000 7f94e7504700 1 -- 172.21.15.144:0/4161634611 wait complete. 2024-09-10T11:48:16.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.298+0000 7f94e7504700 1 Processor -- start 2024-09-10T11:48:16.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.298+0000 7f94e7504700 1 -- start start 2024-09-10T11:48:16.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.298+0000 7f94e7504700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e01003d0 0x7f94e0195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:16.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.298+0000 7f94e7504700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e0100d70 0x7f94e0196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:16.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.298+0000 7f94e7504700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e0196960 con 0x7f94e0100d70 2024-09-10T11:48:16.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.298+0000 7f94e7504700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e019a6f0 con 0x7f94e01003d0 2024-09-10T11:48:16.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.299+0000 7f94e4a9f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e0100d70 0x7f94e0196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:16.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.299+0000 7f94e4a9f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e0100d70 0x7f94e0196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42260/0 (socket says 172.21.15.144:42260) 2024-09-10T11:48:16.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.299+0000 7f94e4a9f700 1 -- 172.21.15.144:0/1794954468 learned_addr learned my addr 172.21.15.144:0/1794954468 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:48:16.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.299+0000 7f94e52a0700 1 --2- 172.21.15.144:0/1794954468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e01003d0 0x7f94e0195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:16.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.299+0000 7f94e4a9f700 1 -- 172.21.15.144:0/1794954468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e01003d0 msgr2=0x7f94e0195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:16.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.299+0000 7f94e4a9f700 1 --2- 172.21.15.144:0/1794954468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e01003d0 0x7f94e0195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.299+0000 7f94e4a9f700 1 -- 172.21.15.144:0/1794954468 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d00095f0 con 0x7f94e0100d70 2024-09-10T11:48:16.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.299+0000 7f94e4a9f700 1 --2- 172.21.15.144:0/1794954468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e0100d70 0x7f94e0196280 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f94d000bda0 tx=0x7f94d000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:16.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.299+0000 7f94d67fc700 1 -- 172.21.15.144:0/1794954468 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94d001d070 con 0x7f94e0100d70 2024-09-10T11:48:16.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.299+0000 7f94d67fc700 1 -- 172.21.15.144:0/1794954468 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94d000f460 con 0x7f94e0100d70 2024-09-10T11:48:16.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.299+0000 7f94e7504700 1 -- 172.21.15.144:0/1794954468 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e019a970 con 0x7f94e0100d70 2024-09-10T11:48:16.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.299+0000 7f94d67fc700 1 -- 172.21.15.144:0/1794954468 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94d0017610 con 0x7f94e0100d70 2024-09-10T11:48:16.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.299+0000 7f94e7504700 1 -- 172.21.15.144:0/1794954468 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e019ae60 con 0x7f94e0100d70 2024-09-10T11:48:16.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.300+0000 7f94e7504700 1 -- 172.21.15.144:0/1794954468 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e0108740 con 0x7f94e0100d70 2024-09-10T11:48:16.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.301+0000 7f94d67fc700 1 -- 172.21.15.144:0/1794954468 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f94d000fa40 con 0x7f94e0100d70 2024-09-10T11:48:16.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.301+0000 7f94d67fc700 1 --2- 172.21.15.144:0/1794954468 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f94cc06e200 0x7f94cc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:16.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.301+0000 7f94d67fc700 1 -- 172.21.15.144:0/1794954468 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f94d008f520 con 0x7f94e0100d70 2024-09-10T11:48:16.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.301+0000 7f94e52a0700 1 --2- 172.21.15.144:0/1794954468 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f94cc06e200 0x7f94cc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:16.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.302+0000 7f94e52a0700 1 --2- 172.21.15.144:0/1794954468 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f94cc06e200 0x7f94cc0706b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f94dc005950 tx=0x7f94dc0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:16.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.306+0000 7f94d67fc700 1 -- 172.21.15.144:0/1794954468 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f94d0058900 con 0x7f94e0100d70 2024-09-10T11:48:16.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.525+0000 7f94e7504700 1 -- 172.21.15.144:0/1794954468 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f94e002cd20 con 0x7f94cc06e200 2024-09-10T11:48:16.528 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.527+0000 7f94d67fc700 1 -- 172.21.15.144:0/1794954468 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f94e002cd20 con 0x7f94cc06e200 2024-09-10T11:48:16.528 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:48:16.528 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:48:16.528 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:48:16.528 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:48:16.528 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:48:16.528 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:48:16.528 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:48:16.528 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:48:16.529 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:48:16.529 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:48:16.529 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:48:16.529 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:48:16.529 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:48:16.529 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:48:16.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.529+0000 7f94e7504700 1 -- 172.21.15.144:0/1794954468 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f94cc06e200 msgr2=0x7f94cc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:16.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.529+0000 7f94e7504700 1 --2- 172.21.15.144:0/1794954468 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f94cc06e200 0x7f94cc0706b0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f94dc005950 tx=0x7f94dc0058e0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.529+0000 7f94e7504700 1 -- 172.21.15.144:0/1794954468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e0100d70 msgr2=0x7f94e0196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:16.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.529+0000 7f94e7504700 1 --2- 172.21.15.144:0/1794954468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e0100d70 0x7f94e0196280 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f94d000bda0 tx=0x7f94d000be80 comp rx=0 tx=0).stop 2024-09-10T11:48:16.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.529+0000 7f94e7504700 1 -- 172.21.15.144:0/1794954468 shutdown_connections 2024-09-10T11:48:16.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.529+0000 7f94e7504700 1 --2- 172.21.15.144:0/1794954468 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f94cc06e200 0x7f94cc0706b0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.529+0000 7f94e7504700 1 --2- 172.21.15.144:0/1794954468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f94e01003d0 0x7f94e0195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.529+0000 7f94e7504700 1 --2- 172.21.15.144:0/1794954468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f94e0100d70 0x7f94e0196280 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.529+0000 7f94e7504700 1 -- 172.21.15.144:0/1794954468 >> 172.21.15.144:0/1794954468 conn(0x7f94e00fbc90 msgr2=0x7f94e00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:16.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.529+0000 7f94e7504700 1 -- 172.21.15.144:0/1794954468 shutdown_connections 2024-09-10T11:48:16.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.529+0000 7f94e7504700 1 -- 172.21.15.144:0/1794954468 wait complete. 2024-09-10T11:48:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.634+0000 7f491203c700 1 -- 172.21.15.144:0/470597841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c0ffd20 msgr2=0x7f490c103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.634+0000 7f491203c700 1 --2- 172.21.15.144:0/470597841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c0ffd20 0x7f490c103cf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f48f4009910 tx=0x7f48f4009c20 comp rx=0 tx=0).stop 2024-09-10T11:48:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.634+0000 7f491203c700 1 -- 172.21.15.144:0/470597841 shutdown_connections 2024-09-10T11:48:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.634+0000 7f491203c700 1 --2- 172.21.15.144:0/470597841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c0ffd20 0x7f490c103cf0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.634+0000 7f491203c700 1 --2- 172.21.15.144:0/470597841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f490c0ff380 0x7f490c0ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.634+0000 7f491203c700 1 -- 172.21.15.144:0/470597841 >> 172.21.15.144:0/470597841 conn(0x7f490c0fac20 msgr2=0x7f490c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:16.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.635+0000 7f491203c700 1 -- 172.21.15.144:0/470597841 shutdown_connections 2024-09-10T11:48:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.635+0000 7f491203c700 1 -- 172.21.15.144:0/470597841 wait complete. 2024-09-10T11:48:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.635+0000 7f491203c700 1 Processor -- start 2024-09-10T11:48:16.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.635+0000 7f491203c700 1 -- start start 2024-09-10T11:48:16.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.635+0000 7f491203c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f490c0ff380 0x7f490c196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:16.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.635+0000 7f491203c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c0ffd20 0x7f490c197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:16.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.635+0000 7f491203c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c197720 con 0x7f490c0ffd20 2024-09-10T11:48:16.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.635+0000 7f491203c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c197860 con 0x7f490c0ff380 2024-09-10T11:48:16.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.636+0000 7f4903fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c0ffd20 0x7f490c197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:16.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.636+0000 7f4903fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c0ffd20 0x7f490c197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42284/0 (socket says 172.21.15.144:42284) 2024-09-10T11:48:16.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.636+0000 7f4903fff700 1 -- 172.21.15.144:0/2626286472 learned_addr learned my addr 172.21.15.144:0/2626286472 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:48:16.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.636+0000 7f490b7fe700 1 --2- 172.21.15.144:0/2626286472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f490c0ff380 0x7f490c196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:16.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.636+0000 7f4903fff700 1 -- 172.21.15.144:0/2626286472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f490c0ff380 msgr2=0x7f490c196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:16.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.636+0000 7f4903fff700 1 --2- 172.21.15.144:0/2626286472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f490c0ff380 0x7f490c196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.636+0000 7f4903fff700 1 -- 172.21.15.144:0/2626286472 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f40095f0 con 0x7f490c0ffd20 2024-09-10T11:48:16.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.636+0000 7f4903fff700 1 --2- 172.21.15.144:0/2626286472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c0ffd20 0x7f490c197190 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f48f400bda0 tx=0x7f48f400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:16.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.636+0000 7f49097fa700 1 -- 172.21.15.144:0/2626286472 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48f401d070 con 0x7f490c0ffd20 2024-09-10T11:48:16.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.636+0000 7f49097fa700 1 -- 172.21.15.144:0/2626286472 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48f400f460 con 0x7f490c0ffd20 2024-09-10T11:48:16.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.636+0000 7f491203c700 1 -- 172.21.15.144:0/2626286472 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c19fcc0 con 0x7f490c0ffd20 2024-09-10T11:48:16.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.636+0000 7f49097fa700 1 -- 172.21.15.144:0/2626286472 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48f4017630 con 0x7f490c0ffd20 2024-09-10T11:48:16.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.636+0000 7f491203c700 1 -- 172.21.15.144:0/2626286472 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c1a0070 con 0x7f490c0ffd20 2024-09-10T11:48:16.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.637+0000 7f491203c700 1 -- 172.21.15.144:0/2626286472 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c1076f0 con 0x7f490c0ffd20 2024-09-10T11:48:16.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.638+0000 7f49097fa700 1 -- 172.21.15.144:0/2626286472 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f48f400fab0 con 0x7f490c0ffd20 2024-09-10T11:48:16.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.638+0000 7f49097fa700 1 --2- 172.21.15.144:0/2626286472 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f48f806e200 0x7f48f80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:16.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.638+0000 7f49097fa700 1 -- 172.21.15.144:0/2626286472 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f48f408f4a0 con 0x7f490c0ffd20 2024-09-10T11:48:16.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.638+0000 7f490b7fe700 1 --2- 172.21.15.144:0/2626286472 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f48f806e200 0x7f48f80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:16.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.639+0000 7f490b7fe700 1 --2- 172.21.15.144:0/2626286472 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f48f806e200 0x7f48f80706b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00ac50 tx=0x7f48fc00a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:16.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.643+0000 7f49097fa700 1 -- 172.21.15.144:0/2626286472 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f48f4058900 con 0x7f490c0ffd20 2024-09-10T11:48:16.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.935+0000 7f491203c700 1 -- 172.21.15.144:0/2626286472 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f490c02cc30 con 0x7f490c0ffd20 2024-09-10T11:48:16.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.935+0000 7f49097fa700 1 -- 172.21.15.144:0/2626286472 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f48f405bfc0 con 0x7f490c0ffd20 2024-09-10T11:48:16.937 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:48:16.937 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:48:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:48:16.937 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:48:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:48:16.937 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:48:16.937 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:48:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.938+0000 7f491203c700 1 -- 172.21.15.144:0/2626286472 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f48f806e200 msgr2=0x7f48f80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.938+0000 7f491203c700 1 --2- 172.21.15.144:0/2626286472 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f48f806e200 0x7f48f80706b0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f48fc00ac50 tx=0x7f48fc00a380 comp rx=0 tx=0).stop 2024-09-10T11:48:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.938+0000 7f491203c700 1 -- 172.21.15.144:0/2626286472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c0ffd20 msgr2=0x7f490c197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.938+0000 7f491203c700 1 --2- 172.21.15.144:0/2626286472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c0ffd20 0x7f490c197190 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f48f400bda0 tx=0x7f48f400bdd0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.938+0000 7f491203c700 1 -- 172.21.15.144:0/2626286472 shutdown_connections 2024-09-10T11:48:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.938+0000 7f491203c700 1 --2- 172.21.15.144:0/2626286472 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f48f806e200 0x7f48f80706b0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.938+0000 7f491203c700 1 --2- 172.21.15.144:0/2626286472 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f490c0ff380 0x7f490c196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.938+0000 7f491203c700 1 --2- 172.21.15.144:0/2626286472 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f490c0ffd20 0x7f490c197190 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.938+0000 7f491203c700 1 -- 172.21.15.144:0/2626286472 >> 172.21.15.144:0/2626286472 conn(0x7f490c0fac20 msgr2=0x7f490c0fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.938+0000 7f491203c700 1 -- 172.21.15.144:0/2626286472 shutdown_connections 2024-09-10T11:48:16.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:16.938+0000 7f491203c700 1 -- 172.21.15.144:0/2626286472 wait complete. 2024-09-10T11:48:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:16 smithi178 ceph-mon[65940]: from='client.34936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:16 smithi178 ceph-mon[65940]: from='client.34940 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:16 smithi178 ceph-mon[65940]: pgmap v488: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:16 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2936205991' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:48:17.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:16 smithi144 ceph-mon[87436]: from='client.34936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:17.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:16 smithi144 ceph-mon[87436]: from='client.34940 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:17.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:16 smithi144 ceph-mon[87436]: pgmap v488: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:17.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:16 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2936205991' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:48:18.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:17 smithi178 ceph-mon[65940]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:17 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2626286472' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:48:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:17 smithi144 ceph-mon[87436]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:17 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2626286472' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:48:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:18 smithi178 ceph-mon[65940]: pgmap v489: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:19.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:18 smithi144 ceph-mon[87436]: pgmap v489: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:20.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:19 smithi178 ceph-mon[65940]: pgmap v490: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:20.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:19 smithi144 ceph-mon[87436]: pgmap v490: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:23.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:22 smithi144 ceph-mon[87436]: pgmap v491: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:23.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:22 smithi178 ceph-mon[65940]: pgmap v491: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:24 smithi178 ceph-mon[65940]: pgmap v492: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:24 smithi144 ceph-mon[87436]: pgmap v492: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:26 smithi178 ceph-mon[65940]: pgmap v493: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:26 smithi144 ceph-mon[87436]: pgmap v493: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:28 smithi178 ceph-mon[65940]: pgmap v494: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:48:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:28 smithi144 ceph-mon[87436]: pgmap v494: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:48:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:30 smithi178 ceph-mon[65940]: pgmap v495: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:31.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:30 smithi144 ceph-mon[87436]: pgmap v495: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:32 smithi144 ceph-mon[87436]: pgmap v496: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:33.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:32 smithi178 ceph-mon[65940]: pgmap v496: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:34 smithi178 ceph-mon[65940]: pgmap v497: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:35.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:34 smithi144 ceph-mon[87436]: pgmap v497: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:36 smithi178 ceph-mon[65940]: pgmap v498: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:36 smithi144 ceph-mon[87436]: pgmap v498: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:39.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:38 smithi178 ceph-mon[65940]: pgmap v499: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:39.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:38 smithi144 ceph-mon[87436]: pgmap v499: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:40 smithi178 ceph-mon[65940]: pgmap v500: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:40 smithi144 ceph-mon[87436]: pgmap v500: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:42.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:41 smithi144 ceph-mon[87436]: pgmap v501: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:41 smithi178 ceph-mon[65940]: pgmap v501: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:43.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:48:44.012 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:48:44.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:44 smithi144 ceph-mon[87436]: pgmap v502: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:44 smithi178 ceph-mon[65940]: pgmap v502: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:47.054 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.053+0000 7f6bd1b44700 1 -- 172.21.15.144:0/4176847667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bcc1013d0 msgr2=0x7f6bcc1017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:47.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.053+0000 7f6bd1b44700 1 --2- 172.21.15.144:0/4176847667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bcc1013d0 0x7f6bcc1017a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009910 tx=0x7f6bbc009c20 comp rx=0 tx=0).stop 2024-09-10T11:48:47.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.053+0000 7f6bd1b44700 1 -- 172.21.15.144:0/4176847667 shutdown_connections 2024-09-10T11:48:47.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.053+0000 7f6bd1b44700 1 --2- 172.21.15.144:0/4176847667 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bcc101ce0 0x7f6bcc10a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.053+0000 7f6bd1b44700 1 --2- 172.21.15.144:0/4176847667 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bcc1013d0 0x7f6bcc1017a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.053+0000 7f6bd1b44700 1 -- 172.21.15.144:0/4176847667 >> 172.21.15.144:0/4176847667 conn(0x7f6bcc0fabb0 msgr2=0x7f6bcc0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:47.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.053+0000 7f6bd1b44700 1 -- 172.21.15.144:0/4176847667 shutdown_connections 2024-09-10T11:48:47.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.053+0000 7f6bd1b44700 1 -- 172.21.15.144:0/4176847667 wait complete. 2024-09-10T11:48:47.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:46 smithi178 ceph-mon[65940]: pgmap v503: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:47.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.053+0000 7f6bd1b44700 1 Processor -- start 2024-09-10T11:48:47.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.054+0000 7f6bd1b44700 1 -- start start 2024-09-10T11:48:47.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.054+0000 7f6bd1b44700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bcc1013d0 0x7f6bcc19bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:47.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.054+0000 7f6bd1b44700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bcc101ce0 0x7f6bcc19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:47.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.054+0000 7f6bd1b44700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc19c680 con 0x7f6bcc101ce0 2024-09-10T11:48:47.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.054+0000 7f6bd1b44700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc195b30 con 0x7f6bcc1013d0 2024-09-10T11:48:47.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.054+0000 7f6bcaffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bcc101ce0 0x7f6bcc19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:47.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.054+0000 7f6bcaffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bcc101ce0 0x7f6bcc19bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45162/0 (socket says 172.21.15.144:45162) 2024-09-10T11:48:47.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.054+0000 7f6bcaffd700 1 -- 172.21.15.144:0/3970757450 learned_addr learned my addr 172.21.15.144:0/3970757450 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:48:47.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.054+0000 7f6bcb7fe700 1 --2- 172.21.15.144:0/3970757450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bcc1013d0 0x7f6bcc19bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:47.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.054+0000 7f6bcaffd700 1 -- 172.21.15.144:0/3970757450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bcc1013d0 msgr2=0x7f6bcc19bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:47.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.054+0000 7f6bcaffd700 1 --2- 172.21.15.144:0/3970757450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bcc1013d0 0x7f6bcc19bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.054+0000 7f6bcaffd700 1 -- 172.21.15.144:0/3970757450 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc0095f0 con 0x7f6bcc101ce0 2024-09-10T11:48:47.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.054+0000 7f6bcb7fe700 1 --2- 172.21.15.144:0/3970757450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bcc1013d0 0x7f6bcc19bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:48:47.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.054+0000 7f6bcaffd700 1 --2- 172.21.15.144:0/3970757450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bcc101ce0 0x7f6bcc19bff0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000d8d0 tx=0x7f6bc000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:47.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.055+0000 7f6bc8ff9700 1 -- 172.21.15.144:0/3970757450 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bc0009940 con 0x7f6bcc101ce0 2024-09-10T11:48:47.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.055+0000 7f6bc8ff9700 1 -- 172.21.15.144:0/3970757450 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bc0010460 con 0x7f6bcc101ce0 2024-09-10T11:48:47.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.055+0000 7f6bd1b44700 1 -- 172.21.15.144:0/3970757450 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bcc195e10 con 0x7f6bcc101ce0 2024-09-10T11:48:47.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.055+0000 7f6bc8ff9700 1 -- 172.21.15.144:0/3970757450 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bc000f5f0 con 0x7f6bcc101ce0 2024-09-10T11:48:47.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.055+0000 7f6bd1b44700 1 -- 172.21.15.144:0/3970757450 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bcc196360 con 0x7f6bcc101ce0 2024-09-10T11:48:47.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.056+0000 7f6bd1b44700 1 -- 172.21.15.144:0/3970757450 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bcc1078b0 con 0x7f6bcc101ce0 2024-09-10T11:48:47.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.056+0000 7f6bc8ff9700 1 -- 172.21.15.144:0/3970757450 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6bc00105d0 con 0x7f6bcc101ce0 2024-09-10T11:48:47.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.057+0000 7f6bc8ff9700 1 --2- 172.21.15.144:0/3970757450 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bb406e200 0x7f6bb40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:47.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.057+0000 7f6bc8ff9700 1 -- 172.21.15.144:0/3970757450 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6bc008df90 con 0x7f6bcc101ce0 2024-09-10T11:48:47.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.057+0000 7f6bcb7fe700 1 --2- 172.21.15.144:0/3970757450 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bb406e200 0x7f6bb40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:47.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.057+0000 7f6bcb7fe700 1 --2- 172.21.15.144:0/3970757450 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bb406e200 0x7f6bb40706b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00b5c0 tx=0x7f6bbc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:47.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.061+0000 7f6bc8ff9700 1 -- 172.21.15.144:0/3970757450 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6bc005af20 con 0x7f6bcc101ce0 2024-09-10T11:48:47.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:46 smithi144 ceph-mon[87436]: pgmap v503: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:47.284 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.283+0000 7f6bd1b44700 1 -- 172.21.15.144:0/3970757450 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bcc196690 con 0x7f6bb406e200 2024-09-10T11:48:47.285 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.284+0000 7f6bc8ff9700 1 -- 172.21.15.144:0/3970757450 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6bcc196690 con 0x7f6bb406e200 2024-09-10T11:48:47.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.286+0000 7f6bd1b44700 1 -- 172.21.15.144:0/3970757450 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bb406e200 msgr2=0x7f6bb40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:47.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.286+0000 7f6bd1b44700 1 --2- 172.21.15.144:0/3970757450 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bb406e200 0x7f6bb40706b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc00b5c0 tx=0x7f6bbc01a040 comp rx=0 tx=0).stop 2024-09-10T11:48:47.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.286+0000 7f6bd1b44700 1 -- 172.21.15.144:0/3970757450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bcc101ce0 msgr2=0x7f6bcc19bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:47.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.286+0000 7f6bd1b44700 1 --2- 172.21.15.144:0/3970757450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bcc101ce0 0x7f6bcc19bff0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000d8d0 tx=0x7f6bc000dc90 comp rx=0 tx=0).stop 2024-09-10T11:48:47.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.286+0000 7f6bd1b44700 1 -- 172.21.15.144:0/3970757450 shutdown_connections 2024-09-10T11:48:47.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.286+0000 7f6bd1b44700 1 --2- 172.21.15.144:0/3970757450 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bb406e200 0x7f6bb40706b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.286+0000 7f6bd1b44700 1 --2- 172.21.15.144:0/3970757450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bcc1013d0 0x7f6bcc19bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.286+0000 7f6bd1b44700 1 --2- 172.21.15.144:0/3970757450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bcc101ce0 0x7f6bcc19bff0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.286+0000 7f6bd1b44700 1 -- 172.21.15.144:0/3970757450 >> 172.21.15.144:0/3970757450 conn(0x7f6bcc0fabb0 msgr2=0x7f6bcc1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:47.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.287+0000 7f6bd1b44700 1 -- 172.21.15.144:0/3970757450 shutdown_connections 2024-09-10T11:48:47.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.287+0000 7f6bd1b44700 1 -- 172.21.15.144:0/3970757450 wait complete. 2024-09-10T11:48:47.298 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:48:47.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.392+0000 7ff18862a700 1 -- 172.21.15.144:0/1779994541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff180101320 msgr2=0x7ff1801016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:47.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.392+0000 7ff18862a700 1 --2- 172.21.15.144:0/1779994541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff180101320 0x7ff1801016f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009910 tx=0x7ff17c009c20 comp rx=0 tx=0).stop 2024-09-10T11:48:47.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.392+0000 7ff18862a700 1 -- 172.21.15.144:0/1779994541 shutdown_connections 2024-09-10T11:48:47.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.392+0000 7ff18862a700 1 --2- 172.21.15.144:0/1779994541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff180101c30 0x7ff18010a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.392+0000 7ff18862a700 1 --2- 172.21.15.144:0/1779994541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff180101320 0x7ff1801016f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.392+0000 7ff18862a700 1 -- 172.21.15.144:0/1779994541 >> 172.21.15.144:0/1779994541 conn(0x7ff1800fab80 msgr2=0x7ff1800fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:47.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.393+0000 7ff18862a700 1 -- 172.21.15.144:0/1779994541 shutdown_connections 2024-09-10T11:48:47.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.393+0000 7ff18862a700 1 -- 172.21.15.144:0/1779994541 wait complete. 2024-09-10T11:48:47.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.393+0000 7ff18862a700 1 Processor -- start 2024-09-10T11:48:47.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.393+0000 7ff18862a700 1 -- start start 2024-09-10T11:48:47.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.393+0000 7ff18862a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff180101320 0x7ff180195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:47.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.393+0000 7ff18862a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff180101c30 0x7ff1801962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:47.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.393+0000 7ff18862a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff180196990 con 0x7ff180101c30 2024-09-10T11:48:47.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.393+0000 7ff18862a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff18019a720 con 0x7ff180101320 2024-09-10T11:48:47.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.393+0000 7ff185bc5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff180101c30 0x7ff1801962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:47.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.393+0000 7ff185bc5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff180101c30 0x7ff1801962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45190/0 (socket says 172.21.15.144:45190) 2024-09-10T11:48:47.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.393+0000 7ff185bc5700 1 -- 172.21.15.144:0/2462078491 learned_addr learned my addr 172.21.15.144:0/2462078491 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:48:47.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.394+0000 7ff1863c6700 1 --2- 172.21.15.144:0/2462078491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff180101320 0x7ff180195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:47.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.394+0000 7ff185bc5700 1 -- 172.21.15.144:0/2462078491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff180101320 msgr2=0x7ff180195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:47.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.394+0000 7ff185bc5700 1 --2- 172.21.15.144:0/2462078491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff180101320 0x7ff180195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.394+0000 7ff185bc5700 1 -- 172.21.15.144:0/2462078491 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c0095f0 con 0x7ff180101c30 2024-09-10T11:48:47.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.394+0000 7ff185bc5700 1 --2- 172.21.15.144:0/2462078491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff180101c30 0x7ff1801962b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff17000d900 tx=0x7ff17000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:47.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.394+0000 7ff1777fe700 1 -- 172.21.15.144:0/2462078491 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff170004a00 con 0x7ff180101c30 2024-09-10T11:48:47.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.394+0000 7ff1777fe700 1 -- 172.21.15.144:0/2462078491 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff1700054e0 con 0x7ff180101c30 2024-09-10T11:48:47.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.394+0000 7ff18862a700 1 -- 172.21.15.144:0/2462078491 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18019aa00 con 0x7ff180101c30 2024-09-10T11:48:47.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.394+0000 7ff1777fe700 1 -- 172.21.15.144:0/2462078491 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff170009d60 con 0x7ff180101c30 2024-09-10T11:48:47.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.394+0000 7ff18862a700 1 -- 172.21.15.144:0/2462078491 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18019af50 con 0x7ff180101c30 2024-09-10T11:48:47.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.395+0000 7ff18862a700 1 -- 172.21.15.144:0/2462078491 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1801077d0 con 0x7ff180101c30 2024-09-10T11:48:47.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.395+0000 7ff1777fe700 1 -- 172.21.15.144:0/2462078491 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff170005020 con 0x7ff180101c30 2024-09-10T11:48:47.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.396+0000 7ff1777fe700 1 --2- 172.21.15.144:0/2462078491 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff16c06e200 0x7ff16c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:47.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.396+0000 7ff1777fe700 1 -- 172.21.15.144:0/2462078491 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff17008e250 con 0x7ff180101c30 2024-09-10T11:48:47.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.396+0000 7ff1863c6700 1 --2- 172.21.15.144:0/2462078491 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff16c06e200 0x7ff16c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:47.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.396+0000 7ff1863c6700 1 --2- 172.21.15.144:0/2462078491 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff16c06e200 0x7ff16c0706b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00b5c0 tx=0x7ff17c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:47.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.400+0000 7ff1777fe700 1 -- 172.21.15.144:0/2462078491 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff170057610 con 0x7ff180101c30 2024-09-10T11:48:47.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.620+0000 7ff18862a700 1 -- 172.21.15.144:0/2462078491 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff18019b230 con 0x7ff16c06e200 2024-09-10T11:48:47.622 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.622+0000 7ff1777fe700 1 -- 172.21.15.144:0/2462078491 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff18019b230 con 0x7ff16c06e200 2024-09-10T11:48:47.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.624+0000 7ff18862a700 1 -- 172.21.15.144:0/2462078491 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff16c06e200 msgr2=0x7ff16c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:47.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.624+0000 7ff18862a700 1 --2- 172.21.15.144:0/2462078491 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff16c06e200 0x7ff16c0706b0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff17c00b5c0 tx=0x7ff17c01a040 comp rx=0 tx=0).stop 2024-09-10T11:48:47.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.624+0000 7ff18862a700 1 -- 172.21.15.144:0/2462078491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff180101c30 msgr2=0x7ff1801962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:47.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.624+0000 7ff18862a700 1 --2- 172.21.15.144:0/2462078491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff180101c30 0x7ff1801962b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff17000d900 tx=0x7ff17000dcc0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.624+0000 7ff18862a700 1 -- 172.21.15.144:0/2462078491 shutdown_connections 2024-09-10T11:48:47.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.624+0000 7ff18862a700 1 --2- 172.21.15.144:0/2462078491 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff16c06e200 0x7ff16c0706b0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.624+0000 7ff18862a700 1 --2- 172.21.15.144:0/2462078491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff180101320 0x7ff180195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.624+0000 7ff18862a700 1 --2- 172.21.15.144:0/2462078491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff180101c30 0x7ff1801962b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.624+0000 7ff18862a700 1 -- 172.21.15.144:0/2462078491 >> 172.21.15.144:0/2462078491 conn(0x7ff1800fab80 msgr2=0x7ff180104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:47.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.624+0000 7ff18862a700 1 -- 172.21.15.144:0/2462078491 shutdown_connections 2024-09-10T11:48:47.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.624+0000 7ff18862a700 1 -- 172.21.15.144:0/2462078491 wait complete. 2024-09-10T11:48:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.730+0000 7f92d9cd2700 1 -- 172.21.15.144:0/3183341457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d4100cb0 msgr2=0x7f92d4109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.730+0000 7f92d9cd2700 1 --2- 172.21.15.144:0/3183341457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d4100cb0 0x7f92d4109410 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009940 tx=0x7f92bc009c50 comp rx=0 tx=0).stop 2024-09-10T11:48:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.730+0000 7f92d9cd2700 1 -- 172.21.15.144:0/3183341457 shutdown_connections 2024-09-10T11:48:47.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.730+0000 7f92d9cd2700 1 --2- 172.21.15.144:0/3183341457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d4100cb0 0x7f92d4109410 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.730+0000 7f92d9cd2700 1 --2- 172.21.15.144:0/3183341457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92d41003a0 0x7f92d4100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.730+0000 7f92d9cd2700 1 -- 172.21.15.144:0/3183341457 >> 172.21.15.144:0/3183341457 conn(0x7f92d4074850 msgr2=0x7f92d4074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:47.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.730+0000 7f92d9cd2700 1 -- 172.21.15.144:0/3183341457 shutdown_connections 2024-09-10T11:48:47.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.730+0000 7f92d9cd2700 1 -- 172.21.15.144:0/3183341457 wait complete. 2024-09-10T11:48:47.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.730+0000 7f92d9cd2700 1 Processor -- start 2024-09-10T11:48:47.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.731+0000 7f92d9cd2700 1 -- start start 2024-09-10T11:48:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.731+0000 7f92d9cd2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d41003a0 0x7f92d4196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.731+0000 7f92d9cd2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92d4100cb0 0x7f92d4197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.731+0000 7f92d9cd2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d41976a0 con 0x7f92d41003a0 2024-09-10T11:48:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.731+0000 7f92d9cd2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d4197810 con 0x7f92d4100cb0 2024-09-10T11:48:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.731+0000 7f92d37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d41003a0 0x7f92d4196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.731+0000 7f92d37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d41003a0 0x7f92d4196b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45194/0 (socket says 172.21.15.144:45194) 2024-09-10T11:48:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.731+0000 7f92d37fe700 1 -- 172.21.15.144:0/13906137 learned_addr learned my addr 172.21.15.144:0/13906137 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:48:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.731+0000 7f92d2ffd700 1 --2- 172.21.15.144:0/13906137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92d4100cb0 0x7f92d4197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.731+0000 7f92d37fe700 1 -- 172.21.15.144:0/13906137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92d4100cb0 msgr2=0x7f92d4197080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.731+0000 7f92d37fe700 1 --2- 172.21.15.144:0/13906137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92d4100cb0 0x7f92d4197080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.731+0000 7f92d37fe700 1 -- 172.21.15.144:0/13906137 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc0095f0 con 0x7f92d41003a0 2024-09-10T11:48:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.731+0000 7f92d37fe700 1 --2- 172.21.15.144:0/13906137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d41003a0 0x7f92d4196b40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f92c40060b0 tx=0x7f92c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:47.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.731+0000 7f92d0ff9700 1 -- 172.21.15.144:0/13906137 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92c400dda0 con 0x7f92d41003a0 2024-09-10T11:48:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.732+0000 7f92d0ff9700 1 -- 172.21.15.144:0/13906137 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f92c400f040 con 0x7f92d41003a0 2024-09-10T11:48:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.732+0000 7f92d9cd2700 1 -- 172.21.15.144:0/13906137 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d419fbb0 con 0x7f92d41003a0 2024-09-10T11:48:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.732+0000 7f92d0ff9700 1 -- 172.21.15.144:0/13906137 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92c40146f0 con 0x7f92d41003a0 2024-09-10T11:48:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.732+0000 7f92d9cd2700 1 -- 172.21.15.144:0/13906137 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d41a0070 con 0x7f92d41003a0 2024-09-10T11:48:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.733+0000 7f92d9cd2700 1 -- 172.21.15.144:0/13906137 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d4106b10 con 0x7f92d41003a0 2024-09-10T11:48:47.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.733+0000 7f92d0ff9700 1 -- 172.21.15.144:0/13906137 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f92c4015400 con 0x7f92d41003a0 2024-09-10T11:48:47.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.734+0000 7f92d0ff9700 1 --2- 172.21.15.144:0/13906137 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f92c006e200 0x7f92c00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:47.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.734+0000 7f92d2ffd700 1 --2- 172.21.15.144:0/13906137 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f92c006e200 0x7f92c00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:47.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.734+0000 7f92d0ff9700 1 -- 172.21.15.144:0/13906137 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f92c408d9c0 con 0x7f92d41003a0 2024-09-10T11:48:47.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.734+0000 7f92d2ffd700 1 --2- 172.21.15.144:0/13906137 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f92c006e200 0x7f92c00706b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009910 tx=0x7f92bc005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:47.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.738+0000 7f92d0ff9700 1 -- 172.21.15.144:0/13906137 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f92c4056ed0 con 0x7f92d41003a0 2024-09-10T11:48:47.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.949+0000 7f92d9cd2700 1 -- 172.21.15.144:0/13906137 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f92d402cd10 con 0x7f92c006e200 2024-09-10T11:48:47.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.958+0000 7f92d0ff9700 1 -- 172.21.15.144:0/13906137 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f92d402cd10 con 0x7f92c006e200 2024-09-10T11:48:47.959 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:48:47.959 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (17m) 10m ago 27m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:48:47.959 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (27m) 10m ago 27m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:48:47.959 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (26m) 10m ago 26m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:48:47.959 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (14m) 10m ago 27m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:48:47.959 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (14m) 10m ago 26m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:48:47.959 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (16m) 10m ago 27m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:48:47.960 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (11m) 10m ago 22m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:48:47.960 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (10m) 10m ago 22m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:48:47.960 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (22m) 10m ago 22m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:48:47.960 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (10m) 10m ago 22m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:48:47.960 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (18m) 10m ago 28m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:48:47.960 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (17m) 10m ago 26m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:48:47.960 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (15m) 10m ago 29m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:48:47.960 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (15m) 10m ago 26m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:48:47.960 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (17m) 10m ago 27m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:48:47.960 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (17m) 10m ago 26m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:48:47.960 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (14m) 10m ago 25m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:48:47.960 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (13m) 10m ago 25m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:48:47.960 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (13m) 10m ago 24m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:48:47.961 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (12m) 10m ago 24m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:48:47.961 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (11m) 10m ago 24m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:48:47.961 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (11m) 10m ago 23m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:48:47.961 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (17m) 10m ago 27m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:48:47.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.960+0000 7f92d9cd2700 1 -- 172.21.15.144:0/13906137 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f92c006e200 msgr2=0x7f92c00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:47.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.960+0000 7f92d9cd2700 1 --2- 172.21.15.144:0/13906137 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f92c006e200 0x7f92c00706b0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f92bc009910 tx=0x7f92bc005a90 comp rx=0 tx=0).stop 2024-09-10T11:48:47.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.960+0000 7f92d9cd2700 1 -- 172.21.15.144:0/13906137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d41003a0 msgr2=0x7f92d4196b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:47.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.960+0000 7f92d9cd2700 1 --2- 172.21.15.144:0/13906137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d41003a0 0x7f92d4196b40 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f92c40060b0 tx=0x7f92c400d3b0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.960+0000 7f92d9cd2700 1 -- 172.21.15.144:0/13906137 shutdown_connections 2024-09-10T11:48:47.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.960+0000 7f92d9cd2700 1 --2- 172.21.15.144:0/13906137 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f92c006e200 0x7f92c00706b0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.960+0000 7f92d9cd2700 1 --2- 172.21.15.144:0/13906137 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f92d41003a0 0x7f92d4196b40 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.960+0000 7f92d9cd2700 1 --2- 172.21.15.144:0/13906137 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f92d4100cb0 0x7f92d4197080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:47.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.960+0000 7f92d9cd2700 1 -- 172.21.15.144:0/13906137 >> 172.21.15.144:0/13906137 conn(0x7f92d4074850 msgr2=0x7f92d4103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:47.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.960+0000 7f92d9cd2700 1 -- 172.21.15.144:0/13906137 shutdown_connections 2024-09-10T11:48:47.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:47.960+0000 7f92d9cd2700 1 -- 172.21.15.144:0/13906137 wait complete. 2024-09-10T11:48:48.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.073+0000 7f5b3b48c700 1 -- 172.21.15.144:0/2473948066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b340ff390 msgr2=0x7f5b340ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:48.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.073+0000 7f5b3b48c700 1 --2- 172.21.15.144:0/2473948066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b340ff390 0x7f5b340ff760 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400b0a0 tx=0x7f5b2400b3b0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.073+0000 7f5b3b48c700 1 -- 172.21.15.144:0/2473948066 shutdown_connections 2024-09-10T11:48:48.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.073+0000 7f5b3b48c700 1 --2- 172.21.15.144:0/2473948066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b340ffd30 0x7f5b34103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.073+0000 7f5b3b48c700 1 --2- 172.21.15.144:0/2473948066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b340ff390 0x7f5b340ff760 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.073+0000 7f5b3b48c700 1 -- 172.21.15.144:0/2473948066 >> 172.21.15.144:0/2473948066 conn(0x7f5b34074f50 msgr2=0x7f5b34075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:48.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.073+0000 7f5b3b48c700 1 -- 172.21.15.144:0/2473948066 shutdown_connections 2024-09-10T11:48:48.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.073+0000 7f5b3b48c700 1 -- 172.21.15.144:0/2473948066 wait complete. 2024-09-10T11:48:48.076 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.074+0000 7f5b3b48c700 1 Processor -- start 2024-09-10T11:48:48.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.074+0000 7f5b3b48c700 1 -- start start 2024-09-10T11:48:48.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.074+0000 7f5b3b48c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b340ff390 0x7f5b3419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:48.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.074+0000 7f5b3b48c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b340ffd30 0x7f5b3419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:48.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.074+0000 7f5b3b48c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3419b640 con 0x7f5b340ff390 2024-09-10T11:48:48.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.074+0000 7f5b3b48c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b34194b00 con 0x7f5b340ffd30 2024-09-10T11:48:48.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.074+0000 7f5b39228700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b340ff390 0x7f5b3419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:48.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.074+0000 7f5b39228700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b340ff390 0x7f5b3419aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45212/0 (socket says 172.21.15.144:45212) 2024-09-10T11:48:48.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.074+0000 7f5b39228700 1 -- 172.21.15.144:0/3036593794 learned_addr learned my addr 172.21.15.144:0/3036593794 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:48:48.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.075+0000 7f5b38a27700 1 --2- 172.21.15.144:0/3036593794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b340ffd30 0x7f5b3419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:48.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.075+0000 7f5b39228700 1 -- 172.21.15.144:0/3036593794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b340ffd30 msgr2=0x7f5b3419af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:48.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.075+0000 7f5b39228700 1 --2- 172.21.15.144:0/3036593794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b340ffd30 0x7f5b3419af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.075+0000 7f5b39228700 1 -- 172.21.15.144:0/3036593794 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b24009d00 con 0x7f5b340ff390 2024-09-10T11:48:48.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.075+0000 7f5b38a27700 1 --2- 172.21.15.144:0/3036593794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b340ffd30 0x7f5b3419af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:48:48.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.075+0000 7f5b39228700 1 --2- 172.21.15.144:0/3036593794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b340ff390 0x7f5b3419aa20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5b24000f80 tx=0x7f5b2400cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:48.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.075+0000 7f5b327fc700 1 -- 172.21.15.144:0/3036593794 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b2400e040 con 0x7f5b340ff390 2024-09-10T11:48:48.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.075+0000 7f5b3b48c700 1 -- 172.21.15.144:0/3036593794 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b34194d80 con 0x7f5b340ff390 2024-09-10T11:48:48.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.075+0000 7f5b327fc700 1 -- 172.21.15.144:0/3036593794 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b24003e60 con 0x7f5b340ff390 2024-09-10T11:48:48.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.075+0000 7f5b327fc700 1 -- 172.21.15.144:0/3036593794 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b2401d910 con 0x7f5b340ff390 2024-09-10T11:48:48.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.075+0000 7f5b3b48c700 1 -- 172.21.15.144:0/3036593794 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b34195270 con 0x7f5b340ff390 2024-09-10T11:48:48.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.076+0000 7f5b3b48c700 1 -- 172.21.15.144:0/3036593794 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b34107650 con 0x7f5b340ff390 2024-09-10T11:48:48.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.077+0000 7f5b327fc700 1 -- 172.21.15.144:0/3036593794 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5b24004820 con 0x7f5b340ff390 2024-09-10T11:48:48.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.077+0000 7f5b327fc700 1 --2- 172.21.15.144:0/3036593794 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5b1c06e200 0x7f5b1c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:48.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.077+0000 7f5b327fc700 1 -- 172.21.15.144:0/3036593794 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5b2408f9c0 con 0x7f5b340ff390 2024-09-10T11:48:48.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.077+0000 7f5b38a27700 1 --2- 172.21.15.144:0/3036593794 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5b1c06e200 0x7f5b1c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:48.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.077+0000 7f5b38a27700 1 --2- 172.21.15.144:0/3036593794 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5b1c06e200 0x7f5b1c0706b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5b28005fd0 tx=0x7f5b28005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:48.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.081+0000 7f5b327fc700 1 -- 172.21.15.144:0/3036593794 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5b2405c8d0 con 0x7f5b340ff390 2024-09-10T11:48:48.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.371+0000 7f5b3b48c700 1 -- 172.21.15.144:0/3036593794 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5b34107030 con 0x7f5b340ff390 2024-09-10T11:48:48.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.372+0000 7f5b327fc700 1 -- 172.21.15.144:0/3036593794 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f5b2405c460 con 0x7f5b340ff390 2024-09-10T11:48:48.373 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:48:48.373 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:48:48.373 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:48:48.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:48:48.373 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:48:48.373 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:48:48.373 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:48:48.374 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:48:48.374 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:48:48.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:48:48.374 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:48:48.374 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:48:48.374 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:48:48.374 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:48:48.374 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:48:48.374 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:48:48.374 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:48:48.374 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:48:48.374 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:48:48.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.374+0000 7f5b3b48c700 1 -- 172.21.15.144:0/3036593794 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5b1c06e200 msgr2=0x7f5b1c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:48.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.374+0000 7f5b3b48c700 1 --2- 172.21.15.144:0/3036593794 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5b1c06e200 0x7f5b1c0706b0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5b28005fd0 tx=0x7f5b28005dc0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.374+0000 7f5b3b48c700 1 -- 172.21.15.144:0/3036593794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b340ff390 msgr2=0x7f5b3419aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:48.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.374+0000 7f5b3b48c700 1 --2- 172.21.15.144:0/3036593794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b340ff390 0x7f5b3419aa20 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f5b24000f80 tx=0x7f5b2400cdd0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.374+0000 7f5b3b48c700 1 -- 172.21.15.144:0/3036593794 shutdown_connections 2024-09-10T11:48:48.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.374+0000 7f5b3b48c700 1 --2- 172.21.15.144:0/3036593794 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5b1c06e200 0x7f5b1c0706b0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.374+0000 7f5b3b48c700 1 --2- 172.21.15.144:0/3036593794 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5b340ff390 0x7f5b3419aa20 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.374+0000 7f5b3b48c700 1 --2- 172.21.15.144:0/3036593794 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5b340ffd30 0x7f5b3419af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.374+0000 7f5b3b48c700 1 -- 172.21.15.144:0/3036593794 >> 172.21.15.144:0/3036593794 conn(0x7f5b34074f50 msgr2=0x7f5b340fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:48.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.375+0000 7f5b3b48c700 1 -- 172.21.15.144:0/3036593794 shutdown_connections 2024-09-10T11:48:48.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.375+0000 7f5b3b48c700 1 -- 172.21.15.144:0/3036593794 wait complete. 2024-09-10T11:48:48.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.479+0000 7f3d783ec700 1 -- 172.21.15.144:0/1564009839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d70101370 msgr2=0x7f3d70101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:48.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.479+0000 7f3d783ec700 1 --2- 172.21.15.144:0/1564009839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d70101370 0x7f3d70101740 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c009910 tx=0x7f3d6c009c20 comp rx=0 tx=0).stop 2024-09-10T11:48:48.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.479+0000 7f3d783ec700 1 -- 172.21.15.144:0/1564009839 shutdown_connections 2024-09-10T11:48:48.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.479+0000 7f3d783ec700 1 --2- 172.21.15.144:0/1564009839 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d70101c80 0x7f3d7010a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.479+0000 7f3d783ec700 1 --2- 172.21.15.144:0/1564009839 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d70101370 0x7f3d70101740 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.479+0000 7f3d783ec700 1 -- 172.21.15.144:0/1564009839 >> 172.21.15.144:0/1564009839 conn(0x7f3d700fabd0 msgr2=0x7f3d700fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:48.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.479+0000 7f3d783ec700 1 -- 172.21.15.144:0/1564009839 shutdown_connections 2024-09-10T11:48:48.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.479+0000 7f3d783ec700 1 -- 172.21.15.144:0/1564009839 wait complete. 2024-09-10T11:48:48.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.479+0000 7f3d783ec700 1 Processor -- start 2024-09-10T11:48:48.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.479+0000 7f3d783ec700 1 -- start start 2024-09-10T11:48:48.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.480+0000 7f3d783ec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d70101370 0x7f3d70197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:48.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.480+0000 7f3d783ec700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d70101c80 0x7f3d701984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:48.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.480+0000 7f3d783ec700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d70198bb0 con 0x7f3d70101370 2024-09-10T11:48:48.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.480+0000 7f3d783ec700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d7019c940 con 0x7f3d70101c80 2024-09-10T11:48:48.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.480+0000 7f3d76188700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d70101370 0x7f3d70197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:48.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.480+0000 7f3d76188700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d70101370 0x7f3d70197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45238/0 (socket says 172.21.15.144:45238) 2024-09-10T11:48:48.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.480+0000 7f3d76188700 1 -- 172.21.15.144:0/1723541175 learned_addr learned my addr 172.21.15.144:0/1723541175 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:48:48.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.480+0000 7f3d76188700 1 -- 172.21.15.144:0/1723541175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d70101c80 msgr2=0x7f3d701984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:48.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.480+0000 7f3d76188700 1 --2- 172.21.15.144:0/1723541175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d70101c80 0x7f3d701984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.480+0000 7f3d76188700 1 -- 172.21.15.144:0/1723541175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c0095f0 con 0x7f3d70101370 2024-09-10T11:48:48.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.480+0000 7f3d76188700 1 --2- 172.21.15.144:0/1723541175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d70101370 0x7f3d70197f90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c006010 tx=0x7f3d6c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:48.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.480+0000 7f3d677fe700 1 -- 172.21.15.144:0/1723541175 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d6c01d070 con 0x7f3d70101370 2024-09-10T11:48:48.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.480+0000 7f3d783ec700 1 -- 172.21.15.144:0/1723541175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d7019cbc0 con 0x7f3d70101370 2024-09-10T11:48:48.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.480+0000 7f3d677fe700 1 -- 172.21.15.144:0/1723541175 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d6c00f460 con 0x7f3d70101370 2024-09-10T11:48:48.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.480+0000 7f3d677fe700 1 -- 172.21.15.144:0/1723541175 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d6c017630 con 0x7f3d70101370 2024-09-10T11:48:48.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.481+0000 7f3d783ec700 1 -- 172.21.15.144:0/1723541175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d7019d0b0 con 0x7f3d70101370 2024-09-10T11:48:48.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.482+0000 7f3d783ec700 1 -- 172.21.15.144:0/1723541175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d70109cb0 con 0x7f3d70101370 2024-09-10T11:48:48.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.482+0000 7f3d677fe700 1 -- 172.21.15.144:0/1723541175 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3d6c00f5d0 con 0x7f3d70101370 2024-09-10T11:48:48.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.482+0000 7f3d677fe700 1 --2- 172.21.15.144:0/1723541175 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d5c06e200 0x7f3d5c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:48.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.482+0000 7f3d677fe700 1 -- 172.21.15.144:0/1723541175 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3d6c08f520 con 0x7f3d70101370 2024-09-10T11:48:48.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.483+0000 7f3d75987700 1 --2- 172.21.15.144:0/1723541175 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d5c06e200 0x7f3d5c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:48.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.483+0000 7f3d75987700 1 --2- 172.21.15.144:0/1723541175 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d5c06e200 0x7f3d5c0706b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3d60005950 tx=0x7f3d600058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:48.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.487+0000 7f3d677fe700 1 -- 172.21.15.144:0/1723541175 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3d6c058900 con 0x7f3d70101370 2024-09-10T11:48:48.734 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.733+0000 7f3d783ec700 1 -- 172.21.15.144:0/1723541175 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3d7002ccf0 con 0x7f3d70101370 2024-09-10T11:48:48.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.734+0000 7f3d677fe700 1 -- 172.21.15.144:0/1723541175 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f3d6c05bfc0 con 0x7f3d70101370 2024-09-10T11:48:48.735 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:48:48.735 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:48:48.735 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:48:48.735 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:48:48.736 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:48:48.736 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:48:48.736 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:48:48.736 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:48:48.736 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:48:48.736 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:48:48.736 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:48:48.736 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:48:48.736 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:48:48.736 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:48:48.736 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:48:48.736 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:48:48.737 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:48:48.737 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:48:48.737 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:48:48.737 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:48:48.737 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:48:48.737 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:48:48.737 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:48:48.737 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:48:48.737 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:48:48.737 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:48:48.737 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:48:48.737 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:48:48.738 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:48:48.738 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:48:48.738 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:48:48.738 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:48:48.738 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:48:48.738 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:48:48.738 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:48:48.738 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:48:48.738 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:48:48.738 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:48:48.738 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:48:48.738 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:48:48.739 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:48:48.739 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:48:48.739 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:48:48.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.736+0000 7f3d783ec700 1 -- 172.21.15.144:0/1723541175 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d5c06e200 msgr2=0x7f3d5c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:48.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.736+0000 7f3d783ec700 1 --2- 172.21.15.144:0/1723541175 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d5c06e200 0x7f3d5c0706b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3d60005950 tx=0x7f3d600058e0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.736+0000 7f3d783ec700 1 -- 172.21.15.144:0/1723541175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d70101370 msgr2=0x7f3d70197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:48.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.736+0000 7f3d783ec700 1 --2- 172.21.15.144:0/1723541175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d70101370 0x7f3d70197f90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c006010 tx=0x7f3d6c00be20 comp rx=0 tx=0).stop 2024-09-10T11:48:48.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.736+0000 7f3d783ec700 1 -- 172.21.15.144:0/1723541175 shutdown_connections 2024-09-10T11:48:48.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.736+0000 7f3d783ec700 1 --2- 172.21.15.144:0/1723541175 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d5c06e200 0x7f3d5c0706b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.736+0000 7f3d783ec700 1 --2- 172.21.15.144:0/1723541175 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d70101370 0x7f3d70197f90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.736+0000 7f3d783ec700 1 --2- 172.21.15.144:0/1723541175 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d70101c80 0x7f3d701984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.736+0000 7f3d783ec700 1 -- 172.21.15.144:0/1723541175 >> 172.21.15.144:0/1723541175 conn(0x7f3d700fabd0 msgr2=0x7f3d700ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:48.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.736+0000 7f3d783ec700 1 -- 172.21.15.144:0/1723541175 shutdown_connections 2024-09-10T11:48:48.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.736+0000 7f3d783ec700 1 -- 172.21.15.144:0/1723541175 wait complete. 2024-09-10T11:48:48.741 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:48:48.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.842+0000 7fa7225a2700 1 -- 172.21.15.144:0/3397223928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa71c073070 msgr2=0x7fa71c073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:48.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.842+0000 7fa7225a2700 1 --2- 172.21.15.144:0/3397223928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa71c073070 0x7fa71c073440 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa70c009940 tx=0x7fa70c009c50 comp rx=0 tx=0).stop 2024-09-10T11:48:48.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.842+0000 7fa7225a2700 1 -- 172.21.15.144:0/3397223928 shutdown_connections 2024-09-10T11:48:48.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.842+0000 7fa7225a2700 1 --2- 172.21.15.144:0/3397223928 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa71c073980 0x7fa71c10c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.842+0000 7fa7225a2700 1 --2- 172.21.15.144:0/3397223928 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa71c073070 0x7fa71c073440 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.842+0000 7fa7225a2700 1 -- 172.21.15.144:0/3397223928 >> 172.21.15.144:0/3397223928 conn(0x7fa71c0fbca0 msgr2=0x7fa71c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:48.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.842+0000 7fa7225a2700 1 -- 172.21.15.144:0/3397223928 shutdown_connections 2024-09-10T11:48:48.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.842+0000 7fa7225a2700 1 -- 172.21.15.144:0/3397223928 wait complete. 2024-09-10T11:48:48.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.843+0000 7fa7225a2700 1 Processor -- start 2024-09-10T11:48:48.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.843+0000 7fa7225a2700 1 -- start start 2024-09-10T11:48:48.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.843+0000 7fa7225a2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa71c073070 0x7fa71c197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:48.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.843+0000 7fa7225a2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa71c073980 0x7fa71c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:48.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.843+0000 7fa7225a2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c198b50 con 0x7fa71c073070 2024-09-10T11:48:48.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.843+0000 7fa7225a2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c19c890 con 0x7fa71c073980 2024-09-10T11:48:48.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.843+0000 7fa71bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa71c073070 0x7fa71c197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:48.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.844+0000 7fa71bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa71c073070 0x7fa71c197f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45248/0 (socket says 172.21.15.144:45248) 2024-09-10T11:48:48.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.844+0000 7fa71bfff700 1 -- 172.21.15.144:0/2472793996 learned_addr learned my addr 172.21.15.144:0/2472793996 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:48:48.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.844+0000 7fa71b7fe700 1 --2- 172.21.15.144:0/2472793996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa71c073980 0x7fa71c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:48.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.844+0000 7fa71bfff700 1 -- 172.21.15.144:0/2472793996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa71c073980 msgr2=0x7fa71c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:48.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.844+0000 7fa71bfff700 1 --2- 172.21.15.144:0/2472793996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa71c073980 0x7fa71c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:48.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.844+0000 7fa71bfff700 1 -- 172.21.15.144:0/2472793996 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c0095f0 con 0x7fa71c073070 2024-09-10T11:48:48.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.844+0000 7fa71b7fe700 1 --2- 172.21.15.144:0/2472793996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa71c073980 0x7fa71c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:48:48.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.844+0000 7fa71bfff700 1 --2- 172.21.15.144:0/2472793996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa71c073070 0x7fa71c197f30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00bb10 tx=0x7fa70c00bbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:48.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.844+0000 7fa7197fa700 1 -- 172.21.15.144:0/2472793996 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa70c01d070 con 0x7fa71c073070 2024-09-10T11:48:48.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.844+0000 7fa7225a2700 1 -- 172.21.15.144:0/2472793996 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa71c19cb10 con 0x7fa71c073070 2024-09-10T11:48:48.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.844+0000 7fa7197fa700 1 -- 172.21.15.144:0/2472793996 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa70c00be80 con 0x7fa71c073070 2024-09-10T11:48:48.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.844+0000 7fa7197fa700 1 -- 172.21.15.144:0/2472793996 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa70c021410 con 0x7fa71c073070 2024-09-10T11:48:48.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.844+0000 7fa7225a2700 1 -- 172.21.15.144:0/2472793996 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71c19d000 con 0x7fa71c073070 2024-09-10T11:48:48.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.845+0000 7fa7225a2700 1 -- 172.21.15.144:0/2472793996 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa71c109c00 con 0x7fa71c073070 2024-09-10T11:48:48.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.846+0000 7fa7197fa700 1 -- 172.21.15.144:0/2472793996 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa70c00f7f0 con 0x7fa71c073070 2024-09-10T11:48:48.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.846+0000 7fa7197fa700 1 --2- 172.21.15.144:0/2472793996 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa70406e200 0x7fa7040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:48.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.846+0000 7fa7197fa700 1 -- 172.21.15.144:0/2472793996 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa70c08f290 con 0x7fa71c073070 2024-09-10T11:48:48.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.846+0000 7fa71b7fe700 1 --2- 172.21.15.144:0/2472793996 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa70406e200 0x7fa7040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:48.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.847+0000 7fa71b7fe700 1 --2- 172.21.15.144:0/2472793996 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa70406e200 0x7fa7040706b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa71c199500 tx=0x7fa710009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:48.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:48.851+0000 7fa7197fa700 1 -- 172.21.15.144:0/2472793996 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa70c058670 con 0x7fa71c073070 2024-09-10T11:48:49.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.072+0000 7fa7225a2700 1 -- 172.21.15.144:0/2472793996 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa71c02cd40 con 0x7fa70406e200 2024-09-10T11:48:49.073 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:48 smithi144 ceph-mon[87436]: from='client.34960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:49.073 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:48 smithi144 ceph-mon[87436]: from='client.34964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:49.074 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:48 smithi144 ceph-mon[87436]: pgmap v504: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:49.074 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:48 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3036593794' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:48:49.074 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:48 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1723541175' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:48:49.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.073+0000 7fa7197fa700 1 -- 172.21.15.144:0/2472793996 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa71c02cd40 con 0x7fa70406e200 2024-09-10T11:48:49.074 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:48:49.075 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:48:49.075 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:48:49.075 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:48:49.075 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:48:49.075 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:48:49.075 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:48:49.075 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:48:49.075 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:48:49.075 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:48:49.075 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:48:49.075 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:48:49.075 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:48:49.075 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:48:49.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.076+0000 7fa7225a2700 1 -- 172.21.15.144:0/2472793996 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa70406e200 msgr2=0x7fa7040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:49.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.076+0000 7fa7225a2700 1 --2- 172.21.15.144:0/2472793996 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa70406e200 0x7fa7040706b0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa71c199500 tx=0x7fa710009260 comp rx=0 tx=0).stop 2024-09-10T11:48:49.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.076+0000 7fa7225a2700 1 -- 172.21.15.144:0/2472793996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa71c073070 msgr2=0x7fa71c197f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.076+0000 7fa7225a2700 1 --2- 172.21.15.144:0/2472793996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa71c073070 0x7fa71c197f30 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00bb10 tx=0x7fa70c00bbf0 comp rx=0 tx=0).stop 2024-09-10T11:48:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.076+0000 7fa7225a2700 1 -- 172.21.15.144:0/2472793996 shutdown_connections 2024-09-10T11:48:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.076+0000 7fa7225a2700 1 --2- 172.21.15.144:0/2472793996 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa70406e200 0x7fa7040706b0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.076+0000 7fa7225a2700 1 --2- 172.21.15.144:0/2472793996 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa71c073070 0x7fa71c197f30 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.076+0000 7fa7225a2700 1 --2- 172.21.15.144:0/2472793996 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa71c073980 0x7fa71c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.076+0000 7fa7225a2700 1 -- 172.21.15.144:0/2472793996 >> 172.21.15.144:0/2472793996 conn(0x7fa71c0fbca0 msgr2=0x7fa71c106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.076+0000 7fa7225a2700 1 -- 172.21.15.144:0/2472793996 shutdown_connections 2024-09-10T11:48:49.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.076+0000 7fa7225a2700 1 -- 172.21.15.144:0/2472793996 wait complete. 2024-09-10T11:48:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:48 smithi178 ceph-mon[65940]: from='client.34960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:48 smithi178 ceph-mon[65940]: from='client.34964 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:48 smithi178 ceph-mon[65940]: pgmap v504: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:48 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3036593794' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:48:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:48 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1723541175' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:48:49.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.180+0000 7f4fd2810700 1 -- 172.21.15.144:0/2828227064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fcc100c90 msgr2=0x7f4fcc1093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:49.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.180+0000 7f4fd2810700 1 --2- 172.21.15.144:0/2828227064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fcc100c90 0x7f4fcc1093f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4009940 tx=0x7f4fb4009c50 comp rx=0 tx=0).stop 2024-09-10T11:48:49.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.181+0000 7f4fd2810700 1 -- 172.21.15.144:0/2828227064 shutdown_connections 2024-09-10T11:48:49.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.181+0000 7f4fd2810700 1 --2- 172.21.15.144:0/2828227064 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fcc100c90 0x7f4fcc1093f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:49.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.181+0000 7f4fd2810700 1 --2- 172.21.15.144:0/2828227064 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fcc100380 0x7f4fcc100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:49.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.181+0000 7f4fd2810700 1 -- 172.21.15.144:0/2828227064 >> 172.21.15.144:0/2828227064 conn(0x7f4fcc074850 msgr2=0x7f4fcc074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:49.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.181+0000 7f4fd2810700 1 -- 172.21.15.144:0/2828227064 shutdown_connections 2024-09-10T11:48:49.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.181+0000 7f4fd2810700 1 -- 172.21.15.144:0/2828227064 wait complete. 2024-09-10T11:48:49.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.181+0000 7f4fd2810700 1 Processor -- start 2024-09-10T11:48:49.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.181+0000 7f4fd2810700 1 -- start start 2024-09-10T11:48:49.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fd2810700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fcc100380 0x7f4fcc101430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:49.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fd2810700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fcc100c90 0x7f4fcc101970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:49.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fd2810700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc101eb0 con 0x7f4fcc100c90 2024-09-10T11:48:49.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fd2810700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc101ff0 con 0x7f4fcc100380 2024-09-10T11:48:49.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fcb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fcc100c90 0x7f4fcc101970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:49.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fcb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fcc100c90 0x7f4fcc101970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45270/0 (socket says 172.21.15.144:45270) 2024-09-10T11:48:49.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fcb7fe700 1 -- 172.21.15.144:0/857860477 learned_addr learned my addr 172.21.15.144:0/857860477 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:48:49.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fcbfff700 1 --2- 172.21.15.144:0/857860477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fcc100380 0x7f4fcc101430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:49.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fcb7fe700 1 -- 172.21.15.144:0/857860477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fcc100380 msgr2=0x7f4fcc101430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fcb7fe700 1 --2- 172.21.15.144:0/857860477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fcc100380 0x7f4fcc101430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fcb7fe700 1 -- 172.21.15.144:0/857860477 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb40095f0 con 0x7f4fcc100c90 2024-09-10T11:48:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fcb7fe700 1 --2- 172.21.15.144:0/857860477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fcc100c90 0x7f4fcc101970 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40055f0 tx=0x7f4fb40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fc97fa700 1 -- 172.21.15.144:0/857860477 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fb401d070 con 0x7f4fcc100c90 2024-09-10T11:48:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fc97fa700 1 -- 172.21.15.144:0/857860477 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fb400f460 con 0x7f4fcc100c90 2024-09-10T11:48:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fd2810700 1 -- 172.21.15.144:0/857860477 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fcc1a1e70 con 0x7f4fcc100c90 2024-09-10T11:48:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fc97fa700 1 -- 172.21.15.144:0/857860477 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fb4021620 con 0x7f4fcc100c90 2024-09-10T11:48:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.182+0000 7f4fd2810700 1 -- 172.21.15.144:0/857860477 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc1a2280 con 0x7f4fcc100c90 2024-09-10T11:48:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.183+0000 7f4fd2810700 1 -- 172.21.15.144:0/857860477 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fcc107d30 con 0x7f4fcc100c90 2024-09-10T11:48:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.184+0000 7f4fc97fa700 1 -- 172.21.15.144:0/857860477 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4fb400fa60 con 0x7f4fcc100c90 2024-09-10T11:48:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.184+0000 7f4fc97fa700 1 --2- 172.21.15.144:0/857860477 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fb806e200 0x7f4fb80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:48:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.184+0000 7f4fc97fa700 1 -- 172.21.15.144:0/857860477 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4fb408f3f0 con 0x7f4fcc100c90 2024-09-10T11:48:49.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.184+0000 7f4fcbfff700 1 --2- 172.21.15.144:0/857860477 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fb806e200 0x7f4fb80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:48:49.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.185+0000 7f4fcbfff700 1 --2- 172.21.15.144:0/857860477 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fb806e200 0x7f4fb80706b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc005950 tx=0x7f4fbc005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:48:49.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.189+0000 7f4fc97fa700 1 -- 172.21.15.144:0/857860477 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4fb40587d0 con 0x7f4fcc100c90 2024-09-10T11:48:49.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.471+0000 7f4fd2810700 1 -- 172.21.15.144:0/857860477 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4fcc04ea50 con 0x7f4fcc100c90 2024-09-10T11:48:49.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.471+0000 7f4fc97fa700 1 -- 172.21.15.144:0/857860477 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f4fb405be90 con 0x7f4fcc100c90 2024-09-10T11:48:49.473 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:48:49.473 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:48:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:48:49.473 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:48:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:48:49.473 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:48:49.473 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:48:49.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.474+0000 7f4fd2810700 1 -- 172.21.15.144:0/857860477 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fb806e200 msgr2=0x7f4fb80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:49.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.474+0000 7f4fd2810700 1 --2- 172.21.15.144:0/857860477 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fb806e200 0x7f4fb80706b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc005950 tx=0x7f4fbc005cf0 comp rx=0 tx=0).stop 2024-09-10T11:48:49.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.474+0000 7f4fd2810700 1 -- 172.21.15.144:0/857860477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fcc100c90 msgr2=0x7f4fcc101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:48:49.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.474+0000 7f4fd2810700 1 --2- 172.21.15.144:0/857860477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fcc100c90 0x7f4fcc101970 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4fb40055f0 tx=0x7f4fb40056d0 comp rx=0 tx=0).stop 2024-09-10T11:48:49.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.474+0000 7f4fd2810700 1 -- 172.21.15.144:0/857860477 shutdown_connections 2024-09-10T11:48:49.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.474+0000 7f4fd2810700 1 --2- 172.21.15.144:0/857860477 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fb806e200 0x7f4fb80706b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:49.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.474+0000 7f4fd2810700 1 --2- 172.21.15.144:0/857860477 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fcc100380 0x7f4fcc101430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:49.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.474+0000 7f4fd2810700 1 --2- 172.21.15.144:0/857860477 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fcc100c90 0x7f4fcc101970 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:48:49.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.474+0000 7f4fd2810700 1 -- 172.21.15.144:0/857860477 >> 172.21.15.144:0/857860477 conn(0x7f4fcc074850 msgr2=0x7f4fcc0ffb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:48:49.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.474+0000 7f4fd2810700 1 -- 172.21.15.144:0/857860477 shutdown_connections 2024-09-10T11:48:49.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:48:49.474+0000 7f4fd2810700 1 -- 172.21.15.144:0/857860477 wait complete. 2024-09-10T11:48:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:49 smithi178 ceph-mon[65940]: from='client.34968 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:49 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/857860477' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:48:50.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:49 smithi144 ceph-mon[87436]: from='client.34968 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:50.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:49 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/857860477' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:48:50.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:50 smithi144 ceph-mon[87436]: from='client.34980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:50.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:50 smithi144 ceph-mon[87436]: pgmap v505: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:50 smithi178 ceph-mon[65940]: from='client.34980 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:48:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:50 smithi178 ceph-mon[65940]: pgmap v505: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:53.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:52 smithi144 ceph-mon[87436]: pgmap v506: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:53.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:52 smithi178 ceph-mon[65940]: pgmap v506: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:54 smithi178 ceph-mon[65940]: pgmap v507: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:54 smithi144 ceph-mon[87436]: pgmap v507: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:56 smithi178 ceph-mon[65940]: pgmap v508: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:56 smithi144 ceph-mon[87436]: pgmap v508: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:58 smithi178 ceph-mon[65940]: pgmap v509: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:48:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:48:59.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:58 smithi144 ceph-mon[87436]: pgmap v509: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:48:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:48:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:49:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:00 smithi178 ceph-mon[65940]: pgmap v510: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:00 smithi144 ceph-mon[87436]: pgmap v510: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:02.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:01 smithi144 ceph-mon[87436]: pgmap v511: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:02.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:01 smithi178 ceph-mon[65940]: pgmap v511: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:04 smithi178 ceph-mon[65940]: pgmap v512: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:05.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:04 smithi144 ceph-mon[87436]: pgmap v512: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:06 smithi178 ceph-mon[65940]: pgmap v513: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:06 smithi144 ceph-mon[87436]: pgmap v513: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:08 smithi178 ceph-mon[65940]: pgmap v514: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:09.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:08 smithi144 ceph-mon[87436]: pgmap v514: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:10 smithi178 ceph-mon[65940]: pgmap v515: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:11.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:10 smithi144 ceph-mon[87436]: pgmap v515: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:12 smithi144 ceph-mon[87436]: pgmap v516: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:13.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:12 smithi178 ceph-mon[65940]: pgmap v516: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:49:14.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:49:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:14 smithi178 ceph-mon[65940]: pgmap v517: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:15.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:14 smithi144 ceph-mon[87436]: pgmap v517: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:16 smithi178 ceph-mon[65940]: pgmap v518: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:17.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:16 smithi144 ceph-mon[87436]: pgmap v518: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:18 smithi178 ceph-mon[65940]: pgmap v519: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:19.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:18 smithi144 ceph-mon[87436]: pgmap v519: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:19.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.591+0000 7f88dc4f4700 1 -- 172.21.15.144:0/3836103632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d40ff390 msgr2=0x7f88d40ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:19.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.591+0000 7f88dc4f4700 1 --2- 172.21.15.144:0/3836103632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d40ff390 0x7f88d40ff760 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f88d0009910 tx=0x7f88d0009c20 comp rx=0 tx=0).stop 2024-09-10T11:49:19.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.591+0000 7f88dc4f4700 1 -- 172.21.15.144:0/3836103632 shutdown_connections 2024-09-10T11:49:19.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.591+0000 7f88dc4f4700 1 --2- 172.21.15.144:0/3836103632 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88d40ffd30 0x7f88d4103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:19.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.591+0000 7f88dc4f4700 1 --2- 172.21.15.144:0/3836103632 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d40ff390 0x7f88d40ff760 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:19.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.591+0000 7f88dc4f4700 1 -- 172.21.15.144:0/3836103632 >> 172.21.15.144:0/3836103632 conn(0x7f88d4074f50 msgr2=0x7f88d4075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:19.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.591+0000 7f88dc4f4700 1 -- 172.21.15.144:0/3836103632 shutdown_connections 2024-09-10T11:49:19.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.591+0000 7f88dc4f4700 1 -- 172.21.15.144:0/3836103632 wait complete. 2024-09-10T11:49:19.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.592+0000 7f88dc4f4700 1 Processor -- start 2024-09-10T11:49:19.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.592+0000 7f88dc4f4700 1 -- start start 2024-09-10T11:49:19.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.592+0000 7f88dc4f4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88d40ff390 0x7f88d4100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:19.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.592+0000 7f88dc4f4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d40ffd30 0x7f88d41012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:19.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.592+0000 7f88dc4f4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d4101800 con 0x7f88d40ffd30 2024-09-10T11:49:19.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.592+0000 7f88dc4f4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d4101940 con 0x7f88d40ff390 2024-09-10T11:49:19.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.592+0000 7f88d9a8f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d40ffd30 0x7f88d41012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:19.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.592+0000 7f88d9a8f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d40ffd30 0x7f88d41012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36408/0 (socket says 172.21.15.144:36408) 2024-09-10T11:49:19.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.592+0000 7f88d9a8f700 1 -- 172.21.15.144:0/1688039404 learned_addr learned my addr 172.21.15.144:0/1688039404 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:49:19.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.593+0000 7f88da290700 1 --2- 172.21.15.144:0/1688039404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88d40ff390 0x7f88d4100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:19.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.593+0000 7f88d9a8f700 1 -- 172.21.15.144:0/1688039404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88d40ff390 msgr2=0x7f88d4100d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:19.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.593+0000 7f88d9a8f700 1 --2- 172.21.15.144:0/1688039404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88d40ff390 0x7f88d4100d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:19.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.593+0000 7f88d9a8f700 1 -- 172.21.15.144:0/1688039404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d00095f0 con 0x7f88d40ffd30 2024-09-10T11:49:19.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.593+0000 7f88da290700 1 --2- 172.21.15.144:0/1688039404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88d40ff390 0x7f88d4100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:49:19.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.593+0000 7f88d9a8f700 1 --2- 172.21.15.144:0/1688039404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d40ffd30 0x7f88d41012c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f88c800c960 tx=0x7f88c800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:19.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.593+0000 7f88c77fe700 1 -- 172.21.15.144:0/1688039404 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88c8007a70 con 0x7f88d40ffd30 2024-09-10T11:49:19.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.593+0000 7f88dc4f4700 1 -- 172.21.15.144:0/1688039404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d41a1ef0 con 0x7f88d40ffd30 2024-09-10T11:49:19.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.593+0000 7f88c77fe700 1 -- 172.21.15.144:0/1688039404 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f88c8007bd0 con 0x7f88d40ffd30 2024-09-10T11:49:19.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.593+0000 7f88c77fe700 1 -- 172.21.15.144:0/1688039404 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88c80186c0 con 0x7f88d40ffd30 2024-09-10T11:49:19.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.593+0000 7f88dc4f4700 1 -- 172.21.15.144:0/1688039404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d41a2250 con 0x7f88d40ffd30 2024-09-10T11:49:19.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.594+0000 7f88dc4f4700 1 -- 172.21.15.144:0/1688039404 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d404ea50 con 0x7f88d40ffd30 2024-09-10T11:49:19.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.595+0000 7f88c77fe700 1 -- 172.21.15.144:0/1688039404 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f88c800faa0 con 0x7f88d40ffd30 2024-09-10T11:49:19.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.595+0000 7f88c77fe700 1 --2- 172.21.15.144:0/1688039404 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f88c006e200 0x7f88c00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:19.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.595+0000 7f88c77fe700 1 -- 172.21.15.144:0/1688039404 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f88c808e090 con 0x7f88d40ffd30 2024-09-10T11:49:19.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.595+0000 7f88da290700 1 --2- 172.21.15.144:0/1688039404 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f88c006e200 0x7f88c00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:19.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.596+0000 7f88da290700 1 --2- 172.21.15.144:0/1688039404 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f88c006e200 0x7f88c00706b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f88d0005b40 tx=0x7f88d001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:19.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.600+0000 7f88c77fe700 1 -- 172.21.15.144:0/1688039404 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f88c805afa0 con 0x7f88d40ffd30 2024-09-10T11:49:19.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.824+0000 7f88dc4f4700 1 -- 172.21.15.144:0/1688039404 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f88d40fcdd0 con 0x7f88c006e200 2024-09-10T11:49:19.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.825+0000 7f88c77fe700 1 -- 172.21.15.144:0/1688039404 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f88d40fcdd0 con 0x7f88c006e200 2024-09-10T11:49:19.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.828+0000 7f88dc4f4700 1 -- 172.21.15.144:0/1688039404 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f88c006e200 msgr2=0x7f88c00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:19.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.828+0000 7f88dc4f4700 1 --2- 172.21.15.144:0/1688039404 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f88c006e200 0x7f88c00706b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f88d0005b40 tx=0x7f88d001a040 comp rx=0 tx=0).stop 2024-09-10T11:49:19.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.828+0000 7f88dc4f4700 1 -- 172.21.15.144:0/1688039404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d40ffd30 msgr2=0x7f88d41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:19.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.828+0000 7f88dc4f4700 1 --2- 172.21.15.144:0/1688039404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d40ffd30 0x7f88d41012c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f88c800c960 tx=0x7f88c800cd20 comp rx=0 tx=0).stop 2024-09-10T11:49:19.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.828+0000 7f88dc4f4700 1 -- 172.21.15.144:0/1688039404 shutdown_connections 2024-09-10T11:49:19.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.828+0000 7f88dc4f4700 1 --2- 172.21.15.144:0/1688039404 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f88c006e200 0x7f88c00706b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:19.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.828+0000 7f88dc4f4700 1 --2- 172.21.15.144:0/1688039404 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f88d40ff390 0x7f88d4100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:19.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.828+0000 7f88dc4f4700 1 --2- 172.21.15.144:0/1688039404 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f88d40ffd30 0x7f88d41012c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:19.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.828+0000 7f88dc4f4700 1 -- 172.21.15.144:0/1688039404 >> 172.21.15.144:0/1688039404 conn(0x7f88d4074f50 msgr2=0x7f88d4103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:19.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.828+0000 7f88dc4f4700 1 -- 172.21.15.144:0/1688039404 shutdown_connections 2024-09-10T11:49:19.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.828+0000 7f88dc4f4700 1 -- 172.21.15.144:0/1688039404 wait complete. 2024-09-10T11:49:19.841 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:49:19.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.934+0000 7fd127bd1700 1 -- 172.21.15.144:0/3233522656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1201003d0 msgr2=0x7fd1201007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:19.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.934+0000 7fd127bd1700 1 --2- 172.21.15.144:0/3233522656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1201003d0 0x7fd1201007a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd118009910 tx=0x7fd118009c20 comp rx=0 tx=0).stop 2024-09-10T11:49:19.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.934+0000 7fd127bd1700 1 -- 172.21.15.144:0/3233522656 shutdown_connections 2024-09-10T11:49:19.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.934+0000 7fd127bd1700 1 --2- 172.21.15.144:0/3233522656 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd120100d70 0x7fd120104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:19.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.934+0000 7fd127bd1700 1 --2- 172.21.15.144:0/3233522656 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd1201003d0 0x7fd1201007a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:19.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.934+0000 7fd127bd1700 1 -- 172.21.15.144:0/3233522656 >> 172.21.15.144:0/3233522656 conn(0x7fd1200fbc90 msgr2=0x7fd1200fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:19.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.934+0000 7fd127bd1700 1 -- 172.21.15.144:0/3233522656 shutdown_connections 2024-09-10T11:49:19.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.934+0000 7fd127bd1700 1 -- 172.21.15.144:0/3233522656 wait complete. 2024-09-10T11:49:19.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.935+0000 7fd127bd1700 1 Processor -- start 2024-09-10T11:49:19.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.935+0000 7fd127bd1700 1 -- start start 2024-09-10T11:49:19.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.935+0000 7fd127bd1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1201003d0 0x7fd120195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:19.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.935+0000 7fd127bd1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd120100d70 0x7fd1201962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:19.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.935+0000 7fd127bd1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd120196980 con 0x7fd120100d70 2024-09-10T11:49:19.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.935+0000 7fd127bd1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12019a710 con 0x7fd1201003d0 2024-09-10T11:49:19.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.935+0000 7fd12516c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd120100d70 0x7fd1201962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:19.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.935+0000 7fd12516c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd120100d70 0x7fd1201962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36426/0 (socket says 172.21.15.144:36426) 2024-09-10T11:49:19.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.935+0000 7fd12516c700 1 -- 172.21.15.144:0/2244128205 learned_addr learned my addr 172.21.15.144:0/2244128205 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:49:19.939 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.936+0000 7fd12596d700 1 --2- 172.21.15.144:0/2244128205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1201003d0 0x7fd120195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:19.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.936+0000 7fd12516c700 1 -- 172.21.15.144:0/2244128205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1201003d0 msgr2=0x7fd120195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:19.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.936+0000 7fd12516c700 1 --2- 172.21.15.144:0/2244128205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1201003d0 0x7fd120195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:19.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.936+0000 7fd12516c700 1 -- 172.21.15.144:0/2244128205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1180095f0 con 0x7fd120100d70 2024-09-10T11:49:19.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.936+0000 7fd12516c700 1 --2- 172.21.15.144:0/2244128205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd120100d70 0x7fd1201962a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd11400d8d0 tx=0x7fd11400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:19.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.936+0000 7fd11effd700 1 -- 172.21.15.144:0/2244128205 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1140041f0 con 0x7fd120100d70 2024-09-10T11:49:19.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.936+0000 7fd11effd700 1 -- 172.21.15.144:0/2244128205 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd114007620 con 0x7fd120100d70 2024-09-10T11:49:19.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.936+0000 7fd127bd1700 1 -- 172.21.15.144:0/2244128205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd12019a9f0 con 0x7fd120100d70 2024-09-10T11:49:19.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.936+0000 7fd11effd700 1 -- 172.21.15.144:0/2244128205 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd114003c50 con 0x7fd120100d70 2024-09-10T11:49:19.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.936+0000 7fd127bd1700 1 -- 172.21.15.144:0/2244128205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd12019af40 con 0x7fd120100d70 2024-09-10T11:49:19.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.937+0000 7fd127bd1700 1 -- 172.21.15.144:0/2244128205 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1201086e0 con 0x7fd120100d70 2024-09-10T11:49:19.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.937+0000 7fd11effd700 1 -- 172.21.15.144:0/2244128205 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd114004350 con 0x7fd120100d70 2024-09-10T11:49:19.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.938+0000 7fd11effd700 1 --2- 172.21.15.144:0/2244128205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd10806e200 0x7fd1080706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:19.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.938+0000 7fd11effd700 1 -- 172.21.15.144:0/2244128205 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd11408dc30 con 0x7fd120100d70 2024-09-10T11:49:19.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.938+0000 7fd12596d700 1 --2- 172.21.15.144:0/2244128205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd10806e200 0x7fd1080706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:19.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.938+0000 7fd12596d700 1 --2- 172.21.15.144:0/2244128205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd10806e200 0x7fd1080706b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd11800b5c0 tx=0x7fd11801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:19.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:19.942+0000 7fd11effd700 1 -- 172.21.15.144:0/2244128205 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd114057010 con 0x7fd120100d70 2024-09-10T11:49:20.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.162+0000 7fd127bd1700 1 -- 172.21.15.144:0/2244128205 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd12019b220 con 0x7fd10806e200 2024-09-10T11:49:20.164 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:19 smithi144 ceph-mon[87436]: pgmap v520: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:20.164 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:19 smithi144 ceph-mon[87436]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:20.165 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.163+0000 7fd11effd700 1 -- 172.21.15.144:0/2244128205 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd12019b220 con 0x7fd10806e200 2024-09-10T11:49:20.168 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.166+0000 7fd127bd1700 1 -- 172.21.15.144:0/2244128205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd10806e200 msgr2=0x7fd1080706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:20.168 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.166+0000 7fd127bd1700 1 --2- 172.21.15.144:0/2244128205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd10806e200 0x7fd1080706b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd11800b5c0 tx=0x7fd11801a040 comp rx=0 tx=0).stop 2024-09-10T11:49:20.168 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.166+0000 7fd127bd1700 1 -- 172.21.15.144:0/2244128205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd120100d70 msgr2=0x7fd1201962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:20.168 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.166+0000 7fd127bd1700 1 --2- 172.21.15.144:0/2244128205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd120100d70 0x7fd1201962a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd11400d8d0 tx=0x7fd11400dc90 comp rx=0 tx=0).stop 2024-09-10T11:49:20.168 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.166+0000 7fd127bd1700 1 -- 172.21.15.144:0/2244128205 shutdown_connections 2024-09-10T11:49:20.168 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.166+0000 7fd127bd1700 1 --2- 172.21.15.144:0/2244128205 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd10806e200 0x7fd1080706b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.168 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.166+0000 7fd127bd1700 1 --2- 172.21.15.144:0/2244128205 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd1201003d0 0x7fd120195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.168 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.166+0000 7fd127bd1700 1 --2- 172.21.15.144:0/2244128205 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd120100d70 0x7fd1201962a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.169 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.166+0000 7fd127bd1700 1 -- 172.21.15.144:0/2244128205 >> 172.21.15.144:0/2244128205 conn(0x7fd1200fbc90 msgr2=0x7fd1200fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:20.169 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.166+0000 7fd127bd1700 1 -- 172.21.15.144:0/2244128205 shutdown_connections 2024-09-10T11:49:20.169 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.166+0000 7fd127bd1700 1 -- 172.21.15.144:0/2244128205 wait complete. 2024-09-10T11:49:20.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.270+0000 7f3fcce52700 1 -- 172.21.15.144:0/3060944255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc8100cb0 msgr2=0x7f3fc8109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:20.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.270+0000 7f3fcce52700 1 --2- 172.21.15.144:0/3060944255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc8100cb0 0x7f3fc8109410 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc009910 tx=0x7f3fbc009c20 comp rx=0 tx=0).stop 2024-09-10T11:49:20.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.271+0000 7f3fcce52700 1 -- 172.21.15.144:0/3060944255 shutdown_connections 2024-09-10T11:49:20.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.271+0000 7f3fcce52700 1 --2- 172.21.15.144:0/3060944255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc8100cb0 0x7f3fc8109410 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.271+0000 7f3fcce52700 1 --2- 172.21.15.144:0/3060944255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fc81003a0 0x7f3fc8100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.271+0000 7f3fcce52700 1 -- 172.21.15.144:0/3060944255 >> 172.21.15.144:0/3060944255 conn(0x7f3fc8074850 msgr2=0x7f3fc8074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:20.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.271+0000 7f3fcce52700 1 -- 172.21.15.144:0/3060944255 shutdown_connections 2024-09-10T11:49:20.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.271+0000 7f3fcce52700 1 -- 172.21.15.144:0/3060944255 wait complete. 2024-09-10T11:49:20.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.271+0000 7f3fcce52700 1 Processor -- start 2024-09-10T11:49:20.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fcce52700 1 -- start start 2024-09-10T11:49:20.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fcce52700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc81003a0 0x7f3fc819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:20.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fcce52700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fc8100cb0 0x7f3fc819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:20.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fcce52700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc819b5d0 con 0x7f3fc81003a0 2024-09-10T11:49:20.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fcce52700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fc8194a80 con 0x7f3fc8100cb0 2024-09-10T11:49:20.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fc659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc81003a0 0x7f3fc819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:20.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fc659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc81003a0 0x7f3fc819aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36450/0 (socket says 172.21.15.144:36450) 2024-09-10T11:49:20.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fc659c700 1 -- 172.21.15.144:0/2170459003 learned_addr learned my addr 172.21.15.144:0/2170459003 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:49:20.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fc5d9b700 1 --2- 172.21.15.144:0/2170459003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fc8100cb0 0x7f3fc819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:20.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fc659c700 1 -- 172.21.15.144:0/2170459003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fc8100cb0 msgr2=0x7f3fc819af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:20.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fc659c700 1 --2- 172.21.15.144:0/2170459003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fc8100cb0 0x7f3fc819af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fc659c700 1 -- 172.21.15.144:0/2170459003 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc0095f0 con 0x7f3fc81003a0 2024-09-10T11:49:20.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fc659c700 1 --2- 172.21.15.144:0/2170459003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc81003a0 0x7f3fc819aa00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800c8a0 tx=0x7f3fb800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:20.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fb77fe700 1 -- 172.21.15.144:0/2170459003 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fb80078e0 con 0x7f3fc81003a0 2024-09-10T11:49:20.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fcce52700 1 -- 172.21.15.144:0/2170459003 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fc8194d60 con 0x7f3fc81003a0 2024-09-10T11:49:20.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fb77fe700 1 -- 172.21.15.144:0/2170459003 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3fb800f450 con 0x7f3fc81003a0 2024-09-10T11:49:20.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.272+0000 7f3fb77fe700 1 -- 172.21.15.144:0/2170459003 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3fb800e5e0 con 0x7f3fc81003a0 2024-09-10T11:49:20.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.273+0000 7f3fcce52700 1 -- 172.21.15.144:0/2170459003 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fc81952b0 con 0x7f3fc81003a0 2024-09-10T11:49:20.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.274+0000 7f3fcce52700 1 -- 172.21.15.144:0/2170459003 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fc8106b10 con 0x7f3fc81003a0 2024-09-10T11:49:20.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.274+0000 7f3fb77fe700 1 -- 172.21.15.144:0/2170459003 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3fb800f5c0 con 0x7f3fc81003a0 2024-09-10T11:49:20.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.274+0000 7f3fb77fe700 1 --2- 172.21.15.144:0/2170459003 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3fb006e200 0x7f3fb00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:20.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.275+0000 7f3fb77fe700 1 -- 172.21.15.144:0/2170459003 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3fb808e0b0 con 0x7f3fc81003a0 2024-09-10T11:49:20.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.275+0000 7f3fc5d9b700 1 --2- 172.21.15.144:0/2170459003 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3fb006e200 0x7f3fb00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:20.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.275+0000 7f3fc5d9b700 1 --2- 172.21.15.144:0/2170459003 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3fb006e200 0x7f3fb00706b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80fde10 tx=0x7f3fbc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:20.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.279+0000 7f3fb77fe700 1 -- 172.21.15.144:0/2170459003 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3fb805afc0 con 0x7f3fc81003a0 2024-09-10T11:49:20.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:19 smithi178 ceph-mon[65940]: pgmap v520: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:20.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:19 smithi178 ceph-mon[65940]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:20.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.490+0000 7f3fcce52700 1 -- 172.21.15.144:0/2170459003 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fc81955f0 con 0x7f3fb006e200 2024-09-10T11:49:20.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.500+0000 7f3fb77fe700 1 -- 172.21.15.144:0/2170459003 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f3fc81955f0 con 0x7f3fb006e200 2024-09-10T11:49:20.502 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:49:20.502 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (17m) 11m ago 28m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:49:20.503 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (28m) 11m ago 28m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:49:20.503 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (27m) 11m ago 27m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:49:20.503 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (15m) 11m ago 28m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:49:20.503 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (15m) 11m ago 27m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:49:20.503 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (17m) 11m ago 27m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:49:20.503 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (11m) 11m ago 22m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:49:20.503 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (11m) 11m ago 22m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:49:20.503 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (22m) 11m ago 22m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:49:20.503 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (11m) 11m ago 22m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:49:20.503 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (19m) 11m ago 29m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:49:20.504 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (18m) 11m ago 27m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:49:20.504 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (16m) 11m ago 29m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:49:20.504 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (15m) 11m ago 27m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:49:20.504 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (18m) 11m ago 28m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:49:20.504 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (18m) 11m ago 27m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:49:20.504 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (14m) 11m ago 26m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:49:20.504 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (14m) 11m ago 25m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:49:20.504 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (13m) 11m ago 25m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:49:20.505 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (13m) 11m ago 24m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:49:20.505 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (12m) 11m ago 24m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:49:20.505 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (11m) 11m ago 24m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:49:20.505 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (17m) 11m ago 27m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:49:20.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.502+0000 7f3fcce52700 1 -- 172.21.15.144:0/2170459003 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3fb006e200 msgr2=0x7f3fb00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:20.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.502+0000 7f3fcce52700 1 --2- 172.21.15.144:0/2170459003 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3fb006e200 0x7f3fb00706b0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f3fc80fde10 tx=0x7f3fbc01a040 comp rx=0 tx=0).stop 2024-09-10T11:49:20.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.502+0000 7f3fcce52700 1 -- 172.21.15.144:0/2170459003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc81003a0 msgr2=0x7f3fc819aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:20.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.502+0000 7f3fcce52700 1 --2- 172.21.15.144:0/2170459003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc81003a0 0x7f3fc819aa00 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800c8a0 tx=0x7f3fb800cbb0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.502+0000 7f3fcce52700 1 -- 172.21.15.144:0/2170459003 shutdown_connections 2024-09-10T11:49:20.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.502+0000 7f3fcce52700 1 --2- 172.21.15.144:0/2170459003 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3fb006e200 0x7f3fb00706b0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.502+0000 7f3fcce52700 1 --2- 172.21.15.144:0/2170459003 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3fc81003a0 0x7f3fc819aa00 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.502+0000 7f3fcce52700 1 --2- 172.21.15.144:0/2170459003 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3fc8100cb0 0x7f3fc819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.502+0000 7f3fcce52700 1 -- 172.21.15.144:0/2170459003 >> 172.21.15.144:0/2170459003 conn(0x7f3fc8074850 msgr2=0x7f3fc8103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:20.506 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.502+0000 7f3fcce52700 1 -- 172.21.15.144:0/2170459003 shutdown_connections 2024-09-10T11:49:20.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.502+0000 7f3fcce52700 1 -- 172.21.15.144:0/2170459003 wait complete. 2024-09-10T11:49:20.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.605+0000 7f4ccdc2e700 1 -- 172.21.15.144:0/578676900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cc80ffd30 msgr2=0x7f4cc8103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:20.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.605+0000 7f4ccdc2e700 1 --2- 172.21.15.144:0/578676900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cc80ffd30 0x7f4cc8103ce0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f4cb0009910 tx=0x7f4cb0009c20 comp rx=0 tx=0).stop 2024-09-10T11:49:20.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.605+0000 7f4ccdc2e700 1 -- 172.21.15.144:0/578676900 shutdown_connections 2024-09-10T11:49:20.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.605+0000 7f4ccdc2e700 1 --2- 172.21.15.144:0/578676900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cc80ffd30 0x7f4cc8103ce0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.605+0000 7f4ccdc2e700 1 --2- 172.21.15.144:0/578676900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc80ff390 0x7f4cc80ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.605+0000 7f4ccdc2e700 1 -- 172.21.15.144:0/578676900 >> 172.21.15.144:0/578676900 conn(0x7f4cc8074f50 msgr2=0x7f4cc8075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:20.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.605+0000 7f4ccdc2e700 1 -- 172.21.15.144:0/578676900 shutdown_connections 2024-09-10T11:49:20.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.605+0000 7f4ccdc2e700 1 -- 172.21.15.144:0/578676900 wait complete. 2024-09-10T11:49:20.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.606+0000 7f4ccdc2e700 1 Processor -- start 2024-09-10T11:49:20.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.606+0000 7f4ccdc2e700 1 -- start start 2024-09-10T11:49:20.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.606+0000 7f4ccdc2e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc80ff390 0x7f4cc819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:20.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.606+0000 7f4ccdc2e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cc80ffd30 0x7f4cc819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:20.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.606+0000 7f4ccdc2e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc819b5b0 con 0x7f4cc80ffd30 2024-09-10T11:49:20.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.606+0000 7f4ccdc2e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cc8194a60 con 0x7f4cc80ff390 2024-09-10T11:49:20.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.606+0000 7f4cc6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cc80ffd30 0x7f4cc819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:20.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.606+0000 7f4cc6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cc80ffd30 0x7f4cc819af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36476/0 (socket says 172.21.15.144:36476) 2024-09-10T11:49:20.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.606+0000 7f4cc6ffd700 1 -- 172.21.15.144:0/4161159936 learned_addr learned my addr 172.21.15.144:0/4161159936 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:49:20.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.606+0000 7f4cc77fe700 1 --2- 172.21.15.144:0/4161159936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc80ff390 0x7f4cc819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:20.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.607+0000 7f4cc6ffd700 1 -- 172.21.15.144:0/4161159936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc80ff390 msgr2=0x7f4cc819a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:20.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.607+0000 7f4cc6ffd700 1 --2- 172.21.15.144:0/4161159936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc80ff390 0x7f4cc819a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.607+0000 7f4cc6ffd700 1 -- 172.21.15.144:0/4161159936 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb00095f0 con 0x7f4cc80ffd30 2024-09-10T11:49:20.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.607+0000 7f4cc6ffd700 1 --2- 172.21.15.144:0/4161159936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cc80ffd30 0x7f4cc819af20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000bda0 tx=0x7f4cb000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:20.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.607+0000 7f4cc4ff9700 1 -- 172.21.15.144:0/4161159936 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cb001d070 con 0x7f4cc80ffd30 2024-09-10T11:49:20.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.607+0000 7f4cc4ff9700 1 -- 172.21.15.144:0/4161159936 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cb000f460 con 0x7f4cc80ffd30 2024-09-10T11:49:20.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.607+0000 7f4ccdc2e700 1 -- 172.21.15.144:0/4161159936 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc8194ce0 con 0x7f4cc80ffd30 2024-09-10T11:49:20.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.607+0000 7f4cc4ff9700 1 -- 172.21.15.144:0/4161159936 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cb0017630 con 0x7f4cc80ffd30 2024-09-10T11:49:20.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.607+0000 7f4ccdc2e700 1 -- 172.21.15.144:0/4161159936 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc81951d0 con 0x7f4cc80ffd30 2024-09-10T11:49:20.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.608+0000 7f4ccdc2e700 1 -- 172.21.15.144:0/4161159936 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cc8107600 con 0x7f4cc80ffd30 2024-09-10T11:49:20.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.608+0000 7f4cc4ff9700 1 -- 172.21.15.144:0/4161159936 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4cb0003a60 con 0x7f4cc80ffd30 2024-09-10T11:49:20.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.609+0000 7f4cc4ff9700 1 --2- 172.21.15.144:0/4161159936 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cb406e200 0x7f4cb40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:20.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.609+0000 7f4cc4ff9700 1 -- 172.21.15.144:0/4161159936 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4cb008f520 con 0x7f4cc80ffd30 2024-09-10T11:49:20.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.609+0000 7f4cc77fe700 1 --2- 172.21.15.144:0/4161159936 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cb406e200 0x7f4cb40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:20.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.609+0000 7f4cc77fe700 1 --2- 172.21.15.144:0/4161159936 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cb406e200 0x7f4cb40706b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8005950 tx=0x7f4cb80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:20.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.613+0000 7f4cc4ff9700 1 -- 172.21.15.144:0/4161159936 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4cb005c430 con 0x7f4cc80ffd30 2024-09-10T11:49:20.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.903+0000 7f4ccdc2e700 1 -- 172.21.15.144:0/4161159936 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4cc804ea50 con 0x7f4cc80ffd30 2024-09-10T11:49:20.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.903+0000 7f4cc4ff9700 1 -- 172.21.15.144:0/4161159936 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f4cb005bfc0 con 0x7f4cc80ffd30 2024-09-10T11:49:20.905 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:49:20.905 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:49:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:49:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:49:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:49:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:49:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:49:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:49:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:49:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:49:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:49:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:49:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:49:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:49:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:49:20.906 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:49:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:49:20.907 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:49:20.907 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:49:20.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.906+0000 7f4ccdc2e700 1 -- 172.21.15.144:0/4161159936 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cb406e200 msgr2=0x7f4cb40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:20.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.906+0000 7f4ccdc2e700 1 --2- 172.21.15.144:0/4161159936 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cb406e200 0x7f4cb40706b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8005950 tx=0x7f4cb80058e0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.906+0000 7f4ccdc2e700 1 -- 172.21.15.144:0/4161159936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cc80ffd30 msgr2=0x7f4cc819af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:20.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.906+0000 7f4ccdc2e700 1 --2- 172.21.15.144:0/4161159936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cc80ffd30 0x7f4cc819af20 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4cb000bda0 tx=0x7f4cb000be80 comp rx=0 tx=0).stop 2024-09-10T11:49:20.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.906+0000 7f4ccdc2e700 1 -- 172.21.15.144:0/4161159936 shutdown_connections 2024-09-10T11:49:20.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.906+0000 7f4ccdc2e700 1 --2- 172.21.15.144:0/4161159936 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cb406e200 0x7f4cb40706b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.906+0000 7f4ccdc2e700 1 --2- 172.21.15.144:0/4161159936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4cc80ff390 0x7f4cc819a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.906+0000 7f4ccdc2e700 1 --2- 172.21.15.144:0/4161159936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4cc80ffd30 0x7f4cc819af20 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:20.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.906+0000 7f4ccdc2e700 1 -- 172.21.15.144:0/4161159936 >> 172.21.15.144:0/4161159936 conn(0x7f4cc8074f50 msgr2=0x7f4cc80fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:20.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.906+0000 7f4ccdc2e700 1 -- 172.21.15.144:0/4161159936 shutdown_connections 2024-09-10T11:49:20.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:20.906+0000 7f4ccdc2e700 1 -- 172.21.15.144:0/4161159936 wait complete. 2024-09-10T11:49:21.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.013+0000 7f5570fc5700 1 -- 172.21.15.144:0/388259391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f556c101320 msgr2=0x7f556c1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:21.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.013+0000 7f5570fc5700 1 --2- 172.21.15.144:0/388259391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f556c101320 0x7f556c1016f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f555c009940 tx=0x7f555c009c50 comp rx=0 tx=0).stop 2024-09-10T11:49:21.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.013+0000 7f5570fc5700 1 -- 172.21.15.144:0/388259391 shutdown_connections 2024-09-10T11:49:21.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.013+0000 7f5570fc5700 1 --2- 172.21.15.144:0/388259391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556c101c30 0x7f556c10a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.013+0000 7f5570fc5700 1 --2- 172.21.15.144:0/388259391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f556c101320 0x7f556c1016f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.013+0000 7f5570fc5700 1 -- 172.21.15.144:0/388259391 >> 172.21.15.144:0/388259391 conn(0x7f556c0fab80 msgr2=0x7f556c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:21.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.013+0000 7f5570fc5700 1 -- 172.21.15.144:0/388259391 shutdown_connections 2024-09-10T11:49:21.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.013+0000 7f5570fc5700 1 -- 172.21.15.144:0/388259391 wait complete. 2024-09-10T11:49:21.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.013+0000 7f5570fc5700 1 Processor -- start 2024-09-10T11:49:21.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.013+0000 7f5570fc5700 1 -- start start 2024-09-10T11:49:21.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.014+0000 7f5570fc5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556c101320 0x7f556c19bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:21.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.014+0000 7f5570fc5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f556c101c30 0x7f556c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:21.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.014+0000 7f5570fc5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c19c680 con 0x7f556c101c30 2024-09-10T11:49:21.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.014+0000 7f5570fc5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c195b30 con 0x7f556c101320 2024-09-10T11:49:21.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.014+0000 7f5569d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f556c101c30 0x7f556c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:21.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.014+0000 7f5569d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f556c101c30 0x7f556c19bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36490/0 (socket says 172.21.15.144:36490) 2024-09-10T11:49:21.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.014+0000 7f5569d9b700 1 -- 172.21.15.144:0/3615632140 learned_addr learned my addr 172.21.15.144:0/3615632140 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:49:21.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.014+0000 7f556a59c700 1 --2- 172.21.15.144:0/3615632140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556c101320 0x7f556c19bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:21.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.014+0000 7f5569d9b700 1 -- 172.21.15.144:0/3615632140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556c101320 msgr2=0x7f556c19bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:21.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.014+0000 7f5569d9b700 1 --2- 172.21.15.144:0/3615632140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556c101320 0x7f556c19bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.014+0000 7f5569d9b700 1 -- 172.21.15.144:0/3615632140 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f555c0095f0 con 0x7f556c101c30 2024-09-10T11:49:21.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.014+0000 7f5569d9b700 1 --2- 172.21.15.144:0/3615632140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f556c101c30 0x7f556c19bff0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f556000d8d0 tx=0x7f556000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:21.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.014+0000 7f555b7fe700 1 -- 172.21.15.144:0/3615632140 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55600098a0 con 0x7f556c101c30 2024-09-10T11:49:21.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.015+0000 7f555b7fe700 1 -- 172.21.15.144:0/3615632140 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5560010460 con 0x7f556c101c30 2024-09-10T11:49:21.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.015+0000 7f5570fc5700 1 -- 172.21.15.144:0/3615632140 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556c195e10 con 0x7f556c101c30 2024-09-10T11:49:21.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.015+0000 7f555b7fe700 1 -- 172.21.15.144:0/3615632140 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f556000f5f0 con 0x7f556c101c30 2024-09-10T11:49:21.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.015+0000 7f5570fc5700 1 -- 172.21.15.144:0/3615632140 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c196360 con 0x7f556c101c30 2024-09-10T11:49:21.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.016+0000 7f5570fc5700 1 -- 172.21.15.144:0/3615632140 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556c107800 con 0x7f556c101c30 2024-09-10T11:49:21.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.016+0000 7f555b7fe700 1 -- 172.21.15.144:0/3615632140 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5560009a00 con 0x7f556c101c30 2024-09-10T11:49:21.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.017+0000 7f555b7fe700 1 --2- 172.21.15.144:0/3615632140 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f555406e200 0x7f55540706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:21.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.017+0000 7f555b7fe700 1 -- 172.21.15.144:0/3615632140 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f556008ded0 con 0x7f556c101c30 2024-09-10T11:49:21.021 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.017+0000 7f556a59c700 1 --2- 172.21.15.144:0/3615632140 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f555406e200 0x7f55540706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:21.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.017+0000 7f556a59c700 1 --2- 172.21.15.144:0/3615632140 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f555406e200 0x7f55540706b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f555c006010 tx=0x7f555c005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:21.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.021+0000 7f555b7fe700 1 -- 172.21.15.144:0/3615632140 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f556005ae60 con 0x7f556c101c30 2024-09-10T11:49:21.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:20 smithi144 ceph-mon[87436]: from='client.34992 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:21.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:20 smithi144 ceph-mon[87436]: from='client.34996 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:21.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.270+0000 7f5570fc5700 1 -- 172.21.15.144:0/3615632140 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f556c04ea50 con 0x7f556c101c30 2024-09-10T11:49:21.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.270+0000 7f555b7fe700 1 -- 172.21.15.144:0/3615632140 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f5560020790 con 0x7f556c101c30 2024-09-10T11:49:21.272 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:49:21.273 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:49:21.273 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:49:21.273 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:49:21.273 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:49:21.273 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:49:21.273 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:49:21.273 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:49:21.273 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:49:21.273 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:49:21.273 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:49:21.273 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:49:21.273 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:49:21.273 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:49:21.274 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:49:21.274 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:49:21.274 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:49:21.274 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:49:21.274 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:49:21.274 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:49:21.274 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:49:21.274 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:49:21.274 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:49:21.274 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:49:21.274 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:49:21.274 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:49:21.274 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:49:21.275 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:49:21.275 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:49:21.275 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:49:21.275 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:49:21.275 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:49:21.275 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:49:21.275 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:49:21.275 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:49:21.275 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:49:21.275 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:49:21.275 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:49:21.275 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:49:21.275 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:49:21.275 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:49:21.276 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:49:21.276 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:49:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.272+0000 7f5570fc5700 1 -- 172.21.15.144:0/3615632140 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f555406e200 msgr2=0x7f55540706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.272+0000 7f5570fc5700 1 --2- 172.21.15.144:0/3615632140 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f555406e200 0x7f55540706b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f555c006010 tx=0x7f555c005a90 comp rx=0 tx=0).stop 2024-09-10T11:49:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.272+0000 7f5570fc5700 1 -- 172.21.15.144:0/3615632140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f556c101c30 msgr2=0x7f556c19bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.272+0000 7f5570fc5700 1 --2- 172.21.15.144:0/3615632140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f556c101c30 0x7f556c19bff0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f556000d8d0 tx=0x7f556000dbe0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.273+0000 7f5570fc5700 1 -- 172.21.15.144:0/3615632140 shutdown_connections 2024-09-10T11:49:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.273+0000 7f5570fc5700 1 --2- 172.21.15.144:0/3615632140 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f555406e200 0x7f55540706b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.273+0000 7f5570fc5700 1 --2- 172.21.15.144:0/3615632140 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f556c101320 0x7f556c19bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.273+0000 7f5570fc5700 1 --2- 172.21.15.144:0/3615632140 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f556c101c30 0x7f556c19bff0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.273+0000 7f5570fc5700 1 -- 172.21.15.144:0/3615632140 >> 172.21.15.144:0/3615632140 conn(0x7f556c0fab80 msgr2=0x7f556c104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.273+0000 7f5570fc5700 1 -- 172.21.15.144:0/3615632140 shutdown_connections 2024-09-10T11:49:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.273+0000 7f5570fc5700 1 -- 172.21.15.144:0/3615632140 wait complete. 2024-09-10T11:49:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:49:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:20 smithi178 ceph-mon[65940]: from='client.34992 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:20 smithi178 ceph-mon[65940]: from='client.34996 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:21.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.379+0000 7fd2a13ec700 1 -- 172.21.15.144:0/2143038770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c101320 msgr2=0x7fd29c1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:21.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.379+0000 7fd2a13ec700 1 --2- 172.21.15.144:0/2143038770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c101320 0x7fd29c1016f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009910 tx=0x7fd28c009c20 comp rx=0 tx=0).stop 2024-09-10T11:49:21.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.379+0000 7fd2a13ec700 1 -- 172.21.15.144:0/2143038770 shutdown_connections 2024-09-10T11:49:21.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.379+0000 7fd2a13ec700 1 --2- 172.21.15.144:0/2143038770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd29c101c30 0x7fd29c10a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.379+0000 7fd2a13ec700 1 --2- 172.21.15.144:0/2143038770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c101320 0x7fd29c1016f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.379+0000 7fd2a13ec700 1 -- 172.21.15.144:0/2143038770 >> 172.21.15.144:0/2143038770 conn(0x7fd29c0fab80 msgr2=0x7fd29c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:21.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.379+0000 7fd2a13ec700 1 -- 172.21.15.144:0/2143038770 shutdown_connections 2024-09-10T11:49:21.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.379+0000 7fd2a13ec700 1 -- 172.21.15.144:0/2143038770 wait complete. 2024-09-10T11:49:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.380+0000 7fd2a13ec700 1 Processor -- start 2024-09-10T11:49:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.380+0000 7fd2a13ec700 1 -- start start 2024-09-10T11:49:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.380+0000 7fd2a13ec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c101320 0x7fd29c195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.380+0000 7fd2a13ec700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd29c101c30 0x7fd29c196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:21.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.380+0000 7fd2a13ec700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c1968e0 con 0x7fd29c101320 2024-09-10T11:49:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.380+0000 7fd2a13ec700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29c196a20 con 0x7fd29c101c30 2024-09-10T11:49:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.380+0000 7fd29affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c101320 0x7fd29c195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.380+0000 7fd29affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c101320 0x7fd29c195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36516/0 (socket says 172.21.15.144:36516) 2024-09-10T11:49:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.380+0000 7fd29affd700 1 -- 172.21.15.144:0/1506098543 learned_addr learned my addr 172.21.15.144:0/1506098543 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:49:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.380+0000 7fd29a7fc700 1 --2- 172.21.15.144:0/1506098543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd29c101c30 0x7fd29c196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.380+0000 7fd29affd700 1 -- 172.21.15.144:0/1506098543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd29c101c30 msgr2=0x7fd29c196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.380+0000 7fd29affd700 1 --2- 172.21.15.144:0/1506098543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd29c101c30 0x7fd29c196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.380+0000 7fd29affd700 1 -- 172.21.15.144:0/1506098543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd28c0095f0 con 0x7fd29c101320 2024-09-10T11:49:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.381+0000 7fd29affd700 1 --2- 172.21.15.144:0/1506098543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c101320 0x7fd29c195d50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd28c006010 tx=0x7fd28c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.381+0000 7fd283fff700 1 -- 172.21.15.144:0/1506098543 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd28c01d070 con 0x7fd29c101320 2024-09-10T11:49:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.381+0000 7fd283fff700 1 -- 172.21.15.144:0/1506098543 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd28c004fb0 con 0x7fd29c101320 2024-09-10T11:49:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.381+0000 7fd2a13ec700 1 -- 172.21.15.144:0/1506098543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29c19a810 con 0x7fd29c101320 2024-09-10T11:49:21.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.381+0000 7fd283fff700 1 -- 172.21.15.144:0/1506098543 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd28c022ea0 con 0x7fd29c101320 2024-09-10T11:49:21.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.381+0000 7fd2a13ec700 1 -- 172.21.15.144:0/1506098543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29c19ad00 con 0x7fd29c101320 2024-09-10T11:49:21.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.382+0000 7fd2a13ec700 1 -- 172.21.15.144:0/1506098543 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29c04ea50 con 0x7fd29c101320 2024-09-10T11:49:21.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.382+0000 7fd283fff700 1 -- 172.21.15.144:0/1506098543 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd28c022470 con 0x7fd29c101320 2024-09-10T11:49:21.386 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.383+0000 7fd283fff700 1 --2- 172.21.15.144:0/1506098543 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd28406e200 0x7fd2840706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:21.386 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.383+0000 7fd283fff700 1 -- 172.21.15.144:0/1506098543 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd28c08f6b0 con 0x7fd29c101320 2024-09-10T11:49:21.386 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.383+0000 7fd29a7fc700 1 --2- 172.21.15.144:0/1506098543 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd28406e200 0x7fd2840706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:21.386 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.383+0000 7fd29a7fc700 1 --2- 172.21.15.144:0/1506098543 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd28406e200 0x7fd2840706b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd29c197370 tx=0x7fd290005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:21.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.387+0000 7fd283fff700 1 -- 172.21.15.144:0/1506098543 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd28c058aa0 con 0x7fd29c101320 2024-09-10T11:49:21.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.608+0000 7fd2a13ec700 1 -- 172.21.15.144:0/1506098543 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd29c107450 con 0x7fd28406e200 2024-09-10T11:49:21.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.609+0000 7fd283fff700 1 -- 172.21.15.144:0/1506098543 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd29c107450 con 0x7fd28406e200 2024-09-10T11:49:21.611 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:49:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:49:21.611 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:49:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:49:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:49:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:49:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:49:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:49:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:49:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:49:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:49:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:49:21.612 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:49:21.612 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:49:21.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.612+0000 7fd2a13ec700 1 -- 172.21.15.144:0/1506098543 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd28406e200 msgr2=0x7fd2840706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:21.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.612+0000 7fd2a13ec700 1 --2- 172.21.15.144:0/1506098543 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd28406e200 0x7fd2840706b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fd29c197370 tx=0x7fd290005d50 comp rx=0 tx=0).stop 2024-09-10T11:49:21.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.612+0000 7fd2a13ec700 1 -- 172.21.15.144:0/1506098543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c101320 msgr2=0x7fd29c195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:21.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.612+0000 7fd2a13ec700 1 --2- 172.21.15.144:0/1506098543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c101320 0x7fd29c195d50 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fd28c006010 tx=0x7fd28c00be20 comp rx=0 tx=0).stop 2024-09-10T11:49:21.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.612+0000 7fd2a13ec700 1 -- 172.21.15.144:0/1506098543 shutdown_connections 2024-09-10T11:49:21.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.612+0000 7fd2a13ec700 1 --2- 172.21.15.144:0/1506098543 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd28406e200 0x7fd2840706b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.612+0000 7fd2a13ec700 1 --2- 172.21.15.144:0/1506098543 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd29c101320 0x7fd29c195d50 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.612+0000 7fd2a13ec700 1 --2- 172.21.15.144:0/1506098543 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd29c101c30 0x7fd29c196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.612+0000 7fd2a13ec700 1 -- 172.21.15.144:0/1506098543 >> 172.21.15.144:0/1506098543 conn(0x7fd29c0fab80 msgr2=0x7fd29c104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:21.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.612+0000 7fd2a13ec700 1 -- 172.21.15.144:0/1506098543 shutdown_connections 2024-09-10T11:49:21.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.612+0000 7fd2a13ec700 1 -- 172.21.15.144:0/1506098543 wait complete. 2024-09-10T11:49:21.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.717+0000 7f78d5dd4700 1 -- 172.21.15.144:0/1054789660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78d0100d50 msgr2=0x7f78d0104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:21.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.717+0000 7f78d5dd4700 1 --2- 172.21.15.144:0/1054789660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78d0100d50 0x7f78d0104d20 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009910 tx=0x7f78c4009c20 comp rx=0 tx=0).stop 2024-09-10T11:49:21.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.717+0000 7f78d5dd4700 1 -- 172.21.15.144:0/1054789660 shutdown_connections 2024-09-10T11:49:21.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.717+0000 7f78d5dd4700 1 --2- 172.21.15.144:0/1054789660 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78d0100d50 0x7f78d0104d20 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.717+0000 7f78d5dd4700 1 --2- 172.21.15.144:0/1054789660 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d01003b0 0x7f78d0100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.717+0000 7f78d5dd4700 1 -- 172.21.15.144:0/1054789660 >> 172.21.15.144:0/1054789660 conn(0x7f78d00fbc30 msgr2=0x7f78d00fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:21.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.717+0000 7f78d5dd4700 1 -- 172.21.15.144:0/1054789660 shutdown_connections 2024-09-10T11:49:21.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.717+0000 7f78d5dd4700 1 -- 172.21.15.144:0/1054789660 wait complete. 2024-09-10T11:49:21.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.717+0000 7f78d5dd4700 1 Processor -- start 2024-09-10T11:49:21.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.717+0000 7f78d5dd4700 1 -- start start 2024-09-10T11:49:21.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78d5dd4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78d01003b0 0x7f78d0195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:21.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78d5dd4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d0100d50 0x7f78d0196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:21.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78d5dd4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d01969e0 con 0x7f78d01003b0 2024-09-10T11:49:21.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78d5dd4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d019a770 con 0x7f78d0100d50 2024-09-10T11:49:21.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78cf7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78d01003b0 0x7f78d0195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:21.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78cf7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78d01003b0 0x7f78d0195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36530/0 (socket says 172.21.15.144:36530) 2024-09-10T11:49:21.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78cf7fe700 1 -- 172.21.15.144:0/3490677958 learned_addr learned my addr 172.21.15.144:0/3490677958 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:49:21.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78cf7fe700 1 -- 172.21.15.144:0/3490677958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d0100d50 msgr2=0x7f78d0196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:21.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78ceffd700 1 --2- 172.21.15.144:0/3490677958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d0100d50 0x7f78d0196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:21.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78cf7fe700 1 --2- 172.21.15.144:0/3490677958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d0100d50 0x7f78d0196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:21.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78cf7fe700 1 -- 172.21.15.144:0/3490677958 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c40095f0 con 0x7f78d01003b0 2024-09-10T11:49:21.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78cf7fe700 1 --2- 172.21.15.144:0/3490677958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78d01003b0 0x7f78d0195dc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f78c000d8d0 tx=0x7f78c000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:21.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78ccff9700 1 -- 172.21.15.144:0/3490677958 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78c0009940 con 0x7f78d01003b0 2024-09-10T11:49:21.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78ccff9700 1 -- 172.21.15.144:0/3490677958 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78c0010460 con 0x7f78d01003b0 2024-09-10T11:49:21.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78ccff9700 1 -- 172.21.15.144:0/3490677958 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78c000f5f0 con 0x7f78d01003b0 2024-09-10T11:49:21.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.718+0000 7f78d5dd4700 1 -- 172.21.15.144:0/3490677958 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d019aa50 con 0x7f78d01003b0 2024-09-10T11:49:21.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.719+0000 7f78d5dd4700 1 -- 172.21.15.144:0/3490677958 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d019afa0 con 0x7f78d01003b0 2024-09-10T11:49:21.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.720+0000 7f78d5dd4700 1 -- 172.21.15.144:0/3490677958 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d0108720 con 0x7f78d01003b0 2024-09-10T11:49:21.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.720+0000 7f78ccff9700 1 -- 172.21.15.144:0/3490677958 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f78c00105d0 con 0x7f78d01003b0 2024-09-10T11:49:21.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.721+0000 7f78ccff9700 1 --2- 172.21.15.144:0/3490677958 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f78b806e200 0x7f78b80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:21.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.721+0000 7f78ccff9700 1 -- 172.21.15.144:0/3490677958 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f78c008df90 con 0x7f78d01003b0 2024-09-10T11:49:21.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.721+0000 7f78ceffd700 1 --2- 172.21.15.144:0/3490677958 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f78b806e200 0x7f78b80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:21.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.721+0000 7f78ceffd700 1 --2- 172.21.15.144:0/3490677958 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f78b806e200 0x7f78b80706b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f78c4006010 tx=0x7f78c400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:21.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:21.725+0000 7f78ccff9700 1 -- 172.21.15.144:0/3490677958 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f78c00573f0 con 0x7f78d01003b0 2024-09-10T11:49:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:22.007+0000 7f78d5dd4700 1 -- 172.21.15.144:0/3490677958 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f78d004ea50 con 0x7f78d01003b0 2024-09-10T11:49:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:22.008+0000 7f78ccff9700 1 -- 172.21.15.144:0/3490677958 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f78c00161c0 con 0x7f78d01003b0 2024-09-10T11:49:22.010 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:49:22.010 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:49:22.010 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:49:22.010 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:49:22.010 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:49:22.010 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:49:22.011 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:49:22.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:22.010+0000 7f78d5dd4700 1 -- 172.21.15.144:0/3490677958 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f78b806e200 msgr2=0x7f78b80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:22.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:22.010+0000 7f78d5dd4700 1 --2- 172.21.15.144:0/3490677958 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f78b806e200 0x7f78b80706b0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f78c4006010 tx=0x7f78c400b540 comp rx=0 tx=0).stop 2024-09-10T11:49:22.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:22.010+0000 7f78d5dd4700 1 -- 172.21.15.144:0/3490677958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78d01003b0 msgr2=0x7f78d0195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:22.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:22.010+0000 7f78d5dd4700 1 --2- 172.21.15.144:0/3490677958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78d01003b0 0x7f78d0195dc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f78c000d8d0 tx=0x7f78c000dc90 comp rx=0 tx=0).stop 2024-09-10T11:49:22.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:22.010+0000 7f78d5dd4700 1 -- 172.21.15.144:0/3490677958 shutdown_connections 2024-09-10T11:49:22.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:22.010+0000 7f78d5dd4700 1 --2- 172.21.15.144:0/3490677958 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f78b806e200 0x7f78b80706b0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:22.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:22.010+0000 7f78d5dd4700 1 --2- 172.21.15.144:0/3490677958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78d01003b0 0x7f78d0195dc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:22.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:22.010+0000 7f78d5dd4700 1 --2- 172.21.15.144:0/3490677958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78d0100d50 0x7f78d0196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:22.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:22.010+0000 7f78d5dd4700 1 -- 172.21.15.144:0/3490677958 >> 172.21.15.144:0/3490677958 conn(0x7f78d00fbc30 msgr2=0x7f78d00fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:22.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:22.010+0000 7f78d5dd4700 1 -- 172.21.15.144:0/3490677958 shutdown_connections 2024-09-10T11:49:22.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:22.010+0000 7f78d5dd4700 1 -- 172.21.15.144:0/3490677958 wait complete. 2024-09-10T11:49:22.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:21 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/4161159936' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:49:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:21 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3615632140' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:49:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:21 smithi144 ceph-mon[87436]: from='client.35008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:21 smithi144 ceph-mon[87436]: pgmap v521: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:22.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:21 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/4161159936' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:49:22.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:21 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3615632140' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:49:22.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:21 smithi178 ceph-mon[65940]: from='client.35008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:22.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:21 smithi178 ceph-mon[65940]: pgmap v521: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:23.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:22 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3490677958' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:49:23.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:22 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3490677958' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:49:24.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:23 smithi144 ceph-mon[87436]: pgmap v522: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:24.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:23 smithi178 ceph-mon[65940]: pgmap v522: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:26 smithi178 ceph-mon[65940]: pgmap v523: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:26 smithi144 ceph-mon[87436]: pgmap v523: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:28 smithi178 ceph-mon[65940]: pgmap v524: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:49:29.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:28 smithi144 ceph-mon[87436]: pgmap v524: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:49:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:30 smithi178 ceph-mon[65940]: pgmap v525: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:31.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:30 smithi144 ceph-mon[87436]: pgmap v525: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:32 smithi144 ceph-mon[87436]: pgmap v526: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:33.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:32 smithi178 ceph-mon[65940]: pgmap v526: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:34 smithi178 ceph-mon[65940]: pgmap v527: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:35.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:34 smithi144 ceph-mon[87436]: pgmap v527: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:37.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:36 smithi178 ceph-mon[65940]: pgmap v528: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:37.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:36 smithi144 ceph-mon[87436]: pgmap v528: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:38 smithi178 ceph-mon[65940]: pgmap v529: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:38 smithi144 ceph-mon[87436]: pgmap v529: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:40 smithi178 ceph-mon[65940]: pgmap v530: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:40 smithi144 ceph-mon[87436]: pgmap v530: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:42.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:41 smithi144 ceph-mon[87436]: pgmap v531: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:41 smithi178 ceph-mon[65940]: pgmap v531: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:43.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:49:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:49:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:44 smithi144 ceph-mon[87436]: pgmap v532: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:44 smithi178 ceph-mon[65940]: pgmap v532: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:46 smithi178 ceph-mon[65940]: pgmap v533: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:47.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:46 smithi144 ceph-mon[87436]: pgmap v533: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:48 smithi178 ceph-mon[65940]: pgmap v534: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:49.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:48 smithi144 ceph-mon[87436]: pgmap v534: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:50 smithi178 ceph-mon[65940]: pgmap v535: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:51.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:50 smithi144 ceph-mon[87436]: pgmap v535: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:52.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.129+0000 7f6bd7314700 1 -- 172.21.15.144:0/2128045308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ffd30 msgr2=0x7f6bd0103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:52.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.129+0000 7f6bd7314700 1 --2- 172.21.15.144:0/2128045308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ffd30 0x7f6bd0103ce0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009910 tx=0x7f6bc0009c20 comp rx=0 tx=0).stop 2024-09-10T11:49:52.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.129+0000 7f6bd7314700 1 -- 172.21.15.144:0/2128045308 shutdown_connections 2024-09-10T11:49:52.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.129+0000 7f6bd7314700 1 --2- 172.21.15.144:0/2128045308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ffd30 0x7f6bd0103ce0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.129+0000 7f6bd7314700 1 --2- 172.21.15.144:0/2128045308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd00ff390 0x7f6bd00ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.129+0000 7f6bd7314700 1 -- 172.21.15.144:0/2128045308 >> 172.21.15.144:0/2128045308 conn(0x7f6bd0074f50 msgr2=0x7f6bd0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:52.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.129+0000 7f6bd7314700 1 -- 172.21.15.144:0/2128045308 shutdown_connections 2024-09-10T11:49:52.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.129+0000 7f6bd7314700 1 -- 172.21.15.144:0/2128045308 wait complete. 2024-09-10T11:49:52.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.130+0000 7f6bd7314700 1 Processor -- start 2024-09-10T11:49:52.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.130+0000 7f6bd7314700 1 -- start start 2024-09-10T11:49:52.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.130+0000 7f6bd7314700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ff390 0x7f6bd019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:52.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.130+0000 7f6bd7314700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd00ffd30 0x7f6bd019af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:52.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.130+0000 7f6bd7314700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd019b5b0 con 0x7f6bd00ff390 2024-09-10T11:49:52.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.130+0000 7f6bd7314700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd0194a60 con 0x7f6bd00ffd30 2024-09-10T11:49:52.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.130+0000 7f6bd50b0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ff390 0x7f6bd019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:52.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.130+0000 7f6bd50b0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ff390 0x7f6bd019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45252/0 (socket says 172.21.15.144:45252) 2024-09-10T11:49:52.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.131+0000 7f6bd50b0700 1 -- 172.21.15.144:0/1644015349 learned_addr learned my addr 172.21.15.144:0/1644015349 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:49:52.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.131+0000 7f6bd48af700 1 --2- 172.21.15.144:0/1644015349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd00ffd30 0x7f6bd019af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:52.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.131+0000 7f6bd50b0700 1 -- 172.21.15.144:0/1644015349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd00ffd30 msgr2=0x7f6bd019af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:52.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.131+0000 7f6bd50b0700 1 --2- 172.21.15.144:0/1644015349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd00ffd30 0x7f6bd019af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.131+0000 7f6bd50b0700 1 -- 172.21.15.144:0/1644015349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc00095f0 con 0x7f6bd00ff390 2024-09-10T11:49:52.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.131+0000 7f6bd50b0700 1 --2- 172.21.15.144:0/1644015349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ff390 0x7f6bd019a9e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00d8d0 tx=0x7f6bcc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:52.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.131+0000 7f6bc67fc700 1 -- 172.21.15.144:0/1644015349 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bcc0098a0 con 0x7f6bd00ff390 2024-09-10T11:49:52.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.131+0000 7f6bc67fc700 1 -- 172.21.15.144:0/1644015349 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bcc010460 con 0x7f6bd00ff390 2024-09-10T11:49:52.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.131+0000 7f6bd7314700 1 -- 172.21.15.144:0/1644015349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd0194d40 con 0x7f6bd00ff390 2024-09-10T11:49:52.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.131+0000 7f6bc67fc700 1 -- 172.21.15.144:0/1644015349 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bcc00f5f0 con 0x7f6bd00ff390 2024-09-10T11:49:52.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.131+0000 7f6bd7314700 1 -- 172.21.15.144:0/1644015349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd0195290 con 0x7f6bd00ff390 2024-09-10T11:49:52.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.132+0000 7f6bd7314700 1 -- 172.21.15.144:0/1644015349 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd004ea50 con 0x7f6bd00ff390 2024-09-10T11:49:52.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.133+0000 7f6bc67fc700 1 -- 172.21.15.144:0/1644015349 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6bcc009a00 con 0x7f6bd00ff390 2024-09-10T11:49:52.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.133+0000 7f6bc67fc700 1 --2- 172.21.15.144:0/1644015349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bbc06e200 0x7f6bbc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:52.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.133+0000 7f6bc67fc700 1 -- 172.21.15.144:0/1644015349 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6bcc08de70 con 0x7f6bd00ff390 2024-09-10T11:49:52.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.133+0000 7f6bd48af700 1 --2- 172.21.15.144:0/1644015349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bbc06e200 0x7f6bbc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:52.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.134+0000 7f6bd48af700 1 --2- 172.21.15.144:0/1644015349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bbc06e200 0x7f6bbc0706b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0103e20 tx=0x7f6bc001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:52.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.138+0000 7f6bc67fc700 1 -- 172.21.15.144:0/1644015349 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6bcc05ae00 con 0x7f6bd00ff390 2024-09-10T11:49:52.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.360+0000 7f6bd7314700 1 -- 172.21.15.144:0/1644015349 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bd0195780 con 0x7f6bbc06e200 2024-09-10T11:49:52.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.361+0000 7f6bc67fc700 1 -- 172.21.15.144:0/1644015349 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f6bd0195780 con 0x7f6bbc06e200 2024-09-10T11:49:52.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.363+0000 7f6bd7314700 1 -- 172.21.15.144:0/1644015349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bbc06e200 msgr2=0x7f6bbc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:52.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.363+0000 7f6bd7314700 1 --2- 172.21.15.144:0/1644015349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bbc06e200 0x7f6bbc0706b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6bd0103e20 tx=0x7f6bc001a040 comp rx=0 tx=0).stop 2024-09-10T11:49:52.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.363+0000 7f6bd7314700 1 -- 172.21.15.144:0/1644015349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ff390 msgr2=0x7f6bd019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:52.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.363+0000 7f6bd7314700 1 --2- 172.21.15.144:0/1644015349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ff390 0x7f6bd019a9e0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f6bcc00d8d0 tx=0x7f6bcc00dbe0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.363+0000 7f6bd7314700 1 -- 172.21.15.144:0/1644015349 shutdown_connections 2024-09-10T11:49:52.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.363+0000 7f6bd7314700 1 --2- 172.21.15.144:0/1644015349 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bbc06e200 0x7f6bbc0706b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.363+0000 7f6bd7314700 1 --2- 172.21.15.144:0/1644015349 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ff390 0x7f6bd019a9e0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.363+0000 7f6bd7314700 1 --2- 172.21.15.144:0/1644015349 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd00ffd30 0x7f6bd019af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.363+0000 7f6bd7314700 1 -- 172.21.15.144:0/1644015349 >> 172.21.15.144:0/1644015349 conn(0x7f6bd0074f50 msgr2=0x7f6bd00fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:52.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.364+0000 7f6bd7314700 1 -- 172.21.15.144:0/1644015349 shutdown_connections 2024-09-10T11:49:52.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.364+0000 7f6bd7314700 1 -- 172.21.15.144:0/1644015349 wait complete. 2024-09-10T11:49:52.375 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:49:52.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.469+0000 7fd20f2b9700 1 -- 172.21.15.144:0/1724139780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd208100d70 msgr2=0x7fd208104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:52.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.469+0000 7fd20f2b9700 1 --2- 172.21.15.144:0/1724139780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd208100d70 0x7fd208104d40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc009910 tx=0x7fd1fc009c20 comp rx=0 tx=0).stop 2024-09-10T11:49:52.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.469+0000 7fd20f2b9700 1 -- 172.21.15.144:0/1724139780 shutdown_connections 2024-09-10T11:49:52.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.469+0000 7fd20f2b9700 1 --2- 172.21.15.144:0/1724139780 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd208100d70 0x7fd208104d40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.469+0000 7fd20f2b9700 1 --2- 172.21.15.144:0/1724139780 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd2081003d0 0x7fd2081007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.469+0000 7fd20f2b9700 1 -- 172.21.15.144:0/1724139780 >> 172.21.15.144:0/1724139780 conn(0x7fd2080fbc90 msgr2=0x7fd2080fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:52.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.470+0000 7fd20f2b9700 1 -- 172.21.15.144:0/1724139780 shutdown_connections 2024-09-10T11:49:52.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.470+0000 7fd20f2b9700 1 -- 172.21.15.144:0/1724139780 wait complete. 2024-09-10T11:49:52.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.470+0000 7fd20f2b9700 1 Processor -- start 2024-09-10T11:49:52.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.470+0000 7fd20f2b9700 1 -- start start 2024-09-10T11:49:52.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.470+0000 7fd20f2b9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2081003d0 0x7fd208195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:52.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.470+0000 7fd20f2b9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd208100d70 0x7fd208196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:52.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.470+0000 7fd20f2b9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd208196960 con 0x7fd2081003d0 2024-09-10T11:49:52.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.470+0000 7fd20f2b9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd20819a6f0 con 0x7fd208100d70 2024-09-10T11:49:52.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.470+0000 7fd20d055700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2081003d0 0x7fd208195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:52.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.470+0000 7fd20d055700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2081003d0 0x7fd208195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36976/0 (socket says 172.21.15.144:36976) 2024-09-10T11:49:52.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.470+0000 7fd20d055700 1 -- 172.21.15.144:0/2805519433 learned_addr learned my addr 172.21.15.144:0/2805519433 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:49:52.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.471+0000 7fd20c854700 1 --2- 172.21.15.144:0/2805519433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd208100d70 0x7fd208196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:52.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.471+0000 7fd20d055700 1 -- 172.21.15.144:0/2805519433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd208100d70 msgr2=0x7fd208196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:52.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.471+0000 7fd20d055700 1 --2- 172.21.15.144:0/2805519433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd208100d70 0x7fd208196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.471+0000 7fd20d055700 1 -- 172.21.15.144:0/2805519433 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1fc0095f0 con 0x7fd2081003d0 2024-09-10T11:49:52.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.471+0000 7fd20c854700 1 --2- 172.21.15.144:0/2805519433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd208100d70 0x7fd208196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:49:52.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.471+0000 7fd20d055700 1 --2- 172.21.15.144:0/2805519433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2081003d0 0x7fd208195d40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd20400e920 tx=0x7fd20400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:52.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.471+0000 7fd1fa7fc700 1 -- 172.21.15.144:0/2805519433 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd204009790 con 0x7fd2081003d0 2024-09-10T11:49:52.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.471+0000 7fd1fa7fc700 1 -- 172.21.15.144:0/2805519433 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd20400cba0 con 0x7fd2081003d0 2024-09-10T11:49:52.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.471+0000 7fd20f2b9700 1 -- 172.21.15.144:0/2805519433 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd20819a9d0 con 0x7fd2081003d0 2024-09-10T11:49:52.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.471+0000 7fd1fa7fc700 1 -- 172.21.15.144:0/2805519433 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd204018650 con 0x7fd2081003d0 2024-09-10T11:49:52.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.471+0000 7fd20f2b9700 1 -- 172.21.15.144:0/2805519433 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd20819af20 con 0x7fd2081003d0 2024-09-10T11:49:52.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.472+0000 7fd20f2b9700 1 -- 172.21.15.144:0/2805519433 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd208108740 con 0x7fd2081003d0 2024-09-10T11:49:52.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.472+0000 7fd1fa7fc700 1 -- 172.21.15.144:0/2805519433 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd2040188c0 con 0x7fd2081003d0 2024-09-10T11:49:52.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.473+0000 7fd1fa7fc700 1 --2- 172.21.15.144:0/2805519433 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd1f406e200 0x7fd1f40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:52.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.473+0000 7fd1fa7fc700 1 -- 172.21.15.144:0/2805519433 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd204014070 con 0x7fd2081003d0 2024-09-10T11:49:52.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.473+0000 7fd20c854700 1 --2- 172.21.15.144:0/2805519433 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd1f406e200 0x7fd1f40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:52.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.473+0000 7fd20c854700 1 --2- 172.21.15.144:0/2805519433 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd1f406e200 0x7fd1f40706b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc005b40 tx=0x7fd1fc005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:52.478 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.477+0000 7fd1fa7fc700 1 -- 172.21.15.144:0/2805519433 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd2040541b0 con 0x7fd2081003d0 2024-09-10T11:49:52.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.699+0000 7fd20f2b9700 1 -- 172.21.15.144:0/2805519433 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd20819b250 con 0x7fd1f406e200 2024-09-10T11:49:52.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.700+0000 7fd1fa7fc700 1 -- 172.21.15.144:0/2805519433 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fd20819b250 con 0x7fd1f406e200 2024-09-10T11:49:52.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.703+0000 7fd20f2b9700 1 -- 172.21.15.144:0/2805519433 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd1f406e200 msgr2=0x7fd1f40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:52.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.703+0000 7fd20f2b9700 1 --2- 172.21.15.144:0/2805519433 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd1f406e200 0x7fd1f40706b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc005b40 tx=0x7fd1fc005a90 comp rx=0 tx=0).stop 2024-09-10T11:49:52.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.703+0000 7fd20f2b9700 1 -- 172.21.15.144:0/2805519433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2081003d0 msgr2=0x7fd208195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:52.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.703+0000 7fd20f2b9700 1 --2- 172.21.15.144:0/2805519433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2081003d0 0x7fd208195d40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd20400e920 tx=0x7fd20400ece0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.703+0000 7fd20f2b9700 1 -- 172.21.15.144:0/2805519433 shutdown_connections 2024-09-10T11:49:52.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.703+0000 7fd20f2b9700 1 --2- 172.21.15.144:0/2805519433 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd1f406e200 0x7fd1f40706b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.703+0000 7fd20f2b9700 1 --2- 172.21.15.144:0/2805519433 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2081003d0 0x7fd208195d40 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.703+0000 7fd20f2b9700 1 --2- 172.21.15.144:0/2805519433 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd208100d70 0x7fd208196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.703+0000 7fd20f2b9700 1 -- 172.21.15.144:0/2805519433 >> 172.21.15.144:0/2805519433 conn(0x7fd2080fbc90 msgr2=0x7fd2080fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:52.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.703+0000 7fd20f2b9700 1 -- 172.21.15.144:0/2805519433 shutdown_connections 2024-09-10T11:49:52.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.703+0000 7fd20f2b9700 1 -- 172.21.15.144:0/2805519433 wait complete. 2024-09-10T11:49:52.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.808+0000 7fcb9693a700 1 -- 172.21.15.144:0/2282653163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb900ff390 msgr2=0x7fcb900ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:52.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.808+0000 7fcb9693a700 1 --2- 172.21.15.144:0/2282653163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb900ff390 0x7fcb900ff760 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fcb80009910 tx=0x7fcb80009c20 comp rx=0 tx=0).stop 2024-09-10T11:49:52.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.808+0000 7fcb9693a700 1 -- 172.21.15.144:0/2282653163 shutdown_connections 2024-09-10T11:49:52.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.808+0000 7fcb9693a700 1 --2- 172.21.15.144:0/2282653163 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb900ffd30 0x7fcb90103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.808+0000 7fcb9693a700 1 --2- 172.21.15.144:0/2282653163 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb900ff390 0x7fcb900ff760 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.808+0000 7fcb9693a700 1 -- 172.21.15.144:0/2282653163 >> 172.21.15.144:0/2282653163 conn(0x7fcb90074f50 msgr2=0x7fcb90075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:52.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.808+0000 7fcb9693a700 1 -- 172.21.15.144:0/2282653163 shutdown_connections 2024-09-10T11:49:52.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.808+0000 7fcb9693a700 1 -- 172.21.15.144:0/2282653163 wait complete. 2024-09-10T11:49:52.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.809+0000 7fcb9693a700 1 Processor -- start 2024-09-10T11:49:52.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.809+0000 7fcb9693a700 1 -- start start 2024-09-10T11:49:52.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.809+0000 7fcb9693a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb900ff390 0x7fcb90195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:52.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.809+0000 7fcb9693a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb900ffd30 0x7fcb90196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:52.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.809+0000 7fcb9693a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb90196960 con 0x7fcb900ffd30 2024-09-10T11:49:52.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.809+0000 7fcb9693a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9019a6f0 con 0x7fcb900ff390 2024-09-10T11:49:52.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.809+0000 7fcb8f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb900ffd30 0x7fcb90196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:52.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.809+0000 7fcb8f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb900ffd30 0x7fcb90196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36994/0 (socket says 172.21.15.144:36994) 2024-09-10T11:49:52.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.809+0000 7fcb8f7fe700 1 -- 172.21.15.144:0/2382959938 learned_addr learned my addr 172.21.15.144:0/2382959938 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:49:52.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.809+0000 7fcb8ffff700 1 --2- 172.21.15.144:0/2382959938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb900ff390 0x7fcb90195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:52.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.810+0000 7fcb8f7fe700 1 -- 172.21.15.144:0/2382959938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb900ff390 msgr2=0x7fcb90195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:52.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.810+0000 7fcb8f7fe700 1 --2- 172.21.15.144:0/2382959938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb900ff390 0x7fcb90195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:52.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.810+0000 7fcb8f7fe700 1 -- 172.21.15.144:0/2382959938 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb800095f0 con 0x7fcb900ffd30 2024-09-10T11:49:52.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.810+0000 7fcb8f7fe700 1 --2- 172.21.15.144:0/2382959938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb900ffd30 0x7fcb90196280 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800d8d0 tx=0x7fcb7800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:52.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.810+0000 7fcb8d7fa700 1 -- 172.21.15.144:0/2382959938 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb78004a00 con 0x7fcb900ffd30 2024-09-10T11:49:52.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.810+0000 7fcb9693a700 1 -- 172.21.15.144:0/2382959938 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb9019a9d0 con 0x7fcb900ffd30 2024-09-10T11:49:52.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.810+0000 7fcb8d7fa700 1 -- 172.21.15.144:0/2382959938 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb780054e0 con 0x7fcb900ffd30 2024-09-10T11:49:52.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.810+0000 7fcb8d7fa700 1 -- 172.21.15.144:0/2382959938 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb7800fc20 con 0x7fcb900ffd30 2024-09-10T11:49:52.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.810+0000 7fcb9693a700 1 -- 172.21.15.144:0/2382959938 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb9019af20 con 0x7fcb900ffd30 2024-09-10T11:49:52.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.811+0000 7fcb9693a700 1 -- 172.21.15.144:0/2382959938 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9004ea50 con 0x7fcb900ffd30 2024-09-10T11:49:52.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.812+0000 7fcb8d7fa700 1 -- 172.21.15.144:0/2382959938 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fcb78005020 con 0x7fcb900ffd30 2024-09-10T11:49:52.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.812+0000 7fcb8d7fa700 1 --2- 172.21.15.144:0/2382959938 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb7c06e200 0x7fcb7c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:52.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.812+0000 7fcb8ffff700 1 --2- 172.21.15.144:0/2382959938 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb7c06e200 0x7fcb7c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:52.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.812+0000 7fcb8d7fa700 1 -- 172.21.15.144:0/2382959938 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcb7808d930 con 0x7fcb900ffd30 2024-09-10T11:49:52.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.813+0000 7fcb8ffff700 1 --2- 172.21.15.144:0/2382959938 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb7c06e200 0x7fcb7c0706b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcb80006010 tx=0x7fcb8001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:52.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:52.817+0000 7fcb8d7fa700 1 -- 172.21.15.144:0/2382959938 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcb78056d10 con 0x7fcb900ffd30 2024-09-10T11:49:53.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.031+0000 7fcb9693a700 1 -- 172.21.15.144:0/2382959938 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb9019b400 con 0x7fcb7c06e200 2024-09-10T11:49:53.042 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.041+0000 7fcb8d7fa700 1 -- 172.21.15.144:0/2382959938 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fcb9019b400 con 0x7fcb7c06e200 2024-09-10T11:49:53.042 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:49:53.042 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (18m) 11m ago 28m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:49:53.042 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (28m) 11m ago 28m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:49:53.042 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (27m) 11m ago 27m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:49:53.042 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (15m) 11m ago 28m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:49:53.042 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (15m) 11m ago 27m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:49:53.043 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (17m) 11m ago 28m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:49:53.043 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (12m) 11m ago 23m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:49:53.043 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (11m) 11m ago 23m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:49:53.043 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (23m) 11m ago 23m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:49:53.043 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (11m) 11m ago 23m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:49:53.043 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (19m) 11m ago 30m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:49:53.043 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (19m) 11m ago 27m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:49:53.043 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (16m) 11m ago 30m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:49:53.043 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (16m) 11m ago 27m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:49:53.043 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (18m) 11m ago 28m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:49:53.043 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (18m) 11m ago 27m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:49:53.043 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (15m) 11m ago 26m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:49:53.043 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (14m) 11m ago 26m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:49:53.043 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (14m) 11m ago 25m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:49:53.043 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (13m) 11m ago 25m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:49:53.044 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (13m) 11m ago 25m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:49:53.044 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (12m) 11m ago 24m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:49:53.044 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (18m) 11m ago 28m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:49:53.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.043+0000 7fcb9693a700 1 -- 172.21.15.144:0/2382959938 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb7c06e200 msgr2=0x7fcb7c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:53.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.043+0000 7fcb9693a700 1 --2- 172.21.15.144:0/2382959938 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb7c06e200 0x7fcb7c0706b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcb80006010 tx=0x7fcb8001a040 comp rx=0 tx=0).stop 2024-09-10T11:49:53.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.043+0000 7fcb9693a700 1 -- 172.21.15.144:0/2382959938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb900ffd30 msgr2=0x7fcb90196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:53.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.043+0000 7fcb9693a700 1 --2- 172.21.15.144:0/2382959938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb900ffd30 0x7fcb90196280 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcb7800d8d0 tx=0x7fcb7800dbe0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.043+0000 7fcb9693a700 1 -- 172.21.15.144:0/2382959938 shutdown_connections 2024-09-10T11:49:53.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.043+0000 7fcb9693a700 1 --2- 172.21.15.144:0/2382959938 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb7c06e200 0x7fcb7c0706b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.043+0000 7fcb9693a700 1 --2- 172.21.15.144:0/2382959938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb900ff390 0x7fcb90195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.043+0000 7fcb9693a700 1 --2- 172.21.15.144:0/2382959938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb900ffd30 0x7fcb90196280 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.043+0000 7fcb9693a700 1 -- 172.21.15.144:0/2382959938 >> 172.21.15.144:0/2382959938 conn(0x7fcb90074f50 msgr2=0x7fcb90103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:53.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.043+0000 7fcb9693a700 1 -- 172.21.15.144:0/2382959938 shutdown_connections 2024-09-10T11:49:53.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.043+0000 7fcb9693a700 1 -- 172.21.15.144:0/2382959938 wait complete. 2024-09-10T11:49:53.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:52 smithi144 ceph-mon[87436]: pgmap v536: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:53.116 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:52 smithi178 ceph-mon[65940]: pgmap v536: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:53.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.147+0000 7f7c4846b700 1 -- 172.21.15.144:0/2005620793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c400ff260 msgr2=0x7f7c400ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:53.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.147+0000 7f7c4846b700 1 --2- 172.21.15.144:0/2005620793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c400ff260 0x7f7c400ff630 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009910 tx=0x7f7c3c009c20 comp rx=0 tx=0).stop 2024-09-10T11:49:53.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.148+0000 7f7c4846b700 1 -- 172.21.15.144:0/2005620793 shutdown_connections 2024-09-10T11:49:53.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.148+0000 7f7c4846b700 1 --2- 172.21.15.144:0/2005620793 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c400ffb70 0x7f7c4010a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.148+0000 7f7c4846b700 1 --2- 172.21.15.144:0/2005620793 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c400ff260 0x7f7c400ff630 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.148+0000 7f7c4846b700 1 -- 172.21.15.144:0/2005620793 >> 172.21.15.144:0/2005620793 conn(0x7f7c400747e0 msgr2=0x7f7c40074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:53.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.148+0000 7f7c4846b700 1 -- 172.21.15.144:0/2005620793 shutdown_connections 2024-09-10T11:49:53.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.148+0000 7f7c4846b700 1 -- 172.21.15.144:0/2005620793 wait complete. 2024-09-10T11:49:53.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.148+0000 7f7c4846b700 1 Processor -- start 2024-09-10T11:49:53.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.148+0000 7f7c4846b700 1 -- start start 2024-09-10T11:49:53.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.148+0000 7f7c4846b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c400ff260 0x7f7c40195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:53.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.148+0000 7f7c4846b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c400ffb70 0x7f7c40196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:53.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.148+0000 7f7c4846b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c40196960 con 0x7f7c400ff260 2024-09-10T11:49:53.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.148+0000 7f7c4846b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4019a6f0 con 0x7f7c400ffb70 2024-09-10T11:49:53.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.149+0000 7f7c46207700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c400ff260 0x7f7c40195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:53.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.149+0000 7f7c46207700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c400ff260 0x7f7c40195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37014/0 (socket says 172.21.15.144:37014) 2024-09-10T11:49:53.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.149+0000 7f7c46207700 1 -- 172.21.15.144:0/347340907 learned_addr learned my addr 172.21.15.144:0/347340907 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:49:53.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.149+0000 7f7c46207700 1 -- 172.21.15.144:0/347340907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c400ffb70 msgr2=0x7f7c40196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:53.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.149+0000 7f7c45a06700 1 --2- 172.21.15.144:0/347340907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c400ffb70 0x7f7c40196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:53.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.149+0000 7f7c46207700 1 --2- 172.21.15.144:0/347340907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c400ffb70 0x7f7c40196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.149+0000 7f7c46207700 1 -- 172.21.15.144:0/347340907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3c0095f0 con 0x7f7c400ff260 2024-09-10T11:49:53.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.149+0000 7f7c45a06700 1 --2- 172.21.15.144:0/347340907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c400ffb70 0x7f7c40196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:49:53.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.149+0000 7f7c46207700 1 --2- 172.21.15.144:0/347340907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c400ff260 0x7f7c40195d40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c006010 tx=0x7f7c3c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:53.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.149+0000 7f7c377fe700 1 -- 172.21.15.144:0/347340907 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3c01d070 con 0x7f7c400ff260 2024-09-10T11:49:53.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.149+0000 7f7c377fe700 1 -- 172.21.15.144:0/347340907 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c3c004fb0 con 0x7f7c400ff260 2024-09-10T11:49:53.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.149+0000 7f7c4846b700 1 -- 172.21.15.144:0/347340907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4019a970 con 0x7f7c400ff260 2024-09-10T11:49:53.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.149+0000 7f7c377fe700 1 -- 172.21.15.144:0/347340907 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3c022ea0 con 0x7f7c400ff260 2024-09-10T11:49:53.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.149+0000 7f7c4846b700 1 -- 172.21.15.144:0/347340907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4019ae60 con 0x7f7c400ff260 2024-09-10T11:49:53.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.150+0000 7f7c4846b700 1 -- 172.21.15.144:0/347340907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c40107ce0 con 0x7f7c400ff260 2024-09-10T11:49:53.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.151+0000 7f7c377fe700 1 -- 172.21.15.144:0/347340907 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7c3c022470 con 0x7f7c400ff260 2024-09-10T11:49:53.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.151+0000 7f7c377fe700 1 --2- 172.21.15.144:0/347340907 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c2c06e200 0x7f7c2c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:53.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.152+0000 7f7c377fe700 1 -- 172.21.15.144:0/347340907 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7c3c08f610 con 0x7f7c400ff260 2024-09-10T11:49:53.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.152+0000 7f7c45a06700 1 --2- 172.21.15.144:0/347340907 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c2c06e200 0x7f7c2c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:53.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.152+0000 7f7c45a06700 1 --2- 172.21.15.144:0/347340907 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c2c06e200 0x7f7c2c0706b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7c30005fd0 tx=0x7f7c30005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:53.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.156+0000 7f7c377fe700 1 -- 172.21.15.144:0/347340907 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7c3c058aa0 con 0x7f7c400ff260 2024-09-10T11:49:53.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.442+0000 7f7c4846b700 1 -- 172.21.15.144:0/347340907 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7c4019b140 con 0x7f7c400ff260 2024-09-10T11:49:53.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.442+0000 7f7c377fe700 1 -- 172.21.15.144:0/347340907 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f7c3c022720 con 0x7f7c400ff260 2024-09-10T11:49:53.444 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:49:53.444 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:49:53.444 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:49:53.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:49:53.444 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:49:53.444 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:49:53.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:49:53.444 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:49:53.444 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:49:53.444 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:49:53.445 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:49:53.445 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:49:53.445 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:49:53.445 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:49:53.445 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:49:53.445 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:49:53.445 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:49:53.445 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:49:53.445 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:49:53.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.444+0000 7f7c4846b700 1 -- 172.21.15.144:0/347340907 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c2c06e200 msgr2=0x7f7c2c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:53.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.444+0000 7f7c4846b700 1 --2- 172.21.15.144:0/347340907 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c2c06e200 0x7f7c2c0706b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7c30005fd0 tx=0x7f7c30005dc0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.444+0000 7f7c4846b700 1 -- 172.21.15.144:0/347340907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c400ff260 msgr2=0x7f7c40195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:53.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.444+0000 7f7c4846b700 1 --2- 172.21.15.144:0/347340907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c400ff260 0x7f7c40195d40 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c006010 tx=0x7f7c3c00be20 comp rx=0 tx=0).stop 2024-09-10T11:49:53.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.445+0000 7f7c4846b700 1 -- 172.21.15.144:0/347340907 shutdown_connections 2024-09-10T11:49:53.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.445+0000 7f7c4846b700 1 --2- 172.21.15.144:0/347340907 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c2c06e200 0x7f7c2c0706b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.445+0000 7f7c4846b700 1 --2- 172.21.15.144:0/347340907 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c400ff260 0x7f7c40195d40 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.445+0000 7f7c4846b700 1 --2- 172.21.15.144:0/347340907 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c400ffb70 0x7f7c40196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.445+0000 7f7c4846b700 1 -- 172.21.15.144:0/347340907 >> 172.21.15.144:0/347340907 conn(0x7f7c400747e0 msgr2=0x7f7c40104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:53.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.445+0000 7f7c4846b700 1 -- 172.21.15.144:0/347340907 shutdown_connections 2024-09-10T11:49:53.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.445+0000 7f7c4846b700 1 -- 172.21.15.144:0/347340907 wait complete. 2024-09-10T11:49:53.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.551+0000 7f17fe33a700 1 -- 172.21.15.144:0/1714609850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f80ff320 msgr2=0x7f17f80ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:53.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.551+0000 7f17fe33a700 1 --2- 172.21.15.144:0/1714609850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f80ff320 0x7f17f80ff790 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009910 tx=0x7f17e0009c20 comp rx=0 tx=0).stop 2024-09-10T11:49:53.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.551+0000 7f17fe33a700 1 -- 172.21.15.144:0/1714609850 shutdown_connections 2024-09-10T11:49:53.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.551+0000 7f17fe33a700 1 --2- 172.21.15.144:0/1714609850 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f80ff320 0x7f17f80ff790 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.551+0000 7f17fe33a700 1 --2- 172.21.15.144:0/1714609850 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17f8103860 0x7f17f8103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.551+0000 7f17fe33a700 1 -- 172.21.15.144:0/1714609850 >> 172.21.15.144:0/1714609850 conn(0x7f17f80748b0 msgr2=0x7f17f8074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:53.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.551+0000 7f17fe33a700 1 -- 172.21.15.144:0/1714609850 shutdown_connections 2024-09-10T11:49:53.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.551+0000 7f17fe33a700 1 -- 172.21.15.144:0/1714609850 wait complete. 2024-09-10T11:49:53.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.552+0000 7f17fe33a700 1 Processor -- start 2024-09-10T11:49:53.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.552+0000 7f17fe33a700 1 -- start start 2024-09-10T11:49:53.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.552+0000 7f17fe33a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17f80ff320 0x7f17f8195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:53.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.552+0000 7f17fe33a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f8103860 0x7f17f8196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:53.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.552+0000 7f17fe33a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f8196890 con 0x7f17f8103860 2024-09-10T11:49:53.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.552+0000 7f17fe33a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f81995c0 con 0x7f17f80ff320 2024-09-10T11:49:53.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.552+0000 7f17effff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f8103860 0x7f17f8196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:53.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.552+0000 7f17effff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f8103860 0x7f17f8196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37038/0 (socket says 172.21.15.144:37038) 2024-09-10T11:49:53.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.552+0000 7f17effff700 1 -- 172.21.15.144:0/3418738942 learned_addr learned my addr 172.21.15.144:0/3418738942 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:49:53.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.552+0000 7f17f7fff700 1 --2- 172.21.15.144:0/3418738942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17f80ff320 0x7f17f8195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:53.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.552+0000 7f17effff700 1 -- 172.21.15.144:0/3418738942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17f80ff320 msgr2=0x7f17f8195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:53.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.552+0000 7f17effff700 1 --2- 172.21.15.144:0/3418738942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17f80ff320 0x7f17f8195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.552+0000 7f17effff700 1 -- 172.21.15.144:0/3418738942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e00095f0 con 0x7f17f8103860 2024-09-10T11:49:53.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.553+0000 7f17effff700 1 --2- 172.21.15.144:0/3418738942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f8103860 0x7f17f8196200 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f17e0006010 tx=0x7f17e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:53.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.553+0000 7f17f5ffb700 1 -- 172.21.15.144:0/3418738942 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17e001d070 con 0x7f17f8103860 2024-09-10T11:49:53.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.553+0000 7f17f5ffb700 1 -- 172.21.15.144:0/3418738942 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f17e0004fb0 con 0x7f17f8103860 2024-09-10T11:49:53.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.553+0000 7f17fe33a700 1 -- 172.21.15.144:0/3418738942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17f8199840 con 0x7f17f8103860 2024-09-10T11:49:53.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.553+0000 7f17f5ffb700 1 -- 172.21.15.144:0/3418738942 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17e0022ea0 con 0x7f17f8103860 2024-09-10T11:49:53.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.553+0000 7f17fe33a700 1 -- 172.21.15.144:0/3418738942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17f8199db0 con 0x7f17f8103860 2024-09-10T11:49:53.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.554+0000 7f17fe33a700 1 -- 172.21.15.144:0/3418738942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17f8108df0 con 0x7f17f8103860 2024-09-10T11:49:53.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.554+0000 7f17f5ffb700 1 -- 172.21.15.144:0/3418738942 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f17e0022470 con 0x7f17f8103860 2024-09-10T11:49:53.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.555+0000 7f17f5ffb700 1 --2- 172.21.15.144:0/3418738942 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f17e406e200 0x7f17e40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:53.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.555+0000 7f17f5ffb700 1 -- 172.21.15.144:0/3418738942 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f17e008f6b0 con 0x7f17f8103860 2024-09-10T11:49:53.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.555+0000 7f17f7fff700 1 --2- 172.21.15.144:0/3418738942 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f17e406e200 0x7f17e40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:53.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.555+0000 7f17f7fff700 1 --2- 172.21.15.144:0/3418738942 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f17e406e200 0x7f17e40706b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f17e8005950 tx=0x7f17e8005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:53.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.559+0000 7f17f5ffb700 1 -- 172.21.15.144:0/3418738942 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f17e0058aa0 con 0x7f17f8103860 2024-09-10T11:49:53.806 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.805+0000 7f17fe33a700 1 -- 172.21.15.144:0/3418738942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f17f804ea50 con 0x7f17f8103860 2024-09-10T11:49:53.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.806+0000 7f17f5ffb700 1 -- 172.21.15.144:0/3418738942 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f17e0022720 con 0x7f17f8103860 2024-09-10T11:49:53.807 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:49:53.807 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:49:53.807 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:49:53.807 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:49:53.807 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:49:53.807 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:49:53.807 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:49:53.808 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:49:53.808 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:49:53.808 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:49:53.808 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:49:53.808 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:49:53.808 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:49:53.808 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:49:53.808 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:49:53.808 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:49:53.808 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:49:53.808 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:49:53.808 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:49:53.808 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:49:53.808 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:49:53.809 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:49:53.809 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:49:53.809 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:49:53.809 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:49:53.809 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:49:53.809 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:49:53.809 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:49:53.809 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:49:53.809 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:49:53.809 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:49:53.809 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:49:53.809 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:49:53.809 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:49:53.809 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:49:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:49:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:49:53.810 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:49:53.810 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:49:53.810 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:49:53.810 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:49:53.810 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:49:53.810 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:49:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.808+0000 7f17fe33a700 1 -- 172.21.15.144:0/3418738942 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f17e406e200 msgr2=0x7f17e40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.808+0000 7f17fe33a700 1 --2- 172.21.15.144:0/3418738942 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f17e406e200 0x7f17e40706b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f17e8005950 tx=0x7f17e8005cf0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.808+0000 7f17fe33a700 1 -- 172.21.15.144:0/3418738942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f8103860 msgr2=0x7f17f8196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.808+0000 7f17fe33a700 1 --2- 172.21.15.144:0/3418738942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f8103860 0x7f17f8196200 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f17e0006010 tx=0x7f17e000be20 comp rx=0 tx=0).stop 2024-09-10T11:49:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.808+0000 7f17fe33a700 1 -- 172.21.15.144:0/3418738942 shutdown_connections 2024-09-10T11:49:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.808+0000 7f17fe33a700 1 --2- 172.21.15.144:0/3418738942 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f17e406e200 0x7f17e40706b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.808+0000 7f17fe33a700 1 --2- 172.21.15.144:0/3418738942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f17f80ff320 0x7f17f8195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.808+0000 7f17fe33a700 1 --2- 172.21.15.144:0/3418738942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f17f8103860 0x7f17f8196200 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.808+0000 7f17fe33a700 1 -- 172.21.15.144:0/3418738942 >> 172.21.15.144:0/3418738942 conn(0x7f17f80748b0 msgr2=0x7f17f80fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.808+0000 7f17fe33a700 1 -- 172.21.15.144:0/3418738942 shutdown_connections 2024-09-10T11:49:53.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.808+0000 7f17fe33a700 1 -- 172.21.15.144:0/3418738942 wait complete. 2024-09-10T11:49:53.812 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:49:53.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.915+0000 7fa14d60b700 1 -- 172.21.15.144:0/160181221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa148100d70 msgr2=0x7fa148104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:53.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.915+0000 7fa14d60b700 1 --2- 172.21.15.144:0/160181221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa148100d70 0x7fa148104d40 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa130009910 tx=0x7fa130009c20 comp rx=0 tx=0).stop 2024-09-10T11:49:53.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.915+0000 7fa14d60b700 1 -- 172.21.15.144:0/160181221 shutdown_connections 2024-09-10T11:49:53.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.915+0000 7fa14d60b700 1 --2- 172.21.15.144:0/160181221 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa148100d70 0x7fa148104d40 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.915+0000 7fa14d60b700 1 --2- 172.21.15.144:0/160181221 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1481003d0 0x7fa1481007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.915+0000 7fa14d60b700 1 -- 172.21.15.144:0/160181221 >> 172.21.15.144:0/160181221 conn(0x7fa1480fbc90 msgr2=0x7fa1480fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:53.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.915+0000 7fa14d60b700 1 -- 172.21.15.144:0/160181221 shutdown_connections 2024-09-10T11:49:53.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.915+0000 7fa14d60b700 1 -- 172.21.15.144:0/160181221 wait complete. 2024-09-10T11:49:53.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.916+0000 7fa14d60b700 1 Processor -- start 2024-09-10T11:49:53.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.916+0000 7fa14d60b700 1 -- start start 2024-09-10T11:49:53.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.916+0000 7fa14d60b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1481003d0 0x7fa148197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:53.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.916+0000 7fa14d60b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa148100d70 0x7fa148198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:53.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.916+0000 7fa14d60b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa148198ab0 con 0x7fa148100d70 2024-09-10T11:49:53.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.916+0000 7fa14d60b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa148198bf0 con 0x7fa1481003d0 2024-09-10T11:49:53.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.916+0000 7fa1467fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa148100d70 0x7fa148198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:53.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.916+0000 7fa1467fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa148100d70 0x7fa148198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37066/0 (socket says 172.21.15.144:37066) 2024-09-10T11:49:53.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.916+0000 7fa1467fc700 1 -- 172.21.15.144:0/2797851066 learned_addr learned my addr 172.21.15.144:0/2797851066 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:49:53.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.917+0000 7fa146ffd700 1 --2- 172.21.15.144:0/2797851066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1481003d0 0x7fa148197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:53.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.917+0000 7fa1467fc700 1 -- 172.21.15.144:0/2797851066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1481003d0 msgr2=0x7fa148197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:53.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.917+0000 7fa1467fc700 1 --2- 172.21.15.144:0/2797851066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1481003d0 0x7fa148197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:53.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.917+0000 7fa1467fc700 1 -- 172.21.15.144:0/2797851066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1300095f0 con 0x7fa148100d70 2024-09-10T11:49:53.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.917+0000 7fa146ffd700 1 --2- 172.21.15.144:0/2797851066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1481003d0 0x7fa148197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:49:53.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.917+0000 7fa1467fc700 1 --2- 172.21.15.144:0/2797851066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa148100d70 0x7fa148198460 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa13000bda0 tx=0x7fa13000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:53.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.917+0000 7fa13ffff700 1 -- 172.21.15.144:0/2797851066 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa13001d070 con 0x7fa148100d70 2024-09-10T11:49:53.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.917+0000 7fa13ffff700 1 -- 172.21.15.144:0/2797851066 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa130004fb0 con 0x7fa148100d70 2024-09-10T11:49:53.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.917+0000 7fa14d60b700 1 -- 172.21.15.144:0/2797851066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14819c9e0 con 0x7fa148100d70 2024-09-10T11:49:53.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.917+0000 7fa13ffff700 1 -- 172.21.15.144:0/2797851066 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa13000f480 con 0x7fa148100d70 2024-09-10T11:49:53.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.917+0000 7fa14d60b700 1 -- 172.21.15.144:0/2797851066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14819ce80 con 0x7fa148100d70 2024-09-10T11:49:53.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.918+0000 7fa14d60b700 1 -- 172.21.15.144:0/2797851066 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa14804ea50 con 0x7fa148100d70 2024-09-10T11:49:53.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.919+0000 7fa13ffff700 1 -- 172.21.15.144:0/2797851066 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa130022870 con 0x7fa148100d70 2024-09-10T11:49:53.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.919+0000 7fa13ffff700 1 --2- 172.21.15.144:0/2797851066 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa13406e200 0x7fa1340706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:53.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.919+0000 7fa13ffff700 1 -- 172.21.15.144:0/2797851066 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa13008f5f0 con 0x7fa148100d70 2024-09-10T11:49:53.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.919+0000 7fa146ffd700 1 --2- 172.21.15.144:0/2797851066 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa13406e200 0x7fa1340706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:53.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.920+0000 7fa146ffd700 1 --2- 172.21.15.144:0/2797851066 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa13406e200 0x7fa1340706b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa13800d340 tx=0x7fa138009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:53.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:53.923+0000 7fa13ffff700 1 -- 172.21.15.144:0/2797851066 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa130058a80 con 0x7fa148100d70 2024-09-10T11:49:54.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:53 smithi178 ceph-mon[65940]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:54.167 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:53 smithi178 ceph-mon[65940]: from='client.35020 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:54.167 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:53 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/347340907' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:49:54.167 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:53 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3418738942' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:49:54.168 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.146+0000 7fa14d60b700 1 -- 172.21.15.144:0/2797851066 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa148104160 con 0x7fa13406e200 2024-09-10T11:49:54.168 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:53 smithi144 ceph-mon[87436]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:54.168 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:53 smithi144 ceph-mon[87436]: from='client.35020 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:54.168 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:53 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/347340907' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:49:54.168 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:53 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3418738942' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:49:54.169 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.147+0000 7fa13ffff700 1 -- 172.21.15.144:0/2797851066 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa148104160 con 0x7fa13406e200 2024-09-10T11:49:54.169 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:49:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:49:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:49:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:49:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:49:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:49:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:49:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:49:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:49:54.169 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:49:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:49:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:49:54.170 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:49:54.170 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:49:54.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.150+0000 7fa14d60b700 1 -- 172.21.15.144:0/2797851066 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa13406e200 msgr2=0x7fa1340706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:54.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.150+0000 7fa14d60b700 1 --2- 172.21.15.144:0/2797851066 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa13406e200 0x7fa1340706b0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa13800d340 tx=0x7fa138009040 comp rx=0 tx=0).stop 2024-09-10T11:49:54.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.150+0000 7fa14d60b700 1 -- 172.21.15.144:0/2797851066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa148100d70 msgr2=0x7fa148198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:54.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.150+0000 7fa14d60b700 1 --2- 172.21.15.144:0/2797851066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa148100d70 0x7fa148198460 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa13000bda0 tx=0x7fa13000be80 comp rx=0 tx=0).stop 2024-09-10T11:49:54.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.150+0000 7fa14d60b700 1 -- 172.21.15.144:0/2797851066 shutdown_connections 2024-09-10T11:49:54.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.150+0000 7fa14d60b700 1 --2- 172.21.15.144:0/2797851066 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa13406e200 0x7fa1340706b0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:54.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.150+0000 7fa14d60b700 1 --2- 172.21.15.144:0/2797851066 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa1481003d0 0x7fa148197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:54.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.150+0000 7fa14d60b700 1 --2- 172.21.15.144:0/2797851066 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa148100d70 0x7fa148198460 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:54.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.150+0000 7fa14d60b700 1 -- 172.21.15.144:0/2797851066 >> 172.21.15.144:0/2797851066 conn(0x7fa1480fbc90 msgr2=0x7fa1480fd2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:54.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.150+0000 7fa14d60b700 1 -- 172.21.15.144:0/2797851066 shutdown_connections 2024-09-10T11:49:54.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.150+0000 7fa14d60b700 1 -- 172.21.15.144:0/2797851066 wait complete. 2024-09-10T11:49:54.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.254+0000 7f987052c700 1 -- 172.21.15.144:0/2238148830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98680ffc60 msgr2=0x7f9868103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:54.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.254+0000 7f987052c700 1 --2- 172.21.15.144:0/2238148830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98680ffc60 0x7f9868103c30 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f9858009910 tx=0x7f9858009c20 comp rx=0 tx=0).stop 2024-09-10T11:49:54.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.255+0000 7f987052c700 1 -- 172.21.15.144:0/2238148830 shutdown_connections 2024-09-10T11:49:54.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.255+0000 7f987052c700 1 --2- 172.21.15.144:0/2238148830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98680ffc60 0x7f9868103c30 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:54.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.255+0000 7f987052c700 1 --2- 172.21.15.144:0/2238148830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98680ff2c0 0x7f98680ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:54.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.255+0000 7f987052c700 1 -- 172.21.15.144:0/2238148830 >> 172.21.15.144:0/2238148830 conn(0x7f98680fab80 msgr2=0x7f98680fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:54.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.255+0000 7f987052c700 1 -- 172.21.15.144:0/2238148830 shutdown_connections 2024-09-10T11:49:54.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.255+0000 7f987052c700 1 -- 172.21.15.144:0/2238148830 wait complete. 2024-09-10T11:49:54.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.255+0000 7f987052c700 1 Processor -- start 2024-09-10T11:49:54.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.255+0000 7f987052c700 1 -- start start 2024-09-10T11:49:54.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.255+0000 7f987052c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98680ff2c0 0x7f9868196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:54.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.255+0000 7f987052c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98680ffc60 0x7f98681970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:54.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.256+0000 7f987052c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9868197700 con 0x7f98680ffc60 2024-09-10T11:49:54.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.256+0000 7f987052c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9868197840 con 0x7f98680ff2c0 2024-09-10T11:49:54.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.256+0000 7f986dac7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98680ffc60 0x7f98681970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:54.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.256+0000 7f986dac7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98680ffc60 0x7f98681970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:37090/0 (socket says 172.21.15.144:37090) 2024-09-10T11:49:54.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.256+0000 7f986dac7700 1 -- 172.21.15.144:0/3271367919 learned_addr learned my addr 172.21.15.144:0/3271367919 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:49:54.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.256+0000 7f986e2c8700 1 --2- 172.21.15.144:0/3271367919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98680ff2c0 0x7f9868196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:54.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.256+0000 7f986dac7700 1 -- 172.21.15.144:0/3271367919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98680ff2c0 msgr2=0x7f9868196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:54.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.256+0000 7f986dac7700 1 --2- 172.21.15.144:0/3271367919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98680ff2c0 0x7f9868196ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:54.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.256+0000 7f986dac7700 1 -- 172.21.15.144:0/3271367919 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98580095f0 con 0x7f98680ffc60 2024-09-10T11:49:54.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.256+0000 7f986dac7700 1 --2- 172.21.15.144:0/3271367919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98680ffc60 0x7f98681970e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f98580092e0 tx=0x7f985800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:54.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.256+0000 7f985f7fe700 1 -- 172.21.15.144:0/3271367919 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f985801d070 con 0x7f98680ffc60 2024-09-10T11:49:54.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.256+0000 7f985f7fe700 1 -- 172.21.15.144:0/3271367919 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9858005dc0 con 0x7f98680ffc60 2024-09-10T11:49:54.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.256+0000 7f987052c700 1 -- 172.21.15.144:0/3271367919 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986819fc10 con 0x7f98680ffc60 2024-09-10T11:49:54.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.256+0000 7f985f7fe700 1 -- 172.21.15.144:0/3271367919 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9858017400 con 0x7f98680ffc60 2024-09-10T11:49:54.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.256+0000 7f987052c700 1 -- 172.21.15.144:0/3271367919 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98681a0020 con 0x7f98680ffc60 2024-09-10T11:49:54.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.257+0000 7f987052c700 1 -- 172.21.15.144:0/3271367919 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9868107630 con 0x7f98680ffc60 2024-09-10T11:49:54.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.258+0000 7f985f7fe700 1 -- 172.21.15.144:0/3271367919 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f985800f6f0 con 0x7f98680ffc60 2024-09-10T11:49:54.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.258+0000 7f985f7fe700 1 --2- 172.21.15.144:0/3271367919 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f985406e200 0x7f98540706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:49:54.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.258+0000 7f985f7fe700 1 -- 172.21.15.144:0/3271367919 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f985808f470 con 0x7f98680ffc60 2024-09-10T11:49:54.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.258+0000 7f986e2c8700 1 --2- 172.21.15.144:0/3271367919 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f985406e200 0x7f98540706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:49:54.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.259+0000 7f986e2c8700 1 --2- 172.21.15.144:0/3271367919 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f985406e200 0x7f98540706b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9864005950 tx=0x7f986400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:49:54.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.263+0000 7f985f7fe700 1 -- 172.21.15.144:0/3271367919 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9858058850 con 0x7f98680ffc60 2024-09-10T11:49:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.546+0000 7f987052c700 1 -- 172.21.15.144:0/3271367919 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f986804ea50 con 0x7f98680ffc60 2024-09-10T11:49:54.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.547+0000 7f985f7fe700 1 -- 172.21.15.144:0/3271367919 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f985805bf10 con 0x7f98680ffc60 2024-09-10T11:49:54.548 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:49:54.548 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:49:54.548 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:49:54.548 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:49:54.548 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:49:54.549 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:49:54.549 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:49:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.549+0000 7f987052c700 1 -- 172.21.15.144:0/3271367919 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f985406e200 msgr2=0x7f98540706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.549+0000 7f987052c700 1 --2- 172.21.15.144:0/3271367919 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f985406e200 0x7f98540706b0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9864005950 tx=0x7f986400b410 comp rx=0 tx=0).stop 2024-09-10T11:49:54.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.549+0000 7f987052c700 1 -- 172.21.15.144:0/3271367919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98680ffc60 msgr2=0x7f98681970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:49:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.549+0000 7f987052c700 1 --2- 172.21.15.144:0/3271367919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98680ffc60 0x7f98681970e0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f98580092e0 tx=0x7f985800bd70 comp rx=0 tx=0).stop 2024-09-10T11:49:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.549+0000 7f987052c700 1 -- 172.21.15.144:0/3271367919 shutdown_connections 2024-09-10T11:49:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.549+0000 7f987052c700 1 --2- 172.21.15.144:0/3271367919 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f985406e200 0x7f98540706b0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.549+0000 7f987052c700 1 --2- 172.21.15.144:0/3271367919 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f98680ff2c0 0x7f9868196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.549+0000 7f987052c700 1 --2- 172.21.15.144:0/3271367919 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f98680ffc60 0x7f98681970e0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:49:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.549+0000 7f987052c700 1 -- 172.21.15.144:0/3271367919 >> 172.21.15.144:0/3271367919 conn(0x7f98680fab80 msgr2=0x7f98680fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:49:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.549+0000 7f987052c700 1 -- 172.21.15.144:0/3271367919 shutdown_connections 2024-09-10T11:49:54.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:49:54.549+0000 7f987052c700 1 -- 172.21.15.144:0/3271367919 wait complete. 2024-09-10T11:49:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:54 smithi178 ceph-mon[65940]: from='client.35024 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:54 smithi178 ceph-mon[65940]: pgmap v537: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:54 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3271367919' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:49:55.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:54 smithi144 ceph-mon[87436]: from='client.35024 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:54 smithi144 ceph-mon[87436]: pgmap v537: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:54 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3271367919' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:49:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:55 smithi144 ceph-mon[87436]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:56.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:55 smithi178 ceph-mon[65940]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:49:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:56 smithi144 ceph-mon[87436]: pgmap v538: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:56 smithi178 ceph-mon[65940]: pgmap v538: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:58.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:57 smithi144 ceph-mon[87436]: pgmap v539: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:57 smithi178 ceph-mon[65940]: pgmap v539: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:49:59.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:49:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:50:00.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:49:59 smithi178 ceph-mon[65940]: pgmap v540: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:00.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:49:59 smithi144 ceph-mon[87436]: pgmap v540: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:00.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:49:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:50:01.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:00 smithi144 ceph-mon[87436]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:50:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:00 smithi178 ceph-mon[65940]: overall HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:50:02.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:01 smithi144 ceph-mon[87436]: pgmap v541: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:02.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:01 smithi178 ceph-mon[65940]: pgmap v541: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:04 smithi178 ceph-mon[65940]: pgmap v542: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:05.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:04 smithi144 ceph-mon[87436]: pgmap v542: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:06 smithi178 ceph-mon[65940]: pgmap v543: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:07.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:06 smithi144 ceph-mon[87436]: pgmap v543: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:08 smithi178 ceph-mon[65940]: pgmap v544: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:08 smithi144 ceph-mon[87436]: pgmap v544: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:10 smithi178 ceph-mon[65940]: pgmap v545: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:11.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:10 smithi144 ceph-mon[87436]: pgmap v545: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:12 smithi178 ceph-mon[65940]: pgmap v546: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:13.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:12 smithi144 ceph-mon[87436]: pgmap v546: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:14.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:50:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:50:15.197 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:14 smithi144 ceph-mon[87436]: pgmap v547: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:14 smithi178 ceph-mon[65940]: pgmap v547: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:16.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:15 smithi144 ceph-mon[87436]: pgmap v548: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:16.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:15 smithi178 ceph-mon[65940]: pgmap v548: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:18 smithi178 ceph-mon[65940]: pgmap v549: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:19.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:18 smithi144 ceph-mon[87436]: pgmap v549: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:20 smithi178 ceph-mon[65940]: pgmap v550: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:21.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:20 smithi144 ceph-mon[87436]: pgmap v550: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:23.117 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:22 smithi144 ceph-mon[87436]: pgmap v551: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:22 smithi178 ceph-mon[65940]: pgmap v551: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:24.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.668+0000 7faac384e700 1 -- 172.21.15.144:0/1162803345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faabc1003d0 msgr2=0x7faabc1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:24.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.668+0000 7faac384e700 1 --2- 172.21.15.144:0/1162803345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faabc1003d0 0x7faabc1007a0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7faab4009910 tx=0x7faab4009c20 comp rx=0 tx=0).stop 2024-09-10T11:50:24.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.668+0000 7faac384e700 1 -- 172.21.15.144:0/1162803345 shutdown_connections 2024-09-10T11:50:24.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.668+0000 7faac384e700 1 --2- 172.21.15.144:0/1162803345 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faabc100d70 0x7faabc104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:24.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.668+0000 7faac384e700 1 --2- 172.21.15.144:0/1162803345 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faabc1003d0 0x7faabc1007a0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:24.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.668+0000 7faac384e700 1 -- 172.21.15.144:0/1162803345 >> 172.21.15.144:0/1162803345 conn(0x7faabc0fbc90 msgr2=0x7faabc0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:24.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.669+0000 7faac384e700 1 -- 172.21.15.144:0/1162803345 shutdown_connections 2024-09-10T11:50:24.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.669+0000 7faac384e700 1 -- 172.21.15.144:0/1162803345 wait complete. 2024-09-10T11:50:24.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.669+0000 7faac384e700 1 Processor -- start 2024-09-10T11:50:24.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.669+0000 7faac384e700 1 -- start start 2024-09-10T11:50:24.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.669+0000 7faac384e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faabc1003d0 0x7faabc195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:24.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.669+0000 7faac384e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faabc100d70 0x7faabc1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:24.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.669+0000 7faac384e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc196980 con 0x7faabc1003d0 2024-09-10T11:50:24.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.669+0000 7faac384e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faabc19a710 con 0x7faabc100d70 2024-09-10T11:50:24.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.670+0000 7faac15ea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faabc1003d0 0x7faabc195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:24.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.670+0000 7faac15ea700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faabc1003d0 0x7faabc195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51510/0 (socket says 172.21.15.144:51510) 2024-09-10T11:50:24.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.670+0000 7faac15ea700 1 -- 172.21.15.144:0/3370932718 learned_addr learned my addr 172.21.15.144:0/3370932718 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:50:24.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.670+0000 7faac0de9700 1 --2- 172.21.15.144:0/3370932718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faabc100d70 0x7faabc1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:24.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.670+0000 7faac15ea700 1 -- 172.21.15.144:0/3370932718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faabc100d70 msgr2=0x7faabc1962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:24.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.670+0000 7faac15ea700 1 --2- 172.21.15.144:0/3370932718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faabc100d70 0x7faabc1962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:24.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.670+0000 7faac15ea700 1 -- 172.21.15.144:0/3370932718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab40095f0 con 0x7faabc1003d0 2024-09-10T11:50:24.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.670+0000 7faac15ea700 1 --2- 172.21.15.144:0/3370932718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faabc1003d0 0x7faabc195d60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faab40092e0 tx=0x7faab400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:24.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.670+0000 7faaba7fc700 1 -- 172.21.15.144:0/3370932718 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faab401d070 con 0x7faabc1003d0 2024-09-10T11:50:24.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.670+0000 7faaba7fc700 1 -- 172.21.15.144:0/3370932718 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faab4005e00 con 0x7faabc1003d0 2024-09-10T11:50:24.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.670+0000 7faac384e700 1 -- 172.21.15.144:0/3370932718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faabc19a990 con 0x7faabc1003d0 2024-09-10T11:50:24.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.670+0000 7faaba7fc700 1 -- 172.21.15.144:0/3370932718 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faab4017400 con 0x7faabc1003d0 2024-09-10T11:50:24.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.670+0000 7faac384e700 1 -- 172.21.15.144:0/3370932718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faabc19ae80 con 0x7faabc1003d0 2024-09-10T11:50:24.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.671+0000 7faac384e700 1 -- 172.21.15.144:0/3370932718 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faabc1086e0 con 0x7faabc1003d0 2024-09-10T11:50:24.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.672+0000 7faaba7fc700 1 -- 172.21.15.144:0/3370932718 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7faab400f6f0 con 0x7faabc1003d0 2024-09-10T11:50:24.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.673+0000 7faaba7fc700 1 --2- 172.21.15.144:0/3370932718 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faaa406e200 0x7faaa40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:24.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.673+0000 7faaba7fc700 1 -- 172.21.15.144:0/3370932718 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7faab408f4f0 con 0x7faabc1003d0 2024-09-10T11:50:24.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.673+0000 7faac0de9700 1 --2- 172.21.15.144:0/3370932718 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faaa406e200 0x7faaa40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:24.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.673+0000 7faac0de9700 1 --2- 172.21.15.144:0/3370932718 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faaa406e200 0x7faaa40706b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faabc197380 tx=0x7faab000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:24.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.677+0000 7faaba7fc700 1 -- 172.21.15.144:0/3370932718 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7faab40588b0 con 0x7faabc1003d0 2024-09-10T11:50:24.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.897+0000 7faac384e700 1 -- 172.21.15.144:0/3370932718 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faabc19b220 con 0x7faaa406e200 2024-09-10T11:50:24.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.898+0000 7faaba7fc700 1 -- 172.21.15.144:0/3370932718 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7faabc19b220 con 0x7faaa406e200 2024-09-10T11:50:24.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.900+0000 7faac384e700 1 -- 172.21.15.144:0/3370932718 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faaa406e200 msgr2=0x7faaa40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:24.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.900+0000 7faac384e700 1 --2- 172.21.15.144:0/3370932718 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faaa406e200 0x7faaa40706b0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7faabc197380 tx=0x7faab000b410 comp rx=0 tx=0).stop 2024-09-10T11:50:24.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.900+0000 7faac384e700 1 -- 172.21.15.144:0/3370932718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faabc1003d0 msgr2=0x7faabc195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:24.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.900+0000 7faac384e700 1 --2- 172.21.15.144:0/3370932718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faabc1003d0 0x7faabc195d60 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7faab40092e0 tx=0x7faab400bd70 comp rx=0 tx=0).stop 2024-09-10T11:50:24.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.900+0000 7faac384e700 1 -- 172.21.15.144:0/3370932718 shutdown_connections 2024-09-10T11:50:24.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.900+0000 7faac384e700 1 --2- 172.21.15.144:0/3370932718 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faaa406e200 0x7faaa40706b0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:24.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.900+0000 7faac384e700 1 --2- 172.21.15.144:0/3370932718 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faabc1003d0 0x7faabc195d60 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:24.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.900+0000 7faac384e700 1 --2- 172.21.15.144:0/3370932718 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faabc100d70 0x7faabc1962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:24.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.900+0000 7faac384e700 1 -- 172.21.15.144:0/3370932718 >> 172.21.15.144:0/3370932718 conn(0x7faabc0fbc90 msgr2=0x7faabc0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:24.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.901+0000 7faac384e700 1 -- 172.21.15.144:0/3370932718 shutdown_connections 2024-09-10T11:50:24.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:24.901+0000 7faac384e700 1 -- 172.21.15.144:0/3370932718 wait complete. 2024-09-10T11:50:24.912 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:50:24.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:24 smithi144 ceph-mon[87436]: pgmap v552: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:25.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.006+0000 7f4126a74700 1 -- 172.21.15.144:0/875074745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200ff260 msgr2=0x7f41200ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:25.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.006+0000 7f4126a74700 1 --2- 172.21.15.144:0/875074745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200ff260 0x7f41200ff630 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f4114009910 tx=0x7f4114009c20 comp rx=0 tx=0).stop 2024-09-10T11:50:25.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.006+0000 7f4126a74700 1 -- 172.21.15.144:0/875074745 shutdown_connections 2024-09-10T11:50:25.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.006+0000 7f4126a74700 1 --2- 172.21.15.144:0/875074745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41200ffb70 0x7f412010a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.006+0000 7f4126a74700 1 --2- 172.21.15.144:0/875074745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200ff260 0x7f41200ff630 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.006+0000 7f4126a74700 1 -- 172.21.15.144:0/875074745 >> 172.21.15.144:0/875074745 conn(0x7f41200747e0 msgr2=0x7f4120074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:25.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.007+0000 7f4126a74700 1 -- 172.21.15.144:0/875074745 shutdown_connections 2024-09-10T11:50:25.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.007+0000 7f4126a74700 1 -- 172.21.15.144:0/875074745 wait complete. 2024-09-10T11:50:25.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.007+0000 7f4126a74700 1 Processor -- start 2024-09-10T11:50:25.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.007+0000 7f4126a74700 1 -- start start 2024-09-10T11:50:25.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.007+0000 7f4126a74700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200ff260 0x7f4120195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:25.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f4126a74700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41200ffb70 0x7f41201962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:25.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f4126a74700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4120196990 con 0x7f41200ff260 2024-09-10T11:50:25.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f4126a74700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412019a720 con 0x7f41200ffb70 2024-09-10T11:50:25.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f4124810700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200ff260 0x7f4120195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:25.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f4124810700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200ff260 0x7f4120195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51528/0 (socket says 172.21.15.144:51528) 2024-09-10T11:50:25.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f4124810700 1 -- 172.21.15.144:0/144027768 learned_addr learned my addr 172.21.15.144:0/144027768 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:50:25.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f411ffff700 1 --2- 172.21.15.144:0/144027768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41200ffb70 0x7f41201962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:25.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f4124810700 1 -- 172.21.15.144:0/144027768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41200ffb70 msgr2=0x7f41201962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:25.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f4124810700 1 --2- 172.21.15.144:0/144027768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41200ffb70 0x7f41201962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f4124810700 1 -- 172.21.15.144:0/144027768 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41140095f0 con 0x7f41200ff260 2024-09-10T11:50:25.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f4124810700 1 --2- 172.21.15.144:0/144027768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200ff260 0x7f4120195d70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4114006010 tx=0x7f411400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:25.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f411dffb700 1 -- 172.21.15.144:0/144027768 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f411401d070 con 0x7f41200ff260 2024-09-10T11:50:25.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f411dffb700 1 -- 172.21.15.144:0/144027768 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4114005dc0 con 0x7f41200ff260 2024-09-10T11:50:25.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f4126a74700 1 -- 172.21.15.144:0/144027768 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f412019aa00 con 0x7f41200ff260 2024-09-10T11:50:25.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f411dffb700 1 -- 172.21.15.144:0/144027768 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4114017400 con 0x7f41200ff260 2024-09-10T11:50:25.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.008+0000 7f4126a74700 1 -- 172.21.15.144:0/144027768 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f412019af50 con 0x7f41200ff260 2024-09-10T11:50:25.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.009+0000 7f4126a74700 1 -- 172.21.15.144:0/144027768 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4120107ce0 con 0x7f41200ff260 2024-09-10T11:50:25.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.010+0000 7f411dffb700 1 -- 172.21.15.144:0/144027768 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4114003a60 con 0x7f41200ff260 2024-09-10T11:50:25.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.010+0000 7f411dffb700 1 --2- 172.21.15.144:0/144027768 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f411006e200 0x7f41100706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:25.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.010+0000 7f411dffb700 1 -- 172.21.15.144:0/144027768 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f411408f4a0 con 0x7f41200ff260 2024-09-10T11:50:25.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.010+0000 7f411ffff700 1 --2- 172.21.15.144:0/144027768 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f411006e200 0x7f41100706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:25.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.011+0000 7f411ffff700 1 --2- 172.21.15.144:0/144027768 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f411006e200 0x7f41100706b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4120197390 tx=0x7f410c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:25.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.014+0000 7f411dffb700 1 -- 172.21.15.144:0/144027768 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4114058880 con 0x7f41200ff260 2024-09-10T11:50:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:24 smithi178 ceph-mon[65940]: pgmap v552: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:25.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.234+0000 7f4126a74700 1 -- 172.21.15.144:0/144027768 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f412019b230 con 0x7f411006e200 2024-09-10T11:50:25.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.236+0000 7f411dffb700 1 -- 172.21.15.144:0/144027768 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f412019b230 con 0x7f411006e200 2024-09-10T11:50:25.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.238+0000 7f4126a74700 1 -- 172.21.15.144:0/144027768 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f411006e200 msgr2=0x7f41100706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:25.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.238+0000 7f4126a74700 1 --2- 172.21.15.144:0/144027768 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f411006e200 0x7f41100706b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4120197390 tx=0x7f410c00b410 comp rx=0 tx=0).stop 2024-09-10T11:50:25.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.238+0000 7f4126a74700 1 -- 172.21.15.144:0/144027768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200ff260 msgr2=0x7f4120195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:25.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.238+0000 7f4126a74700 1 --2- 172.21.15.144:0/144027768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200ff260 0x7f4120195d70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4114006010 tx=0x7f411400bd70 comp rx=0 tx=0).stop 2024-09-10T11:50:25.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.238+0000 7f4126a74700 1 -- 172.21.15.144:0/144027768 shutdown_connections 2024-09-10T11:50:25.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.238+0000 7f4126a74700 1 --2- 172.21.15.144:0/144027768 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f411006e200 0x7f41100706b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.238+0000 7f4126a74700 1 --2- 172.21.15.144:0/144027768 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41200ff260 0x7f4120195d70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.238+0000 7f4126a74700 1 --2- 172.21.15.144:0/144027768 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41200ffb70 0x7f41201962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.238+0000 7f4126a74700 1 -- 172.21.15.144:0/144027768 >> 172.21.15.144:0/144027768 conn(0x7f41200747e0 msgr2=0x7f4120104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:25.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.238+0000 7f4126a74700 1 -- 172.21.15.144:0/144027768 shutdown_connections 2024-09-10T11:50:25.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.238+0000 7f4126a74700 1 -- 172.21.15.144:0/144027768 wait complete. 2024-09-10T11:50:25.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.342+0000 7f57610b6700 1 -- 172.21.15.144:0/2032526052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f575c1003b0 msgr2=0x7f575c100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:25.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.342+0000 7f57610b6700 1 --2- 172.21.15.144:0/2032526052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f575c1003b0 0x7f575c100780 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f5750009910 tx=0x7f5750009c20 comp rx=0 tx=0).stop 2024-09-10T11:50:25.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.342+0000 7f57610b6700 1 -- 172.21.15.144:0/2032526052 shutdown_connections 2024-09-10T11:50:25.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.342+0000 7f57610b6700 1 --2- 172.21.15.144:0/2032526052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f575c100d50 0x7f575c104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.342+0000 7f57610b6700 1 --2- 172.21.15.144:0/2032526052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f575c1003b0 0x7f575c100780 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.342+0000 7f57610b6700 1 -- 172.21.15.144:0/2032526052 >> 172.21.15.144:0/2032526052 conn(0x7f575c0fbc30 msgr2=0x7f575c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:25.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.342+0000 7f57610b6700 1 -- 172.21.15.144:0/2032526052 shutdown_connections 2024-09-10T11:50:25.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.342+0000 7f57610b6700 1 -- 172.21.15.144:0/2032526052 wait complete. 2024-09-10T11:50:25.346 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.343+0000 7f57610b6700 1 Processor -- start 2024-09-10T11:50:25.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.343+0000 7f57610b6700 1 -- start start 2024-09-10T11:50:25.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.343+0000 7f57610b6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f575c1003b0 0x7f575c197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:25.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.343+0000 7f57610b6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f575c100d50 0x7f575c1984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:25.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.343+0000 7f57610b6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c198bc0 con 0x7f575c100d50 2024-09-10T11:50:25.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.343+0000 7f57610b6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f575c19c950 con 0x7f575c1003b0 2024-09-10T11:50:25.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.343+0000 7f575a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f575c100d50 0x7f575c1984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:25.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.344+0000 7f575a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f575c100d50 0x7f575c1984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51542/0 (socket says 172.21.15.144:51542) 2024-09-10T11:50:25.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.344+0000 7f575a59c700 1 -- 172.21.15.144:0/3464389878 learned_addr learned my addr 172.21.15.144:0/3464389878 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:50:25.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.344+0000 7f575ad9d700 1 --2- 172.21.15.144:0/3464389878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f575c1003b0 0x7f575c197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:25.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.344+0000 7f575a59c700 1 -- 172.21.15.144:0/3464389878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f575c1003b0 msgr2=0x7f575c197fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:25.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.344+0000 7f575a59c700 1 --2- 172.21.15.144:0/3464389878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f575c1003b0 0x7f575c197fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.344+0000 7f575a59c700 1 -- 172.21.15.144:0/3464389878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57500095f0 con 0x7f575c100d50 2024-09-10T11:50:25.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.344+0000 7f575a59c700 1 --2- 172.21.15.144:0/3464389878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f575c100d50 0x7f575c1984e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f574c00d900 tx=0x7f574c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:25.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.344+0000 7f5743fff700 1 -- 172.21.15.144:0/3464389878 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f574c009900 con 0x7f575c100d50 2024-09-10T11:50:25.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.344+0000 7f5743fff700 1 -- 172.21.15.144:0/3464389878 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f574c010460 con 0x7f575c100d50 2024-09-10T11:50:25.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.344+0000 7f57610b6700 1 -- 172.21.15.144:0/3464389878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f575c19cc30 con 0x7f575c100d50 2024-09-10T11:50:25.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.344+0000 7f57610b6700 1 -- 172.21.15.144:0/3464389878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f575c19d180 con 0x7f575c100d50 2024-09-10T11:50:25.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.344+0000 7f5743fff700 1 -- 172.21.15.144:0/3464389878 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f574c00f5f0 con 0x7f575c100d50 2024-09-10T11:50:25.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.346+0000 7f57610b6700 1 -- 172.21.15.144:0/3464389878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f575c1044a0 con 0x7f575c100d50 2024-09-10T11:50:25.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.347+0000 7f5743fff700 1 -- 172.21.15.144:0/3464389878 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f574c0105d0 con 0x7f575c100d50 2024-09-10T11:50:25.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.347+0000 7f5743fff700 1 --2- 172.21.15.144:0/3464389878 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f574406e200 0x7f57440706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:25.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.347+0000 7f5743fff700 1 -- 172.21.15.144:0/3464389878 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f574c08dc60 con 0x7f575c100d50 2024-09-10T11:50:25.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.347+0000 7f575ad9d700 1 --2- 172.21.15.144:0/3464389878 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f574406e200 0x7f57440706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:25.349 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.348+0000 7f575ad9d700 1 --2- 172.21.15.144:0/3464389878 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f574406e200 0x7f57440706b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5750006010 tx=0x7f575001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:25.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.352+0000 7f5743fff700 1 -- 172.21.15.144:0/3464389878 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f574c057040 con 0x7f575c100d50 2024-09-10T11:50:25.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.562+0000 7f57610b6700 1 -- 172.21.15.144:0/3464389878 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f575c19d3f0 con 0x7f574406e200 2024-09-10T11:50:25.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.571+0000 7f5743fff700 1 -- 172.21.15.144:0/3464389878 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f575c19d3f0 con 0x7f574406e200 2024-09-10T11:50:25.572 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:50:25.573 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (18m) 12m ago 29m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:50:25.573 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (29m) 12m ago 29m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:50:25.573 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (28m) 12m ago 28m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:50:25.573 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (16m) 12m ago 29m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:50:25.573 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (16m) 12m ago 28m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:50:25.573 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (18m) 12m ago 28m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:50:25.573 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (12m) 12m ago 23m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:50:25.573 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (12m) 12m ago 23m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:50:25.573 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (23m) 12m ago 23m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:50:25.573 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (12m) 12m ago 23m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:50:25.573 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (20m) 12m ago 30m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:50:25.573 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (19m) 12m ago 28m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:50:25.573 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (17m) 12m ago 30m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:50:25.574 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (16m) 12m ago 28m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:50:25.574 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (19m) 12m ago 29m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:50:25.574 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (19m) 12m ago 28m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:50:25.574 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (15m) 12m ago 27m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:50:25.574 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (15m) 12m ago 26m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:50:25.574 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (14m) 12m ago 26m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:50:25.574 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (14m) 12m ago 26m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:50:25.574 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (13m) 12m ago 25m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:50:25.574 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (13m) 12m ago 25m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:50:25.574 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (18m) 12m ago 28m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:50:25.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.574+0000 7f57610b6700 1 -- 172.21.15.144:0/3464389878 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f574406e200 msgr2=0x7f57440706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:25.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.574+0000 7f57610b6700 1 --2- 172.21.15.144:0/3464389878 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f574406e200 0x7f57440706b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f5750006010 tx=0x7f575001a040 comp rx=0 tx=0).stop 2024-09-10T11:50:25.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.574+0000 7f57610b6700 1 -- 172.21.15.144:0/3464389878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f575c100d50 msgr2=0x7f575c1984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:25.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.574+0000 7f57610b6700 1 --2- 172.21.15.144:0/3464389878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f575c100d50 0x7f575c1984e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f574c00d900 tx=0x7f574c00dc10 comp rx=0 tx=0).stop 2024-09-10T11:50:25.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.574+0000 7f57610b6700 1 -- 172.21.15.144:0/3464389878 shutdown_connections 2024-09-10T11:50:25.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.574+0000 7f57610b6700 1 --2- 172.21.15.144:0/3464389878 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f574406e200 0x7f57440706b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.574+0000 7f57610b6700 1 --2- 172.21.15.144:0/3464389878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f575c1003b0 0x7f575c197fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.574+0000 7f57610b6700 1 --2- 172.21.15.144:0/3464389878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f575c100d50 0x7f575c1984e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.574+0000 7f57610b6700 1 -- 172.21.15.144:0/3464389878 >> 172.21.15.144:0/3464389878 conn(0x7f575c0fbc30 msgr2=0x7f575c0fd1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:25.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.574+0000 7f57610b6700 1 -- 172.21.15.144:0/3464389878 shutdown_connections 2024-09-10T11:50:25.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.574+0000 7f57610b6700 1 -- 172.21.15.144:0/3464389878 wait complete. 2024-09-10T11:50:25.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.677+0000 7efd8932e700 1 -- 172.21.15.144:0/3402435029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd840ff340 msgr2=0x7efd840ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:25.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.677+0000 7efd8932e700 1 --2- 172.21.15.144:0/3402435029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd840ff340 0x7efd840ff710 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efd7400b0a0 tx=0x7efd7400b3b0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.677+0000 7efd8932e700 1 -- 172.21.15.144:0/3402435029 shutdown_connections 2024-09-10T11:50:25.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.677+0000 7efd8932e700 1 --2- 172.21.15.144:0/3402435029 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd840ffce0 0x7efd84103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.677+0000 7efd8932e700 1 --2- 172.21.15.144:0/3402435029 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd840ff340 0x7efd840ff710 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.677+0000 7efd8932e700 1 -- 172.21.15.144:0/3402435029 >> 172.21.15.144:0/3402435029 conn(0x7efd840fac00 msgr2=0x7efd840fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:25.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.677+0000 7efd8932e700 1 -- 172.21.15.144:0/3402435029 shutdown_connections 2024-09-10T11:50:25.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.677+0000 7efd8932e700 1 -- 172.21.15.144:0/3402435029 wait complete. 2024-09-10T11:50:25.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.677+0000 7efd8932e700 1 Processor -- start 2024-09-10T11:50:25.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.678+0000 7efd8932e700 1 -- start start 2024-09-10T11:50:25.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.678+0000 7efd8932e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd840ff340 0x7efd84100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:25.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.678+0000 7efd8932e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd840ffce0 0x7efd84101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.678+0000 7efd8932e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd841017d0 con 0x7efd840ff340 2024-09-10T11:50:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.678+0000 7efd8932e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd84101910 con 0x7efd840ffce0 2024-09-10T11:50:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.678+0000 7efd82ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd840ff340 0x7efd84100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.678+0000 7efd82ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd840ff340 0x7efd84100d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51554/0 (socket says 172.21.15.144:51554) 2024-09-10T11:50:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.678+0000 7efd82ffd700 1 -- 172.21.15.144:0/3868864224 learned_addr learned my addr 172.21.15.144:0/3868864224 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:50:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.678+0000 7efd827fc700 1 --2- 172.21.15.144:0/3868864224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd840ffce0 0x7efd84101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.678+0000 7efd82ffd700 1 -- 172.21.15.144:0/3868864224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd840ffce0 msgr2=0x7efd84101290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.678+0000 7efd82ffd700 1 --2- 172.21.15.144:0/3868864224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd840ffce0 0x7efd84101290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.678+0000 7efd82ffd700 1 -- 172.21.15.144:0/3868864224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd74009d00 con 0x7efd840ff340 2024-09-10T11:50:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.678+0000 7efd82ffd700 1 --2- 172.21.15.144:0/3868864224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd840ff340 0x7efd84100d50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efd7400b070 tx=0x7efd7400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.679+0000 7efd7bfff700 1 -- 172.21.15.144:0/3868864224 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd7400e040 con 0x7efd840ff340 2024-09-10T11:50:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.679+0000 7efd7bfff700 1 -- 172.21.15.144:0/3868864224 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efd74003c10 con 0x7efd840ff340 2024-09-10T11:50:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.679+0000 7efd8932e700 1 -- 172.21.15.144:0/3868864224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd84101b90 con 0x7efd840ff340 2024-09-10T11:50:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.679+0000 7efd7bfff700 1 -- 172.21.15.144:0/3868864224 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd7401d740 con 0x7efd840ff340 2024-09-10T11:50:25.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.679+0000 7efd8932e700 1 -- 172.21.15.144:0/3868864224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd841a2350 con 0x7efd840ff340 2024-09-10T11:50:25.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.680+0000 7efd8932e700 1 -- 172.21.15.144:0/3868864224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd84108660 con 0x7efd840ff340 2024-09-10T11:50:25.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.680+0000 7efd7bfff700 1 -- 172.21.15.144:0/3868864224 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7efd74004040 con 0x7efd840ff340 2024-09-10T11:50:25.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.681+0000 7efd7bfff700 1 --2- 172.21.15.144:0/3868864224 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7efd7006e200 0x7efd700706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:25.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.681+0000 7efd7bfff700 1 -- 172.21.15.144:0/3868864224 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7efd7408f7e0 con 0x7efd840ff340 2024-09-10T11:50:25.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.681+0000 7efd827fc700 1 --2- 172.21.15.144:0/3868864224 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7efd7006e200 0x7efd700706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:25.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.681+0000 7efd827fc700 1 --2- 172.21.15.144:0/3868864224 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7efd7006e200 0x7efd700706b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efd84102540 tx=0x7efd6c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:25.686 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.685+0000 7efd7bfff700 1 -- 172.21.15.144:0/3868864224 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7efd74058c40 con 0x7efd840ff340 2024-09-10T11:50:25.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.975+0000 7efd8932e700 1 -- 172.21.15.144:0/3868864224 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efd8404ea50 con 0x7efd840ff340 2024-09-10T11:50:25.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.975+0000 7efd7bfff700 1 -- 172.21.15.144:0/3868864224 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7efd7405c300 con 0x7efd840ff340 2024-09-10T11:50:25.976 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:50:25.976 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:50:25.976 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:50:25.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:50:25.977 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:50:25.977 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:50:25.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:50:25.977 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:50:25.977 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:50:25.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:50:25.977 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:50:25.977 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:50:25.977 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:50:25.977 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:50:25.977 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:50:25.977 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:50:25.977 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:50:25.978 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:50:25.978 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:50:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.977+0000 7efd8932e700 1 -- 172.21.15.144:0/3868864224 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7efd7006e200 msgr2=0x7efd700706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.977+0000 7efd8932e700 1 --2- 172.21.15.144:0/3868864224 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7efd7006e200 0x7efd700706b0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efd84102540 tx=0x7efd6c00b410 comp rx=0 tx=0).stop 2024-09-10T11:50:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.977+0000 7efd8932e700 1 -- 172.21.15.144:0/3868864224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd840ff340 msgr2=0x7efd84100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.977+0000 7efd8932e700 1 --2- 172.21.15.144:0/3868864224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd840ff340 0x7efd84100d50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efd7400b070 tx=0x7efd7400cd20 comp rx=0 tx=0).stop 2024-09-10T11:50:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.977+0000 7efd8932e700 1 -- 172.21.15.144:0/3868864224 shutdown_connections 2024-09-10T11:50:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.977+0000 7efd8932e700 1 --2- 172.21.15.144:0/3868864224 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7efd7006e200 0x7efd700706b0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.978+0000 7efd8932e700 1 --2- 172.21.15.144:0/3868864224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7efd840ff340 0x7efd84100d50 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.978+0000 7efd8932e700 1 --2- 172.21.15.144:0/3868864224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7efd840ffce0 0x7efd84101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.978+0000 7efd8932e700 1 -- 172.21.15.144:0/3868864224 >> 172.21.15.144:0/3868864224 conn(0x7efd840fac00 msgr2=0x7efd84069010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.978+0000 7efd8932e700 1 -- 172.21.15.144:0/3868864224 shutdown_connections 2024-09-10T11:50:25.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:25.978+0000 7efd8932e700 1 -- 172.21.15.144:0/3868864224 wait complete. 2024-09-10T11:50:26.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.082+0000 7f8fdc2e0700 1 -- 172.21.15.144:0/339979088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd40ff390 msgr2=0x7f8fd40ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:26.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.082+0000 7f8fdc2e0700 1 --2- 172.21.15.144:0/339979088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd40ff390 0x7f8fd40ff760 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0009910 tx=0x7f8fd0009c20 comp rx=0 tx=0).stop 2024-09-10T11:50:26.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.082+0000 7f8fdc2e0700 1 -- 172.21.15.144:0/339979088 shutdown_connections 2024-09-10T11:50:26.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.082+0000 7f8fdc2e0700 1 --2- 172.21.15.144:0/339979088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd40ffd30 0x7f8fd4103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.082+0000 7f8fdc2e0700 1 --2- 172.21.15.144:0/339979088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd40ff390 0x7f8fd40ff760 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.082+0000 7f8fdc2e0700 1 -- 172.21.15.144:0/339979088 >> 172.21.15.144:0/339979088 conn(0x7f8fd4074f50 msgr2=0x7f8fd4075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.082+0000 7f8fdc2e0700 1 -- 172.21.15.144:0/339979088 shutdown_connections 2024-09-10T11:50:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.082+0000 7f8fdc2e0700 1 -- 172.21.15.144:0/339979088 wait complete. 2024-09-10T11:50:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.083+0000 7f8fdc2e0700 1 Processor -- start 2024-09-10T11:50:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.083+0000 7f8fdc2e0700 1 -- start start 2024-09-10T11:50:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.083+0000 7f8fdc2e0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd40ff390 0x7f8fd419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.083+0000 7f8fdc2e0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd40ffd30 0x7f8fd419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.083+0000 7f8fdc2e0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd419b640 con 0x7f8fd40ffd30 2024-09-10T11:50:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.083+0000 7f8fdc2e0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd4194b00 con 0x7f8fd40ff390 2024-09-10T11:50:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.083+0000 7f8fd987b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd40ffd30 0x7f8fd419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.083+0000 7f8fd987b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd40ffd30 0x7f8fd419af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51582/0 (socket says 172.21.15.144:51582) 2024-09-10T11:50:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.083+0000 7f8fd987b700 1 -- 172.21.15.144:0/1609193353 learned_addr learned my addr 172.21.15.144:0/1609193353 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:50:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.083+0000 7f8fda07c700 1 --2- 172.21.15.144:0/1609193353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd40ff390 0x7f8fd419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:26.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.083+0000 7f8fd987b700 1 -- 172.21.15.144:0/1609193353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd40ff390 msgr2=0x7f8fd419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.083+0000 7f8fd987b700 1 --2- 172.21.15.144:0/1609193353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd40ff390 0x7f8fd419aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.083+0000 7f8fd987b700 1 -- 172.21.15.144:0/1609193353 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd00095f0 con 0x7f8fd40ffd30 2024-09-10T11:50:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.083+0000 7f8fda07c700 1 --2- 172.21.15.144:0/1609193353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd40ff390 0x7f8fd419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:50:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.084+0000 7f8fd987b700 1 --2- 172.21.15.144:0/1609193353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd40ffd30 0x7f8fd419af60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400d900 tx=0x7f8fc400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.084+0000 7f8fcb7fe700 1 -- 172.21.15.144:0/1609193353 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fc4009900 con 0x7f8fd40ffd30 2024-09-10T11:50:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.084+0000 7f8fdc2e0700 1 -- 172.21.15.144:0/1609193353 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd4194de0 con 0x7f8fd40ffd30 2024-09-10T11:50:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.084+0000 7f8fcb7fe700 1 -- 172.21.15.144:0/1609193353 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8fc4010460 con 0x7f8fd40ffd30 2024-09-10T11:50:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.084+0000 7f8fcb7fe700 1 -- 172.21.15.144:0/1609193353 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fc400f5f0 con 0x7f8fd40ffd30 2024-09-10T11:50:26.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.084+0000 7f8fdc2e0700 1 -- 172.21.15.144:0/1609193353 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd4195330 con 0x7f8fd40ffd30 2024-09-10T11:50:26.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.085+0000 7f8fdc2e0700 1 -- 172.21.15.144:0/1609193353 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd4107650 con 0x7f8fd40ffd30 2024-09-10T11:50:26.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.085+0000 7f8fcb7fe700 1 -- 172.21.15.144:0/1609193353 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8fc40105d0 con 0x7f8fd40ffd30 2024-09-10T11:50:26.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.086+0000 7f8fcb7fe700 1 --2- 172.21.15.144:0/1609193353 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8fc006e200 0x7f8fc00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:26.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.086+0000 7f8fcb7fe700 1 -- 172.21.15.144:0/1609193353 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8fc408df20 con 0x7f8fd40ffd30 2024-09-10T11:50:26.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.086+0000 7f8fda07c700 1 --2- 172.21.15.144:0/1609193353 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8fc006e200 0x7f8fc00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:26.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.086+0000 7f8fda07c700 1 --2- 172.21.15.144:0/1609193353 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8fc006e200 0x7f8fc00706b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0005b40 tx=0x7f8fd001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:26.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.090+0000 7f8fcb7fe700 1 -- 172.21.15.144:0/1609193353 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8fc40c31b0 con 0x7f8fd40ffd30 2024-09-10T11:50:26.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.336+0000 7f8fdc2e0700 1 -- 172.21.15.144:0/1609193353 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8fd404ea50 con 0x7f8fd40ffd30 2024-09-10T11:50:26.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.337+0000 7f8fcb7fe700 1 -- 172.21.15.144:0/1609193353 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f8fc405ae30 con 0x7f8fd40ffd30 2024-09-10T11:50:26.338 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:50:26.338 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:50:26.338 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:50:26.338 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:50:26.338 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:50:26.338 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:50:26.339 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:50:26.339 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:50:26.339 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:50:26.339 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:50:26.339 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:50:26.339 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:50:26.339 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:50:26.339 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:50:26.339 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:50:26.339 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:50:26.339 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:50:26.339 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:50:26.339 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:50:26.339 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:50:26.340 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:50:26.340 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:50:26.340 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:50:26.340 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:50:26.340 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:50:26.340 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:50:26.340 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:50:26.340 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:50:26.340 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:50:26.341 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:50:26.341 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:50:26.341 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:50:26.341 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:50:26.341 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:50:26.341 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:50:26.341 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:50:26.341 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:50:26.341 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:50:26.341 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:50:26.341 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:50:26.341 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:50:26.342 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:50:26.342 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:50:26.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.339+0000 7f8fdc2e0700 1 -- 172.21.15.144:0/1609193353 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8fc006e200 msgr2=0x7f8fc00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:26.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.339+0000 7f8fdc2e0700 1 --2- 172.21.15.144:0/1609193353 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8fc006e200 0x7f8fc00706b0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8fd0005b40 tx=0x7f8fd001a040 comp rx=0 tx=0).stop 2024-09-10T11:50:26.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.339+0000 7f8fdc2e0700 1 -- 172.21.15.144:0/1609193353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd40ffd30 msgr2=0x7f8fd419af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:26.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.339+0000 7f8fdc2e0700 1 --2- 172.21.15.144:0/1609193353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd40ffd30 0x7f8fd419af60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400d900 tx=0x7f8fc400dcc0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.339+0000 7f8fdc2e0700 1 -- 172.21.15.144:0/1609193353 shutdown_connections 2024-09-10T11:50:26.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.339+0000 7f8fdc2e0700 1 --2- 172.21.15.144:0/1609193353 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8fc006e200 0x7f8fc00706b0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.339+0000 7f8fdc2e0700 1 --2- 172.21.15.144:0/1609193353 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8fd40ff390 0x7f8fd419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.339+0000 7f8fdc2e0700 1 --2- 172.21.15.144:0/1609193353 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8fd40ffd30 0x7f8fd419af60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.339+0000 7f8fdc2e0700 1 -- 172.21.15.144:0/1609193353 >> 172.21.15.144:0/1609193353 conn(0x7f8fd4074f50 msgr2=0x7f8fd40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:26.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.339+0000 7f8fdc2e0700 1 -- 172.21.15.144:0/1609193353 shutdown_connections 2024-09-10T11:50:26.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.339+0000 7f8fdc2e0700 1 -- 172.21.15.144:0/1609193353 wait complete. 2024-09-10T11:50:26.344 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:50:26.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.445+0000 7f0e75659700 1 -- 172.21.15.144:0/1411338164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e700ffc60 msgr2=0x7f0e70103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:26.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.445+0000 7f0e75659700 1 --2- 172.21.15.144:0/1411338164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e700ffc60 0x7f0e70103c30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009910 tx=0x7f0e58009c20 comp rx=0 tx=0).stop 2024-09-10T11:50:26.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.445+0000 7f0e75659700 1 -- 172.21.15.144:0/1411338164 shutdown_connections 2024-09-10T11:50:26.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.445+0000 7f0e75659700 1 --2- 172.21.15.144:0/1411338164 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e700ffc60 0x7f0e70103c30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.445+0000 7f0e75659700 1 --2- 172.21.15.144:0/1411338164 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e700ff2c0 0x7f0e700ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.445+0000 7f0e75659700 1 -- 172.21.15.144:0/1411338164 >> 172.21.15.144:0/1411338164 conn(0x7f0e700fab60 msgr2=0x7f0e700fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:26.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.445+0000 7f0e75659700 1 -- 172.21.15.144:0/1411338164 shutdown_connections 2024-09-10T11:50:26.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.445+0000 7f0e75659700 1 -- 172.21.15.144:0/1411338164 wait complete. 2024-09-10T11:50:26.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.446+0000 7f0e75659700 1 Processor -- start 2024-09-10T11:50:26.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.446+0000 7f0e75659700 1 -- start start 2024-09-10T11:50:26.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.446+0000 7f0e75659700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e700ff2c0 0x7f0e70195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:26.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.446+0000 7f0e75659700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e700ffc60 0x7f0e70196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:26.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.446+0000 7f0e75659700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e701968d0 con 0x7f0e700ff2c0 2024-09-10T11:50:26.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.446+0000 7f0e75659700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e70196a10 con 0x7f0e700ffc60 2024-09-10T11:50:26.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.446+0000 7f0e6effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e700ff2c0 0x7f0e70195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:26.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.447+0000 7f0e6effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e700ff2c0 0x7f0e70195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51602/0 (socket says 172.21.15.144:51602) 2024-09-10T11:50:26.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.447+0000 7f0e6effd700 1 -- 172.21.15.144:0/3644568503 learned_addr learned my addr 172.21.15.144:0/3644568503 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:50:26.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.447+0000 7f0e6e7fc700 1 --2- 172.21.15.144:0/3644568503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e700ffc60 0x7f0e70196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:26.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.447+0000 7f0e6effd700 1 -- 172.21.15.144:0/3644568503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e700ffc60 msgr2=0x7f0e70196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:26.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.447+0000 7f0e6effd700 1 --2- 172.21.15.144:0/3644568503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e700ffc60 0x7f0e70196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.447+0000 7f0e6effd700 1 -- 172.21.15.144:0/3644568503 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e580095f0 con 0x7f0e700ff2c0 2024-09-10T11:50:26.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.447+0000 7f0e6e7fc700 1 --2- 172.21.15.144:0/3644568503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e700ffc60 0x7f0e70196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:50:26.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.447+0000 7f0e6effd700 1 --2- 172.21.15.144:0/3644568503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e700ff2c0 0x7f0e70195d40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000cfd0 tx=0x7f0e6000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:26.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.447+0000 7f0e67fff700 1 -- 172.21.15.144:0/3644568503 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e6000ad70 con 0x7f0e700ff2c0 2024-09-10T11:50:26.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.447+0000 7f0e67fff700 1 -- 172.21.15.144:0/3644568503 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e600128e0 con 0x7f0e700ff2c0 2024-09-10T11:50:26.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.447+0000 7f0e75659700 1 -- 172.21.15.144:0/3644568503 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7019a860 con 0x7f0e700ff2c0 2024-09-10T11:50:26.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.447+0000 7f0e67fff700 1 -- 172.21.15.144:0/3644568503 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e60011880 con 0x7f0e700ff2c0 2024-09-10T11:50:26.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.447+0000 7f0e75659700 1 -- 172.21.15.144:0/3644568503 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7019adb0 con 0x7f0e700ff2c0 2024-09-10T11:50:26.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.448+0000 7f0e75659700 1 -- 172.21.15.144:0/3644568503 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7004ea50 con 0x7f0e700ff2c0 2024-09-10T11:50:26.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.449+0000 7f0e67fff700 1 -- 172.21.15.144:0/3644568503 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0e60012420 con 0x7f0e700ff2c0 2024-09-10T11:50:26.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.449+0000 7f0e67fff700 1 --2- 172.21.15.144:0/3644568503 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0e5c06e200 0x7f0e5c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:26.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.449+0000 7f0e67fff700 1 -- 172.21.15.144:0/3644568503 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0e6008e150 con 0x7f0e700ff2c0 2024-09-10T11:50:26.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.449+0000 7f0e6e7fc700 1 --2- 172.21.15.144:0/3644568503 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0e5c06e200 0x7f0e5c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:26.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.450+0000 7f0e6e7fc700 1 --2- 172.21.15.144:0/3644568503 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0e5c06e200 0x7f0e5c0706b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0e70197360 tx=0x7f0e5801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:26.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.454+0000 7f0e67fff700 1 -- 172.21.15.144:0/3644568503 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0e600575b0 con 0x7f0e700ff2c0 2024-09-10T11:50:26.684 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.683+0000 7f0e75659700 1 -- 172.21.15.144:0/3644568503 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e7019b360 con 0x7f0e5c06e200 2024-09-10T11:50:26.685 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.684+0000 7f0e67fff700 1 -- 172.21.15.144:0/3644568503 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f0e7019b360 con 0x7f0e5c06e200 2024-09-10T11:50:26.685 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:50:26.685 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:50:26.685 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:50:26.685 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:50:26.685 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:50:26.685 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:50:26.685 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:50:26.685 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:50:26.686 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:50:26.686 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:50:26.686 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:50:26.686 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:50:26.686 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:50:26.686 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:50:26.687 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.686+0000 7f0e75659700 1 -- 172.21.15.144:0/3644568503 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0e5c06e200 msgr2=0x7f0e5c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:26.687 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.686+0000 7f0e75659700 1 --2- 172.21.15.144:0/3644568503 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0e5c06e200 0x7f0e5c0706b0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f0e70197360 tx=0x7f0e5801a040 comp rx=0 tx=0).stop 2024-09-10T11:50:26.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.686+0000 7f0e75659700 1 -- 172.21.15.144:0/3644568503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e700ff2c0 msgr2=0x7f0e70195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:26.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.686+0000 7f0e75659700 1 --2- 172.21.15.144:0/3644568503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e700ff2c0 0x7f0e70195d40 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f0e6000cfd0 tx=0x7f0e6000a340 comp rx=0 tx=0).stop 2024-09-10T11:50:26.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.686+0000 7f0e75659700 1 -- 172.21.15.144:0/3644568503 shutdown_connections 2024-09-10T11:50:26.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.686+0000 7f0e75659700 1 --2- 172.21.15.144:0/3644568503 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0e5c06e200 0x7f0e5c0706b0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.686+0000 7f0e75659700 1 --2- 172.21.15.144:0/3644568503 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0e700ff2c0 0x7f0e70195d40 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.686+0000 7f0e75659700 1 --2- 172.21.15.144:0/3644568503 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0e700ffc60 0x7f0e70196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.686+0000 7f0e75659700 1 -- 172.21.15.144:0/3644568503 >> 172.21.15.144:0/3644568503 conn(0x7f0e700fab60 msgr2=0x7f0e70068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:26.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.686+0000 7f0e75659700 1 -- 172.21.15.144:0/3644568503 shutdown_connections 2024-09-10T11:50:26.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.687+0000 7f0e75659700 1 -- 172.21.15.144:0/3644568503 wait complete. 2024-09-10T11:50:26.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.791+0000 7f0508713700 1 -- 172.21.15.144:0/3386469590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05000ffcb0 msgr2=0x7f0500103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:26.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.791+0000 7f0508713700 1 --2- 172.21.15.144:0/3386469590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05000ffcb0 0x7f0500103c80 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f04f4009910 tx=0x7f04f4009c20 comp rx=0 tx=0).stop 2024-09-10T11:50:26.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.791+0000 7f0508713700 1 -- 172.21.15.144:0/3386469590 shutdown_connections 2024-09-10T11:50:26.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.791+0000 7f0508713700 1 --2- 172.21.15.144:0/3386469590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05000ffcb0 0x7f0500103c80 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.791+0000 7f0508713700 1 --2- 172.21.15.144:0/3386469590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05000ff310 0x7f05000ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.791+0000 7f0508713700 1 -- 172.21.15.144:0/3386469590 >> 172.21.15.144:0/3386469590 conn(0x7f05000fabb0 msgr2=0x7f05000fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:26.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.791+0000 7f0508713700 1 -- 172.21.15.144:0/3386469590 shutdown_connections 2024-09-10T11:50:26.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.792+0000 7f0508713700 1 -- 172.21.15.144:0/3386469590 wait complete. 2024-09-10T11:50:26.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.792+0000 7f0508713700 1 Processor -- start 2024-09-10T11:50:26.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.792+0000 7f0508713700 1 -- start start 2024-09-10T11:50:26.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.792+0000 7f0508713700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05000ff310 0x7f0500195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:26.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.792+0000 7f0508713700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05000ffcb0 0x7f05001962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:26.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.792+0000 7f0508713700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05001968f0 con 0x7f05000ffcb0 2024-09-10T11:50:26.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.792+0000 7f0508713700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0500196a30 con 0x7f05000ff310 2024-09-10T11:50:26.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.792+0000 7f0505cae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05000ffcb0 0x7f05001962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:26.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.793+0000 7f0505cae700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05000ffcb0 0x7f05001962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51608/0 (socket says 172.21.15.144:51608) 2024-09-10T11:50:26.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.793+0000 7f0505cae700 1 -- 172.21.15.144:0/3507293454 learned_addr learned my addr 172.21.15.144:0/3507293454 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:50:26.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.793+0000 7f05064af700 1 --2- 172.21.15.144:0/3507293454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05000ff310 0x7f0500195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:26.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.793+0000 7f0505cae700 1 -- 172.21.15.144:0/3507293454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05000ff310 msgr2=0x7f0500195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:26.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.793+0000 7f0505cae700 1 --2- 172.21.15.144:0/3507293454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05000ff310 0x7f0500195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:26.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.793+0000 7f0505cae700 1 -- 172.21.15.144:0/3507293454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04f40095f0 con 0x7f05000ffcb0 2024-09-10T11:50:26.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.793+0000 7f0505cae700 1 --2- 172.21.15.144:0/3507293454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05000ffcb0 0x7f05001962a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f04f400bda0 tx=0x7f04f400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:26.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.793+0000 7f04ff7fe700 1 -- 172.21.15.144:0/3507293454 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04f401d070 con 0x7f05000ffcb0 2024-09-10T11:50:26.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.793+0000 7f04ff7fe700 1 -- 172.21.15.144:0/3507293454 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f04f400f460 con 0x7f05000ffcb0 2024-09-10T11:50:26.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.793+0000 7f04ff7fe700 1 -- 172.21.15.144:0/3507293454 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04f4017630 con 0x7f05000ffcb0 2024-09-10T11:50:26.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.793+0000 7f0508713700 1 -- 172.21.15.144:0/3507293454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f050019a820 con 0x7f05000ffcb0 2024-09-10T11:50:26.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.793+0000 7f0508713700 1 -- 172.21.15.144:0/3507293454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f050019ad10 con 0x7f05000ffcb0 2024-09-10T11:50:26.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.794+0000 7f0508713700 1 -- 172.21.15.144:0/3507293454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f050004ea50 con 0x7f05000ffcb0 2024-09-10T11:50:26.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.795+0000 7f04ff7fe700 1 -- 172.21.15.144:0/3507293454 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f04f400fab0 con 0x7f05000ffcb0 2024-09-10T11:50:26.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.795+0000 7f04ff7fe700 1 --2- 172.21.15.144:0/3507293454 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f04e806e200 0x7f04e80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:26.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.795+0000 7f04ff7fe700 1 -- 172.21.15.144:0/3507293454 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f04f408f4a0 con 0x7f05000ffcb0 2024-09-10T11:50:26.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.795+0000 7f05064af700 1 --2- 172.21.15.144:0/3507293454 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f04e806e200 0x7f04e80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:26.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.796+0000 7f05064af700 1 --2- 172.21.15.144:0/3507293454 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f04e806e200 0x7f04e80706b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f04f000f440 tx=0x7f04f00079e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:26.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:26.800+0000 7f04ff7fe700 1 -- 172.21.15.144:0/3507293454 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f04f4058900 con 0x7f05000ffcb0 2024-09-10T11:50:27.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:27.088+0000 7f0508713700 1 -- 172.21.15.144:0/3507293454 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f050019b100 con 0x7f05000ffcb0 2024-09-10T11:50:27.090 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:26 smithi144 ceph-mon[87436]: from='client.35044 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:50:27.090 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:26 smithi144 ceph-mon[87436]: from='client.35048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:50:27.090 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:26 smithi144 ceph-mon[87436]: from='client.35052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:50:27.090 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:26 smithi144 ceph-mon[87436]: pgmap v553: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:27.090 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:26 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3868864224' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:50:27.090 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:26 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1609193353' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:50:27.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:27.089+0000 7f04ff7fe700 1 -- 172.21.15.144:0/3507293454 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f04f405bfc0 con 0x7f05000ffcb0 2024-09-10T11:50:27.091 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:50:27.091 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:50:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:50:27.091 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:50:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:50:27.091 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:50:27.091 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:50:27.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:27.091+0000 7f0508713700 1 -- 172.21.15.144:0/3507293454 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f04e806e200 msgr2=0x7f04e80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:27.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:27.091+0000 7f0508713700 1 --2- 172.21.15.144:0/3507293454 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f04e806e200 0x7f04e80706b0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f04f000f440 tx=0x7f04f00079e0 comp rx=0 tx=0).stop 2024-09-10T11:50:27.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:27.091+0000 7f0508713700 1 -- 172.21.15.144:0/3507293454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05000ffcb0 msgr2=0x7f05001962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:27.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:27.091+0000 7f0508713700 1 --2- 172.21.15.144:0/3507293454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05000ffcb0 0x7f05001962a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f04f400bda0 tx=0x7f04f400bdd0 comp rx=0 tx=0).stop 2024-09-10T11:50:27.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:27.092+0000 7f0508713700 1 -- 172.21.15.144:0/3507293454 shutdown_connections 2024-09-10T11:50:27.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:27.092+0000 7f0508713700 1 --2- 172.21.15.144:0/3507293454 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f04e806e200 0x7f04e80706b0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:27.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:27.092+0000 7f0508713700 1 --2- 172.21.15.144:0/3507293454 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f05000ff310 0x7f0500195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:27.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:27.092+0000 7f0508713700 1 --2- 172.21.15.144:0/3507293454 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f05000ffcb0 0x7f05001962a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:27.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:27.092+0000 7f0508713700 1 -- 172.21.15.144:0/3507293454 >> 172.21.15.144:0/3507293454 conn(0x7f05000fabb0 msgr2=0x7f0500068e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:27.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:27.092+0000 7f0508713700 1 -- 172.21.15.144:0/3507293454 shutdown_connections 2024-09-10T11:50:27.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:27.092+0000 7f0508713700 1 -- 172.21.15.144:0/3507293454 wait complete. 2024-09-10T11:50:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:26 smithi178 ceph-mon[65940]: from='client.35044 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:50:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:26 smithi178 ceph-mon[65940]: from='client.35048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:50:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:26 smithi178 ceph-mon[65940]: from='client.35052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:50:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:26 smithi178 ceph-mon[65940]: pgmap v553: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:26 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3868864224' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:50:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:26 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1609193353' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:50:28.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:27 smithi144 ceph-mon[87436]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:50:28.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:27 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3507293454' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:50:28.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:27 smithi178 ceph-mon[65940]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:50:28.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:27 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3507293454' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:50:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:28 smithi144 ceph-mon[87436]: pgmap v554: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:50:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:28 smithi178 ceph-mon[65940]: pgmap v554: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:50:30.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:29 smithi144 ceph-mon[87436]: pgmap v555: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:30.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:29 smithi178 ceph-mon[65940]: pgmap v555: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:32 smithi144 ceph-mon[87436]: pgmap v556: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:32 smithi178 ceph-mon[65940]: pgmap v556: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:34 smithi178 ceph-mon[65940]: pgmap v557: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:34 smithi144 ceph-mon[87436]: pgmap v557: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:36.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:36 smithi144 ceph-mon[87436]: pgmap v558: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:36 smithi178 ceph-mon[65940]: pgmap v558: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:39.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:38 smithi144 ceph-mon[87436]: pgmap v559: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:39.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:38 smithi178 ceph-mon[65940]: pgmap v559: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:41.188 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:40 smithi144 ceph-mon[87436]: pgmap v560: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:41.385 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:40 smithi178 ceph-mon[65940]: pgmap v560: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:42.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:41 smithi144 ceph-mon[87436]: pgmap v561: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:41 smithi178 ceph-mon[65940]: pgmap v561: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:50:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:50:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:44 smithi144 ceph-mon[87436]: pgmap v562: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:44 smithi178 ceph-mon[65940]: pgmap v562: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:46 smithi178 ceph-mon[65940]: pgmap v563: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:47.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:46 smithi144 ceph-mon[87436]: pgmap v563: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:49.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:48 smithi144 ceph-mon[87436]: pgmap v564: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:48 smithi178 ceph-mon[65940]: pgmap v564: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:50 smithi144 ceph-mon[87436]: pgmap v565: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:50 smithi178 ceph-mon[65940]: pgmap v565: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:53.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:52 smithi144 ceph-mon[87436]: pgmap v566: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:52 smithi178 ceph-mon[65940]: pgmap v566: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:54.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:53 smithi144 ceph-mon[87436]: pgmap v567: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:53 smithi178 ceph-mon[65940]: pgmap v567: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:56 smithi144 ceph-mon[87436]: pgmap v568: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:57.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.208+0000 7ff758577700 1 -- 172.21.15.144:0/2264216254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7500ffb70 msgr2=0x7ff75010a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.208+0000 7ff758577700 1 --2- 172.21.15.144:0/2264216254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7500ffb70 0x7ff75010a5e0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7ff744009910 tx=0x7ff744009c20 comp rx=0 tx=0).stop 2024-09-10T11:50:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.209+0000 7ff758577700 1 -- 172.21.15.144:0/2264216254 shutdown_connections 2024-09-10T11:50:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.209+0000 7ff758577700 1 --2- 172.21.15.144:0/2264216254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7500ffb70 0x7ff75010a5e0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.209+0000 7ff758577700 1 --2- 172.21.15.144:0/2264216254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7500ff260 0x7ff7500ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.209+0000 7ff758577700 1 -- 172.21.15.144:0/2264216254 >> 172.21.15.144:0/2264216254 conn(0x7ff7500747e0 msgr2=0x7ff750074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.209+0000 7ff758577700 1 -- 172.21.15.144:0/2264216254 shutdown_connections 2024-09-10T11:50:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.209+0000 7ff758577700 1 -- 172.21.15.144:0/2264216254 wait complete. 2024-09-10T11:50:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.209+0000 7ff758577700 1 Processor -- start 2024-09-10T11:50:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.209+0000 7ff758577700 1 -- start start 2024-09-10T11:50:57.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.209+0000 7ff758577700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7500ff260 0x7ff750100440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.209+0000 7ff758577700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7500ffb70 0x7ff750100980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff758577700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff750100ec0 con 0x7ff7500ffb70 2024-09-10T11:50:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff758577700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff750101000 con 0x7ff7500ff260 2024-09-10T11:50:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff755b12700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7500ffb70 0x7ff750100980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff755b12700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7500ffb70 0x7ff750100980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34582/0 (socket says 172.21.15.144:34582) 2024-09-10T11:50:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff755b12700 1 -- 172.21.15.144:0/3364780398 learned_addr learned my addr 172.21.15.144:0/3364780398 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:50:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff756313700 1 --2- 172.21.15.144:0/3364780398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7500ff260 0x7ff750100440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff755b12700 1 -- 172.21.15.144:0/3364780398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7500ff260 msgr2=0x7ff750100440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff755b12700 1 --2- 172.21.15.144:0/3364780398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7500ff260 0x7ff750100440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff755b12700 1 -- 172.21.15.144:0/3364780398 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7440095f0 con 0x7ff7500ffb70 2024-09-10T11:50:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff756313700 1 --2- 172.21.15.144:0/3364780398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7500ff260 0x7ff750100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:50:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff755b12700 1 --2- 172.21.15.144:0/3364780398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7500ffb70 0x7ff750100980 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff744006010 tx=0x7ff74400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff7437fe700 1 -- 172.21.15.144:0/3364780398 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74401d070 con 0x7ff7500ffb70 2024-09-10T11:50:57.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff7437fe700 1 -- 172.21.15.144:0/3364780398 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff744004fb0 con 0x7ff7500ffb70 2024-09-10T11:50:57.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff758577700 1 -- 172.21.15.144:0/3364780398 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff750101280 con 0x7ff7500ffb70 2024-09-10T11:50:57.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff7437fe700 1 -- 172.21.15.144:0/3364780398 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff74400f690 con 0x7ff7500ffb70 2024-09-10T11:50:57.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.210+0000 7ff758577700 1 -- 172.21.15.144:0/3364780398 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7501a2350 con 0x7ff7500ffb70 2024-09-10T11:50:57.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.211+0000 7ff758577700 1 -- 172.21.15.144:0/3364780398 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff750107ce0 con 0x7ff7500ffb70 2024-09-10T11:50:57.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.212+0000 7ff7437fe700 1 -- 172.21.15.144:0/3364780398 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff7440228a0 con 0x7ff7500ffb70 2024-09-10T11:50:57.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.212+0000 7ff7437fe700 1 --2- 172.21.15.144:0/3364780398 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff73c06e200 0x7ff73c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:57.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.212+0000 7ff7437fe700 1 -- 172.21.15.144:0/3364780398 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff74408f8f0 con 0x7ff7500ffb70 2024-09-10T11:50:57.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.212+0000 7ff756313700 1 --2- 172.21.15.144:0/3364780398 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff73c06e200 0x7ff73c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:57.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.213+0000 7ff756313700 1 --2- 172.21.15.144:0/3364780398 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff73c06e200 0x7ff73c0706b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0099d0 tx=0x7ff74c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:57.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.217+0000 7ff7437fe700 1 -- 172.21.15.144:0/3364780398 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff744058cd0 con 0x7ff7500ffb70 2024-09-10T11:50:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:56 smithi178 ceph-mon[65940]: pgmap v568: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:57.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.439+0000 7ff758577700 1 -- 172.21.15.144:0/3364780398 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff750061190 con 0x7ff73c06e200 2024-09-10T11:50:57.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.441+0000 7ff7437fe700 1 -- 172.21.15.144:0/3364780398 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7ff750061190 con 0x7ff73c06e200 2024-09-10T11:50:57.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.443+0000 7ff758577700 1 -- 172.21.15.144:0/3364780398 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff73c06e200 msgr2=0x7ff73c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:57.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.443+0000 7ff758577700 1 --2- 172.21.15.144:0/3364780398 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff73c06e200 0x7ff73c0706b0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ff74c0099d0 tx=0x7ff74c008040 comp rx=0 tx=0).stop 2024-09-10T11:50:57.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.443+0000 7ff758577700 1 -- 172.21.15.144:0/3364780398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7500ffb70 msgr2=0x7ff750100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:57.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.443+0000 7ff758577700 1 --2- 172.21.15.144:0/3364780398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7500ffb70 0x7ff750100980 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7ff744006010 tx=0x7ff74400be20 comp rx=0 tx=0).stop 2024-09-10T11:50:57.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.443+0000 7ff758577700 1 -- 172.21.15.144:0/3364780398 shutdown_connections 2024-09-10T11:50:57.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.443+0000 7ff758577700 1 --2- 172.21.15.144:0/3364780398 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff73c06e200 0x7ff73c0706b0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:57.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.443+0000 7ff758577700 1 --2- 172.21.15.144:0/3364780398 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff7500ff260 0x7ff750100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:57.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.443+0000 7ff758577700 1 --2- 172.21.15.144:0/3364780398 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff7500ffb70 0x7ff750100980 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:57.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.443+0000 7ff758577700 1 -- 172.21.15.144:0/3364780398 >> 172.21.15.144:0/3364780398 conn(0x7ff7500747e0 msgr2=0x7ff750104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:57.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.443+0000 7ff758577700 1 -- 172.21.15.144:0/3364780398 shutdown_connections 2024-09-10T11:50:57.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.443+0000 7ff758577700 1 -- 172.21.15.144:0/3364780398 wait complete. 2024-09-10T11:50:57.455 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:50:57.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.548+0000 7f3b9657d700 1 -- 172.21.15.144:0/1292712341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b900ffba0 msgr2=0x7f3b9010a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:57.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.548+0000 7f3b9657d700 1 --2- 172.21.15.144:0/1292712341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b900ffba0 0x7f3b9010a610 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f3b80009910 tx=0x7f3b80009c20 comp rx=0 tx=0).stop 2024-09-10T11:50:57.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.548+0000 7f3b9657d700 1 -- 172.21.15.144:0/1292712341 shutdown_connections 2024-09-10T11:50:57.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.548+0000 7f3b9657d700 1 --2- 172.21.15.144:0/1292712341 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b900ffba0 0x7f3b9010a610 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:57.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.548+0000 7f3b9657d700 1 --2- 172.21.15.144:0/1292712341 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b900ff290 0x7f3b900ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:57.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.548+0000 7f3b9657d700 1 -- 172.21.15.144:0/1292712341 >> 172.21.15.144:0/1292712341 conn(0x7f3b90074850 msgr2=0x7f3b90074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:57.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.548+0000 7f3b9657d700 1 -- 172.21.15.144:0/1292712341 shutdown_connections 2024-09-10T11:50:57.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.548+0000 7f3b9657d700 1 -- 172.21.15.144:0/1292712341 wait complete. 2024-09-10T11:50:57.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.549+0000 7f3b9657d700 1 Processor -- start 2024-09-10T11:50:57.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.549+0000 7f3b9657d700 1 -- start start 2024-09-10T11:50:57.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.550+0000 7f3b9657d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b900ff290 0x7f3b90195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:57.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.550+0000 7f3b9657d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b900ffba0 0x7f3b901962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:57.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.550+0000 7f3b9657d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b90196980 con 0x7f3b900ffba0 2024-09-10T11:50:57.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.550+0000 7f3b9657d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9019a710 con 0x7f3b900ff290 2024-09-10T11:50:57.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.550+0000 7f3b8f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b900ffba0 0x7f3b901962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:57.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.550+0000 7f3b8f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b900ffba0 0x7f3b901962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34606/0 (socket says 172.21.15.144:34606) 2024-09-10T11:50:57.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.550+0000 7f3b8f7fe700 1 -- 172.21.15.144:0/2895508370 learned_addr learned my addr 172.21.15.144:0/2895508370 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:50:57.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.550+0000 7f3b8ffff700 1 --2- 172.21.15.144:0/2895508370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b900ff290 0x7f3b90195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:57.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.550+0000 7f3b8f7fe700 1 -- 172.21.15.144:0/2895508370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b900ff290 msgr2=0x7f3b90195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:57.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.550+0000 7f3b8f7fe700 1 --2- 172.21.15.144:0/2895508370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b900ff290 0x7f3b90195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:57.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.550+0000 7f3b8f7fe700 1 -- 172.21.15.144:0/2895508370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b800095f0 con 0x7f3b900ffba0 2024-09-10T11:50:57.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.550+0000 7f3b8f7fe700 1 --2- 172.21.15.144:0/2895508370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b900ffba0 0x7f3b901962a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3b80006010 tx=0x7f3b8000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:57.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.550+0000 7f3b8d7fa700 1 -- 172.21.15.144:0/2895508370 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b8001d070 con 0x7f3b900ffba0 2024-09-10T11:50:57.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.550+0000 7f3b8d7fa700 1 -- 172.21.15.144:0/2895508370 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3b80005dc0 con 0x7f3b900ffba0 2024-09-10T11:50:57.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.550+0000 7f3b9657d700 1 -- 172.21.15.144:0/2895508370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b9019a990 con 0x7f3b900ffba0 2024-09-10T11:50:57.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.551+0000 7f3b8d7fa700 1 -- 172.21.15.144:0/2895508370 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b80017400 con 0x7f3b900ffba0 2024-09-10T11:50:57.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.551+0000 7f3b9657d700 1 -- 172.21.15.144:0/2895508370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9019ae80 con 0x7f3b900ffba0 2024-09-10T11:50:57.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.551+0000 7f3b9657d700 1 -- 172.21.15.144:0/2895508370 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b90107d10 con 0x7f3b900ffba0 2024-09-10T11:50:57.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.552+0000 7f3b8d7fa700 1 -- 172.21.15.144:0/2895508370 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3b80003a60 con 0x7f3b900ffba0 2024-09-10T11:50:57.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.553+0000 7f3b8d7fa700 1 --2- 172.21.15.144:0/2895508370 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3b7c06e200 0x7f3b7c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:57.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.553+0000 7f3b8d7fa700 1 -- 172.21.15.144:0/2895508370 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3b8008f5a0 con 0x7f3b900ffba0 2024-09-10T11:50:57.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.553+0000 7f3b8ffff700 1 --2- 172.21.15.144:0/2895508370 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3b7c06e200 0x7f3b7c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:57.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.553+0000 7f3b8ffff700 1 --2- 172.21.15.144:0/2895508370 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3b7c06e200 0x7f3b7c0706b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3b9006a5c0 tx=0x7f3b7800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:57.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.557+0000 7f3b8d7fa700 1 -- 172.21.15.144:0/2895508370 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3b800588e0 con 0x7f3b900ffba0 2024-09-10T11:50:57.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.776+0000 7f3b9657d700 1 -- 172.21.15.144:0/2895508370 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3b9002cca0 con 0x7f3b7c06e200 2024-09-10T11:50:57.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.777+0000 7f3b8d7fa700 1 -- 172.21.15.144:0/2895508370 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3b9002cca0 con 0x7f3b7c06e200 2024-09-10T11:50:57.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.779+0000 7f3b9657d700 1 -- 172.21.15.144:0/2895508370 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3b7c06e200 msgr2=0x7f3b7c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:57.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.779+0000 7f3b9657d700 1 --2- 172.21.15.144:0/2895508370 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3b7c06e200 0x7f3b7c0706b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3b9006a5c0 tx=0x7f3b7800b410 comp rx=0 tx=0).stop 2024-09-10T11:50:57.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.779+0000 7f3b9657d700 1 -- 172.21.15.144:0/2895508370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b900ffba0 msgr2=0x7f3b901962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:57.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.779+0000 7f3b9657d700 1 --2- 172.21.15.144:0/2895508370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b900ffba0 0x7f3b901962a0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f3b80006010 tx=0x7f3b8000bd70 comp rx=0 tx=0).stop 2024-09-10T11:50:57.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.780+0000 7f3b9657d700 1 -- 172.21.15.144:0/2895508370 shutdown_connections 2024-09-10T11:50:57.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.780+0000 7f3b9657d700 1 --2- 172.21.15.144:0/2895508370 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3b7c06e200 0x7f3b7c0706b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:57.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.780+0000 7f3b9657d700 1 --2- 172.21.15.144:0/2895508370 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3b900ff290 0x7f3b90195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:57.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.780+0000 7f3b9657d700 1 --2- 172.21.15.144:0/2895508370 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3b900ffba0 0x7f3b901962a0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:57.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.780+0000 7f3b9657d700 1 -- 172.21.15.144:0/2895508370 >> 172.21.15.144:0/2895508370 conn(0x7f3b90074850 msgr2=0x7f3b90104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:57.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.780+0000 7f3b9657d700 1 -- 172.21.15.144:0/2895508370 shutdown_connections 2024-09-10T11:50:57.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.780+0000 7f3b9657d700 1 -- 172.21.15.144:0/2895508370 wait complete. 2024-09-10T11:50:57.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.885+0000 7f3eae401700 1 -- 172.21.15.144:0/118136128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea80ff390 msgr2=0x7f3ea80ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:57.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.885+0000 7f3eae401700 1 --2- 172.21.15.144:0/118136128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea80ff390 0x7f3ea80ff760 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009910 tx=0x7f3e98009c20 comp rx=0 tx=0).stop 2024-09-10T11:50:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.885+0000 7f3eae401700 1 -- 172.21.15.144:0/118136128 shutdown_connections 2024-09-10T11:50:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.885+0000 7f3eae401700 1 --2- 172.21.15.144:0/118136128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea80ffd30 0x7f3ea8103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.885+0000 7f3eae401700 1 --2- 172.21.15.144:0/118136128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea80ff390 0x7f3ea80ff760 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.885+0000 7f3eae401700 1 -- 172.21.15.144:0/118136128 >> 172.21.15.144:0/118136128 conn(0x7f3ea8074f50 msgr2=0x7f3ea8075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.885+0000 7f3eae401700 1 -- 172.21.15.144:0/118136128 shutdown_connections 2024-09-10T11:50:57.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.885+0000 7f3eae401700 1 -- 172.21.15.144:0/118136128 wait complete. 2024-09-10T11:50:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.886+0000 7f3eae401700 1 Processor -- start 2024-09-10T11:50:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.886+0000 7f3eae401700 1 -- start start 2024-09-10T11:50:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.887+0000 7f3eae401700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea80ff390 0x7f3ea8196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.887+0000 7f3eae401700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea80ffd30 0x7f3ea81971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:57.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.887+0000 7f3eae401700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea81977f0 con 0x7f3ea80ff390 2024-09-10T11:50:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.887+0000 7f3eae401700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea8197960 con 0x7f3ea80ffd30 2024-09-10T11:50:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.887+0000 7f3ea7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea80ff390 0x7f3ea8196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.887+0000 7f3ea7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea80ff390 0x7f3ea8196c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34620/0 (socket says 172.21.15.144:34620) 2024-09-10T11:50:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.887+0000 7f3ea7fff700 1 -- 172.21.15.144:0/1034741757 learned_addr learned my addr 172.21.15.144:0/1034741757 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:50:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.887+0000 7f3ea77fe700 1 --2- 172.21.15.144:0/1034741757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea80ffd30 0x7f3ea81971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.887+0000 7f3ea7fff700 1 -- 172.21.15.144:0/1034741757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea80ffd30 msgr2=0x7f3ea81971d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.887+0000 7f3ea7fff700 1 --2- 172.21.15.144:0/1034741757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea80ffd30 0x7f3ea81971d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:57.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.887+0000 7f3ea7fff700 1 -- 172.21.15.144:0/1034741757 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e980095f0 con 0x7f3ea80ff390 2024-09-10T11:50:57.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.887+0000 7f3ea7fff700 1 --2- 172.21.15.144:0/1034741757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea80ff390 0x7f3ea8196c90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3e98006010 tx=0x7f3e9800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:57.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.888+0000 7f3ea57fa700 1 -- 172.21.15.144:0/1034741757 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e9801d070 con 0x7f3ea80ff390 2024-09-10T11:50:57.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.888+0000 7f3ea57fa700 1 -- 172.21.15.144:0/1034741757 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3e9800f460 con 0x7f3ea80ff390 2024-09-10T11:50:57.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.888+0000 7f3ea57fa700 1 -- 172.21.15.144:0/1034741757 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e98017610 con 0x7f3ea80ff390 2024-09-10T11:50:57.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.888+0000 7f3eae401700 1 -- 172.21.15.144:0/1034741757 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea819fd00 con 0x7f3ea80ff390 2024-09-10T11:50:57.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.888+0000 7f3eae401700 1 -- 172.21.15.144:0/1034741757 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea81a0170 con 0x7f3ea80ff390 2024-09-10T11:50:57.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.889+0000 7f3eae401700 1 -- 172.21.15.144:0/1034741757 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea8107650 con 0x7f3ea80ff390 2024-09-10T11:50:57.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.889+0000 7f3ea57fa700 1 -- 172.21.15.144:0/1034741757 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3e98003a60 con 0x7f3ea80ff390 2024-09-10T11:50:57.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.890+0000 7f3ea57fa700 1 --2- 172.21.15.144:0/1034741757 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3e9406e200 0x7f3e940706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:57.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.890+0000 7f3ea57fa700 1 -- 172.21.15.144:0/1034741757 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3e9808f530 con 0x7f3ea80ff390 2024-09-10T11:50:57.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.890+0000 7f3ea77fe700 1 --2- 172.21.15.144:0/1034741757 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3e9406e200 0x7f3e940706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:57.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.890+0000 7f3ea77fe700 1 --2- 172.21.15.144:0/1034741757 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3e9406e200 0x7f3e940706b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8068ca0 tx=0x7f3e90009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:57.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:57.894+0000 7f3ea57fa700 1 -- 172.21.15.144:0/1034741757 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3e98058990 con 0x7f3ea80ff390 2024-09-10T11:50:58.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.103+0000 7f3eae401700 1 -- 172.21.15.144:0/1034741757 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ea8061190 con 0x7f3e9406e200 2024-09-10T11:50:58.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.112+0000 7f3ea57fa700 1 -- 172.21.15.144:0/1034741757 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f3ea8061190 con 0x7f3e9406e200 2024-09-10T11:50:58.113 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:50:58.114 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (19m) 12m ago 29m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:50:58.114 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (29m) 12m ago 29m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:50:58.114 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (29m) 12m ago 28m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:50:58.114 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (16m) 12m ago 29m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:50:58.114 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (16m) 12m ago 28m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:50:58.114 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (18m) 12m ago 29m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:50:58.114 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (13m) 12m ago 24m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:50:58.114 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (12m) 12m ago 24m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:50:58.114 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (24m) 12m ago 24m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:50:58.114 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (12m) 12m ago 24m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:50:58.114 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (20m) 12m ago 31m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:50:58.114 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (20m) 12m ago 28m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:50:58.114 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (17m) 12m ago 31m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:50:58.114 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (17m) 12m ago 28m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:50:58.115 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (19m) 12m ago 29m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:50:58.115 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (19m) 12m ago 28m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:50:58.115 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (16m) 12m ago 27m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:50:58.115 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (15m) 12m ago 27m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:50:58.115 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (15m) 12m ago 27m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:50:58.115 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (14m) 12m ago 26m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:50:58.115 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (14m) 12m ago 26m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:50:58.115 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (13m) 12m ago 25m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:50:58.115 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (19m) 12m ago 29m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:50:58.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.115+0000 7f3eae401700 1 -- 172.21.15.144:0/1034741757 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3e9406e200 msgr2=0x7f3e940706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:58.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.115+0000 7f3eae401700 1 --2- 172.21.15.144:0/1034741757 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3e9406e200 0x7f3e940706b0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3ea8068ca0 tx=0x7f3e90009190 comp rx=0 tx=0).stop 2024-09-10T11:50:58.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.115+0000 7f3eae401700 1 -- 172.21.15.144:0/1034741757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea80ff390 msgr2=0x7f3ea8196c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:58.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.115+0000 7f3eae401700 1 --2- 172.21.15.144:0/1034741757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea80ff390 0x7f3ea8196c90 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f3e98006010 tx=0x7f3e9800bd70 comp rx=0 tx=0).stop 2024-09-10T11:50:58.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.115+0000 7f3eae401700 1 -- 172.21.15.144:0/1034741757 shutdown_connections 2024-09-10T11:50:58.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.115+0000 7f3eae401700 1 --2- 172.21.15.144:0/1034741757 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3e9406e200 0x7f3e940706b0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.115+0000 7f3eae401700 1 --2- 172.21.15.144:0/1034741757 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3ea80ff390 0x7f3ea8196c90 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.115+0000 7f3eae401700 1 --2- 172.21.15.144:0/1034741757 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3ea80ffd30 0x7f3ea81971d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.115+0000 7f3eae401700 1 -- 172.21.15.144:0/1034741757 >> 172.21.15.144:0/1034741757 conn(0x7f3ea8074f50 msgr2=0x7f3ea80fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:58.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.115+0000 7f3eae401700 1 -- 172.21.15.144:0/1034741757 shutdown_connections 2024-09-10T11:50:58.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.115+0000 7f3eae401700 1 -- 172.21.15.144:0/1034741757 wait complete. 2024-09-10T11:50:58.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.218+0000 7fdf20d48700 1 -- 172.21.15.144:0/1561742593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf1c0ff2d0 msgr2=0x7fdf1c0ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:58.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.218+0000 7fdf20d48700 1 --2- 172.21.15.144:0/1561742593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf1c0ff2d0 0x7fdf1c0ff6a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fdf10009910 tx=0x7fdf10009c20 comp rx=0 tx=0).stop 2024-09-10T11:50:58.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.218+0000 7fdf20d48700 1 -- 172.21.15.144:0/1561742593 shutdown_connections 2024-09-10T11:50:58.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.218+0000 7fdf20d48700 1 --2- 172.21.15.144:0/1561742593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf1c0ffc70 0x7fdf1c103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.218+0000 7fdf20d48700 1 --2- 172.21.15.144:0/1561742593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf1c0ff2d0 0x7fdf1c0ff6a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.218+0000 7fdf20d48700 1 -- 172.21.15.144:0/1561742593 >> 172.21.15.144:0/1561742593 conn(0x7fdf1c0fabb0 msgr2=0x7fdf1c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:58.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.218+0000 7fdf20d48700 1 -- 172.21.15.144:0/1561742593 shutdown_connections 2024-09-10T11:50:58.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.218+0000 7fdf20d48700 1 -- 172.21.15.144:0/1561742593 wait complete. 2024-09-10T11:50:58.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.219+0000 7fdf20d48700 1 Processor -- start 2024-09-10T11:50:58.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.219+0000 7fdf20d48700 1 -- start start 2024-09-10T11:50:58.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.219+0000 7fdf20d48700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf1c0ff2d0 0x7fdf1c196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:58.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.219+0000 7fdf20d48700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf1c0ffc70 0x7fdf1c197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:58.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.219+0000 7fdf20d48700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c1977a0 con 0x7fdf1c0ff2d0 2024-09-10T11:50:58.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.219+0000 7fdf20d48700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c197910 con 0x7fdf1c0ffc70 2024-09-10T11:50:58.221 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.219+0000 7fdf1a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf1c0ff2d0 0x7fdf1c196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:58.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.219+0000 7fdf1a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf1c0ff2d0 0x7fdf1c196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34634/0 (socket says 172.21.15.144:34634) 2024-09-10T11:50:58.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.219+0000 7fdf1a59c700 1 -- 172.21.15.144:0/4211404750 learned_addr learned my addr 172.21.15.144:0/4211404750 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:50:58.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.219+0000 7fdf19d9b700 1 --2- 172.21.15.144:0/4211404750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf1c0ffc70 0x7fdf1c197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:58.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.220+0000 7fdf1a59c700 1 -- 172.21.15.144:0/4211404750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf1c0ffc70 msgr2=0x7fdf1c197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:58.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.220+0000 7fdf1a59c700 1 --2- 172.21.15.144:0/4211404750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf1c0ffc70 0x7fdf1c197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.220+0000 7fdf1a59c700 1 -- 172.21.15.144:0/4211404750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf100095f0 con 0x7fdf1c0ff2d0 2024-09-10T11:50:58.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.220+0000 7fdf1a59c700 1 --2- 172.21.15.144:0/4211404750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf1c0ff2d0 0x7fdf1c196c40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdf10006010 tx=0x7fdf1000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:58.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.220+0000 7fdf0b7fe700 1 -- 172.21.15.144:0/4211404750 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf1001d070 con 0x7fdf1c0ff2d0 2024-09-10T11:50:58.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.220+0000 7fdf0b7fe700 1 -- 172.21.15.144:0/4211404750 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf1000f460 con 0x7fdf1c0ff2d0 2024-09-10T11:50:58.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.220+0000 7fdf20d48700 1 -- 172.21.15.144:0/4211404750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1c19fcb0 con 0x7fdf1c0ff2d0 2024-09-10T11:50:58.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.220+0000 7fdf0b7fe700 1 -- 172.21.15.144:0/4211404750 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf10017610 con 0x7fdf1c0ff2d0 2024-09-10T11:50:58.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.220+0000 7fdf20d48700 1 -- 172.21.15.144:0/4211404750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1c1a0120 con 0x7fdf1c0ff2d0 2024-09-10T11:50:58.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.221+0000 7fdf20d48700 1 -- 172.21.15.144:0/4211404750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1c1075e0 con 0x7fdf1c0ff2d0 2024-09-10T11:50:58.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.222+0000 7fdf0b7fe700 1 -- 172.21.15.144:0/4211404750 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fdf1000f5d0 con 0x7fdf1c0ff2d0 2024-09-10T11:50:58.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.222+0000 7fdf0b7fe700 1 --2- 172.21.15.144:0/4211404750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdf0406e200 0x7fdf040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:58.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.222+0000 7fdf0b7fe700 1 -- 172.21.15.144:0/4211404750 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdf1008f510 con 0x7fdf1c0ff2d0 2024-09-10T11:50:58.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.222+0000 7fdf19d9b700 1 --2- 172.21.15.144:0/4211404750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdf0406e200 0x7fdf040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:58.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.223+0000 7fdf19d9b700 1 --2- 172.21.15.144:0/4211404750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdf0406e200 0x7fdf040706b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c069620 tx=0x7fdf0c00a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:58.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.227+0000 7fdf0b7fe700 1 -- 172.21.15.144:0/4211404750 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdf10058970 con 0x7fdf1c0ff2d0 2024-09-10T11:50:58.514 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.513+0000 7fdf20d48700 1 -- 172.21.15.144:0/4211404750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdf1c04ea50 con 0x7fdf1c0ff2d0 2024-09-10T11:50:58.514 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.513+0000 7fdf0b7fe700 1 -- 172.21.15.144:0/4211404750 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fdf1005c030 con 0x7fdf1c0ff2d0 2024-09-10T11:50:58.515 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:50:58.515 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:50:58.515 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:50:58.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:50:58.515 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:50:58.515 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:50:58.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:50:58.515 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:50:58.515 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:50:58.515 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:50:58.515 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:50:58.515 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:50:58.515 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:50:58.516 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:50:58.516 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:50:58.516 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:50:58.516 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:50:58.516 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:50:58.516 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:50:58.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.516+0000 7fdf20d48700 1 -- 172.21.15.144:0/4211404750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdf0406e200 msgr2=0x7fdf040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:58.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.516+0000 7fdf20d48700 1 --2- 172.21.15.144:0/4211404750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdf0406e200 0x7fdf040706b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c069620 tx=0x7fdf0c00a300 comp rx=0 tx=0).stop 2024-09-10T11:50:58.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.516+0000 7fdf20d48700 1 -- 172.21.15.144:0/4211404750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf1c0ff2d0 msgr2=0x7fdf1c196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:58.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.516+0000 7fdf20d48700 1 --2- 172.21.15.144:0/4211404750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf1c0ff2d0 0x7fdf1c196c40 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fdf10006010 tx=0x7fdf1000bd70 comp rx=0 tx=0).stop 2024-09-10T11:50:58.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.516+0000 7fdf20d48700 1 -- 172.21.15.144:0/4211404750 shutdown_connections 2024-09-10T11:50:58.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.516+0000 7fdf20d48700 1 --2- 172.21.15.144:0/4211404750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdf0406e200 0x7fdf040706b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.518 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.516+0000 7fdf20d48700 1 --2- 172.21.15.144:0/4211404750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdf1c0ff2d0 0x7fdf1c196c40 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.518 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.516+0000 7fdf20d48700 1 --2- 172.21.15.144:0/4211404750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdf1c0ffc70 0x7fdf1c197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.518 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.516+0000 7fdf20d48700 1 -- 172.21.15.144:0/4211404750 >> 172.21.15.144:0/4211404750 conn(0x7fdf1c0fabb0 msgr2=0x7fdf1c0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:58.518 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.516+0000 7fdf20d48700 1 -- 172.21.15.144:0/4211404750 shutdown_connections 2024-09-10T11:50:58.518 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.516+0000 7fdf20d48700 1 -- 172.21.15.144:0/4211404750 wait complete. 2024-09-10T11:50:58.622 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.621+0000 7f359be5a700 1 -- 172.21.15.144:0/3516380890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35940ff320 msgr2=0x7f35940ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:58.622 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.621+0000 7f359be5a700 1 --2- 172.21.15.144:0/3516380890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35940ff320 0x7f35940ff6f0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f3590009910 tx=0x7f3590009c20 comp rx=0 tx=0).stop 2024-09-10T11:50:58.622 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.621+0000 7f359be5a700 1 -- 172.21.15.144:0/3516380890 shutdown_connections 2024-09-10T11:50:58.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.621+0000 7f359be5a700 1 --2- 172.21.15.144:0/3516380890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35940ffc30 0x7f359410a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.621+0000 7f359be5a700 1 --2- 172.21.15.144:0/3516380890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35940ff320 0x7f35940ff6f0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.621+0000 7f359be5a700 1 -- 172.21.15.144:0/3516380890 >> 172.21.15.144:0/3516380890 conn(0x7f35940748b0 msgr2=0x7f3594074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:58.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.621+0000 7f359be5a700 1 -- 172.21.15.144:0/3516380890 shutdown_connections 2024-09-10T11:50:58.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.621+0000 7f359be5a700 1 -- 172.21.15.144:0/3516380890 wait complete. 2024-09-10T11:50:58.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.622+0000 7f359be5a700 1 Processor -- start 2024-09-10T11:50:58.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.622+0000 7f359be5a700 1 -- start start 2024-09-10T11:50:58.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.622+0000 7f359be5a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35940ff320 0x7f3594195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:58.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.622+0000 7f359be5a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35940ffc30 0x7f3594196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:58.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.622+0000 7f359be5a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3594196a00 con 0x7f35940ff320 2024-09-10T11:50:58.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.622+0000 7f359be5a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359419a790 con 0x7f35940ffc30 2024-09-10T11:50:58.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.622+0000 7f3599bf6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35940ff320 0x7f3594195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:58.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.622+0000 7f3599bf6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35940ff320 0x7f3594195de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34652/0 (socket says 172.21.15.144:34652) 2024-09-10T11:50:58.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.622+0000 7f3599bf6700 1 -- 172.21.15.144:0/2275624951 learned_addr learned my addr 172.21.15.144:0/2275624951 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:50:58.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.622+0000 7f35993f5700 1 --2- 172.21.15.144:0/2275624951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35940ffc30 0x7f3594196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:58.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.622+0000 7f3599bf6700 1 -- 172.21.15.144:0/2275624951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35940ffc30 msgr2=0x7f3594196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:58.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.622+0000 7f3599bf6700 1 --2- 172.21.15.144:0/2275624951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35940ffc30 0x7f3594196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.622+0000 7f3599bf6700 1 -- 172.21.15.144:0/2275624951 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35900095f0 con 0x7f35940ff320 2024-09-10T11:50:58.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.622+0000 7f35993f5700 1 --2- 172.21.15.144:0/2275624951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35940ffc30 0x7f3594196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:50:58.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.623+0000 7f3599bf6700 1 --2- 172.21.15.144:0/2275624951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35940ff320 0x7f3594195de0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3590006010 tx=0x7f359000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:58.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.623+0000 7f358affd700 1 -- 172.21.15.144:0/2275624951 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f359001d070 con 0x7f35940ff320 2024-09-10T11:50:58.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.623+0000 7f359be5a700 1 -- 172.21.15.144:0/2275624951 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359419aa10 con 0x7f35940ff320 2024-09-10T11:50:58.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.623+0000 7f358affd700 1 -- 172.21.15.144:0/2275624951 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3590004fb0 con 0x7f35940ff320 2024-09-10T11:50:58.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.623+0000 7f358affd700 1 -- 172.21.15.144:0/2275624951 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3590022ea0 con 0x7f35940ff320 2024-09-10T11:50:58.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.623+0000 7f359be5a700 1 -- 172.21.15.144:0/2275624951 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359419af80 con 0x7f35940ff320 2024-09-10T11:50:58.627 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.624+0000 7f359be5a700 1 -- 172.21.15.144:0/2275624951 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3594107da0 con 0x7f35940ff320 2024-09-10T11:50:58.627 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.624+0000 7f358affd700 1 -- 172.21.15.144:0/2275624951 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3590022470 con 0x7f35940ff320 2024-09-10T11:50:58.627 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.625+0000 7f358affd700 1 --2- 172.21.15.144:0/2275624951 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f358006e200 0x7f35800706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:58.627 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.625+0000 7f358affd700 1 -- 172.21.15.144:0/2275624951 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f359008f610 con 0x7f35940ff320 2024-09-10T11:50:58.627 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.625+0000 7f35993f5700 1 --2- 172.21.15.144:0/2275624951 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f358006e200 0x7f35800706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:58.627 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.625+0000 7f35993f5700 1 --2- 172.21.15.144:0/2275624951 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f358006e200 0x7f35800706b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3594197400 tx=0x7f3584008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:58.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.629+0000 7f358affd700 1 -- 172.21.15.144:0/2275624951 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3590058aa0 con 0x7f35940ff320 2024-09-10T11:50:58.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.874+0000 7f359be5a700 1 -- 172.21.15.144:0/2275624951 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f359404ea50 con 0x7f35940ff320 2024-09-10T11:50:58.876 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.875+0000 7f358affd700 1 -- 172.21.15.144:0/2275624951 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f3590022720 con 0x7f35940ff320 2024-09-10T11:50:58.876 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:50:58.876 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:50:58.876 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:50:58.877 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:50:58.877 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:50:58.877 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:50:58.877 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:50:58.877 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:50:58.877 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:50:58.877 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:50:58.877 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:50:58.877 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:50:58.877 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:50:58.877 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:50:58.877 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:50:58.877 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:50:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:50:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:50:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:50:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:50:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:50:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:50:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:50:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:50:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:50:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:50:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:50:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:50:58.878 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:50:58.879 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:50:58.879 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:50:58.879 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:50:58.879 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:50:58.879 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:50:58.879 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:50:58.879 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:50:58.879 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:50:58.879 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:50:58.879 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:50:58.879 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:50:58.879 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:50:58.879 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:50:58.879 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:50:58.880 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.877+0000 7f359be5a700 1 -- 172.21.15.144:0/2275624951 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f358006e200 msgr2=0x7f35800706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:58.880 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.877+0000 7f359be5a700 1 --2- 172.21.15.144:0/2275624951 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f358006e200 0x7f35800706b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3594197400 tx=0x7f3584008040 comp rx=0 tx=0).stop 2024-09-10T11:50:58.880 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.877+0000 7f359be5a700 1 -- 172.21.15.144:0/2275624951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35940ff320 msgr2=0x7f3594195de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:58.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.877+0000 7f359be5a700 1 --2- 172.21.15.144:0/2275624951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35940ff320 0x7f3594195de0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f3590006010 tx=0x7f359000be20 comp rx=0 tx=0).stop 2024-09-10T11:50:58.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.877+0000 7f359be5a700 1 -- 172.21.15.144:0/2275624951 shutdown_connections 2024-09-10T11:50:58.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.877+0000 7f359be5a700 1 --2- 172.21.15.144:0/2275624951 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f358006e200 0x7f35800706b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.877+0000 7f359be5a700 1 --2- 172.21.15.144:0/2275624951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f35940ff320 0x7f3594195de0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.877+0000 7f359be5a700 1 --2- 172.21.15.144:0/2275624951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f35940ffc30 0x7f3594196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.877+0000 7f359be5a700 1 -- 172.21.15.144:0/2275624951 >> 172.21.15.144:0/2275624951 conn(0x7f35940748b0 msgr2=0x7f3594104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:58.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.877+0000 7f359be5a700 1 -- 172.21.15.144:0/2275624951 shutdown_connections 2024-09-10T11:50:58.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.877+0000 7f359be5a700 1 -- 172.21.15.144:0/2275624951 wait complete. 2024-09-10T11:50:58.881 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:50:58.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.983+0000 7f747fac0700 1 -- 172.21.15.144:0/1124909843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74780ffd30 msgr2=0x7f7478103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:58.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.983+0000 7f747fac0700 1 --2- 172.21.15.144:0/1124909843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74780ffd30 0x7f7478103ce0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f7474009910 tx=0x7f7474009c20 comp rx=0 tx=0).stop 2024-09-10T11:50:58.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.983+0000 7f747fac0700 1 -- 172.21.15.144:0/1124909843 shutdown_connections 2024-09-10T11:50:58.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.983+0000 7f747fac0700 1 --2- 172.21.15.144:0/1124909843 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74780ffd30 0x7f7478103ce0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.983+0000 7f747fac0700 1 --2- 172.21.15.144:0/1124909843 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74780ff390 0x7f74780ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.983+0000 7f747fac0700 1 -- 172.21.15.144:0/1124909843 >> 172.21.15.144:0/1124909843 conn(0x7f7478074f50 msgr2=0x7f7478075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:58.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.983+0000 7f747fac0700 1 -- 172.21.15.144:0/1124909843 shutdown_connections 2024-09-10T11:50:58.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.983+0000 7f747fac0700 1 -- 172.21.15.144:0/1124909843 wait complete. 2024-09-10T11:50:58.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.984+0000 7f747fac0700 1 Processor -- start 2024-09-10T11:50:58.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.984+0000 7f747fac0700 1 -- start start 2024-09-10T11:50:58.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.984+0000 7f747fac0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74780ff390 0x7f7478195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:58.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.984+0000 7f747fac0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74780ffd30 0x7f7478196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:58.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.984+0000 7f747fac0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7478196a60 con 0x7f74780ffd30 2024-09-10T11:50:58.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.984+0000 7f747fac0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747819a7f0 con 0x7f74780ff390 2024-09-10T11:50:58.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.984+0000 7f747d05b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74780ffd30 0x7f7478196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:58.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.984+0000 7f747d05b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74780ffd30 0x7f7478196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34662/0 (socket says 172.21.15.144:34662) 2024-09-10T11:50:58.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.984+0000 7f747d05b700 1 -- 172.21.15.144:0/152776269 learned_addr learned my addr 172.21.15.144:0/152776269 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:50:58.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.984+0000 7f747d85c700 1 --2- 172.21.15.144:0/152776269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74780ff390 0x7f7478195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:58.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.984+0000 7f747d05b700 1 -- 172.21.15.144:0/152776269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74780ff390 msgr2=0x7f7478195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:58.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.984+0000 7f747d05b700 1 --2- 172.21.15.144:0/152776269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74780ff390 0x7f7478195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:58.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.984+0000 7f747d05b700 1 -- 172.21.15.144:0/152776269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74740095f0 con 0x7f74780ffd30 2024-09-10T11:50:58.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.984+0000 7f747d05b700 1 --2- 172.21.15.144:0/152776269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74780ffd30 0x7f7478196380 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7474005f50 tx=0x7f747400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:58.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.985+0000 7f746effd700 1 -- 172.21.15.144:0/152776269 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f747401d070 con 0x7f74780ffd30 2024-09-10T11:50:58.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.985+0000 7f746effd700 1 -- 172.21.15.144:0/152776269 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f747400f460 con 0x7f74780ffd30 2024-09-10T11:50:58.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.985+0000 7f746effd700 1 -- 172.21.15.144:0/152776269 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7474017630 con 0x7f74780ffd30 2024-09-10T11:50:58.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.985+0000 7f747fac0700 1 -- 172.21.15.144:0/152776269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f747819aa70 con 0x7f74780ffd30 2024-09-10T11:50:58.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.985+0000 7f747fac0700 1 -- 172.21.15.144:0/152776269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747819af60 con 0x7f74780ffd30 2024-09-10T11:50:58.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.986+0000 7f747fac0700 1 -- 172.21.15.144:0/152776269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f747804ea50 con 0x7f74780ffd30 2024-09-10T11:50:58.989 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.986+0000 7f746effd700 1 -- 172.21.15.144:0/152776269 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f747400f5d0 con 0x7f74780ffd30 2024-09-10T11:50:58.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.987+0000 7f746effd700 1 --2- 172.21.15.144:0/152776269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f746406e200 0x7f74640706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:58.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.987+0000 7f746effd700 1 -- 172.21.15.144:0/152776269 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f747408f690 con 0x7f74780ffd30 2024-09-10T11:50:58.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.987+0000 7f747d85c700 1 --2- 172.21.15.144:0/152776269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f746406e200 0x7f74640706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:58.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.987+0000 7f747d85c700 1 --2- 172.21.15.144:0/152776269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f746406e200 0x7f74640706b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7468006fd0 tx=0x7f7468008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:58.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:58.991+0000 7f746effd700 1 -- 172.21.15.144:0/152776269 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7474058a50 con 0x7f74780ffd30 2024-09-10T11:50:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:58 smithi144 ceph-mon[87436]: from='client.35072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:50:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:58 smithi144 ceph-mon[87436]: from='client.35076 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:50:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:58 smithi144 ceph-mon[87436]: pgmap v569: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:58 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/4211404750' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:50:59.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:50:59.213 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.212+0000 7f747fac0700 1 -- 172.21.15.144:0/152776269 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f747819b510 con 0x7f746406e200 2024-09-10T11:50:59.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.214+0000 7f746effd700 1 -- 172.21.15.144:0/152776269 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f747819b510 con 0x7f746406e200 2024-09-10T11:50:59.215 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:50:59.215 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:50:59.215 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:50:59.215 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:50:59.215 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:50:59.215 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:50:59.216 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:50:59.216 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:50:59.216 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:50:59.216 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:50:59.216 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:50:59.216 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:50:59.216 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:50:59.216 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:50:59.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.216+0000 7f747fac0700 1 -- 172.21.15.144:0/152776269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f746406e200 msgr2=0x7f74640706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:59.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.216+0000 7f747fac0700 1 --2- 172.21.15.144:0/152776269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f746406e200 0x7f74640706b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f7468006fd0 tx=0x7f7468008040 comp rx=0 tx=0).stop 2024-09-10T11:50:59.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.216+0000 7f747fac0700 1 -- 172.21.15.144:0/152776269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74780ffd30 msgr2=0x7f7478196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:59.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.216+0000 7f747fac0700 1 --2- 172.21.15.144:0/152776269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74780ffd30 0x7f7478196380 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7474005f50 tx=0x7f747400be20 comp rx=0 tx=0).stop 2024-09-10T11:50:59.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.216+0000 7f747fac0700 1 -- 172.21.15.144:0/152776269 shutdown_connections 2024-09-10T11:50:59.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.216+0000 7f747fac0700 1 --2- 172.21.15.144:0/152776269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f746406e200 0x7f74640706b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:59.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.216+0000 7f747fac0700 1 --2- 172.21.15.144:0/152776269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f74780ff390 0x7f7478195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:59.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.216+0000 7f747fac0700 1 --2- 172.21.15.144:0/152776269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f74780ffd30 0x7f7478196380 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:59.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.216+0000 7f747fac0700 1 -- 172.21.15.144:0/152776269 >> 172.21.15.144:0/152776269 conn(0x7f7478074f50 msgr2=0x7f7478103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:59.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.216+0000 7f747fac0700 1 -- 172.21.15.144:0/152776269 shutdown_connections 2024-09-10T11:50:59.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.216+0000 7f747fac0700 1 -- 172.21.15.144:0/152776269 wait complete. 2024-09-10T11:50:59.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.320+0000 7fa53d20c700 1 -- 172.21.15.144:0/3359063273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5381003d0 msgr2=0x7fa5381007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.320+0000 7fa53d20c700 1 --2- 172.21.15.144:0/3359063273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5381003d0 0x7fa5381007a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fa520009910 tx=0x7fa520009c20 comp rx=0 tx=0).stop 2024-09-10T11:50:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.320+0000 7fa53d20c700 1 -- 172.21.15.144:0/3359063273 shutdown_connections 2024-09-10T11:50:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.320+0000 7fa53d20c700 1 --2- 172.21.15.144:0/3359063273 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa538100d70 0x7fa538104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.320+0000 7fa53d20c700 1 --2- 172.21.15.144:0/3359063273 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa5381003d0 0x7fa5381007a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.320+0000 7fa53d20c700 1 -- 172.21.15.144:0/3359063273 >> 172.21.15.144:0/3359063273 conn(0x7fa5380fbc50 msgr2=0x7fa5380fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.320+0000 7fa53d20c700 1 -- 172.21.15.144:0/3359063273 shutdown_connections 2024-09-10T11:50:59.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.320+0000 7fa53d20c700 1 -- 172.21.15.144:0/3359063273 wait complete. 2024-09-10T11:50:59.324 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.321+0000 7fa53d20c700 1 Processor -- start 2024-09-10T11:50:59.324 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.321+0000 7fa53d20c700 1 -- start start 2024-09-10T11:50:59.324 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.321+0000 7fa53d20c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5381003d0 0x7fa538195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:59.324 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.321+0000 7fa53d20c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa538100d70 0x7fa5381963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.321+0000 7fa53d20c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa538196a80 con 0x7fa538100d70 2024-09-10T11:50:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.321+0000 7fa53d20c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa53819a810 con 0x7fa5381003d0 2024-09-10T11:50:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.322+0000 7fa53659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa538100d70 0x7fa5381963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.322+0000 7fa53659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa538100d70 0x7fa5381963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34688/0 (socket says 172.21.15.144:34688) 2024-09-10T11:50:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.322+0000 7fa53659c700 1 -- 172.21.15.144:0/2914155313 learned_addr learned my addr 172.21.15.144:0/2914155313 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:50:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.322+0000 7fa536d9d700 1 --2- 172.21.15.144:0/2914155313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5381003d0 0x7fa538195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.322+0000 7fa53659c700 1 -- 172.21.15.144:0/2914155313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5381003d0 msgr2=0x7fa538195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.322+0000 7fa53659c700 1 --2- 172.21.15.144:0/2914155313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5381003d0 0x7fa538195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.322+0000 7fa53659c700 1 -- 172.21.15.144:0/2914155313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5200095f0 con 0x7fa538100d70 2024-09-10T11:50:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.322+0000 7fa53659c700 1 --2- 172.21.15.144:0/2914155313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa538100d70 0x7fa5381963a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa52800b480 tx=0x7fa52800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.322+0000 7fa52ffff700 1 -- 172.21.15.144:0/2914155313 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa528011800 con 0x7fa538100d70 2024-09-10T11:50:59.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.322+0000 7fa53d20c700 1 -- 172.21.15.144:0/2914155313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa53819aaf0 con 0x7fa538100d70 2024-09-10T11:50:59.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.322+0000 7fa52ffff700 1 -- 172.21.15.144:0/2914155313 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa528011e20 con 0x7fa538100d70 2024-09-10T11:50:59.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.322+0000 7fa52ffff700 1 -- 172.21.15.144:0/2914155313 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa528010ea0 con 0x7fa538100d70 2024-09-10T11:50:59.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.322+0000 7fa53d20c700 1 -- 172.21.15.144:0/2914155313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa53819b040 con 0x7fa538100d70 2024-09-10T11:50:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.323+0000 7fa53d20c700 1 -- 172.21.15.144:0/2914155313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5381086e0 con 0x7fa538100d70 2024-09-10T11:50:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.324+0000 7fa52ffff700 1 -- 172.21.15.144:0/2914155313 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa528010380 con 0x7fa538100d70 2024-09-10T11:50:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.324+0000 7fa52ffff700 1 --2- 172.21.15.144:0/2914155313 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa52406e200 0x7fa5240706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:50:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.324+0000 7fa52ffff700 1 -- 172.21.15.144:0/2914155313 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa52808daf0 con 0x7fa538100d70 2024-09-10T11:50:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.325+0000 7fa536d9d700 1 --2- 172.21.15.144:0/2914155313 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa52406e200 0x7fa5240706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:50:59.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.325+0000 7fa536d9d700 1 --2- 172.21.15.144:0/2914155313 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa52406e200 0x7fa5240706b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa520006010 tx=0x7fa52001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:50:59.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.329+0000 7fa52ffff700 1 -- 172.21.15.144:0/2914155313 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa528056f50 con 0x7fa538100d70 2024-09-10T11:50:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:58 smithi178 ceph-mon[65940]: from='client.35072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:50:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:58 smithi178 ceph-mon[65940]: from='client.35076 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:50:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:58 smithi178 ceph-mon[65940]: pgmap v569: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:50:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:58 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/4211404750' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:50:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:50:59.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.611+0000 7fa53d20c700 1 -- 172.21.15.144:0/2914155313 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa53804ea50 con 0x7fa538100d70 2024-09-10T11:50:59.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.612+0000 7fa52ffff700 1 -- 172.21.15.144:0/2914155313 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fa5280151c0 con 0x7fa538100d70 2024-09-10T11:50:59.617 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:50:59.617 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:50:59.617 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:50:59.617 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:50:59.617 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:50:59.617 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:50:59.617 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:50:59.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.615+0000 7fa53d20c700 1 -- 172.21.15.144:0/2914155313 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa52406e200 msgr2=0x7fa5240706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:59.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.615+0000 7fa53d20c700 1 --2- 172.21.15.144:0/2914155313 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa52406e200 0x7fa5240706b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa520006010 tx=0x7fa52001a040 comp rx=0 tx=0).stop 2024-09-10T11:50:59.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.615+0000 7fa53d20c700 1 -- 172.21.15.144:0/2914155313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa538100d70 msgr2=0x7fa5381963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:50:59.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.615+0000 7fa53d20c700 1 --2- 172.21.15.144:0/2914155313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa538100d70 0x7fa5381963a0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fa52800b480 tx=0x7fa52800b790 comp rx=0 tx=0).stop 2024-09-10T11:50:59.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.615+0000 7fa53d20c700 1 -- 172.21.15.144:0/2914155313 shutdown_connections 2024-09-10T11:50:59.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.615+0000 7fa53d20c700 1 --2- 172.21.15.144:0/2914155313 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa52406e200 0x7fa5240706b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:59.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.615+0000 7fa53d20c700 1 --2- 172.21.15.144:0/2914155313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa5381003d0 0x7fa538195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:59.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.615+0000 7fa53d20c700 1 --2- 172.21.15.144:0/2914155313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa538100d70 0x7fa5381963a0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:50:59.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.615+0000 7fa53d20c700 1 -- 172.21.15.144:0/2914155313 >> 172.21.15.144:0/2914155313 conn(0x7fa5380fbc50 msgr2=0x7fa5380fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:50:59.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.615+0000 7fa53d20c700 1 -- 172.21.15.144:0/2914155313 shutdown_connections 2024-09-10T11:50:59.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:50:59.615+0000 7fa53d20c700 1 -- 172.21.15.144:0/2914155313 wait complete. 2024-09-10T11:51:00.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:59 smithi144 ceph-mon[87436]: from='client.35080 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:51:00.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:59 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2275624951' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:51:00.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:50:59 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2914155313' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:51:00.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:59 smithi178 ceph-mon[65940]: from='client.35080 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:51:00.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:59 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2275624951' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:51:00.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:50:59 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2914155313' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:51:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:00 smithi144 ceph-mon[87436]: from='client.35092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:51:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:00 smithi144 ceph-mon[87436]: pgmap v570: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:00 smithi178 ceph-mon[65940]: from='client.35092 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:51:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:00 smithi178 ceph-mon[65940]: pgmap v570: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:02.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:01 smithi144 ceph-mon[87436]: pgmap v571: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:02.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:01 smithi178 ceph-mon[65940]: pgmap v571: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:05.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:04 smithi144 ceph-mon[87436]: pgmap v572: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:04 smithi178 ceph-mon[65940]: pgmap v572: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:07.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:06 smithi144 ceph-mon[87436]: pgmap v573: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:06 smithi178 ceph-mon[65940]: pgmap v573: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:08.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:07 smithi144 ceph-mon[87436]: pgmap v574: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:08.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:07 smithi178 ceph-mon[65940]: pgmap v574: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:11.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:10 smithi144 ceph-mon[87436]: pgmap v575: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:10 smithi178 ceph-mon[65940]: pgmap v575: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:13.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:12 smithi144 ceph-mon[87436]: pgmap v576: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:13.366 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:12 smithi178 ceph-mon[65940]: pgmap v576: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:14.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:51:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:51:15.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:14 smithi144 ceph-mon[87436]: pgmap v577: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:14 smithi178 ceph-mon[65940]: pgmap v577: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:16.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:15 smithi144 ceph-mon[87436]: pgmap v578: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:16.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:15 smithi178 ceph-mon[65940]: pgmap v578: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:19.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:18 smithi144 ceph-mon[87436]: pgmap v579: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:19.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:18 smithi178 ceph-mon[65940]: pgmap v579: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:21.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:20 smithi144 ceph-mon[87436]: pgmap v580: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:20 smithi178 ceph-mon[65940]: pgmap v580: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:23.193 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:22 smithi144 ceph-mon[87436]: pgmap v581: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:23.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:22 smithi178 ceph-mon[65940]: pgmap v581: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:23 smithi144 ceph-mon[87436]: pgmap v582: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:24.366 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:23 smithi178 ceph-mon[65940]: pgmap v582: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:26 smithi144 ceph-mon[87436]: pgmap v583: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:26 smithi178 ceph-mon[65940]: pgmap v583: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:28 smithi144 ceph-mon[87436]: pgmap v584: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:51:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:28 smithi178 ceph-mon[65940]: pgmap v584: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:51:29.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.734+0000 7f0109ae1700 1 -- 172.21.15.144:0/2882865555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01041003d0 msgr2=0x7f01041007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:29.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.734+0000 7f0109ae1700 1 --2- 172.21.15.144:0/2882865555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01041003d0 0x7f01041007a0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f00f400b0a0 tx=0x7f00f400b3b0 comp rx=0 tx=0).stop 2024-09-10T11:51:29.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.734+0000 7f0109ae1700 1 -- 172.21.15.144:0/2882865555 shutdown_connections 2024-09-10T11:51:29.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.734+0000 7f0109ae1700 1 --2- 172.21.15.144:0/2882865555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0104100d70 0x7f0104104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:29.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.734+0000 7f0109ae1700 1 --2- 172.21.15.144:0/2882865555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f01041003d0 0x7f01041007a0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:29.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.734+0000 7f0109ae1700 1 -- 172.21.15.144:0/2882865555 >> 172.21.15.144:0/2882865555 conn(0x7f01040fbc90 msgr2=0x7f01040fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:51:29.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.734+0000 7f0109ae1700 1 -- 172.21.15.144:0/2882865555 shutdown_connections 2024-09-10T11:51:29.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.734+0000 7f0109ae1700 1 -- 172.21.15.144:0/2882865555 wait complete. 2024-09-10T11:51:29.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.735+0000 7f0109ae1700 1 Processor -- start 2024-09-10T11:51:29.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.735+0000 7f0109ae1700 1 -- start start 2024-09-10T11:51:29.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.735+0000 7f0109ae1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01041003d0 0x7f0104073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.735+0000 7f0109ae1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0104100d70 0x7f0104073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.735+0000 7f0109ae1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0104073ad0 con 0x7f0104100d70 2024-09-10T11:51:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.735+0000 7f0109ae1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0104073c10 con 0x7f01041003d0 2024-09-10T11:51:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.735+0000 7f0102ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0104100d70 0x7f0104073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.735+0000 7f0102ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0104100d70 0x7f0104073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36416/0 (socket says 172.21.15.144:36416) 2024-09-10T11:51:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.735+0000 7f0102ffd700 1 -- 172.21.15.144:0/2569804463 learned_addr learned my addr 172.21.15.144:0/2569804463 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:51:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.735+0000 7f01037fe700 1 --2- 172.21.15.144:0/2569804463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01041003d0 0x7f0104073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.736+0000 7f0102ffd700 1 -- 172.21.15.144:0/2569804463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01041003d0 msgr2=0x7f0104073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.736+0000 7f0102ffd700 1 --2- 172.21.15.144:0/2569804463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01041003d0 0x7f0104073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.736+0000 7f0102ffd700 1 -- 172.21.15.144:0/2569804463 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f4009d00 con 0x7f0104100d70 2024-09-10T11:51:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.736+0000 7f0102ffd700 1 --2- 172.21.15.144:0/2569804463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0104100d70 0x7f0104073590 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f00f800d900 tx=0x7f00f800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:51:29.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.736+0000 7f0100ff9700 1 -- 172.21.15.144:0/2569804463 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00f8009900 con 0x7f0104100d70 2024-09-10T11:51:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.736+0000 7f0100ff9700 1 -- 172.21.15.144:0/2569804463 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f00f8010460 con 0x7f0104100d70 2024-09-10T11:51:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.736+0000 7f0100ff9700 1 -- 172.21.15.144:0/2569804463 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00f800f5f0 con 0x7f0104100d70 2024-09-10T11:51:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.736+0000 7f0109ae1700 1 -- 172.21.15.144:0/2569804463 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01041a1f30 con 0x7f0104100d70 2024-09-10T11:51:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.736+0000 7f0109ae1700 1 -- 172.21.15.144:0/2569804463 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01041a2340 con 0x7f0104100d70 2024-09-10T11:51:29.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.737+0000 7f0109ae1700 1 -- 172.21.15.144:0/2569804463 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01041086e0 con 0x7f0104100d70 2024-09-10T11:51:29.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.737+0000 7f0100ff9700 1 -- 172.21.15.144:0/2569804463 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f00f80105d0 con 0x7f0104100d70 2024-09-10T11:51:29.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.738+0000 7f0100ff9700 1 --2- 172.21.15.144:0/2569804463 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f00ec06e200 0x7f00ec0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:29.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.738+0000 7f0100ff9700 1 -- 172.21.15.144:0/2569804463 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f00f808dc60 con 0x7f0104100d70 2024-09-10T11:51:29.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.738+0000 7f01037fe700 1 --2- 172.21.15.144:0/2569804463 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f00ec06e200 0x7f00ec0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:29.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.738+0000 7f01037fe700 1 --2- 172.21.15.144:0/2569804463 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f00ec06e200 0x7f00ec0706b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f00f4000f80 tx=0x7f00f40060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:51:29.743 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.742+0000 7f0100ff9700 1 -- 172.21.15.144:0/2569804463 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f00f8057040 con 0x7f0104100d70 2024-09-10T11:51:29.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.966+0000 7f0109ae1700 1 -- 172.21.15.144:0/2569804463 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f010402cd40 con 0x7f00ec06e200 2024-09-10T11:51:29.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.967+0000 7f0100ff9700 1 -- 172.21.15.144:0/2569804463 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f010402cd40 con 0x7f00ec06e200 2024-09-10T11:51:29.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.969+0000 7f0109ae1700 1 -- 172.21.15.144:0/2569804463 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f00ec06e200 msgr2=0x7f00ec0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:29.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.969+0000 7f0109ae1700 1 --2- 172.21.15.144:0/2569804463 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f00ec06e200 0x7f00ec0706b0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f00f4000f80 tx=0x7f00f40060c0 comp rx=0 tx=0).stop 2024-09-10T11:51:29.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.969+0000 7f0109ae1700 1 -- 172.21.15.144:0/2569804463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0104100d70 msgr2=0x7f0104073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:29.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.969+0000 7f0109ae1700 1 --2- 172.21.15.144:0/2569804463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0104100d70 0x7f0104073590 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f00f800d900 tx=0x7f00f800dc10 comp rx=0 tx=0).stop 2024-09-10T11:51:29.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.969+0000 7f0109ae1700 1 -- 172.21.15.144:0/2569804463 shutdown_connections 2024-09-10T11:51:29.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.969+0000 7f0109ae1700 1 --2- 172.21.15.144:0/2569804463 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f00ec06e200 0x7f00ec0706b0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:29.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.969+0000 7f0109ae1700 1 --2- 172.21.15.144:0/2569804463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f01041003d0 0x7f0104073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:29.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.969+0000 7f0109ae1700 1 --2- 172.21.15.144:0/2569804463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0104100d70 0x7f0104073590 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:29.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.969+0000 7f0109ae1700 1 -- 172.21.15.144:0/2569804463 >> 172.21.15.144:0/2569804463 conn(0x7f01040fbc90 msgr2=0x7f01040fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:51:29.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.969+0000 7f0109ae1700 1 -- 172.21.15.144:0/2569804463 shutdown_connections 2024-09-10T11:51:29.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:29.969+0000 7f0109ae1700 1 -- 172.21.15.144:0/2569804463 wait complete. 2024-09-10T11:51:29.981 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:51:30.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.079+0000 7f1571fff700 1 -- 172.21.15.144:0/2750990232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f156c101cf0 msgr2=0x7f156c10a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:30.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.079+0000 7f1571fff700 1 --2- 172.21.15.144:0/2750990232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f156c101cf0 0x7f156c10a1f0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1554009940 tx=0x7f1554009c50 comp rx=0 tx=0).stop 2024-09-10T11:51:30.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.079+0000 7f1571fff700 1 -- 172.21.15.144:0/2750990232 shutdown_connections 2024-09-10T11:51:30.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.079+0000 7f1571fff700 1 --2- 172.21.15.144:0/2750990232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f156c101cf0 0x7f156c10a1f0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.079+0000 7f1571fff700 1 --2- 172.21.15.144:0/2750990232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156c1013e0 0x7f156c1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.079+0000 7f1571fff700 1 -- 172.21.15.144:0/2750990232 >> 172.21.15.144:0/2750990232 conn(0x7f156c0fac20 msgr2=0x7f156c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:51:30.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.079+0000 7f1571fff700 1 -- 172.21.15.144:0/2750990232 shutdown_connections 2024-09-10T11:51:30.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.079+0000 7f1571fff700 1 -- 172.21.15.144:0/2750990232 wait complete. 2024-09-10T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.080+0000 7f1571fff700 1 Processor -- start 2024-09-10T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.080+0000 7f1571fff700 1 -- start start 2024-09-10T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.080+0000 7f1571fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f156c1013e0 0x7f156c195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.080+0000 7f1571fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156c101cf0 0x7f156c196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.080+0000 7f1571fff700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c196930 con 0x7f156c1013e0 2024-09-10T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.080+0000 7f1571fff700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c19a6c0 con 0x7f156c101cf0 2024-09-10T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.080+0000 7f156b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f156c1013e0 0x7f156c195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.080+0000 7f156b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f156c1013e0 0x7f156c195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36436/0 (socket says 172.21.15.144:36436) 2024-09-10T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.080+0000 7f156b7fe700 1 -- 172.21.15.144:0/203020023 learned_addr learned my addr 172.21.15.144:0/203020023 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.081+0000 7f156affd700 1 --2- 172.21.15.144:0/203020023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156c101cf0 0x7f156c196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:30.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.081+0000 7f156b7fe700 1 -- 172.21.15.144:0/203020023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156c101cf0 msgr2=0x7f156c196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:30.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.081+0000 7f156b7fe700 1 --2- 172.21.15.144:0/203020023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156c101cf0 0x7f156c196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.081+0000 7f156b7fe700 1 -- 172.21.15.144:0/203020023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15540095f0 con 0x7f156c1013e0 2024-09-10T11:51:30.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.081+0000 7f156affd700 1 --2- 172.21.15.144:0/203020023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156c101cf0 0x7f156c196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:51:30.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.081+0000 7f156b7fe700 1 --2- 172.21.15.144:0/203020023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f156c1013e0 0x7f156c195d10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f155c00d8d0 tx=0x7f155c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:51:30.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.081+0000 7f1568ff9700 1 -- 172.21.15.144:0/203020023 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f155c009940 con 0x7f156c1013e0 2024-09-10T11:51:30.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.081+0000 7f1571fff700 1 -- 172.21.15.144:0/203020023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c19a9a0 con 0x7f156c1013e0 2024-09-10T11:51:30.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.081+0000 7f1568ff9700 1 -- 172.21.15.144:0/203020023 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f155c010460 con 0x7f156c1013e0 2024-09-10T11:51:30.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.081+0000 7f1568ff9700 1 -- 172.21.15.144:0/203020023 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f155c00f5f0 con 0x7f156c1013e0 2024-09-10T11:51:30.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.081+0000 7f1571fff700 1 -- 172.21.15.144:0/203020023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c19aef0 con 0x7f156c1013e0 2024-09-10T11:51:30.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.082+0000 7f1571fff700 1 -- 172.21.15.144:0/203020023 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c107860 con 0x7f156c1013e0 2024-09-10T11:51:30.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.083+0000 7f1568ff9700 1 -- 172.21.15.144:0/203020023 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f155c0105d0 con 0x7f156c1013e0 2024-09-10T11:51:30.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.083+0000 7f1568ff9700 1 --2- 172.21.15.144:0/203020023 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f155806e200 0x7f15580706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:30.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.083+0000 7f1568ff9700 1 -- 172.21.15.144:0/203020023 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f155c08df90 con 0x7f156c1013e0 2024-09-10T11:51:30.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.083+0000 7f156affd700 1 --2- 172.21.15.144:0/203020023 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f155806e200 0x7f15580706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:30.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.084+0000 7f156affd700 1 --2- 172.21.15.144:0/203020023 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f155806e200 0x7f15580706b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f156c197330 tx=0x7f1554005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:51:30.089 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.088+0000 7f1568ff9700 1 -- 172.21.15.144:0/203020023 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f155c0573f0 con 0x7f156c1013e0 2024-09-10T11:51:30.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.310+0000 7f1571fff700 1 -- 172.21.15.144:0/203020023 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f156c19b240 con 0x7f155806e200 2024-09-10T11:51:30.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.311+0000 7f1568ff9700 1 -- 172.21.15.144:0/203020023 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f156c19b240 con 0x7f155806e200 2024-09-10T11:51:30.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.313+0000 7f1571fff700 1 -- 172.21.15.144:0/203020023 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f155806e200 msgr2=0x7f15580706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:30.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.313+0000 7f1571fff700 1 --2- 172.21.15.144:0/203020023 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f155806e200 0x7f15580706b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f156c197330 tx=0x7f1554005fd0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.313+0000 7f1571fff700 1 -- 172.21.15.144:0/203020023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f156c1013e0 msgr2=0x7f156c195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:30.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.313+0000 7f1571fff700 1 --2- 172.21.15.144:0/203020023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f156c1013e0 0x7f156c195d10 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f155c00d8d0 tx=0x7f155c00dc90 comp rx=0 tx=0).stop 2024-09-10T11:51:30.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.313+0000 7f1571fff700 1 -- 172.21.15.144:0/203020023 shutdown_connections 2024-09-10T11:51:30.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.313+0000 7f1571fff700 1 --2- 172.21.15.144:0/203020023 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f155806e200 0x7f15580706b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.313+0000 7f1571fff700 1 --2- 172.21.15.144:0/203020023 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f156c1013e0 0x7f156c195d10 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.313+0000 7f1571fff700 1 --2- 172.21.15.144:0/203020023 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f156c101cf0 0x7f156c196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.314+0000 7f1571fff700 1 -- 172.21.15.144:0/203020023 >> 172.21.15.144:0/203020023 conn(0x7f156c0fac20 msgr2=0x7f156c1049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:51:30.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.314+0000 7f1571fff700 1 -- 172.21.15.144:0/203020023 shutdown_connections 2024-09-10T11:51:30.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.314+0000 7f1571fff700 1 -- 172.21.15.144:0/203020023 wait complete. 2024-09-10T11:51:30.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.418+0000 7f635622c700 1 -- 172.21.15.144:0/2129146405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6350100d70 msgr2=0x7f6350104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:30.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.418+0000 7f635622c700 1 --2- 172.21.15.144:0/2129146405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6350100d70 0x7f6350104d40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6340009910 tx=0x7f6340009c20 comp rx=0 tx=0).stop 2024-09-10T11:51:30.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.418+0000 7f635622c700 1 -- 172.21.15.144:0/2129146405 shutdown_connections 2024-09-10T11:51:30.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.418+0000 7f635622c700 1 --2- 172.21.15.144:0/2129146405 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6350100d70 0x7f6350104d40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.418+0000 7f635622c700 1 --2- 172.21.15.144:0/2129146405 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63501003d0 0x7f63501007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.418+0000 7f635622c700 1 -- 172.21.15.144:0/2129146405 >> 172.21.15.144:0/2129146405 conn(0x7f63500fbc90 msgr2=0x7f63500fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:51:30.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.419+0000 7f635622c700 1 -- 172.21.15.144:0/2129146405 shutdown_connections 2024-09-10T11:51:30.420 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.419+0000 7f635622c700 1 -- 172.21.15.144:0/2129146405 wait complete. 2024-09-10T11:51:30.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.419+0000 7f635622c700 1 Processor -- start 2024-09-10T11:51:30.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.420+0000 7f635622c700 1 -- start start 2024-09-10T11:51:30.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.420+0000 7f635622c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63501003d0 0x7f6350195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:30.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.420+0000 7f635622c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6350100d70 0x7f6350196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:30.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.420+0000 7f635622c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6350196960 con 0x7f6350100d70 2024-09-10T11:51:30.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.420+0000 7f635622c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635019a6f0 con 0x7f63501003d0 2024-09-10T11:51:30.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.420+0000 7f634effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6350100d70 0x7f6350196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:30.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.420+0000 7f634effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6350100d70 0x7f6350196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36458/0 (socket says 172.21.15.144:36458) 2024-09-10T11:51:30.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.420+0000 7f634effd700 1 -- 172.21.15.144:0/1754042691 learned_addr learned my addr 172.21.15.144:0/1754042691 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:51:30.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.420+0000 7f634f7fe700 1 --2- 172.21.15.144:0/1754042691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63501003d0 0x7f6350195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:30.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.420+0000 7f634effd700 1 -- 172.21.15.144:0/1754042691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63501003d0 msgr2=0x7f6350195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:30.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.420+0000 7f634effd700 1 --2- 172.21.15.144:0/1754042691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63501003d0 0x7f6350195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.420+0000 7f634effd700 1 -- 172.21.15.144:0/1754042691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63400095f0 con 0x7f6350100d70 2024-09-10T11:51:30.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.421+0000 7f634effd700 1 --2- 172.21.15.144:0/1754042691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6350100d70 0x7f6350196280 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f63400092e0 tx=0x7f634000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:51:30.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.421+0000 7f634cff9700 1 -- 172.21.15.144:0/1754042691 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f634001d070 con 0x7f6350100d70 2024-09-10T11:51:30.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.421+0000 7f634cff9700 1 -- 172.21.15.144:0/1754042691 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f634000f460 con 0x7f6350100d70 2024-09-10T11:51:30.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.421+0000 7f635622c700 1 -- 172.21.15.144:0/1754042691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f635019a970 con 0x7f6350100d70 2024-09-10T11:51:30.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.421+0000 7f634cff9700 1 -- 172.21.15.144:0/1754042691 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63400175f0 con 0x7f6350100d70 2024-09-10T11:51:30.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.421+0000 7f635622c700 1 -- 172.21.15.144:0/1754042691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f635019ae60 con 0x7f6350100d70 2024-09-10T11:51:30.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.422+0000 7f635622c700 1 -- 172.21.15.144:0/1754042691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6350108740 con 0x7f6350100d70 2024-09-10T11:51:30.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.422+0000 7f634cff9700 1 -- 172.21.15.144:0/1754042691 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f634000f5d0 con 0x7f6350100d70 2024-09-10T11:51:30.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.423+0000 7f634cff9700 1 --2- 172.21.15.144:0/1754042691 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f633806e200 0x7f63380706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:30.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.423+0000 7f634cff9700 1 -- 172.21.15.144:0/1754042691 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f634008f4e0 con 0x7f6350100d70 2024-09-10T11:51:30.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.423+0000 7f634f7fe700 1 --2- 172.21.15.144:0/1754042691 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f633806e200 0x7f63380706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:30.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.423+0000 7f634f7fe700 1 --2- 172.21.15.144:0/1754042691 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f633806e200 0x7f63380706b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6344005950 tx=0x7f6344005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:51:30.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.427+0000 7f634cff9700 1 -- 172.21.15.144:0/1754042691 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6340058940 con 0x7f6350100d70 2024-09-10T11:51:30.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.633+0000 7f635622c700 1 -- 172.21.15.144:0/1754042691 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f635002cd20 con 0x7f633806e200 2024-09-10T11:51:30.643 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.642+0000 7f634cff9700 1 -- 172.21.15.144:0/1754042691 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f635002cd20 con 0x7f633806e200 2024-09-10T11:51:30.644 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:51:30.644 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (19m) 13m ago 30m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:51:30.644 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (30m) 13m ago 30m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:51:30.644 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (29m) 13m ago 29m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:51:30.644 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (17m) 13m ago 30m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:51:30.644 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (17m) 13m ago 29m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:51:30.644 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (19m) 13m ago 30m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:51:30.644 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (13m) 13m ago 24m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:51:30.644 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (13m) 13m ago 24m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:51:30.644 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (24m) 13m ago 24m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:51:30.644 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (13m) 13m ago 24m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:51:30.645 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (21m) 13m ago 31m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:51:30.645 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (20m) 13m ago 29m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:51:30.645 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (18m) 13m ago 31m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:51:30.645 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (18m) 13m ago 29m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:51:30.645 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (20m) 13m ago 30m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:51:30.645 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (20m) 13m ago 29m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:51:30.645 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (16m) 13m ago 28m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:51:30.645 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (16m) 13m ago 28m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:51:30.645 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (15m) 13m ago 27m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:51:30.645 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (15m) 13m ago 27m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:51:30.645 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (14m) 13m ago 26m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:51:30.645 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (14m) 13m ago 26m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:51:30.645 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (20m) 13m ago 29m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:51:30.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.645+0000 7f635622c700 1 -- 172.21.15.144:0/1754042691 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f633806e200 msgr2=0x7f63380706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:30.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.645+0000 7f635622c700 1 --2- 172.21.15.144:0/1754042691 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f633806e200 0x7f63380706b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6344005950 tx=0x7f6344005cf0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.645+0000 7f635622c700 1 -- 172.21.15.144:0/1754042691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6350100d70 msgr2=0x7f6350196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:30.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.645+0000 7f635622c700 1 --2- 172.21.15.144:0/1754042691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6350100d70 0x7f6350196280 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f63400092e0 tx=0x7f634000be20 comp rx=0 tx=0).stop 2024-09-10T11:51:30.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.645+0000 7f635622c700 1 -- 172.21.15.144:0/1754042691 shutdown_connections 2024-09-10T11:51:30.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.645+0000 7f635622c700 1 --2- 172.21.15.144:0/1754042691 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f633806e200 0x7f63380706b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.645+0000 7f635622c700 1 --2- 172.21.15.144:0/1754042691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63501003d0 0x7f6350195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.645+0000 7f635622c700 1 --2- 172.21.15.144:0/1754042691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6350100d70 0x7f6350196280 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.645+0000 7f635622c700 1 -- 172.21.15.144:0/1754042691 >> 172.21.15.144:0/1754042691 conn(0x7f63500fbc90 msgr2=0x7f63500fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:51:30.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.645+0000 7f635622c700 1 -- 172.21.15.144:0/1754042691 shutdown_connections 2024-09-10T11:51:30.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.645+0000 7f635622c700 1 -- 172.21.15.144:0/1754042691 wait complete. 2024-09-10T11:51:30.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.748+0000 7f81e0ba8700 1 -- 172.21.15.144:0/949425309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81dc101360 msgr2=0x7f81dc101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:30.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.748+0000 7f81e0ba8700 1 --2- 172.21.15.144:0/949425309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81dc101360 0x7f81dc101730 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009940 tx=0x7f81cc009c50 comp rx=0 tx=0).stop 2024-09-10T11:51:30.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.748+0000 7f81e0ba8700 1 -- 172.21.15.144:0/949425309 shutdown_connections 2024-09-10T11:51:30.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.748+0000 7f81e0ba8700 1 --2- 172.21.15.144:0/949425309 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81dc101c70 0x7f81dc10a170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.748+0000 7f81e0ba8700 1 --2- 172.21.15.144:0/949425309 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81dc101360 0x7f81dc101730 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.748+0000 7f81e0ba8700 1 -- 172.21.15.144:0/949425309 >> 172.21.15.144:0/949425309 conn(0x7f81dc0faba0 msgr2=0x7f81dc0fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:51:30.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.748+0000 7f81e0ba8700 1 -- 172.21.15.144:0/949425309 shutdown_connections 2024-09-10T11:51:30.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.748+0000 7f81e0ba8700 1 -- 172.21.15.144:0/949425309 wait complete. 2024-09-10T11:51:30.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.748+0000 7f81e0ba8700 1 Processor -- start 2024-09-10T11:51:30.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.749+0000 7f81e0ba8700 1 -- start start 2024-09-10T11:51:30.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.749+0000 7f81e0ba8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81dc101360 0x7f81dc0fefc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:30.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.749+0000 7f81e0ba8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81dc101c70 0x7f81dc0ff500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:30.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.749+0000 7f81e0ba8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc0ffa40 con 0x7f81dc101360 2024-09-10T11:51:30.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.749+0000 7f81e0ba8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc0ffb80 con 0x7f81dc101c70 2024-09-10T11:51:30.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.749+0000 7f81da59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81dc101360 0x7f81dc0fefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:30.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.749+0000 7f81da59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81dc101360 0x7f81dc0fefc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36484/0 (socket says 172.21.15.144:36484) 2024-09-10T11:51:30.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.749+0000 7f81da59c700 1 -- 172.21.15.144:0/732912409 learned_addr learned my addr 172.21.15.144:0/732912409 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:51:30.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.749+0000 7f81d1d9b700 1 --2- 172.21.15.144:0/732912409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81dc101c70 0x7f81dc0ff500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:30.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.749+0000 7f81da59c700 1 -- 172.21.15.144:0/732912409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81dc101c70 msgr2=0x7f81dc0ff500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:30.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.749+0000 7f81da59c700 1 --2- 172.21.15.144:0/732912409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81dc101c70 0x7f81dc0ff500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:30.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.749+0000 7f81da59c700 1 -- 172.21.15.144:0/732912409 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc0095f0 con 0x7f81dc101360 2024-09-10T11:51:30.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.749+0000 7f81da59c700 1 --2- 172.21.15.144:0/732912409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81dc101360 0x7f81dc0fefc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00b580 tx=0x7f81cc0055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:51:30.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.750+0000 7f81d3fff700 1 -- 172.21.15.144:0/732912409 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81cc01d070 con 0x7f81dc101360 2024-09-10T11:51:30.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.750+0000 7f81d3fff700 1 -- 172.21.15.144:0/732912409 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f81cc005dc0 con 0x7f81dc101360 2024-09-10T11:51:30.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.750+0000 7f81e0ba8700 1 -- 172.21.15.144:0/732912409 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81dc0ffe00 con 0x7f81dc101360 2024-09-10T11:51:30.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.750+0000 7f81d3fff700 1 -- 172.21.15.144:0/732912409 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81cc021410 con 0x7f81dc101360 2024-09-10T11:51:30.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.750+0000 7f81e0ba8700 1 -- 172.21.15.144:0/732912409 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81dc1a2210 con 0x7f81dc101360 2024-09-10T11:51:30.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.751+0000 7f81e0ba8700 1 -- 172.21.15.144:0/732912409 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81dc107840 con 0x7f81dc101360 2024-09-10T11:51:30.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.751+0000 7f81d3fff700 1 -- 172.21.15.144:0/732912409 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f81cc00f7f0 con 0x7f81dc101360 2024-09-10T11:51:30.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.752+0000 7f81d3fff700 1 --2- 172.21.15.144:0/732912409 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f81c806e200 0x7f81c80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:30.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.752+0000 7f81d3fff700 1 -- 172.21.15.144:0/732912409 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f81cc08f360 con 0x7f81dc101360 2024-09-10T11:51:30.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.752+0000 7f81d1d9b700 1 --2- 172.21.15.144:0/732912409 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f81c806e200 0x7f81c80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:30.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.752+0000 7f81d1d9b700 1 --2- 172.21.15.144:0/732912409 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f81c806e200 0x7f81c80706b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f81dc1007b0 tx=0x7f81c40073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:51:30.757 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:30.756+0000 7f81d3fff700 1 -- 172.21.15.144:0/732912409 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f81cc0587a0 con 0x7f81dc101360 2024-09-10T11:51:31.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.044+0000 7f81e0ba8700 1 -- 172.21.15.144:0/732912409 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f81dc04ea50 con 0x7f81dc101360 2024-09-10T11:51:31.046 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:30 smithi144 ceph-mon[87436]: pgmap v585: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:31.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.045+0000 7f81d3fff700 1 -- 172.21.15.144:0/732912409 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f81cc05be60 con 0x7f81dc101360 2024-09-10T11:51:31.046 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:51:31.046 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:51:31.046 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:51:31.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:51:31.046 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:51:31.047 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:51:31.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:51:31.047 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:51:31.047 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:51:31.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:51:31.047 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:51:31.047 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:51:31.047 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:51:31.047 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:51:31.047 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:51:31.047 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:51:31.047 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:51:31.047 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:51:31.047 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:51:31.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.047+0000 7f81e0ba8700 1 -- 172.21.15.144:0/732912409 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f81c806e200 msgr2=0x7f81c80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:31.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.047+0000 7f81e0ba8700 1 --2- 172.21.15.144:0/732912409 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f81c806e200 0x7f81c80706b0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f81dc1007b0 tx=0x7f81c40073d0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.047+0000 7f81e0ba8700 1 -- 172.21.15.144:0/732912409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81dc101360 msgr2=0x7f81dc0fefc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:31.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.047+0000 7f81e0ba8700 1 --2- 172.21.15.144:0/732912409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81dc101360 0x7f81dc0fefc0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00b580 tx=0x7f81cc0055f0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.047+0000 7f81e0ba8700 1 -- 172.21.15.144:0/732912409 shutdown_connections 2024-09-10T11:51:31.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.047+0000 7f81e0ba8700 1 --2- 172.21.15.144:0/732912409 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f81c806e200 0x7f81c80706b0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.047+0000 7f81e0ba8700 1 --2- 172.21.15.144:0/732912409 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81dc101360 0x7f81dc0fefc0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.047+0000 7f81e0ba8700 1 --2- 172.21.15.144:0/732912409 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81dc101c70 0x7f81dc0ff500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.047+0000 7f81e0ba8700 1 -- 172.21.15.144:0/732912409 >> 172.21.15.144:0/732912409 conn(0x7f81dc0faba0 msgr2=0x7f81dc104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:51:31.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.047+0000 7f81e0ba8700 1 -- 172.21.15.144:0/732912409 shutdown_connections 2024-09-10T11:51:31.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.047+0000 7f81e0ba8700 1 -- 172.21.15.144:0/732912409 wait complete. 2024-09-10T11:51:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:30 smithi178 ceph-mon[65940]: pgmap v585: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:31.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.152+0000 7f7f2e529700 1 -- 172.21.15.144:0/416511154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f280ffba0 msgr2=0x7f7f2810a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:31.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.152+0000 7f7f2e529700 1 --2- 172.21.15.144:0/416511154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f280ffba0 0x7f7f2810a610 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c009910 tx=0x7f7f1c009c20 comp rx=0 tx=0).stop 2024-09-10T11:51:31.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.152+0000 7f7f2e529700 1 -- 172.21.15.144:0/416511154 shutdown_connections 2024-09-10T11:51:31.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.152+0000 7f7f2e529700 1 --2- 172.21.15.144:0/416511154 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f280ffba0 0x7f7f2810a610 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.152+0000 7f7f2e529700 1 --2- 172.21.15.144:0/416511154 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f280ff290 0x7f7f280ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.152+0000 7f7f2e529700 1 -- 172.21.15.144:0/416511154 >> 172.21.15.144:0/416511154 conn(0x7f7f28074850 msgr2=0x7f7f28074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:51:31.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.152+0000 7f7f2e529700 1 -- 172.21.15.144:0/416511154 shutdown_connections 2024-09-10T11:51:31.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.152+0000 7f7f2e529700 1 -- 172.21.15.144:0/416511154 wait complete. 2024-09-10T11:51:31.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.153+0000 7f7f2e529700 1 Processor -- start 2024-09-10T11:51:31.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.153+0000 7f7f2e529700 1 -- start start 2024-09-10T11:51:31.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.153+0000 7f7f2e529700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f280ff290 0x7f7f28195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:31.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.153+0000 7f7f2e529700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f280ffba0 0x7f7f281962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:31.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.153+0000 7f7f2e529700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f28196980 con 0x7f7f280ffba0 2024-09-10T11:51:31.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.153+0000 7f7f2e529700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2819a710 con 0x7f7f280ff290 2024-09-10T11:51:31.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.153+0000 7f7f277fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f280ffba0 0x7f7f281962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:31.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.153+0000 7f7f277fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f280ffba0 0x7f7f281962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36506/0 (socket says 172.21.15.144:36506) 2024-09-10T11:51:31.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.154+0000 7f7f27fff700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f280ff290 0x7f7f28195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:31.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.154+0000 7f7f277fe700 1 -- 172.21.15.144:0/2490956234 learned_addr learned my addr 172.21.15.144:0/2490956234 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:51:31.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.154+0000 7f7f277fe700 1 -- 172.21.15.144:0/2490956234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f280ff290 msgr2=0x7f7f28195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:31.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.154+0000 7f7f277fe700 1 --2- 172.21.15.144:0/2490956234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f280ff290 0x7f7f28195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.154+0000 7f7f277fe700 1 -- 172.21.15.144:0/2490956234 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c0095f0 con 0x7f7f280ffba0 2024-09-10T11:51:31.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.154+0000 7f7f277fe700 1 --2- 172.21.15.144:0/2490956234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f280ffba0 0x7f7f281962a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006010 tx=0x7f7f1c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:51:31.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.154+0000 7f7f257fa700 1 -- 172.21.15.144:0/2490956234 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f1c01d070 con 0x7f7f280ffba0 2024-09-10T11:51:31.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.154+0000 7f7f257fa700 1 -- 172.21.15.144:0/2490956234 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f1c005dc0 con 0x7f7f280ffba0 2024-09-10T11:51:31.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.154+0000 7f7f2e529700 1 -- 172.21.15.144:0/2490956234 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f2819a990 con 0x7f7f280ffba0 2024-09-10T11:51:31.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.154+0000 7f7f257fa700 1 -- 172.21.15.144:0/2490956234 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f1c017400 con 0x7f7f280ffba0 2024-09-10T11:51:31.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.154+0000 7f7f2e529700 1 -- 172.21.15.144:0/2490956234 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f2819ae80 con 0x7f7f280ffba0 2024-09-10T11:51:31.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.155+0000 7f7f2e529700 1 -- 172.21.15.144:0/2490956234 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f28107d10 con 0x7f7f280ffba0 2024-09-10T11:51:31.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.156+0000 7f7f257fa700 1 -- 172.21.15.144:0/2490956234 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7f1c003a60 con 0x7f7f280ffba0 2024-09-10T11:51:31.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.156+0000 7f7f257fa700 1 --2- 172.21.15.144:0/2490956234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f1006e200 0x7f7f100706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:31.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.156+0000 7f7f257fa700 1 -- 172.21.15.144:0/2490956234 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7f1c08f500 con 0x7f7f280ffba0 2024-09-10T11:51:31.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.156+0000 7f7f27fff700 1 --2- 172.21.15.144:0/2490956234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f1006e200 0x7f7f100706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:31.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.157+0000 7f7f27fff700 1 --2- 172.21.15.144:0/2490956234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f1006e200 0x7f7f100706b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7f2806a5c0 tx=0x7f7f1800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:51:31.161 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.161+0000 7f7f257fa700 1 -- 172.21.15.144:0/2490956234 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7f1c0588e0 con 0x7f7f280ffba0 2024-09-10T11:51:31.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.405+0000 7f7f2e529700 1 -- 172.21.15.144:0/2490956234 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7f2804ea50 con 0x7f7f280ffba0 2024-09-10T11:51:31.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.405+0000 7f7f257fa700 1 -- 172.21.15.144:0/2490956234 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f7f1c05bfa0 con 0x7f7f280ffba0 2024-09-10T11:51:31.407 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:51:31.407 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:51:31.407 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:51:31.407 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:51:31.407 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:51:31.407 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:51:31.407 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:51:31.407 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:51:31.407 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:51:31.408 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:51:31.408 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:51:31.408 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:51:31.408 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:51:31.408 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:51:31.408 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:51:31.408 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:51:31.408 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:51:31.408 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:51:31.408 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:51:31.408 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:51:31.408 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:51:31.409 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:51:31.409 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:51:31.409 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:51:31.409 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:51:31.409 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:51:31.409 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:51:31.409 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:51:31.409 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:51:31.409 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:51:31.409 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:51:31.409 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:51:31.409 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:51:31.409 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:51:31.409 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:51:31.409 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:51:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:51:31.410 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:51:31.410 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:51:31.410 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:51:31.410 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:51:31.410 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:51:31.410 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:51:31.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.408+0000 7f7f2e529700 1 -- 172.21.15.144:0/2490956234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f1006e200 msgr2=0x7f7f100706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:31.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.408+0000 7f7f2e529700 1 --2- 172.21.15.144:0/2490956234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f1006e200 0x7f7f100706b0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f7f2806a5c0 tx=0x7f7f1800b410 comp rx=0 tx=0).stop 2024-09-10T11:51:31.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.408+0000 7f7f2e529700 1 -- 172.21.15.144:0/2490956234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f280ffba0 msgr2=0x7f7f281962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:31.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.408+0000 7f7f2e529700 1 --2- 172.21.15.144:0/2490956234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f280ffba0 0x7f7f281962a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c006010 tx=0x7f7f1c00bd70 comp rx=0 tx=0).stop 2024-09-10T11:51:31.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.408+0000 7f7f2e529700 1 -- 172.21.15.144:0/2490956234 shutdown_connections 2024-09-10T11:51:31.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.408+0000 7f7f2e529700 1 --2- 172.21.15.144:0/2490956234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f1006e200 0x7f7f100706b0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.408+0000 7f7f2e529700 1 --2- 172.21.15.144:0/2490956234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f280ff290 0x7f7f28195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.408+0000 7f7f2e529700 1 --2- 172.21.15.144:0/2490956234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f280ffba0 0x7f7f281962a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.408+0000 7f7f2e529700 1 -- 172.21.15.144:0/2490956234 >> 172.21.15.144:0/2490956234 conn(0x7f7f28074850 msgr2=0x7f7f28104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:51:31.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.408+0000 7f7f2e529700 1 -- 172.21.15.144:0/2490956234 shutdown_connections 2024-09-10T11:51:31.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.408+0000 7f7f2e529700 1 -- 172.21.15.144:0/2490956234 wait complete. 2024-09-10T11:51:31.412 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:51:31.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.514+0000 7f09e5ab3700 1 -- 172.21.15.144:0/4070451406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e01003d0 msgr2=0x7f09e01007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:31.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.514+0000 7f09e5ab3700 1 --2- 172.21.15.144:0/4070451406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e01003d0 0x7f09e01007a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f09d000b0a0 tx=0x7f09d000b3b0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.514+0000 7f09e5ab3700 1 -- 172.21.15.144:0/4070451406 shutdown_connections 2024-09-10T11:51:31.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.514+0000 7f09e5ab3700 1 --2- 172.21.15.144:0/4070451406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09e0100d70 0x7f09e0104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.514+0000 7f09e5ab3700 1 --2- 172.21.15.144:0/4070451406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e01003d0 0x7f09e01007a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.514+0000 7f09e5ab3700 1 -- 172.21.15.144:0/4070451406 >> 172.21.15.144:0/4070451406 conn(0x7f09e00fbc90 msgr2=0x7f09e00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:51:31.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.514+0000 7f09e5ab3700 1 -- 172.21.15.144:0/4070451406 shutdown_connections 2024-09-10T11:51:31.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.514+0000 7f09e5ab3700 1 -- 172.21.15.144:0/4070451406 wait complete. 2024-09-10T11:51:31.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.514+0000 7f09e5ab3700 1 Processor -- start 2024-09-10T11:51:31.519 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.515+0000 7f09e5ab3700 1 -- start start 2024-09-10T11:51:31.519 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.515+0000 7f09e5ab3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e01003d0 0x7f09e0195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:31.519 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.515+0000 7f09e5ab3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09e0100d70 0x7f09e01962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:31.519 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.515+0000 7f09e5ab3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e0196980 con 0x7f09e01003d0 2024-09-10T11:51:31.519 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.515+0000 7f09e5ab3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09e019a710 con 0x7f09e0100d70 2024-09-10T11:51:31.519 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.515+0000 7f09df7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e01003d0 0x7f09e0195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:31.519 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.515+0000 7f09df7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e01003d0 0x7f09e0195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36520/0 (socket says 172.21.15.144:36520) 2024-09-10T11:51:31.519 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.515+0000 7f09df7fe700 1 -- 172.21.15.144:0/3861599658 learned_addr learned my addr 172.21.15.144:0/3861599658 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:51:31.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.515+0000 7f09df7fe700 1 -- 172.21.15.144:0/3861599658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09e0100d70 msgr2=0x7f09e01962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:31.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.515+0000 7f09deffd700 1 --2- 172.21.15.144:0/3861599658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09e0100d70 0x7f09e01962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:31.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.515+0000 7f09df7fe700 1 --2- 172.21.15.144:0/3861599658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09e0100d70 0x7f09e01962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.515+0000 7f09df7fe700 1 -- 172.21.15.144:0/3861599658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d0009d00 con 0x7f09e01003d0 2024-09-10T11:51:31.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.515+0000 7f09df7fe700 1 --2- 172.21.15.144:0/3861599658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e01003d0 0x7f09e0195d60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f09d0000f80 tx=0x7f09d000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:51:31.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.516+0000 7f09dcff9700 1 -- 172.21.15.144:0/3861599658 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09d000e040 con 0x7f09e01003d0 2024-09-10T11:51:31.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.516+0000 7f09dcff9700 1 -- 172.21.15.144:0/3861599658 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f09d0003de0 con 0x7f09e01003d0 2024-09-10T11:51:31.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.516+0000 7f09e5ab3700 1 -- 172.21.15.144:0/3861599658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09e019a990 con 0x7f09e01003d0 2024-09-10T11:51:31.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.516+0000 7f09dcff9700 1 -- 172.21.15.144:0/3861599658 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09d001d790 con 0x7f09e01003d0 2024-09-10T11:51:31.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.516+0000 7f09e5ab3700 1 -- 172.21.15.144:0/3861599658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09e019ae80 con 0x7f09e01003d0 2024-09-10T11:51:31.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.517+0000 7f09e5ab3700 1 -- 172.21.15.144:0/3861599658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09e01086e0 con 0x7f09e01003d0 2024-09-10T11:51:31.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.517+0000 7f09dcff9700 1 -- 172.21.15.144:0/3861599658 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f09d0004040 con 0x7f09e01003d0 2024-09-10T11:51:31.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.518+0000 7f09dcff9700 1 --2- 172.21.15.144:0/3861599658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f09c806e200 0x7f09c80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:31.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.518+0000 7f09dcff9700 1 -- 172.21.15.144:0/3861599658 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f09d008f7e0 con 0x7f09e01003d0 2024-09-10T11:51:31.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.518+0000 7f09deffd700 1 --2- 172.21.15.144:0/3861599658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f09c806e200 0x7f09c80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:31.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.518+0000 7f09deffd700 1 --2- 172.21.15.144:0/3861599658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f09c806e200 0x7f09c80706b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f09d4005950 tx=0x7f09d40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:51:31.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.522+0000 7f09dcff9700 1 -- 172.21.15.144:0/3861599658 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f09d0058c40 con 0x7f09e01003d0 2024-09-10T11:51:31.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.745+0000 7f09e5ab3700 1 -- 172.21.15.144:0/3861599658 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f09e019b220 con 0x7f09c806e200 2024-09-10T11:51:31.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.746+0000 7f09dcff9700 1 -- 172.21.15.144:0/3861599658 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f09e019b220 con 0x7f09c806e200 2024-09-10T11:51:31.747 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:51:31.747 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:51:31.747 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:51:31.747 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:51:31.747 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:51:31.747 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:51:31.747 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:51:31.747 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:51:31.748 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:51:31.748 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:51:31.748 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:51:31.748 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:51:31.748 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:51:31.748 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:51:31.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.748+0000 7f09e5ab3700 1 -- 172.21.15.144:0/3861599658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f09c806e200 msgr2=0x7f09c80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:31.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.748+0000 7f09e5ab3700 1 --2- 172.21.15.144:0/3861599658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f09c806e200 0x7f09c80706b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f09d4005950 tx=0x7f09d40058e0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.748+0000 7f09e5ab3700 1 -- 172.21.15.144:0/3861599658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e01003d0 msgr2=0x7f09e0195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:31.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.748+0000 7f09e5ab3700 1 --2- 172.21.15.144:0/3861599658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e01003d0 0x7f09e0195d60 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f09d0000f80 tx=0x7f09d000cdd0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.748+0000 7f09e5ab3700 1 -- 172.21.15.144:0/3861599658 shutdown_connections 2024-09-10T11:51:31.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.748+0000 7f09e5ab3700 1 --2- 172.21.15.144:0/3861599658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f09c806e200 0x7f09c80706b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.748+0000 7f09e5ab3700 1 --2- 172.21.15.144:0/3861599658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f09e01003d0 0x7f09e0195d60 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.748+0000 7f09e5ab3700 1 --2- 172.21.15.144:0/3861599658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f09e0100d70 0x7f09e01962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.748+0000 7f09e5ab3700 1 -- 172.21.15.144:0/3861599658 >> 172.21.15.144:0/3861599658 conn(0x7f09e00fbc90 msgr2=0x7f09e00fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:51:31.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.748+0000 7f09e5ab3700 1 -- 172.21.15.144:0/3861599658 shutdown_connections 2024-09-10T11:51:31.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.748+0000 7f09e5ab3700 1 -- 172.21.15.144:0/3861599658 wait complete. 2024-09-10T11:51:31.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.852+0000 7fa3f8031700 1 -- 172.21.15.144:0/754570606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f00ffc60 msgr2=0x7fa3f0103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:31.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.852+0000 7fa3f8031700 1 --2- 172.21.15.144:0/754570606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f00ffc60 0x7fa3f0103c30 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec009910 tx=0x7fa3ec009c20 comp rx=0 tx=0).stop 2024-09-10T11:51:31.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.852+0000 7fa3f8031700 1 -- 172.21.15.144:0/754570606 shutdown_connections 2024-09-10T11:51:31.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.852+0000 7fa3f8031700 1 --2- 172.21.15.144:0/754570606 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f00ffc60 0x7fa3f0103c30 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.852+0000 7fa3f8031700 1 --2- 172.21.15.144:0/754570606 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f00ff2c0 0x7fa3f00ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.852+0000 7fa3f8031700 1 -- 172.21.15.144:0/754570606 >> 172.21.15.144:0/754570606 conn(0x7fa3f00fab80 msgr2=0x7fa3f00fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:51:31.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.852+0000 7fa3f8031700 1 -- 172.21.15.144:0/754570606 shutdown_connections 2024-09-10T11:51:31.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.852+0000 7fa3f8031700 1 -- 172.21.15.144:0/754570606 wait complete. 2024-09-10T11:51:31.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.853+0000 7fa3f8031700 1 Processor -- start 2024-09-10T11:51:31.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.853+0000 7fa3f8031700 1 -- start start 2024-09-10T11:51:31.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.853+0000 7fa3f8031700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f00ff2c0 0x7fa3f0100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:31.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.853+0000 7fa3f8031700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f00ffc60 0x7fa3f0101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:31.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.853+0000 7fa3f8031700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f0101750 con 0x7fa3f00ffc60 2024-09-10T11:51:31.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.853+0000 7fa3f8031700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3f0101890 con 0x7fa3f00ff2c0 2024-09-10T11:51:31.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.853+0000 7fa3f55cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f00ffc60 0x7fa3f0101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:31.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.853+0000 7fa3f55cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f00ffc60 0x7fa3f0101210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36536/0 (socket says 172.21.15.144:36536) 2024-09-10T11:51:31.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.853+0000 7fa3f55cc700 1 -- 172.21.15.144:0/2239100206 learned_addr learned my addr 172.21.15.144:0/2239100206 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:51:31.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.853+0000 7fa3f5dcd700 1 --2- 172.21.15.144:0/2239100206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f00ff2c0 0x7fa3f0100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:31.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.853+0000 7fa3f55cc700 1 -- 172.21.15.144:0/2239100206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f00ff2c0 msgr2=0x7fa3f0100cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:31.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.853+0000 7fa3f55cc700 1 --2- 172.21.15.144:0/2239100206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f00ff2c0 0x7fa3f0100cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:31.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.853+0000 7fa3f55cc700 1 -- 172.21.15.144:0/2239100206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec0095f0 con 0x7fa3f00ffc60 2024-09-10T11:51:31.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.854+0000 7fa3f55cc700 1 --2- 172.21.15.144:0/2239100206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f00ffc60 0x7fa3f0101210 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00bda0 tx=0x7fa3ec00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:51:31.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.854+0000 7fa3e6ffd700 1 -- 172.21.15.144:0/2239100206 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3ec01d070 con 0x7fa3f00ffc60 2024-09-10T11:51:31.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.854+0000 7fa3e6ffd700 1 -- 172.21.15.144:0/2239100206 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa3ec004fb0 con 0x7fa3f00ffc60 2024-09-10T11:51:31.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.854+0000 7fa3e6ffd700 1 -- 172.21.15.144:0/2239100206 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3ec00f480 con 0x7fa3f00ffc60 2024-09-10T11:51:31.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.854+0000 7fa3f8031700 1 -- 172.21.15.144:0/2239100206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3f0101b10 con 0x7fa3f00ffc60 2024-09-10T11:51:31.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.854+0000 7fa3f8031700 1 -- 172.21.15.144:0/2239100206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3f01a21e0 con 0x7fa3f00ffc60 2024-09-10T11:51:31.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.855+0000 7fa3f8031700 1 -- 172.21.15.144:0/2239100206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3f0108640 con 0x7fa3f00ffc60 2024-09-10T11:51:31.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.855+0000 7fa3e6ffd700 1 -- 172.21.15.144:0/2239100206 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa3ec022870 con 0x7fa3f00ffc60 2024-09-10T11:51:31.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.856+0000 7fa3e6ffd700 1 --2- 172.21.15.144:0/2239100206 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3dc06e200 0x7fa3dc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:51:31.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.856+0000 7fa3e6ffd700 1 -- 172.21.15.144:0/2239100206 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa3ec08f630 con 0x7fa3f00ffc60 2024-09-10T11:51:31.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.856+0000 7fa3f5dcd700 1 --2- 172.21.15.144:0/2239100206 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3dc06e200 0x7fa3dc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:51:31.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.856+0000 7fa3f5dcd700 1 --2- 172.21.15.144:0/2239100206 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3dc06e200 0x7fa3dc0706b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0005ea0 tx=0x7fa3e0005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:51:31.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:31.860+0000 7fa3e6ffd700 1 -- 172.21.15.144:0/2239100206 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa3ec058c60 con 0x7fa3f00ffc60 2024-09-10T11:51:32.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:32.145+0000 7fa3f8031700 1 -- 172.21.15.144:0/2239100206 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa3f0108020 con 0x7fa3f00ffc60 2024-09-10T11:51:32.147 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:31 smithi144 ceph-mon[87436]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:51:32.147 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:31 smithi144 ceph-mon[87436]: from='client.35104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:51:32.147 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:31 smithi144 ceph-mon[87436]: from='client.35108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:51:32.147 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:31 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/732912409' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:51:32.147 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:31 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2490956234' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:51:32.147 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:32.145+0000 7fa3e6ffd700 1 -- 172.21.15.144:0/2239100206 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fa3ec05c320 con 0x7fa3f00ffc60 2024-09-10T11:51:32.148 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:51:32.148 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:51:32.148 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:51:32.148 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:51:32.148 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:51:32.148 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:51:32.148 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:51:32.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:32.149+0000 7fa3f8031700 1 -- 172.21.15.144:0/2239100206 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3dc06e200 msgr2=0x7fa3dc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:32.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:32.149+0000 7fa3f8031700 1 --2- 172.21.15.144:0/2239100206 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3dc06e200 0x7fa3dc0706b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa3e0005ea0 tx=0x7fa3e0005e30 comp rx=0 tx=0).stop 2024-09-10T11:51:32.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:32.149+0000 7fa3f8031700 1 -- 172.21.15.144:0/2239100206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f00ffc60 msgr2=0x7fa3f0101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:51:32.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:32.149+0000 7fa3f8031700 1 --2- 172.21.15.144:0/2239100206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f00ffc60 0x7fa3f0101210 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec00bda0 tx=0x7fa3ec00be80 comp rx=0 tx=0).stop 2024-09-10T11:51:32.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:32.149+0000 7fa3f8031700 1 -- 172.21.15.144:0/2239100206 shutdown_connections 2024-09-10T11:51:32.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:32.149+0000 7fa3f8031700 1 --2- 172.21.15.144:0/2239100206 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa3dc06e200 0x7fa3dc0706b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:32.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:32.149+0000 7fa3f8031700 1 --2- 172.21.15.144:0/2239100206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa3f00ff2c0 0x7fa3f0100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:32.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:32.149+0000 7fa3f8031700 1 --2- 172.21.15.144:0/2239100206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa3f00ffc60 0x7fa3f0101210 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:51:32.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:32.149+0000 7fa3f8031700 1 -- 172.21.15.144:0/2239100206 >> 172.21.15.144:0/2239100206 conn(0x7fa3f00fab80 msgr2=0x7fa3f0068f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:51:32.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:32.149+0000 7fa3f8031700 1 -- 172.21.15.144:0/2239100206 shutdown_connections 2024-09-10T11:51:32.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:51:32.149+0000 7fa3f8031700 1 -- 172.21.15.144:0/2239100206 wait complete. 2024-09-10T11:51:32.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:31 smithi178 ceph-mon[65940]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:51:32.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:31 smithi178 ceph-mon[65940]: from='client.35104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:51:32.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:31 smithi178 ceph-mon[65940]: from='client.35108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:51:32.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:31 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/732912409' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:51:32.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:31 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2490956234' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:51:33.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:32 smithi144 ceph-mon[87436]: from='client.35120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:51:33.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:32 smithi144 ceph-mon[87436]: pgmap v586: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:33.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:32 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2239100206' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:51:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:32 smithi178 ceph-mon[65940]: from='client.35120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:51:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:32 smithi178 ceph-mon[65940]: pgmap v586: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:32 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2239100206' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:51:34.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:33 smithi144 ceph-mon[87436]: pgmap v587: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:34.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:33 smithi178 ceph-mon[65940]: pgmap v587: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:36 smithi144 ceph-mon[87436]: pgmap v588: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:36 smithi178 ceph-mon[65940]: pgmap v588: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:38.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:38 smithi144 ceph-mon[87436]: pgmap v589: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:38.866 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:38 smithi178 ceph-mon[65940]: pgmap v589: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:40 smithi178 ceph-mon[65940]: pgmap v590: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:40 smithi144 ceph-mon[87436]: pgmap v590: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:43.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:42 smithi144 ceph-mon[87436]: pgmap v591: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:42 smithi178 ceph-mon[65940]: pgmap v591: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:44.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:51:44.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:51:45.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:44 smithi144 ceph-mon[87436]: pgmap v592: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:44 smithi178 ceph-mon[65940]: pgmap v592: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:46.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:45 smithi178 ceph-mon[65940]: pgmap v593: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:46.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:45 smithi144 ceph-mon[87436]: pgmap v593: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:49.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:48 smithi144 ceph-mon[87436]: pgmap v594: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:48 smithi178 ceph-mon[65940]: pgmap v594: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:50 smithi144 ceph-mon[87436]: pgmap v595: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:50 smithi178 ceph-mon[65940]: pgmap v595: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:52.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:51 smithi178 ceph-mon[65940]: pgmap v596: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:52.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:51 smithi144 ceph-mon[87436]: pgmap v596: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:55.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:54 smithi144 ceph-mon[87436]: pgmap v597: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:54 smithi178 ceph-mon[65940]: pgmap v597: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:56 smithi144 ceph-mon[87436]: pgmap v598: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:56 smithi178 ceph-mon[65940]: pgmap v598: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:58.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:57 smithi144 ceph-mon[87436]: pgmap v599: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:57 smithi178 ceph-mon[65940]: pgmap v599: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:51:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:51:59.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:52:00.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:51:59 smithi144 ceph-mon[87436]: pgmap v600: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:00.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:51:59 smithi178 ceph-mon[65940]: pgmap v600: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:02.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.268+0000 7fa42a248700 1 -- 172.21.15.144:0/3713710634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4240ff2d0 msgr2=0x7fa4240ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:02.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.268+0000 7fa42a248700 1 --2- 172.21.15.144:0/3713710634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4240ff2d0 0x7fa4240ff6a0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa414009910 tx=0x7fa414009c20 comp rx=0 tx=0).stop 2024-09-10T11:52:02.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.268+0000 7fa42a248700 1 -- 172.21.15.144:0/3713710634 shutdown_connections 2024-09-10T11:52:02.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.268+0000 7fa42a248700 1 --2- 172.21.15.144:0/3713710634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4240ffc70 0x7fa424103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:02.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.268+0000 7fa42a248700 1 --2- 172.21.15.144:0/3713710634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4240ff2d0 0x7fa4240ff6a0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:02.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.268+0000 7fa42a248700 1 -- 172.21.15.144:0/3713710634 >> 172.21.15.144:0/3713710634 conn(0x7fa4240fabb0 msgr2=0x7fa4240fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:02.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.268+0000 7fa42a248700 1 -- 172.21.15.144:0/3713710634 shutdown_connections 2024-09-10T11:52:02.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.268+0000 7fa42a248700 1 -- 172.21.15.144:0/3713710634 wait complete. 2024-09-10T11:52:02.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.269+0000 7fa42a248700 1 Processor -- start 2024-09-10T11:52:02.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.269+0000 7fa42a248700 1 -- start start 2024-09-10T11:52:02.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.269+0000 7fa42a248700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4240ff2d0 0x7fa424197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:02.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.269+0000 7fa42a248700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4240ffc70 0x7fa4241984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:02.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.269+0000 7fa42a248700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa424198b20 con 0x7fa4240ffc70 2024-09-10T11:52:02.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.269+0000 7fa42a248700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa424198c60 con 0x7fa4240ff2d0 2024-09-10T11:52:02.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.269+0000 7fa41bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4240ffc70 0x7fa4241984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:02.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.269+0000 7fa41bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4240ffc70 0x7fa4241984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57644/0 (socket says 172.21.15.144:57644) 2024-09-10T11:52:02.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.269+0000 7fa41bfff700 1 -- 172.21.15.144:0/1736479517 learned_addr learned my addr 172.21.15.144:0/1736479517 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:52:02.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.269+0000 7fa4237fe700 1 --2- 172.21.15.144:0/1736479517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4240ff2d0 0x7fa424197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:02.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.269+0000 7fa41bfff700 1 -- 172.21.15.144:0/1736479517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4240ff2d0 msgr2=0x7fa424197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:02.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.269+0000 7fa41bfff700 1 --2- 172.21.15.144:0/1736479517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4240ff2d0 0x7fa424197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:02.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.269+0000 7fa41bfff700 1 -- 172.21.15.144:0/1736479517 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4140095f0 con 0x7fa4240ffc70 2024-09-10T11:52:02.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.270+0000 7fa41bfff700 1 --2- 172.21.15.144:0/1736479517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4240ffc70 0x7fa4241984d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c960 tx=0x7fa40c00cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:02.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.270+0000 7fa4217fa700 1 -- 172.21.15.144:0/1736479517 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa40c007a70 con 0x7fa4240ffc70 2024-09-10T11:52:02.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.270+0000 7fa4217fa700 1 -- 172.21.15.144:0/1736479517 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa40c007bd0 con 0x7fa4240ffc70 2024-09-10T11:52:02.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.270+0000 7fa42a248700 1 -- 172.21.15.144:0/1736479517 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa42419cab0 con 0x7fa4240ffc70 2024-09-10T11:52:02.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.270+0000 7fa4217fa700 1 -- 172.21.15.144:0/1736479517 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa40c0186c0 con 0x7fa4240ffc70 2024-09-10T11:52:02.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.270+0000 7fa42a248700 1 -- 172.21.15.144:0/1736479517 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa42419d000 con 0x7fa4240ffc70 2024-09-10T11:52:02.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.271+0000 7fa42a248700 1 -- 172.21.15.144:0/1736479517 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa42404ea50 con 0x7fa4240ffc70 2024-09-10T11:52:02.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.271+0000 7fa4217fa700 1 -- 172.21.15.144:0/1736479517 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa40c00faa0 con 0x7fa4240ffc70 2024-09-10T11:52:02.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.272+0000 7fa4217fa700 1 --2- 172.21.15.144:0/1736479517 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa41006e200 0x7fa4100706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:02.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.272+0000 7fa4217fa700 1 -- 172.21.15.144:0/1736479517 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa40c08e070 con 0x7fa4240ffc70 2024-09-10T11:52:02.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.272+0000 7fa4237fe700 1 --2- 172.21.15.144:0/1736479517 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa41006e200 0x7fa4100706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:02.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.272+0000 7fa4237fe700 1 --2- 172.21.15.144:0/1736479517 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa41006e200 0x7fa4100706b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa41400b5c0 tx=0x7fa41401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:02.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.276+0000 7fa4217fa700 1 -- 172.21.15.144:0/1736479517 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa40c05b000 con 0x7fa4240ffc70 2024-09-10T11:52:02.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.498+0000 7fa42a248700 1 -- 172.21.15.144:0/1736479517 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa42419c940 con 0x7fa41006e200 2024-09-10T11:52:02.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.500+0000 7fa4217fa700 1 -- 172.21.15.144:0/1736479517 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa42419c940 con 0x7fa41006e200 2024-09-10T11:52:02.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.502+0000 7fa42a248700 1 -- 172.21.15.144:0/1736479517 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa41006e200 msgr2=0x7fa4100706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:02.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.502+0000 7fa42a248700 1 --2- 172.21.15.144:0/1736479517 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa41006e200 0x7fa4100706b0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fa41400b5c0 tx=0x7fa41401a040 comp rx=0 tx=0).stop 2024-09-10T11:52:02.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.502+0000 7fa42a248700 1 -- 172.21.15.144:0/1736479517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4240ffc70 msgr2=0x7fa4241984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:02.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.502+0000 7fa42a248700 1 --2- 172.21.15.144:0/1736479517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4240ffc70 0x7fa4241984d0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c960 tx=0x7fa40c00cd20 comp rx=0 tx=0).stop 2024-09-10T11:52:02.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.502+0000 7fa42a248700 1 -- 172.21.15.144:0/1736479517 shutdown_connections 2024-09-10T11:52:02.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.502+0000 7fa42a248700 1 --2- 172.21.15.144:0/1736479517 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa41006e200 0x7fa4100706b0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:02.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.502+0000 7fa42a248700 1 --2- 172.21.15.144:0/1736479517 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4240ff2d0 0x7fa424197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:02.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.502+0000 7fa42a248700 1 --2- 172.21.15.144:0/1736479517 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4240ffc70 0x7fa4241984d0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:02.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.502+0000 7fa42a248700 1 -- 172.21.15.144:0/1736479517 >> 172.21.15.144:0/1736479517 conn(0x7fa4240fabb0 msgr2=0x7fa424069660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:02.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.502+0000 7fa42a248700 1 -- 172.21.15.144:0/1736479517 shutdown_connections 2024-09-10T11:52:02.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.502+0000 7fa42a248700 1 -- 172.21.15.144:0/1736479517 wait complete. 2024-09-10T11:52:02.515 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:52:02.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.614+0000 7fbd8e05c700 1 -- 172.21.15.144:0/33151995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd88100d70 msgr2=0x7fbd88104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:02.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.614+0000 7fbd8e05c700 1 --2- 172.21.15.144:0/33151995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd88100d70 0x7fbd88104d40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fbd70009910 tx=0x7fbd70009c20 comp rx=0 tx=0).stop 2024-09-10T11:52:02.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.614+0000 7fbd8e05c700 1 -- 172.21.15.144:0/33151995 shutdown_connections 2024-09-10T11:52:02.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.614+0000 7fbd8e05c700 1 --2- 172.21.15.144:0/33151995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd88100d70 0x7fbd88104d40 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:02.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.614+0000 7fbd8e05c700 1 --2- 172.21.15.144:0/33151995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd881003d0 0x7fbd881007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:02.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.614+0000 7fbd8e05c700 1 -- 172.21.15.144:0/33151995 >> 172.21.15.144:0/33151995 conn(0x7fbd880fbc90 msgr2=0x7fbd880fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:02.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.614+0000 7fbd8e05c700 1 -- 172.21.15.144:0/33151995 shutdown_connections 2024-09-10T11:52:02.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.614+0000 7fbd8e05c700 1 -- 172.21.15.144:0/33151995 wait complete. 2024-09-10T11:52:02.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.615+0000 7fbd8e05c700 1 Processor -- start 2024-09-10T11:52:02.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.615+0000 7fbd8e05c700 1 -- start start 2024-09-10T11:52:02.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.615+0000 7fbd8e05c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd881003d0 0x7fbd88195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:02.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.615+0000 7fbd8e05c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd88100d70 0x7fbd88196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:02.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.615+0000 7fbd8e05c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd88196950 con 0x7fbd881003d0 2024-09-10T11:52:02.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.615+0000 7fbd8e05c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8819a6e0 con 0x7fbd88100d70 2024-09-10T11:52:02.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.615+0000 7fbd877fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd881003d0 0x7fbd88195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:02.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.615+0000 7fbd877fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd881003d0 0x7fbd88195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53398/0 (socket says 172.21.15.144:53398) 2024-09-10T11:52:02.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.615+0000 7fbd877fe700 1 -- 172.21.15.144:0/3874546463 learned_addr learned my addr 172.21.15.144:0/3874546463 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:52:02.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.615+0000 7fbd877fe700 1 -- 172.21.15.144:0/3874546463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd88100d70 msgr2=0x7fbd88196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:02.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.615+0000 7fbd86ffd700 1 --2- 172.21.15.144:0/3874546463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd88100d70 0x7fbd88196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:02.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.615+0000 7fbd877fe700 1 --2- 172.21.15.144:0/3874546463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd88100d70 0x7fbd88196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:02.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.615+0000 7fbd877fe700 1 -- 172.21.15.144:0/3874546463 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd700095f0 con 0x7fbd881003d0 2024-09-10T11:52:02.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.616+0000 7fbd877fe700 1 --2- 172.21.15.144:0/3874546463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd881003d0 0x7fbd88195d30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800ada0 tx=0x7fbd7800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:02.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.616+0000 7fbd84ff9700 1 -- 172.21.15.144:0/3874546463 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd7800f070 con 0x7fbd881003d0 2024-09-10T11:52:02.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.616+0000 7fbd8e05c700 1 -- 172.21.15.144:0/3874546463 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd8819a9c0 con 0x7fbd881003d0 2024-09-10T11:52:02.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.616+0000 7fbd84ff9700 1 -- 172.21.15.144:0/3874546463 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbd78007bf0 con 0x7fbd881003d0 2024-09-10T11:52:02.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.616+0000 7fbd84ff9700 1 -- 172.21.15.144:0/3874546463 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd78005690 con 0x7fbd881003d0 2024-09-10T11:52:02.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.616+0000 7fbd8e05c700 1 -- 172.21.15.144:0/3874546463 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8819af10 con 0x7fbd881003d0 2024-09-10T11:52:02.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.617+0000 7fbd8e05c700 1 -- 172.21.15.144:0/3874546463 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd88108740 con 0x7fbd881003d0 2024-09-10T11:52:02.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.617+0000 7fbd84ff9700 1 -- 172.21.15.144:0/3874546463 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fbd78005900 con 0x7fbd881003d0 2024-09-10T11:52:02.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.618+0000 7fbd84ff9700 1 --2- 172.21.15.144:0/3874546463 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbd7406e200 0x7fbd740706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:02.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.618+0000 7fbd84ff9700 1 -- 172.21.15.144:0/3874546463 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbd7808dac0 con 0x7fbd881003d0 2024-09-10T11:52:02.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.618+0000 7fbd86ffd700 1 --2- 172.21.15.144:0/3874546463 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbd7406e200 0x7fbd740706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:02.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.618+0000 7fbd86ffd700 1 --2- 172.21.15.144:0/3874546463 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbd7406e200 0x7fbd740706b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbd70006010 tx=0x7fbd7000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:02.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.622+0000 7fbd84ff9700 1 -- 172.21.15.144:0/3874546463 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbd78056ea0 con 0x7fbd881003d0 2024-09-10T11:52:02.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.845+0000 7fbd8e05c700 1 -- 172.21.15.144:0/3874546463 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbd8819b210 con 0x7fbd7406e200 2024-09-10T11:52:02.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.846+0000 7fbd84ff9700 1 -- 172.21.15.144:0/3874546463 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fbd8819b210 con 0x7fbd7406e200 2024-09-10T11:52:02.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.849+0000 7fbd8e05c700 1 -- 172.21.15.144:0/3874546463 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbd7406e200 msgr2=0x7fbd740706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:02.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.849+0000 7fbd8e05c700 1 --2- 172.21.15.144:0/3874546463 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbd7406e200 0x7fbd740706b0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbd70006010 tx=0x7fbd7000b540 comp rx=0 tx=0).stop 2024-09-10T11:52:02.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.849+0000 7fbd8e05c700 1 -- 172.21.15.144:0/3874546463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd881003d0 msgr2=0x7fbd88195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:02.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.849+0000 7fbd8e05c700 1 --2- 172.21.15.144:0/3874546463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd881003d0 0x7fbd88195d30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fbd7800ada0 tx=0x7fbd7800c330 comp rx=0 tx=0).stop 2024-09-10T11:52:02.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.849+0000 7fbd8e05c700 1 -- 172.21.15.144:0/3874546463 shutdown_connections 2024-09-10T11:52:02.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.849+0000 7fbd8e05c700 1 --2- 172.21.15.144:0/3874546463 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbd7406e200 0x7fbd740706b0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:02.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.849+0000 7fbd8e05c700 1 --2- 172.21.15.144:0/3874546463 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbd881003d0 0x7fbd88195d30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:02.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.849+0000 7fbd8e05c700 1 --2- 172.21.15.144:0/3874546463 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbd88100d70 0x7fbd88196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:02.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.849+0000 7fbd8e05c700 1 -- 172.21.15.144:0/3874546463 >> 172.21.15.144:0/3874546463 conn(0x7fbd880fbc90 msgr2=0x7fbd880fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:02.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.849+0000 7fbd8e05c700 1 -- 172.21.15.144:0/3874546463 shutdown_connections 2024-09-10T11:52:02.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.849+0000 7fbd8e05c700 1 -- 172.21.15.144:0/3874546463 wait complete. 2024-09-10T11:52:02.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.955+0000 7f72d1388700 1 -- 172.21.15.144:0/3764534144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72cc0ffb80 msgr2=0x7f72cc10a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:02.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.955+0000 7f72d1388700 1 --2- 172.21.15.144:0/3764534144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72cc0ffb80 0x7f72cc10a5f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f72c0009910 tx=0x7f72c0009c20 comp rx=0 tx=0).stop 2024-09-10T11:52:02.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.955+0000 7f72d1388700 1 -- 172.21.15.144:0/3764534144 shutdown_connections 2024-09-10T11:52:02.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.955+0000 7f72d1388700 1 --2- 172.21.15.144:0/3764534144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72cc0ffb80 0x7f72cc10a5f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:02.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.955+0000 7f72d1388700 1 --2- 172.21.15.144:0/3764534144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72cc0ff270 0x7f72cc0ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:02.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.955+0000 7f72d1388700 1 -- 172.21.15.144:0/3764534144 >> 172.21.15.144:0/3764534144 conn(0x7f72cc074850 msgr2=0x7f72cc074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:02.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.955+0000 7f72d1388700 1 -- 172.21.15.144:0/3764534144 shutdown_connections 2024-09-10T11:52:02.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.955+0000 7f72d1388700 1 -- 172.21.15.144:0/3764534144 wait complete. 2024-09-10T11:52:02.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.956+0000 7f72d1388700 1 Processor -- start 2024-09-10T11:52:02.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.956+0000 7f72d1388700 1 -- start start 2024-09-10T11:52:02.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.956+0000 7f72d1388700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72cc0ff270 0x7f72cc100450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:02.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.956+0000 7f72d1388700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72cc0ffb80 0x7f72cc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:02.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.956+0000 7f72d1388700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc100ed0 con 0x7f72cc0ffb80 2024-09-10T11:52:02.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.956+0000 7f72d1388700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc101010 con 0x7f72cc0ff270 2024-09-10T11:52:02.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.956+0000 7f72ca7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72cc0ffb80 0x7f72cc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:02.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.956+0000 7f72ca7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72cc0ffb80 0x7f72cc100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53406/0 (socket says 172.21.15.144:53406) 2024-09-10T11:52:02.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.956+0000 7f72ca7fc700 1 -- 172.21.15.144:0/1126371603 learned_addr learned my addr 172.21.15.144:0/1126371603 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:52:02.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.957+0000 7f72caffd700 1 --2- 172.21.15.144:0/1126371603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72cc0ff270 0x7f72cc100450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:02.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.957+0000 7f72ca7fc700 1 -- 172.21.15.144:0/1126371603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72cc0ff270 msgr2=0x7f72cc100450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:02.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.957+0000 7f72ca7fc700 1 --2- 172.21.15.144:0/1126371603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72cc0ff270 0x7f72cc100450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:02.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.957+0000 7f72ca7fc700 1 -- 172.21.15.144:0/1126371603 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c00095f0 con 0x7f72cc0ffb80 2024-09-10T11:52:02.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.957+0000 7f72ca7fc700 1 --2- 172.21.15.144:0/1126371603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72cc0ffb80 0x7f72cc100990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f72c0006010 tx=0x7f72c000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:02.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.957+0000 7f72b3fff700 1 -- 172.21.15.144:0/1126371603 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72c001d070 con 0x7f72cc0ffb80 2024-09-10T11:52:02.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.957+0000 7f72b3fff700 1 -- 172.21.15.144:0/1126371603 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72c0005dc0 con 0x7f72cc0ffb80 2024-09-10T11:52:02.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.957+0000 7f72d1388700 1 -- 172.21.15.144:0/1126371603 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc101290 con 0x7f72cc0ffb80 2024-09-10T11:52:02.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.957+0000 7f72b3fff700 1 -- 172.21.15.144:0/1126371603 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72c0017400 con 0x7f72cc0ffb80 2024-09-10T11:52:02.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.957+0000 7f72d1388700 1 -- 172.21.15.144:0/1126371603 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc1a21c0 con 0x7f72cc0ffb80 2024-09-10T11:52:02.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.958+0000 7f72d1388700 1 -- 172.21.15.144:0/1126371603 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc107cf0 con 0x7f72cc0ffb80 2024-09-10T11:52:02.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.959+0000 7f72b3fff700 1 -- 172.21.15.144:0/1126371603 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f72c0003a60 con 0x7f72cc0ffb80 2024-09-10T11:52:02.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.959+0000 7f72b3fff700 1 --2- 172.21.15.144:0/1126371603 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f72b406e200 0x7f72b40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:02.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.959+0000 7f72b3fff700 1 -- 172.21.15.144:0/1126371603 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f72c008f400 con 0x7f72cc0ffb80 2024-09-10T11:52:02.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.959+0000 7f72caffd700 1 --2- 172.21.15.144:0/1126371603 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f72b406e200 0x7f72b40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:02.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.960+0000 7f72caffd700 1 --2- 172.21.15.144:0/1126371603 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f72b406e200 0x7f72b40706b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f72bc007c60 tx=0x7f72bc0073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:02.965 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:02.963+0000 7f72b3fff700 1 -- 172.21.15.144:0/1126371603 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f72c0058860 con 0x7f72cc0ffb80 2024-09-10T11:52:03.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:02 smithi144 ceph-mon[87436]: pgmap v601: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:03.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.178+0000 7f72d1388700 1 -- 172.21.15.144:0/1126371603 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f72cc02cca0 con 0x7f72b406e200 2024-09-10T11:52:03.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.187+0000 7f72b3fff700 1 -- 172.21.15.144:0/1126371603 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f72cc02cca0 con 0x7f72b406e200 2024-09-10T11:52:03.189 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:52:03.189 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (20m) 13m ago 30m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:52:03.189 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (31m) 13m ago 31m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:52:03.190 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (30m) 13m ago 30m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:52:03.190 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (17m) 13m ago 31m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:52:03.190 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (17m) 13m ago 30m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:52:03.190 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (19m) 13m ago 30m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:52:03.190 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (14m) 13m ago 25m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:52:03.190 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (14m) 13m ago 25m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:52:03.190 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (25m) 13m ago 25m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:52:03.190 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (13m) 13m ago 25m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:52:03.190 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (22m) 13m ago 32m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:52:03.190 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (21m) 13m ago 29m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:52:03.190 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (18m) 13m ago 32m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:52:03.190 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (18m) 13m ago 29m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:52:03.190 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (20m) 13m ago 30m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:52:03.190 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (20m) 13m ago 29m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:52:03.191 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (17m) 13m ago 29m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:52:03.191 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (16m) 13m ago 28m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:52:03.191 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (16m) 13m ago 28m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:52:03.191 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (15m) 13m ago 27m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:52:03.191 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (15m) 13m ago 27m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:52:03.191 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (14m) 13m ago 26m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:52:03.191 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (20m) 13m ago 30m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:52:03.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.189+0000 7f72d1388700 1 -- 172.21.15.144:0/1126371603 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f72b406e200 msgr2=0x7f72b40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:03.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.189+0000 7f72d1388700 1 --2- 172.21.15.144:0/1126371603 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f72b406e200 0x7f72b40706b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f72bc007c60 tx=0x7f72bc0073d0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.189+0000 7f72d1388700 1 -- 172.21.15.144:0/1126371603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72cc0ffb80 msgr2=0x7f72cc100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:03.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.189+0000 7f72d1388700 1 --2- 172.21.15.144:0/1126371603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72cc0ffb80 0x7f72cc100990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f72c0006010 tx=0x7f72c000bd70 comp rx=0 tx=0).stop 2024-09-10T11:52:03.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.189+0000 7f72d1388700 1 -- 172.21.15.144:0/1126371603 shutdown_connections 2024-09-10T11:52:03.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.189+0000 7f72d1388700 1 --2- 172.21.15.144:0/1126371603 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f72b406e200 0x7f72b40706b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.189+0000 7f72d1388700 1 --2- 172.21.15.144:0/1126371603 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72cc0ff270 0x7f72cc100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.189+0000 7f72d1388700 1 --2- 172.21.15.144:0/1126371603 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72cc0ffb80 0x7f72cc100990 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.189+0000 7f72d1388700 1 -- 172.21.15.144:0/1126371603 >> 172.21.15.144:0/1126371603 conn(0x7f72cc074850 msgr2=0x7f72cc104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:03.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.190+0000 7f72d1388700 1 -- 172.21.15.144:0/1126371603 shutdown_connections 2024-09-10T11:52:03.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.190+0000 7f72d1388700 1 -- 172.21.15.144:0/1126371603 wait complete. 2024-09-10T11:52:03.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.293+0000 7f8bbc4b4700 1 -- 172.21.15.144:0/1717777044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb4100d80 msgr2=0x7f8bb4104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:03.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.293+0000 7f8bbc4b4700 1 --2- 172.21.15.144:0/1717777044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb4100d80 0x7f8bb4104d50 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009910 tx=0x7f8ba8009c20 comp rx=0 tx=0).stop 2024-09-10T11:52:03.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.293+0000 7f8bbc4b4700 1 -- 172.21.15.144:0/1717777044 shutdown_connections 2024-09-10T11:52:03.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.293+0000 7f8bbc4b4700 1 --2- 172.21.15.144:0/1717777044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb4100d80 0x7f8bb4104d50 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.293+0000 7f8bbc4b4700 1 --2- 172.21.15.144:0/1717777044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb41003e0 0x7f8bb41007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.293+0000 7f8bbc4b4700 1 -- 172.21.15.144:0/1717777044 >> 172.21.15.144:0/1717777044 conn(0x7f8bb40fbca0 msgr2=0x7f8bb40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:03.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.293+0000 7f8bbc4b4700 1 -- 172.21.15.144:0/1717777044 shutdown_connections 2024-09-10T11:52:03.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.293+0000 7f8bbc4b4700 1 -- 172.21.15.144:0/1717777044 wait complete. 2024-09-10T11:52:03.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.294+0000 7f8bbc4b4700 1 Processor -- start 2024-09-10T11:52:03.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.294+0000 7f8bbc4b4700 1 -- start start 2024-09-10T11:52:03.298 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.294+0000 7f8bbc4b4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb41003e0 0x7f8bb4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:03.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.294+0000 7f8bbc4b4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb4100d80 0x7f8bb4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:03.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.294+0000 7f8bbc4b4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4073ad0 con 0x7f8bb41003e0 2024-09-10T11:52:03.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.294+0000 7f8bbc4b4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb4073c10 con 0x7f8bb4100d80 2024-09-10T11:52:03.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.294+0000 7f8bba250700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb41003e0 0x7f8bb4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:03.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.294+0000 7f8bba250700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb41003e0 0x7f8bb4073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53414/0 (socket says 172.21.15.144:53414) 2024-09-10T11:52:03.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.294+0000 7f8bba250700 1 -- 172.21.15.144:0/1077208975 learned_addr learned my addr 172.21.15.144:0/1077208975 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:52:03.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.294+0000 7f8bb9a4f700 1 --2- 172.21.15.144:0/1077208975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb4100d80 0x7f8bb4073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:03.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.295+0000 7f8bba250700 1 -- 172.21.15.144:0/1077208975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb4100d80 msgr2=0x7f8bb4073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:03.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.295+0000 7f8bba250700 1 --2- 172.21.15.144:0/1077208975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb4100d80 0x7f8bb4073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.295+0000 7f8bba250700 1 -- 172.21.15.144:0/1077208975 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba80095f0 con 0x7f8bb41003e0 2024-09-10T11:52:03.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.295+0000 7f8bb9a4f700 1 --2- 172.21.15.144:0/1077208975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb4100d80 0x7f8bb4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:52:03.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.295+0000 7f8bba250700 1 --2- 172.21.15.144:0/1077208975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb41003e0 0x7f8bb4073050 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0060b0 tx=0x7f8bac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:03.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.295+0000 7f8bb37fe700 1 -- 172.21.15.144:0/1077208975 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bac00dda0 con 0x7f8bb41003e0 2024-09-10T11:52:03.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.295+0000 7f8bb37fe700 1 -- 172.21.15.144:0/1077208975 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8bac00f040 con 0x7f8bb41003e0 2024-09-10T11:52:03.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.295+0000 7f8bbc4b4700 1 -- 172.21.15.144:0/1077208975 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb41a1fa0 con 0x7f8bb41003e0 2024-09-10T11:52:03.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.295+0000 7f8bb37fe700 1 -- 172.21.15.144:0/1077208975 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bac014790 con 0x7f8bb41003e0 2024-09-10T11:52:03.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.295+0000 7f8bbc4b4700 1 -- 172.21.15.144:0/1077208975 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb41a23b0 con 0x7f8bb41003e0 2024-09-10T11:52:03.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.296+0000 7f8bbc4b4700 1 -- 172.21.15.144:0/1077208975 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb404ea50 con 0x7f8bb41003e0 2024-09-10T11:52:03.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.297+0000 7f8bb37fe700 1 -- 172.21.15.144:0/1077208975 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8bac014980 con 0x7f8bb41003e0 2024-09-10T11:52:03.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.297+0000 7f8bb37fe700 1 --2- 172.21.15.144:0/1077208975 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8b9c06e200 0x7f8b9c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:03.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.297+0000 7f8bb37fe700 1 -- 172.21.15.144:0/1077208975 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8bac08db30 con 0x7f8bb41003e0 2024-09-10T11:52:03.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.297+0000 7f8bb9a4f700 1 --2- 172.21.15.144:0/1077208975 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8b9c06e200 0x7f8b9c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:03.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.298+0000 7f8bb9a4f700 1 --2- 172.21.15.144:0/1077208975 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8b9c06e200 0x7f8b9c0706b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800b5c0 tx=0x7f8ba80053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:03.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.301+0000 7f8bb37fe700 1 -- 172.21.15.144:0/1077208975 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8bac056fc0 con 0x7f8bb41003e0 2024-09-10T11:52:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:02 smithi178 ceph-mon[65940]: pgmap v601: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:03.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.590+0000 7f8bbc4b4700 1 -- 172.21.15.144:0/1077208975 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8bb4074580 con 0x7f8bb41003e0 2024-09-10T11:52:03.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.591+0000 7f8bb37fe700 1 -- 172.21.15.144:0/1077208975 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f8bac05a680 con 0x7f8bb41003e0 2024-09-10T11:52:03.593 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:52:03.593 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:52:03.594 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:52:03.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:52:03.594 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:52:03.594 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:52:03.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:52:03.594 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:52:03.594 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:52:03.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:52:03.594 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:52:03.594 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:52:03.594 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:52:03.594 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:52:03.594 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:52:03.594 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:52:03.595 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:52:03.595 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:52:03.595 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:52:03.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.594+0000 7f8bbc4b4700 1 -- 172.21.15.144:0/1077208975 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8b9c06e200 msgr2=0x7f8b9c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:03.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.594+0000 7f8bbc4b4700 1 --2- 172.21.15.144:0/1077208975 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8b9c06e200 0x7f8b9c0706b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800b5c0 tx=0x7f8ba80053a0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.594+0000 7f8bbc4b4700 1 -- 172.21.15.144:0/1077208975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb41003e0 msgr2=0x7f8bb4073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:03.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.594+0000 7f8bbc4b4700 1 --2- 172.21.15.144:0/1077208975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb41003e0 0x7f8bb4073050 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8bac0060b0 tx=0x7f8bac00d3b0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.594+0000 7f8bbc4b4700 1 -- 172.21.15.144:0/1077208975 shutdown_connections 2024-09-10T11:52:03.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.594+0000 7f8bbc4b4700 1 --2- 172.21.15.144:0/1077208975 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8b9c06e200 0x7f8b9c0706b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.594+0000 7f8bbc4b4700 1 --2- 172.21.15.144:0/1077208975 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8bb41003e0 0x7f8bb4073050 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.594+0000 7f8bbc4b4700 1 --2- 172.21.15.144:0/1077208975 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8bb4100d80 0x7f8bb4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.594+0000 7f8bbc4b4700 1 -- 172.21.15.144:0/1077208975 >> 172.21.15.144:0/1077208975 conn(0x7f8bb40fbca0 msgr2=0x7f8bb40fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:03.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.594+0000 7f8bbc4b4700 1 -- 172.21.15.144:0/1077208975 shutdown_connections 2024-09-10T11:52:03.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.594+0000 7f8bbc4b4700 1 -- 172.21.15.144:0/1077208975 wait complete. 2024-09-10T11:52:03.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.698+0000 7f6a75bfb700 1 -- 172.21.15.144:0/3447161625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a700ff320 msgr2=0x7f6a700ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:03.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.698+0000 7f6a75bfb700 1 --2- 172.21.15.144:0/3447161625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a700ff320 0x7f6a700ff6f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b0d0 tx=0x7f6a6000b3e0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.698+0000 7f6a75bfb700 1 -- 172.21.15.144:0/3447161625 shutdown_connections 2024-09-10T11:52:03.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.698+0000 7f6a75bfb700 1 --2- 172.21.15.144:0/3447161625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a700ffc30 0x7f6a7010a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.698+0000 7f6a75bfb700 1 --2- 172.21.15.144:0/3447161625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a700ff320 0x7f6a700ff6f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.698+0000 7f6a75bfb700 1 -- 172.21.15.144:0/3447161625 >> 172.21.15.144:0/3447161625 conn(0x7f6a700748b0 msgr2=0x7f6a70074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:03.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.698+0000 7f6a75bfb700 1 -- 172.21.15.144:0/3447161625 shutdown_connections 2024-09-10T11:52:03.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.698+0000 7f6a75bfb700 1 -- 172.21.15.144:0/3447161625 wait complete. 2024-09-10T11:52:03.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.699+0000 7f6a75bfb700 1 Processor -- start 2024-09-10T11:52:03.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.699+0000 7f6a75bfb700 1 -- start start 2024-09-10T11:52:03.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.699+0000 7f6a75bfb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a700ff320 0x7f6a7019ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:03.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.699+0000 7f6a75bfb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a700ffc30 0x7f6a7019bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:03.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.699+0000 7f6a75bfb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7019c5f0 con 0x7f6a700ff320 2024-09-10T11:52:03.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.699+0000 7f6a75bfb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a70195aa0 con 0x7f6a700ffc30 2024-09-10T11:52:03.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.699+0000 7f6a6f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a700ff320 0x7f6a7019ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:03.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.699+0000 7f6a6f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a700ff320 0x7f6a7019ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53430/0 (socket says 172.21.15.144:53430) 2024-09-10T11:52:03.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.699+0000 7f6a6f7fe700 1 -- 172.21.15.144:0/2157836157 learned_addr learned my addr 172.21.15.144:0/2157836157 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:52:03.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.699+0000 7f6a6effd700 1 --2- 172.21.15.144:0/2157836157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a700ffc30 0x7f6a7019bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:03.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.700+0000 7f6a6f7fe700 1 -- 172.21.15.144:0/2157836157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a700ffc30 msgr2=0x7f6a7019bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:03.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.700+0000 7f6a6f7fe700 1 --2- 172.21.15.144:0/2157836157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a700ffc30 0x7f6a7019bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.700+0000 7f6a6f7fe700 1 -- 172.21.15.144:0/2157836157 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a60009d00 con 0x7f6a700ff320 2024-09-10T11:52:03.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.700+0000 7f6a6effd700 1 --2- 172.21.15.144:0/2157836157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a700ffc30 0x7f6a7019bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:52:03.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.700+0000 7f6a6f7fe700 1 --2- 172.21.15.144:0/2157836157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a700ff320 0x7f6a7019ba20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b0a0 tx=0x7f6a600076e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:03.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.700+0000 7f6a6cff9700 1 -- 172.21.15.144:0/2157836157 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a6000e040 con 0x7f6a700ff320 2024-09-10T11:52:03.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.700+0000 7f6a6cff9700 1 -- 172.21.15.144:0/2157836157 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a60003e60 con 0x7f6a700ff320 2024-09-10T11:52:03.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.700+0000 7f6a6cff9700 1 -- 172.21.15.144:0/2157836157 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a6001d980 con 0x7f6a700ff320 2024-09-10T11:52:03.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.700+0000 7f6a75bfb700 1 -- 172.21.15.144:0/2157836157 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a70195d20 con 0x7f6a700ff320 2024-09-10T11:52:03.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.700+0000 7f6a75bfb700 1 -- 172.21.15.144:0/2157836157 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a70196270 con 0x7f6a700ff320 2024-09-10T11:52:03.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.701+0000 7f6a75bfb700 1 -- 172.21.15.144:0/2157836157 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a70107da0 con 0x7f6a700ff320 2024-09-10T11:52:03.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.701+0000 7f6a6cff9700 1 -- 172.21.15.144:0/2157836157 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6a6001dbf0 con 0x7f6a700ff320 2024-09-10T11:52:03.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.702+0000 7f6a6cff9700 1 --2- 172.21.15.144:0/2157836157 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6a5c06e200 0x7f6a5c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:03.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.702+0000 7f6a6cff9700 1 -- 172.21.15.144:0/2157836157 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6a6008fb50 con 0x7f6a700ff320 2024-09-10T11:52:03.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.702+0000 7f6a6effd700 1 --2- 172.21.15.144:0/2157836157 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6a5c06e200 0x7f6a5c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:03.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.702+0000 7f6a6effd700 1 --2- 172.21.15.144:0/2157836157 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6a5c06e200 0x7f6a5c0706b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6a700ff0e0 tx=0x7f6a58009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:03.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.706+0000 7f6a6cff9700 1 -- 172.21.15.144:0/2157836157 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6a6005cae0 con 0x7f6a700ff320 2024-09-10T11:52:03.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.952+0000 7f6a75bfb700 1 -- 172.21.15.144:0/2157836157 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6a701965a0 con 0x7f6a700ff320 2024-09-10T11:52:03.955 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.952+0000 7f6a6cff9700 1 -- 172.21.15.144:0/2157836157 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f6a60017790 con 0x7f6a700ff320 2024-09-10T11:52:03.955 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:52:03.955 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:52:03.955 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:52:03.955 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:52:03.955 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:52:03.955 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:52:03.955 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:52:03.955 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:52:03.955 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:52:03.956 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:52:03.956 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:52:03.956 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:52:03.956 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:52:03.956 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:52:03.956 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:52:03.956 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:52:03.956 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:52:03.956 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:52:03.956 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:52:03.956 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:52:03.956 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:52:03.956 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:52:03.957 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:52:03.957 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:52:03.957 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:52:03.957 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:52:03.957 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:52:03.957 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:52:03.957 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:52:03.957 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:52:03.957 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:52:03.957 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:52:03.957 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:52:03.957 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:52:03.957 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:52:03.957 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:52:03.958 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:52:03.958 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:52:03.958 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:52:03.958 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:52:03.958 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:52:03.958 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:52:03.958 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:52:03.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.955+0000 7f6a75bfb700 1 -- 172.21.15.144:0/2157836157 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6a5c06e200 msgr2=0x7f6a5c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:03.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.955+0000 7f6a75bfb700 1 --2- 172.21.15.144:0/2157836157 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6a5c06e200 0x7f6a5c0706b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6a700ff0e0 tx=0x7f6a58009260 comp rx=0 tx=0).stop 2024-09-10T11:52:03.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.955+0000 7f6a75bfb700 1 -- 172.21.15.144:0/2157836157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a700ff320 msgr2=0x7f6a7019ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:03.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.955+0000 7f6a75bfb700 1 --2- 172.21.15.144:0/2157836157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a700ff320 0x7f6a7019ba20 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b0a0 tx=0x7f6a600076e0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.955+0000 7f6a75bfb700 1 -- 172.21.15.144:0/2157836157 shutdown_connections 2024-09-10T11:52:03.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.955+0000 7f6a75bfb700 1 --2- 172.21.15.144:0/2157836157 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6a5c06e200 0x7f6a5c0706b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.955+0000 7f6a75bfb700 1 --2- 172.21.15.144:0/2157836157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a700ff320 0x7f6a7019ba20 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.955+0000 7f6a75bfb700 1 --2- 172.21.15.144:0/2157836157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a700ffc30 0x7f6a7019bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:03.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.955+0000 7f6a75bfb700 1 -- 172.21.15.144:0/2157836157 >> 172.21.15.144:0/2157836157 conn(0x7f6a700748b0 msgr2=0x7f6a70104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:03.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.955+0000 7f6a75bfb700 1 -- 172.21.15.144:0/2157836157 shutdown_connections 2024-09-10T11:52:03.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:03.955+0000 7f6a75bfb700 1 -- 172.21.15.144:0/2157836157 wait complete. 2024-09-10T11:52:03.960 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:52:04.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.060+0000 7f4995786700 1 -- 172.21.15.144:0/19619430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4990068fd0 msgr2=0x7f4990104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:04.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.060+0000 7f4995786700 1 --2- 172.21.15.144:0/19619430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4990068fd0 0x7f4990104df0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4978009910 tx=0x7f4978009c20 comp rx=0 tx=0).stop 2024-09-10T11:52:04.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.060+0000 7f4995786700 1 -- 172.21.15.144:0/19619430 shutdown_connections 2024-09-10T11:52:04.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.060+0000 7f4995786700 1 --2- 172.21.15.144:0/19619430 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4990068fd0 0x7f4990104df0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:04.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.060+0000 7f4995786700 1 --2- 172.21.15.144:0/19619430 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4990068630 0x7f4990068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:04.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.060+0000 7f4995786700 1 -- 172.21.15.144:0/19619430 >> 172.21.15.144:0/19619430 conn(0x7f4990074f50 msgr2=0x7f4990075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:04.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.060+0000 7f4995786700 1 -- 172.21.15.144:0/19619430 shutdown_connections 2024-09-10T11:52:04.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.060+0000 7f4995786700 1 -- 172.21.15.144:0/19619430 wait complete. 2024-09-10T11:52:04.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.061+0000 7f4995786700 1 Processor -- start 2024-09-10T11:52:04.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.061+0000 7f4995786700 1 -- start start 2024-09-10T11:52:04.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.061+0000 7f4995786700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4990068630 0x7f4990195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:04.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.061+0000 7f4995786700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4990068fd0 0x7f4990196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:04.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.061+0000 7f4995786700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4990196a60 con 0x7f4990068fd0 2024-09-10T11:52:04.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.061+0000 7f4995786700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499019a7f0 con 0x7f4990068630 2024-09-10T11:52:04.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.061+0000 7f498e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4990068fd0 0x7f4990196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:04.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.061+0000 7f498e7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4990068fd0 0x7f4990196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53448/0 (socket says 172.21.15.144:53448) 2024-09-10T11:52:04.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.061+0000 7f498e7fc700 1 -- 172.21.15.144:0/2609669938 learned_addr learned my addr 172.21.15.144:0/2609669938 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:52:04.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.061+0000 7f498effd700 1 --2- 172.21.15.144:0/2609669938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4990068630 0x7f4990195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:04.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.061+0000 7f498e7fc700 1 -- 172.21.15.144:0/2609669938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4990068630 msgr2=0x7f4990195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:04.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.061+0000 7f498e7fc700 1 --2- 172.21.15.144:0/2609669938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4990068630 0x7f4990195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:04.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.061+0000 7f498e7fc700 1 -- 172.21.15.144:0/2609669938 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49780095f0 con 0x7f4990068fd0 2024-09-10T11:52:04.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.061+0000 7f498effd700 1 --2- 172.21.15.144:0/2609669938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4990068630 0x7f4990195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:52:04.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.062+0000 7f498e7fc700 1 --2- 172.21.15.144:0/2609669938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4990068fd0 0x7f4990196380 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f49780092e0 tx=0x7f497800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:04.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.062+0000 7f4987fff700 1 -- 172.21.15.144:0/2609669938 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f497801d070 con 0x7f4990068fd0 2024-09-10T11:52:04.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.062+0000 7f4987fff700 1 -- 172.21.15.144:0/2609669938 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f497800f460 con 0x7f4990068fd0 2024-09-10T11:52:04.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.062+0000 7f4995786700 1 -- 172.21.15.144:0/2609669938 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499019aa70 con 0x7f4990068fd0 2024-09-10T11:52:04.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.062+0000 7f4987fff700 1 -- 172.21.15.144:0/2609669938 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49780175f0 con 0x7f4990068fd0 2024-09-10T11:52:04.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.062+0000 7f4995786700 1 -- 172.21.15.144:0/2609669938 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499019af60 con 0x7f4990068fd0 2024-09-10T11:52:04.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.063+0000 7f4995786700 1 -- 172.21.15.144:0/2609669938 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49901086f0 con 0x7f4990068fd0 2024-09-10T11:52:04.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.063+0000 7f4987fff700 1 -- 172.21.15.144:0/2609669938 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f497800f5d0 con 0x7f4990068fd0 2024-09-10T11:52:04.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.064+0000 7f4987fff700 1 --2- 172.21.15.144:0/2609669938 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f497c06e200 0x7f497c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:04.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.064+0000 7f498effd700 1 --2- 172.21.15.144:0/2609669938 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f497c06e200 0x7f497c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:04.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.064+0000 7f4987fff700 1 -- 172.21.15.144:0/2609669938 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f497808f5b0 con 0x7f4990068fd0 2024-09-10T11:52:04.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.064+0000 7f498effd700 1 --2- 172.21.15.144:0/2609669938 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f497c06e200 0x7f497c0706b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4988007eb0 tx=0x7f49880073c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:04.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.068+0000 7f4987fff700 1 -- 172.21.15.144:0/2609669938 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4978058990 con 0x7f4990068fd0 2024-09-10T11:52:04.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:03 smithi144 ceph-mon[87436]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:04.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:03 smithi144 ceph-mon[87436]: from='client.35132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:04.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:03 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1077208975' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:52:04.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.289+0000 7f4995786700 1 -- 172.21.15.144:0/2609669938 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f499002cd10 con 0x7f497c06e200 2024-09-10T11:52:04.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.291+0000 7f4987fff700 1 -- 172.21.15.144:0/2609669938 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f499002cd10 con 0x7f497c06e200 2024-09-10T11:52:04.293 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:52:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:52:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:52:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:52:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:52:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:52:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:52:04.293 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:52:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:52:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:52:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:52:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:52:04.294 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:52:04.294 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:52:04.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.293+0000 7f4995786700 1 -- 172.21.15.144:0/2609669938 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f497c06e200 msgr2=0x7f497c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:04.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.293+0000 7f4995786700 1 --2- 172.21.15.144:0/2609669938 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f497c06e200 0x7f497c0706b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f4988007eb0 tx=0x7f49880073c0 comp rx=0 tx=0).stop 2024-09-10T11:52:04.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.293+0000 7f4995786700 1 -- 172.21.15.144:0/2609669938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4990068fd0 msgr2=0x7f4990196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:04.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.293+0000 7f4995786700 1 --2- 172.21.15.144:0/2609669938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4990068fd0 0x7f4990196380 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f49780092e0 tx=0x7f497800be20 comp rx=0 tx=0).stop 2024-09-10T11:52:04.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.293+0000 7f4995786700 1 -- 172.21.15.144:0/2609669938 shutdown_connections 2024-09-10T11:52:04.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.293+0000 7f4995786700 1 --2- 172.21.15.144:0/2609669938 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f497c06e200 0x7f497c0706b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:04.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.293+0000 7f4995786700 1 --2- 172.21.15.144:0/2609669938 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4990068630 0x7f4990195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:04.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.293+0000 7f4995786700 1 --2- 172.21.15.144:0/2609669938 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4990068fd0 0x7f4990196380 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:04.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.293+0000 7f4995786700 1 -- 172.21.15.144:0/2609669938 >> 172.21.15.144:0/2609669938 conn(0x7f4990074f50 msgr2=0x7f49900fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:04.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.294+0000 7f4995786700 1 -- 172.21.15.144:0/2609669938 shutdown_connections 2024-09-10T11:52:04.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.294+0000 7f4995786700 1 -- 172.21.15.144:0/2609669938 wait complete. 2024-09-10T11:52:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:03 smithi178 ceph-mon[65940]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:03 smithi178 ceph-mon[65940]: from='client.35132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:03 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1077208975' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:52:04.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.399+0000 7f8ab1f70700 1 -- 172.21.15.144:0/1963638359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac1003d0 msgr2=0x7f8aac1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:04.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.399+0000 7f8ab1f70700 1 --2- 172.21.15.144:0/1963638359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac1003d0 0x7f8aac1007a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c009910 tx=0x7f8a9c009c20 comp rx=0 tx=0).stop 2024-09-10T11:52:04.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.399+0000 7f8ab1f70700 1 -- 172.21.15.144:0/1963638359 shutdown_connections 2024-09-10T11:52:04.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.399+0000 7f8ab1f70700 1 --2- 172.21.15.144:0/1963638359 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aac100d70 0x7f8aac104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:04.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.399+0000 7f8ab1f70700 1 --2- 172.21.15.144:0/1963638359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac1003d0 0x7f8aac1007a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:04.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.399+0000 7f8ab1f70700 1 -- 172.21.15.144:0/1963638359 >> 172.21.15.144:0/1963638359 conn(0x7f8aac0fbc90 msgr2=0x7f8aac0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:04.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.399+0000 7f8ab1f70700 1 -- 172.21.15.144:0/1963638359 shutdown_connections 2024-09-10T11:52:04.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.399+0000 7f8ab1f70700 1 -- 172.21.15.144:0/1963638359 wait complete. 2024-09-10T11:52:04.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.399+0000 7f8ab1f70700 1 Processor -- start 2024-09-10T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.399+0000 7f8ab1f70700 1 -- start start 2024-09-10T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8ab1f70700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aac1003d0 0x7f8aac195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8ab1f70700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac100d70 0x7f8aac1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:04.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8ab1f70700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac1968f0 con 0x7f8aac100d70 2024-09-10T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8ab1f70700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aac196a30 con 0x7f8aac1003d0 2024-09-10T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8aaaffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac100d70 0x7f8aac1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8aaaffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac100d70 0x7f8aac1962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53470/0 (socket says 172.21.15.144:53470) 2024-09-10T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8aaaffd700 1 -- 172.21.15.144:0/2418653764 learned_addr learned my addr 172.21.15.144:0/2418653764 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8aab7fe700 1 --2- 172.21.15.144:0/2418653764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aac1003d0 0x7f8aac195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8aaaffd700 1 -- 172.21.15.144:0/2418653764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aac1003d0 msgr2=0x7f8aac195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8aaaffd700 1 --2- 172.21.15.144:0/2418653764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aac1003d0 0x7f8aac195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8aaaffd700 1 -- 172.21.15.144:0/2418653764 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c0095f0 con 0x7f8aac100d70 2024-09-10T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8aab7fe700 1 --2- 172.21.15.144:0/2418653764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aac1003d0 0x7f8aac195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8aaaffd700 1 --2- 172.21.15.144:0/2418653764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac100d70 0x7f8aac1962a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000c960 tx=0x7f8aa000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8aa8ff9700 1 -- 172.21.15.144:0/2418653764 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8aa0007a70 con 0x7f8aac100d70 2024-09-10T11:52:04.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8ab1f70700 1 -- 172.21.15.144:0/2418653764 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aac19a880 con 0x7f8aac100d70 2024-09-10T11:52:04.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8aa8ff9700 1 -- 172.21.15.144:0/2418653764 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8aa0007bd0 con 0x7f8aac100d70 2024-09-10T11:52:04.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8aa8ff9700 1 -- 172.21.15.144:0/2418653764 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8aa00186c0 con 0x7f8aac100d70 2024-09-10T11:52:04.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.400+0000 7f8ab1f70700 1 -- 172.21.15.144:0/2418653764 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aac19add0 con 0x7f8aac100d70 2024-09-10T11:52:04.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.401+0000 7f8ab1f70700 1 -- 172.21.15.144:0/2418653764 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aac1086e0 con 0x7f8aac100d70 2024-09-10T11:52:04.405 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.402+0000 7f8aa8ff9700 1 -- 172.21.15.144:0/2418653764 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8aa000faa0 con 0x7f8aac100d70 2024-09-10T11:52:04.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.402+0000 7f8aa8ff9700 1 --2- 172.21.15.144:0/2418653764 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a9406e200 0x7f8a940706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:04.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.403+0000 7f8aa8ff9700 1 -- 172.21.15.144:0/2418653764 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8aa008e0a0 con 0x7f8aac100d70 2024-09-10T11:52:04.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.403+0000 7f8aab7fe700 1 --2- 172.21.15.144:0/2418653764 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a9406e200 0x7f8a940706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:04.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.403+0000 7f8aab7fe700 1 --2- 172.21.15.144:0/2418653764 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a9406e200 0x7f8a940706b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00b5c0 tx=0x7f8a9c005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:04.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.407+0000 7f8aa8ff9700 1 -- 172.21.15.144:0/2418653764 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8aa005afb0 con 0x7f8aac100d70 2024-09-10T11:52:04.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.690+0000 7f8ab1f70700 1 -- 172.21.15.144:0/2418653764 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8aac04ea50 con 0x7f8aac100d70 2024-09-10T11:52:04.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.691+0000 7f8aa8ff9700 1 -- 172.21.15.144:0/2418653764 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f8aa005ab40 con 0x7f8aac100d70 2024-09-10T11:52:04.693 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:52:04.693 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:52:04.693 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:52:04.693 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:52:04.693 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:52:04.694 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:52:04.694 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:52:04.697 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.693+0000 7f8ab1f70700 1 -- 172.21.15.144:0/2418653764 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a9406e200 msgr2=0x7f8a940706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:04.697 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.693+0000 7f8ab1f70700 1 --2- 172.21.15.144:0/2418653764 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a9406e200 0x7f8a940706b0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c00b5c0 tx=0x7f8a9c005960 comp rx=0 tx=0).stop 2024-09-10T11:52:04.697 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.693+0000 7f8ab1f70700 1 -- 172.21.15.144:0/2418653764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac100d70 msgr2=0x7f8aac1962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:04.697 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.693+0000 7f8ab1f70700 1 --2- 172.21.15.144:0/2418653764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac100d70 0x7f8aac1962a0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f8aa000c960 tx=0x7f8aa000cd20 comp rx=0 tx=0).stop 2024-09-10T11:52:04.697 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.693+0000 7f8ab1f70700 1 -- 172.21.15.144:0/2418653764 shutdown_connections 2024-09-10T11:52:04.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.693+0000 7f8ab1f70700 1 --2- 172.21.15.144:0/2418653764 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a9406e200 0x7f8a940706b0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:04.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.693+0000 7f8ab1f70700 1 --2- 172.21.15.144:0/2418653764 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8aac1003d0 0x7f8aac195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:04.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.693+0000 7f8ab1f70700 1 --2- 172.21.15.144:0/2418653764 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8aac100d70 0x7f8aac1962a0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:04.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.693+0000 7f8ab1f70700 1 -- 172.21.15.144:0/2418653764 >> 172.21.15.144:0/2418653764 conn(0x7f8aac0fbc90 msgr2=0x7f8aac0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:04.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.693+0000 7f8ab1f70700 1 -- 172.21.15.144:0/2418653764 shutdown_connections 2024-09-10T11:52:04.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:04.693+0000 7f8ab1f70700 1 -- 172.21.15.144:0/2418653764 wait complete. 2024-09-10T11:52:05.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:04 smithi144 ceph-mon[87436]: from='client.35136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:05.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:04 smithi144 ceph-mon[87436]: pgmap v602: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:05.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:04 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2157836157' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:52:05.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:04 smithi144 ceph-mon[87436]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:05.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:04 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2418653764' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:52:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:04 smithi178 ceph-mon[65940]: from='client.35136 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:04 smithi178 ceph-mon[65940]: pgmap v602: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:04 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2157836157' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:52:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:04 smithi178 ceph-mon[65940]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:04 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2418653764' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:52:06.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:05 smithi144 ceph-mon[87436]: pgmap v603: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:06.268 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:05 smithi178 ceph-mon[65940]: pgmap v603: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:09.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:08 smithi144 ceph-mon[87436]: pgmap v604: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:08 smithi178 ceph-mon[65940]: pgmap v604: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:10.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:09 smithi178 ceph-mon[65940]: pgmap v605: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:10.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:09 smithi144 ceph-mon[87436]: pgmap v605: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:12.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:12 smithi144 ceph-mon[87436]: pgmap v606: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:12 smithi178 ceph-mon[65940]: pgmap v606: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:14.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:52:14.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:13 smithi144 ceph-mon[87436]: pgmap v607: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:52:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:13 smithi178 ceph-mon[65940]: pgmap v607: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:17.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:16 smithi144 ceph-mon[87436]: pgmap v608: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:16 smithi178 ceph-mon[65940]: pgmap v608: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:18.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:17 smithi144 ceph-mon[87436]: pgmap v609: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:18.366 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:17 smithi178 ceph-mon[65940]: pgmap v609: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:20.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:19 smithi178 ceph-mon[65940]: pgmap v610: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:20.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:19 smithi144 ceph-mon[87436]: pgmap v610: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:22.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:21 smithi178 ceph-mon[65940]: pgmap v611: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:22.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:21 smithi144 ceph-mon[87436]: pgmap v611: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:24.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:23 smithi178 ceph-mon[65940]: pgmap v612: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:24.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:23 smithi144 ceph-mon[87436]: pgmap v612: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:26.274 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:25 smithi178 ceph-mon[65940]: pgmap v613: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:26.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:25 smithi144 ceph-mon[87436]: pgmap v613: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:28.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:27 smithi144 ceph-mon[87436]: pgmap v614: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:28.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:27 smithi178 ceph-mon[65940]: pgmap v614: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:52:29.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:52:30.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:29 smithi178 ceph-mon[65940]: pgmap v615: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:30.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:29 smithi144 ceph-mon[87436]: pgmap v615: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:32.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:31 smithi178 ceph-mon[65940]: pgmap v616: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:32.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:31 smithi144 ceph-mon[87436]: pgmap v616: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:34.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:33 smithi178 ceph-mon[65940]: pgmap v617: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:34.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:33 smithi144 ceph-mon[87436]: pgmap v617: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:34.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.813+0000 7f7fb425d700 1 -- 172.21.15.144:0/2931404363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 msgr2=0x7f7fac104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:34.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.813+0000 7f7fb425d700 1 --2- 172.21.15.144:0/2931404363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac104d40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009910 tx=0x7f7f9c009c20 comp rx=0 tx=0).stop 2024-09-10T11:52:34.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.813+0000 7f7fb425d700 1 -- 172.21.15.144:0/2931404363 shutdown_connections 2024-09-10T11:52:34.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.813+0000 7f7fb425d700 1 --2- 172.21.15.144:0/2931404363 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac104d40 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:34.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.813+0000 7f7fb425d700 1 --2- 172.21.15.144:0/2931404363 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fac1003d0 0x7f7fac1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:34.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.813+0000 7f7fb425d700 1 -- 172.21.15.144:0/2931404363 >> 172.21.15.144:0/2931404363 conn(0x7f7fac0fbc90 msgr2=0x7f7fac0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:34.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.813+0000 7f7fb425d700 1 -- 172.21.15.144:0/2931404363 shutdown_connections 2024-09-10T11:52:34.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.813+0000 7f7fb425d700 1 -- 172.21.15.144:0/2931404363 wait complete. 2024-09-10T11:52:34.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.813+0000 7f7fb425d700 1 Processor -- start 2024-09-10T11:52:34.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.813+0000 7f7fb425d700 1 -- start start 2024-09-10T11:52:34.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.814+0000 7f7fb425d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fac1003d0 0x7f7fac195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:34.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.814+0000 7f7fb425d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:34.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.814+0000 7f7fb425d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac196950 con 0x7f7fac100d70 2024-09-10T11:52:34.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.814+0000 7f7fb425d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac19a6e0 con 0x7f7fac1003d0 2024-09-10T11:52:34.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.814+0000 7f7fb17f8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:34.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.814+0000 7f7fb17f8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50378/0 (socket says 172.21.15.144:50378) 2024-09-10T11:52:34.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.814+0000 7f7fb17f8700 1 -- 172.21.15.144:0/2407924617 learned_addr learned my addr 172.21.15.144:0/2407924617 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:52:34.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.814+0000 7f7fb1ff9700 1 --2- 172.21.15.144:0/2407924617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fac1003d0 0x7f7fac195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:34.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.814+0000 7f7fb17f8700 1 -- 172.21.15.144:0/2407924617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fac1003d0 msgr2=0x7f7fac195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:34.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.814+0000 7f7fb17f8700 1 --2- 172.21.15.144:0/2407924617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fac1003d0 0x7f7fac195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:34.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.814+0000 7f7fb17f8700 1 -- 172.21.15.144:0/2407924617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9c0095f0 con 0x7f7fac100d70 2024-09-10T11:52:34.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.814+0000 7f7fb17f8700 1 --2- 172.21.15.144:0/2407924617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac196270 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0092e0 tx=0x7f7f9c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:34.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.815+0000 7f7fa2ffd700 1 -- 172.21.15.144:0/2407924617 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f9c01d070 con 0x7f7fac100d70 2024-09-10T11:52:34.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.815+0000 7f7fa2ffd700 1 -- 172.21.15.144:0/2407924617 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f9c005dc0 con 0x7f7fac100d70 2024-09-10T11:52:34.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.815+0000 7f7fb425d700 1 -- 172.21.15.144:0/2407924617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fac19a960 con 0x7f7fac100d70 2024-09-10T11:52:34.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.815+0000 7f7fa2ffd700 1 -- 172.21.15.144:0/2407924617 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f9c017400 con 0x7f7fac100d70 2024-09-10T11:52:34.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.815+0000 7f7fb425d700 1 -- 172.21.15.144:0/2407924617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fac19ae50 con 0x7f7fac100d70 2024-09-10T11:52:34.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.816+0000 7f7fb425d700 1 -- 172.21.15.144:0/2407924617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fac108740 con 0x7f7fac100d70 2024-09-10T11:52:34.819 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.816+0000 7f7fa2ffd700 1 -- 172.21.15.144:0/2407924617 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7f9c00f6f0 con 0x7f7fac100d70 2024-09-10T11:52:34.819 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.817+0000 7f7fa2ffd700 1 --2- 172.21.15.144:0/2407924617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f9806e200 0x7f7f980706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:34.819 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.817+0000 7f7fa2ffd700 1 -- 172.21.15.144:0/2407924617 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7f9c08f400 con 0x7f7fac100d70 2024-09-10T11:52:34.819 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.817+0000 7f7fb1ff9700 1 --2- 172.21.15.144:0/2407924617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f9806e200 0x7f7f980706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:34.819 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.817+0000 7f7fb1ff9700 1 --2- 172.21.15.144:0/2407924617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f9806e200 0x7f7f980706b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8005950 tx=0x7f7fa800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:34.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:34.821+0000 7f7fa2ffd700 1 -- 172.21.15.144:0/2407924617 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7f9c058800 con 0x7f7fac100d70 2024-09-10T11:52:35.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.042+0000 7f7fb425d700 1 -- 172.21.15.144:0/2407924617 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7fac02cd20 con 0x7f7f9806e200 2024-09-10T11:52:35.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.044+0000 7f7fa2ffd700 1 -- 172.21.15.144:0/2407924617 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f7fac02cd20 con 0x7f7f9806e200 2024-09-10T11:52:35.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.046+0000 7f7fb425d700 1 -- 172.21.15.144:0/2407924617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f9806e200 msgr2=0x7f7f980706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:35.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.046+0000 7f7fb425d700 1 --2- 172.21.15.144:0/2407924617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f9806e200 0x7f7f980706b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8005950 tx=0x7f7fa800b410 comp rx=0 tx=0).stop 2024-09-10T11:52:35.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.046+0000 7f7fb425d700 1 -- 172.21.15.144:0/2407924617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 msgr2=0x7f7fac196270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:35.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.046+0000 7f7fb425d700 1 --2- 172.21.15.144:0/2407924617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac196270 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0092e0 tx=0x7f7f9c00bd70 comp rx=0 tx=0).stop 2024-09-10T11:52:35.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.046+0000 7f7fb425d700 1 -- 172.21.15.144:0/2407924617 shutdown_connections 2024-09-10T11:52:35.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.046+0000 7f7fb425d700 1 --2- 172.21.15.144:0/2407924617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f9806e200 0x7f7f980706b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.046+0000 7f7fb425d700 1 --2- 172.21.15.144:0/2407924617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fac1003d0 0x7f7fac195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.046+0000 7f7fb425d700 1 --2- 172.21.15.144:0/2407924617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac196270 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.046+0000 7f7fb425d700 1 -- 172.21.15.144:0/2407924617 >> 172.21.15.144:0/2407924617 conn(0x7f7fac0fbc90 msgr2=0x7f7fac0fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:35.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.046+0000 7f7fb425d700 1 -- 172.21.15.144:0/2407924617 shutdown_connections 2024-09-10T11:52:35.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.046+0000 7f7fb425d700 1 -- 172.21.15.144:0/2407924617 wait complete. 2024-09-10T11:52:35.058 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:52:35.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.153+0000 7fa05dc2a700 1 -- 172.21.15.144:0/3940991419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa058100d80 msgr2=0x7fa058104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:35.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.153+0000 7fa05dc2a700 1 --2- 172.21.15.144:0/3940991419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa058100d80 0x7fa058104d50 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa04c009910 tx=0x7fa04c009c20 comp rx=0 tx=0).stop 2024-09-10T11:52:35.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.153+0000 7fa05dc2a700 1 -- 172.21.15.144:0/3940991419 shutdown_connections 2024-09-10T11:52:35.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.153+0000 7fa05dc2a700 1 --2- 172.21.15.144:0/3940991419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa058100d80 0x7fa058104d50 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.153+0000 7fa05dc2a700 1 --2- 172.21.15.144:0/3940991419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0581003e0 0x7fa0581007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.153+0000 7fa05dc2a700 1 -- 172.21.15.144:0/3940991419 >> 172.21.15.144:0/3940991419 conn(0x7fa0580fbca0 msgr2=0x7fa0580fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:35.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.153+0000 7fa05dc2a700 1 -- 172.21.15.144:0/3940991419 shutdown_connections 2024-09-10T11:52:35.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.153+0000 7fa05dc2a700 1 -- 172.21.15.144:0/3940991419 wait complete. 2024-09-10T11:52:35.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.154+0000 7fa05dc2a700 1 Processor -- start 2024-09-10T11:52:35.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.154+0000 7fa05dc2a700 1 -- start start 2024-09-10T11:52:35.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.154+0000 7fa05dc2a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0581003e0 0x7fa058073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:35.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.154+0000 7fa05dc2a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa058100d80 0x7fa058073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:35.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.154+0000 7fa05dc2a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa058073ad0 con 0x7fa0581003e0 2024-09-10T11:52:35.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.154+0000 7fa05dc2a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa058073c10 con 0x7fa058100d80 2024-09-10T11:52:35.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.154+0000 7fa0577fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0581003e0 0x7fa058073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:35.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.155+0000 7fa0577fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0581003e0 0x7fa058073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50404/0 (socket says 172.21.15.144:50404) 2024-09-10T11:52:35.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.155+0000 7fa0577fe700 1 -- 172.21.15.144:0/1834149307 learned_addr learned my addr 172.21.15.144:0/1834149307 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:52:35.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.155+0000 7fa056ffd700 1 --2- 172.21.15.144:0/1834149307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa058100d80 0x7fa058073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:35.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.155+0000 7fa0577fe700 1 -- 172.21.15.144:0/1834149307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa058100d80 msgr2=0x7fa058073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:35.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.155+0000 7fa0577fe700 1 --2- 172.21.15.144:0/1834149307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa058100d80 0x7fa058073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.155+0000 7fa0577fe700 1 -- 172.21.15.144:0/1834149307 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa04c0095f0 con 0x7fa0581003e0 2024-09-10T11:52:35.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.155+0000 7fa0577fe700 1 --2- 172.21.15.144:0/1834149307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0581003e0 0x7fa058073050 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa04800d900 tx=0x7fa04800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:35.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.155+0000 7fa054ff9700 1 -- 172.21.15.144:0/1834149307 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa048004a00 con 0x7fa0581003e0 2024-09-10T11:52:35.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.155+0000 7fa054ff9700 1 -- 172.21.15.144:0/1834149307 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0480054e0 con 0x7fa0581003e0 2024-09-10T11:52:35.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.155+0000 7fa05dc2a700 1 -- 172.21.15.144:0/1834149307 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0581a1f20 con 0x7fa0581003e0 2024-09-10T11:52:35.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.155+0000 7fa054ff9700 1 -- 172.21.15.144:0/1834149307 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa048009d00 con 0x7fa0581003e0 2024-09-10T11:52:35.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.155+0000 7fa05dc2a700 1 -- 172.21.15.144:0/1834149307 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0581a2330 con 0x7fa0581003e0 2024-09-10T11:52:35.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.156+0000 7fa05dc2a700 1 -- 172.21.15.144:0/1834149307 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa058108660 con 0x7fa0581003e0 2024-09-10T11:52:35.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.157+0000 7fa054ff9700 1 -- 172.21.15.144:0/1834149307 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa048005020 con 0x7fa0581003e0 2024-09-10T11:52:35.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.157+0000 7fa054ff9700 1 --2- 172.21.15.144:0/1834149307 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa04006e200 0x7fa0400706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:35.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.157+0000 7fa054ff9700 1 -- 172.21.15.144:0/1834149307 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa04808def0 con 0x7fa0581003e0 2024-09-10T11:52:35.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.157+0000 7fa056ffd700 1 --2- 172.21.15.144:0/1834149307 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa04006e200 0x7fa0400706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:35.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.158+0000 7fa056ffd700 1 --2- 172.21.15.144:0/1834149307 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa04006e200 0x7fa0400706b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa058074840 tx=0x7fa04c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:35.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.162+0000 7fa054ff9700 1 -- 172.21.15.144:0/1834149307 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa048057350 con 0x7fa0581003e0 2024-09-10T11:52:35.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.383+0000 7fa05dc2a700 1 -- 172.21.15.144:0/1834149307 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa058073dd0 con 0x7fa04006e200 2024-09-10T11:52:35.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.384+0000 7fa054ff9700 1 -- 172.21.15.144:0/1834149307 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa058073dd0 con 0x7fa04006e200 2024-09-10T11:52:35.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.387+0000 7fa05dc2a700 1 -- 172.21.15.144:0/1834149307 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa04006e200 msgr2=0x7fa0400706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:35.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.387+0000 7fa05dc2a700 1 --2- 172.21.15.144:0/1834149307 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa04006e200 0x7fa0400706b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa058074840 tx=0x7fa04c01a040 comp rx=0 tx=0).stop 2024-09-10T11:52:35.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.387+0000 7fa05dc2a700 1 -- 172.21.15.144:0/1834149307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0581003e0 msgr2=0x7fa058073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:35.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.387+0000 7fa05dc2a700 1 --2- 172.21.15.144:0/1834149307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0581003e0 0x7fa058073050 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa04800d900 tx=0x7fa04800dc10 comp rx=0 tx=0).stop 2024-09-10T11:52:35.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.387+0000 7fa05dc2a700 1 -- 172.21.15.144:0/1834149307 shutdown_connections 2024-09-10T11:52:35.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.387+0000 7fa05dc2a700 1 --2- 172.21.15.144:0/1834149307 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa04006e200 0x7fa0400706b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.387+0000 7fa05dc2a700 1 --2- 172.21.15.144:0/1834149307 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0581003e0 0x7fa058073050 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.387+0000 7fa05dc2a700 1 --2- 172.21.15.144:0/1834149307 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa058100d80 0x7fa058073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.387+0000 7fa05dc2a700 1 -- 172.21.15.144:0/1834149307 >> 172.21.15.144:0/1834149307 conn(0x7fa0580fbca0 msgr2=0x7fa0581045c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:35.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.387+0000 7fa05dc2a700 1 -- 172.21.15.144:0/1834149307 shutdown_connections 2024-09-10T11:52:35.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.387+0000 7fa05dc2a700 1 -- 172.21.15.144:0/1834149307 wait complete. 2024-09-10T11:52:35.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.492+0000 7f782563a700 1 -- 172.21.15.144:0/3027452958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78200ff640 msgr2=0x7f78200ffab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:35.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.492+0000 7f782563a700 1 --2- 172.21.15.144:0/3027452958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78200ff640 0x7f78200ffab0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7808009910 tx=0x7f7808009c20 comp rx=0 tx=0).stop 2024-09-10T11:52:35.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.492+0000 7f782563a700 1 -- 172.21.15.144:0/3027452958 shutdown_connections 2024-09-10T11:52:35.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.492+0000 7f782563a700 1 --2- 172.21.15.144:0/3027452958 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78200ff640 0x7f78200ffab0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.492+0000 7f782563a700 1 --2- 172.21.15.144:0/3027452958 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78200ff270 0x7f78201062d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.492+0000 7f782563a700 1 -- 172.21.15.144:0/3027452958 >> 172.21.15.144:0/3027452958 conn(0x7f7820074850 msgr2=0x7f7820074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:35.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.492+0000 7f782563a700 1 -- 172.21.15.144:0/3027452958 shutdown_connections 2024-09-10T11:52:35.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.492+0000 7f782563a700 1 -- 172.21.15.144:0/3027452958 wait complete. 2024-09-10T11:52:35.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.493+0000 7f782563a700 1 Processor -- start 2024-09-10T11:52:35.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.493+0000 7f782563a700 1 -- start start 2024-09-10T11:52:35.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.493+0000 7f782563a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78200ff270 0x7f782019c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:35.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.493+0000 7f782563a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78200ff640 0x7f782019c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:35.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.493+0000 7f782563a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f782019cee0 con 0x7f78200ff640 2024-09-10T11:52:35.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.493+0000 7f782563a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78201a0c00 con 0x7f78200ff270 2024-09-10T11:52:35.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.493+0000 7f7817fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78200ff640 0x7f782019c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:35.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.493+0000 7f7817fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78200ff640 0x7f782019c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50410/0 (socket says 172.21.15.144:50410) 2024-09-10T11:52:35.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.493+0000 7f7817fff700 1 -- 172.21.15.144:0/2633382893 learned_addr learned my addr 172.21.15.144:0/2633382893 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:52:35.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.493+0000 7f781effd700 1 --2- 172.21.15.144:0/2633382893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78200ff270 0x7f782019c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:35.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.493+0000 7f7817fff700 1 -- 172.21.15.144:0/2633382893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78200ff270 msgr2=0x7f782019c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:35.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.493+0000 7f7817fff700 1 --2- 172.21.15.144:0/2633382893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78200ff270 0x7f782019c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.493+0000 7f7817fff700 1 -- 172.21.15.144:0/2633382893 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78080095f0 con 0x7f78200ff640 2024-09-10T11:52:35.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.493+0000 7f7817fff700 1 --2- 172.21.15.144:0/2633382893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78200ff640 0x7f782019c800 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7808006010 tx=0x7f780800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:35.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.494+0000 7f781cff9700 1 -- 172.21.15.144:0/2633382893 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f780801d070 con 0x7f78200ff640 2024-09-10T11:52:35.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.494+0000 7f781cff9700 1 -- 172.21.15.144:0/2633382893 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7808005dc0 con 0x7f78200ff640 2024-09-10T11:52:35.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.494+0000 7f782563a700 1 -- 172.21.15.144:0/2633382893 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78201a0ee0 con 0x7f78200ff640 2024-09-10T11:52:35.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.494+0000 7f781cff9700 1 -- 172.21.15.144:0/2633382893 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7808017400 con 0x7f78200ff640 2024-09-10T11:52:35.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.494+0000 7f782563a700 1 -- 172.21.15.144:0/2633382893 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78201a1430 con 0x7f78200ff640 2024-09-10T11:52:35.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.495+0000 7f782563a700 1 -- 172.21.15.144:0/2633382893 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7820103a30 con 0x7f78200ff640 2024-09-10T11:52:35.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.495+0000 7f781cff9700 1 -- 172.21.15.144:0/2633382893 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7808003a60 con 0x7f78200ff640 2024-09-10T11:52:35.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.496+0000 7f781cff9700 1 --2- 172.21.15.144:0/2633382893 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f780c06e200 0x7f780c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:35.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.496+0000 7f781cff9700 1 -- 172.21.15.144:0/2633382893 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f780808f500 con 0x7f78200ff640 2024-09-10T11:52:35.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.496+0000 7f781effd700 1 --2- 172.21.15.144:0/2633382893 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f780c06e200 0x7f780c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:35.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.496+0000 7f781effd700 1 --2- 172.21.15.144:0/2633382893 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f780c06e200 0x7f780c0706b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7810004630 tx=0x7f781000b3c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:35.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.500+0000 7f781cff9700 1 -- 172.21.15.144:0/2633382893 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7808058880 con 0x7f78200ff640 2024-09-10T11:52:35.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.707+0000 7f782563a700 1 -- 172.21.15.144:0/2633382893 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f78201a1710 con 0x7f780c06e200 2024-09-10T11:52:35.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.717+0000 7f781cff9700 1 -- 172.21.15.144:0/2633382893 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f78201a1710 con 0x7f780c06e200 2024-09-10T11:52:35.718 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:52:35.718 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (20m) 14m ago 31m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:52:35.718 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (31m) 14m ago 31m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:52:35.718 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (30m) 14m ago 30m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:52:35.718 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (18m) 14m ago 31m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:52:35.718 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (18m) 14m ago 30m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:52:35.718 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (20m) 14m ago 31m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:52:35.719 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (14m) 14m ago 25m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:52:35.719 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (14m) 14m ago 25m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:52:35.719 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (25m) 14m ago 25m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:52:35.719 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (14m) 14m ago 25m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:52:35.719 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (22m) 14m ago 32m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:52:35.719 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (21m) 14m ago 30m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:52:35.719 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (19m) 14m ago 33m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:52:35.719 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (19m) 14m ago 30m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:52:35.719 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (21m) 14m ago 31m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:52:35.719 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (21m) 14m ago 30m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:52:35.719 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (18m) 14m ago 29m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:52:35.719 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (17m) 14m ago 29m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:52:35.719 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (16m) 14m ago 28m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:52:35.719 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (16m) 14m ago 28m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:52:35.720 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (15m) 14m ago 27m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:52:35.720 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (15m) 14m ago 27m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:52:35.720 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (21m) 14m ago 30m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:52:35.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.719+0000 7f782563a700 1 -- 172.21.15.144:0/2633382893 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f780c06e200 msgr2=0x7f780c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:35.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.719+0000 7f782563a700 1 --2- 172.21.15.144:0/2633382893 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f780c06e200 0x7f780c0706b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f7810004630 tx=0x7f781000b3c0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.719+0000 7f782563a700 1 -- 172.21.15.144:0/2633382893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78200ff640 msgr2=0x7f782019c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:35.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.719+0000 7f782563a700 1 --2- 172.21.15.144:0/2633382893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78200ff640 0x7f782019c800 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7808006010 tx=0x7f780800bd70 comp rx=0 tx=0).stop 2024-09-10T11:52:35.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.719+0000 7f782563a700 1 -- 172.21.15.144:0/2633382893 shutdown_connections 2024-09-10T11:52:35.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.719+0000 7f782563a700 1 --2- 172.21.15.144:0/2633382893 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f780c06e200 0x7f780c0706b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.719+0000 7f782563a700 1 --2- 172.21.15.144:0/2633382893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78200ff270 0x7f782019c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.719+0000 7f782563a700 1 --2- 172.21.15.144:0/2633382893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f78200ff640 0x7f782019c800 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.719+0000 7f782563a700 1 -- 172.21.15.144:0/2633382893 >> 172.21.15.144:0/2633382893 conn(0x7f7820074850 msgr2=0x7f78200fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:35.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.719+0000 7f782563a700 1 -- 172.21.15.144:0/2633382893 shutdown_connections 2024-09-10T11:52:35.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.719+0000 7f782563a700 1 -- 172.21.15.144:0/2633382893 wait complete. 2024-09-10T11:52:35.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.824+0000 7fe892066700 1 -- 172.21.15.144:0/2628155588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe88c0ffd30 msgr2=0x7fe88c103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:35.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.824+0000 7fe892066700 1 --2- 172.21.15.144:0/2628155588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe88c0ffd30 0x7fe88c103ce0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fe87c009910 tx=0x7fe87c009c20 comp rx=0 tx=0).stop 2024-09-10T11:52:35.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.824+0000 7fe892066700 1 -- 172.21.15.144:0/2628155588 shutdown_connections 2024-09-10T11:52:35.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.824+0000 7fe892066700 1 --2- 172.21.15.144:0/2628155588 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe88c0ffd30 0x7fe88c103ce0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.824+0000 7fe892066700 1 --2- 172.21.15.144:0/2628155588 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe88c0ff390 0x7fe88c0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.824+0000 7fe892066700 1 -- 172.21.15.144:0/2628155588 >> 172.21.15.144:0/2628155588 conn(0x7fe88c074f50 msgr2=0x7fe88c075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:35.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.824+0000 7fe892066700 1 -- 172.21.15.144:0/2628155588 shutdown_connections 2024-09-10T11:52:35.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.824+0000 7fe892066700 1 -- 172.21.15.144:0/2628155588 wait complete. 2024-09-10T11:52:35.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.825+0000 7fe892066700 1 Processor -- start 2024-09-10T11:52:35.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.825+0000 7fe892066700 1 -- start start 2024-09-10T11:52:35.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.825+0000 7fe892066700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe88c0ff390 0x7fe88c196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:35.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.825+0000 7fe892066700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe88c0ffd30 0x7fe88c1970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:35.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.825+0000 7fe892066700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c1976e0 con 0x7fe88c0ff390 2024-09-10T11:52:35.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.825+0000 7fe892066700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe88c197820 con 0x7fe88c0ffd30 2024-09-10T11:52:35.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.825+0000 7fe88b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe88c0ff390 0x7fe88c196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:35.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.826+0000 7fe88affd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe88c0ffd30 0x7fe88c1970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:35.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.826+0000 7fe88b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe88c0ff390 0x7fe88c196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50436/0 (socket says 172.21.15.144:50436) 2024-09-10T11:52:35.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.826+0000 7fe88b7fe700 1 -- 172.21.15.144:0/2729328226 learned_addr learned my addr 172.21.15.144:0/2729328226 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:52:35.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.826+0000 7fe88affd700 1 -- 172.21.15.144:0/2729328226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe88c0ff390 msgr2=0x7fe88c196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:35.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.826+0000 7fe88affd700 1 --2- 172.21.15.144:0/2729328226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe88c0ff390 0x7fe88c196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:35.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.826+0000 7fe88affd700 1 -- 172.21.15.144:0/2729328226 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87c0095f0 con 0x7fe88c0ffd30 2024-09-10T11:52:35.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.826+0000 7fe88b7fe700 1 --2- 172.21.15.144:0/2729328226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe88c0ff390 0x7fe88c196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:52:35.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.826+0000 7fe88affd700 1 --2- 172.21.15.144:0/2729328226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe88c0ffd30 0x7fe88c1970c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe88c073930 tx=0x7fe87c00bd70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:35.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.826+0000 7fe888ff9700 1 -- 172.21.15.144:0/2729328226 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe87c01d070 con 0x7fe88c0ffd30 2024-09-10T11:52:35.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.826+0000 7fe892066700 1 -- 172.21.15.144:0/2729328226 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe88c19fbf0 con 0x7fe88c0ffd30 2024-09-10T11:52:35.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.826+0000 7fe892066700 1 -- 172.21.15.144:0/2729328226 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe88c1a0000 con 0x7fe88c0ffd30 2024-09-10T11:52:35.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.826+0000 7fe888ff9700 1 -- 172.21.15.144:0/2729328226 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe87c00f460 con 0x7fe88c0ffd30 2024-09-10T11:52:35.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.827+0000 7fe888ff9700 1 -- 172.21.15.144:0/2729328226 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe87c017630 con 0x7fe88c0ffd30 2024-09-10T11:52:35.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.827+0000 7fe892066700 1 -- 172.21.15.144:0/2729328226 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe88c107600 con 0x7fe88c0ffd30 2024-09-10T11:52:35.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.828+0000 7fe888ff9700 1 -- 172.21.15.144:0/2729328226 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe87c00fab0 con 0x7fe88c0ffd30 2024-09-10T11:52:35.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.829+0000 7fe888ff9700 1 --2- 172.21.15.144:0/2729328226 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe87406e200 0x7fe8740706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:35.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.829+0000 7fe888ff9700 1 -- 172.21.15.144:0/2729328226 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe87c08f3d0 con 0x7fe88c0ffd30 2024-09-10T11:52:35.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.829+0000 7fe88b7fe700 1 --2- 172.21.15.144:0/2729328226 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe87406e200 0x7fe8740706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:35.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.829+0000 7fe88b7fe700 1 --2- 172.21.15.144:0/2729328226 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe87406e200 0x7fe8740706b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe880005950 tx=0x7fe88000a400 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:35.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:35.834+0000 7fe888ff9700 1 -- 172.21.15.144:0/2729328226 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe87c058830 con 0x7fe88c0ffd30 2024-09-10T11:52:36.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.126+0000 7fe892066700 1 -- 172.21.15.144:0/2729328226 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe88c02cc30 con 0x7fe88c0ffd30 2024-09-10T11:52:36.127 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:35 smithi144 ceph-mon[87436]: from='client.35156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:36.127 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:35 smithi144 ceph-mon[87436]: from='client.35160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:36.127 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:35 smithi144 ceph-mon[87436]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:36.128 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:35 smithi144 ceph-mon[87436]: pgmap v618: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:36.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.126+0000 7fe888ff9700 1 -- 172.21.15.144:0/2729328226 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7fe87c05bef0 con 0x7fe88c0ffd30 2024-09-10T11:52:36.128 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:52:36.128 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:52:36.128 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:52:36.128 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:52:36.128 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:52:36.129 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:52:36.129 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:52:36.129 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:52:36.129 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:52:36.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:52:36.133 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:52:36.133 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:52:36.133 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:52:36.133 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:52:36.133 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:52:36.133 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:52:36.133 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:52:36.133 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:52:36.133 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.129+0000 7fe892066700 1 -- 172.21.15.144:0/2729328226 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe87406e200 msgr2=0x7fe8740706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.129+0000 7fe892066700 1 --2- 172.21.15.144:0/2729328226 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe87406e200 0x7fe8740706b0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe880005950 tx=0x7fe88000a400 comp rx=0 tx=0).stop 2024-09-10T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.129+0000 7fe892066700 1 -- 172.21.15.144:0/2729328226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe88c0ffd30 msgr2=0x7fe88c1970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.129+0000 7fe892066700 1 --2- 172.21.15.144:0/2729328226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe88c0ffd30 0x7fe88c1970c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe88c073930 tx=0x7fe87c00bd70 comp rx=0 tx=0).stop 2024-09-10T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.129+0000 7fe892066700 1 -- 172.21.15.144:0/2729328226 shutdown_connections 2024-09-10T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.129+0000 7fe892066700 1 --2- 172.21.15.144:0/2729328226 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe87406e200 0x7fe8740706b0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.129+0000 7fe892066700 1 --2- 172.21.15.144:0/2729328226 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe88c0ff390 0x7fe88c196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.129+0000 7fe892066700 1 --2- 172.21.15.144:0/2729328226 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe88c0ffd30 0x7fe88c1970c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.129+0000 7fe892066700 1 -- 172.21.15.144:0/2729328226 >> 172.21.15.144:0/2729328226 conn(0x7fe88c074f50 msgr2=0x7fe88c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:36.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.129+0000 7fe892066700 1 -- 172.21.15.144:0/2729328226 shutdown_connections 2024-09-10T11:52:36.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.129+0000 7fe892066700 1 -- 172.21.15.144:0/2729328226 wait complete. 2024-09-10T11:52:36.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.234+0000 7ff4a32d5700 1 -- 172.21.15.144:0/1520401833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c1003d0 msgr2=0x7ff49c1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:36.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.234+0000 7ff4a32d5700 1 --2- 172.21.15.144:0/1520401833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c1003d0 0x7ff49c1007a0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7ff498009910 tx=0x7ff498009c20 comp rx=0 tx=0).stop 2024-09-10T11:52:36.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.234+0000 7ff4a32d5700 1 -- 172.21.15.144:0/1520401833 shutdown_connections 2024-09-10T11:52:36.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.234+0000 7ff4a32d5700 1 --2- 172.21.15.144:0/1520401833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff49c100d70 0x7ff49c104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.234+0000 7ff4a32d5700 1 --2- 172.21.15.144:0/1520401833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c1003d0 0x7ff49c1007a0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.234+0000 7ff4a32d5700 1 -- 172.21.15.144:0/1520401833 >> 172.21.15.144:0/1520401833 conn(0x7ff49c0fbc50 msgr2=0x7ff49c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:36.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.234+0000 7ff4a32d5700 1 -- 172.21.15.144:0/1520401833 shutdown_connections 2024-09-10T11:52:36.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.234+0000 7ff4a32d5700 1 -- 172.21.15.144:0/1520401833 wait complete. 2024-09-10T11:52:36.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.234+0000 7ff4a32d5700 1 Processor -- start 2024-09-10T11:52:36.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.234+0000 7ff4a32d5700 1 -- start start 2024-09-10T11:52:36.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4a32d5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff49c1003d0 0x7ff49c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:36.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4a32d5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c100d70 0x7ff49c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:36.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4a32d5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c073ad0 con 0x7ff49c100d70 2024-09-10T11:52:36.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4a32d5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c073c10 con 0x7ff49c1003d0 2024-09-10T11:52:36.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4a0870700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c100d70 0x7ff49c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:36.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4a0870700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c100d70 0x7ff49c073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50454/0 (socket says 172.21.15.144:50454) 2024-09-10T11:52:36.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4a0870700 1 -- 172.21.15.144:0/2816230658 learned_addr learned my addr 172.21.15.144:0/2816230658 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:52:36.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4a1071700 1 --2- 172.21.15.144:0/2816230658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff49c1003d0 0x7ff49c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:36.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4a0870700 1 -- 172.21.15.144:0/2816230658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff49c1003d0 msgr2=0x7ff49c073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:36.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4a0870700 1 --2- 172.21.15.144:0/2816230658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff49c1003d0 0x7ff49c073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4a0870700 1 -- 172.21.15.144:0/2816230658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4980095f0 con 0x7ff49c100d70 2024-09-10T11:52:36.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4a1071700 1 --2- 172.21.15.144:0/2816230658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff49c1003d0 0x7ff49c073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:52:36.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4a0870700 1 --2- 172.21.15.144:0/2816230658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c100d70 0x7ff49c073590 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00d900 tx=0x7ff48c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:36.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4927fc700 1 -- 172.21.15.144:0/2816230658 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff48c009900 con 0x7ff49c100d70 2024-09-10T11:52:36.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4927fc700 1 -- 172.21.15.144:0/2816230658 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff48c010460 con 0x7ff49c100d70 2024-09-10T11:52:36.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4a32d5700 1 -- 172.21.15.144:0/2816230658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c1a1f80 con 0x7ff49c100d70 2024-09-10T11:52:36.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4927fc700 1 -- 172.21.15.144:0/2816230658 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff48c00f5f0 con 0x7ff49c100d70 2024-09-10T11:52:36.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.235+0000 7ff4a32d5700 1 -- 172.21.15.144:0/2816230658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c1a2390 con 0x7ff49c100d70 2024-09-10T11:52:36.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.236+0000 7ff4a32d5700 1 -- 172.21.15.144:0/2816230658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49c1086e0 con 0x7ff49c100d70 2024-09-10T11:52:36.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.237+0000 7ff4927fc700 1 -- 172.21.15.144:0/2816230658 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff48c0105d0 con 0x7ff49c100d70 2024-09-10T11:52:36.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.237+0000 7ff4927fc700 1 --2- 172.21.15.144:0/2816230658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff48806e200 0x7ff4880706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:36.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.237+0000 7ff4927fc700 1 -- 172.21.15.144:0/2816230658 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff48c08df20 con 0x7ff49c100d70 2024-09-10T11:52:36.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.237+0000 7ff4a1071700 1 --2- 172.21.15.144:0/2816230658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff48806e200 0x7ff4880706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:36.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.238+0000 7ff4a1071700 1 --2- 172.21.15.144:0/2816230658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff48806e200 0x7ff4880706b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff498005b40 tx=0x7ff49801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:36.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.242+0000 7ff4927fc700 1 -- 172.21.15.144:0/2816230658 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff48c0c31b0 con 0x7ff49c100d70 2024-09-10T11:52:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:35 smithi178 ceph-mon[65940]: from='client.35156 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:35 smithi178 ceph-mon[65940]: from='client.35160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:35 smithi178 ceph-mon[65940]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:35 smithi178 ceph-mon[65940]: pgmap v618: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:36.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.493+0000 7ff4a32d5700 1 -- 172.21.15.144:0/2816230658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff49c04ea50 con 0x7ff49c100d70 2024-09-10T11:52:36.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.493+0000 7ff4927fc700 1 -- 172.21.15.144:0/2816230658 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7ff48c057300 con 0x7ff49c100d70 2024-09-10T11:52:36.495 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:52:36.495 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:52:36.495 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:52:36.495 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:52:36.495 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:52:36.495 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:52:36.495 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:52:36.495 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:52:36.495 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:52:36.495 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:52:36.495 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:52:36.495 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:52:36.496 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:52:36.496 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:52:36.496 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:52:36.496 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:52:36.496 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:52:36.496 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:52:36.496 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:52:36.496 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:52:36.496 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:52:36.496 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:52:36.496 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:52:36.496 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:52:36.496 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:52:36.496 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:52:36.497 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:52:36.497 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:52:36.497 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:52:36.497 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:52:36.497 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:52:36.497 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:52:36.497 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:52:36.497 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:52:36.497 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:52:36.497 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:52:36.497 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:52:36.497 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:52:36.497 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:52:36.497 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:52:36.498 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:52:36.498 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:52:36.498 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:52:36.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.496+0000 7ff4a32d5700 1 -- 172.21.15.144:0/2816230658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff48806e200 msgr2=0x7ff4880706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:36.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.496+0000 7ff4a32d5700 1 --2- 172.21.15.144:0/2816230658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff48806e200 0x7ff4880706b0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7ff498005b40 tx=0x7ff49801a040 comp rx=0 tx=0).stop 2024-09-10T11:52:36.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.496+0000 7ff4a32d5700 1 -- 172.21.15.144:0/2816230658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c100d70 msgr2=0x7ff49c073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:36.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.496+0000 7ff4a32d5700 1 --2- 172.21.15.144:0/2816230658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c100d70 0x7ff49c073590 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00d900 tx=0x7ff48c00dcc0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.496+0000 7ff4a32d5700 1 -- 172.21.15.144:0/2816230658 shutdown_connections 2024-09-10T11:52:36.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.496+0000 7ff4a32d5700 1 --2- 172.21.15.144:0/2816230658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff48806e200 0x7ff4880706b0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.496+0000 7ff4a32d5700 1 --2- 172.21.15.144:0/2816230658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff49c1003d0 0x7ff49c073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.496+0000 7ff4a32d5700 1 --2- 172.21.15.144:0/2816230658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff49c100d70 0x7ff49c073590 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.496+0000 7ff4a32d5700 1 -- 172.21.15.144:0/2816230658 >> 172.21.15.144:0/2816230658 conn(0x7ff49c0fbc50 msgr2=0x7ff49c0fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:36.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.496+0000 7ff4a32d5700 1 -- 172.21.15.144:0/2816230658 shutdown_connections 2024-09-10T11:52:36.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.496+0000 7ff4a32d5700 1 -- 172.21.15.144:0/2816230658 wait complete. 2024-09-10T11:52:36.499 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:52:36.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.601+0000 7fa84548e700 1 -- 172.21.15.144:0/4177976816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8400ffba0 msgr2=0x7fa84010a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:36.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.601+0000 7fa84548e700 1 --2- 172.21.15.144:0/4177976816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8400ffba0 0x7fa84010a610 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa828009910 tx=0x7fa828009c20 comp rx=0 tx=0).stop 2024-09-10T11:52:36.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.601+0000 7fa84548e700 1 -- 172.21.15.144:0/4177976816 shutdown_connections 2024-09-10T11:52:36.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.601+0000 7fa84548e700 1 --2- 172.21.15.144:0/4177976816 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8400ffba0 0x7fa84010a610 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.601+0000 7fa84548e700 1 --2- 172.21.15.144:0/4177976816 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8400ff290 0x7fa8400ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.601+0000 7fa84548e700 1 -- 172.21.15.144:0/4177976816 >> 172.21.15.144:0/4177976816 conn(0x7fa840074850 msgr2=0x7fa840074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:36.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.601+0000 7fa84548e700 1 -- 172.21.15.144:0/4177976816 shutdown_connections 2024-09-10T11:52:36.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.601+0000 7fa84548e700 1 -- 172.21.15.144:0/4177976816 wait complete. 2024-09-10T11:52:36.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.602+0000 7fa84548e700 1 Processor -- start 2024-09-10T11:52:36.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.602+0000 7fa84548e700 1 -- start start 2024-09-10T11:52:36.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.602+0000 7fa84548e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8400ff290 0x7fa840195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:36.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.602+0000 7fa84548e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8400ffba0 0x7fa8401962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:36.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.602+0000 7fa84548e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa840196980 con 0x7fa8400ff290 2024-09-10T11:52:36.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.602+0000 7fa84548e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84019a710 con 0x7fa8400ffba0 2024-09-10T11:52:36.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.602+0000 7fa83effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8400ff290 0x7fa840195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:36.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.602+0000 7fa83effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8400ff290 0x7fa840195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50470/0 (socket says 172.21.15.144:50470) 2024-09-10T11:52:36.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.602+0000 7fa83effd700 1 -- 172.21.15.144:0/3949660498 learned_addr learned my addr 172.21.15.144:0/3949660498 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:52:36.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.602+0000 7fa83e7fc700 1 --2- 172.21.15.144:0/3949660498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8400ffba0 0x7fa8401962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:36.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.602+0000 7fa83effd700 1 -- 172.21.15.144:0/3949660498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8400ffba0 msgr2=0x7fa8401962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:36.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.602+0000 7fa83effd700 1 --2- 172.21.15.144:0/3949660498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8400ffba0 0x7fa8401962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.602+0000 7fa83effd700 1 -- 172.21.15.144:0/3949660498 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8280095f0 con 0x7fa8400ff290 2024-09-10T11:52:36.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.603+0000 7fa83effd700 1 --2- 172.21.15.144:0/3949660498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8400ff290 0x7fa840195d60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa838007e90 tx=0x7fa83800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:36.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.603+0000 7fa837fff700 1 -- 172.21.15.144:0/3949660498 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa838017070 con 0x7fa8400ff290 2024-09-10T11:52:36.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.603+0000 7fa837fff700 1 -- 172.21.15.144:0/3949660498 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa83800f040 con 0x7fa8400ff290 2024-09-10T11:52:36.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.603+0000 7fa84548e700 1 -- 172.21.15.144:0/3949660498 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa84019a9f0 con 0x7fa8400ff290 2024-09-10T11:52:36.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.603+0000 7fa837fff700 1 -- 172.21.15.144:0/3949660498 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa838004850 con 0x7fa8400ff290 2024-09-10T11:52:36.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.603+0000 7fa84548e700 1 -- 172.21.15.144:0/3949660498 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84019af40 con 0x7fa8400ff290 2024-09-10T11:52:36.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.604+0000 7fa84548e700 1 -- 172.21.15.144:0/3949660498 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa840107d10 con 0x7fa8400ff290 2024-09-10T11:52:36.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.605+0000 7fa837fff700 1 -- 172.21.15.144:0/3949660498 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa838003a00 con 0x7fa8400ff290 2024-09-10T11:52:36.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.605+0000 7fa837fff700 1 --2- 172.21.15.144:0/3949660498 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa82c06e1b0 0x7fa82c070660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:36.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.605+0000 7fa83e7fc700 1 --2- 172.21.15.144:0/3949660498 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa82c06e1b0 0x7fa82c070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:36.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.605+0000 7fa837fff700 1 -- 172.21.15.144:0/3949660498 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa83808db70 con 0x7fa8400ff290 2024-09-10T11:52:36.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.606+0000 7fa83e7fc700 1 --2- 172.21.15.144:0/3949660498 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa82c06e1b0 0x7fa82c070660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa840197380 tx=0x7fa82801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:36.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.610+0000 7fa837fff700 1 -- 172.21.15.144:0/3949660498 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa838056f50 con 0x7fa8400ff290 2024-09-10T11:52:36.831 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.830+0000 7fa84548e700 1 -- 172.21.15.144:0/3949660498 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa84019b220 con 0x7fa82c06e1b0 2024-09-10T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.831+0000 7fa837fff700 1 -- 172.21.15.144:0/3949660498 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa84019b220 con 0x7fa82c06e1b0 2024-09-10T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:52:36.832 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:52:36.833 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:52:36.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.833+0000 7fa84548e700 1 -- 172.21.15.144:0/3949660498 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa82c06e1b0 msgr2=0x7fa82c070660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:36.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.833+0000 7fa84548e700 1 --2- 172.21.15.144:0/3949660498 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa82c06e1b0 0x7fa82c070660 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa840197380 tx=0x7fa82801a040 comp rx=0 tx=0).stop 2024-09-10T11:52:36.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.833+0000 7fa84548e700 1 -- 172.21.15.144:0/3949660498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8400ff290 msgr2=0x7fa840195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:36.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.833+0000 7fa84548e700 1 --2- 172.21.15.144:0/3949660498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8400ff290 0x7fa840195d60 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa838007e90 tx=0x7fa83800d3b0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.833+0000 7fa84548e700 1 -- 172.21.15.144:0/3949660498 shutdown_connections 2024-09-10T11:52:36.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.833+0000 7fa84548e700 1 --2- 172.21.15.144:0/3949660498 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa82c06e1b0 0x7fa82c070660 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.833+0000 7fa84548e700 1 --2- 172.21.15.144:0/3949660498 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa8400ff290 0x7fa840195d60 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.833+0000 7fa84548e700 1 --2- 172.21.15.144:0/3949660498 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa8400ffba0 0x7fa8401962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.833+0000 7fa84548e700 1 -- 172.21.15.144:0/3949660498 >> 172.21.15.144:0/3949660498 conn(0x7fa840074850 msgr2=0x7fa840104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:36.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.833+0000 7fa84548e700 1 -- 172.21.15.144:0/3949660498 shutdown_connections 2024-09-10T11:52:36.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.833+0000 7fa84548e700 1 -- 172.21.15.144:0/3949660498 wait complete. 2024-09-10T11:52:36.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.938+0000 7fb492db4700 1 -- 172.21.15.144:0/3646178439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb48c1003d0 msgr2=0x7fb48c1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:36.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.938+0000 7fb492db4700 1 --2- 172.21.15.144:0/3646178439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb48c1003d0 0x7fb48c1007a0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb480009910 tx=0x7fb480009c20 comp rx=0 tx=0).stop 2024-09-10T11:52:36.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.938+0000 7fb492db4700 1 -- 172.21.15.144:0/3646178439 shutdown_connections 2024-09-10T11:52:36.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.938+0000 7fb492db4700 1 --2- 172.21.15.144:0/3646178439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb48c100d70 0x7fb48c104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.939+0000 7fb492db4700 1 --2- 172.21.15.144:0/3646178439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb48c1003d0 0x7fb48c1007a0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.939+0000 7fb492db4700 1 -- 172.21.15.144:0/3646178439 >> 172.21.15.144:0/3646178439 conn(0x7fb48c0fbc50 msgr2=0x7fb48c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:36.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.939+0000 7fb492db4700 1 -- 172.21.15.144:0/3646178439 shutdown_connections 2024-09-10T11:52:36.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.939+0000 7fb492db4700 1 -- 172.21.15.144:0/3646178439 wait complete. 2024-09-10T11:52:36.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.939+0000 7fb492db4700 1 Processor -- start 2024-09-10T11:52:36.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.939+0000 7fb492db4700 1 -- start start 2024-09-10T11:52:36.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.939+0000 7fb492db4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb48c1003d0 0x7fb48c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:36.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.939+0000 7fb492db4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb48c100d70 0x7fb48c1962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:36.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.939+0000 7fb492db4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c1969d0 con 0x7fb48c1003d0 2024-09-10T11:52:36.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.939+0000 7fb492db4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48c19a760 con 0x7fb48c100d70 2024-09-10T11:52:36.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.939+0000 7fb490b50700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb48c1003d0 0x7fb48c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:36.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.940+0000 7fb490b50700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb48c1003d0 0x7fb48c195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50488/0 (socket says 172.21.15.144:50488) 2024-09-10T11:52:36.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.940+0000 7fb490b50700 1 -- 172.21.15.144:0/1915428822 learned_addr learned my addr 172.21.15.144:0/1915428822 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:52:36.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.940+0000 7fb48bfff700 1 --2- 172.21.15.144:0/1915428822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb48c100d70 0x7fb48c1962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:36.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.940+0000 7fb490b50700 1 -- 172.21.15.144:0/1915428822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb48c100d70 msgr2=0x7fb48c1962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:36.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.940+0000 7fb490b50700 1 --2- 172.21.15.144:0/1915428822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb48c100d70 0x7fb48c1962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:36.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.940+0000 7fb490b50700 1 -- 172.21.15.144:0/1915428822 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4800095f0 con 0x7fb48c1003d0 2024-09-10T11:52:36.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.940+0000 7fb490b50700 1 --2- 172.21.15.144:0/1915428822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb48c1003d0 0x7fb48c195db0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb480006010 tx=0x7fb48000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:36.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.942+0000 7fb489ffb700 1 -- 172.21.15.144:0/1915428822 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb48001d070 con 0x7fb48c1003d0 2024-09-10T11:52:36.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.942+0000 7fb489ffb700 1 -- 172.21.15.144:0/1915428822 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb480005dc0 con 0x7fb48c1003d0 2024-09-10T11:52:36.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.942+0000 7fb489ffb700 1 -- 172.21.15.144:0/1915428822 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb480017400 con 0x7fb48c1003d0 2024-09-10T11:52:36.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.942+0000 7fb492db4700 1 -- 172.21.15.144:0/1915428822 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb48c19a9e0 con 0x7fb48c1003d0 2024-09-10T11:52:36.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.943+0000 7fb492db4700 1 -- 172.21.15.144:0/1915428822 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb48c19aed0 con 0x7fb48c1003d0 2024-09-10T11:52:36.945 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.944+0000 7fb492db4700 1 -- 172.21.15.144:0/1915428822 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48c1086e0 con 0x7fb48c1003d0 2024-09-10T11:52:36.945 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.944+0000 7fb489ffb700 1 -- 172.21.15.144:0/1915428822 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb480003a60 con 0x7fb48c1003d0 2024-09-10T11:52:36.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.945+0000 7fb489ffb700 1 --2- 172.21.15.144:0/1915428822 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb47c06e200 0x7fb47c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:52:36.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.945+0000 7fb489ffb700 1 -- 172.21.15.144:0/1915428822 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb48008f450 con 0x7fb48c1003d0 2024-09-10T11:52:36.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.945+0000 7fb48bfff700 1 --2- 172.21.15.144:0/1915428822 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb47c06e200 0x7fb47c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:52:36.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.945+0000 7fb48bfff700 1 --2- 172.21.15.144:0/1915428822 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb47c06e200 0x7fb47c0706b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb48c1973d0 tx=0x7fb47800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:52:36.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:36.949+0000 7fb489ffb700 1 -- 172.21.15.144:0/1915428822 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb4800588b0 con 0x7fb48c1003d0 2024-09-10T11:52:37.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:36 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2729328226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:52:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:36 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2816230658' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:52:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:36 smithi144 ceph-mon[87436]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:37.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:37.236+0000 7fb492db4700 1 -- 172.21.15.144:0/1915428822 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb48c02cc30 con 0x7fb48c1003d0 2024-09-10T11:52:37.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:37.236+0000 7fb489ffb700 1 -- 172.21.15.144:0/1915428822 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7fb48005bf70 con 0x7fb48c1003d0 2024-09-10T11:52:37.238 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:52:37.238 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:52:37.238 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:52:37.238 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:52:37.238 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:52:37.238 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:52:37.238 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:52:37.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:37.238+0000 7fb492db4700 1 -- 172.21.15.144:0/1915428822 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb47c06e200 msgr2=0x7fb47c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:37.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:37.238+0000 7fb492db4700 1 --2- 172.21.15.144:0/1915428822 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb47c06e200 0x7fb47c0706b0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb48c1973d0 tx=0x7fb47800b410 comp rx=0 tx=0).stop 2024-09-10T11:52:37.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:37.238+0000 7fb492db4700 1 -- 172.21.15.144:0/1915428822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb48c1003d0 msgr2=0x7fb48c195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:52:37.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:37.239+0000 7fb492db4700 1 --2- 172.21.15.144:0/1915428822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb48c1003d0 0x7fb48c195db0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb480006010 tx=0x7fb48000bd70 comp rx=0 tx=0).stop 2024-09-10T11:52:37.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:37.239+0000 7fb492db4700 1 -- 172.21.15.144:0/1915428822 shutdown_connections 2024-09-10T11:52:37.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:37.239+0000 7fb492db4700 1 --2- 172.21.15.144:0/1915428822 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb47c06e200 0x7fb47c0706b0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:37.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:37.239+0000 7fb492db4700 1 --2- 172.21.15.144:0/1915428822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb48c1003d0 0x7fb48c195db0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:37.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:37.239+0000 7fb492db4700 1 --2- 172.21.15.144:0/1915428822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb48c100d70 0x7fb48c1962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:52:37.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:37.239+0000 7fb492db4700 1 -- 172.21.15.144:0/1915428822 >> 172.21.15.144:0/1915428822 conn(0x7fb48c0fbc50 msgr2=0x7fb48c0fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:52:37.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:37.239+0000 7fb492db4700 1 -- 172.21.15.144:0/1915428822 shutdown_connections 2024-09-10T11:52:37.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:52:37.239+0000 7fb492db4700 1 -- 172.21.15.144:0/1915428822 wait complete. 2024-09-10T11:52:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:36 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2729328226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:52:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:36 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2816230658' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:52:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:36 smithi178 ceph-mon[65940]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:52:38.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:37 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1915428822' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:52:38.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:37 smithi144 ceph-mon[87436]: pgmap v619: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:38.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:37 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1915428822' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:52:38.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:37 smithi178 ceph-mon[65940]: pgmap v619: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:40.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:39 smithi178 ceph-mon[65940]: pgmap v620: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:40.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:39 smithi144 ceph-mon[87436]: pgmap v620: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:41 smithi178 ceph-mon[65940]: pgmap v621: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:42.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:41 smithi144 ceph-mon[87436]: pgmap v621: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:52:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:52:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:44 smithi144 ceph-mon[87436]: pgmap v622: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:44 smithi178 ceph-mon[65940]: pgmap v622: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:46.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:45 smithi178 ceph-mon[65940]: pgmap v623: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:46.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:45 smithi144 ceph-mon[87436]: pgmap v623: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:48.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:47 smithi178 ceph-mon[65940]: pgmap v624: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:48.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:47 smithi144 ceph-mon[87436]: pgmap v624: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:50.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:49 smithi178 ceph-mon[65940]: pgmap v625: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:50.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:49 smithi144 ceph-mon[87436]: pgmap v625: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:52.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:51 smithi178 ceph-mon[65940]: pgmap v626: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:52.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:51 smithi144 ceph-mon[87436]: pgmap v626: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:53 smithi178 ceph-mon[65940]: pgmap v627: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:54.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:53 smithi144 ceph-mon[87436]: pgmap v627: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:56.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:55 smithi178 ceph-mon[65940]: pgmap v628: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:56.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:55 smithi144 ceph-mon[87436]: pgmap v628: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:57 smithi178 ceph-mon[65940]: pgmap v629: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:58.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:57 smithi144 ceph-mon[87436]: pgmap v629: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:52:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:52:59.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:53:00.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:52:59 smithi178 ceph-mon[65940]: pgmap v630: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:00.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:52:59 smithi144 ceph-mon[87436]: pgmap v630: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:02.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:01 smithi178 ceph-mon[65940]: pgmap v631: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:02.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:01 smithi144 ceph-mon[87436]: pgmap v631: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:03 smithi178 ceph-mon[65940]: pgmap v632: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:04.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:03 smithi144 ceph-mon[87436]: pgmap v632: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:06.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:05 smithi178 ceph-mon[65940]: pgmap v633: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:06.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:05 smithi144 ceph-mon[87436]: pgmap v633: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:07.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.353+0000 7f879fb41700 1 -- 172.21.15.144:0/1253685142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87980ff390 msgr2=0x7f87980ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:07.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.353+0000 7f879fb41700 1 --2- 172.21.15.144:0/1253685142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87980ff390 0x7f87980ff760 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f879400b0a0 tx=0x7f879400b3b0 comp rx=0 tx=0).stop 2024-09-10T11:53:07.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.353+0000 7f879fb41700 1 -- 172.21.15.144:0/1253685142 shutdown_connections 2024-09-10T11:53:07.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.353+0000 7f879fb41700 1 --2- 172.21.15.144:0/1253685142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87980ffd30 0x7f8798103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:07.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.353+0000 7f879fb41700 1 --2- 172.21.15.144:0/1253685142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87980ff390 0x7f87980ff760 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:07.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.353+0000 7f879fb41700 1 -- 172.21.15.144:0/1253685142 >> 172.21.15.144:0/1253685142 conn(0x7f8798074f50 msgr2=0x7f8798075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:07.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.353+0000 7f879fb41700 1 -- 172.21.15.144:0/1253685142 shutdown_connections 2024-09-10T11:53:07.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.353+0000 7f879fb41700 1 -- 172.21.15.144:0/1253685142 wait complete. 2024-09-10T11:53:07.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.354+0000 7f879fb41700 1 Processor -- start 2024-09-10T11:53:07.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.354+0000 7f879fb41700 1 -- start start 2024-09-10T11:53:07.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.354+0000 7f879fb41700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87980ff390 0x7f8798196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:07.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.354+0000 7f879fb41700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87980ffd30 0x7f8798197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:07.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.354+0000 7f879fb41700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8798197860 con 0x7f87980ffd30 2024-09-10T11:53:07.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.354+0000 7f879fb41700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87981979d0 con 0x7f87980ff390 2024-09-10T11:53:07.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.354+0000 7f879d0dc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87980ffd30 0x7f8798197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:07.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.354+0000 7f879d0dc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87980ffd30 0x7f8798197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50460/0 (socket says 172.21.15.144:50460) 2024-09-10T11:53:07.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.354+0000 7f879d0dc700 1 -- 172.21.15.144:0/2305455717 learned_addr learned my addr 172.21.15.144:0/2305455717 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:53:07.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.354+0000 7f879d8dd700 1 --2- 172.21.15.144:0/2305455717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87980ff390 0x7f8798196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:07.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.355+0000 7f879d0dc700 1 -- 172.21.15.144:0/2305455717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87980ff390 msgr2=0x7f8798196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:07.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.355+0000 7f879d0dc700 1 --2- 172.21.15.144:0/2305455717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87980ff390 0x7f8798196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:07.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.355+0000 7f879d0dc700 1 -- 172.21.15.144:0/2305455717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8794009d00 con 0x7f87980ffd30 2024-09-10T11:53:07.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.355+0000 7f879d0dc700 1 --2- 172.21.15.144:0/2305455717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87980ffd30 0x7f8798197180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f878800d900 tx=0x7f878800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:07.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.355+0000 7f878effd700 1 -- 172.21.15.144:0/2305455717 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8788009900 con 0x7f87980ffd30 2024-09-10T11:53:07.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.355+0000 7f879fb41700 1 -- 172.21.15.144:0/2305455717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879819fcb0 con 0x7f87980ffd30 2024-09-10T11:53:07.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.355+0000 7f878effd700 1 -- 172.21.15.144:0/2305455717 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8788010460 con 0x7f87980ffd30 2024-09-10T11:53:07.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.355+0000 7f878effd700 1 -- 172.21.15.144:0/2305455717 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f878800f5f0 con 0x7f87980ffd30 2024-09-10T11:53:07.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.355+0000 7f879fb41700 1 -- 172.21.15.144:0/2305455717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87981a0200 con 0x7f87980ffd30 2024-09-10T11:53:07.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.356+0000 7f879fb41700 1 -- 172.21.15.144:0/2305455717 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8798107650 con 0x7f87980ffd30 2024-09-10T11:53:07.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.356+0000 7f878effd700 1 -- 172.21.15.144:0/2305455717 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f87880105d0 con 0x7f87980ffd30 2024-09-10T11:53:07.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.357+0000 7f878effd700 1 --2- 172.21.15.144:0/2305455717 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f878406e200 0x7f87840706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:07.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.357+0000 7f878effd700 1 -- 172.21.15.144:0/2305455717 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f878808dc50 con 0x7f87980ffd30 2024-09-10T11:53:07.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.357+0000 7f879d8dd700 1 --2- 172.21.15.144:0/2305455717 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f878406e200 0x7f87840706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:07.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.357+0000 7f879d8dd700 1 --2- 172.21.15.144:0/2305455717 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f878406e200 0x7f87840706b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8794000f80 tx=0x7f87940060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:07.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.361+0000 7f878effd700 1 -- 172.21.15.144:0/2305455717 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8788057030 con 0x7f87980ffd30 2024-09-10T11:53:07.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.582+0000 7f879fb41700 1 -- 172.21.15.144:0/2305455717 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8798061190 con 0x7f878406e200 2024-09-10T11:53:07.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.583+0000 7f878effd700 1 -- 172.21.15.144:0/2305455717 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f8798061190 con 0x7f878406e200 2024-09-10T11:53:07.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.585+0000 7f879fb41700 1 -- 172.21.15.144:0/2305455717 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f878406e200 msgr2=0x7f87840706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:07.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.585+0000 7f879fb41700 1 --2- 172.21.15.144:0/2305455717 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f878406e200 0x7f87840706b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8794000f80 tx=0x7f87940060c0 comp rx=0 tx=0).stop 2024-09-10T11:53:07.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.585+0000 7f879fb41700 1 -- 172.21.15.144:0/2305455717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87980ffd30 msgr2=0x7f8798197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:07.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.585+0000 7f879fb41700 1 --2- 172.21.15.144:0/2305455717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87980ffd30 0x7f8798197180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f878800d900 tx=0x7f878800dc10 comp rx=0 tx=0).stop 2024-09-10T11:53:07.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.586+0000 7f879fb41700 1 -- 172.21.15.144:0/2305455717 shutdown_connections 2024-09-10T11:53:07.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.586+0000 7f879fb41700 1 --2- 172.21.15.144:0/2305455717 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f878406e200 0x7f87840706b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:07.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.586+0000 7f879fb41700 1 --2- 172.21.15.144:0/2305455717 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f87980ff390 0x7f8798196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:07.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.586+0000 7f879fb41700 1 --2- 172.21.15.144:0/2305455717 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f87980ffd30 0x7f8798197180 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:07.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.586+0000 7f879fb41700 1 -- 172.21.15.144:0/2305455717 >> 172.21.15.144:0/2305455717 conn(0x7f8798074f50 msgr2=0x7f87980fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:07.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.586+0000 7f879fb41700 1 -- 172.21.15.144:0/2305455717 shutdown_connections 2024-09-10T11:53:07.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.586+0000 7f879fb41700 1 -- 172.21.15.144:0/2305455717 wait complete. 2024-09-10T11:53:07.647 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:53:07.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.691+0000 7f4eb0ac1700 1 -- 172.21.15.144:0/2247865187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eac0ff340 msgr2=0x7f4eac0ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:07.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.691+0000 7f4eb0ac1700 1 --2- 172.21.15.144:0/2247865187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eac0ff340 0x7f4eac0ff710 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c009910 tx=0x7f4e9c009c20 comp rx=0 tx=0).stop 2024-09-10T11:53:07.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.691+0000 7f4eb0ac1700 1 -- 172.21.15.144:0/2247865187 shutdown_connections 2024-09-10T11:53:07.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.691+0000 7f4eb0ac1700 1 --2- 172.21.15.144:0/2247865187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eac0ffce0 0x7f4eac103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:07.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.691+0000 7f4eb0ac1700 1 --2- 172.21.15.144:0/2247865187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eac0ff340 0x7f4eac0ff710 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:07.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.691+0000 7f4eb0ac1700 1 -- 172.21.15.144:0/2247865187 >> 172.21.15.144:0/2247865187 conn(0x7f4eac0fabe0 msgr2=0x7f4eac0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:07.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.691+0000 7f4eb0ac1700 1 -- 172.21.15.144:0/2247865187 shutdown_connections 2024-09-10T11:53:07.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.692+0000 7f4eb0ac1700 1 -- 172.21.15.144:0/2247865187 wait complete. 2024-09-10T11:53:07.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.692+0000 7f4eb0ac1700 1 Processor -- start 2024-09-10T11:53:07.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.692+0000 7f4eb0ac1700 1 -- start start 2024-09-10T11:53:07.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.692+0000 7f4eb0ac1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eac0ff340 0x7f4eac195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:07.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.692+0000 7f4eb0ac1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eac0ffce0 0x7f4eac196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:07.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.692+0000 7f4eb0ac1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac196a00 con 0x7f4eac0ffce0 2024-09-10T11:53:07.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.692+0000 7f4eb0ac1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eac19a790 con 0x7f4eac0ff340 2024-09-10T11:53:07.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.693+0000 7f4ea9d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eac0ffce0 0x7f4eac196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:07.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.693+0000 7f4ea9d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eac0ffce0 0x7f4eac196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50478/0 (socket says 172.21.15.144:50478) 2024-09-10T11:53:07.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.693+0000 7f4ea9d9b700 1 -- 172.21.15.144:0/3355389671 learned_addr learned my addr 172.21.15.144:0/3355389671 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:53:07.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.693+0000 7f4eaa59c700 1 --2- 172.21.15.144:0/3355389671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eac0ff340 0x7f4eac195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:07.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.693+0000 7f4ea9d9b700 1 -- 172.21.15.144:0/3355389671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eac0ff340 msgr2=0x7f4eac195de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:07.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.693+0000 7f4ea9d9b700 1 --2- 172.21.15.144:0/3355389671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eac0ff340 0x7f4eac195de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:07.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.693+0000 7f4ea9d9b700 1 -- 172.21.15.144:0/3355389671 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c0095f0 con 0x7f4eac0ffce0 2024-09-10T11:53:07.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.693+0000 7f4ea9d9b700 1 --2- 172.21.15.144:0/3355389671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eac0ffce0 0x7f4eac196320 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000e8f0 tx=0x7f4ea000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:07.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.693+0000 7f4e9b7fe700 1 -- 172.21.15.144:0/3355389671 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ea0009790 con 0x7f4eac0ffce0 2024-09-10T11:53:07.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.693+0000 7f4e9b7fe700 1 -- 172.21.15.144:0/3355389671 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ea000c810 con 0x7f4eac0ffce0 2024-09-10T11:53:07.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.693+0000 7f4e9b7fe700 1 -- 172.21.15.144:0/3355389671 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ea0005580 con 0x7f4eac0ffce0 2024-09-10T11:53:07.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.693+0000 7f4eb0ac1700 1 -- 172.21.15.144:0/3355389671 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eac19aa70 con 0x7f4eac0ffce0 2024-09-10T11:53:07.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.693+0000 7f4eb0ac1700 1 -- 172.21.15.144:0/3355389671 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eac19afc0 con 0x7f4eac0ffce0 2024-09-10T11:53:07.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.694+0000 7f4eb0ac1700 1 -- 172.21.15.144:0/3355389671 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eac107650 con 0x7f4eac0ffce0 2024-09-10T11:53:07.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.695+0000 7f4e9b7fe700 1 -- 172.21.15.144:0/3355389671 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4ea000c980 con 0x7f4eac0ffce0 2024-09-10T11:53:07.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.695+0000 7f4e9b7fe700 1 --2- 172.21.15.144:0/3355389671 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4e9406e200 0x7f4e940706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:07.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.695+0000 7f4e9b7fe700 1 -- 172.21.15.144:0/3355389671 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4ea0014070 con 0x7f4eac0ffce0 2024-09-10T11:53:07.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.695+0000 7f4eaa59c700 1 --2- 172.21.15.144:0/3355389671 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4e9406e200 0x7f4e940706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:07.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.696+0000 7f4eaa59c700 1 --2- 172.21.15.144:0/3355389671 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4e9406e200 0x7f4e940706b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0059d0 tx=0x7f4e9c005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:07.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.700+0000 7f4e9b7fe700 1 -- 172.21.15.144:0/3355389671 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4ea00576a0 con 0x7f4eac0ffce0 2024-09-10T11:53:07.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.920+0000 7f4eb0ac1700 1 -- 172.21.15.144:0/3355389671 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4eac19b300 con 0x7f4e9406e200 2024-09-10T11:53:07.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.922+0000 7f4e9b7fe700 1 -- 172.21.15.144:0/3355389671 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4eac19b300 con 0x7f4e9406e200 2024-09-10T11:53:07.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.924+0000 7f4eb0ac1700 1 -- 172.21.15.144:0/3355389671 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4e9406e200 msgr2=0x7f4e940706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:07.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.924+0000 7f4eb0ac1700 1 --2- 172.21.15.144:0/3355389671 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4e9406e200 0x7f4e940706b0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c0059d0 tx=0x7f4e9c005940 comp rx=0 tx=0).stop 2024-09-10T11:53:07.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.924+0000 7f4eb0ac1700 1 -- 172.21.15.144:0/3355389671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eac0ffce0 msgr2=0x7f4eac196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:07.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.924+0000 7f4eb0ac1700 1 --2- 172.21.15.144:0/3355389671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eac0ffce0 0x7f4eac196320 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f4ea000e8f0 tx=0x7f4ea000ec00 comp rx=0 tx=0).stop 2024-09-10T11:53:07.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.924+0000 7f4eb0ac1700 1 -- 172.21.15.144:0/3355389671 shutdown_connections 2024-09-10T11:53:07.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.924+0000 7f4eb0ac1700 1 --2- 172.21.15.144:0/3355389671 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4e9406e200 0x7f4e940706b0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:07.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.924+0000 7f4eb0ac1700 1 --2- 172.21.15.144:0/3355389671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4eac0ff340 0x7f4eac195de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:07.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.924+0000 7f4eb0ac1700 1 --2- 172.21.15.144:0/3355389671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4eac0ffce0 0x7f4eac196320 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:07.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.924+0000 7f4eb0ac1700 1 -- 172.21.15.144:0/3355389671 >> 172.21.15.144:0/3355389671 conn(0x7f4eac0fabe0 msgr2=0x7f4eac0fc220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:07.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.924+0000 7f4eb0ac1700 1 -- 172.21.15.144:0/3355389671 shutdown_connections 2024-09-10T11:53:07.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:07.924+0000 7f4eb0ac1700 1 -- 172.21.15.144:0/3355389671 wait complete. 2024-09-10T11:53:08.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.030+0000 7fe24cac4700 1 -- 172.21.15.144:0/2896414117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe248101370 msgr2=0x7fe248101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:08.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.030+0000 7fe24cac4700 1 --2- 172.21.15.144:0/2896414117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe248101370 0x7fe248101740 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fe238009910 tx=0x7fe238009c20 comp rx=0 tx=0).stop 2024-09-10T11:53:08.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.030+0000 7fe24cac4700 1 -- 172.21.15.144:0/2896414117 shutdown_connections 2024-09-10T11:53:08.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.030+0000 7fe24cac4700 1 --2- 172.21.15.144:0/2896414117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe248101c80 0x7fe24810a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.030+0000 7fe24cac4700 1 --2- 172.21.15.144:0/2896414117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe248101370 0x7fe248101740 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.030+0000 7fe24cac4700 1 -- 172.21.15.144:0/2896414117 >> 172.21.15.144:0/2896414117 conn(0x7fe2480fabb0 msgr2=0x7fe2480fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:08.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.030+0000 7fe24cac4700 1 -- 172.21.15.144:0/2896414117 shutdown_connections 2024-09-10T11:53:08.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.030+0000 7fe24cac4700 1 -- 172.21.15.144:0/2896414117 wait complete. 2024-09-10T11:53:08.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.030+0000 7fe24cac4700 1 Processor -- start 2024-09-10T11:53:08.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.031+0000 7fe24cac4700 1 -- start start 2024-09-10T11:53:08.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.031+0000 7fe24cac4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe248101370 0x7fe248195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:08.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.031+0000 7fe24cac4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe248101c80 0x7fe248196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:08.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.031+0000 7fe24cac4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe248196960 con 0x7fe248101c80 2024-09-10T11:53:08.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.031+0000 7fe24cac4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24819a6f0 con 0x7fe248101370 2024-09-10T11:53:08.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.031+0000 7fe23dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe248101c80 0x7fe248196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:08.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.031+0000 7fe23dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe248101c80 0x7fe248196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50506/0 (socket says 172.21.15.144:50506) 2024-09-10T11:53:08.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.031+0000 7fe23dd9b700 1 -- 172.21.15.144:0/3906727387 learned_addr learned my addr 172.21.15.144:0/3906727387 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:53:08.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.031+0000 7fe24659c700 1 --2- 172.21.15.144:0/3906727387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe248101370 0x7fe248195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:08.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.031+0000 7fe23dd9b700 1 -- 172.21.15.144:0/3906727387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe248101370 msgr2=0x7fe248195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:08.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.031+0000 7fe23dd9b700 1 --2- 172.21.15.144:0/3906727387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe248101370 0x7fe248195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.031+0000 7fe23dd9b700 1 -- 172.21.15.144:0/3906727387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2380095f0 con 0x7fe248101c80 2024-09-10T11:53:08.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.031+0000 7fe23dd9b700 1 --2- 172.21.15.144:0/3906727387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe248101c80 0x7fe248196280 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe23000d8d0 tx=0x7fe23000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:08.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.032+0000 7fe23ffff700 1 -- 172.21.15.144:0/3906727387 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2300098a0 con 0x7fe248101c80 2024-09-10T11:53:08.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.032+0000 7fe24cac4700 1 -- 172.21.15.144:0/3906727387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24819a9d0 con 0x7fe248101c80 2024-09-10T11:53:08.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.032+0000 7fe23ffff700 1 -- 172.21.15.144:0/3906727387 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe230010460 con 0x7fe248101c80 2024-09-10T11:53:08.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.032+0000 7fe23ffff700 1 -- 172.21.15.144:0/3906727387 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe23000f5f0 con 0x7fe248101c80 2024-09-10T11:53:08.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.032+0000 7fe24cac4700 1 -- 172.21.15.144:0/3906727387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24819af20 con 0x7fe248101c80 2024-09-10T11:53:08.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.033+0000 7fe24cac4700 1 -- 172.21.15.144:0/3906727387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe248107850 con 0x7fe248101c80 2024-09-10T11:53:08.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.033+0000 7fe23ffff700 1 -- 172.21.15.144:0/3906727387 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe230009a00 con 0x7fe248101c80 2024-09-10T11:53:08.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.034+0000 7fe23ffff700 1 --2- 172.21.15.144:0/3906727387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe23406e200 0x7fe2340706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:08.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.034+0000 7fe23ffff700 1 -- 172.21.15.144:0/3906727387 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe23008dea0 con 0x7fe248101c80 2024-09-10T11:53:08.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.034+0000 7fe24659c700 1 --2- 172.21.15.144:0/3906727387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe23406e200 0x7fe2340706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:08.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.034+0000 7fe24659c700 1 --2- 172.21.15.144:0/3906727387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe23406e200 0x7fe2340706b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe238006010 tx=0x7fe23801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:08.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.038+0000 7fe23ffff700 1 -- 172.21.15.144:0/3906727387 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe230057300 con 0x7fe248101c80 2024-09-10T11:53:08.213 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:07 smithi144 ceph-mon[87436]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:08.213 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:07 smithi144 ceph-mon[87436]: pgmap v634: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:08.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.261+0000 7fe24cac4700 1 -- 172.21.15.144:0/3906727387 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe24819b280 con 0x7fe23406e200 2024-09-10T11:53:08.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.270+0000 7fe23ffff700 1 -- 172.21.15.144:0/3906727387 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fe24819b280 con 0x7fe23406e200 2024-09-10T11:53:08.272 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:53:08.272 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (21m) 15m ago 31m 24.7M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:53:08.272 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (32m) 15m ago 32m 9198k - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:53:08.272 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (31m) 14m ago 31m 8841k - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:53:08.272 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (19m) 15m ago 32m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:53:08.272 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (18m) 14m ago 31m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:53:08.272 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (21m) 15m ago 31m 81.8M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:53:08.272 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (15m) 15m ago 26m 15.3M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:53:08.272 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (15m) 15m ago 26m 11.5M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:53:08.272 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (26m) 14m ago 26m 11.9M - 18.2.0 dc2bc1663786 517a1450c828 2024-09-10T11:53:08.272 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (14m) 14m ago 26m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:53:08.273 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (23m) 15m ago 33m 503M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:53:08.273 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (22m) 14m ago 30m 442M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:53:08.273 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (20m) 15m ago 33m 55.9M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:53:08.273 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (19m) 14m ago 30m 44.1M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:53:08.273 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (21m) 15m ago 32m 17.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:53:08.273 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (21m) 14m ago 31m 15.5M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:53:08.273 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (18m) 15m ago 30m 93.4M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:53:08.273 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (18m) 15m ago 29m 91.7M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:53:08.273 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (17m) 15m ago 29m 83.6M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:53:08.273 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (16m) 14m ago 28m 103M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:53:08.273 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (16m) 14m ago 28m 88.2M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:53:08.273 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (15m) 14m ago 27m 84.7M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:53:08.273 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (21m) 15m ago 31m 81.3M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:53:08.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.272+0000 7fe24cac4700 1 -- 172.21.15.144:0/3906727387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe23406e200 msgr2=0x7fe2340706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:08.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.272+0000 7fe24cac4700 1 --2- 172.21.15.144:0/3906727387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe23406e200 0x7fe2340706b0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe238006010 tx=0x7fe23801a040 comp rx=0 tx=0).stop 2024-09-10T11:53:08.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.272+0000 7fe24cac4700 1 -- 172.21.15.144:0/3906727387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe248101c80 msgr2=0x7fe248196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:08.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.272+0000 7fe24cac4700 1 --2- 172.21.15.144:0/3906727387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe248101c80 0x7fe248196280 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fe23000d8d0 tx=0x7fe23000dbe0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.273+0000 7fe24cac4700 1 -- 172.21.15.144:0/3906727387 shutdown_connections 2024-09-10T11:53:08.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.273+0000 7fe24cac4700 1 --2- 172.21.15.144:0/3906727387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe23406e200 0x7fe2340706b0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.273+0000 7fe24cac4700 1 --2- 172.21.15.144:0/3906727387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe248101370 0x7fe248195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.273+0000 7fe24cac4700 1 --2- 172.21.15.144:0/3906727387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe248101c80 0x7fe248196280 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.273+0000 7fe24cac4700 1 -- 172.21.15.144:0/3906727387 >> 172.21.15.144:0/3906727387 conn(0x7fe2480fabb0 msgr2=0x7fe248104990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:08.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.273+0000 7fe24cac4700 1 -- 172.21.15.144:0/3906727387 shutdown_connections 2024-09-10T11:53:08.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.273+0000 7fe24cac4700 1 -- 172.21.15.144:0/3906727387 wait complete. 2024-09-10T11:53:08.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:07 smithi178 ceph-mon[65940]: from='client.35184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:08.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:07 smithi178 ceph-mon[65940]: pgmap v634: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:08.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.376+0000 7f51fa4ef700 1 -- 172.21.15.144:0/3598808001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51f40ffd30 msgr2=0x7f51f4103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:08.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.376+0000 7f51fa4ef700 1 --2- 172.21.15.144:0/3598808001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51f40ffd30 0x7f51f4103ce0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f51e8009910 tx=0x7f51e8009c20 comp rx=0 tx=0).stop 2024-09-10T11:53:08.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.376+0000 7f51fa4ef700 1 -- 172.21.15.144:0/3598808001 shutdown_connections 2024-09-10T11:53:08.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.376+0000 7f51fa4ef700 1 --2- 172.21.15.144:0/3598808001 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51f40ffd30 0x7f51f4103ce0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.376+0000 7f51fa4ef700 1 --2- 172.21.15.144:0/3598808001 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51f40ff390 0x7f51f40ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.376+0000 7f51fa4ef700 1 -- 172.21.15.144:0/3598808001 >> 172.21.15.144:0/3598808001 conn(0x7f51f4074f50 msgr2=0x7f51f4075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:08.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.376+0000 7f51fa4ef700 1 -- 172.21.15.144:0/3598808001 shutdown_connections 2024-09-10T11:53:08.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.376+0000 7f51fa4ef700 1 -- 172.21.15.144:0/3598808001 wait complete. 2024-09-10T11:53:08.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.377+0000 7f51fa4ef700 1 Processor -- start 2024-09-10T11:53:08.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.377+0000 7f51fa4ef700 1 -- start start 2024-09-10T11:53:08.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.377+0000 7f51fa4ef700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51f40ff390 0x7f51f419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:08.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.377+0000 7f51fa4ef700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51f40ffd30 0x7f51f419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:08.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.377+0000 7f51fa4ef700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f419b5b0 con 0x7f51f40ffd30 2024-09-10T11:53:08.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.377+0000 7f51fa4ef700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f4194a60 con 0x7f51f40ff390 2024-09-10T11:53:08.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.377+0000 7f51f37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51f40ffd30 0x7f51f419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:08.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.377+0000 7f51f37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51f40ffd30 0x7f51f419af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50524/0 (socket says 172.21.15.144:50524) 2024-09-10T11:53:08.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.377+0000 7f51f37fe700 1 -- 172.21.15.144:0/153071389 learned_addr learned my addr 172.21.15.144:0/153071389 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:53:08.380 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.377+0000 7f51f3fff700 1 --2- 172.21.15.144:0/153071389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51f40ff390 0x7f51f419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:08.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.377+0000 7f51f37fe700 1 -- 172.21.15.144:0/153071389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51f40ff390 msgr2=0x7f51f419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:08.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.377+0000 7f51f37fe700 1 --2- 172.21.15.144:0/153071389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51f40ff390 0x7f51f419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.377+0000 7f51f37fe700 1 -- 172.21.15.144:0/153071389 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e80095f0 con 0x7f51f40ffd30 2024-09-10T11:53:08.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.378+0000 7f51f37fe700 1 --2- 172.21.15.144:0/153071389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51f40ffd30 0x7f51f419af20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f51e80059d0 tx=0x7f51e800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:08.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.378+0000 7f51f17fa700 1 -- 172.21.15.144:0/153071389 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51e801d070 con 0x7f51f40ffd30 2024-09-10T11:53:08.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.378+0000 7f51f17fa700 1 -- 172.21.15.144:0/153071389 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51e800b610 con 0x7f51f40ffd30 2024-09-10T11:53:08.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.378+0000 7f51fa4ef700 1 -- 172.21.15.144:0/153071389 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f4194ce0 con 0x7f51f40ffd30 2024-09-10T11:53:08.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.378+0000 7f51f17fa700 1 -- 172.21.15.144:0/153071389 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51e8022470 con 0x7f51f40ffd30 2024-09-10T11:53:08.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.378+0000 7f51fa4ef700 1 -- 172.21.15.144:0/153071389 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f41951d0 con 0x7f51f40ffd30 2024-09-10T11:53:08.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.379+0000 7f51fa4ef700 1 -- 172.21.15.144:0/153071389 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f4107600 con 0x7f51f40ffd30 2024-09-10T11:53:08.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.379+0000 7f51f17fa700 1 -- 172.21.15.144:0/153071389 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f51e8003710 con 0x7f51f40ffd30 2024-09-10T11:53:08.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.380+0000 7f51f17fa700 1 --2- 172.21.15.144:0/153071389 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f51dc06e200 0x7f51dc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:08.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.380+0000 7f51f17fa700 1 -- 172.21.15.144:0/153071389 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f51e808f570 con 0x7f51f40ffd30 2024-09-10T11:53:08.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.380+0000 7f51f3fff700 1 --2- 172.21.15.144:0/153071389 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f51dc06e200 0x7f51dc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:08.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.380+0000 7f51f3fff700 1 --2- 172.21.15.144:0/153071389 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f51dc06e200 0x7f51dc0706b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009c60 tx=0x7f51e4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:08.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.384+0000 7f51f17fa700 1 -- 172.21.15.144:0/153071389 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f51e805c480 con 0x7f51f40ffd30 2024-09-10T11:53:08.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.668+0000 7f51fa4ef700 1 -- 172.21.15.144:0/153071389 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f51f404ea50 con 0x7f51f40ffd30 2024-09-10T11:53:08.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.669+0000 7f51f17fa700 1 -- 172.21.15.144:0/153071389 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+815 (secure 0 0 0) 0x7f51e805c010 con 0x7f51f40ffd30 2024-09-10T11:53:08.670 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:53:08.670 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:53:08.670 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:53:08.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:53:08.670 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:53:08.670 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:53:08.670 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:53:08.670 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:53:08.670 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:53:08.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:53:08.671 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:53:08.671 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:53:08.671 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 3 2024-09-10T11:53:08.671 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:53:08.671 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:53:08.671 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-09-10T11:53:08.671 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 13 2024-09-10T11:53:08.671 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:53:08.671 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:53:08.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.671+0000 7f51fa4ef700 1 -- 172.21.15.144:0/153071389 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f51dc06e200 msgr2=0x7f51dc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:08.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.671+0000 7f51fa4ef700 1 --2- 172.21.15.144:0/153071389 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f51dc06e200 0x7f51dc0706b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f51e4009c60 tx=0x7f51e4009260 comp rx=0 tx=0).stop 2024-09-10T11:53:08.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.671+0000 7f51fa4ef700 1 -- 172.21.15.144:0/153071389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51f40ffd30 msgr2=0x7f51f419af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:08.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.671+0000 7f51fa4ef700 1 --2- 172.21.15.144:0/153071389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51f40ffd30 0x7f51f419af20 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f51e80059d0 tx=0x7f51e800f690 comp rx=0 tx=0).stop 2024-09-10T11:53:08.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.671+0000 7f51fa4ef700 1 -- 172.21.15.144:0/153071389 shutdown_connections 2024-09-10T11:53:08.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.671+0000 7f51fa4ef700 1 --2- 172.21.15.144:0/153071389 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f51dc06e200 0x7f51dc0706b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.671+0000 7f51fa4ef700 1 --2- 172.21.15.144:0/153071389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51f40ff390 0x7f51f419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.671+0000 7f51fa4ef700 1 --2- 172.21.15.144:0/153071389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51f40ffd30 0x7f51f419af20 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.671+0000 7f51fa4ef700 1 -- 172.21.15.144:0/153071389 >> 172.21.15.144:0/153071389 conn(0x7f51f4074f50 msgr2=0x7f51f40fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:08.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.671+0000 7f51fa4ef700 1 -- 172.21.15.144:0/153071389 shutdown_connections 2024-09-10T11:53:08.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.671+0000 7f51fa4ef700 1 -- 172.21.15.144:0/153071389 wait complete. 2024-09-10T11:53:08.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.776+0000 7f39ffee0700 1 -- 172.21.15.144:0/1014646722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39f80ff290 msgr2=0x7f39f80ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:08.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.776+0000 7f39ffee0700 1 --2- 172.21.15.144:0/1014646722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39f80ff290 0x7f39f80ff660 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f39e8009910 tx=0x7f39e8009c20 comp rx=0 tx=0).stop 2024-09-10T11:53:08.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.776+0000 7f39ffee0700 1 -- 172.21.15.144:0/1014646722 shutdown_connections 2024-09-10T11:53:08.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.776+0000 7f39ffee0700 1 --2- 172.21.15.144:0/1014646722 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39f80ffba0 0x7f39f810a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.776+0000 7f39ffee0700 1 --2- 172.21.15.144:0/1014646722 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39f80ff290 0x7f39f80ff660 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.776+0000 7f39ffee0700 1 -- 172.21.15.144:0/1014646722 >> 172.21.15.144:0/1014646722 conn(0x7f39f8074850 msgr2=0x7f39f8074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:08.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.776+0000 7f39ffee0700 1 -- 172.21.15.144:0/1014646722 shutdown_connections 2024-09-10T11:53:08.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.776+0000 7f39ffee0700 1 -- 172.21.15.144:0/1014646722 wait complete. 2024-09-10T11:53:08.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.776+0000 7f39ffee0700 1 Processor -- start 2024-09-10T11:53:08.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.776+0000 7f39ffee0700 1 -- start start 2024-09-10T11:53:08.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39ffee0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39f80ff290 0x7f39f819bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:08.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39ffee0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39f80ffba0 0x7f39f819c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:08.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39ffee0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f819c690 con 0x7f39f80ff290 2024-09-10T11:53:08.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39ffee0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f8195b40 con 0x7f39f80ffba0 2024-09-10T11:53:08.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39fdc7c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39f80ff290 0x7f39f819bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:08.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39fdc7c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39f80ff290 0x7f39f819bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50534/0 (socket says 172.21.15.144:50534) 2024-09-10T11:53:08.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39fdc7c700 1 -- 172.21.15.144:0/221790343 learned_addr learned my addr 172.21.15.144:0/221790343 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:53:08.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39fd47b700 1 --2- 172.21.15.144:0/221790343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39f80ffba0 0x7f39f819c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:08.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39fdc7c700 1 -- 172.21.15.144:0/221790343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39f80ffba0 msgr2=0x7f39f819c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:08.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39fdc7c700 1 --2- 172.21.15.144:0/221790343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39f80ffba0 0x7f39f819c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:08.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39fdc7c700 1 -- 172.21.15.144:0/221790343 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e80095f0 con 0x7f39f80ff290 2024-09-10T11:53:08.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39fdc7c700 1 --2- 172.21.15.144:0/221790343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39f80ff290 0x7f39f819bac0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f39e80092e0 tx=0x7f39e800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:08.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39eeffd700 1 -- 172.21.15.144:0/221790343 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39e801d070 con 0x7f39f80ff290 2024-09-10T11:53:08.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39eeffd700 1 -- 172.21.15.144:0/221790343 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f39e8005dc0 con 0x7f39f80ff290 2024-09-10T11:53:08.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39eeffd700 1 -- 172.21.15.144:0/221790343 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39e8017400 con 0x7f39f80ff290 2024-09-10T11:53:08.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.777+0000 7f39ffee0700 1 -- 172.21.15.144:0/221790343 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f8195dc0 con 0x7f39f80ff290 2024-09-10T11:53:08.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.778+0000 7f39ffee0700 1 -- 172.21.15.144:0/221790343 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f81962b0 con 0x7f39f80ff290 2024-09-10T11:53:08.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.778+0000 7f39ffee0700 1 -- 172.21.15.144:0/221790343 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f8107d10 con 0x7f39f80ff290 2024-09-10T11:53:08.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.779+0000 7f39eeffd700 1 -- 172.21.15.144:0/221790343 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f39e800f460 con 0x7f39f80ff290 2024-09-10T11:53:08.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.779+0000 7f39eeffd700 1 --2- 172.21.15.144:0/221790343 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f39e406e200 0x7f39e40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:08.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.779+0000 7f39eeffd700 1 -- 172.21.15.144:0/221790343 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f39e808f590 con 0x7f39f80ff290 2024-09-10T11:53:08.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.779+0000 7f39fd47b700 1 --2- 172.21.15.144:0/221790343 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f39e406e200 0x7f39e40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:08.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.780+0000 7f39fd47b700 1 --2- 172.21.15.144:0/221790343 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f39e406e200 0x7f39e40706b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f39f4005fd0 tx=0x7f39f4005f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:08.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:08.784+0000 7f39eeffd700 1 -- 172.21.15.144:0/221790343 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f39e805c4a0 con 0x7f39f80ff290 2024-09-10T11:53:09.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.031+0000 7f39ffee0700 1 -- 172.21.15.144:0/221790343 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f39f802ccf0 con 0x7f39f80ff290 2024-09-10T11:53:09.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.032+0000 7f39eeffd700 1 -- 172.21.15.144:0/221790343 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 32 v32) v1 ==== 76+0+1915 (secure 0 0 0) 0x7f39e805c030 con 0x7f39f80ff290 2024-09-10T11:53:09.035 INFO:teuthology.orchestra.run.smithi144.stdout:e32 2024-09-10T11:53:09.035 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:38:14:670931+0000 2024-09-10T11:53:09.035 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:53:09.035 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:53:09.036 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:53:09.036 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:53:09.036 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:53:09.036 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:53:09.036 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:53:09.037 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:53:09.037 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:53:09.037 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:53:09.037 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:53:09.037 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:53:09.038 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:53:09.038 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:53:09.038 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:53:09.038 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:53:09.038 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:53:09.038 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:53:09.039 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:53:09.039 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:53:09.039 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:53:09.039 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:53:09.039 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:53:09.039 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:53:09.039 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:53:09.040 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:53:09.040 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:53:09.040 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:53:09.040 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:53:09.040 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:53:09.041 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:53:09.041 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:53:09.042 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:53:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:53:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:53:09.042 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:53:09.042 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:53:09.043 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:53:09.043 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:53:09.043 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44315} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2970117696,v1:172.21.15.178:6827/2970117696] compat {c=[1],r=[1],i=[7ff]}] 2024-09-10T11:53:09.043 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:53:09.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.034+0000 7f39ffee0700 1 -- 172.21.15.144:0/221790343 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f39e406e200 msgr2=0x7f39e40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:09.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.034+0000 7f39ffee0700 1 --2- 172.21.15.144:0/221790343 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f39e406e200 0x7f39e40706b0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f39f4005fd0 tx=0x7f39f4005f60 comp rx=0 tx=0).stop 2024-09-10T11:53:09.047 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.034+0000 7f39ffee0700 1 -- 172.21.15.144:0/221790343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39f80ff290 msgr2=0x7f39f819bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:09.047 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.034+0000 7f39ffee0700 1 --2- 172.21.15.144:0/221790343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39f80ff290 0x7f39f819bac0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f39e80092e0 tx=0x7f39e800be20 comp rx=0 tx=0).stop 2024-09-10T11:53:09.047 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.034+0000 7f39ffee0700 1 -- 172.21.15.144:0/221790343 shutdown_connections 2024-09-10T11:53:09.047 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.034+0000 7f39ffee0700 1 --2- 172.21.15.144:0/221790343 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f39e406e200 0x7f39e40706b0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.034+0000 7f39ffee0700 1 --2- 172.21.15.144:0/221790343 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f39f80ff290 0x7f39f819bac0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.034+0000 7f39ffee0700 1 --2- 172.21.15.144:0/221790343 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f39f80ffba0 0x7f39f819c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.034+0000 7f39ffee0700 1 -- 172.21.15.144:0/221790343 >> 172.21.15.144:0/221790343 conn(0x7f39f8074850 msgr2=0x7f39f8104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:09.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.034+0000 7f39ffee0700 1 -- 172.21.15.144:0/221790343 shutdown_connections 2024-09-10T11:53:09.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.034+0000 7f39ffee0700 1 -- 172.21.15.144:0/221790343 wait complete. 2024-09-10T11:53:09.049 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T11:53:09.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.140+0000 7f4b25569700 1 -- 172.21.15.144:0/367940232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b20100d70 msgr2=0x7f4b20104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:09.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.140+0000 7f4b25569700 1 --2- 172.21.15.144:0/367940232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b20100d70 0x7f4b20104d40 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f4b10009910 tx=0x7f4b10009c20 comp rx=0 tx=0).stop 2024-09-10T11:53:09.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.140+0000 7f4b25569700 1 -- 172.21.15.144:0/367940232 shutdown_connections 2024-09-10T11:53:09.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.140+0000 7f4b25569700 1 --2- 172.21.15.144:0/367940232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b20100d70 0x7f4b20104d40 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.140+0000 7f4b25569700 1 --2- 172.21.15.144:0/367940232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b201003d0 0x7f4b201007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.140+0000 7f4b25569700 1 -- 172.21.15.144:0/367940232 >> 172.21.15.144:0/367940232 conn(0x7f4b200fbc90 msgr2=0x7f4b200fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:09.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.140+0000 7f4b25569700 1 -- 172.21.15.144:0/367940232 shutdown_connections 2024-09-10T11:53:09.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.141+0000 7f4b25569700 1 -- 172.21.15.144:0/367940232 wait complete. 2024-09-10T11:53:09.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.141+0000 7f4b25569700 1 Processor -- start 2024-09-10T11:53:09.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.141+0000 7f4b25569700 1 -- start start 2024-09-10T11:53:09.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.141+0000 7f4b25569700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b201003d0 0x7f4b20195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:09.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.141+0000 7f4b25569700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b20100d70 0x7f4b201963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:09.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.142+0000 7f4b25569700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b20196aa0 con 0x7f4b201003d0 2024-09-10T11:53:09.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.142+0000 7f4b25569700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b2019a830 con 0x7f4b20100d70 2024-09-10T11:53:09.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.142+0000 7f4b1effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b201003d0 0x7f4b20195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:09.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.142+0000 7f4b1effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b201003d0 0x7f4b20195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50560/0 (socket says 172.21.15.144:50560) 2024-09-10T11:53:09.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.142+0000 7f4b1effd700 1 -- 172.21.15.144:0/1655384905 learned_addr learned my addr 172.21.15.144:0/1655384905 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:53:09.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.142+0000 7f4b1e7fc700 1 --2- 172.21.15.144:0/1655384905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b20100d70 0x7f4b201963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:09.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.142+0000 7f4b1effd700 1 -- 172.21.15.144:0/1655384905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b20100d70 msgr2=0x7f4b201963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:09.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.142+0000 7f4b1effd700 1 --2- 172.21.15.144:0/1655384905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b20100d70 0x7f4b201963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.142+0000 7f4b1effd700 1 -- 172.21.15.144:0/1655384905 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b100095f0 con 0x7f4b201003d0 2024-09-10T11:53:09.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.142+0000 7f4b1effd700 1 --2- 172.21.15.144:0/1655384905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b201003d0 0x7f4b20195e80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400b4b0 tx=0x7f4b1400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:09.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.142+0000 7f4b07fff700 1 -- 172.21.15.144:0/1655384905 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b14011800 con 0x7f4b201003d0 2024-09-10T11:53:09.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.142+0000 7f4b07fff700 1 -- 172.21.15.144:0/1655384905 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b14011e20 con 0x7f4b201003d0 2024-09-10T11:53:09.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.142+0000 7f4b25569700 1 -- 172.21.15.144:0/1655384905 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b2019ab10 con 0x7f4b201003d0 2024-09-10T11:53:09.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.142+0000 7f4b07fff700 1 -- 172.21.15.144:0/1655384905 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b1400f2e0 con 0x7f4b201003d0 2024-09-10T11:53:09.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.142+0000 7f4b25569700 1 -- 172.21.15.144:0/1655384905 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2019b060 con 0x7f4b201003d0 2024-09-10T11:53:09.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.143+0000 7f4b25569700 1 -- 172.21.15.144:0/1655384905 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b20108670 con 0x7f4b201003d0 2024-09-10T11:53:09.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.144+0000 7f4b07fff700 1 -- 172.21.15.144:0/1655384905 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4b14010380 con 0x7f4b201003d0 2024-09-10T11:53:09.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.144+0000 7f4b07fff700 1 --2- 172.21.15.144:0/1655384905 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4b0806e200 0x7f4b080706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:09.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.144+0000 7f4b07fff700 1 -- 172.21.15.144:0/1655384905 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4b1408dc40 con 0x7f4b201003d0 2024-09-10T11:53:09.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.144+0000 7f4b1e7fc700 1 --2- 172.21.15.144:0/1655384905 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4b0806e200 0x7f4b080706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:09.147 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.145+0000 7f4b1e7fc700 1 --2- 172.21.15.144:0/1655384905 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4b0806e200 0x7f4b080706b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4b100059d0 tx=0x7f4b10005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:09.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.149+0000 7f4b07fff700 1 -- 172.21.15.144:0/1655384905 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4b140570b0 con 0x7f4b201003d0 2024-09-10T11:53:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:08 smithi178 ceph-mon[65940]: from='client.35188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:08 smithi178 ceph-mon[65940]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:08 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/153071389' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:53:09.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.370+0000 7f4b25569700 1 -- 172.21.15.144:0/1655384905 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b2019b3f0 con 0x7f4b0806e200 2024-09-10T11:53:09.371 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:08 smithi144 ceph-mon[87436]: from='client.35188 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:09.371 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:08 smithi144 ceph-mon[87436]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:09.371 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:08 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/153071389' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:53:09.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.372+0000 7f4b07fff700 1 -- 172.21.15.144:0/1655384905 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f4b2019b3f0 con 0x7f4b0806e200 2024-09-10T11:53:09.373 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:53:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:53:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:53:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:53:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:53:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: "osd", 2024-09-10T11:53:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:53:09.373 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:53:09.374 INFO:teuthology.orchestra.run.smithi144.stdout: "mon" 2024-09-10T11:53:09.374 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:53:09.374 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "15/23 daemons upgraded", 2024-09-10T11:53:09.374 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:53:09.374 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:53:09.374 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:53:09.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.374+0000 7f4b25569700 1 -- 172.21.15.144:0/1655384905 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4b0806e200 msgr2=0x7f4b080706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:09.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.374+0000 7f4b25569700 1 --2- 172.21.15.144:0/1655384905 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4b0806e200 0x7f4b080706b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f4b100059d0 tx=0x7f4b10005960 comp rx=0 tx=0).stop 2024-09-10T11:53:09.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.374+0000 7f4b25569700 1 -- 172.21.15.144:0/1655384905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b201003d0 msgr2=0x7f4b20195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:09.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.374+0000 7f4b25569700 1 --2- 172.21.15.144:0/1655384905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b201003d0 0x7f4b20195e80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f4b1400b4b0 tx=0x7f4b1400b870 comp rx=0 tx=0).stop 2024-09-10T11:53:09.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.374+0000 7f4b25569700 1 -- 172.21.15.144:0/1655384905 shutdown_connections 2024-09-10T11:53:09.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.374+0000 7f4b25569700 1 --2- 172.21.15.144:0/1655384905 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4b0806e200 0x7f4b080706b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.374+0000 7f4b25569700 1 --2- 172.21.15.144:0/1655384905 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4b201003d0 0x7f4b20195e80 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.374+0000 7f4b25569700 1 --2- 172.21.15.144:0/1655384905 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4b20100d70 0x7f4b201963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.374+0000 7f4b25569700 1 -- 172.21.15.144:0/1655384905 >> 172.21.15.144:0/1655384905 conn(0x7f4b200fbc90 msgr2=0x7f4b200fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:09.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.374+0000 7f4b25569700 1 -- 172.21.15.144:0/1655384905 shutdown_connections 2024-09-10T11:53:09.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.374+0000 7f4b25569700 1 -- 172.21.15.144:0/1655384905 wait complete. 2024-09-10T11:53:09.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.479+0000 7f56c2083700 1 -- 172.21.15.144:0/3332495385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56bc100d70 msgr2=0x7f56bc104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:09.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.479+0000 7f56c2083700 1 --2- 172.21.15.144:0/3332495385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56bc100d70 0x7f56bc104d40 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f56b0009910 tx=0x7f56b0009c20 comp rx=0 tx=0).stop 2024-09-10T11:53:09.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.479+0000 7f56c2083700 1 -- 172.21.15.144:0/3332495385 shutdown_connections 2024-09-10T11:53:09.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.479+0000 7f56c2083700 1 --2- 172.21.15.144:0/3332495385 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56bc100d70 0x7f56bc104d40 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.479+0000 7f56c2083700 1 --2- 172.21.15.144:0/3332495385 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56bc1003d0 0x7f56bc1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.479+0000 7f56c2083700 1 -- 172.21.15.144:0/3332495385 >> 172.21.15.144:0/3332495385 conn(0x7f56bc0fbc90 msgr2=0x7f56bc0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:09.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.479+0000 7f56c2083700 1 -- 172.21.15.144:0/3332495385 shutdown_connections 2024-09-10T11:53:09.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.479+0000 7f56c2083700 1 -- 172.21.15.144:0/3332495385 wait complete. 2024-09-10T11:53:09.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.479+0000 7f56c2083700 1 Processor -- start 2024-09-10T11:53:09.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.480+0000 7f56c2083700 1 -- start start 2024-09-10T11:53:09.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.480+0000 7f56c2083700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56bc1003d0 0x7f56bc195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:09.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.480+0000 7f56c2083700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56bc100d70 0x7f56bc196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:09.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.480+0000 7f56c2083700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc196960 con 0x7f56bc1003d0 2024-09-10T11:53:09.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.480+0000 7f56c2083700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56bc19a6f0 con 0x7f56bc100d70 2024-09-10T11:53:09.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.480+0000 7f56bb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56bc1003d0 0x7f56bc195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:09.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.480+0000 7f56bb7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56bc1003d0 0x7f56bc195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50584/0 (socket says 172.21.15.144:50584) 2024-09-10T11:53:09.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.480+0000 7f56bb7fe700 1 -- 172.21.15.144:0/1733339384 learned_addr learned my addr 172.21.15.144:0/1733339384 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:53:09.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.480+0000 7f56baffd700 1 --2- 172.21.15.144:0/1733339384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56bc100d70 0x7f56bc196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:09.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.480+0000 7f56bb7fe700 1 -- 172.21.15.144:0/1733339384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56bc100d70 msgr2=0x7f56bc196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:09.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.480+0000 7f56bb7fe700 1 --2- 172.21.15.144:0/1733339384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56bc100d70 0x7f56bc196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.480+0000 7f56bb7fe700 1 -- 172.21.15.144:0/1733339384 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b00095f0 con 0x7f56bc1003d0 2024-09-10T11:53:09.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.480+0000 7f56bb7fe700 1 --2- 172.21.15.144:0/1733339384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56bc1003d0 0x7f56bc195d40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0060b0 tx=0x7f56ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:09.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.481+0000 7f56b8ff9700 1 -- 172.21.15.144:0/1733339384 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56ac00dda0 con 0x7f56bc1003d0 2024-09-10T11:53:09.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.481+0000 7f56b8ff9700 1 -- 172.21.15.144:0/1733339384 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f56ac00f040 con 0x7f56bc1003d0 2024-09-10T11:53:09.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.481+0000 7f56c2083700 1 -- 172.21.15.144:0/1733339384 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56bc19a9d0 con 0x7f56bc1003d0 2024-09-10T11:53:09.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.481+0000 7f56b8ff9700 1 -- 172.21.15.144:0/1733339384 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56ac014790 con 0x7f56bc1003d0 2024-09-10T11:53:09.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.481+0000 7f56c2083700 1 -- 172.21.15.144:0/1733339384 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56bc19af20 con 0x7f56bc1003d0 2024-09-10T11:53:09.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.482+0000 7f56c2083700 1 -- 172.21.15.144:0/1733339384 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56bc108740 con 0x7f56bc1003d0 2024-09-10T11:53:09.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.482+0000 7f56b8ff9700 1 -- 172.21.15.144:0/1733339384 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f56ac014980 con 0x7f56bc1003d0 2024-09-10T11:53:09.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.483+0000 7f56b8ff9700 1 --2- 172.21.15.144:0/1733339384 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f56a406e200 0x7f56a40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:09.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.483+0000 7f56b8ff9700 1 -- 172.21.15.144:0/1733339384 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f56ac08db30 con 0x7f56bc1003d0 2024-09-10T11:53:09.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.483+0000 7f56baffd700 1 --2- 172.21.15.144:0/1733339384 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f56a406e200 0x7f56a40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:09.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.483+0000 7f56baffd700 1 --2- 172.21.15.144:0/1733339384 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f56a406e200 0x7f56a40706b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f56bc197360 tx=0x7f56b001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:09.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.487+0000 7f56b8ff9700 1 -- 172.21.15.144:0/1733339384 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f56ac056fc0 con 0x7f56bc1003d0 2024-09-10T11:53:09.774 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.773+0000 7f56c2083700 1 -- 172.21.15.144:0/1733339384 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f56bc04ea50 con 0x7f56bc1003d0 2024-09-10T11:53:09.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.773+0000 7f56b8ff9700 1 -- 172.21.15.144:0/1733339384 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+423 (secure 0 0 0) 0x7f56ac01a1c0 con 0x7f56bc1003d0 2024-09-10T11:53:09.775 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:53:09.775 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:53:09.775 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:53:09.775 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:53:09.775 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:53:09.775 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:53:09.775 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:53:09.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.776+0000 7f56c2083700 1 -- 172.21.15.144:0/1733339384 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f56a406e200 msgr2=0x7f56a40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:09.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.776+0000 7f56c2083700 1 --2- 172.21.15.144:0/1733339384 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f56a406e200 0x7f56a40706b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f56bc197360 tx=0x7f56b001a040 comp rx=0 tx=0).stop 2024-09-10T11:53:09.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.776+0000 7f56c2083700 1 -- 172.21.15.144:0/1733339384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56bc1003d0 msgr2=0x7f56bc195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:09.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.776+0000 7f56c2083700 1 --2- 172.21.15.144:0/1733339384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56bc1003d0 0x7f56bc195d40 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f56ac0060b0 tx=0x7f56ac00d3b0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.776+0000 7f56c2083700 1 -- 172.21.15.144:0/1733339384 shutdown_connections 2024-09-10T11:53:09.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.776+0000 7f56c2083700 1 --2- 172.21.15.144:0/1733339384 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f56a406e200 0x7f56a40706b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.776+0000 7f56c2083700 1 --2- 172.21.15.144:0/1733339384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f56bc1003d0 0x7f56bc195d40 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.776+0000 7f56c2083700 1 --2- 172.21.15.144:0/1733339384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f56bc100d70 0x7f56bc196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:09.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.776+0000 7f56c2083700 1 -- 172.21.15.144:0/1733339384 >> 172.21.15.144:0/1733339384 conn(0x7f56bc0fbc90 msgr2=0x7f56bc0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:09.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.776+0000 7f56c2083700 1 -- 172.21.15.144:0/1733339384 shutdown_connections 2024-09-10T11:53:09.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:09.776+0000 7f56c2083700 1 -- 172.21.15.144:0/1733339384 wait complete. 2024-09-10T11:53:10.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:09 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/221790343' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:53:10.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:09 smithi144 ceph-mon[87436]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:10.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:09 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1733339384' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:53:10.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:09 smithi144 ceph-mon[87436]: pgmap v635: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:10.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:09 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/221790343' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:53:10.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:09 smithi178 ceph-mon[65940]: from='client.35204 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:10.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:09 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1733339384' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:53:10.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:09 smithi178 ceph-mon[65940]: pgmap v635: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:12.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:11 smithi178 ceph-mon[65940]: pgmap v636: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:12.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:11 smithi144 ceph-mon[87436]: pgmap v636: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:53:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:53:14.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:14 smithi144 ceph-mon[87436]: pgmap v637: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:14 smithi178 ceph-mon[65940]: pgmap v637: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:15.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:15 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:53:15.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:15 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:53:15.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:15 smithi178 ceph-mon[65940]: pgmap v638: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:15.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:15 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:15.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:15 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:53:15.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:15 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:53:15.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:15 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:53:15.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:15 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:53:15.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:15 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:53:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:15 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:53:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:15 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T11:53:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:15 smithi144 ceph-mon[87436]: pgmap v638: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:15 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:15 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:53:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:15 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:53:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:15 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:53:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:15 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:53:15.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:15 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:53:16.850 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:16 smithi178 ceph-mon[65940]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-09-10T11:53:16.850 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:16 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:16.850 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:16 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.czjrsa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:53:16.850 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:16 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:53:16.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:16 smithi144 ceph-mon[87436]: Health check failed: failed to probe daemons or devices (CEPHADM_REFRESH_FAILED) 2024-09-10T11:53:16.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:16 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:16.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:16 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi178.czjrsa", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-10T11:53:16.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:16 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T11:53:17.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:17 smithi178 ceph-mon[65940]: Upgrade: Updating mds.cephfs.smithi178.czjrsa 2024-09-10T11:53:17.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:17 smithi178 ceph-mon[65940]: Deploying daemon mds.cephfs.smithi178.czjrsa on smithi178 2024-09-10T11:53:17.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:17 smithi178 ceph-mon[65940]: pgmap v639: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:18.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:17 smithi144 ceph-mon[87436]: Upgrade: Updating mds.cephfs.smithi178.czjrsa 2024-09-10T11:53:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:17 smithi144 ceph-mon[87436]: Deploying daemon mds.cephfs.smithi178.czjrsa on smithi178 2024-09-10T11:53:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:17 smithi144 ceph-mon[87436]: pgmap v639: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:20.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:19 smithi178 ceph-mon[65940]: osdmap e85: 6 total, 6 up, 6 in 2024-09-10T11:53:20.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:19 smithi178 ceph-mon[65940]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T11:53:20.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:19 smithi178 ceph-mon[65940]: pgmap v641: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:20.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:19 smithi144 ceph-mon[87436]: osdmap e85: 6 total, 6 up, 6 in 2024-09-10T11:53:20.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:19 smithi144 ceph-mon[87436]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-09-10T11:53:20.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:19 smithi144 ceph-mon[87436]: pgmap v641: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:20 smithi178 ceph-mon[65940]: pgmap v642: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:21.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:20 smithi144 ceph-mon[87436]: pgmap v642: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:23.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:23 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] up:boot 2024-09-10T11:53:23.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:23 smithi144 ceph-mon[87436]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T11:53:23.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.czjrsa"}]: dispatch 2024-09-10T11:53:23.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:23.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:23.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:53:23.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:23 smithi144 ceph-mon[87436]: pgmap v643: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:23.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:23.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:23.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:23.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:23.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:23 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] up:boot 2024-09-10T11:53:23.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:23 smithi178 ceph-mon[65940]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T11:53:23.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:23 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi178.czjrsa"}]: dispatch 2024-09-10T11:53:23.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:23 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:23.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:23 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:23.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:23 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T11:53:23.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:23 smithi178 ceph-mon[65940]: pgmap v643: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:23.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:23 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:23.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:23 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:23.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:23 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:23.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:23 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:24 smithi178 ceph-mon[65940]: pgmap v644: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:25.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:24 smithi144 ceph-mon[87436]: pgmap v644: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:26 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:26.798 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:26 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:26.798 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:26 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:26.798 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:26 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:26.798 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:26 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:27.681 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:27 smithi178 ceph-mon[65940]: pgmap v645: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:27.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:27 smithi144 ceph-mon[87436]: pgmap v645: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:28.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:28.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:28.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:28.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:53:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T11:53:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:53:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:29 smithi178 ceph-mon[65940]: pgmap v646: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:30.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:29 smithi144 ceph-mon[87436]: pgmap v646: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:31.340 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:30 smithi178 ceph-mon[65940]: pgmap v647: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:31.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:30 smithi144 ceph-mon[87436]: pgmap v647: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:32 smithi178 ceph-mon[65940]: pgmap v648: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:33.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:32 smithi144 ceph-mon[87436]: pgmap v648: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:34 smithi178 ceph-mon[65940]: pgmap v649: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:35.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:34 smithi144 ceph-mon[87436]: pgmap v649: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:36 smithi178 ceph-mon[65940]: pgmap v650: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:37.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:36 smithi144 ceph-mon[87436]: pgmap v650: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:39.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:38 smithi178 ceph-mon[65940]: pgmap v651: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:39.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:38 smithi144 ceph-mon[87436]: pgmap v651: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:39.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.896+0000 7f23796dc700 1 -- 172.21.15.144:0/1974604070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2374101c30 msgr2=0x7f237410a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:39.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.896+0000 7f23796dc700 1 --2- 172.21.15.144:0/1974604070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2374101c30 0x7f237410a130 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f235c009910 tx=0x7f235c009c20 comp rx=0 tx=0).stop 2024-09-10T11:53:39.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.896+0000 7f23796dc700 1 -- 172.21.15.144:0/1974604070 shutdown_connections 2024-09-10T11:53:39.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.896+0000 7f23796dc700 1 --2- 172.21.15.144:0/1974604070 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2374101c30 0x7f237410a130 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:39.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.896+0000 7f23796dc700 1 --2- 172.21.15.144:0/1974604070 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2374101320 0x7f23741016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.896+0000 7f23796dc700 1 -- 172.21.15.144:0/1974604070 >> 172.21.15.144:0/1974604070 conn(0x7f23740fab60 msgr2=0x7f23740fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.896+0000 7f23796dc700 1 -- 172.21.15.144:0/1974604070 shutdown_connections 2024-09-10T11:53:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.896+0000 7f23796dc700 1 -- 172.21.15.144:0/1974604070 wait complete. 2024-09-10T11:53:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.897+0000 7f23796dc700 1 Processor -- start 2024-09-10T11:53:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.897+0000 7f23796dc700 1 -- start start 2024-09-10T11:53:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.897+0000 7f23796dc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2374101320 0x7f23740fef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.897+0000 7f23796dc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2374101c30 0x7f23740ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.897+0000 7f23796dc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740ffa00 con 0x7f2374101320 2024-09-10T11:53:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.897+0000 7f23796dc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23740ffb40 con 0x7f2374101c30 2024-09-10T11:53:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.897+0000 7f2372ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2374101320 0x7f23740fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.897+0000 7f2372ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2374101320 0x7f23740fef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:46216/0 (socket says 172.21.15.144:46216) 2024-09-10T11:53:39.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.897+0000 7f2372ffd700 1 -- 172.21.15.144:0/2166302693 learned_addr learned my addr 172.21.15.144:0/2166302693 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:53:39.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.897+0000 7f236a7fc700 1 --2- 172.21.15.144:0/2166302693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2374101c30 0x7f23740ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:39.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.898+0000 7f2372ffd700 1 -- 172.21.15.144:0/2166302693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2374101c30 msgr2=0x7f23740ff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:39.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.898+0000 7f2372ffd700 1 --2- 172.21.15.144:0/2166302693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2374101c30 0x7f23740ff4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:39.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.898+0000 7f2372ffd700 1 -- 172.21.15.144:0/2166302693 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f235c0095f0 con 0x7f2374101320 2024-09-10T11:53:39.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.898+0000 7f2372ffd700 1 --2- 172.21.15.144:0/2166302693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2374101320 0x7f23740fef80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f236400d8d0 tx=0x7f236400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:39.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.898+0000 7f2370ff9700 1 -- 172.21.15.144:0/2166302693 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23640098a0 con 0x7f2374101320 2024-09-10T11:53:39.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.898+0000 7f2370ff9700 1 -- 172.21.15.144:0/2166302693 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2364010460 con 0x7f2374101320 2024-09-10T11:53:39.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.898+0000 7f23796dc700 1 -- 172.21.15.144:0/2166302693 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23741a1ed0 con 0x7f2374101320 2024-09-10T11:53:39.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.898+0000 7f2370ff9700 1 -- 172.21.15.144:0/2166302693 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f236400f5f0 con 0x7f2374101320 2024-09-10T11:53:39.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.898+0000 7f23796dc700 1 -- 172.21.15.144:0/2166302693 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23741a22e0 con 0x7f2374101320 2024-09-10T11:53:39.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.899+0000 7f23796dc700 1 -- 172.21.15.144:0/2166302693 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23741077a0 con 0x7f2374101320 2024-09-10T11:53:39.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.899+0000 7f2370ff9700 1 -- 172.21.15.144:0/2166302693 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2364009a00 con 0x7f2374101320 2024-09-10T11:53:39.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.900+0000 7f2370ff9700 1 --2- 172.21.15.144:0/2166302693 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f236006e200 0x7f23600706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:39.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.900+0000 7f2370ff9700 1 -- 172.21.15.144:0/2166302693 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f236408df70 con 0x7f2374101320 2024-09-10T11:53:39.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.900+0000 7f236a7fc700 1 --2- 172.21.15.144:0/2166302693 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f236006e200 0x7f23600706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:39.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.900+0000 7f236a7fc700 1 --2- 172.21.15.144:0/2166302693 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f236006e200 0x7f23600706b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2374100770 tx=0x7f235c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:39.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:39.904+0000 7f2370ff9700 1 -- 172.21.15.144:0/2166302693 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f23640c31b0 con 0x7f2374101320 2024-09-10T11:53:40.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.124+0000 7f23796dc700 1 -- 172.21.15.144:0/2166302693 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f237402cd90 con 0x7f236006e200 2024-09-10T11:53:40.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.126+0000 7f2370ff9700 1 -- 172.21.15.144:0/2166302693 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f237402cd90 con 0x7f236006e200 2024-09-10T11:53:40.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.128+0000 7f23796dc700 1 -- 172.21.15.144:0/2166302693 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f236006e200 msgr2=0x7f23600706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:40.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.128+0000 7f23796dc700 1 --2- 172.21.15.144:0/2166302693 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f236006e200 0x7f23600706b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f2374100770 tx=0x7f235c01a040 comp rx=0 tx=0).stop 2024-09-10T11:53:40.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.128+0000 7f23796dc700 1 -- 172.21.15.144:0/2166302693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2374101320 msgr2=0x7f23740fef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:40.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.128+0000 7f23796dc700 1 --2- 172.21.15.144:0/2166302693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2374101320 0x7f23740fef80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f236400d8d0 tx=0x7f236400dbe0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.128+0000 7f23796dc700 1 -- 172.21.15.144:0/2166302693 shutdown_connections 2024-09-10T11:53:40.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.128+0000 7f23796dc700 1 --2- 172.21.15.144:0/2166302693 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f236006e200 0x7f23600706b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.128+0000 7f23796dc700 1 --2- 172.21.15.144:0/2166302693 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2374101320 0x7f23740fef80 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.128+0000 7f23796dc700 1 --2- 172.21.15.144:0/2166302693 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2374101c30 0x7f23740ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.128+0000 7f23796dc700 1 -- 172.21.15.144:0/2166302693 >> 172.21.15.144:0/2166302693 conn(0x7f23740fab60 msgr2=0x7f23741048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:40.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.128+0000 7f23796dc700 1 -- 172.21.15.144:0/2166302693 shutdown_connections 2024-09-10T11:53:40.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.128+0000 7f23796dc700 1 -- 172.21.15.144:0/2166302693 wait complete. 2024-09-10T11:53:40.140 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:53:40.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.236+0000 7f95a0974700 1 -- 172.21.15.144:0/251980549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c100d70 msgr2=0x7f959c104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:40.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.236+0000 7f95a0974700 1 --2- 172.21.15.144:0/251980549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c100d70 0x7f959c104d40 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9590009910 tx=0x7f9590009c20 comp rx=0 tx=0).stop 2024-09-10T11:53:40.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.237+0000 7f95a0974700 1 -- 172.21.15.144:0/251980549 shutdown_connections 2024-09-10T11:53:40.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.237+0000 7f95a0974700 1 --2- 172.21.15.144:0/251980549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c100d70 0x7f959c104d40 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.237+0000 7f95a0974700 1 --2- 172.21.15.144:0/251980549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959c1003d0 0x7f959c1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.237+0000 7f95a0974700 1 -- 172.21.15.144:0/251980549 >> 172.21.15.144:0/251980549 conn(0x7f959c0fbc90 msgr2=0x7f959c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:40.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.237+0000 7f95a0974700 1 -- 172.21.15.144:0/251980549 shutdown_connections 2024-09-10T11:53:40.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.237+0000 7f95a0974700 1 -- 172.21.15.144:0/251980549 wait complete. 2024-09-10T11:53:40.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.237+0000 7f95a0974700 1 Processor -- start 2024-09-10T11:53:40.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.237+0000 7f95a0974700 1 -- start start 2024-09-10T11:53:40.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f95a0974700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959c1003d0 0x7f959c195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:40.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f95a0974700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c100d70 0x7f959c196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:40.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f95a0974700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c196960 con 0x7f959c100d70 2024-09-10T11:53:40.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f95a0974700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959c19a6f0 con 0x7f959c1003d0 2024-09-10T11:53:40.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f9599d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c100d70 0x7f959c196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:40.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f9599d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c100d70 0x7f959c196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:46236/0 (socket says 172.21.15.144:46236) 2024-09-10T11:53:40.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f9599d9b700 1 -- 172.21.15.144:0/2391391542 learned_addr learned my addr 172.21.15.144:0/2391391542 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:53:40.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f959a59c700 1 --2- 172.21.15.144:0/2391391542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959c1003d0 0x7f959c195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:40.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f9599d9b700 1 -- 172.21.15.144:0/2391391542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959c1003d0 msgr2=0x7f959c195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:40.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f9599d9b700 1 --2- 172.21.15.144:0/2391391542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959c1003d0 0x7f959c195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f9599d9b700 1 -- 172.21.15.144:0/2391391542 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95900095f0 con 0x7f959c100d70 2024-09-10T11:53:40.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f959a59c700 1 --2- 172.21.15.144:0/2391391542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959c1003d0 0x7f959c195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T11:53:40.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f9599d9b700 1 --2- 172.21.15.144:0/2391391542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c100d70 0x7f959c196280 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f95900059d0 tx=0x7f959000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:40.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f958b7fe700 1 -- 172.21.15.144:0/2391391542 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f959001d070 con 0x7f959c100d70 2024-09-10T11:53:40.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f958b7fe700 1 -- 172.21.15.144:0/2391391542 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f959000b610 con 0x7f959c100d70 2024-09-10T11:53:40.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f95a0974700 1 -- 172.21.15.144:0/2391391542 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959c19a970 con 0x7f959c100d70 2024-09-10T11:53:40.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f958b7fe700 1 -- 172.21.15.144:0/2391391542 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9590022470 con 0x7f959c100d70 2024-09-10T11:53:40.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.238+0000 7f95a0974700 1 -- 172.21.15.144:0/2391391542 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959c19ae60 con 0x7f959c100d70 2024-09-10T11:53:40.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.239+0000 7f95a0974700 1 -- 172.21.15.144:0/2391391542 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f959c108740 con 0x7f959c100d70 2024-09-10T11:53:40.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.240+0000 7f958b7fe700 1 -- 172.21.15.144:0/2391391542 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9590003710 con 0x7f959c100d70 2024-09-10T11:53:40.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.241+0000 7f958b7fe700 1 --2- 172.21.15.144:0/2391391542 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f958406e200 0x7f95840706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:40.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.241+0000 7f958b7fe700 1 -- 172.21.15.144:0/2391391542 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f959008f6f0 con 0x7f959c100d70 2024-09-10T11:53:40.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.241+0000 7f959a59c700 1 --2- 172.21.15.144:0/2391391542 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f958406e200 0x7f95840706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:40.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.241+0000 7f959a59c700 1 --2- 172.21.15.144:0/2391391542 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f958406e200 0x7f95840706b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f958c011490 tx=0x7f958c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:40.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.244+0000 7f958b7fe700 1 -- 172.21.15.144:0/2391391542 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9590058b30 con 0x7f959c100d70 2024-09-10T11:53:40.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.467+0000 7f95a0974700 1 -- 172.21.15.144:0/2391391542 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f959c19b2a0 con 0x7f958406e200 2024-09-10T11:53:40.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.468+0000 7f958b7fe700 1 -- 172.21.15.144:0/2391391542 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f959c19b2a0 con 0x7f958406e200 2024-09-10T11:53:40.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.470+0000 7f95a0974700 1 -- 172.21.15.144:0/2391391542 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f958406e200 msgr2=0x7f95840706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:40.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.470+0000 7f95a0974700 1 --2- 172.21.15.144:0/2391391542 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f958406e200 0x7f95840706b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f958c011490 tx=0x7f958c009260 comp rx=0 tx=0).stop 2024-09-10T11:53:40.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.470+0000 7f95a0974700 1 -- 172.21.15.144:0/2391391542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c100d70 msgr2=0x7f959c196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:40.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.470+0000 7f95a0974700 1 --2- 172.21.15.144:0/2391391542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c100d70 0x7f959c196280 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f95900059d0 tx=0x7f959000f690 comp rx=0 tx=0).stop 2024-09-10T11:53:40.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.471+0000 7f95a0974700 1 -- 172.21.15.144:0/2391391542 shutdown_connections 2024-09-10T11:53:40.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.471+0000 7f95a0974700 1 --2- 172.21.15.144:0/2391391542 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f958406e200 0x7f95840706b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.471+0000 7f95a0974700 1 --2- 172.21.15.144:0/2391391542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f959c1003d0 0x7f959c195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.471+0000 7f95a0974700 1 --2- 172.21.15.144:0/2391391542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f959c100d70 0x7f959c196280 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.471+0000 7f95a0974700 1 -- 172.21.15.144:0/2391391542 >> 172.21.15.144:0/2391391542 conn(0x7f959c0fbc90 msgr2=0x7f959c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:40.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.471+0000 7f95a0974700 1 -- 172.21.15.144:0/2391391542 shutdown_connections 2024-09-10T11:53:40.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.471+0000 7f95a0974700 1 -- 172.21.15.144:0/2391391542 wait complete. 2024-09-10T11:53:40.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.576+0000 7f1c087bb700 1 -- 172.21.15.144:0/2797599313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c000ff2c0 msgr2=0x7f1c000ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:40.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.576+0000 7f1c087bb700 1 --2- 172.21.15.144:0/2797599313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c000ff2c0 0x7f1c000ff690 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc009910 tx=0x7f1bfc009c20 comp rx=0 tx=0).stop 2024-09-10T11:53:40.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.577+0000 7f1c087bb700 1 -- 172.21.15.144:0/2797599313 shutdown_connections 2024-09-10T11:53:40.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.577+0000 7f1c087bb700 1 --2- 172.21.15.144:0/2797599313 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c000ffc60 0x7f1c00103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.577+0000 7f1c087bb700 1 --2- 172.21.15.144:0/2797599313 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c000ff2c0 0x7f1c000ff690 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.577+0000 7f1c087bb700 1 -- 172.21.15.144:0/2797599313 >> 172.21.15.144:0/2797599313 conn(0x7f1c000fab80 msgr2=0x7f1c000fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:40.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.577+0000 7f1c087bb700 1 -- 172.21.15.144:0/2797599313 shutdown_connections 2024-09-10T11:53:40.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.577+0000 7f1c087bb700 1 -- 172.21.15.144:0/2797599313 wait complete. 2024-09-10T11:53:40.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.577+0000 7f1c087bb700 1 Processor -- start 2024-09-10T11:53:40.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.577+0000 7f1c087bb700 1 -- start start 2024-09-10T11:53:40.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.578+0000 7f1c087bb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c000ff2c0 0x7f1c00196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:40.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.578+0000 7f1c087bb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c000ffc60 0x7f1c00197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:40.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.578+0000 7f1c087bb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c00197720 con 0x7f1c000ffc60 2024-09-10T11:53:40.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.578+0000 7f1c087bb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c00197860 con 0x7f1c000ff2c0 2024-09-10T11:53:40.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.578+0000 7f1c05d56700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c000ffc60 0x7f1c00197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:40.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.578+0000 7f1c05d56700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c000ffc60 0x7f1c00197190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:46244/0 (socket says 172.21.15.144:46244) 2024-09-10T11:53:40.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.578+0000 7f1c05d56700 1 -- 172.21.15.144:0/2110399524 learned_addr learned my addr 172.21.15.144:0/2110399524 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:53:40.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.578+0000 7f1c06557700 1 --2- 172.21.15.144:0/2110399524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c000ff2c0 0x7f1c00196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:40.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.578+0000 7f1c05d56700 1 -- 172.21.15.144:0/2110399524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c000ff2c0 msgr2=0x7f1c00196c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:40.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.578+0000 7f1c05d56700 1 --2- 172.21.15.144:0/2110399524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c000ff2c0 0x7f1c00196c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.578+0000 7f1c05d56700 1 -- 172.21.15.144:0/2110399524 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bfc0095f0 con 0x7f1c000ffc60 2024-09-10T11:53:40.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.578+0000 7f1c05d56700 1 --2- 172.21.15.144:0/2110399524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c000ffc60 0x7f1c00197190 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000c960 tx=0x7f1bf000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:40.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.578+0000 7f1bf77fe700 1 -- 172.21.15.144:0/2110399524 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bf0007a70 con 0x7f1c000ffc60 2024-09-10T11:53:40.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.579+0000 7f1bf77fe700 1 -- 172.21.15.144:0/2110399524 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1bf0007bd0 con 0x7f1c000ffc60 2024-09-10T11:53:40.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.579+0000 7f1c087bb700 1 -- 172.21.15.144:0/2110399524 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0019fcc0 con 0x7f1c000ffc60 2024-09-10T11:53:40.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.579+0000 7f1bf77fe700 1 -- 172.21.15.144:0/2110399524 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bf0018630 con 0x7f1c000ffc60 2024-09-10T11:53:40.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.579+0000 7f1c087bb700 1 -- 172.21.15.144:0/2110399524 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c001a0130 con 0x7f1c000ffc60 2024-09-10T11:53:40.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.580+0000 7f1c087bb700 1 -- 172.21.15.144:0/2110399524 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c001075d0 con 0x7f1c000ffc60 2024-09-10T11:53:40.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.580+0000 7f1bf77fe700 1 -- 172.21.15.144:0/2110399524 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1bf000f450 con 0x7f1c000ffc60 2024-09-10T11:53:40.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.581+0000 7f1bf77fe700 1 --2- 172.21.15.144:0/2110399524 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1bec06e200 0x7f1bec0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:40.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.581+0000 7f1bf77fe700 1 -- 172.21.15.144:0/2110399524 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1bf008df80 con 0x7f1c000ffc60 2024-09-10T11:53:40.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.581+0000 7f1c06557700 1 --2- 172.21.15.144:0/2110399524 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1bec06e200 0x7f1bec0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:40.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.581+0000 7f1c06557700 1 --2- 172.21.15.144:0/2110399524 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1bec06e200 0x7f1bec0706b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc006010 tx=0x7f1bfc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:40.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.585+0000 7f1bf77fe700 1 -- 172.21.15.144:0/2110399524 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1bf00573e0 con 0x7f1c000ffc60 2024-09-10T11:53:40.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.794+0000 7f1c087bb700 1 -- 172.21.15.144:0/2110399524 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c0002cca0 con 0x7f1bec06e200 2024-09-10T11:53:40.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.803+0000 7f1bf77fe700 1 -- 172.21.15.144:0/2110399524 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f1c0002cca0 con 0x7f1bec06e200 2024-09-10T11:53:40.805 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:53:40.805 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (21m) 13s ago 32m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:53:40.805 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (32m) 13s ago 32m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:53:40.805 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (31m) 15s ago 31m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:53:40.805 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (19m) 13s ago 32m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:53:40.805 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (19m) 15s ago 31m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:53:40.805 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (21m) 13s ago 32m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:53:40.805 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (15m) 13s ago 27m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:53:40.805 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (15m) 13s ago 26m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:53:40.805 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (18s) 15s ago 26m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T11:53:40.805 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (15m) 15s ago 26m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:53:40.805 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (23m) 13s ago 33m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:53:40.805 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (22m) 15s ago 31m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:53:40.806 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (20m) 13s ago 34m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:53:40.806 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (20m) 15s ago 31m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:53:40.806 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (22m) 13s ago 32m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:53:40.806 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (22m) 15s ago 31m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:53:40.806 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (19m) 13s ago 30m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:53:40.806 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (18m) 13s ago 30m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:53:40.806 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (17m) 13s ago 29m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:53:40.806 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (17m) 15s ago 29m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:53:40.806 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (16m) 15s ago 28m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:53:40.806 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (16m) 15s ago 28m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:53:40.806 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (22m) 13s ago 32m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:53:40.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.806+0000 7f1c087bb700 1 -- 172.21.15.144:0/2110399524 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1bec06e200 msgr2=0x7f1bec0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:40.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.806+0000 7f1c087bb700 1 --2- 172.21.15.144:0/2110399524 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1bec06e200 0x7f1bec0706b0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f1bfc006010 tx=0x7f1bfc01a040 comp rx=0 tx=0).stop 2024-09-10T11:53:40.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.806+0000 7f1c087bb700 1 -- 172.21.15.144:0/2110399524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c000ffc60 msgr2=0x7f1c00197190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:40.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.806+0000 7f1c087bb700 1 --2- 172.21.15.144:0/2110399524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c000ffc60 0x7f1c00197190 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1bf000c960 tx=0x7f1bf000cc70 comp rx=0 tx=0).stop 2024-09-10T11:53:40.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.806+0000 7f1c087bb700 1 -- 172.21.15.144:0/2110399524 shutdown_connections 2024-09-10T11:53:40.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.806+0000 7f1c087bb700 1 --2- 172.21.15.144:0/2110399524 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1bec06e200 0x7f1bec0706b0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.806+0000 7f1c087bb700 1 --2- 172.21.15.144:0/2110399524 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1c000ff2c0 0x7f1c00196c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.806+0000 7f1c087bb700 1 --2- 172.21.15.144:0/2110399524 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1c000ffc60 0x7f1c00197190 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.806+0000 7f1c087bb700 1 -- 172.21.15.144:0/2110399524 >> 172.21.15.144:0/2110399524 conn(0x7f1c000fab80 msgr2=0x7f1c000fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:40.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.806+0000 7f1c087bb700 1 -- 172.21.15.144:0/2110399524 shutdown_connections 2024-09-10T11:53:40.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.806+0000 7f1c087bb700 1 -- 172.21.15.144:0/2110399524 wait complete. 2024-09-10T11:53:40.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.909+0000 7f35083d6700 1 -- 172.21.15.144:0/2891930247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3500101c30 msgr2=0x7f350010a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:40.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.909+0000 7f35083d6700 1 --2- 172.21.15.144:0/2891930247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3500101c30 0x7f350010a130 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f34f0009910 tx=0x7f34f0009c20 comp rx=0 tx=0).stop 2024-09-10T11:53:40.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.909+0000 7f35083d6700 1 -- 172.21.15.144:0/2891930247 shutdown_connections 2024-09-10T11:53:40.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.909+0000 7f35083d6700 1 --2- 172.21.15.144:0/2891930247 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3500101c30 0x7f350010a130 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.909+0000 7f35083d6700 1 --2- 172.21.15.144:0/2891930247 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3500101320 0x7f35001016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.909+0000 7f35083d6700 1 -- 172.21.15.144:0/2891930247 >> 172.21.15.144:0/2891930247 conn(0x7f35000fab80 msgr2=0x7f35000fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:40.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.909+0000 7f35083d6700 1 -- 172.21.15.144:0/2891930247 shutdown_connections 2024-09-10T11:53:40.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.909+0000 7f35083d6700 1 -- 172.21.15.144:0/2891930247 wait complete. 2024-09-10T11:53:40.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.910+0000 7f35083d6700 1 Processor -- start 2024-09-10T11:53:40.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.910+0000 7f35083d6700 1 -- start start 2024-09-10T11:53:40.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.910+0000 7f35083d6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3500101320 0x7f35000fef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:40.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.910+0000 7f35083d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3500101c30 0x7f35000ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:40.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.910+0000 7f35083d6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35000ffa00 con 0x7f3500101c30 2024-09-10T11:53:40.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.910+0000 7f35083d6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35000ffb40 con 0x7f3500101320 2024-09-10T11:53:40.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.911+0000 7f3505971700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3500101c30 0x7f35000ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:40.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.911+0000 7f3505971700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3500101c30 0x7f35000ff4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:46262/0 (socket says 172.21.15.144:46262) 2024-09-10T11:53:40.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.911+0000 7f3505971700 1 -- 172.21.15.144:0/3765637314 learned_addr learned my addr 172.21.15.144:0/3765637314 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:53:40.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.911+0000 7f3506172700 1 --2- 172.21.15.144:0/3765637314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3500101320 0x7f35000fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:40.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.911+0000 7f3505971700 1 -- 172.21.15.144:0/3765637314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3500101320 msgr2=0x7f35000fef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:40.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.911+0000 7f3505971700 1 --2- 172.21.15.144:0/3765637314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3500101320 0x7f35000fef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:40.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.911+0000 7f3505971700 1 -- 172.21.15.144:0/3765637314 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f00095f0 con 0x7f3500101c30 2024-09-10T11:53:40.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.911+0000 7f3505971700 1 --2- 172.21.15.144:0/3765637314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3500101c30 0x7f35000ff4c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f34f00092e0 tx=0x7f34f000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:40.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.911+0000 7f34f77fe700 1 -- 172.21.15.144:0/3765637314 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34f001d070 con 0x7f3500101c30 2024-09-10T11:53:40.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.911+0000 7f34f77fe700 1 -- 172.21.15.144:0/3765637314 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f34f0005e00 con 0x7f3500101c30 2024-09-10T11:53:40.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.911+0000 7f35083d6700 1 -- 172.21.15.144:0/3765637314 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35000ffdc0 con 0x7f3500101c30 2024-09-10T11:53:40.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.911+0000 7f34f77fe700 1 -- 172.21.15.144:0/3765637314 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34f0017400 con 0x7f3500101c30 2024-09-10T11:53:40.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.911+0000 7f35083d6700 1 -- 172.21.15.144:0/3765637314 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35001a21e0 con 0x7f3500101c30 2024-09-10T11:53:40.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.912+0000 7f35083d6700 1 -- 172.21.15.144:0/3765637314 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35001077a0 con 0x7f3500101c30 2024-09-10T11:53:40.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.914+0000 7f34f77fe700 1 -- 172.21.15.144:0/3765637314 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f34f000f6f0 con 0x7f3500101c30 2024-09-10T11:53:40.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.914+0000 7f34f77fe700 1 --2- 172.21.15.144:0/3765637314 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f34ec06e200 0x7f34ec0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:40.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.914+0000 7f3506172700 1 --2- 172.21.15.144:0/3765637314 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f34ec06e200 0x7f34ec0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:40.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.914+0000 7f34f77fe700 1 -- 172.21.15.144:0/3765637314 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f34f008f750 con 0x7f3500101c30 2024-09-10T11:53:40.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.915+0000 7f3506172700 1 --2- 172.21.15.144:0/3765637314 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f34ec06e200 0x7f34ec0706b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f34fc007be0 tx=0x7f34fc0073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:40.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:40.917+0000 7f34f77fe700 1 -- 172.21.15.144:0/3765637314 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f34f0058c50 con 0x7f3500101c30 2024-09-10T11:53:41.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:40 smithi144 ceph-mon[87436]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:40 smithi144 ceph-mon[87436]: from='client.35216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:40 smithi144 ceph-mon[87436]: from='client.35220 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:40 smithi144 ceph-mon[87436]: pgmap v652: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:41.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.201+0000 7f35083d6700 1 -- 172.21.15.144:0/3765637314 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f350004ea50 con 0x7f3500101c30 2024-09-10T11:53:41.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.202+0000 7f34f77fe700 1 -- 172.21.15.144:0/3765637314 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f34f005c310 con 0x7f3500101c30 2024-09-10T11:53:41.203 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:53:41.203 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:53:41.203 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:53:41.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:53:41.203 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:53:41.203 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:53:41.203 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:53:41.204 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:53:41.204 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:53:41.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:53:41.204 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:53:41.204 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:53:41.204 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:53:41.204 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:53:41.204 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T11:53:41.204 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:53:41.204 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:53:41.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.204+0000 7f35083d6700 1 -- 172.21.15.144:0/3765637314 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f34ec06e200 msgr2=0x7f34ec0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:41.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.204+0000 7f35083d6700 1 --2- 172.21.15.144:0/3765637314 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f34ec06e200 0x7f34ec0706b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f34fc007be0 tx=0x7f34fc0073d0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.204+0000 7f35083d6700 1 -- 172.21.15.144:0/3765637314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3500101c30 msgr2=0x7f35000ff4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:41.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.204+0000 7f35083d6700 1 --2- 172.21.15.144:0/3765637314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3500101c30 0x7f35000ff4c0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f34f00092e0 tx=0x7f34f000bd70 comp rx=0 tx=0).stop 2024-09-10T11:53:41.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.204+0000 7f35083d6700 1 -- 172.21.15.144:0/3765637314 shutdown_connections 2024-09-10T11:53:41.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.204+0000 7f35083d6700 1 --2- 172.21.15.144:0/3765637314 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f34ec06e200 0x7f34ec0706b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.204+0000 7f35083d6700 1 --2- 172.21.15.144:0/3765637314 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3500101320 0x7f35000fef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.204+0000 7f35083d6700 1 --2- 172.21.15.144:0/3765637314 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3500101c30 0x7f35000ff4c0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.204+0000 7f35083d6700 1 -- 172.21.15.144:0/3765637314 >> 172.21.15.144:0/3765637314 conn(0x7f35000fab80 msgr2=0x7f35001048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:41.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.204+0000 7f35083d6700 1 -- 172.21.15.144:0/3765637314 shutdown_connections 2024-09-10T11:53:41.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.204+0000 7f35083d6700 1 -- 172.21.15.144:0/3765637314 wait complete. 2024-09-10T11:53:41.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.309+0000 7f73dde40700 1 -- 172.21.15.144:0/986650649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73d80ffba0 msgr2=0x7f73d810a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:41.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.309+0000 7f73dde40700 1 --2- 172.21.15.144:0/986650649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73d80ffba0 0x7f73d810a610 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f73cc009910 tx=0x7f73cc009c20 comp rx=0 tx=0).stop 2024-09-10T11:53:41.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.309+0000 7f73dde40700 1 -- 172.21.15.144:0/986650649 shutdown_connections 2024-09-10T11:53:41.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.309+0000 7f73dde40700 1 --2- 172.21.15.144:0/986650649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73d80ffba0 0x7f73d810a610 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.309+0000 7f73dde40700 1 --2- 172.21.15.144:0/986650649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73d80ff290 0x7f73d80ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.309+0000 7f73dde40700 1 -- 172.21.15.144:0/986650649 >> 172.21.15.144:0/986650649 conn(0x7f73d8074850 msgr2=0x7f73d8074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:41.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.309+0000 7f73dde40700 1 -- 172.21.15.144:0/986650649 shutdown_connections 2024-09-10T11:53:41.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.309+0000 7f73dde40700 1 -- 172.21.15.144:0/986650649 wait complete. 2024-09-10T11:53:41.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.309+0000 7f73dde40700 1 Processor -- start 2024-09-10T11:53:41.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.310+0000 7f73dde40700 1 -- start start 2024-09-10T11:53:41.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.310+0000 7f73dde40700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73d80ff290 0x7f73d8195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:41.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.310+0000 7f73dde40700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73d80ffba0 0x7f73d81962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:41.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.310+0000 7f73dde40700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d8196980 con 0x7f73d80ff290 2024-09-10T11:53:41.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.310+0000 7f73dde40700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d819a710 con 0x7f73d80ffba0 2024-09-10T11:53:41.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.310+0000 7f73d77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73d80ff290 0x7f73d8195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:41.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.310+0000 7f73d77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73d80ff290 0x7f73d8195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:46278/0 (socket says 172.21.15.144:46278) 2024-09-10T11:53:41.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.310+0000 7f73d77fe700 1 -- 172.21.15.144:0/43750676 learned_addr learned my addr 172.21.15.144:0/43750676 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:53:41.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.310+0000 7f73d6ffd700 1 --2- 172.21.15.144:0/43750676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73d80ffba0 0x7f73d81962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:41.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.310+0000 7f73d77fe700 1 -- 172.21.15.144:0/43750676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73d80ffba0 msgr2=0x7f73d81962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:41.313 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.310+0000 7f73d77fe700 1 --2- 172.21.15.144:0/43750676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73d80ffba0 0x7f73d81962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.310+0000 7f73d77fe700 1 -- 172.21.15.144:0/43750676 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73cc0095f0 con 0x7f73d80ff290 2024-09-10T11:53:41.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.310+0000 7f73d77fe700 1 --2- 172.21.15.144:0/43750676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73d80ff290 0x7f73d8195d60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f73c800ada0 tx=0x7f73c800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:41.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.311+0000 7f73d4ff9700 1 -- 172.21.15.144:0/43750676 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73c800f070 con 0x7f73d80ff290 2024-09-10T11:53:41.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.311+0000 7f73d4ff9700 1 -- 172.21.15.144:0/43750676 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f73c800ae90 con 0x7f73d80ff290 2024-09-10T11:53:41.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.311+0000 7f73d4ff9700 1 -- 172.21.15.144:0/43750676 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73c80083f0 con 0x7f73d80ff290 2024-09-10T11:53:41.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.311+0000 7f73dde40700 1 -- 172.21.15.144:0/43750676 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73d819a9f0 con 0x7f73d80ff290 2024-09-10T11:53:41.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.311+0000 7f73dde40700 1 -- 172.21.15.144:0/43750676 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73d819af40 con 0x7f73d80ff290 2024-09-10T11:53:41.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.312+0000 7f73dde40700 1 -- 172.21.15.144:0/43750676 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73d8107d10 con 0x7f73d80ff290 2024-09-10T11:53:41.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.312+0000 7f73d4ff9700 1 -- 172.21.15.144:0/43750676 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f73c800cad0 con 0x7f73d80ff290 2024-09-10T11:53:41.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.313+0000 7f73d4ff9700 1 --2- 172.21.15.144:0/43750676 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f73c006e200 0x7f73c00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:41.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.313+0000 7f73d4ff9700 1 -- 172.21.15.144:0/43750676 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f73c808d950 con 0x7f73d80ff290 2024-09-10T11:53:41.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.313+0000 7f73d6ffd700 1 --2- 172.21.15.144:0/43750676 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f73c006e200 0x7f73c00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:41.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.313+0000 7f73d6ffd700 1 --2- 172.21.15.144:0/43750676 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f73c006e200 0x7f73c00706b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f73d8197380 tx=0x7f73cc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:41.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.317+0000 7f73d4ff9700 1 -- 172.21.15.144:0/43750676 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f73c8056d30 con 0x7f73d80ff290 2024-09-10T11:53:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:40 smithi178 ceph-mon[65940]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:40 smithi178 ceph-mon[65940]: from='client.35216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:40 smithi178 ceph-mon[65940]: from='client.35220 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:40 smithi178 ceph-mon[65940]: pgmap v652: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:41.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.564+0000 7f73dde40700 1 -- 172.21.15.144:0/43750676 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f73d804ea50 con 0x7f73d80ff290 2024-09-10T11:53:41.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.564+0000 7f73d4ff9700 1 -- 172.21.15.144:0/43750676 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f73c805a3f0 con 0x7f73d80ff290 2024-09-10T11:53:41.566 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T11:53:41.566 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T11:53:41.566 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:53:41.566 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:53:41.566 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:53:41.566 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:53:41.566 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:53:41.566 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:53:41.566 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:53:41.567 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:53:41.567 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:53:41.567 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:53:41.567 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:53:41.567 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:53:41.567 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:53:41.567 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:53:41.567 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:53:41.567 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:53:41.567 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:53:41.567 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:53:41.567 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:53:41.567 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:53:41.567 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:53:41.568 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:53:41.568 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:53:41.568 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:53:41.568 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:53:41.568 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:53:41.568 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:53:41.568 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:53:41.568 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:53:41.568 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:53:41.568 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:53:41.568 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:53:41.568 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:53:41.568 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:53:41.568 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:53:41.568 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:53:41.569 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:53:41.569 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:53:41.569 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:53:41.569 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:53:41.569 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:53:41.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.567+0000 7f73dde40700 1 -- 172.21.15.144:0/43750676 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f73c006e200 msgr2=0x7f73c00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:41.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.567+0000 7f73dde40700 1 --2- 172.21.15.144:0/43750676 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f73c006e200 0x7f73c00706b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f73d8197380 tx=0x7f73cc01a040 comp rx=0 tx=0).stop 2024-09-10T11:53:41.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.567+0000 7f73dde40700 1 -- 172.21.15.144:0/43750676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73d80ff290 msgr2=0x7f73d8195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:41.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.567+0000 7f73dde40700 1 --2- 172.21.15.144:0/43750676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73d80ff290 0x7f73d8195d60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f73c800ada0 tx=0x7f73c800c330 comp rx=0 tx=0).stop 2024-09-10T11:53:41.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.567+0000 7f73dde40700 1 -- 172.21.15.144:0/43750676 shutdown_connections 2024-09-10T11:53:41.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.567+0000 7f73dde40700 1 --2- 172.21.15.144:0/43750676 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f73c006e200 0x7f73c00706b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.567+0000 7f73dde40700 1 --2- 172.21.15.144:0/43750676 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f73d80ff290 0x7f73d8195d60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.567+0000 7f73dde40700 1 --2- 172.21.15.144:0/43750676 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f73d80ffba0 0x7f73d81962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.567+0000 7f73dde40700 1 -- 172.21.15.144:0/43750676 >> 172.21.15.144:0/43750676 conn(0x7f73d8074850 msgr2=0x7f73d8104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:41.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.567+0000 7f73dde40700 1 -- 172.21.15.144:0/43750676 shutdown_connections 2024-09-10T11:53:41.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.567+0000 7f73dde40700 1 -- 172.21.15.144:0/43750676 wait complete. 2024-09-10T11:53:41.571 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T11:53:41.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.673+0000 7f32a8073700 1 -- 172.21.15.144:0/753160444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a00ff2c0 msgr2=0x7f32a00ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:41.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.673+0000 7f32a8073700 1 --2- 172.21.15.144:0/753160444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a00ff2c0 0x7f32a00ff690 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f3294009910 tx=0x7f3294009c20 comp rx=0 tx=0).stop 2024-09-10T11:53:41.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.673+0000 7f32a8073700 1 -- 172.21.15.144:0/753160444 shutdown_connections 2024-09-10T11:53:41.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.673+0000 7f32a8073700 1 --2- 172.21.15.144:0/753160444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32a00ffc60 0x7f32a0103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.673+0000 7f32a8073700 1 --2- 172.21.15.144:0/753160444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a00ff2c0 0x7f32a00ff690 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.673+0000 7f32a8073700 1 -- 172.21.15.144:0/753160444 >> 172.21.15.144:0/753160444 conn(0x7f32a00fab80 msgr2=0x7f32a00fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:41.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.674+0000 7f32a8073700 1 -- 172.21.15.144:0/753160444 shutdown_connections 2024-09-10T11:53:41.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.674+0000 7f32a8073700 1 -- 172.21.15.144:0/753160444 wait complete. 2024-09-10T11:53:41.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.674+0000 7f32a8073700 1 Processor -- start 2024-09-10T11:53:41.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.674+0000 7f32a8073700 1 -- start start 2024-09-10T11:53:41.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.674+0000 7f32a8073700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32a00ff2c0 0x7f32a0196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:41.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.674+0000 7f32a8073700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a00ffc60 0x7f32a0197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:41.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.674+0000 7f32a8073700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a01977d0 con 0x7f32a00ffc60 2024-09-10T11:53:41.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.674+0000 7f32a8073700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a0197910 con 0x7f32a00ff2c0 2024-09-10T11:53:41.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.674+0000 7f32a560e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a00ffc60 0x7f32a0197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:41.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.675+0000 7f32a560e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a00ffc60 0x7f32a0197200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:46296/0 (socket says 172.21.15.144:46296) 2024-09-10T11:53:41.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.675+0000 7f32a560e700 1 -- 172.21.15.144:0/852810988 learned_addr learned my addr 172.21.15.144:0/852810988 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:53:41.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.675+0000 7f32a5e0f700 1 --2- 172.21.15.144:0/852810988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32a00ff2c0 0x7f32a0196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:41.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.675+0000 7f32a560e700 1 -- 172.21.15.144:0/852810988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32a00ff2c0 msgr2=0x7f32a0196cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:41.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.675+0000 7f32a560e700 1 --2- 172.21.15.144:0/852810988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32a00ff2c0 0x7f32a0196cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.675+0000 7f32a560e700 1 -- 172.21.15.144:0/852810988 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32940095f0 con 0x7f32a00ffc60 2024-09-10T11:53:41.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.675+0000 7f32a5e0f700 1 --2- 172.21.15.144:0/852810988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32a00ff2c0 0x7f32a0196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:53:41.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.675+0000 7f32a560e700 1 --2- 172.21.15.144:0/852810988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a00ffc60 0x7f32a0197200 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f329c00b480 tx=0x7f329c00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:41.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.675+0000 7f3292ffd700 1 -- 172.21.15.144:0/852810988 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f329c0118c0 con 0x7f32a00ffc60 2024-09-10T11:53:41.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.675+0000 7f3292ffd700 1 -- 172.21.15.144:0/852810988 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f329c010380 con 0x7f32a00ffc60 2024-09-10T11:53:41.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.675+0000 7f32a8073700 1 -- 172.21.15.144:0/852810988 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a019fd30 con 0x7f32a00ffc60 2024-09-10T11:53:41.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.675+0000 7f3292ffd700 1 -- 172.21.15.144:0/852810988 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f329c00f4f0 con 0x7f32a00ffc60 2024-09-10T11:53:41.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.675+0000 7f32a8073700 1 -- 172.21.15.144:0/852810988 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a01a01a0 con 0x7f32a00ffc60 2024-09-10T11:53:41.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.676+0000 7f32a8073700 1 -- 172.21.15.144:0/852810988 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a01075d0 con 0x7f32a00ffc60 2024-09-10T11:53:41.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.677+0000 7f3292ffd700 1 -- 172.21.15.144:0/852810988 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f329c0104f0 con 0x7f32a00ffc60 2024-09-10T11:53:41.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.677+0000 7f3292ffd700 1 --2- 172.21.15.144:0/852810988 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f328c06e200 0x7f328c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:41.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.677+0000 7f3292ffd700 1 -- 172.21.15.144:0/852810988 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f329c08dc20 con 0x7f32a00ffc60 2024-09-10T11:53:41.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.677+0000 7f32a5e0f700 1 --2- 172.21.15.144:0/852810988 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f328c06e200 0x7f328c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:41.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.678+0000 7f32a5e0f700 1 --2- 172.21.15.144:0/852810988 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f328c06e200 0x7f328c0706b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f32940059d0 tx=0x7f329401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:41.682 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.682+0000 7f3292ffd700 1 -- 172.21.15.144:0/852810988 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f329c057080 con 0x7f32a00ffc60 2024-09-10T11:53:41.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.901+0000 7f32a8073700 1 -- 172.21.15.144:0/852810988 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f32a002cd40 con 0x7f328c06e200 2024-09-10T11:53:41.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.903+0000 7f3292ffd700 1 -- 172.21.15.144:0/852810988 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f32a002cd40 con 0x7f328c06e200 2024-09-10T11:53:41.904 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:53:41.904 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:53:41.904 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:53:41.905 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:53:41.905 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:53:41.905 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:53:41.905 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T11:53:41.905 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T11:53:41.905 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:53:41.905 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T11:53:41.905 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:53:41.905 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T11:53:41.905 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:53:41.905 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:53:41.905 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:53:41.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.905+0000 7f32a8073700 1 -- 172.21.15.144:0/852810988 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f328c06e200 msgr2=0x7f328c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:41.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.905+0000 7f32a8073700 1 --2- 172.21.15.144:0/852810988 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f328c06e200 0x7f328c0706b0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f32940059d0 tx=0x7f329401a040 comp rx=0 tx=0).stop 2024-09-10T11:53:41.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.905+0000 7f32a8073700 1 -- 172.21.15.144:0/852810988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a00ffc60 msgr2=0x7f32a0197200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:41.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.905+0000 7f32a8073700 1 --2- 172.21.15.144:0/852810988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a00ffc60 0x7f32a0197200 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f329c00b480 tx=0x7f329c00b840 comp rx=0 tx=0).stop 2024-09-10T11:53:41.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.905+0000 7f32a8073700 1 -- 172.21.15.144:0/852810988 shutdown_connections 2024-09-10T11:53:41.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.905+0000 7f32a8073700 1 --2- 172.21.15.144:0/852810988 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f328c06e200 0x7f328c0706b0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.906+0000 7f32a8073700 1 --2- 172.21.15.144:0/852810988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f32a00ff2c0 0x7f32a0196cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.906+0000 7f32a8073700 1 --2- 172.21.15.144:0/852810988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f32a00ffc60 0x7f32a0197200 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:41.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.906+0000 7f32a8073700 1 -- 172.21.15.144:0/852810988 >> 172.21.15.144:0/852810988 conn(0x7f32a00fab80 msgr2=0x7f32a00fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:41.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.906+0000 7f32a8073700 1 -- 172.21.15.144:0/852810988 shutdown_connections 2024-09-10T11:53:41.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:41.906+0000 7f32a8073700 1 -- 172.21.15.144:0/852810988 wait complete. 2024-09-10T11:53:42.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.011+0000 7feb19e1b700 1 -- 172.21.15.144:0/4035914148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb14103450 msgr2=0x7feb141038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:42.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.011+0000 7feb19e1b700 1 --2- 172.21.15.144:0/4035914148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb14103450 0x7feb141038c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7feb0800aa20 tx=0x7feb0800ad30 comp rx=0 tx=0).stop 2024-09-10T11:53:42.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.011+0000 7feb19e1b700 1 -- 172.21.15.144:0/4035914148 shutdown_connections 2024-09-10T11:53:42.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.011+0000 7feb19e1b700 1 --2- 172.21.15.144:0/4035914148 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb14103450 0x7feb141038c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:42.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.011+0000 7feb19e1b700 1 --2- 172.21.15.144:0/4035914148 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb14104790 0x7feb14104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:42.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.011+0000 7feb19e1b700 1 -- 172.21.15.144:0/4035914148 >> 172.21.15.144:0/4035914148 conn(0x7feb14074850 msgr2=0x7feb14074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:42.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.011+0000 7feb19e1b700 1 -- 172.21.15.144:0/4035914148 shutdown_connections 2024-09-10T11:53:42.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.011+0000 7feb19e1b700 1 -- 172.21.15.144:0/4035914148 wait complete. 2024-09-10T11:53:42.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.011+0000 7feb19e1b700 1 Processor -- start 2024-09-10T11:53:42.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.012+0000 7feb19e1b700 1 -- start start 2024-09-10T11:53:42.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.012+0000 7feb19e1b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb14103450 0x7feb1419bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:42.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.012+0000 7feb19e1b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb14104790 0x7feb1419bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:42.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.012+0000 7feb19e1b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb1419c680 con 0x7feb14103450 2024-09-10T11:53:42.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.012+0000 7feb19e1b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb14195b30 con 0x7feb14104790 2024-09-10T11:53:42.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.012+0000 7feb137fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb14103450 0x7feb1419bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:42.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.012+0000 7feb137fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb14103450 0x7feb1419bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:46316/0 (socket says 172.21.15.144:46316) 2024-09-10T11:53:42.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.012+0000 7feb137fe700 1 -- 172.21.15.144:0/1799510649 learned_addr learned my addr 172.21.15.144:0/1799510649 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:53:42.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.012+0000 7feb12ffd700 1 --2- 172.21.15.144:0/1799510649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb14104790 0x7feb1419bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:42.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.012+0000 7feb137fe700 1 -- 172.21.15.144:0/1799510649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb14104790 msgr2=0x7feb1419bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:42.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.012+0000 7feb137fe700 1 --2- 172.21.15.144:0/1799510649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb14104790 0x7feb1419bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:42.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.012+0000 7feb137fe700 1 -- 172.21.15.144:0/1799510649 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0800a700 con 0x7feb14103450 2024-09-10T11:53:42.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.012+0000 7feb137fe700 1 --2- 172.21.15.144:0/1799510649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb14103450 0x7feb1419bab0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7feb0400d8d0 tx=0x7feb0400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:42.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.013+0000 7feb10ff9700 1 -- 172.21.15.144:0/1799510649 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb040098a0 con 0x7feb14103450 2024-09-10T11:53:42.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.013+0000 7feb10ff9700 1 -- 172.21.15.144:0/1799510649 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb04010460 con 0x7feb14103450 2024-09-10T11:53:42.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.013+0000 7feb19e1b700 1 -- 172.21.15.144:0/1799510649 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb14195e10 con 0x7feb14103450 2024-09-10T11:53:42.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.013+0000 7feb10ff9700 1 -- 172.21.15.144:0/1799510649 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb0400f5f0 con 0x7feb14103450 2024-09-10T11:53:42.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.013+0000 7feb19e1b700 1 -- 172.21.15.144:0/1799510649 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb14196330 con 0x7feb14103450 2024-09-10T11:53:42.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.014+0000 7feb19e1b700 1 -- 172.21.15.144:0/1799510649 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb140ffa20 con 0x7feb14103450 2024-09-10T11:53:42.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.014+0000 7feb10ff9700 1 -- 172.21.15.144:0/1799510649 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7feb04009a00 con 0x7feb14103450 2024-09-10T11:53:42.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.015+0000 7feb10ff9700 1 --2- 172.21.15.144:0/1799510649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feafc06e200 0x7feafc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:53:42.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.015+0000 7feb10ff9700 1 -- 172.21.15.144:0/1799510649 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7feb0408df70 con 0x7feb14103450 2024-09-10T11:53:42.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.015+0000 7feb12ffd700 1 --2- 172.21.15.144:0/1799510649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feafc06e200 0x7feafc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:53:42.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.015+0000 7feb12ffd700 1 --2- 172.21.15.144:0/1799510649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feafc06e200 0x7feafc0706b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7feb08000c00 tx=0x7feb08005de0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:53:42.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.019+0000 7feb10ff9700 1 -- 172.21.15.144:0/1799510649 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7feb040c31b0 con 0x7feb14103450 2024-09-10T11:53:42.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:41 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3765637314' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:53:42.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:41 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/43750676' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:53:42.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.316+0000 7feb19e1b700 1 -- 172.21.15.144:0/1799510649 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7feb14196610 con 0x7feb14103450 2024-09-10T11:53:42.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.316+0000 7feb10ff9700 1 -- 172.21.15.144:0/1799510649 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7feb0405ae80 con 0x7feb14103450 2024-09-10T11:53:42.318 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:53:42.318 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T11:53:42.318 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:53:42.318 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T11:53:42.318 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:53:42.318 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T11:53:42.318 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T11:53:42.318 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T11:53:42.318 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T11:53:42.318 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T11:53:42.318 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T11:53:42.318 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T11:53:42.319 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T11:53:42.319 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T11:53:42.319 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T11:53:42.319 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T11:53:42.319 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T11:53:42.319 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T11:53:42.319 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:53:42.319 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:53:42.319 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:53:42.319 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:53:42.319 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:53:42.319 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:53:42.319 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:53:42.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.318+0000 7feb19e1b700 1 -- 172.21.15.144:0/1799510649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feafc06e200 msgr2=0x7feafc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:42.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.318+0000 7feb19e1b700 1 --2- 172.21.15.144:0/1799510649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feafc06e200 0x7feafc0706b0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7feb08000c00 tx=0x7feb08005de0 comp rx=0 tx=0).stop 2024-09-10T11:53:42.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.318+0000 7feb19e1b700 1 -- 172.21.15.144:0/1799510649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb14103450 msgr2=0x7feb1419bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:53:42.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.318+0000 7feb19e1b700 1 --2- 172.21.15.144:0/1799510649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb14103450 0x7feb1419bab0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7feb0400d8d0 tx=0x7feb0400dbe0 comp rx=0 tx=0).stop 2024-09-10T11:53:42.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.318+0000 7feb19e1b700 1 -- 172.21.15.144:0/1799510649 shutdown_connections 2024-09-10T11:53:42.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.318+0000 7feb19e1b700 1 --2- 172.21.15.144:0/1799510649 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feafc06e200 0x7feafc0706b0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:42.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.318+0000 7feb19e1b700 1 --2- 172.21.15.144:0/1799510649 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb14103450 0x7feb1419bab0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:42.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.318+0000 7feb19e1b700 1 --2- 172.21.15.144:0/1799510649 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb14104790 0x7feb1419bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:53:42.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.318+0000 7feb19e1b700 1 -- 172.21.15.144:0/1799510649 >> 172.21.15.144:0/1799510649 conn(0x7feb14074850 msgr2=0x7feb14107ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:53:42.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.318+0000 7feb19e1b700 1 -- 172.21.15.144:0/1799510649 shutdown_connections 2024-09-10T11:53:42.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:53:42.318+0000 7feb19e1b700 1 -- 172.21.15.144:0/1799510649 wait complete. 2024-09-10T11:53:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:41 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3765637314' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:53:42.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:41 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/43750676' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:53:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:42 smithi178 ceph-mon[65940]: from='client.35232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:42 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1799510649' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:53:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:42 smithi178 ceph-mon[65940]: pgmap v653: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:43.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:42 smithi144 ceph-mon[87436]: from='client.35232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:53:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:42 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1799510649' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:53:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:42 smithi144 ceph-mon[87436]: pgmap v653: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:44.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:53:44.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:53:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:44 smithi178 ceph-mon[65940]: pgmap v654: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:45.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:44 smithi144 ceph-mon[87436]: pgmap v654: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:46 smithi178 ceph-mon[65940]: pgmap v655: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:46 smithi144 ceph-mon[87436]: pgmap v655: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:48 smithi178 ceph-mon[65940]: pgmap v656: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:49.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:48 smithi144 ceph-mon[87436]: pgmap v656: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:50 smithi178 ceph-mon[65940]: pgmap v657: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:51.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:50 smithi144 ceph-mon[87436]: pgmap v657: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:52 smithi178 ceph-mon[65940]: pgmap v658: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:53.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:52 smithi144 ceph-mon[87436]: pgmap v658: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:55.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:54 smithi144 ceph-mon[87436]: pgmap v659: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:54 smithi178 ceph-mon[65940]: pgmap v659: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:56 smithi178 ceph-mon[65940]: pgmap v660: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:57.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:56 smithi144 ceph-mon[87436]: pgmap v660: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:53:58.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:53:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:53:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:53:59 smithi144 ceph-mon[87436]: pgmap v661: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:00.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:53:59 smithi178 ceph-mon[65940]: pgmap v661: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:00 smithi178 ceph-mon[65940]: pgmap v662: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:01.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:00 smithi144 ceph-mon[87436]: pgmap v662: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:02 smithi178 ceph-mon[65940]: pgmap v663: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:02 smithi144 ceph-mon[87436]: pgmap v663: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:04 smithi178 ceph-mon[65940]: pgmap v664: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:05.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:04 smithi144 ceph-mon[87436]: pgmap v664: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:07.306 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:06 smithi178 ceph-mon[65940]: pgmap v665: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:07.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:06 smithi144 ceph-mon[87436]: pgmap v665: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:08 smithi144 ceph-mon[87436]: pgmap v666: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:08 smithi178 ceph-mon[65940]: pgmap v666: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:10 smithi178 ceph-mon[65940]: pgmap v667: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:11.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:10 smithi144 ceph-mon[87436]: pgmap v667: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:12.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.434+0000 7f7a51d38700 1 -- 172.21.15.144:0/1653003541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c0ff2e0 msgr2=0x7f7a4c0ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:12.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.434+0000 7f7a51d38700 1 --2- 172.21.15.144:0/1653003541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c0ff2e0 0x7f7a4c0ff6b0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c009910 tx=0x7f7a3c009c20 comp rx=0 tx=0).stop 2024-09-10T11:54:12.436 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.434+0000 7f7a51d38700 1 -- 172.21.15.144:0/1653003541 shutdown_connections 2024-09-10T11:54:12.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.434+0000 7f7a51d38700 1 --2- 172.21.15.144:0/1653003541 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c0ffc80 0x7f7a4c103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:12.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.434+0000 7f7a51d38700 1 --2- 172.21.15.144:0/1653003541 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c0ff2e0 0x7f7a4c0ff6b0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:12.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.434+0000 7f7a51d38700 1 -- 172.21.15.144:0/1653003541 >> 172.21.15.144:0/1653003541 conn(0x7f7a4c0fab80 msgr2=0x7f7a4c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:12.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.435+0000 7f7a51d38700 1 -- 172.21.15.144:0/1653003541 shutdown_connections 2024-09-10T11:54:12.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.435+0000 7f7a51d38700 1 -- 172.21.15.144:0/1653003541 wait complete. 2024-09-10T11:54:12.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.435+0000 7f7a51d38700 1 Processor -- start 2024-09-10T11:54:12.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.435+0000 7f7a51d38700 1 -- start start 2024-09-10T11:54:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a51d38700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c0ff2e0 0x7f7a4c100cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a51d38700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c0ffc80 0x7f7a4c101230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a51d38700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c101770 con 0x7f7a4c0ffc80 2024-09-10T11:54:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a51d38700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c1018b0 con 0x7f7a4c0ff2e0 2024-09-10T11:54:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a4affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c0ffc80 0x7f7a4c101230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a4affd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c0ffc80 0x7f7a4c101230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42842/0 (socket says 172.21.15.144:42842) 2024-09-10T11:54:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a4affd700 1 -- 172.21.15.144:0/1306445971 learned_addr learned my addr 172.21.15.144:0/1306445971 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:54:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a4b7fe700 1 --2- 172.21.15.144:0/1306445971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c0ff2e0 0x7f7a4c100cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a4affd700 1 -- 172.21.15.144:0/1306445971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c0ff2e0 msgr2=0x7f7a4c100cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a4affd700 1 --2- 172.21.15.144:0/1306445971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c0ff2e0 0x7f7a4c100cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a4affd700 1 -- 172.21.15.144:0/1306445971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c0095f0 con 0x7f7a4c0ffc80 2024-09-10T11:54:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a4affd700 1 --2- 172.21.15.144:0/1306445971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c0ffc80 0x7f7a4c101230 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000b480 tx=0x7f7a4000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:12.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a48ff9700 1 -- 172.21.15.144:0/1306445971 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a40011800 con 0x7f7a4c0ffc80 2024-09-10T11:54:12.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a48ff9700 1 -- 172.21.15.144:0/1306445971 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a40011e20 con 0x7f7a4c0ffc80 2024-09-10T11:54:12.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a51d38700 1 -- 172.21.15.144:0/1306445971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4c1a2020 con 0x7f7a4c0ffc80 2024-09-10T11:54:12.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a48ff9700 1 -- 172.21.15.144:0/1306445971 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a40010ea0 con 0x7f7a4c0ffc80 2024-09-10T11:54:12.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.436+0000 7f7a51d38700 1 -- 172.21.15.144:0/1306445971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4c1a2380 con 0x7f7a4c0ffc80 2024-09-10T11:54:12.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.437+0000 7f7a51d38700 1 -- 172.21.15.144:0/1306445971 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4c04ea50 con 0x7f7a4c0ffc80 2024-09-10T11:54:12.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.438+0000 7f7a48ff9700 1 -- 172.21.15.144:0/1306445971 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7a40010380 con 0x7f7a4c0ffc80 2024-09-10T11:54:12.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.438+0000 7f7a48ff9700 1 --2- 172.21.15.144:0/1306445971 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7a3406e200 0x7f7a340706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:12.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.438+0000 7f7a48ff9700 1 -- 172.21.15.144:0/1306445971 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7a4008db90 con 0x7f7a4c0ffc80 2024-09-10T11:54:12.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.438+0000 7f7a4b7fe700 1 --2- 172.21.15.144:0/1306445971 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7a3406e200 0x7f7a340706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:12.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.439+0000 7f7a4b7fe700 1 --2- 172.21.15.144:0/1306445971 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7a3406e200 0x7f7a340706b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c006010 tx=0x7f7a3c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:12.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.443+0000 7f7a48ff9700 1 -- 172.21.15.144:0/1306445971 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7a40056f70 con 0x7f7a4c0ffc80 2024-09-10T11:54:12.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.667+0000 7f7a51d38700 1 -- 172.21.15.144:0/1306445971 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a4c101a70 con 0x7f7a3406e200 2024-09-10T11:54:12.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.668+0000 7f7a48ff9700 1 -- 172.21.15.144:0/1306445971 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7a4c101a70 con 0x7f7a3406e200 2024-09-10T11:54:12.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.671+0000 7f7a51d38700 1 -- 172.21.15.144:0/1306445971 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7a3406e200 msgr2=0x7f7a340706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:12.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.671+0000 7f7a51d38700 1 --2- 172.21.15.144:0/1306445971 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7a3406e200 0x7f7a340706b0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f7a3c006010 tx=0x7f7a3c01a040 comp rx=0 tx=0).stop 2024-09-10T11:54:12.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.671+0000 7f7a51d38700 1 -- 172.21.15.144:0/1306445971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c0ffc80 msgr2=0x7f7a4c101230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:12.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.671+0000 7f7a51d38700 1 --2- 172.21.15.144:0/1306445971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c0ffc80 0x7f7a4c101230 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7a4000b480 tx=0x7f7a4000b790 comp rx=0 tx=0).stop 2024-09-10T11:54:12.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.671+0000 7f7a51d38700 1 -- 172.21.15.144:0/1306445971 shutdown_connections 2024-09-10T11:54:12.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.671+0000 7f7a51d38700 1 --2- 172.21.15.144:0/1306445971 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7a3406e200 0x7f7a340706b0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:12.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.671+0000 7f7a51d38700 1 --2- 172.21.15.144:0/1306445971 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a4c0ff2e0 0x7f7a4c100cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:12.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.671+0000 7f7a51d38700 1 --2- 172.21.15.144:0/1306445971 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a4c0ffc80 0x7f7a4c101230 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:12.672 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.671+0000 7f7a51d38700 1 -- 172.21.15.144:0/1306445971 >> 172.21.15.144:0/1306445971 conn(0x7f7a4c0fab80 msgr2=0x7f7a4c103440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:12.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.671+0000 7f7a51d38700 1 -- 172.21.15.144:0/1306445971 shutdown_connections 2024-09-10T11:54:12.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.671+0000 7f7a51d38700 1 -- 172.21.15.144:0/1306445971 wait complete. 2024-09-10T11:54:12.682 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:54:12.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.776+0000 7f36b57b8700 1 -- 172.21.15.144:0/1503056609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b0100430 msgr2=0x7f36b0100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:12.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.776+0000 7f36b57b8700 1 --2- 172.21.15.144:0/1503056609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b0100430 0x7f36b0100800 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f36a8009910 tx=0x7f36a8009c20 comp rx=0 tx=0).stop 2024-09-10T11:54:12.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.776+0000 7f36b57b8700 1 -- 172.21.15.144:0/1503056609 shutdown_connections 2024-09-10T11:54:12.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.776+0000 7f36b57b8700 1 --2- 172.21.15.144:0/1503056609 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36b0100dd0 0x7f36b0104da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:12.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.776+0000 7f36b57b8700 1 --2- 172.21.15.144:0/1503056609 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b0100430 0x7f36b0100800 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:12.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.776+0000 7f36b57b8700 1 -- 172.21.15.144:0/1503056609 >> 172.21.15.144:0/1503056609 conn(0x7f36b00fbcf0 msgr2=0x7f36b00fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:12.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.776+0000 7f36b57b8700 1 -- 172.21.15.144:0/1503056609 shutdown_connections 2024-09-10T11:54:12.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.776+0000 7f36b57b8700 1 -- 172.21.15.144:0/1503056609 wait complete. 2024-09-10T11:54:12.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.777+0000 7f36b57b8700 1 Processor -- start 2024-09-10T11:54:12.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.777+0000 7f36b57b8700 1 -- start start 2024-09-10T11:54:12.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.777+0000 7f36b57b8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b0100430 0x7f36b0195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:12.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.777+0000 7f36b57b8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36b0100dd0 0x7f36b01962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:12.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.777+0000 7f36b57b8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b0196980 con 0x7f36b0100430 2024-09-10T11:54:12.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.777+0000 7f36b57b8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b019a710 con 0x7f36b0100dd0 2024-09-10T11:54:12.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.777+0000 7f36aeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b0100430 0x7f36b0195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:12.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.777+0000 7f36aeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b0100430 0x7f36b0195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42872/0 (socket says 172.21.15.144:42872) 2024-09-10T11:54:12.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.777+0000 7f36aeffd700 1 -- 172.21.15.144:0/3837955185 learned_addr learned my addr 172.21.15.144:0/3837955185 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:54:12.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.778+0000 7f36ae7fc700 1 --2- 172.21.15.144:0/3837955185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36b0100dd0 0x7f36b01962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:12.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.778+0000 7f36aeffd700 1 -- 172.21.15.144:0/3837955185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36b0100dd0 msgr2=0x7f36b01962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:12.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.778+0000 7f36aeffd700 1 --2- 172.21.15.144:0/3837955185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36b0100dd0 0x7f36b01962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:12.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.778+0000 7f36aeffd700 1 -- 172.21.15.144:0/3837955185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a80095f0 con 0x7f36b0100430 2024-09-10T11:54:12.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.778+0000 7f36aeffd700 1 --2- 172.21.15.144:0/3837955185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b0100430 0x7f36b0195d60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f36a8006010 tx=0x7f36a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:12.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.778+0000 7f36a7fff700 1 -- 172.21.15.144:0/3837955185 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a801d070 con 0x7f36b0100430 2024-09-10T11:54:12.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.778+0000 7f36b57b8700 1 -- 172.21.15.144:0/3837955185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b019a990 con 0x7f36b0100430 2024-09-10T11:54:12.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.778+0000 7f36a7fff700 1 -- 172.21.15.144:0/3837955185 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36a800f460 con 0x7f36b0100430 2024-09-10T11:54:12.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.778+0000 7f36a7fff700 1 -- 172.21.15.144:0/3837955185 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a8017630 con 0x7f36b0100430 2024-09-10T11:54:12.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.778+0000 7f36b57b8700 1 -- 172.21.15.144:0/3837955185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b019ae80 con 0x7f36b0100430 2024-09-10T11:54:12.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.779+0000 7f36b57b8700 1 -- 172.21.15.144:0/3837955185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b0108740 con 0x7f36b0100430 2024-09-10T11:54:12.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.780+0000 7f36a7fff700 1 -- 172.21.15.144:0/3837955185 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f36a800f5d0 con 0x7f36b0100430 2024-09-10T11:54:12.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.780+0000 7f36a7fff700 1 --2- 172.21.15.144:0/3837955185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f369c06e200 0x7f369c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:12.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.780+0000 7f36a7fff700 1 -- 172.21.15.144:0/3837955185 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f36a808f540 con 0x7f36b0100430 2024-09-10T11:54:12.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.780+0000 7f36ae7fc700 1 --2- 172.21.15.144:0/3837955185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f369c06e200 0x7f369c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:12.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.781+0000 7f36ae7fc700 1 --2- 172.21.15.144:0/3837955185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f369c06e200 0x7f369c0706b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009550 tx=0x7f36a0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:12.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:12.785+0000 7f36a7fff700 1 -- 172.21.15.144:0/3837955185 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f36a8058920 con 0x7f36b0100430 2024-09-10T11:54:13.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.005+0000 7f36b57b8700 1 -- 172.21.15.144:0/3837955185 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36b019b220 con 0x7f369c06e200 2024-09-10T11:54:13.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.006+0000 7f36a7fff700 1 -- 172.21.15.144:0/3837955185 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f36b019b220 con 0x7f369c06e200 2024-09-10T11:54:13.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.008+0000 7f36b57b8700 1 -- 172.21.15.144:0/3837955185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f369c06e200 msgr2=0x7f369c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:13.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.008+0000 7f36b57b8700 1 --2- 172.21.15.144:0/3837955185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f369c06e200 0x7f369c0706b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009550 tx=0x7f36a0009040 comp rx=0 tx=0).stop 2024-09-10T11:54:13.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.008+0000 7f36b57b8700 1 -- 172.21.15.144:0/3837955185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b0100430 msgr2=0x7f36b0195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:13.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.008+0000 7f36b57b8700 1 --2- 172.21.15.144:0/3837955185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b0100430 0x7f36b0195d60 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f36a8006010 tx=0x7f36a800be20 comp rx=0 tx=0).stop 2024-09-10T11:54:13.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.008+0000 7f36b57b8700 1 -- 172.21.15.144:0/3837955185 shutdown_connections 2024-09-10T11:54:13.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.008+0000 7f36b57b8700 1 --2- 172.21.15.144:0/3837955185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f369c06e200 0x7f369c0706b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.008+0000 7f36b57b8700 1 --2- 172.21.15.144:0/3837955185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36b0100430 0x7f36b0195d60 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.008+0000 7f36b57b8700 1 --2- 172.21.15.144:0/3837955185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36b0100dd0 0x7f36b01962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.009+0000 7f36b57b8700 1 -- 172.21.15.144:0/3837955185 >> 172.21.15.144:0/3837955185 conn(0x7f36b00fbcf0 msgr2=0x7f36b00fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:13.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.009+0000 7f36b57b8700 1 -- 172.21.15.144:0/3837955185 shutdown_connections 2024-09-10T11:54:13.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.009+0000 7f36b57b8700 1 -- 172.21.15.144:0/3837955185 wait complete. 2024-09-10T11:54:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:12 smithi144 ceph-mon[87436]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:12 smithi144 ceph-mon[87436]: pgmap v668: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:13.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.113+0000 7f715c3be700 1 -- 172.21.15.144:0/2745144267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7154103860 msgr2=0x7f7154103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:13.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.113+0000 7f715c3be700 1 --2- 172.21.15.144:0/2745144267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7154103860 0x7f7154103c30 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f7144009910 tx=0x7f7144009c20 comp rx=0 tx=0).stop 2024-09-10T11:54:13.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.113+0000 7f715c3be700 1 -- 172.21.15.144:0/2745144267 shutdown_connections 2024-09-10T11:54:13.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.113+0000 7f715c3be700 1 --2- 172.21.15.144:0/2745144267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71540ff320 0x7f71540ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.113+0000 7f715c3be700 1 --2- 172.21.15.144:0/2745144267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7154103860 0x7f7154103c30 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.113+0000 7f715c3be700 1 -- 172.21.15.144:0/2745144267 >> 172.21.15.144:0/2745144267 conn(0x7f71540748b0 msgr2=0x7f7154074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:13.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.114+0000 7f715c3be700 1 -- 172.21.15.144:0/2745144267 shutdown_connections 2024-09-10T11:54:13.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.114+0000 7f715c3be700 1 -- 172.21.15.144:0/2745144267 wait complete. 2024-09-10T11:54:13.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.114+0000 7f715c3be700 1 Processor -- start 2024-09-10T11:54:13.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.114+0000 7f715c3be700 1 -- start start 2024-09-10T11:54:13.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.114+0000 7f715c3be700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71540ff320 0x7f7154195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:13.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.114+0000 7f715c3be700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7154103860 0x7f7154196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:13.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f715c3be700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7154196930 con 0x7f7154103860 2024-09-10T11:54:13.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f715c3be700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715419a6c0 con 0x7f71540ff320 2024-09-10T11:54:13.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f7159959700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7154103860 0x7f7154196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:13.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f7159959700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7154103860 0x7f7154196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42900/0 (socket says 172.21.15.144:42900) 2024-09-10T11:54:13.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f7159959700 1 -- 172.21.15.144:0/105540382 learned_addr learned my addr 172.21.15.144:0/105540382 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:54:13.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f715a15a700 1 --2- 172.21.15.144:0/105540382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71540ff320 0x7f7154195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:13.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f7159959700 1 -- 172.21.15.144:0/105540382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71540ff320 msgr2=0x7f7154195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:13.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f7159959700 1 --2- 172.21.15.144:0/105540382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71540ff320 0x7f7154195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f7159959700 1 -- 172.21.15.144:0/105540382 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71440095f0 con 0x7f7154103860 2024-09-10T11:54:13.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f715a15a700 1 --2- 172.21.15.144:0/105540382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71540ff320 0x7f7154195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:54:13.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f7159959700 1 --2- 172.21.15.144:0/105540382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7154103860 0x7f7154196250 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f715000dd40 tx=0x7f715000a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:13.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f714b7fe700 1 -- 172.21.15.144:0/105540382 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f715000ad10 con 0x7f7154103860 2024-09-10T11:54:13.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f715c3be700 1 -- 172.21.15.144:0/105540382 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715419a9a0 con 0x7f7154103860 2024-09-10T11:54:13.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f714b7fe700 1 -- 172.21.15.144:0/105540382 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f715000ae70 con 0x7f7154103860 2024-09-10T11:54:13.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f715c3be700 1 -- 172.21.15.144:0/105540382 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715419af70 con 0x7f7154103860 2024-09-10T11:54:13.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.115+0000 7f714b7fe700 1 -- 172.21.15.144:0/105540382 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7150016960 con 0x7f7154103860 2024-09-10T11:54:13.119 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.117+0000 7f715c3be700 1 -- 172.21.15.144:0/105540382 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7154106e00 con 0x7f7154103860 2024-09-10T11:54:13.119 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.117+0000 7f714b7fe700 1 -- 172.21.15.144:0/105540382 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7150016b50 con 0x7f7154103860 2024-09-10T11:54:13.119 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.117+0000 7f714b7fe700 1 --2- 172.21.15.144:0/105540382 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f714006e270 0x7f7140070720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:13.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.118+0000 7f714b7fe700 1 -- 172.21.15.144:0/105540382 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f715008e080 con 0x7f7154103860 2024-09-10T11:54:13.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.118+0000 7f715a15a700 1 --2- 172.21.15.144:0/105540382 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f714006e270 0x7f7140070720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:13.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.118+0000 7f715a15a700 1 --2- 172.21.15.144:0/105540382 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f714006e270 0x7f7140070720 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7144005b40 tx=0x7f714401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:13.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.122+0000 7f714b7fe700 1 -- 172.21.15.144:0/105540382 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7150057510 con 0x7f7154103860 2024-09-10T11:54:13.333 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.331+0000 7f715c3be700 1 -- 172.21.15.144:0/105540382 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f715419b250 con 0x7f714006e270 2024-09-10T11:54:13.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.341+0000 7f714b7fe700 1 -- 172.21.15.144:0/105540382 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f715419b250 con 0x7f714006e270 2024-09-10T11:54:13.342 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:54:13.342 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (22m) 45s ago 32m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:54:13.342 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (33m) 45s ago 33m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:54:13.342 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (32m) 47s ago 32m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:54:13.342 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (20m) 45s ago 33m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:54:13.342 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (20m) 47s ago 32m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:54:13.342 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (22m) 45s ago 32m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:54:13.343 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (16m) 45s ago 27m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:54:13.343 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (16m) 45s ago 27m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:54:13.343 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (51s) 47s ago 27m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T11:54:13.343 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (15m) 47s ago 27m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:54:13.343 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (24m) 45s ago 34m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:54:13.343 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (23m) 47s ago 32m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:54:13.343 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (21m) 45s ago 34m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:54:13.343 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (20m) 47s ago 31m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:54:13.343 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (23m) 45s ago 33m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:54:13.343 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (23m) 47s ago 32m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:54:13.343 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (19m) 45s ago 31m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:54:13.343 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (19m) 45s ago 30m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:54:13.343 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (18m) 45s ago 30m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:54:13.343 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (17m) 47s ago 29m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:54:13.344 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (17m) 47s ago 29m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:54:13.344 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (16m) 47s ago 29m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:54:13.344 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (22m) 45s ago 32m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:54:13.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.343+0000 7f715c3be700 1 -- 172.21.15.144:0/105540382 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f714006e270 msgr2=0x7f7140070720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:13.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.343+0000 7f715c3be700 1 --2- 172.21.15.144:0/105540382 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f714006e270 0x7f7140070720 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f7144005b40 tx=0x7f714401a040 comp rx=0 tx=0).stop 2024-09-10T11:54:13.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.343+0000 7f715c3be700 1 -- 172.21.15.144:0/105540382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7154103860 msgr2=0x7f7154196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:13.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.343+0000 7f715c3be700 1 --2- 172.21.15.144:0/105540382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7154103860 0x7f7154196250 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f715000dd40 tx=0x7f715000a530 comp rx=0 tx=0).stop 2024-09-10T11:54:13.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.343+0000 7f715c3be700 1 -- 172.21.15.144:0/105540382 shutdown_connections 2024-09-10T11:54:13.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.343+0000 7f715c3be700 1 --2- 172.21.15.144:0/105540382 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f714006e270 0x7f7140070720 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.343+0000 7f715c3be700 1 --2- 172.21.15.144:0/105540382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f71540ff320 0x7f7154195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.343+0000 7f715c3be700 1 --2- 172.21.15.144:0/105540382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7154103860 0x7f7154196250 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.343+0000 7f715c3be700 1 -- 172.21.15.144:0/105540382 >> 172.21.15.144:0/105540382 conn(0x7f71540748b0 msgr2=0x7f71540fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:13.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.343+0000 7f715c3be700 1 -- 172.21.15.144:0/105540382 shutdown_connections 2024-09-10T11:54:13.345 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.343+0000 7f715c3be700 1 -- 172.21.15.144:0/105540382 wait complete. 2024-09-10T11:54:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:12 smithi178 ceph-mon[65940]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:12 smithi178 ceph-mon[65940]: pgmap v668: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:13.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.446+0000 7f97591aa700 1 -- 172.21.15.144:0/624023422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754101380 msgr2=0x7f9754101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:13.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.446+0000 7f97591aa700 1 --2- 172.21.15.144:0/624023422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754101380 0x7f9754101750 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f9744009910 tx=0x7f9744009c20 comp rx=0 tx=0).stop 2024-09-10T11:54:13.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.447+0000 7f97591aa700 1 -- 172.21.15.144:0/624023422 shutdown_connections 2024-09-10T11:54:13.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.447+0000 7f97591aa700 1 --2- 172.21.15.144:0/624023422 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9754101c90 0x7f975410a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.447+0000 7f97591aa700 1 --2- 172.21.15.144:0/624023422 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754101380 0x7f9754101750 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.447+0000 7f97591aa700 1 -- 172.21.15.144:0/624023422 >> 172.21.15.144:0/624023422 conn(0x7f97540fab80 msgr2=0x7f97540fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:13.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.447+0000 7f97591aa700 1 -- 172.21.15.144:0/624023422 shutdown_connections 2024-09-10T11:54:13.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.447+0000 7f97591aa700 1 -- 172.21.15.144:0/624023422 wait complete. 2024-09-10T11:54:13.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.447+0000 7f97591aa700 1 Processor -- start 2024-09-10T11:54:13.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.448+0000 7f97591aa700 1 -- start start 2024-09-10T11:54:13.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.448+0000 7f97591aa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754101380 0x7f9754195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:13.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.448+0000 7f97591aa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9754101c90 0x7f97541962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:13.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.448+0000 7f97591aa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9754196900 con 0x7f9754101380 2024-09-10T11:54:13.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.448+0000 7f97591aa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9754196a40 con 0x7f9754101c90 2024-09-10T11:54:13.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.448+0000 7f9752d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754101380 0x7f9754195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:13.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.448+0000 7f9752d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754101380 0x7f9754195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42926/0 (socket says 172.21.15.144:42926) 2024-09-10T11:54:13.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.448+0000 7f9752d9d700 1 -- 172.21.15.144:0/4199602007 learned_addr learned my addr 172.21.15.144:0/4199602007 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:54:13.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.448+0000 7f975259c700 1 --2- 172.21.15.144:0/4199602007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9754101c90 0x7f97541962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:13.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.448+0000 7f9752d9d700 1 -- 172.21.15.144:0/4199602007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9754101c90 msgr2=0x7f97541962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:13.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.448+0000 7f9752d9d700 1 --2- 172.21.15.144:0/4199602007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9754101c90 0x7f97541962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.448+0000 7f9752d9d700 1 -- 172.21.15.144:0/4199602007 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97440095f0 con 0x7f9754101380 2024-09-10T11:54:13.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.448+0000 7f975259c700 1 --2- 172.21.15.144:0/4199602007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9754101c90 0x7f97541962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:54:13.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.448+0000 7f9752d9d700 1 --2- 172.21.15.144:0/4199602007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754101380 0x7f9754195d70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9744006010 tx=0x7f974400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:13.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.449+0000 7f973bfff700 1 -- 172.21.15.144:0/4199602007 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f974401d070 con 0x7f9754101380 2024-09-10T11:54:13.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.449+0000 7f973bfff700 1 -- 172.21.15.144:0/4199602007 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9744004fb0 con 0x7f9754101380 2024-09-10T11:54:13.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.449+0000 7f973bfff700 1 -- 172.21.15.144:0/4199602007 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9744022ea0 con 0x7f9754101380 2024-09-10T11:54:13.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.449+0000 7f97591aa700 1 -- 172.21.15.144:0/4199602007 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975419a7e0 con 0x7f9754101380 2024-09-10T11:54:13.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.449+0000 7f97591aa700 1 -- 172.21.15.144:0/4199602007 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975419acd0 con 0x7f9754101380 2024-09-10T11:54:13.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.450+0000 7f97591aa700 1 -- 172.21.15.144:0/4199602007 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9754107860 con 0x7f9754101380 2024-09-10T11:54:13.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.450+0000 7f973bfff700 1 -- 172.21.15.144:0/4199602007 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9744022470 con 0x7f9754101380 2024-09-10T11:54:13.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.451+0000 7f973bfff700 1 --2- 172.21.15.144:0/4199602007 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f973c06e200 0x7f973c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:13.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.451+0000 7f975259c700 1 --2- 172.21.15.144:0/4199602007 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f973c06e200 0x7f973c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:13.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.451+0000 7f973bfff700 1 -- 172.21.15.144:0/4199602007 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f974408f540 con 0x7f9754101380 2024-09-10T11:54:13.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.451+0000 7f975259c700 1 --2- 172.21.15.144:0/4199602007 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f973c06e200 0x7f973c0706b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9748005fd0 tx=0x7f9748005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:13.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.458+0000 7f973bfff700 1 -- 172.21.15.144:0/4199602007 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9744058a50 con 0x7f9754101380 2024-09-10T11:54:13.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.747+0000 7f97591aa700 1 -- 172.21.15.144:0/4199602007 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f975402ccd0 con 0x7f9754101380 2024-09-10T11:54:13.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.748+0000 7f973bfff700 1 -- 172.21.15.144:0/4199602007 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f9744022720 con 0x7f9754101380 2024-09-10T11:54:13.749 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:54:13.749 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:54:13.749 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:54:13.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:54:13.750 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:54:13.750 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:54:13.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:54:13.750 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:54:13.750 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:54:13.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:54:13.750 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:54:13.750 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:54:13.750 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:54:13.750 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:54:13.750 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T11:54:13.750 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:54:13.750 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:54:13.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.750+0000 7f97591aa700 1 -- 172.21.15.144:0/4199602007 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f973c06e200 msgr2=0x7f973c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:13.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.750+0000 7f97591aa700 1 --2- 172.21.15.144:0/4199602007 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f973c06e200 0x7f973c0706b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9748005fd0 tx=0x7f9748005e20 comp rx=0 tx=0).stop 2024-09-10T11:54:13.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.750+0000 7f97591aa700 1 -- 172.21.15.144:0/4199602007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754101380 msgr2=0x7f9754195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:13.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.750+0000 7f97591aa700 1 --2- 172.21.15.144:0/4199602007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754101380 0x7f9754195d70 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9744006010 tx=0x7f974400be20 comp rx=0 tx=0).stop 2024-09-10T11:54:13.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.750+0000 7f97591aa700 1 -- 172.21.15.144:0/4199602007 shutdown_connections 2024-09-10T11:54:13.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.750+0000 7f97591aa700 1 --2- 172.21.15.144:0/4199602007 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f973c06e200 0x7f973c0706b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.750+0000 7f97591aa700 1 --2- 172.21.15.144:0/4199602007 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754101380 0x7f9754195d70 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.750+0000 7f97591aa700 1 --2- 172.21.15.144:0/4199602007 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9754101c90 0x7f97541962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.750+0000 7f97591aa700 1 -- 172.21.15.144:0/4199602007 >> 172.21.15.144:0/4199602007 conn(0x7f97540fab80 msgr2=0x7f97541049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:13.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.750+0000 7f97591aa700 1 -- 172.21.15.144:0/4199602007 shutdown_connections 2024-09-10T11:54:13.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.750+0000 7f97591aa700 1 -- 172.21.15.144:0/4199602007 wait complete. 2024-09-10T11:54:13.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.855+0000 7f3ab2f7d700 1 -- 172.21.15.144:0/3420313275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aac104790 msgr2=0x7f3aac104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:13.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.855+0000 7f3ab2f7d700 1 --2- 172.21.15.144:0/3420313275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aac104790 0x7f3aac104b60 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0009910 tx=0x7f3aa0009c20 comp rx=0 tx=0).stop 2024-09-10T11:54:13.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.855+0000 7f3ab2f7d700 1 -- 172.21.15.144:0/3420313275 shutdown_connections 2024-09-10T11:54:13.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.855+0000 7f3ab2f7d700 1 --2- 172.21.15.144:0/3420313275 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aac103450 0x7f3aac1038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.855+0000 7f3ab2f7d700 1 --2- 172.21.15.144:0/3420313275 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aac104790 0x7f3aac104b60 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.855+0000 7f3ab2f7d700 1 -- 172.21.15.144:0/3420313275 >> 172.21.15.144:0/3420313275 conn(0x7f3aac074850 msgr2=0x7f3aac074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:13.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.855+0000 7f3ab2f7d700 1 -- 172.21.15.144:0/3420313275 shutdown_connections 2024-09-10T11:54:13.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.855+0000 7f3ab2f7d700 1 -- 172.21.15.144:0/3420313275 wait complete. 2024-09-10T11:54:13.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.856+0000 7f3ab2f7d700 1 Processor -- start 2024-09-10T11:54:13.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.856+0000 7f3ab2f7d700 1 -- start start 2024-09-10T11:54:13.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.856+0000 7f3ab2f7d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aac103450 0x7f3aac197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:13.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.856+0000 7f3ab2f7d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aac104790 0x7f3aac198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:13.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.856+0000 7f3ab2f7d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac198b60 con 0x7f3aac104790 2024-09-10T11:54:13.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.856+0000 7f3ab2f7d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aac19c8f0 con 0x7f3aac103450 2024-09-10T11:54:13.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.856+0000 7f3aabfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aac104790 0x7f3aac198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:13.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.856+0000 7f3aabfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aac104790 0x7f3aac198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42944/0 (socket says 172.21.15.144:42944) 2024-09-10T11:54:13.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.856+0000 7f3aabfff700 1 -- 172.21.15.144:0/1319436444 learned_addr learned my addr 172.21.15.144:0/1319436444 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:54:13.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.857+0000 7f3ab0d19700 1 --2- 172.21.15.144:0/1319436444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aac103450 0x7f3aac197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:13.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.857+0000 7f3aabfff700 1 -- 172.21.15.144:0/1319436444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aac103450 msgr2=0x7f3aac197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:13.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.857+0000 7f3aabfff700 1 --2- 172.21.15.144:0/1319436444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aac103450 0x7f3aac197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:13.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.857+0000 7f3aabfff700 1 -- 172.21.15.144:0/1319436444 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa00095f0 con 0x7f3aac104790 2024-09-10T11:54:13.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.857+0000 7f3aabfff700 1 --2- 172.21.15.144:0/1319436444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aac104790 0x7f3aac198480 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800d8d0 tx=0x7f3a9800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:13.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.857+0000 7f3aa9ffb700 1 -- 172.21.15.144:0/1319436444 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a980098a0 con 0x7f3aac104790 2024-09-10T11:54:13.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.857+0000 7f3aa9ffb700 1 -- 172.21.15.144:0/1319436444 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a98010460 con 0x7f3aac104790 2024-09-10T11:54:13.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.857+0000 7f3aa9ffb700 1 -- 172.21.15.144:0/1319436444 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a9800f5f0 con 0x7f3aac104790 2024-09-10T11:54:13.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.857+0000 7f3ab2f7d700 1 -- 172.21.15.144:0/1319436444 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aac19cbd0 con 0x7f3aac104790 2024-09-10T11:54:13.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.857+0000 7f3ab2f7d700 1 -- 172.21.15.144:0/1319436444 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aac19d0d0 con 0x7f3aac104790 2024-09-10T11:54:13.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.858+0000 7f3ab2f7d700 1 -- 172.21.15.144:0/1319436444 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aac04ea50 con 0x7f3aac104790 2024-09-10T11:54:13.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.859+0000 7f3aa9ffb700 1 -- 172.21.15.144:0/1319436444 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3a98009a00 con 0x7f3aac104790 2024-09-10T11:54:13.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.859+0000 7f3aa9ffb700 1 --2- 172.21.15.144:0/1319436444 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3a9c06e200 0x7f3a9c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:13.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.859+0000 7f3aa9ffb700 1 -- 172.21.15.144:0/1319436444 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3a9808df70 con 0x7f3aac104790 2024-09-10T11:54:13.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.859+0000 7f3ab0d19700 1 --2- 172.21.15.144:0/1319436444 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3a9c06e200 0x7f3a9c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:13.862 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.860+0000 7f3ab0d19700 1 --2- 172.21.15.144:0/1319436444 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3a9c06e200 0x7f3a9c0706b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0006010 tx=0x7f3aa001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:13.865 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:13.864+0000 7f3aa9ffb700 1 -- 172.21.15.144:0/1319436444 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3a980c31b0 con 0x7f3aac104790 2024-09-10T11:54:14.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.113+0000 7f3ab2f7d700 1 -- 172.21.15.144:0/1319436444 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3aac19d4c0 con 0x7f3aac104790 2024-09-10T11:54:14.115 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:13 smithi144 ceph-mon[87436]: from='client.35244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:14.115 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:13 smithi144 ceph-mon[87436]: from='client.35248 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:14.115 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:54:14.115 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:13 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/4199602007' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:54:14.116 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.114+0000 7f3aa9ffb700 1 -- 172.21.15.144:0/1319436444 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f3a98057350 con 0x7f3aac104790 2024-09-10T11:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T11:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T11:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:54:14.116 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:54:14.117 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:54:14.118 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:54:14.119 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:54:14.119 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.117+0000 7f3ab2f7d700 1 -- 172.21.15.144:0/1319436444 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3a9c06e200 msgr2=0x7f3a9c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:14.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.117+0000 7f3ab2f7d700 1 --2- 172.21.15.144:0/1319436444 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3a9c06e200 0x7f3a9c0706b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3aa0006010 tx=0x7f3aa001a040 comp rx=0 tx=0).stop 2024-09-10T11:54:14.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.117+0000 7f3ab2f7d700 1 -- 172.21.15.144:0/1319436444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aac104790 msgr2=0x7f3aac198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:14.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.117+0000 7f3ab2f7d700 1 --2- 172.21.15.144:0/1319436444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aac104790 0x7f3aac198480 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800d8d0 tx=0x7f3a9800dbe0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.117+0000 7f3ab2f7d700 1 -- 172.21.15.144:0/1319436444 shutdown_connections 2024-09-10T11:54:14.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.117+0000 7f3ab2f7d700 1 --2- 172.21.15.144:0/1319436444 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3a9c06e200 0x7f3a9c0706b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.117+0000 7f3ab2f7d700 1 --2- 172.21.15.144:0/1319436444 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3aac103450 0x7f3aac197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.118+0000 7f3ab2f7d700 1 --2- 172.21.15.144:0/1319436444 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3aac104790 0x7f3aac198480 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.118+0000 7f3ab2f7d700 1 -- 172.21.15.144:0/1319436444 >> 172.21.15.144:0/1319436444 conn(0x7f3aac074850 msgr2=0x7f3aac102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:14.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.118+0000 7f3ab2f7d700 1 -- 172.21.15.144:0/1319436444 shutdown_connections 2024-09-10T11:54:14.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.118+0000 7f3ab2f7d700 1 -- 172.21.15.144:0/1319436444 wait complete. 2024-09-10T11:54:14.120 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T11:54:14.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.229+0000 7f2046806700 1 -- 172.21.15.144:0/1887390417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20400ff390 msgr2=0x7f20400ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:14.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.229+0000 7f2046806700 1 --2- 172.21.15.144:0/1887390417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20400ff390 0x7f20400ff760 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f2034009910 tx=0x7f2034009c20 comp rx=0 tx=0).stop 2024-09-10T11:54:14.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.229+0000 7f2046806700 1 -- 172.21.15.144:0/1887390417 shutdown_connections 2024-09-10T11:54:14.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.229+0000 7f2046806700 1 --2- 172.21.15.144:0/1887390417 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20400ffd30 0x7f2040103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.229+0000 7f2046806700 1 --2- 172.21.15.144:0/1887390417 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20400ff390 0x7f20400ff760 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.229+0000 7f2046806700 1 -- 172.21.15.144:0/1887390417 >> 172.21.15.144:0/1887390417 conn(0x7f2040074f50 msgr2=0x7f2040075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:14.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.229+0000 7f2046806700 1 -- 172.21.15.144:0/1887390417 shutdown_connections 2024-09-10T11:54:14.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.229+0000 7f2046806700 1 -- 172.21.15.144:0/1887390417 wait complete. 2024-09-10T11:54:14.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.230+0000 7f2046806700 1 Processor -- start 2024-09-10T11:54:14.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.230+0000 7f2046806700 1 -- start start 2024-09-10T11:54:14.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.230+0000 7f2046806700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20400ff390 0x7f2040194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:14.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.230+0000 7f2046806700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20400ffd30 0x7f2040195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:14.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.230+0000 7f2046806700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2040195960 con 0x7f20400ffd30 2024-09-10T11:54:14.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.230+0000 7f2046806700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20401996f0 con 0x7f20400ff390 2024-09-10T11:54:14.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.231+0000 7f203f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20400ffd30 0x7f2040195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:14.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.231+0000 7f203f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20400ffd30 0x7f2040195280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42952/0 (socket says 172.21.15.144:42952) 2024-09-10T11:54:14.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.231+0000 7f203f7fe700 1 -- 172.21.15.144:0/1685753963 learned_addr learned my addr 172.21.15.144:0/1685753963 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:54:14.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.231+0000 7f203ffff700 1 --2- 172.21.15.144:0/1685753963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20400ff390 0x7f2040194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:14.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.231+0000 7f203f7fe700 1 -- 172.21.15.144:0/1685753963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20400ff390 msgr2=0x7f2040194d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:14.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.231+0000 7f203f7fe700 1 --2- 172.21.15.144:0/1685753963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20400ff390 0x7f2040194d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.231+0000 7f203f7fe700 1 -- 172.21.15.144:0/1685753963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20340095f0 con 0x7f20400ffd30 2024-09-10T11:54:14.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.231+0000 7f203f7fe700 1 --2- 172.21.15.144:0/1685753963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20400ffd30 0x7f2040195280 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f203000d930 tx=0x7f203000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:14.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.231+0000 7f203d7fa700 1 -- 172.21.15.144:0/1685753963 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f203000ec20 con 0x7f20400ffd30 2024-09-10T11:54:14.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.231+0000 7f203d7fa700 1 -- 172.21.15.144:0/1685753963 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f203000ed80 con 0x7f20400ffd30 2024-09-10T11:54:14.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.231+0000 7f203d7fa700 1 -- 172.21.15.144:0/1685753963 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20300147a0 con 0x7f20400ffd30 2024-09-10T11:54:14.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.231+0000 7f2046806700 1 -- 172.21.15.144:0/1685753963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2040199970 con 0x7f20400ffd30 2024-09-10T11:54:14.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.231+0000 7f2046806700 1 -- 172.21.15.144:0/1685753963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2040199ec0 con 0x7f20400ffd30 2024-09-10T11:54:14.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.232+0000 7f2046806700 1 -- 172.21.15.144:0/1685753963 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f204004ea50 con 0x7f20400ffd30 2024-09-10T11:54:14.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.234+0000 7f203d7fa700 1 -- 172.21.15.144:0/1685753963 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2030014900 con 0x7f20400ffd30 2024-09-10T11:54:14.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.234+0000 7f203d7fa700 1 --2- 172.21.15.144:0/1685753963 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f202806e200 0x7f20280706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:14.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.234+0000 7f203d7fa700 1 -- 172.21.15.144:0/1685753963 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f203008f900 con 0x7f20400ffd30 2024-09-10T11:54:14.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.234+0000 7f203ffff700 1 --2- 172.21.15.144:0/1685753963 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f202806e200 0x7f20280706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:14.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.235+0000 7f203ffff700 1 --2- 172.21.15.144:0/1685753963 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f202806e200 0x7f20280706b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f203400b5c0 tx=0x7f203401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:14.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.239+0000 7f203d7fa700 1 -- 172.21.15.144:0/1685753963 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2030058e10 con 0x7f20400ffd30 2024-09-10T11:54:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:13 smithi178 ceph-mon[65940]: from='client.35244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:13 smithi178 ceph-mon[65940]: from='client.35248 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:54:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:13 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/4199602007' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:54:14.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.471+0000 7f2046806700 1 -- 172.21.15.144:0/1685753963 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f204019a500 con 0x7f202806e200 2024-09-10T11:54:14.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.472+0000 7f203d7fa700 1 -- 172.21.15.144:0/1685753963 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f204019a500 con 0x7f202806e200 2024-09-10T11:54:14.473 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:54:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:54:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:54:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:54:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:54:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:54:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T11:54:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T11:54:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:54:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T11:54:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:54:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T11:54:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:54:14.474 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:54:14.475 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:54:14.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.475+0000 7f2046806700 1 -- 172.21.15.144:0/1685753963 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f202806e200 msgr2=0x7f20280706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:14.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.475+0000 7f2046806700 1 --2- 172.21.15.144:0/1685753963 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f202806e200 0x7f20280706b0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f203400b5c0 tx=0x7f203401a040 comp rx=0 tx=0).stop 2024-09-10T11:54:14.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.475+0000 7f2046806700 1 -- 172.21.15.144:0/1685753963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20400ffd30 msgr2=0x7f2040195280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:14.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.475+0000 7f2046806700 1 --2- 172.21.15.144:0/1685753963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20400ffd30 0x7f2040195280 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f203000d930 tx=0x7f203000dcf0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.475+0000 7f2046806700 1 -- 172.21.15.144:0/1685753963 shutdown_connections 2024-09-10T11:54:14.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.475+0000 7f2046806700 1 --2- 172.21.15.144:0/1685753963 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f202806e200 0x7f20280706b0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.475+0000 7f2046806700 1 --2- 172.21.15.144:0/1685753963 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f20400ff390 0x7f2040194d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.475+0000 7f2046806700 1 --2- 172.21.15.144:0/1685753963 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f20400ffd30 0x7f2040195280 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.475+0000 7f2046806700 1 -- 172.21.15.144:0/1685753963 >> 172.21.15.144:0/1685753963 conn(0x7f2040074f50 msgr2=0x7f20400fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:14.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.475+0000 7f2046806700 1 -- 172.21.15.144:0/1685753963 shutdown_connections 2024-09-10T11:54:14.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.475+0000 7f2046806700 1 -- 172.21.15.144:0/1685753963 wait complete. 2024-09-10T11:54:14.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.585+0000 7fd50d3ca700 1 -- 172.21.15.144:0/2373764946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5081003e0 msgr2=0x7fd5081007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:14.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.585+0000 7fd50d3ca700 1 --2- 172.21.15.144:0/2373764946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5081003e0 0x7fd5081007b0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fd4f8009910 tx=0x7fd4f8009c20 comp rx=0 tx=0).stop 2024-09-10T11:54:14.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.585+0000 7fd50d3ca700 1 -- 172.21.15.144:0/2373764946 shutdown_connections 2024-09-10T11:54:14.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.585+0000 7fd50d3ca700 1 --2- 172.21.15.144:0/2373764946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd508100d80 0x7fd508104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.585+0000 7fd50d3ca700 1 --2- 172.21.15.144:0/2373764946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5081003e0 0x7fd5081007b0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.585+0000 7fd50d3ca700 1 -- 172.21.15.144:0/2373764946 >> 172.21.15.144:0/2373764946 conn(0x7fd5080fbca0 msgr2=0x7fd5080fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:14.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.585+0000 7fd50d3ca700 1 -- 172.21.15.144:0/2373764946 shutdown_connections 2024-09-10T11:54:14.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.585+0000 7fd50d3ca700 1 -- 172.21.15.144:0/2373764946 wait complete. 2024-09-10T11:54:14.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.585+0000 7fd50d3ca700 1 Processor -- start 2024-09-10T11:54:14.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.586+0000 7fd50d3ca700 1 -- start start 2024-09-10T11:54:14.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.586+0000 7fd50d3ca700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5081003e0 0x7fd508195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:14.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.586+0000 7fd50d3ca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd508100d80 0x7fd508196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:14.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.586+0000 7fd50d3ca700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd508196940 con 0x7fd508100d80 2024-09-10T11:54:14.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.586+0000 7fd50d3ca700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd50819a6d0 con 0x7fd5081003e0 2024-09-10T11:54:14.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.586+0000 7fd5067fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd508100d80 0x7fd508196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:14.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.586+0000 7fd5067fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd508100d80 0x7fd508196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42974/0 (socket says 172.21.15.144:42974) 2024-09-10T11:54:14.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.586+0000 7fd5067fc700 1 -- 172.21.15.144:0/3338818436 learned_addr learned my addr 172.21.15.144:0/3338818436 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:54:14.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.586+0000 7fd506ffd700 1 --2- 172.21.15.144:0/3338818436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5081003e0 0x7fd508195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:14.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.586+0000 7fd5067fc700 1 -- 172.21.15.144:0/3338818436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5081003e0 msgr2=0x7fd508195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:14.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.586+0000 7fd5067fc700 1 --2- 172.21.15.144:0/3338818436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5081003e0 0x7fd508195d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.586+0000 7fd5067fc700 1 -- 172.21.15.144:0/3338818436 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f80095f0 con 0x7fd508100d80 2024-09-10T11:54:14.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.586+0000 7fd5067fc700 1 --2- 172.21.15.144:0/3338818436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd508100d80 0x7fd508196260 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00d900 tx=0x7fd4fc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:14.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.587+0000 7fd4effff700 1 -- 172.21.15.144:0/3338818436 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4fc009900 con 0x7fd508100d80 2024-09-10T11:54:14.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.587+0000 7fd4effff700 1 -- 172.21.15.144:0/3338818436 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd4fc010460 con 0x7fd508100d80 2024-09-10T11:54:14.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.587+0000 7fd4effff700 1 -- 172.21.15.144:0/3338818436 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd4fc00f5f0 con 0x7fd508100d80 2024-09-10T11:54:14.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.587+0000 7fd50d3ca700 1 -- 172.21.15.144:0/3338818436 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd50819a9b0 con 0x7fd508100d80 2024-09-10T11:54:14.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.587+0000 7fd50d3ca700 1 -- 172.21.15.144:0/3338818436 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd50819af00 con 0x7fd508100d80 2024-09-10T11:54:14.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.588+0000 7fd50d3ca700 1 -- 172.21.15.144:0/3338818436 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5081086f0 con 0x7fd508100d80 2024-09-10T11:54:14.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.588+0000 7fd4effff700 1 -- 172.21.15.144:0/3338818436 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd4fc0105d0 con 0x7fd508100d80 2024-09-10T11:54:14.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.589+0000 7fd4effff700 1 --2- 172.21.15.144:0/3338818436 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd4f006e200 0x7fd4f00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:14.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.589+0000 7fd4effff700 1 -- 172.21.15.144:0/3338818436 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd4fc08de60 con 0x7fd508100d80 2024-09-10T11:54:14.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.589+0000 7fd506ffd700 1 --2- 172.21.15.144:0/3338818436 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd4f006e200 0x7fd4f00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:14.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.589+0000 7fd506ffd700 1 --2- 172.21.15.144:0/3338818436 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd4f006e200 0x7fd4f00706b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800b5c0 tx=0x7fd4f801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:14.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.593+0000 7fd4effff700 1 -- 172.21.15.144:0/3338818436 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd4fc0572c0 con 0x7fd508100d80 2024-09-10T11:54:14.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.893+0000 7fd50d3ca700 1 -- 172.21.15.144:0/3338818436 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd50804ea50 con 0x7fd508100d80 2024-09-10T11:54:14.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.893+0000 7fd4effff700 1 -- 172.21.15.144:0/3338818436 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7fd4fc017090 con 0x7fd508100d80 2024-09-10T11:54:14.895 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:54:14.895 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T11:54:14.895 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:54:14.895 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T11:54:14.895 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:54:14.895 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T11:54:14.895 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T11:54:14.895 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T11:54:14.896 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T11:54:14.896 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T11:54:14.896 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T11:54:14.896 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T11:54:14.896 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T11:54:14.896 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T11:54:14.896 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T11:54:14.896 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T11:54:14.896 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T11:54:14.897 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T11:54:14.897 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:54:14.897 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:54:14.897 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:54:14.897 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:54:14.897 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:54:14.897 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:54:14.897 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:54:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.895+0000 7fd50d3ca700 1 -- 172.21.15.144:0/3338818436 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd4f006e200 msgr2=0x7fd4f00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.895+0000 7fd50d3ca700 1 --2- 172.21.15.144:0/3338818436 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd4f006e200 0x7fd4f00706b0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd4f800b5c0 tx=0x7fd4f801a040 comp rx=0 tx=0).stop 2024-09-10T11:54:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.895+0000 7fd50d3ca700 1 -- 172.21.15.144:0/3338818436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd508100d80 msgr2=0x7fd508196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.895+0000 7fd50d3ca700 1 --2- 172.21.15.144:0/3338818436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd508100d80 0x7fd508196260 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd4fc00d900 tx=0x7fd4fc00dcc0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.895+0000 7fd50d3ca700 1 -- 172.21.15.144:0/3338818436 shutdown_connections 2024-09-10T11:54:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.895+0000 7fd50d3ca700 1 --2- 172.21.15.144:0/3338818436 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd4f006e200 0x7fd4f00706b0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.895+0000 7fd50d3ca700 1 --2- 172.21.15.144:0/3338818436 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5081003e0 0x7fd508195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.895+0000 7fd50d3ca700 1 --2- 172.21.15.144:0/3338818436 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd508100d80 0x7fd508196260 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.895+0000 7fd50d3ca700 1 -- 172.21.15.144:0/3338818436 >> 172.21.15.144:0/3338818436 conn(0x7fd5080fbca0 msgr2=0x7fd5080fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:14.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.896+0000 7fd50d3ca700 1 -- 172.21.15.144:0/3338818436 shutdown_connections 2024-09-10T11:54:14.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:14.896+0000 7fd50d3ca700 1 -- 172.21.15.144:0/3338818436 wait complete. 2024-09-10T11:54:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:14 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1319436444' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:54:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:14 smithi144 ceph-mon[87436]: from='client.35260 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:14 smithi144 ceph-mon[87436]: pgmap v669: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:14 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3338818436' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:54:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:14 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1319436444' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:54:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:14 smithi178 ceph-mon[65940]: from='client.35260 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:14 smithi178 ceph-mon[65940]: pgmap v669: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:14 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3338818436' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:54:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:16 smithi178 ceph-mon[65940]: pgmap v670: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:17.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:16 smithi144 ceph-mon[87436]: pgmap v670: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:18 smithi144 ceph-mon[87436]: pgmap v671: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:19.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:18 smithi178 ceph-mon[65940]: pgmap v671: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:20 smithi178 ceph-mon[65940]: pgmap v672: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:21.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:20 smithi144 ceph-mon[87436]: pgmap v672: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:23.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:22 smithi178 ceph-mon[65940]: pgmap v673: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:23.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:22 smithi144 ceph-mon[87436]: pgmap v673: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:25.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:24 smithi144 ceph-mon[87436]: pgmap v674: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:24 smithi178 ceph-mon[65940]: pgmap v674: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:26 smithi178 ceph-mon[65940]: pgmap v675: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:27.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:26 smithi144 ceph-mon[87436]: pgmap v675: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:28.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:54:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:54:29.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:29 smithi144 ceph-mon[87436]: pgmap v676: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:29 smithi178 ceph-mon[65940]: pgmap v676: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:30 smithi178 ceph-mon[65940]: pgmap v677: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:31.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:30 smithi144 ceph-mon[87436]: pgmap v677: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:32 smithi178 ceph-mon[65940]: pgmap v678: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:33.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:32 smithi144 ceph-mon[87436]: pgmap v678: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:34 smithi178 ceph-mon[65940]: pgmap v679: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:35.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:34 smithi144 ceph-mon[87436]: pgmap v679: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:36 smithi178 ceph-mon[65940]: pgmap v680: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:37.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:36 smithi144 ceph-mon[87436]: pgmap v680: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:39.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:38 smithi178 ceph-mon[65940]: pgmap v681: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:39.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:38 smithi144 ceph-mon[87436]: pgmap v681: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:40 smithi178 ceph-mon[65940]: pgmap v682: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:41.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:40 smithi144 ceph-mon[87436]: pgmap v682: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:43.236 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:42 smithi144 ceph-mon[87436]: pgmap v683: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:42 smithi178 ceph-mon[65940]: pgmap v683: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:44.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:54:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:54:45.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.014+0000 7f4299939700 1 -- 172.21.15.144:0/3316979951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42941003d0 msgr2=0x7f42941007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:45.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.014+0000 7f4299939700 1 --2- 172.21.15.144:0/3316979951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42941003d0 0x7f42941007a0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f427c009910 tx=0x7f427c009c20 comp rx=0 tx=0).stop 2024-09-10T11:54:45.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.014+0000 7f4299939700 1 -- 172.21.15.144:0/3316979951 shutdown_connections 2024-09-10T11:54:45.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.014+0000 7f4299939700 1 --2- 172.21.15.144:0/3316979951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4294100d70 0x7f4294104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.014+0000 7f4299939700 1 --2- 172.21.15.144:0/3316979951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f42941003d0 0x7f42941007a0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.014+0000 7f4299939700 1 -- 172.21.15.144:0/3316979951 >> 172.21.15.144:0/3316979951 conn(0x7f42940fbc90 msgr2=0x7f42940fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:45.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.014+0000 7f4299939700 1 -- 172.21.15.144:0/3316979951 shutdown_connections 2024-09-10T11:54:45.017 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.014+0000 7f4299939700 1 -- 172.21.15.144:0/3316979951 wait complete. 2024-09-10T11:54:45.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.015+0000 7f4299939700 1 Processor -- start 2024-09-10T11:54:45.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.015+0000 7f4299939700 1 -- start start 2024-09-10T11:54:45.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.015+0000 7f4299939700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42941003d0 0x7f4294073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:45.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.015+0000 7f4299939700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4294100d70 0x7f4294073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:45.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.015+0000 7f4299939700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4294073ad0 con 0x7f4294100d70 2024-09-10T11:54:45.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.015+0000 7f4299939700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4294073c10 con 0x7f42941003d0 2024-09-10T11:54:45.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.016+0000 7f42927fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4294100d70 0x7f4294073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:45.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.016+0000 7f42927fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4294100d70 0x7f4294073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52304/0 (socket says 172.21.15.144:52304) 2024-09-10T11:54:45.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.016+0000 7f42927fc700 1 -- 172.21.15.144:0/693647020 learned_addr learned my addr 172.21.15.144:0/693647020 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:54:45.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.016+0000 7f4292ffd700 1 --2- 172.21.15.144:0/693647020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42941003d0 0x7f4294073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:45.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.016+0000 7f42927fc700 1 -- 172.21.15.144:0/693647020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42941003d0 msgr2=0x7f4294073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:45.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.016+0000 7f42927fc700 1 --2- 172.21.15.144:0/693647020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42941003d0 0x7f4294073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.016+0000 7f42927fc700 1 -- 172.21.15.144:0/693647020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c0095f0 con 0x7f4294100d70 2024-09-10T11:54:45.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.016+0000 7f42927fc700 1 --2- 172.21.15.144:0/693647020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4294100d70 0x7f4294073590 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f428400d900 tx=0x7f428400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:45.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.020+0000 7f4298937700 1 -- 172.21.15.144:0/693647020 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4284009900 con 0x7f4294100d70 2024-09-10T11:54:45.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.021+0000 7f4298937700 1 -- 172.21.15.144:0/693647020 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4284010460 con 0x7f4294100d70 2024-09-10T11:54:45.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.021+0000 7f4298937700 1 -- 172.21.15.144:0/693647020 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f428400f5f0 con 0x7f4294100d70 2024-09-10T11:54:45.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.021+0000 7f4299939700 1 -- 172.21.15.144:0/693647020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42941a1f10 con 0x7f4294100d70 2024-09-10T11:54:45.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.021+0000 7f4299939700 1 -- 172.21.15.144:0/693647020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42941a2320 con 0x7f4294100d70 2024-09-10T11:54:45.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.022+0000 7f4299939700 1 -- 172.21.15.144:0/693647020 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42941086e0 con 0x7f4294100d70 2024-09-10T11:54:45.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.022+0000 7f4298937700 1 -- 172.21.15.144:0/693647020 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f42840105d0 con 0x7f4294100d70 2024-09-10T11:54:45.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.023+0000 7f4298937700 1 --2- 172.21.15.144:0/693647020 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f428006e200 0x7f42800706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:45.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.023+0000 7f4298937700 1 -- 172.21.15.144:0/693647020 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f428408db40 con 0x7f4294100d70 2024-09-10T11:54:45.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.023+0000 7f4292ffd700 1 --2- 172.21.15.144:0/693647020 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f428006e200 0x7f42800706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:45.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.024+0000 7f4292ffd700 1 --2- 172.21.15.144:0/693647020 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f428006e200 0x7f42800706b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f427c006010 tx=0x7f427c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:45.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.028+0000 7f4298937700 1 -- 172.21.15.144:0/693647020 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4284056fa0 con 0x7f4294100d70 2024-09-10T11:54:45.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.249+0000 7f4299939700 1 -- 172.21.15.144:0/693647020 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f429402cca0 con 0x7f428006e200 2024-09-10T11:54:45.252 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:44 smithi144 ceph-mon[87436]: pgmap v684: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:45.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.250+0000 7f4298937700 1 -- 172.21.15.144:0/693647020 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f429402cca0 con 0x7f428006e200 2024-09-10T11:54:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.253+0000 7f4299939700 1 -- 172.21.15.144:0/693647020 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f428006e200 msgr2=0x7f42800706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.253+0000 7f4299939700 1 --2- 172.21.15.144:0/693647020 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f428006e200 0x7f42800706b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f427c006010 tx=0x7f427c01a040 comp rx=0 tx=0).stop 2024-09-10T11:54:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.253+0000 7f4299939700 1 -- 172.21.15.144:0/693647020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4294100d70 msgr2=0x7f4294073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.253+0000 7f4299939700 1 --2- 172.21.15.144:0/693647020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4294100d70 0x7f4294073590 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f428400d900 tx=0x7f428400dc10 comp rx=0 tx=0).stop 2024-09-10T11:54:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.253+0000 7f4299939700 1 -- 172.21.15.144:0/693647020 shutdown_connections 2024-09-10T11:54:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.253+0000 7f4299939700 1 --2- 172.21.15.144:0/693647020 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f428006e200 0x7f42800706b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.253+0000 7f4299939700 1 --2- 172.21.15.144:0/693647020 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f42941003d0 0x7f4294073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.253+0000 7f4299939700 1 --2- 172.21.15.144:0/693647020 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4294100d70 0x7f4294073590 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.253+0000 7f4299939700 1 -- 172.21.15.144:0/693647020 >> 172.21.15.144:0/693647020 conn(0x7f42940fbc90 msgr2=0x7f42940fd360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:45.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.253+0000 7f4299939700 1 -- 172.21.15.144:0/693647020 shutdown_connections 2024-09-10T11:54:45.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.253+0000 7f4299939700 1 -- 172.21.15.144:0/693647020 wait complete. 2024-09-10T11:54:45.265 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:54:45.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.359+0000 7fe4e25ac700 1 -- 172.21.15.144:0/1233223457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc101c40 msgr2=0x7fe4dc10a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:45.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.359+0000 7fe4e25ac700 1 --2- 172.21.15.144:0/1233223457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc101c40 0x7fe4dc10a140 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0009940 tx=0x7fe4d0009c50 comp rx=0 tx=0).stop 2024-09-10T11:54:45.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.359+0000 7fe4e25ac700 1 -- 172.21.15.144:0/1233223457 shutdown_connections 2024-09-10T11:54:45.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.359+0000 7fe4e25ac700 1 --2- 172.21.15.144:0/1233223457 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc101c40 0x7fe4dc10a140 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.359+0000 7fe4e25ac700 1 --2- 172.21.15.144:0/1233223457 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4dc101330 0x7fe4dc101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.359+0000 7fe4e25ac700 1 -- 172.21.15.144:0/1233223457 >> 172.21.15.144:0/1233223457 conn(0x7fe4dc0fabb0 msgr2=0x7fe4dc0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:45.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.359+0000 7fe4e25ac700 1 -- 172.21.15.144:0/1233223457 shutdown_connections 2024-09-10T11:54:45.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.359+0000 7fe4e25ac700 1 -- 172.21.15.144:0/1233223457 wait complete. 2024-09-10T11:54:45.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.359+0000 7fe4e25ac700 1 Processor -- start 2024-09-10T11:54:45.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.359+0000 7fe4e25ac700 1 -- start start 2024-09-10T11:54:45.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.360+0000 7fe4e25ac700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4dc101330 0x7fe4dc19bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:45.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.360+0000 7fe4e25ac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc101c40 0x7fe4dc19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:45.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.360+0000 7fe4e25ac700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc19c690 con 0x7fe4dc101c40 2024-09-10T11:54:45.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.360+0000 7fe4e25ac700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc195b40 con 0x7fe4dc101330 2024-09-10T11:54:45.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.360+0000 7fe4db7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc101c40 0x7fe4dc19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:45.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.360+0000 7fe4db7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc101c40 0x7fe4dc19c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52320/0 (socket says 172.21.15.144:52320) 2024-09-10T11:54:45.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.360+0000 7fe4db7fe700 1 -- 172.21.15.144:0/2739588813 learned_addr learned my addr 172.21.15.144:0/2739588813 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:54:45.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.360+0000 7fe4dbfff700 1 --2- 172.21.15.144:0/2739588813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4dc101330 0x7fe4dc19bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:45.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.360+0000 7fe4db7fe700 1 -- 172.21.15.144:0/2739588813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4dc101330 msgr2=0x7fe4dc19bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:45.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.360+0000 7fe4db7fe700 1 --2- 172.21.15.144:0/2739588813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4dc101330 0x7fe4dc19bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.360+0000 7fe4db7fe700 1 -- 172.21.15.144:0/2739588813 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d00095f0 con 0x7fe4dc101c40 2024-09-10T11:54:45.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.361+0000 7fe4db7fe700 1 --2- 172.21.15.144:0/2739588813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc101c40 0x7fe4dc19c000 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000b580 tx=0x7fe4d00055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:45.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.361+0000 7fe4d97fa700 1 -- 172.21.15.144:0/2739588813 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4d001d070 con 0x7fe4dc101c40 2024-09-10T11:54:45.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.361+0000 7fe4d97fa700 1 -- 172.21.15.144:0/2739588813 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe4d0005dc0 con 0x7fe4dc101c40 2024-09-10T11:54:45.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.361+0000 7fe4e25ac700 1 -- 172.21.15.144:0/2739588813 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4dc195dc0 con 0x7fe4dc101c40 2024-09-10T11:54:45.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.361+0000 7fe4d97fa700 1 -- 172.21.15.144:0/2739588813 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4d0021410 con 0x7fe4dc101c40 2024-09-10T11:54:45.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.361+0000 7fe4e25ac700 1 -- 172.21.15.144:0/2739588813 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4dc1962b0 con 0x7fe4dc101c40 2024-09-10T11:54:45.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.362+0000 7fe4e25ac700 1 -- 172.21.15.144:0/2739588813 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4dc1077b0 con 0x7fe4dc101c40 2024-09-10T11:54:45.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.362+0000 7fe4d97fa700 1 -- 172.21.15.144:0/2739588813 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe4d000f7f0 con 0x7fe4dc101c40 2024-09-10T11:54:45.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.363+0000 7fe4d97fa700 1 --2- 172.21.15.144:0/2739588813 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe4c406e200 0x7fe4c40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:45.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.363+0000 7fe4d97fa700 1 -- 172.21.15.144:0/2739588813 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe4d008f3c0 con 0x7fe4dc101c40 2024-09-10T11:54:45.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.363+0000 7fe4dbfff700 1 --2- 172.21.15.144:0/2739588813 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe4c406e200 0x7fe4c40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:45.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.363+0000 7fe4dbfff700 1 --2- 172.21.15.144:0/2739588813 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe4c406e200 0x7fe4c40706b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005950 tx=0x7fe4cc00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:44 smithi178 ceph-mon[65940]: pgmap v684: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:45.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.368+0000 7fe4d97fa700 1 -- 172.21.15.144:0/2739588813 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe4d005c2f0 con 0x7fe4dc101c40 2024-09-10T11:54:45.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.585+0000 7fe4e25ac700 1 -- 172.21.15.144:0/2739588813 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4dc196590 con 0x7fe4c406e200 2024-09-10T11:54:45.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.587+0000 7fe4d97fa700 1 -- 172.21.15.144:0/2739588813 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fe4dc196590 con 0x7fe4c406e200 2024-09-10T11:54:45.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.590+0000 7fe4e25ac700 1 -- 172.21.15.144:0/2739588813 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe4c406e200 msgr2=0x7fe4c40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:45.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.590+0000 7fe4e25ac700 1 --2- 172.21.15.144:0/2739588813 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe4c406e200 0x7fe4c40706b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc005950 tx=0x7fe4cc00b410 comp rx=0 tx=0).stop 2024-09-10T11:54:45.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.590+0000 7fe4e25ac700 1 -- 172.21.15.144:0/2739588813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc101c40 msgr2=0x7fe4dc19c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:45.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.590+0000 7fe4e25ac700 1 --2- 172.21.15.144:0/2739588813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc101c40 0x7fe4dc19c000 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe4d000b580 tx=0x7fe4d00055f0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.590+0000 7fe4e25ac700 1 -- 172.21.15.144:0/2739588813 shutdown_connections 2024-09-10T11:54:45.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.590+0000 7fe4e25ac700 1 --2- 172.21.15.144:0/2739588813 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe4c406e200 0x7fe4c40706b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.590+0000 7fe4e25ac700 1 --2- 172.21.15.144:0/2739588813 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe4dc101330 0x7fe4dc19bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.590+0000 7fe4e25ac700 1 --2- 172.21.15.144:0/2739588813 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe4dc101c40 0x7fe4dc19c000 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.590+0000 7fe4e25ac700 1 -- 172.21.15.144:0/2739588813 >> 172.21.15.144:0/2739588813 conn(0x7fe4dc0fabb0 msgr2=0x7fe4dc1048f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:45.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.590+0000 7fe4e25ac700 1 -- 172.21.15.144:0/2739588813 shutdown_connections 2024-09-10T11:54:45.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.590+0000 7fe4e25ac700 1 -- 172.21.15.144:0/2739588813 wait complete. 2024-09-10T11:54:45.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.696+0000 7f9e2552c700 1 -- 172.21.15.144:0/1238440394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e200ff390 msgr2=0x7f9e200ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:45.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.696+0000 7f9e2552c700 1 --2- 172.21.15.144:0/1238440394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e200ff390 0x7f9e200ff760 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009910 tx=0x7f9e10009c20 comp rx=0 tx=0).stop 2024-09-10T11:54:45.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.696+0000 7f9e2552c700 1 -- 172.21.15.144:0/1238440394 shutdown_connections 2024-09-10T11:54:45.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.696+0000 7f9e2552c700 1 --2- 172.21.15.144:0/1238440394 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e200ffd30 0x7f9e20103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.696+0000 7f9e2552c700 1 --2- 172.21.15.144:0/1238440394 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e200ff390 0x7f9e200ff760 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.696+0000 7f9e2552c700 1 -- 172.21.15.144:0/1238440394 >> 172.21.15.144:0/1238440394 conn(0x7f9e20074f50 msgr2=0x7f9e20075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:45.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.696+0000 7f9e2552c700 1 -- 172.21.15.144:0/1238440394 shutdown_connections 2024-09-10T11:54:45.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.696+0000 7f9e2552c700 1 -- 172.21.15.144:0/1238440394 wait complete. 2024-09-10T11:54:45.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.697+0000 7f9e2552c700 1 Processor -- start 2024-09-10T11:54:45.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.697+0000 7f9e2552c700 1 -- start start 2024-09-10T11:54:45.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.697+0000 7f9e2552c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e200ff390 0x7f9e20100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:45.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.697+0000 7f9e2552c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e200ffd30 0x7f9e201012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:45.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.697+0000 7f9e2552c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e20101800 con 0x7f9e200ffd30 2024-09-10T11:54:45.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.697+0000 7f9e2552c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e20101940 con 0x7f9e200ff390 2024-09-10T11:54:45.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.697+0000 7f9e17fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e200ffd30 0x7f9e201012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:45.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.697+0000 7f9e17fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e200ffd30 0x7f9e201012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52332/0 (socket says 172.21.15.144:52332) 2024-09-10T11:54:45.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.697+0000 7f9e17fff700 1 -- 172.21.15.144:0/2993768218 learned_addr learned my addr 172.21.15.144:0/2993768218 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:54:45.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.697+0000 7f9e1effd700 1 --2- 172.21.15.144:0/2993768218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e200ff390 0x7f9e20100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:45.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.697+0000 7f9e17fff700 1 -- 172.21.15.144:0/2993768218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e200ff390 msgr2=0x7f9e20100d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:45.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.697+0000 7f9e17fff700 1 --2- 172.21.15.144:0/2993768218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e200ff390 0x7f9e20100d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.697+0000 7f9e17fff700 1 -- 172.21.15.144:0/2993768218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e100095f0 con 0x7f9e200ffd30 2024-09-10T11:54:45.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.697+0000 7f9e1effd700 1 --2- 172.21.15.144:0/2993768218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e200ff390 0x7f9e20100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:54:45.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.698+0000 7f9e17fff700 1 --2- 172.21.15.144:0/2993768218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e200ffd30 0x7f9e201012c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800b480 tx=0x7f9e0800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:45.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.698+0000 7f9e1cff9700 1 -- 172.21.15.144:0/2993768218 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e080118c0 con 0x7f9e200ffd30 2024-09-10T11:54:45.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.698+0000 7f9e1cff9700 1 -- 172.21.15.144:0/2993768218 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e08010380 con 0x7f9e200ffd30 2024-09-10T11:54:45.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.698+0000 7f9e2552c700 1 -- 172.21.15.144:0/2993768218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e201a2000 con 0x7f9e200ffd30 2024-09-10T11:54:45.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.698+0000 7f9e1cff9700 1 -- 172.21.15.144:0/2993768218 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e0800f4f0 con 0x7f9e200ffd30 2024-09-10T11:54:45.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.698+0000 7f9e2552c700 1 -- 172.21.15.144:0/2993768218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e201a2410 con 0x7f9e200ffd30 2024-09-10T11:54:45.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.699+0000 7f9e2552c700 1 -- 172.21.15.144:0/2993768218 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e20108660 con 0x7f9e200ffd30 2024-09-10T11:54:45.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.699+0000 7f9e1cff9700 1 -- 172.21.15.144:0/2993768218 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9e080104f0 con 0x7f9e200ffd30 2024-09-10T11:54:45.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.700+0000 7f9e1cff9700 1 --2- 172.21.15.144:0/2993768218 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e0c06e200 0x7f9e0c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:45.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.700+0000 7f9e1cff9700 1 -- 172.21.15.144:0/2993768218 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9e0808dce0 con 0x7f9e200ffd30 2024-09-10T11:54:45.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.700+0000 7f9e1effd700 1 --2- 172.21.15.144:0/2993768218 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e0c06e200 0x7f9e0c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:45.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.700+0000 7f9e1effd700 1 --2- 172.21.15.144:0/2993768218 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e0c06e200 0x7f9e0c0706b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000b5c0 tx=0x7f9e10005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:45.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.704+0000 7f9e1cff9700 1 -- 172.21.15.144:0/2993768218 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9e080570c0 con 0x7f9e200ffd30 2024-09-10T11:54:45.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.913+0000 7f9e2552c700 1 -- 172.21.15.144:0/2993768218 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e2002cd60 con 0x7f9e0c06e200 2024-09-10T11:54:45.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.923+0000 7f9e1cff9700 1 -- 172.21.15.144:0/2993768218 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f9e2002cd60 con 0x7f9e0c06e200 2024-09-10T11:54:45.925 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:54:45.925 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (23m) 78s ago 33m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:54:45.926 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (33m) 78s ago 33m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:54:45.926 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (32m) 80s ago 32m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:54:45.926 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (20m) 78s ago 33m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:54:45.926 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (20m) 80s ago 32m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:54:45.926 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (22m) 78s ago 33m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:54:45.926 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (17m) 78s ago 28m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:54:45.926 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (16m) 78s ago 28m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:54:45.926 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (83s) 80s ago 27m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T11:54:45.926 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (16m) 80s ago 28m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:54:45.926 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (24m) 78s ago 34m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:54:45.926 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (23m) 80s ago 32m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:54:45.926 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (21m) 78s ago 35m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:54:45.926 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (21m) 80s ago 32m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:54:45.926 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (23m) 78s ago 33m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:54:45.927 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (23m) 80s ago 32m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:54:45.927 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (20m) 78s ago 31m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:54:45.927 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (19m) 78s ago 31m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:54:45.927 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (19m) 78s ago 30m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:54:45.927 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (18m) 80s ago 30m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:54:45.927 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (17m) 80s ago 29m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:54:45.927 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (17m) 80s ago 29m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:54:45.927 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (23m) 78s ago 33m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:54:45.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.926+0000 7f9e2552c700 1 -- 172.21.15.144:0/2993768218 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e0c06e200 msgr2=0x7f9e0c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:45.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.926+0000 7f9e2552c700 1 --2- 172.21.15.144:0/2993768218 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e0c06e200 0x7f9e0c0706b0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9e1000b5c0 tx=0x7f9e10005480 comp rx=0 tx=0).stop 2024-09-10T11:54:45.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.926+0000 7f9e2552c700 1 -- 172.21.15.144:0/2993768218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e200ffd30 msgr2=0x7f9e201012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:45.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.926+0000 7f9e2552c700 1 --2- 172.21.15.144:0/2993768218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e200ffd30 0x7f9e201012c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f9e0800b480 tx=0x7f9e0800b840 comp rx=0 tx=0).stop 2024-09-10T11:54:45.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.926+0000 7f9e2552c700 1 -- 172.21.15.144:0/2993768218 shutdown_connections 2024-09-10T11:54:45.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.926+0000 7f9e2552c700 1 --2- 172.21.15.144:0/2993768218 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e0c06e200 0x7f9e0c0706b0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.926+0000 7f9e2552c700 1 --2- 172.21.15.144:0/2993768218 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e200ff390 0x7f9e20100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.926+0000 7f9e2552c700 1 --2- 172.21.15.144:0/2993768218 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e200ffd30 0x7f9e201012c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:45.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.926+0000 7f9e2552c700 1 -- 172.21.15.144:0/2993768218 >> 172.21.15.144:0/2993768218 conn(0x7f9e20074f50 msgr2=0x7f9e200686d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:45.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.926+0000 7f9e2552c700 1 -- 172.21.15.144:0/2993768218 shutdown_connections 2024-09-10T11:54:45.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:45.926+0000 7f9e2552c700 1 -- 172.21.15.144:0/2993768218 wait complete. 2024-09-10T11:54:46.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.031+0000 7fddda723700 1 -- 172.21.15.144:0/1551475250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd40ffc70 msgr2=0x7fddd4103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:46.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.031+0000 7fddda723700 1 --2- 172.21.15.144:0/1551475250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd40ffc70 0x7fddd4103c40 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009910 tx=0x7fddc8009c20 comp rx=0 tx=0).stop 2024-09-10T11:54:46.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.031+0000 7fddda723700 1 -- 172.21.15.144:0/1551475250 shutdown_connections 2024-09-10T11:54:46.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.031+0000 7fddda723700 1 --2- 172.21.15.144:0/1551475250 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd40ffc70 0x7fddd4103c40 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.031+0000 7fddda723700 1 --2- 172.21.15.144:0/1551475250 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddd40ff2d0 0x7fddd40ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.031+0000 7fddda723700 1 -- 172.21.15.144:0/1551475250 >> 172.21.15.144:0/1551475250 conn(0x7fddd40fabb0 msgr2=0x7fddd40fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:46.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.031+0000 7fddda723700 1 -- 172.21.15.144:0/1551475250 shutdown_connections 2024-09-10T11:54:46.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.031+0000 7fddda723700 1 -- 172.21.15.144:0/1551475250 wait complete. 2024-09-10T11:54:46.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.032+0000 7fddda723700 1 Processor -- start 2024-09-10T11:54:46.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.032+0000 7fddda723700 1 -- start start 2024-09-10T11:54:46.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.032+0000 7fddda723700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddd40ff2d0 0x7fddd4195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:46.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.032+0000 7fddda723700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd40ffc70 0x7fddd4196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:46.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.032+0000 7fddda723700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd4196a60 con 0x7fddd40ffc70 2024-09-10T11:54:46.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.032+0000 7fddda723700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd419a7f0 con 0x7fddd40ff2d0 2024-09-10T11:54:46.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.033+0000 7fddd37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd40ffc70 0x7fddd4196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:46.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.033+0000 7fddd37fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd40ffc70 0x7fddd4196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52344/0 (socket says 172.21.15.144:52344) 2024-09-10T11:54:46.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.033+0000 7fddd37fe700 1 -- 172.21.15.144:0/195300563 learned_addr learned my addr 172.21.15.144:0/195300563 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:54:46.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.033+0000 7fddd3fff700 1 --2- 172.21.15.144:0/195300563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddd40ff2d0 0x7fddd4195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:46.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.033+0000 7fddd37fe700 1 -- 172.21.15.144:0/195300563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddd40ff2d0 msgr2=0x7fddd4195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:46.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.033+0000 7fddd37fe700 1 --2- 172.21.15.144:0/195300563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddd40ff2d0 0x7fddd4195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.033+0000 7fddd37fe700 1 -- 172.21.15.144:0/195300563 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc80095f0 con 0x7fddd40ffc70 2024-09-10T11:54:46.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.033+0000 7fddd37fe700 1 --2- 172.21.15.144:0/195300563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd40ffc70 0x7fddd4196380 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fddc80092e0 tx=0x7fddc800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:46.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.033+0000 7fddd17fa700 1 -- 172.21.15.144:0/195300563 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddc801d070 con 0x7fddd40ffc70 2024-09-10T11:54:46.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.033+0000 7fddda723700 1 -- 172.21.15.144:0/195300563 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd419aa70 con 0x7fddd40ffc70 2024-09-10T11:54:46.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.033+0000 7fddd17fa700 1 -- 172.21.15.144:0/195300563 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fddc800f460 con 0x7fddd40ffc70 2024-09-10T11:54:46.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.033+0000 7fddd17fa700 1 -- 172.21.15.144:0/195300563 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fddc80175d0 con 0x7fddd40ffc70 2024-09-10T11:54:46.036 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.033+0000 7fddda723700 1 -- 172.21.15.144:0/195300563 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd419af60 con 0x7fddd40ffc70 2024-09-10T11:54:46.037 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.034+0000 7fddda723700 1 -- 172.21.15.144:0/195300563 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd404ea50 con 0x7fddd40ffc70 2024-09-10T11:54:46.037 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.035+0000 7fddd17fa700 1 -- 172.21.15.144:0/195300563 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fddc800f5d0 con 0x7fddd40ffc70 2024-09-10T11:54:46.037 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.035+0000 7fddd17fa700 1 --2- 172.21.15.144:0/195300563 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fddbc06e200 0x7fddbc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:46.037 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.035+0000 7fddd17fa700 1 -- 172.21.15.144:0/195300563 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fddc808f570 con 0x7fddd40ffc70 2024-09-10T11:54:46.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.035+0000 7fddd3fff700 1 --2- 172.21.15.144:0/195300563 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fddbc06e200 0x7fddbc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:46.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.036+0000 7fddd3fff700 1 --2- 172.21.15.144:0/195300563 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fddbc06e200 0x7fddbc0706b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009a10 tx=0x7fddc4009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:46.040 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.039+0000 7fddd17fa700 1 -- 172.21.15.144:0/195300563 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fddc8058950 con 0x7fddd40ffc70 2024-09-10T11:54:46.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:45 smithi144 ceph-mon[87436]: from='client.35268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:46.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:45 smithi144 ceph-mon[87436]: from='client.35272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:46.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.327+0000 7fddda723700 1 -- 172.21.15.144:0/195300563 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fddd419b300 con 0x7fddd40ffc70 2024-09-10T11:54:46.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.327+0000 7fddd17fa700 1 -- 172.21.15.144:0/195300563 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fddc805c010 con 0x7fddd40ffc70 2024-09-10T11:54:46.329 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:54:46.329 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:54:46.329 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:54:46.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:54:46.329 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:54:46.329 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:54:46.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:54:46.329 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:54:46.329 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:54:46.329 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:54:46.329 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:54:46.329 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:54:46.330 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:54:46.330 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:54:46.330 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T11:54:46.330 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:54:46.330 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:54:46.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.329+0000 7fddda723700 1 -- 172.21.15.144:0/195300563 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fddbc06e200 msgr2=0x7fddbc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:46.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.329+0000 7fddda723700 1 --2- 172.21.15.144:0/195300563 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fddbc06e200 0x7fddbc0706b0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fddc4009a10 tx=0x7fddc4009190 comp rx=0 tx=0).stop 2024-09-10T11:54:46.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.329+0000 7fddda723700 1 -- 172.21.15.144:0/195300563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd40ffc70 msgr2=0x7fddd4196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:46.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.329+0000 7fddda723700 1 --2- 172.21.15.144:0/195300563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd40ffc70 0x7fddd4196380 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fddc80092e0 tx=0x7fddc800bd70 comp rx=0 tx=0).stop 2024-09-10T11:54:46.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.329+0000 7fddda723700 1 -- 172.21.15.144:0/195300563 shutdown_connections 2024-09-10T11:54:46.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.330+0000 7fddda723700 1 --2- 172.21.15.144:0/195300563 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fddbc06e200 0x7fddbc0706b0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.330+0000 7fddda723700 1 --2- 172.21.15.144:0/195300563 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fddd40ff2d0 0x7fddd4195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.330+0000 7fddda723700 1 --2- 172.21.15.144:0/195300563 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fddd40ffc70 0x7fddd4196380 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.330+0000 7fddda723700 1 -- 172.21.15.144:0/195300563 >> 172.21.15.144:0/195300563 conn(0x7fddd40fabb0 msgr2=0x7fddd41033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:46.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.330+0000 7fddda723700 1 -- 172.21.15.144:0/195300563 shutdown_connections 2024-09-10T11:54:46.332 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.330+0000 7fddda723700 1 -- 172.21.15.144:0/195300563 wait complete. 2024-09-10T11:54:46.366 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:45 smithi178 ceph-mon[65940]: from='client.35268 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:46.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:45 smithi178 ceph-mon[65940]: from='client.35272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:46.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.435+0000 7ff4330dd700 1 -- 172.21.15.144:0/3714278817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff42c100dd0 msgr2=0x7ff42c104da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:46.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.435+0000 7ff4330dd700 1 --2- 172.21.15.144:0/3714278817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff42c100dd0 0x7ff42c104da0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff41c00b0a0 tx=0x7ff41c00b3b0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.435+0000 7ff4330dd700 1 -- 172.21.15.144:0/3714278817 shutdown_connections 2024-09-10T11:54:46.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.435+0000 7ff4330dd700 1 --2- 172.21.15.144:0/3714278817 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff42c100dd0 0x7ff42c104da0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.435+0000 7ff4330dd700 1 --2- 172.21.15.144:0/3714278817 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff42c100430 0x7ff42c100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.435+0000 7ff4330dd700 1 -- 172.21.15.144:0/3714278817 >> 172.21.15.144:0/3714278817 conn(0x7ff42c0fbcf0 msgr2=0x7ff42c0fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:46.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.435+0000 7ff4330dd700 1 -- 172.21.15.144:0/3714278817 shutdown_connections 2024-09-10T11:54:46.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.435+0000 7ff4330dd700 1 -- 172.21.15.144:0/3714278817 wait complete. 2024-09-10T11:54:46.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.436+0000 7ff4330dd700 1 Processor -- start 2024-09-10T11:54:46.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.436+0000 7ff4330dd700 1 -- start start 2024-09-10T11:54:46.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.436+0000 7ff4330dd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff42c100430 0x7ff42c195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:46.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.436+0000 7ff4330dd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff42c100dd0 0x7ff42c1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:46.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.436+0000 7ff4330dd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c1968f0 con 0x7ff42c100dd0 2024-09-10T11:54:46.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.436+0000 7ff4330dd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff42c196a30 con 0x7ff42c100430 2024-09-10T11:54:46.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.436+0000 7ff42bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff42c100dd0 0x7ff42c1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:46.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.436+0000 7ff42bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff42c100dd0 0x7ff42c1962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52354/0 (socket says 172.21.15.144:52354) 2024-09-10T11:54:46.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.436+0000 7ff42bfff700 1 -- 172.21.15.144:0/4096093198 learned_addr learned my addr 172.21.15.144:0/4096093198 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:54:46.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.436+0000 7ff430e79700 1 --2- 172.21.15.144:0/4096093198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff42c100430 0x7ff42c195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:46.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.436+0000 7ff42bfff700 1 -- 172.21.15.144:0/4096093198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff42c100430 msgr2=0x7ff42c195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:46.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.436+0000 7ff42bfff700 1 --2- 172.21.15.144:0/4096093198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff42c100430 0x7ff42c195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.436+0000 7ff42bfff700 1 -- 172.21.15.144:0/4096093198 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c009d00 con 0x7ff42c100dd0 2024-09-10T11:54:46.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.437+0000 7ff42bfff700 1 --2- 172.21.15.144:0/4096093198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff42c100dd0 0x7ff42c1962a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff41c006130 tx=0x7ff41c00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:46.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.437+0000 7ff429ffb700 1 -- 172.21.15.144:0/4096093198 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff41c00e040 con 0x7ff42c100dd0 2024-09-10T11:54:46.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.437+0000 7ff429ffb700 1 -- 172.21.15.144:0/4096093198 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff41c003de0 con 0x7ff42c100dd0 2024-09-10T11:54:46.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.437+0000 7ff429ffb700 1 -- 172.21.15.144:0/4096093198 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff41c01d8e0 con 0x7ff42c100dd0 2024-09-10T11:54:46.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.437+0000 7ff4330dd700 1 -- 172.21.15.144:0/4096093198 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff42c19a820 con 0x7ff42c100dd0 2024-09-10T11:54:46.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.437+0000 7ff4330dd700 1 -- 172.21.15.144:0/4096093198 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff42c19ad10 con 0x7ff42c100dd0 2024-09-10T11:54:46.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.438+0000 7ff4330dd700 1 -- 172.21.15.144:0/4096093198 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff42c1087a0 con 0x7ff42c100dd0 2024-09-10T11:54:46.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.439+0000 7ff429ffb700 1 -- 172.21.15.144:0/4096093198 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff41c004820 con 0x7ff42c100dd0 2024-09-10T11:54:46.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.439+0000 7ff429ffb700 1 --2- 172.21.15.144:0/4096093198 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff41806e200 0x7ff4180706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:46.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.439+0000 7ff429ffb700 1 -- 172.21.15.144:0/4096093198 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff41c08f910 con 0x7ff42c100dd0 2024-09-10T11:54:46.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.439+0000 7ff430e79700 1 --2- 172.21.15.144:0/4096093198 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff41806e200 0x7ff4180706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:46.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.440+0000 7ff430e79700 1 --2- 172.21.15.144:0/4096093198 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff41806e200 0x7ff4180706b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff42400ba60 tx=0x7ff424005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:46.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.444+0000 7ff429ffb700 1 -- 172.21.15.144:0/4096093198 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff41c058d70 con 0x7ff42c100dd0 2024-09-10T11:54:46.690 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.689+0000 7ff4330dd700 1 -- 172.21.15.144:0/4096093198 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff42c02cc30 con 0x7ff42c100dd0 2024-09-10T11:54:46.690 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.689+0000 7ff429ffb700 1 -- 172.21.15.144:0/4096093198 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7ff41c05c430 con 0x7ff42c100dd0 2024-09-10T11:54:46.691 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T11:54:46.691 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T11:54:46.691 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:54:46.691 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:54:46.691 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:54:46.691 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:54:46.691 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:54:46.691 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:54:46.691 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:54:46.691 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:54:46.691 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:54:46.692 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:54:46.692 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:54:46.692 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:54:46.692 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:54:46.692 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:54:46.692 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:54:46.692 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:54:46.692 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:54:46.692 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:54:46.692 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:54:46.692 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:54:46.692 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:54:46.692 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:54:46.692 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:54:46.693 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:54:46.693 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:54:46.693 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:54:46.693 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:54:46.693 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:54:46.693 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:54:46.693 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:54:46.693 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:54:46.693 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:54:46.693 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:54:46.693 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:54:46.693 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:54:46.693 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:54:46.693 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:54:46.694 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:54:46.694 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:54:46.694 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:54:46.694 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:54:46.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.692+0000 7ff4330dd700 1 -- 172.21.15.144:0/4096093198 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff41806e200 msgr2=0x7ff4180706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:46.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.692+0000 7ff4330dd700 1 --2- 172.21.15.144:0/4096093198 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff41806e200 0x7ff4180706b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff42400ba60 tx=0x7ff424005d50 comp rx=0 tx=0).stop 2024-09-10T11:54:46.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.692+0000 7ff4330dd700 1 -- 172.21.15.144:0/4096093198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff42c100dd0 msgr2=0x7ff42c1962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:46.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.692+0000 7ff4330dd700 1 --2- 172.21.15.144:0/4096093198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff42c100dd0 0x7ff42c1962a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff41c006130 tx=0x7ff41c00cdd0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.692+0000 7ff4330dd700 1 -- 172.21.15.144:0/4096093198 shutdown_connections 2024-09-10T11:54:46.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.692+0000 7ff4330dd700 1 --2- 172.21.15.144:0/4096093198 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff41806e200 0x7ff4180706b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.692+0000 7ff4330dd700 1 --2- 172.21.15.144:0/4096093198 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff42c100430 0x7ff42c195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.692+0000 7ff4330dd700 1 --2- 172.21.15.144:0/4096093198 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff42c100dd0 0x7ff42c1962a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.692+0000 7ff4330dd700 1 -- 172.21.15.144:0/4096093198 >> 172.21.15.144:0/4096093198 conn(0x7ff42c0fbcf0 msgr2=0x7ff42c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:46.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.692+0000 7ff4330dd700 1 -- 172.21.15.144:0/4096093198 shutdown_connections 2024-09-10T11:54:46.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.692+0000 7ff4330dd700 1 -- 172.21.15.144:0/4096093198 wait complete. 2024-09-10T11:54:46.695 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T11:54:46.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.803+0000 7fa4c16c5700 1 -- 172.21.15.144:0/2094401542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4bc068630 msgr2=0x7fa4bc068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:46.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.803+0000 7fa4c16c5700 1 --2- 172.21.15.144:0/2094401542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4bc068630 0x7fa4bc068a00 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac009910 tx=0x7fa4ac009c20 comp rx=0 tx=0).stop 2024-09-10T11:54:46.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.803+0000 7fa4c16c5700 1 -- 172.21.15.144:0/2094401542 shutdown_connections 2024-09-10T11:54:46.804 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.803+0000 7fa4c16c5700 1 --2- 172.21.15.144:0/2094401542 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4bc068fd0 0x7fa4bc104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.803+0000 7fa4c16c5700 1 --2- 172.21.15.144:0/2094401542 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4bc068630 0x7fa4bc068a00 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.803+0000 7fa4c16c5700 1 -- 172.21.15.144:0/2094401542 >> 172.21.15.144:0/2094401542 conn(0x7fa4bc074f50 msgr2=0x7fa4bc075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:46.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.803+0000 7fa4c16c5700 1 -- 172.21.15.144:0/2094401542 shutdown_connections 2024-09-10T11:54:46.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.803+0000 7fa4c16c5700 1 -- 172.21.15.144:0/2094401542 wait complete. 2024-09-10T11:54:46.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.803+0000 7fa4c16c5700 1 Processor -- start 2024-09-10T11:54:46.806 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.804+0000 7fa4c16c5700 1 -- start start 2024-09-10T11:54:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.804+0000 7fa4c16c5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4bc068630 0x7fa4bc195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.804+0000 7fa4c16c5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4bc068fd0 0x7fa4bc196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.804+0000 7fa4c16c5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc196b40 con 0x7fa4bc068fd0 2024-09-10T11:54:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.804+0000 7fa4c16c5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4bc19a8d0 con 0x7fa4bc068630 2024-09-10T11:54:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.804+0000 7fa4b27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4bc068fd0 0x7fa4bc196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.804+0000 7fa4b27fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4bc068fd0 0x7fa4bc196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52378/0 (socket says 172.21.15.144:52378) 2024-09-10T11:54:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.804+0000 7fa4b27fc700 1 -- 172.21.15.144:0/2942503375 learned_addr learned my addr 172.21.15.144:0/2942503375 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:54:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.804+0000 7fa4baffd700 1 --2- 172.21.15.144:0/2942503375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4bc068630 0x7fa4bc195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.804+0000 7fa4b27fc700 1 -- 172.21.15.144:0/2942503375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4bc068630 msgr2=0x7fa4bc195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.804+0000 7fa4b27fc700 1 --2- 172.21.15.144:0/2942503375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4bc068630 0x7fa4bc195f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.804+0000 7fa4b27fc700 1 -- 172.21.15.144:0/2942503375 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4ac0095f0 con 0x7fa4bc068fd0 2024-09-10T11:54:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.804+0000 7fa4baffd700 1 --2- 172.21.15.144:0/2942503375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4bc068630 0x7fa4bc195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:54:46.807 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.804+0000 7fa4b27fc700 1 --2- 172.21.15.144:0/2942503375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4bc068fd0 0x7fa4bc196460 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400b480 tx=0x7fa4a400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:46.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.805+0000 7fa4b8ff9700 1 -- 172.21.15.144:0/2942503375 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4a40118c0 con 0x7fa4bc068fd0 2024-09-10T11:54:46.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.805+0000 7fa4c16c5700 1 -- 172.21.15.144:0/2942503375 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4bc19abb0 con 0x7fa4bc068fd0 2024-09-10T11:54:46.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.805+0000 7fa4b8ff9700 1 -- 172.21.15.144:0/2942503375 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4a4010380 con 0x7fa4bc068fd0 2024-09-10T11:54:46.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.805+0000 7fa4b8ff9700 1 -- 172.21.15.144:0/2942503375 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4a400f4f0 con 0x7fa4bc068fd0 2024-09-10T11:54:46.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.805+0000 7fa4c16c5700 1 -- 172.21.15.144:0/2942503375 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4bc19b100 con 0x7fa4bc068fd0 2024-09-10T11:54:46.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.806+0000 7fa4c16c5700 1 -- 172.21.15.144:0/2942503375 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4bc1086f0 con 0x7fa4bc068fd0 2024-09-10T11:54:46.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.806+0000 7fa4b8ff9700 1 -- 172.21.15.144:0/2942503375 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa4a40104f0 con 0x7fa4bc068fd0 2024-09-10T11:54:46.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.807+0000 7fa4b8ff9700 1 --2- 172.21.15.144:0/2942503375 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa4a806e200 0x7fa4a80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:46.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.807+0000 7fa4b8ff9700 1 -- 172.21.15.144:0/2942503375 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa4a408dce0 con 0x7fa4bc068fd0 2024-09-10T11:54:46.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.807+0000 7fa4baffd700 1 --2- 172.21.15.144:0/2942503375 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa4a806e200 0x7fa4a80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:46.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.807+0000 7fa4baffd700 1 --2- 172.21.15.144:0/2942503375 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa4a806e200 0x7fa4a80706b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00b5c0 tx=0x7fa4ac005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:46.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:46.811+0000 7fa4b8ff9700 1 -- 172.21.15.144:0/2942503375 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa4a40570c0 con 0x7fa4bc068fd0 2024-09-10T11:54:47.041 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.040+0000 7fa4c16c5700 1 -- 172.21.15.144:0/2942503375 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa4bc19b380 con 0x7fa4a806e200 2024-09-10T11:54:47.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.042+0000 7fa4b8ff9700 1 -- 172.21.15.144:0/2942503375 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa4bc19b380 con 0x7fa4a806e200 2024-09-10T11:54:47.043 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:54:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:54:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:54:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:54:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:54:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:54:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T11:54:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T11:54:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:54:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T11:54:47.043 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:54:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T11:54:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:54:47.044 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:54:47.044 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:54:47.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.044+0000 7fa4c16c5700 1 -- 172.21.15.144:0/2942503375 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa4a806e200 msgr2=0x7fa4a80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:47.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.044+0000 7fa4c16c5700 1 --2- 172.21.15.144:0/2942503375 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa4a806e200 0x7fa4a80706b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa4ac00b5c0 tx=0x7fa4ac005480 comp rx=0 tx=0).stop 2024-09-10T11:54:47.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.044+0000 7fa4c16c5700 1 -- 172.21.15.144:0/2942503375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4bc068fd0 msgr2=0x7fa4bc196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:47.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.044+0000 7fa4c16c5700 1 --2- 172.21.15.144:0/2942503375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4bc068fd0 0x7fa4bc196460 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fa4a400b480 tx=0x7fa4a400b840 comp rx=0 tx=0).stop 2024-09-10T11:54:47.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.044+0000 7fa4c16c5700 1 -- 172.21.15.144:0/2942503375 shutdown_connections 2024-09-10T11:54:47.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.044+0000 7fa4c16c5700 1 --2- 172.21.15.144:0/2942503375 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa4a806e200 0x7fa4a80706b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:47.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.044+0000 7fa4c16c5700 1 --2- 172.21.15.144:0/2942503375 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa4bc068630 0x7fa4bc195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:47.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.044+0000 7fa4c16c5700 1 --2- 172.21.15.144:0/2942503375 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa4bc068fd0 0x7fa4bc196460 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:47.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.044+0000 7fa4c16c5700 1 -- 172.21.15.144:0/2942503375 >> 172.21.15.144:0/2942503375 conn(0x7fa4bc074f50 msgr2=0x7fa4bc0fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:47.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.044+0000 7fa4c16c5700 1 -- 172.21.15.144:0/2942503375 shutdown_connections 2024-09-10T11:54:47.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.044+0000 7fa4c16c5700 1 -- 172.21.15.144:0/2942503375 wait complete. 2024-09-10T11:54:47.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.149+0000 7f11a7943700 1 -- 172.21.15.144:0/1860987904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11a0068fd0 msgr2=0x7f11a0104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:47.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.149+0000 7f11a7943700 1 --2- 172.21.15.144:0/1860987904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11a0068fd0 0x7f11a0104df0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f1194009910 tx=0x7f1194009c20 comp rx=0 tx=0).stop 2024-09-10T11:54:47.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.149+0000 7f11a7943700 1 -- 172.21.15.144:0/1860987904 shutdown_connections 2024-09-10T11:54:47.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.149+0000 7f11a7943700 1 --2- 172.21.15.144:0/1860987904 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11a0068fd0 0x7f11a0104df0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:47.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.149+0000 7f11a7943700 1 --2- 172.21.15.144:0/1860987904 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a0068630 0x7f11a0068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:47.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.149+0000 7f11a7943700 1 -- 172.21.15.144:0/1860987904 >> 172.21.15.144:0/1860987904 conn(0x7f11a0074f50 msgr2=0x7f11a0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:47.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.149+0000 7f11a7943700 1 -- 172.21.15.144:0/1860987904 shutdown_connections 2024-09-10T11:54:47.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.149+0000 7f11a7943700 1 -- 172.21.15.144:0/1860987904 wait complete. 2024-09-10T11:54:47.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.150+0000 7f11a7943700 1 Processor -- start 2024-09-10T11:54:47.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.150+0000 7f11a7943700 1 -- start start 2024-09-10T11:54:47.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.150+0000 7f11a7943700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a0068630 0x7f11a0197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:47.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.150+0000 7f11a7943700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11a0068fd0 0x7f11a0198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:47.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.150+0000 7f11a7943700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a0198b40 con 0x7f11a0068fd0 2024-09-10T11:54:47.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.150+0000 7f11a7943700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a019c8d0 con 0x7f11a0068630 2024-09-10T11:54:47.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.150+0000 7f11a4ede700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11a0068fd0 0x7f11a0198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:47.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.150+0000 7f11a4ede700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11a0068fd0 0x7f11a0198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52400/0 (socket says 172.21.15.144:52400) 2024-09-10T11:54:47.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.150+0000 7f11a4ede700 1 -- 172.21.15.144:0/1006954512 learned_addr learned my addr 172.21.15.144:0/1006954512 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:54:47.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.150+0000 7f11a56df700 1 --2- 172.21.15.144:0/1006954512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a0068630 0x7f11a0197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:47.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.150+0000 7f11a4ede700 1 -- 172.21.15.144:0/1006954512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a0068630 msgr2=0x7f11a0197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:47.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.150+0000 7f11a4ede700 1 --2- 172.21.15.144:0/1006954512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a0068630 0x7f11a0197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:47.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.150+0000 7f11a4ede700 1 -- 172.21.15.144:0/1006954512 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11940095f0 con 0x7f11a0068fd0 2024-09-10T11:54:47.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.150+0000 7f11a4ede700 1 --2- 172.21.15.144:0/1006954512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11a0068fd0 0x7f11a0198460 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f11940092e0 tx=0x7f119400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:47.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.150+0000 7f119e7fc700 1 -- 172.21.15.144:0/1006954512 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f119401d070 con 0x7f11a0068fd0 2024-09-10T11:54:47.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.151+0000 7f11a7943700 1 -- 172.21.15.144:0/1006954512 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a019cb50 con 0x7f11a0068fd0 2024-09-10T11:54:47.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.151+0000 7f119e7fc700 1 -- 172.21.15.144:0/1006954512 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1194005dc0 con 0x7f11a0068fd0 2024-09-10T11:54:47.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.151+0000 7f119e7fc700 1 -- 172.21.15.144:0/1006954512 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1194017400 con 0x7f11a0068fd0 2024-09-10T11:54:47.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.151+0000 7f11a7943700 1 -- 172.21.15.144:0/1006954512 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a019d040 con 0x7f11a0068fd0 2024-09-10T11:54:47.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.151+0000 7f11a7943700 1 -- 172.21.15.144:0/1006954512 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a0104570 con 0x7f11a0068fd0 2024-09-10T11:54:47.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.152+0000 7f119e7fc700 1 -- 172.21.15.144:0/1006954512 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f119400f6f0 con 0x7f11a0068fd0 2024-09-10T11:54:47.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.152+0000 7f119e7fc700 1 --2- 172.21.15.144:0/1006954512 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f118806e200 0x7f11880706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:54:47.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.153+0000 7f119e7fc700 1 -- 172.21.15.144:0/1006954512 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f119408f470 con 0x7f11a0068fd0 2024-09-10T11:54:47.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.153+0000 7f11a56df700 1 --2- 172.21.15.144:0/1006954512 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f118806e200 0x7f11880706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:54:47.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.153+0000 7f11a56df700 1 --2- 172.21.15.144:0/1006954512 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f118806e200 0x7f11880706b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1198009550 tx=0x7f11980091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:54:47.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.157+0000 7f119e7fc700 1 -- 172.21.15.144:0/1006954512 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1194058850 con 0x7f11a0068fd0 2024-09-10T11:54:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:46 smithi178 ceph-mon[65940]: from='client.35276 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:46 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/195300563' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:54:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:46 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/4096093198' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:54:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:46 smithi178 ceph-mon[65940]: pgmap v685: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:46 smithi144 ceph-mon[87436]: from='client.35276 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:46 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/195300563' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:54:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:46 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/4096093198' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:54:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:46 smithi144 ceph-mon[87436]: pgmap v685: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:47.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.442+0000 7f11a7943700 1 -- 172.21.15.144:0/1006954512 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f11a019d320 con 0x7f11a0068fd0 2024-09-10T11:54:47.444 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.443+0000 7f119e7fc700 1 -- 172.21.15.144:0/1006954512 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f119405bf10 con 0x7f11a0068fd0 2024-09-10T11:54:47.444 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:54:47.444 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T11:54:47.444 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:54:47.444 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T11:54:47.444 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:54:47.444 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T11:54:47.444 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T11:54:47.444 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T11:54:47.445 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T11:54:47.445 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T11:54:47.445 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T11:54:47.445 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T11:54:47.445 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T11:54:47.445 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T11:54:47.445 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T11:54:47.445 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T11:54:47.445 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T11:54:47.445 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T11:54:47.445 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:54:47.445 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:54:47.445 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:54:47.446 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:54:47.446 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:54:47.446 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:54:47.446 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:54:47.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.445+0000 7f11a7943700 1 -- 172.21.15.144:0/1006954512 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f118806e200 msgr2=0x7f11880706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:47.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.445+0000 7f11a7943700 1 --2- 172.21.15.144:0/1006954512 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f118806e200 0x7f11880706b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1198009550 tx=0x7f11980091b0 comp rx=0 tx=0).stop 2024-09-10T11:54:47.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.445+0000 7f11a7943700 1 -- 172.21.15.144:0/1006954512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11a0068fd0 msgr2=0x7f11a0198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:54:47.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.445+0000 7f11a7943700 1 --2- 172.21.15.144:0/1006954512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11a0068fd0 0x7f11a0198460 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f11940092e0 tx=0x7f119400bd70 comp rx=0 tx=0).stop 2024-09-10T11:54:47.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.445+0000 7f11a7943700 1 -- 172.21.15.144:0/1006954512 shutdown_connections 2024-09-10T11:54:47.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.445+0000 7f11a7943700 1 --2- 172.21.15.144:0/1006954512 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f118806e200 0x7f11880706b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:47.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.445+0000 7f11a7943700 1 --2- 172.21.15.144:0/1006954512 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11a0068630 0x7f11a0197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:47.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.445+0000 7f11a7943700 1 --2- 172.21.15.144:0/1006954512 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11a0068fd0 0x7f11a0198460 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:54:47.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.445+0000 7f11a7943700 1 -- 172.21.15.144:0/1006954512 >> 172.21.15.144:0/1006954512 conn(0x7f11a0074f50 msgr2=0x7f11a00fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:54:47.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.445+0000 7f11a7943700 1 -- 172.21.15.144:0/1006954512 shutdown_connections 2024-09-10T11:54:47.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:54:47.445+0000 7f11a7943700 1 -- 172.21.15.144:0/1006954512 wait complete. 2024-09-10T11:54:48.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:47 smithi178 ceph-mon[65940]: from='client.35288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:48.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:47 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1006954512' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:54:48.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:47 smithi144 ceph-mon[87436]: from='client.35288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:54:48.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:47 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1006954512' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:54:49.230 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:48 smithi144 ceph-mon[87436]: pgmap v686: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:48 smithi178 ceph-mon[65940]: pgmap v686: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:50 smithi178 ceph-mon[65940]: pgmap v687: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:51.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:50 smithi144 ceph-mon[87436]: pgmap v687: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:52 smithi178 ceph-mon[65940]: pgmap v688: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:52 smithi144 ceph-mon[87436]: pgmap v688: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:54 smithi178 ceph-mon[65940]: pgmap v689: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:54 smithi144 ceph-mon[87436]: pgmap v689: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:56 smithi178 ceph-mon[65940]: pgmap v690: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:57.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:56 smithi144 ceph-mon[87436]: pgmap v690: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:54:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:54:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:54:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:54:59 smithi144 ceph-mon[87436]: pgmap v691: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:00.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:54:59 smithi178 ceph-mon[65940]: pgmap v691: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:00 smithi178 ceph-mon[65940]: pgmap v692: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:01.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:00 smithi144 ceph-mon[87436]: pgmap v692: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:03.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:03 smithi178 ceph-mon[65940]: pgmap v693: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:03.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:03 smithi144 ceph-mon[87436]: pgmap v693: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:04 smithi178 ceph-mon[65940]: pgmap v694: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:05.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:04 smithi144 ceph-mon[87436]: pgmap v694: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:06 smithi178 ceph-mon[65940]: pgmap v695: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:07.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:06 smithi144 ceph-mon[87436]: pgmap v695: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:08 smithi178 ceph-mon[65940]: pgmap v696: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:09.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:08 smithi144 ceph-mon[87436]: pgmap v696: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:10 smithi178 ceph-mon[65940]: pgmap v697: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:11.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:10 smithi144 ceph-mon[87436]: pgmap v697: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:13.236 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:12 smithi144 ceph-mon[87436]: pgmap v698: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:12 smithi178 ceph-mon[65940]: pgmap v698: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:55:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:55:15.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:14 smithi178 ceph-mon[65940]: pgmap v699: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:15.439 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:14 smithi144 ceph-mon[87436]: pgmap v699: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:16 smithi178 ceph-mon[65940]: pgmap v700: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:17.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:16 smithi144 ceph-mon[87436]: pgmap v700: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:17.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.563+0000 7f8b84c0f700 1 -- 172.21.15.144:0/1119248040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b800ffc70 msgr2=0x7f8b80103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:17.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.563+0000 7f8b84c0f700 1 --2- 172.21.15.144:0/1119248040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b800ffc70 0x7f8b80103c40 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f8b74009910 tx=0x7f8b74009c20 comp rx=0 tx=0).stop 2024-09-10T11:55:17.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.564+0000 7f8b84c0f700 1 -- 172.21.15.144:0/1119248040 shutdown_connections 2024-09-10T11:55:17.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.564+0000 7f8b84c0f700 1 --2- 172.21.15.144:0/1119248040 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b800ffc70 0x7f8b80103c40 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:17.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.564+0000 7f8b84c0f700 1 --2- 172.21.15.144:0/1119248040 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b800ff2d0 0x7f8b800ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:17.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.564+0000 7f8b84c0f700 1 -- 172.21.15.144:0/1119248040 >> 172.21.15.144:0/1119248040 conn(0x7f8b800fabb0 msgr2=0x7f8b800fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:17.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.564+0000 7f8b84c0f700 1 -- 172.21.15.144:0/1119248040 shutdown_connections 2024-09-10T11:55:17.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.564+0000 7f8b84c0f700 1 -- 172.21.15.144:0/1119248040 wait complete. 2024-09-10T11:55:17.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.564+0000 7f8b84c0f700 1 Processor -- start 2024-09-10T11:55:17.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.565+0000 7f8b84c0f700 1 -- start start 2024-09-10T11:55:17.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.565+0000 7f8b84c0f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b800ff2d0 0x7f8b80197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:17.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.565+0000 7f8b84c0f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b800ffc70 0x7f8b80198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:17.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.565+0000 7f8b84c0f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b80198c10 con 0x7f8b800ff2d0 2024-09-10T11:55:17.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.565+0000 7f8b84c0f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b8019c9a0 con 0x7f8b800ffc70 2024-09-10T11:55:17.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.565+0000 7f8b7e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b800ff2d0 0x7f8b80197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:17.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.565+0000 7f8b7e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b800ff2d0 0x7f8b80197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58218/0 (socket says 172.21.15.144:58218) 2024-09-10T11:55:17.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.565+0000 7f8b7e59c700 1 -- 172.21.15.144:0/2319092374 learned_addr learned my addr 172.21.15.144:0/2319092374 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:55:17.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.565+0000 7f8b7dd9b700 1 --2- 172.21.15.144:0/2319092374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b800ffc70 0x7f8b80198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:17.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.565+0000 7f8b7e59c700 1 -- 172.21.15.144:0/2319092374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b800ffc70 msgr2=0x7f8b80198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:17.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.565+0000 7f8b7e59c700 1 --2- 172.21.15.144:0/2319092374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b800ffc70 0x7f8b80198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:17.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.565+0000 7f8b7e59c700 1 -- 172.21.15.144:0/2319092374 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b740095f0 con 0x7f8b800ff2d0 2024-09-10T11:55:17.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.565+0000 7f8b7e59c700 1 --2- 172.21.15.144:0/2319092374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b800ff2d0 0x7f8b80197ff0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8b7000e920 tx=0x7f8b7000ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:17.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.565+0000 7f8b6f7fe700 1 -- 172.21.15.144:0/2319092374 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b70009790 con 0x7f8b800ff2d0 2024-09-10T11:55:17.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.566+0000 7f8b6f7fe700 1 -- 172.21.15.144:0/2319092374 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8b7000cba0 con 0x7f8b800ff2d0 2024-09-10T11:55:17.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.566+0000 7f8b84c0f700 1 -- 172.21.15.144:0/2319092374 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8019cc80 con 0x7f8b800ff2d0 2024-09-10T11:55:17.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.566+0000 7f8b6f7fe700 1 -- 172.21.15.144:0/2319092374 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b70018650 con 0x7f8b800ff2d0 2024-09-10T11:55:17.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.566+0000 7f8b84c0f700 1 -- 172.21.15.144:0/2319092374 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b8019d1d0 con 0x7f8b800ff2d0 2024-09-10T11:55:17.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.567+0000 7f8b84c0f700 1 -- 172.21.15.144:0/2319092374 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8004ea50 con 0x7f8b800ff2d0 2024-09-10T11:55:17.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.567+0000 7f8b6f7fe700 1 -- 172.21.15.144:0/2319092374 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8b70010430 con 0x7f8b800ff2d0 2024-09-10T11:55:17.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.568+0000 7f8b6f7fe700 1 --2- 172.21.15.144:0/2319092374 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8b6806e200 0x7f8b680706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:17.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.568+0000 7f8b7dd9b700 1 --2- 172.21.15.144:0/2319092374 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8b6806e200 0x7f8b680706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:17.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.568+0000 7f8b6f7fe700 1 -- 172.21.15.144:0/2319092374 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8b70014070 con 0x7f8b800ff2d0 2024-09-10T11:55:17.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.568+0000 7f8b7dd9b700 1 --2- 172.21.15.144:0/2319092374 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8b6806e200 0x7f8b680706b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8b80199610 tx=0x7f8b74009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:17.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.572+0000 7f8b6f7fe700 1 -- 172.21.15.144:0/2319092374 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8b700541b0 con 0x7f8b800ff2d0 2024-09-10T11:55:17.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.794+0000 7f8b84c0f700 1 -- 172.21.15.144:0/2319092374 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b8019d660 con 0x7f8b6806e200 2024-09-10T11:55:17.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.795+0000 7f8b6f7fe700 1 -- 172.21.15.144:0/2319092374 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8b8019d660 con 0x7f8b6806e200 2024-09-10T11:55:17.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.798+0000 7f8b84c0f700 1 -- 172.21.15.144:0/2319092374 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8b6806e200 msgr2=0x7f8b680706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:17.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.798+0000 7f8b84c0f700 1 --2- 172.21.15.144:0/2319092374 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8b6806e200 0x7f8b680706b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f8b80199610 tx=0x7f8b74009f70 comp rx=0 tx=0).stop 2024-09-10T11:55:17.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.798+0000 7f8b84c0f700 1 -- 172.21.15.144:0/2319092374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b800ff2d0 msgr2=0x7f8b80197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:17.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.798+0000 7f8b84c0f700 1 --2- 172.21.15.144:0/2319092374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b800ff2d0 0x7f8b80197ff0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8b7000e920 tx=0x7f8b7000ece0 comp rx=0 tx=0).stop 2024-09-10T11:55:17.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.798+0000 7f8b84c0f700 1 -- 172.21.15.144:0/2319092374 shutdown_connections 2024-09-10T11:55:17.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.798+0000 7f8b84c0f700 1 --2- 172.21.15.144:0/2319092374 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8b6806e200 0x7f8b680706b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:17.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.798+0000 7f8b84c0f700 1 --2- 172.21.15.144:0/2319092374 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8b800ff2d0 0x7f8b80197ff0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:17.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.798+0000 7f8b84c0f700 1 --2- 172.21.15.144:0/2319092374 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8b800ffc70 0x7f8b80198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:17.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.798+0000 7f8b84c0f700 1 -- 172.21.15.144:0/2319092374 >> 172.21.15.144:0/2319092374 conn(0x7f8b800fabb0 msgr2=0x7f8b80102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:17.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.798+0000 7f8b84c0f700 1 -- 172.21.15.144:0/2319092374 shutdown_connections 2024-09-10T11:55:17.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.798+0000 7f8b84c0f700 1 -- 172.21.15.144:0/2319092374 wait complete. 2024-09-10T11:55:17.809 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:55:17.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.904+0000 7fb45efda700 1 -- 172.21.15.144:0/4116213491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4580ffc60 msgr2=0x7fb458103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:17.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.904+0000 7fb45efda700 1 --2- 172.21.15.144:0/4116213491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4580ffc60 0x7fb458103c30 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fb448009910 tx=0x7fb448009c20 comp rx=0 tx=0).stop 2024-09-10T11:55:17.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.904+0000 7fb45efda700 1 -- 172.21.15.144:0/4116213491 shutdown_connections 2024-09-10T11:55:17.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.904+0000 7fb45efda700 1 --2- 172.21.15.144:0/4116213491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4580ffc60 0x7fb458103c30 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:17.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.904+0000 7fb45efda700 1 --2- 172.21.15.144:0/4116213491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4580ff2c0 0x7fb4580ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:17.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.904+0000 7fb45efda700 1 -- 172.21.15.144:0/4116213491 >> 172.21.15.144:0/4116213491 conn(0x7fb4580fab80 msgr2=0x7fb4580fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:17.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.904+0000 7fb45efda700 1 -- 172.21.15.144:0/4116213491 shutdown_connections 2024-09-10T11:55:17.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.904+0000 7fb45efda700 1 -- 172.21.15.144:0/4116213491 wait complete. 2024-09-10T11:55:17.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.905+0000 7fb45efda700 1 Processor -- start 2024-09-10T11:55:17.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.905+0000 7fb45efda700 1 -- start start 2024-09-10T11:55:17.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.905+0000 7fb45efda700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4580ff2c0 0x7fb45819aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:17.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.905+0000 7fb45efda700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4580ffc60 0x7fb45819af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:17.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.905+0000 7fb45efda700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb45819b5e0 con 0x7fb4580ffc60 2024-09-10T11:55:17.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.905+0000 7fb45efda700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb458194a90 con 0x7fb4580ff2c0 2024-09-10T11:55:17.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.905+0000 7fb44ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4580ffc60 0x7fb45819af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:17.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.905+0000 7fb44ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4580ffc60 0x7fb45819af50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58232/0 (socket says 172.21.15.144:58232) 2024-09-10T11:55:17.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.905+0000 7fb44ffff700 1 -- 172.21.15.144:0/3736685018 learned_addr learned my addr 172.21.15.144:0/3736685018 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:55:17.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.905+0000 7fb45cd76700 1 --2- 172.21.15.144:0/3736685018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4580ff2c0 0x7fb45819aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:17.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.906+0000 7fb44ffff700 1 -- 172.21.15.144:0/3736685018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4580ff2c0 msgr2=0x7fb45819aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:17.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.906+0000 7fb44ffff700 1 --2- 172.21.15.144:0/3736685018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4580ff2c0 0x7fb45819aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:17.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.906+0000 7fb44ffff700 1 -- 172.21.15.144:0/3736685018 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4480095f0 con 0x7fb4580ffc60 2024-09-10T11:55:17.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.906+0000 7fb44ffff700 1 --2- 172.21.15.144:0/3736685018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4580ffc60 0x7fb45819af50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb4480092e0 tx=0x7fb44800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:17.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.906+0000 7fb44dffb700 1 -- 172.21.15.144:0/3736685018 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb44801d070 con 0x7fb4580ffc60 2024-09-10T11:55:17.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.906+0000 7fb44dffb700 1 -- 172.21.15.144:0/3736685018 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb44800f460 con 0x7fb4580ffc60 2024-09-10T11:55:17.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.906+0000 7fb44dffb700 1 -- 172.21.15.144:0/3736685018 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4480175f0 con 0x7fb4580ffc60 2024-09-10T11:55:17.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.906+0000 7fb45efda700 1 -- 172.21.15.144:0/3736685018 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb458194d10 con 0x7fb4580ffc60 2024-09-10T11:55:17.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.906+0000 7fb45efda700 1 -- 172.21.15.144:0/3736685018 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb458195140 con 0x7fb4580ffc60 2024-09-10T11:55:17.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.907+0000 7fb45efda700 1 -- 172.21.15.144:0/3736685018 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb458107540 con 0x7fb4580ffc60 2024-09-10T11:55:17.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.908+0000 7fb44dffb700 1 -- 172.21.15.144:0/3736685018 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb44800f5d0 con 0x7fb4580ffc60 2024-09-10T11:55:17.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.908+0000 7fb44dffb700 1 --2- 172.21.15.144:0/3736685018 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb44006e200 0x7fb4400706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:17.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.908+0000 7fb44dffb700 1 -- 172.21.15.144:0/3736685018 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb44808f4e0 con 0x7fb4580ffc60 2024-09-10T11:55:17.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.908+0000 7fb45cd76700 1 --2- 172.21.15.144:0/3736685018 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb44006e200 0x7fb4400706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:17.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.909+0000 7fb45cd76700 1 --2- 172.21.15.144:0/3736685018 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb44006e200 0x7fb4400706b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb4540060b0 tx=0x7fb454007790 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:17.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:17.913+0000 7fb44dffb700 1 -- 172.21.15.144:0/3736685018 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb44805c470 con 0x7fb4580ffc60 2024-09-10T11:55:18.144 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.143+0000 7fb45efda700 1 -- 172.21.15.144:0/3736685018 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb45802ccf0 con 0x7fb44006e200 2024-09-10T11:55:18.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.144+0000 7fb44dffb700 1 -- 172.21.15.144:0/3736685018 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb45802ccf0 con 0x7fb44006e200 2024-09-10T11:55:18.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.146+0000 7fb45efda700 1 -- 172.21.15.144:0/3736685018 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb44006e200 msgr2=0x7fb4400706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:18.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.146+0000 7fb45efda700 1 --2- 172.21.15.144:0/3736685018 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb44006e200 0x7fb4400706b0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fb4540060b0 tx=0x7fb454007790 comp rx=0 tx=0).stop 2024-09-10T11:55:18.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.146+0000 7fb45efda700 1 -- 172.21.15.144:0/3736685018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4580ffc60 msgr2=0x7fb45819af50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:18.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.146+0000 7fb45efda700 1 --2- 172.21.15.144:0/3736685018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4580ffc60 0x7fb45819af50 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb4480092e0 tx=0x7fb44800be20 comp rx=0 tx=0).stop 2024-09-10T11:55:18.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.146+0000 7fb45efda700 1 -- 172.21.15.144:0/3736685018 shutdown_connections 2024-09-10T11:55:18.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.147+0000 7fb45efda700 1 --2- 172.21.15.144:0/3736685018 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb44006e200 0x7fb4400706b0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.147+0000 7fb45efda700 1 --2- 172.21.15.144:0/3736685018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb4580ff2c0 0x7fb45819aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.147+0000 7fb45efda700 1 --2- 172.21.15.144:0/3736685018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb4580ffc60 0x7fb45819af50 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.147+0000 7fb45efda700 1 -- 172.21.15.144:0/3736685018 >> 172.21.15.144:0/3736685018 conn(0x7fb4580fab80 msgr2=0x7fb4581034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:18.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.147+0000 7fb45efda700 1 -- 172.21.15.144:0/3736685018 shutdown_connections 2024-09-10T11:55:18.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.147+0000 7fb45efda700 1 -- 172.21.15.144:0/3736685018 wait complete. 2024-09-10T11:55:18.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:17 smithi144 ceph-mon[87436]: from='client.35296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:18.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.254+0000 7f27351ca700 1 -- 172.21.15.144:0/2247026487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730101380 msgr2=0x7f2730101750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:18.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.254+0000 7f27351ca700 1 --2- 172.21.15.144:0/2247026487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730101380 0x7f2730101750 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f272800b0a0 tx=0x7f272800b3b0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.254+0000 7f27351ca700 1 -- 172.21.15.144:0/2247026487 shutdown_connections 2024-09-10T11:55:18.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.254+0000 7f27351ca700 1 --2- 172.21.15.144:0/2247026487 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2730101c90 0x7f273010a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.254+0000 7f27351ca700 1 --2- 172.21.15.144:0/2247026487 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730101380 0x7f2730101750 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.254+0000 7f27351ca700 1 -- 172.21.15.144:0/2247026487 >> 172.21.15.144:0/2247026487 conn(0x7f27300fab80 msgr2=0x7f27300fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:18.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.254+0000 7f27351ca700 1 -- 172.21.15.144:0/2247026487 shutdown_connections 2024-09-10T11:55:18.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.254+0000 7f27351ca700 1 -- 172.21.15.144:0/2247026487 wait complete. 2024-09-10T11:55:18.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.254+0000 7f27351ca700 1 Processor -- start 2024-09-10T11:55:18.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.255+0000 7f27351ca700 1 -- start start 2024-09-10T11:55:18.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.255+0000 7f27351ca700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2730101380 0x7f2730195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:18.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.255+0000 7f27351ca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730101c90 0x7f2730196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:18.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.255+0000 7f27351ca700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27301968b0 con 0x7f2730101c90 2024-09-10T11:55:18.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.255+0000 7f27351ca700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27301969f0 con 0x7f2730101380 2024-09-10T11:55:18.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.255+0000 7f272e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730101c90 0x7f2730196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:18.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.255+0000 7f272e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730101c90 0x7f2730196260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58264/0 (socket says 172.21.15.144:58264) 2024-09-10T11:55:18.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.255+0000 7f272e59c700 1 -- 172.21.15.144:0/909658355 learned_addr learned my addr 172.21.15.144:0/909658355 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:55:18.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.255+0000 7f272ed9d700 1 --2- 172.21.15.144:0/909658355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2730101380 0x7f2730195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:18.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.255+0000 7f272e59c700 1 -- 172.21.15.144:0/909658355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2730101380 msgr2=0x7f2730195d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:18.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.255+0000 7f272e59c700 1 --2- 172.21.15.144:0/909658355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2730101380 0x7f2730195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.255+0000 7f272e59c700 1 -- 172.21.15.144:0/909658355 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2728009d00 con 0x7f2730101c90 2024-09-10T11:55:18.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.255+0000 7f272e59c700 1 --2- 172.21.15.144:0/909658355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730101c90 0x7f2730196260 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f271800c930 tx=0x7f271800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:18.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.256+0000 7f2727fff700 1 -- 172.21.15.144:0/909658355 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2718007ad0 con 0x7f2730101c90 2024-09-10T11:55:18.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.256+0000 7f2727fff700 1 -- 172.21.15.144:0/909658355 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2718007c30 con 0x7f2730101c90 2024-09-10T11:55:18.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.256+0000 7f2727fff700 1 -- 172.21.15.144:0/909658355 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2718018610 con 0x7f2730101c90 2024-09-10T11:55:18.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.256+0000 7f27351ca700 1 -- 172.21.15.144:0/909658355 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273019a840 con 0x7f2730101c90 2024-09-10T11:55:18.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.256+0000 7f27351ca700 1 -- 172.21.15.144:0/909658355 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273019ad90 con 0x7f2730101c90 2024-09-10T11:55:18.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.257+0000 7f27351ca700 1 -- 172.21.15.144:0/909658355 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2730107860 con 0x7f2730101c90 2024-09-10T11:55:18.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.257+0000 7f2727fff700 1 -- 172.21.15.144:0/909658355 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f271800f450 con 0x7f2730101c90 2024-09-10T11:55:18.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.258+0000 7f2727fff700 1 --2- 172.21.15.144:0/909658355 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f271c06e200 0x7f271c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:18.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.258+0000 7f2727fff700 1 -- 172.21.15.144:0/909658355 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f271808e1a0 con 0x7f2730101c90 2024-09-10T11:55:18.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.258+0000 7f272ed9d700 1 --2- 172.21.15.144:0/909658355 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f271c06e200 0x7f271c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:18.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.258+0000 7f272ed9d700 1 --2- 172.21.15.144:0/909658355 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f271c06e200 0x7f271c0706b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2728015040 tx=0x7f2728006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:18.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.262+0000 7f2727fff700 1 -- 172.21.15.144:0/909658355 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2718057600 con 0x7f2730101c90 2024-09-10T11:55:18.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:17 smithi178 ceph-mon[65940]: from='client.35296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:18.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.473+0000 7f27351ca700 1 -- 172.21.15.144:0/909658355 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f273002cd20 con 0x7f271c06e200 2024-09-10T11:55:18.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.482+0000 7f2727fff700 1 -- 172.21.15.144:0/909658355 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f273002cd20 con 0x7f271c06e200 2024-09-10T11:55:18.483 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:55:18.484 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (23m) 110s ago 34m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:55:18.484 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (34m) 110s ago 34m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:55:18.484 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (33m) 112s ago 33m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:55:18.484 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (21m) 110s ago 34m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:55:18.484 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (21m) 112s ago 33m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:55:18.484 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (23m) 110s ago 33m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:55:18.485 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (17m) 110s ago 28m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:55:18.485 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (17m) 110s ago 28m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:55:18.485 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (116s) 112s ago 28m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T11:55:18.485 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (17m) 112s ago 28m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:55:18.485 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (25m) 110s ago 35m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:55:18.485 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (24m) 112s ago 33m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:55:18.485 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (22m) 110s ago 35m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:55:18.486 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (21m) 112s ago 33m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:55:18.486 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (24m) 110s ago 34m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:55:18.486 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (24m) 112s ago 33m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:55:18.486 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (20m) 110s ago 32m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:55:18.486 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (20m) 110s ago 31m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:55:18.486 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (19m) 110s ago 31m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:55:18.486 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (18m) 112s ago 30m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:55:18.486 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (18m) 112s ago 30m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:55:18.487 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (17m) 112s ago 30m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:55:18.487 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (23m) 110s ago 33m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:55:18.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.484+0000 7f27351ca700 1 -- 172.21.15.144:0/909658355 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f271c06e200 msgr2=0x7f271c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:18.488 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.484+0000 7f27351ca700 1 --2- 172.21.15.144:0/909658355 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f271c06e200 0x7f271c0706b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2728015040 tx=0x7f2728006000 comp rx=0 tx=0).stop 2024-09-10T11:55:18.489 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.484+0000 7f27351ca700 1 -- 172.21.15.144:0/909658355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730101c90 msgr2=0x7f2730196260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:18.489 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.484+0000 7f27351ca700 1 --2- 172.21.15.144:0/909658355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730101c90 0x7f2730196260 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f271800c930 tx=0x7f271800ccf0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.489 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.484+0000 7f27351ca700 1 -- 172.21.15.144:0/909658355 shutdown_connections 2024-09-10T11:55:18.489 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.484+0000 7f27351ca700 1 --2- 172.21.15.144:0/909658355 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f271c06e200 0x7f271c0706b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.484+0000 7f27351ca700 1 --2- 172.21.15.144:0/909658355 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2730101380 0x7f2730195d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.484+0000 7f27351ca700 1 --2- 172.21.15.144:0/909658355 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2730101c90 0x7f2730196260 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.484+0000 7f27351ca700 1 -- 172.21.15.144:0/909658355 >> 172.21.15.144:0/909658355 conn(0x7f27300fab80 msgr2=0x7f27301049a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:18.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.484+0000 7f27351ca700 1 -- 172.21.15.144:0/909658355 shutdown_connections 2024-09-10T11:55:18.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.485+0000 7f27351ca700 1 -- 172.21.15.144:0/909658355 wait complete. 2024-09-10T11:55:18.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.588+0000 7f6b51a0c700 1 -- 172.21.15.144:0/1396012874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c101c90 msgr2=0x7f6b4c10a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:18.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.588+0000 7f6b51a0c700 1 --2- 172.21.15.144:0/1396012874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c101c90 0x7f6b4c10a190 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6b34009910 tx=0x7f6b34009c20 comp rx=0 tx=0).stop 2024-09-10T11:55:18.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.588+0000 7f6b51a0c700 1 -- 172.21.15.144:0/1396012874 shutdown_connections 2024-09-10T11:55:18.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.588+0000 7f6b51a0c700 1 --2- 172.21.15.144:0/1396012874 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c101c90 0x7f6b4c10a190 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.588+0000 7f6b51a0c700 1 --2- 172.21.15.144:0/1396012874 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b4c101380 0x7f6b4c101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.588+0000 7f6b51a0c700 1 -- 172.21.15.144:0/1396012874 >> 172.21.15.144:0/1396012874 conn(0x7f6b4c0fab80 msgr2=0x7f6b4c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:18.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.588+0000 7f6b51a0c700 1 -- 172.21.15.144:0/1396012874 shutdown_connections 2024-09-10T11:55:18.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.588+0000 7f6b51a0c700 1 -- 172.21.15.144:0/1396012874 wait complete. 2024-09-10T11:55:18.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.589+0000 7f6b51a0c700 1 Processor -- start 2024-09-10T11:55:18.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.589+0000 7f6b51a0c700 1 -- start start 2024-09-10T11:55:18.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.589+0000 7f6b51a0c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b4c101380 0x7f6b4c195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:18.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.589+0000 7f6b51a0c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c101c90 0x7f6b4c1961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:18.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.589+0000 7f6b51a0c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c196870 con 0x7f6b4c101c90 2024-09-10T11:55:18.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.589+0000 7f6b51a0c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b4c1995a0 con 0x7f6b4c101380 2024-09-10T11:55:18.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.589+0000 7f6b43fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c101c90 0x7f6b4c1961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:18.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.589+0000 7f6b43fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c101c90 0x7f6b4c1961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58294/0 (socket says 172.21.15.144:58294) 2024-09-10T11:55:18.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.589+0000 7f6b43fff700 1 -- 172.21.15.144:0/242890047 learned_addr learned my addr 172.21.15.144:0/242890047 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:55:18.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.589+0000 7f6b4affd700 1 --2- 172.21.15.144:0/242890047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b4c101380 0x7f6b4c195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:18.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.590+0000 7f6b43fff700 1 -- 172.21.15.144:0/242890047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b4c101380 msgr2=0x7f6b4c195ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:18.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.590+0000 7f6b43fff700 1 --2- 172.21.15.144:0/242890047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b4c101380 0x7f6b4c195ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.590+0000 7f6b43fff700 1 -- 172.21.15.144:0/242890047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b340095f0 con 0x7f6b4c101c90 2024-09-10T11:55:18.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.590+0000 7f6b43fff700 1 --2- 172.21.15.144:0/242890047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c101c90 0x7f6b4c1961e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6b34005f50 tx=0x7f6b3400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:18.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.590+0000 7f6b50a0a700 1 -- 172.21.15.144:0/242890047 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b3401d070 con 0x7f6b4c101c90 2024-09-10T11:55:18.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.590+0000 7f6b50a0a700 1 -- 172.21.15.144:0/242890047 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b3400b610 con 0x7f6b4c101c90 2024-09-10T11:55:18.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.590+0000 7f6b50a0a700 1 -- 172.21.15.144:0/242890047 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b34022470 con 0x7f6b4c101c90 2024-09-10T11:55:18.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.590+0000 7f6b51a0c700 1 -- 172.21.15.144:0/242890047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b4c199820 con 0x7f6b4c101c90 2024-09-10T11:55:18.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.590+0000 7f6b51a0c700 1 -- 172.21.15.144:0/242890047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b4c199d10 con 0x7f6b4c101c90 2024-09-10T11:55:18.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.591+0000 7f6b51a0c700 1 -- 172.21.15.144:0/242890047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b4c107800 con 0x7f6b4c101c90 2024-09-10T11:55:18.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.592+0000 7f6b50a0a700 1 -- 172.21.15.144:0/242890047 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6b34003780 con 0x7f6b4c101c90 2024-09-10T11:55:18.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.592+0000 7f6b50a0a700 1 --2- 172.21.15.144:0/242890047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6b3806e200 0x7f6b380706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:18.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.592+0000 7f6b50a0a700 1 -- 172.21.15.144:0/242890047 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6b3408f500 con 0x7f6b4c101c90 2024-09-10T11:55:18.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.592+0000 7f6b4affd700 1 --2- 172.21.15.144:0/242890047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6b3806e200 0x7f6b380706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:18.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.593+0000 7f6b4affd700 1 --2- 172.21.15.144:0/242890047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6b3806e200 0x7f6b380706b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009c60 tx=0x7f6b3c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:18.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.599+0000 7f6b50a0a700 1 -- 172.21.15.144:0/242890047 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6b34058840 con 0x7f6b4c101c90 2024-09-10T11:55:18.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.886+0000 7f6b51a0c700 1 -- 172.21.15.144:0/242890047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6b4c04ea50 con 0x7f6b4c101c90 2024-09-10T11:55:18.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.886+0000 7f6b50a0a700 1 -- 172.21.15.144:0/242890047 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f6b3405bf00 con 0x7f6b4c101c90 2024-09-10T11:55:18.888 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:55:18.888 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:55:18.888 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:55:18.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:55:18.888 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:55:18.888 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:55:18.888 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:55:18.888 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:55:18.888 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:55:18.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:55:18.889 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:55:18.889 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:55:18.889 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:55:18.889 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:55:18.889 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T11:55:18.889 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:55:18.889 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:55:18.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.888+0000 7f6b51a0c700 1 -- 172.21.15.144:0/242890047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6b3806e200 msgr2=0x7f6b380706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:18.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.888+0000 7f6b51a0c700 1 --2- 172.21.15.144:0/242890047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6b3806e200 0x7f6b380706b0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f6b3c009c60 tx=0x7f6b3c009260 comp rx=0 tx=0).stop 2024-09-10T11:55:18.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.888+0000 7f6b51a0c700 1 -- 172.21.15.144:0/242890047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c101c90 msgr2=0x7f6b4c1961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:18.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.888+0000 7f6b51a0c700 1 --2- 172.21.15.144:0/242890047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c101c90 0x7f6b4c1961e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6b34005f50 tx=0x7f6b3400f690 comp rx=0 tx=0).stop 2024-09-10T11:55:18.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.888+0000 7f6b51a0c700 1 -- 172.21.15.144:0/242890047 shutdown_connections 2024-09-10T11:55:18.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.888+0000 7f6b51a0c700 1 --2- 172.21.15.144:0/242890047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6b3806e200 0x7f6b380706b0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.888+0000 7f6b51a0c700 1 --2- 172.21.15.144:0/242890047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6b4c101380 0x7f6b4c195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.888+0000 7f6b51a0c700 1 --2- 172.21.15.144:0/242890047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6b4c101c90 0x7f6b4c1961e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.888+0000 7f6b51a0c700 1 -- 172.21.15.144:0/242890047 >> 172.21.15.144:0/242890047 conn(0x7f6b4c0fab80 msgr2=0x7f6b4c104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:18.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.889+0000 7f6b51a0c700 1 -- 172.21.15.144:0/242890047 shutdown_connections 2024-09-10T11:55:18.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.889+0000 7f6b51a0c700 1 -- 172.21.15.144:0/242890047 wait complete. 2024-09-10T11:55:18.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.993+0000 7f41dc3a1700 1 -- 172.21.15.144:0/1406285818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d40ff310 msgr2=0x7f41d40ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:18.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.993+0000 7f41dc3a1700 1 --2- 172.21.15.144:0/1406285818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d40ff310 0x7f41d40ff6e0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f41d0009910 tx=0x7f41d0009c20 comp rx=0 tx=0).stop 2024-09-10T11:55:18.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.994+0000 7f41dc3a1700 1 -- 172.21.15.144:0/1406285818 shutdown_connections 2024-09-10T11:55:18.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.994+0000 7f41dc3a1700 1 --2- 172.21.15.144:0/1406285818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41d40ffcb0 0x7f41d4103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.994+0000 7f41dc3a1700 1 --2- 172.21.15.144:0/1406285818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d40ff310 0x7f41d40ff6e0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.994+0000 7f41dc3a1700 1 -- 172.21.15.144:0/1406285818 >> 172.21.15.144:0/1406285818 conn(0x7f41d40fabd0 msgr2=0x7f41d40fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:18.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.994+0000 7f41dc3a1700 1 -- 172.21.15.144:0/1406285818 shutdown_connections 2024-09-10T11:55:18.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.994+0000 7f41dc3a1700 1 -- 172.21.15.144:0/1406285818 wait complete. 2024-09-10T11:55:18.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.994+0000 7f41dc3a1700 1 Processor -- start 2024-09-10T11:55:18.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.994+0000 7f41dc3a1700 1 -- start start 2024-09-10T11:55:18.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.994+0000 7f41dc3a1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d40ff310 0x7f41d4194d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:18.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.994+0000 7f41dc3a1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41d40ffcb0 0x7f41d4195250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:18.997 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.994+0000 7f41dc3a1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d4195930 con 0x7f41d40ff310 2024-09-10T11:55:18.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.994+0000 7f41dc3a1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d41996c0 con 0x7f41d40ffcb0 2024-09-10T11:55:18.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.995+0000 7f41da13d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d40ff310 0x7f41d4194d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:18.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.995+0000 7f41da13d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d40ff310 0x7f41d4194d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58306/0 (socket says 172.21.15.144:58306) 2024-09-10T11:55:18.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.995+0000 7f41da13d700 1 -- 172.21.15.144:0/160124822 learned_addr learned my addr 172.21.15.144:0/160124822 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:55:18.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.995+0000 7f41d993c700 1 --2- 172.21.15.144:0/160124822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41d40ffcb0 0x7f41d4195250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:18.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.995+0000 7f41da13d700 1 -- 172.21.15.144:0/160124822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41d40ffcb0 msgr2=0x7f41d4195250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:18.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.995+0000 7f41da13d700 1 --2- 172.21.15.144:0/160124822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41d40ffcb0 0x7f41d4195250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:18.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.995+0000 7f41da13d700 1 -- 172.21.15.144:0/160124822 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d00095f0 con 0x7f41d40ff310 2024-09-10T11:55:18.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.995+0000 7f41d993c700 1 --2- 172.21.15.144:0/160124822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41d40ffcb0 0x7f41d4195250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:55:18.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.995+0000 7f41da13d700 1 --2- 172.21.15.144:0/160124822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d40ff310 0x7f41d4194d10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f41d0006010 tx=0x7f41d000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:18.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.995+0000 7f41cb7fe700 1 -- 172.21.15.144:0/160124822 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41d001f070 con 0x7f41d40ff310 2024-09-10T11:55:18.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.995+0000 7f41cb7fe700 1 -- 172.21.15.144:0/160124822 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f41d0019610 con 0x7f41d40ff310 2024-09-10T11:55:18.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.995+0000 7f41dc3a1700 1 -- 172.21.15.144:0/160124822 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d4199940 con 0x7f41d40ff310 2024-09-10T11:55:18.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.995+0000 7f41cb7fe700 1 -- 172.21.15.144:0/160124822 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41d000f690 con 0x7f41d40ff310 2024-09-10T11:55:18.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.995+0000 7f41dc3a1700 1 -- 172.21.15.144:0/160124822 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d4199e30 con 0x7f41d40ff310 2024-09-10T11:55:18.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.996+0000 7f41dc3a1700 1 -- 172.21.15.144:0/160124822 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d404ea50 con 0x7f41d40ff310 2024-09-10T11:55:18.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.997+0000 7f41cb7fe700 1 -- 172.21.15.144:0/160124822 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f41d0003a60 con 0x7f41d40ff310 2024-09-10T11:55:18.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.997+0000 7f41cb7fe700 1 --2- 172.21.15.144:0/160124822 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f41c006e200 0x7f41c00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:18.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.997+0000 7f41cb7fe700 1 -- 172.21.15.144:0/160124822 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f41d0091800 con 0x7f41d40ff310 2024-09-10T11:55:19.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.997+0000 7f41d993c700 1 --2- 172.21.15.144:0/160124822 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f41c006e200 0x7f41c00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:19.000 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:18.998+0000 7f41d993c700 1 --2- 172.21.15.144:0/160124822 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f41c006e200 0x7f41c00706b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f41c4005950 tx=0x7f41c40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:19.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.002+0000 7f41cb7fe700 1 -- 172.21.15.144:0/160124822 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f41d005abe0 con 0x7f41d40ff310 2024-09-10T11:55:19.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:18 smithi144 ceph-mon[87436]: from='client.35300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:18 smithi144 ceph-mon[87436]: from='client.35304 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:18 smithi144 ceph-mon[87436]: pgmap v701: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:18 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/242890047' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:55:19.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.246+0000 7f41dc3a1700 1 -- 172.21.15.144:0/160124822 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f41d419a1b0 con 0x7f41d40ff310 2024-09-10T11:55:19.247 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.246+0000 7f41cb7fe700 1 -- 172.21.15.144:0/160124822 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f41d005e2a0 con 0x7f41d40ff310 2024-09-10T11:55:19.247 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T11:55:19.248 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T11:55:19.248 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:55:19.248 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:55:19.248 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:55:19.248 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:55:19.248 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:55:19.248 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:55:19.248 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:55:19.248 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:55:19.248 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:55:19.248 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:55:19.248 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:55:19.249 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:55:19.249 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:55:19.249 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:55:19.249 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:55:19.249 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:55:19.249 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:55:19.249 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:55:19.249 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:55:19.249 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:55:19.249 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:55:19.249 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:55:19.249 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:55:19.249 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:55:19.250 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:55:19.250 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:55:19.250 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:55:19.250 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:55:19.250 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:55:19.250 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:55:19.250 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:55:19.250 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:55:19.250 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:55:19.250 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:55:19.250 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:55:19.250 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:55:19.250 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:55:19.250 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:55:19.251 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:55:19.251 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:55:19.251 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:55:19.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.248+0000 7f41dc3a1700 1 -- 172.21.15.144:0/160124822 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f41c006e200 msgr2=0x7f41c00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:19.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.248+0000 7f41dc3a1700 1 --2- 172.21.15.144:0/160124822 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f41c006e200 0x7f41c00706b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f41c4005950 tx=0x7f41c40058e0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.248+0000 7f41dc3a1700 1 -- 172.21.15.144:0/160124822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d40ff310 msgr2=0x7f41d4194d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:19.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.248+0000 7f41dc3a1700 1 --2- 172.21.15.144:0/160124822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d40ff310 0x7f41d4194d10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f41d0006010 tx=0x7f41d000be20 comp rx=0 tx=0).stop 2024-09-10T11:55:19.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.248+0000 7f41dc3a1700 1 -- 172.21.15.144:0/160124822 shutdown_connections 2024-09-10T11:55:19.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.248+0000 7f41dc3a1700 1 --2- 172.21.15.144:0/160124822 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f41c006e200 0x7f41c00706b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.248+0000 7f41dc3a1700 1 --2- 172.21.15.144:0/160124822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f41d40ff310 0x7f41d4194d10 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.248+0000 7f41dc3a1700 1 --2- 172.21.15.144:0/160124822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f41d40ffcb0 0x7f41d4195250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.248+0000 7f41dc3a1700 1 -- 172.21.15.144:0/160124822 >> 172.21.15.144:0/160124822 conn(0x7f41d40fabd0 msgr2=0x7f41d40fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:19.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.248+0000 7f41dc3a1700 1 -- 172.21.15.144:0/160124822 shutdown_connections 2024-09-10T11:55:19.252 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.249+0000 7f41dc3a1700 1 -- 172.21.15.144:0/160124822 wait complete. 2024-09-10T11:55:19.252 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T11:55:19.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.354+0000 7f7d99c83700 1 -- 172.21.15.144:0/3334418703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d94101370 msgr2=0x7f7d94101740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:19.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.354+0000 7f7d99c83700 1 --2- 172.21.15.144:0/3334418703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d94101370 0x7f7d94101740 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400b0d0 tx=0x7f7d8400b3e0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.354+0000 7f7d99c83700 1 -- 172.21.15.144:0/3334418703 shutdown_connections 2024-09-10T11:55:19.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.354+0000 7f7d99c83700 1 --2- 172.21.15.144:0/3334418703 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d94101c80 0x7f7d9410a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.354+0000 7f7d99c83700 1 --2- 172.21.15.144:0/3334418703 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d94101370 0x7f7d94101740 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.354+0000 7f7d99c83700 1 -- 172.21.15.144:0/3334418703 >> 172.21.15.144:0/3334418703 conn(0x7f7d940fabd0 msgr2=0x7f7d940fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:19.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.354+0000 7f7d99c83700 1 -- 172.21.15.144:0/3334418703 shutdown_connections 2024-09-10T11:55:19.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.354+0000 7f7d99c83700 1 -- 172.21.15.144:0/3334418703 wait complete. 2024-09-10T11:55:19.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.354+0000 7f7d99c83700 1 Processor -- start 2024-09-10T11:55:19.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.354+0000 7f7d99c83700 1 -- start start 2024-09-10T11:55:19.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d99c83700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d94101370 0x7f7d940fefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:19.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d99c83700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d94101c80 0x7f7d940ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:19.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d99c83700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d940ffa50 con 0x7f7d94101370 2024-09-10T11:55:19.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d99c83700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d940ffb90 con 0x7f7d94101c80 2024-09-10T11:55:19.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d937fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d94101370 0x7f7d940fefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:19.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d937fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d94101370 0x7f7d940fefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58324/0 (socket says 172.21.15.144:58324) 2024-09-10T11:55:19.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d937fe700 1 -- 172.21.15.144:0/1937565327 learned_addr learned my addr 172.21.15.144:0/1937565327 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:55:19.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d92ffd700 1 --2- 172.21.15.144:0/1937565327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d94101c80 0x7f7d940ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:19.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d937fe700 1 -- 172.21.15.144:0/1937565327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d94101c80 msgr2=0x7f7d940ff510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:19.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d937fe700 1 --2- 172.21.15.144:0/1937565327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d94101c80 0x7f7d940ff510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d937fe700 1 -- 172.21.15.144:0/1937565327 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d84009d00 con 0x7f7d94101370 2024-09-10T11:55:19.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d92ffd700 1 --2- 172.21.15.144:0/1937565327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d94101c80 0x7f7d940ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:55:19.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d937fe700 1 --2- 172.21.15.144:0/1937565327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d94101370 0x7f7d940fefd0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400b0a0 tx=0x7f7d8400cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:19.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d90ff9700 1 -- 172.21.15.144:0/1937565327 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d8400e040 con 0x7f7d94101370 2024-09-10T11:55:19.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d90ff9700 1 -- 172.21.15.144:0/1937565327 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7d84004500 con 0x7f7d94101370 2024-09-10T11:55:19.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.355+0000 7f7d99c83700 1 -- 172.21.15.144:0/1937565327 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d940ffe10 con 0x7f7d94101370 2024-09-10T11:55:19.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.356+0000 7f7d90ff9700 1 -- 172.21.15.144:0/1937565327 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d8401daf0 con 0x7f7d94101370 2024-09-10T11:55:19.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.356+0000 7f7d99c83700 1 -- 172.21.15.144:0/1937565327 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d941a22c0 con 0x7f7d94101370 2024-09-10T11:55:19.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.357+0000 7f7d99c83700 1 -- 172.21.15.144:0/1937565327 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d94107830 con 0x7f7d94101370 2024-09-10T11:55:19.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.357+0000 7f7d90ff9700 1 -- 172.21.15.144:0/1937565327 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7d84004670 con 0x7f7d94101370 2024-09-10T11:55:19.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.357+0000 7f7d90ff9700 1 --2- 172.21.15.144:0/1937565327 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7d7c06e200 0x7f7d7c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:19.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.357+0000 7f7d90ff9700 1 -- 172.21.15.144:0/1937565327 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7d8408fb50 con 0x7f7d94101370 2024-09-10T11:55:19.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.357+0000 7f7d92ffd700 1 --2- 172.21.15.144:0/1937565327 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7d7c06e200 0x7f7d7c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:19.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.358+0000 7f7d92ffd700 1 --2- 172.21.15.144:0/1937565327 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7d7c06e200 0x7f7d7c0706b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7d941007c0 tx=0x7f7d88008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:19.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.362+0000 7f7d90ff9700 1 -- 172.21.15.144:0/1937565327 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7d84058f30 con 0x7f7d94101370 2024-09-10T11:55:19.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:18 smithi178 ceph-mon[65940]: from='client.35300 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:19.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:18 smithi178 ceph-mon[65940]: from='client.35304 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:19.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:18 smithi178 ceph-mon[65940]: pgmap v701: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:19.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:18 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/242890047' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:55:19.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.587+0000 7f7d99c83700 1 -- 172.21.15.144:0/1937565327 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d9402cc80 con 0x7f7d7c06e200 2024-09-10T11:55:19.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.589+0000 7f7d90ff9700 1 -- 172.21.15.144:0/1937565327 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7d9402cc80 con 0x7f7d7c06e200 2024-09-10T11:55:19.590 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:55:19.590 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:55:19.590 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:55:19.590 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:55:19.590 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:55:19.590 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:55:19.590 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T11:55:19.590 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T11:55:19.590 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:55:19.590 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T11:55:19.590 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:55:19.591 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T11:55:19.591 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:55:19.591 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:55:19.591 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:55:19.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.591+0000 7f7d99c83700 1 -- 172.21.15.144:0/1937565327 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7d7c06e200 msgr2=0x7f7d7c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:19.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.591+0000 7f7d99c83700 1 --2- 172.21.15.144:0/1937565327 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7d7c06e200 0x7f7d7c0706b0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f7d941007c0 tx=0x7f7d88008040 comp rx=0 tx=0).stop 2024-09-10T11:55:19.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.591+0000 7f7d99c83700 1 -- 172.21.15.144:0/1937565327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d94101370 msgr2=0x7f7d940fefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:19.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.591+0000 7f7d99c83700 1 --2- 172.21.15.144:0/1937565327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d94101370 0x7f7d940fefd0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7d8400b0a0 tx=0x7f7d8400cdc0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.592 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.591+0000 7f7d99c83700 1 -- 172.21.15.144:0/1937565327 shutdown_connections 2024-09-10T11:55:19.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.591+0000 7f7d99c83700 1 --2- 172.21.15.144:0/1937565327 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7d7c06e200 0x7f7d7c0706b0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.591+0000 7f7d99c83700 1 --2- 172.21.15.144:0/1937565327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7d94101370 0x7f7d940fefd0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.591+0000 7f7d99c83700 1 --2- 172.21.15.144:0/1937565327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7d94101c80 0x7f7d940ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.591+0000 7f7d99c83700 1 -- 172.21.15.144:0/1937565327 >> 172.21.15.144:0/1937565327 conn(0x7f7d940fabd0 msgr2=0x7f7d94104970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:19.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.591+0000 7f7d99c83700 1 -- 172.21.15.144:0/1937565327 shutdown_connections 2024-09-10T11:55:19.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.591+0000 7f7d99c83700 1 -- 172.21.15.144:0/1937565327 wait complete. 2024-09-10T11:55:19.697 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.696+0000 7fae8cbd2700 1 -- 172.21.15.144:0/2158271312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae880ff390 msgr2=0x7fae880ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:19.697 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.696+0000 7fae8cbd2700 1 --2- 172.21.15.144:0/2158271312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae880ff390 0x7fae880ff760 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fae8000b0a0 tx=0x7fae8000b3b0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.696+0000 7fae8cbd2700 1 -- 172.21.15.144:0/2158271312 shutdown_connections 2024-09-10T11:55:19.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.696+0000 7fae8cbd2700 1 --2- 172.21.15.144:0/2158271312 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae880ffd30 0x7fae88103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.696+0000 7fae8cbd2700 1 --2- 172.21.15.144:0/2158271312 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae880ff390 0x7fae880ff760 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.696+0000 7fae8cbd2700 1 -- 172.21.15.144:0/2158271312 >> 172.21.15.144:0/2158271312 conn(0x7fae88074f50 msgr2=0x7fae88075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:19.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.696+0000 7fae8cbd2700 1 -- 172.21.15.144:0/2158271312 shutdown_connections 2024-09-10T11:55:19.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.696+0000 7fae8cbd2700 1 -- 172.21.15.144:0/2158271312 wait complete. 2024-09-10T11:55:19.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.697+0000 7fae8cbd2700 1 Processor -- start 2024-09-10T11:55:19.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.697+0000 7fae8cbd2700 1 -- start start 2024-09-10T11:55:19.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.697+0000 7fae8cbd2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae880ff390 0x7fae8819aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:19.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.697+0000 7fae8cbd2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae880ffd30 0x7fae8819af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:19.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.697+0000 7fae8cbd2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8819b640 con 0x7fae880ffd30 2024-09-10T11:55:19.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.697+0000 7fae8cbd2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae88194b00 con 0x7fae880ff390 2024-09-10T11:55:19.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.697+0000 7fae85d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae880ffd30 0x7fae8819af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:19.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.697+0000 7fae85d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae880ffd30 0x7fae8819af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58350/0 (socket says 172.21.15.144:58350) 2024-09-10T11:55:19.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.697+0000 7fae85d9b700 1 -- 172.21.15.144:0/3338102605 learned_addr learned my addr 172.21.15.144:0/3338102605 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:55:19.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.697+0000 7fae8659c700 1 --2- 172.21.15.144:0/3338102605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae880ff390 0x7fae8819aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:19.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.698+0000 7fae85d9b700 1 -- 172.21.15.144:0/3338102605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae880ff390 msgr2=0x7fae8819aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:19.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.698+0000 7fae85d9b700 1 --2- 172.21.15.144:0/3338102605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae880ff390 0x7fae8819aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.698+0000 7fae85d9b700 1 -- 172.21.15.144:0/3338102605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae80009d00 con 0x7fae880ffd30 2024-09-10T11:55:19.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.698+0000 7fae85d9b700 1 --2- 172.21.15.144:0/3338102605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae880ffd30 0x7fae8819af60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fae7000d900 tx=0x7fae7000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:19.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.698+0000 7fae7f7fe700 1 -- 172.21.15.144:0/3338102605 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae70009900 con 0x7fae880ffd30 2024-09-10T11:55:19.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.698+0000 7fae7f7fe700 1 -- 172.21.15.144:0/3338102605 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fae70010460 con 0x7fae880ffd30 2024-09-10T11:55:19.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.698+0000 7fae8cbd2700 1 -- 172.21.15.144:0/3338102605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae88194de0 con 0x7fae880ffd30 2024-09-10T11:55:19.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.698+0000 7fae7f7fe700 1 -- 172.21.15.144:0/3338102605 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae7000f5f0 con 0x7fae880ffd30 2024-09-10T11:55:19.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.698+0000 7fae8cbd2700 1 -- 172.21.15.144:0/3338102605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae88195330 con 0x7fae880ffd30 2024-09-10T11:55:19.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.699+0000 7fae8cbd2700 1 -- 172.21.15.144:0/3338102605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae88107650 con 0x7fae880ffd30 2024-09-10T11:55:19.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.699+0000 7fae7f7fe700 1 -- 172.21.15.144:0/3338102605 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fae700105d0 con 0x7fae880ffd30 2024-09-10T11:55:19.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.700+0000 7fae7f7fe700 1 --2- 172.21.15.144:0/3338102605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fae7406e200 0x7fae740706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:19.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.700+0000 7fae8659c700 1 --2- 172.21.15.144:0/3338102605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fae7406e200 0x7fae740706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:19.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.700+0000 7fae7f7fe700 1 -- 172.21.15.144:0/3338102605 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fae7008db60 con 0x7fae880ffd30 2024-09-10T11:55:19.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.700+0000 7fae8659c700 1 --2- 172.21.15.144:0/3338102605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fae7406e200 0x7fae740706b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fae80000f80 tx=0x7fae800060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:19.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.704+0000 7fae7f7fe700 1 -- 172.21.15.144:0/3338102605 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fae7005aaf0 con 0x7fae880ffd30 2024-09-10T11:55:19.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.989+0000 7fae8cbd2700 1 -- 172.21.15.144:0/3338102605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fae8804ea50 con 0x7fae880ffd30 2024-09-10T11:55:19.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.989+0000 7fae7f7fe700 1 -- 172.21.15.144:0/3338102605 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7fae70017090 con 0x7fae880ffd30 2024-09-10T11:55:19.991 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:55:19.991 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T11:55:19.991 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:55:19.991 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T11:55:19.991 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:55:19.991 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T11:55:19.991 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T11:55:19.991 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T11:55:19.991 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T11:55:19.991 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T11:55:19.991 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T11:55:19.992 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T11:55:19.992 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T11:55:19.992 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T11:55:19.992 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T11:55:19.992 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T11:55:19.992 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T11:55:19.992 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T11:55:19.992 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:55:19.992 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:55:19.992 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:55:19.992 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:55:19.992 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:55:19.992 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:55:19.992 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:55:19.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.991+0000 7fae8cbd2700 1 -- 172.21.15.144:0/3338102605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fae7406e200 msgr2=0x7fae740706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:19.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.991+0000 7fae8cbd2700 1 --2- 172.21.15.144:0/3338102605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fae7406e200 0x7fae740706b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fae80000f80 tx=0x7fae800060c0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.991+0000 7fae8cbd2700 1 -- 172.21.15.144:0/3338102605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae880ffd30 msgr2=0x7fae8819af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:19.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.991+0000 7fae8cbd2700 1 --2- 172.21.15.144:0/3338102605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae880ffd30 0x7fae8819af60 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fae7000d900 tx=0x7fae7000dc10 comp rx=0 tx=0).stop 2024-09-10T11:55:19.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.991+0000 7fae8cbd2700 1 -- 172.21.15.144:0/3338102605 shutdown_connections 2024-09-10T11:55:19.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.991+0000 7fae8cbd2700 1 --2- 172.21.15.144:0/3338102605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fae7406e200 0x7fae740706b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.991+0000 7fae8cbd2700 1 --2- 172.21.15.144:0/3338102605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fae880ff390 0x7fae8819aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.991+0000 7fae8cbd2700 1 --2- 172.21.15.144:0/3338102605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fae880ffd30 0x7fae8819af60 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:19.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.991+0000 7fae8cbd2700 1 -- 172.21.15.144:0/3338102605 >> 172.21.15.144:0/3338102605 conn(0x7fae88074f50 msgr2=0x7fae880fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:19.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.992+0000 7fae8cbd2700 1 -- 172.21.15.144:0/3338102605 shutdown_connections 2024-09-10T11:55:19.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:19.992+0000 7fae8cbd2700 1 -- 172.21.15.144:0/3338102605 wait complete. 2024-09-10T11:55:20.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:19 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/160124822' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:55:20.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:19 smithi178 ceph-mon[65940]: from='client.35316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:20.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:19 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/160124822' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:55:20.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:19 smithi144 ceph-mon[87436]: from='client.35316 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:20 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3338102605' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:55:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:20 smithi178 ceph-mon[65940]: pgmap v702: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:21.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:20 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3338102605' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:55:21.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:20 smithi144 ceph-mon[87436]: pgmap v702: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:23.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:22 smithi178 ceph-mon[65940]: pgmap v703: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:23.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:22 smithi144 ceph-mon[87436]: pgmap v703: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:24 smithi178 ceph-mon[65940]: pgmap v704: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:25.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:24 smithi144 ceph-mon[87436]: pgmap v704: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:26 smithi178 ceph-mon[65940]: pgmap v705: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:27.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:26 smithi144 ceph-mon[87436]: pgmap v705: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:28.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:55:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:55:29.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:29 smithi144 ceph-mon[87436]: pgmap v706: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:29 smithi178 ceph-mon[65940]: pgmap v706: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:30 smithi178 ceph-mon[65940]: pgmap v707: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:31.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:30 smithi144 ceph-mon[87436]: pgmap v707: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:32 smithi178 ceph-mon[65940]: pgmap v708: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:33.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:32 smithi144 ceph-mon[87436]: pgmap v708: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:34 smithi178 ceph-mon[65940]: pgmap v709: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:35.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:34 smithi144 ceph-mon[87436]: pgmap v709: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:36 smithi178 ceph-mon[65940]: pgmap v710: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:37.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:36 smithi144 ceph-mon[87436]: pgmap v710: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:39.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:38 smithi178 ceph-mon[65940]: pgmap v711: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:39.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:38 smithi144 ceph-mon[87436]: pgmap v711: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:40 smithi178 ceph-mon[65940]: pgmap v712: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:41.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:40 smithi144 ceph-mon[87436]: pgmap v712: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:43.236 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:42 smithi144 ceph-mon[87436]: pgmap v713: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:43.279 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:42 smithi178 ceph-mon[65940]: pgmap v713: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:44.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:55:44.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:55:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:44 smithi144 ceph-mon[87436]: pgmap v714: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:44 smithi178 ceph-mon[65940]: pgmap v714: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:46 smithi178 ceph-mon[65940]: pgmap v715: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:46 smithi144 ceph-mon[87436]: pgmap v715: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:48 smithi178 ceph-mon[65940]: pgmap v716: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:49.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:48 smithi144 ceph-mon[87436]: pgmap v716: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:50.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.105+0000 7f19fac11700 1 -- 172.21.15.144:0/3305013507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f40ffd30 msgr2=0x7f19f4103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:50.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.105+0000 7f19fac11700 1 --2- 172.21.15.144:0/3305013507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f40ffd30 0x7f19f4103ce0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f19e0009910 tx=0x7f19e0009c20 comp rx=0 tx=0).stop 2024-09-10T11:55:50.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.105+0000 7f19fac11700 1 -- 172.21.15.144:0/3305013507 shutdown_connections 2024-09-10T11:55:50.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.105+0000 7f19fac11700 1 --2- 172.21.15.144:0/3305013507 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f40ffd30 0x7f19f4103ce0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:50.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.105+0000 7f19fac11700 1 --2- 172.21.15.144:0/3305013507 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19f40ff390 0x7f19f40ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:50.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.105+0000 7f19fac11700 1 -- 172.21.15.144:0/3305013507 >> 172.21.15.144:0/3305013507 conn(0x7f19f4074f50 msgr2=0x7f19f4075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:50.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.105+0000 7f19fac11700 1 -- 172.21.15.144:0/3305013507 shutdown_connections 2024-09-10T11:55:50.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.105+0000 7f19fac11700 1 -- 172.21.15.144:0/3305013507 wait complete. 2024-09-10T11:55:50.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.106+0000 7f19fac11700 1 Processor -- start 2024-09-10T11:55:50.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.106+0000 7f19fac11700 1 -- start start 2024-09-10T11:55:50.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.106+0000 7f19fac11700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19f40ff390 0x7f19f4100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:50.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.106+0000 7f19fac11700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f40ffd30 0x7f19f41012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:50.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.106+0000 7f19fac11700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f4101800 con 0x7f19f40ffd30 2024-09-10T11:55:50.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.106+0000 7f19fac11700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f4101940 con 0x7f19f40ff390 2024-09-10T11:55:50.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.106+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f40ffd30 0x7f19f41012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:50.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.106+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f40ffd30 0x7f19f41012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38384/0 (socket says 172.21.15.144:38384) 2024-09-10T11:55:50.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.106+0000 7f19f3fff700 1 -- 172.21.15.144:0/3326142383 learned_addr learned my addr 172.21.15.144:0/3326142383 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:55:50.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.106+0000 7f19f89ad700 1 --2- 172.21.15.144:0/3326142383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19f40ff390 0x7f19f4100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:50.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.106+0000 7f19f3fff700 1 -- 172.21.15.144:0/3326142383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19f40ff390 msgr2=0x7f19f4100d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:50.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.106+0000 7f19f3fff700 1 --2- 172.21.15.144:0/3326142383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19f40ff390 0x7f19f4100d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:50.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.106+0000 7f19f3fff700 1 -- 172.21.15.144:0/3326142383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e00095f0 con 0x7f19f40ffd30 2024-09-10T11:55:50.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.107+0000 7f19f3fff700 1 --2- 172.21.15.144:0/3326142383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f40ffd30 0x7f19f41012c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f19e00092e0 tx=0x7f19e000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:50.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.107+0000 7f19f1ffb700 1 -- 172.21.15.144:0/3326142383 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19e001d070 con 0x7f19f40ffd30 2024-09-10T11:55:50.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.107+0000 7f19f1ffb700 1 -- 172.21.15.144:0/3326142383 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19e000f460 con 0x7f19f40ffd30 2024-09-10T11:55:50.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.107+0000 7f19f1ffb700 1 -- 172.21.15.144:0/3326142383 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19e00175f0 con 0x7f19f40ffd30 2024-09-10T11:55:50.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.107+0000 7f19fac11700 1 -- 172.21.15.144:0/3326142383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f4101bc0 con 0x7f19f40ffd30 2024-09-10T11:55:50.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.107+0000 7f19fac11700 1 -- 172.21.15.144:0/3326142383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f41a2340 con 0x7f19f40ffd30 2024-09-10T11:55:50.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.108+0000 7f19fac11700 1 -- 172.21.15.144:0/3326142383 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f404ea50 con 0x7f19f40ffd30 2024-09-10T11:55:50.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.109+0000 7f19f1ffb700 1 -- 172.21.15.144:0/3326142383 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f19e000f5d0 con 0x7f19f40ffd30 2024-09-10T11:55:50.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.109+0000 7f19f1ffb700 1 --2- 172.21.15.144:0/3326142383 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19e406e250 0x7f19e4070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:50.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.110+0000 7f19f1ffb700 1 -- 172.21.15.144:0/3326142383 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f19e008f670 con 0x7f19f40ffd30 2024-09-10T11:55:50.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.110+0000 7f19f89ad700 1 --2- 172.21.15.144:0/3326142383 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19e406e250 0x7f19e4070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:50.112 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.110+0000 7f19f89ad700 1 --2- 172.21.15.144:0/3326142383 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19e406e250 0x7f19e4070700 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f19e8006fd0 tx=0x7f19e8008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:50.115 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.114+0000 7f19f1ffb700 1 -- 172.21.15.144:0/3326142383 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f19e00589b0 con 0x7f19f40ffd30 2024-09-10T11:55:50.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.336+0000 7f19fac11700 1 -- 172.21.15.144:0/3326142383 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f19f40fcbc0 con 0x7f19e406e250 2024-09-10T11:55:50.338 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.337+0000 7f19f1ffb700 1 -- 172.21.15.144:0/3326142383 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f19f40fcbc0 con 0x7f19e406e250 2024-09-10T11:55:50.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.339+0000 7f19fac11700 1 -- 172.21.15.144:0/3326142383 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19e406e250 msgr2=0x7f19e4070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:50.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.339+0000 7f19fac11700 1 --2- 172.21.15.144:0/3326142383 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19e406e250 0x7f19e4070700 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f19e8006fd0 tx=0x7f19e8008040 comp rx=0 tx=0).stop 2024-09-10T11:55:50.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.339+0000 7f19fac11700 1 -- 172.21.15.144:0/3326142383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f40ffd30 msgr2=0x7f19f41012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:50.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.339+0000 7f19fac11700 1 --2- 172.21.15.144:0/3326142383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f40ffd30 0x7f19f41012c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f19e00092e0 tx=0x7f19e000be20 comp rx=0 tx=0).stop 2024-09-10T11:55:50.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.339+0000 7f19fac11700 1 -- 172.21.15.144:0/3326142383 shutdown_connections 2024-09-10T11:55:50.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.339+0000 7f19fac11700 1 --2- 172.21.15.144:0/3326142383 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19e406e250 0x7f19e4070700 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:50.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.339+0000 7f19fac11700 1 --2- 172.21.15.144:0/3326142383 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19f40ff390 0x7f19f4100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:50.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.339+0000 7f19fac11700 1 --2- 172.21.15.144:0/3326142383 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f40ffd30 0x7f19f41012c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:50.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.339+0000 7f19fac11700 1 -- 172.21.15.144:0/3326142383 >> 172.21.15.144:0/3326142383 conn(0x7f19f4074f50 msgr2=0x7f19f4103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:50.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.339+0000 7f19fac11700 1 -- 172.21.15.144:0/3326142383 shutdown_connections 2024-09-10T11:55:50.341 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.340+0000 7f19fac11700 1 -- 172.21.15.144:0/3326142383 wait complete. 2024-09-10T11:55:50.351 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:55:50.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.445+0000 7fdd73020700 1 -- 172.21.15.144:0/620947535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6c100d80 msgr2=0x7fdd6c104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:50.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.445+0000 7fdd73020700 1 --2- 172.21.15.144:0/620947535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6c100d80 0x7fdd6c104d50 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009910 tx=0x7fdd58009c20 comp rx=0 tx=0).stop 2024-09-10T11:55:50.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.446+0000 7fdd73020700 1 -- 172.21.15.144:0/620947535 shutdown_connections 2024-09-10T11:55:50.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.446+0000 7fdd73020700 1 --2- 172.21.15.144:0/620947535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6c100d80 0x7fdd6c104d50 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:50.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.446+0000 7fdd73020700 1 --2- 172.21.15.144:0/620947535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd6c1003e0 0x7fdd6c1007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:50.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.446+0000 7fdd73020700 1 -- 172.21.15.144:0/620947535 >> 172.21.15.144:0/620947535 conn(0x7fdd6c0fbca0 msgr2=0x7fdd6c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:50.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.446+0000 7fdd73020700 1 -- 172.21.15.144:0/620947535 shutdown_connections 2024-09-10T11:55:50.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.446+0000 7fdd73020700 1 -- 172.21.15.144:0/620947535 wait complete. 2024-09-10T11:55:50.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.446+0000 7fdd73020700 1 Processor -- start 2024-09-10T11:55:50.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.446+0000 7fdd73020700 1 -- start start 2024-09-10T11:55:50.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.446+0000 7fdd73020700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd6c1003e0 0x7fdd6c195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:50.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.446+0000 7fdd73020700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6c100d80 0x7fdd6c196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:50.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.446+0000 7fdd73020700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c1969f0 con 0x7fdd6c100d80 2024-09-10T11:55:50.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.446+0000 7fdd73020700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c19a780 con 0x7fdd6c1003e0 2024-09-10T11:55:50.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.447+0000 7fdd6bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6c100d80 0x7fdd6c196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:50.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.447+0000 7fdd6bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6c100d80 0x7fdd6c196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38408/0 (socket says 172.21.15.144:38408) 2024-09-10T11:55:50.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.447+0000 7fdd6bfff700 1 -- 172.21.15.144:0/1489995988 learned_addr learned my addr 172.21.15.144:0/1489995988 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:55:50.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.447+0000 7fdd70dbc700 1 --2- 172.21.15.144:0/1489995988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd6c1003e0 0x7fdd6c195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:50.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.447+0000 7fdd6bfff700 1 -- 172.21.15.144:0/1489995988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd6c1003e0 msgr2=0x7fdd6c195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:50.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.447+0000 7fdd6bfff700 1 --2- 172.21.15.144:0/1489995988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd6c1003e0 0x7fdd6c195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:50.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.447+0000 7fdd6bfff700 1 -- 172.21.15.144:0/1489995988 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd580095f0 con 0x7fdd6c100d80 2024-09-10T11:55:50.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.447+0000 7fdd6bfff700 1 --2- 172.21.15.144:0/1489995988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6c100d80 0x7fdd6c196310 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdd580092e0 tx=0x7fdd5800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:50.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.447+0000 7fdd69ffb700 1 -- 172.21.15.144:0/1489995988 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd5801d070 con 0x7fdd6c100d80 2024-09-10T11:55:50.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.447+0000 7fdd69ffb700 1 -- 172.21.15.144:0/1489995988 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdd58005dc0 con 0x7fdd6c100d80 2024-09-10T11:55:50.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.447+0000 7fdd73020700 1 -- 172.21.15.144:0/1489995988 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd6c19aa00 con 0x7fdd6c100d80 2024-09-10T11:55:50.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.447+0000 7fdd69ffb700 1 -- 172.21.15.144:0/1489995988 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd58017400 con 0x7fdd6c100d80 2024-09-10T11:55:50.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.447+0000 7fdd73020700 1 -- 172.21.15.144:0/1489995988 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6c19aef0 con 0x7fdd6c100d80 2024-09-10T11:55:50.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.448+0000 7fdd73020700 1 -- 172.21.15.144:0/1489995988 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd6c04ea50 con 0x7fdd6c100d80 2024-09-10T11:55:50.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.449+0000 7fdd69ffb700 1 -- 172.21.15.144:0/1489995988 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fdd5800f6f0 con 0x7fdd6c100d80 2024-09-10T11:55:50.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.449+0000 7fdd69ffb700 1 --2- 172.21.15.144:0/1489995988 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdd5c06e200 0x7fdd5c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:50.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.449+0000 7fdd69ffb700 1 -- 172.21.15.144:0/1489995988 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdd5808f470 con 0x7fdd6c100d80 2024-09-10T11:55:50.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.449+0000 7fdd70dbc700 1 --2- 172.21.15.144:0/1489995988 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdd5c06e200 0x7fdd5c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:50.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.450+0000 7fdd70dbc700 1 --2- 172.21.15.144:0/1489995988 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdd5c06e200 0x7fdd5c0706b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdd60005950 tx=0x7fdd6000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:50.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.454+0000 7fdd69ffb700 1 -- 172.21.15.144:0/1489995988 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdd58058850 con 0x7fdd6c100d80 2024-09-10T11:55:50.674 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.672+0000 7fdd73020700 1 -- 172.21.15.144:0/1489995988 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd6c19b3d0 con 0x7fdd5c06e200 2024-09-10T11:55:50.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.674+0000 7fdd69ffb700 1 -- 172.21.15.144:0/1489995988 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fdd6c19b3d0 con 0x7fdd5c06e200 2024-09-10T11:55:50.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.676+0000 7fdd73020700 1 -- 172.21.15.144:0/1489995988 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdd5c06e200 msgr2=0x7fdd5c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:50.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.676+0000 7fdd73020700 1 --2- 172.21.15.144:0/1489995988 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdd5c06e200 0x7fdd5c0706b0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdd60005950 tx=0x7fdd6000b410 comp rx=0 tx=0).stop 2024-09-10T11:55:50.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.676+0000 7fdd73020700 1 -- 172.21.15.144:0/1489995988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6c100d80 msgr2=0x7fdd6c196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:50.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.676+0000 7fdd73020700 1 --2- 172.21.15.144:0/1489995988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6c100d80 0x7fdd6c196310 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fdd580092e0 tx=0x7fdd5800bd70 comp rx=0 tx=0).stop 2024-09-10T11:55:50.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.676+0000 7fdd73020700 1 -- 172.21.15.144:0/1489995988 shutdown_connections 2024-09-10T11:55:50.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.676+0000 7fdd73020700 1 --2- 172.21.15.144:0/1489995988 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdd5c06e200 0x7fdd5c0706b0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:50.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.676+0000 7fdd73020700 1 --2- 172.21.15.144:0/1489995988 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdd6c1003e0 0x7fdd6c195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:50.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.676+0000 7fdd73020700 1 --2- 172.21.15.144:0/1489995988 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdd6c100d80 0x7fdd6c196310 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:50.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.676+0000 7fdd73020700 1 -- 172.21.15.144:0/1489995988 >> 172.21.15.144:0/1489995988 conn(0x7fdd6c0fbca0 msgr2=0x7fdd6c0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:50.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.676+0000 7fdd73020700 1 -- 172.21.15.144:0/1489995988 shutdown_connections 2024-09-10T11:55:50.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.676+0000 7fdd73020700 1 -- 172.21.15.144:0/1489995988 wait complete. 2024-09-10T11:55:50.782 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.781+0000 7f50b271a700 1 -- 172.21.15.144:0/3622639596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50ac068630 msgr2=0x7f50ac068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:50.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.781+0000 7f50b271a700 1 --2- 172.21.15.144:0/3622639596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50ac068630 0x7f50ac068a00 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f509c009910 tx=0x7f509c009c20 comp rx=0 tx=0).stop 2024-09-10T11:55:50.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.781+0000 7f50b271a700 1 -- 172.21.15.144:0/3622639596 shutdown_connections 2024-09-10T11:55:50.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.781+0000 7f50b271a700 1 --2- 172.21.15.144:0/3622639596 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50ac068fd0 0x7f50ac104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:50.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.781+0000 7f50b271a700 1 --2- 172.21.15.144:0/3622639596 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50ac068630 0x7f50ac068a00 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:50.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.781+0000 7f50b271a700 1 -- 172.21.15.144:0/3622639596 >> 172.21.15.144:0/3622639596 conn(0x7f50ac074f50 msgr2=0x7f50ac075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:50.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.781+0000 7f50b271a700 1 -- 172.21.15.144:0/3622639596 shutdown_connections 2024-09-10T11:55:50.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.781+0000 7f50b271a700 1 -- 172.21.15.144:0/3622639596 wait complete. 2024-09-10T11:55:50.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.782+0000 7f50b271a700 1 Processor -- start 2024-09-10T11:55:50.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.782+0000 7f50b271a700 1 -- start start 2024-09-10T11:55:50.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.782+0000 7f50b271a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50ac068630 0x7f50ac195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:50.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.782+0000 7f50b271a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50ac068fd0 0x7f50ac1962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:50.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.782+0000 7f50b271a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac196990 con 0x7f50ac068fd0 2024-09-10T11:55:50.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.782+0000 7f50b271a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50ac19a720 con 0x7f50ac068630 2024-09-10T11:55:50.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.782+0000 7f50ab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50ac068fd0 0x7f50ac1962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:50.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.782+0000 7f50ab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50ac068fd0 0x7f50ac1962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38414/0 (socket says 172.21.15.144:38414) 2024-09-10T11:55:50.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.782+0000 7f50ab7fe700 1 -- 172.21.15.144:0/1656894942 learned_addr learned my addr 172.21.15.144:0/1656894942 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:55:50.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.782+0000 7f50abfff700 1 --2- 172.21.15.144:0/1656894942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50ac068630 0x7f50ac195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:50.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.783+0000 7f50ab7fe700 1 -- 172.21.15.144:0/1656894942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50ac068630 msgr2=0x7f50ac195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:50.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.783+0000 7f50ab7fe700 1 --2- 172.21.15.144:0/1656894942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50ac068630 0x7f50ac195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:50.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.783+0000 7f50ab7fe700 1 -- 172.21.15.144:0/1656894942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f509c0095f0 con 0x7f50ac068fd0 2024-09-10T11:55:50.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.783+0000 7f50ab7fe700 1 --2- 172.21.15.144:0/1656894942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50ac068fd0 0x7f50ac1962b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f50a000d8d0 tx=0x7f50a000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:50.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.783+0000 7f50a97fa700 1 -- 172.21.15.144:0/1656894942 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50a0004a00 con 0x7f50ac068fd0 2024-09-10T11:55:50.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.783+0000 7f50a97fa700 1 -- 172.21.15.144:0/1656894942 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50a00054e0 con 0x7f50ac068fd0 2024-09-10T11:55:50.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.783+0000 7f50a97fa700 1 -- 172.21.15.144:0/1656894942 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50a000fc20 con 0x7f50ac068fd0 2024-09-10T11:55:50.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.783+0000 7f50b271a700 1 -- 172.21.15.144:0/1656894942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50ac19aa00 con 0x7f50ac068fd0 2024-09-10T11:55:50.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.783+0000 7f50b271a700 1 -- 172.21.15.144:0/1656894942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50ac19af50 con 0x7f50ac068fd0 2024-09-10T11:55:50.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.784+0000 7f50b271a700 1 -- 172.21.15.144:0/1656894942 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50ac1086f0 con 0x7f50ac068fd0 2024-09-10T11:55:50.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.785+0000 7f50a97fa700 1 -- 172.21.15.144:0/1656894942 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f50a000fd80 con 0x7f50ac068fd0 2024-09-10T11:55:50.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.785+0000 7f50a97fa700 1 --2- 172.21.15.144:0/1656894942 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5094072610 0x7f5094074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:50.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.785+0000 7f50a97fa700 1 -- 172.21.15.144:0/1656894942 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f50a008d8a0 con 0x7f50ac068fd0 2024-09-10T11:55:50.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.785+0000 7f50abfff700 1 --2- 172.21.15.144:0/1656894942 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5094072610 0x7f5094074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:50.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.786+0000 7f50abfff700 1 --2- 172.21.15.144:0/1656894942 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5094072610 0x7f5094074ac0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f509c006010 tx=0x7f509c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:50.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:50.790+0000 7f50a97fa700 1 -- 172.21.15.144:0/1656894942 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f50a0056ca0 con 0x7f50ac068fd0 2024-09-10T11:55:51.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.000+0000 7f50b271a700 1 -- 172.21.15.144:0/1656894942 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f50ac19b2b0 con 0x7f5094072610 2024-09-10T11:55:51.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.010+0000 7f50a97fa700 1 -- 172.21.15.144:0/1656894942 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f50ac19b2b0 con 0x7f5094072610 2024-09-10T11:55:51.011 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:55:51.011 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (24m) 2m ago 34m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:55:51.011 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (34m) 2m ago 34m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:55:51.011 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (33m) 2m ago 33m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:55:51.011 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (21m) 2m ago 34m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:55:51.012 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (21m) 2m ago 33m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:55:51.012 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (23m) 2m ago 34m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:55:51.012 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (18m) 2m ago 29m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:55:51.012 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (17m) 2m ago 29m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:55:51.012 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (2m) 2m ago 29m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T11:55:51.012 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (17m) 2m ago 29m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:55:51.012 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (25m) 2m ago 36m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:55:51.012 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (24m) 2m ago 33m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:55:51.012 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (22m) 2m ago 36m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:55:51.012 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (22m) 2m ago 33m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:55:51.012 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (24m) 2m ago 34m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:55:51.012 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (24m) 2m ago 33m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:55:51.012 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (21m) 2m ago 32m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:55:51.013 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (20m) 2m ago 32m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:55:51.013 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (20m) 2m ago 31m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:55:51.013 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (19m) 2m ago 31m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:55:51.013 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (19m) 2m ago 31m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:55:51.013 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (18m) 2m ago 30m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:55:51.013 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (24m) 2m ago 34m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:55:51.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.012+0000 7f50b271a700 1 -- 172.21.15.144:0/1656894942 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5094072610 msgr2=0x7f5094074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:51.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.012+0000 7f50b271a700 1 --2- 172.21.15.144:0/1656894942 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5094072610 0x7f5094074ac0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f509c006010 tx=0x7f509c01a040 comp rx=0 tx=0).stop 2024-09-10T11:55:51.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.012+0000 7f50b271a700 1 -- 172.21.15.144:0/1656894942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50ac068fd0 msgr2=0x7f50ac1962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:51.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.012+0000 7f50b271a700 1 --2- 172.21.15.144:0/1656894942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50ac068fd0 0x7f50ac1962b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f50a000d8d0 tx=0x7f50a000dbe0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.012+0000 7f50b271a700 1 -- 172.21.15.144:0/1656894942 shutdown_connections 2024-09-10T11:55:51.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.012+0000 7f50b271a700 1 --2- 172.21.15.144:0/1656894942 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5094072610 0x7f5094074ac0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.012+0000 7f50b271a700 1 --2- 172.21.15.144:0/1656894942 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50ac068630 0x7f50ac195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.012+0000 7f50b271a700 1 --2- 172.21.15.144:0/1656894942 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50ac068fd0 0x7f50ac1962b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.012+0000 7f50b271a700 1 -- 172.21.15.144:0/1656894942 >> 172.21.15.144:0/1656894942 conn(0x7f50ac074f50 msgr2=0x7f50ac0fd730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:51.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.012+0000 7f50b271a700 1 -- 172.21.15.144:0/1656894942 shutdown_connections 2024-09-10T11:55:51.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.012+0000 7f50b271a700 1 -- 172.21.15.144:0/1656894942 wait complete. 2024-09-10T11:55:51.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.119+0000 7f68f0e73700 1 -- 172.21.15.144:0/3859053704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ec100d70 msgr2=0x7f68ec104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:51.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.119+0000 7f68f0e73700 1 --2- 172.21.15.144:0/3859053704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ec100d70 0x7f68ec104d40 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009910 tx=0x7f68e0009c20 comp rx=0 tx=0).stop 2024-09-10T11:55:51.120 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:50 smithi144 ceph-mon[87436]: from='client.35324 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:51.120 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:50 smithi144 ceph-mon[87436]: from='client.35328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:51.120 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:50 smithi144 ceph-mon[87436]: pgmap v717: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:51.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.119+0000 7f68f0e73700 1 -- 172.21.15.144:0/3859053704 shutdown_connections 2024-09-10T11:55:51.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.119+0000 7f68f0e73700 1 --2- 172.21.15.144:0/3859053704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ec100d70 0x7f68ec104d40 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.119+0000 7f68f0e73700 1 --2- 172.21.15.144:0/3859053704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68ec1003d0 0x7f68ec1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.119+0000 7f68f0e73700 1 -- 172.21.15.144:0/3859053704 >> 172.21.15.144:0/3859053704 conn(0x7f68ec0fbc90 msgr2=0x7f68ec0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:51.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.119+0000 7f68f0e73700 1 -- 172.21.15.144:0/3859053704 shutdown_connections 2024-09-10T11:55:51.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.119+0000 7f68f0e73700 1 -- 172.21.15.144:0/3859053704 wait complete. 2024-09-10T11:55:51.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.120+0000 7f68f0e73700 1 Processor -- start 2024-09-10T11:55:51.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.120+0000 7f68f0e73700 1 -- start start 2024-09-10T11:55:51.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.120+0000 7f68f0e73700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68ec1003d0 0x7f68ec195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:51.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.120+0000 7f68f0e73700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ec100d70 0x7f68ec196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:51.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.120+0000 7f68f0e73700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec196960 con 0x7f68ec100d70 2024-09-10T11:55:51.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.120+0000 7f68f0e73700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ec19a6f0 con 0x7f68ec1003d0 2024-09-10T11:55:51.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.120+0000 7f68e9d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ec100d70 0x7f68ec196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:51.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.120+0000 7f68e9d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ec100d70 0x7f68ec196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38422/0 (socket says 172.21.15.144:38422) 2024-09-10T11:55:51.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.120+0000 7f68e9d9b700 1 -- 172.21.15.144:0/669139854 learned_addr learned my addr 172.21.15.144:0/669139854 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:55:51.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.120+0000 7f68ea59c700 1 --2- 172.21.15.144:0/669139854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68ec1003d0 0x7f68ec195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:51.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.120+0000 7f68e9d9b700 1 -- 172.21.15.144:0/669139854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68ec1003d0 msgr2=0x7f68ec195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:51.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.120+0000 7f68e9d9b700 1 --2- 172.21.15.144:0/669139854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68ec1003d0 0x7f68ec195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.120+0000 7f68e9d9b700 1 -- 172.21.15.144:0/669139854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e00095f0 con 0x7f68ec100d70 2024-09-10T11:55:51.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.121+0000 7f68e9d9b700 1 --2- 172.21.15.144:0/669139854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ec100d70 0x7f68ec196280 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f68e00092e0 tx=0x7f68e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:51.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.121+0000 7f68db7fe700 1 -- 172.21.15.144:0/669139854 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68e001d070 con 0x7f68ec100d70 2024-09-10T11:55:51.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.121+0000 7f68f0e73700 1 -- 172.21.15.144:0/669139854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ec19a970 con 0x7f68ec100d70 2024-09-10T11:55:51.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.121+0000 7f68db7fe700 1 -- 172.21.15.144:0/669139854 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f68e0005dc0 con 0x7f68ec100d70 2024-09-10T11:55:51.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.121+0000 7f68f0e73700 1 -- 172.21.15.144:0/669139854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ec19ae60 con 0x7f68ec100d70 2024-09-10T11:55:51.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.121+0000 7f68db7fe700 1 -- 172.21.15.144:0/669139854 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68e0022420 con 0x7f68ec100d70 2024-09-10T11:55:51.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.122+0000 7f68f0e73700 1 -- 172.21.15.144:0/669139854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ec108740 con 0x7f68ec100d70 2024-09-10T11:55:51.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.122+0000 7f68db7fe700 1 -- 172.21.15.144:0/669139854 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f68e0003a60 con 0x7f68ec100d70 2024-09-10T11:55:51.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.123+0000 7f68db7fe700 1 --2- 172.21.15.144:0/669139854 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f68d406e200 0x7f68d40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:51.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.123+0000 7f68db7fe700 1 -- 172.21.15.144:0/669139854 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f68e008f580 con 0x7f68ec100d70 2024-09-10T11:55:51.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.123+0000 7f68ea59c700 1 --2- 172.21.15.144:0/669139854 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f68d406e200 0x7f68d40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:51.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.124+0000 7f68ea59c700 1 --2- 172.21.15.144:0/669139854 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f68d406e200 0x7f68d40706b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f68dc005950 tx=0x7f68dc00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:51.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.127+0000 7f68db7fe700 1 -- 172.21.15.144:0/669139854 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f68e0058960 con 0x7f68ec100d70 2024-09-10T11:55:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:50 smithi178 ceph-mon[65940]: from='client.35324 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:50 smithi178 ceph-mon[65940]: from='client.35328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:50 smithi178 ceph-mon[65940]: pgmap v717: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:51.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.411+0000 7f68f0e73700 1 -- 172.21.15.144:0/669139854 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f68ec04ea50 con 0x7f68ec100d70 2024-09-10T11:55:51.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.412+0000 7f68db7fe700 1 -- 172.21.15.144:0/669139854 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f68e005c020 con 0x7f68ec100d70 2024-09-10T11:55:51.413 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:55:51.413 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:55:51.413 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:55:51.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:55:51.413 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:55:51.413 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:55:51.413 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:55:51.414 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:55:51.414 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:55:51.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:55:51.414 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:55:51.414 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:55:51.414 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:55:51.414 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:55:51.414 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T11:55:51.414 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:55:51.414 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:55:51.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.414+0000 7f68f0e73700 1 -- 172.21.15.144:0/669139854 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f68d406e200 msgr2=0x7f68d40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:51.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.414+0000 7f68f0e73700 1 --2- 172.21.15.144:0/669139854 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f68d406e200 0x7f68d40706b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f68dc005950 tx=0x7f68dc00b410 comp rx=0 tx=0).stop 2024-09-10T11:55:51.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.414+0000 7f68f0e73700 1 -- 172.21.15.144:0/669139854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ec100d70 msgr2=0x7f68ec196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:51.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.414+0000 7f68f0e73700 1 --2- 172.21.15.144:0/669139854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ec100d70 0x7f68ec196280 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f68e00092e0 tx=0x7f68e000bd70 comp rx=0 tx=0).stop 2024-09-10T11:55:51.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.414+0000 7f68f0e73700 1 -- 172.21.15.144:0/669139854 shutdown_connections 2024-09-10T11:55:51.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.414+0000 7f68f0e73700 1 --2- 172.21.15.144:0/669139854 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f68d406e200 0x7f68d40706b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.414+0000 7f68f0e73700 1 --2- 172.21.15.144:0/669139854 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68ec1003d0 0x7f68ec195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.414+0000 7f68f0e73700 1 --2- 172.21.15.144:0/669139854 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ec100d70 0x7f68ec196280 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.414+0000 7f68f0e73700 1 -- 172.21.15.144:0/669139854 >> 172.21.15.144:0/669139854 conn(0x7f68ec0fbc90 msgr2=0x7f68ec0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:51.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.414+0000 7f68f0e73700 1 -- 172.21.15.144:0/669139854 shutdown_connections 2024-09-10T11:55:51.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.414+0000 7f68f0e73700 1 -- 172.21.15.144:0/669139854 wait complete. 2024-09-10T11:55:51.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.519+0000 7f4326cca700 1 -- 172.21.15.144:0/1565533117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43200ff270 msgr2=0x7f43200ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:51.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.519+0000 7f4326cca700 1 --2- 172.21.15.144:0/1565533117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43200ff270 0x7f43200ff6e0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f430c009910 tx=0x7f430c009c20 comp rx=0 tx=0).stop 2024-09-10T11:55:51.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.519+0000 7f4326cca700 1 -- 172.21.15.144:0/1565533117 shutdown_connections 2024-09-10T11:55:51.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.519+0000 7f4326cca700 1 --2- 172.21.15.144:0/1565533117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43200ff270 0x7f43200ff6e0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.519+0000 7f4326cca700 1 --2- 172.21.15.144:0/1565533117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43201049a0 0x7f4320104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.519+0000 7f4326cca700 1 -- 172.21.15.144:0/1565533117 >> 172.21.15.144:0/1565533117 conn(0x7f4320074850 msgr2=0x7f4320074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:51.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.519+0000 7f4326cca700 1 -- 172.21.15.144:0/1565533117 shutdown_connections 2024-09-10T11:55:51.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.519+0000 7f4326cca700 1 -- 172.21.15.144:0/1565533117 wait complete. 2024-09-10T11:55:51.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.519+0000 7f4326cca700 1 Processor -- start 2024-09-10T11:55:51.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.519+0000 7f4326cca700 1 -- start start 2024-09-10T11:55:51.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f4326cca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43200ff270 0x7f4320196be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:51.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f4326cca700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43201049a0 0x7f4320197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:51.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f4326cca700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4320197740 con 0x7f43200ff270 2024-09-10T11:55:51.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f4326cca700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43201978b0 con 0x7f43201049a0 2024-09-10T11:55:51.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f4324a66700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43200ff270 0x7f4320196be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:51.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f4324a66700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43200ff270 0x7f4320196be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38436/0 (socket says 172.21.15.144:38436) 2024-09-10T11:55:51.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f4324a66700 1 -- 172.21.15.144:0/3745528715 learned_addr learned my addr 172.21.15.144:0/3745528715 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:55:51.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f431ffff700 1 --2- 172.21.15.144:0/3745528715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43201049a0 0x7f4320197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:51.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f4324a66700 1 -- 172.21.15.144:0/3745528715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43201049a0 msgr2=0x7f4320197120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:51.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f4324a66700 1 --2- 172.21.15.144:0/3745528715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43201049a0 0x7f4320197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f4324a66700 1 -- 172.21.15.144:0/3745528715 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430c0095f0 con 0x7f43200ff270 2024-09-10T11:55:51.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f4324a66700 1 --2- 172.21.15.144:0/3745528715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43200ff270 0x7f4320196be0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f431400c8f0 tx=0x7f431400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:51.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f431dffb700 1 -- 172.21.15.144:0/3745528715 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4314007a70 con 0x7f43200ff270 2024-09-10T11:55:51.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f431dffb700 1 -- 172.21.15.144:0/3745528715 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4314004cf0 con 0x7f43200ff270 2024-09-10T11:55:51.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f4326cca700 1 -- 172.21.15.144:0/3745528715 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f432019fc50 con 0x7f43200ff270 2024-09-10T11:55:51.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.520+0000 7f431dffb700 1 -- 172.21.15.144:0/3745528715 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4314005610 con 0x7f43200ff270 2024-09-10T11:55:51.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.521+0000 7f4326cca700 1 -- 172.21.15.144:0/3745528715 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43201a0140 con 0x7f43200ff270 2024-09-10T11:55:51.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.522+0000 7f4326cca700 1 -- 172.21.15.144:0/3745528715 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432004ea50 con 0x7f43200ff270 2024-09-10T11:55:51.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.522+0000 7f431dffb700 1 -- 172.21.15.144:0/3745528715 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4314004750 con 0x7f43200ff270 2024-09-10T11:55:51.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.523+0000 7f431dffb700 1 --2- 172.21.15.144:0/3745528715 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f431006e200 0x7f43100706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:51.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.523+0000 7f431dffb700 1 -- 172.21.15.144:0/3745528715 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4314013070 con 0x7f43200ff270 2024-09-10T11:55:51.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.523+0000 7f431ffff700 1 --2- 172.21.15.144:0/3745528715 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f431006e200 0x7f43100706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:51.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.523+0000 7f431ffff700 1 --2- 172.21.15.144:0/3745528715 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f431006e200 0x7f43100706b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:51.528 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.527+0000 7f431dffb700 1 -- 172.21.15.144:0/3745528715 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f43140572c0 con 0x7f43200ff270 2024-09-10T11:55:51.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.771+0000 7f4326cca700 1 -- 172.21.15.144:0/3745528715 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f43201a05a0 con 0x7f43200ff270 2024-09-10T11:55:51.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.772+0000 7f431dffb700 1 -- 172.21.15.144:0/3745528715 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f431405a980 con 0x7f43200ff270 2024-09-10T11:55:51.774 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T11:55:51.774 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T11:55:51.774 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:55:51.774 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:55:51.774 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:55:51.774 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:55:51.774 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:55:51.774 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:55:51.774 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:55:51.774 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:55:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:55:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:55:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:55:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:55:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:55:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:55:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:55:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:55:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:55:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:55:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:55:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:55:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:55:51.775 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:55:51.776 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:55:51.776 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:55:51.776 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:55:51.776 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:55:51.776 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:55:51.776 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:55:51.776 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:55:51.776 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:55:51.776 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:55:51.776 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:55:51.776 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:55:51.777 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:55:51.777 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:55:51.777 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:55:51.777 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:55:51.777 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:55:51.777 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:55:51.777 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:55:51.777 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:55:51.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.774+0000 7f4326cca700 1 -- 172.21.15.144:0/3745528715 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f431006e200 msgr2=0x7f43100706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:51.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.774+0000 7f4326cca700 1 --2- 172.21.15.144:0/3745528715 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f431006e200 0x7f43100706b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f430c006010 tx=0x7f430c00b540 comp rx=0 tx=0).stop 2024-09-10T11:55:51.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.774+0000 7f4326cca700 1 -- 172.21.15.144:0/3745528715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43200ff270 msgr2=0x7f4320196be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:51.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.774+0000 7f4326cca700 1 --2- 172.21.15.144:0/3745528715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43200ff270 0x7f4320196be0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f431400c8f0 tx=0x7f431400ccb0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.774+0000 7f4326cca700 1 -- 172.21.15.144:0/3745528715 shutdown_connections 2024-09-10T11:55:51.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.774+0000 7f4326cca700 1 --2- 172.21.15.144:0/3745528715 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f431006e200 0x7f43100706b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.774+0000 7f4326cca700 1 --2- 172.21.15.144:0/3745528715 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f43200ff270 0x7f4320196be0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.774+0000 7f4326cca700 1 --2- 172.21.15.144:0/3745528715 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f43201049a0 0x7f4320197120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.774+0000 7f4326cca700 1 -- 172.21.15.144:0/3745528715 >> 172.21.15.144:0/3745528715 conn(0x7f4320074850 msgr2=0x7f4320108050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:51.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.774+0000 7f4326cca700 1 -- 172.21.15.144:0/3745528715 shutdown_connections 2024-09-10T11:55:51.780 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.774+0000 7f4326cca700 1 -- 172.21.15.144:0/3745528715 wait complete. 2024-09-10T11:55:51.780 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T11:55:51.883 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.880+0000 7f2e46706700 1 -- 172.21.15.144:0/4235313269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e401003e0 msgr2=0x7f2e401007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:51.883 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.880+0000 7f2e46706700 1 --2- 172.21.15.144:0/4235313269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e401003e0 0x7f2e401007b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f2e30009910 tx=0x7f2e30009c20 comp rx=0 tx=0).stop 2024-09-10T11:55:51.883 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.880+0000 7f2e46706700 1 -- 172.21.15.144:0/4235313269 shutdown_connections 2024-09-10T11:55:51.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.880+0000 7f2e46706700 1 --2- 172.21.15.144:0/4235313269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e40100d80 0x7f2e40104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.880+0000 7f2e46706700 1 --2- 172.21.15.144:0/4235313269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e401003e0 0x7f2e401007b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.880+0000 7f2e46706700 1 -- 172.21.15.144:0/4235313269 >> 172.21.15.144:0/4235313269 conn(0x7f2e400fbca0 msgr2=0x7f2e400fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:51.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.880+0000 7f2e46706700 1 -- 172.21.15.144:0/4235313269 shutdown_connections 2024-09-10T11:55:51.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.880+0000 7f2e46706700 1 -- 172.21.15.144:0/4235313269 wait complete. 2024-09-10T11:55:51.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.880+0000 7f2e46706700 1 Processor -- start 2024-09-10T11:55:51.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.880+0000 7f2e46706700 1 -- start start 2024-09-10T11:55:51.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e46706700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e401003e0 0x7f2e40195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:51.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e46706700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e40100d80 0x7f2e40196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:51.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e46706700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e40196940 con 0x7f2e401003e0 2024-09-10T11:55:51.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e46706700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4019a6d0 con 0x7f2e40100d80 2024-09-10T11:55:51.884 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e3ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e401003e0 0x7f2e40195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:51.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e3ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e401003e0 0x7f2e40195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38444/0 (socket says 172.21.15.144:38444) 2024-09-10T11:55:51.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e3ffff700 1 -- 172.21.15.144:0/1949884863 learned_addr learned my addr 172.21.15.144:0/1949884863 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:55:51.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e3f7fe700 1 --2- 172.21.15.144:0/1949884863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e40100d80 0x7f2e40196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:51.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e3ffff700 1 -- 172.21.15.144:0/1949884863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e40100d80 msgr2=0x7f2e40196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:51.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e3ffff700 1 --2- 172.21.15.144:0/1949884863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e40100d80 0x7f2e40196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:51.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e3ffff700 1 -- 172.21.15.144:0/1949884863 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e300095f0 con 0x7f2e401003e0 2024-09-10T11:55:51.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e3ffff700 1 --2- 172.21.15.144:0/1949884863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e401003e0 0x7f2e40195d20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2e30006010 tx=0x7f2e3000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:51.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e3d7fa700 1 -- 172.21.15.144:0/1949884863 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e3001d070 con 0x7f2e401003e0 2024-09-10T11:55:51.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e3d7fa700 1 -- 172.21.15.144:0/1949884863 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2e3000f460 con 0x7f2e401003e0 2024-09-10T11:55:51.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e46706700 1 -- 172.21.15.144:0/1949884863 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4019a950 con 0x7f2e401003e0 2024-09-10T11:55:51.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.881+0000 7f2e3d7fa700 1 -- 172.21.15.144:0/1949884863 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e30017630 con 0x7f2e401003e0 2024-09-10T11:55:51.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.882+0000 7f2e46706700 1 -- 172.21.15.144:0/1949884863 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4019ae40 con 0x7f2e401003e0 2024-09-10T11:55:51.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.882+0000 7f2e46706700 1 -- 172.21.15.144:0/1949884863 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e401086f0 con 0x7f2e401003e0 2024-09-10T11:55:51.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.883+0000 7f2e3d7fa700 1 -- 172.21.15.144:0/1949884863 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2e3000f5d0 con 0x7f2e401003e0 2024-09-10T11:55:51.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.883+0000 7f2e3d7fa700 1 --2- 172.21.15.144:0/1949884863 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2e2806e200 0x7f2e280706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:51.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.883+0000 7f2e3d7fa700 1 -- 172.21.15.144:0/1949884863 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2e3008f520 con 0x7f2e401003e0 2024-09-10T11:55:51.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.883+0000 7f2e3f7fe700 1 --2- 172.21.15.144:0/1949884863 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2e2806e200 0x7f2e280706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:51.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.884+0000 7f2e3f7fe700 1 --2- 172.21.15.144:0/1949884863 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2e2806e200 0x7f2e280706b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2e34005950 tx=0x7f2e340058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:51.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:51.888+0000 7f2e3d7fa700 1 -- 172.21.15.144:0/1949884863 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2e30058900 con 0x7f2e401003e0 2024-09-10T11:55:52.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.112+0000 7f2e46706700 1 -- 172.21.15.144:0/1949884863 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2e4002ccf0 con 0x7f2e2806e200 2024-09-10T11:55:52.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:51 smithi144 ceph-mon[87436]: from='client.35332 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:52.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:51 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/669139854' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:55:52.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:51 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3745528715' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:55:52.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.113+0000 7f2e3d7fa700 1 -- 172.21.15.144:0/1949884863 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2e4002ccf0 con 0x7f2e2806e200 2024-09-10T11:55:52.115 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:55:52.115 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:55:52.115 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:55:52.115 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:55:52.115 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:55:52.115 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:55:52.115 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T11:55:52.115 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T11:55:52.115 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:55:52.116 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T11:55:52.116 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:55:52.116 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T11:55:52.116 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:55:52.116 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:55:52.116 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:55:52.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.115+0000 7f2e46706700 1 -- 172.21.15.144:0/1949884863 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2e2806e200 msgr2=0x7f2e280706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:52.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.115+0000 7f2e46706700 1 --2- 172.21.15.144:0/1949884863 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2e2806e200 0x7f2e280706b0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2e34005950 tx=0x7f2e340058e0 comp rx=0 tx=0).stop 2024-09-10T11:55:52.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.115+0000 7f2e46706700 1 -- 172.21.15.144:0/1949884863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e401003e0 msgr2=0x7f2e40195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:52.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.115+0000 7f2e46706700 1 --2- 172.21.15.144:0/1949884863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e401003e0 0x7f2e40195d20 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2e30006010 tx=0x7f2e3000be20 comp rx=0 tx=0).stop 2024-09-10T11:55:52.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.116+0000 7f2e46706700 1 -- 172.21.15.144:0/1949884863 shutdown_connections 2024-09-10T11:55:52.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.116+0000 7f2e46706700 1 --2- 172.21.15.144:0/1949884863 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2e2806e200 0x7f2e280706b0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:52.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.116+0000 7f2e46706700 1 --2- 172.21.15.144:0/1949884863 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2e401003e0 0x7f2e40195d20 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:52.117 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.116+0000 7f2e46706700 1 --2- 172.21.15.144:0/1949884863 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2e40100d80 0x7f2e40196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:52.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.116+0000 7f2e46706700 1 -- 172.21.15.144:0/1949884863 >> 172.21.15.144:0/1949884863 conn(0x7f2e400fbca0 msgr2=0x7f2e400fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:52.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.116+0000 7f2e46706700 1 -- 172.21.15.144:0/1949884863 shutdown_connections 2024-09-10T11:55:52.118 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.116+0000 7f2e46706700 1 -- 172.21.15.144:0/1949884863 wait complete. 2024-09-10T11:55:52.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.220+0000 7fe288957700 1 -- 172.21.15.144:0/4291147086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2840ffc60 msgr2=0x7fe284103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:52.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.220+0000 7fe288957700 1 --2- 172.21.15.144:0/4291147086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2840ffc60 0x7fe284103c30 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fe26c009910 tx=0x7fe26c009c20 comp rx=0 tx=0).stop 2024-09-10T11:55:52.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.220+0000 7fe288957700 1 -- 172.21.15.144:0/4291147086 shutdown_connections 2024-09-10T11:55:52.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.220+0000 7fe288957700 1 --2- 172.21.15.144:0/4291147086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2840ffc60 0x7fe284103c30 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:52.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.220+0000 7fe288957700 1 --2- 172.21.15.144:0/4291147086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2840ff2c0 0x7fe2840ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:52.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.220+0000 7fe288957700 1 -- 172.21.15.144:0/4291147086 >> 172.21.15.144:0/4291147086 conn(0x7fe2840fab80 msgr2=0x7fe2840fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:52.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.220+0000 7fe288957700 1 -- 172.21.15.144:0/4291147086 shutdown_connections 2024-09-10T11:55:52.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.220+0000 7fe288957700 1 -- 172.21.15.144:0/4291147086 wait complete. 2024-09-10T11:55:52.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.221+0000 7fe288957700 1 Processor -- start 2024-09-10T11:55:52.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.221+0000 7fe288957700 1 -- start start 2024-09-10T11:55:52.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.221+0000 7fe288957700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2840ff2c0 0x7fe28419aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:52.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.221+0000 7fe288957700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2840ffc60 0x7fe28419af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:52.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.221+0000 7fe288957700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe28419b5f0 con 0x7fe2840ffc60 2024-09-10T11:55:52.224 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.221+0000 7fe288957700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe284194aa0 con 0x7fe2840ff2c0 2024-09-10T11:55:52.225 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.221+0000 7fe281d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2840ffc60 0x7fe28419af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:52.225 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.221+0000 7fe281d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2840ffc60 0x7fe28419af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38454/0 (socket says 172.21.15.144:38454) 2024-09-10T11:55:52.225 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.221+0000 7fe281d9b700 1 -- 172.21.15.144:0/3180179914 learned_addr learned my addr 172.21.15.144:0/3180179914 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:55:52.225 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.222+0000 7fe281d9b700 1 -- 172.21.15.144:0/3180179914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2840ff2c0 msgr2=0x7fe28419aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:52.225 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.222+0000 7fe281d9b700 1 --2- 172.21.15.144:0/3180179914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2840ff2c0 0x7fe28419aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:52.225 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.222+0000 7fe281d9b700 1 -- 172.21.15.144:0/3180179914 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c0095f0 con 0x7fe2840ffc60 2024-09-10T11:55:52.225 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.222+0000 7fe281d9b700 1 --2- 172.21.15.144:0/3180179914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2840ffc60 0x7fe28419af60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0092e0 tx=0x7fe26c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:52.225 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.222+0000 7fe27b7fe700 1 -- 172.21.15.144:0/3180179914 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe26c01d070 con 0x7fe2840ffc60 2024-09-10T11:55:52.225 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.222+0000 7fe27b7fe700 1 -- 172.21.15.144:0/3180179914 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe26c005dc0 con 0x7fe2840ffc60 2024-09-10T11:55:52.225 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.222+0000 7fe288957700 1 -- 172.21.15.144:0/3180179914 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe284194d20 con 0x7fe2840ffc60 2024-09-10T11:55:52.225 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.222+0000 7fe27b7fe700 1 -- 172.21.15.144:0/3180179914 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe26c017400 con 0x7fe2840ffc60 2024-09-10T11:55:52.225 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.222+0000 7fe288957700 1 -- 172.21.15.144:0/3180179914 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe284195210 con 0x7fe2840ffc60 2024-09-10T11:55:52.225 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.223+0000 7fe288957700 1 -- 172.21.15.144:0/3180179914 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe284107630 con 0x7fe2840ffc60 2024-09-10T11:55:52.226 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.223+0000 7fe27b7fe700 1 -- 172.21.15.144:0/3180179914 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe26c00f460 con 0x7fe2840ffc60 2024-09-10T11:55:52.226 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.224+0000 7fe27b7fe700 1 --2- 172.21.15.144:0/3180179914 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe27006e200 0x7fe2700706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:55:52.226 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.224+0000 7fe27b7fe700 1 -- 172.21.15.144:0/3180179914 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe26c08f4d0 con 0x7fe2840ffc60 2024-09-10T11:55:52.226 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.224+0000 7fe28259c700 1 --2- 172.21.15.144:0/3180179914 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe27006e200 0x7fe2700706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:55:52.226 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.224+0000 7fe28259c700 1 --2- 172.21.15.144:0/3180179914 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe27006e200 0x7fe2700706b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0060b0 tx=0x7fe27c006040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:55:52.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.228+0000 7fe27b7fe700 1 -- 172.21.15.144:0/3180179914 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe26c05c460 con 0x7fe2840ffc60 2024-09-10T11:55:52.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:51 smithi178 ceph-mon[65940]: from='client.35332 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:52.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:51 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/669139854' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:55:52.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:51 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3745528715' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:55:52.514 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.512+0000 7fe288957700 1 -- 172.21.15.144:0/3180179914 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe28402cc30 con 0x7fe2840ffc60 2024-09-10T11:55:52.514 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.513+0000 7fe27b7fe700 1 -- 172.21.15.144:0/3180179914 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7fe26c05bff0 con 0x7fe2840ffc60 2024-09-10T11:55:52.514 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:55:52.514 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T11:55:52.514 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:55:52.514 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T11:55:52.515 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:55:52.515 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T11:55:52.515 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T11:55:52.515 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T11:55:52.515 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T11:55:52.515 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T11:55:52.515 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T11:55:52.515 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T11:55:52.515 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T11:55:52.515 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T11:55:52.515 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T11:55:52.515 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T11:55:52.515 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T11:55:52.516 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T11:55:52.516 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:55:52.516 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:55:52.516 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:55:52.516 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:55:52.516 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:55:52.516 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:55:52.516 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:55:52.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.515+0000 7fe288957700 1 -- 172.21.15.144:0/3180179914 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe27006e200 msgr2=0x7fe2700706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:52.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.515+0000 7fe288957700 1 --2- 172.21.15.144:0/3180179914 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe27006e200 0x7fe2700706b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fe27c0060b0 tx=0x7fe27c006040 comp rx=0 tx=0).stop 2024-09-10T11:55:52.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.515+0000 7fe288957700 1 -- 172.21.15.144:0/3180179914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2840ffc60 msgr2=0x7fe28419af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:55:52.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.515+0000 7fe288957700 1 --2- 172.21.15.144:0/3180179914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2840ffc60 0x7fe28419af60 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fe26c0092e0 tx=0x7fe26c00be20 comp rx=0 tx=0).stop 2024-09-10T11:55:52.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.515+0000 7fe288957700 1 -- 172.21.15.144:0/3180179914 shutdown_connections 2024-09-10T11:55:52.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.515+0000 7fe288957700 1 --2- 172.21.15.144:0/3180179914 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe27006e200 0x7fe2700706b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:52.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.515+0000 7fe288957700 1 --2- 172.21.15.144:0/3180179914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe2840ff2c0 0x7fe28419aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:52.518 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.515+0000 7fe288957700 1 --2- 172.21.15.144:0/3180179914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe2840ffc60 0x7fe28419af60 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:55:52.518 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.515+0000 7fe288957700 1 -- 172.21.15.144:0/3180179914 >> 172.21.15.144:0/3180179914 conn(0x7fe2840fab80 msgr2=0x7fe2840fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:55:52.518 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.515+0000 7fe288957700 1 -- 172.21.15.144:0/3180179914 shutdown_connections 2024-09-10T11:55:52.518 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:55:52.515+0000 7fe288957700 1 -- 172.21.15.144:0/3180179914 wait complete. 2024-09-10T11:55:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:52 smithi178 ceph-mon[65940]: from='client.35344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:52 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3180179914' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:55:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:52 smithi178 ceph-mon[65940]: pgmap v718: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:53.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:52 smithi144 ceph-mon[87436]: from='client.35344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:55:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:52 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3180179914' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:55:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:52 smithi144 ceph-mon[87436]: pgmap v718: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:54 smithi144 ceph-mon[87436]: pgmap v719: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:54 smithi178 ceph-mon[65940]: pgmap v719: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:56 smithi178 ceph-mon[65940]: pgmap v720: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:57.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:56 smithi144 ceph-mon[87436]: pgmap v720: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:55:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:55:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:55:59.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:55:59 smithi144 ceph-mon[87436]: pgmap v721: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:00.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:55:59 smithi178 ceph-mon[65940]: pgmap v721: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:00 smithi178 ceph-mon[65940]: pgmap v722: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:01.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:00 smithi144 ceph-mon[87436]: pgmap v722: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:02 smithi178 ceph-mon[65940]: pgmap v723: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:03.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:02 smithi144 ceph-mon[87436]: pgmap v723: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:04 smithi178 ceph-mon[65940]: pgmap v724: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:05.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:04 smithi144 ceph-mon[87436]: pgmap v724: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:06 smithi178 ceph-mon[65940]: pgmap v725: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:07.438 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:06 smithi144 ceph-mon[87436]: pgmap v725: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:08 smithi178 ceph-mon[65940]: pgmap v726: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:09.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:08 smithi144 ceph-mon[87436]: pgmap v726: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:10 smithi178 ceph-mon[65940]: pgmap v727: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:11.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:10 smithi144 ceph-mon[87436]: pgmap v727: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:13.236 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:12 smithi144 ceph-mon[87436]: pgmap v728: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:12 smithi178 ceph-mon[65940]: pgmap v728: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:56:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:56:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:14 smithi178 ceph-mon[65940]: pgmap v729: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:15.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:14 smithi144 ceph-mon[87436]: pgmap v729: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:16 smithi178 ceph-mon[65940]: pgmap v730: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:17.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:16 smithi144 ceph-mon[87436]: pgmap v730: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:19.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:18 smithi178 ceph-mon[65940]: pgmap v731: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:19.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:18 smithi144 ceph-mon[87436]: pgmap v731: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:20 smithi178 ceph-mon[65940]: pgmap v732: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:21.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:20 smithi144 ceph-mon[87436]: pgmap v732: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:22.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.634+0000 7f50c1445700 1 -- 172.21.15.144:0/685049976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50bc0ff340 msgr2=0x7f50bc0ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:22.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.634+0000 7f50c1445700 1 --2- 172.21.15.144:0/685049976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50bc0ff340 0x7f50bc0ff710 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009910 tx=0x7f50ac009c20 comp rx=0 tx=0).stop 2024-09-10T11:56:22.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.634+0000 7f50c1445700 1 -- 172.21.15.144:0/685049976 shutdown_connections 2024-09-10T11:56:22.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.634+0000 7f50c1445700 1 --2- 172.21.15.144:0/685049976 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50bc0ffce0 0x7f50bc103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:22.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.634+0000 7f50c1445700 1 --2- 172.21.15.144:0/685049976 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50bc0ff340 0x7f50bc0ff710 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:22.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.634+0000 7f50c1445700 1 -- 172.21.15.144:0/685049976 >> 172.21.15.144:0/685049976 conn(0x7f50bc0fabe0 msgr2=0x7f50bc0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:22.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.634+0000 7f50c1445700 1 -- 172.21.15.144:0/685049976 shutdown_connections 2024-09-10T11:56:22.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.634+0000 7f50c1445700 1 -- 172.21.15.144:0/685049976 wait complete. 2024-09-10T11:56:22.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.635+0000 7f50c1445700 1 Processor -- start 2024-09-10T11:56:22.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.635+0000 7f50c1445700 1 -- start start 2024-09-10T11:56:22.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.635+0000 7f50c1445700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50bc0ff340 0x7f50bc100d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:22.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.635+0000 7f50c1445700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50bc0ffce0 0x7f50bc101290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.635+0000 7f50c1445700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc102b00 con 0x7f50bc0ff340 2024-09-10T11:56:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.635+0000 7f50c1445700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50bc102c70 con 0x7f50bc0ffce0 2024-09-10T11:56:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.635+0000 7f50baffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50bc0ff340 0x7f50bc100d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.635+0000 7f50baffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50bc0ff340 0x7f50bc100d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42264/0 (socket says 172.21.15.144:42264) 2024-09-10T11:56:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.635+0000 7f50baffd700 1 -- 172.21.15.144:0/2843202075 learned_addr learned my addr 172.21.15.144:0/2843202075 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:56:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.636+0000 7f50b3fff700 1 --2- 172.21.15.144:0/2843202075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50bc0ffce0 0x7f50bc101290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.636+0000 7f50baffd700 1 -- 172.21.15.144:0/2843202075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50bc0ffce0 msgr2=0x7f50bc101290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.636+0000 7f50baffd700 1 --2- 172.21.15.144:0/2843202075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50bc0ffce0 0x7f50bc101290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.636+0000 7f50baffd700 1 -- 172.21.15.144:0/2843202075 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac0095f0 con 0x7f50bc0ff340 2024-09-10T11:56:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.636+0000 7f50b3fff700 1 --2- 172.21.15.144:0/2843202075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50bc0ffce0 0x7f50bc101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:56:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.636+0000 7f50baffd700 1 --2- 172.21.15.144:0/2843202075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50bc0ff340 0x7f50bc100d50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f50ac005f50 tx=0x7f50ac00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:22.639 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.636+0000 7f50b8ff9700 1 -- 172.21.15.144:0/2843202075 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50ac01d070 con 0x7f50bc0ff340 2024-09-10T11:56:22.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.636+0000 7f50b8ff9700 1 -- 172.21.15.144:0/2843202075 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50ac00b610 con 0x7f50bc0ff340 2024-09-10T11:56:22.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.636+0000 7f50c1445700 1 -- 172.21.15.144:0/2843202075 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50bc101830 con 0x7f50bc0ff340 2024-09-10T11:56:22.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.636+0000 7f50b8ff9700 1 -- 172.21.15.144:0/2843202075 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50ac022470 con 0x7f50bc0ff340 2024-09-10T11:56:22.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.636+0000 7f50c1445700 1 -- 172.21.15.144:0/2843202075 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50bc1a6750 con 0x7f50bc0ff340 2024-09-10T11:56:22.640 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.637+0000 7f50c1445700 1 -- 172.21.15.144:0/2843202075 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50bc04ea50 con 0x7f50bc0ff340 2024-09-10T11:56:22.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.638+0000 7f50b8ff9700 1 -- 172.21.15.144:0/2843202075 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f50ac00fca0 con 0x7f50bc0ff340 2024-09-10T11:56:22.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.638+0000 7f50b8ff9700 1 --2- 172.21.15.144:0/2843202075 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f50a806e200 0x7f50a80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:22.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.638+0000 7f50b8ff9700 1 -- 172.21.15.144:0/2843202075 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f50ac08f460 con 0x7f50bc0ff340 2024-09-10T11:56:22.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.639+0000 7f50b3fff700 1 --2- 172.21.15.144:0/2843202075 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f50a806e200 0x7f50a80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:22.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.639+0000 7f50b3fff700 1 --2- 172.21.15.144:0/2843202075 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f50a806e200 0x7f50a80706b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f50bc102540 tx=0x7f50a4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:22.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.643+0000 7f50b8ff9700 1 -- 172.21.15.144:0/2843202075 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f50ac0588c0 con 0x7f50bc0ff340 2024-09-10T11:56:22.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.865+0000 7f50c1445700 1 -- 172.21.15.144:0/2843202075 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f50bc0fb950 con 0x7f50a806e200 2024-09-10T11:56:22.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.867+0000 7f50b8ff9700 1 -- 172.21.15.144:0/2843202075 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f50bc0fb950 con 0x7f50a806e200 2024-09-10T11:56:22.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.869+0000 7f50c1445700 1 -- 172.21.15.144:0/2843202075 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f50a806e200 msgr2=0x7f50a80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:22.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.869+0000 7f50c1445700 1 --2- 172.21.15.144:0/2843202075 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f50a806e200 0x7f50a80706b0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f50bc102540 tx=0x7f50a4009260 comp rx=0 tx=0).stop 2024-09-10T11:56:22.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.869+0000 7f50c1445700 1 -- 172.21.15.144:0/2843202075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50bc0ff340 msgr2=0x7f50bc100d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:22.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.869+0000 7f50c1445700 1 --2- 172.21.15.144:0/2843202075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50bc0ff340 0x7f50bc100d50 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f50ac005f50 tx=0x7f50ac00f740 comp rx=0 tx=0).stop 2024-09-10T11:56:22.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.869+0000 7f50c1445700 1 -- 172.21.15.144:0/2843202075 shutdown_connections 2024-09-10T11:56:22.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.869+0000 7f50c1445700 1 --2- 172.21.15.144:0/2843202075 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f50a806e200 0x7f50a80706b0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:22.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.869+0000 7f50c1445700 1 --2- 172.21.15.144:0/2843202075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50bc0ff340 0x7f50bc100d50 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:22.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.869+0000 7f50c1445700 1 --2- 172.21.15.144:0/2843202075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50bc0ffce0 0x7f50bc101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:22.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.869+0000 7f50c1445700 1 -- 172.21.15.144:0/2843202075 >> 172.21.15.144:0/2843202075 conn(0x7f50bc0fabe0 msgr2=0x7f50bc103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:22.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.869+0000 7f50c1445700 1 -- 172.21.15.144:0/2843202075 shutdown_connections 2024-09-10T11:56:22.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.869+0000 7f50c1445700 1 -- 172.21.15.144:0/2843202075 wait complete. 2024-09-10T11:56:22.882 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:56:22.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.978+0000 7f6c0a2c1700 1 -- 172.21.15.144:0/3885526304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c040ff2c0 msgr2=0x7f6c040ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:22.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.978+0000 7f6c0a2c1700 1 --2- 172.21.15.144:0/3885526304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c040ff2c0 0x7f6c040ff690 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4009910 tx=0x7f6bf4009c20 comp rx=0 tx=0).stop 2024-09-10T11:56:22.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.978+0000 7f6c0a2c1700 1 -- 172.21.15.144:0/3885526304 shutdown_connections 2024-09-10T11:56:22.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.978+0000 7f6c0a2c1700 1 --2- 172.21.15.144:0/3885526304 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c040ffc60 0x7f6c04103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:22.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.978+0000 7f6c0a2c1700 1 --2- 172.21.15.144:0/3885526304 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c040ff2c0 0x7f6c040ff690 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:22.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.978+0000 7f6c0a2c1700 1 -- 172.21.15.144:0/3885526304 >> 172.21.15.144:0/3885526304 conn(0x7f6c040fab80 msgr2=0x7f6c040fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:22.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.978+0000 7f6c0a2c1700 1 -- 172.21.15.144:0/3885526304 shutdown_connections 2024-09-10T11:56:22.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.978+0000 7f6c0a2c1700 1 -- 172.21.15.144:0/3885526304 wait complete. 2024-09-10T11:56:22.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.979+0000 7f6c0a2c1700 1 Processor -- start 2024-09-10T11:56:22.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.979+0000 7f6c0a2c1700 1 -- start start 2024-09-10T11:56:22.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.979+0000 7f6c0a2c1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c040ff2c0 0x7f6c04195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:22.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.979+0000 7f6c0a2c1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c040ffc60 0x7f6c041962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:22.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.979+0000 7f6c0a2c1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c04196990 con 0x7f6c040ffc60 2024-09-10T11:56:22.982 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.979+0000 7f6c0a2c1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0419a6d0 con 0x7f6c040ff2c0 2024-09-10T11:56:22.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.979+0000 7f6c037fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c040ffc60 0x7f6c041962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:22.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.979+0000 7f6c037fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c040ffc60 0x7f6c041962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42282/0 (socket says 172.21.15.144:42282) 2024-09-10T11:56:22.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.979+0000 7f6c037fe700 1 -- 172.21.15.144:0/1211998234 learned_addr learned my addr 172.21.15.144:0/1211998234 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:56:22.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.979+0000 7f6c03fff700 1 --2- 172.21.15.144:0/1211998234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c040ff2c0 0x7f6c04195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:22.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.979+0000 7f6c037fe700 1 -- 172.21.15.144:0/1211998234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c040ff2c0 msgr2=0x7f6c04195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:22.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.979+0000 7f6c037fe700 1 --2- 172.21.15.144:0/1211998234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c040ff2c0 0x7f6c04195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:22.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.979+0000 7f6c037fe700 1 -- 172.21.15.144:0/1211998234 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf40095f0 con 0x7f6c040ffc60 2024-09-10T11:56:22.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.980+0000 7f6c037fe700 1 --2- 172.21.15.144:0/1211998234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c040ffc60 0x7f6c041962b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800d900 tx=0x7f6bf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:22.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.980+0000 7f6c017fa700 1 -- 172.21.15.144:0/1211998234 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bf8009900 con 0x7f6c040ffc60 2024-09-10T11:56:22.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.980+0000 7f6c017fa700 1 -- 172.21.15.144:0/1211998234 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bf8010460 con 0x7f6c040ffc60 2024-09-10T11:56:22.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.980+0000 7f6c0a2c1700 1 -- 172.21.15.144:0/1211998234 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0419a9b0 con 0x7f6c040ffc60 2024-09-10T11:56:22.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.980+0000 7f6c017fa700 1 -- 172.21.15.144:0/1211998234 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bf800f5f0 con 0x7f6c040ffc60 2024-09-10T11:56:22.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.980+0000 7f6c0a2c1700 1 -- 172.21.15.144:0/1211998234 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0419af00 con 0x7f6c040ffc60 2024-09-10T11:56:22.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.981+0000 7f6c0a2c1700 1 -- 172.21.15.144:0/1211998234 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c041085e0 con 0x7f6c040ffc60 2024-09-10T11:56:22.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.981+0000 7f6c017fa700 1 -- 172.21.15.144:0/1211998234 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6bf800f820 con 0x7f6c040ffc60 2024-09-10T11:56:22.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.982+0000 7f6c017fa700 1 --2- 172.21.15.144:0/1211998234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bec06e200 0x7f6bec0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:22.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.982+0000 7f6c017fa700 1 -- 172.21.15.144:0/1211998234 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6bf808dc00 con 0x7f6c040ffc60 2024-09-10T11:56:22.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.982+0000 7f6c03fff700 1 --2- 172.21.15.144:0/1211998234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bec06e200 0x7f6bec0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:22.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.982+0000 7f6c03fff700 1 --2- 172.21.15.144:0/1211998234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bec06e200 0x7f6bec0706b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4000c00 tx=0x7f6bf4005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:22.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:22.986+0000 7f6c017fa700 1 -- 172.21.15.144:0/1211998234 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6bf8057060 con 0x7f6c040ffc60 2024-09-10T11:56:23.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.205+0000 7f6c0a2c1700 1 -- 172.21.15.144:0/1211998234 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c0419b2a0 con 0x7f6bec06e200 2024-09-10T11:56:23.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.206+0000 7f6c017fa700 1 -- 172.21.15.144:0/1211998234 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6c0419b2a0 con 0x7f6bec06e200 2024-09-10T11:56:23.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.208+0000 7f6c0a2c1700 1 -- 172.21.15.144:0/1211998234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bec06e200 msgr2=0x7f6bec0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:23.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.208+0000 7f6c0a2c1700 1 --2- 172.21.15.144:0/1211998234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bec06e200 0x7f6bec0706b0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6bf4000c00 tx=0x7f6bf4005fd0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.208+0000 7f6c0a2c1700 1 -- 172.21.15.144:0/1211998234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c040ffc60 msgr2=0x7f6c041962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:23.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.208+0000 7f6c0a2c1700 1 --2- 172.21.15.144:0/1211998234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c040ffc60 0x7f6c041962b0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f6bf800d900 tx=0x7f6bf800dcc0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.208+0000 7f6c0a2c1700 1 -- 172.21.15.144:0/1211998234 shutdown_connections 2024-09-10T11:56:23.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.208+0000 7f6c0a2c1700 1 --2- 172.21.15.144:0/1211998234 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bec06e200 0x7f6bec0706b0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.208+0000 7f6c0a2c1700 1 --2- 172.21.15.144:0/1211998234 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6c040ff2c0 0x7f6c04195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.208+0000 7f6c0a2c1700 1 --2- 172.21.15.144:0/1211998234 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6c040ffc60 0x7f6c041962b0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.208+0000 7f6c0a2c1700 1 -- 172.21.15.144:0/1211998234 >> 172.21.15.144:0/1211998234 conn(0x7f6c040fab80 msgr2=0x7f6c04068ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:23.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.209+0000 7f6c0a2c1700 1 -- 172.21.15.144:0/1211998234 shutdown_connections 2024-09-10T11:56:23.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.209+0000 7f6c0a2c1700 1 -- 172.21.15.144:0/1211998234 wait complete. 2024-09-10T11:56:23.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.313+0000 7f784248d700 1 -- 172.21.15.144:0/1532161015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f783c0ff2d0 msgr2=0x7f783c0ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:23.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.313+0000 7f784248d700 1 --2- 172.21.15.144:0/1532161015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f783c0ff2d0 0x7f783c0ff6a0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f782c009910 tx=0x7f782c009c20 comp rx=0 tx=0).stop 2024-09-10T11:56:23.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.313+0000 7f784248d700 1 -- 172.21.15.144:0/1532161015 shutdown_connections 2024-09-10T11:56:23.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.313+0000 7f784248d700 1 --2- 172.21.15.144:0/1532161015 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f783c0ffc70 0x7f783c103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.313+0000 7f784248d700 1 --2- 172.21.15.144:0/1532161015 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f783c0ff2d0 0x7f783c0ff6a0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.313+0000 7f784248d700 1 -- 172.21.15.144:0/1532161015 >> 172.21.15.144:0/1532161015 conn(0x7f783c0fabb0 msgr2=0x7f783c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:23.315 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:22 smithi144 ceph-mon[87436]: from='client.35352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:23.315 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:22 smithi144 ceph-mon[87436]: pgmap v733: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:23.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.313+0000 7f784248d700 1 -- 172.21.15.144:0/1532161015 shutdown_connections 2024-09-10T11:56:23.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.313+0000 7f784248d700 1 -- 172.21.15.144:0/1532161015 wait complete. 2024-09-10T11:56:23.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.314+0000 7f784248d700 1 Processor -- start 2024-09-10T11:56:23.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.314+0000 7f784248d700 1 -- start start 2024-09-10T11:56:23.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.314+0000 7f784248d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f783c0ff2d0 0x7f783c195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:23.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.314+0000 7f784248d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f783c0ffc70 0x7f783c1962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:23.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.314+0000 7f784248d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c1969a0 con 0x7f783c0ffc70 2024-09-10T11:56:23.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.314+0000 7f784248d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f783c19a730 con 0x7f783c0ff2d0 2024-09-10T11:56:23.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.314+0000 7f7833fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f783c0ffc70 0x7f783c1962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:23.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.314+0000 7f7833fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f783c0ffc70 0x7f783c1962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42294/0 (socket says 172.21.15.144:42294) 2024-09-10T11:56:23.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.314+0000 7f7833fff700 1 -- 172.21.15.144:0/2371910469 learned_addr learned my addr 172.21.15.144:0/2371910469 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:56:23.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.314+0000 7f783bfff700 1 --2- 172.21.15.144:0/2371910469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f783c0ff2d0 0x7f783c195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:23.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.314+0000 7f7833fff700 1 -- 172.21.15.144:0/2371910469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f783c0ff2d0 msgr2=0x7f783c195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:23.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.314+0000 7f7833fff700 1 --2- 172.21.15.144:0/2371910469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f783c0ff2d0 0x7f783c195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.314+0000 7f7833fff700 1 -- 172.21.15.144:0/2371910469 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c0095f0 con 0x7f783c0ffc70 2024-09-10T11:56:23.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.314+0000 7f783bfff700 1 --2- 172.21.15.144:0/2371910469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f783c0ff2d0 0x7f783c195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:56:23.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.315+0000 7f7833fff700 1 --2- 172.21.15.144:0/2371910469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f783c0ffc70 0x7f783c1962c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f782400e8f0 tx=0x7f782400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:23.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.315+0000 7f7839ffb700 1 -- 172.21.15.144:0/2371910469 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7824009790 con 0x7f783c0ffc70 2024-09-10T11:56:23.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.315+0000 7f7839ffb700 1 -- 172.21.15.144:0/2371910469 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7824004cf0 con 0x7f783c0ffc70 2024-09-10T11:56:23.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.315+0000 7f7839ffb700 1 -- 172.21.15.144:0/2371910469 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7824005690 con 0x7f783c0ffc70 2024-09-10T11:56:23.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.315+0000 7f784248d700 1 -- 172.21.15.144:0/2371910469 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f783c19aa10 con 0x7f783c0ffc70 2024-09-10T11:56:23.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.315+0000 7f784248d700 1 -- 172.21.15.144:0/2371910469 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f783c19af10 con 0x7f783c0ffc70 2024-09-10T11:56:23.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.316+0000 7f784248d700 1 -- 172.21.15.144:0/2371910469 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783c04ea50 con 0x7f783c0ffc70 2024-09-10T11:56:23.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.316+0000 7f7839ffb700 1 -- 172.21.15.144:0/2371910469 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7824004750 con 0x7f783c0ffc70 2024-09-10T11:56:23.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.317+0000 7f7839ffb700 1 --2- 172.21.15.144:0/2371910469 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f782806e200 0x7f78280706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:23.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.317+0000 7f7839ffb700 1 -- 172.21.15.144:0/2371910469 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7824014070 con 0x7f783c0ffc70 2024-09-10T11:56:23.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.317+0000 7f783bfff700 1 --2- 172.21.15.144:0/2371910469 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f782806e200 0x7f78280706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:23.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.317+0000 7f783bfff700 1 --2- 172.21.15.144:0/2371910469 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f782806e200 0x7f78280706b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f782c00b760 tx=0x7f782c005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:23.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.322+0000 7f7839ffb700 1 -- 172.21.15.144:0/2371910469 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7824057860 con 0x7f783c0ffc70 2024-09-10T11:56:23.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:22 smithi178 ceph-mon[65940]: from='client.35352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:23.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:22 smithi178 ceph-mon[65940]: pgmap v733: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:23.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.533+0000 7f784248d700 1 -- 172.21.15.144:0/2371910469 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f783c19b4a0 con 0x7f782806e200 2024-09-10T11:56:23.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.543+0000 7f7839ffb700 1 -- 172.21.15.144:0/2371910469 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f783c19b4a0 con 0x7f782806e200 2024-09-10T11:56:23.544 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:56:23.544 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (24m) 2m ago 35m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:56:23.544 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (35m) 2m ago 35m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:56:23.544 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (34m) 2m ago 34m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:56:23.544 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (22m) 2m ago 35m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:56:23.544 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (22m) 2m ago 34m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:56:23.544 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (24m) 2m ago 34m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:56:23.545 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (18m) 2m ago 29m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:56:23.545 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (18m) 2m ago 29m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:56:23.545 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (3m) 2m ago 29m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T11:56:23.545 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (18m) 2m ago 29m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:56:23.545 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (26m) 2m ago 36m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:56:23.545 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (25m) 2m ago 34m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:56:23.545 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (23m) 2m ago 36m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:56:23.545 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (22m) 2m ago 34m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:56:23.545 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (25m) 2m ago 35m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:56:23.545 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (25m) 2m ago 34m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:56:23.545 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (21m) 2m ago 33m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:56:23.545 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (21m) 2m ago 32m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:56:23.546 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (20m) 2m ago 32m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:56:23.546 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (20m) 2m ago 32m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:56:23.546 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (19m) 2m ago 31m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:56:23.546 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (18m) 2m ago 31m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:56:23.546 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (24m) 2m ago 34m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:56:23.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.545+0000 7f784248d700 1 -- 172.21.15.144:0/2371910469 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f782806e200 msgr2=0x7f78280706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:23.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.545+0000 7f784248d700 1 --2- 172.21.15.144:0/2371910469 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f782806e200 0x7f78280706b0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f782c00b760 tx=0x7f782c005a90 comp rx=0 tx=0).stop 2024-09-10T11:56:23.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.545+0000 7f784248d700 1 -- 172.21.15.144:0/2371910469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f783c0ffc70 msgr2=0x7f783c1962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:23.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.545+0000 7f784248d700 1 --2- 172.21.15.144:0/2371910469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f783c0ffc70 0x7f783c1962c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f782400e8f0 tx=0x7f782400ecb0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.545+0000 7f784248d700 1 -- 172.21.15.144:0/2371910469 shutdown_connections 2024-09-10T11:56:23.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.545+0000 7f784248d700 1 --2- 172.21.15.144:0/2371910469 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f782806e200 0x7f78280706b0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.545+0000 7f784248d700 1 --2- 172.21.15.144:0/2371910469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f783c0ff2d0 0x7f783c195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.545+0000 7f784248d700 1 --2- 172.21.15.144:0/2371910469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f783c0ffc70 0x7f783c1962c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.545+0000 7f784248d700 1 -- 172.21.15.144:0/2371910469 >> 172.21.15.144:0/2371910469 conn(0x7f783c0fabb0 msgr2=0x7f783c068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:23.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.545+0000 7f784248d700 1 -- 172.21.15.144:0/2371910469 shutdown_connections 2024-09-10T11:56:23.547 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.545+0000 7f784248d700 1 -- 172.21.15.144:0/2371910469 wait complete. 2024-09-10T11:56:23.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.649+0000 7f6aba4e9700 1 -- 172.21.15.144:0/932561022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab4101330 msgr2=0x7f6ab4101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:23.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.649+0000 7f6aba4e9700 1 --2- 172.21.15.144:0/932561022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab4101330 0x7f6ab4101700 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f6aa4009940 tx=0x7f6aa4009c50 comp rx=0 tx=0).stop 2024-09-10T11:56:23.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.649+0000 7f6aba4e9700 1 -- 172.21.15.144:0/932561022 shutdown_connections 2024-09-10T11:56:23.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.649+0000 7f6aba4e9700 1 --2- 172.21.15.144:0/932561022 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab4101c40 0x7f6ab410a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.649+0000 7f6aba4e9700 1 --2- 172.21.15.144:0/932561022 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab4101330 0x7f6ab4101700 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.649+0000 7f6aba4e9700 1 -- 172.21.15.144:0/932561022 >> 172.21.15.144:0/932561022 conn(0x7f6ab40fabb0 msgr2=0x7f6ab40fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:23.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.649+0000 7f6aba4e9700 1 -- 172.21.15.144:0/932561022 shutdown_connections 2024-09-10T11:56:23.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.649+0000 7f6aba4e9700 1 -- 172.21.15.144:0/932561022 wait complete. 2024-09-10T11:56:23.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.650+0000 7f6aba4e9700 1 Processor -- start 2024-09-10T11:56:23.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.650+0000 7f6aba4e9700 1 -- start start 2024-09-10T11:56:23.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.650+0000 7f6aba4e9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab4101330 0x7f6ab419bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:23.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.650+0000 7f6aba4e9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab4101c40 0x7f6ab419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:23.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.650+0000 7f6aba4e9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab419c690 con 0x7f6ab4101330 2024-09-10T11:56:23.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.650+0000 7f6aba4e9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ab4195b40 con 0x7f6ab4101c40 2024-09-10T11:56:23.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.650+0000 7f6ab3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab4101330 0x7f6ab419bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:23.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.650+0000 7f6ab3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab4101330 0x7f6ab419bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42314/0 (socket says 172.21.15.144:42314) 2024-09-10T11:56:23.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.650+0000 7f6ab3fff700 1 -- 172.21.15.144:0/3686827546 learned_addr learned my addr 172.21.15.144:0/3686827546 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:56:23.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.650+0000 7f6aabfff700 1 --2- 172.21.15.144:0/3686827546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab4101c40 0x7f6ab419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:23.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.651+0000 7f6ab3fff700 1 -- 172.21.15.144:0/3686827546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab4101c40 msgr2=0x7f6ab419c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:23.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.651+0000 7f6ab3fff700 1 --2- 172.21.15.144:0/3686827546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab4101c40 0x7f6ab419c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.651+0000 7f6ab3fff700 1 -- 172.21.15.144:0/3686827546 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aa40095f0 con 0x7f6ab4101330 2024-09-10T11:56:23.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.651+0000 7f6ab3fff700 1 --2- 172.21.15.144:0/3686827546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab4101330 0x7f6ab419bac0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40055f0 tx=0x7f6aa40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:23.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.651+0000 7f6ab1ffb700 1 -- 172.21.15.144:0/3686827546 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6aa401d070 con 0x7f6ab4101330 2024-09-10T11:56:23.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.651+0000 7f6ab1ffb700 1 -- 172.21.15.144:0/3686827546 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6aa400f460 con 0x7f6ab4101330 2024-09-10T11:56:23.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.651+0000 7f6aba4e9700 1 -- 172.21.15.144:0/3686827546 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ab4195dc0 con 0x7f6ab4101330 2024-09-10T11:56:23.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.651+0000 7f6ab1ffb700 1 -- 172.21.15.144:0/3686827546 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6aa4021640 con 0x7f6ab4101330 2024-09-10T11:56:23.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.651+0000 7f6aba4e9700 1 -- 172.21.15.144:0/3686827546 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ab41962b0 con 0x7f6ab4101330 2024-09-10T11:56:23.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.652+0000 7f6aba4e9700 1 -- 172.21.15.144:0/3686827546 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ab41077e0 con 0x7f6ab4101330 2024-09-10T11:56:23.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.653+0000 7f6ab1ffb700 1 -- 172.21.15.144:0/3686827546 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6aa400fa60 con 0x7f6ab4101330 2024-09-10T11:56:23.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.653+0000 7f6ab1ffb700 1 --2- 172.21.15.144:0/3686827546 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6aa006e200 0x7f6aa00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:23.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.653+0000 7f6ab1ffb700 1 -- 172.21.15.144:0/3686827546 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6aa408f2f0 con 0x7f6ab4101330 2024-09-10T11:56:23.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.653+0000 7f6aabfff700 1 --2- 172.21.15.144:0/3686827546 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6aa006e200 0x7f6aa00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:23.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.654+0000 7f6aabfff700 1 --2- 172.21.15.144:0/3686827546 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6aa006e200 0x7f6aa00706b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c005950 tx=0x7f6a9c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:23.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.657+0000 7f6ab1ffb700 1 -- 172.21.15.144:0/3686827546 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6aa405c200 con 0x7f6ab4101330 2024-09-10T11:56:23.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.947+0000 7f6aba4e9700 1 -- 172.21.15.144:0/3686827546 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6ab404ea50 con 0x7f6ab4101330 2024-09-10T11:56:23.949 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.948+0000 7f6ab1ffb700 1 -- 172.21.15.144:0/3686827546 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f6aa405bd90 con 0x7f6ab4101330 2024-09-10T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:56:23.950 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:56:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:56:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:56:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:56:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:56:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:56:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:56:23.951 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:56:23.952 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T11:56:23.952 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:56:23.952 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:56:23.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.950+0000 7f6aba4e9700 1 -- 172.21.15.144:0/3686827546 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6aa006e200 msgr2=0x7f6aa00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:23.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.950+0000 7f6aba4e9700 1 --2- 172.21.15.144:0/3686827546 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6aa006e200 0x7f6aa00706b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6a9c005950 tx=0x7f6a9c0058e0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.950+0000 7f6aba4e9700 1 -- 172.21.15.144:0/3686827546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab4101330 msgr2=0x7f6ab419bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:23.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.950+0000 7f6aba4e9700 1 --2- 172.21.15.144:0/3686827546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab4101330 0x7f6ab419bac0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6aa40055f0 tx=0x7f6aa40056d0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.950+0000 7f6aba4e9700 1 -- 172.21.15.144:0/3686827546 shutdown_connections 2024-09-10T11:56:23.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.950+0000 7f6aba4e9700 1 --2- 172.21.15.144:0/3686827546 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6aa006e200 0x7f6aa00706b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.950+0000 7f6aba4e9700 1 --2- 172.21.15.144:0/3686827546 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6ab4101330 0x7f6ab419bac0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.950+0000 7f6aba4e9700 1 --2- 172.21.15.144:0/3686827546 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6ab4101c40 0x7f6ab419c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:23.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.950+0000 7f6aba4e9700 1 -- 172.21.15.144:0/3686827546 >> 172.21.15.144:0/3686827546 conn(0x7f6ab40fabb0 msgr2=0x7f6ab4104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:23.954 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.950+0000 7f6aba4e9700 1 -- 172.21.15.144:0/3686827546 shutdown_connections 2024-09-10T11:56:23.955 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:23.950+0000 7f6aba4e9700 1 -- 172.21.15.144:0/3686827546 wait complete. 2024-09-10T11:56:24.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.058+0000 7f3f474f4700 1 -- 172.21.15.144:0/49881547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f400ffc70 msgr2=0x7f3f40103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:24.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.058+0000 7f3f474f4700 1 --2- 172.21.15.144:0/49881547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f400ffc70 0x7f3f40103c40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f3f34009910 tx=0x7f3f34009c20 comp rx=0 tx=0).stop 2024-09-10T11:56:24.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.058+0000 7f3f474f4700 1 -- 172.21.15.144:0/49881547 shutdown_connections 2024-09-10T11:56:24.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.058+0000 7f3f474f4700 1 --2- 172.21.15.144:0/49881547 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f400ffc70 0x7f3f40103c40 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.058+0000 7f3f474f4700 1 --2- 172.21.15.144:0/49881547 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f400ff2d0 0x7f3f400ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.058+0000 7f3f474f4700 1 -- 172.21.15.144:0/49881547 >> 172.21.15.144:0/49881547 conn(0x7f3f400fabb0 msgr2=0x7f3f400fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:24.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.058+0000 7f3f474f4700 1 -- 172.21.15.144:0/49881547 shutdown_connections 2024-09-10T11:56:24.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.058+0000 7f3f474f4700 1 -- 172.21.15.144:0/49881547 wait complete. 2024-09-10T11:56:24.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.059+0000 7f3f474f4700 1 Processor -- start 2024-09-10T11:56:24.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.059+0000 7f3f474f4700 1 -- start start 2024-09-10T11:56:24.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.059+0000 7f3f474f4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f400ff2d0 0x7f3f40196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:24.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.059+0000 7f3f474f4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f400ffc70 0x7f3f401971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:24.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.059+0000 7f3f474f4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f40197800 con 0x7f3f400ff2d0 2024-09-10T11:56:24.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.059+0000 7f3f474f4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f40197970 con 0x7f3f400ffc70 2024-09-10T11:56:24.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.059+0000 7f3f45290700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f400ff2d0 0x7f3f40196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:24.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.059+0000 7f3f45290700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f400ff2d0 0x7f3f40196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42332/0 (socket says 172.21.15.144:42332) 2024-09-10T11:56:24.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.059+0000 7f3f45290700 1 -- 172.21.15.144:0/1892601072 learned_addr learned my addr 172.21.15.144:0/1892601072 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:56:24.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.060+0000 7f3f44a8f700 1 --2- 172.21.15.144:0/1892601072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f400ffc70 0x7f3f401971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:24.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.060+0000 7f3f45290700 1 -- 172.21.15.144:0/1892601072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f400ffc70 msgr2=0x7f3f401971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:24.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.060+0000 7f3f45290700 1 --2- 172.21.15.144:0/1892601072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f400ffc70 0x7f3f401971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.060+0000 7f3f45290700 1 -- 172.21.15.144:0/1892601072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f340095f0 con 0x7f3f400ff2d0 2024-09-10T11:56:24.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.060+0000 7f3f45290700 1 --2- 172.21.15.144:0/1892601072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f400ff2d0 0x7f3f40196ca0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000ba50 tx=0x7f3f3000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:24.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.060+0000 7f3f3e7fc700 1 -- 172.21.15.144:0/1892601072 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f3000ca00 con 0x7f3f400ff2d0 2024-09-10T11:56:24.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.060+0000 7f3f3e7fc700 1 -- 172.21.15.144:0/1892601072 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f3000cb60 con 0x7f3f400ff2d0 2024-09-10T11:56:24.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.060+0000 7f3f474f4700 1 -- 172.21.15.144:0/1892601072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4019fd10 con 0x7f3f400ff2d0 2024-09-10T11:56:24.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.060+0000 7f3f3e7fc700 1 -- 172.21.15.144:0/1892601072 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f300075a0 con 0x7f3f400ff2d0 2024-09-10T11:56:24.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.060+0000 7f3f474f4700 1 -- 172.21.15.144:0/1892601072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f401a01d0 con 0x7f3f400ff2d0 2024-09-10T11:56:24.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.061+0000 7f3f474f4700 1 -- 172.21.15.144:0/1892601072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f40107590 con 0x7f3f400ff2d0 2024-09-10T11:56:24.064 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.062+0000 7f3f3e7fc700 1 -- 172.21.15.144:0/1892601072 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3f3000ccd0 con 0x7f3f400ff2d0 2024-09-10T11:56:24.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.062+0000 7f3f3e7fc700 1 --2- 172.21.15.144:0/1892601072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f2806e200 0x7f3f280706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:24.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.062+0000 7f3f3e7fc700 1 -- 172.21.15.144:0/1892601072 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3f30006e40 con 0x7f3f400ff2d0 2024-09-10T11:56:24.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.062+0000 7f3f44a8f700 1 --2- 172.21.15.144:0/1892601072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f2806e200 0x7f3f280706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:24.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.063+0000 7f3f44a8f700 1 --2- 172.21.15.144:0/1892601072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f2806e200 0x7f3f280706b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3f34006010 tx=0x7f3f3401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:24.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.066+0000 7f3f3e7fc700 1 -- 172.21.15.144:0/1892601072 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3f30056c90 con 0x7f3f400ff2d0 2024-09-10T11:56:24.310 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.309+0000 7f3f474f4700 1 -- 172.21.15.144:0/1892601072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3f4002cc60 con 0x7f3f400ff2d0 2024-09-10T11:56:24.310 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:23 smithi144 ceph-mon[87436]: from='client.35356 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:24.311 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:23 smithi144 ceph-mon[87436]: from='client.35360 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:24.311 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:23 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3686827546' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:56:24.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.309+0000 7f3f3e7fc700 1 -- 172.21.15.144:0/1892601072 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f3f3008be60 con 0x7f3f400ff2d0 2024-09-10T11:56:24.311 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T11:56:24.312 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T11:56:24.312 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:56:24.312 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:56:24.312 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:56:24.312 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:56:24.312 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:56:24.312 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:56:24.312 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:56:24.312 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:56:24.312 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:56:24.312 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:56:24.312 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:56:24.312 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:56:24.313 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:56:24.313 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:56:24.313 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:56:24.313 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:56:24.313 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:56:24.313 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:56:24.313 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:56:24.313 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:56:24.313 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:56:24.313 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:56:24.314 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:56:24.314 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:56:24.314 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:56:24.314 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:56:24.314 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:56:24.314 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:56:24.314 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:56:24.314 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:56:24.314 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:56:24.314 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:56:24.314 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:56:24.314 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:56:24.314 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:56:24.315 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:56:24.315 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:56:24.315 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:56:24.315 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:56:24.315 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:56:24.315 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:56:24.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.311+0000 7f3f474f4700 1 -- 172.21.15.144:0/1892601072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f2806e200 msgr2=0x7f3f280706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:24.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.311+0000 7f3f474f4700 1 --2- 172.21.15.144:0/1892601072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f2806e200 0x7f3f280706b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3f34006010 tx=0x7f3f3401a040 comp rx=0 tx=0).stop 2024-09-10T11:56:24.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.312+0000 7f3f474f4700 1 -- 172.21.15.144:0/1892601072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f400ff2d0 msgr2=0x7f3f40196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:24.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.312+0000 7f3f474f4700 1 --2- 172.21.15.144:0/1892601072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f400ff2d0 0x7f3f40196ca0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000ba50 tx=0x7f3f3000be10 comp rx=0 tx=0).stop 2024-09-10T11:56:24.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.312+0000 7f3f474f4700 1 -- 172.21.15.144:0/1892601072 shutdown_connections 2024-09-10T11:56:24.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.312+0000 7f3f474f4700 1 --2- 172.21.15.144:0/1892601072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f2806e200 0x7f3f280706b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.312+0000 7f3f474f4700 1 --2- 172.21.15.144:0/1892601072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f400ff2d0 0x7f3f40196ca0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.312+0000 7f3f474f4700 1 --2- 172.21.15.144:0/1892601072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f400ffc70 0x7f3f401971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.312+0000 7f3f474f4700 1 -- 172.21.15.144:0/1892601072 >> 172.21.15.144:0/1892601072 conn(0x7f3f400fabb0 msgr2=0x7f3f400fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:24.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.312+0000 7f3f474f4700 1 -- 172.21.15.144:0/1892601072 shutdown_connections 2024-09-10T11:56:24.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.312+0000 7f3f474f4700 1 -- 172.21.15.144:0/1892601072 wait complete. 2024-09-10T11:56:24.317 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T11:56:24.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:23 smithi178 ceph-mon[65940]: from='client.35356 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:24.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:23 smithi178 ceph-mon[65940]: from='client.35360 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:24.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:23 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3686827546' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:56:24.418 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.417+0000 7f24160f2700 1 -- 172.21.15.144:0/4076584223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2410100d50 msgr2=0x7f2410104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:24.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.417+0000 7f24160f2700 1 --2- 172.21.15.144:0/4076584223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2410100d50 0x7f2410104d20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009910 tx=0x7f23f8009c20 comp rx=0 tx=0).stop 2024-09-10T11:56:24.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.417+0000 7f24160f2700 1 -- 172.21.15.144:0/4076584223 shutdown_connections 2024-09-10T11:56:24.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.417+0000 7f24160f2700 1 --2- 172.21.15.144:0/4076584223 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2410100d50 0x7f2410104d20 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.417+0000 7f24160f2700 1 --2- 172.21.15.144:0/4076584223 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24101003b0 0x7f2410100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.417+0000 7f24160f2700 1 -- 172.21.15.144:0/4076584223 >> 172.21.15.144:0/4076584223 conn(0x7f24100fbc30 msgr2=0x7f24100fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:24.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.417+0000 7f24160f2700 1 -- 172.21.15.144:0/4076584223 shutdown_connections 2024-09-10T11:56:24.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.417+0000 7f24160f2700 1 -- 172.21.15.144:0/4076584223 wait complete. 2024-09-10T11:56:24.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.417+0000 7f24160f2700 1 Processor -- start 2024-09-10T11:56:24.419 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.418+0000 7f24160f2700 1 -- start start 2024-09-10T11:56:24.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.418+0000 7f24160f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24101003b0 0x7f2410195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:24.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.418+0000 7f24160f2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2410100d50 0x7f2410196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:24.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.418+0000 7f24160f2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24101969e0 con 0x7f24101003b0 2024-09-10T11:56:24.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.418+0000 7f24160f2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241019a770 con 0x7f2410100d50 2024-09-10T11:56:24.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.418+0000 7f240f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24101003b0 0x7f2410195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:24.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.418+0000 7f240f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24101003b0 0x7f2410195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42360/0 (socket says 172.21.15.144:42360) 2024-09-10T11:56:24.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.418+0000 7f240f7fe700 1 -- 172.21.15.144:0/2641966691 learned_addr learned my addr 172.21.15.144:0/2641966691 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:56:24.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.418+0000 7f240effd700 1 --2- 172.21.15.144:0/2641966691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2410100d50 0x7f2410196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:24.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.418+0000 7f240f7fe700 1 -- 172.21.15.144:0/2641966691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2410100d50 msgr2=0x7f2410196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:24.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.418+0000 7f240f7fe700 1 --2- 172.21.15.144:0/2641966691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2410100d50 0x7f2410196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.418+0000 7f240f7fe700 1 -- 172.21.15.144:0/2641966691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f80095f0 con 0x7f24101003b0 2024-09-10T11:56:24.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.419+0000 7f240f7fe700 1 --2- 172.21.15.144:0/2641966691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24101003b0 0x7f2410195dc0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f24080060b0 tx=0x7f240800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:24.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.419+0000 7f240cff9700 1 -- 172.21.15.144:0/2641966691 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f240800dda0 con 0x7f24101003b0 2024-09-10T11:56:24.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.419+0000 7f240cff9700 1 -- 172.21.15.144:0/2641966691 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f240800f040 con 0x7f24101003b0 2024-09-10T11:56:24.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.419+0000 7f24160f2700 1 -- 172.21.15.144:0/2641966691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241019aa50 con 0x7f24101003b0 2024-09-10T11:56:24.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.419+0000 7f240cff9700 1 -- 172.21.15.144:0/2641966691 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2408014790 con 0x7f24101003b0 2024-09-10T11:56:24.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.419+0000 7f24160f2700 1 -- 172.21.15.144:0/2641966691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241019afa0 con 0x7f24101003b0 2024-09-10T11:56:24.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.420+0000 7f24160f2700 1 -- 172.21.15.144:0/2641966691 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2410108720 con 0x7f24101003b0 2024-09-10T11:56:24.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.420+0000 7f240cff9700 1 -- 172.21.15.144:0/2641966691 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2408014980 con 0x7f24101003b0 2024-09-10T11:56:24.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.421+0000 7f240cff9700 1 --2- 172.21.15.144:0/2641966691 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f23fc06e200 0x7f23fc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:24.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.421+0000 7f240cff9700 1 -- 172.21.15.144:0/2641966691 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f240808dbd0 con 0x7f24101003b0 2024-09-10T11:56:24.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.421+0000 7f240effd700 1 --2- 172.21.15.144:0/2641966691 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f23fc06e200 0x7f23fc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:24.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.421+0000 7f240effd700 1 --2- 172.21.15.144:0/2641966691 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f23fc06e200 0x7f23fc0706b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f24101973e0 tx=0x7f23f801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:24.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.425+0000 7f240cff9700 1 -- 172.21.15.144:0/2641966691 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2408056fc0 con 0x7f24101003b0 2024-09-10T11:56:24.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.649+0000 7f24160f2700 1 -- 172.21.15.144:0/2641966691 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f241019b220 con 0x7f23fc06e200 2024-09-10T11:56:24.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.650+0000 7f240cff9700 1 -- 172.21.15.144:0/2641966691 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f241019b220 con 0x7f23fc06e200 2024-09-10T11:56:24.652 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:56:24.652 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:56:24.652 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:56:24.652 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:56:24.653 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:56:24.653 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:56:24.653 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T11:56:24.653 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T11:56:24.653 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:56:24.653 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T11:56:24.654 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:56:24.654 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T11:56:24.654 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:56:24.654 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:56:24.654 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:56:24.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.653+0000 7f24160f2700 1 -- 172.21.15.144:0/2641966691 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f23fc06e200 msgr2=0x7f23fc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:24.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.653+0000 7f24160f2700 1 --2- 172.21.15.144:0/2641966691 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f23fc06e200 0x7f23fc0706b0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f24101973e0 tx=0x7f23f801a040 comp rx=0 tx=0).stop 2024-09-10T11:56:24.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.653+0000 7f24160f2700 1 -- 172.21.15.144:0/2641966691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24101003b0 msgr2=0x7f2410195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:24.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.653+0000 7f24160f2700 1 --2- 172.21.15.144:0/2641966691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24101003b0 0x7f2410195dc0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f24080060b0 tx=0x7f240800d3b0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.653+0000 7f24160f2700 1 -- 172.21.15.144:0/2641966691 shutdown_connections 2024-09-10T11:56:24.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.653+0000 7f24160f2700 1 --2- 172.21.15.144:0/2641966691 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f23fc06e200 0x7f23fc0706b0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.657 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.653+0000 7f24160f2700 1 --2- 172.21.15.144:0/2641966691 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24101003b0 0x7f2410195dc0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.657 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.653+0000 7f24160f2700 1 --2- 172.21.15.144:0/2641966691 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2410100d50 0x7f2410196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.657 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.653+0000 7f24160f2700 1 -- 172.21.15.144:0/2641966691 >> 172.21.15.144:0/2641966691 conn(0x7f24100fbc30 msgr2=0x7f24100fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:24.657 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.653+0000 7f24160f2700 1 -- 172.21.15.144:0/2641966691 shutdown_connections 2024-09-10T11:56:24.657 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.653+0000 7f24160f2700 1 -- 172.21.15.144:0/2641966691 wait complete. 2024-09-10T11:56:24.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.757+0000 7ff05fc6d700 1 -- 172.21.15.144:0/3258070331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0581048c0 msgr2=0x7ff058104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:24.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.757+0000 7ff05fc6d700 1 --2- 172.21.15.144:0/3258070331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0581048c0 0x7ff058104c90 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7ff05400aa20 tx=0x7ff05400ad30 comp rx=0 tx=0).stop 2024-09-10T11:56:24.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.757+0000 7ff05fc6d700 1 -- 172.21.15.144:0/3258070331 shutdown_connections 2024-09-10T11:56:24.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.758+0000 7ff05fc6d700 1 --2- 172.21.15.144:0/3258070331 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0580ff260 0x7ff0580ff6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.758+0000 7ff05fc6d700 1 --2- 172.21.15.144:0/3258070331 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0581048c0 0x7ff058104c90 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.758+0000 7ff05fc6d700 1 -- 172.21.15.144:0/3258070331 >> 172.21.15.144:0/3258070331 conn(0x7ff0580747e0 msgr2=0x7ff058074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:24.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.758+0000 7ff05fc6d700 1 -- 172.21.15.144:0/3258070331 shutdown_connections 2024-09-10T11:56:24.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.758+0000 7ff05fc6d700 1 -- 172.21.15.144:0/3258070331 wait complete. 2024-09-10T11:56:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.758+0000 7ff05fc6d700 1 Processor -- start 2024-09-10T11:56:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.758+0000 7ff05fc6d700 1 -- start start 2024-09-10T11:56:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.758+0000 7ff05fc6d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0580ff260 0x7ff0581998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.758+0000 7ff05fc6d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0581048c0 0x7ff058199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:24.761 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.758+0000 7ff05fc6d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff05819a4c0 con 0x7ff0581048c0 2024-09-10T11:56:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.758+0000 7ff05fc6d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff058193970 con 0x7ff0580ff260 2024-09-10T11:56:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.759+0000 7ff05d208700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0581048c0 0x7ff058199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.759+0000 7ff05d208700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0581048c0 0x7ff058199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42378/0 (socket says 172.21.15.144:42378) 2024-09-10T11:56:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.759+0000 7ff05d208700 1 -- 172.21.15.144:0/4128575085 learned_addr learned my addr 172.21.15.144:0/4128575085 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:56:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.759+0000 7ff05da09700 1 --2- 172.21.15.144:0/4128575085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0580ff260 0x7ff0581998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.759+0000 7ff05d208700 1 -- 172.21.15.144:0/4128575085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0580ff260 msgr2=0x7ff0581998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.759+0000 7ff05d208700 1 --2- 172.21.15.144:0/4128575085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0580ff260 0x7ff0581998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.759+0000 7ff05d208700 1 -- 172.21.15.144:0/4128575085 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05400a700 con 0x7ff0581048c0 2024-09-10T11:56:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.759+0000 7ff05da09700 1 --2- 172.21.15.144:0/4128575085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0580ff260 0x7ff0581998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:56:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.759+0000 7ff05d208700 1 --2- 172.21.15.144:0/4128575085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0581048c0 0x7ff058199e30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00d900 tx=0x7ff04c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.759+0000 7ff04affd700 1 -- 172.21.15.144:0/4128575085 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff04c0041f0 con 0x7ff0581048c0 2024-09-10T11:56:24.762 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.759+0000 7ff04affd700 1 -- 172.21.15.144:0/4128575085 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff04c004cf0 con 0x7ff0581048c0 2024-09-10T11:56:24.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.759+0000 7ff05fc6d700 1 -- 172.21.15.144:0/4128575085 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff058193c50 con 0x7ff0581048c0 2024-09-10T11:56:24.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.759+0000 7ff04affd700 1 -- 172.21.15.144:0/4128575085 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff04c00b750 con 0x7ff0581048c0 2024-09-10T11:56:24.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.759+0000 7ff05fc6d700 1 -- 172.21.15.144:0/4128575085 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0581941a0 con 0x7ff0581048c0 2024-09-10T11:56:24.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.760+0000 7ff05fc6d700 1 -- 172.21.15.144:0/4128575085 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05804ea50 con 0x7ff0581048c0 2024-09-10T11:56:24.763 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.761+0000 7ff04affd700 1 -- 172.21.15.144:0/4128575085 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff04c004350 con 0x7ff0581048c0 2024-09-10T11:56:24.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.761+0000 7ff04affd700 1 --2- 172.21.15.144:0/4128575085 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff04406e200 0x7ff0440706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:24.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.761+0000 7ff04affd700 1 -- 172.21.15.144:0/4128575085 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff04c08e110 con 0x7ff0581048c0 2024-09-10T11:56:24.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.761+0000 7ff05da09700 1 --2- 172.21.15.144:0/4128575085 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff04406e200 0x7ff0440706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:24.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.762+0000 7ff05da09700 1 --2- 172.21.15.144:0/4128575085 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff04406e200 0x7ff0440706b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff05400c6d0 tx=0x7ff054005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:24.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:24.766+0000 7ff04affd700 1 -- 172.21.15.144:0/4128575085 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff04c05b0a0 con 0x7ff0581048c0 2024-09-10T11:56:25.053 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:25.052+0000 7ff05fc6d700 1 -- 172.21.15.144:0/4128575085 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff058194610 con 0x7ff0581048c0 2024-09-10T11:56:25.053 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:25.052+0000 7ff04affd700 1 -- 172.21.15.144:0/4128575085 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7ff04c05ac30 con 0x7ff0581048c0 2024-09-10T11:56:25.053 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:56:25.054 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T11:56:25.054 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:56:25.054 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T11:56:25.054 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:56:25.054 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T11:56:25.054 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T11:56:25.054 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T11:56:25.054 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T11:56:25.054 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T11:56:25.054 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T11:56:25.054 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T11:56:25.054 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T11:56:25.055 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T11:56:25.055 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T11:56:25.055 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T11:56:25.055 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T11:56:25.055 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T11:56:25.055 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:56:25.055 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:56:25.055 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:56:25.055 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:56:25.055 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:56:25.055 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:56:25.055 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:56:25.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:25.055+0000 7ff05fc6d700 1 -- 172.21.15.144:0/4128575085 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff04406e200 msgr2=0x7ff0440706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:25.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:25.055+0000 7ff05fc6d700 1 --2- 172.21.15.144:0/4128575085 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff04406e200 0x7ff0440706b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff05400c6d0 tx=0x7ff054005dc0 comp rx=0 tx=0).stop 2024-09-10T11:56:25.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:25.055+0000 7ff05fc6d700 1 -- 172.21.15.144:0/4128575085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0581048c0 msgr2=0x7ff058199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:25.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:25.055+0000 7ff05fc6d700 1 --2- 172.21.15.144:0/4128575085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0581048c0 0x7ff058199e30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff04c00d900 tx=0x7ff04c00dcc0 comp rx=0 tx=0).stop 2024-09-10T11:56:25.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:25.055+0000 7ff05fc6d700 1 -- 172.21.15.144:0/4128575085 shutdown_connections 2024-09-10T11:56:25.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:25.055+0000 7ff05fc6d700 1 --2- 172.21.15.144:0/4128575085 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff04406e200 0x7ff0440706b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:25.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:25.055+0000 7ff05fc6d700 1 --2- 172.21.15.144:0/4128575085 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff0580ff260 0x7ff0581998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:25.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:25.055+0000 7ff05fc6d700 1 --2- 172.21.15.144:0/4128575085 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff0581048c0 0x7ff058199e30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:25.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:25.055+0000 7ff05fc6d700 1 -- 172.21.15.144:0/4128575085 >> 172.21.15.144:0/4128575085 conn(0x7ff0580747e0 msgr2=0x7ff0580fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:25.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:25.055+0000 7ff05fc6d700 1 -- 172.21.15.144:0/4128575085 shutdown_connections 2024-09-10T11:56:25.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:25.055+0000 7ff05fc6d700 1 -- 172.21.15.144:0/4128575085 wait complete. 2024-09-10T11:56:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:24 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1892601072' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:56:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:24 smithi178 ceph-mon[65940]: from='client.35372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:24 smithi178 ceph-mon[65940]: pgmap v734: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:25.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:24 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1892601072' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:56:25.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:24 smithi144 ceph-mon[87436]: from='client.35372 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:25.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:24 smithi144 ceph-mon[87436]: pgmap v734: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:26.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:25 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/4128575085' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:56:26.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:25 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/4128575085' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:56:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:26 smithi178 ceph-mon[65940]: pgmap v735: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:27.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:26 smithi144 ceph-mon[87436]: pgmap v735: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:28.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:56:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:56:29.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:29 smithi144 ceph-mon[87436]: pgmap v736: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:29 smithi178 ceph-mon[65940]: pgmap v736: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:30 smithi178 ceph-mon[65940]: pgmap v737: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:31.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:30 smithi144 ceph-mon[87436]: pgmap v737: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:32 smithi178 ceph-mon[65940]: pgmap v738: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:33.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:32 smithi144 ceph-mon[87436]: pgmap v738: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:35.305 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:34 smithi144 ceph-mon[87436]: pgmap v739: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:34 smithi178 ceph-mon[65940]: pgmap v739: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:36 smithi178 ceph-mon[65940]: pgmap v740: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:37.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:36 smithi144 ceph-mon[87436]: pgmap v740: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:38 smithi178 ceph-mon[65940]: pgmap v741: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:39.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:38 smithi144 ceph-mon[87436]: pgmap v741: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:40 smithi178 ceph-mon[65940]: pgmap v742: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:41.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:40 smithi144 ceph-mon[87436]: pgmap v742: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:43.236 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:42 smithi144 ceph-mon[87436]: pgmap v743: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:42 smithi178 ceph-mon[65940]: pgmap v743: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:44.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:56:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:56:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:44 smithi178 ceph-mon[65940]: pgmap v744: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:45.409 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:44 smithi144 ceph-mon[87436]: pgmap v744: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:46 smithi178 ceph-mon[65940]: pgmap v745: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:46 smithi144 ceph-mon[87436]: pgmap v745: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:48 smithi178 ceph-mon[65940]: pgmap v746: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:49.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:48 smithi144 ceph-mon[87436]: pgmap v746: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:50 smithi178 ceph-mon[65940]: pgmap v747: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:51.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:50 smithi144 ceph-mon[87436]: pgmap v747: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:52 smithi178 ceph-mon[65940]: pgmap v748: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:52 smithi144 ceph-mon[87436]: pgmap v748: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:55.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.173+0000 7f297f877700 1 -- 172.21.15.144:0/1331093802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2978101b00 msgr2=0x7f297810a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:55.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.173+0000 7f297f877700 1 --2- 172.21.15.144:0/1331093802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2978101b00 0x7f297810a1d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f2968009910 tx=0x7f2968009c20 comp rx=0 tx=0).stop 2024-09-10T11:56:55.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.173+0000 7f297f877700 1 -- 172.21.15.144:0/1331093802 shutdown_connections 2024-09-10T11:56:55.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.173+0000 7f297f877700 1 --2- 172.21.15.144:0/1331093802 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2978101b00 0x7f297810a1d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.173+0000 7f297f877700 1 --2- 172.21.15.144:0/1331093802 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29781011f0 0x7f29781015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.173+0000 7f297f877700 1 -- 172.21.15.144:0/1331093802 >> 172.21.15.144:0/1331093802 conn(0x7f29780fab60 msgr2=0x7f29780fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:55.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.173+0000 7f297f877700 1 -- 172.21.15.144:0/1331093802 shutdown_connections 2024-09-10T11:56:55.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.173+0000 7f297f877700 1 -- 172.21.15.144:0/1331093802 wait complete. 2024-09-10T11:56:55.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.174+0000 7f297f877700 1 Processor -- start 2024-09-10T11:56:55.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.174+0000 7f297f877700 1 -- start start 2024-09-10T11:56:55.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.174+0000 7f297f877700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29781011f0 0x7f2978195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:55.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.174+0000 7f297f877700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2978101b00 0x7f2978196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:55.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.174+0000 7f297f877700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29781968a0 con 0x7f2978101b00 2024-09-10T11:56:55.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.174+0000 7f297f877700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29781969e0 con 0x7f29781011f0 2024-09-10T11:56:55.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.174+0000 7f297ce12700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2978101b00 0x7f2978196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:55.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.174+0000 7f297ce12700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2978101b00 0x7f2978196250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:46934/0 (socket says 172.21.15.144:46934) 2024-09-10T11:56:55.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.174+0000 7f297ce12700 1 -- 172.21.15.144:0/1285032382 learned_addr learned my addr 172.21.15.144:0/1285032382 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:56:55.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.174+0000 7f297d613700 1 --2- 172.21.15.144:0/1285032382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29781011f0 0x7f2978195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:55.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.175+0000 7f297ce12700 1 -- 172.21.15.144:0/1285032382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29781011f0 msgr2=0x7f2978195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:55.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.175+0000 7f297ce12700 1 --2- 172.21.15.144:0/1285032382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29781011f0 0x7f2978195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.175+0000 7f297ce12700 1 -- 172.21.15.144:0/1285032382 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29680095f0 con 0x7f2978101b00 2024-09-10T11:56:55.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.175+0000 7f297ce12700 1 --2- 172.21.15.144:0/1285032382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2978101b00 0x7f2978196250 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2968006010 tx=0x7f296800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:55.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.175+0000 7f296e7fc700 1 -- 172.21.15.144:0/1285032382 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f296801d070 con 0x7f2978101b00 2024-09-10T11:56:55.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.175+0000 7f296e7fc700 1 -- 172.21.15.144:0/1285032382 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2968004fb0 con 0x7f2978101b00 2024-09-10T11:56:55.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.175+0000 7f297f877700 1 -- 172.21.15.144:0/1285032382 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f297819a7d0 con 0x7f2978101b00 2024-09-10T11:56:55.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.175+0000 7f296e7fc700 1 -- 172.21.15.144:0/1285032382 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2968022ea0 con 0x7f2978101b00 2024-09-10T11:56:55.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.175+0000 7f297f877700 1 -- 172.21.15.144:0/1285032382 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297819acc0 con 0x7f2978101b00 2024-09-10T11:56:55.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.176+0000 7f297f877700 1 -- 172.21.15.144:0/1285032382 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2978107870 con 0x7f2978101b00 2024-09-10T11:56:55.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.177+0000 7f296e7fc700 1 -- 172.21.15.144:0/1285032382 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2968022470 con 0x7f2978101b00 2024-09-10T11:56:55.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.177+0000 7f296e7fc700 1 --2- 172.21.15.144:0/1285032382 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f296406e200 0x7f29640706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:55.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.177+0000 7f296e7fc700 1 -- 172.21.15.144:0/1285032382 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f296808f560 con 0x7f2978101b00 2024-09-10T11:56:55.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.177+0000 7f297d613700 1 --2- 172.21.15.144:0/1285032382 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f296406e200 0x7f29640706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:55.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.178+0000 7f297d613700 1 --2- 172.21.15.144:0/1285032382 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f296406e200 0x7f29640706b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f297400e270 tx=0x7f2974009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:55.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.182+0000 7f296e7fc700 1 -- 172.21.15.144:0/1285032382 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2968058b90 con 0x7f2978101b00 2024-09-10T11:56:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:54 smithi178 ceph-mon[65940]: pgmap v749: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:55.406 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.405+0000 7f297f877700 1 -- 172.21.15.144:0/1285032382 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f297802ccf0 con 0x7f296406e200 2024-09-10T11:56:55.406 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:54 smithi144 ceph-mon[87436]: pgmap v749: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:55.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.406+0000 7f296e7fc700 1 -- 172.21.15.144:0/1285032382 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f297802ccf0 con 0x7f296406e200 2024-09-10T11:56:55.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.408+0000 7f297f877700 1 -- 172.21.15.144:0/1285032382 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f296406e200 msgr2=0x7f29640706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:55.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.408+0000 7f297f877700 1 --2- 172.21.15.144:0/1285032382 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f296406e200 0x7f29640706b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f297400e270 tx=0x7f2974009040 comp rx=0 tx=0).stop 2024-09-10T11:56:55.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.408+0000 7f297f877700 1 -- 172.21.15.144:0/1285032382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2978101b00 msgr2=0x7f2978196250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:55.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.408+0000 7f297f877700 1 --2- 172.21.15.144:0/1285032382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2978101b00 0x7f2978196250 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f2968006010 tx=0x7f296800be20 comp rx=0 tx=0).stop 2024-09-10T11:56:55.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.408+0000 7f297f877700 1 -- 172.21.15.144:0/1285032382 shutdown_connections 2024-09-10T11:56:55.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.408+0000 7f297f877700 1 --2- 172.21.15.144:0/1285032382 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f296406e200 0x7f29640706b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.408+0000 7f297f877700 1 --2- 172.21.15.144:0/1285032382 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29781011f0 0x7f2978195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.408+0000 7f297f877700 1 --2- 172.21.15.144:0/1285032382 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2978101b00 0x7f2978196250 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.408+0000 7f297f877700 1 -- 172.21.15.144:0/1285032382 >> 172.21.15.144:0/1285032382 conn(0x7f29780fab60 msgr2=0x7f29781049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:55.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.408+0000 7f297f877700 1 -- 172.21.15.144:0/1285032382 shutdown_connections 2024-09-10T11:56:55.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.408+0000 7f297f877700 1 -- 172.21.15.144:0/1285032382 wait complete. 2024-09-10T11:56:55.420 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:56:55.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.515+0000 7ff7125b2700 1 -- 172.21.15.144:0/2914430336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c1004c0 msgr2=0x7ff70c10c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:55.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.515+0000 7ff7125b2700 1 --2- 172.21.15.144:0/2914430336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c1004c0 0x7ff70c10c7a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff700009940 tx=0x7ff700009c50 comp rx=0 tx=0).stop 2024-09-10T11:56:55.520 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.515+0000 7ff7125b2700 1 -- 172.21.15.144:0/2914430336 shutdown_connections 2024-09-10T11:56:55.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.515+0000 7ff7125b2700 1 --2- 172.21.15.144:0/2914430336 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c1004c0 0x7ff70c10c7a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.515+0000 7ff7125b2700 1 --2- 172.21.15.144:0/2914430336 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70c0ffbb0 0x7ff70c0fff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.515+0000 7ff7125b2700 1 -- 172.21.15.144:0/2914430336 >> 172.21.15.144:0/2914430336 conn(0x7ff70c074f50 msgr2=0x7ff70c075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:55.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.515+0000 7ff7125b2700 1 -- 172.21.15.144:0/2914430336 shutdown_connections 2024-09-10T11:56:55.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.515+0000 7ff7125b2700 1 -- 172.21.15.144:0/2914430336 wait complete. 2024-09-10T11:56:55.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.516+0000 7ff7125b2700 1 Processor -- start 2024-09-10T11:56:55.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.516+0000 7ff7125b2700 1 -- start start 2024-09-10T11:56:55.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.516+0000 7ff7125b2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70c0ffbb0 0x7ff70c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:55.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.516+0000 7ff7125b2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c1004c0 0x7ff70c1984a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:55.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.516+0000 7ff7125b2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c198b80 con 0x7ff70c1004c0 2024-09-10T11:56:55.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.516+0000 7ff7125b2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c19c910 con 0x7ff70c0ffbb0 2024-09-10T11:56:55.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.516+0000 7ff70b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c1004c0 0x7ff70c1984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:55.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.516+0000 7ff70b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c1004c0 0x7ff70c1984a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:46956/0 (socket says 172.21.15.144:46956) 2024-09-10T11:56:55.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.516+0000 7ff70b7fe700 1 -- 172.21.15.144:0/1168954142 learned_addr learned my addr 172.21.15.144:0/1168954142 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:56:55.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.516+0000 7ff70bfff700 1 --2- 172.21.15.144:0/1168954142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70c0ffbb0 0x7ff70c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:55.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.516+0000 7ff70b7fe700 1 -- 172.21.15.144:0/1168954142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70c0ffbb0 msgr2=0x7ff70c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:55.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.516+0000 7ff70b7fe700 1 --2- 172.21.15.144:0/1168954142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70c0ffbb0 0x7ff70c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.516+0000 7ff70b7fe700 1 -- 172.21.15.144:0/1168954142 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7000095f0 con 0x7ff70c1004c0 2024-09-10T11:56:55.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.516+0000 7ff70bfff700 1 --2- 172.21.15.144:0/1168954142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70c0ffbb0 0x7ff70c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:56:55.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.517+0000 7ff70b7fe700 1 --2- 172.21.15.144:0/1168954142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c1004c0 0x7ff70c1984a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff7000055f0 tx=0x7ff7000056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:55.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.517+0000 7ff7097fa700 1 -- 172.21.15.144:0/1168954142 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff70001d070 con 0x7ff70c1004c0 2024-09-10T11:56:55.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.517+0000 7ff7125b2700 1 -- 172.21.15.144:0/1168954142 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70c19cbf0 con 0x7ff70c1004c0 2024-09-10T11:56:55.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.517+0000 7ff7097fa700 1 -- 172.21.15.144:0/1168954142 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff70000f460 con 0x7ff70c1004c0 2024-09-10T11:56:55.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.517+0000 7ff7097fa700 1 -- 172.21.15.144:0/1168954142 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff700021620 con 0x7ff70c1004c0 2024-09-10T11:56:55.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.517+0000 7ff7125b2700 1 -- 172.21.15.144:0/1168954142 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70c19d140 con 0x7ff70c1004c0 2024-09-10T11:56:55.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.518+0000 7ff7125b2700 1 -- 172.21.15.144:0/1168954142 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70c109f10 con 0x7ff70c1004c0 2024-09-10T11:56:55.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.518+0000 7ff7097fa700 1 -- 172.21.15.144:0/1168954142 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff70000fa60 con 0x7ff70c1004c0 2024-09-10T11:56:55.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.519+0000 7ff7097fa700 1 --2- 172.21.15.144:0/1168954142 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff6f406e200 0x7ff6f40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:55.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.519+0000 7ff7097fa700 1 -- 172.21.15.144:0/1168954142 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff70008f3f0 con 0x7ff70c1004c0 2024-09-10T11:56:55.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.519+0000 7ff70bfff700 1 --2- 172.21.15.144:0/1168954142 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff6f406e200 0x7ff6f40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:55.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.519+0000 7ff70bfff700 1 --2- 172.21.15.144:0/1168954142 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff6f406e200 0x7ff6f40706b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc007900 tx=0x7ff6fc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:55.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.523+0000 7ff7097fa700 1 -- 172.21.15.144:0/1168954142 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff7000587d0 con 0x7ff70c1004c0 2024-09-10T11:56:55.755 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.754+0000 7ff7125b2700 1 -- 172.21.15.144:0/1168954142 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff70c19d420 con 0x7ff6f406e200 2024-09-10T11:56:55.756 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.755+0000 7ff7097fa700 1 -- 172.21.15.144:0/1168954142 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff70c19d420 con 0x7ff6f406e200 2024-09-10T11:56:55.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.757+0000 7ff7125b2700 1 -- 172.21.15.144:0/1168954142 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff6f406e200 msgr2=0x7ff6f40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:55.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.757+0000 7ff7125b2700 1 --2- 172.21.15.144:0/1168954142 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff6f406e200 0x7ff6f40706b0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc007900 tx=0x7ff6fc008040 comp rx=0 tx=0).stop 2024-09-10T11:56:55.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.757+0000 7ff7125b2700 1 -- 172.21.15.144:0/1168954142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c1004c0 msgr2=0x7ff70c1984a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:55.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.757+0000 7ff7125b2700 1 --2- 172.21.15.144:0/1168954142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c1004c0 0x7ff70c1984a0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff7000055f0 tx=0x7ff7000056d0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.757+0000 7ff7125b2700 1 -- 172.21.15.144:0/1168954142 shutdown_connections 2024-09-10T11:56:55.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.757+0000 7ff7125b2700 1 --2- 172.21.15.144:0/1168954142 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff6f406e200 0x7ff6f40706b0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.757+0000 7ff7125b2700 1 --2- 172.21.15.144:0/1168954142 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70c0ffbb0 0x7ff70c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.757+0000 7ff7125b2700 1 --2- 172.21.15.144:0/1168954142 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c1004c0 0x7ff70c1984a0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.757+0000 7ff7125b2700 1 -- 172.21.15.144:0/1168954142 >> 172.21.15.144:0/1168954142 conn(0x7ff70c074f50 msgr2=0x7ff70c0fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:55.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.757+0000 7ff7125b2700 1 -- 172.21.15.144:0/1168954142 shutdown_connections 2024-09-10T11:56:55.759 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.757+0000 7ff7125b2700 1 -- 172.21.15.144:0/1168954142 wait complete. 2024-09-10T11:56:55.865 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.862+0000 7f79d41ee700 1 -- 172.21.15.144:0/859028784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79cc0ffd30 msgr2=0x7f79cc103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:55.865 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.862+0000 7f79d41ee700 1 --2- 172.21.15.144:0/859028784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79cc0ffd30 0x7f79cc103ce0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f79bc009910 tx=0x7f79bc009c20 comp rx=0 tx=0).stop 2024-09-10T11:56:55.865 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.862+0000 7f79d41ee700 1 -- 172.21.15.144:0/859028784 shutdown_connections 2024-09-10T11:56:55.866 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.862+0000 7f79d41ee700 1 --2- 172.21.15.144:0/859028784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79cc0ffd30 0x7f79cc103ce0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.866 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.862+0000 7f79d41ee700 1 --2- 172.21.15.144:0/859028784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79cc0ff390 0x7f79cc0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.866 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.862+0000 7f79d41ee700 1 -- 172.21.15.144:0/859028784 >> 172.21.15.144:0/859028784 conn(0x7f79cc074f50 msgr2=0x7f79cc075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:55.866 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.862+0000 7f79d41ee700 1 -- 172.21.15.144:0/859028784 shutdown_connections 2024-09-10T11:56:55.866 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.863+0000 7f79d41ee700 1 -- 172.21.15.144:0/859028784 wait complete. 2024-09-10T11:56:55.866 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.863+0000 7f79d41ee700 1 Processor -- start 2024-09-10T11:56:55.866 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.863+0000 7f79d41ee700 1 -- start start 2024-09-10T11:56:55.866 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.863+0000 7f79d41ee700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79cc0ff390 0x7f79cc196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:55.866 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.863+0000 7f79d41ee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79cc0ffd30 0x7f79cc1970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:55.866 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.863+0000 7f79d41ee700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc197650 con 0x7f79cc0ffd30 2024-09-10T11:56:55.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.863+0000 7f79d41ee700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79cc197790 con 0x7f79cc0ff390 2024-09-10T11:56:55.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.863+0000 7f79d1789700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79cc0ffd30 0x7f79cc1970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:55.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.863+0000 7f79d1789700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79cc0ffd30 0x7f79cc1970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:46972/0 (socket says 172.21.15.144:46972) 2024-09-10T11:56:55.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.863+0000 7f79d1789700 1 -- 172.21.15.144:0/3431500552 learned_addr learned my addr 172.21.15.144:0/3431500552 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:56:55.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.864+0000 7f79d1f8a700 1 --2- 172.21.15.144:0/3431500552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79cc0ff390 0x7f79cc196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:55.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.864+0000 7f79d1789700 1 -- 172.21.15.144:0/3431500552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79cc0ff390 msgr2=0x7f79cc196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:55.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.864+0000 7f79d1789700 1 --2- 172.21.15.144:0/3431500552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79cc0ff390 0x7f79cc196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:55.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.864+0000 7f79d1789700 1 -- 172.21.15.144:0/3431500552 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79bc0095f0 con 0x7f79cc0ffd30 2024-09-10T11:56:55.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.864+0000 7f79d1789700 1 --2- 172.21.15.144:0/3431500552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79cc0ffd30 0x7f79cc1970c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00bda0 tx=0x7f79bc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:55.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.864+0000 7f79c2ffd700 1 -- 172.21.15.144:0/3431500552 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79bc01d070 con 0x7f79cc0ffd30 2024-09-10T11:56:55.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.864+0000 7f79c2ffd700 1 -- 172.21.15.144:0/3431500552 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79bc004fb0 con 0x7f79cc0ffd30 2024-09-10T11:56:55.867 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.864+0000 7f79d41ee700 1 -- 172.21.15.144:0/3431500552 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79cc19fbf0 con 0x7f79cc0ffd30 2024-09-10T11:56:55.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.864+0000 7f79c2ffd700 1 -- 172.21.15.144:0/3431500552 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79bc00f480 con 0x7f79cc0ffd30 2024-09-10T11:56:55.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.864+0000 7f79d41ee700 1 -- 172.21.15.144:0/3431500552 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79cc19ffa0 con 0x7f79cc0ffd30 2024-09-10T11:56:55.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.865+0000 7f79d41ee700 1 -- 172.21.15.144:0/3431500552 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79cc107600 con 0x7f79cc0ffd30 2024-09-10T11:56:55.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.866+0000 7f79c2ffd700 1 -- 172.21.15.144:0/3431500552 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f79bc022870 con 0x7f79cc0ffd30 2024-09-10T11:56:55.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.866+0000 7f79c2ffd700 1 --2- 172.21.15.144:0/3431500552 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f79b806e200 0x7f79b80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:55.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.866+0000 7f79c2ffd700 1 -- 172.21.15.144:0/3431500552 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f79bc08f4f0 con 0x7f79cc0ffd30 2024-09-10T11:56:55.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.866+0000 7f79d1f8a700 1 --2- 172.21.15.144:0/3431500552 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f79b806e200 0x7f79b80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:55.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.867+0000 7f79d1f8a700 1 --2- 172.21.15.144:0/3431500552 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f79b806e200 0x7f79b80706b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f79c8005ea0 tx=0x7f79c8005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:55.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:55.870+0000 7f79c2ffd700 1 -- 172.21.15.144:0/3431500552 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f79bc058a00 con 0x7f79cc0ffd30 2024-09-10T11:56:56.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.081+0000 7f79d41ee700 1 -- 172.21.15.144:0/3431500552 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f79cc02ccf0 con 0x7f79b806e200 2024-09-10T11:56:56.083 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:55 smithi144 ceph-mon[87436]: from='client.35380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:56.083 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:55 smithi144 ceph-mon[87436]: from='client.35384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:56.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.091+0000 7f79c2ffd700 1 -- 172.21.15.144:0/3431500552 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f79cc02ccf0 con 0x7f79b806e200 2024-09-10T11:56:56.092 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:56:56.092 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (25m) 3m ago 35m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:56:56.093 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (35m) 3m ago 35m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:56:56.093 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (34m) 3m ago 34m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:56:56.093 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (22m) 3m ago 35m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:56:56.093 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (22m) 3m ago 34m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:56:56.093 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (24m) 3m ago 35m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:56:56.093 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (19m) 3m ago 30m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:56:56.093 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (18m) 3m ago 30m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:56:56.093 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (3m) 3m ago 30m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T11:56:56.093 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (18m) 3m ago 30m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:56:56.093 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (26m) 3m ago 37m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:56:56.093 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (26m) 3m ago 34m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:56:56.093 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (23m) 3m ago 37m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:56:56.093 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (23m) 3m ago 34m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:56:56.094 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (25m) 3m ago 35m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:56:56.094 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (25m) 3m ago 34m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:56:56.094 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (22m) 3m ago 33m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:56:56.094 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (21m) 3m ago 33m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:56:56.094 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (21m) 3m ago 32m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:56:56.094 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (20m) 3m ago 32m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:56:56.094 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (20m) 3m ago 32m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:56:56.094 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (19m) 3m ago 31m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:56:56.094 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (25m) 3m ago 35m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:56:56.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.093+0000 7f79d41ee700 1 -- 172.21.15.144:0/3431500552 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f79b806e200 msgr2=0x7f79b80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:56.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.093+0000 7f79d41ee700 1 --2- 172.21.15.144:0/3431500552 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f79b806e200 0x7f79b80706b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f79c8005ea0 tx=0x7f79c8005e30 comp rx=0 tx=0).stop 2024-09-10T11:56:56.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.093+0000 7f79d41ee700 1 -- 172.21.15.144:0/3431500552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79cc0ffd30 msgr2=0x7f79cc1970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:56.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.093+0000 7f79d41ee700 1 --2- 172.21.15.144:0/3431500552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79cc0ffd30 0x7f79cc1970c0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f79bc00bda0 tx=0x7f79bc00be80 comp rx=0 tx=0).stop 2024-09-10T11:56:56.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.093+0000 7f79d41ee700 1 -- 172.21.15.144:0/3431500552 shutdown_connections 2024-09-10T11:56:56.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.093+0000 7f79d41ee700 1 --2- 172.21.15.144:0/3431500552 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f79b806e200 0x7f79b80706b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.093+0000 7f79d41ee700 1 --2- 172.21.15.144:0/3431500552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f79cc0ff390 0x7f79cc196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.093+0000 7f79d41ee700 1 --2- 172.21.15.144:0/3431500552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f79cc0ffd30 0x7f79cc1970c0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.093+0000 7f79d41ee700 1 -- 172.21.15.144:0/3431500552 >> 172.21.15.144:0/3431500552 conn(0x7f79cc074f50 msgr2=0x7f79cc0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:56.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.094+0000 7f79d41ee700 1 -- 172.21.15.144:0/3431500552 shutdown_connections 2024-09-10T11:56:56.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.094+0000 7f79d41ee700 1 -- 172.21.15.144:0/3431500552 wait complete. 2024-09-10T11:56:56.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.197+0000 7f5bd5bc5700 1 -- 172.21.15.144:0/2261265698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00ff390 msgr2=0x7f5bd00ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:56.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.197+0000 7f5bd5bc5700 1 --2- 172.21.15.144:0/2261265698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00ff390 0x7f5bd00ff760 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f5bc000b0a0 tx=0x7f5bc000b3b0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.197+0000 7f5bd5bc5700 1 -- 172.21.15.144:0/2261265698 shutdown_connections 2024-09-10T11:56:56.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.197+0000 7f5bd5bc5700 1 --2- 172.21.15.144:0/2261265698 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd00ffd30 0x7f5bd0103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.197+0000 7f5bd5bc5700 1 --2- 172.21.15.144:0/2261265698 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00ff390 0x7f5bd00ff760 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.199 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.197+0000 7f5bd5bc5700 1 -- 172.21.15.144:0/2261265698 >> 172.21.15.144:0/2261265698 conn(0x7f5bd0074f50 msgr2=0x7f5bd0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:56.199 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.197+0000 7f5bd5bc5700 1 -- 172.21.15.144:0/2261265698 shutdown_connections 2024-09-10T11:56:56.199 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.197+0000 7f5bd5bc5700 1 -- 172.21.15.144:0/2261265698 wait complete. 2024-09-10T11:56:56.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.198+0000 7f5bd5bc5700 1 Processor -- start 2024-09-10T11:56:56.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.198+0000 7f5bd5bc5700 1 -- start start 2024-09-10T11:56:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.198+0000 7f5bd5bc5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd00ff390 0x7f5bd0196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.198+0000 7f5bd5bc5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00ffd30 0x7f5bd0197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.198+0000 7f5bd5bc5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd0197840 con 0x7f5bd00ffd30 2024-09-10T11:56:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.198+0000 7f5bd5bc5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd01979b0 con 0x7f5bd00ff390 2024-09-10T11:56:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.198+0000 7f5bceffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00ffd30 0x7f5bd0197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.198+0000 7f5bceffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00ffd30 0x7f5bd0197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:46986/0 (socket says 172.21.15.144:46986) 2024-09-10T11:56:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.198+0000 7f5bceffd700 1 -- 172.21.15.144:0/2086896418 learned_addr learned my addr 172.21.15.144:0/2086896418 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:56:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.198+0000 7f5bcf7fe700 1 --2- 172.21.15.144:0/2086896418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd00ff390 0x7f5bd0196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.198+0000 7f5bceffd700 1 -- 172.21.15.144:0/2086896418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd00ff390 msgr2=0x7f5bd0196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.198+0000 7f5bceffd700 1 --2- 172.21.15.144:0/2086896418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd00ff390 0x7f5bd0196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.198+0000 7f5bceffd700 1 -- 172.21.15.144:0/2086896418 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc0009d00 con 0x7f5bd00ffd30 2024-09-10T11:56:56.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.199+0000 7f5bceffd700 1 --2- 172.21.15.144:0/2086896418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00ffd30 0x7f5bd0197160 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400c8f0 tx=0x7f5bc400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:56.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.199+0000 7f5bccff9700 1 -- 172.21.15.144:0/2086896418 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bc40078a0 con 0x7f5bd00ffd30 2024-09-10T11:56:56.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.199+0000 7f5bccff9700 1 -- 172.21.15.144:0/2086896418 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5bc400f450 con 0x7f5bd00ffd30 2024-09-10T11:56:56.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.199+0000 7f5bccff9700 1 -- 172.21.15.144:0/2086896418 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bc4018610 con 0x7f5bd00ffd30 2024-09-10T11:56:56.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.199+0000 7f5bd5bc5700 1 -- 172.21.15.144:0/2086896418 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd019fc90 con 0x7f5bd00ffd30 2024-09-10T11:56:56.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.199+0000 7f5bd5bc5700 1 -- 172.21.15.144:0/2086896418 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd01a01e0 con 0x7f5bd00ffd30 2024-09-10T11:56:56.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.200+0000 7f5bd5bc5700 1 -- 172.21.15.144:0/2086896418 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd0107650 con 0x7f5bd00ffd30 2024-09-10T11:56:56.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.200+0000 7f5bccff9700 1 -- 172.21.15.144:0/2086896418 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5bc4007a00 con 0x7f5bd00ffd30 2024-09-10T11:56:56.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.201+0000 7f5bccff9700 1 --2- 172.21.15.144:0/2086896418 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5bb806e200 0x7f5bb80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:56.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.201+0000 7f5bccff9700 1 -- 172.21.15.144:0/2086896418 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5bc408df60 con 0x7f5bd00ffd30 2024-09-10T11:56:56.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.201+0000 7f5bcf7fe700 1 --2- 172.21.15.144:0/2086896418 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5bb806e200 0x7f5bb80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:56.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.201+0000 7f5bcf7fe700 1 --2- 172.21.15.144:0/2086896418 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5bb806e200 0x7f5bb80706b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0000f80 tx=0x7f5bc00060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:56.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.205+0000 7f5bccff9700 1 -- 172.21.15.144:0/2086896418 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5bc40c31b0 con 0x7f5bd00ffd30 2024-09-10T11:56:56.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:55 smithi178 ceph-mon[65940]: from='client.35380 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:56.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:55 smithi178 ceph-mon[65940]: from='client.35384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:56.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.493+0000 7f5bd5bc5700 1 -- 172.21.15.144:0/2086896418 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5bd004ea50 con 0x7f5bd00ffd30 2024-09-10T11:56:56.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.494+0000 7f5bccff9700 1 -- 172.21.15.144:0/2086896418 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f5bc40573c0 con 0x7f5bd00ffd30 2024-09-10T11:56:56.495 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:56:56.495 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:56:56.495 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:56:56.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:56:56.495 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:56:56.495 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:56:56.495 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:56:56.495 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:56:56.496 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:56:56.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:56:56.496 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:56:56.496 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:56:56.496 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:56:56.496 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:56:56.496 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T11:56:56.496 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:56:56.496 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:56:56.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.496+0000 7f5bd5bc5700 1 -- 172.21.15.144:0/2086896418 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5bb806e200 msgr2=0x7f5bb80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:56.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.496+0000 7f5bd5bc5700 1 --2- 172.21.15.144:0/2086896418 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5bb806e200 0x7f5bb80706b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0000f80 tx=0x7f5bc00060c0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.496+0000 7f5bd5bc5700 1 -- 172.21.15.144:0/2086896418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00ffd30 msgr2=0x7f5bd0197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:56.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.496+0000 7f5bd5bc5700 1 --2- 172.21.15.144:0/2086896418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00ffd30 0x7f5bd0197160 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400c8f0 tx=0x7f5bc400cc00 comp rx=0 tx=0).stop 2024-09-10T11:56:56.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.496+0000 7f5bd5bc5700 1 -- 172.21.15.144:0/2086896418 shutdown_connections 2024-09-10T11:56:56.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.496+0000 7f5bd5bc5700 1 --2- 172.21.15.144:0/2086896418 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5bb806e200 0x7f5bb80706b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.496+0000 7f5bd5bc5700 1 --2- 172.21.15.144:0/2086896418 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5bd00ff390 0x7f5bd0196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.496+0000 7f5bd5bc5700 1 --2- 172.21.15.144:0/2086896418 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5bd00ffd30 0x7f5bd0197160 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.496+0000 7f5bd5bc5700 1 -- 172.21.15.144:0/2086896418 >> 172.21.15.144:0/2086896418 conn(0x7f5bd0074f50 msgr2=0x7f5bd00fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:56.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.496+0000 7f5bd5bc5700 1 -- 172.21.15.144:0/2086896418 shutdown_connections 2024-09-10T11:56:56.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.496+0000 7f5bd5bc5700 1 -- 172.21.15.144:0/2086896418 wait complete. 2024-09-10T11:56:56.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.601+0000 7f84a6a43700 1 -- 172.21.15.144:0/1403120232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a00ffc60 msgr2=0x7f84a0103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:56.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.601+0000 7f84a6a43700 1 --2- 172.21.15.144:0/1403120232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a00ffc60 0x7f84a0103c30 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f8494009910 tx=0x7f8494009c20 comp rx=0 tx=0).stop 2024-09-10T11:56:56.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.601+0000 7f84a6a43700 1 -- 172.21.15.144:0/1403120232 shutdown_connections 2024-09-10T11:56:56.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.601+0000 7f84a6a43700 1 --2- 172.21.15.144:0/1403120232 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a00ffc60 0x7f84a0103c30 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.601+0000 7f84a6a43700 1 --2- 172.21.15.144:0/1403120232 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84a00ff2c0 0x7f84a00ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.601+0000 7f84a6a43700 1 -- 172.21.15.144:0/1403120232 >> 172.21.15.144:0/1403120232 conn(0x7f84a00fab80 msgr2=0x7f84a00fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:56.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.601+0000 7f84a6a43700 1 -- 172.21.15.144:0/1403120232 shutdown_connections 2024-09-10T11:56:56.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.601+0000 7f84a6a43700 1 -- 172.21.15.144:0/1403120232 wait complete. 2024-09-10T11:56:56.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.602+0000 7f84a6a43700 1 Processor -- start 2024-09-10T11:56:56.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.602+0000 7f84a6a43700 1 -- start start 2024-09-10T11:56:56.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.602+0000 7f84a6a43700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84a00ff2c0 0x7f84a0196bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:56.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.602+0000 7f84a6a43700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a00ffc60 0x7f84a0197100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:56.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.602+0000 7f84a6a43700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a0197720 con 0x7f84a00ffc60 2024-09-10T11:56:56.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.602+0000 7f84a6a43700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a0197860 con 0x7f84a00ff2c0 2024-09-10T11:56:56.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.602+0000 7f849f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a00ffc60 0x7f84a0197100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:56.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.602+0000 7f849f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a00ffc60 0x7f84a0197100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:47008/0 (socket says 172.21.15.144:47008) 2024-09-10T11:56:56.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.602+0000 7f849f7fe700 1 -- 172.21.15.144:0/646436900 learned_addr learned my addr 172.21.15.144:0/646436900 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:56:56.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.602+0000 7f849ffff700 1 --2- 172.21.15.144:0/646436900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84a00ff2c0 0x7f84a0196bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:56.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.602+0000 7f849f7fe700 1 -- 172.21.15.144:0/646436900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84a00ff2c0 msgr2=0x7f84a0196bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:56.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.602+0000 7f849f7fe700 1 --2- 172.21.15.144:0/646436900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84a00ff2c0 0x7f84a0196bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.602+0000 7f849f7fe700 1 -- 172.21.15.144:0/646436900 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84940095f0 con 0x7f84a00ffc60 2024-09-10T11:56:56.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.602+0000 7f849ffff700 1 --2- 172.21.15.144:0/646436900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84a00ff2c0 0x7f84a0196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:56:56.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.603+0000 7f849f7fe700 1 --2- 172.21.15.144:0/646436900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a00ffc60 0x7f84a0197100 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8494005f50 tx=0x7f849400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:56.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.603+0000 7f849d7fa700 1 -- 172.21.15.144:0/646436900 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f849401d070 con 0x7f84a00ffc60 2024-09-10T11:56:56.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.603+0000 7f84a6a43700 1 -- 172.21.15.144:0/646436900 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a019fc30 con 0x7f84a00ffc60 2024-09-10T11:56:56.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.603+0000 7f849d7fa700 1 -- 172.21.15.144:0/646436900 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f849400f460 con 0x7f84a00ffc60 2024-09-10T11:56:56.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.603+0000 7f849d7fa700 1 -- 172.21.15.144:0/646436900 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8494017630 con 0x7f84a00ffc60 2024-09-10T11:56:56.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.603+0000 7f84a6a43700 1 -- 172.21.15.144:0/646436900 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a01a0040 con 0x7f84a00ffc60 2024-09-10T11:56:56.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.604+0000 7f84a6a43700 1 -- 172.21.15.144:0/646436900 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a0107630 con 0x7f84a00ffc60 2024-09-10T11:56:56.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.604+0000 7f849d7fa700 1 -- 172.21.15.144:0/646436900 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f849400f5d0 con 0x7f84a00ffc60 2024-09-10T11:56:56.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.605+0000 7f849d7fa700 1 --2- 172.21.15.144:0/646436900 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f848806e200 0x7f84880706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:56.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.605+0000 7f849d7fa700 1 -- 172.21.15.144:0/646436900 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f849408f670 con 0x7f84a00ffc60 2024-09-10T11:56:56.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.605+0000 7f849ffff700 1 --2- 172.21.15.144:0/646436900 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f848806e200 0x7f84880706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:56.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.605+0000 7f849ffff700 1 --2- 172.21.15.144:0/646436900 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f848806e200 0x7f84880706b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8490005fd0 tx=0x7f8490005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:56.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.609+0000 7f849d7fa700 1 -- 172.21.15.144:0/646436900 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8494058a50 con 0x7f84a00ffc60 2024-09-10T11:56:56.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.856+0000 7f84a6a43700 1 -- 172.21.15.144:0/646436900 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f84a002cd20 con 0x7f84a00ffc60 2024-09-10T11:56:56.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.857+0000 7f849d7fa700 1 -- 172.21.15.144:0/646436900 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f849405c110 con 0x7f84a00ffc60 2024-09-10T11:56:56.858 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T11:56:56.858 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T11:56:56.858 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:56:56.858 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:56:56.858 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:56:56.859 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:56:56.859 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:56:56.859 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:56:56.859 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:56:56.859 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:56:56.859 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:56:56.859 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:56:56.859 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:56:56.859 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:56:56.859 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:56:56.859 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:56:56.859 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:56:56.859 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:56:56.859 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:56:56.860 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:56:56.860 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:56:56.860 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:56:56.860 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:56:56.860 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:56:56.860 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:56:56.860 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:56:56.860 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:56:56.860 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:56:56.860 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:56:56.860 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:56:56.861 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:56:56.861 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:56:56.861 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:56:56.861 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:56:56.861 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:56:56.861 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:56:56.861 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:56:56.861 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:56:56.861 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:56:56.861 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:56:56.862 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:56:56.862 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:56:56.862 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:56:56.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.859+0000 7f84a6a43700 1 -- 172.21.15.144:0/646436900 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f848806e200 msgr2=0x7f84880706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:56.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.859+0000 7f84a6a43700 1 --2- 172.21.15.144:0/646436900 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f848806e200 0x7f84880706b0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8490005fd0 tx=0x7f8490005dc0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.859+0000 7f84a6a43700 1 -- 172.21.15.144:0/646436900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a00ffc60 msgr2=0x7f84a0197100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:56.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.859+0000 7f84a6a43700 1 --2- 172.21.15.144:0/646436900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a00ffc60 0x7f84a0197100 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f8494005f50 tx=0x7f849400be20 comp rx=0 tx=0).stop 2024-09-10T11:56:56.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.859+0000 7f84a6a43700 1 -- 172.21.15.144:0/646436900 shutdown_connections 2024-09-10T11:56:56.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.859+0000 7f84a6a43700 1 --2- 172.21.15.144:0/646436900 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f848806e200 0x7f84880706b0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.859+0000 7f84a6a43700 1 --2- 172.21.15.144:0/646436900 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84a00ff2c0 0x7f84a0196bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.859+0000 7f84a6a43700 1 --2- 172.21.15.144:0/646436900 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84a00ffc60 0x7f84a0197100 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.859+0000 7f84a6a43700 1 -- 172.21.15.144:0/646436900 >> 172.21.15.144:0/646436900 conn(0x7f84a00fab80 msgr2=0x7f84a00fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:56.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.859+0000 7f84a6a43700 1 -- 172.21.15.144:0/646436900 shutdown_connections 2024-09-10T11:56:56.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.859+0000 7f84a6a43700 1 -- 172.21.15.144:0/646436900 wait complete. 2024-09-10T11:56:56.863 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T11:56:56.967 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.966+0000 7f1399396700 1 -- 172.21.15.144:0/3633677238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13940ff340 msgr2=0x7f13940ff710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:56.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.966+0000 7f1399396700 1 --2- 172.21.15.144:0/3633677238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13940ff340 0x7f13940ff710 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f1388009910 tx=0x7f1388009c20 comp rx=0 tx=0).stop 2024-09-10T11:56:56.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.966+0000 7f1399396700 1 -- 172.21.15.144:0/3633677238 shutdown_connections 2024-09-10T11:56:56.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.966+0000 7f1399396700 1 --2- 172.21.15.144:0/3633677238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13940ffce0 0x7f1394103cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.966+0000 7f1399396700 1 --2- 172.21.15.144:0/3633677238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13940ff340 0x7f13940ff710 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.966+0000 7f1399396700 1 -- 172.21.15.144:0/3633677238 >> 172.21.15.144:0/3633677238 conn(0x7f13940fabe0 msgr2=0x7f13940fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:56.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.966+0000 7f1399396700 1 -- 172.21.15.144:0/3633677238 shutdown_connections 2024-09-10T11:56:56.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.966+0000 7f1399396700 1 -- 172.21.15.144:0/3633677238 wait complete. 2024-09-10T11:56:56.968 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.967+0000 7f1399396700 1 Processor -- start 2024-09-10T11:56:56.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.967+0000 7f1399396700 1 -- start start 2024-09-10T11:56:56.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.967+0000 7f1399396700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13940ff340 0x7f1394196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:56.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.967+0000 7f1399396700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13940ffce0 0x7f13941970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:56.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.967+0000 7f1399396700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13941976e0 con 0x7f13940ff340 2024-09-10T11:56:56.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.967+0000 7f1399396700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1394197850 con 0x7f13940ffce0 2024-09-10T11:56:56.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.967+0000 7f1392ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13940ff340 0x7f1394196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.967+0000 7f1392ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13940ff340 0x7f1394196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:47026/0 (socket says 172.21.15.144:47026) 2024-09-10T11:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.967+0000 7f1392ffd700 1 -- 172.21.15.144:0/3946363282 learned_addr learned my addr 172.21.15.144:0/3946363282 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.967+0000 7f13927fc700 1 --2- 172.21.15.144:0/3946363282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13940ffce0 0x7f13941970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.967+0000 7f1392ffd700 1 -- 172.21.15.144:0/3946363282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13940ffce0 msgr2=0x7f13941970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.967+0000 7f1392ffd700 1 --2- 172.21.15.144:0/3946363282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13940ffce0 0x7f13941970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.967+0000 7f1392ffd700 1 -- 172.21.15.144:0/3946363282 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13880095f0 con 0x7f13940ff340 2024-09-10T11:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.967+0000 7f13927fc700 1 --2- 172.21.15.144:0/3946363282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13940ffce0 0x7f13941970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.968+0000 7f1392ffd700 1 --2- 172.21.15.144:0/3946363282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13940ff340 0x7f1394196b80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1388005f50 tx=0x7f138800f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.968+0000 7f137bfff700 1 -- 172.21.15.144:0/3946363282 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f138801d070 con 0x7f13940ff340 2024-09-10T11:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.968+0000 7f137bfff700 1 -- 172.21.15.144:0/3946363282 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f138800b610 con 0x7f13940ff340 2024-09-10T11:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.968+0000 7f1399396700 1 -- 172.21.15.144:0/3946363282 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f139419fbf0 con 0x7f13940ff340 2024-09-10T11:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.968+0000 7f137bfff700 1 -- 172.21.15.144:0/3946363282 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1388022470 con 0x7f13940ff340 2024-09-10T11:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.968+0000 7f1399396700 1 -- 172.21.15.144:0/3946363282 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13941a0060 con 0x7f13940ff340 2024-09-10T11:56:56.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.969+0000 7f1399396700 1 -- 172.21.15.144:0/3946363282 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1394107650 con 0x7f13940ff340 2024-09-10T11:56:56.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.969+0000 7f137bfff700 1 -- 172.21.15.144:0/3946363282 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f138800fca0 con 0x7f13940ff340 2024-09-10T11:56:56.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.970+0000 7f137bfff700 1 --2- 172.21.15.144:0/3946363282 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f137c06e200 0x7f137c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:56.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.970+0000 7f13927fc700 1 --2- 172.21.15.144:0/3946363282 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f137c06e200 0x7f137c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:56.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.970+0000 7f137bfff700 1 -- 172.21.15.144:0/3946363282 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f138808f520 con 0x7f13940ff340 2024-09-10T11:56:56.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.970+0000 7f13927fc700 1 --2- 172.21.15.144:0/3946363282 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f137c06e200 0x7f137c0706b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1394069680 tx=0x7f1384009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:56.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:56.974+0000 7f137bfff700 1 -- 172.21.15.144:0/3946363282 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1388058900 con 0x7f13940ff340 2024-09-10T11:56:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:56 smithi144 ceph-mon[87436]: from='client.35388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:56 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2086896418' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:56:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:56 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/646436900' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:56:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:56 smithi144 ceph-mon[87436]: pgmap v750: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:57.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.202+0000 7f1399396700 1 -- 172.21.15.144:0/3946363282 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1394061190 con 0x7f137c06e200 2024-09-10T11:56:57.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.204+0000 7f137bfff700 1 -- 172.21.15.144:0/3946363282 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1394061190 con 0x7f137c06e200 2024-09-10T11:56:57.205 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:56:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:56:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:56:57.205 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:56:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:56:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:56:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T11:56:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T11:56:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:56:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T11:56:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:56:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T11:56:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:56:57.206 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:56:57.206 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:56:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.206+0000 7f1399396700 1 -- 172.21.15.144:0/3946363282 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f137c06e200 msgr2=0x7f137c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.206+0000 7f1399396700 1 --2- 172.21.15.144:0/3946363282 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f137c06e200 0x7f137c0706b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1394069680 tx=0x7f1384009260 comp rx=0 tx=0).stop 2024-09-10T11:56:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.206+0000 7f1399396700 1 -- 172.21.15.144:0/3946363282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13940ff340 msgr2=0x7f1394196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.206+0000 7f1399396700 1 --2- 172.21.15.144:0/3946363282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13940ff340 0x7f1394196b80 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f1388005f50 tx=0x7f138800f740 comp rx=0 tx=0).stop 2024-09-10T11:56:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.206+0000 7f1399396700 1 -- 172.21.15.144:0/3946363282 shutdown_connections 2024-09-10T11:56:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.206+0000 7f1399396700 1 --2- 172.21.15.144:0/3946363282 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f137c06e200 0x7f137c0706b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.206+0000 7f1399396700 1 --2- 172.21.15.144:0/3946363282 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f13940ff340 0x7f1394196b80 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.206+0000 7f1399396700 1 --2- 172.21.15.144:0/3946363282 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f13940ffce0 0x7f13941970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.206+0000 7f1399396700 1 -- 172.21.15.144:0/3946363282 >> 172.21.15.144:0/3946363282 conn(0x7f13940fabe0 msgr2=0x7f13940fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.206+0000 7f1399396700 1 -- 172.21.15.144:0/3946363282 shutdown_connections 2024-09-10T11:56:57.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.206+0000 7f1399396700 1 -- 172.21.15.144:0/3946363282 wait complete. 2024-09-10T11:56:57.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.311+0000 7f0aacf96700 1 -- 172.21.15.144:0/3359005707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa81003d0 msgr2=0x7f0aa81007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:57.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.311+0000 7f0aacf96700 1 --2- 172.21.15.144:0/3359005707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa81003d0 0x7f0aa81007a0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009910 tx=0x7f0a9c009c20 comp rx=0 tx=0).stop 2024-09-10T11:56:57.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.311+0000 7f0aacf96700 1 -- 172.21.15.144:0/3359005707 shutdown_connections 2024-09-10T11:56:57.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.311+0000 7f0aacf96700 1 --2- 172.21.15.144:0/3359005707 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa8100d70 0x7f0aa8104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:57.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.311+0000 7f0aacf96700 1 --2- 172.21.15.144:0/3359005707 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa81003d0 0x7f0aa81007a0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:57.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.311+0000 7f0aacf96700 1 -- 172.21.15.144:0/3359005707 >> 172.21.15.144:0/3359005707 conn(0x7f0aa80fbc90 msgr2=0x7f0aa80fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:57.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.311+0000 7f0aacf96700 1 -- 172.21.15.144:0/3359005707 shutdown_connections 2024-09-10T11:56:57.314 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.311+0000 7f0aacf96700 1 -- 172.21.15.144:0/3359005707 wait complete. 2024-09-10T11:56:57.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.312+0000 7f0aacf96700 1 Processor -- start 2024-09-10T11:56:57.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.312+0000 7f0aacf96700 1 -- start start 2024-09-10T11:56:57.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.312+0000 7f0aacf96700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa81003d0 0x7f0aa8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:57.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.312+0000 7f0aacf96700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa8100d70 0x7f0aa8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:57.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.312+0000 7f0aacf96700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa8073ad0 con 0x7f0aa81003d0 2024-09-10T11:56:57.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.312+0000 7f0aacf96700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa8073c10 con 0x7f0aa8100d70 2024-09-10T11:56:57.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.312+0000 7f0aa659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa81003d0 0x7f0aa8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:57.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.313+0000 7f0aa659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa81003d0 0x7f0aa8073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:47040/0 (socket says 172.21.15.144:47040) 2024-09-10T11:56:57.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.313+0000 7f0aa659c700 1 -- 172.21.15.144:0/2382978238 learned_addr learned my addr 172.21.15.144:0/2382978238 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:56:57.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.313+0000 7f0aa5d9b700 1 --2- 172.21.15.144:0/2382978238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa8100d70 0x7f0aa8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:57.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.313+0000 7f0aa659c700 1 -- 172.21.15.144:0/2382978238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa8100d70 msgr2=0x7f0aa8073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:57.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.313+0000 7f0aa659c700 1 --2- 172.21.15.144:0/2382978238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa8100d70 0x7f0aa8073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:57.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.313+0000 7f0aa659c700 1 -- 172.21.15.144:0/2382978238 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a9c0095f0 con 0x7f0aa81003d0 2024-09-10T11:56:57.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.313+0000 7f0aa659c700 1 --2- 172.21.15.144:0/2382978238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa81003d0 0x7f0aa8073050 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c006010 tx=0x7f0a9c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:57.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.313+0000 7f0a977fe700 1 -- 172.21.15.144:0/2382978238 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a9c01d070 con 0x7f0aa81003d0 2024-09-10T11:56:57.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.313+0000 7f0a977fe700 1 -- 172.21.15.144:0/2382978238 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0a9c00f460 con 0x7f0aa81003d0 2024-09-10T11:56:57.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.313+0000 7f0a977fe700 1 -- 172.21.15.144:0/2382978238 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a9c017630 con 0x7f0aa81003d0 2024-09-10T11:56:57.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.313+0000 7f0aacf96700 1 -- 172.21.15.144:0/2382978238 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa8073e90 con 0x7f0aa81003d0 2024-09-10T11:56:57.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.313+0000 7f0aacf96700 1 -- 172.21.15.144:0/2382978238 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa81a2290 con 0x7f0aa81003d0 2024-09-10T11:56:57.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.314+0000 7f0aacf96700 1 -- 172.21.15.144:0/2382978238 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa81086e0 con 0x7f0aa81003d0 2024-09-10T11:56:57.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.315+0000 7f0a977fe700 1 -- 172.21.15.144:0/2382978238 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0a9c00f5d0 con 0x7f0aa81003d0 2024-09-10T11:56:57.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.315+0000 7f0a977fe700 1 --2- 172.21.15.144:0/2382978238 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0a9006e200 0x7f0a900706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:56:57.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.315+0000 7f0aa5d9b700 1 --2- 172.21.15.144:0/2382978238 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0a9006e200 0x7f0a900706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:56:57.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.315+0000 7f0a977fe700 1 -- 172.21.15.144:0/2382978238 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0a9c08f500 con 0x7f0aa81003d0 2024-09-10T11:56:57.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.316+0000 7f0aa5d9b700 1 --2- 172.21.15.144:0/2382978238 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0a9006e200 0x7f0a900706b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0a98005950 tx=0x7f0a980058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:56:57.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.320+0000 7f0a977fe700 1 -- 172.21.15.144:0/2382978238 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0a9c0588e0 con 0x7f0aa81003d0 2024-09-10T11:56:57.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:56 smithi178 ceph-mon[65940]: from='client.35388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:57.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:56 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2086896418' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:56:57.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:56 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/646436900' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:56:57.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:56 smithi178 ceph-mon[65940]: pgmap v750: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:57.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.606+0000 7f0aacf96700 1 -- 172.21.15.144:0/2382978238 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0aa804ea50 con 0x7f0aa81003d0 2024-09-10T11:56:57.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.607+0000 7f0a977fe700 1 -- 172.21.15.144:0/2382978238 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f0a9c05bfa0 con 0x7f0aa81003d0 2024-09-10T11:56:57.608 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:56:57.608 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T11:56:57.608 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:56:57.608 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T11:56:57.608 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:56:57.609 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T11:56:57.609 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T11:56:57.609 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T11:56:57.609 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T11:56:57.609 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T11:56:57.609 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T11:56:57.609 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T11:56:57.609 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T11:56:57.609 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T11:56:57.609 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T11:56:57.609 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T11:56:57.609 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T11:56:57.609 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T11:56:57.610 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:56:57.610 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:56:57.610 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:56:57.610 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:56:57.610 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:56:57.610 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:56:57.610 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:56:57.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.610+0000 7f0aacf96700 1 -- 172.21.15.144:0/2382978238 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0a9006e200 msgr2=0x7f0a900706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:57.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.610+0000 7f0aacf96700 1 --2- 172.21.15.144:0/2382978238 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0a9006e200 0x7f0a900706b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f0a98005950 tx=0x7f0a980058e0 comp rx=0 tx=0).stop 2024-09-10T11:56:57.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.610+0000 7f0aacf96700 1 -- 172.21.15.144:0/2382978238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa81003d0 msgr2=0x7f0aa8073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:56:57.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.610+0000 7f0aacf96700 1 --2- 172.21.15.144:0/2382978238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa81003d0 0x7f0aa8073050 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c006010 tx=0x7f0a9c00be20 comp rx=0 tx=0).stop 2024-09-10T11:56:57.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.610+0000 7f0aacf96700 1 -- 172.21.15.144:0/2382978238 shutdown_connections 2024-09-10T11:56:57.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.610+0000 7f0aacf96700 1 --2- 172.21.15.144:0/2382978238 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0a9006e200 0x7f0a900706b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:57.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.610+0000 7f0aacf96700 1 --2- 172.21.15.144:0/2382978238 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0aa81003d0 0x7f0aa8073050 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:57.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.610+0000 7f0aacf96700 1 --2- 172.21.15.144:0/2382978238 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0aa8100d70 0x7f0aa8073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:56:57.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.610+0000 7f0aacf96700 1 -- 172.21.15.144:0/2382978238 >> 172.21.15.144:0/2382978238 conn(0x7f0aa80fbc90 msgr2=0x7f0aa80fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:56:57.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.610+0000 7f0aacf96700 1 -- 172.21.15.144:0/2382978238 shutdown_connections 2024-09-10T11:56:57.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:56:57.610+0000 7f0aacf96700 1 -- 172.21.15.144:0/2382978238 wait complete. 2024-09-10T11:56:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:57 smithi178 ceph-mon[65940]: from='client.35400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:57 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2382978238' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:56:58.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:57 smithi144 ceph-mon[87436]: from='client.35400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:56:58.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:57 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2382978238' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:56:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:56:59.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:56:58 smithi178 ceph-mon[65940]: pgmap v751: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:56:59.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:56:59.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:56:58 smithi144 ceph-mon[87436]: pgmap v751: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:00 smithi178 ceph-mon[65940]: pgmap v752: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:01.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:00 smithi144 ceph-mon[87436]: pgmap v752: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:02 smithi178 ceph-mon[65940]: pgmap v753: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:03.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:02 smithi144 ceph-mon[87436]: pgmap v753: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:04 smithi178 ceph-mon[65940]: pgmap v754: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:05.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:04 smithi144 ceph-mon[87436]: pgmap v754: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:06 smithi178 ceph-mon[65940]: pgmap v755: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:07.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:06 smithi144 ceph-mon[87436]: pgmap v755: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:08 smithi178 ceph-mon[65940]: pgmap v756: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:09.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:08 smithi144 ceph-mon[87436]: pgmap v756: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:10 smithi178 ceph-mon[65940]: pgmap v757: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:11.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:10 smithi144 ceph-mon[87436]: pgmap v757: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:13.236 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:12 smithi144 ceph-mon[87436]: pgmap v758: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:12 smithi178 ceph-mon[65940]: pgmap v758: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:57:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:57:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:14 smithi178 ceph-mon[65940]: pgmap v759: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:15.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:14 smithi144 ceph-mon[87436]: pgmap v759: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:16 smithi178 ceph-mon[65940]: pgmap v760: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:17.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:16 smithi144 ceph-mon[87436]: pgmap v760: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:19.250 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:18 smithi178 ceph-mon[65940]: pgmap v761: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:19.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:18 smithi144 ceph-mon[87436]: pgmap v761: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:21.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:20 smithi144 ceph-mon[87436]: pgmap v762: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:20 smithi178 ceph-mon[65940]: pgmap v762: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:23.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:22 smithi178 ceph-mon[65940]: pgmap v763: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:23.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:22 smithi144 ceph-mon[87436]: pgmap v763: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:24 smithi178 ceph-mon[65940]: pgmap v764: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:25.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:24 smithi144 ceph-mon[87436]: pgmap v764: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:26 smithi178 ceph-mon[65940]: pgmap v765: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:27.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:26 smithi144 ceph-mon[87436]: pgmap v765: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:27.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.730+0000 7f47e180e700 1 -- 172.21.15.144:0/309677770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ffd30 msgr2=0x7f47dc103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:27.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.730+0000 7f47e180e700 1 --2- 172.21.15.144:0/309677770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ffd30 0x7f47dc103ce0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009910 tx=0x7f47c4009c20 comp rx=0 tx=0).stop 2024-09-10T11:57:27.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.730+0000 7f47e180e700 1 -- 172.21.15.144:0/309677770 shutdown_connections 2024-09-10T11:57:27.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.730+0000 7f47e180e700 1 --2- 172.21.15.144:0/309677770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ffd30 0x7f47dc103ce0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:27.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.730+0000 7f47e180e700 1 --2- 172.21.15.144:0/309677770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47dc0ff390 0x7f47dc0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:27.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.730+0000 7f47e180e700 1 -- 172.21.15.144:0/309677770 >> 172.21.15.144:0/309677770 conn(0x7f47dc074f50 msgr2=0x7f47dc075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:57:27.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.730+0000 7f47e180e700 1 -- 172.21.15.144:0/309677770 shutdown_connections 2024-09-10T11:57:27.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.730+0000 7f47e180e700 1 -- 172.21.15.144:0/309677770 wait complete. 2024-09-10T11:57:27.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.731+0000 7f47e180e700 1 Processor -- start 2024-09-10T11:57:27.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.731+0000 7f47e180e700 1 -- start start 2024-09-10T11:57:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.731+0000 7f47e180e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47dc0ff390 0x7f47dc194970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.731+0000 7f47e180e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ffd30 0x7f47dc194eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.731+0000 7f47e180e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc1953f0 con 0x7f47dc0ffd30 2024-09-10T11:57:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.731+0000 7f47e180e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc195560 con 0x7f47dc0ff390 2024-09-10T11:57:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.731+0000 7f47da7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ffd30 0x7f47dc194eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.732+0000 7f47da7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ffd30 0x7f47dc194eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41854/0 (socket says 172.21.15.144:41854) 2024-09-10T11:57:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.732+0000 7f47da7fc700 1 -- 172.21.15.144:0/809690456 learned_addr learned my addr 172.21.15.144:0/809690456 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:57:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.732+0000 7f47daffd700 1 --2- 172.21.15.144:0/809690456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47dc0ff390 0x7f47dc194970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:27.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.732+0000 7f47da7fc700 1 -- 172.21.15.144:0/809690456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47dc0ff390 msgr2=0x7f47dc194970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.732+0000 7f47da7fc700 1 --2- 172.21.15.144:0/809690456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47dc0ff390 0x7f47dc194970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.732+0000 7f47da7fc700 1 -- 172.21.15.144:0/809690456 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47c40095f0 con 0x7f47dc0ffd30 2024-09-10T11:57:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.732+0000 7f47da7fc700 1 --2- 172.21.15.144:0/809690456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ffd30 0x7f47dc194eb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f47c400bda0 tx=0x7f47c400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:57:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.732+0000 7f47e080c700 1 -- 172.21.15.144:0/809690456 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47c401d070 con 0x7f47dc0ffd30 2024-09-10T11:57:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.732+0000 7f47e080c700 1 -- 172.21.15.144:0/809690456 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f47c4005dc0 con 0x7f47dc0ffd30 2024-09-10T11:57:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.732+0000 7f47e080c700 1 -- 172.21.15.144:0/809690456 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f47c4017400 con 0x7f47dc0ffd30 2024-09-10T11:57:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.732+0000 7f47e180e700 1 -- 172.21.15.144:0/809690456 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47dc1957e0 con 0x7f47dc0ffd30 2024-09-10T11:57:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.732+0000 7f47e180e700 1 -- 172.21.15.144:0/809690456 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47dc1a2260 con 0x7f47dc0ffd30 2024-09-10T11:57:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.733+0000 7f47e180e700 1 -- 172.21.15.144:0/809690456 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc107600 con 0x7f47dc0ffd30 2024-09-10T11:57:27.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.734+0000 7f47e080c700 1 -- 172.21.15.144:0/809690456 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f47c400f6f0 con 0x7f47dc0ffd30 2024-09-10T11:57:27.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.734+0000 7f47e080c700 1 --2- 172.21.15.144:0/809690456 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f47c806e200 0x7f47c80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:27.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.734+0000 7f47e080c700 1 -- 172.21.15.144:0/809690456 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f47c408f2f0 con 0x7f47dc0ffd30 2024-09-10T11:57:27.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.734+0000 7f47daffd700 1 --2- 172.21.15.144:0/809690456 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f47c806e200 0x7f47c80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:27.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.735+0000 7f47daffd700 1 --2- 172.21.15.144:0/809690456 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f47c806e200 0x7f47c80706b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009550 tx=0x7f47cc0091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:57:27.740 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.739+0000 7f47e080c700 1 -- 172.21.15.144:0/809690456 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f47c405c200 con 0x7f47dc0ffd30 2024-09-10T11:57:27.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.960+0000 7f47e180e700 1 -- 172.21.15.144:0/809690456 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f47dc1957e0 con 0x7f47c806e200 2024-09-10T11:57:27.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.961+0000 7f47e080c700 1 -- 172.21.15.144:0/809690456 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f47dc1957e0 con 0x7f47c806e200 2024-09-10T11:57:27.965 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.963+0000 7f47e180e700 1 -- 172.21.15.144:0/809690456 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f47c806e200 msgr2=0x7f47c80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:27.965 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.963+0000 7f47e180e700 1 --2- 172.21.15.144:0/809690456 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f47c806e200 0x7f47c80706b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009550 tx=0x7f47cc0091b0 comp rx=0 tx=0).stop 2024-09-10T11:57:27.965 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.963+0000 7f47e180e700 1 -- 172.21.15.144:0/809690456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ffd30 msgr2=0x7f47dc194eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:27.965 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.963+0000 7f47e180e700 1 --2- 172.21.15.144:0/809690456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ffd30 0x7f47dc194eb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f47c400bda0 tx=0x7f47c400bdd0 comp rx=0 tx=0).stop 2024-09-10T11:57:27.965 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.963+0000 7f47e180e700 1 -- 172.21.15.144:0/809690456 shutdown_connections 2024-09-10T11:57:27.965 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.963+0000 7f47e180e700 1 --2- 172.21.15.144:0/809690456 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f47c806e200 0x7f47c80706b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:27.965 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.963+0000 7f47e180e700 1 --2- 172.21.15.144:0/809690456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47dc0ff390 0x7f47dc194970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:27.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.963+0000 7f47e180e700 1 --2- 172.21.15.144:0/809690456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47dc0ffd30 0x7f47dc194eb0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:27.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.963+0000 7f47e180e700 1 -- 172.21.15.144:0/809690456 >> 172.21.15.144:0/809690456 conn(0x7f47dc074f50 msgr2=0x7f47dc0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:57:27.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.964+0000 7f47e180e700 1 -- 172.21.15.144:0/809690456 shutdown_connections 2024-09-10T11:57:27.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:27.964+0000 7f47e180e700 1 -- 172.21.15.144:0/809690456 wait complete. 2024-09-10T11:57:27.975 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:57:28.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.068+0000 7fb04cc37700 1 -- 172.21.15.144:0/3783444264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb048100d70 msgr2=0x7fb048104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:28.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.068+0000 7fb04cc37700 1 --2- 172.21.15.144:0/3783444264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb048100d70 0x7fb048104d40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fb03c009910 tx=0x7fb03c009c20 comp rx=0 tx=0).stop 2024-09-10T11:57:28.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.069+0000 7fb04cc37700 1 -- 172.21.15.144:0/3783444264 shutdown_connections 2024-09-10T11:57:28.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.069+0000 7fb04cc37700 1 --2- 172.21.15.144:0/3783444264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb048100d70 0x7fb048104d40 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.069+0000 7fb04cc37700 1 --2- 172.21.15.144:0/3783444264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb0481003d0 0x7fb0481007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.069+0000 7fb04cc37700 1 -- 172.21.15.144:0/3783444264 >> 172.21.15.144:0/3783444264 conn(0x7fb0480fbc50 msgr2=0x7fb0480fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:57:28.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.069+0000 7fb04cc37700 1 -- 172.21.15.144:0/3783444264 shutdown_connections 2024-09-10T11:57:28.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.069+0000 7fb04cc37700 1 -- 172.21.15.144:0/3783444264 wait complete. 2024-09-10T11:57:28.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.069+0000 7fb04cc37700 1 Processor -- start 2024-09-10T11:57:28.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.069+0000 7fb04cc37700 1 -- start start 2024-09-10T11:57:28.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.070+0000 7fb04cc37700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0481003d0 0x7fb048073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:28.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.070+0000 7fb04cc37700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb048100d70 0x7fb048073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:28.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.070+0000 7fb04cc37700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb048073ad0 con 0x7fb0481003d0 2024-09-10T11:57:28.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.070+0000 7fb04cc37700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb048073c10 con 0x7fb048100d70 2024-09-10T11:57:28.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.070+0000 7fb04659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0481003d0 0x7fb048073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:28.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.070+0000 7fb04659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0481003d0 0x7fb048073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41884/0 (socket says 172.21.15.144:41884) 2024-09-10T11:57:28.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.070+0000 7fb04659c700 1 -- 172.21.15.144:0/36475447 learned_addr learned my addr 172.21.15.144:0/36475447 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:57:28.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.070+0000 7fb045d9b700 1 --2- 172.21.15.144:0/36475447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb048100d70 0x7fb048073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:28.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.070+0000 7fb04659c700 1 -- 172.21.15.144:0/36475447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb048100d70 msgr2=0x7fb048073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:28.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.070+0000 7fb04659c700 1 --2- 172.21.15.144:0/36475447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb048100d70 0x7fb048073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.070+0000 7fb04659c700 1 -- 172.21.15.144:0/36475447 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb03c0095f0 con 0x7fb0481003d0 2024-09-10T11:57:28.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.070+0000 7fb04659c700 1 --2- 172.21.15.144:0/36475447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0481003d0 0x7fb048073050 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb03800e920 tx=0x7fb03800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:57:28.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.071+0000 7fb0377fe700 1 -- 172.21.15.144:0/36475447 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb03800c9d0 con 0x7fb0481003d0 2024-09-10T11:57:28.073 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.071+0000 7fb0377fe700 1 -- 172.21.15.144:0/36475447 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb038010430 con 0x7fb0481003d0 2024-09-10T11:57:28.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.071+0000 7fb0377fe700 1 -- 172.21.15.144:0/36475447 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb038018630 con 0x7fb0481003d0 2024-09-10T11:57:28.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.071+0000 7fb04cc37700 1 -- 172.21.15.144:0/36475447 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0481a1fd0 con 0x7fb0481003d0 2024-09-10T11:57:28.074 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.071+0000 7fb04cc37700 1 -- 172.21.15.144:0/36475447 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0481a2380 con 0x7fb0481003d0 2024-09-10T11:57:28.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.072+0000 7fb04cc37700 1 -- 172.21.15.144:0/36475447 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb048108740 con 0x7fb0481003d0 2024-09-10T11:57:28.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.072+0000 7fb0377fe700 1 -- 172.21.15.144:0/36475447 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb0380105a0 con 0x7fb0481003d0 2024-09-10T11:57:28.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.073+0000 7fb0377fe700 1 --2- 172.21.15.144:0/36475447 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb03006e200 0x7fb0300706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:28.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.073+0000 7fb0377fe700 1 -- 172.21.15.144:0/36475447 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb038014070 con 0x7fb0481003d0 2024-09-10T11:57:28.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.073+0000 7fb045d9b700 1 --2- 172.21.15.144:0/36475447 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb03006e200 0x7fb0300706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:28.075 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.073+0000 7fb045d9b700 1 --2- 172.21.15.144:0/36475447 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb03006e200 0x7fb0300706b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0059d0 tx=0x7fb03c005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:57:28.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.077+0000 7fb0377fe700 1 -- 172.21.15.144:0/36475447 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb0380541b0 con 0x7fb0481003d0 2024-09-10T11:57:28.298 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.297+0000 7fb04cc37700 1 -- 172.21.15.144:0/36475447 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb048061190 con 0x7fb03006e200 2024-09-10T11:57:28.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.298+0000 7fb0377fe700 1 -- 172.21.15.144:0/36475447 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb048061190 con 0x7fb03006e200 2024-09-10T11:57:28.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.300+0000 7fb04cc37700 1 -- 172.21.15.144:0/36475447 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb03006e200 msgr2=0x7fb0300706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:28.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.300+0000 7fb04cc37700 1 --2- 172.21.15.144:0/36475447 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb03006e200 0x7fb0300706b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb03c0059d0 tx=0x7fb03c005940 comp rx=0 tx=0).stop 2024-09-10T11:57:28.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.300+0000 7fb04cc37700 1 -- 172.21.15.144:0/36475447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0481003d0 msgr2=0x7fb048073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:28.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.300+0000 7fb04cc37700 1 --2- 172.21.15.144:0/36475447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0481003d0 0x7fb048073050 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fb03800e920 tx=0x7fb03800ec30 comp rx=0 tx=0).stop 2024-09-10T11:57:28.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.300+0000 7fb04cc37700 1 -- 172.21.15.144:0/36475447 shutdown_connections 2024-09-10T11:57:28.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.300+0000 7fb04cc37700 1 --2- 172.21.15.144:0/36475447 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb03006e200 0x7fb0300706b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.300+0000 7fb04cc37700 1 --2- 172.21.15.144:0/36475447 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb0481003d0 0x7fb048073050 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.300+0000 7fb04cc37700 1 --2- 172.21.15.144:0/36475447 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb048100d70 0x7fb048073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.300+0000 7fb04cc37700 1 -- 172.21.15.144:0/36475447 >> 172.21.15.144:0/36475447 conn(0x7fb0480fbc50 msgr2=0x7fb0480fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:57:28.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.300+0000 7fb04cc37700 1 -- 172.21.15.144:0/36475447 shutdown_connections 2024-09-10T11:57:28.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.300+0000 7fb04cc37700 1 -- 172.21.15.144:0/36475447 wait complete. 2024-09-10T11:57:28.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.405+0000 7f1de3f50700 1 -- 172.21.15.144:0/1984290639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ddc0ffc60 msgr2=0x7f1ddc103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:28.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.405+0000 7f1de3f50700 1 --2- 172.21.15.144:0/1984290639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ddc0ffc60 0x7f1ddc103c30 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009910 tx=0x7f1dd8009c20 comp rx=0 tx=0).stop 2024-09-10T11:57:28.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.405+0000 7f1de3f50700 1 -- 172.21.15.144:0/1984290639 shutdown_connections 2024-09-10T11:57:28.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.405+0000 7f1de3f50700 1 --2- 172.21.15.144:0/1984290639 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ddc0ffc60 0x7f1ddc103c30 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.405+0000 7f1de3f50700 1 --2- 172.21.15.144:0/1984290639 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ddc0ff2c0 0x7f1ddc0ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.405+0000 7f1de3f50700 1 -- 172.21.15.144:0/1984290639 >> 172.21.15.144:0/1984290639 conn(0x7f1ddc0fab80 msgr2=0x7f1ddc0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:57:28.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.405+0000 7f1de3f50700 1 -- 172.21.15.144:0/1984290639 shutdown_connections 2024-09-10T11:57:28.407 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.405+0000 7f1de3f50700 1 -- 172.21.15.144:0/1984290639 wait complete. 2024-09-10T11:57:28.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.406+0000 7f1de3f50700 1 Processor -- start 2024-09-10T11:57:28.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.406+0000 7f1de3f50700 1 -- start start 2024-09-10T11:57:28.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.406+0000 7f1de3f50700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ddc0ff2c0 0x7f1ddc195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:28.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.406+0000 7f1de3f50700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ddc0ffc60 0x7f1ddc196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:28.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.406+0000 7f1de3f50700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc196950 con 0x7f1ddc0ff2c0 2024-09-10T11:57:28.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.406+0000 7f1de3f50700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ddc19a6e0 con 0x7f1ddc0ffc60 2024-09-10T11:57:28.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.406+0000 7f1de1cec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ddc0ff2c0 0x7f1ddc195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:28.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.406+0000 7f1de1cec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ddc0ff2c0 0x7f1ddc195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41894/0 (socket says 172.21.15.144:41894) 2024-09-10T11:57:28.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.406+0000 7f1de1cec700 1 -- 172.21.15.144:0/737375883 learned_addr learned my addr 172.21.15.144:0/737375883 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:57:28.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.407+0000 7f1de14eb700 1 --2- 172.21.15.144:0/737375883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ddc0ffc60 0x7f1ddc196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:28.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.407+0000 7f1de1cec700 1 -- 172.21.15.144:0/737375883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ddc0ffc60 msgr2=0x7f1ddc196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:28.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.407+0000 7f1de1cec700 1 --2- 172.21.15.144:0/737375883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ddc0ffc60 0x7f1ddc196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.407+0000 7f1de1cec700 1 -- 172.21.15.144:0/737375883 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd80095f0 con 0x7f1ddc0ff2c0 2024-09-10T11:57:28.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.407+0000 7f1de1cec700 1 --2- 172.21.15.144:0/737375883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ddc0ff2c0 0x7f1ddc195d30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00c8a0 tx=0x7f1dcc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:57:28.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.407+0000 7f1dd2ffd700 1 -- 172.21.15.144:0/737375883 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dcc00cea0 con 0x7f1ddc0ff2c0 2024-09-10T11:57:28.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.407+0000 7f1dd2ffd700 1 -- 172.21.15.144:0/737375883 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1dcc004830 con 0x7f1ddc0ff2c0 2024-09-10T11:57:28.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.407+0000 7f1de3f50700 1 -- 172.21.15.144:0/737375883 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ddc19a9c0 con 0x7f1ddc0ff2c0 2024-09-10T11:57:28.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.407+0000 7f1dd2ffd700 1 -- 172.21.15.144:0/737375883 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1dcc0055b0 con 0x7f1ddc0ff2c0 2024-09-10T11:57:28.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.407+0000 7f1de3f50700 1 -- 172.21.15.144:0/737375883 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ddc19af10 con 0x7f1ddc0ff2c0 2024-09-10T11:57:28.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.408+0000 7f1de3f50700 1 -- 172.21.15.144:0/737375883 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ddc108640 con 0x7f1ddc0ff2c0 2024-09-10T11:57:28.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.409+0000 7f1dd2ffd700 1 -- 172.21.15.144:0/737375883 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1dcc0049a0 con 0x7f1ddc0ff2c0 2024-09-10T11:57:28.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.410+0000 7f1dd2ffd700 1 --2- 172.21.15.144:0/737375883 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1dc806e200 0x7f1dc80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:28.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.410+0000 7f1dd2ffd700 1 -- 172.21.15.144:0/737375883 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1dcc08d930 con 0x7f1ddc0ff2c0 2024-09-10T11:57:28.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.410+0000 7f1de14eb700 1 --2- 172.21.15.144:0/737375883 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1dc806e200 0x7f1dc80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:28.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.410+0000 7f1de14eb700 1 --2- 172.21.15.144:0/737375883 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1dc806e200 0x7f1dc80706b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc197350 tx=0x7f1dd801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:57:28.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.414+0000 7f1dd2ffd700 1 -- 172.21.15.144:0/737375883 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1dcc056e30 con 0x7f1ddc0ff2c0 2024-09-10T11:57:28.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.625+0000 7f1de3f50700 1 -- 172.21.15.144:0/737375883 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ddc19b2b0 con 0x7f1dc806e200 2024-09-10T11:57:28.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.641+0000 7f1dd2ffd700 1 -- 172.21.15.144:0/737375883 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f1ddc19b2b0 con 0x7f1dc806e200 2024-09-10T11:57:28.642 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:57:28.643 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (25m) 4m ago 36m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:57:28.643 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (36m) 4m ago 36m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:57:28.643 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (35m) 4m ago 35m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:57:28.643 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (23m) 4m ago 36m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:57:28.643 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (23m) 4m ago 35m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:57:28.643 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (25m) 4m ago 35m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:57:28.643 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (19m) 4m ago 30m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:57:28.643 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (19m) 4m ago 30m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:57:28.643 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (4m) 4m ago 30m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T11:57:28.643 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (19m) 4m ago 30m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:57:28.643 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (27m) 4m ago 37m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:57:28.643 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (26m) 4m ago 35m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:57:28.643 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (24m) 4m ago 37m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:57:28.644 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (24m) 4m ago 35m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:57:28.644 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (26m) 4m ago 36m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:57:28.644 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (26m) 4m ago 35m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:57:28.644 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (22m) 4m ago 34m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:57:28.644 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (22m) 4m ago 33m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:57:28.644 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (21m) 4m ago 33m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:57:28.644 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (21m) 4m ago 33m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:57:28.644 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (20m) 4m ago 32m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:57:28.644 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (20m) 4m ago 32m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:57:28.644 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (25m) 4m ago 35m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:57:28.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.643+0000 7f1de3f50700 1 -- 172.21.15.144:0/737375883 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1dc806e200 msgr2=0x7f1dc80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:28.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.643+0000 7f1de3f50700 1 --2- 172.21.15.144:0/737375883 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1dc806e200 0x7f1dc80706b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1ddc197350 tx=0x7f1dd801a040 comp rx=0 tx=0).stop 2024-09-10T11:57:28.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.643+0000 7f1de3f50700 1 -- 172.21.15.144:0/737375883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ddc0ff2c0 msgr2=0x7f1ddc195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:28.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.643+0000 7f1de3f50700 1 --2- 172.21.15.144:0/737375883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ddc0ff2c0 0x7f1ddc195d30 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f1dcc00c8a0 tx=0x7f1dcc00cc60 comp rx=0 tx=0).stop 2024-09-10T11:57:28.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.644+0000 7f1de3f50700 1 -- 172.21.15.144:0/737375883 shutdown_connections 2024-09-10T11:57:28.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.644+0000 7f1de3f50700 1 --2- 172.21.15.144:0/737375883 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1dc806e200 0x7f1dc80706b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.644+0000 7f1de3f50700 1 --2- 172.21.15.144:0/737375883 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ddc0ff2c0 0x7f1ddc195d30 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.644+0000 7f1de3f50700 1 --2- 172.21.15.144:0/737375883 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ddc0ffc60 0x7f1ddc196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.644+0000 7f1de3f50700 1 -- 172.21.15.144:0/737375883 >> 172.21.15.144:0/737375883 conn(0x7f1ddc0fab80 msgr2=0x7f1ddc068f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:57:28.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.644+0000 7f1de3f50700 1 -- 172.21.15.144:0/737375883 shutdown_connections 2024-09-10T11:57:28.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.644+0000 7f1de3f50700 1 -- 172.21.15.144:0/737375883 wait complete. 2024-09-10T11:57:28.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.748+0000 7f69cf44b700 1 -- 172.21.15.144:0/1083965401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c8100380 msgr2=0x7f69c8100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:28.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.748+0000 7f69cf44b700 1 --2- 172.21.15.144:0/1083965401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c8100380 0x7f69c8100750 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f69b800b0d0 tx=0x7f69b800b3e0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.748+0000 7f69cf44b700 1 -- 172.21.15.144:0/1083965401 shutdown_connections 2024-09-10T11:57:28.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.748+0000 7f69cf44b700 1 --2- 172.21.15.144:0/1083965401 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69c8100c90 0x7f69c81093f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.748+0000 7f69cf44b700 1 --2- 172.21.15.144:0/1083965401 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c8100380 0x7f69c8100750 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.748+0000 7f69cf44b700 1 -- 172.21.15.144:0/1083965401 >> 172.21.15.144:0/1083965401 conn(0x7f69c8074850 msgr2=0x7f69c8074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:57:28.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.748+0000 7f69cf44b700 1 -- 172.21.15.144:0/1083965401 shutdown_connections 2024-09-10T11:57:28.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.748+0000 7f69cf44b700 1 -- 172.21.15.144:0/1083965401 wait complete. 2024-09-10T11:57:28.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.749+0000 7f69cf44b700 1 Processor -- start 2024-09-10T11:57:28.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.749+0000 7f69cf44b700 1 -- start start 2024-09-10T11:57:28.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.749+0000 7f69cf44b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69c8100380 0x7f69c8195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:28.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.749+0000 7f69cf44b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c8100c90 0x7f69c8196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:28.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.749+0000 7f69cf44b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c8196a10 con 0x7f69c8100c90 2024-09-10T11:57:28.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.749+0000 7f69cf44b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69c819a7a0 con 0x7f69c8100380 2024-09-10T11:57:28.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.749+0000 7f69cc9e6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c8100c90 0x7f69c8196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:28.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.749+0000 7f69cc9e6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c8100c90 0x7f69c8196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41908/0 (socket says 172.21.15.144:41908) 2024-09-10T11:57:28.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.749+0000 7f69cc9e6700 1 -- 172.21.15.144:0/1105567911 learned_addr learned my addr 172.21.15.144:0/1105567911 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:57:28.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.749+0000 7f69cd1e7700 1 --2- 172.21.15.144:0/1105567911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69c8100380 0x7f69c8195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:28.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.749+0000 7f69cc9e6700 1 -- 172.21.15.144:0/1105567911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69c8100380 msgr2=0x7f69c8195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:28.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.749+0000 7f69cc9e6700 1 --2- 172.21.15.144:0/1105567911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69c8100380 0x7f69c8195df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:28.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.749+0000 7f69cc9e6700 1 -- 172.21.15.144:0/1105567911 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b8009d00 con 0x7f69c8100c90 2024-09-10T11:57:28.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.749+0000 7f69cd1e7700 1 --2- 172.21.15.144:0/1105567911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69c8100380 0x7f69c8195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:57:28.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.750+0000 7f69cc9e6700 1 --2- 172.21.15.144:0/1105567911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c8100c90 0x7f69c8196330 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00b4b0 tx=0x7f69bc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:57:28.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.750+0000 7f69c67fc700 1 -- 172.21.15.144:0/1105567911 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69bc011800 con 0x7f69c8100c90 2024-09-10T11:57:28.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.750+0000 7f69c67fc700 1 -- 172.21.15.144:0/1105567911 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f69bc011e20 con 0x7f69c8100c90 2024-09-10T11:57:28.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.750+0000 7f69cf44b700 1 -- 172.21.15.144:0/1105567911 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69c819aa80 con 0x7f69c8100c90 2024-09-10T11:57:28.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.750+0000 7f69c67fc700 1 -- 172.21.15.144:0/1105567911 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69bc00f2e0 con 0x7f69c8100c90 2024-09-10T11:57:28.753 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.750+0000 7f69cf44b700 1 -- 172.21.15.144:0/1105567911 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69c819b050 con 0x7f69c8100c90 2024-09-10T11:57:28.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.751+0000 7f69cf44b700 1 -- 172.21.15.144:0/1105567911 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69c8107d60 con 0x7f69c8100c90 2024-09-10T11:57:28.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.751+0000 7f69c67fc700 1 -- 172.21.15.144:0/1105567911 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f69bc010380 con 0x7f69c8100c90 2024-09-10T11:57:28.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.752+0000 7f69c67fc700 1 --2- 172.21.15.144:0/1105567911 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f69b006e200 0x7f69b00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:28.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.752+0000 7f69c67fc700 1 -- 172.21.15.144:0/1105567911 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f69bc08dba0 con 0x7f69c8100c90 2024-09-10T11:57:28.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.752+0000 7f69cd1e7700 1 --2- 172.21.15.144:0/1105567911 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f69b006e200 0x7f69b00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:28.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.752+0000 7f69cd1e7700 1 --2- 172.21.15.144:0/1105567911 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f69b006e200 0x7f69b00706b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f69b8015040 tx=0x7f69b8006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:57:28.757 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:28.756+0000 7f69c67fc700 1 -- 172.21.15.144:0/1105567911 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f69bc0570b0 con 0x7f69c8100c90 2024-09-10T11:57:28.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:57:29.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.044+0000 7f69cf44b700 1 -- 172.21.15.144:0/1105567911 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f69c804ea50 con 0x7f69c8100c90 2024-09-10T11:57:29.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.044+0000 7f69c67fc700 1 -- 172.21.15.144:0/1105567911 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f69bc05a770 con 0x7f69c8100c90 2024-09-10T11:57:29.045 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:57:29.045 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:57:29.045 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:57:29.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:57:29.046 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:57:29.046 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:57:29.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:57:29.046 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:57:29.046 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:57:29.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:57:29.046 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:57:29.046 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:57:29.046 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:57:29.046 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:57:29.046 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T11:57:29.046 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:57:29.046 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:57:29.047 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.046+0000 7f69cf44b700 1 -- 172.21.15.144:0/1105567911 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f69b006e200 msgr2=0x7f69b00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:29.047 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.046+0000 7f69cf44b700 1 --2- 172.21.15.144:0/1105567911 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f69b006e200 0x7f69b00706b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f69b8015040 tx=0x7f69b8006000 comp rx=0 tx=0).stop 2024-09-10T11:57:29.047 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.046+0000 7f69cf44b700 1 -- 172.21.15.144:0/1105567911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c8100c90 msgr2=0x7f69c8196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:29.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.046+0000 7f69cf44b700 1 --2- 172.21.15.144:0/1105567911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c8100c90 0x7f69c8196330 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f69bc00b4b0 tx=0x7f69bc00b870 comp rx=0 tx=0).stop 2024-09-10T11:57:29.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.046+0000 7f69cf44b700 1 -- 172.21.15.144:0/1105567911 shutdown_connections 2024-09-10T11:57:29.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.046+0000 7f69cf44b700 1 --2- 172.21.15.144:0/1105567911 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f69b006e200 0x7f69b00706b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.046+0000 7f69cf44b700 1 --2- 172.21.15.144:0/1105567911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69c8100380 0x7f69c8195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.046+0000 7f69cf44b700 1 --2- 172.21.15.144:0/1105567911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69c8100c90 0x7f69c8196330 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.046+0000 7f69cf44b700 1 -- 172.21.15.144:0/1105567911 >> 172.21.15.144:0/1105567911 conn(0x7f69c8074850 msgr2=0x7f69c80fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:57:29.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.046+0000 7f69cf44b700 1 -- 172.21.15.144:0/1105567911 shutdown_connections 2024-09-10T11:57:29.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.046+0000 7f69cf44b700 1 -- 172.21.15.144:0/1105567911 wait complete. 2024-09-10T11:57:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:57:29.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.150+0000 7f8f5508e700 1 -- 172.21.15.144:0/1879288818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f500ffc50 msgr2=0x7f8f50103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:29.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.150+0000 7f8f5508e700 1 --2- 172.21.15.144:0/1879288818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f500ffc50 0x7f8f50103c20 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f8f44009910 tx=0x7f8f44009c20 comp rx=0 tx=0).stop 2024-09-10T11:57:29.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.151+0000 7f8f5508e700 1 -- 172.21.15.144:0/1879288818 shutdown_connections 2024-09-10T11:57:29.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.151+0000 7f8f5508e700 1 --2- 172.21.15.144:0/1879288818 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f500ffc50 0x7f8f50103c20 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.151+0000 7f8f5508e700 1 --2- 172.21.15.144:0/1879288818 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f500ff2b0 0x7f8f500ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.151+0000 7f8f5508e700 1 -- 172.21.15.144:0/1879288818 >> 172.21.15.144:0/1879288818 conn(0x7f8f500fab50 msgr2=0x7f8f500fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:57:29.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.151+0000 7f8f5508e700 1 -- 172.21.15.144:0/1879288818 shutdown_connections 2024-09-10T11:57:29.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.151+0000 7f8f5508e700 1 -- 172.21.15.144:0/1879288818 wait complete. 2024-09-10T11:57:29.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.151+0000 7f8f5508e700 1 Processor -- start 2024-09-10T11:57:29.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.152+0000 7f8f5508e700 1 -- start start 2024-09-10T11:57:29.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.152+0000 7f8f5508e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f500ff2b0 0x7f8f50195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:29.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.152+0000 7f8f5508e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f500ffc50 0x7f8f50196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:29.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.152+0000 7f8f5508e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f50196a50 con 0x7f8f500ffc50 2024-09-10T11:57:29.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.152+0000 7f8f5508e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5019a790 con 0x7f8f500ff2b0 2024-09-10T11:57:29.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.152+0000 7f8f4e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f500ffc50 0x7f8f50196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:29.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.152+0000 7f8f4e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f500ffc50 0x7f8f50196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41934/0 (socket says 172.21.15.144:41934) 2024-09-10T11:57:29.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.152+0000 7f8f4e59c700 1 -- 172.21.15.144:0/523854267 learned_addr learned my addr 172.21.15.144:0/523854267 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:57:29.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.152+0000 7f8f4ed9d700 1 --2- 172.21.15.144:0/523854267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f500ff2b0 0x7f8f50195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:29.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.152+0000 7f8f4e59c700 1 -- 172.21.15.144:0/523854267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f500ff2b0 msgr2=0x7f8f50195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:29.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.152+0000 7f8f4e59c700 1 --2- 172.21.15.144:0/523854267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f500ff2b0 0x7f8f50195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.152+0000 7f8f4e59c700 1 -- 172.21.15.144:0/523854267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f440095f0 con 0x7f8f500ffc50 2024-09-10T11:57:29.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.152+0000 7f8f4e59c700 1 --2- 172.21.15.144:0/523854267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f500ffc50 0x7f8f50196370 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8f440092e0 tx=0x7f8f4400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:57:29.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.153+0000 7f8f37fff700 1 -- 172.21.15.144:0/523854267 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f4401d070 con 0x7f8f500ffc50 2024-09-10T11:57:29.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.153+0000 7f8f5508e700 1 -- 172.21.15.144:0/523854267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5019aa10 con 0x7f8f500ffc50 2024-09-10T11:57:29.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.153+0000 7f8f37fff700 1 -- 172.21.15.144:0/523854267 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8f4400f460 con 0x7f8f500ffc50 2024-09-10T11:57:29.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.153+0000 7f8f37fff700 1 -- 172.21.15.144:0/523854267 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f440175f0 con 0x7f8f500ffc50 2024-09-10T11:57:29.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.153+0000 7f8f5508e700 1 -- 172.21.15.144:0/523854267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5019af00 con 0x7f8f500ffc50 2024-09-10T11:57:29.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.154+0000 7f8f5508e700 1 -- 172.21.15.144:0/523854267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f5004ea50 con 0x7f8f500ffc50 2024-09-10T11:57:29.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.154+0000 7f8f37fff700 1 -- 172.21.15.144:0/523854267 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8f4400f5d0 con 0x7f8f500ffc50 2024-09-10T11:57:29.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.155+0000 7f8f37fff700 1 --2- 172.21.15.144:0/523854267 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8f3806e200 0x7f8f380706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:29.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.155+0000 7f8f4ed9d700 1 --2- 172.21.15.144:0/523854267 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8f3806e200 0x7f8f380706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:29.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.155+0000 7f8f37fff700 1 -- 172.21.15.144:0/523854267 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8f4408f620 con 0x7f8f500ffc50 2024-09-10T11:57:29.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.155+0000 7f8f4ed9d700 1 --2- 172.21.15.144:0/523854267 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8f3806e200 0x7f8f380706b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8f40006fd0 tx=0x7f8f40008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:57:29.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.159+0000 7f8f37fff700 1 -- 172.21.15.144:0/523854267 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8f44058a00 con 0x7f8f500ffc50 2024-09-10T11:57:29.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.403+0000 7f8f5508e700 1 -- 172.21.15.144:0/523854267 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8f5019b2a0 con 0x7f8f500ffc50 2024-09-10T11:57:29.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.403+0000 7f8f37fff700 1 -- 172.21.15.144:0/523854267 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f8f4405c0c0 con 0x7f8f500ffc50 2024-09-10T11:57:29.405 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T11:57:29.405 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T11:57:29.405 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:57:29.405 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:57:29.405 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:57:29.405 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:57:29.405 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:57:29.405 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:57:29.405 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:57:29.405 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:57:29.405 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:57:29.406 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:57:29.406 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:57:29.406 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:57:29.406 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:57:29.406 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:57:29.406 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:57:29.406 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:57:29.406 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:57:29.406 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:57:29.406 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:57:29.406 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:57:29.406 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:57:29.406 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:57:29.406 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:57:29.407 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:57:29.407 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:57:29.407 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:57:29.407 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:57:29.407 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:57:29.407 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:57:29.407 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:57:29.407 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:57:29.407 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:57:29.407 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:57:29.407 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:57:29.407 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:57:29.407 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:57:29.407 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:57:29.408 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:57:29.408 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:57:29.408 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:57:29.408 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:57:29.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.405+0000 7f8f5508e700 1 -- 172.21.15.144:0/523854267 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8f3806e200 msgr2=0x7f8f380706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:29.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.405+0000 7f8f5508e700 1 --2- 172.21.15.144:0/523854267 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8f3806e200 0x7f8f380706b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f8f40006fd0 tx=0x7f8f40008040 comp rx=0 tx=0).stop 2024-09-10T11:57:29.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.405+0000 7f8f5508e700 1 -- 172.21.15.144:0/523854267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f500ffc50 msgr2=0x7f8f50196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:29.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.405+0000 7f8f5508e700 1 --2- 172.21.15.144:0/523854267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f500ffc50 0x7f8f50196370 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f8f440092e0 tx=0x7f8f4400be20 comp rx=0 tx=0).stop 2024-09-10T11:57:29.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.405+0000 7f8f5508e700 1 -- 172.21.15.144:0/523854267 shutdown_connections 2024-09-10T11:57:29.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.405+0000 7f8f5508e700 1 --2- 172.21.15.144:0/523854267 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8f3806e200 0x7f8f380706b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.405+0000 7f8f5508e700 1 --2- 172.21.15.144:0/523854267 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8f500ff2b0 0x7f8f50195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.405+0000 7f8f5508e700 1 --2- 172.21.15.144:0/523854267 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8f500ffc50 0x7f8f50196370 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.405+0000 7f8f5508e700 1 -- 172.21.15.144:0/523854267 >> 172.21.15.144:0/523854267 conn(0x7f8f500fab50 msgr2=0x7f8f50103400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:57:29.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.406+0000 7f8f5508e700 1 -- 172.21.15.144:0/523854267 shutdown_connections 2024-09-10T11:57:29.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.406+0000 7f8f5508e700 1 -- 172.21.15.144:0/523854267 wait complete. 2024-09-10T11:57:29.410 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T11:57:29.514 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.511+0000 7fb29ae52700 1 -- 172.21.15.144:0/1823067769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2940ff2d0 msgr2=0x7fb2940ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:29.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.511+0000 7fb29ae52700 1 --2- 172.21.15.144:0/1823067769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2940ff2d0 0x7fb2940ff6a0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb288009910 tx=0x7fb288009c20 comp rx=0 tx=0).stop 2024-09-10T11:57:29.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.512+0000 7fb29ae52700 1 -- 172.21.15.144:0/1823067769 shutdown_connections 2024-09-10T11:57:29.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.512+0000 7fb29ae52700 1 --2- 172.21.15.144:0/1823067769 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2940ffc70 0x7fb294103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.512+0000 7fb29ae52700 1 --2- 172.21.15.144:0/1823067769 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2940ff2d0 0x7fb2940ff6a0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.512+0000 7fb29ae52700 1 -- 172.21.15.144:0/1823067769 >> 172.21.15.144:0/1823067769 conn(0x7fb2940fab70 msgr2=0x7fb2940fcf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:57:29.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.512+0000 7fb29ae52700 1 -- 172.21.15.144:0/1823067769 shutdown_connections 2024-09-10T11:57:29.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.512+0000 7fb29ae52700 1 -- 172.21.15.144:0/1823067769 wait complete. 2024-09-10T11:57:29.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.512+0000 7fb29ae52700 1 Processor -- start 2024-09-10T11:57:29.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.512+0000 7fb29ae52700 1 -- start start 2024-09-10T11:57:29.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.512+0000 7fb29ae52700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2940ff2d0 0x7fb294195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:29.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.512+0000 7fb29ae52700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2940ffc70 0x7fb294196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:29.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.512+0000 7fb29ae52700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb294196970 con 0x7fb2940ff2d0 2024-09-10T11:57:29.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.512+0000 7fb29ae52700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29419a700 con 0x7fb2940ffc70 2024-09-10T11:57:29.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.513+0000 7fb298bee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2940ff2d0 0x7fb294195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:29.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.513+0000 7fb298bee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2940ff2d0 0x7fb294195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41944/0 (socket says 172.21.15.144:41944) 2024-09-10T11:57:29.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.513+0000 7fb298bee700 1 -- 172.21.15.144:0/3579784089 learned_addr learned my addr 172.21.15.144:0/3579784089 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:57:29.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.513+0000 7fb293fff700 1 --2- 172.21.15.144:0/3579784089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2940ffc70 0x7fb294196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:29.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.513+0000 7fb298bee700 1 -- 172.21.15.144:0/3579784089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2940ffc70 msgr2=0x7fb294196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:29.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.513+0000 7fb298bee700 1 --2- 172.21.15.144:0/3579784089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2940ffc70 0x7fb294196290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.513+0000 7fb298bee700 1 -- 172.21.15.144:0/3579784089 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2880095f0 con 0x7fb2940ff2d0 2024-09-10T11:57:29.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.513+0000 7fb298bee700 1 --2- 172.21.15.144:0/3579784089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2940ff2d0 0x7fb294195d50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb288006010 tx=0x7fb28800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:57:29.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.513+0000 7fb291ffb700 1 -- 172.21.15.144:0/3579784089 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb28801d070 con 0x7fb2940ff2d0 2024-09-10T11:57:29.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.513+0000 7fb291ffb700 1 -- 172.21.15.144:0/3579784089 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb28800f460 con 0x7fb2940ff2d0 2024-09-10T11:57:29.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.513+0000 7fb29ae52700 1 -- 172.21.15.144:0/3579784089 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29419a980 con 0x7fb2940ff2d0 2024-09-10T11:57:29.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.513+0000 7fb291ffb700 1 -- 172.21.15.144:0/3579784089 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb288017630 con 0x7fb2940ff2d0 2024-09-10T11:57:29.516 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.513+0000 7fb29ae52700 1 -- 172.21.15.144:0/3579784089 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29419ae70 con 0x7fb2940ff2d0 2024-09-10T11:57:29.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.514+0000 7fb29ae52700 1 -- 172.21.15.144:0/3579784089 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29404ea50 con 0x7fb2940ff2d0 2024-09-10T11:57:29.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.515+0000 7fb291ffb700 1 -- 172.21.15.144:0/3579784089 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb28800f5d0 con 0x7fb2940ff2d0 2024-09-10T11:57:29.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.515+0000 7fb291ffb700 1 --2- 172.21.15.144:0/3579784089 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb28406e200 0x7fb2840706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:29.518 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.515+0000 7fb291ffb700 1 -- 172.21.15.144:0/3579784089 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb28808f4d0 con 0x7fb2940ff2d0 2024-09-10T11:57:29.518 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.515+0000 7fb293fff700 1 --2- 172.21.15.144:0/3579784089 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb28406e200 0x7fb2840706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:29.518 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.516+0000 7fb293fff700 1 --2- 172.21.15.144:0/3579784089 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb28406e200 0x7fb2840706b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb280005950 tx=0x7fb2800058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:57:29.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.520+0000 7fb291ffb700 1 -- 172.21.15.144:0/3579784089 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb288058930 con 0x7fb2940ff2d0 2024-09-10T11:57:29.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.742+0000 7fb29ae52700 1 -- 172.21.15.144:0/3579784089 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb29419b350 con 0x7fb28406e200 2024-09-10T11:57:29.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.744+0000 7fb291ffb700 1 -- 172.21.15.144:0/3579784089 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb29419b350 con 0x7fb28406e200 2024-09-10T11:57:29.745 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:57:29.745 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:57:29.745 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:57:29.745 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:57:29.745 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:57:29.745 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:57:29.745 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T11:57:29.745 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T11:57:29.746 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:57:29.746 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T11:57:29.746 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:57:29.746 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T11:57:29.746 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:57:29.746 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:57:29.746 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:57:29.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.746+0000 7fb29ae52700 1 -- 172.21.15.144:0/3579784089 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb28406e200 msgr2=0x7fb2840706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:29.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.746+0000 7fb29ae52700 1 --2- 172.21.15.144:0/3579784089 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb28406e200 0x7fb2840706b0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fb280005950 tx=0x7fb2800058e0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.746+0000 7fb29ae52700 1 -- 172.21.15.144:0/3579784089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2940ff2d0 msgr2=0x7fb294195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:29.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.746+0000 7fb29ae52700 1 --2- 172.21.15.144:0/3579784089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2940ff2d0 0x7fb294195d50 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb288006010 tx=0x7fb28800be20 comp rx=0 tx=0).stop 2024-09-10T11:57:29.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.746+0000 7fb29ae52700 1 -- 172.21.15.144:0/3579784089 shutdown_connections 2024-09-10T11:57:29.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.746+0000 7fb29ae52700 1 --2- 172.21.15.144:0/3579784089 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb28406e200 0x7fb2840706b0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.746+0000 7fb29ae52700 1 --2- 172.21.15.144:0/3579784089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2940ff2d0 0x7fb294195d50 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.746+0000 7fb29ae52700 1 --2- 172.21.15.144:0/3579784089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2940ffc70 0x7fb294196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.746+0000 7fb29ae52700 1 -- 172.21.15.144:0/3579784089 >> 172.21.15.144:0/3579784089 conn(0x7fb2940fab70 msgr2=0x7fb294103360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:57:29.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.746+0000 7fb29ae52700 1 -- 172.21.15.144:0/3579784089 shutdown_connections 2024-09-10T11:57:29.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.746+0000 7fb29ae52700 1 -- 172.21.15.144:0/3579784089 wait complete. 2024-09-10T11:57:29.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.850+0000 7f5adf8b0700 1 -- 172.21.15.144:0/1387257395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad81003d0 msgr2=0x7f5ad81007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:29.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.850+0000 7f5adf8b0700 1 --2- 172.21.15.144:0/1387257395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad81003d0 0x7f5ad81007a0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5ac8009910 tx=0x7f5ac8009c20 comp rx=0 tx=0).stop 2024-09-10T11:57:29.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.850+0000 7f5adf8b0700 1 -- 172.21.15.144:0/1387257395 shutdown_connections 2024-09-10T11:57:29.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.850+0000 7f5adf8b0700 1 --2- 172.21.15.144:0/1387257395 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad8100d70 0x7f5ad8104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.850+0000 7f5adf8b0700 1 --2- 172.21.15.144:0/1387257395 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad81003d0 0x7f5ad81007a0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.850+0000 7f5adf8b0700 1 -- 172.21.15.144:0/1387257395 >> 172.21.15.144:0/1387257395 conn(0x7f5ad80fbc90 msgr2=0x7f5ad80fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:57:29.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.850+0000 7f5adf8b0700 1 -- 172.21.15.144:0/1387257395 shutdown_connections 2024-09-10T11:57:29.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.850+0000 7f5adf8b0700 1 -- 172.21.15.144:0/1387257395 wait complete. 2024-09-10T11:57:29.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.851+0000 7f5adf8b0700 1 Processor -- start 2024-09-10T11:57:29.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.851+0000 7f5adf8b0700 1 -- start start 2024-09-10T11:57:29.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.851+0000 7f5adf8b0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad81003d0 0x7f5ad8195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:29.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.851+0000 7f5adf8b0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad8100d70 0x7f5ad81962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:29.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.851+0000 7f5adf8b0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad81968f0 con 0x7f5ad8100d70 2024-09-10T11:57:29.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.851+0000 7f5adf8b0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ad8196a30 con 0x7f5ad81003d0 2024-09-10T11:57:29.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.851+0000 7f5adce4b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad8100d70 0x7f5ad81962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:29.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.851+0000 7f5adce4b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad8100d70 0x7f5ad81962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41956/0 (socket says 172.21.15.144:41956) 2024-09-10T11:57:29.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.851+0000 7f5adce4b700 1 -- 172.21.15.144:0/3494342927 learned_addr learned my addr 172.21.15.144:0/3494342927 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:57:29.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.852+0000 7f5adce4b700 1 -- 172.21.15.144:0/3494342927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad81003d0 msgr2=0x7f5ad8195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:29.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.852+0000 7f5add64c700 1 --2- 172.21.15.144:0/3494342927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad81003d0 0x7f5ad8195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:29.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.852+0000 7f5adce4b700 1 --2- 172.21.15.144:0/3494342927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad81003d0 0x7f5ad8195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:29.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.852+0000 7f5adce4b700 1 -- 172.21.15.144:0/3494342927 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac80095f0 con 0x7f5ad8100d70 2024-09-10T11:57:29.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.852+0000 7f5adce4b700 1 --2- 172.21.15.144:0/3494342927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad8100d70 0x7f5ad81962a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c960 tx=0x7f5ad400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:57:29.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.852+0000 7f5ace7fc700 1 -- 172.21.15.144:0/3494342927 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ad4007a70 con 0x7f5ad8100d70 2024-09-10T11:57:29.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.852+0000 7f5ace7fc700 1 -- 172.21.15.144:0/3494342927 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5ad4007bd0 con 0x7f5ad8100d70 2024-09-10T11:57:29.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.852+0000 7f5ace7fc700 1 -- 172.21.15.144:0/3494342927 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ad40186c0 con 0x7f5ad8100d70 2024-09-10T11:57:29.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.852+0000 7f5adf8b0700 1 -- 172.21.15.144:0/3494342927 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ad819a880 con 0x7f5ad8100d70 2024-09-10T11:57:29.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.852+0000 7f5adf8b0700 1 -- 172.21.15.144:0/3494342927 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ad819add0 con 0x7f5ad8100d70 2024-09-10T11:57:29.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.853+0000 7f5adf8b0700 1 -- 172.21.15.144:0/3494342927 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ad81086e0 con 0x7f5ad8100d70 2024-09-10T11:57:29.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.854+0000 7f5ace7fc700 1 -- 172.21.15.144:0/3494342927 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5ad400faa0 con 0x7f5ad8100d70 2024-09-10T11:57:29.856 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.854+0000 7f5ace7fc700 1 --2- 172.21.15.144:0/3494342927 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5ac406e200 0x7f5ac40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:57:29.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.854+0000 7f5ace7fc700 1 -- 172.21.15.144:0/3494342927 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5ad408e0d0 con 0x7f5ad8100d70 2024-09-10T11:57:29.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.854+0000 7f5add64c700 1 --2- 172.21.15.144:0/3494342927 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5ac406e200 0x7f5ac40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:57:29.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.855+0000 7f5add64c700 1 --2- 172.21.15.144:0/3494342927 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5ac406e200 0x7f5ac40706b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800b5c0 tx=0x7f5ac8005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:57:29.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:29.858+0000 7f5ace7fc700 1 -- 172.21.15.144:0/3494342927 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5ad405afe0 con 0x7f5ad8100d70 2024-09-10T11:57:29.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:29 smithi144 ceph-mon[87436]: from='client.35408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:57:29.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:29 smithi144 ceph-mon[87436]: from='client.35412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:57:29.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:29 smithi144 ceph-mon[87436]: from='client.35416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:57:29.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:29 smithi144 ceph-mon[87436]: pgmap v766: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:29.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:29 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1105567911' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:57:29.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:29 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/523854267' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:57:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:29 smithi178 ceph-mon[65940]: from='client.35408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:57:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:29 smithi178 ceph-mon[65940]: from='client.35412 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:57:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:29 smithi178 ceph-mon[65940]: from='client.35416 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:57:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:29 smithi178 ceph-mon[65940]: pgmap v766: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:29 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1105567911' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:57:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:29 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/523854267' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:57:30.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:30.149+0000 7f5adf8b0700 1 -- 172.21.15.144:0/3494342927 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5ad804ea50 con 0x7f5ad8100d70 2024-09-10T11:57:30.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:30.149+0000 7f5ace7fc700 1 -- 172.21.15.144:0/3494342927 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f5ad405ab70 con 0x7f5ad8100d70 2024-09-10T11:57:30.151 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:57:30.151 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T11:57:30.151 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:57:30.151 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T11:57:30.151 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:57:30.151 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T11:57:30.151 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T11:57:30.151 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T11:57:30.151 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T11:57:30.151 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T11:57:30.151 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T11:57:30.152 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T11:57:30.152 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T11:57:30.152 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T11:57:30.152 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T11:57:30.152 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T11:57:30.152 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T11:57:30.152 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T11:57:30.152 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:57:30.152 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:57:30.152 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:57:30.152 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:57:30.152 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:57:30.152 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:57:30.153 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:57:30.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:30.151+0000 7f5adf8b0700 1 -- 172.21.15.144:0/3494342927 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5ac406e200 msgr2=0x7f5ac40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:30.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:30.151+0000 7f5adf8b0700 1 --2- 172.21.15.144:0/3494342927 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5ac406e200 0x7f5ac40706b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f5ac800b5c0 tx=0x7f5ac8005960 comp rx=0 tx=0).stop 2024-09-10T11:57:30.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:30.151+0000 7f5adf8b0700 1 -- 172.21.15.144:0/3494342927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad8100d70 msgr2=0x7f5ad81962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:57:30.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:30.151+0000 7f5adf8b0700 1 --2- 172.21.15.144:0/3494342927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad8100d70 0x7f5ad81962a0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5ad400c960 tx=0x7f5ad400cd20 comp rx=0 tx=0).stop 2024-09-10T11:57:30.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:30.151+0000 7f5adf8b0700 1 -- 172.21.15.144:0/3494342927 shutdown_connections 2024-09-10T11:57:30.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:30.151+0000 7f5adf8b0700 1 --2- 172.21.15.144:0/3494342927 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5ac406e200 0x7f5ac40706b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:30.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:30.151+0000 7f5adf8b0700 1 --2- 172.21.15.144:0/3494342927 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5ad81003d0 0x7f5ad8195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:30.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:30.151+0000 7f5adf8b0700 1 --2- 172.21.15.144:0/3494342927 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5ad8100d70 0x7f5ad81962a0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:57:30.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:30.151+0000 7f5adf8b0700 1 -- 172.21.15.144:0/3494342927 >> 172.21.15.144:0/3494342927 conn(0x7f5ad80fbc90 msgr2=0x7f5ad80fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:57:30.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:30.151+0000 7f5adf8b0700 1 -- 172.21.15.144:0/3494342927 shutdown_connections 2024-09-10T11:57:30.154 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:57:30.151+0000 7f5adf8b0700 1 -- 172.21.15.144:0/3494342927 wait complete. 2024-09-10T11:57:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:30 smithi178 ceph-mon[65940]: from='client.35428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:57:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:30 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3494342927' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:57:31.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:30 smithi144 ceph-mon[87436]: from='client.35428 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:57:31.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:30 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3494342927' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:57:32.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:31 smithi178 ceph-mon[65940]: pgmap v767: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:32.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:31 smithi144 ceph-mon[87436]: pgmap v767: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:32 smithi178 ceph-mon[65940]: pgmap v768: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:33.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:32 smithi144 ceph-mon[87436]: pgmap v768: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:34 smithi178 ceph-mon[65940]: pgmap v769: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:35.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:34 smithi144 ceph-mon[87436]: pgmap v769: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:36 smithi178 ceph-mon[65940]: pgmap v770: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:37.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:36 smithi144 ceph-mon[87436]: pgmap v770: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:39.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:38 smithi178 ceph-mon[65940]: pgmap v771: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:39.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:38 smithi144 ceph-mon[87436]: pgmap v771: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:40 smithi178 ceph-mon[65940]: pgmap v772: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:41.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:40 smithi144 ceph-mon[87436]: pgmap v772: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:42 smithi178 ceph-mon[65940]: pgmap v773: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:42 smithi144 ceph-mon[87436]: pgmap v773: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:44.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:57:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:57:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:44 smithi178 ceph-mon[65940]: pgmap v774: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:44 smithi144 ceph-mon[87436]: pgmap v774: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:46 smithi178 ceph-mon[65940]: pgmap v775: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:46 smithi144 ceph-mon[87436]: pgmap v775: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:48 smithi178 ceph-mon[65940]: pgmap v776: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:49.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:48 smithi144 ceph-mon[87436]: pgmap v776: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:50 smithi178 ceph-mon[65940]: pgmap v777: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:51.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:50 smithi144 ceph-mon[87436]: pgmap v777: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:52 smithi178 ceph-mon[65940]: pgmap v778: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:53.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:52 smithi144 ceph-mon[87436]: pgmap v778: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:54 smithi178 ceph-mon[65940]: pgmap v779: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:54 smithi144 ceph-mon[87436]: pgmap v779: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:56 smithi178 ceph-mon[65940]: pgmap v780: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:57.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:56 smithi144 ceph-mon[87436]: pgmap v780: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:57:58.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:57:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:57:59.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:57:59 smithi144 ceph-mon[87436]: pgmap v781: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:00.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:57:59 smithi178 ceph-mon[65940]: pgmap v781: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:00.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.269+0000 7fab7717e700 1 -- 172.21.15.144:0/1180400916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab70100460 msgr2=0x7fab70100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:00.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.269+0000 7fab7717e700 1 --2- 172.21.15.144:0/1180400916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab70100460 0x7fab70100830 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fab68009910 tx=0x7fab68009c20 comp rx=0 tx=0).stop 2024-09-10T11:58:00.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.269+0000 7fab7717e700 1 -- 172.21.15.144:0/1180400916 shutdown_connections 2024-09-10T11:58:00.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.269+0000 7fab7717e700 1 --2- 172.21.15.144:0/1180400916 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab70100e00 0x7fab70104dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.269+0000 7fab7717e700 1 --2- 172.21.15.144:0/1180400916 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab70100460 0x7fab70100830 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.269+0000 7fab7717e700 1 -- 172.21.15.144:0/1180400916 >> 172.21.15.144:0/1180400916 conn(0x7fab700fbd00 msgr2=0x7fab700fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:00.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.269+0000 7fab7717e700 1 -- 172.21.15.144:0/1180400916 shutdown_connections 2024-09-10T11:58:00.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.269+0000 7fab7717e700 1 -- 172.21.15.144:0/1180400916 wait complete. 2024-09-10T11:58:00.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.270+0000 7fab7717e700 1 Processor -- start 2024-09-10T11:58:00.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.270+0000 7fab7717e700 1 -- start start 2024-09-10T11:58:00.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.270+0000 7fab7717e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab70100460 0x7fab70195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:00.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.270+0000 7fab7717e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab70100e00 0x7fab701962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:00.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.270+0000 7fab7717e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab701969c0 con 0x7fab70100e00 2024-09-10T11:58:00.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.270+0000 7fab7717e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7019a750 con 0x7fab70100460 2024-09-10T11:58:00.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.270+0000 7fab6ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab70100e00 0x7fab701962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:00.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.270+0000 7fab6ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab70100e00 0x7fab701962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44180/0 (socket says 172.21.15.144:44180) 2024-09-10T11:58:00.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.270+0000 7fab6ffff700 1 -- 172.21.15.144:0/2567691248 learned_addr learned my addr 172.21.15.144:0/2567691248 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:58:00.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.270+0000 7fab74f1a700 1 --2- 172.21.15.144:0/2567691248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab70100460 0x7fab70195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:00.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.271+0000 7fab6ffff700 1 -- 172.21.15.144:0/2567691248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab70100460 msgr2=0x7fab70195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:00.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.271+0000 7fab6ffff700 1 --2- 172.21.15.144:0/2567691248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab70100460 0x7fab70195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.271+0000 7fab6ffff700 1 -- 172.21.15.144:0/2567691248 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab680095f0 con 0x7fab70100e00 2024-09-10T11:58:00.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.271+0000 7fab74f1a700 1 --2- 172.21.15.144:0/2567691248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab70100460 0x7fab70195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:58:00.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.271+0000 7fab6ffff700 1 --2- 172.21.15.144:0/2567691248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab70100e00 0x7fab701962e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fab60007e90 tx=0x7fab6000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:00.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.271+0000 7fab6dffb700 1 -- 172.21.15.144:0/2567691248 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6000dda0 con 0x7fab70100e00 2024-09-10T11:58:00.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.271+0000 7fab6dffb700 1 -- 172.21.15.144:0/2567691248 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab6000f040 con 0x7fab70100e00 2024-09-10T11:58:00.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.271+0000 7fab7717e700 1 -- 172.21.15.144:0/2567691248 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7019aa30 con 0x7fab70100e00 2024-09-10T11:58:00.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.271+0000 7fab6dffb700 1 -- 172.21.15.144:0/2567691248 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab600125f0 con 0x7fab70100e00 2024-09-10T11:58:00.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.271+0000 7fab7717e700 1 -- 172.21.15.144:0/2567691248 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7019af80 con 0x7fab70100e00 2024-09-10T11:58:00.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.272+0000 7fab7717e700 1 -- 172.21.15.144:0/2567691248 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab70108770 con 0x7fab70100e00 2024-09-10T11:58:00.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.272+0000 7fab6dffb700 1 -- 172.21.15.144:0/2567691248 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fab600128b0 con 0x7fab70100e00 2024-09-10T11:58:00.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.273+0000 7fab6dffb700 1 --2- 172.21.15.144:0/2567691248 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fab5c06e200 0x7fab5c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:00.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.273+0000 7fab74f1a700 1 --2- 172.21.15.144:0/2567691248 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fab5c06e200 0x7fab5c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:00.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.273+0000 7fab6dffb700 1 -- 172.21.15.144:0/2567691248 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fab6008d9e0 con 0x7fab70100e00 2024-09-10T11:58:00.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.274+0000 7fab74f1a700 1 --2- 172.21.15.144:0/2567691248 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fab5c06e200 0x7fab5c0706b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fab68005b40 tx=0x7fab6801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:00.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.278+0000 7fab6dffb700 1 -- 172.21.15.144:0/2567691248 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fab60056dc0 con 0x7fab70100e00 2024-09-10T11:58:00.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.500+0000 7fab7717e700 1 -- 172.21.15.144:0/2567691248 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fab7019b210 con 0x7fab5c06e200 2024-09-10T11:58:00.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.501+0000 7fab6dffb700 1 -- 172.21.15.144:0/2567691248 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fab7019b210 con 0x7fab5c06e200 2024-09-10T11:58:00.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.503+0000 7fab7717e700 1 -- 172.21.15.144:0/2567691248 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fab5c06e200 msgr2=0x7fab5c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:00.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.503+0000 7fab7717e700 1 --2- 172.21.15.144:0/2567691248 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fab5c06e200 0x7fab5c0706b0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fab68005b40 tx=0x7fab6801a040 comp rx=0 tx=0).stop 2024-09-10T11:58:00.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.503+0000 7fab7717e700 1 -- 172.21.15.144:0/2567691248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab70100e00 msgr2=0x7fab701962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:00.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.503+0000 7fab7717e700 1 --2- 172.21.15.144:0/2567691248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab70100e00 0x7fab701962e0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fab60007e90 tx=0x7fab6000d3b0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.503+0000 7fab7717e700 1 -- 172.21.15.144:0/2567691248 shutdown_connections 2024-09-10T11:58:00.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.503+0000 7fab7717e700 1 --2- 172.21.15.144:0/2567691248 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fab5c06e200 0x7fab5c0706b0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.503+0000 7fab7717e700 1 --2- 172.21.15.144:0/2567691248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fab70100460 0x7fab70195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.503+0000 7fab7717e700 1 --2- 172.21.15.144:0/2567691248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fab70100e00 0x7fab701962e0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.503+0000 7fab7717e700 1 -- 172.21.15.144:0/2567691248 >> 172.21.15.144:0/2567691248 conn(0x7fab700fbd00 msgr2=0x7fab700fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:00.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.503+0000 7fab7717e700 1 -- 172.21.15.144:0/2567691248 shutdown_connections 2024-09-10T11:58:00.505 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.503+0000 7fab7717e700 1 -- 172.21.15.144:0/2567691248 wait complete. 2024-09-10T11:58:00.515 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:58:00.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.609+0000 7f684822b700 1 -- 172.21.15.144:0/3348157224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68400ffc60 msgr2=0x7f6840103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:00.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.609+0000 7f684822b700 1 --2- 172.21.15.144:0/3348157224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68400ffc60 0x7f6840103c30 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6830009910 tx=0x7f6830009c20 comp rx=0 tx=0).stop 2024-09-10T11:58:00.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.609+0000 7f684822b700 1 -- 172.21.15.144:0/3348157224 shutdown_connections 2024-09-10T11:58:00.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.609+0000 7f684822b700 1 --2- 172.21.15.144:0/3348157224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68400ffc60 0x7f6840103c30 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.609+0000 7f684822b700 1 --2- 172.21.15.144:0/3348157224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68400ff2c0 0x7f68400ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.609+0000 7f684822b700 1 -- 172.21.15.144:0/3348157224 >> 172.21.15.144:0/3348157224 conn(0x7f68400fab80 msgr2=0x7f68400fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:00.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.610+0000 7f684822b700 1 -- 172.21.15.144:0/3348157224 shutdown_connections 2024-09-10T11:58:00.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.610+0000 7f684822b700 1 -- 172.21.15.144:0/3348157224 wait complete. 2024-09-10T11:58:00.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.610+0000 7f684822b700 1 Processor -- start 2024-09-10T11:58:00.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.610+0000 7f684822b700 1 -- start start 2024-09-10T11:58:00.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.610+0000 7f684822b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68400ff2c0 0x7f6840195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:00.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.610+0000 7f684822b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68400ffc60 0x7f6840196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:00.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.610+0000 7f684822b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6840196a20 con 0x7f68400ffc60 2024-09-10T11:58:00.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.610+0000 7f684822b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684019a7b0 con 0x7f68400ff2c0 2024-09-10T11:58:00.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.611+0000 7f68457c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68400ffc60 0x7f6840196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:00.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.611+0000 7f68457c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68400ffc60 0x7f6840196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44202/0 (socket says 172.21.15.144:44202) 2024-09-10T11:58:00.613 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.611+0000 7f68457c6700 1 -- 172.21.15.144:0/3722302283 learned_addr learned my addr 172.21.15.144:0/3722302283 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:58:00.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.611+0000 7f6845fc7700 1 --2- 172.21.15.144:0/3722302283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68400ff2c0 0x7f6840195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:00.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.611+0000 7f68457c6700 1 -- 172.21.15.144:0/3722302283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68400ff2c0 msgr2=0x7f6840195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:00.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.611+0000 7f68457c6700 1 --2- 172.21.15.144:0/3722302283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68400ff2c0 0x7f6840195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.611+0000 7f68457c6700 1 -- 172.21.15.144:0/3722302283 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68300095f0 con 0x7f68400ffc60 2024-09-10T11:58:00.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.611+0000 7f68457c6700 1 --2- 172.21.15.144:0/3722302283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68400ffc60 0x7f6840196340 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f68300092e0 tx=0x7f683000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:00.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.611+0000 7f6836ffd700 1 -- 172.21.15.144:0/3722302283 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f683001d070 con 0x7f68400ffc60 2024-09-10T11:58:00.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.611+0000 7f6836ffd700 1 -- 172.21.15.144:0/3722302283 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f683000f460 con 0x7f68400ffc60 2024-09-10T11:58:00.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.611+0000 7f684822b700 1 -- 172.21.15.144:0/3722302283 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684019aa30 con 0x7f68400ffc60 2024-09-10T11:58:00.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.611+0000 7f6836ffd700 1 -- 172.21.15.144:0/3722302283 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f68300175d0 con 0x7f68400ffc60 2024-09-10T11:58:00.614 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.611+0000 7f684822b700 1 -- 172.21.15.144:0/3722302283 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684019af20 con 0x7f68400ffc60 2024-09-10T11:58:00.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.612+0000 7f684822b700 1 -- 172.21.15.144:0/3722302283 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6840107630 con 0x7f68400ffc60 2024-09-10T11:58:00.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.613+0000 7f6836ffd700 1 -- 172.21.15.144:0/3722302283 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f683000f5d0 con 0x7f68400ffc60 2024-09-10T11:58:00.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.614+0000 7f6836ffd700 1 --2- 172.21.15.144:0/3722302283 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f682c06e200 0x7f682c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:00.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.614+0000 7f6836ffd700 1 -- 172.21.15.144:0/3722302283 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f683008f570 con 0x7f68400ffc60 2024-09-10T11:58:00.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.614+0000 7f6845fc7700 1 --2- 172.21.15.144:0/3722302283 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f682c06e200 0x7f682c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:00.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.614+0000 7f6845fc7700 1 --2- 172.21.15.144:0/3722302283 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f682c06e200 0x7f682c0706b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f683c005950 tx=0x7f683c00a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:00.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.618+0000 7f6836ffd700 1 -- 172.21.15.144:0/3722302283 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6830058950 con 0x7f68400ffc60 2024-09-10T11:58:00.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.846+0000 7f684822b700 1 -- 172.21.15.144:0/3722302283 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f684002cd10 con 0x7f682c06e200 2024-09-10T11:58:00.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.847+0000 7f6836ffd700 1 -- 172.21.15.144:0/3722302283 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f684002cd10 con 0x7f682c06e200 2024-09-10T11:58:00.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.850+0000 7f684822b700 1 -- 172.21.15.144:0/3722302283 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f682c06e200 msgr2=0x7f682c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:00.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.850+0000 7f684822b700 1 --2- 172.21.15.144:0/3722302283 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f682c06e200 0x7f682c0706b0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f683c005950 tx=0x7f683c00a300 comp rx=0 tx=0).stop 2024-09-10T11:58:00.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.850+0000 7f684822b700 1 -- 172.21.15.144:0/3722302283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68400ffc60 msgr2=0x7f6840196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:00.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.850+0000 7f684822b700 1 --2- 172.21.15.144:0/3722302283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68400ffc60 0x7f6840196340 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f68300092e0 tx=0x7f683000bd70 comp rx=0 tx=0).stop 2024-09-10T11:58:00.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.850+0000 7f684822b700 1 -- 172.21.15.144:0/3722302283 shutdown_connections 2024-09-10T11:58:00.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.850+0000 7f684822b700 1 --2- 172.21.15.144:0/3722302283 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f682c06e200 0x7f682c0706b0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.850+0000 7f684822b700 1 --2- 172.21.15.144:0/3722302283 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68400ff2c0 0x7f6840195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.850+0000 7f684822b700 1 --2- 172.21.15.144:0/3722302283 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68400ffc60 0x7f6840196340 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.850+0000 7f684822b700 1 -- 172.21.15.144:0/3722302283 >> 172.21.15.144:0/3722302283 conn(0x7f68400fab80 msgr2=0x7f68400fcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:00.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.850+0000 7f684822b700 1 -- 172.21.15.144:0/3722302283 shutdown_connections 2024-09-10T11:58:00.852 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.850+0000 7f684822b700 1 -- 172.21.15.144:0/3722302283 wait complete. 2024-09-10T11:58:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.956+0000 7f9f71db5700 1 -- 172.21.15.144:0/4031254641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6c073980 msgr2=0x7f9f6c10c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.956+0000 7f9f71db5700 1 --2- 172.21.15.144:0/4031254641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6c073980 0x7f9f6c10c4f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f9f60009940 tx=0x7f9f60009c50 comp rx=0 tx=0).stop 2024-09-10T11:58:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.956+0000 7f9f71db5700 1 -- 172.21.15.144:0/4031254641 shutdown_connections 2024-09-10T11:58:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.956+0000 7f9f71db5700 1 --2- 172.21.15.144:0/4031254641 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6c073980 0x7f9f6c10c4f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.956+0000 7f9f71db5700 1 --2- 172.21.15.144:0/4031254641 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f6c073070 0x7f9f6c073440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.956+0000 7f9f71db5700 1 -- 172.21.15.144:0/4031254641 >> 172.21.15.144:0/4031254641 conn(0x7f9f6c0fbc90 msgr2=0x7f9f6c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.956+0000 7f9f71db5700 1 -- 172.21.15.144:0/4031254641 shutdown_connections 2024-09-10T11:58:00.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.956+0000 7f9f71db5700 1 -- 172.21.15.144:0/4031254641 wait complete. 2024-09-10T11:58:00.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.956+0000 7f9f71db5700 1 Processor -- start 2024-09-10T11:58:00.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.956+0000 7f9f71db5700 1 -- start start 2024-09-10T11:58:00.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f71db5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6c073070 0x7f9f6c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:00.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f71db5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f6c073980 0x7f9f6c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:00.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f71db5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c198c60 con 0x7f9f6c073070 2024-09-10T11:58:00.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f71db5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f6c19c9f0 con 0x7f9f6c073980 2024-09-10T11:58:00.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f6b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6c073070 0x7f9f6c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:00.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f6b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6c073070 0x7f9f6c198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44218/0 (socket says 172.21.15.144:44218) 2024-09-10T11:58:00.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f6b7fe700 1 -- 172.21.15.144:0/135041565 learned_addr learned my addr 172.21.15.144:0/135041565 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:58:00.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f6affd700 1 --2- 172.21.15.144:0/135041565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f6c073980 0x7f9f6c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:00.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f6b7fe700 1 -- 172.21.15.144:0/135041565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f6c073980 msgr2=0x7f9f6c198580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:00.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f6b7fe700 1 --2- 172.21.15.144:0/135041565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f6c073980 0x7f9f6c198580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:00.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f6b7fe700 1 -- 172.21.15.144:0/135041565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f600095f0 con 0x7f9f6c073070 2024-09-10T11:58:00.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f6b7fe700 1 --2- 172.21.15.144:0/135041565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6c073070 0x7f9f6c198040 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00ba50 tx=0x7f9f5c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:00.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f68ff9700 1 -- 172.21.15.144:0/135041565 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f5c00ca00 con 0x7f9f6c073070 2024-09-10T11:58:00.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f68ff9700 1 -- 172.21.15.144:0/135041565 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f5c00cb60 con 0x7f9f6c073070 2024-09-10T11:58:00.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f68ff9700 1 -- 172.21.15.144:0/135041565 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f5c0075a0 con 0x7f9f6c073070 2024-09-10T11:58:00.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f71db5700 1 -- 172.21.15.144:0/135041565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f6c19ccd0 con 0x7f9f6c073070 2024-09-10T11:58:00.964 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.957+0000 7f9f71db5700 1 -- 172.21.15.144:0/135041565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f6c19d220 con 0x7f9f6c073070 2024-09-10T11:58:00.964 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.959+0000 7f9f71db5700 1 -- 172.21.15.144:0/135041565 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f6c109bf0 con 0x7f9f6c073070 2024-09-10T11:58:00.964 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.959+0000 7f9f68ff9700 1 -- 172.21.15.144:0/135041565 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9f5c00ccd0 con 0x7f9f6c073070 2024-09-10T11:58:00.964 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.959+0000 7f9f68ff9700 1 --2- 172.21.15.144:0/135041565 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f5406e200 0x7f9f540706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:00.964 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.960+0000 7f9f68ff9700 1 -- 172.21.15.144:0/135041565 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9f5c006e40 con 0x7f9f6c073070 2024-09-10T11:58:00.964 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.960+0000 7f9f6affd700 1 --2- 172.21.15.144:0/135041565 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f5406e200 0x7f9f540706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:00.964 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.960+0000 7f9f6affd700 1 --2- 172.21.15.144:0/135041565 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f5406e200 0x7f9f540706b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c199660 tx=0x7f9f60005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:00.965 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:00.964+0000 7f9f68ff9700 1 -- 172.21.15.144:0/135041565 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9f5c056cc0 con 0x7f9f6c073070 2024-09-10T11:58:01.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.175+0000 7f9f71db5700 1 -- 172.21.15.144:0/135041565 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f6c19d560 con 0x7f9f5406e200 2024-09-10T11:58:01.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.187+0000 7f9f68ff9700 1 -- 172.21.15.144:0/135041565 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f9f6c19d560 con 0x7f9f5406e200 2024-09-10T11:58:01.189 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:58:01.189 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (26m) 4m ago 36m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:58:01.189 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (37m) 4m ago 37m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:58:01.189 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (36m) 4m ago 36m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:58:01.189 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (23m) 4m ago 36m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:58:01.189 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (23m) 4m ago 35m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:58:01.190 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (25m) 4m ago 36m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:58:01.190 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (20m) 4m ago 31m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:58:01.190 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (20m) 4m ago 31m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:58:01.190 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (4m) 4m ago 31m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T11:58:01.190 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (19m) 4m ago 31m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:58:01.190 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (27m) 4m ago 38m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:58:01.190 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (27m) 4m ago 35m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:58:01.190 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (24m) 4m ago 38m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:58:01.190 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (24m) 4m ago 35m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:58:01.190 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (26m) 4m ago 36m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:58:01.190 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (26m) 4m ago 35m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:58:01.190 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (23m) 4m ago 34m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:58:01.191 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (22m) 4m ago 34m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:58:01.191 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (22m) 4m ago 34m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:58:01.191 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (21m) 4m ago 33m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:58:01.191 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (21m) 4m ago 33m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:58:01.191 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (20m) 4m ago 32m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:58:01.191 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (26m) 4m ago 36m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:58:01.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.190+0000 7f9f71db5700 1 -- 172.21.15.144:0/135041565 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f5406e200 msgr2=0x7f9f540706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:01.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.190+0000 7f9f71db5700 1 --2- 172.21.15.144:0/135041565 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f5406e200 0x7f9f540706b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c199660 tx=0x7f9f60005a90 comp rx=0 tx=0).stop 2024-09-10T11:58:01.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.190+0000 7f9f71db5700 1 -- 172.21.15.144:0/135041565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6c073070 msgr2=0x7f9f6c198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:01.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.190+0000 7f9f71db5700 1 --2- 172.21.15.144:0/135041565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6c073070 0x7f9f6c198040 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c00ba50 tx=0x7f9f5c00be10 comp rx=0 tx=0).stop 2024-09-10T11:58:01.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.190+0000 7f9f71db5700 1 -- 172.21.15.144:0/135041565 shutdown_connections 2024-09-10T11:58:01.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.190+0000 7f9f71db5700 1 --2- 172.21.15.144:0/135041565 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f5406e200 0x7f9f540706b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:01.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.190+0000 7f9f71db5700 1 --2- 172.21.15.144:0/135041565 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f6c073070 0x7f9f6c198040 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:01.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.190+0000 7f9f71db5700 1 --2- 172.21.15.144:0/135041565 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f6c073980 0x7f9f6c198580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:01.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.190+0000 7f9f71db5700 1 -- 172.21.15.144:0/135041565 >> 172.21.15.144:0/135041565 conn(0x7f9f6c0fbc90 msgr2=0x7f9f6c106d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:01.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.190+0000 7f9f71db5700 1 -- 172.21.15.144:0/135041565 shutdown_connections 2024-09-10T11:58:01.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.190+0000 7f9f71db5700 1 -- 172.21.15.144:0/135041565 wait complete. 2024-09-10T11:58:01.296 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.295+0000 7f9cd0b2d700 1 -- 172.21.15.144:0/3189716276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ccc0ffb70 msgr2=0x7f9ccc10a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:01.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.295+0000 7f9cd0b2d700 1 --2- 172.21.15.144:0/3189716276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ccc0ffb70 0x7f9ccc10a5e0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0009910 tx=0x7f9cc0009c20 comp rx=0 tx=0).stop 2024-09-10T11:58:01.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.295+0000 7f9cd0b2d700 1 -- 172.21.15.144:0/3189716276 shutdown_connections 2024-09-10T11:58:01.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.295+0000 7f9cd0b2d700 1 --2- 172.21.15.144:0/3189716276 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ccc0ffb70 0x7f9ccc10a5e0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:01.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.295+0000 7f9cd0b2d700 1 --2- 172.21.15.144:0/3189716276 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ccc0ff260 0x7f9ccc0ff630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:01.297 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.295+0000 7f9cd0b2d700 1 -- 172.21.15.144:0/3189716276 >> 172.21.15.144:0/3189716276 conn(0x7f9ccc0747e0 msgr2=0x7f9ccc074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:01.297 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:00 smithi144 ceph-mon[87436]: from='client.35436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:01.297 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:00 smithi144 ceph-mon[87436]: from='client.35440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:01.297 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:00 smithi144 ceph-mon[87436]: pgmap v782: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:01.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.295+0000 7f9cd0b2d700 1 -- 172.21.15.144:0/3189716276 shutdown_connections 2024-09-10T11:58:01.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.295+0000 7f9cd0b2d700 1 -- 172.21.15.144:0/3189716276 wait complete. 2024-09-10T11:58:01.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.296+0000 7f9cd0b2d700 1 Processor -- start 2024-09-10T11:58:01.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.296+0000 7f9cd0b2d700 1 -- start start 2024-09-10T11:58:01.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.296+0000 7f9cd0b2d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ccc0ff260 0x7f9ccc100440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:01.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.296+0000 7f9cd0b2d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ccc0ffb70 0x7f9ccc100980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:01.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.296+0000 7f9cd0b2d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc100ec0 con 0x7f9ccc0ffb70 2024-09-10T11:58:01.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.296+0000 7f9cd0b2d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ccc101000 con 0x7f9ccc0ff260 2024-09-10T11:58:01.299 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.297+0000 7f9cc9d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ccc0ffb70 0x7f9ccc100980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:01.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.297+0000 7f9cc9d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ccc0ffb70 0x7f9ccc100980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44238/0 (socket says 172.21.15.144:44238) 2024-09-10T11:58:01.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.297+0000 7f9cc9d9b700 1 -- 172.21.15.144:0/2643583269 learned_addr learned my addr 172.21.15.144:0/2643583269 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:58:01.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.297+0000 7f9cca59c700 1 --2- 172.21.15.144:0/2643583269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ccc0ff260 0x7f9ccc100440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:01.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.297+0000 7f9cc9d9b700 1 -- 172.21.15.144:0/2643583269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ccc0ff260 msgr2=0x7f9ccc100440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:01.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.297+0000 7f9cc9d9b700 1 --2- 172.21.15.144:0/2643583269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ccc0ff260 0x7f9ccc100440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:01.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.297+0000 7f9cc9d9b700 1 -- 172.21.15.144:0/2643583269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc00095f0 con 0x7f9ccc0ffb70 2024-09-10T11:58:01.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.297+0000 7f9cc9d9b700 1 --2- 172.21.15.144:0/2643583269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ccc0ffb70 0x7f9ccc100980 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0006010 tx=0x7f9cc000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:01.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.297+0000 7f9cbb7fe700 1 -- 172.21.15.144:0/2643583269 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cc001d070 con 0x7f9ccc0ffb70 2024-09-10T11:58:01.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.297+0000 7f9cbb7fe700 1 -- 172.21.15.144:0/2643583269 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cc0005dc0 con 0x7f9ccc0ffb70 2024-09-10T11:58:01.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.297+0000 7f9cd0b2d700 1 -- 172.21.15.144:0/2643583269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ccc101280 con 0x7f9ccc0ffb70 2024-09-10T11:58:01.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.297+0000 7f9cbb7fe700 1 -- 172.21.15.144:0/2643583269 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cc0017400 con 0x7f9ccc0ffb70 2024-09-10T11:58:01.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.298+0000 7f9cd0b2d700 1 -- 172.21.15.144:0/2643583269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ccc1a2260 con 0x7f9ccc0ffb70 2024-09-10T11:58:01.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.298+0000 7f9cd0b2d700 1 -- 172.21.15.144:0/2643583269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ccc107ce0 con 0x7f9ccc0ffb70 2024-09-10T11:58:01.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.300+0000 7f9cbb7fe700 1 -- 172.21.15.144:0/2643583269 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9cc0003a60 con 0x7f9ccc0ffb70 2024-09-10T11:58:01.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.300+0000 7f9cbb7fe700 1 --2- 172.21.15.144:0/2643583269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9cb406e200 0x7f9cb40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:01.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.300+0000 7f9cbb7fe700 1 -- 172.21.15.144:0/2643583269 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9cc008f500 con 0x7f9ccc0ffb70 2024-09-10T11:58:01.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.300+0000 7f9cca59c700 1 --2- 172.21.15.144:0/2643583269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9cb406e200 0x7f9cb40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:01.302 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.301+0000 7f9cca59c700 1 --2- 172.21.15.144:0/2643583269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9cb406e200 0x7f9cb40706b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc005950 tx=0x7f9cbc00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:01.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.305+0000 7f9cbb7fe700 1 -- 172.21.15.144:0/2643583269 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9cc00588e0 con 0x7f9ccc0ffb70 2024-09-10T11:58:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:00 smithi178 ceph-mon[65940]: from='client.35436 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:00 smithi178 ceph-mon[65940]: from='client.35440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:00 smithi178 ceph-mon[65940]: pgmap v782: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:01.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.589+0000 7f9cd0b2d700 1 -- 172.21.15.144:0/2643583269 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9ccc04ea50 con 0x7f9ccc0ffb70 2024-09-10T11:58:01.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.589+0000 7f9cbb7fe700 1 -- 172.21.15.144:0/2643583269 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f9cc005bfa0 con 0x7f9ccc0ffb70 2024-09-10T11:58:01.591 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:58:01.591 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:58:01.591 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:58:01.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:58:01.591 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:58:01.591 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:58:01.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:58:01.591 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:58:01.591 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:58:01.591 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:58:01.592 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:58:01.592 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:58:01.592 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:58:01.592 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:58:01.592 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T11:58:01.592 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:58:01.592 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:58:01.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.591+0000 7f9cd0b2d700 1 -- 172.21.15.144:0/2643583269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9cb406e200 msgr2=0x7f9cb40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:01.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.591+0000 7f9cd0b2d700 1 --2- 172.21.15.144:0/2643583269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9cb406e200 0x7f9cb40706b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9cbc005950 tx=0x7f9cbc00b410 comp rx=0 tx=0).stop 2024-09-10T11:58:01.593 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.592+0000 7f9cd0b2d700 1 -- 172.21.15.144:0/2643583269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ccc0ffb70 msgr2=0x7f9ccc100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:01.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.592+0000 7f9cd0b2d700 1 --2- 172.21.15.144:0/2643583269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ccc0ffb70 0x7f9ccc100980 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f9cc0006010 tx=0x7f9cc000bd70 comp rx=0 tx=0).stop 2024-09-10T11:58:01.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.592+0000 7f9cd0b2d700 1 -- 172.21.15.144:0/2643583269 shutdown_connections 2024-09-10T11:58:01.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.592+0000 7f9cd0b2d700 1 --2- 172.21.15.144:0/2643583269 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9cb406e200 0x7f9cb40706b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:01.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.592+0000 7f9cd0b2d700 1 --2- 172.21.15.144:0/2643583269 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9ccc0ff260 0x7f9ccc100440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:01.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.592+0000 7f9cd0b2d700 1 --2- 172.21.15.144:0/2643583269 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9ccc0ffb70 0x7f9ccc100980 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:01.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.592+0000 7f9cd0b2d700 1 -- 172.21.15.144:0/2643583269 >> 172.21.15.144:0/2643583269 conn(0x7f9ccc0747e0 msgr2=0x7f9ccc104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:01.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.592+0000 7f9cd0b2d700 1 -- 172.21.15.144:0/2643583269 shutdown_connections 2024-09-10T11:58:01.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.592+0000 7f9cd0b2d700 1 -- 172.21.15.144:0/2643583269 wait complete. 2024-09-10T11:58:01.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.696+0000 7f1d4aeb6700 1 -- 172.21.15.144:0/189338201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d441003d0 msgr2=0x7f1d441007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:01.699 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.696+0000 7f1d4aeb6700 1 --2- 172.21.15.144:0/189338201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d441003d0 0x7f1d441007a0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009910 tx=0x7f1d38009c20 comp rx=0 tx=0).stop 2024-09-10T11:58:01.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.696+0000 7f1d4aeb6700 1 -- 172.21.15.144:0/189338201 shutdown_connections 2024-09-10T11:58:01.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.696+0000 7f1d4aeb6700 1 --2- 172.21.15.144:0/189338201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d44100d70 0x7f1d44104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:01.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.696+0000 7f1d4aeb6700 1 --2- 172.21.15.144:0/189338201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d441003d0 0x7f1d441007a0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:01.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.696+0000 7f1d4aeb6700 1 -- 172.21.15.144:0/189338201 >> 172.21.15.144:0/189338201 conn(0x7f1d440fbc50 msgr2=0x7f1d440fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:01.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.697+0000 7f1d4aeb6700 1 -- 172.21.15.144:0/189338201 shutdown_connections 2024-09-10T11:58:01.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.697+0000 7f1d4aeb6700 1 -- 172.21.15.144:0/189338201 wait complete. 2024-09-10T11:58:01.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.697+0000 7f1d4aeb6700 1 Processor -- start 2024-09-10T11:58:01.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.697+0000 7f1d4aeb6700 1 -- start start 2024-09-10T11:58:01.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.697+0000 7f1d4aeb6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d441003d0 0x7f1d44195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:01.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.697+0000 7f1d4aeb6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d44100d70 0x7f1d441962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:01.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.697+0000 7f1d4aeb6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d44196940 con 0x7f1d44100d70 2024-09-10T11:58:01.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.697+0000 7f1d4aeb6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d44196a80 con 0x7f1d441003d0 2024-09-10T11:58:01.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.697+0000 7f1d43fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d44100d70 0x7f1d441962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:01.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.697+0000 7f1d43fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d44100d70 0x7f1d441962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44266/0 (socket says 172.21.15.144:44266) 2024-09-10T11:58:01.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.698+0000 7f1d43fff700 1 -- 172.21.15.144:0/1175884255 learned_addr learned my addr 172.21.15.144:0/1175884255 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:58:01.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.698+0000 7f1d48c52700 1 --2- 172.21.15.144:0/1175884255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d441003d0 0x7f1d44195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:01.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.698+0000 7f1d43fff700 1 -- 172.21.15.144:0/1175884255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d441003d0 msgr2=0x7f1d44195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:01.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.698+0000 7f1d43fff700 1 --2- 172.21.15.144:0/1175884255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d441003d0 0x7f1d44195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:01.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.698+0000 7f1d43fff700 1 -- 172.21.15.144:0/1175884255 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d380095f0 con 0x7f1d44100d70 2024-09-10T11:58:01.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.698+0000 7f1d48c52700 1 --2- 172.21.15.144:0/1175884255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d441003d0 0x7f1d44195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:58:01.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.698+0000 7f1d43fff700 1 --2- 172.21.15.144:0/1175884255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d44100d70 0x7f1d441962f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000c960 tx=0x7f1d3000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:01.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.698+0000 7f1d41ffb700 1 -- 172.21.15.144:0/1175884255 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d30007a70 con 0x7f1d44100d70 2024-09-10T11:58:01.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.698+0000 7f1d41ffb700 1 -- 172.21.15.144:0/1175884255 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1d30007bd0 con 0x7f1d44100d70 2024-09-10T11:58:01.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.698+0000 7f1d41ffb700 1 -- 172.21.15.144:0/1175884255 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d300186c0 con 0x7f1d44100d70 2024-09-10T11:58:01.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.698+0000 7f1d4aeb6700 1 -- 172.21.15.144:0/1175884255 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4419a8d0 con 0x7f1d44100d70 2024-09-10T11:58:01.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.698+0000 7f1d4aeb6700 1 -- 172.21.15.144:0/1175884255 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4419ae20 con 0x7f1d44100d70 2024-09-10T11:58:01.702 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.699+0000 7f1d4aeb6700 1 -- 172.21.15.144:0/1175884255 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d441086e0 con 0x7f1d44100d70 2024-09-10T11:58:01.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.700+0000 7f1d41ffb700 1 -- 172.21.15.144:0/1175884255 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1d3000faa0 con 0x7f1d44100d70 2024-09-10T11:58:01.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.700+0000 7f1d41ffb700 1 --2- 172.21.15.144:0/1175884255 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1d3406e200 0x7f1d340706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:01.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.700+0000 7f1d41ffb700 1 -- 172.21.15.144:0/1175884255 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1d3008e070 con 0x7f1d44100d70 2024-09-10T11:58:01.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.700+0000 7f1d48c52700 1 --2- 172.21.15.144:0/1175884255 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1d3406e200 0x7f1d340706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:01.703 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.701+0000 7f1d48c52700 1 --2- 172.21.15.144:0/1175884255 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1d3406e200 0x7f1d340706b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800b5c0 tx=0x7f1d38005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:01.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.705+0000 7f1d41ffb700 1 -- 172.21.15.144:0/1175884255 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1d3005b000 con 0x7f1d44100d70 2024-09-10T11:58:01.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.946+0000 7f1d4aeb6700 1 -- 172.21.15.144:0/1175884255 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1d4404ea50 con 0x7f1d44100d70 2024-09-10T11:58:01.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.947+0000 7f1d41ffb700 1 -- 172.21.15.144:0/1175884255 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f1d3005ab90 con 0x7f1d44100d70 2024-09-10T11:58:01.948 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T11:58:01.948 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T11:58:01.948 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:58:01.948 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:58:01.949 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:58:01.949 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:58:01.949 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:58:01.949 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:58:01.949 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:58:01.949 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:58:01.949 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:58:01.949 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:58:01.949 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:58:01.949 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:58:01.949 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:58:01.949 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:58:01.950 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:58:01.950 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:58:01.950 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:58:01.950 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:58:01.950 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:58:01.950 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:58:01.950 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:58:01.950 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:58:01.950 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:58:01.950 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:58:01.950 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:58:01.950 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:58:01.951 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:58:01.951 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:58:01.951 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:58:01.951 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:58:01.951 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:58:01.951 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:58:01.951 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:58:01.951 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:58:01.951 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:58:01.951 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:58:01.951 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:58:01.952 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:58:01.952 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:58:01.952 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:58:01.952 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:58:01.952 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.949+0000 7f1d4aeb6700 1 -- 172.21.15.144:0/1175884255 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1d3406e200 msgr2=0x7f1d340706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:01.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.949+0000 7f1d4aeb6700 1 --2- 172.21.15.144:0/1175884255 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1d3406e200 0x7f1d340706b0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800b5c0 tx=0x7f1d38005960 comp rx=0 tx=0).stop 2024-09-10T11:58:01.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.949+0000 7f1d4aeb6700 1 -- 172.21.15.144:0/1175884255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d44100d70 msgr2=0x7f1d441962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:01.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.949+0000 7f1d4aeb6700 1 --2- 172.21.15.144:0/1175884255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d44100d70 0x7f1d441962f0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000c960 tx=0x7f1d3000cd20 comp rx=0 tx=0).stop 2024-09-10T11:58:01.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.949+0000 7f1d4aeb6700 1 -- 172.21.15.144:0/1175884255 shutdown_connections 2024-09-10T11:58:01.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.949+0000 7f1d4aeb6700 1 --2- 172.21.15.144:0/1175884255 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1d3406e200 0x7f1d340706b0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:01.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.949+0000 7f1d4aeb6700 1 --2- 172.21.15.144:0/1175884255 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1d441003d0 0x7f1d44195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:01.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.949+0000 7f1d4aeb6700 1 --2- 172.21.15.144:0/1175884255 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1d44100d70 0x7f1d441962f0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:01.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.949+0000 7f1d4aeb6700 1 -- 172.21.15.144:0/1175884255 >> 172.21.15.144:0/1175884255 conn(0x7f1d440fbc50 msgr2=0x7f1d440fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:01.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.949+0000 7f1d4aeb6700 1 -- 172.21.15.144:0/1175884255 shutdown_connections 2024-09-10T11:58:01.953 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:01.949+0000 7f1d4aeb6700 1 -- 172.21.15.144:0/1175884255 wait complete. 2024-09-10T11:58:01.953 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T11:58:02.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.056+0000 7f62401ce700 1 -- 172.21.15.144:0/2172633439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6238100cb0 msgr2=0x7f6238109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:02.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.056+0000 7f62401ce700 1 --2- 172.21.15.144:0/2172633439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6238100cb0 0x7f6238109410 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6228009940 tx=0x7f6228009c50 comp rx=0 tx=0).stop 2024-09-10T11:58:02.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.056+0000 7f62401ce700 1 -- 172.21.15.144:0/2172633439 shutdown_connections 2024-09-10T11:58:02.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.056+0000 7f62401ce700 1 --2- 172.21.15.144:0/2172633439 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6238100cb0 0x7f6238109410 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:02.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.056+0000 7f62401ce700 1 --2- 172.21.15.144:0/2172633439 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62381003a0 0x7f6238100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:02.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.056+0000 7f62401ce700 1 -- 172.21.15.144:0/2172633439 >> 172.21.15.144:0/2172633439 conn(0x7f6238074850 msgr2=0x7f6238074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:02.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.056+0000 7f62401ce700 1 -- 172.21.15.144:0/2172633439 shutdown_connections 2024-09-10T11:58:02.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.056+0000 7f62401ce700 1 -- 172.21.15.144:0/2172633439 wait complete. 2024-09-10T11:58:02.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.056+0000 7f62401ce700 1 Processor -- start 2024-09-10T11:58:02.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.056+0000 7f62401ce700 1 -- start start 2024-09-10T11:58:02.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f62401ce700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62381003a0 0x7f623819bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:02.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f62401ce700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6238100cb0 0x7f623819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:02.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f62401ce700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623819c6a0 con 0x7f62381003a0 2024-09-10T11:58:02.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f62401ce700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6238195b50 con 0x7f6238100cb0 2024-09-10T11:58:02.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f623df6a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62381003a0 0x7f623819bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:02.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f623df6a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62381003a0 0x7f623819bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:44288/0 (socket says 172.21.15.144:44288) 2024-09-10T11:58:02.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f623df6a700 1 -- 172.21.15.144:0/806679750 learned_addr learned my addr 172.21.15.144:0/806679750 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:58:02.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f623d769700 1 --2- 172.21.15.144:0/806679750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6238100cb0 0x7f623819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:02.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f623df6a700 1 -- 172.21.15.144:0/806679750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6238100cb0 msgr2=0x7f623819c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:02.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f623df6a700 1 --2- 172.21.15.144:0/806679750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6238100cb0 0x7f623819c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:02.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f623df6a700 1 -- 172.21.15.144:0/806679750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62280095f0 con 0x7f62381003a0 2024-09-10T11:58:02.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f623df6a700 1 --2- 172.21.15.144:0/806679750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62381003a0 0x7f623819bad0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f623400d8d0 tx=0x7f623400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:02.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f622effd700 1 -- 172.21.15.144:0/806679750 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6234009940 con 0x7f62381003a0 2024-09-10T11:58:02.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f622effd700 1 -- 172.21.15.144:0/806679750 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6234010460 con 0x7f62381003a0 2024-09-10T11:58:02.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f62401ce700 1 -- 172.21.15.144:0/806679750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6238195e30 con 0x7f62381003a0 2024-09-10T11:58:02.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f622effd700 1 -- 172.21.15.144:0/806679750 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f623400f5f0 con 0x7f62381003a0 2024-09-10T11:58:02.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.057+0000 7f62401ce700 1 -- 172.21.15.144:0/806679750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6238196380 con 0x7f62381003a0 2024-09-10T11:58:02.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.058+0000 7f62401ce700 1 -- 172.21.15.144:0/806679750 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623804ea50 con 0x7f62381003a0 2024-09-10T11:58:02.062 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.059+0000 7f622effd700 1 -- 172.21.15.144:0/806679750 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f62340105d0 con 0x7f62381003a0 2024-09-10T11:58:02.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.059+0000 7f622effd700 1 --2- 172.21.15.144:0/806679750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f622406e200 0x7f62240706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:02.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.060+0000 7f622effd700 1 -- 172.21.15.144:0/806679750 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f623408df90 con 0x7f62381003a0 2024-09-10T11:58:02.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.060+0000 7f623d769700 1 --2- 172.21.15.144:0/806679750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f622406e200 0x7f62240706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:02.063 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.060+0000 7f623d769700 1 --2- 172.21.15.144:0/806679750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f622406e200 0x7f62240706b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f62380fdd20 tx=0x7f6228005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:02.065 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.064+0000 7f622effd700 1 -- 172.21.15.144:0/806679750 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f623405af20 con 0x7f62381003a0 2024-09-10T11:58:02.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.287+0000 7f62401ce700 1 -- 172.21.15.144:0/806679750 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6238196850 con 0x7f622406e200 2024-09-10T11:58:02.289 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:01 smithi144 ceph-mon[87436]: from='client.35444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:02.289 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:01 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2643583269' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:58:02.289 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:01 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1175884255' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:58:02.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.289+0000 7f622effd700 1 -- 172.21.15.144:0/806679750 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6238196850 con 0x7f622406e200 2024-09-10T11:58:02.290 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:58:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:58:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:58:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:58:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:58:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:58:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T11:58:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T11:58:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:58:02.290 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T11:58:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:58:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T11:58:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:58:02.291 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:58:02.291 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:58:02.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.291+0000 7f62401ce700 1 -- 172.21.15.144:0/806679750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f622406e200 msgr2=0x7f62240706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:02.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.291+0000 7f62401ce700 1 --2- 172.21.15.144:0/806679750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f622406e200 0x7f62240706b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f62380fdd20 tx=0x7f6228005ab0 comp rx=0 tx=0).stop 2024-09-10T11:58:02.292 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.291+0000 7f62401ce700 1 -- 172.21.15.144:0/806679750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62381003a0 msgr2=0x7f623819bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:02.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.291+0000 7f62401ce700 1 --2- 172.21.15.144:0/806679750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62381003a0 0x7f623819bad0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f623400d8d0 tx=0x7f623400dc90 comp rx=0 tx=0).stop 2024-09-10T11:58:02.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.291+0000 7f62401ce700 1 -- 172.21.15.144:0/806679750 shutdown_connections 2024-09-10T11:58:02.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.291+0000 7f62401ce700 1 --2- 172.21.15.144:0/806679750 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f622406e200 0x7f62240706b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:02.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.291+0000 7f62401ce700 1 --2- 172.21.15.144:0/806679750 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62381003a0 0x7f623819bad0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:02.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.291+0000 7f62401ce700 1 --2- 172.21.15.144:0/806679750 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6238100cb0 0x7f623819c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:02.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.291+0000 7f62401ce700 1 -- 172.21.15.144:0/806679750 >> 172.21.15.144:0/806679750 conn(0x7f6238074850 msgr2=0x7f62380feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:02.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.291+0000 7f62401ce700 1 -- 172.21.15.144:0/806679750 shutdown_connections 2024-09-10T11:58:02.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.291+0000 7f62401ce700 1 -- 172.21.15.144:0/806679750 wait complete. 2024-09-10T11:58:02.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:01 smithi178 ceph-mon[65940]: from='client.35444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:02.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:01 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2643583269' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:58:02.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:01 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1175884255' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:58:02.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.394+0000 7f4fde097700 1 -- 172.21.15.144:0/1961233284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd80ff2c0 msgr2=0x7f4fd80ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:02.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.394+0000 7f4fde097700 1 --2- 172.21.15.144:0/1961233284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd80ff2c0 0x7f4fd80ff690 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009910 tx=0x7f4fc8009c20 comp rx=0 tx=0).stop 2024-09-10T11:58:02.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.394+0000 7f4fde097700 1 -- 172.21.15.144:0/1961233284 shutdown_connections 2024-09-10T11:58:02.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.394+0000 7f4fde097700 1 --2- 172.21.15.144:0/1961233284 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fd80ffc60 0x7f4fd8103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:02.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.394+0000 7f4fde097700 1 --2- 172.21.15.144:0/1961233284 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd80ff2c0 0x7f4fd80ff690 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:02.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.394+0000 7f4fde097700 1 -- 172.21.15.144:0/1961233284 >> 172.21.15.144:0/1961233284 conn(0x7f4fd80fab80 msgr2=0x7f4fd80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:02.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.394+0000 7f4fde097700 1 -- 172.21.15.144:0/1961233284 shutdown_connections 2024-09-10T11:58:02.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.394+0000 7f4fde097700 1 -- 172.21.15.144:0/1961233284 wait complete. 2024-09-10T11:58:02.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.395+0000 7f4fde097700 1 Processor -- start 2024-09-10T11:58:02.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.395+0000 7f4fde097700 1 -- start start 2024-09-10T11:58:02.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.395+0000 7f4fde097700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fd80ff2c0 0x7f4fd8195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:02.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.395+0000 7f4fde097700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd80ffc60 0x7f4fd8196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:02.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.395+0000 7f4fde097700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd81969e0 con 0x7f4fd80ffc60 2024-09-10T11:58:02.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.395+0000 7f4fde097700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd819a770 con 0x7f4fd80ff2c0 2024-09-10T11:58:02.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.396+0000 7f4fd6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd80ffc60 0x7f4fd8196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:02.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.396+0000 7f4fd6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd80ffc60 0x7f4fd8196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:38930/0 (socket says 172.21.15.144:38930) 2024-09-10T11:58:02.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.396+0000 7f4fd6ffd700 1 -- 172.21.15.144:0/407221192 learned_addr learned my addr 172.21.15.144:0/407221192 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:58:02.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.396+0000 7f4fd77fe700 1 --2- 172.21.15.144:0/407221192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fd80ff2c0 0x7f4fd8195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:02.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.396+0000 7f4fd6ffd700 1 -- 172.21.15.144:0/407221192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fd80ff2c0 msgr2=0x7f4fd8195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:02.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.396+0000 7f4fd6ffd700 1 --2- 172.21.15.144:0/407221192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fd80ff2c0 0x7f4fd8195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:02.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.396+0000 7f4fd6ffd700 1 -- 172.21.15.144:0/407221192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc80095f0 con 0x7f4fd80ffc60 2024-09-10T11:58:02.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.396+0000 7f4fd6ffd700 1 --2- 172.21.15.144:0/407221192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd80ffc60 0x7f4fd8196300 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00d900 tx=0x7f4fcc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:02.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.396+0000 7f4fd4ff9700 1 -- 172.21.15.144:0/407221192 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fcc009900 con 0x7f4fd80ffc60 2024-09-10T11:58:02.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.396+0000 7f4fd4ff9700 1 -- 172.21.15.144:0/407221192 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fcc010460 con 0x7f4fd80ffc60 2024-09-10T11:58:02.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.396+0000 7f4fd4ff9700 1 -- 172.21.15.144:0/407221192 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fcc00f5f0 con 0x7f4fd80ffc60 2024-09-10T11:58:02.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.396+0000 7f4fde097700 1 -- 172.21.15.144:0/407221192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd819aa50 con 0x7f4fd80ffc60 2024-09-10T11:58:02.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.396+0000 7f4fde097700 1 -- 172.21.15.144:0/407221192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd819afa0 con 0x7f4fd80ffc60 2024-09-10T11:58:02.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.397+0000 7f4fde097700 1 -- 172.21.15.144:0/407221192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd804ea50 con 0x7f4fd80ffc60 2024-09-10T11:58:02.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.398+0000 7f4fd4ff9700 1 -- 172.21.15.144:0/407221192 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4fcc0105d0 con 0x7f4fd80ffc60 2024-09-10T11:58:02.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.398+0000 7f4fd4ff9700 1 --2- 172.21.15.144:0/407221192 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fc006e200 0x7f4fc00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:02.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.398+0000 7f4fd4ff9700 1 -- 172.21.15.144:0/407221192 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4fcc08db40 con 0x7f4fd80ffc60 2024-09-10T11:58:02.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.398+0000 7f4fd77fe700 1 --2- 172.21.15.144:0/407221192 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fc006e200 0x7f4fc00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:02.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.399+0000 7f4fd77fe700 1 --2- 172.21.15.144:0/407221192 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fc006e200 0x7f4fc00706b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800b5c0 tx=0x7f4fc801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:02.404 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.402+0000 7f4fd4ff9700 1 -- 172.21.15.144:0/407221192 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4fcc056fa0 con 0x7f4fd80ffc60 2024-09-10T11:58:02.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.689+0000 7f4fde097700 1 -- 172.21.15.144:0/407221192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f4fd8197120 con 0x7f4fd80ffc60 2024-09-10T11:58:02.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.690+0000 7f4fd4ff9700 1 -- 172.21.15.144:0/407221192 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f4fcc017090 con 0x7f4fd80ffc60 2024-09-10T11:58:02.691 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:58:02.691 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T11:58:02.691 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:58:02.692 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T11:58:02.692 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:58:02.692 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T11:58:02.692 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T11:58:02.692 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T11:58:02.692 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T11:58:02.692 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T11:58:02.692 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T11:58:02.692 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T11:58:02.692 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T11:58:02.692 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T11:58:02.692 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T11:58:02.692 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T11:58:02.693 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T11:58:02.693 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T11:58:02.693 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:58:02.693 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:58:02.693 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:58:02.693 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:58:02.693 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:58:02.693 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:58:02.693 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:58:02.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.692+0000 7f4fde097700 1 -- 172.21.15.144:0/407221192 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fc006e200 msgr2=0x7f4fc00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:02.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.692+0000 7f4fde097700 1 --2- 172.21.15.144:0/407221192 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fc006e200 0x7f4fc00706b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f4fc800b5c0 tx=0x7f4fc801a040 comp rx=0 tx=0).stop 2024-09-10T11:58:02.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.692+0000 7f4fde097700 1 -- 172.21.15.144:0/407221192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd80ffc60 msgr2=0x7f4fd8196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:02.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.692+0000 7f4fde097700 1 --2- 172.21.15.144:0/407221192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd80ffc60 0x7f4fd8196300 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4fcc00d900 tx=0x7f4fcc00dc10 comp rx=0 tx=0).stop 2024-09-10T11:58:02.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.692+0000 7f4fde097700 1 -- 172.21.15.144:0/407221192 shutdown_connections 2024-09-10T11:58:02.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.692+0000 7f4fde097700 1 --2- 172.21.15.144:0/407221192 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fc006e200 0x7f4fc00706b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:02.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.692+0000 7f4fde097700 1 --2- 172.21.15.144:0/407221192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fd80ff2c0 0x7f4fd8195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:02.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.692+0000 7f4fde097700 1 --2- 172.21.15.144:0/407221192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd80ffc60 0x7f4fd8196300 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:02.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.692+0000 7f4fde097700 1 -- 172.21.15.144:0/407221192 >> 172.21.15.144:0/407221192 conn(0x7f4fd80fab80 msgr2=0x7f4fd8068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:02.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.692+0000 7f4fde097700 1 -- 172.21.15.144:0/407221192 shutdown_connections 2024-09-10T11:58:02.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:02.692+0000 7f4fde097700 1 -- 172.21.15.144:0/407221192 wait complete. 2024-09-10T11:58:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:02 smithi178 ceph-mon[65940]: from='client.35456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:02 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/407221192' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:58:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:02 smithi178 ceph-mon[65940]: pgmap v783: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:03.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:02 smithi144 ceph-mon[87436]: from='client.35456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:02 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/407221192' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:58:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:02 smithi144 ceph-mon[87436]: pgmap v783: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:04 smithi178 ceph-mon[65940]: pgmap v784: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:05.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:04 smithi144 ceph-mon[87436]: pgmap v784: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:06 smithi178 ceph-mon[65940]: pgmap v785: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:07.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:06 smithi144 ceph-mon[87436]: pgmap v785: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:08 smithi178 ceph-mon[65940]: pgmap v786: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:09.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:08 smithi144 ceph-mon[87436]: pgmap v786: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:11.283 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:10 smithi144 ceph-mon[87436]: pgmap v787: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:10 smithi178 ceph-mon[65940]: pgmap v787: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:12 smithi178 ceph-mon[65940]: pgmap v788: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:13.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:12 smithi144 ceph-mon[87436]: pgmap v788: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:58:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:58:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:14 smithi178 ceph-mon[65940]: pgmap v789: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:15.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:14 smithi144 ceph-mon[87436]: pgmap v789: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:17.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:16 smithi178 ceph-mon[65940]: pgmap v790: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:17.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:16 smithi144 ceph-mon[87436]: pgmap v790: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:19.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:18 smithi178 ceph-mon[65940]: pgmap v791: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:19.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:18 smithi144 ceph-mon[87436]: pgmap v791: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:20 smithi178 ceph-mon[65940]: pgmap v792: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:21.387 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:20 smithi144 ceph-mon[87436]: pgmap v792: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:23.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:22 smithi178 ceph-mon[65940]: pgmap v793: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:23.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:22 smithi144 ceph-mon[87436]: pgmap v793: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:24 smithi178 ceph-mon[65940]: pgmap v794: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:25.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:24 smithi144 ceph-mon[87436]: pgmap v794: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:26 smithi178 ceph-mon[65940]: pgmap v795: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:27.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:26 smithi144 ceph-mon[87436]: pgmap v795: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:28.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:58:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:58:29.966 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:29 smithi178 ceph-mon[65940]: pgmap v796: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:30.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:29 smithi144 ceph-mon[87436]: pgmap v796: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:30 smithi178 ceph-mon[65940]: pgmap v797: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:31.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:30 smithi144 ceph-mon[87436]: pgmap v797: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:32.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.810+0000 7fd565d81700 1 -- 172.21.15.144:0/4079451891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ffd30 msgr2=0x7fd560103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:32.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.810+0000 7fd565d81700 1 --2- 172.21.15.144:0/4079451891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ffd30 0x7fd560103ce0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fd548009910 tx=0x7fd548009c20 comp rx=0 tx=0).stop 2024-09-10T11:58:32.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.811+0000 7fd565d81700 1 -- 172.21.15.144:0/4079451891 shutdown_connections 2024-09-10T11:58:32.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.811+0000 7fd565d81700 1 --2- 172.21.15.144:0/4079451891 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ffd30 0x7fd560103ce0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:32.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.811+0000 7fd565d81700 1 --2- 172.21.15.144:0/4079451891 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5600ff390 0x7fd5600ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:32.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.811+0000 7fd565d81700 1 -- 172.21.15.144:0/4079451891 >> 172.21.15.144:0/4079451891 conn(0x7fd560074f50 msgr2=0x7fd560075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:32.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.811+0000 7fd565d81700 1 -- 172.21.15.144:0/4079451891 shutdown_connections 2024-09-10T11:58:32.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.811+0000 7fd565d81700 1 -- 172.21.15.144:0/4079451891 wait complete. 2024-09-10T11:58:32.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.811+0000 7fd565d81700 1 Processor -- start 2024-09-10T11:58:32.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.811+0000 7fd565d81700 1 -- start start 2024-09-10T11:58:32.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd565d81700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5600ff390 0x7fd560195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:32.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd565d81700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ffd30 0x7fd5601962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:32.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd565d81700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5601969b0 con 0x7fd5600ffd30 2024-09-10T11:58:32.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd565d81700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd56019a740 con 0x7fd5600ff390 2024-09-10T11:58:32.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd557fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ffd30 0x7fd5601962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:32.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd557fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ffd30 0x7fd5601962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39948/0 (socket says 172.21.15.144:39948) 2024-09-10T11:58:32.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd557fff700 1 -- 172.21.15.144:0/3029946319 learned_addr learned my addr 172.21.15.144:0/3029946319 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:58:32.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd55f7fe700 1 --2- 172.21.15.144:0/3029946319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5600ff390 0x7fd560195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:32.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd557fff700 1 -- 172.21.15.144:0/3029946319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5600ff390 msgr2=0x7fd560195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:32.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd557fff700 1 --2- 172.21.15.144:0/3029946319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5600ff390 0x7fd560195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:32.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd557fff700 1 -- 172.21.15.144:0/3029946319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5480095f0 con 0x7fd5600ffd30 2024-09-10T11:58:32.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd557fff700 1 --2- 172.21.15.144:0/3029946319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ffd30 0x7fd5601962d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd5480059d0 tx=0x7fd54800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:32.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd55d7fa700 1 -- 172.21.15.144:0/3029946319 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd54801d070 con 0x7fd5600ffd30 2024-09-10T11:58:32.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd55d7fa700 1 -- 172.21.15.144:0/3029946319 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd54800b610 con 0x7fd5600ffd30 2024-09-10T11:58:32.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd565d81700 1 -- 172.21.15.144:0/3029946319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd56019a9c0 con 0x7fd5600ffd30 2024-09-10T11:58:32.816 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd55d7fa700 1 -- 172.21.15.144:0/3029946319 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd548022470 con 0x7fd5600ffd30 2024-09-10T11:58:32.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.812+0000 7fd565d81700 1 -- 172.21.15.144:0/3029946319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd56019aeb0 con 0x7fd5600ffd30 2024-09-10T11:58:32.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.814+0000 7fd565d81700 1 -- 172.21.15.144:0/3029946319 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd56004ea50 con 0x7fd5600ffd30 2024-09-10T11:58:32.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.814+0000 7fd55d7fa700 1 -- 172.21.15.144:0/3029946319 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd548003710 con 0x7fd5600ffd30 2024-09-10T11:58:32.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.814+0000 7fd55d7fa700 1 --2- 172.21.15.144:0/3029946319 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd54c06e200 0x7fd54c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:32.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.815+0000 7fd55d7fa700 1 -- 172.21.15.144:0/3029946319 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd54808f570 con 0x7fd5600ffd30 2024-09-10T11:58:32.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.815+0000 7fd55f7fe700 1 --2- 172.21.15.144:0/3029946319 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd54c06e200 0x7fd54c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:32.818 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.815+0000 7fd55f7fe700 1 --2- 172.21.15.144:0/3029946319 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd54c06e200 0x7fd54c0706b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd550009c60 tx=0x7fd550009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:32.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:32.819+0000 7fd55d7fa700 1 -- 172.21.15.144:0/3029946319 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd548058950 con 0x7fd5600ffd30 2024-09-10T11:58:33.048 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.046+0000 7fd565d81700 1 -- 172.21.15.144:0/3029946319 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd56019b270 con 0x7fd54c06e200 2024-09-10T11:58:33.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.048+0000 7fd55d7fa700 1 -- 172.21.15.144:0/3029946319 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd56019b270 con 0x7fd54c06e200 2024-09-10T11:58:33.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.050+0000 7fd565d81700 1 -- 172.21.15.144:0/3029946319 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd54c06e200 msgr2=0x7fd54c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:33.051 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.050+0000 7fd565d81700 1 --2- 172.21.15.144:0/3029946319 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd54c06e200 0x7fd54c0706b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fd550009c60 tx=0x7fd550009260 comp rx=0 tx=0).stop 2024-09-10T11:58:33.052 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.050+0000 7fd565d81700 1 -- 172.21.15.144:0/3029946319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ffd30 msgr2=0x7fd5601962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:33.052 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.050+0000 7fd565d81700 1 --2- 172.21.15.144:0/3029946319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ffd30 0x7fd5601962d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd5480059d0 tx=0x7fd54800f690 comp rx=0 tx=0).stop 2024-09-10T11:58:33.052 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.050+0000 7fd565d81700 1 -- 172.21.15.144:0/3029946319 shutdown_connections 2024-09-10T11:58:33.052 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.050+0000 7fd565d81700 1 --2- 172.21.15.144:0/3029946319 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd54c06e200 0x7fd54c0706b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.052 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.050+0000 7fd565d81700 1 --2- 172.21.15.144:0/3029946319 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5600ff390 0x7fd560195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.052 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.050+0000 7fd565d81700 1 --2- 172.21.15.144:0/3029946319 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5600ffd30 0x7fd5601962d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.052 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.050+0000 7fd565d81700 1 -- 172.21.15.144:0/3029946319 >> 172.21.15.144:0/3029946319 conn(0x7fd560074f50 msgr2=0x7fd560103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:33.052 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.050+0000 7fd565d81700 1 -- 172.21.15.144:0/3029946319 shutdown_connections 2024-09-10T11:58:33.052 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.050+0000 7fd565d81700 1 -- 172.21.15.144:0/3029946319 wait complete. 2024-09-10T11:58:33.062 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:58:33.063 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:32 smithi144 ceph-mon[87436]: pgmap v798: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:33.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.155+0000 7f7f0fb10700 1 -- 172.21.15.144:0/2478289089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f080ff390 msgr2=0x7f7f080ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:33.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.155+0000 7f7f0fb10700 1 --2- 172.21.15.144:0/2478289089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f080ff390 0x7f7f080ff760 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009910 tx=0x7f7efc009c20 comp rx=0 tx=0).stop 2024-09-10T11:58:33.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.155+0000 7f7f0fb10700 1 -- 172.21.15.144:0/2478289089 shutdown_connections 2024-09-10T11:58:33.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.155+0000 7f7f0fb10700 1 --2- 172.21.15.144:0/2478289089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f080ffd30 0x7f7f08103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.155+0000 7f7f0fb10700 1 --2- 172.21.15.144:0/2478289089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f080ff390 0x7f7f080ff760 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.155+0000 7f7f0fb10700 1 -- 172.21.15.144:0/2478289089 >> 172.21.15.144:0/2478289089 conn(0x7f7f08074f50 msgr2=0x7f7f08075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:33.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.155+0000 7f7f0fb10700 1 -- 172.21.15.144:0/2478289089 shutdown_connections 2024-09-10T11:58:33.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.155+0000 7f7f0fb10700 1 -- 172.21.15.144:0/2478289089 wait complete. 2024-09-10T11:58:33.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.156+0000 7f7f0fb10700 1 Processor -- start 2024-09-10T11:58:33.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.156+0000 7f7f0fb10700 1 -- start start 2024-09-10T11:58:33.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.156+0000 7f7f0fb10700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f080ff390 0x7f7f08197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:33.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.156+0000 7f7f0fb10700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f080ffd30 0x7f7f081984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:33.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.156+0000 7f7f0fb10700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08198b30 con 0x7f7f080ff390 2024-09-10T11:58:33.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.156+0000 7f7f0fb10700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f08198c70 con 0x7f7f080ffd30 2024-09-10T11:58:33.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.156+0000 7f7f0d8ac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f080ff390 0x7f7f08197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:33.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.156+0000 7f7f0d8ac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f080ff390 0x7f7f08197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39966/0 (socket says 172.21.15.144:39966) 2024-09-10T11:58:33.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.156+0000 7f7f0d8ac700 1 -- 172.21.15.144:0/1558817396 learned_addr learned my addr 172.21.15.144:0/1558817396 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:58:33.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.156+0000 7f7f0d0ab700 1 --2- 172.21.15.144:0/1558817396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f080ffd30 0x7f7f081984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:33.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.156+0000 7f7f0d8ac700 1 -- 172.21.15.144:0/1558817396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f080ffd30 msgr2=0x7f7f081984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:33.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.156+0000 7f7f0d8ac700 1 --2- 172.21.15.144:0/1558817396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f080ffd30 0x7f7f081984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.156+0000 7f7f0d8ac700 1 -- 172.21.15.144:0/1558817396 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc0095f0 con 0x7f7f080ff390 2024-09-10T11:58:33.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.156+0000 7f7f0d0ab700 1 --2- 172.21.15.144:0/1558817396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f080ffd30 0x7f7f081984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:58:33.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.156+0000 7f7f0d8ac700 1 --2- 172.21.15.144:0/1558817396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f080ff390 0x7f7f08197fa0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7efc006010 tx=0x7f7efc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:33.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.157+0000 7f7efaffd700 1 -- 172.21.15.144:0/1558817396 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7efc01d070 con 0x7f7f080ff390 2024-09-10T11:58:33.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.157+0000 7f7efaffd700 1 -- 172.21.15.144:0/1558817396 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7efc004fb0 con 0x7f7f080ff390 2024-09-10T11:58:33.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.157+0000 7f7f0fb10700 1 -- 172.21.15.144:0/1558817396 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f0819ca60 con 0x7f7f080ff390 2024-09-10T11:58:33.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.157+0000 7f7efaffd700 1 -- 172.21.15.144:0/1558817396 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7efc022ea0 con 0x7f7f080ff390 2024-09-10T11:58:33.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.157+0000 7f7f0fb10700 1 -- 172.21.15.144:0/1558817396 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0819cf50 con 0x7f7f080ff390 2024-09-10T11:58:33.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.158+0000 7f7f0fb10700 1 -- 172.21.15.144:0/1558817396 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f0804ea50 con 0x7f7f080ff390 2024-09-10T11:58:33.161 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.158+0000 7f7efaffd700 1 -- 172.21.15.144:0/1558817396 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7efc022470 con 0x7f7f080ff390 2024-09-10T11:58:33.161 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.158+0000 7f7efaffd700 1 --2- 172.21.15.144:0/1558817396 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7ef406e200 0x7f7ef40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:33.161 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.159+0000 7f7efaffd700 1 -- 172.21.15.144:0/1558817396 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7efc08f670 con 0x7f7f080ff390 2024-09-10T11:58:33.161 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.159+0000 7f7f0d0ab700 1 --2- 172.21.15.144:0/1558817396 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7ef406e200 0x7f7ef40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:33.161 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.159+0000 7f7f0d0ab700 1 --2- 172.21.15.144:0/1558817396 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7ef406e200 0x7f7ef40706b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7f04005fd0 tx=0x7f7f04005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:33.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.163+0000 7f7efaffd700 1 -- 172.21.15.144:0/1558817396 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7efc058aa0 con 0x7f7f080ff390 2024-09-10T11:58:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:32 smithi178 ceph-mon[65940]: pgmap v798: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:33.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.384+0000 7f7f0fb10700 1 -- 172.21.15.144:0/1558817396 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f0819d450 con 0x7f7ef406e200 2024-09-10T11:58:33.386 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.385+0000 7f7efaffd700 1 -- 172.21.15.144:0/1558817396 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f7f0819d450 con 0x7f7ef406e200 2024-09-10T11:58:33.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.387+0000 7f7f0fb10700 1 -- 172.21.15.144:0/1558817396 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7ef406e200 msgr2=0x7f7ef40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:33.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.387+0000 7f7f0fb10700 1 --2- 172.21.15.144:0/1558817396 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7ef406e200 0x7f7ef40706b0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f7f04005fd0 tx=0x7f7f04005e20 comp rx=0 tx=0).stop 2024-09-10T11:58:33.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.387+0000 7f7f0fb10700 1 -- 172.21.15.144:0/1558817396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f080ff390 msgr2=0x7f7f08197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:33.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.387+0000 7f7f0fb10700 1 --2- 172.21.15.144:0/1558817396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f080ff390 0x7f7f08197fa0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7efc006010 tx=0x7f7efc00be20 comp rx=0 tx=0).stop 2024-09-10T11:58:33.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.387+0000 7f7f0fb10700 1 -- 172.21.15.144:0/1558817396 shutdown_connections 2024-09-10T11:58:33.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.387+0000 7f7f0fb10700 1 --2- 172.21.15.144:0/1558817396 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7ef406e200 0x7f7ef40706b0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.387+0000 7f7f0fb10700 1 --2- 172.21.15.144:0/1558817396 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f080ff390 0x7f7f08197fa0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.387+0000 7f7f0fb10700 1 --2- 172.21.15.144:0/1558817396 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f080ffd30 0x7f7f081984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.387+0000 7f7f0fb10700 1 -- 172.21.15.144:0/1558817396 >> 172.21.15.144:0/1558817396 conn(0x7f7f08074f50 msgr2=0x7f7f080fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:33.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.387+0000 7f7f0fb10700 1 -- 172.21.15.144:0/1558817396 shutdown_connections 2024-09-10T11:58:33.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.387+0000 7f7f0fb10700 1 -- 172.21.15.144:0/1558817396 wait complete. 2024-09-10T11:58:33.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.492+0000 7f81588b3700 1 -- 172.21.15.144:0/1938961805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81540ffd20 msgr2=0x7f8154103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:33.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.492+0000 7f81588b3700 1 --2- 172.21.15.144:0/1938961805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81540ffd20 0x7f8154103cf0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f8144009910 tx=0x7f8144009c20 comp rx=0 tx=0).stop 2024-09-10T11:58:33.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.492+0000 7f81588b3700 1 -- 172.21.15.144:0/1938961805 shutdown_connections 2024-09-10T11:58:33.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.492+0000 7f81588b3700 1 --2- 172.21.15.144:0/1938961805 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81540ffd20 0x7f8154103cf0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.492+0000 7f81588b3700 1 --2- 172.21.15.144:0/1938961805 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81540ff380 0x7f81540ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.492+0000 7f81588b3700 1 -- 172.21.15.144:0/1938961805 >> 172.21.15.144:0/1938961805 conn(0x7f81540fac20 msgr2=0x7f81540fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:33.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.492+0000 7f81588b3700 1 -- 172.21.15.144:0/1938961805 shutdown_connections 2024-09-10T11:58:33.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.492+0000 7f81588b3700 1 -- 172.21.15.144:0/1938961805 wait complete. 2024-09-10T11:58:33.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.493+0000 7f81588b3700 1 Processor -- start 2024-09-10T11:58:33.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.493+0000 7f81588b3700 1 -- start start 2024-09-10T11:58:33.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.493+0000 7f81588b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81540ff380 0x7f8154196c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:33.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.493+0000 7f81588b3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81540ffd20 0x7f8154197190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:33.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.493+0000 7f81588b3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81541977b0 con 0x7f81540ff380 2024-09-10T11:58:33.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.493+0000 7f81588b3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81541978f0 con 0x7f81540ffd20 2024-09-10T11:58:33.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.493+0000 7f815259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81540ff380 0x7f8154196c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:33.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.493+0000 7f815259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81540ff380 0x7f8154196c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39988/0 (socket says 172.21.15.144:39988) 2024-09-10T11:58:33.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.493+0000 7f815259c700 1 -- 172.21.15.144:0/3367667761 learned_addr learned my addr 172.21.15.144:0/3367667761 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:58:33.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.493+0000 7f8151d9b700 1 --2- 172.21.15.144:0/3367667761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81540ffd20 0x7f8154197190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:33.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.494+0000 7f815259c700 1 -- 172.21.15.144:0/3367667761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81540ffd20 msgr2=0x7f8154197190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:33.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.494+0000 7f815259c700 1 --2- 172.21.15.144:0/3367667761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81540ffd20 0x7f8154197190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.494+0000 7f815259c700 1 -- 172.21.15.144:0/3367667761 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81440095f0 con 0x7f81540ff380 2024-09-10T11:58:33.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.494+0000 7f8151d9b700 1 --2- 172.21.15.144:0/3367667761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81540ffd20 0x7f8154197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:58:33.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.494+0000 7f815259c700 1 --2- 172.21.15.144:0/3367667761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81540ff380 0x7f8154196c50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f814800d8d0 tx=0x7f814800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:33.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.494+0000 7f81437fe700 1 -- 172.21.15.144:0/3367667761 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8148009940 con 0x7f81540ff380 2024-09-10T11:58:33.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.494+0000 7f81437fe700 1 -- 172.21.15.144:0/3367667761 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8148010460 con 0x7f81540ff380 2024-09-10T11:58:33.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.494+0000 7f81588b3700 1 -- 172.21.15.144:0/3367667761 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815419fcc0 con 0x7f81540ff380 2024-09-10T11:58:33.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.494+0000 7f81437fe700 1 -- 172.21.15.144:0/3367667761 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f814800f5f0 con 0x7f81540ff380 2024-09-10T11:58:33.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.494+0000 7f81588b3700 1 -- 172.21.15.144:0/3367667761 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81541a0180 con 0x7f81540ff380 2024-09-10T11:58:33.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.495+0000 7f81588b3700 1 -- 172.21.15.144:0/3367667761 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81541076f0 con 0x7f81540ff380 2024-09-10T11:58:33.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.496+0000 7f81437fe700 1 -- 172.21.15.144:0/3367667761 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f81480105d0 con 0x7f81540ff380 2024-09-10T11:58:33.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.496+0000 7f81437fe700 1 --2- 172.21.15.144:0/3367667761 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f813c06e200 0x7f813c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:33.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.496+0000 7f81437fe700 1 -- 172.21.15.144:0/3367667761 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f814808dfe0 con 0x7f81540ff380 2024-09-10T11:58:33.499 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.496+0000 7f8151d9b700 1 --2- 172.21.15.144:0/3367667761 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f813c06e200 0x7f813c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:33.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.497+0000 7f8151d9b700 1 --2- 172.21.15.144:0/3367667761 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f813c06e200 0x7f813c0706b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8144006010 tx=0x7f814401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:33.502 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.501+0000 7f81437fe700 1 -- 172.21.15.144:0/3367667761 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8148057440 con 0x7f81540ff380 2024-09-10T11:58:33.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.716+0000 7f81588b3700 1 -- 172.21.15.144:0/3367667761 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8154061190 con 0x7f813c06e200 2024-09-10T11:58:33.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.725+0000 7f81437fe700 1 -- 172.21.15.144:0/3367667761 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f8154061190 con 0x7f813c06e200 2024-09-10T11:58:33.727 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:58:33.727 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (26m) 5m ago 37m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:58:33.727 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (37m) 5m ago 37m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:58:33.727 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (36m) 5m ago 36m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:58:33.727 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (24m) 5m ago 37m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:58:33.727 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (24m) 5m ago 36m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:58:33.727 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (26m) 5m ago 37m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:58:33.727 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (20m) 5m ago 31m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:58:33.727 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (20m) 5m ago 31m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:58:33.728 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (5m) 5m ago 31m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T11:58:33.728 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (20m) 5m ago 31m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:58:33.728 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (28m) 5m ago 38m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:58:33.728 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (27m) 5m ago 36m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:58:33.728 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (25m) 5m ago 39m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:58:33.728 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (25m) 5m ago 36m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:58:33.728 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (27m) 5m ago 37m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:58:33.728 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (27m) 5m ago 36m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:58:33.728 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (24m) 5m ago 35m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:58:33.728 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (23m) 5m ago 35m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:58:33.728 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (22m) 5m ago 34m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:58:33.728 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (22m) 5m ago 34m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:58:33.728 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (21m) 5m ago 33m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:58:33.729 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (21m) 5m ago 33m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:58:33.729 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (27m) 5m ago 36m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:58:33.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.728+0000 7f81588b3700 1 -- 172.21.15.144:0/3367667761 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f813c06e200 msgr2=0x7f813c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:33.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.728+0000 7f81588b3700 1 --2- 172.21.15.144:0/3367667761 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f813c06e200 0x7f813c0706b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f8144006010 tx=0x7f814401a040 comp rx=0 tx=0).stop 2024-09-10T11:58:33.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.728+0000 7f81588b3700 1 -- 172.21.15.144:0/3367667761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81540ff380 msgr2=0x7f8154196c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:33.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.728+0000 7f81588b3700 1 --2- 172.21.15.144:0/3367667761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81540ff380 0x7f8154196c50 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f814800d8d0 tx=0x7f814800dc90 comp rx=0 tx=0).stop 2024-09-10T11:58:33.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.728+0000 7f81588b3700 1 -- 172.21.15.144:0/3367667761 shutdown_connections 2024-09-10T11:58:33.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.728+0000 7f81588b3700 1 --2- 172.21.15.144:0/3367667761 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f813c06e200 0x7f813c0706b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.728+0000 7f81588b3700 1 --2- 172.21.15.144:0/3367667761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f81540ff380 0x7f8154196c50 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.728+0000 7f81588b3700 1 --2- 172.21.15.144:0/3367667761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f81540ffd20 0x7f8154197190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.728+0000 7f81588b3700 1 -- 172.21.15.144:0/3367667761 >> 172.21.15.144:0/3367667761 conn(0x7f81540fac20 msgr2=0x7f81540fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:33.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.728+0000 7f81588b3700 1 -- 172.21.15.144:0/3367667761 shutdown_connections 2024-09-10T11:58:33.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.728+0000 7f81588b3700 1 -- 172.21.15.144:0/3367667761 wait complete. 2024-09-10T11:58:33.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.831+0000 7f2a544b7700 1 -- 172.21.15.144:0/693236237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a4c0fef40 msgr2=0x7f2a4c0ff310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:33.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.831+0000 7f2a544b7700 1 --2- 172.21.15.144:0/693236237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a4c0fef40 0x7f2a4c0ff310 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f2a48009910 tx=0x7f2a48009c20 comp rx=0 tx=0).stop 2024-09-10T11:58:33.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.831+0000 7f2a544b7700 1 -- 172.21.15.144:0/693236237 shutdown_connections 2024-09-10T11:58:33.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.831+0000 7f2a544b7700 1 --2- 172.21.15.144:0/693236237 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a4c0ff850 0x7f2a4c10c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.831+0000 7f2a544b7700 1 --2- 172.21.15.144:0/693236237 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a4c0fef40 0x7f2a4c0ff310 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.831+0000 7f2a544b7700 1 -- 172.21.15.144:0/693236237 >> 172.21.15.144:0/693236237 conn(0x7f2a4c0fabb0 msgr2=0x7f2a4c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:33.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.831+0000 7f2a544b7700 1 -- 172.21.15.144:0/693236237 shutdown_connections 2024-09-10T11:58:33.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.831+0000 7f2a544b7700 1 -- 172.21.15.144:0/693236237 wait complete. 2024-09-10T11:58:33.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.831+0000 7f2a544b7700 1 Processor -- start 2024-09-10T11:58:33.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a544b7700 1 -- start start 2024-09-10T11:58:33.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a544b7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a4c0fef40 0x7f2a4c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:33.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a544b7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a4c0ff850 0x7f2a4c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:33.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a544b7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c198b60 con 0x7f2a4c0ff850 2024-09-10T11:58:33.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a544b7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4c19c8f0 con 0x7f2a4c0fef40 2024-09-10T11:58:33.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a51a52700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a4c0ff850 0x7f2a4c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:33.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a51a52700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a4c0ff850 0x7f2a4c198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40008/0 (socket says 172.21.15.144:40008) 2024-09-10T11:58:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a51a52700 1 -- 172.21.15.144:0/3135438002 learned_addr learned my addr 172.21.15.144:0/3135438002 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:58:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a52253700 1 --2- 172.21.15.144:0/3135438002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a4c0fef40 0x7f2a4c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a51a52700 1 -- 172.21.15.144:0/3135438002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a4c0fef40 msgr2=0x7f2a4c197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a51a52700 1 --2- 172.21.15.144:0/3135438002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a4c0fef40 0x7f2a4c197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a51a52700 1 -- 172.21.15.144:0/3135438002 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a480095f0 con 0x7f2a4c0ff850 2024-09-10T11:58:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a51a52700 1 --2- 172.21.15.144:0/3135438002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a4c0ff850 0x7f2a4c198480 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00c8a0 tx=0x7f2a3c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a437fe700 1 -- 172.21.15.144:0/3135438002 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3c00cea0 con 0x7f2a4c0ff850 2024-09-10T11:58:33.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a437fe700 1 -- 172.21.15.144:0/3135438002 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a3c004830 con 0x7f2a4c0ff850 2024-09-10T11:58:33.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.832+0000 7f2a544b7700 1 -- 172.21.15.144:0/3135438002 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4c19cbd0 con 0x7f2a4c0ff850 2024-09-10T11:58:33.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.833+0000 7f2a437fe700 1 -- 172.21.15.144:0/3135438002 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a3c0055b0 con 0x7f2a4c0ff850 2024-09-10T11:58:33.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.833+0000 7f2a544b7700 1 -- 172.21.15.144:0/3135438002 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4c19d120 con 0x7f2a4c0ff850 2024-09-10T11:58:33.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.834+0000 7f2a544b7700 1 -- 172.21.15.144:0/3135438002 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a4c109d20 con 0x7f2a4c0ff850 2024-09-10T11:58:33.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.834+0000 7f2a437fe700 1 -- 172.21.15.144:0/3135438002 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2a3c0049a0 con 0x7f2a4c0ff850 2024-09-10T11:58:33.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.835+0000 7f2a437fe700 1 --2- 172.21.15.144:0/3135438002 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2a3806e200 0x7f2a380706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:33.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.835+0000 7f2a52253700 1 --2- 172.21.15.144:0/3135438002 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2a3806e200 0x7f2a380706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:33.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.835+0000 7f2a437fe700 1 -- 172.21.15.144:0/3135438002 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2a3c08dd40 con 0x7f2a4c0ff850 2024-09-10T11:58:33.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.835+0000 7f2a52253700 1 --2- 172.21.15.144:0/3135438002 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2a3806e200 0x7f2a380706b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800b5c0 tx=0x7f2a4801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:33.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:33.839+0000 7f2a437fe700 1 -- 172.21.15.144:0/3135438002 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2a3c057120 con 0x7f2a4c0ff850 2024-09-10T11:58:34.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.129+0000 7f2a544b7700 1 -- 172.21.15.144:0/3135438002 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2a4c04ea50 con 0x7f2a4c0ff850 2024-09-10T11:58:34.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.130+0000 7f2a437fe700 1 -- 172.21.15.144:0/3135438002 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f2a3c01d290 con 0x7f2a4c0ff850 2024-09-10T11:58:34.131 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:58:34.131 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:58:34.131 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:58:34.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:58:34.132 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:58:34.132 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:58:34.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:58:34.132 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:58:34.132 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:58:34.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:58:34.132 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:58:34.132 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:58:34.132 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:58:34.133 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:58:34.133 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T11:58:34.133 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:58:34.133 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:58:34.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.132+0000 7f2a544b7700 1 -- 172.21.15.144:0/3135438002 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2a3806e200 msgr2=0x7f2a380706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:34.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.132+0000 7f2a544b7700 1 --2- 172.21.15.144:0/3135438002 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2a3806e200 0x7f2a380706b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800b5c0 tx=0x7f2a4801a040 comp rx=0 tx=0).stop 2024-09-10T11:58:34.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.132+0000 7f2a544b7700 1 -- 172.21.15.144:0/3135438002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a4c0ff850 msgr2=0x7f2a4c198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:34.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.132+0000 7f2a544b7700 1 --2- 172.21.15.144:0/3135438002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a4c0ff850 0x7f2a4c198480 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2a3c00c8a0 tx=0x7f2a3c00cc60 comp rx=0 tx=0).stop 2024-09-10T11:58:34.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.132+0000 7f2a544b7700 1 -- 172.21.15.144:0/3135438002 shutdown_connections 2024-09-10T11:58:34.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.132+0000 7f2a544b7700 1 --2- 172.21.15.144:0/3135438002 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2a3806e200 0x7f2a380706b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.132+0000 7f2a544b7700 1 --2- 172.21.15.144:0/3135438002 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2a4c0fef40 0x7f2a4c197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.134 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.132+0000 7f2a544b7700 1 --2- 172.21.15.144:0/3135438002 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2a4c0ff850 0x7f2a4c198480 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.132+0000 7f2a544b7700 1 -- 172.21.15.144:0/3135438002 >> 172.21.15.144:0/3135438002 conn(0x7f2a4c0fabb0 msgr2=0x7f2a4c0fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:34.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.132+0000 7f2a544b7700 1 -- 172.21.15.144:0/3135438002 shutdown_connections 2024-09-10T11:58:34.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.132+0000 7f2a544b7700 1 -- 172.21.15.144:0/3135438002 wait complete. 2024-09-10T11:58:34.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.237+0000 7f6851b5b700 1 -- 172.21.15.144:0/1308352450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c068fd0 msgr2=0x7f684c104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:34.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.237+0000 7f6851b5b700 1 --2- 172.21.15.144:0/1308352450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c068fd0 0x7f684c104df0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f6840009910 tx=0x7f6840009c20 comp rx=0 tx=0).stop 2024-09-10T11:58:34.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.237+0000 7f6851b5b700 1 -- 172.21.15.144:0/1308352450 shutdown_connections 2024-09-10T11:58:34.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.237+0000 7f6851b5b700 1 --2- 172.21.15.144:0/1308352450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c068fd0 0x7f684c104df0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.237+0000 7f6851b5b700 1 --2- 172.21.15.144:0/1308352450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f684c068630 0x7f684c068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.237+0000 7f6851b5b700 1 -- 172.21.15.144:0/1308352450 >> 172.21.15.144:0/1308352450 conn(0x7f684c074f50 msgr2=0x7f684c075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:34.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.237+0000 7f6851b5b700 1 -- 172.21.15.144:0/1308352450 shutdown_connections 2024-09-10T11:58:34.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.237+0000 7f6851b5b700 1 -- 172.21.15.144:0/1308352450 wait complete. 2024-09-10T11:58:34.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.237+0000 7f6851b5b700 1 Processor -- start 2024-09-10T11:58:34.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.238+0000 7f6851b5b700 1 -- start start 2024-09-10T11:58:34.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.238+0000 7f6851b5b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c068630 0x7f684c195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:34.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.238+0000 7f6851b5b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f684c068fd0 0x7f684c196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:34.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.238+0000 7f6851b5b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c1969b0 con 0x7f684c068630 2024-09-10T11:58:34.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.238+0000 7f6851b5b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684c196af0 con 0x7f684c068fd0 2024-09-10T11:58:34.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.238+0000 7f684b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c068630 0x7f684c195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:34.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.238+0000 7f684b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c068630 0x7f684c195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40014/0 (socket says 172.21.15.144:40014) 2024-09-10T11:58:34.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.238+0000 7f684b7fe700 1 -- 172.21.15.144:0/2036752073 learned_addr learned my addr 172.21.15.144:0/2036752073 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:58:34.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.238+0000 7f684b7fe700 1 -- 172.21.15.144:0/2036752073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f684c068fd0 msgr2=0x7f684c196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:34.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.238+0000 7f684b7fe700 1 --2- 172.21.15.144:0/2036752073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f684c068fd0 0x7f684c196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.238+0000 7f684b7fe700 1 -- 172.21.15.144:0/2036752073 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68400095f0 con 0x7f684c068630 2024-09-10T11:58:34.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.238+0000 7f684b7fe700 1 --2- 172.21.15.144:0/2036752073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c068630 0x7f684c195e20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f683c00cfd0 tx=0x7f683c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:34.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.239+0000 7f6848ff9700 1 -- 172.21.15.144:0/2036752073 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f683c00ad70 con 0x7f684c068630 2024-09-10T11:58:34.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.239+0000 7f6848ff9700 1 -- 172.21.15.144:0/2036752073 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f683c0128e0 con 0x7f684c068630 2024-09-10T11:58:34.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.239+0000 7f6851b5b700 1 -- 172.21.15.144:0/2036752073 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f684c19a940 con 0x7f684c068630 2024-09-10T11:58:34.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.239+0000 7f6848ff9700 1 -- 172.21.15.144:0/2036752073 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f683c011880 con 0x7f684c068630 2024-09-10T11:58:34.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.239+0000 7f6851b5b700 1 -- 172.21.15.144:0/2036752073 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f684c19ae90 con 0x7f684c068630 2024-09-10T11:58:34.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.240+0000 7f6851b5b700 1 -- 172.21.15.144:0/2036752073 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f684c1086d0 con 0x7f684c068630 2024-09-10T11:58:34.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.240+0000 7f6848ff9700 1 -- 172.21.15.144:0/2036752073 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f683c012420 con 0x7f684c068630 2024-09-10T11:58:34.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.241+0000 7f6848ff9700 1 --2- 172.21.15.144:0/2036752073 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f683406e200 0x7f68340706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:34.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.241+0000 7f6848ff9700 1 -- 172.21.15.144:0/2036752073 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f683c08e500 con 0x7f684c068630 2024-09-10T11:58:34.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.241+0000 7f684affd700 1 --2- 172.21.15.144:0/2036752073 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f683406e200 0x7f68340706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:34.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.241+0000 7f684affd700 1 --2- 172.21.15.144:0/2036752073 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f683406e200 0x7f68340706b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f684000b5c0 tx=0x7f6840005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:34.246 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.245+0000 7f6848ff9700 1 -- 172.21.15.144:0/2036752073 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f683c0578e0 con 0x7f684c068630 2024-09-10T11:58:34.246 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:33 smithi144 ceph-mon[87436]: from='client.35464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:34.246 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:33 smithi144 ceph-mon[87436]: from='client.35468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:34.246 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:33 smithi144 ceph-mon[87436]: from='client.35472 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:34.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:33 smithi178 ceph-mon[65940]: from='client.35464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:34.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:33 smithi178 ceph-mon[65940]: from='client.35468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:34.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:33 smithi178 ceph-mon[65940]: from='client.35472 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:34.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.490+0000 7f6851b5b700 1 -- 172.21.15.144:0/2036752073 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f684c02cc30 con 0x7f684c068630 2024-09-10T11:58:34.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.490+0000 7f6848ff9700 1 -- 172.21.15.144:0/2036752073 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f683c05afa0 con 0x7f684c068630 2024-09-10T11:58:34.492 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T11:58:34.492 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T11:58:34.492 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:58:34.492 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:58:34.492 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:58:34.492 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:58:34.492 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:58:34.492 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:58:34.492 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:58:34.492 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:58:34.492 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:58:34.493 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:58:34.493 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:58:34.493 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:58:34.493 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:58:34.493 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:58:34.493 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:58:34.493 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:58:34.493 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:58:34.493 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:58:34.493 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:58:34.493 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:58:34.493 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:58:34.493 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:58:34.493 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:58:34.494 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:58:34.494 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:58:34.494 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:58:34.494 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:58:34.494 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:58:34.494 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:58:34.494 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:58:34.494 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:58:34.494 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:58:34.494 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:58:34.494 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:58:34.494 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:58:34.494 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:58:34.494 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:58:34.495 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:58:34.495 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:58:34.495 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:58:34.495 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:58:34.495 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.493+0000 7f6851b5b700 1 -- 172.21.15.144:0/2036752073 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f683406e200 msgr2=0x7f68340706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:34.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.493+0000 7f6851b5b700 1 --2- 172.21.15.144:0/2036752073 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f683406e200 0x7f68340706b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f684000b5c0 tx=0x7f6840005960 comp rx=0 tx=0).stop 2024-09-10T11:58:34.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.493+0000 7f6851b5b700 1 -- 172.21.15.144:0/2036752073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c068630 msgr2=0x7f684c195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:34.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.493+0000 7f6851b5b700 1 --2- 172.21.15.144:0/2036752073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c068630 0x7f684c195e20 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f683c00cfd0 tx=0x7f683c00a340 comp rx=0 tx=0).stop 2024-09-10T11:58:34.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.493+0000 7f6851b5b700 1 -- 172.21.15.144:0/2036752073 shutdown_connections 2024-09-10T11:58:34.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.493+0000 7f6851b5b700 1 --2- 172.21.15.144:0/2036752073 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f683406e200 0x7f68340706b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.493+0000 7f6851b5b700 1 --2- 172.21.15.144:0/2036752073 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f684c068630 0x7f684c195e20 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.493+0000 7f6851b5b700 1 --2- 172.21.15.144:0/2036752073 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f684c068fd0 0x7f684c196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.493+0000 7f6851b5b700 1 -- 172.21.15.144:0/2036752073 >> 172.21.15.144:0/2036752073 conn(0x7f684c074f50 msgr2=0x7f684c0fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:34.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.493+0000 7f6851b5b700 1 -- 172.21.15.144:0/2036752073 shutdown_connections 2024-09-10T11:58:34.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.493+0000 7f6851b5b700 1 -- 172.21.15.144:0/2036752073 wait complete. 2024-09-10T11:58:34.496 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T11:58:34.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.598+0000 7f0696419700 1 -- 172.21.15.144:0/563134088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06900730f0 msgr2=0x7f06900734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:34.600 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.598+0000 7f0696419700 1 --2- 172.21.15.144:0/563134088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06900730f0 0x7f06900734c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f0680009910 tx=0x7f0680009c20 comp rx=0 tx=0).stop 2024-09-10T11:58:34.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.598+0000 7f0696419700 1 -- 172.21.15.144:0/563134088 shutdown_connections 2024-09-10T11:58:34.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.598+0000 7f0696419700 1 --2- 172.21.15.144:0/563134088 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0690073a00 0x7f0690110c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.598+0000 7f0696419700 1 --2- 172.21.15.144:0/563134088 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f06900730f0 0x7f06900734c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.598+0000 7f0696419700 1 -- 172.21.15.144:0/563134088 >> 172.21.15.144:0/563134088 conn(0x7f06900fbcf0 msgr2=0x7f06900fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:34.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.599+0000 7f0696419700 1 -- 172.21.15.144:0/563134088 shutdown_connections 2024-09-10T11:58:34.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.599+0000 7f0696419700 1 -- 172.21.15.144:0/563134088 wait complete. 2024-09-10T11:58:34.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.599+0000 7f0696419700 1 Processor -- start 2024-09-10T11:58:34.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.599+0000 7f0696419700 1 -- start start 2024-09-10T11:58:34.604 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.600+0000 7f0696419700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06900730f0 0x7f069019c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:34.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.600+0000 7f0696419700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0690073a00 0x7f069019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:34.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.600+0000 7f0696419700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f069019cfb0 con 0x7f0690073a00 2024-09-10T11:58:34.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.600+0000 7f0696419700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06901a0d40 con 0x7f06900730f0 2024-09-10T11:58:34.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.600+0000 7f068f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0690073a00 0x7f069019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:34.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.600+0000 7f068f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0690073a00 0x7f069019c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40026/0 (socket says 172.21.15.144:40026) 2024-09-10T11:58:34.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.600+0000 7f068f7fe700 1 -- 172.21.15.144:0/4194390825 learned_addr learned my addr 172.21.15.144:0/4194390825 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:58:34.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.600+0000 7f068ffff700 1 --2- 172.21.15.144:0/4194390825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06900730f0 0x7f069019c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:34.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.600+0000 7f068f7fe700 1 -- 172.21.15.144:0/4194390825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06900730f0 msgr2=0x7f069019c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:34.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.600+0000 7f068f7fe700 1 --2- 172.21.15.144:0/4194390825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06900730f0 0x7f069019c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.600+0000 7f068f7fe700 1 -- 172.21.15.144:0/4194390825 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06800095f0 con 0x7f0690073a00 2024-09-10T11:58:34.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.600+0000 7f068f7fe700 1 --2- 172.21.15.144:0/4194390825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0690073a00 0x7f069019c8d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f068400b4b0 tx=0x7f068400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:34.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.600+0000 7f068d7fa700 1 -- 172.21.15.144:0/4194390825 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0684011800 con 0x7f0690073a00 2024-09-10T11:58:34.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.600+0000 7f0696419700 1 -- 172.21.15.144:0/4194390825 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06901a1020 con 0x7f0690073a00 2024-09-10T11:58:34.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.601+0000 7f068d7fa700 1 -- 172.21.15.144:0/4194390825 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0684011e20 con 0x7f0690073a00 2024-09-10T11:58:34.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.601+0000 7f068d7fa700 1 -- 172.21.15.144:0/4194390825 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f068400f2e0 con 0x7f0690073a00 2024-09-10T11:58:34.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.601+0000 7f0696419700 1 -- 172.21.15.144:0/4194390825 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06901a1570 con 0x7f0690073a00 2024-09-10T11:58:34.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.602+0000 7f0696419700 1 -- 172.21.15.144:0/4194390825 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f069010e400 con 0x7f0690073a00 2024-09-10T11:58:34.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.602+0000 7f068d7fa700 1 -- 172.21.15.144:0/4194390825 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0684010380 con 0x7f0690073a00 2024-09-10T11:58:34.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.603+0000 7f068d7fa700 1 --2- 172.21.15.144:0/4194390825 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f067806e200 0x7f06780706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:34.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.603+0000 7f068d7fa700 1 -- 172.21.15.144:0/4194390825 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f068408dba0 con 0x7f0690073a00 2024-09-10T11:58:34.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.603+0000 7f068ffff700 1 --2- 172.21.15.144:0/4194390825 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f067806e200 0x7f06780706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:34.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.603+0000 7f068ffff700 1 --2- 172.21.15.144:0/4194390825 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f067806e200 0x7f06780706b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f068000b5c0 tx=0x7f068001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:34.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.607+0000 7f068d7fa700 1 -- 172.21.15.144:0/4194390825 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0684057000 con 0x7f0690073a00 2024-09-10T11:58:34.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.828+0000 7f0696419700 1 -- 172.21.15.144:0/4194390825 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06901a1900 con 0x7f067806e200 2024-09-10T11:58:34.830 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.829+0000 7f068d7fa700 1 -- 172.21.15.144:0/4194390825 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f06901a1900 con 0x7f067806e200 2024-09-10T11:58:34.830 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:58:34.830 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:58:34.830 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:58:34.830 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:58:34.830 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:58:34.830 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:58:34.830 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T11:58:34.831 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T11:58:34.831 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:58:34.831 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T11:58:34.831 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:58:34.831 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T11:58:34.831 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:58:34.831 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:58:34.831 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:58:34.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.831+0000 7f0696419700 1 -- 172.21.15.144:0/4194390825 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f067806e200 msgr2=0x7f06780706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:34.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.831+0000 7f0696419700 1 --2- 172.21.15.144:0/4194390825 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f067806e200 0x7f06780706b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f068000b5c0 tx=0x7f068001a040 comp rx=0 tx=0).stop 2024-09-10T11:58:34.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.831+0000 7f0696419700 1 -- 172.21.15.144:0/4194390825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0690073a00 msgr2=0x7f069019c8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:34.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.831+0000 7f0696419700 1 --2- 172.21.15.144:0/4194390825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0690073a00 0x7f069019c8d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f068400b4b0 tx=0x7f068400b870 comp rx=0 tx=0).stop 2024-09-10T11:58:34.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.831+0000 7f0696419700 1 -- 172.21.15.144:0/4194390825 shutdown_connections 2024-09-10T11:58:34.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.831+0000 7f0696419700 1 --2- 172.21.15.144:0/4194390825 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f067806e200 0x7f06780706b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.831+0000 7f0696419700 1 --2- 172.21.15.144:0/4194390825 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f06900730f0 0x7f069019c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.833 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.831+0000 7f0696419700 1 --2- 172.21.15.144:0/4194390825 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0690073a00 0x7f069019c8d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.831+0000 7f0696419700 1 -- 172.21.15.144:0/4194390825 >> 172.21.15.144:0/4194390825 conn(0x7f06900fbcf0 msgr2=0x7f06901027b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:34.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.831+0000 7f0696419700 1 -- 172.21.15.144:0/4194390825 shutdown_connections 2024-09-10T11:58:34.834 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.831+0000 7f0696419700 1 -- 172.21.15.144:0/4194390825 wait complete. 2024-09-10T11:58:34.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.936+0000 7f38e6e97700 1 -- 172.21.15.144:0/3019425473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0103860 msgr2=0x7f38e0103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:34.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.936+0000 7f38e6e97700 1 --2- 172.21.15.144:0/3019425473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0103860 0x7f38e0103c30 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f38d400aa20 tx=0x7f38d400ad30 comp rx=0 tx=0).stop 2024-09-10T11:58:34.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.936+0000 7f38e6e97700 1 -- 172.21.15.144:0/3019425473 shutdown_connections 2024-09-10T11:58:34.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.936+0000 7f38e6e97700 1 --2- 172.21.15.144:0/3019425473 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e00ff320 0x7f38e00ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.936+0000 7f38e6e97700 1 --2- 172.21.15.144:0/3019425473 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0103860 0x7f38e0103c30 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.936+0000 7f38e6e97700 1 -- 172.21.15.144:0/3019425473 >> 172.21.15.144:0/3019425473 conn(0x7f38e00748b0 msgr2=0x7f38e0074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:34.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.936+0000 7f38e6e97700 1 -- 172.21.15.144:0/3019425473 shutdown_connections 2024-09-10T11:58:34.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.936+0000 7f38e6e97700 1 -- 172.21.15.144:0/3019425473 wait complete. 2024-09-10T11:58:34.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.936+0000 7f38e6e97700 1 Processor -- start 2024-09-10T11:58:34.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.936+0000 7f38e6e97700 1 -- start start 2024-09-10T11:58:34.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38e6e97700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e00ff320 0x7f38e0195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:34.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38e6e97700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0103860 0x7f38e01962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:34.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38e6e97700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e0196990 con 0x7f38e0103860 2024-09-10T11:58:34.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38e6e97700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e019a720 con 0x7f38e00ff320 2024-09-10T11:58:34.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38dffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0103860 0x7f38e01962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:34.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38dffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0103860 0x7f38e01962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40042/0 (socket says 172.21.15.144:40042) 2024-09-10T11:58:34.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38dffff700 1 -- 172.21.15.144:0/2628177760 learned_addr learned my addr 172.21.15.144:0/2628177760 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:58:34.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38e4c33700 1 --2- 172.21.15.144:0/2628177760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e00ff320 0x7f38e0195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:34.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38dffff700 1 -- 172.21.15.144:0/2628177760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e00ff320 msgr2=0x7f38e0195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:34.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38dffff700 1 --2- 172.21.15.144:0/2628177760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e00ff320 0x7f38e0195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:34.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38dffff700 1 -- 172.21.15.144:0/2628177760 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d400a700 con 0x7f38e0103860 2024-09-10T11:58:34.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38dffff700 1 --2- 172.21.15.144:0/2628177760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0103860 0x7f38e01962b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c8a0 tx=0x7f38cc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:34.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38ddffb700 1 -- 172.21.15.144:0/2628177760 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38cc0078e0 con 0x7f38e0103860 2024-09-10T11:58:34.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38ddffb700 1 -- 172.21.15.144:0/2628177760 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f38cc00f450 con 0x7f38e0103860 2024-09-10T11:58:34.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38e6e97700 1 -- 172.21.15.144:0/2628177760 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38e019aa00 con 0x7f38e0103860 2024-09-10T11:58:34.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.937+0000 7f38ddffb700 1 -- 172.21.15.144:0/2628177760 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38cc00e5e0 con 0x7f38e0103860 2024-09-10T11:58:34.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.938+0000 7f38e6e97700 1 -- 172.21.15.144:0/2628177760 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e019af50 con 0x7f38e0103860 2024-09-10T11:58:34.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.939+0000 7f38e6e97700 1 -- 172.21.15.144:0/2628177760 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e0108e30 con 0x7f38e0103860 2024-09-10T11:58:34.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.939+0000 7f38ddffb700 1 -- 172.21.15.144:0/2628177760 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f38cc00f5c0 con 0x7f38e0103860 2024-09-10T11:58:34.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.939+0000 7f38ddffb700 1 --2- 172.21.15.144:0/2628177760 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38d006e200 0x7f38d00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:58:34.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.940+0000 7f38ddffb700 1 -- 172.21.15.144:0/2628177760 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f38cc08e0b0 con 0x7f38e0103860 2024-09-10T11:58:34.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.940+0000 7f38e4c33700 1 --2- 172.21.15.144:0/2628177760 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38d006e200 0x7f38d00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:58:34.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.940+0000 7f38e4c33700 1 --2- 172.21.15.144:0/2628177760 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38d006e200 0x7f38d00706b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f38d4000c00 tx=0x7f38d4005de0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:58:34.945 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:34.944+0000 7f38ddffb700 1 -- 172.21.15.144:0/2628177760 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f38cc057490 con 0x7f38e0103860 2024-09-10T11:58:35.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:35.230+0000 7f38e6e97700 1 -- 172.21.15.144:0/2628177760 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f38e004ea50 con 0x7f38e0103860 2024-09-10T11:58:35.232 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:35 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3135438002' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:58:35.232 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:35 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2036752073' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:58:35.232 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:35 smithi144 ceph-mon[87436]: from='client.35484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:35.232 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:35 smithi144 ceph-mon[87436]: pgmap v799: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:35.232 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:35.230+0000 7f38ddffb700 1 -- 172.21.15.144:0/2628177760 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f38cc05ab50 con 0x7f38e0103860 2024-09-10T11:58:35.232 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:58:35.233 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T11:58:35.233 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:58:35.233 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T11:58:35.233 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:58:35.233 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T11:58:35.233 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T11:58:35.233 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T11:58:35.233 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T11:58:35.233 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T11:58:35.233 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T11:58:35.233 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T11:58:35.233 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T11:58:35.233 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T11:58:35.234 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T11:58:35.234 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T11:58:35.234 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T11:58:35.234 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T11:58:35.234 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:58:35.234 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:58:35.234 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:58:35.234 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:58:35.234 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:58:35.234 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:58:35.234 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:58:35.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:35.233+0000 7f38e6e97700 1 -- 172.21.15.144:0/2628177760 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38d006e200 msgr2=0x7f38d00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:35.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:35.233+0000 7f38e6e97700 1 --2- 172.21.15.144:0/2628177760 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38d006e200 0x7f38d00706b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f38d4000c00 tx=0x7f38d4005de0 comp rx=0 tx=0).stop 2024-09-10T11:58:35.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:35.233+0000 7f38e6e97700 1 -- 172.21.15.144:0/2628177760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0103860 msgr2=0x7f38e01962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:58:35.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:35.233+0000 7f38e6e97700 1 --2- 172.21.15.144:0/2628177760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0103860 0x7f38e01962b0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f38cc00c8a0 tx=0x7f38cc00cbb0 comp rx=0 tx=0).stop 2024-09-10T11:58:35.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:35.234+0000 7f38e6e97700 1 -- 172.21.15.144:0/2628177760 shutdown_connections 2024-09-10T11:58:35.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:35.234+0000 7f38e6e97700 1 --2- 172.21.15.144:0/2628177760 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f38d006e200 0x7f38d00706b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:35.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:35.234+0000 7f38e6e97700 1 --2- 172.21.15.144:0/2628177760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f38e00ff320 0x7f38e0195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:35.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:35.234+0000 7f38e6e97700 1 --2- 172.21.15.144:0/2628177760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f38e0103860 0x7f38e01962b0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:58:35.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:35.234+0000 7f38e6e97700 1 -- 172.21.15.144:0/2628177760 >> 172.21.15.144:0/2628177760 conn(0x7f38e00748b0 msgr2=0x7f38e00fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:58:35.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:35.234+0000 7f38e6e97700 1 -- 172.21.15.144:0/2628177760 shutdown_connections 2024-09-10T11:58:35.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:58:35.234+0000 7f38e6e97700 1 -- 172.21.15.144:0/2628177760 wait complete. 2024-09-10T11:58:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:35 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3135438002' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:58:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:35 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2036752073' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:58:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:35 smithi178 ceph-mon[65940]: from='client.35484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:58:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:35 smithi178 ceph-mon[65940]: pgmap v799: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:36 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2628177760' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:58:36.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:36 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2628177760' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:58:37.268 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:37 smithi178 ceph-mon[65940]: pgmap v800: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:37.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:37 smithi144 ceph-mon[87436]: pgmap v800: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:39.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:38 smithi178 ceph-mon[65940]: pgmap v801: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:39.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:38 smithi144 ceph-mon[87436]: pgmap v801: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:40 smithi178 ceph-mon[65940]: pgmap v802: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:41.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:40 smithi144 ceph-mon[87436]: pgmap v802: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:42 smithi178 ceph-mon[65940]: pgmap v803: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:42 smithi144 ceph-mon[87436]: pgmap v803: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:44.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:58:44.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:58:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:45 smithi178 ceph-mon[65940]: pgmap v804: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:45.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:45 smithi144 ceph-mon[87436]: pgmap v804: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:47 smithi178 ceph-mon[65940]: pgmap v805: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:46 smithi144 ceph-mon[87436]: pgmap v805: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:49 smithi178 ceph-mon[65940]: pgmap v806: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:49.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:49 smithi144 ceph-mon[87436]: pgmap v806: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:50 smithi178 ceph-mon[65940]: pgmap v807: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:51.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:50 smithi144 ceph-mon[87436]: pgmap v807: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:52 smithi178 ceph-mon[65940]: pgmap v808: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:52 smithi144 ceph-mon[87436]: pgmap v808: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:54 smithi178 ceph-mon[65940]: pgmap v809: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:54 smithi144 ceph-mon[87436]: pgmap v809: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:57 smithi178 ceph-mon[65940]: pgmap v810: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:57.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:57 smithi144 ceph-mon[87436]: pgmap v810: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:58:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:58:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:58:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:58:59 smithi144 ceph-mon[87436]: pgmap v811: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:00.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:58:59 smithi178 ceph-mon[65940]: pgmap v811: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:01 smithi178 ceph-mon[65940]: pgmap v812: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:01.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:01 smithi144 ceph-mon[87436]: pgmap v812: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:03 smithi178 ceph-mon[65940]: pgmap v813: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:03.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:03 smithi144 ceph-mon[87436]: pgmap v813: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:05.325 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:05 smithi178 ceph-mon[65940]: pgmap v814: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:05.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.352+0000 7f45ff8c6700 1 -- 172.21.15.144:0/1678267026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45f8101c30 msgr2=0x7f45f810a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:05.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.352+0000 7f45ff8c6700 1 --2- 172.21.15.144:0/1678267026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45f8101c30 0x7f45f810a130 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f45f4009940 tx=0x7f45f4009c50 comp rx=0 tx=0).stop 2024-09-10T11:59:05.354 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:05 smithi144 ceph-mon[87436]: pgmap v814: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:05.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.352+0000 7f45ff8c6700 1 -- 172.21.15.144:0/1678267026 shutdown_connections 2024-09-10T11:59:05.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.352+0000 7f45ff8c6700 1 --2- 172.21.15.144:0/1678267026 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45f8101c30 0x7f45f810a130 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.352+0000 7f45ff8c6700 1 --2- 172.21.15.144:0/1678267026 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45f8101320 0x7f45f81016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.352+0000 7f45ff8c6700 1 -- 172.21.15.144:0/1678267026 >> 172.21.15.144:0/1678267026 conn(0x7f45f80fab80 msgr2=0x7f45f80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:05.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.352+0000 7f45ff8c6700 1 -- 172.21.15.144:0/1678267026 shutdown_connections 2024-09-10T11:59:05.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.352+0000 7f45ff8c6700 1 -- 172.21.15.144:0/1678267026 wait complete. 2024-09-10T11:59:05.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.353+0000 7f45ff8c6700 1 Processor -- start 2024-09-10T11:59:05.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.353+0000 7f45ff8c6700 1 -- start start 2024-09-10T11:59:05.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.353+0000 7f45ff8c6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45f8101320 0x7f45f80fef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:05.356 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.353+0000 7f45ff8c6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45f8101c30 0x7f45f80ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:05.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.353+0000 7f45ff8c6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f80ffa00 con 0x7f45f8101320 2024-09-10T11:59:05.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.353+0000 7f45ff8c6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45f80ffb40 con 0x7f45f8101c30 2024-09-10T11:59:05.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.353+0000 7f45fd662700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45f8101320 0x7f45f80fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:05.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.353+0000 7f45fd662700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45f8101320 0x7f45f80fef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:60014/0 (socket says 172.21.15.144:60014) 2024-09-10T11:59:05.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.353+0000 7f45fd662700 1 -- 172.21.15.144:0/760159799 learned_addr learned my addr 172.21.15.144:0/760159799 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:59:05.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.353+0000 7f45fce61700 1 --2- 172.21.15.144:0/760159799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45f8101c30 0x7f45f80ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:05.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.353+0000 7f45fd662700 1 -- 172.21.15.144:0/760159799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45f8101c30 msgr2=0x7f45f80ff4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:05.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.353+0000 7f45fd662700 1 --2- 172.21.15.144:0/760159799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45f8101c30 0x7f45f80ff4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.353+0000 7f45fd662700 1 -- 172.21.15.144:0/760159799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f40095f0 con 0x7f45f8101320 2024-09-10T11:59:05.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.354+0000 7f45fd662700 1 --2- 172.21.15.144:0/760159799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45f8101320 0x7f45f80fef80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f45e800e920 tx=0x7f45e800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:05.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.354+0000 7f45ee7fc700 1 -- 172.21.15.144:0/760159799 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45e800c9d0 con 0x7f45f8101320 2024-09-10T11:59:05.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.354+0000 7f45ee7fc700 1 -- 172.21.15.144:0/760159799 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45e8010430 con 0x7f45f8101320 2024-09-10T11:59:05.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.354+0000 7f45ff8c6700 1 -- 172.21.15.144:0/760159799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f81a1eb0 con 0x7f45f8101320 2024-09-10T11:59:05.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.354+0000 7f45ee7fc700 1 -- 172.21.15.144:0/760159799 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45e8018630 con 0x7f45f8101320 2024-09-10T11:59:05.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.354+0000 7f45ff8c6700 1 -- 172.21.15.144:0/760159799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f81a2260 con 0x7f45f8101320 2024-09-10T11:59:05.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.355+0000 7f45ff8c6700 1 -- 172.21.15.144:0/760159799 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45f81077a0 con 0x7f45f8101320 2024-09-10T11:59:05.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.355+0000 7f45ee7fc700 1 -- 172.21.15.144:0/760159799 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f45e80105a0 con 0x7f45f8101320 2024-09-10T11:59:05.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.356+0000 7f45ee7fc700 1 --2- 172.21.15.144:0/760159799 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f45e406e200 0x7f45e40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:05.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.356+0000 7f45ee7fc700 1 -- 172.21.15.144:0/760159799 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f45e8014070 con 0x7f45f8101320 2024-09-10T11:59:05.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.356+0000 7f45fce61700 1 --2- 172.21.15.144:0/760159799 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f45e406e200 0x7f45e40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:05.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.356+0000 7f45fce61700 1 --2- 172.21.15.144:0/760159799 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f45e406e200 0x7f45e40706b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f45f4006010 tx=0x7f45f4005ad0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:05.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.360+0000 7f45ee7fc700 1 -- 172.21.15.144:0/760159799 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f45e80541b0 con 0x7f45f8101320 2024-09-10T11:59:05.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.580+0000 7f45ff8c6700 1 -- 172.21.15.144:0/760159799 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f45f802cd20 con 0x7f45e406e200 2024-09-10T11:59:05.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.581+0000 7f45ee7fc700 1 -- 172.21.15.144:0/760159799 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f45f802cd20 con 0x7f45e406e200 2024-09-10T11:59:05.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.584+0000 7f45ff8c6700 1 -- 172.21.15.144:0/760159799 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f45e406e200 msgr2=0x7f45e40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:05.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.584+0000 7f45ff8c6700 1 --2- 172.21.15.144:0/760159799 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f45e406e200 0x7f45e40706b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f45f4006010 tx=0x7f45f4005ad0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.584+0000 7f45ff8c6700 1 -- 172.21.15.144:0/760159799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45f8101320 msgr2=0x7f45f80fef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:05.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.584+0000 7f45ff8c6700 1 --2- 172.21.15.144:0/760159799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45f8101320 0x7f45f80fef80 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f45e800e920 tx=0x7f45e800ec30 comp rx=0 tx=0).stop 2024-09-10T11:59:05.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.584+0000 7f45ff8c6700 1 -- 172.21.15.144:0/760159799 shutdown_connections 2024-09-10T11:59:05.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.584+0000 7f45ff8c6700 1 --2- 172.21.15.144:0/760159799 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f45e406e200 0x7f45e40706b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.584+0000 7f45ff8c6700 1 --2- 172.21.15.144:0/760159799 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45f8101320 0x7f45f80fef80 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.584+0000 7f45ff8c6700 1 --2- 172.21.15.144:0/760159799 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f45f8101c30 0x7f45f80ff4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.584+0000 7f45ff8c6700 1 -- 172.21.15.144:0/760159799 >> 172.21.15.144:0/760159799 conn(0x7f45f80fab80 msgr2=0x7f45f81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:05.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.584+0000 7f45ff8c6700 1 -- 172.21.15.144:0/760159799 shutdown_connections 2024-09-10T11:59:05.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.584+0000 7f45ff8c6700 1 -- 172.21.15.144:0/760159799 wait complete. 2024-09-10T11:59:05.596 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:59:05.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.689+0000 7f4edbd88700 1 -- 172.21.15.144:0/3305583102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed40730f0 msgr2=0x7f4ed40734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:05.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.689+0000 7f4edbd88700 1 --2- 172.21.15.144:0/3305583102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed40730f0 0x7f4ed40734c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00b0a0 tx=0x7f4ecc00b3b0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.689+0000 7f4edbd88700 1 -- 172.21.15.144:0/3305583102 shutdown_connections 2024-09-10T11:59:05.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.689+0000 7f4edbd88700 1 --2- 172.21.15.144:0/3305583102 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed4073a00 0x7f4ed4110c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.689+0000 7f4edbd88700 1 --2- 172.21.15.144:0/3305583102 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed40730f0 0x7f4ed40734c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.689+0000 7f4edbd88700 1 -- 172.21.15.144:0/3305583102 >> 172.21.15.144:0/3305583102 conn(0x7f4ed40fbc90 msgr2=0x7f4ed40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:05.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.690+0000 7f4edbd88700 1 -- 172.21.15.144:0/3305583102 shutdown_connections 2024-09-10T11:59:05.691 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.690+0000 7f4edbd88700 1 -- 172.21.15.144:0/3305583102 wait complete. 2024-09-10T11:59:05.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.690+0000 7f4edbd88700 1 Processor -- start 2024-09-10T11:59:05.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.690+0000 7f4edbd88700 1 -- start start 2024-09-10T11:59:05.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.690+0000 7f4edbd88700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed40730f0 0x7f4ed419c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:05.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.690+0000 7f4edbd88700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed4073a00 0x7f4ed419c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:05.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.690+0000 7f4edbd88700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed419cef0 con 0x7f4ed4073a00 2024-09-10T11:59:05.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.690+0000 7f4edbd88700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed41a0c80 con 0x7f4ed40730f0 2024-09-10T11:59:05.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.691+0000 7f4ed9323700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed4073a00 0x7f4ed419c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:05.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.691+0000 7f4ed9323700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed4073a00 0x7f4ed419c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:60020/0 (socket says 172.21.15.144:60020) 2024-09-10T11:59:05.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.691+0000 7f4ed9323700 1 -- 172.21.15.144:0/682806909 learned_addr learned my addr 172.21.15.144:0/682806909 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:59:05.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.691+0000 7f4ed9b24700 1 --2- 172.21.15.144:0/682806909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed40730f0 0x7f4ed419c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:05.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.691+0000 7f4ed9323700 1 -- 172.21.15.144:0/682806909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed40730f0 msgr2=0x7f4ed419c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:05.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.691+0000 7f4ed9323700 1 --2- 172.21.15.144:0/682806909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed40730f0 0x7f4ed419c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.691+0000 7f4ed9323700 1 -- 172.21.15.144:0/682806909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc009d00 con 0x7f4ed4073a00 2024-09-10T11:59:05.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.691+0000 7f4ed9323700 1 --2- 172.21.15.144:0/682806909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed4073a00 0x7f4ed419c810 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800d8d0 tx=0x7f4ec800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:05.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.691+0000 7f4ed2ffd700 1 -- 172.21.15.144:0/682806909 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ec80098a0 con 0x7f4ed4073a00 2024-09-10T11:59:05.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.691+0000 7f4edbd88700 1 -- 172.21.15.144:0/682806909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed41a0f60 con 0x7f4ed4073a00 2024-09-10T11:59:05.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.691+0000 7f4ed2ffd700 1 -- 172.21.15.144:0/682806909 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ec8010460 con 0x7f4ed4073a00 2024-09-10T11:59:05.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.691+0000 7f4ed2ffd700 1 -- 172.21.15.144:0/682806909 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ec800f5f0 con 0x7f4ed4073a00 2024-09-10T11:59:05.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.691+0000 7f4edbd88700 1 -- 172.21.15.144:0/682806909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed41a14b0 con 0x7f4ed4073a00 2024-09-10T11:59:05.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.692+0000 7f4edbd88700 1 -- 172.21.15.144:0/682806909 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed410e3c0 con 0x7f4ed4073a00 2024-09-10T11:59:05.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.693+0000 7f4ed2ffd700 1 -- 172.21.15.144:0/682806909 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4ec8009a00 con 0x7f4ed4073a00 2024-09-10T11:59:05.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.693+0000 7f4ed2ffd700 1 --2- 172.21.15.144:0/682806909 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ebc06e200 0x7f4ebc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:05.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.693+0000 7f4ed2ffd700 1 -- 172.21.15.144:0/682806909 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4ec808df70 con 0x7f4ed4073a00 2024-09-10T11:59:05.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.693+0000 7f4ed9b24700 1 --2- 172.21.15.144:0/682806909 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ebc06e200 0x7f4ebc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:05.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.694+0000 7f4ed9b24700 1 --2- 172.21.15.144:0/682806909 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ebc06e200 0x7f4ebc0706b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009710 tx=0x7f4ecc0096a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:05.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.697+0000 7f4ed2ffd700 1 -- 172.21.15.144:0/682806909 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4ec80c31b0 con 0x7f4ed4073a00 2024-09-10T11:59:05.919 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.918+0000 7f4edbd88700 1 -- 172.21.15.144:0/682806909 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ed41a17c0 con 0x7f4ebc06e200 2024-09-10T11:59:05.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.919+0000 7f4ed2ffd700 1 -- 172.21.15.144:0/682806909 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f4ed41a17c0 con 0x7f4ebc06e200 2024-09-10T11:59:05.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.922+0000 7f4edbd88700 1 -- 172.21.15.144:0/682806909 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ebc06e200 msgr2=0x7f4ebc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:05.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.922+0000 7f4edbd88700 1 --2- 172.21.15.144:0/682806909 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ebc06e200 0x7f4ebc0706b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009710 tx=0x7f4ecc0096a0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.922+0000 7f4edbd88700 1 -- 172.21.15.144:0/682806909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed4073a00 msgr2=0x7f4ed419c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:05.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.922+0000 7f4edbd88700 1 --2- 172.21.15.144:0/682806909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed4073a00 0x7f4ed419c810 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f4ec800d8d0 tx=0x7f4ec800dbe0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.922+0000 7f4edbd88700 1 -- 172.21.15.144:0/682806909 shutdown_connections 2024-09-10T11:59:05.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.922+0000 7f4edbd88700 1 --2- 172.21.15.144:0/682806909 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ebc06e200 0x7f4ebc0706b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.922+0000 7f4edbd88700 1 --2- 172.21.15.144:0/682806909 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed40730f0 0x7f4ed419c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.922+0000 7f4edbd88700 1 --2- 172.21.15.144:0/682806909 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed4073a00 0x7f4ed419c810 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:05.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.922+0000 7f4edbd88700 1 -- 172.21.15.144:0/682806909 >> 172.21.15.144:0/682806909 conn(0x7f4ed40fbc90 msgr2=0x7f4ed4102750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:05.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.922+0000 7f4edbd88700 1 -- 172.21.15.144:0/682806909 shutdown_connections 2024-09-10T11:59:05.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:05.922+0000 7f4edbd88700 1 -- 172.21.15.144:0/682806909 wait complete. 2024-09-10T11:59:06.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.026+0000 7f847b54f700 1 -- 172.21.15.144:0/895059411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8474100d80 msgr2=0x7f8474104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:06.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.026+0000 7f847b54f700 1 --2- 172.21.15.144:0/895059411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8474100d80 0x7f8474104d50 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f8470009910 tx=0x7f8470009c20 comp rx=0 tx=0).stop 2024-09-10T11:59:06.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.026+0000 7f847b54f700 1 -- 172.21.15.144:0/895059411 shutdown_connections 2024-09-10T11:59:06.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.026+0000 7f847b54f700 1 --2- 172.21.15.144:0/895059411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8474100d80 0x7f8474104d50 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.026+0000 7f847b54f700 1 --2- 172.21.15.144:0/895059411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84741003e0 0x7f84741007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.026+0000 7f847b54f700 1 -- 172.21.15.144:0/895059411 >> 172.21.15.144:0/895059411 conn(0x7f84740fbca0 msgr2=0x7f84740fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:06.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.026+0000 7f847b54f700 1 -- 172.21.15.144:0/895059411 shutdown_connections 2024-09-10T11:59:06.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.026+0000 7f847b54f700 1 -- 172.21.15.144:0/895059411 wait complete. 2024-09-10T11:59:06.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.027+0000 7f847b54f700 1 Processor -- start 2024-09-10T11:59:06.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.027+0000 7f847b54f700 1 -- start start 2024-09-10T11:59:06.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.027+0000 7f847b54f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84741003e0 0x7f8474073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:06.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.027+0000 7f847b54f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8474100d80 0x7f8474073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:06.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.027+0000 7f847b54f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8474073ad0 con 0x7f84741003e0 2024-09-10T11:59:06.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.027+0000 7f847b54f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8474073c10 con 0x7f8474100d80 2024-09-10T11:59:06.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.027+0000 7f84792eb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84741003e0 0x7f8474073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:06.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.027+0000 7f84792eb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84741003e0 0x7f8474073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:60026/0 (socket says 172.21.15.144:60026) 2024-09-10T11:59:06.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.027+0000 7f84792eb700 1 -- 172.21.15.144:0/640533901 learned_addr learned my addr 172.21.15.144:0/640533901 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:59:06.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.028+0000 7f8478aea700 1 --2- 172.21.15.144:0/640533901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8474100d80 0x7f8474073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:06.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.028+0000 7f84792eb700 1 -- 172.21.15.144:0/640533901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8474100d80 msgr2=0x7f8474073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:06.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.028+0000 7f84792eb700 1 --2- 172.21.15.144:0/640533901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8474100d80 0x7f8474073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.028+0000 7f84792eb700 1 -- 172.21.15.144:0/640533901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84700095f0 con 0x7f84741003e0 2024-09-10T11:59:06.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.028+0000 7f8478aea700 1 --2- 172.21.15.144:0/640533901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8474100d80 0x7f8474073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:59:06.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.028+0000 7f84792eb700 1 --2- 172.21.15.144:0/640533901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84741003e0 0x7f8474073050 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f846400e920 tx=0x7f846400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:06.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.028+0000 7f846a7fc700 1 -- 172.21.15.144:0/640533901 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8464009790 con 0x7f84741003e0 2024-09-10T11:59:06.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.028+0000 7f846a7fc700 1 -- 172.21.15.144:0/640533901 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f846400cba0 con 0x7f84741003e0 2024-09-10T11:59:06.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.028+0000 7f846a7fc700 1 -- 172.21.15.144:0/640533901 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8464018650 con 0x7f84741003e0 2024-09-10T11:59:06.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.028+0000 7f847b54f700 1 -- 172.21.15.144:0/640533901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84741a1f80 con 0x7f84741003e0 2024-09-10T11:59:06.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.028+0000 7f847b54f700 1 -- 172.21.15.144:0/640533901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84741a2390 con 0x7f84741003e0 2024-09-10T11:59:06.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.029+0000 7f847b54f700 1 -- 172.21.15.144:0/640533901 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847404ea50 con 0x7f84741003e0 2024-09-10T11:59:06.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.030+0000 7f846a7fc700 1 -- 172.21.15.144:0/640533901 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f846400cd10 con 0x7f84741003e0 2024-09-10T11:59:06.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.030+0000 7f846a7fc700 1 --2- 172.21.15.144:0/640533901 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f846006e200 0x7f84600706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:06.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.030+0000 7f846a7fc700 1 -- 172.21.15.144:0/640533901 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8464014070 con 0x7f84741003e0 2024-09-10T11:59:06.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.030+0000 7f8478aea700 1 --2- 172.21.15.144:0/640533901 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f846006e200 0x7f84600706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:06.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.031+0000 7f8478aea700 1 --2- 172.21.15.144:0/640533901 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f846006e200 0x7f84600706b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8470000c00 tx=0x7f8470009f70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:06.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.034+0000 7f846a7fc700 1 -- 172.21.15.144:0/640533901 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f846405b910 con 0x7f84741003e0 2024-09-10T11:59:06.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.244+0000 7f847b54f700 1 -- 172.21.15.144:0/640533901 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f84740fdfc0 con 0x7f846006e200 2024-09-10T11:59:06.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.253+0000 7f846a7fc700 1 -- 172.21.15.144:0/640533901 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f84740fdfc0 con 0x7f846006e200 2024-09-10T11:59:06.255 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:06 smithi144 ceph-mon[87436]: from='client.35492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:06.255 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:06 smithi144 ceph-mon[87436]: from='client.35496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:06.255 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:59:06.255 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (27m) 5m ago 37m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:59:06.255 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (38m) 5m ago 38m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:59:06.255 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (37m) 5m ago 37m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:59:06.255 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (24m) 5m ago 38m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:59:06.255 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (24m) 5m ago 37m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:59:06.255 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (26m) 5m ago 37m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:59:06.255 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (21m) 5m ago 32m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:59:06.256 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (21m) 5m ago 32m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:59:06.256 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (5m) 5m ago 32m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T11:59:06.256 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (20m) 5m ago 32m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:59:06.256 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (29m) 5m ago 39m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:59:06.256 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (28m) 5m ago 36m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:59:06.256 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (25m) 5m ago 39m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:59:06.256 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (25m) 5m ago 36m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:59:06.256 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (27m) 5m ago 37m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:59:06.256 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (27m) 5m ago 36m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:59:06.256 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (24m) 5m ago 36m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:59:06.256 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (23m) 5m ago 35m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:59:06.256 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (23m) 5m ago 35m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:59:06.257 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (22m) 5m ago 34m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:59:06.257 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (22m) 5m ago 34m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:59:06.257 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (21m) 5m ago 33m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:59:06.257 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (27m) 5m ago 37m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:59:06.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.255+0000 7f847b54f700 1 -- 172.21.15.144:0/640533901 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f846006e200 msgr2=0x7f84600706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:06.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.255+0000 7f847b54f700 1 --2- 172.21.15.144:0/640533901 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f846006e200 0x7f84600706b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8470000c00 tx=0x7f8470009f70 comp rx=0 tx=0).stop 2024-09-10T11:59:06.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.255+0000 7f847b54f700 1 -- 172.21.15.144:0/640533901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84741003e0 msgr2=0x7f8474073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:06.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.255+0000 7f847b54f700 1 --2- 172.21.15.144:0/640533901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84741003e0 0x7f8474073050 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f846400e920 tx=0x7f846400ece0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.256+0000 7f847b54f700 1 -- 172.21.15.144:0/640533901 shutdown_connections 2024-09-10T11:59:06.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.256+0000 7f847b54f700 1 --2- 172.21.15.144:0/640533901 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f846006e200 0x7f84600706b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.256+0000 7f847b54f700 1 --2- 172.21.15.144:0/640533901 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84741003e0 0x7f8474073050 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.256+0000 7f847b54f700 1 --2- 172.21.15.144:0/640533901 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8474100d80 0x7f8474073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.256+0000 7f847b54f700 1 -- 172.21.15.144:0/640533901 >> 172.21.15.144:0/640533901 conn(0x7f84740fbca0 msgr2=0x7f84740fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:06.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.256+0000 7f847b54f700 1 -- 172.21.15.144:0/640533901 shutdown_connections 2024-09-10T11:59:06.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.256+0000 7f847b54f700 1 -- 172.21.15.144:0/640533901 wait complete. 2024-09-10T11:59:06.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.359+0000 7fec68e49700 1 -- 172.21.15.144:0/1816179248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec640ffc60 msgr2=0x7fec64103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:06.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.359+0000 7fec68e49700 1 --2- 172.21.15.144:0/1816179248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec640ffc60 0x7fec64103c30 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fec54009910 tx=0x7fec54009c20 comp rx=0 tx=0).stop 2024-09-10T11:59:06.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.359+0000 7fec68e49700 1 -- 172.21.15.144:0/1816179248 shutdown_connections 2024-09-10T11:59:06.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.359+0000 7fec68e49700 1 --2- 172.21.15.144:0/1816179248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec640ffc60 0x7fec64103c30 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.359+0000 7fec68e49700 1 --2- 172.21.15.144:0/1816179248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec640ff2c0 0x7fec640ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.359+0000 7fec68e49700 1 -- 172.21.15.144:0/1816179248 >> 172.21.15.144:0/1816179248 conn(0x7fec640fab80 msgr2=0x7fec640fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:06.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.359+0000 7fec68e49700 1 -- 172.21.15.144:0/1816179248 shutdown_connections 2024-09-10T11:59:06.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.359+0000 7fec68e49700 1 -- 172.21.15.144:0/1816179248 wait complete. 2024-09-10T11:59:06.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.360+0000 7fec68e49700 1 Processor -- start 2024-09-10T11:59:06.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.360+0000 7fec68e49700 1 -- start start 2024-09-10T11:59:06.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.360+0000 7fec68e49700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec640ff2c0 0x7fec64196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:06.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.360+0000 7fec68e49700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec640ffc60 0x7fec641970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:06.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.360+0000 7fec68e49700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec64197700 con 0x7fec640ff2c0 2024-09-10T11:59:06.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.360+0000 7fec68e49700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec64197870 con 0x7fec640ffc60 2024-09-10T11:59:06.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.360+0000 7fec6259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec640ff2c0 0x7fec64196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:06.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.360+0000 7fec6259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec640ff2c0 0x7fec64196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:60044/0 (socket says 172.21.15.144:60044) 2024-09-10T11:59:06.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.360+0000 7fec6259c700 1 -- 172.21.15.144:0/1250173047 learned_addr learned my addr 172.21.15.144:0/1250173047 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:59:06.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.361+0000 7fec6259c700 1 -- 172.21.15.144:0/1250173047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec640ffc60 msgr2=0x7fec641970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:06.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.361+0000 7fec6259c700 1 --2- 172.21.15.144:0/1250173047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec640ffc60 0x7fec641970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.361+0000 7fec6259c700 1 -- 172.21.15.144:0/1250173047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec540095f0 con 0x7fec640ff2c0 2024-09-10T11:59:06.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.361+0000 7fec6259c700 1 --2- 172.21.15.144:0/1250173047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec640ff2c0 0x7fec64196ba0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c8a0 tx=0x7fec5800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:06.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.361+0000 7fec537fe700 1 -- 172.21.15.144:0/1250173047 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec5800cea0 con 0x7fec640ff2c0 2024-09-10T11:59:06.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.361+0000 7fec537fe700 1 -- 172.21.15.144:0/1250173047 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec58004830 con 0x7fec640ff2c0 2024-09-10T11:59:06.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.361+0000 7fec537fe700 1 -- 172.21.15.144:0/1250173047 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec580055b0 con 0x7fec640ff2c0 2024-09-10T11:59:06.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.361+0000 7fec68e49700 1 -- 172.21.15.144:0/1250173047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec6419fc10 con 0x7fec640ff2c0 2024-09-10T11:59:06.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.361+0000 7fec68e49700 1 -- 172.21.15.144:0/1250173047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec641a0100 con 0x7fec640ff2c0 2024-09-10T11:59:06.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.362+0000 7fec68e49700 1 -- 172.21.15.144:0/1250173047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec64107630 con 0x7fec640ff2c0 2024-09-10T11:59:06.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.363+0000 7fec537fe700 1 -- 172.21.15.144:0/1250173047 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fec580049a0 con 0x7fec640ff2c0 2024-09-10T11:59:06.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.363+0000 7fec537fe700 1 --2- 172.21.15.144:0/1250173047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec4c06e200 0x7fec4c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:06.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.363+0000 7fec537fe700 1 -- 172.21.15.144:0/1250173047 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fec5808dde0 con 0x7fec640ff2c0 2024-09-10T11:59:06.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.363+0000 7fec61d9b700 1 --2- 172.21.15.144:0/1250173047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec4c06e200 0x7fec4c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:06.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.363+0000 7fec61d9b700 1 --2- 172.21.15.144:0/1250173047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec4c06e200 0x7fec4c0706b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fec54006010 tx=0x7fec5400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:06.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:06 smithi178 ceph-mon[65940]: from='client.35492 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:06.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:06 smithi178 ceph-mon[65940]: from='client.35496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:06.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.367+0000 7fec537fe700 1 -- 172.21.15.144:0/1250173047 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fec58057120 con 0x7fec640ff2c0 2024-09-10T11:59:06.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.654+0000 7fec68e49700 1 -- 172.21.15.144:0/1250173047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fec6404ea50 con 0x7fec640ff2c0 2024-09-10T11:59:06.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.655+0000 7fec537fe700 1 -- 172.21.15.144:0/1250173047 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fec5801d290 con 0x7fec640ff2c0 2024-09-10T11:59:06.656 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:59:06.656 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:59:06.657 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:59:06.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:59:06.657 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:59:06.657 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:59:06.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:59:06.657 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:59:06.657 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:59:06.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:59:06.657 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:59:06.657 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:59:06.657 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:59:06.657 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:59:06.657 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T11:59:06.658 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:59:06.658 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:59:06.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.658+0000 7fec68e49700 1 -- 172.21.15.144:0/1250173047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec4c06e200 msgr2=0x7fec4c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:06.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.658+0000 7fec68e49700 1 --2- 172.21.15.144:0/1250173047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec4c06e200 0x7fec4c0706b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fec54006010 tx=0x7fec5400b540 comp rx=0 tx=0).stop 2024-09-10T11:59:06.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.658+0000 7fec68e49700 1 -- 172.21.15.144:0/1250173047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec640ff2c0 msgr2=0x7fec64196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:06.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.658+0000 7fec68e49700 1 --2- 172.21.15.144:0/1250173047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec640ff2c0 0x7fec64196ba0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fec5800c8a0 tx=0x7fec5800cc60 comp rx=0 tx=0).stop 2024-09-10T11:59:06.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.658+0000 7fec68e49700 1 -- 172.21.15.144:0/1250173047 shutdown_connections 2024-09-10T11:59:06.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.658+0000 7fec68e49700 1 --2- 172.21.15.144:0/1250173047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec4c06e200 0x7fec4c0706b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.658+0000 7fec68e49700 1 --2- 172.21.15.144:0/1250173047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec640ff2c0 0x7fec64196ba0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.658+0000 7fec68e49700 1 --2- 172.21.15.144:0/1250173047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec640ffc60 0x7fec641970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.658+0000 7fec68e49700 1 -- 172.21.15.144:0/1250173047 >> 172.21.15.144:0/1250173047 conn(0x7fec640fab80 msgr2=0x7fec640fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:06.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.658+0000 7fec68e49700 1 -- 172.21.15.144:0/1250173047 shutdown_connections 2024-09-10T11:59:06.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.658+0000 7fec68e49700 1 -- 172.21.15.144:0/1250173047 wait complete. 2024-09-10T11:59:06.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.763+0000 7f114499e700 1 -- 172.21.15.144:0/2378433419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1140103380 msgr2=0x7f11401037f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:06.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.763+0000 7f114499e700 1 --2- 172.21.15.144:0/2378433419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1140103380 0x7f11401037f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f1134009910 tx=0x7f1134009c20 comp rx=0 tx=0).stop 2024-09-10T11:59:06.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.763+0000 7f114499e700 1 -- 172.21.15.144:0/2378433419 shutdown_connections 2024-09-10T11:59:06.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.763+0000 7f114499e700 1 --2- 172.21.15.144:0/2378433419 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1140103380 0x7f11401037f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.763+0000 7f114499e700 1 --2- 172.21.15.144:0/2378433419 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f11401046c0 0x7f1140104a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.763+0000 7f114499e700 1 -- 172.21.15.144:0/2378433419 >> 172.21.15.144:0/2378433419 conn(0x7f1140074690 msgr2=0x7f1140074a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:06.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.763+0000 7f114499e700 1 -- 172.21.15.144:0/2378433419 shutdown_connections 2024-09-10T11:59:06.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.764+0000 7f114499e700 1 -- 172.21.15.144:0/2378433419 wait complete. 2024-09-10T11:59:06.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.764+0000 7f114499e700 1 Processor -- start 2024-09-10T11:59:06.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.764+0000 7f114499e700 1 -- start start 2024-09-10T11:59:06.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.765+0000 7f114499e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1140103380 0x7f11401055e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:06.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.765+0000 7f114499e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11401046c0 0x7f1140105b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:06.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.765+0000 7f114499e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1140106060 con 0x7f11401046c0 2024-09-10T11:59:06.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.765+0000 7f114499e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11401061a0 con 0x7f1140103380 2024-09-10T11:59:06.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.765+0000 7f113effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11401046c0 0x7f1140105b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:06.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.765+0000 7f113effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11401046c0 0x7f1140105b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:60052/0 (socket says 172.21.15.144:60052) 2024-09-10T11:59:06.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.765+0000 7f113effd700 1 -- 172.21.15.144:0/1323093670 learned_addr learned my addr 172.21.15.144:0/1323093670 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:59:06.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.765+0000 7f113f7fe700 1 --2- 172.21.15.144:0/1323093670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1140103380 0x7f11401055e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:06.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.765+0000 7f113effd700 1 -- 172.21.15.144:0/1323093670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1140103380 msgr2=0x7f11401055e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:06.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.765+0000 7f113effd700 1 --2- 172.21.15.144:0/1323093670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1140103380 0x7f11401055e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:06.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.765+0000 7f113effd700 1 -- 172.21.15.144:0/1323093670 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11340095f0 con 0x7f11401046c0 2024-09-10T11:59:06.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.765+0000 7f113effd700 1 --2- 172.21.15.144:0/1323093670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11401046c0 0x7f1140105b20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1134006010 tx=0x7f113400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:06.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.765+0000 7f113cff9700 1 -- 172.21.15.144:0/1323093670 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f113401d070 con 0x7f11401046c0 2024-09-10T11:59:06.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.765+0000 7f113cff9700 1 -- 172.21.15.144:0/1323093670 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1134005dc0 con 0x7f11401046c0 2024-09-10T11:59:06.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.765+0000 7f113cff9700 1 -- 172.21.15.144:0/1323093670 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1134017400 con 0x7f11401046c0 2024-09-10T11:59:06.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.766+0000 7f114499e700 1 -- 172.21.15.144:0/1323093670 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1140106420 con 0x7f11401046c0 2024-09-10T11:59:06.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.766+0000 7f114499e700 1 -- 172.21.15.144:0/1323093670 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11401a2060 con 0x7f11401046c0 2024-09-10T11:59:06.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.767+0000 7f114499e700 1 -- 172.21.15.144:0/1323093670 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11400fe8e0 con 0x7f11401046c0 2024-09-10T11:59:06.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.767+0000 7f113cff9700 1 -- 172.21.15.144:0/1323093670 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1134003a60 con 0x7f11401046c0 2024-09-10T11:59:06.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.767+0000 7f113cff9700 1 --2- 172.21.15.144:0/1323093670 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f112806e200 0x7f11280706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:06.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.767+0000 7f113cff9700 1 -- 172.21.15.144:0/1323093670 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f113408f510 con 0x7f11401046c0 2024-09-10T11:59:06.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.768+0000 7f113f7fe700 1 --2- 172.21.15.144:0/1323093670 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f112806e200 0x7f11280706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:06.770 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.768+0000 7f113f7fe700 1 --2- 172.21.15.144:0/1323093670 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f112806e200 0x7f11280706b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1130005950 tx=0x7f113000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:06.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:06.772+0000 7f113cff9700 1 -- 172.21.15.144:0/1323093670 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f11340588f0 con 0x7f11401046c0 2024-09-10T11:59:07.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.028+0000 7f114499e700 1 -- 172.21.15.144:0/1323093670 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f114004f2a0 con 0x7f11401046c0 2024-09-10T11:59:07.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.028+0000 7f113cff9700 1 -- 172.21.15.144:0/1323093670 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f113405bfb0 con 0x7f11401046c0 2024-09-10T11:59:07.029 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T11:59:07.030 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T11:59:07.030 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:59:07.030 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:59:07.030 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:59:07.030 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:59:07.030 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:59:07.030 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:59:07.030 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:59:07.030 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:59:07.030 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:59:07.030 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:59:07.030 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:59:07.030 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:59:07.030 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:59:07.031 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:59:07.031 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:59:07.031 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:59:07.031 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:59:07.031 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:59:07.031 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:59:07.031 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:59:07.031 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:59:07.031 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:59:07.031 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:59:07.031 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:59:07.031 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:59:07.031 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:59:07.031 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:59:07.032 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:59:07.032 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:59:07.032 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:59:07.032 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:59:07.032 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:59:07.032 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:59:07.032 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:59:07.032 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:59:07.032 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:59:07.032 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:59:07.032 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:59:07.032 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:59:07.032 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:59:07.032 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:59:07.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.030+0000 7f114499e700 1 -- 172.21.15.144:0/1323093670 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f112806e200 msgr2=0x7f11280706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:07.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.030+0000 7f114499e700 1 --2- 172.21.15.144:0/1323093670 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f112806e200 0x7f11280706b0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f1130005950 tx=0x7f113000b410 comp rx=0 tx=0).stop 2024-09-10T11:59:07.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.030+0000 7f114499e700 1 -- 172.21.15.144:0/1323093670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11401046c0 msgr2=0x7f1140105b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:07.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.030+0000 7f114499e700 1 --2- 172.21.15.144:0/1323093670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11401046c0 0x7f1140105b20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f1134006010 tx=0x7f113400bd70 comp rx=0 tx=0).stop 2024-09-10T11:59:07.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.030+0000 7f114499e700 1 -- 172.21.15.144:0/1323093670 shutdown_connections 2024-09-10T11:59:07.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.030+0000 7f114499e700 1 --2- 172.21.15.144:0/1323093670 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f112806e200 0x7f11280706b0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.030+0000 7f114499e700 1 --2- 172.21.15.144:0/1323093670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1140103380 0x7f11401055e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.030+0000 7f114499e700 1 --2- 172.21.15.144:0/1323093670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f11401046c0 0x7f1140105b20 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.030+0000 7f114499e700 1 -- 172.21.15.144:0/1323093670 >> 172.21.15.144:0/1323093670 conn(0x7f1140074690 msgr2=0x7f11401023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:07.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.031+0000 7f114499e700 1 -- 172.21.15.144:0/1323093670 shutdown_connections 2024-09-10T11:59:07.035 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.031+0000 7f114499e700 1 -- 172.21.15.144:0/1323093670 wait complete. 2024-09-10T11:59:07.035 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T11:59:07.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.136+0000 7fb55ade8700 1 -- 172.21.15.144:0/2820586706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5540ff390 msgr2=0x7fb5540ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:07.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.136+0000 7fb55ade8700 1 --2- 172.21.15.144:0/2820586706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5540ff390 0x7fb5540ff760 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fb548009910 tx=0x7fb548009c20 comp rx=0 tx=0).stop 2024-09-10T11:59:07.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.136+0000 7fb55ade8700 1 -- 172.21.15.144:0/2820586706 shutdown_connections 2024-09-10T11:59:07.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.136+0000 7fb55ade8700 1 --2- 172.21.15.144:0/2820586706 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5540ffd30 0x7fb554103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.136+0000 7fb55ade8700 1 --2- 172.21.15.144:0/2820586706 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5540ff390 0x7fb5540ff760 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.136+0000 7fb55ade8700 1 -- 172.21.15.144:0/2820586706 >> 172.21.15.144:0/2820586706 conn(0x7fb554074f50 msgr2=0x7fb554075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:07.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.136+0000 7fb55ade8700 1 -- 172.21.15.144:0/2820586706 shutdown_connections 2024-09-10T11:59:07.138 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.136+0000 7fb55ade8700 1 -- 172.21.15.144:0/2820586706 wait complete. 2024-09-10T11:59:07.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.137+0000 7fb55ade8700 1 Processor -- start 2024-09-10T11:59:07.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.137+0000 7fb55ade8700 1 -- start start 2024-09-10T11:59:07.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.137+0000 7fb55ade8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5540ff390 0x7fb554196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:07.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.137+0000 7fb55ade8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5540ffd30 0x7fb554197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:07.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.137+0000 7fb55ade8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554197860 con 0x7fb5540ffd30 2024-09-10T11:59:07.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.137+0000 7fb55ade8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5541979d0 con 0x7fb5540ff390 2024-09-10T11:59:07.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.137+0000 7fb553fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5540ffd30 0x7fb554197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:07.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.137+0000 7fb553fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5540ffd30 0x7fb554197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:60060/0 (socket says 172.21.15.144:60060) 2024-09-10T11:59:07.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.137+0000 7fb553fff700 1 -- 172.21.15.144:0/1500862564 learned_addr learned my addr 172.21.15.144:0/1500862564 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:59:07.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.137+0000 7fb558b84700 1 --2- 172.21.15.144:0/1500862564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5540ff390 0x7fb554196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:07.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.137+0000 7fb553fff700 1 -- 172.21.15.144:0/1500862564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5540ff390 msgr2=0x7fb554196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:07.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.137+0000 7fb553fff700 1 --2- 172.21.15.144:0/1500862564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5540ff390 0x7fb554196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.137+0000 7fb553fff700 1 -- 172.21.15.144:0/1500862564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5480095f0 con 0x7fb5540ffd30 2024-09-10T11:59:07.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.138+0000 7fb553fff700 1 --2- 172.21.15.144:0/1500862564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5540ffd30 0x7fb554197180 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb54000d900 tx=0x7fb54000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:07.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.138+0000 7fb551ffb700 1 -- 172.21.15.144:0/1500862564 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb540009900 con 0x7fb5540ffd30 2024-09-10T11:59:07.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.138+0000 7fb551ffb700 1 -- 172.21.15.144:0/1500862564 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb540010460 con 0x7fb5540ffd30 2024-09-10T11:59:07.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.138+0000 7fb55ade8700 1 -- 172.21.15.144:0/1500862564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb55419fcb0 con 0x7fb5540ffd30 2024-09-10T11:59:07.141 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.138+0000 7fb551ffb700 1 -- 172.21.15.144:0/1500862564 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb54000f5f0 con 0x7fb5540ffd30 2024-09-10T11:59:07.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.138+0000 7fb55ade8700 1 -- 172.21.15.144:0/1500862564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5541a0200 con 0x7fb5540ffd30 2024-09-10T11:59:07.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.139+0000 7fb55ade8700 1 -- 172.21.15.144:0/1500862564 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb554107650 con 0x7fb5540ffd30 2024-09-10T11:59:07.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.139+0000 7fb551ffb700 1 -- 172.21.15.144:0/1500862564 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb5400105d0 con 0x7fb5540ffd30 2024-09-10T11:59:07.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.140+0000 7fb551ffb700 1 --2- 172.21.15.144:0/1500862564 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb54406e200 0x7fb5440706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:07.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.140+0000 7fb551ffb700 1 -- 172.21.15.144:0/1500862564 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb54008de40 con 0x7fb5540ffd30 2024-09-10T11:59:07.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.140+0000 7fb558b84700 1 --2- 172.21.15.144:0/1500862564 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb54406e200 0x7fb5440706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:07.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.140+0000 7fb558b84700 1 --2- 172.21.15.144:0/1500862564 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb54406e200 0x7fb5440706b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb54800b5c0 tx=0x7fb54801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:07.147 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.144+0000 7fb551ffb700 1 -- 172.21.15.144:0/1500862564 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb5400572a0 con 0x7fb5540ffd30 2024-09-10T11:59:07.268 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:07 smithi178 ceph-mon[65940]: from='client.35500 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:07.269 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:07 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1250173047' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:59:07.269 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:07 smithi178 ceph-mon[65940]: pgmap v815: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:07.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.364+0000 7fb55ade8700 1 -- 172.21.15.144:0/1500862564 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb554061190 con 0x7fb54406e200 2024-09-10T11:59:07.367 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:07 smithi144 ceph-mon[87436]: from='client.35500 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:07.367 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:07 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1250173047' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:59:07.367 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:07 smithi144 ceph-mon[87436]: pgmap v815: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:07.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.365+0000 7fb551ffb700 1 -- 172.21.15.144:0/1500862564 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb554061190 con 0x7fb54406e200 2024-09-10T11:59:07.368 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:59:07.368 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:59:07.368 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:59:07.368 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:59:07.368 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:59:07.368 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:59:07.368 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T11:59:07.368 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T11:59:07.369 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:59:07.369 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T11:59:07.369 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:59:07.369 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T11:59:07.369 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:59:07.369 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:59:07.369 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:59:07.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.367+0000 7fb55ade8700 1 -- 172.21.15.144:0/1500862564 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb54406e200 msgr2=0x7fb5440706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:07.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.367+0000 7fb55ade8700 1 --2- 172.21.15.144:0/1500862564 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb54406e200 0x7fb5440706b0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb54800b5c0 tx=0x7fb54801a040 comp rx=0 tx=0).stop 2024-09-10T11:59:07.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.367+0000 7fb55ade8700 1 -- 172.21.15.144:0/1500862564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5540ffd30 msgr2=0x7fb554197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:07.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.367+0000 7fb55ade8700 1 --2- 172.21.15.144:0/1500862564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5540ffd30 0x7fb554197180 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fb54000d900 tx=0x7fb54000dcc0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.367+0000 7fb55ade8700 1 -- 172.21.15.144:0/1500862564 shutdown_connections 2024-09-10T11:59:07.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.368+0000 7fb55ade8700 1 --2- 172.21.15.144:0/1500862564 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb54406e200 0x7fb5440706b0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.368+0000 7fb55ade8700 1 --2- 172.21.15.144:0/1500862564 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb5540ff390 0x7fb554196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.368+0000 7fb55ade8700 1 --2- 172.21.15.144:0/1500862564 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb5540ffd30 0x7fb554197180 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.368+0000 7fb55ade8700 1 -- 172.21.15.144:0/1500862564 >> 172.21.15.144:0/1500862564 conn(0x7fb554074f50 msgr2=0x7fb5540fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:07.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.368+0000 7fb55ade8700 1 -- 172.21.15.144:0/1500862564 shutdown_connections 2024-09-10T11:59:07.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.368+0000 7fb55ade8700 1 -- 172.21.15.144:0/1500862564 wait complete. 2024-09-10T11:59:07.479 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.474+0000 7f0fd4d40700 1 -- 172.21.15.144:0/327019864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd00ff390 msgr2=0x7f0fd00ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:07.479 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.474+0000 7f0fd4d40700 1 --2- 172.21.15.144:0/327019864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd00ff390 0x7f0fd00ff760 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0009910 tx=0x7f0fc0009c20 comp rx=0 tx=0).stop 2024-09-10T11:59:07.479 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.474+0000 7f0fd4d40700 1 -- 172.21.15.144:0/327019864 shutdown_connections 2024-09-10T11:59:07.479 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.474+0000 7f0fd4d40700 1 --2- 172.21.15.144:0/327019864 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fd00ffd30 0x7f0fd0103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.479 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.474+0000 7f0fd4d40700 1 --2- 172.21.15.144:0/327019864 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd00ff390 0x7f0fd00ff760 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.479 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.474+0000 7f0fd4d40700 1 -- 172.21.15.144:0/327019864 >> 172.21.15.144:0/327019864 conn(0x7f0fd0074f50 msgr2=0x7f0fd0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:07.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.474+0000 7f0fd4d40700 1 -- 172.21.15.144:0/327019864 shutdown_connections 2024-09-10T11:59:07.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.474+0000 7f0fd4d40700 1 -- 172.21.15.144:0/327019864 wait complete. 2024-09-10T11:59:07.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.475+0000 7f0fd4d40700 1 Processor -- start 2024-09-10T11:59:07.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.475+0000 7f0fd4d40700 1 -- start start 2024-09-10T11:59:07.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.475+0000 7f0fd4d40700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fd00ff390 0x7f0fd0196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:07.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.475+0000 7f0fd4d40700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd00ffd30 0x7f0fd0197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:07.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.475+0000 7f0fd4d40700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd01977d0 con 0x7f0fd00ffd30 2024-09-10T11:59:07.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.475+0000 7f0fd4d40700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fd0197940 con 0x7f0fd00ff390 2024-09-10T11:59:07.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.475+0000 7f0fcdd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd00ffd30 0x7f0fd0197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:07.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.475+0000 7f0fcdd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd00ffd30 0x7f0fd0197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:60078/0 (socket says 172.21.15.144:60078) 2024-09-10T11:59:07.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.475+0000 7f0fcdd9b700 1 -- 172.21.15.144:0/1280349522 learned_addr learned my addr 172.21.15.144:0/1280349522 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:59:07.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.476+0000 7f0fce59c700 1 --2- 172.21.15.144:0/1280349522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fd00ff390 0x7f0fd0196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:07.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.476+0000 7f0fcdd9b700 1 -- 172.21.15.144:0/1280349522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fd00ff390 msgr2=0x7f0fd0196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:07.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.476+0000 7f0fcdd9b700 1 --2- 172.21.15.144:0/1280349522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fd00ff390 0x7f0fd0196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.476+0000 7f0fcdd9b700 1 -- 172.21.15.144:0/1280349522 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc00095f0 con 0x7f0fd00ffd30 2024-09-10T11:59:07.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.476+0000 7f0fcdd9b700 1 --2- 172.21.15.144:0/1280349522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd00ffd30 0x7f0fd0197180 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400c960 tx=0x7f0fc400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:07.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.476+0000 7f0fbf7fe700 1 -- 172.21.15.144:0/1280349522 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fc4007a70 con 0x7f0fd00ffd30 2024-09-10T11:59:07.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.476+0000 7f0fbf7fe700 1 -- 172.21.15.144:0/1280349522 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fc4007bd0 con 0x7f0fd00ffd30 2024-09-10T11:59:07.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.476+0000 7f0fd4d40700 1 -- 172.21.15.144:0/1280349522 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fd019fcb0 con 0x7f0fd00ffd30 2024-09-10T11:59:07.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.476+0000 7f0fbf7fe700 1 -- 172.21.15.144:0/1280349522 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fc4018630 con 0x7f0fd00ffd30 2024-09-10T11:59:07.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.476+0000 7f0fd4d40700 1 -- 172.21.15.144:0/1280349522 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fd01a0170 con 0x7f0fd00ffd30 2024-09-10T11:59:07.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.477+0000 7f0fd4d40700 1 -- 172.21.15.144:0/1280349522 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fd0107650 con 0x7f0fd00ffd30 2024-09-10T11:59:07.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.478+0000 7f0fbf7fe700 1 -- 172.21.15.144:0/1280349522 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0fc400f450 con 0x7f0fd00ffd30 2024-09-10T11:59:07.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.478+0000 7f0fbf7fe700 1 --2- 172.21.15.144:0/1280349522 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0fb806e200 0x7f0fb80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:07.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.478+0000 7f0fbf7fe700 1 -- 172.21.15.144:0/1280349522 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f0fc408e000 con 0x7f0fd00ffd30 2024-09-10T11:59:07.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.478+0000 7f0fce59c700 1 --2- 172.21.15.144:0/1280349522 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0fb806e200 0x7f0fb80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:07.483 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.479+0000 7f0fce59c700 1 --2- 172.21.15.144:0/1280349522 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0fb806e200 0x7f0fb80706b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0006010 tx=0x7f0fc001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:07.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.483+0000 7f0fbf7fe700 1 -- 172.21.15.144:0/1280349522 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0fc40573e0 con 0x7f0fd00ffd30 2024-09-10T11:59:07.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.767+0000 7f0fd4d40700 1 -- 172.21.15.144:0/1280349522 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f0fd004ea50 con 0x7f0fd00ffd30 2024-09-10T11:59:07.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.767+0000 7f0fbf7fe700 1 -- 172.21.15.144:0/1280349522 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f0fc405aaa0 con 0x7f0fd00ffd30 2024-09-10T11:59:07.769 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:59:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T11:59:07.770 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:59:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T11:59:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:59:07.770 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T11:59:07.770 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T11:59:07.770 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T11:59:07.770 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T11:59:07.770 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T11:59:07.770 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T11:59:07.770 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T11:59:07.770 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T11:59:07.770 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T11:59:07.771 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T11:59:07.771 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T11:59:07.771 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T11:59:07.771 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T11:59:07.771 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:59:07.771 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:59:07.771 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:59:07.771 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:59:07.771 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:59:07.771 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:59:07.771 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:59:07.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.769+0000 7f0fd4d40700 1 -- 172.21.15.144:0/1280349522 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0fb806e200 msgr2=0x7f0fb80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:07.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.769+0000 7f0fd4d40700 1 --2- 172.21.15.144:0/1280349522 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0fb806e200 0x7f0fb80706b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0006010 tx=0x7f0fc001a040 comp rx=0 tx=0).stop 2024-09-10T11:59:07.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.769+0000 7f0fd4d40700 1 -- 172.21.15.144:0/1280349522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd00ffd30 msgr2=0x7f0fd0197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:07.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.769+0000 7f0fd4d40700 1 --2- 172.21.15.144:0/1280349522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd00ffd30 0x7f0fd0197180 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0fc400c960 tx=0x7f0fc400cc70 comp rx=0 tx=0).stop 2024-09-10T11:59:07.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.769+0000 7f0fd4d40700 1 -- 172.21.15.144:0/1280349522 shutdown_connections 2024-09-10T11:59:07.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.769+0000 7f0fd4d40700 1 --2- 172.21.15.144:0/1280349522 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f0fb806e200 0x7f0fb80706b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.769+0000 7f0fd4d40700 1 --2- 172.21.15.144:0/1280349522 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f0fd00ff390 0x7f0fd0196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.769+0000 7f0fd4d40700 1 --2- 172.21.15.144:0/1280349522 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f0fd00ffd30 0x7f0fd0197180 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:07.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.769+0000 7f0fd4d40700 1 -- 172.21.15.144:0/1280349522 >> 172.21.15.144:0/1280349522 conn(0x7f0fd0074f50 msgr2=0x7f0fd00fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:07.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.769+0000 7f0fd4d40700 1 -- 172.21.15.144:0/1280349522 shutdown_connections 2024-09-10T11:59:07.773 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:07.769+0000 7f0fd4d40700 1 -- 172.21.15.144:0/1280349522 wait complete. 2024-09-10T11:59:08.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:08 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1323093670' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:59:08.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:08 smithi144 ceph-mon[87436]: from='client.35512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:08.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:08 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1280349522' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:59:08.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:08 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1323093670' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:59:08.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:08 smithi178 ceph-mon[65940]: from='client.35512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:08.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:08 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1280349522' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:59:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:09 smithi178 ceph-mon[65940]: pgmap v816: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:09.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:09 smithi144 ceph-mon[87436]: pgmap v816: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:11 smithi178 ceph-mon[65940]: pgmap v817: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:11.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:11 smithi144 ceph-mon[87436]: pgmap v817: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:13 smithi178 ceph-mon[65940]: pgmap v818: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:13.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:13 smithi144 ceph-mon[87436]: pgmap v818: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:14 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:59:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:59:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:15 smithi178 ceph-mon[65940]: pgmap v819: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:15.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:15 smithi144 ceph-mon[87436]: pgmap v819: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:17 smithi178 ceph-mon[65940]: pgmap v820: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:17.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:17 smithi144 ceph-mon[87436]: pgmap v820: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:19.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:19 smithi178 ceph-mon[65940]: pgmap v821: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:19.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:19 smithi144 ceph-mon[87436]: pgmap v821: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:22.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:22 smithi178 ceph-mon[65940]: pgmap v822: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:22.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:22 smithi144 ceph-mon[87436]: pgmap v822: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:24.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:24 smithi178 ceph-mon[65940]: pgmap v823: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:24.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:24 smithi144 ceph-mon[87436]: pgmap v823: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:26.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:26 smithi178 ceph-mon[65940]: pgmap v824: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:26.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:26 smithi144 ceph-mon[87436]: pgmap v824: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:28.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:28 smithi178 ceph-mon[65940]: pgmap v825: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:28.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:28 smithi144 ceph-mon[87436]: pgmap v825: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:29 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:59:29.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:29 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:59:30.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:30 smithi178 ceph-mon[65940]: pgmap v826: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:30.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:30 smithi144 ceph-mon[87436]: pgmap v826: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:32.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:32 smithi178 ceph-mon[65940]: pgmap v827: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:32.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:32 smithi144 ceph-mon[87436]: pgmap v827: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:34.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:34 smithi178 ceph-mon[65940]: pgmap v828: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:34.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:34 smithi144 ceph-mon[87436]: pgmap v828: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:36.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:36 smithi178 ceph-mon[65940]: pgmap v829: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:36.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:36 smithi144 ceph-mon[87436]: pgmap v829: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:37.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.889+0000 7f17970ee700 1 -- 172.21.15.144:0/1013230784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1790068630 msgr2=0x7f1790068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:37.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.889+0000 7f17970ee700 1 --2- 172.21.15.144:0/1013230784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1790068630 0x7f1790068a00 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1784009910 tx=0x7f1784009c20 comp rx=0 tx=0).stop 2024-09-10T11:59:37.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.889+0000 7f17970ee700 1 -- 172.21.15.144:0/1013230784 shutdown_connections 2024-09-10T11:59:37.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.889+0000 7f17970ee700 1 --2- 172.21.15.144:0/1013230784 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1790068fd0 0x7f1790104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:37.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.889+0000 7f17970ee700 1 --2- 172.21.15.144:0/1013230784 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1790068630 0x7f1790068a00 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:37.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.889+0000 7f17970ee700 1 -- 172.21.15.144:0/1013230784 >> 172.21.15.144:0/1013230784 conn(0x7f1790074f50 msgr2=0x7f1790075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:37.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.889+0000 7f17970ee700 1 -- 172.21.15.144:0/1013230784 shutdown_connections 2024-09-10T11:59:37.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.889+0000 7f17970ee700 1 -- 172.21.15.144:0/1013230784 wait complete. 2024-09-10T11:59:37.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.890+0000 7f17970ee700 1 Processor -- start 2024-09-10T11:59:37.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.890+0000 7f17970ee700 1 -- start start 2024-09-10T11:59:37.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.890+0000 7f17970ee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1790068630 0x7f1790195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:37.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.890+0000 7f17970ee700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1790068fd0 0x7f1790196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:37.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.890+0000 7f17970ee700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1790196b40 con 0x7f1790068630 2024-09-10T11:59:37.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.890+0000 7f17970ee700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f179019a8d0 con 0x7f1790068fd0 2024-09-10T11:59:37.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.890+0000 7f1794e8a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1790068630 0x7f1790195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:37.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.890+0000 7f1794e8a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1790068630 0x7f1790195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42704/0 (socket says 172.21.15.144:42704) 2024-09-10T11:59:37.893 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.890+0000 7f1794e8a700 1 -- 172.21.15.144:0/4180251614 learned_addr learned my addr 172.21.15.144:0/4180251614 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:59:37.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.890+0000 7f178ffff700 1 --2- 172.21.15.144:0/4180251614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1790068fd0 0x7f1790196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:37.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.890+0000 7f1794e8a700 1 -- 172.21.15.144:0/4180251614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1790068fd0 msgr2=0x7f1790196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:37.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.890+0000 7f1794e8a700 1 --2- 172.21.15.144:0/4180251614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1790068fd0 0x7f1790196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:37.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.890+0000 7f1794e8a700 1 -- 172.21.15.144:0/4180251614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17840095f0 con 0x7f1790068630 2024-09-10T11:59:37.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.890+0000 7f178ffff700 1 --2- 172.21.15.144:0/4180251614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1790068fd0 0x7f1790196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T11:59:37.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.891+0000 7f1794e8a700 1 --2- 172.21.15.144:0/4180251614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1790068630 0x7f1790195f20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f178400b5c0 tx=0x7f178400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:37.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.891+0000 7f178dffb700 1 -- 172.21.15.144:0/4180251614 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f178401d070 con 0x7f1790068630 2024-09-10T11:59:37.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.891+0000 7f178dffb700 1 -- 172.21.15.144:0/4180251614 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f178400f460 con 0x7f1790068630 2024-09-10T11:59:37.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.891+0000 7f17970ee700 1 -- 172.21.15.144:0/4180251614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f179019ab50 con 0x7f1790068630 2024-09-10T11:59:37.894 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.891+0000 7f178dffb700 1 -- 172.21.15.144:0/4180251614 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1784017610 con 0x7f1790068630 2024-09-10T11:59:37.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.891+0000 7f17970ee700 1 -- 172.21.15.144:0/4180251614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f179019b040 con 0x7f1790068630 2024-09-10T11:59:37.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.892+0000 7f17970ee700 1 -- 172.21.15.144:0/4180251614 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17901086d0 con 0x7f1790068630 2024-09-10T11:59:37.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.893+0000 7f178dffb700 1 -- 172.21.15.144:0/4180251614 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1784003a60 con 0x7f1790068630 2024-09-10T11:59:37.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.893+0000 7f178dffb700 1 --2- 172.21.15.144:0/4180251614 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f178006e200 0x7f17800706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:37.896 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.893+0000 7f178dffb700 1 -- 172.21.15.144:0/4180251614 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f178408f5c0 con 0x7f1790068630 2024-09-10T11:59:37.896 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.893+0000 7f178ffff700 1 --2- 172.21.15.144:0/4180251614 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f178006e200 0x7f17800706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:37.896 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.894+0000 7f178ffff700 1 --2- 172.21.15.144:0/4180251614 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f178006e200 0x7f17800706b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1790197540 tx=0x7f177c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:37.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:37.898+0000 7f178dffb700 1 -- 172.21.15.144:0/4180251614 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1784058a20 con 0x7f1790068630 2024-09-10T11:59:38.120 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.119+0000 7f17970ee700 1 -- 172.21.15.144:0/4180251614 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f179019b370 con 0x7f178006e200 2024-09-10T11:59:38.121 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.120+0000 7f178dffb700 1 -- 172.21.15.144:0/4180251614 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f179019b370 con 0x7f178006e200 2024-09-10T11:59:38.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.122+0000 7f17970ee700 1 -- 172.21.15.144:0/4180251614 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f178006e200 msgr2=0x7f17800706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:38.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.122+0000 7f17970ee700 1 --2- 172.21.15.144:0/4180251614 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f178006e200 0x7f17800706b0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1790197540 tx=0x7f177c008040 comp rx=0 tx=0).stop 2024-09-10T11:59:38.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.122+0000 7f17970ee700 1 -- 172.21.15.144:0/4180251614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1790068630 msgr2=0x7f1790195f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:38.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.122+0000 7f17970ee700 1 --2- 172.21.15.144:0/4180251614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1790068630 0x7f1790195f20 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f178400b5c0 tx=0x7f178400be00 comp rx=0 tx=0).stop 2024-09-10T11:59:38.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.122+0000 7f17970ee700 1 -- 172.21.15.144:0/4180251614 shutdown_connections 2024-09-10T11:59:38.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.123+0000 7f17970ee700 1 --2- 172.21.15.144:0/4180251614 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f178006e200 0x7f17800706b0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.124 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.123+0000 7f17970ee700 1 --2- 172.21.15.144:0/4180251614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1790068630 0x7f1790195f20 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.123+0000 7f17970ee700 1 --2- 172.21.15.144:0/4180251614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1790068fd0 0x7f1790196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.123+0000 7f17970ee700 1 -- 172.21.15.144:0/4180251614 >> 172.21.15.144:0/4180251614 conn(0x7f1790074f50 msgr2=0x7f17900fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:38.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.123+0000 7f17970ee700 1 -- 172.21.15.144:0/4180251614 shutdown_connections 2024-09-10T11:59:38.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.123+0000 7f17970ee700 1 -- 172.21.15.144:0/4180251614 wait complete. 2024-09-10T11:59:38.134 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T11:59:38.232 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.229+0000 7f5a66e34700 1 -- 172.21.15.144:0/431753890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a600ff2c0 msgr2=0x7f5a600ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:38.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.229+0000 7f5a66e34700 1 --2- 172.21.15.144:0/431753890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a600ff2c0 0x7f5a600ff690 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5a54009910 tx=0x7f5a54009c20 comp rx=0 tx=0).stop 2024-09-10T11:59:38.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.229+0000 7f5a66e34700 1 -- 172.21.15.144:0/431753890 shutdown_connections 2024-09-10T11:59:38.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.229+0000 7f5a66e34700 1 --2- 172.21.15.144:0/431753890 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a600ffc60 0x7f5a60103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.229+0000 7f5a66e34700 1 --2- 172.21.15.144:0/431753890 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a600ff2c0 0x7f5a600ff690 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.229+0000 7f5a66e34700 1 -- 172.21.15.144:0/431753890 >> 172.21.15.144:0/431753890 conn(0x7f5a600fab80 msgr2=0x7f5a600fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:38.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.229+0000 7f5a66e34700 1 -- 172.21.15.144:0/431753890 shutdown_connections 2024-09-10T11:59:38.233 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.229+0000 7f5a66e34700 1 -- 172.21.15.144:0/431753890 wait complete. 2024-09-10T11:59:38.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.230+0000 7f5a66e34700 1 Processor -- start 2024-09-10T11:59:38.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.230+0000 7f5a66e34700 1 -- start start 2024-09-10T11:59:38.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.230+0000 7f5a66e34700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a600ff2c0 0x7f5a60195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:38.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.230+0000 7f5a66e34700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a600ffc60 0x7f5a601962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:38.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.230+0000 7f5a66e34700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a60196980 con 0x7f5a600ffc60 2024-09-10T11:59:38.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.230+0000 7f5a66e34700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6019a710 con 0x7f5a600ff2c0 2024-09-10T11:59:38.234 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.230+0000 7f5a5ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a600ffc60 0x7f5a601962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:38.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.230+0000 7f5a5ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a600ffc60 0x7f5a601962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42720/0 (socket says 172.21.15.144:42720) 2024-09-10T11:59:38.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.230+0000 7f5a5ffff700 1 -- 172.21.15.144:0/2773010231 learned_addr learned my addr 172.21.15.144:0/2773010231 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:59:38.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.231+0000 7f5a64bd0700 1 --2- 172.21.15.144:0/2773010231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a600ff2c0 0x7f5a60195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:38.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.231+0000 7f5a5ffff700 1 -- 172.21.15.144:0/2773010231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a600ff2c0 msgr2=0x7f5a60195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:38.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.231+0000 7f5a5ffff700 1 --2- 172.21.15.144:0/2773010231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a600ff2c0 0x7f5a60195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.231+0000 7f5a5ffff700 1 -- 172.21.15.144:0/2773010231 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a540095f0 con 0x7f5a600ffc60 2024-09-10T11:59:38.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.231+0000 7f5a5ffff700 1 --2- 172.21.15.144:0/2773010231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a600ffc60 0x7f5a601962a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00c8f0 tx=0x7f5a4c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:38.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.231+0000 7f5a5dffb700 1 -- 172.21.15.144:0/2773010231 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a4c0078a0 con 0x7f5a600ffc60 2024-09-10T11:59:38.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.231+0000 7f5a5dffb700 1 -- 172.21.15.144:0/2773010231 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5a4c00f450 con 0x7f5a600ffc60 2024-09-10T11:59:38.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.231+0000 7f5a66e34700 1 -- 172.21.15.144:0/2773010231 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a6019a9f0 con 0x7f5a600ffc60 2024-09-10T11:59:38.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.231+0000 7f5a5dffb700 1 -- 172.21.15.144:0/2773010231 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a4c018610 con 0x7f5a600ffc60 2024-09-10T11:59:38.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.231+0000 7f5a66e34700 1 -- 172.21.15.144:0/2773010231 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6019af40 con 0x7f5a600ffc60 2024-09-10T11:59:38.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.232+0000 7f5a66e34700 1 -- 172.21.15.144:0/2773010231 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a601085e0 con 0x7f5a600ffc60 2024-09-10T11:59:38.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.233+0000 7f5a5dffb700 1 -- 172.21.15.144:0/2773010231 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5a4c00f5c0 con 0x7f5a600ffc60 2024-09-10T11:59:38.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.233+0000 7f5a5dffb700 1 --2- 172.21.15.144:0/2773010231 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a5006e200 0x7f5a500706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:38.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.233+0000 7f5a64bd0700 1 --2- 172.21.15.144:0/2773010231 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a5006e200 0x7f5a500706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:38.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.233+0000 7f5a5dffb700 1 -- 172.21.15.144:0/2773010231 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5a4c08dfe0 con 0x7f5a600ffc60 2024-09-10T11:59:38.237 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.234+0000 7f5a64bd0700 1 --2- 172.21.15.144:0/2773010231 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a5006e200 0x7f5a500706b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5a54006010 tx=0x7f5a5401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:38.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.238+0000 7f5a5dffb700 1 -- 172.21.15.144:0/2773010231 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5a4c0573c0 con 0x7f5a600ffc60 2024-09-10T11:59:38.458 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.457+0000 7f5a66e34700 1 -- 172.21.15.144:0/2773010231 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a6019b280 con 0x7f5a5006e200 2024-09-10T11:59:38.459 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.458+0000 7f5a5dffb700 1 -- 172.21.15.144:0/2773010231 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5a6019b280 con 0x7f5a5006e200 2024-09-10T11:59:38.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.461+0000 7f5a66e34700 1 -- 172.21.15.144:0/2773010231 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a5006e200 msgr2=0x7f5a500706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:38.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.461+0000 7f5a66e34700 1 --2- 172.21.15.144:0/2773010231 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a5006e200 0x7f5a500706b0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f5a54006010 tx=0x7f5a5401a040 comp rx=0 tx=0).stop 2024-09-10T11:59:38.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.461+0000 7f5a66e34700 1 -- 172.21.15.144:0/2773010231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a600ffc60 msgr2=0x7f5a601962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:38.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.461+0000 7f5a66e34700 1 --2- 172.21.15.144:0/2773010231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a600ffc60 0x7f5a601962a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c00c8f0 tx=0x7f5a4c00cc00 comp rx=0 tx=0).stop 2024-09-10T11:59:38.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.461+0000 7f5a66e34700 1 -- 172.21.15.144:0/2773010231 shutdown_connections 2024-09-10T11:59:38.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.461+0000 7f5a66e34700 1 --2- 172.21.15.144:0/2773010231 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5a5006e200 0x7f5a500706b0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.461+0000 7f5a66e34700 1 --2- 172.21.15.144:0/2773010231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a600ff2c0 0x7f5a60195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.461+0000 7f5a66e34700 1 --2- 172.21.15.144:0/2773010231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a600ffc60 0x7f5a601962a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.461+0000 7f5a66e34700 1 -- 172.21.15.144:0/2773010231 >> 172.21.15.144:0/2773010231 conn(0x7f5a600fab80 msgr2=0x7f5a600690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:38.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.461+0000 7f5a66e34700 1 -- 172.21.15.144:0/2773010231 shutdown_connections 2024-09-10T11:59:38.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.461+0000 7f5a66e34700 1 -- 172.21.15.144:0/2773010231 wait complete. 2024-09-10T11:59:38.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.567+0000 7f625612b700 1 -- 172.21.15.144:0/1611524261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62501011f0 msgr2=0x7f62501015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:38.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.567+0000 7f625612b700 1 --2- 172.21.15.144:0/1611524261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62501011f0 0x7f62501015c0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f624800b0a0 tx=0x7f624800b3b0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.567+0000 7f625612b700 1 -- 172.21.15.144:0/1611524261 shutdown_connections 2024-09-10T11:59:38.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.567+0000 7f625612b700 1 --2- 172.21.15.144:0/1611524261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6250101b00 0x7f625010a1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.567+0000 7f625612b700 1 --2- 172.21.15.144:0/1611524261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f62501011f0 0x7f62501015c0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.567+0000 7f625612b700 1 -- 172.21.15.144:0/1611524261 >> 172.21.15.144:0/1611524261 conn(0x7f62500fab80 msgr2=0x7f62500fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:38.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.567+0000 7f625612b700 1 -- 172.21.15.144:0/1611524261 shutdown_connections 2024-09-10T11:59:38.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.567+0000 7f625612b700 1 -- 172.21.15.144:0/1611524261 wait complete. 2024-09-10T11:59:38.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.567+0000 7f625612b700 1 Processor -- start 2024-09-10T11:59:38.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.568+0000 7f625612b700 1 -- start start 2024-09-10T11:59:38.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.568+0000 7f625612b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62501011f0 0x7f6250195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:38.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.568+0000 7f625612b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6250101b00 0x7f62501962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:38.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.568+0000 7f625612b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6250196980 con 0x7f6250101b00 2024-09-10T11:59:38.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.568+0000 7f625612b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f625019a710 con 0x7f62501011f0 2024-09-10T11:59:38.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.568+0000 7f624effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6250101b00 0x7f62501962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:38.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.568+0000 7f624effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6250101b00 0x7f62501962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42732/0 (socket says 172.21.15.144:42732) 2024-09-10T11:59:38.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.568+0000 7f624effd700 1 -- 172.21.15.144:0/2558086514 learned_addr learned my addr 172.21.15.144:0/2558086514 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:59:38.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.568+0000 7f624f7fe700 1 --2- 172.21.15.144:0/2558086514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62501011f0 0x7f6250195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:38.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.568+0000 7f624effd700 1 -- 172.21.15.144:0/2558086514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62501011f0 msgr2=0x7f6250195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:38.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.568+0000 7f624effd700 1 --2- 172.21.15.144:0/2558086514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62501011f0 0x7f6250195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.568+0000 7f624effd700 1 -- 172.21.15.144:0/2558086514 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6248009d00 con 0x7f6250101b00 2024-09-10T11:59:38.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.568+0000 7f624effd700 1 --2- 172.21.15.144:0/2558086514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6250101b00 0x7f62501962a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f623800c8a0 tx=0x7f623800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:38.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.569+0000 7f624cff9700 1 -- 172.21.15.144:0/2558086514 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f623800cea0 con 0x7f6250101b00 2024-09-10T11:59:38.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.569+0000 7f624cff9700 1 -- 172.21.15.144:0/2558086514 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6238004830 con 0x7f6250101b00 2024-09-10T11:59:38.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.569+0000 7f624cff9700 1 -- 172.21.15.144:0/2558086514 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62380055b0 con 0x7f6250101b00 2024-09-10T11:59:38.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.569+0000 7f625612b700 1 -- 172.21.15.144:0/2558086514 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625019a9f0 con 0x7f6250101b00 2024-09-10T11:59:38.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.569+0000 7f625612b700 1 -- 172.21.15.144:0/2558086514 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f625019af40 con 0x7f6250101b00 2024-09-10T11:59:38.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.570+0000 7f625612b700 1 -- 172.21.15.144:0/2558086514 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6250107870 con 0x7f6250101b00 2024-09-10T11:59:38.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.570+0000 7f624cff9700 1 -- 172.21.15.144:0/2558086514 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f623800f450 con 0x7f6250101b00 2024-09-10T11:59:38.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.570+0000 7f624cff9700 1 --2- 172.21.15.144:0/2558086514 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f623c06e200 0x7f623c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:38.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.571+0000 7f624cff9700 1 -- 172.21.15.144:0/2558086514 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f623808dca0 con 0x7f6250101b00 2024-09-10T11:59:38.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.571+0000 7f624f7fe700 1 --2- 172.21.15.144:0/2558086514 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f623c06e200 0x7f623c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:38.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.571+0000 7f624f7fe700 1 --2- 172.21.15.144:0/2558086514 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f623c06e200 0x7f623c0706b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f624800b830 tx=0x7f62480097c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:38.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.575+0000 7f624cff9700 1 -- 172.21.15.144:0/2558086514 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6238092070 con 0x7f6250101b00 2024-09-10T11:59:38.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.784+0000 7f625612b700 1 -- 172.21.15.144:0/2558086514 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f625019b280 con 0x7f623c06e200 2024-09-10T11:59:38.785 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:38 smithi144 ceph-mon[87436]: pgmap v830: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:38.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.794+0000 7f624cff9700 1 -- 172.21.15.144:0/2558086514 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f625019b280 con 0x7f623c06e200 2024-09-10T11:59:38.795 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T11:59:38.796 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (27m) 6m ago 38m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T11:59:38.796 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (38m) 6m ago 38m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T11:59:38.796 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (37m) 6m ago 37m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T11:59:38.796 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (25m) 6m ago 38m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T11:59:38.796 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (25m) 6m ago 37m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T11:59:38.796 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (27m) 6m ago 38m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T11:59:38.796 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (21m) 6m ago 33m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T11:59:38.796 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (21m) 6m ago 32m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T11:59:38.796 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (6m) 6m ago 32m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T11:59:38.796 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (21m) 6m ago 32m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T11:59:38.796 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (29m) 6m ago 39m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T11:59:38.796 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (28m) 6m ago 37m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T11:59:38.796 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (26m) 6m ago 40m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T11:59:38.796 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (26m) 6m ago 37m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T11:59:38.797 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (28m) 6m ago 38m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T11:59:38.797 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (28m) 6m ago 37m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T11:59:38.797 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (25m) 6m ago 36m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T11:59:38.797 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (24m) 6m ago 36m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T11:59:38.797 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (23m) 6m ago 35m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T11:59:38.797 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (23m) 6m ago 35m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T11:59:38.797 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (22m) 6m ago 34m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T11:59:38.797 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (22m) 6m ago 34m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T11:59:38.797 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (28m) 6m ago 37m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T11:59:38.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.796+0000 7f625612b700 1 -- 172.21.15.144:0/2558086514 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f623c06e200 msgr2=0x7f623c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:38.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.796+0000 7f625612b700 1 --2- 172.21.15.144:0/2558086514 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f623c06e200 0x7f623c0706b0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f624800b830 tx=0x7f62480097c0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.797+0000 7f625612b700 1 -- 172.21.15.144:0/2558086514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6250101b00 msgr2=0x7f62501962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:38.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.797+0000 7f625612b700 1 --2- 172.21.15.144:0/2558086514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6250101b00 0x7f62501962a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f623800c8a0 tx=0x7f623800cc60 comp rx=0 tx=0).stop 2024-09-10T11:59:38.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.797+0000 7f625612b700 1 -- 172.21.15.144:0/2558086514 shutdown_connections 2024-09-10T11:59:38.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.797+0000 7f625612b700 1 --2- 172.21.15.144:0/2558086514 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f623c06e200 0x7f623c0706b0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.797+0000 7f625612b700 1 --2- 172.21.15.144:0/2558086514 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f62501011f0 0x7f6250195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.797+0000 7f625612b700 1 --2- 172.21.15.144:0/2558086514 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6250101b00 0x7f62501962a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.797+0000 7f625612b700 1 -- 172.21.15.144:0/2558086514 >> 172.21.15.144:0/2558086514 conn(0x7f62500fab80 msgr2=0x7f62501049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:38.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.797+0000 7f625612b700 1 -- 172.21.15.144:0/2558086514 shutdown_connections 2024-09-10T11:59:38.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.797+0000 7f625612b700 1 -- 172.21.15.144:0/2558086514 wait complete. 2024-09-10T11:59:38.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:38 smithi178 ceph-mon[65940]: pgmap v830: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:38.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.900+0000 7f68b2494700 1 -- 172.21.15.144:0/2983337322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ac0ffc70 msgr2=0x7f68ac103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:38.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.900+0000 7f68b2494700 1 --2- 172.21.15.144:0/2983337322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ac0ffc70 0x7f68ac103c40 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6894009910 tx=0x7f6894009c20 comp rx=0 tx=0).stop 2024-09-10T11:59:38.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.900+0000 7f68b2494700 1 -- 172.21.15.144:0/2983337322 shutdown_connections 2024-09-10T11:59:38.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.900+0000 7f68b2494700 1 --2- 172.21.15.144:0/2983337322 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ac0ffc70 0x7f68ac103c40 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.900+0000 7f68b2494700 1 --2- 172.21.15.144:0/2983337322 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68ac0ff2d0 0x7f68ac0ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.900+0000 7f68b2494700 1 -- 172.21.15.144:0/2983337322 >> 172.21.15.144:0/2983337322 conn(0x7f68ac0fabb0 msgr2=0x7f68ac0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:38.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.900+0000 7f68b2494700 1 -- 172.21.15.144:0/2983337322 shutdown_connections 2024-09-10T11:59:38.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.900+0000 7f68b2494700 1 -- 172.21.15.144:0/2983337322 wait complete. 2024-09-10T11:59:38.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.901+0000 7f68b2494700 1 Processor -- start 2024-09-10T11:59:38.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.901+0000 7f68b2494700 1 -- start start 2024-09-10T11:59:38.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.901+0000 7f68b2494700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ac0ff2d0 0x7f68ac100ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:38.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.901+0000 7f68b2494700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68ac0ffc70 0x7f68ac101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:38.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.901+0000 7f68b2494700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac101760 con 0x7f68ac0ff2d0 2024-09-10T11:59:38.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.901+0000 7f68b2494700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac1018a0 con 0x7f68ac0ffc70 2024-09-10T11:59:38.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.901+0000 7f68abfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ac0ff2d0 0x7f68ac100ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:38.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.901+0000 7f68abfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ac0ff2d0 0x7f68ac100ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42748/0 (socket says 172.21.15.144:42748) 2024-09-10T11:59:38.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.901+0000 7f68abfff700 1 -- 172.21.15.144:0/203192862 learned_addr learned my addr 172.21.15.144:0/203192862 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:59:38.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.901+0000 7f68a3fff700 1 --2- 172.21.15.144:0/203192862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68ac0ffc70 0x7f68ac101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:38.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.901+0000 7f68abfff700 1 -- 172.21.15.144:0/203192862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68ac0ffc70 msgr2=0x7f68ac101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:38.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.901+0000 7f68abfff700 1 --2- 172.21.15.144:0/203192862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68ac0ffc70 0x7f68ac101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:38.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.902+0000 7f68abfff700 1 -- 172.21.15.144:0/203192862 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68940095f0 con 0x7f68ac0ff2d0 2024-09-10T11:59:38.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.902+0000 7f68abfff700 1 --2- 172.21.15.144:0/203192862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ac0ff2d0 0x7f68ac100ce0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f689c00e920 tx=0x7f689c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:38.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.902+0000 7f68a9ffb700 1 -- 172.21.15.144:0/203192862 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f689c00c9d0 con 0x7f68ac0ff2d0 2024-09-10T11:59:38.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.902+0000 7f68a9ffb700 1 -- 172.21.15.144:0/203192862 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f689c010430 con 0x7f68ac0ff2d0 2024-09-10T11:59:38.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.902+0000 7f68a9ffb700 1 -- 172.21.15.144:0/203192862 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f689c018630 con 0x7f68ac0ff2d0 2024-09-10T11:59:38.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.902+0000 7f68b2494700 1 -- 172.21.15.144:0/203192862 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac1a1f00 con 0x7f68ac0ff2d0 2024-09-10T11:59:38.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.902+0000 7f68b2494700 1 -- 172.21.15.144:0/203192862 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac1a22b0 con 0x7f68ac0ff2d0 2024-09-10T11:59:38.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.903+0000 7f68b2494700 1 -- 172.21.15.144:0/203192862 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac108650 con 0x7f68ac0ff2d0 2024-09-10T11:59:38.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.904+0000 7f68a9ffb700 1 -- 172.21.15.144:0/203192862 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f689c0105a0 con 0x7f68ac0ff2d0 2024-09-10T11:59:38.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.904+0000 7f68a9ffb700 1 --2- 172.21.15.144:0/203192862 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f689806e200 0x7f68980706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:38.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.904+0000 7f68a9ffb700 1 -- 172.21.15.144:0/203192862 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f689c014070 con 0x7f68ac0ff2d0 2024-09-10T11:59:38.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.904+0000 7f68a3fff700 1 --2- 172.21.15.144:0/203192862 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f689806e200 0x7f68980706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:38.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.904+0000 7f68a3fff700 1 --2- 172.21.15.144:0/203192862 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f689806e200 0x7f68980706b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f68940059d0 tx=0x7f6894005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:38.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:38.908+0000 7f68a9ffb700 1 -- 172.21.15.144:0/203192862 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f689c0541b0 con 0x7f68ac0ff2d0 2024-09-10T11:59:39.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.193+0000 7f68b2494700 1 -- 172.21.15.144:0/203192862 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f68ac02cd20 con 0x7f68ac0ff2d0 2024-09-10T11:59:39.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.193+0000 7f68a9ffb700 1 -- 172.21.15.144:0/203192862 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f689c0576e0 con 0x7f68ac0ff2d0 2024-09-10T11:59:39.194 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:59:39.195 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T11:59:39.195 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:59:39.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:59:39.195 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T11:59:39.195 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T11:59:39.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:59:39.195 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T11:59:39.195 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T11:59:39.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:59:39.195 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T11:59:39.195 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T11:59:39.195 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T11:59:39.195 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T11:59:39.196 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T11:59:39.196 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T11:59:39.196 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:59:39.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.195+0000 7f68b2494700 1 -- 172.21.15.144:0/203192862 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f689806e200 msgr2=0x7f68980706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:39.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.195+0000 7f68b2494700 1 --2- 172.21.15.144:0/203192862 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f689806e200 0x7f68980706b0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f68940059d0 tx=0x7f6894005940 comp rx=0 tx=0).stop 2024-09-10T11:59:39.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.195+0000 7f68b2494700 1 -- 172.21.15.144:0/203192862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ac0ff2d0 msgr2=0x7f68ac100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:39.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.195+0000 7f68b2494700 1 --2- 172.21.15.144:0/203192862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ac0ff2d0 0x7f68ac100ce0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f689c00e920 tx=0x7f689c00ec30 comp rx=0 tx=0).stop 2024-09-10T11:59:39.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.195+0000 7f68b2494700 1 -- 172.21.15.144:0/203192862 shutdown_connections 2024-09-10T11:59:39.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.195+0000 7f68b2494700 1 --2- 172.21.15.144:0/203192862 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f689806e200 0x7f68980706b0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.195+0000 7f68b2494700 1 --2- 172.21.15.144:0/203192862 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f68ac0ff2d0 0x7f68ac100ce0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.195+0000 7f68b2494700 1 --2- 172.21.15.144:0/203192862 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f68ac0ffc70 0x7f68ac101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.195+0000 7f68b2494700 1 -- 172.21.15.144:0/203192862 >> 172.21.15.144:0/203192862 conn(0x7f68ac0fabb0 msgr2=0x7f68ac068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:39.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.195+0000 7f68b2494700 1 -- 172.21.15.144:0/203192862 shutdown_connections 2024-09-10T11:59:39.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.195+0000 7f68b2494700 1 -- 172.21.15.144:0/203192862 wait complete. 2024-09-10T11:59:39.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.299+0000 7faee020e700 1 -- 172.21.15.144:0/3265688770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed80ff260 msgr2=0x7faed80ff630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:39.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.299+0000 7faee020e700 1 --2- 172.21.15.144:0/3265688770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed80ff260 0x7faed80ff630 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7faec8009910 tx=0x7faec8009c20 comp rx=0 tx=0).stop 2024-09-10T11:59:39.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.299+0000 7faee020e700 1 -- 172.21.15.144:0/3265688770 shutdown_connections 2024-09-10T11:59:39.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.299+0000 7faee020e700 1 --2- 172.21.15.144:0/3265688770 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faed80ffb70 0x7faed810a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.299+0000 7faee020e700 1 --2- 172.21.15.144:0/3265688770 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed80ff260 0x7faed80ff630 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.299+0000 7faee020e700 1 -- 172.21.15.144:0/3265688770 >> 172.21.15.144:0/3265688770 conn(0x7faed80747e0 msgr2=0x7faed8074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:39.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.299+0000 7faee020e700 1 -- 172.21.15.144:0/3265688770 shutdown_connections 2024-09-10T11:59:39.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.299+0000 7faee020e700 1 -- 172.21.15.144:0/3265688770 wait complete. 2024-09-10T11:59:39.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.300+0000 7faee020e700 1 Processor -- start 2024-09-10T11:59:39.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.300+0000 7faee020e700 1 -- start start 2024-09-10T11:59:39.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.300+0000 7faee020e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed80ff260 0x7faed8195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:39.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.300+0000 7faee020e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faed80ffb70 0x7faed81962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:39.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.300+0000 7faee020e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed8196990 con 0x7faed80ff260 2024-09-10T11:59:39.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.300+0000 7faee020e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faed819a720 con 0x7faed80ffb70 2024-09-10T11:59:39.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.300+0000 7faeddfaa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed80ff260 0x7faed8195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:39.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.300+0000 7faeddfaa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed80ff260 0x7faed8195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42770/0 (socket says 172.21.15.144:42770) 2024-09-10T11:59:39.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.300+0000 7faeddfaa700 1 -- 172.21.15.144:0/3527334605 learned_addr learned my addr 172.21.15.144:0/3527334605 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:59:39.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.300+0000 7faedd7a9700 1 --2- 172.21.15.144:0/3527334605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faed80ffb70 0x7faed81962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:39.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.301+0000 7faeddfaa700 1 -- 172.21.15.144:0/3527334605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faed80ffb70 msgr2=0x7faed81962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:39.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.301+0000 7faeddfaa700 1 --2- 172.21.15.144:0/3527334605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faed80ffb70 0x7faed81962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.301+0000 7faeddfaa700 1 -- 172.21.15.144:0/3527334605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec80095f0 con 0x7faed80ff260 2024-09-10T11:59:39.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.301+0000 7faeddfaa700 1 --2- 172.21.15.144:0/3527334605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed80ff260 0x7faed8195d70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7faec8006010 tx=0x7faec800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:39.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.301+0000 7faeceffd700 1 -- 172.21.15.144:0/3527334605 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faec801d070 con 0x7faed80ff260 2024-09-10T11:59:39.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.301+0000 7faeceffd700 1 -- 172.21.15.144:0/3527334605 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faec800f460 con 0x7faed80ff260 2024-09-10T11:59:39.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.301+0000 7faee020e700 1 -- 172.21.15.144:0/3527334605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faed819aa00 con 0x7faed80ff260 2024-09-10T11:59:39.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.301+0000 7faeceffd700 1 -- 172.21.15.144:0/3527334605 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faec8017630 con 0x7faed80ff260 2024-09-10T11:59:39.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.301+0000 7faee020e700 1 -- 172.21.15.144:0/3527334605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faed819af50 con 0x7faed80ff260 2024-09-10T11:59:39.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.302+0000 7faee020e700 1 -- 172.21.15.144:0/3527334605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faed8107ce0 con 0x7faed80ff260 2024-09-10T11:59:39.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.302+0000 7faeceffd700 1 -- 172.21.15.144:0/3527334605 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7faec800f5d0 con 0x7faed80ff260 2024-09-10T11:59:39.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.303+0000 7faeceffd700 1 --2- 172.21.15.144:0/3527334605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faec406e200 0x7faec40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:39.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.303+0000 7faeceffd700 1 -- 172.21.15.144:0/3527334605 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7faec808f4f0 con 0x7faed80ff260 2024-09-10T11:59:39.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.303+0000 7faedd7a9700 1 --2- 172.21.15.144:0/3527334605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faec406e200 0x7faec40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:39.305 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.303+0000 7faedd7a9700 1 --2- 172.21.15.144:0/3527334605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faec406e200 0x7faec40706b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faed4005950 tx=0x7faed40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:39.308 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.307+0000 7faeceffd700 1 -- 172.21.15.144:0/3527334605 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7faec80588b0 con 0x7faed80ff260 2024-09-10T11:59:39.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.554+0000 7faee020e700 1 -- 172.21.15.144:0/3527334605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7faed819b170 con 0x7faed80ff260 2024-09-10T11:59:39.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.555+0000 7faeceffd700 1 -- 172.21.15.144:0/3527334605 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7faec805bf70 con 0x7faed80ff260 2024-09-10T11:59:39.556 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T11:59:39.556 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T11:59:39.557 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T11:59:39.557 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:59:39.557 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T11:59:39.557 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:59:39.557 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T11:59:39.557 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T11:59:39.557 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T11:59:39.557 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T11:59:39.557 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T11:59:39.557 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T11:59:39.557 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T11:59:39.557 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T11:59:39.557 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T11:59:39.557 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T11:59:39.558 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T11:59:39.558 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T11:59:39.558 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T11:59:39.558 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T11:59:39.558 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T11:59:39.558 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T11:59:39.558 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T11:59:39.558 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T11:59:39.558 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T11:59:39.558 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T11:59:39.558 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T11:59:39.558 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T11:59:39.558 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T11:59:39.558 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T11:59:39.559 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T11:59:39.559 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T11:59:39.559 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T11:59:39.559 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T11:59:39.559 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T11:59:39.559 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:59:39.559 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:59:39.559 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T11:59:39.559 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T11:59:39.560 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:59:39.560 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:59:39.560 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:59:39.560 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T11:59:39.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.558+0000 7faee020e700 1 -- 172.21.15.144:0/3527334605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faec406e200 msgr2=0x7faec40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:39.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.558+0000 7faee020e700 1 --2- 172.21.15.144:0/3527334605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faec406e200 0x7faec40706b0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7faed4005950 tx=0x7faed40058e0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.558+0000 7faee020e700 1 -- 172.21.15.144:0/3527334605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed80ff260 msgr2=0x7faed8195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:39.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.558+0000 7faee020e700 1 --2- 172.21.15.144:0/3527334605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed80ff260 0x7faed8195d70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7faec8006010 tx=0x7faec800be20 comp rx=0 tx=0).stop 2024-09-10T11:59:39.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.558+0000 7faee020e700 1 -- 172.21.15.144:0/3527334605 shutdown_connections 2024-09-10T11:59:39.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.558+0000 7faee020e700 1 --2- 172.21.15.144:0/3527334605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faec406e200 0x7faec40706b0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.558+0000 7faee020e700 1 --2- 172.21.15.144:0/3527334605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faed80ff260 0x7faed8195d70 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.558+0000 7faee020e700 1 --2- 172.21.15.144:0/3527334605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faed80ffb70 0x7faed81962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.558+0000 7faee020e700 1 -- 172.21.15.144:0/3527334605 >> 172.21.15.144:0/3527334605 conn(0x7faed80747e0 msgr2=0x7faed8104e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:39.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.558+0000 7faee020e700 1 -- 172.21.15.144:0/3527334605 shutdown_connections 2024-09-10T11:59:39.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.558+0000 7faee020e700 1 -- 172.21.15.144:0/3527334605 wait complete. 2024-09-10T11:59:39.561 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T11:59:39.665 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.664+0000 7f2c97c83700 1 -- 172.21.15.144:0/1398833875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c901003d0 msgr2=0x7f2c901007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:39.665 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.664+0000 7f2c97c83700 1 --2- 172.21.15.144:0/1398833875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c901003d0 0x7f2c901007a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c009910 tx=0x7f2c8c009c20 comp rx=0 tx=0).stop 2024-09-10T11:59:39.665 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.664+0000 7f2c97c83700 1 -- 172.21.15.144:0/1398833875 shutdown_connections 2024-09-10T11:59:39.665 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.664+0000 7f2c97c83700 1 --2- 172.21.15.144:0/1398833875 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c90100d70 0x7f2c90104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.664+0000 7f2c97c83700 1 --2- 172.21.15.144:0/1398833875 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c901003d0 0x7f2c901007a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.664+0000 7f2c97c83700 1 -- 172.21.15.144:0/1398833875 >> 172.21.15.144:0/1398833875 conn(0x7f2c900fbc90 msgr2=0x7f2c900fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:39.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.664+0000 7f2c97c83700 1 -- 172.21.15.144:0/1398833875 shutdown_connections 2024-09-10T11:59:39.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.664+0000 7f2c97c83700 1 -- 172.21.15.144:0/1398833875 wait complete. 2024-09-10T11:59:39.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.665+0000 7f2c97c83700 1 Processor -- start 2024-09-10T11:59:39.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.665+0000 7f2c97c83700 1 -- start start 2024-09-10T11:59:39.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.665+0000 7f2c97c83700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c901003d0 0x7f2c90195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:39.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.665+0000 7f2c97c83700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c90100d70 0x7f2c901962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:39.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.665+0000 7f2c97c83700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c90196980 con 0x7f2c90100d70 2024-09-10T11:59:39.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.665+0000 7f2c97c83700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c9019a710 con 0x7f2c901003d0 2024-09-10T11:59:39.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.665+0000 7f2c9521e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c90100d70 0x7f2c901962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:39.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.665+0000 7f2c9521e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c90100d70 0x7f2c901962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42782/0 (socket says 172.21.15.144:42782) 2024-09-10T11:59:39.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.665+0000 7f2c9521e700 1 -- 172.21.15.144:0/1036422859 learned_addr learned my addr 172.21.15.144:0/1036422859 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:59:39.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.665+0000 7f2c95a1f700 1 --2- 172.21.15.144:0/1036422859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c901003d0 0x7f2c90195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:39.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.665+0000 7f2c9521e700 1 -- 172.21.15.144:0/1036422859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c901003d0 msgr2=0x7f2c90195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:39.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.665+0000 7f2c9521e700 1 --2- 172.21.15.144:0/1036422859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c901003d0 0x7f2c90195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.665+0000 7f2c9521e700 1 -- 172.21.15.144:0/1036422859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8c0095f0 con 0x7f2c90100d70 2024-09-10T11:59:39.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.665+0000 7f2c9521e700 1 --2- 172.21.15.144:0/1036422859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c90100d70 0x7f2c901962a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000d900 tx=0x7f2c8000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:39.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.666+0000 7f2c86ffd700 1 -- 172.21.15.144:0/1036422859 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c80009900 con 0x7f2c90100d70 2024-09-10T11:59:39.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.666+0000 7f2c86ffd700 1 -- 172.21.15.144:0/1036422859 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2c80010460 con 0x7f2c90100d70 2024-09-10T11:59:39.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.666+0000 7f2c97c83700 1 -- 172.21.15.144:0/1036422859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9019a9f0 con 0x7f2c90100d70 2024-09-10T11:59:39.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.666+0000 7f2c86ffd700 1 -- 172.21.15.144:0/1036422859 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c8000f5f0 con 0x7f2c90100d70 2024-09-10T11:59:39.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.666+0000 7f2c97c83700 1 -- 172.21.15.144:0/1036422859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9019af40 con 0x7f2c90100d70 2024-09-10T11:59:39.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.667+0000 7f2c97c83700 1 -- 172.21.15.144:0/1036422859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c901086e0 con 0x7f2c90100d70 2024-09-10T11:59:39.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.667+0000 7f2c86ffd700 1 -- 172.21.15.144:0/1036422859 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2c800105d0 con 0x7f2c90100d70 2024-09-10T11:59:39.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.668+0000 7f2c86ffd700 1 --2- 172.21.15.144:0/1036422859 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2c7c06e200 0x7f2c7c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:39.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.668+0000 7f2c86ffd700 1 -- 172.21.15.144:0/1036422859 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2c8008dd00 con 0x7f2c90100d70 2024-09-10T11:59:39.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.668+0000 7f2c95a1f700 1 --2- 172.21.15.144:0/1036422859 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2c7c06e200 0x7f2c7c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:39.670 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.668+0000 7f2c95a1f700 1 --2- 172.21.15.144:0/1036422859 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2c7c06e200 0x7f2c7c0706b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00b5c0 tx=0x7f2c8c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:39.673 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.672+0000 7f2c86ffd700 1 -- 172.21.15.144:0/1036422859 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2c80057040 con 0x7f2c90100d70 2024-09-10T11:59:39.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:39 smithi178 ceph-mon[65940]: from='client.35520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:39.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:39 smithi178 ceph-mon[65940]: from='client.35524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:39.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:39 smithi178 ceph-mon[65940]: from='client.35528 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:39.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:39 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/203192862' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:59:39.896 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.895+0000 7f2c97c83700 1 -- 172.21.15.144:0/1036422859 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2c9019b2e0 con 0x7f2c7c06e200 2024-09-10T11:59:39.897 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:39 smithi144 ceph-mon[87436]: from='client.35520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:39.897 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:39 smithi144 ceph-mon[87436]: from='client.35524 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:39.897 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:39 smithi144 ceph-mon[87436]: from='client.35528 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:39.897 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:39 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/203192862' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T11:59:39.897 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.896+0000 7f2c86ffd700 1 -- 172.21.15.144:0/1036422859 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f2c9019b2e0 con 0x7f2c7c06e200 2024-09-10T11:59:39.898 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T11:59:39.898 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T11:59:39.898 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T11:59:39.898 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T11:59:39.898 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T11:59:39.898 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T11:59:39.898 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T11:59:39.898 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T11:59:39.898 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T11:59:39.898 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T11:59:39.898 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T11:59:39.898 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T11:59:39.898 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T11:59:39.899 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T11:59:39.899 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T11:59:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.899+0000 7f2c97c83700 1 -- 172.21.15.144:0/1036422859 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2c7c06e200 msgr2=0x7f2c7c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.899+0000 7f2c97c83700 1 --2- 172.21.15.144:0/1036422859 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2c7c06e200 0x7f2c7c0706b0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2c8c00b5c0 tx=0x7f2c8c01a040 comp rx=0 tx=0).stop 2024-09-10T11:59:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.899+0000 7f2c97c83700 1 -- 172.21.15.144:0/1036422859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c90100d70 msgr2=0x7f2c901962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.899+0000 7f2c97c83700 1 --2- 172.21.15.144:0/1036422859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c90100d70 0x7f2c901962a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2c8000d900 tx=0x7f2c8000dc10 comp rx=0 tx=0).stop 2024-09-10T11:59:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.899+0000 7f2c97c83700 1 -- 172.21.15.144:0/1036422859 shutdown_connections 2024-09-10T11:59:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.899+0000 7f2c97c83700 1 --2- 172.21.15.144:0/1036422859 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2c7c06e200 0x7f2c7c0706b0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.899+0000 7f2c97c83700 1 --2- 172.21.15.144:0/1036422859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2c901003d0 0x7f2c90195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.899+0000 7f2c97c83700 1 --2- 172.21.15.144:0/1036422859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2c90100d70 0x7f2c901962a0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:39.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.899+0000 7f2c97c83700 1 -- 172.21.15.144:0/1036422859 >> 172.21.15.144:0/1036422859 conn(0x7f2c900fbc90 msgr2=0x7f2c900fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:39.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.899+0000 7f2c97c83700 1 -- 172.21.15.144:0/1036422859 shutdown_connections 2024-09-10T11:59:39.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:39.899+0000 7f2c97c83700 1 -- 172.21.15.144:0/1036422859 wait complete. 2024-09-10T11:59:40.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.003+0000 7f5f51e3e700 1 -- 172.21.15.144:0/1919948254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f4c0ffba0 msgr2=0x7f5f4c10a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:40.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.003+0000 7f5f51e3e700 1 --2- 172.21.15.144:0/1919948254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f4c0ffba0 0x7f5f4c10a610 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f5f40009940 tx=0x7f5f40009c50 comp rx=0 tx=0).stop 2024-09-10T11:59:40.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.003+0000 7f5f51e3e700 1 -- 172.21.15.144:0/1919948254 shutdown_connections 2024-09-10T11:59:40.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.003+0000 7f5f51e3e700 1 --2- 172.21.15.144:0/1919948254 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f4c0ffba0 0x7f5f4c10a610 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:40.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.003+0000 7f5f51e3e700 1 --2- 172.21.15.144:0/1919948254 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f4c0ff290 0x7f5f4c0ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:40.005 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.003+0000 7f5f51e3e700 1 -- 172.21.15.144:0/1919948254 >> 172.21.15.144:0/1919948254 conn(0x7f5f4c074850 msgr2=0x7f5f4c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:40.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.003+0000 7f5f51e3e700 1 -- 172.21.15.144:0/1919948254 shutdown_connections 2024-09-10T11:59:40.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.003+0000 7f5f51e3e700 1 -- 172.21.15.144:0/1919948254 wait complete. 2024-09-10T11:59:40.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.004+0000 7f5f51e3e700 1 Processor -- start 2024-09-10T11:59:40.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.004+0000 7f5f51e3e700 1 -- start start 2024-09-10T11:59:40.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.004+0000 7f5f51e3e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f4c0ff290 0x7f5f4c195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:40.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.004+0000 7f5f51e3e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f4c0ffba0 0x7f5f4c1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:40.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.004+0000 7f5f51e3e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c1969b0 con 0x7f5f4c0ff290 2024-09-10T11:59:40.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.004+0000 7f5f51e3e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4c19a740 con 0x7f5f4c0ffba0 2024-09-10T11:59:40.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.005+0000 7f5f4b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f4c0ff290 0x7f5f4c195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:40.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.005+0000 7f5f4b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f4c0ff290 0x7f5f4c195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42790/0 (socket says 172.21.15.144:42790) 2024-09-10T11:59:40.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.005+0000 7f5f4b7fe700 1 -- 172.21.15.144:0/3757673231 learned_addr learned my addr 172.21.15.144:0/3757673231 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T11:59:40.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.005+0000 7f5f4affd700 1 --2- 172.21.15.144:0/3757673231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f4c0ffba0 0x7f5f4c1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:40.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.005+0000 7f5f4b7fe700 1 -- 172.21.15.144:0/3757673231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f4c0ffba0 msgr2=0x7f5f4c1962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:40.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.005+0000 7f5f4b7fe700 1 --2- 172.21.15.144:0/3757673231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f4c0ffba0 0x7f5f4c1962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:40.008 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.005+0000 7f5f4b7fe700 1 -- 172.21.15.144:0/3757673231 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f400095f0 con 0x7f5f4c0ff290 2024-09-10T11:59:40.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.005+0000 7f5f4affd700 1 --2- 172.21.15.144:0/3757673231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f4c0ffba0 0x7f5f4c1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T11:59:40.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.005+0000 7f5f4b7fe700 1 --2- 172.21.15.144:0/3757673231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f4c0ff290 0x7f5f4c195d90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00c8a0 tx=0x7f5f3c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:40.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.005+0000 7f5f48ff9700 1 -- 172.21.15.144:0/3757673231 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f3c00cea0 con 0x7f5f4c0ff290 2024-09-10T11:59:40.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.005+0000 7f5f48ff9700 1 -- 172.21.15.144:0/3757673231 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5f3c004830 con 0x7f5f4c0ff290 2024-09-10T11:59:40.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.005+0000 7f5f51e3e700 1 -- 172.21.15.144:0/3757673231 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f4c19aa20 con 0x7f5f4c0ff290 2024-09-10T11:59:40.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.005+0000 7f5f48ff9700 1 -- 172.21.15.144:0/3757673231 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f3c0055b0 con 0x7f5f4c0ff290 2024-09-10T11:59:40.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.005+0000 7f5f51e3e700 1 -- 172.21.15.144:0/3757673231 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f4c19af70 con 0x7f5f4c0ff290 2024-09-10T11:59:40.009 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.006+0000 7f5f51e3e700 1 -- 172.21.15.144:0/3757673231 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4c107d10 con 0x7f5f4c0ff290 2024-09-10T11:59:40.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.007+0000 7f5f48ff9700 1 -- 172.21.15.144:0/3757673231 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f5f3c0049a0 con 0x7f5f4c0ff290 2024-09-10T11:59:40.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.007+0000 7f5f48ff9700 1 --2- 172.21.15.144:0/3757673231 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5f3406e200 0x7f5f340706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T11:59:40.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.007+0000 7f5f48ff9700 1 -- 172.21.15.144:0/3757673231 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f5f3c08dc80 con 0x7f5f4c0ff290 2024-09-10T11:59:40.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.007+0000 7f5f4affd700 1 --2- 172.21.15.144:0/3757673231 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5f3406e200 0x7f5f340706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T11:59:40.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.008+0000 7f5f4affd700 1 --2- 172.21.15.144:0/3757673231 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5f3406e200 0x7f5f340706b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000b870 tx=0x7f5f40005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T11:59:40.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.012+0000 7f5f48ff9700 1 -- 172.21.15.144:0/3757673231 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5f3c0570e0 con 0x7f5f4c0ff290 2024-09-10T11:59:40.300 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.299+0000 7f5f51e3e700 1 -- 172.21.15.144:0/3757673231 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5f4c04ea50 con 0x7f5f4c0ff290 2024-09-10T11:59:40.301 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.299+0000 7f5f48ff9700 1 -- 172.21.15.144:0/3757673231 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f5f3c01d020 con 0x7f5f4c0ff290 2024-09-10T11:59:40.301 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T11:59:40.301 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T11:59:40.301 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T11:59:40.301 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T11:59:40.301 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:59:40.301 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T11:59:40.301 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T11:59:40.301 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T11:59:40.301 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T11:59:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T11:59:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T11:59:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T11:59:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T11:59:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T11:59:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T11:59:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T11:59:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T11:59:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T11:59:40.302 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T11:59:40.302 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T11:59:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T11:59:40.302 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T11:59:40.302 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T11:59:40.303 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T11:59:40.303 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T11:59:40.303 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.302+0000 7f5f51e3e700 1 -- 172.21.15.144:0/3757673231 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5f3406e200 msgr2=0x7f5f340706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:40.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.302+0000 7f5f51e3e700 1 --2- 172.21.15.144:0/3757673231 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5f3406e200 0x7f5f340706b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5f4000b870 tx=0x7f5f40005960 comp rx=0 tx=0).stop 2024-09-10T11:59:40.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.302+0000 7f5f51e3e700 1 -- 172.21.15.144:0/3757673231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f4c0ff290 msgr2=0x7f5f4c195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T11:59:40.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.302+0000 7f5f51e3e700 1 --2- 172.21.15.144:0/3757673231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f4c0ff290 0x7f5f4c195d90 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5f3c00c8a0 tx=0x7f5f3c00cc60 comp rx=0 tx=0).stop 2024-09-10T11:59:40.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.302+0000 7f5f51e3e700 1 -- 172.21.15.144:0/3757673231 shutdown_connections 2024-09-10T11:59:40.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.302+0000 7f5f51e3e700 1 --2- 172.21.15.144:0/3757673231 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f5f3406e200 0x7f5f340706b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:40.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.302+0000 7f5f51e3e700 1 --2- 172.21.15.144:0/3757673231 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5f4c0ff290 0x7f5f4c195d90 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:40.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.302+0000 7f5f51e3e700 1 --2- 172.21.15.144:0/3757673231 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5f4c0ffba0 0x7f5f4c1962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T11:59:40.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.302+0000 7f5f51e3e700 1 -- 172.21.15.144:0/3757673231 >> 172.21.15.144:0/3757673231 conn(0x7f5f4c074850 msgr2=0x7f5f4c104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T11:59:40.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.302+0000 7f5f51e3e700 1 -- 172.21.15.144:0/3757673231 shutdown_connections 2024-09-10T11:59:40.304 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T11:59:40.302+0000 7f5f51e3e700 1 -- 172.21.15.144:0/3757673231 wait complete. 2024-09-10T11:59:40.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:40 smithi144 ceph-mon[87436]: pgmap v831: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:40.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:40 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3527334605' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:59:40.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:40 smithi144 ceph-mon[87436]: from='client.35540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:40.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:40 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3757673231' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:59:40.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:40 smithi178 ceph-mon[65940]: pgmap v831: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:40.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:40 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3527334605' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T11:59:40.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:40 smithi178 ceph-mon[65940]: from='client.35540 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T11:59:40.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:40 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3757673231' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T11:59:42.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:42 smithi178 ceph-mon[65940]: pgmap v832: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:42 smithi144 ceph-mon[87436]: pgmap v832: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:44 smithi178 ceph-mon[65940]: pgmap v833: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:59:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:44 smithi144 ceph-mon[87436]: pgmap v833: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:59:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:46 smithi178 ceph-mon[65940]: pgmap v834: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:46.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:46 smithi144 ceph-mon[87436]: pgmap v834: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:48.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:48 smithi178 ceph-mon[65940]: pgmap v835: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:48 smithi144 ceph-mon[87436]: pgmap v835: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:50.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:50 smithi178 ceph-mon[65940]: pgmap v836: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:50.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:50 smithi144 ceph-mon[87436]: pgmap v836: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:52.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:52 smithi144 ceph-mon[87436]: pgmap v837: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:52.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:52 smithi178 ceph-mon[65940]: pgmap v837: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:54.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:54 smithi178 ceph-mon[65940]: pgmap v838: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:54 smithi144 ceph-mon[87436]: pgmap v838: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:56.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:56 smithi178 ceph-mon[65940]: pgmap v839: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:56.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:56 smithi144 ceph-mon[87436]: pgmap v839: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:58.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:58 smithi178 ceph-mon[65940]: pgmap v840: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:58 smithi144 ceph-mon[87436]: pgmap v840: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T11:59:59.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 11:59:59 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T11:59:59.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 11:59:59 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:00:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:00:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:00:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:00:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : /usr/bin/podman: timeout after 895 seconds 2024-09-10T12:00:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:00:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : Traceback (most recent call last): 2024-09-10T12:00:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:00:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : return _run_code(code, main_globals, None, 2024-09-10T12:00:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:00:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : exec(code, run_globals) 2024-09-10T12:00:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:00:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:00:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:00:00.435 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:00:00.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:00:00.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:00:00.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:00:00.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:00:00.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:00:00.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:00:00.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : fs cephfs is degraded 2024-09-10T12:00:00.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:00:00.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:00:00.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:00:00.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T11:59:59.999+0000 7f80c5155640 -1 log_channel(cluster) log [ERR] : fs cephfs is offline because no MDS is active for it. 2024-09-10T12:00:00.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: pgmap v841: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:00.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:00:00.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:00:00.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:00:00.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: /usr/bin/podman: timeout after 895 seconds 2024-09-10T12:00:00.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:00:00.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: Traceback (most recent call last): 2024-09-10T12:00:00.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:00:00.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: return _run_code(code, main_globals, None, 2024-09-10T12:00:00.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:00:00.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: exec(code, run_globals) 2024-09-10T12:00:00.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:00:00.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:00:00.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:00:00.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:00:00.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:00:00.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:00:00.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:00:00.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:00:00.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:00:00.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:00:00.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: fs cephfs is degraded 2024-09-10T12:00:00.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:00:00.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:00:00.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:00:00.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:00 smithi178 ceph-mon[65940]: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:00:00.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: pgmap v841: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:00.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: Health detail: HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:00:00.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: [WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:00:00.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:00:00.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: /usr/bin/podman: timeout after 895 seconds 2024-09-10T12:00:00.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:00:00.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: Traceback (most recent call last): 2024-09-10T12:00:00.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:00:00.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: return _run_code(code, main_globals, None, 2024-09-10T12:00:00.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:00:00.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: exec(code, run_globals) 2024-09-10T12:00:00.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:00:00.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:00:00.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:00:00.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:00:00.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:00:00.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:00:00.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:00:00.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:00:00.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:00:00.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:00:00.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: fs cephfs is degraded 2024-09-10T12:00:00.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:00:00.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:00:00.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: [ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:00:00.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:00 smithi144 ceph-mon[87436]: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:00:02.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:02 smithi144 ceph-mon[87436]: pgmap v842: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:02.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:02 smithi178 ceph-mon[65940]: pgmap v842: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:04.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:04 smithi178 ceph-mon[65940]: pgmap v843: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:04.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:04 smithi144 ceph-mon[87436]: pgmap v843: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:06.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:06 smithi178 ceph-mon[65940]: pgmap v844: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:06 smithi144 ceph-mon[87436]: pgmap v844: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:08.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:08 smithi178 ceph-mon[65940]: pgmap v845: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:08 smithi144 ceph-mon[87436]: pgmap v845: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:10.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.421+0000 7fcd08d88700 1 -- 172.21.15.144:0/943082820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd040ff2c0 msgr2=0x7fcd040ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:10.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.421+0000 7fcd08d88700 1 --2- 172.21.15.144:0/943082820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd040ff2c0 0x7fcd040ff690 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fccf4009910 tx=0x7fccf4009c20 comp rx=0 tx=0).stop 2024-09-10T12:00:10.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.421+0000 7fcd08d88700 1 -- 172.21.15.144:0/943082820 shutdown_connections 2024-09-10T12:00:10.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.421+0000 7fcd08d88700 1 --2- 172.21.15.144:0/943082820 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd040ffc60 0x7fcd04103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:10.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.421+0000 7fcd08d88700 1 --2- 172.21.15.144:0/943082820 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd040ff2c0 0x7fcd040ff690 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:10.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.421+0000 7fcd08d88700 1 -- 172.21.15.144:0/943082820 >> 172.21.15.144:0/943082820 conn(0x7fcd040fab80 msgr2=0x7fcd040fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:10.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.421+0000 7fcd08d88700 1 -- 172.21.15.144:0/943082820 shutdown_connections 2024-09-10T12:00:10.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.421+0000 7fcd08d88700 1 -- 172.21.15.144:0/943082820 wait complete. 2024-09-10T12:00:10.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.422+0000 7fcd08d88700 1 Processor -- start 2024-09-10T12:00:10.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.422+0000 7fcd08d88700 1 -- start start 2024-09-10T12:00:10.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.422+0000 7fcd08d88700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd040ff2c0 0x7fcd04195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:10.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.422+0000 7fcd08d88700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd040ffc60 0x7fcd04196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:10.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.422+0000 7fcd08d88700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd04196ae0 con 0x7fcd040ffc60 2024-09-10T12:00:10.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.422+0000 7fcd08d88700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0419a870 con 0x7fcd040ff2c0 2024-09-10T12:00:10.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.422+0000 7fcd01d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd040ffc60 0x7fcd04196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:10.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.422+0000 7fcd01d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd040ffc60 0x7fcd04196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41588/0 (socket says 172.21.15.144:41588) 2024-09-10T12:00:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.422+0000 7fcd01d9b700 1 -- 172.21.15.144:0/3147512299 learned_addr learned my addr 172.21.15.144:0/3147512299 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:00:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.422+0000 7fcd0259c700 1 --2- 172.21.15.144:0/3147512299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd040ff2c0 0x7fcd04195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.422+0000 7fcd01d9b700 1 -- 172.21.15.144:0/3147512299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd040ff2c0 msgr2=0x7fcd04195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.422+0000 7fcd01d9b700 1 --2- 172.21.15.144:0/3147512299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd040ff2c0 0x7fcd04195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.422+0000 7fcd01d9b700 1 -- 172.21.15.144:0/3147512299 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccf40095f0 con 0x7fcd040ffc60 2024-09-10T12:00:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.422+0000 7fcd0259c700 1 --2- 172.21.15.144:0/3147512299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd040ff2c0 0x7fcd04195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:00:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.423+0000 7fcd01d9b700 1 --2- 172.21.15.144:0/3147512299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd040ffc60 0x7fcd04196400 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fccf800b480 tx=0x7fccf800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.423+0000 7fccf37fe700 1 -- 172.21.15.144:0/3147512299 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccf80118c0 con 0x7fcd040ffc60 2024-09-10T12:00:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.423+0000 7fccf37fe700 1 -- 172.21.15.144:0/3147512299 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fccf8010380 con 0x7fcd040ffc60 2024-09-10T12:00:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.423+0000 7fcd08d88700 1 -- 172.21.15.144:0/3147512299 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0419ab50 con 0x7fcd040ffc60 2024-09-10T12:00:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.423+0000 7fccf37fe700 1 -- 172.21.15.144:0/3147512299 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccf800f4f0 con 0x7fcd040ffc60 2024-09-10T12:00:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.423+0000 7fcd08d88700 1 -- 172.21.15.144:0/3147512299 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd0419b0a0 con 0x7fcd040ffc60 2024-09-10T12:00:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.424+0000 7fcd08d88700 1 -- 172.21.15.144:0/3147512299 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0404ea50 con 0x7fcd040ffc60 2024-09-10T12:00:10.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.425+0000 7fccf37fe700 1 -- 172.21.15.144:0/3147512299 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fccf80104f0 con 0x7fcd040ffc60 2024-09-10T12:00:10.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.425+0000 7fccf37fe700 1 --2- 172.21.15.144:0/3147512299 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fccec06e200 0x7fccec0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:10.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.425+0000 7fcd0259c700 1 --2- 172.21.15.144:0/3147512299 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fccec06e200 0x7fccec0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:10.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.425+0000 7fccf37fe700 1 -- 172.21.15.144:0/3147512299 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fccf808dce0 con 0x7fcd040ffc60 2024-09-10T12:00:10.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.426+0000 7fcd0259c700 1 --2- 172.21.15.144:0/3147512299 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fccec06e200 0x7fccec0706b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fccf4005100 tx=0x7fccf401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:10.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.430+0000 7fccf37fe700 1 -- 172.21.15.144:0/3147512299 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fccf80570c0 con 0x7fcd040ffc60 2024-09-10T12:00:10.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.653+0000 7fcd08d88700 1 -- 172.21.15.144:0/3147512299 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd0419b5e0 con 0x7fccec06e200 2024-09-10T12:00:10.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.654+0000 7fccf37fe700 1 -- 172.21.15.144:0/3147512299 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fcd0419b5e0 con 0x7fccec06e200 2024-09-10T12:00:10.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.656+0000 7fcd08d88700 1 -- 172.21.15.144:0/3147512299 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fccec06e200 msgr2=0x7fccec0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:10.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.656+0000 7fcd08d88700 1 --2- 172.21.15.144:0/3147512299 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fccec06e200 0x7fccec0706b0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fccf4005100 tx=0x7fccf401a040 comp rx=0 tx=0).stop 2024-09-10T12:00:10.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.656+0000 7fcd08d88700 1 -- 172.21.15.144:0/3147512299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd040ffc60 msgr2=0x7fcd04196400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:10.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.656+0000 7fcd08d88700 1 --2- 172.21.15.144:0/3147512299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd040ffc60 0x7fcd04196400 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fccf800b480 tx=0x7fccf800b840 comp rx=0 tx=0).stop 2024-09-10T12:00:10.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.656+0000 7fcd08d88700 1 -- 172.21.15.144:0/3147512299 shutdown_connections 2024-09-10T12:00:10.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.656+0000 7fcd08d88700 1 --2- 172.21.15.144:0/3147512299 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fccec06e200 0x7fccec0706b0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:10.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.656+0000 7fcd08d88700 1 --2- 172.21.15.144:0/3147512299 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcd040ff2c0 0x7fcd04195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:10.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.656+0000 7fcd08d88700 1 --2- 172.21.15.144:0/3147512299 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcd040ffc60 0x7fcd04196400 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:10.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.657+0000 7fcd08d88700 1 -- 172.21.15.144:0/3147512299 >> 172.21.15.144:0/3147512299 conn(0x7fcd040fab80 msgr2=0x7fcd041033a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:10.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.657+0000 7fcd08d88700 1 -- 172.21.15.144:0/3147512299 shutdown_connections 2024-09-10T12:00:10.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.657+0000 7fcd08d88700 1 -- 172.21.15.144:0/3147512299 wait complete. 2024-09-10T12:00:10.668 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:00:10.683 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:10 smithi144 ceph-mon[87436]: pgmap v846: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:10.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.763+0000 7fa2d9089700 1 -- 172.21.15.144:0/4249146945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d41013a0 msgr2=0x7fa2d4101770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:10.764 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.763+0000 7fa2d9089700 1 --2- 172.21.15.144:0/4249146945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d41013a0 0x7fa2d4101770 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc009910 tx=0x7fa2cc009c20 comp rx=0 tx=0).stop 2024-09-10T12:00:10.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.763+0000 7fa2d9089700 1 -- 172.21.15.144:0/4249146945 shutdown_connections 2024-09-10T12:00:10.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.763+0000 7fa2d9089700 1 --2- 172.21.15.144:0/4249146945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2d4101cb0 0x7fa2d410a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:10.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.763+0000 7fa2d9089700 1 --2- 172.21.15.144:0/4249146945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d41013a0 0x7fa2d4101770 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:10.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.763+0000 7fa2d9089700 1 -- 172.21.15.144:0/4249146945 >> 172.21.15.144:0/4249146945 conn(0x7fa2d40fab80 msgr2=0x7fa2d40fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:10.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.763+0000 7fa2d9089700 1 -- 172.21.15.144:0/4249146945 shutdown_connections 2024-09-10T12:00:10.765 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.763+0000 7fa2d9089700 1 -- 172.21.15.144:0/4249146945 wait complete. 2024-09-10T12:00:10.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.764+0000 7fa2d9089700 1 Processor -- start 2024-09-10T12:00:10.766 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.764+0000 7fa2d9089700 1 -- start start 2024-09-10T12:00:10.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.764+0000 7fa2d9089700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2d41013a0 0x7fa2d40ff000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:10.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.764+0000 7fa2d9089700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d4101cb0 0x7fa2d40ff540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:10.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.764+0000 7fa2d9089700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d40ffa80 con 0x7fa2d4101cb0 2024-09-10T12:00:10.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.764+0000 7fa2d9089700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d40ffbc0 con 0x7fa2d41013a0 2024-09-10T12:00:10.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.764+0000 7fa2d259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d4101cb0 0x7fa2d40ff540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:10.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.764+0000 7fa2d259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d4101cb0 0x7fa2d40ff540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41610/0 (socket says 172.21.15.144:41610) 2024-09-10T12:00:10.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.764+0000 7fa2d259c700 1 -- 172.21.15.144:0/1204334480 learned_addr learned my addr 172.21.15.144:0/1204334480 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:00:10.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.764+0000 7fa2d2d9d700 1 --2- 172.21.15.144:0/1204334480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2d41013a0 0x7fa2d40ff000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:10.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.764+0000 7fa2d259c700 1 -- 172.21.15.144:0/1204334480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2d41013a0 msgr2=0x7fa2d40ff000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:10.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.764+0000 7fa2d259c700 1 --2- 172.21.15.144:0/1204334480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2d41013a0 0x7fa2d40ff000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:10.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.765+0000 7fa2d259c700 1 -- 172.21.15.144:0/1204334480 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc0095f0 con 0x7fa2d4101cb0 2024-09-10T12:00:10.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.765+0000 7fa2d259c700 1 --2- 172.21.15.144:0/1204334480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d4101cb0 0x7fa2d40ff540 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00c8a0 tx=0x7fa2bc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:10.767 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.765+0000 7fa2cbfff700 1 -- 172.21.15.144:0/1204334480 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2bc00cea0 con 0x7fa2d4101cb0 2024-09-10T12:00:10.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.765+0000 7fa2d9089700 1 -- 172.21.15.144:0/1204334480 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d41a1ef0 con 0x7fa2d4101cb0 2024-09-10T12:00:10.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.765+0000 7fa2cbfff700 1 -- 172.21.15.144:0/1204334480 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa2bc004830 con 0x7fa2d4101cb0 2024-09-10T12:00:10.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.765+0000 7fa2cbfff700 1 -- 172.21.15.144:0/1204334480 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2bc0055b0 con 0x7fa2d4101cb0 2024-09-10T12:00:10.768 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.765+0000 7fa2d9089700 1 -- 172.21.15.144:0/1204334480 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d41a2300 con 0x7fa2d4101cb0 2024-09-10T12:00:10.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.766+0000 7fa2d9089700 1 -- 172.21.15.144:0/1204334480 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2d4107880 con 0x7fa2d4101cb0 2024-09-10T12:00:10.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.766+0000 7fa2cbfff700 1 -- 172.21.15.144:0/1204334480 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa2bc0049a0 con 0x7fa2d4101cb0 2024-09-10T12:00:10.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.767+0000 7fa2cbfff700 1 --2- 172.21.15.144:0/1204334480 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa2c006e200 0x7fa2c00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:10.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.767+0000 7fa2cbfff700 1 -- 172.21.15.144:0/1204334480 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa2bc08dd40 con 0x7fa2d4101cb0 2024-09-10T12:00:10.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.767+0000 7fa2d2d9d700 1 --2- 172.21.15.144:0/1204334480 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa2c006e200 0x7fa2c00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:10.769 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.767+0000 7fa2d2d9d700 1 --2- 172.21.15.144:0/1204334480 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa2c006e200 0x7fa2c00706b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc006010 tx=0x7fa2cc00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:10.772 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.771+0000 7fa2cbfff700 1 -- 172.21.15.144:0/1204334480 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa2bc057120 con 0x7fa2d4101cb0 2024-09-10T12:00:10.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:10 smithi178 ceph-mon[65940]: pgmap v846: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:10.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.996+0000 7fa2d9089700 1 -- 172.21.15.144:0/1204334480 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2d402ccf0 con 0x7fa2c006e200 2024-09-10T12:00:10.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:10.998+0000 7fa2cbfff700 1 -- 172.21.15.144:0/1204334480 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa2d402ccf0 con 0x7fa2c006e200 2024-09-10T12:00:11.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.000+0000 7fa2d9089700 1 -- 172.21.15.144:0/1204334480 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa2c006e200 msgr2=0x7fa2c00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:11.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.000+0000 7fa2d9089700 1 --2- 172.21.15.144:0/1204334480 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa2c006e200 0x7fa2c00706b0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa2cc006010 tx=0x7fa2cc00b540 comp rx=0 tx=0).stop 2024-09-10T12:00:11.001 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.000+0000 7fa2d9089700 1 -- 172.21.15.144:0/1204334480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d4101cb0 msgr2=0x7fa2d40ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:11.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.000+0000 7fa2d9089700 1 --2- 172.21.15.144:0/1204334480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d4101cb0 0x7fa2d40ff540 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc00c8a0 tx=0x7fa2bc00cc60 comp rx=0 tx=0).stop 2024-09-10T12:00:11.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.000+0000 7fa2d9089700 1 -- 172.21.15.144:0/1204334480 shutdown_connections 2024-09-10T12:00:11.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.000+0000 7fa2d9089700 1 --2- 172.21.15.144:0/1204334480 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa2c006e200 0x7fa2c00706b0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.000+0000 7fa2d9089700 1 --2- 172.21.15.144:0/1204334480 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa2d41013a0 0x7fa2d40ff000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.000+0000 7fa2d9089700 1 --2- 172.21.15.144:0/1204334480 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa2d4101cb0 0x7fa2d40ff540 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.000+0000 7fa2d9089700 1 -- 172.21.15.144:0/1204334480 >> 172.21.15.144:0/1204334480 conn(0x7fa2d40fab80 msgr2=0x7fa2d41049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:11.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.000+0000 7fa2d9089700 1 -- 172.21.15.144:0/1204334480 shutdown_connections 2024-09-10T12:00:11.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.000+0000 7fa2d9089700 1 -- 172.21.15.144:0/1204334480 wait complete. 2024-09-10T12:00:11.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.104+0000 7fa604d65700 1 -- 172.21.15.144:0/1364853751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6000ff2d0 msgr2=0x7fa6000ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:11.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.104+0000 7fa604d65700 1 --2- 172.21.15.144:0/1364853751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6000ff2d0 0x7fa6000ff6a0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0009910 tx=0x7fa5f0009c20 comp rx=0 tx=0).stop 2024-09-10T12:00:11.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.104+0000 7fa604d65700 1 -- 172.21.15.144:0/1364853751 shutdown_connections 2024-09-10T12:00:11.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.104+0000 7fa604d65700 1 --2- 172.21.15.144:0/1364853751 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6000ffc70 0x7fa600103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.104+0000 7fa604d65700 1 --2- 172.21.15.144:0/1364853751 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6000ff2d0 0x7fa6000ff6a0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.104+0000 7fa604d65700 1 -- 172.21.15.144:0/1364853751 >> 172.21.15.144:0/1364853751 conn(0x7fa6000fabb0 msgr2=0x7fa6000fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:11.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.104+0000 7fa604d65700 1 -- 172.21.15.144:0/1364853751 shutdown_connections 2024-09-10T12:00:11.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.104+0000 7fa604d65700 1 -- 172.21.15.144:0/1364853751 wait complete. 2024-09-10T12:00:11.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.105+0000 7fa604d65700 1 Processor -- start 2024-09-10T12:00:11.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.105+0000 7fa604d65700 1 -- start start 2024-09-10T12:00:11.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.105+0000 7fa604d65700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6000ff2d0 0x7fa600195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:11.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.105+0000 7fa604d65700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6000ffc70 0x7fa6001962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:11.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.105+0000 7fa604d65700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6001969a0 con 0x7fa6000ffc70 2024-09-10T12:00:11.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.105+0000 7fa604d65700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60019a730 con 0x7fa6000ff2d0 2024-09-10T12:00:11.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.105+0000 7fa5fdd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6000ffc70 0x7fa6001962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:11.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.105+0000 7fa5fdd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6000ffc70 0x7fa6001962c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41636/0 (socket says 172.21.15.144:41636) 2024-09-10T12:00:11.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.105+0000 7fa5fdd9b700 1 -- 172.21.15.144:0/1063923634 learned_addr learned my addr 172.21.15.144:0/1063923634 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:00:11.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.106+0000 7fa5fe59c700 1 --2- 172.21.15.144:0/1063923634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6000ff2d0 0x7fa600195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:11.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.106+0000 7fa5fdd9b700 1 -- 172.21.15.144:0/1063923634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6000ff2d0 msgr2=0x7fa600195d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:11.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.106+0000 7fa5fdd9b700 1 --2- 172.21.15.144:0/1063923634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6000ff2d0 0x7fa600195d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.106+0000 7fa5fdd9b700 1 -- 172.21.15.144:0/1063923634 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f00095f0 con 0x7fa6000ffc70 2024-09-10T12:00:11.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.106+0000 7fa5fdd9b700 1 --2- 172.21.15.144:0/1063923634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6000ffc70 0x7fa6001962c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400d900 tx=0x7fa5f400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:11.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.106+0000 7fa5ef7fe700 1 -- 172.21.15.144:0/1063923634 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5f4009900 con 0x7fa6000ffc70 2024-09-10T12:00:11.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.106+0000 7fa5ef7fe700 1 -- 172.21.15.144:0/1063923634 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa5f4010460 con 0x7fa6000ffc70 2024-09-10T12:00:11.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.106+0000 7fa604d65700 1 -- 172.21.15.144:0/1063923634 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa60019aa10 con 0x7fa6000ffc70 2024-09-10T12:00:11.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.106+0000 7fa5ef7fe700 1 -- 172.21.15.144:0/1063923634 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5f400f5f0 con 0x7fa6000ffc70 2024-09-10T12:00:11.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.106+0000 7fa604d65700 1 -- 172.21.15.144:0/1063923634 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60019af60 con 0x7fa6000ffc70 2024-09-10T12:00:11.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.107+0000 7fa604d65700 1 -- 172.21.15.144:0/1063923634 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60004ea50 con 0x7fa6000ffc70 2024-09-10T12:00:11.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.108+0000 7fa5ef7fe700 1 -- 172.21.15.144:0/1063923634 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa5f40105d0 con 0x7fa6000ffc70 2024-09-10T12:00:11.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.108+0000 7fa5ef7fe700 1 --2- 172.21.15.144:0/1063923634 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa5e806e200 0x7fa5e80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:11.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.108+0000 7fa5ef7fe700 1 -- 172.21.15.144:0/1063923634 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa5f408db60 con 0x7fa6000ffc70 2024-09-10T12:00:11.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.108+0000 7fa5fe59c700 1 --2- 172.21.15.144:0/1063923634 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa5e806e200 0x7fa5e80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:11.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.109+0000 7fa5fe59c700 1 --2- 172.21.15.144:0/1063923634 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa5e806e200 0x7fa5e80706b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0006010 tx=0x7fa5f001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:11.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.113+0000 7fa5ef7fe700 1 -- 172.21.15.144:0/1063923634 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa5f4056fc0 con 0x7fa6000ffc70 2024-09-10T12:00:11.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.321+0000 7fa604d65700 1 -- 172.21.15.144:0/1063923634 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa60019b490 con 0x7fa5e806e200 2024-09-10T12:00:11.332 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.331+0000 7fa5ef7fe700 1 -- 172.21.15.144:0/1063923634 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fa60019b490 con 0x7fa5e806e200 2024-09-10T12:00:11.332 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:00:11.332 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (28m) 6m ago 38m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:00:11.332 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (39m) 6m ago 39m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:00:11.332 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (38m) 6m ago 38m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:00:11.332 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (26m) 6m ago 39m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:00:11.332 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (25m) 6m ago 38m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:00:11.332 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (28m) 6m ago 38m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:00:11.333 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (22m) 6m ago 33m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:00:11.333 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (22m) 6m ago 33m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:00:11.333 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (6m) 6m ago 33m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:00:11.333 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (21m) 6m ago 33m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:00:11.333 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (30m) 6m ago 40m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:00:11.333 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (29m) 6m ago 38m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:00:11.333 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (27m) 6m ago 40m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:00:11.333 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (26m) 6m ago 37m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:00:11.333 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (29m) 6m ago 39m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:00:11.333 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (28m) 6m ago 38m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:00:11.333 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (25m) 6m ago 37m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:00:11.333 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (25m) 6m ago 36m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:00:11.333 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (24m) 6m ago 36m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:00:11.334 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (23m) 6m ago 35m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:00:11.334 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (23m) 6m ago 35m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:00:11.334 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (22m) 6m ago 34m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:00:11.334 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (28m) 6m ago 38m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:00:11.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.333+0000 7fa604d65700 1 -- 172.21.15.144:0/1063923634 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa5e806e200 msgr2=0x7fa5e80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:11.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.333+0000 7fa604d65700 1 --2- 172.21.15.144:0/1063923634 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa5e806e200 0x7fa5e80706b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa5f0006010 tx=0x7fa5f001a040 comp rx=0 tx=0).stop 2024-09-10T12:00:11.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.333+0000 7fa604d65700 1 -- 172.21.15.144:0/1063923634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6000ffc70 msgr2=0x7fa6001962c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:11.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.333+0000 7fa604d65700 1 --2- 172.21.15.144:0/1063923634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6000ffc70 0x7fa6001962c0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa5f400d900 tx=0x7fa5f400dc10 comp rx=0 tx=0).stop 2024-09-10T12:00:11.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.333+0000 7fa604d65700 1 -- 172.21.15.144:0/1063923634 shutdown_connections 2024-09-10T12:00:11.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.333+0000 7fa604d65700 1 --2- 172.21.15.144:0/1063923634 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa5e806e200 0x7fa5e80706b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.333+0000 7fa604d65700 1 --2- 172.21.15.144:0/1063923634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6000ff2d0 0x7fa600195d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.333+0000 7fa604d65700 1 --2- 172.21.15.144:0/1063923634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6000ffc70 0x7fa6001962c0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.333+0000 7fa604d65700 1 -- 172.21.15.144:0/1063923634 >> 172.21.15.144:0/1063923634 conn(0x7fa6000fabb0 msgr2=0x7fa600068e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:11.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.333+0000 7fa604d65700 1 -- 172.21.15.144:0/1063923634 shutdown_connections 2024-09-10T12:00:11.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.333+0000 7fa604d65700 1 -- 172.21.15.144:0/1063923634 wait complete. 2024-09-10T12:00:11.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.436+0000 7f300c157700 1 -- 172.21.15.144:0/2528156948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30040ff390 msgr2=0x7f30040ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:11.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.436+0000 7f300c157700 1 --2- 172.21.15.144:0/2528156948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30040ff390 0x7f30040ff760 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3000009910 tx=0x7f3000009c20 comp rx=0 tx=0).stop 2024-09-10T12:00:11.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.436+0000 7f300c157700 1 -- 172.21.15.144:0/2528156948 shutdown_connections 2024-09-10T12:00:11.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.436+0000 7f300c157700 1 --2- 172.21.15.144:0/2528156948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30040ffd30 0x7f3004103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.436+0000 7f300c157700 1 --2- 172.21.15.144:0/2528156948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30040ff390 0x7f30040ff760 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.436+0000 7f300c157700 1 -- 172.21.15.144:0/2528156948 >> 172.21.15.144:0/2528156948 conn(0x7f3004074f50 msgr2=0x7f3004075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:11.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.436+0000 7f300c157700 1 -- 172.21.15.144:0/2528156948 shutdown_connections 2024-09-10T12:00:11.438 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.437+0000 7f300c157700 1 -- 172.21.15.144:0/2528156948 wait complete. 2024-09-10T12:00:11.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.437+0000 7f300c157700 1 Processor -- start 2024-09-10T12:00:11.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.437+0000 7f300c157700 1 -- start start 2024-09-10T12:00:11.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.437+0000 7f300c157700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30040ff390 0x7f3004195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:11.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.437+0000 7f300c157700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30040ffd30 0x7f3004196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:11.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.437+0000 7f300c157700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30041969e0 con 0x7f30040ffd30 2024-09-10T12:00:11.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.437+0000 7f300c157700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300419a770 con 0x7f30040ff390 2024-09-10T12:00:11.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.437+0000 7f30096f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30040ffd30 0x7f3004196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:11.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.437+0000 7f30096f2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30040ffd30 0x7f3004196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41654/0 (socket says 172.21.15.144:41654) 2024-09-10T12:00:11.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.437+0000 7f30096f2700 1 -- 172.21.15.144:0/495490143 learned_addr learned my addr 172.21.15.144:0/495490143 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:00:11.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.438+0000 7f3009ef3700 1 --2- 172.21.15.144:0/495490143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30040ff390 0x7f3004195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:11.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.438+0000 7f30096f2700 1 -- 172.21.15.144:0/495490143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30040ff390 msgr2=0x7f3004195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:11.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.438+0000 7f30096f2700 1 --2- 172.21.15.144:0/495490143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30040ff390 0x7f3004195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.438+0000 7f30096f2700 1 -- 172.21.15.144:0/495490143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30000095f0 con 0x7f30040ffd30 2024-09-10T12:00:11.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.438+0000 7f3009ef3700 1 --2- 172.21.15.144:0/495490143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30040ff390 0x7f3004195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:00:11.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.438+0000 7f30096f2700 1 --2- 172.21.15.144:0/495490143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30040ffd30 0x7f3004196300 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400d900 tx=0x7f2ff400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:11.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.438+0000 7f2ffaffd700 1 -- 172.21.15.144:0/495490143 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ff4009900 con 0x7f30040ffd30 2024-09-10T12:00:11.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.438+0000 7f2ffaffd700 1 -- 172.21.15.144:0/495490143 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2ff4010460 con 0x7f30040ffd30 2024-09-10T12:00:11.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.438+0000 7f2ffaffd700 1 -- 172.21.15.144:0/495490143 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ff400f5f0 con 0x7f30040ffd30 2024-09-10T12:00:11.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.438+0000 7f300c157700 1 -- 172.21.15.144:0/495490143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300419aa50 con 0x7f30040ffd30 2024-09-10T12:00:11.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.438+0000 7f300c157700 1 -- 172.21.15.144:0/495490143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300419af50 con 0x7f30040ffd30 2024-09-10T12:00:11.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.439+0000 7f300c157700 1 -- 172.21.15.144:0/495490143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f300404ea50 con 0x7f30040ffd30 2024-09-10T12:00:11.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.440+0000 7f2ffaffd700 1 -- 172.21.15.144:0/495490143 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2ff40105d0 con 0x7f30040ffd30 2024-09-10T12:00:11.442 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.440+0000 7f2ffaffd700 1 --2- 172.21.15.144:0/495490143 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2ff006e200 0x7f2ff00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:11.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.440+0000 7f2ffaffd700 1 -- 172.21.15.144:0/495490143 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2ff408df20 con 0x7f30040ffd30 2024-09-10T12:00:11.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.440+0000 7f3009ef3700 1 --2- 172.21.15.144:0/495490143 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2ff006e200 0x7f2ff00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:11.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.441+0000 7f3009ef3700 1 --2- 172.21.15.144:0/495490143 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2ff006e200 0x7f2ff00706b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f300000b5c0 tx=0x7f30000053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:11.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.444+0000 7f2ffaffd700 1 -- 172.21.15.144:0/495490143 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2ff40c31b0 con 0x7f30040ffd30 2024-09-10T12:00:11.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.728+0000 7f300c157700 1 -- 172.21.15.144:0/495490143 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3004197120 con 0x7f30040ffd30 2024-09-10T12:00:11.730 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:11 smithi144 ceph-mon[87436]: from='client.35548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:11.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.729+0000 7f2ffaffd700 1 -- 172.21.15.144:0/495490143 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f2ff4057300 con 0x7f30040ffd30 2024-09-10T12:00:11.730 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:00:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:00:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:00:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:00:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:00:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:00:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:00:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:00:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:00:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:00:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:00:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:00:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:00:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:00:11.731 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:00:11.732 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:00:11.732 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:00:11.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.731+0000 7f300c157700 1 -- 172.21.15.144:0/495490143 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2ff006e200 msgr2=0x7f2ff00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:11.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.731+0000 7f300c157700 1 --2- 172.21.15.144:0/495490143 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2ff006e200 0x7f2ff00706b0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f300000b5c0 tx=0x7f30000053a0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.731+0000 7f300c157700 1 -- 172.21.15.144:0/495490143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30040ffd30 msgr2=0x7f3004196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:11.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.731+0000 7f300c157700 1 --2- 172.21.15.144:0/495490143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30040ffd30 0x7f3004196300 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f2ff400d900 tx=0x7f2ff400dcc0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.731+0000 7f300c157700 1 -- 172.21.15.144:0/495490143 shutdown_connections 2024-09-10T12:00:11.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.731+0000 7f300c157700 1 --2- 172.21.15.144:0/495490143 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2ff006e200 0x7f2ff00706b0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.731+0000 7f300c157700 1 --2- 172.21.15.144:0/495490143 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f30040ff390 0x7f3004195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.731+0000 7f300c157700 1 --2- 172.21.15.144:0/495490143 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f30040ffd30 0x7f3004196300 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.731+0000 7f300c157700 1 -- 172.21.15.144:0/495490143 >> 172.21.15.144:0/495490143 conn(0x7f3004074f50 msgr2=0x7f30040fe430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:11.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.731+0000 7f300c157700 1 -- 172.21.15.144:0/495490143 shutdown_connections 2024-09-10T12:00:11.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.731+0000 7f300c157700 1 -- 172.21.15.144:0/495490143 wait complete. 2024-09-10T12:00:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.835+0000 7fa6eb771700 1 -- 172.21.15.144:0/646629018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e40ffba0 msgr2=0x7fa6e410a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.835+0000 7fa6eb771700 1 --2- 172.21.15.144:0/646629018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e40ffba0 0x7fa6e410a610 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009910 tx=0x7fa6d4009c20 comp rx=0 tx=0).stop 2024-09-10T12:00:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.835+0000 7fa6eb771700 1 -- 172.21.15.144:0/646629018 shutdown_connections 2024-09-10T12:00:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.835+0000 7fa6eb771700 1 --2- 172.21.15.144:0/646629018 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e40ffba0 0x7fa6e410a610 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.835+0000 7fa6eb771700 1 --2- 172.21.15.144:0/646629018 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6e40ff290 0x7fa6e40ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.835+0000 7fa6eb771700 1 -- 172.21.15.144:0/646629018 >> 172.21.15.144:0/646629018 conn(0x7fa6e4074850 msgr2=0x7fa6e4074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.836+0000 7fa6eb771700 1 -- 172.21.15.144:0/646629018 shutdown_connections 2024-09-10T12:00:11.837 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.836+0000 7fa6eb771700 1 -- 172.21.15.144:0/646629018 wait complete. 2024-09-10T12:00:11.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.836+0000 7fa6eb771700 1 Processor -- start 2024-09-10T12:00:11.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.836+0000 7fa6eb771700 1 -- start start 2024-09-10T12:00:11.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.836+0000 7fa6eb771700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6e40ff290 0x7fa6e4195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:11.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.836+0000 7fa6eb771700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e40ffba0 0x7fa6e41962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:11.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.836+0000 7fa6eb771700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e41968f0 con 0x7fa6e40ffba0 2024-09-10T12:00:11.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.836+0000 7fa6eb771700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e4196a30 con 0x7fa6e40ff290 2024-09-10T12:00:11.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.836+0000 7fa6e8d0c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e40ffba0 0x7fa6e41962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:11.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.836+0000 7fa6e8d0c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e40ffba0 0x7fa6e41962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41664/0 (socket says 172.21.15.144:41664) 2024-09-10T12:00:11.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.836+0000 7fa6e8d0c700 1 -- 172.21.15.144:0/2094520908 learned_addr learned my addr 172.21.15.144:0/2094520908 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:00:11.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.837+0000 7fa6e950d700 1 --2- 172.21.15.144:0/2094520908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6e40ff290 0x7fa6e4195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:11.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.837+0000 7fa6e8d0c700 1 -- 172.21.15.144:0/2094520908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6e40ff290 msgr2=0x7fa6e4195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:11.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.837+0000 7fa6e8d0c700 1 --2- 172.21.15.144:0/2094520908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6e40ff290 0x7fa6e4195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:11.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.837+0000 7fa6e8d0c700 1 -- 172.21.15.144:0/2094520908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d40095f0 con 0x7fa6e40ffba0 2024-09-10T12:00:11.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.837+0000 7fa6e8d0c700 1 --2- 172.21.15.144:0/2094520908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e40ffba0 0x7fa6e41962a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400b5c0 tx=0x7fa6d400bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:11.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.837+0000 7fa6da7fc700 1 -- 172.21.15.144:0/2094520908 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d401d070 con 0x7fa6e40ffba0 2024-09-10T12:00:11.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.837+0000 7fa6da7fc700 1 -- 172.21.15.144:0/2094520908 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa6d400f460 con 0x7fa6e40ffba0 2024-09-10T12:00:11.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.837+0000 7fa6eb771700 1 -- 172.21.15.144:0/2094520908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e419a820 con 0x7fa6e40ffba0 2024-09-10T12:00:11.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.837+0000 7fa6da7fc700 1 -- 172.21.15.144:0/2094520908 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d4017630 con 0x7fa6e40ffba0 2024-09-10T12:00:11.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.837+0000 7fa6eb771700 1 -- 172.21.15.144:0/2094520908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e419ad10 con 0x7fa6e40ffba0 2024-09-10T12:00:11.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.838+0000 7fa6eb771700 1 -- 172.21.15.144:0/2094520908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e4107d10 con 0x7fa6e40ffba0 2024-09-10T12:00:11.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.839+0000 7fa6da7fc700 1 -- 172.21.15.144:0/2094520908 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa6d400f5d0 con 0x7fa6e40ffba0 2024-09-10T12:00:11.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.839+0000 7fa6da7fc700 1 --2- 172.21.15.144:0/2094520908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa6d006e200 0x7fa6d00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:11.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.839+0000 7fa6da7fc700 1 -- 172.21.15.144:0/2094520908 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa6d408f510 con 0x7fa6e40ffba0 2024-09-10T12:00:11.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.839+0000 7fa6e950d700 1 --2- 172.21.15.144:0/2094520908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa6d006e200 0x7fa6d00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:11.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.840+0000 7fa6e950d700 1 --2- 172.21.15.144:0/2094520908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa6d006e200 0x7fa6d00706b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa6e406a5c0 tx=0x7fa6e000a3b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:11.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:11.844+0000 7fa6da7fc700 1 -- 172.21.15.144:0/2094520908 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa6d4058970 con 0x7fa6e40ffba0 2024-09-10T12:00:11.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:11 smithi178 ceph-mon[65940]: from='client.35548 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:12.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.096+0000 7fa6eb771700 1 -- 172.21.15.144:0/2094520908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa6e404ea50 con 0x7fa6e40ffba0 2024-09-10T12:00:12.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.097+0000 7fa6da7fc700 1 -- 172.21.15.144:0/2094520908 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7fa6d405c030 con 0x7fa6e40ffba0 2024-09-10T12:00:12.098 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:00:12.098 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:00:12.099 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:00:12.099 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:00:12.099 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:00:12.099 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:00:12.099 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:00:12.099 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:00:12.099 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:00:12.099 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:00:12.099 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:00:12.099 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:00:12.099 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:00:12.100 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:00:12.100 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:00:12.100 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:00:12.100 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:00:12.100 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:00:12.100 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:00:12.100 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:00:12.100 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:00:12.100 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:00:12.100 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:00:12.101 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:00:12.101 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:00:12.101 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:00:12.101 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:00:12.101 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:00:12.101 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:00:12.101 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:00:12.101 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:00:12.102 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:00:12.102 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:00:12.102 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:00:12.102 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:00:12.102 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:00:12.102 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:00:12.102 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:00:12.102 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:00:12.102 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:00:12.102 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:00:12.102 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:00:12.102 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:00:12.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.099+0000 7fa6eb771700 1 -- 172.21.15.144:0/2094520908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa6d006e200 msgr2=0x7fa6d00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:12.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.099+0000 7fa6eb771700 1 --2- 172.21.15.144:0/2094520908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa6d006e200 0x7fa6d00706b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fa6e406a5c0 tx=0x7fa6e000a3b0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.099+0000 7fa6eb771700 1 -- 172.21.15.144:0/2094520908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e40ffba0 msgr2=0x7fa6e41962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:12.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.099+0000 7fa6eb771700 1 --2- 172.21.15.144:0/2094520908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e40ffba0 0x7fa6e41962a0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fa6d400b5c0 tx=0x7fa6d400bd50 comp rx=0 tx=0).stop 2024-09-10T12:00:12.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.099+0000 7fa6eb771700 1 -- 172.21.15.144:0/2094520908 shutdown_connections 2024-09-10T12:00:12.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.099+0000 7fa6eb771700 1 --2- 172.21.15.144:0/2094520908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa6d006e200 0x7fa6d00706b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.099+0000 7fa6eb771700 1 --2- 172.21.15.144:0/2094520908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa6e40ff290 0x7fa6e4195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.099+0000 7fa6eb771700 1 --2- 172.21.15.144:0/2094520908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa6e40ffba0 0x7fa6e41962a0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.099+0000 7fa6eb771700 1 -- 172.21.15.144:0/2094520908 >> 172.21.15.144:0/2094520908 conn(0x7fa6e4074850 msgr2=0x7fa6e4104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:12.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.099+0000 7fa6eb771700 1 -- 172.21.15.144:0/2094520908 shutdown_connections 2024-09-10T12:00:12.104 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.099+0000 7fa6eb771700 1 -- 172.21.15.144:0/2094520908 wait complete. 2024-09-10T12:00:12.104 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:00:12.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.205+0000 7fa3a3caf700 1 -- 172.21.15.144:0/1578909830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa39c100d50 msgr2=0x7fa39c104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:12.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.205+0000 7fa3a3caf700 1 --2- 172.21.15.144:0/1578909830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa39c100d50 0x7fa39c104d20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009910 tx=0x7fa38c009c20 comp rx=0 tx=0).stop 2024-09-10T12:00:12.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.205+0000 7fa3a3caf700 1 -- 172.21.15.144:0/1578909830 shutdown_connections 2024-09-10T12:00:12.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.205+0000 7fa3a3caf700 1 --2- 172.21.15.144:0/1578909830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa39c100d50 0x7fa39c104d20 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.205+0000 7fa3a3caf700 1 --2- 172.21.15.144:0/1578909830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa39c1003b0 0x7fa39c100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.205+0000 7fa3a3caf700 1 -- 172.21.15.144:0/1578909830 >> 172.21.15.144:0/1578909830 conn(0x7fa39c0fbc30 msgr2=0x7fa39c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:12.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.205+0000 7fa3a3caf700 1 -- 172.21.15.144:0/1578909830 shutdown_connections 2024-09-10T12:00:12.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.205+0000 7fa3a3caf700 1 -- 172.21.15.144:0/1578909830 wait complete. 2024-09-10T12:00:12.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.206+0000 7fa3a3caf700 1 Processor -- start 2024-09-10T12:00:12.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.206+0000 7fa3a3caf700 1 -- start start 2024-09-10T12:00:12.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.206+0000 7fa3a3caf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa39c1003b0 0x7fa39c195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:12.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.206+0000 7fa3a3caf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa39c100d50 0x7fa39c196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:12.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.206+0000 7fa3a3caf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c1969e0 con 0x7fa39c100d50 2024-09-10T12:00:12.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.206+0000 7fa3a3caf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa39c19a770 con 0x7fa39c1003b0 2024-09-10T12:00:12.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.206+0000 7fa3a124a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa39c100d50 0x7fa39c196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:12.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.206+0000 7fa3a124a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa39c100d50 0x7fa39c196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41682/0 (socket says 172.21.15.144:41682) 2024-09-10T12:00:12.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.206+0000 7fa3a124a700 1 -- 172.21.15.144:0/2701538549 learned_addr learned my addr 172.21.15.144:0/2701538549 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:00:12.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.206+0000 7fa3a1a4b700 1 --2- 172.21.15.144:0/2701538549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa39c1003b0 0x7fa39c195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:12.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.206+0000 7fa3a124a700 1 -- 172.21.15.144:0/2701538549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa39c1003b0 msgr2=0x7fa39c195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:12.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.206+0000 7fa3a124a700 1 --2- 172.21.15.144:0/2701538549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa39c1003b0 0x7fa39c195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.206+0000 7fa3a124a700 1 -- 172.21.15.144:0/2701538549 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38c0095f0 con 0x7fa39c100d50 2024-09-10T12:00:12.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.206+0000 7fa3a124a700 1 --2- 172.21.15.144:0/2701538549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa39c100d50 0x7fa39c196300 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0092e0 tx=0x7fa38c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:12.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.207+0000 7fa392ffd700 1 -- 172.21.15.144:0/2701538549 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa38c01d070 con 0x7fa39c100d50 2024-09-10T12:00:12.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.207+0000 7fa392ffd700 1 -- 172.21.15.144:0/2701538549 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa38c005dc0 con 0x7fa39c100d50 2024-09-10T12:00:12.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.207+0000 7fa3a3caf700 1 -- 172.21.15.144:0/2701538549 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39c19a9f0 con 0x7fa39c100d50 2024-09-10T12:00:12.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.207+0000 7fa392ffd700 1 -- 172.21.15.144:0/2701538549 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa38c017400 con 0x7fa39c100d50 2024-09-10T12:00:12.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.207+0000 7fa3a3caf700 1 -- 172.21.15.144:0/2701538549 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39c19aee0 con 0x7fa39c100d50 2024-09-10T12:00:12.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.208+0000 7fa3a3caf700 1 -- 172.21.15.144:0/2701538549 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39c108720 con 0x7fa39c100d50 2024-09-10T12:00:12.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.208+0000 7fa392ffd700 1 -- 172.21.15.144:0/2701538549 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa38c00f6f0 con 0x7fa39c100d50 2024-09-10T12:00:12.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.209+0000 7fa392ffd700 1 --2- 172.21.15.144:0/2701538549 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa38806e200 0x7fa3880706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:12.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.209+0000 7fa392ffd700 1 -- 172.21.15.144:0/2701538549 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa38c08f470 con 0x7fa39c100d50 2024-09-10T12:00:12.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.209+0000 7fa3a1a4b700 1 --2- 172.21.15.144:0/2701538549 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa38806e200 0x7fa3880706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:12.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.209+0000 7fa3a1a4b700 1 --2- 172.21.15.144:0/2701538549 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa38806e200 0x7fa3880706b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa398009550 tx=0x7fa3980091b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:12.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.213+0000 7fa392ffd700 1 -- 172.21.15.144:0/2701538549 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa38c058850 con 0x7fa39c100d50 2024-09-10T12:00:12.434 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.433+0000 7fa3a3caf700 1 -- 172.21.15.144:0/2701538549 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa39c02cca0 con 0x7fa38806e200 2024-09-10T12:00:12.435 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.434+0000 7fa392ffd700 1 -- 172.21.15.144:0/2701538549 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa39c02cca0 con 0x7fa38806e200 2024-09-10T12:00:12.436 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:00:12.436 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:00:12.436 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:00:12.436 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:00:12.436 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:00:12.436 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:00:12.436 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:00:12.436 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:00:12.436 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:00:12.437 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:00:12.437 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:00:12.437 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:00:12.437 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:00:12.437 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:00:12.437 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:00:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.436+0000 7fa3a3caf700 1 -- 172.21.15.144:0/2701538549 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa38806e200 msgr2=0x7fa3880706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.436+0000 7fa3a3caf700 1 --2- 172.21.15.144:0/2701538549 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa38806e200 0x7fa3880706b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fa398009550 tx=0x7fa3980091b0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.436+0000 7fa3a3caf700 1 -- 172.21.15.144:0/2701538549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa39c100d50 msgr2=0x7fa39c196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.436+0000 7fa3a3caf700 1 --2- 172.21.15.144:0/2701538549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa39c100d50 0x7fa39c196300 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fa38c0092e0 tx=0x7fa38c00bd70 comp rx=0 tx=0).stop 2024-09-10T12:00:12.439 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.436+0000 7fa3a3caf700 1 -- 172.21.15.144:0/2701538549 shutdown_connections 2024-09-10T12:00:12.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.436+0000 7fa3a3caf700 1 --2- 172.21.15.144:0/2701538549 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa38806e200 0x7fa3880706b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.436+0000 7fa3a3caf700 1 --2- 172.21.15.144:0/2701538549 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa39c1003b0 0x7fa39c195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.436+0000 7fa3a3caf700 1 --2- 172.21.15.144:0/2701538549 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa39c100d50 0x7fa39c196300 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.436+0000 7fa3a3caf700 1 -- 172.21.15.144:0/2701538549 >> 172.21.15.144:0/2701538549 conn(0x7fa39c0fbc30 msgr2=0x7fa39c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:12.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.436+0000 7fa3a3caf700 1 -- 172.21.15.144:0/2701538549 shutdown_connections 2024-09-10T12:00:12.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.436+0000 7fa3a3caf700 1 -- 172.21.15.144:0/2701538549 wait complete. 2024-09-10T12:00:12.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.544+0000 7fb67516b700 1 -- 172.21.15.144:0/35300201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6700ff2c0 msgr2=0x7fb6700ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:12.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.544+0000 7fb67516b700 1 --2- 172.21.15.144:0/35300201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6700ff2c0 0x7fb6700ff690 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb66000b0a0 tx=0x7fb66000b3b0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.544+0000 7fb67516b700 1 -- 172.21.15.144:0/35300201 shutdown_connections 2024-09-10T12:00:12.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.544+0000 7fb67516b700 1 --2- 172.21.15.144:0/35300201 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6700ffc60 0x7fb670103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.544+0000 7fb67516b700 1 --2- 172.21.15.144:0/35300201 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6700ff2c0 0x7fb6700ff690 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.544+0000 7fb67516b700 1 -- 172.21.15.144:0/35300201 >> 172.21.15.144:0/35300201 conn(0x7fb6700fab80 msgr2=0x7fb6700fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:12.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.544+0000 7fb67516b700 1 -- 172.21.15.144:0/35300201 shutdown_connections 2024-09-10T12:00:12.546 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.544+0000 7fb67516b700 1 -- 172.21.15.144:0/35300201 wait complete. 2024-09-10T12:00:12.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.545+0000 7fb67516b700 1 Processor -- start 2024-09-10T12:00:12.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.545+0000 7fb67516b700 1 -- start start 2024-09-10T12:00:12.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.545+0000 7fb67516b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6700ff2c0 0x7fb670195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:12.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.545+0000 7fb67516b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6700ffc60 0x7fb670196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:12.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.545+0000 7fb67516b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6701968b0 con 0x7fb6700ff2c0 2024-09-10T12:00:12.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.545+0000 7fb67516b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6701969f0 con 0x7fb6700ffc60 2024-09-10T12:00:12.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.545+0000 7fb66ed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6700ff2c0 0x7fb670195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:12.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.545+0000 7fb66ed9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6700ff2c0 0x7fb670195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53902/0 (socket says 172.21.15.144:53902) 2024-09-10T12:00:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.545+0000 7fb66ed9d700 1 -- 172.21.15.144:0/2660611538 learned_addr learned my addr 172.21.15.144:0/2660611538 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:00:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.546+0000 7fb66e59c700 1 --2- 172.21.15.144:0/2660611538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6700ffc60 0x7fb670196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.546+0000 7fb66ed9d700 1 -- 172.21.15.144:0/2660611538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6700ffc60 msgr2=0x7fb670196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.546+0000 7fb66ed9d700 1 --2- 172.21.15.144:0/2660611538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6700ffc60 0x7fb670196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.546+0000 7fb66ed9d700 1 -- 172.21.15.144:0/2660611538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb660009d00 con 0x7fb6700ff2c0 2024-09-10T12:00:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.546+0000 7fb66e59c700 1 --2- 172.21.15.144:0/2660611538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6700ffc60 0x7fb670196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:00:12.550 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.546+0000 7fb66ed9d700 1 --2- 172.21.15.144:0/2660611538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6700ff2c0 0x7fb670195d20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb660000f80 tx=0x7fb66000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:12.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.546+0000 7fb667fff700 1 -- 172.21.15.144:0/2660611538 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb66000e040 con 0x7fb6700ff2c0 2024-09-10T12:00:12.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.546+0000 7fb67516b700 1 -- 172.21.15.144:0/2660611538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67019a770 con 0x7fb6700ff2c0 2024-09-10T12:00:12.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.546+0000 7fb667fff700 1 -- 172.21.15.144:0/2660611538 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb660003e60 con 0x7fb6700ff2c0 2024-09-10T12:00:12.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.546+0000 7fb67516b700 1 -- 172.21.15.144:0/2660611538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67019ac60 con 0x7fb6700ff2c0 2024-09-10T12:00:12.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.546+0000 7fb667fff700 1 -- 172.21.15.144:0/2660611538 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb66001d910 con 0x7fb6700ff2c0 2024-09-10T12:00:12.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.547+0000 7fb67516b700 1 -- 172.21.15.144:0/2660611538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67004ea50 con 0x7fb6700ff2c0 2024-09-10T12:00:12.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.548+0000 7fb667fff700 1 -- 172.21.15.144:0/2660611538 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb660004820 con 0x7fb6700ff2c0 2024-09-10T12:00:12.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.548+0000 7fb667fff700 1 --2- 172.21.15.144:0/2660611538 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb65c06e200 0x7fb65c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:12.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.548+0000 7fb667fff700 1 -- 172.21.15.144:0/2660611538 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb66008f890 con 0x7fb6700ff2c0 2024-09-10T12:00:12.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.548+0000 7fb66e59c700 1 --2- 172.21.15.144:0/2660611538 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb65c06e200 0x7fb65c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:12.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.549+0000 7fb66e59c700 1 --2- 172.21.15.144:0/2660611538 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb65c06e200 0x7fb65c0706b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb658005fd0 tx=0x7fb658005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:12.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.555+0000 7fb667fff700 1 -- 172.21.15.144:0/2660611538 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb660058cf0 con 0x7fb6700ff2c0 2024-09-10T12:00:12.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.843+0000 7fb67516b700 1 -- 172.21.15.144:0/2660611538 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb67019b200 con 0x7fb6700ff2c0 2024-09-10T12:00:12.845 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:12 smithi144 ceph-mon[87436]: pgmap v847: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:12.845 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:12 smithi144 ceph-mon[87436]: from='client.35552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:12.845 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:12 smithi144 ceph-mon[87436]: from='client.35556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:12.845 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:12 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/495490143' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:00:12.845 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:12 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2094520908' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:00:12.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.843+0000 7fb667fff700 1 -- 172.21.15.144:0/2660611538 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7fb66005c3b0 con 0x7fb6700ff2c0 2024-09-10T12:00:12.845 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:00:12.846 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:00:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:00:12.846 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:00:12.846 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:00:12.846 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:00:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:00:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:00:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:00:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:00:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:00:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:00:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:00:12.846 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:00:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:00:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:00:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:00:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:00:12.847 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:00:12.847 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:00:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:00:12.847 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:00:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:00:12.847 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:00:12.847 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:00:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.845+0000 7fb67516b700 1 -- 172.21.15.144:0/2660611538 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb65c06e200 msgr2=0x7fb65c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.845+0000 7fb67516b700 1 --2- 172.21.15.144:0/2660611538 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb65c06e200 0x7fb65c0706b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fb658005fd0 tx=0x7fb658005e30 comp rx=0 tx=0).stop 2024-09-10T12:00:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.845+0000 7fb67516b700 1 -- 172.21.15.144:0/2660611538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6700ff2c0 msgr2=0x7fb670195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.845+0000 7fb67516b700 1 --2- 172.21.15.144:0/2660611538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6700ff2c0 0x7fb670195d20 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb660000f80 tx=0x7fb66000cdd0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.846+0000 7fb67516b700 1 -- 172.21.15.144:0/2660611538 shutdown_connections 2024-09-10T12:00:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.846+0000 7fb67516b700 1 --2- 172.21.15.144:0/2660611538 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb65c06e200 0x7fb65c0706b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.846+0000 7fb67516b700 1 --2- 172.21.15.144:0/2660611538 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb6700ff2c0 0x7fb670195d20 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.846+0000 7fb67516b700 1 --2- 172.21.15.144:0/2660611538 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb6700ffc60 0x7fb670196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:12.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.846+0000 7fb67516b700 1 -- 172.21.15.144:0/2660611538 >> 172.21.15.144:0/2660611538 conn(0x7fb6700fab80 msgr2=0x7fb670103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:12.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.846+0000 7fb67516b700 1 -- 172.21.15.144:0/2660611538 shutdown_connections 2024-09-10T12:00:12.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:12.846+0000 7fb67516b700 1 -- 172.21.15.144:0/2660611538 wait complete. 2024-09-10T12:00:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:12 smithi178 ceph-mon[65940]: pgmap v847: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:12 smithi178 ceph-mon[65940]: from='client.35552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:12 smithi178 ceph-mon[65940]: from='client.35556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:12.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:12 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/495490143' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:00:12.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:12 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2094520908' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:00:13.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:13 smithi178 ceph-mon[65940]: from='client.35568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:13.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:13 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2660611538' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:00:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:13 smithi144 ceph-mon[87436]: from='client.35568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:13.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:13 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2660611538' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:00:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:14 smithi178 ceph-mon[65940]: pgmap v848: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:14 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:00:14.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:14 smithi144 ceph-mon[87436]: pgmap v848: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:14.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:00:16.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:16 smithi178 ceph-mon[65940]: pgmap v849: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:16.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:16 smithi144 ceph-mon[87436]: pgmap v849: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:18.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:18 smithi178 ceph-mon[65940]: pgmap v850: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:18 smithi144 ceph-mon[87436]: pgmap v850: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:20 smithi178 ceph-mon[65940]: pgmap v851: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:20.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:20 smithi144 ceph-mon[87436]: pgmap v851: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:22.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:22 smithi178 ceph-mon[65940]: pgmap v852: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:22 smithi144 ceph-mon[87436]: pgmap v852: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:24.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:24 smithi178 ceph-mon[65940]: pgmap v853: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:24.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:24 smithi144 ceph-mon[87436]: pgmap v853: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:26.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:26 smithi178 ceph-mon[65940]: pgmap v854: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:26.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:26 smithi144 ceph-mon[87436]: pgmap v854: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:28.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:28 smithi178 ceph-mon[65940]: pgmap v855: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:28.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:28 smithi144 ceph-mon[87436]: pgmap v855: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:29.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:29 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:00:29.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:29 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:00:30.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:30 smithi178 ceph-mon[65940]: pgmap v856: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:30.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:30 smithi144 ceph-mon[87436]: pgmap v856: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:32.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:32 smithi178 ceph-mon[65940]: pgmap v857: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:32.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:32 smithi144 ceph-mon[87436]: pgmap v857: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:34.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:34 smithi144 ceph-mon[87436]: pgmap v858: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:34 smithi178 ceph-mon[65940]: pgmap v858: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:36.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:36 smithi144 ceph-mon[87436]: pgmap v859: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:36 smithi178 ceph-mon[65940]: pgmap v859: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:38.919 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:38 smithi178 ceph-mon[65940]: pgmap v860: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:38.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:38 smithi144 ceph-mon[87436]: pgmap v860: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:40.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:40 smithi144 ceph-mon[87436]: pgmap v861: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:40 smithi178 ceph-mon[65940]: pgmap v861: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:42 smithi144 ceph-mon[87436]: pgmap v862: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:42.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.966+0000 7fb2b47ac700 1 -- 172.21.15.144:0/2053972455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2ac0ffcb0 msgr2=0x7fb2ac103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:42.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.966+0000 7fb2b47ac700 1 --2- 172.21.15.144:0/2053972455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2ac0ffcb0 0x7fb2ac103c80 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb29c009910 tx=0x7fb29c009c20 comp rx=0 tx=0).stop 2024-09-10T12:00:42.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.966+0000 7fb2b47ac700 1 -- 172.21.15.144:0/2053972455 shutdown_connections 2024-09-10T12:00:42.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.966+0000 7fb2b47ac700 1 --2- 172.21.15.144:0/2053972455 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2ac0ffcb0 0x7fb2ac103c80 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:42.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.966+0000 7fb2b47ac700 1 --2- 172.21.15.144:0/2053972455 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2ac0ff310 0x7fb2ac0ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:42.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.966+0000 7fb2b47ac700 1 -- 172.21.15.144:0/2053972455 >> 172.21.15.144:0/2053972455 conn(0x7fb2ac0fabb0 msgr2=0x7fb2ac0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:42.969 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.966+0000 7fb2b47ac700 1 -- 172.21.15.144:0/2053972455 shutdown_connections 2024-09-10T12:00:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.966+0000 7fb2b47ac700 1 -- 172.21.15.144:0/2053972455 wait complete. 2024-09-10T12:00:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.966+0000 7fb2b47ac700 1 Processor -- start 2024-09-10T12:00:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.966+0000 7fb2b47ac700 1 -- start start 2024-09-10T12:00:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.967+0000 7fb2b47ac700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2ac0ff310 0x7fb2ac196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.967+0000 7fb2b47ac700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2ac0ffcb0 0x7fb2ac197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.967+0000 7fb2b47ac700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac197750 con 0x7fb2ac0ffcb0 2024-09-10T12:00:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.967+0000 7fb2b47ac700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac197890 con 0x7fb2ac0ff310 2024-09-10T12:00:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.967+0000 7fb2b1d47700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2ac0ffcb0 0x7fb2ac197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.967+0000 7fb2b1d47700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2ac0ffcb0 0x7fb2ac197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52432/0 (socket says 172.21.15.144:52432) 2024-09-10T12:00:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.967+0000 7fb2b1d47700 1 -- 172.21.15.144:0/2852588100 learned_addr learned my addr 172.21.15.144:0/2852588100 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:00:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.967+0000 7fb2b2548700 1 --2- 172.21.15.144:0/2852588100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2ac0ff310 0x7fb2ac196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.967+0000 7fb2b1d47700 1 -- 172.21.15.144:0/2852588100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2ac0ff310 msgr2=0x7fb2ac196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:42.970 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.967+0000 7fb2b1d47700 1 --2- 172.21.15.144:0/2852588100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2ac0ff310 0x7fb2ac196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.967+0000 7fb2b1d47700 1 -- 172.21.15.144:0/2852588100 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c0095f0 con 0x7fb2ac0ffcb0 2024-09-10T12:00:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.967+0000 7fb2b2548700 1 --2- 172.21.15.144:0/2852588100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2ac0ff310 0x7fb2ac196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:00:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.967+0000 7fb2b1d47700 1 --2- 172.21.15.144:0/2852588100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2ac0ffcb0 0x7fb2ac197130 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0092e0 tx=0x7fb29c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.967+0000 7fb2a37fe700 1 -- 172.21.15.144:0/2852588100 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb29c01d070 con 0x7fb2ac0ffcb0 2024-09-10T12:00:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.968+0000 7fb2b47ac700 1 -- 172.21.15.144:0/2852588100 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ac19fc60 con 0x7fb2ac0ffcb0 2024-09-10T12:00:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.968+0000 7fb2a37fe700 1 -- 172.21.15.144:0/2852588100 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb29c005dc0 con 0x7fb2ac0ffcb0 2024-09-10T12:00:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.968+0000 7fb2a37fe700 1 -- 172.21.15.144:0/2852588100 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb29c017400 con 0x7fb2ac0ffcb0 2024-09-10T12:00:42.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.968+0000 7fb2b47ac700 1 -- 172.21.15.144:0/2852588100 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ac1a0070 con 0x7fb2ac0ffcb0 2024-09-10T12:00:42.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.968+0000 7fb2b47ac700 1 -- 172.21.15.144:0/2852588100 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ac107680 con 0x7fb2ac0ffcb0 2024-09-10T12:00:42.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.969+0000 7fb2a37fe700 1 -- 172.21.15.144:0/2852588100 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb29c00f460 con 0x7fb2ac0ffcb0 2024-09-10T12:00:42.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.970+0000 7fb2a37fe700 1 --2- 172.21.15.144:0/2852588100 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb29806e200 0x7fb2980706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:42.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.970+0000 7fb2a37fe700 1 -- 172.21.15.144:0/2852588100 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb29c08f570 con 0x7fb2ac0ffcb0 2024-09-10T12:00:42.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.970+0000 7fb2b2548700 1 --2- 172.21.15.144:0/2852588100 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb29806e200 0x7fb2980706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:42.973 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.970+0000 7fb2b2548700 1 --2- 172.21.15.144:0/2852588100 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb29806e200 0x7fb2980706b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8005950 tx=0x7fb2a80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:42.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:42.974+0000 7fb2a37fe700 1 -- 172.21.15.144:0/2852588100 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb29c058950 con 0x7fb2ac0ffcb0 2024-09-10T12:00:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:42 smithi178 ceph-mon[65940]: pgmap v862: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:43.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.194+0000 7fb2b47ac700 1 -- 172.21.15.144:0/2852588100 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2ac02cd70 con 0x7fb29806e200 2024-09-10T12:00:43.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.196+0000 7fb2a37fe700 1 -- 172.21.15.144:0/2852588100 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb2ac02cd70 con 0x7fb29806e200 2024-09-10T12:00:43.199 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.198+0000 7fb2b47ac700 1 -- 172.21.15.144:0/2852588100 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb29806e200 msgr2=0x7fb2980706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:43.199 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.198+0000 7fb2b47ac700 1 --2- 172.21.15.144:0/2852588100 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb29806e200 0x7fb2980706b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8005950 tx=0x7fb2a80058e0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.199 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.198+0000 7fb2b47ac700 1 -- 172.21.15.144:0/2852588100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2ac0ffcb0 msgr2=0x7fb2ac197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:43.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.198+0000 7fb2b47ac700 1 --2- 172.21.15.144:0/2852588100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2ac0ffcb0 0x7fb2ac197130 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fb29c0092e0 tx=0x7fb29c00be20 comp rx=0 tx=0).stop 2024-09-10T12:00:43.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.198+0000 7fb2b47ac700 1 -- 172.21.15.144:0/2852588100 shutdown_connections 2024-09-10T12:00:43.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.198+0000 7fb2b47ac700 1 --2- 172.21.15.144:0/2852588100 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb29806e200 0x7fb2980706b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.198+0000 7fb2b47ac700 1 --2- 172.21.15.144:0/2852588100 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2ac0ff310 0x7fb2ac196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.198+0000 7fb2b47ac700 1 --2- 172.21.15.144:0/2852588100 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2ac0ffcb0 0x7fb2ac197130 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.198+0000 7fb2b47ac700 1 -- 172.21.15.144:0/2852588100 >> 172.21.15.144:0/2852588100 conn(0x7fb2ac0fabb0 msgr2=0x7fb2ac0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:43.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.198+0000 7fb2b47ac700 1 -- 172.21.15.144:0/2852588100 shutdown_connections 2024-09-10T12:00:43.200 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.198+0000 7fb2b47ac700 1 -- 172.21.15.144:0/2852588100 wait complete. 2024-09-10T12:00:43.210 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:00:43.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.304+0000 7f9f52d50700 1 -- 172.21.15.144:0/3860934224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f4c100d70 msgr2=0x7f9f4c104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:43.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.304+0000 7f9f52d50700 1 --2- 172.21.15.144:0/3860934224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f4c100d70 0x7f9f4c104d40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c00b0a0 tx=0x7f9f3c00b3b0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.304+0000 7f9f52d50700 1 -- 172.21.15.144:0/3860934224 shutdown_connections 2024-09-10T12:00:43.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.304+0000 7f9f52d50700 1 --2- 172.21.15.144:0/3860934224 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f4c100d70 0x7f9f4c104d40 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.307 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.304+0000 7f9f52d50700 1 --2- 172.21.15.144:0/3860934224 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f4c1003d0 0x7f9f4c1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.307 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.304+0000 7f9f52d50700 1 -- 172.21.15.144:0/3860934224 >> 172.21.15.144:0/3860934224 conn(0x7f9f4c0fbc50 msgr2=0x7f9f4c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:43.307 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.304+0000 7f9f52d50700 1 -- 172.21.15.144:0/3860934224 shutdown_connections 2024-09-10T12:00:43.307 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.304+0000 7f9f52d50700 1 -- 172.21.15.144:0/3860934224 wait complete. 2024-09-10T12:00:43.307 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.305+0000 7f9f52d50700 1 Processor -- start 2024-09-10T12:00:43.307 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.305+0000 7f9f52d50700 1 -- start start 2024-09-10T12:00:43.310 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.305+0000 7f9f52d50700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f4c1003d0 0x7f9f4c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:43.310 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.305+0000 7f9f52d50700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f4c100d70 0x7f9f4c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:43.310 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.305+0000 7f9f52d50700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c073ad0 con 0x7f9f4c1003d0 2024-09-10T12:00:43.310 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.305+0000 7f9f52d50700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4c073c10 con 0x7f9f4c100d70 2024-09-10T12:00:43.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.305+0000 7f9f50aec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f4c1003d0 0x7f9f4c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:43.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.305+0000 7f9f50aec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f4c1003d0 0x7f9f4c073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52444/0 (socket says 172.21.15.144:52444) 2024-09-10T12:00:43.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.305+0000 7f9f50aec700 1 -- 172.21.15.144:0/4268536178 learned_addr learned my addr 172.21.15.144:0/4268536178 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:00:43.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.306+0000 7f9f4bfff700 1 --2- 172.21.15.144:0/4268536178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f4c100d70 0x7f9f4c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:43.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.306+0000 7f9f50aec700 1 -- 172.21.15.144:0/4268536178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f4c100d70 msgr2=0x7f9f4c073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:43.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.306+0000 7f9f50aec700 1 --2- 172.21.15.144:0/4268536178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f4c100d70 0x7f9f4c073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.306+0000 7f9f50aec700 1 -- 172.21.15.144:0/4268536178 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f3c009d00 con 0x7f9f4c1003d0 2024-09-10T12:00:43.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.306+0000 7f9f4bfff700 1 --2- 172.21.15.144:0/4268536178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f4c100d70 0x7f9f4c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:00:43.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.306+0000 7f9f50aec700 1 --2- 172.21.15.144:0/4268536178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f4c1003d0 0x7f9f4c073050 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400d8d0 tx=0x7f9f3400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:43.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.306+0000 7f9f49ffb700 1 -- 172.21.15.144:0/4268536178 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f34009940 con 0x7f9f4c1003d0 2024-09-10T12:00:43.311 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.306+0000 7f9f49ffb700 1 -- 172.21.15.144:0/4268536178 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f34010460 con 0x7f9f4c1003d0 2024-09-10T12:00:43.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.306+0000 7f9f49ffb700 1 -- 172.21.15.144:0/4268536178 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f3400f5f0 con 0x7f9f4c1003d0 2024-09-10T12:00:43.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.306+0000 7f9f52d50700 1 -- 172.21.15.144:0/4268536178 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f4c1a1ff0 con 0x7f9f4c1003d0 2024-09-10T12:00:43.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.306+0000 7f9f52d50700 1 -- 172.21.15.144:0/4268536178 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f4c1a23a0 con 0x7f9f4c1003d0 2024-09-10T12:00:43.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.308+0000 7f9f52d50700 1 -- 172.21.15.144:0/4268536178 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f4c108740 con 0x7f9f4c1003d0 2024-09-10T12:00:43.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.308+0000 7f9f49ffb700 1 -- 172.21.15.144:0/4268536178 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9f340105d0 con 0x7f9f4c1003d0 2024-09-10T12:00:43.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.308+0000 7f9f49ffb700 1 --2- 172.21.15.144:0/4268536178 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f3806e200 0x7f9f380706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:43.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.308+0000 7f9f49ffb700 1 -- 172.21.15.144:0/4268536178 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9f3408df20 con 0x7f9f4c1003d0 2024-09-10T12:00:43.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.308+0000 7f9f4bfff700 1 --2- 172.21.15.144:0/4268536178 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f3806e200 0x7f9f380706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:43.312 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.309+0000 7f9f4bfff700 1 --2- 172.21.15.144:0/4268536178 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f3806e200 0x7f9f380706b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c000f80 tx=0x7f9f3c006210 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:43.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.314+0000 7f9f49ffb700 1 -- 172.21.15.144:0/4268536178 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9f340c31b0 con 0x7f9f4c1003d0 2024-09-10T12:00:43.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.536+0000 7f9f52d50700 1 -- 172.21.15.144:0/4268536178 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f4c061190 con 0x7f9f3806e200 2024-09-10T12:00:43.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.537+0000 7f9f49ffb700 1 -- 172.21.15.144:0/4268536178 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9f4c061190 con 0x7f9f3806e200 2024-09-10T12:00:43.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.539+0000 7f9f52d50700 1 -- 172.21.15.144:0/4268536178 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f3806e200 msgr2=0x7f9f380706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:43.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.539+0000 7f9f52d50700 1 --2- 172.21.15.144:0/4268536178 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f3806e200 0x7f9f380706b0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c000f80 tx=0x7f9f3c006210 comp rx=0 tx=0).stop 2024-09-10T12:00:43.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.539+0000 7f9f52d50700 1 -- 172.21.15.144:0/4268536178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f4c1003d0 msgr2=0x7f9f4c073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:43.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.539+0000 7f9f52d50700 1 --2- 172.21.15.144:0/4268536178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f4c1003d0 0x7f9f4c073050 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400d8d0 tx=0x7f9f3400dc90 comp rx=0 tx=0).stop 2024-09-10T12:00:43.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.539+0000 7f9f52d50700 1 -- 172.21.15.144:0/4268536178 shutdown_connections 2024-09-10T12:00:43.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.539+0000 7f9f52d50700 1 --2- 172.21.15.144:0/4268536178 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9f3806e200 0x7f9f380706b0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.539+0000 7f9f52d50700 1 --2- 172.21.15.144:0/4268536178 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9f4c1003d0 0x7f9f4c073050 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.539+0000 7f9f52d50700 1 --2- 172.21.15.144:0/4268536178 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9f4c100d70 0x7f9f4c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.539+0000 7f9f52d50700 1 -- 172.21.15.144:0/4268536178 >> 172.21.15.144:0/4268536178 conn(0x7f9f4c0fbc50 msgr2=0x7f9f4c0fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:43.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.539+0000 7f9f52d50700 1 -- 172.21.15.144:0/4268536178 shutdown_connections 2024-09-10T12:00:43.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.539+0000 7f9f52d50700 1 -- 172.21.15.144:0/4268536178 wait complete. 2024-09-10T12:00:43.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.645+0000 7fc1d43b9700 1 -- 172.21.15.144:0/730800213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1cc0ff320 msgr2=0x7fc1cc0ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:43.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.645+0000 7fc1d43b9700 1 --2- 172.21.15.144:0/730800213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1cc0ff320 0x7fc1cc0ff6f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8009940 tx=0x7fc1c8009c50 comp rx=0 tx=0).stop 2024-09-10T12:00:43.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.646+0000 7fc1d43b9700 1 -- 172.21.15.144:0/730800213 shutdown_connections 2024-09-10T12:00:43.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.646+0000 7fc1d43b9700 1 --2- 172.21.15.144:0/730800213 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1cc0ffc30 0x7fc1cc10a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.646+0000 7fc1d43b9700 1 --2- 172.21.15.144:0/730800213 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1cc0ff320 0x7fc1cc0ff6f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.646+0000 7fc1d43b9700 1 -- 172.21.15.144:0/730800213 >> 172.21.15.144:0/730800213 conn(0x7fc1cc0748b0 msgr2=0x7fc1cc074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:43.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.646+0000 7fc1d43b9700 1 -- 172.21.15.144:0/730800213 shutdown_connections 2024-09-10T12:00:43.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.646+0000 7fc1d43b9700 1 -- 172.21.15.144:0/730800213 wait complete. 2024-09-10T12:00:43.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.646+0000 7fc1d43b9700 1 Processor -- start 2024-09-10T12:00:43.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.646+0000 7fc1d43b9700 1 -- start start 2024-09-10T12:00:43.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.646+0000 7fc1d43b9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1cc0ff320 0x7fc1cc100500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:43.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1d43b9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1cc0ffc30 0x7fc1cc100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:43.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1d43b9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc100f80 con 0x7fc1cc0ff320 2024-09-10T12:00:43.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1d43b9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc1010c0 con 0x7fc1cc0ffc30 2024-09-10T12:00:43.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1d2155700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1cc0ff320 0x7fc1cc100500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:43.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1d2155700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1cc0ff320 0x7fc1cc100500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52464/0 (socket says 172.21.15.144:52464) 2024-09-10T12:00:43.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1d2155700 1 -- 172.21.15.144:0/2971325838 learned_addr learned my addr 172.21.15.144:0/2971325838 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:00:43.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1d1954700 1 --2- 172.21.15.144:0/2971325838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1cc0ffc30 0x7fc1cc100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:43.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1d2155700 1 -- 172.21.15.144:0/2971325838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1cc0ffc30 msgr2=0x7fc1cc100a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:43.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1d2155700 1 --2- 172.21.15.144:0/2971325838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1cc0ffc30 0x7fc1cc100a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1d2155700 1 -- 172.21.15.144:0/2971325838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c80095f0 con 0x7fc1cc0ff320 2024-09-10T12:00:43.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1d2155700 1 --2- 172.21.15.144:0/2971325838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1cc0ff320 0x7fc1cc100500 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80055f0 tx=0x7fc1c80056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:43.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1c37fe700 1 -- 172.21.15.144:0/2971325838 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1c801d070 con 0x7fc1cc0ff320 2024-09-10T12:00:43.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1c37fe700 1 -- 172.21.15.144:0/2971325838 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc1c800f460 con 0x7fc1cc0ff320 2024-09-10T12:00:43.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1d43b9700 1 -- 172.21.15.144:0/2971325838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1cc1a1e20 con 0x7fc1cc0ff320 2024-09-10T12:00:43.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1c37fe700 1 -- 172.21.15.144:0/2971325838 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1c8021640 con 0x7fc1cc0ff320 2024-09-10T12:00:43.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.647+0000 7fc1d43b9700 1 -- 172.21.15.144:0/2971325838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc1a2230 con 0x7fc1cc0ff320 2024-09-10T12:00:43.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.648+0000 7fc1d43b9700 1 -- 172.21.15.144:0/2971325838 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1cc107da0 con 0x7fc1cc0ff320 2024-09-10T12:00:43.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.649+0000 7fc1c37fe700 1 -- 172.21.15.144:0/2971325838 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc1c800f5d0 con 0x7fc1cc0ff320 2024-09-10T12:00:43.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.649+0000 7fc1c37fe700 1 --2- 172.21.15.144:0/2971325838 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc1b806e200 0x7fc1b80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:43.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.649+0000 7fc1c37fe700 1 -- 172.21.15.144:0/2971325838 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc1c808f370 con 0x7fc1cc0ff320 2024-09-10T12:00:43.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.649+0000 7fc1d1954700 1 --2- 172.21.15.144:0/2971325838 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc1b806e200 0x7fc1b80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:43.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.650+0000 7fc1d1954700 1 --2- 172.21.15.144:0/2971325838 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc1b806e200 0x7fc1b80706b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc005fd0 tx=0x7fc1bc00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:43.655 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.654+0000 7fc1c37fe700 1 -- 172.21.15.144:0/2971325838 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc1c80587d0 con 0x7fc1cc0ff320 2024-09-10T12:00:43.865 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.864+0000 7fc1d43b9700 1 -- 172.21.15.144:0/2971325838 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1cc02cc80 con 0x7fc1b806e200 2024-09-10T12:00:43.874 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.873+0000 7fc1c37fe700 1 -- 172.21.15.144:0/2971325838 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fc1cc02cc80 con 0x7fc1b806e200 2024-09-10T12:00:43.875 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:00:43.875 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (28m) 7m ago 39m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:00:43.875 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (39m) 7m ago 39m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:00:43.875 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (38m) 7m ago 38m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:00:43.876 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (26m) 7m ago 39m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:00:43.876 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (26m) 7m ago 38m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:00:43.876 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (28m) 7m ago 39m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:00:43.876 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (23m) 7m ago 34m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:00:43.876 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (22m) 7m ago 33m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:00:43.876 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (7m) 7m ago 33m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:00:43.876 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (22m) 7m ago 34m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:00:43.877 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (30m) 7m ago 40m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:00:43.877 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (29m) 7m ago 38m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:00:43.877 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (27m) 7m ago 41m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:00:43.877 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (27m) 7m ago 38m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:00:43.877 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (29m) 7m ago 39m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:00:43.877 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (29m) 7m ago 38m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:00:43.877 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (26m) 7m ago 37m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:00:43.877 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (25m) 7m ago 37m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:00:43.877 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (25m) 7m ago 36m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:00:43.877 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (24m) 7m ago 36m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:00:43.877 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (23m) 7m ago 35m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:00:43.877 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (23m) 7m ago 35m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:00:43.877 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (29m) 7m ago 39m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:00:43.880 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.876+0000 7fc1d43b9700 1 -- 172.21.15.144:0/2971325838 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc1b806e200 msgr2=0x7fc1b80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:43.880 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.876+0000 7fc1d43b9700 1 --2- 172.21.15.144:0/2971325838 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc1b806e200 0x7fc1b80706b0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc005fd0 tx=0x7fc1bc00a560 comp rx=0 tx=0).stop 2024-09-10T12:00:43.880 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.876+0000 7fc1d43b9700 1 -- 172.21.15.144:0/2971325838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1cc0ff320 msgr2=0x7fc1cc100500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:43.880 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.876+0000 7fc1d43b9700 1 --2- 172.21.15.144:0/2971325838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1cc0ff320 0x7fc1cc100500 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fc1c80055f0 tx=0x7fc1c80056d0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.880 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.876+0000 7fc1d43b9700 1 -- 172.21.15.144:0/2971325838 shutdown_connections 2024-09-10T12:00:43.880 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.876+0000 7fc1d43b9700 1 --2- 172.21.15.144:0/2971325838 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc1b806e200 0x7fc1b80706b0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.880 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.876+0000 7fc1d43b9700 1 --2- 172.21.15.144:0/2971325838 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc1cc0ff320 0x7fc1cc100500 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.880 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.876+0000 7fc1d43b9700 1 --2- 172.21.15.144:0/2971325838 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc1cc0ffc30 0x7fc1cc100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.880 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.876+0000 7fc1d43b9700 1 -- 172.21.15.144:0/2971325838 >> 172.21.15.144:0/2971325838 conn(0x7fc1cc0748b0 msgr2=0x7fc1cc104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:43.880 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.876+0000 7fc1d43b9700 1 -- 172.21.15.144:0/2971325838 shutdown_connections 2024-09-10T12:00:43.880 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.876+0000 7fc1d43b9700 1 -- 172.21.15.144:0/2971325838 wait complete. 2024-09-10T12:00:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:00:43.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.984+0000 7f60aca90700 1 -- 172.21.15.144:0/1573958803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a80ffd30 msgr2=0x7f60a8103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:43.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.984+0000 7f60aca90700 1 --2- 172.21.15.144:0/1573958803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a80ffd30 0x7f60a8103ce0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f6098009910 tx=0x7f6098009c20 comp rx=0 tx=0).stop 2024-09-10T12:00:43.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.984+0000 7f60aca90700 1 -- 172.21.15.144:0/1573958803 shutdown_connections 2024-09-10T12:00:43.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.984+0000 7f60aca90700 1 --2- 172.21.15.144:0/1573958803 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a80ffd30 0x7f60a8103ce0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.984+0000 7f60aca90700 1 --2- 172.21.15.144:0/1573958803 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60a80ff390 0x7f60a80ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.984+0000 7f60aca90700 1 -- 172.21.15.144:0/1573958803 >> 172.21.15.144:0/1573958803 conn(0x7f60a8074f50 msgr2=0x7f60a8075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:43.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.984+0000 7f60aca90700 1 -- 172.21.15.144:0/1573958803 shutdown_connections 2024-09-10T12:00:43.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.984+0000 7f60aca90700 1 -- 172.21.15.144:0/1573958803 wait complete. 2024-09-10T12:00:43.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.985+0000 7f60aca90700 1 Processor -- start 2024-09-10T12:00:43.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.985+0000 7f60aca90700 1 -- start start 2024-09-10T12:00:43.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.985+0000 7f60aca90700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a80ff390 0x7f60a8195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:43.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.985+0000 7f60aca90700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60a80ffd30 0x7f60a81962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:43.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.985+0000 7f60aca90700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a81969b0 con 0x7f60a80ff390 2024-09-10T12:00:43.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.985+0000 7f60aca90700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a819a740 con 0x7f60a80ffd30 2024-09-10T12:00:43.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.985+0000 7f60a659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a80ff390 0x7f60a8195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:43.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.985+0000 7f60a659c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a80ff390 0x7f60a8195d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52486/0 (socket says 172.21.15.144:52486) 2024-09-10T12:00:43.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.985+0000 7f60a659c700 1 -- 172.21.15.144:0/323980745 learned_addr learned my addr 172.21.15.144:0/323980745 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:00:43.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.986+0000 7f60a5d9b700 1 --2- 172.21.15.144:0/323980745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60a80ffd30 0x7f60a81962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:43.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.986+0000 7f60a659c700 1 -- 172.21.15.144:0/323980745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60a80ffd30 msgr2=0x7f60a81962d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:43.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.986+0000 7f60a659c700 1 --2- 172.21.15.144:0/323980745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60a80ffd30 0x7f60a81962d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:43.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.986+0000 7f60a659c700 1 -- 172.21.15.144:0/323980745 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60980095f0 con 0x7f60a80ff390 2024-09-10T12:00:43.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.986+0000 7f60a5d9b700 1 --2- 172.21.15.144:0/323980745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60a80ffd30 0x7f60a81962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:00:43.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.986+0000 7f60a659c700 1 --2- 172.21.15.144:0/323980745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a80ff390 0x7f60a8195d90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f609c00d900 tx=0x7f609c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:43.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.986+0000 7f60977fe700 1 -- 172.21.15.144:0/323980745 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f609c0041f0 con 0x7f60a80ff390 2024-09-10T12:00:43.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.986+0000 7f60aca90700 1 -- 172.21.15.144:0/323980745 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a819aa20 con 0x7f60a80ff390 2024-09-10T12:00:43.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.986+0000 7f60977fe700 1 -- 172.21.15.144:0/323980745 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f609c004cf0 con 0x7f60a80ff390 2024-09-10T12:00:43.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.986+0000 7f60977fe700 1 -- 172.21.15.144:0/323980745 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f609c00b750 con 0x7f60a80ff390 2024-09-10T12:00:43.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.986+0000 7f60aca90700 1 -- 172.21.15.144:0/323980745 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a819af70 con 0x7f60a80ff390 2024-09-10T12:00:43.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.987+0000 7f60aca90700 1 -- 172.21.15.144:0/323980745 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a804ea50 con 0x7f60a80ff390 2024-09-10T12:00:43.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.988+0000 7f60977fe700 1 -- 172.21.15.144:0/323980745 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f609c004350 con 0x7f60a80ff390 2024-09-10T12:00:43.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.988+0000 7f60977fe700 1 --2- 172.21.15.144:0/323980745 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f609006e200 0x7f60900706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:43.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.988+0000 7f60977fe700 1 -- 172.21.15.144:0/323980745 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f609c08dfc0 con 0x7f60a80ff390 2024-09-10T12:00:43.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.988+0000 7f60a5d9b700 1 --2- 172.21.15.144:0/323980745 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f609006e200 0x7f60900706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:43.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.989+0000 7f60a5d9b700 1 --2- 172.21.15.144:0/323980745 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f609006e200 0x7f60900706b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f609800b5c0 tx=0x7f60980053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:43.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:43.992+0000 7f60977fe700 1 -- 172.21.15.144:0/323980745 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f609c0573a0 con 0x7f60a80ff390 2024-09-10T12:00:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:00:44.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.275+0000 7f60aca90700 1 -- 172.21.15.144:0/323980745 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f60a81970f0 con 0x7f60a80ff390 2024-09-10T12:00:44.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.276+0000 7f60977fe700 1 -- 172.21.15.144:0/323980745 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f609c05aa60 con 0x7f60a80ff390 2024-09-10T12:00:44.277 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:00:44.277 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:00:44.277 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:00:44.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:00:44.277 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:00:44.277 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:00:44.277 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:00:44.278 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:00:44.278 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:00:44.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:00:44.278 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:00:44.278 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:00:44.278 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:00:44.278 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:00:44.278 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:00:44.278 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:00:44.278 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:00:44.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.278+0000 7f60aca90700 1 -- 172.21.15.144:0/323980745 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f609006e200 msgr2=0x7f60900706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:44.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.278+0000 7f60aca90700 1 --2- 172.21.15.144:0/323980745 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f609006e200 0x7f60900706b0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f609800b5c0 tx=0x7f60980053a0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.278+0000 7f60aca90700 1 -- 172.21.15.144:0/323980745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a80ff390 msgr2=0x7f60a8195d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:44.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.278+0000 7f60aca90700 1 --2- 172.21.15.144:0/323980745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a80ff390 0x7f60a8195d90 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f609c00d900 tx=0x7f609c00dcc0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.278+0000 7f60aca90700 1 -- 172.21.15.144:0/323980745 shutdown_connections 2024-09-10T12:00:44.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.278+0000 7f60aca90700 1 --2- 172.21.15.144:0/323980745 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f609006e200 0x7f60900706b0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.278+0000 7f60aca90700 1 --2- 172.21.15.144:0/323980745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f60a80ff390 0x7f60a8195d90 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.278+0000 7f60aca90700 1 --2- 172.21.15.144:0/323980745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f60a80ffd30 0x7f60a81962d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.278+0000 7f60aca90700 1 -- 172.21.15.144:0/323980745 >> 172.21.15.144:0/323980745 conn(0x7f60a8074f50 msgr2=0x7f60a8103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:44.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.279+0000 7f60aca90700 1 -- 172.21.15.144:0/323980745 shutdown_connections 2024-09-10T12:00:44.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.279+0000 7f60aca90700 1 -- 172.21.15.144:0/323980745 wait complete. 2024-09-10T12:00:44.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.383+0000 7fd2906b6700 1 -- 172.21.15.144:0/3818129951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2881003d0 msgr2=0x7fd2881007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:44.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.383+0000 7fd2906b6700 1 --2- 172.21.15.144:0/3818129951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2881003d0 0x7fd2881007a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd284009910 tx=0x7fd284009c20 comp rx=0 tx=0).stop 2024-09-10T12:00:44.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.383+0000 7fd2906b6700 1 -- 172.21.15.144:0/3818129951 shutdown_connections 2024-09-10T12:00:44.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.383+0000 7fd2906b6700 1 --2- 172.21.15.144:0/3818129951 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd288100d70 0x7fd288104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.383+0000 7fd2906b6700 1 --2- 172.21.15.144:0/3818129951 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2881003d0 0x7fd2881007a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.383+0000 7fd2906b6700 1 -- 172.21.15.144:0/3818129951 >> 172.21.15.144:0/3818129951 conn(0x7fd2880fbc90 msgr2=0x7fd2880fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:44.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.383+0000 7fd2906b6700 1 -- 172.21.15.144:0/3818129951 shutdown_connections 2024-09-10T12:00:44.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.383+0000 7fd2906b6700 1 -- 172.21.15.144:0/3818129951 wait complete. 2024-09-10T12:00:44.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.383+0000 7fd2906b6700 1 Processor -- start 2024-09-10T12:00:44.385 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd2906b6700 1 -- start start 2024-09-10T12:00:44.386 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd2906b6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2881003d0 0x7fd288195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:44.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd2906b6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd288100d70 0x7fd2881962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:44.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd2906b6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd288196980 con 0x7fd2881003d0 2024-09-10T12:00:44.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd2906b6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28819a710 con 0x7fd288100d70 2024-09-10T12:00:44.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd28e452700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2881003d0 0x7fd288195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:44.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd28e452700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2881003d0 0x7fd288195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52504/0 (socket says 172.21.15.144:52504) 2024-09-10T12:00:44.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd28e452700 1 -- 172.21.15.144:0/3213071229 learned_addr learned my addr 172.21.15.144:0/3213071229 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:00:44.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd28e452700 1 -- 172.21.15.144:0/3213071229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd288100d70 msgr2=0x7fd2881962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:44.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd28e452700 1 --2- 172.21.15.144:0/3213071229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd288100d70 0x7fd2881962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd28e452700 1 -- 172.21.15.144:0/3213071229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2840095f0 con 0x7fd2881003d0 2024-09-10T12:00:44.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd28e452700 1 --2- 172.21.15.144:0/3213071229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2881003d0 0x7fd288195d60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd284005f50 tx=0x7fd28400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:44.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd27f7fe700 1 -- 172.21.15.144:0/3213071229 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd28401d070 con 0x7fd2881003d0 2024-09-10T12:00:44.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd2906b6700 1 -- 172.21.15.144:0/3213071229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd28819a990 con 0x7fd2881003d0 2024-09-10T12:00:44.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd27f7fe700 1 -- 172.21.15.144:0/3213071229 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd28400b610 con 0x7fd2881003d0 2024-09-10T12:00:44.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd27f7fe700 1 -- 172.21.15.144:0/3213071229 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd284022470 con 0x7fd2881003d0 2024-09-10T12:00:44.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.384+0000 7fd2906b6700 1 -- 172.21.15.144:0/3213071229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd28819ae80 con 0x7fd2881003d0 2024-09-10T12:00:44.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.385+0000 7fd2906b6700 1 -- 172.21.15.144:0/3213071229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2881086e0 con 0x7fd2881003d0 2024-09-10T12:00:44.388 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.386+0000 7fd27f7fe700 1 -- 172.21.15.144:0/3213071229 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd28400fca0 con 0x7fd2881003d0 2024-09-10T12:00:44.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.386+0000 7fd27f7fe700 1 --2- 172.21.15.144:0/3213071229 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd27406e200 0x7fd2740706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:44.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.386+0000 7fd27f7fe700 1 -- 172.21.15.144:0/3213071229 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd28408f540 con 0x7fd2881003d0 2024-09-10T12:00:44.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.387+0000 7fd28dc51700 1 --2- 172.21.15.144:0/3213071229 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd27406e200 0x7fd2740706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:44.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.387+0000 7fd28dc51700 1 --2- 172.21.15.144:0/3213071229 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd27406e200 0x7fd2740706b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd278005950 tx=0x7fd2780058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:44.392 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.391+0000 7fd27f7fe700 1 -- 172.21.15.144:0/3213071229 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd284058920 con 0x7fd2881003d0 2024-09-10T12:00:44.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.636+0000 7fd2906b6700 1 -- 172.21.15.144:0/3213071229 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd28802cc60 con 0x7fd2881003d0 2024-09-10T12:00:44.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.636+0000 7fd27f7fe700 1 -- 172.21.15.144:0/3213071229 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7fd28405bfe0 con 0x7fd2881003d0 2024-09-10T12:00:44.638 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:00:44.638 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:00:44.638 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:00:44.638 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:00:44.638 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:00:44.638 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:00:44.638 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:00:44.638 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:00:44.639 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:00:44.639 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:00:44.639 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:00:44.639 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:00:44.639 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:00:44.639 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:00:44.639 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:00:44.639 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:00:44.639 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:00:44.639 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:00:44.639 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:00:44.639 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:00:44.639 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:00:44.639 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:00:44.640 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:00:44.640 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:00:44.640 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:00:44.640 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:00:44.640 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:00:44.640 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:00:44.640 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:00:44.640 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:00:44.640 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:00:44.640 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:00:44.640 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:00:44.640 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:00:44.640 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:00:44.640 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:00:44.641 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:00:44.641 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:00:44.641 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:00:44.641 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:00:44.641 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:00:44.641 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:00:44.641 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:00:44.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.640+0000 7fd2906b6700 1 -- 172.21.15.144:0/3213071229 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd27406e200 msgr2=0x7fd2740706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:44.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.640+0000 7fd2906b6700 1 --2- 172.21.15.144:0/3213071229 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd27406e200 0x7fd2740706b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fd278005950 tx=0x7fd2780058e0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.640+0000 7fd2906b6700 1 -- 172.21.15.144:0/3213071229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2881003d0 msgr2=0x7fd288195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:44.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.640+0000 7fd2906b6700 1 --2- 172.21.15.144:0/3213071229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2881003d0 0x7fd288195d60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fd284005f50 tx=0x7fd28400f740 comp rx=0 tx=0).stop 2024-09-10T12:00:44.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.640+0000 7fd2906b6700 1 -- 172.21.15.144:0/3213071229 shutdown_connections 2024-09-10T12:00:44.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.640+0000 7fd2906b6700 1 --2- 172.21.15.144:0/3213071229 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd27406e200 0x7fd2740706b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.640+0000 7fd2906b6700 1 --2- 172.21.15.144:0/3213071229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd2881003d0 0x7fd288195d60 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.640+0000 7fd2906b6700 1 --2- 172.21.15.144:0/3213071229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd288100d70 0x7fd2881962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.640+0000 7fd2906b6700 1 -- 172.21.15.144:0/3213071229 >> 172.21.15.144:0/3213071229 conn(0x7fd2880fbc90 msgr2=0x7fd2880fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:44.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.640+0000 7fd2906b6700 1 -- 172.21.15.144:0/3213071229 shutdown_connections 2024-09-10T12:00:44.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.640+0000 7fd2906b6700 1 -- 172.21.15.144:0/3213071229 wait complete. 2024-09-10T12:00:44.643 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:00:44.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.746+0000 7f36a7b89700 1 -- 172.21.15.144:0/3628638391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0104790 msgr2=0x7f36a0104b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:44.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.746+0000 7f36a7b89700 1 --2- 172.21.15.144:0/3628638391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0104790 0x7f36a0104b60 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f369c009910 tx=0x7f369c009c20 comp rx=0 tx=0).stop 2024-09-10T12:00:44.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.746+0000 7f36a7b89700 1 -- 172.21.15.144:0/3628638391 shutdown_connections 2024-09-10T12:00:44.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.746+0000 7f36a7b89700 1 --2- 172.21.15.144:0/3628638391 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36a0103450 0x7f36a01038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.746+0000 7f36a7b89700 1 --2- 172.21.15.144:0/3628638391 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0104790 0x7f36a0104b60 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.746+0000 7f36a7b89700 1 -- 172.21.15.144:0/3628638391 >> 172.21.15.144:0/3628638391 conn(0x7f36a0074850 msgr2=0x7f36a0074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:44.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.746+0000 7f36a7b89700 1 -- 172.21.15.144:0/3628638391 shutdown_connections 2024-09-10T12:00:44.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.746+0000 7f36a7b89700 1 -- 172.21.15.144:0/3628638391 wait complete. 2024-09-10T12:00:44.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.746+0000 7f36a7b89700 1 Processor -- start 2024-09-10T12:00:44.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.746+0000 7f36a7b89700 1 -- start start 2024-09-10T12:00:44.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f36a7b89700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36a0103450 0x7f36a019bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:44.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f36a7b89700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0104790 0x7f36a019bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:44.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f36a7b89700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a019c680 con 0x7f36a0104790 2024-09-10T12:00:44.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f36a7b89700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36a0195b30 con 0x7f36a0103450 2024-09-10T12:00:44.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f36a5124700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0104790 0x7f36a019bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:44.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f36a5124700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0104790 0x7f36a019bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52520/0 (socket says 172.21.15.144:52520) 2024-09-10T12:00:44.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f36a5124700 1 -- 172.21.15.144:0/2468103308 learned_addr learned my addr 172.21.15.144:0/2468103308 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:00:44.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f36a5925700 1 --2- 172.21.15.144:0/2468103308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36a0103450 0x7f36a019bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:44.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f36a5124700 1 -- 172.21.15.144:0/2468103308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36a0103450 msgr2=0x7f36a019bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:44.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f36a5124700 1 --2- 172.21.15.144:0/2468103308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36a0103450 0x7f36a019bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f36a5124700 1 -- 172.21.15.144:0/2468103308 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c0095f0 con 0x7f36a0104790 2024-09-10T12:00:44.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f36a5925700 1 --2- 172.21.15.144:0/2468103308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36a0103450 0x7f36a019bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:00:44.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f36a5124700 1 --2- 172.21.15.144:0/2468103308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0104790 0x7f36a019bff0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f369000db60 tx=0x7f369000df20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:44.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f3696ffd700 1 -- 172.21.15.144:0/2468103308 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3690007ce0 con 0x7f36a0104790 2024-09-10T12:00:44.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f3696ffd700 1 -- 172.21.15.144:0/2468103308 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3690007e40 con 0x7f36a0104790 2024-09-10T12:00:44.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f3696ffd700 1 -- 172.21.15.144:0/2468103308 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3690005320 con 0x7f36a0104790 2024-09-10T12:00:44.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.747+0000 7f36a7b89700 1 -- 172.21.15.144:0/2468103308 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36a0195e10 con 0x7f36a0104790 2024-09-10T12:00:44.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.748+0000 7f36a7b89700 1 -- 172.21.15.144:0/2468103308 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36a0196360 con 0x7f36a0104790 2024-09-10T12:00:44.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.748+0000 7f36a7b89700 1 -- 172.21.15.144:0/2468103308 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36a004ea50 con 0x7f36a0104790 2024-09-10T12:00:44.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.749+0000 7f3696ffd700 1 -- 172.21.15.144:0/2468103308 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3690004750 con 0x7f36a0104790 2024-09-10T12:00:44.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.749+0000 7f3696ffd700 1 --2- 172.21.15.144:0/2468103308 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f368c06e200 0x7f368c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:44.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.750+0000 7f3696ffd700 1 -- 172.21.15.144:0/2468103308 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f369008de70 con 0x7f36a0104790 2024-09-10T12:00:44.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.750+0000 7f36a5925700 1 --2- 172.21.15.144:0/2468103308 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f368c06e200 0x7f368c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:44.752 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.750+0000 7f36a5925700 1 --2- 172.21.15.144:0/2468103308 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f368c06e200 0x7f368c0706b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f369c006010 tx=0x7f369c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:44.755 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.754+0000 7f3696ffd700 1 -- 172.21.15.144:0/2468103308 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f369005ae00 con 0x7f36a0104790 2024-09-10T12:00:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:44 smithi144 ceph-mon[87436]: pgmap v863: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:44 smithi144 ceph-mon[87436]: from='client.35576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:44 smithi144 ceph-mon[87436]: from='client.35580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:44 smithi144 ceph-mon[87436]: from='client.35584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:44 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/323980745' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:00:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:44 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3213071229' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:00:44.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.975+0000 7f36a7b89700 1 -- 172.21.15.144:0/2468103308 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36a01967e0 con 0x7f368c06e200 2024-09-10T12:00:44.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.977+0000 7f3696ffd700 1 -- 172.21.15.144:0/2468103308 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f36a01967e0 con 0x7f368c06e200 2024-09-10T12:00:44.978 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:00:44.978 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:00:44.978 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:00:44.978 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:00:44.978 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:00:44.978 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:00:44.978 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:00:44.978 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:00:44.978 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:00:44.979 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:00:44.979 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:00:44.979 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:00:44.979 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:00:44.979 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:00:44.979 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:00:44.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.979+0000 7f36a7b89700 1 -- 172.21.15.144:0/2468103308 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f368c06e200 msgr2=0x7f368c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:44.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.979+0000 7f36a7b89700 1 --2- 172.21.15.144:0/2468103308 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f368c06e200 0x7f368c0706b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f369c006010 tx=0x7f369c00b540 comp rx=0 tx=0).stop 2024-09-10T12:00:44.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.979+0000 7f36a7b89700 1 -- 172.21.15.144:0/2468103308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0104790 msgr2=0x7f36a019bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:44.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.979+0000 7f36a7b89700 1 --2- 172.21.15.144:0/2468103308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0104790 0x7f36a019bff0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f369000db60 tx=0x7f369000df20 comp rx=0 tx=0).stop 2024-09-10T12:00:44.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.979+0000 7f36a7b89700 1 -- 172.21.15.144:0/2468103308 shutdown_connections 2024-09-10T12:00:44.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.979+0000 7f36a7b89700 1 --2- 172.21.15.144:0/2468103308 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f368c06e200 0x7f368c0706b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.979+0000 7f36a7b89700 1 --2- 172.21.15.144:0/2468103308 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36a0103450 0x7f36a019bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.979+0000 7f36a7b89700 1 --2- 172.21.15.144:0/2468103308 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36a0104790 0x7f36a019bff0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:44.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.979+0000 7f36a7b89700 1 -- 172.21.15.144:0/2468103308 >> 172.21.15.144:0/2468103308 conn(0x7f36a0074850 msgr2=0x7f36a01021f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:44.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.979+0000 7f36a7b89700 1 -- 172.21.15.144:0/2468103308 shutdown_connections 2024-09-10T12:00:44.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:44.979+0000 7f36a7b89700 1 -- 172.21.15.144:0/2468103308 wait complete. 2024-09-10T12:00:45.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.091+0000 7f6466db8700 1 -- 172.21.15.144:0/2573202150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64600ff310 msgr2=0x7f64600ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:45.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.091+0000 7f6466db8700 1 --2- 172.21.15.144:0/2573202150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64600ff310 0x7f64600ff6e0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f6454009910 tx=0x7f6454009c20 comp rx=0 tx=0).stop 2024-09-10T12:00:45.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.091+0000 7f6466db8700 1 -- 172.21.15.144:0/2573202150 shutdown_connections 2024-09-10T12:00:45.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.091+0000 7f6466db8700 1 --2- 172.21.15.144:0/2573202150 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64600ffcb0 0x7f6460103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:45.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.091+0000 7f6466db8700 1 --2- 172.21.15.144:0/2573202150 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64600ff310 0x7f64600ff6e0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:45.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.091+0000 7f6466db8700 1 -- 172.21.15.144:0/2573202150 >> 172.21.15.144:0/2573202150 conn(0x7f64600fac10 msgr2=0x7f64600fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:45.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.091+0000 7f6466db8700 1 -- 172.21.15.144:0/2573202150 shutdown_connections 2024-09-10T12:00:45.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.091+0000 7f6466db8700 1 -- 172.21.15.144:0/2573202150 wait complete. 2024-09-10T12:00:45.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.091+0000 7f6466db8700 1 Processor -- start 2024-09-10T12:00:45.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.091+0000 7f6466db8700 1 -- start start 2024-09-10T12:00:45.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.092+0000 7f6466db8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64600ff310 0x7f6460196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:45.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.092+0000 7f6466db8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64600ffcb0 0x7f64601970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:45.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.092+0000 7f6466db8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6460197780 con 0x7f64600ffcb0 2024-09-10T12:00:45.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.092+0000 7f6466db8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64601978c0 con 0x7f64600ff310 2024-09-10T12:00:45.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.092+0000 7f645ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64600ffcb0 0x7f64601970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:45.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.092+0000 7f645ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64600ffcb0 0x7f64601970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52546/0 (socket says 172.21.15.144:52546) 2024-09-10T12:00:45.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.092+0000 7f645ffff700 1 -- 172.21.15.144:0/2865741704 learned_addr learned my addr 172.21.15.144:0/2865741704 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:00:45.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.092+0000 7f6464b54700 1 --2- 172.21.15.144:0/2865741704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64600ff310 0x7f6460196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:45.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.092+0000 7f645ffff700 1 -- 172.21.15.144:0/2865741704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64600ff310 msgr2=0x7f6460196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:45.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.092+0000 7f645ffff700 1 --2- 172.21.15.144:0/2865741704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64600ff310 0x7f6460196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:45.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.092+0000 7f645ffff700 1 -- 172.21.15.144:0/2865741704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64540095f0 con 0x7f64600ffcb0 2024-09-10T12:00:45.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.092+0000 7f645ffff700 1 --2- 172.21.15.144:0/2865741704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64600ffcb0 0x7f64601970a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f644c00c8f0 tx=0x7f644c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:45.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.092+0000 7f645dffb700 1 -- 172.21.15.144:0/2865741704 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f644c0078a0 con 0x7f64600ffcb0 2024-09-10T12:00:45.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.093+0000 7f6466db8700 1 -- 172.21.15.144:0/2865741704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f646019fbd0 con 0x7f64600ffcb0 2024-09-10T12:00:45.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.093+0000 7f645dffb700 1 -- 172.21.15.144:0/2865741704 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f644c00f450 con 0x7f64600ffcb0 2024-09-10T12:00:45.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.093+0000 7f645dffb700 1 -- 172.21.15.144:0/2865741704 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f644c018610 con 0x7f64600ffcb0 2024-09-10T12:00:45.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.093+0000 7f6466db8700 1 -- 172.21.15.144:0/2865741704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64601a00f0 con 0x7f64600ffcb0 2024-09-10T12:00:45.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.094+0000 7f6466db8700 1 -- 172.21.15.144:0/2865741704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6460107620 con 0x7f64600ffcb0 2024-09-10T12:00:45.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.094+0000 7f645dffb700 1 -- 172.21.15.144:0/2865741704 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f644c00f5c0 con 0x7f64600ffcb0 2024-09-10T12:00:45.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.095+0000 7f645dffb700 1 --2- 172.21.15.144:0/2865741704 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f645006e200 0x7f64500706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:00:45.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.095+0000 7f6464b54700 1 --2- 172.21.15.144:0/2865741704 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f645006e200 0x7f64500706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:00:45.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.095+0000 7f645dffb700 1 -- 172.21.15.144:0/2865741704 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f644c08dfe0 con 0x7f64600ffcb0 2024-09-10T12:00:45.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.095+0000 7f6464b54700 1 --2- 172.21.15.144:0/2865741704 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f645006e200 0x7f64500706b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6454005f50 tx=0x7f645401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:00:45.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.099+0000 7f645dffb700 1 -- 172.21.15.144:0/2865741704 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f644c0573c0 con 0x7f64600ffcb0 2024-09-10T12:00:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:44 smithi178 ceph-mon[65940]: pgmap v863: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:44 smithi178 ceph-mon[65940]: from='client.35576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:44 smithi178 ceph-mon[65940]: from='client.35580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:44 smithi178 ceph-mon[65940]: from='client.35584 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:44 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/323980745' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:00:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:44 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3213071229' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:00:45.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.385+0000 7f6466db8700 1 -- 172.21.15.144:0/2865741704 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f646004ea50 con 0x7f64600ffcb0 2024-09-10T12:00:45.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.386+0000 7f645dffb700 1 -- 172.21.15.144:0/2865741704 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f644c05aa80 con 0x7f64600ffcb0 2024-09-10T12:00:45.387 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:00:45.387 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:00:45.387 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:00:45.388 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:00:45.388 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:00:45.388 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:00:45.388 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:00:45.388 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:00:45.388 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:00:45.388 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:00:45.388 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:00:45.388 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:00:45.388 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:00:45.388 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:00:45.389 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:00:45.389 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:00:45.389 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:00:45.389 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:00:45.389 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:00:45.389 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:00:45.389 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:00:45.389 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:00:45.389 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:00:45.389 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:00:45.389 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:00:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.388+0000 7f6466db8700 1 -- 172.21.15.144:0/2865741704 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f645006e200 msgr2=0x7f64500706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.388+0000 7f6466db8700 1 --2- 172.21.15.144:0/2865741704 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f645006e200 0x7f64500706b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6454005f50 tx=0x7f645401a040 comp rx=0 tx=0).stop 2024-09-10T12:00:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.388+0000 7f6466db8700 1 -- 172.21.15.144:0/2865741704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64600ffcb0 msgr2=0x7f64601970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:00:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.388+0000 7f6466db8700 1 --2- 172.21.15.144:0/2865741704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64600ffcb0 0x7f64601970a0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f644c00c8f0 tx=0x7f644c00cc00 comp rx=0 tx=0).stop 2024-09-10T12:00:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.388+0000 7f6466db8700 1 -- 172.21.15.144:0/2865741704 shutdown_connections 2024-09-10T12:00:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.388+0000 7f6466db8700 1 --2- 172.21.15.144:0/2865741704 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f645006e200 0x7f64500706b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.388+0000 7f6466db8700 1 --2- 172.21.15.144:0/2865741704 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64600ff310 0x7f6460196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:45.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.388+0000 7f6466db8700 1 --2- 172.21.15.144:0/2865741704 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64600ffcb0 0x7f64601970a0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:00:45.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.388+0000 7f6466db8700 1 -- 172.21.15.144:0/2865741704 >> 172.21.15.144:0/2865741704 conn(0x7f64600fac10 msgr2=0x7f64600fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:00:45.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.388+0000 7f6466db8700 1 -- 172.21.15.144:0/2865741704 shutdown_connections 2024-09-10T12:00:45.391 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:00:45.388+0000 7f6466db8700 1 -- 172.21.15.144:0/2865741704 wait complete. 2024-09-10T12:00:45.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:45 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2865741704' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:00:46.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:45 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2865741704' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:00:46.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:46 smithi144 ceph-mon[87436]: pgmap v864: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:46.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:46 smithi144 ceph-mon[87436]: from='client.35596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:46 smithi178 ceph-mon[65940]: pgmap v864: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:46 smithi178 ceph-mon[65940]: from='client.35596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:00:48.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:48 smithi144 ceph-mon[87436]: pgmap v865: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:48 smithi178 ceph-mon[65940]: pgmap v865: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:50.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:50 smithi144 ceph-mon[87436]: pgmap v866: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:50 smithi178 ceph-mon[65940]: pgmap v866: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:52.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:52 smithi144 ceph-mon[87436]: pgmap v867: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:52 smithi178 ceph-mon[65940]: pgmap v867: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:54 smithi144 ceph-mon[87436]: pgmap v868: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:54 smithi178 ceph-mon[65940]: pgmap v868: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:56.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:56 smithi178 ceph-mon[65940]: pgmap v869: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:56.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:56 smithi144 ceph-mon[87436]: pgmap v869: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:58 smithi144 ceph-mon[87436]: pgmap v870: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:00:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:00:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:58 smithi178 ceph-mon[65940]: pgmap v870: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:00:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:00:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:01:00.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:00 smithi144 ceph-mon[87436]: pgmap v871: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:00 smithi178 ceph-mon[65940]: pgmap v871: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:02.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:02 smithi144 ceph-mon[87436]: pgmap v872: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:02 smithi178 ceph-mon[65940]: pgmap v872: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:04.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:04 smithi144 ceph-mon[87436]: pgmap v873: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:04 smithi178 ceph-mon[65940]: pgmap v873: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:06.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:06 smithi178 ceph-mon[65940]: pgmap v874: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:06 smithi144 ceph-mon[87436]: pgmap v874: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:08 smithi144 ceph-mon[87436]: pgmap v875: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:08 smithi178 ceph-mon[65940]: pgmap v875: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:10 smithi178 ceph-mon[65940]: pgmap v876: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:11.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:10 smithi144 ceph-mon[87436]: pgmap v876: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:13.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:12 smithi144 ceph-mon[87436]: pgmap v877: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:12 smithi178 ceph-mon[65940]: pgmap v877: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:01:14.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:01:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:14 smithi178 ceph-mon[65940]: pgmap v878: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:14 smithi144 ceph-mon[87436]: pgmap v878: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:15.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.507+0000 7f9e7ee70700 1 -- 172.21.15.144:0/939772190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e780ffd30 msgr2=0x7f9e78103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:15.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.507+0000 7f9e7ee70700 1 --2- 172.21.15.144:0/939772190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e780ffd30 0x7f9e78103ce0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f9e6800b1a0 tx=0x7f9e6800b4b0 comp rx=0 tx=0).stop 2024-09-10T12:01:15.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.507+0000 7f9e7ee70700 1 -- 172.21.15.144:0/939772190 shutdown_connections 2024-09-10T12:01:15.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.507+0000 7f9e7ee70700 1 --2- 172.21.15.144:0/939772190 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e780ffd30 0x7f9e78103ce0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:15.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.507+0000 7f9e7ee70700 1 --2- 172.21.15.144:0/939772190 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e780ff390 0x7f9e780ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:15.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.507+0000 7f9e7ee70700 1 -- 172.21.15.144:0/939772190 >> 172.21.15.144:0/939772190 conn(0x7f9e78074f50 msgr2=0x7f9e78075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:15.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.507+0000 7f9e7ee70700 1 -- 172.21.15.144:0/939772190 shutdown_connections 2024-09-10T12:01:15.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.507+0000 7f9e7ee70700 1 -- 172.21.15.144:0/939772190 wait complete. 2024-09-10T12:01:15.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.508+0000 7f9e7ee70700 1 Processor -- start 2024-09-10T12:01:15.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.508+0000 7f9e7ee70700 1 -- start start 2024-09-10T12:01:15.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.508+0000 7f9e7ee70700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e780ff390 0x7f9e78199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:15.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.508+0000 7f9e7ee70700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e780ffd30 0x7f9e78199e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:15.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.508+0000 7f9e7ee70700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7819a4d0 con 0x7f9e780ff390 2024-09-10T12:01:15.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.508+0000 7f9e7ee70700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e78193980 con 0x7f9e780ffd30 2024-09-10T12:01:15.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.508+0000 7f9e7cc0c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e780ff390 0x7f9e78199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:15.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.508+0000 7f9e7cc0c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e780ff390 0x7f9e78199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56612/0 (socket says 172.21.15.144:56612) 2024-09-10T12:01:15.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.508+0000 7f9e7cc0c700 1 -- 172.21.15.144:0/4088103072 learned_addr learned my addr 172.21.15.144:0/4088103072 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:01:15.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.509+0000 7f9e77fff700 1 --2- 172.21.15.144:0/4088103072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e780ffd30 0x7f9e78199e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:15.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.509+0000 7f9e7cc0c700 1 -- 172.21.15.144:0/4088103072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e780ffd30 msgr2=0x7f9e78199e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:15.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.509+0000 7f9e7cc0c700 1 --2- 172.21.15.144:0/4088103072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e780ffd30 0x7f9e78199e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:15.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.509+0000 7f9e7cc0c700 1 -- 172.21.15.144:0/4088103072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e68009d00 con 0x7f9e780ff390 2024-09-10T12:01:15.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.509+0000 7f9e77fff700 1 --2- 172.21.15.144:0/4088103072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e780ffd30 0x7f9e78199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:01:15.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.509+0000 7f9e7cc0c700 1 --2- 172.21.15.144:0/4088103072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e780ff390 0x7f9e78199900 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000d8d0 tx=0x7f9e6000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:15.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.509+0000 7f9e75ffb700 1 -- 172.21.15.144:0/4088103072 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e60009940 con 0x7f9e780ff390 2024-09-10T12:01:15.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.509+0000 7f9e75ffb700 1 -- 172.21.15.144:0/4088103072 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e60010460 con 0x7f9e780ff390 2024-09-10T12:01:15.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.509+0000 7f9e75ffb700 1 -- 172.21.15.144:0/4088103072 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e6000f5f0 con 0x7f9e780ff390 2024-09-10T12:01:15.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.509+0000 7f9e7ee70700 1 -- 172.21.15.144:0/4088103072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e78193c60 con 0x7f9e780ff390 2024-09-10T12:01:15.514 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.509+0000 7f9e7ee70700 1 -- 172.21.15.144:0/4088103072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e781941b0 con 0x7f9e780ff390 2024-09-10T12:01:15.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.510+0000 7f9e7ee70700 1 -- 172.21.15.144:0/4088103072 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e781075c0 con 0x7f9e780ff390 2024-09-10T12:01:15.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.511+0000 7f9e75ffb700 1 -- 172.21.15.144:0/4088103072 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9e600105d0 con 0x7f9e780ff390 2024-09-10T12:01:15.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.511+0000 7f9e75ffb700 1 --2- 172.21.15.144:0/4088103072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e6406e200 0x7f9e640706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:15.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.511+0000 7f9e75ffb700 1 -- 172.21.15.144:0/4088103072 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9e6008df90 con 0x7f9e780ff390 2024-09-10T12:01:15.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.511+0000 7f9e77fff700 1 --2- 172.21.15.144:0/4088103072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e6406e200 0x7f9e640706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:15.515 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.512+0000 7f9e77fff700 1 --2- 172.21.15.144:0/4088103072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e6406e200 0x7f9e640706b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9e680114a0 tx=0x7f9e6801c040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:15.517 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.516+0000 7f9e75ffb700 1 -- 172.21.15.144:0/4088103072 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9e6005af20 con 0x7f9e780ff390 2024-09-10T12:01:15.738 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.737+0000 7f9e7ee70700 1 -- 172.21.15.144:0/4088103072 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e781944e0 con 0x7f9e6406e200 2024-09-10T12:01:15.739 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.738+0000 7f9e75ffb700 1 -- 172.21.15.144:0/4088103072 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9e781944e0 con 0x7f9e6406e200 2024-09-10T12:01:15.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.741+0000 7f9e7ee70700 1 -- 172.21.15.144:0/4088103072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e6406e200 msgr2=0x7f9e640706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:15.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.741+0000 7f9e7ee70700 1 --2- 172.21.15.144:0/4088103072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e6406e200 0x7f9e640706b0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9e680114a0 tx=0x7f9e6801c040 comp rx=0 tx=0).stop 2024-09-10T12:01:15.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.741+0000 7f9e7ee70700 1 -- 172.21.15.144:0/4088103072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e780ff390 msgr2=0x7f9e78199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:15.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.741+0000 7f9e7ee70700 1 --2- 172.21.15.144:0/4088103072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e780ff390 0x7f9e78199900 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f9e6000d8d0 tx=0x7f9e6000dc90 comp rx=0 tx=0).stop 2024-09-10T12:01:15.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.741+0000 7f9e7ee70700 1 -- 172.21.15.144:0/4088103072 shutdown_connections 2024-09-10T12:01:15.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.741+0000 7f9e7ee70700 1 --2- 172.21.15.144:0/4088103072 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e6406e200 0x7f9e640706b0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:15.743 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.741+0000 7f9e7ee70700 1 --2- 172.21.15.144:0/4088103072 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e780ff390 0x7f9e78199900 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:15.743 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.741+0000 7f9e7ee70700 1 --2- 172.21.15.144:0/4088103072 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e780ffd30 0x7f9e78199e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:15.743 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.741+0000 7f9e7ee70700 1 -- 172.21.15.144:0/4088103072 >> 172.21.15.144:0/4088103072 conn(0x7f9e78074f50 msgr2=0x7f9e780fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:15.743 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.741+0000 7f9e7ee70700 1 -- 172.21.15.144:0/4088103072 shutdown_connections 2024-09-10T12:01:15.743 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.741+0000 7f9e7ee70700 1 -- 172.21.15.144:0/4088103072 wait complete. 2024-09-10T12:01:15.753 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:01:15.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.846+0000 7f69a17b2700 1 -- 172.21.15.144:0/4005885251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699c068fd0 msgr2=0x7f699c104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:15.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.846+0000 7f69a17b2700 1 --2- 172.21.15.144:0/4005885251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699c068fd0 0x7f699c104df0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f698c009910 tx=0x7f698c009c20 comp rx=0 tx=0).stop 2024-09-10T12:01:15.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.847+0000 7f69a17b2700 1 -- 172.21.15.144:0/4005885251 shutdown_connections 2024-09-10T12:01:15.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.847+0000 7f69a17b2700 1 --2- 172.21.15.144:0/4005885251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699c068fd0 0x7f699c104df0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:15.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.847+0000 7f69a17b2700 1 --2- 172.21.15.144:0/4005885251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f699c068630 0x7f699c068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:15.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.847+0000 7f69a17b2700 1 -- 172.21.15.144:0/4005885251 >> 172.21.15.144:0/4005885251 conn(0x7f699c074f50 msgr2=0x7f699c075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:15.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.847+0000 7f69a17b2700 1 -- 172.21.15.144:0/4005885251 shutdown_connections 2024-09-10T12:01:15.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.847+0000 7f69a17b2700 1 -- 172.21.15.144:0/4005885251 wait complete. 2024-09-10T12:01:15.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.848+0000 7f69a17b2700 1 Processor -- start 2024-09-10T12:01:15.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.848+0000 7f69a17b2700 1 -- start start 2024-09-10T12:01:15.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.848+0000 7f69a17b2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f699c068630 0x7f699c195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:15.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.848+0000 7f69a17b2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699c068fd0 0x7f699c196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:15.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.848+0000 7f69a17b2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c196a60 con 0x7f699c068fd0 2024-09-10T12:01:15.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.848+0000 7f69a17b2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699c19a7f0 con 0x7f699c068630 2024-09-10T12:01:15.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.848+0000 7f699a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699c068fd0 0x7f699c196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:15.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.849+0000 7f699a7fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699c068fd0 0x7f699c196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56620/0 (socket says 172.21.15.144:56620) 2024-09-10T12:01:15.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.849+0000 7f699a7fc700 1 -- 172.21.15.144:0/530376728 learned_addr learned my addr 172.21.15.144:0/530376728 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:01:15.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.849+0000 7f699affd700 1 --2- 172.21.15.144:0/530376728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f699c068630 0x7f699c195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:15.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.849+0000 7f699a7fc700 1 -- 172.21.15.144:0/530376728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f699c068630 msgr2=0x7f699c195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:15.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.849+0000 7f699a7fc700 1 --2- 172.21.15.144:0/530376728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f699c068630 0x7f699c195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:15.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.849+0000 7f699a7fc700 1 -- 172.21.15.144:0/530376728 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f698c0095f0 con 0x7f699c068fd0 2024-09-10T12:01:15.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.849+0000 7f699a7fc700 1 --2- 172.21.15.144:0/530376728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699c068fd0 0x7f699c196380 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f698c0092e0 tx=0x7f698c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:15.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.849+0000 7f6983fff700 1 -- 172.21.15.144:0/530376728 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f698c01d070 con 0x7f699c068fd0 2024-09-10T12:01:15.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.849+0000 7f69a17b2700 1 -- 172.21.15.144:0/530376728 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699c19aa70 con 0x7f699c068fd0 2024-09-10T12:01:15.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.849+0000 7f6983fff700 1 -- 172.21.15.144:0/530376728 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f698c00f460 con 0x7f699c068fd0 2024-09-10T12:01:15.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.849+0000 7f6983fff700 1 -- 172.21.15.144:0/530376728 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f698c0175d0 con 0x7f699c068fd0 2024-09-10T12:01:15.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.849+0000 7f69a17b2700 1 -- 172.21.15.144:0/530376728 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699c19af60 con 0x7f699c068fd0 2024-09-10T12:01:15.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.850+0000 7f69a17b2700 1 -- 172.21.15.144:0/530376728 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f699c1086d0 con 0x7f699c068fd0 2024-09-10T12:01:15.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.851+0000 7f6983fff700 1 -- 172.21.15.144:0/530376728 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f698c00f5d0 con 0x7f699c068fd0 2024-09-10T12:01:15.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.851+0000 7f6983fff700 1 --2- 172.21.15.144:0/530376728 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f698406e200 0x7f69840706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:15.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.851+0000 7f6983fff700 1 -- 172.21.15.144:0/530376728 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f698c08f590 con 0x7f699c068fd0 2024-09-10T12:01:15.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.851+0000 7f699affd700 1 --2- 172.21.15.144:0/530376728 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f698406e200 0x7f69840706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:15.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.852+0000 7f699affd700 1 --2- 172.21.15.144:0/530376728 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f698406e200 0x7f69840706b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6990009a10 tx=0x7f6990009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:15.857 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:15.856+0000 7f6983fff700 1 -- 172.21.15.144:0/530376728 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f698c058970 con 0x7f699c068fd0 2024-09-10T12:01:16.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.077+0000 7f69a17b2700 1 -- 172.21.15.144:0/530376728 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f699c02ccf0 con 0x7f698406e200 2024-09-10T12:01:16.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.078+0000 7f6983fff700 1 -- 172.21.15.144:0/530376728 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f699c02ccf0 con 0x7f698406e200 2024-09-10T12:01:16.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.081+0000 7f69a17b2700 1 -- 172.21.15.144:0/530376728 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f698406e200 msgr2=0x7f69840706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:16.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.081+0000 7f69a17b2700 1 --2- 172.21.15.144:0/530376728 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f698406e200 0x7f69840706b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f6990009a10 tx=0x7f6990009190 comp rx=0 tx=0).stop 2024-09-10T12:01:16.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.081+0000 7f69a17b2700 1 -- 172.21.15.144:0/530376728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699c068fd0 msgr2=0x7f699c196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:16.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.081+0000 7f69a17b2700 1 --2- 172.21.15.144:0/530376728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699c068fd0 0x7f699c196380 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f698c0092e0 tx=0x7f698c00bd70 comp rx=0 tx=0).stop 2024-09-10T12:01:16.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.081+0000 7f69a17b2700 1 -- 172.21.15.144:0/530376728 shutdown_connections 2024-09-10T12:01:16.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.081+0000 7f69a17b2700 1 --2- 172.21.15.144:0/530376728 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f698406e200 0x7f69840706b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.081+0000 7f69a17b2700 1 --2- 172.21.15.144:0/530376728 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f699c068630 0x7f699c195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.081+0000 7f69a17b2700 1 --2- 172.21.15.144:0/530376728 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f699c068fd0 0x7f699c196380 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.081+0000 7f69a17b2700 1 -- 172.21.15.144:0/530376728 >> 172.21.15.144:0/530376728 conn(0x7f699c074f50 msgr2=0x7f699c0fd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:16.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.081+0000 7f69a17b2700 1 -- 172.21.15.144:0/530376728 shutdown_connections 2024-09-10T12:01:16.083 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.081+0000 7f69a17b2700 1 -- 172.21.15.144:0/530376728 wait complete. 2024-09-10T12:01:16.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.185+0000 7f51a9148700 1 -- 172.21.15.144:0/314727491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51a40ff390 msgr2=0x7f51a40ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:16.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.185+0000 7f51a9148700 1 --2- 172.21.15.144:0/314727491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51a40ff390 0x7f51a40ff760 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5194009910 tx=0x7f5194009c20 comp rx=0 tx=0).stop 2024-09-10T12:01:16.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.185+0000 7f51a9148700 1 -- 172.21.15.144:0/314727491 shutdown_connections 2024-09-10T12:01:16.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.185+0000 7f51a9148700 1 --2- 172.21.15.144:0/314727491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51a40ffd30 0x7f51a4103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.185+0000 7f51a9148700 1 --2- 172.21.15.144:0/314727491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51a40ff390 0x7f51a40ff760 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.185+0000 7f51a9148700 1 -- 172.21.15.144:0/314727491 >> 172.21.15.144:0/314727491 conn(0x7f51a4074f50 msgr2=0x7f51a4075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:16.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.185+0000 7f51a9148700 1 -- 172.21.15.144:0/314727491 shutdown_connections 2024-09-10T12:01:16.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.186+0000 7f51a9148700 1 -- 172.21.15.144:0/314727491 wait complete. 2024-09-10T12:01:16.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.186+0000 7f51a9148700 1 Processor -- start 2024-09-10T12:01:16.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.186+0000 7f51a9148700 1 -- start start 2024-09-10T12:01:16.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f51a9148700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51a40ff390 0x7f51a4196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:16.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f51a9148700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51a40ffd30 0x7f51a4197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:16.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f51a9148700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a4197840 con 0x7f51a40ffd30 2024-09-10T12:01:16.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f51a9148700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a41979b0 con 0x7f51a40ff390 2024-09-10T12:01:16.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f51a259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51a40ffd30 0x7f51a4197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:16.190 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f51a259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51a40ffd30 0x7f51a4197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56638/0 (socket says 172.21.15.144:56638) 2024-09-10T12:01:16.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f51a259c700 1 -- 172.21.15.144:0/3189193830 learned_addr learned my addr 172.21.15.144:0/3189193830 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:01:16.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f51a2d9d700 1 --2- 172.21.15.144:0/3189193830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51a40ff390 0x7f51a4196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:16.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f51a259c700 1 -- 172.21.15.144:0/3189193830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51a40ff390 msgr2=0x7f51a4196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:16.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f51a259c700 1 --2- 172.21.15.144:0/3189193830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51a40ff390 0x7f51a4196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f51a259c700 1 -- 172.21.15.144:0/3189193830 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51940095f0 con 0x7f51a40ffd30 2024-09-10T12:01:16.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f51a259c700 1 --2- 172.21.15.144:0/3189193830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51a40ffd30 0x7f51a4197160 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f519800d900 tx=0x7f519800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:16.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f518bfff700 1 -- 172.21.15.144:0/3189193830 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5198009900 con 0x7f51a40ffd30 2024-09-10T12:01:16.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f51a9148700 1 -- 172.21.15.144:0/3189193830 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a419fc90 con 0x7f51a40ffd30 2024-09-10T12:01:16.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f518bfff700 1 -- 172.21.15.144:0/3189193830 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5198010460 con 0x7f51a40ffd30 2024-09-10T12:01:16.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.187+0000 7f518bfff700 1 -- 172.21.15.144:0/3189193830 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f519800f5f0 con 0x7f51a40ffd30 2024-09-10T12:01:16.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.188+0000 7f51a9148700 1 -- 172.21.15.144:0/3189193830 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a41a01e0 con 0x7f51a40ffd30 2024-09-10T12:01:16.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.189+0000 7f51a9148700 1 -- 172.21.15.144:0/3189193830 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a4107650 con 0x7f51a40ffd30 2024-09-10T12:01:16.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.189+0000 7f518bfff700 1 -- 172.21.15.144:0/3189193830 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f51980105d0 con 0x7f51a40ffd30 2024-09-10T12:01:16.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.190+0000 7f518bfff700 1 --2- 172.21.15.144:0/3189193830 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f518c06e200 0x7f518c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:16.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.190+0000 7f518bfff700 1 -- 172.21.15.144:0/3189193830 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f519808df10 con 0x7f51a40ffd30 2024-09-10T12:01:16.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.190+0000 7f51a2d9d700 1 --2- 172.21.15.144:0/3189193830 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f518c06e200 0x7f518c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:16.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.190+0000 7f51a2d9d700 1 --2- 172.21.15.144:0/3189193830 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f518c06e200 0x7f518c0706b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5194006010 tx=0x7f519401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:16.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.194+0000 7f518bfff700 1 -- 172.21.15.144:0/3189193830 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f51980572f0 con 0x7f51a40ffd30 2024-09-10T12:01:16.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.401+0000 7f51a9148700 1 -- 172.21.15.144:0/3189193830 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f51a4061190 con 0x7f518c06e200 2024-09-10T12:01:16.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.411+0000 7f518bfff700 1 -- 172.21.15.144:0/3189193830 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f51a4061190 con 0x7f518c06e200 2024-09-10T12:01:16.412 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:01:16.412 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (29m) 7m ago 39m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:01:16.412 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (40m) 7m ago 40m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:01:16.412 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (39m) 7m ago 39m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:01:16.412 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (27m) 7m ago 40m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:01:16.413 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (27m) 7m ago 39m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:01:16.413 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (29m) 7m ago 39m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:01:16.413 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (23m) 7m ago 34m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:01:16.413 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (23m) 7m ago 34m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:01:16.413 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (7m) 7m ago 34m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:01:16.413 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (23m) 7m ago 34m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:01:16.413 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (31m) 7m ago 41m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:01:16.413 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (30m) 7m ago 39m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:01:16.413 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (28m) 7m ago 41m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:01:16.413 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (27m) 7m ago 39m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:01:16.413 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (30m) 7m ago 40m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:01:16.413 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (30m) 7m ago 39m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:01:16.413 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (26m) 7m ago 38m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:01:16.414 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (26m) 7m ago 37m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:01:16.414 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (25m) 7m ago 37m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:01:16.414 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (24m) 7m ago 36m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:01:16.414 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (24m) 7m ago 36m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:01:16.414 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (23m) 7m ago 36m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:01:16.414 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (29m) 7m ago 39m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:01:16.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.413+0000 7f51a9148700 1 -- 172.21.15.144:0/3189193830 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f518c06e200 msgr2=0x7f518c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:16.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.413+0000 7f51a9148700 1 --2- 172.21.15.144:0/3189193830 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f518c06e200 0x7f518c0706b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f5194006010 tx=0x7f519401a040 comp rx=0 tx=0).stop 2024-09-10T12:01:16.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.413+0000 7f51a9148700 1 -- 172.21.15.144:0/3189193830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51a40ffd30 msgr2=0x7f51a4197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:16.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.413+0000 7f51a9148700 1 --2- 172.21.15.144:0/3189193830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51a40ffd30 0x7f51a4197160 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f519800d900 tx=0x7f519800dcc0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.413+0000 7f51a9148700 1 -- 172.21.15.144:0/3189193830 shutdown_connections 2024-09-10T12:01:16.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.413+0000 7f51a9148700 1 --2- 172.21.15.144:0/3189193830 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f518c06e200 0x7f518c0706b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.413+0000 7f51a9148700 1 --2- 172.21.15.144:0/3189193830 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f51a40ff390 0x7f51a4196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.413+0000 7f51a9148700 1 --2- 172.21.15.144:0/3189193830 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f51a40ffd30 0x7f51a4197160 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.413+0000 7f51a9148700 1 -- 172.21.15.144:0/3189193830 >> 172.21.15.144:0/3189193830 conn(0x7f51a4074f50 msgr2=0x7f51a40fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:16.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.413+0000 7f51a9148700 1 -- 172.21.15.144:0/3189193830 shutdown_connections 2024-09-10T12:01:16.415 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.414+0000 7f51a9148700 1 -- 172.21.15.144:0/3189193830 wait complete. 2024-09-10T12:01:16.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.515+0000 7fb8c4315700 1 -- 172.21.15.144:0/2976184652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc100d80 msgr2=0x7fb8bc104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:16.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.515+0000 7fb8c4315700 1 --2- 172.21.15.144:0/2976184652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc100d80 0x7fb8bc104d50 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fb8ac009910 tx=0x7fb8ac009c20 comp rx=0 tx=0).stop 2024-09-10T12:01:16.521 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.516+0000 7fb8c4315700 1 -- 172.21.15.144:0/2976184652 shutdown_connections 2024-09-10T12:01:16.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.516+0000 7fb8c4315700 1 --2- 172.21.15.144:0/2976184652 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc100d80 0x7fb8bc104d50 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.516+0000 7fb8c4315700 1 --2- 172.21.15.144:0/2976184652 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8bc1003e0 0x7fb8bc1007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.516+0000 7fb8c4315700 1 -- 172.21.15.144:0/2976184652 >> 172.21.15.144:0/2976184652 conn(0x7fb8bc0fbca0 msgr2=0x7fb8bc0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:16.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.516+0000 7fb8c4315700 1 -- 172.21.15.144:0/2976184652 shutdown_connections 2024-09-10T12:01:16.522 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.516+0000 7fb8c4315700 1 -- 172.21.15.144:0/2976184652 wait complete. 2024-09-10T12:01:16.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.516+0000 7fb8c4315700 1 Processor -- start 2024-09-10T12:01:16.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.516+0000 7fb8c4315700 1 -- start start 2024-09-10T12:01:16.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8c4315700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc1003e0 0x7fb8bc195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:16.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8c4315700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8bc100d80 0x7fb8bc196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:16.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8c4315700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc1969f0 con 0x7fb8bc1003e0 2024-09-10T12:01:16.523 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8c4315700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8bc19a780 con 0x7fb8bc100d80 2024-09-10T12:01:16.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8c20b1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc1003e0 0x7fb8bc195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:16.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8c20b1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc1003e0 0x7fb8bc195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56656/0 (socket says 172.21.15.144:56656) 2024-09-10T12:01:16.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8c20b1700 1 -- 172.21.15.144:0/1282338908 learned_addr learned my addr 172.21.15.144:0/1282338908 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:01:16.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8c18b0700 1 --2- 172.21.15.144:0/1282338908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8bc100d80 0x7fb8bc196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:16.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8c20b1700 1 -- 172.21.15.144:0/1282338908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8bc100d80 msgr2=0x7fb8bc196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:16.524 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8c20b1700 1 --2- 172.21.15.144:0/1282338908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8bc100d80 0x7fb8bc196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8c20b1700 1 -- 172.21.15.144:0/1282338908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ac0095f0 con 0x7fb8bc1003e0 2024-09-10T12:01:16.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8c20b1700 1 --2- 172.21.15.144:0/1282338908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc1003e0 0x7fb8bc195dd0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c8a0 tx=0x7fb8b800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:16.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8b37fe700 1 -- 172.21.15.144:0/1282338908 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8b800cea0 con 0x7fb8bc1003e0 2024-09-10T12:01:16.525 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8b37fe700 1 -- 172.21.15.144:0/1282338908 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8b8004830 con 0x7fb8bc1003e0 2024-09-10T12:01:16.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8b37fe700 1 -- 172.21.15.144:0/1282338908 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8b80055b0 con 0x7fb8bc1003e0 2024-09-10T12:01:16.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8c4315700 1 -- 172.21.15.144:0/1282338908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8bc19aa60 con 0x7fb8bc1003e0 2024-09-10T12:01:16.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.517+0000 7fb8c4315700 1 -- 172.21.15.144:0/1282338908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8bc19afb0 con 0x7fb8bc1003e0 2024-09-10T12:01:16.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.518+0000 7fb8c4315700 1 -- 172.21.15.144:0/1282338908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8bc04ea50 con 0x7fb8bc1003e0 2024-09-10T12:01:16.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.519+0000 7fb8b37fe700 1 -- 172.21.15.144:0/1282338908 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb8b80049a0 con 0x7fb8bc1003e0 2024-09-10T12:01:16.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.519+0000 7fb8b37fe700 1 --2- 172.21.15.144:0/1282338908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb8a806e200 0x7fb8a80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:16.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.519+0000 7fb8b37fe700 1 -- 172.21.15.144:0/1282338908 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb8b808dc80 con 0x7fb8bc1003e0 2024-09-10T12:01:16.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.520+0000 7fb8c18b0700 1 --2- 172.21.15.144:0/1282338908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb8a806e200 0x7fb8a80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:16.526 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.520+0000 7fb8c18b0700 1 --2- 172.21.15.144:0/1282338908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb8a806e200 0x7fb8a80706b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc1973f0 tx=0x7fb8ac01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:16.527 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.524+0000 7fb8b37fe700 1 -- 172.21.15.144:0/1282338908 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb8b80570e0 con 0x7fb8bc1003e0 2024-09-10T12:01:16.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.807+0000 7fb8c4315700 1 -- 172.21.15.144:0/1282338908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb8bc19b4b0 con 0x7fb8bc1003e0 2024-09-10T12:01:16.809 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:16 smithi144 ceph-mon[87436]: pgmap v879: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:16.809 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:16 smithi144 ceph-mon[87436]: from='client.35604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:16.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.808+0000 7fb8b37fe700 1 -- 172.21.15.144:0/1282338908 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fb8b801d070 con 0x7fb8bc1003e0 2024-09-10T12:01:16.810 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:01:16.810 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:01:16.810 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:01:16.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:01:16.810 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:01:16.810 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:01:16.810 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:01:16.810 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:01:16.810 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:01:16.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:01:16.811 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:01:16.811 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:01:16.811 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:01:16.811 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:01:16.811 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:01:16.811 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:01:16.811 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:01:16.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.810+0000 7fb8c4315700 1 -- 172.21.15.144:0/1282338908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb8a806e200 msgr2=0x7fb8a80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:16.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.810+0000 7fb8c4315700 1 --2- 172.21.15.144:0/1282338908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb8a806e200 0x7fb8a80706b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fb8bc1973f0 tx=0x7fb8ac01a040 comp rx=0 tx=0).stop 2024-09-10T12:01:16.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.810+0000 7fb8c4315700 1 -- 172.21.15.144:0/1282338908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc1003e0 msgr2=0x7fb8bc195dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:16.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.810+0000 7fb8c4315700 1 --2- 172.21.15.144:0/1282338908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc1003e0 0x7fb8bc195dd0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fb8b800c8a0 tx=0x7fb8b800cc60 comp rx=0 tx=0).stop 2024-09-10T12:01:16.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.810+0000 7fb8c4315700 1 -- 172.21.15.144:0/1282338908 shutdown_connections 2024-09-10T12:01:16.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.810+0000 7fb8c4315700 1 --2- 172.21.15.144:0/1282338908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb8a806e200 0x7fb8a80706b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.810+0000 7fb8c4315700 1 --2- 172.21.15.144:0/1282338908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8bc1003e0 0x7fb8bc195dd0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.810+0000 7fb8c4315700 1 --2- 172.21.15.144:0/1282338908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8bc100d80 0x7fb8bc196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.810+0000 7fb8c4315700 1 -- 172.21.15.144:0/1282338908 >> 172.21.15.144:0/1282338908 conn(0x7fb8bc0fbca0 msgr2=0x7fb8bc0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:16.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.810+0000 7fb8c4315700 1 -- 172.21.15.144:0/1282338908 shutdown_connections 2024-09-10T12:01:16.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.810+0000 7fb8c4315700 1 -- 172.21.15.144:0/1282338908 wait complete. 2024-09-10T12:01:16.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.915+0000 7f29c5998700 1 -- 172.21.15.144:0/2530090292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29c0100da0 msgr2=0x7f29c0104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:16.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.915+0000 7f29c5998700 1 --2- 172.21.15.144:0/2530090292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29c0100da0 0x7f29c0104d70 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f29a8009910 tx=0x7f29a8009c20 comp rx=0 tx=0).stop 2024-09-10T12:01:16.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.915+0000 7f29c5998700 1 -- 172.21.15.144:0/2530090292 shutdown_connections 2024-09-10T12:01:16.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.915+0000 7f29c5998700 1 --2- 172.21.15.144:0/2530090292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29c0100da0 0x7f29c0104d70 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.915+0000 7f29c5998700 1 --2- 172.21.15.144:0/2530090292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29c0100400 0x7f29c01007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.915+0000 7f29c5998700 1 -- 172.21.15.144:0/2530090292 >> 172.21.15.144:0/2530090292 conn(0x7f29c00fbca0 msgr2=0x7f29c00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:16.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.915+0000 7f29c5998700 1 -- 172.21.15.144:0/2530090292 shutdown_connections 2024-09-10T12:01:16.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.915+0000 7f29c5998700 1 -- 172.21.15.144:0/2530090292 wait complete. 2024-09-10T12:01:16.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.916+0000 7f29c5998700 1 Processor -- start 2024-09-10T12:01:16.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.916+0000 7f29c5998700 1 -- start start 2024-09-10T12:01:16.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.916+0000 7f29c5998700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29c0100400 0x7f29c0195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:16.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.916+0000 7f29c5998700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29c0100da0 0x7f29c01963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:16.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.916+0000 7f29c5998700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c0196a90 con 0x7f29c0100da0 2024-09-10T12:01:16.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.916+0000 7f29c5998700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c019a820 con 0x7f29c0100400 2024-09-10T12:01:16.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.916+0000 7f29b7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29c0100da0 0x7f29c01963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:16.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.916+0000 7f29b7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29c0100da0 0x7f29c01963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56680/0 (socket says 172.21.15.144:56680) 2024-09-10T12:01:16.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.916+0000 7f29b7fff700 1 -- 172.21.15.144:0/1579589692 learned_addr learned my addr 172.21.15.144:0/1579589692 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:01:16.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.916+0000 7f29b7fff700 1 -- 172.21.15.144:0/1579589692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29c0100400 msgr2=0x7f29c0195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:16.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.916+0000 7f29beffd700 1 --2- 172.21.15.144:0/1579589692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29c0100400 0x7f29c0195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:16.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.916+0000 7f29b7fff700 1 --2- 172.21.15.144:0/1579589692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29c0100400 0x7f29c0195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:16.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.916+0000 7f29b7fff700 1 -- 172.21.15.144:0/1579589692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a80095f0 con 0x7f29c0100da0 2024-09-10T12:01:16.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.916+0000 7f29b7fff700 1 --2- 172.21.15.144:0/1579589692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29c0100da0 0x7f29c01963b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f29a8005f50 tx=0x7f29a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:16.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.916+0000 7f29c4996700 1 -- 172.21.15.144:0/1579589692 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29a801d070 con 0x7f29c0100da0 2024-09-10T12:01:16.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.917+0000 7f29c4996700 1 -- 172.21.15.144:0/1579589692 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f29a800f460 con 0x7f29c0100da0 2024-09-10T12:01:16.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.917+0000 7f29c5998700 1 -- 172.21.15.144:0/1579589692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c019aaa0 con 0x7f29c0100da0 2024-09-10T12:01:16.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.917+0000 7f29c4996700 1 -- 172.21.15.144:0/1579589692 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29a8017630 con 0x7f29c0100da0 2024-09-10T12:01:16.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.917+0000 7f29c5998700 1 -- 172.21.15.144:0/1579589692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c019af90 con 0x7f29c0100da0 2024-09-10T12:01:16.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.917+0000 7f29c5998700 1 -- 172.21.15.144:0/1579589692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c01086c0 con 0x7f29c0100da0 2024-09-10T12:01:16.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.918+0000 7f29c4996700 1 -- 172.21.15.144:0/1579589692 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f29a800f5d0 con 0x7f29c0100da0 2024-09-10T12:01:16.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.918+0000 7f29c4996700 1 --2- 172.21.15.144:0/1579589692 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f29ac06e200 0x7f29ac0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:16.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.919+0000 7f29c4996700 1 -- 172.21.15.144:0/1579589692 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f29a808f6a0 con 0x7f29c0100da0 2024-09-10T12:01:16.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.919+0000 7f29beffd700 1 --2- 172.21.15.144:0/1579589692 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f29ac06e200 0x7f29ac0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:16.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.919+0000 7f29beffd700 1 --2- 172.21.15.144:0/1579589692 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f29ac06e200 0x7f29ac0706b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f29b0006fd0 tx=0x7f29b0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:16.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:16.923+0000 7f29c4996700 1 -- 172.21.15.144:0/1579589692 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f29a8058a80 con 0x7f29c0100da0 2024-09-10T12:01:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:16 smithi178 ceph-mon[65940]: pgmap v879: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:16 smithi178 ceph-mon[65940]: from='client.35604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:17.168 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.167+0000 7f29c5998700 1 -- 172.21.15.144:0/1579589692 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f29c004ea50 con 0x7f29c0100da0 2024-09-10T12:01:17.169 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.168+0000 7f29c4996700 1 -- 172.21.15.144:0/1579589692 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f29a805c140 con 0x7f29c0100da0 2024-09-10T12:01:17.169 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:01:17.169 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:01:17.169 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:01:17.169 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:01:17.170 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:01:17.170 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:01:17.170 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:01:17.170 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:01:17.170 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:01:17.170 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:01:17.170 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:01:17.170 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:01:17.170 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:01:17.170 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:01:17.170 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:01:17.170 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:01:17.170 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:01:17.170 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:01:17.171 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:01:17.171 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:01:17.171 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:01:17.171 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:01:17.171 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:01:17.171 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:01:17.171 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:01:17.171 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:01:17.171 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:01:17.171 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:01:17.171 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:01:17.171 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:01:17.171 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:01:17.171 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:01:17.172 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:01:17.172 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:01:17.172 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:01:17.172 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:01:17.172 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:01:17.172 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:01:17.172 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:01:17.172 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:01:17.172 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:01:17.172 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:01:17.172 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:01:17.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.170+0000 7f29c5998700 1 -- 172.21.15.144:0/1579589692 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f29ac06e200 msgr2=0x7f29ac0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:17.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.170+0000 7f29c5998700 1 --2- 172.21.15.144:0/1579589692 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f29ac06e200 0x7f29ac0706b0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f29b0006fd0 tx=0x7f29b0008040 comp rx=0 tx=0).stop 2024-09-10T12:01:17.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.170+0000 7f29c5998700 1 -- 172.21.15.144:0/1579589692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29c0100da0 msgr2=0x7f29c01963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:17.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.170+0000 7f29c5998700 1 --2- 172.21.15.144:0/1579589692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29c0100da0 0x7f29c01963b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f29a8005f50 tx=0x7f29a800be20 comp rx=0 tx=0).stop 2024-09-10T12:01:17.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.170+0000 7f29c5998700 1 -- 172.21.15.144:0/1579589692 shutdown_connections 2024-09-10T12:01:17.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.170+0000 7f29c5998700 1 --2- 172.21.15.144:0/1579589692 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f29ac06e200 0x7f29ac0706b0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:17.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.170+0000 7f29c5998700 1 --2- 172.21.15.144:0/1579589692 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29c0100400 0x7f29c0195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:17.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.170+0000 7f29c5998700 1 --2- 172.21.15.144:0/1579589692 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29c0100da0 0x7f29c01963b0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:17.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.170+0000 7f29c5998700 1 -- 172.21.15.144:0/1579589692 >> 172.21.15.144:0/1579589692 conn(0x7f29c00fbca0 msgr2=0x7f29c00fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:17.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.171+0000 7f29c5998700 1 -- 172.21.15.144:0/1579589692 shutdown_connections 2024-09-10T12:01:17.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.171+0000 7f29c5998700 1 -- 172.21.15.144:0/1579589692 wait complete. 2024-09-10T12:01:17.174 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:01:17.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.275+0000 7f975addd700 1 -- 172.21.15.144:0/533207206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754100430 msgr2=0x7f9754100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:17.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.275+0000 7f975addd700 1 --2- 172.21.15.144:0/533207206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754100430 0x7f9754100800 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9748009940 tx=0x7f9748009c50 comp rx=0 tx=0).stop 2024-09-10T12:01:17.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.275+0000 7f975addd700 1 -- 172.21.15.144:0/533207206 shutdown_connections 2024-09-10T12:01:17.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.275+0000 7f975addd700 1 --2- 172.21.15.144:0/533207206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9754100d40 0x7f97541094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:17.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.275+0000 7f975addd700 1 --2- 172.21.15.144:0/533207206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754100430 0x7f9754100800 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:17.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.275+0000 7f975addd700 1 -- 172.21.15.144:0/533207206 >> 172.21.15.144:0/533207206 conn(0x7f97540748b0 msgr2=0x7f9754074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:17.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.275+0000 7f975addd700 1 -- 172.21.15.144:0/533207206 shutdown_connections 2024-09-10T12:01:17.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.275+0000 7f975addd700 1 -- 172.21.15.144:0/533207206 wait complete. 2024-09-10T12:01:17.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.276+0000 7f975addd700 1 Processor -- start 2024-09-10T12:01:17.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.276+0000 7f975addd700 1 -- start start 2024-09-10T12:01:17.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.276+0000 7f975addd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9754100430 0x7f9754196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:17.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.276+0000 7f975addd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754100d40 0x7f9754196f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:17.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.276+0000 7f975addd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9754197590 con 0x7f9754100d40 2024-09-10T12:01:17.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.276+0000 7f975addd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97541976d0 con 0x7f9754100430 2024-09-10T12:01:17.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.276+0000 7f9753fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754100d40 0x7f9754196f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:17.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.276+0000 7f9753fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754100d40 0x7f9754196f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56698/0 (socket says 172.21.15.144:56698) 2024-09-10T12:01:17.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.276+0000 7f9753fff700 1 -- 172.21.15.144:0/279796144 learned_addr learned my addr 172.21.15.144:0/279796144 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:01:17.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.276+0000 7f9758b79700 1 --2- 172.21.15.144:0/279796144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9754100430 0x7f9754196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:17.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.276+0000 7f9753fff700 1 -- 172.21.15.144:0/279796144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9754100430 msgr2=0x7f9754196a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:17.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.276+0000 7f9753fff700 1 --2- 172.21.15.144:0/279796144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9754100430 0x7f9754196a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:17.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.276+0000 7f9753fff700 1 -- 172.21.15.144:0/279796144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97480095f0 con 0x7f9754100d40 2024-09-10T12:01:17.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.277+0000 7f9753fff700 1 --2- 172.21.15.144:0/279796144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754100d40 0x7f9754196f70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f974000c8a0 tx=0x7f974000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:17.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.277+0000 7f9751ffb700 1 -- 172.21.15.144:0/279796144 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f974000cea0 con 0x7f9754100d40 2024-09-10T12:01:17.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.277+0000 7f9751ffb700 1 -- 172.21.15.144:0/279796144 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9740004830 con 0x7f9754100d40 2024-09-10T12:01:17.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.277+0000 7f975addd700 1 -- 172.21.15.144:0/279796144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f975419faa0 con 0x7f9754100d40 2024-09-10T12:01:17.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.277+0000 7f9751ffb700 1 -- 172.21.15.144:0/279796144 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97400055b0 con 0x7f9754100d40 2024-09-10T12:01:17.280 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.277+0000 7f975addd700 1 -- 172.21.15.144:0/279796144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f975419ffe0 con 0x7f9754100d40 2024-09-10T12:01:17.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.278+0000 7f975addd700 1 -- 172.21.15.144:0/279796144 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9754106ba0 con 0x7f9754100d40 2024-09-10T12:01:17.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.278+0000 7f9751ffb700 1 -- 172.21.15.144:0/279796144 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f97400049a0 con 0x7f9754100d40 2024-09-10T12:01:17.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.279+0000 7f9751ffb700 1 --2- 172.21.15.144:0/279796144 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f974406e200 0x7f97440706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:17.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.279+0000 7f9751ffb700 1 -- 172.21.15.144:0/279796144 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f974008dd40 con 0x7f9754100d40 2024-09-10T12:01:17.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.279+0000 7f9758b79700 1 --2- 172.21.15.144:0/279796144 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f974406e200 0x7f97440706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:17.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.279+0000 7f9758b79700 1 --2- 172.21.15.144:0/279796144 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f974406e200 0x7f97440706b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9748005cd0 tx=0x7f9748005c00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:17.284 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.283+0000 7f9751ffb700 1 -- 172.21.15.144:0/279796144 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9740057120 con 0x7f9754100d40 2024-09-10T12:01:17.507 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.506+0000 7f975addd700 1 -- 172.21.15.144:0/279796144 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f975402cca0 con 0x7f974406e200 2024-09-10T12:01:17.508 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.507+0000 7f9751ffb700 1 -- 172.21.15.144:0/279796144 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f975402cca0 con 0x7f974406e200 2024-09-10T12:01:17.509 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:01:17.509 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:01:17.509 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:01:17.509 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:01:17.509 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:01:17.509 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:01:17.509 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:01:17.509 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:01:17.509 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:01:17.509 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:01:17.510 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:01:17.510 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:01:17.510 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:01:17.510 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:01:17.510 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:01:17.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.510+0000 7f975addd700 1 -- 172.21.15.144:0/279796144 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f974406e200 msgr2=0x7f97440706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:17.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.510+0000 7f975addd700 1 --2- 172.21.15.144:0/279796144 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f974406e200 0x7f97440706b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f9748005cd0 tx=0x7f9748005c00 comp rx=0 tx=0).stop 2024-09-10T12:01:17.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.510+0000 7f975addd700 1 -- 172.21.15.144:0/279796144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754100d40 msgr2=0x7f9754196f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:17.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.510+0000 7f975addd700 1 --2- 172.21.15.144:0/279796144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754100d40 0x7f9754196f70 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f974000c8a0 tx=0x7f974000cc60 comp rx=0 tx=0).stop 2024-09-10T12:01:17.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.510+0000 7f975addd700 1 -- 172.21.15.144:0/279796144 shutdown_connections 2024-09-10T12:01:17.511 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.510+0000 7f975addd700 1 --2- 172.21.15.144:0/279796144 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f974406e200 0x7f97440706b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:17.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.510+0000 7f975addd700 1 --2- 172.21.15.144:0/279796144 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9754100430 0x7f9754196a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:17.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.510+0000 7f975addd700 1 --2- 172.21.15.144:0/279796144 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9754100d40 0x7f9754196f70 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:17.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.510+0000 7f975addd700 1 -- 172.21.15.144:0/279796144 >> 172.21.15.144:0/279796144 conn(0x7f97540748b0 msgr2=0x7f9754103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:17.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.510+0000 7f975addd700 1 -- 172.21.15.144:0/279796144 shutdown_connections 2024-09-10T12:01:17.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.510+0000 7f975addd700 1 -- 172.21.15.144:0/279796144 wait complete. 2024-09-10T12:01:17.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.614+0000 7f12ec0aa700 1 -- 172.21.15.144:0/1412361894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e40ffba0 msgr2=0x7f12e410a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:17.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.614+0000 7f12ec0aa700 1 --2- 172.21.15.144:0/1412361894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e40ffba0 0x7f12e410a610 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f12e0009910 tx=0x7f12e0009c20 comp rx=0 tx=0).stop 2024-09-10T12:01:17.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.614+0000 7f12ec0aa700 1 -- 172.21.15.144:0/1412361894 shutdown_connections 2024-09-10T12:01:17.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.614+0000 7f12ec0aa700 1 --2- 172.21.15.144:0/1412361894 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e40ffba0 0x7f12e410a610 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:17.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.614+0000 7f12ec0aa700 1 --2- 172.21.15.144:0/1412361894 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12e40ff290 0x7f12e40ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:17.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.614+0000 7f12ec0aa700 1 -- 172.21.15.144:0/1412361894 >> 172.21.15.144:0/1412361894 conn(0x7f12e4074850 msgr2=0x7f12e4074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:17.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.614+0000 7f12ec0aa700 1 -- 172.21.15.144:0/1412361894 shutdown_connections 2024-09-10T12:01:17.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.614+0000 7f12ec0aa700 1 -- 172.21.15.144:0/1412361894 wait complete. 2024-09-10T12:01:17.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.615+0000 7f12ec0aa700 1 Processor -- start 2024-09-10T12:01:17.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.615+0000 7f12ec0aa700 1 -- start start 2024-09-10T12:01:17.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.615+0000 7f12ec0aa700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12e40ff290 0x7f12e4195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.615+0000 7f12ec0aa700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e40ffba0 0x7f12e4196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.615+0000 7f12ec0aa700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e41968a0 con 0x7f12e40ffba0 2024-09-10T12:01:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.615+0000 7f12ec0aa700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e41995d0 con 0x7f12e40ff290 2024-09-10T12:01:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.615+0000 7f12e9645700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e40ffba0 0x7f12e4196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.615+0000 7f12e9645700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e40ffba0 0x7f12e4196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56724/0 (socket says 172.21.15.144:56724) 2024-09-10T12:01:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.615+0000 7f12e9645700 1 -- 172.21.15.144:0/354177192 learned_addr learned my addr 172.21.15.144:0/354177192 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:01:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.615+0000 7f12e9e46700 1 --2- 172.21.15.144:0/354177192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12e40ff290 0x7f12e4195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.615+0000 7f12e9645700 1 -- 172.21.15.144:0/354177192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12e40ff290 msgr2=0x7f12e4195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.615+0000 7f12e9645700 1 --2- 172.21.15.144:0/354177192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12e40ff290 0x7f12e4195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.615+0000 7f12e9645700 1 -- 172.21.15.144:0/354177192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e00095f0 con 0x7f12e40ffba0 2024-09-10T12:01:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.616+0000 7f12e9645700 1 --2- 172.21.15.144:0/354177192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e40ffba0 0x7f12e4196210 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f12e0005f50 tx=0x7f12e000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:17.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.616+0000 7f12daffd700 1 -- 172.21.15.144:0/354177192 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12e001d070 con 0x7f12e40ffba0 2024-09-10T12:01:17.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.616+0000 7f12daffd700 1 -- 172.21.15.144:0/354177192 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f12e000b610 con 0x7f12e40ffba0 2024-09-10T12:01:17.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.616+0000 7f12ec0aa700 1 -- 172.21.15.144:0/354177192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e4199850 con 0x7f12e40ffba0 2024-09-10T12:01:17.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.616+0000 7f12daffd700 1 -- 172.21.15.144:0/354177192 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12e0022470 con 0x7f12e40ffba0 2024-09-10T12:01:17.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.616+0000 7f12ec0aa700 1 -- 172.21.15.144:0/354177192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e4199dc0 con 0x7f12e40ffba0 2024-09-10T12:01:17.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.617+0000 7f12ec0aa700 1 -- 172.21.15.144:0/354177192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e4107d10 con 0x7f12e40ffba0 2024-09-10T12:01:17.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.617+0000 7f12daffd700 1 -- 172.21.15.144:0/354177192 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f12e000fca0 con 0x7f12e40ffba0 2024-09-10T12:01:17.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.618+0000 7f12daffd700 1 --2- 172.21.15.144:0/354177192 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f12d006e200 0x7f12d00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:17.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.618+0000 7f12daffd700 1 -- 172.21.15.144:0/354177192 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f12e008f540 con 0x7f12e40ffba0 2024-09-10T12:01:17.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.618+0000 7f12e9e46700 1 --2- 172.21.15.144:0/354177192 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f12d006e200 0x7f12d00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:17.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.618+0000 7f12e9e46700 1 --2- 172.21.15.144:0/354177192 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f12d006e200 0x7f12d00706b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f12e4077330 tx=0x7f12d4009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:17.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.622+0000 7f12daffd700 1 -- 172.21.15.144:0/354177192 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f12e0058920 con 0x7f12e40ffba0 2024-09-10T12:01:17.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.909+0000 7f12ec0aa700 1 -- 172.21.15.144:0/354177192 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f12e402cc30 con 0x7f12e40ffba0 2024-09-10T12:01:17.911 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:17 smithi144 ceph-mon[87436]: from='client.35608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:17.911 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:17 smithi144 ceph-mon[87436]: from='client.35612 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:17.911 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:17 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1282338908' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:01:17.911 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:17 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1579589692' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:01:17.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.909+0000 7f12daffd700 1 -- 172.21.15.144:0/354177192 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f12e005bfe0 con 0x7f12e40ffba0 2024-09-10T12:01:17.911 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:01:17.911 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:01:17.912 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:01:17.912 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:01:17.912 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:01:17.912 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:01:17.912 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:01:17.912 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:01:17.912 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:01:17.912 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:01:17.912 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:01:17.912 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:01:17.912 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:01:17.912 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:01:17.913 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:01:17.913 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:01:17.913 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:01:17.913 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:01:17.913 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:01:17.913 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:01:17.913 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:01:17.913 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:01:17.913 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:01:17.913 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:01:17.913 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:01:17.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.912+0000 7f12ec0aa700 1 -- 172.21.15.144:0/354177192 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f12d006e200 msgr2=0x7f12d00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:17.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.912+0000 7f12ec0aa700 1 --2- 172.21.15.144:0/354177192 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f12d006e200 0x7f12d00706b0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f12e4077330 tx=0x7f12d4009260 comp rx=0 tx=0).stop 2024-09-10T12:01:17.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.913+0000 7f12ec0aa700 1 -- 172.21.15.144:0/354177192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e40ffba0 msgr2=0x7f12e4196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:17.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.913+0000 7f12ec0aa700 1 --2- 172.21.15.144:0/354177192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e40ffba0 0x7f12e4196210 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f12e0005f50 tx=0x7f12e000f740 comp rx=0 tx=0).stop 2024-09-10T12:01:17.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.913+0000 7f12ec0aa700 1 -- 172.21.15.144:0/354177192 shutdown_connections 2024-09-10T12:01:17.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.913+0000 7f12ec0aa700 1 --2- 172.21.15.144:0/354177192 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f12d006e200 0x7f12d00706b0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:17.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.913+0000 7f12ec0aa700 1 --2- 172.21.15.144:0/354177192 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f12e40ff290 0x7f12e4195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:17.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.913+0000 7f12ec0aa700 1 --2- 172.21.15.144:0/354177192 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f12e40ffba0 0x7f12e4196210 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:17.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.913+0000 7f12ec0aa700 1 -- 172.21.15.144:0/354177192 >> 172.21.15.144:0/354177192 conn(0x7f12e4074850 msgr2=0x7f12e4104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:17.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.913+0000 7f12ec0aa700 1 -- 172.21.15.144:0/354177192 shutdown_connections 2024-09-10T12:01:17.915 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:17.913+0000 7f12ec0aa700 1 -- 172.21.15.144:0/354177192 wait complete. 2024-09-10T12:01:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:17 smithi178 ceph-mon[65940]: from='client.35608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:17 smithi178 ceph-mon[65940]: from='client.35612 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:17 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1282338908' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:01:18.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:17 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1579589692' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:01:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:18 smithi178 ceph-mon[65940]: pgmap v880: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:18 smithi178 ceph-mon[65940]: from='client.35624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:18 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/354177192' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:01:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:18 smithi144 ceph-mon[87436]: pgmap v880: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:18 smithi144 ceph-mon[87436]: from='client.35624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:18 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/354177192' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:01:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:20 smithi178 ceph-mon[65940]: pgmap v881: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:21.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:20 smithi144 ceph-mon[87436]: pgmap v881: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:23.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:22 smithi144 ceph-mon[87436]: pgmap v882: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:23.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:22 smithi178 ceph-mon[65940]: pgmap v882: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:24 smithi178 ceph-mon[65940]: pgmap v883: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:24 smithi144 ceph-mon[87436]: pgmap v883: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:26 smithi178 ceph-mon[65940]: pgmap v884: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:26 smithi144 ceph-mon[87436]: pgmap v884: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:28 smithi178 ceph-mon[65940]: pgmap v885: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:01:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:28 smithi144 ceph-mon[87436]: pgmap v885: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:01:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:30 smithi178 ceph-mon[65940]: pgmap v886: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:31.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:30 smithi144 ceph-mon[87436]: pgmap v886: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:32 smithi144 ceph-mon[87436]: pgmap v887: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:32 smithi178 ceph-mon[65940]: pgmap v887: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:34 smithi178 ceph-mon[65940]: pgmap v888: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:34 smithi144 ceph-mon[87436]: pgmap v888: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:36 smithi178 ceph-mon[65940]: pgmap v889: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:36 smithi144 ceph-mon[87436]: pgmap v889: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:38 smithi178 ceph-mon[65940]: pgmap v890: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:38 smithi144 ceph-mon[87436]: pgmap v890: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:40 smithi178 ceph-mon[65940]: pgmap v891: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:40 smithi144 ceph-mon[87436]: pgmap v891: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:43.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:42 smithi144 ceph-mon[87436]: pgmap v892: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:42 smithi178 ceph-mon[65940]: pgmap v892: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:01:44.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:01:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:44 smithi178 ceph-mon[65940]: pgmap v893: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:44 smithi144 ceph-mon[87436]: pgmap v893: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:46 smithi178 ceph-mon[65940]: pgmap v894: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:46 smithi144 ceph-mon[87436]: pgmap v894: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:48.037 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.034+0000 7f8a47282700 1 -- 172.21.15.144:0/1747541468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a400ffc60 msgr2=0x7f8a40103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:48.037 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.034+0000 7f8a47282700 1 --2- 172.21.15.144:0/1747541468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a400ffc60 0x7f8a40103c30 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009910 tx=0x7f8a3c009c20 comp rx=0 tx=0).stop 2024-09-10T12:01:48.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.034+0000 7f8a47282700 1 -- 172.21.15.144:0/1747541468 shutdown_connections 2024-09-10T12:01:48.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.034+0000 7f8a47282700 1 --2- 172.21.15.144:0/1747541468 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a400ffc60 0x7f8a40103c30 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.034+0000 7f8a47282700 1 --2- 172.21.15.144:0/1747541468 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a400ff2c0 0x7f8a400ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.034+0000 7f8a47282700 1 -- 172.21.15.144:0/1747541468 >> 172.21.15.144:0/1747541468 conn(0x7f8a400fab80 msgr2=0x7f8a400fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:48.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.034+0000 7f8a47282700 1 -- 172.21.15.144:0/1747541468 shutdown_connections 2024-09-10T12:01:48.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.034+0000 7f8a47282700 1 -- 172.21.15.144:0/1747541468 wait complete. 2024-09-10T12:01:48.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.035+0000 7f8a47282700 1 Processor -- start 2024-09-10T12:01:48.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.035+0000 7f8a47282700 1 -- start start 2024-09-10T12:01:48.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.035+0000 7f8a47282700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a400ff2c0 0x7f8a40195df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:48.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.035+0000 7f8a47282700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a400ffc60 0x7f8a40196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:48.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.035+0000 7f8a47282700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a40196a10 con 0x7f8a400ffc60 2024-09-10T12:01:48.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.035+0000 7f8a47282700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4019a7a0 con 0x7f8a400ff2c0 2024-09-10T12:01:48.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.035+0000 7f8a4481d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a400ffc60 0x7f8a40196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:48.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.035+0000 7f8a4481d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a400ffc60 0x7f8a40196330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42086/0 (socket says 172.21.15.144:42086) 2024-09-10T12:01:48.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.035+0000 7f8a4481d700 1 -- 172.21.15.144:0/3004000711 learned_addr learned my addr 172.21.15.144:0/3004000711 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:01:48.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.035+0000 7f8a4501e700 1 --2- 172.21.15.144:0/3004000711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a400ff2c0 0x7f8a40195df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:48.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.035+0000 7f8a4481d700 1 -- 172.21.15.144:0/3004000711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a400ff2c0 msgr2=0x7f8a40195df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:48.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.035+0000 7f8a4481d700 1 --2- 172.21.15.144:0/3004000711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a400ff2c0 0x7f8a40195df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.035+0000 7f8a4481d700 1 -- 172.21.15.144:0/3004000711 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c0095f0 con 0x7f8a400ffc60 2024-09-10T12:01:48.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.036+0000 7f8a4481d700 1 --2- 172.21.15.144:0/3004000711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a400ffc60 0x7f8a40196330 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0092e0 tx=0x7f8a3c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:48.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.036+0000 7f8a367fc700 1 -- 172.21.15.144:0/3004000711 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3c01d070 con 0x7f8a400ffc60 2024-09-10T12:01:48.039 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.036+0000 7f8a47282700 1 -- 172.21.15.144:0/3004000711 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4019aa20 con 0x7f8a400ffc60 2024-09-10T12:01:48.040 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.036+0000 7f8a367fc700 1 -- 172.21.15.144:0/3004000711 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a3c005dc0 con 0x7f8a400ffc60 2024-09-10T12:01:48.040 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.036+0000 7f8a367fc700 1 -- 172.21.15.144:0/3004000711 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3c017400 con 0x7f8a400ffc60 2024-09-10T12:01:48.040 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.036+0000 7f8a47282700 1 -- 172.21.15.144:0/3004000711 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4019af10 con 0x7f8a400ffc60 2024-09-10T12:01:48.041 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.037+0000 7f8a47282700 1 -- 172.21.15.144:0/3004000711 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a4004ea50 con 0x7f8a400ffc60 2024-09-10T12:01:48.041 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.037+0000 7f8a367fc700 1 -- 172.21.15.144:0/3004000711 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8a3c00f460 con 0x7f8a400ffc60 2024-09-10T12:01:48.041 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.038+0000 7f8a367fc700 1 --2- 172.21.15.144:0/3004000711 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a2c06e200 0x7f8a2c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:48.041 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.038+0000 7f8a4501e700 1 --2- 172.21.15.144:0/3004000711 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a2c06e200 0x7f8a2c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:48.041 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.038+0000 7f8a367fc700 1 -- 172.21.15.144:0/3004000711 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8a3c08f590 con 0x7f8a400ffc60 2024-09-10T12:01:48.041 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.039+0000 7f8a4501e700 1 --2- 172.21.15.144:0/3004000711 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a2c06e200 0x7f8a2c0706b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8a30005950 tx=0x7f8a300058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:48.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.043+0000 7f8a367fc700 1 -- 172.21.15.144:0/3004000711 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8a3c058970 con 0x7f8a400ffc60 2024-09-10T12:01:48.268 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.267+0000 7f8a47282700 1 -- 172.21.15.144:0/3004000711 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a4019b3f0 con 0x7f8a2c06e200 2024-09-10T12:01:48.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.268+0000 7f8a367fc700 1 -- 172.21.15.144:0/3004000711 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f8a4019b3f0 con 0x7f8a2c06e200 2024-09-10T12:01:48.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.270+0000 7f8a47282700 1 -- 172.21.15.144:0/3004000711 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a2c06e200 msgr2=0x7f8a2c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:48.271 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.270+0000 7f8a47282700 1 --2- 172.21.15.144:0/3004000711 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a2c06e200 0x7f8a2c0706b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8a30005950 tx=0x7f8a300058e0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.270+0000 7f8a47282700 1 -- 172.21.15.144:0/3004000711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a400ffc60 msgr2=0x7f8a40196330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:48.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.270+0000 7f8a47282700 1 --2- 172.21.15.144:0/3004000711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a400ffc60 0x7f8a40196330 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c0092e0 tx=0x7f8a3c00be20 comp rx=0 tx=0).stop 2024-09-10T12:01:48.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.270+0000 7f8a47282700 1 -- 172.21.15.144:0/3004000711 shutdown_connections 2024-09-10T12:01:48.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.270+0000 7f8a47282700 1 --2- 172.21.15.144:0/3004000711 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a2c06e200 0x7f8a2c0706b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.270+0000 7f8a47282700 1 --2- 172.21.15.144:0/3004000711 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a400ff2c0 0x7f8a40195df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.270+0000 7f8a47282700 1 --2- 172.21.15.144:0/3004000711 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a400ffc60 0x7f8a40196330 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.270+0000 7f8a47282700 1 -- 172.21.15.144:0/3004000711 >> 172.21.15.144:0/3004000711 conn(0x7f8a400fab80 msgr2=0x7f8a400fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:48.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.270+0000 7f8a47282700 1 -- 172.21.15.144:0/3004000711 shutdown_connections 2024-09-10T12:01:48.272 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.270+0000 7f8a47282700 1 -- 172.21.15.144:0/3004000711 wait complete. 2024-09-10T12:01:48.282 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:01:48.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.377+0000 7f6a9ff1a700 1 -- 172.21.15.144:0/1132250229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a981003e0 msgr2=0x7f6a981007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:48.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.377+0000 7f6a9ff1a700 1 --2- 172.21.15.144:0/1132250229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a981003e0 0x7f6a981007b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f6a94009910 tx=0x7f6a94009c20 comp rx=0 tx=0).stop 2024-09-10T12:01:48.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.377+0000 7f6a9ff1a700 1 -- 172.21.15.144:0/1132250229 shutdown_connections 2024-09-10T12:01:48.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.377+0000 7f6a9ff1a700 1 --2- 172.21.15.144:0/1132250229 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a98100cf0 0x7f6a98109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.377+0000 7f6a9ff1a700 1 --2- 172.21.15.144:0/1132250229 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a981003e0 0x7f6a981007b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.377+0000 7f6a9ff1a700 1 -- 172.21.15.144:0/1132250229 >> 172.21.15.144:0/1132250229 conn(0x7f6a980747e0 msgr2=0x7f6a98074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:48.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.377+0000 7f6a9ff1a700 1 -- 172.21.15.144:0/1132250229 shutdown_connections 2024-09-10T12:01:48.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.377+0000 7f6a9ff1a700 1 -- 172.21.15.144:0/1132250229 wait complete. 2024-09-10T12:01:48.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.378+0000 7f6a9ff1a700 1 Processor -- start 2024-09-10T12:01:48.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.378+0000 7f6a9ff1a700 1 -- start start 2024-09-10T12:01:48.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.378+0000 7f6a9ff1a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a981003e0 0x7f6a98196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:48.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.378+0000 7f6a9ff1a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a98100cf0 0x7f6a98196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:48.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.378+0000 7f6a9ff1a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a981975e0 con 0x7f6a98100cf0 2024-09-10T12:01:48.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.378+0000 7f6a9ff1a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a98197720 con 0x7f6a981003e0 2024-09-10T12:01:48.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.378+0000 7f6a9d4b5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a98100cf0 0x7f6a98196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:48.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.378+0000 7f6a9d4b5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a98100cf0 0x7f6a98196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42104/0 (socket says 172.21.15.144:42104) 2024-09-10T12:01:48.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.378+0000 7f6a9d4b5700 1 -- 172.21.15.144:0/619543149 learned_addr learned my addr 172.21.15.144:0/619543149 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:01:48.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.378+0000 7f6a9dcb6700 1 --2- 172.21.15.144:0/619543149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a981003e0 0x7f6a98196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:48.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.379+0000 7f6a9d4b5700 1 -- 172.21.15.144:0/619543149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a981003e0 msgr2=0x7f6a98196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:48.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.379+0000 7f6a9d4b5700 1 --2- 172.21.15.144:0/619543149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a981003e0 0x7f6a98196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.379+0000 7f6a9d4b5700 1 -- 172.21.15.144:0/619543149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a940095f0 con 0x7f6a98100cf0 2024-09-10T12:01:48.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.379+0000 7f6a9d4b5700 1 --2- 172.21.15.144:0/619543149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a98100cf0 0x7f6a98196fc0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800d900 tx=0x7f6a8800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:48.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.379+0000 7f6a8effd700 1 -- 172.21.15.144:0/619543149 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a88004a00 con 0x7f6a98100cf0 2024-09-10T12:01:48.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.379+0000 7f6a8effd700 1 -- 172.21.15.144:0/619543149 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a880054e0 con 0x7f6a98100cf0 2024-09-10T12:01:48.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.379+0000 7f6a9ff1a700 1 -- 172.21.15.144:0/619543149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a9819faf0 con 0x7f6a98100cf0 2024-09-10T12:01:48.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.379+0000 7f6a8effd700 1 -- 172.21.15.144:0/619543149 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a88009d00 con 0x7f6a98100cf0 2024-09-10T12:01:48.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.379+0000 7f6a9ff1a700 1 -- 172.21.15.144:0/619543149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a981a0030 con 0x7f6a98100cf0 2024-09-10T12:01:48.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.380+0000 7f6a9ff1a700 1 -- 172.21.15.144:0/619543149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a98106b50 con 0x7f6a98100cf0 2024-09-10T12:01:48.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.381+0000 7f6a8effd700 1 -- 172.21.15.144:0/619543149 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6a88005020 con 0x7f6a98100cf0 2024-09-10T12:01:48.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.381+0000 7f6a8effd700 1 --2- 172.21.15.144:0/619543149 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6a8406e200 0x7f6a840706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:48.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.381+0000 7f6a8effd700 1 -- 172.21.15.144:0/619543149 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6a8808df50 con 0x7f6a98100cf0 2024-09-10T12:01:48.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.381+0000 7f6a9dcb6700 1 --2- 172.21.15.144:0/619543149 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6a8406e200 0x7f6a840706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:48.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.381+0000 7f6a9dcb6700 1 --2- 172.21.15.144:0/619543149 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6a8406e200 0x7f6a840706b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6a94006010 tx=0x7f6a9401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:48.386 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.385+0000 7f6a8effd700 1 -- 172.21.15.144:0/619543149 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6a88057350 con 0x7f6a98100cf0 2024-09-10T12:01:48.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.632+0000 7f6a9ff1a700 1 -- 172.21.15.144:0/619543149 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a98061190 con 0x7f6a8406e200 2024-09-10T12:01:48.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.633+0000 7f6a8effd700 1 -- 172.21.15.144:0/619543149 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f6a98061190 con 0x7f6a8406e200 2024-09-10T12:01:48.636 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.635+0000 7f6a9ff1a700 1 -- 172.21.15.144:0/619543149 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6a8406e200 msgr2=0x7f6a840706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:48.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.635+0000 7f6a9ff1a700 1 --2- 172.21.15.144:0/619543149 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6a8406e200 0x7f6a840706b0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6a94006010 tx=0x7f6a9401a040 comp rx=0 tx=0).stop 2024-09-10T12:01:48.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.635+0000 7f6a9ff1a700 1 -- 172.21.15.144:0/619543149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a98100cf0 msgr2=0x7f6a98196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:48.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.635+0000 7f6a9ff1a700 1 --2- 172.21.15.144:0/619543149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a98100cf0 0x7f6a98196fc0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f6a8800d900 tx=0x7f6a8800dc10 comp rx=0 tx=0).stop 2024-09-10T12:01:48.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.635+0000 7f6a9ff1a700 1 -- 172.21.15.144:0/619543149 shutdown_connections 2024-09-10T12:01:48.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.635+0000 7f6a9ff1a700 1 --2- 172.21.15.144:0/619543149 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6a8406e200 0x7f6a840706b0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.635+0000 7f6a9ff1a700 1 --2- 172.21.15.144:0/619543149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6a981003e0 0x7f6a98196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.635+0000 7f6a9ff1a700 1 --2- 172.21.15.144:0/619543149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6a98100cf0 0x7f6a98196fc0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.635+0000 7f6a9ff1a700 1 -- 172.21.15.144:0/619543149 >> 172.21.15.144:0/619543149 conn(0x7f6a980747e0 msgr2=0x7f6a98103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:48.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.635+0000 7f6a9ff1a700 1 -- 172.21.15.144:0/619543149 shutdown_connections 2024-09-10T12:01:48.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.635+0000 7f6a9ff1a700 1 -- 172.21.15.144:0/619543149 wait complete. 2024-09-10T12:01:48.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.740+0000 7f9a76753700 1 -- 172.21.15.144:0/110227812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70100d70 msgr2=0x7f9a70104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:48.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.740+0000 7f9a76753700 1 --2- 172.21.15.144:0/110227812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70100d70 0x7f9a70104d40 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f9a64009910 tx=0x7f9a64009c20 comp rx=0 tx=0).stop 2024-09-10T12:01:48.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.740+0000 7f9a76753700 1 -- 172.21.15.144:0/110227812 shutdown_connections 2024-09-10T12:01:48.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.740+0000 7f9a76753700 1 --2- 172.21.15.144:0/110227812 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70100d70 0x7f9a70104d40 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.740+0000 7f9a76753700 1 --2- 172.21.15.144:0/110227812 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a701003d0 0x7f9a701007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.740+0000 7f9a76753700 1 -- 172.21.15.144:0/110227812 >> 172.21.15.144:0/110227812 conn(0x7f9a700fbc90 msgr2=0x7f9a700fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:48.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.740+0000 7f9a76753700 1 -- 172.21.15.144:0/110227812 shutdown_connections 2024-09-10T12:01:48.742 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.740+0000 7f9a76753700 1 -- 172.21.15.144:0/110227812 wait complete. 2024-09-10T12:01:48.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.741+0000 7f9a76753700 1 Processor -- start 2024-09-10T12:01:48.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.741+0000 7f9a76753700 1 -- start start 2024-09-10T12:01:48.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.741+0000 7f9a76753700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a701003d0 0x7f9a70195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:48.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.741+0000 7f9a76753700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70100d70 0x7f9a70196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:48.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.741+0000 7f9a76753700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a70196960 con 0x7f9a70100d70 2024-09-10T12:01:48.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.741+0000 7f9a76753700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a7019a6f0 con 0x7f9a701003d0 2024-09-10T12:01:48.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.741+0000 7f9a6f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70100d70 0x7f9a70196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:48.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.742+0000 7f9a6f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70100d70 0x7f9a70196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42126/0 (socket says 172.21.15.144:42126) 2024-09-10T12:01:48.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.742+0000 7f9a6f7fe700 1 -- 172.21.15.144:0/3962236885 learned_addr learned my addr 172.21.15.144:0/3962236885 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:01:48.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.742+0000 7f9a6ffff700 1 --2- 172.21.15.144:0/3962236885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a701003d0 0x7f9a70195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:48.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.742+0000 7f9a6f7fe700 1 -- 172.21.15.144:0/3962236885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a701003d0 msgr2=0x7f9a70195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:48.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.742+0000 7f9a6f7fe700 1 --2- 172.21.15.144:0/3962236885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a701003d0 0x7f9a70195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.742+0000 7f9a6f7fe700 1 -- 172.21.15.144:0/3962236885 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a640095f0 con 0x7f9a70100d70 2024-09-10T12:01:48.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.742+0000 7f9a6ffff700 1 --2- 172.21.15.144:0/3962236885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a701003d0 0x7f9a70195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:01:48.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.742+0000 7f9a6f7fe700 1 --2- 172.21.15.144:0/3962236885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70100d70 0x7f9a70196280 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9a640092e0 tx=0x7f9a6400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:48.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.742+0000 7f9a6d7fa700 1 -- 172.21.15.144:0/3962236885 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a6401d070 con 0x7f9a70100d70 2024-09-10T12:01:48.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.742+0000 7f9a6d7fa700 1 -- 172.21.15.144:0/3962236885 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9a64005dc0 con 0x7f9a70100d70 2024-09-10T12:01:48.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.742+0000 7f9a76753700 1 -- 172.21.15.144:0/3962236885 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a7019a970 con 0x7f9a70100d70 2024-09-10T12:01:48.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.742+0000 7f9a6d7fa700 1 -- 172.21.15.144:0/3962236885 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a64017400 con 0x7f9a70100d70 2024-09-10T12:01:48.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.742+0000 7f9a76753700 1 -- 172.21.15.144:0/3962236885 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7019ae60 con 0x7f9a70100d70 2024-09-10T12:01:48.746 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.743+0000 7f9a76753700 1 -- 172.21.15.144:0/3962236885 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a70108740 con 0x7f9a70100d70 2024-09-10T12:01:48.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.744+0000 7f9a6d7fa700 1 -- 172.21.15.144:0/3962236885 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9a6400f460 con 0x7f9a70100d70 2024-09-10T12:01:48.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.744+0000 7f9a6d7fa700 1 --2- 172.21.15.144:0/3962236885 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9a5806e200 0x7f9a580706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:48.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.744+0000 7f9a6d7fa700 1 -- 172.21.15.144:0/3962236885 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9a6408f4d0 con 0x7f9a70100d70 2024-09-10T12:01:48.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.744+0000 7f9a6ffff700 1 --2- 172.21.15.144:0/3962236885 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9a5806e200 0x7f9a580706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:48.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.745+0000 7f9a6ffff700 1 --2- 172.21.15.144:0/3962236885 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9a5806e200 0x7f9a580706b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9a60005fd0 tx=0x7f9a60005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:48.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.749+0000 7f9a6d7fa700 1 -- 172.21.15.144:0/3962236885 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9a64058930 con 0x7f9a70100d70 2024-09-10T12:01:48.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:48 smithi144 ceph-mon[87436]: pgmap v895: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:48.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.959+0000 7f9a76753700 1 -- 172.21.15.144:0/3962236885 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a7002cd20 con 0x7f9a5806e200 2024-09-10T12:01:48.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.968+0000 7f9a6d7fa700 1 -- 172.21.15.144:0/3962236885 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f9a7002cd20 con 0x7f9a5806e200 2024-09-10T12:01:48.972 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:01:48.972 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (30m) 8m ago 40m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:01:48.973 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (40m) 8m ago 40m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:01:48.973 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (39m) 8m ago 39m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:01:48.973 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (27m) 8m ago 40m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:01:48.973 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (27m) 8m ago 39m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:01:48.973 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (29m) 8m ago 40m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:01:48.973 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (24m) 8m ago 35m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:01:48.973 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (23m) 8m ago 35m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:01:48.973 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (8m) 8m ago 35m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:01:48.973 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (23m) 8m ago 35m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:01:48.973 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (31m) 8m ago 41m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:01:48.973 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (30m) 8m ago 39m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:01:48.974 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (28m) 8m ago 42m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:01:48.974 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (28m) 8m ago 39m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:01:48.974 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (30m) 8m ago 40m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:01:48.974 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (30m) 8m ago 39m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:01:48.974 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (27m) 8m ago 38m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:01:48.974 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (26m) 8m ago 38m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:01:48.974 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (26m) 8m ago 37m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:01:48.974 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (25m) 8m ago 37m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:01:48.974 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (24m) 8m ago 37m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:01:48.974 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (24m) 8m ago 36m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:01:48.974 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (30m) 8m ago 40m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:01:48.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.970+0000 7f9a76753700 1 -- 172.21.15.144:0/3962236885 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9a5806e200 msgr2=0x7f9a580706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:48.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.970+0000 7f9a76753700 1 --2- 172.21.15.144:0/3962236885 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9a5806e200 0x7f9a580706b0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f9a60005fd0 tx=0x7f9a60005ee0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.970+0000 7f9a76753700 1 -- 172.21.15.144:0/3962236885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70100d70 msgr2=0x7f9a70196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:48.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.970+0000 7f9a76753700 1 --2- 172.21.15.144:0/3962236885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70100d70 0x7f9a70196280 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f9a640092e0 tx=0x7f9a6400be20 comp rx=0 tx=0).stop 2024-09-10T12:01:48.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.971+0000 7f9a76753700 1 -- 172.21.15.144:0/3962236885 shutdown_connections 2024-09-10T12:01:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.971+0000 7f9a76753700 1 --2- 172.21.15.144:0/3962236885 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9a5806e200 0x7f9a580706b0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.971+0000 7f9a76753700 1 --2- 172.21.15.144:0/3962236885 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9a701003d0 0x7f9a70195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.971+0000 7f9a76753700 1 --2- 172.21.15.144:0/3962236885 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9a70100d70 0x7f9a70196280 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.971+0000 7f9a76753700 1 -- 172.21.15.144:0/3962236885 >> 172.21.15.144:0/3962236885 conn(0x7f9a700fbc90 msgr2=0x7f9a700fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.971+0000 7f9a76753700 1 -- 172.21.15.144:0/3962236885 shutdown_connections 2024-09-10T12:01:48.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:48.971+0000 7f9a76753700 1 -- 172.21.15.144:0/3962236885 wait complete. 2024-09-10T12:01:49.076 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.074+0000 7f477cffd700 1 -- 172.21.15.144:0/3388379934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47780ffd30 msgr2=0x7f4778103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:49.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.074+0000 7f477cffd700 1 --2- 172.21.15.144:0/3388379934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47780ffd30 0x7f4778103ce0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4768009910 tx=0x7f4768009c20 comp rx=0 tx=0).stop 2024-09-10T12:01:49.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.074+0000 7f477cffd700 1 -- 172.21.15.144:0/3388379934 shutdown_connections 2024-09-10T12:01:49.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.074+0000 7f477cffd700 1 --2- 172.21.15.144:0/3388379934 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47780ffd30 0x7f4778103ce0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:49.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.074+0000 7f477cffd700 1 --2- 172.21.15.144:0/3388379934 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47780ff390 0x7f47780ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:49.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.074+0000 7f477cffd700 1 -- 172.21.15.144:0/3388379934 >> 172.21.15.144:0/3388379934 conn(0x7f4778074f50 msgr2=0x7f4778075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:49.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.074+0000 7f477cffd700 1 -- 172.21.15.144:0/3388379934 shutdown_connections 2024-09-10T12:01:49.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.074+0000 7f477cffd700 1 -- 172.21.15.144:0/3388379934 wait complete. 2024-09-10T12:01:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.075+0000 7f477cffd700 1 Processor -- start 2024-09-10T12:01:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.075+0000 7f477cffd700 1 -- start start 2024-09-10T12:01:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.075+0000 7f477cffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47780ff390 0x7f4778196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.075+0000 7f477cffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47780ffd30 0x7f47781970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.075+0000 7f477cffd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47781976e0 con 0x7f47780ffd30 2024-09-10T12:01:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.075+0000 7f477cffd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4778197820 con 0x7f47780ff390 2024-09-10T12:01:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.075+0000 7f4775d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47780ffd30 0x7f47781970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.075+0000 7f4775d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47780ffd30 0x7f47781970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42150/0 (socket says 172.21.15.144:42150) 2024-09-10T12:01:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.075+0000 7f4775d9b700 1 -- 172.21.15.144:0/3183514491 learned_addr learned my addr 172.21.15.144:0/3183514491 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:01:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.075+0000 7f477659c700 1 --2- 172.21.15.144:0/3183514491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47780ff390 0x7f4778196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:49.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.076+0000 7f4775d9b700 1 -- 172.21.15.144:0/3183514491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47780ff390 msgr2=0x7f4778196b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:49.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.076+0000 7f4775d9b700 1 --2- 172.21.15.144:0/3183514491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47780ff390 0x7f4778196b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:49.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.076+0000 7f4775d9b700 1 -- 172.21.15.144:0/3183514491 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47680095f0 con 0x7f47780ffd30 2024-09-10T12:01:49.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.076+0000 7f4775d9b700 1 --2- 172.21.15.144:0/3183514491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47780ffd30 0x7f47781970c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f47680092e0 tx=0x7f476800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:49.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.076+0000 7f476f7fe700 1 -- 172.21.15.144:0/3183514491 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f476801d070 con 0x7f47780ffd30 2024-09-10T12:01:49.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.076+0000 7f476f7fe700 1 -- 172.21.15.144:0/3183514491 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4768005dc0 con 0x7f47780ffd30 2024-09-10T12:01:49.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.076+0000 7f477cffd700 1 -- 172.21.15.144:0/3183514491 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f477819fbf0 con 0x7f47780ffd30 2024-09-10T12:01:49.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.076+0000 7f476f7fe700 1 -- 172.21.15.144:0/3183514491 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4768017400 con 0x7f47780ffd30 2024-09-10T12:01:49.080 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.076+0000 7f477cffd700 1 -- 172.21.15.144:0/3183514491 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47781a0000 con 0x7f47780ffd30 2024-09-10T12:01:49.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.077+0000 7f477cffd700 1 -- 172.21.15.144:0/3183514491 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4778107600 con 0x7f47780ffd30 2024-09-10T12:01:49.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.078+0000 7f476f7fe700 1 -- 172.21.15.144:0/3183514491 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f476800f6f0 con 0x7f47780ffd30 2024-09-10T12:01:49.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.078+0000 7f476f7fe700 1 --2- 172.21.15.144:0/3183514491 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f476406e200 0x7f47640706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:49.081 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.078+0000 7f476f7fe700 1 -- 172.21.15.144:0/3183514491 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f476808f470 con 0x7f47780ffd30 2024-09-10T12:01:49.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.078+0000 7f477659c700 1 --2- 172.21.15.144:0/3183514491 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f476406e200 0x7f47640706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:49.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.079+0000 7f477659c700 1 --2- 172.21.15.144:0/3183514491 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f476406e200 0x7f47640706b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4760005950 tx=0x7f476000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:49.084 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.082+0000 7f476f7fe700 1 -- 172.21.15.144:0/3183514491 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4768058850 con 0x7f47780ffd30 2024-09-10T12:01:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:48 smithi178 ceph-mon[65940]: pgmap v895: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:49.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.364+0000 7f477cffd700 1 -- 172.21.15.144:0/3183514491 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f477804ea50 con 0x7f47780ffd30 2024-09-10T12:01:49.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.364+0000 7f476f7fe700 1 -- 172.21.15.144:0/3183514491 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f476805bf10 con 0x7f47780ffd30 2024-09-10T12:01:49.367 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:01:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:01:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:01:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:01:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:01:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:01:49.367 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:01:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:01:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:01:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:01:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:01:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:01:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:01:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:01:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:01:49.368 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:01:49.368 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:01:49.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.367+0000 7f477cffd700 1 -- 172.21.15.144:0/3183514491 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f476406e200 msgr2=0x7f47640706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:49.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.367+0000 7f477cffd700 1 --2- 172.21.15.144:0/3183514491 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f476406e200 0x7f47640706b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4760005950 tx=0x7f476000b410 comp rx=0 tx=0).stop 2024-09-10T12:01:49.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.367+0000 7f477cffd700 1 -- 172.21.15.144:0/3183514491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47780ffd30 msgr2=0x7f47781970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:49.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.367+0000 7f477cffd700 1 --2- 172.21.15.144:0/3183514491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47780ffd30 0x7f47781970c0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f47680092e0 tx=0x7f476800bd70 comp rx=0 tx=0).stop 2024-09-10T12:01:49.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.367+0000 7f477cffd700 1 -- 172.21.15.144:0/3183514491 shutdown_connections 2024-09-10T12:01:49.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.367+0000 7f477cffd700 1 --2- 172.21.15.144:0/3183514491 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f476406e200 0x7f47640706b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:49.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.367+0000 7f477cffd700 1 --2- 172.21.15.144:0/3183514491 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47780ff390 0x7f4778196b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:49.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.367+0000 7f477cffd700 1 --2- 172.21.15.144:0/3183514491 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47780ffd30 0x7f47781970c0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:49.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.367+0000 7f477cffd700 1 -- 172.21.15.144:0/3183514491 >> 172.21.15.144:0/3183514491 conn(0x7f4778074f50 msgr2=0x7f47780fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:49.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.367+0000 7f477cffd700 1 -- 172.21.15.144:0/3183514491 shutdown_connections 2024-09-10T12:01:49.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.367+0000 7f477cffd700 1 -- 172.21.15.144:0/3183514491 wait complete. 2024-09-10T12:01:49.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.471+0000 7f9ce3a62700 1 -- 172.21.15.144:0/784222629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cdc0ffd30 msgr2=0x7f9cdc103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:49.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.471+0000 7f9ce3a62700 1 --2- 172.21.15.144:0/784222629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cdc0ffd30 0x7f9cdc103ce0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009910 tx=0x7f9ccc009c20 comp rx=0 tx=0).stop 2024-09-10T12:01:49.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.471+0000 7f9ce3a62700 1 -- 172.21.15.144:0/784222629 shutdown_connections 2024-09-10T12:01:49.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.471+0000 7f9ce3a62700 1 --2- 172.21.15.144:0/784222629 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cdc0ffd30 0x7f9cdc103ce0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:49.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.471+0000 7f9ce3a62700 1 --2- 172.21.15.144:0/784222629 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cdc0ff390 0x7f9cdc0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:49.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.471+0000 7f9ce3a62700 1 -- 172.21.15.144:0/784222629 >> 172.21.15.144:0/784222629 conn(0x7f9cdc074f50 msgr2=0x7f9cdc075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:49.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.471+0000 7f9ce3a62700 1 -- 172.21.15.144:0/784222629 shutdown_connections 2024-09-10T12:01:49.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.471+0000 7f9ce3a62700 1 -- 172.21.15.144:0/784222629 wait complete. 2024-09-10T12:01:49.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.472+0000 7f9ce3a62700 1 Processor -- start 2024-09-10T12:01:49.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.472+0000 7f9ce3a62700 1 -- start start 2024-09-10T12:01:49.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.472+0000 7f9ce3a62700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cdc0ff390 0x7f9cdc194970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:49.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.472+0000 7f9ce3a62700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cdc0ffd30 0x7f9cdc194eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:49.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.472+0000 7f9ce3a62700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc1953f0 con 0x7f9cdc0ff390 2024-09-10T12:01:49.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.472+0000 7f9ce3a62700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cdc195560 con 0x7f9cdc0ffd30 2024-09-10T12:01:49.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.472+0000 7f9ce17fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cdc0ff390 0x7f9cdc194970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:49.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.472+0000 7f9ce17fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cdc0ff390 0x7f9cdc194970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42156/0 (socket says 172.21.15.144:42156) 2024-09-10T12:01:49.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.472+0000 7f9ce17fe700 1 -- 172.21.15.144:0/3703440948 learned_addr learned my addr 172.21.15.144:0/3703440948 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:01:49.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.473+0000 7f9ce0ffd700 1 --2- 172.21.15.144:0/3703440948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cdc0ffd30 0x7f9cdc194eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:49.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.473+0000 7f9ce17fe700 1 -- 172.21.15.144:0/3703440948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cdc0ffd30 msgr2=0x7f9cdc194eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:49.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.473+0000 7f9ce17fe700 1 --2- 172.21.15.144:0/3703440948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cdc0ffd30 0x7f9cdc194eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:49.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.473+0000 7f9ce17fe700 1 -- 172.21.15.144:0/3703440948 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc0095f0 con 0x7f9cdc0ff390 2024-09-10T12:01:49.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.473+0000 7f9ce17fe700 1 --2- 172.21.15.144:0/3703440948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cdc0ff390 0x7f9cdc194970 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800ada0 tx=0x7f9cd800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:49.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.473+0000 7f9cd27fc700 1 -- 172.21.15.144:0/3703440948 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cd800f070 con 0x7f9cdc0ff390 2024-09-10T12:01:49.478 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.473+0000 7f9ce3a62700 1 -- 172.21.15.144:0/3703440948 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cdc1a1fe0 con 0x7f9cdc0ff390 2024-09-10T12:01:49.478 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.473+0000 7f9cd27fc700 1 -- 172.21.15.144:0/3703440948 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9cd8007bf0 con 0x7f9cdc0ff390 2024-09-10T12:01:49.478 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.473+0000 7f9cd27fc700 1 -- 172.21.15.144:0/3703440948 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9cd8005690 con 0x7f9cdc0ff390 2024-09-10T12:01:49.478 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.473+0000 7f9ce3a62700 1 -- 172.21.15.144:0/3703440948 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cdc1a2340 con 0x7f9cdc0ff390 2024-09-10T12:01:49.478 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.474+0000 7f9ce3a62700 1 -- 172.21.15.144:0/3703440948 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cdc107600 con 0x7f9cdc0ff390 2024-09-10T12:01:49.479 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.474+0000 7f9cd27fc700 1 -- 172.21.15.144:0/3703440948 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9cd8005900 con 0x7f9cdc0ff390 2024-09-10T12:01:49.479 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.475+0000 7f9cd27fc700 1 --2- 172.21.15.144:0/3703440948 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9cc806e200 0x7f9cc80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:49.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.475+0000 7f9cd27fc700 1 -- 172.21.15.144:0/3703440948 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9cd808da20 con 0x7f9cdc0ff390 2024-09-10T12:01:49.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.475+0000 7f9ce0ffd700 1 --2- 172.21.15.144:0/3703440948 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9cc806e200 0x7f9cc80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:49.480 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.475+0000 7f9ce0ffd700 1 --2- 172.21.15.144:0/3703440948 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9cc806e200 0x7f9cc80706b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc068dc0 tx=0x7f9ccc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:49.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.479+0000 7f9cd27fc700 1 -- 172.21.15.144:0/3703440948 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9cd805a930 con 0x7f9cdc0ff390 2024-09-10T12:01:49.728 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.726+0000 7f9ce3a62700 1 -- 172.21.15.144:0/3703440948 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9cdc02cc60 con 0x7f9cdc0ff390 2024-09-10T12:01:49.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.726+0000 7f9cd27fc700 1 -- 172.21.15.144:0/3703440948 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f9cd805a4c0 con 0x7f9cdc0ff390 2024-09-10T12:01:49.729 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:01:49.729 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:01:49.729 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:01:49.729 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:01:49.729 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:01:49.729 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:01:49.730 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:01:49.730 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:01:49.730 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:01:49.730 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:01:49.730 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:01:49.730 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:01:49.730 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:01:49.730 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:01:49.730 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:01:49.730 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:01:49.730 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:01:49.730 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:01:49.731 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:01:49.731 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:01:49.731 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:01:49.731 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:01:49.731 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:01:49.731 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:01:49.731 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:01:49.731 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:01:49.731 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:01:49.732 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:01:49.732 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:01:49.732 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:01:49.732 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:01:49.732 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:01:49.733 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:01:49.733 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:01:49.733 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:01:49.733 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:01:49.733 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:01:49.733 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:01:49.734 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:01:49.734 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:01:49.734 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:01:49.734 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:01:49.734 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:01:49.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.728+0000 7f9ce3a62700 1 -- 172.21.15.144:0/3703440948 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9cc806e200 msgr2=0x7f9cc80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:49.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.729+0000 7f9ce3a62700 1 --2- 172.21.15.144:0/3703440948 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9cc806e200 0x7f9cc80706b0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc068dc0 tx=0x7f9ccc01a040 comp rx=0 tx=0).stop 2024-09-10T12:01:49.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.729+0000 7f9ce3a62700 1 -- 172.21.15.144:0/3703440948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cdc0ff390 msgr2=0x7f9cdc194970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:49.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.729+0000 7f9ce3a62700 1 --2- 172.21.15.144:0/3703440948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cdc0ff390 0x7f9cdc194970 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f9cd800ada0 tx=0x7f9cd800c330 comp rx=0 tx=0).stop 2024-09-10T12:01:49.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.729+0000 7f9ce3a62700 1 -- 172.21.15.144:0/3703440948 shutdown_connections 2024-09-10T12:01:49.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.729+0000 7f9ce3a62700 1 --2- 172.21.15.144:0/3703440948 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9cc806e200 0x7f9cc80706b0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:49.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.729+0000 7f9ce3a62700 1 --2- 172.21.15.144:0/3703440948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9cdc0ff390 0x7f9cdc194970 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:49.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.729+0000 7f9ce3a62700 1 --2- 172.21.15.144:0/3703440948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9cdc0ffd30 0x7f9cdc194eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:49.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.729+0000 7f9ce3a62700 1 -- 172.21.15.144:0/3703440948 >> 172.21.15.144:0/3703440948 conn(0x7f9cdc074f50 msgr2=0x7f9cdc0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:49.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.729+0000 7f9ce3a62700 1 -- 172.21.15.144:0/3703440948 shutdown_connections 2024-09-10T12:01:49.737 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.729+0000 7f9ce3a62700 1 -- 172.21.15.144:0/3703440948 wait complete. 2024-09-10T12:01:49.737 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:01:49.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.836+0000 7fb2ceefd700 1 -- 172.21.15.144:0/2818701573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c80ffd30 msgr2=0x7fb2c8103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:49.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.836+0000 7fb2ceefd700 1 --2- 172.21.15.144:0/2818701573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c80ffd30 0x7fb2c8103ce0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009910 tx=0x7fb2b4009c20 comp rx=0 tx=0).stop 2024-09-10T12:01:49.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.836+0000 7fb2ceefd700 1 -- 172.21.15.144:0/2818701573 shutdown_connections 2024-09-10T12:01:49.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.836+0000 7fb2ceefd700 1 --2- 172.21.15.144:0/2818701573 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c80ffd30 0x7fb2c8103ce0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:49.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.836+0000 7fb2ceefd700 1 --2- 172.21.15.144:0/2818701573 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2c80ff390 0x7fb2c80ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:49.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.836+0000 7fb2ceefd700 1 -- 172.21.15.144:0/2818701573 >> 172.21.15.144:0/2818701573 conn(0x7fb2c8074f50 msgr2=0x7fb2c8075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:49.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.836+0000 7fb2ceefd700 1 -- 172.21.15.144:0/2818701573 shutdown_connections 2024-09-10T12:01:49.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.836+0000 7fb2ceefd700 1 -- 172.21.15.144:0/2818701573 wait complete. 2024-09-10T12:01:49.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.837+0000 7fb2ceefd700 1 Processor -- start 2024-09-10T12:01:49.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.837+0000 7fb2ceefd700 1 -- start start 2024-09-10T12:01:49.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.837+0000 7fb2ceefd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c80ff390 0x7fb2c8196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:49.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.837+0000 7fb2ceefd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2c80ffd30 0x7fb2c81970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:49.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.837+0000 7fb2ceefd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c81976e0 con 0x7fb2c80ff390 2024-09-10T12:01:49.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.837+0000 7fb2ceefd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2c8197820 con 0x7fb2c80ffd30 2024-09-10T12:01:49.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.837+0000 7fb2ccc99700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c80ff390 0x7fb2c8196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:49.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.837+0000 7fb2ccc99700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c80ff390 0x7fb2c8196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42162/0 (socket says 172.21.15.144:42162) 2024-09-10T12:01:49.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.837+0000 7fb2ccc99700 1 -- 172.21.15.144:0/3612623167 learned_addr learned my addr 172.21.15.144:0/3612623167 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:01:49.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.837+0000 7fb2c7fff700 1 --2- 172.21.15.144:0/3612623167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2c80ffd30 0x7fb2c81970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:49.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.837+0000 7fb2ccc99700 1 -- 172.21.15.144:0/3612623167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2c80ffd30 msgr2=0x7fb2c81970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:49.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.837+0000 7fb2ccc99700 1 --2- 172.21.15.144:0/3612623167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2c80ffd30 0x7fb2c81970c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:49.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.837+0000 7fb2ccc99700 1 -- 172.21.15.144:0/3612623167 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b40095f0 con 0x7fb2c80ff390 2024-09-10T12:01:49.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.837+0000 7fb2ccc99700 1 --2- 172.21.15.144:0/3612623167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c80ff390 0x7fb2c8196b80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00d8d0 tx=0x7fb2bc00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:49.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.838+0000 7fb2c5ffb700 1 -- 172.21.15.144:0/3612623167 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2bc0098a0 con 0x7fb2c80ff390 2024-09-10T12:01:49.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.838+0000 7fb2c5ffb700 1 -- 172.21.15.144:0/3612623167 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb2bc010460 con 0x7fb2c80ff390 2024-09-10T12:01:49.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.838+0000 7fb2c5ffb700 1 -- 172.21.15.144:0/3612623167 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2bc00f5f0 con 0x7fb2c80ff390 2024-09-10T12:01:49.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.838+0000 7fb2ceefd700 1 -- 172.21.15.144:0/3612623167 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c819fbf0 con 0x7fb2c80ff390 2024-09-10T12:01:49.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.838+0000 7fb2ceefd700 1 -- 172.21.15.144:0/3612623167 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2c81a00b0 con 0x7fb2c80ff390 2024-09-10T12:01:49.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.839+0000 7fb2ceefd700 1 -- 172.21.15.144:0/3612623167 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2c8107600 con 0x7fb2c80ff390 2024-09-10T12:01:49.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.839+0000 7fb2c5ffb700 1 -- 172.21.15.144:0/3612623167 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb2bc009a00 con 0x7fb2c80ff390 2024-09-10T12:01:49.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.840+0000 7fb2c5ffb700 1 --2- 172.21.15.144:0/3612623167 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb2b806e200 0x7fb2b80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:49.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.840+0000 7fb2c5ffb700 1 -- 172.21.15.144:0/3612623167 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb2bc08de50 con 0x7fb2c80ff390 2024-09-10T12:01:49.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.840+0000 7fb2c7fff700 1 --2- 172.21.15.144:0/3612623167 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb2b806e200 0x7fb2b80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:49.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.840+0000 7fb2c7fff700 1 --2- 172.21.15.144:0/3612623167 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb2b806e200 0x7fb2b80706b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009fd0 tx=0x7fb2b401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:49.846 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:49.844+0000 7fb2c5ffb700 1 -- 172.21.15.144:0/3612623167 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb2bc0572b0 con 0x7fb2c80ff390 2024-09-10T12:01:50.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.063+0000 7fb2ceefd700 1 -- 172.21.15.144:0/3612623167 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2c802cd40 con 0x7fb2b806e200 2024-09-10T12:01:50.066 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:49 smithi144 ceph-mon[87436]: from='client.35632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:50.066 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:49 smithi144 ceph-mon[87436]: from='client.35636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:50.066 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:49 smithi144 ceph-mon[87436]: from='client.35640 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:50.066 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:49 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3183514491' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:01:50.066 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:49 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3703440948' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:01:50.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.065+0000 7fb2c5ffb700 1 -- 172.21.15.144:0/3612623167 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb2c802cd40 con 0x7fb2b806e200 2024-09-10T12:01:50.067 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:01:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:01:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:01:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:01:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:01:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:01:50.067 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:01:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:01:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:01:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:01:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:01:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:01:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:01:50.068 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:01:50.068 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:01:50.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.067+0000 7fb2ceefd700 1 -- 172.21.15.144:0/3612623167 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb2b806e200 msgr2=0x7fb2b80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:50.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.067+0000 7fb2ceefd700 1 --2- 172.21.15.144:0/3612623167 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb2b806e200 0x7fb2b80706b0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fb2b4009fd0 tx=0x7fb2b401a040 comp rx=0 tx=0).stop 2024-09-10T12:01:50.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.067+0000 7fb2ceefd700 1 -- 172.21.15.144:0/3612623167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c80ff390 msgr2=0x7fb2c8196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:50.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.067+0000 7fb2ceefd700 1 --2- 172.21.15.144:0/3612623167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c80ff390 0x7fb2c8196b80 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc00d8d0 tx=0x7fb2bc00dbe0 comp rx=0 tx=0).stop 2024-09-10T12:01:50.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.067+0000 7fb2ceefd700 1 -- 172.21.15.144:0/3612623167 shutdown_connections 2024-09-10T12:01:50.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.067+0000 7fb2ceefd700 1 --2- 172.21.15.144:0/3612623167 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb2b806e200 0x7fb2b80706b0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:50.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.067+0000 7fb2ceefd700 1 --2- 172.21.15.144:0/3612623167 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb2c80ff390 0x7fb2c8196b80 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:50.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.067+0000 7fb2ceefd700 1 --2- 172.21.15.144:0/3612623167 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb2c80ffd30 0x7fb2c81970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:50.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.067+0000 7fb2ceefd700 1 -- 172.21.15.144:0/3612623167 >> 172.21.15.144:0/3612623167 conn(0x7fb2c8074f50 msgr2=0x7fb2c80fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:50.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.068+0000 7fb2ceefd700 1 -- 172.21.15.144:0/3612623167 shutdown_connections 2024-09-10T12:01:50.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.068+0000 7fb2ceefd700 1 -- 172.21.15.144:0/3612623167 wait complete. 2024-09-10T12:01:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:49 smithi178 ceph-mon[65940]: from='client.35632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:49 smithi178 ceph-mon[65940]: from='client.35636 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:49 smithi178 ceph-mon[65940]: from='client.35640 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:49 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3183514491' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:01:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:49 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3703440948' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:01:50.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.173+0000 7f19fecfc700 1 -- 172.21.15.144:0/324642225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f80ff2e0 msgr2=0x7f19f80ff6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:50.175 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.173+0000 7f19fecfc700 1 --2- 172.21.15.144:0/324642225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f80ff2e0 0x7f19f80ff6b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009910 tx=0x7f19ec009c20 comp rx=0 tx=0).stop 2024-09-10T12:01:50.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.173+0000 7f19fecfc700 1 -- 172.21.15.144:0/324642225 shutdown_connections 2024-09-10T12:01:50.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.173+0000 7f19fecfc700 1 --2- 172.21.15.144:0/324642225 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19f80ffc80 0x7f19f8103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:50.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.173+0000 7f19fecfc700 1 --2- 172.21.15.144:0/324642225 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f80ff2e0 0x7f19f80ff6b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:50.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.173+0000 7f19fecfc700 1 -- 172.21.15.144:0/324642225 >> 172.21.15.144:0/324642225 conn(0x7f19f80fab80 msgr2=0x7f19f80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:50.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.173+0000 7f19fecfc700 1 -- 172.21.15.144:0/324642225 shutdown_connections 2024-09-10T12:01:50.176 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.173+0000 7f19fecfc700 1 -- 172.21.15.144:0/324642225 wait complete. 2024-09-10T12:01:50.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.174+0000 7f19fecfc700 1 Processor -- start 2024-09-10T12:01:50.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.174+0000 7f19fecfc700 1 -- start start 2024-09-10T12:01:50.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.174+0000 7f19fecfc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19f80ff2e0 0x7f19f8196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:50.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.174+0000 7f19fecfc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f80ffc80 0x7f19f8197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:50.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.174+0000 7f19fecfc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f8197810 con 0x7f19f80ffc80 2024-09-10T12:01:50.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.174+0000 7f19fecfc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f8197950 con 0x7f19f80ff2e0 2024-09-10T12:01:50.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.174+0000 7f19f7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f80ffc80 0x7f19f8197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:50.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.174+0000 7f19f7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f80ffc80 0x7f19f8197130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:42184/0 (socket says 172.21.15.144:42184) 2024-09-10T12:01:50.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.174+0000 7f19f7fff700 1 -- 172.21.15.144:0/946531259 learned_addr learned my addr 172.21.15.144:0/946531259 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:01:50.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.174+0000 7f19fca98700 1 --2- 172.21.15.144:0/946531259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19f80ff2e0 0x7f19f8196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:50.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.174+0000 7f19f7fff700 1 -- 172.21.15.144:0/946531259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19f80ff2e0 msgr2=0x7f19f8196bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:50.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.174+0000 7f19f7fff700 1 --2- 172.21.15.144:0/946531259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19f80ff2e0 0x7f19f8196bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:50.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.174+0000 7f19f7fff700 1 -- 172.21.15.144:0/946531259 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19ec0095f0 con 0x7f19f80ffc80 2024-09-10T12:01:50.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.174+0000 7f19fca98700 1 --2- 172.21.15.144:0/946531259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19f80ff2e0 0x7f19f8196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:01:50.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.175+0000 7f19f7fff700 1 --2- 172.21.15.144:0/946531259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f80ffc80 0x7f19f8197130 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f19e400d8d0 tx=0x7f19e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:50.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.175+0000 7f19f5ffb700 1 -- 172.21.15.144:0/946531259 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19e40041f0 con 0x7f19f80ffc80 2024-09-10T12:01:50.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.175+0000 7f19fecfc700 1 -- 172.21.15.144:0/946531259 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f819fc60 con 0x7f19f80ffc80 2024-09-10T12:01:50.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.175+0000 7f19f5ffb700 1 -- 172.21.15.144:0/946531259 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f19e4007620 con 0x7f19f80ffc80 2024-09-10T12:01:50.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.175+0000 7f19f5ffb700 1 -- 172.21.15.144:0/946531259 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19e4003c50 con 0x7f19f80ffc80 2024-09-10T12:01:50.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.175+0000 7f19fecfc700 1 -- 172.21.15.144:0/946531259 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f81a0180 con 0x7f19f80ffc80 2024-09-10T12:01:50.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.176+0000 7f19fecfc700 1 -- 172.21.15.144:0/946531259 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f81075f0 con 0x7f19f80ffc80 2024-09-10T12:01:50.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.177+0000 7f19f5ffb700 1 -- 172.21.15.144:0/946531259 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f19e4004350 con 0x7f19f80ffc80 2024-09-10T12:01:50.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.177+0000 7f19f5ffb700 1 --2- 172.21.15.144:0/946531259 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19e806e200 0x7f19e80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:01:50.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.177+0000 7f19f5ffb700 1 -- 172.21.15.144:0/946531259 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f19e408dc30 con 0x7f19f80ffc80 2024-09-10T12:01:50.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.177+0000 7f19fca98700 1 --2- 172.21.15.144:0/946531259 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19e806e200 0x7f19e80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:01:50.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.177+0000 7f19fca98700 1 --2- 172.21.15.144:0/946531259 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19e806e200 0x7f19e80706b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00b5c0 tx=0x7f19ec005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:01:50.184 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.181+0000 7f19f5ffb700 1 -- 172.21.15.144:0/946531259 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f19e4057010 con 0x7f19f80ffc80 2024-09-10T12:01:50.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.468+0000 7f19fecfc700 1 -- 172.21.15.144:0/946531259 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f19f804ea50 con 0x7f19f80ffc80 2024-09-10T12:01:50.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.468+0000 7f19f5ffb700 1 -- 172.21.15.144:0/946531259 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f19e405a6d0 con 0x7f19f80ffc80 2024-09-10T12:01:50.471 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:01:50.471 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:01:50.471 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:01:50.471 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:01:50.471 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:01:50.471 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:01:50.471 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:01:50.472 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:01:50.472 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:01:50.472 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:01:50.472 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:01:50.472 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:01:50.472 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:01:50.472 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:01:50.472 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:01:50.472 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:01:50.472 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:01:50.472 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:01:50.472 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:01:50.472 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:01:50.473 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:01:50.473 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:01:50.473 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:01:50.473 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:01:50.473 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:01:50.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.471+0000 7f19fecfc700 1 -- 172.21.15.144:0/946531259 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19e806e200 msgr2=0x7f19e80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:50.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.471+0000 7f19fecfc700 1 --2- 172.21.15.144:0/946531259 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19e806e200 0x7f19e80706b0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f19ec00b5c0 tx=0x7f19ec005960 comp rx=0 tx=0).stop 2024-09-10T12:01:50.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.471+0000 7f19fecfc700 1 -- 172.21.15.144:0/946531259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f80ffc80 msgr2=0x7f19f8197130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:01:50.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.471+0000 7f19fecfc700 1 --2- 172.21.15.144:0/946531259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f80ffc80 0x7f19f8197130 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f19e400d8d0 tx=0x7f19e400dc90 comp rx=0 tx=0).stop 2024-09-10T12:01:50.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.471+0000 7f19fecfc700 1 -- 172.21.15.144:0/946531259 shutdown_connections 2024-09-10T12:01:50.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.471+0000 7f19fecfc700 1 --2- 172.21.15.144:0/946531259 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f19e806e200 0x7f19e80706b0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:50.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.471+0000 7f19fecfc700 1 --2- 172.21.15.144:0/946531259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f19f80ff2e0 0x7f19f8196bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:50.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.471+0000 7f19fecfc700 1 --2- 172.21.15.144:0/946531259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f19f80ffc80 0x7f19f8197130 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:01:50.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.471+0000 7f19fecfc700 1 -- 172.21.15.144:0/946531259 >> 172.21.15.144:0/946531259 conn(0x7f19f80fab80 msgr2=0x7f19f80fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:01:50.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.471+0000 7f19fecfc700 1 -- 172.21.15.144:0/946531259 shutdown_connections 2024-09-10T12:01:50.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:01:50.471+0000 7f19fecfc700 1 -- 172.21.15.144:0/946531259 wait complete. 2024-09-10T12:01:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:50 smithi178 ceph-mon[65940]: pgmap v896: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:50 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/946531259' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:01:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:50 smithi144 ceph-mon[87436]: pgmap v896: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:50 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/946531259' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:01:52.029 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:51 smithi178 ceph-mon[65940]: from='client.35652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:52.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:51 smithi144 ceph-mon[87436]: from='client.35652 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:01:53.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:52 smithi144 ceph-mon[87436]: pgmap v897: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:52 smithi178 ceph-mon[65940]: pgmap v897: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:54 smithi178 ceph-mon[65940]: pgmap v898: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:54 smithi144 ceph-mon[87436]: pgmap v898: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:57.080 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:56 smithi178 ceph-mon[65940]: pgmap v899: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:56 smithi144 ceph-mon[87436]: pgmap v899: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:58 smithi144 ceph-mon[87436]: pgmap v900: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:01:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:01:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:58 smithi178 ceph-mon[65940]: pgmap v900: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:01:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:01:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:02:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:00 smithi178 ceph-mon[65940]: pgmap v901: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:00 smithi144 ceph-mon[87436]: pgmap v901: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:03.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:02 smithi144 ceph-mon[87436]: pgmap v902: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:02 smithi178 ceph-mon[65940]: pgmap v902: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:04 smithi178 ceph-mon[65940]: pgmap v903: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:05.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:04 smithi144 ceph-mon[87436]: pgmap v903: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:06 smithi178 ceph-mon[65940]: pgmap v904: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:06 smithi144 ceph-mon[87436]: pgmap v904: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:08 smithi178 ceph-mon[65940]: pgmap v905: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:08 smithi144 ceph-mon[87436]: pgmap v905: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:10 smithi178 ceph-mon[65940]: pgmap v906: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:11.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:10 smithi144 ceph-mon[87436]: pgmap v906: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:12 smithi144 ceph-mon[87436]: pgmap v907: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:12 smithi178 ceph-mon[65940]: pgmap v907: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:02:14.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:02:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:14 smithi178 ceph-mon[65940]: pgmap v908: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:14 smithi144 ceph-mon[87436]: pgmap v908: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:16 smithi178 ceph-mon[65940]: pgmap v909: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:17.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:16 smithi144 ceph-mon[87436]: pgmap v909: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:18 smithi178 ceph-mon[65940]: pgmap v910: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:18 smithi144 ceph-mon[87436]: pgmap v910: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:20.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.591+0000 7f4112b64700 1 -- 172.21.15.144:0/2650743857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f410c1003e0 msgr2=0x7f410c1007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:20.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.591+0000 7f4112b64700 1 --2- 172.21.15.144:0/2650743857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f410c1003e0 0x7f410c1007b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f4100009910 tx=0x7f4100009c20 comp rx=0 tx=0).stop 2024-09-10T12:02:20.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.591+0000 7f4112b64700 1 -- 172.21.15.144:0/2650743857 shutdown_connections 2024-09-10T12:02:20.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.591+0000 7f4112b64700 1 --2- 172.21.15.144:0/2650743857 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f410c100d80 0x7f410c104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:20.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.591+0000 7f4112b64700 1 --2- 172.21.15.144:0/2650743857 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f410c1003e0 0x7f410c1007b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:20.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.591+0000 7f4112b64700 1 -- 172.21.15.144:0/2650743857 >> 172.21.15.144:0/2650743857 conn(0x7f410c0fbca0 msgr2=0x7f410c0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:20.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.592+0000 7f4112b64700 1 -- 172.21.15.144:0/2650743857 shutdown_connections 2024-09-10T12:02:20.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.592+0000 7f4112b64700 1 -- 172.21.15.144:0/2650743857 wait complete. 2024-09-10T12:02:20.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.592+0000 7f4112b64700 1 Processor -- start 2024-09-10T12:02:20.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.592+0000 7f4112b64700 1 -- start start 2024-09-10T12:02:20.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.592+0000 7f4112b64700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f410c1003e0 0x7f410c195d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:20.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.592+0000 7f4112b64700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f410c100d80 0x7f410c196260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:20.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.592+0000 7f4112b64700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c196940 con 0x7f410c1003e0 2024-09-10T12:02:20.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.592+0000 7f4112b64700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410c19a6d0 con 0x7f410c100d80 2024-09-10T12:02:20.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.593+0000 7f4110900700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f410c1003e0 0x7f410c195d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:20.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.593+0000 7f4110900700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f410c1003e0 0x7f410c195d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45168/0 (socket says 172.21.15.144:45168) 2024-09-10T12:02:20.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.593+0000 7f4110900700 1 -- 172.21.15.144:0/3843264679 learned_addr learned my addr 172.21.15.144:0/3843264679 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:02:20.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.593+0000 7f410bfff700 1 --2- 172.21.15.144:0/3843264679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f410c100d80 0x7f410c196260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:20.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.593+0000 7f4110900700 1 -- 172.21.15.144:0/3843264679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f410c100d80 msgr2=0x7f410c196260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:20.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.593+0000 7f4110900700 1 --2- 172.21.15.144:0/3843264679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f410c100d80 0x7f410c196260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:20.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.593+0000 7f4110900700 1 -- 172.21.15.144:0/3843264679 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41000095f0 con 0x7f410c1003e0 2024-09-10T12:02:20.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.593+0000 7f410bfff700 1 --2- 172.21.15.144:0/3843264679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f410c100d80 0x7f410c196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:02:20.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.593+0000 7f4110900700 1 --2- 172.21.15.144:0/3843264679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f410c1003e0 0x7f410c195d20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f4100006010 tx=0x7f410000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:20.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.593+0000 7f4109ffb700 1 -- 172.21.15.144:0/3843264679 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f410001d070 con 0x7f410c1003e0 2024-09-10T12:02:20.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.593+0000 7f4109ffb700 1 -- 172.21.15.144:0/3843264679 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4100004fb0 con 0x7f410c1003e0 2024-09-10T12:02:20.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.593+0000 7f4112b64700 1 -- 172.21.15.144:0/3843264679 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f410c19a950 con 0x7f410c1003e0 2024-09-10T12:02:20.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.593+0000 7f4109ffb700 1 -- 172.21.15.144:0/3843264679 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4100022ea0 con 0x7f410c1003e0 2024-09-10T12:02:20.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.594+0000 7f4112b64700 1 -- 172.21.15.144:0/3843264679 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f410c19ae40 con 0x7f410c1003e0 2024-09-10T12:02:20.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.595+0000 7f4112b64700 1 -- 172.21.15.144:0/3843264679 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f410c1086f0 con 0x7f410c1003e0 2024-09-10T12:02:20.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.595+0000 7f4109ffb700 1 -- 172.21.15.144:0/3843264679 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4100022470 con 0x7f410c1003e0 2024-09-10T12:02:20.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.596+0000 7f4109ffb700 1 --2- 172.21.15.144:0/3843264679 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f40fc06e200 0x7f40fc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:20.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.596+0000 7f4109ffb700 1 -- 172.21.15.144:0/3843264679 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f410008f510 con 0x7f410c1003e0 2024-09-10T12:02:20.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.596+0000 7f410bfff700 1 --2- 172.21.15.144:0/3843264679 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f40fc06e200 0x7f40fc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:20.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.596+0000 7f410bfff700 1 --2- 172.21.15.144:0/3843264679 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f40fc06e200 0x7f40fc0706b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f410c197340 tx=0x7f40f8005f50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:20.601 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.600+0000 7f4109ffb700 1 -- 172.21.15.144:0/3843264679 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4100058b40 con 0x7f410c1003e0 2024-09-10T12:02:20.822 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.821+0000 7f4112b64700 1 -- 172.21.15.144:0/3843264679 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f410c19b170 con 0x7f40fc06e200 2024-09-10T12:02:20.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.822+0000 7f4109ffb700 1 -- 172.21.15.144:0/3843264679 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f410c19b170 con 0x7f40fc06e200 2024-09-10T12:02:20.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.825+0000 7f4112b64700 1 -- 172.21.15.144:0/3843264679 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f40fc06e200 msgr2=0x7f40fc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:20.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.825+0000 7f4112b64700 1 --2- 172.21.15.144:0/3843264679 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f40fc06e200 0x7f40fc0706b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f410c197340 tx=0x7f40f8005f50 comp rx=0 tx=0).stop 2024-09-10T12:02:20.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.825+0000 7f4112b64700 1 -- 172.21.15.144:0/3843264679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f410c1003e0 msgr2=0x7f410c195d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:20.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.825+0000 7f4112b64700 1 --2- 172.21.15.144:0/3843264679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f410c1003e0 0x7f410c195d20 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f4100006010 tx=0x7f410000be20 comp rx=0 tx=0).stop 2024-09-10T12:02:20.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.825+0000 7f4112b64700 1 -- 172.21.15.144:0/3843264679 shutdown_connections 2024-09-10T12:02:20.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.825+0000 7f4112b64700 1 --2- 172.21.15.144:0/3843264679 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f40fc06e200 0x7f40fc0706b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:20.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.825+0000 7f4112b64700 1 --2- 172.21.15.144:0/3843264679 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f410c1003e0 0x7f410c195d20 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:20.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.825+0000 7f4112b64700 1 --2- 172.21.15.144:0/3843264679 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f410c100d80 0x7f410c196260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:20.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.825+0000 7f4112b64700 1 -- 172.21.15.144:0/3843264679 >> 172.21.15.144:0/3843264679 conn(0x7f410c0fbca0 msgr2=0x7f410c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:20.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.826+0000 7f4112b64700 1 -- 172.21.15.144:0/3843264679 shutdown_connections 2024-09-10T12:02:20.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.826+0000 7f4112b64700 1 -- 172.21.15.144:0/3843264679 wait complete. 2024-09-10T12:02:20.837 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:02:20.932 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.931+0000 7ff92a564700 1 -- 172.21.15.144:0/47921372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9240ffd30 msgr2=0x7ff924103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:20.932 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.931+0000 7ff92a564700 1 --2- 172.21.15.144:0/47921372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9240ffd30 0x7ff924103ce0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7ff914009910 tx=0x7ff914009c20 comp rx=0 tx=0).stop 2024-09-10T12:02:20.933 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.931+0000 7ff92a564700 1 -- 172.21.15.144:0/47921372 shutdown_connections 2024-09-10T12:02:20.933 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.931+0000 7ff92a564700 1 --2- 172.21.15.144:0/47921372 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9240ffd30 0x7ff924103ce0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:20.933 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.931+0000 7ff92a564700 1 --2- 172.21.15.144:0/47921372 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9240ff390 0x7ff9240ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:20.933 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.931+0000 7ff92a564700 1 -- 172.21.15.144:0/47921372 >> 172.21.15.144:0/47921372 conn(0x7ff924074f50 msgr2=0x7ff924075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:20.933 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:20 smithi144 ceph-mon[87436]: pgmap v911: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:20.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.931+0000 7ff92a564700 1 -- 172.21.15.144:0/47921372 shutdown_connections 2024-09-10T12:02:20.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.931+0000 7ff92a564700 1 -- 172.21.15.144:0/47921372 wait complete. 2024-09-10T12:02:20.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.932+0000 7ff92a564700 1 Processor -- start 2024-09-10T12:02:20.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.932+0000 7ff92a564700 1 -- start start 2024-09-10T12:02:20.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.932+0000 7ff92a564700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9240ff390 0x7ff92419a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:20.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.932+0000 7ff92a564700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9240ffd30 0x7ff92419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:20.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.932+0000 7ff92a564700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92419b5b0 con 0x7ff9240ffd30 2024-09-10T12:02:20.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.932+0000 7ff92a564700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff924194a60 con 0x7ff9240ff390 2024-09-10T12:02:20.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.932+0000 7ff9237fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9240ffd30 0x7ff92419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:20.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.932+0000 7ff9237fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9240ffd30 0x7ff92419af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45182/0 (socket says 172.21.15.144:45182) 2024-09-10T12:02:20.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.932+0000 7ff9237fe700 1 -- 172.21.15.144:0/1948237670 learned_addr learned my addr 172.21.15.144:0/1948237670 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:02:20.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.933+0000 7ff923fff700 1 --2- 172.21.15.144:0/1948237670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9240ff390 0x7ff92419a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:20.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.933+0000 7ff9237fe700 1 -- 172.21.15.144:0/1948237670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9240ff390 msgr2=0x7ff92419a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:20.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.933+0000 7ff9237fe700 1 --2- 172.21.15.144:0/1948237670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9240ff390 0x7ff92419a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:20.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.933+0000 7ff9237fe700 1 -- 172.21.15.144:0/1948237670 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9140095f0 con 0x7ff9240ffd30 2024-09-10T12:02:20.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.933+0000 7ff9237fe700 1 --2- 172.21.15.144:0/1948237670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9240ffd30 0x7ff92419af20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff9140092e0 tx=0x7ff91400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:20.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.933+0000 7ff9217fa700 1 -- 172.21.15.144:0/1948237670 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff91401f070 con 0x7ff9240ffd30 2024-09-10T12:02:20.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.933+0000 7ff9217fa700 1 -- 172.21.15.144:0/1948237670 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff914019610 con 0x7ff9240ffd30 2024-09-10T12:02:20.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.933+0000 7ff92a564700 1 -- 172.21.15.144:0/1948237670 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff924194ce0 con 0x7ff9240ffd30 2024-09-10T12:02:20.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.933+0000 7ff9217fa700 1 -- 172.21.15.144:0/1948237670 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff91400f7a0 con 0x7ff9240ffd30 2024-09-10T12:02:20.936 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.933+0000 7ff92a564700 1 -- 172.21.15.144:0/1948237670 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9241951d0 con 0x7ff9240ffd30 2024-09-10T12:02:20.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.934+0000 7ff92a564700 1 -- 172.21.15.144:0/1948237670 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92404ea50 con 0x7ff9240ffd30 2024-09-10T12:02:20.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.935+0000 7ff9217fa700 1 -- 172.21.15.144:0/1948237670 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff914019780 con 0x7ff9240ffd30 2024-09-10T12:02:20.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.935+0000 7ff9217fa700 1 --2- 172.21.15.144:0/1948237670 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff91006e200 0x7ff9100706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:20.937 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.935+0000 7ff9217fa700 1 -- 172.21.15.144:0/1948237670 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff914091870 con 0x7ff9240ffd30 2024-09-10T12:02:20.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.935+0000 7ff923fff700 1 --2- 172.21.15.144:0/1948237670 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff91006e200 0x7ff9100706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:20.938 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.936+0000 7ff923fff700 1 --2- 172.21.15.144:0/1948237670 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff91006e200 0x7ff9100706b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005fd0 tx=0x7ff90c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:20.940 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:20.939+0000 7ff9217fa700 1 -- 172.21.15.144:0/1948237670 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff91405e800 con 0x7ff9240ffd30 2024-09-10T12:02:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:20 smithi178 ceph-mon[65940]: pgmap v911: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:21.166 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.165+0000 7ff92a564700 1 -- 172.21.15.144:0/1948237670 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff924195600 con 0x7ff91006e200 2024-09-10T12:02:21.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.166+0000 7ff9217fa700 1 -- 172.21.15.144:0/1948237670 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff924195600 con 0x7ff91006e200 2024-09-10T12:02:21.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.168+0000 7ff92a564700 1 -- 172.21.15.144:0/1948237670 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff91006e200 msgr2=0x7ff9100706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:21.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.168+0000 7ff92a564700 1 --2- 172.21.15.144:0/1948237670 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff91006e200 0x7ff9100706b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff90c005fd0 tx=0x7ff90c00a560 comp rx=0 tx=0).stop 2024-09-10T12:02:21.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.168+0000 7ff92a564700 1 -- 172.21.15.144:0/1948237670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9240ffd30 msgr2=0x7ff92419af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:21.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.168+0000 7ff92a564700 1 --2- 172.21.15.144:0/1948237670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9240ffd30 0x7ff92419af20 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7ff9140092e0 tx=0x7ff91400be20 comp rx=0 tx=0).stop 2024-09-10T12:02:21.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.169+0000 7ff92a564700 1 -- 172.21.15.144:0/1948237670 shutdown_connections 2024-09-10T12:02:21.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.169+0000 7ff92a564700 1 --2- 172.21.15.144:0/1948237670 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff91006e200 0x7ff9100706b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.169+0000 7ff92a564700 1 --2- 172.21.15.144:0/1948237670 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff9240ff390 0x7ff92419a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.169+0000 7ff92a564700 1 --2- 172.21.15.144:0/1948237670 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff9240ffd30 0x7ff92419af20 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.169+0000 7ff92a564700 1 -- 172.21.15.144:0/1948237670 >> 172.21.15.144:0/1948237670 conn(0x7ff924074f50 msgr2=0x7ff9240fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:21.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.169+0000 7ff92a564700 1 -- 172.21.15.144:0/1948237670 shutdown_connections 2024-09-10T12:02:21.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.169+0000 7ff92a564700 1 -- 172.21.15.144:0/1948237670 wait complete. 2024-09-10T12:02:21.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.273+0000 7f167d632700 1 -- 172.21.15.144:0/1363012481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1678100410 msgr2=0x7f16781007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:21.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.273+0000 7f167d632700 1 --2- 172.21.15.144:0/1363012481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1678100410 0x7f16781007e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f1668009910 tx=0x7f1668009c20 comp rx=0 tx=0).stop 2024-09-10T12:02:21.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.273+0000 7f167d632700 1 -- 172.21.15.144:0/1363012481 shutdown_connections 2024-09-10T12:02:21.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.273+0000 7f167d632700 1 --2- 172.21.15.144:0/1363012481 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1678100db0 0x7f1678104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.273+0000 7f167d632700 1 --2- 172.21.15.144:0/1363012481 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1678100410 0x7f16781007e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.273+0000 7f167d632700 1 -- 172.21.15.144:0/1363012481 >> 172.21.15.144:0/1363012481 conn(0x7f16780fbcf0 msgr2=0x7f16780fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:21.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.273+0000 7f167d632700 1 -- 172.21.15.144:0/1363012481 shutdown_connections 2024-09-10T12:02:21.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.273+0000 7f167d632700 1 -- 172.21.15.144:0/1363012481 wait complete. 2024-09-10T12:02:21.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.273+0000 7f167d632700 1 Processor -- start 2024-09-10T12:02:21.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.273+0000 7f167d632700 1 -- start start 2024-09-10T12:02:21.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.274+0000 7f167d632700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1678100410 0x7f1678195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:21.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.274+0000 7f167d632700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1678100db0 0x7f1678196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:21.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.274+0000 7f167d632700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16781969e0 con 0x7f1678100db0 2024-09-10T12:02:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.274+0000 7f167d632700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167819a770 con 0x7f1678100410 2024-09-10T12:02:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.274+0000 7f166ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1678100db0 0x7f1678196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.274+0000 7f166ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1678100db0 0x7f1678196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45202/0 (socket says 172.21.15.144:45202) 2024-09-10T12:02:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.274+0000 7f166ffff700 1 -- 172.21.15.144:0/2900490119 learned_addr learned my addr 172.21.15.144:0/2900490119 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:02:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.274+0000 7f1676ffd700 1 --2- 172.21.15.144:0/2900490119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1678100410 0x7f1678195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.274+0000 7f166ffff700 1 -- 172.21.15.144:0/2900490119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1678100410 msgr2=0x7f1678195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.274+0000 7f166ffff700 1 --2- 172.21.15.144:0/2900490119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1678100410 0x7f1678195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.274+0000 7f166ffff700 1 -- 172.21.15.144:0/2900490119 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16680095f0 con 0x7f1678100db0 2024-09-10T12:02:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.274+0000 7f166ffff700 1 --2- 172.21.15.144:0/2900490119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1678100db0 0x7f1678196300 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f166000d900 tx=0x7f166000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.274+0000 7f1674ff9700 1 -- 172.21.15.144:0/2900490119 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1660009900 con 0x7f1678100db0 2024-09-10T12:02:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.274+0000 7f1674ff9700 1 -- 172.21.15.144:0/2900490119 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1660010460 con 0x7f1678100db0 2024-09-10T12:02:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.274+0000 7f1674ff9700 1 -- 172.21.15.144:0/2900490119 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f166000f5f0 con 0x7f1678100db0 2024-09-10T12:02:21.277 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.275+0000 7f167d632700 1 -- 172.21.15.144:0/2900490119 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167819aa50 con 0x7f1678100db0 2024-09-10T12:02:21.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.275+0000 7f167d632700 1 -- 172.21.15.144:0/2900490119 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167819afa0 con 0x7f1678100db0 2024-09-10T12:02:21.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.275+0000 7f167d632700 1 -- 172.21.15.144:0/2900490119 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167804ea50 con 0x7f1678100db0 2024-09-10T12:02:21.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.276+0000 7f1674ff9700 1 -- 172.21.15.144:0/2900490119 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f16600105d0 con 0x7f1678100db0 2024-09-10T12:02:21.278 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.276+0000 7f1674ff9700 1 --2- 172.21.15.144:0/2900490119 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f166406e200 0x7f16640706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:21.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.277+0000 7f1674ff9700 1 -- 172.21.15.144:0/2900490119 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f166008db40 con 0x7f1678100db0 2024-09-10T12:02:21.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.277+0000 7f1676ffd700 1 --2- 172.21.15.144:0/2900490119 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f166406e200 0x7f16640706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:21.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.277+0000 7f1676ffd700 1 --2- 172.21.15.144:0/2900490119 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f166406e200 0x7f16640706b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1668006010 tx=0x7f166801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:21.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.281+0000 7f1674ff9700 1 -- 172.21.15.144:0/2900490119 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1660056fa0 con 0x7f1678100db0 2024-09-10T12:02:21.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.490+0000 7f167d632700 1 -- 172.21.15.144:0/2900490119 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f167819b4d0 con 0x7f166406e200 2024-09-10T12:02:21.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.499+0000 7f1674ff9700 1 -- 172.21.15.144:0/2900490119 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f167819b4d0 con 0x7f166406e200 2024-09-10T12:02:21.501 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:02:21.501 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (30m) 8m ago 41m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:02:21.501 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (41m) 8m ago 41m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:02:21.501 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (40m) 8m ago 40m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:02:21.501 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (28m) 8m ago 41m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:02:21.501 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (28m) 8m ago 40m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:02:21.501 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (30m) 8m ago 40m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:02:21.501 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (24m) 8m ago 35m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:02:21.501 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (24m) 8m ago 35m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:02:21.502 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (8m) 8m ago 35m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:02:21.502 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (24m) 8m ago 35m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:02:21.502 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (32m) 8m ago 42m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:02:21.502 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (31m) 8m ago 40m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:02:21.502 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (29m) 8m ago 42m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:02:21.502 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (28m) 8m ago 40m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:02:21.502 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (31m) 8m ago 41m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:02:21.502 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (31m) 8m ago 40m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:02:21.502 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (27m) 8m ago 39m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:02:21.502 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (27m) 8m ago 38m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:02:21.502 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (26m) 8m ago 38m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:02:21.502 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (26m) 8m ago 38m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:02:21.502 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (25m) 8m ago 37m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:02:21.502 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (24m) 8m ago 37m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:02:21.503 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (30m) 8m ago 40m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:02:21.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.501+0000 7f167d632700 1 -- 172.21.15.144:0/2900490119 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f166406e200 msgr2=0x7f16640706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:21.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.501+0000 7f167d632700 1 --2- 172.21.15.144:0/2900490119 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f166406e200 0x7f16640706b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f1668006010 tx=0x7f166801a040 comp rx=0 tx=0).stop 2024-09-10T12:02:21.503 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.501+0000 7f167d632700 1 -- 172.21.15.144:0/2900490119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1678100db0 msgr2=0x7f1678196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:21.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.501+0000 7f167d632700 1 --2- 172.21.15.144:0/2900490119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1678100db0 0x7f1678196300 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f166000d900 tx=0x7f166000dc10 comp rx=0 tx=0).stop 2024-09-10T12:02:21.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.501+0000 7f167d632700 1 -- 172.21.15.144:0/2900490119 shutdown_connections 2024-09-10T12:02:21.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.501+0000 7f167d632700 1 --2- 172.21.15.144:0/2900490119 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f166406e200 0x7f16640706b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.501+0000 7f167d632700 1 --2- 172.21.15.144:0/2900490119 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1678100410 0x7f1678195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.501+0000 7f167d632700 1 --2- 172.21.15.144:0/2900490119 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1678100db0 0x7f1678196300 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.501+0000 7f167d632700 1 -- 172.21.15.144:0/2900490119 >> 172.21.15.144:0/2900490119 conn(0x7f16780fbcf0 msgr2=0x7f16780fd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:21.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.501+0000 7f167d632700 1 -- 172.21.15.144:0/2900490119 shutdown_connections 2024-09-10T12:02:21.504 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.501+0000 7f167d632700 1 -- 172.21.15.144:0/2900490119 wait complete. 2024-09-10T12:02:21.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.604+0000 7fb63ff37700 1 -- 172.21.15.144:0/1718405157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb638101c30 msgr2=0x7fb63810a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:21.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.604+0000 7fb63ff37700 1 --2- 172.21.15.144:0/1718405157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb638101c30 0x7fb63810a130 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fb628009940 tx=0x7fb628009c50 comp rx=0 tx=0).stop 2024-09-10T12:02:21.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.605+0000 7fb63ff37700 1 -- 172.21.15.144:0/1718405157 shutdown_connections 2024-09-10T12:02:21.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.605+0000 7fb63ff37700 1 --2- 172.21.15.144:0/1718405157 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb638101c30 0x7fb63810a130 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.605+0000 7fb63ff37700 1 --2- 172.21.15.144:0/1718405157 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb638101320 0x7fb6381016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.605+0000 7fb63ff37700 1 -- 172.21.15.144:0/1718405157 >> 172.21.15.144:0/1718405157 conn(0x7fb6380fab80 msgr2=0x7fb6380fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:21.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.605+0000 7fb63ff37700 1 -- 172.21.15.144:0/1718405157 shutdown_connections 2024-09-10T12:02:21.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.605+0000 7fb63ff37700 1 -- 172.21.15.144:0/1718405157 wait complete. 2024-09-10T12:02:21.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.605+0000 7fb63ff37700 1 Processor -- start 2024-09-10T12:02:21.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.606+0000 7fb63ff37700 1 -- start start 2024-09-10T12:02:21.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.606+0000 7fb63ff37700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb638101320 0x7fb63819bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:21.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.606+0000 7fb63ff37700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb638101c30 0x7fb63819bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:21.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.606+0000 7fb63ff37700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63819c680 con 0x7fb638101320 2024-09-10T12:02:21.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.606+0000 7fb63ff37700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb638195b30 con 0x7fb638101c30 2024-09-10T12:02:21.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.606+0000 7fb63dcd3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb638101320 0x7fb63819bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:21.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.606+0000 7fb63dcd3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb638101320 0x7fb63819bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45214/0 (socket says 172.21.15.144:45214) 2024-09-10T12:02:21.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.606+0000 7fb63dcd3700 1 -- 172.21.15.144:0/1698602880 learned_addr learned my addr 172.21.15.144:0/1698602880 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:02:21.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.606+0000 7fb63d4d2700 1 --2- 172.21.15.144:0/1698602880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb638101c30 0x7fb63819bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:21.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.606+0000 7fb63dcd3700 1 -- 172.21.15.144:0/1698602880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb638101c30 msgr2=0x7fb63819bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:21.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.606+0000 7fb63dcd3700 1 --2- 172.21.15.144:0/1698602880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb638101c30 0x7fb63819bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.606+0000 7fb63dcd3700 1 -- 172.21.15.144:0/1698602880 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6280095f0 con 0x7fb638101320 2024-09-10T12:02:21.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.606+0000 7fb63dcd3700 1 --2- 172.21.15.144:0/1698602880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb638101320 0x7fb63819bab0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fb63400e920 tx=0x7fb63400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:21.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.607+0000 7fb62effd700 1 -- 172.21.15.144:0/1698602880 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb634009790 con 0x7fb638101320 2024-09-10T12:02:21.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.607+0000 7fb63ff37700 1 -- 172.21.15.144:0/1698602880 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb638195e10 con 0x7fb638101320 2024-09-10T12:02:21.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.607+0000 7fb62effd700 1 -- 172.21.15.144:0/1698602880 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb63400cba0 con 0x7fb638101320 2024-09-10T12:02:21.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.607+0000 7fb62effd700 1 -- 172.21.15.144:0/1698602880 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb634018650 con 0x7fb638101320 2024-09-10T12:02:21.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.607+0000 7fb63ff37700 1 -- 172.21.15.144:0/1698602880 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb638196360 con 0x7fb638101320 2024-09-10T12:02:21.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.608+0000 7fb63ff37700 1 -- 172.21.15.144:0/1698602880 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6381077a0 con 0x7fb638101320 2024-09-10T12:02:21.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.608+0000 7fb62effd700 1 -- 172.21.15.144:0/1698602880 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb63400cd10 con 0x7fb638101320 2024-09-10T12:02:21.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.609+0000 7fb62effd700 1 --2- 172.21.15.144:0/1698602880 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb62406e200 0x7fb6240706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:21.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.609+0000 7fb62effd700 1 -- 172.21.15.144:0/1698602880 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb634014070 con 0x7fb638101320 2024-09-10T12:02:21.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.609+0000 7fb63d4d2700 1 --2- 172.21.15.144:0/1698602880 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb62406e200 0x7fb6240706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:21.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.609+0000 7fb63d4d2700 1 --2- 172.21.15.144:0/1698602880 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb62406e200 0x7fb6240706b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb6380fc300 tx=0x7fb628005e60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:21.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.615+0000 7fb62effd700 1 -- 172.21.15.144:0/1698602880 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb6340541b0 con 0x7fb638101320 2024-09-10T12:02:21.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.901+0000 7fb63ff37700 1 -- 172.21.15.144:0/1698602880 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb638196640 con 0x7fb638101320 2024-09-10T12:02:21.903 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:21 smithi144 ceph-mon[87436]: from='client.35660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:21.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.902+0000 7fb62effd700 1 -- 172.21.15.144:0/1698602880 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fb63407b290 con 0x7fb638101320 2024-09-10T12:02:21.903 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:02:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:02:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:02:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:02:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:02:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:02:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:02:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:02:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:02:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:02:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:02:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:02:21.904 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:02:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:02:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:02:21.905 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:02:21.905 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:02:21.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.904+0000 7fb63ff37700 1 -- 172.21.15.144:0/1698602880 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb62406e200 msgr2=0x7fb6240706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:21.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.904+0000 7fb63ff37700 1 --2- 172.21.15.144:0/1698602880 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb62406e200 0x7fb6240706b0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fb6380fc300 tx=0x7fb628005e60 comp rx=0 tx=0).stop 2024-09-10T12:02:21.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.904+0000 7fb63ff37700 1 -- 172.21.15.144:0/1698602880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb638101320 msgr2=0x7fb63819bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:21.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.904+0000 7fb63ff37700 1 --2- 172.21.15.144:0/1698602880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb638101320 0x7fb63819bab0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fb63400e920 tx=0x7fb63400ece0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.904+0000 7fb63ff37700 1 -- 172.21.15.144:0/1698602880 shutdown_connections 2024-09-10T12:02:21.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.904+0000 7fb63ff37700 1 --2- 172.21.15.144:0/1698602880 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb62406e200 0x7fb6240706b0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.904+0000 7fb63ff37700 1 --2- 172.21.15.144:0/1698602880 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb638101320 0x7fb63819bab0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.904+0000 7fb63ff37700 1 --2- 172.21.15.144:0/1698602880 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb638101c30 0x7fb63819bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:21.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.904+0000 7fb63ff37700 1 -- 172.21.15.144:0/1698602880 >> 172.21.15.144:0/1698602880 conn(0x7fb6380fab80 msgr2=0x7fb6381048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:21.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.904+0000 7fb63ff37700 1 -- 172.21.15.144:0/1698602880 shutdown_connections 2024-09-10T12:02:21.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:21.904+0000 7fb63ff37700 1 -- 172.21.15.144:0/1698602880 wait complete. 2024-09-10T12:02:22.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.009+0000 7f4fda4b2700 1 -- 172.21.15.144:0/638421664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd41003d0 msgr2=0x7f4fd41007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:22.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.009+0000 7f4fda4b2700 1 --2- 172.21.15.144:0/638421664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd41003d0 0x7f4fd41007a0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4009910 tx=0x7f4fc4009c20 comp rx=0 tx=0).stop 2024-09-10T12:02:22.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.009+0000 7f4fda4b2700 1 -- 172.21.15.144:0/638421664 shutdown_connections 2024-09-10T12:02:22.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.009+0000 7f4fda4b2700 1 --2- 172.21.15.144:0/638421664 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fd4100d70 0x7f4fd4104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.009+0000 7f4fda4b2700 1 --2- 172.21.15.144:0/638421664 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd41003d0 0x7f4fd41007a0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.009+0000 7f4fda4b2700 1 -- 172.21.15.144:0/638421664 >> 172.21.15.144:0/638421664 conn(0x7f4fd40fbc90 msgr2=0x7f4fd40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:22.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.009+0000 7f4fda4b2700 1 -- 172.21.15.144:0/638421664 shutdown_connections 2024-09-10T12:02:22.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.009+0000 7f4fda4b2700 1 -- 172.21.15.144:0/638421664 wait complete. 2024-09-10T12:02:22.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.009+0000 7f4fda4b2700 1 Processor -- start 2024-09-10T12:02:22.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.010+0000 7f4fda4b2700 1 -- start start 2024-09-10T12:02:22.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.010+0000 7f4fda4b2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd41003d0 0x7f4fd4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:22.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.010+0000 7f4fda4b2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fd4100d70 0x7f4fd4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:22.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.010+0000 7f4fda4b2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd4073ad0 con 0x7f4fd41003d0 2024-09-10T12:02:22.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.010+0000 7f4fda4b2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd4073c10 con 0x7f4fd4100d70 2024-09-10T12:02:22.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.010+0000 7f4fd3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd41003d0 0x7f4fd4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:22.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.010+0000 7f4fd3fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd41003d0 0x7f4fd4073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:45240/0 (socket says 172.21.15.144:45240) 2024-09-10T12:02:22.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.010+0000 7f4fd3fff700 1 -- 172.21.15.144:0/2301682387 learned_addr learned my addr 172.21.15.144:0/2301682387 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:02:22.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.010+0000 7f4fcbfff700 1 --2- 172.21.15.144:0/2301682387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fd4100d70 0x7f4fd4073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:22.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.010+0000 7f4fd3fff700 1 -- 172.21.15.144:0/2301682387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fd4100d70 msgr2=0x7f4fd4073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:22.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.010+0000 7f4fd3fff700 1 --2- 172.21.15.144:0/2301682387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fd4100d70 0x7f4fd4073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.010+0000 7f4fd3fff700 1 -- 172.21.15.144:0/2301682387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fc40095f0 con 0x7f4fd41003d0 2024-09-10T12:02:22.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.010+0000 7f4fd3fff700 1 --2- 172.21.15.144:0/2301682387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd41003d0 0x7f4fd4073050 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4006010 tx=0x7f4fc400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:22.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.010+0000 7f4fd1ffb700 1 -- 172.21.15.144:0/2301682387 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fc401d070 con 0x7f4fd41003d0 2024-09-10T12:02:22.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.011+0000 7f4fd1ffb700 1 -- 172.21.15.144:0/2301682387 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4fc4004fb0 con 0x7f4fd41003d0 2024-09-10T12:02:22.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.011+0000 7f4fda4b2700 1 -- 172.21.15.144:0/2301682387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd4073e90 con 0x7f4fd41003d0 2024-09-10T12:02:22.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.011+0000 7f4fd1ffb700 1 -- 172.21.15.144:0/2301682387 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fc4022ea0 con 0x7f4fd41003d0 2024-09-10T12:02:22.014 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.011+0000 7f4fda4b2700 1 -- 172.21.15.144:0/2301682387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd41a23d0 con 0x7f4fd41003d0 2024-09-10T12:02:22.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.012+0000 7f4fda4b2700 1 -- 172.21.15.144:0/2301682387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd41086e0 con 0x7f4fd41003d0 2024-09-10T12:02:22.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.012+0000 7f4fd1ffb700 1 -- 172.21.15.144:0/2301682387 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4fc4022470 con 0x7f4fd41003d0 2024-09-10T12:02:22.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.013+0000 7f4fd1ffb700 1 --2- 172.21.15.144:0/2301682387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fc006e200 0x7f4fc00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:22.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.013+0000 7f4fd1ffb700 1 -- 172.21.15.144:0/2301682387 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4fc408f5f0 con 0x7f4fd41003d0 2024-09-10T12:02:22.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.013+0000 7f4fcbfff700 1 --2- 172.21.15.144:0/2301682387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fc006e200 0x7f4fc00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:22.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.013+0000 7f4fcbfff700 1 --2- 172.21.15.144:0/2301682387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fc006e200 0x7f4fc00706b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc006fd0 tx=0x7f4fbc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:22.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.017+0000 7f4fd1ffb700 1 -- 172.21.15.144:0/2301682387 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4fc4058a80 con 0x7f4fd41003d0 2024-09-10T12:02:22.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:21 smithi178 ceph-mon[65940]: from='client.35660 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:22.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.269+0000 7f4fda4b2700 1 -- 172.21.15.144:0/2301682387 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4fd41080c0 con 0x7f4fd41003d0 2024-09-10T12:02:22.270 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.269+0000 7f4fd1ffb700 1 -- 172.21.15.144:0/2301682387 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f4fc4022720 con 0x7f4fd41003d0 2024-09-10T12:02:22.271 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:02:22.271 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:02:22.271 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:02:22.271 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:02:22.271 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:02:22.271 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:02:22.271 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:02:22.271 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:02:22.271 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:02:22.271 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:02:22.271 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:02:22.271 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:02:22.272 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:02:22.272 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:02:22.272 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:02:22.272 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:02:22.272 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:02:22.272 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:02:22.272 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:02:22.272 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:02:22.272 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:02:22.272 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:02:22.272 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:02:22.272 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:02:22.273 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:02:22.273 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:02:22.273 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:02:22.273 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:02:22.273 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:02:22.273 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:02:22.273 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:02:22.273 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:02:22.273 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:02:22.273 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:02:22.273 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:02:22.273 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:02:22.273 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:02:22.274 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:02:22.274 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:02:22.274 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:02:22.274 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:02:22.274 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:02:22.274 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:02:22.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.271+0000 7f4fda4b2700 1 -- 172.21.15.144:0/2301682387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fc006e200 msgr2=0x7f4fc00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:22.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.271+0000 7f4fda4b2700 1 --2- 172.21.15.144:0/2301682387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fc006e200 0x7f4fc00706b0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc006fd0 tx=0x7f4fbc008040 comp rx=0 tx=0).stop 2024-09-10T12:02:22.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.271+0000 7f4fda4b2700 1 -- 172.21.15.144:0/2301682387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd41003d0 msgr2=0x7f4fd4073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:22.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.271+0000 7f4fda4b2700 1 --2- 172.21.15.144:0/2301682387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd41003d0 0x7f4fd4073050 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4006010 tx=0x7f4fc400be20 comp rx=0 tx=0).stop 2024-09-10T12:02:22.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.271+0000 7f4fda4b2700 1 -- 172.21.15.144:0/2301682387 shutdown_connections 2024-09-10T12:02:22.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.271+0000 7f4fda4b2700 1 --2- 172.21.15.144:0/2301682387 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4fc006e200 0x7f4fc00706b0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.271+0000 7f4fda4b2700 1 --2- 172.21.15.144:0/2301682387 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4fd41003d0 0x7f4fd4073050 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.271+0000 7f4fda4b2700 1 --2- 172.21.15.144:0/2301682387 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4fd4100d70 0x7f4fd4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.275 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.271+0000 7f4fda4b2700 1 -- 172.21.15.144:0/2301682387 >> 172.21.15.144:0/2301682387 conn(0x7f4fd40fbc90 msgr2=0x7f4fd40fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:22.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.271+0000 7f4fda4b2700 1 -- 172.21.15.144:0/2301682387 shutdown_connections 2024-09-10T12:02:22.276 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.271+0000 7f4fda4b2700 1 -- 172.21.15.144:0/2301682387 wait complete. 2024-09-10T12:02:22.276 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:02:22.378 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.377+0000 7f592328c700 1 -- 172.21.15.144:0/814191333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f591c100d50 msgr2=0x7f591c104d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:22.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.377+0000 7f592328c700 1 --2- 172.21.15.144:0/814191333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f591c100d50 0x7f591c104d20 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f5918009910 tx=0x7f5918009c20 comp rx=0 tx=0).stop 2024-09-10T12:02:22.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.377+0000 7f592328c700 1 -- 172.21.15.144:0/814191333 shutdown_connections 2024-09-10T12:02:22.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.377+0000 7f592328c700 1 --2- 172.21.15.144:0/814191333 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f591c100d50 0x7f591c104d20 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.377+0000 7f592328c700 1 --2- 172.21.15.144:0/814191333 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591c1003b0 0x7f591c100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.377+0000 7f592328c700 1 -- 172.21.15.144:0/814191333 >> 172.21.15.144:0/814191333 conn(0x7f591c0fbc30 msgr2=0x7f591c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:22.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.377+0000 7f592328c700 1 -- 172.21.15.144:0/814191333 shutdown_connections 2024-09-10T12:02:22.379 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.377+0000 7f592328c700 1 -- 172.21.15.144:0/814191333 wait complete. 2024-09-10T12:02:22.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.378+0000 7f592328c700 1 Processor -- start 2024-09-10T12:02:22.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.378+0000 7f592328c700 1 -- start start 2024-09-10T12:02:22.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.378+0000 7f592328c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591c1003b0 0x7f591c195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:22.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.378+0000 7f592328c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f591c100d50 0x7f591c196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:22.381 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.378+0000 7f592328c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c1969e0 con 0x7f591c100d50 2024-09-10T12:02:22.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.378+0000 7f592328c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c19a770 con 0x7f591c1003b0 2024-09-10T12:02:22.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.378+0000 7f5920827700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f591c100d50 0x7f591c196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:22.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.378+0000 7f5920827700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f591c100d50 0x7f591c196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51414/0 (socket says 172.21.15.144:51414) 2024-09-10T12:02:22.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.378+0000 7f5920827700 1 -- 172.21.15.144:0/2074977425 learned_addr learned my addr 172.21.15.144:0/2074977425 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:02:22.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.378+0000 7f5921028700 1 --2- 172.21.15.144:0/2074977425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591c1003b0 0x7f591c195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:22.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.378+0000 7f5920827700 1 -- 172.21.15.144:0/2074977425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591c1003b0 msgr2=0x7f591c195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:22.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.378+0000 7f5920827700 1 --2- 172.21.15.144:0/2074977425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591c1003b0 0x7f591c195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.378+0000 7f5920827700 1 -- 172.21.15.144:0/2074977425 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59180095f0 con 0x7f591c100d50 2024-09-10T12:02:22.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.378+0000 7f5920827700 1 --2- 172.21.15.144:0/2074977425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f591c100d50 0x7f591c196300 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f591800bd70 tx=0x7f591800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:22.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.379+0000 7f59127fc700 1 -- 172.21.15.144:0/2074977425 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f591801d070 con 0x7f591c100d50 2024-09-10T12:02:22.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.379+0000 7f59127fc700 1 -- 172.21.15.144:0/2074977425 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5918005dc0 con 0x7f591c100d50 2024-09-10T12:02:22.382 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.379+0000 7f592328c700 1 -- 172.21.15.144:0/2074977425 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591c19a9f0 con 0x7f591c100d50 2024-09-10T12:02:22.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.379+0000 7f59127fc700 1 -- 172.21.15.144:0/2074977425 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5918017400 con 0x7f591c100d50 2024-09-10T12:02:22.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.379+0000 7f592328c700 1 -- 172.21.15.144:0/2074977425 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591c19aee0 con 0x7f591c100d50 2024-09-10T12:02:22.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.380+0000 7f592328c700 1 -- 172.21.15.144:0/2074977425 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f591c108720 con 0x7f591c100d50 2024-09-10T12:02:22.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.380+0000 7f59127fc700 1 -- 172.21.15.144:0/2074977425 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f591800f460 con 0x7f591c100d50 2024-09-10T12:02:22.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.380+0000 7f59127fc700 1 --2- 172.21.15.144:0/2074977425 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f590806e200 0x7f59080706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:22.383 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.381+0000 7f59127fc700 1 -- 172.21.15.144:0/2074977425 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f591808f520 con 0x7f591c100d50 2024-09-10T12:02:22.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.381+0000 7f5921028700 1 --2- 172.21.15.144:0/2074977425 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f590806e200 0x7f59080706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:22.384 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.381+0000 7f5921028700 1 --2- 172.21.15.144:0/2074977425 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f590806e200 0x7f59080706b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f590c005950 tx=0x7f590c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:22.386 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.385+0000 7f59127fc700 1 -- 172.21.15.144:0/2074977425 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5918058900 con 0x7f591c100d50 2024-09-10T12:02:22.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.606+0000 7f592328c700 1 -- 172.21.15.144:0/2074977425 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f591c02cca0 con 0x7f590806e200 2024-09-10T12:02:22.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.607+0000 7f59127fc700 1 -- 172.21.15.144:0/2074977425 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f591c02cca0 con 0x7f590806e200 2024-09-10T12:02:22.609 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:02:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:02:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:02:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:02:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:02:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:02:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:02:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:02:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:02:22.609 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:02:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:02:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:02:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:02:22.610 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:02:22.610 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:02:22.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.610+0000 7f592328c700 1 -- 172.21.15.144:0/2074977425 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f590806e200 msgr2=0x7f59080706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:22.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.610+0000 7f592328c700 1 --2- 172.21.15.144:0/2074977425 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f590806e200 0x7f59080706b0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f590c005950 tx=0x7f590c00b410 comp rx=0 tx=0).stop 2024-09-10T12:02:22.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.610+0000 7f592328c700 1 -- 172.21.15.144:0/2074977425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f591c100d50 msgr2=0x7f591c196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:22.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.610+0000 7f592328c700 1 --2- 172.21.15.144:0/2074977425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f591c100d50 0x7f591c196300 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f591800bd70 tx=0x7f591800bda0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.610+0000 7f592328c700 1 -- 172.21.15.144:0/2074977425 shutdown_connections 2024-09-10T12:02:22.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.610+0000 7f592328c700 1 --2- 172.21.15.144:0/2074977425 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f590806e200 0x7f59080706b0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.610+0000 7f592328c700 1 --2- 172.21.15.144:0/2074977425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f591c1003b0 0x7f591c195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.610+0000 7f592328c700 1 --2- 172.21.15.144:0/2074977425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f591c100d50 0x7f591c196300 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.610+0000 7f592328c700 1 -- 172.21.15.144:0/2074977425 >> 172.21.15.144:0/2074977425 conn(0x7f591c0fbc30 msgr2=0x7f591c0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:22.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.610+0000 7f592328c700 1 -- 172.21.15.144:0/2074977425 shutdown_connections 2024-09-10T12:02:22.612 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.610+0000 7f592328c700 1 -- 172.21.15.144:0/2074977425 wait complete. 2024-09-10T12:02:22.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.714+0000 7fc41f6c1700 1 -- 172.21.15.144:0/1285616153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc418101c50 msgr2=0x7fc41810a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:22.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.714+0000 7fc41f6c1700 1 --2- 172.21.15.144:0/1285616153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc418101c50 0x7fc41810a150 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fc40c009940 tx=0x7fc40c009c50 comp rx=0 tx=0).stop 2024-09-10T12:02:22.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.714+0000 7fc41f6c1700 1 -- 172.21.15.144:0/1285616153 shutdown_connections 2024-09-10T12:02:22.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.714+0000 7fc41f6c1700 1 --2- 172.21.15.144:0/1285616153 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc418101c50 0x7fc41810a150 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.714+0000 7fc41f6c1700 1 --2- 172.21.15.144:0/1285616153 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc418101340 0x7fc418101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.714+0000 7fc41f6c1700 1 -- 172.21.15.144:0/1285616153 >> 172.21.15.144:0/1285616153 conn(0x7fc4180fab80 msgr2=0x7fc4180fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:22.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.714+0000 7fc41f6c1700 1 -- 172.21.15.144:0/1285616153 shutdown_connections 2024-09-10T12:02:22.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.714+0000 7fc41f6c1700 1 -- 172.21.15.144:0/1285616153 wait complete. 2024-09-10T12:02:22.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.715+0000 7fc41f6c1700 1 Processor -- start 2024-09-10T12:02:22.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.715+0000 7fc41f6c1700 1 -- start start 2024-09-10T12:02:22.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.715+0000 7fc41f6c1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc418101340 0x7fc418195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:22.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.715+0000 7fc41f6c1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc418101c50 0x7fc418196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:22.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.715+0000 7fc41f6c1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc418196890 con 0x7fc418101340 2024-09-10T12:02:22.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.715+0000 7fc41f6c1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4181995c0 con 0x7fc418101c50 2024-09-10T12:02:22.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.715+0000 7fc41d45d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc418101340 0x7fc418195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:22.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.715+0000 7fc41d45d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc418101340 0x7fc418195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51424/0 (socket says 172.21.15.144:51424) 2024-09-10T12:02:22.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.715+0000 7fc41d45d700 1 -- 172.21.15.144:0/1516673714 learned_addr learned my addr 172.21.15.144:0/1516673714 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:02:22.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.715+0000 7fc41cc5c700 1 --2- 172.21.15.144:0/1516673714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc418101c50 0x7fc418196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:22.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.715+0000 7fc41d45d700 1 -- 172.21.15.144:0/1516673714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc418101c50 msgr2=0x7fc418196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:22.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.715+0000 7fc41d45d700 1 --2- 172.21.15.144:0/1516673714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc418101c50 0x7fc418196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:22.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.715+0000 7fc41d45d700 1 -- 172.21.15.144:0/1516673714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40c0095f0 con 0x7fc418101340 2024-09-10T12:02:22.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.715+0000 7fc41d45d700 1 --2- 172.21.15.144:0/1516673714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc418101340 0x7fc418195cc0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc4100060b0 tx=0x7fc41000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:22.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.716+0000 7fc4167fc700 1 -- 172.21.15.144:0/1516673714 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc41000dda0 con 0x7fc418101340 2024-09-10T12:02:22.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.716+0000 7fc4167fc700 1 -- 172.21.15.144:0/1516673714 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc41000f040 con 0x7fc418101340 2024-09-10T12:02:22.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.716+0000 7fc41f6c1700 1 -- 172.21.15.144:0/1516673714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4181998a0 con 0x7fc418101340 2024-09-10T12:02:22.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.716+0000 7fc4167fc700 1 -- 172.21.15.144:0/1516673714 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc410014790 con 0x7fc418101340 2024-09-10T12:02:22.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.716+0000 7fc41f6c1700 1 -- 172.21.15.144:0/1516673714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc418199dc0 con 0x7fc418101340 2024-09-10T12:02:22.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.717+0000 7fc41f6c1700 1 -- 172.21.15.144:0/1516673714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4181077c0 con 0x7fc418101340 2024-09-10T12:02:22.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.717+0000 7fc4167fc700 1 -- 172.21.15.144:0/1516673714 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc410014980 con 0x7fc418101340 2024-09-10T12:02:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.718+0000 7fc4167fc700 1 --2- 172.21.15.144:0/1516673714 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc40006e200 0x7fc4000706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.718+0000 7fc4167fc700 1 -- 172.21.15.144:0/1516673714 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc41008db30 con 0x7fc418101340 2024-09-10T12:02:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.718+0000 7fc41cc5c700 1 --2- 172.21.15.144:0/1516673714 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc40006e200 0x7fc4000706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:22.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.718+0000 7fc41cc5c700 1 --2- 172.21.15.144:0/1516673714 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc40006e200 0x7fc4000706b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc418197340 tx=0x7fc40c005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:22.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:22.722+0000 7fc4167fc700 1 -- 172.21.15.144:0/1516673714 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc410056fc0 con 0x7fc418101340 2024-09-10T12:02:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:22 smithi144 ceph-mon[87436]: pgmap v912: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:22 smithi144 ceph-mon[87436]: from='client.35664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:22 smithi144 ceph-mon[87436]: from='client.35668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:22 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1698602880' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:02:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:22 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2301682387' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:02:23.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:23.005+0000 7fc41f6c1700 1 -- 172.21.15.144:0/1516673714 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc41804ea50 con 0x7fc418101340 2024-09-10T12:02:23.007 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:23.005+0000 7fc4167fc700 1 -- 172.21.15.144:0/1516673714 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7fc41005a680 con 0x7fc418101340 2024-09-10T12:02:23.007 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:02:23.007 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:02:23.007 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:02:23.007 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:02:23.007 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:02:23.007 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:02:23.008 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:02:23.008 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:02:23.008 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:02:23.008 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:02:23.008 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:02:23.008 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:02:23.008 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:02:23.008 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:02:23.008 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:02:23.008 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:02:23.008 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:02:23.008 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:02:23.009 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:02:23.009 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:02:23.009 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:02:23.009 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:02:23.009 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:02:23.009 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:02:23.009 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:02:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:23.008+0000 7fc41f6c1700 1 -- 172.21.15.144:0/1516673714 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc40006e200 msgr2=0x7fc4000706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:23.008+0000 7fc41f6c1700 1 --2- 172.21.15.144:0/1516673714 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc40006e200 0x7fc4000706b0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc418197340 tx=0x7fc40c005a90 comp rx=0 tx=0).stop 2024-09-10T12:02:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:23.008+0000 7fc41f6c1700 1 -- 172.21.15.144:0/1516673714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc418101340 msgr2=0x7fc418195cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:23.008+0000 7fc41f6c1700 1 --2- 172.21.15.144:0/1516673714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc418101340 0x7fc418195cc0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fc4100060b0 tx=0x7fc41000d3b0 comp rx=0 tx=0).stop 2024-09-10T12:02:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:23.008+0000 7fc41f6c1700 1 -- 172.21.15.144:0/1516673714 shutdown_connections 2024-09-10T12:02:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:23.008+0000 7fc41f6c1700 1 --2- 172.21.15.144:0/1516673714 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc40006e200 0x7fc4000706b0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:23.008+0000 7fc41f6c1700 1 --2- 172.21.15.144:0/1516673714 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc418101340 0x7fc418195cc0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:23.008+0000 7fc41f6c1700 1 --2- 172.21.15.144:0/1516673714 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc418101c50 0x7fc418196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:23.008+0000 7fc41f6c1700 1 -- 172.21.15.144:0/1516673714 >> 172.21.15.144:0/1516673714 conn(0x7fc4180fab80 msgr2=0x7fc418104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:23.010 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:23.008+0000 7fc41f6c1700 1 -- 172.21.15.144:0/1516673714 shutdown_connections 2024-09-10T12:02:23.011 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:23.008+0000 7fc41f6c1700 1 -- 172.21.15.144:0/1516673714 wait complete. 2024-09-10T12:02:23.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:22 smithi178 ceph-mon[65940]: pgmap v912: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:23.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:22 smithi178 ceph-mon[65940]: from='client.35664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:23.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:22 smithi178 ceph-mon[65940]: from='client.35668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:23.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:22 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1698602880' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:02:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:22 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2301682387' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:02:24.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:23 smithi178 ceph-mon[65940]: from='client.35680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:24.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:23 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1516673714' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:02:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:23 smithi144 ceph-mon[87436]: from='client.35680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:23 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1516673714' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:02:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:24 smithi178 ceph-mon[65940]: pgmap v913: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:24 smithi144 ceph-mon[87436]: pgmap v913: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:26 smithi178 ceph-mon[65940]: pgmap v914: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:26 smithi144 ceph-mon[87436]: pgmap v914: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:28 smithi178 ceph-mon[65940]: pgmap v915: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:02:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:28 smithi144 ceph-mon[87436]: pgmap v915: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:02:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:30 smithi178 ceph-mon[65940]: pgmap v916: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:31.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:30 smithi144 ceph-mon[87436]: pgmap v916: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:32 smithi144 ceph-mon[87436]: pgmap v917: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:32 smithi178 ceph-mon[65940]: pgmap v917: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:34 smithi178 ceph-mon[65940]: pgmap v918: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:34 smithi144 ceph-mon[87436]: pgmap v918: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:36 smithi178 ceph-mon[65940]: pgmap v919: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:36 smithi144 ceph-mon[87436]: pgmap v919: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:38 smithi178 ceph-mon[65940]: pgmap v920: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:38 smithi144 ceph-mon[87436]: pgmap v920: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:40 smithi178 ceph-mon[65940]: pgmap v921: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:40 smithi144 ceph-mon[87436]: pgmap v921: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:43.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:42 smithi144 ceph-mon[87436]: pgmap v922: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:42 smithi178 ceph-mon[65940]: pgmap v922: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:44.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:02:44.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:02:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:44 smithi144 ceph-mon[87436]: pgmap v923: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:44 smithi178 ceph-mon[65940]: pgmap v923: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:46 smithi144 ceph-mon[87436]: pgmap v924: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:46 smithi178 ceph-mon[65940]: pgmap v924: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:49.143 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:48 smithi144 ceph-mon[87436]: pgmap v925: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:48 smithi178 ceph-mon[65940]: pgmap v925: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:50 smithi144 ceph-mon[87436]: pgmap v926: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:50 smithi178 ceph-mon[65940]: pgmap v926: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:52 smithi178 ceph-mon[65940]: pgmap v927: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:53.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.125+0000 7f5a0c715700 1 -- 172.21.15.144:0/730876460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a04100d70 msgr2=0x7f5a04104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:53.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.125+0000 7f5a0c715700 1 --2- 172.21.15.144:0/730876460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a04100d70 0x7f5a04104d40 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f5a00009910 tx=0x7f5a00009c20 comp rx=0 tx=0).stop 2024-09-10T12:02:53.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.125+0000 7f5a0c715700 1 -- 172.21.15.144:0/730876460 shutdown_connections 2024-09-10T12:02:53.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.125+0000 7f5a0c715700 1 --2- 172.21.15.144:0/730876460 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a04100d70 0x7f5a04104d40 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:53.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.125+0000 7f5a0c715700 1 --2- 172.21.15.144:0/730876460 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a041003d0 0x7f5a041007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:53.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.125+0000 7f5a0c715700 1 -- 172.21.15.144:0/730876460 >> 172.21.15.144:0/730876460 conn(0x7f5a040fbc50 msgr2=0x7f5a040fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:53.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.126+0000 7f5a0c715700 1 -- 172.21.15.144:0/730876460 shutdown_connections 2024-09-10T12:02:53.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.126+0000 7f5a0c715700 1 -- 172.21.15.144:0/730876460 wait complete. 2024-09-10T12:02:53.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.126+0000 7f5a0c715700 1 Processor -- start 2024-09-10T12:02:53.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.126+0000 7f5a0c715700 1 -- start start 2024-09-10T12:02:53.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.126+0000 7f5a0c715700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a041003d0 0x7f5a04195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:53.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.126+0000 7f5a0c715700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a04100d70 0x7f5a04196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:53.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.126+0000 7f5a0c715700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a04196b00 con 0x7f5a041003d0 2024-09-10T12:02:53.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.126+0000 7f5a0c715700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0419a890 con 0x7f5a04100d70 2024-09-10T12:02:53.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.127+0000 7f5a0a4b1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a041003d0 0x7f5a04195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:53.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.127+0000 7f5a0a4b1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a041003d0 0x7f5a04195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36564/0 (socket says 172.21.15.144:36564) 2024-09-10T12:02:53.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.127+0000 7f5a0a4b1700 1 -- 172.21.15.144:0/898187889 learned_addr learned my addr 172.21.15.144:0/898187889 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:02:53.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.127+0000 7f5a09cb0700 1 --2- 172.21.15.144:0/898187889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a04100d70 0x7f5a04196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:53.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.127+0000 7f5a0a4b1700 1 -- 172.21.15.144:0/898187889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a04100d70 msgr2=0x7f5a04196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:53.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.127+0000 7f5a0a4b1700 1 --2- 172.21.15.144:0/898187889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a04100d70 0x7f5a04196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:53.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.127+0000 7f5a0a4b1700 1 -- 172.21.15.144:0/898187889 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a000095f0 con 0x7f5a041003d0 2024-09-10T12:02:53.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.127+0000 7f5a09cb0700 1 --2- 172.21.15.144:0/898187889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a04100d70 0x7f5a04196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:02:53.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.127+0000 7f5a0a4b1700 1 --2- 172.21.15.144:0/898187889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a041003d0 0x7f5a04195ee0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f59f800b4b0 tx=0x7f59f800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:53.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.127+0000 7f59f77fe700 1 -- 172.21.15.144:0/898187889 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59f8011800 con 0x7f5a041003d0 2024-09-10T12:02:53.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.127+0000 7f59f77fe700 1 -- 172.21.15.144:0/898187889 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f59f8011e20 con 0x7f5a041003d0 2024-09-10T12:02:53.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.127+0000 7f5a0c715700 1 -- 172.21.15.144:0/898187889 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a0419ab70 con 0x7f5a041003d0 2024-09-10T12:02:53.131 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.127+0000 7f59f77fe700 1 -- 172.21.15.144:0/898187889 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59f800f2e0 con 0x7f5a041003d0 2024-09-10T12:02:53.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.127+0000 7f5a0c715700 1 -- 172.21.15.144:0/898187889 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a0419b0c0 con 0x7f5a041003d0 2024-09-10T12:02:53.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.128+0000 7f5a0c715700 1 -- 172.21.15.144:0/898187889 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a04108670 con 0x7f5a041003d0 2024-09-10T12:02:53.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.129+0000 7f59f77fe700 1 -- 172.21.15.144:0/898187889 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f59f8010380 con 0x7f5a041003d0 2024-09-10T12:02:53.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.129+0000 7f59f77fe700 1 --2- 172.21.15.144:0/898187889 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f59f006e200 0x7f59f00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:53.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.129+0000 7f59f77fe700 1 -- 172.21.15.144:0/898187889 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f59f808dba0 con 0x7f5a041003d0 2024-09-10T12:02:53.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.129+0000 7f5a09cb0700 1 --2- 172.21.15.144:0/898187889 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f59f006e200 0x7f59f00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:53.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.130+0000 7f5a09cb0700 1 --2- 172.21.15.144:0/898187889 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f59f006e200 0x7f59f00706b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5a04197500 tx=0x7f5a0001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:53.135 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.134+0000 7f59f77fe700 1 -- 172.21.15.144:0/898187889 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f59f80570b0 con 0x7f5a041003d0 2024-09-10T12:02:53.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:52 smithi144 ceph-mon[87436]: pgmap v927: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:53.357 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.355+0000 7f5a0c715700 1 -- 172.21.15.144:0/898187889 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5a0419b390 con 0x7f59f006e200 2024-09-10T12:02:53.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.357+0000 7f59f77fe700 1 -- 172.21.15.144:0/898187889 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f5a0419b390 con 0x7f59f006e200 2024-09-10T12:02:53.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.359+0000 7f5a0c715700 1 -- 172.21.15.144:0/898187889 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f59f006e200 msgr2=0x7f59f00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:53.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.359+0000 7f5a0c715700 1 --2- 172.21.15.144:0/898187889 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f59f006e200 0x7f59f00706b0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f5a04197500 tx=0x7f5a0001a040 comp rx=0 tx=0).stop 2024-09-10T12:02:53.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.359+0000 7f5a0c715700 1 -- 172.21.15.144:0/898187889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a041003d0 msgr2=0x7f5a04195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:53.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.359+0000 7f5a0c715700 1 --2- 172.21.15.144:0/898187889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a041003d0 0x7f5a04195ee0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f59f800b4b0 tx=0x7f59f800b870 comp rx=0 tx=0).stop 2024-09-10T12:02:53.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.359+0000 7f5a0c715700 1 -- 172.21.15.144:0/898187889 shutdown_connections 2024-09-10T12:02:53.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.359+0000 7f5a0c715700 1 --2- 172.21.15.144:0/898187889 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f59f006e200 0x7f59f00706b0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:53.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.359+0000 7f5a0c715700 1 --2- 172.21.15.144:0/898187889 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5a041003d0 0x7f5a04195ee0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:53.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.359+0000 7f5a0c715700 1 --2- 172.21.15.144:0/898187889 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5a04100d70 0x7f5a04196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:53.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.359+0000 7f5a0c715700 1 -- 172.21.15.144:0/898187889 >> 172.21.15.144:0/898187889 conn(0x7f5a040fbc50 msgr2=0x7f5a040fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:53.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.359+0000 7f5a0c715700 1 -- 172.21.15.144:0/898187889 shutdown_connections 2024-09-10T12:02:53.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.359+0000 7f5a0c715700 1 -- 172.21.15.144:0/898187889 wait complete. 2024-09-10T12:02:53.371 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:02:53.467 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.465+0000 7ff710a9f700 1 -- 172.21.15.144:0/2753696841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c101440 msgr2=0x7ff70c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:53.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.465+0000 7ff710a9f700 1 --2- 172.21.15.144:0/2753696841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c101440 0x7ff70c101810 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc009910 tx=0x7ff6fc009c20 comp rx=0 tx=0).stop 2024-09-10T12:02:53.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.466+0000 7ff710a9f700 1 -- 172.21.15.144:0/2753696841 shutdown_connections 2024-09-10T12:02:53.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.466+0000 7ff710a9f700 1 --2- 172.21.15.144:0/2753696841 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70c101d50 0x7ff70c10a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:53.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.466+0000 7ff710a9f700 1 --2- 172.21.15.144:0/2753696841 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c101440 0x7ff70c101810 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:53.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.466+0000 7ff710a9f700 1 -- 172.21.15.144:0/2753696841 >> 172.21.15.144:0/2753696841 conn(0x7ff70c0fac20 msgr2=0x7ff70c0fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:53.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.466+0000 7ff710a9f700 1 -- 172.21.15.144:0/2753696841 shutdown_connections 2024-09-10T12:02:53.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.466+0000 7ff710a9f700 1 -- 172.21.15.144:0/2753696841 wait complete. 2024-09-10T12:02:53.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.466+0000 7ff710a9f700 1 Processor -- start 2024-09-10T12:02:53.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.466+0000 7ff710a9f700 1 -- start start 2024-09-10T12:02:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff710a9f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c101440 0x7ff70c195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff710a9f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70c101d50 0x7ff70c196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff710a9f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c196960 con 0x7ff70c101440 2024-09-10T12:02:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff710a9f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c19a6f0 con 0x7ff70c101d50 2024-09-10T12:02:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff70a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c101440 0x7ff70c195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff70a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c101440 0x7ff70c195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36582/0 (socket says 172.21.15.144:36582) 2024-09-10T12:02:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff70a59c700 1 -- 172.21.15.144:0/2980813605 learned_addr learned my addr 172.21.15.144:0/2980813605 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:02:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff709d9b700 1 --2- 172.21.15.144:0/2980813605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70c101d50 0x7ff70c196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff70a59c700 1 -- 172.21.15.144:0/2980813605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70c101d50 msgr2=0x7ff70c196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:53.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff70a59c700 1 --2- 172.21.15.144:0/2980813605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70c101d50 0x7ff70c196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:53.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff70a59c700 1 -- 172.21.15.144:0/2980813605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6fc0095f0 con 0x7ff70c101440 2024-09-10T12:02:53.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff70a59c700 1 --2- 172.21.15.144:0/2980813605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c101440 0x7ff70c195d40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc006010 tx=0x7ff6fc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:53.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff6fb7fe700 1 -- 172.21.15.144:0/2980813605 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6fc01d070 con 0x7ff70c101440 2024-09-10T12:02:53.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff6fb7fe700 1 -- 172.21.15.144:0/2980813605 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff6fc00f460 con 0x7ff70c101440 2024-09-10T12:02:53.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff710a9f700 1 -- 172.21.15.144:0/2980813605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70c19a970 con 0x7ff70c101440 2024-09-10T12:02:53.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.467+0000 7ff6fb7fe700 1 -- 172.21.15.144:0/2980813605 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6fc017610 con 0x7ff70c101440 2024-09-10T12:02:53.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.468+0000 7ff710a9f700 1 -- 172.21.15.144:0/2980813605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70c19ae60 con 0x7ff70c101440 2024-09-10T12:02:53.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.469+0000 7ff710a9f700 1 -- 172.21.15.144:0/2980813605 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70c107920 con 0x7ff70c101440 2024-09-10T12:02:53.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.469+0000 7ff6fb7fe700 1 -- 172.21.15.144:0/2980813605 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff6fc00f5d0 con 0x7ff70c101440 2024-09-10T12:02:53.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.470+0000 7ff6fb7fe700 1 --2- 172.21.15.144:0/2980813605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff6f406e200 0x7ff6f40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:53.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.470+0000 7ff6fb7fe700 1 -- 172.21.15.144:0/2980813605 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff6fc08f630 con 0x7ff70c101440 2024-09-10T12:02:53.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.470+0000 7ff709d9b700 1 --2- 172.21.15.144:0/2980813605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff6f406e200 0x7ff6f40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:53.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.470+0000 7ff709d9b700 1 --2- 172.21.15.144:0/2980813605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff6f406e200 0x7ff6f40706b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff70c197360 tx=0x7ff70000a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:53.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.474+0000 7ff6fb7fe700 1 -- 172.21.15.144:0/2980813605 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff6fc058a10 con 0x7ff70c101440 2024-09-10T12:02:53.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.699+0000 7ff710a9f700 1 -- 172.21.15.144:0/2980813605 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff70c19b140 con 0x7ff6f406e200 2024-09-10T12:02:53.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.700+0000 7ff6fb7fe700 1 -- 172.21.15.144:0/2980813605 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7ff70c19b140 con 0x7ff6f406e200 2024-09-10T12:02:53.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.702+0000 7ff710a9f700 1 -- 172.21.15.144:0/2980813605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff6f406e200 msgr2=0x7ff6f40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:53.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.702+0000 7ff710a9f700 1 --2- 172.21.15.144:0/2980813605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff6f406e200 0x7ff6f40706b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff70c197360 tx=0x7ff70000a300 comp rx=0 tx=0).stop 2024-09-10T12:02:53.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.702+0000 7ff710a9f700 1 -- 172.21.15.144:0/2980813605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c101440 msgr2=0x7ff70c195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:53.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.702+0000 7ff710a9f700 1 --2- 172.21.15.144:0/2980813605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c101440 0x7ff70c195d40 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc006010 tx=0x7ff6fc00bd70 comp rx=0 tx=0).stop 2024-09-10T12:02:53.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.702+0000 7ff710a9f700 1 -- 172.21.15.144:0/2980813605 shutdown_connections 2024-09-10T12:02:53.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.702+0000 7ff710a9f700 1 --2- 172.21.15.144:0/2980813605 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff6f406e200 0x7ff6f40706b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:53.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.702+0000 7ff710a9f700 1 --2- 172.21.15.144:0/2980813605 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff70c101440 0x7ff70c195d40 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:53.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.702+0000 7ff710a9f700 1 --2- 172.21.15.144:0/2980813605 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff70c101d50 0x7ff70c196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:53.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.702+0000 7ff710a9f700 1 -- 172.21.15.144:0/2980813605 >> 172.21.15.144:0/2980813605 conn(0x7ff70c0fac20 msgr2=0x7ff70c104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:53.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.703+0000 7ff710a9f700 1 -- 172.21.15.144:0/2980813605 shutdown_connections 2024-09-10T12:02:53.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.703+0000 7ff710a9f700 1 -- 172.21.15.144:0/2980813605 wait complete. 2024-09-10T12:02:53.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.806+0000 7fb7baf65700 1 -- 172.21.15.144:0/607594985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b40ffc70 msgr2=0x7fb7b4103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:53.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.806+0000 7fb7baf65700 1 --2- 172.21.15.144:0/607594985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b40ffc70 0x7fb7b4103c40 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7fb7a0009910 tx=0x7fb7a0009c20 comp rx=0 tx=0).stop 2024-09-10T12:02:53.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.806+0000 7fb7baf65700 1 -- 172.21.15.144:0/607594985 shutdown_connections 2024-09-10T12:02:53.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.806+0000 7fb7baf65700 1 --2- 172.21.15.144:0/607594985 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b40ffc70 0x7fb7b4103c40 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:53.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.806+0000 7fb7baf65700 1 --2- 172.21.15.144:0/607594985 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7b40ff2d0 0x7fb7b40ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:53.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.806+0000 7fb7baf65700 1 -- 172.21.15.144:0/607594985 >> 172.21.15.144:0/607594985 conn(0x7fb7b40fabb0 msgr2=0x7fb7b40fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:53.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.806+0000 7fb7baf65700 1 -- 172.21.15.144:0/607594985 shutdown_connections 2024-09-10T12:02:53.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.806+0000 7fb7baf65700 1 -- 172.21.15.144:0/607594985 wait complete. 2024-09-10T12:02:53.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.807+0000 7fb7baf65700 1 Processor -- start 2024-09-10T12:02:53.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.807+0000 7fb7baf65700 1 -- start start 2024-09-10T12:02:53.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.807+0000 7fb7baf65700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b40ff2d0 0x7fb7b4100ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:53.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.807+0000 7fb7baf65700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7b40ffc70 0x7fb7b4101220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:53.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.807+0000 7fb7baf65700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b4101760 con 0x7fb7b40ff2d0 2024-09-10T12:02:53.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.807+0000 7fb7baf65700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7b41018a0 con 0x7fb7b40ffc70 2024-09-10T12:02:53.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.807+0000 7fb7b8d01700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b40ff2d0 0x7fb7b4100ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:53.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.807+0000 7fb7b8d01700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b40ff2d0 0x7fb7b4100ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36598/0 (socket says 172.21.15.144:36598) 2024-09-10T12:02:53.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.807+0000 7fb7b8d01700 1 -- 172.21.15.144:0/1823123765 learned_addr learned my addr 172.21.15.144:0/1823123765 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:02:53.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.807+0000 7fb7b3fff700 1 --2- 172.21.15.144:0/1823123765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7b40ffc70 0x7fb7b4101220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.808+0000 7fb7b8d01700 1 -- 172.21.15.144:0/1823123765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7b40ffc70 msgr2=0x7fb7b4101220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.808+0000 7fb7b8d01700 1 --2- 172.21.15.144:0/1823123765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7b40ffc70 0x7fb7b4101220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.808+0000 7fb7b8d01700 1 -- 172.21.15.144:0/1823123765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a00095f0 con 0x7fb7b40ff2d0 2024-09-10T12:02:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.808+0000 7fb7b3fff700 1 --2- 172.21.15.144:0/1823123765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7b40ffc70 0x7fb7b4101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:02:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.808+0000 7fb7b8d01700 1 --2- 172.21.15.144:0/1823123765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b40ff2d0 0x7fb7b4100ce0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800d8d0 tx=0x7fb7a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.808+0000 7fb7b1ffb700 1 -- 172.21.15.144:0/1823123765 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7a8009940 con 0x7fb7b40ff2d0 2024-09-10T12:02:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.808+0000 7fb7b1ffb700 1 -- 172.21.15.144:0/1823123765 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb7a8010460 con 0x7fb7b40ff2d0 2024-09-10T12:02:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.808+0000 7fb7baf65700 1 -- 172.21.15.144:0/1823123765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7b41a1f00 con 0x7fb7b40ff2d0 2024-09-10T12:02:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.808+0000 7fb7b1ffb700 1 -- 172.21.15.144:0/1823123765 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb7a800f5f0 con 0x7fb7b40ff2d0 2024-09-10T12:02:53.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.808+0000 7fb7baf65700 1 -- 172.21.15.144:0/1823123765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7b41a2310 con 0x7fb7b40ff2d0 2024-09-10T12:02:53.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.809+0000 7fb7baf65700 1 -- 172.21.15.144:0/1823123765 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b4108650 con 0x7fb7b40ff2d0 2024-09-10T12:02:53.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.809+0000 7fb7b1ffb700 1 -- 172.21.15.144:0/1823123765 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb7a80105d0 con 0x7fb7b40ff2d0 2024-09-10T12:02:53.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.810+0000 7fb7b1ffb700 1 --2- 172.21.15.144:0/1823123765 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb7a406e200 0x7fb7a40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:53.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.810+0000 7fb7b3fff700 1 --2- 172.21.15.144:0/1823123765 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb7a406e200 0x7fb7a40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:53.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.810+0000 7fb7b1ffb700 1 -- 172.21.15.144:0/1823123765 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb7a808df90 con 0x7fb7b40ff2d0 2024-09-10T12:02:53.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.810+0000 7fb7b3fff700 1 --2- 172.21.15.144:0/1823123765 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb7a406e200 0x7fb7a40706b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb7b41024d0 tx=0x7fb7a001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:53.815 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:53.814+0000 7fb7b1ffb700 1 -- 172.21.15.144:0/1823123765 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb7a80573f0 con 0x7fb7b40ff2d0 2024-09-10T12:02:54.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.029+0000 7fb7baf65700 1 -- 172.21.15.144:0/1823123765 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7b402cd90 con 0x7fb7a406e200 2024-09-10T12:02:54.040 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.039+0000 7fb7b1ffb700 1 -- 172.21.15.144:0/1823123765 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fb7b402cd90 con 0x7fb7a406e200 2024-09-10T12:02:54.041 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:02:54.041 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (31m) 9m ago 41m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:02:54.041 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (41m) 9m ago 41m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:02:54.041 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (40m) 9m ago 40m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:02:54.041 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (28m) 9m ago 41m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:02:54.041 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (28m) 9m ago 40m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:02:54.041 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (30m) 9m ago 41m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:02:54.041 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (25m) 9m ago 36m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:02:54.041 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (24m) 9m ago 36m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:02:54.042 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (9m) 9m ago 36m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:02:54.042 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (24m) 9m ago 36m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:02:54.042 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (32m) 9m ago 43m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:02:54.042 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (32m) 9m ago 40m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:02:54.042 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (29m) 9m ago 43m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:02:54.042 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (29m) 9m ago 40m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:02:54.042 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (31m) 9m ago 41m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:02:54.042 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (31m) 9m ago 40m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:02:54.042 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (28m) 9m ago 39m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:02:54.042 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (27m) 9m ago 39m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:02:54.042 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (27m) 9m ago 38m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:02:54.042 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (26m) 9m ago 38m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:02:54.042 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (26m) 9m ago 38m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:02:54.043 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (25m) 9m ago 37m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:02:54.043 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (31m) 9m ago 41m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:02:54.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.041+0000 7fb7baf65700 1 -- 172.21.15.144:0/1823123765 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb7a406e200 msgr2=0x7fb7a40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:54.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.041+0000 7fb7baf65700 1 --2- 172.21.15.144:0/1823123765 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb7a406e200 0x7fb7a40706b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb7b41024d0 tx=0x7fb7a001a040 comp rx=0 tx=0).stop 2024-09-10T12:02:54.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.041+0000 7fb7baf65700 1 -- 172.21.15.144:0/1823123765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b40ff2d0 msgr2=0x7fb7b4100ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:54.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.041+0000 7fb7baf65700 1 --2- 172.21.15.144:0/1823123765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b40ff2d0 0x7fb7b4100ce0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fb7a800d8d0 tx=0x7fb7a800dc90 comp rx=0 tx=0).stop 2024-09-10T12:02:54.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.041+0000 7fb7baf65700 1 -- 172.21.15.144:0/1823123765 shutdown_connections 2024-09-10T12:02:54.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.041+0000 7fb7baf65700 1 --2- 172.21.15.144:0/1823123765 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb7a406e200 0x7fb7a40706b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.041+0000 7fb7baf65700 1 --2- 172.21.15.144:0/1823123765 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb7b40ff2d0 0x7fb7b4100ce0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.042+0000 7fb7baf65700 1 --2- 172.21.15.144:0/1823123765 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb7b40ffc70 0x7fb7b4101220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.042+0000 7fb7baf65700 1 -- 172.21.15.144:0/1823123765 >> 172.21.15.144:0/1823123765 conn(0x7fb7b40fabb0 msgr2=0x7fb7b4068f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:54.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.042+0000 7fb7baf65700 1 -- 172.21.15.144:0/1823123765 shutdown_connections 2024-09-10T12:02:54.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.042+0000 7fb7baf65700 1 -- 172.21.15.144:0/1823123765 wait complete. 2024-09-10T12:02:54.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.146+0000 7fec5706b700 1 -- 172.21.15.144:0/2628005696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec50100d80 msgr2=0x7fec50104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:54.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.146+0000 7fec5706b700 1 --2- 172.21.15.144:0/2628005696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec50100d80 0x7fec50104d50 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009910 tx=0x7fec3c009c20 comp rx=0 tx=0).stop 2024-09-10T12:02:54.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.146+0000 7fec5706b700 1 -- 172.21.15.144:0/2628005696 shutdown_connections 2024-09-10T12:02:54.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.146+0000 7fec5706b700 1 --2- 172.21.15.144:0/2628005696 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec50100d80 0x7fec50104d50 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.146+0000 7fec5706b700 1 --2- 172.21.15.144:0/2628005696 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec501003e0 0x7fec501007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.146+0000 7fec5706b700 1 -- 172.21.15.144:0/2628005696 >> 172.21.15.144:0/2628005696 conn(0x7fec500fbca0 msgr2=0x7fec500fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:54.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.146+0000 7fec5706b700 1 -- 172.21.15.144:0/2628005696 shutdown_connections 2024-09-10T12:02:54.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.146+0000 7fec5706b700 1 -- 172.21.15.144:0/2628005696 wait complete. 2024-09-10T12:02:54.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.147+0000 7fec5706b700 1 Processor -- start 2024-09-10T12:02:54.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.147+0000 7fec5706b700 1 -- start start 2024-09-10T12:02:54.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.147+0000 7fec5706b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec501003e0 0x7fec50073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:54.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.147+0000 7fec5706b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec50100d80 0x7fec50073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:54.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.147+0000 7fec5706b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec50073ad0 con 0x7fec501003e0 2024-09-10T12:02:54.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.147+0000 7fec5706b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec50073c10 con 0x7fec50100d80 2024-09-10T12:02:54.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.147+0000 7fec54e07700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec501003e0 0x7fec50073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:54.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.147+0000 7fec54e07700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec501003e0 0x7fec50073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36624/0 (socket says 172.21.15.144:36624) 2024-09-10T12:02:54.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.147+0000 7fec54e07700 1 -- 172.21.15.144:0/167478031 learned_addr learned my addr 172.21.15.144:0/167478031 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:02:54.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.148+0000 7fec4ffff700 1 --2- 172.21.15.144:0/167478031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec50100d80 0x7fec50073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:54.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.148+0000 7fec54e07700 1 -- 172.21.15.144:0/167478031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec50100d80 msgr2=0x7fec50073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:54.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.148+0000 7fec54e07700 1 --2- 172.21.15.144:0/167478031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec50100d80 0x7fec50073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.148+0000 7fec54e07700 1 -- 172.21.15.144:0/167478031 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c0095f0 con 0x7fec501003e0 2024-09-10T12:02:54.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.148+0000 7fec4ffff700 1 --2- 172.21.15.144:0/167478031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec50100d80 0x7fec50073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:02:54.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.148+0000 7fec54e07700 1 --2- 172.21.15.144:0/167478031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec501003e0 0x7fec50073050 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fec4400d8d0 tx=0x7fec4400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:54.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.148+0000 7fec4dffb700 1 -- 172.21.15.144:0/167478031 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec44009940 con 0x7fec501003e0 2024-09-10T12:02:54.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.148+0000 7fec4dffb700 1 -- 172.21.15.144:0/167478031 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fec44010460 con 0x7fec501003e0 2024-09-10T12:02:54.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.148+0000 7fec5706b700 1 -- 172.21.15.144:0/167478031 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec501a1fa0 con 0x7fec501003e0 2024-09-10T12:02:54.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.148+0000 7fec4dffb700 1 -- 172.21.15.144:0/167478031 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec4400f5f0 con 0x7fec501003e0 2024-09-10T12:02:54.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.148+0000 7fec5706b700 1 -- 172.21.15.144:0/167478031 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec501a23b0 con 0x7fec501003e0 2024-09-10T12:02:54.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.149+0000 7fec5706b700 1 -- 172.21.15.144:0/167478031 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5004ea50 con 0x7fec501003e0 2024-09-10T12:02:54.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.150+0000 7fec4dffb700 1 -- 172.21.15.144:0/167478031 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fec440105d0 con 0x7fec501003e0 2024-09-10T12:02:54.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.150+0000 7fec4dffb700 1 --2- 172.21.15.144:0/167478031 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec4006e200 0x7fec400706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:54.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.150+0000 7fec4dffb700 1 -- 172.21.15.144:0/167478031 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fec4408e0a0 con 0x7fec501003e0 2024-09-10T12:02:54.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.150+0000 7fec4ffff700 1 --2- 172.21.15.144:0/167478031 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec4006e200 0x7fec400706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:54.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.150+0000 7fec4ffff700 1 --2- 172.21.15.144:0/167478031 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec4006e200 0x7fec400706b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00b5c0 tx=0x7fec3c0053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:54.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.154+0000 7fec4dffb700 1 -- 172.21.15.144:0/167478031 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fec44057480 con 0x7fec501003e0 2024-09-10T12:02:54.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.446+0000 7fec5706b700 1 -- 172.21.15.144:0/167478031 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fec501a2730 con 0x7fec501003e0 2024-09-10T12:02:54.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.447+0000 7fec4dffb700 1 -- 172.21.15.144:0/167478031 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fec4405ab40 con 0x7fec501003e0 2024-09-10T12:02:54.448 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:02:54.448 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:02:54.448 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:02:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:02:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:02:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:02:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:02:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:02:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:02:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:02:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:02:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:02:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:02:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:02:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:02:54.449 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:02:54.449 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:02:54.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.449+0000 7fec5706b700 1 -- 172.21.15.144:0/167478031 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec4006e200 msgr2=0x7fec400706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:54.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.449+0000 7fec5706b700 1 --2- 172.21.15.144:0/167478031 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec4006e200 0x7fec400706b0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00b5c0 tx=0x7fec3c0053a0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.449+0000 7fec5706b700 1 -- 172.21.15.144:0/167478031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec501003e0 msgr2=0x7fec50073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:54.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.449+0000 7fec5706b700 1 --2- 172.21.15.144:0/167478031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec501003e0 0x7fec50073050 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fec4400d8d0 tx=0x7fec4400dc90 comp rx=0 tx=0).stop 2024-09-10T12:02:54.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.449+0000 7fec5706b700 1 -- 172.21.15.144:0/167478031 shutdown_connections 2024-09-10T12:02:54.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.449+0000 7fec5706b700 1 --2- 172.21.15.144:0/167478031 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fec4006e200 0x7fec400706b0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.449+0000 7fec5706b700 1 --2- 172.21.15.144:0/167478031 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fec501003e0 0x7fec50073050 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.449+0000 7fec5706b700 1 --2- 172.21.15.144:0/167478031 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fec50100d80 0x7fec50073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.449+0000 7fec5706b700 1 -- 172.21.15.144:0/167478031 >> 172.21.15.144:0/167478031 conn(0x7fec500fbca0 msgr2=0x7fec500fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:54.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.449+0000 7fec5706b700 1 -- 172.21.15.144:0/167478031 shutdown_connections 2024-09-10T12:02:54.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.449+0000 7fec5706b700 1 -- 172.21.15.144:0/167478031 wait complete. 2024-09-10T12:02:54.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.554+0000 7f6338fca700 1 -- 172.21.15.144:0/3309482946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63340ff320 msgr2=0x7f63340ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:54.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.554+0000 7f6338fca700 1 --2- 172.21.15.144:0/3309482946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63340ff320 0x7f63340ff6f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f632c00b0d0 tx=0x7f632c00b3e0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.554+0000 7f6338fca700 1 -- 172.21.15.144:0/3309482946 shutdown_connections 2024-09-10T12:02:54.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.554+0000 7f6338fca700 1 --2- 172.21.15.144:0/3309482946 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63340ffc30 0x7f633410a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.554+0000 7f6338fca700 1 --2- 172.21.15.144:0/3309482946 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63340ff320 0x7f63340ff6f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.554+0000 7f6338fca700 1 -- 172.21.15.144:0/3309482946 >> 172.21.15.144:0/3309482946 conn(0x7f63340748b0 msgr2=0x7f6334074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:54.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.554+0000 7f6338fca700 1 -- 172.21.15.144:0/3309482946 shutdown_connections 2024-09-10T12:02:54.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.554+0000 7f6338fca700 1 -- 172.21.15.144:0/3309482946 wait complete. 2024-09-10T12:02:54.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.555+0000 7f6338fca700 1 Processor -- start 2024-09-10T12:02:54.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.555+0000 7f6338fca700 1 -- start start 2024-09-10T12:02:54.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.555+0000 7f6338fca700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63340ff320 0x7f6334195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:54.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.555+0000 7f6338fca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63340ffc30 0x7f63341961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:54.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.555+0000 7f6338fca700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6334196870 con 0x7f63340ffc30 2024-09-10T12:02:54.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.555+0000 7f6338fca700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63341995a0 con 0x7f63340ff320 2024-09-10T12:02:54.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.555+0000 7f6331d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63340ffc30 0x7f63341961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:54.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.555+0000 7f6331d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63340ffc30 0x7f63341961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36648/0 (socket says 172.21.15.144:36648) 2024-09-10T12:02:54.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.555+0000 7f6331d9b700 1 -- 172.21.15.144:0/302729292 learned_addr learned my addr 172.21.15.144:0/302729292 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:02:54.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.556+0000 7f633259c700 1 --2- 172.21.15.144:0/302729292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63340ff320 0x7f6334195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:54.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.556+0000 7f6331d9b700 1 -- 172.21.15.144:0/302729292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63340ff320 msgr2=0x7f6334195ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:54.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.556+0000 7f6331d9b700 1 --2- 172.21.15.144:0/302729292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63340ff320 0x7f6334195ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.556+0000 7f6331d9b700 1 -- 172.21.15.144:0/302729292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f632c009d00 con 0x7f63340ffc30 2024-09-10T12:02:54.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.556+0000 7f6331d9b700 1 --2- 172.21.15.144:0/302729292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63340ffc30 0x7f63341961e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f631c00d900 tx=0x7f631c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:54.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.556+0000 7f632b7fe700 1 -- 172.21.15.144:0/302729292 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f631c004a00 con 0x7f63340ffc30 2024-09-10T12:02:54.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.556+0000 7f632b7fe700 1 -- 172.21.15.144:0/302729292 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f631c0054e0 con 0x7f63340ffc30 2024-09-10T12:02:54.559 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.556+0000 7f6338fca700 1 -- 172.21.15.144:0/302729292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6334199880 con 0x7f63340ffc30 2024-09-10T12:02:54.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.556+0000 7f632b7fe700 1 -- 172.21.15.144:0/302729292 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f631c009d60 con 0x7f63340ffc30 2024-09-10T12:02:54.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.556+0000 7f6338fca700 1 -- 172.21.15.144:0/302729292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6334199dd0 con 0x7f63340ffc30 2024-09-10T12:02:54.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.557+0000 7f6338fca700 1 -- 172.21.15.144:0/302729292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6334107da0 con 0x7f63340ffc30 2024-09-10T12:02:54.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.558+0000 7f632b7fe700 1 -- 172.21.15.144:0/302729292 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f631c005020 con 0x7f63340ffc30 2024-09-10T12:02:54.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.558+0000 7f632b7fe700 1 --2- 172.21.15.144:0/302729292 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f632006e200 0x7f63200706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:54.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.558+0000 7f632b7fe700 1 -- 172.21.15.144:0/302729292 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f631c08e300 con 0x7f63340ffc30 2024-09-10T12:02:54.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.558+0000 7f633259c700 1 --2- 172.21.15.144:0/302729292 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f632006e200 0x7f63200706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:54.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.558+0000 7f633259c700 1 --2- 172.21.15.144:0/302729292 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f632006e200 0x7f63200706b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f632c00b820 tx=0x7f632c0092e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:54.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.562+0000 7f632b7fe700 1 -- 172.21.15.144:0/302729292 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f631c057680 con 0x7f63340ffc30 2024-09-10T12:02:54.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.806+0000 7f6338fca700 1 -- 172.21.15.144:0/302729292 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f633404ea50 con 0x7f63340ffc30 2024-09-10T12:02:54.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.807+0000 7f632b7fe700 1 -- 172.21.15.144:0/302729292 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f631c05ad40 con 0x7f63340ffc30 2024-09-10T12:02:54.808 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:02:54.808 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:02:54.808 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:02:54.809 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:02:54.809 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:02:54.809 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:02:54.809 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:02:54.809 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:02:54.809 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:02:54.809 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:02:54.809 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:02:54.809 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:02:54.809 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:02:54.809 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:02:54.810 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:02:54.810 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:02:54.810 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:02:54.810 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:02:54.810 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:02:54.810 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:02:54.810 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:02:54.810 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:02:54.810 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:02:54.810 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:02:54.810 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:02:54.811 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:02:54.811 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:02:54.811 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:02:54.811 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:02:54.811 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:02:54.811 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:02:54.811 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:02:54.811 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:02:54.811 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:02:54.812 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:02:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:02:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:02:54.812 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:02:54.812 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:02:54.812 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:02:54.812 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:02:54.812 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:02:54.812 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:02:54.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.810+0000 7f6338fca700 1 -- 172.21.15.144:0/302729292 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f632006e200 msgr2=0x7f63200706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:54.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.810+0000 7f6338fca700 1 --2- 172.21.15.144:0/302729292 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f632006e200 0x7f63200706b0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f632c00b820 tx=0x7f632c0092e0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.810+0000 7f6338fca700 1 -- 172.21.15.144:0/302729292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63340ffc30 msgr2=0x7f63341961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:54.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.810+0000 7f6338fca700 1 --2- 172.21.15.144:0/302729292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63340ffc30 0x7f63341961e0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f631c00d900 tx=0x7f631c00dcc0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.810+0000 7f6338fca700 1 -- 172.21.15.144:0/302729292 shutdown_connections 2024-09-10T12:02:54.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.810+0000 7f6338fca700 1 --2- 172.21.15.144:0/302729292 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f632006e200 0x7f63200706b0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.810+0000 7f6338fca700 1 --2- 172.21.15.144:0/302729292 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f63340ff320 0x7f6334195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.810+0000 7f6338fca700 1 --2- 172.21.15.144:0/302729292 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f63340ffc30 0x7f63341961e0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.810+0000 7f6338fca700 1 -- 172.21.15.144:0/302729292 >> 172.21.15.144:0/302729292 conn(0x7f63340748b0 msgr2=0x7f6334104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:54.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.810+0000 7f6338fca700 1 -- 172.21.15.144:0/302729292 shutdown_connections 2024-09-10T12:02:54.814 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.810+0000 7f6338fca700 1 -- 172.21.15.144:0/302729292 wait complete. 2024-09-10T12:02:54.814 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:02:54.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.917+0000 7fde82ad6700 1 -- 172.21.15.144:0/3286676074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c100cb0 msgr2=0x7fde7c109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:54.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.917+0000 7fde82ad6700 1 --2- 172.21.15.144:0/3286676074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c100cb0 0x7fde7c109410 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fde68009910 tx=0x7fde68009c20 comp rx=0 tx=0).stop 2024-09-10T12:02:54.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.918+0000 7fde82ad6700 1 -- 172.21.15.144:0/3286676074 shutdown_connections 2024-09-10T12:02:54.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.918+0000 7fde82ad6700 1 --2- 172.21.15.144:0/3286676074 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c100cb0 0x7fde7c109410 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.918+0000 7fde82ad6700 1 --2- 172.21.15.144:0/3286676074 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde7c1003a0 0x7fde7c100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.918+0000 7fde82ad6700 1 -- 172.21.15.144:0/3286676074 >> 172.21.15.144:0/3286676074 conn(0x7fde7c074850 msgr2=0x7fde7c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:54.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.918+0000 7fde82ad6700 1 -- 172.21.15.144:0/3286676074 shutdown_connections 2024-09-10T12:02:54.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.918+0000 7fde82ad6700 1 -- 172.21.15.144:0/3286676074 wait complete. 2024-09-10T12:02:54.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.918+0000 7fde82ad6700 1 Processor -- start 2024-09-10T12:02:54.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.918+0000 7fde82ad6700 1 -- start start 2024-09-10T12:02:54.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.918+0000 7fde82ad6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde7c1003a0 0x7fde7c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:54.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.918+0000 7fde82ad6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c100cb0 0x7fde7c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:54.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.918+0000 7fde82ad6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c198ac0 con 0x7fde7c100cb0 2024-09-10T12:02:54.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.918+0000 7fde82ad6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde7c19c850 con 0x7fde7c1003a0 2024-09-10T12:02:54.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.919+0000 7fde7bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c100cb0 0x7fde7c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:54.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.919+0000 7fde7bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c100cb0 0x7fde7c1983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36670/0 (socket says 172.21.15.144:36670) 2024-09-10T12:02:54.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.919+0000 7fde7bfff700 1 -- 172.21.15.144:0/1242664829 learned_addr learned my addr 172.21.15.144:0/1242664829 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:02:54.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.919+0000 7fde80872700 1 --2- 172.21.15.144:0/1242664829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde7c1003a0 0x7fde7c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:54.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.919+0000 7fde7bfff700 1 -- 172.21.15.144:0/1242664829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde7c1003a0 msgr2=0x7fde7c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:54.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.919+0000 7fde7bfff700 1 --2- 172.21.15.144:0/1242664829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde7c1003a0 0x7fde7c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:54.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.919+0000 7fde7bfff700 1 -- 172.21.15.144:0/1242664829 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde680095f0 con 0x7fde7c100cb0 2024-09-10T12:02:54.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.919+0000 7fde7bfff700 1 --2- 172.21.15.144:0/1242664829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c100cb0 0x7fde7c1983e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fde68006010 tx=0x7fde6800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:54.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.919+0000 7fde79ffb700 1 -- 172.21.15.144:0/1242664829 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde6801d070 con 0x7fde7c100cb0 2024-09-10T12:02:54.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.919+0000 7fde82ad6700 1 -- 172.21.15.144:0/1242664829 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde7c19cad0 con 0x7fde7c100cb0 2024-09-10T12:02:54.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.919+0000 7fde79ffb700 1 -- 172.21.15.144:0/1242664829 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fde68005dc0 con 0x7fde7c100cb0 2024-09-10T12:02:54.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.919+0000 7fde79ffb700 1 -- 172.21.15.144:0/1242664829 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fde68017400 con 0x7fde7c100cb0 2024-09-10T12:02:54.923 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.920+0000 7fde82ad6700 1 -- 172.21.15.144:0/1242664829 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde7c19d040 con 0x7fde7c100cb0 2024-09-10T12:02:54.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.920+0000 7fde82ad6700 1 -- 172.21.15.144:0/1242664829 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde7c04ea50 con 0x7fde7c100cb0 2024-09-10T12:02:54.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.921+0000 7fde79ffb700 1 -- 172.21.15.144:0/1242664829 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fde68003a60 con 0x7fde7c100cb0 2024-09-10T12:02:54.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.921+0000 7fde79ffb700 1 --2- 172.21.15.144:0/1242664829 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fde6c06e200 0x7fde6c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:54.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.921+0000 7fde79ffb700 1 -- 172.21.15.144:0/1242664829 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fde6808f450 con 0x7fde7c100cb0 2024-09-10T12:02:54.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.921+0000 7fde80872700 1 --2- 172.21.15.144:0/1242664829 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fde6c06e200 0x7fde6c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:54.924 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.922+0000 7fde80872700 1 --2- 172.21.15.144:0/1242664829 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fde6c06e200 0x7fde6c0706b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fde70005950 tx=0x7fde7000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:54.927 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:54.926+0000 7fde79ffb700 1 -- 172.21.15.144:0/1242664829 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fde680588b0 con 0x7fde7c100cb0 2024-09-10T12:02:55.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.149+0000 7fde82ad6700 1 -- 172.21.15.144:0/1242664829 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fde7c19d4a0 con 0x7fde6c06e200 2024-09-10T12:02:55.151 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:54 smithi144 ceph-mon[87436]: pgmap v928: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:55.151 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:54 smithi144 ceph-mon[87436]: from='client.35688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:55.151 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:54 smithi144 ceph-mon[87436]: from='client.35692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:55.151 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:54 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/167478031' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:02:55.151 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:54 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/302729292' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:02:55.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.150+0000 7fde79ffb700 1 -- 172.21.15.144:0/1242664829 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fde7c19d4a0 con 0x7fde6c06e200 2024-09-10T12:02:55.152 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:02:55.152 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:02:55.152 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:02:55.152 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:02:55.152 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:02:55.152 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:02:55.152 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:02:55.153 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:02:55.153 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:02:55.153 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:02:55.153 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:02:55.153 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:02:55.153 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:02:55.153 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:02:55.153 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:02:55.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.153+0000 7fde82ad6700 1 -- 172.21.15.144:0/1242664829 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fde6c06e200 msgr2=0x7fde6c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:55.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.153+0000 7fde82ad6700 1 --2- 172.21.15.144:0/1242664829 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fde6c06e200 0x7fde6c0706b0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fde70005950 tx=0x7fde7000b410 comp rx=0 tx=0).stop 2024-09-10T12:02:55.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.153+0000 7fde82ad6700 1 -- 172.21.15.144:0/1242664829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c100cb0 msgr2=0x7fde7c1983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:55.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.153+0000 7fde82ad6700 1 --2- 172.21.15.144:0/1242664829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c100cb0 0x7fde7c1983e0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fde68006010 tx=0x7fde6800bd70 comp rx=0 tx=0).stop 2024-09-10T12:02:55.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.153+0000 7fde82ad6700 1 -- 172.21.15.144:0/1242664829 shutdown_connections 2024-09-10T12:02:55.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.153+0000 7fde82ad6700 1 --2- 172.21.15.144:0/1242664829 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fde6c06e200 0x7fde6c0706b0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:55.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.153+0000 7fde82ad6700 1 --2- 172.21.15.144:0/1242664829 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fde7c1003a0 0x7fde7c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:55.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.153+0000 7fde82ad6700 1 --2- 172.21.15.144:0/1242664829 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fde7c100cb0 0x7fde7c1983e0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:55.155 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.153+0000 7fde82ad6700 1 -- 172.21.15.144:0/1242664829 >> 172.21.15.144:0/1242664829 conn(0x7fde7c074850 msgr2=0x7fde7c107cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:55.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.153+0000 7fde82ad6700 1 -- 172.21.15.144:0/1242664829 shutdown_connections 2024-09-10T12:02:55.156 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.153+0000 7fde82ad6700 1 -- 172.21.15.144:0/1242664829 wait complete. 2024-09-10T12:02:55.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.258+0000 7f297988c700 1 -- 172.21.15.144:0/233334625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2974103860 msgr2=0x7f2974103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:55.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.258+0000 7f297988c700 1 --2- 172.21.15.144:0/233334625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2974103860 0x7f2974103c30 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f2964009910 tx=0x7f2964009c20 comp rx=0 tx=0).stop 2024-09-10T12:02:55.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.258+0000 7f297988c700 1 -- 172.21.15.144:0/233334625 shutdown_connections 2024-09-10T12:02:55.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.258+0000 7f297988c700 1 --2- 172.21.15.144:0/233334625 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29740ff320 0x7f29740ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:55.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.258+0000 7f297988c700 1 --2- 172.21.15.144:0/233334625 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2974103860 0x7f2974103c30 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:55.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.258+0000 7f297988c700 1 -- 172.21.15.144:0/233334625 >> 172.21.15.144:0/233334625 conn(0x7f29740748b0 msgr2=0x7f2974074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:55.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.258+0000 7f297988c700 1 -- 172.21.15.144:0/233334625 shutdown_connections 2024-09-10T12:02:55.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.258+0000 7f297988c700 1 -- 172.21.15.144:0/233334625 wait complete. 2024-09-10T12:02:55.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.258+0000 7f297988c700 1 Processor -- start 2024-09-10T12:02:55.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.259+0000 7f297988c700 1 -- start start 2024-09-10T12:02:55.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.259+0000 7f297988c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29740ff320 0x7f2974100510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:55.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.259+0000 7f297988c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2974103860 0x7f2974100a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:55.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.259+0000 7f297988c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2974100f90 con 0x7f29740ff320 2024-09-10T12:02:55.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.259+0000 7f297988c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29741010d0 con 0x7f2974103860 2024-09-10T12:02:55.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.259+0000 7f2972ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29740ff320 0x7f2974100510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:55.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.259+0000 7f2972ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29740ff320 0x7f2974100510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36688/0 (socket says 172.21.15.144:36688) 2024-09-10T12:02:55.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.259+0000 7f2972ffd700 1 -- 172.21.15.144:0/2032226125 learned_addr learned my addr 172.21.15.144:0/2032226125 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:02:55.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.259+0000 7f296a7fc700 1 --2- 172.21.15.144:0/2032226125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2974103860 0x7f2974100a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:55.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.259+0000 7f2972ffd700 1 -- 172.21.15.144:0/2032226125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2974103860 msgr2=0x7f2974100a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:55.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.259+0000 7f2972ffd700 1 --2- 172.21.15.144:0/2032226125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2974103860 0x7f2974100a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:55.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.259+0000 7f2972ffd700 1 -- 172.21.15.144:0/2032226125 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29640095f0 con 0x7f29740ff320 2024-09-10T12:02:55.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.259+0000 7f296a7fc700 1 --2- 172.21.15.144:0/2032226125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2974103860 0x7f2974100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:02:55.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.260+0000 7f2972ffd700 1 --2- 172.21.15.144:0/2032226125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29740ff320 0x7f2974100510 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2964006010 tx=0x7f296400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:55.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.260+0000 7f297888a700 1 -- 172.21.15.144:0/2032226125 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f296401d070 con 0x7f29740ff320 2024-09-10T12:02:55.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.260+0000 7f297888a700 1 -- 172.21.15.144:0/2032226125 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2964004fb0 con 0x7f29740ff320 2024-09-10T12:02:55.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.260+0000 7f297988c700 1 -- 172.21.15.144:0/2032226125 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2974101350 con 0x7f29740ff320 2024-09-10T12:02:55.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.260+0000 7f297888a700 1 -- 172.21.15.144:0/2032226125 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2964022ea0 con 0x7f29740ff320 2024-09-10T12:02:55.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.260+0000 7f297988c700 1 -- 172.21.15.144:0/2032226125 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29741a2270 con 0x7f29740ff320 2024-09-10T12:02:55.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.261+0000 7f297988c700 1 -- 172.21.15.144:0/2032226125 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2974108e30 con 0x7f29740ff320 2024-09-10T12:02:55.264 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.261+0000 7f297888a700 1 -- 172.21.15.144:0/2032226125 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2964022470 con 0x7f29740ff320 2024-09-10T12:02:55.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.262+0000 7f297888a700 1 --2- 172.21.15.144:0/2032226125 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f296006e200 0x7f29600706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:02:55.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.262+0000 7f297888a700 1 -- 172.21.15.144:0/2032226125 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f296408f5d0 con 0x7f29740ff320 2024-09-10T12:02:55.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.262+0000 7f296a7fc700 1 --2- 172.21.15.144:0/2032226125 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f296006e200 0x7f29600706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:02:55.265 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.262+0000 7f296a7fc700 1 --2- 172.21.15.144:0/2032226125 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f296006e200 0x7f29600706b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f295c005fd0 tx=0x7f295c005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:02:55.267 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.266+0000 7f297888a700 1 -- 172.21.15.144:0/2032226125 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2964058a60 con 0x7f29740ff320 2024-09-10T12:02:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:54 smithi178 ceph-mon[65940]: pgmap v928: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:54 smithi178 ceph-mon[65940]: from='client.35688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:54 smithi178 ceph-mon[65940]: from='client.35692 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:54 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/167478031' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:02:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:54 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/302729292' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:02:55.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.552+0000 7f297988c700 1 -- 172.21.15.144:0/2032226125 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f297404ea50 con 0x7f29740ff320 2024-09-10T12:02:55.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.552+0000 7f297888a700 1 -- 172.21.15.144:0/2032226125 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f296405c120 con 0x7f29740ff320 2024-09-10T12:02:55.554 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:02:55.554 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:02:55.554 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:02:55.554 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:02:55.555 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:02:55.555 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:02:55.555 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:02:55.555 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:02:55.555 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:02:55.555 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:02:55.555 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:02:55.555 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:02:55.555 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:02:55.555 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:02:55.555 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:02:55.555 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:02:55.555 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:02:55.556 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:02:55.556 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:02:55.556 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:02:55.556 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:02:55.556 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:02:55.556 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:02:55.556 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:02:55.556 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:02:55.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.555+0000 7f297988c700 1 -- 172.21.15.144:0/2032226125 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f296006e200 msgr2=0x7f29600706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:55.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.555+0000 7f297988c700 1 --2- 172.21.15.144:0/2032226125 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f296006e200 0x7f29600706b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f295c005fd0 tx=0x7f295c005dc0 comp rx=0 tx=0).stop 2024-09-10T12:02:55.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.555+0000 7f297988c700 1 -- 172.21.15.144:0/2032226125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29740ff320 msgr2=0x7f2974100510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:02:55.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.555+0000 7f297988c700 1 --2- 172.21.15.144:0/2032226125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29740ff320 0x7f2974100510 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f2964006010 tx=0x7f296400be20 comp rx=0 tx=0).stop 2024-09-10T12:02:55.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.555+0000 7f297988c700 1 -- 172.21.15.144:0/2032226125 shutdown_connections 2024-09-10T12:02:55.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.555+0000 7f297988c700 1 --2- 172.21.15.144:0/2032226125 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f296006e200 0x7f29600706b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:55.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.555+0000 7f297988c700 1 --2- 172.21.15.144:0/2032226125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29740ff320 0x7f2974100510 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:55.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.555+0000 7f297988c700 1 --2- 172.21.15.144:0/2032226125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2974103860 0x7f2974100a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:02:55.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.555+0000 7f297988c700 1 -- 172.21.15.144:0/2032226125 >> 172.21.15.144:0/2032226125 conn(0x7f29740748b0 msgr2=0x7f29740fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:02:55.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.555+0000 7f297988c700 1 -- 172.21.15.144:0/2032226125 shutdown_connections 2024-09-10T12:02:55.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:02:55.555+0000 7f297988c700 1 -- 172.21.15.144:0/2032226125 wait complete. 2024-09-10T12:02:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:55 smithi144 ceph-mon[87436]: from='client.35696 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:55 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2032226125' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:02:56.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:55 smithi178 ceph-mon[65940]: from='client.35696 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:56.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:55 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2032226125' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:02:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:56 smithi144 ceph-mon[87436]: pgmap v929: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:56 smithi144 ceph-mon[87436]: from='client.35708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:56 smithi178 ceph-mon[65940]: pgmap v929: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:56 smithi178 ceph-mon[65940]: from='client.35708 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:02:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:58 smithi144 ceph-mon[87436]: pgmap v930: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:02:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:02:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:58 smithi178 ceph-mon[65940]: pgmap v930: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:02:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:02:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:03:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:00 smithi144 ceph-mon[87436]: pgmap v931: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:00 smithi178 ceph-mon[65940]: pgmap v931: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:02 smithi178 ceph-mon[65940]: pgmap v932: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:03.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:02 smithi144 ceph-mon[87436]: pgmap v932: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:05.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:04 smithi144 ceph-mon[87436]: pgmap v933: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:04 smithi178 ceph-mon[65940]: pgmap v933: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:06 smithi144 ceph-mon[87436]: pgmap v934: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:06 smithi178 ceph-mon[65940]: pgmap v934: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:08 smithi144 ceph-mon[87436]: pgmap v935: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:08 smithi178 ceph-mon[65940]: pgmap v935: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:11.169 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:10 smithi178 ceph-mon[65940]: pgmap v936: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:11.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:10 smithi144 ceph-mon[87436]: pgmap v936: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:12 smithi178 ceph-mon[65940]: pgmap v937: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:13.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:12 smithi144 ceph-mon[87436]: pgmap v937: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:14.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:03:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:03:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:14 smithi144 ceph-mon[87436]: pgmap v938: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:14 smithi178 ceph-mon[65940]: pgmap v938: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:17.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:16 smithi144 ceph-mon[87436]: pgmap v939: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:16 smithi178 ceph-mon[65940]: pgmap v939: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:19.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:18 smithi178 ceph-mon[65940]: pgmap v940: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:19.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:18 smithi144 ceph-mon[87436]: pgmap v940: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:20 smithi178 ceph-mon[65940]: pgmap v941: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:21.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:20 smithi144 ceph-mon[87436]: pgmap v941: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:23.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:22 smithi178 ceph-mon[65940]: pgmap v942: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:23.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:22 smithi144 ceph-mon[87436]: pgmap v942: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:24 smithi178 ceph-mon[65940]: pgmap v943: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:25.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:24 smithi144 ceph-mon[87436]: pgmap v943: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:25.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.674+0000 7f61ec0bd700 1 -- 172.21.15.144:0/64765650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e40ffd20 msgr2=0x7f61e4103cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:25.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.674+0000 7f61ec0bd700 1 --2- 172.21.15.144:0/64765650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e40ffd20 0x7f61e4103cf0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f61d4009910 tx=0x7f61d4009c20 comp rx=0 tx=0).stop 2024-09-10T12:03:25.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.674+0000 7f61ec0bd700 1 -- 172.21.15.144:0/64765650 shutdown_connections 2024-09-10T12:03:25.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.674+0000 7f61ec0bd700 1 --2- 172.21.15.144:0/64765650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e40ffd20 0x7f61e4103cf0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:25.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.674+0000 7f61ec0bd700 1 --2- 172.21.15.144:0/64765650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e40ff380 0x7f61e40ff750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:25.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.674+0000 7f61ec0bd700 1 -- 172.21.15.144:0/64765650 >> 172.21.15.144:0/64765650 conn(0x7f61e40fac20 msgr2=0x7f61e40fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:25.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.674+0000 7f61ec0bd700 1 -- 172.21.15.144:0/64765650 shutdown_connections 2024-09-10T12:03:25.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.674+0000 7f61ec0bd700 1 -- 172.21.15.144:0/64765650 wait complete. 2024-09-10T12:03:25.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.674+0000 7f61ec0bd700 1 Processor -- start 2024-09-10T12:03:25.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.675+0000 7f61ec0bd700 1 -- start start 2024-09-10T12:03:25.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.675+0000 7f61ec0bd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e40ff380 0x7f61e419aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:25.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.675+0000 7f61ec0bd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e40ffd20 0x7f61e419aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:25.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.675+0000 7f61ec0bd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e419b680 con 0x7f61e40ff380 2024-09-10T12:03:25.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.675+0000 7f61ec0bd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e4194b30 con 0x7f61e40ffd20 2024-09-10T12:03:25.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.675+0000 7f61e9e59700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e40ff380 0x7f61e419aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:25.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.675+0000 7f61e9e59700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e40ff380 0x7f61e419aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40462/0 (socket says 172.21.15.144:40462) 2024-09-10T12:03:25.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.675+0000 7f61e9e59700 1 -- 172.21.15.144:0/696235917 learned_addr learned my addr 172.21.15.144:0/696235917 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:03:25.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.675+0000 7f61e9658700 1 --2- 172.21.15.144:0/696235917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e40ffd20 0x7f61e419aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:25.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.675+0000 7f61e9e59700 1 -- 172.21.15.144:0/696235917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e40ffd20 msgr2=0x7f61e419aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:25.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.675+0000 7f61e9e59700 1 --2- 172.21.15.144:0/696235917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e40ffd20 0x7f61e419aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:25.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.675+0000 7f61e9e59700 1 -- 172.21.15.144:0/696235917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d40095f0 con 0x7f61e40ff380 2024-09-10T12:03:25.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.675+0000 7f61e9e59700 1 --2- 172.21.15.144:0/696235917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e40ff380 0x7f61e419aab0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f61e000d8d0 tx=0x7f61e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:25.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.676+0000 7f61daffd700 1 -- 172.21.15.144:0/696235917 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61e0009940 con 0x7f61e40ff380 2024-09-10T12:03:25.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.676+0000 7f61daffd700 1 -- 172.21.15.144:0/696235917 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61e0010460 con 0x7f61e40ff380 2024-09-10T12:03:25.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.676+0000 7f61daffd700 1 -- 172.21.15.144:0/696235917 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61e000f5f0 con 0x7f61e40ff380 2024-09-10T12:03:25.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.676+0000 7f61ec0bd700 1 -- 172.21.15.144:0/696235917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e4194e10 con 0x7f61e40ff380 2024-09-10T12:03:25.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.676+0000 7f61ec0bd700 1 -- 172.21.15.144:0/696235917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e4195360 con 0x7f61e40ff380 2024-09-10T12:03:25.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.677+0000 7f61ec0bd700 1 -- 172.21.15.144:0/696235917 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e41076f0 con 0x7f61e40ff380 2024-09-10T12:03:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.677+0000 7f61daffd700 1 -- 172.21.15.144:0/696235917 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f61e00105d0 con 0x7f61e40ff380 2024-09-10T12:03:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.678+0000 7f61daffd700 1 --2- 172.21.15.144:0/696235917 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f61d006e200 0x7f61d00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.678+0000 7f61daffd700 1 -- 172.21.15.144:0/696235917 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f61e008e0a0 con 0x7f61e40ff380 2024-09-10T12:03:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.678+0000 7f61e9658700 1 --2- 172.21.15.144:0/696235917 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f61d006e200 0x7f61d00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:25.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.678+0000 7f61e9658700 1 --2- 172.21.15.144:0/696235917 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f61d006e200 0x7f61d00706b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f61d40059d0 tx=0x7f61d4005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:25.683 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.682+0000 7f61daffd700 1 -- 172.21.15.144:0/696235917 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f61e005afb0 con 0x7f61e40ff380 2024-09-10T12:03:25.910 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.908+0000 7f61ec0bd700 1 -- 172.21.15.144:0/696235917 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f61e4195580 con 0x7f61d006e200 2024-09-10T12:03:25.911 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.910+0000 7f61daffd700 1 -- 172.21.15.144:0/696235917 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f61e4195580 con 0x7f61d006e200 2024-09-10T12:03:25.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.912+0000 7f61ec0bd700 1 -- 172.21.15.144:0/696235917 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f61d006e200 msgr2=0x7f61d00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:25.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.912+0000 7f61ec0bd700 1 --2- 172.21.15.144:0/696235917 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f61d006e200 0x7f61d00706b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f61d40059d0 tx=0x7f61d4005960 comp rx=0 tx=0).stop 2024-09-10T12:03:25.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.912+0000 7f61ec0bd700 1 -- 172.21.15.144:0/696235917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e40ff380 msgr2=0x7f61e419aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:25.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.912+0000 7f61ec0bd700 1 --2- 172.21.15.144:0/696235917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e40ff380 0x7f61e419aab0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f61e000d8d0 tx=0x7f61e000dc90 comp rx=0 tx=0).stop 2024-09-10T12:03:25.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.912+0000 7f61ec0bd700 1 -- 172.21.15.144:0/696235917 shutdown_connections 2024-09-10T12:03:25.913 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.912+0000 7f61ec0bd700 1 --2- 172.21.15.144:0/696235917 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f61d006e200 0x7f61d00706b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:25.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.912+0000 7f61ec0bd700 1 --2- 172.21.15.144:0/696235917 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f61e40ff380 0x7f61e419aab0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:25.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.912+0000 7f61ec0bd700 1 --2- 172.21.15.144:0/696235917 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f61e40ffd20 0x7f61e419aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:25.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.912+0000 7f61ec0bd700 1 -- 172.21.15.144:0/696235917 >> 172.21.15.144:0/696235917 conn(0x7f61e40fac20 msgr2=0x7f61e40fd010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:25.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.912+0000 7f61ec0bd700 1 -- 172.21.15.144:0/696235917 shutdown_connections 2024-09-10T12:03:25.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:25.912+0000 7f61ec0bd700 1 -- 172.21.15.144:0/696235917 wait complete. 2024-09-10T12:03:25.924 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:03:26.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.018+0000 7fc24d762700 1 -- 172.21.15.144:0/364898914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2480ffd30 msgr2=0x7fc248103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:26.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.018+0000 7fc24d762700 1 --2- 172.21.15.144:0/364898914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2480ffd30 0x7fc248103ce0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc230009910 tx=0x7fc230009c20 comp rx=0 tx=0).stop 2024-09-10T12:03:26.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.018+0000 7fc24d762700 1 -- 172.21.15.144:0/364898914 shutdown_connections 2024-09-10T12:03:26.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.018+0000 7fc24d762700 1 --2- 172.21.15.144:0/364898914 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2480ffd30 0x7fc248103ce0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.018+0000 7fc24d762700 1 --2- 172.21.15.144:0/364898914 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2480ff390 0x7fc2480ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.018+0000 7fc24d762700 1 -- 172.21.15.144:0/364898914 >> 172.21.15.144:0/364898914 conn(0x7fc248074f50 msgr2=0x7fc248075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:26.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.018+0000 7fc24d762700 1 -- 172.21.15.144:0/364898914 shutdown_connections 2024-09-10T12:03:26.020 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.018+0000 7fc24d762700 1 -- 172.21.15.144:0/364898914 wait complete. 2024-09-10T12:03:26.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.019+0000 7fc24d762700 1 Processor -- start 2024-09-10T12:03:26.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.019+0000 7fc24d762700 1 -- start start 2024-09-10T12:03:26.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.019+0000 7fc24d762700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2480ff390 0x7fc24819a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:26.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.019+0000 7fc24d762700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2480ffd30 0x7fc24819af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:26.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.019+0000 7fc24d762700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24819b520 con 0x7fc2480ff390 2024-09-10T12:03:26.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.019+0000 7fc24d762700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc248194a60 con 0x7fc2480ffd30 2024-09-10T12:03:26.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.019+0000 7fc246ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2480ff390 0x7fc24819a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:26.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.019+0000 7fc246ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2480ff390 0x7fc24819a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40484/0 (socket says 172.21.15.144:40484) 2024-09-10T12:03:26.022 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.019+0000 7fc246ffd700 1 -- 172.21.15.144:0/1510885993 learned_addr learned my addr 172.21.15.144:0/1510885993 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:03:26.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.019+0000 7fc23ffff700 1 --2- 172.21.15.144:0/1510885993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2480ffd30 0x7fc24819af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:26.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.019+0000 7fc246ffd700 1 -- 172.21.15.144:0/1510885993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2480ffd30 msgr2=0x7fc24819af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:26.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.019+0000 7fc246ffd700 1 --2- 172.21.15.144:0/1510885993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2480ffd30 0x7fc24819af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.019+0000 7fc246ffd700 1 -- 172.21.15.144:0/1510885993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2300095f0 con 0x7fc2480ff390 2024-09-10T12:03:26.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.019+0000 7fc23ffff700 1 --2- 172.21.15.144:0/1510885993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2480ffd30 0x7fc24819af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:03:26.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.020+0000 7fc246ffd700 1 --2- 172.21.15.144:0/1510885993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2480ff390 0x7fc24819a9e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc23800c930 tx=0x7fc23800ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:26.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.020+0000 7fc244ff9700 1 -- 172.21.15.144:0/1510885993 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc238007ad0 con 0x7fc2480ff390 2024-09-10T12:03:26.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.020+0000 7fc244ff9700 1 -- 172.21.15.144:0/1510885993 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc238007c30 con 0x7fc2480ff390 2024-09-10T12:03:26.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.020+0000 7fc244ff9700 1 -- 172.21.15.144:0/1510885993 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc238018610 con 0x7fc2480ff390 2024-09-10T12:03:26.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.020+0000 7fc24d762700 1 -- 172.21.15.144:0/1510885993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc248194d40 con 0x7fc2480ff390 2024-09-10T12:03:26.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.020+0000 7fc24d762700 1 -- 172.21.15.144:0/1510885993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc248195290 con 0x7fc2480ff390 2024-09-10T12:03:26.023 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.021+0000 7fc24d762700 1 -- 172.21.15.144:0/1510885993 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc248107600 con 0x7fc2480ff390 2024-09-10T12:03:26.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.021+0000 7fc244ff9700 1 -- 172.21.15.144:0/1510885993 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc23800f450 con 0x7fc2480ff390 2024-09-10T12:03:26.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.022+0000 7fc244ff9700 1 --2- 172.21.15.144:0/1510885993 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc23406e200 0x7fc2340706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:26.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.022+0000 7fc244ff9700 1 -- 172.21.15.144:0/1510885993 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc23808e1a0 con 0x7fc2480ff390 2024-09-10T12:03:26.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.022+0000 7fc23ffff700 1 --2- 172.21.15.144:0/1510885993 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc23406e200 0x7fc2340706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:26.024 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.022+0000 7fc23ffff700 1 --2- 172.21.15.144:0/1510885993 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc23406e200 0x7fc2340706b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc248103e20 tx=0x7fc23001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:26.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.026+0000 7fc244ff9700 1 -- 172.21.15.144:0/1510885993 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc23805b130 con 0x7fc2480ff390 2024-09-10T12:03:26.250 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.249+0000 7fc24d762700 1 -- 172.21.15.144:0/1510885993 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc248195580 con 0x7fc23406e200 2024-09-10T12:03:26.251 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.250+0000 7fc244ff9700 1 -- 172.21.15.144:0/1510885993 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc248195580 con 0x7fc23406e200 2024-09-10T12:03:26.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.252+0000 7fc24d762700 1 -- 172.21.15.144:0/1510885993 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc23406e200 msgr2=0x7fc2340706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:26.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.252+0000 7fc24d762700 1 --2- 172.21.15.144:0/1510885993 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc23406e200 0x7fc2340706b0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc248103e20 tx=0x7fc23001a040 comp rx=0 tx=0).stop 2024-09-10T12:03:26.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.252+0000 7fc24d762700 1 -- 172.21.15.144:0/1510885993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2480ff390 msgr2=0x7fc24819a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:26.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.252+0000 7fc24d762700 1 --2- 172.21.15.144:0/1510885993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2480ff390 0x7fc24819a9e0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc23800c930 tx=0x7fc23800ccf0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.252+0000 7fc24d762700 1 -- 172.21.15.144:0/1510885993 shutdown_connections 2024-09-10T12:03:26.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.252+0000 7fc24d762700 1 --2- 172.21.15.144:0/1510885993 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc23406e200 0x7fc2340706b0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.252+0000 7fc24d762700 1 --2- 172.21.15.144:0/1510885993 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc2480ff390 0x7fc24819a9e0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.252+0000 7fc24d762700 1 --2- 172.21.15.144:0/1510885993 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc2480ffd30 0x7fc24819af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.252+0000 7fc24d762700 1 -- 172.21.15.144:0/1510885993 >> 172.21.15.144:0/1510885993 conn(0x7fc248074f50 msgr2=0x7fc2480fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:26.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.252+0000 7fc24d762700 1 -- 172.21.15.144:0/1510885993 shutdown_connections 2024-09-10T12:03:26.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.252+0000 7fc24d762700 1 -- 172.21.15.144:0/1510885993 wait complete. 2024-09-10T12:03:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.358+0000 7fb1505ad700 1 -- 172.21.15.144:0/1505556719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1480ff290 msgr2=0x7fb1480ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.358+0000 7fb1505ad700 1 --2- 172.21.15.144:0/1505556719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1480ff290 0x7fb1480ff660 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb144009910 tx=0x7fb144009c20 comp rx=0 tx=0).stop 2024-09-10T12:03:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.358+0000 7fb1505ad700 1 -- 172.21.15.144:0/1505556719 shutdown_connections 2024-09-10T12:03:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.358+0000 7fb1505ad700 1 --2- 172.21.15.144:0/1505556719 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1480ffba0 0x7fb14810a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.358+0000 7fb1505ad700 1 --2- 172.21.15.144:0/1505556719 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1480ff290 0x7fb1480ff660 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.358+0000 7fb1505ad700 1 -- 172.21.15.144:0/1505556719 >> 172.21.15.144:0/1505556719 conn(0x7fb148074850 msgr2=0x7fb148074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:26.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.358+0000 7fb1505ad700 1 -- 172.21.15.144:0/1505556719 shutdown_connections 2024-09-10T12:03:26.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.358+0000 7fb1505ad700 1 -- 172.21.15.144:0/1505556719 wait complete. 2024-09-10T12:03:26.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.358+0000 7fb1505ad700 1 Processor -- start 2024-09-10T12:03:26.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.358+0000 7fb1505ad700 1 -- start start 2024-09-10T12:03:26.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.359+0000 7fb1505ad700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1480ff290 0x7fb148195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:26.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.359+0000 7fb1505ad700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1480ffba0 0x7fb1481962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:26.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.359+0000 7fb1505ad700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb148196990 con 0x7fb1480ff290 2024-09-10T12:03:26.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.359+0000 7fb1505ad700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14819a720 con 0x7fb1480ffba0 2024-09-10T12:03:26.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.359+0000 7fb14e349700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1480ff290 0x7fb148195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:26.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.359+0000 7fb14e349700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1480ff290 0x7fb148195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40512/0 (socket says 172.21.15.144:40512) 2024-09-10T12:03:26.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.359+0000 7fb14e349700 1 -- 172.21.15.144:0/753031965 learned_addr learned my addr 172.21.15.144:0/753031965 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:03:26.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.359+0000 7fb14db48700 1 --2- 172.21.15.144:0/753031965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1480ffba0 0x7fb1481962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:26.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.359+0000 7fb14e349700 1 -- 172.21.15.144:0/753031965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1480ffba0 msgr2=0x7fb1481962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:26.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.359+0000 7fb14e349700 1 --2- 172.21.15.144:0/753031965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1480ffba0 0x7fb1481962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.359+0000 7fb14e349700 1 -- 172.21.15.144:0/753031965 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1440095f0 con 0x7fb1480ff290 2024-09-10T12:03:26.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.359+0000 7fb14db48700 1 --2- 172.21.15.144:0/753031965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1480ffba0 0x7fb1481962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:03:26.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.359+0000 7fb14e349700 1 --2- 172.21.15.144:0/753031965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1480ff290 0x7fb148195d70 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb144005f50 tx=0x7fb14400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:26.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.360+0000 7fb13f7fe700 1 -- 172.21.15.144:0/753031965 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb14401d070 con 0x7fb1480ff290 2024-09-10T12:03:26.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.360+0000 7fb1505ad700 1 -- 172.21.15.144:0/753031965 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb14819a9a0 con 0x7fb1480ff290 2024-09-10T12:03:26.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.360+0000 7fb13f7fe700 1 -- 172.21.15.144:0/753031965 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb14400f460 con 0x7fb1480ff290 2024-09-10T12:03:26.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.360+0000 7fb13f7fe700 1 -- 172.21.15.144:0/753031965 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb144017630 con 0x7fb1480ff290 2024-09-10T12:03:26.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.360+0000 7fb1505ad700 1 -- 172.21.15.144:0/753031965 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb14819ae90 con 0x7fb1480ff290 2024-09-10T12:03:26.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.361+0000 7fb1505ad700 1 -- 172.21.15.144:0/753031965 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb148107d10 con 0x7fb1480ff290 2024-09-10T12:03:26.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.362+0000 7fb13f7fe700 1 -- 172.21.15.144:0/753031965 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb14400f5d0 con 0x7fb1480ff290 2024-09-10T12:03:26.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.362+0000 7fb13f7fe700 1 --2- 172.21.15.144:0/753031965 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb13406e200 0x7fb1340706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:26.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.362+0000 7fb13f7fe700 1 -- 172.21.15.144:0/753031965 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb14408f850 con 0x7fb1480ff290 2024-09-10T12:03:26.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.362+0000 7fb14db48700 1 --2- 172.21.15.144:0/753031965 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb13406e200 0x7fb1340706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:26.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.363+0000 7fb14db48700 1 --2- 172.21.15.144:0/753031965 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb13406e200 0x7fb1340706b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb148197390 tx=0x7fb138005f50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:26.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.366+0000 7fb13f7fe700 1 -- 172.21.15.144:0/753031965 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb144058cb0 con 0x7fb1480ff290 2024-09-10T12:03:26.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.577+0000 7fb1505ad700 1 -- 172.21.15.144:0/753031965 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb14802cd40 con 0x7fb13406e200 2024-09-10T12:03:26.587 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.586+0000 7fb13f7fe700 1 -- 172.21.15.144:0/753031965 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fb14802cd40 con 0x7fb13406e200 2024-09-10T12:03:26.587 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:03:26.587 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (31m) 9m ago 42m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:03:26.588 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (42m) 9m ago 42m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:03:26.588 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (41m) 10m ago 41m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:03:26.588 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (29m) 9m ago 42m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:03:26.588 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (29m) 10m ago 41m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:03:26.588 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (31m) 9m ago 41m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:03:26.588 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (25m) 9m ago 36m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:03:26.588 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (25m) 9m ago 36m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:03:26.588 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (10m) 10m ago 36m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:03:26.588 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (25m) 10m ago 36m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:03:26.588 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (33m) 9m ago 43m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:03:26.589 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (32m) 10m ago 41m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:03:26.589 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (30m) 9m ago 43m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:03:26.589 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (29m) 10m ago 41m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:03:26.589 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (32m) 9m ago 42m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:03:26.589 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (32m) 10m ago 41m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:03:26.589 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (28m) 9m ago 40m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:03:26.589 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (28m) 9m ago 39m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:03:26.589 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (27m) 9m ago 39m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:03:26.589 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (27m) 10m ago 39m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:03:26.589 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (26m) 10m ago 38m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:03:26.589 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (26m) 10m ago 38m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:03:26.589 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (31m) 9m ago 41m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:03:26.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.588+0000 7fb1505ad700 1 -- 172.21.15.144:0/753031965 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb13406e200 msgr2=0x7fb1340706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:26.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.588+0000 7fb1505ad700 1 --2- 172.21.15.144:0/753031965 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb13406e200 0x7fb1340706b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb148197390 tx=0x7fb138005f50 comp rx=0 tx=0).stop 2024-09-10T12:03:26.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.588+0000 7fb1505ad700 1 -- 172.21.15.144:0/753031965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1480ff290 msgr2=0x7fb148195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:26.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.588+0000 7fb1505ad700 1 --2- 172.21.15.144:0/753031965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1480ff290 0x7fb148195d70 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fb144005f50 tx=0x7fb14400be20 comp rx=0 tx=0).stop 2024-09-10T12:03:26.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.588+0000 7fb1505ad700 1 -- 172.21.15.144:0/753031965 shutdown_connections 2024-09-10T12:03:26.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.588+0000 7fb1505ad700 1 --2- 172.21.15.144:0/753031965 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb13406e200 0x7fb1340706b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.590 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.588+0000 7fb1505ad700 1 --2- 172.21.15.144:0/753031965 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1480ff290 0x7fb148195d70 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.588+0000 7fb1505ad700 1 --2- 172.21.15.144:0/753031965 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1480ffba0 0x7fb1481962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.588+0000 7fb1505ad700 1 -- 172.21.15.144:0/753031965 >> 172.21.15.144:0/753031965 conn(0x7fb148074850 msgr2=0x7fb148104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:26.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.589+0000 7fb1505ad700 1 -- 172.21.15.144:0/753031965 shutdown_connections 2024-09-10T12:03:26.591 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.589+0000 7fb1505ad700 1 -- 172.21.15.144:0/753031965 wait complete. 2024-09-10T12:03:26.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.691+0000 7f50b8d65700 1 -- 172.21.15.144:0/85966882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50b40ffd30 msgr2=0x7f50b4103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:26.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.691+0000 7f50b8d65700 1 --2- 172.21.15.144:0/85966882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50b40ffd30 0x7f50b4103ce0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f50a4009910 tx=0x7f50a4009c20 comp rx=0 tx=0).stop 2024-09-10T12:03:26.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.691+0000 7f50b8d65700 1 -- 172.21.15.144:0/85966882 shutdown_connections 2024-09-10T12:03:26.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.691+0000 7f50b8d65700 1 --2- 172.21.15.144:0/85966882 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50b40ffd30 0x7f50b4103ce0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.691+0000 7f50b8d65700 1 --2- 172.21.15.144:0/85966882 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50b40ff390 0x7f50b40ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.691+0000 7f50b8d65700 1 -- 172.21.15.144:0/85966882 >> 172.21.15.144:0/85966882 conn(0x7f50b4074f50 msgr2=0x7f50b4075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:26.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.691+0000 7f50b8d65700 1 -- 172.21.15.144:0/85966882 shutdown_connections 2024-09-10T12:03:26.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.691+0000 7f50b8d65700 1 -- 172.21.15.144:0/85966882 wait complete. 2024-09-10T12:03:26.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.692+0000 7f50b8d65700 1 Processor -- start 2024-09-10T12:03:26.694 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.692+0000 7f50b8d65700 1 -- start start 2024-09-10T12:03:26.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.692+0000 7f50b8d65700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50b40ff390 0x7f50b4196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:26.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.692+0000 7f50b8d65700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50b40ffd30 0x7f50b41970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:26.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.692+0000 7f50b8d65700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b41976c0 con 0x7f50b40ffd30 2024-09-10T12:03:26.695 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.692+0000 7f50b8d65700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b4197830 con 0x7f50b40ff390 2024-09-10T12:03:26.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.692+0000 7f50b1d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50b40ffd30 0x7f50b41970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:26.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.692+0000 7f50b1d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50b40ffd30 0x7f50b41970a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40536/0 (socket says 172.21.15.144:40536) 2024-09-10T12:03:26.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.692+0000 7f50b1d9b700 1 -- 172.21.15.144:0/2223664593 learned_addr learned my addr 172.21.15.144:0/2223664593 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:03:26.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.692+0000 7f50b259c700 1 --2- 172.21.15.144:0/2223664593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50b40ff390 0x7f50b4196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:26.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.693+0000 7f50b1d9b700 1 -- 172.21.15.144:0/2223664593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50b40ff390 msgr2=0x7f50b4196b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:26.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.693+0000 7f50b1d9b700 1 --2- 172.21.15.144:0/2223664593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50b40ff390 0x7f50b4196b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.693+0000 7f50b1d9b700 1 -- 172.21.15.144:0/2223664593 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50a40095f0 con 0x7f50b40ffd30 2024-09-10T12:03:26.696 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.693+0000 7f50b1d9b700 1 --2- 172.21.15.144:0/2223664593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50b40ffd30 0x7f50b41970a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f50a40092e0 tx=0x7f50a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:26.697 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.693+0000 7f50a37fe700 1 -- 172.21.15.144:0/2223664593 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50a401d070 con 0x7f50b40ffd30 2024-09-10T12:03:26.697 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.693+0000 7f50a37fe700 1 -- 172.21.15.144:0/2223664593 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f50a400f460 con 0x7f50b40ffd30 2024-09-10T12:03:26.697 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.693+0000 7f50b8d65700 1 -- 172.21.15.144:0/2223664593 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b419fbd0 con 0x7f50b40ffd30 2024-09-10T12:03:26.697 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.693+0000 7f50a37fe700 1 -- 172.21.15.144:0/2223664593 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50a40175f0 con 0x7f50b40ffd30 2024-09-10T12:03:26.697 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.693+0000 7f50b8d65700 1 -- 172.21.15.144:0/2223664593 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b419fff0 con 0x7f50b40ffd30 2024-09-10T12:03:26.697 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.694+0000 7f50b8d65700 1 -- 172.21.15.144:0/2223664593 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b4107600 con 0x7f50b40ffd30 2024-09-10T12:03:26.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.695+0000 7f50a37fe700 1 -- 172.21.15.144:0/2223664593 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f50a40178b0 con 0x7f50b40ffd30 2024-09-10T12:03:26.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.695+0000 7f50a37fe700 1 --2- 172.21.15.144:0/2223664593 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f509c06e200 0x7f509c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:26.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.695+0000 7f50a37fe700 1 -- 172.21.15.144:0/2223664593 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f50a408f5f0 con 0x7f50b40ffd30 2024-09-10T12:03:26.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.695+0000 7f50b259c700 1 --2- 172.21.15.144:0/2223664593 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f509c06e200 0x7f509c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:26.698 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.696+0000 7f50b259c700 1 --2- 172.21.15.144:0/2223664593 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f509c06e200 0x7f509c0706b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f50a8005fd0 tx=0x7f50a800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:26.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.700+0000 7f50a37fe700 1 -- 172.21.15.144:0/2223664593 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f50a4058a40 con 0x7f50b40ffd30 2024-09-10T12:03:26.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.991+0000 7f50b8d65700 1 -- 172.21.15.144:0/2223664593 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f50b402cc30 con 0x7f50b40ffd30 2024-09-10T12:03:26.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.991+0000 7f50a37fe700 1 -- 172.21.15.144:0/2223664593 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f50a405c100 con 0x7f50b40ffd30 2024-09-10T12:03:26.993 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:03:26.993 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:03:26.993 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:03:26.993 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:03:26.993 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:03:26.993 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:03:26.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:03:26.994 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:03:26.994 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:03:26.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:03:26.994 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:03:26.994 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:03:26.994 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:03:26.994 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:03:26.994 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:03:26.994 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:03:26.994 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:03:26.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.994+0000 7f50b8d65700 1 -- 172.21.15.144:0/2223664593 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f509c06e200 msgr2=0x7f509c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:26.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.994+0000 7f50b8d65700 1 --2- 172.21.15.144:0/2223664593 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f509c06e200 0x7f509c0706b0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f50a8005fd0 tx=0x7f50a800a560 comp rx=0 tx=0).stop 2024-09-10T12:03:26.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.994+0000 7f50b8d65700 1 -- 172.21.15.144:0/2223664593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50b40ffd30 msgr2=0x7f50b41970a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:26.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.994+0000 7f50b8d65700 1 --2- 172.21.15.144:0/2223664593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50b40ffd30 0x7f50b41970a0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f50a40092e0 tx=0x7f50a400be20 comp rx=0 tx=0).stop 2024-09-10T12:03:26.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.994+0000 7f50b8d65700 1 -- 172.21.15.144:0/2223664593 shutdown_connections 2024-09-10T12:03:26.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.994+0000 7f50b8d65700 1 --2- 172.21.15.144:0/2223664593 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f509c06e200 0x7f509c0706b0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.994+0000 7f50b8d65700 1 --2- 172.21.15.144:0/2223664593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f50b40ff390 0x7f50b4196b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.994+0000 7f50b8d65700 1 --2- 172.21.15.144:0/2223664593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f50b40ffd30 0x7f50b41970a0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:26.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.994+0000 7f50b8d65700 1 -- 172.21.15.144:0/2223664593 >> 172.21.15.144:0/2223664593 conn(0x7f50b4074f50 msgr2=0x7f50b40fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:26.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.994+0000 7f50b8d65700 1 -- 172.21.15.144:0/2223664593 shutdown_connections 2024-09-10T12:03:26.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:26.994+0000 7f50b8d65700 1 -- 172.21.15.144:0/2223664593 wait complete. 2024-09-10T12:03:27.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.105+0000 7fa528c3c700 1 -- 172.21.15.144:0/2530664437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa524100400 msgr2=0x7fa5241007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:27.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.105+0000 7fa528c3c700 1 --2- 172.21.15.144:0/2530664437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa524100400 0x7fa5241007d0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fa514009910 tx=0x7fa514009c20 comp rx=0 tx=0).stop 2024-09-10T12:03:27.107 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.105+0000 7fa528c3c700 1 -- 172.21.15.144:0/2530664437 shutdown_connections 2024-09-10T12:03:27.107 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.105+0000 7fa528c3c700 1 --2- 172.21.15.144:0/2530664437 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa524100da0 0x7fa524104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:27.107 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.105+0000 7fa528c3c700 1 --2- 172.21.15.144:0/2530664437 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa524100400 0x7fa5241007d0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:27.107 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.105+0000 7fa528c3c700 1 -- 172.21.15.144:0/2530664437 >> 172.21.15.144:0/2530664437 conn(0x7fa5240fbca0 msgr2=0x7fa5240fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:27.107 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.105+0000 7fa528c3c700 1 -- 172.21.15.144:0/2530664437 shutdown_connections 2024-09-10T12:03:27.107 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.105+0000 7fa528c3c700 1 -- 172.21.15.144:0/2530664437 wait complete. 2024-09-10T12:03:27.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.106+0000 7fa528c3c700 1 Processor -- start 2024-09-10T12:03:27.108 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.106+0000 7fa528c3c700 1 -- start start 2024-09-10T12:03:27.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.106+0000 7fa528c3c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa524100400 0x7fa524195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:27.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.106+0000 7fa528c3c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa524100da0 0x7fa524196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:27.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.106+0000 7fa528c3c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa524196a60 con 0x7fa524100400 2024-09-10T12:03:27.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.106+0000 7fa528c3c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52419a7f0 con 0x7fa524100da0 2024-09-10T12:03:27.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.106+0000 7fa52259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa524100400 0x7fa524195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:27.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.106+0000 7fa52259c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa524100400 0x7fa524195e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40558/0 (socket says 172.21.15.144:40558) 2024-09-10T12:03:27.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.106+0000 7fa52259c700 1 -- 172.21.15.144:0/3668810159 learned_addr learned my addr 172.21.15.144:0/3668810159 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:03:27.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.106+0000 7fa51bfff700 1 --2- 172.21.15.144:0/3668810159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa524100da0 0x7fa524196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:27.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.106+0000 7fa52259c700 1 -- 172.21.15.144:0/3668810159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa524100da0 msgr2=0x7fa524196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:27.109 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.106+0000 7fa52259c700 1 --2- 172.21.15.144:0/3668810159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa524100da0 0x7fa524196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:27.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.106+0000 7fa52259c700 1 -- 172.21.15.144:0/3668810159 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5140095f0 con 0x7fa524100400 2024-09-10T12:03:27.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.106+0000 7fa51bfff700 1 --2- 172.21.15.144:0/3668810159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa524100da0 0x7fa524196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:03:27.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.107+0000 7fa52259c700 1 --2- 172.21.15.144:0/3668810159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa524100400 0x7fa524195e40 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa514006010 tx=0x7fa51400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:27.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.107+0000 7fa51b7fe700 1 -- 172.21.15.144:0/3668810159 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa51401d070 con 0x7fa524100400 2024-09-10T12:03:27.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.107+0000 7fa51b7fe700 1 -- 172.21.15.144:0/3668810159 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa514004fb0 con 0x7fa524100400 2024-09-10T12:03:27.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.107+0000 7fa51b7fe700 1 -- 172.21.15.144:0/3668810159 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa514022ea0 con 0x7fa524100400 2024-09-10T12:03:27.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.107+0000 7fa528c3c700 1 -- 172.21.15.144:0/3668810159 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52419aa70 con 0x7fa524100400 2024-09-10T12:03:27.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.107+0000 7fa528c3c700 1 -- 172.21.15.144:0/3668810159 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52419af10 con 0x7fa524100400 2024-09-10T12:03:27.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.108+0000 7fa528c3c700 1 -- 172.21.15.144:0/3668810159 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa524108710 con 0x7fa524100400 2024-09-10T12:03:27.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.108+0000 7fa51b7fe700 1 -- 172.21.15.144:0/3668810159 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa514022470 con 0x7fa524100400 2024-09-10T12:03:27.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.109+0000 7fa51b7fe700 1 --2- 172.21.15.144:0/3668810159 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa51006e200 0x7fa5100706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:27.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.109+0000 7fa51b7fe700 1 -- 172.21.15.144:0/3668810159 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa51408f610 con 0x7fa524100400 2024-09-10T12:03:27.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.109+0000 7fa51bfff700 1 --2- 172.21.15.144:0/3668810159 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa51006e200 0x7fa5100706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:27.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.109+0000 7fa51bfff700 1 --2- 172.21.15.144:0/3668810159 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa51006e200 0x7fa5100706b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa524197460 tx=0x7fa50c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:27.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.113+0000 7fa51b7fe700 1 -- 172.21.15.144:0/3668810159 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa514058aa0 con 0x7fa524100400 2024-09-10T12:03:27.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.360+0000 7fa528c3c700 1 -- 172.21.15.144:0/3668810159 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa52404ea50 con 0x7fa524100400 2024-09-10T12:03:27.361 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:26 smithi144 ceph-mon[87436]: pgmap v944: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:27.361 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:26 smithi144 ceph-mon[87436]: from='client.35716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:03:27.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.360+0000 7fa51b7fe700 1 -- 172.21.15.144:0/3668810159 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7fa514022720 con 0x7fa524100400 2024-09-10T12:03:27.362 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:03:27.362 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:03:27.362 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:03:27.362 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:03:27.362 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:03:27.362 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:03:27.362 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:03:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:03:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:03:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:03:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:03:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:03:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:03:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:03:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:03:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:03:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:03:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:03:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:03:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:03:27.363 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:03:27.364 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:03:27.364 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:03:27.364 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:03:27.364 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:03:27.364 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:03:27.364 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:03:27.364 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:03:27.364 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:03:27.364 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:03:27.364 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:03:27.364 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:03:27.364 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:03:27.364 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:03:27.364 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:03:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:03:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:03:27.365 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:03:27.365 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:03:27.365 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:03:27.365 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:03:27.365 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:03:27.365 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:03:27.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.362+0000 7fa528c3c700 1 -- 172.21.15.144:0/3668810159 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa51006e200 msgr2=0x7fa5100706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:27.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.362+0000 7fa528c3c700 1 --2- 172.21.15.144:0/3668810159 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa51006e200 0x7fa5100706b0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fa524197460 tx=0x7fa50c008040 comp rx=0 tx=0).stop 2024-09-10T12:03:27.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.362+0000 7fa528c3c700 1 -- 172.21.15.144:0/3668810159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa524100400 msgr2=0x7fa524195e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:27.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.362+0000 7fa528c3c700 1 --2- 172.21.15.144:0/3668810159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa524100400 0x7fa524195e40 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa514006010 tx=0x7fa51400be20 comp rx=0 tx=0).stop 2024-09-10T12:03:27.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.362+0000 7fa528c3c700 1 -- 172.21.15.144:0/3668810159 shutdown_connections 2024-09-10T12:03:27.366 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.362+0000 7fa528c3c700 1 --2- 172.21.15.144:0/3668810159 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa51006e200 0x7fa5100706b0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:27.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.362+0000 7fa528c3c700 1 --2- 172.21.15.144:0/3668810159 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa524100400 0x7fa524195e40 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:27.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.362+0000 7fa528c3c700 1 --2- 172.21.15.144:0/3668810159 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa524100da0 0x7fa524196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:27.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.362+0000 7fa528c3c700 1 -- 172.21.15.144:0/3668810159 >> 172.21.15.144:0/3668810159 conn(0x7fa5240fbca0 msgr2=0x7fa5240fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:27.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.363+0000 7fa528c3c700 1 -- 172.21.15.144:0/3668810159 shutdown_connections 2024-09-10T12:03:27.367 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.363+0000 7fa528c3c700 1 -- 172.21.15.144:0/3668810159 wait complete. 2024-09-10T12:03:27.367 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:03:27.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:26 smithi178 ceph-mon[65940]: pgmap v944: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:27.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:26 smithi178 ceph-mon[65940]: from='client.35716 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:03:27.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.469+0000 7f379c0f1700 1 -- 172.21.15.144:0/1591804379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3794101b00 msgr2=0x7f379410a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:27.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.469+0000 7f379c0f1700 1 --2- 172.21.15.144:0/1591804379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3794101b00 0x7f379410a1d0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f3788009910 tx=0x7f3788009c20 comp rx=0 tx=0).stop 2024-09-10T12:03:27.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.469+0000 7f379c0f1700 1 -- 172.21.15.144:0/1591804379 shutdown_connections 2024-09-10T12:03:27.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.469+0000 7f379c0f1700 1 --2- 172.21.15.144:0/1591804379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3794101b00 0x7f379410a1d0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:27.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.469+0000 7f379c0f1700 1 --2- 172.21.15.144:0/1591804379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f37941011f0 0x7f37941015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:27.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.469+0000 7f379c0f1700 1 -- 172.21.15.144:0/1591804379 >> 172.21.15.144:0/1591804379 conn(0x7f37940fab80 msgr2=0x7f37940fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:27.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.469+0000 7f379c0f1700 1 -- 172.21.15.144:0/1591804379 shutdown_connections 2024-09-10T12:03:27.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.469+0000 7f379c0f1700 1 -- 172.21.15.144:0/1591804379 wait complete. 2024-09-10T12:03:27.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.470+0000 7f379c0f1700 1 Processor -- start 2024-09-10T12:03:27.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.470+0000 7f379c0f1700 1 -- start start 2024-09-10T12:03:27.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.470+0000 7f379c0f1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37941011f0 0x7f379419bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:27.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.470+0000 7f379c0f1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3794101b00 0x7f379419c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:27.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.470+0000 7f379c0f1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379419c690 con 0x7f37941011f0 2024-09-10T12:03:27.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.470+0000 7f379c0f1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3794195b40 con 0x7f3794101b00 2024-09-10T12:03:27.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.470+0000 7f3799e8d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37941011f0 0x7f379419bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:27.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.470+0000 7f3799e8d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37941011f0 0x7f379419bac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40578/0 (socket says 172.21.15.144:40578) 2024-09-10T12:03:27.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.470+0000 7f3799e8d700 1 -- 172.21.15.144:0/235593047 learned_addr learned my addr 172.21.15.144:0/235593047 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:03:27.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.470+0000 7f379968c700 1 --2- 172.21.15.144:0/235593047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3794101b00 0x7f379419c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:27.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.470+0000 7f3799e8d700 1 -- 172.21.15.144:0/235593047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3794101b00 msgr2=0x7f379419c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:27.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.470+0000 7f3799e8d700 1 --2- 172.21.15.144:0/235593047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3794101b00 0x7f379419c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:27.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.470+0000 7f3799e8d700 1 -- 172.21.15.144:0/235593047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37880095f0 con 0x7f37941011f0 2024-09-10T12:03:27.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.470+0000 7f3799e8d700 1 --2- 172.21.15.144:0/235593047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37941011f0 0x7f379419bac0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f378c00ada0 tx=0x7f378c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:27.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.471+0000 7f3792ffd700 1 -- 172.21.15.144:0/235593047 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f378c00f070 con 0x7f37941011f0 2024-09-10T12:03:27.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.471+0000 7f3792ffd700 1 -- 172.21.15.144:0/235593047 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f378c007bf0 con 0x7f37941011f0 2024-09-10T12:03:27.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.471+0000 7f379c0f1700 1 -- 172.21.15.144:0/235593047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3794195e20 con 0x7f37941011f0 2024-09-10T12:03:27.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.471+0000 7f3792ffd700 1 -- 172.21.15.144:0/235593047 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f378c005690 con 0x7f37941011f0 2024-09-10T12:03:27.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.471+0000 7f379c0f1700 1 -- 172.21.15.144:0/235593047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3794196370 con 0x7f37941011f0 2024-09-10T12:03:27.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.472+0000 7f379c0f1700 1 -- 172.21.15.144:0/235593047 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3794107870 con 0x7f37941011f0 2024-09-10T12:03:27.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.472+0000 7f3792ffd700 1 -- 172.21.15.144:0/235593047 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f378c005830 con 0x7f37941011f0 2024-09-10T12:03:27.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.473+0000 7f3792ffd700 1 --2- 172.21.15.144:0/235593047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f377c06e200 0x7f377c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:27.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.473+0000 7f379968c700 1 --2- 172.21.15.144:0/235593047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f377c06e200 0x7f377c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:27.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.473+0000 7f3792ffd700 1 -- 172.21.15.144:0/235593047 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f378c08d9e0 con 0x7f37941011f0 2024-09-10T12:03:27.476 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.473+0000 7f379968c700 1 --2- 172.21.15.144:0/235593047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f377c06e200 0x7f377c0706b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f378800b5c0 tx=0x7f3788005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:27.478 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.477+0000 7f3792ffd700 1 -- 172.21.15.144:0/235593047 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f378c05a970 con 0x7f37941011f0 2024-09-10T12:03:27.700 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.699+0000 7f379c0f1700 1 -- 172.21.15.144:0/235593047 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f37941966a0 con 0x7f377c06e200 2024-09-10T12:03:27.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.700+0000 7f3792ffd700 1 -- 172.21.15.144:0/235593047 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f37941966a0 con 0x7f377c06e200 2024-09-10T12:03:27.705 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:03:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:03:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:03:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:03:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:03:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:03:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:03:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:03:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:03:27.705 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:03:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:03:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:03:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:03:27.706 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:03:27.706 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:03:27.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.702+0000 7f379c0f1700 1 -- 172.21.15.144:0/235593047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f377c06e200 msgr2=0x7f377c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:27.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.702+0000 7f379c0f1700 1 --2- 172.21.15.144:0/235593047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f377c06e200 0x7f377c0706b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f378800b5c0 tx=0x7f3788005960 comp rx=0 tx=0).stop 2024-09-10T12:03:27.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.702+0000 7f379c0f1700 1 -- 172.21.15.144:0/235593047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37941011f0 msgr2=0x7f379419bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:27.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.702+0000 7f379c0f1700 1 --2- 172.21.15.144:0/235593047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37941011f0 0x7f379419bac0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f378c00ada0 tx=0x7f378c00c330 comp rx=0 tx=0).stop 2024-09-10T12:03:27.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.703+0000 7f379c0f1700 1 -- 172.21.15.144:0/235593047 shutdown_connections 2024-09-10T12:03:27.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.703+0000 7f379c0f1700 1 --2- 172.21.15.144:0/235593047 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f377c06e200 0x7f377c0706b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:27.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.703+0000 7f379c0f1700 1 --2- 172.21.15.144:0/235593047 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f37941011f0 0x7f379419bac0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:27.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.703+0000 7f379c0f1700 1 --2- 172.21.15.144:0/235593047 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3794101b00 0x7f379419c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:27.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.703+0000 7f379c0f1700 1 -- 172.21.15.144:0/235593047 >> 172.21.15.144:0/235593047 conn(0x7f37940fab80 msgr2=0x7f37941049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:27.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.703+0000 7f379c0f1700 1 -- 172.21.15.144:0/235593047 shutdown_connections 2024-09-10T12:03:27.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.703+0000 7f379c0f1700 1 -- 172.21.15.144:0/235593047 wait complete. 2024-09-10T12:03:27.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.807+0000 7f7a124c7700 1 -- 172.21.15.144:0/3969254034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a0c101320 msgr2=0x7f7a0c1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:27.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.807+0000 7f7a124c7700 1 --2- 172.21.15.144:0/3969254034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a0c101320 0x7f7a0c1016f0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f79fc009940 tx=0x7f79fc009c50 comp rx=0 tx=0).stop 2024-09-10T12:03:27.808 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.807+0000 7f7a124c7700 1 -- 172.21.15.144:0/3969254034 shutdown_connections 2024-09-10T12:03:27.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.807+0000 7f7a124c7700 1 --2- 172.21.15.144:0/3969254034 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c101c30 0x7f7a0c10a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:27.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.807+0000 7f7a124c7700 1 --2- 172.21.15.144:0/3969254034 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a0c101320 0x7f7a0c1016f0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:27.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.807+0000 7f7a124c7700 1 -- 172.21.15.144:0/3969254034 >> 172.21.15.144:0/3969254034 conn(0x7f7a0c0fab80 msgr2=0x7f7a0c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:27.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.807+0000 7f7a124c7700 1 -- 172.21.15.144:0/3969254034 shutdown_connections 2024-09-10T12:03:27.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.807+0000 7f7a124c7700 1 -- 172.21.15.144:0/3969254034 wait complete. 2024-09-10T12:03:27.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.807+0000 7f7a124c7700 1 Processor -- start 2024-09-10T12:03:27.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.807+0000 7f7a124c7700 1 -- start start 2024-09-10T12:03:27.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.808+0000 7f7a124c7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c101320 0x7f7a0c19bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:27.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.808+0000 7f7a124c7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a0c101c30 0x7f7a0c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:27.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.808+0000 7f7a124c7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c19c680 con 0x7f7a0c101c30 2024-09-10T12:03:27.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.808+0000 7f7a124c7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c195b30 con 0x7f7a0c101320 2024-09-10T12:03:27.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.808+0000 7f7a0b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a0c101c30 0x7f7a0c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:27.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.808+0000 7f7a0b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a0c101c30 0x7f7a0c19bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:40588/0 (socket says 172.21.15.144:40588) 2024-09-10T12:03:27.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.808+0000 7f7a0b7fe700 1 -- 172.21.15.144:0/2029947837 learned_addr learned my addr 172.21.15.144:0/2029947837 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:03:27.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.808+0000 7f7a0bfff700 1 --2- 172.21.15.144:0/2029947837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c101320 0x7f7a0c19bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:27.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.808+0000 7f7a0b7fe700 1 -- 172.21.15.144:0/2029947837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c101320 msgr2=0x7f7a0c19bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:27.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.808+0000 7f7a0b7fe700 1 --2- 172.21.15.144:0/2029947837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c101320 0x7f7a0c19bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:27.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.808+0000 7f7a0b7fe700 1 -- 172.21.15.144:0/2029947837 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79fc0095f0 con 0x7f7a0c101c30 2024-09-10T12:03:27.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.808+0000 7f7a0bfff700 1 --2- 172.21.15.144:0/2029947837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c101320 0x7f7a0c19bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:03:27.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.809+0000 7f7a0b7fe700 1 --2- 172.21.15.144:0/2029947837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a0c101c30 0x7f7a0c19bff0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000d8d0 tx=0x7f7a0000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:27.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.809+0000 7f7a097fa700 1 -- 172.21.15.144:0/2029947837 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a00009940 con 0x7f7a0c101c30 2024-09-10T12:03:27.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.809+0000 7f7a097fa700 1 -- 172.21.15.144:0/2029947837 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7a00010460 con 0x7f7a0c101c30 2024-09-10T12:03:27.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.809+0000 7f7a124c7700 1 -- 172.21.15.144:0/2029947837 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0c195e10 con 0x7f7a0c101c30 2024-09-10T12:03:27.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.809+0000 7f7a097fa700 1 -- 172.21.15.144:0/2029947837 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a0000f5f0 con 0x7f7a0c101c30 2024-09-10T12:03:27.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.809+0000 7f7a124c7700 1 -- 172.21.15.144:0/2029947837 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0c196360 con 0x7f7a0c101c30 2024-09-10T12:03:27.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.810+0000 7f7a124c7700 1 -- 172.21.15.144:0/2029947837 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0c107800 con 0x7f7a0c101c30 2024-09-10T12:03:27.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.810+0000 7f7a097fa700 1 -- 172.21.15.144:0/2029947837 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7a000105d0 con 0x7f7a0c101c30 2024-09-10T12:03:27.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.811+0000 7f7a097fa700 1 --2- 172.21.15.144:0/2029947837 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f79f406e200 0x7f79f40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:27.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.811+0000 7f7a097fa700 1 -- 172.21.15.144:0/2029947837 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7a0008e0a0 con 0x7f7a0c101c30 2024-09-10T12:03:27.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.811+0000 7f7a0bfff700 1 --2- 172.21.15.144:0/2029947837 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f79f406e200 0x7f79f40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:27.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.811+0000 7f7a0bfff700 1 --2- 172.21.15.144:0/2029947837 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f79f406e200 0x7f79f40706b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00b870 tx=0x7f79fc005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:27.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:27.816+0000 7f7a097fa700 1 -- 172.21.15.144:0/2029947837 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7a0005afb0 con 0x7f7a0c101c30 2024-09-10T12:03:27.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:27 smithi144 ceph-mon[87436]: from='client.35720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:03:27.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:27 smithi144 ceph-mon[87436]: from='client.35724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:03:27.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:27 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2223664593' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:03:27.951 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:27 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3668810159' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:03:28.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:28.101+0000 7f7a124c7700 1 -- 172.21.15.144:0/2029947837 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f7a0c04ea50 con 0x7f7a0c101c30 2024-09-10T12:03:28.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:28.101+0000 7f7a097fa700 1 -- 172.21.15.144:0/2029947837 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f7a0005ab40 con 0x7f7a0c101c30 2024-09-10T12:03:28.103 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:03:28.103 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:03:28.103 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:03:28.103 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:03:28.103 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:03:28.103 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:03:28.103 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:03:28.103 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:03:28.104 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:03:28.104 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:03:28.104 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:03:28.104 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:03:28.104 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:03:28.104 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:03:28.104 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:03:28.104 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:03:28.104 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:03:28.104 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:03:28.104 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:03:28.104 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:03:28.104 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:03:28.104 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:03:28.105 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:03:28.105 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:03:28.105 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:03:28.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:28.104+0000 7f7a124c7700 1 -- 172.21.15.144:0/2029947837 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f79f406e200 msgr2=0x7f79f40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:28.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:28.104+0000 7f7a124c7700 1 --2- 172.21.15.144:0/2029947837 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f79f406e200 0x7f79f40706b0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f79fc00b870 tx=0x7f79fc005960 comp rx=0 tx=0).stop 2024-09-10T12:03:28.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:28.104+0000 7f7a124c7700 1 -- 172.21.15.144:0/2029947837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a0c101c30 msgr2=0x7f7a0c19bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:28.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:28.104+0000 7f7a124c7700 1 --2- 172.21.15.144:0/2029947837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a0c101c30 0x7f7a0c19bff0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f7a0000d8d0 tx=0x7f7a0000dc90 comp rx=0 tx=0).stop 2024-09-10T12:03:28.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:28.104+0000 7f7a124c7700 1 -- 172.21.15.144:0/2029947837 shutdown_connections 2024-09-10T12:03:28.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:28.104+0000 7f7a124c7700 1 --2- 172.21.15.144:0/2029947837 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f79f406e200 0x7f79f40706b0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:28.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:28.104+0000 7f7a124c7700 1 --2- 172.21.15.144:0/2029947837 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7a0c101320 0x7f7a0c19bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:28.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:28.104+0000 7f7a124c7700 1 --2- 172.21.15.144:0/2029947837 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7a0c101c30 0x7f7a0c19bff0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:28.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:28.104+0000 7f7a124c7700 1 -- 172.21.15.144:0/2029947837 >> 172.21.15.144:0/2029947837 conn(0x7f7a0c0fab80 msgr2=0x7f7a0c104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:28.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:28.104+0000 7f7a124c7700 1 -- 172.21.15.144:0/2029947837 shutdown_connections 2024-09-10T12:03:28.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:28.104+0000 7f7a124c7700 1 -- 172.21.15.144:0/2029947837 wait complete. 2024-09-10T12:03:28.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:27 smithi178 ceph-mon[65940]: from='client.35720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:03:28.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:27 smithi178 ceph-mon[65940]: from='client.35724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:03:28.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:27 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2223664593' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:03:28.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:27 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3668810159' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:03:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:28 smithi178 ceph-mon[65940]: pgmap v945: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:28 smithi178 ceph-mon[65940]: from='client.35736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:03:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:28 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2029947837' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:03:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:03:29.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:28 smithi144 ceph-mon[87436]: pgmap v945: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:29.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:28 smithi144 ceph-mon[87436]: from='client.35736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:03:29.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:28 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2029947837' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:03:29.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:03:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:30 smithi178 ceph-mon[65940]: pgmap v946: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:31.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:30 smithi144 ceph-mon[87436]: pgmap v946: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:32 smithi178 ceph-mon[65940]: pgmap v947: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:33.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:32 smithi144 ceph-mon[87436]: pgmap v947: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:34 smithi178 ceph-mon[65940]: pgmap v948: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:35.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:34 smithi144 ceph-mon[87436]: pgmap v948: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:36 smithi178 ceph-mon[65940]: pgmap v949: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:37.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:36 smithi144 ceph-mon[87436]: pgmap v949: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:39.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:38 smithi178 ceph-mon[65940]: pgmap v950: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:39.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:38 smithi144 ceph-mon[87436]: pgmap v950: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:40 smithi178 ceph-mon[65940]: pgmap v951: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:41.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:40 smithi144 ceph-mon[87436]: pgmap v951: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:43.236 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:42 smithi144 ceph-mon[87436]: pgmap v952: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:42 smithi178 ceph-mon[65940]: pgmap v952: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:44.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:03:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:03:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:44 smithi178 ceph-mon[65940]: pgmap v953: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:44 smithi144 ceph-mon[87436]: pgmap v953: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:46 smithi178 ceph-mon[65940]: pgmap v954: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:46 smithi144 ceph-mon[87436]: pgmap v954: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:48 smithi178 ceph-mon[65940]: pgmap v955: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:49.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:48 smithi144 ceph-mon[87436]: pgmap v955: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:50 smithi178 ceph-mon[65940]: pgmap v956: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:51.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:50 smithi144 ceph-mon[87436]: pgmap v956: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:53.261 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:52 smithi178 ceph-mon[65940]: pgmap v957: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:52 smithi144 ceph-mon[87436]: pgmap v957: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:54 smithi144 ceph-mon[87436]: pgmap v958: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:54 smithi178 ceph-mon[65940]: pgmap v958: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:56.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:56 smithi178 ceph-mon[65940]: pgmap v959: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:56.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:56 smithi144 ceph-mon[87436]: pgmap v959: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:58.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.225+0000 7fb8b5606700 1 -- 172.21.15.144:0/1200809733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b0100d80 msgr2=0x7fb8b0104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:58.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.225+0000 7fb8b5606700 1 --2- 172.21.15.144:0/1200809733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b0100d80 0x7fb8b0104d50 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb898009910 tx=0x7fb898009c20 comp rx=0 tx=0).stop 2024-09-10T12:03:58.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.225+0000 7fb8b5606700 1 -- 172.21.15.144:0/1200809733 shutdown_connections 2024-09-10T12:03:58.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.225+0000 7fb8b5606700 1 --2- 172.21.15.144:0/1200809733 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b0100d80 0x7fb8b0104d50 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:58.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.225+0000 7fb8b5606700 1 --2- 172.21.15.144:0/1200809733 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8b01003e0 0x7fb8b01007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:58.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.225+0000 7fb8b5606700 1 -- 172.21.15.144:0/1200809733 >> 172.21.15.144:0/1200809733 conn(0x7fb8b00fbca0 msgr2=0x7fb8b00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:58.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.225+0000 7fb8b5606700 1 -- 172.21.15.144:0/1200809733 shutdown_connections 2024-09-10T12:03:58.227 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.225+0000 7fb8b5606700 1 -- 172.21.15.144:0/1200809733 wait complete. 2024-09-10T12:03:58.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.226+0000 7fb8b5606700 1 Processor -- start 2024-09-10T12:03:58.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.226+0000 7fb8b5606700 1 -- start start 2024-09-10T12:03:58.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.226+0000 7fb8b5606700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8b01003e0 0x7fb8b0195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:58.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.226+0000 7fb8b5606700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b0100d80 0x7fb8b0196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:58.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.226+0000 7fb8b5606700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0196960 con 0x7fb8b0100d80 2024-09-10T12:03:58.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.226+0000 7fb8b5606700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8b0196aa0 con 0x7fb8b01003e0 2024-09-10T12:03:58.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.226+0000 7fb8a7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b0100d80 0x7fb8b0196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:58.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.226+0000 7fb8a7fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b0100d80 0x7fb8b0196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35786/0 (socket says 172.21.15.144:35786) 2024-09-10T12:03:58.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.226+0000 7fb8a7fff700 1 -- 172.21.15.144:0/3242169500 learned_addr learned my addr 172.21.15.144:0/3242169500 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:03:58.229 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.227+0000 7fb8aeffd700 1 --2- 172.21.15.144:0/3242169500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8b01003e0 0x7fb8b0195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:58.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.227+0000 7fb8a7fff700 1 -- 172.21.15.144:0/3242169500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8b01003e0 msgr2=0x7fb8b0195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:58.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.227+0000 7fb8a7fff700 1 --2- 172.21.15.144:0/3242169500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8b01003e0 0x7fb8b0195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:58.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.227+0000 7fb8a7fff700 1 -- 172.21.15.144:0/3242169500 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8980095f0 con 0x7fb8b0100d80 2024-09-10T12:03:58.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.227+0000 7fb8a7fff700 1 --2- 172.21.15.144:0/3242169500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b0100d80 0x7fb8b0196310 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb89800bda0 tx=0x7fb89800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:58.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.227+0000 7fb8acff9700 1 -- 172.21.15.144:0/3242169500 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb89801d070 con 0x7fb8b0100d80 2024-09-10T12:03:58.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.227+0000 7fb8acff9700 1 -- 172.21.15.144:0/3242169500 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb898004fb0 con 0x7fb8b0100d80 2024-09-10T12:03:58.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.227+0000 7fb8b5606700 1 -- 172.21.15.144:0/3242169500 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8b019a890 con 0x7fb8b0100d80 2024-09-10T12:03:58.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.227+0000 7fb8acff9700 1 -- 172.21.15.144:0/3242169500 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb89800f480 con 0x7fb8b0100d80 2024-09-10T12:03:58.230 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.227+0000 7fb8b5606700 1 -- 172.21.15.144:0/3242169500 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8b019ad80 con 0x7fb8b0100d80 2024-09-10T12:03:58.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.228+0000 7fb8b5606700 1 -- 172.21.15.144:0/3242169500 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8b004ea50 con 0x7fb8b0100d80 2024-09-10T12:03:58.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.229+0000 7fb8acff9700 1 -- 172.21.15.144:0/3242169500 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb898022870 con 0x7fb8b0100d80 2024-09-10T12:03:58.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.229+0000 7fb8acff9700 1 --2- 172.21.15.144:0/3242169500 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb89c06e200 0x7fb89c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:58.232 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.229+0000 7fb8acff9700 1 -- 172.21.15.144:0/3242169500 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb89808f5f0 con 0x7fb8b0100d80 2024-09-10T12:03:58.232 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.229+0000 7fb8aeffd700 1 --2- 172.21.15.144:0/3242169500 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb89c06e200 0x7fb89c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:58.232 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.230+0000 7fb8aeffd700 1 --2- 172.21.15.144:0/3242169500 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb89c06e200 0x7fb89c0706b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000ba60 tx=0x7fb8a0005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:58.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.234+0000 7fb8acff9700 1 -- 172.21.15.144:0/3242169500 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb898058a80 con 0x7fb8b0100d80 2024-09-10T12:03:58.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:58 smithi178 ceph-mon[65940]: pgmap v960: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:58.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:58 smithi144 ceph-mon[87436]: pgmap v960: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:03:58.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.455+0000 7fb8b5606700 1 -- 172.21.15.144:0/3242169500 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8b019b260 con 0x7fb89c06e200 2024-09-10T12:03:58.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.456+0000 7fb8acff9700 1 -- 172.21.15.144:0/3242169500 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb8b019b260 con 0x7fb89c06e200 2024-09-10T12:03:58.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.458+0000 7fb8b5606700 1 -- 172.21.15.144:0/3242169500 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb89c06e200 msgr2=0x7fb89c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:58.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.458+0000 7fb8b5606700 1 --2- 172.21.15.144:0/3242169500 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb89c06e200 0x7fb89c0706b0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000ba60 tx=0x7fb8a0005d50 comp rx=0 tx=0).stop 2024-09-10T12:03:58.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.458+0000 7fb8b5606700 1 -- 172.21.15.144:0/3242169500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b0100d80 msgr2=0x7fb8b0196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:58.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.458+0000 7fb8b5606700 1 --2- 172.21.15.144:0/3242169500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b0100d80 0x7fb8b0196310 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb89800bda0 tx=0x7fb89800be80 comp rx=0 tx=0).stop 2024-09-10T12:03:58.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.458+0000 7fb8b5606700 1 -- 172.21.15.144:0/3242169500 shutdown_connections 2024-09-10T12:03:58.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.458+0000 7fb8b5606700 1 --2- 172.21.15.144:0/3242169500 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb89c06e200 0x7fb89c0706b0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:58.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.458+0000 7fb8b5606700 1 --2- 172.21.15.144:0/3242169500 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb8b01003e0 0x7fb8b0195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:58.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.458+0000 7fb8b5606700 1 --2- 172.21.15.144:0/3242169500 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb8b0100d80 0x7fb8b0196310 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:58.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.458+0000 7fb8b5606700 1 -- 172.21.15.144:0/3242169500 >> 172.21.15.144:0/3242169500 conn(0x7fb8b00fbca0 msgr2=0x7fb8b00fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:58.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.459+0000 7fb8b5606700 1 -- 172.21.15.144:0/3242169500 shutdown_connections 2024-09-10T12:03:58.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.459+0000 7fb8b5606700 1 -- 172.21.15.144:0/3242169500 wait complete. 2024-09-10T12:03:58.470 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:03:58.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.565+0000 7fd53c67c700 1 -- 172.21.15.144:0/3994192166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5340ff2d0 msgr2=0x7fd5340ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:58.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.565+0000 7fd53c67c700 1 --2- 172.21.15.144:0/3994192166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5340ff2d0 0x7fd5340ff6a0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fd524009910 tx=0x7fd524009c20 comp rx=0 tx=0).stop 2024-09-10T12:03:58.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.565+0000 7fd53c67c700 1 -- 172.21.15.144:0/3994192166 shutdown_connections 2024-09-10T12:03:58.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.565+0000 7fd53c67c700 1 --2- 172.21.15.144:0/3994192166 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5340ffc70 0x7fd534103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:58.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.565+0000 7fd53c67c700 1 --2- 172.21.15.144:0/3994192166 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5340ff2d0 0x7fd5340ff6a0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:58.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.565+0000 7fd53c67c700 1 -- 172.21.15.144:0/3994192166 >> 172.21.15.144:0/3994192166 conn(0x7fd5340fabb0 msgr2=0x7fd5340fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:58.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.565+0000 7fd53c67c700 1 -- 172.21.15.144:0/3994192166 shutdown_connections 2024-09-10T12:03:58.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.565+0000 7fd53c67c700 1 -- 172.21.15.144:0/3994192166 wait complete. 2024-09-10T12:03:58.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.566+0000 7fd53c67c700 1 Processor -- start 2024-09-10T12:03:58.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.566+0000 7fd53c67c700 1 -- start start 2024-09-10T12:03:58.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.566+0000 7fd53c67c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5340ff2d0 0x7fd534194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:58.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.566+0000 7fd53c67c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5340ffc70 0x7fd534195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:58.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.566+0000 7fd53c67c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd534195960 con 0x7fd5340ff2d0 2024-09-10T12:03:58.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.566+0000 7fd53c67c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5341996f0 con 0x7fd5340ffc70 2024-09-10T12:03:58.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.566+0000 7fd53a418700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5340ff2d0 0x7fd534194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:58.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.566+0000 7fd53a418700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5340ff2d0 0x7fd534194d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35808/0 (socket says 172.21.15.144:35808) 2024-09-10T12:03:58.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.566+0000 7fd53a418700 1 -- 172.21.15.144:0/975956303 learned_addr learned my addr 172.21.15.144:0/975956303 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:03:58.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.566+0000 7fd539c17700 1 --2- 172.21.15.144:0/975956303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5340ffc70 0x7fd534195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:58.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.566+0000 7fd53a418700 1 -- 172.21.15.144:0/975956303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5340ffc70 msgr2=0x7fd534195280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:58.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.566+0000 7fd53a418700 1 --2- 172.21.15.144:0/975956303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5340ffc70 0x7fd534195280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:58.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.567+0000 7fd53a418700 1 -- 172.21.15.144:0/975956303 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5240095f0 con 0x7fd5340ff2d0 2024-09-10T12:03:58.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.567+0000 7fd539c17700 1 --2- 172.21.15.144:0/975956303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5340ffc70 0x7fd534195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:03:58.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.567+0000 7fd53a418700 1 --2- 172.21.15.144:0/975956303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5340ff2d0 0x7fd534194d40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd524006010 tx=0x7fd52400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:58.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.567+0000 7fd52b7fe700 1 -- 172.21.15.144:0/975956303 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd52401d070 con 0x7fd5340ff2d0 2024-09-10T12:03:58.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.567+0000 7fd53c67c700 1 -- 172.21.15.144:0/975956303 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd534199970 con 0x7fd5340ff2d0 2024-09-10T12:03:58.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.567+0000 7fd52b7fe700 1 -- 172.21.15.144:0/975956303 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd524004fb0 con 0x7fd5340ff2d0 2024-09-10T12:03:58.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.567+0000 7fd52b7fe700 1 -- 172.21.15.144:0/975956303 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd524022ea0 con 0x7fd5340ff2d0 2024-09-10T12:03:58.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.567+0000 7fd53c67c700 1 -- 172.21.15.144:0/975956303 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd534199e60 con 0x7fd5340ff2d0 2024-09-10T12:03:58.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.568+0000 7fd53c67c700 1 -- 172.21.15.144:0/975956303 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5341075e0 con 0x7fd5340ff2d0 2024-09-10T12:03:58.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.568+0000 7fd52b7fe700 1 -- 172.21.15.144:0/975956303 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd524022470 con 0x7fd5340ff2d0 2024-09-10T12:03:58.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.569+0000 7fd52b7fe700 1 --2- 172.21.15.144:0/975956303 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd52006e200 0x7fd5200706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:58.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.569+0000 7fd52b7fe700 1 -- 172.21.15.144:0/975956303 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd52408f610 con 0x7fd5340ff2d0 2024-09-10T12:03:58.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.569+0000 7fd539c17700 1 --2- 172.21.15.144:0/975956303 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd52006e200 0x7fd5200706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:58.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.569+0000 7fd539c17700 1 --2- 172.21.15.144:0/975956303 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd52006e200 0x7fd5200706b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd534196360 tx=0x7fd530008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:58.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.573+0000 7fd52b7fe700 1 -- 172.21.15.144:0/975956303 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd524058aa0 con 0x7fd5340ff2d0 2024-09-10T12:03:58.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.794+0000 7fd53c67c700 1 -- 172.21.15.144:0/975956303 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd53402ccf0 con 0x7fd52006e200 2024-09-10T12:03:58.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.795+0000 7fd52b7fe700 1 -- 172.21.15.144:0/975956303 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd53402ccf0 con 0x7fd52006e200 2024-09-10T12:03:58.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.797+0000 7fd53c67c700 1 -- 172.21.15.144:0/975956303 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd52006e200 msgr2=0x7fd5200706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:58.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.797+0000 7fd53c67c700 1 --2- 172.21.15.144:0/975956303 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd52006e200 0x7fd5200706b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd534196360 tx=0x7fd530008040 comp rx=0 tx=0).stop 2024-09-10T12:03:58.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.797+0000 7fd53c67c700 1 -- 172.21.15.144:0/975956303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5340ff2d0 msgr2=0x7fd534194d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:58.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.798+0000 7fd53c67c700 1 --2- 172.21.15.144:0/975956303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5340ff2d0 0x7fd534194d40 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd524006010 tx=0x7fd52400be20 comp rx=0 tx=0).stop 2024-09-10T12:03:58.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.798+0000 7fd53c67c700 1 -- 172.21.15.144:0/975956303 shutdown_connections 2024-09-10T12:03:58.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.798+0000 7fd53c67c700 1 --2- 172.21.15.144:0/975956303 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd52006e200 0x7fd5200706b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:58.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.798+0000 7fd53c67c700 1 --2- 172.21.15.144:0/975956303 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd5340ff2d0 0x7fd534194d40 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:58.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.798+0000 7fd53c67c700 1 --2- 172.21.15.144:0/975956303 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd5340ffc70 0x7fd534195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:58.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.798+0000 7fd53c67c700 1 -- 172.21.15.144:0/975956303 >> 172.21.15.144:0/975956303 conn(0x7fd5340fabb0 msgr2=0x7fd5340fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:58.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.798+0000 7fd53c67c700 1 -- 172.21.15.144:0/975956303 shutdown_connections 2024-09-10T12:03:58.800 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.798+0000 7fd53c67c700 1 -- 172.21.15.144:0/975956303 wait complete. 2024-09-10T12:03:58.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.903+0000 7f7785886700 1 -- 172.21.15.144:0/3059255259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77800ff2c0 msgr2=0x7f77800ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:58.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.903+0000 7f7785886700 1 --2- 172.21.15.144:0/3059255259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77800ff2c0 0x7f77800ff690 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f7770009910 tx=0x7f7770009c20 comp rx=0 tx=0).stop 2024-09-10T12:03:58.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.903+0000 7f7785886700 1 -- 172.21.15.144:0/3059255259 shutdown_connections 2024-09-10T12:03:58.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.903+0000 7f7785886700 1 --2- 172.21.15.144:0/3059255259 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77800ffc60 0x7f7780103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:58.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.903+0000 7f7785886700 1 --2- 172.21.15.144:0/3059255259 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77800ff2c0 0x7f77800ff690 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:58.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.903+0000 7f7785886700 1 -- 172.21.15.144:0/3059255259 >> 172.21.15.144:0/3059255259 conn(0x7f77800fab80 msgr2=0x7f77800fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:58.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.903+0000 7f7785886700 1 -- 172.21.15.144:0/3059255259 shutdown_connections 2024-09-10T12:03:58.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.903+0000 7f7785886700 1 -- 172.21.15.144:0/3059255259 wait complete. 2024-09-10T12:03:58.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.904+0000 7f7785886700 1 Processor -- start 2024-09-10T12:03:58.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.904+0000 7f7785886700 1 -- start start 2024-09-10T12:03:58.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.904+0000 7f7785886700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77800ff2c0 0x7f7780100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:58.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.904+0000 7f7785886700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77800ffc60 0x7f7780101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:58.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.904+0000 7f7785886700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7780101750 con 0x7f77800ffc60 2024-09-10T12:03:58.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.904+0000 7f7785886700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7780101890 con 0x7f77800ff2c0 2024-09-10T12:03:58.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.904+0000 7f7777fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77800ffc60 0x7f7780101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:58.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.904+0000 7f7777fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77800ffc60 0x7f7780101210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35828/0 (socket says 172.21.15.144:35828) 2024-09-10T12:03:58.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.904+0000 7f7777fff700 1 -- 172.21.15.144:0/2181071645 learned_addr learned my addr 172.21.15.144:0/2181071645 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:03:58.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.904+0000 7f777effd700 1 --2- 172.21.15.144:0/2181071645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77800ff2c0 0x7f7780100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:58.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.904+0000 7f7777fff700 1 -- 172.21.15.144:0/2181071645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77800ff2c0 msgr2=0x7f7780100cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:58.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.904+0000 7f7777fff700 1 --2- 172.21.15.144:0/2181071645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77800ff2c0 0x7f7780100cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:58.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.904+0000 7f7777fff700 1 -- 172.21.15.144:0/2181071645 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77700095f0 con 0x7f77800ffc60 2024-09-10T12:03:58.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.904+0000 7f7777fff700 1 --2- 172.21.15.144:0/2181071645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77800ffc60 0x7f7780101210 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f776800c8f0 tx=0x7f776800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:58.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.905+0000 7f7784884700 1 -- 172.21.15.144:0/2181071645 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7768007a50 con 0x7f77800ffc60 2024-09-10T12:03:58.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.905+0000 7f7784884700 1 -- 172.21.15.144:0/2181071645 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7768007bb0 con 0x7f77800ffc60 2024-09-10T12:03:58.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.905+0000 7f7785886700 1 -- 172.21.15.144:0/2181071645 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77801a1f30 con 0x7f77800ffc60 2024-09-10T12:03:58.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.905+0000 7f7784884700 1 -- 172.21.15.144:0/2181071645 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7768018630 con 0x7f77800ffc60 2024-09-10T12:03:58.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.905+0000 7f7785886700 1 -- 172.21.15.144:0/2181071645 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77801a2340 con 0x7f77800ffc60 2024-09-10T12:03:58.908 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.906+0000 7f7785886700 1 -- 172.21.15.144:0/2181071645 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77801085e0 con 0x7f77800ffc60 2024-09-10T12:03:58.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.906+0000 7f7784884700 1 -- 172.21.15.144:0/2181071645 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7768007d20 con 0x7f77800ffc60 2024-09-10T12:03:58.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.907+0000 7f7784884700 1 --2- 172.21.15.144:0/2181071645 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f776c06e200 0x7f776c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:58.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.907+0000 7f7784884700 1 -- 172.21.15.144:0/2181071645 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f776808dfa0 con 0x7f77800ffc60 2024-09-10T12:03:58.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.907+0000 7f777effd700 1 --2- 172.21.15.144:0/2181071645 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f776c06e200 0x7f776c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:58.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.907+0000 7f777effd700 1 --2- 172.21.15.144:0/2181071645 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f776c06e200 0x7f776c0706b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f777000b5c0 tx=0x7f777001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:58.912 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:58.911+0000 7f7784884700 1 -- 172.21.15.144:0/2181071645 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7768057400 con 0x7f77800ffc60 2024-09-10T12:03:59.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.121+0000 7f7785886700 1 -- 172.21.15.144:0/2181071645 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f778002cd40 con 0x7f776c06e200 2024-09-10T12:03:59.133 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.131+0000 7f7784884700 1 -- 172.21.15.144:0/2181071645 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f778002cd40 con 0x7f776c06e200 2024-09-10T12:03:59.133 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:03:59.133 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (32m) 10m ago 42m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:03:59.133 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (43m) 10m ago 42m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:03:59.133 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (42m) 10m ago 42m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:03:59.133 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (29m) 10m ago 42m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:03:59.133 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (29m) 10m ago 41m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:03:59.133 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (31m) 10m ago 42m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:03:59.134 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (26m) 10m ago 37m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:03:59.134 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (26m) 10m ago 37m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:03:59.134 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (10m) 10m ago 37m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:03:59.134 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (25m) 10m ago 37m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:03:59.134 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (33m) 10m ago 44m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:03:59.134 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (33m) 10m ago 41m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:03:59.134 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (30m) 10m ago 44m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:03:59.134 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (30m) 10m ago 41m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:03:59.134 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (32m) 10m ago 42m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:03:59.134 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (32m) 10m ago 41m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:03:59.135 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (29m) 10m ago 40m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:03:59.135 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (28m) 10m ago 40m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:03:59.135 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (28m) 10m ago 40m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:03:59.135 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (27m) 10m ago 39m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:03:59.135 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (27m) 10m ago 39m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:03:59.135 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (26m) 10m ago 38m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:03:59.135 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (32m) 10m ago 42m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:03:59.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.134+0000 7f7785886700 1 -- 172.21.15.144:0/2181071645 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f776c06e200 msgr2=0x7f776c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:59.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.134+0000 7f7785886700 1 --2- 172.21.15.144:0/2181071645 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f776c06e200 0x7f776c0706b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f777000b5c0 tx=0x7f777001a040 comp rx=0 tx=0).stop 2024-09-10T12:03:59.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.134+0000 7f7785886700 1 -- 172.21.15.144:0/2181071645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77800ffc60 msgr2=0x7f7780101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:59.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.134+0000 7f7785886700 1 --2- 172.21.15.144:0/2181071645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77800ffc60 0x7f7780101210 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f776800c8f0 tx=0x7f776800ccb0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.134+0000 7f7785886700 1 -- 172.21.15.144:0/2181071645 shutdown_connections 2024-09-10T12:03:59.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.134+0000 7f7785886700 1 --2- 172.21.15.144:0/2181071645 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f776c06e200 0x7f776c0706b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.134+0000 7f7785886700 1 --2- 172.21.15.144:0/2181071645 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f77800ff2c0 0x7f7780100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.136 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.134+0000 7f7785886700 1 --2- 172.21.15.144:0/2181071645 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f77800ffc60 0x7f7780101210 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.134+0000 7f7785886700 1 -- 172.21.15.144:0/2181071645 >> 172.21.15.144:0/2181071645 conn(0x7f77800fab80 msgr2=0x7f77800690d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:59.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.134+0000 7f7785886700 1 -- 172.21.15.144:0/2181071645 shutdown_connections 2024-09-10T12:03:59.137 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.134+0000 7f7785886700 1 -- 172.21.15.144:0/2181071645 wait complete. 2024-09-10T12:03:59.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.237+0000 7fce4cddf700 1 -- 172.21.15.144:0/2109335833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce481003b0 msgr2=0x7fce48100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.237+0000 7fce4cddf700 1 --2- 172.21.15.144:0/2109335833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce481003b0 0x7fce48100780 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fce4000b0a0 tx=0x7fce4000b3b0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.237+0000 7fce4cddf700 1 -- 172.21.15.144:0/2109335833 shutdown_connections 2024-09-10T12:03:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.237+0000 7fce4cddf700 1 --2- 172.21.15.144:0/2109335833 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce48100d50 0x7fce48104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.237+0000 7fce4cddf700 1 --2- 172.21.15.144:0/2109335833 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce481003b0 0x7fce48100780 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.237+0000 7fce4cddf700 1 -- 172.21.15.144:0/2109335833 >> 172.21.15.144:0/2109335833 conn(0x7fce480fbc30 msgr2=0x7fce480fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.237+0000 7fce4cddf700 1 -- 172.21.15.144:0/2109335833 shutdown_connections 2024-09-10T12:03:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.237+0000 7fce4cddf700 1 -- 172.21.15.144:0/2109335833 wait complete. 2024-09-10T12:03:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.238+0000 7fce4cddf700 1 Processor -- start 2024-09-10T12:03:59.243 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.238+0000 7fce4cddf700 1 -- start start 2024-09-10T12:03:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.238+0000 7fce4cddf700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce481003b0 0x7fce48073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.238+0000 7fce4cddf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce48100d50 0x7fce48073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.238+0000 7fce4cddf700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce48073ad0 con 0x7fce48100d50 2024-09-10T12:03:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.238+0000 7fce4cddf700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce48073c10 con 0x7fce481003b0 2024-09-10T12:03:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.238+0000 7fce45d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce48100d50 0x7fce48073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.238+0000 7fce45d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce48100d50 0x7fce48073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35844/0 (socket says 172.21.15.144:35844) 2024-09-10T12:03:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.238+0000 7fce45d9b700 1 -- 172.21.15.144:0/2529439114 learned_addr learned my addr 172.21.15.144:0/2529439114 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:03:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.238+0000 7fce4659c700 1 --2- 172.21.15.144:0/2529439114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce481003b0 0x7fce48073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.238+0000 7fce45d9b700 1 -- 172.21.15.144:0/2529439114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce481003b0 msgr2=0x7fce48073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.238+0000 7fce45d9b700 1 --2- 172.21.15.144:0/2529439114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce481003b0 0x7fce48073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.238+0000 7fce45d9b700 1 -- 172.21.15.144:0/2529439114 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce40009d00 con 0x7fce48100d50 2024-09-10T12:03:59.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.239+0000 7fce45d9b700 1 --2- 172.21.15.144:0/2529439114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce48100d50 0x7fce48073590 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fce3000d900 tx=0x7fce3000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:59.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.239+0000 7fce3f7fe700 1 -- 172.21.15.144:0/2529439114 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce30009900 con 0x7fce48100d50 2024-09-10T12:03:59.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.239+0000 7fce3f7fe700 1 -- 172.21.15.144:0/2529439114 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce30010460 con 0x7fce48100d50 2024-09-10T12:03:59.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.239+0000 7fce4cddf700 1 -- 172.21.15.144:0/2529439114 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce481a1ee0 con 0x7fce48100d50 2024-09-10T12:03:59.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.239+0000 7fce3f7fe700 1 -- 172.21.15.144:0/2529439114 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce3000f5f0 con 0x7fce48100d50 2024-09-10T12:03:59.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.239+0000 7fce4cddf700 1 -- 172.21.15.144:0/2529439114 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce481a22f0 con 0x7fce48100d50 2024-09-10T12:03:59.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.240+0000 7fce4cddf700 1 -- 172.21.15.144:0/2529439114 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce481086c0 con 0x7fce48100d50 2024-09-10T12:03:59.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.240+0000 7fce3f7fe700 1 -- 172.21.15.144:0/2529439114 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fce300105d0 con 0x7fce48100d50 2024-09-10T12:03:59.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.241+0000 7fce3f7fe700 1 --2- 172.21.15.144:0/2529439114 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fce3406e200 0x7fce340706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:59.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.241+0000 7fce3f7fe700 1 -- 172.21.15.144:0/2529439114 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fce3008dc60 con 0x7fce48100d50 2024-09-10T12:03:59.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.241+0000 7fce4659c700 1 --2- 172.21.15.144:0/2529439114 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fce3406e200 0x7fce340706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:59.245 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.241+0000 7fce4659c700 1 --2- 172.21.15.144:0/2529439114 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fce3406e200 0x7fce340706b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fce40000f80 tx=0x7fce400060c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:59.246 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.245+0000 7fce3f7fe700 1 -- 172.21.15.144:0/2529439114 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fce30057040 con 0x7fce48100d50 2024-09-10T12:03:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:59 smithi178 ceph-mon[65940]: from='client.35744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:03:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:59 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:03:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:03:59 smithi178 ceph-mon[65940]: from='client.35748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:03:59.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:59 smithi144 ceph-mon[87436]: from='client.35744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:03:59.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:59 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:03:59.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:03:59 smithi144 ceph-mon[87436]: from='client.35748 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:03:59.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.532+0000 7fce4cddf700 1 -- 172.21.15.144:0/2529439114 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fce4804ea50 con 0x7fce48100d50 2024-09-10T12:03:59.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.532+0000 7fce3f7fe700 1 -- 172.21.15.144:0/2529439114 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fce3005a700 con 0x7fce48100d50 2024-09-10T12:03:59.534 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:03:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:03:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:03:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:03:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:03:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:03:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:03:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:03:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:03:59.534 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:03:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:03:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:03:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:03:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:03:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:03:59.535 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:03:59.535 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:03:59.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.535+0000 7fce4cddf700 1 -- 172.21.15.144:0/2529439114 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fce3406e200 msgr2=0x7fce340706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:59.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.535+0000 7fce4cddf700 1 --2- 172.21.15.144:0/2529439114 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fce3406e200 0x7fce340706b0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fce40000f80 tx=0x7fce400060c0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.535+0000 7fce4cddf700 1 -- 172.21.15.144:0/2529439114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce48100d50 msgr2=0x7fce48073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:59.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.535+0000 7fce4cddf700 1 --2- 172.21.15.144:0/2529439114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce48100d50 0x7fce48073590 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fce3000d900 tx=0x7fce3000dc10 comp rx=0 tx=0).stop 2024-09-10T12:03:59.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.535+0000 7fce4cddf700 1 -- 172.21.15.144:0/2529439114 shutdown_connections 2024-09-10T12:03:59.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.535+0000 7fce4cddf700 1 --2- 172.21.15.144:0/2529439114 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fce3406e200 0x7fce340706b0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.535+0000 7fce4cddf700 1 --2- 172.21.15.144:0/2529439114 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce481003b0 0x7fce48073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.535+0000 7fce4cddf700 1 --2- 172.21.15.144:0/2529439114 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce48100d50 0x7fce48073590 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.535+0000 7fce4cddf700 1 -- 172.21.15.144:0/2529439114 >> 172.21.15.144:0/2529439114 conn(0x7fce480fbc30 msgr2=0x7fce480fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:59.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.535+0000 7fce4cddf700 1 -- 172.21.15.144:0/2529439114 shutdown_connections 2024-09-10T12:03:59.537 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.535+0000 7fce4cddf700 1 -- 172.21.15.144:0/2529439114 wait complete. 2024-09-10T12:03:59.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.640+0000 7f6fdd1cc700 1 -- 172.21.15.144:0/2870827434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fd80ff2c0 msgr2=0x7f6fd80ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:59.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.640+0000 7f6fdd1cc700 1 --2- 172.21.15.144:0/2870827434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fd80ff2c0 0x7f6fd80ff690 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0009910 tx=0x7f6fd0009c20 comp rx=0 tx=0).stop 2024-09-10T12:03:59.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.640+0000 7f6fdd1cc700 1 -- 172.21.15.144:0/2870827434 shutdown_connections 2024-09-10T12:03:59.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.640+0000 7f6fdd1cc700 1 --2- 172.21.15.144:0/2870827434 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fd80ffc60 0x7f6fd8103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.640+0000 7f6fdd1cc700 1 --2- 172.21.15.144:0/2870827434 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fd80ff2c0 0x7f6fd80ff690 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.640+0000 7f6fdd1cc700 1 -- 172.21.15.144:0/2870827434 >> 172.21.15.144:0/2870827434 conn(0x7f6fd80fab80 msgr2=0x7f6fd80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:59.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.640+0000 7f6fdd1cc700 1 -- 172.21.15.144:0/2870827434 shutdown_connections 2024-09-10T12:03:59.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.640+0000 7f6fdd1cc700 1 -- 172.21.15.144:0/2870827434 wait complete. 2024-09-10T12:03:59.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.641+0000 7f6fdd1cc700 1 Processor -- start 2024-09-10T12:03:59.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.641+0000 7f6fdd1cc700 1 -- start start 2024-09-10T12:03:59.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.641+0000 7f6fdd1cc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fd80ff2c0 0x7f6fd8195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:59.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.641+0000 7f6fdd1cc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fd80ffc60 0x7f6fd81962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:59.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.641+0000 7f6fdd1cc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd8196990 con 0x7f6fd80ff2c0 2024-09-10T12:03:59.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.641+0000 7f6fdd1cc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd819a6d0 con 0x7f6fd80ffc60 2024-09-10T12:03:59.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.641+0000 7f6fd6d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fd80ff2c0 0x7f6fd8195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:59.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.641+0000 7f6fd6d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fd80ff2c0 0x7f6fd8195d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35856/0 (socket says 172.21.15.144:35856) 2024-09-10T12:03:59.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.641+0000 7f6fd6d9d700 1 -- 172.21.15.144:0/3699469752 learned_addr learned my addr 172.21.15.144:0/3699469752 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:03:59.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.641+0000 7f6fd659c700 1 --2- 172.21.15.144:0/3699469752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fd80ffc60 0x7f6fd81962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:59.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.641+0000 7f6fd6d9d700 1 -- 172.21.15.144:0/3699469752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fd80ffc60 msgr2=0x7f6fd81962b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:59.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.641+0000 7f6fd6d9d700 1 --2- 172.21.15.144:0/3699469752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fd80ffc60 0x7f6fd81962b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.641+0000 7f6fd6d9d700 1 -- 172.21.15.144:0/3699469752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd00095f0 con 0x7f6fd80ff2c0 2024-09-10T12:03:59.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.641+0000 7f6fd659c700 1 --2- 172.21.15.144:0/3699469752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fd80ffc60 0x7f6fd81962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:03:59.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.642+0000 7f6fd6d9d700 1 --2- 172.21.15.144:0/3699469752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fd80ff2c0 0x7f6fd8195d70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0005f50 tx=0x7f6fd000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:59.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.642+0000 7f6fcffff700 1 -- 172.21.15.144:0/3699469752 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fd001d070 con 0x7f6fd80ff2c0 2024-09-10T12:03:59.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.642+0000 7f6fdd1cc700 1 -- 172.21.15.144:0/3699469752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd819a950 con 0x7f6fd80ff2c0 2024-09-10T12:03:59.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.642+0000 7f6fcffff700 1 -- 172.21.15.144:0/3699469752 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6fd000b610 con 0x7f6fd80ff2c0 2024-09-10T12:03:59.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.642+0000 7f6fcffff700 1 -- 172.21.15.144:0/3699469752 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6fd0022470 con 0x7f6fd80ff2c0 2024-09-10T12:03:59.645 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.642+0000 7f6fdd1cc700 1 -- 172.21.15.144:0/3699469752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd819ae40 con 0x7f6fd80ff2c0 2024-09-10T12:03:59.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.643+0000 7f6fdd1cc700 1 -- 172.21.15.144:0/3699469752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd81085e0 con 0x7f6fd80ff2c0 2024-09-10T12:03:59.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.643+0000 7f6fcffff700 1 -- 172.21.15.144:0/3699469752 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6fd0003680 con 0x7f6fd80ff2c0 2024-09-10T12:03:59.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.644+0000 7f6fcffff700 1 --2- 172.21.15.144:0/3699469752 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6fc406e200 0x7f6fc40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:03:59.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.644+0000 7f6fcffff700 1 -- 172.21.15.144:0/3699469752 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6fd008f510 con 0x7f6fd80ff2c0 2024-09-10T12:03:59.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.644+0000 7f6fd659c700 1 --2- 172.21.15.144:0/3699469752 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6fc406e200 0x7f6fc40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:03:59.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.644+0000 7f6fd659c700 1 --2- 172.21.15.144:0/3699469752 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6fc406e200 0x7f6fc40706b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8197340 tx=0x7f6fc0009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:03:59.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.648+0000 7f6fcffff700 1 -- 172.21.15.144:0/3699469752 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6fd00588f0 con 0x7f6fd80ff2c0 2024-09-10T12:03:59.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.915+0000 7f6fdd1cc700 1 -- 172.21.15.144:0/3699469752 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6fd802ccf0 con 0x7f6fd80ff2c0 2024-09-10T12:03:59.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.915+0000 7f6fcffff700 1 -- 172.21.15.144:0/3699469752 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f6fd005bfb0 con 0x7f6fd80ff2c0 2024-09-10T12:03:59.917 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:03:59.917 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:03:59.917 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:03:59.917 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:03:59.917 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:03:59.917 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:03:59.918 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:03:59.918 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:03:59.918 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:03:59.918 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:03:59.918 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:03:59.918 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:03:59.918 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:03:59.918 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:03:59.918 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:03:59.918 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:03:59.918 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:03:59.918 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:03:59.918 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:03:59.918 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:03:59.918 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:03:59.919 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:03:59.919 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:03:59.919 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:03:59.919 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:03:59.919 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:03:59.919 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:03:59.919 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:03:59.919 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:03:59.919 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:03:59.919 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:03:59.919 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:03:59.919 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:03:59.919 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:03:59.920 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:03:59.920 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:03:59.920 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:03:59.920 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:03:59.920 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:03:59.920 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:03:59.920 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:03:59.920 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:03:59.920 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:03:59.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.918+0000 7f6fdd1cc700 1 -- 172.21.15.144:0/3699469752 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6fc406e200 msgr2=0x7f6fc40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:59.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.918+0000 7f6fdd1cc700 1 --2- 172.21.15.144:0/3699469752 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6fc406e200 0x7f6fc40706b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f6fd8197340 tx=0x7f6fc0009260 comp rx=0 tx=0).stop 2024-09-10T12:03:59.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.918+0000 7f6fdd1cc700 1 -- 172.21.15.144:0/3699469752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fd80ff2c0 msgr2=0x7f6fd8195d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:03:59.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.918+0000 7f6fdd1cc700 1 --2- 172.21.15.144:0/3699469752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fd80ff2c0 0x7f6fd8195d70 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0005f50 tx=0x7f6fd000f740 comp rx=0 tx=0).stop 2024-09-10T12:03:59.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.918+0000 7f6fdd1cc700 1 -- 172.21.15.144:0/3699469752 shutdown_connections 2024-09-10T12:03:59.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.918+0000 7f6fdd1cc700 1 --2- 172.21.15.144:0/3699469752 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6fc406e200 0x7f6fc40706b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.918+0000 7f6fdd1cc700 1 --2- 172.21.15.144:0/3699469752 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6fd80ff2c0 0x7f6fd8195d70 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.918+0000 7f6fdd1cc700 1 --2- 172.21.15.144:0/3699469752 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6fd80ffc60 0x7f6fd81962b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:03:59.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.918+0000 7f6fdd1cc700 1 -- 172.21.15.144:0/3699469752 >> 172.21.15.144:0/3699469752 conn(0x7f6fd80fab80 msgr2=0x7f6fd8068ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:03:59.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.918+0000 7f6fdd1cc700 1 -- 172.21.15.144:0/3699469752 shutdown_connections 2024-09-10T12:03:59.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:03:59.918+0000 7f6fdd1cc700 1 -- 172.21.15.144:0/3699469752 wait complete. 2024-09-10T12:03:59.922 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:04:00.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.023+0000 7f2135952700 1 -- 172.21.15.144:0/1104630899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21300ffc60 msgr2=0x7f2130103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:00.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.023+0000 7f2135952700 1 --2- 172.21.15.144:0/1104630899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21300ffc60 0x7f2130103c30 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f2124009910 tx=0x7f2124009c20 comp rx=0 tx=0).stop 2024-09-10T12:04:00.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.023+0000 7f2135952700 1 -- 172.21.15.144:0/1104630899 shutdown_connections 2024-09-10T12:04:00.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.023+0000 7f2135952700 1 --2- 172.21.15.144:0/1104630899 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21300ffc60 0x7f2130103c30 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:00.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.023+0000 7f2135952700 1 --2- 172.21.15.144:0/1104630899 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21300ff2c0 0x7f21300ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:00.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.023+0000 7f2135952700 1 -- 172.21.15.144:0/1104630899 >> 172.21.15.144:0/1104630899 conn(0x7f21300fab80 msgr2=0x7f21300fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:00.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.023+0000 7f2135952700 1 -- 172.21.15.144:0/1104630899 shutdown_connections 2024-09-10T12:04:00.025 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.023+0000 7f2135952700 1 -- 172.21.15.144:0/1104630899 wait complete. 2024-09-10T12:04:00.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.024+0000 7f2135952700 1 Processor -- start 2024-09-10T12:04:00.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.024+0000 7f2135952700 1 -- start start 2024-09-10T12:04:00.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.024+0000 7f2135952700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21300ff2c0 0x7f2130196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:00.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.024+0000 7f2135952700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21300ffc60 0x7f21301970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:00.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.024+0000 7f2135952700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2130197700 con 0x7f21300ff2c0 2024-09-10T12:04:00.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.024+0000 7f2135952700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2130197840 con 0x7f21300ffc60 2024-09-10T12:04:00.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.025+0000 7f212effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21300ff2c0 0x7f2130196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:00.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.025+0000 7f212effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21300ff2c0 0x7f2130196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35882/0 (socket says 172.21.15.144:35882) 2024-09-10T12:04:00.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.025+0000 7f212effd700 1 -- 172.21.15.144:0/3269790859 learned_addr learned my addr 172.21.15.144:0/3269790859 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:04:00.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.025+0000 7f212e7fc700 1 --2- 172.21.15.144:0/3269790859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21300ffc60 0x7f21301970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:00.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.025+0000 7f212effd700 1 -- 172.21.15.144:0/3269790859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21300ffc60 msgr2=0x7f21301970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:00.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.025+0000 7f212effd700 1 --2- 172.21.15.144:0/3269790859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21300ffc60 0x7f21301970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:00.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.025+0000 7f212effd700 1 -- 172.21.15.144:0/3269790859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21240095f0 con 0x7f21300ff2c0 2024-09-10T12:04:00.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.025+0000 7f212effd700 1 --2- 172.21.15.144:0/3269790859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21300ff2c0 0x7f2130196ba0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f212000d8d0 tx=0x7f212000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:00.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.025+0000 7f2134950700 1 -- 172.21.15.144:0/3269790859 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21200098a0 con 0x7f21300ff2c0 2024-09-10T12:04:00.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.025+0000 7f2135952700 1 -- 172.21.15.144:0/3269790859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f213019fc10 con 0x7f21300ff2c0 2024-09-10T12:04:00.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.025+0000 7f2134950700 1 -- 172.21.15.144:0/3269790859 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2120010460 con 0x7f21300ff2c0 2024-09-10T12:04:00.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.025+0000 7f2134950700 1 -- 172.21.15.144:0/3269790859 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f212000f5f0 con 0x7f21300ff2c0 2024-09-10T12:04:00.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.025+0000 7f2135952700 1 -- 172.21.15.144:0/3269790859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21301a00d0 con 0x7f21300ff2c0 2024-09-10T12:04:00.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.026+0000 7f2135952700 1 -- 172.21.15.144:0/3269790859 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2130107630 con 0x7f21300ff2c0 2024-09-10T12:04:00.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.027+0000 7f2134950700 1 -- 172.21.15.144:0/3269790859 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2120009a00 con 0x7f21300ff2c0 2024-09-10T12:04:00.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.027+0000 7f2134950700 1 --2- 172.21.15.144:0/3269790859 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f211806e200 0x7f21180706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:00.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.028+0000 7f2134950700 1 -- 172.21.15.144:0/3269790859 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f212008df50 con 0x7f21300ff2c0 2024-09-10T12:04:00.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.028+0000 7f212e7fc700 1 --2- 172.21.15.144:0/3269790859 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f211806e200 0x7f21180706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:00.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.028+0000 7f212e7fc700 1 --2- 172.21.15.144:0/3269790859 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f211806e200 0x7f21180706b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2124006010 tx=0x7f212401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:00.033 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.032+0000 7f2134950700 1 -- 172.21.15.144:0/3269790859 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2120057460 con 0x7f21300ff2c0 2024-09-10T12:04:00.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:00 smithi144 ceph-mon[87436]: pgmap v961: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:00.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:00 smithi144 ceph-mon[87436]: from='client.35752 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:04:00.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:00 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2529439114' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:04:00.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:00 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3699469752' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:04:00.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.257+0000 7f2135952700 1 -- 172.21.15.144:0/3269790859 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f213002cca0 con 0x7f211806e200 2024-09-10T12:04:00.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.258+0000 7f2134950700 1 -- 172.21.15.144:0/3269790859 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f213002cca0 con 0x7f211806e200 2024-09-10T12:04:00.260 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:04:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:04:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:04:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:04:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:04:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:04:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:04:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:04:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:04:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:04:00.260 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:04:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:04:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:04:00.261 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:04:00.261 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:04:00.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.261+0000 7f2135952700 1 -- 172.21.15.144:0/3269790859 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f211806e200 msgr2=0x7f21180706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:00.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.261+0000 7f2135952700 1 --2- 172.21.15.144:0/3269790859 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f211806e200 0x7f21180706b0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f2124006010 tx=0x7f212401a040 comp rx=0 tx=0).stop 2024-09-10T12:04:00.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.261+0000 7f2135952700 1 -- 172.21.15.144:0/3269790859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21300ff2c0 msgr2=0x7f2130196ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:00.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.261+0000 7f2135952700 1 --2- 172.21.15.144:0/3269790859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21300ff2c0 0x7f2130196ba0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f212000d8d0 tx=0x7f212000dbe0 comp rx=0 tx=0).stop 2024-09-10T12:04:00.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.261+0000 7f2135952700 1 -- 172.21.15.144:0/3269790859 shutdown_connections 2024-09-10T12:04:00.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.261+0000 7f2135952700 1 --2- 172.21.15.144:0/3269790859 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f211806e200 0x7f21180706b0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:00.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.261+0000 7f2135952700 1 --2- 172.21.15.144:0/3269790859 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21300ff2c0 0x7f2130196ba0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:00.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.261+0000 7f2135952700 1 --2- 172.21.15.144:0/3269790859 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21300ffc60 0x7f21301970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:00.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.261+0000 7f2135952700 1 -- 172.21.15.144:0/3269790859 >> 172.21.15.144:0/3269790859 conn(0x7f21300fab80 msgr2=0x7f21300fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:00.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.261+0000 7f2135952700 1 -- 172.21.15.144:0/3269790859 shutdown_connections 2024-09-10T12:04:00.263 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.261+0000 7f2135952700 1 -- 172.21.15.144:0/3269790859 wait complete. 2024-09-10T12:04:00.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:00 smithi178 ceph-mon[65940]: pgmap v961: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:00.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:00 smithi178 ceph-mon[65940]: from='client.35752 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:04:00.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:00 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2529439114' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:04:00.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:00 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3699469752' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:04:00.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.366+0000 7f21c48c8700 1 -- 172.21.15.144:0/3889425978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c0100dd0 msgr2=0x7f21c0104da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:00.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.366+0000 7f21c48c8700 1 --2- 172.21.15.144:0/3889425978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c0100dd0 0x7f21c0104da0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009910 tx=0x7f21b0009c20 comp rx=0 tx=0).stop 2024-09-10T12:04:00.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.366+0000 7f21c48c8700 1 -- 172.21.15.144:0/3889425978 shutdown_connections 2024-09-10T12:04:00.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.366+0000 7f21c48c8700 1 --2- 172.21.15.144:0/3889425978 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c0100dd0 0x7f21c0104da0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:00.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.366+0000 7f21c48c8700 1 --2- 172.21.15.144:0/3889425978 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21c0100430 0x7f21c0100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:00.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.366+0000 7f21c48c8700 1 -- 172.21.15.144:0/3889425978 >> 172.21.15.144:0/3889425978 conn(0x7f21c00fbcf0 msgr2=0x7f21c00fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:00.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.366+0000 7f21c48c8700 1 -- 172.21.15.144:0/3889425978 shutdown_connections 2024-09-10T12:04:00.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.366+0000 7f21c48c8700 1 -- 172.21.15.144:0/3889425978 wait complete. 2024-09-10T12:04:00.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.367+0000 7f21c48c8700 1 Processor -- start 2024-09-10T12:04:00.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.367+0000 7f21c48c8700 1 -- start start 2024-09-10T12:04:00.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.367+0000 7f21c48c8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c0100430 0x7f21c0195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:00.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.367+0000 7f21c48c8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21c0100dd0 0x7f21c01962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:00.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.367+0000 7f21c48c8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c0196980 con 0x7f21c0100430 2024-09-10T12:04:00.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.367+0000 7f21c48c8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c019a710 con 0x7f21c0100dd0 2024-09-10T12:04:00.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.367+0000 7f21be59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c0100430 0x7f21c0195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:00.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.367+0000 7f21be59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c0100430 0x7f21c0195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35896/0 (socket says 172.21.15.144:35896) 2024-09-10T12:04:00.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.367+0000 7f21be59c700 1 -- 172.21.15.144:0/2562288056 learned_addr learned my addr 172.21.15.144:0/2562288056 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:04:00.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.368+0000 7f21bdd9b700 1 --2- 172.21.15.144:0/2562288056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21c0100dd0 0x7f21c01962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:00.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.368+0000 7f21be59c700 1 -- 172.21.15.144:0/2562288056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21c0100dd0 msgr2=0x7f21c01962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:00.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.368+0000 7f21be59c700 1 --2- 172.21.15.144:0/2562288056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21c0100dd0 0x7f21c01962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:00.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.368+0000 7f21be59c700 1 -- 172.21.15.144:0/2562288056 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21b00095f0 con 0x7f21c0100430 2024-09-10T12:04:00.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.368+0000 7f21be59c700 1 --2- 172.21.15.144:0/2562288056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c0100430 0x7f21c0195d60 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f21b400c8a0 tx=0x7f21b400cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:00.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.368+0000 7f21af7fe700 1 -- 172.21.15.144:0/2562288056 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21b40078e0 con 0x7f21c0100430 2024-09-10T12:04:00.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.368+0000 7f21c48c8700 1 -- 172.21.15.144:0/2562288056 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c019a9f0 con 0x7f21c0100430 2024-09-10T12:04:00.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.368+0000 7f21af7fe700 1 -- 172.21.15.144:0/2562288056 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f21b400f450 con 0x7f21c0100430 2024-09-10T12:04:00.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.368+0000 7f21af7fe700 1 -- 172.21.15.144:0/2562288056 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21b400e5e0 con 0x7f21c0100430 2024-09-10T12:04:00.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.368+0000 7f21c48c8700 1 -- 172.21.15.144:0/2562288056 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c019af10 con 0x7f21c0100430 2024-09-10T12:04:00.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.369+0000 7f21c48c8700 1 -- 172.21.15.144:0/2562288056 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c01087a0 con 0x7f21c0100430 2024-09-10T12:04:00.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.370+0000 7f21af7fe700 1 -- 172.21.15.144:0/2562288056 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f21b400f5c0 con 0x7f21c0100430 2024-09-10T12:04:00.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.370+0000 7f21af7fe700 1 --2- 172.21.15.144:0/2562288056 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f21a806e200 0x7f21a80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:00.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.370+0000 7f21af7fe700 1 -- 172.21.15.144:0/2562288056 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f21b408e0b0 con 0x7f21c0100430 2024-09-10T12:04:00.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.370+0000 7f21bdd9b700 1 --2- 172.21.15.144:0/2562288056 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f21a806e200 0x7f21a80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:00.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.371+0000 7f21bdd9b700 1 --2- 172.21.15.144:0/2562288056 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f21a806e200 0x7f21a80706b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f21b0006010 tx=0x7f21b001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:00.376 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.375+0000 7f21af7fe700 1 -- 172.21.15.144:0/2562288056 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f21b4057490 con 0x7f21c0100430 2024-09-10T12:04:00.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.658+0000 7f21c48c8700 1 -- 172.21.15.144:0/2562288056 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f21c004ea50 con 0x7f21c0100430 2024-09-10T12:04:00.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.658+0000 7f21af7fe700 1 -- 172.21.15.144:0/2562288056 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f21c004ea50 con 0x7f21c0100430 2024-09-10T12:04:00.660 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:04:00.660 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:04:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:04:00.660 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:04:00.660 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:04:00.660 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:04:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:04:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:04:00.660 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:04:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:04:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:04:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:04:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:04:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:04:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:04:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:04:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:04:00.661 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:04:00.661 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:04:00.661 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:04:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:04:00.662 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:04:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:04:00.662 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:04:00.662 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:04:00.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.661+0000 7f21c48c8700 1 -- 172.21.15.144:0/2562288056 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f21a806e200 msgr2=0x7f21a80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:00.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.661+0000 7f21c48c8700 1 --2- 172.21.15.144:0/2562288056 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f21a806e200 0x7f21a80706b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f21b0006010 tx=0x7f21b001a040 comp rx=0 tx=0).stop 2024-09-10T12:04:00.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.661+0000 7f21c48c8700 1 -- 172.21.15.144:0/2562288056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c0100430 msgr2=0x7f21c0195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:00.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.661+0000 7f21c48c8700 1 --2- 172.21.15.144:0/2562288056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c0100430 0x7f21c0195d60 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f21b400c8a0 tx=0x7f21b400cbb0 comp rx=0 tx=0).stop 2024-09-10T12:04:00.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.662+0000 7f21c48c8700 1 -- 172.21.15.144:0/2562288056 shutdown_connections 2024-09-10T12:04:00.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.662+0000 7f21c48c8700 1 --2- 172.21.15.144:0/2562288056 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f21a806e200 0x7f21a80706b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:00.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.662+0000 7f21c48c8700 1 --2- 172.21.15.144:0/2562288056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f21c0100430 0x7f21c0195d60 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:00.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.662+0000 7f21c48c8700 1 --2- 172.21.15.144:0/2562288056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f21c0100dd0 0x7f21c01962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:00.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.662+0000 7f21c48c8700 1 -- 172.21.15.144:0/2562288056 >> 172.21.15.144:0/2562288056 conn(0x7f21c00fbcf0 msgr2=0x7f21c00fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:00.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.662+0000 7f21c48c8700 1 -- 172.21.15.144:0/2562288056 shutdown_connections 2024-09-10T12:04:00.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:00.662+0000 7f21c48c8700 1 -- 172.21.15.144:0/2562288056 wait complete. 2024-09-10T12:04:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:01 smithi178 ceph-mon[65940]: from='client.35764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:04:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:01 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2562288056' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:04:01.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:01 smithi144 ceph-mon[87436]: from='client.35764 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:04:01.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:01 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2562288056' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:04:02.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:02 smithi178 ceph-mon[65940]: pgmap v962: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:02.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:02 smithi144 ceph-mon[87436]: pgmap v962: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:04 smithi178 ceph-mon[65940]: pgmap v963: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:04.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:04 smithi144 ceph-mon[87436]: pgmap v963: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:06.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:06 smithi178 ceph-mon[65940]: pgmap v964: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:06.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:06 smithi144 ceph-mon[87436]: pgmap v964: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:08.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:08 smithi178 ceph-mon[65940]: pgmap v965: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:08.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:08 smithi144 ceph-mon[87436]: pgmap v965: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:10.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:10 smithi178 ceph-mon[65940]: pgmap v966: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:10.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:10 smithi144 ceph-mon[87436]: pgmap v966: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:12.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:12 smithi178 ceph-mon[65940]: pgmap v967: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:12.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:12 smithi144 ceph-mon[87436]: pgmap v967: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:14 smithi178 ceph-mon[65940]: pgmap v968: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:14 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:04:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:14 smithi144 ceph-mon[87436]: pgmap v968: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:04:16.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:16 smithi144 ceph-mon[87436]: pgmap v969: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:16.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:16 smithi178 ceph-mon[65940]: pgmap v969: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:18.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:18 smithi144 ceph-mon[87436]: pgmap v970: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:18.520 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:18 smithi178 ceph-mon[65940]: pgmap v970: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:20.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:20 smithi144 ceph-mon[87436]: pgmap v971: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:20.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:20 smithi178 ceph-mon[65940]: pgmap v971: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:22.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:22 smithi144 ceph-mon[87436]: pgmap v972: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:22.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:22 smithi178 ceph-mon[65940]: pgmap v972: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:24.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:24 smithi144 ceph-mon[87436]: pgmap v973: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:24.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:24 smithi178 ceph-mon[65940]: pgmap v973: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:26.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:26 smithi144 ceph-mon[87436]: pgmap v974: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:26.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:26 smithi178 ceph-mon[65940]: pgmap v974: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:28.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:28 smithi144 ceph-mon[87436]: pgmap v975: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:28.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:28 smithi178 ceph-mon[65940]: pgmap v975: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:29.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:29 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:04:29.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:29 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:04:30.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:30 smithi144 ceph-mon[87436]: pgmap v976: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:30.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:30 smithi178 ceph-mon[65940]: pgmap v976: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:30.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.779+0000 7f93b70e6700 1 -- 172.21.15.144:0/1680779872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93b0068fd0 msgr2=0x7f93b0104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:30.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.779+0000 7f93b70e6700 1 --2- 172.21.15.144:0/1680779872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93b0068fd0 0x7f93b0104df0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f93a000b0a0 tx=0x7f93a000b3b0 comp rx=0 tx=0).stop 2024-09-10T12:04:30.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.779+0000 7f93b70e6700 1 -- 172.21.15.144:0/1680779872 shutdown_connections 2024-09-10T12:04:30.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.779+0000 7f93b70e6700 1 --2- 172.21.15.144:0/1680779872 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93b0068fd0 0x7f93b0104df0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:30.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.779+0000 7f93b70e6700 1 --2- 172.21.15.144:0/1680779872 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b0068630 0x7f93b0068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:30.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.779+0000 7f93b70e6700 1 -- 172.21.15.144:0/1680779872 >> 172.21.15.144:0/1680779872 conn(0x7f93b0074f50 msgr2=0x7f93b0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:30.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.779+0000 7f93b70e6700 1 -- 172.21.15.144:0/1680779872 shutdown_connections 2024-09-10T12:04:30.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.779+0000 7f93b70e6700 1 -- 172.21.15.144:0/1680779872 wait complete. 2024-09-10T12:04:30.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.780+0000 7f93b70e6700 1 Processor -- start 2024-09-10T12:04:30.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.780+0000 7f93b70e6700 1 -- start start 2024-09-10T12:04:30.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93b70e6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b0068630 0x7f93b0195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:30.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93b70e6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93b0068fd0 0x7f93b0196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:30.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93b70e6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b0196a40 con 0x7f93b0068fd0 2024-09-10T12:04:30.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93b70e6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b019a7d0 con 0x7f93b0068630 2024-09-10T12:04:30.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93affff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93b0068fd0 0x7f93b0196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:30.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93affff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93b0068fd0 0x7f93b0196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41650/0 (socket says 172.21.15.144:41650) 2024-09-10T12:04:30.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93affff700 1 -- 172.21.15.144:0/3027582939 learned_addr learned my addr 172.21.15.144:0/3027582939 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:04:30.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93b4e82700 1 --2- 172.21.15.144:0/3027582939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b0068630 0x7f93b0195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:30.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93affff700 1 -- 172.21.15.144:0/3027582939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b0068630 msgr2=0x7f93b0195e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:30.785 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93affff700 1 --2- 172.21.15.144:0/3027582939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b0068630 0x7f93b0195e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:30.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93affff700 1 -- 172.21.15.144:0/3027582939 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a0009d00 con 0x7f93b0068fd0 2024-09-10T12:04:30.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93affff700 1 --2- 172.21.15.144:0/3027582939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93b0068fd0 0x7f93b0196360 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f93a0006130 tx=0x7f93a000cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:30.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93adffb700 1 -- 172.21.15.144:0/3027582939 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93a000e040 con 0x7f93b0068fd0 2024-09-10T12:04:30.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93b70e6700 1 -- 172.21.15.144:0/3027582939 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b019aa50 con 0x7f93b0068fd0 2024-09-10T12:04:30.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93adffb700 1 -- 172.21.15.144:0/3027582939 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f93a0003c10 con 0x7f93b0068fd0 2024-09-10T12:04:30.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.781+0000 7f93adffb700 1 -- 172.21.15.144:0/3027582939 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93a001d6b0 con 0x7f93b0068fd0 2024-09-10T12:04:30.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.782+0000 7f93b70e6700 1 -- 172.21.15.144:0/3027582939 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b019af40 con 0x7f93b0068fd0 2024-09-10T12:04:30.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.783+0000 7f93b70e6700 1 -- 172.21.15.144:0/3027582939 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b01086d0 con 0x7f93b0068fd0 2024-09-10T12:04:30.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.783+0000 7f93adffb700 1 -- 172.21.15.144:0/3027582939 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f93a0004600 con 0x7f93b0068fd0 2024-09-10T12:04:30.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.784+0000 7f93adffb700 1 --2- 172.21.15.144:0/3027582939 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f939c06e200 0x7f939c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:30.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.784+0000 7f93b4e82700 1 --2- 172.21.15.144:0/3027582939 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f939c06e200 0x7f939c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:30.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.784+0000 7f93adffb700 1 -- 172.21.15.144:0/3027582939 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f93a008f8c0 con 0x7f93b0068fd0 2024-09-10T12:04:30.787 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.785+0000 7f93b4e82700 1 --2- 172.21.15.144:0/3027582939 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f939c06e200 0x7f939c0706b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f93a8005950 tx=0x7f93a800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:30.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:30.789+0000 7f93adffb700 1 -- 172.21.15.144:0/3027582939 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f93a0058c40 con 0x7f93b0068fd0 2024-09-10T12:04:31.012 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.009+0000 7f93b70e6700 1 -- 172.21.15.144:0/3027582939 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f93b019b220 con 0x7f939c06e200 2024-09-10T12:04:31.013 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.011+0000 7f93adffb700 1 -- 172.21.15.144:0/3027582939 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f93b019b220 con 0x7f939c06e200 2024-09-10T12:04:31.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.013+0000 7f93b70e6700 1 -- 172.21.15.144:0/3027582939 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f939c06e200 msgr2=0x7f939c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:31.015 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.013+0000 7f93b70e6700 1 --2- 172.21.15.144:0/3027582939 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f939c06e200 0x7f939c0706b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f93a8005950 tx=0x7f93a800b410 comp rx=0 tx=0).stop 2024-09-10T12:04:31.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.013+0000 7f93b70e6700 1 -- 172.21.15.144:0/3027582939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93b0068fd0 msgr2=0x7f93b0196360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:31.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.013+0000 7f93b70e6700 1 --2- 172.21.15.144:0/3027582939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93b0068fd0 0x7f93b0196360 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f93a0006130 tx=0x7f93a000cd00 comp rx=0 tx=0).stop 2024-09-10T12:04:31.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.013+0000 7f93b70e6700 1 -- 172.21.15.144:0/3027582939 shutdown_connections 2024-09-10T12:04:31.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.013+0000 7f93b70e6700 1 --2- 172.21.15.144:0/3027582939 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f939c06e200 0x7f939c0706b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.013+0000 7f93b70e6700 1 --2- 172.21.15.144:0/3027582939 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f93b0068630 0x7f93b0195e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.013+0000 7f93b70e6700 1 --2- 172.21.15.144:0/3027582939 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f93b0068fd0 0x7f93b0196360 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.013+0000 7f93b70e6700 1 -- 172.21.15.144:0/3027582939 >> 172.21.15.144:0/3027582939 conn(0x7f93b0074f50 msgr2=0x7f93b00fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:31.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.013+0000 7f93b70e6700 1 -- 172.21.15.144:0/3027582939 shutdown_connections 2024-09-10T12:04:31.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.013+0000 7f93b70e6700 1 -- 172.21.15.144:0/3027582939 wait complete. 2024-09-10T12:04:31.026 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:04:31.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.119+0000 7f696527c700 1 -- 172.21.15.144:0/2300792109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960101320 msgr2=0x7f69601016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:31.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.119+0000 7f696527c700 1 --2- 172.21.15.144:0/2300792109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960101320 0x7f69601016f0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f6950009910 tx=0x7f6950009c20 comp rx=0 tx=0).stop 2024-09-10T12:04:31.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.119+0000 7f696527c700 1 -- 172.21.15.144:0/2300792109 shutdown_connections 2024-09-10T12:04:31.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.119+0000 7f696527c700 1 --2- 172.21.15.144:0/2300792109 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6960101c30 0x7f696010a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.119+0000 7f696527c700 1 --2- 172.21.15.144:0/2300792109 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960101320 0x7f69601016f0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.119+0000 7f696527c700 1 -- 172.21.15.144:0/2300792109 >> 172.21.15.144:0/2300792109 conn(0x7f69600fab80 msgr2=0x7f69600fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:31.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.119+0000 7f696527c700 1 -- 172.21.15.144:0/2300792109 shutdown_connections 2024-09-10T12:04:31.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.119+0000 7f696527c700 1 -- 172.21.15.144:0/2300792109 wait complete. 2024-09-10T12:04:31.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.119+0000 7f696527c700 1 Processor -- start 2024-09-10T12:04:31.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.120+0000 7f696527c700 1 -- start start 2024-09-10T12:04:31.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.120+0000 7f696527c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960101320 0x7f6960195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:31.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.120+0000 7f696527c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6960101c30 0x7f6960196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:31.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.120+0000 7f696527c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6960196970 con 0x7f6960101320 2024-09-10T12:04:31.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.120+0000 7f696527c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f696019a700 con 0x7f6960101c30 2024-09-10T12:04:31.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.120+0000 7f695effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960101320 0x7f6960195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:31.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.120+0000 7f695effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960101320 0x7f6960195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41676/0 (socket says 172.21.15.144:41676) 2024-09-10T12:04:31.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.120+0000 7f695effd700 1 -- 172.21.15.144:0/1956806240 learned_addr learned my addr 172.21.15.144:0/1956806240 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:04:31.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.120+0000 7f69567fc700 1 --2- 172.21.15.144:0/1956806240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6960101c30 0x7f6960196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:31.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.120+0000 7f695effd700 1 -- 172.21.15.144:0/1956806240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6960101c30 msgr2=0x7f6960196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:31.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.120+0000 7f695effd700 1 --2- 172.21.15.144:0/1956806240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6960101c30 0x7f6960196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.120+0000 7f695effd700 1 -- 172.21.15.144:0/1956806240 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69500095f0 con 0x7f6960101320 2024-09-10T12:04:31.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.121+0000 7f695effd700 1 --2- 172.21.15.144:0/1956806240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960101320 0x7f6960195d50 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6950006010 tx=0x7f695000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:31.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.121+0000 7f695cff9700 1 -- 172.21.15.144:0/1956806240 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f695001d070 con 0x7f6960101320 2024-09-10T12:04:31.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.121+0000 7f695cff9700 1 -- 172.21.15.144:0/1956806240 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6950005dc0 con 0x7f6960101320 2024-09-10T12:04:31.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.121+0000 7f696527c700 1 -- 172.21.15.144:0/1956806240 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f696019a980 con 0x7f6960101320 2024-09-10T12:04:31.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.121+0000 7f695cff9700 1 -- 172.21.15.144:0/1956806240 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6950017400 con 0x7f6960101320 2024-09-10T12:04:31.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.121+0000 7f696527c700 1 -- 172.21.15.144:0/1956806240 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f696019ae70 con 0x7f6960101320 2024-09-10T12:04:31.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.122+0000 7f696527c700 1 -- 172.21.15.144:0/1956806240 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6960107800 con 0x7f6960101320 2024-09-10T12:04:31.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.122+0000 7f695cff9700 1 -- 172.21.15.144:0/1956806240 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6950003a60 con 0x7f6960101320 2024-09-10T12:04:31.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.123+0000 7f695cff9700 1 --2- 172.21.15.144:0/1956806240 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f694c06e200 0x7f694c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:31.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.123+0000 7f695cff9700 1 -- 172.21.15.144:0/1956806240 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f695008f450 con 0x7f6960101320 2024-09-10T12:04:31.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.123+0000 7f69567fc700 1 --2- 172.21.15.144:0/1956806240 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f694c06e200 0x7f694c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:31.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.124+0000 7f69567fc700 1 --2- 172.21.15.144:0/1956806240 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f694c06e200 0x7f694c0706b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6960197370 tx=0x7f694800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:31.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.127+0000 7f695cff9700 1 -- 172.21.15.144:0/1956806240 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f69500588b0 con 0x7f6960101320 2024-09-10T12:04:31.350 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.347+0000 7f696527c700 1 -- 172.21.15.144:0/1956806240 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f696019b210 con 0x7f694c06e200 2024-09-10T12:04:31.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.349+0000 7f695cff9700 1 -- 172.21.15.144:0/1956806240 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f696019b210 con 0x7f694c06e200 2024-09-10T12:04:31.353 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.351+0000 7f696527c700 1 -- 172.21.15.144:0/1956806240 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f694c06e200 msgr2=0x7f694c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:31.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.351+0000 7f696527c700 1 --2- 172.21.15.144:0/1956806240 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f694c06e200 0x7f694c0706b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6960197370 tx=0x7f694800b410 comp rx=0 tx=0).stop 2024-09-10T12:04:31.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.351+0000 7f696527c700 1 -- 172.21.15.144:0/1956806240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960101320 msgr2=0x7f6960195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:31.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.351+0000 7f696527c700 1 --2- 172.21.15.144:0/1956806240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960101320 0x7f6960195d50 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6950006010 tx=0x7f695000bd70 comp rx=0 tx=0).stop 2024-09-10T12:04:31.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.351+0000 7f696527c700 1 -- 172.21.15.144:0/1956806240 shutdown_connections 2024-09-10T12:04:31.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.351+0000 7f696527c700 1 --2- 172.21.15.144:0/1956806240 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f694c06e200 0x7f694c0706b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.351+0000 7f696527c700 1 --2- 172.21.15.144:0/1956806240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6960101320 0x7f6960195d50 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.354 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.351+0000 7f696527c700 1 --2- 172.21.15.144:0/1956806240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6960101c30 0x7f6960196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.351+0000 7f696527c700 1 -- 172.21.15.144:0/1956806240 >> 172.21.15.144:0/1956806240 conn(0x7f69600fab80 msgr2=0x7f6960104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:31.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.351+0000 7f696527c700 1 -- 172.21.15.144:0/1956806240 shutdown_connections 2024-09-10T12:04:31.355 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.351+0000 7f696527c700 1 -- 172.21.15.144:0/1956806240 wait complete. 2024-09-10T12:04:31.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.456+0000 7f9e9cd85700 1 -- 172.21.15.144:0/3393924915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e98100d80 msgr2=0x7f9e98104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:31.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.456+0000 7f9e9cd85700 1 --2- 172.21.15.144:0/3393924915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e98100d80 0x7f9e98104d50 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009910 tx=0x7f9e8c009c20 comp rx=0 tx=0).stop 2024-09-10T12:04:31.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.456+0000 7f9e9cd85700 1 -- 172.21.15.144:0/3393924915 shutdown_connections 2024-09-10T12:04:31.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.456+0000 7f9e9cd85700 1 --2- 172.21.15.144:0/3393924915 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e98100d80 0x7f9e98104d50 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.456+0000 7f9e9cd85700 1 --2- 172.21.15.144:0/3393924915 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e981003e0 0x7f9e981007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.456+0000 7f9e9cd85700 1 -- 172.21.15.144:0/3393924915 >> 172.21.15.144:0/3393924915 conn(0x7f9e980fbca0 msgr2=0x7f9e980fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:31.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.456+0000 7f9e9cd85700 1 -- 172.21.15.144:0/3393924915 shutdown_connections 2024-09-10T12:04:31.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.456+0000 7f9e9cd85700 1 -- 172.21.15.144:0/3393924915 wait complete. 2024-09-10T12:04:31.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.457+0000 7f9e9cd85700 1 Processor -- start 2024-09-10T12:04:31.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.457+0000 7f9e9cd85700 1 -- start start 2024-09-10T12:04:31.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.457+0000 7f9e9cd85700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e981003e0 0x7f9e98195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:31.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.457+0000 7f9e9cd85700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e98100d80 0x7f9e98196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:31.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.457+0000 7f9e9cd85700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e981969f0 con 0x7f9e98100d80 2024-09-10T12:04:31.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.457+0000 7f9e9cd85700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9819a780 con 0x7f9e981003e0 2024-09-10T12:04:31.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.457+0000 7f9e95d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e98100d80 0x7f9e98196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:31.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.457+0000 7f9e95d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e98100d80 0x7f9e98196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41694/0 (socket says 172.21.15.144:41694) 2024-09-10T12:04:31.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.457+0000 7f9e95d9b700 1 -- 172.21.15.144:0/699760240 learned_addr learned my addr 172.21.15.144:0/699760240 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:04:31.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.457+0000 7f9e9659c700 1 --2- 172.21.15.144:0/699760240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e981003e0 0x7f9e98195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:31.463 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.458+0000 7f9e95d9b700 1 -- 172.21.15.144:0/699760240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e981003e0 msgr2=0x7f9e98195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:31.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.458+0000 7f9e95d9b700 1 --2- 172.21.15.144:0/699760240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e981003e0 0x7f9e98195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.458+0000 7f9e95d9b700 1 -- 172.21.15.144:0/699760240 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e8c0095f0 con 0x7f9e98100d80 2024-09-10T12:04:31.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.458+0000 7f9e95d9b700 1 --2- 172.21.15.144:0/699760240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e98100d80 0x7f9e98196310 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009fd0 tx=0x7f9e8c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:31.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.458+0000 7f9e877fe700 1 -- 172.21.15.144:0/699760240 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e8c01d070 con 0x7f9e98100d80 2024-09-10T12:04:31.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.458+0000 7f9e877fe700 1 -- 172.21.15.144:0/699760240 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9e8c022470 con 0x7f9e98100d80 2024-09-10T12:04:31.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.458+0000 7f9e9cd85700 1 -- 172.21.15.144:0/699760240 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9819aa00 con 0x7f9e98100d80 2024-09-10T12:04:31.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.458+0000 7f9e877fe700 1 -- 172.21.15.144:0/699760240 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e8c017650 con 0x7f9e98100d80 2024-09-10T12:04:31.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.458+0000 7f9e9cd85700 1 -- 172.21.15.144:0/699760240 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e9819ae90 con 0x7f9e98100d80 2024-09-10T12:04:31.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.459+0000 7f9e9cd85700 1 -- 172.21.15.144:0/699760240 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e9804ea50 con 0x7f9e98100d80 2024-09-10T12:04:31.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.459+0000 7f9e877fe700 1 -- 172.21.15.144:0/699760240 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9e8c0225e0 con 0x7f9e98100d80 2024-09-10T12:04:31.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.460+0000 7f9e877fe700 1 --2- 172.21.15.144:0/699760240 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e8006e200 0x7f9e800706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:31.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.460+0000 7f9e877fe700 1 -- 172.21.15.144:0/699760240 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9e8c08f5e0 con 0x7f9e98100d80 2024-09-10T12:04:31.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.460+0000 7f9e9659c700 1 --2- 172.21.15.144:0/699760240 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e8006e200 0x7f9e800706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:31.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.460+0000 7f9e9659c700 1 --2- 172.21.15.144:0/699760240 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e8006e200 0x7f9e800706b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009c60 tx=0x7f9e88009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:31.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.464+0000 7f9e877fe700 1 -- 172.21.15.144:0/699760240 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9e8c0589c0 con 0x7f9e98100d80 2024-09-10T12:04:31.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.676+0000 7f9e9cd85700 1 -- 172.21.15.144:0/699760240 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e9819b430 con 0x7f9e8006e200 2024-09-10T12:04:31.688 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.686+0000 7f9e877fe700 1 -- 172.21.15.144:0/699760240 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f9e9819b430 con 0x7f9e8006e200 2024-09-10T12:04:31.688 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:04:31.689 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (32m) 11m ago 43m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:04:31.689 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (43m) 11m ago 43m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:04:31.689 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (42m) 11m ago 42m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:04:31.689 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (30m) 11m ago 43m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:04:31.689 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (30m) 11m ago 42m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:04:31.689 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (32m) 11m ago 43m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:04:31.689 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (26m) 11m ago 37m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:04:31.690 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (26m) 11m ago 37m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:04:31.690 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (11m) 11m ago 37m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:04:31.690 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (26m) 11m ago 37m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:04:31.690 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (34m) 11m ago 44m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:04:31.690 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (33m) 11m ago 42m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:04:31.690 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (31m) 11m ago 44m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:04:31.690 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (31m) 11m ago 42m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:04:31.690 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (33m) 11m ago 43m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:04:31.691 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (33m) 11m ago 42m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:04:31.691 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (29m) 11m ago 41m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:04:31.691 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (29m) 11m ago 41m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:04:31.691 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (28m) 11m ago 40m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:04:31.691 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (28m) 11m ago 40m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:04:31.691 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (27m) 11m ago 39m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:04:31.691 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (27m) 11m ago 39m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:04:31.691 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (33m) 11m ago 42m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:04:31.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.688+0000 7f9e9cd85700 1 -- 172.21.15.144:0/699760240 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e8006e200 msgr2=0x7f9e800706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:31.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.688+0000 7f9e9cd85700 1 --2- 172.21.15.144:0/699760240 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e8006e200 0x7f9e800706b0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f9e88009c60 tx=0x7f9e88009260 comp rx=0 tx=0).stop 2024-09-10T12:04:31.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.688+0000 7f9e9cd85700 1 -- 172.21.15.144:0/699760240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e98100d80 msgr2=0x7f9e98196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:31.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.688+0000 7f9e9cd85700 1 --2- 172.21.15.144:0/699760240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e98100d80 0x7f9e98196310 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009fd0 tx=0x7f9e8c00f690 comp rx=0 tx=0).stop 2024-09-10T12:04:31.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.688+0000 7f9e9cd85700 1 -- 172.21.15.144:0/699760240 shutdown_connections 2024-09-10T12:04:31.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.688+0000 7f9e9cd85700 1 --2- 172.21.15.144:0/699760240 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9e8006e200 0x7f9e800706b0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.688+0000 7f9e9cd85700 1 --2- 172.21.15.144:0/699760240 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9e981003e0 0x7f9e98195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.692 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.688+0000 7f9e9cd85700 1 --2- 172.21.15.144:0/699760240 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9e98100d80 0x7f9e98196310 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.688+0000 7f9e9cd85700 1 -- 172.21.15.144:0/699760240 >> 172.21.15.144:0/699760240 conn(0x7f9e980fbca0 msgr2=0x7f9e980fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:31.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.688+0000 7f9e9cd85700 1 -- 172.21.15.144:0/699760240 shutdown_connections 2024-09-10T12:04:31.693 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.688+0000 7f9e9cd85700 1 -- 172.21.15.144:0/699760240 wait complete. 2024-09-10T12:04:31.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.792+0000 7f645a0b3700 1 -- 172.21.15.144:0/3466878245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6454100d70 msgr2=0x7f6454104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:31.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.792+0000 7f645a0b3700 1 --2- 172.21.15.144:0/3466878245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6454100d70 0x7f6454104d40 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f643c009910 tx=0x7f643c009c20 comp rx=0 tx=0).stop 2024-09-10T12:04:31.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.792+0000 7f645a0b3700 1 -- 172.21.15.144:0/3466878245 shutdown_connections 2024-09-10T12:04:31.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.792+0000 7f645a0b3700 1 --2- 172.21.15.144:0/3466878245 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6454100d70 0x7f6454104d40 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.792+0000 7f645a0b3700 1 --2- 172.21.15.144:0/3466878245 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64541003d0 0x7f64541007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.792+0000 7f645a0b3700 1 -- 172.21.15.144:0/3466878245 >> 172.21.15.144:0/3466878245 conn(0x7f64540fbc90 msgr2=0x7f64540fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:31.795 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.793+0000 7f645a0b3700 1 -- 172.21.15.144:0/3466878245 shutdown_connections 2024-09-10T12:04:31.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.793+0000 7f645a0b3700 1 -- 172.21.15.144:0/3466878245 wait complete. 2024-09-10T12:04:31.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.793+0000 7f645a0b3700 1 Processor -- start 2024-09-10T12:04:31.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.793+0000 7f645a0b3700 1 -- start start 2024-09-10T12:04:31.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.793+0000 7f645a0b3700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64541003d0 0x7f6454195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:31.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.793+0000 7f645a0b3700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6454100d70 0x7f6454196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:31.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.793+0000 7f645a0b3700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6454196960 con 0x7f6454100d70 2024-09-10T12:04:31.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.793+0000 7f645a0b3700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645419a6f0 con 0x7f64541003d0 2024-09-10T12:04:31.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.794+0000 7f644bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6454100d70 0x7f6454196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:31.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.794+0000 7f644bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6454100d70 0x7f6454196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41710/0 (socket says 172.21.15.144:41710) 2024-09-10T12:04:31.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.794+0000 7f644bfff700 1 -- 172.21.15.144:0/2174365425 learned_addr learned my addr 172.21.15.144:0/2174365425 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:04:31.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.794+0000 7f64537fe700 1 --2- 172.21.15.144:0/2174365425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64541003d0 0x7f6454195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:31.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.794+0000 7f644bfff700 1 -- 172.21.15.144:0/2174365425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64541003d0 msgr2=0x7f6454195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:31.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.794+0000 7f644bfff700 1 --2- 172.21.15.144:0/2174365425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64541003d0 0x7f6454195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:31.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.794+0000 7f644bfff700 1 -- 172.21.15.144:0/2174365425 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c0095f0 con 0x7f6454100d70 2024-09-10T12:04:31.798 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.794+0000 7f644bfff700 1 --2- 172.21.15.144:0/2174365425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6454100d70 0x7f6454196280 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f643c0092e0 tx=0x7f643c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:31.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.794+0000 7f64517fa700 1 -- 172.21.15.144:0/2174365425 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f643c01d070 con 0x7f6454100d70 2024-09-10T12:04:31.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.794+0000 7f64517fa700 1 -- 172.21.15.144:0/2174365425 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f643c00f460 con 0x7f6454100d70 2024-09-10T12:04:31.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.794+0000 7f64517fa700 1 -- 172.21.15.144:0/2174365425 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f643c0175f0 con 0x7f6454100d70 2024-09-10T12:04:31.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.794+0000 7f645a0b3700 1 -- 172.21.15.144:0/2174365425 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645419a970 con 0x7f6454100d70 2024-09-10T12:04:31.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.794+0000 7f645a0b3700 1 -- 172.21.15.144:0/2174365425 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645419ae60 con 0x7f6454100d70 2024-09-10T12:04:31.799 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.795+0000 7f645a0b3700 1 -- 172.21.15.144:0/2174365425 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6454108740 con 0x7f6454100d70 2024-09-10T12:04:31.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.796+0000 7f64517fa700 1 -- 172.21.15.144:0/2174365425 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f643c00f5d0 con 0x7f6454100d70 2024-09-10T12:04:31.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.796+0000 7f64517fa700 1 --2- 172.21.15.144:0/2174365425 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f644006e200 0x7f64400706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:31.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.796+0000 7f64517fa700 1 -- 172.21.15.144:0/2174365425 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f643c08f620 con 0x7f6454100d70 2024-09-10T12:04:31.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.796+0000 7f64537fe700 1 --2- 172.21.15.144:0/2174365425 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f644006e200 0x7f64400706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:31.801 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.797+0000 7f64537fe700 1 --2- 172.21.15.144:0/2174365425 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f644006e200 0x7f64400706b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6444005fd0 tx=0x7f644400a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:31.803 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:31.801+0000 7f64517fa700 1 -- 172.21.15.144:0/2174365425 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f643c058a00 con 0x7f6454100d70 2024-09-10T12:04:32.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.092+0000 7f645a0b3700 1 -- 172.21.15.144:0/2174365425 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f645404ea50 con 0x7f6454100d70 2024-09-10T12:04:32.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.093+0000 7f64517fa700 1 -- 172.21.15.144:0/2174365425 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f643c05c0c0 con 0x7f6454100d70 2024-09-10T12:04:32.095 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:04:32.096 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:04:32.096 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:04:32.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:04:32.096 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:04:32.096 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:04:32.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:04:32.096 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:04:32.096 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:04:32.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:04:32.096 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:04:32.096 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:04:32.096 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:04:32.096 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:04:32.097 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:04:32.097 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:04:32.097 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:04:32.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.095+0000 7f645a0b3700 1 -- 172.21.15.144:0/2174365425 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f644006e200 msgr2=0x7f64400706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:32.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.095+0000 7f645a0b3700 1 --2- 172.21.15.144:0/2174365425 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f644006e200 0x7f64400706b0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f6444005fd0 tx=0x7f644400a560 comp rx=0 tx=0).stop 2024-09-10T12:04:32.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.095+0000 7f645a0b3700 1 -- 172.21.15.144:0/2174365425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6454100d70 msgr2=0x7f6454196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:32.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.095+0000 7f645a0b3700 1 --2- 172.21.15.144:0/2174365425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6454100d70 0x7f6454196280 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f643c0092e0 tx=0x7f643c00be20 comp rx=0 tx=0).stop 2024-09-10T12:04:32.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.095+0000 7f645a0b3700 1 -- 172.21.15.144:0/2174365425 shutdown_connections 2024-09-10T12:04:32.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.095+0000 7f645a0b3700 1 --2- 172.21.15.144:0/2174365425 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f644006e200 0x7f64400706b0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.095+0000 7f645a0b3700 1 --2- 172.21.15.144:0/2174365425 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64541003d0 0x7f6454195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.095+0000 7f645a0b3700 1 --2- 172.21.15.144:0/2174365425 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6454100d70 0x7f6454196280 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.096+0000 7f645a0b3700 1 -- 172.21.15.144:0/2174365425 >> 172.21.15.144:0/2174365425 conn(0x7f64540fbc90 msgr2=0x7f64540fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:32.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.096+0000 7f645a0b3700 1 -- 172.21.15.144:0/2174365425 shutdown_connections 2024-09-10T12:04:32.099 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.096+0000 7f645a0b3700 1 -- 172.21.15.144:0/2174365425 wait complete. 2024-09-10T12:04:32.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.202+0000 7f1fedc37700 1 -- 172.21.15.144:0/3054069379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe81003d0 msgr2=0x7f1fe81007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:32.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.202+0000 7f1fedc37700 1 --2- 172.21.15.144:0/3054069379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe81003d0 0x7f1fe81007a0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800b0a0 tx=0x7f1fd800b3b0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.202+0000 7f1fedc37700 1 -- 172.21.15.144:0/3054069379 shutdown_connections 2024-09-10T12:04:32.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.202+0000 7f1fedc37700 1 --2- 172.21.15.144:0/3054069379 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fe8100d70 0x7f1fe8104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.202+0000 7f1fedc37700 1 --2- 172.21.15.144:0/3054069379 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe81003d0 0x7f1fe81007a0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.202+0000 7f1fedc37700 1 -- 172.21.15.144:0/3054069379 >> 172.21.15.144:0/3054069379 conn(0x7f1fe80fbc90 msgr2=0x7f1fe80fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:32.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.202+0000 7f1fedc37700 1 -- 172.21.15.144:0/3054069379 shutdown_connections 2024-09-10T12:04:32.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.202+0000 7f1fedc37700 1 -- 172.21.15.144:0/3054069379 wait complete. 2024-09-10T12:04:32.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.202+0000 7f1fedc37700 1 Processor -- start 2024-09-10T12:04:32.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.203+0000 7f1fedc37700 1 -- start start 2024-09-10T12:04:32.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.203+0000 7f1fedc37700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe81003d0 0x7f1fe8197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:32.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.203+0000 7f1fedc37700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fe8100d70 0x7f1fe81984e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:32.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.203+0000 7f1fedc37700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe8198b30 con 0x7f1fe81003d0 2024-09-10T12:04:32.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.203+0000 7f1fedc37700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe8198c70 con 0x7f1fe8100d70 2024-09-10T12:04:32.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.203+0000 7f1fe77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe81003d0 0x7f1fe8197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:32.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.203+0000 7f1fe77fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe81003d0 0x7f1fe8197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41720/0 (socket says 172.21.15.144:41720) 2024-09-10T12:04:32.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.203+0000 7f1fe77fe700 1 -- 172.21.15.144:0/1270554659 learned_addr learned my addr 172.21.15.144:0/1270554659 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:04:32.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.203+0000 7f1fe6ffd700 1 --2- 172.21.15.144:0/1270554659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fe8100d70 0x7f1fe81984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:32.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.203+0000 7f1fe77fe700 1 -- 172.21.15.144:0/1270554659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fe8100d70 msgr2=0x7f1fe81984e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:32.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.203+0000 7f1fe77fe700 1 --2- 172.21.15.144:0/1270554659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fe8100d70 0x7f1fe81984e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.203+0000 7f1fe77fe700 1 -- 172.21.15.144:0/1270554659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd8009d00 con 0x7f1fe81003d0 2024-09-10T12:04:32.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.203+0000 7f1fe77fe700 1 --2- 172.21.15.144:0/1270554659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe81003d0 0x7f1fe8197fa0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800b070 tx=0x7f1fd800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:32.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.204+0000 7f1fe4ff9700 1 -- 172.21.15.144:0/1270554659 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fd800e040 con 0x7f1fe81003d0 2024-09-10T12:04:32.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.204+0000 7f1fe4ff9700 1 -- 172.21.15.144:0/1270554659 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1fd8003e60 con 0x7f1fe81003d0 2024-09-10T12:04:32.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.204+0000 7f1fedc37700 1 -- 172.21.15.144:0/1270554659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe819ca60 con 0x7f1fe81003d0 2024-09-10T12:04:32.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.204+0000 7f1fe4ff9700 1 -- 172.21.15.144:0/1270554659 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1fd801d910 con 0x7f1fe81003d0 2024-09-10T12:04:32.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.204+0000 7f1fedc37700 1 -- 172.21.15.144:0/1270554659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe819cf50 con 0x7f1fe81003d0 2024-09-10T12:04:32.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.205+0000 7f1fedc37700 1 -- 172.21.15.144:0/1270554659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe81044c0 con 0x7f1fe81003d0 2024-09-10T12:04:32.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.205+0000 7f1fe4ff9700 1 -- 172.21.15.144:0/1270554659 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1fd8004820 con 0x7f1fe81003d0 2024-09-10T12:04:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.205+0000 7f1fe4ff9700 1 --2- 172.21.15.144:0/1270554659 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1fd406e200 0x7f1fd40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.206+0000 7f1fe4ff9700 1 -- 172.21.15.144:0/1270554659 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1fd808f900 con 0x7f1fe81003d0 2024-09-10T12:04:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.206+0000 7f1fe6ffd700 1 --2- 172.21.15.144:0/1270554659 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1fd406e200 0x7f1fd40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:32.211 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.206+0000 7f1fe6ffd700 1 --2- 172.21.15.144:0/1270554659 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1fd406e200 0x7f1fd40706b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0005ea0 tx=0x7f1fd0005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:32.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.210+0000 7f1fe4ff9700 1 -- 172.21.15.144:0/1270554659 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1fd8058d60 con 0x7f1fe81003d0 2024-09-10T12:04:32.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:32 smithi144 ceph-mon[87436]: from='client.35772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:04:32.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:32 smithi144 ceph-mon[87436]: pgmap v977: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:32.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:32 smithi144 ceph-mon[87436]: from='client.35776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:04:32.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:32 smithi144 ceph-mon[87436]: from='client.35780 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:04:32.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:32 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2174365425' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:04:32.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.454+0000 7f1fedc37700 1 -- 172.21.15.144:0/1270554659 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1fe802ccf0 con 0x7f1fe81003d0 2024-09-10T12:04:32.458 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.455+0000 7f1fe4ff9700 1 -- 172.21.15.144:0/1270554659 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f1fd805c420 con 0x7f1fe81003d0 2024-09-10T12:04:32.458 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:04:32.458 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:04:32.458 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:04:32.459 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:04:32.459 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:04:32.459 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:04:32.459 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:04:32.459 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:04:32.459 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:04:32.459 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:04:32.460 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:04:32.460 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:04:32.460 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:04:32.460 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:04:32.460 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:04:32.460 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:04:32.460 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:04:32.461 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:04:32.461 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:04:32.461 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:04:32.461 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:04:32.461 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:04:32.461 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:04:32.461 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:04:32.461 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:04:32.462 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:04:32.462 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:04:32.462 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:04:32.462 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:04:32.462 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:04:32.462 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:04:32.463 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:04:32.463 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:04:32.463 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:04:32.463 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:04:32.463 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:04:32.463 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:04:32.463 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:04:32.464 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:04:32.464 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:04:32.464 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:04:32.464 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:04:32.464 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:04:32.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.457+0000 7f1fedc37700 1 -- 172.21.15.144:0/1270554659 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1fd406e200 msgr2=0x7f1fd40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:32.465 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.457+0000 7f1fedc37700 1 --2- 172.21.15.144:0/1270554659 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1fd406e200 0x7f1fd40706b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0005ea0 tx=0x7f1fd0005e30 comp rx=0 tx=0).stop 2024-09-10T12:04:32.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.457+0000 7f1fedc37700 1 -- 172.21.15.144:0/1270554659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe81003d0 msgr2=0x7f1fe8197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:32.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.457+0000 7f1fedc37700 1 --2- 172.21.15.144:0/1270554659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe81003d0 0x7f1fe8197fa0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800b070 tx=0x7f1fd800cdd0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.457+0000 7f1fedc37700 1 -- 172.21.15.144:0/1270554659 shutdown_connections 2024-09-10T12:04:32.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.457+0000 7f1fedc37700 1 --2- 172.21.15.144:0/1270554659 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1fd406e200 0x7f1fd40706b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.457+0000 7f1fedc37700 1 --2- 172.21.15.144:0/1270554659 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1fe81003d0 0x7f1fe8197fa0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.457+0000 7f1fedc37700 1 --2- 172.21.15.144:0/1270554659 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1fe8100d70 0x7f1fe81984e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.457+0000 7f1fedc37700 1 -- 172.21.15.144:0/1270554659 >> 172.21.15.144:0/1270554659 conn(0x7f1fe80fbc90 msgr2=0x7f1fe80fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:32.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.457+0000 7f1fedc37700 1 -- 172.21.15.144:0/1270554659 shutdown_connections 2024-09-10T12:04:32.466 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.457+0000 7f1fedc37700 1 -- 172.21.15.144:0/1270554659 wait complete. 2024-09-10T12:04:32.466 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:04:32.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.562+0000 7f25ef6b5700 1 -- 172.21.15.144:0/3594366808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e80ffcb0 msgr2=0x7f25e8103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:32.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.562+0000 7f25ef6b5700 1 --2- 172.21.15.144:0/3594366808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e80ffcb0 0x7f25e8103c80 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f25e4009910 tx=0x7f25e4009c20 comp rx=0 tx=0).stop 2024-09-10T12:04:32.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.562+0000 7f25ef6b5700 1 -- 172.21.15.144:0/3594366808 shutdown_connections 2024-09-10T12:04:32.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.562+0000 7f25ef6b5700 1 --2- 172.21.15.144:0/3594366808 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e80ffcb0 0x7f25e8103c80 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.562+0000 7f25ef6b5700 1 --2- 172.21.15.144:0/3594366808 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25e80ff310 0x7f25e80ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.562+0000 7f25ef6b5700 1 -- 172.21.15.144:0/3594366808 >> 172.21.15.144:0/3594366808 conn(0x7f25e80fabd0 msgr2=0x7f25e80fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:32.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.562+0000 7f25ef6b5700 1 -- 172.21.15.144:0/3594366808 shutdown_connections 2024-09-10T12:04:32.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.562+0000 7f25ef6b5700 1 -- 172.21.15.144:0/3594366808 wait complete. 2024-09-10T12:04:32.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.563+0000 7f25ef6b5700 1 Processor -- start 2024-09-10T12:04:32.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.563+0000 7f25ef6b5700 1 -- start start 2024-09-10T12:04:32.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.563+0000 7f25ef6b5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25e80ff310 0x7f25e8194cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:32.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.563+0000 7f25ef6b5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e80ffcb0 0x7f25e8195200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:32.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.563+0000 7f25ef6b5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e8195800 con 0x7f25e80ffcb0 2024-09-10T12:04:32.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.563+0000 7f25ef6b5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e8195970 con 0x7f25e80ff310 2024-09-10T12:04:32.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.563+0000 7f25ecc50700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e80ffcb0 0x7f25e8195200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:32.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.563+0000 7f25ecc50700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e80ffcb0 0x7f25e8195200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52998/0 (socket says 172.21.15.144:52998) 2024-09-10T12:04:32.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.563+0000 7f25ecc50700 1 -- 172.21.15.144:0/4201096456 learned_addr learned my addr 172.21.15.144:0/4201096456 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:04:32.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.563+0000 7f25ecc50700 1 -- 172.21.15.144:0/4201096456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25e80ff310 msgr2=0x7f25e8194cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:32.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.563+0000 7f25ed451700 1 --2- 172.21.15.144:0/4201096456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25e80ff310 0x7f25e8194cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:32.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.564+0000 7f25ecc50700 1 --2- 172.21.15.144:0/4201096456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25e80ff310 0x7f25e8194cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.564+0000 7f25ecc50700 1 -- 172.21.15.144:0/4201096456 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e40095f0 con 0x7f25e80ffcb0 2024-09-10T12:04:32.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.564+0000 7f25ed451700 1 --2- 172.21.15.144:0/4201096456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25e80ff310 0x7f25e8194cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:04:32.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.564+0000 7f25ecc50700 1 --2- 172.21.15.144:0/4201096456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e80ffcb0 0x7f25e8195200 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f25e400bda0 tx=0x7f25e400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:32.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.564+0000 7f25de7fc700 1 -- 172.21.15.144:0/4201096456 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25e401d070 con 0x7f25e80ffcb0 2024-09-10T12:04:32.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.564+0000 7f25ef6b5700 1 -- 172.21.15.144:0/4201096456 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e81986b0 con 0x7f25e80ffcb0 2024-09-10T12:04:32.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.564+0000 7f25de7fc700 1 -- 172.21.15.144:0/4201096456 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25e4004fb0 con 0x7f25e80ffcb0 2024-09-10T12:04:32.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.564+0000 7f25de7fc700 1 -- 172.21.15.144:0/4201096456 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25e400f460 con 0x7f25e80ffcb0 2024-09-10T12:04:32.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.564+0000 7f25ef6b5700 1 -- 172.21.15.144:0/4201096456 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e8198ba0 con 0x7f25e80ffcb0 2024-09-10T12:04:32.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.565+0000 7f25ef6b5700 1 -- 172.21.15.144:0/4201096456 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e804ea50 con 0x7f25e80ffcb0 2024-09-10T12:04:32.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.565+0000 7f25de7fc700 1 -- 172.21.15.144:0/4201096456 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f25e4022800 con 0x7f25e80ffcb0 2024-09-10T12:04:32.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.566+0000 7f25de7fc700 1 --2- 172.21.15.144:0/4201096456 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25d406e200 0x7f25d40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:32.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.566+0000 7f25de7fc700 1 -- 172.21.15.144:0/4201096456 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f25e408f630 con 0x7f25e80ffcb0 2024-09-10T12:04:32.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.566+0000 7f25ed451700 1 --2- 172.21.15.144:0/4201096456 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25d406e200 0x7f25d40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:32.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.566+0000 7f25ed451700 1 --2- 172.21.15.144:0/4201096456 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25d406e200 0x7f25d40706b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f25d8005fd0 tx=0x7f25d8005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:32.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.570+0000 7f25de7fc700 1 -- 172.21.15.144:0/4201096456 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f25e4058ac0 con 0x7f25e80ffcb0 2024-09-10T12:04:32.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:32 smithi178 ceph-mon[65940]: from='client.35772 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:04:32.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:32 smithi178 ceph-mon[65940]: pgmap v977: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:32.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:32 smithi178 ceph-mon[65940]: from='client.35776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:04:32.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:32 smithi178 ceph-mon[65940]: from='client.35780 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:04:32.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:32 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2174365425' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:04:32.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.790+0000 7f25ef6b5700 1 -- 172.21.15.144:0/4201096456 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f25e8198f90 con 0x7f25d406e200 2024-09-10T12:04:32.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.791+0000 7f25de7fc700 1 -- 172.21.15.144:0/4201096456 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f25e8198f90 con 0x7f25d406e200 2024-09-10T12:04:32.793 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:04:32.794 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:04:32.794 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:04:32.794 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:04:32.794 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:04:32.794 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:04:32.794 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:04:32.794 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:04:32.794 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:04:32.794 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:04:32.794 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:04:32.794 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:04:32.794 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:04:32.794 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:04:32.795 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:04:32.796 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.794+0000 7f25ef6b5700 1 -- 172.21.15.144:0/4201096456 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25d406e200 msgr2=0x7f25d40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:32.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.794+0000 7f25ef6b5700 1 --2- 172.21.15.144:0/4201096456 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25d406e200 0x7f25d40706b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f25d8005fd0 tx=0x7f25d8005e20 comp rx=0 tx=0).stop 2024-09-10T12:04:32.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.794+0000 7f25ef6b5700 1 -- 172.21.15.144:0/4201096456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e80ffcb0 msgr2=0x7f25e8195200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:32.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.794+0000 7f25ef6b5700 1 --2- 172.21.15.144:0/4201096456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e80ffcb0 0x7f25e8195200 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f25e400bda0 tx=0x7f25e400be80 comp rx=0 tx=0).stop 2024-09-10T12:04:32.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.794+0000 7f25ef6b5700 1 -- 172.21.15.144:0/4201096456 shutdown_connections 2024-09-10T12:04:32.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.794+0000 7f25ef6b5700 1 --2- 172.21.15.144:0/4201096456 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25d406e200 0x7f25d40706b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.794+0000 7f25ef6b5700 1 --2- 172.21.15.144:0/4201096456 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25e80ff310 0x7f25e8194cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.794+0000 7f25ef6b5700 1 --2- 172.21.15.144:0/4201096456 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25e80ffcb0 0x7f25e8195200 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.794+0000 7f25ef6b5700 1 -- 172.21.15.144:0/4201096456 >> 172.21.15.144:0/4201096456 conn(0x7f25e80fabd0 msgr2=0x7f25e80fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:32.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.794+0000 7f25ef6b5700 1 -- 172.21.15.144:0/4201096456 shutdown_connections 2024-09-10T12:04:32.797 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.794+0000 7f25ef6b5700 1 -- 172.21.15.144:0/4201096456 wait complete. 2024-09-10T12:04:32.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.899+0000 7fa05cfd9700 1 -- 172.21.15.144:0/3720549653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0581003d0 msgr2=0x7fa0581007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:32.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.899+0000 7fa05cfd9700 1 --2- 172.21.15.144:0/3720549653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0581003d0 0x7fa0581007a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fa048009910 tx=0x7fa048009c20 comp rx=0 tx=0).stop 2024-09-10T12:04:32.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.899+0000 7fa05cfd9700 1 -- 172.21.15.144:0/3720549653 shutdown_connections 2024-09-10T12:04:32.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.899+0000 7fa05cfd9700 1 --2- 172.21.15.144:0/3720549653 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa058100d70 0x7fa058104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.899+0000 7fa05cfd9700 1 --2- 172.21.15.144:0/3720549653 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa0581003d0 0x7fa0581007a0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.899+0000 7fa05cfd9700 1 -- 172.21.15.144:0/3720549653 >> 172.21.15.144:0/3720549653 conn(0x7fa0580fbc90 msgr2=0x7fa0580fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:32.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.899+0000 7fa05cfd9700 1 -- 172.21.15.144:0/3720549653 shutdown_connections 2024-09-10T12:04:32.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.899+0000 7fa05cfd9700 1 -- 172.21.15.144:0/3720549653 wait complete. 2024-09-10T12:04:32.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.900+0000 7fa05cfd9700 1 Processor -- start 2024-09-10T12:04:32.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.900+0000 7fa05cfd9700 1 -- start start 2024-09-10T12:04:32.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.900+0000 7fa05cfd9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0581003d0 0x7fa058195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:32.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.900+0000 7fa05cfd9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa058100d70 0x7fa0581962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:32.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.900+0000 7fa05cfd9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa058196980 con 0x7fa058100d70 2024-09-10T12:04:32.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.900+0000 7fa05cfd9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05819a710 con 0x7fa0581003d0 2024-09-10T12:04:32.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.900+0000 7fa055d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa058100d70 0x7fa0581962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:32.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.900+0000 7fa055d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa058100d70 0x7fa0581962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:53014/0 (socket says 172.21.15.144:53014) 2024-09-10T12:04:32.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.900+0000 7fa055d9b700 1 -- 172.21.15.144:0/66288761 learned_addr learned my addr 172.21.15.144:0/66288761 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:04:32.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.900+0000 7fa05659c700 1 --2- 172.21.15.144:0/66288761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0581003d0 0x7fa058195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:32.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.901+0000 7fa055d9b700 1 -- 172.21.15.144:0/66288761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0581003d0 msgr2=0x7fa058195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:32.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.901+0000 7fa055d9b700 1 --2- 172.21.15.144:0/66288761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0581003d0 0x7fa058195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:32.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.901+0000 7fa055d9b700 1 -- 172.21.15.144:0/66288761 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0480095f0 con 0x7fa058100d70 2024-09-10T12:04:32.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.901+0000 7fa055d9b700 1 --2- 172.21.15.144:0/66288761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa058100d70 0x7fa0581962a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00d900 tx=0x7fa04c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:32.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.901+0000 7fa0477fe700 1 -- 172.21.15.144:0/66288761 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa04c009900 con 0x7fa058100d70 2024-09-10T12:04:32.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.901+0000 7fa0477fe700 1 -- 172.21.15.144:0/66288761 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa04c010460 con 0x7fa058100d70 2024-09-10T12:04:32.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.901+0000 7fa05cfd9700 1 -- 172.21.15.144:0/66288761 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05819a9f0 con 0x7fa058100d70 2024-09-10T12:04:32.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.901+0000 7fa0477fe700 1 -- 172.21.15.144:0/66288761 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa04c00f5f0 con 0x7fa058100d70 2024-09-10T12:04:32.905 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.901+0000 7fa05cfd9700 1 -- 172.21.15.144:0/66288761 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05819af40 con 0x7fa058100d70 2024-09-10T12:04:32.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.902+0000 7fa05cfd9700 1 -- 172.21.15.144:0/66288761 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0581086e0 con 0x7fa058100d70 2024-09-10T12:04:32.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.902+0000 7fa0477fe700 1 -- 172.21.15.144:0/66288761 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa04c0105d0 con 0x7fa058100d70 2024-09-10T12:04:32.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.903+0000 7fa0477fe700 1 --2- 172.21.15.144:0/66288761 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa04006e200 0x7fa0400706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:04:32.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.903+0000 7fa0477fe700 1 -- 172.21.15.144:0/66288761 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa04c08dd00 con 0x7fa058100d70 2024-09-10T12:04:32.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.903+0000 7fa05659c700 1 --2- 172.21.15.144:0/66288761 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa04006e200 0x7fa0400706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:04:32.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.903+0000 7fa05659c700 1 --2- 172.21.15.144:0/66288761 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa04006e200 0x7fa0400706b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa048006010 tx=0x7fa04801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:04:32.909 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:32.907+0000 7fa0477fe700 1 -- 172.21.15.144:0/66288761 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa04c057040 con 0x7fa058100d70 2024-09-10T12:04:33.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:33.196+0000 7fa05cfd9700 1 -- 172.21.15.144:0/66288761 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa05819b220 con 0x7fa058100d70 2024-09-10T12:04:33.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:33.196+0000 7fa0477fe700 1 -- 172.21.15.144:0/66288761 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7fa04c05a700 con 0x7fa058100d70 2024-09-10T12:04:33.199 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:04:33.199 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:04:33.199 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:04:33.199 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:04:33.199 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:04:33.199 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:04:33.199 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:04:33.199 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:04:33.199 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:04:33.199 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:04:33.199 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:04:33.200 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:04:33.200 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:04:33.200 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:04:33.200 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:04:33.200 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:04:33.200 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:04:33.200 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:04:33.200 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:04:33.200 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:04:33.200 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:04:33.200 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:04:33.201 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:04:33.201 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:04:33.201 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:04:33.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:33.198+0000 7fa05cfd9700 1 -- 172.21.15.144:0/66288761 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa04006e200 msgr2=0x7fa0400706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:33.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:33.198+0000 7fa05cfd9700 1 --2- 172.21.15.144:0/66288761 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa04006e200 0x7fa0400706b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fa048006010 tx=0x7fa04801a040 comp rx=0 tx=0).stop 2024-09-10T12:04:33.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:33.198+0000 7fa05cfd9700 1 -- 172.21.15.144:0/66288761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa058100d70 msgr2=0x7fa0581962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:04:33.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:33.198+0000 7fa05cfd9700 1 --2- 172.21.15.144:0/66288761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa058100d70 0x7fa0581962a0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00d900 tx=0x7fa04c00dc10 comp rx=0 tx=0).stop 2024-09-10T12:04:33.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:33.198+0000 7fa05cfd9700 1 -- 172.21.15.144:0/66288761 shutdown_connections 2024-09-10T12:04:33.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:33.198+0000 7fa05cfd9700 1 --2- 172.21.15.144:0/66288761 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa04006e200 0x7fa0400706b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:33.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:33.198+0000 7fa05cfd9700 1 --2- 172.21.15.144:0/66288761 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa0581003d0 0x7fa058195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:33.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:33.198+0000 7fa05cfd9700 1 --2- 172.21.15.144:0/66288761 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa058100d70 0x7fa0581962a0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:04:33.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:33.198+0000 7fa05cfd9700 1 -- 172.21.15.144:0/66288761 >> 172.21.15.144:0/66288761 conn(0x7fa0580fbc90 msgr2=0x7fa0580fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:04:33.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:33.198+0000 7fa05cfd9700 1 -- 172.21.15.144:0/66288761 shutdown_connections 2024-09-10T12:04:33.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:04:33.198+0000 7fa05cfd9700 1 -- 172.21.15.144:0/66288761 wait complete. 2024-09-10T12:04:33.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:33 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1270554659' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:04:33.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:33 smithi144 ceph-mon[87436]: from='client.35792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:04:33.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:33 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1270554659' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:04:33.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:33 smithi178 ceph-mon[65940]: from='client.35792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:04:34.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:34 smithi144 ceph-mon[87436]: pgmap v978: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:34.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:34 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/66288761' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:04:34.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:34 smithi178 ceph-mon[65940]: pgmap v978: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:34.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:34 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/66288761' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:04:36.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:36 smithi144 ceph-mon[87436]: pgmap v979: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:36.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:36 smithi178 ceph-mon[65940]: pgmap v979: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:38.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:38 smithi144 ceph-mon[87436]: pgmap v980: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:38 smithi178 ceph-mon[65940]: pgmap v980: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:40.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:40 smithi144 ceph-mon[87436]: pgmap v981: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:40.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:40 smithi178 ceph-mon[65940]: pgmap v981: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:42.419 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:42 smithi178 ceph-mon[65940]: pgmap v982: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:42.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:42 smithi144 ceph-mon[87436]: pgmap v982: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:44 smithi144 ceph-mon[87436]: pgmap v983: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:04:44.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:44 smithi178 ceph-mon[65940]: pgmap v983: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:44.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:04:46.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:46 smithi144 ceph-mon[87436]: pgmap v984: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:46.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:46 smithi178 ceph-mon[65940]: pgmap v984: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:48.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:48 smithi144 ceph-mon[87436]: pgmap v985: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:48 smithi178 ceph-mon[65940]: pgmap v985: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:50.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:50 smithi144 ceph-mon[87436]: pgmap v986: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:50.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:50 smithi178 ceph-mon[65940]: pgmap v986: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:52.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:52 smithi144 ceph-mon[87436]: pgmap v987: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:52.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:52 smithi178 ceph-mon[65940]: pgmap v987: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:54.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:54 smithi178 ceph-mon[65940]: pgmap v988: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:54.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:54 smithi144 ceph-mon[87436]: pgmap v988: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:56.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:56 smithi178 ceph-mon[65940]: pgmap v989: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:56.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:56 smithi144 ceph-mon[87436]: pgmap v989: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:58.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:58 smithi178 ceph-mon[65940]: pgmap v990: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:58 smithi144 ceph-mon[87436]: pgmap v990: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:04:59.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:04:59 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:04:59.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:04:59 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:05:00.492 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:00 smithi144 ceph-mon[87436]: pgmap v991: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:00 smithi178 ceph-mon[65940]: pgmap v991: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:02.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:02 smithi178 ceph-mon[65940]: pgmap v992: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:02.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:02 smithi144 ceph-mon[87436]: pgmap v992: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:03.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.316+0000 7fa7f6cfd700 1 -- 172.21.15.144:0/2376990339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f0100d70 msgr2=0x7fa7f0104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:03.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.316+0000 7fa7f6cfd700 1 --2- 172.21.15.144:0/2376990339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f0100d70 0x7fa7f0104d40 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc009910 tx=0x7fa7dc009c20 comp rx=0 tx=0).stop 2024-09-10T12:05:03.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.316+0000 7fa7f6cfd700 1 -- 172.21.15.144:0/2376990339 shutdown_connections 2024-09-10T12:05:03.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.316+0000 7fa7f6cfd700 1 --2- 172.21.15.144:0/2376990339 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f0100d70 0x7fa7f0104d40 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:03.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.316+0000 7fa7f6cfd700 1 --2- 172.21.15.144:0/2376990339 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f01003d0 0x7fa7f01007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:03.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.316+0000 7fa7f6cfd700 1 -- 172.21.15.144:0/2376990339 >> 172.21.15.144:0/2376990339 conn(0x7fa7f00fbc90 msgr2=0x7fa7f00fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:03.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.316+0000 7fa7f6cfd700 1 -- 172.21.15.144:0/2376990339 shutdown_connections 2024-09-10T12:05:03.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.317+0000 7fa7f6cfd700 1 -- 172.21.15.144:0/2376990339 wait complete. 2024-09-10T12:05:03.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.317+0000 7fa7f6cfd700 1 Processor -- start 2024-09-10T12:05:03.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.317+0000 7fa7f6cfd700 1 -- start start 2024-09-10T12:05:03.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.317+0000 7fa7f6cfd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f01003d0 0x7fa7f0195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:03.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.317+0000 7fa7f6cfd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f0100d70 0x7fa7f01963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:03.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.317+0000 7fa7f6cfd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f0196aa0 con 0x7fa7f01003d0 2024-09-10T12:05:03.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.317+0000 7fa7f6cfd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7f019a830 con 0x7fa7f0100d70 2024-09-10T12:05:03.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.317+0000 7fa7f4a99700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f01003d0 0x7fa7f0195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:03.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.318+0000 7fa7f4a99700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f01003d0 0x7fa7f0195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51576/0 (socket says 172.21.15.144:51576) 2024-09-10T12:05:03.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.318+0000 7fa7f4a99700 1 -- 172.21.15.144:0/3051454709 learned_addr learned my addr 172.21.15.144:0/3051454709 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:05:03.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.318+0000 7fa7effff700 1 --2- 172.21.15.144:0/3051454709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f0100d70 0x7fa7f01963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:03.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.318+0000 7fa7f4a99700 1 -- 172.21.15.144:0/3051454709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f0100d70 msgr2=0x7fa7f01963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:03.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.318+0000 7fa7f4a99700 1 --2- 172.21.15.144:0/3051454709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f0100d70 0x7fa7f01963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:03.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.318+0000 7fa7f4a99700 1 -- 172.21.15.144:0/3051454709 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc0095f0 con 0x7fa7f01003d0 2024-09-10T12:05:03.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.318+0000 7fa7f4a99700 1 --2- 172.21.15.144:0/3051454709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f01003d0 0x7fa7f0195e80 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400b4b0 tx=0x7fa7e400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:03.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.318+0000 7fa7edffb700 1 -- 172.21.15.144:0/3051454709 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7e4011800 con 0x7fa7f01003d0 2024-09-10T12:05:03.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.318+0000 7fa7edffb700 1 -- 172.21.15.144:0/3051454709 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa7e4011e20 con 0x7fa7f01003d0 2024-09-10T12:05:03.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.318+0000 7fa7f6cfd700 1 -- 172.21.15.144:0/3051454709 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7f019ab10 con 0x7fa7f01003d0 2024-09-10T12:05:03.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.318+0000 7fa7edffb700 1 -- 172.21.15.144:0/3051454709 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7e400f2e0 con 0x7fa7f01003d0 2024-09-10T12:05:03.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.318+0000 7fa7f6cfd700 1 -- 172.21.15.144:0/3051454709 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7f019b060 con 0x7fa7f01003d0 2024-09-10T12:05:03.323 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.319+0000 7fa7f6cfd700 1 -- 172.21.15.144:0/3051454709 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7f0108670 con 0x7fa7f01003d0 2024-09-10T12:05:03.323 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.320+0000 7fa7edffb700 1 -- 172.21.15.144:0/3051454709 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa7e4010380 con 0x7fa7f01003d0 2024-09-10T12:05:03.323 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.320+0000 7fa7edffb700 1 --2- 172.21.15.144:0/3051454709 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa7e006e200 0x7fa7e00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:03.323 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.320+0000 7fa7edffb700 1 -- 172.21.15.144:0/3051454709 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa7e408dc50 con 0x7fa7f01003d0 2024-09-10T12:05:03.323 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.321+0000 7fa7effff700 1 --2- 172.21.15.144:0/3051454709 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa7e006e200 0x7fa7e00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:03.323 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.321+0000 7fa7effff700 1 --2- 172.21.15.144:0/3051454709 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa7e006e200 0x7fa7e00706b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc006010 tx=0x7fa7dc00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:03.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.325+0000 7fa7edffb700 1 -- 172.21.15.144:0/3051454709 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa7e4057030 con 0x7fa7f01003d0 2024-09-10T12:05:03.548 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.547+0000 7fa7f6cfd700 1 -- 172.21.15.144:0/3051454709 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7f019b3a0 con 0x7fa7e006e200 2024-09-10T12:05:03.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.548+0000 7fa7edffb700 1 -- 172.21.15.144:0/3051454709 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fa7f019b3a0 con 0x7fa7e006e200 2024-09-10T12:05:03.551 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.550+0000 7fa7f6cfd700 1 -- 172.21.15.144:0/3051454709 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa7e006e200 msgr2=0x7fa7e00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:03.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.550+0000 7fa7f6cfd700 1 --2- 172.21.15.144:0/3051454709 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa7e006e200 0x7fa7e00706b0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc006010 tx=0x7fa7dc00b540 comp rx=0 tx=0).stop 2024-09-10T12:05:03.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.550+0000 7fa7f6cfd700 1 -- 172.21.15.144:0/3051454709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f01003d0 msgr2=0x7fa7f0195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:03.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.550+0000 7fa7f6cfd700 1 --2- 172.21.15.144:0/3051454709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f01003d0 0x7fa7f0195e80 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fa7e400b4b0 tx=0x7fa7e400b870 comp rx=0 tx=0).stop 2024-09-10T12:05:03.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.550+0000 7fa7f6cfd700 1 -- 172.21.15.144:0/3051454709 shutdown_connections 2024-09-10T12:05:03.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.550+0000 7fa7f6cfd700 1 --2- 172.21.15.144:0/3051454709 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa7e006e200 0x7fa7e00706b0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:03.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.550+0000 7fa7f6cfd700 1 --2- 172.21.15.144:0/3051454709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa7f01003d0 0x7fa7f0195e80 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:03.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.550+0000 7fa7f6cfd700 1 --2- 172.21.15.144:0/3051454709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa7f0100d70 0x7fa7f01963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:03.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.550+0000 7fa7f6cfd700 1 -- 172.21.15.144:0/3051454709 >> 172.21.15.144:0/3051454709 conn(0x7fa7f00fbc90 msgr2=0x7fa7f00fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:03.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.550+0000 7fa7f6cfd700 1 -- 172.21.15.144:0/3051454709 shutdown_connections 2024-09-10T12:05:03.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.550+0000 7fa7f6cfd700 1 -- 172.21.15.144:0/3051454709 wait complete. 2024-09-10T12:05:03.562 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:05:03.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.654+0000 7f1ac647d700 1 -- 172.21.15.144:0/2193528145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ac0101c90 msgr2=0x7f1ac010a190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:03.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.654+0000 7f1ac647d700 1 --2- 172.21.15.144:0/2193528145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ac0101c90 0x7f1ac010a190 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8009910 tx=0x7f1aa8009c20 comp rx=0 tx=0).stop 2024-09-10T12:05:03.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.654+0000 7f1ac647d700 1 -- 172.21.15.144:0/2193528145 shutdown_connections 2024-09-10T12:05:03.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.654+0000 7f1ac647d700 1 --2- 172.21.15.144:0/2193528145 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ac0101c90 0x7f1ac010a190 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:03.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.654+0000 7f1ac647d700 1 --2- 172.21.15.144:0/2193528145 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ac0101380 0x7f1ac0101750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:03.657 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.655+0000 7f1ac647d700 1 -- 172.21.15.144:0/2193528145 >> 172.21.15.144:0/2193528145 conn(0x7f1ac00fab80 msgr2=0x7f1ac00fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:03.657 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.655+0000 7f1ac647d700 1 -- 172.21.15.144:0/2193528145 shutdown_connections 2024-09-10T12:05:03.657 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.655+0000 7f1ac647d700 1 -- 172.21.15.144:0/2193528145 wait complete. 2024-09-10T12:05:03.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.655+0000 7f1ac647d700 1 Processor -- start 2024-09-10T12:05:03.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.655+0000 7f1ac647d700 1 -- start start 2024-09-10T12:05:03.658 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.656+0000 7f1ac647d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ac0101380 0x7f1ac00fefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:03.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.656+0000 7f1ac647d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ac0101c90 0x7f1ac00ff520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:03.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.656+0000 7f1ac647d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00ffa60 con 0x7f1ac0101c90 2024-09-10T12:05:03.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.656+0000 7f1ac647d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac00ffba0 con 0x7f1ac0101380 2024-09-10T12:05:03.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.656+0000 7f1abf7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ac0101c90 0x7f1ac00ff520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:03.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.656+0000 7f1abf7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ac0101c90 0x7f1ac00ff520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51590/0 (socket says 172.21.15.144:51590) 2024-09-10T12:05:03.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.656+0000 7f1abf7fe700 1 -- 172.21.15.144:0/3081510239 learned_addr learned my addr 172.21.15.144:0/3081510239 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:05:03.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.656+0000 7f1abffff700 1 --2- 172.21.15.144:0/3081510239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ac0101380 0x7f1ac00fefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:03.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.656+0000 7f1abf7fe700 1 -- 172.21.15.144:0/3081510239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ac0101380 msgr2=0x7f1ac00fefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:03.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.656+0000 7f1abf7fe700 1 --2- 172.21.15.144:0/3081510239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ac0101380 0x7f1ac00fefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:03.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.656+0000 7f1abf7fe700 1 -- 172.21.15.144:0/3081510239 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa80095f0 con 0x7f1ac0101c90 2024-09-10T12:05:03.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.656+0000 7f1abffff700 1 --2- 172.21.15.144:0/3081510239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ac0101380 0x7f1ac00fefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:05:03.659 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.656+0000 7f1abf7fe700 1 --2- 172.21.15.144:0/3081510239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ac0101c90 0x7f1ac00ff520 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8006010 tx=0x7f1aa800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:03.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.656+0000 7f1abd7fa700 1 -- 172.21.15.144:0/3081510239 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1aa801d070 con 0x7f1ac0101c90 2024-09-10T12:05:03.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.657+0000 7f1abd7fa700 1 -- 172.21.15.144:0/3081510239 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1aa8004fb0 con 0x7f1ac0101c90 2024-09-10T12:05:03.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.657+0000 7f1ac647d700 1 -- 172.21.15.144:0/3081510239 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac00ffe20 con 0x7f1ac0101c90 2024-09-10T12:05:03.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.657+0000 7f1abd7fa700 1 -- 172.21.15.144:0/3081510239 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1aa8022ea0 con 0x7f1ac0101c90 2024-09-10T12:05:03.660 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.657+0000 7f1ac647d700 1 -- 172.21.15.144:0/3081510239 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac01a2390 con 0x7f1ac0101c90 2024-09-10T12:05:03.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.658+0000 7f1ac647d700 1 -- 172.21.15.144:0/3081510239 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac0107800 con 0x7f1ac0101c90 2024-09-10T12:05:03.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.658+0000 7f1abd7fa700 1 -- 172.21.15.144:0/3081510239 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1aa8022470 con 0x7f1ac0101c90 2024-09-10T12:05:03.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.658+0000 7f1abd7fa700 1 --2- 172.21.15.144:0/3081510239 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1aac06e200 0x7f1aac0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:03.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.659+0000 7f1abd7fa700 1 -- 172.21.15.144:0/3081510239 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1aa808f5f0 con 0x7f1ac0101c90 2024-09-10T12:05:03.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.659+0000 7f1abffff700 1 --2- 172.21.15.144:0/3081510239 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1aac06e200 0x7f1aac0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:03.661 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.659+0000 7f1abffff700 1 --2- 172.21.15.144:0/3081510239 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1aac06e200 0x7f1aac0706b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0007900 tx=0x7f1ab0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:03.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.663+0000 7f1abd7fa700 1 -- 172.21.15.144:0/3081510239 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1aa8058a80 con 0x7f1ac0101c90 2024-09-10T12:05:03.883 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.881+0000 7f1ac647d700 1 -- 172.21.15.144:0/3081510239 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ac0061190 con 0x7f1aac06e200 2024-09-10T12:05:03.883 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.882+0000 7f1abd7fa700 1 -- 172.21.15.144:0/3081510239 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1ac0061190 con 0x7f1aac06e200 2024-09-10T12:05:03.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.885+0000 7f1ac647d700 1 -- 172.21.15.144:0/3081510239 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1aac06e200 msgr2=0x7f1aac0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:03.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.885+0000 7f1ac647d700 1 --2- 172.21.15.144:0/3081510239 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1aac06e200 0x7f1aac0706b0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1ab0007900 tx=0x7f1ab0008040 comp rx=0 tx=0).stop 2024-09-10T12:05:03.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.885+0000 7f1ac647d700 1 -- 172.21.15.144:0/3081510239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ac0101c90 msgr2=0x7f1ac00ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:03.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.885+0000 7f1ac647d700 1 --2- 172.21.15.144:0/3081510239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ac0101c90 0x7f1ac00ff520 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f1aa8006010 tx=0x7f1aa800be20 comp rx=0 tx=0).stop 2024-09-10T12:05:03.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.885+0000 7f1ac647d700 1 -- 172.21.15.144:0/3081510239 shutdown_connections 2024-09-10T12:05:03.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.885+0000 7f1ac647d700 1 --2- 172.21.15.144:0/3081510239 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1aac06e200 0x7f1aac0706b0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:03.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.885+0000 7f1ac647d700 1 --2- 172.21.15.144:0/3081510239 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ac0101380 0x7f1ac00fefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:03.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.885+0000 7f1ac647d700 1 --2- 172.21.15.144:0/3081510239 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ac0101c90 0x7f1ac00ff520 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:03.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.885+0000 7f1ac647d700 1 -- 172.21.15.144:0/3081510239 >> 172.21.15.144:0/3081510239 conn(0x7f1ac00fab80 msgr2=0x7f1ac0104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:03.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.885+0000 7f1ac647d700 1 -- 172.21.15.144:0/3081510239 shutdown_connections 2024-09-10T12:05:03.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.885+0000 7f1ac647d700 1 -- 172.21.15.144:0/3081510239 wait complete. 2024-09-10T12:05:03.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.989+0000 7f7c528e1700 1 -- 172.21.15.144:0/4217819532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c0ffd30 msgr2=0x7f7c4c103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:03.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.989+0000 7f7c528e1700 1 --2- 172.21.15.144:0/4217819532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c0ffd30 0x7f7c4c103ce0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c009910 tx=0x7f7c3c009c20 comp rx=0 tx=0).stop 2024-09-10T12:05:03.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.989+0000 7f7c528e1700 1 -- 172.21.15.144:0/4217819532 shutdown_connections 2024-09-10T12:05:03.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.989+0000 7f7c528e1700 1 --2- 172.21.15.144:0/4217819532 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c0ffd30 0x7f7c4c103ce0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:03.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.989+0000 7f7c528e1700 1 --2- 172.21.15.144:0/4217819532 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c4c0ff390 0x7f7c4c0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:03.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.989+0000 7f7c528e1700 1 -- 172.21.15.144:0/4217819532 >> 172.21.15.144:0/4217819532 conn(0x7f7c4c074f50 msgr2=0x7f7c4c075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:03.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.989+0000 7f7c528e1700 1 -- 172.21.15.144:0/4217819532 shutdown_connections 2024-09-10T12:05:03.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.989+0000 7f7c528e1700 1 -- 172.21.15.144:0/4217819532 wait complete. 2024-09-10T12:05:03.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.990+0000 7f7c528e1700 1 Processor -- start 2024-09-10T12:05:03.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.990+0000 7f7c528e1700 1 -- start start 2024-09-10T12:05:03.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.990+0000 7f7c528e1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c4c0ff390 0x7f7c4c195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:03.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.990+0000 7f7c528e1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c0ffd30 0x7f7c4c1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:03.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.990+0000 7f7c528e1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c1969b0 con 0x7f7c4c0ffd30 2024-09-10T12:05:03.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.990+0000 7f7c528e1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c4c19a740 con 0x7f7c4c0ff390 2024-09-10T12:05:03.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.990+0000 7f7c4b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c0ffd30 0x7f7c4c1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:03.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.990+0000 7f7c4b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c0ffd30 0x7f7c4c1962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51594/0 (socket says 172.21.15.144:51594) 2024-09-10T12:05:03.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.990+0000 7f7c4b7fe700 1 -- 172.21.15.144:0/4202034991 learned_addr learned my addr 172.21.15.144:0/4202034991 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:05:03.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.990+0000 7f7c4bfff700 1 --2- 172.21.15.144:0/4202034991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c4c0ff390 0x7f7c4c195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:03.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.991+0000 7f7c4b7fe700 1 -- 172.21.15.144:0/4202034991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c4c0ff390 msgr2=0x7f7c4c195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:03.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.991+0000 7f7c4b7fe700 1 --2- 172.21.15.144:0/4202034991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c4c0ff390 0x7f7c4c195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:03.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.991+0000 7f7c4b7fe700 1 -- 172.21.15.144:0/4202034991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3c0095f0 con 0x7f7c4c0ffd30 2024-09-10T12:05:03.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.991+0000 7f7c4b7fe700 1 --2- 172.21.15.144:0/4202034991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c0ffd30 0x7f7c4c1962d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0059d0 tx=0x7f7c3c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:03.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.991+0000 7f7c497fa700 1 -- 172.21.15.144:0/4202034991 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3c01d070 con 0x7f7c4c0ffd30 2024-09-10T12:05:03.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.991+0000 7f7c497fa700 1 -- 172.21.15.144:0/4202034991 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c3c00b610 con 0x7f7c4c0ffd30 2024-09-10T12:05:03.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.991+0000 7f7c528e1700 1 -- 172.21.15.144:0/4202034991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c4c19a9c0 con 0x7f7c4c0ffd30 2024-09-10T12:05:03.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.991+0000 7f7c497fa700 1 -- 172.21.15.144:0/4202034991 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3c022470 con 0x7f7c4c0ffd30 2024-09-10T12:05:03.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.991+0000 7f7c528e1700 1 -- 172.21.15.144:0/4202034991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c4c19aeb0 con 0x7f7c4c0ffd30 2024-09-10T12:05:03.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.992+0000 7f7c528e1700 1 -- 172.21.15.144:0/4202034991 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c4c04ea50 con 0x7f7c4c0ffd30 2024-09-10T12:05:03.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.993+0000 7f7c497fa700 1 -- 172.21.15.144:0/4202034991 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7c3c003710 con 0x7f7c4c0ffd30 2024-09-10T12:05:03.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.993+0000 7f7c497fa700 1 --2- 172.21.15.144:0/4202034991 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c3406e200 0x7f7c340706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:03.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.993+0000 7f7c497fa700 1 -- 172.21.15.144:0/4202034991 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7c3c08f570 con 0x7f7c4c0ffd30 2024-09-10T12:05:03.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.993+0000 7f7c4bfff700 1 --2- 172.21.15.144:0/4202034991 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c3406e200 0x7f7c340706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:03.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.994+0000 7f7c4bfff700 1 --2- 172.21.15.144:0/4202034991 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c3406e200 0x7f7c340706b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009c60 tx=0x7f7c40009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:03.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:03.998+0000 7f7c497fa700 1 -- 172.21.15.144:0/4202034991 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7c3c058950 con 0x7f7c4c0ffd30 2024-09-10T12:05:04.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.209+0000 7f7c528e1700 1 -- 172.21.15.144:0/4202034991 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c4c19b390 con 0x7f7c3406e200 2024-09-10T12:05:04.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.218+0000 7f7c497fa700 1 -- 172.21.15.144:0/4202034991 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f7c4c19b390 con 0x7f7c3406e200 2024-09-10T12:05:04.220 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:05:04.220 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (33m) 11m ago 43m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:05:04.220 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (44m) 11m ago 44m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:05:04.220 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (43m) 11m ago 43m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:05:04.220 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (30m) 11m ago 44m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:05:04.220 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (30m) 11m ago 43m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:05:04.220 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (32m) 11m ago 43m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:05:04.220 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (27m) 11m ago 38m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:05:04.220 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (27m) 11m ago 38m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:05:04.221 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (11m) 11m ago 38m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:05:04.221 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (26m) 11m ago 38m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:05:04.221 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (35m) 11m ago 45m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:05:04.221 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (34m) 11m ago 42m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:05:04.221 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (31m) 11m ago 45m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:05:04.221 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (31m) 11m ago 42m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:05:04.221 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (33m) 11m ago 43m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:05:04.221 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (33m) 11m ago 42m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:05:04.221 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (30m) 11m ago 42m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:05:04.221 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (29m) 11m ago 41m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:05:04.221 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (29m) 11m ago 41m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:05:04.221 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (28m) 11m ago 40m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:05:04.221 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (28m) 11m ago 40m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:05:04.222 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (27m) 11m ago 39m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:05:04.222 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (33m) 11m ago 43m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:05:04.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.221+0000 7f7c528e1700 1 -- 172.21.15.144:0/4202034991 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c3406e200 msgr2=0x7f7c340706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:04.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.221+0000 7f7c528e1700 1 --2- 172.21.15.144:0/4202034991 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c3406e200 0x7f7c340706b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7c40009c60 tx=0x7f7c40009260 comp rx=0 tx=0).stop 2024-09-10T12:05:04.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.221+0000 7f7c528e1700 1 -- 172.21.15.144:0/4202034991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c0ffd30 msgr2=0x7f7c4c1962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:04.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.221+0000 7f7c528e1700 1 --2- 172.21.15.144:0/4202034991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c0ffd30 0x7f7c4c1962d0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7c3c0059d0 tx=0x7f7c3c00f690 comp rx=0 tx=0).stop 2024-09-10T12:05:04.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.221+0000 7f7c528e1700 1 -- 172.21.15.144:0/4202034991 shutdown_connections 2024-09-10T12:05:04.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.221+0000 7f7c528e1700 1 --2- 172.21.15.144:0/4202034991 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7c3406e200 0x7f7c340706b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.221+0000 7f7c528e1700 1 --2- 172.21.15.144:0/4202034991 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7c4c0ff390 0x7f7c4c195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.221+0000 7f7c528e1700 1 --2- 172.21.15.144:0/4202034991 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7c4c0ffd30 0x7f7c4c1962d0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.221+0000 7f7c528e1700 1 -- 172.21.15.144:0/4202034991 >> 172.21.15.144:0/4202034991 conn(0x7f7c4c074f50 msgr2=0x7f7c4c103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:04.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.221+0000 7f7c528e1700 1 -- 172.21.15.144:0/4202034991 shutdown_connections 2024-09-10T12:05:04.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.221+0000 7f7c528e1700 1 -- 172.21.15.144:0/4202034991 wait complete. 2024-09-10T12:05:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.324+0000 7fdccfd4b700 1 -- 172.21.15.144:0/764996736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc8101c30 msgr2=0x7fdcc810a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.324+0000 7fdccfd4b700 1 --2- 172.21.15.144:0/764996736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc8101c30 0x7fdcc810a130 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4009940 tx=0x7fdcc4009c50 comp rx=0 tx=0).stop 2024-09-10T12:05:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.324+0000 7fdccfd4b700 1 -- 172.21.15.144:0/764996736 shutdown_connections 2024-09-10T12:05:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.324+0000 7fdccfd4b700 1 --2- 172.21.15.144:0/764996736 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc8101c30 0x7fdcc810a130 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.324+0000 7fdccfd4b700 1 --2- 172.21.15.144:0/764996736 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcc8101320 0x7fdcc81016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.324+0000 7fdccfd4b700 1 -- 172.21.15.144:0/764996736 >> 172.21.15.144:0/764996736 conn(0x7fdcc80fab80 msgr2=0x7fdcc80fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:04.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.324+0000 7fdccfd4b700 1 -- 172.21.15.144:0/764996736 shutdown_connections 2024-09-10T12:05:04.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.324+0000 7fdccfd4b700 1 -- 172.21.15.144:0/764996736 wait complete. 2024-09-10T12:05:04.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.325+0000 7fdccfd4b700 1 Processor -- start 2024-09-10T12:05:04.327 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.325+0000 7fdccfd4b700 1 -- start start 2024-09-10T12:05:04.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.325+0000 7fdccfd4b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc8101320 0x7fdcc8195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:04.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.325+0000 7fdccfd4b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcc8101c30 0x7fdcc8196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:04.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.325+0000 7fdccfd4b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc8196a20 con 0x7fdcc8101320 2024-09-10T12:05:04.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.325+0000 7fdccfd4b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcc819a7b0 con 0x7fdcc8101c30 2024-09-10T12:05:04.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.325+0000 7fdccdae7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc8101320 0x7fdcc8195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:04.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.325+0000 7fdccdae7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc8101320 0x7fdcc8195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51610/0 (socket says 172.21.15.144:51610) 2024-09-10T12:05:04.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.325+0000 7fdccdae7700 1 -- 172.21.15.144:0/2878431613 learned_addr learned my addr 172.21.15.144:0/2878431613 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:05:04.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.326+0000 7fdccd2e6700 1 --2- 172.21.15.144:0/2878431613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcc8101c30 0x7fdcc8196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:04.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.326+0000 7fdccdae7700 1 -- 172.21.15.144:0/2878431613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcc8101c30 msgr2=0x7fdcc8196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:04.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.326+0000 7fdccdae7700 1 --2- 172.21.15.144:0/2878431613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcc8101c30 0x7fdcc8196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.326+0000 7fdccdae7700 1 -- 172.21.15.144:0/2878431613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcc40095f0 con 0x7fdcc8101320 2024-09-10T12:05:04.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.326+0000 7fdccdae7700 1 --2- 172.21.15.144:0/2878431613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc8101320 0x7fdcc8195e00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800b4b0 tx=0x7fdcb800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:04.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.326+0000 7fdcbeffd700 1 -- 172.21.15.144:0/2878431613 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcb8011800 con 0x7fdcc8101320 2024-09-10T12:05:04.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.326+0000 7fdcbeffd700 1 -- 172.21.15.144:0/2878431613 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdcb8011e20 con 0x7fdcc8101320 2024-09-10T12:05:04.329 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.326+0000 7fdccfd4b700 1 -- 172.21.15.144:0/2878431613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcc819aa90 con 0x7fdcc8101320 2024-09-10T12:05:04.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.326+0000 7fdcbeffd700 1 -- 172.21.15.144:0/2878431613 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcb800f2e0 con 0x7fdcc8101320 2024-09-10T12:05:04.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.326+0000 7fdccfd4b700 1 -- 172.21.15.144:0/2878431613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcc819afe0 con 0x7fdcc8101320 2024-09-10T12:05:04.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.327+0000 7fdccfd4b700 1 -- 172.21.15.144:0/2878431613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcc81077a0 con 0x7fdcc8101320 2024-09-10T12:05:04.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.328+0000 7fdcbeffd700 1 -- 172.21.15.144:0/2878431613 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fdcb8010380 con 0x7fdcc8101320 2024-09-10T12:05:04.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.328+0000 7fdcbeffd700 1 --2- 172.21.15.144:0/2878431613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdcb406e200 0x7fdcb40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:04.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.328+0000 7fdccd2e6700 1 --2- 172.21.15.144:0/2878431613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdcb406e200 0x7fdcb40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:04.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.328+0000 7fdcbeffd700 1 -- 172.21.15.144:0/2878431613 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdcb808dab0 con 0x7fdcc8101320 2024-09-10T12:05:04.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.328+0000 7fdccd2e6700 1 --2- 172.21.15.144:0/2878431613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdcb406e200 0x7fdcb40706b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8197420 tx=0x7fdcc400b7b0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:04.333 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.332+0000 7fdcbeffd700 1 -- 172.21.15.144:0/2878431613 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdcb8056f10 con 0x7fdcc8101320 2024-09-10T12:05:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:04 smithi178 ceph-mon[65940]: pgmap v993: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:04 smithi178 ceph-mon[65940]: from='client.35800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:04.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:04 smithi178 ceph-mon[65940]: from='client.35804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:04.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.618+0000 7fdccfd4b700 1 -- 172.21.15.144:0/2878431613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdcc804ea50 con 0x7fdcc8101320 2024-09-10T12:05:04.620 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:04 smithi144 ceph-mon[87436]: pgmap v993: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:04.620 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:04 smithi144 ceph-mon[87436]: from='client.35800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:04.620 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:04 smithi144 ceph-mon[87436]: from='client.35804 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:04.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.618+0000 7fdcbeffd700 1 -- 172.21.15.144:0/2878431613 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fdcb805a5d0 con 0x7fdcc8101320 2024-09-10T12:05:04.620 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:05:04.620 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:05:04.620 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:05:04.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:05:04.621 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:05:04.621 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:05:04.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:05:04.621 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:05:04.621 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:05:04.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:05:04.621 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:05:04.621 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:05:04.621 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:05:04.621 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:05:04.621 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:05:04.621 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:05:04.622 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:05:04.622 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.621+0000 7fdccfd4b700 1 -- 172.21.15.144:0/2878431613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdcb406e200 msgr2=0x7fdcb40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:04.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.621+0000 7fdccfd4b700 1 --2- 172.21.15.144:0/2878431613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdcb406e200 0x7fdcb40706b0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fdcc8197420 tx=0x7fdcc400b7b0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.621+0000 7fdccfd4b700 1 -- 172.21.15.144:0/2878431613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc8101320 msgr2=0x7fdcc8195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:04.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.621+0000 7fdccfd4b700 1 --2- 172.21.15.144:0/2878431613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc8101320 0x7fdcc8195e00 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800b4b0 tx=0x7fdcb800b7c0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.621+0000 7fdccfd4b700 1 -- 172.21.15.144:0/2878431613 shutdown_connections 2024-09-10T12:05:04.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.621+0000 7fdccfd4b700 1 --2- 172.21.15.144:0/2878431613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdcb406e200 0x7fdcb40706b0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.621+0000 7fdccfd4b700 1 --2- 172.21.15.144:0/2878431613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdcc8101320 0x7fdcc8195e00 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.621+0000 7fdccfd4b700 1 --2- 172.21.15.144:0/2878431613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdcc8101c30 0x7fdcc8196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.621+0000 7fdccfd4b700 1 -- 172.21.15.144:0/2878431613 >> 172.21.15.144:0/2878431613 conn(0x7fdcc80fab80 msgr2=0x7fdcc81048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:04.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.621+0000 7fdccfd4b700 1 -- 172.21.15.144:0/2878431613 shutdown_connections 2024-09-10T12:05:04.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.621+0000 7fdccfd4b700 1 -- 172.21.15.144:0/2878431613 wait complete. 2024-09-10T12:05:04.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.725+0000 7faacf50c700 1 -- 172.21.15.144:0/2663982716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faac81003e0 msgr2=0x7faac81007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:04.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.725+0000 7faacf50c700 1 --2- 172.21.15.144:0/2663982716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faac81003e0 0x7faac81007b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7faab8009910 tx=0x7faab8009c20 comp rx=0 tx=0).stop 2024-09-10T12:05:04.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.725+0000 7faacf50c700 1 -- 172.21.15.144:0/2663982716 shutdown_connections 2024-09-10T12:05:04.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.725+0000 7faacf50c700 1 --2- 172.21.15.144:0/2663982716 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faac8100d80 0x7faac8104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.725+0000 7faacf50c700 1 --2- 172.21.15.144:0/2663982716 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faac81003e0 0x7faac81007b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.725+0000 7faacf50c700 1 -- 172.21.15.144:0/2663982716 >> 172.21.15.144:0/2663982716 conn(0x7faac80fbca0 msgr2=0x7faac80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:04.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.725+0000 7faacf50c700 1 -- 172.21.15.144:0/2663982716 shutdown_connections 2024-09-10T12:05:04.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.725+0000 7faacf50c700 1 -- 172.21.15.144:0/2663982716 wait complete. 2024-09-10T12:05:04.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.726+0000 7faacf50c700 1 Processor -- start 2024-09-10T12:05:04.729 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.726+0000 7faacf50c700 1 -- start start 2024-09-10T12:05:04.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.726+0000 7faacf50c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faac81003e0 0x7faac8073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:04.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.726+0000 7faacf50c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faac8100d80 0x7faac8073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:04.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.726+0000 7faacf50c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac8073ad0 con 0x7faac8100d80 2024-09-10T12:05:04.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.726+0000 7faacf50c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac8073c10 con 0x7faac81003e0 2024-09-10T12:05:04.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.726+0000 7faaccaa7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faac8100d80 0x7faac8073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:04.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.726+0000 7faaccaa7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faac8100d80 0x7faac8073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51622/0 (socket says 172.21.15.144:51622) 2024-09-10T12:05:04.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.726+0000 7faaccaa7700 1 -- 172.21.15.144:0/3055350424 learned_addr learned my addr 172.21.15.144:0/3055350424 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:05:04.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.727+0000 7faacd2a8700 1 --2- 172.21.15.144:0/3055350424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faac81003e0 0x7faac8073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:04.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.727+0000 7faaccaa7700 1 -- 172.21.15.144:0/3055350424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faac81003e0 msgr2=0x7faac8073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:04.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.727+0000 7faaccaa7700 1 --2- 172.21.15.144:0/3055350424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faac81003e0 0x7faac8073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.727+0000 7faaccaa7700 1 -- 172.21.15.144:0/3055350424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab80095f0 con 0x7faac8100d80 2024-09-10T12:05:04.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.727+0000 7faacd2a8700 1 --2- 172.21.15.144:0/3055350424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faac81003e0 0x7faac8073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:05:04.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.727+0000 7faaccaa7700 1 --2- 172.21.15.144:0/3055350424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faac8100d80 0x7faac8073590 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7faac400c8f0 tx=0x7faac400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:04.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.727+0000 7faabe7fc700 1 -- 172.21.15.144:0/3055350424 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faac4007a50 con 0x7faac8100d80 2024-09-10T12:05:04.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.727+0000 7faabe7fc700 1 -- 172.21.15.144:0/3055350424 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faac4007bb0 con 0x7faac8100d80 2024-09-10T12:05:04.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.727+0000 7faabe7fc700 1 -- 172.21.15.144:0/3055350424 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faac4018630 con 0x7faac8100d80 2024-09-10T12:05:04.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.727+0000 7faacf50c700 1 -- 172.21.15.144:0/3055350424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac81a1ef0 con 0x7faac8100d80 2024-09-10T12:05:04.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.727+0000 7faacf50c700 1 -- 172.21.15.144:0/3055350424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac81a2300 con 0x7faac8100d80 2024-09-10T12:05:04.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.728+0000 7faacf50c700 1 -- 172.21.15.144:0/3055350424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac81086f0 con 0x7faac8100d80 2024-09-10T12:05:04.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.729+0000 7faabe7fc700 1 -- 172.21.15.144:0/3055350424 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7faac4007d20 con 0x7faac8100d80 2024-09-10T12:05:04.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.729+0000 7faabe7fc700 1 --2- 172.21.15.144:0/3055350424 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faab406e200 0x7faab40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:04.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.729+0000 7faabe7fc700 1 -- 172.21.15.144:0/3055350424 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7faac408df70 con 0x7faac8100d80 2024-09-10T12:05:04.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.729+0000 7faacd2a8700 1 --2- 172.21.15.144:0/3055350424 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faab406e200 0x7faab40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:04.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.730+0000 7faacd2a8700 1 --2- 172.21.15.144:0/3055350424 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faab406e200 0x7faab40706b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faab800b5c0 tx=0x7faab8005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:04.735 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.734+0000 7faabe7fc700 1 -- 172.21.15.144:0/3055350424 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7faac40c31b0 con 0x7faac8100d80 2024-09-10T12:05:04.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.977+0000 7faacf50c700 1 -- 172.21.15.144:0/3055350424 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7faac804ea50 con 0x7faac8100d80 2024-09-10T12:05:04.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.978+0000 7faabe7fc700 1 -- 172.21.15.144:0/3055350424 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7faac40573d0 con 0x7faac8100d80 2024-09-10T12:05:04.979 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:05:04.979 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:05:04.979 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:05:04.979 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:05:04.980 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:05:04.980 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:05:04.980 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:05:04.980 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:05:04.980 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:05:04.980 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:05:04.980 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:05:04.980 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:05:04.980 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:05:04.980 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:05:04.980 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:05:04.980 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:05:04.980 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:05:04.981 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:05:04.981 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:05:04.981 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:05:04.981 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:05:04.981 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:05:04.981 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:05:04.981 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:05:04.981 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:05:04.981 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:05:04.981 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:05:04.981 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:05:04.981 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:05:04.981 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:05:04.982 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:05:04.982 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:05:04.982 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:05:04.982 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:05:04.982 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:05:04.982 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:05:04.982 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:05:04.982 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:05:04.982 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:05:04.982 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:05:04.982 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:05:04.982 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:05:04.982 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:05:04.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.980+0000 7faacf50c700 1 -- 172.21.15.144:0/3055350424 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faab406e200 msgr2=0x7faab40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:04.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.980+0000 7faacf50c700 1 --2- 172.21.15.144:0/3055350424 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faab406e200 0x7faab40706b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7faab800b5c0 tx=0x7faab8005480 comp rx=0 tx=0).stop 2024-09-10T12:05:04.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.980+0000 7faacf50c700 1 -- 172.21.15.144:0/3055350424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faac8100d80 msgr2=0x7faac8073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:04.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.980+0000 7faacf50c700 1 --2- 172.21.15.144:0/3055350424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faac8100d80 0x7faac8073590 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7faac400c8f0 tx=0x7faac400ccb0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.980+0000 7faacf50c700 1 -- 172.21.15.144:0/3055350424 shutdown_connections 2024-09-10T12:05:04.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.980+0000 7faacf50c700 1 --2- 172.21.15.144:0/3055350424 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7faab406e200 0x7faab40706b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.980+0000 7faacf50c700 1 --2- 172.21.15.144:0/3055350424 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7faac81003e0 0x7faac8073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.980+0000 7faacf50c700 1 --2- 172.21.15.144:0/3055350424 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7faac8100d80 0x7faac8073590 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:04.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.980+0000 7faacf50c700 1 -- 172.21.15.144:0/3055350424 >> 172.21.15.144:0/3055350424 conn(0x7faac80fbca0 msgr2=0x7faac80fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:04.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.980+0000 7faacf50c700 1 -- 172.21.15.144:0/3055350424 shutdown_connections 2024-09-10T12:05:04.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:04.980+0000 7faacf50c700 1 -- 172.21.15.144:0/3055350424 wait complete. 2024-09-10T12:05:04.984 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:05:05.087 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.086+0000 7f823241b700 1 -- 172.21.15.144:0/3619368932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c1048c0 msgr2=0x7f822c104c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:05.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.086+0000 7f823241b700 1 --2- 172.21.15.144:0/3619368932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c1048c0 0x7f822c104c90 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f821c009910 tx=0x7f821c009c20 comp rx=0 tx=0).stop 2024-09-10T12:05:05.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.086+0000 7f823241b700 1 -- 172.21.15.144:0/3619368932 shutdown_connections 2024-09-10T12:05:05.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.086+0000 7f823241b700 1 --2- 172.21.15.144:0/3619368932 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f822c0ff270 0x7f822c0ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:05.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.086+0000 7f823241b700 1 --2- 172.21.15.144:0/3619368932 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c1048c0 0x7f822c104c90 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:05.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.086+0000 7f823241b700 1 -- 172.21.15.144:0/3619368932 >> 172.21.15.144:0/3619368932 conn(0x7f822c074850 msgr2=0x7f822c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:05.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.086+0000 7f823241b700 1 -- 172.21.15.144:0/3619368932 shutdown_connections 2024-09-10T12:05:05.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.086+0000 7f823241b700 1 -- 172.21.15.144:0/3619368932 wait complete. 2024-09-10T12:05:05.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.086+0000 7f823241b700 1 Processor -- start 2024-09-10T12:05:05.088 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.086+0000 7f823241b700 1 -- start start 2024-09-10T12:05:05.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.087+0000 7f823241b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f822c0ff270 0x7f822c19bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:05.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.087+0000 7f823241b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c1048c0 0x7f822c19c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:05.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.087+0000 7f823241b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c19c6a0 con 0x7f822c1048c0 2024-09-10T12:05:05.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.087+0000 7f823241b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f822c195b50 con 0x7f822c0ff270 2024-09-10T12:05:05.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.087+0000 7f822b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c1048c0 0x7f822c19c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:05.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.087+0000 7f822b7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c1048c0 0x7f822c19c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51638/0 (socket says 172.21.15.144:51638) 2024-09-10T12:05:05.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.087+0000 7f822b7fe700 1 -- 172.21.15.144:0/1063740278 learned_addr learned my addr 172.21.15.144:0/1063740278 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:05:05.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.087+0000 7f822bfff700 1 --2- 172.21.15.144:0/1063740278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f822c0ff270 0x7f822c19bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:05.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.087+0000 7f822b7fe700 1 -- 172.21.15.144:0/1063740278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f822c0ff270 msgr2=0x7f822c19bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:05.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.087+0000 7f822b7fe700 1 --2- 172.21.15.144:0/1063740278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f822c0ff270 0x7f822c19bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:05.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.087+0000 7f822b7fe700 1 -- 172.21.15.144:0/1063740278 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c0095f0 con 0x7f822c1048c0 2024-09-10T12:05:05.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.087+0000 7f822b7fe700 1 --2- 172.21.15.144:0/1063740278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c1048c0 0x7f822c19c010 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f822000d8d0 tx=0x7f822000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:05.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.088+0000 7f82297fa700 1 -- 172.21.15.144:0/1063740278 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8220009940 con 0x7f822c1048c0 2024-09-10T12:05:05.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.088+0000 7f82297fa700 1 -- 172.21.15.144:0/1063740278 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8220010460 con 0x7f822c1048c0 2024-09-10T12:05:05.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.088+0000 7f823241b700 1 -- 172.21.15.144:0/1063740278 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f822c195e30 con 0x7f822c1048c0 2024-09-10T12:05:05.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.088+0000 7f82297fa700 1 -- 172.21.15.144:0/1063740278 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f822000f5f0 con 0x7f822c1048c0 2024-09-10T12:05:05.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.088+0000 7f823241b700 1 -- 172.21.15.144:0/1063740278 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f822c196380 con 0x7f822c1048c0 2024-09-10T12:05:05.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.089+0000 7f823241b700 1 -- 172.21.15.144:0/1063740278 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f822c04ea50 con 0x7f822c1048c0 2024-09-10T12:05:05.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.089+0000 7f82297fa700 1 -- 172.21.15.144:0/1063740278 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f82200105d0 con 0x7f822c1048c0 2024-09-10T12:05:05.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.090+0000 7f82297fa700 1 --2- 172.21.15.144:0/1063740278 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f821406e200 0x7f82140706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:05.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.090+0000 7f82297fa700 1 -- 172.21.15.144:0/1063740278 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f822008df90 con 0x7f822c1048c0 2024-09-10T12:05:05.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.090+0000 7f822bfff700 1 --2- 172.21.15.144:0/1063740278 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f821406e200 0x7f82140706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:05.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.090+0000 7f822bfff700 1 --2- 172.21.15.144:0/1063740278 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f821406e200 0x7f82140706b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f821c006010 tx=0x7f821c00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:05.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.094+0000 7f82297fa700 1 -- 172.21.15.144:0/1063740278 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f822005af20 con 0x7f822c1048c0 2024-09-10T12:05:05.316 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.314+0000 7f823241b700 1 -- 172.21.15.144:0/1063740278 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f822c1967a0 con 0x7f821406e200 2024-09-10T12:05:05.316 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:05 smithi144 ceph-mon[87436]: from='client.35808 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:05.316 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:05 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2878431613' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:05:05.316 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:05 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3055350424' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:05:05.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.316+0000 7f82297fa700 1 -- 172.21.15.144:0/1063740278 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f822c1967a0 con 0x7f821406e200 2024-09-10T12:05:05.317 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:05:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:05:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:05:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:05:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:05:05.317 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:05:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:05:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:05:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:05:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:05:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:05:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:05:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:05:05.318 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:05:05.318 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:05:05.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.318+0000 7f823241b700 1 -- 172.21.15.144:0/1063740278 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f821406e200 msgr2=0x7f82140706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:05.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.318+0000 7f823241b700 1 --2- 172.21.15.144:0/1063740278 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f821406e200 0x7f82140706b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f821c006010 tx=0x7f821c00b540 comp rx=0 tx=0).stop 2024-09-10T12:05:05.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.318+0000 7f823241b700 1 -- 172.21.15.144:0/1063740278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c1048c0 msgr2=0x7f822c19c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:05.319 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.318+0000 7f823241b700 1 --2- 172.21.15.144:0/1063740278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c1048c0 0x7f822c19c010 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f822000d8d0 tx=0x7f822000dc90 comp rx=0 tx=0).stop 2024-09-10T12:05:05.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.318+0000 7f823241b700 1 -- 172.21.15.144:0/1063740278 shutdown_connections 2024-09-10T12:05:05.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.318+0000 7f823241b700 1 --2- 172.21.15.144:0/1063740278 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f821406e200 0x7f82140706b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:05.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.318+0000 7f823241b700 1 --2- 172.21.15.144:0/1063740278 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f822c0ff270 0x7f822c19bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:05.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.318+0000 7f823241b700 1 --2- 172.21.15.144:0/1063740278 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f822c1048c0 0x7f822c19c010 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:05.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.318+0000 7f823241b700 1 -- 172.21.15.144:0/1063740278 >> 172.21.15.144:0/1063740278 conn(0x7f822c074850 msgr2=0x7f822c0fe880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:05.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.318+0000 7f823241b700 1 -- 172.21.15.144:0/1063740278 shutdown_connections 2024-09-10T12:05:05.320 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.318+0000 7f823241b700 1 -- 172.21.15.144:0/1063740278 wait complete. 2024-09-10T12:05:05.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.422+0000 7fe66485b700 1 -- 172.21.15.144:0/3376740371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff2c0 msgr2=0x7fe6600ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:05.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.422+0000 7fe66485b700 1 --2- 172.21.15.144:0/3376740371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff2c0 0x7fe6600ff690 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fe654009910 tx=0x7fe654009c20 comp rx=0 tx=0).stop 2024-09-10T12:05:05.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.422+0000 7fe66485b700 1 -- 172.21.15.144:0/3376740371 shutdown_connections 2024-09-10T12:05:05.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.422+0000 7fe66485b700 1 --2- 172.21.15.144:0/3376740371 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600ffc60 0x7fe660103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:05.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.422+0000 7fe66485b700 1 --2- 172.21.15.144:0/3376740371 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff2c0 0x7fe6600ff690 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:05.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.422+0000 7fe66485b700 1 -- 172.21.15.144:0/3376740371 >> 172.21.15.144:0/3376740371 conn(0x7fe6600fab80 msgr2=0x7fe6600fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:05.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.422+0000 7fe66485b700 1 -- 172.21.15.144:0/3376740371 shutdown_connections 2024-09-10T12:05:05.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.422+0000 7fe66485b700 1 -- 172.21.15.144:0/3376740371 wait complete. 2024-09-10T12:05:05.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.423+0000 7fe66485b700 1 Processor -- start 2024-09-10T12:05:05.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.423+0000 7fe66485b700 1 -- start start 2024-09-10T12:05:05.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.423+0000 7fe66485b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff2c0 0x7fe660194a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:05.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.423+0000 7fe66485b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600ffc60 0x7fe660194fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:05.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.423+0000 7fe66485b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe660195500 con 0x7fe6600ff2c0 2024-09-10T12:05:05.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.423+0000 7fe66485b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe660195670 con 0x7fe6600ffc60 2024-09-10T12:05:05.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.423+0000 7fe65e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff2c0 0x7fe660194a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:05.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.424+0000 7fe65e59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff2c0 0x7fe660194a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51654/0 (socket says 172.21.15.144:51654) 2024-09-10T12:05:05.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.424+0000 7fe65e59c700 1 -- 172.21.15.144:0/1999666279 learned_addr learned my addr 172.21.15.144:0/1999666279 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:05:05.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.424+0000 7fe65dd9b700 1 --2- 172.21.15.144:0/1999666279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600ffc60 0x7fe660194fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:05.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.424+0000 7fe65e59c700 1 -- 172.21.15.144:0/1999666279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600ffc60 msgr2=0x7fe660194fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:05.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.424+0000 7fe65e59c700 1 --2- 172.21.15.144:0/1999666279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600ffc60 0x7fe660194fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:05.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.424+0000 7fe65e59c700 1 -- 172.21.15.144:0/1999666279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6540095f0 con 0x7fe6600ff2c0 2024-09-10T12:05:05.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.424+0000 7fe65e59c700 1 --2- 172.21.15.144:0/1999666279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff2c0 0x7fe660194a80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe654006010 tx=0x7fe65400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:05.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.424+0000 7fe64f7fe700 1 -- 172.21.15.144:0/1999666279 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe65401d070 con 0x7fe6600ff2c0 2024-09-10T12:05:05.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.424+0000 7fe64f7fe700 1 -- 172.21.15.144:0/1999666279 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe654005dc0 con 0x7fe6600ff2c0 2024-09-10T12:05:05.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.424+0000 7fe64f7fe700 1 -- 172.21.15.144:0/1999666279 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe654017400 con 0x7fe6600ff2c0 2024-09-10T12:05:05.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.424+0000 7fe66485b700 1 -- 172.21.15.144:0/1999666279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6601958f0 con 0x7fe6600ff2c0 2024-09-10T12:05:05.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.424+0000 7fe66485b700 1 -- 172.21.15.144:0/1999666279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6601a2370 con 0x7fe6600ff2c0 2024-09-10T12:05:05.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.425+0000 7fe66485b700 1 -- 172.21.15.144:0/1999666279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6601075d0 con 0x7fe6600ff2c0 2024-09-10T12:05:05.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.426+0000 7fe64f7fe700 1 -- 172.21.15.144:0/1999666279 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe654003a60 con 0x7fe6600ff2c0 2024-09-10T12:05:05.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.426+0000 7fe64f7fe700 1 --2- 172.21.15.144:0/1999666279 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe64806e200 0x7fe6480706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:05.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.426+0000 7fe64f7fe700 1 -- 172.21.15.144:0/1999666279 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe65408f370 con 0x7fe6600ff2c0 2024-09-10T12:05:05.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.426+0000 7fe65dd9b700 1 --2- 172.21.15.144:0/1999666279 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe64806e200 0x7fe6480706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:05.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.427+0000 7fe65dd9b700 1 --2- 172.21.15.144:0/1999666279 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe64806e200 0x7fe6480706b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe6600696d0 tx=0x7fe65000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:05.432 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.430+0000 7fe64f7fe700 1 -- 172.21.15.144:0/1999666279 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe65405c280 con 0x7fe6600ff2c0 2024-09-10T12:05:05.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:05 smithi178 ceph-mon[65940]: from='client.35808 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:05.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:05 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2878431613' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:05:05.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:05 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3055350424' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:05:05.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.722+0000 7fe66485b700 1 -- 172.21.15.144:0/1999666279 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe66004ea50 con 0x7fe6600ff2c0 2024-09-10T12:05:05.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.722+0000 7fe64f7fe700 1 -- 172.21.15.144:0/1999666279 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7fe65405be10 con 0x7fe6600ff2c0 2024-09-10T12:05:05.724 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:05:05.724 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:05:05.724 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:05:05.724 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:05:05.724 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:05:05.724 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:05:05.724 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:05:05.724 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:05:05.724 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:05:05.724 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:05:05.725 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:05:05.725 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:05:05.725 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:05:05.725 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:05:05.725 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:05:05.725 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:05:05.725 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:05:05.725 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:05:05.725 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:05:05.725 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:05:05.725 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:05:05.725 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:05:05.725 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:05:05.726 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:05:05.726 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:05:05.726 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.725+0000 7fe66485b700 1 -- 172.21.15.144:0/1999666279 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe64806e200 msgr2=0x7fe6480706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:05.726 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.725+0000 7fe66485b700 1 --2- 172.21.15.144:0/1999666279 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe64806e200 0x7fe6480706b0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fe6600696d0 tx=0x7fe65000b410 comp rx=0 tx=0).stop 2024-09-10T12:05:05.726 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.725+0000 7fe66485b700 1 -- 172.21.15.144:0/1999666279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff2c0 msgr2=0x7fe660194a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:05.726 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.725+0000 7fe66485b700 1 --2- 172.21.15.144:0/1999666279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff2c0 0x7fe660194a80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fe654006010 tx=0x7fe65400bd70 comp rx=0 tx=0).stop 2024-09-10T12:05:05.726 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.725+0000 7fe66485b700 1 -- 172.21.15.144:0/1999666279 shutdown_connections 2024-09-10T12:05:05.726 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.725+0000 7fe66485b700 1 --2- 172.21.15.144:0/1999666279 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe64806e200 0x7fe6480706b0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:05.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.725+0000 7fe66485b700 1 --2- 172.21.15.144:0/1999666279 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe6600ff2c0 0x7fe660194a80 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:05.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.725+0000 7fe66485b700 1 --2- 172.21.15.144:0/1999666279 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe6600ffc60 0x7fe660194fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:05.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.725+0000 7fe66485b700 1 -- 172.21.15.144:0/1999666279 >> 172.21.15.144:0/1999666279 conn(0x7fe6600fab80 msgr2=0x7fe6600fc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:05.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.725+0000 7fe66485b700 1 -- 172.21.15.144:0/1999666279 shutdown_connections 2024-09-10T12:05:05.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:05.725+0000 7fe66485b700 1 -- 172.21.15.144:0/1999666279 wait complete. 2024-09-10T12:05:06.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:06 smithi178 ceph-mon[65940]: pgmap v994: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:06.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:06 smithi178 ceph-mon[65940]: from='client.35820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:06.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:06 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1999666279' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:05:06.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:06 smithi144 ceph-mon[87436]: pgmap v994: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:06.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:06 smithi144 ceph-mon[87436]: from='client.35820 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:06.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:06 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1999666279' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:05:08.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:08 smithi178 ceph-mon[65940]: pgmap v995: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:08.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:08 smithi144 ceph-mon[87436]: pgmap v995: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:10.593 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:10 smithi144 ceph-mon[87436]: pgmap v996: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:10.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:10 smithi178 ceph-mon[65940]: pgmap v996: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:12.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:12 smithi178 ceph-mon[65940]: pgmap v997: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:12.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:12 smithi144 ceph-mon[87436]: pgmap v997: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:14 smithi178 ceph-mon[65940]: pgmap v998: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:14 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:05:14.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:14 smithi144 ceph-mon[87436]: pgmap v998: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:14.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:05:16.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:16 smithi178 ceph-mon[65940]: pgmap v999: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:16.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:16 smithi144 ceph-mon[87436]: pgmap v999: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:18.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:18 smithi178 ceph-mon[65940]: pgmap v1000: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:18 smithi144 ceph-mon[87436]: pgmap v1000: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:20.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:20 smithi178 ceph-mon[65940]: pgmap v1001: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:20.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:20 smithi144 ceph-mon[87436]: pgmap v1001: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:22.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:22 smithi178 ceph-mon[65940]: pgmap v1002: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:22.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:22 smithi144 ceph-mon[87436]: pgmap v1002: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:24.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:24 smithi178 ceph-mon[65940]: pgmap v1003: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:24.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:24 smithi144 ceph-mon[87436]: pgmap v1003: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:26.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:26 smithi178 ceph-mon[65940]: pgmap v1004: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:26 smithi144 ceph-mon[87436]: pgmap v1004: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:28.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:28 smithi178 ceph-mon[65940]: pgmap v1005: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:28.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:28 smithi144 ceph-mon[87436]: pgmap v1005: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:29.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:29 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:05:29.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:29 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:05:30.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:30 smithi178 ceph-mon[65940]: pgmap v1006: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:30.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:30 smithi144 ceph-mon[87436]: pgmap v1006: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:32.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:32 smithi178 ceph-mon[65940]: pgmap v1007: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:32.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:32 smithi144 ceph-mon[87436]: pgmap v1007: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:34.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:34 smithi178 ceph-mon[65940]: pgmap v1008: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:34.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:34 smithi144 ceph-mon[87436]: pgmap v1008: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:35.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.843+0000 7fd055e86700 1 -- 172.21.15.144:0/1450593128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd050101cd0 msgr2=0x7fd05010a1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:35.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.843+0000 7fd055e86700 1 --2- 172.21.15.144:0/1450593128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd050101cd0 0x7fd05010a1d0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7fd044009910 tx=0x7fd044009c20 comp rx=0 tx=0).stop 2024-09-10T12:05:35.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.843+0000 7fd055e86700 1 -- 172.21.15.144:0/1450593128 shutdown_connections 2024-09-10T12:05:35.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.843+0000 7fd055e86700 1 --2- 172.21.15.144:0/1450593128 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd050101cd0 0x7fd05010a1d0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:35.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.843+0000 7fd055e86700 1 --2- 172.21.15.144:0/1450593128 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0501013c0 0x7fd050101790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:35.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.843+0000 7fd055e86700 1 -- 172.21.15.144:0/1450593128 >> 172.21.15.144:0/1450593128 conn(0x7fd0500fab80 msgr2=0x7fd0500fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:35.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.843+0000 7fd055e86700 1 -- 172.21.15.144:0/1450593128 shutdown_connections 2024-09-10T12:05:35.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.843+0000 7fd055e86700 1 -- 172.21.15.144:0/1450593128 wait complete. 2024-09-10T12:05:35.847 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.844+0000 7fd055e86700 1 Processor -- start 2024-09-10T12:05:35.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.844+0000 7fd055e86700 1 -- start start 2024-09-10T12:05:35.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.844+0000 7fd055e86700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0501013c0 0x7fd050195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:35.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.844+0000 7fd055e86700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd050101cd0 0x7fd050196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:35.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.844+0000 7fd055e86700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd050196810 con 0x7fd050101cd0 2024-09-10T12:05:35.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.844+0000 7fd055e86700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd050196980 con 0x7fd0501013c0 2024-09-10T12:05:35.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.844+0000 7fd04effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd050101cd0 0x7fd050196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:35.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.844+0000 7fd04effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd050101cd0 0x7fd050196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35812/0 (socket says 172.21.15.144:35812) 2024-09-10T12:05:35.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.844+0000 7fd04effd700 1 -- 172.21.15.144:0/1217846189 learned_addr learned my addr 172.21.15.144:0/1217846189 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:05:35.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.844+0000 7fd04f7fe700 1 --2- 172.21.15.144:0/1217846189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0501013c0 0x7fd050195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:35.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.845+0000 7fd04effd700 1 -- 172.21.15.144:0/1217846189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0501013c0 msgr2=0x7fd050195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:35.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.845+0000 7fd04effd700 1 --2- 172.21.15.144:0/1217846189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0501013c0 0x7fd050195cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:35.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.845+0000 7fd04effd700 1 -- 172.21.15.144:0/1217846189 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0440095f0 con 0x7fd050101cd0 2024-09-10T12:05:35.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.845+0000 7fd04f7fe700 1 --2- 172.21.15.144:0/1217846189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0501013c0 0x7fd050195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:05:35.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.845+0000 7fd04effd700 1 --2- 172.21.15.144:0/1217846189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd050101cd0 0x7fd050196210 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd044006010 tx=0x7fd04400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:35.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.845+0000 7fd04cff9700 1 -- 172.21.15.144:0/1217846189 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd04401d070 con 0x7fd050101cd0 2024-09-10T12:05:35.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.845+0000 7fd055e86700 1 -- 172.21.15.144:0/1217846189 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0501996c0 con 0x7fd050101cd0 2024-09-10T12:05:35.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.845+0000 7fd04cff9700 1 -- 172.21.15.144:0/1217846189 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd044004fb0 con 0x7fd050101cd0 2024-09-10T12:05:35.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.845+0000 7fd04cff9700 1 -- 172.21.15.144:0/1217846189 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd044022ea0 con 0x7fd050101cd0 2024-09-10T12:05:35.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.845+0000 7fd055e86700 1 -- 172.21.15.144:0/1217846189 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd050199bb0 con 0x7fd050101cd0 2024-09-10T12:05:35.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.846+0000 7fd055e86700 1 -- 172.21.15.144:0/1217846189 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd05004ea50 con 0x7fd050101cd0 2024-09-10T12:05:35.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.847+0000 7fd04cff9700 1 -- 172.21.15.144:0/1217846189 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd044022470 con 0x7fd050101cd0 2024-09-10T12:05:35.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.847+0000 7fd04cff9700 1 --2- 172.21.15.144:0/1217846189 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd03806e200 0x7fd0380706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:35.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.847+0000 7fd04cff9700 1 -- 172.21.15.144:0/1217846189 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd04408f610 con 0x7fd050101cd0 2024-09-10T12:05:35.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.847+0000 7fd04f7fe700 1 --2- 172.21.15.144:0/1217846189 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd03806e200 0x7fd0380706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:35.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.848+0000 7fd04f7fe700 1 --2- 172.21.15.144:0/1217846189 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd03806e200 0x7fd0380706b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd04000e300 tx=0x7fd040009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:35.855 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:35.854+0000 7fd04cff9700 1 -- 172.21.15.144:0/1217846189 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd044058aa0 con 0x7fd050101cd0 2024-09-10T12:05:36.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.100+0000 7fd055e86700 1 -- 172.21.15.144:0/1217846189 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd0501073e0 con 0x7fd03806e200 2024-09-10T12:05:36.103 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.102+0000 7fd04cff9700 1 -- 172.21.15.144:0/1217846189 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fd0501073e0 con 0x7fd03806e200 2024-09-10T12:05:36.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.104+0000 7fd055e86700 1 -- 172.21.15.144:0/1217846189 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd03806e200 msgr2=0x7fd0380706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:36.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.104+0000 7fd055e86700 1 --2- 172.21.15.144:0/1217846189 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd03806e200 0x7fd0380706b0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd04000e300 tx=0x7fd040009040 comp rx=0 tx=0).stop 2024-09-10T12:05:36.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.104+0000 7fd055e86700 1 -- 172.21.15.144:0/1217846189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd050101cd0 msgr2=0x7fd050196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:36.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.104+0000 7fd055e86700 1 --2- 172.21.15.144:0/1217846189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd050101cd0 0x7fd050196210 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fd044006010 tx=0x7fd04400be20 comp rx=0 tx=0).stop 2024-09-10T12:05:36.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.104+0000 7fd055e86700 1 -- 172.21.15.144:0/1217846189 shutdown_connections 2024-09-10T12:05:36.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.104+0000 7fd055e86700 1 --2- 172.21.15.144:0/1217846189 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd03806e200 0x7fd0380706b0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.104+0000 7fd055e86700 1 --2- 172.21.15.144:0/1217846189 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0501013c0 0x7fd050195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.104+0000 7fd055e86700 1 --2- 172.21.15.144:0/1217846189 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd050101cd0 0x7fd050196210 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.104+0000 7fd055e86700 1 -- 172.21.15.144:0/1217846189 >> 172.21.15.144:0/1217846189 conn(0x7fd0500fab80 msgr2=0x7fd050104980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:36.106 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.105+0000 7fd055e86700 1 -- 172.21.15.144:0/1217846189 shutdown_connections 2024-09-10T12:05:36.107 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.105+0000 7fd055e86700 1 -- 172.21.15.144:0/1217846189 wait complete. 2024-09-10T12:05:36.116 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:05:36.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.210+0000 7f36b27f8700 1 -- 172.21.15.144:0/166425910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ac100d80 msgr2=0x7f36ac104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:36.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.210+0000 7f36b27f8700 1 --2- 172.21.15.144:0/166425910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ac100d80 0x7f36ac104d50 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f36a0009910 tx=0x7f36a0009c20 comp rx=0 tx=0).stop 2024-09-10T12:05:36.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.210+0000 7f36b27f8700 1 -- 172.21.15.144:0/166425910 shutdown_connections 2024-09-10T12:05:36.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.210+0000 7f36b27f8700 1 --2- 172.21.15.144:0/166425910 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ac100d80 0x7f36ac104d50 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.210+0000 7f36b27f8700 1 --2- 172.21.15.144:0/166425910 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ac1003e0 0x7f36ac1007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.210+0000 7f36b27f8700 1 -- 172.21.15.144:0/166425910 >> 172.21.15.144:0/166425910 conn(0x7f36ac0fbca0 msgr2=0x7f36ac0fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:36.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.210+0000 7f36b27f8700 1 -- 172.21.15.144:0/166425910 shutdown_connections 2024-09-10T12:05:36.212 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.210+0000 7f36b27f8700 1 -- 172.21.15.144:0/166425910 wait complete. 2024-09-10T12:05:36.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.211+0000 7f36b27f8700 1 Processor -- start 2024-09-10T12:05:36.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.211+0000 7f36b27f8700 1 -- start start 2024-09-10T12:05:36.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36b27f8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ac1003e0 0x7f36ac195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:36.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36b27f8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ac100d80 0x7f36ac196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:36.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36b27f8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac1969f0 con 0x7f36ac100d80 2024-09-10T12:05:36.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36b27f8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac19a780 con 0x7f36ac1003e0 2024-09-10T12:05:36.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36ab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ac100d80 0x7f36ac196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:36.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36ab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ac100d80 0x7f36ac196310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35832/0 (socket says 172.21.15.144:35832) 2024-09-10T12:05:36.214 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36ab7fe700 1 -- 172.21.15.144:0/3417819617 learned_addr learned my addr 172.21.15.144:0/3417819617 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:05:36.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36abfff700 1 --2- 172.21.15.144:0/3417819617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ac1003e0 0x7f36ac195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:36.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36ab7fe700 1 -- 172.21.15.144:0/3417819617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ac1003e0 msgr2=0x7f36ac195dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:36.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36ab7fe700 1 --2- 172.21.15.144:0/3417819617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ac1003e0 0x7f36ac195dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36ab7fe700 1 -- 172.21.15.144:0/3417819617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a00095f0 con 0x7f36ac100d80 2024-09-10T12:05:36.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36ab7fe700 1 --2- 172.21.15.144:0/3417819617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ac100d80 0x7f36ac196310 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f36a00092e0 tx=0x7f36a000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:36.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36a97fa700 1 -- 172.21.15.144:0/3417819617 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a001d070 con 0x7f36ac100d80 2024-09-10T12:05:36.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36a97fa700 1 -- 172.21.15.144:0/3417819617 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36a0005dc0 con 0x7f36ac100d80 2024-09-10T12:05:36.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36b27f8700 1 -- 172.21.15.144:0/3417819617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ac19aa00 con 0x7f36ac100d80 2024-09-10T12:05:36.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36a97fa700 1 -- 172.21.15.144:0/3417819617 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a0017400 con 0x7f36ac100d80 2024-09-10T12:05:36.215 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.212+0000 7f36b27f8700 1 -- 172.21.15.144:0/3417819617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ac19aef0 con 0x7f36ac100d80 2024-09-10T12:05:36.216 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.213+0000 7f36b27f8700 1 -- 172.21.15.144:0/3417819617 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ac04ea50 con 0x7f36ac100d80 2024-09-10T12:05:36.216 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.214+0000 7f36a97fa700 1 -- 172.21.15.144:0/3417819617 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f36a000f6f0 con 0x7f36ac100d80 2024-09-10T12:05:36.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.214+0000 7f36a97fa700 1 --2- 172.21.15.144:0/3417819617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f369406e200 0x7f36940706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:36.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.214+0000 7f36a97fa700 1 -- 172.21.15.144:0/3417819617 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f36a008f470 con 0x7f36ac100d80 2024-09-10T12:05:36.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.214+0000 7f36abfff700 1 --2- 172.21.15.144:0/3417819617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f369406e200 0x7f36940706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:36.217 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.215+0000 7f36abfff700 1 --2- 172.21.15.144:0/3417819617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f369406e200 0x7f36940706b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f369c005950 tx=0x7f369c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:36.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.219+0000 7f36a97fa700 1 -- 172.21.15.144:0/3417819617 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f36a0058850 con 0x7f36ac100d80 2024-09-10T12:05:36.433 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:36 smithi144 ceph-mon[87436]: pgmap v1009: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:36.443 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.442+0000 7f36b27f8700 1 -- 172.21.15.144:0/3417819617 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f36ac19b3d0 con 0x7f369406e200 2024-09-10T12:05:36.445 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.443+0000 7f36a97fa700 1 -- 172.21.15.144:0/3417819617 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f36ac19b3d0 con 0x7f369406e200 2024-09-10T12:05:36.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.446+0000 7f36b27f8700 1 -- 172.21.15.144:0/3417819617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f369406e200 msgr2=0x7f36940706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:36.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.446+0000 7f36b27f8700 1 --2- 172.21.15.144:0/3417819617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f369406e200 0x7f36940706b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f369c005950 tx=0x7f369c00b410 comp rx=0 tx=0).stop 2024-09-10T12:05:36.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.446+0000 7f36b27f8700 1 -- 172.21.15.144:0/3417819617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ac100d80 msgr2=0x7f36ac196310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:36.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.446+0000 7f36b27f8700 1 --2- 172.21.15.144:0/3417819617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ac100d80 0x7f36ac196310 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f36a00092e0 tx=0x7f36a000bd70 comp rx=0 tx=0).stop 2024-09-10T12:05:36.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.446+0000 7f36b27f8700 1 -- 172.21.15.144:0/3417819617 shutdown_connections 2024-09-10T12:05:36.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.446+0000 7f36b27f8700 1 --2- 172.21.15.144:0/3417819617 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f369406e200 0x7f36940706b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.446+0000 7f36b27f8700 1 --2- 172.21.15.144:0/3417819617 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f36ac1003e0 0x7f36ac195dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.446+0000 7f36b27f8700 1 --2- 172.21.15.144:0/3417819617 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f36ac100d80 0x7f36ac196310 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.446+0000 7f36b27f8700 1 -- 172.21.15.144:0/3417819617 >> 172.21.15.144:0/3417819617 conn(0x7f36ac0fbca0 msgr2=0x7f36ac0fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:36.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.446+0000 7f36b27f8700 1 -- 172.21.15.144:0/3417819617 shutdown_connections 2024-09-10T12:05:36.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.446+0000 7f36b27f8700 1 -- 172.21.15.144:0/3417819617 wait complete. 2024-09-10T12:05:36.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.551+0000 7f4ed74b4700 1 -- 172.21.15.144:0/2091111180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed00ff290 msgr2=0x7f4ed00ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:36.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.551+0000 7f4ed74b4700 1 --2- 172.21.15.144:0/2091111180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed00ff290 0x7f4ed00ff660 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc009910 tx=0x7f4ecc009c20 comp rx=0 tx=0).stop 2024-09-10T12:05:36.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.551+0000 7f4ed74b4700 1 -- 172.21.15.144:0/2091111180 shutdown_connections 2024-09-10T12:05:36.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.551+0000 7f4ed74b4700 1 --2- 172.21.15.144:0/2091111180 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed00ffba0 0x7f4ed010a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.552 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.551+0000 7f4ed74b4700 1 --2- 172.21.15.144:0/2091111180 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed00ff290 0x7f4ed00ff660 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.551+0000 7f4ed74b4700 1 -- 172.21.15.144:0/2091111180 >> 172.21.15.144:0/2091111180 conn(0x7f4ed0074850 msgr2=0x7f4ed0074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:36.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.551+0000 7f4ed74b4700 1 -- 172.21.15.144:0/2091111180 shutdown_connections 2024-09-10T12:05:36.553 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.551+0000 7f4ed74b4700 1 -- 172.21.15.144:0/2091111180 wait complete. 2024-09-10T12:05:36.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.551+0000 7f4ed74b4700 1 Processor -- start 2024-09-10T12:05:36.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.551+0000 7f4ed74b4700 1 -- start start 2024-09-10T12:05:36.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ed74b4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed00ff290 0x7f4ed0195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:36.554 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ed74b4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed00ffba0 0x7f4ed01962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:36.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ed74b4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed0196990 con 0x7f4ed00ffba0 2024-09-10T12:05:36.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ed74b4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed019a720 con 0x7f4ed00ff290 2024-09-10T12:05:36.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ed4a4f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed00ffba0 0x7f4ed01962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:36.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ed4a4f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed00ffba0 0x7f4ed01962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35850/0 (socket says 172.21.15.144:35850) 2024-09-10T12:05:36.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ed4a4f700 1 -- 172.21.15.144:0/807248103 learned_addr learned my addr 172.21.15.144:0/807248103 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:05:36.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ed5250700 1 --2- 172.21.15.144:0/807248103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed00ff290 0x7f4ed0195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:36.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ed4a4f700 1 -- 172.21.15.144:0/807248103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed00ff290 msgr2=0x7f4ed0195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:36.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ed4a4f700 1 --2- 172.21.15.144:0/807248103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed00ff290 0x7f4ed0195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ed4a4f700 1 -- 172.21.15.144:0/807248103 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ecc0095f0 con 0x7f4ed00ffba0 2024-09-10T12:05:36.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ed5250700 1 --2- 172.21.15.144:0/807248103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed00ff290 0x7f4ed0195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:05:36.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ed4a4f700 1 --2- 172.21.15.144:0/807248103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed00ffba0 0x7f4ed01962b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000d8d0 tx=0x7f4ec000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:36.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ec67fc700 1 -- 172.21.15.144:0/807248103 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ec0009940 con 0x7f4ed00ffba0 2024-09-10T12:05:36.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ec67fc700 1 -- 172.21.15.144:0/807248103 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4ec0010460 con 0x7f4ed00ffba0 2024-09-10T12:05:36.555 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ed74b4700 1 -- 172.21.15.144:0/807248103 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed019aa00 con 0x7f4ed00ffba0 2024-09-10T12:05:36.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ec67fc700 1 -- 172.21.15.144:0/807248103 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4ec000f5f0 con 0x7f4ed00ffba0 2024-09-10T12:05:36.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.552+0000 7f4ed74b4700 1 -- 172.21.15.144:0/807248103 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed019af50 con 0x7f4ed00ffba0 2024-09-10T12:05:36.556 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.554+0000 7f4ed74b4700 1 -- 172.21.15.144:0/807248103 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed0107d10 con 0x7f4ed00ffba0 2024-09-10T12:05:36.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.554+0000 7f4ec67fc700 1 -- 172.21.15.144:0/807248103 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4ec00105d0 con 0x7f4ed00ffba0 2024-09-10T12:05:36.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.554+0000 7f4ec67fc700 1 --2- 172.21.15.144:0/807248103 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ebc06e200 0x7f4ebc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:36.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.554+0000 7f4ec67fc700 1 -- 172.21.15.144:0/807248103 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4ec008e0a0 con 0x7f4ed00ffba0 2024-09-10T12:05:36.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.555+0000 7f4ed5250700 1 --2- 172.21.15.144:0/807248103 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ebc06e200 0x7f4ebc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:36.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.555+0000 7f4ed5250700 1 --2- 172.21.15.144:0/807248103 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ebc06e200 0x7f4ebc0706b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc005b40 tx=0x7f4ecc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:36.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.559+0000 7f4ec67fc700 1 -- 172.21.15.144:0/807248103 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4ec0057480 con 0x7f4ed00ffba0 2024-09-10T12:05:36.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:36 smithi178 ceph-mon[65940]: pgmap v1009: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:36.771 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.769+0000 7f4ed74b4700 1 -- 172.21.15.144:0/807248103 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ed019b230 con 0x7f4ebc06e200 2024-09-10T12:05:36.781 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.780+0000 7f4ec67fc700 1 -- 172.21.15.144:0/807248103 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f4ed019b230 con 0x7f4ebc06e200 2024-09-10T12:05:36.781 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:05:36.781 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (33m) 12m ago 44m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:05:36.781 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (44m) 12m ago 44m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:05:36.781 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (43m) 12m ago 43m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:05:36.782 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (31m) 12m ago 44m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:05:36.782 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (31m) 12m ago 43m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:05:36.782 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (33m) 12m ago 44m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:05:36.782 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (27m) 12m ago 38m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:05:36.782 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (27m) 12m ago 38m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:05:36.782 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (12m) 12m ago 38m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:05:36.782 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (27m) 12m ago 38m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:05:36.782 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (35m) 12m ago 45m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:05:36.782 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (34m) 12m ago 43m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:05:36.782 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (32m) 12m ago 46m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:05:36.782 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (32m) 12m ago 43m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:05:36.782 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (34m) 12m ago 44m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:05:36.782 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (34m) 12m ago 43m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:05:36.782 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (31m) 12m ago 42m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:05:36.783 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (30m) 12m ago 42m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:05:36.783 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (29m) 12m ago 41m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:05:36.783 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (29m) 12m ago 41m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:05:36.783 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (28m) 12m ago 40m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:05:36.783 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (28m) 12m ago 40m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:05:36.783 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (34m) 12m ago 43m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:05:36.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.782+0000 7f4ed74b4700 1 -- 172.21.15.144:0/807248103 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ebc06e200 msgr2=0x7f4ebc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:36.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.782+0000 7f4ed74b4700 1 --2- 172.21.15.144:0/807248103 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ebc06e200 0x7f4ebc0706b0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc005b40 tx=0x7f4ecc01a040 comp rx=0 tx=0).stop 2024-09-10T12:05:36.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.782+0000 7f4ed74b4700 1 -- 172.21.15.144:0/807248103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed00ffba0 msgr2=0x7f4ed01962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:36.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.782+0000 7f4ed74b4700 1 --2- 172.21.15.144:0/807248103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed00ffba0 0x7f4ed01962b0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f4ec000d8d0 tx=0x7f4ec000dc90 comp rx=0 tx=0).stop 2024-09-10T12:05:36.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.782+0000 7f4ed74b4700 1 -- 172.21.15.144:0/807248103 shutdown_connections 2024-09-10T12:05:36.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.782+0000 7f4ed74b4700 1 --2- 172.21.15.144:0/807248103 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4ebc06e200 0x7f4ebc0706b0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.782+0000 7f4ed74b4700 1 --2- 172.21.15.144:0/807248103 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ed00ff290 0x7f4ed0195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.782+0000 7f4ed74b4700 1 --2- 172.21.15.144:0/807248103 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ed00ffba0 0x7f4ed01962b0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.782+0000 7f4ed74b4700 1 -- 172.21.15.144:0/807248103 >> 172.21.15.144:0/807248103 conn(0x7f4ed0074850 msgr2=0x7f4ed0104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:36.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.782+0000 7f4ed74b4700 1 -- 172.21.15.144:0/807248103 shutdown_connections 2024-09-10T12:05:36.784 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.782+0000 7f4ed74b4700 1 -- 172.21.15.144:0/807248103 wait complete. 2024-09-10T12:05:36.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.886+0000 7fac58e71700 1 -- 172.21.15.144:0/1556240893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac54101320 msgr2=0x7fac541016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:36.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.886+0000 7fac58e71700 1 --2- 172.21.15.144:0/1556240893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac54101320 0x7fac541016f0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fac4c00b0a0 tx=0x7fac4c00b3b0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.886+0000 7fac58e71700 1 -- 172.21.15.144:0/1556240893 shutdown_connections 2024-09-10T12:05:36.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.886+0000 7fac58e71700 1 --2- 172.21.15.144:0/1556240893 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac54101c30 0x7fac5410a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.886+0000 7fac58e71700 1 --2- 172.21.15.144:0/1556240893 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac54101320 0x7fac541016f0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.886+0000 7fac58e71700 1 -- 172.21.15.144:0/1556240893 >> 172.21.15.144:0/1556240893 conn(0x7fac540fab80 msgr2=0x7fac540fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:36.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.886+0000 7fac58e71700 1 -- 172.21.15.144:0/1556240893 shutdown_connections 2024-09-10T12:05:36.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.886+0000 7fac58e71700 1 -- 172.21.15.144:0/1556240893 wait complete. 2024-09-10T12:05:36.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.887+0000 7fac58e71700 1 Processor -- start 2024-09-10T12:05:36.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.887+0000 7fac58e71700 1 -- start start 2024-09-10T12:05:36.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.887+0000 7fac58e71700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac54101320 0x7fac540fef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:36.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.887+0000 7fac58e71700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac54101c30 0x7fac540ff4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:36.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.887+0000 7fac58e71700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac540ffa00 con 0x7fac54101c30 2024-09-10T12:05:36.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.887+0000 7fac58e71700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac540ffb40 con 0x7fac54101320 2024-09-10T12:05:36.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.887+0000 7fac51d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac54101c30 0x7fac540ff4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:36.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.887+0000 7fac51d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac54101c30 0x7fac540ff4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35866/0 (socket says 172.21.15.144:35866) 2024-09-10T12:05:36.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.887+0000 7fac51d9b700 1 -- 172.21.15.144:0/3843730995 learned_addr learned my addr 172.21.15.144:0/3843730995 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:05:36.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.887+0000 7fac5259c700 1 --2- 172.21.15.144:0/3843730995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac54101320 0x7fac540fef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:36.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.887+0000 7fac51d9b700 1 -- 172.21.15.144:0/3843730995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac54101320 msgr2=0x7fac540fef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:36.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.887+0000 7fac51d9b700 1 --2- 172.21.15.144:0/3843730995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac54101320 0x7fac540fef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:36.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.887+0000 7fac51d9b700 1 -- 172.21.15.144:0/3843730995 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac4c009d00 con 0x7fac54101c30 2024-09-10T12:05:36.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.887+0000 7fac51d9b700 1 --2- 172.21.15.144:0/3843730995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac54101c30 0x7fac540ff4c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00c930 tx=0x7fac3c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:36.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.888+0000 7fac4b7fe700 1 -- 172.21.15.144:0/3843730995 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac3c007ad0 con 0x7fac54101c30 2024-09-10T12:05:36.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.888+0000 7fac4b7fe700 1 -- 172.21.15.144:0/3843730995 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fac3c007c30 con 0x7fac54101c30 2024-09-10T12:05:36.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.888+0000 7fac58e71700 1 -- 172.21.15.144:0/3843730995 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac541a1f20 con 0x7fac54101c30 2024-09-10T12:05:36.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.888+0000 7fac4b7fe700 1 -- 172.21.15.144:0/3843730995 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac3c018610 con 0x7fac54101c30 2024-09-10T12:05:36.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.888+0000 7fac58e71700 1 -- 172.21.15.144:0/3843730995 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac541a2280 con 0x7fac54101c30 2024-09-10T12:05:36.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.889+0000 7fac58e71700 1 -- 172.21.15.144:0/3843730995 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac54107800 con 0x7fac54101c30 2024-09-10T12:05:36.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.889+0000 7fac4b7fe700 1 -- 172.21.15.144:0/3843730995 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fac3c00f450 con 0x7fac54101c30 2024-09-10T12:05:36.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.890+0000 7fac4b7fe700 1 --2- 172.21.15.144:0/3843730995 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fac4006e200 0x7fac400706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:36.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.890+0000 7fac5259c700 1 --2- 172.21.15.144:0/3843730995 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fac4006e200 0x7fac400706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:36.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.890+0000 7fac4b7fe700 1 -- 172.21.15.144:0/3843730995 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fac3c08e1a0 con 0x7fac54101c30 2024-09-10T12:05:36.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.890+0000 7fac5259c700 1 --2- 172.21.15.144:0/3843730995 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fac4006e200 0x7fac400706b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fac4c015040 tx=0x7fac4c006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:36.895 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:36.894+0000 7fac4b7fe700 1 -- 172.21.15.144:0/3843730995 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fac3c057600 con 0x7fac54101c30 2024-09-10T12:05:37.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.177+0000 7fac58e71700 1 -- 172.21.15.144:0/3843730995 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fac541071e0 con 0x7fac54101c30 2024-09-10T12:05:37.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.178+0000 7fac4b7fe700 1 -- 172.21.15.144:0/3843730995 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fac3c05acc0 con 0x7fac54101c30 2024-09-10T12:05:37.179 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:05:37.179 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:05:37.179 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:05:37.179 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:05:37.179 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:05:37.179 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:05:37.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:05:37.180 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:05:37.180 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:05:37.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:05:37.180 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:05:37.180 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:05:37.180 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:05:37.180 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:05:37.180 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:05:37.180 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:05:37.180 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:05:37.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.180+0000 7fac58e71700 1 -- 172.21.15.144:0/3843730995 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fac4006e200 msgr2=0x7fac400706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:37.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.180+0000 7fac58e71700 1 --2- 172.21.15.144:0/3843730995 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fac4006e200 0x7fac400706b0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fac4c015040 tx=0x7fac4c006000 comp rx=0 tx=0).stop 2024-09-10T12:05:37.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.180+0000 7fac58e71700 1 -- 172.21.15.144:0/3843730995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac54101c30 msgr2=0x7fac540ff4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:37.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.180+0000 7fac58e71700 1 --2- 172.21.15.144:0/3843730995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac54101c30 0x7fac540ff4c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fac3c00c930 tx=0x7fac3c00ccf0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.180+0000 7fac58e71700 1 -- 172.21.15.144:0/3843730995 shutdown_connections 2024-09-10T12:05:37.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.180+0000 7fac58e71700 1 --2- 172.21.15.144:0/3843730995 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fac4006e200 0x7fac400706b0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.180+0000 7fac58e71700 1 --2- 172.21.15.144:0/3843730995 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fac54101320 0x7fac540fef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.180+0000 7fac58e71700 1 --2- 172.21.15.144:0/3843730995 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fac54101c30 0x7fac540ff4c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.180+0000 7fac58e71700 1 -- 172.21.15.144:0/3843730995 >> 172.21.15.144:0/3843730995 conn(0x7fac540fab80 msgr2=0x7fac54104940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.180+0000 7fac58e71700 1 -- 172.21.15.144:0/3843730995 shutdown_connections 2024-09-10T12:05:37.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.180+0000 7fac58e71700 1 -- 172.21.15.144:0/3843730995 wait complete. 2024-09-10T12:05:37.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.284+0000 7fa31bf52700 1 -- 172.21.15.144:0/2511147126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa314101320 msgr2=0x7fa3141016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:37.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.284+0000 7fa31bf52700 1 --2- 172.21.15.144:0/2511147126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa314101320 0x7fa3141016f0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fa310009940 tx=0x7fa310009c50 comp rx=0 tx=0).stop 2024-09-10T12:05:37.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.284+0000 7fa31bf52700 1 -- 172.21.15.144:0/2511147126 shutdown_connections 2024-09-10T12:05:37.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.284+0000 7fa31bf52700 1 --2- 172.21.15.144:0/2511147126 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa314101c30 0x7fa31410a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.284+0000 7fa31bf52700 1 --2- 172.21.15.144:0/2511147126 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa314101320 0x7fa3141016f0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.284+0000 7fa31bf52700 1 -- 172.21.15.144:0/2511147126 >> 172.21.15.144:0/2511147126 conn(0x7fa3140fab80 msgr2=0x7fa3140fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:37.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.284+0000 7fa31bf52700 1 -- 172.21.15.144:0/2511147126 shutdown_connections 2024-09-10T12:05:37.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.284+0000 7fa31bf52700 1 -- 172.21.15.144:0/2511147126 wait complete. 2024-09-10T12:05:37.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.285+0000 7fa31bf52700 1 Processor -- start 2024-09-10T12:05:37.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.285+0000 7fa31bf52700 1 -- start start 2024-09-10T12:05:37.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.285+0000 7fa31bf52700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa314101320 0x7fa314195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:37.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.285+0000 7fa31bf52700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa314101c30 0x7fa314196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:37.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.285+0000 7fa31bf52700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa314196a30 con 0x7fa314101320 2024-09-10T12:05:37.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.285+0000 7fa31bf52700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31419a7c0 con 0x7fa314101c30 2024-09-10T12:05:37.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.285+0000 7fa319cee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa314101320 0x7fa314195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:37.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.285+0000 7fa319cee700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa314101320 0x7fa314195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35884/0 (socket says 172.21.15.144:35884) 2024-09-10T12:05:37.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.285+0000 7fa319cee700 1 -- 172.21.15.144:0/1999574529 learned_addr learned my addr 172.21.15.144:0/1999574529 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:05:37.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.286+0000 7fa319cee700 1 -- 172.21.15.144:0/1999574529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa314101c30 msgr2=0x7fa314196350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:37.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.286+0000 7fa3194ed700 1 --2- 172.21.15.144:0/1999574529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa314101c30 0x7fa314196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:37.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.286+0000 7fa319cee700 1 --2- 172.21.15.144:0/1999574529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa314101c30 0x7fa314196350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.286+0000 7fa319cee700 1 -- 172.21.15.144:0/1999574529 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3100095f0 con 0x7fa314101320 2024-09-10T12:05:37.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.286+0000 7fa3194ed700 1 --2- 172.21.15.144:0/1999574529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa314101c30 0x7fa314196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:05:37.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.286+0000 7fa319cee700 1 --2- 172.21.15.144:0/1999574529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa314101320 0x7fa314195e10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa310005620 tx=0x7fa310005700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:37.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.286+0000 7fa30affd700 1 -- 172.21.15.144:0/1999574529 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa31001d070 con 0x7fa314101320 2024-09-10T12:05:37.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.286+0000 7fa30affd700 1 -- 172.21.15.144:0/1999574529 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa31000f460 con 0x7fa314101320 2024-09-10T12:05:37.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.286+0000 7fa30affd700 1 -- 172.21.15.144:0/1999574529 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa310021640 con 0x7fa314101320 2024-09-10T12:05:37.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.286+0000 7fa31bf52700 1 -- 172.21.15.144:0/1999574529 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31419aa40 con 0x7fa314101320 2024-09-10T12:05:37.290 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.286+0000 7fa31bf52700 1 -- 172.21.15.144:0/1999574529 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31419af30 con 0x7fa314101320 2024-09-10T12:05:37.290 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:37 smithi144 ceph-mon[87436]: from='client.35828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:37.290 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:37 smithi144 ceph-mon[87436]: from='client.35832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:37.290 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:37 smithi144 ceph-mon[87436]: from='client.35836 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:37.290 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:37 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3843730995' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:05:37.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.287+0000 7fa31bf52700 1 -- 172.21.15.144:0/1999574529 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3141077e0 con 0x7fa314101320 2024-09-10T12:05:37.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.288+0000 7fa30affd700 1 -- 172.21.15.144:0/1999574529 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa31000fa40 con 0x7fa314101320 2024-09-10T12:05:37.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.289+0000 7fa30affd700 1 --2- 172.21.15.144:0/1999574529 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa30006e200 0x7fa3000706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:37.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.289+0000 7fa30affd700 1 -- 172.21.15.144:0/1999574529 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa31008f710 con 0x7fa314101320 2024-09-10T12:05:37.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.289+0000 7fa3194ed700 1 --2- 172.21.15.144:0/1999574529 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa30006e200 0x7fa3000706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:37.291 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.289+0000 7fa3194ed700 1 --2- 172.21.15.144:0/1999574529 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa30006e200 0x7fa3000706b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa314197430 tx=0x7fa304008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:37.293 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.292+0000 7fa30affd700 1 -- 172.21.15.144:0/1999574529 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa310058b90 con 0x7fa314101320 2024-09-10T12:05:37.538 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.537+0000 7fa31bf52700 1 -- 172.21.15.144:0/1999574529 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa31404ea50 con 0x7fa314101320 2024-09-10T12:05:37.539 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.538+0000 7fa30affd700 1 -- 172.21.15.144:0/1999574529 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7fa31005c250 con 0x7fa314101320 2024-09-10T12:05:37.539 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:05:37.539 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:05:37.539 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:05:37.539 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:05:37.539 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:05:37.539 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:05:37.540 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:05:37.540 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:05:37.540 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:05:37.540 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:05:37.540 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:05:37.540 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:05:37.540 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:05:37.540 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:05:37.540 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:05:37.540 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:05:37.540 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:05:37.540 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:05:37.540 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:05:37.540 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:05:37.541 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:05:37.541 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:05:37.541 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:05:37.541 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:05:37.541 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:05:37.541 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:05:37.541 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:05:37.541 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:05:37.541 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:05:37.541 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:05:37.541 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:05:37.541 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:05:37.541 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:05:37.542 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:05:37.542 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:05:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:05:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:05:37.542 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:05:37.542 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:05:37.542 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:05:37.542 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:05:37.542 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:05:37.542 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:05:37.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.540+0000 7fa31bf52700 1 -- 172.21.15.144:0/1999574529 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa30006e200 msgr2=0x7fa3000706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:37.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.540+0000 7fa31bf52700 1 --2- 172.21.15.144:0/1999574529 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa30006e200 0x7fa3000706b0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa314197430 tx=0x7fa304008040 comp rx=0 tx=0).stop 2024-09-10T12:05:37.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.540+0000 7fa31bf52700 1 -- 172.21.15.144:0/1999574529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa314101320 msgr2=0x7fa314195e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:37.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.540+0000 7fa31bf52700 1 --2- 172.21.15.144:0/1999574529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa314101320 0x7fa314195e10 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fa310005620 tx=0x7fa310005700 comp rx=0 tx=0).stop 2024-09-10T12:05:37.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.540+0000 7fa31bf52700 1 -- 172.21.15.144:0/1999574529 shutdown_connections 2024-09-10T12:05:37.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.540+0000 7fa31bf52700 1 --2- 172.21.15.144:0/1999574529 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa30006e200 0x7fa3000706b0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.540+0000 7fa31bf52700 1 --2- 172.21.15.144:0/1999574529 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa314101320 0x7fa314195e10 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.540+0000 7fa31bf52700 1 --2- 172.21.15.144:0/1999574529 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa314101c30 0x7fa314196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.540+0000 7fa31bf52700 1 -- 172.21.15.144:0/1999574529 >> 172.21.15.144:0/1999574529 conn(0x7fa3140fab80 msgr2=0x7fa314104920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:37.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.540+0000 7fa31bf52700 1 -- 172.21.15.144:0/1999574529 shutdown_connections 2024-09-10T12:05:37.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.540+0000 7fa31bf52700 1 -- 172.21.15.144:0/1999574529 wait complete. 2024-09-10T12:05:37.544 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:05:37.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:37 smithi178 ceph-mon[65940]: from='client.35828 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:37.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:37 smithi178 ceph-mon[65940]: from='client.35832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:37.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:37 smithi178 ceph-mon[65940]: from='client.35836 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:37.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:37 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3843730995' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:05:37.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.647+0000 7f96069dd700 1 -- 172.21.15.144:0/508074362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9600068fd0 msgr2=0x7f9600104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:37.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.647+0000 7f96069dd700 1 --2- 172.21.15.144:0/508074362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9600068fd0 0x7f9600104df0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f95f4009910 tx=0x7f95f4009c20 comp rx=0 tx=0).stop 2024-09-10T12:05:37.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.647+0000 7f96069dd700 1 -- 172.21.15.144:0/508074362 shutdown_connections 2024-09-10T12:05:37.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.647+0000 7f96069dd700 1 --2- 172.21.15.144:0/508074362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9600068fd0 0x7f9600104df0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.648 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.647+0000 7f96069dd700 1 --2- 172.21.15.144:0/508074362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9600068630 0x7f9600068a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.647+0000 7f96069dd700 1 -- 172.21.15.144:0/508074362 >> 172.21.15.144:0/508074362 conn(0x7f9600074f50 msgr2=0x7f9600075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:37.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.647+0000 7f96069dd700 1 -- 172.21.15.144:0/508074362 shutdown_connections 2024-09-10T12:05:37.649 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.647+0000 7f96069dd700 1 -- 172.21.15.144:0/508074362 wait complete. 2024-09-10T12:05:37.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.647+0000 7f96069dd700 1 Processor -- start 2024-09-10T12:05:37.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.647+0000 7f96069dd700 1 -- start start 2024-09-10T12:05:37.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f96069dd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9600068630 0x7f9600195e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:37.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f96069dd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9600068fd0 0x7f9600196360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:37.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f96069dd700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9600196a40 con 0x7f9600068630 2024-09-10T12:05:37.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f96069dd700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f960019a7d0 con 0x7f9600068fd0 2024-09-10T12:05:37.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f95fffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9600068630 0x7f9600195e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:37.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f95fffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9600068630 0x7f9600195e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35908/0 (socket says 172.21.15.144:35908) 2024-09-10T12:05:37.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f95fffff700 1 -- 172.21.15.144:0/1045470860 learned_addr learned my addr 172.21.15.144:0/1045470860 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:05:37.651 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f95ff7fe700 1 --2- 172.21.15.144:0/1045470860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9600068fd0 0x7f9600196360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f95fffff700 1 -- 172.21.15.144:0/1045470860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9600068fd0 msgr2=0x7f9600196360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f95fffff700 1 --2- 172.21.15.144:0/1045470860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9600068fd0 0x7f9600196360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f95fffff700 1 -- 172.21.15.144:0/1045470860 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f40095f0 con 0x7f9600068630 2024-09-10T12:05:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f95fffff700 1 --2- 172.21.15.144:0/1045470860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9600068630 0x7f9600195e20 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f95f000d8d0 tx=0x7f95f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f95fd7fa700 1 -- 172.21.15.144:0/1045470860 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95f0009940 con 0x7f9600068630 2024-09-10T12:05:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f95fd7fa700 1 -- 172.21.15.144:0/1045470860 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f95f0010460 con 0x7f9600068630 2024-09-10T12:05:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f96069dd700 1 -- 172.21.15.144:0/1045470860 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f960019aab0 con 0x7f9600068630 2024-09-10T12:05:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.648+0000 7f95fd7fa700 1 -- 172.21.15.144:0/1045470860 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95f000f5f0 con 0x7f9600068630 2024-09-10T12:05:37.652 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.649+0000 7f96069dd700 1 -- 172.21.15.144:0/1045470860 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f960019b000 con 0x7f9600068630 2024-09-10T12:05:37.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.649+0000 7f96069dd700 1 -- 172.21.15.144:0/1045470860 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96001086d0 con 0x7f9600068630 2024-09-10T12:05:37.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.650+0000 7f95fd7fa700 1 -- 172.21.15.144:0/1045470860 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f95f00105d0 con 0x7f9600068630 2024-09-10T12:05:37.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.650+0000 7f95fd7fa700 1 --2- 172.21.15.144:0/1045470860 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f95e806e200 0x7f95e80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:37.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.651+0000 7f95fd7fa700 1 -- 172.21.15.144:0/1045470860 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f95f008e0a0 con 0x7f9600068630 2024-09-10T12:05:37.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.651+0000 7f95ff7fe700 1 --2- 172.21.15.144:0/1045470860 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f95e806e200 0x7f95e80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:37.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.651+0000 7f95ff7fe700 1 --2- 172.21.15.144:0/1045470860 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f95e806e200 0x7f95e80706b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f95f4006010 tx=0x7f95f400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:37.656 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.655+0000 7f95fd7fa700 1 -- 172.21.15.144:0/1045470860 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f95f0057480 con 0x7f9600068630 2024-09-10T12:05:37.877 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.875+0000 7f96069dd700 1 -- 172.21.15.144:0/1045470860 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f960019b220 con 0x7f95e806e200 2024-09-10T12:05:37.878 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.877+0000 7f95fd7fa700 1 -- 172.21.15.144:0/1045470860 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f960019b220 con 0x7f95e806e200 2024-09-10T12:05:37.878 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:05:37.878 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:05:37.878 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:05:37.878 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:05:37.878 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:05:37.878 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:05:37.878 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:05:37.879 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:05:37.879 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:05:37.879 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:05:37.879 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:05:37.879 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:05:37.879 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:05:37.879 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:05:37.879 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:05:37.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.880+0000 7f96069dd700 1 -- 172.21.15.144:0/1045470860 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f95e806e200 msgr2=0x7f95e80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:37.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.880+0000 7f96069dd700 1 --2- 172.21.15.144:0/1045470860 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f95e806e200 0x7f95e80706b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f95f4006010 tx=0x7f95f400b540 comp rx=0 tx=0).stop 2024-09-10T12:05:37.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.880+0000 7f96069dd700 1 -- 172.21.15.144:0/1045470860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9600068630 msgr2=0x7f9600195e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:37.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.880+0000 7f96069dd700 1 --2- 172.21.15.144:0/1045470860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9600068630 0x7f9600195e20 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f95f000d8d0 tx=0x7f95f000dc90 comp rx=0 tx=0).stop 2024-09-10T12:05:37.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.880+0000 7f96069dd700 1 -- 172.21.15.144:0/1045470860 shutdown_connections 2024-09-10T12:05:37.882 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.880+0000 7f96069dd700 1 --2- 172.21.15.144:0/1045470860 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f95e806e200 0x7f95e80706b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.882 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.880+0000 7f96069dd700 1 --2- 172.21.15.144:0/1045470860 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9600068630 0x7f9600195e20 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.882 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.880+0000 7f96069dd700 1 --2- 172.21.15.144:0/1045470860 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9600068fd0 0x7f9600196360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.882 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.880+0000 7f96069dd700 1 -- 172.21.15.144:0/1045470860 >> 172.21.15.144:0/1045470860 conn(0x7f9600074f50 msgr2=0x7f96000fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:37.882 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.880+0000 7f96069dd700 1 -- 172.21.15.144:0/1045470860 shutdown_connections 2024-09-10T12:05:37.882 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.880+0000 7f96069dd700 1 -- 172.21.15.144:0/1045470860 wait complete. 2024-09-10T12:05:37.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.986+0000 7f07228a8700 1 -- 172.21.15.144:0/3054839450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f071c103450 msgr2=0x7f071c1038c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:37.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.986+0000 7f07228a8700 1 --2- 172.21.15.144:0/3054839450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f071c103450 0x7f071c1038c0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f0710009910 tx=0x7f0710009c20 comp rx=0 tx=0).stop 2024-09-10T12:05:37.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.986+0000 7f07228a8700 1 -- 172.21.15.144:0/3054839450 shutdown_connections 2024-09-10T12:05:37.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.986+0000 7f07228a8700 1 --2- 172.21.15.144:0/3054839450 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f071c103450 0x7f071c1038c0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.986+0000 7f07228a8700 1 --2- 172.21.15.144:0/3054839450 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f071c104790 0x7f071c104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.986+0000 7f07228a8700 1 -- 172.21.15.144:0/3054839450 >> 172.21.15.144:0/3054839450 conn(0x7f071c074850 msgr2=0x7f071c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:37.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.986+0000 7f07228a8700 1 -- 172.21.15.144:0/3054839450 shutdown_connections 2024-09-10T12:05:37.988 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.986+0000 7f07228a8700 1 -- 172.21.15.144:0/3054839450 wait complete. 2024-09-10T12:05:37.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.986+0000 7f07228a8700 1 Processor -- start 2024-09-10T12:05:37.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.987+0000 7f07228a8700 1 -- start start 2024-09-10T12:05:37.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.987+0000 7f07228a8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f071c103450 0x7f071c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:37.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.987+0000 7f07228a8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f071c104790 0x7f071c19af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:37.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.987+0000 7f07228a8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c19b5d0 con 0x7f071c103450 2024-09-10T12:05:37.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.987+0000 7f07228a8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c194a80 con 0x7f071c104790 2024-09-10T12:05:37.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.987+0000 7f071bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f071c103450 0x7f071c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:37.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.987+0000 7f071bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f071c103450 0x7f071c19aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:35922/0 (socket says 172.21.15.144:35922) 2024-09-10T12:05:37.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.987+0000 7f071bfff700 1 -- 172.21.15.144:0/1685430613 learned_addr learned my addr 172.21.15.144:0/1685430613 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:05:37.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.987+0000 7f071b7fe700 1 --2- 172.21.15.144:0/1685430613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f071c104790 0x7f071c19af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:37.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.987+0000 7f071bfff700 1 -- 172.21.15.144:0/1685430613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f071c104790 msgr2=0x7f071c19af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:37.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.987+0000 7f071bfff700 1 --2- 172.21.15.144:0/1685430613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f071c104790 0x7f071c19af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:37.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.987+0000 7f071bfff700 1 -- 172.21.15.144:0/1685430613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07100095f0 con 0x7f071c103450 2024-09-10T12:05:37.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.987+0000 7f071b7fe700 1 --2- 172.21.15.144:0/1685430613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f071c104790 0x7f071c19af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:05:37.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.987+0000 7f071bfff700 1 --2- 172.21.15.144:0/1685430613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f071c103450 0x7f071c19aa00 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f070c00d900 tx=0x7f070c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:37.991 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.988+0000 7f07197fa700 1 -- 172.21.15.144:0/1685430613 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f070c0041f0 con 0x7f071c103450 2024-09-10T12:05:37.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.988+0000 7f07197fa700 1 -- 172.21.15.144:0/1685430613 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f070c004cf0 con 0x7f071c103450 2024-09-10T12:05:37.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.988+0000 7f07197fa700 1 -- 172.21.15.144:0/1685430613 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f070c00b750 con 0x7f071c103450 2024-09-10T12:05:37.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.988+0000 7f07228a8700 1 -- 172.21.15.144:0/1685430613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071c194d60 con 0x7f071c103450 2024-09-10T12:05:37.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.988+0000 7f07228a8700 1 -- 172.21.15.144:0/1685430613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c1952b0 con 0x7f071c103450 2024-09-10T12:05:37.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.989+0000 7f07228a8700 1 -- 172.21.15.144:0/1685430613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071c04ea50 con 0x7f071c103450 2024-09-10T12:05:37.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.989+0000 7f07197fa700 1 -- 172.21.15.144:0/1685430613 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f070c004350 con 0x7f071c103450 2024-09-10T12:05:37.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.990+0000 7f07197fa700 1 --2- 172.21.15.144:0/1685430613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f070406e200 0x7f07040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:05:37.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.990+0000 7f07197fa700 1 -- 172.21.15.144:0/1685430613 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f070c08e260 con 0x7f071c103450 2024-09-10T12:05:37.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.990+0000 7f071b7fe700 1 --2- 172.21.15.144:0/1685430613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f070406e200 0x7f07040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:05:37.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.990+0000 7f071b7fe700 1 --2- 172.21.15.144:0/1685430613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f070406e200 0x7f07040706b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f071000b5c0 tx=0x7f071001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:05:37.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:37.994+0000 7f07197fa700 1 -- 172.21.15.144:0/1685430613 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f070c05b170 con 0x7f071c103450 2024-09-10T12:05:38.285 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:38.283+0000 7f07228a8700 1 -- 172.21.15.144:0/1685430613 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f071c195750 con 0x7f071c103450 2024-09-10T12:05:38.285 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:38.284+0000 7f07197fa700 1 -- 172.21.15.144:0/1685430613 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f070c05ad00 con 0x7f071c103450 2024-09-10T12:05:38.285 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:05:38.285 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:05:38.286 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:05:38.286 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:05:38.286 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:05:38.286 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:05:38.286 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:05:38.286 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:05:38.286 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:05:38.286 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:05:38.286 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:05:38.286 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:05:38.286 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:05:38.286 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:05:38.287 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:05:38.287 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:05:38.287 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:05:38.287 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:05:38.287 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:05:38.287 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:05:38.287 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:05:38.287 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:05:38.287 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:05:38.287 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:05:38.287 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:05:38.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:38.286+0000 7f07228a8700 1 -- 172.21.15.144:0/1685430613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f070406e200 msgr2=0x7f07040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:38.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:38.286+0000 7f07228a8700 1 --2- 172.21.15.144:0/1685430613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f070406e200 0x7f07040706b0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f071000b5c0 tx=0x7f071001a040 comp rx=0 tx=0).stop 2024-09-10T12:05:38.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:38.286+0000 7f07228a8700 1 -- 172.21.15.144:0/1685430613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f071c103450 msgr2=0x7f071c19aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:05:38.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:38.286+0000 7f07228a8700 1 --2- 172.21.15.144:0/1685430613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f071c103450 0x7f071c19aa00 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f070c00d900 tx=0x7f070c00dcc0 comp rx=0 tx=0).stop 2024-09-10T12:05:38.288 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:38.286+0000 7f07228a8700 1 -- 172.21.15.144:0/1685430613 shutdown_connections 2024-09-10T12:05:38.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:38.286+0000 7f07228a8700 1 --2- 172.21.15.144:0/1685430613 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f070406e200 0x7f07040706b0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:38.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:38.286+0000 7f07228a8700 1 --2- 172.21.15.144:0/1685430613 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f071c103450 0x7f071c19aa00 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:38.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:38.286+0000 7f07228a8700 1 --2- 172.21.15.144:0/1685430613 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f071c104790 0x7f071c19af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:05:38.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:38.286+0000 7f07228a8700 1 -- 172.21.15.144:0/1685430613 >> 172.21.15.144:0/1685430613 conn(0x7f071c074850 msgr2=0x7f071c107fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:05:38.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:38.286+0000 7f07228a8700 1 -- 172.21.15.144:0/1685430613 shutdown_connections 2024-09-10T12:05:38.289 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:05:38.286+0000 7f07228a8700 1 -- 172.21.15.144:0/1685430613 wait complete. 2024-09-10T12:05:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:38 smithi178 ceph-mon[65940]: pgmap v1010: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:38.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:38 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1999574529' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:05:38.618 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:38 smithi178 ceph-mon[65940]: from='client.35848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:38.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:38 smithi144 ceph-mon[87436]: pgmap v1010: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:38.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:38 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1999574529' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:05:38.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:38 smithi144 ceph-mon[87436]: from='client.35848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:05:39.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:39 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1685430613' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:05:39.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:39 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1685430613' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:05:40.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:40 smithi178 ceph-mon[65940]: pgmap v1011: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:40.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:40 smithi144 ceph-mon[87436]: pgmap v1011: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:42.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:42 smithi178 ceph-mon[65940]: pgmap v1012: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:42.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:42 smithi144 ceph-mon[87436]: pgmap v1012: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:44.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:44 smithi178 ceph-mon[65940]: pgmap v1013: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:44.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:05:44.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:44 smithi144 ceph-mon[87436]: pgmap v1013: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:44.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:05:46.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:46 smithi178 ceph-mon[65940]: pgmap v1014: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:46.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:46 smithi144 ceph-mon[87436]: pgmap v1014: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:48.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:48 smithi178 ceph-mon[65940]: pgmap v1015: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:48 smithi144 ceph-mon[87436]: pgmap v1015: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:50.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:50 smithi178 ceph-mon[65940]: pgmap v1016: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:50.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:50 smithi144 ceph-mon[87436]: pgmap v1016: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:52.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:52 smithi178 ceph-mon[65940]: pgmap v1017: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:52.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:52 smithi144 ceph-mon[87436]: pgmap v1017: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:54.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:54 smithi178 ceph-mon[65940]: pgmap v1018: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:54.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:54 smithi144 ceph-mon[87436]: pgmap v1018: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:56.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:56 smithi144 ceph-mon[87436]: pgmap v1019: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:56.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:56 smithi178 ceph-mon[65940]: pgmap v1019: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:58.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:58 smithi178 ceph-mon[65940]: pgmap v1020: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:58 smithi144 ceph-mon[87436]: pgmap v1020: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:05:59.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:05:59 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:05:59.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:05:59 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:06:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:00 smithi178 ceph-mon[65940]: pgmap v1021: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:00.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:00 smithi144 ceph-mon[87436]: pgmap v1021: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:02.621 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:02 smithi178 ceph-mon[65940]: pgmap v1022: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:02.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:02 smithi144 ceph-mon[87436]: pgmap v1022: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:04.598 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:04 smithi178 ceph-mon[65940]: pgmap v1023: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:04.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:04 smithi144 ceph-mon[87436]: pgmap v1023: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:06.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:06 smithi144 ceph-mon[87436]: pgmap v1024: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:06.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:06 smithi178 ceph-mon[65940]: pgmap v1024: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:08.409 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.406+0000 7eff3b12a700 1 -- 172.21.15.144:0/242571207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff34100c80 msgr2=0x7eff341093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:08.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.406+0000 7eff3b12a700 1 --2- 172.21.15.144:0/242571207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff34100c80 0x7eff341093e0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7eff20009910 tx=0x7eff20009c20 comp rx=0 tx=0).stop 2024-09-10T12:06:08.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.406+0000 7eff3b12a700 1 -- 172.21.15.144:0/242571207 shutdown_connections 2024-09-10T12:06:08.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.406+0000 7eff3b12a700 1 --2- 172.21.15.144:0/242571207 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff34100c80 0x7eff341093e0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:08.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.406+0000 7eff3b12a700 1 --2- 172.21.15.144:0/242571207 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff34100370 0x7eff34100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:08.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.406+0000 7eff3b12a700 1 -- 172.21.15.144:0/242571207 >> 172.21.15.144:0/242571207 conn(0x7eff340747e0 msgr2=0x7eff34074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:08.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.407+0000 7eff3b12a700 1 -- 172.21.15.144:0/242571207 shutdown_connections 2024-09-10T12:06:08.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.407+0000 7eff3b12a700 1 -- 172.21.15.144:0/242571207 wait complete. 2024-09-10T12:06:08.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.407+0000 7eff3b12a700 1 Processor -- start 2024-09-10T12:06:08.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.407+0000 7eff3b12a700 1 -- start start 2024-09-10T12:06:08.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.407+0000 7eff3b12a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff34100370 0x7eff341948d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:08.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.407+0000 7eff3b12a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff34100c80 0x7eff34194e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:08.410 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.407+0000 7eff3b12a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff34195350 con 0x7eff34100370 2024-09-10T12:06:08.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.407+0000 7eff3b12a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff341954c0 con 0x7eff34100c80 2024-09-10T12:06:08.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.408+0000 7eff38ec6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff34100370 0x7eff341948d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:08.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.408+0000 7eff38ec6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff34100370 0x7eff341948d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34056/0 (socket says 172.21.15.144:34056) 2024-09-10T12:06:08.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.408+0000 7eff38ec6700 1 -- 172.21.15.144:0/997334098 learned_addr learned my addr 172.21.15.144:0/997334098 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:06:08.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.408+0000 7eff33fff700 1 --2- 172.21.15.144:0/997334098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff34100c80 0x7eff34194e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:08.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.408+0000 7eff38ec6700 1 -- 172.21.15.144:0/997334098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff34100c80 msgr2=0x7eff34194e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:08.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.408+0000 7eff38ec6700 1 --2- 172.21.15.144:0/997334098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff34100c80 0x7eff34194e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:08.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.408+0000 7eff38ec6700 1 -- 172.21.15.144:0/997334098 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff200095f0 con 0x7eff34100370 2024-09-10T12:06:08.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.408+0000 7eff38ec6700 1 --2- 172.21.15.144:0/997334098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff34100370 0x7eff341948d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7eff2800d8d0 tx=0x7eff2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:08.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.408+0000 7eff31ffb700 1 -- 172.21.15.144:0/997334098 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff28009940 con 0x7eff34100370 2024-09-10T12:06:08.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.408+0000 7eff31ffb700 1 -- 172.21.15.144:0/997334098 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7eff28010460 con 0x7eff34100370 2024-09-10T12:06:08.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.408+0000 7eff3b12a700 1 -- 172.21.15.144:0/997334098 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff341a1f40 con 0x7eff34100370 2024-09-10T12:06:08.411 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.408+0000 7eff31ffb700 1 -- 172.21.15.144:0/997334098 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff2800f5f0 con 0x7eff34100370 2024-09-10T12:06:08.412 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.409+0000 7eff3b12a700 1 -- 172.21.15.144:0/997334098 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff341a2320 con 0x7eff34100370 2024-09-10T12:06:08.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.410+0000 7eff3b12a700 1 -- 172.21.15.144:0/997334098 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff34106ae0 con 0x7eff34100370 2024-09-10T12:06:08.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.410+0000 7eff31ffb700 1 -- 172.21.15.144:0/997334098 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7eff280105d0 con 0x7eff34100370 2024-09-10T12:06:08.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.410+0000 7eff31ffb700 1 --2- 172.21.15.144:0/997334098 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7eff2406e200 0x7eff240706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:08.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.411+0000 7eff31ffb700 1 -- 172.21.15.144:0/997334098 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7eff2808df20 con 0x7eff34100370 2024-09-10T12:06:08.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.411+0000 7eff33fff700 1 --2- 172.21.15.144:0/997334098 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7eff2406e200 0x7eff240706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:08.413 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.411+0000 7eff33fff700 1 --2- 172.21.15.144:0/997334098 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7eff2406e200 0x7eff240706b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7eff340ffbc0 tx=0x7eff2001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:08.416 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.415+0000 7eff31ffb700 1 -- 172.21.15.144:0/997334098 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7eff280c31b0 con 0x7eff34100370 2024-09-10T12:06:08.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:08 smithi178 ceph-mon[65940]: pgmap v1025: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:08.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.635+0000 7eff3b12a700 1 -- 172.21.15.144:0/997334098 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7eff3402ccf0 con 0x7eff2406e200 2024-09-10T12:06:08.638 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:08 smithi144 ceph-mon[87436]: pgmap v1025: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:08.638 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.637+0000 7eff31ffb700 1 -- 172.21.15.144:0/997334098 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7eff3402ccf0 con 0x7eff2406e200 2024-09-10T12:06:08.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.639+0000 7eff3b12a700 1 -- 172.21.15.144:0/997334098 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7eff2406e200 msgr2=0x7eff240706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:08.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.639+0000 7eff3b12a700 1 --2- 172.21.15.144:0/997334098 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7eff2406e200 0x7eff240706b0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7eff340ffbc0 tx=0x7eff2001a040 comp rx=0 tx=0).stop 2024-09-10T12:06:08.641 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.639+0000 7eff3b12a700 1 -- 172.21.15.144:0/997334098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff34100370 msgr2=0x7eff341948d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:08.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.639+0000 7eff3b12a700 1 --2- 172.21.15.144:0/997334098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff34100370 0x7eff341948d0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7eff2800d8d0 tx=0x7eff2800dc90 comp rx=0 tx=0).stop 2024-09-10T12:06:08.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.639+0000 7eff3b12a700 1 -- 172.21.15.144:0/997334098 shutdown_connections 2024-09-10T12:06:08.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.639+0000 7eff3b12a700 1 --2- 172.21.15.144:0/997334098 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7eff2406e200 0x7eff240706b0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:08.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.639+0000 7eff3b12a700 1 --2- 172.21.15.144:0/997334098 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7eff34100370 0x7eff341948d0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:08.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.639+0000 7eff3b12a700 1 --2- 172.21.15.144:0/997334098 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7eff34100c80 0x7eff34194e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:08.642 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.639+0000 7eff3b12a700 1 -- 172.21.15.144:0/997334098 >> 172.21.15.144:0/997334098 conn(0x7eff340747e0 msgr2=0x7eff34103c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:08.643 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.639+0000 7eff3b12a700 1 -- 172.21.15.144:0/997334098 shutdown_connections 2024-09-10T12:06:08.643 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.639+0000 7eff3b12a700 1 -- 172.21.15.144:0/997334098 wait complete. 2024-09-10T12:06:08.651 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:06:08.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.745+0000 7fb9cd852700 1 -- 172.21.15.144:0/2166884094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c81003b0 msgr2=0x7fb9c8100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:08.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.745+0000 7fb9cd852700 1 --2- 172.21.15.144:0/2166884094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c81003b0 0x7fb9c8100780 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc009910 tx=0x7fb9bc009c20 comp rx=0 tx=0).stop 2024-09-10T12:06:08.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.745+0000 7fb9cd852700 1 -- 172.21.15.144:0/2166884094 shutdown_connections 2024-09-10T12:06:08.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.745+0000 7fb9cd852700 1 --2- 172.21.15.144:0/2166884094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9c8100d50 0x7fb9c8104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:08.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.745+0000 7fb9cd852700 1 --2- 172.21.15.144:0/2166884094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c81003b0 0x7fb9c8100780 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:08.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.745+0000 7fb9cd852700 1 -- 172.21.15.144:0/2166884094 >> 172.21.15.144:0/2166884094 conn(0x7fb9c80fbc30 msgr2=0x7fb9c80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:08.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.745+0000 7fb9cd852700 1 -- 172.21.15.144:0/2166884094 shutdown_connections 2024-09-10T12:06:08.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.745+0000 7fb9cd852700 1 -- 172.21.15.144:0/2166884094 wait complete. 2024-09-10T12:06:08.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.746+0000 7fb9cd852700 1 Processor -- start 2024-09-10T12:06:08.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.746+0000 7fb9cd852700 1 -- start start 2024-09-10T12:06:08.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.746+0000 7fb9cd852700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c81003b0 0x7fb9c8195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:08.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.746+0000 7fb9cd852700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9c8100d50 0x7fb9c8196250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:08.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.746+0000 7fb9cd852700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c8196930 con 0x7fb9c81003b0 2024-09-10T12:06:08.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.746+0000 7fb9cd852700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9c819a6c0 con 0x7fb9c8100d50 2024-09-10T12:06:08.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.746+0000 7fb9c6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c81003b0 0x7fb9c8195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:08.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.746+0000 7fb9c6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c81003b0 0x7fb9c8195d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34080/0 (socket says 172.21.15.144:34080) 2024-09-10T12:06:08.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.746+0000 7fb9c6ffd700 1 -- 172.21.15.144:0/3590770552 learned_addr learned my addr 172.21.15.144:0/3590770552 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:06:08.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.747+0000 7fb9c67fc700 1 --2- 172.21.15.144:0/3590770552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9c8100d50 0x7fb9c8196250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:08.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.747+0000 7fb9c6ffd700 1 -- 172.21.15.144:0/3590770552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9c8100d50 msgr2=0x7fb9c8196250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:08.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.747+0000 7fb9c6ffd700 1 --2- 172.21.15.144:0/3590770552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9c8100d50 0x7fb9c8196250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:08.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.747+0000 7fb9c6ffd700 1 -- 172.21.15.144:0/3590770552 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9bc0095f0 con 0x7fb9c81003b0 2024-09-10T12:06:08.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.747+0000 7fb9c6ffd700 1 --2- 172.21.15.144:0/3590770552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c81003b0 0x7fb9c8195d10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc006010 tx=0x7fb9bc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:08.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.747+0000 7fb9cc850700 1 -- 172.21.15.144:0/3590770552 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9bc01d070 con 0x7fb9c81003b0 2024-09-10T12:06:08.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.747+0000 7fb9cc850700 1 -- 172.21.15.144:0/3590770552 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb9bc00f460 con 0x7fb9c81003b0 2024-09-10T12:06:08.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.747+0000 7fb9cd852700 1 -- 172.21.15.144:0/3590770552 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9c819a940 con 0x7fb9c81003b0 2024-09-10T12:06:08.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.747+0000 7fb9cc850700 1 -- 172.21.15.144:0/3590770552 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9bc017630 con 0x7fb9c81003b0 2024-09-10T12:06:08.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.747+0000 7fb9cd852700 1 -- 172.21.15.144:0/3590770552 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9c819ae30 con 0x7fb9c81003b0 2024-09-10T12:06:08.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.748+0000 7fb9cd852700 1 -- 172.21.15.144:0/3590770552 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9c81086c0 con 0x7fb9c81003b0 2024-09-10T12:06:08.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.748+0000 7fb9cc850700 1 -- 172.21.15.144:0/3590770552 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb9bc00f5d0 con 0x7fb9c81003b0 2024-09-10T12:06:08.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.749+0000 7fb9cc850700 1 --2- 172.21.15.144:0/3590770552 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb9b006e200 0x7fb9b00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:08.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.749+0000 7fb9cc850700 1 -- 172.21.15.144:0/3590770552 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb9bc08f5c0 con 0x7fb9c81003b0 2024-09-10T12:06:08.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.749+0000 7fb9c67fc700 1 --2- 172.21.15.144:0/3590770552 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb9b006e200 0x7fb9b00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:08.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.749+0000 7fb9c67fc700 1 --2- 172.21.15.144:0/3590770552 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb9b006e200 0x7fb9b00706b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8005950 tx=0x7fb9b80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:08.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.753+0000 7fb9cc850700 1 -- 172.21.15.144:0/3590770552 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb9bc058900 con 0x7fb9c81003b0 2024-09-10T12:06:08.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.976+0000 7fb9cd852700 1 -- 172.21.15.144:0/3590770552 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb9c802ccf0 con 0x7fb9b006e200 2024-09-10T12:06:08.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.977+0000 7fb9cc850700 1 -- 172.21.15.144:0/3590770552 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb9c802ccf0 con 0x7fb9b006e200 2024-09-10T12:06:08.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.979+0000 7fb9cd852700 1 -- 172.21.15.144:0/3590770552 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb9b006e200 msgr2=0x7fb9b00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:08.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.979+0000 7fb9cd852700 1 --2- 172.21.15.144:0/3590770552 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb9b006e200 0x7fb9b00706b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fb9b8005950 tx=0x7fb9b80058e0 comp rx=0 tx=0).stop 2024-09-10T12:06:08.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.979+0000 7fb9cd852700 1 -- 172.21.15.144:0/3590770552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c81003b0 msgr2=0x7fb9c8195d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:08.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.979+0000 7fb9cd852700 1 --2- 172.21.15.144:0/3590770552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c81003b0 0x7fb9c8195d10 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc006010 tx=0x7fb9bc00be20 comp rx=0 tx=0).stop 2024-09-10T12:06:08.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.979+0000 7fb9cd852700 1 -- 172.21.15.144:0/3590770552 shutdown_connections 2024-09-10T12:06:08.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.979+0000 7fb9cd852700 1 --2- 172.21.15.144:0/3590770552 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb9b006e200 0x7fb9b00706b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:08.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.979+0000 7fb9cd852700 1 --2- 172.21.15.144:0/3590770552 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb9c81003b0 0x7fb9c8195d10 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:08.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.979+0000 7fb9cd852700 1 --2- 172.21.15.144:0/3590770552 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb9c8100d50 0x7fb9c8196250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:08.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.979+0000 7fb9cd852700 1 -- 172.21.15.144:0/3590770552 >> 172.21.15.144:0/3590770552 conn(0x7fb9c80fbc30 msgr2=0x7fb9c80fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:08.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.980+0000 7fb9cd852700 1 -- 172.21.15.144:0/3590770552 shutdown_connections 2024-09-10T12:06:08.981 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:08.980+0000 7fb9cd852700 1 -- 172.21.15.144:0/3590770552 wait complete. 2024-09-10T12:06:09.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.088+0000 7f9199cb6700 1 -- 172.21.15.144:0/769755334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91940ffd30 msgr2=0x7f9194103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:09.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.088+0000 7f9199cb6700 1 --2- 172.21.15.144:0/769755334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91940ffd30 0x7f9194103ce0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f917c009910 tx=0x7f917c009c20 comp rx=0 tx=0).stop 2024-09-10T12:06:09.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.088+0000 7f9199cb6700 1 -- 172.21.15.144:0/769755334 shutdown_connections 2024-09-10T12:06:09.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.088+0000 7f9199cb6700 1 --2- 172.21.15.144:0/769755334 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91940ffd30 0x7f9194103ce0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:09.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.088+0000 7f9199cb6700 1 --2- 172.21.15.144:0/769755334 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91940ff390 0x7f91940ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:09.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.088+0000 7f9199cb6700 1 -- 172.21.15.144:0/769755334 >> 172.21.15.144:0/769755334 conn(0x7f9194074f50 msgr2=0x7f9194075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:09.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.088+0000 7f9199cb6700 1 -- 172.21.15.144:0/769755334 shutdown_connections 2024-09-10T12:06:09.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.088+0000 7f9199cb6700 1 -- 172.21.15.144:0/769755334 wait complete. 2024-09-10T12:06:09.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.089+0000 7f9199cb6700 1 Processor -- start 2024-09-10T12:06:09.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.089+0000 7f9199cb6700 1 -- start start 2024-09-10T12:06:09.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.089+0000 7f9199cb6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91940ff390 0x7f9194196cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:09.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.089+0000 7f9199cb6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91940ffd30 0x7f9194197200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:09.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.089+0000 7f9199cb6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9194197820 con 0x7f91940ff390 2024-09-10T12:06:09.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.089+0000 7f9199cb6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9194197960 con 0x7f91940ffd30 2024-09-10T12:06:09.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.089+0000 7f91937fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91940ff390 0x7f9194196cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:09.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.089+0000 7f91937fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91940ff390 0x7f9194196cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34088/0 (socket says 172.21.15.144:34088) 2024-09-10T12:06:09.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.089+0000 7f91937fe700 1 -- 172.21.15.144:0/3441046620 learned_addr learned my addr 172.21.15.144:0/3441046620 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:06:09.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.089+0000 7f91937fe700 1 -- 172.21.15.144:0/3441046620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91940ffd30 msgr2=0x7f9194197200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:09.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.089+0000 7f91937fe700 1 --2- 172.21.15.144:0/3441046620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91940ffd30 0x7f9194197200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:09.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.090+0000 7f91937fe700 1 -- 172.21.15.144:0/3441046620 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f917c0095f0 con 0x7f91940ff390 2024-09-10T12:06:09.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.090+0000 7f91937fe700 1 --2- 172.21.15.144:0/3441046620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91940ff390 0x7f9194196cc0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f918400b4b0 tx=0x7f918400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:09.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.090+0000 7f9190ff9700 1 -- 172.21.15.144:0/3441046620 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9184011800 con 0x7f91940ff390 2024-09-10T12:06:09.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.090+0000 7f9199cb6700 1 -- 172.21.15.144:0/3441046620 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f919419fd30 con 0x7f91940ff390 2024-09-10T12:06:09.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.090+0000 7f9190ff9700 1 -- 172.21.15.144:0/3441046620 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9184011e20 con 0x7f91940ff390 2024-09-10T12:06:09.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.090+0000 7f9190ff9700 1 -- 172.21.15.144:0/3441046620 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f918400f2e0 con 0x7f91940ff390 2024-09-10T12:06:09.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.090+0000 7f9199cb6700 1 -- 172.21.15.144:0/3441046620 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91941a01f0 con 0x7f91940ff390 2024-09-10T12:06:09.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.091+0000 7f9199cb6700 1 -- 172.21.15.144:0/3441046620 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9194107670 con 0x7f91940ff390 2024-09-10T12:06:09.094 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.091+0000 7f9190ff9700 1 -- 172.21.15.144:0/3441046620 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9184010380 con 0x7f91940ff390 2024-09-10T12:06:09.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.092+0000 7f9190ff9700 1 --2- 172.21.15.144:0/3441046620 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f918006e200 0x7f91800706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:09.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.092+0000 7f9190ff9700 1 -- 172.21.15.144:0/3441046620 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f918408dc70 con 0x7f91940ff390 2024-09-10T12:06:09.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.092+0000 7f9192ffd700 1 --2- 172.21.15.144:0/3441046620 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f918006e200 0x7f91800706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:09.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.092+0000 7f9192ffd700 1 --2- 172.21.15.144:0/3441046620 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f918006e200 0x7f91800706b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f917c0059d0 tx=0x7f917c005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:09.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.096+0000 7f9190ff9700 1 -- 172.21.15.144:0/3441046620 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9184057050 con 0x7f91940ff390 2024-09-10T12:06:09.306 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.305+0000 7f9199cb6700 1 -- 172.21.15.144:0/3441046620 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f919402cd20 con 0x7f918006e200 2024-09-10T12:06:09.315 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.314+0000 7f9190ff9700 1 -- 172.21.15.144:0/3441046620 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f919402cd20 con 0x7f918006e200 2024-09-10T12:06:09.316 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:06:09.316 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (34m) 12m ago 44m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:06:09.316 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (45m) 12m ago 45m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:06:09.316 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (44m) 12m ago 44m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:06:09.316 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (32m) 12m ago 45m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:06:09.316 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (31m) 12m ago 44m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:06:09.316 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (34m) 12m ago 44m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:06:09.316 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (28m) 12m ago 39m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:06:09.317 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (28m) 12m ago 39m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:06:09.317 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (12m) 12m ago 39m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:06:09.317 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (27m) 12m ago 39m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:06:09.317 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (36m) 12m ago 46m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:06:09.317 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (35m) 12m ago 43m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:06:09.317 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (33m) 12m ago 46m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:06:09.317 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (32m) 12m ago 43m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:06:09.317 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (35m) 12m ago 45m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:06:09.317 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (34m) 12m ago 44m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:06:09.317 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (31m) 12m ago 43m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:06:09.317 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (31m) 12m ago 42m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:06:09.317 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (30m) 12m ago 42m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:06:09.317 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (29m) 12m ago 41m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:06:09.317 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (29m) 12m ago 41m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:06:09.318 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (28m) 12m ago 40m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:06:09.318 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (34m) 12m ago 44m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:06:09.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.316+0000 7f9199cb6700 1 -- 172.21.15.144:0/3441046620 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f918006e200 msgr2=0x7f91800706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:09.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.317+0000 7f9199cb6700 1 --2- 172.21.15.144:0/3441046620 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f918006e200 0x7f91800706b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f917c0059d0 tx=0x7f917c005960 comp rx=0 tx=0).stop 2024-09-10T12:06:09.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.317+0000 7f9199cb6700 1 -- 172.21.15.144:0/3441046620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91940ff390 msgr2=0x7f9194196cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:09.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.317+0000 7f9199cb6700 1 --2- 172.21.15.144:0/3441046620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91940ff390 0x7f9194196cc0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f918400b4b0 tx=0x7f918400b870 comp rx=0 tx=0).stop 2024-09-10T12:06:09.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.317+0000 7f9199cb6700 1 -- 172.21.15.144:0/3441046620 shutdown_connections 2024-09-10T12:06:09.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.317+0000 7f9199cb6700 1 --2- 172.21.15.144:0/3441046620 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f918006e200 0x7f91800706b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:09.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.317+0000 7f9199cb6700 1 --2- 172.21.15.144:0/3441046620 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91940ff390 0x7f9194196cc0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:09.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.317+0000 7f9199cb6700 1 --2- 172.21.15.144:0/3441046620 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91940ffd30 0x7f9194197200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:09.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.317+0000 7f9199cb6700 1 -- 172.21.15.144:0/3441046620 >> 172.21.15.144:0/3441046620 conn(0x7f9194074f50 msgr2=0x7f91940fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:09.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.317+0000 7f9199cb6700 1 -- 172.21.15.144:0/3441046620 shutdown_connections 2024-09-10T12:06:09.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.317+0000 7f9199cb6700 1 -- 172.21.15.144:0/3441046620 wait complete. 2024-09-10T12:06:09.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.420+0000 7f7de6776700 1 -- 172.21.15.144:0/2652317634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de00ff2d0 msgr2=0x7f7de00ff6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:09.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.420+0000 7f7de6776700 1 --2- 172.21.15.144:0/2652317634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de00ff2d0 0x7f7de00ff6a0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f7dd0009910 tx=0x7f7dd0009c20 comp rx=0 tx=0).stop 2024-09-10T12:06:09.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.420+0000 7f7de6776700 1 -- 172.21.15.144:0/2652317634 shutdown_connections 2024-09-10T12:06:09.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.420+0000 7f7de6776700 1 --2- 172.21.15.144:0/2652317634 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7de00ffc70 0x7f7de0103c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:09.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.421+0000 7f7de6776700 1 --2- 172.21.15.144:0/2652317634 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de00ff2d0 0x7f7de00ff6a0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:09.423 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.421+0000 7f7de6776700 1 -- 172.21.15.144:0/2652317634 >> 172.21.15.144:0/2652317634 conn(0x7f7de00fabb0 msgr2=0x7f7de00fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:09.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.421+0000 7f7de6776700 1 -- 172.21.15.144:0/2652317634 shutdown_connections 2024-09-10T12:06:09.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.421+0000 7f7de6776700 1 -- 172.21.15.144:0/2652317634 wait complete. 2024-09-10T12:06:09.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.421+0000 7f7de6776700 1 Processor -- start 2024-09-10T12:06:09.424 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.421+0000 7f7de6776700 1 -- start start 2024-09-10T12:06:09.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.422+0000 7f7de6776700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7de00ff2d0 0x7f7de01939b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:09.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.422+0000 7f7de6776700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de00ffc70 0x7f7de0193ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:09.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.422+0000 7f7de6776700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de0195ad0 con 0x7f7de00ffc70 2024-09-10T12:06:09.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.422+0000 7f7de6776700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7de0194430 con 0x7f7de00ff2d0 2024-09-10T12:06:09.425 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:09 smithi144 ceph-mon[87436]: from='client.35856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:09.426 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:09 smithi144 ceph-mon[87436]: from='client.35860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:09.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.422+0000 7f7ddf7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de00ffc70 0x7f7de0193ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:09.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.422+0000 7f7ddf7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de00ffc70 0x7f7de0193ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34104/0 (socket says 172.21.15.144:34104) 2024-09-10T12:06:09.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.422+0000 7f7ddf7fe700 1 -- 172.21.15.144:0/2293637913 learned_addr learned my addr 172.21.15.144:0/2293637913 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:06:09.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.422+0000 7f7ddffff700 1 --2- 172.21.15.144:0/2293637913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7de00ff2d0 0x7f7de01939b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:09.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.422+0000 7f7ddf7fe700 1 -- 172.21.15.144:0/2293637913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7de00ff2d0 msgr2=0x7f7de01939b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:09.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.422+0000 7f7ddf7fe700 1 --2- 172.21.15.144:0/2293637913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7de00ff2d0 0x7f7de01939b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:09.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.422+0000 7f7ddf7fe700 1 -- 172.21.15.144:0/2293637913 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dd00095f0 con 0x7f7de00ffc70 2024-09-10T12:06:09.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.423+0000 7f7ddf7fe700 1 --2- 172.21.15.144:0/2293637913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de00ffc70 0x7f7de0193ef0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400e8f0 tx=0x7f7dd400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:09.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.423+0000 7f7ddd7fa700 1 -- 172.21.15.144:0/2293637913 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dd4009790 con 0x7f7de00ffc70 2024-09-10T12:06:09.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.423+0000 7f7ddd7fa700 1 -- 172.21.15.144:0/2293637913 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7dd400c810 con 0x7f7de00ffc70 2024-09-10T12:06:09.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.423+0000 7f7de6776700 1 -- 172.21.15.144:0/2293637913 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7de0194710 con 0x7f7de00ffc70 2024-09-10T12:06:09.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.423+0000 7f7ddd7fa700 1 -- 172.21.15.144:0/2293637913 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7dd4005580 con 0x7f7de00ffc70 2024-09-10T12:06:09.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.423+0000 7f7de6776700 1 -- 172.21.15.144:0/2293637913 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7de01a6780 con 0x7f7de00ffc70 2024-09-10T12:06:09.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.424+0000 7f7de6776700 1 -- 172.21.15.144:0/2293637913 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7de004ea50 con 0x7f7de00ffc70 2024-09-10T12:06:09.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.425+0000 7f7ddd7fa700 1 -- 172.21.15.144:0/2293637913 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7dd4004a00 con 0x7f7de00ffc70 2024-09-10T12:06:09.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.425+0000 7f7ddd7fa700 1 --2- 172.21.15.144:0/2293637913 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7dc806e200 0x7f7dc80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:09.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.426+0000 7f7ddffff700 1 --2- 172.21.15.144:0/2293637913 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7dc806e200 0x7f7dc80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:09.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.426+0000 7f7ddd7fa700 1 -- 172.21.15.144:0/2293637913 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7dd4014070 con 0x7f7de00ffc70 2024-09-10T12:06:09.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.426+0000 7f7ddffff700 1 --2- 172.21.15.144:0/2293637913 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7dc806e200 0x7f7dc80706b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00059d0 tx=0x7f7dd0005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:09.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.430+0000 7f7ddd7fa700 1 -- 172.21.15.144:0/2293637913 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7dd40576f0 con 0x7f7de00ffc70 2024-09-10T12:06:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:09 smithi178 ceph-mon[65940]: from='client.35856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:09.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:09 smithi178 ceph-mon[65940]: from='client.35860 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:09.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.726+0000 7f7de6776700 1 -- 172.21.15.144:0/2293637913 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7de0194ee0 con 0x7f7de00ffc70 2024-09-10T12:06:09.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.726+0000 7f7ddd7fa700 1 -- 172.21.15.144:0/2293637913 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f7dd405adb0 con 0x7f7de00ffc70 2024-09-10T12:06:09.728 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:06:09.728 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:06:09.728 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:06:09.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:06:09.728 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:06:09.728 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:06:09.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:06:09.728 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:06:09.728 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:06:09.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:06:09.728 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:06:09.728 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:06:09.728 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:06:09.729 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:06:09.729 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:06:09.729 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:06:09.729 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:06:09.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.729+0000 7f7de6776700 1 -- 172.21.15.144:0/2293637913 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7dc806e200 msgr2=0x7f7dc80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:09.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.729+0000 7f7de6776700 1 --2- 172.21.15.144:0/2293637913 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7dc806e200 0x7f7dc80706b0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7dd00059d0 tx=0x7f7dd0005940 comp rx=0 tx=0).stop 2024-09-10T12:06:09.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.729+0000 7f7de6776700 1 -- 172.21.15.144:0/2293637913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de00ffc70 msgr2=0x7f7de0193ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:09.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.729+0000 7f7de6776700 1 --2- 172.21.15.144:0/2293637913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de00ffc70 0x7f7de0193ef0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f7dd400e8f0 tx=0x7f7dd400ec00 comp rx=0 tx=0).stop 2024-09-10T12:06:09.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.729+0000 7f7de6776700 1 -- 172.21.15.144:0/2293637913 shutdown_connections 2024-09-10T12:06:09.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.729+0000 7f7de6776700 1 --2- 172.21.15.144:0/2293637913 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7dc806e200 0x7f7dc80706b0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:09.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.729+0000 7f7de6776700 1 --2- 172.21.15.144:0/2293637913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7de00ff2d0 0x7f7de01939b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:09.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.729+0000 7f7de6776700 1 --2- 172.21.15.144:0/2293637913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7de00ffc70 0x7f7de0193ef0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:09.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.729+0000 7f7de6776700 1 -- 172.21.15.144:0/2293637913 >> 172.21.15.144:0/2293637913 conn(0x7f7de00fabb0 msgr2=0x7f7de00fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:09.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.729+0000 7f7de6776700 1 -- 172.21.15.144:0/2293637913 shutdown_connections 2024-09-10T12:06:09.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.729+0000 7f7de6776700 1 -- 172.21.15.144:0/2293637913 wait complete. 2024-09-10T12:06:09.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.833+0000 7fce4e9e1700 1 -- 172.21.15.144:0/3028223050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce481003a0 msgr2=0x7fce48100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:09.835 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.833+0000 7fce4e9e1700 1 --2- 172.21.15.144:0/3028223050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce481003a0 0x7fce48100770 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fce38009910 tx=0x7fce38009c20 comp rx=0 tx=0).stop 2024-09-10T12:06:09.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.834+0000 7fce4e9e1700 1 -- 172.21.15.144:0/3028223050 shutdown_connections 2024-09-10T12:06:09.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.834+0000 7fce4e9e1700 1 --2- 172.21.15.144:0/3028223050 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce48100cb0 0x7fce48109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:09.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.834+0000 7fce4e9e1700 1 --2- 172.21.15.144:0/3028223050 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce481003a0 0x7fce48100770 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:09.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.834+0000 7fce4e9e1700 1 -- 172.21.15.144:0/3028223050 >> 172.21.15.144:0/3028223050 conn(0x7fce48074850 msgr2=0x7fce48074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:09.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.834+0000 7fce4e9e1700 1 -- 172.21.15.144:0/3028223050 shutdown_connections 2024-09-10T12:06:09.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.834+0000 7fce4e9e1700 1 -- 172.21.15.144:0/3028223050 wait complete. 2024-09-10T12:06:09.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.834+0000 7fce4e9e1700 1 Processor -- start 2024-09-10T12:06:09.836 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.834+0000 7fce4e9e1700 1 -- start start 2024-09-10T12:06:09.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce4e9e1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce481003a0 0x7fce48196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:09.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce4e9e1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce48100cb0 0x7fce481971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:09.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce4e9e1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce48197800 con 0x7fce481003a0 2024-09-10T12:06:09.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce4e9e1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce48197940 con 0x7fce48100cb0 2024-09-10T12:06:09.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce47fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce481003a0 0x7fce48196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:09.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce47fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce481003a0 0x7fce48196ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34112/0 (socket says 172.21.15.144:34112) 2024-09-10T12:06:09.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce47fff700 1 -- 172.21.15.144:0/2328814945 learned_addr learned my addr 172.21.15.144:0/2328814945 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:06:09.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce47fff700 1 -- 172.21.15.144:0/2328814945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce48100cb0 msgr2=0x7fce481971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:09.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce477fe700 1 --2- 172.21.15.144:0/2328814945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce48100cb0 0x7fce481971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:09.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce47fff700 1 --2- 172.21.15.144:0/2328814945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce48100cb0 0x7fce481971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:09.838 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce47fff700 1 -- 172.21.15.144:0/2328814945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce380095f0 con 0x7fce481003a0 2024-09-10T12:06:09.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce47fff700 1 --2- 172.21.15.144:0/2328814945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce481003a0 0x7fce48196ca0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fce3800b5c0 tx=0x7fce3800be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:09.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce457fa700 1 -- 172.21.15.144:0/2328814945 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce3801d070 con 0x7fce481003a0 2024-09-10T12:06:09.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce4e9e1700 1 -- 172.21.15.144:0/2328814945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4819fd10 con 0x7fce481003a0 2024-09-10T12:06:09.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce457fa700 1 -- 172.21.15.144:0/2328814945 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fce3800f460 con 0x7fce481003a0 2024-09-10T12:06:09.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.835+0000 7fce457fa700 1 -- 172.21.15.144:0/2328814945 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce38017610 con 0x7fce481003a0 2024-09-10T12:06:09.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.836+0000 7fce4e9e1700 1 -- 172.21.15.144:0/2328814945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce481a0120 con 0x7fce481003a0 2024-09-10T12:06:09.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.836+0000 7fce4e9e1700 1 -- 172.21.15.144:0/2328814945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce48106b10 con 0x7fce481003a0 2024-09-10T12:06:09.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.837+0000 7fce457fa700 1 -- 172.21.15.144:0/2328814945 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fce3800f5d0 con 0x7fce481003a0 2024-09-10T12:06:09.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.837+0000 7fce457fa700 1 --2- 172.21.15.144:0/2328814945 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fce3006e200 0x7fce300706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:09.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.838+0000 7fce457fa700 1 -- 172.21.15.144:0/2328814945 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fce3808f6e0 con 0x7fce481003a0 2024-09-10T12:06:09.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.838+0000 7fce477fe700 1 --2- 172.21.15.144:0/2328814945 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fce3006e200 0x7fce300706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:09.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.838+0000 7fce477fe700 1 --2- 172.21.15.144:0/2328814945 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fce3006e200 0x7fce300706b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fce480ffb40 tx=0x7fce3c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:09.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:09.842+0000 7fce457fa700 1 -- 172.21.15.144:0/2328814945 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fce38058ac0 con 0x7fce481003a0 2024-09-10T12:06:10.085 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.084+0000 7fce4e9e1700 1 -- 172.21.15.144:0/2328814945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fce4802cd00 con 0x7fce481003a0 2024-09-10T12:06:10.086 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.084+0000 7fce457fa700 1 -- 172.21.15.144:0/2328814945 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7fce3805c180 con 0x7fce481003a0 2024-09-10T12:06:10.086 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:06:10.086 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:06:10.086 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:06:10.086 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:06:10.086 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:06:10.086 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:06:10.086 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:06:10.086 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:06:10.087 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:06:10.087 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:06:10.087 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:06:10.087 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:06:10.087 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:06:10.087 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:06:10.087 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:06:10.087 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:06:10.087 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:06:10.087 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:06:10.087 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:06:10.088 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:06:10.088 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:06:10.088 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:06:10.088 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:06:10.088 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:06:10.088 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:06:10.088 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:06:10.088 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:06:10.088 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:06:10.088 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:06:10.088 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:06:10.088 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:06:10.088 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:06:10.088 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:06:10.089 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:06:10.089 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:06:10.089 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:06:10.089 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:06:10.089 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:06:10.089 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:06:10.089 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:06:10.089 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:06:10.089 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:06:10.089 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:06:10.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.086+0000 7fce4e9e1700 1 -- 172.21.15.144:0/2328814945 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fce3006e200 msgr2=0x7fce300706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:10.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.086+0000 7fce4e9e1700 1 --2- 172.21.15.144:0/2328814945 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fce3006e200 0x7fce300706b0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fce480ffb40 tx=0x7fce3c008040 comp rx=0 tx=0).stop 2024-09-10T12:06:10.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.086+0000 7fce4e9e1700 1 -- 172.21.15.144:0/2328814945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce481003a0 msgr2=0x7fce48196ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:10.090 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.086+0000 7fce4e9e1700 1 --2- 172.21.15.144:0/2328814945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce481003a0 0x7fce48196ca0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fce3800b5c0 tx=0x7fce3800be00 comp rx=0 tx=0).stop 2024-09-10T12:06:10.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.087+0000 7fce4e9e1700 1 -- 172.21.15.144:0/2328814945 shutdown_connections 2024-09-10T12:06:10.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.087+0000 7fce4e9e1700 1 --2- 172.21.15.144:0/2328814945 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fce3006e200 0x7fce300706b0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.087+0000 7fce4e9e1700 1 --2- 172.21.15.144:0/2328814945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fce481003a0 0x7fce48196ca0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.087+0000 7fce4e9e1700 1 --2- 172.21.15.144:0/2328814945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fce48100cb0 0x7fce481971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.087+0000 7fce4e9e1700 1 -- 172.21.15.144:0/2328814945 >> 172.21.15.144:0/2328814945 conn(0x7fce48074850 msgr2=0x7fce48103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:10.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.087+0000 7fce4e9e1700 1 -- 172.21.15.144:0/2328814945 shutdown_connections 2024-09-10T12:06:10.091 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.087+0000 7fce4e9e1700 1 -- 172.21.15.144:0/2328814945 wait complete. 2024-09-10T12:06:10.091 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:06:10.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.192+0000 7f57bd5e6700 1 -- 172.21.15.144:0/359941199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b80ff310 msgr2=0x7f57b80ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:10.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.192+0000 7f57bd5e6700 1 --2- 172.21.15.144:0/359941199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b80ff310 0x7f57b80ff6e0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f57a0009910 tx=0x7f57a0009c20 comp rx=0 tx=0).stop 2024-09-10T12:06:10.193 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.192+0000 7f57bd5e6700 1 -- 172.21.15.144:0/359941199 shutdown_connections 2024-09-10T12:06:10.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.192+0000 7f57bd5e6700 1 --2- 172.21.15.144:0/359941199 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57b80ffcb0 0x7f57b8103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.192+0000 7f57bd5e6700 1 --2- 172.21.15.144:0/359941199 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b80ff310 0x7f57b80ff6e0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.192+0000 7f57bd5e6700 1 -- 172.21.15.144:0/359941199 >> 172.21.15.144:0/359941199 conn(0x7f57b80fabb0 msgr2=0x7f57b80fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:10.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.192+0000 7f57bd5e6700 1 -- 172.21.15.144:0/359941199 shutdown_connections 2024-09-10T12:06:10.194 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.192+0000 7f57bd5e6700 1 -- 172.21.15.144:0/359941199 wait complete. 2024-09-10T12:06:10.195 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.193+0000 7f57bd5e6700 1 Processor -- start 2024-09-10T12:06:10.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.193+0000 7f57bd5e6700 1 -- start start 2024-09-10T12:06:10.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.193+0000 7f57bd5e6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b80ff310 0x7f57b8100d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:10.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.193+0000 7f57bd5e6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57b80ffcb0 0x7f57b8101260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:10.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.193+0000 7f57bd5e6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b81017a0 con 0x7f57b80ff310 2024-09-10T12:06:10.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.193+0000 7f57bd5e6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b81018e0 con 0x7f57b80ffcb0 2024-09-10T12:06:10.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.193+0000 7f57b6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b80ff310 0x7f57b8100d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:10.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.193+0000 7f57b6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b80ff310 0x7f57b8100d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34128/0 (socket says 172.21.15.144:34128) 2024-09-10T12:06:10.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.193+0000 7f57b6ffd700 1 -- 172.21.15.144:0/1758343263 learned_addr learned my addr 172.21.15.144:0/1758343263 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:06:10.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.193+0000 7f57b67fc700 1 --2- 172.21.15.144:0/1758343263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57b80ffcb0 0x7f57b8101260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:10.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.193+0000 7f57b6ffd700 1 -- 172.21.15.144:0/1758343263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57b80ffcb0 msgr2=0x7f57b8101260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:10.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.193+0000 7f57b6ffd700 1 --2- 172.21.15.144:0/1758343263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57b80ffcb0 0x7f57b8101260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.193+0000 7f57b6ffd700 1 -- 172.21.15.144:0/1758343263 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57a00095f0 con 0x7f57b80ff310 2024-09-10T12:06:10.196 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.194+0000 7f57b6ffd700 1 --2- 172.21.15.144:0/1758343263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b80ff310 0x7f57b8100d20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f57a0006010 tx=0x7f57a000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:10.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.194+0000 7f57affff700 1 -- 172.21.15.144:0/1758343263 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57a001d070 con 0x7f57b80ff310 2024-09-10T12:06:10.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.194+0000 7f57bd5e6700 1 -- 172.21.15.144:0/1758343263 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b8101b60 con 0x7f57b80ff310 2024-09-10T12:06:10.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.194+0000 7f57affff700 1 -- 172.21.15.144:0/1758343263 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f57a000f460 con 0x7f57b80ff310 2024-09-10T12:06:10.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.194+0000 7f57affff700 1 -- 172.21.15.144:0/1758343263 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57a0017630 con 0x7f57b80ff310 2024-09-10T12:06:10.197 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.194+0000 7f57bd5e6700 1 -- 172.21.15.144:0/1758343263 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b81a2280 con 0x7f57b80ff310 2024-09-10T12:06:10.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.195+0000 7f57bd5e6700 1 -- 172.21.15.144:0/1758343263 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b8108630 con 0x7f57b80ff310 2024-09-10T12:06:10.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.196+0000 7f57affff700 1 -- 172.21.15.144:0/1758343263 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f57a000f5d0 con 0x7f57b80ff310 2024-09-10T12:06:10.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.196+0000 7f57affff700 1 --2- 172.21.15.144:0/1758343263 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f57a406e200 0x7f57a40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:10.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.196+0000 7f57affff700 1 -- 172.21.15.144:0/1758343263 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f57a008f730 con 0x7f57b80ff310 2024-09-10T12:06:10.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.196+0000 7f57b67fc700 1 --2- 172.21.15.144:0/1758343263 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f57a406e200 0x7f57a40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:10.198 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.197+0000 7f57b67fc700 1 --2- 172.21.15.144:0/1758343263 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f57a406e200 0x7f57a40706b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f57a8005950 tx=0x7f57a80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:10.201 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.200+0000 7f57affff700 1 -- 172.21.15.144:0/1758343263 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f57a0058af0 con 0x7f57b80ff310 2024-09-10T12:06:10.421 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.420+0000 7f57bd5e6700 1 -- 172.21.15.144:0/1758343263 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f57b802cda0 con 0x7f57a406e200 2024-09-10T12:06:10.422 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.421+0000 7f57affff700 1 -- 172.21.15.144:0/1758343263 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f57b802cda0 con 0x7f57a406e200 2024-09-10T12:06:10.423 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:06:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:06:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:06:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:06:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:06:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:06:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:06:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:06:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:06:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:06:10.423 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:06:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:06:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:06:10.424 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:06:10.424 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:06:10.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.424+0000 7f57bd5e6700 1 -- 172.21.15.144:0/1758343263 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f57a406e200 msgr2=0x7f57a40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:10.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.424+0000 7f57bd5e6700 1 --2- 172.21.15.144:0/1758343263 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f57a406e200 0x7f57a40706b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f57a8005950 tx=0x7f57a80058e0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.424+0000 7f57bd5e6700 1 -- 172.21.15.144:0/1758343263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b80ff310 msgr2=0x7f57b8100d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.424+0000 7f57bd5e6700 1 --2- 172.21.15.144:0/1758343263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b80ff310 0x7f57b8100d20 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f57a0006010 tx=0x7f57a000be20 comp rx=0 tx=0).stop 2024-09-10T12:06:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.424+0000 7f57bd5e6700 1 -- 172.21.15.144:0/1758343263 shutdown_connections 2024-09-10T12:06:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.424+0000 7f57bd5e6700 1 --2- 172.21.15.144:0/1758343263 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f57a406e200 0x7f57a40706b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.424+0000 7f57bd5e6700 1 --2- 172.21.15.144:0/1758343263 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f57b80ff310 0x7f57b8100d20 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.424+0000 7f57bd5e6700 1 --2- 172.21.15.144:0/1758343263 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f57b80ffcb0 0x7f57b8101260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.424+0000 7f57bd5e6700 1 -- 172.21.15.144:0/1758343263 >> 172.21.15.144:0/1758343263 conn(0x7f57b80fabb0 msgr2=0x7f57b8069030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.424+0000 7f57bd5e6700 1 -- 172.21.15.144:0/1758343263 shutdown_connections 2024-09-10T12:06:10.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.424+0000 7f57bd5e6700 1 -- 172.21.15.144:0/1758343263 wait complete. 2024-09-10T12:06:10.530 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.527+0000 7f6517b53700 1 -- 172.21.15.144:0/586329228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6510100d70 msgr2=0x7f6510104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:10.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.527+0000 7f6517b53700 1 --2- 172.21.15.144:0/586329228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6510100d70 0x7f6510104d40 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f6500009910 tx=0x7f6500009c20 comp rx=0 tx=0).stop 2024-09-10T12:06:10.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.527+0000 7f6517b53700 1 -- 172.21.15.144:0/586329228 shutdown_connections 2024-09-10T12:06:10.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.527+0000 7f6517b53700 1 --2- 172.21.15.144:0/586329228 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6510100d70 0x7f6510104d40 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.527+0000 7f6517b53700 1 --2- 172.21.15.144:0/586329228 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f65101003d0 0x7f65101007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.527+0000 7f6517b53700 1 -- 172.21.15.144:0/586329228 >> 172.21.15.144:0/586329228 conn(0x7f65100fbc90 msgr2=0x7f65100fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:10.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.528+0000 7f6517b53700 1 -- 172.21.15.144:0/586329228 shutdown_connections 2024-09-10T12:06:10.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.528+0000 7f6517b53700 1 -- 172.21.15.144:0/586329228 wait complete. 2024-09-10T12:06:10.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.528+0000 7f6517b53700 1 Processor -- start 2024-09-10T12:06:10.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.528+0000 7f6517b53700 1 -- start start 2024-09-10T12:06:10.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.528+0000 7f6517b53700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65101003d0 0x7f6510195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:10.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.528+0000 7f6517b53700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6510100d70 0x7f6510196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:10.531 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.528+0000 7f6517b53700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6510196960 con 0x7f65101003d0 2024-09-10T12:06:10.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.528+0000 7f6517b53700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651019a6f0 con 0x7f6510100d70 2024-09-10T12:06:10.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.528+0000 7f65158ef700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65101003d0 0x7f6510195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:10.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.528+0000 7f65158ef700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65101003d0 0x7f6510195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34152/0 (socket says 172.21.15.144:34152) 2024-09-10T12:06:10.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.528+0000 7f65158ef700 1 -- 172.21.15.144:0/3558016599 learned_addr learned my addr 172.21.15.144:0/3558016599 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:06:10.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.529+0000 7f65150ee700 1 --2- 172.21.15.144:0/3558016599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6510100d70 0x7f6510196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:10.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.529+0000 7f65158ef700 1 -- 172.21.15.144:0/3558016599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6510100d70 msgr2=0x7f6510196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:10.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.529+0000 7f65158ef700 1 --2- 172.21.15.144:0/3558016599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6510100d70 0x7f6510196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.529+0000 7f65158ef700 1 -- 172.21.15.144:0/3558016599 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65000095f0 con 0x7f65101003d0 2024-09-10T12:06:10.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.529+0000 7f65158ef700 1 --2- 172.21.15.144:0/3558016599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65101003d0 0x7f6510195d40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c8a0 tx=0x7f650c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:10.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.529+0000 7f6506ffd700 1 -- 172.21.15.144:0/3558016599 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f650c00cea0 con 0x7f65101003d0 2024-09-10T12:06:10.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.529+0000 7f6506ffd700 1 -- 172.21.15.144:0/3558016599 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f650c004830 con 0x7f65101003d0 2024-09-10T12:06:10.532 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.529+0000 7f6517b53700 1 -- 172.21.15.144:0/3558016599 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f651019a9d0 con 0x7f65101003d0 2024-09-10T12:06:10.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.529+0000 7f6506ffd700 1 -- 172.21.15.144:0/3558016599 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f650c0055b0 con 0x7f65101003d0 2024-09-10T12:06:10.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.529+0000 7f6517b53700 1 -- 172.21.15.144:0/3558016599 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651019af20 con 0x7f65101003d0 2024-09-10T12:06:10.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.530+0000 7f6517b53700 1 -- 172.21.15.144:0/3558016599 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6510108740 con 0x7f65101003d0 2024-09-10T12:06:10.533 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.530+0000 7f6506ffd700 1 -- 172.21.15.144:0/3558016599 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f650c0049a0 con 0x7f65101003d0 2024-09-10T12:06:10.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.531+0000 7f6506ffd700 1 --2- 172.21.15.144:0/3558016599 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f64fc06e200 0x7f64fc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:10.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.531+0000 7f6506ffd700 1 -- 172.21.15.144:0/3558016599 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f650c08dc40 con 0x7f65101003d0 2024-09-10T12:06:10.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.531+0000 7f65150ee700 1 --2- 172.21.15.144:0/3558016599 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f64fc06e200 0x7f64fc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:10.534 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.531+0000 7f65150ee700 1 --2- 172.21.15.144:0/3558016599 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f64fc06e200 0x7f64fc0706b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6510197360 tx=0x7f650001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:10.536 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.535+0000 7f6506ffd700 1 -- 172.21.15.144:0/3558016599 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f650c0570a0 con 0x7f65101003d0 2024-09-10T12:06:10.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:10 smithi178 ceph-mon[65940]: pgmap v1026: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:10.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:10 smithi178 ceph-mon[65940]: from='client.35864 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:10.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:10 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2293637913' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:06:10.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:10 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2328814945' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:06:10.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:10 smithi144 ceph-mon[87436]: pgmap v1026: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:10.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:10 smithi144 ceph-mon[87436]: from='client.35864 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:10.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:10 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2293637913' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:06:10.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:10 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2328814945' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:06:10.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.816+0000 7f6517b53700 1 -- 172.21.15.144:0/3558016599 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f651004ea50 con 0x7f65101003d0 2024-09-10T12:06:10.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.816+0000 7f6506ffd700 1 -- 172.21.15.144:0/3558016599 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f650c01d020 con 0x7f65101003d0 2024-09-10T12:06:10.818 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:06:10.818 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:06:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:06:10.818 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:06:10.818 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:06:10.818 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:06:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:06:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:06:10.818 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:06:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:06:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:06:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:06:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:06:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:06:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:06:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:06:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:06:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:06:10.819 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:06:10.819 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:06:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:06:10.819 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:06:10.819 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:06:10.820 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:06:10.820 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:06:10.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.818+0000 7f6517b53700 1 -- 172.21.15.144:0/3558016599 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f64fc06e200 msgr2=0x7f64fc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:10.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.818+0000 7f6517b53700 1 --2- 172.21.15.144:0/3558016599 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f64fc06e200 0x7f64fc0706b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f6510197360 tx=0x7f650001a040 comp rx=0 tx=0).stop 2024-09-10T12:06:10.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.818+0000 7f6517b53700 1 -- 172.21.15.144:0/3558016599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65101003d0 msgr2=0x7f6510195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:10.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.818+0000 7f6517b53700 1 --2- 172.21.15.144:0/3558016599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65101003d0 0x7f6510195d40 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f650c00c8a0 tx=0x7f650c00cc60 comp rx=0 tx=0).stop 2024-09-10T12:06:10.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.818+0000 7f6517b53700 1 -- 172.21.15.144:0/3558016599 shutdown_connections 2024-09-10T12:06:10.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.818+0000 7f6517b53700 1 --2- 172.21.15.144:0/3558016599 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f64fc06e200 0x7f64fc0706b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.818+0000 7f6517b53700 1 --2- 172.21.15.144:0/3558016599 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f65101003d0 0x7f6510195d40 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.818+0000 7f6517b53700 1 --2- 172.21.15.144:0/3558016599 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6510100d70 0x7f6510196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:10.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.818+0000 7f6517b53700 1 -- 172.21.15.144:0/3558016599 >> 172.21.15.144:0/3558016599 conn(0x7f65100fbc90 msgr2=0x7f65100fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:10.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.819+0000 7f6517b53700 1 -- 172.21.15.144:0/3558016599 shutdown_connections 2024-09-10T12:06:10.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:10.819+0000 7f6517b53700 1 -- 172.21.15.144:0/3558016599 wait complete. 2024-09-10T12:06:11.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:11 smithi178 ceph-mon[65940]: from='client.35876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:11.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:11 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3558016599' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:06:11.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:11 smithi144 ceph-mon[87436]: from='client.35876 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:11.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:11 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3558016599' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:06:12.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:12 smithi178 ceph-mon[65940]: pgmap v1027: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:12.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:12 smithi144 ceph-mon[87436]: pgmap v1027: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:14.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:14 smithi178 ceph-mon[65940]: pgmap v1028: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:14.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:14 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:06:14.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:14 smithi144 ceph-mon[87436]: pgmap v1028: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:14.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:06:16.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:16 smithi178 ceph-mon[65940]: pgmap v1029: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:16.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:16 smithi144 ceph-mon[87436]: pgmap v1029: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:18.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:18 smithi178 ceph-mon[65940]: pgmap v1030: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:18.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:18 smithi144 ceph-mon[87436]: pgmap v1030: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:20.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:20 smithi178 ceph-mon[65940]: pgmap v1031: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:20.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:20 smithi144 ceph-mon[87436]: pgmap v1031: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:22.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:22 smithi178 ceph-mon[65940]: pgmap v1032: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:22.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:22 smithi144 ceph-mon[87436]: pgmap v1032: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:24.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:24 smithi144 ceph-mon[87436]: pgmap v1033: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:24.804 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:24 smithi178 ceph-mon[65940]: pgmap v1033: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:26.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:26 smithi144 ceph-mon[87436]: pgmap v1034: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:26.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:26 smithi178 ceph-mon[65940]: pgmap v1034: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:28.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:28 smithi144 ceph-mon[87436]: pgmap v1035: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:28.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:28 smithi178 ceph-mon[65940]: pgmap v1035: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:29.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:29 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:06:29.855 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:29 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:06:30.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:30 smithi144 ceph-mon[87436]: pgmap v1036: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:30.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:30 smithi178 ceph-mon[65940]: pgmap v1036: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:32.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:32 smithi144 ceph-mon[87436]: pgmap v1037: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:32.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:32 smithi178 ceph-mon[65940]: pgmap v1037: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:34.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:34 smithi144 ceph-mon[87436]: pgmap v1038: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:34.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:34 smithi178 ceph-mon[65940]: pgmap v1038: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:36.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:36 smithi144 ceph-mon[87436]: pgmap v1039: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:36.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:36 smithi178 ceph-mon[65940]: pgmap v1039: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:38.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:38 smithi144 ceph-mon[87436]: pgmap v1040: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:38.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:38 smithi178 ceph-mon[65940]: pgmap v1040: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:40.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:40 smithi144 ceph-mon[87436]: pgmap v1041: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:40.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:40 smithi178 ceph-mon[65940]: pgmap v1041: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:40.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.939+0000 7fb495c76700 1 -- 172.21.15.144:0/776022555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb490100c80 msgr2=0x7fb4901093e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:40.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.939+0000 7fb495c76700 1 --2- 172.21.15.144:0/776022555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb490100c80 0x7fb4901093e0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fb478009940 tx=0x7fb478009c50 comp rx=0 tx=0).stop 2024-09-10T12:06:40.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.939+0000 7fb495c76700 1 -- 172.21.15.144:0/776022555 shutdown_connections 2024-09-10T12:06:40.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.939+0000 7fb495c76700 1 --2- 172.21.15.144:0/776022555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb490100c80 0x7fb4901093e0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:40.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.939+0000 7fb495c76700 1 --2- 172.21.15.144:0/776022555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb490100370 0x7fb490100740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:40.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.939+0000 7fb495c76700 1 -- 172.21.15.144:0/776022555 >> 172.21.15.144:0/776022555 conn(0x7fb4900747e0 msgr2=0x7fb490074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:40.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.939+0000 7fb495c76700 1 -- 172.21.15.144:0/776022555 shutdown_connections 2024-09-10T12:06:40.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.939+0000 7fb495c76700 1 -- 172.21.15.144:0/776022555 wait complete. 2024-09-10T12:06:40.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.940+0000 7fb495c76700 1 Processor -- start 2024-09-10T12:06:40.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.940+0000 7fb495c76700 1 -- start start 2024-09-10T12:06:40.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.940+0000 7fb495c76700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb490100370 0x7fb49019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:40.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.940+0000 7fb495c76700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb490100c80 0x7fb49019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:40.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.940+0000 7fb495c76700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb49019c580 con 0x7fb490100370 2024-09-10T12:06:40.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.940+0000 7fb495c76700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb490195a30 con 0x7fb490100c80 2024-09-10T12:06:40.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.940+0000 7fb48f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb490100370 0x7fb49019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:40.943 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.940+0000 7fb48f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb490100370 0x7fb49019b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57722/0 (socket says 172.21.15.144:57722) 2024-09-10T12:06:40.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.940+0000 7fb48f7fe700 1 -- 172.21.15.144:0/2573934032 learned_addr learned my addr 172.21.15.144:0/2573934032 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:06:40.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.941+0000 7fb48effd700 1 --2- 172.21.15.144:0/2573934032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb490100c80 0x7fb49019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:40.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.941+0000 7fb48f7fe700 1 -- 172.21.15.144:0/2573934032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb490100c80 msgr2=0x7fb49019bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:40.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.941+0000 7fb48f7fe700 1 --2- 172.21.15.144:0/2573934032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb490100c80 0x7fb49019bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:40.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.941+0000 7fb48f7fe700 1 -- 172.21.15.144:0/2573934032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4780095f0 con 0x7fb490100370 2024-09-10T12:06:40.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.941+0000 7fb48f7fe700 1 --2- 172.21.15.144:0/2573934032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb490100370 0x7fb49019b9b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb48000ef10 tx=0x7fb48000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:40.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.941+0000 7fb48cff9700 1 -- 172.21.15.144:0/2573934032 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb48000ca50 con 0x7fb490100370 2024-09-10T12:06:40.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.941+0000 7fb48cff9700 1 -- 172.21.15.144:0/2573934032 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb48000f040 con 0x7fb490100370 2024-09-10T12:06:40.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.941+0000 7fb495c76700 1 -- 172.21.15.144:0/2573934032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb490195cb0 con 0x7fb490100370 2024-09-10T12:06:40.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.941+0000 7fb48cff9700 1 -- 172.21.15.144:0/2573934032 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4800146b0 con 0x7fb490100370 2024-09-10T12:06:40.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.941+0000 7fb495c76700 1 -- 172.21.15.144:0/2573934032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb490196280 con 0x7fb490100370 2024-09-10T12:06:40.944 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.942+0000 7fb495c76700 1 -- 172.21.15.144:0/2573934032 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb490107d20 con 0x7fb490100370 2024-09-10T12:06:40.945 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.943+0000 7fb48cff9700 1 -- 172.21.15.144:0/2573934032 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb48000cbb0 con 0x7fb490100370 2024-09-10T12:06:40.945 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.943+0000 7fb48cff9700 1 --2- 172.21.15.144:0/2573934032 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb47c06e200 0x7fb47c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:40.945 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.943+0000 7fb48cff9700 1 -- 172.21.15.144:0/2573934032 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb48008e490 con 0x7fb490100370 2024-09-10T12:06:40.945 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.943+0000 7fb48effd700 1 --2- 172.21.15.144:0/2573934032 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb47c06e200 0x7fb47c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:40.945 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.944+0000 7fb48effd700 1 --2- 172.21.15.144:0/2573934032 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb47c06e200 0x7fb47c0706b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb4900fdd90 tx=0x7fb478005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:40.948 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:40.947+0000 7fb48cff9700 1 -- 172.21.15.144:0/2573934032 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb48005b3a0 con 0x7fb490100370 2024-09-10T12:06:41.170 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.169+0000 7fb495c76700 1 -- 172.21.15.144:0/2573934032 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb490196560 con 0x7fb47c06e200 2024-09-10T12:06:41.171 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.170+0000 7fb48cff9700 1 -- 172.21.15.144:0/2573934032 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb490196560 con 0x7fb47c06e200 2024-09-10T12:06:41.173 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.172+0000 7fb495c76700 1 -- 172.21.15.144:0/2573934032 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb47c06e200 msgr2=0x7fb47c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:41.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.172+0000 7fb495c76700 1 --2- 172.21.15.144:0/2573934032 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb47c06e200 0x7fb47c0706b0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fb4900fdd90 tx=0x7fb478005a90 comp rx=0 tx=0).stop 2024-09-10T12:06:41.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.172+0000 7fb495c76700 1 -- 172.21.15.144:0/2573934032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb490100370 msgr2=0x7fb49019b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:41.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.172+0000 7fb495c76700 1 --2- 172.21.15.144:0/2573934032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb490100370 0x7fb49019b9b0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fb48000ef10 tx=0x7fb48000c3a0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.172+0000 7fb495c76700 1 -- 172.21.15.144:0/2573934032 shutdown_connections 2024-09-10T12:06:41.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.172+0000 7fb495c76700 1 --2- 172.21.15.144:0/2573934032 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb47c06e200 0x7fb47c0706b0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.172+0000 7fb495c76700 1 --2- 172.21.15.144:0/2573934032 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb490100370 0x7fb49019b9b0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.172+0000 7fb495c76700 1 --2- 172.21.15.144:0/2573934032 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb490100c80 0x7fb49019bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.172+0000 7fb495c76700 1 -- 172.21.15.144:0/2573934032 >> 172.21.15.144:0/2573934032 conn(0x7fb4900747e0 msgr2=0x7fb4900fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:41.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.172+0000 7fb495c76700 1 -- 172.21.15.144:0/2573934032 shutdown_connections 2024-09-10T12:06:41.174 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.172+0000 7fb495c76700 1 -- 172.21.15.144:0/2573934032 wait complete. 2024-09-10T12:06:41.184 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:06:41.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.277+0000 7f9d8a9e4700 1 -- 172.21.15.144:0/3168623446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d841003d0 msgr2=0x7f9d841007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:41.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.277+0000 7f9d8a9e4700 1 --2- 172.21.15.144:0/3168623446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d841003d0 0x7f9d841007a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009910 tx=0x7f9d74009c20 comp rx=0 tx=0).stop 2024-09-10T12:06:41.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.277+0000 7f9d8a9e4700 1 -- 172.21.15.144:0/3168623446 shutdown_connections 2024-09-10T12:06:41.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.277+0000 7f9d8a9e4700 1 --2- 172.21.15.144:0/3168623446 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84100d70 0x7f9d84104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.277+0000 7f9d8a9e4700 1 --2- 172.21.15.144:0/3168623446 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d841003d0 0x7f9d841007a0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.277+0000 7f9d8a9e4700 1 -- 172.21.15.144:0/3168623446 >> 172.21.15.144:0/3168623446 conn(0x7f9d840fbc90 msgr2=0x7f9d840fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:41.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.278+0000 7f9d8a9e4700 1 -- 172.21.15.144:0/3168623446 shutdown_connections 2024-09-10T12:06:41.279 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.278+0000 7f9d8a9e4700 1 -- 172.21.15.144:0/3168623446 wait complete. 2024-09-10T12:06:41.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.278+0000 7f9d8a9e4700 1 Processor -- start 2024-09-10T12:06:41.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.278+0000 7f9d8a9e4700 1 -- start start 2024-09-10T12:06:41.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.279+0000 7f9d8a9e4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d841003d0 0x7f9d84195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:41.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.279+0000 7f9d8a9e4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84100d70 0x7f9d84196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:41.281 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.279+0000 7f9d8a9e4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d84196b40 con 0x7f9d841003d0 2024-09-10T12:06:41.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.279+0000 7f9d8a9e4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8419a830 con 0x7f9d84100d70 2024-09-10T12:06:41.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.279+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d841003d0 0x7f9d84195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:41.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.279+0000 7f9d83fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d841003d0 0x7f9d84195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57736/0 (socket says 172.21.15.144:57736) 2024-09-10T12:06:41.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.279+0000 7f9d83fff700 1 -- 172.21.15.144:0/3355981149 learned_addr learned my addr 172.21.15.144:0/3355981149 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:06:41.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.279+0000 7f9d837fe700 1 --2- 172.21.15.144:0/3355981149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84100d70 0x7f9d84196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:41.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.279+0000 7f9d83fff700 1 -- 172.21.15.144:0/3355981149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84100d70 msgr2=0x7f9d84196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:41.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.279+0000 7f9d83fff700 1 --2- 172.21.15.144:0/3355981149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84100d70 0x7f9d84196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.279+0000 7f9d83fff700 1 -- 172.21.15.144:0/3355981149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d740095f0 con 0x7f9d841003d0 2024-09-10T12:06:41.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.279+0000 7f9d83fff700 1 --2- 172.21.15.144:0/3355981149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d841003d0 0x7f9d84195f20 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9d74006010 tx=0x7f9d7400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:41.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.280+0000 7f9d817fa700 1 -- 172.21.15.144:0/3355981149 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d7401d070 con 0x7f9d841003d0 2024-09-10T12:06:41.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.280+0000 7f9d8a9e4700 1 -- 172.21.15.144:0/3355981149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8419aab0 con 0x7f9d841003d0 2024-09-10T12:06:41.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.280+0000 7f9d817fa700 1 -- 172.21.15.144:0/3355981149 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9d7400f460 con 0x7f9d841003d0 2024-09-10T12:06:41.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.280+0000 7f9d8a9e4700 1 -- 172.21.15.144:0/3355981149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8419afa0 con 0x7f9d841003d0 2024-09-10T12:06:41.282 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.280+0000 7f9d817fa700 1 -- 172.21.15.144:0/3355981149 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9d7402b610 con 0x7f9d841003d0 2024-09-10T12:06:41.283 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.281+0000 7f9d8a9e4700 1 -- 172.21.15.144:0/3355981149 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d841086e0 con 0x7f9d841003d0 2024-09-10T12:06:41.283 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.281+0000 7f9d817fa700 1 -- 172.21.15.144:0/3355981149 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9d7400f5d0 con 0x7f9d841003d0 2024-09-10T12:06:41.283 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.282+0000 7f9d817fa700 1 --2- 172.21.15.144:0/3355981149 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9d6c06e200 0x7f9d6c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:41.283 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.282+0000 7f9d817fa700 1 -- 172.21.15.144:0/3355981149 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9d7408f590 con 0x7f9d841003d0 2024-09-10T12:06:41.283 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.282+0000 7f9d837fe700 1 --2- 172.21.15.144:0/3355981149 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9d6c06e200 0x7f9d6c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:41.284 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.282+0000 7f9d837fe700 1 --2- 172.21.15.144:0/3355981149 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9d6c06e200 0x7f9d6c0706b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9d841974a0 tx=0x7f9d78009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:41.287 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.286+0000 7f9d817fa700 1 -- 172.21.15.144:0/3355981149 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9d74058970 con 0x7f9d841003d0 2024-09-10T12:06:41.509 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.507+0000 7f9d8a9e4700 1 -- 172.21.15.144:0/3355981149 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d8402cca0 con 0x7f9d6c06e200 2024-09-10T12:06:41.510 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.509+0000 7f9d817fa700 1 -- 172.21.15.144:0/3355981149 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f9d8402cca0 con 0x7f9d6c06e200 2024-09-10T12:06:41.512 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.511+0000 7f9d8a9e4700 1 -- 172.21.15.144:0/3355981149 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9d6c06e200 msgr2=0x7f9d6c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:41.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.511+0000 7f9d8a9e4700 1 --2- 172.21.15.144:0/3355981149 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9d6c06e200 0x7f9d6c0706b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f9d841974a0 tx=0x7f9d78009190 comp rx=0 tx=0).stop 2024-09-10T12:06:41.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.511+0000 7f9d8a9e4700 1 -- 172.21.15.144:0/3355981149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d841003d0 msgr2=0x7f9d84195f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:41.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.511+0000 7f9d8a9e4700 1 --2- 172.21.15.144:0/3355981149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d841003d0 0x7f9d84195f20 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f9d74006010 tx=0x7f9d7400bd70 comp rx=0 tx=0).stop 2024-09-10T12:06:41.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.511+0000 7f9d8a9e4700 1 -- 172.21.15.144:0/3355981149 shutdown_connections 2024-09-10T12:06:41.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.511+0000 7f9d8a9e4700 1 --2- 172.21.15.144:0/3355981149 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9d6c06e200 0x7f9d6c0706b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.511+0000 7f9d8a9e4700 1 --2- 172.21.15.144:0/3355981149 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9d841003d0 0x7f9d84195f20 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.511+0000 7f9d8a9e4700 1 --2- 172.21.15.144:0/3355981149 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9d84100d70 0x7f9d84196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.511+0000 7f9d8a9e4700 1 -- 172.21.15.144:0/3355981149 >> 172.21.15.144:0/3355981149 conn(0x7f9d840fbc90 msgr2=0x7f9d840fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:41.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.511+0000 7f9d8a9e4700 1 -- 172.21.15.144:0/3355981149 shutdown_connections 2024-09-10T12:06:41.513 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.511+0000 7f9d8a9e4700 1 -- 172.21.15.144:0/3355981149 wait complete. 2024-09-10T12:06:41.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.619+0000 7f9b6d3bc700 1 -- 172.21.15.144:0/3140490261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b681003d0 msgr2=0x7f9b681007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:41.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.619+0000 7f9b6d3bc700 1 --2- 172.21.15.144:0/3140490261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b681003d0 0x7f9b681007a0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f9b58009910 tx=0x7f9b58009c20 comp rx=0 tx=0).stop 2024-09-10T12:06:41.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.619+0000 7f9b6d3bc700 1 -- 172.21.15.144:0/3140490261 shutdown_connections 2024-09-10T12:06:41.622 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.619+0000 7f9b6d3bc700 1 --2- 172.21.15.144:0/3140490261 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b68100d70 0x7f9b68104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.622 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.619+0000 7f9b6d3bc700 1 --2- 172.21.15.144:0/3140490261 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b681003d0 0x7f9b681007a0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.622 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.619+0000 7f9b6d3bc700 1 -- 172.21.15.144:0/3140490261 >> 172.21.15.144:0/3140490261 conn(0x7f9b680fbc90 msgr2=0x7f9b680fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:41.622 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.620+0000 7f9b6d3bc700 1 -- 172.21.15.144:0/3140490261 shutdown_connections 2024-09-10T12:06:41.622 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.620+0000 7f9b6d3bc700 1 -- 172.21.15.144:0/3140490261 wait complete. 2024-09-10T12:06:41.622 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.620+0000 7f9b6d3bc700 1 Processor -- start 2024-09-10T12:06:41.622 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.620+0000 7f9b6d3bc700 1 -- start start 2024-09-10T12:06:41.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b6d3bc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b681003d0 0x7f9b68195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:41.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b6d3bc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b68100d70 0x7f9b681962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:41.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b6d3bc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b68196980 con 0x7f9b68100d70 2024-09-10T12:06:41.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b6d3bc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6819a710 con 0x7f9b681003d0 2024-09-10T12:06:41.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b667fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b68100d70 0x7f9b681962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:41.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b667fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b68100d70 0x7f9b681962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57752/0 (socket says 172.21.15.144:57752) 2024-09-10T12:06:41.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b667fc700 1 -- 172.21.15.144:0/1877954204 learned_addr learned my addr 172.21.15.144:0/1877954204 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:06:41.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b66ffd700 1 --2- 172.21.15.144:0/1877954204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b681003d0 0x7f9b68195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:41.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b667fc700 1 -- 172.21.15.144:0/1877954204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b681003d0 msgr2=0x7f9b68195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:41.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b667fc700 1 --2- 172.21.15.144:0/1877954204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b681003d0 0x7f9b68195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b667fc700 1 -- 172.21.15.144:0/1877954204 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b580095f0 con 0x7f9b68100d70 2024-09-10T12:06:41.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b66ffd700 1 --2- 172.21.15.144:0/1877954204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b681003d0 0x7f9b68195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:06:41.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b667fc700 1 --2- 172.21.15.144:0/1877954204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b68100d70 0x7f9b681962a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00c8f0 tx=0x7f9b5c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:41.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b4ffff700 1 -- 172.21.15.144:0/1877954204 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b5c007a50 con 0x7f9b68100d70 2024-09-10T12:06:41.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b6d3bc700 1 -- 172.21.15.144:0/1877954204 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b6819a9f0 con 0x7f9b68100d70 2024-09-10T12:06:41.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b4ffff700 1 -- 172.21.15.144:0/1877954204 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b5c007bb0 con 0x7f9b68100d70 2024-09-10T12:06:41.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b6d3bc700 1 -- 172.21.15.144:0/1877954204 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b6819af40 con 0x7f9b68100d70 2024-09-10T12:06:41.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.621+0000 7f9b4ffff700 1 -- 172.21.15.144:0/1877954204 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b5c004a00 con 0x7f9b68100d70 2024-09-10T12:06:41.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.622+0000 7f9b6d3bc700 1 -- 172.21.15.144:0/1877954204 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b681086e0 con 0x7f9b68100d70 2024-09-10T12:06:41.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.623+0000 7f9b4ffff700 1 -- 172.21.15.144:0/1877954204 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9b5c007d20 con 0x7f9b68100d70 2024-09-10T12:06:41.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.623+0000 7f9b4ffff700 1 --2- 172.21.15.144:0/1877954204 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9b5006e200 0x7f9b500706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:41.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.623+0000 7f9b4ffff700 1 -- 172.21.15.144:0/1877954204 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9b5c08e0a0 con 0x7f9b68100d70 2024-09-10T12:06:41.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.623+0000 7f9b66ffd700 1 --2- 172.21.15.144:0/1877954204 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9b5006e200 0x7f9b500706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:41.626 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.624+0000 7f9b66ffd700 1 --2- 172.21.15.144:0/1877954204 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9b5006e200 0x7f9b500706b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9b580059d0 tx=0x7f9b5801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:41.629 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.628+0000 7f9b4ffff700 1 -- 172.21.15.144:0/1877954204 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9b5c057480 con 0x7f9b68100d70 2024-09-10T12:06:41.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.839+0000 7f9b6d3bc700 1 -- 172.21.15.144:0/1877954204 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b6819b2d0 con 0x7f9b5006e200 2024-09-10T12:06:41.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.849+0000 7f9b4ffff700 1 -- 172.21.15.144:0/1877954204 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f9b6819b2d0 con 0x7f9b5006e200 2024-09-10T12:06:41.851 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:06:41.851 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (34m) 13m ago 45m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:06:41.851 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (45m) 13m ago 45m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:06:41.851 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (44m) 13m ago 44m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:06:41.851 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (32m) 13m ago 45m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:06:41.851 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (32m) 13m ago 44m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:06:41.851 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (34m) 13m ago 45m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:06:41.851 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (29m) 13m ago 40m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:06:41.851 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (28m) 13m ago 39m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:06:41.851 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (13m) 13m ago 39m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:06:41.851 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (28m) 13m ago 40m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:06:41.852 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (36m) 13m ago 46m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:06:41.852 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (35m) 13m ago 44m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:06:41.852 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (33m) 13m ago 47m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:06:41.852 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (33m) 13m ago 44m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:06:41.852 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (35m) 13m ago 45m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:06:41.852 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (35m) 13m ago 44m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:06:41.852 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (32m) 13m ago 43m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:06:41.852 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (31m) 13m ago 43m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:06:41.852 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (30m) 13m ago 42m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:06:41.853 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (30m) 13m ago 42m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:06:41.853 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (29m) 13m ago 41m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:06:41.853 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (29m) 13m ago 41m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:06:41.853 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (35m) 13m ago 45m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:06:41.853 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.852+0000 7f9b6d3bc700 1 -- 172.21.15.144:0/1877954204 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9b5006e200 msgr2=0x7f9b500706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:41.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.852+0000 7f9b6d3bc700 1 --2- 172.21.15.144:0/1877954204 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9b5006e200 0x7f9b500706b0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f9b580059d0 tx=0x7f9b5801a040 comp rx=0 tx=0).stop 2024-09-10T12:06:41.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.852+0000 7f9b6d3bc700 1 -- 172.21.15.144:0/1877954204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b68100d70 msgr2=0x7f9b681962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:41.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.852+0000 7f9b6d3bc700 1 --2- 172.21.15.144:0/1877954204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b68100d70 0x7f9b681962a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00c8f0 tx=0x7f9b5c00ccb0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.852+0000 7f9b6d3bc700 1 -- 172.21.15.144:0/1877954204 shutdown_connections 2024-09-10T12:06:41.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.852+0000 7f9b6d3bc700 1 --2- 172.21.15.144:0/1877954204 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9b5006e200 0x7f9b500706b0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.852+0000 7f9b6d3bc700 1 --2- 172.21.15.144:0/1877954204 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9b681003d0 0x7f9b68195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.852+0000 7f9b6d3bc700 1 --2- 172.21.15.144:0/1877954204 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9b68100d70 0x7f9b681962a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.852+0000 7f9b6d3bc700 1 -- 172.21.15.144:0/1877954204 >> 172.21.15.144:0/1877954204 conn(0x7f9b680fbc90 msgr2=0x7f9b680fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:41.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.852+0000 7f9b6d3bc700 1 -- 172.21.15.144:0/1877954204 shutdown_connections 2024-09-10T12:06:41.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.852+0000 7f9b6d3bc700 1 -- 172.21.15.144:0/1877954204 wait complete. 2024-09-10T12:06:41.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.956+0000 7f7440b26700 1 -- 172.21.15.144:0/3617197584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f743c0ffd30 msgr2=0x7f743c103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:41.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.956+0000 7f7440b26700 1 --2- 172.21.15.144:0/3617197584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f743c0ffd30 0x7f743c103ce0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f7424009910 tx=0x7f7424009c20 comp rx=0 tx=0).stop 2024-09-10T12:06:41.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.956+0000 7f7440b26700 1 -- 172.21.15.144:0/3617197584 shutdown_connections 2024-09-10T12:06:41.957 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.956+0000 7f7440b26700 1 --2- 172.21.15.144:0/3617197584 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f743c0ffd30 0x7f743c103ce0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.956+0000 7f7440b26700 1 --2- 172.21.15.144:0/3617197584 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f743c0ff390 0x7f743c0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.956+0000 7f7440b26700 1 -- 172.21.15.144:0/3617197584 >> 172.21.15.144:0/3617197584 conn(0x7f743c074f50 msgr2=0x7f743c075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:41.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.956+0000 7f7440b26700 1 -- 172.21.15.144:0/3617197584 shutdown_connections 2024-09-10T12:06:41.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.956+0000 7f7440b26700 1 -- 172.21.15.144:0/3617197584 wait complete. 2024-09-10T12:06:41.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.956+0000 7f7440b26700 1 Processor -- start 2024-09-10T12:06:41.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.957+0000 7f7440b26700 1 -- start start 2024-09-10T12:06:41.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.957+0000 7f7440b26700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f743c0ff390 0x7f743c100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:41.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.957+0000 7f7440b26700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f743c0ffd30 0x7f743c1012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:41.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.957+0000 7f7440b26700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c102b30 con 0x7f743c0ffd30 2024-09-10T12:06:41.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.957+0000 7f7440b26700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c102ca0 con 0x7f743c0ff390 2024-09-10T12:06:41.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.957+0000 7f7431d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f743c0ffd30 0x7f743c1012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:41.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.957+0000 7f7431d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f743c0ffd30 0x7f743c1012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57772/0 (socket says 172.21.15.144:57772) 2024-09-10T12:06:41.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.957+0000 7f7431d9b700 1 -- 172.21.15.144:0/1887626280 learned_addr learned my addr 172.21.15.144:0/1887626280 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:06:41.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.957+0000 7f743a59c700 1 --2- 172.21.15.144:0/1887626280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f743c0ff390 0x7f743c100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:41.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.957+0000 7f7431d9b700 1 -- 172.21.15.144:0/1887626280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f743c0ff390 msgr2=0x7f743c100d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:41.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.957+0000 7f7431d9b700 1 --2- 172.21.15.144:0/1887626280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f743c0ff390 0x7f743c100d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:41.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.957+0000 7f7431d9b700 1 -- 172.21.15.144:0/1887626280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74240095f0 con 0x7f743c0ffd30 2024-09-10T12:06:41.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.958+0000 7f7431d9b700 1 --2- 172.21.15.144:0/1887626280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f743c0ffd30 0x7f743c1012c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f74240092e0 tx=0x7f742400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:41.960 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.958+0000 7f7433fff700 1 -- 172.21.15.144:0/1887626280 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f742401d070 con 0x7f743c0ffd30 2024-09-10T12:06:41.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.958+0000 7f7433fff700 1 -- 172.21.15.144:0/1887626280 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f742400f460 con 0x7f743c0ffd30 2024-09-10T12:06:41.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.958+0000 7f7440b26700 1 -- 172.21.15.144:0/1887626280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743c101860 con 0x7f743c0ffd30 2024-09-10T12:06:41.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.958+0000 7f7433fff700 1 -- 172.21.15.144:0/1887626280 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74240175f0 con 0x7f743c0ffd30 2024-09-10T12:06:41.961 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.958+0000 7f7440b26700 1 -- 172.21.15.144:0/1887626280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743c1a6730 con 0x7f743c0ffd30 2024-09-10T12:06:41.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.959+0000 7f7440b26700 1 -- 172.21.15.144:0/1887626280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743c04ea50 con 0x7f743c0ffd30 2024-09-10T12:06:41.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.959+0000 7f7433fff700 1 -- 172.21.15.144:0/1887626280 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f742400f5d0 con 0x7f743c0ffd30 2024-09-10T12:06:41.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.960+0000 7f7433fff700 1 --2- 172.21.15.144:0/1887626280 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f742806e200 0x7f74280706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:41.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.960+0000 7f7433fff700 1 -- 172.21.15.144:0/1887626280 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f742408f5d0 con 0x7f743c0ffd30 2024-09-10T12:06:41.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.960+0000 7f743a59c700 1 --2- 172.21.15.144:0/1887626280 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f742806e200 0x7f74280706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:41.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.960+0000 7f743a59c700 1 --2- 172.21.15.144:0/1887626280 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f742806e200 0x7f74280706b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f742c005fd0 tx=0x7f742c00a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:41.965 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:41.964+0000 7f7433fff700 1 -- 172.21.15.144:0/1887626280 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f74240589b0 con 0x7f743c0ffd30 2024-09-10T12:06:42.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.255+0000 7f7440b26700 1 -- 172.21.15.144:0/1887626280 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f743c1a69e0 con 0x7f743c0ffd30 2024-09-10T12:06:42.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.255+0000 7f7433fff700 1 -- 172.21.15.144:0/1887626280 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f742405c070 con 0x7f743c0ffd30 2024-09-10T12:06:42.257 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:06:42.257 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:06:42.257 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:06:42.257 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:06:42.257 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:06:42.258 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:06:42.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:06:42.258 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:06:42.258 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:06:42.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:06:42.258 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:06:42.258 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:06:42.258 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:06:42.258 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:06:42.258 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:06:42.259 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:06:42.259 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:06:42.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.258+0000 7f7440b26700 1 -- 172.21.15.144:0/1887626280 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f742806e200 msgr2=0x7f74280706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:42.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.258+0000 7f7440b26700 1 --2- 172.21.15.144:0/1887626280 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f742806e200 0x7f74280706b0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f742c005fd0 tx=0x7f742c00a560 comp rx=0 tx=0).stop 2024-09-10T12:06:42.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.258+0000 7f7440b26700 1 -- 172.21.15.144:0/1887626280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f743c0ffd30 msgr2=0x7f743c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:42.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.258+0000 7f7440b26700 1 --2- 172.21.15.144:0/1887626280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f743c0ffd30 0x7f743c1012c0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f74240092e0 tx=0x7f742400be20 comp rx=0 tx=0).stop 2024-09-10T12:06:42.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.258+0000 7f7440b26700 1 -- 172.21.15.144:0/1887626280 shutdown_connections 2024-09-10T12:06:42.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.258+0000 7f7440b26700 1 --2- 172.21.15.144:0/1887626280 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f742806e200 0x7f74280706b0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.258+0000 7f7440b26700 1 --2- 172.21.15.144:0/1887626280 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f743c0ff390 0x7f743c100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.258+0000 7f7440b26700 1 --2- 172.21.15.144:0/1887626280 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f743c0ffd30 0x7f743c1012c0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.258+0000 7f7440b26700 1 -- 172.21.15.144:0/1887626280 >> 172.21.15.144:0/1887626280 conn(0x7f743c074f50 msgr2=0x7f743c103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:42.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.258+0000 7f7440b26700 1 -- 172.21.15.144:0/1887626280 shutdown_connections 2024-09-10T12:06:42.260 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.258+0000 7f7440b26700 1 -- 172.21.15.144:0/1887626280 wait complete. 2024-09-10T12:06:42.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.362+0000 7f7fb277e700 1 -- 172.21.15.144:0/419864581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 msgr2=0x7f7fac104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:42.364 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.362+0000 7f7fb277e700 1 --2- 172.21.15.144:0/419864581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac104d40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009910 tx=0x7f7fa0009c20 comp rx=0 tx=0).stop 2024-09-10T12:06:42.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.362+0000 7f7fb277e700 1 -- 172.21.15.144:0/419864581 shutdown_connections 2024-09-10T12:06:42.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.362+0000 7f7fb277e700 1 --2- 172.21.15.144:0/419864581 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac104d40 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.362+0000 7f7fb277e700 1 --2- 172.21.15.144:0/419864581 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fac1003d0 0x7f7fac1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.362+0000 7f7fb277e700 1 -- 172.21.15.144:0/419864581 >> 172.21.15.144:0/419864581 conn(0x7f7fac0fbc90 msgr2=0x7f7fac0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:42.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.362+0000 7f7fb277e700 1 -- 172.21.15.144:0/419864581 shutdown_connections 2024-09-10T12:06:42.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.362+0000 7f7fb277e700 1 -- 172.21.15.144:0/419864581 wait complete. 2024-09-10T12:06:42.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.363+0000 7f7fb277e700 1 Processor -- start 2024-09-10T12:06:42.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.363+0000 7f7fb277e700 1 -- start start 2024-09-10T12:06:42.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.363+0000 7f7fb277e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fac1003d0 0x7f7fac195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:42.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.363+0000 7f7fb277e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:42.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.363+0000 7f7fb277e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac196960 con 0x7f7fac100d70 2024-09-10T12:06:42.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.363+0000 7f7fb277e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac19a6f0 con 0x7f7fac1003d0 2024-09-10T12:06:42.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.364+0000 7f7fab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:42.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.364+0000 7f7fab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39844/0 (socket says 172.21.15.144:39844) 2024-09-10T12:06:42.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.364+0000 7f7fab7fe700 1 -- 172.21.15.144:0/2586299878 learned_addr learned my addr 172.21.15.144:0/2586299878 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:06:42.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.364+0000 7f7fabfff700 1 --2- 172.21.15.144:0/2586299878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fac1003d0 0x7f7fac195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:42.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.364+0000 7f7fab7fe700 1 -- 172.21.15.144:0/2586299878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fac1003d0 msgr2=0x7f7fac195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:42.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.364+0000 7f7fab7fe700 1 --2- 172.21.15.144:0/2586299878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fac1003d0 0x7f7fac195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.364+0000 7f7fab7fe700 1 -- 172.21.15.144:0/2586299878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa00095f0 con 0x7f7fac100d70 2024-09-10T12:06:42.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.364+0000 7f7fab7fe700 1 --2- 172.21.15.144:0/2586299878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac196280 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00092e0 tx=0x7f7fa000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:42.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.364+0000 7f7fa97fa700 1 -- 172.21.15.144:0/2586299878 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa001d070 con 0x7f7fac100d70 2024-09-10T12:06:42.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.364+0000 7f7fa97fa700 1 -- 172.21.15.144:0/2586299878 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7fa0005dc0 con 0x7f7fac100d70 2024-09-10T12:06:42.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.364+0000 7f7fa97fa700 1 -- 172.21.15.144:0/2586299878 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fa0017400 con 0x7f7fac100d70 2024-09-10T12:06:42.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.364+0000 7f7fb277e700 1 -- 172.21.15.144:0/2586299878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fac19a970 con 0x7f7fac100d70 2024-09-10T12:06:42.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.364+0000 7f7fb277e700 1 -- 172.21.15.144:0/2586299878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fac19ae60 con 0x7f7fac100d70 2024-09-10T12:06:42.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.365+0000 7f7fb277e700 1 -- 172.21.15.144:0/2586299878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fac108740 con 0x7f7fac100d70 2024-09-10T12:06:42.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.366+0000 7f7fa97fa700 1 -- 172.21.15.144:0/2586299878 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7fa0003a60 con 0x7f7fac100d70 2024-09-10T12:06:42.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.366+0000 7f7fa97fa700 1 --2- 172.21.15.144:0/2586299878 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f9406e200 0x7f7f940706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:42.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.366+0000 7f7fa97fa700 1 -- 172.21.15.144:0/2586299878 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7fa008f4d0 con 0x7f7fac100d70 2024-09-10T12:06:42.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.366+0000 7f7fabfff700 1 --2- 172.21.15.144:0/2586299878 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f9406e200 0x7f7f940706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:42.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.367+0000 7f7fabfff700 1 --2- 172.21.15.144:0/2586299878 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f9406e200 0x7f7f940706b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c005950 tx=0x7f7f9c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:42.373 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.370+0000 7f7fa97fa700 1 -- 172.21.15.144:0/2586299878 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7fa0058930 con 0x7f7fac100d70 2024-09-10T12:06:42.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.618+0000 7f7fb277e700 1 -- 172.21.15.144:0/2586299878 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7fac04ea50 con 0x7f7fac100d70 2024-09-10T12:06:42.620 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:42 smithi144 ceph-mon[87436]: pgmap v1042: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:42.620 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:42 smithi144 ceph-mon[87436]: from='client.35884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:42.620 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:42 smithi144 ceph-mon[87436]: from='client.35888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:42.620 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:42 smithi144 ceph-mon[87436]: from='client.35892 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:42.620 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:42 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1887626280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:06:42.620 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.618+0000 7f7fa97fa700 1 -- 172.21.15.144:0/2586299878 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f7fa005bff0 con 0x7f7fac100d70 2024-09-10T12:06:42.620 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:06:42.621 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:06:42.621 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:06:42.621 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:06:42.621 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:06:42.621 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:06:42.621 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:06:42.621 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:06:42.621 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:06:42.621 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:06:42.621 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:06:42.621 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:06:42.621 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:06:42.621 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:06:42.622 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:06:42.622 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:06:42.622 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:06:42.622 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:06:42.622 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:06:42.622 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:06:42.622 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:06:42.622 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:06:42.622 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:06:42.622 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:06:42.622 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:06:42.622 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:06:42.622 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:06:42.622 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:06:42.623 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:06:42.623 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:06:42.623 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:06:42.623 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:06:42.623 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:06:42.623 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:06:42.623 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:06:42.623 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:06:42.623 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:06:42.623 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:06:42.623 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:06:42.623 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:06:42.623 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:06:42.623 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:06:42.624 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:06:42.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.621+0000 7f7fb277e700 1 -- 172.21.15.144:0/2586299878 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f9406e200 msgr2=0x7f7f940706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:42.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.621+0000 7f7fb277e700 1 --2- 172.21.15.144:0/2586299878 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f9406e200 0x7f7f940706b0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c005950 tx=0x7f7f9c0058e0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.624 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.621+0000 7f7fb277e700 1 -- 172.21.15.144:0/2586299878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 msgr2=0x7f7fac196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.621+0000 7f7fb277e700 1 --2- 172.21.15.144:0/2586299878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac196280 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f7fa00092e0 tx=0x7f7fa000be20 comp rx=0 tx=0).stop 2024-09-10T12:06:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.621+0000 7f7fb277e700 1 -- 172.21.15.144:0/2586299878 shutdown_connections 2024-09-10T12:06:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.621+0000 7f7fb277e700 1 --2- 172.21.15.144:0/2586299878 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f9406e200 0x7f7f940706b0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.621+0000 7f7fb277e700 1 --2- 172.21.15.144:0/2586299878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7fac1003d0 0x7f7fac195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.621+0000 7f7fb277e700 1 --2- 172.21.15.144:0/2586299878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7fac100d70 0x7f7fac196280 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.621+0000 7f7fb277e700 1 -- 172.21.15.144:0/2586299878 >> 172.21.15.144:0/2586299878 conn(0x7f7fac0fbc90 msgr2=0x7f7fac0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.621+0000 7f7fb277e700 1 -- 172.21.15.144:0/2586299878 shutdown_connections 2024-09-10T12:06:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.621+0000 7f7fb277e700 1 -- 172.21.15.144:0/2586299878 wait complete. 2024-09-10T12:06:42.625 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:06:42.669 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:42 smithi178 ceph-mon[65940]: pgmap v1042: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:42.670 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:42 smithi178 ceph-mon[65940]: from='client.35884 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:42.670 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:42 smithi178 ceph-mon[65940]: from='client.35888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:42.670 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:42 smithi178 ceph-mon[65940]: from='client.35892 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:42.670 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:42 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1887626280' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:06:42.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.727+0000 7fb78359d700 1 -- 172.21.15.144:0/1824197760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c0ff320 msgr2=0x7fb77c0ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:42.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.727+0000 7fb78359d700 1 --2- 172.21.15.144:0/1824197760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c0ff320 0x7fb77c0ff6f0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fb77800b0a0 tx=0x7fb77800b3b0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.727+0000 7fb78359d700 1 -- 172.21.15.144:0/1824197760 shutdown_connections 2024-09-10T12:06:42.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.727+0000 7fb78359d700 1 --2- 172.21.15.144:0/1824197760 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb77c0ffc30 0x7fb77c10a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.727+0000 7fb78359d700 1 --2- 172.21.15.144:0/1824197760 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c0ff320 0x7fb77c0ff6f0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.730 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.727+0000 7fb78359d700 1 -- 172.21.15.144:0/1824197760 >> 172.21.15.144:0/1824197760 conn(0x7fb77c0748b0 msgr2=0x7fb77c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:42.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.727+0000 7fb78359d700 1 -- 172.21.15.144:0/1824197760 shutdown_connections 2024-09-10T12:06:42.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.727+0000 7fb78359d700 1 -- 172.21.15.144:0/1824197760 wait complete. 2024-09-10T12:06:42.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.728+0000 7fb78359d700 1 Processor -- start 2024-09-10T12:06:42.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.728+0000 7fb78359d700 1 -- start start 2024-09-10T12:06:42.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.728+0000 7fb78359d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb77c0ff320 0x7fb77c100500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:42.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.728+0000 7fb78359d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c0ffc30 0x7fb77c100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:42.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.728+0000 7fb78359d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c100f80 con 0x7fb77c0ffc30 2024-09-10T12:06:42.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.728+0000 7fb78359d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb77c1010c0 con 0x7fb77c0ff320 2024-09-10T12:06:42.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.728+0000 7fb780b38700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c0ffc30 0x7fb77c100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:42.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.728+0000 7fb780b38700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c0ffc30 0x7fb77c100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39862/0 (socket says 172.21.15.144:39862) 2024-09-10T12:06:42.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.728+0000 7fb780b38700 1 -- 172.21.15.144:0/2988889222 learned_addr learned my addr 172.21.15.144:0/2988889222 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:06:42.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.728+0000 7fb781339700 1 --2- 172.21.15.144:0/2988889222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb77c0ff320 0x7fb77c100500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:42.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.728+0000 7fb780b38700 1 -- 172.21.15.144:0/2988889222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb77c0ff320 msgr2=0x7fb77c100500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:42.731 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.728+0000 7fb780b38700 1 --2- 172.21.15.144:0/2988889222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb77c0ff320 0x7fb77c100500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.728+0000 7fb780b38700 1 -- 172.21.15.144:0/2988889222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb778009d00 con 0x7fb77c0ffc30 2024-09-10T12:06:42.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.728+0000 7fb780b38700 1 --2- 172.21.15.144:0/2988889222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c0ffc30 0x7fb77c100a40 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00c8a0 tx=0x7fb76c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:42.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.729+0000 7fb7727fc700 1 -- 172.21.15.144:0/2988889222 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb76c0078e0 con 0x7fb77c0ffc30 2024-09-10T12:06:42.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.729+0000 7fb7727fc700 1 -- 172.21.15.144:0/2988889222 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb76c00f450 con 0x7fb77c0ffc30 2024-09-10T12:06:42.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.729+0000 7fb78359d700 1 -- 172.21.15.144:0/2988889222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb77c1a1df0 con 0x7fb77c0ffc30 2024-09-10T12:06:42.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.729+0000 7fb7727fc700 1 -- 172.21.15.144:0/2988889222 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb76c00e5e0 con 0x7fb77c0ffc30 2024-09-10T12:06:42.732 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.729+0000 7fb78359d700 1 -- 172.21.15.144:0/2988889222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77c1a2280 con 0x7fb77c0ffc30 2024-09-10T12:06:42.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.730+0000 7fb78359d700 1 -- 172.21.15.144:0/2988889222 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb77c107da0 con 0x7fb77c0ffc30 2024-09-10T12:06:42.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.730+0000 7fb7727fc700 1 -- 172.21.15.144:0/2988889222 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb76c00f5c0 con 0x7fb77c0ffc30 2024-09-10T12:06:42.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.731+0000 7fb7727fc700 1 --2- 172.21.15.144:0/2988889222 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb76806e200 0x7fb7680706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:42.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.731+0000 7fb7727fc700 1 -- 172.21.15.144:0/2988889222 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb76c08df10 con 0x7fb77c0ffc30 2024-09-10T12:06:42.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.731+0000 7fb781339700 1 --2- 172.21.15.144:0/2988889222 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb76806e200 0x7fb7680706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:42.733 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.731+0000 7fb781339700 1 --2- 172.21.15.144:0/2988889222 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb76806e200 0x7fb7680706b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb778006090 tx=0x7fb778006000 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:42.736 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.735+0000 7fb7727fc700 1 -- 172.21.15.144:0/2988889222 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb76c057370 con 0x7fb77c0ffc30 2024-09-10T12:06:42.958 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.957+0000 7fb78359d700 1 -- 172.21.15.144:0/2988889222 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb77c02cd10 con 0x7fb76806e200 2024-09-10T12:06:42.959 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.958+0000 7fb7727fc700 1 -- 172.21.15.144:0/2988889222 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb77c02cd10 con 0x7fb76806e200 2024-09-10T12:06:42.960 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:06:42.960 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:06:42.960 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:06:42.960 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:06:42.960 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:06:42.960 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:06:42.960 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:06:42.960 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:06:42.960 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:06:42.960 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:06:42.960 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:06:42.960 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:06:42.960 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:06:42.961 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:06:42.961 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:06:42.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.960+0000 7fb78359d700 1 -- 172.21.15.144:0/2988889222 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb76806e200 msgr2=0x7fb7680706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:42.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.960+0000 7fb78359d700 1 --2- 172.21.15.144:0/2988889222 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb76806e200 0x7fb7680706b0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb778006090 tx=0x7fb778006000 comp rx=0 tx=0).stop 2024-09-10T12:06:42.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.960+0000 7fb78359d700 1 -- 172.21.15.144:0/2988889222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c0ffc30 msgr2=0x7fb77c100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:42.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.960+0000 7fb78359d700 1 --2- 172.21.15.144:0/2988889222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c0ffc30 0x7fb77c100a40 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fb76c00c8a0 tx=0x7fb76c00cbb0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.962 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.960+0000 7fb78359d700 1 -- 172.21.15.144:0/2988889222 shutdown_connections 2024-09-10T12:06:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.961+0000 7fb78359d700 1 --2- 172.21.15.144:0/2988889222 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb76806e200 0x7fb7680706b0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.961+0000 7fb78359d700 1 --2- 172.21.15.144:0/2988889222 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb77c0ff320 0x7fb77c100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.961+0000 7fb78359d700 1 --2- 172.21.15.144:0/2988889222 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb77c0ffc30 0x7fb77c100a40 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.961+0000 7fb78359d700 1 -- 172.21.15.144:0/2988889222 >> 172.21.15.144:0/2988889222 conn(0x7fb77c0748b0 msgr2=0x7fb77c104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.961+0000 7fb78359d700 1 -- 172.21.15.144:0/2988889222 shutdown_connections 2024-09-10T12:06:42.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:42.961+0000 7fb78359d700 1 -- 172.21.15.144:0/2988889222 wait complete. 2024-09-10T12:06:43.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.065+0000 7f702eb50700 1 -- 172.21.15.144:0/3195292755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7028068630 msgr2=0x7f7028068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:43.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.065+0000 7f702eb50700 1 --2- 172.21.15.144:0/3195292755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7028068630 0x7f7028068a00 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f7020009910 tx=0x7f7020009c20 comp rx=0 tx=0).stop 2024-09-10T12:06:43.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.065+0000 7f702eb50700 1 -- 172.21.15.144:0/3195292755 shutdown_connections 2024-09-10T12:06:43.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.065+0000 7f702eb50700 1 --2- 172.21.15.144:0/3195292755 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7028068fd0 0x7f7028104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:43.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.065+0000 7f702eb50700 1 --2- 172.21.15.144:0/3195292755 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7028068630 0x7f7028068a00 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:43.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.065+0000 7f702eb50700 1 -- 172.21.15.144:0/3195292755 >> 172.21.15.144:0/3195292755 conn(0x7f7028074f50 msgr2=0x7f7028075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:43.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.065+0000 7f702eb50700 1 -- 172.21.15.144:0/3195292755 shutdown_connections 2024-09-10T12:06:43.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.065+0000 7f702eb50700 1 -- 172.21.15.144:0/3195292755 wait complete. 2024-09-10T12:06:43.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.065+0000 7f702eb50700 1 Processor -- start 2024-09-10T12:06:43.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.066+0000 7f702eb50700 1 -- start start 2024-09-10T12:06:43.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.066+0000 7f702eb50700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7028068630 0x7f7028195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:43.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.066+0000 7f702eb50700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7028068fd0 0x7f7028196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:43.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.066+0000 7f702eb50700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7028196b40 con 0x7f7028068630 2024-09-10T12:06:43.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.066+0000 7f702eb50700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f702819a8d0 con 0x7f7028068fd0 2024-09-10T12:06:43.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.066+0000 7f702c8ec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7028068630 0x7f7028195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:43.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.066+0000 7f702c8ec700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7028068630 0x7f7028195f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:39876/0 (socket says 172.21.15.144:39876) 2024-09-10T12:06:43.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.066+0000 7f702c8ec700 1 -- 172.21.15.144:0/3734334017 learned_addr learned my addr 172.21.15.144:0/3734334017 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:06:43.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.066+0000 7f702c8ec700 1 -- 172.21.15.144:0/3734334017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7028068fd0 msgr2=0x7f7028196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:43.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.066+0000 7f702c8ec700 1 --2- 172.21.15.144:0/3734334017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7028068fd0 0x7f7028196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:43.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.066+0000 7f702c8ec700 1 -- 172.21.15.144:0/3734334017 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70200095f0 con 0x7f7028068630 2024-09-10T12:06:43.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.066+0000 7f702c8ec700 1 --2- 172.21.15.144:0/3734334017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7028068630 0x7f7028195f20 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7020006010 tx=0x7f702000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:43.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.066+0000 7f7025ffb700 1 -- 172.21.15.144:0/3734334017 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f702001d070 con 0x7f7028068630 2024-09-10T12:06:43.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.067+0000 7f702eb50700 1 -- 172.21.15.144:0/3734334017 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702819ab50 con 0x7f7028068630 2024-09-10T12:06:43.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.067+0000 7f7025ffb700 1 -- 172.21.15.144:0/3734334017 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7020004fb0 con 0x7f7028068630 2024-09-10T12:06:43.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.067+0000 7f7025ffb700 1 -- 172.21.15.144:0/3734334017 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7020022ea0 con 0x7f7028068630 2024-09-10T12:06:43.070 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.067+0000 7f702eb50700 1 -- 172.21.15.144:0/3734334017 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702819b040 con 0x7f7028068630 2024-09-10T12:06:43.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.068+0000 7f702eb50700 1 -- 172.21.15.144:0/3734334017 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70281086d0 con 0x7f7028068630 2024-09-10T12:06:43.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.068+0000 7f7025ffb700 1 -- 172.21.15.144:0/3734334017 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7020022470 con 0x7f7028068630 2024-09-10T12:06:43.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.069+0000 7f7025ffb700 1 --2- 172.21.15.144:0/3734334017 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f701406e200 0x7f70140706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:06:43.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.069+0000 7f7025ffb700 1 -- 172.21.15.144:0/3734334017 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f702008f610 con 0x7f7028068630 2024-09-10T12:06:43.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.069+0000 7f7027fff700 1 --2- 172.21.15.144:0/3734334017 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f701406e200 0x7f70140706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:06:43.071 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.069+0000 7f7027fff700 1 --2- 172.21.15.144:0/3734334017 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f701406e200 0x7f70140706b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7028197540 tx=0x7f7018009250 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:06:43.076 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.073+0000 7f7025ffb700 1 -- 172.21.15.144:0/3734334017 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7020058aa0 con 0x7f7028068630 2024-09-10T12:06:43.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.358+0000 7f702eb50700 1 -- 172.21.15.144:0/3734334017 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f702804ea50 con 0x7f7028068630 2024-09-10T12:06:43.360 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.359+0000 7f7025ffb700 1 -- 172.21.15.144:0/3734334017 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f7020022720 con 0x7f7028068630 2024-09-10T12:06:43.360 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:06:43.360 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:06:43.360 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:06:43.360 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:06:43.361 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:06:43.361 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:06:43.361 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:06:43.361 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:06:43.361 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:06:43.361 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:06:43.361 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:06:43.361 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:06:43.361 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:06:43.361 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:06:43.361 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:06:43.361 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:06:43.361 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:06:43.361 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:06:43.362 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:06:43.362 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:06:43.362 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:06:43.362 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:06:43.362 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:06:43.362 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:06:43.362 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:06:43.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.361+0000 7f702eb50700 1 -- 172.21.15.144:0/3734334017 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f701406e200 msgr2=0x7f70140706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:43.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.361+0000 7f702eb50700 1 --2- 172.21.15.144:0/3734334017 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f701406e200 0x7f70140706b0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f7028197540 tx=0x7f7018009250 comp rx=0 tx=0).stop 2024-09-10T12:06:43.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.361+0000 7f702eb50700 1 -- 172.21.15.144:0/3734334017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7028068630 msgr2=0x7f7028195f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:06:43.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.361+0000 7f702eb50700 1 --2- 172.21.15.144:0/3734334017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7028068630 0x7f7028195f20 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f7020006010 tx=0x7f702000be20 comp rx=0 tx=0).stop 2024-09-10T12:06:43.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.361+0000 7f702eb50700 1 -- 172.21.15.144:0/3734334017 shutdown_connections 2024-09-10T12:06:43.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.361+0000 7f702eb50700 1 --2- 172.21.15.144:0/3734334017 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f701406e200 0x7f70140706b0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:43.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.361+0000 7f702eb50700 1 --2- 172.21.15.144:0/3734334017 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7028068630 0x7f7028195f20 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:43.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.361+0000 7f702eb50700 1 --2- 172.21.15.144:0/3734334017 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7028068fd0 0x7f7028196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:06:43.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.361+0000 7f702eb50700 1 -- 172.21.15.144:0/3734334017 >> 172.21.15.144:0/3734334017 conn(0x7f7028074f50 msgr2=0x7f70280fd7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:06:43.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.361+0000 7f702eb50700 1 -- 172.21.15.144:0/3734334017 shutdown_connections 2024-09-10T12:06:43.363 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:06:43.361+0000 7f702eb50700 1 -- 172.21.15.144:0/3734334017 wait complete. 2024-09-10T12:06:43.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:43 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2586299878' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:06:43.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:43 smithi144 ceph-mon[87436]: from='client.35904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:43.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:43 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3734334017' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:06:43.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:43 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2586299878' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:06:43.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:43 smithi178 ceph-mon[65940]: from='client.35904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:06:43.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:43 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3734334017' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:06:44.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:44 smithi144 ceph-mon[87436]: pgmap v1043: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:44.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:06:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:44 smithi178 ceph-mon[65940]: pgmap v1043: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:06:46.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:46 smithi144 ceph-mon[87436]: pgmap v1044: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:46 smithi178 ceph-mon[65940]: pgmap v1044: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:48.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:48 smithi144 ceph-mon[87436]: pgmap v1045: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:48.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:48 smithi178 ceph-mon[65940]: pgmap v1045: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:50.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:50 smithi144 ceph-mon[87436]: pgmap v1046: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:50.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:50 smithi178 ceph-mon[65940]: pgmap v1046: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:52.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:52 smithi144 ceph-mon[87436]: pgmap v1047: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:52.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:52 smithi178 ceph-mon[65940]: pgmap v1047: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:54.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:54 smithi144 ceph-mon[87436]: pgmap v1048: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:54.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:54 smithi178 ceph-mon[65940]: pgmap v1048: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:56.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:56 smithi144 ceph-mon[87436]: pgmap v1049: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:56.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:56 smithi178 ceph-mon[65940]: pgmap v1049: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:58.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:58 smithi144 ceph-mon[87436]: pgmap v1050: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:58.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:58 smithi178 ceph-mon[65940]: pgmap v1050: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:06:59.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:06:59 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:06:59.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:06:59 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:07:00.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:00 smithi178 ceph-mon[65940]: pgmap v1051: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:00.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:00 smithi144 ceph-mon[87436]: pgmap v1051: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:02.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:02 smithi178 ceph-mon[65940]: pgmap v1052: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:02.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:02 smithi144 ceph-mon[87436]: pgmap v1052: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:04.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:04 smithi178 ceph-mon[65940]: pgmap v1053: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:04.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:04 smithi144 ceph-mon[87436]: pgmap v1053: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:06.795 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:06 smithi144 ceph-mon[87436]: pgmap v1054: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:06.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:06 smithi178 ceph-mon[65940]: pgmap v1054: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:08.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:08 smithi178 ceph-mon[65940]: pgmap v1055: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:08 smithi144 ceph-mon[87436]: pgmap v1055: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:10.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:10 smithi178 ceph-mon[65940]: pgmap v1056: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:10.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:10 smithi144 ceph-mon[87436]: pgmap v1056: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:12 smithi178 ceph-mon[65940]: pgmap v1057: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:12 smithi144 ceph-mon[87436]: pgmap v1057: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:13.481 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.479+0000 7fc76a13d700 1 -- 172.21.15.144:0/2580323384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764101c80 msgr2=0x7fc76410a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:13.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.479+0000 7fc76a13d700 1 --2- 172.21.15.144:0/2580323384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764101c80 0x7fc76410a180 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc758009910 tx=0x7fc758009c20 comp rx=0 tx=0).stop 2024-09-10T12:07:13.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.479+0000 7fc76a13d700 1 -- 172.21.15.144:0/2580323384 shutdown_connections 2024-09-10T12:07:13.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.479+0000 7fc76a13d700 1 --2- 172.21.15.144:0/2580323384 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764101c80 0x7fc76410a180 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:13.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.479+0000 7fc76a13d700 1 --2- 172.21.15.144:0/2580323384 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc764101370 0x7fc764101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:13.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.479+0000 7fc76a13d700 1 -- 172.21.15.144:0/2580323384 >> 172.21.15.144:0/2580323384 conn(0x7fc7640fabd0 msgr2=0x7fc7640fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:13.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.479+0000 7fc76a13d700 1 -- 172.21.15.144:0/2580323384 shutdown_connections 2024-09-10T12:07:13.482 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.479+0000 7fc76a13d700 1 -- 172.21.15.144:0/2580323384 wait complete. 2024-09-10T12:07:13.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.479+0000 7fc76a13d700 1 Processor -- start 2024-09-10T12:07:13.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.480+0000 7fc76a13d700 1 -- start start 2024-09-10T12:07:13.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.480+0000 7fc76a13d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764101370 0x7fc764195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:13.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.480+0000 7fc76a13d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc764101c80 0x7fc764196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:13.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.480+0000 7fc76a13d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7641968a0 con 0x7fc764101370 2024-09-10T12:07:13.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.480+0000 7fc76a13d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7641995d0 con 0x7fc764101c80 2024-09-10T12:07:13.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.480+0000 7fc7637fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764101370 0x7fc764195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:13.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.480+0000 7fc7637fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764101370 0x7fc764195cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55480/0 (socket says 172.21.15.144:55480) 2024-09-10T12:07:13.484 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.480+0000 7fc7637fe700 1 -- 172.21.15.144:0/1139225133 learned_addr learned my addr 172.21.15.144:0/1139225133 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:07:13.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.480+0000 7fc762ffd700 1 --2- 172.21.15.144:0/1139225133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc764101c80 0x7fc764196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:13.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.480+0000 7fc7637fe700 1 -- 172.21.15.144:0/1139225133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc764101c80 msgr2=0x7fc764196210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:13.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.480+0000 7fc7637fe700 1 --2- 172.21.15.144:0/1139225133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc764101c80 0x7fc764196210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:13.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.480+0000 7fc7637fe700 1 -- 172.21.15.144:0/1139225133 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7580095f0 con 0x7fc764101370 2024-09-10T12:07:13.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.481+0000 7fc7637fe700 1 --2- 172.21.15.144:0/1139225133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764101370 0x7fc764195cd0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc75400ada0 tx=0x7fc75400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:13.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.481+0000 7fc760ff9700 1 -- 172.21.15.144:0/1139225133 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc75400f070 con 0x7fc764101370 2024-09-10T12:07:13.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.481+0000 7fc760ff9700 1 -- 172.21.15.144:0/1139225133 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc75400ae90 con 0x7fc764101370 2024-09-10T12:07:13.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.481+0000 7fc76a13d700 1 -- 172.21.15.144:0/1139225133 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7641998b0 con 0x7fc764101370 2024-09-10T12:07:13.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.481+0000 7fc760ff9700 1 -- 172.21.15.144:0/1139225133 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc754005690 con 0x7fc764101370 2024-09-10T12:07:13.485 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.481+0000 7fc76a13d700 1 -- 172.21.15.144:0/1139225133 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc764199e00 con 0x7fc764101370 2024-09-10T12:07:13.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.482+0000 7fc76a13d700 1 -- 172.21.15.144:0/1139225133 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7641077f0 con 0x7fc764101370 2024-09-10T12:07:13.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.482+0000 7fc760ff9700 1 -- 172.21.15.144:0/1139225133 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc754005900 con 0x7fc764101370 2024-09-10T12:07:13.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.483+0000 7fc760ff9700 1 --2- 172.21.15.144:0/1139225133 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc74c06e200 0x7fc74c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:13.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.483+0000 7fc762ffd700 1 --2- 172.21.15.144:0/1139225133 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc74c06e200 0x7fc74c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:13.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.483+0000 7fc760ff9700 1 -- 172.21.15.144:0/1139225133 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc75408dae0 con 0x7fc764101370 2024-09-10T12:07:13.486 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.483+0000 7fc762ffd700 1 --2- 172.21.15.144:0/1139225133 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc74c06e200 0x7fc74c0706b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc7641972b0 tx=0x7fc75801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:13.489 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.487+0000 7fc760ff9700 1 -- 172.21.15.144:0/1139225133 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc754056e20 con 0x7fc764101370 2024-09-10T12:07:13.714 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.712+0000 7fc76a13d700 1 -- 172.21.15.144:0/1139225133 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc76419a020 con 0x7fc74c06e200 2024-09-10T12:07:13.715 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.713+0000 7fc760ff9700 1 -- 172.21.15.144:0/1139225133 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fc76419a020 con 0x7fc74c06e200 2024-09-10T12:07:13.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.715+0000 7fc76a13d700 1 -- 172.21.15.144:0/1139225133 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc74c06e200 msgr2=0x7fc74c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:13.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.715+0000 7fc76a13d700 1 --2- 172.21.15.144:0/1139225133 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc74c06e200 0x7fc74c0706b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc7641972b0 tx=0x7fc75801a040 comp rx=0 tx=0).stop 2024-09-10T12:07:13.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.715+0000 7fc76a13d700 1 -- 172.21.15.144:0/1139225133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764101370 msgr2=0x7fc764195cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:13.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.715+0000 7fc76a13d700 1 --2- 172.21.15.144:0/1139225133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764101370 0x7fc764195cd0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc75400ada0 tx=0x7fc75400c330 comp rx=0 tx=0).stop 2024-09-10T12:07:13.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.715+0000 7fc76a13d700 1 -- 172.21.15.144:0/1139225133 shutdown_connections 2024-09-10T12:07:13.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.715+0000 7fc76a13d700 1 --2- 172.21.15.144:0/1139225133 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc74c06e200 0x7fc74c0706b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:13.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.715+0000 7fc76a13d700 1 --2- 172.21.15.144:0/1139225133 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc764101370 0x7fc764195cd0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:13.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.715+0000 7fc76a13d700 1 --2- 172.21.15.144:0/1139225133 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc764101c80 0x7fc764196210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:13.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.715+0000 7fc76a13d700 1 -- 172.21.15.144:0/1139225133 >> 172.21.15.144:0/1139225133 conn(0x7fc7640fabd0 msgr2=0x7fc764104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:13.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.715+0000 7fc76a13d700 1 -- 172.21.15.144:0/1139225133 shutdown_connections 2024-09-10T12:07:13.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.715+0000 7fc76a13d700 1 -- 172.21.15.144:0/1139225133 wait complete. 2024-09-10T12:07:13.728 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:07:13.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.822+0000 7f72c7c92700 1 -- 172.21.15.144:0/278420359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72c00ffce0 msgr2=0x7f72c0103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:13.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.822+0000 7f72c7c92700 1 --2- 172.21.15.144:0/278420359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72c00ffce0 0x7f72c0103cb0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f72b0009910 tx=0x7f72b0009c20 comp rx=0 tx=0).stop 2024-09-10T12:07:13.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.822+0000 7f72c7c92700 1 -- 172.21.15.144:0/278420359 shutdown_connections 2024-09-10T12:07:13.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.822+0000 7f72c7c92700 1 --2- 172.21.15.144:0/278420359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72c00ffce0 0x7f72c0103cb0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:13.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.822+0000 7f72c7c92700 1 --2- 172.21.15.144:0/278420359 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72c00ff340 0x7f72c00ff710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:13.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.822+0000 7f72c7c92700 1 -- 172.21.15.144:0/278420359 >> 172.21.15.144:0/278420359 conn(0x7f72c00fabe0 msgr2=0x7f72c00fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:13.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.822+0000 7f72c7c92700 1 -- 172.21.15.144:0/278420359 shutdown_connections 2024-09-10T12:07:13.825 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.822+0000 7f72c7c92700 1 -- 172.21.15.144:0/278420359 wait complete. 2024-09-10T12:07:13.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.822+0000 7f72c7c92700 1 Processor -- start 2024-09-10T12:07:13.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.822+0000 7f72c7c92700 1 -- start start 2024-09-10T12:07:13.826 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.823+0000 7f72c7c92700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72c00ff340 0x7f72c0194d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:13.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.823+0000 7f72c7c92700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72c00ffce0 0x7f72c01952b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:13.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.823+0000 7f72c7c92700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0195990 con 0x7f72c00ffce0 2024-09-10T12:07:13.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.823+0000 7f72c7c92700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c0199720 con 0x7f72c00ff340 2024-09-10T12:07:13.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.823+0000 7f72c522d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72c00ffce0 0x7f72c01952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:13.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.823+0000 7f72c522d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72c00ffce0 0x7f72c01952b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55504/0 (socket says 172.21.15.144:55504) 2024-09-10T12:07:13.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.823+0000 7f72c522d700 1 -- 172.21.15.144:0/2818027908 learned_addr learned my addr 172.21.15.144:0/2818027908 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:07:13.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.823+0000 7f72c5a2e700 1 --2- 172.21.15.144:0/2818027908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72c00ff340 0x7f72c0194d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:13.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.823+0000 7f72c522d700 1 -- 172.21.15.144:0/2818027908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72c00ff340 msgr2=0x7f72c0194d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:13.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.823+0000 7f72c522d700 1 --2- 172.21.15.144:0/2818027908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72c00ff340 0x7f72c0194d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:13.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.823+0000 7f72c522d700 1 -- 172.21.15.144:0/2818027908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b00095f0 con 0x7f72c00ffce0 2024-09-10T12:07:13.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.823+0000 7f72c522d700 1 --2- 172.21.15.144:0/2818027908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72c00ffce0 0x7f72c01952b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f72b000bda0 tx=0x7f72b000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:13.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.823+0000 7f72b6ffd700 1 -- 172.21.15.144:0/2818027908 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b001d070 con 0x7f72c00ffce0 2024-09-10T12:07:13.827 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.824+0000 7f72b6ffd700 1 -- 172.21.15.144:0/2818027908 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f72b000f460 con 0x7f72c00ffce0 2024-09-10T12:07:13.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.824+0000 7f72c7c92700 1 -- 172.21.15.144:0/2818027908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c01999a0 con 0x7f72c00ffce0 2024-09-10T12:07:13.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.824+0000 7f72b6ffd700 1 -- 172.21.15.144:0/2818027908 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72b0017630 con 0x7f72c00ffce0 2024-09-10T12:07:13.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.824+0000 7f72c7c92700 1 -- 172.21.15.144:0/2818027908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c0199e90 con 0x7f72c00ffce0 2024-09-10T12:07:13.828 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.825+0000 7f72c7c92700 1 -- 172.21.15.144:0/2818027908 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c0107600 con 0x7f72c00ffce0 2024-09-10T12:07:13.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.825+0000 7f72b6ffd700 1 -- 172.21.15.144:0/2818027908 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f72b000fab0 con 0x7f72c00ffce0 2024-09-10T12:07:13.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.825+0000 7f72b6ffd700 1 --2- 172.21.15.144:0/2818027908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f72ac06e200 0x7f72ac0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:13.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.826+0000 7f72b6ffd700 1 -- 172.21.15.144:0/2818027908 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f72b008f570 con 0x7f72c00ffce0 2024-09-10T12:07:13.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.826+0000 7f72c5a2e700 1 --2- 172.21.15.144:0/2818027908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f72ac06e200 0x7f72ac0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:13.829 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.826+0000 7f72c5a2e700 1 --2- 172.21.15.144:0/2818027908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f72ac06e200 0x7f72ac0706b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009a10 tx=0x7f72bc009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:13.832 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:13.830+0000 7f72b6ffd700 1 -- 172.21.15.144:0/2818027908 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f72b0058950 con 0x7f72c00ffce0 2024-09-10T12:07:14.052 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.049+0000 7f72c7c92700 1 -- 172.21.15.144:0/2818027908 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f72c019a230 con 0x7f72ac06e200 2024-09-10T12:07:14.053 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.051+0000 7f72b6ffd700 1 -- 172.21.15.144:0/2818027908 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f72c019a230 con 0x7f72ac06e200 2024-09-10T12:07:14.055 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.053+0000 7f72c7c92700 1 -- 172.21.15.144:0/2818027908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f72ac06e200 msgr2=0x7f72ac0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:14.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.053+0000 7f72c7c92700 1 --2- 172.21.15.144:0/2818027908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f72ac06e200 0x7f72ac0706b0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009a10 tx=0x7f72bc009190 comp rx=0 tx=0).stop 2024-09-10T12:07:14.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.053+0000 7f72c7c92700 1 -- 172.21.15.144:0/2818027908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72c00ffce0 msgr2=0x7f72c01952b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:14.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.053+0000 7f72c7c92700 1 --2- 172.21.15.144:0/2818027908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72c00ffce0 0x7f72c01952b0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f72b000bda0 tx=0x7f72b000bdd0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.053+0000 7f72c7c92700 1 -- 172.21.15.144:0/2818027908 shutdown_connections 2024-09-10T12:07:14.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.053+0000 7f72c7c92700 1 --2- 172.21.15.144:0/2818027908 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f72ac06e200 0x7f72ac0706b0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.053+0000 7f72c7c92700 1 --2- 172.21.15.144:0/2818027908 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f72c00ff340 0x7f72c0194d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.053+0000 7f72c7c92700 1 --2- 172.21.15.144:0/2818027908 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f72c00ffce0 0x7f72c01952b0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.053+0000 7f72c7c92700 1 -- 172.21.15.144:0/2818027908 >> 172.21.15.144:0/2818027908 conn(0x7f72c00fabe0 msgr2=0x7f72c00fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:14.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.053+0000 7f72c7c92700 1 -- 172.21.15.144:0/2818027908 shutdown_connections 2024-09-10T12:07:14.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.053+0000 7f72c7c92700 1 -- 172.21.15.144:0/2818027908 wait complete. 2024-09-10T12:07:14.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.157+0000 7f1ede1b2700 1 -- 172.21.15.144:0/1546854590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed80ffce0 msgr2=0x7f1ed8103cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:14.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.157+0000 7f1ede1b2700 1 --2- 172.21.15.144:0/1546854590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed80ffce0 0x7f1ed8103cb0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc009910 tx=0x7f1ecc009c20 comp rx=0 tx=0).stop 2024-09-10T12:07:14.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.157+0000 7f1ede1b2700 1 -- 172.21.15.144:0/1546854590 shutdown_connections 2024-09-10T12:07:14.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.157+0000 7f1ede1b2700 1 --2- 172.21.15.144:0/1546854590 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed80ffce0 0x7f1ed8103cb0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.159 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.157+0000 7f1ede1b2700 1 --2- 172.21.15.144:0/1546854590 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed80ff340 0x7f1ed80ff710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.157+0000 7f1ede1b2700 1 -- 172.21.15.144:0/1546854590 >> 172.21.15.144:0/1546854590 conn(0x7f1ed80fabe0 msgr2=0x7f1ed80fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:14.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.157+0000 7f1ede1b2700 1 -- 172.21.15.144:0/1546854590 shutdown_connections 2024-09-10T12:07:14.160 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.157+0000 7f1ede1b2700 1 -- 172.21.15.144:0/1546854590 wait complete. 2024-09-10T12:07:14.161 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.157+0000 7f1ede1b2700 1 Processor -- start 2024-09-10T12:07:14.161 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.157+0000 7f1ede1b2700 1 -- start start 2024-09-10T12:07:14.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.158+0000 7f1ede1b2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed80ff340 0x7f1ed8194a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:14.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.158+0000 7f1ede1b2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed80ffce0 0x7f1ed8194f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:14.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.158+0000 7f1ede1b2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed8069d70 con 0x7f1ed80ffce0 2024-09-10T12:07:14.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.158+0000 7f1ede1b2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed8195480 con 0x7f1ed80ff340 2024-09-10T12:07:14.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.158+0000 7f1ed6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed80ffce0 0x7f1ed8194f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:14.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.158+0000 7f1ed6ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed80ffce0 0x7f1ed8194f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55518/0 (socket says 172.21.15.144:55518) 2024-09-10T12:07:14.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.158+0000 7f1ed6ffd700 1 -- 172.21.15.144:0/287531657 learned_addr learned my addr 172.21.15.144:0/287531657 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:07:14.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.158+0000 7f1ed77fe700 1 --2- 172.21.15.144:0/287531657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed80ff340 0x7f1ed8194a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:14.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.158+0000 7f1ed6ffd700 1 -- 172.21.15.144:0/287531657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed80ff340 msgr2=0x7f1ed8194a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:14.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.158+0000 7f1ed6ffd700 1 --2- 172.21.15.144:0/287531657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed80ff340 0x7f1ed8194a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.158+0000 7f1ed6ffd700 1 -- 172.21.15.144:0/287531657 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc0095f0 con 0x7f1ed80ffce0 2024-09-10T12:07:14.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.158+0000 7f1ed77fe700 1 --2- 172.21.15.144:0/287531657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed80ff340 0x7f1ed8194a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:07:14.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.158+0000 7f1ed6ffd700 1 --2- 172.21.15.144:0/287531657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed80ffce0 0x7f1ed8194f40 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00bda0 tx=0x7f1ecc00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:14.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.158+0000 7f1ed4ff9700 1 -- 172.21.15.144:0/287531657 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ecc01d070 con 0x7f1ed80ffce0 2024-09-10T12:07:14.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.158+0000 7f1ede1b2700 1 -- 172.21.15.144:0/287531657 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed8195700 con 0x7f1ed80ffce0 2024-09-10T12:07:14.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.159+0000 7f1ed4ff9700 1 -- 172.21.15.144:0/287531657 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1ecc004fb0 con 0x7f1ed80ffce0 2024-09-10T12:07:14.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.159+0000 7f1ed4ff9700 1 -- 172.21.15.144:0/287531657 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ecc00f480 con 0x7f1ed80ffce0 2024-09-10T12:07:14.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.159+0000 7f1ede1b2700 1 -- 172.21.15.144:0/287531657 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed81a2290 con 0x7f1ed80ffce0 2024-09-10T12:07:14.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.159+0000 7f1ede1b2700 1 -- 172.21.15.144:0/287531657 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed81076b0 con 0x7f1ed80ffce0 2024-09-10T12:07:14.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.160+0000 7f1ed4ff9700 1 -- 172.21.15.144:0/287531657 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1ecc022870 con 0x7f1ed80ffce0 2024-09-10T12:07:14.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.160+0000 7f1ed4ff9700 1 --2- 172.21.15.144:0/287531657 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1ec006e200 0x7f1ec00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:14.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.161+0000 7f1ed4ff9700 1 -- 172.21.15.144:0/287531657 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1ecc08f620 con 0x7f1ed80ffce0 2024-09-10T12:07:14.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.161+0000 7f1ed77fe700 1 --2- 172.21.15.144:0/287531657 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1ec006e200 0x7f1ec00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:14.164 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.161+0000 7f1ed77fe700 1 --2- 172.21.15.144:0/287531657 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1ec006e200 0x7f1ec00706b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800e300 tx=0x7f1ec8009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:14.167 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.165+0000 7f1ed4ff9700 1 -- 172.21.15.144:0/287531657 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1ecc05c700 con 0x7f1ed80ffce0 2024-09-10T12:07:14.377 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.375+0000 7f1ede1b2700 1 -- 172.21.15.144:0/287531657 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ed802cca0 con 0x7f1ec006e200 2024-09-10T12:07:14.387 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.385+0000 7f1ed4ff9700 1 -- 172.21.15.144:0/287531657 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f1ed802cca0 con 0x7f1ec006e200 2024-09-10T12:07:14.387 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:07:14.387 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (35m) 13m ago 45m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:07:14.387 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (46m) 13m ago 46m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:07:14.387 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (45m) 13m ago 45m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:07:14.387 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (33m) 13m ago 46m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:07:14.388 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (33m) 13m ago 45m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:07:14.388 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (35m) 13m ago 45m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:07:14.388 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (29m) 13m ago 40m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:07:14.388 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (29m) 13m ago 40m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:07:14.388 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (13m) 13m ago 40m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:07:14.388 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (29m) 13m ago 40m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:07:14.388 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (37m) 13m ago 47m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:07:14.388 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (36m) 13m ago 45m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:07:14.388 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (34m) 13m ago 47m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:07:14.388 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (33m) 13m ago 44m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:07:14.388 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (36m) 13m ago 46m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:07:14.388 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (36m) 13m ago 45m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:07:14.388 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (32m) 13m ago 44m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:07:14.388 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (32m) 13m ago 43m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:07:14.389 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (31m) 13m ago 43m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:07:14.389 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (30m) 13m ago 42m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:07:14.389 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (30m) 13m ago 42m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:07:14.389 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (29m) 13m ago 42m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:07:14.389 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (35m) 13m ago 45m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:07:14.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.387+0000 7f1ede1b2700 1 -- 172.21.15.144:0/287531657 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1ec006e200 msgr2=0x7f1ec00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:14.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.387+0000 7f1ede1b2700 1 --2- 172.21.15.144:0/287531657 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1ec006e200 0x7f1ec00706b0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1ec800e300 tx=0x7f1ec8009040 comp rx=0 tx=0).stop 2024-09-10T12:07:14.389 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.387+0000 7f1ede1b2700 1 -- 172.21.15.144:0/287531657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed80ffce0 msgr2=0x7f1ed8194f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:14.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.387+0000 7f1ede1b2700 1 --2- 172.21.15.144:0/287531657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed80ffce0 0x7f1ed8194f40 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00bda0 tx=0x7f1ecc00be80 comp rx=0 tx=0).stop 2024-09-10T12:07:14.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.387+0000 7f1ede1b2700 1 -- 172.21.15.144:0/287531657 shutdown_connections 2024-09-10T12:07:14.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.387+0000 7f1ede1b2700 1 --2- 172.21.15.144:0/287531657 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1ec006e200 0x7f1ec00706b0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.387+0000 7f1ede1b2700 1 --2- 172.21.15.144:0/287531657 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1ed80ff340 0x7f1ed8194a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.387+0000 7f1ede1b2700 1 --2- 172.21.15.144:0/287531657 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1ed80ffce0 0x7f1ed8194f40 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.387+0000 7f1ede1b2700 1 -- 172.21.15.144:0/287531657 >> 172.21.15.144:0/287531657 conn(0x7f1ed80fabe0 msgr2=0x7f1ed80fcfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:14.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.387+0000 7f1ede1b2700 1 -- 172.21.15.144:0/287531657 shutdown_connections 2024-09-10T12:07:14.390 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.387+0000 7f1ede1b2700 1 -- 172.21.15.144:0/287531657 wait complete. 2024-09-10T12:07:14.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.491+0000 7f290a1f8700 1 -- 172.21.15.144:0/2351578936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29040ffc60 msgr2=0x7f2904103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:14.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.491+0000 7f290a1f8700 1 --2- 172.21.15.144:0/2351578936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29040ffc60 0x7f2904103c30 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f28ec009910 tx=0x7f28ec009c20 comp rx=0 tx=0).stop 2024-09-10T12:07:14.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.491+0000 7f290a1f8700 1 -- 172.21.15.144:0/2351578936 shutdown_connections 2024-09-10T12:07:14.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.491+0000 7f290a1f8700 1 --2- 172.21.15.144:0/2351578936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29040ffc60 0x7f2904103c30 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.491+0000 7f290a1f8700 1 --2- 172.21.15.144:0/2351578936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29040ff2c0 0x7f29040ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.491+0000 7f290a1f8700 1 -- 172.21.15.144:0/2351578936 >> 172.21.15.144:0/2351578936 conn(0x7f29040fab80 msgr2=0x7f29040fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:14.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.491+0000 7f290a1f8700 1 -- 172.21.15.144:0/2351578936 shutdown_connections 2024-09-10T12:07:14.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.491+0000 7f290a1f8700 1 -- 172.21.15.144:0/2351578936 wait complete. 2024-09-10T12:07:14.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.491+0000 7f290a1f8700 1 Processor -- start 2024-09-10T12:07:14.494 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.492+0000 7f290a1f8700 1 -- start start 2024-09-10T12:07:14.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.492+0000 7f290a1f8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29040ff2c0 0x7f2904196ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:14.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.492+0000 7f290a1f8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29040ffc60 0x7f29041970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:14.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.492+0000 7f290a1f8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2904197700 con 0x7f29040ffc60 2024-09-10T12:07:14.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.492+0000 7f290a1f8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2904197870 con 0x7f29040ff2c0 2024-09-10T12:07:14.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.492+0000 7f2902ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29040ffc60 0x7f29041970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:14.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.492+0000 7f2902ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29040ffc60 0x7f29041970e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55530/0 (socket says 172.21.15.144:55530) 2024-09-10T12:07:14.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.492+0000 7f2902ffd700 1 -- 172.21.15.144:0/3346833607 learned_addr learned my addr 172.21.15.144:0/3346833607 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:07:14.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.492+0000 7f29037fe700 1 --2- 172.21.15.144:0/3346833607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29040ff2c0 0x7f2904196ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:14.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.492+0000 7f2902ffd700 1 -- 172.21.15.144:0/3346833607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29040ff2c0 msgr2=0x7f2904196ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:14.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.492+0000 7f2902ffd700 1 --2- 172.21.15.144:0/3346833607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29040ff2c0 0x7f2904196ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.492+0000 7f2902ffd700 1 -- 172.21.15.144:0/3346833607 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec0095f0 con 0x7f29040ffc60 2024-09-10T12:07:14.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.492+0000 7f2902ffd700 1 --2- 172.21.15.144:0/3346833607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29040ffc60 0x7f29041970e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f28ec005f50 tx=0x7f28ec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:14.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.493+0000 7f2900ff9700 1 -- 172.21.15.144:0/3346833607 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28ec01d070 con 0x7f29040ffc60 2024-09-10T12:07:14.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.493+0000 7f2900ff9700 1 -- 172.21.15.144:0/3346833607 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f28ec00f460 con 0x7f29040ffc60 2024-09-10T12:07:14.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.493+0000 7f2900ff9700 1 -- 172.21.15.144:0/3346833607 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28ec017630 con 0x7f29040ffc60 2024-09-10T12:07:14.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.493+0000 7f290a1f8700 1 -- 172.21.15.144:0/3346833607 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290419fc10 con 0x7f29040ffc60 2024-09-10T12:07:14.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.493+0000 7f290a1f8700 1 -- 172.21.15.144:0/3346833607 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29041a0080 con 0x7f29040ffc60 2024-09-10T12:07:14.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.494+0000 7f290a1f8700 1 -- 172.21.15.144:0/3346833607 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2904107630 con 0x7f29040ffc60 2024-09-10T12:07:14.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.494+0000 7f2900ff9700 1 -- 172.21.15.144:0/3346833607 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f28ec00f5d0 con 0x7f29040ffc60 2024-09-10T12:07:14.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.495+0000 7f2900ff9700 1 --2- 172.21.15.144:0/3346833607 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f28f006e200 0x7f28f00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:14.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.495+0000 7f2900ff9700 1 -- 172.21.15.144:0/3346833607 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f28ec08f5d0 con 0x7f29040ffc60 2024-09-10T12:07:14.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.495+0000 7f29037fe700 1 --2- 172.21.15.144:0/3346833607 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f28f006e200 0x7f28f00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:14.498 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.495+0000 7f29037fe700 1 --2- 172.21.15.144:0/3346833607 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f28f006e200 0x7f28f00706b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f28f40060b0 tx=0x7f28f4008640 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:14.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.499+0000 7f2900ff9700 1 -- 172.21.15.144:0/3346833607 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f28ec058a30 con 0x7f29040ffc60 2024-09-10T12:07:14.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:14 smithi144 ceph-mon[87436]: pgmap v1058: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:14.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:07:14.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:14 smithi144 ceph-mon[87436]: from='client.35912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:14.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.788+0000 7f290a1f8700 1 -- 172.21.15.144:0/3346833607 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f290404ea50 con 0x7f29040ffc60 2024-09-10T12:07:14.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.788+0000 7f2900ff9700 1 -- 172.21.15.144:0/3346833607 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f28ec05c0f0 con 0x7f29040ffc60 2024-09-10T12:07:14.791 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:07:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:07:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:07:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:07:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:07:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:07:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:07:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:07:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:07:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:07:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:07:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:07:14.791 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:07:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:07:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:07:14.792 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:07:14.792 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:07:14.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.790+0000 7f290a1f8700 1 -- 172.21.15.144:0/3346833607 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f28f006e200 msgr2=0x7f28f00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:14.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.790+0000 7f290a1f8700 1 --2- 172.21.15.144:0/3346833607 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f28f006e200 0x7f28f00706b0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f28f40060b0 tx=0x7f28f4008640 comp rx=0 tx=0).stop 2024-09-10T12:07:14.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.790+0000 7f290a1f8700 1 -- 172.21.15.144:0/3346833607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29040ffc60 msgr2=0x7f29041970e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:14.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.790+0000 7f290a1f8700 1 --2- 172.21.15.144:0/3346833607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29040ffc60 0x7f29041970e0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f28ec005f50 tx=0x7f28ec00be20 comp rx=0 tx=0).stop 2024-09-10T12:07:14.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.790+0000 7f290a1f8700 1 -- 172.21.15.144:0/3346833607 shutdown_connections 2024-09-10T12:07:14.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.790+0000 7f290a1f8700 1 --2- 172.21.15.144:0/3346833607 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f28f006e200 0x7f28f00706b0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.790+0000 7f290a1f8700 1 --2- 172.21.15.144:0/3346833607 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f29040ff2c0 0x7f2904196ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.790+0000 7f290a1f8700 1 --2- 172.21.15.144:0/3346833607 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f29040ffc60 0x7f29041970e0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.790+0000 7f290a1f8700 1 -- 172.21.15.144:0/3346833607 >> 172.21.15.144:0/3346833607 conn(0x7f29040fab80 msgr2=0x7f29040fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:14.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.790+0000 7f290a1f8700 1 -- 172.21.15.144:0/3346833607 shutdown_connections 2024-09-10T12:07:14.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.790+0000 7f290a1f8700 1 -- 172.21.15.144:0/3346833607 wait complete. 2024-09-10T12:07:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:14 smithi178 ceph-mon[65940]: pgmap v1058: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:14 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:07:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:14 smithi178 ceph-mon[65940]: from='client.35912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.895+0000 7fbc59802700 1 -- 172.21.15.144:0/126193485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc54100270 msgr2=0x7fbc54100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.895+0000 7fbc59802700 1 --2- 172.21.15.144:0/126193485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc54100270 0x7fbc54100640 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fbc48009910 tx=0x7fbc48009c20 comp rx=0 tx=0).stop 2024-09-10T12:07:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.895+0000 7fbc59802700 1 -- 172.21.15.144:0/126193485 shutdown_connections 2024-09-10T12:07:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.895+0000 7fbc59802700 1 --2- 172.21.15.144:0/126193485 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc54100c10 0x7fbc54104be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.895+0000 7fbc59802700 1 --2- 172.21.15.144:0/126193485 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc54100270 0x7fbc54100640 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.895+0000 7fbc59802700 1 -- 172.21.15.144:0/126193485 >> 172.21.15.144:0/126193485 conn(0x7fbc540fbb50 msgr2=0x7fbc540fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.895+0000 7fbc59802700 1 -- 172.21.15.144:0/126193485 shutdown_connections 2024-09-10T12:07:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.895+0000 7fbc59802700 1 -- 172.21.15.144:0/126193485 wait complete. 2024-09-10T12:07:14.898 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.896+0000 7fbc59802700 1 Processor -- start 2024-09-10T12:07:14.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.896+0000 7fbc59802700 1 -- start start 2024-09-10T12:07:14.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.896+0000 7fbc59802700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc54100270 0x7fbc5419ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:14.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.896+0000 7fbc59802700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc54100c10 0x7fbc5419bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:14.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.896+0000 7fbc59802700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc5419c610 con 0x7fbc54100270 2024-09-10T12:07:14.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.896+0000 7fbc59802700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc54195ac0 con 0x7fbc54100c10 2024-09-10T12:07:14.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.896+0000 7fbc52ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc54100270 0x7fbc5419ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:14.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.896+0000 7fbc52ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc54100270 0x7fbc5419ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55554/0 (socket says 172.21.15.144:55554) 2024-09-10T12:07:14.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.897+0000 7fbc52ffd700 1 -- 172.21.15.144:0/2392967842 learned_addr learned my addr 172.21.15.144:0/2392967842 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:07:14.900 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.897+0000 7fbc527fc700 1 --2- 172.21.15.144:0/2392967842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc54100c10 0x7fbc5419bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:14.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.897+0000 7fbc52ffd700 1 -- 172.21.15.144:0/2392967842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc54100c10 msgr2=0x7fbc5419bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:14.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.897+0000 7fbc52ffd700 1 --2- 172.21.15.144:0/2392967842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc54100c10 0x7fbc5419bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:14.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.897+0000 7fbc52ffd700 1 -- 172.21.15.144:0/2392967842 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc480095f0 con 0x7fbc54100270 2024-09-10T12:07:14.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.897+0000 7fbc52ffd700 1 --2- 172.21.15.144:0/2392967842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc54100270 0x7fbc5419ba40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbc48006010 tx=0x7fbc4800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:14.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.897+0000 7fbc58800700 1 -- 172.21.15.144:0/2392967842 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc4801d070 con 0x7fbc54100270 2024-09-10T12:07:14.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.897+0000 7fbc58800700 1 -- 172.21.15.144:0/2392967842 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbc4800f460 con 0x7fbc54100270 2024-09-10T12:07:14.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.897+0000 7fbc59802700 1 -- 172.21.15.144:0/2392967842 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc54195d40 con 0x7fbc54100270 2024-09-10T12:07:14.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.897+0000 7fbc58800700 1 -- 172.21.15.144:0/2392967842 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc48017610 con 0x7fbc54100270 2024-09-10T12:07:14.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.897+0000 7fbc59802700 1 -- 172.21.15.144:0/2392967842 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc54196230 con 0x7fbc54100270 2024-09-10T12:07:14.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.898+0000 7fbc59802700 1 -- 172.21.15.144:0/2392967842 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc54108580 con 0x7fbc54100270 2024-09-10T12:07:14.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.899+0000 7fbc58800700 1 -- 172.21.15.144:0/2392967842 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fbc4800f5d0 con 0x7fbc54100270 2024-09-10T12:07:14.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.899+0000 7fbc58800700 1 --2- 172.21.15.144:0/2392967842 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc3c06e200 0x7fbc3c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:14.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.899+0000 7fbc58800700 1 -- 172.21.15.144:0/2392967842 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fbc4808f600 con 0x7fbc54100270 2024-09-10T12:07:14.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.899+0000 7fbc527fc700 1 --2- 172.21.15.144:0/2392967842 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc3c06e200 0x7fbc3c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:14.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.900+0000 7fbc527fc700 1 --2- 172.21.15.144:0/2392967842 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc3c06e200 0x7fbc3c0706b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbc540fc310 tx=0x7fbc4400a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:14.906 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:14.904+0000 7fbc58800700 1 -- 172.21.15.144:0/2392967842 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fbc4805c510 con 0x7fbc54100270 2024-09-10T12:07:15.157 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.155+0000 7fbc59802700 1 -- 172.21.15.144:0/2392967842 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbc5404f2a0 con 0x7fbc54100270 2024-09-10T12:07:15.158 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.155+0000 7fbc58800700 1 -- 172.21.15.144:0/2392967842 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7fbc4805c0a0 con 0x7fbc54100270 2024-09-10T12:07:15.158 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:07:15.158 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:07:15.158 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:07:15.158 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:07:15.158 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:07:15.158 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:07:15.159 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:07:15.159 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:07:15.159 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:07:15.159 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:07:15.159 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:07:15.159 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:07:15.159 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:07:15.159 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:07:15.159 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:07:15.159 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:07:15.159 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:07:15.159 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:07:15.159 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:07:15.159 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:07:15.160 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:07:15.160 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:07:15.160 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:07:15.160 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:07:15.160 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:07:15.160 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:07:15.160 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:07:15.160 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:07:15.160 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:07:15.160 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:07:15.160 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:07:15.160 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:07:15.161 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:07:15.161 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:07:15.161 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:07:15.161 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:07:15.161 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:07:15.161 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:07:15.161 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:07:15.161 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:07:15.161 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:07:15.161 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:07:15.161 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:07:15.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.158+0000 7fbc59802700 1 -- 172.21.15.144:0/2392967842 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc3c06e200 msgr2=0x7fbc3c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:15.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.158+0000 7fbc59802700 1 --2- 172.21.15.144:0/2392967842 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc3c06e200 0x7fbc3c0706b0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fbc540fc310 tx=0x7fbc4400a300 comp rx=0 tx=0).stop 2024-09-10T12:07:15.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.158+0000 7fbc59802700 1 -- 172.21.15.144:0/2392967842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc54100270 msgr2=0x7fbc5419ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:15.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.158+0000 7fbc59802700 1 --2- 172.21.15.144:0/2392967842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc54100270 0x7fbc5419ba40 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fbc48006010 tx=0x7fbc4800bd70 comp rx=0 tx=0).stop 2024-09-10T12:07:15.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.158+0000 7fbc59802700 1 -- 172.21.15.144:0/2392967842 shutdown_connections 2024-09-10T12:07:15.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.158+0000 7fbc59802700 1 --2- 172.21.15.144:0/2392967842 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fbc3c06e200 0x7fbc3c0706b0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.162 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.158+0000 7fbc59802700 1 --2- 172.21.15.144:0/2392967842 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fbc54100270 0x7fbc5419ba40 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.158+0000 7fbc59802700 1 --2- 172.21.15.144:0/2392967842 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fbc54100c10 0x7fbc5419bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.158+0000 7fbc59802700 1 -- 172.21.15.144:0/2392967842 >> 172.21.15.144:0/2392967842 conn(0x7fbc540fbb50 msgr2=0x7fbc540fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:15.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.159+0000 7fbc59802700 1 -- 172.21.15.144:0/2392967842 shutdown_connections 2024-09-10T12:07:15.163 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.159+0000 7fbc59802700 1 -- 172.21.15.144:0/2392967842 wait complete. 2024-09-10T12:07:15.163 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:07:15.267 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.264+0000 7f1e0df0d700 1 -- 172.21.15.144:0/1619944125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e080ff2c0 msgr2=0x7f1e080ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:15.268 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.264+0000 7f1e0df0d700 1 --2- 172.21.15.144:0/1619944125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e080ff2c0 0x7f1e080ff690 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f1df8009910 tx=0x7f1df8009c20 comp rx=0 tx=0).stop 2024-09-10T12:07:15.268 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.264+0000 7f1e0df0d700 1 -- 172.21.15.144:0/1619944125 shutdown_connections 2024-09-10T12:07:15.268 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.264+0000 7f1e0df0d700 1 --2- 172.21.15.144:0/1619944125 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e080ffc60 0x7f1e08103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.268 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.264+0000 7f1e0df0d700 1 --2- 172.21.15.144:0/1619944125 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e080ff2c0 0x7f1e080ff690 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.268 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.264+0000 7f1e0df0d700 1 -- 172.21.15.144:0/1619944125 >> 172.21.15.144:0/1619944125 conn(0x7f1e080fab80 msgr2=0x7f1e080fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:15.268 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.264+0000 7f1e0df0d700 1 -- 172.21.15.144:0/1619944125 shutdown_connections 2024-09-10T12:07:15.268 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.264+0000 7f1e0df0d700 1 -- 172.21.15.144:0/1619944125 wait complete. 2024-09-10T12:07:15.268 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.265+0000 7f1e0df0d700 1 Processor -- start 2024-09-10T12:07:15.268 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.265+0000 7f1e0df0d700 1 -- start start 2024-09-10T12:07:15.268 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.265+0000 7f1e0df0d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e080ff2c0 0x7f1e08100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:15.268 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.265+0000 7f1e0df0d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e080ffc60 0x7f1e08101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:15.268 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.265+0000 7f1e0df0d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e08101750 con 0x7f1e080ff2c0 2024-09-10T12:07:15.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.265+0000 7f1e0df0d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e08101890 con 0x7f1e080ffc60 2024-09-10T12:07:15.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.265+0000 7f1e077fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e080ff2c0 0x7f1e08100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:15.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.265+0000 7f1e077fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e080ff2c0 0x7f1e08100cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55574/0 (socket says 172.21.15.144:55574) 2024-09-10T12:07:15.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.265+0000 7f1e077fe700 1 -- 172.21.15.144:0/530031550 learned_addr learned my addr 172.21.15.144:0/530031550 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:07:15.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.265+0000 7f1dfffff700 1 --2- 172.21.15.144:0/530031550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e080ffc60 0x7f1e08101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:15.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.265+0000 7f1e077fe700 1 -- 172.21.15.144:0/530031550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e080ffc60 msgr2=0x7f1e08101210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:15.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.265+0000 7f1e077fe700 1 --2- 172.21.15.144:0/530031550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e080ffc60 0x7f1e08101210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.269 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.265+0000 7f1e077fe700 1 -- 172.21.15.144:0/530031550 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1df80095f0 con 0x7f1e080ff2c0 2024-09-10T12:07:15.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.266+0000 7f1e077fe700 1 --2- 172.21.15.144:0/530031550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e080ff2c0 0x7f1e08100cd0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1df8006010 tx=0x7f1df800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:15.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.266+0000 7f1e057fa700 1 -- 172.21.15.144:0/530031550 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1df801d070 con 0x7f1e080ff2c0 2024-09-10T12:07:15.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.266+0000 7f1e0df0d700 1 -- 172.21.15.144:0/530031550 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e08101b10 con 0x7f1e080ff2c0 2024-09-10T12:07:15.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.266+0000 7f1e057fa700 1 -- 172.21.15.144:0/530031550 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1df8005dc0 con 0x7f1e080ff2c0 2024-09-10T12:07:15.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.266+0000 7f1e057fa700 1 -- 172.21.15.144:0/530031550 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1df8017400 con 0x7f1e080ff2c0 2024-09-10T12:07:15.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.266+0000 7f1e0df0d700 1 -- 172.21.15.144:0/530031550 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e081a2220 con 0x7f1e080ff2c0 2024-09-10T12:07:15.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.267+0000 7f1e0df0d700 1 -- 172.21.15.144:0/530031550 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e081085e0 con 0x7f1e080ff2c0 2024-09-10T12:07:15.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.267+0000 7f1e057fa700 1 -- 172.21.15.144:0/530031550 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1df8003a60 con 0x7f1e080ff2c0 2024-09-10T12:07:15.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.268+0000 7f1e057fa700 1 --2- 172.21.15.144:0/530031550 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1df406e200 0x7f1df40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:15.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.268+0000 7f1e057fa700 1 -- 172.21.15.144:0/530031550 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1df808f560 con 0x7f1e080ff2c0 2024-09-10T12:07:15.273 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.268+0000 7f1dfffff700 1 --2- 172.21.15.144:0/530031550 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1df406e200 0x7f1df40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:15.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.268+0000 7f1dfffff700 1 --2- 172.21.15.144:0/530031550 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1df406e200 0x7f1df40706b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1e08102470 tx=0x7f1df000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:15.274 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.272+0000 7f1e057fa700 1 -- 172.21.15.144:0/530031550 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1df80588e0 con 0x7f1e080ff2c0 2024-09-10T12:07:15.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.494+0000 7f1e0df0d700 1 -- 172.21.15.144:0/530031550 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e0802ccf0 con 0x7f1df406e200 2024-09-10T12:07:15.497 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.495+0000 7f1e057fa700 1 -- 172.21.15.144:0/530031550 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f1e0802ccf0 con 0x7f1df406e200 2024-09-10T12:07:15.498 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:07:15.498 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:07:15.498 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:07:15.498 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:07:15.498 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:07:15.498 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:07:15.498 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:07:15.498 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:07:15.498 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:07:15.498 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:07:15.498 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:07:15.499 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:07:15.499 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:07:15.499 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:07:15.499 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:07:15.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.498+0000 7f1e0df0d700 1 -- 172.21.15.144:0/530031550 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1df406e200 msgr2=0x7f1df40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:15.500 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.498+0000 7f1e0df0d700 1 --2- 172.21.15.144:0/530031550 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1df406e200 0x7f1df40706b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f1e08102470 tx=0x7f1df000b410 comp rx=0 tx=0).stop 2024-09-10T12:07:15.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.498+0000 7f1e0df0d700 1 -- 172.21.15.144:0/530031550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e080ff2c0 msgr2=0x7f1e08100cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:15.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.498+0000 7f1e0df0d700 1 --2- 172.21.15.144:0/530031550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e080ff2c0 0x7f1e08100cd0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f1df8006010 tx=0x7f1df800bd70 comp rx=0 tx=0).stop 2024-09-10T12:07:15.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.498+0000 7f1e0df0d700 1 -- 172.21.15.144:0/530031550 shutdown_connections 2024-09-10T12:07:15.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.498+0000 7f1e0df0d700 1 --2- 172.21.15.144:0/530031550 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1df406e200 0x7f1df40706b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.498+0000 7f1e0df0d700 1 --2- 172.21.15.144:0/530031550 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1e080ff2c0 0x7f1e08100cd0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.498+0000 7f1e0df0d700 1 --2- 172.21.15.144:0/530031550 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1e080ffc60 0x7f1e08101210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.498+0000 7f1e0df0d700 1 -- 172.21.15.144:0/530031550 >> 172.21.15.144:0/530031550 conn(0x7f1e080fab80 msgr2=0x7f1e08068ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:15.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.498+0000 7f1e0df0d700 1 -- 172.21.15.144:0/530031550 shutdown_connections 2024-09-10T12:07:15.501 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.498+0000 7f1e0df0d700 1 -- 172.21.15.144:0/530031550 wait complete. 2024-09-10T12:07:15.605 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.602+0000 7f2be6b90700 1 -- 172.21.15.144:0/836879056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be00ffd30 msgr2=0x7f2be0103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:15.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.602+0000 7f2be6b90700 1 --2- 172.21.15.144:0/836879056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be00ffd30 0x7f2be0103ce0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc009910 tx=0x7f2bcc009c20 comp rx=0 tx=0).stop 2024-09-10T12:07:15.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.602+0000 7f2be6b90700 1 -- 172.21.15.144:0/836879056 shutdown_connections 2024-09-10T12:07:15.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.602+0000 7f2be6b90700 1 --2- 172.21.15.144:0/836879056 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be00ffd30 0x7f2be0103ce0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.602+0000 7f2be6b90700 1 --2- 172.21.15.144:0/836879056 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00ff390 0x7f2be00ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.602+0000 7f2be6b90700 1 -- 172.21.15.144:0/836879056 >> 172.21.15.144:0/836879056 conn(0x7f2be0074f50 msgr2=0x7f2be0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:15.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.602+0000 7f2be6b90700 1 -- 172.21.15.144:0/836879056 shutdown_connections 2024-09-10T12:07:15.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.602+0000 7f2be6b90700 1 -- 172.21.15.144:0/836879056 wait complete. 2024-09-10T12:07:15.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.603+0000 7f2be6b90700 1 Processor -- start 2024-09-10T12:07:15.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.603+0000 7f2be6b90700 1 -- start start 2024-09-10T12:07:15.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.603+0000 7f2be6b90700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00ff390 0x7f2be0195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:15.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.603+0000 7f2be6b90700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be00ffd30 0x7f2be01962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:15.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.603+0000 7f2be6b90700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be0196920 con 0x7f2be00ffd30 2024-09-10T12:07:15.606 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.603+0000 7f2be6b90700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2be0196a60 con 0x7f2be00ff390 2024-09-10T12:07:15.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.603+0000 7f2bdffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be00ffd30 0x7f2be01962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:15.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.603+0000 7f2bdffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be00ffd30 0x7f2be01962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:55586/0 (socket says 172.21.15.144:55586) 2024-09-10T12:07:15.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.603+0000 7f2bdffff700 1 -- 172.21.15.144:0/2112809086 learned_addr learned my addr 172.21.15.144:0/2112809086 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:07:15.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.603+0000 7f2be492c700 1 --2- 172.21.15.144:0/2112809086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00ff390 0x7f2be0195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:15.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.603+0000 7f2bdffff700 1 -- 172.21.15.144:0/2112809086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00ff390 msgr2=0x7f2be0195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:15.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.603+0000 7f2bdffff700 1 --2- 172.21.15.144:0/2112809086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00ff390 0x7f2be0195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.604+0000 7f2bdffff700 1 -- 172.21.15.144:0/2112809086 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bcc0095f0 con 0x7f2be00ffd30 2024-09-10T12:07:15.607 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.604+0000 7f2bdffff700 1 --2- 172.21.15.144:0/2112809086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be00ffd30 0x7f2be01962d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00bda0 tx=0x7f2bcc00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:15.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.606+0000 7f2bddffb700 1 -- 172.21.15.144:0/2112809086 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bcc01d070 con 0x7f2be00ffd30 2024-09-10T12:07:15.608 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.606+0000 7f2bddffb700 1 -- 172.21.15.144:0/2112809086 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2bcc00f460 con 0x7f2be00ffd30 2024-09-10T12:07:15.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.606+0000 7f2bddffb700 1 -- 172.21.15.144:0/2112809086 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bcc017630 con 0x7f2be00ffd30 2024-09-10T12:07:15.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.606+0000 7f2be6b90700 1 -- 172.21.15.144:0/2112809086 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2be019a850 con 0x7f2be00ffd30 2024-09-10T12:07:15.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.606+0000 7f2be6b90700 1 -- 172.21.15.144:0/2112809086 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2be019ad40 con 0x7f2be00ffd30 2024-09-10T12:07:15.609 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.607+0000 7f2be6b90700 1 -- 172.21.15.144:0/2112809086 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2be004ea50 con 0x7f2be00ffd30 2024-09-10T12:07:15.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.608+0000 7f2bddffb700 1 -- 172.21.15.144:0/2112809086 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2bcc00fab0 con 0x7f2be00ffd30 2024-09-10T12:07:15.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.608+0000 7f2bddffb700 1 --2- 172.21.15.144:0/2112809086 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2bd006e200 0x7f2bd00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:15.610 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.608+0000 7f2bddffb700 1 -- 172.21.15.144:0/2112809086 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f2bcc08f4a0 con 0x7f2be00ffd30 2024-09-10T12:07:15.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.608+0000 7f2be492c700 1 --2- 172.21.15.144:0/2112809086 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2bd006e200 0x7f2bd00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:15.611 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.609+0000 7f2be492c700 1 --2- 172.21.15.144:0/2112809086 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2bd006e200 0x7f2bd00706b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400ac50 tx=0x7f2bd400a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:15.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.615+0000 7f2bddffb700 1 -- 172.21.15.144:0/2112809086 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2bcc058900 con 0x7f2be00ffd30 2024-09-10T12:07:15.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:15 smithi178 ceph-mon[65940]: from='client.35916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:15.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:15 smithi178 ceph-mon[65940]: from='client.35920 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:15.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:15 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3346833607' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:07:15.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:15 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2392967842' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:07:15.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.900+0000 7f2be6b90700 1 -- 172.21.15.144:0/2112809086 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f2be019b070 con 0x7f2be00ffd30 2024-09-10T12:07:15.903 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:15 smithi144 ceph-mon[87436]: from='client.35916 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:15.903 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:15 smithi144 ceph-mon[87436]: from='client.35920 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:15.903 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:15 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3346833607' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:07:15.903 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:15 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2392967842' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:07:15.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.900+0000 7f2bddffb700 1 -- 172.21.15.144:0/2112809086 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f2bcc05bfc0 con 0x7f2be00ffd30 2024-09-10T12:07:15.904 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:07:15.904 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:07:15.904 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:07:15.904 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:07:15.904 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:07:15.904 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:07:15.904 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:07:15.904 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:07:15.904 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:07:15.904 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:07:15.905 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:07:15.905 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:07:15.905 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:07:15.905 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:07:15.905 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:07:15.905 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:07:15.905 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:07:15.905 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:07:15.905 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:07:15.905 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:07:15.905 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:07:15.906 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:07:15.906 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:07:15.906 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:07:15.906 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:07:15.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.903+0000 7f2be6b90700 1 -- 172.21.15.144:0/2112809086 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2bd006e200 msgr2=0x7f2bd00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:15.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.903+0000 7f2be6b90700 1 --2- 172.21.15.144:0/2112809086 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2bd006e200 0x7f2bd00706b0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400ac50 tx=0x7f2bd400a380 comp rx=0 tx=0).stop 2024-09-10T12:07:15.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.903+0000 7f2be6b90700 1 -- 172.21.15.144:0/2112809086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be00ffd30 msgr2=0x7f2be01962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:15.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.903+0000 7f2be6b90700 1 --2- 172.21.15.144:0/2112809086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be00ffd30 0x7f2be01962d0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc00bda0 tx=0x7f2bcc00bdd0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.903+0000 7f2be6b90700 1 -- 172.21.15.144:0/2112809086 shutdown_connections 2024-09-10T12:07:15.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.903+0000 7f2be6b90700 1 --2- 172.21.15.144:0/2112809086 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f2bd006e200 0x7f2bd00706b0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.903+0000 7f2be6b90700 1 --2- 172.21.15.144:0/2112809086 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2be00ff390 0x7f2be0195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.903+0000 7f2be6b90700 1 --2- 172.21.15.144:0/2112809086 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2be00ffd30 0x7f2be01962d0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:15.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.903+0000 7f2be6b90700 1 -- 172.21.15.144:0/2112809086 >> 172.21.15.144:0/2112809086 conn(0x7f2be0074f50 msgr2=0x7f2be0103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:15.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.904+0000 7f2be6b90700 1 -- 172.21.15.144:0/2112809086 shutdown_connections 2024-09-10T12:07:15.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:15.904+0000 7f2be6b90700 1 -- 172.21.15.144:0/2112809086 wait complete. 2024-09-10T12:07:16.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:16 smithi178 ceph-mon[65940]: pgmap v1059: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:16.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:16 smithi178 ceph-mon[65940]: from='client.35932 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:16.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:16 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2112809086' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:07:16.897 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:16 smithi144 ceph-mon[87436]: pgmap v1059: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:16.897 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:16 smithi144 ceph-mon[87436]: from='client.35932 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:16.897 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:16 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2112809086' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:07:18.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:18 smithi178 ceph-mon[65940]: pgmap v1060: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:18 smithi144 ceph-mon[87436]: pgmap v1060: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:20 smithi178 ceph-mon[65940]: pgmap v1061: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:20.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:20 smithi144 ceph-mon[87436]: pgmap v1061: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:22 smithi144 ceph-mon[87436]: pgmap v1062: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:23.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:22 smithi178 ceph-mon[65940]: pgmap v1062: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:24.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:24 smithi144 ceph-mon[87436]: pgmap v1063: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:24 smithi178 ceph-mon[65940]: pgmap v1063: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:26.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:26 smithi144 ceph-mon[87436]: pgmap v1064: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:26 smithi178 ceph-mon[65940]: pgmap v1064: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:28.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:28 smithi144 ceph-mon[87436]: pgmap v1065: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:28 smithi178 ceph-mon[65940]: pgmap v1065: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:29.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:29 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:07:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:29 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:07:30.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:30 smithi178 ceph-mon[65940]: pgmap v1066: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:30.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:30 smithi144 ceph-mon[87436]: pgmap v1066: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:32.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:32 smithi144 ceph-mon[87436]: pgmap v1067: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:32 smithi178 ceph-mon[65940]: pgmap v1067: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:34.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:34 smithi144 ceph-mon[87436]: pgmap v1068: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:34 smithi178 ceph-mon[65940]: pgmap v1068: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:36.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:36 smithi144 ceph-mon[87436]: pgmap v1069: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:36 smithi178 ceph-mon[65940]: pgmap v1069: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:38.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:38 smithi144 ceph-mon[87436]: pgmap v1070: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:38 smithi178 ceph-mon[65940]: pgmap v1070: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:40.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:40 smithi178 ceph-mon[65940]: pgmap v1071: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:40.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:40 smithi144 ceph-mon[87436]: pgmap v1071: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:42 smithi144 ceph-mon[87436]: pgmap v1072: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:42 smithi178 ceph-mon[65940]: pgmap v1072: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:07:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:07:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:44 smithi144 ceph-mon[87436]: pgmap v1073: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:44 smithi178 ceph-mon[65940]: pgmap v1073: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:46.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.024+0000 7fb1cadf9700 1 -- 172.21.15.144:0/1888021241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c4100430 msgr2=0x7fb1c4100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:46.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.024+0000 7fb1cadf9700 1 --2- 172.21.15.144:0/1888021241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c4100430 0x7fb1c4100800 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc009910 tx=0x7fb1bc009c20 comp rx=0 tx=0).stop 2024-09-10T12:07:46.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.024+0000 7fb1cadf9700 1 -- 172.21.15.144:0/1888021241 shutdown_connections 2024-09-10T12:07:46.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.024+0000 7fb1cadf9700 1 --2- 172.21.15.144:0/1888021241 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1c4100d40 0x7fb1c41094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.026 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.024+0000 7fb1cadf9700 1 --2- 172.21.15.144:0/1888021241 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c4100430 0x7fb1c4100800 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.024+0000 7fb1cadf9700 1 -- 172.21.15.144:0/1888021241 >> 172.21.15.144:0/1888021241 conn(0x7fb1c40748b0 msgr2=0x7fb1c4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:46.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.024+0000 7fb1cadf9700 1 -- 172.21.15.144:0/1888021241 shutdown_connections 2024-09-10T12:07:46.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.024+0000 7fb1cadf9700 1 -- 172.21.15.144:0/1888021241 wait complete. 2024-09-10T12:07:46.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.025+0000 7fb1cadf9700 1 Processor -- start 2024-09-10T12:07:46.027 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.025+0000 7fb1cadf9700 1 -- start start 2024-09-10T12:07:46.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.025+0000 7fb1cadf9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c4100430 0x7fb1c4196a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:46.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.025+0000 7fb1cadf9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1c4100d40 0x7fb1c4196f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:46.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.025+0000 7fb1cadf9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4197630 con 0x7fb1c4100430 2024-09-10T12:07:46.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.025+0000 7fb1cadf9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4197770 con 0x7fb1c4100d40 2024-09-10T12:07:46.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.025+0000 7fb1c8b95700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c4100430 0x7fb1c4196a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:46.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.025+0000 7fb1c8b95700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c4100430 0x7fb1c4196a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56858/0 (socket says 172.21.15.144:56858) 2024-09-10T12:07:46.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.025+0000 7fb1c8b95700 1 -- 172.21.15.144:0/3448130389 learned_addr learned my addr 172.21.15.144:0/3448130389 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:07:46.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.026+0000 7fb1c3fff700 1 --2- 172.21.15.144:0/3448130389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1c4100d40 0x7fb1c4196f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:46.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.026+0000 7fb1c8b95700 1 -- 172.21.15.144:0/3448130389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1c4100d40 msgr2=0x7fb1c4196f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:46.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.026+0000 7fb1c8b95700 1 --2- 172.21.15.144:0/3448130389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1c4100d40 0x7fb1c4196f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.026+0000 7fb1c8b95700 1 -- 172.21.15.144:0/3448130389 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1bc0095f0 con 0x7fb1c4100430 2024-09-10T12:07:46.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.026+0000 7fb1c3fff700 1 --2- 172.21.15.144:0/3448130389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1c4100d40 0x7fb1c4196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:07:46.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.026+0000 7fb1c8b95700 1 --2- 172.21.15.144:0/3448130389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c4100430 0x7fb1c4196a10 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc006010 tx=0x7fb1bc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:46.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.026+0000 7fb1c1ffb700 1 -- 172.21.15.144:0/3448130389 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1bc01d070 con 0x7fb1c4100430 2024-09-10T12:07:46.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.026+0000 7fb1c1ffb700 1 -- 172.21.15.144:0/3448130389 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb1bc00f460 con 0x7fb1c4100430 2024-09-10T12:07:46.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.026+0000 7fb1cadf9700 1 -- 172.21.15.144:0/3448130389 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c419fa80 con 0x7fb1c4100430 2024-09-10T12:07:46.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.026+0000 7fb1c1ffb700 1 -- 172.21.15.144:0/3448130389 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1bc017630 con 0x7fb1c4100430 2024-09-10T12:07:46.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.026+0000 7fb1cadf9700 1 -- 172.21.15.144:0/3448130389 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c41a0020 con 0x7fb1c4100430 2024-09-10T12:07:46.030 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.027+0000 7fb1cadf9700 1 -- 172.21.15.144:0/3448130389 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c4106ba0 con 0x7fb1c4100430 2024-09-10T12:07:46.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.028+0000 7fb1c1ffb700 1 -- 172.21.15.144:0/3448130389 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fb1bc00f5d0 con 0x7fb1c4100430 2024-09-10T12:07:46.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.028+0000 7fb1c1ffb700 1 --2- 172.21.15.144:0/3448130389 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb1b006e200 0x7fb1b00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:46.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.028+0000 7fb1c1ffb700 1 -- 172.21.15.144:0/3448130389 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fb1bc08f4a0 con 0x7fb1c4100430 2024-09-10T12:07:46.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.028+0000 7fb1c3fff700 1 --2- 172.21.15.144:0/3448130389 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb1b006e200 0x7fb1b00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:46.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.029+0000 7fb1c3fff700 1 --2- 172.21.15.144:0/3448130389 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb1b006e200 0x7fb1b00706b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4100400 tx=0x7fb1b4009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:46.034 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.033+0000 7fb1c1ffb700 1 -- 172.21.15.144:0/3448130389 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fb1bc058880 con 0x7fb1c4100430 2024-09-10T12:07:46.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.256+0000 7fb1cadf9700 1 -- 172.21.15.144:0/3448130389 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1c402cd40 con 0x7fb1b006e200 2024-09-10T12:07:46.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.257+0000 7fb1c1ffb700 1 -- 172.21.15.144:0/3448130389 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7fb1c402cd40 con 0x7fb1b006e200 2024-09-10T12:07:46.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.259+0000 7fb1cadf9700 1 -- 172.21.15.144:0/3448130389 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb1b006e200 msgr2=0x7fb1b00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:46.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.259+0000 7fb1cadf9700 1 --2- 172.21.15.144:0/3448130389 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb1b006e200 0x7fb1b00706b0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fb1c4100400 tx=0x7fb1b4009040 comp rx=0 tx=0).stop 2024-09-10T12:07:46.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.259+0000 7fb1cadf9700 1 -- 172.21.15.144:0/3448130389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c4100430 msgr2=0x7fb1c4196a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:46.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.259+0000 7fb1cadf9700 1 --2- 172.21.15.144:0/3448130389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c4100430 0x7fb1c4196a10 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fb1bc006010 tx=0x7fb1bc00be20 comp rx=0 tx=0).stop 2024-09-10T12:07:46.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.259+0000 7fb1cadf9700 1 -- 172.21.15.144:0/3448130389 shutdown_connections 2024-09-10T12:07:46.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.259+0000 7fb1cadf9700 1 --2- 172.21.15.144:0/3448130389 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fb1b006e200 0x7fb1b00706b0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.259+0000 7fb1cadf9700 1 --2- 172.21.15.144:0/3448130389 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fb1c4100430 0x7fb1c4196a10 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.259+0000 7fb1cadf9700 1 --2- 172.21.15.144:0/3448130389 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fb1c4100d40 0x7fb1c4196f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.259+0000 7fb1cadf9700 1 -- 172.21.15.144:0/3448130389 >> 172.21.15.144:0/3448130389 conn(0x7fb1c40748b0 msgr2=0x7fb1c4103ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:46.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.259+0000 7fb1cadf9700 1 -- 172.21.15.144:0/3448130389 shutdown_connections 2024-09-10T12:07:46.261 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.259+0000 7fb1cadf9700 1 -- 172.21.15.144:0/3448130389 wait complete. 2024-09-10T12:07:46.271 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:07:46.368 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.365+0000 7f439fbd2700 1 -- 172.21.15.144:0/2363081094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4398100c90 msgr2=0x7f43981093f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:46.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.365+0000 7f439fbd2700 1 --2- 172.21.15.144:0/2363081094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4398100c90 0x7f43981093f0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4388009910 tx=0x7f4388009c20 comp rx=0 tx=0).stop 2024-09-10T12:07:46.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.365+0000 7f439fbd2700 1 -- 172.21.15.144:0/2363081094 shutdown_connections 2024-09-10T12:07:46.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.365+0000 7f439fbd2700 1 --2- 172.21.15.144:0/2363081094 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4398100c90 0x7f43981093f0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.365+0000 7f439fbd2700 1 --2- 172.21.15.144:0/2363081094 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4398100380 0x7f4398100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.365+0000 7f439fbd2700 1 -- 172.21.15.144:0/2363081094 >> 172.21.15.144:0/2363081094 conn(0x7f4398074850 msgr2=0x7f4398074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:46.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.365+0000 7f439fbd2700 1 -- 172.21.15.144:0/2363081094 shutdown_connections 2024-09-10T12:07:46.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.365+0000 7f439fbd2700 1 -- 172.21.15.144:0/2363081094 wait complete. 2024-09-10T12:07:46.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.366+0000 7f439fbd2700 1 Processor -- start 2024-09-10T12:07:46.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.366+0000 7f439fbd2700 1 -- start start 2024-09-10T12:07:46.369 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.366+0000 7f439fbd2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4398100380 0x7f4398101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:46.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.366+0000 7f439fbd2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4398100c90 0x7f43981019a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:46.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.366+0000 7f439fbd2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4398101ee0 con 0x7f4398100c90 2024-09-10T12:07:46.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.366+0000 7f439fbd2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4398102020 con 0x7f4398100380 2024-09-10T12:07:46.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.366+0000 7f439d16d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4398100c90 0x7f43981019a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:46.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.366+0000 7f439d16d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4398100c90 0x7f43981019a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56874/0 (socket says 172.21.15.144:56874) 2024-09-10T12:07:46.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.366+0000 7f439d16d700 1 -- 172.21.15.144:0/796430735 learned_addr learned my addr 172.21.15.144:0/796430735 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:07:46.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.366+0000 7f439d96e700 1 --2- 172.21.15.144:0/796430735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4398100380 0x7f4398101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:46.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.366+0000 7f439d16d700 1 -- 172.21.15.144:0/796430735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4398100380 msgr2=0x7f4398101460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:46.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.366+0000 7f439d16d700 1 --2- 172.21.15.144:0/796430735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4398100380 0x7f4398101460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.366+0000 7f439d16d700 1 -- 172.21.15.144:0/796430735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43880095f0 con 0x7f4398100c90 2024-09-10T12:07:46.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.366+0000 7f439d16d700 1 --2- 172.21.15.144:0/796430735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4398100c90 0x7f43981019a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f4388006010 tx=0x7f438800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:46.370 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.367+0000 7f438effd700 1 -- 172.21.15.144:0/796430735 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f438801d070 con 0x7f4398100c90 2024-09-10T12:07:46.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.367+0000 7f438effd700 1 -- 172.21.15.144:0/796430735 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f438800f460 con 0x7f4398100c90 2024-09-10T12:07:46.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.367+0000 7f439fbd2700 1 -- 172.21.15.144:0/796430735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43981a1e30 con 0x7f4398100c90 2024-09-10T12:07:46.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.367+0000 7f438effd700 1 -- 172.21.15.144:0/796430735 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4388017610 con 0x7f4398100c90 2024-09-10T12:07:46.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.367+0000 7f439fbd2700 1 -- 172.21.15.144:0/796430735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43981a2240 con 0x7f4398100c90 2024-09-10T12:07:46.371 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.368+0000 7f439fbd2700 1 -- 172.21.15.144:0/796430735 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4398107d60 con 0x7f4398100c90 2024-09-10T12:07:46.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.369+0000 7f438effd700 1 -- 172.21.15.144:0/796430735 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f438800f5d0 con 0x7f4398100c90 2024-09-10T12:07:46.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.369+0000 7f438effd700 1 --2- 172.21.15.144:0/796430735 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f438406e200 0x7f43840706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:46.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.369+0000 7f438effd700 1 -- 172.21.15.144:0/796430735 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f438808f4f0 con 0x7f4398100c90 2024-09-10T12:07:46.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.369+0000 7f439d96e700 1 --2- 172.21.15.144:0/796430735 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f438406e200 0x7f43840706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:46.372 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.370+0000 7f439d96e700 1 --2- 172.21.15.144:0/796430735 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f438406e200 0x7f43840706b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f43940060b0 tx=0x7f439400c040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:46.375 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.374+0000 7f438effd700 1 -- 172.21.15.144:0/796430735 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4388058950 con 0x7f4398100c90 2024-09-10T12:07:46.595 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.593+0000 7f439fbd2700 1 -- 172.21.15.144:0/796430735 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f439802cd40 con 0x7f438406e200 2024-09-10T12:07:46.596 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.595+0000 7f438effd700 1 -- 172.21.15.144:0/796430735 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f439802cd40 con 0x7f438406e200 2024-09-10T12:07:46.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.597+0000 7f439fbd2700 1 -- 172.21.15.144:0/796430735 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f438406e200 msgr2=0x7f43840706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:46.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.597+0000 7f439fbd2700 1 --2- 172.21.15.144:0/796430735 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f438406e200 0x7f43840706b0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f43940060b0 tx=0x7f439400c040 comp rx=0 tx=0).stop 2024-09-10T12:07:46.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.597+0000 7f439fbd2700 1 -- 172.21.15.144:0/796430735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4398100c90 msgr2=0x7f43981019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:46.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.597+0000 7f439fbd2700 1 --2- 172.21.15.144:0/796430735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4398100c90 0x7f43981019a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f4388006010 tx=0x7f438800bd70 comp rx=0 tx=0).stop 2024-09-10T12:07:46.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.597+0000 7f439fbd2700 1 -- 172.21.15.144:0/796430735 shutdown_connections 2024-09-10T12:07:46.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.597+0000 7f439fbd2700 1 --2- 172.21.15.144:0/796430735 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f438406e200 0x7f43840706b0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.597+0000 7f439fbd2700 1 --2- 172.21.15.144:0/796430735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4398100380 0x7f4398101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.597+0000 7f439fbd2700 1 --2- 172.21.15.144:0/796430735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4398100c90 0x7f43981019a0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.597+0000 7f439fbd2700 1 -- 172.21.15.144:0/796430735 >> 172.21.15.144:0/796430735 conn(0x7f4398074850 msgr2=0x7f43980ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:46.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.597+0000 7f439fbd2700 1 -- 172.21.15.144:0/796430735 shutdown_connections 2024-09-10T12:07:46.599 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.597+0000 7f439fbd2700 1 -- 172.21.15.144:0/796430735 wait complete. 2024-09-10T12:07:46.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.705+0000 7f23d3fc1700 1 -- 172.21.15.144:0/2497801535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23cc1003d0 msgr2=0x7f23cc1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:46.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.705+0000 7f23d3fc1700 1 --2- 172.21.15.144:0/2497801535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23cc1003d0 0x7f23cc1007a0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f23c8009910 tx=0x7f23c8009c20 comp rx=0 tx=0).stop 2024-09-10T12:07:46.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.705+0000 7f23d3fc1700 1 -- 172.21.15.144:0/2497801535 shutdown_connections 2024-09-10T12:07:46.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.705+0000 7f23d3fc1700 1 --2- 172.21.15.144:0/2497801535 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23cc100d70 0x7f23cc104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.705+0000 7f23d3fc1700 1 --2- 172.21.15.144:0/2497801535 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23cc1003d0 0x7f23cc1007a0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.705+0000 7f23d3fc1700 1 -- 172.21.15.144:0/2497801535 >> 172.21.15.144:0/2497801535 conn(0x7f23cc0fbc90 msgr2=0x7f23cc0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:46.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.705+0000 7f23d3fc1700 1 -- 172.21.15.144:0/2497801535 shutdown_connections 2024-09-10T12:07:46.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.705+0000 7f23d3fc1700 1 -- 172.21.15.144:0/2497801535 wait complete. 2024-09-10T12:07:46.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.706+0000 7f23d3fc1700 1 Processor -- start 2024-09-10T12:07:46.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.706+0000 7f23d3fc1700 1 -- start start 2024-09-10T12:07:46.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.706+0000 7f23d3fc1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23cc1003d0 0x7f23cc195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:46.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.706+0000 7f23d3fc1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23cc100d70 0x7f23cc1962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:46.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.706+0000 7f23d3fc1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc196980 con 0x7f23cc100d70 2024-09-10T12:07:46.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.706+0000 7f23d3fc1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc19a710 con 0x7f23cc1003d0 2024-09-10T12:07:46.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.706+0000 7f23d155c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23cc100d70 0x7f23cc1962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:46.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.706+0000 7f23d155c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23cc100d70 0x7f23cc1962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56894/0 (socket says 172.21.15.144:56894) 2024-09-10T12:07:46.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.706+0000 7f23d155c700 1 -- 172.21.15.144:0/786210090 learned_addr learned my addr 172.21.15.144:0/786210090 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:07:46.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.707+0000 7f23d1d5d700 1 --2- 172.21.15.144:0/786210090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23cc1003d0 0x7f23cc195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:46.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.707+0000 7f23d155c700 1 -- 172.21.15.144:0/786210090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23cc1003d0 msgr2=0x7f23cc195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:46.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.707+0000 7f23d155c700 1 --2- 172.21.15.144:0/786210090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23cc1003d0 0x7f23cc195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.707+0000 7f23d155c700 1 -- 172.21.15.144:0/786210090 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c80095f0 con 0x7f23cc100d70 2024-09-10T12:07:46.713 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.707+0000 7f23d155c700 1 --2- 172.21.15.144:0/786210090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23cc100d70 0x7f23cc1962a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00d900 tx=0x7f23bc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:46.713 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.707+0000 7f23c2ffd700 1 -- 172.21.15.144:0/786210090 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23bc009900 con 0x7f23cc100d70 2024-09-10T12:07:46.713 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.707+0000 7f23c2ffd700 1 -- 172.21.15.144:0/786210090 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23bc010460 con 0x7f23cc100d70 2024-09-10T12:07:46.713 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.707+0000 7f23d3fc1700 1 -- 172.21.15.144:0/786210090 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc19a9f0 con 0x7f23cc100d70 2024-09-10T12:07:46.713 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.708+0000 7f23c2ffd700 1 -- 172.21.15.144:0/786210090 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23bc00f5f0 con 0x7f23cc100d70 2024-09-10T12:07:46.713 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.708+0000 7f23d3fc1700 1 -- 172.21.15.144:0/786210090 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc19af40 con 0x7f23cc100d70 2024-09-10T12:07:46.713 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.708+0000 7f23c2ffd700 1 -- 172.21.15.144:0/786210090 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f23bc00f870 con 0x7f23cc100d70 2024-09-10T12:07:46.713 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.709+0000 7f23c2ffd700 1 --2- 172.21.15.144:0/786210090 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f23b806e200 0x7f23b80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:46.713 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.709+0000 7f23c2ffd700 1 -- 172.21.15.144:0/786210090 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f23bc08d1a0 con 0x7f23cc100d70 2024-09-10T12:07:46.713 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.709+0000 7f23d3fc1700 1 -- 172.21.15.144:0/786210090 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc1086e0 con 0x7f23cc100d70 2024-09-10T12:07:46.713 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.709+0000 7f23d1d5d700 1 --2- 172.21.15.144:0/786210090 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f23b806e200 0x7f23b80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:46.713 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.709+0000 7f23d1d5d700 1 --2- 172.21.15.144:0/786210090 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f23b806e200 0x7f23b80706b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f23c8006010 tx=0x7f23c801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:46.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.715+0000 7f23c2ffd700 1 -- 172.21.15.144:0/786210090 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f23bc092060 con 0x7f23cc100d70 2024-09-10T12:07:46.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.924+0000 7f23d3fc1700 1 -- 172.21.15.144:0/786210090 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f23cc19b240 con 0x7f23b806e200 2024-09-10T12:07:46.927 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:46 smithi144 ceph-mon[87436]: pgmap v1074: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:46.935 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.934+0000 7f23c2ffd700 1 -- 172.21.15.144:0/786210090 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f23cc19b240 con 0x7f23b806e200 2024-09-10T12:07:46.936 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:07:46.936 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (36m) 14m ago 46m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:07:46.936 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (46m) 14m ago 46m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:07:46.936 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (45m) 14m ago 45m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:07:46.936 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (33m) 14m ago 46m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:07:46.936 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (33m) 14m ago 45m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:07:46.936 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (35m) 14m ago 46m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:07:46.937 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (30m) 14m ago 41m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:07:46.937 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (29m) 14m ago 41m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:07:46.937 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (14m) 14m ago 40m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:07:46.937 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (29m) 14m ago 41m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:07:46.937 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (37m) 14m ago 47m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:07:46.937 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (36m) 14m ago 45m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:07:46.938 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (34m) 14m ago 48m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:07:46.938 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (34m) 14m ago 45m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:07:46.938 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (36m) 14m ago 46m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:07:46.938 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (36m) 14m ago 45m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:07:46.938 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (33m) 14m ago 44m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:07:46.938 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (32m) 14m ago 44m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:07:46.938 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (32m) 14m ago 43m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:07:46.939 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (31m) 14m ago 43m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:07:46.939 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (30m) 14m ago 42m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:07:46.939 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (30m) 14m ago 42m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:07:46.939 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (36m) 14m ago 46m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:07:46.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.936+0000 7f23d3fc1700 1 -- 172.21.15.144:0/786210090 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f23b806e200 msgr2=0x7f23b80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:46.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.936+0000 7f23d3fc1700 1 --2- 172.21.15.144:0/786210090 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f23b806e200 0x7f23b80706b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f23c8006010 tx=0x7f23c801a040 comp rx=0 tx=0).stop 2024-09-10T12:07:46.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.936+0000 7f23d3fc1700 1 -- 172.21.15.144:0/786210090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23cc100d70 msgr2=0x7f23cc1962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:46.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.936+0000 7f23d3fc1700 1 --2- 172.21.15.144:0/786210090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23cc100d70 0x7f23cc1962a0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00d900 tx=0x7f23bc00dc10 comp rx=0 tx=0).stop 2024-09-10T12:07:46.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.936+0000 7f23d3fc1700 1 -- 172.21.15.144:0/786210090 shutdown_connections 2024-09-10T12:07:46.941 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.936+0000 7f23d3fc1700 1 --2- 172.21.15.144:0/786210090 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f23b806e200 0x7f23b80706b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.936+0000 7f23d3fc1700 1 --2- 172.21.15.144:0/786210090 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f23cc1003d0 0x7f23cc195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.936+0000 7f23d3fc1700 1 --2- 172.21.15.144:0/786210090 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f23cc100d70 0x7f23cc1962a0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:46.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.936+0000 7f23d3fc1700 1 -- 172.21.15.144:0/786210090 >> 172.21.15.144:0/786210090 conn(0x7f23cc0fbc90 msgr2=0x7f23cc0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:46.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.936+0000 7f23d3fc1700 1 -- 172.21.15.144:0/786210090 shutdown_connections 2024-09-10T12:07:46.942 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:46.936+0000 7f23d3fc1700 1 -- 172.21.15.144:0/786210090 wait complete. 2024-09-10T12:07:47.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.040+0000 7f474f0a0700 1 -- 172.21.15.144:0/2892068987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47480ff390 msgr2=0x7f47480ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:47.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.040+0000 7f474f0a0700 1 --2- 172.21.15.144:0/2892068987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47480ff390 0x7f47480ff760 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f473c009910 tx=0x7f473c009c20 comp rx=0 tx=0).stop 2024-09-10T12:07:47.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.040+0000 7f474f0a0700 1 -- 172.21.15.144:0/2892068987 shutdown_connections 2024-09-10T12:07:47.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.040+0000 7f474f0a0700 1 --2- 172.21.15.144:0/2892068987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47480ffd30 0x7f4748103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.040+0000 7f474f0a0700 1 --2- 172.21.15.144:0/2892068987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47480ff390 0x7f47480ff760 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.040+0000 7f474f0a0700 1 -- 172.21.15.144:0/2892068987 >> 172.21.15.144:0/2892068987 conn(0x7f4748074f50 msgr2=0x7f4748075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:47.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.040+0000 7f474f0a0700 1 -- 172.21.15.144:0/2892068987 shutdown_connections 2024-09-10T12:07:47.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.040+0000 7f474f0a0700 1 -- 172.21.15.144:0/2892068987 wait complete. 2024-09-10T12:07:47.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f474f0a0700 1 Processor -- start 2024-09-10T12:07:47.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f474f0a0700 1 -- start start 2024-09-10T12:07:47.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f474f0a0700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47480ff390 0x7f4748196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:47.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f474f0a0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47480ffd30 0x7f4748197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:47.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f474f0a0700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4748197860 con 0x7f47480ffd30 2024-09-10T12:07:47.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f474f0a0700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47481979d0 con 0x7f47480ff390 2024-09-10T12:07:47.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f4747fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47480ffd30 0x7f4748197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:47.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f4747fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47480ffd30 0x7f4748197180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56914/0 (socket says 172.21.15.144:56914) 2024-09-10T12:07:47.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f4747fff700 1 -- 172.21.15.144:0/3002866186 learned_addr learned my addr 172.21.15.144:0/3002866186 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:07:47.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f474ce3c700 1 --2- 172.21.15.144:0/3002866186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47480ff390 0x7f4748196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:47.044 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f4747fff700 1 -- 172.21.15.144:0/3002866186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47480ff390 msgr2=0x7f4748196c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:47.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f4747fff700 1 --2- 172.21.15.144:0/3002866186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47480ff390 0x7f4748196c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f4747fff700 1 -- 172.21.15.144:0/3002866186 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f473c0095f0 con 0x7f47480ffd30 2024-09-10T12:07:47.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f474ce3c700 1 --2- 172.21.15.144:0/3002866186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47480ff390 0x7f4748196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:07:47.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f4747fff700 1 --2- 172.21.15.144:0/3002866186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47480ffd30 0x7f4748197180 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f473400d900 tx=0x7f473400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:47.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.041+0000 7f4745ffb700 1 -- 172.21.15.144:0/3002866186 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4734009900 con 0x7f47480ffd30 2024-09-10T12:07:47.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.042+0000 7f4745ffb700 1 -- 172.21.15.144:0/3002866186 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4734010460 con 0x7f47480ffd30 2024-09-10T12:07:47.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.042+0000 7f4745ffb700 1 -- 172.21.15.144:0/3002866186 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f473400f5f0 con 0x7f47480ffd30 2024-09-10T12:07:47.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.042+0000 7f474f0a0700 1 -- 172.21.15.144:0/3002866186 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f474819fcb0 con 0x7f47480ffd30 2024-09-10T12:07:47.045 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.042+0000 7f474f0a0700 1 -- 172.21.15.144:0/3002866186 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47481a0200 con 0x7f47480ffd30 2024-09-10T12:07:47.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.043+0000 7f474f0a0700 1 -- 172.21.15.144:0/3002866186 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4748107650 con 0x7f47480ffd30 2024-09-10T12:07:47.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.043+0000 7f4745ffb700 1 -- 172.21.15.144:0/3002866186 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f47340105d0 con 0x7f47480ffd30 2024-09-10T12:07:47.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.043+0000 7f4745ffb700 1 --2- 172.21.15.144:0/3002866186 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f473806e200 0x7f47380706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:47.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.044+0000 7f4745ffb700 1 -- 172.21.15.144:0/3002866186 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f473408de40 con 0x7f47480ffd30 2024-09-10T12:07:47.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.044+0000 7f474ce3c700 1 --2- 172.21.15.144:0/3002866186 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f473806e200 0x7f47380706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:47.046 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.044+0000 7f474ce3c700 1 --2- 172.21.15.144:0/3002866186 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f473806e200 0x7f47380706b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f473c00b5c0 tx=0x7f473c0053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:47.049 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.048+0000 7f4745ffb700 1 -- 172.21.15.144:0/3002866186 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f47340572a0 con 0x7f47480ffd30 2024-09-10T12:07:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:46 smithi178 ceph-mon[65940]: pgmap v1074: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:47.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.335+0000 7f474f0a0700 1 -- 172.21.15.144:0/3002866186 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4748107030 con 0x7f47480ffd30 2024-09-10T12:07:47.337 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.336+0000 7f4745ffb700 1 -- 172.21.15.144:0/3002866186 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f4734017290 con 0x7f47480ffd30 2024-09-10T12:07:47.337 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:07:47.337 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:07:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:07:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:07:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:07:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:07:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:07:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:07:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:07:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:07:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:07:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:07:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:07:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:07:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:07:47.338 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:07:47.339 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:07:47.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.338+0000 7f474f0a0700 1 -- 172.21.15.144:0/3002866186 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f473806e200 msgr2=0x7f47380706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:47.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.338+0000 7f474f0a0700 1 --2- 172.21.15.144:0/3002866186 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f473806e200 0x7f47380706b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f473c00b5c0 tx=0x7f473c0053a0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.338+0000 7f474f0a0700 1 -- 172.21.15.144:0/3002866186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47480ffd30 msgr2=0x7f4748197180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:47.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.338+0000 7f474f0a0700 1 --2- 172.21.15.144:0/3002866186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47480ffd30 0x7f4748197180 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f473400d900 tx=0x7f473400dcc0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.338+0000 7f474f0a0700 1 -- 172.21.15.144:0/3002866186 shutdown_connections 2024-09-10T12:07:47.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.338+0000 7f474f0a0700 1 --2- 172.21.15.144:0/3002866186 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f473806e200 0x7f47380706b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.338+0000 7f474f0a0700 1 --2- 172.21.15.144:0/3002866186 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f47480ff390 0x7f4748196c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.338+0000 7f474f0a0700 1 --2- 172.21.15.144:0/3002866186 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f47480ffd30 0x7f4748197180 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.338+0000 7f474f0a0700 1 -- 172.21.15.144:0/3002866186 >> 172.21.15.144:0/3002866186 conn(0x7f4748074f50 msgr2=0x7f47480fd680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:47.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.338+0000 7f474f0a0700 1 -- 172.21.15.144:0/3002866186 shutdown_connections 2024-09-10T12:07:47.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.338+0000 7f474f0a0700 1 -- 172.21.15.144:0/3002866186 wait complete. 2024-09-10T12:07:47.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.445+0000 7fef14950700 1 -- 172.21.15.144:0/4227390671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef100ff390 msgr2=0x7fef100ff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:47.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.445+0000 7fef14950700 1 --2- 172.21.15.144:0/4227390671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef100ff390 0x7fef100ff760 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fef00009910 tx=0x7fef00009c20 comp rx=0 tx=0).stop 2024-09-10T12:07:47.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.445+0000 7fef14950700 1 -- 172.21.15.144:0/4227390671 shutdown_connections 2024-09-10T12:07:47.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.445+0000 7fef14950700 1 --2- 172.21.15.144:0/4227390671 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef100ffd30 0x7fef10103ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.446 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.445+0000 7fef14950700 1 --2- 172.21.15.144:0/4227390671 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef100ff390 0x7fef100ff760 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.445+0000 7fef14950700 1 -- 172.21.15.144:0/4227390671 >> 172.21.15.144:0/4227390671 conn(0x7fef10074f50 msgr2=0x7fef10075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:47.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.445+0000 7fef14950700 1 -- 172.21.15.144:0/4227390671 shutdown_connections 2024-09-10T12:07:47.447 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.445+0000 7fef14950700 1 -- 172.21.15.144:0/4227390671 wait complete. 2024-09-10T12:07:47.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.445+0000 7fef14950700 1 Processor -- start 2024-09-10T12:07:47.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.446+0000 7fef14950700 1 -- start start 2024-09-10T12:07:47.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.446+0000 7fef14950700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef100ff390 0x7fef10194aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:47.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.446+0000 7fef14950700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef100ffd30 0x7fef10194fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:47.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.446+0000 7fef14950700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef10195520 con 0x7fef100ffd30 2024-09-10T12:07:47.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.446+0000 7fef14950700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef10195690 con 0x7fef100ff390 2024-09-10T12:07:47.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.446+0000 7fef05d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef100ffd30 0x7fef10194fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:47.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.446+0000 7fef05d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef100ffd30 0x7fef10194fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56932/0 (socket says 172.21.15.144:56932) 2024-09-10T12:07:47.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.446+0000 7fef05d9b700 1 -- 172.21.15.144:0/4161756406 learned_addr learned my addr 172.21.15.144:0/4161756406 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:07:47.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.446+0000 7fef0e59c700 1 --2- 172.21.15.144:0/4161756406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef100ff390 0x7fef10194aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:47.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.446+0000 7fef05d9b700 1 -- 172.21.15.144:0/4161756406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef100ff390 msgr2=0x7fef10194aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:47.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.446+0000 7fef05d9b700 1 --2- 172.21.15.144:0/4161756406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef100ff390 0x7fef10194aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.446+0000 7fef05d9b700 1 -- 172.21.15.144:0/4161756406 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef000095f0 con 0x7fef100ffd30 2024-09-10T12:07:47.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.447+0000 7fef05d9b700 1 --2- 172.21.15.144:0/4161756406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef100ffd30 0x7fef10194fe0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7feef800b480 tx=0x7feef800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:47.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.447+0000 7fef07fff700 1 -- 172.21.15.144:0/4161756406 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feef80118c0 con 0x7fef100ffd30 2024-09-10T12:07:47.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.447+0000 7fef07fff700 1 -- 172.21.15.144:0/4161756406 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feef8010380 con 0x7fef100ffd30 2024-09-10T12:07:47.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.447+0000 7fef14950700 1 -- 172.21.15.144:0/4161756406 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef101a2110 con 0x7fef100ffd30 2024-09-10T12:07:47.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.447+0000 7fef07fff700 1 -- 172.21.15.144:0/4161756406 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feef800f4f0 con 0x7fef100ffd30 2024-09-10T12:07:47.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.447+0000 7fef14950700 1 -- 172.21.15.144:0/4161756406 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef101a2470 con 0x7fef100ffd30 2024-09-10T12:07:47.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.448+0000 7fef14950700 1 -- 172.21.15.144:0/4161756406 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef10107650 con 0x7fef100ffd30 2024-09-10T12:07:47.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.448+0000 7fef07fff700 1 -- 172.21.15.144:0/4161756406 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7feef80104f0 con 0x7fef100ffd30 2024-09-10T12:07:47.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.449+0000 7fef07fff700 1 --2- 172.21.15.144:0/4161756406 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feefc06e200 0x7feefc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:47.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.449+0000 7fef07fff700 1 -- 172.21.15.144:0/4161756406 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7feef808db80 con 0x7fef100ffd30 2024-09-10T12:07:47.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.449+0000 7fef0e59c700 1 --2- 172.21.15.144:0/4161756406 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feefc06e200 0x7feefc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:47.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.449+0000 7fef0e59c700 1 --2- 172.21.15.144:0/4161756406 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feefc06e200 0x7feefc0706b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fef00006010 tx=0x7fef0001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:47.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.453+0000 7fef07fff700 1 -- 172.21.15.144:0/4161756406 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7feef805ab10 con 0x7fef100ffd30 2024-09-10T12:07:47.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.699+0000 7fef14950700 1 -- 172.21.15.144:0/4161756406 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fef1004ea50 con 0x7fef100ffd30 2024-09-10T12:07:47.701 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.700+0000 7fef07fff700 1 -- 172.21.15.144:0/4161756406 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7feef8015790 con 0x7fef100ffd30 2024-09-10T12:07:47.701 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:07:47.701 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:07:47.702 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:07:47.702 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:07:47.702 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:07:47.702 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:07:47.702 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:07:47.702 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:07:47.702 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:07:47.702 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:07:47.702 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:07:47.702 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:07:47.702 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:07:47.702 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:07:47.703 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:07:47.703 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:07:47.703 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:07:47.703 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:07:47.703 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:07:47.703 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:07:47.703 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:07:47.703 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:07:47.703 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:07:47.703 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:07:47.703 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:07:47.703 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:07:47.703 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:07:47.703 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:07:47.704 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:07:47.704 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:07:47.704 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:07:47.704 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:07:47.704 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:07:47.704 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:07:47.704 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:07:47.704 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:07:47.704 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:07:47.704 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:07:47.704 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:07:47.704 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:07:47.704 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:07:47.705 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:07:47.705 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:07:47.705 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.702+0000 7fef14950700 1 -- 172.21.15.144:0/4161756406 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feefc06e200 msgr2=0x7feefc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:47.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.702+0000 7fef14950700 1 --2- 172.21.15.144:0/4161756406 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feefc06e200 0x7feefc0706b0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fef00006010 tx=0x7fef0001a040 comp rx=0 tx=0).stop 2024-09-10T12:07:47.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.702+0000 7fef14950700 1 -- 172.21.15.144:0/4161756406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef100ffd30 msgr2=0x7fef10194fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:47.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.702+0000 7fef14950700 1 --2- 172.21.15.144:0/4161756406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef100ffd30 0x7fef10194fe0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7feef800b480 tx=0x7feef800b840 comp rx=0 tx=0).stop 2024-09-10T12:07:47.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.702+0000 7fef14950700 1 -- 172.21.15.144:0/4161756406 shutdown_connections 2024-09-10T12:07:47.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.702+0000 7fef14950700 1 --2- 172.21.15.144:0/4161756406 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feefc06e200 0x7feefc0706b0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.702+0000 7fef14950700 1 --2- 172.21.15.144:0/4161756406 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fef100ff390 0x7fef10194aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.702+0000 7fef14950700 1 --2- 172.21.15.144:0/4161756406 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fef100ffd30 0x7fef10194fe0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.702+0000 7fef14950700 1 -- 172.21.15.144:0/4161756406 >> 172.21.15.144:0/4161756406 conn(0x7fef10074f50 msgr2=0x7fef100fd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:47.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.702+0000 7fef14950700 1 -- 172.21.15.144:0/4161756406 shutdown_connections 2024-09-10T12:07:47.706 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.702+0000 7fef14950700 1 -- 172.21.15.144:0/4161756406 wait complete. 2024-09-10T12:07:47.706 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:07:47.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.807+0000 7f77a09f1700 1 -- 172.21.15.144:0/3198061414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779c0ffd30 msgr2=0x7f779c103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:47.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.807+0000 7f77a09f1700 1 --2- 172.21.15.144:0/3198061414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779c0ffd30 0x7f779c103ce0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f7790009910 tx=0x7f7790009c20 comp rx=0 tx=0).stop 2024-09-10T12:07:47.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.807+0000 7f77a09f1700 1 -- 172.21.15.144:0/3198061414 shutdown_connections 2024-09-10T12:07:47.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.807+0000 7f77a09f1700 1 --2- 172.21.15.144:0/3198061414 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779c0ffd30 0x7f779c103ce0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.807+0000 7f77a09f1700 1 --2- 172.21.15.144:0/3198061414 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f779c0ff390 0x7f779c0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.807+0000 7f77a09f1700 1 -- 172.21.15.144:0/3198061414 >> 172.21.15.144:0/3198061414 conn(0x7f779c074f50 msgr2=0x7f779c075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:47.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.807+0000 7f77a09f1700 1 -- 172.21.15.144:0/3198061414 shutdown_connections 2024-09-10T12:07:47.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.807+0000 7f77a09f1700 1 -- 172.21.15.144:0/3198061414 wait complete. 2024-09-10T12:07:47.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.808+0000 7f77a09f1700 1 Processor -- start 2024-09-10T12:07:47.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.808+0000 7f77a09f1700 1 -- start start 2024-09-10T12:07:47.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.808+0000 7f77a09f1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779c0ff390 0x7f779c196b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:47.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.808+0000 7f77a09f1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f779c0ffd30 0x7f779c1970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:47.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.808+0000 7f77a09f1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c197650 con 0x7f779c0ff390 2024-09-10T12:07:47.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.808+0000 7f77a09f1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779c197790 con 0x7f779c0ffd30 2024-09-10T12:07:47.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.808+0000 7f779a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779c0ff390 0x7f779c196b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:47.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.808+0000 7f779a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779c0ff390 0x7f779c196b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56954/0 (socket says 172.21.15.144:56954) 2024-09-10T12:07:47.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.808+0000 7f779a59c700 1 -- 172.21.15.144:0/1318045810 learned_addr learned my addr 172.21.15.144:0/1318045810 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:07:47.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.809+0000 7f779a59c700 1 -- 172.21.15.144:0/1318045810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f779c0ffd30 msgr2=0x7f779c1970c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:47.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.809+0000 7f779a59c700 1 --2- 172.21.15.144:0/1318045810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f779c0ffd30 0x7f779c1970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:47.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.809+0000 7f779a59c700 1 -- 172.21.15.144:0/1318045810 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77900095f0 con 0x7f779c0ff390 2024-09-10T12:07:47.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.809+0000 7f779a59c700 1 --2- 172.21.15.144:0/1318045810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779c0ff390 0x7f779c196b80 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f778c00c930 tx=0x7f778c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:47.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.809+0000 7f778b7fe700 1 -- 172.21.15.144:0/1318045810 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f778c007ad0 con 0x7f779c0ff390 2024-09-10T12:07:47.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.809+0000 7f77a09f1700 1 -- 172.21.15.144:0/1318045810 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779c19fbf0 con 0x7f779c0ff390 2024-09-10T12:07:47.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.809+0000 7f778b7fe700 1 -- 172.21.15.144:0/1318045810 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f778c007c30 con 0x7f779c0ff390 2024-09-10T12:07:47.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.809+0000 7f778b7fe700 1 -- 172.21.15.144:0/1318045810 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f778c018610 con 0x7f779c0ff390 2024-09-10T12:07:47.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.809+0000 7f77a09f1700 1 -- 172.21.15.144:0/1318045810 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779c1a0060 con 0x7f779c0ff390 2024-09-10T12:07:47.812 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.810+0000 7f77a09f1700 1 -- 172.21.15.144:0/1318045810 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f779c107600 con 0x7f779c0ff390 2024-09-10T12:07:47.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.810+0000 7f778b7fe700 1 -- 172.21.15.144:0/1318045810 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f778c00f450 con 0x7f779c0ff390 2024-09-10T12:07:47.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.811+0000 7f778b7fe700 1 --2- 172.21.15.144:0/1318045810 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f778406e200 0x7f77840706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:47.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.811+0000 7f778b7fe700 1 -- 172.21.15.144:0/1318045810 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f778c08e2a0 con 0x7f779c0ff390 2024-09-10T12:07:47.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.811+0000 7f7799d9b700 1 --2- 172.21.15.144:0/1318045810 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f778406e200 0x7f77840706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:47.813 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.811+0000 7f7799d9b700 1 --2- 172.21.15.144:0/1318045810 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f778406e200 0x7f77840706b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7790006010 tx=0x7f7790005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:47.817 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:47.816+0000 7f778b7fe700 1 -- 172.21.15.144:0/1318045810 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f778c057680 con 0x7f779c0ff390 2024-09-10T12:07:47.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:47 smithi144 ceph-mon[87436]: from='client.35940 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:47.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:47 smithi144 ceph-mon[87436]: from='client.35944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:47.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:47 smithi144 ceph-mon[87436]: from='client.35948 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:47.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:47 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3002866186' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:07:48.038 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.037+0000 7f77a09f1700 1 -- 172.21.15.144:0/1318045810 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f779c02ccd0 con 0x7f778406e200 2024-09-10T12:07:48.040 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.038+0000 7f778b7fe700 1 -- 172.21.15.144:0/1318045810 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f779c02ccd0 con 0x7f778406e200 2024-09-10T12:07:48.041 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:07:48.041 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:07:48.041 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:07:48.041 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:07:48.041 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:07:48.041 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:07:48.041 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:07:48.041 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:07:48.041 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:07:48.041 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:07:48.041 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:07:48.041 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:07:48.042 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:07:48.042 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:07:48.042 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:07:48.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.040+0000 7f77a09f1700 1 -- 172.21.15.144:0/1318045810 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f778406e200 msgr2=0x7f77840706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:48.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.040+0000 7f77a09f1700 1 --2- 172.21.15.144:0/1318045810 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f778406e200 0x7f77840706b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f7790006010 tx=0x7f7790005960 comp rx=0 tx=0).stop 2024-09-10T12:07:48.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.040+0000 7f77a09f1700 1 -- 172.21.15.144:0/1318045810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779c0ff390 msgr2=0x7f779c196b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:48.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.040+0000 7f77a09f1700 1 --2- 172.21.15.144:0/1318045810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779c0ff390 0x7f779c196b80 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f778c00c930 tx=0x7f778c00ccf0 comp rx=0 tx=0).stop 2024-09-10T12:07:48.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.040+0000 7f77a09f1700 1 -- 172.21.15.144:0/1318045810 shutdown_connections 2024-09-10T12:07:48.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.040+0000 7f77a09f1700 1 --2- 172.21.15.144:0/1318045810 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f778406e200 0x7f77840706b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:48.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.040+0000 7f77a09f1700 1 --2- 172.21.15.144:0/1318045810 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f779c0ff390 0x7f779c196b80 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:48.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.040+0000 7f77a09f1700 1 --2- 172.21.15.144:0/1318045810 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f779c0ffd30 0x7f779c1970c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:48.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.040+0000 7f77a09f1700 1 -- 172.21.15.144:0/1318045810 >> 172.21.15.144:0/1318045810 conn(0x7f779c074f50 msgr2=0x7f779c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:48.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.040+0000 7f77a09f1700 1 -- 172.21.15.144:0/1318045810 shutdown_connections 2024-09-10T12:07:48.043 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.040+0000 7f77a09f1700 1 -- 172.21.15.144:0/1318045810 wait complete. 2024-09-10T12:07:48.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:47 smithi178 ceph-mon[65940]: from='client.35940 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:48.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:47 smithi178 ceph-mon[65940]: from='client.35944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:48.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:47 smithi178 ceph-mon[65940]: from='client.35948 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:48.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:47 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3002866186' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:07:48.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.144+0000 7ff544f8a700 1 -- 172.21.15.144:0/3181105203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5400ff320 msgr2=0x7ff5400ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:48.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.144+0000 7ff544f8a700 1 --2- 172.21.15.144:0/3181105203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5400ff320 0x7ff5400ff6f0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ff53800b0d0 tx=0x7ff53800b3e0 comp rx=0 tx=0).stop 2024-09-10T12:07:48.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.144+0000 7ff544f8a700 1 -- 172.21.15.144:0/3181105203 shutdown_connections 2024-09-10T12:07:48.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.144+0000 7ff544f8a700 1 --2- 172.21.15.144:0/3181105203 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5400ffc30 0x7ff54010a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:48.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.144+0000 7ff544f8a700 1 --2- 172.21.15.144:0/3181105203 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5400ff320 0x7ff5400ff6f0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:48.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.144+0000 7ff544f8a700 1 -- 172.21.15.144:0/3181105203 >> 172.21.15.144:0/3181105203 conn(0x7ff5400748b0 msgr2=0x7ff540074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:48.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.144+0000 7ff544f8a700 1 -- 172.21.15.144:0/3181105203 shutdown_connections 2024-09-10T12:07:48.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.144+0000 7ff544f8a700 1 -- 172.21.15.144:0/3181105203 wait complete. 2024-09-10T12:07:48.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.145+0000 7ff544f8a700 1 Processor -- start 2024-09-10T12:07:48.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.145+0000 7ff544f8a700 1 -- start start 2024-09-10T12:07:48.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.145+0000 7ff544f8a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5400ff320 0x7ff540195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:48.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.145+0000 7ff544f8a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5400ffc30 0x7ff540196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:48.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.145+0000 7ff544f8a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff540196890 con 0x7ff5400ffc30 2024-09-10T12:07:48.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.145+0000 7ff544f8a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff540199550 con 0x7ff5400ff320 2024-09-10T12:07:48.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.145+0000 7ff53dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5400ffc30 0x7ff540196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:48.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.145+0000 7ff53dd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5400ffc30 0x7ff540196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:56968/0 (socket says 172.21.15.144:56968) 2024-09-10T12:07:48.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.145+0000 7ff53dd9b700 1 -- 172.21.15.144:0/2837869686 learned_addr learned my addr 172.21.15.144:0/2837869686 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:07:48.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.145+0000 7ff53e59c700 1 --2- 172.21.15.144:0/2837869686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5400ff320 0x7ff540195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:48.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.145+0000 7ff53dd9b700 1 -- 172.21.15.144:0/2837869686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5400ff320 msgr2=0x7ff540195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:48.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.145+0000 7ff53dd9b700 1 --2- 172.21.15.144:0/2837869686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5400ff320 0x7ff540195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:48.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.145+0000 7ff53dd9b700 1 -- 172.21.15.144:0/2837869686 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff538009d00 con 0x7ff5400ffc30 2024-09-10T12:07:48.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.146+0000 7ff53dd9b700 1 --2- 172.21.15.144:0/2837869686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5400ffc30 0x7ff540196200 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff52800d900 tx=0x7ff52800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:48.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.146+0000 7ff5377fe700 1 -- 172.21.15.144:0/2837869686 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff528004a00 con 0x7ff5400ffc30 2024-09-10T12:07:48.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.146+0000 7ff5377fe700 1 -- 172.21.15.144:0/2837869686 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff5280054e0 con 0x7ff5400ffc30 2024-09-10T12:07:48.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.146+0000 7ff544f8a700 1 -- 172.21.15.144:0/2837869686 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff540199830 con 0x7ff5400ffc30 2024-09-10T12:07:48.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.146+0000 7ff5377fe700 1 -- 172.21.15.144:0/2837869686 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff528009d00 con 0x7ff5400ffc30 2024-09-10T12:07:48.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.146+0000 7ff544f8a700 1 -- 172.21.15.144:0/2837869686 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff540199d80 con 0x7ff5400ffc30 2024-09-10T12:07:48.150 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.147+0000 7ff544f8a700 1 -- 172.21.15.144:0/2837869686 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff540107da0 con 0x7ff5400ffc30 2024-09-10T12:07:48.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.147+0000 7ff5377fe700 1 -- 172.21.15.144:0/2837869686 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7ff528005020 con 0x7ff5400ffc30 2024-09-10T12:07:48.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.148+0000 7ff5377fe700 1 --2- 172.21.15.144:0/2837869686 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff52c06e200 0x7ff52c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:07:48.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.148+0000 7ff5377fe700 1 -- 172.21.15.144:0/2837869686 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7ff52808dfa0 con 0x7ff5400ffc30 2024-09-10T12:07:48.151 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.148+0000 7ff53e59c700 1 --2- 172.21.15.144:0/2837869686 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff52c06e200 0x7ff52c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:07:48.152 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.148+0000 7ff53e59c700 1 --2- 172.21.15.144:0/2837869686 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff52c06e200 0x7ff52c0706b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff538009480 tx=0x7ff53800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:07:48.153 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.152+0000 7ff5377fe700 1 -- 172.21.15.144:0/2837869686 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7ff528057380 con 0x7ff5400ffc30 2024-09-10T12:07:48.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.436+0000 7ff544f8a700 1 -- 172.21.15.144:0/2837869686 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7ff54019a000 con 0x7ff5400ffc30 2024-09-10T12:07:48.437 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.436+0000 7ff5377fe700 1 -- 172.21.15.144:0/2837869686 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7ff528009e60 con 0x7ff5400ffc30 2024-09-10T12:07:48.438 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:07:48.438 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:07:48.438 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:07:48.438 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:07:48.438 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:07:48.438 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:07:48.438 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:07:48.438 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:07:48.438 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:07:48.438 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:07:48.438 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:07:48.438 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:07:48.439 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:07:48.439 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:07:48.439 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:07:48.439 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:07:48.439 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:07:48.439 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:07:48.439 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:07:48.439 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:07:48.439 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:07:48.439 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:07:48.439 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:07:48.439 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:07:48.439 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:07:48.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.439+0000 7ff544f8a700 1 -- 172.21.15.144:0/2837869686 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff52c06e200 msgr2=0x7ff52c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:48.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.439+0000 7ff544f8a700 1 --2- 172.21.15.144:0/2837869686 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff52c06e200 0x7ff52c0706b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7ff538009480 tx=0x7ff53800b040 comp rx=0 tx=0).stop 2024-09-10T12:07:48.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.439+0000 7ff544f8a700 1 -- 172.21.15.144:0/2837869686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5400ffc30 msgr2=0x7ff540196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:07:48.440 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.439+0000 7ff544f8a700 1 --2- 172.21.15.144:0/2837869686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5400ffc30 0x7ff540196200 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7ff52800d900 tx=0x7ff52800dc10 comp rx=0 tx=0).stop 2024-09-10T12:07:48.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.439+0000 7ff544f8a700 1 -- 172.21.15.144:0/2837869686 shutdown_connections 2024-09-10T12:07:48.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.439+0000 7ff544f8a700 1 --2- 172.21.15.144:0/2837869686 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7ff52c06e200 0x7ff52c0706b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:48.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.439+0000 7ff544f8a700 1 --2- 172.21.15.144:0/2837869686 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7ff5400ff320 0x7ff540195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:48.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.439+0000 7ff544f8a700 1 --2- 172.21.15.144:0/2837869686 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7ff5400ffc30 0x7ff540196200 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:07:48.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.439+0000 7ff544f8a700 1 -- 172.21.15.144:0/2837869686 >> 172.21.15.144:0/2837869686 conn(0x7ff5400748b0 msgr2=0x7ff540104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:07:48.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.439+0000 7ff544f8a700 1 -- 172.21.15.144:0/2837869686 shutdown_connections 2024-09-10T12:07:48.441 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:07:48.439+0000 7ff544f8a700 1 -- 172.21.15.144:0/2837869686 wait complete. 2024-09-10T12:07:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:48 smithi144 ceph-mon[87436]: pgmap v1075: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:48 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/4161756406' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:07:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:48 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2837869686' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:07:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:48 smithi178 ceph-mon[65940]: pgmap v1075: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:48 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/4161756406' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:07:49.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:48 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2837869686' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:07:49.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:49 smithi144 ceph-mon[87436]: from='client.35960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:50.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:49 smithi178 ceph-mon[65940]: from='client.35960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:07:50.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:50 smithi144 ceph-mon[87436]: pgmap v1076: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:50 smithi178 ceph-mon[65940]: pgmap v1076: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:53.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:52 smithi144 ceph-mon[87436]: pgmap v1077: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:52 smithi178 ceph-mon[65940]: pgmap v1077: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:54 smithi178 ceph-mon[65940]: pgmap v1078: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:54 smithi144 ceph-mon[87436]: pgmap v1078: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:56 smithi178 ceph-mon[65940]: pgmap v1079: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:56 smithi144 ceph-mon[87436]: pgmap v1079: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:59.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:58 smithi178 ceph-mon[65940]: pgmap v1080: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:59.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:07:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:07:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:58 smithi144 ceph-mon[87436]: pgmap v1080: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:07:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:07:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:08:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:00 smithi178 ceph-mon[65940]: pgmap v1081: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:00 smithi144 ceph-mon[87436]: pgmap v1081: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:03.115 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:02 smithi144 ceph-mon[87436]: pgmap v1082: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:02 smithi178 ceph-mon[65940]: pgmap v1082: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:04 smithi178 ceph-mon[65940]: pgmap v1083: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:05.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:04 smithi144 ceph-mon[87436]: pgmap v1083: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:06 smithi178 ceph-mon[65940]: pgmap v1084: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:06 smithi144 ceph-mon[87436]: pgmap v1084: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:08 smithi178 ceph-mon[65940]: pgmap v1085: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:08 smithi144 ceph-mon[87436]: pgmap v1085: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:10 smithi178 ceph-mon[65940]: pgmap v1086: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:11.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:10 smithi144 ceph-mon[87436]: pgmap v1086: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:12 smithi144 ceph-mon[87436]: pgmap v1087: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:12 smithi178 ceph-mon[65940]: pgmap v1087: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:08:14.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:08:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:14 smithi178 ceph-mon[65940]: pgmap v1088: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:14 smithi144 ceph-mon[87436]: pgmap v1088: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:16 smithi178 ceph-mon[65940]: pgmap v1089: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:17.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:16 smithi144 ceph-mon[87436]: pgmap v1089: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:18.557 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.556+0000 7f05724b5700 1 -- 172.21.15.144:0/2011203069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f056c1049a0 msgr2=0x7f056c104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:18.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.556+0000 7f05724b5700 1 --2- 172.21.15.144:0/2011203069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f056c1049a0 0x7f056c104d70 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f055c009910 tx=0x7f055c009c20 comp rx=0 tx=0).stop 2024-09-10T12:08:18.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.556+0000 7f05724b5700 1 -- 172.21.15.144:0/2011203069 shutdown_connections 2024-09-10T12:08:18.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.556+0000 7f05724b5700 1 --2- 172.21.15.144:0/2011203069 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f056c0ff270 0x7f056c0ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:18.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.556+0000 7f05724b5700 1 --2- 172.21.15.144:0/2011203069 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f056c1049a0 0x7f056c104d70 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:18.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.556+0000 7f05724b5700 1 -- 172.21.15.144:0/2011203069 >> 172.21.15.144:0/2011203069 conn(0x7f056c074850 msgr2=0x7f056c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:18.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.556+0000 7f05724b5700 1 -- 172.21.15.144:0/2011203069 shutdown_connections 2024-09-10T12:08:18.558 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.556+0000 7f05724b5700 1 -- 172.21.15.144:0/2011203069 wait complete. 2024-09-10T12:08:18.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.557+0000 7f05724b5700 1 Processor -- start 2024-09-10T12:08:18.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.557+0000 7f05724b5700 1 -- start start 2024-09-10T12:08:18.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.557+0000 7f05724b5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f056c0ff270 0x7f056c105740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:18.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.557+0000 7f05724b5700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f056c1049a0 0x7f056c105c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:18.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.557+0000 7f05724b5700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c1061c0 con 0x7f056c0ff270 2024-09-10T12:08:18.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.557+0000 7f05724b5700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f056c106300 con 0x7f056c1049a0 2024-09-10T12:08:18.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.557+0000 7f056bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f056c0ff270 0x7f056c105740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:18.560 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.557+0000 7f056bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f056c0ff270 0x7f056c105740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36032/0 (socket says 172.21.15.144:36032) 2024-09-10T12:08:18.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.557+0000 7f056bfff700 1 -- 172.21.15.144:0/1046393700 learned_addr learned my addr 172.21.15.144:0/1046393700 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:08:18.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.557+0000 7f056b7fe700 1 --2- 172.21.15.144:0/1046393700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f056c1049a0 0x7f056c105c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:18.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.558+0000 7f056bfff700 1 -- 172.21.15.144:0/1046393700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f056c1049a0 msgr2=0x7f056c105c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:18.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.558+0000 7f056bfff700 1 --2- 172.21.15.144:0/1046393700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f056c1049a0 0x7f056c105c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:18.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.558+0000 7f056bfff700 1 -- 172.21.15.144:0/1046393700 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f055c0095f0 con 0x7f056c0ff270 2024-09-10T12:08:18.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.558+0000 7f056bfff700 1 --2- 172.21.15.144:0/1046393700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f056c0ff270 0x7f056c105740 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f055c006010 tx=0x7f055c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:18.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.558+0000 7f05697fa700 1 -- 172.21.15.144:0/1046393700 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f055c01d070 con 0x7f056c0ff270 2024-09-10T12:08:18.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.558+0000 7f05697fa700 1 -- 172.21.15.144:0/1046393700 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f055c005dc0 con 0x7f056c0ff270 2024-09-10T12:08:18.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.558+0000 7f05724b5700 1 -- 172.21.15.144:0/1046393700 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f056c106580 con 0x7f056c0ff270 2024-09-10T12:08:18.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.558+0000 7f05697fa700 1 -- 172.21.15.144:0/1046393700 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f055c017400 con 0x7f056c0ff270 2024-09-10T12:08:18.561 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.558+0000 7f05724b5700 1 -- 172.21.15.144:0/1046393700 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f056c1a22a0 con 0x7f056c0ff270 2024-09-10T12:08:18.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.559+0000 7f05724b5700 1 -- 172.21.15.144:0/1046393700 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056c103bb0 con 0x7f056c0ff270 2024-09-10T12:08:18.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.560+0000 7f05697fa700 1 -- 172.21.15.144:0/1046393700 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f055c003a60 con 0x7f056c0ff270 2024-09-10T12:08:18.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.560+0000 7f05697fa700 1 --2- 172.21.15.144:0/1046393700 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f055406e200 0x7f05540706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:18.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.560+0000 7f05697fa700 1 -- 172.21.15.144:0/1046393700 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f055c08f500 con 0x7f056c0ff270 2024-09-10T12:08:18.562 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.560+0000 7f056b7fe700 1 --2- 172.21.15.144:0/1046393700 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f055406e200 0x7f05540706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:18.563 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.561+0000 7f056b7fe700 1 --2- 172.21.15.144:0/1046393700 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f055406e200 0x7f05540706b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f056c106f30 tx=0x7f056000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:18.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.565+0000 7f05697fa700 1 -- 172.21.15.144:0/1046393700 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f055c0588e0 con 0x7f056c0ff270 2024-09-10T12:08:18.788 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.787+0000 7f05724b5700 1 -- 172.21.15.144:0/1046393700 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f056c02cc80 con 0x7f055406e200 2024-09-10T12:08:18.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.788+0000 7f05697fa700 1 -- 172.21.15.144:0/1046393700 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f056c02cc80 con 0x7f055406e200 2024-09-10T12:08:18.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.790+0000 7f05724b5700 1 -- 172.21.15.144:0/1046393700 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f055406e200 msgr2=0x7f05540706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:18.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.790+0000 7f05724b5700 1 --2- 172.21.15.144:0/1046393700 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f055406e200 0x7f05540706b0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f056c106f30 tx=0x7f056000b410 comp rx=0 tx=0).stop 2024-09-10T12:08:18.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.790+0000 7f05724b5700 1 -- 172.21.15.144:0/1046393700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f056c0ff270 msgr2=0x7f056c105740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:18.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.790+0000 7f05724b5700 1 --2- 172.21.15.144:0/1046393700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f056c0ff270 0x7f056c105740 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f055c006010 tx=0x7f055c00bd70 comp rx=0 tx=0).stop 2024-09-10T12:08:18.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.790+0000 7f05724b5700 1 -- 172.21.15.144:0/1046393700 shutdown_connections 2024-09-10T12:08:18.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.790+0000 7f05724b5700 1 --2- 172.21.15.144:0/1046393700 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f055406e200 0x7f05540706b0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:18.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.791+0000 7f05724b5700 1 --2- 172.21.15.144:0/1046393700 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f056c0ff270 0x7f056c105740 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:18.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.791+0000 7f05724b5700 1 --2- 172.21.15.144:0/1046393700 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f056c1049a0 0x7f056c105c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:18.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.791+0000 7f05724b5700 1 -- 172.21.15.144:0/1046393700 >> 172.21.15.144:0/1046393700 conn(0x7f056c074850 msgr2=0x7f056c0fe280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:18.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.791+0000 7f05724b5700 1 -- 172.21.15.144:0/1046393700 shutdown_connections 2024-09-10T12:08:18.793 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.791+0000 7f05724b5700 1 -- 172.21.15.144:0/1046393700 wait complete. 2024-09-10T12:08:18.803 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:08:18.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.897+0000 7f7898ff2700 1 -- 172.21.15.144:0/1002153075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7894100d70 msgr2=0x7f7894104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:18.899 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.897+0000 7f7898ff2700 1 --2- 172.21.15.144:0/1002153075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7894100d70 0x7f7894104d40 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f787c009910 tx=0x7f787c009c20 comp rx=0 tx=0).stop 2024-09-10T12:08:18.899 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:18 smithi144 ceph-mon[87436]: pgmap v1090: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:18.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.897+0000 7f7898ff2700 1 -- 172.21.15.144:0/1002153075 shutdown_connections 2024-09-10T12:08:18.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.897+0000 7f7898ff2700 1 --2- 172.21.15.144:0/1002153075 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7894100d70 0x7f7894104d40 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:18.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.897+0000 7f7898ff2700 1 --2- 172.21.15.144:0/1002153075 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78941003d0 0x7f78941007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:18.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.897+0000 7f7898ff2700 1 -- 172.21.15.144:0/1002153075 >> 172.21.15.144:0/1002153075 conn(0x7f78940fbc90 msgr2=0x7f78940fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:18.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.897+0000 7f7898ff2700 1 -- 172.21.15.144:0/1002153075 shutdown_connections 2024-09-10T12:08:18.901 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.897+0000 7f7898ff2700 1 -- 172.21.15.144:0/1002153075 wait complete. 2024-09-10T12:08:18.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.898+0000 7f7898ff2700 1 Processor -- start 2024-09-10T12:08:18.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.898+0000 7f7898ff2700 1 -- start start 2024-09-10T12:08:18.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.898+0000 7f7898ff2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78941003d0 0x7f7894195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:18.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.898+0000 7f7898ff2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7894100d70 0x7f7894196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:18.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.898+0000 7f7898ff2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78941968d0 con 0x7f7894100d70 2024-09-10T12:08:18.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.898+0000 7f7898ff2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7894196a10 con 0x7f78941003d0 2024-09-10T12:08:18.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.898+0000 7f7891d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7894100d70 0x7f7894196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:18.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.899+0000 7f7891d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7894100d70 0x7f7894196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36048/0 (socket says 172.21.15.144:36048) 2024-09-10T12:08:18.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.899+0000 7f7891d9b700 1 -- 172.21.15.144:0/3367394117 learned_addr learned my addr 172.21.15.144:0/3367394117 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:08:18.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.899+0000 7f789259c700 1 --2- 172.21.15.144:0/3367394117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78941003d0 0x7f7894195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:18.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.899+0000 7f7891d9b700 1 -- 172.21.15.144:0/3367394117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78941003d0 msgr2=0x7f7894195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:18.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.899+0000 7f7891d9b700 1 --2- 172.21.15.144:0/3367394117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78941003d0 0x7f7894195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:18.902 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.899+0000 7f7891d9b700 1 -- 172.21.15.144:0/3367394117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c0095f0 con 0x7f7894100d70 2024-09-10T12:08:18.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.899+0000 7f7891d9b700 1 --2- 172.21.15.144:0/3367394117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7894100d70 0x7f7894196280 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f787c00bda0 tx=0x7f787c00be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:18.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.899+0000 7f788b7fe700 1 -- 172.21.15.144:0/3367394117 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f787c01d070 con 0x7f7894100d70 2024-09-10T12:08:18.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.899+0000 7f788b7fe700 1 -- 172.21.15.144:0/3367394117 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f787c004fb0 con 0x7f7894100d70 2024-09-10T12:08:18.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.899+0000 7f7898ff2700 1 -- 172.21.15.144:0/3367394117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789419a800 con 0x7f7894100d70 2024-09-10T12:08:18.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.899+0000 7f788b7fe700 1 -- 172.21.15.144:0/3367394117 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f787c00f460 con 0x7f7894100d70 2024-09-10T12:08:18.903 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.899+0000 7f7898ff2700 1 -- 172.21.15.144:0/3367394117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789419acf0 con 0x7f7894100d70 2024-09-10T12:08:18.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.900+0000 7f7898ff2700 1 -- 172.21.15.144:0/3367394117 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7894108740 con 0x7f7894100d70 2024-09-10T12:08:18.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.901+0000 7f788b7fe700 1 -- 172.21.15.144:0/3367394117 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f787c004af0 con 0x7f7894100d70 2024-09-10T12:08:18.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.901+0000 7f788b7fe700 1 --2- 172.21.15.144:0/3367394117 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f788006e200 0x7f78800706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:18.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.901+0000 7f788b7fe700 1 -- 172.21.15.144:0/3367394117 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f787c08f540 con 0x7f7894100d70 2024-09-10T12:08:18.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.901+0000 7f789259c700 1 --2- 172.21.15.144:0/3367394117 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f788006e200 0x7f78800706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:18.904 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.902+0000 7f789259c700 1 --2- 172.21.15.144:0/3367394117 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f788006e200 0x7f78800706b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7884005ea0 tx=0x7f7884005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:18.907 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:18.905+0000 7f788b7fe700 1 -- 172.21.15.144:0/3367394117 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f787c058a50 con 0x7f7894100d70 2024-09-10T12:08:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:18 smithi178 ceph-mon[65940]: pgmap v1090: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:19.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.124+0000 7f7898ff2700 1 -- 172.21.15.144:0/3367394117 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f789402ccf0 con 0x7f788006e200 2024-09-10T12:08:19.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.125+0000 7f788b7fe700 1 -- 172.21.15.144:0/3367394117 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f789402ccf0 con 0x7f788006e200 2024-09-10T12:08:19.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.127+0000 7f7898ff2700 1 -- 172.21.15.144:0/3367394117 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f788006e200 msgr2=0x7f78800706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:19.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.127+0000 7f7898ff2700 1 --2- 172.21.15.144:0/3367394117 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f788006e200 0x7f78800706b0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7884005ea0 tx=0x7f7884005e30 comp rx=0 tx=0).stop 2024-09-10T12:08:19.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.127+0000 7f7898ff2700 1 -- 172.21.15.144:0/3367394117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7894100d70 msgr2=0x7f7894196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:19.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.127+0000 7f7898ff2700 1 --2- 172.21.15.144:0/3367394117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7894100d70 0x7f7894196280 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f787c00bda0 tx=0x7f787c00be80 comp rx=0 tx=0).stop 2024-09-10T12:08:19.129 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.128+0000 7f7898ff2700 1 -- 172.21.15.144:0/3367394117 shutdown_connections 2024-09-10T12:08:19.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.128+0000 7f7898ff2700 1 --2- 172.21.15.144:0/3367394117 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f788006e200 0x7f78800706b0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.128+0000 7f7898ff2700 1 --2- 172.21.15.144:0/3367394117 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f78941003d0 0x7f7894195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.128+0000 7f7898ff2700 1 --2- 172.21.15.144:0/3367394117 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7894100d70 0x7f7894196280 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.128+0000 7f7898ff2700 1 -- 172.21.15.144:0/3367394117 >> 172.21.15.144:0/3367394117 conn(0x7f78940fbc90 msgr2=0x7f78940fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:19.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.128+0000 7f7898ff2700 1 -- 172.21.15.144:0/3367394117 shutdown_connections 2024-09-10T12:08:19.130 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.128+0000 7f7898ff2700 1 -- 172.21.15.144:0/3367394117 wait complete. 2024-09-10T12:08:19.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.235+0000 7f457ad7a700 1 -- 172.21.15.144:0/2090076936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45741003b0 msgr2=0x7f4574100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:19.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.235+0000 7f457ad7a700 1 --2- 172.21.15.144:0/2090076936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45741003b0 0x7f4574100780 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f457000b0a0 tx=0x7f457000b3b0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.235+0000 7f457ad7a700 1 -- 172.21.15.144:0/2090076936 shutdown_connections 2024-09-10T12:08:19.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.235+0000 7f457ad7a700 1 --2- 172.21.15.144:0/2090076936 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4574100d50 0x7f4574104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.238 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.235+0000 7f457ad7a700 1 --2- 172.21.15.144:0/2090076936 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45741003b0 0x7f4574100780 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.235+0000 7f457ad7a700 1 -- 172.21.15.144:0/2090076936 >> 172.21.15.144:0/2090076936 conn(0x7f45740fbc70 msgr2=0x7f45740fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:19.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.235+0000 7f457ad7a700 1 -- 172.21.15.144:0/2090076936 shutdown_connections 2024-09-10T12:08:19.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.235+0000 7f457ad7a700 1 -- 172.21.15.144:0/2090076936 wait complete. 2024-09-10T12:08:19.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.235+0000 7f457ad7a700 1 Processor -- start 2024-09-10T12:08:19.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.236+0000 7f457ad7a700 1 -- start start 2024-09-10T12:08:19.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.236+0000 7f457ad7a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45741003b0 0x7f4574195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:19.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.236+0000 7f457ad7a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4574100d50 0x7f4574196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:19.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.236+0000 7f457ad7a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4574196960 con 0x7f45741003b0 2024-09-10T12:08:19.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.236+0000 7f457ad7a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457419a6f0 con 0x7f4574100d50 2024-09-10T12:08:19.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.236+0000 7f4578b16700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45741003b0 0x7f4574195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:19.239 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.236+0000 7f4578b16700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45741003b0 0x7f4574195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36066/0 (socket says 172.21.15.144:36066) 2024-09-10T12:08:19.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.236+0000 7f4578b16700 1 -- 172.21.15.144:0/177456504 learned_addr learned my addr 172.21.15.144:0/177456504 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:08:19.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.236+0000 7f456bfff700 1 --2- 172.21.15.144:0/177456504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4574100d50 0x7f4574196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:19.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.236+0000 7f4578b16700 1 -- 172.21.15.144:0/177456504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4574100d50 msgr2=0x7f4574196280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:19.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.236+0000 7f4578b16700 1 --2- 172.21.15.144:0/177456504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4574100d50 0x7f4574196280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.236+0000 7f4578b16700 1 -- 172.21.15.144:0/177456504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4570009d00 con 0x7f45741003b0 2024-09-10T12:08:19.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.236+0000 7f456bfff700 1 --2- 172.21.15.144:0/177456504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4574100d50 0x7f4574196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:08:19.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.236+0000 7f4578b16700 1 --2- 172.21.15.144:0/177456504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45741003b0 0x7f4574195d40 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f457000b070 tx=0x7f457000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:19.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.236+0000 7f4569ffb700 1 -- 172.21.15.144:0/177456504 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f457000e040 con 0x7f45741003b0 2024-09-10T12:08:19.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.237+0000 7f457ad7a700 1 -- 172.21.15.144:0/177456504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457419a970 con 0x7f45741003b0 2024-09-10T12:08:19.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.237+0000 7f4569ffb700 1 -- 172.21.15.144:0/177456504 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4570003de0 con 0x7f45741003b0 2024-09-10T12:08:19.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.237+0000 7f4569ffb700 1 -- 172.21.15.144:0/177456504 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f457001d8e0 con 0x7f45741003b0 2024-09-10T12:08:19.240 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.237+0000 7f457ad7a700 1 -- 172.21.15.144:0/177456504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f457419ae60 con 0x7f45741003b0 2024-09-10T12:08:19.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.238+0000 7f457ad7a700 1 -- 172.21.15.144:0/177456504 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45741086c0 con 0x7f45741003b0 2024-09-10T12:08:19.241 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.238+0000 7f4569ffb700 1 -- 172.21.15.144:0/177456504 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4570004040 con 0x7f45741003b0 2024-09-10T12:08:19.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.238+0000 7f4569ffb700 1 --2- 172.21.15.144:0/177456504 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f455c06e200 0x7f455c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:19.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.239+0000 7f4569ffb700 1 -- 172.21.15.144:0/177456504 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f457008f8d0 con 0x7f45741003b0 2024-09-10T12:08:19.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.239+0000 7f456bfff700 1 --2- 172.21.15.144:0/177456504 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f455c06e200 0x7f455c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:19.242 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.239+0000 7f456bfff700 1 --2- 172.21.15.144:0/177456504 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f455c06e200 0x7f455c0706b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4574197360 tx=0x7f456400b2c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:19.244 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.243+0000 7f4569ffb700 1 -- 172.21.15.144:0/177456504 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4570058cb0 con 0x7f45741003b0 2024-09-10T12:08:19.455 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.453+0000 7f457ad7a700 1 -- 172.21.15.144:0/177456504 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f457419b140 con 0x7f455c06e200 2024-09-10T12:08:19.464 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.463+0000 7f4569ffb700 1 -- 172.21.15.144:0/177456504 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f457419b140 con 0x7f455c06e200 2024-09-10T12:08:19.464 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:08:19.464 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (36m) 14m ago 47m 24.6M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:08:19.464 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (47m) 14m ago 47m 9.86M - 18.2.0 dc2bc1663786 26cccc2383be 2024-09-10T12:08:19.464 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (46m) 14m ago 46m 9.86M - 18.2.0 dc2bc1663786 b4705c4ae9ea 2024-09-10T12:08:19.465 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (34m) 14m ago 47m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:08:19.465 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (34m) 14m ago 46m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:08:19.465 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (36m) 14m ago 46m 80.9M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:08:19.465 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (30m) 14m ago 41m 21.9M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:08:19.465 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (30m) 14m ago 41m 19.7M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:08:19.465 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (14m) 14m ago 41m 13.0M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:08:19.465 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (30m) 14m ago 41m 20.4M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:08:19.465 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (38m) 14m ago 48m 507M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:08:19.465 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (37m) 14m ago 46m 448M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:08:19.465 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (35m) 14m ago 48m 73.7M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:08:19.465 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (34m) 14m ago 46m 53.4M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:08:19.465 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (37m) 14m ago 47m 16.0M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:08:19.465 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (37m) 14m ago 46m 15.4M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:08:19.466 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (33m) 14m ago 45m 96.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:08:19.466 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (33m) 14m ago 44m 94.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:08:19.466 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (32m) 14m ago 44m 86.0M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:08:19.466 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (32m) 14m ago 43m 107M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:08:19.466 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (31m) 14m ago 43m 92.3M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:08:19.466 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (30m) 14m ago 43m 88.9M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:08:19.466 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (36m) 14m ago 46m 86.4M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:08:19.467 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.465+0000 7f457ad7a700 1 -- 172.21.15.144:0/177456504 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f455c06e200 msgr2=0x7f455c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:19.467 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.465+0000 7f457ad7a700 1 --2- 172.21.15.144:0/177456504 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f455c06e200 0x7f455c0706b0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f4574197360 tx=0x7f456400b2c0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.467 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.465+0000 7f457ad7a700 1 -- 172.21.15.144:0/177456504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45741003b0 msgr2=0x7f4574195d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:19.467 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.465+0000 7f457ad7a700 1 --2- 172.21.15.144:0/177456504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45741003b0 0x7f4574195d40 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f457000b070 tx=0x7f457000cdd0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.467 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.465+0000 7f457ad7a700 1 -- 172.21.15.144:0/177456504 shutdown_connections 2024-09-10T12:08:19.467 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.465+0000 7f457ad7a700 1 --2- 172.21.15.144:0/177456504 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f455c06e200 0x7f455c0706b0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.467 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.465+0000 7f457ad7a700 1 --2- 172.21.15.144:0/177456504 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f45741003b0 0x7f4574195d40 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.465+0000 7f457ad7a700 1 --2- 172.21.15.144:0/177456504 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4574100d50 0x7f4574196280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.465+0000 7f457ad7a700 1 -- 172.21.15.144:0/177456504 >> 172.21.15.144:0/177456504 conn(0x7f45740fbc70 msgr2=0x7f45740fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:19.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.465+0000 7f457ad7a700 1 -- 172.21.15.144:0/177456504 shutdown_connections 2024-09-10T12:08:19.468 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.465+0000 7f457ad7a700 1 -- 172.21.15.144:0/177456504 wait complete. 2024-09-10T12:08:19.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.569+0000 7f84b6106700 1 -- 172.21.15.144:0/3844246290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b00ffd30 msgr2=0x7f84b0103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:19.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.569+0000 7f84b6106700 1 --2- 172.21.15.144:0/3844246290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b00ffd30 0x7f84b0103ce0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f8498009910 tx=0x7f8498009c20 comp rx=0 tx=0).stop 2024-09-10T12:08:19.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.569+0000 7f84b6106700 1 -- 172.21.15.144:0/3844246290 shutdown_connections 2024-09-10T12:08:19.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.569+0000 7f84b6106700 1 --2- 172.21.15.144:0/3844246290 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b00ffd30 0x7f84b0103ce0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.569+0000 7f84b6106700 1 --2- 172.21.15.144:0/3844246290 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84b00ff390 0x7f84b00ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.569+0000 7f84b6106700 1 -- 172.21.15.144:0/3844246290 >> 172.21.15.144:0/3844246290 conn(0x7f84b0074f50 msgr2=0x7f84b0075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:19.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.569+0000 7f84b6106700 1 -- 172.21.15.144:0/3844246290 shutdown_connections 2024-09-10T12:08:19.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.569+0000 7f84b6106700 1 -- 172.21.15.144:0/3844246290 wait complete. 2024-09-10T12:08:19.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.570+0000 7f84b6106700 1 Processor -- start 2024-09-10T12:08:19.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.570+0000 7f84b6106700 1 -- start start 2024-09-10T12:08:19.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.570+0000 7f84b6106700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b00ff390 0x7f84b0196b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:19.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.570+0000 7f84b6106700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84b00ffd30 0x7f84b01970a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:19.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.570+0000 7f84b6106700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b01976c0 con 0x7f84b00ff390 2024-09-10T12:08:19.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.570+0000 7f84b6106700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b0197830 con 0x7f84b00ffd30 2024-09-10T12:08:19.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.570+0000 7f84af7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b00ff390 0x7f84b0196b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:19.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.570+0000 7f84af7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b00ff390 0x7f84b0196b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36094/0 (socket says 172.21.15.144:36094) 2024-09-10T12:08:19.573 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.570+0000 7f84af7fe700 1 -- 172.21.15.144:0/1090209937 learned_addr learned my addr 172.21.15.144:0/1090209937 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:08:19.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.570+0000 7f84aeffd700 1 --2- 172.21.15.144:0/1090209937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84b00ffd30 0x7f84b01970a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:19.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.571+0000 7f84af7fe700 1 -- 172.21.15.144:0/1090209937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84b00ffd30 msgr2=0x7f84b01970a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:19.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.571+0000 7f84af7fe700 1 --2- 172.21.15.144:0/1090209937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84b00ffd30 0x7f84b01970a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.571+0000 7f84af7fe700 1 -- 172.21.15.144:0/1090209937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84980095f0 con 0x7f84b00ff390 2024-09-10T12:08:19.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.571+0000 7f84af7fe700 1 --2- 172.21.15.144:0/1090209937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b00ff390 0x7f84b0196b60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f84a00060b0 tx=0x7f84a000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:19.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.571+0000 7f84acff9700 1 -- 172.21.15.144:0/1090209937 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84a000dda0 con 0x7f84b00ff390 2024-09-10T12:08:19.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.571+0000 7f84acff9700 1 -- 172.21.15.144:0/1090209937 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f84a000f040 con 0x7f84b00ff390 2024-09-10T12:08:19.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.571+0000 7f84b6106700 1 -- 172.21.15.144:0/1090209937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b019fbd0 con 0x7f84b00ff390 2024-09-10T12:08:19.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.571+0000 7f84acff9700 1 -- 172.21.15.144:0/1090209937 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84a00146f0 con 0x7f84b00ff390 2024-09-10T12:08:19.574 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.571+0000 7f84b6106700 1 -- 172.21.15.144:0/1090209937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b01a0090 con 0x7f84b00ff390 2024-09-10T12:08:19.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.572+0000 7f84b6106700 1 -- 172.21.15.144:0/1090209937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b0107600 con 0x7f84b00ff390 2024-09-10T12:08:19.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.573+0000 7f84acff9700 1 -- 172.21.15.144:0/1090209937 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f84a0015400 con 0x7f84b00ff390 2024-09-10T12:08:19.575 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.574+0000 7f84acff9700 1 --2- 172.21.15.144:0/1090209937 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f849c06e200 0x7f849c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:19.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.574+0000 7f84acff9700 1 -- 172.21.15.144:0/1090209937 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f84a008da70 con 0x7f84b00ff390 2024-09-10T12:08:19.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.574+0000 7f84aeffd700 1 --2- 172.21.15.144:0/1090209937 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f849c06e200 0x7f849c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:19.576 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.574+0000 7f84aeffd700 1 --2- 172.21.15.144:0/1090209937 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f849c06e200 0x7f849c0706b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8498009fd0 tx=0x7f849801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:19.579 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.578+0000 7f84acff9700 1 -- 172.21.15.144:0/1090209937 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f84a0056f00 con 0x7f84b00ff390 2024-09-10T12:08:19.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.866+0000 7f84b6106700 1 -- 172.21.15.144:0/1090209937 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f84b002ccd0 con 0x7f84b00ff390 2024-09-10T12:08:19.868 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:19 smithi144 ceph-mon[87436]: from='client.35968 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:19.868 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.867+0000 7f84acff9700 1 -- 172.21.15.144:0/1090209937 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f84a005a5c0 con 0x7f84b00ff390 2024-09-10T12:08:19.869 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:08:19.869 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:08:19.869 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:08:19.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:08:19.869 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:08:19.869 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:08:19.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:08:19.869 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:08:19.869 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:08:19.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:08:19.869 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:08:19.869 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:08:19.869 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:08:19.870 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:08:19.870 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:08:19.870 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:08:19.870 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:08:19.870 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.869+0000 7f84b6106700 1 -- 172.21.15.144:0/1090209937 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f849c06e200 msgr2=0x7f849c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:19.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.869+0000 7f84b6106700 1 --2- 172.21.15.144:0/1090209937 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f849c06e200 0x7f849c0706b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8498009fd0 tx=0x7f849801a040 comp rx=0 tx=0).stop 2024-09-10T12:08:19.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.869+0000 7f84b6106700 1 -- 172.21.15.144:0/1090209937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b00ff390 msgr2=0x7f84b0196b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:19.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.869+0000 7f84b6106700 1 --2- 172.21.15.144:0/1090209937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b00ff390 0x7f84b0196b60 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f84a00060b0 tx=0x7f84a000d3b0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.869+0000 7f84b6106700 1 -- 172.21.15.144:0/1090209937 shutdown_connections 2024-09-10T12:08:19.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.869+0000 7f84b6106700 1 --2- 172.21.15.144:0/1090209937 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f849c06e200 0x7f849c0706b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.869+0000 7f84b6106700 1 --2- 172.21.15.144:0/1090209937 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f84b00ff390 0x7f84b0196b60 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.869+0000 7f84b6106700 1 --2- 172.21.15.144:0/1090209937 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f84b00ffd30 0x7f84b01970a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.869+0000 7f84b6106700 1 -- 172.21.15.144:0/1090209937 >> 172.21.15.144:0/1090209937 conn(0x7f84b0074f50 msgr2=0x7f84b00fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:19.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.869+0000 7f84b6106700 1 -- 172.21.15.144:0/1090209937 shutdown_connections 2024-09-10T12:08:19.871 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.869+0000 7f84b6106700 1 -- 172.21.15.144:0/1090209937 wait complete. 2024-09-10T12:08:19.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.973+0000 7f9c05910700 1 -- 172.21.15.144:0/3116845219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c000ff380 msgr2=0x7f9c000ff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:19.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.973+0000 7f9c05910700 1 --2- 172.21.15.144:0/3116845219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c000ff380 0x7f9c000ff750 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0009910 tx=0x7f9bf0009c20 comp rx=0 tx=0).stop 2024-09-10T12:08:19.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.974+0000 7f9c05910700 1 -- 172.21.15.144:0/3116845219 shutdown_connections 2024-09-10T12:08:19.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.974+0000 7f9c05910700 1 --2- 172.21.15.144:0/3116845219 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c000ffd20 0x7f9c00103cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.974+0000 7f9c05910700 1 --2- 172.21.15.144:0/3116845219 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c000ff380 0x7f9c000ff750 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.974+0000 7f9c05910700 1 -- 172.21.15.144:0/3116845219 >> 172.21.15.144:0/3116845219 conn(0x7f9c000fac20 msgr2=0x7f9c000fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:19.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.974+0000 7f9c05910700 1 -- 172.21.15.144:0/3116845219 shutdown_connections 2024-09-10T12:08:19.976 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.974+0000 7f9c05910700 1 -- 172.21.15.144:0/3116845219 wait complete. 2024-09-10T12:08:19.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.974+0000 7f9c05910700 1 Processor -- start 2024-09-10T12:08:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.974+0000 7f9c05910700 1 -- start start 2024-09-10T12:08:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9c05910700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c000ff380 0x7f9c00195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9c05910700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c000ffd20 0x7f9c00196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9c05910700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c00196a30 con 0x7f9c000ff380 2024-09-10T12:08:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9c05910700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0019a7c0 con 0x7f9c000ffd20 2024-09-10T12:08:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9bfeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c000ff380 0x7f9c00195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9bfeffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c000ff380 0x7f9c00195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36100/0 (socket says 172.21.15.144:36100) 2024-09-10T12:08:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9bfeffd700 1 -- 172.21.15.144:0/3485119776 learned_addr learned my addr 172.21.15.144:0/3485119776 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:08:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9bfe7fc700 1 --2- 172.21.15.144:0/3485119776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c000ffd20 0x7f9c00196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9bfeffd700 1 -- 172.21.15.144:0/3485119776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c000ffd20 msgr2=0x7f9c00196350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:19.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9bfeffd700 1 --2- 172.21.15.144:0/3485119776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c000ffd20 0x7f9c00196350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:19.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9bfeffd700 1 -- 172.21.15.144:0/3485119776 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf00095f0 con 0x7f9c000ff380 2024-09-10T12:08:19.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9bfeffd700 1 --2- 172.21.15.144:0/3485119776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c000ff380 0x7f9c00195e10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0006010 tx=0x7f9bf000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:19.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9c0490e700 1 -- 172.21.15.144:0/3485119776 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bf001d070 con 0x7f9c000ff380 2024-09-10T12:08:19.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9c0490e700 1 -- 172.21.15.144:0/3485119776 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9bf0005dc0 con 0x7f9c000ff380 2024-09-10T12:08:19.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9c05910700 1 -- 172.21.15.144:0/3485119776 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0019aa40 con 0x7f9c000ff380 2024-09-10T12:08:19.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9c05910700 1 -- 172.21.15.144:0/3485119776 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0019af30 con 0x7f9c000ff380 2024-09-10T12:08:19.979 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.975+0000 7f9c0490e700 1 -- 172.21.15.144:0/3485119776 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9bf0017400 con 0x7f9c000ff380 2024-09-10T12:08:19.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.977+0000 7f9c05910700 1 -- 172.21.15.144:0/3485119776 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c00107690 con 0x7f9c000ff380 2024-09-10T12:08:19.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.977+0000 7f9c0490e700 1 -- 172.21.15.144:0/3485119776 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f9bf0003a60 con 0x7f9c000ff380 2024-09-10T12:08:19.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.977+0000 7f9c0490e700 1 --2- 172.21.15.144:0/3485119776 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9bec06e200 0x7f9bec0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:19.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.978+0000 7f9c0490e700 1 -- 172.21.15.144:0/3485119776 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f9bf008f470 con 0x7f9c000ff380 2024-09-10T12:08:19.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.978+0000 7f9bfe7fc700 1 --2- 172.21.15.144:0/3485119776 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9bec06e200 0x7f9bec0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:19.980 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.978+0000 7f9bfe7fc700 1 --2- 172.21.15.144:0/3485119776 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9bec06e200 0x7f9bec0706b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9c00197430 tx=0x7f9be800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:19.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:19.982+0000 7f9c0490e700 1 -- 172.21.15.144:0/3485119776 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f9bf00588d0 con 0x7f9c000ff380 2024-09-10T12:08:20.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:19 smithi178 ceph-mon[65940]: from='client.35968 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:20.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.229+0000 7f9c05910700 1 -- 172.21.15.144:0/3485119776 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9c0004ea50 con 0x7f9c000ff380 2024-09-10T12:08:20.231 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.229+0000 7f9c0490e700 1 -- 172.21.15.144:0/3485119776 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1916 (secure 0 0 0) 0x7f9bf005bf90 con 0x7f9c000ff380 2024-09-10T12:08:20.231 INFO:teuthology.orchestra.run.smithi144.stdout:e34 2024-09-10T12:08:20.231 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T11:53:22:534815+0000 2024-09-10T12:08:20.231 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:08:20.231 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:08:20.232 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:08:20.232 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:08:20.232 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:08:20.232 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:08:20.232 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 25 2024-09-10T12:08:20.232 INFO:teuthology.orchestra.run.smithi144.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-09-10T12:08:20.232 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:08:20.232 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T11:37:33.047941+0000 2024-09-10T12:08:20.232 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:08:20.232 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:08:20.233 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:08:20.233 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:08:20.233 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:08:20.233 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:08:20.233 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:08:20.233 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:08:20.233 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:08:20.233 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:08:20.233 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:08:20.233 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:08:20.233 INFO:teuthology.orchestra.run.smithi144.stdout:up {} 2024-09-10T12:08:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:failed 0,1 2024-09-10T12:08:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:08:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:08:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:08:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:08:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:08:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:08:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:08:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:08:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 0 members: 2024-09-10T12:08:20.234 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:08:20.234 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:08:20.234 INFO:teuthology.orchestra.run.smithi144.stdout:Standby daemons: 2024-09-10T12:08:20.235 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:08:20.235 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{-1:34390} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:08:20.235 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{-1:34422} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:08:20.235 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{-1:44355} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:08:20.235 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{-1:44887} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:08:20.235 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.232+0000 7f9c05910700 1 -- 172.21.15.144:0/3485119776 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9bec06e200 msgr2=0x7f9bec0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:20.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.232+0000 7f9c05910700 1 --2- 172.21.15.144:0/3485119776 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9bec06e200 0x7f9bec0706b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9c00197430 tx=0x7f9be800b410 comp rx=0 tx=0).stop 2024-09-10T12:08:20.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.232+0000 7f9c05910700 1 -- 172.21.15.144:0/3485119776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c000ff380 msgr2=0x7f9c00195e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:20.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.232+0000 7f9c05910700 1 --2- 172.21.15.144:0/3485119776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c000ff380 0x7f9c00195e10 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f9bf0006010 tx=0x7f9bf000bd70 comp rx=0 tx=0).stop 2024-09-10T12:08:20.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.232+0000 7f9c05910700 1 -- 172.21.15.144:0/3485119776 shutdown_connections 2024-09-10T12:08:20.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.232+0000 7f9c05910700 1 --2- 172.21.15.144:0/3485119776 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f9bec06e200 0x7f9bec0706b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.232+0000 7f9c05910700 1 --2- 172.21.15.144:0/3485119776 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9c000ff380 0x7f9c00195e10 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.232+0000 7f9c05910700 1 --2- 172.21.15.144:0/3485119776 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9c000ffd20 0x7f9c00196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.232+0000 7f9c05910700 1 -- 172.21.15.144:0/3485119776 >> 172.21.15.144:0/3485119776 conn(0x7f9c000fac20 msgr2=0x7f9c000fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:20.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.232+0000 7f9c05910700 1 -- 172.21.15.144:0/3485119776 shutdown_connections 2024-09-10T12:08:20.236 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.232+0000 7f9c05910700 1 -- 172.21.15.144:0/3485119776 wait complete. 2024-09-10T12:08:20.236 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:08:20.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.338+0000 7f0220e9a700 1 -- 172.21.15.144:0/4258011567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f021c1003d0 msgr2=0x7f021c1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:20.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.338+0000 7f0220e9a700 1 --2- 172.21.15.144:0/4258011567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f021c1003d0 0x7f021c1007a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f020c009910 tx=0x7f020c009c20 comp rx=0 tx=0).stop 2024-09-10T12:08:20.339 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.338+0000 7f0220e9a700 1 -- 172.21.15.144:0/4258011567 shutdown_connections 2024-09-10T12:08:20.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.338+0000 7f0220e9a700 1 --2- 172.21.15.144:0/4258011567 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f021c100d70 0x7f021c104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.338+0000 7f0220e9a700 1 --2- 172.21.15.144:0/4258011567 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f021c1003d0 0x7f021c1007a0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.338+0000 7f0220e9a700 1 -- 172.21.15.144:0/4258011567 >> 172.21.15.144:0/4258011567 conn(0x7f021c0fbc90 msgr2=0x7f021c0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:20.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.338+0000 7f0220e9a700 1 -- 172.21.15.144:0/4258011567 shutdown_connections 2024-09-10T12:08:20.340 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.338+0000 7f0220e9a700 1 -- 172.21.15.144:0/4258011567 wait complete. 2024-09-10T12:08:20.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.339+0000 7f0220e9a700 1 Processor -- start 2024-09-10T12:08:20.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.339+0000 7f0220e9a700 1 -- start start 2024-09-10T12:08:20.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.339+0000 7f0220e9a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f021c1003d0 0x7f021c195d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:20.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.339+0000 7f0220e9a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f021c100d70 0x7f021c1962d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:20.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.339+0000 7f0220e9a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c1969b0 con 0x7f021c100d70 2024-09-10T12:08:20.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.339+0000 7f0220e9a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f021c19a740 con 0x7f021c1003d0 2024-09-10T12:08:20.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.339+0000 7f0219d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f021c100d70 0x7f021c1962d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:20.342 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.339+0000 7f0219d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f021c100d70 0x7f021c1962d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36124/0 (socket says 172.21.15.144:36124) 2024-09-10T12:08:20.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.339+0000 7f0219d9b700 1 -- 172.21.15.144:0/4135944945 learned_addr learned my addr 172.21.15.144:0/4135944945 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:08:20.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.339+0000 7f021a59c700 1 --2- 172.21.15.144:0/4135944945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f021c1003d0 0x7f021c195d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:20.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.339+0000 7f0219d9b700 1 -- 172.21.15.144:0/4135944945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f021c1003d0 msgr2=0x7f021c195d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:20.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.339+0000 7f0219d9b700 1 --2- 172.21.15.144:0/4135944945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f021c1003d0 0x7f021c195d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.339+0000 7f0219d9b700 1 -- 172.21.15.144:0/4135944945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0095f0 con 0x7f021c100d70 2024-09-10T12:08:20.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.339+0000 7f0219d9b700 1 --2- 172.21.15.144:0/4135944945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f021c100d70 0x7f021c1962d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f021000d8d0 tx=0x7f021000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:20.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.340+0000 7f020b7fe700 1 -- 172.21.15.144:0/4135944945 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0210004a00 con 0x7f021c100d70 2024-09-10T12:08:20.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.340+0000 7f020b7fe700 1 -- 172.21.15.144:0/4135944945 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f02100054e0 con 0x7f021c100d70 2024-09-10T12:08:20.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.340+0000 7f020b7fe700 1 -- 172.21.15.144:0/4135944945 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f021000fc20 con 0x7f021c100d70 2024-09-10T12:08:20.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.340+0000 7f0220e9a700 1 -- 172.21.15.144:0/4135944945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f021c19aa20 con 0x7f021c100d70 2024-09-10T12:08:20.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.340+0000 7f0220e9a700 1 -- 172.21.15.144:0/4135944945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f021c19af70 con 0x7f021c100d70 2024-09-10T12:08:20.343 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.341+0000 7f0220e9a700 1 -- 172.21.15.144:0/4135944945 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f021c1086e0 con 0x7f021c100d70 2024-09-10T12:08:20.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.341+0000 7f020b7fe700 1 -- 172.21.15.144:0/4135944945 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f021000fd80 con 0x7f021c100d70 2024-09-10T12:08:20.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.342+0000 7f020b7fe700 1 --2- 172.21.15.144:0/4135944945 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f020406e200 0x7f02040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:20.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.342+0000 7f021a59c700 1 --2- 172.21.15.144:0/4135944945 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f020406e200 0x7f02040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:20.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.342+0000 7f020b7fe700 1 -- 172.21.15.144:0/4135944945 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f021008d840 con 0x7f021c100d70 2024-09-10T12:08:20.344 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.342+0000 7f021a59c700 1 --2- 172.21.15.144:0/4135944945 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f020406e200 0x7f02040706b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f020c00b5c0 tx=0x7f020c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:20.347 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.346+0000 7f020b7fe700 1 -- 172.21.15.144:0/4135944945 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f0210056ca0 con 0x7f021c100d70 2024-09-10T12:08:20.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.566+0000 7f0220e9a700 1 -- 172.21.15.144:0/4135944945 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f021c19b2a0 con 0x7f020406e200 2024-09-10T12:08:20.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.567+0000 7f020b7fe700 1 -- 172.21.15.144:0/4135944945 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+431 (secure 0 0 0) 0x7f021c19b2a0 con 0x7f020406e200 2024-09-10T12:08:20.568 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:08:20.569 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559", 2024-09-10T12:08:20.569 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": true, 2024-09-10T12:08:20.569 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "Upgrading all daemon types on all hosts", 2024-09-10T12:08:20.569 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [ 2024-09-10T12:08:20.569 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr", 2024-09-10T12:08:20.569 INFO:teuthology.orchestra.run.smithi144.stdout: "mon", 2024-09-10T12:08:20.569 INFO:teuthology.orchestra.run.smithi144.stdout: "mds", 2024-09-10T12:08:20.569 INFO:teuthology.orchestra.run.smithi144.stdout: "crash", 2024-09-10T12:08:20.569 INFO:teuthology.orchestra.run.smithi144.stdout: "osd" 2024-09-10T12:08:20.569 INFO:teuthology.orchestra.run.smithi144.stdout: ], 2024-09-10T12:08:20.569 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": "16/23 daemons upgraded", 2024-09-10T12:08:20.569 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "Currently upgrading mds daemons", 2024-09-10T12:08:20.569 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:08:20.569 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:08:20.570 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.569+0000 7f0220e9a700 1 -- 172.21.15.144:0/4135944945 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f020406e200 msgr2=0x7f02040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:20.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.569+0000 7f0220e9a700 1 --2- 172.21.15.144:0/4135944945 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f020406e200 0x7f02040706b0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f020c00b5c0 tx=0x7f020c01a040 comp rx=0 tx=0).stop 2024-09-10T12:08:20.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.569+0000 7f0220e9a700 1 -- 172.21.15.144:0/4135944945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f021c100d70 msgr2=0x7f021c1962d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:20.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.569+0000 7f0220e9a700 1 --2- 172.21.15.144:0/4135944945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f021c100d70 0x7f021c1962d0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f021000d8d0 tx=0x7f021000dbe0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.569+0000 7f0220e9a700 1 -- 172.21.15.144:0/4135944945 shutdown_connections 2024-09-10T12:08:20.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.569+0000 7f0220e9a700 1 --2- 172.21.15.144:0/4135944945 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f020406e200 0x7f02040706b0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.569+0000 7f0220e9a700 1 --2- 172.21.15.144:0/4135944945 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f021c1003d0 0x7f021c195d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.569+0000 7f0220e9a700 1 --2- 172.21.15.144:0/4135944945 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f021c100d70 0x7f021c1962d0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.569+0000 7f0220e9a700 1 -- 172.21.15.144:0/4135944945 >> 172.21.15.144:0/4135944945 conn(0x7f021c0fbc90 msgr2=0x7f021c0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:20.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.569+0000 7f0220e9a700 1 -- 172.21.15.144:0/4135944945 shutdown_connections 2024-09-10T12:08:20.571 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.569+0000 7f0220e9a700 1 -- 172.21.15.144:0/4135944945 wait complete. 2024-09-10T12:08:20.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.674+0000 7f582c183700 1 -- 172.21.15.144:0/1635851555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5824100d80 msgr2=0x7f5824104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:20.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.674+0000 7f582c183700 1 --2- 172.21.15.144:0/1635851555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5824100d80 0x7f5824104d50 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f5820009910 tx=0x7f5820009c20 comp rx=0 tx=0).stop 2024-09-10T12:08:20.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.674+0000 7f582c183700 1 -- 172.21.15.144:0/1635851555 shutdown_connections 2024-09-10T12:08:20.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.674+0000 7f582c183700 1 --2- 172.21.15.144:0/1635851555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f5824100d80 0x7f5824104d50 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.674+0000 7f582c183700 1 --2- 172.21.15.144:0/1635851555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58241003e0 0x7f58241007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.674+0000 7f582c183700 1 -- 172.21.15.144:0/1635851555 >> 172.21.15.144:0/1635851555 conn(0x7f58240fbca0 msgr2=0x7f58240fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:20.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.674+0000 7f582c183700 1 -- 172.21.15.144:0/1635851555 shutdown_connections 2024-09-10T12:08:20.676 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.674+0000 7f582c183700 1 -- 172.21.15.144:0/1635851555 wait complete. 2024-09-10T12:08:20.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.675+0000 7f582c183700 1 Processor -- start 2024-09-10T12:08:20.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.675+0000 7f582c183700 1 -- start start 2024-09-10T12:08:20.678 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.675+0000 7f582c183700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58241003e0 0x7f5824195dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:20.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.675+0000 7f582c183700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5824100d80 0x7f5824196310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:20.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.675+0000 7f582c183700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58241969f0 con 0x7f58241003e0 2024-09-10T12:08:20.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.675+0000 7f582c183700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582419a780 con 0x7f5824100d80 2024-09-10T12:08:20.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.675+0000 7f5829f1f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58241003e0 0x7f5824195dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:20.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.675+0000 7f5829f1f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58241003e0 0x7f5824195dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36140/0 (socket says 172.21.15.144:36140) 2024-09-10T12:08:20.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.675+0000 7f5829f1f700 1 -- 172.21.15.144:0/3386899051 learned_addr learned my addr 172.21.15.144:0/3386899051 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:08:20.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.676+0000 7f582971e700 1 --2- 172.21.15.144:0/3386899051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5824100d80 0x7f5824196310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:20.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.676+0000 7f5829f1f700 1 -- 172.21.15.144:0/3386899051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5824100d80 msgr2=0x7f5824196310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:20.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.676+0000 7f5829f1f700 1 --2- 172.21.15.144:0/3386899051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5824100d80 0x7f5824196310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.676+0000 7f5829f1f700 1 -- 172.21.15.144:0/3386899051 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58200095f0 con 0x7f58241003e0 2024-09-10T12:08:20.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.676+0000 7f582971e700 1 --2- 172.21.15.144:0/3386899051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5824100d80 0x7f5824196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:08:20.679 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.676+0000 7f5829f1f700 1 --2- 172.21.15.144:0/3386899051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58241003e0 0x7f5824195dd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f581400c8a0 tx=0x7f581400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:20.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.676+0000 7f581affd700 1 -- 172.21.15.144:0/3386899051 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f581400cea0 con 0x7f58241003e0 2024-09-10T12:08:20.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.676+0000 7f581affd700 1 -- 172.21.15.144:0/3386899051 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5814004830 con 0x7f58241003e0 2024-09-10T12:08:20.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.676+0000 7f582c183700 1 -- 172.21.15.144:0/3386899051 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582419aa60 con 0x7f58241003e0 2024-09-10T12:08:20.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.676+0000 7f581affd700 1 -- 172.21.15.144:0/3386899051 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58140055b0 con 0x7f58241003e0 2024-09-10T12:08:20.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.676+0000 7f582c183700 1 -- 172.21.15.144:0/3386899051 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582419afb0 con 0x7f58241003e0 2024-09-10T12:08:20.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.677+0000 7f582c183700 1 -- 172.21.15.144:0/3386899051 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582404ea50 con 0x7f58241003e0 2024-09-10T12:08:20.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.678+0000 7f581affd700 1 -- 172.21.15.144:0/3386899051 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f58140049a0 con 0x7f58241003e0 2024-09-10T12:08:20.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.678+0000 7f581affd700 1 --2- 172.21.15.144:0/3386899051 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f581006e200 0x7f58100706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:20.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.678+0000 7f582971e700 1 --2- 172.21.15.144:0/3386899051 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f581006e200 0x7f58100706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:20.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.678+0000 7f581affd700 1 -- 172.21.15.144:0/3386899051 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f581408dd40 con 0x7f58241003e0 2024-09-10T12:08:20.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.679+0000 7f582971e700 1 --2- 172.21.15.144:0/3386899051 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f581006e200 0x7f58100706b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f582000b5c0 tx=0x7f5820005480 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:20.684 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.683+0000 7f581affd700 1 -- 172.21.15.144:0/3386899051 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5814057120 con 0x7f58241003e0 2024-09-10T12:08:20.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:20 smithi144 ceph-mon[87436]: pgmap v1091: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:20.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:20 smithi144 ceph-mon[87436]: from='client.35972 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:20.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:20 smithi144 ceph-mon[87436]: from='client.35976 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:20.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:20 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1090209937' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:20.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:20 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3485119776' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:08:20.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.972+0000 7f582c183700 1 -- 172.21.15.144:0/3386899051 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f5824197130 con 0x7f58241003e0 2024-09-10T12:08:20.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.972+0000 7f581affd700 1 -- 172.21.15.144:0/3386899051 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+3795 (secure 0 0 0) 0x7f581401d090 con 0x7f58241003e0 2024-09-10T12:08:20.974 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_ERR failed to probe daemons or devices; 1 filesystem is degraded; 1 filesystem with deprecated feature inline_data; 1 filesystem is offline 2024-09-10T12:08:20.974 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] CEPHADM_REFRESH_FAILED: failed to probe daemons or devices 2024-09-10T12:08:20.974 INFO:teuthology.orchestra.run.smithi144.stdout: host smithi178 `cephadm ceph-volume` failed: cephadm exited with an error code: 1, stderr: Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi178/config 2024-09-10T12:08:20.974 INFO:teuthology.orchestra.run.smithi144.stdout:/usr/bin/podman: timeout after 895 seconds 2024-09-10T12:08:20.974 INFO:teuthology.orchestra.run.smithi144.stdout:Non-zero exit code 124 from /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:08:20.974 INFO:teuthology.orchestra.run.smithi144.stdout:Traceback (most recent call last): 2024-09-10T12:08:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main 2024-09-10T12:08:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: return _run_code(code, main_globals, None, 2024-09-10T12:08:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code 2024-09-10T12:08:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: exec(code, run_globals) 2024-09-10T12:08:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5579, in 2024-09-10T12:08:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 5567, in main 2024-09-10T12:08:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 409, in _infer_config 2024-09-10T12:08:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 324, in _infer_fsid 2024-09-10T12:08:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 437, in _infer_image 2024-09-10T12:08:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 311, in _validate_fsid 2024-09-10T12:08:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/__main__.py", line 3312, in command_ceph_volume 2024-09-10T12:08:20.975 INFO:teuthology.orchestra.run.smithi144.stdout: File "/tmp/tmp1vs9i6c0.cephadm.build/app/cephadmlib/call_wrappers.py", line 310, in call_throws 2024-09-10T12:08:20.975 INFO:teuthology.orchestra.run.smithi144.stdout:RuntimeError: Failed command: /usr/bin/podman run --rm --ipc=host --stop-signal=SIGTERM --net=host --entrypoint /usr/sbin/ceph-volume --privileged --group-add=disk --init -e CONTAINER_IMAGE=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 -e NODE_NAME=smithi178 -e CEPH_VOLUME_SKIP_RESTORECON=yes -e CEPH_VOLUME_DEBUG=1 -v /var/run/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/run/ceph:z -v /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968:/var/log/ceph:z -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash:/var/lib/ceph/crash:z -v /run/systemd/journal:/run/systemd/journal -v /dev:/dev -v /run/udev:/run/udev -v /sys:/sys -v /run/lvm:/run/lvm -v /run/lock/lvm:/run/lock/lvm -v /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/selinux:/sys/fs/selinux:ro -v /:/rootfs:rslave -v /etc/hosts:/etc/hosts:ro -v /tmp/ceph-tmp3cgj990c:/etc/ceph/ceph.conf:z quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 inventory --format=json-pretty --filter-for-batch 2024-09-10T12:08:20.976 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-09-10T12:08:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is degraded 2024-09-10T12:08:20.976 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:08:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:08:20.976 INFO:teuthology.orchestra.run.smithi144.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-09-10T12:08:20.976 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs is offline because no MDS is active for it. 2024-09-10T12:08:20.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.975+0000 7f582c183700 1 -- 172.21.15.144:0/3386899051 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f581006e200 msgr2=0x7f58100706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:20.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.975+0000 7f582c183700 1 --2- 172.21.15.144:0/3386899051 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f581006e200 0x7f58100706b0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f582000b5c0 tx=0x7f5820005480 comp rx=0 tx=0).stop 2024-09-10T12:08:20.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.975+0000 7f582c183700 1 -- 172.21.15.144:0/3386899051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58241003e0 msgr2=0x7f5824195dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:20.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.975+0000 7f582c183700 1 --2- 172.21.15.144:0/3386899051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58241003e0 0x7f5824195dd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f581400c8a0 tx=0x7f581400cc60 comp rx=0 tx=0).stop 2024-09-10T12:08:20.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.975+0000 7f582c183700 1 -- 172.21.15.144:0/3386899051 shutdown_connections 2024-09-10T12:08:20.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.975+0000 7f582c183700 1 --2- 172.21.15.144:0/3386899051 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f581006e200 0x7f58100706b0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.975+0000 7f582c183700 1 --2- 172.21.15.144:0/3386899051 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58241003e0 0x7f5824195dd0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.975+0000 7f582c183700 1 --2- 172.21.15.144:0/3386899051 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f5824100d80 0x7f5824196310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:20.977 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.975+0000 7f582c183700 1 -- 172.21.15.144:0/3386899051 >> 172.21.15.144:0/3386899051 conn(0x7f58240fbca0 msgr2=0x7f58240fd2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:20.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.975+0000 7f582c183700 1 -- 172.21.15.144:0/3386899051 shutdown_connections 2024-09-10T12:08:20.978 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:20.975+0000 7f582c183700 1 -- 172.21.15.144:0/3386899051 wait complete. 2024-09-10T12:08:20.989 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:20 smithi178 ceph-mon[65940]: pgmap v1091: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:20.989 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:20 smithi178 ceph-mon[65940]: from='client.35972 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:20.989 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:20 smithi178 ceph-mon[65940]: from='client.35976 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:20.989 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:20 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1090209937' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:20.989 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:20 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3485119776' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:08:22.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:21 smithi178 ceph-mon[65940]: from='client.35988 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:22.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:21 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3386899051' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:08:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:21 smithi144 ceph-mon[87436]: from='client.35988 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:21 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3386899051' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:08:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: pgmap v1092: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:08:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:08:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:08:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:22.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:22.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:22.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:22.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:22.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T12:08:22.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.bfyvwv"}]': finished 2024-09-10T12:08:22.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.jaoyvs"}]: dispatch 2024-09-10T12:08:22.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.jaoyvs"}]': finished 2024-09-10T12:08:22.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi178.czjrsa"}]: dispatch 2024-09-10T12:08:22.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi178.czjrsa"}]': finished 2024-09-10T12:08:22.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi178.nmhrmh"}]: dispatch 2024-09-10T12:08:22.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi178.nmhrmh"}]': finished 2024-09-10T12:08:22.935 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:22 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-09-10T12:08:23.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: pgmap v1092: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:23.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:08:23.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:08:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:08:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.bfyvwv"}]: dispatch 2024-09-10T12:08:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.bfyvwv"}]': finished 2024-09-10T12:08:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.jaoyvs"}]: dispatch 2024-09-10T12:08:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi144.jaoyvs"}]': finished 2024-09-10T12:08:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi178.czjrsa"}]: dispatch 2024-09-10T12:08:23.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi178.czjrsa"}]': finished 2024-09-10T12:08:23.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi178.nmhrmh"}]: dispatch 2024-09-10T12:08:23.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi178.nmhrmh"}]': finished 2024-09-10T12:08:23.119 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:22 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-09-10T12:08:24.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:23 smithi178 ceph-mon[65940]: pgmap v1093: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:24.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:23 smithi178 ceph-mon[65940]: pgmap v1094: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:24.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:23 smithi178 ceph-mon[65940]: Upgrade: Setting container_image for all mds 2024-09-10T12:08:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:23 smithi178 ceph-mon[65940]: Upgrade: Setting filesystem cephfs Joinable 2024-09-10T12:08:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:23 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-10T12:08:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:23 smithi178 ceph-mon[65940]: Standby daemon mds.cephfs.smithi144.bfyvwv assigned to filesystem cephfs as rank 0 2024-09-10T12:08:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:23 smithi178 ceph-mon[65940]: Standby daemon mds.cephfs.smithi144.jaoyvs assigned to filesystem cephfs as rank 1 2024-09-10T12:08:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:23 smithi178 ceph-mon[65940]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T12:08:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:23 smithi178 ceph-mon[65940]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T12:08:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:23 smithi178 ceph-mon[65940]: fsmap cephfs:2/2 {0=cephfs.smithi144.bfyvwv=up:replay,1=cephfs.smithi144.jaoyvs=up:replay} 2 up:standby 2024-09-10T12:08:24.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:23 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-09-10T12:08:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:23 smithi144 ceph-mon[87436]: pgmap v1093: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:23 smithi144 ceph-mon[87436]: pgmap v1094: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:23 smithi144 ceph-mon[87436]: Upgrade: Setting container_image for all mds 2024-09-10T12:08:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:23 smithi144 ceph-mon[87436]: Upgrade: Setting filesystem cephfs Joinable 2024-09-10T12:08:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-10T12:08:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:23 smithi144 ceph-mon[87436]: Standby daemon mds.cephfs.smithi144.bfyvwv assigned to filesystem cephfs as rank 0 2024-09-10T12:08:24.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:23 smithi144 ceph-mon[87436]: Standby daemon mds.cephfs.smithi144.jaoyvs assigned to filesystem cephfs as rank 1 2024-09-10T12:08:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:23 smithi144 ceph-mon[87436]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-10T12:08:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:23 smithi144 ceph-mon[87436]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-09-10T12:08:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:23 smithi144 ceph-mon[87436]: fsmap cephfs:2/2 {0=cephfs.smithi144.bfyvwv=up:replay,1=cephfs.smithi144.jaoyvs=up:replay} 2 up:standby 2024-09-10T12:08:24.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:23 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]: dispatch 2024-09-10T12:08:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:24 smithi178 ceph-mon[65940]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-09-10T12:08:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:24 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-09-10T12:08:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:24 smithi178 ceph-mon[65940]: fsmap cephfs:2/2 {0=cephfs.smithi144.bfyvwv=up:replay,1=cephfs.smithi144.jaoyvs=up:replay} 2 up:standby 2024-09-10T12:08:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:24 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:24 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:25.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:24 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:25.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:24 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:25.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:24 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:25.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:24 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:25.169 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:24 smithi144 ceph-mon[87436]: Upgrade: Enabling allow_standby_replay on filesystem cephfs 2024-09-10T12:08:25.169 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:24 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "1"}]': finished 2024-09-10T12:08:25.169 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:24 smithi144 ceph-mon[87436]: fsmap cephfs:2/2 {0=cephfs.smithi144.bfyvwv=up:replay,1=cephfs.smithi144.jaoyvs=up:replay} 2 up:standby 2024-09-10T12:08:25.169 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:24 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:25.169 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:24 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:25.169 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:24 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:25.169 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:24 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:25.169 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:24 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:25.169 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:24 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:26.040 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:25 smithi178 ceph-mon[65940]: pgmap v1095: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:26.040 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:25 smithi178 ceph-mon[65940]: Upgrade: Setting container_image for all rgw 2024-09-10T12:08:26.041 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:25 smithi178 ceph-mon[65940]: Upgrade: Setting container_image for all rbd-mirror 2024-09-10T12:08:26.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:25 smithi144 ceph-mon[87436]: pgmap v1095: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-09-10T12:08:26.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:25 smithi144 ceph-mon[87436]: Upgrade: Setting container_image for all rgw 2024-09-10T12:08:26.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:25 smithi144 ceph-mon[87436]: Upgrade: Setting container_image for all rbd-mirror 2024-09-10T12:08:27.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:27 smithi178 ceph-mon[65940]: Upgrade: Updating ceph-exporter.smithi144 (1/2) 2024-09-10T12:08:27.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:27 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:27.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:27 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T12:08:27.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:27 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:08:27.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:27 smithi178 ceph-mon[65940]: Deploying daemon ceph-exporter.smithi144 on smithi144 2024-09-10T12:08:27.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:27 smithi178 ceph-mon[65940]: pgmap v1096: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 9.3 MiB/s rd, 3 op/s 2024-09-10T12:08:27.936 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:27 smithi144 ceph-mon[87436]: Upgrade: Updating ceph-exporter.smithi144 (1/2) 2024-09-10T12:08:27.937 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:27 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:27.937 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:27 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi144", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T12:08:27.937 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:27 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:08:27.937 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:27 smithi144 ceph-mon[87436]: Deploying daemon ceph-exporter.smithi144 on smithi144 2024-09-10T12:08:27.937 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:27 smithi144 ceph-mon[87436]: pgmap v1096: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 9.3 MiB/s rd, 3 op/s 2024-09-10T12:08:29.860 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:29 smithi144 ceph-mon[87436]: pgmap v1097: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-09-10T12:08:29.860 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:29 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:29.860 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:29 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:08:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:29 smithi178 ceph-mon[65940]: pgmap v1097: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 27 MiB/s rd, 9 op/s 2024-09-10T12:08:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:29 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:30.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:29 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:08:30.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:30 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] up:resolve 2024-09-10T12:08:30.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:30 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] up:resolve 2024-09-10T12:08:30.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:30 smithi144 ceph-mon[87436]: fsmap cephfs:2/2 {0=cephfs.smithi144.bfyvwv=up:resolve,1=cephfs.smithi144.jaoyvs=up:resolve} 2 up:standby 2024-09-10T12:08:31.091 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:30 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] up:resolve 2024-09-10T12:08:31.108 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:30 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] up:resolve 2024-09-10T12:08:31.108 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:30 smithi178 ceph-mon[65940]: fsmap cephfs:2/2 {0=cephfs.smithi144.bfyvwv=up:resolve,1=cephfs.smithi144.jaoyvs=up:resolve} 2 up:standby 2024-09-10T12:08:31.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:31 smithi144 ceph-mon[87436]: pgmap v1098: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 28 MiB/s rd, 9 op/s 2024-09-10T12:08:31.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:31 smithi144 ceph-mon[87436]: reconnect by client.24305 192.168.0.1:0/2242343795 after 0.000999982 2024-09-10T12:08:31.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:31 smithi144 ceph-mon[87436]: reconnect by client.24305 192.168.0.1:0/2242343795 after 0.000999982 2024-09-10T12:08:31.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:31 smithi144 ceph-mon[87436]: reconnect by client.14522 192.168.0.1:0/3895676988 after 0.00199996 2024-09-10T12:08:31.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:31 smithi144 ceph-mon[87436]: reconnect by client.14522 192.168.0.1:0/3895676988 after 0.00199996 2024-09-10T12:08:31.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:31 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] up:reconnect 2024-09-10T12:08:31.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:31 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] up:reconnect 2024-09-10T12:08:31.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:31 smithi144 ceph-mon[87436]: fsmap cephfs:2/2 {0=cephfs.smithi144.bfyvwv=up:reconnect,1=cephfs.smithi144.jaoyvs=up:reconnect} 2 up:standby 2024-09-10T12:08:32.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:31 smithi178 ceph-mon[65940]: pgmap v1098: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 28 MiB/s rd, 9 op/s 2024-09-10T12:08:32.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:31 smithi178 ceph-mon[65940]: reconnect by client.24305 192.168.0.1:0/2242343795 after 0.000999982 2024-09-10T12:08:32.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:31 smithi178 ceph-mon[65940]: reconnect by client.24305 192.168.0.1:0/2242343795 after 0.000999982 2024-09-10T12:08:32.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:31 smithi178 ceph-mon[65940]: reconnect by client.14522 192.168.0.1:0/3895676988 after 0.00199996 2024-09-10T12:08:32.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:31 smithi178 ceph-mon[65940]: reconnect by client.14522 192.168.0.1:0/3895676988 after 0.00199996 2024-09-10T12:08:32.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:31 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] up:reconnect 2024-09-10T12:08:32.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:31 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] up:reconnect 2024-09-10T12:08:32.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:31 smithi178 ceph-mon[65940]: fsmap cephfs:2/2 {0=cephfs.smithi144.bfyvwv=up:reconnect,1=cephfs.smithi144.jaoyvs=up:reconnect} 2 up:standby 2024-09-10T12:08:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:32 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] up:rejoin 2024-09-10T12:08:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:32 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] up:rejoin 2024-09-10T12:08:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:32 smithi144 ceph-mon[87436]: fsmap cephfs:2/2 {0=cephfs.smithi144.bfyvwv=up:rejoin,1=cephfs.smithi144.jaoyvs=up:rejoin} 2 up:standby 2024-09-10T12:08:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:32 smithi144 ceph-mon[87436]: daemon mds.cephfs.smithi144.bfyvwv is now active in filesystem cephfs as rank 0 2024-09-10T12:08:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:32 smithi144 ceph-mon[87436]: daemon mds.cephfs.smithi144.jaoyvs is now active in filesystem cephfs as rank 1 2024-09-10T12:08:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:32 smithi144 ceph-mon[87436]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T12:08:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:32 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] up:active 2024-09-10T12:08:33.115 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:32 smithi144 ceph-mon[87436]: mds.? [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] up:active 2024-09-10T12:08:33.115 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:32 smithi144 ceph-mon[87436]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi144.jaoyvs=up:active} 2 up:standby-replay 2024-09-10T12:08:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:32 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] up:rejoin 2024-09-10T12:08:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:32 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] up:rejoin 2024-09-10T12:08:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:32 smithi178 ceph-mon[65940]: fsmap cephfs:2/2 {0=cephfs.smithi144.bfyvwv=up:rejoin,1=cephfs.smithi144.jaoyvs=up:rejoin} 2 up:standby 2024-09-10T12:08:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:32 smithi178 ceph-mon[65940]: daemon mds.cephfs.smithi144.bfyvwv is now active in filesystem cephfs as rank 0 2024-09-10T12:08:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:32 smithi178 ceph-mon[65940]: daemon mds.cephfs.smithi144.jaoyvs is now active in filesystem cephfs as rank 1 2024-09-10T12:08:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:32 smithi178 ceph-mon[65940]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-10T12:08:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:32 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] up:active 2024-09-10T12:08:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:32 smithi178 ceph-mon[65940]: mds.? [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] up:active 2024-09-10T12:08:33.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:32 smithi178 ceph-mon[65940]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi144.jaoyvs=up:active} 2 up:standby-replay 2024-09-10T12:08:33.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:33 smithi144 ceph-mon[87436]: pgmap v1099: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31 MiB/s rd, 204 B/s wr, 10 op/s 2024-09-10T12:08:33.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:33.706 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:33 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:33 smithi178 ceph-mon[65940]: pgmap v1099: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 31 MiB/s rd, 204 B/s wr, 10 op/s 2024-09-10T12:08:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:34.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:33 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:35.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:35 smithi178 ceph-mon[65940]: pgmap v1100: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 28 MiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T12:08:35.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:35 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:35.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:35 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi178", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T12:08:35.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:35 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:08:36.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:35 smithi144 ceph-mon[87436]: pgmap v1100: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 28 MiB/s rd, 255 B/s wr, 11 op/s 2024-09-10T12:08:36.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:35 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:36.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:35 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi178", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-09-10T12:08:36.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:35 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:08:37.051 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:36 smithi178 ceph-mon[65940]: Upgrade: Updating ceph-exporter.smithi178 (2/2) 2024-09-10T12:08:37.051 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:36 smithi178 ceph-mon[65940]: Deploying daemon ceph-exporter.smithi178 on smithi178 2024-09-10T12:08:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:36 smithi144 ceph-mon[87436]: Upgrade: Updating ceph-exporter.smithi178 (2/2) 2024-09-10T12:08:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:36 smithi144 ceph-mon[87436]: Deploying daemon ceph-exporter.smithi178 on smithi178 2024-09-10T12:08:37.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:37 smithi178 ceph-mon[65940]: pgmap v1101: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38 MiB/s rd, 9.8 KiB/s wr, 21 op/s 2024-09-10T12:08:38.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:37 smithi144 ceph-mon[87436]: pgmap v1101: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 38 MiB/s rd, 9.8 KiB/s wr, 21 op/s 2024-09-10T12:08:40.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:39 smithi178 ceph-mon[65940]: pgmap v1102: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 42 MiB/s rd, 9.8 KiB/s wr, 22 op/s 2024-09-10T12:08:40.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:39 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:40.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:39 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:40.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:39 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:08:40.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:39 smithi144 ceph-mon[87436]: pgmap v1102: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 42 MiB/s rd, 9.8 KiB/s wr, 22 op/s 2024-09-10T12:08:40.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:39 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:40.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:39 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:40.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:39 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:08:41.098 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:40 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:41.098 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:40 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:41.098 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:40 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:41.098 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:40 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:41.152 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:40 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:41.152 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:40 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:41.152 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:40 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:41.153 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:40 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:42.060 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:41 smithi178 ceph-mon[65940]: pgmap v1103: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 29 MiB/s rd, 9.8 KiB/s wr, 18 op/s 2024-09-10T12:08:42.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:41 smithi144 ceph-mon[87436]: pgmap v1103: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 29 MiB/s rd, 9.8 KiB/s wr, 18 op/s 2024-09-10T12:08:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:43 smithi144 ceph-mon[87436]: pgmap v1104: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 29 MiB/s rd, 14 KiB/s wr, 21 op/s 2024-09-10T12:08:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:08:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:43 smithi178 ceph-mon[65940]: pgmap v1104: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 29 MiB/s rd, 14 KiB/s wr, 21 op/s 2024-09-10T12:08:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:08:45.249 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:45.250 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:45.250 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:45.250 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:46.249 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:46 smithi144 ceph-mon[87436]: pgmap v1105: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 14 KiB/s wr, 20 op/s 2024-09-10T12:08:46.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:46 smithi178 ceph-mon[65940]: pgmap v1105: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 25 MiB/s rd, 14 KiB/s wr, 20 op/s 2024-09-10T12:08:48.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:48 smithi178 ceph-mon[65940]: pgmap v1106: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-10T12:08:48.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:48 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:48.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:48 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:48.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:48 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:48.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:48 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:48.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:48 smithi144 ceph-mon[87436]: pgmap v1106: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 14 KiB/s wr, 17 op/s 2024-09-10T12:08:48.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:48 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:48.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:48 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:48.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:48 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:48.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:48 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:49.557 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:49 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:49.557 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:49 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:49.557 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:49 smithi178 ceph-mon[65940]: pgmap v1107: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-10T12:08:49.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:49 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:49.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:49 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:49.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:49 smithi144 ceph-mon[87436]: pgmap v1107: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 4.3 KiB/s wr, 9 op/s 2024-09-10T12:08:51.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.095+0000 7f913c22b700 1 -- 172.21.15.144:0/3777806411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91340ff2c0 msgr2=0x7f91340ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:51.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.095+0000 7f913c22b700 1 --2- 172.21.15.144:0/3777806411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91340ff2c0 0x7f91340ff690 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f912c00b0a0 tx=0x7f912c00b3b0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.095+0000 7f913c22b700 1 -- 172.21.15.144:0/3777806411 shutdown_connections 2024-09-10T12:08:51.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.095+0000 7f913c22b700 1 --2- 172.21.15.144:0/3777806411 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91340ffc60 0x7f9134103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.095+0000 7f913c22b700 1 --2- 172.21.15.144:0/3777806411 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91340ff2c0 0x7f91340ff690 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.095+0000 7f913c22b700 1 -- 172.21.15.144:0/3777806411 >> 172.21.15.144:0/3777806411 conn(0x7f91340fab80 msgr2=0x7f91340fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:51.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.095+0000 7f913c22b700 1 -- 172.21.15.144:0/3777806411 shutdown_connections 2024-09-10T12:08:51.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.095+0000 7f913c22b700 1 -- 172.21.15.144:0/3777806411 wait complete. 2024-09-10T12:08:51.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.096+0000 7f913c22b700 1 Processor -- start 2024-09-10T12:08:51.098 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.096+0000 7f913c22b700 1 -- start start 2024-09-10T12:08:51.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.096+0000 7f913c22b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91340ff2c0 0x7f9134195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:51.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.096+0000 7f913c22b700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91340ffc60 0x7f91341962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:51.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.096+0000 7f913c22b700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91341969c0 con 0x7f91340ff2c0 2024-09-10T12:08:51.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.096+0000 7f913c22b700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913419a750 con 0x7f91340ffc60 2024-09-10T12:08:51.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.096+0000 7f9139fc7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91340ff2c0 0x7f9134195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:51.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.097+0000 7f9139fc7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91340ff2c0 0x7f9134195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57496/0 (socket says 172.21.15.144:57496) 2024-09-10T12:08:51.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.097+0000 7f9139fc7700 1 -- 172.21.15.144:0/2902744695 learned_addr learned my addr 172.21.15.144:0/2902744695 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:08:51.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.097+0000 7f91397c6700 1 --2- 172.21.15.144:0/2902744695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91340ffc60 0x7f91341962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:51.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.097+0000 7f9139fc7700 1 -- 172.21.15.144:0/2902744695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91340ffc60 msgr2=0x7f91341962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:51.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.097+0000 7f9139fc7700 1 --2- 172.21.15.144:0/2902744695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91340ffc60 0x7f91341962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.097+0000 7f9139fc7700 1 -- 172.21.15.144:0/2902744695 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c009d00 con 0x7f91340ff2c0 2024-09-10T12:08:51.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.097+0000 7f91397c6700 1 --2- 172.21.15.144:0/2902744695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91340ffc60 0x7f91341962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:08:51.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.097+0000 7f9139fc7700 1 --2- 172.21.15.144:0/2902744695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91340ff2c0 0x7f9134195da0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f912c00b070 tx=0x7f912c00bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:51.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.097+0000 7f9132ffd700 1 -- 172.21.15.144:0/2902744695 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f912c00e040 con 0x7f91340ff2c0 2024-09-10T12:08:51.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.097+0000 7f9132ffd700 1 -- 172.21.15.144:0/2902744695 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f912c003c10 con 0x7f91340ff2c0 2024-09-10T12:08:51.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.097+0000 7f913c22b700 1 -- 172.21.15.144:0/2902744695 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913419a9d0 con 0x7f91340ff2c0 2024-09-10T12:08:51.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.097+0000 7f9132ffd700 1 -- 172.21.15.144:0/2902744695 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f912c01d6a0 con 0x7f91340ff2c0 2024-09-10T12:08:51.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.097+0000 7f913c22b700 1 -- 172.21.15.144:0/2902744695 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913419aec0 con 0x7f91340ff2c0 2024-09-10T12:08:51.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.098+0000 7f913c22b700 1 -- 172.21.15.144:0/2902744695 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913404ea50 con 0x7f91340ff2c0 2024-09-10T12:08:51.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.099+0000 7f9132ffd700 1 -- 172.21.15.144:0/2902744695 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f912c019070 con 0x7f91340ff2c0 2024-09-10T12:08:51.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.099+0000 7f9132ffd700 1 --2- 172.21.15.144:0/2902744695 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f911c06e200 0x7f911c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:51.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.099+0000 7f9132ffd700 1 -- 172.21.15.144:0/2902744695 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f912c08f4e0 con 0x7f91340ff2c0 2024-09-10T12:08:51.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.099+0000 7f91397c6700 1 --2- 172.21.15.144:0/2902744695 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f911c06e200 0x7f911c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:51.102 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.100+0000 7f91397c6700 1 --2- 172.21.15.144:0/2902744695 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f911c06e200 0x7f911c0706b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f91341973c0 tx=0x7f9128009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:51.105 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.104+0000 7f9132ffd700 1 -- 172.21.15.144:0/2902744695 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f912c058940 con 0x7f91340ff2c0 2024-09-10T12:08:51.326 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.324+0000 7f913c22b700 1 -- 172.21.15.144:0/2902744695 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f913419b390 con 0x7f911c06e200 2024-09-10T12:08:51.328 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.327+0000 7f9132ffd700 1 -- 172.21.15.144:0/2902744695 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f913419b390 con 0x7f911c06e200 2024-09-10T12:08:51.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.329+0000 7f913c22b700 1 -- 172.21.15.144:0/2902744695 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f911c06e200 msgr2=0x7f911c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:51.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.329+0000 7f913c22b700 1 --2- 172.21.15.144:0/2902744695 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f911c06e200 0x7f911c0706b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f91341973c0 tx=0x7f9128009260 comp rx=0 tx=0).stop 2024-09-10T12:08:51.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.329+0000 7f913c22b700 1 -- 172.21.15.144:0/2902744695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91340ff2c0 msgr2=0x7f9134195da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:51.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.329+0000 7f913c22b700 1 --2- 172.21.15.144:0/2902744695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91340ff2c0 0x7f9134195da0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f912c00b070 tx=0x7f912c00bd10 comp rx=0 tx=0).stop 2024-09-10T12:08:51.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.329+0000 7f913c22b700 1 -- 172.21.15.144:0/2902744695 shutdown_connections 2024-09-10T12:08:51.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.329+0000 7f913c22b700 1 --2- 172.21.15.144:0/2902744695 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f911c06e200 0x7f911c0706b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.329+0000 7f913c22b700 1 --2- 172.21.15.144:0/2902744695 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f91340ff2c0 0x7f9134195da0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.329+0000 7f913c22b700 1 --2- 172.21.15.144:0/2902744695 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f91340ffc60 0x7f91341962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.329+0000 7f913c22b700 1 -- 172.21.15.144:0/2902744695 >> 172.21.15.144:0/2902744695 conn(0x7f91340fab80 msgr2=0x7f9134068d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:51.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.329+0000 7f913c22b700 1 -- 172.21.15.144:0/2902744695 shutdown_connections 2024-09-10T12:08:51.331 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.329+0000 7f913c22b700 1 -- 172.21.15.144:0/2902744695 wait complete. 2024-09-10T12:08:51.342 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:08:51.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.446+0000 7f31cd139700 1 -- 172.21.15.144:0/3439283878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c80ff220 msgr2=0x7f31c80ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:51.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.446+0000 7f31cd139700 1 --2- 172.21.15.144:0/3439283878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c80ff220 0x7f31c80ff690 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f31b800aa20 tx=0x7f31b800ad30 comp rx=0 tx=0).stop 2024-09-10T12:08:51.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.447+0000 7f31cd139700 1 -- 172.21.15.144:0/3439283878 shutdown_connections 2024-09-10T12:08:51.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.447+0000 7f31cd139700 1 --2- 172.21.15.144:0/3439283878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c80ff220 0x7f31c80ff690 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.447+0000 7f31cd139700 1 --2- 172.21.15.144:0/3439283878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c8104870 0x7f31c8104c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.447+0000 7f31cd139700 1 -- 172.21.15.144:0/3439283878 >> 172.21.15.144:0/3439283878 conn(0x7f31c80747e0 msgr2=0x7f31c8074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:51.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.447+0000 7f31cd139700 1 -- 172.21.15.144:0/3439283878 shutdown_connections 2024-09-10T12:08:51.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.447+0000 7f31cd139700 1 -- 172.21.15.144:0/3439283878 wait complete. 2024-09-10T12:08:51.449 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.447+0000 7f31cd139700 1 Processor -- start 2024-09-10T12:08:51.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.448+0000 7f31cd139700 1 -- start start 2024-09-10T12:08:51.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.448+0000 7f31cd139700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c80ff220 0x7f31c8195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:51.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.448+0000 7f31cd139700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c8104870 0x7f31c8196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:51.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.448+0000 7f31cd139700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c8196890 con 0x7f31c80ff220 2024-09-10T12:08:51.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.448+0000 7f31cd139700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31c81995c0 con 0x7f31c8104870 2024-09-10T12:08:51.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.448+0000 7f31c6d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c80ff220 0x7f31c8195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:51.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.448+0000 7f31c6d9d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c80ff220 0x7f31c8195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57506/0 (socket says 172.21.15.144:57506) 2024-09-10T12:08:51.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.448+0000 7f31c6d9d700 1 -- 172.21.15.144:0/1051316486 learned_addr learned my addr 172.21.15.144:0/1051316486 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:08:51.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.448+0000 7f31c659c700 1 --2- 172.21.15.144:0/1051316486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c8104870 0x7f31c8196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:51.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.448+0000 7f31c6d9d700 1 -- 172.21.15.144:0/1051316486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c8104870 msgr2=0x7f31c8196200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:51.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.448+0000 7f31c6d9d700 1 --2- 172.21.15.144:0/1051316486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c8104870 0x7f31c8196200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.448+0000 7f31c6d9d700 1 -- 172.21.15.144:0/1051316486 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b800a700 con 0x7f31c80ff220 2024-09-10T12:08:51.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.448+0000 7f31c659c700 1 --2- 172.21.15.144:0/1051316486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c8104870 0x7f31c8196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:08:51.451 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.448+0000 7f31c6d9d700 1 --2- 172.21.15.144:0/1051316486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c80ff220 0x7f31c8195cc0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00e920 tx=0x7f31bc00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:51.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.449+0000 7f31affff700 1 -- 172.21.15.144:0/1051316486 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31bc009790 con 0x7f31c80ff220 2024-09-10T12:08:51.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.449+0000 7f31cd139700 1 -- 172.21.15.144:0/1051316486 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31c81998a0 con 0x7f31c80ff220 2024-09-10T12:08:51.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.449+0000 7f31affff700 1 -- 172.21.15.144:0/1051316486 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f31bc00cba0 con 0x7f31c80ff220 2024-09-10T12:08:51.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.449+0000 7f31affff700 1 -- 172.21.15.144:0/1051316486 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31bc018650 con 0x7f31c80ff220 2024-09-10T12:08:51.452 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.449+0000 7f31cd139700 1 -- 172.21.15.144:0/1051316486 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31c8199df0 con 0x7f31c80ff220 2024-09-10T12:08:51.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.450+0000 7f31cd139700 1 -- 172.21.15.144:0/1051316486 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31c8101c30 con 0x7f31c80ff220 2024-09-10T12:08:51.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.450+0000 7f31affff700 1 -- 172.21.15.144:0/1051316486 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f31bc0188c0 con 0x7f31c80ff220 2024-09-10T12:08:51.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.450+0000 7f31affff700 1 --2- 172.21.15.144:0/1051316486 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f31b006e200 0x7f31b00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:51.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.451+0000 7f31affff700 1 -- 172.21.15.144:0/1051316486 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f31bc014070 con 0x7f31c80ff220 2024-09-10T12:08:51.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.451+0000 7f31c659c700 1 --2- 172.21.15.144:0/1051316486 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f31b006e200 0x7f31b00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:51.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.451+0000 7f31c659c700 1 --2- 172.21.15.144:0/1051316486 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f31b006e200 0x7f31b00706b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f31b800d010 tx=0x7f31b800c650 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:51.456 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.455+0000 7f31affff700 1 -- 172.21.15.144:0/1051316486 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f31bc0541b0 con 0x7f31c80ff220 2024-09-10T12:08:51.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.675+0000 7f31cd139700 1 -- 172.21.15.144:0/1051316486 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31c819a060 con 0x7f31b006e200 2024-09-10T12:08:51.677 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.676+0000 7f31affff700 1 -- 172.21.15.144:0/1051316486 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+466 (secure 0 0 0) 0x7f31c819a060 con 0x7f31b006e200 2024-09-10T12:08:51.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.679+0000 7f31cd139700 1 -- 172.21.15.144:0/1051316486 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f31b006e200 msgr2=0x7f31b00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:51.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.679+0000 7f31cd139700 1 --2- 172.21.15.144:0/1051316486 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f31b006e200 0x7f31b00706b0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f31b800d010 tx=0x7f31b800c650 comp rx=0 tx=0).stop 2024-09-10T12:08:51.680 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.679+0000 7f31cd139700 1 -- 172.21.15.144:0/1051316486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c80ff220 msgr2=0x7f31c8195cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:51.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.679+0000 7f31cd139700 1 --2- 172.21.15.144:0/1051316486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c80ff220 0x7f31c8195cc0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f31bc00e920 tx=0x7f31bc00ece0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.679+0000 7f31cd139700 1 -- 172.21.15.144:0/1051316486 shutdown_connections 2024-09-10T12:08:51.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.679+0000 7f31cd139700 1 --2- 172.21.15.144:0/1051316486 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f31b006e200 0x7f31b00706b0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.679+0000 7f31cd139700 1 --2- 172.21.15.144:0/1051316486 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f31c80ff220 0x7f31c8195cc0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.679+0000 7f31cd139700 1 --2- 172.21.15.144:0/1051316486 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f31c8104870 0x7f31c8196200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.679+0000 7f31cd139700 1 -- 172.21.15.144:0/1051316486 >> 172.21.15.144:0/1051316486 conn(0x7f31c80747e0 msgr2=0x7f31c80fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:51.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.679+0000 7f31cd139700 1 -- 172.21.15.144:0/1051316486 shutdown_connections 2024-09-10T12:08:51.681 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.679+0000 7f31cd139700 1 -- 172.21.15.144:0/1051316486 wait complete. 2024-09-10T12:08:51.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.784+0000 7f3f4113d700 1 -- 172.21.15.144:0/1454263236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f3c0ff290 msgr2=0x7f3f3c0ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:51.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.784+0000 7f3f4113d700 1 --2- 172.21.15.144:0/1454263236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f3c0ff290 0x7f3f3c0ff660 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3f30009910 tx=0x7f3f30009c20 comp rx=0 tx=0).stop 2024-09-10T12:08:51.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.784+0000 7f3f4113d700 1 -- 172.21.15.144:0/1454263236 shutdown_connections 2024-09-10T12:08:51.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.784+0000 7f3f4113d700 1 --2- 172.21.15.144:0/1454263236 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f3c0ffc30 0x7f3f3c103c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.784+0000 7f3f4113d700 1 --2- 172.21.15.144:0/1454263236 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f3c0ff290 0x7f3f3c0ff660 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.784+0000 7f3f4113d700 1 -- 172.21.15.144:0/1454263236 >> 172.21.15.144:0/1454263236 conn(0x7f3f3c0fabb0 msgr2=0x7f3f3c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:51.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.784+0000 7f3f4113d700 1 -- 172.21.15.144:0/1454263236 shutdown_connections 2024-09-10T12:08:51.786 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.784+0000 7f3f4113d700 1 -- 172.21.15.144:0/1454263236 wait complete. 2024-09-10T12:08:51.787 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:51 smithi144 ceph-mon[87436]: pgmap v1108: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-09-10T12:08:51.787 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:51.787 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:51.787 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:08:51.787 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:08:51.787 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:51.787 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:08:51.787 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:51.787 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:51.787 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:51.787 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:51.787 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:51.788 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:51 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-09-10T12:08:51.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.785+0000 7f3f4113d700 1 Processor -- start 2024-09-10T12:08:51.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.785+0000 7f3f4113d700 1 -- start start 2024-09-10T12:08:51.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f4113d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f3c0ff290 0x7f3f3c19bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:51.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f4113d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f3c0ffc30 0x7f3f3c19c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:51.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f4113d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c19c690 con 0x7f3f3c0ffc30 2024-09-10T12:08:51.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f4113d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3c195b40 con 0x7f3f3c0ff290 2024-09-10T12:08:51.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f3a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f3c0ffc30 0x7f3f3c19c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:51.789 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f3a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f3c0ffc30 0x7f3f3c19c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57522/0 (socket says 172.21.15.144:57522) 2024-09-10T12:08:51.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f3a59c700 1 -- 172.21.15.144:0/3040186449 learned_addr learned my addr 172.21.15.144:0/3040186449 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:08:51.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f3ad9d700 1 --2- 172.21.15.144:0/3040186449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f3c0ff290 0x7f3f3c19bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:51.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f3a59c700 1 -- 172.21.15.144:0/3040186449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f3c0ff290 msgr2=0x7f3f3c19bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:51.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f3a59c700 1 --2- 172.21.15.144:0/3040186449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f3c0ff290 0x7f3f3c19bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:51.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f3a59c700 1 -- 172.21.15.144:0/3040186449 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f300095f0 con 0x7f3f3c0ffc30 2024-09-10T12:08:51.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f3a59c700 1 --2- 172.21.15.144:0/3040186449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f3c0ffc30 0x7f3f3c19c000 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00d900 tx=0x7f3f2c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:51.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f23fff700 1 -- 172.21.15.144:0/3040186449 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f2c009900 con 0x7f3f3c0ffc30 2024-09-10T12:08:51.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f23fff700 1 -- 172.21.15.144:0/3040186449 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f2c010460 con 0x7f3f3c0ffc30 2024-09-10T12:08:51.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f4113d700 1 -- 172.21.15.144:0/3040186449 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3c195e20 con 0x7f3f3c0ffc30 2024-09-10T12:08:51.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.786+0000 7f3f23fff700 1 -- 172.21.15.144:0/3040186449 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f2c00f5f0 con 0x7f3f3c0ffc30 2024-09-10T12:08:51.790 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.787+0000 7f3f4113d700 1 -- 172.21.15.144:0/3040186449 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f3c196370 con 0x7f3f3c0ffc30 2024-09-10T12:08:51.791 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.788+0000 7f3f4113d700 1 -- 172.21.15.144:0/3040186449 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f3c1085b0 con 0x7f3f3c0ffc30 2024-09-10T12:08:51.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.788+0000 7f3f23fff700 1 -- 172.21.15.144:0/3040186449 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3f2c0105d0 con 0x7f3f3c0ffc30 2024-09-10T12:08:51.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.789+0000 7f3f23fff700 1 --2- 172.21.15.144:0/3040186449 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f2406e200 0x7f3f240706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:51.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.789+0000 7f3f23fff700 1 -- 172.21.15.144:0/3040186449 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3f2c08dc60 con 0x7f3f3c0ffc30 2024-09-10T12:08:51.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.789+0000 7f3f3ad9d700 1 --2- 172.21.15.144:0/3040186449 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f2406e200 0x7f3f240706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:51.792 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.789+0000 7f3f3ad9d700 1 --2- 172.21.15.144:0/3040186449 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f2406e200 0x7f3f240706b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3f30006010 tx=0x7f3f3001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:51.794 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:51.793+0000 7f3f23fff700 1 -- 172.21.15.144:0/3040186449 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3f2c05ab70 con 0x7f3f3c0ffc30 2024-09-10T12:08:51.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:51 smithi178 ceph-mon[65940]: pgmap v1108: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.5 MiB/s rd, 4.3 KiB/s wr, 5 op/s 2024-09-10T12:08:51.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:51.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:51.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:08:51.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:08:51.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:51.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:08:51.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:51.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:51.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:51.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:51.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:51.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:51 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-09-10T12:08:52.006 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.005+0000 7f3f4113d700 1 -- 172.21.15.144:0/3040186449 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f3c1966c0 con 0x7f3f2406e200 2024-09-10T12:08:52.016 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.014+0000 7f3f23fff700 1 -- 172.21.15.144:0/3040186449 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f3f3c1966c0 con 0x7f3f2406e200 2024-09-10T12:08:52.016 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:08:52.016 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (37m) 8s ago 47m 23.5M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:08:52.016 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (22s) 8s ago 47m 8132k - 19.1.1-305-g1cf819b7 41d671128357 59d89239373f 2024-09-10T12:08:52.016 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (12s) 4s ago 46m 7415k - 19.1.1-305-g1cf819b7 41d671128357 79ad3d40fa4d 2024-09-10T12:08:52.016 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (34m) 8s ago 47m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:08:52.016 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (34m) 4s ago 46m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:08:52.016 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (36m) 8s ago 47m 79.3M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:08:52.016 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (31m) 8s ago 42m 62.2M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:08:52.016 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (30m) 8s ago 42m 65.2M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:08:52.017 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (15m) 4s ago 42m 200M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:08:52.017 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (30m) 4s ago 42m 204M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:08:52.017 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (38m) 8s ago 49m 511M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:08:52.017 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (37m) 4s ago 46m 449M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:08:52.017 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (35m) 8s ago 49m 88.6M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:08:52.017 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (35m) 4s ago 46m 59.8M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:08:52.017 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (37m) 8s ago 47m 16.8M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:08:52.017 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (37m) 4s ago 46m 15.6M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:08:52.017 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (34m) 8s ago 45m 104M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:08:52.017 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (33m) 8s ago 45m 105M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:08:52.017 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (33m) 8s ago 44m 95.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:08:52.017 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (32m) 4s ago 44m 119M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:08:52.017 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (32m) 4s ago 44m 101M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:08:52.018 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (31m) 4s ago 43m 98.4M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:08:52.018 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (37m) 8s ago 47m 101M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:08:52.018 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.017+0000 7f3f4113d700 1 -- 172.21.15.144:0/3040186449 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f2406e200 msgr2=0x7f3f240706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:52.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.017+0000 7f3f4113d700 1 --2- 172.21.15.144:0/3040186449 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f2406e200 0x7f3f240706b0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f3f30006010 tx=0x7f3f3001a040 comp rx=0 tx=0).stop 2024-09-10T12:08:52.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.017+0000 7f3f4113d700 1 -- 172.21.15.144:0/3040186449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f3c0ffc30 msgr2=0x7f3f3c19c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:52.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.017+0000 7f3f4113d700 1 --2- 172.21.15.144:0/3040186449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f3c0ffc30 0x7f3f3c19c000 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c00d900 tx=0x7f3f2c00dc10 comp rx=0 tx=0).stop 2024-09-10T12:08:52.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.017+0000 7f3f4113d700 1 -- 172.21.15.144:0/3040186449 shutdown_connections 2024-09-10T12:08:52.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.017+0000 7f3f4113d700 1 --2- 172.21.15.144:0/3040186449 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3f2406e200 0x7f3f240706b0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.017+0000 7f3f4113d700 1 --2- 172.21.15.144:0/3040186449 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3f3c0ff290 0x7f3f3c19bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.017+0000 7f3f4113d700 1 --2- 172.21.15.144:0/3040186449 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3f3c0ffc30 0x7f3f3c19c000 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.017+0000 7f3f4113d700 1 -- 172.21.15.144:0/3040186449 >> 172.21.15.144:0/3040186449 conn(0x7f3f3c0fabb0 msgr2=0x7f3f3c068f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:52.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.017+0000 7f3f4113d700 1 -- 172.21.15.144:0/3040186449 shutdown_connections 2024-09-10T12:08:52.019 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.017+0000 7f3f4113d700 1 -- 172.21.15.144:0/3040186449 wait complete. 2024-09-10T12:08:52.122 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.121+0000 7f1a1efdc700 1 -- 172.21.15.144:0/2519989048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a18100400 msgr2=0x7f1a181007d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:52.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.121+0000 7f1a1efdc700 1 --2- 172.21.15.144:0/2519989048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a18100400 0x7f1a181007d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f1a10009910 tx=0x7f1a10009c20 comp rx=0 tx=0).stop 2024-09-10T12:08:52.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.121+0000 7f1a1efdc700 1 -- 172.21.15.144:0/2519989048 shutdown_connections 2024-09-10T12:08:52.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.121+0000 7f1a1efdc700 1 --2- 172.21.15.144:0/2519989048 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a18100da0 0x7f1a18104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.121+0000 7f1a1efdc700 1 --2- 172.21.15.144:0/2519989048 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a18100400 0x7f1a181007d0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.123 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.121+0000 7f1a1efdc700 1 -- 172.21.15.144:0/2519989048 >> 172.21.15.144:0/2519989048 conn(0x7f1a180fbca0 msgr2=0x7f1a180fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:52.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.122+0000 7f1a1efdc700 1 -- 172.21.15.144:0/2519989048 shutdown_connections 2024-09-10T12:08:52.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.122+0000 7f1a1efdc700 1 -- 172.21.15.144:0/2519989048 wait complete. 2024-09-10T12:08:52.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.122+0000 7f1a1efdc700 1 Processor -- start 2024-09-10T12:08:52.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.122+0000 7f1a1efdc700 1 -- start start 2024-09-10T12:08:52.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.122+0000 7f1a1efdc700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a18100400 0x7f1a18195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:52.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.122+0000 7f1a1efdc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a18100da0 0x7f1a18196290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:52.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.122+0000 7f1a1efdc700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a18196970 con 0x7f1a18100da0 2024-09-10T12:08:52.125 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.122+0000 7f1a1efdc700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1819a700 con 0x7f1a18100400 2024-09-10T12:08:52.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.123+0000 7f1a1cd78700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a18100400 0x7f1a18195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:52.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.123+0000 7f1a17fff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a18100da0 0x7f1a18196290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:52.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.123+0000 7f1a1cd78700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a18100400 0x7f1a18195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.178:3300/0 says I am v2:172.21.15.144:55946/0 (socket says 172.21.15.144:55946) 2024-09-10T12:08:52.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.123+0000 7f1a1cd78700 1 -- 172.21.15.144:0/590942593 learned_addr learned my addr 172.21.15.144:0/590942593 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:08:52.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.123+0000 7f1a1cd78700 1 -- 172.21.15.144:0/590942593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a18100da0 msgr2=0x7f1a18196290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:52.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.123+0000 7f1a1cd78700 1 --2- 172.21.15.144:0/590942593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a18100da0 0x7f1a18196290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.123+0000 7f1a1cd78700 1 -- 172.21.15.144:0/590942593 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a100095f0 con 0x7f1a18100400 2024-09-10T12:08:52.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.123+0000 7f1a17fff700 1 --2- 172.21.15.144:0/590942593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a18100da0 0x7f1a18196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T12:08:52.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.123+0000 7f1a1cd78700 1 --2- 172.21.15.144:0/590942593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a18100400 0x7f1a18195d50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1a10005f50 tx=0x7f1a1000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:52.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.123+0000 7f1a15ffb700 1 -- 172.21.15.144:0/590942593 <== mon.1 v2:172.21.15.178:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a1001d070 con 0x7f1a18100400 2024-09-10T12:08:52.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.123+0000 7f1a15ffb700 1 -- 172.21.15.144:0/590942593 <== mon.1 v2:172.21.15.178:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a1000b610 con 0x7f1a18100400 2024-09-10T12:08:52.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.123+0000 7f1a1efdc700 1 -- 172.21.15.144:0/590942593 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1819a980 con 0x7f1a18100400 2024-09-10T12:08:52.126 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.123+0000 7f1a15ffb700 1 -- 172.21.15.144:0/590942593 <== mon.1 v2:172.21.15.178:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a10022470 con 0x7f1a18100400 2024-09-10T12:08:52.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.124+0000 7f1a1efdc700 1 -- 172.21.15.144:0/590942593 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1819ae70 con 0x7f1a18100400 2024-09-10T12:08:52.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.124+0000 7f1a1efdc700 1 -- 172.21.15.144:0/590942593 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a18108710 con 0x7f1a18100400 2024-09-10T12:08:52.127 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.125+0000 7f1a15ffb700 1 -- 172.21.15.144:0/590942593 <== mon.1 v2:172.21.15.178:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1a10003780 con 0x7f1a18100400 2024-09-10T12:08:52.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.125+0000 7f1a15ffb700 1 --2- 172.21.15.144:0/590942593 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1a0406e200 0x7f1a040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:52.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.126+0000 7f1a15ffb700 1 -- 172.21.15.144:0/590942593 <== mon.1 v2:172.21.15.178:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1a1008f500 con 0x7f1a18100400 2024-09-10T12:08:52.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.126+0000 7f1a17fff700 1 --2- 172.21.15.144:0/590942593 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1a0406e200 0x7f1a040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:52.128 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.126+0000 7f1a17fff700 1 --2- 172.21.15.144:0/590942593 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1a0406e200 0x7f1a040706b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1a18197370 tx=0x7f1a0800b040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:52.132 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.130+0000 7f1a15ffb700 1 -- 172.21.15.144:0/590942593 <== mon.1 v2:172.21.15.178:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1a10058840 con 0x7f1a18100400 2024-09-10T12:08:52.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.423+0000 7f1a1efdc700 1 -- 172.21.15.144:0/590942593 --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1a1802ccf0 con 0x7f1a18100400 2024-09-10T12:08:52.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.425+0000 7f1a15ffb700 1 -- 172.21.15.144:0/590942593 <== mon.1 v2:172.21.15.178:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f1a1005bf00 con 0x7f1a18100400 2024-09-10T12:08:52.426 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:08:52.426 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:08:52.426 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:08:52.426 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:08:52.426 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:08:52.427 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:08:52.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:08:52.427 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:08:52.427 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:08:52.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:08:52.427 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:08:52.427 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:08:52.427 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:08:52.427 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:08:52.427 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:08:52.427 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:08:52.427 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:08:52.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.427+0000 7f1a1efdc700 1 -- 172.21.15.144:0/590942593 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1a0406e200 msgr2=0x7f1a040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:52.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.427+0000 7f1a1efdc700 1 --2- 172.21.15.144:0/590942593 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1a0406e200 0x7f1a040706b0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f1a18197370 tx=0x7f1a0800b040 comp rx=0 tx=0).stop 2024-09-10T12:08:52.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.427+0000 7f1a1efdc700 1 -- 172.21.15.144:0/590942593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a18100400 msgr2=0x7f1a18195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:52.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.427+0000 7f1a1efdc700 1 --2- 172.21.15.144:0/590942593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a18100400 0x7f1a18195d50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f1a10005f50 tx=0x7f1a1000f690 comp rx=0 tx=0).stop 2024-09-10T12:08:52.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.427+0000 7f1a1efdc700 1 -- 172.21.15.144:0/590942593 shutdown_connections 2024-09-10T12:08:52.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.427+0000 7f1a1efdc700 1 --2- 172.21.15.144:0/590942593 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1a0406e200 0x7f1a040706b0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.427+0000 7f1a1efdc700 1 --2- 172.21.15.144:0/590942593 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1a18100400 0x7f1a18195d50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.427+0000 7f1a1efdc700 1 --2- 172.21.15.144:0/590942593 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1a18100da0 0x7f1a18196290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.427+0000 7f1a1efdc700 1 -- 172.21.15.144:0/590942593 >> 172.21.15.144:0/590942593 conn(0x7f1a180fbca0 msgr2=0x7f1a180fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:52.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.428+0000 7f1a1efdc700 1 -- 172.21.15.144:0/590942593 shutdown_connections 2024-09-10T12:08:52.430 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.428+0000 7f1a1efdc700 1 -- 172.21.15.144:0/590942593 wait complete. 2024-09-10T12:08:52.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.539+0000 7fd521046700 1 -- 172.21.15.144:0/2140339206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51c0ffd30 msgr2=0x7fd51c103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:52.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.539+0000 7fd521046700 1 --2- 172.21.15.144:0/2140339206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51c0ffd30 0x7fd51c103ce0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fd510009910 tx=0x7fd510009c20 comp rx=0 tx=0).stop 2024-09-10T12:08:52.540 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.539+0000 7fd521046700 1 -- 172.21.15.144:0/2140339206 shutdown_connections 2024-09-10T12:08:52.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.539+0000 7fd521046700 1 --2- 172.21.15.144:0/2140339206 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51c0ffd30 0x7fd51c103ce0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.539+0000 7fd521046700 1 --2- 172.21.15.144:0/2140339206 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd51c0ff390 0x7fd51c0ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.539+0000 7fd521046700 1 -- 172.21.15.144:0/2140339206 >> 172.21.15.144:0/2140339206 conn(0x7fd51c074f50 msgr2=0x7fd51c075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:52.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.539+0000 7fd521046700 1 -- 172.21.15.144:0/2140339206 shutdown_connections 2024-09-10T12:08:52.541 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.539+0000 7fd521046700 1 -- 172.21.15.144:0/2140339206 wait complete. 2024-09-10T12:08:52.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.540+0000 7fd521046700 1 Processor -- start 2024-09-10T12:08:52.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.540+0000 7fd521046700 1 -- start start 2024-09-10T12:08:52.542 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.540+0000 7fd521046700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd51c0ff390 0x7fd51c100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.540+0000 7fd521046700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51c0ffd30 0x7fd51c1012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.540+0000 7fd521046700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c101800 con 0x7fd51c0ffd30 2024-09-10T12:08:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.540+0000 7fd521046700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd51c101940 con 0x7fd51c0ff390 2024-09-10T12:08:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.540+0000 7fd51a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51c0ffd30 0x7fd51c1012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.540+0000 7fd51a59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51c0ffd30 0x7fd51c1012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41860/0 (socket says 172.21.15.144:41860) 2024-09-10T12:08:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.540+0000 7fd51ad9d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd51c0ff390 0x7fd51c100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.540+0000 7fd51a59c700 1 -- 172.21.15.144:0/2291481758 learned_addr learned my addr 172.21.15.144:0/2291481758 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:08:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.540+0000 7fd51a59c700 1 -- 172.21.15.144:0/2291481758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd51c0ff390 msgr2=0x7fd51c100d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.540+0000 7fd51a59c700 1 --2- 172.21.15.144:0/2291481758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd51c0ff390 0x7fd51c100d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.541+0000 7fd51a59c700 1 -- 172.21.15.144:0/2291481758 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5100095f0 con 0x7fd51c0ffd30 2024-09-10T12:08:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.541+0000 7fd51a59c700 1 --2- 172.21.15.144:0/2291481758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51c0ffd30 0x7fd51c1012c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd5100092e0 tx=0x7fd51000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.541+0000 7fd503fff700 1 -- 172.21.15.144:0/2291481758 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd51001d070 con 0x7fd51c0ffd30 2024-09-10T12:08:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.541+0000 7fd503fff700 1 -- 172.21.15.144:0/2291481758 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd510005dc0 con 0x7fd51c0ffd30 2024-09-10T12:08:52.543 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.541+0000 7fd521046700 1 -- 172.21.15.144:0/2291481758 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd51c101bc0 con 0x7fd51c0ffd30 2024-09-10T12:08:52.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.541+0000 7fd503fff700 1 -- 172.21.15.144:0/2291481758 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd510017400 con 0x7fd51c0ffd30 2024-09-10T12:08:52.544 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.541+0000 7fd521046700 1 -- 172.21.15.144:0/2291481758 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd51c1a2290 con 0x7fd51c0ffd30 2024-09-10T12:08:52.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.542+0000 7fd521046700 1 -- 172.21.15.144:0/2291481758 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd51c04ea50 con 0x7fd51c0ffd30 2024-09-10T12:08:52.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.543+0000 7fd503fff700 1 -- 172.21.15.144:0/2291481758 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd51000f6f0 con 0x7fd51c0ffd30 2024-09-10T12:08:52.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.543+0000 7fd503fff700 1 --2- 172.21.15.144:0/2291481758 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd50406e200 0x7fd5040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:52.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.543+0000 7fd503fff700 1 -- 172.21.15.144:0/2291481758 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd51008f450 con 0x7fd51c0ffd30 2024-09-10T12:08:52.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.543+0000 7fd51ad9d700 1 --2- 172.21.15.144:0/2291481758 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd50406e200 0x7fd5040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:52.545 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.544+0000 7fd51ad9d700 1 --2- 172.21.15.144:0/2291481758 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd50406e200 0x7fd5040706b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd50c005950 tx=0x7fd50c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:52.549 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.547+0000 7fd503fff700 1 -- 172.21.15.144:0/2291481758 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd510058830 con 0x7fd51c0ffd30 2024-09-10T12:08:52.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='client.36000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:52.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: pgmap v1109: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.8 KiB/s wr, 6 op/s 2024-09-10T12:08:52.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: Upgrade: Setting filesystem cephfs Joinable 2024-09-10T12:08:52.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='client.36004 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:52.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='client.36008 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:52.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-09-10T12:08:52.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-10T12:08:52.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi144.jaoyvs=up:active} 2 up:standby-replay 2024-09-10T12:08:52.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:52.685 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi144"}]: dispatch 2024-09-10T12:08:52.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi144"}]': finished 2024-09-10T12:08:52.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi178"}]: dispatch 2024-09-10T12:08:52.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi178"}]': finished 2024-09-10T12:08:52.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:52.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:52.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:52.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.686 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-10T12:08:52.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-10T12:08:52.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-10T12:08:52.687 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-10T12:08:52.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-09-10T12:08:52.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-10T12:08:52.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-09-10T12:08:52.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-10T12:08:52.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-09-10T12:08:52.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-10T12:08:52.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-10T12:08:52.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-10T12:08:52.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-10T12:08:52.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.688 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-10T12:08:52.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-10T12:08:52.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-10T12:08:52.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/590942593' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-10T12:08:52.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-10T12:08:52.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-10T12:08:52.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-10T12:08:52.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-10T12:08:52.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.689 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.690 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-10T12:08:52.690 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-10T12:08:52.690 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:08:52.690 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:08:52.690 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:08:52.690 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:52.690 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:08:52.690 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:08:52.690 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:08:52.690 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:52.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.803+0000 7fd521046700 1 -- 172.21.15.144:0/2291481758 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd51c1a2680 con 0x7fd51c0ffd30 2024-09-10T12:08:52.805 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.804+0000 7fd503fff700 1 -- 172.21.15.144:0/2291481758 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 42 v42) v1 ==== 76+0+2010 (secure 0 0 0) 0x7fd51005bef0 con 0x7fd51c0ffd30 2024-09-10T12:08:52.805 INFO:teuthology.orchestra.run.smithi144.stdout:e42 2024-09-10T12:08:52.805 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T12:08:52:341881+0000 2024-09-10T12:08:52.805 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:08:52.806 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:08:52.806 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:08:52.806 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:08:52.806 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:08:52.806 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:08:52.806 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 42 2024-09-10T12:08:52.806 INFO:teuthology.orchestra.run.smithi144.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T12:08:52.806 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:08:52.806 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T12:08:51.469973+0000 2024-09-10T12:08:52.806 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:08:52.806 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:08:52.806 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:08:52.806 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:08:52.806 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:08:52.807 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:08:52.807 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:08:52.807 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:08:52.807 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:08:52.807 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T12:08:52.807 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:08:52.807 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:08:52.807 INFO:teuthology.orchestra.run.smithi144.stdout:up {0=34390,1=34422} 2024-09-10T12:08:52.807 INFO:teuthology.orchestra.run.smithi144.stdout:failed 2024-09-10T12:08:52.807 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:08:52.807 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:08:52.807 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:08:52.807 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:08:52.808 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:08:52.808 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:08:52.808 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:08:52.808 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:08:52.808 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 34390 members: 34422,34390 2024-09-10T12:08:52.808 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{0:34390} state up:active seq 466 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:08:52.808 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{0:44355} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:08:52.808 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{1:34422} state up:active seq 462 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:08:52.808 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{1:44887} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:08:52.808 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:08:52.808 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:08:52.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.806+0000 7fd521046700 1 -- 172.21.15.144:0/2291481758 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd50406e200 msgr2=0x7fd5040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:52.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.806+0000 7fd521046700 1 --2- 172.21.15.144:0/2291481758 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd50406e200 0x7fd5040706b0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fd50c005950 tx=0x7fd50c00b410 comp rx=0 tx=0).stop 2024-09-10T12:08:52.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.806+0000 7fd521046700 1 -- 172.21.15.144:0/2291481758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51c0ffd30 msgr2=0x7fd51c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:52.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.806+0000 7fd521046700 1 --2- 172.21.15.144:0/2291481758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51c0ffd30 0x7fd51c1012c0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fd5100092e0 tx=0x7fd51000bd70 comp rx=0 tx=0).stop 2024-09-10T12:08:52.809 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.806+0000 7fd521046700 1 -- 172.21.15.144:0/2291481758 shutdown_connections 2024-09-10T12:08:52.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.806+0000 7fd521046700 1 --2- 172.21.15.144:0/2291481758 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd50406e200 0x7fd5040706b0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.806+0000 7fd521046700 1 --2- 172.21.15.144:0/2291481758 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd51c0ff390 0x7fd51c100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.806+0000 7fd521046700 1 --2- 172.21.15.144:0/2291481758 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd51c0ffd30 0x7fd51c1012c0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.806+0000 7fd521046700 1 -- 172.21.15.144:0/2291481758 >> 172.21.15.144:0/2291481758 conn(0x7fd51c074f50 msgr2=0x7fd51c103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:52.810 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.806+0000 7fd521046700 1 -- 172.21.15.144:0/2291481758 shutdown_connections 2024-09-10T12:08:52.811 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.806+0000 7fd521046700 1 -- 172.21.15.144:0/2291481758 wait complete. 2024-09-10T12:08:52.811 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 42 2024-09-10T12:08:52.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='client.36000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:52.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: pgmap v1109: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 4.8 KiB/s wr, 6 op/s 2024-09-10T12:08:52.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: Upgrade: Setting filesystem cephfs Joinable 2024-09-10T12:08:52.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='client.36004 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:52.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='client.36008 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:52.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: Health check cleared: CEPHADM_REFRESH_FAILED (was: failed to probe daemons or devices) 2024-09-10T12:08:52.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-09-10T12:08:52.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: fsmap cephfs:2 {0=cephfs.smithi144.bfyvwv=up:active,1=cephfs.smithi144.jaoyvs=up:active} 2 up:standby-replay 2024-09-10T12:08:52.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:52.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi144"}]: dispatch 2024-09-10T12:08:52.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi144"}]': finished 2024-09-10T12:08:52.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi178"}]: dispatch 2024-09-10T12:08:52.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi178"}]': finished 2024-09-10T12:08:52.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:52.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:52.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.869 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:52.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-09-10T12:08:52.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-09-10T12:08:52.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-09-10T12:08:52.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-09-10T12:08:52.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-09-10T12:08:52.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-09-10T12:08:52.870 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-09-10T12:08:52.871 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-09-10T12:08:52.871 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-09-10T12:08:52.871 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-09-10T12:08:52.871 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-09-10T12:08:52.871 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-09-10T12:08:52.871 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-09-10T12:08:52.871 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.871 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-09-10T12:08:52.871 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-09-10T12:08:52.872 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-09-10T12:08:52.872 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/590942593' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:08:52.872 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-09-10T12:08:52.872 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-09-10T12:08:52.872 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-09-10T12:08:52.872 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-09-10T12:08:52.872 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-09-10T12:08:52.872 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.873 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.873 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.873 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.873 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.873 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-09-10T12:08:52.873 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-09-10T12:08:52.873 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-09-10T12:08:52.873 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:08:52.873 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:08:52.873 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:08:52.873 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:52.873 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:08:52.873 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:08:52.873 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:08:52.874 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:52.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.912+0000 7f3d26184700 1 -- 172.21.15.144:0/1992651033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d20100d70 msgr2=0x7f3d20104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:52.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.912+0000 7f3d26184700 1 --2- 172.21.15.144:0/1992651033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d20100d70 0x7f3d20104d40 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f3d14009910 tx=0x7f3d14009c20 comp rx=0 tx=0).stop 2024-09-10T12:08:52.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.912+0000 7f3d26184700 1 -- 172.21.15.144:0/1992651033 shutdown_connections 2024-09-10T12:08:52.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.912+0000 7f3d26184700 1 --2- 172.21.15.144:0/1992651033 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d20100d70 0x7f3d20104d40 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.912+0000 7f3d26184700 1 --2- 172.21.15.144:0/1992651033 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d201003d0 0x7f3d201007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.912+0000 7f3d26184700 1 -- 172.21.15.144:0/1992651033 >> 172.21.15.144:0/1992651033 conn(0x7f3d200fbc90 msgr2=0x7f3d200fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:52.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.912+0000 7f3d26184700 1 -- 172.21.15.144:0/1992651033 shutdown_connections 2024-09-10T12:08:52.914 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.912+0000 7f3d26184700 1 -- 172.21.15.144:0/1992651033 wait complete. 2024-09-10T12:08:52.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.913+0000 7f3d26184700 1 Processor -- start 2024-09-10T12:08:52.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.913+0000 7f3d26184700 1 -- start start 2024-09-10T12:08:52.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.913+0000 7f3d26184700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d201003d0 0x7f3d20195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:52.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.913+0000 7f3d26184700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d20100d70 0x7f3d201963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:52.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.913+0000 7f3d26184700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d20196ac0 con 0x7f3d201003d0 2024-09-10T12:08:52.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.913+0000 7f3d26184700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2019a850 con 0x7f3d20100d70 2024-09-10T12:08:52.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.913+0000 7f3d1f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d201003d0 0x7f3d20195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:52.916 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.913+0000 7f3d1f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d201003d0 0x7f3d20195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41882/0 (socket says 172.21.15.144:41882) 2024-09-10T12:08:52.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.913+0000 7f3d1f7fe700 1 -- 172.21.15.144:0/3270603709 learned_addr learned my addr 172.21.15.144:0/3270603709 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:08:52.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.914+0000 7f3d1effd700 1 --2- 172.21.15.144:0/3270603709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d20100d70 0x7f3d201963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:52.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.914+0000 7f3d1f7fe700 1 -- 172.21.15.144:0/3270603709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d20100d70 msgr2=0x7f3d201963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:52.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.914+0000 7f3d1f7fe700 1 --2- 172.21.15.144:0/3270603709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d20100d70 0x7f3d201963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:52.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.914+0000 7f3d1f7fe700 1 -- 172.21.15.144:0/3270603709 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d140095f0 con 0x7f3d201003d0 2024-09-10T12:08:52.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.914+0000 7f3d1f7fe700 1 --2- 172.21.15.144:0/3270603709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d201003d0 0x7f3d20195ea0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000ba50 tx=0x7f3d1000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:52.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.914+0000 7f3d1cff9700 1 -- 172.21.15.144:0/3270603709 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d1000ca00 con 0x7f3d201003d0 2024-09-10T12:08:52.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.914+0000 7f3d26184700 1 -- 172.21.15.144:0/3270603709 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d2019ab30 con 0x7f3d201003d0 2024-09-10T12:08:52.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.914+0000 7f3d1cff9700 1 -- 172.21.15.144:0/3270603709 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3d1000cb60 con 0x7f3d201003d0 2024-09-10T12:08:52.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.914+0000 7f3d1cff9700 1 -- 172.21.15.144:0/3270603709 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d100075a0 con 0x7f3d201003d0 2024-09-10T12:08:52.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.914+0000 7f3d26184700 1 -- 172.21.15.144:0/3270603709 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d2019b080 con 0x7f3d201003d0 2024-09-10T12:08:52.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.915+0000 7f3d26184700 1 -- 172.21.15.144:0/3270603709 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d20108690 con 0x7f3d201003d0 2024-09-10T12:08:52.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.916+0000 7f3d1cff9700 1 -- 172.21.15.144:0/3270603709 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f3d1000ccd0 con 0x7f3d201003d0 2024-09-10T12:08:52.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.916+0000 7f3d1cff9700 1 --2- 172.21.15.144:0/3270603709 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d0806e200 0x7f3d080706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:52.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.916+0000 7f3d1cff9700 1 -- 172.21.15.144:0/3270603709 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f3d10006e40 con 0x7f3d201003d0 2024-09-10T12:08:52.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.916+0000 7f3d1effd700 1 --2- 172.21.15.144:0/3270603709 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d0806e200 0x7f3d080706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:52.918 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.917+0000 7f3d1effd700 1 --2- 172.21.15.144:0/3270603709 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d0806e200 0x7f3d080706b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3d201974c0 tx=0x7f3d1401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:52.922 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:52.921+0000 7f3d1cff9700 1 -- 172.21.15.144:0/3270603709 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f3d10056cc0 con 0x7f3d201003d0 2024-09-10T12:08:53.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.143+0000 7f3d26184700 1 -- 172.21.15.144:0/3270603709 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d2019b3c0 con 0x7f3d0806e200 2024-09-10T12:08:53.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.144+0000 7f3d1cff9700 1 -- 172.21.15.144:0/3270603709 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f3d2019b3c0 con 0x7f3d0806e200 2024-09-10T12:08:53.146 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:08:53.146 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": null, 2024-09-10T12:08:53.146 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": false, 2024-09-10T12:08:53.146 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "", 2024-09-10T12:08:53.146 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [], 2024-09-10T12:08:53.146 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": null, 2024-09-10T12:08:53.146 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "", 2024-09-10T12:08:53.146 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:08:53.146 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:08:53.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.146+0000 7f3d26184700 1 -- 172.21.15.144:0/3270603709 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d0806e200 msgr2=0x7f3d080706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:53.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.147+0000 7f3d26184700 1 --2- 172.21.15.144:0/3270603709 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d0806e200 0x7f3d080706b0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f3d201974c0 tx=0x7f3d1401a040 comp rx=0 tx=0).stop 2024-09-10T12:08:53.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.147+0000 7f3d26184700 1 -- 172.21.15.144:0/3270603709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d201003d0 msgr2=0x7f3d20195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:53.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.147+0000 7f3d26184700 1 --2- 172.21.15.144:0/3270603709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d201003d0 0x7f3d20195ea0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f3d1000ba50 tx=0x7f3d1000be10 comp rx=0 tx=0).stop 2024-09-10T12:08:53.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.147+0000 7f3d26184700 1 -- 172.21.15.144:0/3270603709 shutdown_connections 2024-09-10T12:08:53.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.147+0000 7f3d26184700 1 --2- 172.21.15.144:0/3270603709 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f3d0806e200 0x7f3d080706b0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:53.148 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.147+0000 7f3d26184700 1 --2- 172.21.15.144:0/3270603709 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f3d201003d0 0x7f3d20195ea0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:53.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.147+0000 7f3d26184700 1 --2- 172.21.15.144:0/3270603709 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3d20100d70 0x7f3d201963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:53.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.147+0000 7f3d26184700 1 -- 172.21.15.144:0/3270603709 >> 172.21.15.144:0/3270603709 conn(0x7f3d200fbc90 msgr2=0x7f3d200fd470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:53.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.147+0000 7f3d26184700 1 -- 172.21.15.144:0/3270603709 shutdown_connections 2024-09-10T12:08:53.149 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.147+0000 7f3d26184700 1 -- 172.21.15.144:0/3270603709 wait complete. 2024-09-10T12:08:53.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.252+0000 7f8a4d5d6700 1 -- 172.21.15.144:0/528424895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a480ff290 msgr2=0x7f8a480ff660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:53.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.252+0000 7f8a4d5d6700 1 --2- 172.21.15.144:0/528424895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a480ff290 0x7f8a480ff660 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f8a38009910 tx=0x7f8a38009c20 comp rx=0 tx=0).stop 2024-09-10T12:08:53.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.252+0000 7f8a4d5d6700 1 -- 172.21.15.144:0/528424895 shutdown_connections 2024-09-10T12:08:53.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.252+0000 7f8a4d5d6700 1 --2- 172.21.15.144:0/528424895 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a480ffba0 0x7f8a4810a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:53.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.252+0000 7f8a4d5d6700 1 --2- 172.21.15.144:0/528424895 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a480ff290 0x7f8a480ff660 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:53.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.252+0000 7f8a4d5d6700 1 -- 172.21.15.144:0/528424895 >> 172.21.15.144:0/528424895 conn(0x7f8a48074850 msgr2=0x7f8a48074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:53.254 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.252+0000 7f8a4d5d6700 1 -- 172.21.15.144:0/528424895 shutdown_connections 2024-09-10T12:08:53.255 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.252+0000 7f8a4d5d6700 1 -- 172.21.15.144:0/528424895 wait complete. 2024-09-10T12:08:53.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.253+0000 7f8a4d5d6700 1 Processor -- start 2024-09-10T12:08:53.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.253+0000 7f8a4d5d6700 1 -- start start 2024-09-10T12:08:53.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a4d5d6700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a480ff290 0x7f8a48195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:53.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a4d5d6700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a480ffba0 0x7f8a481962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:53.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a4d5d6700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a48196990 con 0x7f8a480ffba0 2024-09-10T12:08:53.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a4d5d6700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4819a720 con 0x7f8a480ff290 2024-09-10T12:08:53.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a467fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a480ffba0 0x7f8a481962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:53.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a467fc700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a480ffba0 0x7f8a481962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41886/0 (socket says 172.21.15.144:41886) 2024-09-10T12:08:53.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a46ffd700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a480ff290 0x7f8a48195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:53.256 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a467fc700 1 -- 172.21.15.144:0/567633611 learned_addr learned my addr 172.21.15.144:0/567633611 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:08:53.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a467fc700 1 -- 172.21.15.144:0/567633611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a480ff290 msgr2=0x7f8a48195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:53.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a467fc700 1 --2- 172.21.15.144:0/567633611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a480ff290 0x7f8a48195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:53.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a467fc700 1 -- 172.21.15.144:0/567633611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a380095f0 con 0x7f8a480ffba0 2024-09-10T12:08:53.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a467fc700 1 --2- 172.21.15.144:0/567633611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a480ffba0 0x7f8a481962b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00d8d0 tx=0x7f8a3c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:53.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a2ffff700 1 -- 172.21.15.144:0/567633611 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3c0098a0 con 0x7f8a480ffba0 2024-09-10T12:08:53.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a2ffff700 1 -- 172.21.15.144:0/567633611 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8a3c010460 con 0x7f8a480ffba0 2024-09-10T12:08:53.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a4d5d6700 1 -- 172.21.15.144:0/567633611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a4819aa00 con 0x7f8a480ffba0 2024-09-10T12:08:53.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.254+0000 7f8a2ffff700 1 -- 172.21.15.144:0/567633611 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a3c00f5f0 con 0x7f8a480ffba0 2024-09-10T12:08:53.257 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.255+0000 7f8a4d5d6700 1 -- 172.21.15.144:0/567633611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4819af50 con 0x7f8a480ffba0 2024-09-10T12:08:53.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.256+0000 7f8a4d5d6700 1 -- 172.21.15.144:0/567633611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a48107d10 con 0x7f8a480ffba0 2024-09-10T12:08:53.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.256+0000 7f8a2ffff700 1 -- 172.21.15.144:0/567633611 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8a3c009a00 con 0x7f8a480ffba0 2024-09-10T12:08:53.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.257+0000 7f8a2ffff700 1 --2- 172.21.15.144:0/567633611 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a3006e200 0x7f8a300706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:08:53.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.257+0000 7f8a2ffff700 1 -- 172.21.15.144:0/567633611 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8a3c08df70 con 0x7f8a480ffba0 2024-09-10T12:08:53.258 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.257+0000 7f8a46ffd700 1 --2- 172.21.15.144:0/567633611 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a3006e200 0x7f8a300706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:08:53.259 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.258+0000 7f8a46ffd700 1 --2- 172.21.15.144:0/567633611 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a3006e200 0x7f8a300706b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8a38006010 tx=0x7f8a3801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:08:53.262 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.261+0000 7f8a2ffff700 1 -- 172.21.15.144:0/567633611 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8a3c0c31b0 con 0x7f8a480ffba0 2024-09-10T12:08:53.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.583+0000 7f8a4d5d6700 1 -- 172.21.15.144:0/567633611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f8a4804ea50 con 0x7f8a480ffba0 2024-09-10T12:08:53.585 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.584+0000 7f8a2ffff700 1 -- 172.21.15.144:0/567633611 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7f8a3c057350 con 0x7f8a480ffba0 2024-09-10T12:08:53.585 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T12:08:53.586 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:08:53.586 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:08:53.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.586+0000 7f8a4d5d6700 1 -- 172.21.15.144:0/567633611 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a3006e200 msgr2=0x7f8a300706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:53.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.586+0000 7f8a4d5d6700 1 --2- 172.21.15.144:0/567633611 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a3006e200 0x7f8a300706b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f8a38006010 tx=0x7f8a3801a040 comp rx=0 tx=0).stop 2024-09-10T12:08:53.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.586+0000 7f8a4d5d6700 1 -- 172.21.15.144:0/567633611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a480ffba0 msgr2=0x7f8a481962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:08:53.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.586+0000 7f8a4d5d6700 1 --2- 172.21.15.144:0/567633611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a480ffba0 0x7f8a481962b0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c00d8d0 tx=0x7f8a3c00dbe0 comp rx=0 tx=0).stop 2024-09-10T12:08:53.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.586+0000 7f8a4d5d6700 1 -- 172.21.15.144:0/567633611 shutdown_connections 2024-09-10T12:08:53.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.586+0000 7f8a4d5d6700 1 --2- 172.21.15.144:0/567633611 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8a3006e200 0x7f8a300706b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:53.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.586+0000 7f8a4d5d6700 1 --2- 172.21.15.144:0/567633611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8a480ff290 0x7f8a48195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:53.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.586+0000 7f8a4d5d6700 1 --2- 172.21.15.144:0/567633611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8a480ffba0 0x7f8a481962b0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:08:53.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.586+0000 7f8a4d5d6700 1 -- 172.21.15.144:0/567633611 >> 172.21.15.144:0/567633611 conn(0x7f8a48074850 msgr2=0x7f8a48104e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:08:53.588 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.586+0000 7f8a4d5d6700 1 -- 172.21.15.144:0/567633611 shutdown_connections 2024-09-10T12:08:53.589 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:08:53.586+0000 7f8a4d5d6700 1 -- 172.21.15.144:0/567633611 wait complete. 2024-09-10T12:08:53.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:53 smithi144 ceph-mon[87436]: Upgrade: Setting container_image for all ceph-exporter 2024-09-10T12:08:53.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:53 smithi144 ceph-mon[87436]: Upgrade: Setting container_image for all iscsi 2024-09-10T12:08:53.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:53 smithi144 ceph-mon[87436]: Upgrade: Setting container_image for all nfs 2024-09-10T12:08:53.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:53 smithi144 ceph-mon[87436]: Upgrade: Setting container_image for all nvmeof 2024-09-10T12:08:53.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:53 smithi144 ceph-mon[87436]: Upgrade: Finalizing container_image settings 2024-09-10T12:08:53.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:53 smithi144 ceph-mon[87436]: Upgrade: Complete! 2024-09-10T12:08:53.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:53 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2291481758' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:08:53.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:53 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/567633611' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:08:53.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:53 smithi178 ceph-mon[65940]: Upgrade: Setting container_image for all ceph-exporter 2024-09-10T12:08:53.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:53 smithi178 ceph-mon[65940]: Upgrade: Setting container_image for all iscsi 2024-09-10T12:08:53.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:53 smithi178 ceph-mon[65940]: Upgrade: Setting container_image for all nfs 2024-09-10T12:08:53.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:53 smithi178 ceph-mon[65940]: Upgrade: Setting container_image for all nvmeof 2024-09-10T12:08:53.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:53 smithi178 ceph-mon[65940]: Upgrade: Finalizing container_image settings 2024-09-10T12:08:53.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:53 smithi178 ceph-mon[65940]: Upgrade: Complete! 2024-09-10T12:08:53.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:53 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2291481758' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:08:53.868 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:53 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/567633611' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:08:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:54 smithi178 ceph-mon[65940]: from='client.36020 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:54 smithi178 ceph-mon[65940]: pgmap v1110: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-10T12:08:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:54 smithi144 ceph-mon[87436]: from='client.36020 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:08:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:54 smithi144 ceph-mon[87436]: pgmap v1110: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.2 KiB/s rd, 4 op/s 2024-09-10T12:08:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:56 smithi178 ceph-mon[65940]: pgmap v1111: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T12:08:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:56 smithi144 ceph-mon[87436]: pgmap v1111: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T12:08:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:58 smithi178 ceph-mon[65940]: pgmap v1112: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-10T12:08:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:08:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:08:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:58 smithi144 ceph-mon[87436]: pgmap v1112: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 4 op/s 2024-09-10T12:08:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:08:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:08:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:09:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:00 smithi178 ceph-mon[65940]: pgmap v1113: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T12:09:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:00 smithi144 ceph-mon[87436]: pgmap v1113: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 3 op/s 2024-09-10T12:09:03.113 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:02 smithi144 ceph-mon[87436]: pgmap v1114: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-09-10T12:09:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:02 smithi178 ceph-mon[65940]: pgmap v1114: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.9 KiB/s rd, 4 op/s 2024-09-10T12:09:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:04 smithi178 ceph-mon[65940]: pgmap v1115: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:09:05.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:04 smithi144 ceph-mon[87436]: pgmap v1115: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:09:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:06 smithi178 ceph-mon[65940]: pgmap v1116: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:06 smithi144 ceph-mon[87436]: pgmap v1116: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:08 smithi178 ceph-mon[65940]: pgmap v1117: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:09:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:08 smithi144 ceph-mon[87436]: pgmap v1117: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:09:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:10 smithi178 ceph-mon[65940]: pgmap v1118: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:09:11.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:10 smithi144 ceph-mon[87436]: pgmap v1118: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:09:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:12 smithi144 ceph-mon[87436]: pgmap v1119: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:12 smithi178 ceph-mon[65940]: pgmap v1119: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:09:14.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:09:14.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:14 smithi144 ceph-mon[87436]: pgmap v1120: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:09:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:14 smithi178 ceph-mon[65940]: pgmap v1120: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:09:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:16 smithi178 ceph-mon[65940]: pgmap v1121: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:17.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:16 smithi144 ceph-mon[87436]: pgmap v1121: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:18 smithi178 ceph-mon[65940]: pgmap v1122: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:09:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:18 smithi144 ceph-mon[87436]: pgmap v1122: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:09:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:20 smithi178 ceph-mon[65940]: pgmap v1123: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:09:21.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:20 smithi144 ceph-mon[87436]: pgmap v1123: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:09:23.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:22 smithi144 ceph-mon[87436]: pgmap v1124: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:23.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:22 smithi178 ceph-mon[65940]: pgmap v1124: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:23.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.706+0000 7f031362e700 1 -- 172.21.15.144:0/3502716016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c1003a0 msgr2=0x7f030c100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:23.709 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.706+0000 7f031362e700 1 --2- 172.21.15.144:0/3502716016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c1003a0 0x7f030c100770 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f030400b0d0 tx=0x7f030400b3e0 comp rx=0 tx=0).stop 2024-09-10T12:09:23.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.706+0000 7f031362e700 1 -- 172.21.15.144:0/3502716016 shutdown_connections 2024-09-10T12:09:23.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.706+0000 7f031362e700 1 --2- 172.21.15.144:0/3502716016 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f030c100cb0 0x7f030c109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:23.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.706+0000 7f031362e700 1 --2- 172.21.15.144:0/3502716016 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c1003a0 0x7f030c100770 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:23.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.706+0000 7f031362e700 1 -- 172.21.15.144:0/3502716016 >> 172.21.15.144:0/3502716016 conn(0x7f030c074850 msgr2=0x7f030c074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:09:23.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.706+0000 7f031362e700 1 -- 172.21.15.144:0/3502716016 shutdown_connections 2024-09-10T12:09:23.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.706+0000 7f031362e700 1 -- 172.21.15.144:0/3502716016 wait complete. 2024-09-10T12:09:23.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.707+0000 7f031362e700 1 Processor -- start 2024-09-10T12:09:23.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.707+0000 7f031362e700 1 -- start start 2024-09-10T12:09:23.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.707+0000 7f031362e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c1003a0 0x7f030c101480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:09:23.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.707+0000 7f031362e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f030c100cb0 0x7f030c1019c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:09:23.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.707+0000 7f031362e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c101f00 con 0x7f030c1003a0 2024-09-10T12:09:23.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.707+0000 7f031362e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f030c102040 con 0x7f030c100cb0 2024-09-10T12:09:23.710 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.707+0000 7f03113ca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c1003a0 0x7f030c101480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:09:23.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.707+0000 7f03113ca700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c1003a0 0x7f030c101480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41644/0 (socket says 172.21.15.144:41644) 2024-09-10T12:09:23.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.707+0000 7f03113ca700 1 -- 172.21.15.144:0/1203808888 learned_addr learned my addr 172.21.15.144:0/1203808888 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:09:23.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.707+0000 7f0310bc9700 1 --2- 172.21.15.144:0/1203808888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f030c100cb0 0x7f030c1019c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:09:23.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.707+0000 7f03113ca700 1 -- 172.21.15.144:0/1203808888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f030c100cb0 msgr2=0x7f030c1019c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:23.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.707+0000 7f03113ca700 1 --2- 172.21.15.144:0/1203808888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f030c100cb0 0x7f030c1019c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:23.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.707+0000 7f03113ca700 1 -- 172.21.15.144:0/1203808888 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0304009d00 con 0x7f030c1003a0 2024-09-10T12:09:23.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.708+0000 7f03113ca700 1 --2- 172.21.15.144:0/1203808888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c1003a0 0x7f030c101480 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f030400bd00 tx=0x7f030400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:09:23.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.708+0000 7f030a7fc700 1 -- 172.21.15.144:0/1203808888 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f030400e040 con 0x7f030c1003a0 2024-09-10T12:09:23.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.708+0000 7f030a7fc700 1 -- 172.21.15.144:0/1203808888 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0304003c10 con 0x7f030c1003a0 2024-09-10T12:09:23.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.708+0000 7f031362e700 1 -- 172.21.15.144:0/1203808888 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030c1022c0 con 0x7f030c1003a0 2024-09-10T12:09:23.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.708+0000 7f030a7fc700 1 -- 172.21.15.144:0/1203808888 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f030401b830 con 0x7f030c1003a0 2024-09-10T12:09:23.711 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.708+0000 7f031362e700 1 -- 172.21.15.144:0/1203808888 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f030c1a2290 con 0x7f030c1003a0 2024-09-10T12:09:23.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.709+0000 7f031362e700 1 -- 172.21.15.144:0/1203808888 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030c107d80 con 0x7f030c1003a0 2024-09-10T12:09:23.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.709+0000 7f030a7fc700 1 -- 172.21.15.144:0/1203808888 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f0304019070 con 0x7f030c1003a0 2024-09-10T12:09:23.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.710+0000 7f030a7fc700 1 --2- 172.21.15.144:0/1203808888 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f02f406e200 0x7f02f40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:09:23.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.710+0000 7f0310bc9700 1 --2- 172.21.15.144:0/1203808888 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f02f406e200 0x7f02f40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:09:23.712 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.710+0000 7f030a7fc700 1 -- 172.21.15.144:0/1203808888 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f030408ef20 con 0x7f030c1003a0 2024-09-10T12:09:23.713 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.711+0000 7f0310bc9700 1 --2- 172.21.15.144:0/1203808888 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f02f406e200 0x7f02f40706b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f030c102c70 tx=0x7f030000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:09:23.716 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.715+0000 7f030a7fc700 1 -- 172.21.15.144:0/1203808888 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f03040c41b0 con 0x7f030c1003a0 2024-09-10T12:09:23.946 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.945+0000 7f031362e700 1 -- 172.21.15.144:0/1203808888 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f030c02cd10 con 0x7f02f406e200 2024-09-10T12:09:23.947 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.946+0000 7f030a7fc700 1 -- 172.21.15.144:0/1203808888 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f030c02cd10 con 0x7f02f406e200 2024-09-10T12:09:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.948+0000 7f031362e700 1 -- 172.21.15.144:0/1203808888 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f02f406e200 msgr2=0x7f02f40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.948+0000 7f031362e700 1 --2- 172.21.15.144:0/1203808888 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f02f406e200 0x7f02f40706b0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f030c102c70 tx=0x7f030000b410 comp rx=0 tx=0).stop 2024-09-10T12:09:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.948+0000 7f031362e700 1 -- 172.21.15.144:0/1203808888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c1003a0 msgr2=0x7f030c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.948+0000 7f031362e700 1 --2- 172.21.15.144:0/1203808888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c1003a0 0x7f030c101480 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f030400bd00 tx=0x7f030400bd30 comp rx=0 tx=0).stop 2024-09-10T12:09:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.948+0000 7f031362e700 1 -- 172.21.15.144:0/1203808888 shutdown_connections 2024-09-10T12:09:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.948+0000 7f031362e700 1 --2- 172.21.15.144:0/1203808888 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f02f406e200 0x7f02f40706b0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:23.950 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.948+0000 7f031362e700 1 --2- 172.21.15.144:0/1203808888 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f030c1003a0 0x7f030c101480 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:23.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.948+0000 7f031362e700 1 --2- 172.21.15.144:0/1203808888 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f030c100cb0 0x7f030c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:23.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.948+0000 7f031362e700 1 -- 172.21.15.144:0/1203808888 >> 172.21.15.144:0/1203808888 conn(0x7f030c074850 msgr2=0x7f030c0fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:09:23.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.949+0000 7f031362e700 1 -- 172.21.15.144:0/1203808888 shutdown_connections 2024-09-10T12:09:23.951 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:23.949+0000 7f031362e700 1 -- 172.21.15.144:0/1203808888 wait complete. 2024-09-10T12:09:24.614 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph orch ps' 2024-09-10T12:09:24.820 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:09:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:24 smithi178 ceph-mon[65940]: pgmap v1125: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:09:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:24 smithi178 ceph-mon[65940]: from='client.36028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:09:25.154 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:24 smithi144 ceph-mon[87436]: pgmap v1125: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:09:25.154 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:24 smithi144 ceph-mon[87436]: from='client.36028 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:09:26.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.355+0000 7f264e46e700 1 -- 172.21.15.144:0/3290450651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26480ffc60 msgr2=0x7f2648103c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:26.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.355+0000 7f264e46e700 1 --2- 172.21.15.144:0/3290450651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26480ffc60 0x7f2648103c30 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f2630009910 tx=0x7f2630009c20 comp rx=0 tx=0).stop 2024-09-10T12:09:26.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.355+0000 7f264e46e700 1 -- 172.21.15.144:0/3290450651 shutdown_connections 2024-09-10T12:09:26.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.355+0000 7f264e46e700 1 --2- 172.21.15.144:0/3290450651 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26480ffc60 0x7f2648103c30 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:26.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.355+0000 7f264e46e700 1 --2- 172.21.15.144:0/3290450651 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26480ff2c0 0x7f26480ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:26.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.355+0000 7f264e46e700 1 -- 172.21.15.144:0/3290450651 >> 172.21.15.144:0/3290450651 conn(0x7f26480fab80 msgr2=0x7f26480fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:09:26.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.355+0000 7f264e46e700 1 -- 172.21.15.144:0/3290450651 shutdown_connections 2024-09-10T12:09:26.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.355+0000 7f264e46e700 1 -- 172.21.15.144:0/3290450651 wait complete. 2024-09-10T12:09:26.358 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.356+0000 7f264e46e700 1 Processor -- start 2024-09-10T12:09:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.356+0000 7f264e46e700 1 -- start start 2024-09-10T12:09:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.356+0000 7f264e46e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26480ff2c0 0x7f2648100cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:09:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.356+0000 7f264e46e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26480ffc60 0x7f2648101210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:09:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.356+0000 7f264e46e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2648101750 con 0x7f26480ffc60 2024-09-10T12:09:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.356+0000 7f264e46e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2648101890 con 0x7f26480ff2c0 2024-09-10T12:09:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.356+0000 7f263ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26480ffc60 0x7f2648101210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:09:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.356+0000 7f263ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26480ffc60 0x7f2648101210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41662/0 (socket says 172.21.15.144:41662) 2024-09-10T12:09:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.356+0000 7f263ffff700 1 -- 172.21.15.144:0/629836658 learned_addr learned my addr 172.21.15.144:0/629836658 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:09:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.357+0000 7f2647fff700 1 --2- 172.21.15.144:0/629836658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26480ff2c0 0x7f2648100cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:09:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.357+0000 7f263ffff700 1 -- 172.21.15.144:0/629836658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26480ff2c0 msgr2=0x7f2648100cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.357+0000 7f263ffff700 1 --2- 172.21.15.144:0/629836658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26480ff2c0 0x7f2648100cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:26.359 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.357+0000 7f263ffff700 1 -- 172.21.15.144:0/629836658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26300095f0 con 0x7f26480ffc60 2024-09-10T12:09:26.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.357+0000 7f263ffff700 1 --2- 172.21.15.144:0/629836658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26480ffc60 0x7f2648101210 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f26300059d0 tx=0x7f263000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:09:26.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.357+0000 7f2645ffb700 1 -- 172.21.15.144:0/629836658 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f263001d070 con 0x7f26480ffc60 2024-09-10T12:09:26.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.357+0000 7f2645ffb700 1 -- 172.21.15.144:0/629836658 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f263000b610 con 0x7f26480ffc60 2024-09-10T12:09:26.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.357+0000 7f264e46e700 1 -- 172.21.15.144:0/629836658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2648101b10 con 0x7f26480ffc60 2024-09-10T12:09:26.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.357+0000 7f2645ffb700 1 -- 172.21.15.144:0/629836658 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2630022470 con 0x7f26480ffc60 2024-09-10T12:09:26.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.357+0000 7f264e46e700 1 -- 172.21.15.144:0/629836658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26481a2280 con 0x7f26480ffc60 2024-09-10T12:09:26.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.358+0000 7f264e46e700 1 -- 172.21.15.144:0/629836658 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2648108640 con 0x7f26480ffc60 2024-09-10T12:09:26.361 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.359+0000 7f2645ffb700 1 -- 172.21.15.144:0/629836658 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2630003710 con 0x7f26480ffc60 2024-09-10T12:09:26.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.359+0000 7f2645ffb700 1 --2- 172.21.15.144:0/629836658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f263406e200 0x7f26340706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:09:26.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.359+0000 7f2647fff700 1 --2- 172.21.15.144:0/629836658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f263406e200 0x7f26340706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:09:26.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.359+0000 7f2645ffb700 1 -- 172.21.15.144:0/629836658 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f263008f4f0 con 0x7f26480ffc60 2024-09-10T12:09:26.362 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.360+0000 7f2647fff700 1 --2- 172.21.15.144:0/629836658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f263406e200 0x7f26340706b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2638009c60 tx=0x7f2638009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:09:26.365 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.364+0000 7f2645ffb700 1 -- 172.21.15.144:0/629836658 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f26300588d0 con 0x7f26480ffc60 2024-09-10T12:09:26.584 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.582+0000 7f264e46e700 1 -- 172.21.15.144:0/629836658 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f264802cca0 con 0x7f263406e200 2024-09-10T12:09:26.594 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.593+0000 7f2645ffb700 1 -- 172.21.15.144:0/629836658 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7f264802cca0 con 0x7f263406e200 2024-09-10T12:09:26.594 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:09:26.594 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (37m) 42s ago 48m 23.5M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:09:26.595 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (56s) 42s ago 48m 8132k - 19.1.1-305-g1cf819b7 41d671128357 59d89239373f 2024-09-10T12:09:26.595 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (47s) 39s ago 47m 7415k - 19.1.1-305-g1cf819b7 41d671128357 79ad3d40fa4d 2024-09-10T12:09:26.595 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (35m) 42s ago 48m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:09:26.595 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (35m) 39s ago 47m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:09:26.595 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (37m) 42s ago 47m 79.3M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:09:26.595 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (31m) 42s ago 42m 62.2M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:09:26.595 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (31m) 42s ago 42m 65.2M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:09:26.595 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (16m) 39s ago 42m 200M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:09:26.595 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (31m) 39s ago 42m 204M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:09:26.595 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (39m) 42s ago 49m 511M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:09:26.595 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (38m) 39s ago 47m 449M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:09:26.595 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (36m) 42s ago 49m 88.6M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:09:26.595 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (35m) 39s ago 47m 59.8M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:09:26.596 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (38m) 42s ago 48m 16.8M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:09:26.596 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (38m) 39s ago 47m 15.6M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:09:26.596 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (34m) 42s ago 46m 104M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:09:26.596 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (34m) 42s ago 45m 105M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:09:26.596 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (33m) 42s ago 45m 95.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:09:26.596 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (33m) 39s ago 45m 119M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:09:26.596 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (32m) 39s ago 44m 101M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:09:26.596 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (32m) 39s ago 44m 98.4M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:09:26.596 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (37m) 42s ago 47m 101M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:09:26.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.595+0000 7f264e46e700 1 -- 172.21.15.144:0/629836658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f263406e200 msgr2=0x7f26340706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:26.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.595+0000 7f264e46e700 1 --2- 172.21.15.144:0/629836658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f263406e200 0x7f26340706b0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2638009c60 tx=0x7f2638009260 comp rx=0 tx=0).stop 2024-09-10T12:09:26.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.595+0000 7f264e46e700 1 -- 172.21.15.144:0/629836658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26480ffc60 msgr2=0x7f2648101210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:26.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.595+0000 7f264e46e700 1 --2- 172.21.15.144:0/629836658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26480ffc60 0x7f2648101210 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f26300059d0 tx=0x7f263000f690 comp rx=0 tx=0).stop 2024-09-10T12:09:26.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.595+0000 7f264e46e700 1 -- 172.21.15.144:0/629836658 shutdown_connections 2024-09-10T12:09:26.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.595+0000 7f264e46e700 1 --2- 172.21.15.144:0/629836658 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f263406e200 0x7f26340706b0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:26.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.595+0000 7f264e46e700 1 --2- 172.21.15.144:0/629836658 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f26480ff2c0 0x7f2648100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:26.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.595+0000 7f264e46e700 1 --2- 172.21.15.144:0/629836658 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f26480ffc60 0x7f2648101210 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:26.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.595+0000 7f264e46e700 1 -- 172.21.15.144:0/629836658 >> 172.21.15.144:0/629836658 conn(0x7f26480fab80 msgr2=0x7f2648068f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:09:26.597 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.595+0000 7f264e46e700 1 -- 172.21.15.144:0/629836658 shutdown_connections 2024-09-10T12:09:26.598 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:26.595+0000 7f264e46e700 1 -- 172.21.15.144:0/629836658 wait complete. 2024-09-10T12:09:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:26 smithi178 ceph-mon[65940]: pgmap v1126: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:26 smithi144 ceph-mon[87436]: pgmap v1126: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:27.230 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph orch upgrade status' 2024-09-10T12:09:27.441 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:09:28.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:27 smithi178 ceph-mon[65940]: from='client.36032 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:09:28.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:27 smithi144 ceph-mon[87436]: from='client.36032 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:09:28.983 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.980+0000 7f2535d8e700 1 -- 172.21.15.144:0/4207994179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2530100430 msgr2=0x7f2530100800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:28.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.980+0000 7f2535d8e700 1 --2- 172.21.15.144:0/4207994179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2530100430 0x7f2530100800 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f2520009910 tx=0x7f2520009c20 comp rx=0 tx=0).stop 2024-09-10T12:09:28.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.980+0000 7f2535d8e700 1 -- 172.21.15.144:0/4207994179 shutdown_connections 2024-09-10T12:09:28.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.980+0000 7f2535d8e700 1 --2- 172.21.15.144:0/4207994179 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2530100d40 0x7f25301094a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:28.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.980+0000 7f2535d8e700 1 --2- 172.21.15.144:0/4207994179 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2530100430 0x7f2530100800 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:28.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.980+0000 7f2535d8e700 1 -- 172.21.15.144:0/4207994179 >> 172.21.15.144:0/4207994179 conn(0x7f25300748b0 msgr2=0x7f2530074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:09:28.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.980+0000 7f2535d8e700 1 -- 172.21.15.144:0/4207994179 shutdown_connections 2024-09-10T12:09:28.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.980+0000 7f2535d8e700 1 -- 172.21.15.144:0/4207994179 wait complete. 2024-09-10T12:09:28.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.981+0000 7f2535d8e700 1 Processor -- start 2024-09-10T12:09:28.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.981+0000 7f2535d8e700 1 -- start start 2024-09-10T12:09:28.984 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.982+0000 7f2535d8e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2530100430 0x7f2530195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:09:28.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.982+0000 7f2535d8e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2530100d40 0x7f2530196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:09:28.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.982+0000 7f2535d8e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2530196a50 con 0x7f2530100d40 2024-09-10T12:09:28.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.982+0000 7f2535d8e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253019a7e0 con 0x7f2530100430 2024-09-10T12:09:28.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.982+0000 7f252effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2530100d40 0x7f2530196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:09:28.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.982+0000 7f252effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2530100d40 0x7f2530196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41676/0 (socket says 172.21.15.144:41676) 2024-09-10T12:09:28.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.982+0000 7f252effd700 1 -- 172.21.15.144:0/2723102264 learned_addr learned my addr 172.21.15.144:0/2723102264 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:09:28.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.982+0000 7f252f7fe700 1 --2- 172.21.15.144:0/2723102264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2530100430 0x7f2530195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:09:28.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.982+0000 7f252effd700 1 -- 172.21.15.144:0/2723102264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2530100430 msgr2=0x7f2530195e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:28.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.982+0000 7f252effd700 1 --2- 172.21.15.144:0/2723102264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2530100430 0x7f2530195e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:28.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.982+0000 7f252effd700 1 -- 172.21.15.144:0/2723102264 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25200095f0 con 0x7f2530100d40 2024-09-10T12:09:28.985 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.982+0000 7f252effd700 1 --2- 172.21.15.144:0/2723102264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2530100d40 0x7f2530196370 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f252400b4b0 tx=0x7f252400b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:09:28.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.983+0000 7f252cff9700 1 -- 172.21.15.144:0/2723102264 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2524011800 con 0x7f2530100d40 2024-09-10T12:09:28.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.983+0000 7f252cff9700 1 -- 172.21.15.144:0/2723102264 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2524011e20 con 0x7f2530100d40 2024-09-10T12:09:28.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.983+0000 7f2535d8e700 1 -- 172.21.15.144:0/2723102264 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253019aac0 con 0x7f2530100d40 2024-09-10T12:09:28.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.983+0000 7f2535d8e700 1 -- 172.21.15.144:0/2723102264 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253019b010 con 0x7f2530100d40 2024-09-10T12:09:28.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.983+0000 7f252cff9700 1 -- 172.21.15.144:0/2723102264 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f252400f2e0 con 0x7f2530100d40 2024-09-10T12:09:28.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.984+0000 7f2535d8e700 1 -- 172.21.15.144:0/2723102264 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2530107e10 con 0x7f2530100d40 2024-09-10T12:09:28.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.984+0000 7f252cff9700 1 -- 172.21.15.144:0/2723102264 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f2524010380 con 0x7f2530100d40 2024-09-10T12:09:28.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.985+0000 7f252cff9700 1 --2- 172.21.15.144:0/2723102264 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f251806e200 0x7f25180706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:09:28.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.985+0000 7f252cff9700 1 -- 172.21.15.144:0/2723102264 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f252408dc80 con 0x7f2530100d40 2024-09-10T12:09:28.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.985+0000 7f252f7fe700 1 --2- 172.21.15.144:0/2723102264 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f251806e200 0x7f25180706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:09:28.987 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.985+0000 7f252f7fe700 1 --2- 172.21.15.144:0/2723102264 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f251806e200 0x7f25180706b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2520006010 tx=0x7f252001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:09:28.990 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:28.989+0000 7f252cff9700 1 -- 172.21.15.144:0/2723102264 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f2524057060 con 0x7f2530100d40 2024-09-10T12:09:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:28 smithi178 ceph-mon[65940]: pgmap v1127: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:09:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:09:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:28 smithi144 ceph-mon[87436]: pgmap v1127: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:09:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:09:29.219 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:29.218+0000 7f2535d8e700 1 -- 172.21.15.144:0/2723102264 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f253019b350 con 0x7f251806e200 2024-09-10T12:09:29.220 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:29.219+0000 7f252cff9700 1 -- 172.21.15.144:0/2723102264 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f253019b350 con 0x7f251806e200 2024-09-10T12:09:29.220 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:09:29.221 INFO:teuthology.orchestra.run.smithi144.stdout: "target_image": null, 2024-09-10T12:09:29.221 INFO:teuthology.orchestra.run.smithi144.stdout: "in_progress": false, 2024-09-10T12:09:29.221 INFO:teuthology.orchestra.run.smithi144.stdout: "which": "", 2024-09-10T12:09:29.221 INFO:teuthology.orchestra.run.smithi144.stdout: "services_complete": [], 2024-09-10T12:09:29.221 INFO:teuthology.orchestra.run.smithi144.stdout: "progress": null, 2024-09-10T12:09:29.221 INFO:teuthology.orchestra.run.smithi144.stdout: "message": "", 2024-09-10T12:09:29.221 INFO:teuthology.orchestra.run.smithi144.stdout: "is_paused": false 2024-09-10T12:09:29.221 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:09:29.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:29.221+0000 7f2535d8e700 1 -- 172.21.15.144:0/2723102264 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f251806e200 msgr2=0x7f25180706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:29.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:29.221+0000 7f2535d8e700 1 --2- 172.21.15.144:0/2723102264 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f251806e200 0x7f25180706b0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f2520006010 tx=0x7f252001a040 comp rx=0 tx=0).stop 2024-09-10T12:09:29.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:29.221+0000 7f2535d8e700 1 -- 172.21.15.144:0/2723102264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2530100d40 msgr2=0x7f2530196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:29.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:29.221+0000 7f2535d8e700 1 --2- 172.21.15.144:0/2723102264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2530100d40 0x7f2530196370 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f252400b4b0 tx=0x7f252400b870 comp rx=0 tx=0).stop 2024-09-10T12:09:29.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:29.221+0000 7f2535d8e700 1 -- 172.21.15.144:0/2723102264 shutdown_connections 2024-09-10T12:09:29.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:29.221+0000 7f2535d8e700 1 --2- 172.21.15.144:0/2723102264 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f251806e200 0x7f25180706b0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:29.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:29.221+0000 7f2535d8e700 1 --2- 172.21.15.144:0/2723102264 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f2530100430 0x7f2530195e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:29.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:29.221+0000 7f2535d8e700 1 --2- 172.21.15.144:0/2723102264 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f2530100d40 0x7f2530196370 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:29.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:29.221+0000 7f2535d8e700 1 -- 172.21.15.144:0/2723102264 >> 172.21.15.144:0/2723102264 conn(0x7f25300748b0 msgr2=0x7f25300fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:09:29.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:29.221+0000 7f2535d8e700 1 -- 172.21.15.144:0/2723102264 shutdown_connections 2024-09-10T12:09:29.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:29.221+0000 7f2535d8e700 1 -- 172.21.15.144:0/2723102264 wait complete. 2024-09-10T12:09:29.894 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph health detail' 2024-09-10T12:09:30.096 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:09:31.121 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:30 smithi144 ceph-mon[87436]: from='client.36036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:09:31.121 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:30 smithi144 ceph-mon[87436]: pgmap v1128: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:09:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:30 smithi178 ceph-mon[65940]: from='client.36036 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:09:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:30 smithi178 ceph-mon[65940]: pgmap v1128: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:09:31.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.665+0000 7fd9bc44d700 1 -- 172.21.15.144:0/1515645627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9b41003d0 msgr2=0x7fd9b41007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:31.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.665+0000 7fd9bc44d700 1 --2- 172.21.15.144:0/1515645627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9b41003d0 0x7fd9b41007a0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009910 tx=0x7fd9b0009c20 comp rx=0 tx=0).stop 2024-09-10T12:09:31.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.666+0000 7fd9bc44d700 1 -- 172.21.15.144:0/1515645627 shutdown_connections 2024-09-10T12:09:31.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.666+0000 7fd9bc44d700 1 --2- 172.21.15.144:0/1515645627 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b4100d70 0x7fd9b4104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:31.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.666+0000 7fd9bc44d700 1 --2- 172.21.15.144:0/1515645627 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9b41003d0 0x7fd9b41007a0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:31.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.666+0000 7fd9bc44d700 1 -- 172.21.15.144:0/1515645627 >> 172.21.15.144:0/1515645627 conn(0x7fd9b40fbc90 msgr2=0x7fd9b40fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:09:31.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.666+0000 7fd9bc44d700 1 -- 172.21.15.144:0/1515645627 shutdown_connections 2024-09-10T12:09:31.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.666+0000 7fd9bc44d700 1 -- 172.21.15.144:0/1515645627 wait complete. 2024-09-10T12:09:31.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.666+0000 7fd9bc44d700 1 Processor -- start 2024-09-10T12:09:31.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.666+0000 7fd9bc44d700 1 -- start start 2024-09-10T12:09:31.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.666+0000 7fd9bc44d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b41003d0 0x7fd9b4195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:09:31.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.666+0000 7fd9bc44d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9b4100d70 0x7fd9b41962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:09:31.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.666+0000 7fd9bc44d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b4196980 con 0x7fd9b4100d70 2024-09-10T12:09:31.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.666+0000 7fd9bc44d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b419a710 con 0x7fd9b41003d0 2024-09-10T12:09:31.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.667+0000 7fd9b99e8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9b4100d70 0x7fd9b41962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:09:31.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.667+0000 7fd9b99e8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9b4100d70 0x7fd9b41962a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:41682/0 (socket says 172.21.15.144:41682) 2024-09-10T12:09:31.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.667+0000 7fd9b99e8700 1 -- 172.21.15.144:0/1694785987 learned_addr learned my addr 172.21.15.144:0/1694785987 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:09:31.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.667+0000 7fd9ba1e9700 1 --2- 172.21.15.144:0/1694785987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b41003d0 0x7fd9b4195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:09:31.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.667+0000 7fd9b99e8700 1 -- 172.21.15.144:0/1694785987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b41003d0 msgr2=0x7fd9b4195d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:31.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.667+0000 7fd9b99e8700 1 --2- 172.21.15.144:0/1694785987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b41003d0 0x7fd9b4195d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:31.669 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.667+0000 7fd9b99e8700 1 -- 172.21.15.144:0/1694785987 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b00095f0 con 0x7fd9b4100d70 2024-09-10T12:09:31.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.667+0000 7fd9b99e8700 1 --2- 172.21.15.144:0/1694785987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9b4100d70 0x7fd9b41962a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400c8f0 tx=0x7fd9a400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:09:31.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.667+0000 7fd9ab7fe700 1 -- 172.21.15.144:0/1694785987 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9a4007a50 con 0x7fd9b4100d70 2024-09-10T12:09:31.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.667+0000 7fd9bc44d700 1 -- 172.21.15.144:0/1694785987 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b419a9f0 con 0x7fd9b4100d70 2024-09-10T12:09:31.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.667+0000 7fd9ab7fe700 1 -- 172.21.15.144:0/1694785987 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd9a4007bb0 con 0x7fd9b4100d70 2024-09-10T12:09:31.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.667+0000 7fd9ab7fe700 1 -- 172.21.15.144:0/1694785987 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9a4018630 con 0x7fd9b4100d70 2024-09-10T12:09:31.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.667+0000 7fd9bc44d700 1 -- 172.21.15.144:0/1694785987 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b419af40 con 0x7fd9b4100d70 2024-09-10T12:09:31.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.668+0000 7fd9bc44d700 1 -- 172.21.15.144:0/1694785987 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b41086e0 con 0x7fd9b4100d70 2024-09-10T12:09:31.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.669+0000 7fd9ab7fe700 1 -- 172.21.15.144:0/1694785987 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd9a4007d20 con 0x7fd9b4100d70 2024-09-10T12:09:31.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.669+0000 7fd9ab7fe700 1 --2- 172.21.15.144:0/1694785987 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd9a006e200 0x7fd9a00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:09:31.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.669+0000 7fd9ab7fe700 1 -- 172.21.15.144:0/1694785987 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd9a408e070 con 0x7fd9b4100d70 2024-09-10T12:09:31.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.669+0000 7fd9ba1e9700 1 --2- 172.21.15.144:0/1694785987 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd9a006e200 0x7fd9a00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:09:31.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.670+0000 7fd9ba1e9700 1 --2- 172.21.15.144:0/1694785987 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd9a006e200 0x7fd9a00706b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0006010 tx=0x7fd9b001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:09:31.675 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.674+0000 7fd9ab7fe700 1 -- 172.21.15.144:0/1694785987 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd9a4057450 con 0x7fd9b4100d70 2024-09-10T12:09:31.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.961+0000 7fd9bc44d700 1 -- 172.21.15.144:0/1694785987 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fd9b404ea50 con 0x7fd9b4100d70 2024-09-10T12:09:31.963 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.962+0000 7fd9ab7fe700 1 -- 172.21.15.144:0/1694785987 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+201 (secure 0 0 0) 0x7fd9a405ab10 con 0x7fd9b4100d70 2024-09-10T12:09:31.963 INFO:teuthology.orchestra.run.smithi144.stdout:HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T12:09:31.963 INFO:teuthology.orchestra.run.smithi144.stdout:[WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:09:31.964 INFO:teuthology.orchestra.run.smithi144.stdout: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:09:31.965 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.964+0000 7fd9bc44d700 1 -- 172.21.15.144:0/1694785987 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd9a006e200 msgr2=0x7fd9a00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:31.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.964+0000 7fd9bc44d700 1 --2- 172.21.15.144:0/1694785987 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd9a006e200 0x7fd9a00706b0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0006010 tx=0x7fd9b001a040 comp rx=0 tx=0).stop 2024-09-10T12:09:31.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.964+0000 7fd9bc44d700 1 -- 172.21.15.144:0/1694785987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9b4100d70 msgr2=0x7fd9b41962a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:31.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.964+0000 7fd9bc44d700 1 --2- 172.21.15.144:0/1694785987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9b4100d70 0x7fd9b41962a0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fd9a400c8f0 tx=0x7fd9a400ccb0 comp rx=0 tx=0).stop 2024-09-10T12:09:31.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.964+0000 7fd9bc44d700 1 -- 172.21.15.144:0/1694785987 shutdown_connections 2024-09-10T12:09:31.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.964+0000 7fd9bc44d700 1 --2- 172.21.15.144:0/1694785987 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd9a006e200 0x7fd9a00706b0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:31.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.964+0000 7fd9bc44d700 1 --2- 172.21.15.144:0/1694785987 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd9b41003d0 0x7fd9b4195d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:31.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.964+0000 7fd9bc44d700 1 --2- 172.21.15.144:0/1694785987 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd9b4100d70 0x7fd9b41962a0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:31.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.964+0000 7fd9bc44d700 1 -- 172.21.15.144:0/1694785987 >> 172.21.15.144:0/1694785987 conn(0x7fd9b40fbc90 msgr2=0x7fd9b40fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:09:31.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.964+0000 7fd9bc44d700 1 -- 172.21.15.144:0/1694785987 shutdown_connections 2024-09-10T12:09:31.966 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:31.964+0000 7fd9bc44d700 1 -- 172.21.15.144:0/1694785987 wait complete. 2024-09-10T12:09:32.642 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph versions' 2024-09-10T12:09:32.853 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:09:33.156 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:32 smithi144 ceph-mon[87436]: pgmap v1129: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:33.156 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:32 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1694785987' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:09:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:32 smithi178 ceph-mon[65940]: pgmap v1129: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:32 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1694785987' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-09-10T12:09:34.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.421+0000 7fc02a502700 1 -- 172.21.15.144:0/2039415970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0240ffcb0 msgr2=0x7fc024103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:34.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.421+0000 7fc02a502700 1 --2- 172.21.15.144:0/2039415970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0240ffcb0 0x7fc024103c80 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009910 tx=0x7fc00c009c20 comp rx=0 tx=0).stop 2024-09-10T12:09:34.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.421+0000 7fc02a502700 1 -- 172.21.15.144:0/2039415970 shutdown_connections 2024-09-10T12:09:34.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.421+0000 7fc02a502700 1 --2- 172.21.15.144:0/2039415970 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0240ffcb0 0x7fc024103c80 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:34.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.421+0000 7fc02a502700 1 --2- 172.21.15.144:0/2039415970 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0240ff310 0x7fc0240ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:34.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.421+0000 7fc02a502700 1 -- 172.21.15.144:0/2039415970 >> 172.21.15.144:0/2039415970 conn(0x7fc0240fabd0 msgr2=0x7fc0240fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:09:34.425 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.421+0000 7fc02a502700 1 -- 172.21.15.144:0/2039415970 shutdown_connections 2024-09-10T12:09:34.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.421+0000 7fc02a502700 1 -- 172.21.15.144:0/2039415970 wait complete. 2024-09-10T12:09:34.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.422+0000 7fc02a502700 1 Processor -- start 2024-09-10T12:09:34.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.422+0000 7fc02a502700 1 -- start start 2024-09-10T12:09:34.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc02a502700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0240ff310 0x7fc024194e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:09:34.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc02a502700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0240ffcb0 0x7fc024195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:09:34.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc02a502700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc024195a40 con 0x7fc0240ffcb0 2024-09-10T12:09:34.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc02a502700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0241997d0 con 0x7fc0240ff310 2024-09-10T12:09:34.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc0237fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0240ffcb0 0x7fc024195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:09:34.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc0237fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0240ffcb0 0x7fc024195360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:54392/0 (socket says 172.21.15.144:54392) 2024-09-10T12:09:34.426 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc0237fe700 1 -- 172.21.15.144:0/3891903873 learned_addr learned my addr 172.21.15.144:0/3891903873 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:09:34.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc023fff700 1 --2- 172.21.15.144:0/3891903873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0240ff310 0x7fc024194e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:09:34.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc0237fe700 1 -- 172.21.15.144:0/3891903873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0240ff310 msgr2=0x7fc024194e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:34.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc0237fe700 1 --2- 172.21.15.144:0/3891903873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0240ff310 0x7fc024194e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:34.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc0237fe700 1 -- 172.21.15.144:0/3891903873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c0095f0 con 0x7fc0240ffcb0 2024-09-10T12:09:34.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc023fff700 1 --2- 172.21.15.144:0/3891903873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0240ff310 0x7fc024194e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:09:34.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc0237fe700 1 --2- 172.21.15.144:0/3891903873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0240ffcb0 0x7fc024195360 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0092e0 tx=0x7fc00c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:09:34.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc0217fa700 1 -- 172.21.15.144:0/3891903873 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc00c01f070 con 0x7fc0240ffcb0 2024-09-10T12:09:34.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc0217fa700 1 -- 172.21.15.144:0/3891903873 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc00c019610 con 0x7fc0240ffcb0 2024-09-10T12:09:34.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc02a502700 1 -- 172.21.15.144:0/3891903873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc024199a50 con 0x7fc0240ffcb0 2024-09-10T12:09:34.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc0217fa700 1 -- 172.21.15.144:0/3891903873 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc00c00f7a0 con 0x7fc0240ffcb0 2024-09-10T12:09:34.427 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.423+0000 7fc02a502700 1 -- 172.21.15.144:0/3891903873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc024199f40 con 0x7fc0240ffcb0 2024-09-10T12:09:34.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.424+0000 7fc02a502700 1 -- 172.21.15.144:0/3891903873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc02404ea50 con 0x7fc0240ffcb0 2024-09-10T12:09:34.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.425+0000 7fc0217fa700 1 -- 172.21.15.144:0/3891903873 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc00c019780 con 0x7fc0240ffcb0 2024-09-10T12:09:34.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.425+0000 7fc0217fa700 1 --2- 172.21.15.144:0/3891903873 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc01006e200 0x7fc0100706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:09:34.428 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.425+0000 7fc0217fa700 1 -- 172.21.15.144:0/3891903873 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc00c0918e0 con 0x7fc0240ffcb0 2024-09-10T12:09:34.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.425+0000 7fc023fff700 1 --2- 172.21.15.144:0/3891903873 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc01006e200 0x7fc0100706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:09:34.429 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.426+0000 7fc023fff700 1 --2- 172.21.15.144:0/3891903873 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc01006e200 0x7fc0100706b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc0140099d0 tx=0x7fc014008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:09:34.431 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.430+0000 7fc0217fa700 1 -- 172.21.15.144:0/3891903873 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc00c05ad70 con 0x7fc0240ffcb0 2024-09-10T12:09:34.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.718+0000 7fc02a502700 1 -- 172.21.15.144:0/3891903873 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc024196180 con 0x7fc0240ffcb0 2024-09-10T12:09:34.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.718+0000 7fc0217fa700 1 -- 172.21.15.144:0/3891903873 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7fc00c05e430 con 0x7fc0240ffcb0 2024-09-10T12:09:34.720 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:09:34.720 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:09:34.720 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:09:34.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:09:34.720 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:09:34.720 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:09:34.720 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:09:34.720 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:09:34.720 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:09:34.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:09:34.721 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:09:34.721 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:09:34.721 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:09:34.721 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:09:34.721 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:09:34.721 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:09:34.721 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:09:34.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.720+0000 7fc02a502700 1 -- 172.21.15.144:0/3891903873 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc01006e200 msgr2=0x7fc0100706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:34.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.720+0000 7fc02a502700 1 --2- 172.21.15.144:0/3891903873 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc01006e200 0x7fc0100706b0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc0140099d0 tx=0x7fc014008040 comp rx=0 tx=0).stop 2024-09-10T12:09:34.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.720+0000 7fc02a502700 1 -- 172.21.15.144:0/3891903873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0240ffcb0 msgr2=0x7fc024195360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:09:34.722 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.720+0000 7fc02a502700 1 --2- 172.21.15.144:0/3891903873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0240ffcb0 0x7fc024195360 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fc00c0092e0 tx=0x7fc00c00be20 comp rx=0 tx=0).stop 2024-09-10T12:09:34.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.721+0000 7fc02a502700 1 -- 172.21.15.144:0/3891903873 shutdown_connections 2024-09-10T12:09:34.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.721+0000 7fc02a502700 1 --2- 172.21.15.144:0/3891903873 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc01006e200 0x7fc0100706b0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:34.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.721+0000 7fc02a502700 1 --2- 172.21.15.144:0/3891903873 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc0240ff310 0x7fc024194e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:34.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.721+0000 7fc02a502700 1 --2- 172.21.15.144:0/3891903873 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc0240ffcb0 0x7fc024195360 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:09:34.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.721+0000 7fc02a502700 1 -- 172.21.15.144:0/3891903873 >> 172.21.15.144:0/3891903873 conn(0x7fc0240fabd0 msgr2=0x7fc0240fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:09:34.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.721+0000 7fc02a502700 1 -- 172.21.15.144:0/3891903873 shutdown_connections 2024-09-10T12:09:34.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:09:34.721+0000 7fc02a502700 1 -- 172.21.15.144:0/3891903873 wait complete. 2024-09-10T12:09:35.092 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:34 smithi144 ceph-mon[87436]: pgmap v1130: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:09:35.093 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:34 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3891903873' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:09:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:34 smithi178 ceph-mon[65940]: pgmap v1130: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:09:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:34 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3891903873' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:09:35.408 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-09-10T12:09:35.608 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:09:36.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:36 smithi144 ceph-mon[87436]: pgmap v1131: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:37.187 INFO:teuthology.orchestra.run.smithi144.stdout:wait for servicemap items w/ changing names to refresh 2024-09-10T12:09:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:36 smithi178 ceph-mon[65940]: pgmap v1131: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:37.781 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'sleep 60' 2024-09-10T12:09:37.986 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:09:39.064 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:38 smithi144 ceph-mon[87436]: pgmap v1132: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:09:39.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:38 smithi178 ceph-mon[65940]: pgmap v1132: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:09:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:40 smithi144 ceph-mon[87436]: pgmap v1133: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:09:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:40 smithi178 ceph-mon[65940]: pgmap v1133: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:09:43.169 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:42 smithi178 ceph-mon[65940]: pgmap v1134: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:43.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:42 smithi144 ceph-mon[87436]: pgmap v1134: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:44.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:09:44.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:09:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:44 smithi144 ceph-mon[87436]: pgmap v1135: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:09:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:44 smithi178 ceph-mon[65940]: pgmap v1135: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:09:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:46 smithi144 ceph-mon[87436]: pgmap v1136: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:46 smithi178 ceph-mon[65940]: pgmap v1136: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:49.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:48 smithi144 ceph-mon[87436]: pgmap v1137: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:09:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:48 smithi178 ceph-mon[65940]: pgmap v1137: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:09:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:50 smithi178 ceph-mon[65940]: pgmap v1138: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:09:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:50 smithi144 ceph-mon[87436]: pgmap v1138: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:09:53.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:52 smithi144 ceph-mon[87436]: pgmap v1139: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:53.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:52 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:09:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:52 smithi178 ceph-mon[65940]: pgmap v1139: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:52 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:09:54.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:09:54.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:09:54.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:53 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:09:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:09:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:09:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:53 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:09:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:54 smithi144 ceph-mon[87436]: pgmap v1140: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:09:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:54 smithi178 ceph-mon[65940]: pgmap v1140: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:09:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:56 smithi144 ceph-mon[87436]: pgmap v1141: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:56 smithi178 ceph-mon[65940]: pgmap v1141: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:09:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:58 smithi144 ceph-mon[87436]: pgmap v1142: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:09:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:09:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:09:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:58 smithi178 ceph-mon[65940]: pgmap v1142: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:09:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:09:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:10:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:00 smithi178 ceph-mon[65940]: pgmap v1143: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:10:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:00 smithi178 ceph-mon[65940]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T12:10:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:00 smithi178 ceph-mon[65940]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:10:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:00 smithi178 ceph-mon[65940]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:10:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:00 smithi144 ceph-mon[87436]: pgmap v1143: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:10:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:00 smithi144 ceph-mon[87436]: Health detail: HEALTH_WARN 1 filesystem with deprecated feature inline_data 2024-09-10T12:10:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:00 smithi144 ceph-mon[87436]: [WRN] FS_INLINE_DATA_DEPRECATED: 1 filesystem with deprecated feature inline_data 2024-09-10T12:10:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:00 smithi144 ceph-mon[87436]: fs cephfs has deprecated feature inline_data enabled. 2024-09-10T12:10:03.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:02 smithi144 ceph-mon[87436]: pgmap v1144: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:02 smithi178 ceph-mon[65940]: pgmap v1144: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:05.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:04 smithi144 ceph-mon[87436]: pgmap v1145: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:10:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:04 smithi178 ceph-mon[65940]: pgmap v1145: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:10:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:06 smithi144 ceph-mon[87436]: pgmap v1146: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:06 smithi178 ceph-mon[65940]: pgmap v1146: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:08 smithi144 ceph-mon[87436]: pgmap v1147: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:10:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:08 smithi178 ceph-mon[65940]: pgmap v1147: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:10:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:10 smithi178 ceph-mon[65940]: pgmap v1148: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:10:11.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:10 smithi144 ceph-mon[87436]: pgmap v1148: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:10:13.236 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:12 smithi144 ceph-mon[87436]: pgmap v1149: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:12 smithi178 ceph-mon[65940]: pgmap v1149: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:10:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:10:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:14 smithi144 ceph-mon[87436]: pgmap v1150: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:10:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:14 smithi178 ceph-mon[65940]: pgmap v1150: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:10:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:16 smithi178 ceph-mon[65940]: pgmap v1151: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:17.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:16 smithi144 ceph-mon[87436]: pgmap v1151: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:19.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:18 smithi178 ceph-mon[65940]: pgmap v1152: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:10:19.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:18 smithi144 ceph-mon[87436]: pgmap v1152: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:10:21.232 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:20 smithi178 ceph-mon[65940]: pgmap v1153: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:10:21.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:20 smithi144 ceph-mon[87436]: pgmap v1153: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:10:23.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:22 smithi178 ceph-mon[65940]: pgmap v1154: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:23.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:22 smithi144 ceph-mon[87436]: pgmap v1154: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:24 smithi178 ceph-mon[65940]: pgmap v1155: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:10:25.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:24 smithi144 ceph-mon[87436]: pgmap v1155: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:10:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:26 smithi178 ceph-mon[65940]: pgmap v1156: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:27.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:26 smithi144 ceph-mon[87436]: pgmap v1156: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:28 smithi178 ceph-mon[65940]: pgmap v1157: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:10:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:10:29.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:28 smithi144 ceph-mon[87436]: pgmap v1157: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:10:29.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:10:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:30 smithi178 ceph-mon[65940]: pgmap v1158: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:10:31.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:30 smithi144 ceph-mon[87436]: pgmap v1158: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:10:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:32 smithi178 ceph-mon[65940]: pgmap v1159: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:33.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:32 smithi144 ceph-mon[87436]: pgmap v1159: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:34 smithi178 ceph-mon[65940]: pgmap v1160: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:10:35.437 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:34 smithi144 ceph-mon[87436]: pgmap v1160: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:10:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:36 smithi178 ceph-mon[65940]: pgmap v1161: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:37.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:36 smithi144 ceph-mon[87436]: pgmap v1161: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:39.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:38 smithi178 ceph-mon[65940]: pgmap v1162: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:10:39.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:38 smithi144 ceph-mon[87436]: pgmap v1162: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:10:40.179 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph orch ps' 2024-09-10T12:10:40.390 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:10:41.117 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:40 smithi144 ceph-mon[87436]: pgmap v1163: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:10:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:40 smithi178 ceph-mon[65940]: pgmap v1163: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:10:41.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.988+0000 7fc230f48700 1 -- 172.21.15.144:0/1517590187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc22c0ff320 msgr2=0x7fc22c0ff6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:41.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.988+0000 7fc230f48700 1 --2- 172.21.15.144:0/1517590187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc22c0ff320 0x7fc22c0ff6f0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009940 tx=0x7fc21c009c50 comp rx=0 tx=0).stop 2024-09-10T12:10:41.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.988+0000 7fc230f48700 1 -- 172.21.15.144:0/1517590187 shutdown_connections 2024-09-10T12:10:41.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.988+0000 7fc230f48700 1 --2- 172.21.15.144:0/1517590187 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc22c0ffc30 0x7fc22c10a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:41.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.988+0000 7fc230f48700 1 --2- 172.21.15.144:0/1517590187 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc22c0ff320 0x7fc22c0ff6f0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:41.992 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.988+0000 7fc230f48700 1 -- 172.21.15.144:0/1517590187 >> 172.21.15.144:0/1517590187 conn(0x7fc22c0748b0 msgr2=0x7fc22c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:10:41.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.988+0000 7fc230f48700 1 -- 172.21.15.144:0/1517590187 shutdown_connections 2024-09-10T12:10:41.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.988+0000 7fc230f48700 1 -- 172.21.15.144:0/1517590187 wait complete. 2024-09-10T12:10:41.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.989+0000 7fc230f48700 1 Processor -- start 2024-09-10T12:10:41.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.989+0000 7fc230f48700 1 -- start start 2024-09-10T12:10:41.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.989+0000 7fc230f48700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc22c0ff320 0x7fc22c100500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:41.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.989+0000 7fc230f48700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc22c0ffc30 0x7fc22c100a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:41.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.989+0000 7fc230f48700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c100f80 con 0x7fc22c0ffc30 2024-09-10T12:10:41.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.989+0000 7fc230f48700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc22c1010c0 con 0x7fc22c0ff320 2024-09-10T12:10:41.993 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.990+0000 7fc229d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc22c0ffc30 0x7fc22c100a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:41.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.990+0000 7fc22a59c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc22c0ff320 0x7fc22c100500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:41.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.990+0000 7fc229d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc22c0ffc30 0x7fc22c100a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57176/0 (socket says 172.21.15.144:57176) 2024-09-10T12:10:41.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.990+0000 7fc229d9b700 1 -- 172.21.15.144:0/1925213044 learned_addr learned my addr 172.21.15.144:0/1925213044 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:10:41.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.990+0000 7fc229d9b700 1 -- 172.21.15.144:0/1925213044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc22c0ff320 msgr2=0x7fc22c100500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:41.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.990+0000 7fc229d9b700 1 --2- 172.21.15.144:0/1925213044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc22c0ff320 0x7fc22c100500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:41.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.990+0000 7fc229d9b700 1 -- 172.21.15.144:0/1925213044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc21c0095f0 con 0x7fc22c0ffc30 2024-09-10T12:10:41.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.990+0000 7fc229d9b700 1 --2- 172.21.15.144:0/1925213044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc22c0ffc30 0x7fc22c100a40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc22000d900 tx=0x7fc22000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:10:41.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.990+0000 7fc21b7fe700 1 -- 172.21.15.144:0/1925213044 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2200041f0 con 0x7fc22c0ffc30 2024-09-10T12:10:41.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.990+0000 7fc21b7fe700 1 -- 172.21.15.144:0/1925213044 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc220004cf0 con 0x7fc22c0ffc30 2024-09-10T12:10:41.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.990+0000 7fc230f48700 1 -- 172.21.15.144:0/1925213044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc22c1a1e20 con 0x7fc22c0ffc30 2024-09-10T12:10:41.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.990+0000 7fc21b7fe700 1 -- 172.21.15.144:0/1925213044 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc22000b750 con 0x7fc22c0ffc30 2024-09-10T12:10:41.994 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.990+0000 7fc230f48700 1 -- 172.21.15.144:0/1925213044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc22c1a2230 con 0x7fc22c0ffc30 2024-09-10T12:10:41.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.991+0000 7fc230f48700 1 -- 172.21.15.144:0/1925213044 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc22c107da0 con 0x7fc22c0ffc30 2024-09-10T12:10:41.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.992+0000 7fc21b7fe700 1 -- 172.21.15.144:0/1925213044 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fc220004750 con 0x7fc22c0ffc30 2024-09-10T12:10:41.995 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.992+0000 7fc21b7fe700 1 --2- 172.21.15.144:0/1925213044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc21406e200 0x7fc2140706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:41.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.992+0000 7fc21b7fe700 1 -- 172.21.15.144:0/1925213044 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fc22008df70 con 0x7fc22c0ffc30 2024-09-10T12:10:41.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.992+0000 7fc22a59c700 1 --2- 172.21.15.144:0/1925213044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc21406e200 0x7fc2140706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:41.996 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.993+0000 7fc22a59c700 1 --2- 172.21.15.144:0/1925213044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc21406e200 0x7fc2140706b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009910 tx=0x7fc21c005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:10:41.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:41.997+0000 7fc21b7fe700 1 -- 172.21.15.144:0/1925213044 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fc2200c31b0 con 0x7fc22c0ffc30 2024-09-10T12:10:42.209 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:42.208+0000 7fc230f48700 1 -- 172.21.15.144:0/1925213044 --> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc22c02cd60 con 0x7fc21406e200 2024-09-10T12:10:42.218 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:42.217+0000 7fc21b7fe700 1 -- 172.21.15.144:0/1925213044 <== mgr.34104 v2:172.21.15.144:6800/2497095016 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3792 (secure 0 0 0) 0x7fc22c02cd60 con 0x7fc21406e200 2024-09-10T12:10:42.219 INFO:teuthology.orchestra.run.smithi144.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-10T12:10:42.219 INFO:teuthology.orchestra.run.smithi144.stdout:alertmanager.smithi144 smithi144 *:9093,9094 running (38m) 118s ago 49m 23.5M - 0.25.0 c8568f914cd2 0410ed2f90fa 2024-09-10T12:10:42.219 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi144 smithi144 running (2m) 118s ago 49m 8132k - 19.1.1-305-g1cf819b7 41d671128357 59d89239373f 2024-09-10T12:10:42.219 INFO:teuthology.orchestra.run.smithi144.stdout:ceph-exporter.smithi178 smithi178 running (2m) 114s ago 48m 7415k - 19.1.1-305-g1cf819b7 41d671128357 79ad3d40fa4d 2024-09-10T12:10:42.219 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi144 smithi144 running (36m) 118s ago 49m 6895k - 19.1.1-305-g1cf819b7 41d671128357 eadfd96ed03a 2024-09-10T12:10:42.220 INFO:teuthology.orchestra.run.smithi144.stdout:crash.smithi178 smithi178 running (36m) 114s ago 48m 6891k - 19.1.1-305-g1cf819b7 41d671128357 7cf8abbbae02 2024-09-10T12:10:42.220 INFO:teuthology.orchestra.run.smithi144.stdout:grafana.smithi144 smithi144 *:3000 running (38m) 118s ago 49m 79.3M - 9.4.12 2bacad6d85d8 b4c257a3c0ec 2024-09-10T12:10:42.220 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.bfyvwv smithi144 running (33m) 118s ago 44m 62.2M - 19.1.1-305-g1cf819b7 41d671128357 8af49bd0efc5 2024-09-10T12:10:42.220 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi144.jaoyvs smithi144 running (32m) 118s ago 43m 65.2M - 19.1.1-305-g1cf819b7 41d671128357 8ac710062b31 2024-09-10T12:10:42.220 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.czjrsa smithi178 running (17m) 114s ago 43m 200M - 19.1.1-305-g1cf819b7 41d671128357 e1a0ca425edf 2024-09-10T12:10:42.220 INFO:teuthology.orchestra.run.smithi144.stdout:mds.cephfs.smithi178.nmhrmh smithi178 running (32m) 114s ago 44m 204M - 19.1.1-305-g1cf819b7 41d671128357 0a199d75c378 2024-09-10T12:10:42.220 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi144.bkogfs smithi144 *:8443,9283,8765 running (40m) 118s ago 50m 511M - 19.1.1-305-g1cf819b7 41d671128357 eda7db996233 2024-09-10T12:10:42.220 INFO:teuthology.orchestra.run.smithi144.stdout:mgr.smithi178.vmvxuf smithi178 *:8443,9283,8765 running (39m) 114s ago 48m 449M - 19.1.1-305-g1cf819b7 41d671128357 e99885a6177f 2024-09-10T12:10:42.220 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi144 smithi144 running (37m) 118s ago 51m 88.6M 2048M 19.1.1-305-g1cf819b7 41d671128357 0fc028b6cc37 2024-09-10T12:10:42.220 INFO:teuthology.orchestra.run.smithi144.stdout:mon.smithi178 smithi178 running (37m) 114s ago 48m 59.8M 2048M 19.1.1-305-g1cf819b7 41d671128357 a1b7b65c960c 2024-09-10T12:10:42.220 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi144 smithi144 *:9100 running (39m) 118s ago 49m 16.8M - 1.5.0 0da6a335fe13 2d2e839ae7a0 2024-09-10T12:10:42.220 INFO:teuthology.orchestra.run.smithi144.stdout:node-exporter.smithi178 smithi178 *:9100 running (39m) 114s ago 48m 15.6M - 1.5.0 0da6a335fe13 de852f408486 2024-09-10T12:10:42.221 INFO:teuthology.orchestra.run.smithi144.stdout:osd.0 smithi144 running (36m) 118s ago 47m 104M 1216M 19.1.1-305-g1cf819b7 41d671128357 b59b6d7c85be 2024-09-10T12:10:42.221 INFO:teuthology.orchestra.run.smithi144.stdout:osd.1 smithi144 running (35m) 118s ago 47m 105M 1216M 19.1.1-305-g1cf819b7 41d671128357 e5d70992955a 2024-09-10T12:10:42.221 INFO:teuthology.orchestra.run.smithi144.stdout:osd.2 smithi144 running (34m) 118s ago 46m 95.3M 1216M 19.1.1-305-g1cf819b7 41d671128357 8485d374ff44 2024-09-10T12:10:42.221 INFO:teuthology.orchestra.run.smithi144.stdout:osd.3 smithi178 running (34m) 114s ago 46m 119M 2239M 19.1.1-305-g1cf819b7 41d671128357 867cd5897d94 2024-09-10T12:10:42.221 INFO:teuthology.orchestra.run.smithi144.stdout:osd.4 smithi178 running (33m) 114s ago 45m 101M 2239M 19.1.1-305-g1cf819b7 41d671128357 400e18bd8874 2024-09-10T12:10:42.221 INFO:teuthology.orchestra.run.smithi144.stdout:osd.5 smithi178 running (33m) 114s ago 45m 98.4M 2239M 19.1.1-305-g1cf819b7 41d671128357 4a6125da2507 2024-09-10T12:10:42.221 INFO:teuthology.orchestra.run.smithi144.stdout:prometheus.smithi144 smithi144 *:9095 running (39m) 118s ago 49m 101M - 2.43.0 a07b618ecd1d 1336f50c7fe2 2024-09-10T12:10:42.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:42.219+0000 7fc230f48700 1 -- 172.21.15.144:0/1925213044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc21406e200 msgr2=0x7fc2140706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:42.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:42.219+0000 7fc230f48700 1 --2- 172.21.15.144:0/1925213044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc21406e200 0x7fc2140706b0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009910 tx=0x7fc21c005ab0 comp rx=0 tx=0).stop 2024-09-10T12:10:42.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:42.219+0000 7fc230f48700 1 -- 172.21.15.144:0/1925213044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc22c0ffc30 msgr2=0x7fc22c100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:42.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:42.219+0000 7fc230f48700 1 --2- 172.21.15.144:0/1925213044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc22c0ffc30 0x7fc22c100a40 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc22000d900 tx=0x7fc22000dc10 comp rx=0 tx=0).stop 2024-09-10T12:10:42.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:42.219+0000 7fc230f48700 1 -- 172.21.15.144:0/1925213044 shutdown_connections 2024-09-10T12:10:42.222 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:42.219+0000 7fc230f48700 1 --2- 172.21.15.144:0/1925213044 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fc21406e200 0x7fc2140706b0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:42.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:42.219+0000 7fc230f48700 1 --2- 172.21.15.144:0/1925213044 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fc22c0ff320 0x7fc22c100500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:42.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:42.219+0000 7fc230f48700 1 --2- 172.21.15.144:0/1925213044 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fc22c0ffc30 0x7fc22c100a40 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:42.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:42.219+0000 7fc230f48700 1 -- 172.21.15.144:0/1925213044 >> 172.21.15.144:0/1925213044 conn(0x7fc22c0748b0 msgr2=0x7fc22c104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:10:42.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:42.219+0000 7fc230f48700 1 -- 172.21.15.144:0/1925213044 shutdown_connections 2024-09-10T12:10:42.223 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:42.219+0000 7fc230f48700 1 -- 172.21.15.144:0/1925213044 wait complete. 2024-09-10T12:10:42.980 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph versions' 2024-09-10T12:10:43.195 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:10:43.315 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:42 smithi144 ceph-mon[87436]: pgmap v1164: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:42 smithi178 ceph-mon[65940]: pgmap v1164: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:44.307 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:43 smithi144 ceph-mon[87436]: from='client.36048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:10:44.307 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:10:44.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:43 smithi178 ceph-mon[65940]: from='client.36048 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T12:10:44.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:10:44.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.844+0000 7f64b1e87700 1 -- 172.21.15.144:0/3522865545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64ac100d70 msgr2=0x7f64ac104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:44.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.844+0000 7f64b1e87700 1 --2- 172.21.15.144:0/3522865545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64ac100d70 0x7f64ac104d40 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f6494009910 tx=0x7f6494009c20 comp rx=0 tx=0).stop 2024-09-10T12:10:44.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.844+0000 7f64b1e87700 1 -- 172.21.15.144:0/3522865545 shutdown_connections 2024-09-10T12:10:44.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.844+0000 7f64b1e87700 1 --2- 172.21.15.144:0/3522865545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64ac100d70 0x7f64ac104d40 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:44.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.844+0000 7f64b1e87700 1 --2- 172.21.15.144:0/3522865545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64ac1003d0 0x7f64ac1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:44.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.844+0000 7f64b1e87700 1 -- 172.21.15.144:0/3522865545 >> 172.21.15.144:0/3522865545 conn(0x7f64ac0fbc90 msgr2=0x7f64ac0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:10:44.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.845+0000 7f64b1e87700 1 -- 172.21.15.144:0/3522865545 shutdown_connections 2024-09-10T12:10:44.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.845+0000 7f64b1e87700 1 -- 172.21.15.144:0/3522865545 wait complete. 2024-09-10T12:10:44.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.845+0000 7f64b1e87700 1 Processor -- start 2024-09-10T12:10:44.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.845+0000 7f64b1e87700 1 -- start start 2024-09-10T12:10:44.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.845+0000 7f64b1e87700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64ac1003d0 0x7f64ac195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:44.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.845+0000 7f64b1e87700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64ac100d70 0x7f64ac1963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:44.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.845+0000 7f64b1e87700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac196aa0 con 0x7f64ac1003d0 2024-09-10T12:10:44.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.845+0000 7f64b1e87700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64ac19a830 con 0x7f64ac100d70 2024-09-10T12:10:44.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.846+0000 7f64ab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64ac1003d0 0x7f64ac195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:44.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.846+0000 7f64ab7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64ac1003d0 0x7f64ac195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:49768/0 (socket says 172.21.15.144:49768) 2024-09-10T12:10:44.849 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.846+0000 7f64ab7fe700 1 -- 172.21.15.144:0/3534488359 learned_addr learned my addr 172.21.15.144:0/3534488359 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:10:44.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.846+0000 7f64a3fff700 1 --2- 172.21.15.144:0/3534488359 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64ac100d70 0x7f64ac1963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:44.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.846+0000 7f64ab7fe700 1 -- 172.21.15.144:0/3534488359 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64ac100d70 msgr2=0x7f64ac1963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:44.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.846+0000 7f64ab7fe700 1 --2- 172.21.15.144:0/3534488359 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64ac100d70 0x7f64ac1963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:44.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.846+0000 7f64ab7fe700 1 -- 172.21.15.144:0/3534488359 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64940095f0 con 0x7f64ac1003d0 2024-09-10T12:10:44.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.846+0000 7f64ab7fe700 1 --2- 172.21.15.144:0/3534488359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64ac1003d0 0x7f64ac195e80 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f649c00b4b0 tx=0x7f649c00b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:10:44.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.846+0000 7f64a97fa700 1 -- 172.21.15.144:0/3534488359 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f649c011800 con 0x7f64ac1003d0 2024-09-10T12:10:44.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.846+0000 7f64a97fa700 1 -- 172.21.15.144:0/3534488359 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f649c011e20 con 0x7f64ac1003d0 2024-09-10T12:10:44.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.846+0000 7f64b1e87700 1 -- 172.21.15.144:0/3534488359 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64ac19ab10 con 0x7f64ac1003d0 2024-09-10T12:10:44.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.846+0000 7f64a97fa700 1 -- 172.21.15.144:0/3534488359 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f649c00f2e0 con 0x7f64ac1003d0 2024-09-10T12:10:44.850 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.846+0000 7f64b1e87700 1 -- 172.21.15.144:0/3534488359 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64ac19b030 con 0x7f64ac1003d0 2024-09-10T12:10:44.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.847+0000 7f64b1e87700 1 -- 172.21.15.144:0/3534488359 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64ac108670 con 0x7f64ac1003d0 2024-09-10T12:10:44.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.848+0000 7f64a97fa700 1 -- 172.21.15.144:0/3534488359 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f649c010380 con 0x7f64ac1003d0 2024-09-10T12:10:44.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.848+0000 7f64a97fa700 1 --2- 172.21.15.144:0/3534488359 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f649806e200 0x7f64980706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:44.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.848+0000 7f64a97fa700 1 -- 172.21.15.144:0/3534488359 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f649c08dab0 con 0x7f64ac1003d0 2024-09-10T12:10:44.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.848+0000 7f64a3fff700 1 --2- 172.21.15.144:0/3534488359 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f649806e200 0x7f64980706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:44.851 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.849+0000 7f64a3fff700 1 --2- 172.21.15.144:0/3534488359 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f649806e200 0x7f64980706b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6494006010 tx=0x7f649401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:10:44.854 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:44.853+0000 7f64a97fa700 1 -- 172.21.15.144:0/3534488359 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f649c056f10 con 0x7f64ac1003d0 2024-09-10T12:10:45.139 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:45.138+0000 7f64b1e87700 1 -- 172.21.15.144:0/3534488359 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f64ac19b300 con 0x7f64ac1003d0 2024-09-10T12:10:45.140 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:45.139+0000 7f64a97fa700 1 -- 172.21.15.144:0/3534488359 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f649c05a5d0 con 0x7f64ac1003d0 2024-09-10T12:10:45.140 INFO:teuthology.orchestra.run.smithi144.stdout:{ 2024-09-10T12:10:45.140 INFO:teuthology.orchestra.run.smithi144.stdout: "mon": { 2024-09-10T12:10:45.140 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:10:45.140 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:10:45.141 INFO:teuthology.orchestra.run.smithi144.stdout: "mgr": { 2024-09-10T12:10:45.141 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 2 2024-09-10T12:10:45.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:10:45.141 INFO:teuthology.orchestra.run.smithi144.stdout: "osd": { 2024-09-10T12:10:45.141 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 6 2024-09-10T12:10:45.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:10:45.141 INFO:teuthology.orchestra.run.smithi144.stdout: "mds": { 2024-09-10T12:10:45.141 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 4 2024-09-10T12:10:45.141 INFO:teuthology.orchestra.run.smithi144.stdout: }, 2024-09-10T12:10:45.141 INFO:teuthology.orchestra.run.smithi144.stdout: "overall": { 2024-09-10T12:10:45.141 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)": 14 2024-09-10T12:10:45.141 INFO:teuthology.orchestra.run.smithi144.stdout: } 2024-09-10T12:10:45.141 INFO:teuthology.orchestra.run.smithi144.stdout:} 2024-09-10T12:10:45.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:45.141+0000 7f64b1e87700 1 -- 172.21.15.144:0/3534488359 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f649806e200 msgr2=0x7f64980706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:45.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:45.141+0000 7f64b1e87700 1 --2- 172.21.15.144:0/3534488359 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f649806e200 0x7f64980706b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6494006010 tx=0x7f649401a040 comp rx=0 tx=0).stop 2024-09-10T12:10:45.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:45.141+0000 7f64b1e87700 1 -- 172.21.15.144:0/3534488359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64ac1003d0 msgr2=0x7f64ac195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:45.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:45.141+0000 7f64b1e87700 1 --2- 172.21.15.144:0/3534488359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64ac1003d0 0x7f64ac195e80 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f649c00b4b0 tx=0x7f649c00b7c0 comp rx=0 tx=0).stop 2024-09-10T12:10:45.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:45.141+0000 7f64b1e87700 1 -- 172.21.15.144:0/3534488359 shutdown_connections 2024-09-10T12:10:45.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:45.141+0000 7f64b1e87700 1 --2- 172.21.15.144:0/3534488359 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f649806e200 0x7f64980706b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:45.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:45.141+0000 7f64b1e87700 1 --2- 172.21.15.144:0/3534488359 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f64ac1003d0 0x7f64ac195e80 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:45.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:45.141+0000 7f64b1e87700 1 --2- 172.21.15.144:0/3534488359 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f64ac100d70 0x7f64ac1963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:45.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:45.141+0000 7f64b1e87700 1 -- 172.21.15.144:0/3534488359 >> 172.21.15.144:0/3534488359 conn(0x7f64ac0fbc90 msgr2=0x7f64ac0fd1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:10:45.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:45.141+0000 7f64b1e87700 1 -- 172.21.15.144:0/3534488359 shutdown_connections 2024-09-10T12:10:45.143 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:45.141+0000 7f64b1e87700 1 -- 172.21.15.144:0/3534488359 wait complete. 2024-09-10T12:10:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:44 smithi178 ceph-mon[65940]: pgmap v1165: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:10:45.424 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:44 smithi144 ceph-mon[87436]: pgmap v1165: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:10:45.811 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-09-10T12:10:46.012 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:10:46.132 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:45 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3534488359' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:10:46.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:45 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3534488359' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:10:47.072 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:46 smithi144 ceph-mon[87436]: pgmap v1166: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:46 smithi178 ceph-mon[65940]: pgmap v1166: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:47.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.576+0000 7f69b7192700 1 -- 172.21.15.144:0/3394317735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b00ffc70 msgr2=0x7f69b0103c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:47.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.576+0000 7f69b7192700 1 --2- 172.21.15.144:0/3394317735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b00ffc70 0x7f69b0103c40 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f699c009910 tx=0x7f699c009c20 comp rx=0 tx=0).stop 2024-09-10T12:10:47.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.576+0000 7f69b7192700 1 -- 172.21.15.144:0/3394317735 shutdown_connections 2024-09-10T12:10:47.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.576+0000 7f69b7192700 1 --2- 172.21.15.144:0/3394317735 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b00ffc70 0x7f69b0103c40 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:47.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.576+0000 7f69b7192700 1 --2- 172.21.15.144:0/3394317735 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69b00ff2d0 0x7f69b00ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:47.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.576+0000 7f69b7192700 1 -- 172.21.15.144:0/3394317735 >> 172.21.15.144:0/3394317735 conn(0x7f69b00fabb0 msgr2=0x7f69b00fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:10:47.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.576+0000 7f69b7192700 1 -- 172.21.15.144:0/3394317735 shutdown_connections 2024-09-10T12:10:47.578 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.576+0000 7f69b7192700 1 -- 172.21.15.144:0/3394317735 wait complete. 2024-09-10T12:10:47.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.577+0000 7f69b7192700 1 Processor -- start 2024-09-10T12:10:47.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.577+0000 7f69b7192700 1 -- start start 2024-09-10T12:10:47.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.577+0000 7f69b7192700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69b00ff2d0 0x7f69b01949c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:47.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.577+0000 7f69b7192700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b00ffc70 0x7f69b0194f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:47.580 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.577+0000 7f69b7192700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b0195440 con 0x7f69b00ffc70 2024-09-10T12:10:47.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.577+0000 7f69b7192700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b01955b0 con 0x7f69b00ff2d0 2024-09-10T12:10:47.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.577+0000 7f69affff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b00ffc70 0x7f69b0194f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:47.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.577+0000 7f69affff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b00ffc70 0x7f69b0194f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:49794/0 (socket says 172.21.15.144:49794) 2024-09-10T12:10:47.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.577+0000 7f69affff700 1 -- 172.21.15.144:0/2256588045 learned_addr learned my addr 172.21.15.144:0/2256588045 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:10:47.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.577+0000 7f69b4f2e700 1 --2- 172.21.15.144:0/2256588045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69b00ff2d0 0x7f69b01949c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:47.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.578+0000 7f69affff700 1 -- 172.21.15.144:0/2256588045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69b00ff2d0 msgr2=0x7f69b01949c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:47.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.578+0000 7f69affff700 1 --2- 172.21.15.144:0/2256588045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69b00ff2d0 0x7f69b01949c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:47.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.578+0000 7f69affff700 1 -- 172.21.15.144:0/2256588045 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f699c0095f0 con 0x7f69b00ffc70 2024-09-10T12:10:47.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.578+0000 7f69b4f2e700 1 --2- 172.21.15.144:0/2256588045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69b00ff2d0 0x7f69b01949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:10:47.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.578+0000 7f69affff700 1 --2- 172.21.15.144:0/2256588045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b00ffc70 0x7f69b0194f00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f699c0092e0 tx=0x7f699c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:10:47.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.578+0000 7f69adffb700 1 -- 172.21.15.144:0/2256588045 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f699c01d070 con 0x7f69b00ffc70 2024-09-10T12:10:47.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.578+0000 7f69adffb700 1 -- 172.21.15.144:0/2256588045 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f699c005dc0 con 0x7f69b00ffc70 2024-09-10T12:10:47.581 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.578+0000 7f69b7192700 1 -- 172.21.15.144:0/2256588045 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b0195830 con 0x7f69b00ffc70 2024-09-10T12:10:47.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.578+0000 7f69adffb700 1 -- 172.21.15.144:0/2256588045 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f699c017400 con 0x7f69b00ffc70 2024-09-10T12:10:47.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.578+0000 7f69b7192700 1 -- 172.21.15.144:0/2256588045 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b01a22b0 con 0x7f69b00ffc70 2024-09-10T12:10:47.582 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.579+0000 7f69b7192700 1 -- 172.21.15.144:0/2256588045 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b004ea50 con 0x7f69b00ffc70 2024-09-10T12:10:47.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.580+0000 7f69adffb700 1 -- 172.21.15.144:0/2256588045 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f699c00f460 con 0x7f69b00ffc70 2024-09-10T12:10:47.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.580+0000 7f69adffb700 1 --2- 172.21.15.144:0/2256588045 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f69a006e200 0x7f69a00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:47.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.580+0000 7f69adffb700 1 -- 172.21.15.144:0/2256588045 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f699c08f380 con 0x7f69b00ffc70 2024-09-10T12:10:47.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.580+0000 7f69b4f2e700 1 --2- 172.21.15.144:0/2256588045 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f69a006e200 0x7f69a00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:47.583 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.581+0000 7f69b4f2e700 1 --2- 172.21.15.144:0/2256588045 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f69a006e200 0x7f69a00706b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f69a4005fd0 tx=0x7f69a4005ee0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:10:47.586 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.584+0000 7f69adffb700 1 -- 172.21.15.144:0/2256588045 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f699c05c310 con 0x7f69b00ffc70 2024-09-10T12:10:47.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.873+0000 7f69b7192700 1 -- 172.21.15.144:0/2256588045 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f69b0195830 con 0x7f69b00ffc70 2024-09-10T12:10:47.875 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.874+0000 7f69adffb700 1 -- 172.21.15.144:0/2256588045 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f69b0195830 con 0x7f69b00ffc70 2024-09-10T12:10:47.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.876+0000 7f69b7192700 1 -- 172.21.15.144:0/2256588045 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f69a006e200 msgr2=0x7f69a00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:47.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.876+0000 7f69b7192700 1 --2- 172.21.15.144:0/2256588045 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f69a006e200 0x7f69a00706b0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f69a4005fd0 tx=0x7f69a4005ee0 comp rx=0 tx=0).stop 2024-09-10T12:10:47.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.876+0000 7f69b7192700 1 -- 172.21.15.144:0/2256588045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b00ffc70 msgr2=0x7f69b0194f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:47.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.876+0000 7f69b7192700 1 --2- 172.21.15.144:0/2256588045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b00ffc70 0x7f69b0194f00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f699c0092e0 tx=0x7f699c00be20 comp rx=0 tx=0).stop 2024-09-10T12:10:47.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.876+0000 7f69b7192700 1 -- 172.21.15.144:0/2256588045 shutdown_connections 2024-09-10T12:10:47.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.876+0000 7f69b7192700 1 --2- 172.21.15.144:0/2256588045 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f69a006e200 0x7f69a00706b0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:47.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.876+0000 7f69b7192700 1 --2- 172.21.15.144:0/2256588045 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f69b00ff2d0 0x7f69b01949c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:47.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.876+0000 7f69b7192700 1 --2- 172.21.15.144:0/2256588045 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69b00ffc70 0x7f69b0194f00 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:47.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.876+0000 7f69b7192700 1 -- 172.21.15.144:0/2256588045 >> 172.21.15.144:0/2256588045 conn(0x7f69b00fabb0 msgr2=0x7f69b00fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:10:47.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.876+0000 7f69b7192700 1 -- 172.21.15.144:0/2256588045 shutdown_connections 2024-09-10T12:10:47.881 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:47.876+0000 7f69b7192700 1 -- 172.21.15.144:0/2256588045 wait complete. 2024-09-10T12:10:47.888 INFO:teuthology.orchestra.run.smithi144.stdout:true 2024-09-10T12:10:48.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:48 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2256588045' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:10:48.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:48 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2256588045' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:10:48.601 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -e sha1=1cf819b7212a6048e92fc78c4eff963344f6d56b -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-09-10T12:10:48.801 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:10:49.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:49 smithi144 ceph-mon[87436]: pgmap v1167: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:10:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:49 smithi178 ceph-mon[65940]: pgmap v1167: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:10:50.392 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.388+0000 7f69a0433700 1 -- 172.21.15.144:0/1587958614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69981003d0 msgr2=0x7f69981007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:50.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.388+0000 7f69a0433700 1 --2- 172.21.15.144:0/1587958614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69981003d0 0x7f69981007a0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f6988009910 tx=0x7f6988009c20 comp rx=0 tx=0).stop 2024-09-10T12:10:50.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.388+0000 7f69a0433700 1 -- 172.21.15.144:0/1587958614 shutdown_connections 2024-09-10T12:10:50.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.388+0000 7f69a0433700 1 --2- 172.21.15.144:0/1587958614 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6998100d70 0x7f6998104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:50.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.388+0000 7f69a0433700 1 --2- 172.21.15.144:0/1587958614 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69981003d0 0x7f69981007a0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:50.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.388+0000 7f69a0433700 1 -- 172.21.15.144:0/1587958614 >> 172.21.15.144:0/1587958614 conn(0x7f69980fbc90 msgr2=0x7f69980fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:10:50.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.388+0000 7f69a0433700 1 -- 172.21.15.144:0/1587958614 shutdown_connections 2024-09-10T12:10:50.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.388+0000 7f69a0433700 1 -- 172.21.15.144:0/1587958614 wait complete. 2024-09-10T12:10:50.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.389+0000 7f69a0433700 1 Processor -- start 2024-09-10T12:10:50.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.389+0000 7f69a0433700 1 -- start start 2024-09-10T12:10:50.393 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.389+0000 7f69a0433700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69981003d0 0x7f6998195d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:50.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.389+0000 7f69a0433700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6998100d70 0x7f69981962a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:50.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.389+0000 7f69a0433700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6998196980 con 0x7f69981003d0 2024-09-10T12:10:50.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.389+0000 7f69a0433700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699819a710 con 0x7f6998100d70 2024-09-10T12:10:50.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.389+0000 7f699e1cf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69981003d0 0x7f6998195d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:50.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.390+0000 7f699e1cf700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69981003d0 0x7f6998195d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:49814/0 (socket says 172.21.15.144:49814) 2024-09-10T12:10:50.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.390+0000 7f699e1cf700 1 -- 172.21.15.144:0/780396644 learned_addr learned my addr 172.21.15.144:0/780396644 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:10:50.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.390+0000 7f699d9ce700 1 --2- 172.21.15.144:0/780396644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6998100d70 0x7f69981962a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:50.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.390+0000 7f699e1cf700 1 -- 172.21.15.144:0/780396644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6998100d70 msgr2=0x7f69981962a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:50.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.390+0000 7f699e1cf700 1 --2- 172.21.15.144:0/780396644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6998100d70 0x7f69981962a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:50.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.390+0000 7f699e1cf700 1 -- 172.21.15.144:0/780396644 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69880095f0 con 0x7f69981003d0 2024-09-10T12:10:50.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.390+0000 7f699e1cf700 1 --2- 172.21.15.144:0/780396644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69981003d0 0x7f6998195d60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6988006010 tx=0x7f698800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:10:50.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.390+0000 7f698f7fe700 1 -- 172.21.15.144:0/780396644 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f698801d070 con 0x7f69981003d0 2024-09-10T12:10:50.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.390+0000 7f69a0433700 1 -- 172.21.15.144:0/780396644 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699819a990 con 0x7f69981003d0 2024-09-10T12:10:50.395 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.390+0000 7f698f7fe700 1 -- 172.21.15.144:0/780396644 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f698800f460 con 0x7f69981003d0 2024-09-10T12:10:50.395 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.390+0000 7f698f7fe700 1 -- 172.21.15.144:0/780396644 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6988017610 con 0x7f69981003d0 2024-09-10T12:10:50.395 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.390+0000 7f69a0433700 1 -- 172.21.15.144:0/780396644 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699819ae80 con 0x7f69981003d0 2024-09-10T12:10:50.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.391+0000 7f69a0433700 1 -- 172.21.15.144:0/780396644 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69981086e0 con 0x7f69981003d0 2024-09-10T12:10:50.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.392+0000 7f698f7fe700 1 -- 172.21.15.144:0/780396644 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f698800f5d0 con 0x7f69981003d0 2024-09-10T12:10:50.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.392+0000 7f698f7fe700 1 --2- 172.21.15.144:0/780396644 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f698406e200 0x7f69840706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:50.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.392+0000 7f698f7fe700 1 -- 172.21.15.144:0/780396644 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f698808f620 con 0x7f69981003d0 2024-09-10T12:10:50.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.392+0000 7f699d9ce700 1 --2- 172.21.15.144:0/780396644 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f698406e200 0x7f69840706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:50.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.393+0000 7f699d9ce700 1 --2- 172.21.15.144:0/780396644 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f698406e200 0x7f69840706b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6998197380 tx=0x7f699400a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:10:50.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.397+0000 7f698f7fe700 1 -- 172.21.15.144:0/780396644 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f69880589e0 con 0x7f69981003d0 2024-09-10T12:10:50.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.702+0000 7f69a0433700 1 -- 172.21.15.144:0/780396644 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f699802cc30 con 0x7f69981003d0 2024-09-10T12:10:50.704 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.702+0000 7f698f7fe700 1 -- 172.21.15.144:0/780396644 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+633 (secure 0 0 0) 0x7f698805c0a0 con 0x7f69981003d0 2024-09-10T12:10:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.704+0000 7f69a0433700 1 -- 172.21.15.144:0/780396644 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f698406e200 msgr2=0x7f69840706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.704+0000 7f69a0433700 1 --2- 172.21.15.144:0/780396644 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f698406e200 0x7f69840706b0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f6998197380 tx=0x7f699400a300 comp rx=0 tx=0).stop 2024-09-10T12:10:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.704+0000 7f69a0433700 1 -- 172.21.15.144:0/780396644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69981003d0 msgr2=0x7f6998195d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.704+0000 7f69a0433700 1 --2- 172.21.15.144:0/780396644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69981003d0 0x7f6998195d60 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f6988006010 tx=0x7f698800bd70 comp rx=0 tx=0).stop 2024-09-10T12:10:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.705+0000 7f69a0433700 1 -- 172.21.15.144:0/780396644 shutdown_connections 2024-09-10T12:10:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.705+0000 7f69a0433700 1 --2- 172.21.15.144:0/780396644 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f698406e200 0x7f69840706b0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:50.707 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.705+0000 7f69a0433700 1 --2- 172.21.15.144:0/780396644 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f69981003d0 0x7f6998195d60 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:50.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.705+0000 7f69a0433700 1 --2- 172.21.15.144:0/780396644 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6998100d70 0x7f69981962a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:50.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.705+0000 7f69a0433700 1 -- 172.21.15.144:0/780396644 >> 172.21.15.144:0/780396644 conn(0x7f69980fbc90 msgr2=0x7f69980fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:10:50.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.705+0000 7f69a0433700 1 -- 172.21.15.144:0/780396644 shutdown_connections 2024-09-10T12:10:50.708 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:50.705+0000 7f69a0433700 1 -- 172.21.15.144:0/780396644 wait complete. 2024-09-10T12:10:50.718 INFO:teuthology.orchestra.run.smithi144.stdout: "ceph version 19.1.1-305-g1cf819b7 (1cf819b7212a6048e92fc78c4eff963344f6d56b) squid (stable)" 2024-09-10T12:10:51.310 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:51 smithi144 ceph-mon[87436]: pgmap v1168: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:10:51.310 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:51 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/780396644' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:10:51.332 DEBUG:teuthology.parallel:result is None 2024-09-10T12:10:51.332 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-10T12:10:51.342 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T12:10:51.342 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-10T12:10:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:51 smithi178 ceph-mon[65940]: pgmap v1168: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:10:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:51 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/780396644' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-09-10T12:10:51.541 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:10:53.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.181+0000 7fdc5031f700 1 -- 172.21.15.144:0/3737509754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc48068630 msgr2=0x7fdc48068a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:53.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.181+0000 7fdc5031f700 1 --2- 172.21.15.144:0/3737509754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc48068630 0x7fdc48068a00 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009910 tx=0x7fdc44009c20 comp rx=0 tx=0).stop 2024-09-10T12:10:53.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.181+0000 7fdc5031f700 1 -- 172.21.15.144:0/3737509754 shutdown_connections 2024-09-10T12:10:53.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.181+0000 7fdc5031f700 1 --2- 172.21.15.144:0/3737509754 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc48068fd0 0x7fdc48104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:53.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.181+0000 7fdc5031f700 1 --2- 172.21.15.144:0/3737509754 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc48068630 0x7fdc48068a00 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:53.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.181+0000 7fdc5031f700 1 -- 172.21.15.144:0/3737509754 >> 172.21.15.144:0/3737509754 conn(0x7fdc48074f50 msgr2=0x7fdc48075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:10:53.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.181+0000 7fdc5031f700 1 -- 172.21.15.144:0/3737509754 shutdown_connections 2024-09-10T12:10:53.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.181+0000 7fdc5031f700 1 -- 172.21.15.144:0/3737509754 wait complete. 2024-09-10T12:10:53.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.182+0000 7fdc5031f700 1 Processor -- start 2024-09-10T12:10:53.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.182+0000 7fdc5031f700 1 -- start start 2024-09-10T12:10:53.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.182+0000 7fdc5031f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc48068630 0x7fdc48195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:53.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.182+0000 7fdc5031f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc48068fd0 0x7fdc48196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:53.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.182+0000 7fdc5031f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc481968c0 con 0x7fdc48068630 2024-09-10T12:10:53.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.182+0000 7fdc5031f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc48196a00 con 0x7fdc48068fd0 2024-09-10T12:10:53.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.182+0000 7fdc4e0bb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc48068630 0x7fdc48195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:53.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.182+0000 7fdc4e0bb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc48068630 0x7fdc48195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:48778/0 (socket says 172.21.15.144:48778) 2024-09-10T12:10:53.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.182+0000 7fdc4e0bb700 1 -- 172.21.15.144:0/3460526911 learned_addr learned my addr 172.21.15.144:0/3460526911 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:10:53.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.182+0000 7fdc4d8ba700 1 --2- 172.21.15.144:0/3460526911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc48068fd0 0x7fdc48196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:53.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.182+0000 7fdc4e0bb700 1 -- 172.21.15.144:0/3460526911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc48068fd0 msgr2=0x7fdc48196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:53.186 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.182+0000 7fdc4e0bb700 1 --2- 172.21.15.144:0/3460526911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc48068fd0 0x7fdc48196270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:53.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.182+0000 7fdc4e0bb700 1 -- 172.21.15.144:0/3460526911 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc440095f0 con 0x7fdc48068630 2024-09-10T12:10:53.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.182+0000 7fdc4e0bb700 1 --2- 172.21.15.144:0/3460526911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc48068630 0x7fdc48195d30 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fdc44006010 tx=0x7fdc4400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:10:53.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.183+0000 7fdc3f7fe700 1 -- 172.21.15.144:0/3460526911 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc4401d070 con 0x7fdc48068630 2024-09-10T12:10:53.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.183+0000 7fdc3f7fe700 1 -- 172.21.15.144:0/3460526911 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdc44004fb0 con 0x7fdc48068630 2024-09-10T12:10:53.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.183+0000 7fdc5031f700 1 -- 172.21.15.144:0/3460526911 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc4819a7f0 con 0x7fdc48068630 2024-09-10T12:10:53.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.183+0000 7fdc3f7fe700 1 -- 172.21.15.144:0/3460526911 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc44022ea0 con 0x7fdc48068630 2024-09-10T12:10:53.187 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.183+0000 7fdc5031f700 1 -- 172.21.15.144:0/3460526911 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc4819ace0 con 0x7fdc48068630 2024-09-10T12:10:53.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.184+0000 7fdc5031f700 1 -- 172.21.15.144:0/3460526911 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc481086d0 con 0x7fdc48068630 2024-09-10T12:10:53.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.184+0000 7fdc3f7fe700 1 -- 172.21.15.144:0/3460526911 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fdc44022470 con 0x7fdc48068630 2024-09-10T12:10:53.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.185+0000 7fdc3f7fe700 1 --2- 172.21.15.144:0/3460526911 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdc3406e200 0x7fdc340706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:53.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.185+0000 7fdc3f7fe700 1 -- 172.21.15.144:0/3460526911 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fdc4408f540 con 0x7fdc48068630 2024-09-10T12:10:53.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.185+0000 7fdc4d8ba700 1 --2- 172.21.15.144:0/3460526911 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdc3406e200 0x7fdc340706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:53.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.185+0000 7fdc4d8ba700 1 --2- 172.21.15.144:0/3460526911 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdc3406e200 0x7fdc340706b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdc48197350 tx=0x7fdc38005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:10:53.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.189+0000 7fdc3f7fe700 1 -- 172.21.15.144:0/3460526911 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fdc44058a50 con 0x7fdc48068630 2024-09-10T12:10:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:53 smithi178 ceph-mon[65940]: pgmap v1169: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:53 smithi144 ceph-mon[87436]: pgmap v1169: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:53.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.459+0000 7fdc5031f700 1 -- 172.21.15.144:0/3460526911 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdc4802cd20 con 0x7fdc48068630 2024-09-10T12:10:53.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.460+0000 7fdc3f7fe700 1 -- 172.21.15.144:0/3460526911 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 42 v42) v1 ==== 76+0+2010 (secure 0 0 0) 0x7fdc440277f0 con 0x7fdc48068630 2024-09-10T12:10:53.463 INFO:teuthology.orchestra.run.smithi144.stdout:e42 2024-09-10T12:10:53.463 INFO:teuthology.orchestra.run.smithi144.stdout:btime 2024-09-10T12:08:52:341881+0000 2024-09-10T12:10:53.463 INFO:teuthology.orchestra.run.smithi144.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T12:10:53.463 INFO:teuthology.orchestra.run.smithi144.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T12:10:53.463 INFO:teuthology.orchestra.run.smithi144.stdout:legacy client fscid: 1 2024-09-10T12:10:53.463 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:10:53.463 INFO:teuthology.orchestra.run.smithi144.stdout:Filesystem 'cephfs' (1) 2024-09-10T12:10:53.463 INFO:teuthology.orchestra.run.smithi144.stdout:fs_name cephfs 2024-09-10T12:10:53.463 INFO:teuthology.orchestra.run.smithi144.stdout:epoch 42 2024-09-10T12:10:53.463 INFO:teuthology.orchestra.run.smithi144.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-10T12:10:53.463 INFO:teuthology.orchestra.run.smithi144.stdout:created 2024-09-10T11:26:34.004497+0000 2024-09-10T12:10:53.463 INFO:teuthology.orchestra.run.smithi144.stdout:modified 2024-09-10T12:08:51.469973+0000 2024-09-10T12:10:53.463 INFO:teuthology.orchestra.run.smithi144.stdout:tableserver 0 2024-09-10T12:10:53.464 INFO:teuthology.orchestra.run.smithi144.stdout:root 0 2024-09-10T12:10:53.464 INFO:teuthology.orchestra.run.smithi144.stdout:session_timeout 60 2024-09-10T12:10:53.464 INFO:teuthology.orchestra.run.smithi144.stdout:session_autoclose 300 2024-09-10T12:10:53.464 INFO:teuthology.orchestra.run.smithi144.stdout:max_file_size 1099511627776 2024-09-10T12:10:53.464 INFO:teuthology.orchestra.run.smithi144.stdout:max_xattr_size 65536 2024-09-10T12:10:53.464 INFO:teuthology.orchestra.run.smithi144.stdout:required_client_features {} 2024-09-10T12:10:53.464 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure 0 2024-09-10T12:10:53.464 INFO:teuthology.orchestra.run.smithi144.stdout:last_failure_osd_epoch 80 2024-09-10T12:10:53.464 INFO:teuthology.orchestra.run.smithi144.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-10T12:10:53.464 INFO:teuthology.orchestra.run.smithi144.stdout:max_mds 2 2024-09-10T12:10:53.464 INFO:teuthology.orchestra.run.smithi144.stdout:in 0,1 2024-09-10T12:10:53.464 INFO:teuthology.orchestra.run.smithi144.stdout:up {0=34390,1=34422} 2024-09-10T12:10:53.465 INFO:teuthology.orchestra.run.smithi144.stdout:failed 2024-09-10T12:10:53.465 INFO:teuthology.orchestra.run.smithi144.stdout:damaged 2024-09-10T12:10:53.466 INFO:teuthology.orchestra.run.smithi144.stdout:stopped 2024-09-10T12:10:53.466 INFO:teuthology.orchestra.run.smithi144.stdout:data_pools [3] 2024-09-10T12:10:53.466 INFO:teuthology.orchestra.run.smithi144.stdout:metadata_pool 2 2024-09-10T12:10:53.466 INFO:teuthology.orchestra.run.smithi144.stdout:inline_data enabled 2024-09-10T12:10:53.466 INFO:teuthology.orchestra.run.smithi144.stdout:balancer 2024-09-10T12:10:53.466 INFO:teuthology.orchestra.run.smithi144.stdout:bal_rank_mask -1 2024-09-10T12:10:53.466 INFO:teuthology.orchestra.run.smithi144.stdout:standby_count_wanted 1 2024-09-10T12:10:53.466 INFO:teuthology.orchestra.run.smithi144.stdout:qdb_cluster leader: 34390 members: 34422,34390 2024-09-10T12:10:53.467 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.bfyvwv{0:34390} state up:active seq 466 join_fscid=1 addr [v2:172.21.15.144:6826/2724506882,v1:172.21.15.144:6827/2724506882] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:10:53.467 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.nmhrmh{0:44355} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.178:6824/706731549,v1:172.21.15.178:6825/706731549] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:10:53.467 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi144.jaoyvs{1:34422} state up:active seq 462 join_fscid=1 addr [v2:172.21.15.144:6828/4254390011,v1:172.21.15.144:6829/4254390011] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:10:53.467 INFO:teuthology.orchestra.run.smithi144.stdout:[mds.cephfs.smithi178.czjrsa{1:44887} state up:standby-replay seq 1 join_fscid=1 addr [v2:172.21.15.178:6826/2967786976,v1:172.21.15.178:6827/2967786976] compat {c=[1],r=[1],i=[1fff]}] 2024-09-10T12:10:53.467 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:10:53.468 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:10:53.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.462+0000 7fdc5031f700 1 -- 172.21.15.144:0/3460526911 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdc3406e200 msgr2=0x7fdc340706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:53.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.462+0000 7fdc5031f700 1 --2- 172.21.15.144:0/3460526911 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdc3406e200 0x7fdc340706b0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fdc48197350 tx=0x7fdc38005d50 comp rx=0 tx=0).stop 2024-09-10T12:10:53.469 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.462+0000 7fdc5031f700 1 -- 172.21.15.144:0/3460526911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc48068630 msgr2=0x7fdc48195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.462+0000 7fdc5031f700 1 --2- 172.21.15.144:0/3460526911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc48068630 0x7fdc48195d30 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fdc44006010 tx=0x7fdc4400be20 comp rx=0 tx=0).stop 2024-09-10T12:10:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.462+0000 7fdc5031f700 1 -- 172.21.15.144:0/3460526911 shutdown_connections 2024-09-10T12:10:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.462+0000 7fdc5031f700 1 --2- 172.21.15.144:0/3460526911 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fdc3406e200 0x7fdc340706b0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.462+0000 7fdc5031f700 1 --2- 172.21.15.144:0/3460526911 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fdc48068630 0x7fdc48195d30 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.462+0000 7fdc5031f700 1 --2- 172.21.15.144:0/3460526911 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fdc48068fd0 0x7fdc48196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.462+0000 7fdc5031f700 1 -- 172.21.15.144:0/3460526911 >> 172.21.15.144:0/3460526911 conn(0x7fdc48074f50 msgr2=0x7fdc480fd710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:10:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.463+0000 7fdc5031f700 1 -- 172.21.15.144:0/3460526911 shutdown_connections 2024-09-10T12:10:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:53.463+0000 7fdc5031f700 1 -- 172.21.15.144:0/3460526911 wait complete. 2024-09-10T12:10:53.470 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 42 2024-09-10T12:10:54.122 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-09-10T12:10:54.134 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 2024-09-10T12:10:54.332 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:10:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:10:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:54 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3460526911' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:10:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:10:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:10:54.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:10:54.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:10:54.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:54 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3460526911' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-10T12:10:54.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:10:54.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:10:54.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:10:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:55 smithi178 ceph-mon[65940]: pgmap v1170: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:10:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:55 smithi144 ceph-mon[87436]: pgmap v1170: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:10:56.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.061+0000 7f08de70f700 1 -- 172.21.15.144:0/1177752251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d80ffd30 msgr2=0x7f08d8103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:56.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.061+0000 7f08de70f700 1 --2- 172.21.15.144:0/1177752251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d80ffd30 0x7f08d8103ce0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f08c0009910 tx=0x7f08c0009c20 comp rx=0 tx=0).stop 2024-09-10T12:10:56.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.061+0000 7f08de70f700 1 -- 172.21.15.144:0/1177752251 shutdown_connections 2024-09-10T12:10:56.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.061+0000 7f08de70f700 1 --2- 172.21.15.144:0/1177752251 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d80ffd30 0x7f08d8103ce0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:56.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.061+0000 7f08de70f700 1 --2- 172.21.15.144:0/1177752251 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08d80ff390 0x7f08d80ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:56.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.061+0000 7f08de70f700 1 -- 172.21.15.144:0/1177752251 >> 172.21.15.144:0/1177752251 conn(0x7f08d8074f50 msgr2=0x7f08d8075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:10:56.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.061+0000 7f08de70f700 1 -- 172.21.15.144:0/1177752251 shutdown_connections 2024-09-10T12:10:56.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.061+0000 7f08de70f700 1 -- 172.21.15.144:0/1177752251 wait complete. 2024-09-10T12:10:56.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.062+0000 7f08de70f700 1 Processor -- start 2024-09-10T12:10:56.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.062+0000 7f08de70f700 1 -- start start 2024-09-10T12:10:56.066 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.062+0000 7f08de70f700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08d80ff390 0x7f08d8100d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:56.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.062+0000 7f08de70f700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d80ffd30 0x7f08d81012c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:56.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.062+0000 7f08de70f700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d8101800 con 0x7f08d80ffd30 2024-09-10T12:10:56.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.062+0000 7f08de70f700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d8101940 con 0x7f08d80ff390 2024-09-10T12:10:56.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.062+0000 7f08cffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d80ffd30 0x7f08d81012c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:56.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.062+0000 7f08cffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d80ffd30 0x7f08d81012c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:48800/0 (socket says 172.21.15.144:48800) 2024-09-10T12:10:56.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.062+0000 7f08cffff700 1 -- 172.21.15.144:0/3278622185 learned_addr learned my addr 172.21.15.144:0/3278622185 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:10:56.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.063+0000 7f08d7fff700 1 --2- 172.21.15.144:0/3278622185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08d80ff390 0x7f08d8100d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:56.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.063+0000 7f08cffff700 1 -- 172.21.15.144:0/3278622185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08d80ff390 msgr2=0x7f08d8100d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:56.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.063+0000 7f08cffff700 1 --2- 172.21.15.144:0/3278622185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08d80ff390 0x7f08d8100d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:56.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.063+0000 7f08cffff700 1 -- 172.21.15.144:0/3278622185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c00095f0 con 0x7f08d80ffd30 2024-09-10T12:10:56.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.063+0000 7f08cffff700 1 --2- 172.21.15.144:0/3278622185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d80ffd30 0x7f08d81012c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f08c00059d0 tx=0x7f08c000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:10:56.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.063+0000 7f08d5ffb700 1 -- 172.21.15.144:0/3278622185 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08c001d070 con 0x7f08d80ffd30 2024-09-10T12:10:56.067 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.063+0000 7f08de70f700 1 -- 172.21.15.144:0/3278622185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d8101bc0 con 0x7f08d80ffd30 2024-09-10T12:10:56.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.063+0000 7f08d5ffb700 1 -- 172.21.15.144:0/3278622185 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08c000b610 con 0x7f08d80ffd30 2024-09-10T12:10:56.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.063+0000 7f08d5ffb700 1 -- 172.21.15.144:0/3278622185 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08c0022470 con 0x7f08d80ffd30 2024-09-10T12:10:56.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.063+0000 7f08de70f700 1 -- 172.21.15.144:0/3278622185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d81a2290 con 0x7f08d80ffd30 2024-09-10T12:10:56.068 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.064+0000 7f08de70f700 1 -- 172.21.15.144:0/3278622185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d804ea50 con 0x7f08d80ffd30 2024-09-10T12:10:56.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.065+0000 7f08d5ffb700 1 -- 172.21.15.144:0/3278622185 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f08c000fc70 con 0x7f08d80ffd30 2024-09-10T12:10:56.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.065+0000 7f08d5ffb700 1 --2- 172.21.15.144:0/3278622185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f08c406e200 0x7f08c40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:56.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.065+0000 7f08d5ffb700 1 -- 172.21.15.144:0/3278622185 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f08c008f4f0 con 0x7f08d80ffd30 2024-09-10T12:10:56.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.065+0000 7f08d7fff700 1 --2- 172.21.15.144:0/3278622185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f08c406e200 0x7f08c40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:56.069 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.066+0000 7f08d7fff700 1 --2- 172.21.15.144:0/3278622185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f08c406e200 0x7f08c40706b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009b30 tx=0x7f08c8009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:10:56.072 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.069+0000 7f08d5ffb700 1 -- 172.21.15.144:0/3278622185 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f08c00588d0 con 0x7f08d80ffd30 2024-09-10T12:10:56.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.318+0000 7f08de70f700 1 -- 172.21.15.144:0/3278622185 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08d81022b0 con 0x7f08d80ffd30 2024-09-10T12:10:56.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.319+0000 7f08d5ffb700 1 -- 172.21.15.144:0/3278622185 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+5299 (secure 0 0 0) 0x7f08c005bf90 con 0x7f08d80ffd30 2024-09-10T12:10:56.322 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:10:56.322 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":42,"btime":"2024-09-10T12:08:52:341881+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T12:08:51.469973+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34390,"mds_1":34422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34390":{"gid":34390,"name":"cephfs.smithi144.bfyvwv","rank":0,"incarnation":36,"state":"up:active","state_seq":466,"addr":"172.21.15.144:6827/2724506882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2724506882},{"type":"v1","addr":"172.21.15.144:6827","nonce":2724506882}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34422":{"gid":34422,"name":"cephfs.smithi144.jaoyvs","rank":1,"incarnation":36,"state":"up:active","state_seq":462,"addr":"172.21.15.144:6829/4254390011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4254390011},{"type":"v1","addr":"172.21.15.144:6829","nonce":4254390011}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44355":{"gid":44355,"name":"cephfs.smithi178.nmhrmh","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6825/706731549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":706731549},{"type":"v1","addr":"172.21.15.178:6825","nonce":706731549}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44887":{"gid":44887,"name":"cephfs.smithi178.czjrsa","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6827/2967786976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2967786976},{"type":"v1","addr":"172.21.15.178:6827","nonce":2967786976}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34390,"qdb_cluster":[34422,34390]},"id":1}]} 2024-09-10T12:10:56.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.321+0000 7f08de70f700 1 -- 172.21.15.144:0/3278622185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f08c406e200 msgr2=0x7f08c40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:56.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.321+0000 7f08de70f700 1 --2- 172.21.15.144:0/3278622185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f08c406e200 0x7f08c40706b0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f08c8009b30 tx=0x7f08c8009260 comp rx=0 tx=0).stop 2024-09-10T12:10:56.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.321+0000 7f08de70f700 1 -- 172.21.15.144:0/3278622185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d80ffd30 msgr2=0x7f08d81012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:56.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.321+0000 7f08de70f700 1 --2- 172.21.15.144:0/3278622185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d80ffd30 0x7f08d81012c0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f08c00059d0 tx=0x7f08c000f740 comp rx=0 tx=0).stop 2024-09-10T12:10:56.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.321+0000 7f08de70f700 1 -- 172.21.15.144:0/3278622185 shutdown_connections 2024-09-10T12:10:56.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.321+0000 7f08de70f700 1 --2- 172.21.15.144:0/3278622185 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f08c406e200 0x7f08c40706b0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:56.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.321+0000 7f08de70f700 1 --2- 172.21.15.144:0/3278622185 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f08d80ff390 0x7f08d8100d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:56.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.321+0000 7f08de70f700 1 --2- 172.21.15.144:0/3278622185 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f08d80ffd30 0x7f08d81012c0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:56.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.321+0000 7f08de70f700 1 -- 172.21.15.144:0/3278622185 >> 172.21.15.144:0/3278622185 conn(0x7f08d8074f50 msgr2=0x7f08d8103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:10:56.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.322+0000 7f08de70f700 1 -- 172.21.15.144:0/3278622185 shutdown_connections 2024-09-10T12:10:56.325 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:56.322+0000 7f08de70f700 1 -- 172.21.15.144:0/3278622185 wait complete. 2024-09-10T12:10:56.325 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 42 2024-09-10T12:10:56.965 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 19, 'max_mds': 2, 'flags': 50} 2024-09-10T12:10:56.965 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 20 2024-09-10T12:10:57.174 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:10:57.295 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:57 smithi144 ceph-mon[87436]: pgmap v1171: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:57.295 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:57 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3278622185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T12:10:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:57 smithi178 ceph-mon[65940]: pgmap v1171: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:10:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:57 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3278622185' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-10T12:10:58.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.744+0000 7f25c460a700 1 -- 172.21.15.144:0/4009100745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25bc100d40 msgr2=0x7f25bc1094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:58.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.744+0000 7f25c460a700 1 --2- 172.21.15.144:0/4009100745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25bc100d40 0x7f25bc1094a0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009910 tx=0x7f25b0009c20 comp rx=0 tx=0).stop 2024-09-10T12:10:58.747 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.744+0000 7f25c460a700 1 -- 172.21.15.144:0/4009100745 shutdown_connections 2024-09-10T12:10:58.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.744+0000 7f25c460a700 1 --2- 172.21.15.144:0/4009100745 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25bc100d40 0x7f25bc1094a0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:58.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.744+0000 7f25c460a700 1 --2- 172.21.15.144:0/4009100745 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25bc100430 0x7f25bc100800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:58.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.744+0000 7f25c460a700 1 -- 172.21.15.144:0/4009100745 >> 172.21.15.144:0/4009100745 conn(0x7f25bc0748b0 msgr2=0x7f25bc074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:10:58.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.744+0000 7f25c460a700 1 -- 172.21.15.144:0/4009100745 shutdown_connections 2024-09-10T12:10:58.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.744+0000 7f25c460a700 1 -- 172.21.15.144:0/4009100745 wait complete. 2024-09-10T12:10:58.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.745+0000 7f25c460a700 1 Processor -- start 2024-09-10T12:10:58.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.745+0000 7f25c460a700 1 -- start start 2024-09-10T12:10:58.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.745+0000 7f25c460a700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25bc100430 0x7f25bc195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:58.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.745+0000 7f25c460a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25bc100d40 0x7f25bc196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:58.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.745+0000 7f25c460a700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc196960 con 0x7f25bc100d40 2024-09-10T12:10:58.748 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.745+0000 7f25c460a700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc19a6f0 con 0x7f25bc100430 2024-09-10T12:10:58.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.745+0000 7f25c1ba5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25bc100d40 0x7f25bc196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:58.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.745+0000 7f25c1ba5700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25bc100d40 0x7f25bc196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:48820/0 (socket says 172.21.15.144:48820) 2024-09-10T12:10:58.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.745+0000 7f25c1ba5700 1 -- 172.21.15.144:0/921723475 learned_addr learned my addr 172.21.15.144:0/921723475 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:10:58.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.745+0000 7f25c23a6700 1 --2- 172.21.15.144:0/921723475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25bc100430 0x7f25bc195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:58.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.745+0000 7f25c1ba5700 1 -- 172.21.15.144:0/921723475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25bc100430 msgr2=0x7f25bc195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:58.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.745+0000 7f25c1ba5700 1 --2- 172.21.15.144:0/921723475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25bc100430 0x7f25bc195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:58.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.745+0000 7f25c1ba5700 1 -- 172.21.15.144:0/921723475 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b00095f0 con 0x7f25bc100d40 2024-09-10T12:10:58.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.746+0000 7f25c1ba5700 1 --2- 172.21.15.144:0/921723475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25bc100d40 0x7f25bc196280 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f25b0000c00 tx=0x7f25b00056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:10:58.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.746+0000 7f25bb7fe700 1 -- 172.21.15.144:0/921723475 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25b001d070 con 0x7f25bc100d40 2024-09-10T12:10:58.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.746+0000 7f25bb7fe700 1 -- 172.21.15.144:0/921723475 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25b000f460 con 0x7f25bc100d40 2024-09-10T12:10:58.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.746+0000 7f25c460a700 1 -- 172.21.15.144:0/921723475 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc19a970 con 0x7f25bc100d40 2024-09-10T12:10:58.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.746+0000 7f25bb7fe700 1 -- 172.21.15.144:0/921723475 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25b0021640 con 0x7f25bc100d40 2024-09-10T12:10:58.749 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.746+0000 7f25c460a700 1 -- 172.21.15.144:0/921723475 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc19aec0 con 0x7f25bc100d40 2024-09-10T12:10:58.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.747+0000 7f25c460a700 1 -- 172.21.15.144:0/921723475 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25bc107df0 con 0x7f25bc100d40 2024-09-10T12:10:58.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.747+0000 7f25bb7fe700 1 -- 172.21.15.144:0/921723475 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f25b000fa60 con 0x7f25bc100d40 2024-09-10T12:10:58.750 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.748+0000 7f25bb7fe700 1 --2- 172.21.15.144:0/921723475 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25a406e200 0x7f25a40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:10:58.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.748+0000 7f25bb7fe700 1 -- 172.21.15.144:0/921723475 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f25b008f280 con 0x7f25bc100d40 2024-09-10T12:10:58.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.748+0000 7f25c23a6700 1 --2- 172.21.15.144:0/921723475 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25a406e200 0x7f25a40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:10:58.751 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.748+0000 7f25c23a6700 1 --2- 172.21.15.144:0/921723475 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25a406e200 0x7f25a40706b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f25b4005950 tx=0x7f25b40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:10:58.754 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.752+0000 7f25bb7fe700 1 -- 172.21.15.144:0/921723475 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f25b0058660 con 0x7f25bc100d40 2024-09-10T12:10:58.998 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.997+0000 7f25c460a700 1 -- 172.21.15.144:0/921723475 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f25bc19b1a0 con 0x7f25bc100d40 2024-09-10T12:10:58.999 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:58.998+0000 7f25bb7fe700 1 -- 172.21.15.144:0/921723475 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v42) v1 ==== 107+0+4943 (secure 0 0 0) 0x7f25b005bd20 con 0x7f25bc100d40 2024-09-10T12:10:58.999 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:10:58.999 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":20,"btime":"1970-01-01T00:00:00:000000+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:28:40.999990+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14488,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14488":{"gid":14488,"name":"cephfs.smithi144.jaoyvs","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6829/2138997251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2138997251},{"type":"v1","addr":"172.21.15.144:6829","nonce":2138997251}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14498":{"gid":14498,"name":"cephfs.smithi144.bfyvwv","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6827/1752150537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1752150537},{"type":"v1","addr":"172.21.15.144:6827","nonce":1752150537}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi178.nmhrmh","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6825/2986758450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2986758450},{"type":"v1","addr":"172.21.15.178:6825","nonce":2986758450}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi178.czjrsa","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.178:6827/3262518638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3262518638},{"type":"v1","addr":"172.21.15.178:6827","nonce":3262518638}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:10:59.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:59.000+0000 7f25c460a700 1 -- 172.21.15.144:0/921723475 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25a406e200 msgr2=0x7f25a40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:59.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:59.000+0000 7f25c460a700 1 --2- 172.21.15.144:0/921723475 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25a406e200 0x7f25a40706b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f25b4005950 tx=0x7f25b40058e0 comp rx=0 tx=0).stop 2024-09-10T12:10:59.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:59.000+0000 7f25c460a700 1 -- 172.21.15.144:0/921723475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25bc100d40 msgr2=0x7f25bc196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:10:59.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:59.000+0000 7f25c460a700 1 --2- 172.21.15.144:0/921723475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25bc100d40 0x7f25bc196280 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f25b0000c00 tx=0x7f25b00056a0 comp rx=0 tx=0).stop 2024-09-10T12:10:59.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:59.000+0000 7f25c460a700 1 -- 172.21.15.144:0/921723475 shutdown_connections 2024-09-10T12:10:59.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:59.000+0000 7f25c460a700 1 --2- 172.21.15.144:0/921723475 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f25a406e200 0x7f25a40706b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:59.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:59.000+0000 7f25c460a700 1 --2- 172.21.15.144:0/921723475 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f25bc100430 0x7f25bc195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:59.002 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:59.000+0000 7f25c460a700 1 --2- 172.21.15.144:0/921723475 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f25bc100d40 0x7f25bc196280 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:10:59.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:59.000+0000 7f25c460a700 1 -- 172.21.15.144:0/921723475 >> 172.21.15.144:0/921723475 conn(0x7f25bc0748b0 msgr2=0x7f25bc0ffc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:10:59.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:59.000+0000 7f25c460a700 1 -- 172.21.15.144:0/921723475 shutdown_connections 2024-09-10T12:10:59.003 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:10:59.000+0000 7f25c460a700 1 -- 172.21.15.144:0/921723475 wait complete. 2024-09-10T12:10:59.003 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 20 2024-09-10T12:10:59.311 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:59 smithi144 ceph-mon[87436]: pgmap v1172: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:10:59.311 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:59 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:10:59.311 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:10:59 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/921723475' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-09-10T12:10:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:59 smithi178 ceph-mon[65940]: pgmap v1172: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:10:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:59 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:10:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:10:59 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/921723475' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-09-10T12:10:59.634 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 21 2024-09-10T12:10:59.836 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:01.253 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:01 smithi144 ceph-mon[87436]: pgmap v1173: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:11:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:01 smithi178 ceph-mon[65940]: pgmap v1173: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:11:01.470 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.467+0000 7f581d94d700 1 -- 172.21.15.144:0/1575167878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58180ff2c0 msgr2=0x7f58180ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:01.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.467+0000 7f581d94d700 1 --2- 172.21.15.144:0/1575167878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58180ff2c0 0x7f58180ff690 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f5808009910 tx=0x7f5808009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:01.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.467+0000 7f581d94d700 1 -- 172.21.15.144:0/1575167878 shutdown_connections 2024-09-10T12:11:01.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.467+0000 7f581d94d700 1 --2- 172.21.15.144:0/1575167878 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58180ffc60 0x7f5818103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:01.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.467+0000 7f581d94d700 1 --2- 172.21.15.144:0/1575167878 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58180ff2c0 0x7f58180ff690 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:01.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.467+0000 7f581d94d700 1 -- 172.21.15.144:0/1575167878 >> 172.21.15.144:0/1575167878 conn(0x7f58180fab80 msgr2=0x7f58180fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:01.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.467+0000 7f581d94d700 1 -- 172.21.15.144:0/1575167878 shutdown_connections 2024-09-10T12:11:01.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.467+0000 7f581d94d700 1 -- 172.21.15.144:0/1575167878 wait complete. 2024-09-10T12:11:01.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.468+0000 7f581d94d700 1 Processor -- start 2024-09-10T12:11:01.471 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.468+0000 7f581d94d700 1 -- start start 2024-09-10T12:11:01.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.468+0000 7f581d94d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58180ff2c0 0x7f5818195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:01.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.468+0000 7f581d94d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58180ffc60 0x7f58181962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:01.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.468+0000 7f581d94d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5818196990 con 0x7f58180ffc60 2024-09-10T12:11:01.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.468+0000 7f581d94d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581819a6d0 con 0x7f58180ff2c0 2024-09-10T12:11:01.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.468+0000 7f580ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58180ffc60 0x7f58181962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:01.472 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.468+0000 7f580ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58180ffc60 0x7f58181962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:48838/0 (socket says 172.21.15.144:48838) 2024-09-10T12:11:01.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.468+0000 7f580ffff700 1 -- 172.21.15.144:0/3652098362 learned_addr learned my addr 172.21.15.144:0/3652098362 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:01.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.468+0000 7f5816ffd700 1 --2- 172.21.15.144:0/3652098362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58180ff2c0 0x7f5818195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:01.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.468+0000 7f580ffff700 1 -- 172.21.15.144:0/3652098362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58180ff2c0 msgr2=0x7f5818195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:01.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.468+0000 7f580ffff700 1 --2- 172.21.15.144:0/3652098362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58180ff2c0 0x7f5818195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:01.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.468+0000 7f580ffff700 1 -- 172.21.15.144:0/3652098362 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58080095f0 con 0x7f58180ffc60 2024-09-10T12:11:01.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.469+0000 7f580ffff700 1 --2- 172.21.15.144:0/3652098362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58180ffc60 0x7f58181962b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f580000d900 tx=0x7f580000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:01.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.469+0000 7f581c94b700 1 -- 172.21.15.144:0/3652098362 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5800009900 con 0x7f58180ffc60 2024-09-10T12:11:01.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.469+0000 7f581d94d700 1 -- 172.21.15.144:0/3652098362 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f581819a9b0 con 0x7f58180ffc60 2024-09-10T12:11:01.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.469+0000 7f581c94b700 1 -- 172.21.15.144:0/3652098362 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5800010460 con 0x7f58180ffc60 2024-09-10T12:11:01.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.469+0000 7f581c94b700 1 -- 172.21.15.144:0/3652098362 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f580000f5f0 con 0x7f58180ffc60 2024-09-10T12:11:01.473 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.469+0000 7f581d94d700 1 -- 172.21.15.144:0/3652098362 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581819af00 con 0x7f58180ffc60 2024-09-10T12:11:01.474 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.470+0000 7f581d94d700 1 -- 172.21.15.144:0/3652098362 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58181085e0 con 0x7f58180ffc60 2024-09-10T12:11:01.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.470+0000 7f581c94b700 1 -- 172.21.15.144:0/3652098362 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f58000105d0 con 0x7f58180ffc60 2024-09-10T12:11:01.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.471+0000 7f581c94b700 1 --2- 172.21.15.144:0/3652098362 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f580406e200 0x7f58040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:01.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.471+0000 7f581c94b700 1 -- 172.21.15.144:0/3652098362 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f580008dc60 con 0x7f58180ffc60 2024-09-10T12:11:01.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.471+0000 7f5816ffd700 1 --2- 172.21.15.144:0/3652098362 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f580406e200 0x7f58040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:01.475 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.471+0000 7f5816ffd700 1 --2- 172.21.15.144:0/3652098362 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f580406e200 0x7f58040706b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f580800b5c0 tx=0x7f580801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:01.477 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.475+0000 7f581c94b700 1 -- 172.21.15.144:0/3652098362 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f5800057040 con 0x7f58180ffc60 2024-09-10T12:11:01.723 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.722+0000 7f581d94d700 1 -- 172.21.15.144:0/3652098362 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f581819b1e0 con 0x7f58180ffc60 2024-09-10T12:11:01.724 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.723+0000 7f581c94b700 1 -- 172.21.15.144:0/3652098362 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v42) v1 ==== 107+0+4944 (secure 0 0 0) 0x7f580005a700 con 0x7f58180ffc60 2024-09-10T12:11:01.724 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:01.724 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":21,"btime":"1970-01-01T00:00:00:000000+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":21,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:30:16.315952+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14488,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14488":{"gid":14488,"name":"cephfs.smithi144.jaoyvs","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6829/2138997251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2138997251},{"type":"v1","addr":"172.21.15.144:6829","nonce":2138997251}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14498":{"gid":14498,"name":"cephfs.smithi144.bfyvwv","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6827/1752150537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1752150537},{"type":"v1","addr":"172.21.15.144:6827","nonce":1752150537}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi178.nmhrmh","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6825/2986758450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2986758450},{"type":"v1","addr":"172.21.15.178:6825","nonce":2986758450}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi178.czjrsa","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.178:6827/3262518638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3262518638},{"type":"v1","addr":"172.21.15.178:6827","nonce":3262518638}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:01.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.725+0000 7f581d94d700 1 -- 172.21.15.144:0/3652098362 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f580406e200 msgr2=0x7f58040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:01.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.725+0000 7f581d94d700 1 --2- 172.21.15.144:0/3652098362 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f580406e200 0x7f58040706b0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f580800b5c0 tx=0x7f580801a040 comp rx=0 tx=0).stop 2024-09-10T12:11:01.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.725+0000 7f581d94d700 1 -- 172.21.15.144:0/3652098362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58180ffc60 msgr2=0x7f58181962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:01.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.725+0000 7f581d94d700 1 --2- 172.21.15.144:0/3652098362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58180ffc60 0x7f58181962b0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f580000d900 tx=0x7f580000dc10 comp rx=0 tx=0).stop 2024-09-10T12:11:01.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.725+0000 7f581d94d700 1 -- 172.21.15.144:0/3652098362 shutdown_connections 2024-09-10T12:11:01.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.725+0000 7f581d94d700 1 --2- 172.21.15.144:0/3652098362 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f580406e200 0x7f58040706b0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:01.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.725+0000 7f581d94d700 1 --2- 172.21.15.144:0/3652098362 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f58180ff2c0 0x7f5818195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:01.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.725+0000 7f581d94d700 1 --2- 172.21.15.144:0/3652098362 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f58180ffc60 0x7f58181962b0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:01.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.725+0000 7f581d94d700 1 -- 172.21.15.144:0/3652098362 >> 172.21.15.144:0/3652098362 conn(0x7f58180fab80 msgr2=0x7f5818068ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:01.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.725+0000 7f581d94d700 1 -- 172.21.15.144:0/3652098362 shutdown_connections 2024-09-10T12:11:01.727 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:01.725+0000 7f581d94d700 1 -- 172.21.15.144:0/3652098362 wait complete. 2024-09-10T12:11:01.728 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 21 2024-09-10T12:11:02.286 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:02 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3652098362' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-09-10T12:11:02.308 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 22 2024-09-10T12:11:02.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:02 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3652098362' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-09-10T12:11:02.507 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:03 smithi178 ceph-mon[65940]: pgmap v1174: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:03 smithi144 ceph-mon[87436]: pgmap v1174: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:04.076 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.072+0000 7f7e4b38c700 1 -- 172.21.15.144:0/844671058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e441003d0 msgr2=0x7f7e441007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:04.076 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.072+0000 7f7e4b38c700 1 --2- 172.21.15.144:0/844671058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e441003d0 0x7f7e441007a0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f7e34009910 tx=0x7f7e34009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:04.076 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.073+0000 7f7e4b38c700 1 -- 172.21.15.144:0/844671058 shutdown_connections 2024-09-10T12:11:04.076 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.073+0000 7f7e4b38c700 1 --2- 172.21.15.144:0/844671058 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e44100d70 0x7f7e44104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:04.076 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.073+0000 7f7e4b38c700 1 --2- 172.21.15.144:0/844671058 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e441003d0 0x7f7e441007a0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:04.076 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.073+0000 7f7e4b38c700 1 -- 172.21.15.144:0/844671058 >> 172.21.15.144:0/844671058 conn(0x7f7e440fbc90 msgr2=0x7f7e440fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:04.076 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.073+0000 7f7e4b38c700 1 -- 172.21.15.144:0/844671058 shutdown_connections 2024-09-10T12:11:04.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.073+0000 7f7e4b38c700 1 -- 172.21.15.144:0/844671058 wait complete. 2024-09-10T12:11:04.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.073+0000 7f7e4b38c700 1 Processor -- start 2024-09-10T12:11:04.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.073+0000 7f7e4b38c700 1 -- start start 2024-09-10T12:11:04.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.073+0000 7f7e4b38c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e441003d0 0x7f7e44073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:04.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e4b38c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e44100d70 0x7f7e44073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:04.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e4b38c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e44073ad0 con 0x7f7e441003d0 2024-09-10T12:11:04.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e4b38c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e44073c10 con 0x7f7e44100d70 2024-09-10T12:11:04.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e49128700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e441003d0 0x7f7e44073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:04.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e49128700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e441003d0 0x7f7e44073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51490/0 (socket says 172.21.15.144:51490) 2024-09-10T12:11:04.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e49128700 1 -- 172.21.15.144:0/1277387200 learned_addr learned my addr 172.21.15.144:0/1277387200 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:04.077 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e48927700 1 --2- 172.21.15.144:0/1277387200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e44100d70 0x7f7e44073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:04.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e49128700 1 -- 172.21.15.144:0/1277387200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e44100d70 msgr2=0x7f7e44073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:04.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e49128700 1 --2- 172.21.15.144:0/1277387200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e44100d70 0x7f7e44073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:04.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e49128700 1 -- 172.21.15.144:0/1277387200 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e340095f0 con 0x7f7e441003d0 2024-09-10T12:11:04.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e49128700 1 --2- 172.21.15.144:0/1277387200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e441003d0 0x7f7e44073050 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7e34006010 tx=0x7f7e3400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:04.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e3a7fc700 1 -- 172.21.15.144:0/1277387200 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e3401d070 con 0x7f7e441003d0 2024-09-10T12:11:04.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e3a7fc700 1 -- 172.21.15.144:0/1277387200 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7e3400f460 con 0x7f7e441003d0 2024-09-10T12:11:04.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e4b38c700 1 -- 172.21.15.144:0/1277387200 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e44073e90 con 0x7f7e441003d0 2024-09-10T12:11:04.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e3a7fc700 1 -- 172.21.15.144:0/1277387200 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7e34017610 con 0x7f7e441003d0 2024-09-10T12:11:04.078 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.074+0000 7f7e4b38c700 1 -- 172.21.15.144:0/1277387200 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e441a2260 con 0x7f7e441003d0 2024-09-10T12:11:04.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.075+0000 7f7e4b38c700 1 -- 172.21.15.144:0/1277387200 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e441086e0 con 0x7f7e441003d0 2024-09-10T12:11:04.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.076+0000 7f7e3a7fc700 1 -- 172.21.15.144:0/1277387200 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7e34003a60 con 0x7f7e441003d0 2024-09-10T12:11:04.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.076+0000 7f7e3a7fc700 1 --2- 172.21.15.144:0/1277387200 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7e3006e200 0x7f7e300706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:04.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.076+0000 7f7e3a7fc700 1 -- 172.21.15.144:0/1277387200 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7e3408f500 con 0x7f7e441003d0 2024-09-10T12:11:04.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.076+0000 7f7e48927700 1 --2- 172.21.15.144:0/1277387200 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7e3006e200 0x7f7e300706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:04.079 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.077+0000 7f7e48927700 1 --2- 172.21.15.144:0/1277387200 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7e3006e200 0x7f7e300706b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7e44074840 tx=0x7f7e4000a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:04.082 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.081+0000 7f7e3a7fc700 1 -- 172.21.15.144:0/1277387200 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7e34058960 con 0x7f7e441003d0 2024-09-10T12:11:04.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.328+0000 7f7e4b38c700 1 -- 172.21.15.144:0/1277387200 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f7e4402cc30 con 0x7f7e441003d0 2024-09-10T12:11:04.330 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.329+0000 7f7e3a7fc700 1 -- 172.21.15.144:0/1277387200 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v42) v1 ==== 107+0+4943 (secure 0 0 0) 0x7f7e3405c020 con 0x7f7e441003d0 2024-09-10T12:11:04.331 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:04.331 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":22,"btime":"1970-01-01T00:00:00:000000+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":22,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:31:50.444180+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14488,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14488":{"gid":14488,"name":"cephfs.smithi144.jaoyvs","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6829/2138997251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2138997251},{"type":"v1","addr":"172.21.15.144:6829","nonce":2138997251}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14498":{"gid":14498,"name":"cephfs.smithi144.bfyvwv","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6827/1752150537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1752150537},{"type":"v1","addr":"172.21.15.144:6827","nonce":1752150537}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi178.nmhrmh","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6825/2986758450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2986758450},{"type":"v1","addr":"172.21.15.178:6825","nonce":2986758450}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi178.czjrsa","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.178:6827/3262518638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3262518638},{"type":"v1","addr":"172.21.15.178:6827","nonce":3262518638}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:04.333 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.331+0000 7f7e4b38c700 1 -- 172.21.15.144:0/1277387200 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7e3006e200 msgr2=0x7f7e300706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:04.333 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.331+0000 7f7e4b38c700 1 --2- 172.21.15.144:0/1277387200 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7e3006e200 0x7f7e300706b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7e44074840 tx=0x7f7e4000a300 comp rx=0 tx=0).stop 2024-09-10T12:11:04.334 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.331+0000 7f7e4b38c700 1 -- 172.21.15.144:0/1277387200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e441003d0 msgr2=0x7f7e44073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:04.334 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.331+0000 7f7e4b38c700 1 --2- 172.21.15.144:0/1277387200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e441003d0 0x7f7e44073050 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f7e34006010 tx=0x7f7e3400bd70 comp rx=0 tx=0).stop 2024-09-10T12:11:04.334 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.331+0000 7f7e4b38c700 1 -- 172.21.15.144:0/1277387200 shutdown_connections 2024-09-10T12:11:04.334 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.331+0000 7f7e4b38c700 1 --2- 172.21.15.144:0/1277387200 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7e3006e200 0x7f7e300706b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:04.334 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.331+0000 7f7e4b38c700 1 --2- 172.21.15.144:0/1277387200 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7e441003d0 0x7f7e44073050 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:04.334 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.331+0000 7f7e4b38c700 1 --2- 172.21.15.144:0/1277387200 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7e44100d70 0x7f7e44073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:04.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.331+0000 7f7e4b38c700 1 -- 172.21.15.144:0/1277387200 >> 172.21.15.144:0/1277387200 conn(0x7f7e440fbc90 msgr2=0x7f7e440fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:04.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.331+0000 7f7e4b38c700 1 -- 172.21.15.144:0/1277387200 shutdown_connections 2024-09-10T12:11:04.335 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:04.331+0000 7f7e4b38c700 1 -- 172.21.15.144:0/1277387200 wait complete. 2024-09-10T12:11:04.335 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 22 2024-09-10T12:11:05.066 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 23 2024-09-10T12:11:05.266 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:05 smithi178 ceph-mon[65940]: pgmap v1175: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:11:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:05 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1277387200' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-09-10T12:11:05.386 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:05 smithi144 ceph-mon[87436]: pgmap v1175: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:11:05.386 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:05 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1277387200' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-09-10T12:11:06.841 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.838+0000 7f75d5b93700 1 -- 172.21.15.144:0/498309052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0100cb0 msgr2=0x7f75d0109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:06.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.838+0000 7f75d5b93700 1 --2- 172.21.15.144:0/498309052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0100cb0 0x7f75d0109410 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009940 tx=0x7f75c4009c50 comp rx=0 tx=0).stop 2024-09-10T12:11:06.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.838+0000 7f75d5b93700 1 -- 172.21.15.144:0/498309052 shutdown_connections 2024-09-10T12:11:06.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.838+0000 7f75d5b93700 1 --2- 172.21.15.144:0/498309052 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0100cb0 0x7f75d0109410 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:06.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.838+0000 7f75d5b93700 1 --2- 172.21.15.144:0/498309052 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75d01003a0 0x7f75d0100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:06.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.838+0000 7f75d5b93700 1 -- 172.21.15.144:0/498309052 >> 172.21.15.144:0/498309052 conn(0x7f75d0074850 msgr2=0x7f75d0074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:06.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.838+0000 7f75d5b93700 1 -- 172.21.15.144:0/498309052 shutdown_connections 2024-09-10T12:11:06.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.838+0000 7f75d5b93700 1 -- 172.21.15.144:0/498309052 wait complete. 2024-09-10T12:11:06.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.838+0000 7f75d5b93700 1 Processor -- start 2024-09-10T12:11:06.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.839+0000 7f75d5b93700 1 -- start start 2024-09-10T12:11:06.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.839+0000 7f75d5b93700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75d01003a0 0x7f75d0196b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:06.842 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.839+0000 7f75d5b93700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0100cb0 0x7f75d0197080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:06.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.839+0000 7f75d5b93700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d01976a0 con 0x7f75d0100cb0 2024-09-10T12:11:06.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.839+0000 7f75d5b93700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d0197810 con 0x7f75d01003a0 2024-09-10T12:11:06.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.839+0000 7f75ceffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0100cb0 0x7f75d0197080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:06.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.839+0000 7f75ceffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0100cb0 0x7f75d0197080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51504/0 (socket says 172.21.15.144:51504) 2024-09-10T12:11:06.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.839+0000 7f75ceffd700 1 -- 172.21.15.144:0/3241196650 learned_addr learned my addr 172.21.15.144:0/3241196650 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:06.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.839+0000 7f75cf7fe700 1 --2- 172.21.15.144:0/3241196650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75d01003a0 0x7f75d0196b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:06.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.839+0000 7f75ceffd700 1 -- 172.21.15.144:0/3241196650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75d01003a0 msgr2=0x7f75d0196b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:06.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.839+0000 7f75ceffd700 1 --2- 172.21.15.144:0/3241196650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75d01003a0 0x7f75d0196b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:06.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.839+0000 7f75ceffd700 1 -- 172.21.15.144:0/3241196650 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c40095f0 con 0x7f75d0100cb0 2024-09-10T12:11:06.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.840+0000 7f75ceffd700 1 --2- 172.21.15.144:0/3241196650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0100cb0 0x7f75d0197080 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f75c40055f0 tx=0x7f75c40056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:06.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.840+0000 7f75ccff9700 1 -- 172.21.15.144:0/3241196650 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75c401d070 con 0x7f75d0100cb0 2024-09-10T12:11:06.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.840+0000 7f75ccff9700 1 -- 172.21.15.144:0/3241196650 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75c400f460 con 0x7f75d0100cb0 2024-09-10T12:11:06.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.840+0000 7f75d5b93700 1 -- 172.21.15.144:0/3241196650 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d019fbb0 con 0x7f75d0100cb0 2024-09-10T12:11:06.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.840+0000 7f75ccff9700 1 -- 172.21.15.144:0/3241196650 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75c4021640 con 0x7f75d0100cb0 2024-09-10T12:11:06.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.840+0000 7f75d5b93700 1 -- 172.21.15.144:0/3241196650 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d01a00a0 con 0x7f75d0100cb0 2024-09-10T12:11:06.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.841+0000 7f75d5b93700 1 -- 172.21.15.144:0/3241196650 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d0106b10 con 0x7f75d0100cb0 2024-09-10T12:11:06.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.841+0000 7f75ccff9700 1 -- 172.21.15.144:0/3241196650 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f75c402b440 con 0x7f75d0100cb0 2024-09-10T12:11:06.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.842+0000 7f75ccff9700 1 --2- 172.21.15.144:0/3241196650 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f75b806e200 0x7f75b80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:06.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.842+0000 7f75ccff9700 1 -- 172.21.15.144:0/3241196650 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f75c408f1c0 con 0x7f75d0100cb0 2024-09-10T12:11:06.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.842+0000 7f75cf7fe700 1 --2- 172.21.15.144:0/3241196650 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f75b806e200 0x7f75b80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:06.845 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.842+0000 7f75cf7fe700 1 --2- 172.21.15.144:0/3241196650 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f75b806e200 0x7f75b80706b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009d10 tx=0x7f75c0009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:06.848 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:06.847+0000 7f75ccff9700 1 -- 172.21.15.144:0/3241196650 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f75c4058690 con 0x7f75d0100cb0 2024-09-10T12:11:07.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:07.094+0000 7f75d5b93700 1 -- 172.21.15.144:0/3241196650 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7f75d004ea50 con 0x7f75d0100cb0 2024-09-10T12:11:07.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:07.094+0000 7f75ccff9700 1 -- 172.21.15.144:0/3241196650 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v42) v1 ==== 107+0+4957 (secure 0 0 0) 0x7f75c405bd50 con 0x7f75d0100cb0 2024-09-10T12:11:07.096 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:07.096 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":23,"btime":"2024-09-10T11:33:11:119769+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:33:11.119766+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":47,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14488,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14488":{"gid":14488,"name":"cephfs.smithi144.jaoyvs","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6829/2138997251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2138997251},{"type":"v1","addr":"172.21.15.144:6829","nonce":2138997251}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_14498":{"gid":14498,"name":"cephfs.smithi144.bfyvwv","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.144:6827/1752150537","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":1752150537},{"type":"v1","addr":"172.21.15.144:6827","nonce":1752150537}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi178.nmhrmh","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6825/2986758450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2986758450},{"type":"v1","addr":"172.21.15.178:6825","nonce":2986758450}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi178.czjrsa","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":2,"addr":"172.21.15.178:6827/3262518638","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":3262518638},{"type":"v1","addr":"172.21.15.178:6827","nonce":3262518638}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14488,"qdb_cluster":[14488,24267]},"id":1}]} 2024-09-10T12:11:07.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:07.097+0000 7f75d5b93700 1 -- 172.21.15.144:0/3241196650 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f75b806e200 msgr2=0x7f75b80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:07.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:07.097+0000 7f75d5b93700 1 --2- 172.21.15.144:0/3241196650 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f75b806e200 0x7f75b80706b0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f75c0009d10 tx=0x7f75c0009040 comp rx=0 tx=0).stop 2024-09-10T12:11:07.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:07.097+0000 7f75d5b93700 1 -- 172.21.15.144:0/3241196650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0100cb0 msgr2=0x7f75d0197080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:07.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:07.097+0000 7f75d5b93700 1 --2- 172.21.15.144:0/3241196650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0100cb0 0x7f75d0197080 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f75c40055f0 tx=0x7f75c40056d0 comp rx=0 tx=0).stop 2024-09-10T12:11:07.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:07.097+0000 7f75d5b93700 1 -- 172.21.15.144:0/3241196650 shutdown_connections 2024-09-10T12:11:07.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:07.097+0000 7f75d5b93700 1 --2- 172.21.15.144:0/3241196650 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f75b806e200 0x7f75b80706b0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:07.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:07.097+0000 7f75d5b93700 1 --2- 172.21.15.144:0/3241196650 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f75d01003a0 0x7f75d0196b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:07.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:07.097+0000 7f75d5b93700 1 --2- 172.21.15.144:0/3241196650 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f75d0100cb0 0x7f75d0197080 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:07.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:07.097+0000 7f75d5b93700 1 -- 172.21.15.144:0/3241196650 >> 172.21.15.144:0/3241196650 conn(0x7f75d0074850 msgr2=0x7f75d0103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:07.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:07.097+0000 7f75d5b93700 1 -- 172.21.15.144:0/3241196650 shutdown_connections 2024-09-10T12:11:07.101 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:07.097+0000 7f75d5b93700 1 -- 172.21.15.144:0/3241196650 wait complete. 2024-09-10T12:11:07.101 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 23 2024-09-10T12:11:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:07 smithi178 ceph-mon[65940]: pgmap v1176: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:07.399 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:07 smithi144 ceph-mon[87436]: pgmap v1176: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:07.779 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 24 2024-09-10T12:11:07.979 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:08.353 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:08 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3241196650' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-09-10T12:11:08.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:08 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3241196650' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-09-10T12:11:09.089 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:09 smithi144 ceph-mon[87436]: pgmap v1177: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:11:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:09 smithi178 ceph-mon[65940]: pgmap v1177: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:11:09.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.628+0000 7f4916344700 1 -- 172.21.15.144:0/313530413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49100ff2c0 msgr2=0x7f49100ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:09.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.628+0000 7f4916344700 1 --2- 172.21.15.144:0/313530413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49100ff2c0 0x7f49100ff690 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f4900009910 tx=0x7f4900009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:09.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.628+0000 7f4916344700 1 -- 172.21.15.144:0/313530413 shutdown_connections 2024-09-10T12:11:09.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.628+0000 7f4916344700 1 --2- 172.21.15.144:0/313530413 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49100ffc60 0x7f4910103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:09.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.628+0000 7f4916344700 1 --2- 172.21.15.144:0/313530413 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49100ff2c0 0x7f49100ff690 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:09.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.628+0000 7f4916344700 1 -- 172.21.15.144:0/313530413 >> 172.21.15.144:0/313530413 conn(0x7f49100fab80 msgr2=0x7f49100fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:09.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.628+0000 7f4916344700 1 -- 172.21.15.144:0/313530413 shutdown_connections 2024-09-10T12:11:09.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.628+0000 7f4916344700 1 -- 172.21.15.144:0/313530413 wait complete. 2024-09-10T12:11:09.630 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.628+0000 7f4916344700 1 Processor -- start 2024-09-10T12:11:09.631 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.629+0000 7f4916344700 1 -- start start 2024-09-10T12:11:09.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.629+0000 7f4916344700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49100ff2c0 0x7f4910194a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:09.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.629+0000 7f4916344700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49100ffc60 0x7f4910194fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:09.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.629+0000 7f4916344700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4910195500 con 0x7f49100ffc60 2024-09-10T12:11:09.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.629+0000 7f4916344700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4910195670 con 0x7f49100ff2c0 2024-09-10T12:11:09.632 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.629+0000 7f490f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49100ffc60 0x7f4910194fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:09.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.629+0000 7f490f7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49100ffc60 0x7f4910194fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51516/0 (socket says 172.21.15.144:51516) 2024-09-10T12:11:09.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.629+0000 7f490f7fe700 1 -- 172.21.15.144:0/1197653731 learned_addr learned my addr 172.21.15.144:0/1197653731 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:09.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.629+0000 7f490f7fe700 1 -- 172.21.15.144:0/1197653731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49100ff2c0 msgr2=0x7f4910194a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:09.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.629+0000 7f490ffff700 1 --2- 172.21.15.144:0/1197653731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49100ff2c0 0x7f4910194a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:09.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.629+0000 7f490f7fe700 1 --2- 172.21.15.144:0/1197653731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49100ff2c0 0x7f4910194a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:09.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.629+0000 7f490f7fe700 1 -- 172.21.15.144:0/1197653731 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49000095f0 con 0x7f49100ffc60 2024-09-10T12:11:09.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.629+0000 7f490ffff700 1 --2- 172.21.15.144:0/1197653731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49100ff2c0 0x7f4910194a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T12:11:09.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.630+0000 7f490f7fe700 1 --2- 172.21.15.144:0/1197653731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49100ffc60 0x7f4910194fc0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f490400d900 tx=0x7f490400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:09.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.630+0000 7f490d7fa700 1 -- 172.21.15.144:0/1197653731 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4904009900 con 0x7f49100ffc60 2024-09-10T12:11:09.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.630+0000 7f4916344700 1 -- 172.21.15.144:0/1197653731 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49101a20f0 con 0x7f49100ffc60 2024-09-10T12:11:09.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.630+0000 7f490d7fa700 1 -- 172.21.15.144:0/1197653731 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4904010460 con 0x7f49100ffc60 2024-09-10T12:11:09.633 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.630+0000 7f490d7fa700 1 -- 172.21.15.144:0/1197653731 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f490400f5f0 con 0x7f49100ffc60 2024-09-10T12:11:09.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.630+0000 7f4916344700 1 -- 172.21.15.144:0/1197653731 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49101a2450 con 0x7f49100ffc60 2024-09-10T12:11:09.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.631+0000 7f4916344700 1 -- 172.21.15.144:0/1197653731 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49101075d0 con 0x7f49100ffc60 2024-09-10T12:11:09.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.631+0000 7f490d7fa700 1 -- 172.21.15.144:0/1197653731 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f49040105d0 con 0x7f49100ffc60 2024-09-10T12:11:09.634 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.632+0000 7f490d7fa700 1 --2- 172.21.15.144:0/1197653731 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f48f806e200 0x7f48f80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.632+0000 7f490d7fa700 1 -- 172.21.15.144:0/1197653731 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f490408dea0 con 0x7f49100ffc60 2024-09-10T12:11:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.632+0000 7f490ffff700 1 --2- 172.21.15.144:0/1197653731 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f48f806e200 0x7f48f80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:09.635 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.632+0000 7f490ffff700 1 --2- 172.21.15.144:0/1197653731 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f48f806e200 0x7f48f80706b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f490000b5c0 tx=0x7f49000053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:09.637 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.636+0000 7f490d7fa700 1 -- 172.21.15.144:0/1197653731 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f490405adb0 con 0x7f49100ffc60 2024-09-10T12:11:09.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.885+0000 7f4916344700 1 -- 172.21.15.144:0/1197653731 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7f491004ea50 con 0x7f49100ffc60 2024-09-10T12:11:09.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.886+0000 7f490d7fa700 1 -- 172.21.15.144:0/1197653731 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v42) v1 ==== 107+0+3379 (secure 0 0 0) 0x7f4904017d50 con 0x7f49100ffc60 2024-09-10T12:11:09.887 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:09.888 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":24,"btime":"2024-09-10T11:37:32:906660+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:37:31.909658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14488,"mds_1":24267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14488":{"gid":14488,"name":"cephfs.smithi144.jaoyvs","rank":0,"incarnation":14,"state":"up:active","state_seq":6,"addr":"172.21.15.144:6829/2138997251","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":2138997251},{"type":"v1","addr":"172.21.15.144:6829","nonce":2138997251}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24267":{"gid":24267,"name":"cephfs.smithi178.nmhrmh","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.178:6825/2986758450","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":2986758450},{"type":"v1","addr":"172.21.15.178:6825","nonce":2986758450}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":14488,"qdb_cluster":[24267,14488]},"id":1}]} 2024-09-10T12:11:09.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.888+0000 7f4916344700 1 -- 172.21.15.144:0/1197653731 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f48f806e200 msgr2=0x7f48f80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:09.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.888+0000 7f4916344700 1 --2- 172.21.15.144:0/1197653731 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f48f806e200 0x7f48f80706b0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f490000b5c0 tx=0x7f49000053a0 comp rx=0 tx=0).stop 2024-09-10T12:11:09.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.888+0000 7f4916344700 1 -- 172.21.15.144:0/1197653731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49100ffc60 msgr2=0x7f4910194fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:09.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.888+0000 7f4916344700 1 --2- 172.21.15.144:0/1197653731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49100ffc60 0x7f4910194fc0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f490400d900 tx=0x7f490400dcc0 comp rx=0 tx=0).stop 2024-09-10T12:11:09.890 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.888+0000 7f4916344700 1 -- 172.21.15.144:0/1197653731 shutdown_connections 2024-09-10T12:11:09.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.888+0000 7f4916344700 1 --2- 172.21.15.144:0/1197653731 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f48f806e200 0x7f48f80706b0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:09.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.888+0000 7f4916344700 1 --2- 172.21.15.144:0/1197653731 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f49100ff2c0 0x7f4910194a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:09.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.888+0000 7f4916344700 1 --2- 172.21.15.144:0/1197653731 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f49100ffc60 0x7f4910194fc0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:09.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.888+0000 7f4916344700 1 -- 172.21.15.144:0/1197653731 >> 172.21.15.144:0/1197653731 conn(0x7f49100fab80 msgr2=0x7f49100fc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:09.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.889+0000 7f4916344700 1 -- 172.21.15.144:0/1197653731 shutdown_connections 2024-09-10T12:11:09.891 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:09.889+0000 7f4916344700 1 -- 172.21.15.144:0/1197653731 wait complete. 2024-09-10T12:11:09.891 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 24 2024-09-10T12:11:10.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:10 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1197653731' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-09-10T12:11:10.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:10 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1197653731' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-09-10T12:11:10.468 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 24 2024-09-10T12:11:10.468 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 25 2024-09-10T12:11:10.668 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:11 smithi178 ceph-mon[65940]: pgmap v1178: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:11:11.395 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:11 smithi144 ceph-mon[87436]: pgmap v1178: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:11:12.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.201+0000 7f4dd099d700 1 -- 172.21.15.144:0/3023520327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dcc0ffba0 msgr2=0x7f4dcc10a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:12.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.201+0000 7f4dd099d700 1 --2- 172.21.15.144:0/3023520327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dcc0ffba0 0x7f4dcc10a610 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009910 tx=0x7f4dc0009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:12.202 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.201+0000 7f4dd099d700 1 -- 172.21.15.144:0/3023520327 shutdown_connections 2024-09-10T12:11:12.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.201+0000 7f4dd099d700 1 --2- 172.21.15.144:0/3023520327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dcc0ffba0 0x7f4dcc10a610 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:12.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.201+0000 7f4dd099d700 1 --2- 172.21.15.144:0/3023520327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dcc0ff290 0x7f4dcc0ff660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:12.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.201+0000 7f4dd099d700 1 -- 172.21.15.144:0/3023520327 >> 172.21.15.144:0/3023520327 conn(0x7f4dcc074850 msgr2=0x7f4dcc074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:12.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.201+0000 7f4dd099d700 1 -- 172.21.15.144:0/3023520327 shutdown_connections 2024-09-10T12:11:12.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.201+0000 7f4dd099d700 1 -- 172.21.15.144:0/3023520327 wait complete. 2024-09-10T12:11:12.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.202+0000 7f4dd099d700 1 Processor -- start 2024-09-10T12:11:12.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.202+0000 7f4dd099d700 1 -- start start 2024-09-10T12:11:12.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.202+0000 7f4dd099d700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dcc0ff290 0x7f4dcc197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:12.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.202+0000 7f4dd099d700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dcc0ffba0 0x7f4dcc198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:12.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.202+0000 7f4dd099d700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc198bf0 con 0x7f4dcc0ff290 2024-09-10T12:11:12.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.202+0000 7f4dd099d700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc19c980 con 0x7f4dcc0ffba0 2024-09-10T12:11:12.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.202+0000 7f4dca59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dcc0ff290 0x7f4dcc197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:12.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.202+0000 7f4dca59c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dcc0ff290 0x7f4dcc197fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:51536/0 (socket says 172.21.15.144:51536) 2024-09-10T12:11:12.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.202+0000 7f4dca59c700 1 -- 172.21.15.144:0/1768766181 learned_addr learned my addr 172.21.15.144:0/1768766181 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:12.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.202+0000 7f4dc9d9b700 1 --2- 172.21.15.144:0/1768766181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dcc0ffba0 0x7f4dcc198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:12.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.202+0000 7f4dca59c700 1 -- 172.21.15.144:0/1768766181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dcc0ffba0 msgr2=0x7f4dcc198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:12.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.202+0000 7f4dca59c700 1 --2- 172.21.15.144:0/1768766181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dcc0ffba0 0x7f4dcc198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:12.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.202+0000 7f4dca59c700 1 -- 172.21.15.144:0/1768766181 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc00095f0 con 0x7f4dcc0ff290 2024-09-10T12:11:12.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.202+0000 7f4dca59c700 1 --2- 172.21.15.144:0/1768766181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dcc0ff290 0x7f4dcc197fd0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00d900 tx=0x7f4dbc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:12.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.203+0000 7f4dbb7fe700 1 -- 172.21.15.144:0/1768766181 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dbc00b920 con 0x7f4dcc0ff290 2024-09-10T12:11:12.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.203+0000 7f4dbb7fe700 1 -- 172.21.15.144:0/1768766181 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4dbc010460 con 0x7f4dcc0ff290 2024-09-10T12:11:12.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.203+0000 7f4dd099d700 1 -- 172.21.15.144:0/1768766181 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dcc19cc60 con 0x7f4dcc0ff290 2024-09-10T12:11:12.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.203+0000 7f4dbb7fe700 1 -- 172.21.15.144:0/1768766181 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4dbc00f590 con 0x7f4dcc0ff290 2024-09-10T12:11:12.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.203+0000 7f4dd099d700 1 -- 172.21.15.144:0/1768766181 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dcc19d1b0 con 0x7f4dcc0ff290 2024-09-10T12:11:12.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.204+0000 7f4dd099d700 1 -- 172.21.15.144:0/1768766181 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dcc10a170 con 0x7f4dcc0ff290 2024-09-10T12:11:12.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.204+0000 7f4dbb7fe700 1 -- 172.21.15.144:0/1768766181 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4dbc0105d0 con 0x7f4dcc0ff290 2024-09-10T12:11:12.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.205+0000 7f4dbb7fe700 1 --2- 172.21.15.144:0/1768766181 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4db406e200 0x7f4db40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:12.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.205+0000 7f4dbb7fe700 1 -- 172.21.15.144:0/1768766181 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4dbc08def0 con 0x7f4dcc0ff290 2024-09-10T12:11:12.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.205+0000 7f4dc9d9b700 1 --2- 172.21.15.144:0/1768766181 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4db406e200 0x7f4db40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:12.208 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.205+0000 7f4dc9d9b700 1 --2- 172.21.15.144:0/1768766181 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4db406e200 0x7f4db40706b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc1995f0 tx=0x7f4dc001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:12.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.209+0000 7f4dbb7fe700 1 -- 172.21.15.144:0/1768766181 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4dbc057350 con 0x7f4dcc0ff290 2024-09-10T12:11:12.458 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.456+0000 7f4dd099d700 1 -- 172.21.15.144:0/1768766181 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7f4dcc04ea50 con 0x7f4dcc0ff290 2024-09-10T12:11:12.458 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.457+0000 7f4dbb7fe700 1 -- 172.21.15.144:0/1768766181 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v42) v1 ==== 107+0+3343 (secure 0 0 0) 0x7f4dbc020d20 con 0x7f4dcc0ff290 2024-09-10T12:11:12.459 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:12.459 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":25,"btime":"2024-09-10T11:37:33:047944+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34378,"name":"cephfs.smithi144.bfyvwv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/4013687114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4013687114},{"type":"v1","addr":"172.21.15.144:6827","nonce":4013687114}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44315,"name":"cephfs.smithi178.czjrsa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6827/2970117696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2970117696},{"type":"v1","addr":"172.21.15.178:6827","nonce":2970117696}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:37:33.047941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:12.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.459+0000 7f4dd099d700 1 -- 172.21.15.144:0/1768766181 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4db406e200 msgr2=0x7f4db40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:12.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.459+0000 7f4dd099d700 1 --2- 172.21.15.144:0/1768766181 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4db406e200 0x7f4db40706b0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4dcc1995f0 tx=0x7f4dc001a040 comp rx=0 tx=0).stop 2024-09-10T12:11:12.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.459+0000 7f4dd099d700 1 -- 172.21.15.144:0/1768766181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dcc0ff290 msgr2=0x7f4dcc197fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:12.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.459+0000 7f4dd099d700 1 --2- 172.21.15.144:0/1768766181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dcc0ff290 0x7f4dcc197fd0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc00d900 tx=0x7f4dbc00dc10 comp rx=0 tx=0).stop 2024-09-10T12:11:12.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.459+0000 7f4dd099d700 1 -- 172.21.15.144:0/1768766181 shutdown_connections 2024-09-10T12:11:12.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.459+0000 7f4dd099d700 1 --2- 172.21.15.144:0/1768766181 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4db406e200 0x7f4db40706b0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:12.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.459+0000 7f4dd099d700 1 --2- 172.21.15.144:0/1768766181 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4dcc0ff290 0x7f4dcc197fd0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:12.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.459+0000 7f4dd099d700 1 --2- 172.21.15.144:0/1768766181 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4dcc0ffba0 0x7f4dcc198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:12.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.459+0000 7f4dd099d700 1 -- 172.21.15.144:0/1768766181 >> 172.21.15.144:0/1768766181 conn(0x7f4dcc074850 msgr2=0x7f4dcc100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:12.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.460+0000 7f4dd099d700 1 -- 172.21.15.144:0/1768766181 shutdown_connections 2024-09-10T12:11:12.462 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:12.460+0000 7f4dd099d700 1 -- 172.21.15.144:0/1768766181 wait complete. 2024-09-10T12:11:12.462 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 25 2024-09-10T12:11:13.066 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T12:11:13.067 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 25 2024-09-10T12:11:13.067 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 26 2024-09-10T12:11:13.272 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:13.335 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:13 smithi144 ceph-mon[87436]: pgmap v1179: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:13.335 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:13 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1768766181' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-09-10T12:11:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:13 smithi178 ceph-mon[65940]: pgmap v1179: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:13 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1768766181' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-09-10T12:11:14.320 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:11:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:14 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:11:14.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.924+0000 7f1aab714700 1 -- 172.21.15.144:0/1073973555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa41003e0 msgr2=0x7f1aa41007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:14.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.924+0000 7f1aab714700 1 --2- 172.21.15.144:0/1073973555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa41003e0 0x7f1aa41007b0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f1a9400b0a0 tx=0x7f1a9400b3b0 comp rx=0 tx=0).stop 2024-09-10T12:11:14.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.924+0000 7f1aab714700 1 -- 172.21.15.144:0/1073973555 shutdown_connections 2024-09-10T12:11:14.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.924+0000 7f1aab714700 1 --2- 172.21.15.144:0/1073973555 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aa4100d80 0x7f1aa4104d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:14.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.924+0000 7f1aab714700 1 --2- 172.21.15.144:0/1073973555 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa41003e0 0x7f1aa41007b0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:14.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.924+0000 7f1aab714700 1 -- 172.21.15.144:0/1073973555 >> 172.21.15.144:0/1073973555 conn(0x7f1aa40fbca0 msgr2=0x7f1aa40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:14.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.924+0000 7f1aab714700 1 -- 172.21.15.144:0/1073973555 shutdown_connections 2024-09-10T12:11:14.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.924+0000 7f1aab714700 1 -- 172.21.15.144:0/1073973555 wait complete. 2024-09-10T12:11:14.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.925+0000 7f1aab714700 1 Processor -- start 2024-09-10T12:11:14.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.925+0000 7f1aab714700 1 -- start start 2024-09-10T12:11:14.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.925+0000 7f1aab714700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa41003e0 0x7f1aa4195d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:14.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.925+0000 7f1aab714700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aa4100d80 0x7f1aa41962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:14.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.925+0000 7f1aab714700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa41969a0 con 0x7f1aa41003e0 2024-09-10T12:11:14.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.925+0000 7f1aab714700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aa419a730 con 0x7f1aa4100d80 2024-09-10T12:11:14.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.925+0000 7f1aa94b0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa41003e0 0x7f1aa4195d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:14.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.925+0000 7f1aa94b0700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa41003e0 0x7f1aa4195d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52670/0 (socket says 172.21.15.144:52670) 2024-09-10T12:11:14.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.925+0000 7f1aa94b0700 1 -- 172.21.15.144:0/3767701992 learned_addr learned my addr 172.21.15.144:0/3767701992 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:14.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.925+0000 7f1aa8caf700 1 --2- 172.21.15.144:0/3767701992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aa4100d80 0x7f1aa41962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:14.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.926+0000 7f1aa94b0700 1 -- 172.21.15.144:0/3767701992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aa4100d80 msgr2=0x7f1aa41962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:14.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.926+0000 7f1aa94b0700 1 --2- 172.21.15.144:0/3767701992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aa4100d80 0x7f1aa41962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:14.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.926+0000 7f1aa94b0700 1 -- 172.21.15.144:0/3767701992 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a94009d00 con 0x7f1aa41003e0 2024-09-10T12:11:14.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.926+0000 7f1aa94b0700 1 --2- 172.21.15.144:0/3767701992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa41003e0 0x7f1aa4195d80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1a94000f80 tx=0x7f1a9400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:14.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.926+0000 7f1aa27fc700 1 -- 172.21.15.144:0/3767701992 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a9400e040 con 0x7f1aa41003e0 2024-09-10T12:11:14.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.926+0000 7f1aa27fc700 1 -- 172.21.15.144:0/3767701992 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1a94003c10 con 0x7f1aa41003e0 2024-09-10T12:11:14.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.926+0000 7f1aa27fc700 1 -- 172.21.15.144:0/3767701992 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a9401d6b0 con 0x7f1aa41003e0 2024-09-10T12:11:14.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.926+0000 7f1aab714700 1 -- 172.21.15.144:0/3767701992 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aa419a9b0 con 0x7f1aa41003e0 2024-09-10T12:11:14.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.926+0000 7f1aab714700 1 -- 172.21.15.144:0/3767701992 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aa419aea0 con 0x7f1aa41003e0 2024-09-10T12:11:14.930 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.927+0000 7f1aab714700 1 -- 172.21.15.144:0/3767701992 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aa41086f0 con 0x7f1aa41003e0 2024-09-10T12:11:14.930 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.928+0000 7f1aa27fc700 1 -- 172.21.15.144:0/3767701992 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f1a94004040 con 0x7f1aa41003e0 2024-09-10T12:11:14.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.928+0000 7f1aa27fc700 1 --2- 172.21.15.144:0/3767701992 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1a8c06e200 0x7f1a8c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:14.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.928+0000 7f1aa27fc700 1 -- 172.21.15.144:0/3767701992 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f1a9408f790 con 0x7f1aa41003e0 2024-09-10T12:11:14.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.928+0000 7f1aa8caf700 1 --2- 172.21.15.144:0/3767701992 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1a8c06e200 0x7f1a8c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:14.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.928+0000 7f1aa8caf700 1 --2- 172.21.15.144:0/3767701992 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1a8c06e200 0x7f1a8c0706b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1aa41973a0 tx=0x7f1a9800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:14.934 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:14.932+0000 7f1aa27fc700 1 -- 172.21.15.144:0/3767701992 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f1a94058bf0 con 0x7f1aa41003e0 2024-09-10T12:11:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:15 smithi144 ceph-mon[87436]: pgmap v1180: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:11:15.188 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:15.187+0000 7f1aab714700 1 -- 172.21.15.144:0/3767701992 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f1aa404ea50 con 0x7f1aa41003e0 2024-09-10T12:11:15.189 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:15.188+0000 7f1aa27fc700 1 -- 172.21.15.144:0/3767701992 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v42) v1 ==== 107+0+4904 (secure 0 0 0) 0x7f1aa404ea50 con 0x7f1aa41003e0 2024-09-10T12:11:15.189 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:15.189 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":26,"btime":"2024-09-10T11:37:34:049972+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34378,"name":"cephfs.smithi144.bfyvwv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/4013687114","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":4013687114},{"type":"v1","addr":"172.21.15.144:6827","nonce":4013687114}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":34384,"name":"cephfs.smithi144.jaoyvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6829/1022937641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1022937641},{"type":"v1","addr":"172.21.15.144:6829","nonce":1022937641}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44315,"name":"cephfs.smithi178.czjrsa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6827/2970117696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2970117696},{"type":"v1","addr":"172.21.15.178:6827","nonce":2970117696}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44325,"name":"cephfs.smithi178.nmhrmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6825/559040332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":559040332},{"type":"v1","addr":"172.21.15.178:6825","nonce":559040332}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:37:33.047941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:15.191 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:15.190+0000 7f1aab714700 1 -- 172.21.15.144:0/3767701992 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1a8c06e200 msgr2=0x7f1a8c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:15.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:15.190+0000 7f1aab714700 1 --2- 172.21.15.144:0/3767701992 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1a8c06e200 0x7f1a8c0706b0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1aa41973a0 tx=0x7f1a9800b410 comp rx=0 tx=0).stop 2024-09-10T12:11:15.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:15.190+0000 7f1aab714700 1 -- 172.21.15.144:0/3767701992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa41003e0 msgr2=0x7f1aa4195d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:15.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:15.190+0000 7f1aab714700 1 --2- 172.21.15.144:0/3767701992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa41003e0 0x7f1aa4195d80 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f1a94000f80 tx=0x7f1a9400cd20 comp rx=0 tx=0).stop 2024-09-10T12:11:15.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:15.190+0000 7f1aab714700 1 -- 172.21.15.144:0/3767701992 shutdown_connections 2024-09-10T12:11:15.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:15.190+0000 7f1aab714700 1 --2- 172.21.15.144:0/3767701992 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f1a8c06e200 0x7f1a8c0706b0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:15.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:15.190+0000 7f1aab714700 1 --2- 172.21.15.144:0/3767701992 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f1aa41003e0 0x7f1aa4195d80 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:15.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:15.190+0000 7f1aab714700 1 --2- 172.21.15.144:0/3767701992 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f1aa4100d80 0x7f1aa41962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:15.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:15.190+0000 7f1aab714700 1 -- 172.21.15.144:0/3767701992 >> 172.21.15.144:0/3767701992 conn(0x7f1aa40fbca0 msgr2=0x7f1aa40fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:15.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:15.190+0000 7f1aab714700 1 -- 172.21.15.144:0/3767701992 shutdown_connections 2024-09-10T12:11:15.192 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:15.190+0000 7f1aab714700 1 -- 172.21.15.144:0/3767701992 wait complete. 2024-09-10T12:11:15.193 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 26 2024-09-10T12:11:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:15 smithi178 ceph-mon[65940]: pgmap v1180: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:11:15.872 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T12:11:15.873 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 26 2024-09-10T12:11:15.873 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 27 2024-09-10T12:11:16.078 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:16.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:16 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3767701992' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-09-10T12:11:16.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:16 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3767701992' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-09-10T12:11:17.125 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:17 smithi144 ceph-mon[87436]: pgmap v1181: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:17 smithi178 ceph-mon[65940]: pgmap v1181: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:17.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.612+0000 7fa9d3767700 1 -- 172.21.15.144:0/1645594080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc0ff270 msgr2=0x7fa9cc0ff640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:17.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.612+0000 7fa9d3767700 1 --2- 172.21.15.144:0/1645594080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc0ff270 0x7fa9cc0ff640 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc009910 tx=0x7fa9bc009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:17.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.612+0000 7fa9d3767700 1 -- 172.21.15.144:0/1645594080 shutdown_connections 2024-09-10T12:11:17.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.612+0000 7fa9d3767700 1 --2- 172.21.15.144:0/1645594080 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9cc0ffb80 0x7fa9cc10a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:17.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.612+0000 7fa9d3767700 1 --2- 172.21.15.144:0/1645594080 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc0ff270 0x7fa9cc0ff640 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:17.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.612+0000 7fa9d3767700 1 -- 172.21.15.144:0/1645594080 >> 172.21.15.144:0/1645594080 conn(0x7fa9cc074850 msgr2=0x7fa9cc074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:17.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.612+0000 7fa9d3767700 1 -- 172.21.15.144:0/1645594080 shutdown_connections 2024-09-10T12:11:17.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.612+0000 7fa9d3767700 1 -- 172.21.15.144:0/1645594080 wait complete. 2024-09-10T12:11:17.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.613+0000 7fa9d3767700 1 Processor -- start 2024-09-10T12:11:17.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.613+0000 7fa9d3767700 1 -- start start 2024-09-10T12:11:17.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.613+0000 7fa9d3767700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc0ff270 0x7fa9cc195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:17.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.613+0000 7fa9d3767700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9cc0ffb80 0x7fa9cc1961e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:17.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.613+0000 7fa9d3767700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc196870 con 0x7fa9cc0ff270 2024-09-10T12:11:17.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.613+0000 7fa9d3767700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9cc1995a0 con 0x7fa9cc0ffb80 2024-09-10T12:11:17.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.613+0000 7fa9d1503700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc0ff270 0x7fa9cc195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:17.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.613+0000 7fa9d1503700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc0ff270 0x7fa9cc195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52684/0 (socket says 172.21.15.144:52684) 2024-09-10T12:11:17.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.613+0000 7fa9d1503700 1 -- 172.21.15.144:0/3241473327 learned_addr learned my addr 172.21.15.144:0/3241473327 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:17.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.614+0000 7fa9d1503700 1 -- 172.21.15.144:0/3241473327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9cc0ffb80 msgr2=0x7fa9cc1961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:17.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.614+0000 7fa9d1503700 1 --2- 172.21.15.144:0/3241473327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9cc0ffb80 0x7fa9cc1961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:17.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.614+0000 7fa9d1503700 1 -- 172.21.15.144:0/3241473327 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9bc0095f0 con 0x7fa9cc0ff270 2024-09-10T12:11:17.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.614+0000 7fa9d1503700 1 --2- 172.21.15.144:0/3241473327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc0ff270 0x7fa9cc195ca0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00b5c0 tx=0x7fa9bc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.614+0000 7fa9c27fc700 1 -- 172.21.15.144:0/3241473327 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9bc01d070 con 0x7fa9cc0ff270 2024-09-10T12:11:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.614+0000 7fa9d3767700 1 -- 172.21.15.144:0/3241473327 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9cc199820 con 0x7fa9cc0ff270 2024-09-10T12:11:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.614+0000 7fa9c27fc700 1 -- 172.21.15.144:0/3241473327 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa9bc005e00 con 0x7fa9cc0ff270 2024-09-10T12:11:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.614+0000 7fa9c27fc700 1 -- 172.21.15.144:0/3241473327 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9bc017400 con 0x7fa9cc0ff270 2024-09-10T12:11:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.614+0000 7fa9d3767700 1 -- 172.21.15.144:0/3241473327 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9cc199d90 con 0x7fa9cc0ff270 2024-09-10T12:11:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.615+0000 7fa9d3767700 1 -- 172.21.15.144:0/3241473327 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9cc107cf0 con 0x7fa9cc0ff270 2024-09-10T12:11:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.616+0000 7fa9c27fc700 1 -- 172.21.15.144:0/3241473327 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa9bc00f460 con 0x7fa9cc0ff270 2024-09-10T12:11:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.616+0000 7fa9c27fc700 1 --2- 172.21.15.144:0/3241473327 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa9b806e200 0x7fa9b80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.616+0000 7fa9c27fc700 1 -- 172.21.15.144:0/3241473327 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa9bc08f650 con 0x7fa9cc0ff270 2024-09-10T12:11:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.616+0000 7fa9d0d02700 1 --2- 172.21.15.144:0/3241473327 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa9b806e200 0x7fa9b80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:17.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.616+0000 7fa9d0d02700 1 --2- 172.21.15.144:0/3241473327 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa9b806e200 0x7fa9b80706b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8005950 tx=0x7fa9c80058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:17.621 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.620+0000 7fa9c27fc700 1 -- 172.21.15.144:0/3241473327 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa9bc058990 con 0x7fa9cc0ff270 2024-09-10T12:11:17.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.867+0000 7fa9d3767700 1 -- 172.21.15.144:0/3241473327 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7fa9cc04ea50 con 0x7fa9cc0ff270 2024-09-10T12:11:17.869 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.868+0000 7fa9c27fc700 1 -- 172.21.15.144:0/3241473327 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v42) v1 ==== 107+0+4122 (secure 0 0 0) 0x7fa9bc05c050 con 0x7fa9cc0ff270 2024-09-10T12:11:17.870 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:17.870 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":27,"btime":"2024-09-10T11:37:37:063607+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34384,"name":"cephfs.smithi144.jaoyvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6829/1022937641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1022937641},{"type":"v1","addr":"172.21.15.144:6829","nonce":1022937641}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":44315,"name":"cephfs.smithi178.czjrsa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6827/2970117696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2970117696},{"type":"v1","addr":"172.21.15.178:6827","nonce":2970117696}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44325,"name":"cephfs.smithi178.nmhrmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6825/559040332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":559040332},{"type":"v1","addr":"172.21.15.178:6825","nonce":559040332}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:37:33.047941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:17.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.870+0000 7fa9d3767700 1 -- 172.21.15.144:0/3241473327 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa9b806e200 msgr2=0x7fa9b80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:17.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.870+0000 7fa9d3767700 1 --2- 172.21.15.144:0/3241473327 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa9b806e200 0x7fa9b80706b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fa9c8005950 tx=0x7fa9c80058e0 comp rx=0 tx=0).stop 2024-09-10T12:11:17.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.870+0000 7fa9d3767700 1 -- 172.21.15.144:0/3241473327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc0ff270 msgr2=0x7fa9cc195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:17.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.870+0000 7fa9d3767700 1 --2- 172.21.15.144:0/3241473327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc0ff270 0x7fa9cc195ca0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fa9bc00b5c0 tx=0x7fa9bc00be20 comp rx=0 tx=0).stop 2024-09-10T12:11:17.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.870+0000 7fa9d3767700 1 -- 172.21.15.144:0/3241473327 shutdown_connections 2024-09-10T12:11:17.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.870+0000 7fa9d3767700 1 --2- 172.21.15.144:0/3241473327 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa9b806e200 0x7fa9b80706b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:17.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.870+0000 7fa9d3767700 1 --2- 172.21.15.144:0/3241473327 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa9cc0ff270 0x7fa9cc195ca0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:17.872 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.870+0000 7fa9d3767700 1 --2- 172.21.15.144:0/3241473327 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa9cc0ffb80 0x7fa9cc1961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:17.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.870+0000 7fa9d3767700 1 -- 172.21.15.144:0/3241473327 >> 172.21.15.144:0/3241473327 conn(0x7fa9cc074850 msgr2=0x7fa9cc104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:17.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.870+0000 7fa9d3767700 1 -- 172.21.15.144:0/3241473327 shutdown_connections 2024-09-10T12:11:17.873 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:17.870+0000 7fa9d3767700 1 -- 172.21.15.144:0/3241473327 wait complete. 2024-09-10T12:11:17.873 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 27 2024-09-10T12:11:18.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:18 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3241473327' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-09-10T12:11:18.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:18 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3241473327' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-09-10T12:11:18.554 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T12:11:18.554 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 27 2024-09-10T12:11:18.554 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 28 2024-09-10T12:11:18.753 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:19 smithi144 ceph-mon[87436]: pgmap v1182: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:11:19.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:19 smithi178 ceph-mon[65940]: pgmap v1182: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:11:20.164 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:20 smithi144 ceph-mon[87436]: pgmap v1183: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:11:20.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:20 smithi178 ceph-mon[65940]: pgmap v1183: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:11:20.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.392+0000 7fd8f6ace700 1 -- 172.21.15.144:0/3781433571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8f00ffb80 msgr2=0x7fd8f010a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:20.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.392+0000 7fd8f6ace700 1 --2- 172.21.15.144:0/3781433571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8f00ffb80 0x7fd8f010a5f0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fd8dc009910 tx=0x7fd8dc009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:20.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.392+0000 7fd8f6ace700 1 -- 172.21.15.144:0/3781433571 shutdown_connections 2024-09-10T12:11:20.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.392+0000 7fd8f6ace700 1 --2- 172.21.15.144:0/3781433571 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8f00ffb80 0x7fd8f010a5f0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:20.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.392+0000 7fd8f6ace700 1 --2- 172.21.15.144:0/3781433571 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8f00ff270 0x7fd8f00ff640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:20.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.392+0000 7fd8f6ace700 1 -- 172.21.15.144:0/3781433571 >> 172.21.15.144:0/3781433571 conn(0x7fd8f0074850 msgr2=0x7fd8f0074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:20.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.392+0000 7fd8f6ace700 1 -- 172.21.15.144:0/3781433571 shutdown_connections 2024-09-10T12:11:20.394 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.392+0000 7fd8f6ace700 1 -- 172.21.15.144:0/3781433571 wait complete. 2024-09-10T12:11:20.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.393+0000 7fd8f6ace700 1 Processor -- start 2024-09-10T12:11:20.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.393+0000 7fd8f6ace700 1 -- start start 2024-09-10T12:11:20.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.393+0000 7fd8f6ace700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8f00ff270 0x7fd8f0195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:20.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.393+0000 7fd8f6ace700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8f00ffb80 0x7fd8f0196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:20.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.393+0000 7fd8f6ace700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f0196b00 con 0x7fd8f00ff270 2024-09-10T12:11:20.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.393+0000 7fd8f6ace700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8f019a890 con 0x7fd8f00ffb80 2024-09-10T12:11:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.393+0000 7fd8f486a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8f00ff270 0x7fd8f0195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.393+0000 7fd8f486a700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8f00ff270 0x7fd8f0195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:52694/0 (socket says 172.21.15.144:52694) 2024-09-10T12:11:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.393+0000 7fd8f486a700 1 -- 172.21.15.144:0/1090393611 learned_addr learned my addr 172.21.15.144:0/1090393611 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.393+0000 7fd8effff700 1 --2- 172.21.15.144:0/1090393611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8f00ffb80 0x7fd8f0196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.393+0000 7fd8f486a700 1 -- 172.21.15.144:0/1090393611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8f00ffb80 msgr2=0x7fd8f0196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.393+0000 7fd8f486a700 1 --2- 172.21.15.144:0/1090393611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8f00ffb80 0x7fd8f0196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.393+0000 7fd8f486a700 1 -- 172.21.15.144:0/1090393611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc0095f0 con 0x7fd8f00ff270 2024-09-10T12:11:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.394+0000 7fd8f486a700 1 --2- 172.21.15.144:0/1090393611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8f00ff270 0x7fd8f0195ee0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400b480 tx=0x7fd8e400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.394+0000 7fd8edffb700 1 -- 172.21.15.144:0/1090393611 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8e4011800 con 0x7fd8f00ff270 2024-09-10T12:11:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.394+0000 7fd8edffb700 1 -- 172.21.15.144:0/1090393611 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd8e4011e20 con 0x7fd8f00ff270 2024-09-10T12:11:20.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.394+0000 7fd8f6ace700 1 -- 172.21.15.144:0/1090393611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8f019ab70 con 0x7fd8f00ff270 2024-09-10T12:11:20.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.394+0000 7fd8edffb700 1 -- 172.21.15.144:0/1090393611 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8e4010ea0 con 0x7fd8f00ff270 2024-09-10T12:11:20.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.394+0000 7fd8f6ace700 1 -- 172.21.15.144:0/1090393611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8f019b140 con 0x7fd8f00ff270 2024-09-10T12:11:20.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.395+0000 7fd8f6ace700 1 -- 172.21.15.144:0/1090393611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8f0107cf0 con 0x7fd8f00ff270 2024-09-10T12:11:20.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.395+0000 7fd8edffb700 1 -- 172.21.15.144:0/1090393611 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd8e4010380 con 0x7fd8f00ff270 2024-09-10T12:11:20.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.396+0000 7fd8edffb700 1 --2- 172.21.15.144:0/1090393611 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd8e006e200 0x7fd8e00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:20.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.396+0000 7fd8edffb700 1 -- 172.21.15.144:0/1090393611 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd8e408db70 con 0x7fd8f00ff270 2024-09-10T12:11:20.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.396+0000 7fd8effff700 1 --2- 172.21.15.144:0/1090393611 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd8e006e200 0x7fd8e00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:20.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.396+0000 7fd8effff700 1 --2- 172.21.15.144:0/1090393611 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd8e006e200 0x7fd8e00706b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0197500 tx=0x7fd8dc00b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:20.401 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.400+0000 7fd8edffb700 1 -- 172.21.15.144:0/1090393611 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd8e4056f70 con 0x7fd8f00ff270 2024-09-10T12:11:20.643 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.642+0000 7fd8f6ace700 1 -- 172.21.15.144:0/1090393611 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7fd8f004ea50 con 0x7fd8f00ff270 2024-09-10T12:11:20.644 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.643+0000 7fd8edffb700 1 -- 172.21.15.144:0/1090393611 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v42) v1 ==== 107+0+4972 (secure 0 0 0) 0x7fd8e405a630 con 0x7fd8f00ff270 2024-09-10T12:11:20.644 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:20.644 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":28,"btime":"2024-09-10T11:37:42:715131+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34384,"name":"cephfs.smithi144.jaoyvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6829/1022937641","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":1022937641},{"type":"v1","addr":"172.21.15.144:6829","nonce":1022937641}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":34390,"name":"cephfs.smithi144.bfyvwv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/2724506882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2724506882},{"type":"v1","addr":"172.21.15.144:6827","nonce":2724506882}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44315,"name":"cephfs.smithi178.czjrsa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6827/2970117696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2970117696},{"type":"v1","addr":"172.21.15.178:6827","nonce":2970117696}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44325,"name":"cephfs.smithi178.nmhrmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6825/559040332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":559040332},{"type":"v1","addr":"172.21.15.178:6825","nonce":559040332}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:37:33.047941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:20.646 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.645+0000 7fd8f6ace700 1 -- 172.21.15.144:0/1090393611 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd8e006e200 msgr2=0x7fd8e00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:20.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.645+0000 7fd8f6ace700 1 --2- 172.21.15.144:0/1090393611 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd8e006e200 0x7fd8e00706b0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0197500 tx=0x7fd8dc00b540 comp rx=0 tx=0).stop 2024-09-10T12:11:20.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.645+0000 7fd8f6ace700 1 -- 172.21.15.144:0/1090393611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8f00ff270 msgr2=0x7fd8f0195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:20.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.645+0000 7fd8f6ace700 1 --2- 172.21.15.144:0/1090393611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8f00ff270 0x7fd8f0195ee0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd8e400b480 tx=0x7fd8e400b790 comp rx=0 tx=0).stop 2024-09-10T12:11:20.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.645+0000 7fd8f6ace700 1 -- 172.21.15.144:0/1090393611 shutdown_connections 2024-09-10T12:11:20.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.645+0000 7fd8f6ace700 1 --2- 172.21.15.144:0/1090393611 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd8e006e200 0x7fd8e00706b0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:20.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.645+0000 7fd8f6ace700 1 --2- 172.21.15.144:0/1090393611 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd8f00ff270 0x7fd8f0195ee0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:20.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.645+0000 7fd8f6ace700 1 --2- 172.21.15.144:0/1090393611 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd8f00ffb80 0x7fd8f0196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:20.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.645+0000 7fd8f6ace700 1 -- 172.21.15.144:0/1090393611 >> 172.21.15.144:0/1090393611 conn(0x7fd8f0074850 msgr2=0x7fd8f0104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:20.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.645+0000 7fd8f6ace700 1 -- 172.21.15.144:0/1090393611 shutdown_connections 2024-09-10T12:11:20.647 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:20.645+0000 7fd8f6ace700 1 -- 172.21.15.144:0/1090393611 wait complete. 2024-09-10T12:11:20.648 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 28 2024-09-10T12:11:21.313 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:21 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1090393611' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-09-10T12:11:21.333 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T12:11:21.333 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 28 2024-09-10T12:11:21.334 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 29 2024-09-10T12:11:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:21 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1090393611' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-09-10T12:11:21.534 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:22.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:22 smithi144 ceph-mon[87436]: pgmap v1184: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:22.345 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:22 smithi178 ceph-mon[65940]: pgmap v1184: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:23.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.091+0000 7f8eb0118700 1 -- 172.21.15.144:0/171406265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea8100d80 msgr2=0x7f8ea8104d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:23.092 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.091+0000 7f8eb0118700 1 --2- 172.21.15.144:0/171406265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea8100d80 0x7f8ea8104d50 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009910 tx=0x7f8e98009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:23.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.091+0000 7f8eb0118700 1 -- 172.21.15.144:0/171406265 shutdown_connections 2024-09-10T12:11:23.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.091+0000 7f8eb0118700 1 --2- 172.21.15.144:0/171406265 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea8100d80 0x7f8ea8104d50 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:23.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.091+0000 7f8eb0118700 1 --2- 172.21.15.144:0/171406265 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea81003e0 0x7f8ea81007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:23.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.091+0000 7f8eb0118700 1 -- 172.21.15.144:0/171406265 >> 172.21.15.144:0/171406265 conn(0x7f8ea80fbca0 msgr2=0x7f8ea80fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:23.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.091+0000 7f8eb0118700 1 -- 172.21.15.144:0/171406265 shutdown_connections 2024-09-10T12:11:23.093 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.091+0000 7f8eb0118700 1 -- 172.21.15.144:0/171406265 wait complete. 2024-09-10T12:11:23.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.092+0000 7f8eb0118700 1 Processor -- start 2024-09-10T12:11:23.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.092+0000 7f8eb0118700 1 -- start start 2024-09-10T12:11:23.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.092+0000 7f8eb0118700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea81003e0 0x7f8ea8195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:23.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.092+0000 7f8eb0118700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea8100d80 0x7f8ea8196270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:23.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.092+0000 7f8eb0118700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea8196950 con 0x7f8ea81003e0 2024-09-10T12:11:23.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.092+0000 7f8eb0118700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea819a6e0 con 0x7f8ea8100d80 2024-09-10T12:11:23.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.092+0000 7f8eadeb4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea81003e0 0x7f8ea8195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:23.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.092+0000 7f8eadeb4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea81003e0 0x7f8ea8195d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50004/0 (socket says 172.21.15.144:50004) 2024-09-10T12:11:23.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.092+0000 7f8eadeb4700 1 -- 172.21.15.144:0/3393800710 learned_addr learned my addr 172.21.15.144:0/3393800710 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:23.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.092+0000 7f8ead6b3700 1 --2- 172.21.15.144:0/3393800710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea8100d80 0x7f8ea8196270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:23.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.092+0000 7f8eadeb4700 1 -- 172.21.15.144:0/3393800710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea8100d80 msgr2=0x7f8ea8196270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:23.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.092+0000 7f8eadeb4700 1 --2- 172.21.15.144:0/3393800710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea8100d80 0x7f8ea8196270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:23.095 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.092+0000 7f8eadeb4700 1 -- 172.21.15.144:0/3393800710 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e980095f0 con 0x7f8ea81003e0 2024-09-10T12:11:23.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.092+0000 7f8eadeb4700 1 --2- 172.21.15.144:0/3393800710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea81003e0 0x7f8ea8195d30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40060b0 tx=0x7f8ea400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:23.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.093+0000 7f8e9effd700 1 -- 172.21.15.144:0/3393800710 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ea400dda0 con 0x7f8ea81003e0 2024-09-10T12:11:23.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.093+0000 7f8e9effd700 1 -- 172.21.15.144:0/3393800710 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8ea400f040 con 0x7f8ea81003e0 2024-09-10T12:11:23.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.093+0000 7f8eb0118700 1 -- 172.21.15.144:0/3393800710 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea819a9c0 con 0x7f8ea81003e0 2024-09-10T12:11:23.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.093+0000 7f8e9effd700 1 -- 172.21.15.144:0/3393800710 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ea40146f0 con 0x7f8ea81003e0 2024-09-10T12:11:23.096 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.093+0000 7f8eb0118700 1 -- 172.21.15.144:0/3393800710 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea819aee0 con 0x7f8ea81003e0 2024-09-10T12:11:23.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.094+0000 7f8eb0118700 1 -- 172.21.15.144:0/3393800710 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea8108750 con 0x7f8ea81003e0 2024-09-10T12:11:23.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.094+0000 7f8e9effd700 1 -- 172.21.15.144:0/3393800710 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f8ea4015400 con 0x7f8ea81003e0 2024-09-10T12:11:23.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.095+0000 7f8e9effd700 1 --2- 172.21.15.144:0/3393800710 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8e9406e200 0x7f8e940706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:23.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.095+0000 7f8e9effd700 1 -- 172.21.15.144:0/3393800710 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f8ea408da20 con 0x7f8ea81003e0 2024-09-10T12:11:23.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.095+0000 7f8ead6b3700 1 --2- 172.21.15.144:0/3393800710 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8e9406e200 0x7f8e940706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:23.097 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.095+0000 7f8ead6b3700 1 --2- 172.21.15.144:0/3393800710 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8e9406e200 0x7f8e940706b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8e98006010 tx=0x7f8e9801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:23.100 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.099+0000 7f8e9effd700 1 -- 172.21.15.144:0/3393800710 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f8ea4056eb0 con 0x7f8ea81003e0 2024-09-10T12:11:23.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.346+0000 7f8eb0118700 1 -- 172.21.15.144:0/3393800710 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f8ea819b280 con 0x7f8ea81003e0 2024-09-10T12:11:23.348 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.347+0000 7f8e9effd700 1 -- 172.21.15.144:0/3393800710 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v42) v1 ==== 107+0+4190 (secure 0 0 0) 0x7f8ea405a570 con 0x7f8ea81003e0 2024-09-10T12:11:23.349 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:23.349 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":29,"btime":"2024-09-10T11:37:54:629649+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34390,"name":"cephfs.smithi144.bfyvwv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/2724506882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2724506882},{"type":"v1","addr":"172.21.15.144:6827","nonce":2724506882}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":44315,"name":"cephfs.smithi178.czjrsa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6827/2970117696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2970117696},{"type":"v1","addr":"172.21.15.178:6827","nonce":2970117696}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44325,"name":"cephfs.smithi178.nmhrmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6825/559040332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":559040332},{"type":"v1","addr":"172.21.15.178:6825","nonce":559040332}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:37:33.047941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:23.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.349+0000 7f8eb0118700 1 -- 172.21.15.144:0/3393800710 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8e9406e200 msgr2=0x7f8e940706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:23.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.349+0000 7f8eb0118700 1 --2- 172.21.15.144:0/3393800710 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8e9406e200 0x7f8e940706b0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f8e98006010 tx=0x7f8e9801a040 comp rx=0 tx=0).stop 2024-09-10T12:11:23.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.349+0000 7f8eb0118700 1 -- 172.21.15.144:0/3393800710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea81003e0 msgr2=0x7f8ea8195d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:23.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.349+0000 7f8eb0118700 1 --2- 172.21.15.144:0/3393800710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea81003e0 0x7f8ea8195d30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f8ea40060b0 tx=0x7f8ea400d3b0 comp rx=0 tx=0).stop 2024-09-10T12:11:23.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.349+0000 7f8eb0118700 1 -- 172.21.15.144:0/3393800710 shutdown_connections 2024-09-10T12:11:23.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.349+0000 7f8eb0118700 1 --2- 172.21.15.144:0/3393800710 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f8e9406e200 0x7f8e940706b0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:23.351 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.349+0000 7f8eb0118700 1 --2- 172.21.15.144:0/3393800710 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f8ea81003e0 0x7f8ea8195d30 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:23.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.349+0000 7f8eb0118700 1 --2- 172.21.15.144:0/3393800710 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f8ea8100d80 0x7f8ea8196270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:23.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.349+0000 7f8eb0118700 1 -- 172.21.15.144:0/3393800710 >> 172.21.15.144:0/3393800710 conn(0x7f8ea80fbca0 msgr2=0x7f8ea80fd250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:23.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.350+0000 7f8eb0118700 1 -- 172.21.15.144:0/3393800710 shutdown_connections 2024-09-10T12:11:23.352 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:23.350+0000 7f8eb0118700 1 -- 172.21.15.144:0/3393800710 wait complete. 2024-09-10T12:11:23.352 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 29 2024-09-10T12:11:23.644 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:23 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3393800710' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-09-10T12:11:23.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:23 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3393800710' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-09-10T12:11:23.975 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T12:11:23.975 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 29 2024-09-10T12:11:23.975 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 30 2024-09-10T12:11:24.177 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:24.630 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:24 smithi144 ceph-mon[87436]: pgmap v1185: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:11:24.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:24 smithi178 ceph-mon[65940]: pgmap v1185: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:11:25.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.773+0000 7f9505ef4700 1 -- 172.21.15.144:0/2614178580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9500073070 msgr2=0x7f9500073440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:25.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.773+0000 7f9505ef4700 1 --2- 172.21.15.144:0/2614178580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9500073070 0x7f9500073440 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f94f000b0d0 tx=0x7f94f000b3e0 comp rx=0 tx=0).stop 2024-09-10T12:11:25.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.773+0000 7f9505ef4700 1 -- 172.21.15.144:0/2614178580 shutdown_connections 2024-09-10T12:11:25.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.773+0000 7f9505ef4700 1 --2- 172.21.15.144:0/2614178580 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9500073980 0x7f950010c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:25.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.773+0000 7f9505ef4700 1 --2- 172.21.15.144:0/2614178580 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9500073070 0x7f9500073440 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:25.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.773+0000 7f9505ef4700 1 -- 172.21.15.144:0/2614178580 >> 172.21.15.144:0/2614178580 conn(0x7f95000fbca0 msgr2=0x7f95000fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:25.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.773+0000 7f9505ef4700 1 -- 172.21.15.144:0/2614178580 shutdown_connections 2024-09-10T12:11:25.775 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.773+0000 7f9505ef4700 1 -- 172.21.15.144:0/2614178580 wait complete. 2024-09-10T12:11:25.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.774+0000 7f9505ef4700 1 Processor -- start 2024-09-10T12:11:25.776 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.774+0000 7f9505ef4700 1 -- start start 2024-09-10T12:11:25.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.774+0000 7f9505ef4700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9500073070 0x7f9500198010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:25.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.774+0000 7f9505ef4700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9500073980 0x7f9500198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:25.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.774+0000 7f9505ef4700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9500198c30 con 0x7f9500073070 2024-09-10T12:11:25.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.774+0000 7f9505ef4700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f950019c9c0 con 0x7f9500073980 2024-09-10T12:11:25.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.774+0000 7f94ff7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9500073070 0x7f9500198010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:25.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.774+0000 7f94ff7fe700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9500073070 0x7f9500198010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50036/0 (socket says 172.21.15.144:50036) 2024-09-10T12:11:25.777 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.774+0000 7f94ff7fe700 1 -- 172.21.15.144:0/4041939216 learned_addr learned my addr 172.21.15.144:0/4041939216 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:25.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.774+0000 7f94ff7fe700 1 -- 172.21.15.144:0/4041939216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9500073980 msgr2=0x7f9500198550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:25.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.774+0000 7f94ff7fe700 1 --2- 172.21.15.144:0/4041939216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9500073980 0x7f9500198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:25.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.774+0000 7f94ff7fe700 1 -- 172.21.15.144:0/4041939216 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f0009d00 con 0x7f9500073070 2024-09-10T12:11:25.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.775+0000 7f94ff7fe700 1 --2- 172.21.15.144:0/4041939216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9500073070 0x7f9500198010 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f94f000b0a0 tx=0x7f94f000cdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:25.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.775+0000 7f94fd7fa700 1 -- 172.21.15.144:0/4041939216 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94f000e040 con 0x7f9500073070 2024-09-10T12:11:25.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.775+0000 7f9505ef4700 1 -- 172.21.15.144:0/4041939216 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f950019cc40 con 0x7f9500073070 2024-09-10T12:11:25.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.775+0000 7f94fd7fa700 1 -- 172.21.15.144:0/4041939216 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f94f0004500 con 0x7f9500073070 2024-09-10T12:11:25.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.775+0000 7f94fd7fa700 1 -- 172.21.15.144:0/4041939216 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94f001daf0 con 0x7f9500073070 2024-09-10T12:11:25.778 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.775+0000 7f9505ef4700 1 -- 172.21.15.144:0/4041939216 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f950019d130 con 0x7f9500073070 2024-09-10T12:11:25.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.776+0000 7f9505ef4700 1 -- 172.21.15.144:0/4041939216 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9500109c00 con 0x7f9500073070 2024-09-10T12:11:25.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.777+0000 7f94fd7fa700 1 -- 172.21.15.144:0/4041939216 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f94f001dc50 con 0x7f9500073070 2024-09-10T12:11:25.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.777+0000 7f94fd7fa700 1 --2- 172.21.15.144:0/4041939216 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f94ec06e200 0x7f94ec0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:25.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.777+0000 7f94fd7fa700 1 -- 172.21.15.144:0/4041939216 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f94f008fae0 con 0x7f9500073070 2024-09-10T12:11:25.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.777+0000 7f94f6ffd700 1 --2- 172.21.15.144:0/4041939216 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f94ec06e200 0x7f94ec0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:25.779 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.778+0000 7f94f6ffd700 1 --2- 172.21.15.144:0/4041939216 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f94ec06e200 0x7f94ec0706b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9500199630 tx=0x7f94e8006d20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:25.783 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:25.782+0000 7f94fd7fa700 1 -- 172.21.15.144:0/4041939216 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f94f0058f40 con 0x7f9500073070 2024-09-10T12:11:26.028 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:26.027+0000 7f9505ef4700 1 -- 172.21.15.144:0/4041939216 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f950002cd20 con 0x7f9500073070 2024-09-10T12:11:26.029 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:26.027+0000 7f94fd7fa700 1 -- 172.21.15.144:0/4041939216 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v42) v1 ==== 107+0+5040 (secure 0 0 0) 0x7f94f005c600 con 0x7f9500073070 2024-09-10T12:11:26.029 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:26.029 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":30,"btime":"2024-09-10T11:37:59:643990+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34390,"name":"cephfs.smithi144.bfyvwv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/2724506882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2724506882},{"type":"v1","addr":"172.21.15.144:6827","nonce":2724506882}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":34422,"name":"cephfs.smithi144.jaoyvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6829/4254390011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4254390011},{"type":"v1","addr":"172.21.15.144:6829","nonce":4254390011}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44315,"name":"cephfs.smithi178.czjrsa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6827/2970117696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2970117696},{"type":"v1","addr":"172.21.15.178:6827","nonce":2970117696}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44325,"name":"cephfs.smithi178.nmhrmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6825/559040332","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":559040332},{"type":"v1","addr":"172.21.15.178:6825","nonce":559040332}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:37:33.047941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:26.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:26.030+0000 7f9505ef4700 1 -- 172.21.15.144:0/4041939216 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f94ec06e200 msgr2=0x7f94ec0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:26.031 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:26.030+0000 7f9505ef4700 1 --2- 172.21.15.144:0/4041939216 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f94ec06e200 0x7f94ec0706b0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9500199630 tx=0x7f94e8006d20 comp rx=0 tx=0).stop 2024-09-10T12:11:26.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:26.030+0000 7f9505ef4700 1 -- 172.21.15.144:0/4041939216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9500073070 msgr2=0x7f9500198010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:26.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:26.030+0000 7f9505ef4700 1 --2- 172.21.15.144:0/4041939216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9500073070 0x7f9500198010 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f94f000b0a0 tx=0x7f94f000cdc0 comp rx=0 tx=0).stop 2024-09-10T12:11:26.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:26.030+0000 7f9505ef4700 1 -- 172.21.15.144:0/4041939216 shutdown_connections 2024-09-10T12:11:26.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:26.030+0000 7f9505ef4700 1 --2- 172.21.15.144:0/4041939216 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f94ec06e200 0x7f94ec0706b0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:26.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:26.030+0000 7f9505ef4700 1 --2- 172.21.15.144:0/4041939216 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9500073070 0x7f9500198010 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:26.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:26.030+0000 7f9505ef4700 1 --2- 172.21.15.144:0/4041939216 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9500073980 0x7f9500198550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:26.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:26.030+0000 7f9505ef4700 1 -- 172.21.15.144:0/4041939216 >> 172.21.15.144:0/4041939216 conn(0x7f95000fbca0 msgr2=0x7f9500106d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:26.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:26.030+0000 7f9505ef4700 1 -- 172.21.15.144:0/4041939216 shutdown_connections 2024-09-10T12:11:26.032 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:26.030+0000 7f9505ef4700 1 -- 172.21.15.144:0/4041939216 wait complete. 2024-09-10T12:11:26.033 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 30 2024-09-10T12:11:26.666 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:26 smithi144 ceph-mon[87436]: pgmap v1186: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:26.666 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:26 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/4041939216' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-09-10T12:11:26.687 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T12:11:26.687 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 30 2024-09-10T12:11:26.687 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 31 2024-09-10T12:11:26.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:26 smithi178 ceph-mon[65940]: pgmap v1186: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:26.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:26 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/4041939216' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-09-10T12:11:26.887 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:28.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.486+0000 7f7480f71700 1 -- 172.21.15.144:0/1600225853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c101cb0 msgr2=0x7f747c10a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:28.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.486+0000 7f7480f71700 1 --2- 172.21.15.144:0/1600225853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c101cb0 0x7f747c10a1b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f7464009940 tx=0x7f7464009c50 comp rx=0 tx=0).stop 2024-09-10T12:11:28.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.486+0000 7f7480f71700 1 -- 172.21.15.144:0/1600225853 shutdown_connections 2024-09-10T12:11:28.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.486+0000 7f7480f71700 1 --2- 172.21.15.144:0/1600225853 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c101cb0 0x7f747c10a1b0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:28.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.486+0000 7f7480f71700 1 --2- 172.21.15.144:0/1600225853 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f747c1013a0 0x7f747c101770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:28.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.486+0000 7f7480f71700 1 -- 172.21.15.144:0/1600225853 >> 172.21.15.144:0/1600225853 conn(0x7f747c0fabe0 msgr2=0x7f747c0fcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:28.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.486+0000 7f7480f71700 1 -- 172.21.15.144:0/1600225853 shutdown_connections 2024-09-10T12:11:28.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.486+0000 7f7480f71700 1 -- 172.21.15.144:0/1600225853 wait complete. 2024-09-10T12:11:28.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.487+0000 7f7480f71700 1 Processor -- start 2024-09-10T12:11:28.490 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.487+0000 7f7480f71700 1 -- start start 2024-09-10T12:11:28.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.487+0000 7f7480f71700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f747c1013a0 0x7f747c19bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:28.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.487+0000 7f7480f71700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c101cb0 0x7f747c19bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:28.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.487+0000 7f7480f71700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c19c5f0 con 0x7f747c101cb0 2024-09-10T12:11:28.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.487+0000 7f7480f71700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f747c195b30 con 0x7f747c1013a0 2024-09-10T12:11:28.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.487+0000 7f7479d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c101cb0 0x7f747c19bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:28.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.487+0000 7f7479d9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c101cb0 0x7f747c19bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50048/0 (socket says 172.21.15.144:50048) 2024-09-10T12:11:28.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.487+0000 7f7479d9b700 1 -- 172.21.15.144:0/2091801368 learned_addr learned my addr 172.21.15.144:0/2091801368 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:28.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.488+0000 7f747a59c700 1 --2- 172.21.15.144:0/2091801368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f747c1013a0 0x7f747c19bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:28.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.488+0000 7f7479d9b700 1 -- 172.21.15.144:0/2091801368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f747c1013a0 msgr2=0x7f747c19bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:28.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.488+0000 7f7479d9b700 1 --2- 172.21.15.144:0/2091801368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f747c1013a0 0x7f747c19bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:28.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.488+0000 7f7479d9b700 1 -- 172.21.15.144:0/2091801368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74640095f0 con 0x7f747c101cb0 2024-09-10T12:11:28.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.488+0000 7f7479d9b700 1 --2- 172.21.15.144:0/2091801368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c101cb0 0x7f747c19bff0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f74640055f0 tx=0x7f74640056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:28.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.488+0000 7f74737fe700 1 -- 172.21.15.144:0/2091801368 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f746401d070 con 0x7f747c101cb0 2024-09-10T12:11:28.491 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.488+0000 7f7480f71700 1 -- 172.21.15.144:0/2091801368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f747c195db0 con 0x7f747c101cb0 2024-09-10T12:11:28.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.488+0000 7f7480f71700 1 -- 172.21.15.144:0/2091801368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f747c1962a0 con 0x7f747c101cb0 2024-09-10T12:11:28.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.488+0000 7f74737fe700 1 -- 172.21.15.144:0/2091801368 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f746400f460 con 0x7f747c101cb0 2024-09-10T12:11:28.492 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.488+0000 7f74737fe700 1 -- 172.21.15.144:0/2091801368 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7464021640 con 0x7f747c101cb0 2024-09-10T12:11:28.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.489+0000 7f7480f71700 1 -- 172.21.15.144:0/2091801368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f745c005320 con 0x7f747c101cb0 2024-09-10T12:11:28.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.490+0000 7f74737fe700 1 -- 172.21.15.144:0/2091801368 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7464003a60 con 0x7f747c101cb0 2024-09-10T12:11:28.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.490+0000 7f74737fe700 1 --2- 172.21.15.144:0/2091801368 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f746806e200 0x7f74680706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:28.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.490+0000 7f74737fe700 1 -- 172.21.15.144:0/2091801368 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f746408f320 con 0x7f747c101cb0 2024-09-10T12:11:28.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.490+0000 7f747a59c700 1 --2- 172.21.15.144:0/2091801368 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f746806e200 0x7f74680706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:28.493 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.491+0000 7f747a59c700 1 --2- 172.21.15.144:0/2091801368 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f746806e200 0x7f74680706b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f747400d260 tx=0x7f7474009040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:28.496 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.495+0000 7f74737fe700 1 -- 172.21.15.144:0/2091801368 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f746405c230 con 0x7f747c101cb0 2024-09-10T12:11:28.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:28 smithi144 ceph-mon[87436]: pgmap v1187: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:11:28.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.739+0000 7f7480f71700 1 -- 172.21.15.144:0/2091801368 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f745c005190 con 0x7f747c101cb0 2024-09-10T12:11:28.741 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.740+0000 7f74737fe700 1 -- 172.21.15.144:0/2091801368 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v42) v1 ==== 107+0+4261 (secure 0 0 0) 0x7f746405bdc0 con 0x7f747c101cb0 2024-09-10T12:11:28.742 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:28.742 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":31,"btime":"2024-09-10T11:38:11:461953+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34390,"name":"cephfs.smithi144.bfyvwv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/2724506882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2724506882},{"type":"v1","addr":"172.21.15.144:6827","nonce":2724506882}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":34422,"name":"cephfs.smithi144.jaoyvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6829/4254390011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4254390011},{"type":"v1","addr":"172.21.15.144:6829","nonce":4254390011}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44315,"name":"cephfs.smithi178.czjrsa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6827/2970117696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2970117696},{"type":"v1","addr":"172.21.15.178:6827","nonce":2970117696}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:37:33.047941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:28.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.743+0000 7f7480f71700 1 -- 172.21.15.144:0/2091801368 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f746806e200 msgr2=0x7f74680706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:28.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.743+0000 7f7480f71700 1 --2- 172.21.15.144:0/2091801368 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f746806e200 0x7f74680706b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f747400d260 tx=0x7f7474009040 comp rx=0 tx=0).stop 2024-09-10T12:11:28.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.743+0000 7f7480f71700 1 -- 172.21.15.144:0/2091801368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c101cb0 msgr2=0x7f747c19bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:28.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.743+0000 7f7480f71700 1 --2- 172.21.15.144:0/2091801368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c101cb0 0x7f747c19bff0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f74640055f0 tx=0x7f74640056d0 comp rx=0 tx=0).stop 2024-09-10T12:11:28.744 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.743+0000 7f7480f71700 1 -- 172.21.15.144:0/2091801368 shutdown_connections 2024-09-10T12:11:28.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.743+0000 7f7480f71700 1 --2- 172.21.15.144:0/2091801368 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f746806e200 0x7f74680706b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:28.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.743+0000 7f7480f71700 1 --2- 172.21.15.144:0/2091801368 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f747c1013a0 0x7f747c19bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:28.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.743+0000 7f7480f71700 1 --2- 172.21.15.144:0/2091801368 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f747c101cb0 0x7f747c19bff0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:28.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.743+0000 7f7480f71700 1 -- 172.21.15.144:0/2091801368 >> 172.21.15.144:0/2091801368 conn(0x7f747c0fabe0 msgr2=0x7f747c104960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:28.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.743+0000 7f7480f71700 1 -- 172.21.15.144:0/2091801368 shutdown_connections 2024-09-10T12:11:28.745 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:28.743+0000 7f7480f71700 1 -- 172.21.15.144:0/2091801368 wait complete. 2024-09-10T12:11:28.745 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 31 2024-09-10T12:11:28.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:28 smithi178 ceph-mon[65940]: pgmap v1187: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:11:29.351 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T12:11:29.352 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 31 2024-09-10T12:11:29.352 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 32 2024-09-10T12:11:29.552 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:29.673 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:29 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:11:29.673 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:29 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2091801368' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-09-10T12:11:29.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:29 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:11:29.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:29 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2091801368' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-09-10T12:11:30.662 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:30 smithi144 ceph-mon[87436]: pgmap v1188: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:11:30.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:30 smithi178 ceph-mon[65940]: pgmap v1188: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:11:31.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.175+0000 7f33793a7700 1 -- 172.21.15.144:0/1028618527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33741004a0 msgr2=0x7f3374100870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:31.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.175+0000 7f33793a7700 1 --2- 172.21.15.144:0/1028618527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33741004a0 0x7f3374100870 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f335c009910 tx=0x7f335c009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:31.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.175+0000 7f33793a7700 1 -- 172.21.15.144:0/1028618527 shutdown_connections 2024-09-10T12:11:31.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.175+0000 7f33793a7700 1 --2- 172.21.15.144:0/1028618527 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3374100e40 0x7f3374104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:31.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.175+0000 7f33793a7700 1 --2- 172.21.15.144:0/1028618527 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33741004a0 0x7f3374100870 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:31.177 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.175+0000 7f33793a7700 1 -- 172.21.15.144:0/1028618527 >> 172.21.15.144:0/1028618527 conn(0x7f33740fbd60 msgr2=0x7f33740fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:31.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.175+0000 7f33793a7700 1 -- 172.21.15.144:0/1028618527 shutdown_connections 2024-09-10T12:11:31.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.175+0000 7f33793a7700 1 -- 172.21.15.144:0/1028618527 wait complete. 2024-09-10T12:11:31.178 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.176+0000 7f33793a7700 1 Processor -- start 2024-09-10T12:11:31.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.176+0000 7f33793a7700 1 -- start start 2024-09-10T12:11:31.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.176+0000 7f33793a7700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33741004a0 0x7f3374195de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:31.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.176+0000 7f33793a7700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3374100e40 0x7f3374196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:31.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.176+0000 7f33793a7700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3374196a00 con 0x7f33741004a0 2024-09-10T12:11:31.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.176+0000 7f33793a7700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f337419a790 con 0x7f3374100e40 2024-09-10T12:11:31.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.176+0000 7f3372ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33741004a0 0x7f3374195de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:31.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.176+0000 7f3372ffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33741004a0 0x7f3374195de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:50068/0 (socket says 172.21.15.144:50068) 2024-09-10T12:11:31.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.176+0000 7f3372ffd700 1 -- 172.21.15.144:0/2572592948 learned_addr learned my addr 172.21.15.144:0/2572592948 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:31.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.176+0000 7f33727fc700 1 --2- 172.21.15.144:0/2572592948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3374100e40 0x7f3374196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:31.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.177+0000 7f3372ffd700 1 -- 172.21.15.144:0/2572592948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3374100e40 msgr2=0x7f3374196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:31.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.177+0000 7f3372ffd700 1 --2- 172.21.15.144:0/2572592948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3374100e40 0x7f3374196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:31.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.177+0000 7f3372ffd700 1 -- 172.21.15.144:0/2572592948 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f335c0095f0 con 0x7f33741004a0 2024-09-10T12:11:31.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.177+0000 7f3372ffd700 1 --2- 172.21.15.144:0/2572592948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33741004a0 0x7f3374195de0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f335c0092e0 tx=0x7f335c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:31.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.177+0000 7f336bfff700 1 -- 172.21.15.144:0/2572592948 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f335c01d070 con 0x7f33741004a0 2024-09-10T12:11:31.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.177+0000 7f336bfff700 1 -- 172.21.15.144:0/2572592948 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f335c005e00 con 0x7f33741004a0 2024-09-10T12:11:31.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.177+0000 7f33793a7700 1 -- 172.21.15.144:0/2572592948 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f337419aa10 con 0x7f33741004a0 2024-09-10T12:11:31.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.177+0000 7f336bfff700 1 -- 172.21.15.144:0/2572592948 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f335c017400 con 0x7f33741004a0 2024-09-10T12:11:31.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.177+0000 7f33793a7700 1 -- 172.21.15.144:0/2572592948 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f337419af00 con 0x7f33741004a0 2024-09-10T12:11:31.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.178+0000 7f33793a7700 1 -- 172.21.15.144:0/2572592948 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33741087b0 con 0x7f33741004a0 2024-09-10T12:11:31.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.179+0000 7f336bfff700 1 -- 172.21.15.144:0/2572592948 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f335c00f6f0 con 0x7f33741004a0 2024-09-10T12:11:31.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.179+0000 7f336bfff700 1 --2- 172.21.15.144:0/2572592948 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f336006e250 0x7f3360070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:31.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.179+0000 7f33727fc700 1 --2- 172.21.15.144:0/2572592948 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f336006e250 0x7f3360070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:31.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.179+0000 7f336bfff700 1 -- 172.21.15.144:0/2572592948 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f335c08f520 con 0x7f33741004a0 2024-09-10T12:11:31.181 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.180+0000 7f33727fc700 1 --2- 172.21.15.144:0/2572592948 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f336006e250 0x7f3360070700 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3374197400 tx=0x7f336400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:31.185 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.183+0000 7f336bfff700 1 -- 172.21.15.144:0/2572592948 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f335c0588e0 con 0x7f33741004a0 2024-09-10T12:11:31.448 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.447+0000 7f33793a7700 1 -- 172.21.15.144:0/2572592948 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f337402cc60 con 0x7f33741004a0 2024-09-10T12:11:31.450 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.449+0000 7f336bfff700 1 -- 172.21.15.144:0/2572592948 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v42) v1 ==== 107+0+5108 (secure 0 0 0) 0x7f335c05bfa0 con 0x7f33741004a0 2024-09-10T12:11:31.451 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:31.451 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":32,"btime":"2024-09-10T11:38:14:670931+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34390,"name":"cephfs.smithi144.bfyvwv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/2724506882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2724506882},{"type":"v1","addr":"172.21.15.144:6827","nonce":2724506882}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":34422,"name":"cephfs.smithi144.jaoyvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6829/4254390011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4254390011},{"type":"v1","addr":"172.21.15.144:6829","nonce":4254390011}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44315,"name":"cephfs.smithi178.czjrsa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6827/2970117696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2970117696},{"type":"v1","addr":"172.21.15.178:6827","nonce":2970117696}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25},{"gid":44355,"name":"cephfs.smithi178.nmhrmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6825/706731549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":706731549},{"type":"v1","addr":"172.21.15.178:6825","nonce":706731549}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:37:33.047941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:31.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.452+0000 7f33793a7700 1 -- 172.21.15.144:0/2572592948 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f336006e250 msgr2=0x7f3360070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:31.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.452+0000 7f33793a7700 1 --2- 172.21.15.144:0/2572592948 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f336006e250 0x7f3360070700 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3374197400 tx=0x7f336400b410 comp rx=0 tx=0).stop 2024-09-10T12:11:31.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.452+0000 7f33793a7700 1 -- 172.21.15.144:0/2572592948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33741004a0 msgr2=0x7f3374195de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:31.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.452+0000 7f33793a7700 1 --2- 172.21.15.144:0/2572592948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33741004a0 0x7f3374195de0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f335c0092e0 tx=0x7f335c00bd70 comp rx=0 tx=0).stop 2024-09-10T12:11:31.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.452+0000 7f33793a7700 1 -- 172.21.15.144:0/2572592948 shutdown_connections 2024-09-10T12:11:31.453 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.452+0000 7f33793a7700 1 --2- 172.21.15.144:0/2572592948 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f336006e250 0x7f3360070700 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:31.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.452+0000 7f33793a7700 1 --2- 172.21.15.144:0/2572592948 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f33741004a0 0x7f3374195de0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:31.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.452+0000 7f33793a7700 1 --2- 172.21.15.144:0/2572592948 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f3374100e40 0x7f3374196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:31.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.452+0000 7f33793a7700 1 -- 172.21.15.144:0/2572592948 >> 172.21.15.144:0/2572592948 conn(0x7f33740fbd60 msgr2=0x7f33740fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:31.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.452+0000 7f33793a7700 1 -- 172.21.15.144:0/2572592948 shutdown_connections 2024-09-10T12:11:31.454 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:31.452+0000 7f33793a7700 1 -- 172.21.15.144:0/2572592948 wait complete. 2024-09-10T12:11:31.454 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 32 2024-09-10T12:11:32.025 DEBUG:tasks.fs:max_mds not reduced in epoch 32 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T12:11:32.025 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 32 2024-09-10T12:11:32.026 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 33 2024-09-10T12:11:32.226 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:32.674 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:32 smithi144 ceph-mon[87436]: pgmap v1189: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:32.674 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:32 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2572592948' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-09-10T12:11:32.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:32 smithi178 ceph-mon[65940]: pgmap v1189: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:32.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:32 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2572592948' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-09-10T12:11:33.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.856+0000 7f6bd4c0c700 1 -- 172.21.15.144:0/999008177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ff2c0 msgr2=0x7f6bd00ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:33.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.856+0000 7f6bd4c0c700 1 --2- 172.21.15.144:0/999008177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ff2c0 0x7f6bd00ff690 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f6bc0009910 tx=0x7f6bc0009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:33.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.856+0000 7f6bd4c0c700 1 -- 172.21.15.144:0/999008177 shutdown_connections 2024-09-10T12:11:33.858 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.856+0000 7f6bd4c0c700 1 --2- 172.21.15.144:0/999008177 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd00ffc60 0x7f6bd0103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:33.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.856+0000 7f6bd4c0c700 1 --2- 172.21.15.144:0/999008177 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ff2c0 0x7f6bd00ff690 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:33.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.856+0000 7f6bd4c0c700 1 -- 172.21.15.144:0/999008177 >> 172.21.15.144:0/999008177 conn(0x7f6bd00fab80 msgr2=0x7f6bd00fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:33.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.857+0000 7f6bd4c0c700 1 -- 172.21.15.144:0/999008177 shutdown_connections 2024-09-10T12:11:33.859 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.857+0000 7f6bd4c0c700 1 -- 172.21.15.144:0/999008177 wait complete. 2024-09-10T12:11:33.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.857+0000 7f6bd4c0c700 1 Processor -- start 2024-09-10T12:11:33.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.857+0000 7f6bd4c0c700 1 -- start start 2024-09-10T12:11:33.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.858+0000 7f6bd4c0c700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd00ff2c0 0x7f6bd0195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:33.860 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.858+0000 7f6bd4c0c700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ffc60 0x7f6bd01962b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.858+0000 7f6bd4c0c700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd0196990 con 0x7f6bd00ffc60 2024-09-10T12:11:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.858+0000 7f6bd4c0c700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd019a6d0 con 0x7f6bd00ff2c0 2024-09-10T12:11:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.858+0000 7f6bcdd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ffc60 0x7f6bd01962b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.858+0000 7f6bcdd9b700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ffc60 0x7f6bd01962b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57962/0 (socket says 172.21.15.144:57962) 2024-09-10T12:11:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.858+0000 7f6bcdd9b700 1 -- 172.21.15.144:0/1459282089 learned_addr learned my addr 172.21.15.144:0/1459282089 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.858+0000 7f6bce59c700 1 --2- 172.21.15.144:0/1459282089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd00ff2c0 0x7f6bd0195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.858+0000 7f6bcdd9b700 1 -- 172.21.15.144:0/1459282089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd00ff2c0 msgr2=0x7f6bd0195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.858+0000 7f6bcdd9b700 1 --2- 172.21.15.144:0/1459282089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd00ff2c0 0x7f6bd0195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.858+0000 7f6bcdd9b700 1 -- 172.21.15.144:0/1459282089 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc00095f0 con 0x7f6bd00ffc60 2024-09-10T12:11:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.858+0000 7f6bcdd9b700 1 --2- 172.21.15.144:0/1459282089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ffc60 0x7f6bd01962b0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400d900 tx=0x7f6bc400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.858+0000 7f6bbf7fe700 1 -- 172.21.15.144:0/1459282089 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bc4009900 con 0x7f6bd00ffc60 2024-09-10T12:11:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.859+0000 7f6bd4c0c700 1 -- 172.21.15.144:0/1459282089 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd019a9b0 con 0x7f6bd00ffc60 2024-09-10T12:11:33.861 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.859+0000 7f6bbf7fe700 1 -- 172.21.15.144:0/1459282089 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6bc4010460 con 0x7f6bd00ffc60 2024-09-10T12:11:33.862 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.859+0000 7f6bbf7fe700 1 -- 172.21.15.144:0/1459282089 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bc400f5f0 con 0x7f6bd00ffc60 2024-09-10T12:11:33.862 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.859+0000 7f6bd4c0c700 1 -- 172.21.15.144:0/1459282089 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd019af00 con 0x7f6bd00ffc60 2024-09-10T12:11:33.862 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.860+0000 7f6bd4c0c700 1 -- 172.21.15.144:0/1459282089 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd01085e0 con 0x7f6bd00ffc60 2024-09-10T12:11:33.862 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.860+0000 7f6bbf7fe700 1 -- 172.21.15.144:0/1459282089 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f6bc4009a60 con 0x7f6bd00ffc60 2024-09-10T12:11:33.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.861+0000 7f6bbf7fe700 1 --2- 172.21.15.144:0/1459282089 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bb806e200 0x7f6bb80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:33.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.861+0000 7f6bce59c700 1 --2- 172.21.15.144:0/1459282089 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bb806e200 0x7f6bb80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:33.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.861+0000 7f6bbf7fe700 1 -- 172.21.15.144:0/1459282089 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f6bc408dc50 con 0x7f6bd00ffc60 2024-09-10T12:11:33.863 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.861+0000 7f6bce59c700 1 --2- 172.21.15.144:0/1459282089 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bb806e200 0x7f6bb80706b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000b5c0 tx=0x7f6bc001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:33.866 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:33.865+0000 7f6bbf7fe700 1 -- 172.21.15.144:0/1459282089 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f6bc4057030 con 0x7f6bd00ffc60 2024-09-10T12:11:34.110 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:34.109+0000 7f6bd4c0c700 1 -- 172.21.15.144:0/1459282089 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f6bd004ea50 con 0x7f6bd00ffc60 2024-09-10T12:11:34.111 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:34.109+0000 7f6bbf7fe700 1 -- 172.21.15.144:0/1459282089 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v42) v1 ==== 107+0+4326 (secure 0 0 0) 0x7f6bc405a6f0 con 0x7f6bd00ffc60 2024-09-10T12:11:34.111 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:34.111 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":33,"btime":"2024-09-10T11:53:18:688824+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34390,"name":"cephfs.smithi144.bfyvwv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/2724506882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2724506882},{"type":"v1","addr":"172.21.15.144:6827","nonce":2724506882}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":34422,"name":"cephfs.smithi144.jaoyvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6829/4254390011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4254390011},{"type":"v1","addr":"172.21.15.144:6829","nonce":4254390011}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44355,"name":"cephfs.smithi178.nmhrmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6825/706731549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":706731549},{"type":"v1","addr":"172.21.15.178:6825","nonce":706731549}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:37:33.047941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:34.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:34.112+0000 7f6bd4c0c700 1 -- 172.21.15.144:0/1459282089 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bb806e200 msgr2=0x7f6bb80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:34.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:34.112+0000 7f6bd4c0c700 1 --2- 172.21.15.144:0/1459282089 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bb806e200 0x7f6bb80706b0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000b5c0 tx=0x7f6bc001a040 comp rx=0 tx=0).stop 2024-09-10T12:11:34.113 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:34.112+0000 7f6bd4c0c700 1 -- 172.21.15.144:0/1459282089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ffc60 msgr2=0x7f6bd01962b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:34.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:34.112+0000 7f6bd4c0c700 1 --2- 172.21.15.144:0/1459282089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ffc60 0x7f6bd01962b0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f6bc400d900 tx=0x7f6bc400dc10 comp rx=0 tx=0).stop 2024-09-10T12:11:34.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:34.112+0000 7f6bd4c0c700 1 -- 172.21.15.144:0/1459282089 shutdown_connections 2024-09-10T12:11:34.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:34.112+0000 7f6bd4c0c700 1 --2- 172.21.15.144:0/1459282089 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f6bb806e200 0x7f6bb80706b0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:34.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:34.112+0000 7f6bd4c0c700 1 --2- 172.21.15.144:0/1459282089 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f6bd00ff2c0 0x7f6bd0195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:34.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:34.112+0000 7f6bd4c0c700 1 --2- 172.21.15.144:0/1459282089 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f6bd00ffc60 0x7f6bd01962b0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:34.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:34.112+0000 7f6bd4c0c700 1 -- 172.21.15.144:0/1459282089 >> 172.21.15.144:0/1459282089 conn(0x7f6bd00fab80 msgr2=0x7f6bd0068ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:34.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:34.112+0000 7f6bd4c0c700 1 -- 172.21.15.144:0/1459282089 shutdown_connections 2024-09-10T12:11:34.114 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:34.112+0000 7f6bd4c0c700 1 -- 172.21.15.144:0/1459282089 wait complete. 2024-09-10T12:11:34.114 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 33 2024-09-10T12:11:34.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:34 smithi144 ceph-mon[87436]: pgmap v1190: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:11:34.696 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:34 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1459282089' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-09-10T12:11:34.697 DEBUG:tasks.fs:max_mds not reduced in epoch 33 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T12:11:34.698 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 33 2024-09-10T12:11:34.698 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 34 2024-09-10T12:11:34.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:34 smithi178 ceph-mon[65940]: pgmap v1190: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:11:34.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:34 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1459282089' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-09-10T12:11:34.897 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:36.663 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.661+0000 7f7f86870700 1 -- 172.21.15.144:0/917805248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f800ff310 msgr2=0x7f7f800ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:36.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.661+0000 7f7f86870700 1 --2- 172.21.15.144:0/917805248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f800ff310 0x7f7f800ff6e0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f7f68009910 tx=0x7f7f68009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:36.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.661+0000 7f7f86870700 1 -- 172.21.15.144:0/917805248 shutdown_connections 2024-09-10T12:11:36.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.661+0000 7f7f86870700 1 --2- 172.21.15.144:0/917805248 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f800ffcb0 0x7f7f80103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:36.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.661+0000 7f7f86870700 1 --2- 172.21.15.144:0/917805248 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f800ff310 0x7f7f800ff6e0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:36.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.661+0000 7f7f86870700 1 -- 172.21.15.144:0/917805248 >> 172.21.15.144:0/917805248 conn(0x7f7f800fabd0 msgr2=0x7f7f800fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:36.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.661+0000 7f7f86870700 1 -- 172.21.15.144:0/917805248 shutdown_connections 2024-09-10T12:11:36.664 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.661+0000 7f7f86870700 1 -- 172.21.15.144:0/917805248 wait complete. 2024-09-10T12:11:36.664 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:36 smithi144 ceph-mon[87436]: pgmap v1191: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:36.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.662+0000 7f7f86870700 1 Processor -- start 2024-09-10T12:11:36.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.662+0000 7f7f86870700 1 -- start start 2024-09-10T12:11:36.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.662+0000 7f7f86870700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f800ff310 0x7f7f80198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:36.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.662+0000 7f7f86870700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f800ffcb0 0x7f7f80198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:36.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.662+0000 7f7f86870700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f80198c40 con 0x7f7f800ff310 2024-09-10T12:11:36.666 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.662+0000 7f7f86870700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8019c9d0 con 0x7f7f800ffcb0 2024-09-10T12:11:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.663+0000 7f7f7ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f800ff310 0x7f7f80198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.663+0000 7f7f7ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f800ff310 0x7f7f80198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:57992/0 (socket says 172.21.15.144:57992) 2024-09-10T12:11:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.663+0000 7f7f7ffff700 1 -- 172.21.15.144:0/1226812654 learned_addr learned my addr 172.21.15.144:0/1226812654 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.663+0000 7f7f7f7fe700 1 --2- 172.21.15.144:0/1226812654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f800ffcb0 0x7f7f80198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.663+0000 7f7f7ffff700 1 -- 172.21.15.144:0/1226812654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f800ffcb0 msgr2=0x7f7f80198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.663+0000 7f7f7ffff700 1 --2- 172.21.15.144:0/1226812654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f800ffcb0 0x7f7f80198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.663+0000 7f7f7ffff700 1 -- 172.21.15.144:0/1226812654 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f680095f0 con 0x7f7f800ff310 2024-09-10T12:11:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.663+0000 7f7f7ffff700 1 --2- 172.21.15.144:0/1226812654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f800ff310 0x7f7f80198020 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f7f68006010 tx=0x7f7f6800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.663+0000 7f7f7d7fa700 1 -- 172.21.15.144:0/1226812654 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f6801d070 con 0x7f7f800ff310 2024-09-10T12:11:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.663+0000 7f7f7d7fa700 1 -- 172.21.15.144:0/1226812654 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7f6800f460 con 0x7f7f800ff310 2024-09-10T12:11:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.663+0000 7f7f86870700 1 -- 172.21.15.144:0/1226812654 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f8019cc50 con 0x7f7f800ff310 2024-09-10T12:11:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.663+0000 7f7f7d7fa700 1 -- 172.21.15.144:0/1226812654 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f68017610 con 0x7f7f800ff310 2024-09-10T12:11:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.664+0000 7f7f86870700 1 -- 172.21.15.144:0/1226812654 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f8019d140 con 0x7f7f800ff310 2024-09-10T12:11:36.667 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.665+0000 7f7f86870700 1 -- 172.21.15.144:0/1226812654 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f8004ea50 con 0x7f7f800ff310 2024-09-10T12:11:36.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.665+0000 7f7f7d7fa700 1 -- 172.21.15.144:0/1226812654 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f7f6800f5d0 con 0x7f7f800ff310 2024-09-10T12:11:36.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.665+0000 7f7f7d7fa700 1 --2- 172.21.15.144:0/1226812654 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f6c06e200 0x7f7f6c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:36.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.665+0000 7f7f7d7fa700 1 -- 172.21.15.144:0/1226812654 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f7f6808f580 con 0x7f7f800ff310 2024-09-10T12:11:36.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.666+0000 7f7f7f7fe700 1 --2- 172.21.15.144:0/1226812654 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f6c06e200 0x7f7f6c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:36.668 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.666+0000 7f7f7f7fe700 1 --2- 172.21.15.144:0/1226812654 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f6c06e200 0x7f7f6c0706b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7f80199640 tx=0x7f7f7000a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:36.671 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.670+0000 7f7f7d7fa700 1 -- 172.21.15.144:0/1226812654 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f7f680589e0 con 0x7f7f800ff310 2024-09-10T12:11:36.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:36 smithi178 ceph-mon[65940]: pgmap v1191: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:36.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.915+0000 7f7f86870700 1 -- 172.21.15.144:0/1226812654 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f7f80199380 con 0x7f7f800ff310 2024-09-10T12:11:36.917 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.916+0000 7f7f7d7fa700 1 -- 172.21.15.144:0/1226812654 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v42) v1 ==== 107+0+5176 (secure 0 0 0) 0x7f7f6805c0a0 con 0x7f7f800ff310 2024-09-10T12:11:36.917 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:36.918 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":34,"btime":"2024-09-10T11:53:22:534815+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34390,"name":"cephfs.smithi144.bfyvwv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/2724506882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2724506882},{"type":"v1","addr":"172.21.15.144:6827","nonce":2724506882}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":34422,"name":"cephfs.smithi144.jaoyvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6829/4254390011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4254390011},{"type":"v1","addr":"172.21.15.144:6829","nonce":4254390011}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44355,"name":"cephfs.smithi178.nmhrmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6825/706731549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":706731549},{"type":"v1","addr":"172.21.15.178:6825","nonce":706731549}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44887,"name":"cephfs.smithi178.czjrsa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6827/2967786976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2967786976},{"type":"v1","addr":"172.21.15.178:6827","nonce":2967786976}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":25,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T11:37:33.047941+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:36.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.918+0000 7f7f86870700 1 -- 172.21.15.144:0/1226812654 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f6c06e200 msgr2=0x7f7f6c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:36.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.918+0000 7f7f86870700 1 --2- 172.21.15.144:0/1226812654 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f6c06e200 0x7f7f6c0706b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f7f80199640 tx=0x7f7f7000a300 comp rx=0 tx=0).stop 2024-09-10T12:11:36.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.918+0000 7f7f86870700 1 -- 172.21.15.144:0/1226812654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f800ff310 msgr2=0x7f7f80198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:36.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.918+0000 7f7f86870700 1 --2- 172.21.15.144:0/1226812654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f800ff310 0x7f7f80198020 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f7f68006010 tx=0x7f7f6800bd70 comp rx=0 tx=0).stop 2024-09-10T12:11:36.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.918+0000 7f7f86870700 1 -- 172.21.15.144:0/1226812654 shutdown_connections 2024-09-10T12:11:36.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.918+0000 7f7f86870700 1 --2- 172.21.15.144:0/1226812654 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f7f6c06e200 0x7f7f6c0706b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:36.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.918+0000 7f7f86870700 1 --2- 172.21.15.144:0/1226812654 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f7f800ff310 0x7f7f80198020 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:36.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.918+0000 7f7f86870700 1 --2- 172.21.15.144:0/1226812654 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f7f800ffcb0 0x7f7f80198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:36.920 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.918+0000 7f7f86870700 1 -- 172.21.15.144:0/1226812654 >> 172.21.15.144:0/1226812654 conn(0x7f7f800fabd0 msgr2=0x7f7f80102c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:36.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.918+0000 7f7f86870700 1 -- 172.21.15.144:0/1226812654 shutdown_connections 2024-09-10T12:11:36.921 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:36.918+0000 7f7f86870700 1 -- 172.21.15.144:0/1226812654 wait complete. 2024-09-10T12:11:36.921 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 34 2024-09-10T12:11:37.565 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:37 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1226812654' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-09-10T12:11:37.586 DEBUG:tasks.fs:max_mds not reduced in epoch 34 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-09-10T12:11:37.586 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 34 2024-09-10T12:11:37.586 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 35 2024-09-10T12:11:37.786 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:37.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:37 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1226812654' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-09-10T12:11:38.847 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:38 smithi144 ceph-mon[87436]: pgmap v1192: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:11:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:38 smithi178 ceph-mon[65940]: pgmap v1192: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:11:39.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.393+0000 7f4cec3a8700 1 -- 172.21.15.144:0/778207822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce4100da0 msgr2=0x7f4ce4104d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:39.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.393+0000 7f4cec3a8700 1 --2- 172.21.15.144:0/778207822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce4100da0 0x7f4ce4104d70 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009910 tx=0x7f4cd4009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:39.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.393+0000 7f4cec3a8700 1 -- 172.21.15.144:0/778207822 shutdown_connections 2024-09-10T12:11:39.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.393+0000 7f4cec3a8700 1 --2- 172.21.15.144:0/778207822 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce4100da0 0x7f4ce4104d70 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:39.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.393+0000 7f4cec3a8700 1 --2- 172.21.15.144:0/778207822 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ce4100400 0x7f4ce41007d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:39.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.393+0000 7f4cec3a8700 1 -- 172.21.15.144:0/778207822 >> 172.21.15.144:0/778207822 conn(0x7f4ce40fbca0 msgr2=0x7f4ce40fe0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:39.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.393+0000 7f4cec3a8700 1 -- 172.21.15.144:0/778207822 shutdown_connections 2024-09-10T12:11:39.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.393+0000 7f4cec3a8700 1 -- 172.21.15.144:0/778207822 wait complete. 2024-09-10T12:11:39.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.394+0000 7f4cec3a8700 1 Processor -- start 2024-09-10T12:11:39.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.394+0000 7f4cec3a8700 1 -- start start 2024-09-10T12:11:39.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.394+0000 7f4cec3a8700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ce4100400 0x7f4ce4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:39.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.394+0000 7f4cec3a8700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce4100da0 0x7f4ce4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:39.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.394+0000 7f4cec3a8700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce4073ad0 con 0x7f4ce4100da0 2024-09-10T12:11:39.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.394+0000 7f4cec3a8700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce4073c10 con 0x7f4ce4100400 2024-09-10T12:11:39.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.394+0000 7f4ce9943700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce4100da0 0x7f4ce4073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:39.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.394+0000 7f4ce9943700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce4100da0 0x7f4ce4073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58008/0 (socket says 172.21.15.144:58008) 2024-09-10T12:11:39.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.394+0000 7f4ce9943700 1 -- 172.21.15.144:0/1166884933 learned_addr learned my addr 172.21.15.144:0/1166884933 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:39.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.394+0000 7f4cea144700 1 --2- 172.21.15.144:0/1166884933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ce4100400 0x7f4ce4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:39.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.394+0000 7f4ce9943700 1 -- 172.21.15.144:0/1166884933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ce4100400 msgr2=0x7f4ce4073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:39.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.394+0000 7f4ce9943700 1 --2- 172.21.15.144:0/1166884933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ce4100400 0x7f4ce4073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:39.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.394+0000 7f4ce9943700 1 -- 172.21.15.144:0/1166884933 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd40095f0 con 0x7f4ce4100da0 2024-09-10T12:11:39.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.394+0000 7f4cea144700 1 --2- 172.21.15.144:0/1166884933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ce4100400 0x7f4ce4073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:11:39.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.395+0000 7f4ce9943700 1 --2- 172.21.15.144:0/1166884933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce4100da0 0x7f4ce4073590 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40092e0 tx=0x7f4cd400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:39.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.395+0000 7f4cdb7fe700 1 -- 172.21.15.144:0/1166884933 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cd401d070 con 0x7f4ce4100da0 2024-09-10T12:11:39.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.395+0000 7f4cdb7fe700 1 -- 172.21.15.144:0/1166884933 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4cd400f460 con 0x7f4ce4100da0 2024-09-10T12:11:39.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.395+0000 7f4cec3a8700 1 -- 172.21.15.144:0/1166884933 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce4073e90 con 0x7f4ce4100da0 2024-09-10T12:11:39.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.395+0000 7f4cdb7fe700 1 -- 172.21.15.144:0/1166884933 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4cd40175f0 con 0x7f4ce4100da0 2024-09-10T12:11:39.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.395+0000 7f4cec3a8700 1 -- 172.21.15.144:0/1166884933 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce41a2240 con 0x7f4ce4100da0 2024-09-10T12:11:39.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.396+0000 7f4cec3a8700 1 -- 172.21.15.144:0/1166884933 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce4108680 con 0x7f4ce4100da0 2024-09-10T12:11:39.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.396+0000 7f4cdb7fe700 1 -- 172.21.15.144:0/1166884933 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f4cd400f5d0 con 0x7f4ce4100da0 2024-09-10T12:11:39.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.397+0000 7f4cdb7fe700 1 --2- 172.21.15.144:0/1166884933 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cd006e200 0x7f4cd00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:39.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.397+0000 7f4cdb7fe700 1 -- 172.21.15.144:0/1166884933 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f4cd408f590 con 0x7f4ce4100da0 2024-09-10T12:11:39.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.397+0000 7f4cea144700 1 --2- 172.21.15.144:0/1166884933 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cd006e200 0x7f4cd00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:39.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.397+0000 7f4cea144700 1 --2- 172.21.15.144:0/1166884933 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cd006e200 0x7f4cd00706b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0007fd0 tx=0x7f4ce0007810 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:39.402 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.401+0000 7f4cdb7fe700 1 -- 172.21.15.144:0/1166884933 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f4cd4058970 con 0x7f4ce4100da0 2024-09-10T12:11:39.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.648+0000 7f4cec3a8700 1 -- 172.21.15.144:0/1166884933 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f4ce404ea50 con 0x7f4ce4100da0 2024-09-10T12:11:39.650 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.649+0000 7f4cdb7fe700 1 -- 172.21.15.144:0/1166884933 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v42) v1 ==== 107+0+5175 (secure 0 0 0) 0x7f4cd405c030 con 0x7f4ce4100da0 2024-09-10T12:11:39.650 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:39.651 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":35,"btime":"2024-09-10T12:08:23:697495+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34390,"name":"cephfs.smithi144.bfyvwv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6827/2724506882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2724506882},{"type":"v1","addr":"172.21.15.144:6827","nonce":2724506882}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":28},{"gid":34422,"name":"cephfs.smithi144.jaoyvs","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.144:6829/4254390011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4254390011},{"type":"v1","addr":"172.21.15.144:6829","nonce":4254390011}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30},{"gid":44355,"name":"cephfs.smithi178.nmhrmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6825/706731549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":706731549},{"type":"v1","addr":"172.21.15.178:6825","nonce":706731549}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44887,"name":"cephfs.smithi178.czjrsa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6827/2967786976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2967786976},{"type":"v1","addr":"172.21.15.178:6827","nonce":2967786976}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T12:08:22.700328+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:39.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.651+0000 7f4cec3a8700 1 -- 172.21.15.144:0/1166884933 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cd006e200 msgr2=0x7f4cd00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:39.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.651+0000 7f4cec3a8700 1 --2- 172.21.15.144:0/1166884933 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cd006e200 0x7f4cd00706b0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f4ce0007fd0 tx=0x7f4ce0007810 comp rx=0 tx=0).stop 2024-09-10T12:11:39.653 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.651+0000 7f4cec3a8700 1 -- 172.21.15.144:0/1166884933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce4100da0 msgr2=0x7f4ce4073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:39.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.651+0000 7f4cec3a8700 1 --2- 172.21.15.144:0/1166884933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce4100da0 0x7f4ce4073590 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40092e0 tx=0x7f4cd400be20 comp rx=0 tx=0).stop 2024-09-10T12:11:39.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.652+0000 7f4cec3a8700 1 -- 172.21.15.144:0/1166884933 shutdown_connections 2024-09-10T12:11:39.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.652+0000 7f4cec3a8700 1 --2- 172.21.15.144:0/1166884933 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f4cd006e200 0x7f4cd00706b0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:39.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.652+0000 7f4cec3a8700 1 --2- 172.21.15.144:0/1166884933 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f4ce4100400 0x7f4ce4073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:39.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.652+0000 7f4cec3a8700 1 --2- 172.21.15.144:0/1166884933 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f4ce4100da0 0x7f4ce4073590 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:39.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.652+0000 7f4cec3a8700 1 -- 172.21.15.144:0/1166884933 >> 172.21.15.144:0/1166884933 conn(0x7f4ce40fbca0 msgr2=0x7f4ce41045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:39.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.652+0000 7f4cec3a8700 1 -- 172.21.15.144:0/1166884933 shutdown_connections 2024-09-10T12:11:39.654 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:39.652+0000 7f4cec3a8700 1 -- 172.21.15.144:0/1166884933 wait complete. 2024-09-10T12:11:39.654 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 35 2024-09-10T12:11:39.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:39 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1166884933' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-09-10T12:11:40.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:39 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1166884933' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-09-10T12:11:40.202 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 35 2024-09-10T12:11:40.202 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 36 2024-09-10T12:11:40.403 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:40.790 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:40 smithi144 ceph-mon[87436]: pgmap v1193: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:11:41.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:40 smithi178 ceph-mon[65940]: pgmap v1193: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:11:42.054 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.050+0000 7fe096da2700 1 -- 172.21.15.144:0/3990639913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0901003a0 msgr2=0x7fe090100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:42.055 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.050+0000 7fe096da2700 1 --2- 172.21.15.144:0/3990639913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0901003a0 0x7fe090100770 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe078009940 tx=0x7fe078009c50 comp rx=0 tx=0).stop 2024-09-10T12:11:42.055 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.050+0000 7fe096da2700 1 -- 172.21.15.144:0/3990639913 shutdown_connections 2024-09-10T12:11:42.055 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.050+0000 7fe096da2700 1 --2- 172.21.15.144:0/3990639913 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe090100cb0 0x7fe090109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:42.055 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.050+0000 7fe096da2700 1 --2- 172.21.15.144:0/3990639913 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe0901003a0 0x7fe090100770 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:42.055 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.050+0000 7fe096da2700 1 -- 172.21.15.144:0/3990639913 >> 172.21.15.144:0/3990639913 conn(0x7fe090074850 msgr2=0x7fe090074c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:42.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.050+0000 7fe096da2700 1 -- 172.21.15.144:0/3990639913 shutdown_connections 2024-09-10T12:11:42.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.050+0000 7fe096da2700 1 -- 172.21.15.144:0/3990639913 wait complete. 2024-09-10T12:11:42.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.051+0000 7fe096da2700 1 Processor -- start 2024-09-10T12:11:42.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.051+0000 7fe096da2700 1 -- start start 2024-09-10T12:11:42.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.051+0000 7fe096da2700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0901003a0 0x7fe090194d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:42.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.051+0000 7fe096da2700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe090100cb0 0x7fe0901952a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:42.056 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.051+0000 7fe096da2700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe090195980 con 0x7fe090100cb0 2024-09-10T12:11:42.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.051+0000 7fe096da2700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe090199710 con 0x7fe0901003a0 2024-09-10T12:11:42.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.051+0000 7fe08ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe090100cb0 0x7fe0901952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:42.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.051+0000 7fe08ffff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe090100cb0 0x7fe0901952a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:58016/0 (socket says 172.21.15.144:58016) 2024-09-10T12:11:42.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.051+0000 7fe08ffff700 1 -- 172.21.15.144:0/2352159705 learned_addr learned my addr 172.21.15.144:0/2352159705 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:42.057 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.051+0000 7fe094b3e700 1 --2- 172.21.15.144:0/2352159705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0901003a0 0x7fe090194d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:42.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.051+0000 7fe08ffff700 1 -- 172.21.15.144:0/2352159705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0901003a0 msgr2=0x7fe090194d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:42.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.051+0000 7fe08ffff700 1 --2- 172.21.15.144:0/2352159705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0901003a0 0x7fe090194d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:42.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.051+0000 7fe08ffff700 1 -- 172.21.15.144:0/2352159705 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0780095f0 con 0x7fe090100cb0 2024-09-10T12:11:42.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.052+0000 7fe08ffff700 1 --2- 172.21.15.144:0/2352159705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe090100cb0 0x7fe0901952a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe08000ba50 tx=0x7fe08000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:42.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.052+0000 7fe08dffb700 1 -- 172.21.15.144:0/2352159705 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe08000ca00 con 0x7fe090100cb0 2024-09-10T12:11:42.058 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.052+0000 7fe08dffb700 1 -- 172.21.15.144:0/2352159705 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe08000cb60 con 0x7fe090100cb0 2024-09-10T12:11:42.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.052+0000 7fe096da2700 1 -- 172.21.15.144:0/2352159705 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0901999f0 con 0x7fe090100cb0 2024-09-10T12:11:42.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.052+0000 7fe08dffb700 1 -- 172.21.15.144:0/2352159705 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0800075a0 con 0x7fe090100cb0 2024-09-10T12:11:42.059 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.052+0000 7fe096da2700 1 -- 172.21.15.144:0/2352159705 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe090199f40 con 0x7fe090100cb0 2024-09-10T12:11:42.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.053+0000 7fe096da2700 1 -- 172.21.15.144:0/2352159705 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe090106b10 con 0x7fe090100cb0 2024-09-10T12:11:42.060 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.053+0000 7fe08dffb700 1 -- 172.21.15.144:0/2352159705 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fe08000ccd0 con 0x7fe090100cb0 2024-09-10T12:11:42.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.054+0000 7fe08dffb700 1 --2- 172.21.15.144:0/2352159705 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe07c06e200 0x7fe07c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:42.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.054+0000 7fe08dffb700 1 -- 172.21.15.144:0/2352159705 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fe080006e40 con 0x7fe090100cb0 2024-09-10T12:11:42.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.054+0000 7fe094b3e700 1 --2- 172.21.15.144:0/2352159705 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe07c06e200 0x7fe07c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:42.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.054+0000 7fe094b3e700 1 --2- 172.21.15.144:0/2352159705 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe07c06e200 0x7fe07c0706b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe078005cd0 tx=0x7fe078005c00 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:42.061 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.058+0000 7fe08dffb700 1 -- 172.21.15.144:0/2352159705 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fe080056cc0 con 0x7fe090100cb0 2024-09-10T12:11:42.317 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.316+0000 7fe096da2700 1 -- 172.21.15.144:0/2352159705 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7fe09019a1c0 con 0x7fe090100cb0 2024-09-10T12:11:42.318 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.317+0000 7fe08dffb700 1 -- 172.21.15.144:0/2352159705 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v42) v1 ==== 107+0+5266 (secure 0 0 0) 0x7fe08005a380 con 0x7fe090100cb0 2024-09-10T12:11:42.318 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:42.319 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":36,"btime":"2024-09-10T12:08:23:702329+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44355,"name":"cephfs.smithi178.nmhrmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6825/706731549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":706731549},{"type":"v1","addr":"172.21.15.178:6825","nonce":706731549}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44887,"name":"cephfs.smithi178.czjrsa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6827/2967786976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2967786976},{"type":"v1","addr":"172.21.15.178:6827","nonce":2967786976}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T12:08:23.702318+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34390,"mds_1":34422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34390":{"gid":34390,"name":"cephfs.smithi144.bfyvwv","rank":0,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6827/2724506882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2724506882},{"type":"v1","addr":"172.21.15.144:6827","nonce":2724506882}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34422":{"gid":34422,"name":"cephfs.smithi144.jaoyvs","rank":1,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6829/4254390011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4254390011},{"type":"v1","addr":"172.21.15.144:6829","nonce":4254390011}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:42.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.320+0000 7fe096da2700 1 -- 172.21.15.144:0/2352159705 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe07c06e200 msgr2=0x7fe07c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:42.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.320+0000 7fe096da2700 1 --2- 172.21.15.144:0/2352159705 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe07c06e200 0x7fe07c0706b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fe078005cd0 tx=0x7fe078005c00 comp rx=0 tx=0).stop 2024-09-10T12:11:42.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.320+0000 7fe096da2700 1 -- 172.21.15.144:0/2352159705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe090100cb0 msgr2=0x7fe0901952a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:42.321 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.320+0000 7fe096da2700 1 --2- 172.21.15.144:0/2352159705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe090100cb0 0x7fe0901952a0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe08000ba50 tx=0x7fe08000be10 comp rx=0 tx=0).stop 2024-09-10T12:11:42.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.320+0000 7fe096da2700 1 -- 172.21.15.144:0/2352159705 shutdown_connections 2024-09-10T12:11:42.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.320+0000 7fe096da2700 1 --2- 172.21.15.144:0/2352159705 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fe07c06e200 0x7fe07c0706b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:42.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.320+0000 7fe096da2700 1 --2- 172.21.15.144:0/2352159705 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fe0901003a0 0x7fe090194d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:42.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.320+0000 7fe096da2700 1 --2- 172.21.15.144:0/2352159705 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fe090100cb0 0x7fe0901952a0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:42.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.320+0000 7fe096da2700 1 -- 172.21.15.144:0/2352159705 >> 172.21.15.144:0/2352159705 conn(0x7fe090074850 msgr2=0x7fe090103c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:42.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.320+0000 7fe096da2700 1 -- 172.21.15.144:0/2352159705 shutdown_connections 2024-09-10T12:11:42.322 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:42.320+0000 7fe096da2700 1 -- 172.21.15.144:0/2352159705 wait complete. 2024-09-10T12:11:42.322 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 36 2024-09-10T12:11:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:42 smithi144 ceph-mon[87436]: pgmap v1194: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:42 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2352159705' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-09-10T12:11:42.994 DEBUG:tasks.fs:allow_standby_replay disabled in epoch 36 2024-09-10T12:11:42.994 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 37 2024-09-10T12:11:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:42 smithi178 ceph-mon[65940]: pgmap v1194: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:42 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2352159705' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-09-10T12:11:43.200 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:43.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:11:44.018 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:11:44.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.923+0000 7feb45e7e700 1 -- 172.21.15.144:0/282586994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb401003d0 msgr2=0x7feb401007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:44.925 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.923+0000 7feb45e7e700 1 --2- 172.21.15.144:0/282586994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb401003d0 0x7feb401007a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7feb34009910 tx=0x7feb34009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:44.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.923+0000 7feb45e7e700 1 -- 172.21.15.144:0/282586994 shutdown_connections 2024-09-10T12:11:44.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.923+0000 7feb45e7e700 1 --2- 172.21.15.144:0/282586994 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb40100d70 0x7feb40104d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:44.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.923+0000 7feb45e7e700 1 --2- 172.21.15.144:0/282586994 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb401003d0 0x7feb401007a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:44.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.923+0000 7feb45e7e700 1 -- 172.21.15.144:0/282586994 >> 172.21.15.144:0/282586994 conn(0x7feb400fbc50 msgr2=0x7feb400fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:44.926 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.923+0000 7feb45e7e700 1 -- 172.21.15.144:0/282586994 shutdown_connections 2024-09-10T12:11:44.926 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:44 smithi144 ceph-mon[87436]: pgmap v1195: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:11:44.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.923+0000 7feb45e7e700 1 -- 172.21.15.144:0/282586994 wait complete. 2024-09-10T12:11:44.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.924+0000 7feb45e7e700 1 Processor -- start 2024-09-10T12:11:44.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.924+0000 7feb45e7e700 1 -- start start 2024-09-10T12:11:44.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.924+0000 7feb45e7e700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb401003d0 0x7feb40195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:44.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.924+0000 7feb45e7e700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb40100d70 0x7feb401962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:44.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.924+0000 7feb45e7e700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb401969d0 con 0x7feb40100d70 2024-09-10T12:11:44.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.924+0000 7feb45e7e700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4019a760 con 0x7feb401003d0 2024-09-10T12:11:44.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.924+0000 7feb3effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb40100d70 0x7feb401962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:44.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.924+0000 7feb3effd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb40100d70 0x7feb401962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36260/0 (socket says 172.21.15.144:36260) 2024-09-10T12:11:44.928 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.924+0000 7feb3effd700 1 -- 172.21.15.144:0/3970800397 learned_addr learned my addr 172.21.15.144:0/3970800397 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:44.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.924+0000 7feb3f7fe700 1 --2- 172.21.15.144:0/3970800397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb401003d0 0x7feb40195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:44.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.925+0000 7feb3effd700 1 -- 172.21.15.144:0/3970800397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb401003d0 msgr2=0x7feb40195db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:44.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.925+0000 7feb3effd700 1 --2- 172.21.15.144:0/3970800397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb401003d0 0x7feb40195db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:44.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.925+0000 7feb3effd700 1 -- 172.21.15.144:0/3970800397 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb340095f0 con 0x7feb40100d70 2024-09-10T12:11:44.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.925+0000 7feb3f7fe700 1 --2- 172.21.15.144:0/3970800397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb401003d0 0x7feb40195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:11:44.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.925+0000 7feb3effd700 1 --2- 172.21.15.144:0/3970800397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb40100d70 0x7feb401962f0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7feb3000c8f0 tx=0x7feb3000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:44.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.925+0000 7feb3cff9700 1 -- 172.21.15.144:0/3970800397 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb30007a50 con 0x7feb40100d70 2024-09-10T12:11:44.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.925+0000 7feb45e7e700 1 -- 172.21.15.144:0/3970800397 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4019aa40 con 0x7feb40100d70 2024-09-10T12:11:44.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.925+0000 7feb45e7e700 1 -- 172.21.15.144:0/3970800397 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4019af90 con 0x7feb40100d70 2024-09-10T12:11:44.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.925+0000 7feb3cff9700 1 -- 172.21.15.144:0/3970800397 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb30007bb0 con 0x7feb40100d70 2024-09-10T12:11:44.929 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.925+0000 7feb3cff9700 1 -- 172.21.15.144:0/3970800397 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb30018630 con 0x7feb40100d70 2024-09-10T12:11:44.930 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.926+0000 7feb45e7e700 1 -- 172.21.15.144:0/3970800397 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb2c005320 con 0x7feb40100d70 2024-09-10T12:11:44.930 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.927+0000 7feb3cff9700 1 -- 172.21.15.144:0/3970800397 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7feb30007d20 con 0x7feb40100d70 2024-09-10T12:11:44.930 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.927+0000 7feb3cff9700 1 --2- 172.21.15.144:0/3970800397 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feb2806e200 0x7feb280706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:44.930 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.927+0000 7feb3cff9700 1 -- 172.21.15.144:0/3970800397 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7feb3008df00 con 0x7feb40100d70 2024-09-10T12:11:44.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.927+0000 7feb3f7fe700 1 --2- 172.21.15.144:0/3970800397 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feb2806e200 0x7feb280706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:44.931 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.928+0000 7feb3f7fe700 1 --2- 172.21.15.144:0/3970800397 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feb2806e200 0x7feb280706b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7feb34005100 tx=0x7feb3401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:44.934 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:44.932+0000 7feb3cff9700 1 -- 172.21.15.144:0/3970800397 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7feb30057360 con 0x7feb40100d70 2024-09-10T12:11:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:44 smithi178 ceph-mon[65940]: pgmap v1195: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:11:45.179 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:45.177+0000 7feb45e7e700 1 -- 172.21.15.144:0/3970800397 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7feb2c005190 con 0x7feb40100d70 2024-09-10T12:11:45.180 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:45.177+0000 7feb3cff9700 1 -- 172.21.15.144:0/3970800397 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v42) v1 ==== 107+0+5265 (secure 0 0 0) 0x7feb3005aa20 con 0x7feb40100d70 2024-09-10T12:11:45.180 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:45.180 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":37,"btime":"2024-09-10T12:08:24:703246+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44355,"name":"cephfs.smithi178.nmhrmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6825/706731549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":706731549},{"type":"v1","addr":"172.21.15.178:6825","nonce":706731549}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44887,"name":"cephfs.smithi178.czjrsa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6827/2967786976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2967786976},{"type":"v1","addr":"172.21.15.178:6827","nonce":2967786976}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":37,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T12:08:23.706844+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34390,"mds_1":34422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34390":{"gid":34390,"name":"cephfs.smithi144.bfyvwv","rank":0,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6827/2724506882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2724506882},{"type":"v1","addr":"172.21.15.144:6827","nonce":2724506882}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34422":{"gid":34422,"name":"cephfs.smithi144.jaoyvs","rank":1,"incarnation":36,"state":"up:replay","state_seq":1,"addr":"172.21.15.144:6829/4254390011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4254390011},{"type":"v1","addr":"172.21.15.144:6829","nonce":4254390011}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:45.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:45.179+0000 7feb45e7e700 1 -- 172.21.15.144:0/3970800397 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feb2806e200 msgr2=0x7feb280706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:45.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:45.179+0000 7feb45e7e700 1 --2- 172.21.15.144:0/3970800397 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feb2806e200 0x7feb280706b0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7feb34005100 tx=0x7feb3401a040 comp rx=0 tx=0).stop 2024-09-10T12:11:45.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:45.179+0000 7feb45e7e700 1 -- 172.21.15.144:0/3970800397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb40100d70 msgr2=0x7feb401962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:45.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:45.179+0000 7feb45e7e700 1 --2- 172.21.15.144:0/3970800397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb40100d70 0x7feb401962f0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7feb3000c8f0 tx=0x7feb3000ccb0 comp rx=0 tx=0).stop 2024-09-10T12:11:45.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:45.179+0000 7feb45e7e700 1 -- 172.21.15.144:0/3970800397 shutdown_connections 2024-09-10T12:11:45.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:45.179+0000 7feb45e7e700 1 --2- 172.21.15.144:0/3970800397 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7feb2806e200 0x7feb280706b0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:45.182 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:45.179+0000 7feb45e7e700 1 --2- 172.21.15.144:0/3970800397 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7feb401003d0 0x7feb40195db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:45.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:45.179+0000 7feb45e7e700 1 --2- 172.21.15.144:0/3970800397 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7feb40100d70 0x7feb401962f0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:45.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:45.179+0000 7feb45e7e700 1 -- 172.21.15.144:0/3970800397 >> 172.21.15.144:0/3970800397 conn(0x7feb400fbc50 msgr2=0x7feb400fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:45.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:45.180+0000 7feb45e7e700 1 -- 172.21.15.144:0/3970800397 shutdown_connections 2024-09-10T12:11:45.183 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:45.180+0000 7feb45e7e700 1 -- 172.21.15.144:0/3970800397 wait complete. 2024-09-10T12:11:45.183 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 37 2024-09-10T12:11:45.775 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:45 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3970800397' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-09-10T12:11:45.796 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 38 2024-09-10T12:11:45.995 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:46.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:45 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3970800397' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-09-10T12:11:46.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:46 smithi144 ceph-mon[87436]: pgmap v1196: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:46 smithi178 ceph-mon[65940]: pgmap v1196: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:47.564 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.562+0000 7f24f2227700 1 -- 172.21.15.144:0/1570182488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec100d70 msgr2=0x7f24ec104d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:47.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.562+0000 7f24f2227700 1 --2- 172.21.15.144:0/1570182488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec100d70 0x7f24ec104d40 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f24e0009910 tx=0x7f24e0009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:47.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.562+0000 7f24f2227700 1 -- 172.21.15.144:0/1570182488 shutdown_connections 2024-09-10T12:11:47.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.562+0000 7f24f2227700 1 --2- 172.21.15.144:0/1570182488 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec100d70 0x7f24ec104d40 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:47.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.562+0000 7f24f2227700 1 --2- 172.21.15.144:0/1570182488 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24ec1003d0 0x7f24ec1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:47.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.562+0000 7f24f2227700 1 -- 172.21.15.144:0/1570182488 >> 172.21.15.144:0/1570182488 conn(0x7f24ec0fbc90 msgr2=0x7f24ec0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:47.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.562+0000 7f24f2227700 1 -- 172.21.15.144:0/1570182488 shutdown_connections 2024-09-10T12:11:47.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.562+0000 7f24f2227700 1 -- 172.21.15.144:0/1570182488 wait complete. 2024-09-10T12:11:47.565 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.562+0000 7f24f2227700 1 Processor -- start 2024-09-10T12:11:47.566 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.563+0000 7f24f2227700 1 -- start start 2024-09-10T12:11:47.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.563+0000 7f24f2227700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24ec1003d0 0x7f24ec195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:47.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.563+0000 7f24f2227700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec100d70 0x7f24ec196280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:47.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.563+0000 7f24f2227700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec1968d0 con 0x7f24ec100d70 2024-09-10T12:11:47.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.563+0000 7f24f2227700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24ec196a10 con 0x7f24ec1003d0 2024-09-10T12:11:47.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.563+0000 7f24eaffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec100d70 0x7f24ec196280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:47.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.563+0000 7f24eaffd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec100d70 0x7f24ec196280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36288/0 (socket says 172.21.15.144:36288) 2024-09-10T12:11:47.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.563+0000 7f24eaffd700 1 -- 172.21.15.144:0/3793368451 learned_addr learned my addr 172.21.15.144:0/3793368451 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:47.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.563+0000 7f24eb7fe700 1 --2- 172.21.15.144:0/3793368451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24ec1003d0 0x7f24ec195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:47.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.563+0000 7f24eaffd700 1 -- 172.21.15.144:0/3793368451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24ec1003d0 msgr2=0x7f24ec195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:47.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.563+0000 7f24eaffd700 1 --2- 172.21.15.144:0/3793368451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24ec1003d0 0x7f24ec195d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:47.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.563+0000 7f24eaffd700 1 -- 172.21.15.144:0/3793368451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e00095f0 con 0x7f24ec100d70 2024-09-10T12:11:47.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.563+0000 7f24eaffd700 1 --2- 172.21.15.144:0/3793368451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec100d70 0x7f24ec196280 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f24e000bda0 tx=0x7f24e000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:47.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.564+0000 7f24e8ff9700 1 -- 172.21.15.144:0/3793368451 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24e001d070 con 0x7f24ec100d70 2024-09-10T12:11:47.567 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.564+0000 7f24e8ff9700 1 -- 172.21.15.144:0/3793368451 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f24e0004fb0 con 0x7f24ec100d70 2024-09-10T12:11:47.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.564+0000 7f24f2227700 1 -- 172.21.15.144:0/3793368451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24ec19a800 con 0x7f24ec100d70 2024-09-10T12:11:47.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.564+0000 7f24e8ff9700 1 -- 172.21.15.144:0/3793368451 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24e000f480 con 0x7f24ec100d70 2024-09-10T12:11:47.568 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.564+0000 7f24f2227700 1 -- 172.21.15.144:0/3793368451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24ec19acf0 con 0x7f24ec100d70 2024-09-10T12:11:47.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.565+0000 7f24f2227700 1 -- 172.21.15.144:0/3793368451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24ec04ea50 con 0x7f24ec100d70 2024-09-10T12:11:47.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.565+0000 7f24e8ff9700 1 -- 172.21.15.144:0/3793368451 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f24e0022870 con 0x7f24ec100d70 2024-09-10T12:11:47.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.566+0000 7f24e8ff9700 1 --2- 172.21.15.144:0/3793368451 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f24d4072610 0x7f24d4074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:47.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.566+0000 7f24e8ff9700 1 -- 172.21.15.144:0/3793368451 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f24e008f5f0 con 0x7f24ec100d70 2024-09-10T12:11:47.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.566+0000 7f24eb7fe700 1 --2- 172.21.15.144:0/3793368451 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f24d4072610 0x7f24d4074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:47.569 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.566+0000 7f24eb7fe700 1 --2- 172.21.15.144:0/3793368451 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f24d4072610 0x7f24d4074ac0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00ba60 tx=0x7f24dc005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:47.572 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.570+0000 7f24e8ff9700 1 -- 172.21.15.144:0/3793368451 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f24e0058a80 con 0x7f24ec100d70 2024-09-10T12:11:47.820 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.818+0000 7f24f2227700 1 -- 172.21.15.144:0/3793368451 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f24ec1082a0 con 0x7f24ec100d70 2024-09-10T12:11:47.821 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.818+0000 7f24e8ff9700 1 -- 172.21.15.144:0/3793368451 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v42) v1 ==== 107+0+5271 (secure 0 0 0) 0x7f24e005c140 con 0x7f24ec100d70 2024-09-10T12:11:47.821 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:47.821 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":38,"btime":"2024-09-10T12:08:29:638727+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44355,"name":"cephfs.smithi178.nmhrmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6825/706731549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":706731549},{"type":"v1","addr":"172.21.15.178:6825","nonce":706731549}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44887,"name":"cephfs.smithi178.czjrsa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6827/2967786976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2967786976},{"type":"v1","addr":"172.21.15.178:6827","nonce":2967786976}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T12:08:28.700037+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34390,"mds_1":34422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34390":{"gid":34390,"name":"cephfs.smithi144.bfyvwv","rank":0,"incarnation":36,"state":"up:resolve","state_seq":463,"addr":"172.21.15.144:6827/2724506882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2724506882},{"type":"v1","addr":"172.21.15.144:6827","nonce":2724506882}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34422":{"gid":34422,"name":"cephfs.smithi144.jaoyvs","rank":1,"incarnation":36,"state":"up:resolve","state_seq":459,"addr":"172.21.15.144:6829/4254390011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4254390011},{"type":"v1","addr":"172.21.15.144:6829","nonce":4254390011}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:47.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.821+0000 7f24f2227700 1 -- 172.21.15.144:0/3793368451 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f24d4072610 msgr2=0x7f24d4074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:47.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.821+0000 7f24f2227700 1 --2- 172.21.15.144:0/3793368451 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f24d4072610 0x7f24d4074ac0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f24dc00ba60 tx=0x7f24dc005d50 comp rx=0 tx=0).stop 2024-09-10T12:11:47.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.821+0000 7f24f2227700 1 -- 172.21.15.144:0/3793368451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec100d70 msgr2=0x7f24ec196280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:47.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.821+0000 7f24f2227700 1 --2- 172.21.15.144:0/3793368451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec100d70 0x7f24ec196280 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f24e000bda0 tx=0x7f24e000be80 comp rx=0 tx=0).stop 2024-09-10T12:11:47.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.821+0000 7f24f2227700 1 -- 172.21.15.144:0/3793368451 shutdown_connections 2024-09-10T12:11:47.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.821+0000 7f24f2227700 1 --2- 172.21.15.144:0/3793368451 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f24d4072610 0x7f24d4074ac0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:47.823 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.821+0000 7f24f2227700 1 --2- 172.21.15.144:0/3793368451 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f24ec1003d0 0x7f24ec195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:47.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.821+0000 7f24f2227700 1 --2- 172.21.15.144:0/3793368451 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f24ec100d70 0x7f24ec196280 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:47.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.821+0000 7f24f2227700 1 -- 172.21.15.144:0/3793368451 >> 172.21.15.144:0/3793368451 conn(0x7f24ec0fbc90 msgr2=0x7f24ec0fe0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:47.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.821+0000 7f24f2227700 1 -- 172.21.15.144:0/3793368451 shutdown_connections 2024-09-10T12:11:47.824 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:47.821+0000 7f24f2227700 1 -- 172.21.15.144:0/3793368451 wait complete. 2024-09-10T12:11:47.824 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 38 2024-09-10T12:11:48.402 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 39 2024-09-10T12:11:48.601 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:48 smithi144 ceph-mon[87436]: pgmap v1197: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:11:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:48 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/3793368451' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-09-10T12:11:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:48 smithi178 ceph-mon[65940]: pgmap v1197: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:11:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:48 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/3793368451' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-09-10T12:11:50.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.199+0000 7fa242266700 1 -- 172.21.15.144:0/380766340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa23c0ffcb0 msgr2=0x7fa23c103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:50.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.199+0000 7fa242266700 1 --2- 172.21.15.144:0/380766340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa23c0ffcb0 0x7fa23c103c80 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fa230009910 tx=0x7fa230009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:50.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.199+0000 7fa242266700 1 -- 172.21.15.144:0/380766340 shutdown_connections 2024-09-10T12:11:50.203 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.199+0000 7fa242266700 1 --2- 172.21.15.144:0/380766340 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa23c0ffcb0 0x7fa23c103c80 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:50.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.199+0000 7fa242266700 1 --2- 172.21.15.144:0/380766340 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa23c0ff310 0x7fa23c0ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:50.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.199+0000 7fa242266700 1 -- 172.21.15.144:0/380766340 >> 172.21.15.144:0/380766340 conn(0x7fa23c0fabb0 msgr2=0x7fa23c0fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:50.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.199+0000 7fa242266700 1 -- 172.21.15.144:0/380766340 shutdown_connections 2024-09-10T12:11:50.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.200+0000 7fa242266700 1 -- 172.21.15.144:0/380766340 wait complete. 2024-09-10T12:11:50.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.200+0000 7fa242266700 1 Processor -- start 2024-09-10T12:11:50.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.200+0000 7fa242266700 1 -- start start 2024-09-10T12:11:50.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa242266700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa23c0ff310 0x7fa23c196bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:50.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa242266700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa23c0ffcb0 0x7fa23c197130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:50.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa242266700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c1976c0 con 0x7fa23c0ff310 2024-09-10T12:11:50.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa242266700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c197800 con 0x7fa23c0ffcb0 2024-09-10T12:11:50.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa23bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa23c0ff310 0x7fa23c196bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:50.204 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa23bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa23c0ff310 0x7fa23c196bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:36300/0 (socket says 172.21.15.144:36300) 2024-09-10T12:11:50.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa23bfff700 1 -- 172.21.15.144:0/2438185631 learned_addr learned my addr 172.21.15.144:0/2438185631 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:50.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa23b7fe700 1 --2- 172.21.15.144:0/2438185631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa23c0ffcb0 0x7fa23c197130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:50.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa23bfff700 1 -- 172.21.15.144:0/2438185631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa23c0ffcb0 msgr2=0x7fa23c197130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:50.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa23bfff700 1 --2- 172.21.15.144:0/2438185631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa23c0ffcb0 0x7fa23c197130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:50.205 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa23bfff700 1 -- 172.21.15.144:0/2438185631 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2300095f0 con 0x7fa23c0ff310 2024-09-10T12:11:50.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa23bfff700 1 --2- 172.21.15.144:0/2438185631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa23c0ff310 0x7fa23c196bf0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00c930 tx=0x7fa22c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:50.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa2397fa700 1 -- 172.21.15.144:0/2438185631 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa22c007ad0 con 0x7fa23c0ff310 2024-09-10T12:11:50.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa2397fa700 1 -- 172.21.15.144:0/2438185631 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa22c00ce80 con 0x7fa23c0ff310 2024-09-10T12:11:50.206 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa242266700 1 -- 172.21.15.144:0/2438185631 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa23c19fc60 con 0x7fa23c0ff310 2024-09-10T12:11:50.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.201+0000 7fa2397fa700 1 -- 172.21.15.144:0/2438185631 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa22c018630 con 0x7fa23c0ff310 2024-09-10T12:11:50.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.202+0000 7fa242266700 1 -- 172.21.15.144:0/2438185631 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23c1a0070 con 0x7fa23c0ff310 2024-09-10T12:11:50.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.203+0000 7fa242266700 1 -- 172.21.15.144:0/2438185631 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa23c107680 con 0x7fa23c0ff310 2024-09-10T12:11:50.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.203+0000 7fa2397fa700 1 -- 172.21.15.144:0/2438185631 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fa22c007c30 con 0x7fa23c0ff310 2024-09-10T12:11:50.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.203+0000 7fa2397fa700 1 --2- 172.21.15.144:0/2438185631 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa22406e200 0x7fa2240706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:50.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.204+0000 7fa2397fa700 1 -- 172.21.15.144:0/2438185631 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fa22c08e080 con 0x7fa23c0ff310 2024-09-10T12:11:50.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.204+0000 7fa23b7fe700 1 --2- 172.21.15.144:0/2438185631 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa22406e200 0x7fa2240706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:50.207 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.204+0000 7fa23b7fe700 1 --2- 172.21.15.144:0/2438185631 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa22406e200 0x7fa2240706b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa230006010 tx=0x7fa23001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:50.210 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.209+0000 7fa2397fa700 1 -- 172.21.15.144:0/2438185631 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fa22c057460 con 0x7fa23c0ff310 2024-09-10T12:11:50.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.455+0000 7fa242266700 1 -- 172.21.15.144:0/2438185631 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 39, "format": "json"} v 0) v1 -- 0x7fa23c04ea50 con 0x7fa23c0ff310 2024-09-10T12:11:50.457 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.455+0000 7fa2397fa700 1 -- 172.21.15.144:0/2438185631 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 39, "format": "json"}]=0 dumped fsmap epoch 39 v42) v1 ==== 107+0+5275 (secure 0 0 0) 0x7fa22c05ab20 con 0x7fa23c0ff310 2024-09-10T12:11:50.458 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:50.458 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":39,"btime":"2024-09-10T12:08:30:644306+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44355,"name":"cephfs.smithi178.nmhrmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6825/706731549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":706731549},{"type":"v1","addr":"172.21.15.178:6825","nonce":706731549}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44887,"name":"cephfs.smithi178.czjrsa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6827/2967786976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2967786976},{"type":"v1","addr":"172.21.15.178:6827","nonce":2967786976}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T12:08:29.687841+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34390,"mds_1":34422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34390":{"gid":34390,"name":"cephfs.smithi144.bfyvwv","rank":0,"incarnation":36,"state":"up:reconnect","state_seq":464,"addr":"172.21.15.144:6827/2724506882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2724506882},{"type":"v1","addr":"172.21.15.144:6827","nonce":2724506882}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34422":{"gid":34422,"name":"cephfs.smithi144.jaoyvs","rank":1,"incarnation":36,"state":"up:reconnect","state_seq":460,"addr":"172.21.15.144:6829/4254390011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4254390011},{"type":"v1","addr":"172.21.15.144:6829","nonce":4254390011}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:50.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.458+0000 7fa242266700 1 -- 172.21.15.144:0/2438185631 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa22406e200 msgr2=0x7fa2240706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:50.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.458+0000 7fa242266700 1 --2- 172.21.15.144:0/2438185631 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa22406e200 0x7fa2240706b0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fa230006010 tx=0x7fa23001a040 comp rx=0 tx=0).stop 2024-09-10T12:11:50.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.458+0000 7fa242266700 1 -- 172.21.15.144:0/2438185631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa23c0ff310 msgr2=0x7fa23c196bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:50.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.458+0000 7fa242266700 1 --2- 172.21.15.144:0/2438185631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa23c0ff310 0x7fa23c196bf0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00c930 tx=0x7fa22c00cc40 comp rx=0 tx=0).stop 2024-09-10T12:11:50.460 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.458+0000 7fa242266700 1 -- 172.21.15.144:0/2438185631 shutdown_connections 2024-09-10T12:11:50.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.458+0000 7fa242266700 1 --2- 172.21.15.144:0/2438185631 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fa22406e200 0x7fa2240706b0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:50.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.458+0000 7fa242266700 1 --2- 172.21.15.144:0/2438185631 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fa23c0ff310 0x7fa23c196bf0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:50.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.458+0000 7fa242266700 1 --2- 172.21.15.144:0/2438185631 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fa23c0ffcb0 0x7fa23c197130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:50.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.458+0000 7fa242266700 1 -- 172.21.15.144:0/2438185631 >> 172.21.15.144:0/2438185631 conn(0x7fa23c0fabb0 msgr2=0x7fa23c0fcf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:50.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.458+0000 7fa242266700 1 -- 172.21.15.144:0/2438185631 shutdown_connections 2024-09-10T12:11:50.461 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:50.458+0000 7fa242266700 1 -- 172.21.15.144:0/2438185631 wait complete. 2024-09-10T12:11:50.461 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 39 2024-09-10T12:11:50.770 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:50 smithi144 ceph-mon[87436]: pgmap v1198: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:11:50.770 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:50 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2438185631' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-09-10T12:11:51.066 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 40 2024-09-10T12:11:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:50 smithi178 ceph-mon[65940]: pgmap v1198: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:11:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:50 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2438185631' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 39, "format": "json"}]: dispatch 2024-09-10T12:11:51.265 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:52.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.882+0000 7f978a4d1700 1 -- 172.21.15.144:0/2705932421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784101390 msgr2=0x7f9784101760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:52.885 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.882+0000 7f978a4d1700 1 --2- 172.21.15.144:0/2705932421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784101390 0x7f9784101760 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f9774009910 tx=0x7f9774009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:52.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.882+0000 7f978a4d1700 1 -- 172.21.15.144:0/2705932421 shutdown_connections 2024-09-10T12:11:52.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.882+0000 7f978a4d1700 1 --2- 172.21.15.144:0/2705932421 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9784101ca0 0x7f978410a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:52.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.882+0000 7f978a4d1700 1 --2- 172.21.15.144:0/2705932421 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784101390 0x7f9784101760 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:52.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.882+0000 7f978a4d1700 1 -- 172.21.15.144:0/2705932421 >> 172.21.15.144:0/2705932421 conn(0x7f97840fabb0 msgr2=0x7f97840fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:52.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.882+0000 7f978a4d1700 1 -- 172.21.15.144:0/2705932421 shutdown_connections 2024-09-10T12:11:52.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.882+0000 7f978a4d1700 1 -- 172.21.15.144:0/2705932421 wait complete. 2024-09-10T12:11:52.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.883+0000 7f978a4d1700 1 Processor -- start 2024-09-10T12:11:52.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.883+0000 7f978a4d1700 1 -- start start 2024-09-10T12:11:52.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.883+0000 7f978a4d1700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9784101390 0x7f97840feff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:52.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.883+0000 7f978a4d1700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784101ca0 0x7f97840ff530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:52.886 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.883+0000 7f978a4d1700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97840ffa70 con 0x7f9784101ca0 2024-09-10T12:11:52.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.883+0000 7f978a4d1700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97840ffbb0 con 0x7f9784101390 2024-09-10T12:11:52.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.883+0000 7f977bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784101ca0 0x7f97840ff530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:52.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.883+0000 7f977bfff700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784101ca0 0x7f97840ff530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34494/0 (socket says 172.21.15.144:34494) 2024-09-10T12:11:52.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.883+0000 7f977bfff700 1 -- 172.21.15.144:0/2065645469 learned_addr learned my addr 172.21.15.144:0/2065645469 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:52.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.883+0000 7f9783fff700 1 --2- 172.21.15.144:0/2065645469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9784101390 0x7f97840feff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:52.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.884+0000 7f977bfff700 1 -- 172.21.15.144:0/2065645469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9784101390 msgr2=0x7f97840feff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:52.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.884+0000 7f977bfff700 1 --2- 172.21.15.144:0/2065645469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9784101390 0x7f97840feff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:52.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.884+0000 7f977bfff700 1 -- 172.21.15.144:0/2065645469 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97740095f0 con 0x7f9784101ca0 2024-09-10T12:11:52.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.884+0000 7f977bfff700 1 --2- 172.21.15.144:0/2065645469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784101ca0 0x7f97840ff530 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f976c00d900 tx=0x7f976c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:52.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.884+0000 7f9781ffb700 1 -- 172.21.15.144:0/2065645469 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f976c0041f0 con 0x7f9784101ca0 2024-09-10T12:11:52.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.884+0000 7f978a4d1700 1 -- 172.21.15.144:0/2065645469 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97841a1f20 con 0x7f9784101ca0 2024-09-10T12:11:52.887 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.884+0000 7f9781ffb700 1 -- 172.21.15.144:0/2065645469 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f976c004cf0 con 0x7f9784101ca0 2024-09-10T12:11:52.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.884+0000 7f9781ffb700 1 -- 172.21.15.144:0/2065645469 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f976c00b750 con 0x7f9784101ca0 2024-09-10T12:11:52.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.884+0000 7f978a4d1700 1 -- 172.21.15.144:0/2065645469 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97841a2330 con 0x7f9784101ca0 2024-09-10T12:11:52.888 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.885+0000 7f978a4d1700 1 -- 172.21.15.144:0/2065645469 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9784107870 con 0x7f9784101ca0 2024-09-10T12:11:52.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.885+0000 7f9781ffb700 1 -- 172.21.15.144:0/2065645469 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f976c004350 con 0x7f9784101ca0 2024-09-10T12:11:52.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.886+0000 7f9781ffb700 1 --2- 172.21.15.144:0/2065645469 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f977006e200 0x7f97700706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:52.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.886+0000 7f9781ffb700 1 -- 172.21.15.144:0/2065645469 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f976c08e1a0 con 0x7f9784101ca0 2024-09-10T12:11:52.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.886+0000 7f9783fff700 1 --2- 172.21.15.144:0/2065645469 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f977006e200 0x7f97700706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:52.889 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.886+0000 7f9783fff700 1 --2- 172.21.15.144:0/2065645469 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f977006e200 0x7f97700706b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f977400b5c0 tx=0x7f9774005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:52.892 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:52.890+0000 7f9781ffb700 1 -- 172.21.15.144:0/2065645469 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f976c057580 con 0x7f9784101ca0 2024-09-10T12:11:52.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:52 smithi144 ceph-mon[87436]: pgmap v1199: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:52 smithi178 ceph-mon[65940]: pgmap v1199: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:53.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:53.140+0000 7f978a4d1700 1 -- 172.21.15.144:0/2065645469 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 40, "format": "json"} v 0) v1 -- 0x7f978404ea50 con 0x7f9784101ca0 2024-09-10T12:11:53.142 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:53.141+0000 7f9781ffb700 1 -- 172.21.15.144:0/2065645469 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 40, "format": "json"}]=0 dumped fsmap epoch 40 v42) v1 ==== 107+0+5269 (secure 0 0 0) 0x7f976c05ac40 con 0x7f9784101ca0 2024-09-10T12:11:53.143 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:53.143 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":40,"btime":"2024-09-10T12:08:31:649491+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44355,"name":"cephfs.smithi178.nmhrmh","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6825/706731549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":706731549},{"type":"v1","addr":"172.21.15.178:6825","nonce":706731549}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":32},{"gid":44887,"name":"cephfs.smithi178.czjrsa","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.178:6827/2967786976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2967786976},{"type":"v1","addr":"172.21.15.178:6827","nonce":2967786976}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":40,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T12:08:30.654288+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34390,"mds_1":34422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34390":{"gid":34390,"name":"cephfs.smithi144.bfyvwv","rank":0,"incarnation":36,"state":"up:rejoin","state_seq":465,"addr":"172.21.15.144:6827/2724506882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2724506882},{"type":"v1","addr":"172.21.15.144:6827","nonce":2724506882}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34422":{"gid":34422,"name":"cephfs.smithi144.jaoyvs","rank":1,"incarnation":36,"state":"up:rejoin","state_seq":461,"addr":"172.21.15.144:6829/4254390011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4254390011},{"type":"v1","addr":"172.21.15.144:6829","nonce":4254390011}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-09-10T12:11:53.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:53.143+0000 7f978a4d1700 1 -- 172.21.15.144:0/2065645469 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f977006e200 msgr2=0x7f97700706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:53.145 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:53.143+0000 7f978a4d1700 1 --2- 172.21.15.144:0/2065645469 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f977006e200 0x7f97700706b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f977400b5c0 tx=0x7f9774005960 comp rx=0 tx=0).stop 2024-09-10T12:11:53.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:53.143+0000 7f978a4d1700 1 -- 172.21.15.144:0/2065645469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784101ca0 msgr2=0x7f97840ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:53.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:53.143+0000 7f978a4d1700 1 --2- 172.21.15.144:0/2065645469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784101ca0 0x7f97840ff530 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f976c00d900 tx=0x7f976c00dcc0 comp rx=0 tx=0).stop 2024-09-10T12:11:53.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:53.143+0000 7f978a4d1700 1 -- 172.21.15.144:0/2065645469 shutdown_connections 2024-09-10T12:11:53.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:53.143+0000 7f978a4d1700 1 --2- 172.21.15.144:0/2065645469 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f977006e200 0x7f97700706b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:53.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:53.143+0000 7f978a4d1700 1 --2- 172.21.15.144:0/2065645469 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f9784101390 0x7f97840feff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:53.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:53.143+0000 7f978a4d1700 1 --2- 172.21.15.144:0/2065645469 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f9784101ca0 0x7f97840ff530 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:53.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:53.143+0000 7f978a4d1700 1 -- 172.21.15.144:0/2065645469 >> 172.21.15.144:0/2065645469 conn(0x7f97840fabb0 msgr2=0x7f97841049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:53.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:53.143+0000 7f978a4d1700 1 -- 172.21.15.144:0/2065645469 shutdown_connections 2024-09-10T12:11:53.146 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:53.143+0000 7f978a4d1700 1 -- 172.21.15.144:0/2065645469 wait complete. 2024-09-10T12:11:53.146 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 40 2024-09-10T12:11:53.834 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:53 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/2065645469' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-09-10T12:11:53.858 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph fs dump --format=json 41 2024-09-10T12:11:54.058 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:54.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:53 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/2065645469' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 40, "format": "json"}]: dispatch 2024-09-10T12:11:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:54 smithi178 ceph-mon[65940]: pgmap v1200: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:11:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:11:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:11:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:11:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:11:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:54 smithi144 ceph-mon[87436]: pgmap v1200: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:11:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:11:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:11:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:11:55.185 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:11:55.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.715+0000 7fd0ef6b9700 1 -- 172.21.15.144:0/864250461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e8101c80 msgr2=0x7fd0e810a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:55.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.715+0000 7fd0ef6b9700 1 --2- 172.21.15.144:0/864250461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e8101c80 0x7fd0e810a180 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8009910 tx=0x7fd0d8009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:55.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.715+0000 7fd0ef6b9700 1 -- 172.21.15.144:0/864250461 shutdown_connections 2024-09-10T12:11:55.717 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.715+0000 7fd0ef6b9700 1 --2- 172.21.15.144:0/864250461 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e8101c80 0x7fd0e810a180 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:55.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.715+0000 7fd0ef6b9700 1 --2- 172.21.15.144:0/864250461 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e8101370 0x7fd0e8101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:55.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.715+0000 7fd0ef6b9700 1 -- 172.21.15.144:0/864250461 >> 172.21.15.144:0/864250461 conn(0x7fd0e80fabb0 msgr2=0x7fd0e80fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:55.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.715+0000 7fd0ef6b9700 1 -- 172.21.15.144:0/864250461 shutdown_connections 2024-09-10T12:11:55.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.715+0000 7fd0ef6b9700 1 -- 172.21.15.144:0/864250461 wait complete. 2024-09-10T12:11:55.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.716+0000 7fd0ef6b9700 1 Processor -- start 2024-09-10T12:11:55.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.716+0000 7fd0ef6b9700 1 -- start start 2024-09-10T12:11:55.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.716+0000 7fd0ef6b9700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e8101370 0x7fd0e80fefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:55.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.716+0000 7fd0ef6b9700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e8101c80 0x7fd0e80ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:55.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.716+0000 7fd0ef6b9700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80ffa50 con 0x7fd0e8101370 2024-09-10T12:11:55.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.716+0000 7fd0ef6b9700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e80ffb90 con 0x7fd0e8101c80 2024-09-10T12:11:55.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.716+0000 7fd0ed455700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e8101370 0x7fd0e80fefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:55.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.716+0000 7fd0ed455700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e8101370 0x7fd0e80fefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34512/0 (socket says 172.21.15.144:34512) 2024-09-10T12:11:55.718 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.716+0000 7fd0ed455700 1 -- 172.21.15.144:0/993504306 learned_addr learned my addr 172.21.15.144:0/993504306 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:55.719 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.716+0000 7fd0ecc54700 1 --2- 172.21.15.144:0/993504306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e8101c80 0x7fd0e80ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:55.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.716+0000 7fd0ed455700 1 -- 172.21.15.144:0/993504306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e8101c80 msgr2=0x7fd0e80ff510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:55.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.716+0000 7fd0ed455700 1 --2- 172.21.15.144:0/993504306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e8101c80 0x7fd0e80ff510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:55.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.716+0000 7fd0ed455700 1 -- 172.21.15.144:0/993504306 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d80095f0 con 0x7fd0e8101370 2024-09-10T12:11:55.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.716+0000 7fd0ecc54700 1 --2- 172.21.15.144:0/993504306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e8101c80 0x7fd0e80ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T12:11:55.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.716+0000 7fd0ed455700 1 --2- 172.21.15.144:0/993504306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e8101370 0x7fd0e80fefd0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400d8d0 tx=0x7fd0e400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:55.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.717+0000 7fd0de7fc700 1 -- 172.21.15.144:0/993504306 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0e4009940 con 0x7fd0e8101370 2024-09-10T12:11:55.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.717+0000 7fd0de7fc700 1 -- 172.21.15.144:0/993504306 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd0e4010460 con 0x7fd0e8101370 2024-09-10T12:11:55.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.717+0000 7fd0de7fc700 1 -- 172.21.15.144:0/993504306 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0e400f5f0 con 0x7fd0e8101370 2024-09-10T12:11:55.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.717+0000 7fd0ef6b9700 1 -- 172.21.15.144:0/993504306 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e81a1ec0 con 0x7fd0e8101370 2024-09-10T12:11:55.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.717+0000 7fd0ef6b9700 1 -- 172.21.15.144:0/993504306 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e81a22d0 con 0x7fd0e8101370 2024-09-10T12:11:55.720 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.718+0000 7fd0ef6b9700 1 -- 172.21.15.144:0/993504306 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e81077f0 con 0x7fd0e8101370 2024-09-10T12:11:55.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.718+0000 7fd0de7fc700 1 -- 172.21.15.144:0/993504306 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fd0e40105d0 con 0x7fd0e8101370 2024-09-10T12:11:55.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.719+0000 7fd0de7fc700 1 --2- 172.21.15.144:0/993504306 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd0d406e200 0x7fd0d40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:55.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.719+0000 7fd0ecc54700 1 --2- 172.21.15.144:0/993504306 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd0d406e200 0x7fd0d40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:55.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.719+0000 7fd0de7fc700 1 -- 172.21.15.144:0/993504306 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fd0e408e0a0 con 0x7fd0e8101370 2024-09-10T12:11:55.721 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.719+0000 7fd0ecc54700 1 --2- 172.21.15.144:0/993504306 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd0d406e200 0x7fd0d40706b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800b5c0 tx=0x7fd0d801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:55.725 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.723+0000 7fd0de7fc700 1 -- 172.21.15.144:0/993504306 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fd0e4057480 con 0x7fd0e8101370 2024-09-10T12:11:55.971 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.970+0000 7fd0ef6b9700 1 -- 172.21.15.144:0/993504306 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 41, "format": "json"} v 0) v1 -- 0x7fd0e80ffd50 con 0x7fd0e8101370 2024-09-10T12:11:55.972 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.971+0000 7fd0de7fc700 1 -- 172.21.15.144:0/993504306 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 41, "format": "json"}]=0 dumped fsmap epoch 41 v42) v1 ==== 107+0+5299 (secure 0 0 0) 0x7fd0e405ab40 con 0x7fd0e8101370 2024-09-10T12:11:55.972 INFO:teuthology.orchestra.run.smithi144.stdout: 2024-09-10T12:11:55.972 INFO:teuthology.orchestra.run.smithi144.stdout:{"epoch":41,"btime":"2024-09-10T12:08:32:615743+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-10T11:26:34.004497+0000","modified":"2024-09-10T12:08:32.615741+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":80,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34390,"mds_1":34422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34390":{"gid":34390,"name":"cephfs.smithi144.bfyvwv","rank":0,"incarnation":36,"state":"up:active","state_seq":466,"addr":"172.21.15.144:6827/2724506882","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6826","nonce":2724506882},{"type":"v1","addr":"172.21.15.144:6827","nonce":2724506882}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34422":{"gid":34422,"name":"cephfs.smithi144.jaoyvs","rank":1,"incarnation":36,"state":"up:active","state_seq":462,"addr":"172.21.15.144:6829/4254390011","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.144:6828","nonce":4254390011},{"type":"v1","addr":"172.21.15.144:6829","nonce":4254390011}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44355":{"gid":44355,"name":"cephfs.smithi178.nmhrmh","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6825/706731549","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6824","nonce":706731549},{"type":"v1","addr":"172.21.15.178:6825","nonce":706731549}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_44887":{"gid":44887,"name":"cephfs.smithi178.czjrsa","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.178:6827/2967786976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.178:6826","nonce":2967786976},{"type":"v1","addr":"172.21.15.178:6827","nonce":2967786976}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34390,"qdb_cluster":[34390,34422]},"id":1}]} 2024-09-10T12:11:55.974 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.973+0000 7fd0ef6b9700 1 -- 172.21.15.144:0/993504306 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd0d406e200 msgr2=0x7fd0d40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:55.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.973+0000 7fd0ef6b9700 1 --2- 172.21.15.144:0/993504306 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd0d406e200 0x7fd0d40706b0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fd0d800b5c0 tx=0x7fd0d801a040 comp rx=0 tx=0).stop 2024-09-10T12:11:55.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.973+0000 7fd0ef6b9700 1 -- 172.21.15.144:0/993504306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e8101370 msgr2=0x7fd0e80fefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:55.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.973+0000 7fd0ef6b9700 1 --2- 172.21.15.144:0/993504306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e8101370 0x7fd0e80fefd0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd0e400d8d0 tx=0x7fd0e400dc90 comp rx=0 tx=0).stop 2024-09-10T12:11:55.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.973+0000 7fd0ef6b9700 1 -- 172.21.15.144:0/993504306 shutdown_connections 2024-09-10T12:11:55.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.973+0000 7fd0ef6b9700 1 --2- 172.21.15.144:0/993504306 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fd0d406e200 0x7fd0d40706b0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:55.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.973+0000 7fd0ef6b9700 1 --2- 172.21.15.144:0/993504306 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fd0e8101370 0x7fd0e80fefd0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:55.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.973+0000 7fd0ef6b9700 1 --2- 172.21.15.144:0/993504306 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fd0e8101c80 0x7fd0e80ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:55.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.973+0000 7fd0ef6b9700 1 -- 172.21.15.144:0/993504306 >> 172.21.15.144:0/993504306 conn(0x7fd0e80fabb0 msgr2=0x7fd0e8104930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:55.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.973+0000 7fd0ef6b9700 1 -- 172.21.15.144:0/993504306 shutdown_connections 2024-09-10T12:11:55.975 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:55.973+0000 7fd0ef6b9700 1 -- 172.21.15.144:0/993504306 wait complete. 2024-09-10T12:11:55.976 INFO:teuthology.orchestra.run.smithi144.stderr:dumped fsmap epoch 41 2024-09-10T12:11:56.576 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2024-09-10T12:11:56.587 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2024-09-10T12:11:56.587 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T12:11:56.587 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T12:11:56.607 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T12:11:56.607 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T12:11:56.665 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T12:11:56.806 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:56 smithi144 ceph-mon[87436]: pgmap v1201: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:56.806 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:56 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/993504306' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-09-10T12:11:56.916 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:56 smithi178 ceph-mon[65940]: pgmap v1201: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:11:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:56 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/993504306' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 41, "format": "json"}]: dispatch 2024-09-10T12:11:58.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.612+0000 7f4848762700 1 -- 172.21.15.144:0/3851307165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48400ffd30 msgr2=0x7f4840103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:58.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.612+0000 7f4848762700 1 --2- 172.21.15.144:0/3851307165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48400ffd30 0x7f4840103ce0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f483c009910 tx=0x7f483c009c20 comp rx=0 tx=0).stop 2024-09-10T12:11:58.615 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.612+0000 7f4848762700 1 -- 172.21.15.144:0/3851307165 shutdown_connections 2024-09-10T12:11:58.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.612+0000 7f4848762700 1 --2- 172.21.15.144:0/3851307165 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48400ffd30 0x7f4840103ce0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:58.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.612+0000 7f4848762700 1 --2- 172.21.15.144:0/3851307165 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48400ff390 0x7f48400ff760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:58.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.612+0000 7f4848762700 1 -- 172.21.15.144:0/3851307165 >> 172.21.15.144:0/3851307165 conn(0x7f4840074f50 msgr2=0x7f4840075350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:58.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.612+0000 7f4848762700 1 -- 172.21.15.144:0/3851307165 shutdown_connections 2024-09-10T12:11:58.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.612+0000 7f4848762700 1 -- 172.21.15.144:0/3851307165 wait complete. 2024-09-10T12:11:58.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.613+0000 7f4848762700 1 Processor -- start 2024-09-10T12:11:58.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.613+0000 7f4848762700 1 -- start start 2024-09-10T12:11:58.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.613+0000 7f4848762700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48400ff390 0x7f4840195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:58.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.613+0000 7f4848762700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48400ffd30 0x7f4840196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:58.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.613+0000 7f4848762700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4840196a60 con 0x7f48400ffd30 2024-09-10T12:11:58.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.613+0000 7f4848762700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f484019a7f0 con 0x7f48400ff390 2024-09-10T12:11:58.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.613+0000 7f4845cfd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48400ffd30 0x7f4840196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:58.616 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.613+0000 7f4845cfd700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48400ffd30 0x7f4840196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34528/0 (socket says 172.21.15.144:34528) 2024-09-10T12:11:58.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.613+0000 7f4845cfd700 1 -- 172.21.15.144:0/1510519577 learned_addr learned my addr 172.21.15.144:0/1510519577 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:11:58.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.613+0000 7f48464fe700 1 --2- 172.21.15.144:0/1510519577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48400ff390 0x7f4840195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:58.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.614+0000 7f4845cfd700 1 -- 172.21.15.144:0/1510519577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48400ff390 msgr2=0x7f4840195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:58.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.614+0000 7f4845cfd700 1 --2- 172.21.15.144:0/1510519577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48400ff390 0x7f4840195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:58.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.614+0000 7f4845cfd700 1 -- 172.21.15.144:0/1510519577 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c0095f0 con 0x7f48400ffd30 2024-09-10T12:11:58.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.614+0000 7f4845cfd700 1 --2- 172.21.15.144:0/1510519577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48400ffd30 0x7f4840196380 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f483c0092e0 tx=0x7f483c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:58.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.614+0000 7f48337fe700 1 -- 172.21.15.144:0/1510519577 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f483c01d070 con 0x7f48400ffd30 2024-09-10T12:11:58.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.614+0000 7f4848762700 1 -- 172.21.15.144:0/1510519577 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f484019aa70 con 0x7f48400ffd30 2024-09-10T12:11:58.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.614+0000 7f48337fe700 1 -- 172.21.15.144:0/1510519577 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f483c00f460 con 0x7f48400ffd30 2024-09-10T12:11:58.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.614+0000 7f48337fe700 1 -- 172.21.15.144:0/1510519577 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f483c0175d0 con 0x7f48400ffd30 2024-09-10T12:11:58.617 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.614+0000 7f4848762700 1 -- 172.21.15.144:0/1510519577 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f484019af60 con 0x7f48400ffd30 2024-09-10T12:11:58.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.615+0000 7f4848762700 1 -- 172.21.15.144:0/1510519577 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f484004ea50 con 0x7f48400ffd30 2024-09-10T12:11:58.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.615+0000 7f48337fe700 1 -- 172.21.15.144:0/1510519577 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7f483c00f5d0 con 0x7f48400ffd30 2024-09-10T12:11:58.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.616+0000 7f48337fe700 1 --2- 172.21.15.144:0/1510519577 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f482c06e200 0x7f482c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:11:58.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.616+0000 7f48337fe700 1 -- 172.21.15.144:0/1510519577 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7f483c08f500 con 0x7f48400ffd30 2024-09-10T12:11:58.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.616+0000 7f48464fe700 1 --2- 172.21.15.144:0/1510519577 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f482c06e200 0x7f482c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:11:58.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.617+0000 7f48464fe700 1 --2- 172.21.15.144:0/1510519577 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f482c06e200 0x7f482c0706b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4834009a10 tx=0x7f4834009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:11:58.622 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.621+0000 7f48337fe700 1 -- 172.21.15.144:0/1510519577 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7f483c0588e0 con 0x7f48400ffd30 2024-09-10T12:11:58.839 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.838+0000 7f4848762700 1 -- 172.21.15.144:0/1510519577 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f484019b2c0 con 0x7f48400ffd30 2024-09-10T12:11:58.840 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.838+0000 7f48337fe700 1 -- 172.21.15.144:0/1510519577 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 35 entries v85) v1 ==== 81+0+2075 (secure 0 0 0) 0x7f483c05bfa0 con 0x7f48400ffd30 2024-09-10T12:11:58.840 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:6825/2986758450 2024-09-11T11:37:33.047531+0000 2024-09-10T12:11:58.840 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6829/2138997251 2024-09-11T11:37:33.047514+0000 2024-09-10T12:11:58.840 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6828/2138997251 2024-09-11T11:37:33.047514+0000 2024-09-10T12:11:58.840 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/815653583 2024-09-11T11:33:13.536690+0000 2024-09-10T12:11:58.840 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/3063000005 2024-09-11T11:33:13.536690+0000 2024-09-10T12:11:58.840 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:0/107770944 2024-09-11T11:30:13.307420+0000 2024-09-10T12:11:58.840 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:6824/2986758450 2024-09-11T11:37:33.047531+0000 2024-09-10T12:11:58.841 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/2988003891 2024-09-11T11:28:51.554066+0000 2024-09-10T12:11:58.841 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:6829/2974090415 2024-09-11T11:30:13.307420+0000 2024-09-10T12:11:58.841 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/3102311729 2024-09-11T11:20:42.965172+0000 2024-09-10T12:11:58.841 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/1470901008 2024-09-11T11:28:51.554066+0000 2024-09-10T12:11:58.841 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6826/3978020411 2024-09-11T11:26:49.926479+0000 2024-09-10T12:11:58.841 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/2543679831 2024-09-11T11:20:09.967048+0000 2024-09-10T12:11:58.841 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6801/2 2024-09-11T11:20:09.967048+0000 2024-09-10T12:11:58.841 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6801/1923800240 2024-09-11T11:33:13.536690+0000 2024-09-10T12:11:58.841 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:0/58974248 2024-09-11T11:30:13.307420+0000 2024-09-10T12:11:58.841 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/2667523333 2024-09-11T11:20:42.965172+0000 2024-09-10T12:11:58.841 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6800/2 2024-09-11T11:20:09.967048+0000 2024-09-10T12:11:58.841 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:0/102845028 2024-09-11T11:30:13.307420+0000 2024-09-10T12:11:58.841 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/1513284721 2024-09-11T11:20:09.967048+0000 2024-09-10T12:11:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/337525076 2024-09-11T11:28:51.554066+0000 2024-09-10T12:11:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:0/1342690675 2024-09-11T11:30:13.307420+0000 2024-09-10T12:11:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6827/3978020411 2024-09-11T11:26:49.926479+0000 2024-09-10T12:11:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/1183923569 2024-09-11T11:33:13.536690+0000 2024-09-10T12:11:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/898696771 2024-09-11T11:21:47.868365+0000 2024-09-10T12:11:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/3946165961 2024-09-11T11:20:09.967048+0000 2024-09-10T12:11:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:0/3450800846 2024-09-11T11:30:13.307420+0000 2024-09-10T12:11:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:0/1972404970 2024-09-11T11:30:13.307420+0000 2024-09-10T12:11:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/2490688975 2024-09-11T11:28:51.554066+0000 2024-09-10T12:11:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/3499523310 2024-09-11T11:21:47.868365+0000 2024-09-10T12:11:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:6828/2974090415 2024-09-11T11:30:13.307420+0000 2024-09-10T12:11:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/859227033 2024-09-11T11:33:13.536690+0000 2024-09-10T12:11:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/4053102056 2024-09-11T11:21:47.868365+0000 2024-09-10T12:11:58.842 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/2387813590 2024-09-11T11:20:42.965172+0000 2024-09-10T12:11:58.843 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6800/1923800240 2024-09-11T11:33:13.536690+0000 2024-09-10T12:11:58.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.841+0000 7f4848762700 1 -- 172.21.15.144:0/1510519577 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f482c06e200 msgr2=0x7f482c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:58.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.841+0000 7f4848762700 1 --2- 172.21.15.144:0/1510519577 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f482c06e200 0x7f482c0706b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f4834009a10 tx=0x7f4834009190 comp rx=0 tx=0).stop 2024-09-10T12:11:58.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.841+0000 7f4848762700 1 -- 172.21.15.144:0/1510519577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48400ffd30 msgr2=0x7f4840196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:11:58.843 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.841+0000 7f4848762700 1 --2- 172.21.15.144:0/1510519577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48400ffd30 0x7f4840196380 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f483c0092e0 tx=0x7f483c00bd70 comp rx=0 tx=0).stop 2024-09-10T12:11:58.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.841+0000 7f4848762700 1 -- 172.21.15.144:0/1510519577 shutdown_connections 2024-09-10T12:11:58.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.841+0000 7f4848762700 1 --2- 172.21.15.144:0/1510519577 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7f482c06e200 0x7f482c0706b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:58.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.841+0000 7f4848762700 1 --2- 172.21.15.144:0/1510519577 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7f48400ff390 0x7f4840195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:58.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.841+0000 7f4848762700 1 --2- 172.21.15.144:0/1510519577 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7f48400ffd30 0x7f4840196380 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:11:58.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.841+0000 7f4848762700 1 -- 172.21.15.144:0/1510519577 >> 172.21.15.144:0/1510519577 conn(0x7f4840074f50 msgr2=0x7f4840103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:11:58.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.841+0000 7f4848762700 1 -- 172.21.15.144:0/1510519577 shutdown_connections 2024-09-10T12:11:58.844 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:11:58.841+0000 7f4848762700 1 -- 172.21.15.144:0/1510519577 wait complete. 2024-09-10T12:11:58.844 INFO:teuthology.orchestra.run.smithi144.stderr:listed 35 entries 2024-09-10T12:11:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:58 smithi144 ceph-mon[87436]: pgmap v1202: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:11:58.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:11:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:58 smithi178 ceph-mon[65940]: pgmap v1202: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:11:59.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:11:59.521 DEBUG:teuthology.orchestra.run.smithi144:> set -ex 2024-09-10T12:11:59.521 DEBUG:teuthology.orchestra.run.smithi144:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-10T12:11:59.540 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 -- ceph osd blocklist ls 2024-09-10T12:11:59.782 INFO:teuthology.orchestra.run.smithi144.stderr:Inferring config /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/mon.smithi144/config 2024-09-10T12:11:59.902 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:11:59 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/1510519577' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T12:12:00.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:11:59 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/1510519577' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T12:12:00.874 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:00 smithi144 ceph-mon[87436]: pgmap v1203: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:12:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:00 smithi178 ceph-mon[65940]: pgmap v1203: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:12:01.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.393+0000 7fcb942bb700 1 -- 172.21.15.144:0/3090665243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0ffbe0 msgr2=0x7fcb8c10a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:12:01.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.393+0000 7fcb942bb700 1 --2- 172.21.15.144:0/3090665243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0ffbe0 0x7fcb8c10a650 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c009910 tx=0x7fcb7c009c20 comp rx=0 tx=0).stop 2024-09-10T12:12:01.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.393+0000 7fcb942bb700 1 -- 172.21.15.144:0/3090665243 shutdown_connections 2024-09-10T12:12:01.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.394+0000 7fcb942bb700 1 --2- 172.21.15.144:0/3090665243 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0ffbe0 0x7fcb8c10a650 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:12:01.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.394+0000 7fcb942bb700 1 --2- 172.21.15.144:0/3090665243 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb8c0ff2d0 0x7fcb8c0ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:12:01.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.394+0000 7fcb942bb700 1 -- 172.21.15.144:0/3090665243 >> 172.21.15.144:0/3090665243 conn(0x7fcb8c0747e0 msgr2=0x7fcb8c074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:12:01.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.394+0000 7fcb942bb700 1 -- 172.21.15.144:0/3090665243 shutdown_connections 2024-09-10T12:12:01.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.394+0000 7fcb942bb700 1 -- 172.21.15.144:0/3090665243 wait complete. 2024-09-10T12:12:01.396 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.394+0000 7fcb942bb700 1 Processor -- start 2024-09-10T12:12:01.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.394+0000 7fcb942bb700 1 -- start start 2024-09-10T12:12:01.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.394+0000 7fcb942bb700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0ff2d0 0x7fcb8c195db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:12:01.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.394+0000 7fcb942bb700 1 --2- >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb8c0ffbe0 0x7fcb8c1962f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:12:01.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.394+0000 7fcb942bb700 1 -- --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c1969d0 con 0x7fcb8c0ff2d0 2024-09-10T12:12:01.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.394+0000 7fcb942bb700 1 -- --> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c19a760 con 0x7fcb8c0ffbe0 2024-09-10T12:12:01.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.394+0000 7fcb92057700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0ff2d0 0x7fcb8c195db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:12:01.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.395+0000 7fcb92057700 1 --2- >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0ff2d0 0x7fcb8c195db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.144:3300/0 says I am v2:172.21.15.144:34542/0 (socket says 172.21.15.144:34542) 2024-09-10T12:12:01.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.395+0000 7fcb92057700 1 -- 172.21.15.144:0/906983545 learned_addr learned my addr 172.21.15.144:0/906983545 (peer_addr_for_me v2:172.21.15.144:0/0) 2024-09-10T12:12:01.397 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.395+0000 7fcb91856700 1 --2- 172.21.15.144:0/906983545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb8c0ffbe0 0x7fcb8c1962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:12:01.398 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.395+0000 7fcb92057700 1 -- 172.21.15.144:0/906983545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb8c0ffbe0 msgr2=0x7fcb8c1962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:12:01.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.395+0000 7fcb92057700 1 --2- 172.21.15.144:0/906983545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb8c0ffbe0 0x7fcb8c1962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:12:01.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.395+0000 7fcb92057700 1 -- 172.21.15.144:0/906983545 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb7c0095f0 con 0x7fcb8c0ff2d0 2024-09-10T12:12:01.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.395+0000 7fcb92057700 1 --2- 172.21.15.144:0/906983545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0ff2d0 0x7fcb8c195db0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800d8d0 tx=0x7fcb8800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:12:01.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.395+0000 7fcb837fe700 1 -- 172.21.15.144:0/906983545 <== mon.0 v2:172.21.15.144:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb880098a0 con 0x7fcb8c0ff2d0 2024-09-10T12:12:01.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.395+0000 7fcb837fe700 1 -- 172.21.15.144:0/906983545 <== mon.0 v2:172.21.15.144:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcb88010460 con 0x7fcb8c0ff2d0 2024-09-10T12:12:01.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.395+0000 7fcb942bb700 1 -- 172.21.15.144:0/906983545 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c19aa40 con 0x7fcb8c0ff2d0 2024-09-10T12:12:01.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.395+0000 7fcb837fe700 1 -- 172.21.15.144:0/906983545 <== mon.0 v2:172.21.15.144:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb8800f5f0 con 0x7fcb8c0ff2d0 2024-09-10T12:12:01.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.395+0000 7fcb942bb700 1 -- 172.21.15.144:0/906983545 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8c19af90 con 0x7fcb8c0ff2d0 2024-09-10T12:12:01.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.396+0000 7fcb942bb700 1 -- 172.21.15.144:0/906983545 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8c107d50 con 0x7fcb8c0ff2d0 2024-09-10T12:12:01.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.397+0000 7fcb837fe700 1 -- 172.21.15.144:0/906983545 <== mon.0 v2:172.21.15.144:3300/0 4 ==== mgrmap(e 39) v1 ==== 92205+0+0 (secure 0 0 0) 0x7fcb88009a00 con 0x7fcb8c0ff2d0 2024-09-10T12:12:01.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.397+0000 7fcb837fe700 1 --2- 172.21.15.144:0/906983545 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb7806e200 0x7fcb780706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T12:12:01.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.397+0000 7fcb837fe700 1 -- 172.21.15.144:0/906983545 <== mon.0 v2:172.21.15.144:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6328+0+0 (secure 0 0 0) 0x7fcb8808de70 con 0x7fcb8c0ff2d0 2024-09-10T12:12:01.399 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.397+0000 7fcb91856700 1 --2- 172.21.15.144:0/906983545 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb7806e200 0x7fcb780706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T12:12:01.400 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.398+0000 7fcb91856700 1 --2- 172.21.15.144:0/906983545 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb7806e200 0x7fcb780706b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c1973d0 tx=0x7fcb7c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T12:12:01.403 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.402+0000 7fcb837fe700 1 -- 172.21.15.144:0/906983545 <== mon.0 v2:172.21.15.144:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+188821 (secure 0 0 0) 0x7fcb880572d0 con 0x7fcb8c0ff2d0 2024-09-10T12:12:01.618 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.617+0000 7fcb942bb700 1 -- 172.21.15.144:0/906983545 --> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcb8c04ea50 con 0x7fcb8c0ff2d0 2024-09-10T12:12:01.619 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.618+0000 7fcb837fe700 1 -- 172.21.15.144:0/906983545 <== mon.0 v2:172.21.15.144:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 35 entries v85) v1 ==== 81+0+2075 (secure 0 0 0) 0x7fcb88020830 con 0x7fcb8c0ff2d0 2024-09-10T12:12:01.619 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:6825/2986758450 2024-09-11T11:37:33.047531+0000 2024-09-10T12:12:01.619 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6829/2138997251 2024-09-11T11:37:33.047514+0000 2024-09-10T12:12:01.619 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6828/2138997251 2024-09-11T11:37:33.047514+0000 2024-09-10T12:12:01.619 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/815653583 2024-09-11T11:33:13.536690+0000 2024-09-10T12:12:01.620 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/3063000005 2024-09-11T11:33:13.536690+0000 2024-09-10T12:12:01.620 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:0/107770944 2024-09-11T11:30:13.307420+0000 2024-09-10T12:12:01.620 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:6824/2986758450 2024-09-11T11:37:33.047531+0000 2024-09-10T12:12:01.620 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/2988003891 2024-09-11T11:28:51.554066+0000 2024-09-10T12:12:01.620 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:6829/2974090415 2024-09-11T11:30:13.307420+0000 2024-09-10T12:12:01.620 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/3102311729 2024-09-11T11:20:42.965172+0000 2024-09-10T12:12:01.620 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/1470901008 2024-09-11T11:28:51.554066+0000 2024-09-10T12:12:01.620 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6826/3978020411 2024-09-11T11:26:49.926479+0000 2024-09-10T12:12:01.620 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/2543679831 2024-09-11T11:20:09.967048+0000 2024-09-10T12:12:01.620 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6801/2 2024-09-11T11:20:09.967048+0000 2024-09-10T12:12:01.620 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6801/1923800240 2024-09-11T11:33:13.536690+0000 2024-09-10T12:12:01.620 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:0/58974248 2024-09-11T11:30:13.307420+0000 2024-09-10T12:12:01.620 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/2667523333 2024-09-11T11:20:42.965172+0000 2024-09-10T12:12:01.621 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6800/2 2024-09-11T11:20:09.967048+0000 2024-09-10T12:12:01.621 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:0/102845028 2024-09-11T11:30:13.307420+0000 2024-09-10T12:12:01.621 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/1513284721 2024-09-11T11:20:09.967048+0000 2024-09-10T12:12:01.621 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/337525076 2024-09-11T11:28:51.554066+0000 2024-09-10T12:12:01.621 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:0/1342690675 2024-09-11T11:30:13.307420+0000 2024-09-10T12:12:01.621 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6827/3978020411 2024-09-11T11:26:49.926479+0000 2024-09-10T12:12:01.621 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/1183923569 2024-09-11T11:33:13.536690+0000 2024-09-10T12:12:01.621 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/898696771 2024-09-11T11:21:47.868365+0000 2024-09-10T12:12:01.621 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/3946165961 2024-09-11T11:20:09.967048+0000 2024-09-10T12:12:01.621 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:0/3450800846 2024-09-11T11:30:13.307420+0000 2024-09-10T12:12:01.621 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:0/1972404970 2024-09-11T11:30:13.307420+0000 2024-09-10T12:12:01.621 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/2490688975 2024-09-11T11:28:51.554066+0000 2024-09-10T12:12:01.621 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/3499523310 2024-09-11T11:21:47.868365+0000 2024-09-10T12:12:01.621 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.178:6828/2974090415 2024-09-11T11:30:13.307420+0000 2024-09-10T12:12:01.621 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/859227033 2024-09-11T11:33:13.536690+0000 2024-09-10T12:12:01.622 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/4053102056 2024-09-11T11:21:47.868365+0000 2024-09-10T12:12:01.622 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:0/2387813590 2024-09-11T11:20:42.965172+0000 2024-09-10T12:12:01.622 INFO:teuthology.orchestra.run.smithi144.stdout:172.21.15.144:6800/1923800240 2024-09-11T11:33:13.536690+0000 2024-09-10T12:12:01.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.620+0000 7fcb942bb700 1 -- 172.21.15.144:0/906983545 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb7806e200 msgr2=0x7fcb780706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:12:01.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.620+0000 7fcb942bb700 1 --2- 172.21.15.144:0/906983545 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb7806e200 0x7fcb780706b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fcb8c1973d0 tx=0x7fcb7c01a040 comp rx=0 tx=0).stop 2024-09-10T12:12:01.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.620+0000 7fcb942bb700 1 -- 172.21.15.144:0/906983545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0ff2d0 msgr2=0x7fcb8c195db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T12:12:01.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.620+0000 7fcb942bb700 1 --2- 172.21.15.144:0/906983545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0ff2d0 0x7fcb8c195db0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fcb8800d8d0 tx=0x7fcb8800dbe0 comp rx=0 tx=0).stop 2024-09-10T12:12:01.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.620+0000 7fcb942bb700 1 -- 172.21.15.144:0/906983545 shutdown_connections 2024-09-10T12:12:01.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.620+0000 7fcb942bb700 1 --2- 172.21.15.144:0/906983545 >> [v2:172.21.15.144:6800/2497095016,v1:172.21.15.144:6801/2497095016] conn(0x7fcb7806e200 0x7fcb780706b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:12:01.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.620+0000 7fcb942bb700 1 --2- 172.21.15.144:0/906983545 >> [v2:172.21.15.144:3300/0,v1:172.21.15.144:6789/0] conn(0x7fcb8c0ff2d0 0x7fcb8c195db0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:12:01.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.620+0000 7fcb942bb700 1 --2- 172.21.15.144:0/906983545 >> [v2:172.21.15.178:3300/0,v1:172.21.15.178:6789/0] conn(0x7fcb8c0ffbe0 0x7fcb8c1962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T12:12:01.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.620+0000 7fcb942bb700 1 -- 172.21.15.144:0/906983545 >> 172.21.15.144:0/906983545 conn(0x7fcb8c0747e0 msgr2=0x7fcb8c104e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T12:12:01.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.620+0000 7fcb942bb700 1 -- 172.21.15.144:0/906983545 shutdown_connections 2024-09-10T12:12:01.623 INFO:teuthology.orchestra.run.smithi144.stderr:2024-09-10T12:12:01.620+0000 7fcb942bb700 1 -- 172.21.15.144:0/906983545 wait complete. 2024-09-10T12:12:01.623 INFO:teuthology.orchestra.run.smithi144.stderr:listed 35 entries 2024-09-10T12:12:01.913 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:01 smithi144 ceph-mon[87436]: from='client.? 172.21.15.144:0/906983545' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T12:12:02.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:01 smithi178 ceph-mon[65940]: from='client.? 172.21.15.144:0/906983545' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T12:12:02.285 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi144.front.sepia.ceph.com... 2024-09-10T12:12:02.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-10T12:12:02.286 DEBUG:teuthology.orchestra.run.smithi144:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2024-09-10T12:12:02.313 INFO:teuthology.orchestra.run:waiting for 300 2024-09-10T12:12:03.081 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:02 smithi178 ceph-mon[65940]: pgmap v1204: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:03.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:02 smithi144 ceph-mon[87436]: pgmap v1204: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:04 smithi178 ceph-mon[65940]: pgmap v1205: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:12:05.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:04 smithi144 ceph-mon[87436]: pgmap v1205: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:12:07.008 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:06 smithi144 ceph-mon[87436]: pgmap v1206: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:06 smithi178 ceph-mon[65940]: pgmap v1206: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:08 smithi178 ceph-mon[65940]: pgmap v1207: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:12:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:08 smithi144 ceph-mon[87436]: pgmap v1207: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:12:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:10 smithi178 ceph-mon[65940]: pgmap v1208: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:12:11.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:10 smithi144 ceph-mon[87436]: pgmap v1208: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:12:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:12 smithi144 ceph-mon[87436]: pgmap v1209: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:12 smithi178 ceph-mon[65940]: pgmap v1209: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:12:14.183 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:12:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:14 smithi178 ceph-mon[65940]: pgmap v1210: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:12:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:14 smithi144 ceph-mon[87436]: pgmap v1210: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:12:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:16 smithi178 ceph-mon[65940]: pgmap v1211: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:17.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:16 smithi144 ceph-mon[87436]: pgmap v1211: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:18 smithi178 ceph-mon[65940]: pgmap v1212: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:12:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:18 smithi144 ceph-mon[87436]: pgmap v1212: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:12:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:20 smithi178 ceph-mon[65940]: pgmap v1213: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:12:21.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:20 smithi144 ceph-mon[87436]: pgmap v1213: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:12:23.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:22 smithi144 ceph-mon[87436]: pgmap v1214: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:23.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:22 smithi178 ceph-mon[65940]: pgmap v1214: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:24 smithi178 ceph-mon[65940]: pgmap v1215: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:12:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:24 smithi144 ceph-mon[87436]: pgmap v1215: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:12:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:26 smithi178 ceph-mon[65940]: pgmap v1216: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:26 smithi144 ceph-mon[87436]: pgmap v1216: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:28.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:28 smithi178 ceph-mon[65940]: pgmap v1217: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:12:28.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:12:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:28 smithi144 ceph-mon[87436]: pgmap v1217: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:12:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:12:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:30 smithi178 ceph-mon[65940]: pgmap v1218: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:12:31.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:30 smithi144 ceph-mon[87436]: pgmap v1218: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:12:33.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:32 smithi144 ceph-mon[87436]: pgmap v1219: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:33.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:32 smithi178 ceph-mon[65940]: pgmap v1219: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:34 smithi178 ceph-mon[65940]: pgmap v1220: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:12:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:34 smithi144 ceph-mon[87436]: pgmap v1220: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:12:37.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:36 smithi178 ceph-mon[65940]: pgmap v1221: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:36 smithi144 ceph-mon[87436]: pgmap v1221: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:39.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:38 smithi178 ceph-mon[65940]: pgmap v1222: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:12:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:38 smithi144 ceph-mon[87436]: pgmap v1222: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:12:41.120 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:40 smithi178 ceph-mon[65940]: pgmap v1223: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:12:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:40 smithi144 ceph-mon[87436]: pgmap v1223: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:12:43.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:42 smithi144 ceph-mon[87436]: pgmap v1224: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:43.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:42 smithi178 ceph-mon[65940]: pgmap v1224: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:44.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:12:44.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:12:45.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:44 smithi178 ceph-mon[65940]: pgmap v1225: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:12:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:44 smithi144 ceph-mon[87436]: pgmap v1225: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:12:47.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:46 smithi178 ceph-mon[65940]: pgmap v1226: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:46 smithi144 ceph-mon[87436]: pgmap v1226: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:49.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:48 smithi178 ceph-mon[65940]: pgmap v1227: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:12:49.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:48 smithi144 ceph-mon[87436]: pgmap v1227: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:12:51.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:50 smithi178 ceph-mon[65940]: pgmap v1228: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:12:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:50 smithi144 ceph-mon[87436]: pgmap v1228: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:12:53.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:52 smithi144 ceph-mon[87436]: pgmap v1229: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:53.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:52 smithi178 ceph-mon[65940]: pgmap v1229: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:54 smithi178 ceph-mon[65940]: pgmap v1230: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:12:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:12:55.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:12:55.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:12:55.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:12:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:54 smithi144 ceph-mon[87436]: pgmap v1230: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:12:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:12:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:12:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:12:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:12:57.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:56 smithi178 ceph-mon[65940]: pgmap v1231: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:56 smithi144 ceph-mon[87436]: pgmap v1231: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:12:59.106 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:58 smithi178 ceph-mon[65940]: pgmap v1232: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:12:59.106 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:12:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:12:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:58 smithi144 ceph-mon[87436]: pgmap v1232: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:12:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:12:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:13:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:00 smithi178 ceph-mon[65940]: pgmap v1233: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:13:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:00 smithi144 ceph-mon[87436]: pgmap v1233: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:13:03.071 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:02 smithi144 ceph-mon[87436]: pgmap v1234: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:03.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:02 smithi178 ceph-mon[65940]: pgmap v1234: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:05.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:04 smithi178 ceph-mon[65940]: pgmap v1235: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:13:05.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:04 smithi144 ceph-mon[87436]: pgmap v1235: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:13:07.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:06 smithi178 ceph-mon[65940]: pgmap v1236: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:06 smithi144 ceph-mon[87436]: pgmap v1236: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:09.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:08 smithi178 ceph-mon[65940]: pgmap v1237: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:13:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:08 smithi144 ceph-mon[87436]: pgmap v1237: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:13:11.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:10 smithi178 ceph-mon[65940]: pgmap v1238: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:13:11.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:10 smithi144 ceph-mon[87436]: pgmap v1238: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:13:13.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:12 smithi144 ceph-mon[87436]: pgmap v1239: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:13.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:12 smithi178 ceph-mon[65940]: pgmap v1239: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:14.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:13:14.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:13:15.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:14 smithi178 ceph-mon[65940]: pgmap v1240: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:13:15.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:14 smithi144 ceph-mon[87436]: pgmap v1240: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:13:17.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:16 smithi178 ceph-mon[65940]: pgmap v1241: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:17.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:16 smithi144 ceph-mon[87436]: pgmap v1241: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:19.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:18 smithi178 ceph-mon[65940]: pgmap v1242: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:13:19.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:18 smithi144 ceph-mon[87436]: pgmap v1242: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:13:21.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:20 smithi178 ceph-mon[65940]: pgmap v1243: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:13:21.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:20 smithi144 ceph-mon[87436]: pgmap v1243: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:13:23.114 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:22 smithi144 ceph-mon[87436]: pgmap v1244: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:23.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:22 smithi178 ceph-mon[65940]: pgmap v1244: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:25.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:24 smithi178 ceph-mon[65940]: pgmap v1245: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:13:25.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:24 smithi144 ceph-mon[87436]: pgmap v1245: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:13:27.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:26 smithi178 ceph-mon[65940]: pgmap v1246: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:27.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:26 smithi144 ceph-mon[87436]: pgmap v1246: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:29.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:28 smithi178 ceph-mon[65940]: pgmap v1247: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:13:29.118 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:13:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:28 smithi144 ceph-mon[87436]: pgmap v1247: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:13:29.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:13:31.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:30 smithi144 ceph-mon[87436]: pgmap v1248: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:13:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:30 smithi178 ceph-mon[65940]: pgmap v1248: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:13:33.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:32 smithi144 ceph-mon[87436]: pgmap v1249: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:32 smithi178 ceph-mon[65940]: pgmap v1249: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:35.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:34 smithi178 ceph-mon[65940]: pgmap v1250: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:13:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:34 smithi144 ceph-mon[87436]: pgmap v1250: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:13:37.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:36 smithi144 ceph-mon[87436]: pgmap v1251: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:36 smithi178 ceph-mon[65940]: pgmap v1251: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:39.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:38 smithi144 ceph-mon[87436]: pgmap v1252: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:13:39.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:38 smithi178 ceph-mon[65940]: pgmap v1252: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:13:41.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:40 smithi144 ceph-mon[87436]: pgmap v1253: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:13:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:40 smithi178 ceph-mon[65940]: pgmap v1253: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:13:43.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:42 smithi144 ceph-mon[87436]: pgmap v1254: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:42 smithi178 ceph-mon[65940]: pgmap v1254: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:44.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:13:44.268 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:13:45.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:44 smithi144 ceph-mon[87436]: pgmap v1255: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:13:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:44 smithi178 ceph-mon[65940]: pgmap v1255: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:13:47.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:46 smithi144 ceph-mon[87436]: pgmap v1256: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:46 smithi178 ceph-mon[65940]: pgmap v1256: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:49.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:48 smithi144 ceph-mon[87436]: pgmap v1257: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:13:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:48 smithi178 ceph-mon[65940]: pgmap v1257: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:13:51.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:50 smithi144 ceph-mon[87436]: pgmap v1258: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:13:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:50 smithi178 ceph-mon[65940]: pgmap v1258: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:13:53.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:52 smithi144 ceph-mon[87436]: pgmap v1259: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:52 smithi178 ceph-mon[65940]: pgmap v1259: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:54 smithi144 ceph-mon[87436]: pgmap v1260: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:13:55.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:54 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:13:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:54 smithi178 ceph-mon[65940]: pgmap v1260: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:13:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:54 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:13:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:55 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:13:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:55 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:13:56.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:55 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:13:56.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:55 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:13:56.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:55 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:13:56.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:55 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:13:57.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:56 smithi144 ceph-mon[87436]: pgmap v1261: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:56 smithi178 ceph-mon[65940]: pgmap v1261: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:13:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:58 smithi144 ceph-mon[87436]: pgmap v1262: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:13:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:13:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:13:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:58 smithi178 ceph-mon[65940]: pgmap v1262: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:13:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:13:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:14:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:00 smithi144 ceph-mon[87436]: pgmap v1263: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:14:01.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:00 smithi178 ceph-mon[65940]: pgmap v1263: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:14:03.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:02 smithi144 ceph-mon[87436]: pgmap v1264: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:02 smithi178 ceph-mon[65940]: pgmap v1264: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:05.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:04 smithi144 ceph-mon[87436]: pgmap v1265: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:14:05.326 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:04 smithi178 ceph-mon[65940]: pgmap v1265: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:14:07.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:06 smithi144 ceph-mon[87436]: pgmap v1266: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:06 smithi178 ceph-mon[65940]: pgmap v1266: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:09.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:08 smithi144 ceph-mon[87436]: pgmap v1267: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:14:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:08 smithi178 ceph-mon[65940]: pgmap v1267: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:14:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:10 smithi178 ceph-mon[65940]: pgmap v1268: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:14:11.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:10 smithi144 ceph-mon[87436]: pgmap v1268: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:14:13.236 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:12 smithi144 ceph-mon[87436]: pgmap v1269: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:12 smithi178 ceph-mon[65940]: pgmap v1269: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:14.268 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:13 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:14:14.425 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:13 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:14:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:14 smithi178 ceph-mon[65940]: pgmap v1270: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:14:15.436 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:14 smithi144 ceph-mon[87436]: pgmap v1270: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:14:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:16 smithi178 ceph-mon[65940]: pgmap v1271: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:17.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:16 smithi144 ceph-mon[87436]: pgmap v1271: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:19.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:18 smithi178 ceph-mon[65940]: pgmap v1272: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:14:19.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:18 smithi144 ceph-mon[87436]: pgmap v1272: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:14:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:20 smithi178 ceph-mon[65940]: pgmap v1273: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:14:21.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:20 smithi144 ceph-mon[87436]: pgmap v1273: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:14:23.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:22 smithi178 ceph-mon[65940]: pgmap v1274: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:23.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:22 smithi144 ceph-mon[87436]: pgmap v1274: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:24 smithi178 ceph-mon[65940]: pgmap v1275: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:14:25.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:24 smithi144 ceph-mon[87436]: pgmap v1275: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:14:27.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:26 smithi178 ceph-mon[65940]: pgmap v1276: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:27.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:26 smithi144 ceph-mon[87436]: pgmap v1276: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:29.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:28 smithi178 ceph-mon[65940]: pgmap v1277: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:14:29.368 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:28 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:14:29.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:28 smithi144 ceph-mon[87436]: pgmap v1277: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:14:29.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:28 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:14:31.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:30 smithi178 ceph-mon[65940]: pgmap v1278: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:14:31.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:30 smithi144 ceph-mon[87436]: pgmap v1278: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:14:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:32 smithi178 ceph-mon[65940]: pgmap v1279: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:33.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:32 smithi144 ceph-mon[87436]: pgmap v1279: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:35.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:34 smithi144 ceph-mon[87436]: pgmap v1280: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:14:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:34 smithi178 ceph-mon[65940]: pgmap v1280: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:14:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:36 smithi178 ceph-mon[65940]: pgmap v1281: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:37.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:36 smithi144 ceph-mon[87436]: pgmap v1281: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:39.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:38 smithi178 ceph-mon[65940]: pgmap v1282: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:14:39.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:38 smithi144 ceph-mon[87436]: pgmap v1282: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:14:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:40 smithi178 ceph-mon[65940]: pgmap v1283: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:14:41.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:40 smithi144 ceph-mon[87436]: pgmap v1283: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:14:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:42 smithi178 ceph-mon[65940]: pgmap v1284: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:42 smithi144 ceph-mon[87436]: pgmap v1284: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:44.268 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:43 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:14:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:43 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:14:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:44 smithi178 ceph-mon[65940]: pgmap v1285: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:14:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:44 smithi144 ceph-mon[87436]: pgmap v1285: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:14:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:46 smithi178 ceph-mon[65940]: pgmap v1286: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:46 smithi144 ceph-mon[87436]: pgmap v1286: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:49 smithi178 ceph-mon[65940]: pgmap v1287: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:14:49.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:49 smithi144 ceph-mon[87436]: pgmap v1287: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:14:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:51 smithi178 ceph-mon[65940]: pgmap v1288: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:14:51.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:51 smithi144 ceph-mon[87436]: pgmap v1288: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:14:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:53 smithi178 ceph-mon[65940]: pgmap v1289: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:53 smithi144 ceph-mon[87436]: pgmap v1289: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:55 smithi178 ceph-mon[65940]: pgmap v1290: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:14:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:55 smithi144 ceph-mon[87436]: pgmap v1290: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:14:56.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:56 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:14:56.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:56 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:14:56.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:56 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:14:56.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:56 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:14:56.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:56 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:14:56.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:56 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:14:56.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:56 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:14:56.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:56 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:14:57.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:57 smithi178 ceph-mon[65940]: pgmap v1291: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:57.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:57 smithi144 ceph-mon[87436]: pgmap v1291: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:14:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:59 smithi178 ceph-mon[65940]: pgmap v1292: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:14:59.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:14:59 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:14:59.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:59 smithi144 ceph-mon[87436]: pgmap v1292: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:14:59.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:14:59 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:15:01.349 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:01 smithi178 ceph-mon[65940]: pgmap v1293: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:15:01.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:01 smithi144 ceph-mon[87436]: pgmap v1293: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:15:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:03 smithi178 ceph-mon[65940]: pgmap v1294: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:03.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:03 smithi144 ceph-mon[87436]: pgmap v1294: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:05.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:05 smithi178 ceph-mon[65940]: pgmap v1295: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:15:05.398 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:05 smithi144 ceph-mon[87436]: pgmap v1295: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:15:07.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:07 smithi178 ceph-mon[65940]: pgmap v1296: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:07.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:07 smithi144 ceph-mon[87436]: pgmap v1296: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:09.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:09 smithi178 ceph-mon[65940]: pgmap v1297: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:15:09.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:09 smithi144 ceph-mon[87436]: pgmap v1297: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:15:11.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:11 smithi178 ceph-mon[65940]: pgmap v1298: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:15:11.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:11 smithi144 ceph-mon[87436]: pgmap v1298: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:15:13.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:13 smithi178 ceph-mon[65940]: pgmap v1299: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:13.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:13 smithi144 ceph-mon[87436]: pgmap v1299: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:14.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:14 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:15:14.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:15:15.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:15 smithi178 ceph-mon[65940]: pgmap v1300: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:15:15.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:15 smithi144 ceph-mon[87436]: pgmap v1300: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:15:17.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:17 smithi178 ceph-mon[65940]: pgmap v1301: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:17.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:17 smithi144 ceph-mon[87436]: pgmap v1301: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:19.370 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:19 smithi178 ceph-mon[65940]: pgmap v1302: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:15:19.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:19 smithi144 ceph-mon[87436]: pgmap v1302: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:15:21.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:21 smithi178 ceph-mon[65940]: pgmap v1303: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:15:21.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:21 smithi144 ceph-mon[87436]: pgmap v1303: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:15:23.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:23 smithi178 ceph-mon[65940]: pgmap v1304: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:23.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:23 smithi144 ceph-mon[87436]: pgmap v1304: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:25.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:25 smithi178 ceph-mon[65940]: pgmap v1305: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:15:25.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:25 smithi144 ceph-mon[87436]: pgmap v1305: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:15:27.369 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:27 smithi178 ceph-mon[65940]: pgmap v1306: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:27.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:27 smithi144 ceph-mon[87436]: pgmap v1306: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:29 smithi178 ceph-mon[65940]: pgmap v1307: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:15:29.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:29 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:15:29.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:29 smithi144 ceph-mon[87436]: pgmap v1307: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:15:29.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:29 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:15:31.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:31 smithi144 ceph-mon[87436]: pgmap v1308: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:15:31.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:31 smithi178 ceph-mon[65940]: pgmap v1308: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:15:33.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:33 smithi178 ceph-mon[65940]: pgmap v1309: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:33.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:33 smithi144 ceph-mon[87436]: pgmap v1309: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:35.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:35 smithi178 ceph-mon[65940]: pgmap v1310: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:15:35.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:35 smithi144 ceph-mon[87436]: pgmap v1310: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:15:37.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:37 smithi178 ceph-mon[65940]: pgmap v1311: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:37.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:37 smithi144 ceph-mon[87436]: pgmap v1311: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:39.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:39 smithi178 ceph-mon[65940]: pgmap v1312: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:15:39.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:39 smithi144 ceph-mon[87436]: pgmap v1312: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:15:41.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:41 smithi178 ceph-mon[65940]: pgmap v1313: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:15:41.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:41 smithi144 ceph-mon[87436]: pgmap v1313: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:15:43.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:43 smithi178 ceph-mon[65940]: pgmap v1314: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:43.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:43 smithi144 ceph-mon[87436]: pgmap v1314: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:44.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:15:44.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:15:45.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:45 smithi178 ceph-mon[65940]: pgmap v1315: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:15:45.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:45 smithi144 ceph-mon[87436]: pgmap v1315: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:15:47.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:47 smithi178 ceph-mon[65940]: pgmap v1316: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:47.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:47 smithi144 ceph-mon[87436]: pgmap v1316: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:49.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:49 smithi178 ceph-mon[65940]: pgmap v1317: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:15:49.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:49 smithi144 ceph-mon[87436]: pgmap v1317: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:15:51.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:51 smithi178 ceph-mon[65940]: pgmap v1318: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:15:51.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:51 smithi144 ceph-mon[87436]: pgmap v1318: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:15:53.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:53 smithi178 ceph-mon[65940]: pgmap v1319: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:53.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:53 smithi144 ceph-mon[87436]: pgmap v1319: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:55.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:55 smithi178 ceph-mon[65940]: pgmap v1320: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:15:55.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:55 smithi144 ceph-mon[87436]: pgmap v1320: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:15:56.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:56 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:15:56.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:56 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:15:57.379 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:57 smithi178 ceph-mon[65940]: pgmap v1321: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:57.380 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:57 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:15:57.380 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:57 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:15:57.380 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:57 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:15:57.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:57 smithi144 ceph-mon[87436]: pgmap v1321: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:15:57.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:57 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:15:57.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:57 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:15:57.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:57 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:15:58.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:58 smithi144 ceph-mon[87436]: pgmap v1322: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:15:58.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:58 smithi178 ceph-mon[65940]: pgmap v1322: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:15:59.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:15:59 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:15:59.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:15:59 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:16:00.434 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:00 smithi144 ceph-mon[87436]: pgmap v1323: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:16:00.617 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:00 smithi178 ceph-mon[65940]: pgmap v1323: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:16:02.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:02 smithi178 ceph-mon[65940]: pgmap v1324: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:02.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:02 smithi144 ceph-mon[87436]: pgmap v1324: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:04.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:04 smithi178 ceph-mon[65940]: pgmap v1325: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:16:04.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:04 smithi144 ceph-mon[87436]: pgmap v1325: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:16:06.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:06 smithi178 ceph-mon[65940]: pgmap v1326: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:06.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:06 smithi144 ceph-mon[87436]: pgmap v1326: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:08.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:08 smithi178 ceph-mon[65940]: pgmap v1327: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:16:08.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:08 smithi144 ceph-mon[87436]: pgmap v1327: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:16:10.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:10 smithi178 ceph-mon[65940]: pgmap v1328: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:16:10.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:10 smithi144 ceph-mon[87436]: pgmap v1328: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:16:12.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:12 smithi178 ceph-mon[65940]: pgmap v1329: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:12.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:12 smithi144 ceph-mon[87436]: pgmap v1329: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:14 smithi178 ceph-mon[65940]: pgmap v1330: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:16:14.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:14 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:16:14.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:14 smithi144 ceph-mon[87436]: pgmap v1330: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:16:14.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:14 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:16:16.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:16 smithi178 ceph-mon[65940]: pgmap v1331: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:16.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:16 smithi144 ceph-mon[87436]: pgmap v1331: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:18.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:18 smithi178 ceph-mon[65940]: pgmap v1332: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:16:18.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:18 smithi144 ceph-mon[87436]: pgmap v1332: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:16:20.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:20 smithi178 ceph-mon[65940]: pgmap v1333: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:16:20.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:20 smithi144 ceph-mon[87436]: pgmap v1333: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:16:22.849 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:22 smithi178 ceph-mon[65940]: pgmap v1334: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:22.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:22 smithi144 ceph-mon[87436]: pgmap v1334: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:24.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:24 smithi178 ceph-mon[65940]: pgmap v1335: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:16:24.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:24 smithi144 ceph-mon[87436]: pgmap v1335: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:16:26.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:26 smithi178 ceph-mon[65940]: pgmap v1336: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:26.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:26 smithi144 ceph-mon[87436]: pgmap v1336: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:28.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:28 smithi178 ceph-mon[65940]: pgmap v1337: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:16:28.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:28 smithi144 ceph-mon[87436]: pgmap v1337: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:16:29.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:29 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:16:29.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:29 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:16:30.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:30 smithi178 ceph-mon[65940]: pgmap v1338: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:16:30.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:30 smithi144 ceph-mon[87436]: pgmap v1338: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:16:32.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:32 smithi178 ceph-mon[65940]: pgmap v1339: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:32.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:32 smithi144 ceph-mon[87436]: pgmap v1339: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:34.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:34 smithi178 ceph-mon[65940]: pgmap v1340: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:16:34.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:34 smithi144 ceph-mon[87436]: pgmap v1340: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:16:36.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:36 smithi178 ceph-mon[65940]: pgmap v1341: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:36.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:36 smithi144 ceph-mon[87436]: pgmap v1341: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:38.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:38 smithi178 ceph-mon[65940]: pgmap v1342: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:16:38.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:38 smithi144 ceph-mon[87436]: pgmap v1342: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:16:40.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:40 smithi178 ceph-mon[65940]: pgmap v1343: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:16:40.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:40 smithi144 ceph-mon[87436]: pgmap v1343: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:16:42.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:42 smithi178 ceph-mon[65940]: pgmap v1344: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:42.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:42 smithi144 ceph-mon[87436]: pgmap v1344: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:44 smithi178 ceph-mon[65940]: pgmap v1345: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:16:44.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:44 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:16:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:44 smithi144 ceph-mon[87436]: pgmap v1345: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:16:44.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:44 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:16:46.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:46 smithi178 ceph-mon[65940]: pgmap v1346: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:46.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:46 smithi144 ceph-mon[87436]: pgmap v1346: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:48.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:48 smithi178 ceph-mon[65940]: pgmap v1347: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:16:48.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:48 smithi144 ceph-mon[87436]: pgmap v1347: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:16:50.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:50 smithi178 ceph-mon[65940]: pgmap v1348: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:16:50.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:50 smithi144 ceph-mon[87436]: pgmap v1348: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:16:52.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:52 smithi144 ceph-mon[87436]: pgmap v1349: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:52.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:52 smithi178 ceph-mon[65940]: pgmap v1349: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:54.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:54 smithi178 ceph-mon[65940]: pgmap v1350: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:16:54.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:54 smithi144 ceph-mon[87436]: pgmap v1350: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-10T12:16:56.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:56 smithi178 ceph-mon[65940]: pgmap v1351: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:56.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:56 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:16:56.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:56 smithi144 ceph-mon[87436]: pgmap v1351: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.7 KiB/s rd, 3 op/s 2024-09-10T12:16:56.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:56 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T12:16:57.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:57 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:16:57.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:57 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:16:57.867 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:57 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:16:57.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:57 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T12:16:57.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:57 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T12:16:57.934 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:57 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' 2024-09-10T12:16:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:58 smithi178 ceph-mon[65940]: pgmap v1352: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:16:59.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:16:58 smithi178 ceph-mon[65940]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:16:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:58 smithi144 ceph-mon[87436]: pgmap v1352: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 4 op/s 2024-09-10T12:16:59.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:16:58 smithi144 ceph-mon[87436]: from='mgr.34104 172.21.15.144:0/29416049' entity='mgr.smithi144.bkogfs' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T12:17:01.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:17:00 smithi178 ceph-mon[65940]: pgmap v1353: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:17:01.184 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:17:00 smithi144 ceph-mon[87436]: pgmap v1353: 65 pgs: 65 active+clean; 309 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.1 KiB/s rd, 3 op/s 2024-09-10T12:17:01.386 ERROR:tasks.cephfs.fuse_mount:process failed to terminate after unmount. This probably indicates a bug within ceph-fuse. 2024-09-10T12:17:01.387 ERROR:teuthology.run_tasks:Manager failed: ceph-fuse Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/ceph_fuse.py", line 248, in task File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-10T12:17:01.388 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T12:17:01.399 INFO:tasks.cephadm:Teardown begin 2024-09-10T12:17:01.399 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/cephadm.py", line 2236, in task File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/ceph_fuse.py", line 248, in task File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-10T12:17:01.400 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T12:17:01.434 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T12:17:01.468 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T12:17:01.468 DEBUG:teuthology.orchestra.run.smithi144:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T12:17:01.492 DEBUG:teuthology.orchestra.run.smithi178:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T12:17:01.526 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T12:17:01.527 INFO:tasks.cephadm.mon.smithi144:Stopping mon.smithi144... 2024-09-10T12:17:01.527 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi144 2024-09-10T12:17:01.684 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:17:01 smithi144 systemd[1]: Stopping Ceph mon.smithi144 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T12:17:02.133 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:17:01 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T12:17:01.856+0000 7f80c815b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi144 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T12:17:02.133 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:17:01 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144[87432]: 2024-09-10T12:17:01.856+0000 7f80c815b640 -1 mon.smithi144@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T12:17:02.133 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:17:01 smithi144 podman[146350]: 2024-09-10 12:17:01.917654159 +0000 UTC m=+0.194361962 container died 0fc028b6cc37b8df63d191420f95d55ca82542a929a59d69fb10fd62a573b54c (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T12:17:02.392 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:17:02 smithi144 podman[146350]: 2024-09-10 12:17:02.373050794 +0000 UTC m=+0.649758585 container cleanup 0fc028b6cc37b8df63d191420f95d55ca82542a929a59d69fb10fd62a573b54c (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T12:17:02.392 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:17:02 smithi144 bash[146350]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144 2024-09-10T12:17:02.854 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi144.service' 2024-09-10T12:17:02.877 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:17:02 smithi144 podman[146362]: 2024-09-10 12:17:02.683183782 +0000 UTC m=+0.761549428 container remove 0fc028b6cc37b8df63d191420f95d55ca82542a929a59d69fb10fd62a573b54c (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi144, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T12:17:02.877 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:17:02 smithi144 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi144.service: Deactivated successfully. 2024-09-10T12:17:02.877 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:17:02 smithi144 systemd[1]: Stopped Ceph mon.smithi144 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T12:17:02.877 INFO:journalctl@ceph.mon.smithi144.smithi144.stdout:Sep 10 12:17:02 smithi144 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi144.service: Consumed 28.073s CPU time. 2024-09-10T12:17:02.917 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T12:17:02.917 INFO:tasks.cephadm.mon.smithi144:Stopped mon.smithi144 2024-09-10T12:17:02.917 INFO:tasks.cephadm.mon.smithi178:Stopping mon.smithi178... 2024-09-10T12:17:02.918 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi178 2024-09-10T12:17:03.367 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:17:02 smithi178 systemd[1]: Stopping Ceph mon.smithi178 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T12:17:03.811 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:17:03 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178[65936]: 2024-09-10T12:17:03.418+0000 7f8535142640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi178 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T12:17:03.811 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:17:03 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178[65936]: 2024-09-10T12:17:03.418+0000 7f8535142640 -1 mon.smithi178@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T12:17:03.811 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:17:03 smithi178 podman[102521]: 2024-09-10 12:17:03.562344214 +0000 UTC m=+0.445928136 container died a1b7b65c960cf7ec7ca9eddcf74c6a14ce0a9878a28924bd5e7af2fc170317c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T12:17:04.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:17:03 smithi178 podman[102521]: 2024-09-10 12:17:03.993682733 +0000 UTC m=+0.877266698 container cleanup a1b7b65c960cf7ec7ca9eddcf74c6a14ce0a9878a28924bd5e7af2fc170317c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T12:17:04.117 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:17:03 smithi178 bash[102521]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178 2024-09-10T12:17:04.473 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi178.service' 2024-09-10T12:17:04.495 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:17:04 smithi178 podman[102534]: 2024-09-10 12:17:04.23675538 +0000 UTC m=+0.674511618 container remove a1b7b65c960cf7ec7ca9eddcf74c6a14ce0a9878a28924bd5e7af2fc170317c1 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-mon-smithi178, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T12:17:04.495 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:17:04 smithi178 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi178.service: Deactivated successfully. 2024-09-10T12:17:04.495 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:17:04 smithi178 systemd[1]: Stopped Ceph mon.smithi178 for 7eacd86c-6f66-11ef-bcea-c7b262605968. 2024-09-10T12:17:04.495 INFO:journalctl@ceph.mon.smithi178.smithi178.stdout:Sep 10 12:17:04 smithi178 systemd[1]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@mon.smithi178.service: Consumed 18.211s CPU time. 2024-09-10T12:17:04.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T12:17:04.521 INFO:tasks.cephadm.mon.smithi178:Stopped mon.smithi178 2024-09-10T12:17:04.521 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T12:17:04.521 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.0 2024-09-10T12:17:04.845 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 12:17:04 smithi144 systemd[1]: Stopping Ceph osd.0 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T12:17:05.184 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 12:17:04 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0[98838]: 2024-09-10T12:17:04.844+0000 7fa4c7f13640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T12:17:05.184 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 12:17:04 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0[98838]: 2024-09-10T12:17:04.844+0000 7fa4c7f13640 -1 osd.0 85 *** Got signal Terminated *** 2024-09-10T12:17:05.184 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 12:17:04 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0[98838]: 2024-09-10T12:17:04.844+0000 7fa4c7f13640 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T12:17:10.515 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 12:17:10 smithi144 podman[146483]: 2024-09-10 12:17:10.235953739 +0000 UTC m=+5.521848100 container died b59b6d7c85be8431611e56a640620c4986cae932653a0e234da48bbd1911b09c (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True) 2024-09-10T12:17:10.888 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 12:17:10 smithi144 podman[146483]: 2024-09-10 12:17:10.628470801 +0000 UTC m=+5.914365151 container cleanup b59b6d7c85be8431611e56a640620c4986cae932653a0e234da48bbd1911b09c (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-10T12:17:10.889 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 12:17:10 smithi144 bash[146483]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0 2024-09-10T12:17:11.184 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 12:17:10 smithi144 podman[146499]: 2024-09-10 12:17:10.88822717 +0000 UTC m=+0.646666913 container remove b59b6d7c85be8431611e56a640620c4986cae932653a0e234da48bbd1911b09c (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T12:17:11.518 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 12:17:11 smithi144 podman[146588]: 2024-09-10 12:17:11.155435843 +0000 UTC m=+0.016240156 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T12:17:11.518 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 12:17:11 smithi144 podman[146588]: 2024-09-10 12:17:11.289138935 +0000 UTC m=+0.149943246 container create e317854dd6614432a3c7e1b077814c0b2e2715b4019376947d07c1fd1c12ba0d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T12:17:11.519 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 12:17:11 smithi144 podman[146588]: 2024-09-10 12:17:11.517548156 +0000 UTC m=+0.378352463 container init e317854dd6614432a3c7e1b077814c0b2e2715b4019376947d07c1fd1c12ba0d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-09-10T12:17:11.934 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 12:17:11 smithi144 podman[146588]: 2024-09-10 12:17:11.520965888 +0000 UTC m=+0.381770204 container start e317854dd6614432a3c7e1b077814c0b2e2715b4019376947d07c1fd1c12ba0d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T12:17:11.934 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 12:17:11 smithi144 podman[146588]: 2024-09-10 12:17:11.600863836 +0000 UTC m=+0.461668159 container attach e317854dd6614432a3c7e1b077814c0b2e2715b4019376947d07c1fd1c12ba0d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T12:17:11.934 INFO:journalctl@ceph.osd.0.smithi144.stdout:Sep 10 12:17:11 smithi144 podman[146588]: 2024-09-10 12:17:11.709267431 +0000 UTC m=+0.570071739 container died e317854dd6614432a3c7e1b077814c0b2e2715b4019376947d07c1fd1c12ba0d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T12:17:12.364 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T12:17:12.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T12:17:12.404 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T12:17:12.404 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T12:17:12.405 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.1 2024-09-10T12:17:12.619 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 12:17:12 smithi144 systemd[1]: Stopping Ceph osd.1 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T12:17:12.934 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 12:17:12 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1[104366]: 2024-09-10T12:17:12.747+0000 7fc04e7a7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T12:17:12.934 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 12:17:12 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1[104366]: 2024-09-10T12:17:12.747+0000 7fc04e7a7640 -1 osd.1 85 *** Got signal Terminated *** 2024-09-10T12:17:12.934 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 12:17:12 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1[104366]: 2024-09-10T12:17:12.747+0000 7fc04e7a7640 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T12:17:18.552 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 12:17:18 smithi144 podman[146692]: 2024-09-10 12:17:18.266125842 +0000 UTC m=+5.633462814 container died e5d70992955a9ebbd82c7aaf32aa7ef4d5e1efe4be41f4ff6b8cf1f1cf120555 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T12:17:18.885 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 12:17:18 smithi144 podman[146692]: 2024-09-10 12:17:18.667834665 +0000 UTC m=+6.035171638 container cleanup e5d70992955a9ebbd82c7aaf32aa7ef4d5e1efe4be41f4ff6b8cf1f1cf120555 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T12:17:18.885 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 12:17:18 smithi144 bash[146692]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1 2024-09-10T12:17:19.184 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 12:17:18 smithi144 podman[146717]: 2024-09-10 12:17:18.884132037 +0000 UTC m=+0.616818636 container remove e5d70992955a9ebbd82c7aaf32aa7ef4d5e1efe4be41f4ff6b8cf1f1cf120555 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T12:17:19.588 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 12:17:19 smithi144 podman[146807]: 2024-09-10 12:17:19.195989456 +0000 UTC m=+0.016062770 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T12:17:19.588 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 12:17:19 smithi144 podman[146807]: 2024-09-10 12:17:19.301718823 +0000 UTC m=+0.121792126 container create ac60886a271a763aea929806cd427b1522b2c730792ae5bca5eb68efbb6be674 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-10T12:17:19.588 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 12:17:19 smithi144 podman[146807]: 2024-09-10 12:17:19.512632261 +0000 UTC m=+0.332705564 container init ac60886a271a763aea929806cd427b1522b2c730792ae5bca5eb68efbb6be674 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-10T12:17:19.588 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 12:17:19 smithi144 podman[146807]: 2024-09-10 12:17:19.515889309 +0000 UTC m=+0.335962613 container start ac60886a271a763aea929806cd427b1522b2c730792ae5bca5eb68efbb6be674 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T12:17:19.588 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 12:17:19 smithi144 podman[146807]: 2024-09-10 12:17:19.587464119 +0000 UTC m=+0.407537451 container attach ac60886a271a763aea929806cd427b1522b2c730792ae5bca5eb68efbb6be674 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T12:17:19.934 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 12:17:19 smithi144 podman[146830]: 2024-09-10 12:17:19.730383239 +0000 UTC m=+0.016724786 container died ac60886a271a763aea929806cd427b1522b2c730792ae5bca5eb68efbb6be674 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-10T12:17:20.278 INFO:journalctl@ceph.osd.1.smithi144.stdout:Sep 10 12:17:20 smithi144 podman[146830]: 2024-09-10 12:17:20.274625398 +0000 UTC m=+0.560966933 container remove ac60886a271a763aea929806cd427b1522b2c730792ae5bca5eb68efbb6be674 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T12:17:20.294 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T12:17:20.335 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T12:17:20.335 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T12:17:20.335 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T12:17:20.335 DEBUG:teuthology.orchestra.run.smithi144:> sudo systemctl stop ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.2 2024-09-10T12:17:20.679 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 12:17:20 smithi144 systemd[1]: Stopping Ceph osd.2 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T12:17:20.934 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 12:17:20 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2[110067]: 2024-09-10T12:17:20.677+0000 7f11e8ddb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T12:17:20.934 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 12:17:20 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2[110067]: 2024-09-10T12:17:20.677+0000 7f11e8ddb640 -1 osd.2 85 *** Got signal Terminated *** 2024-09-10T12:17:20.934 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 12:17:20 smithi144 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2[110067]: 2024-09-10T12:17:20.677+0000 7f11e8ddb640 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T12:17:26.353 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 12:17:26 smithi144 podman[146910]: 2024-09-10 12:17:26.017087325 +0000 UTC m=+5.455494536 container died 8485d374ff4439a004c0723d8822bf6c528ecbbf1b3000a563e6f06465f02f4d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T12:17:26.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 12:17:26 smithi144 podman[146910]: 2024-09-10 12:17:26.352661076 +0000 UTC m=+5.791068276 container cleanup 8485d374ff4439a004c0723d8822bf6c528ecbbf1b3000a563e6f06465f02f4d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-10T12:17:26.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 12:17:26 smithi144 bash[146910]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2 2024-09-10T12:17:26.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 12:17:26 smithi144 podman[146927]: 2024-09-10 12:17:26.537268503 +0000 UTC m=+0.519673340 container remove 8485d374ff4439a004c0723d8822bf6c528ecbbf1b3000a563e6f06465f02f4d (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-10T12:17:27.291 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 12:17:26 smithi144 podman[147015]: 2024-09-10 12:17:26.963009278 +0000 UTC m=+0.106592311 container create e50a6f39ce0b60f780aa122327d5303de2b29fa6ba55199cee07e54d424ef590 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-10T12:17:27.291 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 12:17:26 smithi144 podman[147015]: 2024-09-10 12:17:26.872275518 +0000 UTC m=+0.015858552 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T12:17:27.291 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 12:17:27 smithi144 podman[147015]: 2024-09-10 12:17:27.199103818 +0000 UTC m=+0.342686856 container init e50a6f39ce0b60f780aa122327d5303de2b29fa6ba55199cee07e54d424ef590 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T12:17:27.292 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 12:17:27 smithi144 podman[147015]: 2024-09-10 12:17:27.202508282 +0000 UTC m=+0.346091329 container start e50a6f39ce0b60f780aa122327d5303de2b29fa6ba55199cee07e54d424ef590 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-09-10T12:17:27.292 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 12:17:27 smithi144 podman[147015]: 2024-09-10 12:17:27.290751368 +0000 UTC m=+0.434334417 container attach e50a6f39ce0b60f780aa122327d5303de2b29fa6ba55199cee07e54d424ef590 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T12:17:27.684 INFO:journalctl@ceph.osd.2.smithi144.stdout:Sep 10 12:17:27 smithi144 podman[147015]: 2024-09-10 12:17:27.396841627 +0000 UTC m=+0.540424662 container died e50a6f39ce0b60f780aa122327d5303de2b29fa6ba55199cee07e54d424ef590 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-2-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T12:17:28.139 DEBUG:teuthology.orchestra.run.smithi144:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T12:17:28.178 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T12:17:28.178 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T12:17:28.178 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T12:17:28.178 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.3 2024-09-10T12:17:28.530 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 12:17:28 smithi178 systemd[1]: Stopping Ceph osd.3 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T12:17:28.867 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 12:17:28 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3[75322]: 2024-09-10T12:17:28.528+0000 7f2117064640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T12:17:28.867 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 12:17:28 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3[75322]: 2024-09-10T12:17:28.528+0000 7f2117064640 -1 osd.3 85 *** Got signal Terminated *** 2024-09-10T12:17:28.867 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 12:17:28 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3[75322]: 2024-09-10T12:17:28.528+0000 7f2117064640 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T12:17:31.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:31 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:31.369+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:32.867 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 12:17:32 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3[75322]: 2024-09-10T12:17:32.566+0000 7f210f661640 -1 osd.3 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.039227+0000 front 2024-09-10T12:17:08.039362+0000 (oldest deadline 2024-09-10T12:17:31.539158+0000) 2024-09-10T12:17:32.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:32 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:32.365+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:33.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:33 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:33.392+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:34.214 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 12:17:33 smithi178 podman[102642]: 2024-09-10 12:17:33.961731628 +0000 UTC m=+5.596921445 container died 867cd5897d94f2f526c285cb7669bfd7292f94728510e828abd0bdfbf36c96bd (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T12:17:34.505 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:34 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:34.440+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:34.506 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 12:17:34 smithi178 podman[102642]: 2024-09-10 12:17:34.311907948 +0000 UTC m=+5.947097764 container cleanup 867cd5897d94f2f526c285cb7669bfd7292f94728510e828abd0bdfbf36c96bd (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS) 2024-09-10T12:17:34.506 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 12:17:34 smithi178 bash[102642]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3 2024-09-10T12:17:34.867 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 12:17:34 smithi178 podman[102659]: 2024-09-10 12:17:34.50470086 +0000 UTC m=+0.540689513 container remove 867cd5897d94f2f526c285cb7669bfd7292f94728510e828abd0bdfbf36c96bd (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T12:17:35.134 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 12:17:34 smithi178 podman[102749]: 2024-09-10 12:17:34.805525897 +0000 UTC m=+0.016002899 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T12:17:35.134 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 12:17:34 smithi178 podman[102749]: 2024-09-10 12:17:34.905430885 +0000 UTC m=+0.115907874 container create 20fc897b27df436522ba6d9e621d48827de9e7dacd5bef2de453e0dd6405f86f (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-09-10T12:17:35.134 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 12:17:35 smithi178 podman[102749]: 2024-09-10 12:17:35.132964839 +0000 UTC m=+0.343441841 container init 20fc897b27df436522ba6d9e621d48827de9e7dacd5bef2de453e0dd6405f86f (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-10T12:17:35.134 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:34 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[80225]: 2024-09-10T12:17:34.872+0000 7fd57de3f640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.155415+0000 front 2024-09-10T12:17:08.155327+0000 (oldest deadline 2024-09-10T12:17:34.055264+0000) 2024-09-10T12:17:35.408 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 12:17:35 smithi178 podman[102749]: 2024-09-10 12:17:35.136545164 +0000 UTC m=+0.347022154 container start 20fc897b27df436522ba6d9e621d48827de9e7dacd5bef2de453e0dd6405f86f (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T12:17:35.408 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 12:17:35 smithi178 podman[102749]: 2024-09-10 12:17:35.207920685 +0000 UTC m=+0.418397684 container attach 20fc897b27df436522ba6d9e621d48827de9e7dacd5bef2de453e0dd6405f86f (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-10T12:17:35.408 INFO:journalctl@ceph.osd.3.smithi178.stdout:Sep 10 12:17:35 smithi178 podman[102749]: 2024-09-10 12:17:35.32356417 +0000 UTC m=+0.534041169 container died 20fc897b27df436522ba6d9e621d48827de9e7dacd5bef2de453e0dd6405f86f (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T12:17:35.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:35 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:35.406+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:35.891 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T12:17:35.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T12:17:35.929 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T12:17:35.929 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T12:17:35.929 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.4 2024-09-10T12:17:36.249 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:35 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[80225]: 2024-09-10T12:17:35.909+0000 7fd57de3f640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.155415+0000 front 2024-09-10T12:17:08.155327+0000 (oldest deadline 2024-09-10T12:17:34.055264+0000) 2024-09-10T12:17:36.249 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:36 smithi178 systemd[1]: Stopping Ceph osd.4 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T12:17:36.249 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:36 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[80225]: 2024-09-10T12:17:36.247+0000 7fd58183a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T12:17:36.250 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:36 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[80225]: 2024-09-10T12:17:36.247+0000 7fd58183a640 -1 osd.4 85 *** Got signal Terminated *** 2024-09-10T12:17:36.250 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:36 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[80225]: 2024-09-10T12:17:36.247+0000 7fd58183a640 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T12:17:36.617 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:36 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:36.397+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:37.367 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:36 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[80225]: 2024-09-10T12:17:36.905+0000 7fd57de3f640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.155415+0000 front 2024-09-10T12:17:08.155327+0000 (oldest deadline 2024-09-10T12:17:34.055264+0000) 2024-09-10T12:17:37.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:37 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:37.402+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:38.367 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:37 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[80225]: 2024-09-10T12:17:37.931+0000 7fd57de3f640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.155415+0000 front 2024-09-10T12:17:08.155327+0000 (oldest deadline 2024-09-10T12:17:34.055264+0000) 2024-09-10T12:17:38.870 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:38 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:38.384+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:39.172 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:38 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[80225]: 2024-09-10T12:17:38.899+0000 7fd57de3f640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.155415+0000 front 2024-09-10T12:17:08.155327+0000 (oldest deadline 2024-09-10T12:17:34.055264+0000) 2024-09-10T12:17:39.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:39 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:39.435+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:40.367 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:39 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[80225]: 2024-09-10T12:17:39.866+0000 7fd57de3f640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.155415+0000 front 2024-09-10T12:17:08.155327+0000 (oldest deadline 2024-09-10T12:17:34.055264+0000) 2024-09-10T12:17:40.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:40 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:40.443+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:40.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:40 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:40.443+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6814 osd.1 since back 2024-09-10T12:17:14.789492+0000 front 2024-09-10T12:17:14.789479+0000 (oldest deadline 2024-09-10T12:17:40.089344+0000) 2024-09-10T12:17:41.367 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:40 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[80225]: 2024-09-10T12:17:40.871+0000 7fd57de3f640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.155415+0000 front 2024-09-10T12:17:08.155327+0000 (oldest deadline 2024-09-10T12:17:34.055264+0000) 2024-09-10T12:17:41.367 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:40 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4[80225]: 2024-09-10T12:17:40.871+0000 7fd57de3f640 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.144:6814 osd.1 since back 2024-09-10T12:17:14.055808+0000 front 2024-09-10T12:17:14.055783+0000 (oldest deadline 2024-09-10T12:17:39.955682+0000) 2024-09-10T12:17:41.841 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:41 smithi178 podman[102851]: 2024-09-10 12:17:41.584777325 +0000 UTC m=+5.435676916 container died 400e18bd887448c18a10b414078e80ccb46d00c8b5d1acb2b27a932014aec215 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=) 2024-09-10T12:17:41.841 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:41 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:41.419+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:41.841 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:41 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:41.419+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6814 osd.1 since back 2024-09-10T12:17:14.789492+0000 front 2024-09-10T12:17:14.789479+0000 (oldest deadline 2024-09-10T12:17:40.089344+0000) 2024-09-10T12:17:42.117 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:41 smithi178 podman[102851]: 2024-09-10 12:17:41.939242535 +0000 UTC m=+5.790142120 container cleanup 400e18bd887448c18a10b414078e80ccb46d00c8b5d1acb2b27a932014aec215 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T12:17:42.117 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:41 smithi178 bash[102851]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4 2024-09-10T12:17:42.470 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:42 smithi178 podman[102865]: 2024-09-10 12:17:42.207037427 +0000 UTC m=+0.619614534 container remove 400e18bd887448c18a10b414078e80ccb46d00c8b5d1acb2b27a932014aec215 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T12:17:42.770 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:42 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:42.468+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:42.770 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:42 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:42.468+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6814 osd.1 since back 2024-09-10T12:17:14.789492+0000 front 2024-09-10T12:17:14.789479+0000 (oldest deadline 2024-09-10T12:17:40.089344+0000) 2024-09-10T12:17:42.770 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:42 smithi178 podman[102958]: 2024-09-10 12:17:42.450014309 +0000 UTC m=+0.015973018 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T12:17:42.770 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:42 smithi178 podman[102958]: 2024-09-10 12:17:42.566391707 +0000 UTC m=+0.132350404 container create 023bf02143ac56af2af4bc66be0e05840b91724c9b08237370ae3f3d5a30cff5 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T12:17:42.771 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:42 smithi178 podman[102958]: 2024-09-10 12:17:42.768913018 +0000 UTC m=+0.334871716 container init 023bf02143ac56af2af4bc66be0e05840b91724c9b08237370ae3f3d5a30cff5 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-10T12:17:43.117 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:42 smithi178 podman[102958]: 2024-09-10 12:17:42.772535009 +0000 UTC m=+0.338493711 container start 023bf02143ac56af2af4bc66be0e05840b91724c9b08237370ae3f3d5a30cff5 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux ) 2024-09-10T12:17:43.117 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:42 smithi178 podman[102958]: 2024-09-10 12:17:42.851970777 +0000 UTC m=+0.417929484 container attach 023bf02143ac56af2af4bc66be0e05840b91724c9b08237370ae3f3d5a30cff5 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T12:17:43.117 INFO:journalctl@ceph.osd.4.smithi178.stdout:Sep 10 12:17:42 smithi178 podman[102958]: 2024-09-10 12:17:42.956294955 +0000 UTC m=+0.522253663 container died 023bf02143ac56af2af4bc66be0e05840b91724c9b08237370ae3f3d5a30cff5 (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True) 2024-09-10T12:17:43.506 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T12:17:43.543 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T12:17:43.543 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T12:17:43.543 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T12:17:43.543 DEBUG:teuthology.orchestra.run.smithi178:> sudo systemctl stop ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.5 2024-09-10T12:17:43.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:43 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:43.489+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:43.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:43 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:43.489+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6814 osd.1 since back 2024-09-10T12:17:14.789492+0000 front 2024-09-10T12:17:14.789479+0000 (oldest deadline 2024-09-10T12:17:40.089344+0000) 2024-09-10T12:17:43.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:43 smithi178 systemd[1]: Stopping Ceph osd.5 for 7eacd86c-6f66-11ef-bcea-c7b262605968... 2024-09-10T12:17:44.238 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:43 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:43.866+0000 7f82f5bbd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T12:17:44.238 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:43 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:43.866+0000 7f82f5bbd640 -1 osd.5 85 *** Got signal Terminated *** 2024-09-10T12:17:44.238 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:43 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:43.866+0000 7f82f5bbd640 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T12:17:44.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:44 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:44.510+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:44.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:44 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:44.510+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6814 osd.1 since back 2024-09-10T12:17:14.789492+0000 front 2024-09-10T12:17:14.789479+0000 (oldest deadline 2024-09-10T12:17:40.089344+0000) 2024-09-10T12:17:45.617 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:45 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:45.481+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:45.617 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:45 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:45.481+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6814 osd.1 since back 2024-09-10T12:17:14.789492+0000 front 2024-09-10T12:17:14.789479+0000 (oldest deadline 2024-09-10T12:17:40.089344+0000) 2024-09-10T12:17:46.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:46 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:46.493+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:46.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:46 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:46.493+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6814 osd.1 since back 2024-09-10T12:17:14.789492+0000 front 2024-09-10T12:17:14.789479+0000 (oldest deadline 2024-09-10T12:17:40.089344+0000) 2024-09-10T12:17:47.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:47 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:47.523+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:47.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:47 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:47.523+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6814 osd.1 since back 2024-09-10T12:17:14.789492+0000 front 2024-09-10T12:17:14.789479+0000 (oldest deadline 2024-09-10T12:17:40.089344+0000) 2024-09-10T12:17:48.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:48 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:48.527+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6806 osd.0 since back 2024-09-10T12:17:08.988977+0000 front 2024-09-10T12:17:08.989036+0000 (oldest deadline 2024-09-10T12:17:30.688663+0000) 2024-09-10T12:17:48.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:48 smithi178 ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5[85301]: 2024-09-10T12:17:48.527+0000 7f82ee1ba640 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.144:6814 osd.1 since back 2024-09-10T12:17:14.789492+0000 front 2024-09-10T12:17:14.789479+0000 (oldest deadline 2024-09-10T12:17:40.089344+0000) 2024-09-10T12:17:49.551 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:49 smithi178 podman[103061]: 2024-09-10 12:17:49.317640737 +0000 UTC m=+5.558819970 container died 4a6125da2507a4954980f0c0d476ac03f0a6927b76a293ae79279fa9f2991c7a (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T12:17:49.826 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:49 smithi178 podman[103061]: 2024-09-10 12:17:49.638360764 +0000 UTC m=+5.879539996 container cleanup 4a6125da2507a4954980f0c0d476ac03f0a6927b76a293ae79279fa9f2991c7a (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-10T12:17:49.827 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:49 smithi178 bash[103061]: ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5 2024-09-10T12:17:50.117 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:49 smithi178 podman[103089]: 2024-09-10 12:17:49.825613255 +0000 UTC m=+0.503904605 container remove 4a6125da2507a4954980f0c0d476ac03f0a6927b76a293ae79279fa9f2991c7a (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T12:17:50.579 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:50 smithi178 podman[103178]: 2024-09-10 12:17:50.155241035 +0000 UTC m=+0.015602680 image pull 41d67112835771c3efca60c6b8fe297c7ddaecdc26cea26fd28aa18636e49005 quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559 2024-09-10T12:17:50.579 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:50 smithi178 podman[103178]: 2024-09-10 12:17:50.259713422 +0000 UTC m=+0.120075097 container create d21f86b1a1bc71f35df63620a63f014f1c7839a03b72b30efb01d9f77d4e01db (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.license=GPLv2) 2024-09-10T12:17:50.579 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:50 smithi178 podman[103178]: 2024-09-10 12:17:50.495489369 +0000 UTC m=+0.355850999 container init d21f86b1a1bc71f35df63620a63f014f1c7839a03b72b30efb01d9f77d4e01db (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T12:17:50.579 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:50 smithi178 podman[103178]: 2024-09-10 12:17:50.499038025 +0000 UTC m=+0.359399655 container start d21f86b1a1bc71f35df63620a63f014f1c7839a03b72b30efb01d9f77d4e01db (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b) 2024-09-10T12:17:50.580 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:50 smithi178 podman[103178]: 2024-09-10 12:17:50.57862804 +0000 UTC m=+0.438989678 container attach d21f86b1a1bc71f35df63620a63f014f1c7839a03b72b30efb01d9f77d4e01db (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-10T12:17:50.867 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:50 smithi178 podman[103178]: 2024-09-10 12:17:50.689240603 +0000 UTC m=+0.549602243 container died d21f86b1a1bc71f35df63620a63f014f1c7839a03b72b30efb01d9f77d4e01db (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T12:17:51.306 INFO:journalctl@ceph.osd.5.smithi178.stdout:Sep 10 12:17:51 smithi178 podman[103202]: 2024-09-10 12:17:51.28218582 +0000 UTC m=+0.586889071 container remove d21f86b1a1bc71f35df63620a63f014f1c7839a03b72b30efb01d9f77d4e01db (image=quay.ceph.io/ceph-ci/ceph@sha256:1c016bcec6a352c6df797523a67ab86655d59690b95308490e636572fc527559, name=ceph-7eacd86c-6f66-11ef-bcea-c7b262605968-osd-5-deactivate, RELEASE=wip-rishabh-testing-20240909.165402-squid-1cf819b, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T12:17:51.311 DEBUG:teuthology.orchestra.run.smithi178:> sudo pkill -f 'journalctl -f -n 0 -u ceph-7eacd86c-6f66-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T12:17:51.352 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T12:17:51.352 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T12:17:51.352 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T12:17:51.508 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T12:17:55.735 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi144.stderr:ceph-fuse[65863]: fuse finished with error 0 and tester_r 0 2024-09-10T12:18:20.582 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T12:18:20.741 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T12:18:36.207 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T12:18:36.241 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T12:18:36.269 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T12:18:36.271 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899074/remote/smithi144/crash 2024-09-10T12:18:36.271 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash -- . 2024-09-10T12:18:36.309 INFO:teuthology.orchestra.run.smithi144.stderr:tar: /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T12:18:36.309 INFO:teuthology.orchestra.run.smithi144.stderr:tar: Error is not recoverable: exiting now 2024-09-10T12:18:36.311 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899074/remote/smithi178/crash 2024-09-10T12:18:36.312 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash -- . 2024-09-10T12:18:36.335 INFO:teuthology.orchestra.run.smithi178.stderr:tar: /var/lib/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T12:18:36.335 INFO:teuthology.orchestra.run.smithi178.stderr:tar: Error is not recoverable: exiting now 2024-09-10T12:18:36.336 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T12:18:36.337 DEBUG:teuthology.orchestra.run.smithi144:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | head -n 1 2024-09-10T12:18:36.385 INFO:teuthology.orchestra.run.smithi144.stdout:2024-09-10T11:40:00.000229+0000 mon.smithi144 (mon.0) 681 : cluster [ERR] fs cephfs is degraded 2024-09-10T12:18:36.386 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-10T12:18:36.386 INFO:tasks.cephadm:Compressing logs... 2024-09-10T12:18:36.386 DEBUG:teuthology.orchestra.run.smithi144:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T12:18:36.428 DEBUG:teuthology.orchestra.run.smithi178:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T12:18:36.451 INFO:teuthology.orchestra.run.smithi144.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T12:18:36.451 INFO:teuthology.orchestra.run.smithi144.stderr:: No such file or directory 2024-09-10T12:18:36.451 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mon.smithi144.log 2024-09-10T12:18:36.452 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T12:18:36.452 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.0.log: gzip/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mon.smithi144.log: -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T12:18:36.452 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T12:18:36.455 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T12:18:36.456 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mgr.smithi144.bkogfs.log 2024-09-10T12:18:36.456 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi144.bfyvwv.log 2024-09-10T12:18:36.456 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mgr.smithi144.bkogfs.log: gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T12:18:36.456 INFO:teuthology.orchestra.run.smithi144.stderr: 85.9% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T12:18:36.456 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi144.bfyvwv.log: gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.log 2024-09-10T12:18:36.458 INFO:teuthology.orchestra.run.smithi178.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T12:18:36.458 INFO:teuthology.orchestra.run.smithi178.stderr:: No such file or directory 2024-09-10T12:18:36.458 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T12:18:36.458 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.audit.log 2024-09-10T12:18:36.458 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-volume.log: /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mon.smithi178.log 2024-09-10T12:18:36.458 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi178.nmhrmh.log 2024-09-10T12:18:36.458 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mon.smithi178.log: gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mgr.smithi178.vmvxuf.log 2024-09-10T12:18:36.458 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi178.nmhrmh.log: gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.cephadm.log 2024-09-10T12:18:36.458 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mgr.smithi178.vmvxuf.log: gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T12:18:36.458 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.log 2024-09-10T12:18:36.458 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.5.log: 85.8% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.cephadm.log.gz 2024-09-10T12:18:36.459 INFO:teuthology.orchestra.run.smithi178.stderr: 91.4% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T12:18:36.459 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi178.czjrsa.log 2024-09-10T12:18:36.460 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-client.ceph-exporter.smithi178.log 2024-09-10T12:18:36.461 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.2.log: 91.3% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.audit.log.gz 2024-09-10T12:18:36.461 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi144.jaoyvs.log 2024-09-10T12:18:36.465 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi178.czjrsa.log: 89.7% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T12:18:36.466 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T12:18:36.467 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.log: 89.6% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph.log.gz 2024-09-10T12:18:36.467 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-client.ceph-exporter.smithi144.log 2024-09-10T12:18:36.468 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-client.ceph-exporter.smithi178.log: 91.4% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T12:18:36.468 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.log 2024-09-10T12:18:36.469 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi144.jaoyvs.log: 91.6% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T12:18:36.470 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2024-09-10T12:18:36.472 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.3.log: 94.5% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-client.ceph-exporter.smithi178.log.gz 2024-09-10T12:18:36.472 INFO:teuthology.orchestra.run.smithi178.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T12:18:36.474 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-client.ceph-exporter.smithi144.log: 94.5% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-client.ceph-exporter.smithi144.log.gz 2024-09-10T12:18:36.475 INFO:teuthology.orchestra.run.smithi144.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T12:18:36.487 INFO:teuthology.orchestra.run.smithi178.stderr:/var/log/ceph/ceph-client.1.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T12:18:36.514 INFO:teuthology.orchestra.run.smithi178.stderr: 90.1% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mgr.smithi178.vmvxuf.log.gz 2024-09-10T12:18:36.544 INFO:teuthology.orchestra.run.smithi144.stderr:/var/log/ceph/ceph-client.0.log: /var/log/ceph/cephadm.log: 91.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T12:18:37.105 INFO:teuthology.orchestra.run.smithi144.stderr: 89.7% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mgr.smithi144.bkogfs.log.gz 2024-09-10T12:18:37.308 INFO:teuthology.orchestra.run.smithi178.stderr: 92.5% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mon.smithi178.log.gz 2024-09-10T12:18:38.266 INFO:teuthology.orchestra.run.smithi144.stderr: 89.9% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mon.smithi144.log.gz 2024-09-10T12:18:41.686 INFO:teuthology.orchestra.run.smithi144.stderr: 93.7% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T12:18:42.217 INFO:teuthology.orchestra.run.smithi144.stderr: 93.6% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T12:18:42.568 INFO:teuthology.orchestra.run.smithi178.stderr: 93.7% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T12:18:42.652 INFO:teuthology.orchestra.run.smithi178.stderr: 93.7% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T12:18:43.367 INFO:teuthology.orchestra.run.smithi144.stderr: 93.7% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T12:18:43.612 INFO:teuthology.orchestra.run.smithi178.stderr: 93.8% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T12:18:45.278 INFO:teuthology.orchestra.run.smithi144.stderr: 95.2% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi144.bfyvwv.log.gz 2024-09-10T12:18:45.287 INFO:teuthology.orchestra.run.smithi178.stderr: 95.2% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi178.czjrsa.log.gz 2024-09-10T12:18:51.303 INFO:teuthology.orchestra.run.smithi144.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2024-09-10T12:18:52.386 INFO:teuthology.orchestra.run.smithi178.stderr:gzip: /var/log/ceph/ceph-client.1.log: file size changed while zipping 2024-09-10T12:18:52.386 INFO:teuthology.orchestra.run.smithi178.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.1.log.gz 2024-09-10T12:19:20.932 INFO:teuthology.orchestra.run.smithi178.stderr: 93.5% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi178.nmhrmh.log.gz 2024-09-10T12:19:20.933 INFO:teuthology.orchestra.run.smithi178.stderr: 2024-09-10T12:19:20.933 INFO:teuthology.orchestra.run.smithi178.stderr:real 0m44.490s 2024-09-10T12:19:20.934 INFO:teuthology.orchestra.run.smithi178.stderr:user 1m26.777s 2024-09-10T12:19:20.934 INFO:teuthology.orchestra.run.smithi178.stderr:sys 0m2.689s 2024-09-10T12:19:23.572 INFO:teuthology.orchestra.run.smithi144.stderr: 93.3% -- replaced with /var/log/ceph/7eacd86c-6f66-11ef-bcea-c7b262605968/ceph-mds.cephfs.smithi144.jaoyvs.log.gz 2024-09-10T12:19:23.574 INFO:teuthology.orchestra.run.smithi144.stderr: 2024-09-10T12:19:23.574 INFO:teuthology.orchestra.run.smithi144.stderr:real 0m47.133s 2024-09-10T12:19:23.574 INFO:teuthology.orchestra.run.smithi144.stderr:user 1m28.064s 2024-09-10T12:19:23.575 INFO:teuthology.orchestra.run.smithi144.stderr:sys 0m2.953s 2024-09-10T12:19:23.575 INFO:tasks.cephadm:Archiving logs... 2024-09-10T12:19:23.575 DEBUG:teuthology.misc:Transferring archived files from smithi144:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899074/remote/smithi144/log 2024-09-10T12:19:23.576 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T12:19:35.528 DEBUG:teuthology.misc:Transferring archived files from smithi178:/var/log/ceph to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899074/remote/smithi178/log 2024-09-10T12:19:35.529 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T12:19:46.878 INFO:tasks.cephadm:Removing cluster... 2024-09-10T12:19:46.878 DEBUG:teuthology.orchestra.run.smithi144:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 --force 2024-09-10T12:19:47.032 INFO:teuthology.orchestra.run.smithi144.stdout:Deleting cluster with fsid: 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T12:19:47.386 DEBUG:teuthology.orchestra.run.smithi178:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 7eacd86c-6f66-11ef-bcea-c7b262605968 --force 2024-09-10T12:19:47.536 INFO:teuthology.orchestra.run.smithi178.stdout:Deleting cluster with fsid: 7eacd86c-6f66-11ef-bcea-c7b262605968 2024-09-10T12:19:47.935 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T12:19:47.935 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T12:19:47.952 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T12:19:47.968 INFO:tasks.cephadm:Teardown complete 2024-09-10T12:19:47.968 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T12:19:47.979 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/install/__init__.py", line 641, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/ceph_fuse.py", line 248, in task File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-10T12:19:47.980 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T12:19:47.980 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T12:19:47.995 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T12:19:48.076 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T12:19:48.087 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T12:19:48.087 DEBUG:teuthology.orchestra.run.smithi144:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T12:19:48.090 DEBUG:teuthology.orchestra.run.smithi178:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T12:19:48.102 INFO:teuthology.orchestra.run.smithi144.stderr:bash: line 1: ntpq: command not found 2024-09-10T12:19:48.104 INFO:teuthology.orchestra.run.smithi178.stderr:bash: line 1: ntpq: command not found 2024-09-10T12:19:48.107 INFO:teuthology.orchestra.run.smithi144.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T12:19:48.107 INFO:teuthology.orchestra.run.smithi144.stdout:=============================================================================== 2024-09-10T12:19:48.107 INFO:teuthology.orchestra.run.smithi144.stdout:^* hv01.front.sepia.ceph.com 3 7 377 74 +94us[ +108us] +/- 41ms 2024-09-10T12:19:48.107 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv02.front.sepia.ceph.com 3 8 377 140 +78us[ +86us] +/- 79ms 2024-09-10T12:19:48.107 INFO:teuthology.orchestra.run.smithi144.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 75 -282us[ -268us] +/- 82ms 2024-09-10T12:19:48.107 INFO:teuthology.orchestra.run.smithi144.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T12:19:48.108 INFO:teuthology.orchestra.run.smithi178.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T12:19:48.108 INFO:teuthology.orchestra.run.smithi178.stdout:=============================================================================== 2024-09-10T12:19:48.108 INFO:teuthology.orchestra.run.smithi178.stdout:^* hv01.front.sepia.ceph.com 3 7 377 69 +89us[ +95us] +/- 41ms 2024-09-10T12:19:48.108 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 5 +121us[ +121us] +/- 53ms 2024-09-10T12:19:48.108 INFO:teuthology.orchestra.run.smithi178.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 3 -239us[ -239us] +/- 84ms 2024-09-10T12:19:48.108 INFO:teuthology.orchestra.run.smithi178.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T12:19:48.109 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T12:19:48.119 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T12:19:48.119 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T12:19:48.130 DEBUG:teuthology.orchestra.run.smithi144:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T12:19:48.232 DEBUG:teuthology.orchestra.run.smithi178:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T12:19:48.306 DEBUG:teuthology.orchestra.run.smithi144:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T12:19:48.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T12:19:48.336 DEBUG:teuthology.orchestra.run.smithi178:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-10T12:19:48.364 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T12:19:48.364 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T12:19:48.375 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T12:19:48.389 INFO:teuthology.task.internal:Duration was 3987.907208 seconds 2024-09-10T12:19:48.389 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T12:19:48.399 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T12:19:48.399 DEBUG:teuthology.orchestra.run.smithi144:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T12:19:48.402 DEBUG:teuthology.orchestra.run.smithi178:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T12:19:48.439 INFO:teuthology.orchestra.run.smithi144.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T12:19:48.443 INFO:teuthology.orchestra.run.smithi178.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T12:19:48.725 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T12:19:48.725 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi144.front.sepia.ceph.com 2024-09-10T12:19:48.725 DEBUG:teuthology.orchestra.run.smithi144:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T12:19:48.785 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi178.front.sepia.ceph.com 2024-09-10T12:19:48.785 DEBUG:teuthology.orchestra.run.smithi178:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T12:19:48.813 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T12:19:48.813 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T12:19:48.827 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T12:19:48.879 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T12:19:48.879 DEBUG:teuthology.orchestra.run.smithi144:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T12:19:48.894 DEBUG:teuthology.orchestra.run.smithi178:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T12:19:50.149 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T12:19:50.161 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T12:19:50.161 DEBUG:teuthology.orchestra.run.smithi144:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T12:19:50.186 DEBUG:teuthology.orchestra.run.smithi178:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T12:19:50.216 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T12:19:50.227 DEBUG:teuthology.orchestra.run.smithi144:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T12:19:50.229 DEBUG:teuthology.orchestra.run.smithi178:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T12:19:50.250 INFO:teuthology.orchestra.run.smithi144.stdout:kernel.core_pattern = core 2024-09-10T12:19:50.284 INFO:teuthology.orchestra.run.smithi178.stdout:kernel.core_pattern = core 2024-09-10T12:19:50.578 DEBUG:teuthology.orchestra.run.smithi144:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T12:19:50.634 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T12:19:50.635 DEBUG:teuthology.orchestra.run.smithi178:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T12:19:50.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T12:19:50.653 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T12:19:50.666 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T12:19:50.667 DEBUG:teuthology.misc:Transferring archived files from smithi144:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899074/remote/smithi144 2024-09-10T12:19:50.667 DEBUG:teuthology.orchestra.run.smithi144:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T12:19:50.734 DEBUG:teuthology.misc:Transferring archived files from smithi178:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2024-09-10_06:58:10-fs-wip-rishabh-testing-20240909.165402-squid-distro-default-smithi/7899074/remote/smithi178 2024-09-10T12:19:50.734 DEBUG:teuthology.orchestra.run.smithi178:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T12:19:50.778 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T12:19:50.778 DEBUG:teuthology.orchestra.run.smithi144:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T12:19:50.780 DEBUG:teuthology.orchestra.run.smithi178:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T12:19:50.829 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T12:19:50.842 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T12:19:50.842 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T12:19:50.853 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T12:19:50.853 DEBUG:teuthology.orchestra.run.smithi144:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T12:19:50.856 DEBUG:teuthology.orchestra.run.smithi178:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T12:19:50.868 INFO:teuthology.orchestra.run.smithi144.stdout: 262169 4 drwxr-xr-x 3 ubuntu ubuntu 4096 Sep 10 12:19 /home/ubuntu/cephtest 2024-09-10T12:19:50.869 INFO:teuthology.orchestra.run.smithi144.stdout: 528175 4 d--------- 2 ubuntu ubuntu 4096 Sep 10 11:27 /home/ubuntu/cephtest/mnt.0 2024-09-10T12:19:50.869 INFO:teuthology.orchestra.run.smithi144.stderr:find: ‘/home/ubuntu/cephtest/mnt.0’: Permission denied 2024-09-10T12:19:50.870 INFO:teuthology.orchestra.run.smithi144.stderr:rmdir: failed to remove '/home/ubuntu/cephtest': Directory not empty 2024-09-10T12:19:50.872 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T12:19:50.872 ERROR:teuthology.run_tasks:Manager failed: internal.base Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/internal/__init__.py", line 48, in base yield File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 142, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/ceph_fuse.py", line 248, in task File "/home/teuthworker/src/git.ceph.com_ceph-c_1cf819b7212a6048e92fc78c4eff963344f6d56b/qa/tasks/cephfs/fuse_mount.py", line 403, in umount_wait File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 473, in wait check_time() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/run_tasks.py", line 160, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__ self.gen.throw(typ, value, traceback) File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/task/internal/__init__.py", line 53, in base run.wait( File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 479, in wait proc.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed on smithi144 with status 1: 'find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest' 2024-09-10T12:19:50.873 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T12:19:50.884 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T12:19:50.896 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: MaxWhileTries: reached maximum tries (50) after waiting for 300 seconds 2024-09-10T12:19:50.899 INFO:teuthology.run:Summary data: description: fs/upgrade/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn pg_health syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client/fuse 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 3987.907207965851 failure_reason: reached maximum tries (50) after waiting for 300 seconds flavor: default owner: scheduled_rishabh@teuthology status: fail success: false 2024-09-10T12:19:50.899 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T12:19:50.978 INFO:teuthology.run:FAIL